v1.2.0

[stephenroller]

This Saturday marks the 4 year anniversary since the initial release of ParlAI. I'd like to offer my sincere gratitude to our users, our contributors, and all of the core development team. ParlAI wouldn't be what it is without all of you. -@stephenroller

Major new features

Background Preprocessing

Improve your training speeds by 1.25x-5.0x by switching from --num-workers 0 to --num-workers N. See our Speeding up training docs for details. (#3527, #3586, #3575, #3533, #3389)

(Beta) Support for torch.jit
Deploy faster models by exporting models with TorchScript. Currently limited to BART models only. (#3459)

Support for T5
We now have agents for Google's T5 models (#3519)

Opt Presets
Opt presets. Easily use prepackaged opt files as shorthand for long command line arguments (#3564)

Log/validate/stop based on number of steps
Get up to a 10% speedup of distributed training by switching from -vtim or -veps to -vstep (#3379, #3555)

Backwards-incompatible changes

• DictionaryAgent.text2vec now requires input to be a string (RuntimeError: Need to set model argument to use create_agent #3472)
• A number of older projects have been archived: DrQA (Retire DrQA agent #3559), Controllable Dialogue (Retire Self-feeding and Controllable Dialogue #3557), and Self-Feeding Chatbot (Retire Self-feeding and Controllable Dialogue #3557).

Minor improvements

• Performance speedup in generation using Transformer Generators (Optimize MultiHeadAttention forward method #3550)
• Improvements to the Transformer API, making Transformer models more easily extensible. More to come soon. (Pass Opt to TransformerEncoder/Decoder #3486, Type annotate transformer/modules.py #3545, Breaking transformer modules.py into smaller files #3466, [Transformer Opt] Add back default values to underspecified call sites #3501)
• Various performance improvements when loading ParlAI or performing some activities (Marginally faster display data #3544, Improve parlai cold start #3482)
• Metrics:
  • New truncation metrics show you how much context/label you're losing (Add truncate stats to training logs #3458, Add length of truncation to logs #3508)
  • Additional metrics in the Wizard of Wikipedia teacher (Rare Word F1 #3566, [RareF1] Check for null inputs #3592, WoW Custom Metrics #3503)
  • New token_em metric, an equivalent to accuracy with --skip-generation true (Add new token_em metric #3497)
• Self-chat can now use seed messages (Self chat from messages in file #3580)
• New "normalized" ConvAI2 teachers for the non-standard variants (Normalized all convai2 teachers #3509)
• Update FusedAdam support to use FairScale ([fairscale] FusedAdam support #3522)
• Add --wandb-entity flag to the logger (Add --wandb-entity flag to the logger #3562)
• Tensorboard now provides nicer names of metrics (Full Tensorboard metric titles #3534)

Bugfixes

• [core] Fix a bug when resuming with the cosine LR scheduler after preemption ([TA] Fix preemption of cosine scheduler #3599)
• [core] Improve robustness to serialization of Observations ([World Logs] Don't crash if a value can't be serialized #3591)
• [core] ParlaiDialogTeacher now parses the rewards field as a number (Support a rewards field in parlai format #3517)
• [core] Fix recently introduced ChunkTeacher bugs ([ChunkTeacher] Fix a hang #3549, [ChunkTeacher] Fix infinite padding output #3542, [ChunkTeacher] Test that would have caught broken padding #3543)
• [core] Minor FP16 fixes when converting old checkpoints (Minor FP16 fixes when converting old checkpoints #3514)
• [core] Fix annoying ambiguity issues with commandline parsing in Python 3.8 ([params] Fix parser in python 3.8 #3598)
• [core] Fix a rare situation in case a dictionary contained tokens with leading whitespace (Modify dictionary load to only rstrip to handle space prefixed tokens #3613)
• [mutators] Fix a bug with the flatten mutator providing the wrong history ([Mutators] full conversation history in flatten mutator #3578, redundant loop #3584)
• [metrics] Fix a bug with computation of fairseq metrics ([Metrics] Fairseq BLEU Re-Implementation #3518)
• [task] Fix a bug with Wizard of Wikipedia teacher causing some data to be omitted (Update wizard_of_wikipedia agents.py #3585)
• [task] Fix a crash in Wizard of Wikipedia end2end agent when given zero documents ( fixed universal_sentence_embedding function #3602)
• [task] Update a dead dataset to a new link (mirrored dialog babi plus dataset #3520)
• [task] Fix an issue with CCPE (Make CCPE work #3487)
• [tga] Fix a case where TGA used as a ranking model could crash ([TGA] rank candidates fix #3541)
• [agent] Fix a crash in BertDictionaryAgent ([bug-fix] fixed is_ prebuilt function name typo in BertDictionaryAgent #3560)
• [other] Various rare issues (Fix issue #3502: "Message already contains key dyn_batch_idx" #3505) and quality improvements (Messages instead of dicts for get() in ED, DNLI #3496)

Crowdsourcing improvements

• Add new option to avoid randomizing pairs in ACUTE-Eval (Add a randomize_conversations to ACUTE blueprint + use #3528)
• ACUTE-Eval provides additional warnings when options are not set to recommended values (Warning on unrecommended ACUTE-Eval value #3536, [ACUTE] Update default maximum_units_per_worker #3524)

New Datasets

• MetaLWoZ ([teacher] MetaLWoZ #3583)
• CMU_DoG (CMU_DoG #3593, [CMU_DoG] Download data from source #3615)

Doc improvements

• New tutorial: How to write a PR contributing (moving to a fork) tutorial (How to write a PR contributing (moving to a fork) tutorial #3490)
• New reference manual of standard ParlAI metrics ([docs] Add docs explaining metrics. #3498)
• Fix a bug keeping some tasks from being listed in our docs ([docs] Fix task list #3495)
• Various minor doc changes ([docs] remove redundant --skip-generation True #3513, Add distillation arg #3561, [RAD] Project Page #3596)

Developer improvements

• Update to pytorch 1.8. No changes necessary. ([tests] Update to pytorch 1.8 #3607)
• Various reliability improvements and speed ups to our CI and tests ([tests] Improve reliability of test_generator_distributed. #3603, [tests] Kill test_eval_fixed #3597, [tests] Speed up TestWizardModel #3604, [tests] Speed up test_chunkteacher #3606, [tests] Speed up test_unlikelihood #3605, Speed up test_crossencoder #3587, Dependency bumps to fix cleaninstall CI #3588, [gputests] Don't use temporary datapath in test_torchscript #3590, Fix Wizard unit test GPU OOMs #3539, Skip a test that's causing headaches. #3535, Update test_tutorial_generator.py #3526, Bump CI checks to Torch 1.7 #3521)
• Various dependency bumps (Bump pyyaml from 5.3.1 to 5.4 #3553, Bump pillow from 7.2.0 to 8.1.1 #3537, Attrs version bump #3515)
• Switched logging.warn to logging.warning (Switched logging.warn to logging.warning #3569)
• TorchAgent.history prettier debug printing (Implement __str__ for History #3510)
• Small other improvements (Add cmdline args method to DialogPartnerWorld #3506)

---

This discussion was created from the release v1.2.0.