[Bug Fix] trainer.update(1) should be used after loss.mean() is called #1000

liuzh47 · 2019-11-12T10:32:14Z

Description

(Brief description on what this PR is about)
[BUGFIX] A bug fix of sentiment analysis training script. trainer.update(1) should be used after loss.mean() is called.

Checklist

Essentials

[ X] PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
[ X] Changes are complete (i.e. I finished coding on this PR)
[ X] All changes have test coverage
[ X] Code is well-documented

Changes

Feature1, tests, (and when applicable, API doc)
Feature2, tests, (and when applicable, API doc)

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

cc @dmlc/gluon-nlp-team

codecov · 2019-11-12T10:32:18Z

Codecov Report

Merging #1000 into v0.x will decrease coverage by 2.55%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             v0.x    #1000      +/-   ##
==========================================
- Coverage   87.26%   84.70%   -2.56%     
==========================================
  Files          81       43      -38     
  Lines        7371     6701     -670     
==========================================
- Hits         6432     5676     -756     
- Misses        939     1025      +86

Impacted Files	Coverage Δ
conftest.py	`85.29% <100.00%> (ø)`
src/gluonnlp/data/sampler.py	`96.55% <0.00%> (-0.27%)`	⬇️
src/gluonnlp/utils/__init__.py	`100.00% <0.00%> (ø)`
src/gluonnlp/data/bert/squad.py
src/gluonnlp/model/utils.py
src/gluonnlp/data/conll.py
src/gluonnlp/data/word_embedding_evaluation.py
src/gluonnlp/calibration/collector.py
src/gluonnlp/data/registry.py
src/gluonnlp/embedding/evaluation.py
... and 109 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c99061d...189bbdc. Read the comment docs.

mli · 2019-11-13T03:03:05Z

Job PR-1000/2 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-1000/2/index.html

leezu · 2019-11-13T03:10:01Z

@astonzhang FYI

The results at https://github.com/dmlc/gluon-nlp/blob/master/scripts/sentiment_analysis/index.rst#textcnn are generated without this change. Could you confirm (on a sample) that the results remain unchanged?

eric-haibin-lin

@xiaotinghe

mli · 2019-11-14T04:25:07Z

Job PR-1000/4 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-1000/4/index.html

xiaotinghe · 2019-11-17T07:44:09Z

I will reconfirm the results on all sample.

szha · 2019-12-10T01:08:27Z

@xiaotinghe any update?

xiaotinghe · 2019-12-11T11:41:42Z

@szha @eric-haibin-lin I have reconfirmed the results for all the data. I will update the results later.

leezu · 2020-02-10T18:13:58Z

Ping @xiaotinghe

* numpy version * Enable Github Actions * Update unittests.yml * Update unittests.yml * Update setup.py * fix test * Update README.md * Update test_models_bert.py * Update tmpdir * Enable codecov * fix a commit id * Separate codecov per platform * Revert "Update tmpdir" This reverts commit 6625af9. pytest-dev/pytest#1120 * Remove files * add symlinks * update Merge conversion toolkits update unittests by fixing the version update datasets add scripts Delete __init__.py add src update Update setup.py Update setup.py update all tests revise test cases Update unittests.yml Update initializer.py Create preprocessing.py Update __init__.py Update attention_cell.py Update prepare_wmt.py move ubuntu + windows to TODO * Update unittests.yml * fix alpha in sentencepiece * fix bug * update * fix README * Update unittests.yml * Update README.md * update Co-authored-by: Leonard Lausen <lausen@amazon.com>

* fix bert cfg * fix lowercase * re-test

* try to fix the CI of the export test * re-enable 3.8 * use skipif to skip the test of python3.8

…S3 + Add Ubuntu test (dmlc#1249) * add match_tokens_with_char_spans to utility + add ability to download from S3 * Update lazy_imports.py * Update lazy_imports.py * Revise broken link * test downloading * enable ubuntu test * update * Update unittests.yml * Update .coveragerc * Create codecov.yml * Update test_models.py * fix bug * Update test_models.py * Update codecov.yml * Delete codecov.yml * do not paralleize the backbone forward test * update test cases * use a smaller batch_size + seq_length for testing

* fix bert cfg * fix lowercase * re-test * restart * fix * update gluon_electra_small_owt * remove plau_answer * fix * get_backbone * eta * fix * add match_tokens_with_char_spans to utility + add ability to download from S3 * Update lazy_imports.py * Update lazy_imports.py * update * fix squad * hotpotqa * update hotpotqa * update electra results * triviaqa * searchqa * remove newsqa * revise * fix * move * fix * upload fasttext to s3 * Update filtering.py * Update filtering.py Co-authored-by: Xingjian Shi <xshiab@connect.ust.hk>

* AWS batch job tool for GluonNLP * limit range Co-authored-by: Xingjian Shi <xshiab@connect.ust.hk>

…c#1254) codecov/codecov-action#80 (comment)

* back translation bash * split "lang-pair" para in clean_tok_para_corpus * added clean_tok_mono_corpus * fix * add num_process para * fix * fix * add yml * rm yml * update cfg name * update evaluate * added max_update / save_interval_update params * fix * fix * multi gpu inference * fix * update * update multi gpu inference * fix * fix * split evaluate and parallel infer * fix * test * fix * update * add comments * fix * remove todo comment * revert remove todo comment * raw lines remove duplicated '\n' * update multinomaial sampler * fix * fix * fix * fix * sampling * update script * fix * add test_case with k > 1 in topk sampling * fix multinomial sampler * update docs * comments situation eos_id = None * fix Co-authored-by: Hu <huta@a483e74650ff.ant.amazon.com>

* Some fixes to make the CI more stable * add retries * Update tokenizers.py

- Remove params and prefix arguments for MXNet 2 and update parameter sharing implementation - Remove Block.name_scope() for MXNet 2 - Remove self.params.get() and self.params.get_constant()

* Add fp16 support for Bert QA inference * change cfg dtype setting from run_squad script * pass dtype as argument to get_backbone

* update batch to gluonnlp-dev * add more types

…ECTRA, MobileBERT, RoBERTA, XLMR (dmlc#1258) * Add layout support * fix test * Update transformer.py * Update transformer.py * Update README.md * try to add set_layout * update test case * fix * update * update * update * Update bert.py * fix bug * update * Update test_models_bert.py * Update tokenizers.py * add compute layout * Update xlmr.py * Update test_models_bert.py * revise test cases * Update layers.py * move jieba to try import * fix * Update transformer.py * fix * Update bert.py * Update setup.py * Update test_models_bert.py * Update test_models_bert.py * fix * update * Revise * Update electra.py * Update electra.py * Update test_models_electra.py * fix * fix bug * Update test_models_albert.py * add more testcases * fix * Update albert.py * Update albert.py * fix bug * fix testcase * Update test_models_electra.py * Update bert.py * update * Update test_models_electra.py * Update mobilebert.py * Update mobilebert.py * update mobilebert * Update test_models_mobilebert.py * Update mobilebert.py * fix bug * Update roberta.py * fix roberta * update * update * fix import * fix bug * update * reduce test workloads * address comment * address comment

* Update run_squad.py * Update run_squad.py * Update prepare_glue.py

* init * fix convert roberta * rename TransformerNMTModel as TransformerModel * update bart * fix * fix * update init * add layernorm_embedding for transformer * convert script * encoder * fix * fix vocab * fix roberta * fix * fix electra * add conversion bash for roberta and xlmr * ELECTRA SETUP * convert bart decoder * fix * update * testing output * remove arange_like for embeddings * fix * update * use_pooler for bart * fix * upload params for bart * add test_models_bart * fix cfg * test bart * update * fix transformer * Squashed commit of the following: commit 510d991 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 30 02:33:22 2020 +0800 test commit 1b5fa7b Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 30 01:48:01 2020 +0800 fix comment1 commit 6533601 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 30 01:27:44 2020 +0800 fix comment commit a8853f9 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 30 01:10:06 2020 +0800 Squashed commit of the following: commit 232e0b6 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 30 01:05:17 2020 +0800 update commit 995e5d7 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 30 01:01:56 2020 +0800 fix commit 9623240 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 30 00:52:17 2020 +0800 fix commit d9c4140 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Wed Jul 29 23:07:10 2020 +0800 fix transformer commit e49fbe1 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Wed Jul 29 22:18:12 2020 +0800 update commit 1f75b26 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Wed Jul 29 22:04:08 2020 +0800 test bart commit 5bab516 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Wed Jul 29 21:34:47 2020 +0800 fix cfg commit 6c62a29 Merge: 3366cf3 033214e Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Wed Jul 29 21:33:10 2020 +0800 Merge remote-tracking branch 'upstream/numpy' into bart commit 033214e Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Wed Jul 29 00:36:57 2020 -0700 [Numpy] Fix SQuAD + Fix GLUE downloading (dmlc#1280) * Update run_squad.py * Update run_squad.py * Update prepare_glue.py commit 3c87457 Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Tue Jul 28 18:03:21 2020 -0700 Add layout + compute_layout support: TransformerNMT, BERT, ALBERT, ELECTRA, MobileBERT, RoBERTA, XLMR (dmlc#1258) * Add layout support * fix test * Update transformer.py * Update transformer.py * Update README.md * try to add set_layout * update test case * fix * update * update * update * Update bert.py * fix bug * update * Update test_models_bert.py * Update tokenizers.py * add compute layout * Update xlmr.py * Update test_models_bert.py * revise test cases * Update layers.py * move jieba to try import * fix * Update transformer.py * fix * Update bert.py * Update setup.py * Update test_models_bert.py * Update test_models_bert.py * fix * update * Revise * Update electra.py * Update electra.py * Update test_models_electra.py * fix * fix bug * Update test_models_albert.py * add more testcases * fix * Update albert.py * Update albert.py * fix bug * fix testcase * Update test_models_electra.py * Update bert.py * update * Update test_models_electra.py * Update mobilebert.py * Update mobilebert.py * update mobilebert * Update test_models_mobilebert.py * Update mobilebert.py * fix bug * Update roberta.py * fix roberta * update * update * fix import * fix bug * update * reduce test workloads * address comment * address comment commit 4d43f82 Author: Sheng Zha <szha@users.noreply.github.com> Date: Mon Jul 27 20:21:00 2020 -0700 add subversion/wget to docker, add readme (dmlc#1279) commit d76897b Author: phile <phile_999@126.com> Date: Tue Jul 28 10:10:13 2020 +0800 Add embedding related methods in numpy version (dmlc#1263) * A draft for embedding * fix embed_loader * add hyperbolic space and some updates * revise evaluation * fix * simple fixes * move l2norm to op.py * new features * fix * update * add tests, update * newline * Squashed commit of the following: commit 9e1ffde Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 30 11:42:01 2020 +0800 todo commit 9a7c343 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 30 10:53:15 2020 +0800 revert gelu commit 0425346 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 30 10:49:52 2020 +0800 re-upload bart commit 516ae84 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 30 03:32:35 2020 +0800 use_qkv_bias for transformer commit 9d60cda Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 30 03:17:28 2020 +0800 classifier_activation commit 510d991 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 30 02:33:22 2020 +0800 test commit 1b5fa7b Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 30 01:48:01 2020 +0800 fix comment1 commit 6533601 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 30 01:27:44 2020 +0800 fix comment commit a8853f9 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 30 01:10:06 2020 +0800 Squashed commit of the following: commit 232e0b6 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 30 01:05:17 2020 +0800 update commit 995e5d7 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 30 01:01:56 2020 +0800 fix commit 9623240 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 30 00:52:17 2020 +0800 fix commit d9c4140 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Wed Jul 29 23:07:10 2020 +0800 fix transformer commit e49fbe1 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Wed Jul 29 22:18:12 2020 +0800 update commit 1f75b26 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Wed Jul 29 22:04:08 2020 +0800 test bart commit 5bab516 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Wed Jul 29 21:34:47 2020 +0800 fix cfg commit 6c62a29 Merge: 3366cf3 033214e Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Wed Jul 29 21:33:10 2020 +0800 Merge remote-tracking branch 'upstream/numpy' into bart commit 033214e Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Wed Jul 29 00:36:57 2020 -0700 [Numpy] Fix SQuAD + Fix GLUE downloading (dmlc#1280) * Update run_squad.py * Update run_squad.py * Update prepare_glue.py commit 3c87457 Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Tue Jul 28 18:03:21 2020 -0700 Add layout + compute_layout support: TransformerNMT, BERT, ALBERT, ELECTRA, MobileBERT, RoBERTA, XLMR (dmlc#1258) * Add layout support * fix test * Update transformer.py * Update transformer.py * Update README.md * try to add set_layout * update test case * fix * update * update * update * Update bert.py * fix bug * update * Update test_models_bert.py * Update tokenizers.py * add compute layout * Update xlmr.py * Update test_models_bert.py * revise test cases * Update layers.py * move jieba to try import * fix * Update transformer.py * fix * Update bert.py * Update setup.py * Update test_models_bert.py * Update test_models_bert.py * fix * update * Revise * Update electra.py * Update electra.py * Update test_models_electra.py * fix * fix bug * Update test_models_albert.py * add more testcases * fix * Update albert.py * Update albert.py * fix bug * fix testcase * Update test_models_electra.py * Update bert.py * update * Update test_models_electra.py * Update mobilebert.py * Update mobilebert.py * update mobilebert * Update test_models_mobilebert.py * Update mobilebert.py * fix bug * Update roberta.py * fix roberta * update * update * fix import * fix bug * update * reduce test workloads * address comment * address comment commit 4d43f82 Author: Sheng Zha <szha@users.noreply.github.com> Date: Mon Jul 27 20:21:00 2020 -0700 add subversion/wget to docker, add readme (dmlc#1279) commit d76897b Author: phile <phile_999@126.com> Date: Tue Jul 28 10:10:13 2020 +0800 Add embedding related methods in numpy version (dmlc#1263) * A draft for embedding * fix embed_loader * add hyperbolic space and some updates * revise evaluation * fix * simple fixes * move l2norm to op.py * new features * fix * update * add tests, update * newline * fix comment * use xavier for embedding initializer

* fix roberta * fix xlmr * fix token_ids * fix * use_segmentation * fix roberta * update * fix * fix mobilebert * repeat * repeat for pretraining * revise * revise train_transformer * upload gluon_electra_small_owt * fix openwebtext * fix wiki * fix bookcorpus * multiprocessing for wiki * update * rename * index_update * topk * revise * layer-wise decay * fix mobilebert * try * update hyper-parameters of adamw * fix roberta * clip_grad_global_norm with zeros max_grad_norm * fix ModelForQABasic * multiply_grads * remove multiply_grads * fix * horovod for squad * update * inference without horovod * fix * update * re-upload roberta * fix get_pretrained * re-upload xlmr * update testings * tiny update on run_squad * test * lowercase * CharTokenizer * Squashed commit of the following: commit 35a586676036f627bffd0d3c753c6cd0a70d63cf Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Fri Jul 17 10:10:14 2020 +0800 Squashed commit of the following: commit 673344d Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Wed Jul 15 22:43:07 2020 +0800 CharTokenizer commit 8dabfd6 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Wed Jul 15 15:47:24 2020 +0800 lowercase commit f5c94a6 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Jul 14 17:45:28 2020 +0800 test commit dc55fc9 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Jul 14 05:45:01 2020 +0800 tiny update on run_squad commit 4defc7a Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Mon Jul 13 23:18:08 2020 +0800 update testings commit 2719e81 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Mon Jul 13 23:08:32 2020 +0800 re-upload xlmr commit cd0509d Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Mon Jul 13 22:30:47 2020 +0800 fix get_pretrained commit 8ed8a72 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Mon Jul 13 22:28:13 2020 +0800 re-upload roberta commit 5811d40 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Mon Jul 13 18:27:23 2020 +0800 update commit 44a09a3 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Sat Jul 11 15:06:33 2020 +0800 fix commit 4074a26 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Fri Jul 10 16:08:49 2020 +0800 inference without horovod commit 31cb953 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 9 18:41:55 2020 +0800 update commit 838be2a Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 9 15:14:39 2020 +0800 horovod for squad commit 1d374a2 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 9 12:09:19 2020 +0800 fix commit e4fba39 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 9 10:35:08 2020 +0800 remove multiply_grads commit 007f07e Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Jul 7 11:26:38 2020 +0800 multiply_grads commit b8c85bb Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Mon Jul 6 12:28:56 2020 +0800 fix ModelForQABasic commit 0e13a58 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Sat Jul 4 18:42:12 2020 +0800 clip_grad_global_norm with zeros max_grad_norm commit bd270f2 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Fri Jul 3 20:21:31 2020 +0800 fix roberta commit 4fc564c Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Fri Jul 3 19:36:08 2020 +0800 update hyper-parameters of adamw commit 59cffbf Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Fri Jul 3 16:25:46 2020 +0800 try commit a84f782 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 2 20:39:03 2020 +0800 fix mobilebert commit 4bc3a96 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 2 11:14:39 2020 +0800 layer-wise decay commit 07186d5 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Jul 2 02:14:43 2020 +0800 revise commit a5a6475 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Wed Jul 1 19:50:20 2020 +0800 topk commit 34ee884 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Wed Jul 1 19:25:09 2020 +0800 index_update commit 74178e2 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Wed Jul 1 00:48:32 2020 +0800 rename commit fa011aa Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Jun 30 23:40:28 2020 +0800 update commit 402d625 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Jun 30 21:40:30 2020 +0800 multiprocessing for wiki commit ddbde75 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Jun 30 20:41:35 2020 +0800 fix bookcorpus commit 6cc5ccd Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Jun 30 16:39:12 2020 +0800 fix wiki commit 9773efd Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Jun 30 15:52:13 2020 +0800 fix openwebtext commit 1fb8eb8 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Mon Jun 29 19:51:25 2020 +0800 upload gluon_electra_small_owt commit ca83fac Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Mon Jun 29 18:09:48 2020 +0800 revise train_transformer commit 1450f5c Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Mon Jun 29 18:07:04 2020 +0800 revise commit b460bbe Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Mon Jun 29 17:24:00 2020 +0800 repeat for pretraining commit 8ee381b Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Mon Jun 29 17:06:43 2020 +0800 repeat commit aea936f Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Mon Jun 29 16:39:22 2020 +0800 fix mobilebert commit eead164 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Sun Jun 28 18:44:28 2020 +0800 fix commit 8645115 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Sun Jun 28 17:27:43 2020 +0800 update commit 2b7f7a3 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Sun Jun 28 17:18:00 2020 +0800 fix roberta commit 86702fe Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Sun Jun 28 16:27:43 2020 +0800 use_segmentation commit 6d03d7a Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Sun Jun 28 15:52:40 2020 +0800 fix commit 5c0ca43 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Sun Jun 28 15:49:48 2020 +0800 fix token_ids commit ff7aae8 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Sun Jun 28 13:56:07 2020 +0800 fix xlmr commit 2070b86 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Sun Jun 28 13:54:26 2020 +0800 fix roberta commit 70a1887 Author: Leonard Lausen <lausen@amazon.com> Date: Fri Jul 17 00:07:08 2020 +0000 Update for Block API (dmlc#1261) - Remove params and prefix arguments for MXNet 2 and update parameter sharing implementation - Remove Block.name_scope() for MXNet 2 - Remove self.params.get() and self.params.get_constant() commit ea9152b Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Thu Jul 16 15:42:04 2020 -0700 Fixes to make the CI more stable (dmlc#1265) * Some fixes to make the CI more stable * add retries * Update tokenizers.py commit a646c34 Author: ht <wawawa@akane.waseda.jp> Date: Sun Jul 12 02:49:53 2020 +0800 [FEATURE] update backtranslation and add multinomial sampler (dmlc#1259) * back translation bash * split "lang-pair" para in clean_tok_para_corpus * added clean_tok_mono_corpus * fix * add num_process para * fix * fix * add yml * rm yml * update cfg name * update evaluate * added max_update / save_interval_update params * fix * fix * multi gpu inference * fix * update * update multi gpu inference * fix * fix * split evaluate and parallel infer * fix * test * fix * update * add comments * fix * remove todo comment * revert remove todo comment * raw lines remove duplicated '\n' * update multinomaial sampler * fix * fix * fix * fix * sampling * update script * fix * add test_case with k > 1 in topk sampling * fix multinomial sampler * update docs * comments situation eos_id = None * fix Co-authored-by: Hu <huta@a483e74650ff.ant.amazon.com> commit 83e1f13 Author: Leonard Lausen <lausen@amazon.com> Date: Thu Jul 9 20:57:55 2020 -0700 Use Amazon S3 Transfer Acceleration (dmlc#1260) commit cd48efd Author: Leonard Lausen <lausen@amazon.com> Date: Tue Jul 7 17:39:42 2020 -0700 Update codecov action to handle different OS and Python versions (dmlc#1254) codecov/codecov-action#80 (comment) commit 689eba9 Author: Sheng Zha <szha@users.noreply.github.com> Date: Tue Jul 7 09:55:34 2020 -0700 [CI] AWS batch job tool for GluonNLP (Part I) (dmlc#1251) * AWS batch job tool for GluonNLP * limit range Co-authored-by: Xingjian Shi <xshiab@connect.ust.hk> commit e06ff01 Author: Leonard Lausen <lausen@amazon.com> Date: Tue Jul 7 08:36:24 2020 -0700 Pin mxnet version range on CI (dmlc#1257) * frozen_params * remove conversion to a sperate pr * fix * fix * update * test * revise * update performance numbers * update apply_layerwisw_decay * use shuffle * fix mobilebert * fix vocab_file

This does not yet include the fully functional Makefile with the docs_local target so that the notebook compilation step can be executed.

* fix leaky_relu * update mxnet as 0b20200802

…or wmt (PART 1) (dmlc#1284) * set default shuffle=True for boundedbudgetsampler * fix * fix log condition * use horovod to train transformer * fix * add mirror wmt dataset * fix * rename wmt.txt to wmt.json and remove part of urls * fix * tuning params * use get_repo_url() * update average checkpoint cli * paste result of transformer large * fix * fix logging in train_transformer * fix * fix * fix * add transformer base config Co-authored-by: Hu <huta@a483e74650ff.ant.amazon.com>

* update Dockerfile * fix num_out_files * fix run_electra * Revert "update Dockerfile" This reverts commit 80593a2.

…n3 + Fix conversion tool (dmlc#1292) * update update Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Create requirements.txt Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update requirements.txt update Update README.md Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py fix fix Update test_models_bart.py Update test_models_bart.py Update bart.py update Update __init__.py Update electra.py update update Update convert_bert_from_tf_hub.sh update Update unittests.yml fix conversion update fix bert conversion update fix fix Update __init__.py fix bug fix css Update benchmark_utils.py Update benchmark_utils.py update update Update misc.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py no multiprocessing Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py fix bug Update benchmark_utils.py Update benchmark_utils.py try to use mxnet profiler Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py fix update Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py fix Update benchmark_utils.py Update bart.py Update bart.py fix fix Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_gluonnlp.py Update benchmark_gluonnlp.py Update benchmark_gluonnlp.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update README.md * Update benchmark_utils.py * Update benchmark_utils.py * Update requirements.txt * Update benchmark_utils.py * Update benchmark_utils.py * Update benchmark_utils.py * Update benchmark_utils.py * Update benchmark_utils.py * Update benchmark_utils.py * debug * Update benchmark_utils.py * Update benchmark_gluonnlp.py * Update benchmark_gluonnlp.py * Update benchmark_utils.py * Update pretraining_utils.py * Update benchmark_utils.py * update * Update benchmark_utils.py * Update benchmark_utils.py * fix convert * tiny fix * python3 * fix * lower tolerance for albert large and xlarge * Update benchmark_utils.py * fix xlmr * lower tolerance for albert large * update * Update benchmark_utils.py * Update benchmark_utils.py * Update benchmark_utils.py * Update benchmark_utils.py * fix * Squashed commit of the following: commit bd05969 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Aug 11 23:44:53 2020 +0800 lower tolerance for albert large commit f0f9cd6 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Aug 11 14:59:06 2020 +0800 fix xlmr commit edd6655 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Aug 11 14:49:36 2020 +0800 lower tolerance for albert large and xlarge commit d651730 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Aug 11 14:34:55 2020 +0800 fix commit e097c3b Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Aug 11 14:02:13 2020 +0800 python3 commit d6f3fc4 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Aug 11 14:00:28 2020 +0800 tiny fix commit 93bd659 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Aug 11 13:08:34 2020 +0800 fix convert commit 9238d56 Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Mon Aug 10 21:03:13 2020 -0700 Update benchmark_utils.py commit 9bbc581 Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Mon Aug 10 12:58:04 2020 -0700 Update benchmark_utils.py commit b1f5955 Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Mon Aug 10 11:18:43 2020 -0700 update commit a43e65b Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Mon Aug 10 10:32:55 2020 -0700 Update benchmark_utils.py commit 13db82f Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Mon Aug 10 10:16:46 2020 -0700 Update pretraining_utils.py commit fdd9df5 Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Mon Aug 10 08:49:17 2020 -0700 Update benchmark_utils.py commit 44f9c8b Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Mon Aug 10 05:07:45 2020 -0700 Update benchmark_gluonnlp.py commit 45c58b6 Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Mon Aug 10 05:06:05 2020 -0700 Update benchmark_gluonnlp.py commit f0ae933 Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Mon Aug 10 05:04:41 2020 -0700 Update benchmark_utils.py commit 9735edb Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Mon Aug 10 04:59:58 2020 -0700 debug commit d9daf58 Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Mon Aug 10 04:57:17 2020 -0700 Update benchmark_utils.py commit 9e0f631 Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Mon Aug 10 04:56:52 2020 -0700 Update benchmark_utils.py commit 37f224f Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Mon Aug 10 04:56:06 2020 -0700 Update benchmark_utils.py commit 1cf5c7b Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Mon Aug 10 04:54:34 2020 -0700 Update benchmark_utils.py commit 15272f1 Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Mon Aug 10 04:49:28 2020 -0700 Update benchmark_utils.py commit 8215df6 Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Mon Aug 10 04:48:20 2020 -0700 Update benchmark_utils.py commit 1451f03 Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Mon Aug 10 04:42:21 2020 -0700 Update requirements.txt commit 626739d Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Mon Aug 10 04:38:54 2020 -0700 Update benchmark_utils.py commit 1955197 Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Mon Aug 10 04:31:30 2020 -0700 Update benchmark_utils.py commit 2fd7e3b Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Thu Aug 6 23:56:49 2020 -0700 update update Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Create requirements.txt Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update requirements.txt update Update README.md Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py Update benchmark_hf.py fix fix Update test_models_bart.py Update test_models_bart.py Update bart.py update Update __init__.py Update electra.py update update Update convert_bert_from_tf_hub.sh update Update unittests.yml fix conversion update fix bert conversion update fix fix Update __init__.py fix bug fix css Update benchmark_utils.py Update benchmark_utils.py update update Update misc.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py no multiprocessing Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py fix bug Update benchmark_utils.py Update benchmark_utils.py try to use mxnet profiler Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py fix update Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py fix Update benchmark_utils.py Update bart.py Update bart.py fix fix Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_gluonnlp.py Update benchmark_gluonnlp.py Update benchmark_gluonnlp.py Update benchmark_utils.py Update benchmark_utils.py Update benchmark_utils.py Update README.md * fix squad * fix typo * Update benchmark_utils.py * Update benchmark_hf.py * Update benchmark_gluonnlp.py * Update benchmark_gluonnlp.py * Update benchmark_gluonnlp.py * Update benchmark_utils.py * Update benchmark_gluonnlp.py * update * Update benchmark_gluonnlp.py * Update benchmark_gluonnlp.py * Update benchmark_gluonnlp.py * Update benchmark_gluonnlp.py * Update README.md * update * Update benchmark_hf.py * Update benchmark_hf.py * Update requirements.txt * Update benchmark_hf.py * Delete conversion_tool_test.yml * Update README.md * Update README.md * Update README.md * move python --> python3 * try to fix test * fix test case * add test cases * Update README.md * update * update logging config * fix logging config Co-authored-by: ZheyuYe <zheyu.ye1995@gmail.com>

* set default shuffle=True for boundedbudgetsampler * fix * fix log condition * use horovod to train transformer * fix * add mirror wmt dataset * fix * rename wmt.txt to wmt.json and remove part of urls * fix * tuning params * use get_repo_url() * update average checkpoint cli * paste result of transformer large * fix * fix logging in train_transformer * fix * fix * fix * add transformer base config * fix * change to wmt14/full * print more sacrebleu info * fix * add test for num_parts and update behavior of boundedbudgetsampler with even_size * fix * fix * fix * fix logging when using horovd * udpate doc of train transformer * add test case for fail downloading * add a ShardedIterator * fix * fix * fix * change mpirun to horovodrun * make the horovod command complete * use print(sampler) to cover the codes of __repr__ func * empty commit * add test case test_sharded_iterator_even_size Co-authored-by: Hu <huta@a483e74650ff.ant.amazon.com>

* Update submit-job.py Add LICESE + Examples for batch Update docker image update Update README.md Update README.md Update ubuntu18.04-devel.Dockerfile Update ubuntu18.04-devel.Dockerfile Update ubuntu18.04-devel.Dockerfile update Update ubuntu18.04-devel-gpu.Dockerfile fix Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update submit-job.py Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile update Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile update update Update submit-job.py Update submit-job.py Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile try to fix fix batch Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update submit-job.py Update ubuntu18.04-devel-gpu.Dockerfile simplify bert test add files Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile fix Update ubuntu18.04-devel-gpu.Dockerfile * Update ubuntu18.04-devel-gpu.Dockerfile * try to add back mxnet support * Update ubuntu18.04-devel-gpu.Dockerfile * Update ubuntu18.04-devel-gpu.Dockerfile * update * Update ubuntu18.04-devel-gpu.Dockerfile * Update ubuntu18.04-devel-gpu.Dockerfile * Update ubuntu18.04-devel-gpu.Dockerfile * fix issues * update

…on method (dmlc#1307)

* Squashed commit of the following: commit 7525618 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Fri Aug 21 11:25:38 2020 +0800 Squashed commit of the following: commit d8b68c6 Author: Xingjian Shi <xshiab@connect.ust.hk> Date: Thu Aug 20 08:47:56 2020 -0700 [Numpy] Fix AWS Batch + Add Docker Support (dmlc#1302) * Update submit-job.py Add LICESE + Examples for batch Update docker image update Update README.md Update README.md Update ubuntu18.04-devel.Dockerfile Update ubuntu18.04-devel.Dockerfile Update ubuntu18.04-devel.Dockerfile update Update ubuntu18.04-devel-gpu.Dockerfile fix Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update submit-job.py Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile update Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile update update Update submit-job.py Update submit-job.py Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile try to fix fix batch Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update submit-job.py Update ubuntu18.04-devel-gpu.Dockerfile simplify bert test add files Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile fix Update ubuntu18.04-devel-gpu.Dockerfile * Update ubuntu18.04-devel-gpu.Dockerfile * try to add back mxnet support * Update ubuntu18.04-devel-gpu.Dockerfile * Update ubuntu18.04-devel-gpu.Dockerfile * update * Update ubuntu18.04-devel-gpu.Dockerfile * Update ubuntu18.04-devel-gpu.Dockerfile * Update ubuntu18.04-devel-gpu.Dockerfile * fix issues * update commit 6ae558e Author: ht <wawawa@akane.waseda.jp> Date: Thu Aug 20 23:47:30 2020 +0800 [FEATURE]Horovod support for training transformer (PART 2) (dmlc#1301) * set default shuffle=True for boundedbudgetsampler * fix * fix log condition * use horovod to train transformer * fix * add mirror wmt dataset * fix * rename wmt.txt to wmt.json and remove part of urls * fix * tuning params * use get_repo_url() * update average checkpoint cli * paste result of transformer large * fix * fix logging in train_transformer * fix * fix * fix * add transformer base config * fix * change to wmt14/full * print more sacrebleu info * fix * add test for num_parts and update behavior of boundedbudgetsampler with even_size * fix * fix * fix * fix logging when using horovd * udpate doc of train transformer * add test case for fail downloading * add a ShardedIterator * fix * fix * fix * change mpirun to horovodrun * make the horovod command complete * use print(sampler) to cover the codes of __repr__ func * empty commit * add test case test_sharded_iterator_even_size Co-authored-by: Hu <huta@a483e74650ff.ant.amazon.com> commit 1403c6e Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Fri Aug 21 11:15:44 2020 +0800 update uncased_bert_large commit 733a4b6 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Aug 20 20:16:39 2020 +0800 adjust uncased_bert_large commit 770f079 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Aug 20 15:10:57 2020 +0800 Revert "merge xingjian's" This reverts commit ea1f1aa. commit fe74dda Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Aug 20 14:07:36 2020 +0800 update electra small commit 8972343 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Aug 20 14:00:57 2020 +0800 add command to readme commit 8fcde49 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Aug 20 12:30:47 2020 +0800 revise commit 7a625c4 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Thu Aug 20 12:21:58 2020 +0800 update reamde commit 071c6dd Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Wed Aug 19 17:14:53 2020 +0800 update bert squad command commit ea1f1aa Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Aug 18 18:07:01 2020 +0800 merge xingjian's commit 859ab4d Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Aug 18 17:47:01 2020 +0800 dummy example commit 633e683 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Aug 18 17:36:31 2020 +0800 list_backbone_names commit b4aac59 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Aug 18 17:32:51 2020 +0800 update readme commit 54301d9 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Aug 18 13:59:06 2020 +0800 revise batch squad commit e019e27 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Aug 18 13:58:49 2020 +0800 bash convert commit e01eda0 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Aug 18 11:10:51 2020 +0800 update roberta commit 1730ff7 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Tue Aug 18 10:15:27 2020 +0800 revise submit commit de0b4c9 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Mon Aug 17 16:07:58 2020 +0800 upload batch files commit 175de01 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Mon Aug 17 16:05:02 2020 +0800 fix commit 0460ed3 Author: ZheyuYe <zheyu.ye1995@gmail.com> Date: Mon Aug 17 15:48:52 2020 +0800 upload commands * add mobilebert * replace remote * fix branch * fix typo Co-authored-by: Yuma1L <Yuma1L@Zheyu-mac.local>

* make beam search a hybrid block * use mx.np/mx.npx * early_return default to True

* Update README.md Update README.md Update ubuntu18.04-devel-gpu.Dockerfile Update README.md update Update README.md Update README.md Update README.md use python3 -m Update benchmark_utils.py Update benchmark_utils.py Update ubuntu18.04-devel-gpu.Dockerfile Update ubuntu18.04-devel-gpu.Dockerfile * Update ubuntu18.04-devel-gpu.Dockerfile * Update ubuntu18.04-devel-gpu.Dockerfile * Update ubuntu18.04-devel-gpu.Dockerfile * Update ubuntu18.04-devel-gpu.Dockerfile * update * Update README.md * Update README.md * Update ubuntu18.04-devel-gpu.Dockerfile * Update README.md

…line (dmlc#1308) * [CI] Add GPU pytest + Submit jobs to AWS Batch through GitHub Actions * [CI] Update GPU tests and parameters use * [CI] Update CI pipeline * [CI] Add new line * [CI] Update pytest command for cpu test * [CI] Update use_gpu to ctx + add permissions to test.sh * [CI] Update submitted command * [CI] De-stringify input to mxnet attribute * [CI] Change pull_request event to pull_request_target event * [CI] Add new workflow for GPU unit tests

* [CI] Add GPU pytest + Submit jobs to AWS Batch through GitHub Actions * [CI] Update GPU tests and parameters use * [CI] Update CI pipeline * [CI] Add new line * [CI] Update pytest command for cpu test * [CI] Update use_gpu to ctx + add permissions to test.sh * [CI] Update submitted command * [CI] De-stringify input to mxnet attribute * [CI] Change pull_request event to pull_request_target event * [CI] Add new workflow for GPU unit tests * [CI] Update unittests-gpu.yml * [CI] Update unittests-gpu.yml

Co-authored-by: Ubuntu <ubuntu@ip-10-20-2-34.ec2.internal>

* [CI] Add GPU pytest + Submit jobs to AWS Batch through GitHub Actions * [CI] Update GPU tests and parameters use * [CI] Update CI pipeline * [CI] Add new line * [CI] Update pytest command for cpu test * [CI] Update use_gpu to ctx + add permissions to test.sh * [CI] Update submitted command * [CI] De-stringify input to mxnet attribute * [CI] Change pull_request event to pull_request_target event * [CI] Add new workflow for GPU unit tests * [CI] Update unittests-gpu.yml * [CI] Update unittests-gpu.yml * [CI] Update path of test.sh * [CI] Update path of /test * [CI] Update remote to barry-jin/gluon-nlp * [CI] Update remote to dmlc/gluon-nlp * [CI] Add gpu tests for attention cells, bert, electra + Update README * [CI] Change remote from dmlc to barry-jin * [CI] Bug Fix * [CI] Truncate logs + Add failure test * [CI] Duplicate script to submit test and get logs * [CI] Update unittest-gpu * [CI] Quiet the pip install + Redirect the logs to script.log * [CI] Remove asserts * [CI] Simplify ctx statement * [CI] Simplify ctx statement * [CI] test_multi_head_rel_attn_score failed for gpu test * [CI] Finalize gpu test - change remote from barry-jin to dmlc * Delete submit-test.py * [CI] Update test working directory * [CI] Update AWS Batch job type * [CI] Allow test logs downloading

* fix * logging

* [CI] Fix reference issues * [CI] Fix reference issues * [CI] Fix reference issues

* fix valid candidates issue * replace numpy with mxnet numpy * update gumbel trick Co-authored-by: Ubuntu <ubuntu@ip-10-20-2-34.ec2.internal>

* convert gpt2 model * update * update * Update test_models_gpt2.py Co-authored-by: Hu <huta@a483e74650ff.ant.amazon.com> Co-authored-by: Xingjian Shi <xshiab@connect.ust.hk>

szha · 2020-09-01T19:20:50Z

@liuzh91 I tried changing the base to master and got this error message: There are no new commits between base branch 'master' and head branch 'master'. It might be easier to close this one and create a new branch and PR

liuzh47 requested a review from a team as a code owner November 12, 2019 10:32

eric-haibin-lin reviewed Nov 13, 2019

View reviewed changes

liuzh47 force-pushed the master branch from e502764 to 9df613e Compare November 14, 2019 03:27

leezu force-pushed the master branch from 9df613e to 8d27b82 Compare November 14, 2019 03:50

xiaotinghe requested a review from eric-haibin-lin December 11, 2019 11:29

szha and others added 17 commits March 18, 2020 15:31

clean slate for 1.x

de7b23d

Fix bert cfg (dmlc#1245)

982a416

* fix bert cfg * fix lowercase * re-test

fix download

789e2b9

[Numpy] Try to fix the CI (dmlc#1248)

b714eac

* try to fix the CI of the export test * re-enable 3.8 * use skipif to skip the test of python3.8

Pin mxnet version range on CI (dmlc#1257)

e06ff01

[CI] AWS batch job tool for GluonNLP (Part I) (dmlc#1251)

689eba9

* AWS batch job tool for GluonNLP * limit range Co-authored-by: Xingjian Shi <xshiab@connect.ust.hk>

Update codecov action to handle different OS and Python versions (dml…

cd48efd

…c#1254) codecov/codecov-action#80 (comment)

Use Amazon S3 Transfer Acceleration (dmlc#1260)

83e1f13

Fixes to make the CI more stable (dmlc#1265)

ea9152b

* Some fixes to make the CI more stable * add retries * Update tokenizers.py

Update for Block API (dmlc#1261)

70a1887

- Remove params and prefix arguments for MXNet 2 and update parameter sharing implementation - Remove Block.name_scope() for MXNet 2 - Remove self.params.get() and self.params.get_constant()

Fix parameter share regex (dmlc#1267)

9d83fe6

Add fp16 support for Bert QA inference (dmlc#1264)

4743afc

* Add fp16 support for Bert QA inference * change cfg dtype setting from run_squad script * pass dtype as argument to get_backbone

[CI] update batch to gluonnlp-dev (dmlc#1268)

e78a24e

* update batch to gluonnlp-dev * add more types

szha and others added 10 commits July 27, 2020 20:21

add subversion/wget to docker, add readme (dmlc#1279)

4d43f82

[Numpy] Fix SQuAD + Fix GLUE downloading (dmlc#1280)

033214e

* Update run_squad.py * Update run_squad.py * Update prepare_glue.py

[DOC] Add the basic documentation for the embedding API (dmlc#1281)

7e1f9d0

This does not yet include the fully functional Makefile with the docs_local target so that the notebook compilation step can be executed.

Fix gelu (dmlc#1287)

20af58f

* fix leaky_relu * update mxnet as 0b20200802

fix prepare_openwebtext (dmlc#1289)

ded0f99

Fix electra (dmlc#1291)

9e268c0

* update Dockerfile * fix num_out_files * fix run_electra * Revert "update Dockerfile" This reverts commit 80593a2.

szha changed the base branch from master to v0.x August 13, 2020 02:18

liuzh47 force-pushed the master branch from 8d27b82 to 32e87d4 Compare August 20, 2020 03:53

hutao965 and others added 16 commits August 20, 2020 08:47

minor fix for run_electra.py & remove hybridization in the constructi…

d17ec4c

…on method (dmlc#1307)

[MODEL] make beam search a hybrid block (dmlc#1310)

d93356f

* make beam search a hybrid block * use mx.np/mx.npx * early_return default to True

automatically generate date suffix for dev versions (dmlc#1314)

dca17ee

fix typo (dmlc#1317)

39ec921

Co-authored-by: Ubuntu <ubuntu@ip-10-20-2-34.ec2.internal>

fix typo (dmlc#1318)

970318d

Co-authored-by: Ubuntu <ubuntu@ip-10-20-2-34.ec2.internal>

fix dmlc#1315 (dmlc#1319)

66e5e05

* fix * logging

[CI] Fix Source Reference Issues (dmlc#1332)

ff95fb4

* [CI] Fix reference issues * [CI] Fix reference issues * [CI] Fix reference issues

[BUGFIX] fix valid candidates issue (dmlc#1323)

1bd85b6

* fix valid candidates issue * replace numpy with mxnet numpy * update gumbel trick Co-authored-by: Ubuntu <ubuntu@ip-10-20-2-34.ec2.internal>

[MODEL] convert gpt2 model (dmlc#1328)

189bbdc

* convert gpt2 model * update * update * Update test_models_gpt2.py Co-authored-by: Hu <huta@a483e74650ff.ant.amazon.com> Co-authored-by: Xingjian Shi <xshiab@connect.ust.hk>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug Fix] trainer.update(1) should be used after loss.mean() is called #1000

[Bug Fix] trainer.update(1) should be used after loss.mean() is called #1000

liuzh47 commented Nov 12, 2019

codecov bot commented Nov 12, 2019 •

edited

mli commented Nov 13, 2019

leezu commented Nov 13, 2019

eric-haibin-lin left a comment

mli commented Nov 14, 2019

xiaotinghe commented Nov 17, 2019

szha commented Dec 10, 2019

xiaotinghe commented Dec 11, 2019

leezu commented Feb 10, 2020

szha commented Sep 1, 2020 •

edited

[Bug Fix] trainer.update(1) should be used after loss.mean() is called #1000

Are you sure you want to change the base?

[Bug Fix] trainer.update(1) should be used after loss.mean() is called #1000

Conversation

liuzh47 commented Nov 12, 2019

Description

Checklist

Essentials

Changes

Comments

codecov bot commented Nov 12, 2019 • edited

Codecov Report

mli commented Nov 13, 2019

leezu commented Nov 13, 2019

eric-haibin-lin left a comment

Choose a reason for hiding this comment

mli commented Nov 14, 2019

xiaotinghe commented Nov 17, 2019

szha commented Dec 10, 2019

xiaotinghe commented Dec 11, 2019

leezu commented Feb 10, 2020

szha commented Sep 1, 2020 • edited

codecov bot commented Nov 12, 2019 •

edited

szha commented Sep 1, 2020 •

edited