Skip to content

Latest commit

 

History

History

deepspeed

Folders and files

NameName
Last commit message
Last commit date

parent directory

..

README.md

README.md

 
 

__init__.py

__init__.py

 
 

deepspeed_config.json

deepspeed_config.json

 
 

ds_fairseq.py

ds_fairseq.py

 
 

ds_fairseq_argument.py

ds_fairseq_argument.py

 
 

ds_fairseq_data.py

ds_fairseq_data.py

 
 

ds_fairseq_wmt14en2de.sh

ds_fairseq_wmt14en2de.sh

 
 

README.md

LightSeq for Fairseq+DeepSpeed

This repo contains an example for how to use LightSeq to accerate the training of translation task in Fairseq, together with DeepSpeed for distributed strategies and optimizers. We provide a new trainer for translation task to connect Fairseq and DeepSpeed.

First you should install these requirements.

pip install torch ninja fairseq deepspeed

Then you can train a translation task on wmt14 en2de dataset by running the following script:

sh examples/training/deepspeed/ds_fairseq_wmt14en2de.sh

This script firstly download the dataset, and then run native Fairseq training script using DeepSpeed launcher without any other parameter modifications.