Flow Matching for Conditional Text Generation in a Few Sampling Steps

EACL 2024

This repository represents the official implementation of the EACL2024 paper titled "Flow Matching for Conditional Text Generation in a Few Sampling Steps".

Vincent Tao Hu, Di Wu, Yuki M. Asano, Pascal Mettes, Basura Fernando, Bjorn Ommer, Cees G.M. Snoek

Dataset

https://drive.google.com/drive/folders/1sU8CcOJE_aaaKLijBNzr-4y1i1YoZet2?usp=drive_link

Run

CUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --nnodes=1 --nproc-per-node=4 flow_train.py 
CUDA_VISIBLE_DEVICES=0,2 torchrun --nnodes=1 --nproc-per-node=2 flow_train.py  data=qg
CUDA_VISIBLE_DEVICES=6 torchrun --nnodes=1 --nproc-per-node=1 flow_train.py  data=qg

Evaluation

python flow_sample_eval_s2s.py   --config.eval.is_debug=0 --config=cfgs/rflow_xxx.py --config.eval.model_path='xxxx' --config.eval.ode_stepnum=1

Environment Preparation

conda create -n flowseq  python=3.10
conda install -c "nvidia/label/cuda-11.8.0" cuda-toolkit
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia
pip install  torchdiffeq  matplotlib h5py  accelerate loguru blobfile ml_collections
pip install hydra-core wandb einops scikit-learn --upgrade
pip install einops 
pip install transformers
pip install nltk bert_score datasets torchmetrics

Optional

pip install diffusers

Common Issue

Typical Issue:

https://github.com/Shark-NLP/DiffuSeq/issues/5
https://github.com/Shark-NLP/DiffuSeq/issues/22

Citation

Please add the citation if our paper or code helps you.

@inproceedings{HuEACL2024,
        title = {Flow Matching for Conditional Text Generation in a Few Sampling Steps},
        author = {Vincent Tao Hu and Di Wu and Yuki M Asano and Pascal Mettes and Basura Fernando and Björn Ommer and Cees G M Snoek},
        year = {2024},
        date = {2024-03-27},
        booktitle = {EACL},
        tppubtype = {inproceedings}
        }