Skip to content

MFTCoder v0.3.0: Support more open source models, support Self-Paced Loss, support FSDP

Latest
Compare
Choose a tag to compare
@chencyudel chencyudel released this 19 Jan 11:16
· 14 commits to main since this release
e5243da

Updates:

  1. Mainly for MFTCoder-accelerate.
  2. It now supports more open source models like Mistral, Mixtral(MoE), DeepSeek-coder, chatglm3.
  3. It supports FSDP as an option.
  4. It also supports Self-paced Loss as a solution for convergence balance in Multitask Fine-tuning.