民间版中文羊驼模型 v4.0

[ymcui]

本次重点发布了33B版本中文LLaMA和Alpaca模型。

🚀 发布中文LLaMA-33B、Alpaca-33B

• 与其他基础版一致，中文LLaMA-33B使用了20G的中文语料进行预训练
• 指令数据仍然是4M级别，并添加了OASST1数据（中文翻译，约10K）
• 生成效果评价：Plus-7B获得75.3，Plus-13B获得79.4，33B获得82.0👍🏻
• Alpaca-33B在数值计算、代码理解等7B/13B不擅长的任务上获得了显著性能提升

具体评测结果和生成示例请参考效果评测。

💯 提供了C-Eval客观评价指标结果

C-Eval是近期推出的一个包含14K样本的选择型问答数据集，涵盖了52个不同的学科和四个难度级别。中文Alpaca-Plus-13B和Alpaca-33B模型在该测试集上达到了较好水平，甚至超越了LLaMA-65B、Bloomz-mt等参数量大几倍甚至十几倍的千亿级模型。

详细结果请参考：https://github.com/ymcui/Chinese-LLaMA-Alpaca/tree/main#客观效果评测

📚 功能性更新

• 提供了privateGPT示例（Add support for privateGPT #410）：privateGPT是基于LangChain二次开发的项目，可以对本地文档进行分析并利用大模型进行交互问答。相关使用方法请查看本项目Wiki
• 提供了Colab上可运行的Gradio示例（Add quick try Colab Notebook #364、Add Gradio notebook #536）：在Colab上运行notebook，启动基于Gradio的交互式Web服务，可迅速进行部署与体验。Contribution by @sunyuhan19981208

其他更新

• 更新了训练脚本，提升了训练之后进一步合并模型的便利性
• 重构了scripts文件夹，使目录结构更清晰
• 更新了合并模型脚本，使之适配33B/65B模型

For English release note, please refer to Discussion.

---

This discussion was created from the release 民间版中文羊驼模型 v4.0.

[ymcui]

English Release Note v4.0

In this release, we primarily launched the 33B version of the Chinese LLaMA and Alpaca models.

🚀 Launch of Chinese LLaMA-33B and Alpaca-33B

• Consistent with other base versions, Chinese LLaMA-33B is pre-trained using 20G of Chinese corpora.
• Instruction data remains at the 4M level, and OASST1 data (Chinese translation, about 10K) has been added.
• Generation effect evaluation: Plus-7B scored 75.3, Plus-13B scored 79.4, and 33B scored 82.0👍🏻
• Alpaca-33B has achieved significant performance improvement in tasks such as numerical calculation and code understanding, which 7B/13B are not proficient at.

For specific evaluation results and generation examples, please refer to evaluation of results.

💯 Provided C-Eval objective evaluation index results

C-Eval is a recently released multiple-choice question and answer dataset that includes 14K samples, covering 52 different subjects and four levels of difficulty. The Chinese Alpaca-Plus-13B and Alpaca-33B models have achieved a good level on this test set, even surpassing LLaMA-65B, Bloomz-mt, and other models that have several or even more than ten times more parameters.

For detailed results, please refer to: https://github.com/ymcui/Chinese-LLaMA-Alpaca/blob/main/README_EN.md#nlu-performance-test

📚 Functional updates

• Provided a privateGPT example (Add support for privateGPT #410): privateGPT is a project developed based on LangChain, which can analyze local documents and use large models for interactive Q&A. For usage methods, please check this project's Wiki.
• Provided a runnable Gradio example on Colab (Add quick try Colab Notebook #364, Add Gradio notebook #536): Run the notebook on Colab, start the Gradio-based interactive web service for quick deployment and experience. Contribution by @sunyuhan19981208

Other updates

• Updated training scripts, enhancing the convenience of further merging models after training.
• Refactored the scripts folder to make the directory structure clearer.
• Updated the model merging script to adapt to 33B/65B models.