Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

问下readme中32机的吞吐对应的参数可以提供下吗，目前没有复现出来 #49

Open
jianzi123 opened this issue Nov 8, 2023 · 5 comments
Open

问下readme中32机的吞吐对应的参数可以提供下吗，目前没有复现出来 #49

jianzi123 opened this issue Nov 8, 2023 · 5 comments

Comments

@jianzi123
Copy link

No description provided.

@li-yi-dong
Copy link
Collaborator

https://github.com/alibaba/Megatron-LLaMA/blob/main/examples/LLaMA/LLaMA_13_slurm.sh

@HydraQYH
Copy link

180TFlops是如何计算出来呢？

@HydraQYH
Copy link

从4机（32卡）扩展到64机（512卡），扩展的线性度为0.85。根据README中提供的吞吐量1845 tokens / gpu / s，1845 / 0.85 = 2170 tokens / gpu / s。2170 * 6 * 13 / 1000约为169TFlops。我想了解下180TFlops是如何得到的呢？

@li-yi-dong
Copy link
Collaborator

li-yi-dong commented Nov 29, 2023 •
edited

直接参数量*6 有点过于粗糙了。。。。可以根据这个算一下
image

@HydraQYH
Copy link

感谢您的回复。想问下方便提供一下4机（32卡）的token/sec/GPU数据吗？

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jianzi123 @HydraQYH @li-yi-dong