Question about the tokens/per second/GPU

[P3ngLiu]

❓ The question

Thanks for your great work. I have a question about the tokens/second/GPU when training the 7B model. I checked your TRAINLOG.md, it saids the tokens/second/GPU for 7B is 1200. But when I check the wandb logs, the tokens/second/GPU for 7B model is about 2500. How did you get this dramatic improvements?