Config file of ViT-B/16

[shashankvkt]

Hi Hugo,
Thanks for this sharing your amazing work. I was trying to train ViT-B/16 from scratch on ImageNet-1k using the hyperparams reported in your DeIT paper. I'm pretty sure I'm missing something, but I'm unable to reach 81.8%. With the hyperparams I use, I get around 78.6% which is even worse than ViT-S/16.

Could you please share the training command line for ViT-B/16 or share the config file for the same?
Thanks a lot.

[TouvronHugo]

Hi @shashankvkt,
Thanks for your message. The command line is:
python run_with_submitit.py --model deit_base_patch16_224 --data-path /path/to/imagenet
Best,
Hugo

(It's here in the README.)

[shashankvkt]

Hi Hugo,
Thanks for your response. I had used exactly the same command line, but this time I get 79.2% (0.6% increase than previous) and still dont get close to 81.8. I use a smaller batch size of 1024 on 4 GPUs, with a learning rate 1e-3, warm up of 5 epochs. Do you think it's due to the batch size that is causing this poor performance?
Thanks