how to implement cosub training use deit-III

[xiaoguang-1]

Hi,
i want to use cosub trainging method for custom images classification work, but i don't find implemention in the code. Does this work contains the details of cosub training method. If yes, my computer can't support RASamper ,does it have an influence?
Best,
xiaoguang

[xiaoguang-1]

Trouble you again,
I would like to know how the residual function at efficient stochastic depth is implemented? Thank you so much!
yours best,
xiaoguang

[TouvronHugo]

Hi @xiaoguang-1 ,
Thanks for your message.
I have added cosub in the code you need to use stochastic depth with cosub but RASampler is not mandatory.
I still need to add the efficient version of Stochastic depth but the code works without it.
Best,
Hugo