Would this work to train flan-t5-xxl on multiple GPUs?

[experimarketing]

No description provided.

[emendoza2]

Bump

[Shivanandroy]

@experimarketing : Right now, it only works with T5 and mT5 on single GPU. I was away from the development for a couple of months. So, I didn't upgrade it to support FlanT5 and multi GPU.

But, I will integrate it ASAP.

[experimarketing]

Would it work with the -xxl version? I believe model parrellism would be required to run it. As it is too large to run on a single GPU.

[Shivanandroy]

@experimarketing : I'm afraid, It won't!

[SomasekharDS]

@experimarketing : Right now, it only works with T5 and mT5 on single GPU. I was away from the development for a couple of months. So, I didn't upgrade it to support FlanT5 and multi GPU.

But, I will integrate it ASAP.

Thanks for looking into this. Kindly let us know after completion. one more thing really many thanks for developing this library. It simplified the usage of the T5 model.