[AI][Megtron] Train GPT models with NVIDIA Megatron-LM

[mozhgan-kch]

This Bootcamp is designed to give NLP researchers an overview of the fundamentals of NVIDIA Megatron-LM ( NVIDIA open-source framework for training very large language models). The focus will be on training GPT Megatron models specifically.

It will consist of intro to Megatron-LM code base, converting data to mmap format, understanding model parallel, data-parallel and how to config your training, then train and profile GPT Megatron models.

[Zenodia]

I forked it and created a branch called megatron , i am now ready to create a PR , but i am not sure how you want me to do this. the ready repo can be found here https://github.com/Zenodia/gpubootcamp/tree/megatron

[mozhgan-kch]

Add a link in the start notebook to our other materials. Check the AI materials for reference. Assigned to @mozhgan-kch