This repository is my personal attempt at learning how to build a GPT-like model using PyTorch. The goal is to understand the architecture and the training process of a generative model.
- Neural Networks: Zero to Hero
- Create a Large Language Model from Scratch with Python
- Attention is All You Need
- Python 3.10+
- Accelerator (CUDA, MPS, etc.)
A makefile is include in the project that will create the virtual environment and install the dependencies.
makeA setup.ps1 script is included in the project that will create the virtual environment and install the dependencies.
.\setup.ps1Note: Reload VSCode using the command palette
Developer: Reload Windowto allow the virtual environment to be auto-selected.