Perplexed

A fork of the simple & hack-able nanoGPT, to illustrate perplexity in this blog post.

installation

pip install torch numpy transformers datasets tiktoken wandb tqdm
# ala Karpathy

¯\_(ツ)_/¯

#TODO: add the commands for folks to reimplement this work

python data/shakespeare_char/prepare.py

Name		Name	Last commit message	Last commit date
Latest commit History 214 Commits
assets		assets
config		config
data		data
.gitattributes		.gitattributes
.gitignore		.gitignore
.pylintrc		.pylintrc
LICENSE		LICENSE
README.md		README.md
bench.py		bench.py
configurator.py		configurator.py
model.py		model.py
sample.py		sample.py
scaling_laws.ipynb		scaling_laws.ipynb
train.py		train.py
transformer_sizing.ipynb		transformer_sizing.ipynb
utils.py		utils.py
visualization.py		visualization.py