👋 Hi, I’m geronimo

trying to understand LLMs. This is my journey so far:

🚀 Tutorials and Repositories

• A failed experiment with LISA: "Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning", code, paper
• 🛠️ Memory-efficient LLM Training with GaLore, yet another PEFT approach, code
• ⚖️ Evaluating LLMs with Semantic Similarity, code
• 🛠️ Finetune TinyLlama and StableLM 2, code
• 🛠️ Finetune Microsoft's Phi-2, code
• 🛠️ Finetune Mamba, code
• 🛠️ Finetune Llama2 and Mistral using QLoRA, code
• ⚖️ Evaluate LLM language capabilities with meta's Belebele benchmark, code
• ⚖️ Evaluate LLM language capabilities with BLEU, code
• ⚖️ Llama2-70B as a judge of LLMs performs almost as good as GPT-4, code
• ⚖️ Validation loss is not a good metric for chatbot quality
• ⚖️ Use GPT3.5 as a judge of open-source LLMs, code
• 🛠️ Finetune Llama on podcast transripts with QLoRA, code
• 💅 Use Stable Diffusion for sketch-guided image generation, code

💎 Other Repositories

• Collection of minimal scripts for 24GB GPUs
• Needle in a haystack test for local LLMs
• FSDP multi-GPU training of Mamba
• Evaluate an LLMs ability to follow instructions
• Multi-GPU inference with 🤗 accelerate
• Trial run of ReLoRA for low-resource continued pretraining of llama2-7b

Twitter Hugging Face