T81 559:Applications of Generative Artificial Intelligence

Washington University in St. Louis

Instructor: Jeff Heaton

• Section 1. Fall 2024, Tuesday, 6:00 PM, Location: TBD

Course Description

This course covers the dynamic world of Generative Artificial Intelligence providing hands-on practical applications of Large Language Models (LLMs) and advanced text-to-image networks. Using Python as the primary tool, students will interact with OpenAI's models for both text and images. The course begins with a solid foundation in generative AI principles, moving swiftly into the utilization of LangChain for model-agnostic access and the management of prompts, indexes, chains, and agents. A significant focus is placed on the integration of the Retrieval-Augmented Generation (RAG) model with graph databases, unlocking new possibilities in AI applications.

As the course progresses, students will delve into sophisticated image generation and augmentation techniques, including LORA (LOw-Rank Adaptation), and learn the art of fine-tuning generative neural networks for specific needs. The final part of the course is dedicated to mastering prompt engineering, a critical skill for optimizing the efficiency and creativity of AI outputs. Ideal for students, researchers, and professionals in computer science or related fields, this course offers a transformative learning experience where technology meets creativity, paving the way for innovative applications in the realm of Generative AI.

Note: This course will require the purchase of up to $100 in OpenAI API credits to complete the coruse.

Objectives

1. Learn how Generative AI fits into the landscape of deep learning and predictive AI.
2. Be able to create ChatBots, Agents, and other LLM-based automation assistants.
3. Understand how to make use of image generative AI programatically.

Syllabus

This syllabus presents the expected class schedule, due dates, and reading assignments. Download current syllabus.

Module	Content
Module 1 Meet on 08/27/2024	Module 1: Introduction to Generative AI 1.1: Course Overview 1.2: Generative AI Overview 1.3: Introduction to OpenAI 1.4: Introduction to LangChain 1.5: Prompt Engineering We will meet on campus this week! (first meeting)
Module 2 Week of 09/03/2024	Module 2: Prompt Based Development 2.1: Prompting for Code Generation 2.2: Handling Revision Prompts 2.3: Using a LLM to Help Debug 2.4: Tracking Prompts in Software Development 2.5: Limits of LLM Code Generation Module 1 Assignment due: 09/04/2024 Icebreaker due: 09/04/2024
Module 3 Week of 09/10/2024	Module 3: Introduction to Large Language Models 3.1: Foundation Models 3.2: Text Generation 3.3: Text Summarization 3.4: LLM Writes a Book 3.5 Small Large Language Models Module 2 Program due: 09/11/2024
Module 4 Week of 09/17/2024	Module 4: LangChain: Chat and Memory 4.1: LangChain Conversations 4.2: Conversation Buffer Window Memory 4.3: Conversation Token Buffer Memory 4.4: Conversation Summary Memory 4.5: Persisting Langchain Memory Module 3: Program due: 09/18/2024
Module 5 Meet on 09/24/2024	Module 5: LangChain: Data Extraction 5.1: Structured Output Parser 5.2: Other Parsers (CSV, JSON, Pandas, Datetime) 5.3: Pydantic parser 5.4: Custom Output Parser 5.5: Output-Fixing Parser Module 4 Program due: 09/25/2024 We will meet on campus this week! (second meeting)
Module 6 Week of 10/01/2024	Module 6: Retrieval-Augmented Generation (RAG) 6.1 Introduction to RAG 6.2 Embeddings 6.3 Indexing Networks 6.4 Q&A Over Documents 6.5 Vector Databases Module 5 Program due: 10/02/2024
Module 7 Week of 10/15/2024	Module 7: LangChain: Agents 7.1: Introduction to Transformers 7.2: Accessing the ChatGPT API 7.3: LLM Memory 7.4: Introduction to Embeddings 7.5: Prompt Engineering Module 6 Program due: 10/16/2024
Module 8 Meet on 10/22/2024	Module 8: Introduction to StreamLit 8.1: Running StreamLit in Google Colab 8.2: StreamLit Introduction 8.3: Understanding Streamlit State 8.4: Creating a Chat Application 8.5: More Advanced Chat Application Module 7 Assignment due: 10/23/2024 We will meet on campus this week! (third meeting)
Module 9 Week of 10/29/2024	Module 9: Kaggle Assignment 9.1: Introduction to Kaggle 9.2: Kaggle Notebooks 9.3: Accessing Small LLM from Kaggle 9.4: Evaluating LLM Performance 9.5: Current Semester's Kaggle Module 8 Assignment due: 10/30/2024
Module 10 Week of 11/05/2024	Module 10: Text to Image Generative AI 10.1: Stable Diffusion 10.2: Calling Dall-E 10.3: Upscaling Images 10.4: Introduction to Dreambooth 10.5: Adding Images to ChatBot Module 9 Assignment due: 11/06/2024
Module 11 Week of 11/12/2024	Module 11: Fine Tuning 11.1: When is fine tuning necessary 11.2: Preparing a dataset for fine tuning 11.3: OepnAI Fine Tuning 11.4: Application of Fine Tuning 11.5: Evaluating Fine Tuning and Optimization Module 10 Assignment due: 11/13/2024
Module 12 Week of 11/19/2024	Module 12: Prompt Engineering Kaggle Assignment due: 04/15/2024 (approx 4-6PM, due to Kaggle GMT timezone) 12.1 Towards the Perfect Prompt 12.2 Prompt Engineering for Images 12.3 Prompt Engineering for LLM 12.4 Advanced Tips for Prompt Engineering 12.5 Evaluating Bias in Prompts
Module 13 Meet on 11/26/2024	Module 13: Speech Processing 13.1: Audio Generation and Recognition 13.2: OpenAI Speech Generation 13.3: OpenAI Speech Recognition Part 13.4: A Voice-Based ChatBot 13.5: Future Directions in GenAI We will meet on campus this week! (fourth meeting) Final project due: 12/03/2024