t81_559_class_03_2_text_gen.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "whjsJasuhstV"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/jeffheaton/app_generative_ai/blob/main/t81_559_class_03_2_text_gen.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "euOZxlIMhstX"
   },
   "source": [
    "# T81-559: Applications of Generative Artificial Intelligence\n",
    "**Module 3: Large Language Models**\n",
    "* Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/), McKelvey School of Engineering, [Washington University in St. Louis](https://engineering.wustl.edu/Programs/Pages/default.aspx)\n",
    "* For more information visit the [class website](https://sites.wustl.edu/jeffheaton/t81-558/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "d4Yov72PhstY"
   },
   "source": [
    "# Module 3 Material\n",
    "\n",
    "* Part 3.1: Foundation Models [[Video]]() [[Notebook]](t81_559_class_03_1_llm.ipynb)\n",
    "* **Part 3.2: Text Generation** [[Video]]() [[Notebook]](t81_559_class_03_2_text_gen.ipynb)\n",
    "* Part 3.3: Text Summarization [[Video]]() [[Notebook]](t81_559_class_03_3_text_summary.ipynb)\n",
    "* Part 3.4: Text Classification [[Video]]() [[Notebook]](t81_559_class_03_4_classification.ipynb)\n",
    "* Part 3.5: LLM Writes a Book [[Video]]() [[Notebook]](t81_559_class_03_5_book.ipynb)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AcAUP0c3hstY"
   },
   "source": [
    "# Google CoLab Instructions\n",
    "\n",
    "The following code ensures that Google CoLab is running and maps Google Drive if needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "xsI496h5hstZ",
    "outputId": "17d351c0-acb9-4b51-dd06-beef6878dcb9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: using Google CoLab\n",
      "Collecting langchain\n",
      "  Downloading langchain-0.1.16-py3-none-any.whl (817 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m817.7/817.7 kB\u001b[0m \u001b[31m4.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting langchain_openai\n",
      "  Downloading langchain_openai-0.1.4-py3-none-any.whl (33 kB)\n",
      "Requirement already satisfied: PyYAML>=5.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (6.0.1)\n",
      "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.0.29)\n",
      "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (3.9.5)\n",
      "Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (4.0.3)\n",
      "Collecting dataclasses-json<0.7,>=0.5.7 (from langchain)\n",
      "  Downloading dataclasses_json-0.6.4-py3-none-any.whl (28 kB)\n",
      "Collecting jsonpatch<2.0,>=1.33 (from langchain)\n",
      "  Downloading jsonpatch-1.33-py2.py3-none-any.whl (12 kB)\n",
      "Collecting langchain-community<0.1,>=0.0.32 (from langchain)\n",
      "  Downloading langchain_community-0.0.34-py3-none-any.whl (1.9 MB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.9/1.9 MB\u001b[0m \u001b[31m22.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting langchain-core<0.2.0,>=0.1.42 (from langchain)\n",
      "  Downloading langchain_core-0.1.46-py3-none-any.whl (299 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m299.3/299.3 kB\u001b[0m \u001b[31m11.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting langchain-text-splitters<0.1,>=0.0.1 (from langchain)\n",
      "  Downloading langchain_text_splitters-0.0.1-py3-none-any.whl (21 kB)\n",
      "Collecting langsmith<0.2.0,>=0.1.17 (from langchain)\n",
      "  Downloading langsmith-0.1.51-py3-none-any.whl (115 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m116.0/116.0 kB\u001b[0m \u001b[31m2.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: numpy<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.25.2)\n",
      "Requirement already satisfied: pydantic<3,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.7.0)\n",
      "Requirement already satisfied: requests<3,>=2 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.31.0)\n",
      "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (8.2.3)\n",
      "Collecting openai<2.0.0,>=1.10.0 (from langchain_openai)\n",
      "  Downloading openai-1.23.6-py3-none-any.whl (311 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m311.6/311.6 kB\u001b[0m \u001b[31m12.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting tiktoken<1,>=0.5.2 (from langchain_openai)\n",
      "  Downloading tiktoken-0.6.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.8 MB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.8/1.8 MB\u001b[0m \u001b[31m27.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.3.1)\n",
      "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (23.2.0)\n",
      "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.4.1)\n",
      "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (6.0.5)\n",
      "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.9.4)\n",
      "Collecting marshmallow<4.0.0,>=3.18.0 (from dataclasses-json<0.7,>=0.5.7->langchain)\n",
      "  Downloading marshmallow-3.21.1-py3-none-any.whl (49 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.4/49.4 kB\u001b[0m \u001b[31m2.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting typing-inspect<1,>=0.4.0 (from dataclasses-json<0.7,>=0.5.7->langchain)\n",
      "  Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB)\n",
      "Collecting jsonpointer>=1.9 (from jsonpatch<2.0,>=1.33->langchain)\n",
      "  Downloading jsonpointer-2.4-py2.py3-none-any.whl (7.8 kB)\n",
      "Collecting packaging<24.0,>=23.2 (from langchain-core<0.2.0,>=0.1.42->langchain)\n",
      "  Downloading packaging-23.2-py3-none-any.whl (53 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m53.0/53.0 kB\u001b[0m \u001b[31m3.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting orjson<4.0.0,>=3.9.14 (from langsmith<0.2.0,>=0.1.17->langchain)\n",
      "  Downloading orjson-3.10.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (141 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m141.1/141.1 kB\u001b[0m \u001b[31m2.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (3.7.1)\n",
      "Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (1.7.0)\n",
      "Collecting httpx<1,>=0.23.0 (from openai<2.0.0,>=1.10.0->langchain_openai)\n",
      "  Downloading httpx-0.27.0-py3-none-any.whl (75 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.6/75.6 kB\u001b[0m \u001b[31m5.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hRequirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (1.3.1)\n",
      "Requirement already satisfied: tqdm>4 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (4.66.2)\n",
      "Requirement already satisfied: typing-extensions<5,>=4.7 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.10.0->langchain_openai) (4.11.0)\n",
      "Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain) (0.6.0)\n",
      "Requirement already satisfied: pydantic-core==2.18.1 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1->langchain) (2.18.1)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.3.2)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.7)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2.0.7)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2024.2.2)\n",
      "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain) (3.0.3)\n",
      "Requirement already satisfied: regex>=2022.1.18 in /usr/local/lib/python3.10/dist-packages (from tiktoken<1,>=0.5.2->langchain_openai) (2023.12.25)\n",
      "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai<2.0.0,>=1.10.0->langchain_openai) (1.2.1)\n",
      "Collecting httpcore==1.* (from httpx<1,>=0.23.0->openai<2.0.0,>=1.10.0->langchain_openai)\n",
      "  Downloading httpcore-1.0.5-py3-none-any.whl (77 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m5.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting h11<0.15,>=0.13 (from httpcore==1.*->httpx<1,>=0.23.0->openai<2.0.0,>=1.10.0->langchain_openai)\n",
      "  Downloading h11-0.14.0-py3-none-any.whl (58 kB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m4.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25hCollecting mypy-extensions>=0.3.0 (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain)\n",
      "  Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)\n",
      "Installing collected packages: packaging, orjson, mypy-extensions, jsonpointer, h11, typing-inspect, tiktoken, marshmallow, jsonpatch, httpcore, langsmith, httpx, dataclasses-json, openai, langchain-core, langchain-text-splitters, langchain_openai, langchain-community, langchain\n",
      "  Attempting uninstall: packaging\n",
      "    Found existing installation: packaging 24.0\n",
      "    Uninstalling packaging-24.0:\n",
      "      Successfully uninstalled packaging-24.0\n",
      "Successfully installed dataclasses-json-0.6.4 h11-0.14.0 httpcore-1.0.5 httpx-0.27.0 jsonpatch-1.33 jsonpointer-2.4 langchain-0.1.16 langchain-community-0.0.34 langchain-core-0.1.46 langchain-text-splitters-0.0.1 langchain_openai-0.1.4 langsmith-0.1.51 marshmallow-3.21.1 mypy-extensions-1.0.0 openai-1.23.6 orjson-3.10.1 packaging-23.2 tiktoken-0.6.0 typing-inspect-0.9.0\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "try:\n",
    "    from google.colab import drive, userdata\n",
    "    COLAB = True\n",
    "    print(\"Note: using Google CoLab\")\n",
    "except:\n",
    "    print(\"Note: not using Google CoLab\")\n",
    "    COLAB = False\n",
    "\n",
    "# OpenAI Secrets\n",
    "if COLAB:\n",
    "    os.environ[\"OPENAI_API_KEY\"] = userdata.get('OPENAI_API_KEY')\n",
    "\n",
    "# Install needed libraries in CoLab\n",
    "if COLAB:\n",
    "    !pip install langchain langchain_openai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pC9A-LaYhsta"
   },
   "source": [
    "# 3.2: Text Generation\n",
    "\n",
    "Text generation is one of the most common tasks for LLMs. We've already seen how to use the LLM to generate code; generating regular text for human consumption is similar. To generate text, we will not use a conversational chat style; instead, we will send prompts to LangChain and receive the generated text.\n",
    "\n",
    "We use the following code to query the LLM for text generation.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "TMF-rtxgRAea"
   },
   "outputs": [],
   "source": [
    "from langchain_core.messages import HumanMessage, SystemMessage\n",
    "from langchain_core.prompts.chat import (\n",
    "    ChatPromptTemplate,\n",
    "    HumanMessagePromptTemplate,\n",
    "    SystemMessagePromptTemplate,\n",
    ")\n",
    "from langchain_openai import ChatOpenAI\n",
    "from IPython.display import display_markdown\n",
    "\n",
    "MODEL = 'gpt-3.5-turbo'\n",
    "TEMPERATURE = 0.2\n",
    "\n",
    "def get_response(llm, prompt):\n",
    "  messages = [\n",
    "      SystemMessage(\n",
    "          content=\"You are a helpful assistant that answers questions accurately.\"\n",
    "      ),\n",
    "      HumanMessage(content=prompt),\n",
    "  ]\n",
    "\n",
    "  print(\"Model response:\")\n",
    "  output = llm.invoke(messages)\n",
    "  display_markdown(output.content, raw=True)\n",
    "\n",
    "# Initialize the OpenAI LLM with your API key\n",
    "llm = ChatOpenAI(\n",
    "  model=MODEL,\n",
    "  temperature=TEMPERATURE,\n",
    "  n= 1,\n",
    "  max_tokens= 256)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DB0IAW8vBJLV"
   },
   "source": [
    "## Text Generation Patterns\n",
    "\n",
    "For simple text generation, you will see several different prompting patterns. These patterns vary depending on the amount of information you provide the LLM. The patterns we will examine in this module are listed here.\n",
    "\n",
    "* Zero-Shot\n",
    "* One-Shot\n",
    "* Few-Shot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9GeVTcwLl4xi"
   },
   "source": [
    "\n",
    "## Zero-Shot Text Generation\n",
    "\n",
    "A zero-shot prompt for text generation is a method where you provide a language model with a single prompt to generate text, without any prior fine-tuning or specific training on related tasks. To use this approach effectively, you should craft a detailed and clear prompt that communicates exactly what you want the model to generate. Include the type of content, style, and any specific information or constraints that are important to the task. For instance, if you're asking for a business email, you might specify the tone (formal or informal), the main points to cover (meeting time, purpose, attendees), and any call to action. The key is to be explicit about the desired output to guide the model's response accurately, as it relies solely on the information provided in the prompt to produce relevant and coherent text. This method is highly versatile and can be applied across various text generation tasks without the need for customized training.\n",
    "\n",
    "The following text is an example of a zero-shot prompt. I make many requests and provide information about the student, but I do not give the LLM a sample to work from."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 280
    },
    "id": "bt_Ra8TOw1SP",
    "outputId": "0b75ae66-468e-4fb7-85eb-16ab7f67f8d9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model response:\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "```markdown\n",
       "Dear Admissions Committee,\n",
       "\n",
       "It is with great pleasure that I write this letter of recommendation for John Smith, a former student of mine at Washington University in St. Louis. I had the privilege of teaching John in INFO 558: Applications of Deep Neural Networks during the spring semester of 2020, where he demonstrated exceptional dedication, intelligence, and a strong work ethic.\n",
       "\n",
       "John's academic performance in my course was outstanding, earning him an A+ grade. His enthusiasm for the subject matter was evident in his active participation in class discussions and his ability to grasp complex concepts quickly. John consistently displayed a high level of analytical thinking and problem-solving skills, which are essential qualities for success in the field of Computer Science.\n",
       "\n",
       "Since graduating with a Master's degree in Quantitative Finance from WashU, John has excelled in his role as a Senior Financial Risk Analyst at RGA. His responsibilities involve developing automation tools and programming for strategic analysis, showcasing his proficiency in coding and data analysis. John's decision to pursue a part-time Master's in Computer Science while working full-time is a testament to his commitment to continuous learning and professional growth.\n",
       "\n",
       "I have no doubt that John's passion for technology, coupled with his strong academic background and practical experience, make him an ideal candidate for the"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "print(get_response(llm, \"\"\"\n",
    "Generate a positive letter of reccomendation for John Smith, a student of mine\n",
    "for INFO 558 at Washington University, my name is Jeff Heaton. He is applying\n",
    "for a Master of Science in Computer Science. Just give me the\n",
    "body text of the letter, no header or footer. Format in markdown.\n",
    "Below is his request.\n",
    "\n",
    "I hope this message finds you well and that you are enjoying the holiday season!\n",
    "I am John Smith (ID: 1234), a proud alumnus of WashU, having graduated in\n",
    "January 2021 with a Master’s degree in Quantitative Finance.\n",
    "\n",
    "During the spring semester of 2020, I had the pleasure of attending your course,\n",
    "INFO 558: Applications of Deep Neural Networks, which was an elective for my\n",
    "master's program. I thoroughly enjoyed the content and was deeply engaged\n",
    "throughout, culminating in an A+ grade.\n",
    "\n",
    "Since graduating with a 3.99 GPA—top of my major—I have been working as a Senior\n",
    "Financial Risk Analyst at RGA. My role primarily involves developing automation\n",
    "tools and programming for strategic analysis and other analytical tasks. To\n",
    "further enhance my programming skills and knowledge, I am planning to pursue a\n",
    "part-time Master's in Computer Science while continuing to work at RGA.\n",
    "\n",
    "I am a great admirer of your work (I’m a regular viewer of your YouTube channel\n",
    "and have recommended it to my colleagues), and your insights would be invaluable\n",
    "in my application. I am applying to the following programs:\n",
    "\n",
    "Georgia Tech, Master of Science in Computer Science\n",
    "University of Pennsylvania, Master of Computer & Information Technology\n",
    "Could I possibly ask for your support with a recommendation letter for these\n",
    "applications? I have attached my resume for your reference and am happy to\n",
    "provide any additional information you might need.\n",
    "\n",
    "Thank you very much for considering my request. I look forward to your\n",
    "positive response.\n",
    "\n",
    "Warm regards,\n",
    "\n",
    "John\n",
    "\"\"\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XhpXOo-xOair"
   },
   "source": [
    "## One-Shot Text Generation\n",
    "\n",
    "A one-shot prompt for text generation is a technique where you provide a single, detailed input to a language model to generate text based on that prompt. To use this effectively, start by crafting a clear and concise prompt that includes all necessary details and context needed for the output you desire. Specify the style, tone, and specific elements you want to include. For example, if you want a descriptive paragraph about a seaside town, mention key details like the time of day, the atmosphere, and any particular imagery or emotions you want to evoke. This precision helps the model understand your expectations and produce more relevant and focused content. Once you've prepared your prompt, simply input it into the text generation tool and evaluate the generated text, tweaking your prompt as needed to refine the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 280
    },
    "id": "Jc5ICsAUOdm0",
    "outputId": "20b910a5-ded6-449d-f5b9-a8983b52f0c6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model response:\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "```markdown\n",
       "To Whom It May Concern:\n",
       "\n",
       "I am pleased to write this letter of recommendation for John Smith, a former student of mine in INFO 558: Applications of Deep Neural Networks at Washington University in St. Louis. John excelled in my course, earning an A+ grade during the Fall 2019 semester.\n",
       "\n",
       "Throughout the semester, I had the opportunity to engage with John in discussions related to the course material and his research interests. Despite not having a background in computer science, John demonstrated strong Python programming skills and the ability to effectively translate his ideas into code. As a VP of data science at RGA, a Fortune 500 insurance company, I recognize the value of individuals like John who possess a blend of finance knowledge, expertise in advanced machine learning concepts, and proficient coding abilities essential for roles in data science.\n",
       "\n",
       "John's dedication to learning and his impressive academic performance make him a standout candidate for the Master of Science in Computer Science program. I have no doubt that he will continue to excel in his academic pursuits and contribute positively to any program he chooses to undertake.\n",
       "\n",
       "I highly recommend John for admission to the Master of Science in Computer Science program at Georgia Tech and the Master of Computer & Information Technology program at the University of Pennsylvania. His passion for learning, strong"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "print(get_response(llm, \"\"\"\n",
    "Generate a positive letter of reccomendation for John Smith, a student of mine\n",
    "for INFO 558 at Washington University, my name is Jeff Heaton. He is applying\n",
    "for a Master of Science in Computer Science. Just give me the\n",
    "body text of the letter, no header or footer. Format in markdown.\n",
    "\n",
    "-----------------\n",
    "This is an example letter of reccomendation, written by me.\n",
    "\n",
    "To Whom It May Concern:\n",
    "John earned an A+ in my course Applications of Deep Neural Networks for the\n",
    "Fall 2019 semester at Washington University in St. Louis. During the semester\n",
    "I got a chance to know John through several discussions, both about my course\n",
    "and his research interests. While John did not come from a computer science\n",
    "background he has demonstrated himself as a capable Python programmer and was\n",
    "able to express his ideas in code.  My primary career is as a VP of data science\n",
    "at RGA, a Fortune 500 insurance company.  In this role I know the value of\n",
    "individuals, such as John, who have a background in finance, understand\n",
    "advanced machine learning topics, and can code sufficiently well to function\n",
    "as a data scientist.\n",
    "\n",
    "-----------\n",
    "The details of this student's request follows.\n",
    "\n",
    "I hope this message finds you well and that you are enjoying the holiday season!\n",
    "I am John Smith (ID: 1234), a proud alumnus of WashU, having graduated in\n",
    "January 2021 with a Master’s degree in Quantitative Finance.\n",
    "\n",
    "During the spring semester of 2020, I had the pleasure of attending your course,\n",
    "INFO 558: Applications of Deep Neural Networks, which was an elective for my\n",
    "master's program. I thoroughly enjoyed the content and was deeply engaged\n",
    "throughout, culminating in an A+ grade.\n",
    "\n",
    "Since graduating with a 3.99 GPA—top of my major—I have been working as a Senior\n",
    "Financial Risk Analyst at RGA. My role primarily involves developing automation\n",
    "tools and programming for strategic analysis and other analytical tasks. To\n",
    "further enhance my programming skills and knowledge, I am planning to pursue a\n",
    "part-time Master's in Computer Science while continuing to work at RGA.\n",
    "\n",
    "I am a great admirer of your work (I’m a regular viewer of your YouTube channel\n",
    "and have recommended it to my colleagues), and your insights would be invaluable\n",
    "in my application. I am applying to the following programs:\n",
    "\n",
    "Georgia Tech, Master of Science in Computer Science\n",
    "University of Pennsylvania, Master of Computer & Information Technology\n",
    "Could I possibly ask for your support with a recommendation letter for these\n",
    "applications? I have attached my resume for your reference and am happy to\n",
    "provide any additional information you might need.\n",
    "\n",
    "Thank you very much for considering my request. I look forward to your\n",
    "positive response.\n",
    "\n",
    "Warm regards,\n",
    "\n",
    "John\n",
    "\"\"\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DYr9PH7ZOeH3"
   },
   "source": [
    "## Few-Shot Text Generation\n",
    "\n",
    "A few-shot prompt involves presenting a model with a small set of examples to guide its behavior in generating responses or predictions. This technique is particularly useful in machine learning models like language or image generation systems, where the prompt acts as a mini-training session, enabling the model to understand and replicate a desired pattern or style with limited input. For instance, in a text generation model, a few-shot prompt might include a handful of sentences along with the desired outputs, setting the stage for the model to continue producing similar results. This approach helps in refining the model's outputs without the need for extensive training data, making it adaptable and efficient for specific tasks or creative nuances."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 297
    },
    "id": "uaYfdi_cOkS3",
    "outputId": "e3bfbcaa-8a5f-46f4-9cec-dc89110e1d6f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model response:\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Dear Admissions Committee,\n",
       "\n",
       "I am writing to highly recommend John Smith for admission to the Master of Science in Computer Science program at your esteemed institution. I had the pleasure of teaching John in my course, INFO 558: Applications of Deep Neural Networks, at Washington University in St. Louis. Throughout the semester, John consistently demonstrated exceptional dedication, intelligence, and a strong work ethic.\n",
       "\n",
       "Despite not having a background in computer science, John quickly adapted and excelled in the course, earning an outstanding A+ grade. His ability to grasp complex concepts, coupled with his proficiency in Python programming, was truly impressive. John's passion for learning and his commitment to excellence were evident in his performance and interactions within the class.\n",
       "\n",
       "Having graduated with a Master's degree in Quantitative Finance and excelling in his professional role as a Senior Financial Risk Analyst at RGA, John has proven himself to be a highly motivated and capable individual. His decision to pursue further education in Computer Science speaks volumes about his ambition and determination to broaden his skill set.\n",
       "\n",
       "I have no doubt that John's analytical skills, programming expertise, and strong academic background make him an ideal candidate for your program. His enthusiasm for learning, coupled with his ability to apply theoretical knowledge to practical scenarios, sets him apart as a promising candidate who"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "print(get_response(llm, \"\"\"\n",
    "Generate a positive letter of reccomendation for John Smith, a student of mine\n",
    "for INFO 558 at Washington University, my name is Jeff Heaton. He is applying\n",
    "for a Master of Science in Computer Science. Just give me the\n",
    "body text of the letter, no header or footer. Format in markdown.\n",
    "\n",
    "-----------------\n",
    "Examples of letters of reccomendation, written by me.\n",
    "\n",
    "To Whom It May Concern:\n",
    "John earned an A+ in my course Applications of Deep Neural Networks for the\n",
    "Fall 2019 semester at Washington University in St. Louis. During the semester\n",
    "I got a chance to know John through several discussions, both about my course\n",
    "and his research interests. While John did not come from a computer science\n",
    "background he has demonstrated himself as a capable Python programmer and was\n",
    "able to express his ideas in code.  My primary career is as a VP of data science\n",
    "at RGA, a Fortune 500 insurance company.  In this role I know the value of\n",
    "individuals, such as John, who have a background in finance, understand\n",
    "advanced machine learning topics, and can code sufficiently well to function\n",
    "as a data scientist.\n",
    "\n",
    "John was a student that in my class, T81-558: Application of Deep Neural Networks,\n",
    "for the Spring 2017 semester. This is a technical graduate class which includes\n",
    "students from the Masters of Science lnformation Systems, Management,\n",
    "computer science, and other disciplines. The course teaches students to\n",
    "implement deep neural networks using Google TensorFlow and Keras in the Python\n",
    "programming language. Students are expected to complete four computer programs\n",
    "and complete a final project. John did well in my course and earned an A+ (4.0).\n",
    "\n",
    "-----------\n",
    "The details of this student's request follows.\n",
    "\n",
    "I hope this message finds you well and that you are enjoying the holiday season!\n",
    "I am John Smith (ID: 1234), a proud alumnus of WashU, having graduated in\n",
    "January 2021 with a Master’s degree in Quantitative Finance.\n",
    "\n",
    "During the spring semester of 2020, I had the pleasure of attending your course,\n",
    "INFO 558: Applications of Deep Neural Networks, which was an elective for my\n",
    "master's program. I thoroughly enjoyed the content and was deeply engaged\n",
    "throughout, culminating in an A+ grade.\n",
    "\n",
    "Since graduating with a 3.99 GPA—top of my major—I have been working as a Senior\n",
    "Financial Risk Analyst at RGA. My role primarily involves developing automation\n",
    "tools and programming for strategic analysis and other analytical tasks. To\n",
    "further enhance my programming skills and knowledge, I am planning to pursue a\n",
    "part-time Master's in Computer Science while continuing to work at RGA.\n",
    "\n",
    "I am a great admirer of your work (I’m a regular viewer of your YouTube channel\n",
    "and have recommended it to my colleagues), and your insights would be invaluable\n",
    "in my application. I am applying to the following programs:\n",
    "\n",
    "Georgia Tech, Master of Science in Computer Science\n",
    "University of Pennsylvania, Master of Computer & Information Technology\n",
    "Could I possibly ask for your support with a recommendation letter for these\n",
    "applications? I have attached my resume for your reference and am happy to\n",
    "provide any additional information you might need.\n",
    "\n",
    "Thank you very much for considering my request. I look forward to your\n",
    "positive response.\n",
    "\n",
    "Warm regards,\n",
    "\n",
    "John\n",
    "\"\"\"))"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3.11 (genai)",
   "language": "python",
   "name": "pytorch"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}