t81_558_class_06_2_chat_gpt.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YjrBimlZK_ih"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/jeffheaton/app_deep_learning/blob/main/t81_558_class_06_2_chat_gpt.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a92SeNfkK_ij"
      },
      "source": [
        "# T81-558: Applications of Deep Neural Networks\n",
        "**Module 6: ChatGPT and Large Language Models**\n",
        "* Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/), McKelvey School of Engineering, [Washington University in St. Louis](https://engineering.wustl.edu/Programs/Pages/default.aspx)\n",
        "* For more information visit the [class website](https://sites.wustl.edu/jeffheaton/t81-558/)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zT1sGKjFK_ij"
      },
      "source": [
        "# Module 6 Material\n",
        "\n",
        "* Part 6.1: Introduction to Transformers [[Video]](https://www.youtube.com/watch?v=mn6r5PYJcu0&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_06_1_transformers.ipynb)\n",
        "* **Part 6.2: Accessing the ChatGPT API** [[Video]](https://www.youtube.com/watch?v=tcdscXl4o5w&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_06_2_chat_gpt.ipynb)\n",
        "* Part 6.3: LLM Memory [[Video]](https://www.youtube.com/watch?v=oGQ3TQx1Qs8&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_06_3_llm_memory.ipynb)\n",
        "* Part 6.4: Introduction to Embeddings [[Video]](https://www.youtube.com/watch?v=e6kcs9Uj_ps&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_06_4_embedding.ipynb)\n",
        "* Part 6.5: Prompt Engineering [[Video]](https://www.youtube.com/watch?v=miTpIDR7k6c&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_06_5_prompt_engineering.ipynb)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KM2XCFm-K_ij"
      },
      "source": [
        "# Google CoLab Instructions\n",
        "\n",
        "The following code ensures that Google CoLab is running the correct version of TensorFlow.\n",
        "  Running the following code will map your GDrive to ```/content/drive```."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XqifKgVsK_ij",
        "outputId": "067dac2d-f858-45b9-d009-324455313872"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Note: using Google CoLab\n"
          ]
        }
      ],
      "source": [
        "try:\n",
        "    from google.colab import drive\n",
        "    COLAB = True\n",
        "    print(\"Note: using Google CoLab\")\n",
        "except:\n",
        "    print(\"Note: not using Google CoLab\")\n",
        "    COLAB = False"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JyFjANq1v-2z"
      },
      "source": [
        "# Part 6.2: LangChain, ChatGPT and NLP\n",
        "\n",
        "Large Language Models (LLMs) such as GPT have brought AI into mainstream use. LLMs allow regular users to interact with AI using natural language. Most of these language models require extreme processing capabilities and hardware. Because of this, application programming interfaces (APIs) accessed through the Internet are becoming common entry points for these models. One of the most compelling features of services like ChatGPT is their availability as an API. But before we dive into the depths of coding and integration, let's understand what an API is and its significance in the AI domain.\n",
        "\n",
        "API stands for Application Programming Interface. Think of it as a bridge or a messenger that allows two different software applications to communicate. In the context of AI and machine learning, APIs often allow developers to access a particular model or service without having to house the model on their local machine. This technique can be beneficial when the model in question, like ChatGPT, is large and resource-intensive.\n",
        "\n",
        "In the realm of AI, APIs have several distinct advantages:\n",
        "\n",
        "* Scalability: Since the actual model runs on external servers, developers don't need to worry about scaling infrastructure.\n",
        "* Maintenance: You get to use the latest and greatest version of the model without constantly updating your local copy.\n",
        "* Cost-Effective: Leveraging external computational resources can be more cost-effective than maintaining high-end infrastructure locally, especially for sporadic or one-off tasks.\n",
        "* Ease of Use: Instead of diving into the nitty-gritty details of model implementation and optimization, developers can directly utilize its capabilities with a few lines of code.\n",
        "\n",
        "In this section, we won't be running the neural network computations locally. Instead, our PyTorch code will communicate with the OpenAI API to access and harness the abilities of ChatGPT. The actual execution of the neural network code happens on OpenAI servers, bringing forth a unique synergy of PyTorch's flexibility and ChatGPT's conversational mastery.\n",
        "\n",
        "In this section, we will make use of the OpenAI ChatGPT API. Further information on this API can be found here:\n",
        "\n",
        "* [OpenAI API Login/Registration](https://platform.openai.com/apps)\n",
        "* [OpenAI API Reference](https://platform.openai.com/docs/introduction/overview)\n",
        "* [OpenAI Python API Reference](https://platform.openai.com/docs/api-reference/introduction?lang=python)\n",
        "* [OpenAI Python Library](https://github.com/openai/openai-python)\n",
        "* [OpenAI Cookbook for Python](https://github.com/openai/openai-cookbook/)\n",
        "* [LangChain](https://www.langchain.com/)\n",
        "\n",
        "## Installing LangChain to use the OpenAI Python Library\n",
        "\n",
        "As we delve deeper into the intricacies of deep learning, it's crucial to understand that the tools and platforms we use are as versatile as the concepts themselves. When it comes to accessing ChatGPT, a state-of-the-art conversational AI model developed by OpenAI, there are two predominant pathways:\n",
        "\n",
        "Direct API Access using Python's HTTP Capabilities: Python, with its rich library ecosystem, provides utilities like requests to directly communicate with APIs over HTTP. This method involves crafting the necessary API calls, handling responses, and error checking, giving the developer a granular control over the process.\n",
        "\n",
        "Using the Official OpenAI Python Library: OpenAI offers an official Python library, aptly named openai, that simplifies the process of integrating with ChatGPT and other OpenAI services. This library abstracts many of the intricacies and boilerplate steps of direct API access, offering a streamlined and user-friendly approach to interacting with the model.\n",
        "\n",
        "Each approach has its advantages. Direct API access provides a more hands-on, granular approach, allowing developers to intimately understand the intricacies of each API call. On the other hand, using the openai library can accelerate development, reduce potential errors, and allow for a more straightforward integration, especially for those new to API interactions.\n",
        "\n",
        "We will make use of the OpenAI API through a library called LangChain. LangChain is a framework designed to simplify the creation of applications using LLMs. As a language model integration framework, LangChain's use-cases largely overlap with those of language models in general, including document analysis and summarization, chatbots, and code analysis. LangChain allows you to quickly change between different underlying LLMs with minimal code changes.\n",
        "\n",
        "The following command installs the **LangChain** library and needed OpenAI LLM connectors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FJdVstEjlIJx",
        "outputId": "a5e3fcf7-5daf-4406-f35a-c8c4a98e5ef2"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Requirement already satisfied: langchain in /usr/local/lib/python3.10/dist-packages (0.1.0)\n",
            "Requirement already satisfied: langchain_openai in /usr/local/lib/python3.10/dist-packages (0.0.2.post1)\n",
            "Requirement already satisfied: PyYAML>=5.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (6.0.1)\n",
            "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.0.24)\n",
            "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (3.9.1)\n",
            "Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (4.0.3)\n",
            "Requirement already satisfied: dataclasses-json<0.7,>=0.5.7 in /usr/local/lib/python3.10/dist-packages (from langchain) (0.6.3)\n",
            "Requirement already satisfied: jsonpatch<2.0,>=1.33 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.33)\n",
            "Requirement already satisfied: langchain-community<0.1,>=0.0.9 in /usr/local/lib/python3.10/dist-packages (from langchain) (0.0.12)\n",
            "Requirement already satisfied: langchain-core<0.2,>=0.1.7 in /usr/local/lib/python3.10/dist-packages (from langchain) (0.1.10)\n",
            "Requirement already satisfied: langsmith<0.1.0,>=0.0.77 in /usr/local/lib/python3.10/dist-packages (from langchain) (0.0.80)\n",
            "Requirement already satisfied: numpy<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.23.5)\n",
            "Requirement already satisfied: pydantic<3,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.10.13)\n",
            "Requirement already satisfied: requests<3,>=2 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.31.0)\n",
            "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (8.2.3)\n",
            "Requirement already satisfied: openai<2.0.0,>=1.6.1 in /usr/local/lib/python3.10/dist-packages (from langchain_openai) (1.7.2)\n",
            "Requirement already satisfied: tiktoken<0.6.0,>=0.5.2 in /usr/local/lib/python3.10/dist-packages (from langchain_openai) (0.5.2)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (23.2.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (6.0.4)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.9.4)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.4.1)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.3.1)\n",
            "Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /usr/local/lib/python3.10/dist-packages (from dataclasses-json<0.7,>=0.5.7->langchain) (3.20.2)\n",
            "Requirement already satisfied: typing-inspect<1,>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from dataclasses-json<0.7,>=0.5.7->langchain) (0.9.0)\n",
            "Requirement already satisfied: jsonpointer>=1.9 in /usr/local/lib/python3.10/dist-packages (from jsonpatch<2.0,>=1.33->langchain) (2.4)\n",
            "Requirement already satisfied: anyio<5,>=3 in /usr/local/lib/python3.10/dist-packages (from langchain-core<0.2,>=0.1.7->langchain) (3.7.1)\n",
            "Requirement already satisfied: packaging<24.0,>=23.2 in /usr/local/lib/python3.10/dist-packages (from langchain-core<0.2,>=0.1.7->langchain) (23.2)\n",
            "Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai<2.0.0,>=1.6.1->langchain_openai) (1.7.0)\n",
            "Requirement already satisfied: httpx<1,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.6.1->langchain_openai) (0.26.0)\n",
            "Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.6.1->langchain_openai) (1.3.0)\n",
            "Requirement already satisfied: tqdm>4 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.6.1->langchain_openai) (4.66.1)\n",
            "Requirement already satisfied: typing-extensions<5,>=4.7 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.6.1->langchain_openai) (4.9.0)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.6)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2.0.7)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2023.11.17)\n",
            "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain) (3.0.3)\n",
            "Requirement already satisfied: regex>=2022.1.18 in /usr/local/lib/python3.10/dist-packages (from tiktoken<0.6.0,>=0.5.2->langchain_openai) (2023.6.3)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3->langchain-core<0.2,>=0.1.7->langchain) (1.2.0)\n",
            "Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.23.0->openai<2.0.0,>=1.6.1->langchain_openai) (1.0.2)\n",
            "Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/lib/python3.10/dist-packages (from httpcore==1.*->httpx<1,>=0.23.0->openai<2.0.0,>=1.6.1->langchain_openai) (0.14.0)\n",
            "Requirement already satisfied: mypy-extensions>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain) (1.0.0)\n"
          ]
        }
      ],
      "source": [
        "!pip install langchain langchain_openai"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EBqaIJSJ3BJD"
      },
      "source": [
        "## Obtaining an OpenAI API Key\n",
        "\n",
        "In order to delve into the practical exercises and code demonstrations within this section, students will need to obtain an OpenAI API key. This key grants access to OpenAI's services, including the ChatGPT functionality we'll be exploring. It's important to note that there is a nominal cost associated with the usage of this key, depending on the volume and intensity of requests made to OpenAI's servers. However, securing and using this key is entirely optional for this course. Engaging with this segment is not mandatory, nor will it be a part of any course assignments. The decision to obtain and use an OpenAI key rests solely with the student, allowing for a personalized learning journey tailored to individual interests and resources.\n",
        "\n",
        "To obtain an OpenAI API key, access this [site](https://platform.openai.com/apps)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "NSOgT81AlAUs"
      },
      "outputs": [],
      "source": [
        "# Your OpenAI API key\n",
        "# If you are in my class at WUSTL, get this key from the Assignment 6 description in Canvas.\n",
        "\n",
        "OPENAI_KEY = '[insert your token]'\n",
        "\n",
        "# This is the model you will generally use for this class\n",
        "LLM_MODEL = 'gpt-3.5-turbo-1106'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UyC-6nxu4u_I"
      },
      "source": [
        "We begin with a very basic query to LangChain, we ask LangChain what are the 5 largest cities in the USA."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "BXYrysJ54tA5",
        "outputId": "b500ca57-e103-4972-8527-5cc322fc7de3"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "'1. New York City, New York\\n2. Los Angeles, California\\n3. Chicago, Illinois\\n4. Houston, Texas\\n5. Phoenix, Arizona'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "from langchain_openai import OpenAI, ChatOpenAI\n",
        "\n",
        "LLM_MODEL = 'gpt-3.5-turbo-1106'\n",
        "\n",
        "# Initialize the OpenAI LLM (Language Learning Model) with your API key\n",
        "llm = ChatOpenAI(openai_api_key=OPENAI_KEY, model=LLM_MODEL, temperature=0)\n",
        "\n",
        "# Define the question\n",
        "question = \"What are the five largest cities in the USA by population?\"\n",
        "\n",
        "# Use Langchain to call the OpenAI API\n",
        "# The method and parameters might differ based on the Langchain version\n",
        "response = llm.invoke(question)\n",
        "\n",
        "# Print the response\n",
        "display(response.content)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SIWbUIv3UIAH"
      },
      "source": [
        "As you can see, the response from LangChain is in regular English, complete with formatting. While the formatting may make it easier to read, we often have to parse the results given to us by LLMs. Later, we will see that LangChain can help with this as well. You will also notice that we specified a value of 0 for **temperature**; this instructs the LLM to be less creative with its responses and more consistent. Because we are working primarily with data extraction in this section, a low temperature will give us more consistent results.\n",
        "\n",
        "## Working with Prompts\n",
        "\n",
        "We will often need to construct complex prompts that incorporate multiple variables into the final prompt. We can use normal Python string handling to achieve this. Lets use ChatGPT to translate from French to English, using normal Python F-Strings to build the prompt."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "text = \"\"\"Laissez les bons temps rouler\"\"\"\n",
        "style = \"American English\"\n",
        "\n",
        "prompt = f\"\"\"Translate the text \\\n",
        "that is delimited by triple backticks \\\n",
        "into a style that is {style}. \\\n",
        "text: ```{text}```\n",
        "\"\"\"\n",
        "\n",
        "response = llm.invoke(prompt)\n",
        "\n",
        "# Print the response\n",
        "print(response.content)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VvMF3vj5L1nl",
        "outputId": "82875276-2b70-4aca-d564-966de8ce884f"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\"Let the good times roll\"\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "We can use LangChain to help us build dynamic prompts."
      ],
      "metadata": {
        "id": "XKg0RKzZMe8i"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain.prompts import ChatPromptTemplate\n",
        "\n",
        "template_text = \"\"\"Translate the text \\\n",
        "that is delimited by triple backticks \\\n",
        "into a style that is {style}. \\\n",
        "text: ```{text}```\n",
        "\"\"\"\n",
        "\n",
        "prompt_template = ChatPromptTemplate.from_template(template_text)\n"
      ],
      "metadata": {
        "id": "ZS3orjWUMZ1-"
      },
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can now fill in the blanks for this prompt and observe the prompt created, which is a text string."
      ],
      "metadata": {
        "id": "uXkAH-OcMivw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "prompt = prompt_template.format_messages(\n",
        "                    style=\"American English\",\n",
        "                    text=\"千里之行，始于足下。\")\n",
        "\n",
        "print(type(prompt))\n",
        "print(type(prompt[0]))\n",
        "\n",
        "print(prompt[0])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "n4BnJKsCMmm6",
        "outputId": "288c1de4-b732-48bb-9b57-23af3c52963e"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<class 'list'>\n",
            "<class 'langchain_core.messages.human.HumanMessage'>\n",
            "content='Translate the text that is delimited by triple backticks into a style that is American English. text: ```千里之行，始于足下。```\\n'\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This newly constructed prompt can now perform the intended task of translation."
      ],
      "metadata": {
        "id": "3rn7ck6PMtkW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Call the LLM to translate to the style of the customer message\n",
        "response = llm.invoke(prompt)\n",
        "print(response)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9_P10PpgMu3-",
        "outputId": "78b77901-9a8f-4443-ad7c-429dd46d1938"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "content='\"千里之行，始于足下。\" translates to \"A journey of a thousand miles begins with a single step.\"'\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Processing Output\n",
        "\n",
        "We will now consider a more complex text extraction example and see how LangChain can help us extract multiple values returned by ChatGPT. Here, we will see how three fields can be extracted from a product description."
      ],
      "metadata": {
        "id": "6q1K8DUCN7ND"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain.output_parsers import ResponseSchema\n",
        "from langchain.output_parsers import StructuredOutputParser\n",
        "\n",
        "material_schema = ResponseSchema(name=\"material\",\n",
        "                             description=\"What is the material that this \\\n",
        "                             item is made of? If unknown, make an estimate.\")\n",
        "description_schema = ResponseSchema(name=\"shape\",\n",
        "                                      description=\"What is the shape of this \\\n",
        "                                      item? If unknown, return null.\")\n",
        "who_schema = ResponseSchema(name=\"who\",\n",
        "                                    description=\"Who is the likely user of \\\n",
        "                                    this item? If unkown, make an estimate.\")\n",
        "\n",
        "response_schemas = [material_schema,\n",
        "                    description_schema,\n",
        "                    who_schema]"
      ],
      "metadata": {
        "id": "vmLhCnnGOBwC"
      },
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "As you can see from the above code, we are extracting three fields from the product description: material, shape, and who. We describe LangChain for each to instruct LangChain of what each field is, which helps to find it in the product description. Next we construct a **StructuredOutputParser** to actually obtain this data."
      ],
      "metadata": {
        "id": "CsTBu7b2O4qh"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "output_parser = StructuredOutputParser.from_response_schemas(response_schemas)\n",
        "format_instructions = output_parser.get_format_instructions()\n",
        "\n",
        "prompt = ChatPromptTemplate.from_template(template=\"\"\"\n",
        "For the product text, extract this information in valid JSON, with commas. \\\n",
        "{format_instructions}\n",
        "Text:\n",
        "{text}\"\"\")\n",
        "\n",
        "product_description = \"\"\"\\\n",
        "Ross Brachiosaurus has a gentle spirit that any dog will quickly love. \\\n",
        "His long body lends itself for tossing or tugging alone or with friends, \\\n",
        "and his size makes him an excellent cuddle companion.\n",
        "\"\"\""
      ],
      "metadata": {
        "id": "ub0yyFZcPY_8"
      },
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can now execute these three parts: the prompt builder, invoke the LLM itself, and then finally parse the output from the LLM. This gives us a dictionary containing these three fields."
      ],
      "metadata": {
        "id": "lsp281YtPtg_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "result1 = prompt.invoke({'text':product_description,'format_instructions':format_instructions})\n",
        "result2 = llm.invoke(result1)\n",
        "result3 = output_parser.invoke(result2)\n",
        "print(result3)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hshH8WB-Prp8",
        "outputId": "aeab4865-5ad2-438f-fcb7-2b36e1520811"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'material': 'plush', 'shape': 'dinosaur', 'who': 'dog'}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Often you will have multiple components in langchain that you must call in a \"chain\", to do this you can construct a chain."
      ],
      "metadata": {
        "id": "q96aBRDrQBNi"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "chain = prompt | llm | output_parser\n",
        "chain.invoke({'text':product_description,'format_instructions':format_instructions})"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jtvvijCcQJc3",
        "outputId": "4feea36d-d418-4299-eea7-921324762d7c"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'material': 'plush', 'shape': 'dinosaur', 'who': 'dog'}"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u4oI42e4PqKl"
      },
      "source": [
        "## Application to Text Extraction\n",
        "\n",
        "Language model-based learning, commonly abbreviated as LLM, has numerous applications in the world of business. One prevalent utilization of LLM is in the domain of text extraction. Text extraction focuses on the retrieval of specific pieces of information from a larger body of text. For instance, in scenarios where a dataset contains varied information about individuals—ranging from birthdays to job details—one can employ LLM to zero in on just the birthdays, efficiently filtering out extraneous data. The power of LLM lies in its ability to discern context and extract relevant details based on the user's requirements, as showcased in the code that adeptly identifies and extracts birthday details while disregarding other particulars."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "c2ds0AQ9PvmI",
        "outputId": "db41444b-7fe8-4b12-8d9f-700d8a005fa3"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "14-JUN-1995\n"
          ]
        }
      ],
      "source": [
        "from langchain.prompts import ChatPromptTemplate\n",
        "\n",
        "PROMPT = \"\"\"\n",
        "You are to extract any birthdays from the provided text, return the \" \\\n",
        "date in the form 10-FEB-1990, or NONE if no birthday.\n",
        "\n",
        "text: {text}\"\"\"\n",
        "\n",
        "prompt_template = ChatPromptTemplate.from_template(PROMPT)\n",
        "\n",
        "INPUT = \"John was born on June 14, 1995, he was married on May 8, 2015.\"\n",
        "\n",
        "chain = prompt_template | llm\n",
        "\n",
        "result = chain.invoke({'text':INPUT})\n",
        "print(result.content)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8vXeREOYUgqB"
      },
      "source": [
        "The same code can process a series of text strings. The dates in these strings are in a variety of different formats. The LLM is able to parse and find the needed birthdays and ignore other information. Notice that sometimes the date is not formatted as requested or multiple dates return. Soon we will learn about prompt engineering, wich solves some of these problems."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "I2y7WfO_J8Mz",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "44eec16c-e8dd-493a-8ea9-48d5238364b2"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "5-MAR-1990\n",
            "20-JUL-1985\n",
            "22-OCT-1992\n",
            "02/17/1987\n",
            "03-DEC-1994\n",
            "12-MAY-1988\n",
            "25-APR-1991\n",
            "08-JAN-1984\n",
            "16-SEP-1993\n",
            "11/30/1989\n"
          ]
        }
      ],
      "source": [
        "\n",
        "LIST = [\n",
        "  \"Anna started her first job on 15th January 2012. She was born on March 5, 1990.\",\n",
        "  \"On 04/14/2007, Michael graduated from college. He was born on 20th July 1985.\",\n",
        "  \"Born on 22nd October 1992, Sophia got married on 11.11.2016.\",\n",
        "  \"Graduating from high school on June 5, 2005, was a big moment for Lucas. His birth date is 02/17/1987.\",\n",
        "  \"Isabelle began her professional journey on 01/09/2016, having been born on December 3, 1994.\",\n",
        "  \"Liam was born on May 12, 1988. He celebrated his wedding on 07-15-2014.\",\n",
        "  \"Eva celebrated her college graduation on 20-05-2013. Her birthday falls on April 25, 1991.\",\n",
        "  \"In 2006, specifically on 03.03.2006, Daniel started his first job. He came into this world on January 8, 1984.\",\n",
        "  \"On 05.25.2011, Emily donned her graduation gown. Her birthdate is September 16, 1993.\",\n",
        "  \"Henry marked his birthday on 11/30/1989. He tied the knot on October 10, 2017.\"\n",
        "]\n",
        "\n",
        "for item in LIST:\n",
        "  response = chain.invoke({'text':item})\n",
        "\n",
        "  print(response.content)\n",
        "\n",
        "\n"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3.11 (torch)",
      "language": "python",
      "name": "pytorch"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.18"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}