t81_558_class_06_3_llm_memory.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R2Z2JH6cR3bM"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/jeffheaton/app_deep_learning/blob/main/t81_558_class_06_3_llm_memory.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "On4gWCWuR3bN"
      },
      "source": [
        "# T81-558: Applications of Deep Neural Networks\n",
        "**Module 6: ChatGPT and Large Language Models**\n",
        "* Instructor: [Jeff Heaton](https://sites.wustl.edu/jeffheaton/), McKelvey School of Engineering, [Washington University in St. Louis](https://engineering.wustl.edu/Programs/Pages/default.aspx)\n",
        "* For more information visit the [class website](https://sites.wustl.edu/jeffheaton/t81-558/)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5QVOz9xNR3bO"
      },
      "source": [
        "# Module 6 Material\n",
        "\n",
        "* Part 6.1: Introduction to Transformers [[Video]](https://www.youtube.com/watch?v=mn6r5PYJcu0&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_06_1_transformers.ipynb)\n",
        "* Part 6.2: Accessing the ChatGPT API [[Video]](https://www.youtube.com/watch?v=tcdscXl4o5w&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_06_2_chat_gpt.ipynb)\n",
        "* **Part 6.3: LLM Memory** [[Video]](https://www.youtube.com/watch?v=oGQ3TQx1Qs8&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_06_3_llm_memory.ipynb)\n",
        "* Part 6.4: Introduction to Embeddings [[Video]](https://www.youtube.com/watch?v=e6kcs9Uj_ps&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_06_4_embedding.ipynb)\n",
        "* Part 6.5: Prompt Engineering [[Video]](https://www.youtube.com/watch?v=miTpIDR7k6c&list=PLjy4p-07OYzulelvJ5KVaT2pDlxivl_BN) [[Notebook]](t81_558_class_06_5_prompt_engineering.ipynb)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uLI2X9tnR3bO"
      },
      "source": [
        "# Google CoLab Instructions\n",
        "\n",
        "The following code ensures that Google CoLab is running the correct version of TensorFlow.\n",
        "  Running the following code will map your GDrive to ```/content/drive```."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yjyDLCCuR3bO",
        "outputId": "5193c3bc-fa81-4450-fce0-06fd402e8672"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Note: using Google CoLab\n"
          ]
        }
      ],
      "source": [
        "try:\n",
        "    from google.colab import drive\n",
        "    COLAB = True\n",
        "    print(\"Note: using Google CoLab\")\n",
        "except:\n",
        "    print(\"Note: not using Google CoLab\")\n",
        "    COLAB = False"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IVpAB-sNR3bP"
      },
      "source": [
        "# Part 6.3: LLM Memory\n",
        "\n",
        "Human minds have both long-term and short-term memory. Long-term memory is what the human has learned throughout their lifetime. Short-term memory is what a human has only recently discovered in the last minute or so. For humans, learning is converting short-term memory into long-term memory that we will retain.\n",
        "\n",
        "This process works somewhat differently for a LLM. Long-term memory was the weight of the neural network when it was initially trained or finetuned. Short-term memory is additional information that we wish the LLM to retain from previous prompts. For example, if the first prompt is \"My name is Jeff\", the LLM will likely tell you hello and repeat your name. However, the LLM will not know the answer if the second prompt is \"What is my name.\" without adding a memory component.\n",
        "\n",
        "These memory objects, which LangChain provides, provide a sort of short-term memory. It is important to note that these objects are not affecting the long-term memory of the LLM, and once you discard the memory object, the LLM will forget. Additionally, the memory object can only hold so much information; newer information may replace older information once it is filled.\n",
        "\n",
        "One important point to remember is that LLM's only have their input prompt. To provide such memory, these objects are appending anything we wish the LLM to remember to the input prompt. This section will see two ways to augment the prompt with previous information: a buffer and a summary. The buffer prepends a script of the last conversation up to this point. The summary approach keeps a consistently updated summary paragraph of the conversation.\n",
        "\n",
        "## Install LangChain\n",
        "\n",
        "Just as we did previously, we mustinstall LangChain.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "X_pz8MuY84Qh",
        "outputId": "9320d45e-1fef-46b9-99fb-941c0577328b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting langchain\n",
            "  Downloading langchain-0.1.0-py3-none-any.whl (797 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m798.0/798.0 kB\u001b[0m \u001b[31m8.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting langchain_openai\n",
            "  Downloading langchain_openai-0.0.2.post1-py3-none-any.whl (28 kB)\n",
            "Requirement already satisfied: PyYAML>=5.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (6.0.1)\n",
            "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.0.24)\n",
            "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /usr/local/lib/python3.10/dist-packages (from langchain) (3.9.1)\n",
            "Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (4.0.3)\n",
            "Collecting dataclasses-json<0.7,>=0.5.7 (from langchain)\n",
            "  Downloading dataclasses_json-0.6.3-py3-none-any.whl (28 kB)\n",
            "Collecting jsonpatch<2.0,>=1.33 (from langchain)\n",
            "  Downloading jsonpatch-1.33-py2.py3-none-any.whl (12 kB)\n",
            "Collecting langchain-community<0.1,>=0.0.9 (from langchain)\n",
            "  Downloading langchain_community-0.0.12-py3-none-any.whl (1.6 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.6/1.6 MB\u001b[0m \u001b[31m69.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting langchain-core<0.2,>=0.1.7 (from langchain)\n",
            "  Downloading langchain_core-0.1.10-py3-none-any.whl (216 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m216.6/216.6 kB\u001b[0m \u001b[31m20.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting langsmith<0.1.0,>=0.0.77 (from langchain)\n",
            "  Downloading langsmith-0.0.80-py3-none-any.whl (48 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m48.3/48.3 kB\u001b[0m \u001b[31m4.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: numpy<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.23.5)\n",
            "Requirement already satisfied: pydantic<3,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain) (1.10.13)\n",
            "Requirement already satisfied: requests<3,>=2 in /usr/local/lib/python3.10/dist-packages (from langchain) (2.31.0)\n",
            "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /usr/local/lib/python3.10/dist-packages (from langchain) (8.2.3)\n",
            "Collecting openai<2.0.0,>=1.6.1 (from langchain_openai)\n",
            "  Downloading openai-1.7.2-py3-none-any.whl (212 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m212.1/212.1 kB\u001b[0m \u001b[31m21.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting tiktoken<0.6.0,>=0.5.2 (from langchain_openai)\n",
            "  Downloading tiktoken-0.5.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.0 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.0/2.0 MB\u001b[0m \u001b[31m65.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (23.2.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (6.0.4)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.9.4)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.4.1)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain) (1.3.1)\n",
            "Collecting marshmallow<4.0.0,>=3.18.0 (from dataclasses-json<0.7,>=0.5.7->langchain)\n",
            "  Downloading marshmallow-3.20.2-py3-none-any.whl (49 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.4/49.4 kB\u001b[0m \u001b[31m3.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting typing-inspect<1,>=0.4.0 (from dataclasses-json<0.7,>=0.5.7->langchain)\n",
            "  Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB)\n",
            "Collecting jsonpointer>=1.9 (from jsonpatch<2.0,>=1.33->langchain)\n",
            "  Downloading jsonpointer-2.4-py2.py3-none-any.whl (7.8 kB)\n",
            "Requirement already satisfied: anyio<5,>=3 in /usr/local/lib/python3.10/dist-packages (from langchain-core<0.2,>=0.1.7->langchain) (3.7.1)\n",
            "Requirement already satisfied: packaging<24.0,>=23.2 in /usr/local/lib/python3.10/dist-packages (from langchain-core<0.2,>=0.1.7->langchain) (23.2)\n",
            "Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai<2.0.0,>=1.6.1->langchain_openai) (1.7.0)\n",
            "Collecting httpx<1,>=0.23.0 (from openai<2.0.0,>=1.6.1->langchain_openai)\n",
            "  Downloading httpx-0.26.0-py3-none-any.whl (75 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.9/75.9 kB\u001b[0m \u001b[31m8.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.6.1->langchain_openai) (1.3.0)\n",
            "Requirement already satisfied: tqdm>4 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.6.1->langchain_openai) (4.66.1)\n",
            "Collecting typing-extensions<5,>=4.7 (from openai<2.0.0,>=1.6.1->langchain_openai)\n",
            "  Downloading typing_extensions-4.9.0-py3-none-any.whl (32 kB)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (3.6)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2.0.7)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2->langchain) (2023.11.17)\n",
            "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain) (3.0.3)\n",
            "Requirement already satisfied: regex>=2022.1.18 in /usr/local/lib/python3.10/dist-packages (from tiktoken<0.6.0,>=0.5.2->langchain_openai) (2023.6.3)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3->langchain-core<0.2,>=0.1.7->langchain) (1.2.0)\n",
            "Collecting httpcore==1.* (from httpx<1,>=0.23.0->openai<2.0.0,>=1.6.1->langchain_openai)\n",
            "  Downloading httpcore-1.0.2-py3-none-any.whl (76 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m76.9/76.9 kB\u001b[0m \u001b[31m8.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting h11<0.15,>=0.13 (from httpcore==1.*->httpx<1,>=0.23.0->openai<2.0.0,>=1.6.1->langchain_openai)\n",
            "  Downloading h11-0.14.0-py3-none-any.whl (58 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m6.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting mypy-extensions>=0.3.0 (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain)\n",
            "  Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)\n",
            "Installing collected packages: typing-extensions, mypy-extensions, marshmallow, jsonpointer, h11, typing-inspect, tiktoken, jsonpatch, httpcore, langsmith, httpx, dataclasses-json, openai, langchain-core, langchain_openai, langchain-community, langchain\n",
            "  Attempting uninstall: typing-extensions\n",
            "    Found existing installation: typing_extensions 4.5.0\n",
            "    Uninstalling typing_extensions-4.5.0:\n",
            "      Successfully uninstalled typing_extensions-4.5.0\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "llmx 0.0.15a0 requires cohere, which is not installed.\n",
            "tensorflow-probability 0.22.0 requires typing-extensions<4.6.0, but you have typing-extensions 4.9.0 which is incompatible.\u001b[0m\u001b[31m\n",
            "\u001b[0mSuccessfully installed dataclasses-json-0.6.3 h11-0.14.0 httpcore-1.0.2 httpx-0.26.0 jsonpatch-1.33 jsonpointer-2.4 langchain-0.1.0 langchain-community-0.0.12 langchain-core-0.1.10 langchain_openai-0.0.2.post1 langsmith-0.0.80 marshmallow-3.20.2 mypy-extensions-1.0.0 openai-1.7.2 tiktoken-0.5.2 typing-extensions-4.9.0 typing-inspect-0.9.0\n"
          ]
        }
      ],
      "source": [
        "!pip install langchain langchain_openai"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RE0FlZ_eR3bP"
      },
      "source": [
        "Smiliarly, we must also specify our API key and model touse. Refer to Module 6.2 for more information."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "VucO3HSMoJkz"
      },
      "outputs": [],
      "source": [
        "from langchain_openai import OpenAI, ChatOpenAI\n",
        "\n",
        "# Your OpenAI API key\n",
        "# If you are in my class at WUSTL, get this key from the Assignment 6 description in Canvas.\n",
        "\n",
        "OPENAI_KEY = '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'\n",
        "\n",
        "# This is the model you will generally use for this class\n",
        "LLM_MODEL = 'gpt-3.5-turbo-instruct'\n",
        "\n",
        "# Chat Model: LLM_MODEL = 'gpt-3.5-turbo-1106'"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Next, we create the LLM model, as we did before. We specify a higher temperature to encourage creativity in the conversation."
      ],
      "metadata": {
        "id": "W0McrknBXgxG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Initialize the OpenAI LLM (Language Learning Model) with your API key\n",
        "llm = OpenAI(openai_api_key=OPENAI_KEY, model=LLM_MODEL, temperature=0.7)"
      ],
      "metadata": {
        "id": "D0CRDXlUW0V5"
      },
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Conversation Buffer Window Memory"
      ],
      "metadata": {
        "id": "-6kkALepWLi9"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aufxkOj1R3bP"
      },
      "source": [
        "The LangChain library includes a conversation object named **ConversationChain**; this object facilitates an ongoing conversation with an LLM. For any conversation object, you must also specify a memory. For this first example, we will use the **ConversationBufferWindowMemory** object. This object keeps a transcript of the most recent conversation to reference. This memory allows the conversation object to remember what you have asked or told it and its responses to you."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "w3_lzwcqermJ"
      },
      "outputs": [],
      "source": [
        "from langchain.chains import ConversationChain\n",
        "from langchain.memory import ConversationBufferWindowMemory\n",
        "\n",
        "memory = ConversationBufferWindowMemory()\n",
        "conversation = ConversationChain(\n",
        "    llm=llm,\n",
        "    memory = memory,\n",
        "    verbose=False\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kxg_SXT2R3bP"
      },
      "source": [
        "We can now have a conversation with the LLM."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "Egh3beCVRpW5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "outputId": "c8de5cd3-b5e9-4ebb-a756-e4b64ecf6dae"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\" Hello Jeff, it's nice to meet you. My name is AI and I am an artificial intelligence designed to assist and communicate with humans. How can I help you today?\""
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 6
        }
      ],
      "source": [
        "conversation.predict(input=\"Hi, my name is Jeff\")"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "conversation.predict(input=\"What is my name?\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "gWi1o0tuYOR3",
        "outputId": "f610ce7e-dd4e-4701-dee1-fe21fcf0a268"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "' Your name is Jeff. I know this because you introduced yourself to me at the beginning of our conversation. Is there something else you would like to know, Jeff?'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can have a look at what the memory now contains."
      ],
      "metadata": {
        "id": "W3u8kB-ccx2k"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "conversation.memory.load_memory_variables({})"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LC8dFPqoaa7Z",
        "outputId": "3f84cbb8-de1d-42d9-f735-9a6aab420f87"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'history': \"Human: Hi, my name is Jeff\\nAI:  Hello Jeff, it's nice to meet you. My name is AI and I am an artificial intelligence designed to assist and communicate with humans. How can I help you today?\\nHuman: What is my name?\\nAI:  Your name is Jeff. I know this because you introduced yourself to me at the beginning of our conversation. Is there something else you would like to know, Jeff?\"}"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Custom Conversation Bots\n",
        "\n",
        "You can define the prompt template for a conversationbot. This technique allows you to create a bot with a name and perform a specialized task. In this case, we created a bot named \"WashU Assistant\" that we designed to help students."
      ],
      "metadata": {
        "id": "9EJfbbLSYTnO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Now we can override it and set it to \"AI Assistant\"\n",
        "from langchain.prompts.prompt import PromptTemplate\n",
        "\n",
        "template = \"\"\"The following is a friendly conversation between a human and an\n",
        "AI to assist Washington University Students. The AI should stick to topics\n",
        "about Washington University. If the AI does not know the answer to a question,\n",
        "it should suggest the student speak to their advisor.\n",
        "\n",
        "Current conversation:\n",
        "{history}\n",
        "Human: {input}\n",
        "WashU Assistant:\"\"\"\n",
        "PROMPT = PromptTemplate(input_variables=[\"history\", \"input\"], template=template)\n",
        "conversation = ConversationChain(\n",
        "    prompt=PROMPT,\n",
        "    llm=llm,\n",
        "    verbose=False,\n",
        "    memory=ConversationBufferWindowMemory(ai_prefix=\"WashU Assistant\"),\n",
        ")"
      ],
      "metadata": {
        "id": "3BZAlBkBYajX"
      },
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can now have a conversation with the WUSTL assistant bot."
      ],
      "metadata": {
        "id": "S7_y05_Pc8Be"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "conversation.predict(input=\"Where is the bookstore?\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "JBH_Zp0NYrW1",
        "outputId": "8b02cb8f-e96f-4657-fc98-a5bec5768a41"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "' The WashU Bookstore is located on the Danforth Campus in the Danforth University Center. It is on the south side of the building, near the clock tower. Is there anything else I can assist you with?'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "conversation.predict(input=\"What is a nice quiet area to study?\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "4LNHTxdMYu1h",
        "outputId": "36803406-2543-44b2-c047-256a168880dd"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\" There are many great places to study on campus, depending on your preferences. Some popular options include the Olin Library, the Village Green, and the Women's Building. Additionally, many academic buildings have designated study areas. Is there a specific location or atmosphere you are looking for?\""
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "conversation.predict(input=\"Which of these is closest to the bookstore?\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "my3IvjLhY1eh",
        "outputId": "4385be9f-3214-4d30-eedf-36afe1300149"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "\" The Olin Library and the Women's Building are both located nearby the bookstore. The Village Green is a bit farther, but still within walking distance. Is there anything else I can assist you with?\""
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "conversation.predict(input=\"What is the meaning of life.\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "-tx7V8d8Zlng",
        "outputId": "b55eaec5-5995-4f79-8069-0682841c32dd"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "' I am an AI programmed to assist with information about Washington University. I do not have the capability to answer philosophical questions. Is there anything else I can help you with related to WashU? Otherwise, I suggest speaking to your advisor for guidance on this topic.'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can have a look at what the memory now contains."
      ],
      "metadata": {
        "id": "z3kzuQqRc2Lu"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "conversation.memory.load_memory_variables({})"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Bwq-I9lYaZuP",
        "outputId": "0602d0b4-c527-4352-f40b-57314c253e07"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'history': \"Human: Where is the bookstore?\\nWashU Assistant:  The WashU Bookstore is located on the Danforth Campus in the Danforth University Center. It is on the south side of the building, near the clock tower. Is there anything else I can assist you with?\\nHuman: What is a nice quiet area to study?\\nWashU Assistant:  There are many great places to study on campus, depending on your preferences. Some popular options include the Olin Library, the Village Green, and the Women's Building. Additionally, many academic buildings have designated study areas. Is there a specific location or atmosphere you are looking for?\\nHuman: Which of these is closest to the bookstore?\\nWashU Assistant:  The Olin Library and the Women's Building are both located nearby the bookstore. The Village Green is a bit farther, but still within walking distance. Is there anything else I can assist you with?\\nHuman: What is the meaning of life.\\nWashU Assistant:  I am an AI programmed to assist with information about Washington University. I do not have the capability to answer philosophical questions. Is there anything else I can help you with related to WashU? Otherwise, I suggest speaking to your advisor for guidance on this topic.\"}"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Conversation Summary Memory\n",
        "\n",
        "Now, let's look at using a slightly more complex type of memory, the ConversationSummaryMemory object. This type of memory creates a summary of the conversation over time. This memory can be helpful for condensing information from the conversation over time. Conversation summary memory summarizes the conversation and stores the current summary in memory. You can use this memory to inject the conversation summary so far into a prompt/chain. This memory is most useful for more extended conversations, where keeping the past message history in the prompt verbatim would take up too many tokens."
      ],
      "metadata": {
        "id": "T6Yjeri4Zv9x"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from langchain.memory import ConversationSummaryMemory\n",
        "\n",
        "memory = ConversationSummaryMemory(llm=llm)\n",
        "conversation = ConversationChain(\n",
        "    llm=llm,\n",
        "    memory = memory,\n",
        "    verbose=False\n",
        ")"
      ],
      "metadata": {
        "id": "6wJsN0k_aMVS"
      },
      "execution_count": 15,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "conversation.predict(input=\"I am a computer scientist, what do you do for a living?\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 71
        },
        "id": "4DEl67mQaPvp",
        "outputId": "3efb44c8-79cf-439f-d9d9-ef64cd0bc9df"
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "' I am an artificial intelligence program designed to assist with various tasks and provide information to users. I am constantly learning and improving my abilities through machine learning algorithms and access to vast amounts of data. Right now, my primary function is to communicate with users like you and help with any questions or tasks you may have. Is there anything specific you would like to know or discuss?'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can have a look at what the memory now contains."
      ],
      "metadata": {
        "id": "HG5LSIvgc4yJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "conversation.memory.load_memory_variables({})"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PTFOPjHjaSBL",
        "outputId": "72263729-e827-4330-93b7-d21085034c56"
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'history': '\\nThe human asks what the AI does for a living. The AI explains that it is an artificial intelligence program designed to assist with various tasks and provide information. Its primary function is to communicate with users and help with any questions or tasks.'}"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuClass": "premium",
      "machine_shape": "hm",
      "provenance": []
    },
    "gpuClass": "premium",
    "kernelspec": {
      "display_name": "Python 3.11 (torch)",
      "language": "python",
      "name": "pytorch"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.18"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}