notebooks/knowledge-graph.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install graphviz -q"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/kylekelley/Library/Caches/pypoetry/virtualenvs/chatlab-0EWIBOuo-py3.11/lib/python3.11/site-packages/pydantic/main.py:1455: RuntimeWarning: fields may not start with an underscore, ignoring \"__required__\"\n",
      "  warnings.warn(f'fields may not start with an underscore, ignoring \"{f_name}\"', RuntimeWarning)\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "`nteract papermill` is a tool that enables you to parameterize and execute Jupyter Notebooks. This is particularly useful for automating and scaling the execution of notebooks for data processing, analysis, and reporting. `papermill` allows users to parameterize notebooks via the command line or programmatically in Python, execute them, and even save the output to a new notebook. This makes it an excellent tool for automating repetitive tasks and workflows in data science and analysis.\n",
       "\n",
       "### Key Concepts of Papermill:\n",
       "\n",
       "1. **Parameterization**: `papermill` allows you to pass parameters into your Jupyter Notebooks. This enables you to reuse and automate notebooks for different inputs without manually changing the code.\n",
       "\n",
       "2. **Execution**: It can execute notebooks programmatically or from the command line, making it a powerful tool for automating workflows.\n",
       "\n",
       "3. **Logging and Reporting**: `papermill` records the execution and enables you to output the results to a new notebook, facilitating easy sharing and reporting of the results.\n",
       "\n",
       "### Contributors to Papermill:\n",
       "\n",
       "The development and maintenance of `papermill` is a collaborative effort. Contributors include a mix of individual developers, data scientists, and contributions from organizations. The core contributors are often affiliated with `nteract`, an open-source organization dedicated to creating tools for interactive computing.\n",
       "\n",
       "To give you a visual representation of `nteract papermill` and its contributors, I will create a knowledge graph that illustrates the key components of `papermill` and highlights some of the contributors and their roles. Let's proceed with creating that graph."
      ],
      "text/plain": [
       "AssistantMessageView(display_id='7815eac9-6139-4b0e-924e-e1099d1aa41d', content=\"`nteract papermill` is a tool that enables you to parameterize and execute Jupyter Notebooks. This is particularly useful for automating and scaling the execution of notebooks for data processing, analysis, and reporting. `papermill` allows users to parameterize notebooks via the command line or programmatically in Python, execute them, and even save the output to a new notebook. This makes it an excellent tool for automating repetitive tasks and workflows in data science and analysis.\\n\\n### Key Concepts of Papermill:\\n\\n1. **Parameterization**: `papermill` allows you to pass parameters into your Jupyter Notebooks. This enables you to reuse and automate notebooks for different inputs without manually changing the code.\\n\\n2. **Execution**: It can execute notebooks programmatically or from the command line, making it a powerful tool for automating workflows.\\n\\n3. **Logging and Reporting**: `papermill` records the execution and enables you to output the results to a new notebook, facilitating easy sharing and reporting of the results.\\n\\n### Contributors to Papermill:\\n\\nThe development and maintenance of `papermill` is a collaborative effort. Contributors include a mix of individual developers, data scientists, and contributions from organizations. The core contributors are often affiliated with `nteract`, an open-source organization dedicated to creating tools for interactive computing.\\n\\nTo give you a visual representation of `nteract papermill` and its contributors, I will create a knowledge graph that illustrates the key components of `papermill` and highlights some of the contributors and their roles. Let's proceed with creating that graph.\", finished=True, has_displayed=True)"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 9.0.0 (20230911.1827)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"857pt\" height=\"310pt\"\n",
       " viewBox=\"0.00 0.00 857.15 309.50\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 305.5)\">\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-305.5 853.15,-305.5 853.15,4 -4,4\"/>\n",
       "<!-- 1 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>1</title>\n",
       "<ellipse fill=\"none\" stroke=\"lightblue\" cx=\"292.87\" cy=\"-283.5\" rx=\"74.38\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"292.87\" y=\"-278.45\" font-family=\"Times,serif\" font-size=\"14.00\">nteract papermill</text>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>2</title>\n",
       "<ellipse fill=\"none\" stroke=\"lightgreen\" cx=\"73.87\" cy=\"-195\" rx=\"73.87\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"73.87\" y=\"-189.95\" font-family=\"Times,serif\" font-size=\"14.00\">Parameterization</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;2 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>1&#45;&gt;2</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M254.52,-267.79C238.4,-261.57 219.46,-254.23 202.37,-247.5 175.61,-236.96 145.79,-225.02 121.61,-215.29\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"122.92,-212.05 112.34,-211.56 120.31,-218.54 122.92,-212.05\"/>\n",
       "<text text-anchor=\"middle\" x=\"223.12\" y=\"-234.2\" font-family=\"Times,serif\" font-size=\"14.00\">enables</text>\n",
       "</g>\n",
       "<!-- 3 -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>3</title>\n",
       "<ellipse fill=\"none\" stroke=\"lightgreen\" cx=\"213.87\" cy=\"-195\" rx=\"48.28\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"213.87\" y=\"-189.95\" font-family=\"Times,serif\" font-size=\"14.00\">Execution</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;3 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>1&#45;&gt;3</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M277.26,-265.41C265.64,-252.69 249.61,-235.13 236.57,-220.86\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"239.26,-218.61 229.93,-213.58 234.09,-223.33 239.26,-218.61\"/>\n",
       "<text text-anchor=\"middle\" x=\"285.62\" y=\"-234.2\" font-family=\"Times,serif\" font-size=\"14.00\">performs</text>\n",
       "</g>\n",
       "<!-- 4 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>4</title>\n",
       "<ellipse fill=\"none\" stroke=\"lightgreen\" cx=\"371.87\" cy=\"-195\" rx=\"91.78\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"371.87\" y=\"-189.95\" font-family=\"Times,serif\" font-size=\"14.00\">Logging &amp; Reporting</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;4 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>1&#45;&gt;4</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M308.48,-265.41C319.95,-252.85 335.72,-235.58 348.67,-221.41\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"351.12,-223.92 355.27,-214.17 345.95,-219.2 351.12,-223.92\"/>\n",
       "<text text-anchor=\"middle\" x=\"366.12\" y=\"-234.2\" font-family=\"Times,serif\" font-size=\"14.00\">facilitates</text>\n",
       "</g>\n",
       "<!-- 5 -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>5</title>\n",
       "<ellipse fill=\"none\" stroke=\"orange\" cx=\"538.87\" cy=\"-195\" rx=\"57.49\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"538.87\" y=\"-189.95\" font-family=\"Times,serif\" font-size=\"14.00\">Contributors</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;5 -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>1&#45;&gt;5</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M335.28,-268.28C353.89,-261.99 375.99,-254.46 395.87,-247.5 427.97,-236.27 464,-223.29 491.86,-213.17\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"492.87,-216.53 501.07,-209.82 490.48,-209.95 492.87,-216.53\"/>\n",
       "<text text-anchor=\"middle\" x=\"475.12\" y=\"-234.2\" font-family=\"Times,serif\" font-size=\"14.00\">supported by</text>\n",
       "</g>\n",
       "<!-- 6 -->\n",
       "<g id=\"node6\" class=\"node\">\n",
       "<title>6</title>\n",
       "<ellipse fill=\"none\" stroke=\"yellow\" cx=\"360.87\" cy=\"-106.5\" rx=\"93.32\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"360.87\" y=\"-101.45\" font-family=\"Times,serif\" font-size=\"14.00\">Individual Developers</text>\n",
       "</g>\n",
       "<!-- 5&#45;&gt;6 -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>5&#45;&gt;6</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M508.68,-179.33C480.04,-165.41 436.69,-144.34 404.33,-128.62\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"406.13,-125.6 395.61,-124.38 403.07,-131.9 406.13,-125.6\"/>\n",
       "<text text-anchor=\"middle\" x=\"485.75\" y=\"-145.7\" font-family=\"Times,serif\" font-size=\"14.00\">include</text>\n",
       "</g>\n",
       "<!-- 7 -->\n",
       "<g id=\"node7\" class=\"node\">\n",
       "<title>7</title>\n",
       "<ellipse fill=\"none\" stroke=\"yellow\" cx=\"538.87\" cy=\"-106.5\" rx=\"66.19\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"538.87\" y=\"-101.45\" font-family=\"Times,serif\" font-size=\"14.00\">Data Scientists</text>\n",
       "</g>\n",
       "<!-- 5&#45;&gt;7 -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>5&#45;&gt;7</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M538.87,-176.91C538.87,-165.26 538.87,-149.55 538.87,-136.02\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"542.37,-136.36 538.87,-126.36 535.37,-136.36 542.37,-136.36\"/>\n",
       "<text text-anchor=\"middle\" x=\"558.75\" y=\"-145.7\" font-family=\"Times,serif\" font-size=\"14.00\">include</text>\n",
       "</g>\n",
       "<!-- 8 -->\n",
       "<g id=\"node8\" class=\"node\">\n",
       "<title>8</title>\n",
       "<ellipse fill=\"none\" stroke=\"yellow\" cx=\"735.87\" cy=\"-106.5\" rx=\"113.28\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"735.87\" y=\"-101.45\" font-family=\"Times,serif\" font-size=\"14.00\">Open Source Organizations</text>\n",
       "</g>\n",
       "<!-- 5&#45;&gt;8 -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>5&#45;&gt;8</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M571.4,-179.72C603.23,-165.74 652.07,-144.3 688.26,-128.4\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"689.45,-131.71 697.2,-124.48 686.64,-125.3 689.45,-131.71\"/>\n",
       "<text text-anchor=\"middle\" x=\"674.75\" y=\"-145.7\" font-family=\"Times,serif\" font-size=\"14.00\">include</text>\n",
       "</g>\n",
       "<!-- 9 -->\n",
       "<g id=\"node9\" class=\"node\">\n",
       "<title>9</title>\n",
       "<ellipse fill=\"none\" stroke=\"yellow\" cx=\"735.87\" cy=\"-18\" rx=\"36\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"735.87\" y=\"-12.95\" font-family=\"Times,serif\" font-size=\"14.00\">nteract</text>\n",
       "</g>\n",
       "<!-- 8&#45;&gt;9 -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>8&#45;&gt;9</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M735.87,-88.41C735.87,-76.76 735.87,-61.05 735.87,-47.52\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"739.37,-47.86 735.87,-37.86 732.37,-47.86 739.37,-47.86\"/>\n",
       "<text text-anchor=\"middle\" x=\"759.12\" y=\"-57.2\" font-family=\"Times,serif\" font-size=\"14.00\">example</text>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x11ed421d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "The knowledge graph illustrates the key components of `nteract papermill` and its contributors. Here's a breakdown of what the graph represents:\n",
       "\n",
       "- **nteract papermill**: The central node, representing the tool itself.\n",
       "- **Parameterization, Execution, Logging & Reporting**: These nodes represent the key functionalities of `papermill`. They are linked to the main node, showing that `papermill` enables parameterization, performs execution, and facilitates logging and reporting.\n",
       "- **Contributors**: This node is linked to the main `nteract papermill` node, indicating that the tool is supported by a variety of contributors.\n",
       "  - **Individual Developers, Data Scientists, Open Source Organizations**: These nodes highlight the types of contributors involved in the development and maintenance of `papermill`. The \"nteract\" node under \"Open Source Organizations\" specifically points out one of the major contributors.\n",
       "\n",
       "This visualization serves as a simplified overview, focusing on the functionalities of `papermill` and the broad categories of its contributors. The collaborative effort behind `papermill` involves many individuals and organizations, all contributing towards its development, documentation, and maintenance."
      ],
      "text/plain": [
       "AssistantMessageView(display_id='286d40c4-3183-4983-934b-9689dd64524f', content='The knowledge graph illustrates the key components of `nteract papermill` and its contributors. Here\\'s a breakdown of what the graph represents:\\n\\n- **nteract papermill**: The central node, representing the tool itself.\\n- **Parameterization, Execution, Logging & Reporting**: These nodes represent the key functionalities of `papermill`. They are linked to the main node, showing that `papermill` enables parameterization, performs execution, and facilitates logging and reporting.\\n- **Contributors**: This node is linked to the main `nteract papermill` node, indicating that the tool is supported by a variety of contributors.\\n  - **Individual Developers, Data Scientists, Open Source Organizations**: These nodes highlight the types of contributors involved in the development and maintenance of `papermill`. The \"nteract\" node under \"Open Source Organizations\" specifically points out one of the major contributors.\\n\\nThis visualization serves as a simplified overview, focusing on the functionalities of `papermill` and the broad categories of its contributors. The collaborative effort behind `papermill` involves many individuals and organizations, all contributing towards its development, documentation, and maintenance.', finished=False, has_displayed=True)"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import warnings\n",
    "from graphviz import Digraph\n",
    "from pydantic import BaseModel, Field\n",
    "from typing import List\n",
    "from chatlab import Chat\n",
    "from chatlab.decorators import incremental_display\n",
    "\n",
    "class Node(BaseModel):\n",
    "    id: int\n",
    "    label: str\n",
    "    color: str\n",
    "\n",
    "class Edge(BaseModel):\n",
    "    source: int\n",
    "    target: int\n",
    "    label: str\n",
    "    color: str = Field(\"black\", description=\"A valid Graphviz color name or hex code.\")\n",
    "\n",
    "class KnowledgeGraph(BaseModel):\n",
    "    nodes: List[Node] = Field(..., default_factory=list)\n",
    "    edges: List[Edge] = Field(..., default_factory=list)\n",
    "\n",
    "\n",
    "def visualize_knowledge_graph(kg: KnowledgeGraph, comment: str = \"Knowledge Graph\"):\n",
    "    \"\"\"Visualizes a knowledge graph using graphviz.\"\"\"\n",
    "    if kg is None:\n",
    "        return\n",
    "\n",
    "    dot = Digraph(comment=comment)\n",
    "\n",
    "    for node in kg.nodes:\n",
    "        dot.node(str(node.id), node.label, color=node.color)\n",
    "\n",
    "    for edge in kg.edges:\n",
    "        dot.edge(str(edge.source), str(edge.target), label=edge.label, color=edge.color)\n",
    "\n",
    "    return dot\n",
    "\n",
    "\n",
    "@incremental_display(visualize_knowledge_graph)\n",
    "def store_knowledge_graph(kg: KnowledgeGraph, comment: str):\n",
    "    \"\"\"Creates a graphviz diagram for the user and stores it in their database.\"\"\"\n",
    "    return \"Stored graph, visual shown\"\n",
    "\n",
    "chat = Chat(\n",
    "    model=\"gpt-4-turbo-preview\",\n",
    "    chat_functions=[store_knowledge_graph],\n",
    ")\n",
    "\n",
    "await chat(\"Visually teach me about the nteract papermill and its contributors\", temperature=0.7)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "The `nteract` project, which `papermill` is a part of, is an open-source initiative without a traditional single-founder structure. It was started and is maintained by a community of contributors. The project is collaborative, with contributions from many individuals and support from various organizations. \n",
       "\n",
       "However, some key figures have played significant roles in its development and in the development of `papermill` specifically:\n",
       "\n",
       "- **Matthias Bussonnier** is one of the notable contributors to the Jupyter ecosystem and has been involved with `nteract` projects. While not a founder in the traditional sense, his contributions to the broader Jupyter and interactive computing community are significant.\n",
       "- **Kyle Kelley** is another prominent figure in the `nteract` community, contributing to various aspects of the project and its tools.\n",
       "- **Safia Abdalla** is also a key contributor to `nteract` and has been involved in the development and advocacy of open-source tools for interactive computing.\n",
       "\n",
       "It's important to note that `nteract` and `papermill` are community-driven projects. This means that they rely on the contributions of many individuals, including developers, data scientists, and enthusiasts in the open-source community. The success and development of these projects are attributed to the collective effort of the community rather than to individual founders."
      ],
      "text/plain": [
       "AssistantMessageView(display_id='d05ed621-d0e8-4047-8c2d-08c1deab8222', content=\"The `nteract` project, which `papermill` is a part of, is an open-source initiative without a traditional single-founder structure. It was started and is maintained by a community of contributors. The project is collaborative, with contributions from many individuals and support from various organizations. \\n\\nHowever, some key figures have played significant roles in its development and in the development of `papermill` specifically:\\n\\n- **Matthias Bussonnier** is one of the notable contributors to the Jupyter ecosystem and has been involved with `nteract` projects. While not a founder in the traditional sense, his contributions to the broader Jupyter and interactive computing community are significant.\\n- **Kyle Kelley** is another prominent figure in the `nteract` community, contributing to various aspects of the project and its tools.\\n- **Safia Abdalla** is also a key contributor to `nteract` and has been involved in the development and advocacy of open-source tools for interactive computing.\\n\\nIt's important to note that `nteract` and `papermill` are community-driven projects. This means that they rely on the contributions of many individuals, including developers, data scientists, and enthusiasts in the open-source community. The success and development of these projects are attributed to the collective effort of the community rather than to individual founders.\", finished=False, has_displayed=True)"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "await chat(\"Who are the founders?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "display(\"above here\", update=True, display_id=\"230945832\")\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with warnings.catch_warnings():\n",
    "    warnings.simplefilter(\"ignore\")\n",
    "    await chat(\"Ok now make two knowledge graphs. One about cats and one about dogs.\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "chatlab-3kMKfU-i-py3.11",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}