create_public_vectordb.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create Public Vector Database"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "from langchain.embeddings import CohereEmbeddings\n",
    "import json\n",
    "import random\n",
    "from langchain.vectorstores import Chroma\n",
    "\n",
    "# Set the game name \n",
    "game_name = 'Cyberpunk_2077'\n",
    "\n",
    "# Load the file\n",
    "with open(f'{game_name}/public_info.txt', errors='ignore') as f:\n",
    "    public_info = f.read()\n",
    "\n",
    "# Randomly select an API key\n",
    "selected_key = json.load(open('apikeys.json', 'r'))['api_keys'][random.randint(\n",
    "    0, len(json.load(open('apikeys.json', 'r'))['api_keys'])-1)]\n",
    "\n",
    "# Initialise the embedding model\n",
    "embeddings = CohereEmbeddings(cohere_api_key=selected_key)\n",
    "\n",
    "# Create Text Splitter, Chunk size is number of characters in each item of the list \n",
    "# While chunk overlap is how many characters from the previous item should be repeated in the next item \n",
    "text_splitter = RecursiveCharacterTextSplitter(\n",
    "    chunk_size = 1500,\n",
    "    chunk_overlap  = 500,\n",
    "    length_function = len,\n",
    "    separators = ['\\n\\n\\n', '\\n\\n', '\\n', '.', ',', ' ']\n",
    ")\n",
    "\n",
    "# Split the text \n",
    "public_info_list = text_splitter.split_text(public_info)\n",
    "\n",
    "# Path must be inside the game folder with the name public_vectordb\n",
    "persist_directory = f'{game_name}/public_vectordb'\n",
    "\n",
    "# Create the Vector Database\n",
    "vectordb = Chroma.from_texts(public_info_list, embeddings, persist_directory=persist_directory)\n",
    "vectordb.persist()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test the Vector Database"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Johnny Silverhand, born Robert John Linder,[3] was a famous influential rockerboy and the lead singer of the band Samurai before its breakup in 2008.[4] A military veteran who defined the rockerboy movement to what it is today, he was the most prominent figure that fought against the corrupted NUSA government and megacorporations, often being described as a terrorist. Despite being charismatic and quite the charmer Johnny is also known to be irrational, impulsive, and a manipulator.[5] Silverhand got his nickname from a cybernetic implant left arm he had installed after losing it in the Second Central American War. One of the early (and most probable) victims of cyberpsychosis, he then became very temperamental and \"on the edge\". He was described as driven by \"dedication and ambitions, but at the end of the day he doesn't care much for the people around him as long as they are used to accomplish his goals\" in one interview.[6] Eventually, Johnny was killed by Adam Smasher during the Night City Holocaust on August 20, 2023.\n",
      "\n",
      "Rosalind Myers is the current president of the New United States of America, the former CEO of Militech, and a former Marine.\n",
      "Adam Smasher is a full borg solo and rival of Morgan Blackhand. He is employed by Arasaka and by 2077, has risen to the position of head of security and the personal bodyguard of Yorinobu Arasaka. Smasher is a towering cyborg, with little humanity left to be seen - not that he ever had much. After being reduced to mush by an RPG blast, Arasaka offered him a choice - either pull the plug or become a full body conversion cyborg. With little to no options and a lack of care for his human side, he agreed and became more machine than man. Adam has no empathy for others - including his fellow employees - but Arasaka kept him alive, so he lives to repay their act by killing any enemies of the corporation Arasaka puts in front of him.\n",
      "\n",
      "Delamain is an artificial intelligence that runs the taxi company known as the Delamain Corporation. Based in Night City, Delamain's core is found at its corp headquarters in Vista del Rey, Heywood.\n",
      "\n",
      "Dexter DeShawn, commonly known as Dex, was a well-known fixer in Night City. In 2077, he recently returned to the city after spending a few years off the grid. He has never personally worked as a mercenary before becoming a fixer, but had no need to thanks to his charisma and keen eye for potential.\n",
      "Johnny Silverhand and Kerry Eurodyne led the post-2008 resurgence of the genre with their band Samurai, and have made it onto the Top Ten. Since then, Rocker music has covered political, social, and economic activism. With the rise of cyberware, songs about technohumanism have found their way into the Rocker scene.[1]\n",
      "\n",
      "Sports\n",
      "The '93-'94 Super Bowl saw all pretense of \"drug free\" professional sports pushed aside, with six players dying in the game. During the collapse, professional sports died for over a decade. Violent blood sports became the norm amidst the chaos. Around roughly 2002, the military government formalized augmented sports. The great stadiums of the pre-Collapse largely disappeared save for the Arrowhead complex in Kansas City, or the Giants stadium in New York; and the \"golden age\" of professional sports was considered over; with the last Olympic Games held in 1992.\n"
     ]
    }
   ],
   "source": [
    "# search for a topic and see if you get relevant information, k is for number of similar items to return\n",
    "topic = \"Johnny Silverhand\"\n",
    "docs = vectordb.similarity_search(topic, k = 3)\n",
    "similar_story_string = \"\\n\".join(doc.page_content for doc in docs)\n",
    "print(similar_story_string)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### To add text to existing Public vector database \n",
    "#### (Optional, only if you want to add something that was not previously there in the above text file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# from langchain.embeddings import CohereEmbeddings\n",
    "# import json\n",
    "# import random\n",
    "# from langchain.vectorstores import Chroma\n",
    "\n",
    "#selected_key = json.load(open('apikeys.json', 'r'))['api_keys'][random.randint(\n",
    "#    0, len(json.load(open('apikeys.json', 'r'))['api_keys'])-1)]\n",
    "#embeddings = CohereEmbeddings(cohere_api_key=selected_key)\n",
    "#persist_directory = 'Cyberpunk_2077/public_vectordb'\n",
    "#vectordb = Chroma(persist_directory=persist_directory, embedding_function=embeddings)\n",
    "#vectordb.add_texts([\"pineapples are yummy\", \"tomatoes are evil\"])\n",
    "#vectordb.persist()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### To delete vectordb entirely, delete the public_vectordb folder inside the game folder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import shutil\n",
    "# path to public_vectordb inside your game folder\n",
    "folder_path = 'Some_game/public_vectordb'\n",
    "shutil.rmtree(folder_path)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}