c964-capstone-content-based-video-game-recommend(1).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**This Jupyter Notebook will serve as a proof of concept for a content based video game recommendation system. It was created in part with a guide found [here](http://www.datacamp.com/tutorial/recommender-systems-python). It will pull from data from IMDB on over 20k video games, including user ratings, as well as total number of users who rated each game. The recommender system will use Natural Language Processing on the 'plot' column of each game. It will then use cosine similarity to calculate the numerical difference between an example game and every other game in our dataset. Mathematically, cosine similarity is represented by the following equation:**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](https://res.cloudinary.com/dyd911kmh/image/upload/f_auto,q_auto:best/v1590782185/cos_aalkpq.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
    "execution": {
     "iopub.execute_input": "2023-02-01T15:41:13.059261Z",
     "iopub.status.busy": "2023-02-01T15:41:13.058796Z",
     "iopub.status.idle": "2023-02-01T15:41:15.070491Z",
     "shell.execute_reply": "2023-02-01T15:41:15.069542Z",
     "shell.execute_reply.started": "2023-02-01T15:41:13.059184Z"
    }
   },
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'numpy'",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[1], line 1\u001b[0m\n\u001b[1;32m----> 1\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mnumpy\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01mnp\u001b[39;00m \u001b[38;5;66;03m# linear algebra\u001b[39;00m\n\u001b[0;32m      2\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mpandas\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01mpd\u001b[39;00m \u001b[38;5;66;03m# data processing, CSV file I/O (e.g. pd.read_csv)\u001b[39;00m\n\u001b[0;32m      3\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01msklearn\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mfeature_extraction\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mtext\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m TfidfVectorizer \u001b[38;5;66;03m# NLP and TF-IDF Vectorization\u001b[39;00m\n",
      "\u001b[1;31mModuleNotFoundError\u001b[0m: No module named 'numpy'"
     ]
    }
   ],
   "source": [
    "import numpy as np # linear algebra\n",
    "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer # NLP and TF-IDF Vectorization\n",
    "from sklearn.metrics.pairwise import linear_kernel # Import linear_kernel\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import string\n",
    "import re\n",
    "import nltk\n",
    "\n",
    "# Input data files are available in the read-only \"../input/\" directory\n",
    "# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n",
    "\n",
    "import os\n",
    "for dirname, _, filenames in os.walk('/kaggle/input'):\n",
    "    for filename in filenames:\n",
    "        print(os.path.join(dirname, filename))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-02-01T15:41:15.073937Z",
     "iopub.status.busy": "2023-02-01T15:41:15.071885Z",
     "iopub.status.idle": "2023-02-01T15:41:15.393933Z",
     "shell.execute_reply": "2023-02-01T15:41:15.390906Z",
     "shell.execute_reply.started": "2023-02-01T15:41:15.073899Z"
    }
   },
   "outputs": [],
   "source": [
    "games_df = pd.read_csv('../input/imdb-video-games/imdb-videogames.csv')\n",
    "\n",
    "games_df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Now we need to remove the video game entries that have no info added into their 'plot' section. In this dataset, these entries have a string \"Add a Plot\" in this column. We also need to remove any entries that have dulplicate names, as this will confuse the cosine similarity function, and try to return more than one array, which will cause a ValueError. After dropping these, we will need to re-index our dataframe, so that any search will remain within range.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-02-01T15:41:15.405189Z",
     "iopub.status.busy": "2023-02-01T15:41:15.401742Z",
     "iopub.status.idle": "2023-02-01T15:41:15.507625Z",
     "shell.execute_reply": "2023-02-01T15:41:15.506529Z",
     "shell.execute_reply.started": "2023-02-01T15:41:15.405142Z"
    }
   },
   "outputs": [],
   "source": [
    "no_plot = games_df[games_df['plot'].__eq__('Add a Plot')].index\n",
    "\n",
    "games_df.drop(index=no_plot, inplace=True)\n",
    "games_df = games_df.drop_duplicates(subset='name', keep=\"first\", inplace=False)\n",
    "games_df = games_df.reset_index()\n",
    "\n",
    "games_df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**We have greatly shortened our list of games, from 20,803 to 11,096.** \n",
    "\n",
    "**Now we will perform the language processing on the 'plot' column, removing the \"Stop words\" from each plot, and use each plot to create a TF-IDF matrix:**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-02-01T15:41:15.517767Z",
     "iopub.status.busy": "2023-02-01T15:41:15.514184Z",
     "iopub.status.idle": "2023-02-01T15:41:16.395634Z",
     "shell.execute_reply": "2023-02-01T15:41:16.394523Z",
     "shell.execute_reply.started": "2023-02-01T15:41:15.517711Z"
    }
   },
   "outputs": [],
   "source": [
    "#Define a TF-IDF Vectorizer Object. We will also get rid of all english \"stop words\" such as 'the', 'a', or 'an'\n",
    "tfidf = TfidfVectorizer(stop_words='english')\n",
    "\n",
    "#Replace any NaN values with an empty string\n",
    "games_df['plot'] = games_df['plot'].fillna('')\n",
    "\n",
    "#Create the required TF-IDF matrix by fitting and transforming the data, taking the 'plot' column as an input.\n",
    "tfidf_matrix = tfidf.fit_transform(games_df['plot'])\n",
    "\n",
    "#Output the shape of tfidf_matrix\n",
    "tfidf_matrix.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Next, we can take a look at some of the words that included in our TD-IDF matrix:**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-02-01T15:41:16.400600Z",
     "iopub.status.busy": "2023-02-01T15:41:16.397033Z",
     "iopub.status.idle": "2023-02-01T15:41:16.431244Z",
     "shell.execute_reply": "2023-02-01T15:41:16.429701Z",
     "shell.execute_reply.started": "2023-02-01T15:41:16.400556Z"
    }
   },
   "outputs": [],
   "source": [
    "tfidf.get_feature_names_out()[2000:2100]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**We can also look at the shape of our cosine similarity matrix, and see a few of its entries, to make sure it looks correct:**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-02-01T15:41:16.435362Z",
     "iopub.status.busy": "2023-02-01T15:41:16.434265Z",
     "iopub.status.idle": "2023-02-01T15:41:18.766177Z",
     "shell.execute_reply": "2023-02-01T15:41:18.765272Z",
     "shell.execute_reply.started": "2023-02-01T15:41:16.435324Z"
    }
   },
   "outputs": [],
   "source": [
    "# Compute the cosine similarity matrix\n",
    "cosine_sim = linear_kernel(tfidf_matrix, tfidf_matrix)\n",
    "\n",
    "cosine_sim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-02-01T15:41:18.773769Z",
     "iopub.status.busy": "2023-02-01T15:41:18.771071Z",
     "iopub.status.idle": "2023-02-01T15:41:18.786327Z",
     "shell.execute_reply": "2023-02-01T15:41:18.785270Z",
     "shell.execute_reply.started": "2023-02-01T15:41:18.773700Z"
    }
   },
   "outputs": [],
   "source": [
    "cosine_sim.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**We can get a look, albeit not very detailed, of this large matrix by plotting it:**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-02-01T15:41:18.797055Z",
     "iopub.status.busy": "2023-02-01T15:41:18.794803Z",
     "iopub.status.idle": "2023-02-01T15:41:19.251395Z",
     "shell.execute_reply": "2023-02-01T15:41:19.250446Z",
     "shell.execute_reply.started": "2023-02-01T15:41:18.797002Z"
    }
   },
   "outputs": [],
   "source": [
    "sns.set_theme(style=\"white\")\n",
    "fig, ax = plt.subplots(figsize=(10,10))\n",
    "\n",
    "plot_cos = cosine_sim[900:1050, 900:1050]\n",
    "\n",
    "cax = ax.matshow(plot_cos, interpolation = 'nearest', cmap = sns.diverging_palette(230, 30, as_cmap=True), vmax = 0.4)\n",
    "\n",
    "plt.title('Video Game Similarity matrix')\n",
    "\n",
    "fig.colorbar(cax, ticks=[0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.75, 1])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**As can be seen, a large majority of games' plots don't have a great amount of similarity. The graph actually has been adjusted to show more similarity; the max value has been set to 0.4, rather than the usual 1.0. Anything higher than 0.4 will register as 1. Additionally, we have zoomed in on a slice of the 11,096 x 11,096 matrix, to show more granularity. It's important to note that if 1 were the maximum value, any other 1:1 matches were visible in the above matrix plot, it would indicate a duplicate entry, rather than two separate, similar data points, given the amount of dissimilarity in the dataset.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Now, in order to create a function which will return a list of similar games after inputing the title of one game, we need a reference of the indices of each game based on the title string. We will create that now:**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-02-01T15:41:19.253649Z",
     "iopub.status.busy": "2023-02-01T15:41:19.253252Z",
     "iopub.status.idle": "2023-02-01T15:41:19.259712Z",
     "shell.execute_reply": "2023-02-01T15:41:19.258633Z",
     "shell.execute_reply.started": "2023-02-01T15:41:19.253610Z"
    }
   },
   "outputs": [],
   "source": [
    "indices = pd.Series(games_df.index, index=games_df['name'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Now we need to create our recommendation function. It will have to do a couple of things:**\n",
    "1. Retrieve the index of the game inputted, given its title\n",
    "2. Get that game's cosine similarity to all other games on our list (11,929)\n",
    "3. Convert that cosine similarity list into a tuple list, where the first element is the position, and the second is its similarity score\n",
    "4. Sort the tuple list based on their similarity scores\n",
    "5. Get the top 10 results, ignoring the 1st result, as it will refer to the inputted game itself (it has a score of 1 in similarity, after all)\n",
    "6. Print the names that match the indices of those top 10 results\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-02-01T15:41:19.264076Z",
     "iopub.status.busy": "2023-02-01T15:41:19.263260Z",
     "iopub.status.idle": "2023-02-01T15:41:19.272666Z",
     "shell.execute_reply": "2023-02-01T15:41:19.271636Z",
     "shell.execute_reply.started": "2023-02-01T15:41:19.264039Z"
    }
   },
   "outputs": [],
   "source": [
    "def game_recommendation(name, cosine_sim=cosine_sim):\n",
    "    #Get the index of the inputted game\n",
    "    indx = indices[name]\n",
    "    \n",
    "    #Get the inputted game's list of similarity scores with all other games\n",
    "    similarity_scores = list(enumerate(cosine_sim[indx]))\n",
    "    \n",
    "    #Sort the games by similarity score\n",
    "    similarity_scores = sorted(similarity_scores, key=lambda x:x[1], reverse=True)\n",
    "    \n",
    "    #Get the top ten results\n",
    "    similarity_scores = similarity_scores[1:11]\n",
    "    \n",
    "    #Get the game indices\n",
    "    game_indices = [i[0] for i in similarity_scores]\n",
    "    \n",
    "    #Return top 10 most similar games\n",
    "    return games_df['name'].iloc[game_indices], similarity_scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-02-01T15:41:19.275356Z",
     "iopub.status.busy": "2023-02-01T15:41:19.274922Z",
     "iopub.status.idle": "2023-02-01T15:41:19.294775Z",
     "shell.execute_reply": "2023-02-01T15:41:19.293516Z",
     "shell.execute_reply.started": "2023-02-01T15:41:19.275321Z"
    }
   },
   "outputs": [],
   "source": [
    "game_recommendation(\"Samurai Warriors 3\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**The system seems to have correctly picked up on a 'Samurai' theme.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-02-01T15:41:19.296875Z",
     "iopub.status.busy": "2023-02-01T15:41:19.296488Z",
     "iopub.status.idle": "2023-02-01T15:41:19.311433Z",
     "shell.execute_reply": "2023-02-01T15:41:19.310195Z",
     "shell.execute_reply.started": "2023-02-01T15:41:19.296840Z"
    }
   },
   "outputs": [],
   "source": [
    "game_recommendation(\"Yakuza Kiwami\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-02-01T15:41:19.313475Z",
     "iopub.status.busy": "2023-02-01T15:41:19.313106Z",
     "iopub.status.idle": "2023-02-01T15:41:19.329966Z",
     "shell.execute_reply": "2023-02-01T15:41:19.328801Z",
     "shell.execute_reply.started": "2023-02-01T15:41:19.313425Z"
    }
   },
   "outputs": [],
   "source": [
    "game_recommendation(\"The Last of Us\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-02-01T15:41:19.331720Z",
     "iopub.status.busy": "2023-02-01T15:41:19.331425Z",
     "iopub.status.idle": "2023-02-01T15:54:17.102312Z",
     "shell.execute_reply": "2023-02-01T15:54:17.101104Z",
     "shell.execute_reply.started": "2023-02-01T15:41:19.331695Z"
    }
   },
   "outputs": [],
   "source": [
    "videogame = input(\"Please enter a game to search: \")\n",
    "\n",
    "game_recommendation(videogame)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}