Optional Challenge 2. KNN - Customer Churn.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Classification Problem - Predicting Customer Churn 🛫"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Quick reminder on `Jupyter Notebook` and `Python` basics 🚴‍♀️"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notebook consists of two main parts.\n",
    "\n",
    "1. Text instructions like this one - these are made using a text formatting language called [Markdown](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet)\n",
    "\n",
    "2. Code cells like the one below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "1 + 1 * 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. To run a code cell, click into it with your mouse and press the `► Run` button in the navbar at the top of the notebook. \n",
    "2. You can also use the shortcut `Shift + Enter` to run a cell!\n",
    "3. A cell that has been run will get a `In [number]` next to it\n",
    "4. An output (returned value) of a cell will be displayed below with a `Out[number]` next to it\n",
    "5. If you want to add another code cell - look for the `➕` button in the navbar."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In **Python** we have **built-in data types** to help us work with different kinds of data:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Strings** (`str` in Python) for **literal text, column or file names**. Made by putting quotes (`\"\"`) around the text."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"hello!\"\n",
    "\"ML like a pro\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Integers** (`int` in Python) for **whole numbers**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "42\n",
    "-10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Floats** (`float` in Python) for **numbers with decimal points**. The decimal delimeter is always `.`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "3.14"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have **variables** to help store data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "name = \"Alan Turing\"\n",
    "age = 42\n",
    "new_hire = [[0, 30, 3, 7.1, 12]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "...and **re-use** it later!:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"Hi, my name is \" + name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also **update** variables:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# getting one year older :(\n",
    "age = age + 1\n",
    "age"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we have **methods** to perform actions on data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "name.upper()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "number_of_n = name.count('n') # creating a new variable as a result of the method call\n",
    "number_of_n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A new model - K-Nearest Neighbors 🤜 🤛 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[K-Nearest Neighbors (or KNN)](https://scikit-learn.org/0.24/modules/generated/sklearn.neighbors.KNeighborsClassifier.html) is a distance-based model that can be used both for regression (predict a number) or classification (predict a category)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**And the best part?** The steps are exactly the same for KNN model as for the Linear Regression we just did! 🙌"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Let's start by importing the necessary Python libraries again"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. Let's get the data from `CSV` into a nice `DataFrame`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "churn = pd.read_csv('clean_data/churn.csv')\n",
    "churn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. Always a good idea to do some **visual exploration** first 📊\n",
    "\n",
    "In this case, we care about whether the customer has `Exited` or **churned**. So we can try to plot the different features (inputs) keeping `Exited` as the color differentiator (hue).\n",
    "\n",
    "### (Optional) Visualize the data! 📊\n",
    "\n",
    "Experiment with the `scatterplot` below by changing which columns will be `x` axis and `y` axis. Make sure to type the column names correctly! 👀"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.scatterplot(data=churn, x='PICK A COLUMN', y='PICK ANOTHER COLUMN', hue='Exited')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4. Time for Machine Learning! Import the model from Scikit-learn:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5. Initialize the model and **pick a number of neighbors** `n_neighbors` to match against:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = KNeighborsClassifier(n_neighbors=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "6. Create our `inputs` and `output`. This time let's call them `x` and `y`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = churn.drop([\"Exited\"], axis=\"columns\") # dropping the output column to create the inputs (features)\n",
    "y = churn[\"Exited\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Feel free to check your `x` and `y` below 👇"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Your turn! 🚀\n",
    "\n",
    "The **training** and **scoring** part is exactly the same as with the first model - try to solve it!\n",
    "\n",
    "**💡Tip:** try adjusting the number of neighbors - `n_neighbors` - above until you get the best result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>Reveal Solution 🙈</summary>\n",
    "\n",
    "<p> \n",
    "<pre>\n",
    "model.fit(x,y) # this trains the model\n",
    "model.score(x,y) # this scores the model\n",
    "</pre>\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This time score metric is **accuracy** - how many predictions did the model get right in our dataset.\n",
    "\n",
    "How is your score? Pretty great for 10 minutes of work, right?! 🤩 Well..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## We've been cheating! 😳"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We've been scoring the model on the same data as it is trained on - too easy! That is called **data leakage**.\n",
    "\n",
    "Scikit-learn library saves us again - let's import and use the [`train_test_split` method](https://scikit-learn.org/0.24/modules/generated/sklearn.model_selection.train_test_split.html) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below line might look a little crazy. Don't worry.\n",
    "\n",
    "The `train_test_split` method gives us all the datasets we need - inputs and outputs for both training and testing. So we simultaneously create four new variables to store that."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xtrain, xtest, ytrain, ytest = train_test_split(x, y, test_size=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Your turn - this time no cheating! 🚀\n",
    "\n",
    "Now that we have training and testing datasets, you should **initialize** a new model, **train** and **score** it again with the right datasets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>Reveal Solution 🙈</summary>\n",
    "\n",
    "<p> \n",
    "<pre>\n",
    "model = KNeighborsClassifier(n_neighbors=3)\n",
    "\n",
    "model.fit(xtrain, ytrain) # using only the training data\n",
    "model.score(xtest, ytest) # using unseen testing data to score\n",
    "</pre>\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----------"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1. Your turn - prediction! 🚀\n",
    "\n",
    "The prediction also works exactly as the previous model, so we let you do that. We created an example customer which you can tune."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "customer = pd.DataFrame({\n",
    "    'CreditScore': [608],\n",
    "    'Gender': [1],\n",
    "    'Age': [31],\n",
    "    'Tenure': [20],\n",
    "    'Balance': [81207.86],\n",
    "    'NumOfProducts': [2],\n",
    "    'HasCrCard': [1],\n",
    "    'IsActiveMember': [0],\n",
    "    'EstimatedSalary': [11142.58]\n",
    "})\n",
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>Reveal Solution 🙈</summary>\n",
    "\n",
    "<p> \n",
    "<pre>\n",
    "model.predict(customer)\n",
    "</pre>\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2. Your turn! 🚀Adding probability\n",
    "\n",
    "To understand the model's decision a bit more, we can use the `predict_proba()` method. It works exactly the same as the `predict()` method you used above. Can you write the code?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>Reveal Solution 🙈</summary>\n",
    "\n",
    "<p> \n",
    "<pre>\n",
    "model.predict_proba(customer)\n",
    "</pre>\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Your turn! 🚀\n",
    "\n",
    "Try to tune the features inside `customer` variable to get different results. What insights can you see? 🔍"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Explainability? Already harder with KNN model 😓"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to use a [feature permutation](https://scikit-learn.org/0.24/modules/generated/sklearn.inspection.permutation_importance.html) method provided by `Scikit-learn`.\n",
    "\n",
    "This method runs the model scoring many times, by changing (*permutating*) one feature at a time, to see which one causes the most change in the target."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from sklearn.inspection import permutation_importance\n",
    "\n",
    "permutation_score = permutation_importance(model, xtrain, ytrain, n_repeats=10)\n",
    "\n",
    "np.vstack((x.columns, permutation_score.importances_mean)).T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Congratulations on your first ML models, heroes! 🦸‍♀️🦸‍♂️"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 🕵️‍♀️ Going further? (Optional Challenge) KNN only on categorical columns!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You probably noticed that the **categorical columns** - such as `HasCrCard` or `NumOfProducts` - have virtually no influence on the above model.\n",
    "\n",
    "It might just be that they are strongly outweighed by the other columns, like `Balance` or `EstimatedSalary`. It would be a good idea **create a new model** *only* for the categorical columns."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Your turn - build your own model! 🚀\n",
    "\n",
    "Create a new KNN model **from scratch** and repeat the steps all the way to prediction but this time **only on the categorical columns**. We listed them below:\n",
    "\n",
    "`['NumOfProducts', 'HasCrCard', 'IsActiveMember', 'Gender']`\n",
    "\n",
    "#### **💡Tip:**  Don't be shy to re-use from before and adapt it to the current challenge!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Create our `x` (inputs) and `y` (output) datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# your code goes here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>Reveal Solution 🙈</summary>\n",
    "\n",
    "<p> \n",
    "<pre>\n",
    "x = churn.drop(['CreditScore', 'Age', 'Tenure', 'Balance', 'EstimatedSalary'], axis='columns')\n",
    "  # or instead of dropping columns, we can also just select the ones we need:\n",
    "x = churn[['NumOfProducts', 'HasCrCard', 'IsActiveMember', 'Gender']]\n",
    "\n",
    "y = churn.Exited\n",
    "</pre>\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. Split our data into training and testing data with `train_test_split`   (*P.S. no need to `import` a second time*)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>Reveal Solution 🙈</summary>\n",
    "\n",
    "<p> \n",
    "<pre>\n",
    "xtrain, xtest, ytrain, ytest = train_test_split(x, y, test_size=0.2)\n",
    "</pre>\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. Initialize a KNN model. Pick the number of neighbors `n_neighbors` you like 🙂"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>Reveal Solution 🙈</summary>\n",
    "\n",
    "<p> \n",
    "<pre>\n",
    "classifier = KNeighborsClassifier(n_neighbors=3)\n",
    "</pre>\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4. Train (or fit) the model with the training `x` and `y` datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>Reveal Solution 🙈</summary>\n",
    "\n",
    "<p> \n",
    "<pre>\n",
    "classifier.fit(xtrain, ytrain)\n",
    "</pre>\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5. Score the model on the testing data. How is the accuracy? Is it comparable to the first model?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>Reveal Solution 🙈</summary>\n",
    "\n",
    "<p> \n",
    "<pre>\n",
    "classifier.score(xtest, ytest)\n",
    "</pre>\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "6. Make predictions and checking prediction probability! 🔮 \n",
    "\n",
    "We left a `customer` sample below but **make sure to adjust the numbers to your column order** - you can check the column order by checking your `x` (inputs) dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "customer = pd.DataFrame({\n",
    "    'Gender': [1],\n",
    "    'NumOfProducts': [1],\n",
    "    'HasCrCard': [1],\n",
    "    'IsActiveMember': [1],\n",
    "})\n",
    "# your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "    <summary>Reveal Solution 🙈</summary>\n",
    "\n",
    "<p> \n",
    "<pre>\n",
    "classifier.predict(customer)\n",
    "   # and in another code cell:\n",
    "classifier.predict_proba(customer)\n",
    "</pre>\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "7. **Explainability** - let's check importance of each input with `permutation_importance` once again. What do you find? 🔍\n",
    "\n",
    "⚠️ We left the variables that **you need to replace with your variables** in `UPCASED_LETTERS` 👇"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "permutation_score = permutation_importance(YOUR_MODEL, TRAINING_X_DATA, TRAINING_Y_DATA, n_repeats=10)\n",
    "\n",
    "np.vstack((TRAINING_X_DATA.columns, permutation_score.importances_mean)).T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Bonus:** we can visualize the findings above with another `scatterplot`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.scatterplot(data=churn, x='NumOfProducts', y='Balance', hue='Exited')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Congratulations on completing the optional challenge! 🏋️‍♀️🏋️‍♂️"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}