Lab4-Exercises.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Decision Trees\n",
    "## Exercise 1 \n",
    "Ternary classification\n",
    "\n",
    "The following code creates a small dataset with two attributes and a target\n",
    "variable with three possible values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "X = pd.DataFrame({'X1': [1, 1, 1, 1, 0, 0],\n",
    "                  'X2': [1, 1, 1, 0, 0, 0]})\n",
    "Y = pd.Series([1, 1, 2, 3, 2, 3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Calculate the information gain for `X1` and `X2` relative to `Y`.\n",
    "1. Based on these calculations, what attribute will be used for the first node of the ID3 tree?\n",
    "1. Learn the entire tree and classify the instance `{'X1': 0, 'X2': 1}`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2\n",
    "ID3 as a \"greedy\" algorithm\n",
    "\n",
    "The following code creates a dataset with features `A, B, C` and target variable `Y`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "X = pd.DataFrame({'A': [1, 1, 0, 0],\n",
    "                  'B': [1, 0, 1, 0],\n",
    "                  'C': [0, 1, 1, 1]})\n",
    "Y = pd.Series([0, 1, 1, 0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Find the decision tree using ID3. Is it _consistent_ with the training data (does it have 100% accuracy)?\n",
    "1. Is there a less deep decision tree consistent with the above data? If so, what logic concept does it represent?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3\n",
    "\n",
    "Titanic dataset\n",
    "\n",
    "The table bellow shows a few statistics on the survivors of the Titanic:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Class</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Age</th>\n",
       "      <th>Passengers</th>\n",
       "      <th>Survivors</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Upper</td>\n",
       "      <td>Male</td>\n",
       "      <td>Child</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Upper</td>\n",
       "      <td>Male</td>\n",
       "      <td>Adult</td>\n",
       "      <td>175</td>\n",
       "      <td>57</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Upper</td>\n",
       "      <td>Female</td>\n",
       "      <td>Child</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Upper</td>\n",
       "      <td>Female</td>\n",
       "      <td>Adult</td>\n",
       "      <td>144</td>\n",
       "      <td>140</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Lower</td>\n",
       "      <td>Male</td>\n",
       "      <td>Child</td>\n",
       "      <td>59</td>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Lower</td>\n",
       "      <td>Male</td>\n",
       "      <td>Adult</td>\n",
       "      <td>1492</td>\n",
       "      <td>281</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Lower</td>\n",
       "      <td>Female</td>\n",
       "      <td>Child</td>\n",
       "      <td>44</td>\n",
       "      <td>27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Lower</td>\n",
       "      <td>Female</td>\n",
       "      <td>Adult</td>\n",
       "      <td>281</td>\n",
       "      <td>176</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Class  Gender    Age  Passengers  Survivors\n",
       "0  Upper    Male  Child           5          5\n",
       "1  Upper    Male  Adult         175         57\n",
       "2  Upper  Female  Child           1          1\n",
       "3  Upper  Female  Adult         144        140\n",
       "4  Lower    Male  Child          59         24\n",
       "5  Lower    Male  Adult        1492        281\n",
       "6  Lower  Female  Child          44         27\n",
       "7  Lower  Female  Adult         281        176"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "titanic = pd.DataFrame([\n",
    "  ('Upper', 'Male', 'Child', 5, 5),\n",
    "  ('Upper', 'Male', 'Adult', 175, 57),\n",
    "  ('Upper', 'Female', 'Child', 1, 1),\n",
    "  ('Upper', 'Female', 'Adult', 144, 140),\n",
    "  ('Lower', 'Male', 'Child', 59, 24),\n",
    "  ('Lower', 'Male', 'Adult', 1492, 281),\n",
    "  ('Lower', 'Female', 'Child', 44, 27),\n",
    "  ('Lower', 'Female', 'Adult', 281, 176)\n",
    "],\n",
    "columns=['Class', 'Gender', 'Age', 'Passengers', 'Survivors'])\n",
    "titanic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. We want to build a decision tree to predict the target variable `Y` (survived) based on variables `C` (class), `G` (gender) and `A` (age). Using information gain, determine which of the three variables will be used in the root node.\n",
    "2. What is the training accuracy of the decision tree consisting only of the root node above?\n",
    "3. If you were to build the full tree using all attributes, what would be the training accuracy? Note that you don’t have to actually build the full tree!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 4\n",
    "Exoplanets, one-hot encoding\n",
    "\n",
    "Given a dataset with data regarding 800 exoplanets, fit a decision tree to find how well `Size` and `Orbit` describe if a planet is habitable. In other words, find the training accuracy of a decision tree model that uses those two variables to predict `Habitable` and also print the resulting tree."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Big</th>\n",
       "      <th>Orbit</th>\n",
       "      <th>Habitable</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Big</td>\n",
       "      <td>Near</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Big</td>\n",
       "      <td>Near</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Big</td>\n",
       "      <td>Near</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Big</td>\n",
       "      <td>Near</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Big</td>\n",
       "      <td>Near</td>\n",
       "      <td>Yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>795</th>\n",
       "      <td>Small</td>\n",
       "      <td>Far</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>796</th>\n",
       "      <td>Small</td>\n",
       "      <td>Far</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>797</th>\n",
       "      <td>Small</td>\n",
       "      <td>Far</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>798</th>\n",
       "      <td>Small</td>\n",
       "      <td>Far</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>799</th>\n",
       "      <td>Small</td>\n",
       "      <td>Far</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>800 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       Big Orbit Habitable\n",
       "0      Big  Near       Yes\n",
       "1      Big  Near       Yes\n",
       "2      Big  Near       Yes\n",
       "3      Big  Near       Yes\n",
       "4      Big  Near       Yes\n",
       "..     ...   ...       ...\n",
       "795  Small   Far        No\n",
       "796  Small   Far        No\n",
       "797  Small   Far        No\n",
       "798  Small   Far        No\n",
       "799  Small   Far        No\n",
       "\n",
       "[800 rows x 3 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tools.pd_helpers import apply_counts\n",
    "exoplanets = pd.DataFrame([\n",
    "  ('Big', 'Near', 'Yes', 20),\n",
    "  ('Big', 'Far', 'Yes', 170),\n",
    "  ('Small', 'Near', 'Yes', 139),\n",
    "  ('Small', 'Far', 'Yes', 45),\n",
    "  ('Big', 'Near', 'No', 130),\n",
    "  ('Big', 'Far', 'No', 30),\n",
    "  ('Small', 'Near', 'No', 11),\n",
    "  ('Small', 'Far', 'No', 255)\n",
    "],\n",
    "columns=['Big', 'Orbit', 'Habitable', 'Count'])\n",
    "exoplanets = apply_counts(exoplanets, 'Count')\n",
    "exoplanets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 5\n",
    "\n",
    "Exoplanets, continuous variable\n",
    "\n",
    "Given a dataset with 9 exoplanets for which we know the `Temperature` as well as the target variable `Habitable`.\n",
    "\n",
    "1. Find the training accuracy of a decision tree that predicts `Habitable` using `Temperature` and print the resulting tree.\n",
    "1. Independently calculate the split points that the algorithm will use for `Temperature` and check it against the generated tree. (The solution does not need to be general, can be \"hard-coded\" for this dataset.)\n",
    "1. Independently calculate the entropy of the root node of the generated tree."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Temperature</th>\n",
       "      <th>Habitable</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>205</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>205</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>260</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>380</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>205</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>260</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>260</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>380</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>380</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Temperature  Habitable\n",
       "0          205          0\n",
       "1          205          0\n",
       "2          260          1\n",
       "3          380          1\n",
       "4          205          0\n",
       "5          260          1\n",
       "6          260          1\n",
       "7          380          0\n",
       "8          380          0"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exoplanets = pd.DataFrame([\n",
    "  (205, 0),\n",
    "  (205, 0),\n",
    "  (260, 1),\n",
    "  (380, 1),\n",
    "  (205, 0),\n",
    "  (260, 1),\n",
    "  (260, 1),\n",
    "  (380, 0),\n",
    "  (380, 0)\n",
    "],\n",
    "columns=['Temperature', 'Habitable'])\n",
    "exoplanets"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}