Lab5-Exercises.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1\n",
    "\n",
    "Given the following dataset with two input random variables $X_1$ and $X_2$ and a target variable $Y$, we want to compare two extreme decision tree algorithms:\n",
    "\n",
    "* OVERFIT will build a full standard ID3 decision tree, with no pruning;\n",
    "* UNDERFIT will make no splits at all, always having a single node (which is both root and decision).\n",
    "\n",
    "1. Plot the full OVERFIT tree.\n",
    "1. What is the CVLOO error for OVERFIT?\n",
    "1. What is the CVLOO error for UNDERFIT?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "d = pd.DataFrame({'X1': [1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8],\n",
    "                  'X2': [1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2],\n",
    "                  'Y' : [0, 1, 1, 0, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1]})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2\n",
    "\n",
    "Suppose we learned a decision tree from a training set with binary output values (either 0 or 1). We find that for a leaf node $l$, \n",
    "\n",
    "* there are $M$ training examples falling into it (labeled either 0 or 1); \n",
    "* its entropy is $H$. \n",
    "\n",
    "1. Create a graph using `matplotlib` that shows the entropy $H$ as a function of the proportion of 1s in $M$. The proportion should be on the $x$ axis (from 0 to 1), while the entropy should be on the $y$ axis.\n",
    "1. Create a simple algorithm which takes as input $M$ and $H$ and that outputs the number of training examples misclassified by leaf node $l$.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3\n",
    "\n",
    "Given the dataset below:\n",
    "1. plot the points and the labels using `matplotib.pyplot.scatter`;\n",
    "1. train a regular decision tree, then plot its decision surface;\n",
    "1. create a new dataset with 1000 random points with coordinates between 0 and 10, which the diagonal line $X1 = X2$ perfectly separates in two classes. See [numpy.random.random_sample](https://numpy.org/doc/stable/reference/random/generated/numpy.random.random_sample.html#numpy.random.random_sample) for easily generating random numbers between 0 and 1.\n",
    "1. train a regular decision tree, then plot its decision surface on the new dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "d = pd.DataFrame({'X1': [1, 2, 3, 3, 3, 4, 5, 5, 5],\n",
    "                  'X2': [2, 3, 1, 2, 4, 4, 1, 2, 4],\n",
    "                  'Y':  [1, 1, 0, 0, 0, 0, 1, 1, 0]})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 4\n",
    "\n",
    "Given the following dataset, with input attributes $A$, $B$, and $C$ and target attribute $Y$, predict the entry $A=0, B=0, C=1$ using `BernoulliNB(alpha=1e-10)` and `predict_proba()` then manually calculate the probabilities using the formulas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "d = pd.DataFrame({'A': [0, 0, 1, 0, 1, 1, 1],\n",
    "                  'B': [0, 1, 1, 0, 1, 0, 1],\n",
    "                  'C': [1, 0, 0, 1, 1, 0, 0],\n",
    "                  'Y': [0, 0, 0, 1, 1, 1, 1]})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 5\n",
    "\n",
    "Consider two random variables $X_1$ and $X_2$ and a label $Y$ assigned to each instance as in the dataset `d` created below.\n",
    "\n",
    "1. Classify the instance $X_1=0,X_2=0$ using Naive Bayes.\n",
    "\n",
    "1. According to Naive Bayes, what is the probability of this classification?\n",
    "\n",
    "1. How many probabilities are estimated by the model (check the `class_log_prior_` and `feature_log_prob_` attributes)?\n",
    "\n",
    "1. How many probabilities would be estimated by the model if there were $n$ features instead of 2?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from tools.pd_helpers import apply_counts\n",
    "\n",
    "d_grouped = pd.DataFrame({\n",
    "    'X1': [0, 0, 1, 1, 0, 0, 1, 1],\n",
    "    'X2': [0, 0, 0, 0, 1, 1, 1, 1],\n",
    "    'C' : [2, 18, 4, 1, 4, 1, 2, 18],\n",
    "    'Y' : [0, 1, 0, 1, 0, 1, 0, 1]})\n",
    "d = apply_counts(d_grouped, 'C')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}