7_CreditCardDefaults_AdaptiveBoosting.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "# Boosting\n",
    "#### Implementing Adaptive Boosting with a simple classifier\n",
    "\n",
    "---------------------------------------\n",
    "\n",
    "### Assignment Contents:\n",
    "- [Building a Simple Classifier](#Building-a-Simple-Binary-Tree-Classifier)\n",
    "- [Building Adaptive Boosting](#Building-Adaptive-Boosting)\n",
    "- [Census Data](#Census-Data)\n",
    "- [AdaBoost in `sklearn`](#sklearn)\n",
    "\n",
    "### Overview\n",
    "This project extends the work done in my previous project with Random Forests.  \n",
    "\n",
    "Initially this notebook is theory heavy, both a simple classifier and an adaptive boosting model will be built from scratch. This will primarily involve creating `Python` implementations of the algorithms. As usual, pre-built versions of these algorithms will be demonstrated at the end of the notebook. Building the algorithms from scratch help to ensure depth of theoretical knowledge before allowing `Python` and its packages to handle the heavy lifting.\n",
    "\n",
    "### Activities in this Project\n",
    "- Create a simple Classifier\n",
    "    - Find potential splits in data\n",
    "    - Find the best split according to entropy\n",
    "    - Create binary predictions given a chosen split\n",
    "- Create an Adaptive Boosting Algorithm\n",
    "    - Create Weights\n",
    "    - Calculate Epsilon and Alph\n",
    "    - Update Weights\n",
    "   \n",
    "- Use Adaptive Boosting to Create Predictions on Census Data\n",
    "\n",
    "#### Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "### Building a Simple Binary Tree Classifier\n",
    "\n",
    "Below, pseudo-code for a simple binary tree classifier class is provided.  \n",
    "\n",
    "The structure of this pseudo-code mimics the structure of the `sklearn` classifiers in that it creates \"fit\" and \"predict\" methods.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "class Simple_Binary(object):\n",
    "    \n",
    "    def __init__(self):\n",
    "        pass\n",
    "    \n",
    "    def fit(self, X,y):\n",
    "        \"\"\"\n",
    "            1. Find best split in X\n",
    "                - According to entropy\n",
    "            2. After finding split, assign:\n",
    "                - self.col_idx\n",
    "                - self.split_value\n",
    "                - self.left_pred\n",
    "                - self.right_pred\n",
    "        \"\"\"\n",
    "        pass\n",
    "    \n",
    "    def predict(self, X):\n",
    "        \"\"\"\n",
    "            1. Make predictions given values calculated\n",
    "                in the `.fit(X,y)` method.\n",
    "            2. return predictions as numpy array.\n",
    "        \"\"\"\n",
    "        pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "Instead of building a class like `sklearn`, our `.fit()` and `.predict()` methods will be written as independent `simple_binary_tree_fit()` and `simple_binary_tree_predict()` functions.\n",
    "\n",
    "In `simple_binary_tree_fit`, **3 steps** must be accomplished:  \n",
    "\n",
    "1. Find all of the potential values for splitting.\n",
    "2. Find the column and split_value that results in the lowest entropy.\n",
    "3. Given that \"best split\", determine which predictions should be made when data is \"<=\" and \">\" that split value.  \n",
    "\n",
    "The below provides framework for returning the column and split value that yeilds the lowest entropy, and the predictions indicated by that split.  \n",
    "After correctly defining `find_splits()`, `ent_from_split()`, and `pred_from_split()` -- marked with \"`### <------`\" -- the `simple_binary_tree_fit()` function should work.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "def simple_binary_tree_fit(X,y):\n",
    "    \"\"\"\n",
    "    \n",
    "    Positional arguments -\n",
    "        X -- a numpy array of numeric observations:\n",
    "            Assume rows are separate observations, columns are features\n",
    "        y -- a numpy array of binary labels:\n",
    "            *Assume labels are 1 for \"True\" and 0 for \"False\"*\n",
    "            \n",
    "    1. Find best split in X\n",
    "        - According to entropy\n",
    "    2. After finding split, return:\n",
    "        - col_idx - index of column used to split data\n",
    "        - split_value - value upon which data is split\n",
    "        - left_pred - The prediction for observation <= split_value\n",
    "        - right_pred - The prediciton for observation > split_value\n",
    "        \n",
    "    \"\"\"\n",
    "    \n",
    "    # create variable \"best_split\" which will hold:\n",
    "    # (col_number, split_value, entropy)\n",
    "    best_split = (-1,-1,1)\n",
    "    \n",
    "    # loop through each column in X, keeping track of the column index.\n",
    "    # # # Note, taking the transpose of X -- X.T -- yeilds columns in this \"for\" loop\n",
    "    for col_idx, col in enumerate(X.T):\n",
    "        \n",
    "        # Find potential split values within column using `find_splits(col)`\n",
    "        splits = find_splits(col) ### <------\n",
    "        \n",
    "        # For each split, calculate entropy\n",
    "        for s in splits:\n",
    "            ent = ent_from_split(col, s, y) ### <------\n",
    "            \n",
    "            # Check if calculated entropy is less than previous \"best\"\n",
    "            if ent < best_split[2]:\n",
    "                best_split = (col_idx, s, ent)\n",
    "    \n",
    "    # Now, the \"best split\" has been found.\n",
    "    # create \"left\" and \"right\" predictions for the best_split\n",
    "    # The \"left\" predictions is for when `observation` <= `split_value`\n",
    "    # The \"right\" prediction is for when `observation` > `split_value`\n",
    "    # Each prediction will either be 1 for \"True\" or 0 for \"False\"\n",
    "    \n",
    "    left_pred, right_pred = pred_from_split(X, y, *best_split[:2]) ### <------\n",
    "    \n",
    "    col_idx, split_value = best_split[:2]\n",
    "    \n",
    "    # return:\n",
    "    # - the index of the column to split on.\n",
    "    # - the value to split that column on\n",
    "    # - the prediction for rows with observations in that column less than or equal to the split\n",
    "    # - the prediction for rows with observations in that column greater than the split\n",
    "    \n",
    "    return col_idx, split_value, left_pred, right_pred"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "Build the `find_splits()` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Code a function called `find_splits`.\n",
    "### ACCEPT a 1-dimensional numpy array as input.\n",
    "### RETURN a numpy.array of \"split values\"\n",
    "\n",
    "### \"Split values\" are the mid-points between the values in the sorted list of unique values.\n",
    "\n",
    "### e.g., Input of np.array([1, 3, 2, 3, 4, 6])\n",
    "### Yields a sorted-unique list of: np.array([1, 2, 3, 4, 6])\n",
    "### Then the \"splits\" in between those values will be: np.array([1.5, 2.5, 3.5, 5])\n",
    "\n",
    "def find_splits(col):\n",
    "    \"\"\"\n",
    "    Calculate and return all possible split values given a column of numeric data\n",
    "    \n",
    "    Positional argument:\n",
    "        col -- a 1-dimensional numpy array, corresponding to a numeric\n",
    "            predictor variable.\n",
    "    \n",
    "    Example:\n",
    "        col = np.array([0.5, 1. , 3. , 2. , 3. , 3.5, 3.6, 4. , 4.5, 4.7])\n",
    "        splits  = find_splits(col)\n",
    "        print(splits) # --> np.array([0.75, 1.5, 2.5, 3.25, 3.55, 3.8, 4.25, 4.6])\n",
    "        \n",
    "    \"\"\"\n",
    "    \n",
    "    # col = np.array([0.5, 1. , 3. , 2. , 3. , 3.5, 3.6, 4. , 4.5, 4.7])\n",
    "    # Sort list of unique values\n",
    "    srt = np.sort(np.unique(col))\n",
    "    # srt = np.array([0.5 1. 2. 3. 3.5 3.6 4. 4.5 4.7])\n",
    "    \n",
    "    # Shift values by inserting value at begining, also deleting final value\n",
    "    shift = np.delete(np.insert(srt,0,0), -1)\n",
    "    # shift = np.array([0. 0.5 1. 2. 3. 3.5 3.6 4. 4.5])\n",
    "    # Take the difference divided by two of the sorted and shifted list, and remove initial placeholder\n",
    "    # Add to the sorted list with final element removed\n",
    "    splits = np.delete(srt,-1) + np.delete((srt-shift)/2,0)\n",
    "    # splits =\n",
    "    # # np.delete(srt,-1) --> np.array([0.5 1. 2. 3. 3.5 3.6 4. 4.5])\n",
    "    # # +\n",
    "    # # np.delete((srt-shift)/2,0) -- > np.array([0.25 0.5 0.5 0.25 0.05 0.2 0.25 0.1 ])\n",
    "    # # =\n",
    "    # # np.array([0.75, 1.5, 2.5, 3.25, 3.55, 3.8, 4.25, 4.6])\n",
    "    \n",
    "    return splits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "#### Equation for Entropy for binary classification at binary split:\n",
    "The entropy at a node containing only two classes is calculated by:\n",
    "\n",
    "$Entropy(node) = -p_{class1}*log_2(p_{class1}) + -p_{class2}*log_2(p_{class2})$  \n",
    "\n",
    "Suppose a node contains the observations [1,0,1,1]. Then:  \n",
    "\n",
    "$Entropy(node) = -p_{class1}*log_2(p_{class1}) + -p_{class2}*log_2(p_{class2})$  \n",
    "\n",
    "$\\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ = -.75*log_2(.75) + -.25*log_2(.25)$\n",
    "\n",
    "$\\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ = .311 + .5$\n",
    "\n",
    "$\\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\approx .811$  \n",
    "\n",
    "\n",
    "This calculation is already programmed into the supplied `entropy()` function below.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.8112781244591328\n"
     ]
    }
   ],
   "source": [
    "def entropy(class1_n, class2_n):\n",
    "    # If all of one category, log2(0) does not exist,\n",
    "    # and entropy = 0\n",
    "    if (class1_n == 0) or (class2_n == 0):\n",
    "        return 0\n",
    "\n",
    "    # Find total number of observations \n",
    "    total = class1_n + class2_n\n",
    "\n",
    "    # find proportion of both classes\n",
    "    class1_proprtion = class1_n/total\n",
    "    class2_proportion = class2_n/total\n",
    "\n",
    "    # implement entropy function\n",
    "    return  sum([-1 * prop * np.log2(prop)\n",
    "                 for prop in [class1_proprtion, class2_proportion] ])\n",
    "\n",
    "print(entropy(3,1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "The entropy of a split is:  \n",
    "$Entropy(split) = p_{node1} * Entropy(node1)+ p_{node2}* Entropy(node2)$  \n",
    "\n",
    "\n",
    "Where $p_{node}$ is the proportion of observations at that node\n",
    "\n",
    "Suppose:  \n",
    "Node 1 contains the observations - [1,0,1,1]  \n",
    "Node 2 contains the observations - [0,0,0,1,1,0]\n",
    "\n",
    "Then:  \n",
    "$Entropy(split) = p_{node1} * Entropy(node1)+ p_{node2}* Entropy(node2)$  \n",
    "\n",
    "$\\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ = .4 *Entropy(node1) + .6 * Entropy(node2)$\n",
    "\n",
    "--------------------------------  \n",
    "\n",
    "For our purposes, the two classes in each node will be defined by:\n",
    "1. Observations with values less than or equal to the split value  \n",
    "2. Observations with values greater than the split value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Code a function called `ent_from_split`\n",
    "### ACCEPT three inputs:\n",
    "### 1. A numpy array of values\n",
    "### 2. A value on which to split the values in the first array, (into two groups; <= and >)\n",
    "### 3. Labels for the observations corresponding to each value in the first array.\n",
    "### ### Assume the labels are \"0\"s and \"1\"s\n",
    "\n",
    "### RETURN the entropy resulting from that split: a float between 0 and 1.\n",
    "\n",
    "def ent_from_split(col, split_value, labels):\n",
    "    \n",
    "    \"\"\"\n",
    "    Calculate the entropy of a split.\n",
    "    \n",
    "    Positional arguments:\n",
    "        col -- a 1-dimensional numpy array, corresponding to a numeric\n",
    "            predictor variable.\n",
    "        split_value --  number, defining where the spliting should occur\n",
    "        labels -- a 1-dimensional numpy array, corresponding to the class\n",
    "            labels associated with the observations in `col`.\n",
    "            assume they will be \"0\"s and \"1\"s\n",
    "    Example:\n",
    "        col = np.array([1,1,2,2,3,3,4])\n",
    "        split = 2.5\n",
    "        labels = np.array([0,1,0,0,1,0,1])\n",
    "        \n",
    "        ent = ent_from_split(col, split, labels)\n",
    "        \n",
    "        print(ent) # --> 0.8571428571428571\n",
    "    \n",
    "    \"\"\"\n",
    "    # Define entropy function\n",
    "    def entropy(class1_n, class2_n):\n",
    "        # If all of one category, log2(0) does not exist,\n",
    "        # and entropy = 0\n",
    "        if (class1_n == 0) or (class2_n == 0):\n",
    "            return 0\n",
    "        # Find total number of observations\n",
    "        total = class1_n + class2_n\n",
    "        # find proportion of both classes\n",
    "        class1_proprtion = class1_n/total\n",
    "        class2_proportion = class2_n/total\n",
    "        # implement entropy function\n",
    "        return sum([-1 * prop * np.log2(prop)\n",
    "                    for prop in [class1_proprtion, class2_proportion] ])\n",
    "    \n",
    "    # Helper function to find number in each class at each node.\n",
    "    def node_classes(col, split_value, labels):\n",
    "        # subset labels by observations that are <= / > the split value\n",
    "        le_node = labels[col <= split_value]\n",
    "        g_node = labels[col > split_value]\n",
    "        # count members of each class at each node\n",
    "        # c1 corresponds to \"1's\"\n",
    "        # c2 corresponds to \"0's\n",
    "        le_c1 = np.count_nonzero(le_node)\n",
    "        le_c2 = len(le_node) - le_c1\n",
    "        g_c1 = np.count_nonzero(g_node)\n",
    "        g_c2 = len(g_node) - g_c1\n",
    "        return le_c1, le_c2, g_c1, g_c2\n",
    "    \n",
    "    # Find number in each class at both nodes\n",
    "    le_c1, le_c2, g_c1, g_c2 = node_classes(col, split_value, labels)\n",
    "    #print(le_c1, le_c2, g_c1, g_c2)\n",
    "    # calculate proportion of total at each node\n",
    "    total = len(col)\n",
    "    p_g = (g_c1 + g_c2) / total\n",
    "    p_le = (le_c1+ le_c2) / total\n",
    "    \n",
    "    # implement entropy calculation\n",
    "    return (p_le * entropy(le_c1, le_c2)) + (p_g * entropy(g_c1, g_c2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "Creating predictions from the observed majority class at each node."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Code a function called `pred_from_split`\n",
    "### ACCEPT four inputs:\n",
    "### 1. a numpy array of observations\n",
    "### 2. a numpy array of labels: 0's and 1's\n",
    "### 3. a column index\n",
    "### 4. a value to split that column specified by the index\n",
    "\n",
    "### RETURN a tuple of (left_pred, right_pred) where:\n",
    "### left_pred is the majority class of labels where observations are <= split_value\n",
    "### right_pred is the majority class of labels where observations are > split_value\n",
    "\n",
    "### If the split yeilds equal number of observations of each class in BOTH nodes,\n",
    "### ### let both `left_pred` and `right_pred` be 1.\n",
    "### If the split yeilds equal number of observations of each class in ONLY ONE node,\n",
    "### ### predict the opposite of the other node. e.g.\n",
    "\n",
    "### ### node 1    |   node 2\n",
    "### ###  c1  | c2 |  c1 | c2\n",
    "### ###  5  | 4   |  3  |  3\n",
    "\n",
    "### The prediction for node 1 would be \"class 1\".\n",
    "### Because of the equal numbers of each class in node 2,\n",
    "### the prediction for node 2 would be the opposite of the node 1 prediction.\n",
    "### e.g. the prediction for node 2 would be \"class 2\"\n",
    "\n",
    "def pred_from_split(X, y, col_idx, split_value):\n",
    "    \n",
    "    \"\"\"\n",
    "    Return predictions for the nodes defined by the given split.\n",
    "    \n",
    "    Positional argument:\n",
    "        X -- a 2-dimensional numpy array of predictor variable observations.\n",
    "            rows are observations, columns are features.\n",
    "        y -- a 1-dimensional numpy array of labels, associated with observations\n",
    "             in X.\n",
    "        col_idx -- an integer index, such that X[:,col_idx] yeilds all the observations\n",
    "            of a single feature.\n",
    "        split_value -- a numeric split, such that the values of X[:,col_idx] that are\n",
    "            <= split_value are in the left node. Those > split_value are in the right node.\n",
    "    \n",
    "    Example:\n",
    "        X = np.array([[0.5, 3. ], [1.,  2. ], [3.,  0.5],\n",
    "                      [2.,  3. ], [3.,  4. ]])\n",
    "            \n",
    "        y = np.array([ 1, 1, 0, 0, 1])\n",
    "        \n",
    "        col_idx = 0\n",
    "        \n",
    "        split_value = 1.5\n",
    "        \n",
    "        pred_at_nodes = pred_from_split(X, y, col_idx, split_value)\n",
    "        print(pred_at_nodes) # --> (1, 0)\n",
    "\n",
    "    \"\"\"\n",
    "    # Helper function to find number in each class at each node.\n",
    "    def node_classes(col, split_value, labels):\n",
    "        # subset labels by observations that are <= / > the split value\n",
    "        le_node = labels[col <= split_value]\n",
    "        g_node = labels[col > split_value]\n",
    "        # count members of each class at each node\n",
    "        # c1 corresponds to \"1's\"\n",
    "        # c2 corresponds to \"0's\n",
    "        le_c1 = np.count_nonzero(le_node)\n",
    "        le_c2 = len(le_node) - le_c1\n",
    "        g_c1 = np.count_nonzero(g_node)\n",
    "        g_c2 = len(g_node) - g_c1\n",
    "        \n",
    "        return le_c1, le_c2, g_c1, g_c2\n",
    "    # Return count of each class at each node using helper function\n",
    "    le_c1, le_c2, g_c1, g_c2 = node_classes(X[:,col_idx], split_value, y)\n",
    "    \n",
    "    # if more of class 1 than class 2, return True (1)\n",
    "    # if more of class 2 than class 1, return False (0)\n",
    "    # if equal number, return None\n",
    "    def pred_for_node(cl1, cl2):\n",
    "        if cl1 > cl2: return True\n",
    "        elif cl1 < cl2: return False\n",
    "        else: return None\n",
    "        \n",
    "    # Create intial predictions\n",
    "    left = pred_for_node(le_c1,le_c2)\n",
    "    right = pred_for_node(g_c1, g_c2)\n",
    "    \n",
    "    # Check predictions for ties\n",
    "    # Check to see if one of the predictions came back as \"None\"\n",
    "    if ((right == None) or (left == None)) and (right != left):\n",
    "        if left == None:\n",
    "            left = not right\n",
    "        else:\n",
    "            right = not left\n",
    "    # Check to see if both predictions came back as \"None\"\n",
    "    if (right == None) and (left == None):\n",
    "        right = True; left = True\n",
    "    # Convert predictions to integers, and return as tuple\n",
    "    return (int(left), int(right))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "\n",
    "Creating Predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Code a function called \"simple_binary_tree_predict\"\n",
    "### ACCEPT five inputs:\n",
    "### 1. A numpy array of observations\n",
    "### 2. A column index\n",
    "### 3. A value to split the column specified by the index\n",
    "### 4/5. Two values, 1 or 0, denoting the predictions at left and right nodes\n",
    "\n",
    "### RETURN a numpy array of predictions for each observation\n",
    "\n",
    "### Predictions are created for each row in x:\n",
    "### 1. For a row in X, find the value in the \"col_idx\" column\n",
    "### 2. Compare to \"split_value\"\n",
    "### 3. If <= \"split_value\", predict \"left_pred\"\n",
    "### 4. Else predict \"right_pred\"\n",
    "\n",
    "def simple_binary_tree_predict(X, col_idx, split_value, left_pred, right_pred):\n",
    "    \"\"\"\n",
    "    Create an array of predictions built from: observations in one column of X,\n",
    "        a given split value, and given predictions for when observations\n",
    "        are less-than-or-equal-to that split or greater-than that split value\n",
    "        \n",
    "    Positional arguments:\n",
    "        X -- a 2-dimensional numpy array of predictor variable observations.\n",
    "            rows are observations, columns are different features\n",
    "        col_idx -- an integer index, such that X[:,col_idx] yeilds all the observations\n",
    "            in a single feature.\n",
    "        split_value -- a numeric split, such that the values of X[:,col_idx] that are\n",
    "            <= split_value are in the left node, and those > are in the right node.   \n",
    "        left_pred -- class (0 or 1), that is predicted when observations\n",
    "            are less-than-or-equal-to the split value\n",
    "        right_pred -- class (0 or 1), that is predicted when observations\n",
    "            are greater-than the split value\n",
    "            \n",
    "    Example:\n",
    "        X = np.array([[0.5, 3. ], [1.,  2. ], [3.,  0.5],\n",
    "                [2.,  3. ], [3.,  4. ]])\n",
    "        col_idx = 0\n",
    "        split_value = 1.5\n",
    "        left_pred = 1\n",
    "        right_pred = 0\n",
    "\n",
    "        preds = simple_binary_tree_predict(X, col_idx, split_value, left_pred, right_pred)\n",
    "\n",
    "        print(preds) #--> np.array([1,1,0,0,0])\n",
    "    \n",
    "    \"\"\"\n",
    "    def simple_binary_tree_predict_T(X, col_idx, split_value, left_pred, right_pred):\n",
    "        # take out column\n",
    "        col = X[:,col_idx]\n",
    "        # If both predictions are same, return all that prediction\n",
    "        if left_pred == right_pred:\n",
    "            preds = np.ones(len(col))\n",
    "            preds *= int(left_pred)\n",
    "        # Otherwise, predictions are different.\n",
    "        # if left_pred ( == 1)\n",
    "        # return 1's where col is <= split\n",
    "        # otherwise, 1's where col is > split\n",
    "        elif left_pred:\n",
    "            preds = (col<= split_value)*1\n",
    "        else:\n",
    "            preds = (col >split_value)*1\n",
    "        return preds "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "At this point, we have a functioning binary-tree classifier that can be fit on data, and then given that fit, make predictions on out-of-sample data.  \n",
    "\n",
    "However, our ultimate goal is creation of an Adaptive Boosting algorithm.  \n",
    "\n",
    "\n",
    "Our Adaptive Boosting algorithm's prediction for out of sample data will be:  \n",
    "\n",
    "$$f_{boost}(x_0) = sign(\\sum_{t=1}^T\\alpha_tf_t(X_0))$$  \n",
    "\n",
    "The alpha is equal to:\n",
    "$$\\alpha_t = \\frac12ln(\\frac{1-\\epsilon_t}{\\epsilon_t})$$  \n",
    "\n",
    "The Epsilon is equal to:  \n",
    "\n",
    "$$\\epsilon_t = \\sum_{i=1}^nw_t(i)\\mathbb{1}\\{y_i\\ne f_t(x_i)\\}$$  \n",
    "\n",
    "Where all weights starts at$\\frac1n$  \n",
    "\n",
    "And weights update by:\n",
    "$$w_{t+1}(i) = \\frac{\\hat{w}_{t+1}(i)}{\\sum_j\\hat{w}_{t+1}(j)}$$  \n",
    "\n",
    "Where:\n",
    "$$\\hat{w}_{t+1}(i) = w_t(i)e^{-\\alpha_ty_if_t(x_i)}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "While we could use the `simple_binary_tree` functions created above, in the interest of speed, we will use sklearn's `DecisionTreeClassifier` as a the simple predictor to boost.\n",
    "\n",
    "The below gives a short example of using `DecisionTreeClassifier` that:  \n",
    "\n",
    "- Splits toy data in two\n",
    "- Builds two Trees each on 1/2 of data\n",
    "- Saves each tree with an associated \"alpha\" in a dictionary (As will be done in boosting)\n",
    "- Creates predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAEICAYAAAB/KknhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAF0ZJREFUeJzt3X2UXHddx/H3p9PdbegDWFqgD0C6QSFCYoIrUSrQVA62pVJUCAiNJAdoAijF4ukRPSoqiI/Yc7QSqzUtKQKLgA9VPFRIWhvaLZsmTUITLdm2PDTQtLG0oTEblq9//O7S283uzp3dO3N39n5e58zJztw7v/udO/czv3vv3PlFEYGZ1ctxVRdgZp3n4JvVkINvVkMOvlkNOfhmNeTgm9WQg1+ApEOS+quuY66QtFBSSDp+iun3SXplp+uapI7rJH2gQ8t6v6QbOrGsMnQs+JLeJGk4C9F+SZ+T9NMFnxuSntfuGqcSESdFxEiZbWav6bvZ+nhY0hckvaHMZUyz7K7ZSCWdIukqSV/L1tVXs/undWj5L5X0pezvSyTtkPSopIey92xhJ+ooW0eCL+kK4CrgD4FnAs8B/hq4pBPLn6mperQS/VhEnAQ8H7gO+CtJv9vmZXYNSb3AF4AXAhcApwAvBR4GXtLG5ebf94uAf886no8C7wWeCpxD2oa/36462ioi2nojraRDwOunmeclwG3AI8B+4K+A3mzaLUAA383aeUP2+MXAjuw5XwKW5tp7MbAdeAz4FPBJ4AO56W8HvgocBP4FODM3LYB3AfcA9+Yee172dx/wZ8DXgG8DG4AF2bTTgBuzmg4C/wUcN8Vr/kGbucdeB/wf8PTcurs2WyffBD4ANLJpa4Bbs1r+F7gXuDDX1pnZazuYvda3Z49fAIwCR7P1eVeBZTWy5TwEjGTrJ4Djp3ht9wHvA+7OatsInJBN2w38XG7enqzdZZO087ZsHZ80zbazGNiSrfOvAK/JTbtuNu979vid2fb0OmDHNHW8HxgkfTg8ltUykJv+G8C+bNrdwM/npq0BtgJ/CXwH2Av8zIQMTfrezDiXHQj+BcD3ptpIsnl+HPhJ4HhgIbAHeM9UIcneiAeBFdlG+ZZsY+sDeoH7gcuzjeoXsg39A9lzz882tBdn8/8lcMuEZd0EnMoTgc4H/6psozkVOBn4V+BD2bQPkT4IerLbywC1EPyebF1dmN3/J+BvgBOBZwB3AOtyG8vRbGNuAO8AHhhfHnAzqUc6AVgGHBjfmLKN9IYJy55uWeuzjfHZ2eveTPPg787NvzW3/q8EPpmb9xJg1xTtfAK4fprtpocU5N/M3vfzScF6/sTgz/B9P4MUNAH9pA/lvwBWMuHDKFun/0faQ2hk28LtuemvJ30YHwe8gdSRnZF7L78H/Fr2mt5A+gA4tdl7M5eD/2bgWy0+5z3AZ6cJ/keAP5jwnP8GXgG8fPzNyk27NbcBXAv8SW7aSaQALcwt6/zJQpptAN8FFuWm/RRP7Bn8PvDPTAh00eBnj38rW2fPBI6Mb4TZtF8CNuc2lq/mpj0la/NZpMCNASfnpn8IuG6y4BdY1heB9blpr6J58PPzXwTsy/4+kxTOU7L7/whcOUU7NwF/NM06fFm2vo7LPfZx4P3Z39fN8n1/K3Bt7v5Pknr1A6SQX0f2AZCt0//MzfujwOFpat8BXJJ7L3/woZ09dgewutl7M9Nbu49hIR2PnSbp+Ij43mQzSPoR4MPAAGkDPh7YNk2bzwXeIulXc4/1kjaqAL4Z2RrKfD3395mk3TcAIuKQpIeBs0gb7MT5807P6tsm6Qflkz7hAf6UtAF8Ppt+TUT80TSv40kk9WTLOJi9xh5gf25Zx02o7Vu51/F4Nt9JwNOBgxHxWG7e+0nrdzLNlnXmhOXeX+DlTJz/zKzOByRtBX5R0meBC0l7Z5N5mNTrTuVM4OsRkT/Ovp/0Xk42b6vv+0XAP+SeczuwCkDST5AOIX+LdFgDufcDeBw4YXy7l/TLwBWkPVpI71P+BOXEbXZ8nRXZDlrWiZN7t5E+HV87zTwfIe1K/nBEnELaddM0838d+GBEPC13e0pEfJx0HHSWcmuJ1AOOe4C0MgGQdCIpKN/MzZN/A/IeAg4DL8wt96mRTtAREY9FxHsjoh/4OeAKST8zzeuY6BLSLt8d2Ws8ApyWW9YpEfHCAu08AJwq6eTcY8/JvcaJr6/Zsvbz5HX4nAI1TJz/gdz964FLSbu/t0VEft3n/Sfws9l7NJkHgGdLym/H+dc5cd7C73v2IfwK0l7HMSLiy8BngBdNUdsPSHou8LfAr5DO3zyNdCiU30YnbrPj62w228GU2h78iPgO8DvA1ZJeK+kpknokXSjpT7LZTgYeBQ5JegHpeDXv26RjrHF/C6yXtELJiZJenW3ot5F2c39F0vGSLuHJZ4D/AVgraZmkPtI3DUMRcV+B1/L9bNl/IekZAJLOkvSz2d8XS3pe9gY+mtUx1qxdSadKejNwNfDHEfFwROwHPg/8efaV1nGSFkl6RYE6v0464fkhSSdIWkrabf1YNsu3gYXjgSmwrEHg3ZLOlvRDpBNVzbwrm/9U0gf5J3PT/ol0rH056WTYVDaRNvxPS3pBVtfTJf2mpIuAIdKh15XZNnUe6QP3E5O01er7/jJgZ0Q8CiDppyW9Pfe+vwB4DXB781XBiaQPlQPZc9dy7AfGM0jruEfS60knLf99NtvBtGZznNDKjXTcOkx6o74F/Bvw0mzay0k9/iHSmfDfB27NPXc9qdd5BFiVPXYB8GWe+CbgU2THtKRd2h1Ze58ifTL/9oT29pF2qW8Ezs5Nm+ykW/7k3gmkjWaEFO49wLuzab9G2m38LvCN/DInWR/BE99UHCSdMHvThHmeStob+gbpZM924I2548Jbp6nz7Oy1Hcxea/6Y++mk8x7/C9xZYFnHk05qPUz69qCVs/qPkHr4p0yY5++y1z/lGftcXVeRPgAOZa/lwzzxzccLSScyv8OxZ8uv48ln9Qu/76RvMX49d/9FpBO5387quA/4Y6AnnjjGz583WZhfR8AHs+U+lNV/M/C23Hu5lfRt1neA/wFeVWQ7mOlt/AzwvCZpCNgQERurrsUSSb8D/EhEXFp1LZORdDfwuoi4uwPLWkP6ECh0QVsZ5uUlu5JeIelZ2a7+W4ClwH9UXZcl2e7/W4Frqq5lMtmFQx/tROirMi+DT7oS7i7SbtF7SZ/c+6styQAkvZ202/65iLil6nomExGj0cK3Md2oFrv6ZvZk87XHN7NptOUCntNOOy0WLlzYjqbNbBrbtm17KCJObzZfW4K/cOFChoeH29G0mU1DUpGrKr2rb93pvPPSzWbGwTerIQffrIYcfLMacvDNasjBN6shB9+shhx8sxpy8M1qqBNj7pm1pMiFOTffXHzeLVtmUUy77NoFH/kIbN4MIyMwOgq9vdDfDytXwjveAUuWtG3xDr5ZJ42MwOrVsGMHHDkCY7mR2UZHYe9euOceuP56WLYMNm1KHwYlc/BtzinSQ4/39HOyN5/K4CCsXXts4CcaG4PHH4ehodTrb9wIq1aVWoqDb9YJg4OwZg0cPlz8OeMfAGvWpPslht8n98zabWQk9fSthD7v8OH0/HvvLa0kB9+s3VavTrv3s3HkCFxa3rikDr5ZO+3cmU7kTXdMX8TYGGzfnr4NKIGDb9ZOGzbMvrcfNzqa2iuBg2/WTps3z763Hzc2ltorgYNv1k4jI+W2t29fKc04+GbtNDpabntHj5bSjL/Ht67UNRfu9PaWG/6enlKacY9v1k5lX267aFEpzTj4Zu20ciU0GuW01Wik9krg4FuttX2Y7vXroa+vnLZ6e1N7JXDwzdpp6dL0K7vZ9vqNBixfXtpPdR18s3bbtGn2vX5fH9xwQzn14OCbtV9/f/pp7YIFM3v+ggXp+eecU1pJ/jrPrBPGf1Jb5Pf44xqN1NO34ff47vHNOmXVqvQjmxUrUi8+1XF/o5Gmr1gBu3eXHnpw8M06q78ftm5No+usWweLF6ez9VL6d/Hi9PjQUJqvxN37PO/qm1VhyRK4+urKFu8e36yG3ONbKebi4Je1GKZ7htzjm9WQe3ybt+btMN0lcI9vVkMOvlkNOfhmNeTgm9VQ4eBLakjaLunGdhZkZu3XSo9/ObCnXYWYWecU+jpP0tnAq4EPAle0tSKbc3whzPxTtMe/CrgS+P5UM0i6TNKwpOEDBw6UUpyZtUfTHl/SxcCDEbFN0nlTzRcR1wDXAAwMDERpFVrl5vOFMN1Wb1mK9PjnAq+RdB/wCeB8SeWNAWRmHdc0+BHxvog4OyIWAm8EvhgR5f1/vWbWcf4e36yGWvqRTkRsAba0pRIz6xj3+GY15OCb1ZCDb1ZDDr5ZDXkEHitFXS+E6Vbu8c1qyME3qyEH36yGHHyzGnLwzWrIwTerIQffrIYcfLMacvDNasjBN6shB9+shhx8sxpy8M1qyME3qyEH36yGHHyzGnLwzWrIwTerIQffrIYcfLMacvDNasjBN6shB9+shhx8sxqqPPjnnZdutbdrF7zznbB4MfT1gZT+Xbw4Pb5rV9UV2jzi/0mnaiMjsHo17NgBR47A2NgT00ZHYe9euOceuP56WLYMNm2C/v7q6rV5ofIev9YGB2HJEhgagscff3Lo88bG0vShoTT/4GBn67R5xz1+VQYHYc0aOHy4+HPGPwDWrEn3V61qR2VWA+7xqzAyAmvXthb6vMOH0/Pvvbfcuqw2HPwqrF6djudn48gRuPTScuqx2nHwO23nznQib6rj+aLGxmD7dp/ttxlpGnxJJ0i6Q9Jdkr4i6fc6Udi8tWHD7Hv7caOjqT2zFhU5uXcEOD8iDknqAW6V9LmIuL3Ntc1PmzfPvrcfNzaW2jNrUdPgR0QAh7K7PdktijRe5MKcm28uPu+WLUWWOseNjJTb3r595bZntVDoGF9SQ9IO4EHgpogYmmSeyyQNSxo+cOBA2XXOH6Oj5bZ39Gi57VktKHXoBWeWngZ8FvjViNg91XwDAwMxPDxcqM3xnn5e9OZF9PWVG/7e3vLOGVjXk7QtIgaazdfSWf2IeATYAlwww7qs7MttFy0qtz2rhSJn9U/PenokLQBeCextd2Hz1sqV0GiU01ajkdoza1GRHv8MYLOkncCXScf4N7a3rHls/fq0u1+G3t7UnlmLipzV3wks70At9bB0afqV3dDQ7L7WazRg+fL0ox2zFvnKvSps2jT7Xr+vD264oZx6rHYc/Cr098PGjbBgwcyev2BBev4555Rbl9WGf5ZblfGf1K5de+wAHFNpNFJPv3Gjf5Jrs+Iev0qrVqUf2axYkXrxqc72Nxpp+ooVsHu3Q2+zVnmPX5sLd6bS3w9bt6YPgA0b0rX3+/alK/J6etL39CtXprP3PpFnJak8+JZZsgSuvrrqKqwmvKtvc5NHHW4r9/g2t3jU4Y5wj29zh0cd7hj3+DY3eNThjnKPb9XzqMMd5+Bb9TzqcMc5+FYtjzpcCQffquVRhyvh4Fu1POpwJRx8q5ZHHa6Eg2/V8qjDlXDwrVq9veW219NTbnvzlINv1fKow5Vw8K1aHnW4Eg6+VcujDlfCwbdqjY86PNte36MOt8TBt+p51OGOc/Cteh51uOP8s1ybGzzqcEe5x7e5w6MOd4yDb3PL+KjDQ0Owbl0aY6+3N42519ub7q9bl6Zv3erd+xnyrr7NTR51uK3c45vV0LwO/nnnpZtZx3TJsODe1TcrQ5cNCz6ve3yzjujCYcHd45vNRpcOC+4e32ymunhYcAffbKa6eFhwB99sJrp8WPCmwZf0bEmbJe2R9BVJl3eiMLM5rcuHBS9ycu97wHsj4k5JJwPbJN0UEXe3uTazuavLhwVvGvyI2A/sz/5+TNIe4Cyg0uAXuTDn5puLz7tlyyyKsfrp8mHBWzrGl7QQWA4MTTLtMknDkoYPHDhQTnVmc1WXDwte+Ht8SScBnwbeExGPTpweEdcA1wAMDAxEaRVOoUgPPd7Tuze30vX2lhv+Dg8LXqjHl9RDCv3HIuIz7S3JrAt0+bDgRc7qC7gW2BMRH25/SWZdoMuHBS/S458LrAbOl7Qju13U5rrM5rYuHxa8yFn9WwF1oBaz7jE+LPjQ0Oy+1qtoWHBfuWc2U108LLiDbzZTXTwsuH+WazYbXTosuHt8s9nqwmHB53WP7wt3rGPGhwXftSv94Gbz5nQZ7tGj6eKcRYvSV3br18+J/99vXgffrOO6ZFhw7+qb1ZCDb1ZDDr5ZDTn4ZjXk4JvVkINvVkMOvlkNOfhmNeTgm9WQg29WQw6+WQ05+GY15OCb1ZCDb1ZDDr5ZDTn4ZjXk4JvVkINvVkMOvlkNOfhmNeTgm9WQg29WQw6+WQ05+GY15OCb1ZCDb1ZDDr5ZDTn4ZjXk4JvVkINvVkNNgy/p7yU9KGl3Jwoys/Yr0uNfB1zQ5jrMrIOaBj8ibgEOdqAWM+uQ0o7xJV0maVjS8IEDB8pq1szaoLTgR8Q1ETEQEQOnn356Wc2aWRv4rL5ZDTn4ZjVU5Ou8jwO3Ac+X9A1Jb21/WWbWTsc3myEifqkThZhZ53hX36yGHHyzGnLwzWrIwTerIQffrIYcfLMacvDNasjBt5nbtQve+U5YvBj6+kBK/y5enB7ftavqCm0KTS/gMTvGyAisXg07dsCRIzA29sS00VHYuxfuuQeuvx6WLYNNm6C/v7p67Rju8a01g4OwZAkMDcHjjz859HljY2n60FCaf3Cws3XatNzjW3GDg7BmDRw+XPw54x8Aa9ak+6tWtaMya5F7fCtmZATWrm0t9HmHD6fn33tvuXXZjDj4Vszq1el4fjaOHIFLLy2nHpsVB9+a27kzncib6ni+qLEx2L7dZ/vnAAffmtuwYfa9/bjR0dSeVcrBt+Y2b559bz9ubCy1Z5Vy8K25kZFy29u3r9z2rGUOvjU3Olpue0ePltuetczBt+Z6e8ttr6en3PasZQ6+NVf25baLFpXbnrXMwbfmVq6ERqOcthqN1J5VysG35tavT7+6K0Nvb2rPKuXgW3NLl6Zf2c221280YPny9KMdq5SDb8Vs2jT7Xr+vD264oZx6bFYcfCumvx82boQFC2b2/AUL0vPPOafcumxG/LNcK278J7Vr1x47AMdUGo3U02/c6J/kziHu8a01q1alH9msWJF68amO+xuNNH3FCti926GfYxx8a11/P2zdmkbXWbcujbHX25vG3OvtTffXrUvTt2717v0c5F19m7klS+Dqq6uuwmbAPb5ZDTn4ZjXk4JvVkINvVkOKiPIblQ4A95fe8MycBjxUdREz0I11d2PNML/qfm5EnN7siW0J/lwiaTgiBqquo1XdWHc31gz1rNu7+mY15OCb1VAdgn9N1QXMUDfW3Y01Qw3rnvfH+GZ2rDr0+GY2gYNvVkPzNviS/l7Sg5J2V11LUZKeLWmzpD2SviLp8qprKkLSCZLukHRXVvfvVV1TKyQ1JG2XdGPVtRQl6T5JuyTtkDTc8vPn6zG+pJcDh4CPRsSLqq6nCElnAGdExJ2STga2Aa+NiLsrLm1akgScGBGHJPUAtwKXR8TtFZdWiKQrgAHglIi4uOp6ipB0HzAQETO68Gje9vgRcQtwsOo6WhER+yPizuzvx4A9wFnVVtVcJIeyuz3ZrSt6FElnA68G/q7qWjpp3ga/20laCCwHhqqtpJhsd3kH8CBwU0R0Rd3AVcCVwPerLqRFAXxe0jZJl7X6ZAd/DpJ0EvBp4D0R8WjV9RQREWMRsQw4G3iJpDl/eCXpYuDBiNhWdS0zcG5EvBi4EHhXdmhbmIM/x2THyJ8GPhYRn6m6nlZFxCPAFuCCiksp4lzgNdnx8ieA8yV1xfjfEfFA9u+DwGeBl7TyfAd/DslOkl0L7ImID1ddT1GSTpf0tOzvBcArgb3VVtVcRLwvIs6OiIXAG4EvRsSlFZfVlKQTs5O/SDoReBXQ0rdX8zb4kj4O3AY8X9I3JL216poKOBdYTep5dmS3i6ouqoAzgM2SdgJfJh3jd81XY13omcCtku4C7gD+LSL+o5UG5u3XeWY2tXnb45vZ1Bx8sxpy8M1qyME3qyEH36yGHHyzGnLwzWro/wGnqIqWhOqaLgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 288x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "df:\n",
      "      x    y  classification\n",
      "0  0.5  3.0               1\n",
      "1  1.0  2.0               1\n",
      "2  3.0  0.5              -1\n",
      "3  2.0  3.0              -1\n",
      "4  3.0  4.0               1\n",
      "5  3.5  2.5              -1\n",
      "6  3.6  4.7               1\n",
      "7  4.0  4.2               1\n",
      "8  4.5  2.0              -1\n",
      "9  4.7  4.5              -1 \n",
      "\n",
      "threshold: 1.5 feature: 0\n",
      "threshold: 3.3499999046325684 feature: 1\n",
      "\n",
      "tree1 predictions on all elements: [1 1 0 0 0 0 0 0 0 0]\n",
      "tree2 predictions on all elements: [0 0 0 0 1 0 1 1 0 1]\n",
      "\n",
      "Entropy of different splits for observations 5-9\n",
      "Col 1, @ 3.35: 0.5509775004326937\n",
      "Col 0, # 4.25: 0.5509775004326937\n"
     ]
    }
   ],
   "source": [
    "### This helper function will return an instance of a `DecisionTreeClassifier` with\n",
    "### our specifications - split on entropy, and grown to depth of 1.\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "def simple_tree():\n",
    "    return DecisionTreeClassifier(criterion = 'entropy', max_depth= 1)\n",
    "\n",
    "\n",
    "### Our example dataset, inspired from lecture\n",
    "pts = [[.5, 3,1],[1,2,1],[3,.5,-1],[2,3,-1],[3,4,1],\n",
    " [3.5,2.5,-1],[3.6,4.7,1],[4,4.2,1],[4.5,2,-1],[4.7,4.5,-1]]\n",
    "\n",
    "df = pd.DataFrame(pts, columns = ['x','y','classification'])\n",
    "\n",
    "# Plotting by category\n",
    "\n",
    "b = df[df.classification ==1]\n",
    "r = df[df.classification ==-1]\n",
    "plt.figure(figsize = (4,4))\n",
    "plt.scatter(b.x, b.y, color = 'b', marker=\"+\", s = 400)\n",
    "plt.scatter(r.x, r.y, color = 'r', marker = \"o\", s = 400)\n",
    "plt.title(\"Categories Denoted by Color/Shape\")\n",
    "plt.show()\n",
    "\n",
    "\n",
    "print(\"df:\\n\",df, \"\\n\")\n",
    "\n",
    "### split out X and y\n",
    "X = df[['x','y']]\n",
    "\n",
    "# Change from -1 and 1 to 0 and 1\n",
    "y = np.array([1 if x == 1 else 0 for x in df['classification']])\n",
    "\n",
    "### Split data in half\n",
    "X1 = X.iloc[:len(X.index)//2, :]\n",
    "X2 = X.iloc[len(X.index)//2:, :]\n",
    "\n",
    "y1 = y[:len(y)//2]\n",
    "y2 = y[len(X)//2:]\n",
    "\n",
    "\n",
    "### Fit classifier to both sets of data, save to dictionary:\n",
    "\n",
    "tree_dict = {}\n",
    "\n",
    "tree1 = simple_tree()\n",
    "tree1.fit(X1,y1)\n",
    "print(\"threshold:\", tree1.tree_.threshold[0], \"feature:\", tree1.tree_.feature[0])\n",
    "\n",
    "### made up alpha, for example\n",
    "alpha1 = .6\n",
    "tree_dict[1] = (tree1, alpha1)\n",
    "\n",
    "tree2 = simple_tree()\n",
    "tree2.fit(X2,y2)\n",
    "print(\"threshold:\", tree2.tree_.threshold[0], \"feature:\" ,tree2.tree_.feature[0])\n",
    "\n",
    "### made up alpha, again.\n",
    "alpha2 = .35\n",
    "\n",
    "tree_dict[2] = (tree2, alpha2)\n",
    "\n",
    "### Create predictions using trees stored in dictionary\n",
    "print(\"\\ntree1 predictions on all elements:\", tree_dict[1][0].predict(X))\n",
    "print(\"tree2 predictions on all elements:\", tree_dict[2][0].predict(X))\n",
    "\n",
    "### Showing Ent\n",
    "print(\"\\nEntropy of different splits for observations 5-9\")\n",
    "print(\"Col 1, @ 3.35:\", ent_from_split(X2.iloc[:,1].values,3.35, y2))\n",
    "print(\"Col 0, # 4.25:\", ent_from_split(X2.iloc[:,0].values, 4.25, y2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "Running the above cell a number of times, you might notice that the threshold and feature for `tree2` change.  \n",
    "\n",
    "At the bottom of the cell, the entropy for two different splits is shown to be identical. This is unlikely to happen with \"real\" data.  \n",
    "\n",
    "#### Bootstrapping\n",
    "\n",
    "Taking a bootstrap sample in adaptive boosting requires selecting observation with pre-defined probabilities.  \n",
    "\n",
    "\n",
    "Below offers an example of selecting random numbers with numpy given pre-defined probabilities.  \n",
    "\n",
    "This will be done with `np.random.choice()`, documentation below:\n",
    "![Boosting](Boosting.PNG)  \n",
    "\n",
    "Try running the below cell a few times, to gain a sense of how `.choice()` works while passing a value for the `<p>` argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "selecting from: [0, 1, 2, 3, 4] \n",
      "\n",
      "Equal Weights:\n",
      " [0 3 2 1 3]\n",
      "\n",
      "Weights of [.9,.05,.03,.02,0]:\n",
      " [0 3 0 0 0]\n"
     ]
    }
   ],
   "source": [
    "select_from = list(range(5))\n",
    "print(\"selecting from:\", select_from, \"\\n\")\n",
    "\n",
    "### Implement 1/n weights (which is the np.random.choice default)\n",
    "### Also note:\n",
    "### replace = True (default, used for boot-strapping)\n",
    "### size = len(array) - This will be the sample size used in our algorithms.\n",
    "\n",
    "print(\"Equal Weights:\\n\",\n",
    "    np.random.choice(select_from,\n",
    "                size = len(select_from),\n",
    "                replace = True,\n",
    "                p = np.array([.2,.2,.2,.2,.2,])\n",
    "                )\n",
    ")\n",
    "\n",
    "### Now, using uneven weights\n",
    "\n",
    "print(\"\\nWeights of [.9,.05,.03,.02,0]:\\n\",\n",
    "    np.random.choice(select_from,\n",
    "                size = len(select_from),\n",
    "                p = np.array([.9,.05,.03,.02,0]))\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "### Building Adaptive Boosting\n",
    "\n",
    "Below Gives the outline of the fitting process for the adaptive boosting algorithm.  \n",
    "\n",
    "Again, the functions next to \"`###<------`\" will be created in the exercises below. They include:  \n",
    "\n",
    "- `default_weights()`\n",
    "- `calc_epsilon()`\n",
    "- `calc_alpha()`\n",
    "- `update_weights()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "def simple_adaboost_fit(X,y, n_estimators):\n",
    "    \"\"\"\n",
    "    Positional arguments :\n",
    "        X -- a numpy array of numeric observations:\n",
    "            rows are observations, columns are features\n",
    "        y -- a numpy array of binary labels:\n",
    "            *Assume labels are 1 for \"True\" and 0 for \"False\"*\n",
    "        estimator -- a model capable of binary classification, implementing\n",
    "            the `.fit()` and `.predict()` methods.\n",
    "        n_estimators -- The number of estimators to fit.\n",
    "\n",
    "    Steps:\n",
    "        1. Create probability weights for selection during boot-straping.\n",
    "        2. Create boot-strap sample of observations according to weights\n",
    "        3. Fit estimator model with boot-strap sample.\n",
    "        4. Calculate model error: epsilon\n",
    "        5. Calculate alpha to associate with model\n",
    "        6. Re-calculate probability weights\n",
    "        7. Repeat 2-6 unil creation of n_estimators models. \n",
    "\n",
    "    \"\"\"\n",
    "    \n",
    "    def simple_tree():\n",
    "        return DecisionTreeClassifier(criterion = 'entropy', max_depth= 1)\n",
    "    \n",
    "    # Create default weights array where all are equal to 1/n\n",
    "    weights = default_weights(len(y)) ### <------\n",
    "    \n",
    "    est_dict = {}\n",
    "    for i in range(n_estimators):\n",
    "        # Create bootstrap sample\n",
    "        bs_X, bs_y = boot_strap_selection(X, y, weights)\n",
    "        \n",
    "        mod = simple_tree()\n",
    "        mod.fit(bs_X, bs_y)\n",
    "        \n",
    "        # Note: Predicting on all values of X, NOT boot-strap\n",
    "        preds = mod.predict(X)\n",
    "        \n",
    "        epsilon = calc_epsilon(y, preds, weights) ### <------\n",
    "        alpha = calc_alpha(epsilon) ### <------\n",
    "        \n",
    "        # Note that the i+1-th model will be keyed to the int i,\n",
    "        # and will store a tuple of the fit model and the alpha value\n",
    "        est_dict[i] = (mod, alpha)\n",
    "        \n",
    "        weights = update_weights(weights, alpha, y, preds) ### <------\n",
    "    \n",
    "    return est_dict "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "Creating vector of default weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Code a function called 'default_weights\n",
    "### ACCEPT a single integer, `n`,  as input\n",
    "### RETURN default weights; a numpy array of lenth n, where each value is equal to 1/n\n",
    "\n",
    "def default_weights(n):\n",
    "    \"\"\"\n",
    "    Create the default list of weights, a numpy array of length n\n",
    "    with each value equal to 1/n\n",
    "    \n",
    "    Example:\n",
    "        n = 10\n",
    "        dw = default_weights(n)\n",
    "        print(dw) #--> np.array([0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1])\n",
    "        \n",
    "    \"\"\"\n",
    "    return np.ones(n)*(1/n)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "#### boot_strap_selection  \n",
    "\n",
    "Below, the \"`boot_strap_selection`\" algorithm is provided. The function creates a boot-strap sample given the passed-in weights.  \n",
    "\n",
    "Example given"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(array([[4, 4],\n",
      "       [5, 5],\n",
      "       [4, 4],\n",
      "       [1, 1],\n",
      "       [1, 1]]), array([0, 1, 0, 1, 1]))\n"
     ]
    }
   ],
   "source": [
    "def boot_strap_selection(X, y, weights):\n",
    "    \"\"\"\n",
    "    Create and return a boot-strapped sample of the given data,\n",
    "    According to the provided weights.\n",
    "    \n",
    "    Positional Arguments:\n",
    "        X -- a numpy array, corresponding to the matrix of x-observations\n",
    "        y -- a numpy array, corresponding to a vector of y-labels\n",
    "            All either 0 or 1\n",
    "        weights -- a numpy array, corresponding to the rate at which the observations\n",
    "            should be sampled for the boot-strap. \n",
    "            \n",
    "    Example: \n",
    "    \n",
    "        X = np.array([[1,1],[2,2],[3,3],[4,4],[5,5]])\n",
    "        y = np.array([1,0,1,0,1])\n",
    "        weights = np.array([.35,.1,.1,.35,.1])\n",
    "        \n",
    "        print(boot_strap_selection(X,y, weights))\n",
    "        #-->(\n",
    "            np.array([[4, 4],\n",
    "                   [2, 2],\n",
    "                   [4, 4],\n",
    "                   [5, 5],\n",
    "                   [5, 5]]),\n",
    "            np.array([0, 0, 0, 1, 1]))\n",
    "        ### Actual results will vary\n",
    "    \"\"\"\n",
    "    \n",
    "    # Take random sample of indicies, with replacement\n",
    "    bss_indicies = np.random.choice(range(len(y)), size = len(y), p = weights)\n",
    "    \n",
    "    # Subset arrays with indicies\n",
    "    return X[bss_indicies,:], y[bss_indicies]\n",
    "\n",
    "### Example of use\n",
    "X = np.array([[1,1],[2,2],[3,3],[4,4],[5,5]])\n",
    "y = np.array([1,0,1,0,1])\n",
    "weights = np.array([.35,.1,.1,.35,.1])\n",
    "\n",
    "print(boot_strap_selection(X,y, weights))    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "#### Calculating Epsilon\n",
    "\n",
    "The Epsilon is equal to:  \n",
    "$$\\epsilon_t = \\sum_{i=1}^nw_t(i)\\mathbb{1}\\{y_i\\ne f_t(x_i)\\}$$  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Code a function called `calc_epsilon` \n",
    "### ACCEPT three inputs:\n",
    "### 1. The True labels\n",
    "### 2. The Predicted labels\n",
    "### 3. The current Weights\n",
    "\n",
    "### RETURN the epsilon value, calculated according to the above equation.\n",
    "### ### Will be a float between 0 and 1\n",
    "\n",
    "### The epsilon is the sum of the weights where the true-label DOES NOT EQUAL the predicted-label\n",
    "\n",
    "def calc_epsilon(y_true, y_pred, weights):\n",
    "    \n",
    "    \"\"\"\n",
    "    Calculate the value of epsilon, given the above equation \n",
    "    \n",
    "    Positional Arguments:\n",
    "        y_true -- An np.array of 1's and 0's corresponding to whether each observation is\n",
    "            a member of class 1 or class 2\n",
    "        y_pred -- An np.array of 1's and 0's corresponding to whether each observation was\n",
    "            predicted to be a member of class 1 or class 2\n",
    "        weights -- An np.array of floats corresponding to each observation's weight. \n",
    "            All the weights will sum up to 1.\n",
    "            \n",
    "    Example:\n",
    "        y_true = np.array([1,0,1,1,0])\n",
    "        y_pred = np.array([0,0,0,1,0])\n",
    "        weights = np.array([.4,.4,.1,.05,.05])\n",
    "        \n",
    "        ep = calc_epsilon(y_true, y_pred, weights)\n",
    "        \n",
    "        print(ep) # --> .5\n",
    "        \n",
    "    Assumptions:\n",
    "        Assume both the true labels and the predictions are both all 0's and 1's.\n",
    "    \"\"\"\n",
    "    # subset weights taking only values where y_true != y_pred\n",
    "    # then sum\n",
    "    return weights[y_true != y_pred].sum()\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "#### Calculating alpha.\n",
    "\n",
    "Alpha is equal to:\n",
    "$$\\alpha_t = \\frac12ln(\\frac{1-\\epsilon_t}{\\epsilon_t})$$  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Code a function called `calc_alpha`\n",
    "### ACCEPT a non-negative float (epsilon) as input\n",
    "### RETURN the alpha (float) calculated using the equation above.\n",
    "### HOWEVER, if epsilon equals 0, return np.inf\n",
    "\n",
    "def calc_alpha(epsilon):\n",
    "    \"\"\"\n",
    "    Calculate the alpha value given the epsilon observed from a model\n",
    "    \n",
    "    Positional Argument:\n",
    "        epsilon -- The epsilon value calculated from a particular model\n",
    "    Example:\n",
    "        ep = .4\n",
    "        alpha = calc_alpha(ep)\n",
    "        print(alpha) # --> 0.2027325540540821\n",
    "    \"\"\"\n",
    "    if epsilon == 0:\n",
    "        return np.inf\n",
    "    return .5 * np.log(((1-epsilon)/epsilon))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "#### Updating weights\n",
    "\n",
    "To update weights:\n",
    "$$w_{t+1}(i) = \\frac{\\hat{w}_{t+1}(i)}{\\sum_j\\hat{w}_{t+1}(j)}$$  \n",
    "\n",
    "Where:\n",
    "$$\\hat{w}_{t+1}(i) = w_t(i)e^{-\\alpha_ty_if_t(x_i)}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Code a function \"update_weights\"\n",
    "### ACCEPT four inputs:\n",
    "### 1. A numpy array of a weight vector\n",
    "### 2. An alpha value (float)\n",
    "### 3/4. numpy arrays of true labels and predicted labels vectors.\n",
    "\n",
    "### NB: Labels will need to be converted from 0s and 1s to -1s and 1s\n",
    "\n",
    "### RETURN an updated array of weights, according to equation above.\n",
    "\n",
    "def update_weights(weights, alpha, y_true, y_pred):\n",
    "    \"\"\"\n",
    "    Create an updated vector of weights according to the above equations\n",
    "    \n",
    "    Positional Arguments:\n",
    "        weights -- a 1-d numpy array of positive floats, corresponding to \n",
    "            observation weights\n",
    "        alpha -- a positive float\n",
    "        y_true -- a 1-d numpy array of true labels, all 0s and 1s\n",
    "        y_pred -- a 1-d numpy array of labels predicted by the last model;\n",
    "             all 0s and 1s. \n",
    "    \n",
    "    Example:\n",
    "        y_true = np.array([1,0,1,1,0])\n",
    "        y_pred = np.array([0,0,1,1,1])\n",
    "        weights = np.array([.4,.4,.1,.05,.05])\n",
    "        alpha = 0.10033534773107562\n",
    "        \n",
    "        print(update_weights(weights, alpha, y_true, y_pred))\n",
    "        #-->np.array([0.44444444 0.36363636 0.09090909 0.04545455 0.05555556])\n",
    "        \n",
    "    \"\"\"\n",
    "    # helper function to convert target of 1's and 0's to 1's and -1's\n",
    "    def target_for_weight_update(y):\n",
    "        toRet = y.copy()\n",
    "        toRet[toRet == 0] = -1\n",
    "        return toRet\n",
    "    \n",
    "    # convert predictions and actual labels to 1's and -1's\n",
    "    y = target_for_weight_update(y_true)\n",
    "    \n",
    "    pred = target_for_weight_update(y_pred)\n",
    "    #print(\"sol:\", y[:10], pred[:10])\n",
    "    # implement weight update equation\n",
    "    weights = weights * np.e**(-alpha*y*pred)\n",
    "    #print(\"sol:\",-alpha*y*pred)\n",
    "    # normalize weights by dividing by sum of all weights\n",
    "    weights = weights / sum(weights)\n",
    "    return weights"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "With the above functions created, the \"`simple_adaboost_fit()`\" method should work correctly.  \n",
    "\n",
    "`simple_adaboost_fit()` returns a dictionary where the keys are 0 through n-1 where n is the `n_estimators` from the function signature.  \n",
    "\n",
    "The values of the dictionaries are (model, alpha) where `model`is a `DecisionTreeClassifier`, and `alpha` is a float.  \n",
    "\n",
    "#### Creating a Prediction from boosted trees\n",
    "\n",
    "Our prediction will be:  \n",
    "\n",
    "$$f_{boost}(x_0) = sign(\\sum_{t=1}^T\\alpha_tf_t(X_0))$$  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Code a function called `predict`\n",
    "\n",
    "### ACCEPT two inputs:\n",
    "### 1. a 2-d numpy array of x-obervations\n",
    "### 2. a dictionary that contains classifiers and alphas (described more below and above)\n",
    "\n",
    "### Combine the models as in the manner described in the equation above\n",
    "### to create predictions for the observations.\n",
    "\n",
    "### RETURN a 1-d numpy array of observations (all 0s and 1s)\n",
    "\n",
    "def predict(X, est_dict):\n",
    "    \"\"\"\n",
    "    Create a np.array list of predictions for all of the observations in x,\n",
    "    according to the above equation.\n",
    "    \n",
    "    Positional Arguments:\n",
    "        X -- a 2-d numpy array of X observations. Features in columns, \n",
    "            observations in rows.\n",
    "        est_dict -- a dictionary consists of keys 0 through n with tuples as values\n",
    "            The tuples will be (<mod>, alpha), where alpha is a float, and \n",
    "            <mod> is a sklearn DecisionTreeClassifier\n",
    "    Example:\n",
    "    \n",
    "        ### Our example dataset, inspired from lecture\n",
    "        pts = [[.5, 3,1],[1,2,1],[3,.5,0],[2,3,0],[3,4,1],\n",
    "         [3.5,2.5,0],[3.6,4.7,1],[4,4.2,1],[4.5,2,0],[4.7,4.5,0]]\n",
    "\n",
    "        df = pd.DataFrame(pts, columns = ['x','y','classification'])\n",
    "        \n",
    "        ### split out X and labels\n",
    "        X = df[['x','y']]\n",
    "        y = df['classification']\n",
    "        ### Split data in half\n",
    "        X1 = X.iloc[:len(X.index)//2, :]\n",
    "        X2 = X.iloc[len(X.index)//2:, :]\n",
    "\n",
    "        y1 = y[:len(y)//2]\n",
    "        y2 = y[len(X)//2:]\n",
    "\n",
    "\n",
    "        ### Fit classifiers to both sets of data, save to dictionary:\n",
    "        \n",
    "        ### Tree-creator helper function\n",
    "        def simple_tree():\n",
    "            return DecisionTreeClassifier(criterion = 'entropy', max_depth= 1)\n",
    "            \n",
    "        tree_dict = {}\n",
    "\n",
    "        tree1 = simple_tree()\n",
    "        tree1.fit(X1,y1)\n",
    "        print(\"threshold:\", tree1.tree_.threshold[0], \"feature:\", tree1.tree_.feature[0])\n",
    "\n",
    "        ### made up alpha, for example\n",
    "        alpha1 = .6\n",
    "        tree_dict[1] = (tree1, alpha1)\n",
    "\n",
    "        tree2 = simple_tree()\n",
    "        tree2.fit(X2,y2)\n",
    "        print(\"threshold:\", tree2.tree_.threshold[0], \"feature:\" ,tree2.tree_.feature[0])\n",
    "        \n",
    "        ### made up alpha, again.\n",
    "        alpha2 = .35\n",
    "        tree_dict[2] = (tree2, alpha2)\n",
    "    \n",
    "        print(predict(X, tree_dict))\n",
    "        #--> np.array([1., 1., 0., 0., 0., 0., 0., 0., 0., 0.])\n",
    "        \n",
    "        ###############################\n",
    "        ### For Further Checking of your function:\n",
    "        ### The sum of predictions from the two models should be:\n",
    "        \n",
    "        # If tree2 splits on feature 1:\n",
    "        # np.array([ 0.25  0.25 -0.95 -0.95 -0.25 -0.95 -0.25 -0.25 -0.95 -0.25])\n",
    "        \n",
    "        # If tree2 splits on feature 0:\n",
    "        # np.array([ 0.95  0.95 -0.25 -0.25 -0.25 -0.25 -0.25 -0.25 -0.95 -0.95])\n",
    "        ###############################\n",
    "        \n",
    "    Assumptions:\n",
    "        The models in the `est-dict` tuple will return 0s and 1s.\n",
    "            HOWEVER, the prediction equation depends upon predictions\n",
    "            of -1s and 1s.\n",
    "            FINALLY, the returned predictions should be 0s and 1s.            \n",
    "    \"\"\"\n",
    "    fin_preds = np.zeros(X.shape[0])\n",
    "    for k in est_dict:\n",
    "        preds = est_dict[k][0].predict(X)\n",
    "        preds[preds<1] = -1\n",
    "        #print(preds, est_dict[k][1])\n",
    "        preds = preds *est_dict[k][1]\n",
    "        fin_preds += preds\n",
    "    #print(fin_preds)\n",
    "    fin_preds[fin_preds >= 0 ] = 1\n",
    "    fin_preds[fin_preds <0] = 0\n",
    "    \n",
    "    return fin_preds"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "### Census Data\n",
    "This project now on will use the [**`Census Income Data`**](https://archive.ics.uci.edu/ml/datasets/census+income) from the [UCI Machine Learning Repository](https://archive.ics.uci.edu/ml/index.html). A thorough description of the data and its features may be accessed either at the link above, or [this text file](https://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.names).  \n",
    "\n",
    "In particular, this classification attempts to predict whether or not a particular census respondant has an income of more or less than $50,000.  \n",
    "  \n",
    "#### The Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>fnlwgt</th>\n",
       "      <th>education</th>\n",
       "      <th>education-num</th>\n",
       "      <th>marital-status</th>\n",
       "      <th>occupation</th>\n",
       "      <th>relationship</th>\n",
       "      <th>race</th>\n",
       "      <th>sex</th>\n",
       "      <th>capital-gain</th>\n",
       "      <th>capital-loss</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>native-country</th>\n",
       "      <th>income</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>39</td>\n",
       "      <td>State-gov</td>\n",
       "      <td>77516</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Never-married</td>\n",
       "      <td>Adm-clerical</td>\n",
       "      <td>Not-in-family</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>2174</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>50</td>\n",
       "      <td>Self-emp-not-inc</td>\n",
       "      <td>83311</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Exec-managerial</td>\n",
       "      <td>Husband</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>38</td>\n",
       "      <td>Private</td>\n",
       "      <td>215646</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>9</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>Handlers-cleaners</td>\n",
       "      <td>Not-in-family</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>53</td>\n",
       "      <td>Private</td>\n",
       "      <td>234721</td>\n",
       "      <td>11th</td>\n",
       "      <td>7</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Handlers-cleaners</td>\n",
       "      <td>Husband</td>\n",
       "      <td>Black</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>28</td>\n",
       "      <td>Private</td>\n",
       "      <td>338409</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Prof-specialty</td>\n",
       "      <td>Wife</td>\n",
       "      <td>Black</td>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>Cuba</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age          workclass  fnlwgt   education  education-num  \\\n",
       "0   39          State-gov   77516   Bachelors             13   \n",
       "1   50   Self-emp-not-inc   83311   Bachelors             13   \n",
       "2   38            Private  215646     HS-grad              9   \n",
       "3   53            Private  234721        11th              7   \n",
       "4   28            Private  338409   Bachelors             13   \n",
       "\n",
       "        marital-status          occupation    relationship    race      sex  \\\n",
       "0        Never-married        Adm-clerical   Not-in-family   White     Male   \n",
       "1   Married-civ-spouse     Exec-managerial         Husband   White     Male   \n",
       "2             Divorced   Handlers-cleaners   Not-in-family   White     Male   \n",
       "3   Married-civ-spouse   Handlers-cleaners         Husband   Black     Male   \n",
       "4   Married-civ-spouse      Prof-specialty            Wife   Black   Female   \n",
       "\n",
       "   capital-gain  capital-loss  hours-per-week  native-country  income  \n",
       "0          2174             0              40   United-States   <=50K  \n",
       "1             0             0              13   United-States   <=50K  \n",
       "2             0             0              40   United-States   <=50K  \n",
       "3             0             0              40   United-States   <=50K  \n",
       "4             0             0              40            Cuba   <=50K  "
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "col_names = [\n",
    "\"age\", \"workclass\", \"fnlwgt\", \"education\",\n",
    "\"education-num\", \"marital-status\", \"occupation\", \"relationship\",\n",
    "\"race\", \"sex\", \"capital-gain\", \"capital-loss\", \"hours-per-week\",\n",
    "\"native-country\", \"income\"\n",
    "]\n",
    "\n",
    "data_path = \"adult.data.csv\"\n",
    "\n",
    "data = pd.read_csv(data_path, header = None, names = col_names)\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "Taking a subset of the data relevant to the prediction problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>education-num</th>\n",
       "      <th>occupation</th>\n",
       "      <th>sex</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>income</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>39</td>\n",
       "      <td>State-gov</td>\n",
       "      <td>13</td>\n",
       "      <td>Adm-clerical</td>\n",
       "      <td>Male</td>\n",
       "      <td>40</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>50</td>\n",
       "      <td>Self-emp-not-inc</td>\n",
       "      <td>13</td>\n",
       "      <td>Exec-managerial</td>\n",
       "      <td>Male</td>\n",
       "      <td>13</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>38</td>\n",
       "      <td>Private</td>\n",
       "      <td>9</td>\n",
       "      <td>Handlers-cleaners</td>\n",
       "      <td>Male</td>\n",
       "      <td>40</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>53</td>\n",
       "      <td>Private</td>\n",
       "      <td>7</td>\n",
       "      <td>Handlers-cleaners</td>\n",
       "      <td>Male</td>\n",
       "      <td>40</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>28</td>\n",
       "      <td>Private</td>\n",
       "      <td>13</td>\n",
       "      <td>Prof-specialty</td>\n",
       "      <td>Female</td>\n",
       "      <td>40</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age          workclass  education-num          occupation      sex  \\\n",
       "0   39          State-gov             13        Adm-clerical     Male   \n",
       "1   50   Self-emp-not-inc             13     Exec-managerial     Male   \n",
       "2   38            Private              9   Handlers-cleaners     Male   \n",
       "3   53            Private              7   Handlers-cleaners     Male   \n",
       "4   28            Private             13      Prof-specialty   Female   \n",
       "\n",
       "   hours-per-week  income  \n",
       "0              40   <=50K  \n",
       "1              13   <=50K  \n",
       "2              40   <=50K  \n",
       "3              40   <=50K  \n",
       "4              40   <=50K  "
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cols = [\"age\", \"workclass\", \"education-num\", \"occupation\", \"sex\", \"hours-per-week\", \"income\"]\n",
    "data = data[cols]\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "A quick look at the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>education-num</th>\n",
       "      <th>occupation</th>\n",
       "      <th>sex</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>income</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>32561.000000</td>\n",
       "      <td>32561</td>\n",
       "      <td>32561.000000</td>\n",
       "      <td>32561</td>\n",
       "      <td>32561</td>\n",
       "      <td>32561.000000</td>\n",
       "      <td>32561</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>unique</th>\n",
       "      <td>NaN</td>\n",
       "      <td>9</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>top</th>\n",
       "      <td>NaN</td>\n",
       "      <td>Private</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Prof-specialty</td>\n",
       "      <td>Male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>freq</th>\n",
       "      <td>NaN</td>\n",
       "      <td>22696</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4140</td>\n",
       "      <td>21790</td>\n",
       "      <td>NaN</td>\n",
       "      <td>24720</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>38.581647</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10.080679</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40.437456</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>13.640433</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.572720</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>12.347429</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>17.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>28.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>37.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>48.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>12.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>45.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>90.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>99.000000</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 age workclass  education-num       occupation    sex  \\\n",
       "count   32561.000000     32561   32561.000000            32561  32561   \n",
       "unique           NaN         9            NaN               15      2   \n",
       "top              NaN   Private            NaN   Prof-specialty   Male   \n",
       "freq             NaN     22696            NaN             4140  21790   \n",
       "mean       38.581647       NaN      10.080679              NaN    NaN   \n",
       "std        13.640433       NaN       2.572720              NaN    NaN   \n",
       "min        17.000000       NaN       1.000000              NaN    NaN   \n",
       "25%        28.000000       NaN       9.000000              NaN    NaN   \n",
       "50%        37.000000       NaN      10.000000              NaN    NaN   \n",
       "75%        48.000000       NaN      12.000000              NaN    NaN   \n",
       "max        90.000000       NaN      16.000000              NaN    NaN   \n",
       "\n",
       "        hours-per-week  income  \n",
       "count     32561.000000   32561  \n",
       "unique             NaN       2  \n",
       "top                NaN   <=50K  \n",
       "freq               NaN   24720  \n",
       "mean         40.437456     NaN  \n",
       "std          12.347429     NaN  \n",
       "min           1.000000     NaN  \n",
       "25%          40.000000     NaN  \n",
       "50%          40.000000     NaN  \n",
       "75%          45.000000     NaN  \n",
       "max          99.000000     NaN  "
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.describe(include = 'all')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    },
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlkAAAJRCAYAAAB2q6IqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGwlJREFUeJzt3X+wZ3dd3/HXm6ykBDXBJFJIUhZL/NGqIO4gymiR0A5hHaMOFJ1WKRMm7RQqLe3IajtDO9N21pkq4ExrjcQO/qhIUEvqUpRBoVMHkOWHIAlCgDWJSXApMSiiGHj3j3u27uZe2JvNfd/v/fF4zOzc7/ec8733c8/c793n/ZzzPd/q7gAAsLUesuoBAADsRSILAGCAyAIAGCCyAAAGiCwAgAEiCwBggMgCABggsgAABogsAIABIgvYlarqSFV9uKr+pKpurqrvXpafV1U/VlUfr6qPVtULq6qr6sCy/sKquqGq7qqqP6yqf19V5632uwH2ogOrHgDAOfpwkm9NcneSZyf5+ap6XJJrklyd5AlJPpXkxvs97lVJPpbkcUkenuTXktye5Ke2Z9jAflHeuxDYC6rqPUlemuRFSX6pu39qWf70JG9M8kVJLk5yW5KLuvvTy/rvS3Jdd3/7SgYO7FlmsoBdqap+IMmLkxxcFn1xkkuSPDprM1OnnH77MVmLrbuq6tSyh9xvG4AtIbKAXaeqHpPkp5NcleSt3f3ZZSarktyV5PLTNr/itNu3J/mLJJd0933bNV5gf3LiO7AbPTxJJzmZJFX1vCRfu6x7TZIXVdVlVXVRkpecelB335XkN5L8WFV9aVU9pKr+ZlX9ne0dPrAfiCxg1+num5P8WJK3Zu0k9q9L8tvL6p/OWki9N8m7k7w+yX1JPrus/4EkD01yc5J7krw2yaO2a+zA/uHEd2BPq6qrk/zX7n7MqscC7C9msoA9paoeVlXPrKoDVXVZ1l5x+KurHhew/5jJAvaUqrogyVuSfHWSTyc5luRF3f3JlQ4M2HdEFgDAAIcLAQAGiCwAgAE74mKkl1xySR88eHDVwwAAOKt3vvOdH+/uS8+23Y6IrIMHD+b48eOrHgYAwFlV1R9sZjuHCwEABogsAIABIgsAYIDIAgAYILIAAAaILACAASILAGCAyAIAGCCyAAAGiCwAgAEiCwBggMgCABggsgAABogsAIABIgsAYIDIAgAYILIAAAaILACAASILAGCAyAIAGCCyAAAGiCwAgAEiCwBgwIFVDwC228EjxzZcfuLo4W0eCQB7mZksAIABIgsAYIDIAgAYILIAAAaILACAAV5dyLbZ6FV9XtEHwF4lsti1XIoBgJ3M4UIAgAEiCwBggMgCABggsgAABogsAIABXl24w7jMwdbwykMAVk1ksaOJJQB2K4cLAQAGiCwAgAEiCwBggHOy2JBzoQDgwTGTBQAwQGQBAAwQWQAAA5yTxZZyMVUAWGMmCwBggJksOI2ZOAC2ipksAIABIgsAYIDIAgAYILIAAAaILACAASILAGCAyAIAGCCyAAAGuBjpHrDRBTQTF9EEgFUykwUAMEBkAQAMEFkAAAOck7WPOZcLAOaYyQIAGCCyAAAGiCwAgAHOyWJH2Oj8sJ12bthuGCMAO4eZLACAASILAGCAyAIAGCCyAAAGiCwAgAEiCwBggEs4wBZweQcA7k9k8YAJCgA4O4cLAQAGiCwAgAEOF8KKbHTYNXHoFWCvMJMFADBgU5FVVf+iqt5fVb9XVb9YVX+tqh5bVW+vqg9V1S9V1UOXbc9f7t+6rD84+Q0AAOxEZ42sqrosyQ8mOdTdX5vkvCTfm+RHk7ysu69Mck+Sa5eHXJvknu5+XJKXLdsBAOwrmz1ceCDJw6rqQJILktyV5GlJXrusf1WS71puX7Pcz7L+qqqqrRkuAMDucNbI6u4/TPKfktyWtbi6N8k7k/xxd9+3bHZHksuW25cluX157H3L9hff//NW1XVVdbyqjp88efLBfh8AADvKWV9dWFWPyNrs1GOT/HGSG5NcvcGmfeohX2DdXy3ovj7J9Uly6NChdetZz0VAAWD32Mzhwqcn+Wh3n+zuv0zyK0m+JclFy+HDJLk8yZ3L7TuSXJEky/oLk3xiS0cNALDDbeY6WbcleXJVXZDk00muSnI8yW8leVaSVyd5bpLXLdvftNx/67L+N7vbTBX7lhlIgP1pM+dkvT1rJ7C/K8n7lsdcn+QlSV5cVbdm7ZyrG5aH3JDk4mX5i5McGRg3AMCOtqkrvnf3S5O89H6LP5LkSRts++dJnv3ghwYAsHu54jsAwACRBQAwQGQBAAzY1DlZ7F4bvbIt8eo2AJhmJgsAYICZLNiBzEAC7H5msgAABogsAIABIgsAYIDIAgAYILIAAAZ4dSHsMl55CLA7mMkCABggsgAABjhcuAIbHe5xqAcA9hYzWQAAA0QWAMAAkQUAMEBkAQAMEFkAAANEFgDAAJdweBBceRsA+HzMZAEADBBZAAADRBYAwACRBQAwQGQBAAwQWQAAA0QWAMAAkQUAMMDFSIENL6zroroAD46ZLACAAWayYJ8wWwWwvcxkAQAMEFkAAANEFgDAAJEFADBAZAEADPDqQthDNnoFYeJVhACrYCYLAGCAmawhrkkEAPubmSwAgAEiCwBggMgCABggsgAABjjx/Sy8JB4AOBdmsgAABpjJAr4glyMBODdmsgAABogsAIABIgsAYIDIAgAYILIAAAaILACAASILAGCAyAIAGCCyAAAGiCwAgAEiCwBggMgCABggsgAABogsAIABIgsAYIDIAgAYILIAAAaILACAASILAGCAyAIAGCCyAAAGiCwAgAEiCwBggMgCABggsgAABogsAIABIgsAYIDIAgAYILIAAAaILACAASILAGCAyAIAGCCyAAAGiCwAgAEiCwBggMgCABggsgAABogsAIABIgsAYIDIAgAYILIAAAaILACAASILAGCAyAIAGCCyAAAGiCwAgAEiCwBggMgCABggsgAABogsAIABIgsAYIDIAgAYILIAAAaILACAASILAGCAyAIAGCCyAAAGiCwAgAGbiqyquqiqXltVH6iqW6rqm6vqy6rqjVX1oeXjI5Ztq6p+oqpurar3VtUTZ78FAICdZ7MzWa9I8obu/uokj09yS5IjSd7U3VcmedNyP0muTnLl8u+6JD+5pSMGANgFDpxtg6r60iTfluQfJUl3fybJZ6rqmiRPXTZ7VZI3J3lJkmuS/Gx3d5K3LbNgj+ruu7Z89MCuc/DIsQ2Xnzh6eJtHAjBrMzNZX5HkZJL/VlXvrqpXVtXDkzzyVDgtH7982f6yJLef9vg7lmVnqKrrqup4VR0/efLkg/omAAB2ms1E1oEkT0zyk939DUk+lb86NLiR2mBZr1vQfX13H+ruQ5deeummBgsAsFtsJrLuSHJHd799uf/arEXXx6rqUUmyfPyj07a/4rTHX57kzq0ZLgDA7nDWc7K6++6qur2qvqq7fz/JVUluXv49N8nR5ePrlofclOSFVfXqJN+U5F7nYwGb4XwtYC85a2Qt/lmSX6iqhyb5SJLnZW0W7DVVdW2S25I8e9n29UmemeTWJH+2bAsAsK9sKrK6+z1JDm2w6qoNtu0kL3iQ4wIA2NVc8R0AYIDIAgAYILIAAAaILACAASILAGCAyAIAGCCyAAAGiCwAgAGbveI7wKZ5exwAM1kAACNEFgDAAJEFADBAZAEADBBZAAADRBYAwACRBQAwQGQBAAwQWQAAA0QWAMAAkQUAMEBkAQAMEFkAAANEFgDAgAOrHsBOcPDIsQ2Xnzh6eJtHAgDsFWayAAAGiCwAgAEiCwBggHOygHO20fmMzmUEWGMmCwBggMgCABggsgAABogsAIABIgsAYIDIAgAYILIAAAaILACAASILAGCAyAIAGCCyAAAGiCwAgAEiCwBggMgCABggsgAABogsAIABIgsAYIDIAgAYILIAAAaILACAASILAGCAyAIAGCCyAAAGiCwAgAEiCwBggMgCABggsgAABogsAIABIgsAYIDIAgAYILIAAAaILACAASILAGCAyAIAGCCyAAAGiCwAgAEiCwBgwIFVDwBgMw4eObbh8hNHD2/zSAA2x0wWAMAAkQUAMEBkAQAMEFkAAANEFgDAAJEFADBAZAEADHCdLGDX2+gaWq6fBayamSwAgAEiCwBggMgCABggsgAABogsAIABIgsAYIDIAgAYILIAAAaILACAASILAGCAyAIAGCCyAAAGiCwAgAEiCwBggMgCABggsgAABogsAIABIgsAYIDIAgAYILIAAAaILACAASILAGCAyAIAGCCyAAAGiCwAgAEHVj0AgFU5eOTYumUnjh5ewUiAvchMFgDAADNZwJ5mtgpYFTNZAAADRBYAwACRBQAwQGQBAAzYdGRV1XlV9e6q+rXl/mOr6u1V9aGq+qWqeuiy/Pzl/q3L+oMzQwcA2LkeyEzWi5Lcctr9H03ysu6+Msk9Sa5dll+b5J7uflySly3bAQDsK5uKrKq6PMnhJK9c7leSpyV57bLJq5J813L7muV+lvVXLdsDAOwbm53JenmSH0ryueX+xUn+uLvvW+7fkeSy5fZlSW5PkmX9vcv2AAD7xlkjq6q+I8kfdfc7T1+8waa9iXWnf97rqup4VR0/efLkpgYLALBbbGYm6ylJvrOqTiR5ddYOE748yUVVdeqK8ZcnuXO5fUeSK5JkWX9hkk/c/5N29/Xdfai7D1166aUP6psAANhpzvq2Ot39w0l+OEmq6qlJ/lV3/4OqujHJs7IWXs9N8rrlITct99+6rP/N7l43k7XdNnprjcTbawAAMx7MdbJekuTFVXVr1s65umFZfkOSi5flL05y5MENEQBg93lAbxDd3W9O8ubl9keSPGmDbf48ybO3YGwAALuWK74DAAwQWQAAA0QWAMAAkQUAMEBkAQAMEFkAAANEFgDAAJEFADBAZAEADBBZAAADRBYAwACRBQAwQGQBAAwQWQAAAw6segAAO9HBI8fWLTtx9PADfsxmHgfsTWayAAAGiCwAgAEiCwBggMgCABjgxHeAB8DJ7cBmmckCABhgJgtgG5gBg/3HTBYAwACRBQAwQGQBAAwQWQAAA0QWAMAAkQUAMEBkAQAMEFkAAANEFgDAAJEFADBAZAEADBBZAAADRBYAwACRBQAwQGQBAAwQWQAAA0QWAMAAkQUAMEBkAQAMEFkAAANEFgDAAJEFADBAZAEADDiw6gEA7HcHjxzbcPmJo4e3eSTAVjKTBQAwQGQBAAwQWQAAA0QWAMAAkQUAMEBkAQAMcAkHgB1so8s7uLQD7A5msgAABogsAIABIgsAYIDIAgAYILIAAAaILACAASILAGCAyAIAGCCyAAAGiCwAgAEiCwBggMgCABggsgAABogsAIABIgsAYIDIAgAYILIAAAaILACAASILAGCAyAIAGCCyAAAGiCwAgAEiCwBgwIFVDwCAc3PwyLF1y04cPbyCkQAbMZMFADBAZAEADBBZAAADRBYAwACRBQAwQGQBAAwQWQAAA0QWAMAAkQUAMEBkAQAMEFkAAANEFgDAAJEFADBAZAEADBBZAAADRBYAwIADqx4AAFvv4JFj65adOHp4BSOB/ctMFgDAAJEFADBAZAEADBBZAAADRBYAwACRBQAwQGQBAAwQWQAAA0QWAMAAkQUAMEBkAQAM8N6FAPvIRu9pmHhfQ5hw1pmsqrqiqn6rqm6pqvdX1YuW5V9WVW+sqg8tHx+xLK+q+omqurWq3ltVT5z+JgAAdprNHC68L8m/7O6vSfLkJC+oqr+V5EiSN3X3lUnetNxPkquTXLn8uy7JT275qAEAdrizRlZ339Xd71pu/0mSW5JcluSaJK9aNntVku9abl+T5Gd7zduSXFRVj9rykQMA7GAP6MT3qjqY5BuSvD3JI7v7rmQtxJJ8+bLZZUluP+1hdyzLAAD2jU1HVlV9cZJfTvLPu/uTX2jTDZb1Bp/vuqo6XlXHT548udlhAADsCpuKrKr6oqwF1i90968siz926jDg8vGPluV3JLnitIdfnuTO+3/O7r6+uw9196FLL730XMcPALAjbebVhZXkhiS3dPePn7bqpiTPXW4/N8nrTlv+A8urDJ+c5N5ThxUBAPaLzVwn6ylJvj/J+6rqPcuyH0lyNMlrquraJLclefay7vVJnpnk1iR/luR5WzpiAIBd4KyR1d3/JxufZ5UkV22wfSd5wYMcFwDAruZtdQAABogsAIABIgsAYIDIAgAYILIAAAaILACAASILAGCAyAIAGLCZK74DsA8cPHJsw+Unjh7e5pHA3mAmCwBggMgCABggsgAABogsAIABIgsAYIDIAgAYILIAAAaILACAASILAGCAyAIAGCCyAAAGiCwAgAEiCwBggMgCABhwYNUDAGB3O3jk2IbLTxw9vM0jgZ1FZAFwVkIKHjiHCwEABogsAIABIgsAYIDIAgAYILIAAAaILACAASILAGCAyAIAGCCyAAAGiCwAgAEiCwBggMgCABggsgAABogsAIABIgsAYIDIAgAYcGDVAwBg7zp45Ni6ZSeOHl7BSGD7mckCABggsgAABogsAIABIgsAYIDIAgAYILIAAAa4hAMAK+HyDux1ZrIAAAaILACAASILAGCAyAIAGCCyAAAGiCwAgAEiCwBggMgCABggsgAABogsAIAB3lYHgB3HW+6wF5jJAgAYILIAAAaILACAASILAGCAyAIAGCCyAAAGiCwAgAEiCwBggMgCABjgiu8A7CquBs9uYSYLAGCAyAIAGOBwIQB7wkaHEROHElkdkQXAnifAWAWHCwEABogsAIABIgsAYIDIAgAYILIAAAaILACAASILAGCA62QBsK+5hhZTzGQBAAwQWQAAAxwuBIDP4wsdStxonUOMnE5kAQC7zm44l87hQgCAASILAGCAyAIAGCCyAAAGiCwAgAEiCwBggMgCABjgOlkAsMVcqJTETBYAwAiRBQAwQGQBAAxwThYAbKMvdL7W51u3G96nj/XMZAEADDCTBQC7mFmunctMFgDAAJEFADBAZAEADHBOFgDsUc7XWi2RBQCcwdsCbQ2HCwEABogsAIABI4cLq+oZSV6R5Lwkr+zuoxNfBwA4N87XmrflkVVV5yX5z0n+bpI7kryjqm7q7pu3+msBANtrq8/X2suxN3G48ElJbu3uj3T3Z5K8Osk1A18HAGDHqu7e2k9Y9awkz+ju5y/3vz/JN3X3C++33XVJrlvuflWS39/SgWy/S5J8fNWD2GHskzPZH+vZJ+vZJ2eyP9azT9bb7n3ymO6+9GwbTZyTVRssW1dy3X19kusHvv5KVNXx7j606nHsJPbJmeyP9eyT9eyTM9kf69kn6+3UfTJxuPCOJFecdv/yJHcOfB0AgB1rIrLekeTKqnpsVT00yfcmuWng6wAA7Fhbfriwu++rqhcm+fWsXcLhZ7r7/Vv9dXagPXPocwvZJ2eyP9azT9azT85kf6xnn6y3I/fJlp/4DgCAK74DAIwQWQAAA0QWAMAAkQUAMEBknYOqurCqjlbVB6rq/y7/blmWXbTq8W23qjpQVf+4qt5QVe+tqt+tqv9VVf+kqr5o1eNbBftkPc+b9eyTM9kf6/ldst5u+jkRWefmNUnuSfLU7r64uy9O8u3LshtXOrLV+LkkT0jyb5M8M8nhJP8uyeOT/PzqhrVS9sl6njfr2Sdnsj/W87tkvV3zc+ISDuegqn6/u7/qga7bq86yPz7Y3V+53WNaNftkPc+b9eyTM9kf6/ldst5u+jkxk3Vu/qCqfqiqHnlqQVU9sqpekuT2FY5rVe6pqmdX1f//eaqqh1TVc7L2l8V+ZJ+s53mznn1yJvtjPb9L1ts1Pyci69w8J8nFSd5SVfdU1SeSvDnJlyX5+6sc2Ip8b5JnJbm7qj5YVR9McneS71nW7Uen9snHln3yodgnnjfr2Sdnsj/W8/t1vVM/J2+uqk/s5J8ThwvPUVV9ddbe/Ppt3f2npy1/Rne/YXUjW42q+qYkneTDSb4myZOT3Nzdr1/pwHaAqro4SSV5eXf/w1WPZ6eoqm9N8qQk7+vu31j1eFZhed58oLvvraoLkhxJ8sQk70/yH7v73pUOcJtV1Q8m+dXu3lGzEatUa+8B/H1J7kzyriRXJ/mWrP2MXN/df7nC4a1MVT0uyXcnuSLJfUk+mOQXd9pzRmSdg+UXwQuS3JK1ExJf1N2vW9a9q7ufuMrxbbeqemnWnvgHkrwxa/9xviXJ05P8enf/hxUObyWqaqM3RX9akt9Mku7+zu0d0epV1e9095OW28/P2nPofyT5e0n+Z3cfXeX4VqGq3p/k8ct7vl6f5FNJfjnJVcvy71npALdZVd2btX3w4ST/PcmN3f3x1Y5qtarqF7L2u/VhSe5N8vAkv5q1n5Hq7ueucHgrsfwf/B1J/nfWXgzwnqwdOv3uJP+0u9+8utGdSWSdg6p6X5Jv7u4/raqDSV6b5Oe6+xVV9e7u/oaVDnCbLfvjCUnOz9o09uXd/cmqeliSt3f31690gCtQVe9KcnOSV2Zthq+S/GKW6f3ufsvqRrcapz83quodSZ7Z3Ser6uFZmxH+utWOcPtV1S3d/TXL7TP+QKuq93T3E1Y3uu1XVe9O8o1Z+wPtOUm+M8k7s/bc+ZXu/pMVDm8lquq93f31VXUgyR8meXR3f7aqKsnv7tPfr+9L8oRlP1yQ5PXd/dSq+htJXreT/g92Tta5Oe/UIcLuPpHkqUmurqofz9p/pvvNfd392e7+syQf7u5PJkl3fzrJ51Y7tJU5lLX/HP51knuXv6w+3d1v2Y+BtXhIVT3i1OHT7j6ZJN39qaxN9+9Hv1dVz1tu/25VHUqSqvrKJPvxMFB39+e6+ze6+9okj07yX5I8I8lHVju0lXnIcsjwS5JckOTCZfn5SfbldbIWB5aP52dt36S7b8sO2ycHzr4JG7i7qp7Q3e9JkmVG6zuS/EySfffXeJLPVNUFS2R946mFVXVh9mlkdffnkrysqm5cPn4snm8XZi08K0lX1V/v7rur6ouzP/84SZLnJ3lFVf2bJB9P8taquj1rr5B6/kpHthpn/Bws5xvdlOSmZWZ8P7ohyQeSnJe1P9purKqPZO2811evcmAr9Mok76iqtyX5tiQ/miRVdWmST6xyYPfncOE5qKrLszZ7c/cG657S3b+9gmGtTFWd391/scHyS5I8qrvft4Jh7ShVdTjJU7r7R1Y9lp1mme5/ZHd/dNVjWZWq+pIkX5G1EL+juz+24iGtRFV9ZXd/cNXj2Gmq6tFJ0t131toVzZ+e5Lbu/p3Vjmx1qupvZ+1FVr/X3R9Y9Xg+H5EFADDAOVkAAANEFgDAAJEFADBAZAEADBBZAAAD/h+Qsd9scOpiQgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAFUCAYAAADPtPD/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xm4XFWZ7/Hvj0lRCAQIXuaABhEQokSIoIhNN4PIJNJAtxJxiLYgeNW2waFBaFtsRVr6CgptJKAytSKRwUBHCPOQQAizxCgQghBkiiLI8N4/1iqyc3adnJOTZK9d5Pd5nnqq9qpdtd+ck1NvrVkRgZmZWdUKpQMwM7P2cXIwM7MaJwczM6txcjAzsxonBzMzq3FyMDOzGicHsyGQdJWkjy/ma3aRNGdZxWS2NDk5mJlZzUqlAzDrJZIEqHQcZsuaaw72qibpMEm/rBzPknR+5fghSaMl7SjpFklP5/sdK+dcJenrkq4DngU263ON9STNlPSFfLyWpB9JmivpSUm/6Ce2oyX9VtJ8SXdL2r/y3JskTc3xPC7pvFwuSSdLeiw/N1PS1kvr52XW4ZqDvdpNBU6WtALwBmBlYCcASZsBqwEPAr8FjgTOAQ4ELpH0poj4Y36fDwN7AvdRqTlIGglcDnw7Ik7PxWcDfwK2yvevJJo+fgu8G/hDvuaP8zUfAU7I7/teYBVgTH7NbsDOwObA08AWwFOL/2MxWzTXHOxVLSJmA/OB0cB7gMnAw5K2yMfXAHsB90fE2RHxYkScA9wL7F15qzMj4q78/Au5bEvgKuDYTmKQtB4piXwqIp6MiBciYmo/sV0QEXMj4uWIOA+4H9g+P/0CsAmwfkQ8FxHXVspXJyUFRcQ9OZmYLVVODrY8mArsQvrGPZX0gf6efJsKrA880Oc1DwAbVI4f6vK+/wg8DPxPpWwj4ImIeHKgoCQdKmmGpKckPQVsDayTn/4iqYZys6S7JH0UICJ+Dfw/4HvAo5JOlzRsoGuZLS4nB1sedJLDu/PjqSycHOaSvqVXbUz64O/otnzxccDjwE8lrZjLHgLWkrTmogKStAlwBnAEsHZErAncSW6yiog/RMQnImJ94JPAqZLelJ87JSK2IzVbbQ788wD/frPF5uRgy4OppLb7VSNiDqkpaQ9gbeA24FJgc0n/IGklSQeRmowuHuB9XyD1FbweOFvSCrmJ5zLSh/lwSStL2rnLa19PSjjzIHWck2oO5OMDJW2YD5/M574k6R2SdpC0MvBn4DngpcX9gZgNxMnBXvUi4jekjuFr8vEzwGzguoh4KXc6vx/4PPBHUpPO+yPi8UG891+BDwDrAhNyx/eHSYnjXuAx4LNdXnc3cBJwA/Ao8Fbgusop7wBukvQnYBJwVET8DhhGqnE8SWr6+iPw7cX5eZgNhrzZj5mZ9eWag5mZ1Tg5mJlZjZODmZnVODmYmVlNzy6fsc4668TIkSNLh2Fm1lOmT5/+eESMGOi8nk0OI0eOZNq0aaXDMDPrKZL6rgbQlZuVzMysxsnBzMxqnBzMzKzGycHMzGqcHMzMrMbJwczMapwczMysxsnBzMxqnBzMzKymZ2dImy1rI4++pNHr/f7EvRq9ntmiuOZgZmY1Tg5mZlbj5GBmZjVODmZmVuPkYGZmNU4OZmZW4+RgZmY1Tg5mZlbj5GBmZjVODmZmVuPkYGZmNU4OZmZW4+RgZmY1Tg5mZlbj5GBmZjVODmZmVuPkYGZmNU4OZmZW4+RgZmY1Tg5mZlYzYHKQtJGkKyXdI+kuSUfl8rUkXSHp/nw/PJdL0imSZkmaKentlfcal8+/X9K4Svl2ku7IrzlFkpbFP9bMzAZnMDWHF4HPR8RbgLHA4ZK2BI4GpkTEKGBKPgbYExiVb+OB0yAlE+BYYAdge+DYTkLJ54yvvG6PJf+nmZnZUA2YHCLikYi4NT+eD9wDbADsC0zMp00E9suP9wXOiuRGYE1J6wG7A1dExBMR8SRwBbBHfm5YRNwQEQGcVXkvMzMrYLH6HCSNBN4G3AS8ISIegZRAgHXzaRsAD1VeNieXLap8TpfybtcfL2mapGnz5s1bnNDNzGwxDDo5SFoN+Bnw2Yh4ZlGndimLIZTXCyNOj4gxETFmxIgRA4VsZmZDNKjkIGllUmL4SUT8PBc/mpuEyPeP5fI5wEaVl28IzB2gfMMu5WZmVshgRisJ+CFwT0R8p/LUJKAz4mgccFGl/NA8amks8HRudpoM7CZpeO6I3g2YnJ+bL2lsvtahlfcyM7MCVhrEOTsBHwbukDQjl30JOBE4X9LHgAeBA/NzlwLvA2YBzwKHAUTEE5JOAG7J5x0fEU/kx/8EnAmsClyWb2ZmVsiAySEirqV7vwDArl3OD+Dwft5rAjChS/k0YOuBYjEzs2Z4hrSZmdU4OZiZWY2Tg5mZ1Tg5mJlZjZODmZnVODmYmVmNk4OZmdU4OZiZWY2Tg5mZ1Tg5mJlZjZODmZnVODmYmVmNk4OZmdU4OZiZWY2Tg5mZ1Tg5mJlZjZODmZnVODmYmVmNk4OZmdU4OZiZWY2Tg5mZ1Tg5mJlZjZODmZnVODmYmVmNk4OZmdU4OZiZWY2Tg5mZ1Tg5mJlZjZODmZnVODmYmVmNk4OZmdU4OZiZWY2Tg5mZ1Tg5mJlZjZODmZnVODmYmVmNk4OZmdUMmBwkTZD0mKQ7K2XHSXpY0ox8e1/luWMkzZJ0n6TdK+V75LJZko6ulG8q6SZJ90s6T9IqS/MfaGZmi28wNYczgT26lJ8cEaPz7VIASVsCBwNb5decKmlFSSsC3wP2BLYEDsnnAnwzv9co4EngY0vyDzIzsyU3YHKIiKuBJwb5fvsC50bE8xHxO2AWsH2+zYqI2RHxV+BcYF9JAv4G+J/8+onAfov5bzAzs6VsSfocjpA0Mzc7Dc9lGwAPVc6Zk8v6K18beCoiXuxT3pWk8ZKmSZo2b968JQjdzMwWZajJ4TTgjcBo4BHgpFyuLufGEMq7iojTI2JMRIwZMWLE4kVsZmaDttJQXhQRj3YeSzoDuDgfzgE2qpy6ITA3P+5W/jiwpqSVcu2her6ZmRUypJqDpPUqh/sDnZFMk4CDJb1G0qbAKOBm4BZgVB6ZtAqp03pSRARwJfDB/PpxwEVDicnMzJaeAWsOks4BdgHWkTQHOBbYRdJoUhPQ74FPAkTEXZLOB+4GXgQOj4iX8vscAUwGVgQmRMRd+RL/Apwr6d+A24AfLrV/nZmZDcmAySEiDulS3O8HeER8Hfh6l/JLgUu7lM8mjWYyM7OW8AxpMzOrcXIwM7MaJwczM6txcjAzsxonBzMzq3FyMDOzGicHMzOrcXIwM7MaJwczM6txcjAzsxonBzMzq3FyMDOzGicHMzOrcXIwM7MaJwczM6txcjAzsxonBzMzq3FyMDOzGicHMzOrcXIwM7MaJwczM6txcjAzsxonBzMzq3FyMDOzGicHMzOrcXIwM7MaJwczM6txcjAzsxonBzMzq3FyMDOzGicHMzOrcXIwM7MaJwczM6txcjAzsxonBzMzq3FyMDOzmgGTg6QJkh6TdGelbC1JV0i6P98Pz+WSdIqkWZJmSnp75TXj8vn3SxpXKd9O0h35NadI0tL+R5qZ2eIZTM3hTGCPPmVHA1MiYhQwJR8D7AmMyrfxwGmQkglwLLADsD1wbCeh5HPGV17X91pmZtawAZNDRFwNPNGneF9gYn48EdivUn5WJDcCa0paD9gduCIinoiIJ4ErgD3yc8Mi4oaICOCsynuZmVkhQ+1zeENEPAKQ79fN5RsAD1XOm5PLFlU+p0u5mZkVtLQ7pLv1F8QQyru/uTRe0jRJ0+bNmzfEEM3MbCBDTQ6P5iYh8v1juXwOsFHlvA2BuQOUb9ilvKuIOD0ixkTEmBEjRgwxdDMzG8hQk8MkoDPiaBxwUaX80DxqaSzwdG52mgzsJml47ojeDZicn5svaWwepXRo5b3MzKyQlQY6QdI5wC7AOpLmkEYdnQicL+ljwIPAgfn0S4H3AbOAZ4HDACLiCUknALfk846PiE4n9z+RRkStClyWb2ZmVtCAySEiDunnqV27nBvA4f28zwRgQpfyacDWA8VhZmbN8QxpMzOrcXIwM7MaJwczM6txcjAzsxonBzMzq3FyMDOzGicHMzOrcXIwM7MaJwczM6txcjAzsxonBzMzq3FyMDOzGicHMzOrcXIwM7MaJwczM6txcjAzsxonBzMzq3FyMDOzGicHMzOrcXIwM7MaJwczM6txcjAzsxonBzMzq3FyMDOzGicHMzOrcXIwM7MaJwczM6txcjAzsxonBzMzq3FyMDOzGicHMzOrcXIwM7MaJwczM6txcjAzsxonBzMzq3FyMDOzGicHMzOrWaLkIOn3ku6QNEPStFy2lqQrJN2f74fnckk6RdIsSTMlvb3yPuPy+fdLGrdk/yQzM1tSS6Pm8N6IGB0RY/Lx0cCUiBgFTMnHAHsCo/JtPHAapGQCHAvsAGwPHNtJKGZmVsayaFbaF5iYH08E9quUnxXJjcCaktYDdgeuiIgnIuJJ4Apgj2UQl5mZDdKSJocALpc0XdL4XPaGiHgEIN+vm8s3AB6qvHZOLuuvvEbSeEnTJE2bN2/eEoZuZmb9WWkJX79TRMyVtC5whaR7F3GuupTFIsrrhRGnA6cDjBkzpus5Zma25Jao5hARc/P9Y8CFpD6DR3NzEfn+sXz6HGCjyss3BOYuotzMzAoZcnKQ9HpJq3ceA7sBdwKTgM6Io3HARfnxJODQPGppLPB0bnaaDOwmaXjuiN4tl5mZWSFL0qz0BuBCSZ33+WlE/ErSLcD5kj4GPAgcmM+/FHgfMAt4FjgMICKekHQCcEs+7/iIeGIJ4jIzsyU05OQQEbOBbbuU/xHYtUt5AIf3814TgAlDjcXMzJYuz5A2M7MaJwczM6txcjAzsxonBzMzq3FyMDOzGicHMzOrcXIwM7MaJwczM6txcjAzsxonBzMzq3FyMDOzGicHMzOrcXIwM7MaJwczM6txcjAzsxonBzMzq3FyMDOzmiXZJtTMljMjj76k0ev9/sS9Gr2eLeCag5mZ1Tg5mJlZjZODmZnVuM/BALclm9nCXHMwM7Oa5bLm4G/JZmaL5pqDmZnVODmYmVmNk4OZmdU4OZiZWc1y2SFtZr3PA0uWLdcczMysxjUHsx7gb8nWNCeHwpr8o/cfvJkNlpuVzMysxsnBzMxqnBzMzKzGycHMzGqcHMzMrKY1yUHSHpLukzRL0tGl4zEzW561IjlIWhH4HrAnsCVwiKQty0ZlZrb8akVyALYHZkXE7Ij4K3AusG/hmMzMlluKiNIxIOmDwB4R8fF8/GFgh4g4os9544Hx+fDNwH2NBgrrAI83fM1u2hIHOJZu2hIHtCeWtsQB7YmlVBybRMSIgU5qywxpdSmrZa2IOB04fdmH052kaRExptT12xYHOJY2xwHtiaUtcUB7YmlLHP1pS7PSHGCjyvGGwNxCsZiZLffakhxuAUZJ2lTSKsDBwKTCMZmZLbda0awUES9KOgKYDKwITIiIuwqH1U2xJq0+2hIHOJZu2hIHtCeWtsQB7YmlLXF01YoOaTMza5e2NCuZmVmLODmYmVmNk0MPkDTgmGQzs6XJyaELSZtJOkPSKZI2Lh0PcL2kyyV9TNLwkoFI+q6kHQvHsNaibgXikaQPSfrXfLyxpO2bjiNfe6yk1SvHq0vaoUAcm0p6beV4VUkjm44jX/t2SV+S9MYS16/EMU3S4aX/hgfLyaG7c0nDa2cBv5a0U8lgImIU8BVgK2C6pIslfahQOLcCX8kLJH5LUolJPNOBafl+HvAb4P78eHqBeE4F3gkcko/nk9YKK+E04E+V4z/nsqZdALxcOX4pl5WwD/AicL6kWyR9odCXvoOB9YFbJJ0raXdJ3SYAt0NE+NbnBsysPB5N+sB5CvgAcG3h2NYBzgJeKhzHWsAngCnA/YVi+D7wvsrxnsBJBeK4Nd/fVim7vdDPZEaXspktiaPIz6RPDKNK//2QvpTvAzwMPAR8DVir9M+m7801h+4elbQNQETMiIjtImLNiPh5RLyr6WAkDZM0TtJlwPXAI6TFCkt6E7AFMBK4t1AM74iISzsHEXEZ8J4CcbyQVxYOeKWP6OVFv2SZmS3pSEkr59tRwOwCccyTtE/nQNK+FFzPSNJISV8ktQpsAXyxUBzbACcB3wJ+BnwQeAb4dYl4FsXzHLrIf9wrRcQjpWMBkPQ74BfA+RFxQ+FYvkmqQf0WOB/4eUQ8VSiWycA1wI9JH8wfAnaOiN0bjuMfgYOAtwMTSX/wX42I85uMI8eyLnAK8Dekn8kU4LMR8VjDcbwR+AmpGUWkb8iHRsSsJuPIsdwErExq1jovIkokSyR1WiB+CPwsIp6vPPfziPhAibj64+TQAyQpIiJ3NEZE/GnAFy27WD4F/E9EFF/VMnc+HwvsTPogvBo4PiKeKBDLFsCupA/CKRFxT9MxtJGk1UifM/MLxrBFRJSq3Vbj2KxUYhoKJ4ceIGlr4GxSO79IHa/jIuLOQvHsQ/pABpgaEb8sEUclntUKJ8yzI+LDA5U1FMsIUl/QSCrL40TERxuO4zXAAV3iOL7JOHIsa7DgSwTAVNKXiKcLxLIXaWDJKyO5SvxMBsN9Dr3hdOBzEbFJRGwMfJ5C67JI+gZwFHB3vh2Zy0rEsqOkThxI2lbSqQVC2apPXCsC2xWIA+AiYA3gf4FLKrcScexLGiX058qthAmkEWR/n2/PAD9qOghJ3yc1P36G9CXvQGCTpuMYLNcceoCk2yNi24HKGoplJjA6Il7OxyuSRulsUyCWm0jt+5Mi4m257M6I2Lqh6x8DfAlYFXiWBfuS/BU4PSKOaSKOPjHNiIjRTV+3SxyN/R4G0u1nUuLnJGlmRGxTuV+N1Ge3W5NxDJZrDr1htqSv5hEXIyV9BfhdwXjWrDxeo1gUQEQ81KfopQav/Y2IWB34VkQMi4jV823tEokhu1jS+wpdu+p6SW8tHUT2F0mvjDLM85b+UiKOfP+spPWBF4BNC8QxKK1YstsG9FHSWOifk76dXg0cViiWbwC3Sboyx7IzUOqD8KE8WzuU9gE5Emi8IzgijsmzXkexcFvy1U3HQmry+5Kk50kfPkqhxLCG43gX8JE80u75ShyN1zCBfwIm5r4HAU8AHykQx8WS1iQNY72VNIjijAJxDIqblWyxSVoPeAfpD+2miPhDoTjWAb4L/G2O5XLgqIj4Y8NxfJz0obwhMAMYC9wQEX/TZBxtIqlrW3pEPNB0LB2ShuUYnikVQyWW1wCvLdEpPlhODj1A0i+p76n9NGkJiR9ExHMNxvL2LsVPAw9ExItNxZFjWavvsFVJm0ZEo01uku4gJcsbI2J0Htb6tYg4qMEYtoiIe/v5/RARtzYUx7CIeEb9rHFVaJjx57oUPw1Mj4gZDcbxWuDTpFpVANcCpzX597s4nBx6gKTvAiOAc3LRQcAfSB2hw5ocMinpRtJkr5mkb+tb58drA5+KiMsbjOU6YM/ON0FJbwEuaLojVNItEfEOSTOAHSLi+aY7PCWdHhHjc3NfX9FULUbSxRHx/tycFCzopO/EsVkTcfSJ6afAGKAz5Hov0tppW5D+v/xHQ3GcTxo19eNcdAgwPCIObOL6i21pr8fh29K/AVf3Vwbc1XAs5wJbVY63JA0L3Iwu6+ks41j2Io1ZX400dPQu0kiqpn8/F5I66Y8j9QddBFxa+v+Nb6/8fiYDq1WOVwN+RfpydXeDcdTWlupW1pabO6R7wwhJG0fEg5CWhCYtwAdp2GSTtojK/t4Rcbekt0XE7KYXmIyISyStTOprWB3YLyLubzSIFMf++eFx+Zv7GqQPn6I6tYkWxHFcRBxXMISNWfjv5AVgk4j4S+64b8ptksZGxI0ASkupX9fg9ReLk0Nv+DxwraTfkqrpmwKflvR60lo+TbpP0mmkGgSkJq7f5A62F5oIQNJ/sXAfzDDS4nKfkUREHNlEHJV4vktas+f6iJja5LUHUGI59W72IdWqSvkpcKOki/Lx3sA5+e/n7gbj2AE4VNKD+Xhj4J7cZxVRZiRXv9zn0CPyh+8WpORwbxTqxJK0Kgs61UTqVDsVeA54XTSwjIWkcYt6PiIaTZg5noOAzUlNTOdFxLQmY+hG0q8iYo8WxHFb5EmKBWPYjsr/2RK/n/5GcHVEwZFc3Tg59Ji2NBVA+oOLiBKb63RmZk+MiFKbHtXkEToHkDZ12TjSJk2lYhlG+jZaZME7STtFxHX58QoR8XK1rBRJ4yOiyNIzbYxjUTxDuve0pakACk7giYiXSH0xq5SKoYvie1xIGpObKWYCdyhtkVlinaf/6jyIvNRKtaygT5UOIGtLHP1yn0PvaXRd/gGU3uLw98B1kiZRWdQtIr7TZBCq73FxQhTa44K0yNynI+KaHNu7SKPJGmnPlvROYEdS4q7OLxgGrNhEDAMo/X+2oy1x9MvJoce0oQ254muFrz8331YgjVYq5XfAO6MFe1wA8zuJASAirpXUZNPSKqShoiux8O/kGdIiiaXtXerCfSZo7t2lrFXc59Bi/cyMfkVE7NPfc8sglq4zbyuxNDIDtxsV2gSpLbOS+8R0MvA60oTJIHWUP0nakrKxmCRtEhEPlPrd9InlDcC/A+tHxJ6StiQl8x82HMetEfH2PmXTI6LU8u6L5JpDu327dAAVJy3iuSBtS9koLbwJEpIeJ21FedciX7j0fA4YT/efTZGfCdCZlX1sn/IdaTam1SXdxsK/m3FRZoOqM0lNa1/Ox78BziNt17nM5eVUtgLWkFTdCnQYlYUa28Y1B+tZkq4HvhwRV+bjXYB/j4gdG4xhBdK30NZOZiqhDb+bSiyd5U1eGVLb5PImkvYF9iPN95hUeWo+cG5EXN9EHIvLNYceIGkUaansLVl4SejG16nJ8WzdJZazCoTy+s6HT47hqjyxqTF5iOa3gXc2ed3+5CWhD6W+PWejEwNpwe+m4s+S1iY30UoaS1p4rxERcRFwkaR3RsQNTV13STk59IYfkZoJTgbeS9rLochoB0nHAruQksOlwJ6kiXAlksNsSV8lNS0BfIgymyBdLukA0q5epavilwI3AncALw9w7rLUlt8NpOa/ScAb82KNI0hbdDZtvKRP9C2Mhvf3Hiw3K/WATqeVpDsi4q257JqIeHeBWO4AtiVtDbpt7uz774hofBSI0gY7X2PBzNeppKWyn2w4jvnA60n7JT9HuQ12unZ6ltDld3M1cFzTv5scy2tIOwS+OcdyH7BCRDS5rhL5C0THa4H9gbkFanWD4ppDb3gut23fL+kI4GFg3UKx/CU3pbyYZ+E+RlqRtYThbfjDirRVaFucnb+dXkzagQ1ofh+FnASK/26yG3LCfGWggqRbSUvPNyYiflY9lnQO8L9NxrA4nBx6w2dJwxOPBE4gNS0tcn2hZWhabtc+A5gO/Am4uVAsZ0ragLQ2/9XANRFxR1MXl7Qu8CXSzOiZwIlRfpexv5K2ofwyC4ZBBw0ncEmbA1+g3vfR2AguSf8H2ABYVdLbWNAUO4z091TaKNLie63kZqWWy2sInRgR/1w6lr4kjSRtNjSzYAyrkHZh2wX4JGnd/q67kC2Da/+KlCCvBt4PrB4RH2ni2ouI6bekDYeKTsiTdDvwfdLP56VOeZNrceUFET9CWnKmutDefODMiPh5U7HkeOazYAOkIG3YdUzfGkVbODn0AEm/BnZtQWcnkvYHfh1579tci9glIn5RIJZ3Ae/OtzVJ+zdfExHnLPKFS+/6Cw2HbEN7f15K5OCIeLZwHK2Z3CXpgLZ+ALeZk0MPkHQSqQp6AQuvIdToN58cS218eKklmSW9RPpG+A3SzmuNbnyUvx3vwoLmiiurx0238+eYLiRNuLqShfscmt7j4jhSf9SFFOz7qMSzF+nnUh1+fXyBOPYBds6HV0XExU3HMFjuc+gNawF/ZOHZrQE0nhzovpJvqf9HawM7kf7YjpT0Mqnz8asNXX8NUrNJdVhxZ3mKxtv5s1/kW2mdPrFqc2iRn4mk75P6GN4L/DdpjafG+8kknUhqAv1JLjoqL2N+TNOxDIZrDrZYJE0AngK+R/pj/wxp1NBHCsXzFuA9pKalHYEHI+I9JWJpC6UNmTaOiPtKx9IGkmZGxDaV+9VIc1J2azoO0h7nL+fjFUlDwlu1A1yH93PoAZI2lzRF0p35eBtJXykUzmdII2LOIzVzPQccXiKQ3Pl6EjCc1Pn5ZicG7U3qe/lVPh6d+yFKxlR6U5u/5PtnJa1P2s5200KxrFl5vEahGAbFzUq94QxS9fwHABExU9JPgX9rOpCI+DNwdJ7j8HLJ1TaBUZWNZCw5DtgeuAogImZIKvVB2FF6g6qL88CJb5Ga/YLUvNS0bwC3SbqS1BS5M9DKJiVwcugVr4uIm6WFVsx4sUQgkt5KWiqj+Gqb1cTQhpFCLfFiRDzd5/9K6bbj0htU/UeeDf0zSReTOqUb34M9Is6RdBWp30HAv0TEH5qOY7DcrNQbHpf0RhYsHPZB4JFCsfwA+FxEbBIRmwCfB0o3G0AP7KzVkDsl/QOwoqRRkv4LKLbqZ65hlljHqOqVxe4i4vk8DLvUAngrAI+T9tjYXNLOA5xfjGsOveFw0gfwFpIeJi1g9qFCsbRptc2qS0oH0BKfIc2Ofh74KTCZAs2PksaQFoxcPR8/DXy04UlwrZohrbSd7EGkZTw6td4gTaJsHY9W6iH5Q3iFiGhy28e+MVxIaretrrY5JiL2aziOFYHJEfG3TV63l0haLyKK1DDzyJzDY+G9rE9tcmROnxnSt7AgOZSaIX0fsE3TC/4NlWsOLaaFN2ivlgMQEd9pNKDko6TVNjt/WFeT/gAbFREvSXpW0hqd2dpWcwkNLy5XUXovayJiIjCxRTOkZwMrU5kU2GZODu3WWe3zzaROrM6QxL0pVBXtttpm3uzmCwXCeQ64Q9IVLDxzvC2rgZZWsh/mZkk/YOG9rK9S3m87GtjLOg/rndlJDJL+FTgAeAA4KiIa2V8i9/sE8CwwQ9IUCs5eHyw3K/UASZcDB3Sak5Q2bb8gIvYoG1ki6cGIaHx1ydyWisH2AAAGYklEQVRsUJO/MS5XJG3a98NO0qcj4tRC8Vy5iKejidVZc9PW2Ih4VtL7ge8AhwBvAw6MiN2XdQw5jkWtoBxRZhfFATk59ABJ9wLbdtoq8+Ylt0fEFmUjSyQ9FBEbFbq2ZwOz0IZQUyJi19LxtIGk2yNi2/x4AnBfRHwzHzc+9FnSURHx3YHK2sLNSr3hbFI1/UJS9XR/Gt6WU1J/y2CLcluW7g18G1gF2FTSaOD4iNinRDyFraC0hevm3fqqmu6fUjv2slZeKuNZYFegWot6bfeXLFPjgL6J4CNdylrByaEHRMTXJV1GWj8I4LCIuK3hMKazYC36vhpdDbXiONo3G7iUg4H9SH/TbdiZrg17Wf8naSmRZ4B7ImIaQB7W2tgoLkmHAP9A+gJTXcpkddKCmq3k5NA7Xgc8ExE/kjSiWxvzshQRbfzQbeNs4CJys9o38+Jyl5WOB3htRHQdbdeUiJggaTJpS93bK0/9ATiswVCuJyWjdUhrgXXMJ+0g2Eruc+gBublgDGlhuc3z4mEXRMROhUMrStIPgSnA0aRRKEcCK0fEp4oGVkB/w547CjQr/V/SFrJF97K2ofPyGb1hf2Af8nDNiJhLO5oOSvsMaQOXzmzgp0n7bS+PVh/g1rTOXtY3kJokp7PwVp3LDUnX5vv5kp6p3OZLKr3neL9cc+gBkm6OiO07IyzyTOkb2roOfFMkva1A34sNglqyl7UNnWsOveH8PKFoTUmfAP6XtIz38u47ku6VdIKkrUoH0wYt2vvjLtIooeWepP+UdGBuDu4Zrjn0CEl/B+xGGi00OSKuKBxSK+TF1f6eNAN3GHBeRDS+0FxbSJpK3vujs6+3pDsjYuuG42jFXtZtIOkI0i6FO+ai64Hr8v3tbd2TxMmhxSSNjYgbS8fRC/I+E18EDoqIVUrHU4qkWyLiHZJuqySHGRExuuE4PHu9C0nrkfY935HUj7huRAwrG1V3HsrabqeSF06TdENEvLNwPK2itH/0QaT9Ah4HziXtL7E8a8XeHxEx0bPXF1Aab/1WUlLYCdgSmMWC1Y1bx8mh3aoD+EvM6Gy7H5EWdvu7PILLuu/98Y9NB+HZ6wvkhSGHkSbk3Qj8e0TcUzaqgTk5tNsKkoaTBg50Hr+SMJb3MeMRMbbz7bR0LG0REbOBv23B3h/H4dnrHbOBbYFRpBnRj0ua1/aRXB6t1G5rsGB8+DDSJjvL9ZjxqvztdAbwq3w8us/yBMsNSXtL2qRS9HngWkmTCn0ov9hln43lsoMzIj4ZEWNJy5tcBWwH/FjSdEmt7YNxzaHFImJk6Rha7jjq305HlgunqK8DYwHy8tQfYsHy1N8HGlmeumKhvaxJs9eL7WXdEs+Thvf+JT/ekNTs1kquOVgv6/btdHkVEdGZV/AB4IcRMT0i/hsYUSAez17PJJ0s6SbSwIDjSTPWf0BaDuetRYNbBNccrJf52+kCrVqeOieqLwNfVsG9rFvid8BPgNsi4qXSwQyWaw7Wy6rfTs8hLc28XH47ZcHy1NMouDx1Py4pfP2iIuKUiJjWS4kBPAnO7FVD0gbk5ak7s27zpKuVI+LBgnG9MiHPeoeblaznSPoR/Y98iYj4WJPxtEVEPAw83Kes0VpDP/uMeB2wHuSag/UcSQd0Kd6Y1KS0YkRs2HBIlnkv61cPJwfraZI2A74E7AycTBqlU2rb0uWepNuAXwAfJ/0+FtL0pkM2dO6Qtp4k6S2Sfgz8ErgW2DIiTnNiKO5g4DkW7GVdetMhGyLXHKznSLqAtG3qt4HzgYVGgSzvy4q0gaQ9W7KXtQ2Rk4P1HEm/Z0GHdOe+s+ZURMRmjQdlQPv2srah82gl6zleVqTV3HT0KuGag5mZ1bhD2syWuhbtZW1D5ORgZsvCGcAxwAsAETGTNJLJeoSTg5ktC6+LiJv7lL1YJBIbEicHM1sWWrGXtQ2dO6TNbKnLM9dPB3YEniTvZR0RDxQNzAbNycHMlpkW7GVtQ+RmJTNbalq4l7UNkZODmS1NXwfmwUJ7WX8UmETay9p6hJODmS1NbdvL2obIycHMliZJWk3SCqS9rKdUnmt8L2sbOq+tZGZLU2cv62do317Wthg8WsnMlqq27mVti8fJwczMatznYGZmNU4OZmZW4+RgZmY1Tg5mZlbj5GBmZjVODmZmVuPkYGZmNf8fvDi+b98EKXAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAELCAYAAAAybErdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAFXhJREFUeJzt3X3Q3WV95/H3RyKIUgkPESVBgxpr1W6VjcjqrLLFQiJimK5uo65EFic7HXza6bai7Uy6KN2425VKZ3WGAgrqiIhWaNEq4sOOrQIBEcWoiUBJDEJsAj5QHyLf/eNcd3u4rztP9wk5J+b9mjlzfr/rd12/+3vfSe7Pua7f75ykqpAkadgjxl2AJGnyGA6SpI7hIEnqGA6SpI7hIEnqGA6SpI7hoF8ZSU5MsnEMX/dtSS7a219XejjNGXcB0r4kyYnAB6tqwVRbVf3Z+CqSHh7OHCRJHcNBEy/J0Uk+lmRzkjuSvLG1H5zk/Um2Jvkm8Nxp4yrJU4f235/kHUP7y5LckuSHSb6bZElrPzPJ2iQ/SnJ7kv/a2h8DfAo4OsmP2+PoJH+a5IND531ZktuS3JfkC0l+Y+jYnUn+e5Jbk9yf5CNJHrWD7327/ZO8NsmXtvc9t+/3PUk+1Wr9+ySPT/IX7Wf2rSTPmcUfifYDhoMmWpJHAH8DfA2YD5wEvDnJKcAq4CntcQqwYjfOezxwGfCHwFzghcCd7fC9wEuBxwJnAucnOa6qfgIsBTZV1SHtsWnaeZ8GfBh4MzAP+CTwN0kOHOr2n4AlwLHAvwFeu5Nyd7f/9LF/AhwJ/Az4MnBz278SeNdunEv7EcNBk+65wLyqOreqfl5VtwN/BSxn8IvvvKraUlUbgAt247xnAZdU1bVV9WBVfa+qvgVQVddU1Xdr4IvAZ4B/v4vn/T3gmnbeXwB/DhwMPH+ozwVVtamqtjAIvmfv5Jy723/YX1fVTVX1U+CvgZ9W1WVV9UvgI4AzB83IcNCkexKDZZz7ph7A24CjgKOBDUN9/3E3znsM8N2ZDiRZmuQrSba0r/cSBq+0d8XRw3VU1YOtxvlDfb4/tP0AcEj7ulPLPz9O8uqd9d9F9wxt//MM+7tzLu1HvFtJk24DcEdVLZp+IMkdDH7J39aanjitywPAo4f2Hw9M3eq6gcFy1PRzHgR8DDgDuKqqfpHkE0Bal519jPEm4DeHzpdW4/d2Mo6qWrqzPtP8hKHvL8njd3O8tF3OHDTpbgB+mOQt7QL0AUmeleS5wBXAW5MclmQB8IZpY28BXtXGLAFeNHTsYuDMJCcleUSS+UmeDhwIHARsBrYlWQqcPDTuHuCIJIdup94rgFPbeR8J/AGDtf5/GOmnMLOvAc9M8ux2kfpPH4avof2U4aCJ1tbGT2Owzn4H8APgIuBQ4H8wWMK5g8F1gQ9MG/6mNvY+4NXAJ4bOewPtYjNwP/BF4ElV9SPgjQx+yW8FXgVcPTTuWwwuON/elrmOnlbvt4H/DPxlq/U04LSq+vmIP4pOVX0HOBf4LLAO+NKOR0i7Lv5nP5Kk6Zw5SJI6hoMkqWM4SJI6hoMkqWM4SJI6++yb4I488shauHDhuMuQpH3GTTfd9IOqmrcrfffZcFi4cCFr1qwZdxmStM9IsssfMeOykiSpYzhIkjqGgySpYzhIkjqGgySpYzhIkjqGgySpYzhIkjr77JvgpEm38JxrZj32ztWn7sFKpN3nzEGS1DEcJEkdw0GS1DEcJEkdw0GS1NlpOCS5JMm9Sb4x1HZ4kmuTrGvPh7X2JLkgyfoktyY5bmjMitZ/XZIVQ+3/NsnX25gLkmRPf5OSpN2zKzOH9wNLprWdA1xXVYuA69o+wFJgUXusBN4LgzABVgHPA44HVk0FSuuzcmjc9K8lSdrLdhoOVfX/gC3TmpcBl7btS4HTh9ovq4GvAHOTPAE4Bbi2qrZU1VbgWmBJO/bYqvpyVRVw2dC5JEljMttrDkdV1d0A7flxrX0+sGGo38bWtqP2jTO0S5LGaE9fkJ7pekHNon3mkycrk6xJsmbz5s2zLFGStDOzDYd72pIQ7fne1r4ROGao3wJg007aF8zQPqOqurCqFlfV4nnzdun/yJYkzcJsw+FqYOqOoxXAVUPtZ7S7lk4A7m/LTp8GTk5yWLsQfTLw6XbsR0lOaHcpnTF0LknSmOz0g/eSfBg4ETgyyUYGdx2tBq5IchZwF/CK1v2TwEuA9cADwJkAVbUlyduBG1u/c6tq6iL37zO4I+pg4FPtIUkao52GQ1W9cjuHTpqhbwFnb+c8lwCXzNC+BnjWzuqQJO09vkNaktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQZKRyS/LcktyX5RpIPJ3lUkmOTXJ9kXZKPJDmw9T2o7a9vxxcOneetrf3bSU4Z7VuSJI1q1uGQZD7wRmBxVT0LOABYDrwTOL+qFgFbgbPakLOArVX1VOD81o8kz2jjngksAd6T5IDZ1iVJGt2oy0pzgIOTzAEeDdwN/DZwZTt+KXB6217W9mnHT0qS1n55Vf2squ4A1gPHj1iXJGkEsw6Hqvoe8OfAXQxC4X7gJuC+qtrWum0E5rft+cCGNnZb63/EcPsMYx4iycoka5Ks2bx582xLlyTtxCjLSocxeNV/LHA08Bhg6Qxda2rIdo5tr71vrLqwqhZX1eJ58+btftGSpF0yyrLSi4E7qmpzVf0C+DjwfGBuW2YCWABsatsbgWMA2vFDgS3D7TOMkSSNwSjhcBdwQpJHt2sHJwHfBD4PvLz1WQFc1bavbvu045+rqmrty9vdTMcCi4AbRqhLkjSiOTvvMrOquj7JlcDNwDbgq8CFwDXA5Une0doubkMuBj6QZD2DGcPydp7bklzBIFi2AWdX1S9nW5ckaXSzDgeAqloFrJrWfDsz3G1UVT8FXrGd85wHnDdKLZKkPcd3SEuSOoaDJKljOEiSOoaDJKljOEiSOoaDJKljOEiSOoaDJKkz0pvgJO0dC8+5ZqTxd64+dQ9Vov2FMwdJUsdwkCR1DAdJUsdwkCR1DAdJUsdwkCR1DAdJUsdwkCR1DAdJUsdwkCR1DAdJUsdwkCR1DAdJUsdwkCR1DAdJUsdwkCR1DAdJUsdwkCR1DAdJUsdwkCR1DAdJUmekcEgyN8mVSb6VZG2Sf5fk8CTXJlnXng9rfZPkgiTrk9ya5Lih86xo/dclWTHqNyVJGs2oM4d3A39XVU8HfgtYC5wDXFdVi4Dr2j7AUmBRe6wE3guQ5HBgFfA84Hhg1VSgSJLGY9bhkOSxwAuBiwGq6udVdR+wDLi0dbsUOL1tLwMuq4GvAHOTPAE4Bbi2qrZU1VbgWmDJbOuSJI1ulJnDk4HNwPuSfDXJRUkeAxxVVXcDtOfHtf7zgQ1D4ze2tu21S5LGZJRwmAMcB7y3qp4D/IR/XUKaSWZoqx209ydIViZZk2TN5s2bd7deSdIuGiUcNgIbq+r6tn8lg7C4py0X0Z7vHep/zND4BcCmHbR3qurCqlpcVYvnzZs3QumSpB2ZdThU1feBDUl+vTWdBHwTuBqYuuNoBXBV274aOKPdtXQCcH9bdvo0cHKSw9qF6JNbmyRpTOaMOP4NwIeSHAjcDpzJIHCuSHIWcBfwitb3k8BLgPXAA60vVbUlyduBG1u/c6tqy4h1SZJGMFI4VNUtwOIZDp00Q98Czt7OeS4BLhmlFknSnuM7pCVJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJnVHfBCdJe8zCc66Z9dg7V5+6ByuRMwdJUsdwkCR1DAdJUsdwkCR1DAdJUsdwkCR1DAdJUsdwkCR1DAdJUsdwkCR1DAdJUsdwkCR1DAdJUsdwkCR1DAdJUsdwkCR1DAdJUsdwkCR1DAdJUsdwkCR1DAdJUsdwkCR1DAdJUmfkcEhyQJKvJvnbtn9skuuTrEvykSQHtvaD2v76dnzh0Dne2tq/neSUUWuSJI1mT8wc3gSsHdp/J3B+VS0CtgJntfazgK1V9VTg/NaPJM8AlgPPBJYA70lywB6oS5I0SyOFQ5IFwKnARW0/wG8DV7YulwKnt+1lbZ92/KTWfxlweVX9rKruANYDx49SlyRpNKPOHP4C+CPgwbZ/BHBfVW1r+xuB+W17PrABoB2/v/X/l/YZxjxEkpVJ1iRZs3nz5hFLlyRtz6zDIclLgXur6qbh5hm61k6O7WjMQxurLqyqxVW1eN68ebtVryRp180ZYewLgJcleQnwKOCxDGYSc5PMabODBcCm1n8jcAywMckc4FBgy1D7lOExkqQxmPXMoareWlULqmohgwvKn6uqVwOfB17euq0ArmrbV7d92vHPVVW19uXtbqZjgUXADbOtS5I0ulFmDtvzFuDyJO8Avgpc3NovBj6QZD2DGcNygKq6LckVwDeBbcDZVfXLh6EuSdIu2iPhUFVfAL7Qtm9nhruNquqnwCu2M/484Lw9UYskaXS+Q1qS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEmdWYdDkmOSfD7J2iS3JXlTaz88ybVJ1rXnw1p7klyQZH2SW5McN3SuFa3/uiQrRv+2JEmjGGXmsA34g6r6DeAE4OwkzwDOAa6rqkXAdW0fYCmwqD1WAu+FQZgAq4DnAccDq6YCRZI0HrMOh6q6u6pubts/AtYC84FlwKWt26XA6W17GXBZDXwFmJvkCcApwLVVtaWqtgLXAktmW5ckaXR75JpDkoXAc4DrgaOq6m4YBAjwuNZtPrBhaNjG1ra9dknSmIwcDkkOAT4GvLmqfrijrjO01Q7aZ/paK5OsSbJm8+bNu1+sJGmXjBQOSR7JIBg+VFUfb833tOUi2vO9rX0jcMzQ8AXAph20d6rqwqpaXFWL582bN0rpkqQdGOVupQAXA2ur6l1Dh64Gpu44WgFcNdR+Rrtr6QTg/rbs9Gng5CSHtQvRJ7c2SdKYzBlh7AuA1wBfT3JLa3sbsBq4IslZwF3AK9qxTwIvAdYDDwBnAlTVliRvB25s/c6tqi0j1CVJGtGsw6GqvsTM1wsATpqhfwFnb+dclwCXzLYWSdKeNcrMQZIm1sJzrhlp/J2rT91Dleyb/PgMSVLHcJAkdQwHSVLHcJAkdQwHSVLHcJAkdbyVVdrPeIundoUzB0lSx3CQJHUMB0lSx2sO2qe5fi49PJw5SJI6zhwkjWSU2Zszt8nlzEGS1DEcJEkdl5W013kRWZp8zhwkSR3DQZLUMRwkSR3DQZLU8YL0rzDvP5c0W84cJEkdZw6StBP74+3XhoN2iUtU0v7FZSVJUseZgzTEGZL2hn3h75nhMEH2x3VNSZPJZSVJUsdwkCR1DAdJUmdirjkkWQK8GzgAuKiqVo+5pF2yL1xYkqTdNREzhyQHAP8XWAo8A3hlkmeMtypJ2n9NyszheGB9Vd0OkORyYBnwzYfji/lqX5J2LFU17hpI8nJgSVW9ru2/BnheVb1+Wr+VwMq2++vAt9v2kcAP9lK5u2tSa5vUumBya5vUumBya5vUumBya3s463pSVc3blY6TMnPIDG1dalXVhcCF3eBkTVUtfjgKG9Wk1japdcHk1japdcHk1japdcHk1jYpdU3ENQdgI3DM0P4CYNOYapGk/d6khMONwKIkxyY5EFgOXD3mmiRpvzURy0pVtS3J64FPM7iV9ZKqum03TtEtNU2QSa1tUuuCya1tUuuCya1tUuuCya1tIuqaiAvSkqTJMinLSpKkCWI4SJI6hoMkqTMRF6R3V5KnM3gH9XwG74fYBFxdVWvHWhj/Utt84Pqq+vFQ+5Kq+rvxVfZQSS6rqjMmoI7nAWur6odJDgbOAY5j8O74P6uq+8dU19Rdc5uq6rNJXgU8H1gLXFhVvxhHXdLess9dkE7yFuCVwOUM3h8Bg/dFLAcuH+cH9iV5I3A2g18gzwbeVFVXtWM3V9VxY6pr+m3BAf4D8DmAqnrZXi9qqpDkNuC32h1rFwIPAFcCJ7X23x1TXR9i8OLp0cB9wCHAx1tdqaoV46hLe06Sx1XVveOuYyZJjqiqfxprEVW1Tz2A7wCPnKH9QGDdmGv7OnBI214IrGEQEABfHWNdNwMfBE4EXtSe727bLxrzz2ztcJ3Tjt0yxrpubc9zgHuAA9p+po6NsbZDgdXAt4B/ao+1rW3umGt7LPA/gQ8Ar5p27D1jrOvwaY8jgDuBw4DDx/wzWw0c2bYXA7cD64F/HOe/z33xmsODwNEztD+hHRunA6otJVXVnQx+CS9N8i5m/oiQvWUxcBPwx8D9VfUF4J+r6otV9cUx1gXwjSRntu2vJVkMkORpwDiXbh7RlpZ+jcHs4dDWfhDwyLFVNXAFsBU4saqOqKojGMwEtwIfHWtl8D4Gf9c/BixP8rEkB7VjJ4yvLH7A4N/A1GMNg+Xfm9v2OJ1aVVOfpfS/gd+rqqcCvwP8n3EVtS9ec3gzcF2SdcCG1vZE4KnA67c7au/4fpJnV9UtAFX14yQvBS4BfnNcRVXVg8D5ST7anu9hcv7sXwe8O8mfMPgH/OUkGxj82b5ujHVdzOCV+QEMQvWjSW5n8Avu8jHWBbCwqt453FBV3wfemeS/jKmmKU+pqv/Ytj+R5I+BzyUZ29Jl80fAi4E/rKqvAyS5o6qOHW9ZADwyyZyq2gYcXFU3AlTVd4aCda/b5645ACR5BIOP+Z7P4FXKRuDGqvrlmOtaAGxr/1CnH3tBVf39GMrqJDkVeEFVvW3ctUxJ8mvAkxmE1saqumfMJZHkaICq2pRkLoNfLndV1Q1jruszwGeBS6d+TkmOAl4L/E5VvXiMta0FntlekEy1rWDwy/mQqnrSGGtbAJzP4IXHKuBrVfXkcdUzJckbgNMYLC+9EJjLv17fenJVvWYsde2L4SDtz5IcxuCurmXA41rzPQw+j2x1VW0dY23/C/hMVX12WvsS4C+ratF4KntILacxmA0urKrHj7segCQnAr8PPI3BC6QNwCcYfJTQtrHUZDhIvzqSnFlV7xt3HTOZpNrabdNPqapvTFJd042zNsNB+hWS5K6qeuK465jJpNY2qXXBeGublIuSknZRklu3dwg4am/W0hUwobVNal0wubUZDtK+5yjgFAa3rg4L8A97v5yHmNTaJrUumNDaDAdp3/O3DO78uWX6gSRf2PvlPMSk1japdcGE1uY1B0lSZ198h7Qk6WFmOEiSOoaDJKljOEiSOoaDJKnz/wGUwUwuK50ODgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAFdCAYAAAAKZ7pOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJztnXe4XFXZt+9fQpcOASnBAEYUkWZowisISpWmKKAiIop8gmJDwQaC2AVBBV8UEBREVJCAKIQuSEsgdHiJtAQQIr2X8Pv+WGvIzsnk5Mxee3JOmOe+rrnO7DV7P7PmnDn7Weupsk0QBEHQewwb7AkEQRAEg0MogCAIgh4lFEAQBEGPEgogCIKgRwkFEARB0KOEAgiCIOhRQgEEwRxG0kqSnpE0fLDnEvQ2oQCCoMtIulfSe1vHtu+3vbDtaYM5ryAIBRAEQdCjhAII5lokvU3SpZKekHSrpB3y+IKSfirpPklPSrpC0oL5tU0k/StfM1nSJ/L4pZI+VZH9CUlXVI4t6fOS7pb0X0k/ljQsv7aqpIslPZpfO1XS4vm13wErAedks89XJY3K8ubJ5ywvaaykxyRNkvTpyvseKukMSadIejp/zjFd/+UGPUEogGCuRNK8wDnABcAywOeAUyWtBvwEeCfwLmBJ4KvAq5JWAv4O/BwYAawNTOzgbXcGxgDrAjsCn2xNB/g+sDzwNmAkcCiA7T2A+4Hts9nnR23k/gGYkq/fBfiepC0qr+8AnA4sDowFftHBnINgloQCCOZWNgQWBn5g+yXbFwPnAh8l3ZgPsP2A7Wm2/2X7xfzahbb/YPtl24/a7kQB/ND2Y7bvB34G7A5ge5LtcbZftD0VOBLYdCACJY0ENgG+ZvuFPJ/fAHtUTrvC9nnZZ/A7YK0O5hwEs2SewZ5AENRkeWCy7VcrY/eRVt8LAP9uc83IWYwPlMl93mt5AEnLAMcA/wMsQlpYPT5AmcsDj9l+uo/sqpnnP5XnzwELSJrH9iudTT8IZiR2AMHcyoPAyJYdPrMS6Sb9ArBqm2smz2Ic4FlgocrxG9ucM7LPez2Yn38fMLCm7UWBj5HMQi36K7n7ILCkpEX6yH6gn2uCoBFCAQRzK9eQbtpflTSvpM2A7YHTgBOBI7NzdbikjSTND5wKvFfShyXNI2kpSWtneROBD0haSNKbgb3bvOeBkpbIZpsDgD/m8UWAZ4AnJK0AHNjnuoeBVdp9CNuTgX8B35e0gKQ183ufWuN3EgQdEQogmCux/RLJOboN8F/gWODjtu8AvgLcDFwHPAb8EBiWbffbAl/O4xOZbk8/CniJdLM+mfY34LOBCfm6vwEn5PHvkBzDT+bxM/tc933gmzny6Ctt5O4OjCLtBs4CDrE9boC/iiCojaIhTBDMHkkGRtueNNhzCYKmiB1AEARBjxIKIAiCoEcJE1AQBEGPEjuAIAiCHiUUQBAEQY8ypDOBl156aY8aNWqwpxEEQTBXMWHChP/aHjG784a0Ahg1ahTjx48f7GkEQRDMVUi6byDnhQkoCIKgRwkFEARB0KOEAgiCIOhRQgEEQRD0KKEAgiAIepRQAEEQBD1KKIAgCIIeJRRAEARBjzKkE8GCIJizjDrob8Uy7v3Bdg3MJJgTxA4gCIKgR4kdwOuMWMEFQTBQYgcQBEHQowx4ByBpODAeeMD2+yWtDJwOLAlcD+xh+yVJ8wOnAO8EHgV2tX1vlnEwsDcwDfi87fOb/DBBdyjdVcSOIgiGJp3sAA4Abq8c/xA4yvZo4HHSjZ3883HbbwaOyuchaXVgN+DtwNbAsVmpBEEQBIPAgBSApBWB7YDf5GMBmwN/zqecDOyUn++Yj8mvb5HP3xE43faLtu8BJgHrN/EhgiAIgs4Z6A7gZ8BXgVfz8VLAE7ZfycdTgBXy8xWAyQD59Sfz+a+Nt7nmNSTtI2m8pPFTp07t4KMEQRAEnTBbBSDp/cAjtidUh9uc6tm81t810wfs422PsT1mxIjZNrQJgiAIajIQJ/DGwA6StgUWABYl7QgWlzRPXuWvCDyYz58CjASmSJoHWAx4rDLeonpNEARBMIeZ7Q7A9sG2V7Q9iuTEvdj2R4FLgF3yaXsCZ+fnY/Mx+fWLbTuP7yZp/hxBNBq4trFPEgRBEHRESSLY14DTJX0XuAE4IY+fAPxO0iTSyn83ANu3SjoDuA14BdjP9rSC9w+CIAgK6EgB2L4UuDQ/v5s2UTy2XwA+NIvrjwCO6HSSQRAEQfNEJnAQBEGPEgogCIKgRwkFEARB0KOEAgiCIOhRQgEEQRD0KNEPoAOi1n4QBK8nYgcQBEHQo4QCCIIg6FFCAQRBEPQooQCCIAh6lFAAQRAEPUpEAQVBMFcR0XjNETuAIAiCHiUUQBAEQY8SCiAIgqBHCQUQBEHQowykKfwCkq6VdKOkWyV9J4//VtI9kibmx9p5XJKOkTRJ0k2S1q3I2lPSXfmx56zeMwiCIOg+A4kCehHY3PYzkuYFrpD09/zagbb/3Of8bUj9fkcDGwDHARtIWhI4BBgDGJggaaztx5v4IEEQBEFnDKQpvG0/kw/nzQ/3c8mOwCn5uquBxSUtB2wFjLP9WL7pjwO2Lpt+EARBUJcB+QAkDZc0EXiEdBO/Jr90RDbzHCVp/jy2AjC5cvmUPDar8b7vtY+k8ZLGT506tcOPEwRBEAyUASkA29Nsrw2sCKwvaQ3gYOCtwHrAksDX8ulqJ6Kf8b7vdbztMbbHjBgxYiDTC4IgCGrQURSQ7SeAS4GtbT+UzTwvAicB6+fTpgAjK5etCDzYz3gQBEEwCAwkCmiEpMXz8wWB9wJ3ZLs+kgTsBNySLxkLfDxHA20IPGn7IeB8YEtJS0haAtgyjwVBEASDwECigJYDTpY0nKQwzrB9rqSLJY0gmXYmAvvm888DtgUmAc8BewHYfkzS4cB1+bzDbD/W3EcJgiAIOmG2CsD2TcA6bcY3n8X5BvabxWsnAid2OMcgCIKgC0QmcBAEQY8SCiAIgqBHCQUQBEHQo4QCCIIg6FGiI1gQzCGik1Uw1IgdQBAEQY8SCiAIgqBHCQUQBEHQo4QCCIIg6FFCAQRBEPQooQCCIAh6lFAAQRAEPUoogCAIgh4lFEAQBEGPEgogCIKgRwkFEARB0KMMpCXkApKulXSjpFslfSePryzpGkl3SfqjpPny+Pz5eFJ+fVRF1sF5/E5JW3XrQwVBEASzZyA7gBeBzW2vBawNbJ17/f4QOMr2aOBxYO98/t7A47bfDByVz0PS6sBuwNuBrYFjc5vJIAiCYBCYrQJw4pl8OG9+GNgc+HMeP5nUGB5gx3xMfn2L3Dh+R+B02y/avofUM3j9Rj5FEARB0DED8gFIGi5pIvAIMA74N/CE7VfyKVOAFfLzFYDJAPn1J4GlquNtrqm+1z6SxksaP3Xq1M4/URAEQTAgBqQAbE+zvTawImnV/rZ2p+WfmsVrsxrv+17H2x5je8yIESMGMr0gCIKgBh1FAdl+ArgU2BBYXFKrocyKwIP5+RRgJEB+fTHgsep4m2uCIAiCOcxAooBGSFo8P18QeC9wO3AJsEs+bU/g7Px8bD4mv36xbefx3XKU0MrAaODapj5IEARB0BkDaQm5HHByjtgZBpxh+1xJtwGnS/oucANwQj7/BOB3kiaRVv67Adi+VdIZwG3AK8B+tqc1+3GCIAiCgTJbBWD7JmCdNuN30yaKx/YLwIdmIesI4IjOpxkEQRA0TWQCB0EQ9CihAIIgCHqUUABBEAQ9SiiAIAiCHiUUQBAEQY8SCiAIgqBHCQUQBEHQo4QCCIIg6FFCAQRBEPQooQCCIAh6lFAAQRAEPUoogCAIgh4lFEAQBEGPEgogCIKgRwkFEARB0KOEAgiCIOhRBtIScqSkSyTdLulWSQfk8UMlPSBpYn5sW7nmYEmTJN0paavK+NZ5bJKkg7rzkYIgCIKBMJCWkK8AX7Z9vaRFgAmSxuXXjrL9k+rJklYntYF8O7A8cKGkt+SXfwm8j9Qg/jpJY23f1sQHCYIgCDpjIC0hHwIeys+flnQ7sEI/l+wInG77ReCe3Bu41TpyUm4liaTT87mhAIIgCAaBjnwAkkaR+gNfk4f2l3STpBMlLZHHVgAmVy6bksdmNd73PfaRNF7S+KlTp3YyvSAIgqADBqwAJC0M/AX4gu2ngOOAVYG1STuEn7ZObXO5+xmfccA+3vYY22NGjBgx0OkFQRAEHTIQHwCS5iXd/E+1fSaA7Ycrr/8aODcfTgFGVi5fEXgwP5/VeBAEQTCHGUgUkIATgNttH1kZX65y2s7ALfn5WGA3SfNLWhkYDVwLXAeMlrSypPlIjuKxzXyMIAiCoFMGsgPYGNgDuFnSxDz2dWB3SWuTzDj3Ap8BsH2rpDNIzt1XgP1sTwOQtD9wPjAcONH2rQ1+liAIgqADBhIFdAXt7ffn9XPNEcARbcbP6++6IAiCYM4RmcBBEAQ9SiiAIAiCHiUUQBAEQY8SCiAIgqBHCQUQBEHQo4QCCIIg6FFCAQRBEPQooQCCIAh6lFAAQRAEPUoogCAIgh4lFEAQBEGPEgogCIKgRwkFEARB0KOEAgiCIOhRQgEEQRD0KKEAgiAIepSBtIQcKekSSbdLulXSAXl8SUnjJN2Vfy6RxyXpGEmTJN0kad2KrD3z+XdJ2rN7HysIgiCYHQPZAbwCfNn224ANgf0krQ4cBFxkezRwUT4G2IbUB3g0sA9wHCSFARwCbACsDxzSUhpBEATBnGe2CsD2Q7avz8+fBm4HVgB2BE7Op50M7JSf7wic4sTVwOK5gfxWwDjbj9l+HBgHbN3opwmCIAgGzECawr+GpFHAOsA1wLK2H4KkJCQtk09bAZhcuWxKHpvVeN/32Ie0c2CllVbqZHpB0CijDvpb0fX3/mC7hmYSBN1hwE5gSQsDfwG+YPup/k5tM+Z+xmccsI+3Pcb2mBEjRgx0ekEQBEGHDEgBSJqXdPM/1faZefjhbNoh/3wkj08BRlYuXxF4sJ/xIAiCYBAYSBSQgBOA220fWXlpLNCK5NkTOLsy/vEcDbQh8GQ2FZ0PbClpiez83TKPBUEQBIPAQHwAGwN7ADdLmpjHvg78ADhD0t7A/cCH8mvnAdsCk4DngL0AbD8m6XDgunzeYbYfa+RTBEEQBB0zWwVg+wra2+8BtmhzvoH9ZiHrRODETiYYBEEQdIfIBA6CIOhRQgEEQRD0KB3lAcxtRBx3EATBrIkdQBAEQY8SCiAIgqBHCQUQBEHQo4QCCIIg6FFCAQRBEPQooQCCIAh6lFAAQRAEPUoogCAIgh4lFEAQBEGPEgogCIKgRwkFEARB0KO8rmsBBUEQDBZzQy2y2AEEQRD0KANpCXmipEck3VIZO1TSA5Im5se2ldcOljRJ0p2StqqMb53HJkk6qPmPEgRBEHTCQHYAvwW2bjN+lO218+M8AEmrA7sBb8/XHCtpuKThwC+BbYDVgd3zuUEQBMEgMZCWkJdLGjVAeTsCp9t+EbhH0iRg/fzaJNt3A0g6PZ97W8czDoIgCBqhxAewv6SbsoloiTy2AjC5cs6UPDar8ZmQtI+k8ZLGT506tWB6QRAEQX/UVQDHAasCawMPAT/N4+2ax7uf8ZkH7eNtj7E9ZsSIETWnFwRBEMyOWmGgth9uPZf0a+DcfDgFGFk5dUXgwfx8VuNBEATBIFBrByBpucrhzkArQmgssJuk+SWtDIwGrgWuA0ZLWlnSfCRH8dj60w6CIAhKme0OQNIfgM2ApSVNAQ4BNpO0NsmMcy/wGQDbt0o6g+TcfQXYz/a0LGd/4HxgOHCi7Vsb/zRBEATBgBlIFNDubYZP6Of8I4Aj2oyfB5zX0eyCIAiCrhGZwEEQBD1K1AIKXheU1l2BOVN7JQiGErEDCIIg6FFCAQRBEPQooQCCIAh6lFAAQRAEPUoogCAIgh4lFEAQBEGPEgogCIKgRwkFEARB0KOEAgiCIOhRQgEEQRD0KKEAgiAIepRQAEEQBD1KKIAgCIIeJRRAEARBjzJbBSDpREmPSLqlMrakpHGS7so/l8jjknSMpEmSbpK0buWaPfP5d0naszsfJwiCIBgoA9kB/BbYus/YQcBFtkcDF+VjgG1IfYBHA/sAx0FSGKRWkhsA6wOHtJRGEARBMDjMVgHYvhx4rM/wjsDJ+fnJwE6V8VOcuBpYPDeQ3woYZ/sx248D45hZqQRBEARzkLodwZa1/RCA7YckLZPHVwAmV86bksdmNT4TkvYh7R5YaaWVak4vGMpE964gGBo07QRWmzH3Mz7zoH287TG2x4wYMaLRyQVBEATTqasAHs6mHfLPR/L4FGBk5bwVgQf7GQ+CIAgGiboKYCzQiuTZEzi7Mv7xHA20IfBkNhWdD2wpaYns/N0yjwVBEASDxGx9AJL+AGwGLC1pCima5wfAGZL2Bu4HPpRPPw/YFpgEPAfsBWD7MUmHA9fl8w6z3dexHARBMCiU+qXmVp/UbBWA7d1n8dIWbc41sN8s5JwInNjR7IIgCIKuEZnAQRAEPUoogCAIgh4lFEAQBEGPEgogCIKgRwkFEARB0KOEAgiCIOhRQgEEQRD0KKEAgiAIepRQAEEQBD1KKIAgCIIeJRRAEARBjxIKIAiCoEep2xEsCIJgQPRqpc25gdgBBEEQ9CihAIIgCHqUUABBEAQ9SpECkHSvpJslTZQ0Po8tKWmcpLvyzyXyuCQdI2mSpJskrdvEBwiCIAjq0cQO4D2217Y9Jh8fBFxkezRwUT4G2AYYnR/7AMc18N5BEARBTbphAtoRODk/PxnYqTJ+ihNXA4tLWq4L7x8EQRAMgFIFYOACSRMk7ZPHlrX9EED+uUweXwGYXLl2Sh6bAUn7SBovafzUqVMLpxcEQRDMitI8gI1tPyhpGWCcpDv6OVdtxjzTgH08cDzAmDFjZno9CIIgaIaiHYDtB/PPR4CzgPWBh1umnfzzkXz6FGBk5fIVgQdL3j8IgiCoT20FIOkNkhZpPQe2BG4BxgJ75tP2BM7Oz8cCH8/RQBsCT7ZMRUEQBMGcp8QEtCxwlqSWnNNs/0PSdcAZkvYG7gc+lM8/D9gWmAQ8B+xV8N5BEARBIbUVgO27gbXajD8KbNFm3MB+dd8vCIIgaJbIBA6CIOhRQgEEQRD0KKEAgiAIepRQAEEQBD1KKIAgCIIeJRRAEARBjxIKIAiCoEcJBRAEQdCjhAIIgiDoUUIBBEEQ9CihAIIgCHqUUABBEAQ9SmlDmCAIBpFRB/2t6Pp7f7BdQzMJ5kZiBxAEQdCjhAIIgiDoUUIBBEEQ9ChzXAFI2lrSnZImSTpoTr9/EARBkJijCkDScOCXwDbA6sDuklafk3MIgiAIEnN6B7A+MMn23bZfAk4HdpzDcwiCIAgApVa9c+jNpF2ArW1/Kh/vAWxge//KOfsA++TD1YA7uzilpYH/DmF5c4vMmOPQlDe3yIw5Ns+bbI+Y3UlzOg9AbcZm0EC2jweOnyOTkcbbHjNU5c0tMmOOQ1Pe3CIz5jh4zGkT0BRgZOV4ReDBOTyHIAiCgDmvAK4DRktaWdJ8wG7A2Dk8hyAIgoA5bAKy/Yqk/YHzgeHAibZvnZNz6EPTpqZumK7mBpkxx6Epb26RGXMcJOaoEzgIgiAYOkQmcBAEQY8SCiAIgqBHCQUwhJE0TNKigz2PICHpgIGMDSaSdpY0/2DPoz8kzTY+vVD+EpLWLJTxvYGMze30hA9A0irAwcDzwE9s318o7xz65C9Usb1DgezTgH2BacAEYDHgSNs/rilvGHCT7TXqzqki6wP9vW77zALZNzPz7/RJYDzwXduPdihvVWCK7RclbQasCZxi+4mCOV5ve90+YzfYXqdA5puA0bYvlLQgMI/tpwvknQRsDlxOyrQ/3/YrdeVlmeOBk4DTbD9eIivLuwu4B/gjcGZDMi8FdiAFtkwEpgKX2f5STXnt/tY32l6rYI7tFnPP2p5WV2YpvaIArgV+AywAfB7Y0/aVBfI27e9125cVyJ5oe21JHwXeCXwNmGC79opG0qnAwQ0ovpP6edm2P1kg+0ckpXdaHtot/3wK2MT29h3KmwiMAUaRos7GAqvZ3rbG3HYHPgJsAvyz8tIiwDTb7+1UZpb7aVLW+5K2V5U0GviV7S3qyKvInZdUb2vXPOdxrez7mvLeDOyV5bWUwQUuuHlIWp/0N94JuA043fbvC+TdYHsdSZ8CRto+RNJNnf7fSPoMaQG2GnBH5aVFgPG2dy+Y4xRgOeBpUlLswsDDpPyoz9i+oa7s2th+3T9IK+DW87VJK+sngA8AVwz2/PrM9VZgXuBPwKZ57MZCmReTvnQXkW6EY4Gxg/1Z+8zxylmNATfXkHd9/nkg8Ln8/Iaac3sTsBlwFbBp5bEuacVe9zNPBOarzqvOZ52F7HmB7YEzgakNyRxGWmU/AEwGvkNSXiUylwZOISnSEjk3k26uFwDr5bGbashZAnhz/v9btfJYpoHf3y+B7SrH2wI/BjYGrmnib9Tpo1daQj4saU3bN9meSFpZtygxW4wGvk+qbLpAa9z2KrVnCv8L3AvcCFyeTQRPFciD9I/aKJK2A97OjJ/7sAKRC0vawPY1Wf76pBUSQB0Txst55b4n6UYI6abYMbbvA+7Lu7IHbb+Q57ggKZv93jpygRdtvySlCimS5qEf0+JAkLQ1aWX9HuBS0s73wyUys9w1SbuAbYG/AKeSdhcXkxZVnchaFNg5z3NV4CxSocgSDiPt9K60fV02+97VqRDbj0t6CniL7X8XzqkvG9jer/Je50k6zPaBkhbo78KuMRhaZ04/gBHAcl2QewWwBXATaZV4KPCdLrxP7VVml36fvyKt2iYDh5BWXycUylwvy7mHdEO9iXRTeAPw4RryVgeOAXbPxysDBxXOcTwwX+V4PuC6Ank/Ar5OMjW8j3QjPKJwjn8gmVXmb/DvPYG0e/xIX7kkG36n8u4BjgI26sb3s6HP/AdghYZlXgh8GVghP76Ux4aTd6xz/HMO9i96bn6QbPNQ2bYD/yyUuSxwAvD3fLw6sHdNWVfkn0+TdhGtx9PAUwVzvKnPz4VJNuEmfqeLAYsXyhgO/L4Lf++JbcZqm+dIJpVPk8wNf87PVfi5L+zC516lzdjKBXM8sgtzfEtWUrfk4zWBbxbIG5f/V84nWQnOrKPs+sgcARyXFzq35OfLAvOT/FON/k4G8ugVE1C3eCFH2dyVS1w8ACxTKPO3JCfbN/Lx/5GiJU7oVJDtTfLPRQrn1Jfn88/nJC0PPEpaYdcmhy5+kOS0nadlFnENs5LtaZJGSJrPqe9EU0yVtIPtsXnOO1JWzndBUjmUX2d5w/PYc3WE5c/9nKTFbD9ZMK++/Jnk7+g79s425/ZLnmPtSJp++DXJ3/O/+X1uyhF1360p7wdNTayF7anA/5vFy90sez9LQgGU8QVgIVJk0eEku+uehTKXtn2GpIPhtfpJjYSJSVqGGW32daOCzpW0OMmBdT3Jbv2bwumdTQr7nAC8WCgLkhnpSkljgWdbg7aPLJC5L3CqpF/m48nAxwvkXQS8F3gmHy9IcmK+q0DmC8DNksYx4+f+fKeCJL2V5OdZrE8I8KJUvkc1mJj/Ln/qM8fa/jhgIdvXthYOmdrhr7YvkrQ0KZIMUgRQUe3+HE31JfIip/JeW5bILSEUQAG2r8tPnyE5yJrgWUlLkZ2BkjYk3RhrI2kH4KfA8sAjJH/F7aR/7o6xfXh++hdJ5wILNLDiXNH21oUyqjyYH8NIIXzFODkFN5S0MMlUUzteP7OA7dbNH9vPSFqoUObf8qMJVgPeDyzOdEc6JBPipwvkLknaNW5eGTMFARnAf3PuR+v/ZhfgobrCJH2Q5Kf4Jylk81eSvmj7rII5/pm0k/89KeR58BkMu9Pr5UGyEy5eOV6ClHhTInNd4ErSTf9KkglozUKZNwJLkcMNSTuV42vI2Tz//EC7R+Ecjwfe0YW/0RsalLUUybF8PWmncjSwVIG8K4F1K8fvBK5qYJ4L0qBNmSHsrK3McRWSQ/U5kin2CmBUgbwbgWUrx8tSHo49KI7e/h6xAyhjaVcyS51CyIp8ALavz4lmq5FWHnfafrlwni/bfjSXlhhm+xJJP6whZ1NS2F+7pKzSFdwmwCck3UMyAYmUXFYrAU7SRqTV1sLAStnu/Bnbny2Y4+mkDNsP5uOPkvwztRLBSCbEP0lqNUVajpRsVRtJ2wM/IUUorSxpbeAw18hOl/Rzpq+oZ0qAcg2zUpb1FrID1PYaOcR0B9t17fXYvht4r6Q3AMNcvjsbZvvhyvFUykvnnK3U8vYsKmZO26Vh3rXpiUzgbiFpArCzsy09x+yf5T4p5B3K3A84taVYJC1BCmU8tkDmhaTQwB+QVrGPkJJlOrY1Z6f3LrbPqDufWch9U7txpxj8OvKuAXYhJbytk8ducUFJDEkTbL+zz1hRW7+ctdtS9neUKvv8ndwcuLTyuW+2/Y4asvr1Z9k+ueYcLyM7bEv/NpL6LfXgmj4fST8F3saMmel32P5KHXlZ5uQ2w7a9Ul2ZpcQOoIxvAFfkLzTAu5ne0L4un7bdcjK2dhWfBmorAGBHknPwC6RV62KkxJmOsf1qjnhqRAFIWjSvgEpXbDNhe3Ifp2Cp3fUSSbsx/bPvQg17u6TNbV+smWsrjZaEy5yhr9h+ss/nrrXKq3uDHwBNOmybjnBr8RXgQ6SdqYCTSTb82tgeOfuz5iyhAAqw/Q9J6wIbkr4kX3RhpAAwTJKct2Y5NHC+wnk+K+mNpMSqx0h+io6Kq/VhnKSvkMwf1SiOx2rIOo3kaJxAulFV7wom2XbrMFnSuwArtR/9PMnxXcJnSFEcvyPNcxjJaf8l0kpuoJVbu2lKu0XSR4DhOVP988C/CuS1qnd+jZkz3jef5UX905jD1nbjWe5ZriVdQvp+v0qKAqqlSCVtavuyHIzR7r0GrS1umIBqIOmttu/IN/+ZsH19geyfkKJ0fkX6B9kXmGz7ywUyPwV8m3TTEekGdJjtE2vKu6fNsF1WAqNRcgjf0ST7vEjhlQcUKr5GkTTcDVeCzFFE3wC2JH3u84HDnctX1JR5AUnZf4X0fdyTVF/oazXlrUIMF8h/AAAgAElEQVRy+r8LeJyUGfwx2/cWzHEBYG9mLk9Sq0ChpL1Iu+TLSL/HTYBv19kVSfqu7W9K+l2bl227JJS4iFAANZB0vO198gqhLy5YGbVs7Psw443rNyU3Ckl3Au9q3fxymOm/bK9WV2Y3kLQCSflVY6QvH7wZzYikjUnZwM9K+hgpYutnrplPIel+4B+km+vFdVeY/cgfToqCKnIytnwfqlTXlHSZ7U0L5TblsEXSn0glNT5CunF/FLjddq1+Dfl/ZhOn5K3WguLKkv+ZHIDxat3ru0K3wote7w/S9n/jhmV2q4TBRcxcw6Z2yQBS8ts3yaGkwGjg/YVz/CEpees84Jz8qF2xlJR2/3XSSvPE1qNwjjeRlPJa+fkBpJrzdeUtSCrUdmb+7L8g3XRK5ngaKVHrDaQb4kPAgYUyr84/zwe2A9YB/l0g74A8R5ESCK8HtiycYyvEuVWeZF6SUq0r72Jg3srxvMBFhXO8j+TL27RETpOP8AHUxMkZ+hNgowZldquEwQPANZLOJpmVdgSubUVQuPNIiZNINvtWFNEUUlbnuQVz3IkUu95EFjCkzOJ/kmLDmzKzvGLbSiUgjrZ9wuwiZfrD9vMkh/IZOdrraJLJYXjBHFe3/ZRS5dLzyP0kSFnbdfmupMVIhcx+Trp5f7FA3idtHy1pK1LplL3IPQYKZLaip56QtAbwH1LGbV3uB66S9FfS/8xOwHWSPg9g+5gaMlcn/e99Wam3xljgj7avKphnEaEAyrggZwye6aziG+Bemi9h8O/8aHF2/lk3gmJV27u2YsNtP68+IR01uJu0ympKASzkmjbqfnhaqUTHx4B3ZxNLrRLTLXLOx66kBi7XUV66ed4cWroT8AvbL0sq+m7abin2J0lJhKW0vivbAifZvrGB78/xWYl+i3RjXTg/r8vk/Gi11/xH/lm7naXtZ0k7tNMkLQn8jJSwVqLwiwgFUMaXSFvtaZKeZ3ryUkkf326UMPgOJJtr/hKW8pJSLfxWFMeq1LxxV5KNniPViLmIGZNkaiUbkeoVbWv7vJrXt2NXko15b9v/kbQSBSvr7EyfSNoFHNjQ36axfhKSvmr7R9WEsCoFf5sJ2bG8MnCwpEVIkTa1sd2qRXUZ9SPHqvK+Ba851Z13a8VkP9KupMi3iaTv06ARTuAhSv6nsCu1YgpkvZYVa7s4K1bS+0g+gNVJ2/aNgU/YvrSGrG4lGz1NUs4vkswDTSjnxsi7h2+4rInOQN5HwHDX6AssaXvb58zqb1TwtxlGaiJzt+0nclDCCrZvqjNHkt3/vnz8bVKm9n2kqK92EWsDkbsu6X+mteJ/GPiUC9o2Srqb1PHvDOCvbsD5XcxgOyHm5gfppvIx4Fv5eCSwfqHMNYAbSF/g+0j227cXyrwmz63aevCWQplLkRyC7yeVxGjy97oEhfWPuvT3/gCpy9STNNNX4ZIuz/fcwf6dDWCOhxZefxPJ3Ef+Lv4fqabSpyioy0XaQb2ncrwZ5bWAlhjs33ffR2lti17nWJITuLWNe4bU97OE44Ev2X6T7TeRHG+/LpSJ7b5p6CVhpeuSwjUfIpmrVpK0qlJLw7oyL5W0aLaN3gicJKnE74GkJSStL+ndrUeJPFIHrx1sL2Z7UduLuGxH8S9Jv5D0P5LWbT0K51hlhSaESBqnVP67dbyEpPObkE3qMVyCbbf6J3yA1JlugpNJqLa9HnjW9mth3k6729Ld+IKS/iTpofz4o1I/jUEjfABlbGB7XUk3wGtlG4qydklx2zN88XK8dAlNZ8UeS4qBb4VFrpGfLyVpX9t1ojkWc4pe+RTJMXiIpI5NAi2ynANIPXsnkrK1r2LGEsSd8rDt0mziKq0oqqoZyJTNsUptc0UfRrjhoocVSp2/UirP/RypPWu1ZEpJz4JrlPo+/IH0N9mVVApkTUgNZ2rIPIlUTuJj+XiPPLZVwTyLCAVQxsvZlttyho6g0JkF3C3pW6RyA5C+LLXsmBX2JYUYrkAK2bwA2K/fK/rnXpIj9FYASauTinsdToppr6MA5pG0HCkK5huzO3kAHEDqM3y17fcoNTcpLRswXtIfgb8yo6O6VukG201E1MxEdtCv5JpZsG2YJmklz1j0sCnnYcddxfrwM5KCf4qU+DUeQNI6FPQDYHojmL7VaDclffY6u8llnbu/ZX6jVFdr0AgFUMYxpNKuy0g6glQc7JuFMj9JulG1biqXU9BsJiuoPWx/tHBeVd7auvkD2L5N0jq27y6I5juMlGh0he3rlMoF3FUwxxdsvyAJSfM7le4ozXxelLTSrHZwql27R9KywPeA5W1vkxXpRrY7bv9ZkdlYOegKjRY9VIPloG2fmM1Ry5BMhy3+Q8H/je3/qXttPzymVEzwj/n4w6TaXINGRAEVkleWW5C2shfVNRFImsc1IjUGKPtS25s1KO+PpC/u6XloV2Bp0pb2CtvrNfVedZF0FukG8AWSSeVxUmbntoM6sQqS/k7u/2x7rexDucE1SjdXZLYrB/1aCYcCuUszvejhVS4oeqgGy0F3C0lfbzdu+3sFMkeRTFQbkBYOVwOfc81IpSaIHUANsqOyxSMkO+Frr7leVcxryY23Jf3c9ufKZjkDV0r6BTNX76xbtO4TwGdJN1eRklm+Qgq37Mis0a1Yc9s756eHKtVsWozpyTwd0cV4+G70f25XDrqIHEq6NbCK7cMkrSRpfdvX1hTZaP/eLlH9OyxAini7dRbnDginYndDZgECoQDq0rd0ceumIOqXMK7+N2xcf2ptadTZ6JQU89P86EunkRK35Z/j68ylPyRtAoy2fVL2z6xAPX9Ka1fX9Bwb7/9MF8pBk1atr5K+L4eRwl//QvKx1KHR/r3dwPYMHfOUOuj9tUSmUuLg/szcFL5vX4g5RiiAGtheuRtiuyAzCW7Y2ZhvLN9n5vrwdRTfrqQaQovbPrqZGYKkQ0iOvNVIZpZ5Sc24O1auts/JP19LfJL0Rtv/KZzml0hlC1aVdCUpbHGXQpmfI9nsXyTtTM8nOedLaDrabT9SuPNbJT1AUspN+qi6wfzAqoUyxgKnkHqJD4mqoKEACpC0M6ni4JP5eHFgM9t1VgpvzWGPIt0QWmFmRb1x28z5XNvvLxRzEnAIcBTJ5LMX9cP53pmjSj4p6ZS+cmqa0wB2JlWtvD7LeVApu7opziOb7OriLvR/zjHx3wC+oenloGv3Asg0He12n+0m+/c2TlZ2rUXZcFK/5tr2/8xLLqvp1TihAMo4xPZZrQOntPZDqLdVfFtz0+qXJpKDFrR9kSQ5peAfKumfJKXQKb8i2eZXIZnWmuoI9pJtKxdCayCXoi/FRnZJHwL+YftWSd8E1lVqHlLSUOg0UtjvNNLvczFJR9ouqQbainZbtqFot3skvdYHoUBON6nuxF4B/uPySrU/z3/n85kxjLh2vkspoQDKaJdJXet36jbNzyW939MrMTZFE8lBLyjVc7krxzE/QArD6xinsrrHSDrO9v9rYG4tzpD0v8DiSj2VP0kDGdUVmpD1Ldt/yr6KrUjhm8eRokTq0ng5aNun5uiiLfLQToUJcauR2mHuB5wg6VzgdNtXFMhsmmnAg7Zfyn+f7ST93mXNdd5CKlGxDdN3UHVzChohSkGUMV7SkUplEFaRdBTpn60pGikUJum1rkit5KDqWA2+QGoK83lSIs8epDaBtane/CXVjjGvyPsJKevyL6Qbzrdt/7xUrlK5hs+TnKylZRtakSbbAcfZPpvC/s/MWA767GxSasK/tBDJFDKM1MimNraft31Gdn6uQ8qvuGw2l81p/krKnF+VZLd/G6mUcwkfBkbZ3tj2/+THoN38IRRAKZ8DXiJtZc8Anqcsw7YvTcXytbs5f6KuMNvX2X7G9hTbe9n+gO2r609vJvZtQojtcbYPtP0V2+NK5SlVmjyZVAhvaVK9ohJTyAN5l/Jh4DxJ81P+P9kqB/0GCstBt6h87iVp5nMjaVNJx5J8NAtQ3gehaV7NyvMDpLafn6PcfHoTDZV4b4pIBGsIScvZbjSUrTDWGqWGLR8hNbT+Z+WlRYBptt9bU+5bSIk8ffv3NlLDRtINrQShGtc+zfQQ3eqXu7gctKTbgXVaTlWlkgvX267lv1GqNb81cLPtu5RKYbzD9Wop9fc+RUmGXfjc1T4IY91MH4RGkXQtyWz2LZLJ6+7SZDVJF5NKS1zDjD6ACAN9HfA3CqJCJLX9EkhaEWrXm7meFF+9NDPG7D9NWo3U5U8k5+2vaa7dYpXt615ou5srrHtJq9VWVM38zNhprSNyxM6ZkMxeto+nMB4+r9bbUWJOvJcGPzewVqEtfU7wSVKy44/yzX9lKgmfNTmifFrNEjuAhihZtebrT8pPlyElbrWiI95DSuvveJUg6focv/172x+b/RUDljvBdmkRr74yv9Rm+Elggu2JNeRtCNzaCjFUqhj5dtvXFMzxr6Tkp3Gk3cX7SFnQj0BRRvBrf6u611fkfLlyuACpRv7tLigM19Tn7mJGdVeRtGbTkTqStrZdKzO9SWIH0BxFUSG294IUp0+K5HgoHy9H/R4D8yl1c9qo3Q6j5q4C4BxJnyWFBla3siWFrcbkxzn5eDtSj9x9Jf3J9o86lHccM+7Inmsz1iln5UeLSwtk9aURf4/tGbKzJf2ElIBUQlOfu1sZ1d3mtxTmfLThe9QsTdIkoQAKUWpEPRK4uhUVUhLHTYoSqJoBHiaFj9VhX1KG5eLMbFapXcWS6U7lA/vIK+nFuhSwrnMLzJxP8WdSiNwEUjOWTpAr21vbr6qgYU2WcXLOgG39PWonbuUw2l1sn5GHapu9ZsNCFPbI7ZMBvQQwss6KuJVRDTxn+0/V13JOxFClucJK3ZXZMaEACpB0OCma5t9M39KWNvS4VKm8basRxW7AJf1f0p4cV32FpPEuKDHcRm43SmGsRIqoavEy8Cbbz0uqk4Bzdw7XPC4ffxa4u2SCkjYjRcPcS/oHHilpT9uXdyorK6T9SY5QbE8pmVtljjczYwbrCArDiSVdSurcNQ/JeTtV0mW225ntBsLBJD/S7MaGCh2XqW5HH2f8Z9uMzXHCB1CApDtJURsvzfbkzuTuzPTkkMur2cYFMtdg5to9pwwhed8ilW84Ow9tTzJd/BQ43h32M1DqWHUMSRkbuAj4gu1HCuY4AfiI7Tvz8VuAP9T1h+TP/DwzV2mtbUrLYZ8tXiF1MSu6wbT8W0pd1kY6d2tzh+VJJG1Dqob5YabXxIeUB7C67fVL5tk0kt5IWphUI91qF9Zr5+dpyvdTl9gBlHELybxS+6YyC64HnrZ9oaSFJC3ignop2ZyyGemGfR4pE/EKUoLLoMsDsH24Un38jUmr632duztRo1BYvtHvVnc+s2De1s0/v8f/5aSrurQcs9XckSJTmu37JK0FtBqaXE5ZxBc0163tQZL9fwdmTJh8GvhigdzGkfQ9Uje+O5ge6WZqlHPOi5HlSD2B38F088+iJBPdoBE7gAIkjSGtWG9hRmdo7e5LuWzBPsCStldVqrz5K9tbzObS/mTeDKxFajayllInqt/YrmV3blpeRe5wYFlmXHHdXyIzy20qwuZE0k2g1a7zo8A8LQf+UCBneH+a6f6dnUk7qNpZ0Nk+/y1Ss5/PKnVr+7HtD9aUtyip6fq0fDwcmN/Tm7sPOnl3v5bLC+khaS+Ssl+bVIqlpQCeAn7b1x8yJwkFUICkW0mZlzdTqY5ou3Zau6SJwPrANZ7eLelml3WJutb2+tmE8R7SiusW228fCvKyzM+Risk9TFpxNVYFtTREtyJnftJqfZM8v8uBY12zSFhOBPsSqX/vPlnZr+aC+k9KVWQ3aiVXKRXBu6qJ32NTSLoaeG/F4b8wcIHtd/V/5ZxDqVjdB5pSSlnJfbDi9B8ShAmojP86FTNrkhedClAByUlEeS2X8Uqlqn9N2no/Q+pANlTkQWrivprtRwvltONvTQjJN/ojgSMlrVsY7QWprPYEpjfsmUJyhJYUABQzJue1lGkjNLSbWqB18wew/UxWhoOOUj0vkxY1N0i6kBl397Uc37an5UVOKIDXERMkfZ/krKx+SUpuDJcp9SNdUNL7SNEC58zmmlmipEm+b/sJ4Fd5ZbNoSWKL7c/mp43Iy0ymvBvWa+SV7/O2XwVOkbQD8Pe6YZtt+A3lseGr2t5VqWQHOeKp9GZ9EnCNUk9kSEXhGosAoxll8mxVgUp6J8kZPhS4Jf+8lebj9M+X9AVmdvoPWlZ0mIAKUOo12xe7oCZOjg/fG9iS9M92Psm+XvsPpe5k7q7JzK3t6uYVIOkEUtXOvzGjMq3VQCObp/4HWILUfHs8Kf68kc5TTZiVJP2LVGL5SqeM7VVJUUVF0TD5htpypl9uu1YJcEnz9zVvKfUrKC0Etx5wOskpDMlBuqvtJivpFiFpAVJPiVfz8TBgvhKfgKTJbYZte6W6MksJBdADSPolydl0XUPyTiQVtbqVSl1zl5UbaNtMxvZ3asprlcH4HKmBzY+a8gVk+Tu5Xue3qoz3kRqrrA5cQLppf8L2pYVyG3GmV36Hv7O9R8mc2siel+md0O5ocGfWCJKuArb09FIiiwDnDyU/RROECagGal+35jXqrlqz7PeTeri2Km0WV7EkOWo/I+k+0taz1MG6oe3VC+YzE3Vv9P0gSRuRInX2zmNF3/dsnvkosIrtw5SafL/RNSu22h4n6XpgQ9Lf5ADb/y2cY1tnOklhd0qrlMi71GApkYrz+022Py1ptKQi53cXWLAaem376VI/Rfbn7cP0HJ9LSbv7QUsECwVQj1bFydVIRbJatVa2J0WGlPAzUg3ym0vMPn3YpiE5La6StLrt20oFSfqZ7S9IOof2BcLqhtQeQMouPcup5eIq1MyornAsacezOSm79mlSw5n1CmQuADxO+l9cXRKukVlcoUlnerdKibSc3xvl4yac303znKS1bN8IIGltpldDrcsvSX0aTszHHyP5kYobINUlFEANWqtVSReQ6te0tomHUp7OPpkUUll888+21qVt/73P+PYk++tMbSgHyMkkJfAfkr2+ZEfRiqn/Sc25zEQ2gWxfVR627yZ1MCthg2wSuSHLfFypNlDdef4Q2JU+pjTKFhGNOdPdpVIidMf53TRfBM7Ku2ZIGcG7F8rc0PZaleMLJN1YKLOIUABl9K1f8xLJMVrCV0ndoS6j3Bn6Y9p3/rodOJ76NYtOJLWBnCH/oQ4tx19J7kQbmdOyI7RpXs7KpdVofgRln38n0mq9tNl41Sx5N6meVCPO9MzvlOoqtUwXl5GSE+va7V9SairT+j2uWp3rUMD2NZLeRmoFKVJp8dKSL69KGmX7XgBJoyj8/yklFEAZvwOuzSF3JmVd1i6HkDmCFFe/AOX9YZdqfdmq2J4kaakCuffbLi0xPANd8H3cIGksaUdWDbmrHalEqi10FrCMpCOAXUhO3LrcDcxLMze/llny/vyYj/LvT4tjSfM8Nh/vQSqy96ma8g4lhViOlHQqyfk9ZLKpAbKCOoBUnXdfSW+WNLrvbrpDvkpq0/l/pO/3m5nunxoUIgqoEKUS0K/VXakbcleRN972mPKZgaRJtt/c6WsDkHssyS58DjOuMkvCQCfRoO9D0xvsVCmKVMpy30oK3RRwke3bZ3NJOxmthigrkEpqXMSMv8dGGqNIeqPt/zQg58Y+pou2Yx3KXIrpzu+rS53fTSPpD6Qd7kdsr5EdwFc2EPq7INN3FbfZHtT8h9gBFJKTWa7PiUc75zjp7QpEXihpSzfTF/bCvFL9ZvWmKuk7TO84VocFSTesLStjJU5BaND3AdMb7HSBu0g1XOYBkLRSjRDLVpG7CczcrKXJFdl5NNPIZJqkVW3/GyA71Gu3ApV0kVNtq7+1GRsqjLa9u3KfAtvPlfoplEqJfJpUSsTAPyX9ugkTYF1CARSQHYDbkhqvb02KCPlVodj9gK8q1cB/mTJTyJdJGauTlGoMQVpxjqf+9r1bN9cmfR+tUs3HAcvmFdyawA62a9d2byrE0rnBiqQDbB/d5z0OqDu/NjTlWD0QuETS3Vnmm6hhssnJVQsBSys1lqlWxVy+obk2xUt5vi0/xcrM6O+rw8mk73are+DueazpqrUDJkxANcgJPLsDW5FCC/8I/Nz2qEK5ItVbL66A2UfuKkCrUNutOSKmKdlNVdq8gOT76FtYr24i2GWkG9f/enpRvVtsr1Ewx0mkSKBG6hW1+901nKz2WdvHzv7MfmUMI5lqJjBj4lbHq9as3L5Autk/WHnpKeDXtn9RMtcmkbQ1cBApSe/vwKbA3rYvKpA5Uw+FUlNaKbEDqMf5wD+BTWzfAyDp6P4vmT22nR3KjUaw5Bt+Yzf9PjS1ylzS9pazP23ALGT72j679tKEm0ZCLHP440eAlbOjusUiQJFyyd/DP9r+V+nNH17rXPZT2xtR2Fcg73aOlvQ5F5Sn7iYtk57tfyiVE3kX6Tt+oAuaCWUmSlrPOSM/R6pdVSiziFAA9Xgnadt2Yd4Wn05qv9cEV1e/JHMBjVTapFnfB8B/c3hhawu/C/BQ/5fMlqZCLP+V57I0qeNZi6cpb95yPfDNbAI7i6QMSpuwXyDpg8CZDflo/rdPWOmlpJ3aUCgH8Vey38T2VKZ3qGuCdUn/3/fk45WBW3NeiZvYSXdKmIAKkbQxyRz0QVK/1LNsH18g7zZS0/GmyjY0TraHPuRcGCtHNizbLuS0A5lPk7Ikm/B9tMxex5NWcI8D9wAfK5xj0/WKViGZGAzc3rBpbknSd3I3Ur+B0QWyWn+bV0jZsKV/m9+Qwkpbzeb3AKbZru2XaoomTXBtZK/a3+stJ/ucJBRAQ2Rb6fuA3UqcpJqxp+tr2K6btds4ksYD72olxmRn+JW2a5VE6JbvI8t+AzDMBS01m0apI9ZvSDvJG0k31LVIdva93UB5YEnrk7KMdyKFGxZ1a2uSboSVNoWkR0g7+raUhOjmxK8Hnfp9bEIKHvh9E3/vuoQJqCGcysaenx8lcl670eeb104ke3FJaGnTzFPNisxf6NpJR036PjSLQn0tX0CdqCI1X6/oGOA20mKhVW5YpLaLvwA+3ukcK3P9ISmf4t+k5iOHO/WCqCNrGeDrpISlm4AfNHSzajSstGGeZ8Z+xU3yV2C9vBM4hWQ+PQ14f5feb7aEAhhidCm0tGmmStrBORtY0o5AaSJPU76PRWZ/Ssc0Xa9oY9ufqA5k2/phku4qlH0PqSVkE4lVp5Buhj8n3aSOoX1pkU5pJKy0SzzaCtPtAq/aflmpsurPbB+T7f+DRpiAhgjdCi3tBnkFcyopk9Wkao4ftz2pQOaQ9300hfrP0L6rjr1e0ltt36GUmT4TrtGlTtJE22tXjotDfpsMK+0Gkq62vWGXZF9Lqs/1LWAn23eXhiYXzykUwNBA0quk0NJPVEJL77a9yuDObNYoNfNWE/b1pnwfkvrt0Vxow92YVMemb72ijv5Gkk4mmWgOr0bVSPoW8BbXaL4i6XinxvKNdalTqlS5GdNDfS+pHtt+rFOZWe5VOay0p5C0BqnF679s/z4HU3zE9hGDNqdQAEMDSeuQIjZ2IYUbng5823bbG+NgImlZ4HvA8ra3kbQ6yexQXDI4250XaB136hhWamACqcDY6qSdFMCHgAm2v1gwtztIZYInULFZd5oYlp3AJ5DCAieSdlHrADeQnMC1cg3y6noj21fWub6NvHtJSXntcj06VnwVud8h+RSaCisNahIKYAjSdGhp00j6O6mpxzdsr6XU6egG2+8okLkDKSZ+eeAR0ir7dttv7/fCWcu7hNTS7+V8PC9wge33FMzxGtsb1L2+jbxVSUqqVW64OAxwblhdNx1WOrcgaUPg20zvpd363G8ZtDmFAhi6NBVa2jSSrrO9XjVmuq+9uIbMG0n9CS60vY6k9wC7267VLUnSnaTV8GP5eAlS1cnVashq2b0/TEr4O5MZE8E6tq93i1hdD10k3U6qedV3B/nwYM0pooCGME2FlnaBZ5XK+baybDekvETCy7YflTRM0jDbl+SQxrr8gNQToGUT35Rkv6/DT/scV8t1m/qNdbrBl8ira0lDanXdxbDSuYWnbJ8z2JOoEjuAoGPyivjnwBrALcAI4EPO/VNryryQlPPwfVKJhEeA9Wy/q0DmG4GWyeYaN1AbP6iPpH+QVr+Xk8JKF+kbDvt6RtL389O+O8jS8h+1CQUQ1CLb/VthfHe6sI5LTnprrVg/CiwGnNqpg7WPzBWYHrEDUNRwXdL3gB+1EquyWenLtku6gjXC3LC67kZY6dyEpH+2Gbbtd7cZnyOEAgg6RtLhwKG2p+XjRYGj6/gpJG1o++ouzLFtw/UaWbtVmTPViRkqN7G5YXXdrbDSoD7hAwjqMA+pF/JewBtJ5qC65X2PJVdfbDiCpbGG6xWGS5q/JVOpCN78Dcov4Y22v5Gfny9pyDimKyxGUlLVsNLWPA0M2ZyXppC0Fak3RzXU+XuDNZ9QAEHH2D5Y0kXANaRKm+8uyAKu3gwWmOVZndNkw/UWvwcuUuo3bOCTTK9oOdhIM3bZGl49Hgqr66GY1T4n0fRe2u8mhVF/EGh899vRnMIEFHSKpHeT2i3+HngHsCTwSdsP9nthe1kts8AwUp/izagohYJs07/QhYbrkrZhelP4C2wPiQitbiVtBc2h3BFMufKppEWAv7jZRkgdETuAoA4/IUX93AaQi1tdDLy1hqy+ZoGq6aLELDCWmRuuF2P776QWgUOKXl9dzyW80PqZI9QeJSWFDRqhAII6bNRyAAPYPlOpB2/HdOvG5S5UdMz5Dj8H3gbMR0oKe3YoxNgHcwXnSVqctICaSEoGG1QT4rDBfPNg7kLSzwBsT1Nq8F2lb7LUoCJptKQ/S7pN0t2tR6HYX5BKdNwFLAh8ivrO76CHyFn9f7f9hO0/kT3psr0AAAIWSURBVNpBvsP21wdzXqEAgk6oxivv2ee1oVa2+SSSn+IV4D2k2va/6/eKAZCd3cNtT7N9UpYdBP2Ss/qPrhw/PxQc86EAgk7QLJ4PRRa0fREp0OE+24dSXrLhOaWGPRMl/UjSF0llF4JgIIxTap40ZAgfQNAJw3Jo4bDK89fCDgdvWm15IW+775K0P/AAsEyhzD1In3N/UlnokaRQviAYCPsDi0l6kdR6slWnacnBmlCEgQYDZm4KNZS0HnA7Ke76cFK00Y+6kXUcBP0haSXb90tqu0iqBlTMaUIBBMFskNRvsS6/DttWBs0xVMqFtCNMQMHrCknnkMtUt6NmLaBXs8zTgHNI2/cgGChD1l8WO4DgdYWkTVtPgV+TQjVfw3atfAVJbyWFgG4P3EZSBhfYfqX+bINeQNIjpBavbSnNTi8hFEDwuqVd9c6G5O4K/BL4oe0fNy0/eH0h6T5SK8i2dCNpcaCECSh4PdPY6ib3FtgN2JlUAO+LwFlNyQ9e1zw6mDf5/ggFELyukFQNqRveJ1S1VnG5XOZiEeAM4BNAS8Z8kpYcCgk9wZDmpcGewKwIE1DwukLSPaSVf2Ohqjn8tfWPUv2HacVxD5nw1yDohFAAQRAEPUqUggiCIOhRQgEEQRD0KKEAgiAIepRQAEEQBD1KKIAgCIIeJRRAEARBjxIKIAiCoEcJBRAEQdCjhAIIgiDoUf4/P/jVGQNj3FcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEmCAYAAACJXlw1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAERlJREFUeJzt3X+s3XV9x/Hni1akczLKKARbpM51KLqNaQNN+EdhKwWjxUQjmI0OmTUK29zMYuf+qJEZcZtbglMmzirMH4hOR1W01oZoNsRRNkJBxtohQinSahFxqFh474/zvXrgc9t7e1vu97DzfCQn53ve5/P93vc3Obmv+/1+P+d7U1VIkjTskL4bkCSNHsNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNB2g9J3prk3iQPJbkjyelJDkmyJsn/JPlekquTHNmNf02SO5Mc3r0+M8l3kizod0+kfYu3z5CmJ8kJwFeAU6pqR5LFwBzg5cA5wKuAXcClwOFVdW633seAnwJvAbYAq6vq87O+A9J+MBykaUryq8D1wGuBr1bVT7v67cBFVbWpe30scDcwr6r2JDkCuAV4ELi+qt7Qyw5I+8FwkPZDktcCbwJeAGwA/hTYBuwBHhsaehjw3Kq6t1vvPd3YE6rqv2e1aWkGDAdpBrprCB9gEAonA6+rqn/by9iTgOuALwJHVtWKWWtUmiEvSEvTlOSEJKcleTrwY+BHwKPAPwDvTHJ8N25BkpXd8mHAR4G3AecDC5O8qZcdkPbD3L4bkJ5Cng5cAjyfwQXm64HVwHeAAF9O8ixgJ/BJ4BrgXcD2qroMIMnvAtcl2VhVW2d/F6Tp8bSSJKnhaSVJUsNwkCQ1DAdJUsNwkCQ1DAdJUuMpO5X1qKOOqsWLF/fdhiQ9pdx0003fraopb/z4lA2HxYsXs3nz5r7bkKSnlCTfns44TytJkhqGgySpYThIkhqGgySpYThIkhqGgySpYThIkhqGgySp8ZT9Epz0/9niNV/ouwWNqLsuedms/ByPHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJjSnDIclxSa5LcnuS25L8cVc/MsnGJFu75/ldPUkuTbItyS1JXjS0rVXd+K1JVg3VX5xkS7fOpUnyZOysJGl6pnPksAd4S1U9H1gGXJjkRGANsKmqlgCbutcAZwJLusdq4DIYhAmwFjgFOBlYOxEo3ZjVQ+utOPBdkyTN1JThUFX3VdV/dMsPAbcDC4GVwBXdsCuAs7vllcCVNXADcESSY4EzgI1VtbuqHgA2Aiu69w6vqq9XVQFXDm1LktSD/brmkGQx8FvAN4Bjquo+GAQIcHQ3bCFwz9Bq27vavurbJ6lP9vNXJ9mcZPOuXbv2p3VJ0n6Ydjgk+UXgn4E3V9UP9jV0klrNoN4Wqy6vqqVVtXTBggVTtSxJmqFphUOSpzEIho9V1We68v3dKSG6551dfTtw3NDqi4AdU9QXTVKXJPVkOrOVAnwIuL2q/nborfXAxIyjVcA1Q/XzullLy4AHu9NOG4DlSeZ3F6KXAxu69x5Ksqz7WecNbUuS1IO50xhzKvB7wJYkN3e1twGXAFcnuQC4G3h19961wFnANuBh4HyAqtqd5GLgxm7cO6pqd7f8RuAjwDzgi91DktSTKcOhqv6Vya8LAJw+yfgCLtzLttYB6yapbwZeOFUvkqTZ4TekJUkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1JgyHJKsS7Izya1DtbcnuTfJzd3jrKH3/jzJtiR3JDljqL6iq21Lsmao/pwk30iyNcknkxx6MHdQkrT/pnPk8BFgxST1v6uqk7rHtQBJTgTOAV7QrfP+JHOSzAHeB5wJnAic240FeHe3rSXAA8AFB7JDkqQDN2U4VNXXgN3T3N5K4Kqq+klVfQvYBpzcPbZV1Z1V9QhwFbAySYDTgE93618BnL2f+yBJOsgO5JrDRUlu6U47ze9qC4F7hsZs72p7q/8y8P2q2vOE+qSSrE6yOcnmXbt2HUDrkqR9mWk4XAY8FzgJuA94T1fPJGNrBvVJVdXlVbW0qpYuWLBg/zqWJE3b3JmsVFX3Tywn+SDw+e7lduC4oaGLgB3d8mT17wJHJJnbHT0Mj5ck9WRGRw5Jjh16+UpgYibTeuCcJE9P8hxgCfDvwI3Akm5m0qEMLlqvr6oCrgNe1a2/CrhmJj1Jkg6eKY8cknwCeAlwVJLtwFrgJUlOYnAK6C7gDQBVdVuSq4FvAnuAC6vq0W47FwEbgDnAuqq6rfsRbwWuSvKXwH8CHzpoeydJmpEpw6Gqzp2kvNdf4FX1TuCdk9SvBa6dpH4ng9lMkqQR4TekJUkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEkNw0GS1DAcJEmNuX030IfFa77QdwsaUXdd8rK+W5BGgkcOkqSG4SBJahgOkqTGlOGQZF2SnUluHaodmWRjkq3d8/yuniSXJtmW5JYkLxpaZ1U3fmuSVUP1FyfZ0q1zaZIc7J2UJO2f6Rw5fARY8YTaGmBTVS0BNnWvAc4ElnSP1cBlMAgTYC1wCnAysHYiULoxq4fWe+LPkiTNsinDoaq+Bux+QnklcEW3fAVw9lD9yhq4ATgiybHAGcDGqtpdVQ8AG4EV3XuHV9XXq6qAK4e2JUnqyUyvORxTVfcBdM9Hd/WFwD1D47Z3tX3Vt09Sn1SS1Uk2J9m8a9euGbYuSZrKwb4gPdn1gppBfVJVdXlVLa2qpQsWLJhhi5Kkqcw0HO7vTgnRPe/s6tuB44bGLQJ2TFFfNEldktSjmYbDemBixtEq4Jqh+nndrKVlwIPdaacNwPIk87sL0cuBDd17DyVZ1s1SOm9oW5Kknkx5+4wknwBeAhyVZDuDWUeXAFcnuQC4G3h1N/xa4CxgG/AwcD5AVe1OcjFwYzfuHVU1cZH7jQxmRM0Dvtg9JEk9mjIcqurcvbx1+iRjC7hwL9tZB6ybpL4ZeOFUfUiSZo/fkJYkNQwHSVLDcJAkNQwHSVLDcJAkNQwHSVLDcJAkNQwHSVLDcJAkNQwHSVLDcJAkNQwHSVLDcJAkNQwHSVLDcJAkNQwHSVLDcJAkNQwHSVLDcJAkNQwHSVLDcJAkNQwHSVLDcJAkNQwHSVLDcJAkNQwHSVLDcJAkNQwHSVLDcJAkNQwHSVLDcJAkNQwHSVLDcJAkNQwHSVLDcJAkNQwHSVLDcJAkNQwHSVLjgMIhyV1JtiS5OcnmrnZkko1JtnbP87t6klyaZFuSW5K8aGg7q7rxW5OsOrBdkiQdqINx5PDSqjqpqpZ2r9cAm6pqCbCpew1wJrCke6wGLoNBmABrgVOAk4G1E4EiSerHk3FaaSVwRbd8BXD2UP3KGrgBOCLJscAZwMaq2l1VDwAbgRVPQl+SpGk60HAo4MtJbkqyuqsdU1X3AXTPR3f1hcA9Q+tu72p7q0uSejL3ANc/tap2JDka2Jjkv/YxNpPUah/1dgODAFoN8OxnP3t/e5UkTdMBHTlU1Y7ueSfwWQbXDO7vThfRPe/shm8HjhtafRGwYx/1yX7e5VW1tKqWLliw4EBalyTtw4zDIckzkjxzYhlYDtwKrAcmZhytAq7pltcD53WzlpYBD3annTYAy5PM7y5EL+9qkqSeHMhppWOAzyaZ2M7Hq+pLSW4Erk5yAXA38Opu/LXAWcA24GHgfICq2p3kYuDGbtw7qmr3AfQlSTpAMw6HqroT+M1J6t8DTp+kXsCFe9nWOmDdTHuRJB1cfkNaktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQwHCRJDcNBktQYmXBIsiLJHUm2JVnTdz+SNM5GIhySzAHeB5wJnAicm+TEfruSpPE1EuEAnAxsq6o7q+oR4CpgZc89SdLYmtt3A52FwD1Dr7cDpzxxUJLVwOru5Q+T3DELvY2Do4Dv9t3EKMi7++5Ae+FntHMQPqPHT2fQqIRDJqlVU6i6HLj8yW9nvCTZXFVL++5D2hs/o7NvVE4rbQeOG3q9CNjRUy+SNPZGJRxuBJYkeU6SQ4FzgPU99yRJY2skTitV1Z4kFwEbgDnAuqq6ree2xomn6jTq/IzOslQ1p/YlSWNuVE4rSZJGiOEgSWoYDmMqybwkJ/Tdh6TRZDiMoSQvB24GvtS9PimJs8M0UpIcn+S3u+V5SZ7Zd0/jxHAYT29ncMuS7wNU1c3A4h77kR4nyeuBTwMf6EqLgH/pr6PxYziMpz1V9WDfTUj7cCFwKvADgKraChzda0djxnAYT7cmeS0wJ8mSJO8Fru+7KWnIT7qbcAKQZC6T3FJHTx7DYTz9IfAC4CfAJxj8dfbmXjuSHu+rSd4GzEvyO8CngM/13NNY8UtwkkZOkkOAC4DlDG7MuQH4x/IX1qwxHMZIks+xj0PzqnrFLLYjaYSNxL2VNGv+pu8GpH1JsoV9/wHzG7PYzljzyEHSyEiyz39EU1Xfnq1exp3hMIaSLAHexeD/dR82Ua+qX+mtKUkjxdlK4+nDwGXAHuClwJXAP/XakTQkybIkNyb5YZJHkjya5Ad99zVODIfxNK+qNjE4cvx2Vb0dOK3nnqRhfw+cC2wF5gF/ALy3147GjBekx9OPu6mCW7t/snQvfvtUI6aqtiWZU1WPAh9O4hc1Z5HhMJ7eDPwC8EfAxQyOGlb12pH0eA93/zL45iR/BdwHPKPnnsaKF6QljZxu1tJO4GnAnwC/BLy/qrb12tgYMRzGUJKlwF8AxzN09OgcckkTDIcxlOQO4M+ALcBjE3XnkKtvSW7Z1/v+ATN7vOYwnnZVlf/cR6PoMQbfkP44gxvt/ajfdsaXRw5jKMnpDKYJbmJwZ1YAquozvTUldZI8j8Hn8+XANxkExZerak+vjY0Zw2EMJfko8DzgNn5+Wqmq6nX9dSW1krwGeB/w7qr66777GSeGwxhKsqWqfr3vPqTJJFkInAO8EngAuBr4bFX9sNfGxozXHMbTDUlOrKpv9t2INCzJV4FnMgiE3wd2d28dmuTIqtq9t3V1cHnkMIaS3A48F/gWg2sOYXBayZkg6lWSu/j5LbuHfzlNfEa9OeQsMRzG0N5ui+xUVkkTvPHeGOpC4DjgtG75YfwsSBrikcMYSrIWWAqcUFW/luRZwKeq6tSeW5M0IvxrcTy9EngF8L8AVbWDwUVASQIMh3H1SA0OGQsgiXe7lPQ4hsN4ujrJB4Ajkrwe+ArwwZ57kjRCvOYwppL8DrCcwRTBDVW1seeWJI0Qw2GMJFlWVTf03Yek0edppfHy/omFJF/vsxFJo81wGC8ZWj6sty4kjTzvrTReDkkyn8EfBRPLPwsM71sjaYLXHMZId9+ax3j8EcQE71sj6WcMB0lSw2sOkqSG4SBJahgOkqSG4SBJahgOkqSG4SBJahgOkqSG4SBJahgOkqTG/wGUv2dFTWkJEgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAElCAYAAAAPyi6bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAE9NJREFUeJzt3X+w3XV95/Hny0Qqo8XEEliapIZq2oKdabQR2NFpqToQ6LiJndKVnS2p4sRtYcd2+4fo7gyO1g78obLMCruxpIYdARl/lEwbmmYyzmjHqgTLgEBpMggSghANKA6rFue9f5zPleP9nOTe3Bvuucx5PmbOnO95fz/f73kfJsmL7+9UFZIkDXvRuBuQJC0+hoMkqWM4SJI6hoMkqWM4SJI6hoMkqWM4aKIluTfJuePuQ1ps4nUOkqTp3HKQJHUMB020JA8leUuSDyS5NcmNSZ5uu5vWD41bneRzSQ4l+W6S/9XqL0ryP5I8nOSJtvzL27w1SSrJO5I8kuTJJP8lyeuT3J3kqan1DH3PO5Pc38buSvLKhf0vIg0YDtJz/gNwC7AM2AFMBcAS4G+Bh4E1wMo2DuCP2ut3gF8GXja13JCzgbXAfwSuAf478BbgNcAfJPnt9j2bgPcDvwesAL4E3Hycf6M0Kx5z0ERL8hDwLuCNwBur6i2tfiZwZ1WdmOTfMwiL06rq2WnL7wE+W1XXtc+/CnwDOBFYBXwTWFVVj7b53wX+pKo+3T5/FvhSVV2T5HbgM1V1Q5v3IuAHwBlV9fDz+d9Bms4tB+k53x6afgZ4SZKlwGrg4enB0Pwigy2KKQ8DS4FTh2qPD03/vxGfX9amXwn8z7a76SngMBAGWyrSgjIcpJk9AvxSC4rpDjL4R33KLwHP8rMBcCzf8+6qWjb0OrGqvjyHdUnzYjhIM/sa8BhwVZKXJnlJkje0eTcDf5bk9CQvA/4S+PQRtjJm8r+B9yV5DUCSlye56Hj8AOlYGQ7SDKrqJ8BbgVcD3wIOMDi4DLAN+L/AFxkcX/gh8F/n+D2fB64GbknyfQbHLi6YV/PSHHlAWpLUcctBktQxHCRJHcNBktQxHCRJnVHnbb8gnHzyybVmzZpxtyFJLyh33nnnd6pqxUzjXrDhsGbNGvbu3TvuNiTpBSXJrG7F4m4lSVLHcJAkdQwHSVJnxnBoDzn5QnsAyb1J3tPqH0jyaJK72uvCoWXel2R/kgeSnD9U39Bq+5NcMVQ/PclXk+xL8ukkJxzvHypJmr3ZbDk8C/x5VZ0BnANc1u51D/CxqlrXXjvhp/fBfzuDB5lsAK5LsqQ9MOXjDO4VcyZw8dB6rm7rWgs8CVx6nH6fJGkOZgyHqnqsqr7epp8G7ufo95ffCNxSVT+qqm8C+4Gz2mt/VT1YVT9m8CStjUkCvAn4TFt+O7Bprj9IkjR/x3TMIcka4LXAV1vp8vYs3G1JlrfaSgb3pZ9yoNWOVP8F4KmhWxxP1Ud9/5Yke5PsPXTo0LG0Lkk6BrMOh3av+s8Cf1pV3weuB14FrGNwr/uPTA0dsXjNod4Xq7ZW1fqqWr9ixYzXcEiS5mhWF8EleTGDYPhUVX0OoKoeH5r/CQYPYIfB//mvHlp8FYOnZXGE+neAZUmWtq2H4fGSpDGYMRzaMYEbgPur6qND9dOq6rH28W0MHkwCgwex35Tkowyer7uWwZO0AqxNcjrwKIOD1v+pqirJF4DfZ3AcYjNw2/H4cUey5oq/ez5Xrxewh6763XG3IC0Ks9lyeAPwh8A9Se5qtfczONtoHYNdQA8B7waoqnuT3Arcx+BMp8vak7RIcjmwC1gCbKuqe9v63svg6Vd/AfwzgzCSJI3JjOFQVf/I6OMCO4+yzIeBD4+o7xy1XFU9yOBsJknSIuAV0pKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkjuEgSeoYDpKkzozhkGR1ki8kuT/JvUne0+qvSLI7yb72vrzVk+TaJPuT3J3kdUPr2tzG70uyeaj+m0nuactcmyTPx4+VJM3ObLYcngX+vKrOAM4BLktyJnAFsKeq1gJ72meAC4C17bUFuB4GYQJcCZwNnAVcORUobcyWoeU2zP+nSZLmasZwqKrHqurrbfpp4H5gJbAR2N6GbQc2temNwI018BVgWZLTgPOB3VV1uKqeBHYDG9q8k6rqn6qqgBuH1iVJGoNjOuaQZA3wWuCrwKlV9RgMAgQ4pQ1bCTwytNiBVjta/cCI+qjv35Jkb5K9hw4dOpbWJUnHYNbhkORlwGeBP62q7x9t6IhazaHeF6u2VtX6qlq/YsWKmVqWJM3RrMIhyYsZBMOnqupzrfx42yVEe3+i1Q8Aq4cWXwUcnKG+akRdkjQmszlbKcANwP1V9dGhWTuAqTOONgO3DdUvaWctnQN8r+122gWcl2R5OxB9HrCrzXs6yTntuy4ZWpckaQyWzmLMG4A/BO5JclervR+4Crg1yaXAt4CL2rydwIXAfuAZ4B0AVXU4yYeAO9q4D1bV4Tb9x8AngROB29tLkjQmM4ZDVf0jo48LALx5xPgCLjvCurYB20bU9wK/PlMvkqSF4RXSkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqSO4SBJ6hgOkqTOjOGQZFuSJ5J8Y6j2gSSPJrmrvS4cmve+JPuTPJDk/KH6hlbbn+SKofrpSb6aZF+STyc54Xj+QEnSsZvNlsMngQ0j6h+rqnXttRMgyZnA24HXtGWuS7IkyRLg48AFwJnAxW0swNVtXWuBJ4FL5/ODJEnzN2M4VNUXgcOzXN9G4Jaq+lFVfRPYD5zVXvur6sGq+jFwC7AxSYA3AZ9py28HNh3jb5AkHWfzOeZweZK7226n5a22EnhkaMyBVjtS/ReAp6rq2Wn1kZJsSbI3yd5Dhw7No3VJ0tHMNRyuB14FrAMeAz7S6hkxtuZQH6mqtlbV+qpav2LFimPrWJI0a0vnslBVPT41neQTwN+2jweA1UNDVwEH2/So+neAZUmWtq2H4fGSpDGZ05ZDktOGPr4NmDqTaQfw9iQ/l+R0YC3wNeAOYG07M+kEBgetd1RVAV8Afr8tvxm4bS49SZKOnxm3HJLcDJwLnJzkAHAlcG6SdQx2AT0EvBugqu5NcitwH/AscFlV/aSt53JgF7AE2FZV97aveC9wS5K/AP4ZuOG4/TpJ0pzMGA5VdfGI8hH/Aa+qDwMfHlHfCewcUX+QwdlMkqRFwiukJUkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEkdw0GS1DEcJEmdGcMhybYkTyT5xlDtFUl2J9nX3pe3epJcm2R/kruTvG5omc1t/L4km4fqv5nknrbMtUlyvH+kJOnYzGbL4ZPAhmm1K4A9VbUW2NM+A1wArG2vLcD1MAgT4ErgbOAs4MqpQGljtgwtN/27JEkLbMZwqKovAoenlTcC29v0dmDTUP3GGvgKsCzJacD5wO6qOlxVTwK7gQ1t3klV9U9VVcCNQ+uSJI3JXI85nFpVjwG091NafSXwyNC4A612tPqBEfWRkmxJsjfJ3kOHDs2xdUnSTI73AelRxwtqDvWRqmprVa2vqvUrVqyYY4uSpJnMNRweb7uEaO9PtPoBYPXQuFXAwRnqq0bUJUljNNdw2AFMnXG0GbhtqH5JO2vpHOB7bbfTLuC8JMvbgejzgF1t3tNJzmlnKV0ytC5J0pgsnWlAkpuBc4GTkxxgcNbRVcCtSS4FvgVc1IbvBC4E9gPPAO8AqKrDST4E3NHGfbCqpg5y/zGDM6JOBG5vL0nSGM0YDlV18RFmvXnE2AIuO8J6tgHbRtT3Ar8+Ux+SpIXjFdKSpI7hIEnqGA6SpI7hIEnqGA6SpI7hIEnqGA6SpI7hIEnqGA6SpI7hIEnqGA6SpI7hIEnqzHjjPUkLb80VfzfuFrRIPXTV7y7I97jlIEnqGA6SpI7hIEnqGA6SpI7hIEnqGA6SpI7hIEnqGA6SpI7hIEnqGA6SpI7hIEnqGA6SpI7hIEnqGA6SpI7hIEnqGA6SpI7hIEnqGA6SpI7hIEnqGA6SpM68wiHJQ0nuSXJXkr2t9ooku5Psa+/LWz1Jrk2yP8ndSV43tJ7Nbfy+JJvn95MkSfN1PLYcfqeq1lXV+vb5CmBPVa0F9rTPABcAa9trC3A9DMIEuBI4GzgLuHIqUCRJ4/F87FbaCGxv09uBTUP1G2vgK8CyJKcB5wO7q+pwVT0J7AY2PA99SZJmab7hUMA/JLkzyZZWO7WqHgNo76e0+krgkaFlD7TakeqdJFuS7E2y99ChQ/NsXZJ0JEvnufwbqupgklOA3Un+5ShjM6JWR6n3xaqtwFaA9evXjxwjSZq/eW05VNXB9v4E8HkGxwweb7uLaO9PtOEHgNVDi68CDh6lLkkakzmHQ5KXJvn5qWngPOAbwA5g6oyjzcBtbXoHcEk7a+kc4Httt9Mu4Lwky9uB6PNaTZI0JvPZrXQq8PkkU+u5qar+PskdwK1JLgW+BVzUxu8ELgT2A88A7wCoqsNJPgTc0cZ9sKoOz6MvSdI8zTkcqupB4DdG1L8LvHlEvYDLjrCubcC2ufYiSTq+vEJaktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktQxHCRJHcNBktRZNOGQZEOSB5LsT3LFuPuRpEm2KMIhyRLg48AFwJnAxUnOHG9XkjS5FkU4AGcB+6vqwar6MXALsHHMPUnSxFo67gaalcAjQ58PAGdPH5RkC7ClffxBkgcWoLdJcDLwnXE3sRjk6nF3oCPwz2hzHP6MvnI2gxZLOGRErbpC1VZg6/PfzmRJsreq1o+7D+lI/DO68BbLbqUDwOqhz6uAg2PqRZIm3mIJhzuAtUlOT3IC8HZgx5h7kqSJtSh2K1XVs0kuB3YBS4BtVXXvmNuaJO6q02Lnn9EFlqpu174kacItlt1KkqRFxHCQJHUMB0mLRpJVR5n31oXsZdIZDhMkyeYj1F+c5OaF7kcaYU+SNdOLSd4JXLPg3Uwww2GyvKddZf5TSV4K7ASeGU9L0s/4M2B3krVThSTva/XfHltXE2hRnMqqBfMW4O+TvKSqrk2ygkEw7Kkq74SrsauqnUl+BNyeZBPwLuD1wG9V1ZPj7W6yeCrrhElyEnA78CUGNze8vqquHW9X0s9K8kbgb4AvA39QVT8cc0sTx3CYIEl+r03+PPBRYA+DO+ACUFWfG0df0pQkTzO4r1qAnwP+DfhJ+1xVddIY25sohsMESfLXR5ldVfXOBWtG0qJmOEhaVJKEwTNeVjLYijgIfK38x2pBGQ4TJsn5wCZ+9i/e31TVrrE2JgFJzgOuA/YBj7byKuDVwJ9U1T+Mq7dJYzhMkCTXAL8C3MjgNukw+It3CbCvqt4zrt4kgCT3AxdU1UPT6qcDO6vqjLE0NoEMhwmS5F+r6ldG1AP8a1WtHbGYtGCS7APOqKpnp9VPAO6rqlePp7PJ43UOk+WHSc6qqq9Nq78e8FRBLQbbgDuS3MJzjw5ezeAZLzeMrasJ5JbDBEnyOuB6BqeyTu1WWg18n8H+3DvH1Zs0JckZDK7BWcngFNYDwI6qum+sjU0Yw2ECJfl3DP3Fq6pvj7klSYuM91aaQFX17baV8C/ALyZZNu6eJIAkG4amX57kr5LcneSmJKeOs7dJYzhMkCTXDU2/EbgP+AhwT5ILx9aY9Jy/HJr+CPBt4K0MnjP/f8bS0YTygPRkOWdo+kPApqr6epJfBm5lcBM+abFYX1Xr2vTHjnTLeT0/DIfJdVJVfR2gqh5MsmTcDUnAKUn+G4PjYSclydCV0e7pWECGw2T5tSR3M/iLtybJ8qp6MsmLgBePuTcJ4BMMzqYD2A6cDBxqJ1HcNbauJpBnK02QJK+cVjpYVf+W5GQG98v3rqySAMNB0iKW5G3A7qr6wbh7mTTuw5tQ7T5LP32XFpskr2JwosR/Hncvk8hwmFy/1d59Lq8Wq3cCV7d3LTDDQdKi086eu4hBOHwvyW+MuaWJYzhIWowuBL5cVU8zuBnfu8bcz8QxHCQtRpfy3F1YPw9c2G7brQViOEhaVNq9vpZV1ZcAquqHwGeAN421sQnjRXCT66b2/qmxdiFNU1VPAedOq713PN1MLq9zkCR13K00YZKclOSCabV1SX51XD1JWnwMh8nzNHBNklcM1a470mBJk8lwmDDtDpc30a46TfJrrfzAWBuTtKgYDpNpGzB1b/w/wge3S5rGs5UmUFU9kuRQktcDbwNeO+6eJC0ubjlMrr8C/hr4YlU9M+5mJC0uhsPkug34CbB13I1IWny8zkGS1HHLQZLUMRwkSR3DQZLUMRwkSR3DQZLUMRwkSR3DQZLUMRwkSZ3/D2dwVJ1wZRPAAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEICAYAAAC0+DhzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGg5JREFUeJzt3X+0XWV95/H3p4moaDUgF4tJauKY2gLTVppCWqctFQvBX2F1ZBXGDqlDJ7M6WO0PR6Fdq3RUZmDGVSrLSheFaHAckIW2ZCotTRFLXZVIEEUBndyCJVeQXA2g1QpGv/PHee54yD43Nznnhpvc+36tddfd+7ufvfezs2/O5+wf5+xUFZIk9fuBue6AJOngYzhIkjoMB0lSh+EgSeowHCRJHYaDJKnDcJAWqCQfT/Lrc90PHZwMBx2UknwpySv2qP1akk/MVZ/6+vHxJN9O8s9JvprkI0mOmYXlLm7LPLGv9vokNaD2hVHXJ+2N4aAFL8niIWZ7Y1U9G/gRYAlw6ajrrardwCeBX+gr/zzwhQG1W/d3fdL+MBx0yEryY+1d/KNJ7k7y2r5pTzplsudRR3s3fl6S7cD29FyaZGeSx5LcleT4mfpQVbuADwPHt+U+Pcm7kjyQ5OEkf5rkmW3ayUkmkrwtyVeA9w1Y5K30Xvyn/BxwyYDarW2ZP5Dk/CT/mORrSa5LcmTfdq5J8g/t3+izSU6e5t/ymLbNb5lpm7UwGA46JCV5GvB/gL8BjgZ+E/hgkpfsx2LOAE4CjgVOpfcCPHUk8CvA1/ahH0cB/xa4s5Uuacv4SeDFwFLgD/pm+SHgSOCFwIYBi7wVeFl70T8KeBZwHXBiX+1H+f6Rw5vadvwC8ALgEeBPWt+WAh8F3tnW+Rbgw0nG9tiGFcDfAe+pqnfNtM1aGAwHHcz+or3jfTTJo8B7+6atAZ4NXFxVT1TVx4C/BM7ej+X/96raVVX/AnwH+EF6L7ypqnur6qG9zHtZ69NngYeA30kS4D8Cv92W+w3gvwFn9c33PeDCqnq8rXdPW4HDgX9N7wjhE1X1LeD+vto/VdUDrf1/An6/qiaq6nHgD4HXtVNWvwrcWFU3VtX3qmoLsA14Zd/6jgU+3vp0xcz/ZFoohjnXKj1Vzqiqv50aSfJrwNSpohcAO6rqe33t/4neO/V9tWNqoKo+luQ99N51/3CSPwfeUlVfn2beN1XVlf2FJEfTe2G/o5cTvTKwqK/ZZFV9e7oOVdW3k3yK3lHMi4C/b5M+0Vfrv97wQuDPk/T/O3wXeH6bdmaS1/RNexpwS9/464Fx4Prp+qSFySMHHaoeBJYn6f8b/mHgy234m/ReqKf80IBlPOkriavqsqr6KeA4eqeG/st+9umrwL8Ax1XVkvbz3HbheuA6pzF13eHn+H44/H1frT8cdgCn961vSVU9o6q+3KZ9YI9pz6qqi/vm/8PW7/+dpD/EtMAZDjpUbaUXAG9N8rR2ofU1wLVt+meAX05yeJIXA+fubWFJfjrJSe1axjeBb9N7B77P2lHMnwGXtqMIkixNctr+LIfei/8vAsuBe1rtE8DJ9K5l9IfDnwIXJXlhW99YknVt2v8CXpPktCSLkjyjXRRf1jf/d4Az6V3b+MAeYasFzD8EHZKq6gngtcDp9N75vhc4p6qm7v+/FHgCeBjYBHxwhkU+h94L+yP0Tk99DRjm4uzb6J2muS3J14G/BfbnIjnAPwDPBbZWe+BKVX0NmAR2VtX2vrbvBjYDf5PkG8Bt9C6yU1U7gHXA77V5d9A7GnrS//v2b/nL9C7sbzQgBL0Lb3PdB0nSQcZ3CJKkDsNBktRhOEiSOgwHSVLHIfshuKOOOqpWrFgx192QpEPKHXfc8dWqGpup3SEbDitWrGDbtm1z3Q1JOqQk+ad9aedpJUlSh+EgSeowHCRJHYaDJKnDcJAkdRgOkqQOw0GS1DFjOCTZ2B66/vk96r+Z5Ivtwe7/o69+QZLxNu20vvraVhtPcn5ffWWSrUm2J/lQksNma+MkScPZlyOH9wNr+wtJfpHe98T/eFUdR/ve+yTH0nte7nFtnve2h4wsovf4xdPpPbP27NYWeg9kv7SqVtH7Lv29PpRFknTgzfgJ6aq6NcmKPcq/Qe/B7o+3NjtbfR1wbavfn2QcOLFNG6+q+wCSXAusS3Iv8HLg37U2m+g9tvDyYTdI88eK8z860vxfuvhVs9QTaeEZ9prDjwA/104H/V2Sn271pfQ9tB2YaLXp6s8DHq2q3XvUB0qyIcm2JNsmJyeH7LokaSbDhsNi4AhgDb3HDl6XJEAGtK0h6gNV1RVVtbqqVo+Nzfi9UZKkIQ37xXsTwEfa820/leR7wFGtvryv3TLgwTY8qP5VYEmSxe3oob+9JGmODHvk8Bf0rhWQ5EeAw+i90G8Gzkry9CQrgVXAp4DbgVXtzqTD6F203tzC5RbgdW2564Ebht0YSdLsmPHIIck1wMnAUUkmgAuBjcDGdnvrE8D69kJ/d5LrgHuA3cB5VfXdtpw3AjcBi4CNVXV3W8XbgGuTvBO4E7hqFrdPkjSEfblb6expJv3qNO0vAi4aUL8RuHFA/T6+f0eTJOkg4CekJUkdhoMkqcNwkCR1GA6SpA7DQZLUYThIkjoMB0lSh+EgSeowHCRJHYaDJKnDcJAkdRgOkqQOw0GS1GE4SJI6DAdJUofhIEnqMBwkSR0zhkOSjUl2tkeC7jntLUkqyVFtPEkuSzKe5K4kJ/S1XZ9ke/tZ31f/qSSfa/NcliSztXGSpOHsy5HD+4G1exaTLAd+CXigr3w6sKr9bAAub22PpPfs6ZPoPRL0wiRHtHkub22n5uusS5L01JoxHKrqVmDXgEmXAm8Fqq+2Dri6em4DliQ5BjgN2FJVu6rqEWALsLZNe05VfbKqCrgaOGO0TZIkjWqoaw5JXgt8uao+u8ekpcCOvvGJVttbfWJAfbr1bkiyLcm2ycnJYbouSdoH+x0OSQ4Hfh/4g0GTB9RqiPpAVXVFVa2uqtVjY2P70l1J0hCGOXL4V8BK4LNJvgQsAz6d5IfovfNf3td2GfDgDPVlA+qSpDm03+FQVZ+rqqOrakVVraD3An9CVX0F2Ayc0+5aWgM8VlUPATcBpyY5ol2IPhW4qU37RpI17S6lc4AbZmnbJElD2pdbWa8BPgm8JMlEknP30vxG4D5gHPgz4D8DVNUu4B3A7e3n7a0G8BvAlW2efwT+arhNkSTNlsUzNaiqs2eYvqJvuIDzpmm3Edg4oL4NOH6mfkiSnjp+QlqS1GE4SJI6DAdJUofhIEnqMBwkSR2GgySpw3CQJHUYDpKkDsNBktRhOEiSOgwHSVKH4SBJ6jAcJEkdhoMkqcNwkCR1GA6SpI59eRLcxiQ7k3y+r/Y/k3whyV1J/jzJkr5pFyQZT/LFJKf11de22niS8/vqK5NsTbI9yYeSHDabGyhJ2n/7cuTwfmDtHrUtwPFV9ePA/wUuAEhyLHAWcFyb571JFiVZBPwJcDpwLHB2awtwCXBpVa0CHgH29hhSSdJTYMZwqKpbgV171P6mqna30duAZW14HXBtVT1eVffTey70ie1nvKruq6ongGuBdUkCvBy4vs2/CThjxG2SJI1oNq45/Afgr9rwUmBH37SJVpuu/jzg0b6gmaoPlGRDkm1Jtk1OTs5C1yVJg4wUDkl+H9gNfHCqNKBZDVEfqKquqKrVVbV6bGxsf7srSdpHi4edMcl64NXAKVU19YI+ASzva7YMeLAND6p/FViSZHE7euhvL0maI0MdOSRZC7wNeG1Vfatv0mbgrCRPT7ISWAV8CrgdWNXuTDqM3kXrzS1UbgFe1+ZfD9ww3KZIkmbLvtzKeg3wSeAlSSaSnAu8B/hBYEuSzyT5U4Cquhu4DrgH+GvgvKr6bjsqeCNwE3AvcF1rC72Q+Z0k4/SuQVw1q1soSdpvM55WqqqzB5SnfQGvqouAiwbUbwRuHFC/j97dTJKkg4SfkJYkdRgOkqQOw0GS1GE4SJI6DAdJUofhIEnqMBwkSR2GgySpw3CQJHUYDpKkDsNBktRhOEiSOgwHSVKH4SBJ6jAcJEkdhoMkqcNwkCR17MtjQjcm2Znk8321I5NsSbK9/T6i1ZPksiTjSe5KckLfPOtb++1J1vfVfyrJ59o8lyXJbG+kJGn/7MuRw/uBtXvUzgdurqpVwM1tHOB0YFX72QBcDr0wAS4ETqL3SNALpwKltdnQN9+e65IkPcVmDIequhXYtUd5HbCpDW8CzuirX109twFLkhwDnAZsqapdVfUIsAVY26Y9p6o+WVUFXN23LEnSHBn2msPzq+ohgPb76FZfCuzoazfRanurTwyoD5RkQ5JtSbZNTk4O2XVJ0kxm+4L0oOsFNUR9oKq6oqpWV9XqsbGxIbsoSZrJsOHwcDslRPu9s9UngOV97ZYBD85QXzagLkmaQ8OGw2Zg6o6j9cANffVz2l1La4DH2mmnm4BTkxzRLkSfCtzUpn0jyZp2l9I5fcuSJM2RxTM1SHINcDJwVJIJencdXQxcl+Rc4AHgzNb8RuCVwDjwLeANAFW1K8k7gNtbu7dX1dRF7t+gd0fUM4G/aj+SpDk0YzhU1dnTTDplQNsCzptmORuBjQPq24DjZ+qHJOmp4yekJUkdhoMkqcNwkCR1GA6SpA7DQZLUYThIkjoMB0lSh+EgSeowHCRJHYaDJKnDcJAkdRgOkqQOw0GS1GE4SJI6DAdJUofhIEnqMBwkSR0jhUOS305yd5LPJ7kmyTOSrEyyNcn2JB9Kclhr+/Q2Pt6mr+hbzgWt/sUkp422SZKkUQ0dDkmWAm8CVlfV8cAi4CzgEuDSqloFPAKc22Y5F3ikql4MXNrakeTYNt9xwFrgvUkWDdsvSdLoRj2ttBh4ZpLFwOHAQ8DLgevb9E3AGW14XRunTT8lSVr92qp6vKruB8aBE0fslyRpBEOHQ1V9GXgX8AC9UHgMuAN4tKp2t2YTwNI2vBTY0ebd3do/r78+YJ4nSbIhybYk2yYnJ4ftuiRpBqOcVjqC3rv+lcALgGcBpw9oWlOzTDNtunq3WHVFVa2uqtVjY2P732lJ0j4Z5bTSK4D7q2qyqr4DfAT4WWBJO80EsAx4sA1PAMsB2vTnArv66wPmkSTNgVHC4QFgTZLD27WDU4B7gFuA17U264Eb2vDmNk6b/rGqqlY/q93NtBJYBXxqhH5Jkka0eOYmg1XV1iTXA58GdgN3AlcAHwWuTfLOVruqzXIV8IEk4/SOGM5qy7k7yXX0gmU3cF5VfXfYfkmSRjd0OABU1YXAhXuU72PA3UZV9W3gzGmWcxFw0Sh9kSTNHj8hLUnqMBwkSR2GgySpw3CQJHUYDpKkDsNBktRhOEiSOgwHSVKH4SBJ6jAcJEkdhoMkqcNwkCR1GA6SpA7DQZLUYThIkjoMB0lSx0jhkGRJkuuTfCHJvUl+JsmRSbYk2d5+H9HaJsllScaT3JXkhL7lrG/ttydZP/0aJUlPhVGPHN4N/HVV/SjwE8C9wPnAzVW1Cri5jQOcTu/50KuADcDlAEmOpPc0uZPoPUHuwqlAkSTNjaHDIclzgJ+nPSO6qp6oqkeBdcCm1mwTcEYbXgdcXT23AUuSHAOcBmypql1V9QiwBVg7bL8kSaMb5cjhRcAk8L4kdya5MsmzgOdX1UMA7ffRrf1SYEff/BOtNl1dkjRHRgmHxcAJwOVV9VLgm3z/FNIgGVCrvdS7C0g2JNmWZNvk5OT+9leStI9GCYcJYKKqtrbx6+mFxcPtdBHt986+9sv75l8GPLiXekdVXVFVq6tq9djY2AhdlyTtzdDhUFVfAXYkeUkrnQLcA2wGpu44Wg/c0IY3A+e0u5bWAI+10043AacmOaJdiD611SRJc2TxiPP/JvDBJIcB9wFvoBc41yU5F3gAOLO1vRF4JTAOfKu1pap2JXkHcHtr9/aq2jVivyRJIxgpHKrqM8DqAZNOGdC2gPOmWc5GYOMofZEkzR4/IS1J6jAcJEkdhoMkqcNwkCR1GA6SpA7DQZLUYThIkjoMB0lSh+EgSeowHCRJHYaDJKnDcJAkdRgOkqQOw0GS1GE4SJI6DAdJUofhIEnqGDkckixKcmeSv2zjK5NsTbI9yYfaI0RJ8vQ2Pt6mr+hbxgWt/sUkp43aJ0nSaGbjyOHNwL1945cAl1bVKuAR4NxWPxd4pKpeDFza2pHkWOAs4DhgLfDeJItmoV+SpCGNFA5JlgGvAq5s4wFeDlzfmmwCzmjD69o4bfoprf064Nqqeryq7gfGgRNH6ZckaTSjHjn8MfBW4Htt/HnAo1W1u41PAEvb8FJgB0Cb/lhr///rA+Z5kiQbkmxLsm1ycnLErkuSpjN0OCR5NbCzqu7oLw9oWjNM29s8Ty5WXVFVq6tq9djY2H71V5K07xaPMO/LgNcmeSXwDOA59I4kliRZ3I4OlgEPtvYTwHJgIsli4LnArr76lP55JElzYOgjh6q6oKqWVdUKeheUP1ZVrwduAV7Xmq0HbmjDm9s4bfrHqqpa/ax2N9NKYBXwqWH7JUka3ShHDtN5G3BtkncCdwJXtfpVwAeSjNM7YjgLoKruTnIdcA+wGzivqr57APolSdpHsxIOVfVx4ONt+D4G3G1UVd8Gzpxm/ouAi2ajL5Kk0fkJaUlSh+EgSeowHCRJHYaDJKnDcJAkdRgOkqQOw0GS1HEgPgQnLWgrzv/o0PN+6eJXzWJPpOF55CBJ6jAcJEkdhoMkqcNwkCR1GA6SpA7DQZLU4a2smre8pVQankcOkqQOw0GS1DF0OCRZnuSWJPcmuTvJm1v9yCRbkmxvv49o9SS5LMl4kruSnNC3rPWt/fYk66dbpyTpqTHKkcNu4Her6seANcB5SY4FzgdurqpVwM1tHOB0YFX72QBcDr0wAS4ETqL3eNELpwJFkjQ3hg6Hqnqoqj7dhr8B3AssBdYBm1qzTcAZbXgdcHX13AYsSXIMcBqwpap2VdUjwBZg7bD9kiSNblauOSRZAbwU2Ao8v6oegl6AAEe3ZkuBHX2zTbTadPVB69mQZFuSbZOTk7PRdUnSACOHQ5JnAx8Gfquqvr63pgNqtZd6t1h1RVWtrqrVY2Nj+99ZSdI+GSkckjyNXjB8sKo+0soPt9NFtN87W30CWN43+zLgwb3UJUlzZJS7lQJcBdxbVX/UN2kzMHXH0Xrghr76Oe2upTXAY+20003AqUmOaBeiT201SdIcGeUT0i8D/j3wuSSfabXfAy4GrktyLvAAcGabdiPwSmAc+BbwBoCq2pXkHcDtrd3bq2rXCP2SJI1o6HCoqk8w+HoBwCkD2hdw3jTL2ghsHLYvOniN8hUWkuaO3610CPG7gjQT/0Y0W/z6DElSh+EgSerwtJJm5HUDaeHxyEGS1GE4SJI6DAdJUofXHBYIrxtI2h8eOUiSOjxykA4iHuHpYOGRgySpwyMHaQDfwWuhMxyeQr7gSDpUGA77yRd4SQuB1xwkSR0eOUgamV8VPv8YDpIAT5nqyQ6acEiyFng3sAi4sqouPlDr8j+BND+M+n/Zo5bpHRThkGQR8CfALwETwO1JNlfVPXPbM0mafYfCabiDIhyAE4HxqroPIMm1wDrAcJDmuUP1SP5Q7fe+OljCYSmwo298Ajhpz0ZJNgAb2ug/J/nifqzjKOCrQ/fw0OV2Lyxu937IJQegJwfYHn0eZrtfuC+NDpZwyIBadQpVVwBXDLWCZFtVrR5m3kOZ272wuN0Ly4Hc7oPlcw4TwPK+8WXAg3PUF0la8A6WcLgdWJVkZZLDgLOAzXPcJ0lasA6K00pVtTvJG4Gb6N3KurGq7p7l1Qx1OmoecLsXFrd7YTlg252qzql9SdICd7CcVpIkHUQMB0lSx7wPhyRrk3wxyXiS8+e6PwdKkuVJbklyb5K7k7y51Y9MsiXJ9vb7iLnu64GQZFGSO5P8ZRtfmWRr2+4PtRsd5p0kS5Jcn+QLbd//zELY50l+u/2dfz7JNUmeMR/3eZKNSXYm+XxfbeD+Tc9l7bXuriQnjLLueR0OfV/LcTpwLHB2kmPntlcHzG7gd6vqx4A1wHltW88Hbq6qVcDNbXw+ejNwb9/4JcClbbsfAc6dk14deO8G/rqqfhT4CXr/BvN6nydZCrwJWF1Vx9O7ieUs5uc+fz+wdo/adPv3dGBV+9kAXD7Kiud1OND3tRxV9QQw9bUc805VPVRVn27D36D3IrGU3vZuas02AWfMTQ8PnCTLgFcBV7bxAC8Hrm9N5ut2Pwf4eeAqgKp6oqoeZQHsc3p3Wj4zyWLgcOAh5uE+r6pbgV17lKfbv+uAq6vnNmBJkmOGXfd8D4dBX8uxdI768pRJsgJ4KbAVeH5VPQS9AAGOnrueHTB/DLwV+F4bfx7waFXtbuPzdb+/CJgE3tdOqV2Z5FnM831eVV8G3gU8QC8UHgPuYGHsc5h+/87q6918D4d9+lqO+STJs4EPA79VVV+f6/4caEleDeysqjv6ywOazsf9vhg4Abi8ql4KfJN5dgppkHaOfR2wEngB8Cx6p1T2NB/3+d7M6t/9fA+HBfW1HEmeRi8YPlhVH2nlh6cOLdvvnXPVvwPkZcBrk3yJ3mnDl9M7kljSTjnA/N3vE8BEVW1t49fTC4v5vs9fAdxfVZNV9R3gI8DPsjD2OUy/f2f19W6+h8OC+VqOdp79KuDeqvqjvkmbgfVteD1ww1PdtwOpqi6oqmVVtYLe/v1YVb0euAV4XWs277YboKq+AuxI8pJWOoXe19zP631O73TSmiSHt7/7qe2e9/u8mW7/bgbOaXctrQEemzr9NIx5/wnpJK+k905y6ms5LprjLh0QSf4N8PfA5/j+ufffo3fd4Trgh+n9pzqzqva8wDUvJDkZeEtVvTrJi+gdSRwJ3An8alU9Ppf9OxCS/CS9C/GHAfcBb6D3pm9e7/Mk/xX4FXp36d0J/Dq98+vzap8nuQY4md5Xcz8MXAj8BQP2bwvK99C7u+lbwBuqatvQ657v4SBJ2n/z/bSSJGkIhoMkqcNwkCR1GA6SpA7DQZLUYThIkjoMB0lSx/8DoD8u4IZhO9IAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(10,10))\n",
    "for i, col in enumerate(data.drop(\"hours-per-week\", axis = 1)):\n",
    "    d = data[col].value_counts().sort_index()\n",
    "    plt.bar(d.index, d)\n",
    "    plt.xticks(rotation = 90)\n",
    "    plt.title(col)\n",
    "    plt.show()\n",
    "\n",
    "plt.hist(data['hours-per-week'], bins = 20);\n",
    "plt.title(\"Hours Per Week\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "#### Preprocessing\n",
    "The following cells  demonstrates the functions used in preprocessing, concluding with the division of data into a training and testing sets, that are then preprocessed.  \n",
    "\n",
    "##### Dummy Variables\n",
    "\n",
    "In the creation of dummy variables, the most frequently occuring class will be dropped as a way of avoiding multicollinearity.  \n",
    "\n",
    "Thus, for **`n`** categories, **`n-1`** features will be created. Below demonstrates finding the most frequent category."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "All Value Counts:\n",
      " Private             22696\n",
      " Self-emp-not-inc     2541\n",
      " Local-gov            2093\n",
      " ?                    1836\n",
      " State-gov            1298\n",
      " Self-emp-inc         1116\n",
      " Federal-gov           960\n",
      " Without-pay            14\n",
      " Never-worked            7\n",
      "Name: workclass, dtype: int64\n",
      "\n",
      "Top category: 22696\n"
     ]
    }
   ],
   "source": [
    "val_count = data['workclass'].value_counts()\n",
    "print('All Value Counts:')\n",
    "print(val_count)\n",
    "\n",
    "top = val_count[0]\n",
    "print(\"\\nTop category:\", top)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### One Hot Encoder  \n",
    "\n",
    "Dummies will be created using `sklearn`'s `OneHotEncoder`.  The following cell demonstrates fitting and transforming data using the One Hot Encoder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Initial DataFrame:\n",
      "    beg end\n",
      "500   b   z\n",
      "501   a   y\n",
      "502   c   y\n",
      "503   a   y\n",
      "504   c   z\n",
      "\n",
      "Test df\n",
      "   beg end\n",
      "56   c   y\n",
      "72   b   y\n"
     ]
    }
   ],
   "source": [
    "from sklearn.preprocessing import OneHotEncoder\n",
    "\n",
    "ex_df = pd.DataFrame(\n",
    "    np.array([['b', 'a', 'c', 'a', 'c',], [\"z\",\"y\",\"y\",\"y\",\"z\"]]).T,\n",
    "    columns = [\"beg\",\"end\"],\n",
    "    index = range(500,505))\n",
    "\n",
    "test_df = pd.DataFrame(\n",
    "    np.array([[\"c\",\"b\"],[\"y\",\"y\"]]).T,\n",
    "    columns = [\"beg\",\"end\"],\n",
    "    index = [56,72])\n",
    "\n",
    "print(\"Initial DataFrame:\")\n",
    "print(ex_df)\n",
    "\n",
    "print(\"\\nTest df\")\n",
    "print(test_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Transformed values\n",
      "[[0. 1. 0. 0. 1.]\n",
      " [1. 0. 0. 1. 0.]\n",
      " [0. 0. 1. 1. 0.]\n",
      " [1. 0. 0. 1. 0.]\n",
      " [0. 0. 1. 0. 1.]]\n",
      "\n",
      "Categories\n",
      "[array(['a', 'b', 'c'], dtype=object), array(['y', 'z'], dtype=object)]\n",
      "\n",
      "Final DataFrame\n",
      "     a    b    c    y    z\n",
      "0  0.0  1.0  0.0  0.0  1.0\n",
      "1  1.0  0.0  0.0  1.0  0.0\n",
      "2  0.0  0.0  1.0  1.0  0.0\n",
      "3  1.0  0.0  0.0  1.0  0.0\n",
      "4  0.0  0.0  1.0  0.0  1.0\n",
      "\n",
      "Transformed test df\n",
      "     a    b    c    y    z\n",
      "0  0.0  0.0  1.0  1.0  0.0\n",
      "1  0.0  1.0  0.0  1.0  0.0\n"
     ]
    }
   ],
   "source": [
    "# Instantiate OneHotEncoder\n",
    "# sparse = False means data will not be stored in sparse matrix\n",
    "ohe = OneHotEncoder(sparse = False)\n",
    "\n",
    "# Fitting OHE with the \"training\" data\n",
    "ohe.fit(ex_df)\n",
    "\n",
    "# Transforming the \"training\" dat\n",
    "tr_vals = ohe.transform(ex_df)\n",
    "\n",
    "print(\"\\nTransformed values\")\n",
    "print(tr_vals)\n",
    "\n",
    "print(\"\\nCategories\")\n",
    "print(ohe.categories_)\n",
    "\n",
    "# Creating column names from `.categories_`\n",
    "ohe_cats = np.concatenate(ohe.categories_)\n",
    "\n",
    "# In creation of new df. Note the use of np.concatenate\n",
    "final_df = pd.DataFrame(tr_vals, columns = ohe_cats)\n",
    "\n",
    "print(\"\\nFinal DataFrame\")\n",
    "print(final_df)\n",
    "\n",
    "# Putting everything together to transform test data\n",
    "print(\"\\nTransformed test df\")\n",
    "print(pd.DataFrame(ohe.transform(test_df), columns= ohe_cats))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "##### LabelEncoder\n",
    "\n",
    "`sklearn`'s `LabelEncoder` will be used to transform our income variable from strings to 0s and 1s.  \n",
    "\n",
    "Demonstrated below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Target Train Series\n",
      "0      >50K\n",
      "1     <=50K\n",
      "2     <=50K\n",
      "3      >50K\n",
      "4      >50K\n",
      "5      >50K\n",
      "6      >50K\n",
      "7     <=50K\n",
      "8      >50K\n",
      "9     <=50K\n",
      "dtype: object\n",
      "\n",
      "Target Test Series\n",
      "0     <=50K\n",
      "1      >50K\n",
      "2      >50K\n",
      "3      >50K\n",
      "4     <=50K\n",
      "dtype: object\n"
     ]
    }
   ],
   "source": [
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "# Create target Series\n",
    "target_train = pd.Series(np.random.choice(data['income'].unique(), size = 10))\n",
    "target_test = pd.Series(np.random.choice(data['income'].unique(), size = 5))\n",
    "print(\"Target Train Series\")\n",
    "print(target_train)\n",
    "\n",
    "print(\"\\nTarget Test Series\")\n",
    "print(target_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Transformed training values\n",
      "[1 0 0 1 1 1 1 0 1 0]\n",
      "\n",
      "Transformed test values\n",
      "[0 1 1 1 0]\n",
      "\n",
      "LabelEncoder `.classes_`\n",
      "[' <=50K' ' >50K']\n"
     ]
    }
   ],
   "source": [
    "# Instantiate encoder\n",
    "le = LabelEncoder()\n",
    "\n",
    "# Fit with training data\n",
    "le.fit(target_train)\n",
    "\n",
    "# Transform training and test data\n",
    "trans_train = le.transform(target_train)\n",
    "trans_test = le.transform(target_test)\n",
    "\n",
    "print(\"Transformed training values\")\n",
    "print(trans_train)\n",
    "\n",
    "print(\"\\nTransformed test values\")\n",
    "print(trans_test)\n",
    "\n",
    "print(\"\\nLabelEncoder `.classes_`\")\n",
    "print(le.classes_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "##### Define a custom preprocessing function\n",
    "Using the processes demonstrated above, a function is created to preprocess the census data.  \n",
    "\n",
    "The function is then used to create a training and testing dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "def preprocess_census(X_train, X_test, y_train, y_test):\n",
    "\n",
    "    ### Hardcode variables which need categorical encoding\n",
    "    to_encode = [\"workclass\", \"occupation\", \"sex\"]\n",
    "\n",
    "    ### Find top categories in categorical columns\n",
    "    ### Used for dropping majority class to prevent multi-colinearity\n",
    "    top_categories = []\n",
    "\n",
    "    for col in to_encode:\n",
    "        top_categories.append(X_train[col].value_counts().index[0])\n",
    "\n",
    "    ### Create and fit one-hot encoder for categoricals\n",
    "    OHE = OneHotEncoder(sparse = False)\n",
    "    OHE.fit(X_train[to_encode])\n",
    "\n",
    "    ## Create and fit Label encoder for target\n",
    "    LabEnc = LabelEncoder()\n",
    "    LabEnc.fit(y_train)\n",
    "\n",
    "    def create_encoded_df(X, to_encode = to_encode, OHE = OHE, top_categories = top_categories):\n",
    "        # Return columns which need encoding.\n",
    "        def return_encoded_cols(X, to_encode = to_encode, OHE = OHE, top_categories = top_categories):\n",
    "            # Use onehotencoder to transform.\n",
    "            # Use \"categories\" to name\n",
    "            toRet = pd.DataFrame(OHE.transform(X[to_encode]), columns = np.concatenate(OHE.categories_))\n",
    "\n",
    "            # Drop top_categories and return\n",
    "            return toRet.drop(top_categories, axis = 1)\n",
    "\n",
    "        # create encoded columns\n",
    "        ret_cols = return_encoded_cols(X)\n",
    "\n",
    "        # Drop columns that were encoded\n",
    "        dr_enc = X.drop(to_encode, axis = 1)\n",
    "\n",
    "        # Concatenate values\n",
    "        # use index from original data\n",
    "        # use combined column names\n",
    "        return pd.DataFrame(np.concatenate([ret_cols.values, dr_enc.values],axis = 1),\n",
    "                            index = dr_enc.index,\n",
    "                            columns = list(ret_cols.columns) + list(dr_enc.columns))\n",
    "\n",
    "\n",
    "    def encode_target(y, LabEnc = LabEnc):\n",
    "        # Use label encoder, and supply with original index\n",
    "        return pd.Series(LabEnc.transform(y), index= y.index)\n",
    "\n",
    "    return create_encoded_df(X_train), create_encoded_df(X_test), encode_target(y_train), encode_target(y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# Create training and testing sets; preprocess them.\n",
    "target = data['income']\n",
    "predictors = data.drop(\"income\", axis = 'columns')\n",
    "\n",
    "X_train, X_test, y_train, y_test = preprocess_census(*train_test_split(predictors, target, test_size = .2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "### Fitting Models to Data  \n",
    "\n",
    "If the above functions are defined correctly, the following cells should work; creating predictions from your adaptive-boosted model.\n",
    "\n",
    "Try playing around with the number of estimators"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.83      0.93      0.88      4916\n",
      "           1       0.67      0.43      0.52      1597\n",
      "\n",
      "   micro avg       0.81      0.81      0.81      6513\n",
      "   macro avg       0.75      0.68      0.70      6513\n",
      "weighted avg       0.79      0.81      0.79      6513\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import classification_report\n",
    "d = simple_adaboost_fit(X_train.values.copy(), y_train.values.copy(), 50)\n",
    "preds = predict(X_test, d)\n",
    "print(classification_report(y_test,preds))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "This image gives an idea of how the precision and recall on both the training and test set changes as the number of estimators is changed.  \n",
    "\n",
    "![Boosting2](Boosting2.PNG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "<a id = \"sklearn\"></a>\n",
    "### `sklearn` Implementation of Adaptive Boosting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Random Forest:\n",
      "\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.84      0.89      0.87      4916\n",
      "           1       0.59      0.47      0.52      1597\n",
      "\n",
      "   micro avg       0.79      0.79      0.79      6513\n",
      "   macro avg       0.71      0.68      0.69      6513\n",
      "weighted avg       0.78      0.79      0.78      6513\n",
      "\n",
      "\n",
      "AdaBoost:\n",
      "\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.84      0.93      0.89      4916\n",
      "           1       0.69      0.46      0.55      1597\n",
      "\n",
      "   micro avg       0.82      0.82      0.82      6513\n",
      "   macro avg       0.77      0.70      0.72      6513\n",
      "weighted avg       0.81      0.82      0.80      6513\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.ensemble import AdaBoostClassifier, RandomForestClassifier\n",
    "\n",
    "RF = RandomForestClassifier(n_estimators = 50)\n",
    "RF.fit(X_train, y_train)\n",
    "\n",
    "print(\"Random Forest:\\n\")\n",
    "print(classification_report(y_test, RF.predict(X_test)))\n",
    "ABC = AdaBoostClassifier(n_estimators = 50)\n",
    "ABC.fit(X_train, y_train)\n",
    "print(\"\\nAdaBoost:\\n\")\n",
    "print(classification_report(y_test, ABC.predict(X_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": false,
     "locked": true,
     "solution": false
    }
   },
   "source": [
    "You should find that the precision and recall of the your custom Adaptive Boosting are very similar to the `sklearn` adaboost.\n",
    "\n",
    "Notice the higher precision of the AdaBoost compared to the Random Forest"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}