2-Conventional_or_static_machine_learning_algorithms.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A Lightweight Concept Drift Detection and Adaptation Framework for IoT Data Streams \n",
    "This is the code for the paper entitled \"**A Lightweight Concept Drift Detection and Adaptation Framework for IoT Data Streams**\" accepted in IEEE Internet of Things Magazine.  \n",
    "Authors: Li Yang (lyang339@uwo.ca) and Abdallah Shami (Abdallah.Shami@uwo.ca)  \n",
    "Organization: The Optimized Computing and Communications (OC2) Lab, ECE Department, Western University\n",
    "\n",
    "**Notebook 2: Static machine learning algorithms**  \n",
    "Aims:  \n",
    "&nbsp; 1): Find the best performing static machine learning algorithm by considering both accuracy and time  \n",
    "&nbsp; 2): Obtain the optimized learner using hyperparameter optimization techniques  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.model_selection import train_test_split,cross_val_score\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.metrics import classification_report,confusion_matrix,accuracy_score\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from xgboost import XGBClassifier\n",
    "import lightgbm as lgb\n",
    "import time\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Read the NSL-KDD dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#Read the training and test set\n",
    "df1 = pd.read_csv(\"NSL_KDD_binary_train.csv\")\n",
    "df2 = pd.read_csv(\"NSL_KDD_binary_test.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train = df1.iloc[:, :-1].values\n",
    "y_train = df1.iloc[:, -1].values\n",
    "X_test = df2.iloc[:, :-1].values\n",
    "y_test = df2.iloc[:, -1].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>duration</th>\n",
       "      <th>protocol_type</th>\n",
       "      <th>service</th>\n",
       "      <th>flag</th>\n",
       "      <th>src_bytes</th>\n",
       "      <th>dst_bytes</th>\n",
       "      <th>land</th>\n",
       "      <th>wrong_fragment</th>\n",
       "      <th>urgent</th>\n",
       "      <th>hot</th>\n",
       "      <th>...</th>\n",
       "      <th>dst_host_srv_count</th>\n",
       "      <th>dst_host_same_srv_rate</th>\n",
       "      <th>dst_host_diff_srv_rate</th>\n",
       "      <th>dst_host_same_src_port_rate</th>\n",
       "      <th>dst_host_srv_diff_host_rate</th>\n",
       "      <th>dst_host_serror_rate</th>\n",
       "      <th>dst_host_srv_serror_rate</th>\n",
       "      <th>dst_host_rerror_rate</th>\n",
       "      <th>dst_host_srv_rerror_rate</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>45</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>10</td>\n",
       "      <td>0.04</td>\n",
       "      <td>0.06</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>45</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.06</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>19</td>\n",
       "      <td>9</td>\n",
       "      <td>12983</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>86</td>\n",
       "      <td>0.61</td>\n",
       "      <td>0.04</td>\n",
       "      <td>0.61</td>\n",
       "      <td>0.02</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "      <td>9</td>\n",
       "      <td>20</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>57</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.28</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>55</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>15</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>86</td>\n",
       "      <td>0.31</td>\n",
       "      <td>0.17</td>\n",
       "      <td>0.03</td>\n",
       "      <td>0.02</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.83</td>\n",
       "      <td>0.71</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22539</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>49</td>\n",
       "      <td>9</td>\n",
       "      <td>794</td>\n",
       "      <td>333</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>141</td>\n",
       "      <td>0.72</td>\n",
       "      <td>0.06</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22540</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>22</td>\n",
       "      <td>9</td>\n",
       "      <td>317</td>\n",
       "      <td>938</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>255</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22541</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>22</td>\n",
       "      <td>9</td>\n",
       "      <td>54540</td>\n",
       "      <td>8314</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>...</td>\n",
       "      <td>255</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.07</td>\n",
       "      <td>0.07</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22542</th>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>11</td>\n",
       "      <td>9</td>\n",
       "      <td>42</td>\n",
       "      <td>42</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>252</td>\n",
       "      <td>0.99</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22543</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>52</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>21</td>\n",
       "      <td>0.08</td>\n",
       "      <td>0.03</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.44</td>\n",
       "      <td>1.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>22544 rows × 42 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       duration  protocol_type  service  flag  src_bytes  dst_bytes  land  \\\n",
       "0             0              1       45     1          0          0     0   \n",
       "1             0              1       45     1          0          0     0   \n",
       "2             2              1       19     9      12983          0     0   \n",
       "3             0              0       13     9         20          0     0   \n",
       "4             1              1       55     2          0         15     0   \n",
       "...         ...            ...      ...   ...        ...        ...   ...   \n",
       "22539         0              1       49     9        794        333     0   \n",
       "22540         0              1       22     9        317        938     0   \n",
       "22541         0              1       22     9      54540       8314     0   \n",
       "22542         0              2       11     9         42         42     0   \n",
       "22543         0              1       52     1          0          0     0   \n",
       "\n",
       "       wrong_fragment  urgent  hot  ...  dst_host_srv_count  \\\n",
       "0                   0       0    0  ...                  10   \n",
       "1                   0       0    0  ...                   1   \n",
       "2                   0       0    0  ...                  86   \n",
       "3                   0       0    0  ...                  57   \n",
       "4                   0       0    0  ...                  86   \n",
       "...               ...     ...  ...  ...                 ...   \n",
       "22539               0       0    0  ...                 141   \n",
       "22540               0       0    0  ...                 255   \n",
       "22541               0       0    2  ...                 255   \n",
       "22542               0       0    0  ...                 252   \n",
       "22543               0       0    0  ...                  21   \n",
       "\n",
       "       dst_host_same_srv_rate  dst_host_diff_srv_rate  \\\n",
       "0                        0.04                    0.06   \n",
       "1                        0.00                    0.06   \n",
       "2                        0.61                    0.04   \n",
       "3                        1.00                    0.00   \n",
       "4                        0.31                    0.17   \n",
       "...                       ...                     ...   \n",
       "22539                    0.72                    0.06   \n",
       "22540                    1.00                    0.00   \n",
       "22541                    1.00                    0.00   \n",
       "22542                    0.99                    0.01   \n",
       "22543                    0.08                    0.03   \n",
       "\n",
       "       dst_host_same_src_port_rate  dst_host_srv_diff_host_rate  \\\n",
       "0                             0.00                         0.00   \n",
       "1                             0.00                         0.00   \n",
       "2                             0.61                         0.02   \n",
       "3                             1.00                         0.28   \n",
       "4                             0.03                         0.02   \n",
       "...                            ...                          ...   \n",
       "22539                         0.01                         0.01   \n",
       "22540                         0.01                         0.01   \n",
       "22541                         0.00                         0.00   \n",
       "22542                         0.00                         0.00   \n",
       "22543                         0.00                         0.00   \n",
       "\n",
       "       dst_host_serror_rate  dst_host_srv_serror_rate  dst_host_rerror_rate  \\\n",
       "0                      0.00                       0.0                  1.00   \n",
       "1                      0.00                       0.0                  1.00   \n",
       "2                      0.00                       0.0                  0.00   \n",
       "3                      0.00                       0.0                  0.00   \n",
       "4                      0.00                       0.0                  0.83   \n",
       "...                     ...                       ...                   ...   \n",
       "22539                  0.01                       0.0                  0.00   \n",
       "22540                  0.01                       0.0                  0.00   \n",
       "22541                  0.00                       0.0                  0.07   \n",
       "22542                  0.00                       0.0                  0.00   \n",
       "22543                  0.00                       0.0                  0.44   \n",
       "\n",
       "       dst_host_srv_rerror_rate  label  \n",
       "0                          1.00      1  \n",
       "1                          1.00      1  \n",
       "2                          0.00      0  \n",
       "3                          0.00      1  \n",
       "4                          0.71      1  \n",
       "...                         ...    ...  \n",
       "22539                      0.00      0  \n",
       "22540                      0.00      0  \n",
       "22541                      0.07      1  \n",
       "22542                      0.00      0  \n",
       "22543                      1.00      1  \n",
       "\n",
       "[22544 rows x 42 columns]"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training 5 static machine learning algorithms: \n",
    "### KNN, LR, RF, XGBoost, LightGBM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.66      0.97      0.79      9711\n",
      "           1       0.96      0.62      0.76     12833\n",
      "\n",
      "    accuracy                           0.77     22544\n",
      "   macro avg       0.81      0.80      0.77     22544\n",
      "weighted avg       0.83      0.77      0.77     22544\n",
      "\n",
      "Accuracy: 0.7720014194464159\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUgAAAE+CAYAAADvb4nvAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAbv0lEQVR4nO3deXwV1f3/8dcnJJQIyCYg+y6I+wJisdVv6RfRWsG1WJdIaVNb677g1lIVW9t+W5eftRXccKlAUZHaqlXAtQhYQBRQgygSNlkEZZMk9/P74w4YaE5I0rm5Nzfvp495cOfMmZlzCfn4OXNmzpi7IyIi/ykn3Q0QEclUCpAiIgEKkCIiAQqQIiIBCpAiIgEKkCIiAbnpbkClzHQPkkg6uFtNditZt7Tav7N5+3Wv0blqQ0YHyJK1H6a7CVIDea17AJCb1z7NLZGaKC1Zme4mZIyMDpAiUsckytLdglgpQIpIfDyR7hbESgFSROKTUIAUEamQK4MUEQlQBikiEqAMUkQkQKPYIiIByiBFRAJ0DVJEpGIaxRYRCVEGKSISoAxSRCRAo9giIgFZlkFqwlwRkQBlkCISHw3SiIgEZFkXWwFSROKjDFJEpGLuGsUWEamYutgiIgHqYouIBCiDFBEJ0JM0IiIByiBFRAJ0DVJEJEAZpIhIgDJIEZEABUgRkYrpSRoRkRBlkCIiAVk2SKMJc0VEApRBikh81MUWEQnIsi62AqSIxEcZpIhIQJZlkBqkEZH4JBLVX6rAzK4ws4Vm9q6ZPWFmjcysm5nNMrMlZjbRzBpGdb8WrS+Jtnctd5zro/L3zezEvZ1XAVJE4pOCAGlmHYBLgaPd/WCgATAc+A1wh7v3BD4DRka7jAQ+i8rviOphZn2j/Q4ChgD3mlmDys6tACki8fFE9ZeqyQXyzSwX2AdYBXwLmBxtHw8Miz4PjdaJtg8yM4vKJ7j7l+7+EbAE6F/ZSRUgRSQ+Kcgg3X0F8H/AJyQD4ybg38BGdy+NqhUDHaLPHYDl0b6lUf1W5csr2KdCCpAiEp8aZJBmVmhmb5VbCssf0sxakMz+ugHtgcYku8gpp1FsEYlPDW7zcfexwNhKqnwb+Mjd1wKY2VPAQKC5meVGWWJHYEVUfwXQCSiOuuTNgPXlyncqv0+FlEGKSHxScw3yE2CAme0TXUscBCwCZgBnRnUKgGeiz1OjdaLt093do/Lh0Sh3N6AXMLuyEyuDFJH4pOBGcXefZWaTgblAKTCPZMb5d2CCmY2Jyh6IdnkAeNTMlgAbSI5c4+4LzWwSyeBaClzse5mfzZKBNTOVrFuauY2ToLzWPQDIzWuf5pZITZSWrAR3q8m+2ybdUu3f2fyzf1Gjc9UGZZAiEp8MTrhqQgFSROKjZ7FFRAIUIEVEAjRZhYhI/aAMUkTioy62iEiARrFFRAKUQYqIBChAiogEZNkotgKkiMTGE7oGKSJSMXWxRUQC1MUWEQlQF1tEJEBdbBGRgCwLkHoWO0aPTprCsPMuYui5P+bRiU/vtu3hJ57k4IEn8dnGTQAsXbaccwuv4IgTvstDf5m8q96qNWsZ8bNRnHpuYfI4k6bU5lcQoGPH9rz0z7+y4O0ZvD1/Opf8LPm65UMP7cvrr05l3tyXmPL0wzRt2gSAvLw87h/3B+bNfYl/v/Uix3/z2HQ2P73cq79kMGWQMSla+jFPTn2eJ+6/k7zcPC666iaOH3gMnTu2Z9Watfxr9lzatW2zq36zfZty3RUXMf3VmbsdJ7dBA6655Ef07d2TLVu2cvbIS/l6vyPo0a1LbX+lequ0tJRrrr2ZefPfpUmTxsye9TwvTXuV+/78O0aNupVXX3uTCwu+x9VX/YTRv/wdPxz5fQCOOPLbtG7dimf/9hgDjj2ZTJ6tP2WUQVaNmfUxs1Fmdne0jDKzA1N1vnRb+vFyDjmoN/mNGpGb24CjDz+El155A4Df3n0fV/50JFZuYvlWLZpzyIG9yc3d/f9RrfdrSd/ePQFo3HgfunfpxJq162vtewisXv0p8+a/C8DmzVt4770iOrTfnwN6defV194E4KVpr3HaaScDcOCBBzDj5eTPeu3a9Wza+DlHH3VYehqfbgmv/pLBUhIgzWwUMAEwkm8Nmx19fsLMrkvFOdOtZ/cuzH17IRs3fc627dt5beYcVq9Zy/TXZtKm9X706dW92sdcsWoNi4s+5NCDeqegxVIVXbp05PDDDmbW7HksWvQBp556IgBnnnEKnTom37mzYMEivnvKYBo0aEDXrp048shD6Nipnr6PJzVvNUybVHWxRwIHuXtJ+UIz+wOwELg9RedNmx5dO/ODc8+i8IobyW/UiN69urOjpIRxj0xk7B23Vft4W7du44obxzDq0h/TpHHjFLRY9qZx432YNHEcV149mi++2MwPC6/kzj/cyo03XM6zz/6THTuS/7wfengCB/bpxaw3n+OTT4qZOfMtysoqfVle9srwjLC6UhUgE0B7YNke5e2ibUFmVggUAtz7+zH88IJzUtLAVDjjuydyxneTGcadf36YVi2bM/3VmZxR8FMA1qxdx1k/uIQJ4+5kv1Ytg8cpKS3l8hvH8J3B/8P/njCwVtouu8vNzeWvE8fxxBNPM2XKcwC8//6HnPSd5PXGXr26c/JJgwAoKyvjqmt+uWvf1155hqKipbXeZolfqgLk5cA0MysClkdlnYGewM8q29Hdx5J8522de+3r+s820qpFc1at/pRpr7zB42Pv4Pyzh+3aPviMAiY+cDctmjcLHsPd+cWv76R7l04UDD+9FlotFRk39vcsfm8Jd941dldZ69atWLt2PWbGDddfxn1jHwUgP78RZsbWrdv49qBvUFpayuLFRelqelp5lg3SpCRAuvvzZnYA0B/oEBWvAObs7UXdddkVN4xh4+efk5uby41X/ZR9o9tAKrJu/Qa+N/JSNm/ZSk5ODo9NmsIzj9/HB0s+4m/PT6NXj66cUXAxAJf9uIBvfr1/bX2Nem/g1/tx/nlnsuCdRbw1558A/Pznt9OzZzd+8pMLAZgy5R88PH4iAG3a7Mc//v4XEokEK1espmDEpelqevplWRfbMvlWhLqWQUpSXuseAOTm1dOBijqutGQluNvea/6nLWPOq/bvbOObHqvRuWqD7oMUkfhkWQapACki8dE1SBGRAGWQIiIBGX7jd3UpQIpIfJRBiohUTPdBioiEKIMUEQlQgBQRCdAgjYhIgDJIEZGKuQKkiEiAAqSISECW3eajtxqKiAQogxSR+KiLLSISoAApIlKxTJ6AuyYUIEUkPsogRUQCFCBFRCqmG8VFREIUIEVEArLrPnEFSBGJj7rYIiIhCpAiIgHqYouIVCzbutiarEJE4pOowVIFZtbczCab2XtmttjMjjWzlmb2opkVRX+2iOqamd1tZkvMbIGZHVnuOAVR/SIzK9jbeRUgRSQ2nvBqL1V0F/C8u/cBDgMWA9cB09y9FzAtWgc4CegVLYXAnwDMrCUwGjgG6A+M3hlUQxQgRSQ+KcggzawZ8E3gAQB33+HuG4GhwPio2nhgWPR5KPCIJ70JNDezdsCJwIvuvsHdPwNeBIZUdm4FSBGJjSeqv1RBN2At8JCZzTOz+82sMdDW3VdFdVYDbaPPHYDl5fYvjspC5UEKkCKSVmZWaGZvlVsK96iSCxwJ/MndjwC28FV3GgBPTiMU+wiRRrFFJD41uM3H3ccCYyupUgwUu/usaH0yyQC5xszaufuqqAv9abR9BdCp3P4do7IVwAl7lL9cWduUQYpIbFLRxXb31cByM+sdFQ0CFgFTgZ0j0QXAM9HnqcAF0Wj2AGBT1BV/ARhsZi2iwZnBUVmQMkgRiU/qbhS/BHjczBoCS4ERJBO8SWY2ElgGnB3V/QdwMrAE2BrVxd03mNmtwJyo3i3uvqGykypAikhsqjjoUv3jus8Hjq5g06AK6jpwceA4DwIPVvW8CpAiEptUBch0UYAUkdgoQIqIhLiluwWxUoAUkdgogxQRCfCEMkgRkQopgxQRCXBdgxQRqZgySBGRAF2DFBEJ8Ox644ICpIjERxmkiEhAtgVITXcmIhKgDFJEYqNrkCIiAdnWxVaAFJHYZNuN4nu9Bmlmbc3sATN7LlrvG83gKyKymxS91TBtqjJI8zDJ9za0j9Y/AC5PUXtEpA5LuFV7yWRVCZD7ufskordNuHspUJbSVolIneRu1V4yWVWuQW4xs1ZE75zd+ZawlLZKROqk+jhIcyXJ1yj2MLM3gNbAmSltlYjUSfXuNh93n2tmxwO9AQPed/eSlLdMROqcepdBmtkFexQdaWa4+yMpapOI1FGZPuhSXVXpYvcr97kRyffQzgUUIEVkN5k+6FJdVeliX1J+3cyaAxNS1SARqbvq3TXICmwBusXdEBGp++pdF9vM/kZ0iw/J+yb7ApNS2SgRqZvqXRcb+L9yn0uBZe5enKL2iEgdVq+62GbWAPilu/9PLbVnN3mte6TjtBKT0pKV6W6C1LJ61cV29zIzS5hZM3fX0zMiUqn62MXeDLxjZi+SHKABwN0vTVmrInPaD0v1KSQF+q2cAsC2Kb9Jb0OkRvKHjUp3EzJGVQLkU9FSXpZdaRCRONSrLnakubvfVb7AzC5LUXtEpA7LtsypKtOdFVRQdmHM7RCRLJBt80EGM0gzOwf4PtDNzKaW29QU2JDqholI3VOfBmn+BawC9gN+X678C2BBKhslInVThr9BodqCAdLdlwHLgGMrO4CZzXT3SuuISP3g1J8MsqoaxXAMEckCiSwbpYkjQGbZX4mI1FRCGaSISMWyrYtdlfdiX2JmLSqrEmN7RKQOS9RgyWRVuQ+yLTDHzCaZ2RAz2zMgnp+CdolIHeRYtZdMttcA6e43Ab2AB0jeIF5kZr8ysx7R9ndT2kIRqTPqYwaJuzuwOlpKgRbAZDP7bQrbJiJ1TLYFyKrMKH4ZcAGwDrgfuMbdS8wsBygCrk1tE0Wkrsj0LnN1VWUUuyVwenTj+C7unjCzU1LTLBGpi7LstdhVeqvh6Eq2LY63OSJSl2XbfZBVugYpIlIf6UZxEYlNtj1WpwApIrHJ9FHp6lKAFJHYJP7jOZK6TdcgRSQ2XoOlqsysgZnNM7Nno/VuZjbLzJaY2UQzaxiVfy1aXxJt71ruGNdH5e+b2Yl7O6cCpIjEJsU3il8GlL9z5jfAHe7eE/gMGBmVjwQ+i8rviOphZn2B4cBBwBDgXjNrUNkJFSBFJDYJq/5SFWbWEfgOyYdViOaE+BYwOaoyHhgWfR4arRNtHxTVHwpMcPcv3f0jYAnQv7LzKkCKSGwSWLWXKrqT5FN7O5POVsBGdy+N1ouBDtHnDsBygGj7pqj+rvIK9qmQAqSIxKYm1yDNrNDM3iq3FJY/ZvTE3qfu/u/a+yZJGsUWkdjU5FFDdx8LjK2kykDgVDM7meQrXvYF7gKam1lulCV2BFZE9VcAnYBiM8sFmgHry5XvVH6fCimDFJHYpGKQxt2vd/eO7t6V5CDLdHc/F5gBnBlVKwCeiT5PjdaJtk+PZiSbCgyPRrm7kZzGcXZl51YGKSKxqeUnaUYBE8xsDDCP5Jy1RH8+amZLgA0kgyruvtDMJgGLSE7beLG7l1V2AgVIEYlNqmfzcfeXgZejz0upYBTa3bcDZwX2vw24rarnU4AUkdjoUUMRkQAFSBGRAM+uR7EVIEUkPtmWQeo2HxGRAGWQIhKbbMsgFSBFJDaaUVxEJKDevdVQRKSq1MUWEQlQgBQRCdA1SBGRAF2DFBEJUBdbRCRAXWwRkYBEloVIBUgRiY262CIiAdmVPypAikiMlEGKiAToNh8RkQAN0oiIBGRXeNSEuSIiQcogRSQ2GqQREQnQNUgRkYDsCo8KkCISI3WxRUQC1MUWEQnIrvCoACkiMVIXW0QkwLMsh1SAFJHYKIMUEQnQII1ULieHvs/9jpLVGygquI2mxx1Cp5sKsJwcyrZs56Mr7ubLj1fvqt7i5AH0HDeKhSddzdYFH9KgRVN6jr2Gxof1ZN2kGXxy07g0fpn64+NPN3Lt4zN2ra/Y8AU/GXwk/Xq047an3mDrjlLat2jCr845gSaNGlJSWsatT73BouJ15JhxzakD6NejHdt2lHLNY9MoXv8FOTnG8Qd25rKT+6Xxm9Wu7AqPCpCxa/vDU9heVEyDpvsA0PXXF1E04tdsX1JM64IhtL/sLD664v8BkNO4EW1HnsLmue/v2t+372DFb58gv09n8nt3Tst3qI+6tmnOpCtOA6AskWDwmAl86+AuXP3odK78Tn+O7tGOKXM+YPwr73DxiUfx5Ozkz2zylaezYfM2Ln7gBR6/ZCgABd88hH4921NSWkbh2Od4/b3lHNenU9q+W23KtgxSk1XEKK9dK5oPOoq1T7y0q8zdadA0H4DcpvuwY82GXds6XPt9Vt37NIntJbvKEtu+ZPOcxSS+3FF7DZfdzFqyko6tmtK+RVM+WbeJo7rvD8CAXu2Z9s7HACxds5H+PdoB0LJJPk3zG7KweB35DXPp17M9AHm5DejToRVrNm1Jy/dIh0QNlkyWlgBpZiPScd5U63zzD1g+Zjwkvvqxf3z1Hzng0Z9z2FvjaHXGCay65ykA9jm4Ow3b7cemaf9OV3Ml4IX5Sznp8B4AdG/bghkLlwHw4oKPWL0xGewOaNeSlxd9QmlZghUbvmBR8XrWbNq823E+3/Ylry5ezjFRwKwPvAb/ZbJ0ZZA3p+m8KdPs20dTum4TW99Zulv5/j86lQ/Ov5W3j/4R6yZOp/PoEWBGp9EjWH7LQ2lqrYSUlJbxyqJP+N9DuwFw81nfYNLMxZxz1xS2fFlCXm7yV2ZYvwNo26wx37/7GX439U0O69KGHPtqOu3SsgTX/+VlzhnYl46t9k3Ld0mHbMsgU3YN0swWhDYBbSvZrxAoBLi+2eGc1rhr/I1LgaZH96H54H40+9ZR5Hwtj5ym+9DrkRtp1KMjW+YVAbBh6usc8PgvaNAkn/w+nekzeQwAea2b0+uhGyga8Su2LvgwnV+j3nv9/WL6dGhFq+iySLc2zfnzj04CYNnaTbz23nIAchvkcM2pA3btd8Ef/0aX1s12rd/65Ot03m9fzvvGwbXY+vTL9IywulI5SNMWOBH4bI9yA/4V2sndxwJjAeZ0OK3O/G0X3/4Yxbc/BkDTYw9i/4uGUfSDX3P4/If4Wvf2fLl0Jft+8zC2FRVT9sVW5h9SsGvf3n+9leW3PqzgmAGen/8hQ6LuNcCGzdto2SSfRMIZN20+Zw04EIBtO0oBJ79hHjM/WEFujtGjbQsA7nn+LTZvL2H0md9Ix1eQGKUyQD4LNHH3+XtuMLOXU3jezFGW4ONr7qXn2GvBE5Ru3MLHV92z190OffM+GjTJxxrm0mJIf94/52a2FxXXQoPrt207SnizaCU3nX7crrLn5n/IxH8tBmDQwV0ZenQvIBk4f3r/C+TkQJt9GzNm+PEArNm4hfunv023Ns0YftcUAIZ/vS+nH9O7dr9MmmR6l7m6zD1zk7S6lEHKV/qtnALAtim/SW9DpEbyh40C9xq9n/D8LqdX+3f20WVPZey7EHUfpIjEJtsyGgVIEYlNtt0orgApIrHRKLaISEC2DdIoQIpIbNTFFhEJUBdbRCRAXWwRkYBMvq+6JhQgRSQ22XYNUvNBikhsUjGbj5l1MrMZZrbIzBaa2WVReUsze9HMiqI/W0TlZmZ3m9kSM1tgZkeWO1ZBVL/IzApC59xJAVJEYpOi+SBLgavcvS8wALjYzPoC1wHT3L0XMC1aBzgJ6BUthcCfIBlQgdHAMUB/YPTOoBqiACkisUng1V72xt1Xufvc6PMXwGKgAzAUGB9VGw8Miz4PBR7xpDeB5mbWjuTsYi+6+wZ3/wx4ERhS2bl1DVJEYpPqQRoz6wocAcwC2rr7qmjTar6aZ7YDsLzcbsVRWag8SBmkiMSmJtcgzazQzN4qtxRWdGwzawI8CVzu7p+X3+bJyBx7dFYGKSKxqcmN4uUnyQ4xszySwfFxd38qKl5jZu3cfVXUhf40Kl8BlH+NZMeobAVwwh7lL1d2XmWQIpLRzMyAB4DF7v6HcpumAjtHoguAZ8qVXxCNZg8ANkVd8ReAwWbWIhqcGRyVBSmDFJHYpOg+yIHA+cA7ZjY/KrsBuB2YZGYjgWXA2dG2fwAnA0uArcAIAHffYGa3AnOiere4+1fvYa6AAqSIxCYVgzTu/jrJd1lVZFAF9R24OHCsB4EHq3puBUgRiU22PUmjACkisdFsPiIiAQlNViEiUrHsCo8KkCISI12DFBEJUIAUEQnQhLkiIgHKIEVEAnSbj4hIgLrYIiIB6mKLiAQogxQRCVAGKSISkG2DNJowV0QkQBmkiMRGk1WIiARkWxdbAVJEYqMMUkQkQBmkiEiAMkgRkQBlkCIiAcogRUQClEGKiAS4J9LdhFgpQIpIbPQstohIgGbzEREJUAYpIhKgDFJEJEC3+YiIBOg2HxGRgGzrYmvCXBGRAGWQIhIbjWKLiARkWxdbAVJEYqNRbBGRAGWQIiIBugYpIhKgDFJEJEDXIEVEAvQkjYhIgDJIEZEAXYMUEQlQF1tEJEAZpIhIQLYFSMvoL2SWwY0TyWLuVpPdcht2qPbvbOmOFTU6V23I7ACZxcys0N3HprsdUjP6+dUPmg8yfQrT3QD5r+jnVw8oQIqIBChAiogEKECmj65f1W36+dUDGqQREQlQBikiEqAAmQZmNsTM3jezJWZ2XbrbI1VnZg+a2adm9m662yKppwBZy8ysAfBH4CSgL3COmfVNb6ukGh4GhqS7EVI7FCBrX39gibsvdfcdwARgaJrbJFXk7q8CG9LdDqkdCpC1rwOwvNx6cVQmIhlGAVJEJEABsvatADqVW+8YlYlIhlGArH1zgF5m1s3MGgLDgalpbpOIVEABspa5eynwM+AFYDEwyd0XprdVUlVm9gQwE+htZsVmNjLdbZLU0ZM0IiIByiBFRAIUIEVEAhQgRUQCFCBFRAIUIEVEAhQgRUQCFCAlo5jZhWZ2T7rbIQIKkFJLomneROoUBUipkJndYmaXl1u/zcwuq6DeCWb2qpn9PZoE+M9mlhNt22xmvzezt4Fjzew8M5ttZvPN7L6dQdPMRpjZB2Y2GxhYS19RZK8UICXkQeACgCjgDQceC9TtD1xCcgLgHsDpUXljYJa7HwasB74HDHT3w4Ey4FwzawfcTDIwHhcdQyQj5Ka7AZKZ3P1jM1tvZkcAbYF57r4+UH22uy+FXc8qHwdMJhkEn4zqDAKOAuaYGUA+8ClwDPCyu6+N9p8IHJCabyVSPQqQUpn7gQuB/UlmlCF7PtC/c327u5dFnw0Y7+7Xl69oZsP++2aKpIa62FKZp0m+f6UfydmHQvpH07flkOxGv15BnWnAmWbWBsDMWppZF2AWcLyZtTKzPOCsWL+ByH9BGaQEufsOM5sBbCyXCVZkDnAP0BOYQTKw7nmsRWZ2E/DPKJCWABe7+5tm9kuSU4htBObH+iVE/gua7kyCokA2FzjL3YsCdU4Arnb3U2qxaSK1Ql1sqVD0KtolwLRQcBTJdsogpUrM7BDg0T2Kv3T3Y9LRHpHaoAApIhKgLraISIACpIhIgAKkiEiAAqSISIACpIhIwP8HKVA+FF2aH5sAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 360x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 13 s\n"
     ]
    }
   ],
   "source": [
    "%%time \n",
    "# Print out execution time\n",
    "\n",
    "# KNN algorithm\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "classifier = KNeighborsClassifier()\n",
    "classifier.fit(X_train, y_train) # Model training\n",
    "y_pred = classifier.predict(X_test) # Model testing\n",
    "print(classification_report(y_test,y_pred))\n",
    "print(\"Accuracy: \"+ str(accuracy_score(y_test, y_pred)))\n",
    "\n",
    "# Plot the confusion matrix\n",
    "cm=confusion_matrix(y_test,y_pred)\n",
    "f,ax=plt.subplots(figsize=(5,5))\n",
    "sns.heatmap(cm,annot=True,linewidth=0.5,linecolor=\"red\",fmt=\".0f\",ax=ax)\n",
    "plt.xlabel(\"y_pred\")\n",
    "plt.ylabel(\"y_true\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.60      0.90      0.72      9711\n",
      "           1       0.88      0.55      0.68     12833\n",
      "\n",
      "    accuracy                           0.70     22544\n",
      "   macro avg       0.74      0.73      0.70     22544\n",
      "weighted avg       0.76      0.70      0.70     22544\n",
      "\n",
      "Accuracy: 0.7026703335699077\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUgAAAE+CAYAAADvb4nvAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAhIElEQVR4nO3deZwV1Z338c+XbnaQXcL2TFCJBp1oXHCNawQ1UXhi4pjEiIY8ZOI+k5loluchGpNxsjnjK4kTEhc0RiROjJhoFBE0JlEgghvqgKiRFhXZlLXp7t/zxz20V+jqLXV7uXzfvurVVadO1TlX6B+/U6eqriICMzPbVZf27oCZWUflAGlmlsEB0swsgwOkmVkGB0gzswwOkGZmGSrbuwONknwPkll7iFBrDtv+1ooW/852HbxXq9pqCx06QG5f/WJ7d8FaoeuQvQGo7Dq8nXtirVGz/bX27kKH0aEDpJl1MnW17d2DXDlAmll+oq69e5ArB0gzy0+dA6SZWYPCGaSZWQZnkGZmGZxBmpll8Cy2mVkGZ5BmZhl8DdLMrGGexTYzy+IM0swsgzNIM7MMnsU2M8tQZhmkX5hrZpbBGaSZ5ceTNGZmGcpsiO0AaWb5cQZpZtawCM9im5k1rMyG2J7FNrP81NW1fGkGSf8k6VlJz0i6XVIPSaMlPS5puaQ7JHVLdbun7eVp//uLzvPVVP6CpAlNtesAaWb5ibqWL02QNAK4BDg0Ig4AKoCzgX8Hro2IfYB1wJR0yBRgXSq/NtVD0th03P7AKcBPJFU01rYDpJnlp6625UvzVAI9JVUCvYBVwInAnWn/DGBSWp+Ytkn7T5KkVD4zIrZFxEvAcmBcY406QJpZfkqQQUZEFfB94K8UAuMG4C/A+oioSdVWAiPS+gjg1XRsTao/qLi8gWMa5ABpZvlpxTVISVMlLSpaphafUtIACtnfaGA40JvCELnkPIttZvlpxSx2REwHpjdS5aPASxGxGkDSr4Gjgf6SKlOWOBKoSvWrgFHAyjQk7wesKSrfofiYBjmDNLP8lGYW+6/AEZJ6pWuJJwFLgXnAJ1OdycDdaX122ibtfygiIpWfnWa5RwNjgAWNNewM0szyU4InaSLicUl3Ak8ANcBiChnn74CZkq5OZTekQ24AbpW0HFhLYeaaiHhW0iwKwbUGuDCauLNdhcDaMW1/a0XH7Zxl6jpkbwAquw5v555Ya9Rsfw0i1Jpjtzxyc4t/Z3see16r2moLziDNLD9+FtvMLIMfNTQz2z04gzSz/HiIbWaWocyG2A6QZpYfZ5BmZhmcQZqZZXAGaWaWwQHSzCyDh9hmZhmcQZqZZXAGaWaWwRmkmVkGZ5BmZhmcQZqZZXCANDPL0IFfwN0aDpBmlh9nkGZmGRwgzcwylNkstt8obmaWwRmkmeXHQ2wzswyexTYzy+AM0swsgwOkmVkGz2KbmTUs6qLFS1Mk7StpSdHytqTLJA2UNEfSsvRzQKovSddJWi7pKUkHF51rcqq/TNLkptp2gDSz/NTVtXxpQkS8EBEHRcRBwCHAZuAu4ApgbkSMAeambYBTgTFpmQpcDyBpIDANOBwYB0zbEVSzOECaWX6iruVLy5wEvBgRrwATgRmpfAYwKa1PBG6JgseA/pKGAROAORGxNiLWAXOAUxprzNcgzSw/zRgy/43OBm5P60MjYlVafx0YmtZHAK8WHbMylWWVZ3IGaWb5acUQW9JUSYuKlqkNnVpSN+AM4Fc774uIAHKPzs4gzSw/rbjNJyKmA9ObUfVU4ImIeCNtvyFpWESsSkPoN1N5FTCq6LiRqawKOH6n8vmNNegMMke3zLyLiZ/9IpPO+Uf+ddo1bNtWzblf+hfOnHwhZ06+kBPO+CyXXHEVAAueeIojxp9Zv+/6G2+rP883vvNDjv3Y2Uw65x/b66Ps9i6+aApLFs/lySUPccnFX6gvv/CC83nm6Yd5cslDXPNvX68vv/wrF/H80kd59plHGH/yce3R5Y4houVL832ad4fXALOBHTPRk4G7i8rPTbPZRwAb0lD8fmC8pAFpcmZ8KsvkDDInb6x+i9vuvJu7b/spPbp358v/9zvc9+DD3HL99+vrXPa1qznhI0fUbx984AH85HtX7nKuSaedzGfOPIOvfev7u+yz0tt//32ZMuUzHHnUx6iu3s69v72N3937IKNGDueM0ydw8CEnU11dzZAhgwD44AfHcNZZE/nQQScyfPhQ7r9vJh/c/yPUldlN081Sos8sqTdwMvDFouJrgFmSpgCvAGel8nuB04DlFGa8zweIiLWSvgUsTPWuioi1jbVbsgApaT8Ks0k7LoJWAbMj4rlStdneampr2batmsqKSrZs3caQwQPr923ctIkFTzzJ1V//pybPc+hBf0/VqjearGelsd9+Y1iwYDFbtmwF4JE/PMb/nnQqhxxyIN/93o+prq4GYPXqNQCccfoEZs26m+rqal5++VVefPFlxh32YR57/C/t9hnaTYkmaSJiEzBop7I1FGa1d64bwIUZ57kRuLG57ZZkiC3pcmAmIGBBWgTcLumKxo7trIYOGcx5nz6Tj37iXE6Y+Bn69u7F0YcfUr9/7iN/5vBDDqRP7971ZU8+8xyfmHwB//jl/8vyFa+0R7etAc8++zzHHHM4AwcOoGfPHpx6yomMHDmcMWP24phjxvGnR+/hoQfv5NBDDgRg+PD38erK1+qPX1m1iuEj3tde3W9fpb/Np02VKoOcAuwfEduLCyX9EHiWQmpcVja8/Q7z/vAY9//qJvr27cOXv/Ed7rn/IU6fcCIA9z34MGd+fEJ9/bH77s2c/55Br149eeRPC7jkq1dx7x03tFf3rcjzzy/ne9/7Mffd+0s2b9rMkiefpba2jsrKCgYM6M9Rx5zOYYcexO2//C/G7Htke3e3Yyn9bT5tqlSTNHXA8AbKh6V9mYqn/H9+y+2NVe1QHlu0hBHDhzJwQH+6VlZy0nFHseTppQCsW7+Bp5e+wLFHjauv36d3b3r16gnAsUeNo6amhnXrN7RL321XN908k8OPOJUTTjqT9es3sGzZCqpWruI3v7kPgIWLllBXV8fgwQN57bXXGTXy3b/uI0cM47Wq19ur65ajUmWQlwFzJS3j3Rsz/xewD3BRYwcWT/lvf2tFp/nnaNjQITz1zPNs2bqVHt278/iiJey/3xgAHpj3KMcdNY7u3bvV139rzVoGDRyAJJ5e+gJ1EfTvt0d7dd92MmTIIFavXsOoUcOZNOlUjj7mdOrq6jj++KOY//CfGDNmL7p168Zbb63lnt8+wK23/Jhr/2M6w4cPZZ99RrNg4eL2/gjtIspsYqokATIifi/pAxSedyyepFkYEbWlaLO9fWj//Tj5hGM46/yLqaioYL8P7M2nJp4KwH1zH+YL55z1nvoPzHuUO+76HRWVFfTo1o3vXXkFkgD412nXsHDxU6xf/zYnTTqHC6Z8jjNPn7BLm1Y6v7rjZwwcNIDt22u45JKvs2HD29x080x+/rMfsGTxXKqrt/P5KZcBsHTp/3Dnnffw9JPzqKmt5ZJLv757zmBD2Q2xFR34DcCdKYO0d3UdsjcAlV0buspiHV3N9tcgQq05dtPV57T4d7b3N37Rqrbagu+DNLP8lFkG6QBpZvkps0sLDpBmlh9nkGZmGTr4jd8t5QBpZvlxBmlm1jDfB2lmlsUZpJlZBgdIM7MMnqQxM8vgDNLMrGHhAGlmlsEB0swsQ5nd5uNvNTQzy+AM0szy4yG2mVkGB0gzs4Z15Bdwt4YDpJnlxxmkmVkGB0gzs4aV243ivs3HzPJTFy1fmkFSf0l3Snpe0nOSjpQ0UNIcScvSzwGpriRdJ2m5pKckHVx0nsmp/jJJk5tq1wHSzPJT14qlef4T+H1E7AccCDwHXAHMjYgxwNy0DXAqMCYtU4HrASQNBKYBh1P4SuppO4JqFgdIM8tN1EWLl6ZI6gccC9wAEBHVEbEemAjMSNVmAJPS+kTglih4DOgvaRgwAZgTEWsjYh0wBzilsbYdIM0sP6UZYo8GVgM3SVos6eeSegNDI2JVqvM6MDStjwBeLTp+ZSrLKs/kAGlm+WnFEFvSVEmLipapO521EjgYuD4iPgxs4t3hNABRuAEz9xkiz2KbWW5aM4sdEdOB6Y1UWQmsjIjH0/adFALkG5KGRcSqNIR+M+2vAkYVHT8ylVUBx+9UPr+xvjmDNLP8lGCSJiJeB16VtG8qOglYCswGdsxETwbuTuuzgXPTbPYRwIY0FL8fGC9pQJqcGZ/KMjmDNLPclPA+yIuB2yR1A1YA51NI8GZJmgK8ApyV6t4LnAYsBzanukTEWknfAhameldFxNrGGnWANLP8lOh1kBGxBDi0gV0nNVA3gAszznMjcGNz23WANLPclNl3dvkapJlZFmeQZpafMssgHSDNLDflNsR2gDSz/DhAmpk1zBmkmVkGB0gzswwOkGZmWULt3YNcOUCaWW6cQZqZZYg6Z5BmZg1yBmlmliF8DdLMrGHOIM3MMvgapJlZhijZ+3LbhwOkmeXGGaSZWYZyC5B+Ya6ZWQZnkGaWG1+DNDPLUG5DbAdIM8tNud0o3uQ1SElDJd0g6b60PTZ9D62Z2XtEXcuXjqw5kzQ3A/cDw9P2/wCXlag/ZtaJ1YVavHRkzQmQgyNiFunbJiKiBqgtaa/MrFOKUIuXjqw51yA3SRoEBICkI4ANJe2VmXVK5TZJ05wM8p+B2cDekv4I3AJcXNJemVmnFNHypTkkvSzpaUlLJC1KZQMlzZG0LP0ckMol6TpJyyU9JengovNMTvWXSZrcVLtNZpAR8YSk44B9AQEvRMT25n0sM9udlDiDPCEi3iravgKYGxHXSLoibV8OnAqMScvhwPXA4ZIGAtOAQymMiP8iaXZErMtqsMkAKencnYoOlkRE3NKCD2Zmu4E2nnSZCByf1mcA8ykEyInALRERwGOS+ksalurOiYi1AJLmAKcAt2c10JxrkIcVrfcATgKeoDDUNjOrV8JJlwAekBTATyNiOjA0Ilal/a8DQ9P6CODVomNXprKs8kzNGWK/53qjpP7AzKaOM7PdT2seNZQ0FZhaVDQ9BcBix0RElaQ9gTmSnn9vuxEpeOaqNU/SbAJG590RM+v8WjPETsFw54C4c52q9PNNSXcB44A3JA2LiFVpCP1mql4FjCo6fGQqq+LdIfmO8vmNtducJ2nukTQ7Lb8FXgDuauo4M9v9lOI+SEm9JfXdsQ6MB56hcHfNjpnoycDdaX02cG6azT4C2JCG4vcD4yUNSDPe41NZpuZkkN8vWq8BXomIlc04zsx2MyV6m89Q4C5JUIhZv4yI30taCMxKjz6/ApyV6t8LnAYsBzYD5xf6FmslfQtYmOpdtWPCJouikU8kqQJ4MCJOaO0n+5uU4JqCmTVDK2dbFo2c1OLf2UNX/qbD3l3eaAYZEbWS6iT1iwg/PWNmjerojw62VHOG2BuBp9M9Q5t2FEbEJSXrVfLW+GNL3YSVwOAHHgFg8/Ul/ytiJdDrS9e1dxc6jOYEyF+npZiHvma2i47+dp6Wak6A7B8R/1lcIOnSEvXHzDqxcsucmvOyioYe6D4v536YWRkot/dBZmaQkj4NfAYYLWl20a6+QKNT42a2e9qdJmn+BKwCBgM/KCp/B3iqlJ0ys86pg3+DQotlBsiIeIXCzZdHNnYCSX+OiEbrmNnuIdh9Msjm6pHDOcysDNSV2SxNHgGyzP6XmFlr1TmDNDNrWLkNsZvzNp+Ld3zXQ1aVHPtjZp1YXSuWjqw590EOBRZKmiXpFKVXahT5XAn6ZWadUKAWLx1ZkwEyIr5B4ctvbqBwg/gySd+RtHfa/0xJe2hmncbumEGSvvzm9bTUAAOAOyV9t4R9M7NOptwCZHO+1fBS4FzgLeDnwL9GxHZJXYBlwFdK20Uz6yw6+pC5pZoziz0Q+ES6cbxeRNRJ+nhpumVmnVFpvxa77TXnWw2nNbLvuXy7Y2adWbndB9msa5BmZrsj3yhuZrkpt8fqHCDNLDcdfVa6pRwgzSw3dbs8R9K5OUCaWW48xDYzy+AhtplZht3uPkgzs+byfZBmZhmiFUtzSaqQtFjSb9P2aEmPS1ou6Q5J3VJ597S9PO1/f9E5vprKX5A0oak2HSDNLDd1avnSApcCxU/v/TtwbUTsA6wDpqTyKcC6VH5tqoekscDZwP7AKcBPJFU01qADpJnlplRv85E0EvgYhRfmkN5LeyJwZ6oyA5iU1iembdL+k1L9icDMiNgWES8By4FxjbXrAGlmuSnhEPs/KLw5bEdMHQSsj4iatL0SGJHWRwCvAqT9G1L9+vIGjmmQA6SZ5aY1Q2xJUyUtKlqmFp8zvTXszYj4S1t/Hs9im1luWnMfZERMB6Y3UuVo4AxJp1H4muk9gP8E+kuqTFniSKAq1a8CRgErJVUC/YA1ReU7FB/TIGeQZpabUlyDjIivRsTIiHg/hUmWhyLis8A84JOp2mTg7rQ+O22T9j+UvhVhNnB2muUeTeGrZBY01rYzSDPLTbTtbZCXAzMlXQ0spvC9WaSft0paDqylEFSJiGclzQKWUvjqmAsjoraxBhwgzSw3pX7UMCLmA/PT+goamIWOiK3ApzKO/zbw7ea25yG2mVkGZ5Bmlhu/rMLMLINfd2ZmlsFv8zEzy+AhtplZBgdIM7MMvgZpZpbB1yDNzDJ4iG1mlsFDbDOzDHVlFiIdIM0sNx5im5llKK/80QHSzHLkDNLMLINv8zEzy+BJGjOzDOUVHv3CXDOzTM4gzSw3nqQxM8vga5BmZhnKKzw6QJpZjjzENjPL4CG2mVmG8gqPDpBmliMPsc3MMkSZ5ZC+UdzMclPXiqUpknpIWiDpSUnPSroylY+W9Lik5ZLukNQtlXdP28vT/vcXneurqfwFSROaatsB0sxyU0e0eGmGbcCJEXEgcBBwiqQjgH8Hro2IfYB1wJRUfwqwLpVfm+ohaSxwNrA/cArwE0kVjTXsIXbOBsyYSWzZAnW1RG0tGy7+In2/No2KkaMAUO8+xKaNrL/gC/XHdBmyJwN+NoPNv7iZLXfeAV270e8H16GuXaGiguo/PMzmW29qr4+0W3h57UYuv/fJ+u2qDZv50pFj+PjY4Vz+uyd57e0tDN+jJ9/92EHs0aMrb2/dzjcfeJqVGzbTraKCb44/gH0G9wXgjy+v5nvzn6OuDiYdMJLPj9urvT5WmyvFADsiAtiYNrumJYATgc+k8hnAN4HrgYlpHeBO4EeSlMpnRsQ24CVJy4FxwJ+z2naALIENX7mMeHtD/fY737myfr331Auo27TpPfV7f/FCqhcueLdgezUbvvJPsHULVFTQ74c/onLh49Q8v7Tkfd9dvX9gH+4452gAauuCCT+bxwn7DOWmBS8xbtQgPj9uL25csIKbFq7g0o/syw0LXmTfIXvwwzMO5qW1G7nmoaX89JPjqK0LrnloKdd/4jCG9u3BZ3/5Z47be0/2HtSnnT9h2yjVbT4p0/sLsA/wY+BFYH1E1KQqK4ERaX0E8CpARNRI2gAMSuWPFZ22+JgGeYjdxrodewLb5j347vaRx1D7+ipqX3npvRW3bin8rKxEFZUQ5XXxuyNb8OoaRvbrxfA9ejJ/xRucPnY4AKePHc68F98AYMXaTRw2aiAAowf24bW3t7Bm0zaeeX09o/r3YmT/XnSt6MKEfd/H/HTM7qA11yAlTZW0qGiZuvN5I6I2Ig4CRlLI+vZrg4/TPgFS0vnt0W5b6fed79P/R9Ppfurp7ymvPOBD1K1bS91rVYWCHj3pedZn2PyLGbuepEsX+v/k5wy64zdUL15EzQvPtUHPDeD+F1Zxyn7DAFizuZohfXoAMLh3d9ZsrgbgA4P78tDyQuB75vX1rHp7K29s3MqbG7cxtG/P+nMN7dOD1Ru3tfEnaD/Rmv8ipkfEoUXL9MzzR6wH5gFHAv0l7RgFjwTSLxZVwCiAtL8fsKa4vIFjGtReGeSVTVfpnDb880Wsv+j/sOHrX6HnGZOoPOBD9fu6n/BRqufPrd/u9bnz2HLXr97NFovV1bH+gi+w9rOfonLfD1Lxd6Pbovu7ve21dTz84pucPOZ9u+yTxI4XZp9/2F68s62Gf/jFH5m5+K/su2dfKlRmr9NuhRLNYg+R1D+t9wROBp6jECg/mapNBu5O67PTNmn/Q+k65mzg7DTLPRoYAxRd29pVya5BSnoqaxcwtJHjpgJTAX4wdgznjhxWgt6VTt2atwCIDeup/uMf6LrfB6l55inoUkH3oz/C+oveHT103W8s3Y85jt5Tvoj69IEIorqarbPvqq8Tmzay/cnFdDtsHFt2HoZb7h59eTX77bkHg3p3B2BQr26s3riVIX16sHrjVgb26gZAn+6VXDnh7wGICD5248OM6NeLrTW1vPHOu//gvbFxK0P6dG/7D9JOSnQf5DBgRroO2QWYFRG/lbQUmCnpamAxcEOqfwNwa5qEWUth5pqIeFbSLGApUANcGBG1jTVcykmaocAECtPvxQT8KeuglF5PB3hrwnGd68Jb9x6oiwqz2N170PWQw9h8W2H43PXgQ6h99a/UvbW6vvqGL19cv97rnPOIrVvYOvsu1K8f1NQSmzZCt250O/hQNs/6ZZt/nN3R759/d3gNcNxee3LP0tf4/Li9uGfpaxy/V+Hf9ne2bqdH1wq6VnThrmdWcvCIgfTpXsn+7+vHX9dtpmrDZvbs04P7X3idfzv1Q1nNWTNExFPAhxsoX0HheuTO5VuBT2Wc69vAt5vbdikD5G+BPhGxZOcdkuaXsN1202XAAPaYdnVho6KCbfMeZPuiQgbf/bgT2VY0vG70PAMH0fdfvgZdukAXse2R+Wx/PPNOBMvJlu01PP7XNXzjo/vXl51/2F5c/rsl/ObZlQzr25PvfvxAAFas3cj/u/9pJNh7UF+mnXwAAJVdunD5iWO54NeLqItg4v4j2Tvd/rM7KLdHDRUdeHa002WQBsDgBx4BYPP1l7RzT6w1en3pOoho1QXVz/3dJ1r8O3vrK7/usBdvfR+kmeWm3DIaB0gzy43fB2lmlqHc3ubjAGlmuSm3SRoHSDPLjYfYZmYZPMQ2M8vgIbaZWYaOfF91azhAmllufA3SzCyDh9hmZhk8SWNmlsFDbDOzDJ6kMTPL4GuQZmYZyu0apL/V0MwsgzNIM8uNJ2nMzDJ4ksbMLIMzSDOzDOU2SeMAaWa5qfMQ28ysYeUVHh0gzSxHvgZpZpah3AKkbxQ3s9xERIuXpkgaJWmepKWSnpV0aSofKGmOpGXp54BULknXSVou6SlJBxeda3Kqv0zS5KbadoA0s9zUES1emqEG+HJEjAWOAC6UNBa4ApgbEWOAuWkb4FRgTFqmAtdDIaAC04DDgXHAtB1BNYsDpJnlJlrxX5PnjFgVEU+k9XeA54ARwERgRqo2A5iU1icCt0TBY0B/ScOACcCciFgbEeuAOcApjbXta5BmlptSP0kj6f3Ah4HHgaERsSrteh0YmtZHAK8WHbYylWWVZ3IGaWa5ac0QW9JUSYuKlqkNnVtSH+C/gcsi4u3ifVGIzLlHZ2eQZpab1mSQETEdmN5YHUldKQTH2yLi16n4DUnDImJVGkK/mcqrgFFFh49MZVXA8TuVz2+sXWeQZpabUkzSSBJwA/BcRPywaNdsYMdM9GTg7qLyc9Ns9hHAhjQUvx8YL2lAmpwZn8oyOYM0s9yU6Fnso4HPAU9LWpLKvgZcA8ySNAV4BTgr7bsXOA1YDmwGzgeIiLWSvgUsTPWuioi1jTXsAGlmHVpEPAooY/dJDdQP4MKMc90I3Njcth0gzSw3flmFmVkGv+7MzCyDM0gzswzOIM3MMjiDNDPL4AzSzCyDM0gzswzOIM3MMkTUtXcXcuUAaWa5KbevXHCANLPclPp9kG3NAdLMcuMM0swsgzNIM7MMvs3HzCyDb/MxM8tQbkNsf+WCmVkGZ5BmlhvPYpuZZSi3IbYDpJnlxrPYZmYZnEGamWXwNUgzswzOIM3MMvgapJlZBj9JY2aWodwySD9JY2a5iYgWL02RdKOkNyU9U1Q2UNIcScvSzwGpXJKuk7Rc0lOSDi46ZnKqv0zS5OZ8HgdIM8tNtOK/ZrgZOGWnsiuAuRExBpibtgFOBcakZSpwPRQCKjANOBwYB0zbEVQb4wBpZrkpRQYZEY8Aa3cqngjMSOszgElF5bdEwWNAf0nDgAnAnIhYGxHrgDnsGnR34QBpZrlpTYCUNFXSoqJlajOaGhoRq9L668DQtD4CeLWo3spUllXeqA49STP4gUfauwv2N+j1pevauwvWxlozRRMR04HprW4zIiSVZHaoQwdIItTeXSgVSVPTXwzrhPzn17Ca6qq2+p19Q9KwiFiVhtBvpvIqYFRRvZGprAo4fqfy+U014iF2+2nOMMI6Lv/5ta/ZwI6Z6MnA3UXl56bZ7COADWkofj8wXtKANDkzPpU1qmNnkGa225N0O4Xsb7CklRRmo68BZkmaArwCnJWq3wucBiwHNgPnA0TEWknfAhameldFxM4TP7u2XW7PTnYWkhZFxKHt3Q9rHf/57R48xG4/vn7VufnPbzfgDNLMLIMzSDOzDA6Q7UDSKZJeSM+LXtH0EdZRNPRcsJUvB8g2JqkC+DGFZ0bHAp+WNLZ9e2UtcDPNeETNyoMDZNsbByyPiBURUQ3MpPD8qHUCGc8FW5lygGx7rXom1MzangOkmVkGB8i2l/WsqJl1MA6QbW8hMEbSaEndgLMpPD9qZh2MA2Qbi4ga4CIKD8o/B8yKiGfbt1fWXOm54D8D+0pamZ4FtjLlJ2nMzDI4gzQzy+AAaWaWwQHSzCyDA6SZWQYHSDOzDA6QZmYZHCCtQ5F0nqQftXc/zMAB0tpIes2bWafiAGkNknSVpMuKtr8t6dIG6h0v6RFJv0svAf4vSV3Svo2SfiDpSeBISedIWiBpiaSf7giaks6X9D+SFgBHt9FHNGuSA6RluRE4FyAFvLOBX2TUHQdcTOEFwHsDn0jlvYHHI+JAYA3wD8DREXEQUAt8Nn3p+5UUAuMx6RxmHYK/F9saFBEvS1oj6cPAUGBxRKzJqL4gIlZA/bPKxwB3UgiC/53qnAQcAiyUBNATeBM4HJgfEavT8XcAHyjNpzJrGQdIa8zPgfOA91HIKLPs/ED/ju2tEVGb1gXMiIivFleUNOlv76ZZaXiIbY25i8L3rxxG4e1DWcal17d1oTCMfrSBOnOBT0raE0DSQEl/BzwOHCdpkKSuwKdy/QRmfwNnkJYpIqolzQPWF2WCDVkI/AjYB5hHIbDufK6lkr4BPJAC6Xbgwoh4TNI3KbxCbD2wJNcPYfY38OvOLFMKZE8An4qIZRl1jgf+JSI+3oZdM2sTHmJbg9JX0S4H5mYFR7Ny5wzSmkXS3wO37lS8LSIOb4/+mLUFB0gzswweYpuZZXCANDPL4ABpZpbBAdLMLIMDpJlZhv8PhsDly5QR5MYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 360x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 2.02 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# Logistic regression\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "classifier = LogisticRegression()\n",
    "classifier.fit(X_train, y_train)\n",
    "y_pred = classifier.predict(X_test)\n",
    "print(classification_report(y_test,y_pred))\n",
    "print(\"Accuracy: \"+ str(accuracy_score(y_test, y_pred)))\n",
    "\n",
    "# Plot the confusion matrix\n",
    "cm=confusion_matrix(y_test,y_pred)\n",
    "f,ax=plt.subplots(figsize=(5,5))\n",
    "sns.heatmap(cm,annot=True,linewidth=0.5,linecolor=\"red\",fmt=\".0f\",ax=ax)\n",
    "plt.xlabel(\"y_pred\")\n",
    "plt.ylabel(\"y_true\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.66      0.97      0.78      9711\n",
      "           1       0.97      0.61      0.75     12833\n",
      "\n",
      "    accuracy                           0.77     22544\n",
      "   macro avg       0.81      0.79      0.77     22544\n",
      "weighted avg       0.83      0.77      0.76     22544\n",
      "\n",
      "Accuracy: 0.7682753726046841\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUgAAAE+CAYAAADvb4nvAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAcOElEQVR4nO3dd5wV1d3H8c9vd0F6FZAOCnYsgIpBHwsEwUQhiiUxSpA8JLHXRGNvSTQa66NPEI1ofCh27AV7VIpC7MhKkd6WIgiyu/f3/HGHzYr3LLvr3L13737fvubFnTNn7pxh2Z+/M2fmjLk7IiLyfXmZboCISLZSgBQRCVCAFBEJUIAUEQlQgBQRCVCAFBEJKMh0AypkpnuQRDLB3aqzW/GquVX+na23487VOlZNyOoAWbzyy0w3QaqhXptdACio1yHDLZHqKClekukmZI2sDpAiUsskSjPdglgpQIpIfDyR6RbESgFSROKTUIAUEUnJlUGKiAQogxQRCVAGKSISoFFsEZEAZZAiIgG6BikikppGsUVEQpRBiogEKIMUEQnQKLaISECOZZCaMFdEJEAZpIjER4M0IiIBOdbFVoAUkfgogxQRSc1do9giIqmpiy0iEqAutohIgDJIEZEAPUkjIhKgDFJEJEDXIEVEApRBiogEKIMUEQlQgBQRSU1P0oiIhCiDFBEJyLFBGk2YKyISoAxSROKjLraISECOdbEVIEUkPsogRUQClEGKiATkWAapUWwRiU8iUfWlEszsfDP7xMw+NrPxZtbAzLqb2VQzKzSziWZWP6q7Q7ReGG3vVu57Lo3KZ5vZUds7rgKkiMTHE1VftsPMOgLnAH3dfW8gHzgZuBG41d17AGuAUdEuo4A1UfmtUT3MbM9ov72AwcDdZpZf0bEVIEUkPmnKIEleDmxoZgVAI2ApcCTwaLR9HDAs+jw0WifaPsDMLCqf4O7fuvs8oBA4sKKDKkCKSHzSkEG6+2LgZuArkoFxHfA+sNbdS6Jqi4CO0eeOwMJo35Kofuvy5Sn2SUkBUkTiU40M0sxGm9mMcsvo8l9pZi1JZn/dgQ5AY5Jd5LTTKLaIxKcat/m4+xhgTAVVBgLz3H0lgJk9DvQHWphZQZQldgIWR/UXA52BRVGXvDmwulz5VuX3SUkZpIjEJz3XIL8C+plZo+ha4gDgU+A1YHhUZwTwVPR5crROtP1Vd/eo/ORolLs70BOYVtGBlUGKSHzScB+ku081s0eBD4ASYCbJjPNZYIKZXR+V3Rftch/wkJkVAkUkR65x90/MbBLJ4FoCnOnbmcDSkoE1OxWvmpu9jZOgem12AaCgXocMt0Sqo6R4CbhbdfbdNPGaKv/ONjzpqmodqyYogxSR+OTYkzQKkCISHwVIEZGAHJusQqPYIiIByiBFJD7qYouIBGTxXTHVoQApIvFRBikiEqAAKSISkGOj2AqQIhIbT+gapIhIaupii4gEqIstIhKgLraISIC62CIiAQqQEvLQpCd5bPILuDvDjx3MqSf9rGzbA+Mf4+a7xvLWsxNo2aI5r771Lnfe+yB5lkd+fj6XnDua3vvuzedffMl1N9/Fho3fkJefx+jTTmbIwMMyeFZ1T6dOHXjg/ttp225H3J2xYx/mzrvu4/8evoddd03OddmieTPWrltP3wMGUa9ePe65+0b69NmHRMK54IIreePNdzN8FhmiJ2kklTlz5/PY5BcYP/Y26hXU47cXXs5h/Q+iS6cOLF2+knemfUD7dm3L6vfrsx9HHNIPM2N24TwuuuJPPD3+Xho02IE/XXERXTt3ZMXK1Zw46mz6H9SHZk2bZPDs6paSkhIu/v01zJz1MU2aNGba1Bd4Zcqb/OKU35XV+euNV7Ju/XoAfj3qFwDs33sgbdq05pmn/0m/g48mmyejTpscyyDTNpuPme1uZn8wszui5Q9mtke6jpdpc+cvpNdeu9GwQQMKCvLpu18vXnnjXwDcdMffueCMUVi5eZMbNWqIRQWbNm9m68ZuXTrRtXPyTZRt27SmVcsWrFm7rmZPpo5btmwFM2d9DMCGDRv5/PM5dOyw03fqDB9+DBMmJl+Bssceu/La68mf9cqVq1m3dj19++xbs43OFgmv+pLF0hIgzewPwATASL4UZ1r0ebyZXZKOY2Zaj5278sG/P2HtuvVs2ryZt96dzrLlK3n1rXdp22ZHdu+58/f2eeWNf3HMz/+bMy66kuv+eP73tn/06WyKi0vo3LF9TZyCpNC1ayf223dvpk6bWVZ26CEHsXzFSgoL5wHw4YefcsxPB5Gfn0+3bp3p3bsXnTrX0ddNpOG92JmUri72KGAvdy8uX2hmfwM+Af6SpuNmzC7dunD6KScw+vzLaNigAbv13JktxcXc++BExtx6Q8p9Bh7Wn4GH9WfGrI+4694HGXv7n8u2rVxVxKXX/pUbLr+QvDxN25kJjRs3YtLEe7ngoqv4+usNZeUnnTSMiROfKlv/xwMT2GP3nkx973m++moR7747g9LSCt8FlbuyPCOsqnT95iVIvuB7W+2jbUHlXyI+9sHxaWlcuhx/zFFMuv9Oxt39V5o1bcou3buyeMkyjh9xBoOOH8Hylas44fSzWbW66Dv79d2vF4uWLCvrSm/YuJEzLr6Sc34zgn33ztmrElmtoKCARybey/jxT/Dkk8+Xlefn5/OzYUOY9MjksrLS0lIuvPhq+h4wiOOOP50WLZozZ87cTDRbYpauDPI8YIqZzQEWRmVdgB7AWRXtWP4l4rXtrYar16yldcsWLF22gilv/IuHx9zKqScOK9s+6PgRTLzvDlq2aM5Xi5bQuWN7zIxPZxeyZUsxLZo3o7i4mHMvvY5jBw9g0BGHZu5k6rh7x9zCZ58Xctvt332f/cABhzJ7diGLFy8tK2vYsAFmxjffbGLggEMpKSnhs8/m1HSTs4Ln2CBNWgKku79gZrsCBwIdo+LFwPTtvYe2Njv/j9ezdv16CgoKuOzCMyoceX759beZ/PwUCgoKaLBDfW6+9hLMjBdefYv3Z33M2nVf8+RzrwBww2UXsHt0e4mkX/8fHcCpvxzOhx99yozpLwFwxRV/4fkXXuXEE4eWDc5s1bbtjjz37P+RSCRYsngZI0aek4lmZ4cc62LrvdgSO70Xu3b7Ie/F3nj9L6v8O9v48n/qvdgiUgfkWAapACki8dE1SBGRAGWQIiIBWX7jd1UpQIpIfJRBioikpvsgRURClEGKiAQoQIqIBGiQRkQkQBmkiEhqrgApIhKgACkiEpBjt/loqmoRkQBlkCISH3WxRUQCFCBFRFLL5gm4q0MBUkTiowxSRCRAAVJEJDXdKC4iEqIAKSISkFv3iStAikh81MUWEQlRgBQRCVAXW0QktVzrYmuyChGJT6IaSyWYWQsze9TMPjezz8zsYDNrZWYvm9mc6M+WUV0zszvMrNDMPjSz3uW+Z0RUf46ZjdjecRUgRSQ2nvAqL5V0O/CCu+8O7At8BlwCTHH3nsCUaB1gCNAzWkYD9wCYWSvgKuAg4EDgqq1BNUQBUkTik4YM0syaA/8F3Afg7lvcfS0wFBgXVRsHDIs+DwUe9KT3gBZm1h44CnjZ3YvcfQ3wMjC4omMrQIpIbDxR9aUSugMrgX+Y2UwzG2tmjYF27r40qrMMaBd97ggsLLf/oqgsVB6kACkiGWVmo81sRrll9DZVCoDewD3uvj+wkf90pwHw5DRCsY8QaRRbROJTjdt83H0MMKaCKouARe4+NVp/lGSAXG5m7d19adSFXhFtXwx0Lrd/p6hsMXD4NuWvV9Q2ZZAiEpt0dLHdfRmw0Mx2i4oGAJ8Ck4GtI9EjgKeiz5OB06LR7H7Auqgr/iIwyMxaRoMzg6KyIGWQIhKf9N0ofjbwsJnVB+YCI0kmeJPMbBSwADgxqvsccDRQCHwT1cXdi8zsOmB6VO9ady+q6KAKkCISm0oOulT9e91nAX1TbBqQoq4DZwa+537g/soeVwFSRGKTrgCZKQqQIhIbBUgRkRC3TLcgVgqQIhIbZZAiIgGeUAYpIpKSMkgRkQDXNUgRkdSUQYqIBOgapIhIgOfWGxcUIEUkPsogRUQCci1AarozEZEAZZAiEhtdgxQRCci1LrYCpIjEJtduFN/uNUgza2dm95nZ89H6ntEMviIi35GmtxpmTGUGaR4g+d6GDtH6F8B5aWqPiNRiCbcqL9msMgFyR3efRPS2CXcvAUrT2ioRqZXcrcpLNqvMNciNZtaa6J2zW98SltZWiUitVBcHaS4g+RrFXczsX0AbYHhaWyUitVKdu83H3T8ws8OA3QADZrt7cdpbJiK1Tp3LIM3stG2KepsZ7v5gmtokIrVUtg+6VFVlutgHlPvcgOR7aD8AFCBF5DuyfdClqirTxT67/LqZtQAmpKtBIlJ71blrkClsBLrH3RARqf3qXBfbzJ4musWH5H2TewKT0tkoEamd6lwXG7i53OcSYIG7L0pTe0SkFqtTXWwzyweudvcjaqg931GvzS6ZOKzEpKR4SaabIDWsTnWx3b3UzBJm1tzd9fSMiFSoLnaxNwAfmdnLJAdoAHD3c9LWqsjMzsem+xCSBvsvnAzAponXZLglUh0NT7oq003IGpUJkI9HS3k5dqVBROJQp7rYkRbufnv5AjM7N03tEZFaLNcyp8pMdzYiRdmvYm6HiOSAXJsPMphBmtnPgV8A3c1scrlNTYGidDdMRGqfujRI8w6wFNgRuKVc+dfAh+lslIjUTln+BoUqCwZId18ALAAOrugLzOxdd6+wjojUDU7dySArq0EM3yEiOSCRY6M0cQTIHPsrEZHqSiiDFBFJLde62JV5L/bZZtayoioxtkdEarFENZZsVpn7INsB081skpkNNrNtA+KpaWiXiNRCjlV5yWbbDZDufjnQE7iP5A3ic8zsT2a2S7T947S2UERqjbqYQeLuDiyLlhKgJfComd2UxraJSC2TawGyMjOKnwucBqwCxgIXu3uxmeUBc4Dfp7eJIlJbZHuXuaoqM4rdCjguunG8jLsnzOyn6WmWiNRGOfZa7Eq91TA4OZy7fxZvc0SkNsu1+yArdQ1SRKQu0o3iIhKbXHusTgFSRGKT7aPSVaUAKSKxSXzvOZLaTdcgRSQ2Xo2lssws38xmmtkz0Xp3M5tqZoVmNtHM6kflO0TrhdH2buW+49KofLaZHbW9YypAikhs0nyj+LlA+TtnbgRudfcewBpgVFQ+ClgTld8a1cPM9gROBvYCBgN3m1l+RQdUgBSR2CSs6ktlmFkn4CckH1YhmhPiSODRqMo4YFj0eWi0TrR9QFR/KDDB3b9193lAIXBgRcdVgBSR2CSwKi9mNtrMZpRbRqf46ttIPrW3NelsDax195JofRHQMfrcEVgIEG1fF9UvK0+xT0oapBGR2FTnNh93HwOMCW2Pnthb4e7vm9nh1WxatShAikhs0vSoYX/gWDM7muQrXpoBtwMtzKwgyhI7AYuj+ouBzsAiMysAmgOry5VvVX6flNTFFpHYpGOQxt0vdfdO7t6N5CDLq+5+CvAaMDyqNgJ4Kvo8OVon2v5qNCPZZODkaJS7O8lpHKdVdGxlkCISmxp+kuYPwAQzux6YSXLOWqI/HzKzQqCIZFDF3T8xs0nApySnbTzT3UsrOoACpIjEJt2z+bj768Dr0ee5pBiFdvfNwAmB/W8Abqjs8RQgRSQ2etRQRCRAAVJEJMBz61FsBUgRiU+uZZC6zUdEJEAZpIjEJtcySAVIEYmNZhQXEQmoc281FBGpLHWxRUQCFCBFRAJ0DVJEJEDXIEVEAtTFFhEJUBdbRCQgkWMhUgFSRGKjLraISEBu5Y8KkCISI2WQIiIBus1HRCRAgzQiIgG5FR41Ya6ISJAySBGJjQZpREQCdA1SRCQgt8KjAqSIxEhdbBGRAHWxRUQCcis8KkCKSIzUxRYRCfAcyyEVIEUkNsogRUQCNEgjFcvLY7dnbqF4+WrmjryeJj/qRcfLRmL1C/jmoy/56uI7ofQ//59ttE8Pdn3yJuafdTNrn3sHgP3mPc6mzxcAULxkFXNH3ZCRU6lL5q9cz+8nvV22vnjN1/zuyH3p270tN0yexrclCQryjEuPOYBenXbk681buOzRd1i2diMlCee0Q/ZgWO9dALj1xQ94a/YS3J1+Pdrz+6P7YJZj09wE5FZ4VICMXZvTf8rmwoXkN20EZnT923kU/vwKvp23hJ0u+AWthh9J0cRXkpXz8uhw6QjWvznzO9+R2LyF2UPOz0Dr665ubZox6cyjAShNJBj01yc4cs9OXPvkVH5zRC8O2bUjb32xmNtenMl9o37MxKlfsHOb5tzxy8Mp2riZYbc/zU/26cYnS4qY9dVKHjkr+V0jx77MjPkrOKB7u0yeXo3JtQxSk1XEqN5OrWk+oC+rJ7wMQEHLpnhxMd/OWwLA12/PosWQg8vqtxn5E9Y+/y4lq9dlpL2S2tS5y+nUqgkdWjTBzNj4bTEAGzYX06ZpQwAM2LilGHdn05YSmjesT35eHgZsKUlQXJpgS0mCktIErRs3yNzJ1LBENZZslpEM0sxGuvs/MnHsdOp49a9Z/Kdx5DdO/hKVFK3H8vNpuE8PNn1YSIujf0T9DjsCUK9dK5of1Y/Cky6ny75nf+d78naoz27P3IKXlLL87sdY99LUGj+XuuzFj+YzpFc3AC4e0oczHnyVv70wk4Q740YPAuDkfrtx7sNv8OObHmfjlhJuPPEQ8vKMfbu04YDu7Rh40+PgcNJBu7Jz2+YZPJuapVHseFwD5FSAbDagLyWr1rLpoy9p0m/vsvJ5Z91MpytPx+rX4+s3Z+HR9ceOV/+aJX8eB/79f1CfHPxripcXUb9LO3qMv45NsxewZcGyGjuXuqy4pJQ3Pl/MOT/eD4BHps/hoiF9GLhXF178aAHXPDGVv48cwDtzlrLbTi25d+QAFhZt4LcPTKF317YUbdzM3JXreOminwHw23Gv8sH8FfTu1jaDZ1Vzsj0jrKq0BUgz+zC0CQhekDGz0cBogMta7sPxTbrF37g0aNx3D5r/+ECaHdGHvB3qk9+0EV1vO58F593KnOF/BKDpofuxw84dAGjUqwfd7roIgIJWzWh2RB+8pJR1L02leHkRAFu+Ws6G9z6m0V47K0DWkLfnLGH39i1p3STZC3h65lx+f3QfAAbt3YVrn3oPgKdmfsnph+6FmdGldVM6tmzCvFXreH/+CvbptCONdqgHQP+eHfj3wpV1JkAqg6y8dsBRwJptyg14J7STu48BxgDM7DK01vxtL73xIZbe+BAATfrtTdvfDGPBebdS0Lo5JavXYfULaHfGcSy78xEAPj1kdNm+XW45h/VTZrDupankN29MYtO3+JYS8ls2pXHfPVjxv09k5Jzqohc+XMDgfbqVrbdp2rBskGXa3OV0ad0MgPbNGzN17jJ6d2vL6g2bmL9qPZ1aNmHJmg08PuNLSkqTwxXvz1/OKQfvnpmTkR8snQHyGaCJu8/adoOZvZ7G42aVtr/5Gc0H9IW8PFb983k2vPNRhfUb9OhM5z//DhIOecbyux9j85yFNdTaum3TlhLe+3Iplw89sKzsymEHcdNz71NamqB+vXyuODa57b8P35srH3+X4Xc+gwPnDdqflo0bMHCvLkybu5wT7noWM/hRzw4ctnunDJ1Rzcu1LrZ5imtg2aI2ZZDyH/svnAzAponXZLglUh0NT7oK3Kt14+apXY+r8u/sQwsez9qbRHUfpIjEJtcyGgVIEYlNrt0orgApIrHRKLaISECuDdIoQIpIbNTFFhEJUBdbRCRAXWwRkYBsvq+6OhQgRSQ2uXYNUvNBikhs0jEfpJl1NrPXzOxTM/vEzM6NyluZ2ctmNif6s2VUbmZ2h5kVmtmHZta73HeNiOrPMbMR2zu2AqSIxMar8V8llAAXuvueQD/gTDPbE7gEmOLuPYEp0TrAEKBntIwG7oFkQAWuAg4CDgSu2hpUQxQgRSQ2CbzKy/a4+1J3/yD6/DXwGdARGAqMi6qNA4ZFn4cCD3rSe0ALM2tPcnaxl929yN3XAC8Dgys6tq5Bikhs0j1IY2bdgP2BqUA7d18abVrGf+aZ7QiUnwJrUVQWKg9SBikisanONUgzG21mM8oto1N9t5k1AR4DznP39eW3eTIyxx6dlUGKSGyqc6N4+UmyQ8ysHsng+LC7Px4VLzez9u6+NOpCr4jKFwOdy+3eKSpbDBy+TfnrFR1XGaSIZDVLvlT8PuAzd/9buU2Tga0j0SOAp8qVnxaNZvcD1kVd8ReBQWbWMhqcGRSVBSmDFJHYpOk+yP7AqcBHZjYrKvsj8BdgkpmNAhYAJ0bbngOOBgqBb4CRAO5eZGbXAdOjete6e1FFB1aAFJHYpGOQxt3fJvkuq1QGpKjvwJmB77ofuL+yx1aAFJHY5NqTNAqQIhIbzeYjIhKQ0GQVIiKp5VZ4VIAUkRjpGqSISIACpIhIgCbMFREJUAYpIhKg23xERALUxRYRCVAXW0QkQBmkiEiAMkgRkYBcG6TRhLkiIgHKIEUkNpqsQkQkINe62AqQIhIbZZAiIgHKIEVEApRBiogEKIMUEQlQBikiEqAMUkQkwD2R6SbESgFSRGKjZ7FFRAI0m4+ISIAySBGRAGWQIiIBus1HRCRAt/mIiATkWhdbE+aKiAQogxSR2GgUW0QkINe62AqQIhIbjWKLiAQogxQRCdA1SBGRAGWQIiIBugYpIhKgJ2lERAKUQYqIBOgapIhIgLrYIiIByiBFRAJyLUBaVp+QWRY3TiSHuVt1diuo37HKv7MlWxZX61g1IbsDZA4zs9HuPibT7ZDq0c+vbtB8kJkzOtMNkB9EP786QAFSRCRAAVJEJEABMnN0/ap208+vDtAgjYhIgDJIEZEABcgMMLPBZjbbzArN7JJMt0cqz8zuN7MVZvZxptsi6acAWcPMLB/4H2AIsCfwczPbM7Otkip4ABic6UZIzVCArHkHAoXuPtfdtwATgKEZbpNUkru/CRRluh1SMxQga15HYGG59UVRmYhkGQVIEZEABciatxjoXG69U1QmIllGAbLmTQd6mll3M6sPnAxMznCbRCQFBcga5u4lwFnAi8BnwCR3/ySzrZLKMrPxwLvAbma2yMxGZbpNkj56kkZEJEAZpIhIgAKkiEiAAqSISIACpIhIgAKkiEiAAqSISIACpGQVM/uVmd2V6XaIgAKk1JBomjeRWkUBUlIys2vN7Lxy6zeY2bkp6h1uZm+a2bPRJMD/a2Z50bYNZnaLmf0bONjMfmlm08xslpn9fWvQNLORZvaFmU0D+tfQKYpslwKkhNwPnAYQBbyTgX8G6h4InE1yAuBdgOOi8sbAVHffF1gNnAT0d/f9gFLgFDNrD1xDMjAeEn2HSFYoyHQDJDu5+3wzW21m+wPtgJnuvjpQfZq7z4WyZ5UPAR4lGQQfi+oMAPoA080MoCGwAjgIeN3dV0b7TwR2Tc9ZiVSNAqRUZCzwK2AnkhllyLYP9G9d3+zupdFnA8a5+6XlK5rZsB/eTJH0UBdbKvIEyfevHEBy9qGQA6Pp2/JIdqPfTlFnCjDczNoCmFkrM+sKTAUOM7PWZlYPOCHWMxD5AZRBSpC7bzGz14C15TLBVKYDdwE9gNdIBtZtv+tTM7sceCkKpMXAme7+npldTXIKsbXArFhPQuQH0HRnEhQFsg+AE9x9TqDO4cBF7v7TGmyaSI1QF1tSil5FWwhMCQVHkVynDFIqxcx6AQ9tU/ytux+UifaI1AQFSBGRAHWxRUQCFCBFRAIUIEVEAhQgRUQCFCBFRAL+H61tT2ZkMGj6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 360x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 7.67 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# Random forest algorithm\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "classifier = RandomForestClassifier()\n",
    "classifier.fit(X_train, y_train)\n",
    "y_pred = classifier.predict(X_test)\n",
    "print(classification_report(y_test,y_pred))\n",
    "print(\"Accuracy: \"+ str(accuracy_score(y_test, y_pred)))\n",
    "\n",
    "# Plot the confusion matrix\n",
    "cm=confusion_matrix(y_test,y_pred)\n",
    "f,ax=plt.subplots(figsize=(5,5))\n",
    "sns.heatmap(cm,annot=True,linewidth=0.5,linecolor=\"red\",fmt=\".0f\",ax=ax)\n",
    "plt.xlabel(\"y_pred\")\n",
    "plt.ylabel(\"y_true\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.68      0.97      0.80      9711\n",
      "           1       0.97      0.66      0.79     12833\n",
      "\n",
      "    accuracy                           0.79     22544\n",
      "   macro avg       0.83      0.82      0.79     22544\n",
      "weighted avg       0.84      0.79      0.79     22544\n",
      "\n",
      "Accuracy: 0.7940028388928317\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUgAAAE+CAYAAADvb4nvAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAbMUlEQVR4nO3deZwU1bn/8c8DMwiyo4AsKoi4oHGLogZv1IsBUSPG3Yii4Xe5URRFY1wT45aYeN0TF1wiImGRKBCjBkSMGmULIAqKjCCBYd9E9hn6+f3RBTbYZ5gZq6d7er5vXvWi69SpqtMM87yeU6fqlLk7IiLybbWy3QARkVylACkiEqAAKSISoAApIhKgACkiEqAAKSISUJDtBpTJTPcgiWSDu1Vmt5KV8yr8O1u49wGVOldVyOkAWbLii2w3QSqhsHkHAAoKW2e5JVIZpSWLs92EnJHTAVJEqpnEtmy3IFYKkCISH09kuwWxUoAUkfgkFCBFRNJyZZAiIgHKIEVEApRBiogEaBRbRCRAGaSISICuQYqIpKdRbBGREGWQIiIByiBFRAI0ii0iEpBnGaQmzBURCVAGKSLx0SCNiEhAnnWxFSBFJD7KIEVE0nPXKLaISHrqYouIBKiLLSISoAxSRCRAT9KIiAQogxQRCdA1SBGRAGWQIiIByiBFRAIUIEVE0tOTNCIiIcogRUQC8myQRhPmiogEKIMUkfioiy0iEpBnXWwFSBGJjzJIEZGAPMsgNUgjIvFJJCq+lIOZDTCzWWb2iZkNNbO6ZtbezCaZWZGZDTezOlHdPaL1omh7u5Tj3BqVzzGz7rs7rwKkiMQnAwHSzNoA/YFj3f1woDZwMfB74GF3PxBYA/SJdukDrInKH47qYWadov0OA04HnjCz2mWdWwFSROLjiYov5VMA1DOzAmBPYAnw38DIaPsg4Jzoc89onWh7VzOzqHyYu29x9/lAEdC5rJMqQIpIfDKQQbp7MfB/wH9IBsavgH8Da929NKq2CGgTfW4DLIz2LY3q75VanmaftBQgRSQ+lcggzayvmU1NWfqmHtLMmpLM/toDrYH6JLvIGadRbBGJTyVu83H3gcDAMqqcBsx39xUAZvYK0AVoYmYFUZbYFiiO6hcD+wKLoi55Y2BVSvl2qfukpQxSROKTmWuQ/wFOMLM9o2uJXYHZwATg/KhOb2B09HlMtE60/W1396j84miUuz3QEZhc1omVQYpIfDJwo7i7TzKzkcA0oBSYTjLj/DswzMzujcqei3Z5DhhsZkXAapIj17j7LDMbQTK4lgL9fDfzs1kysOamkpXzcrdxElTYvAMABYWts9wSqYzSksXgbpXZd9OIuyv8O1vvwl9X6lxVQRmkiMQnhxOuylCAFJH46FlsEZEABUgRkQBNViEiUjMogxSR+KiLLSISoFFsEZEAZZAiIgEKkCIiAXk2iq0AKSKx8YSuQYqIpKcutohIgLrYIiIB6mKLiASoiy0iEpBnAVLPYsdo8IhRnNPr5/S89H8ZPPzVnba9MPSvHN6lB2vWfgXAvAULubTvAI4+5cf8+S8jy30cyby2bVvz1tiXmfnRBD6a8TbXXpN83fIRR3Ti/XfHMH3aW4x69QUaNmwAwGld/4tJE99g+rS3mDTxDU49pUs2m59d7hVfcpgyyJjMnfclfx3zJkOffYTCgkJ+fuMdnNzlePZr25oly1bwweRptGrZYkf9xo0acsuAn/P2ux+W+zhSNUpLS7npl3cxfcYnNGhQn8mT3uSt8e/y9FMPcPPN9/DuexO5ovdF/OLGq7jzNw+wctVqzvnJFSxZsozDDjuY118bwv7tj83218gOZZDlY2aHmNnNZvZYtNxsZodm6nzZNu/LhXzvsIOpV7cuBQW1Ofao7/HWP/8FwB8ee5obru6DpUwsv1fTJnzv0IMpKCgo93GkaixdupzpMz4BYP36DXz22VzatN6HgzoewLvvTQTgrfHv8ZOfnAHAjBmzWLJkGQCzZs2hXr261KlTJzuNz7aEV3zJYRkJkGZ2MzAMMJJvDZscfR5qZrdk4pzZduAB+zPto1ms/WodmzZv5r0Pp7B02Qrefu9DWjTfm0M6HvCdjiPZsf/+bTnqyMOZNHk6s2d/ztlndwfg/PPOYt80Wf25557J9OmfsHXr1qpuam7IzFsNsyZTXew+wGHuXpJaaGYPAbOA+zN03qzp0G4/fnbpBfQdcDv16tbl4I4HsLWkhGdeHM7Ah+/7TsepVUuXirOhfv09GTH8GW74xZ18/fV6/l/fG3jkoXu4/bbree21sWzdutN/bzp1Oojf3XcbPc78aZZanANyPCOsqEz95iWAdBfNWkXbgsysr5lNNbOpz744NCONy5TzftydEc8/zqAnHqBRw4Z0aL8/xYuXcl7vq+l2Xm+WrVjJBT+7lpWrVlfoOO32a1tF30C2Kygo4OXhzzB06KuMGvUGAHPmfEGPM3/K8Sf0YNjw0cyb9+WO+m3atGLky89x5c+uY968BVlqtcQtUxnk9cB4M5sLLIzK9gMOBK4pa0d3H0jynbfV7rWvq9asZa+mTViydDnj//kvhgx8mMsuPGfH9m7n9Wb4c4/RtEnjCh9HqtYzAx/k08+KeOTRgTvKmjffixUrVmFm3HbrdTw9cDAAjRs3YszoF7nt9t/ywYdTs9XknOB5NkiTkQDp7m+a2UFAZ6BNVFwMTNndi7qrswG33cvadesoKCjg9huvplF0G0g6K1et5qI+/Vm/YSO1atXipRGjGD3kaRrUr1+h40j8uvzgOC7rdT4zP57N1CljAfjVr+7nwAPbc9VVVwAwatTrvDBoOAD9rr6SAzu0447bB3DH7QMA6HHGJaxYsSor7c+qPOtim+fwfUjVLYOUpMLmHQAoKNStSdVRaclicLfd1/y2Dff2qvDvbP07XqrUuaqC7oMUkfjkWQapACki8dE1SBGRAGWQIiIBOX7jd0UpQIpIfJRBioikp/sgRURClEGKiAQoQIqIBGiQRkQkQBmkiEh6rgApIhKgACkiEpBnt/loqmoRkQBlkCISH3WxRUQCFCBFRNLL5Qm4K0MBUkTiowxSRCRAAVJEJD3dKC4iEqIAKSISkF/3iStAikh81MUWEQlRgBQRCVAXW0QkvXzrYmuyChGJT6ISSzmYWRMzG2lmn5nZp2Z2opk1M7NxZjY3+rtpVNfM7DEzKzKzmWZ2TMpxekf155pZ792dVwFSRGLjCa/wUk6PAm+6+yHAkcCnwC3AeHfvCIyP1gF6AB2jpS/wJICZNQPuBI4HOgN3bg+qIQqQIhKfDGSQZtYY+CHwHIC7b3X3tUBPYFBUbRBwTvS5J/CiJ00EmphZK6A7MM7dV7v7GmAccHpZ51aAFJHYeKLiSzm0B1YAfzaz6Wb2rJnVB1q6+5KozlKgZfS5DbAwZf9FUVmoPEgBUkSyysz6mtnUlKXvLlUKgGOAJ939aGAD33SnAfDkNEKxjxBpFFtE4lOJ23zcfSAwsIwqi4BF7j4pWh9JMkAuM7NW7r4k6kIvj7YXA/um7N82KisGTtml/J2y2qYMUkRik4kutrsvBRaa2cFRUVdgNjAG2D4S3RsYHX0eA1wejWafAHwVdcX/AXQzs6bR4Ey3qCxIGaSIxCdzN4pfCwwxszrAPOBKkgneCDPrAywALozqvg6cARQBG6O6uPtqM7sHmBLVu9vdV5d1UgVIEYlNOQddKn5c9xnAsWk2dU1T14F+geM8Dzxf3vMqQIpIbDIVILNFAVJEYqMAKSIS4pbtFsRKAVJEYqMMUkQkwBPKIEVE0lIGKSIS4LoGKSKSnjJIEZEAXYMUEQnw/HrjggKkiMRHGaSISEC+BUhNdyYiEqAMUkRio2uQIiIB+dbFVoAUkdjk243iu70GaWYtzew5M3sjWu8UzeArIrKTDL3VMGvKM0jzAsn3NrSO1j8Hrs9Qe0SkGku4VXjJZeUJkHu7+wiit024eymwLaOtEpFqyd0qvOSy8lyD3GBmexG9c3b7W8Iy2ioRqZZq4iDNDSRfo9jBzP4FNAfOz2irRKRaqnG3+bj7NDM7GTgYMGCOu5dkvGUiUu3UuAzSzC7fpegYM8PdX8xQm0Skmsr1QZeKKk8X+7iUz3VJvod2GqAAKSI7yfVBl4oqTxf72tR1M2sCDMtUg0Sk+qpx1yDT2AC0j7shIlL91bgutpn9jegWH5L3TXYCRmSyUSJSPdW4LjbwfymfS4EF7r4oQ+0RkWqsRnWxzaw28Bt3P7WK2rOTwuYdsnFaiUlpyeJsN0GqWI3qYrv7NjNLmFljd9fTMyJSpprYxV4PfGxm40gO0ADg7v0z1qrI31tcnOlTSAacuTx5k8Pm91/KckukMuqe1CvbTcgZ5QmQr0RLqjy70iAicahRXexIE3d/NLXAzK7LUHtEpBrLt8ypPNOd9U5TdkXM7RCRPJBv80EGM0gzuwT4KdDezMakbGoIrM50w0Sk+qlJgzQfAEuAvYEHU8q/BmZmslEiUj3l+BsUKiwYIN19AbAAOLGsA5jZh+5eZh0RqRmcmpNBllfdGI4hInkgkWejNHEEyDz7JxGRykoogxQRSS/futjleS/2tWbWtKwqMbZHRKqxRCWWXFae+yBbAlPMbISZnW5muwbEyzLQLhGphhyr8JLLdhsg3f0OoCPwHMkbxOea2W/NrEO0/ZOMtlBEqo2amEHi7g4sjZZSoCkw0sz+kMG2iUg1k28Bsjwzil8HXA6sBJ4FbnL3EjOrBcwFfpnZJopIdZHrXeaKKs8odjPg3OjG8R3cPWFmZ2WmWSJSHeXZa7HL9VbDO8vY9mm8zRGR6izf7oMs1zVIEZGaSDeKi0hs8u2xOgVIEYlNro9KV5QCpIjEJvGt50iqN12DFJHYeCWW8jKz2mY23cxei9bbm9kkMysys+FmVicq3yNaL4q2t0s5xq1R+Rwz6767cypAikhsMnyj+HVA6p0zvwcedvcDgTVAn6i8D7AmKn84qoeZdQIuBg4DTgeeMLPaZZ1QAVJEYpOwii/lYWZtgTNJPqxCNCfEfwMjoyqDgHOizz2jdaLtXaP6PYFh7r7F3ecDRUDnss6rACkisUlgFV7MrK+ZTU1Z+qY59CMkn9rbnnTuBax199JofRHQJvrcBlgIEG3/Kqq/ozzNPmlpkEZEYlOZ23zcfSAwMLQ9emJvubv/28xOqWTTKkUBUkRik6FHDbsAZ5vZGSRf8dIIeBRoYmYFUZbYFiiO6hcD+wKLzKwAaAysSinfLnWftNTFFpHYZGKQxt1vdfe27t6O5CDL2+5+KTABOD+q1hsYHX0eE60TbX87mpFsDHBxNMrdnuQ0jpPLOrcySBGJTRU/SXMzMMzM7gWmk5yzlujvwWZWBKwmGVRx91lmNgKYTXLaxn7uvq2sEyhAikhsMj2bj7u/A7wTfZ5HmlFod98MXBDY/z7gvvKeTwFSRGKjRw1FRAIUIEVEAjy/HsVWgBSR+ORbBqnbfEREApRBikhs8i2DVIAUkdhoRnERkYAa91ZDEZHyUhdbRCRAAVJEJEDXIEVEAnQNUkQkQF1sEZEAdbFFRAISeRYiFSBFJDbqYouIBORX/qgAKSIxUgYpIhKg23xERAI0SCMiEpBf4VET5oqIBCmDFJHYaJBGRCRA1yBFRALyKzwqQIpIjNTFFhEJUBdbRCQgv8KjAqSIxEhdbBGRAM+zHFIBUkRiowxSRCRAgzRStlrGSWN/y+alq5na6wGOeLgvjY88AMzY8MUSPur/JNs2bqHtRT/kkF9fyualqwFY8PxYFg6ZQKPD9ufwP/yMggZ74okERY+8ypLRE7P8pWqGwWMn8sq70zGDjm1acHefnuxRmPwVuX/Im4x6fzoTn7wVgCWrvuKO50bx9cYtJBIJrju/K/91REfWrt/IjU+8zKz5izm7y1Hc1qtHNr9Slcuv8KgAGbv2/9OD9XOLKWhYD4DZvxpM6fpNABx6Vy/a9enOF4+PAWDJ6A+ZddsLO+2/bdMWZlzzJBvnL2WPlk05adx9rJgwk9J1G6v0e9Q0y9as4y9vTebVe6+ibp1CbnpiJG9O+oSeJx3FrPmLWbdx0071n/nbe3Q/7jAuPPVYvihewTWP/IU3HriOOoUF9DvnVIqKl1NUvCJL3yZ78i2D1GQVMarbqhktfnQ0C4dM2FG2PTgC1K5bB/ey/wNtmLeUjfOXArBl2Rq2rlxHnb0aZabBspNt2xJs2VpK6bYEm7aW0LxJQ7YlEjz08jgGXHDazpUN1m/aAsD6TZtp3qQhAHvuUYdjDtpvR+ZZ0yQqseSyrPwUzexKd/9zNs6dSZ3uuZxP7/4LBQ3q7lR+xCP/S4vTjmb954uY/ZuXdpTvc1Znmp14KBu+WMLsX7/I5sWrd9qv8dEdqFVYwMYvl1VJ+2uylk0b0fv0E+l+0yPULSzkxMMP4AeHd2DIuEmcctTBOwLgdlf1PJmfPziEoeMns2lLCQN/0StLLc8t+TaKna0M8q4snTdjWvzoaLauXMe6mfO/tW3m9U/z1hFXsf7zxbTueSIAy8ZOY8Kx/Xnv1JtZ+c+POfLxq3faZ48WTTjqj1fz0fVPwW6yTvnu1m3YxITpc3j99/0Z99AANm0p4W//+oixU2ZzSdfO36r/xqRPOLvLkYx7cAB/uv4Sbn9mFImEfk75lkFmLECa2czA8jHQsoz9+prZVDOb+uamokw1L3ZNOx9Mi+7HcOqUxzj66f7s3eUwjvpTv28qJJzFoz5gn7OSv2wla9aT2FoKwH+GvE3jI9rvqFrQoB7HDfklc343nLX/rj7/BtXZxNnzabN3E5o1qk9hQW26HnMIT4z+JwuXr+bHtzxOj5seZfPWEs665XEAXn1vBt07dwLgyAP3ZUtJKWvW6zqxV+JPLstkF7sl0B1Ys0u5AR+EdnL3gcBAgL+3vCS3//VSzLlvGHPuGwZAsx8cygFXn8WMfn9iz3Ytd3SRW3b/PhvmLgaSGeKW5Wt3lK+fWwyAFdbm+y/cwKKX32Ppa5Or/ovUUPs0a8TMecVs2lJC3ToFTPp0Ppd1O4GfnvZN9njCVb/jtfuvBaBVs0ZMmj2fnicdxbzFK9haUkqzhntmq/mSIZkMkK8BDdx9xq4bzOydDJ43d5hx5ONXUdCwHmbGulkL+OSXzwPQ7n9Op2W37+PbtlGydj0f9X8KgNZnn0izEw6hsGkD2l70QwBm9n+KdbMWZO1r1ARHdGjLj449lIvvGkjt2rU4ZL99OP/kY4L1b7yoG3cP+hsvjZ2EGdzdpydmyTdW9bjpUdZv3kJJ6TYmTP+Mp27oRYc2zavqq2RVrneZK8p2N6qaTdUpg5RvnLk8mUlvfv+l3dSUXFT3pF7gXqn3E162/7kV/p0dvOCVnH0XYs28F0FEMiLfMhoFSBGJTb7dKK4AKSKxyfVR6YpSgBSR2OTbII0CpIjERl1sEZEAdbFFRALUxRYRCcjl+6orQwFSRGKTb9cgNR+kiMQmE7P5mNm+ZjbBzGab2Swzuy4qb2Zm48xsbvR306jczOwxMyuKJsg5JuVYvaP6c82s9+7OrQApIrHJ0Gw+pcCN7t4JOAHoZ2adgFuA8e7eERgfrQP0ADpGS1/gSUgGVOBO4HigM3Dn9qAaogApIrFJ4BVedsfdl7j7tOjz18CnQBugJzAoqjYIOCf63BN40ZMmAk3MrBXJ2cXGuftqd18DjANOL+vcugYpIrHJ9CCNmbUDjgYmAS3dfUm0aSnfzDPbBliYstuiqCxUHqQMUkRiU5lrkKmTZEdL33THNrMGwF+B6919Xeo2T0bm2KOzMkgRiU1lbhRPnSQ7xMwKSQbHIe7+SlS8zMxaufuSqAu9PCovBvZN2b1tVFYMnLJL+TtlnVcZpIjkNEvORPwc8Km7P5SyaQywfSS6NzA6pfzyaDT7BOCrqCv+D6CbmTWNBme6RWVByiBFJDYZug+yC3AZ8LGZzYjKbgPuB0aYWR9gAXBhtO114AygCNgIXAng7qvN7B5gSlTvbnff+VWiu1CAFJHYZGKQxt3fJ/kuq3S6pqnvQL80dXH354Hny3tuBUgRiU2+PUmjACkisdFsPiIiAQlNViEikl5+hUcFSBGJka5BiogEKECKiARowlwRkQBlkCIiAbrNR0QkQF1sEZEAdbFFRAKUQYqIBCiDFBEJyLdBGk2YKyISoAxSRGKjySpERALyrYutACkisVEGKSISoAxSRCRAGaSISIAySBGRAGWQIiIByiBFRALcE9luQqwUIEUkNnoWW0QkQLP5iIgEKIMUEQlQBikiEqDbfEREAnSbj4hIQL51sTVhrohIgDJIEYmNRrFFRALyrYutACkisdEotohIgDJIEZEAXYMUEQlQBikiEqBrkCIiAXqSRkQkQBmkiEiArkGKiASoiy0iEqAMUkQkIN8CpOX0FzLL4caJ5DF3q8xuBXXaVPh3tnRrcaXOVRVyO0DmMTPr6+4Ds90OqRz9/GoGzQeZPX2z3QD5TvTzqwEUIEVEAhQgRUQCFCCzR9evqjf9/GoADdKIiAQogxQRCVCAzAIzO93M5phZkZndku32SPmZ2fNmttzMPsl2WyTzFCCrmJnVBv4E9AA6AZeYWafstkoq4AXg9Gw3QqqGAmTV6wwUufs8d98KDAN6ZrlNUk7u/i6wOtvtkKqhAFn12gALU9YXRWUikmMUIEVEAhQgq14xsG/KetuoTERyjAJk1ZsCdDSz9mZWB7gYGJPlNolIGgqQVczdS4FrgH8AnwIj3H1Wdlsl5WVmQ4EPgYPNbJGZ9cl2myRz9CSNiEiAMkgRkQAFSBGRAAVIEZEABUgRkQAFSBGRAAVIEZEABUjJKWZ2hZn9MdvtEAEFSKki0TRvItWKAqSkZWZ3m9n1Kev3mdl1aeqdYmbvmtnfo0mAnzKzWtG29Wb2oJl9BJxoZr3MbLKZzTCzp7cHTTO70sw+N7PJQJcq+ooiu6UAKSHPA5cDRAHvYuClQN3OwLUkJwDuAJwbldcHJrn7kcAq4CKgi7sfBWwDLjWzVsBdJAPjSdExRHJCQbYbILnJ3b80s1VmdjTQEpju7qsC1Se7+zzY8azyScBIkkHwr1GdrsD3gSlmBlAPWA4cD7zj7iui/YcDB2XmW4lUjAKklOVZ4ApgH5IZZciuD/RvX9/s7tuizwYMcvdbUyua2TnfvZkimaEutpTlVZLvXzmO5OxDIZ2j6dtqkexGv5+mznjgfDNrAWBmzcxsf2AScLKZ7WVmhcAFsX4Dke9AGaQEuftWM5sArE3JBNOZAvwROBCYQDKw7nqs2WZ2BzA2CqQlQD93n2hmvyE5hdhaYEasX0LkO9B0ZxIUBbJpwAXuPjdQ5xTgF+5+VhU2TaRKqIstaUWvoi0CxoeCo0i+UwYp5WJm3wMG71K8xd2Pz0Z7RKqCAqSISIC62CIiAQqQIiIBCpAiIgEKkCIiAQqQIiIB/x/mDWAAdSldWgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 360x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 4.13 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# XGBoost algorithm\n",
    "from xgboost import XGBClassifier\n",
    "classifier = XGBClassifier()\n",
    "classifier.fit(X_train, y_train)\n",
    "y_pred = classifier.predict(X_test)\n",
    "print(classification_report(y_test,y_pred))\n",
    "print(\"Accuracy: \"+ str(accuracy_score(y_test, y_pred)))\n",
    "\n",
    "# Plot the confusion matrix\n",
    "cm=confusion_matrix(y_test,y_pred)\n",
    "f,ax=plt.subplots(figsize=(5,5))\n",
    "sns.heatmap(cm,annot=True,linewidth=0.5,linecolor=\"red\",fmt=\".0f\",ax=ax)\n",
    "plt.xlabel(\"y_pred\")\n",
    "plt.ylabel(\"y_true\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.68      0.97      0.80      9711\n",
      "           1       0.97      0.66      0.78     12833\n",
      "\n",
      "    accuracy                           0.79     22544\n",
      "   macro avg       0.82      0.81      0.79     22544\n",
      "weighted avg       0.84      0.79      0.79     22544\n",
      "\n",
      "Accuracy: 0.7935149041873669\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUgAAAE+CAYAAADvb4nvAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAbTElEQVR4nO3deZgU1bnH8e87CwwqsiggghEEFHFBkaBGYkxQwKgXr1s0LmiIk6vGheQmoslVY9TEmKjRq0auEBVRRDSK0aCI4i5LFBdABVGEYZdNWWem3/tHF9iQPsPMpHqZnt/Hpx66Tp2qOg32+7ynTtUpc3dERORfFeW6ASIi+UoBUkQkQAFSRCRAAVJEJEABUkQkQAFSRCSgJNcNqJGZ7kESyQV3q89ulSvm1fk3W7r7PvU6VzbkdYCsXP5Jrpsg9VDapgsAJaV75rglUh9VlYty3YS8kdcBUkQamER1rlsQKwVIEYmPJ3LdglgpQIpIfBIKkCIiabkySBGRAGWQIiIByiBFRAI0ii0iEqAMUkQkQNcgRUTS0yi2iEiIMkgRkQBlkCIiARrFFhEJKLAMUhPmiogEKIMUkfhokEZEJKDAutgKkCISH2WQIiLpuWsUW0QkPXWxRUQC1MUWEQlQBikiEqAnaUREApRBiogE6BqkiEiAMkgRkQBlkCIiAQqQIiLp6UkaEZEQZZAiIgEFNkijCXNFRAKUQYpIfNTFFhEJKLAutgKkiMRHGaSISECBZZAapBGR+CQSdV9qwcyGmtlMM/vAzB4xszIz62xmU8xsrpk9amZNorpNo/W50fZOKce5Kir/yMwG7Oi8CpAiEp8MBEgz6wBcBvR29wOBYuBM4GbgNnfvCqwChkS7DAFWReW3RfUwsx7RfgcAA4G7zay4pnMrQIpIfDxR96V2SoBmZlYC7AQsBr4HjIu2PwCcHH0eFK0Tbe9nZhaVj3H3Te7+KTAX6FPTSRUgRSQ+Gcgg3b0C+CPwOcnAuAb4J7Da3auiaguBDtHnDsCCaN+qqP5uqeVp9klLAVJE4lOPDNLMys1sespSnnpIM2tFMvvrDOwJ7Eyyi5xxGsUWkfjU4zYfdx8ODK+hyrHAp+6+HMDMngCOAlqaWUmUJXYEKqL6FcBewMKoS94C+CKlfIvUfdJSBiki8cnMNcjPgSPMbKfoWmI/YBbwEnBaVGcw8FT0eXy0TrT9RXf3qPzMaJS7M9ANmFrTiZVBikh8MnCjuLtPMbNxwNtAFfAOyYzzGWCMmd0QlY2IdhkBjDKzucBKkiPXuPtMMxtLMrhWAZf4DuZns2RgzU+VK+blb+MkqLRNFwBKSvfMcUukPqoqF4G71WffDWOvr/NvttkZ19TrXNmgDFJE4pPHCVd9KECKSHz0LLaISIACpIhIgCarEBFpHJRBikh81MUWEQnQKLaISIAySBGRAAVIEZGAAhvFVoAUkdh4QtcgRUTSUxdbRCRAXWwRkQB1sUVEAtTFFhEJKLAAqWexYzRq7JOcfM5/MejsnzDq0b9ts+3+Rx7nwKOOZ9XqNQDMm7+As8uHcugxJ/HXh8dtU/e1t6Zz4pk/5vgzfsR9o8Zmrf2S1LRpU958/e/8c/pE3p3xItde83MAOnXaizdee5oPZ73Gw6PvobS0FIBv9z2cqVMmsHH9fE455YRcNj333Ou+5DEFyJjMmfcZj4+fwCP33c7jD9zNy29M5fOFiwBYvHQ5b0x9m/bt2m6t32LX5gwb+l+cf9ap2xynurqaG/50F/f86beMH30vz74wmU8+nZ/V79LYbdq0iWP7n8FhvY/jsN79GdD/GA7v04vf3fQrbr/j/+jeoy+rVq3hRxecBcDnCyoY8uOhPDLmydw2PB9k4LWvuZSxAGlm3c3sSjO7I1quNLP9M3W+XJv32QIOOmA/mpWVUVJSTO9DDuKFl18H4A933MvPLh6CpUwsv1urlhy0/36UlGx7leP92R/zjY57sleH9pSWlnJ8v+/w4qtvZfOrCLBu3XoASktLKCktxd357jFH8fjjzwAwatRjDPqPAQDMn7+Q99+fTSLPf+xZkfC6L3ksIwHSzK4ExgBG8q1hU6PPj5jZsEycM9e67rM3b787k9Vr1rJh40ZefXMaS5Yu58VX36Rtm93p3m2fWh1n2fIV7NG2zdb1dm13Z9nyLzLVbAkoKipi+rTnWVzxHpMmvcIn8z5j9eo1VFcn3/G0sGIxe3bYI8etzEOZeathzmRqkGYIcIC7V6YWmtmtwEzg9xk6b8506fQNfnT26ZQP/RXNysrYr9s+bK6s5P8efJTht92Y6+ZJHSUSCXp/sz8tWuzK44+NoPt+XXPdpIYhzzPCuspUFzsBpHulXftoW5CZlZvZdDObft+Dj2SkcZly6kkDGDvyTh64+xZ2bd6cLp33pmLREk4dfDH9Tx3M0uUrOP1Hl7Lii5XBY7RtsztLli3fur502QrattktG82XNNasWcvkl1/niCMOo2XLFhQXFwPQsUN7FlUsyXHrJNMylUFeAUwysznAgqjsG0BX4Kc17ejuw0m+87bBvfb1i1Wr2a1VSxYvWcakl19n9PDbOPeMk7du73/qYB4dcQetWrYIHuPA7vvy+cJFLFy0hHZtduMfk17mD9demYXWyxa7796aysoq1qxZS1lZGcf2O5pb/ng3k19+g1NPPYGxY8dz7rmnM/7p53Pd1LzjBXYdNiMB0t0nmNm+QB+gQ1RcAUzb0Yu6G7KhV9/A6rVrKSkp4Vc/v5hdm+8SrLvii5X8YMhlfLVuPUVFRTw09kmeGn0vu+y8M1cPvYif/OzXVFdX858n9qfrPntn8VtI+/btGDnidoqLiygqKmLcuKd55tkXmDX7Yx5+6G6uv+6XzHh3JiP/muzh9D6sJ+MeG0GrVi048YTjuPaan9PzkO/l+FvkSIF1sc3z+D6khpZBSlJpmy4AlJSmu8oi+a6qchG4245r/qt1N5xT59/szr9+qF7nygY9SSMi8SmwDFIBUkTio2uQIiIByiBFRALy/MbvulKAFJH4KIMUEUlP90GKiIQogxQRCVCAFBEJ0CCNiEiAMkgRkfRcAVJEJEABUkQkoMBu89FLu0REApRBikh81MUWEQlQgBQRSS+fJ+CuDwVIEYmPMkgRkQAFSBGR9HSjuIhIiAKkiEhAYd0nrgApIvFRF1tEJEQBUkQkQF1sEZH0Cq2LrckqRCQ+iXostWBmLc1snJl9aGazzexIM2ttZhPNbE70Z6uorpnZHWY218zeM7NeKccZHNWfY2aDd3ReBUgRiY0nvM5LLf0ZmODu3YGewGxgGDDJ3bsBk6J1gOOBbtFSDtwDYGatgWuBw4E+wLVbgmqIAqSIxCcDGaSZtQCOBkYAuPtmd18NDAIeiKo9AJwcfR4EPOhJbwEtzaw9MACY6O4r3X0VMBEYWNO5FSBFJDaeqPtSC52B5cBfzewdM7vPzHYG2rn74qjOEqBd9LkDsCBl/4VRWag8SAFSRHLKzMrNbHrKUr5dlRKgF3CPux8KrOPr7jQAnpxGKPYRIo1ii0h86nGbj7sPB4bXUGUhsNDdp0Tr40gGyKVm1t7dF0dd6GXR9gpgr5T9O0ZlFcAx25VPrqltyiBFJDaZ6GK7+xJggZntFxX1A2YB44EtI9GDgaeiz+OB86LR7COANVFX/Dmgv5m1igZn+kdlQcogRSQ+mbtR/FJgtJk1AeYBF5BM8Maa2RBgPnBGVPdZ4PvAXGB9VBd3X2lmvwWmRfWud/eVNZ1UAVJEYlPLQZe6H9d9BtA7zaZ+aeo6cEngOCOBkbU9rwKkiMQmUwEyVxQgRSQ2CpAiIiFuuW5BrBQgRSQ2yiBFRAI8oQxSRCQtZZAiIgGua5AiIukpgxQRCdA1SBGRAC+sNy4oQIpIfJRBiogEFFqA1HRnIiIByiBFJDa6BikiElBoXWwFSBGJTaHdKL7Da5Bm1s7MRpjZP6L1HtEMviIi28jQWw1zpjaDNPeTfG/DntH6x8AVGWqPiDRgCbc6L/msNgFyd3cfS/S2CXevAqoz2ioRaZDcrc5LPqvNNch1ZrYb0Ttnt7wlLKOtEpEGqTEO0vyM5GsUu5jZ60Ab4LSMtkpEGqRGd5uPu79tZt8B9gMM+MjdKzPeMhFpcBpdBmlm521X1MvMcPcHM9QmEWmg8n3Qpa5q08X+ZsrnMpLvoX0bUIAUkW3k+6BLXdWmi31p6rqZtQTGZKpBItJwNbprkGmsAzrH3RARafgaXRfbzJ4musWH5H2TPYCxmWyUiDRMja6LDfwx5XMVMN/dF2aoPSLSgDWqLraZFQPXuft3s9SebZS26ZKL00pMqioX5boJkmWNqovt7tVmljCzFu6up2dEpEaNsYv9FfC+mU0kOUADgLtflrFWRZ5pe2amTyEZcMKy5E0OG197KMctkfoo63tOrpuQN2oTIJ+IllQFdqVBROLQqLrYkZbu/ufUAjO7PEPtEZEGrNAyp9pMdzY4Tdn5MbdDRApAoc0HGcwgzews4IdAZzMbn7KpObAy0w0TkYanMQ3SvAEsBnYH/pRS/iXwXiYbJSINU56/QaHOggHS3ecD84EjazqAmb3p7jXWEZHGwWk8GWRtlcVwDBEpAIkCG6WJI0AW2F+JiNRXQhmkiEh6hdbFrs17sS81s1Y1VYmxPSLSgCXqseSz2twH2Q6YZmZjzWygmW0fEM/NQLtEpAFyrM5LPtthgHT3XwPdgBEkbxCfY2Y3mVmXaPsHGW2hiDQYjTGDxN0dWBItVUArYJyZ/SGDbRORBqbQAmRtZhS/HDgPWAHcB/zC3SvNrAiYA/wys00UkYYi37vMdVWbUezWwCnRjeNbuXvCzE7MTLNEpCEqsNdi1+qthtfWsG12vM0RkYas0O6DrNU1SBGRxkg3iotIbArtsToFSBGJTb6PSteVAqSIxCbxL8+RNGy6BikisfF6LLVlZsVm9o6Z/T1a72xmU8xsrpk9amZNovKm0frcaHunlGNcFZV/ZGYDdnROBUgRiU2GbxS/HEi9c+Zm4DZ37wqsAoZE5UOAVVH5bVE9zKwHcCZwADAQuNvMims6oQKkiMQmYXVfasPMOgInkHxYhWhOiO8B46IqDwAnR58HRetE2/tF9QcBY9x9k7t/CswF+tR0XgVIEYlNAqvzUku3k3xqb0vSuRuw2t2rovWFQIfocwdgAUC0fU1Uf2t5mn3SUoAUkdjU5xqkmZWb2fSUpTz1mNETe8vc/Z/Z+yZJGsUWkdjU51FDdx8ODK+hylHAf5jZ90m+4mVX4M9ASzMribLEjkBFVL8C2AtYaGYlQAvgi5TyLVL3SUsZpIjEJhODNO5+lbt3dPdOJAdZXnT3s4GXgNOiaoOBp6LP46N1ou0vRjOSjQfOjEa5O5OcxnFqTedWBikiscnykzRXAmPM7AbgHZJz1hL9OcrM5gIrSQZV3H2mmY0FZpGctvESd6+u6QQKkCISm0zP5uPuk4HJ0ed5pBmFdveNwOmB/W8Ebqzt+RQgRSQ2etRQRCRAAVJEJMAL61FsBUgRiU+hZZC6zUdEJEAZpIjEptAySAVIEYmNZhQXEQlodG81FBGpLXWxRUQCFCBFRAJ0DVJEJEDXIEVEAtTFFhEJUBdbRCQgUWAhUgFSRGKjLraISEBh5Y8KkCISI2WQIiIBus1HRCRAgzQiIgGFFR41Ya6ISJAySBGJjQZpREQCdA1SRCSgsMKjAqSIxEhdbBGRAHWxRUQCCis8KkCKSIzUxRYRCfACyyEVIEUkNsogRUQCNEgjNSsy+j5/ExuXrGT6Obdw8G3ltOi5D5ix7pPFvHvZPVSv30THHxxN92vOZuOSlQDMH/k8C0a/tPUwJbs04+hXb2HpP6Yz8+r7c/RlGpdRz7/FE6+8gxl069CW64cMomlp8ify+9ETePK1d3jrnqsAuOWR55j24WcAbNhcyaq163jtrisBuO2xF3j1vTkAlJ90NAP7HJD9L5MjhRUeFSBj1/nC4/lqTgUlzZsBMOt/RlH11QYA9v/NOXQaMoBP7hwPwOKn3gwGv32Hnc7Ktz7MSpsFlq5ay8MvTOVvN1xEWZNSfnH3OCZM+YBBfQ9h5qeLWLt+wzb1f3HWgK2fH35hKh9+vgSAV979mA/nL2bsdT9hc1UVP775Qfoe1JVdmjXN6vfJlULLIDVZRYzK2rem7XGHbpMJbgmOAMVlTXDf8f9Aux7cmaZtWrBi8nsZaaekV12dYNPmKqqqE2zYXEmbls2pTiS49bGJDD392OB+E6Z8wPGHJ7PEeYtW0GvfvSkpLmKnpk3o1rEtr78/N1tfIecS9VjyWU4CpJldkIvzZlqP357H7OsfxhPb/rMffPtPOPaDv7BLtz35bMRzW8v3OLEP337pZnrddwVle7ZOFprR47pzmH3d6Gw2vdFr12pXBg88kgG/uJ1jh95K852a8q0DuzBm0jSOOWQ/2rRsnna/RStWU7FiNX327wzAvnu1440P5rJhUyWrvlzPtA8/Y8nKtdn8Kjnl9fgvn+Uqg/xNjs6bMW2PO5TNK9ay9r1P/2Xbe1fcywsHX8RXHy9iz0FHArD0+bd5qfdlvPrdK1nx8vv0vPNiAPa+4DiWTZrBxsUrs9r+xm7tug289M5HPHvzZUy8dSgbNlXy9Ovv8vy0WZzVr09wvwlTZ3Js7/0pLkr+lL51YBf6HtSNwTeNZNi9j9Oza0eKiwpsmu0aFFoGmbFrkGYW6h8a0K6G/cqBcoCfNu/NwGZdM9C6+LXqsx9tB/Tiu/0OoaislNJdmnHIXZcw45K7khUSzqIn32Cfn57EwjEvU7nqq637fj76Rbpf88PkcXp3o/Xh3dn7/OMo2bkMa1JM1fqNfHTDmFx8rUbjrVmf0mH3lrTedWcA+vXqzt1PvcymzZWcNOxOADZuruTEYXfy999funW/CVNncvU5x29zrAtP+jYXnvRtAIbd+wR777Fblr5F7uV7RlhXmRykaQcMAFZtV27AG6Gd3H04MBzgmXZnNZi/7Y9uHMNHNyaDWOtv7c8+F5/IjEvuYqdO7Vj/2VIA2g04jHVzFgHQtG1LNi1bvbX8qzkVAMy4+K6tx+z4g6Np0XMfBccs2KP1rrw3r4INmyopa1LClNmfcm7/I/jhsV9nj0dc9LttguOni1fw5boN9OzScWtZdSLBl+s30nKXnfh4wVI+XriUGw44OZtfRWKUyQD5d2AXd5+x/QYzm5zB8+YPM3reeRElzZthZqydOZ8PfjkSgE4XDqRd/8Pw6moqV3/Fu5f9JceNbdwO7tKR43rvz5m/GU5xcRHdv7EHp32nV437TJjyAQP6HIDZ113oquoEF/zufgB2btaUmy78T0qKG89YaL53mevKajOqmisNKYOUr52wLJnxbnztoRy3ROqjrO854F6vC6fn7n1KnX+zo+Y/kbcXaXUfpIjEptAyGgVIEYlNod0orgApIrHRKLaISEChDdIoQIpIbNTFFhEJUBdbRCRAXWwRkYB8vq+6PhQgRSQ2hXYNsvE8AyUiGZeJ2XzMbC8ze8nMZpnZTDO7PCpvbWYTzWxO9GerqNzM7A4zm2tm75lZr5RjDY7qzzGzwTs6twKkiMQmQ/NBVgE/d/cewBHAJWbWAxgGTHL3bsCkaB3geKBbtJQD90AyoALXAocDfYBrtwTVEAVIEYlNAq/zsiPuvtjd344+fwnMBjoAg4AHomoPACdHnwcBD3rSW0BLM2tPcnaxie6+0t1XAROBgTWdW9cgRSQ2mR6kMbNOwKHAFKCduy+ONi3h63lmOwALUnZbGJWFyoOUQYpIbOpzDdLMys1sespSnu7YZrYL8Dhwhbtv8x4LT0bm2KOzMkgRiU19bhRPnSQ7xMxKSQbH0e7+RFS81Mzau/viqAu9LCqvAPZK2b1jVFYBHLNd+eSazqsMUkTymiVnJB4BzHb3W1M2jQe2jEQPBp5KKT8vGs0+AlgTdcWfA/qbWatocKZ/VBakDFJEYpOh+yCPAs4F3jezGVHZ1cDvgbFmNgSYD5wRbXsW+D4wF1gPXADg7ivN7LfAtKje9e5e49vxFCBFJDaZGKRx99dIvssqnX5p6jtwSeBYI4GRtT23AqSIxKbQnqRRgBSR2Gg2HxGRgIQmqxARSa+wwqMCpIjESNcgRUQCFCBFRAI0Ya6ISIAySBGRAN3mIyISoC62iEiAutgiIgHKIEVEApRBiogEFNogjSbMFREJUAYpIrHRZBUiIgGF1sVWgBSR2CiDFBEJUAYpIhKgDFJEJEAZpIhIgDJIEZEAZZAiIgHuiVw3IVYKkCISGz2LLSISoNl8REQClEGKiAQogxQRCdBtPiIiAbrNR0QkoNC62JowV0QkQBmkiMRGo9giIgGF1sVWgBSR2GgUW0QkQBmkiEiArkGKiAQogxQRCdA1SBGRAD1JIyISoAxSRCRA1yBFRALUxRYRCVAGKSISUGgB0vL6C5nlceNECpi71We3kiYd6vybrdpcUa9zZUN+B8gCZmbl7j481+2Q+tG/X+Og+SBzpzzXDZB/i/79GgEFSBGRAAVIEZEABcjc0fWrhk3/fo2ABmlERAKUQYqIBChA5oCZDTSzj8xsrpkNy3V7pPbMbKSZLTOzD3LdFsk8BcgsM7Ni4C7geKAHcJaZ9chtq6QO7gcG5roRkh0KkNnXB5jr7vPcfTMwBhiU4zZJLbn7K8DKXLdDskMBMvs6AAtS1hdGZSKSZxQgRUQCFCCzrwLYK2W9Y1QmInlGATL7pgHdzKyzmTUBzgTG57hNIpKGAmSWuXsV8FPgOWA2MNbdZ+a2VVJbZvYI8Cawn5ktNLMhuW6TZI6epBERCVAGKSISoAApIhKgACkiEqAAKSISoAApIhKgACkiEqAAKXnFzM43s//NdTtEQAFSsiSa5k2kQVGAlLTM7HozuyJl/UYzuzxNvWPM7BUzeyaaBPgvZlYUbfvKzP5kZu8CR5rZOWY21cxmmNm9W4KmmV1gZh+b2VTgqCx9RZEdUoCUkJHAeQBRwDsTeChQtw9wKckJgLsAp0TlOwNT3L0n8AXwA+Aodz8EqAbONrP2wG9IBsa+0TFE8kJJrhsg+cndPzOzL8zsUKAd8I67fxGoPtXd58HWZ5X7AuNIBsHHozr9gMOAaWYG0AxYBhwOTHb35dH+jwL7ZuZbidSNAqTU5D7gfGAPkhllyPYP9G9Z3+ju1dFnAx5w96tSK5rZyf9+M0UyQ11sqcnfSL5/5ZskZx8K6RNN31ZEshv9Wpo6k4DTzKwtgJm1NrO9gSnAd8xsNzMrBU6P9RuI/BuUQUqQu282s5eA1SmZYDrTgP8FugIvkQys2x9rlpn9Gng+CqSVwCXu/paZXUdyCrHVwIxYv4TIv0HTnUlQFMjeBk539zmBOscA/+3uJ2axaSJZoS62pBW9inYuMCkUHEUKnTJIqRUzOwgYtV3xJnc/PBftEckGBUgRkQB1sUVEAhQgRUQCFCBFRAIUIEVEAhQgRUQC/h/MZjoZojxKQAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 360x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wall time: 598 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# LightGBM algorithm\n",
    "import lightgbm as lgb\n",
    "classifier = lgb.LGBMClassifier()\n",
    "classifier.fit(X_train, y_train)\n",
    "y_pred = classifier.predict(X_test)\n",
    "print(classification_report(y_test,y_pred))\n",
    "print(\"Accuracy: \"+ str(accuracy_score(y_test, y_pred)))\n",
    "\n",
    "# Plot the confusion matrix\n",
    "cm=confusion_matrix(y_test,y_pred)\n",
    "f,ax=plt.subplots(figsize=(5,5))\n",
    "sns.heatmap(cm,annot=True,linewidth=0.5,linecolor=\"red\",fmt=\".0f\",ax=ax)\n",
    "plt.xlabel(\"y_pred\")\n",
    "plt.ylabel(\"y_true\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Considering both accuracy and execution time, **LightGBM** performs the best among the five algorithms:  \n",
    "Accuracy: 79.35%, time: 0.6s.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hyperparameter optimization\n",
    "Tune the hyperparameters of LightGBM to obtain optimal accuracy."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Optimize LightGBM by Bayesian Optimization with Tree-structured Parzen Estimator (BO-TPE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100%|███████████████████████████████████████████████| 50/50 [01:22<00:00,  1.64s/trial, best loss: -0.8021202980837473]\n",
      "LightGBM: Hyperopt estimated optimum {'learning_rate': 0.014116868973595653, 'max_depth': 18.0, 'min_child_samples': 45.0, 'n_estimators': 340.0, 'num_leaves': 600.0}\n"
     ]
    }
   ],
   "source": [
    "from hyperopt import hp, fmin, tpe, STATUS_OK, Trials\n",
    "from sklearn.model_selection import cross_val_score, StratifiedKFold\n",
    "\n",
    "# Define the objective function\n",
    "def objective(params):\n",
    "    params = {\n",
    "        'n_estimators': int(params['n_estimators']), \n",
    "        'max_depth': int(params['max_depth']),\n",
    "        'learning_rate': abs(float(params['learning_rate'])),\n",
    "        \"num_leaves\": int(params['num_leaves']),\n",
    "        \"min_child_samples\": int(params['min_child_samples']),\n",
    "    }\n",
    "    clf = lgb.LGBMClassifier( **params)\n",
    "    clf.fit(X_train,y_train)\n",
    "    prediction = clf.predict(X_test)\n",
    "    score = accuracy_score(y_test,prediction)\n",
    "    return {'loss':-score, 'status': STATUS_OK }\n",
    "\n",
    "# Define the hyperparameter configuration space\n",
    "space = {\n",
    "    'n_estimators': hp.quniform('n_estimators', 50, 500, 20),\n",
    "    'max_depth': hp.quniform('max_depth', 5, 50, 1),\n",
    "    \"learning_rate\":hp.uniform('learning_rate', 0, 1),\n",
    "    \"num_leaves\":hp.quniform('num_leaves',100,2000,100),\n",
    "    \"min_child_samples\":hp.quniform('min_child_samples',10,50,5),\n",
    "}\n",
    "\n",
    "# Detect the optimal hyperparameter values\n",
    "best = fmin(fn=objective,\n",
    "            space=space,\n",
    "            algo=tpe.suggest,\n",
    "            max_evals=50)\n",
    "print(\"LightGBM: Hyperopt estimated optimum {}\".format(best))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.69      0.97      0.81      9711\n",
      "           1       0.97      0.68      0.80     12833\n",
      "\n",
      "    accuracy                           0.80     22544\n",
      "   macro avg       0.83      0.82      0.80     22544\n",
      "weighted avg       0.85      0.80      0.80     22544\n",
      "\n",
      "Accuracy: 0.8021202980837473\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUgAAAE+CAYAAADvb4nvAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAbjElEQVR4nO3deXwV5b3H8c8vJMi+iiwBFQGpuFtBLe1Viwtob12xuKLlNq1aq9Z6VfSW1q22tnW52l5RrIjKUtzQuiOKG5uAyCISUcoSEAiLIJLtd/84A430PCGJc3JOTr5vX/PizDPPzDzHkB+/Z56ZZ8zdERGRf5eT7gaIiGQqBUgRkQAFSBGRAAVIEZEABUgRkQAFSBGRgNx0N6BKZroHSSQd3K02u5WuW1rj39m8Pfer1bnqQkYHyNK1n6S7CVILeR16AJCb1yXNLZHaKCtdle4mZIyMDpAiUs9UlKe7BbFSgBSR+HhFulsQKwVIEYlPhQKkiEhSrgxSRCRAGaSISIAySBGRAI1ii4gEKIMUEQnQNUgRkeQ0ii0iEqIMUkQkQBmkiEiARrFFRAKyLIPUhLkiIgHKIEUkPhqkEREJyLIutgKkiMRHGaSISHLuGsUWEUlOXWwRkQB1sUVEApRBiogE6EkaEZEAZZAiIgG6BikiEqAMUkQkQBmkiEiAAqSISHJ6kkZEJEQZpIhIQJYN0mjCXBGRAGWQIhIfdbFFRAKyrIutACki8VEGKSISkGUZpAZpRCQ+FRU1X6rBzK42swVmNt/MxppZEzPrbmbTzazQzMabWeOo7h7RemG0fd9Kx7khKl9sZifv7rwKkCISnxQESDPLB34BHOnuBwGNgCHA74G73L0nsAEYFu0yDNgQld8V1cPM+kT7HQgMBP5iZo2qOrcCpIjExytqvlRPLtDUzHKBZkAR8H1gYrR9NHB69Pm0aJ1o+wAzs6h8nLtvd/dPgUKgX1UnVYAUkfikIIN095XAH4F/kgiMm4D3gY3uXhZVWwHkR5/zgeXRvmVR/faVy5Psk5QCpIjEpxYZpJkVmNmsSktB5UOaWVsS2V93oAvQnEQXOeU0ii0i8anFbT7uPhIYWUWVE4BP3X0tgJk9BfQH2phZbpQldgVWRvVXAt2AFVGXvDWwvlL5DpX3SUoZpIjEJzXXIP8JHG1mzaJriQOAhcAU4OyozlDg2ejzpGidaPvr7u5R+ZBolLs70AuYUdWJlUGKSHxScKO4u083s4nAbKAMmEMi4/wHMM7Mbo3KRkW7jALGmFkhUExi5Bp3X2BmE0gE1zLgct/N/GyWCKyZqXTd0sxtnATldegBQG5elzS3RGqjrHQVuFtt9t024eYa/842PefXtTpXXVAGKSLxyeCEqzYUIEUkPnoWW0QkQAFSRCRAk1WIiDQMyiBFJD7qYouIBGgUW0QkQBmkiEiAAqSISECWjWIrQIpIbLxC1yBFRJJTF1tEJEBdbBGRAHWxRUQC1MUWEQnIsgCpZ7FjNGbCM5x+wc847fyfMmb801/b9sjYJzmo/yA2bNwEwNJlyzm/4GoOP+4/+dsTE6t9HEm9rl278Norf2feB1P4YO7rXPHzxOuWDzmkD29PncSc2a/xzNOP0LJlCwBOGPA9pk97kTmzX2P6tBc5/rj+6Wx+ernXfMlgyiBjsmTpZzw56SXGPnQ3ebl5/Oyamzi2/1Hs3bULRWvW8u6M2XTuuNfO+q1bteT6q3/G61Pfq/ZxpG6UlZVx7X//ljlz59OiRXNmTH+J1yZP5YH/u5PrrruFqW9N4+KhP+JX11zKiN/cybr1xZx+xsUUFa3hwAN788Lzj7NP9yPT/TXSQxlk9ZjZt8zsOjO7N1quM7MDUnW+dFv62XIOPrA3TZs0ITe3EUcedjCvvfkOAH+49wF+edkwrNLE8u3btuHgA3qTm5tb7eNI3Vi9+nPmzJ0PwJYtW/nooyXkd+nE/r32Y+pb0wB4bfJbnHHGKQDMnbuAoqI1ACxYsJimTZvQuHHj9DQ+3Sq85ksGS0mANLPrgHGAkXhr2Izo81gzuz4V50y3nvvtw+wPFrBx02a2ffUVb703k9Vr1vL6W++xV4c9+Vav/b7RcSQ99tmnK4cdehDTZ8xh4cKP+eEPTwbg7LN+QLckWf2ZZ57KnDnzKSkpqeumZobUvNUwbVLVxR4GHOjupZULzezPwALgjhSdN2167Ls3Pz5/MAVX30jTJk3o3Ws/SkpLefDR8Yy867ZvdJycHF0qTofmzZsxYfyD/PJXI/jiiy38V8EvufvPt3Dj8Kt4/vlXKCn52l9v+vTZn9/dNpxBp56XphZngAzPCGsqVb95FUCyi2ado21BZlZgZrPMbNZDj45NSeNS5az/PJkJD/8vo/9yJ61atqRH931YuWo1Zw29jJPOGsqatesY/OMrWLe+uEbH2XfvrnX0DWSH3Nxc/j7+QcaOfZpnnnkRgMWLP2HQqedx1NGDGDf+WZYu/Wxn/fz8zkz8+ygu+fGVLF26LE2tlrilKoO8CphsZkuA5VHZ3kBP4OdV7ejuI0m887bevfZ1/YaNtG/bhqLVnzP5zXd4fORdXHjO6Tu3n3TWUMaPupe2bVrX+DhStx4c+ScWfVTI3feM3FnWoUN71q5dj5kx/IYreWDkGABat27FpGcfZfiNt/Pue7PS1eSM4Fk2SJOSAOnuL5nZ/kA/ID8qXgnM3N2Luuuzq4ffysbNm8nNzeXGay6jVXQbSDLr1hfzo2G/YMvWL8nJyeGxCc/w7OMP0KJ58xodR+LX/zt9ufCCs5n34UJmzXwFgP/5nzvo2bM7l156MQDPPPMCj4weD8Dll11Czx77ctONV3PTjVcDMOiUc1m7dn1a2p9WWdbFNs/g+5DqWwYpCXkdegCQm6dbk+qjstJV4G67r/nvtt56QY1/Z5vf9FitzlUXdB+kiMQnyzJIBUgRiY+uQYqIBCiDFBEJyPAbv2tKAVJE4qMMUkQkOd0HKSISogxSRCRAAVJEJECDNCIiAcogRUSScwVIEZEABUgRkYAsu81HU1WLiAQogxSR+KiLLSISoAApIpJcJk/AXRsKkCISH2WQIiIBCpAiIsnpRnERkRAFSBGRgOy6T1wBUkTioy62iEiIAqSISIC62CIiyWVbF1uTVYhIfCpqsVSDmbUxs4lm9pGZLTKzY8ysnZm9amZLoj/bRnXNzO41s0Izm2dmR1Q6ztCo/hIzG7q78ypAikhsvMJrvFTTPcBL7v4t4FBgEXA9MNndewGTo3WAQUCvaCkA/gpgZu2AEcBRQD9gxI6gGqIAKSLxSUEGaWatgf8ARgG4e4m7bwROA0ZH1UYDp0efTwMe9YRpQBsz6wycDLzq7sXuvgF4FRhY1bkVIEUkNl5R86UaugNrgb+Z2Rwze8jMmgMd3b0oqrMa6Bh9zgeWV9p/RVQWKg9SgBSRtDKzAjObVWkp2KVKLnAE8Fd3PxzYyr+60wB4Yhqh2EeINIotIvGpxW0+7j4SGFlFlRXACnefHq1PJBEg15hZZ3cvirrQn0fbVwLdKu3fNSpbCRy3S/kbVbVNGaSIxCYVXWx3Xw0sN7PeUdEAYCEwCdgxEj0UeDb6PAm4KBrNPhrYFHXFXwZOMrO20eDMSVFZkDJIEYlP6m4UvwJ43MwaA0uBS0gkeBPMbBiwDDgnqvsCcApQCHwZ1cXdi83sFmBmVO9mdy+u6qQKkCISm2oOutT8uO5zgSOTbBqQpK4DlweO8zDwcHXPqwApIrFJVYBMFwVIEYmNAqSISIhbulsQKwVIEYmNMkgRkQCvUAYpIpKUMkgRkQDXNUgRkeSUQYqIBOgapIhIgGfXGxcUIEUkPsogRUQCsi1AarozEZEAZZAiEhtdgxQRCci2LrYCpIjEJttuFN/tNUgz62hmo8zsxWi9TzSDr4jI16TorYZpU51BmkdIvLehS7T+MXBVitojIvVYhVuNl0xWnQC5p7tPIHrbhLuXAeUpbZWI1EvuVuMlk1XnGuRWM2tP9M7ZHW8JS2mrRKReaoiDNL8k8RrFHmb2DtABODulrRKReqnB3ebj7rPN7FigN2DAYncvTXnLRKTeaXAZpJldtEvREWaGuz+aojaJSD2V6YMuNVWdLnbfSp+bkHgP7WxAAVJEvibTB11qqjpd7Csqr5tZG2BcqhokIvVXg7sGmcRWoHvcDRGR+q/BdbHN7DmiW3xI3DfZB5iQykaJSP3U4LrYwB8rfS4Dlrn7ihS1R0TqsQbVxTazRsBv3P34OmrP1+R16JGO00pMykpXpbsJUscaVBfb3cvNrMLMWru7np4RkSo1xC72FuBDM3uVxAANAO7+i5S1KjKx03mpPoWkwNmrnwDgq1lPp7klUhtNjjwj3U3IGNUJkE9FS2VZdqVBROLQoLrYkTbufk/lAjO7MkXtEZF6LNsyp+pMdzY0SdnFMbdDRLJAts0HGcwgzexc4Dygu5lNqrSpJVCc6oaJSP3TkAZp3gWKgD2BP1Uq/wKYl8pGiUj9lOFvUKixYIB092XAMuCYqg5gZu+5e5V1RKRhcBpOBlldTWI4hohkgYosG6WJI0Bm2f8SEamtCmWQIiLJZVsXuzrvxb7CzNpWVSXG9ohIPVZRiyWTVec+yI7ATDObYGYDzWzXgHhhCtolIvWQYzVeMtluA6S73wT0AkaRuEF8iZndbmY9ou3zU9pCEak3GmIGibs7sDpayoC2wEQz+0MK2yYi9Uy2BcjqzCh+JXARsA54CLjW3UvNLAdYAvx3apsoIvVFpneZa6o6o9jtgDOjG8d3cvcKM/tBapolIvVRlr0Wu1pvNRxRxbZF8TZHROqzbLsPslrXIEVEGiLdKC4iscm2x+oUIEUkNpk+Kl1TCpAiEpuKf3uOpH7TNUgRiY3XYqkuM2tkZnPM7PlovbuZTTezQjMbb2aNo/I9ovXCaPu+lY5xQ1S+2MxO3t05FSBFJDYpvlH8SqDynTO/B+5y957ABmBYVD4M2BCV3xXVw8z6AEOAA4GBwF/MrFFVJ1SAFJHYVFjNl+ows67AqSQeViGaE+L7wMSoymjg9OjzadE60fYBUf3TgHHuvt3dPwUKgX5VnVcBUkRiU4HVeDGzAjObVWkpSHLou0k8tbcj6WwPbHT3smh9BZAffc4HlgNE2zdF9XeWJ9knKQ3SiEhsanObj7uPBEaGtkdP7H3u7u+b2XG1bFqtKECKSGxS9Khhf+CHZnYKiVe8tALuAdqYWW6UJXYFVkb1VwLdgBVmlgu0BtZXKt+h8j5JqYstIrFJxSCNu9/g7l3dfV8Sgyyvu/v5wBTg7KjaUODZ6POkaJ1o++vRjGSTgCHRKHd3EtM4zqjq3MogRSQ2dfwkzXXAODO7FZhDYs5aoj/HmFkhUEwiqOLuC8xsArCQxLSNl7t7eVUnUIAUkdikejYfd38DeCP6vJQko9Du/hUwOLD/bcBt1T2fAqSIxEaPGoqIBChAiogEeHY9iq0AKSLxybYMUrf5iIgEKIMUkdhkWwapACkisdGM4iIiAQ3urYYiItWlLraISIACpIhIgK5BiogE6BqkiEiAutgiIgHqYouIBFRkWYhUgBSR2KiLLSISkF35owKkiMRIGaSISIBu8xERCdAgjYhIQHaFR02YKyISpAxSRGKjQRoRkQBdgxQRCciu8KgAKSIxUhdbRCRAXWwRkYDsCo8KkCISI3WxRUQCPMtySAVIEYmNMkgRkQAN0kjVcowBL93KV6s38M5Ff6THJSfS6ycDadG9E5MO/CklxVt2Vu1wzAEcevOFWF4jSoq/4M0zbwWg53+dTPfzjwczPn18CoUPvpSub9OgjHnhLZ6aMgMzo1e3Ttz808E0zsvlvgkv88r0D2mUYww+4RjOH9gfgJkLP+HOMc9RWlZO25bNefjXP9t5rPKKCs698X/Zq10r7rv2knR9pTqXXeFRATJ2vX4ykC+WrCKvZVMA1s/8mKJX53DsUzd9rV5eq2YcfsclvHXe79m2cj17tG8FQKveXel+/vG8fsqvqSgp47tPXEfRq3PY+tmaOv8uDcma4k088fI7PH3nNTRpnMe19zzGS+99gLuzev0mnv3jNeTk5LB+U+IfuM1bt3H7357hL9f9mM57tt1ZvsPjL77Nfvl7sWXbV+n4OmmTbRmkJquIUdPO7eg84DA+fWLKzrKN85fx5Yp1/1a32xnfYeULM9m2cj0A29dvBqBlry4Uz/6E8m0leHkF66YtIv+UvnXzBRq48vIKtpeUUlZezraSUjq0bcWE16bx0zMHkJOT+FVp37oFAC++O5cBfQ+i855tv1YOsGb9Rt6a+xFnHN/wfm4VtVgyWVoCpJllZZ/j0JsvZN6tY6Fi9/+KtuzRibzWzTn2yRsZ8PKt7D34uwBsXryCPY/qTeO2LWjUtDGdvn8Yzbq0S3XTG7yO7Voz9NT/4OQrfscJl91Gy6ZN+M4h+7Pi82JenjaPc2+8l8t+P4plRYl/7JYVrWXz1m0Mu+UBhgy/l+emvr/zWH8Y8xxXn3sKOZZls8dWg9fiv0yWri72b4G/pencKdH5hMPZvm4TG+d9RodjDthtfWvUiLaHdGfq4Ntp1DSP45/7LcXvF/LFklUsvv85vjfuesq+3M7GBcvwikz/d7b+27zlS6a8v5AX7rmOls2acu09j/H827MpKS2jcV4uY2/7Ba/NmM+IkX/nkRGXUlZewcJPVzByeAHbS0q5aMT9HNxrb5YVraNdqxb02a8rMxd+ku6vVeey7W9qygKkmc0LbQI6VrFfAVAAUNCqHyc265mC1sWvfb/96XzSt+k04DAa7ZFHbsum9L3vUmb+/K9J628rKqZkwxbKt22nfNt21k37iNZ99mbL0tV8NvZNPhv7JgAH3XAOX64qrsuv0iBNm19I/l5tadcq0VUe0PcgPvh4GR3btWZA34OisgMZ8cAEADq2b02bls1o1qQxzZo05ogDuvPxsiIWfbaSN2Yv5O25i9leWsrWbdu54f5x/O7yIWn7bnUp0zPCmkplBtkROBnYsEu5Ae+GdnL3kcBIgImdz683/7fn3z6e+bePBxKj0/tfemowOAKsevl9Dr9tKNYoh5zGubQ7ogdLRr4IwB7tW7F9/Waa5renyyl9mXLqiDr5Dg1Zpz3bMG/JP9m2vYQmjfOYvqCQPvt1pXnTJsxc+Ald92rHrEVL2adzBwCO/3YffvfIs5SVl1NaVs6Hhcu5YND3OOnoQ7hyyCAgMco9+h9TG0xwzEapDJDPAy3cfe6uG8zsjRSeN6P0HHYy+1/2A5rs1ZoTJ9/B6slzef9XD/HFklWsnjKPE1+/A6+o4NMn3mDz4hUAHDPqShq3bUlFaRlzb3iE0s1fpvlbZL9Deu7NiUcdzJDh99KoUQ7f2rcLZ3//KL4qKWX4/eN47MW3abZHY0b85CwA9svvSP9DejP4+rsxM848vi+9unVK87dIv2zrYpt75iZp9SmDlH85e/UTAHw16+k0t0Rqo8mRZ4B7rUaYLtznzBr/zo5Z9lTGjmbpPkgRiU22ZTQKkCISm2y7UVwBUkRio1FsEZGAbBukUYAUkdioiy0iEqAutohIgLrYIiIBmXxfdW0oQIpIbLLtGqTmgxSR2KRiPkgz62ZmU8xsoZktMLMro/J2ZvaqmS2J/mwblZuZ3WtmhWY2z8yOqHSsoVH9JWY2dHfnVoAUkdikaD7IMuAad+8DHA1cbmZ9gOuBye7eC5gcrQMMAnpFSwHwV0gEVGAEcBTQDxixI6iGKECKSGwq8Bovu+PuRe4+O/r8BbAIyAdOA0ZH1UYDp0efTwMe9YRpQBsz60xidrFX3b3Y3TcArwIDqzq3rkGKSGxSPUhjZvsChwPTgY7uXhRtWs2/5pnNB5ZX2m1FVBYqD1IGKSKxqc01SDMrMLNZlZaCZMc2sxbAk8BV7r658jZPRObYo7MySBGJTW1uFK88SXaImeWRCI6Pu/tTUfEaM+vs7kVRF/rzqHwl0K3S7l2jspXAcbuUv1HVeZVBikhGMzMDRgGL3P3PlTZNAnaMRA8Fnq1UflE0mn00sCnqir8MnGRmbaPBmZOisiBlkCISmxTdB9kfuBD40MzmRmXDgTuACWY2DFgGnBNtewE4BSgEvgQuAXD3YjO7BZgZ1bvZ3at84ZMCpIjEJhWDNO7+Nol3WSUzIEl9By4PHOth4OHqnlsBUkRik21P0ihAikhsNJuPiEhAhSarEBFJLrvCowKkiMRI1yBFRAIUIEVEAjRhrohIgDJIEZEA3eYjIhKgLraISIC62CIiAcogRUQClEGKiARk2yCNJswVEQlQBikisdFkFSIiAdnWxVaAFJHYKIMUEQlQBikiEqAMUkQkQBmkiEiAMkgRkQBlkCIiAe4V6W5CrBQgRSQ2ehZbRCRAs/mIiAQogxQRCVAGKSISoNt8REQCdJuPiEhAtnWxNWGuiEiAMkgRiY1GsUVEArKti60AKSKx0Si2iEiAMkgRkQBdgxQRCVAGKSISoGuQIiIBepJGRCRAGaSISICuQYqIBKiLLSISoAxSRCQg2wKkZfQXMsvgxolkMXerzW65jfNr/DtbVrKyVueqC5kdILOYmRW4+8h0t0NqRz+/hkHzQaZPQbobIN+Ifn4NgAKkiEiAAqSISIACZPro+lX9pp9fA6BBGhGRAGWQIiIBCpBpYGYDzWyxmRWa2fXpbo9Un5k9bGafm9n8dLdFUk8Bso6ZWSPgfmAQ0Ac418z6pLdVUgOPAAPT3QipGwqQda8fUOjuS929BBgHnJbmNkk1uftUoDjd7ZC6oQBZ9/KB5ZXWV0RlIpJhFCBFRAIUIOveSqBbpfWuUZmIZBgFyLo3E+hlZt3NrDEwBJiU5jaJSBIKkHXM3cuAnwMvA4uACe6+IL2tkuoys7HAe0BvM1thZsPS3SZJHT1JIyISoAxSRCRAAVJEJEABUkQkQAFSRCRAAVJEJEABUkQkQAFSMoqZXWxm96W7HSKgACl1JJrmTaReUYCUpMzsZjO7qtL6bWZ2ZZJ6x5nZVDP7RzQJ8P+ZWU60bYuZ/cnMPgCOMbMLzGyGmc01swd2BE0zu8TMPjazGUD/OvqKIrulACkhDwMXAUQBbwjwWKBuP+AKEhMA9wDOjMqbA9Pd/VBgPfAjoL+7HwaUA+ebWWfgtyQC43ejY4hkhNx0N0Ayk7t/ZmbrzexwoCMwx93XB6rPcPelsPNZ5e8CE0kEwSejOgOAbwMzzQygKfA5cBTwhruvjfYfD+yfmm8lUjMKkFKVh4CLgU4kMsqQXR/o37H+lbuXR58NGO3uN1SuaGanf/NmiqSGuthSladJvH+lL4nZh0L6RdO35ZDoRr+dpM5k4Gwz2wvAzNqZ2T7AdOBYM2tvZnnA4Fi/gcg3oAxSgty9xMymABsrZYLJzATuA3oCU0gE1l2PtdDMbgJeiQJpKXC5u08zs9+QmEJsIzA31i8h8g1oujMJigLZbGCwuy8J1DkO+JW7/6AOmyZSJ9TFlqSiV9EWApNDwVEk2ymDlGoxs4OBMbsUb3f3o9LRHpG6oAApIhKgLraISIACpIhIgAKkiEiAAqSISIACpIhIwP8DaHJmUWUBu9kAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 360x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Use the optimal hyperparameter values to train the optimized LightGBM model\n",
    "clf = lgb.LGBMClassifier(max_depth=18, learning_rate= 0.014116868973595653, n_estimators = 340, \n",
    "                         num_leaves = 600, min_child_samples = 45)\n",
    "clf.fit(X_train,y_train)\n",
    "y_pred = clf.predict(X_test)\n",
    "print(classification_report(y_test,y_pred))\n",
    "print(\"Accuracy: \"+ str(accuracy_score(y_test, y_pred)))\n",
    "\n",
    "# Plot the confusion matrix\n",
    "cm=confusion_matrix(y_test,y_pred)\n",
    "f,ax=plt.subplots(figsize=(5,5))\n",
    "sns.heatmap(cm,annot=True,linewidth=0.5,linecolor=\"red\",fmt=\".0f\",ax=ax)\n",
    "plt.xlabel(\"y_pred\")\n",
    "plt.ylabel(\"y_true\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Optimize LightGBM by particle swarm optimization (PSO)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'n_estimators': 78.1513671875, 'max_depth': 6.808439403826421, 'learning_rate': 0.20349557055687642, 'num_leaves': 1302.232421875, 'min_child_samples': 27.56027628043679}\n",
      "Accuracy:0.8069109297374024\n"
     ]
    }
   ],
   "source": [
    "#PSO\n",
    "import optunity\n",
    "import optunity.metrics\n",
    "\n",
    "# Define the hyperparameter configuration space\n",
    "search = {\n",
    "    'n_estimators': [50, 500],\n",
    "    'max_depth': [5, 50],\n",
    "    'learning_rate': (0, 1),\n",
    "    \"num_leaves\":[100, 2000],\n",
    "    \"min_child_samples\":[10, 50],\n",
    "         }\n",
    "# Define the objective function\n",
    "def performance(n_estimators=None, max_depth=None,learning_rate=None,num_leaves=None,min_child_samples=None):\n",
    "    clf = lgb.LGBMClassifier(n_estimators=int(n_estimators),\n",
    "                                   max_depth=int(max_depth),\n",
    "                                   learning_rate=float(learning_rate),\n",
    "                                   num_leaves=int(num_leaves),\n",
    "                                   min_child_samples=int(min_child_samples),\n",
    "                                  )\n",
    "    clf.fit(X_train,y_train)\n",
    "    prediction = clf.predict(X_test)\n",
    "    score = accuracy_score(y_test,prediction)\n",
    "    return score\n",
    "\n",
    "# Detect the optimal hyperparameter values\n",
    "optimal_configuration, info, _ = optunity.maximize(performance,\n",
    "                                                  solver_name='particle swarm',\n",
    "                                                  num_evals=50,\n",
    "                                                   **search\n",
    "                                                  )\n",
    "print(optimal_configuration)\n",
    "print(\"Accuracy:\"+ str(info.optimum))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.70      0.97      0.81      9711\n",
      "           1       0.97      0.68      0.80     12833\n",
      "\n",
      "    accuracy                           0.81     22544\n",
      "   macro avg       0.83      0.83      0.81     22544\n",
      "weighted avg       0.85      0.81      0.81     22544\n",
      "\n",
      "Accuracy: 0.8069109297374024\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUgAAAE+CAYAAADvb4nvAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAcPklEQVR4nO3deXwV1f3/8deHhFV2RJaAgki1uKEiqPh1oxVw+eK31apfFWppca/70qpfFLEuFVF/VixuqFUWUSulLlAEcWVRkM2FiKJgWGSVnSSf3x93iAFzQhLn5t7cvJ8+5sGdM2funDHkw+fMmTlj7o6IiPxYjVQ3QEQkXSlAiogEKECKiAQoQIqIBChAiogEKECKiARkp7oBpTLTPUgiqeBuFdlt+3eLyv07W3PPfSt0rMqQ1gFy+8ovUt0EqYCazTsAkF2zdYpbIhWRv/3bVDchbaR1gBSRKqawINUtiJUCpIjExwtT3YJYKUCKSHwKFSBFRErkyiBFRAKUQYqIBCiDFBEJ0Ci2iEiAMkgRkQBdgxQRKZlGsUVEQpRBiogEKIMUEQnQKLaISECGZZCaMFdEJEAZpIjER4M0IiIBGdbFVoAUkfgogxQRKZm7RrFFREqmLraISIC62CIiAcogRUQC9CSNiEiAMkgRkQBdgxQRCVAGKSISoAxSRCRAAVJEpGR6kkZEJEQZpIhIQIYN0mjCXBGRAGWQIhIfdbFFRAIyrIutACki8VEGKSISoAxSRCQgwzJIjWKLSHwKC8u/lIGZXW1m881snpmNNLM6ZtbezKaZWa6ZjTazWlHd2tF6brS9XbHv+VNU/pmZ9dzdcRUgRSQ+Xlj+ZTfMLAf4I9DF3Q8CsoBzgHuAoe6+H7AG6B/t0h9YE5UPjephZp2i/Q4EegGPmFlWacdWgBSR+CQpgyRxObCumWUD9YA84CRgbLT9aeCM6HOfaJ1oew8zs6h8lLtvdfcvgVyga2kHVYAUkfhUIIM0swFmNrPYMmCnr3RfCtwHfE0iMK4DPgTWunt+VG0JkBN9zgG+ifbNj+o3K15ewj4l0iCNiMSnAoM07j4cGB7abmZNSGR/7YG1wAskushJpwApIvFJzm0+vwC+dPeVAGb2EtAdaGxm2VGW2AZYGtVfCrQFlkRd8kbAqmLlOxTfp0TqYotIfJJzDfJr4CgzqxddS+wBLAAmA2dGdfoBr0Sfx0XrRNvfdHePys+JRrnbAx2B6aUdWBmkiMQnCfdBuvs0MxsLfATkA7NIdMn/DYwys8FR2RPRLk8Az5pZLrCaxMg17j7fzMaQCK75wGW+mwksLRFY09P27xalb+MkqGbzDgBk12yd4pZIReRv/xbcrSL7bh59e7l/Z+uePbBCx6oMyiBFJD4Z9iSNAqSIxEcBUkQkIMMmq9AotohIgDJIEYmPutgiIgFpfFdMRShAikh8lEGKiAQoQIqIBGTYKLYCpIjExgt1DVJEpGTqYouIBKiLLSISoC62iEiAutgiIgEZFiD1LHaMnh3zT844/2L6nHcRz45+eadtI0a+yEHde7Nm7ToAxr/xJv/T9xL+54JLOO+ia/h04aKd6hcUFHDmby/j0usHVlr7JaFNm9b8Z8ILzPl4Mh/PfpMrLk+8TfTQQw/k3bf/xcwZE/jg/Vc5sktnAK695mJmzpjAzBkTmD1rEls3f02TJo1TdwKp5F7+JY0pg4zJwkVf8eK41xn5+APUzK7JxdfewvHdu7F3m9bkLV/Je9M/olWLvYrq57RuyYiH76VRwwa8/f4Mbr/3IUY+9kDR9n+88Ar7ttubDRs3peBsqrf8/Hyuv+F2Zs2eR/36ezB92uv8Z9JU7v7Lzdwx+H5ef2MyvXudxN133UyPX57FkPsfZcj9jwJw2qm/5Mo//oE1a9am9iRSRRlk2ZjZAWZ2o5k9FC03mtnPk3W8VFv01TccfOD+1K1Th+zsLLp0Ppj/vPUuAPc+9HeuubQ/Vmze5MMO7kSjhg0AOOTAA1i+4ruibctWrGTqe9P59ek9K/UcJGHZshXMmj0PgA0bNvLppwvJad0Sd6dB9DNr2KgB3+Yt/9G+Z5/dh1Gj/1mZzU0vhV7+JY0lJUCa2Y3AKMBIvBRnevR5pJndlIxjptp+++7DRx/PZ+269WzesoW335/BsuUrefPt99mr+Z4c0HHf4L4vjX+DY4/qUrR+z4M7AqqugKTaPvu0ofOhBzFt+iyuuW4g99x1C19+MYN7776Vm2+5a6e6devWoefJJ/DSy6+mqLVpoALvxU5nyepi9wcOdPftxQvN7H5gPnB3ko6bMh3a7c3vzjuLAVffTN06ddi/475s276dx54ZzfChdwb3m/7hx7w0fgLPDrsPgCnvTqNpk8YceEBHpn80p7KaLyXYY496jBn9GNdcN5Dvv9/ARQP6cu31t/Hyy69y5pmn89jfh9Cz9zlF9U877WTee39m9e1eQ9pnhOWVlJd2mdmnQE93X7xL+T7ABHffv5R9BwADAB4ZMviI3/c9N/b2VYYHHh1Bs6aNeezpUdSpUxuA5Su/o/mezRj12APs2awpn+V+yZV/GsSjQ+6g3d5tABg67CnGvzGJrKwstm7bzsaNm+hx/DHcM/CGVJ5OuWTCS7uys7MZ98+nmTDxLR54MPFO+1UrP6FZ8x+uEq3+7lOa7nlA0frYFx5n7IvjGTXqn5Xd3Fj9lJd2bbrnwnIHlHo3PlXtXtp1FTDJzBYC30RlewP7AZeXtqO7DyfxSscq91bDVWvW0qxJY/KWrWDSW+/y3PChXPCbM4q2n/zrfox+4iGaNG5E3rIVXPXnO7jr/64vCo4AV19yIVdfciEA0z+aw4iRL1ap4JgpHhs+hE8+zS0KjgDf5i3n+OOO5q2p73PSiceyMPfLom0NGzbguP86ir79rkhFc9OGZ9ggTVICpLu/bmY/A7oCOVHxUmDG7t5DW5Vd/efBrF2/nuzsbG6+9lIaNqgfrDvsqedZt/57Bt/3NwCysrIY8+RDldVUKUX3Y47kgvPPZM7cBcycMQGAW2+9m4svvp777x9EdnY2W7ds4ZJLfviH64w+vZn4n6ls2rQ5Vc1OD+piV56qlkFKQiZ0sauzn9LF3jj4/HL/zu5xyz+qXRdbRKqjDMsgFSBFJD66BikiEqAMUkQkIM1v/C4vBUgRiY8ySBGRkuk+SBGREGWQIiIBCpAiIgEapBERCVAGKSJSMleAFBEJUIAUEQnIsNt8NKe/iEiAMkgRiY+62CIiAQqQIiIlS+cJuCtCAVJE4qMMUkQkQAFSRKRkulFcRCREAVJEJCCz7hNXgBSR+KiLLSISogApIhKgLraISMkyrYutySpEJD6FFVjKwMwam9lYM/vUzD4xs6PNrKmZTTSzhdGfTaK6ZmYPmVmumc0xs8OLfU+/qP5CM+u3u+MqQIpIbLzQy72U0YPA6+5+AHAo8AlwEzDJ3TsCk6J1gN5Ax2gZAAwDMLOmwECgG9AVGLgjqIYoQIpIfJKQQZpZI+A44AkAd9/m7muBPsDTUbWngTOiz32AZzzhA6CxmbUCegIT3X21u68BJgK9Sju2AqSIxMYLy7+UQXtgJfCUmc0ys8fNbA+ghbvnRXWWAS2izznAN8X2XxKVhcqDFCBFJKXMbICZzSy2DNilSjZwODDM3Q8DNvJDdxoAT0wjFPsIkUaxRSQ+FbjNx92HA8NLqbIEWOLu06L1sSQC5HIza+XueVEXekW0fSnQttj+baKypcAJu5RPKa1tyiBFJDbJ6GK7+zLgGzPbPyrqASwAxgE7RqL7Aa9En8cBfaPR7KOAdVFX/A3gZDNrEg3OnByVBSmDFJH4JO9G8SuA58ysFrAIuJBEgjfGzPoDi4HfRHVfBU4BcoFNUV3cfbWZ3QHMiOoNcvfVpR1UAVJEYlPGQZfyf6/7bKBLCZt6lFDXgcsC3/Mk8GRZj6sAKSKxSVaATBUFSBGJjQKkiEiIW6pbECsFSBGJjTJIEZEAL1QGKSJSImWQIiIBrmuQIiIlUwYpIhKga5AiIgGeWW9cUIAUkfgogxQRCci0AKnpzkREApRBikhsdA1SRCQg07rYCpAiEptMu1F8t9cgzayFmT1hZq9F652iGXxFRHaSpLcapkxZBmlGkHhvQ+to/XPgqiS1R0SqsEK3ci/prCwBck93H0P0tgl3zwcKktoqEamS3K3cSzoryzXIjWbWjOidszveEpbUVolIlVQdB2muIfEaxQ5m9i7QHDgzqa0SkSqp2t3m4+4fmdnxwP6AAZ+5+/akt0xEqpxql0GaWd9dig43M9z9mSS1SUSqqHQfdCmvsnSxjyz2uQ6J99B+BChAishO0n3QpbzK0sW+ovi6mTUGRiWrQSJSdVW7a5Al2Ai0j7shIlL1Vbsutpn9i+gWHxL3TXYCxiSzUSJSNVW7LjZwX7HP+cBid1+SpPaISBVWrbrYZpYF3ObuJ1ZSe3ZSs3mHVBxWYpK//dtUN0EqWbXqYrt7gZkVmlkjd9fTMyJSqurYxd4AzDWziSQGaABw9z8mrVWRka3+N9mHkCQ4N+95ALbM/neKWyIVUafzqaluQtooS4B8KVqKy7ArDSISh2rVxY40dvcHixeY2ZVJao+IVGGZljmVZbqzfiWU/TbmdohIBsi0+SCDGaSZnQv8L9DezMYV29QAWJ3sholI1VOdBmneA/KAPYEhxcq/B+Yks1EiUjWl+RsUyi0YIN19MbAYOLq0LzCz99291DoiUj041SeDLKs6MXyHiGSAwgwbpYkjQGbY/xIRqahCZZAiIiXLtC52Wd6LfYWZNSmtSoztEZEqrLACSzory32QLYAZZjbGzHqZ2a4B8YIktEtEqiDHyr2ks90GSHe/BegIPEHiBvGFZvYXM+sQbZ+X1BaKSJVRHTNI3N2BZdGSDzQBxprZvUlsm4hUMZkWIMsyo/iVQF/gO+Bx4Hp3325mNYCFwA3JbaKIVBXp3mUur7KMYjcFfhXdOF7E3QvN7LTkNEtEqqIMey12md5qOLCUbZ/E2xwRqcoy7T7IMl2DFBGpjnSjuIjEJtMeq1OAFJHYpPuodHkpQIpIbAp/9BxJ1aZrkCISG6/AUlZmlmVms8xsfLTe3symmVmumY02s1pRee1oPTfa3q7Yd/wpKv/MzHru7pgKkCISmyTfKH4lUPzOmXuAoe6+H7AG6B+V9wfWROVDo3qYWSfgHOBAoBfwiJlllXZABUgRiU2hlX8pCzNrA5xK4mEVojkhTgLGRlWeBs6IPveJ1om294jq9wFGuftWd/8SyAW6lnZcBUgRiU0hVu7FzAaY2cxiy4ASvvoBEk/t7Ug6mwFr3T0/Wl8C5ESfc4BvAKLt66L6ReUl7FMiDdKISGwqcpuPuw8Hhoe2R0/srXD3D83shAo2rUIUIEUkNkl61LA78N9mdgqJV7w0BB4EGptZdpQltgGWRvWXAm2BJWaWDTQCVhUr36H4PiVSF1tEYpOMQRp3/5O7t3H3diQGWd509/OAycCZUbV+wCvR53HROtH2N6MZycYB50Sj3O1JTOM4vbRjK4MUkdhU8pM0NwKjzGwwMIvEnLVEfz5rZrnAahJBFXefb2ZjgAUkpm28zN0LSjuAAqSIxCbZs/m4+xRgSvR5ESWMQrv7FuCswP53AneW9XgKkCISGz1qKCISoAApIhLgmfUotgKkiMQn0zJI3eYjIhKgDFJEYpNpGaQCpIjERjOKi4gEVLu3GoqIlJW62CIiAQqQIiIBugYpIhKga5AiIgHqYouIBKiLLSISUJhhIVIBUkRioy62iEhAZuWPCpAiEiNlkCIiAbrNR0QkQIM0IiIBmRUeNWGuiEiQMkgRiY0GaUREAnQNUkQkILPCowKkiMRIXWwRkQB1sUVEAjIrPCpAikiM1MUWEQnwDMshFSBFJDbKIEVEAjRII6WyGkbP1wezKW8NU/vdxx5tm3PMsMup3aQ+q+d+xQdXPELh9gIA2p7ejYOv/TW4s2bB17x/2d8AqJfTjK73/YF6rZuCw1vn38vGJd+l8rSqhWfHT+GlNz/AzOjYthWDLj2XiwYPY9PmrQCsXr+BgzrszQM39Adgxvxc/jriZbYXFNCkQX2evP1yvvp2BTcMfbroO5esWMWlv+nN+acen5JzqmyZFR4VIGP3s9/3Yt3Cb6lZvy4AnW8+h88ee42vX/mALnf/jn3PPYHcZyZRv30LDrziv5nY5za2r9tE7WYNi77jqAcvZsFDr7Bs6jyy69XGPdP+2qWf5avX8vxrb/Py0BupU6sW198/gtffm8WIQX8sqnPNfU9x4pEHAbB+42b+8vhYHrn5Ilrt2YRV674HoF3rvRjz1+sBKCgs5JcX3cZJXQ+u/BNKkUzLIDVZRYzqtmpK6x6dWfT85KKyFsceyDfjpwPw5QtTadOrCwD7nXcSn4+YyPZ1mwDYumo9AA075lAjO4tlU+cBkL9pKwWbt1XmaVRbBYWFbN22nfyCAjZv207zJj/8o7Vh0xamz1/IiUcmgt1r73xIj26H0GrPJgA0a9TgR983be7ntG3ZjNbNm1bOCaSBwgos6SwlGaSZXejuT6Xi2Ml0+O0XMHvwyKLssVbT+mxbtxEvSPw12Jy3mrotE79QDfZtCcAvXhmI1ajBvCEvkjdlDg06tGTbuk0c+/hV1N+7OcvensfHd47CCzPrX+Z006JpY/qdfgI9LxlEnVo1OfrQ/Tnm0AOKtk+eMZduB3Wkfr06ACzOW0l+fgH9b3uYjZu3ct4px3H68Ufu9J2vvzuLXt0Pr9TzSLVMG8VOVQZ5e4qOmzStf3EYW79bx5q5X5WpvmVlUb99Cyb9ejDvXfowR973e2o2rEeNrCyad9ufWYOe443et1J/771of/ZxyW28sH7DJibPmMerf7uViX+/nc1btjF+6syi7a+9+xG9iwW7/IJCFny5hP930x8YdvNFDH9xAl99u6Jo+/b8fN76cD4nH9W5Mk8j5ZRBlpGZzQltAlqUst8AYABA/0Zd6VFvvyS0Ln7Nj/wZOScfQasencmqXZOaDepyxKC+1Gq0B5ZVAy8opG6rpmxetgaATXmrWTUrF88vYOM3K/n+izwatG/JprzVrJ2/mI1frwRgyesf0uyI/WDkW6k8vYz3wdzPydmrGU0b1gegR7dD+PjzrzjtuC6sWb+BeblfM/S63xXVb9GsEY0b1KNendrUq1Obw3/egc8Xf0u71nsB8M6sTzigfQ7NGv+4653JlEGWXQugL3B6Ccuq0E7uPtzdu7h7l6oSHAE+vms0r3S5gn91u4r3LnmY5e8s4P3LH2H5uwtoe1pXANqfdRxL3vgQgKWvz6TF0T8HEl3xBh1aseHrFaye/QU1G9ajdtPEL1aLYzux/vOlqTmpaqTlnk2Ys/ArNm/dhrszbe7ntM9JBLuJH3zMcYd3onatmkX1T+xyMLM+/TJxvXLrNubmLqZ9zg//7r/27qydMk6pmpJ5DXI8UN/dZ++6wcymJPG4aWX2nSPpPuwKDrnhLNbMW8yikVMAyJsyh5bHH8wpU+7FCwqZfcfzbFuzIbHPHc9z0pg/gxmr53zJF8+9mcIzqB4O6bgPvzzqUM65cQhZWTU4oF0OZ/7iGADeeG8Wvzujx071923Tgu6dD+Cs6/6K1TB+ddJRdNy7FQCbtmzlgzmfceuAsyr9PFIt3bvM5WXpfAvJyNbnpW/jJOjcvOcB2DL73yluiVREnc6ngnuF3k94wT6/Kvfv7LOLX0rbdyHqPkgRiU2mZTQKkCISm0y7UVwBUkRik2mj2AqQIhKbTBukUYAUkdioiy0iEqAutohIgLrYIiIB6XxfdUUoQIpIbDLtGqTmgxSR2CRjNh8za2tmk81sgZnNN7Mro/KmZjbRzBZGfzaJys3MHjKzXDObY2aHF/uuflH9hWbWb3fHVoAUkdh4Bf4rg3zgWnfvBBwFXGZmnYCbgEnu3hGYFK0D9AY6RssAYBgkAiowEOgGdAUG7giqIQqQIhKbQrzcy+64e567fxR9/h74BMgB+gA7XgD0NHBG9LkP8IwnfAA0NrNWQE9goruvdvc1wESgV2nH1jVIEYlNsgdpzKwdcBgwDWjh7nnRpmX8MM9sDvBNsd2WRGWh8iBlkCISm4pcgzSzAWY2s9gyoKTvNrP6wIvAVe6+vvg2T0Tm2KOzMkgRiU1FbhR39+HA8NLqmFlNEsHxOXd/KSpebmat3D0v6kLveOfFUqBtsd3bRGVLgRN2KZ9S2nGVQYpIWjMzA54APnH3+4ttGgfsGInuB7xSrLxvNJp9FLAu6oq/AZxsZk2iwZmTo7IgZZAiEpsk3QfZHbgAmGtms6OyPwN3A2PMrD+wGPhNtO1V4BQgF9gEXAjg7qvN7A5gRlRvkLuvLu3ACpAiEptkDNK4+zskXvZXkh67FkTXIy8LfNeTwJNlPbYCpIjEJtOepFGAFJHYaDYfEZGAQk1WISJSsswKjwqQIhIjXYMUEQlQgBQRCdCEuSIiAcogRUQCdJuPiEiAutgiIgHqYouIBCiDFBEJUAYpIhKQaYM0mjBXRCRAGaSIxEaTVYiIBGRaF1sBUkRiowxSRCRAGaSISIAySBGRAGWQIiIByiBFRAKUQYqIBLgXproJsVKAFJHY6FlsEZEAzeYjIhKgDFJEJEAZpIhIgG7zEREJ0G0+IiIBmdbF1oS5IiIByiBFJDYaxRYRCci0LrYCpIjERqPYIiIByiBFRAJ0DVJEJEAZpIhIgK5BiogE6EkaEZEAZZAiIgG6BikiEqAutohIgDJIEZGATAuQltYnZJbGjRPJYO5Wkd2ya+WU+3c2f9vSCh2rMqR3gMxgZjbA3Yenuh1SMfr5VQ+aDzJ1BqS6AfKT6OdXDShAiogEKECKiAQoQKaOrl9Vbfr5VQMapBERCVAGKSISoACZAmbWy8w+M7NcM7sp1e2RsjOzJ81shZnNS3VbJPkUICuZmWUBfwN6A52Ac82sU2pbJeUwAuiV6kZI5VCArHxdgVx3X+Tu24BRQJ8Ut0nKyN2nAqtT3Q6pHAqQlS8H+KbY+pKoTETSjAKkiEiAAmTlWwq0LbbeJioTkTSjAFn5ZgAdzay9mdUCzgHGpbhNIlICBchK5u75wOXAG8AnwBh3n5/aVklZmdlI4H1gfzNbYmb9U90mSR49SSMiEqAMUkQkQAFSRCRAAVJEJEABUkQkQAFSRCRAAVJEJEABUtKKmf3WzB5OdTtEQAFSKkk0zZtIlaIAKSUys0FmdlWx9TvN7MoS6p1gZlPN7N/RJMCPmlmNaNsGMxtiZh8DR5vZ+WY23cxmm9nfdwRNM7vQzD43s+lA90o6RZHdUoCUkCeBvgBRwDsH+EegblfgChITAHcAfhWV7wFMc/dDgVXA2UB3d+8MFADnmVkr4HYSgfHY6DtE0kJ2qhsg6cndvzKzVWZ2GNACmOXuqwLVp7v7Iih6VvlYYCyJIPhiVKcHcAQww8wA6gIrgG7AFHdfGe0/GvhZcs5KpHwUIKU0jwO/BVqSyChDdn2gf8f6FncviD4b8LS7/6l4RTM746c3UyQ51MWW0rxM4v0rR5KYfSikazR9Ww0S3eh3SqgzCTjTzPYCMLOmZrYPMA043syamVlN4KxYz0DkJ1AGKUHuvs3MJgNri2WCJZkBPAzsB0wmEVh3/a4FZnYLMCEKpNuBy9z9AzO7jcQUYmuB2bGehMhPoOnOJCgKZB8BZ7n7wkCdE4Dr3P20SmyaSKVQF1tKFL2KNheYFAqOIplOGaSUiZkdDDy7S/FWd++WivaIVAYFSBGRAHWxRUQCFCBFRAIUIEVEAhQgRUQCFCBFRAL+PxU4Ye/mZ/5VAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 360x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "clf = lgb.LGBMClassifier(max_depth=6, learning_rate= 0.20349557055687642, n_estimators = 78, \n",
    "                         num_leaves = 1302, min_child_samples = 27)\n",
    "clf.fit(X_train,y_train)\n",
    "y_pred = clf.predict(X_test)\n",
    "print(classification_report(y_test,y_pred))\n",
    "print(\"Accuracy: \"+ str(accuracy_score(y_test, y_pred)))\n",
    "\n",
    "# Plot the confusion matrix\n",
    "cm=confusion_matrix(y_test,y_pred)\n",
    "f,ax=plt.subplots(figsize=(5,5))\n",
    "sns.heatmap(cm,annot=True,linewidth=0.5,linecolor=\"red\",fmt=\".0f\",ax=ax)\n",
    "plt.xlabel(\"y_pred\")\n",
    "plt.ylabel(\"y_true\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After hyperparameter optimization, the accuracy of LightGBM can be improved from 79.35% to 80.69%."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Save and load the optimized model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Optimized_lightGBM.pkl']"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import joblib\n",
    "# Output a pickle file to save the trained model\n",
    "joblib.dump(clf, 'Optimized_lightGBM.pkl') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# The trained model can be loaded directly for future testing\n",
    "savestkrf = joblib.load('Optimized_lightGBM.pkl') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[9424  287]\n",
      " [4066 8767]]\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.70      0.97      0.81      9711\n",
      "           1       0.97      0.68      0.80     12833\n",
      "\n",
      "    accuracy                           0.81     22544\n",
      "   macro avg       0.83      0.83      0.81     22544\n",
      "weighted avg       0.85      0.81      0.81     22544\n",
      "\n",
      "0.8069109297374024\n"
     ]
    }
   ],
   "source": [
    "y_pred = savestkrf.predict(X_test)\n",
    "print(confusion_matrix(y_test,y_pred))\n",
    "print(classification_report(y_test,y_pred))\n",
    "print(accuracy_score(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, the accuracy is still at a low level. A model with the accuracy of 80% cannot detect attacks accurately."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Find the reason why the accuracy is low"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Read the entire dataset (training+test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"NSL_KDD_binary(train+test).csv\")\n",
    "X = df.iloc[:, :-1].values\n",
    "y = df.iloc[:, -1].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X,y, train_size = 0.1, test_size = 0.9, random_state = 0,shuffle=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.93      0.99      0.96     69135\n",
      "           1       0.99      0.92      0.96     64531\n",
      "\n",
      "    accuracy                           0.96    133666\n",
      "   macro avg       0.96      0.96      0.96    133666\n",
      "weighted avg       0.96      0.96      0.96    133666\n",
      "\n",
      "Accuracy: 0.9604237427618093\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU4AAAE+CAYAAADicfmoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAg2klEQVR4nO3deXwV1fnH8c8TkgBVIWwi2w9RqQruIoK4oCggXUDrWhW0aFyoYqut2mqpa9Xqz11bqlRAK/CjLtQNEaGurIoiIJpSKYkoyiaLQJbn98c94BVzQyadm5t4v29f88rMmTNzzwh5eM6cM3PN3RERkerLyXQDRETqGwVOEZGIFDhFRCJS4BQRiUiBU0QkIgVOEZGIcjPdgCqZaa6USCa4W00OK/1iSeTf2byWe9ToszKpTgfO0s//lekmSA3ktdoTgNy8thluidREWeknmW5CnVenA6eI1DMV5ZluQa1Q4BSR+HhFpltQKxQ4RSQ+FQqcIiKRuDJOEZGIlHGKiESkjFNEJCKNqouIRKSMU0QkIt3jFBGJRqPqIiJRKeMUEYlIGaeISEQaVRcRiShLMk69yFhEJCJlnCISHw0OiYhElCVddQVOEYmPMk4RkWjcNaouIhKNuuoiIhGpqy4iEpEyThGRiPTkkIhIRMo4RUQi0j1OEZGIlHGKiESkjFNEJCIFThGRaLLlySG9Vk5E4lNREX2pBjMrMLOJZvaBmS0ys55m1tzMppjZR+Fns1DXzOxeMysys/fM7JCk8wwJ9T8ysyFJ5Yea2fxwzL1mZlW1R4FTROLjFdGX6rkHeNHd9wEOBBYBVwNT3b0zMDVsA5wIdA5LIfAQgJk1B0YAhwPdgRFbg22oc0HScf2raowCp4jUaWbWFDgaeATA3be4+xpgIDA6VBsNDArrA4ExnjADKDCzNkA/YIq7r3L31cAUoH/Y18TdZ7i7A2OSzlUp3eMUkfikZ3CoE/A58FczOxCYCwwHWrv78lDnU6B1WG8HLEs6vjiUVVVeXEl5Sso4RSQ+Neiqm1mhmc1JWgq3O2sucAjwkLsfDGzg62554mMTmaLXzkUq4xSRONUg43T3kcDIKqoUA8XuPjNsTyQROD8zszbuvjx0t1eE/SVAh6Tj24eyEqD3duXTQ3n7SuqnpIxTROKThsEhd/8UWGZme4eiPsBCYBKwdWR8CPBMWJ8EDA6j6z2AtaFLPxnoa2bNwqBQX2By2PelmfUIo+mDk85VKWWcIhKf9E2AvxR43MzygSXAeSQSvwlmNhRYCpwW6j4PDACKgI2hLu6+ysxuBGaHeje4+6qwfgnwKNAYeCEsKSlwikh80hQ43X0e0K2SXX0qqevAsBTnGQWMqqR8DrBfddujwCki8dFLPkREItKz6iIiESnjFBGJSBmniEhEyjhFRCJSxikiEpECp4hIRF5rj4tnlAKniMRHGaeISEQKnCIiEWXJqLrejiQiEpEyThGJj7rqIiIRaVRdRCQiZZwiIhEpcIqIRJQlo+oKnCISG6/QPU4RkWjUVRcRiUhddRGRiNRVFxGJSF11EZGIFDilOr5ct54Rt95N0ZKlYMaNv/kFjfLzueGP97F5SykNGjTguiuHsX+XvRn1+ESee2kaAOXl5SxZuozXnhtH0ya7VHqeg/bbF4DH/+8Zxj35LDk5ORx9RHeuGDY0k5f8ndewYUOmv/J38hs2JDe3AU8++RzX33AnADfecBU/+ckPKS8v589/HsP9D4yioKApD//lTvbYoyObN23m/MIrWLBgcYavIkP05JBUx613/4leh3fjrpuvpbS0lK82beaK627h4p+dxVE9D+PVN2dx54OP8Oj9t/Ozs07hZ2edAsD012cwZvzTNG2yS8rzAMya+y7TXp/B30c/QH5+PitXr8nUpWaNzZs3c3zf09iwYSO5ubm8Ov0pXnxxGvvssxft27el635H4+60atUCgGuuupR3313AKaeez95778l999xC3/6nZ/gqMiRLMs60vR3JzPYxs6vM7N6wXGVm+6br8zJh3foNzH33fX7yo34A5OXl0WSXnTEz1m/YCMD6DRvZtWWLbx37/Mv/ZMAJx1R5HoDxTz/H0LNPIz8/H4AWzQrSfVkCbAh/fnl5ueTm5eHuXHThYG66+S48ZFWff74SgH33/T7Tpr0BwOLF/6Jjx/bsumvLzDQ80yo8+lIPpSVwmtlVwDjAgFlhMeAJM7s6HZ+ZCSWffEqzgqZce/P/csq5w/jdH+5m41ebuGr4hdz54CP0Oekc7rj/YS6/6NxvHPfVpk28PmMOJ/Q+ssrzAHz8nxLmvvs+Z15wOecO+xXzF2VpF7CW5eTkMGf2SywveY+pU19l1ux32GOP3Tnt1B8z463neXbSWPbaqxMA781fyEmDBgBwWLeD6NixPe3btclk8zPHK6Iv9VC6Ms6hwGHufqu7PxaWW4HuYd93Qll5OYs+LOL0k37AxEcfoHHjRjwydgLjn3qOqy4tZOpTY/n1ZYX87g93f+O46a/P5OADumzrpqc6DyTuhX755Tr+NvIurhh2Plde94dtGY+kT0VFBd0O60vHTt04rNvBdO26Nw0b5rNp02Z69BzAw6P+xsMjE/c9b7v9fpoWNGHO7JcYNuxnvDPvfcqzpMv6Lco4/ysVQNtKytuEfSmZWaGZzTGzOQ+PeSItjYvLbru2pHWrlhzQdR8A+vY+koUfFjHphZc5vncvAPoddxTzF34zS3xh6j8ZcHzvHZ4HoPWuLTn+mF6YGft32RszY/WatbVwdQKwdu2XTP/nG/Tr25vikuU89fTzADz99Avsv3/iztO6des5/4Jf0u2wvpx73mW0atmCJUuWZrLZkmbpCpyXA1PN7AUzGxmWF4GpwPCqDnT3ke7ezd27nT/4zDQ1Lx4tWzRnt11b8e+lxQDMmDuPPXf/H1q1bMHsd+YDMHPuPDp2aLftmHXrNzDnnfkce1TPHZ4H4LijejLr7XcB+Pg/xZSWldGsoGmtXF+2atmyOU2bNgGgUaNGHN/naBYv/heTJr1I72OOAOCYo3vy4UdLAGjatAl5eXkADP3ZT3nt9ZmsW7c+M43PMK+oiLzUR2kZVXf3F83s+yS65lujRgkw293L0/GZmfKbX1zMVdffTmlZKR3atuHG3/yC447qwa33/Jmy8nIa5ucz4teXbas/9Z9vckT3Q/he40Y7PA/AyT/sy7W33MWgsy8iLy+XW669AjOr1WvMNm3atGbUI3fToEEOOTk5TJz4D557/mVef2MWY0ffz/DhF7Bh/UYuvOhXAOy7T2dGjbobd2fhwsVcUHhlhq8gg+pp1zsqq8v3y0q/WFJ3Gycp5bXaE4DcvMru1khdV1b6CbjX6F/nDTedHfl3dqdrH9vhZ5nZx8A6oBwoc/duZtYcGA/sDnwMnObuqy2RWdwDDAA2Aue6+9vhPEOAa8Npb3L30aH8UOBRoDHwPDDcqwiO+rI2EYlPegeHjnX3g9y9W9i+Gpjq7p1J3AbcOmPnRKBzWAqBhwBCoB0BHE6iNzzCzJqFYx4CLkg6rn9VDVHgFJH4VFREX2puIDA6rI8GBiWVj/GEGUCBmbUB+gFT3H2Vu68GpgD9w74m7j4jZJljks5VKQVOEYlP+jJOB14ys7lmVhjKWrv78rD+KdA6rLcDliUdWxzKqiovrqQ8JT1yKSLxqcGE9hAIC5OKRrr7yO2qHenuJWa2KzDFzD74xse6u5nV2piIAqeIxKcGo+ohSG4fKLevUxJ+rjCzp0jco/zMzNq4+/LQ3V4RqpcAHZIObx/KSoDe25VPD+XtK6mfkrrqIhKbdMzjNLOdzGyXretAX+B9YBIwJFQbAjwT1icBgy2hB7A2dOknA33NrFkYFOoLTA77vjSzHmFEfnDSuSqljFNE4pOeeZytgafC/OVc4G9hrvhsYIKZDQWWAqeF+s+TmIpURGI60nkA7r7KzG4EZod6N7j7qrB+CV9PR3ohLCkpcIpIfNIQON19CXBgJeUrgT6VlDswLMW5RgGjKimfA+xX3TYpcIpIfOrp246iUuAUkfhkySOXCpwiEhtX4BQRiUiBU0Qkonr6mrioNI9TRCQiZZwiEh911UVEIlLgFBGJpi6/GD1OCpwiEh9lnCIiESlwiohEownwIiJRKXCKiESUHfPfFThFJD7qqouIRKXAKSISkbrqIiLRqKsuIhKVMk4RkWiUcYqIRKWMU0Qkmiz5rja9yFhEJCplnCISnyzJOBU4RSQ22dJVV+AUkfgocIqIRKOMU0QkIgVOEZGIFDhFRKJyy3QLaoUCp4jEJlsyTk2AF5HYeIVFXqrLzBqY2Ttm9mzY7mRmM82syMzGm1l+KG8YtovC/t2TznFNKF9sZv2SyvuHsiIzu3pHbVHgFJHYeEX0JYLhwKKk7duAu9x9L2A1MDSUDwVWh/K7Qj3MrAtwBtAV6A88GIJxA+AB4ESgC3BmqJuSAqeIxMbdIi/VYWbtgR8AD4dtA44DJoYqo4FBYX1g2Cbs7xPqDwTGuftmd/83UAR0D0uRuy9x9y3AuFA3JQVOEYlNTTJOMys0szlJS2Elp74b+DVfT7FvAaxx97KwXQy0C+vtgGUAYf/aUH9b+XbHpCpPSYNDIhKbKPcstx3jPhIYmWq/mf0QWOHuc82sd40bFyMFThGJjafnPca9gB+b2QCgEdAEuAcoMLPckFW2B0pC/RKgA1BsZrlAU2BlUvlWycekKq+UuuoiEpt0jKq7+zXu3t7ddycxuPOKu58FTANOCdWGAM+E9Ulhm7D/FXf3UH5GGHXvBHQGZgGzgc5hlD4/fMakqtqkjFNEYlOTrvp/4SpgnJndBLwDPBLKHwHGmlkRsIpEIMTdF5jZBGAhUAYMc/dyADP7OTAZaACMcvcFVX2weZpy6ziUfrGk7jZOUsprtScAuXltM9wSqYmy0k+o9nD3dj4+6ITIv7O7z5tS7x43UsYpIrGpw3lYrBQ4RSQ2tdxVzxgFThGJTQ17+PXODkfVzay1mT1iZi+E7S5mNnRHx4lI9knzI5d1RnWmIz1KYrRp653+D4HL09QeEanHKtwiL/VRdQJnS3efQHjUKUw2LU9rq0SkXkrXs+p1TXXucW4wsxaAA5hZDxLPfoqIfIMGh772SxKz6Pc0szeAVnw9W19EZBtNRwrc/W0zOwbYGzBgsbuXpr1lIlLvKOMMzGzwdkWHmBnuPiZNbRKReqq+DvZEVZ2u+mFJ642APsDbgAKniHxDfR3siao6XfVLk7fNrIDEG5JFRL5B9zhT2wB0irshIlL/qasemNk/CFORSMz77AJMSGejRKR+Ulf9a3ckrZcBS929OE3tEZF6TF11Et9jDPze3Y+tpfZ8w9b3Okr9VFb6SaabILVMXXXA3cvNrMLMmrq7nhYSkSqpq/619cB8M5tCYmAIAHe/LG2tCtoVVPmd8FJHlaxZCMBXL96X4ZZITTTuf+mOK2W56gTOJ8OSLEvuZIhIFOqqf63A3e9JLjCz4Wlqj4jUY9mSUVXntXJDKik7N+Z2iMh3QLa8jzNlxmlmZwI/BTqZWfJ3DO9C4is3RUS+QYND8CawHGgJ3JlUvg54L52NEpH6qZ5+E0ZkKQOnuy8FlgI9qzqBmb3l7lXWEZHs4CjjrK5GMZxDRL4DKrJkdCiOwJkl/6tEZEcqlHGKiESTLV316nyv+qVm1qyqKjG2R0TqsYoaLPVRdeZxtgZmm9kEM+tvZtsHynPS0C4RqYcci7zURzsMnO5+LdAZeITExPePzOwWM9sz7H8/rS0UkXpDGWcSd3fg07CUAc2AiWZ2exrbJiL1TDoCp5k1MrNZZvaumS0ws+tDeSczm2lmRWY23szyQ3nDsF0U9u+edK5rQvliM+uXVN4/lBWZ2dU7alN17nEON7O5wO3AG8D+7n4xcCjwk2pct4hkiTR11TcDx7n7gcBBQH8z6wHcBtzl7nsBq4Ghof5QYHUovyvUw8y6AGcAXYH+wINm1iC8d/gB4EQS33BxZqibUnUyzubAye7ez93/b+t3qrt7BfDD6ly1iGSHCou+7IgnrA+beWFx4DhgYigfDQwK6wPDNmF/nzA2MxAY5+6b3f3fQBHQPSxF7r7E3beQ+DLKgVW1qTr3OEeEp4gq27doR8eLSPaowCIv1REyw3nACmAK8C9gjbuXhSrFQLuw3g5YBhD2rwVaJJdvd0yq8pSqdY9TRCRdzKzQzOYkLYXb13H3cnc/CGhPIkPcp7bbmUwT4EUkNjV5jNDdRwIjq1l3jZlNI/EOjQIzyw1ZZXugJFQrAToAxWaWCzQFViaVb5V8TKrySinjFJHYpGlUvZWZFYT1xsAJwCJgGnBKqDYEeCasT+Lr9wifArwSZgZNAs4Io+6dSEyznAXMBjqHUfp8EgNIya/S/BZlnCISm4pvPR8TizbA6DD6nQNMcPdnzWwhMM7MbgLeITHXnPBzrJkVkXh38BkA7r7AzCYAC0lMqxzm7uUAZvZzYDLQABjl7guqapACp4jEJh1v/HH394CDKylfQuJ+5/blm4BTU5zrZuDmSsqfB56vbpsUOEUkNvX1SaCoFDhFJDbVmZf5XaDAKSKx0fs4RUQiypa3mitwikhs1FUXEYlIg0MiIhGpqy4iEpG66iIiEamrLiISkQKniEhErq66iEg02ZJx6rVyIiIRKeMUkdhkS8apwCkisdE8ThGRiDSPU0QkInXVRUQiUuAUEYlI9zhFRCLSPU4RkYjUVRcRiUhddRGRiCqyJHQqcIpIbNRVFxGJKDvyTQVOEYmRMk4RkYg0HUlEJCINDomIRJQdYVMvMhYRiUwZp4jEJlsGh5RxikhsKvDIy46YWQczm2ZmC81sgZkND+XNzWyKmX0UfjYL5WZm95pZkZm9Z2aHJJ1rSKj/kZkNSSo/1Mzmh2PuNbMqh7kUOEUkNl6DpRrKgCvcvQvQAxhmZl2Aq4Gp7t4ZmBq2AU4EOoelEHgIEoEWGAEcDnQHRmwNtqHOBUnH9a+qQQqcIhKbihosO+Luy9397bC+DlgEtAMGAqNDtdHAoLA+EBjjCTOAAjNrA/QDprj7KndfDUwB+od9Tdx9hrs7MCbpXJXSPU4RiU26pyOZ2e7AwcBMoLW7Lw+7PgVah/V2wLKkw4pDWVXlxZWUp6SMU0RiU5OuupkVmtmcpKWwsnOb2c7A34HL3f3Lb3xuIlOstdlQyjhFJDY1GVV395HAyKrqmFkeiaD5uLs/GYo/M7M27r48dLdXhPISoEPS4e1DWQnQe7vy6aG8fSX1U1LGKSKx8Rr8tyNhhPsRYJG7/2/SrknA1pHxIcAzSeWDw+h6D2Bt6NJPBvqaWbMwKNQXmBz2fWlmPcJnDU46V6WUcYpIbNI0j7MXcA4w38zmhbLfALcCE8xsKLAUOC3sex4YABQBG4HzANx9lZndCMwO9W5w91Vh/RLgUaAx8EJYUlLgFJHYpGNwyN1fB1LNq+xTSX0HhqU41yhgVCXlc4D9qtsmBc6Y5eTk8MK0CXy6/DOGnDGMXkcdznU3Xklefh7z5y3kikuvo7y8nJNO/QGXDB+KmbFh/QauueJGFr6/GIA777uR4/sdwxdfrKLPEYMye0HfcSf+/lF2aphPTo6Rm5PD3351OotLPufm8dPZuLmUts134ZbB/di5cT4AH5Z8wU3jp7F+0xZyzHj8ytNomJfLfc++xbOzPuDLjZt5646Ltp1/S2k51z72EouWfU7TnRpx27n9adeiSaYuN+30rLrUyPkXncNHHy4BwMy4+6GbuWTolfQ5YhDFyz7h1DMHArBsaQmn/OBcju91Enf/8U/cdtfvt51jwhNPc9YpF2ai+VnpL5eexISrzuRvvzodgOufeIXLfnQEE6/5KccdsCejX3kbgLLyCn479iV+e3pvnvzNWTx82UnkNkj8Ch3TtROPXXHat8791IwFNPleI/7xu8Gc3fsg7pn0Ru1dWAak48mhukiBM0Zt2ramT9+jeWLM3wFo1ryALVtKWfKvpQC8Ov1NBvz4BADmzJrH2rWJGRVvz36PNm1bbzvPzDfnsmb12lpuvWz1nxVrOHSvtgD02KcDU+cVAfDWB/+hc9uW7N2uFQAFOzWmQU7iV+iATrvRqulO3zrX9Pn/5kfd9wHg+IP2YtaHxSR6kt9N6ZgAXxdlJHCa2XmZ+Nx0u/6Wq7lpxJ1UVCT+OqxauZrc3FwOOKgrAD/4cV/attvtW8edcc7JTHv5tVptqyQYxsUPPsOZt49j4hvvA7DHbs2ZNj/Ra5jyThGfrlkPwNIVazDg4gef4Yzbx/HXl+fu8Pwr1q5nt4JdAMhtkMPOjfJZs2FTei6mDkjHqHpdlKl7nNcDf83QZ6fF1nuS899dSM9eh20rv2Tolfz+lqvIz8/n1WlvUlH+zX9jjziyO2eefTInnXhObTdZgL9e/hNaF+zMqnUbueiBp+nUuhnXn9WH2ya+yl9enM0x+3cir0EDAMorKnhnyXIev/I0GuXncuH9T9Olw64cvneHHXxK9qivGWRUaQucZvZeql18/WhUZccVkngwn6aN27BTw2apqtYp3Q4/mL79e3PcCUfRsGFDdtllJ+79861cduHVnDxgMABHH3sEe+zZcdsx+3b9Pn+893rOOfUiVqtrnhGtC3YGoPku3+PYA/bk/aWfMaTPIfxp2CAAlq5YzWsLPt5W95C92tJs58YAHNmlI4uKP68ycO7adGc+XbOO1s12pqy8gvWbtlCwU6O0XlMm1dcMMqp0dtVbk5hI+qNKlpWpDnL3ke7ezd271ZegCXDrDXfTbb8+9DiwL5cMvZI3XpvJZRdeTYuWzQHIz89j2PChjP3rBADatm/DX8bcw/CLrtl2D1Rq11ebS9mwacu29bc++A97tWnBqnUbAaiocP4yeTan9tofgCP2/R+KPlnJV1tKKSuvYG5RCXvsVvXf0WP268Q/Zn0AwMvzijisc3t28MYyqQfS2VV/FtjZ3edtv8PMpqfxc+uUiy87j+P7HkNOTg5jRo3njddmAvCLX11Es+ZNueWO6wAoKytjwHGJUd0HHv4jPXsdRvMWBcx5fyp33PoA4x57MuVnSM2sXLeRXz78HABlFc6Jh36fXl068vj0eYx/bT4AfQ7cg4E99gWgyfcacc6xB3HWHRMwgyO77M7RXTsBcNczb/DCnMVsKi2l73WjOKlnVy4ecDgn9ezCb8dO4Uc3jKHJ9xpy27lVvq2s3suWrrrV5RG+ds261t3GSUolaxYC8NWL92W4JVITjftfCu41SovP6Xhy5N/ZsUufrHcpuCbAi0hssiXTUeAUkdjU1wntUSlwikhssmVUXYFTRGKTLYNDCpwiEht11UVEIlJXXUQkInXVRUQiqsvzwuOkwCkisdE9ThGRiNRVFxGJSINDIiIRqasuIhKRBodERCLSPU4RkYiy5R6nvuVSRCQiZZwiEhsNDomIRKTBIRGRiJRxiohElC2DQwqcIhKbCnXVRUSiyY6wqelIIhKjCjzysiNmNsrMVpjZ+0llzc1sipl9FH42C+VmZveaWZGZvWdmhyQdMyTU/8jMhiSVH2pm88Mx95rZDr+uWIFTRGKTjsAJPAr0367samCqu3cGpoZtgBOBzmEpBB6CRKAFRgCHA92BEVuDbahzQdJx23/Wtyhwikhs3D3yUo1zvgqs2q54IDA6rI8GBiWVj/GEGUCBmbUB+gFT3H2Vu68GpgD9w74m7j7DE40Zk3SulHSPU0RiU4vTkVq7+/Kw/inQOqy3A5Yl1SsOZVWVF1dSXiVlnCISG6/Bf2ZWaGZzkpbCSJ+ZyBRrdVxKGaeIxKYmTw65+0hgZMTDPjOzNu6+PHS3V4TyEqBDUr32oawE6L1d+fRQ3r6S+lVSxikisUnT4FBlJgFbR8aHAM8klQ8Oo+s9gLWhSz8Z6GtmzcKgUF9gctj3pZn1CKPpg5POlZIyThGJTTqeVTezJ0hkiy3NrJjE6PitwAQzGwosBU4L1Z8HBgBFwEbgvNCuVWZ2IzA71LvB3bcOOF1CYuS+MfBCWKqkwCkisUnH4JC7n5liV59K6jowLMV5RgGjKimfA+wXpU0KnCISm2x5Vl33OEVEIlLGKSKx0Us+REQiypauugKniMRGGaeISETKOEVEIlLGKSISkTJOEZGIlHGKiESkjFNEJCL3ikw3oVYocIpIbPS96iIiEaXj7Uh1kQKniMRGGaeISETKOEVEItJ0JBGRiDQdSUQkomzpqutFxiIiESnjFJHYaFRdRCSibOmqK3CKSGw0qi4iEpEyThGRiHSPU0QkImWcIiIR6R6niEhEenJIRCQiZZwiIhHpHqeISETqqouIRKSMU0QkomwJnFanL9SsDjdO5DvM3WpyWG5+u8i/s2VbSmr0WZlUtwPnd5iZFbr7yEy3Q2pGf37ZTe/jzJzCTDdA/iv688tiCpwiIhEpcIqIRKTAmTm6P1a/6c8vi2lwSEQkImWcIiIRKXBmgJn1N7PFZlZkZldnuj1SfWY2ysxWmNn7mW6LZI4CZy0zswbAA8CJQBfgTDPrktlWSQSPAv0z3QjJLAXO2tcdKHL3Je6+BRgHDMxwm6Sa3P1VYFWm2yGZpcBZ+9oBy5K2i0OZiNQTCpwiIhEpcNa+EqBD0nb7UCYi9YQCZ+2bDXQ2s05mlg+cAUzKcJtEJAIFzlrm7mXAz4HJwCJggrsvyGyrpLrM7AngLWBvMys2s6GZbpPUPj05JCISkTJOEZGIFDhFRCJS4BQRiUiBU0QkIgVOEZGIFDhFRCJS4JQ6xczONbP7M90OkaoocEqtCK/TE/lOUOCUSpnZDWZ2edL2zWY2vJJ6vc3sVTN7Lryc+U9mlhP2rTezO83sXaCnmZ1tZrPMbJ6Z/XlrMDWz88zsQzObBfSqpUsUqTEFTkllFDAYIATCM4DHUtTtDlxK4sXMewInh/KdgJnufiCwEjgd6OXuBwHlwFlm1ga4nkTAPDKcQ6ROy810A6RucvePzWylmR0MtAbecfeVKarPcvclsO1Z7iOBiSSC499DnT7AocBsMwNoDKwADgemu/vn4fjxwPfTc1Ui8VDglKo8DJwL7EYiA01l+xcebN3e5O7lYd2A0e5+TXJFMxv03zdTpHapqy5VeYrE9+scRuJtTql0D6/JyyHRHX+9kjpTgVPMbFcAM2tuZh2BmcAxZtbCzPKAU2O9ApE0UMYpKbn7FjObBqxJyhwrMxu4H9gLmEYi4G5/roVmdi3wUgiwpcAwd59hZr8n8aq2NcC8WC9CJA30WjlJKQS4t4FT3f2jFHV6A1e6+w9rsWkiGaWuulQqfGVxETA1VdAUyVbKOKVazGx/YOx2xZvd/fBMtEckkxQ4RUQiUlddRCQiBU4RkYgUOEVEIlLgFBGJSIFTRCSi/wf62JfhTpNx0AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 360x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# LightGBM algorithm\n",
    "import lightgbm as lgb\n",
    "classifier = lgb.LGBMClassifier()\n",
    "classifier.fit(X_train, y_train)\n",
    "y_pred = classifier.predict(X_test)\n",
    "print(classification_report(y_test,y_pred))\n",
    "print(\"Accuracy: \"+ str(accuracy_score(y_test, y_pred)))\n",
    "\n",
    "# Plot the confusion matrix\n",
    "cm=confusion_matrix(y_test,y_pred)\n",
    "f,ax=plt.subplots(figsize=(5,5))\n",
    "sns.heatmap(cm,annot=True,linewidth=0.5,linecolor=\"red\",fmt=\".0f\",ax=ax)\n",
    "plt.xlabel(\"y_pred\")\n",
    "plt.ylabel(\"y_true\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Monitor how the accuracy changes over time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Record the real-time accuracy based on timestamp/sample index\n",
    "acc = []\n",
    "acc_sum = 0\n",
    "for i in range(0, len(y_test)):\n",
    "    if y_test[i] == y_pred[i]:\n",
    "        acc_sum = acc_sum + 1  \n",
    "    accuracy = acc_sum/(i+1)\n",
    "    acc.append(accuracy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABcQAAAODCAYAAACSRAJgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAADJEElEQVR4nOzdd5hU5fnG8fsBlr70qoKAFDUqiCsoKGIvib1giREV7DX6MxpJAKPGGDX2KKKCvUVRotgBC7GAXbFRFKR3pJf398dzhp2dne3Lnt2d7+e6zvXOnPpM3Z173nmPhRAEAAAAAAAAAEB1VyPuAgAAAAAAAAAAqAgE4gAAAAAAAACAjEAgDgAAAAAAAADICATiAAAAAAAAAICMQCAOAAAAAAAAAMgIBOIAAAAAAAAAgIxAIA4AAIAqwcyCmU2Iuw7kZ2b9o8dnWNy1oHows4HRc2pg3LUAAIDqhUAcAACUWRRaBDP7yczqFrDOzGidWinza5rZYDObaGZLzGyDmS0wsy/MbKSZHZWyfv/yCEaL2o+Z/dbMVpnZGjM7OprXIem2JqY1Ub0fmNndZrZvIceckLLtRjNbambfmtkzZnammTUsxW0JZhZKul2a/RBAlVCa58M6M1toZp9Ez9/DzaxmOR0rtscn6bk/qqKPXV7M7I3oNswqr8cEJWNmB5vZ42Y2w8xWR++fP5rZo2Z2eNz1AQCAzFCr6FUAAACKrb2kyyTdVJyVo1Dqv5IOk7RM0suSZkuqLek3kk6VtKOkl8q/1ELrOlPSCEkrJR0ZQng/ZZXlkm6PLteS1ExSd0nnS7rQzF6X9IcQwvwCDjFa0kxJJilbUidJB0k6UdKNZnZ2COGVcrtB1cdOklbHXUQBhkdtTUlN5M/f0yWdLWmymZ0WQvg+ptoqwkfyx2dR3IWkY2adJB0oKUjaTtLh8vceVAAzy5b0iKRjJK2V9Lak5yVtkNRR0hGSfm9mt4YQroyrTgAAkBkIxAEAQHlZKg+brjazkSGE4gRjp8jD8M8l7RdCWJ680MzqS+pd7pUWwsyukXSjPJg/LITwdZrVloUQhqXZtpOkByUdIulVM9s7hLA2zfajQggTUratK+kKSddJesHMDg4hvFOmG1PNhBC+jbuGghTwfGgt6S75Fx1vmllOCGFBRddWEUIIqyVV2sdH0mD5F1A3Sbpa0jkiEK8QZlZD0rOSDpU0XtLvQwhzUtapI+k8SV0rvkIAAJBpGDIFAACUl9WS/iapsaShxdymT9SOSg3DJQ/ZQgjjy6m+Qpm7Qx6GfyOpTwFheIFCCNMl/VYeDPaQBzzF3XZtCOEGSdfLe8jfUZJjp0oe4iK6/JSZLTKztWY22cx+l7L+BEkPR1cfThkGpEO0zrDoen8zO9XMPjSzX81sZrS80HGko2FzZqbMq21ml0RDjCyNhlGYaWYvmtlBKeumHeLGzBqb2d/N7Lvo9i01s9dSt0+t0cx6mNnLZrYsOu5EM+uTuk1pRb8QOFnSBEntJP05TT3NotqnRsNHLDezt8zskJT1JqiIxydar5aZXWA+hM+K6HZ9amYXRcFkPmbWy8yeNrNfoiFf5prZ62Z2UrR8mKQZ0epnpBx7YLROgY+9mXUxs0ei/a83sznR9S5p1k1+jp1gZh9Ft2FJ9BzetrD7vIDbV0vSQEkr5F84TZF0RGH7Kuo+Kem6pXxtbBkix8wOMx9yabklDY9kZseY2WNm9r35EE+rzGxK9Joq6PGub2Z/Mn8fWBm9hqea2Z3mX+LIzJ6Mjr1fAfs4Plp+d0H3YZJT5GH4j/Jf3MxJXSGEsC6EcIekPxZwvP2j278yel6/bGY7pVmvq5ndFN22hdHj8ZOZjTCz7dKsX6r3AzNra2YPmw+XtcbMPjOzM4p4HRTrtQ4AALY+AnEAAFCe7pE0TdK56cKuNBZHbay9As2stqQnJV0iaZKkfUMIs0qzr6in7C3R1dNKsYtbJK2R1MPMflOaGlJsLx/OooOkRyU9LWkXSS+a2f5J642S9GJ0+UX5ECCJaVnKPq+Q9JCknyXdLWlcGeobJQ//s+RDKtwp6R1Ju8p/PVAoM2sif8yuVu5QNv+RtLek183s3AI2zYm2qytppLy38D6S3jKzbqW9MalCCJvlX3JI0ilmZkm1by8PZ6+WtFDSffLHZyf5LwwGJ+1qlIp4fMwsK7od98iHbXlCPvRPDXlP9dGp9UXHmCQfymKSpFvlQxe1knRBtNoE5X5B83nKsT8r7Pab2Z6SJkv6vaSP5c/vD6Lrk6Pl6Vwg6TH50EL3SPpK0gB5T/s6hR0zjaMktZH0dAhhjfy+rCnprAJqLs59UuJ1y+AE+eO6UrnPkYSbJPWU9KH8MX5EUkP545Xu8W4a1XlTtN5Dkv4taaqkM+XPPUXzJO9Jn07idXVfMepP7OOWEMKqwlYMIaxLM/t3kl6Xf6Fxn6R35UOsTDSzFinrHif/InKW/D39LvkXnIMkfVzIlyDFfj8ws1aS/if/kmWq/D3nU0n3Sro03c5L+FoHAABbWwiBiYmJiYmJialMk3yolNnR5ROi68+nrDMzml8rad7uktZL2iwPa4+TtH0Rx+of7WdCGWtO7GeypDeiyy9JqlfINh2i9WYWse8dovU2ptzeCdH8/kVs/2603pkluP9DAbUGSUNTlh0azX8lZf7AaP7AAo4zLFq+StLuhdynwwrYfmbyfSf/NcHm6DGomWb95mlu54SUefdH8++XZEnzu8gD8nWSOqSpMd/tlId8QdK9JXzuhyLWqSMfKzlI6pjyfNgs6eSU9ZvIg+Y1klqX4vG5K/n+lIe/D0bLjk6av3NU1xJJv0mzv+3SPJ9GFfF6GpY0z+SBYZB0Wsr6A6L530qqkeY2rJC0a8o2T0TLTiru4xNt92q03d7R9WbR82Jm8rFLcZ+UZN18909hr42Ux3uzfPimtO81aebVkIfhQVLvAu7Df6e57Q0lNU66/pV8vO/U12GnqKb3i3Hf14ru6yCpcwkft8Tt3yjpwJRlf4+WXZUyf1tJddLs6xBJmyT9u4DnbbHfD5T7WvpHyvzuSbd1WMqyCSrBa52JiYmJiYlp6070EAcAAOUqhPCcvPfcsWa2TxHrfirvKTo/av8jaaaZLTazF8zsyK1esLSH/ISW30k6LngP0rL6JWprysO30m7fshxq+Um5PZQlSSGE1+S9u3uVcp8joseurII8NF0nD4vyLgxhcb4tkkQ9+38v6VdJ14QQtgwlEUL4Qd7bvLakP6TZ/P0QwqiUeQ/Jw7fS3i9pBe/1mrgtLaPau0vaT9J/QghPpay/TD7sUF1JxxfnGNHwGBdLmifp8hDCpqT9bZL36g/K+6uF8+WB5d9CmuGBQgizi3PsQvSRnxT3fyGEx1P2/bSk9yR1k/fETXVnCOHLlHkPRG2xH5+oZ+7Bkr4LIfwvOvYSSWPlv544NGWTktwnW/v+S3gxhPBqugUhhGlp5m1Wbo/+Lbcv6tk8QNJcSVdG6yVv92vIO3TVv+Vf5gxMOURiPPb7i1F7M/lrUPLzMpTGUyGEt1LmjYjaPM+FEMIvIU0v8xDC65K+Vv7HO6FY7wfRe84p8i/bUt9XP5f30M+jvF/rAACg7DipJgAA2BqukP/8/BZJexW2YgjhGTN7QdL+8mBs96g9RtIxZvaIvOdeKHgv+ZlZf3nvv2Qz04Qe38lDn26S7jSzC0t6rHSHT7pcmn0lti9rHZL0WXI4mmSWfFiR0vioDPVsEUJYYWZjJR0p6TMz+4+8d/yHwYeeKUo3SfXlYdaSNMvfljRE/pxKNTlNPRvMbL6kpsW9DSWQ+pgm7vvGBYwrnfgyJN84yQXoKg8ff5A0JGlklmRrUvaXeG2WZcibwvSM2rcLWP62cl/zqSeQzff4yJ+zUsken0HyHtOjUuaPkgeQg5X39pfkPtna919Cga83M2su6f/kQ4h0ktQgZZXkIUL2lN8X74Qihi6JPCIfWuUc+VAwiWF5BspPovxM8covs2I/F6IhiU6T19g9Wl4zaZX1xT1GAe8H3STVkzQ5hLAyzX7ekz/nkpX3ax0AAJQRgTgAACh3IYT/mdlzkk4wswFRb9DC1t8gHyP2dUkys5rysOohee/eFySNKWEZ/ZX/5J4TlT8YmyfpdElvyXt81jOzs1N7T5bQNlG7SR4clXb7hWWoIWFZAfM3qvTnk5lXyu3SGSDpT5JOlY9JLUlro+fPlcFPTFmQxlE7t4DliflN0ixbVsA2G5U3QCszM6ur3F8KJB7T5lF7cDQVpGExD5PYXxcVflLb5P01idpf0qxXHsr78dkYtcV6fKL3kbOUOyRTslflz+MjzaxNCCHxnE7UUpz7pCTrlkXa11s0fv7HkjrKQ/NH5MO3bIxqu1T+ZV9Ck6gtVr0hhJVm9pik88xs/+AnOE6Mx357CGFtMXazRB5C15aH8/l6tBfDsjS1bYy+9El9Ltwm6TL5c+s1+W1N/OpnoPxXAcU6RiT1/SDxnC7ofSnd/PJ+rQMAgDJiyBQAALC1XCMfX/fv0c/Miy2EsCmE8Iykf0WzDijpwUMIw0IIljL1L2DdWZL6yU++NlDS42ZWlo4DiZNVTgkhbCx0zRRmli0fxkXyE+VVRgX1XE98iVDQfdck345CWBM9Vl0ltZcPgfJe1D5XRB2J4R3aFLC8bcp6cdlHfp/MDyHMjOYlaro0zfM0eTqzmMdI7O+FIvbXMWmbZVFb0IkGyyrux+d38i+XakiabWYhMcnfm9rIH5fkk2sui9ri3CclWbfEr40kBb3eBsnD8OEhhN4hhAtCCENCCMOU98SbCcuitiSPd+LkmuemtCPSrJtP9P73QXT1wBIct8SiIWEukY993i2E8PsQwp+i95dh8qGZympF1LYuYHm6+eX9WgcAAGVEIA4AALaKEMKPku6VBzYXl3I3iZ+kpx3/oTxFPUT3k5/g7GRJz5Y0yJckM6svHzJGkh4vbN0C/J/8J/mfhBCmlmL7skgMrVLaHtKJ3vDtUheYWWfl9q5MK4QwKxpr+lBJP0raJxoSoiDfSVotqXvUWzZV4ouJT4qoe6uJxva+Nrr6RNKiREi4bwl2V9jj86088NwrGtaiOBI1HF7GYxckMc58/wKWb+3HZ3DU/ld+IsTUaVS0/GzLHWOmJPdJSdYt02ujAJ2j9j9plu2XZt5H8mC+n5mlDq2SVgjhC0nvy88J0Vt+voV3SvjelAjPr4zeHwtkZnUKW16ETvLPt6+nDmdiZttFy8vqW3mP892iLy9TpRsPvzSvdQAAsBURiAMAgK3pOnlId63S/BzczE4xs4Oj0DB1WRvlBlqp4wtvFSGERfLe6B/JxzAfEw13USxm1lHSy/ITCX6q4p10LrFtXTP7s/y+Wi8f7qCiJU782L6U238r70F5dNRbU5JkZvXkJ7jMw8xamtmuafbTQP582aiCx/xVCGG9/EuHbEl/S9n3DvLeohuUf7iMChHdB0/JA+GfJd2YWBZCmCwfL/04MzurgO13Tb4fVcjjE/XEvUve6/rO6D5P3V9bM9s5ada/5ffxX1LmJ9bfLunqUnlP5ZI8N96Xf2mxj5mdkLLvE+QB4ffyXwSUKzNrJ+kwed0nhhAGpZnOjI7dSR70SiW7T0qyboleG8U0M2r7pxx3d/kvdPIIISyUPx/bSrol9X3XzBqaWbpg/t/yIU/+I/9y8r4S1vmkfPiSLpJeNLO2qSuYWW0zu1DRWOWlNDNq94mGy0nsu6H8hKxlHi40es95Wv4FxpDkZdHJM/OdwLeUr3UAALAVMYY4AADYakIIS8zsRkk3F7BKb3nwO8/M3pM0I5rfUdJv5T2lX1T6oTN2NLNRBez35xDCX0tZ81IzO0gebB8u6WUzOyrlJHRNkk6OVkt+0rXu8pOn1ZCPT3xGCKGgn+gPjE76KXmY20k+ZEsz+di3Z4UQyj0kLIb/yXtcXxb1zE6MXXxXCKHIYS2ik9DdIekvkj6NTpZaSz5u7pxoSrZttN6Xkr6QnyivkXyoizaS7izgxHXJrpYHqxeZ2Z6SxktqIekk+X17UQhhRiHbl4uk50MN+fAXv5H3Fq0t/4LltOgLl2Snyk8s+aCZXSIfImeZpO0k7SZpF/lzakG0flGPz9/kz8Pz5GNjvy0fQ7mVPIzsK//C5RtJCiF8Y2YXyAPOT83sRflJOZvLT8C4QlEv7hDCr2b2oaR9zexxeZC9SdJLUS/ifEIIwczOkPSGpKej/X8rPzHhMfJfgPyhjOP1F+RseW/2x4oY63qk/HE6R9IbJbxPSrJuSV8bxfGI/Bclt5vZ/tGxu8hfP8/Lx+dPdZH8eXWepP5m9pr8S6eO8l9mHCVpQso2z8qHr9pW0qJo38UWQthsZifKv5g6WtJ0M3tL0lT5c6iD/IvIlvITMZdKCGGemT0l/4XPZ2b2ujy4PljSWvmvf3qUdv9JrpbXe1XUa36S/EuGkyS9In9upz6nS/paBwAAW1MIgYmJiYmJiYmpTJO85+jsApbVkQfdIZpqJS1rJ+lC+Ukzv5MHSOvlofAr8nGka6Tsr3/SvgqaPitGzYn9TChgeX15kBfkvUgbyYOb1GOtlYcYH8h76O5TyDEnpGy7UR6KfCvvdThQUoNS3v8hZV6i1lGF1ZJm/mHy4PXXpDo7RMuGRdf7F1KLyQOjadFj+bP8C5H68h6cM5PWbSLpr/Kg6Bf5GL9zo9pOkWRpbme+xyvazz/kgeC66D59Q9IhhTzuwwqoP0+Nxb3vk6Z18tBwirxX6mGpz+GU7bMl/Tla/1f5cAwz5F/InJP6fCjs8Um6/xMniU2c0PAX+XP4z5Lapalhb3nv3wXR+nPkX+qckLJeZ0lj5T3VN0fHHljU/SoPwB+NHtsNUfuYfJzn1HULfI6piOd00no1ouddkLRbEevWj54v6yW1Kul9UsL7r9ivjWj9gcn3cQH17yzppejYq6Ln0aDC7iv5LzCulX8JtVr+xcQ3km5Pvg9StvlXtL9/Fve1UcB+DpEPHTRD/lxfK2l6NO+wktx+pXk/iO7LG+RDLq2Vf8l2j/xLignK/z5Z4PO2sPcD+ZcDo+UnyV0jD9vPkHRCtL/LyvpaZ2JiYmJiYtp6k4VQ0DlaAAAAAACZzswmyH/F0i2E8EPM5VRaZnaDPPQ+LITwWtz1AACA9AjEAQAAAABpmVkv+RAfr4YQinMC0WrPzLYJIcxJmberfPiU9ZK2DYUP1QMAAGLEGOIAAAAAgDzM7Hz50CBnyofIGRpvRZXKZDP7UdJX8qFqusjPe1FD0rmE4QAAVG70EAcAAAAA5GFmM+UnfZwuH2P7iXgrqjzMbKj85Jkd5GODL5OfR+KWEMKEuOoCAADFQyAOAAAAAAAAAMgIDJkSoxYtWoQOHTrEXQYAAAAAAAAAlMmUKVMWhRBaxl1HUQjEY9ShQwdNnjw57jIAAAAAAAAAoEzM7Ke4ayiOGnEXAAAAAAAAAABARSAQBwAAAAAAAABkBAJxAAAAAAAAAEBGIBAHAAAAAAAAAGQEAnEAAAAAAAAAQEYgEAcAAAAAAAAAZAQCcQAAAAAAAABARiAQBwAAAAAAAABkhFpxFwAAAAAAAIB4rFu3TkuWLNHKlSu1adOmuMsBUEnUrFlT2dnZatasmerUqRN3OeWKQBwAAAAAACADrVu3Tj///LOaNm2qDh06KCsrS2YWd1kAYhZC0IYNG7RixQr9/PPPat++fbUKxRkyBQAAAAAAIAMtWbJETZs2VYsWLVS7dm3CcACSJDNT7dq11aJFCzVt2lRLliyJu6RyRSAOAAAAAACQgVauXKlGjRrFXQaASqxRo0ZauXJl3GWUKwJxAAAAAACADLRp0yZlZWXFXQaASiwrK6vanV+AQBwAAAAAACBDMUwKgMJUx/cIAnEAAAAAAAAAQEYgEAcAAAAAAAAAZAQCcQAAAAAAAABARiAQBwAAAAAAAIrBzNS/f/8y76d///7VcmxmoCogEAcAAAAAAEDGMrMqEU4XN4z//vvv9cc//lE9e/ZUs2bNlJWVpWbNmql379668sorNWXKlHzbDBs2bMv9kJhq1Kihxo0bq0+fPrrnnnu0cePGfNslgn0z00MPPVRgTcOHD9+y3sCBA0tys7e44YYbtuzju+++K9U+AEmqFXcBAAAAAAAAQFUwdepU1a9fP+4y0goh6LrrrtN1112nzZs3q2fPnhowYICaNWumlStX6osvvtBdd92lW2+9VXfffbcuvPDCfPvYb7/9toTuGzdu1KxZs/TSSy/poosu0qRJk/T444+nPXatWrU0cuRInXXWWfmWbd68WQ899JBq1aqVNlQv7m0bOXKkzEwhBD3wwAO65ZZbSrUvgEAcAAAAAAAAKIYdd9wx7hIKdN1112nYsGFq166dnnzySfXt2zffOgsWLNDtt9+u5cuXp91H//79NWzYsDzzfv75Z+2888564okndMMNN6hDhw75tvvd736nMWPG6Ouvv9ZvfvObPMtee+01/fzzzzr22GP1wgsvlOq2vf7665o5c6YGDhyoV199VaNHj9aNN96o2rVrl2p/yGwMmQIAAAAAAAAUQ0HDlsydO1dnnnmmWrVqpXr16qlHjx4aPXq0JkyYIDPLFzInbNy4UTfeeKO6dOmiOnXqqF27dvrTn/6k9evXb1ln1KhRW4Z0mThxYp5hTRL7nT59uq6//nrVrl1b48aNSxuGS1KrVq1044036qqrrir2bW7fvr26desmSVq4cGHadQYNGiRJeuCBB/Ite+CBB1S/fn2ddtppxT5mun1I0uDBg3Xaaadp0aJFhYbrs2fP1iWXXKIuXbqoXr16atasmXr16qW//e1vpV63sCFrBg4cKDPTzJkzt8ybOXPmliFivv/+ew0YMECtWrVSjRo1NGHCBEnSlClTdOmll6p79+5q1qyZ6tatqy5duuiKK67Q0qVLC7x9Tz/9tA488MAt23To0EGnnHKKJk+eLEm6//77ZWYaPnx42u3nzZunrKws7brrrgUeozqjhzgAAAAAAABQSgsWLNDee++tn376Sf369VOfPn00b948XXDBBTrkkEMK3fbUU0/Vu+++q8MPP1yNGjXSK6+8optvvlkLFizQww8/LEnq0aOHhg4dquHDh2v77bfPMwZ3IqB9+OGHtXHjRp166qn5eminU6tW8SPBWbNm6bvvvlN2dvaWYDxVt27d1K9fPz322GP6xz/+oTp16kjy4HXs2LE67bTT1Lhx42IfM9n8+fP10ksvqWvXrurTp48aNWqkW2+9VSNGjNCAAQPyrT958mQdeuihWrJkifr166fjjjtOq1ev1jfffKNhw4bpL3/5S6nWLa1p06apd+/e6tq1q0477TStWbNGjRo1kuRB/wsvvKD99ttPBx10kDZv3qwpU6botttu07hx4/Thhx8qOzt7y75CCDrzzDM1evRotWjRQscdd5xatmyp2bNna/z48erWrZtycnJ02mmn6aqrrtKDDz6oIUOGqGbNmnlqeuihh7Rx40ade+65Zb59VRGBOAAAAAAAAPK47DLps8/irqJwPXpIt98edxXSNddco59++klXXXWV/vGPf2yZf9lll6lXr16Fbjtt2jR9/fXXatasmSQ/cWT37t31yCOP6O9//7vatGmjHj16qEePHho+fLg6dOiQtrf5+++/L0k64IADynRbJkyYsGX/Gzdu1C+//KKXXnpJderU0QMPPLAlyE1n8ODBOv300/X888/rlFNOkeS92zdu3KjBgwdrzZo1parp4Ycf1oYNG7Z8EbDLLrtojz320Pjx4/Xjjz+qc+fOW9Zdv369TjzxRC1ZskSPP/64Tj311Dz7mj17dqnWLYv33ntP11xzjW688cZ8y6655hrdc889+QLrBx98UIMGDdK9996rP/3pT1vmP/DAAxo9erT23HNPvfHGG3m+ZNi0aZMWLFggSWrYsKFOP/103XPPPRo3bpx+97vfbVkvMR57/fr1dfrpp5fLbaxqGDIFAAAAAAAAKIX169frySefVOPGjTVkyJA8y7p3764//OEPhW7/j3/8Y0sYLkkNGjTQaaedps2bN28Z/qI45s2bJ0nadttt8y2bOXOmhg0blme6vYBvEiZOnKjhw4dr+PDhuuGGGzRq1CitWLFCJ510UpHh/gknnKCmTZtuGd4kEbzutNNOBQ7hUpTEPmrUqJHnvhw4cOCWk2smGzt2rGbOnKmjjjoqX8AtSdttt12p1i2L1q1ba+jQoWmXbb/99vnCcEk666yz1KhRI7322mt55t91112SfEiU1B73NWvWVNu2bbdcP//887esm+z111/XjBkzNGDAgFL32q/q6CEOAAAAAACAPCpDz+uq4LvvvtOaNWuUk5OTZ2iLhH322UcjR44scPucnJx889q1aydJhY4hXRIzZ87MN5b09ttvr8suuyzfukOHDt3SQ3zz5s2aO3euxowZoyuuuEJjxozRRx99tKW+VHXr1tXvf/973X333frxxx/1008/adq0abrttttKXfvbb7+tadOm6dBDD80T9p966qm64oorNGrUKF1//fXKysqSJH3wwQeSpMMPP7zIfZdk3bLo3r37liFkUm3YsEH333+/nnrqKX3zzTdavny5Nm/evGX5L7/8suXyqlWr9NVXX6l169bafffdizzub37zG/Xr10/jxo3TrFmztjxuI0aMkCSdd955ZblZVRo9xAEAAAAAAIBSWL58uSTvBZxOQfMTmjRpkm9eYnzvTZs2FbuONm3aSJLmzJmTb1n//v0VQlAIQRs2bCj2PmvUqKFtt91WF154oa688krNmzdP119/faHbDB48WCEEPfjgg3rggQdUp06dInvJFyYR3iaPmy5JzZo105FHHqkFCxboxRdf3DJ/2bJlktL3lE9VknXLIvHYpDNgwABdfPHFmjt3ro4++mhdddVVGjp0qIYOHarGjRtr3bp1Zar3ggsu0KZNm7Z8KTNv3jy99NJL6tGjR5E9/qszeogDAAAAAAAApZAYU3v+/Plplxc0v7z17dtX48eP11tvvaWzzjqr3Pffu3dvSdJHH31U6Hq77rqr9tprLz344INavny5jj/+eDVv3rxUx1y4cKHGjBkjSTrllFO2jEueasSIETrhhBMk5X7BkNyzuiAlWVeSzEwbN25MuywRVhe0XTqTJ0/WCy+8oIMOOkjjxo3Lc6LTzZs36+abby5TvZJ03HHHqXXr1nrwwQf117/+NeNPpplAD3EAAAAAAACgFHbccUfVq1dPX3zxhVauXJlv+XvvvVdux6pRo0aBvcYHDhyoWrVq6bnnntPUqVPL7ZgJieFbkofzKMjgwYO1cOFCrV+/XoMHDy71MUePHq3169drjz320Nlnn512atmypd58803NmDFDkrTXXntJksaNG1fk/kuyriQ1bdpUs2bNyjd/06ZN+qwUZ6D98ccfJUlHHXVUnjBc8i8eUk9C2qBBA+2yyy6aP3++Pv3002IdIysrS4MGDdIvv/yisWPHauTIkWrYsKFOO+20EtdbnRCIAwAAAAAAAKVQu3ZtDRgwQMuXL883nMjnn3+uRx55pNyO1bx587SBrCTtsMMOGjJkiNavX6/DDz9ckyZNSrteYT2ZC7Ju3Trde++9knz4laKcfPLJeuGFF/Tiiy8Wa/2CJE6Yee+992rkyJFpp3PPPXfLiTcl6cgjj1SHDh300ksv6cknn8y3z9mzZ2+5XJJ1JalXr176+eef9frrr+eZf/311+unn34q8e3r0KGDJGnChAl55i9YsEAXXnhh2m0uueQSSdK55567ZbiehMSY76nOOecc1axZUxdddJFmzJihU089Ne1495kk9iFTzOwESftJ6iGpu6RsSY+HEH5fin1tJ+k6SYdJai5prqQxkoaHENKeicDMdpY0TFJ/SY0k/STpKUk3hRDWFLBNH0lDJO0lqZ6kHyQ9JOmuEELxB3gCAAAAAABApZA6TnWye++9V/Xr10+77KabbtLbb7+tm2++WR9++KH69OmjuXPn6plnntERRxyhMWPGqEaNsvdJPfDAA/XUU0/pyCOPVM+ePZWVlaV+/fqpX79+kqS//vWvCiHob3/7m/r27as99thDvXr1UrNmzbRs2TLNnDlTb775piRt2SbVhAkTtpxUM4SguXPnaty4cZo9e7Y6deqka6+9tsg669evr2OOOaZMt3XChAn6/vvvteuuuxY61vXZZ5+tG264QQ8//LCGDx+u2rVr69lnn9UhhxyiU089Vffff7/22msvrV27VlOnTtVbb721ZdiTkqwrSVdeeaVee+01HX300RowYICaNWumSZMmacaMGerfv3++YLsoe+65p/r27avnn39effr00T777KP58+dr3Lhx6tatm7bZZpt82wwaNEjvvvuuHn30UXXp0kVHH320WrZsqTlz5ujtt9/WWWedteXxS2jfvr1++9vf6qWXXpKkjB8uRaoEgbg8WO4u6VdJsyXtWJqdmNkOkiZJaiXpRUnfSuol6VJJh5lZ3xDC4pRtekt6W1KWpOckzZJ0gKS/SjrQzA4MIaxL2eZoSf+RtFbS05KWSDpS0r8k9ZV0YmnqBwAAAAAAQHxGjx5d4LLbb7+9wEC8devWmjRpkv785z/rlVde0Ycffqhu3brp3nvvVYMGDTRmzJgtY42XxR133CEz01tvvaVXXnlFmzdv1tChQ7eE22amYcOG6ZRTTtF9992n8ePH64knntCqVauUnZ2tHXbYQeeff75OP/109ezZM+0xJk6cqIkTJ265Xr9+fXXu3FlnnHGGrrzyyrQnAd0aEr3DBw0aVOh6HTp00EEHHaQ33nhDY8eO1bHHHqucnBx99tlnuummmzRu3DhNmjRJ2dnZ6ty5s6677ro825dk3QMPPFBjxozRddddp6eeekoNGjTQwQcfrKefflpDhw4t8W2sWbOmXnrpJQ0ZMkSvvPKK7rzzTm277bYaNGiQhgwZop133jnfNmamRx55RIceeqhGjBihZ555RuvWrVPbtm2177776qijjkp7rLPOOksvvfSScnJyCnzsM4mFEOItwGx/eRD+o7yn+HiVooe4mb0m6RBJl4QQ7kqaf5ukyyXdH0I4L2l+TUlfStpJ0tEhhJei+TUkPSPpeEnXhBBuStqmUVRnY0l9QwiTo/l15cH63pJOCSE8VZyac3JywuTJk0tyMwEAAAAAAMrF1KlTtdNOO8VdRrV27bXX6sYbb9Srr76qQw89NO5ykKGGDRum4cOHa+TIkTr77LNLvH1x3yvMbEoIIac0NVak2McQDyGMDyH8EMqQzEe9ww+RNFPSPSmLh0paJel0M2uQNH8/eRj+TiIMj+rZLOmq6Op5lvdUsCdIainpqUQYHm2zVt7TXZLOL+3tAAAAAAAAQNUzZ86cfPO+/PJL3XnnnWrWrJn222+/GKoCpJUrV+q+++5Ts2bNdMopp8RdTqVQGYZMKQ/7R+3rUaC9RQhhpZm9Lw/M95L0VrTogKh9NXVnIYTpZva9pK6SOkmaVtQ2kt6RtFpSHzOrkzrUSqbbuFF64w2/3KmT1K1bvPUAAAAAAACUl5ycHHXu3Fm77LKLGjRooB9++EEvv/yyNm/erPvvv19169aNu0RkmJdfflmffPKJxo4dq/nz5+uWW24pcNifTFNdAvFEvPp9Act/kAfiXZUbiBdnm67RlAjEC9wmhLDRzGZI+o08RJ+abqdmdo6kcyQf1D5TrFkjHXGEX27dWpo3L956AAAAAAAAysu5556rMWPG6Mknn9TKlSvVpEkTHXroobryyivVv3//uMtDBnr22Wc1evRotW7dWtdcc40uv/zyuEuqNKpLIN44apcXsDwxv0kM2+QRQhghaYTkY4gXtF51U7++9MEH0h13SGPHxl0NAAAAAABA+Rk6dGipTqwIbC2jRo3SqFGj4i6jUop9DHFkhpo1pd69pbZt464EAAAAAAAAQKaqLoF4omd24wKWJ+Yvi2EbAAAAAAAAAEAlUF0C8e+itmsBy7tEbfLY3+W6jZnVktRR0kZJ0wsrFgAAAAAAoDIIIWNGcwVQCtXxPaK6BOLjo/YQM8tzm8wsW1JfSaslfZC06O2oPSx1Z2bWSR56/6S84XaB20jqJ6m+pEkhhHUlvQEAAAAAAAAVqWbNmtqwYUPcZQCoxDZs2KCaNWvGXUa5qlKBuJllmdmOZrZD8vwQwjRJr0vqIOnClM2GS2og6dEQwqqk+RMlTZXUz8yOSjpGDUn/iK7eF/J+DfKcpEWSTjaznKRt6kq6Prr671LePAAAAAAAgAqTnZ2tFStWxF0GgEpsxYoVys7OjruMclUr7gLM7BhJx0RX20Tt3mY2Krq8KIRwZXR5W3mI/ZM8/E52gaRJku40swOj9XpL2l8+7Mm1ySuHEDaZ2ZnyXt/Pmdlzkn6WdKCkHEnvS/pXyjYrzGywPBifYGZPSVoi6ShJ3aL5T5f0PgAAAAAAAKhozZo1088//yxJatSokbKysmRmMVcFIG4hBG3YsEErVqzQ0qVL1b59+7hLKlexB+KSekg6I2Vep2iSPPy+UkUIIUyLem1fJx/S5AhJcyXdIWl4CGFpmm0+NLM95b3ID5GUHR3vOkk3pRv6JIQwxsz2kwfsx0uqK+lHSX+UdGeojgPrAAAAAACAaqdOnTpq3769lixZopkzZ2rTpk1xlwSgkqhZs6ays7PVvn171alTJ+5yylXsgXgIYZikYcVcd6akAr+qDCHMknRmCY//jaQTS7jN+/LAHQAAAAAAoMqqU6eO2rZtq7Zt28ZdCgBUiCo1hjgAAAAAAAAAAKVFIA4AAAAAAAAAyAgE4gAAAAAAAACAjEAgDgAAAAAAAADICATiAAAAAAAAAICMQCAOAAAAAAAAAMgIBOIAAAAAAAAAgIxAIA4AAAAAAAAAyAgE4gAAAAAAAACAjEAgDgAAAAAAAADICATiAAAAAAAAKNrMmZKZNHBgxR972DA/9oQJFX/sglRETR06+FRRJkzw2zRsWMUdE6hgBOIAAAAAAAAAgIxAIA4AAAAAAAAAyAgE4gAAAAAAACjcsGFSx45+efRoH1YjMY0alXfd116TjjhCatFCqlNH2mEH6f/+T1q2LP9+v/hCOuUUHxakTh2pZUupZ0/pssukDRt8nQ4dpOHD/fL+++c9dlFC8Hr79PF9160rtWsnHXqo9PTT+defPVu66CKpUyevp3lz6aijpI8/zrve1q4pMXTJTz/5lLz/5CFrxoyRfv97qWtXqUEDn/bYQ7rzTmnz5vzHHjjQ9zF9unTXXdJuu0n16kn9+/uy/ff39YYPz3vMxLAw69f7vnv2lJo2lerX9/vi6KOlN98s+rYDlUCtuAsAAAAAAABAJde/vwfad9whde8uHXNM7rIePXIvDx/u4XmzZtLvfie1auWh9y23SK+8Iv3vf1KjRr7uF19IvXt74HrUUR64r1gh/fijdO+90vXXS1lZHo6PGSNNnCidcUbJxtS+9lrp73/3fZ90ktS4sTR3rgfczz4rDRiQu+4nn0iHHCItWeLh9HHHSYsW+bH32Ud64QUP+qWtX1OHDtLQodLtt+ceL939ffXVUo0afj9uu620fLn09tvSpZf6/h59NH0Nl14qvfuu9Nvf+m2qWVPac09fNnq0tN9+/pgnJG7fwIHSk09Ku+wi/eEPHqbPmSO995706qvSQQcV/34AYmIhhLhryFg5OTlh8uTJcZdRoa64QhoxQlq5Mu5KAAAAAABAicyc6SHuGWfk7xUuSePHSwccIO29t4ffTZrkLhs1SjrzTA92//Uvn3fFFdJtt3mwfPTRefe1dKkHxTWiwQ2GDfOwffz4vEFtUZo399D2+++9N3OyRYu8F7skbdwo7bij9xB/7TUPhBPmzPGwePNmvw/q1KmYmqTcIHrmzPT7mjbNe+An27zZ7+tHHpE++MDD8oSBAz3w3mYbD7ETvf4TJkzwXuJDh+Y/seby5d4rvGdP6cMPPURPtnix3zZkLDObEkLIibuOojBkCgAAAAAAAMruzju9feCBvGG45EFsjx7S44/n365evfzzmjbNDcPLKisrf3gr5Q2eX37Zw+WLL84bhkseHl91lTRvnvTWWxVXU3GkhuGS32+XXuqXX3st/XZXXZU/DC+KmQ/3UqdO+seGMBxVBEOmAAAAAAAAoOz+9z8Pep991qdU69dLCxfm9iQeMMCHYDnmGOmEE3y4jb5904e8pXXaaT5W9s47+/Ak++3nPdgbN85fu+Tjdaf2jJakH37wdurU3GFTtnZNxbF4sfTPf3qP/OnTpVWr8i7/5Zf02/XqVfJjNWokHXmkNHasf7lx/PHSvvt6D/TUnu5AJUYgDgAAAAAAgLJbvNiHHkmcbLIgv/7qgXivXj6O9Q03SM89lzvedbduPmTHKaeUvaZ//ctPkPnww9JNN/lUq5aH2rfeKnXunFu7lD7IT629omoqyrJlPpTLjBl+X/7hDz52e61aueO9r1uXfts2bUpX+9NPS//4h/TEE/4YSX5S0BNO8HHiW7cu3X6BCkQgDgAAAAAAgLJr3NjHr16ypPjb7L239N//enA7ZYqfmPGuu6RTT5Vatiz7SRpr1vRxyy+7TFqwwMfNfuopD76//tqnOnVye2e/+KKf4HNrKm5NRRk50sPwdON9/+9/HogXxKx0tder58caNkyaNUt65x0fH/6xx3yc83ffLd1+gQrEGOIAAAAAAAAoWmLM602b0i/fay8/GebXX5d833XqSH36SNddlzsW+YsvFv/YxdGqlXTccdIzz/jJP6dNk776Krd2qWSB7tauKXGMgvb/44/eHn98/mUTJ5aunpLcpnbtfPiX117zXu3vvZfb0x6oxAjEAQAAAAAAULSmTb1n8c8/p19++eXeDh4szZmTf/mqVdIHH+RenzRJWrMm/3rz53ubPC514oSNBR07nXXrpPffzz9/w4bcXuyJYxx9tI9dfs89Ph53Ov/7n7R6dcXVlDjGwoXp76cOHbydMCHv/E8/lf7+9+LXlKyw27RwofTll/nnr1rlQ8nUqiXVrl264wIViCFTAAAAAAAAULSGDf0Eiu++6z2Du3b1HsVHHSXttpt04IE+HvY110hduviY2B07elj600/ea3mffXxYFEm6+Wbp7bf9xIwdO/r+v/5aGjfOw/dzzsk99v77SzVq+L6/+sqXS9KQIQXXu2aNH69zZ2mPPaTtt5fWrpXeeMNPjnnUUdJOO/m6WVnS889Lhx4q/fa33lu9Rw8Pp2fNkj7+2E9aOXdubmC9tWuS/D79+GPpsMOkfv28J3337n5yyz/8wU+oedll0vjxfp//8IMPQXPccT7ed0l16yZtu60P4ZKV5fWZSaef7r3/d99d2nVXf7zbtZNWrPDjzZsnXXKJlJ1d8mMCFcxCCHHXkLFycnLC5MmT4y6jQl1xhTRihLRyZdyVAAAAAACAEvvxR+8JPmmSB6Qh+MkhBw7MXee993zYk/fekxYt8vG5t93WhwQ59VQpJ8fXe/116cknpQ8/lH75xU/Iud12HkpfcYWHsckee8xP3Pjddx4iS378gmzY4CewHD/eg/YFCzyw3WEHr/ess/L3aF6wQLrtNg95p0/3wLttWw+hjzlGOvlk7wldUTWtWiVdeaU0dqyHzps2SWec4eN2S9I330hXX+0971etknbcUTr/fB97vWPHvOtKfozRo33s8UQP81Qff+z7nDzZA5wQvN4ePfxxnTDBb++iRX4Sz27dpHPP9fumtGOTo1owsykhhJy46ygKgXiMCMQBAAAAAAAAVAdVJRBnDHEAAAAAAAAAQEYgEAcAAAAAAAAAZAQCcVQ4RukBAAAAAKAKmDZNuuACqVEjH0u7USO/Pm1a3JUBQKkRiAMAAAAAACCvceOk3XaTRo7MPbHiypV+fbfdfDkAVEEE4qhQnGwYAAAAAIBKbto06YQTpNWrpQ0b8i7bsMHnn3ACPcUBVEkE4gAAAAAAAMh16635g/BUGzZI//pXxdQDAOWoVtwFACg/s2ZJt9wiffih9OabUsOGcVdUuI0bpc8+k3baSWrQIO5q0ps7V5o0SdpjD6lDh/zLlyzx5e+9J737rrTLLtLvfif973/S++9LbdtKAwdKH3zg6zVsKD3/fEXfCqDyCEFavtyn9u193ooV0qZNUrNmpdvnqlXS0qXSttv6L5FWr5bmzPFp2TJp332l7GypFv/1AABQbWze7H/zf/1VWrPG/69Ys8avr1rly9q3l2rXlurVK/vxQpDWrcvd/6pVuZfTzcvK8v9tVq+WGjeW6tTxZQ0a+GeCli2l7t1zt9u82T87JO8rMdWsKbVo4ZdbtvT/dxK3MXXd5ClxrOR5zZpJu+7qtyV1H4nLa9ZITUd3Ua0NQ7VKDbRKDbRG9TRII9VfE3PvlA0bpEcfle6+u+x3MABUID4aAtXA999L//iH9MgjHjJL0kknSX/9q7TXXvHWlmrjRmn8eOmZZ6QXXpAWL5b69ZPOPVc69VT/5+vNN6WxY33+73+/deuZM0eaPFnaf38PzObOlSZOlCZM8Om773LX7d3b79PFiz0Af+896ZtvfFlWlv8/OGmSNGKE/9O6aZMve/ppP//M5s1+/Te/ke69V+rbV/r6a+mTT/z4220nffmlf6HxySfSscdKrVr55f32k7bfXvr8c2nKFKlHD2nvvbfufYPKIQRp3jxp+nRpxgxvv/9eysmRmjaVZs70afNmqX9/accdpdatpZ9+8vnbbit16uT72GOP8vlAmGzlSn/ddOrkgfOKFf7l3KxZ0uzZedvE5V9/zd2+YcPc67vuKh11lPTLL77ezz/76+13v/N506f7B7uePf36L7/4a3j58tz9NWniIXg63btL55/vbc2aPvTlvHm+j+xsqUsXvy2JMH3RImmfffw2desmNW9evvcdAKBwiQAyK8vft+O2aZP/zVq50v82JNrE5Ro1/G9wy5Yewq5c6X+HE8Mul3ZautSP27On/2/YsKH/D5CVJW2zjf8tO+AAD0wT9SWmX3/1v4s1avi6v/7qf+8S6xXUrlwpzZ/v509s1crnJ6Zly3xq1sw/a6xfn3d5w4Y+1avnf5eTl61c6fvcccfcgHjVKl8vcT15WrvW//7WqJF/2erVxX/sEgF07dpS58654XmzZv7cSg22Cwq9E//PV1dZWVL9+n7frF9/uWpqYxSHr9JcbaPH9XsFpYyDmvyPHQBUERZCiLuGjJWTkxMmT54cdxkV6sorpfvu429meZkxQxo2THrsMf/nbvBgqWNH6Y9/9OW9evmyLl3K/9jr1vk/remE4MHwI49IL77o4W/jxh6CP/+8h0wNG3rw9cQTudsdfrj0zjv+D2fCnnv6vLp1y6fuVat8f6+9Jr3+ujR1au6yHXeUvv3WL2dneyDfv7/0z39KCxbk3U+jRh5o77OPT3vuKX3xhe+7d28PK2fOlF55xUPInBwP9jp2zN1HgwZ5b2u9ev7PZ0ESoXvC0UdLBx0kffyx7+fii/05MWCAf3j45BMPzz/+2C+fc46H7qeemnc8/82b/TXZqFFJ783MFULh50TYtMnD3B9/zJ2WL5eOOcaHWaxVy385kPhlxMqV/tglAu9EO326P48Ke16YeT0lcffd0oUX+naLF/vzu1s3/7C5YIGH6anTlCn+BdXKlX7bZs3yNjl8zs725an1tWkjtWvnz7927fwD6ejRHkxvt51v8+CDvn7iA/t22/kvKyS/vO22/mVR8vVtt/V127b1/f3mN7nzEtN//+vvR+Xx5/bII6XLL5d2392D87lz/Zi1avnl1av99b5okQfqdepIXbt6oDB3rtfdpk3efa5d68vq1vXbURYbN0oLF3owUreuv6dx7g4gs23a5GFt48b+/pr4PyIry9v163N/tbNsWe7l1GnWLP8ift06aYcdfN6qVf4+k1hnxYq8bbrLS5d6YLzffrlB8vLl/rdo9Wp/T02sm5gSNbds6b/WO+00v37qqbmBZceO/rcyuYZly3z9xH1Q2DRzptSnT96QO13onfx/W3mqVcv/hmZn+/9jicuJ6dtv/W9i27b+N6O81a7tx2nYMLfNyvJjdu3qNSUv++yz3NC8devc5Q0b+uO1dKm3ixf7/7eJZdnZPv/bb32dxP7WrPHHLTs7t0dzYlq2zP9WJq6nLk/MS4TWyfNWrcr9P2fePH/+Jv4/q1vXH9N16/xyYpvE/tO1xZ2XaH/5xR/bWrX8/4L69f2+njvXQ/hJk/z+SOxn8WL/W564npjq1fP/JRIdbpYs8Xmp6zVo4MdIvjx7tv8/krheq5Z/LlizJv02ifeGEKQNjZqr9q9LtjxPTtLTelYnaYp6qqc+zX0CNWqUt3cCgIxmZlNCCDlx11EUAvEYEYijtObOla6/XnrgAf/H6MILpauu8t4ba9dKzz0nnX66r7vPPtI993hPSMn/6atVq3S9bFav9lB75EgfDuTllz1Q/ukn6fbb/cPEo4/6NG1a/gC3QQMPwU86STr0UP9H7uWXPTh/5hnv1XLiid4z+qWX8v7ybuxY/yCWrofmkiV+gvOxY/2D1KmnehD9/vvS1Vd7IPT66x6Cv/uuf/irW9c/jO23n/TnP/t+fvtbD8D79/ce2InhFULw0PiGG7zGffbx45TmPpw+3R+Pd9/1HjV77eWviXXr/INYr14eqE+c6L3Pe/Xy+2bxYr+8557+T+yNNxb/mKmPQ+fO0pAh0qef+r7efdfnH3KIh31ZWf6Ba+edS3bbVq70Hu4//+yPYUFfmFQly5Z5b+zE9N133n72mT+vpk718DoRev/wg7czZhQ95KLkj+mMGR5iJsvO9h7XnTr5h/zkyx06+HNjyRK/3K6d3/dffOHvDd995/M7dPDX7JQp/uuCp57y10DCzjv7azf5w32dOv5cTNa4cd7POM2aea+q5Onxx722du3yBt/t2vlzqXbtou+Ldes8SG7dOv9rL/FaS/zqojSvvU2b/BcZH3/sj90773ios802fh98/rnfhkSQ3ratvw6zs/0XOMm/FimrXr1ye9Ynf6Fw7LHSmWf6/HnzckP3uXP9A+2cOd7DvWtXX548zZ3rz6PUf+tOPNHfr7Ky/EuPxPrz5/vzd999/UNxIuiaPz93+bx5/iH5mms8TOjd2x+fkkr08Ex8sZmoMRHWb9rkNSSG+SrqCydkrhD8fa1+/dwvA1es8PfDpUt9WrzYvyxu1sz/3ifmZ2d7D9TE9dq1/TVR2W3YkDesTvTMTZ0KWpb8JWX9+rk9aps39/e+tWvLt94aNTwba9TI/34k2sTlhx/2xyJ5WaNG0ltv+eORuJ687Mcf/X+8xo39vam81Kzp+1wS5X21auUeOzmYLmhe6rJVq3K/PKhZ0+f9+KPf76kBd+pUp07x3/cS75Fr1uSGmu+/7493Yn+J8Dlxed06f97XrOl/T5LD7YYNi/d3ujpK/T8DKS64wD/4Rf/UfqXfaFd9pf/TzbpZf/J1srK81w1DpgCIEIijSATiSGfBAv/AVru2/5M2YYKHs02b+v3297/7eUs2bJAGDfJgc9tt8+/n/vul887Lvf7mm9Krr/r9f+aZ0p13FlzD4sXSqFEewB17rId8993nH2ISPXtSP5B07uz/9Jv58B9/+IN03HHe1qzpIfXhh6cfrmHDBg8FO3f2D1IJU6fmDWUbNpQOPth7ae6zj384euklH7okEZQVZpddPIg/9FDfPlHLqlX+QaQqjS/8zTce0OXk+HPlscc8fLz+ev9FQE6O91Tt2dND/Fde8bBs4MDcfdSv78+t2rX9eZYqO1u69loPTWfN8h7oX33lQXqbNh6kff65h8Off57/BPOPPeaP++rVuT2hN270L04S4znGITEEydSpudOnn3qP+xDyBuDJvwyoUcMD065d/QuYVA0a+H3fuXP+qV496e23PSiuX9+/ePj6ax8iJDXw7tTJg5ytFQi+/bZ/CdSmTW6I/eCDHppuv33+qXFj327evNwPzpnq2Wf9/aZ9ew/NP//cP0hvs40/ZmPH+vOjbVt/rk+e7KF727b+Pvr++/4+2aWLv2+3bevTt9/6F4mpWrTw5W3a+Osx+fwDWVk+P3lKrLthgw9Jle51LeUdwimdevX8fX7mzPzLbrzRXxf9+/txdtvN/x4kT4lAPXlK/Mphm2082Jf8ObdgQd4vhOrW9YAu0fOtTRu/r6ZN8y+BQ/DQbMGCvJdXrfLzUSxf7q+xOnX8MVi0yPe/Zo0fb/Fiv544bnL7009+rBYtfLu99vLefQcd5O+pCxfmTu3b+y8cJA9dE/MTPVwbNUr/Gk70mFy40I+RqG/RIr+NiV6zS5b463T58tx1lizxv4lr1uRuu2iR137EEf5cW7w4d37i8tq10mWX+faJ+YsX+5co++3nj2Fi3qpV/hxO9ITt1s0D1SZN8v59TmfzZg/kFi/2/1kSQWPytHhx+stLlvjtmD3bv5zu0iU35E5eZ+lSv78lD3SXLi34uZwc/hama1d/Lxw0yG9jIjD/+Wd/j16/3n8Vkpi/Zo1vk7i+dKnf7hkz/G9uonfsokX+f8V223kd3brlrrtsWe7lRLtwoT/eu++eN9AuqkdyjRp+Xzdp4lPy5cT06af+vG7SxI+zdKm/XySC6tQpsZ9EIF2jhj9fNm70x6hxY79tM2bk3S7RS3hrfqH188/+mDz8sHcEadTIvxBu3jxviN6okf8db9gw93rqVLdubq3r1/v7Kl/GAUmmTfM/EklvpqagHvpUn6qnz6hf31+EO+wQU5EAKhsCcRSJQLxqCsFD6I8+kt54o/z2u2qVh4933OFh9UEHeaD9zjseQLRv773A58zxkPG664r+v2P4cB9GYMYMv54cgjz9tPfUTjZ1qoftjz2WG14cfLDfzlq1POA+/3z/wHfssR6mtmzpvQe32847EZx2Wu6J8srDkiX+JcAtt6Rfnhhz+Kij/IPP0097T+fDDvO6atXyy4cc4kFMpvv5Zz/h5267+Qf6RI+Y1as9bPvlF/+fNnkom8KY+fOwRw8Ph7p183HWE0PPJHoc77CDf0j9+uvcHsht2kgXXeQfzj//3HvPX3KJ9zL/9FOvY6+9/Dm2227+vPrpJ38e1qrlx0v0bEv0PF240J/v3bv7B9uZM/MG34mpoDGmJQ8Cu3b129K1a+7UqVNur/fVq30ooBYtckPv1q35II3SW7nSw/YWLfy10bp1/i+NfvzRQ5s2bTxwLOr5tnChr79pkz/vE8F5ixb+2l+wwF9L337rfxtat/YpO9v3vWCB/4qndWvprLOK1zOzRg3ff2JfienFFz2wbtXKX++1avl7cqtWfryXX/bXeYMGPsxNt27+65XkAL0kmjf3vx/F+Te3dm0PAFOHxiqORo08PE4n8euKRJawaFHh7z1Vwe67+/9wNWr4+14iSE+E1cUdW7dGDf8SqXlzb5s186D18899eXa2z2vaNHd54vqUKf4cS16e3I4a5XU0b547f9Uq//vWtq3Pq1HD/99JnAekvNWqlXtOl3Sys72OJk1y2/Hj/T5NXC8o3E5d1rBh0V9WAECpjRsnnXCCf9u3YYMO0FsarwO0uVZtWe0s/2ny4YfHXSWASoRAHEUiEK96Nm3y0HfECA/gPvlE+stffLiQbbf1sG/nnUv+weTdd73XdnIP26ws/5CzdGnuvD32kO66q2QnU/zxRx/r+pBDfLzxSy/14EHycQF79/b2pps8sKhb18cJ/uILD/23285/BTdoUPrxbTdt8g+x7dtv3TBw6lQPP1980UOjo4/2nuIdOmy9Y2ayTZv8y5j27T2weuIJ70HXo4eHH+++619G7Lpr+h7D117r/z936OA9VevV8175u+0m3Xpr3nVTe/G1aeMB+vvvF15j8olL+/f311+6nqYJrVr563OnnfJO22zjY1PXquW9EhM9ogHkNXeufymUGDLo44/99ZMcfCfC9vK0YYP/ykXy12r79v6eMmmSf1HVooX/AqpzZw84a9b095FWrXxq3tyHY+jY0a+3bOlT4nLiC4A1a7ytUcP/djZv7uHra6/ljn3fsqX/zbvrLn//S8xr2dLrmDjRfxnQtq0H4L/84u+Dyeu0aJF7OdE2a+a/IkiE89ddl3f9Fi38fm/c2OtKzGve3N9rp0/POy9xuX59/1K5bdvc+c2b+/Tll75tx465y/73v9x133nHb/eUKR54J27Lu+/6/zyJ+zYRaicuz5+fO65wauiduJydnf5/pY0b/UuMxBi2FWHzZr9NmzfnDdbnzPEvcL/+2q83bep/J6ZNyw3bE1OjRv76CMEfvwYN/O/ar7/6bZkxIzf8btSoav0iDQA0bZp/i/joo/q/FX/RLbpSX5x8o3a9fgA9wwHkQyCOIhGIp3fHHf7T3g0bKtcHho0bfbiJxx/PPaFN8+YeDr7wgvdoe/BB/zC8114FDwcxf75/yK1Rwz8sXXONf7Du2FF66CEP9iQfauSWW/xD54YNvu+BA8veCygE3+fcuf6BtGdPr7lZM++te/HF/qE4MRbywQdXrscBVc+mTfkDshA8eGnVygOtNWu8Z2DXrv78k7xnZZ06/mXIL7/4kDcvvOC9OEeP9i9zpkzx4Vx69/blO+/s7yE77eTDFiQH382aVfxtBwAAAKqLDz/0z7p//7ufqwkAUhGIo0gE4uklehmvWuU9myqDzZu9B/cjj/jwJT/95GN0Z2fnPVmR5EOGPP+8/3rsiCNy54fgAfc110j//refQHHAAO95dNFF3kO7QQPvibvttj6up+THSvQoKi9ff+3hoeQ9wK+4wnuAZ/LYwAAAAACAgm3Y4B2/Djkk78naASChqgTijDgHpLF0qQ8jInmQffHFHoZfd50H2kccIR14YN5xlgcN8vbxx72369SpuWNpLl/u4x5fdZX3lr31VmnPPX14h9de8x7iiZMOnnpqbhgu+UntyjMMl7zn7Esv+W2aNs175BOGAwAAAAAKkpXlw3i+/nrxztMBAJUVgTgqlVWr4jnuzJnSm2/65dWrfZiQ/ff360OG+Mnz/u///LLkJ3B8800fy7tXL+nZZ733u+RDNkh+/Z57PHDeay8fUuWGG3zZd995D/HPP/dv1+Nw5JHS6acXPLQLAAAAAADJevf2dtKkeOsAgLJgZGBUKp99VvHHXLnSQ+lly3x877PP9nGJJWnkSB8i5ZxzpH/8I/9JI5s393HUEmbO9J7eLVv69aee8l7lmzd7gL7vvn5ir379PFwv7xOPAQAAAACwtVx2mXcY++c/pb59464GAEqHQByVyqefVvwxL77YTyDZpIl0990eYm+3nTR7tnT++R6W33NP/jA8ne239/azz6QePfxb8y5dpJdf9lbyQBwAAAAAgKqmSxc/l9ZLL8VdCQCUHkOmoFL5+uvcy4nxt7emZ5+VRo/2MbqXLfOTSx55pPTb3/ryrl2lZ56RapXwq6Pu3aU6daTWrX0s8kQYDgAAAABAVXbGGT6GeBy/8AaA8kAgjlhcd1363uBTp+Ze3mMPP/nj1rJokXThhVJOjofgktS2rTRqlP/0y0waO1Zq3Lh0+587V5ozR2rWrNxKBgAAAAAgVuec4+0zz8RbBwCUFoE4KtyGDdLQoenHG/vmm9zL33+f93p5u+QS7xX+0EO5J5Z86ikPsE8/Xdq0SerUqfT7b9pUqsErDAAAAABQjeyyi7dPPhlvHQBQWsR1qHDr13ubOgzJokXSwoVSmzZbv4axY/2P95Ah0q67+glBPv1U2nvv3HWKM2Y4AAAAAACZxExq106aOdOHTgGAqoZAHLFp2DDv9cRwKTvttHWPu3atdOml0s47S1df7fOaN/eTYAIAAAAAgMKddZa3r7wSbx0AUBoE4ohNQYH4zjvnzttxx5Ltc+lS6b77Cv+W+rbbpBkzpDvuyB0qBQAAAAAAFM/RR3t7wgnx1gEApUEgjthkZ+e9/uOPHlC3b587r379ku1z332l88/3n26l88sv0o03SsccIx10UMn2DQAAAAAApN1393bt2njrAIDSIBBHbFJ7iM+YIXXoUPoTUa5ZI339tV8uqIf4kCHSxo3SrbeW7hgAAAAAAED6y1+8TfzaGwCqCgJxxCY1EJ85U+rYUVq+vHT7GzOm8OXffSc98oh04YVSp06lOwYAAAAAAJAOOMDbN96Itw4AKCkCccQmXQ/xjh2lL77Inffzz9L06cXb30MP5V5O10N82DCpXj3pT38qcakAAAAAACBJnz7evvdevHUAQEkRiCM2yYH4ypXS4sU+ZEoizN5xR2nRImmHHYre18yZ0ltvFbzul19KTz8tXXKJ1KpVWSsHAAAAACCz1a4ttWwpPfssY4kDqFoIxBGb5BNmzpjhbceO0gMPSE8+WbwgPGH0aG8HDvR23bq8y//2Nz+J55VXlrpcAAAAAACQ5JhjvH300VjLAIASIRBHbOrWzb08c6a3HTtKrVtLJ58sTZtWvP2E4AF6//7S3Lk+7+mnc5dPny795z/S+edLzZqVR+UAAAAAAOCOO7y95JJ46wCAkiAQR2zq1Mm9nAjEO3TInfftt7nrDR8uvf56+v189ZWfMPOkk6QBA3xet265Q6/cfrtUs6Z08cXlWDwAAAAAABmuXj1v166VNm+OtxYAKC4CccSmRtKz75dfPPhu0SL/eu3a+QkxDz00/X6efdb3ddxxUvPmPu/mm6Xdd5eWLPGTbZ56qrTttuV+EwAAAAAAyGhDhnj77rvx1gEAxUUgjkph7lypbVvJLHde9+7e/vijt+3b598uBA/E99vPT5aZ2P7zz32olBEjpFWrpCuu2Lr1AwAAAACQiU45xduRI+OtAwCKi0AclcKcOR6IJ/vwQ2nffXOv5+Tk3+7rr31olRNPzL9s82bp/vul/feXdt21fOsFAAAAAADSzjt7+9xz8dYBAMVFII5KYe5caZtt8s6rU0dq0CD3errhVMaO9TZxZuvkHuarVvnY5OeeW56VAgAAAACAZL16+Tji69bFXQkAFI1AHJVCuh7ixTFunI8Vnth248a8y1u0yA3LAQAAAABA+Tv9dG///Od46wCA4iAQR+zWrJGWLcvfQ1zyXt4JI0ZIt96ae33ZMmnSJOnww3Pn7bST9Kc/SQce6NfPOMN7mgMAAAAAgK3jrLO8ve22eOsAgOIgEEfs5s71Nl0P8S++yHv9yitzL7/xhrRpU95AvFYt6aabPCyXpMGDy7VUAAAAAACQon793LHEV66MtxYAKAqBOGKXCMTT9RBfvrzg7caNk5o0kfbaK/+yUaOk++6TunUrjwoBAAAAAEBhrrjC2xdeiLcOACgKgTgqVPJJLxPmzPG2OGOIt2vnbQjSa69JBx/svcJT7bILJ9MEAAAAAKCiHHecty+9FG8dAFAUAnHEbsECb1u1yr/s1luliy7Kvb7TTt7++KMH6QccsPXrAwAAAAAAhWvSRGrRQnr11bgrAYDCEYgjdosXe9usWf5lf/yjdNddudcTvcHfecfb/fbburUBAAAAAIDi2X9/adUqad68uCsBgIIRiCN2ixZJjRtLWVkFr5MIvidNkhYulCZOlFq2lHbcsWJqBAAAAAAAhTv+eG/HjIm1DAAoFIE4Yrd4sf+sqjATJvgZq5ctk0480QPxfv3Sj0kOAAAAAAAq3tFHe5v4VTcAVEYE4ojdokVS8+ZFr/fNN95OmSL9/DPDpQAAAAAAUJnUrSs1bCiNHRt3JQBQMAJxxK44PcSTrVnjLYE4AAAAAACVy267Sb/+6mOJA0BlRCCOChVC/svF7SGesGmTf+P8m9+Ub20AAAAAAKBszjzT24kT460DAApCII4KtXFj/nnF7SGek5P3cs2a5VcXAAAAAAAou379vH3vvXjrAICCEIijQm3YkPf6unX+U6ri9BD/+OPcy716lW9dAAAAAACg7Lbf3tvnnou3DgAoCIE4KlRqIL54sbclGUNckvbcs3zqAQAAAAAA5adOHR9H/Icf4q4EANIjEEeFSg3EFy3ytiRjiEv0EAcAAAAAoLI68EBvE5/5AaAyqTSBuJltZ2YPmdkcM1tnZjPN7HYza1rC/RxvZhPMbLmZrTGzr83sGjOrXcD62WZ2g5l9a2ZrzWypmb1mZgcWsP4wMwuFTIeV5vZnivXr815fssTbZs1Ktp927cqnHgAAAAAAUL769vX2o4/irQMA0qkVdwGSZGY7SJokqZWkFyV9K6mXpEslHWZmfUMIi4uxnxslXSPpV0n/kbRE0r6SbpR0oJkdHkLYkLR+U0nvSdpZ0teS7pPUUNLRkt40s0EhhAcLONxoSTPTzP+xyBucwVJ7iC9f7m2TJiXbj1m5lAMAAAAAAMpZ797evvqqdMQR8dYCAKkqRSAu6V55GH5JCOGuxEwzu03S5ZJukHReYTsws57yMHyZpD1CCNOj+Rbt/zxJF0u6LWmzYfIw/HlJA0IIG6Nt/ixpsqS7zOy1EMLsNIccFUKYUNIbmulSA/EVK7xt1Kh42zdqJF14YfnWBAAAAAAAys+223r79tvx1gEA6cQ+ZErUO/wQeW/re1IWD5W0StLpZtagiF0dE7UjE2G4JIUQgqQ/R1dTo9Rjo/aviTA82maBPDivJ+msYt0QFEtBPcQbNy7e9suXSzfeWL41AQAAAACA8mMm9eghff21tHlz3NUAQF6xB+KS9o/a10MIed4mQwgrJb0vqb6kvYrYT5uonZ66IISwVNJSSZ3MrGNxtkmal3YscUn7mNmVZvYnMxtgZi2KqA8qOBAvbg9xAAAAAABQ+SWGShk7Nt46ACBVZQjEu0Xt9wUs/yFquxaxn8S5izumLjCzJpISJ+fslrSowG0kdUqzfrK/SfqnpJskPSVptpn9LRqiBQVIN2RKnTo+AQAAAACA6uHyy729//546wCAVJUhEE8MlrG8gOWJ+U2K2M/LUTvYzDokZkYB9Q1J6zVNs81wM6uZtE1L+djlqetL0ufyYVQ6yYdU2V7SYPnY5UNSjpWPmZ1jZpPNbPLChQuLuEnVT7oe4sUdLgUAAAAAAFQNLVr40CnjxsVdCQDkVRkC8XIRQnhf0oPy4PwLM3vYzG6V9KGksyV9G62aPCzLXyXNknSCpM/M7HYze0DS15KWpFlfIYQXQggPhxBmhBDWhhB+DiGMlHSEpA2Srixs+JQQwogQQk4IIadly5ZlvdlVzvr1ea8TiAMAAAAAUD316+ft7Nnx1gEAySpDIJ7oAV5QLJqYv6wY+xos6VxJ30k6Kbq8QlJ/SdOidRYkVg4hzJW0p/xkntmSLpD0W0lPSzoxdf3ChBA+kfSRpCxJexdnm0yUbsgUxg8HAAAAAKD6ufpqbz/4IN46ACBZZQjEv4vagsYI7xK1BY0xvkVwI0IIe4YQGoQQGoYQDgohfCBpV3lv709StpkfQrgohNAhhFA7hLBNCOFiSe2jVT4uwW1JjIHSoATbZBSGTAEAAAAAIDPsvru3H34Ybx0AkKwyBOLjo/YQM8tTj5llS+orabWkUn+faGb95QH3yyGEgsYqT/WHqH2imMfIktQzujq9JPVlkuRA/PrrvYc4gTgAAAAAANVPq1bePvVUvHUAQLLYA/EQwjRJr0vqIOnClMXD5b2tHw0hrErMNLMdzWzH1H2ZWb7BN8xse0kjJa2Xn/QyeVkNM2uYZpvT5YH4JEljkuZnm1m3NOvXlnS7PHT/VtLktDcWaXuIM2QKAAAAAADVj5nUubOPIc444gAqi1pxFxC5QB4+32lmB0qaKqm3pP3lQ6Vcm7L+1Ki1lPkPRgH4J/KTYnaUdJR8XO/TQwhfpKxfX9J8M3tDPsb4ZnmP9L2jY5wYQkg+qWZzSVPNbHK0fK6kllGdHSUtknRKyjZIwpApAAAAAABkjltvlY4+WrrlFun22+OuBgAqSSAeQphmZjmSrpN0mKQj5GHzHZKGhxCWFnNX/5V0jvyEmNmS5kt6TtJNIYSpadZfJ+kpSftIOjia94M8gL89hLA6Zf0lku6W1EvSoZKayXueT5P0D0m3hRCKdRLOTJXupJoE4gAAAAAAVE9HHukt44gDqCwqRSAuSSGEWZLOLOa6qT3DE/NHSxpdgmNukHR2CdZfIemS4q6P/Navz3s9BCk7O55aAAAAAADA1mUm7bmn9MEH0rJlUpMmcVcEINPFPoY4MktqD3FJatCg4usAAAAAAAAVo3t3b//973jrAACJQBwVjEAcAAAAAIDMct993n76abx1AIBEII4KtnFj/nkE4gAAAAAAVF81a0pNm0rvvx93JQBAII4Kli4Qr1+/4usAAAAAAAAVJydHmjNHWrs27koAZDoCccSOHuIAAAAAAFRvBx3k7YQJsZYBAATiiB+BOAAAAAAA1dtJJ3l7113x1gEABOKIHYE4AAAAAADVW4cOkpn0yitxVwIg0xGII3aMIQ4AAAAAQPV36KHeMo44gDgRiCN29BAHAAAAAKD6O+oob7/9Nt46AGQ2AnFUqF128fa883LnEYgDAAAAAFD97byzt5dfHm8dADIbgTgq1DvvSF98kXvdTKpbN756AAAAAABAxejTx9sJE6QQYi0FQAYjEEeFatpU2nXX3OsNGngoDgAAAAAAqresLOnSS/3yl1/GWwuAzEUgjlhxQk0AAAAAADLHySd7O3p0vHUAyFwE4ogV44cDAAAAAJA59tjD21dfjbcOAJmLQByxIhAHAAAAACBzZGVJu+8uffMN44gDiAeBOGJFIA4AAAAAQGY57jhvP/ss1jIAZCgCccSKMcQBAAAAAMgs++3n7TPPxFsHgMxEII5Y1asXdwUAAAAAAKAi9e3rLeOIA4gDgThiRSAOAAAAAEBmqVFD6tXLh0xhHHEAFY1AHLGqUyfuCgAAAAAAQEXr1cvbX36Jtw4AmYdAHLGqWzfuCgAAAAAAQEU7+GBvn3su3joAZB4CccTCzFsCcQAAAAAAMs/++3v72mvx1gEg8xCII1YMmQIAAAAAQObJzpaaN/cTa27aFHc1ADIJgThiRQ9xAAAAAAAy0/HHe/vBB/HWASCzEIgjFolvfwnEAQAAAADITJdf7u2IEfHWASCzEIgjFuvWecuQKQAAAAAAZKYdd/SWHuIAKhKBOGKxdq239BAHAAAAACBz9e8vff+9FELclQDIFATiiEWihziBOAAAAAAAmatvX2/fey/eOgBkDgJxxCLRQ5whUwAAAAAAyFynnurtzTfHWweAzEEgjlgwZAoAAAAAANhpJ2/feSfeOgBkDgJxxIIhUwAAAAAAgJl00EHSihW5necAYGsiEEcsGDIFAAAAAABIUr9+3jJsCoCKQCCOWDBkCgAAAAAAkKQrr/T2v/+Ntw4AmYFAHLFgyBQAAAAAACBJ9epJjRpJH38shRB3NQCqOwJxxIIhUwAAAAAAQMLxx3vLyTUBbG0E4ogFQ6YAAAAAAICE//s/bw8+ON46AFR/BOKIBUOmAAAAAACAhJ128nbDBmn9+nhrAVC9EYgjFgyZAgAAAAAAkt1+u7fvvx9rGQCqOQJxxGLTJm/pIQ4AAAAAACTpsMO8nTQp3joAVG8E4ogVgTgAAAAAAJCkdu28vemmeOsAUL0RiCNWtWvHXQEAAAAAAKgM6teXmjSRfv1VCiHuagBUVwTiiFWtWnFXAAAAAAAAKos//cnbr76Ktw4A1ReBOGJlFncFAAAAAACgsth7b29fey3eOgBUXwTiAAAAAAAAqBT69PH200/jrQNA9UUgjljsuGPcFQAAAAAAgMomK8tPrvn883FXAqC6IhBHLD78UJo5M+4qAAAAAABAZdOli7R2rbRuXdyVAKiOCMQRi0aNpO23j7sKAAAAAABQ2RxwgLfjx8dbB4DqiUAcAAAAAAAAlcaAAd6OGBFvHQCqJwJxAAAAAAAAVBqdO3v74Yfx1gGgeiIQBwAAAAAAQKXSq5c0Z44UQtyVAKhuCMQBAAAAAABQqRx0kLf/+1+8dQCofgjEAQAAAAAAUKmceKK3L7wQbx0Aqh8CcQAAAAAAAFQq3bt7+9lnsZYBoBoiEAcAAAAAAEClYiZ16ya9+WbclQCobgjEAQAAAAAAUOl07ertjBnx1gGgeiEQBwAAAAAAQKVz5pneTpwYbx0AqhcCcQAAAAAAAFQ6hx3m7dtvx1sHgOqFQBwAAAAAAACVTr163k6YEGsZAKoZAnEAAAAAAABUSl26SLNmxV0FgOqEQBwAAAAAAACVUmLYlKVL460DQPVBIA4AAAAAAIBKad99vf3oo3jrAFB9EIgDAAAAAACgUurZ09vHHou3DgDVB4E4AAAAAAAAKqUddvB20qR46wBQfRCIAwAAAAAAoNLae29p+nQphLgrAVAdEIgDAAAAAACg0kqMI3711fHWAaB6IBAHAAAAAABApXXWWd7ecku8dQCoHgjEAQAAAAAAUGl16yadfLK0ebP0yy9xVwOgqiMQBwAAAAAAQKV29NHePvVUvHUAqPoIxAEAAAAAAFCpHXust2PGxFoGgGqAQBwAAAAAAACVWp06Uq1a0ocfxl0JgKqOQBwAAAAAAACV3gknSBs2SD/8EHclAKoyAnEAAAAAAABUeoMGefvQQ/HWAaBqIxAHAAAAAABApXfggd5OmBBrGQCqOAJxAAAAAAAAVAk77yx98EHcVQCoygjEAQAAAAAAUCXsuqu3338fbx0Aqi4CcQAAAAAAAFQJ553n7dtvx1sHgKqLQBwAAAAAAABVwp57evvee/HWAaDqIhAHAAAAAABAldCggbfvvBNvHQCqLgJxAAAAAAAAVBm9e0uzZkkhxF0JgKqo0gTiZradmT1kZnPMbJ2ZzTSz282saQn3c7yZTTCz5Wa2xsy+NrNrzKx2Aetnm9kNZvatma01s6Vm9pqZHVjIMWqa2eVm9kV0jCVm9oqZ9Snp7QYAAAAAAEDx9YnSl59+ircOAFVTpQjEzWwHSVMknSnpI0n/kjRd0qWS/mdmzYu5nxslPSdpD0kvSPq3pNWSbpT0ipllpazfVNIHkv4saaOk+yT9R1JPSW+a2dlpjmGSnpJ0m6Taku6OjtVP0jtmdnRJbjsAAAAAAACK75BDvD3uuHjrAFA1WagEvy8xs9ckHSLpkhDCXUnzb5N0uaT7QwjnFbGPnvJQfZmkPUII06P5JuleSedJuiKEcFvSNndIukTS85IGhBA2RvNbSZosqYWkriGE2UnbnCLpCUmTJB0YQlgbzd9T0nuSlkvaIYSwsqjbnZOTEyZPnlzUagAAAAAAAIiEINWokXsZQOVgZlNCCDlx11GU2HuIR73DD5E0U9I9KYuHSlol6XQza1DEro6J2pGJMFySgif+f46uXpiyzbFR+9dEGB5ts0DeA7yepLNStjk/aockwvBom48lPS2ppaQTiqgVAAAAAAAApWAmDR7slz/7LNZSAFRBsQfikvaP2tdDCJuTF0S9rN+XVF/SXkXsp03UTk9dEEJYKmmppE5m1rE42yTN2zKWuJnVldRHPgzLu2m2GRe1BxRRKwAAAAAAAErpvGgcgVdfjbcOAFVPZQjEu0Xt9wUs/yFquxaxn0VR2zF1gZk1kZQ4OWe3pEUFbiOpU5r1d5BUU9L05B7lpagVAAAAAAAApdS9u7cffhhvHQCqnsoQiDeO2uUFLE/Mb1LEfl6O2sFm1iExMxpD/Iak9Zqm2Wa4mdVM2qalfOzy1PXLXKuZnWNmk81s8sKFCwtaDQAAAAAAAAWoWVOqW1datKjodQEgWWUIxMtFCOF9SQ/Kw+gvzOxhM7tV0oeSzpb0bbRq8rAsf5U0Sz7m92dmdruZPSDpa0lL0qxfHnWOCCHkhBByWrZsWZ67BgAAAAAAyBi//a303ntxVwGgqqkMgXiiV3XjApYn5i8rxr4GSzpX0neSToour5DUX9K0aJ0FiZVDCHMl7Sk/mWe2pAsk/VZ+cswTU9cv51oBAAAAAABQSrVreztvXrx1AKhaKkMg/l3UFjTudpeoLWiM8S2CGxFC2DOE0CCE0DCEcFAI4QNJu8p7e3+Sss38EMJFIYQOIYTaIYRtQggXS2ofrfJx0urTJG2Sn5yzVllqBQAAAAAAQOmdfrq3o0fHWweAqqUyBOLjo/YQM8tTj5llS+orabWkD0p7ADPrLw+4Xw4hFDT+d6o/RO0TiRkhhLWSJkmqL2nfNNscHrVvl6pQAAAAAAAAFMv++3s7YUKsZQCoYmIPxEMI0yS9LqmDpAtTFg+X1EDSoyGEVYmZZrajme2Yui8za5Rm3vaSRkpaL2lIyrIaZtYwzTanywPxSZLGpCz+d9Reb2Z1k7bZU9IASQsl/SfNTQUAAAAAAEA5qVtXattWevXVuCsBUJWkG/YjDhfIw+c7zexASVMl9Za0v3z4kWtT1p8atZYy/8EoAP9EflLMjpKOkpQl6fQQwhcp69eXNN/M3pAPh7JZ3iN97+gYJ4YQUk+q+ZSk4+Qn4vzUzMZKai4Pw2tKGhxCWFGymw8AAAAAAICSOugg6dFHpTlzpG22ibsaAFVB7D3EpS29xHMkjZIH4VdI2kHSHZL2CiEsLuau/itpg/yEmFdK2kfSc5K6hxCeTrP+OnnAvZOk8+TBfH15AJ8TQpiTptYg6RRJf5S0UdLF8oD8HUn9QggvFrNWAAAAAAAAlMGJJ3r7738Xvh4AJJjnu4hDTk5OmDx5ctxlAAAAAAAAVEnr10t16vhlIi4gXmY2JYSQE3cdRakUPcQBAAAAAACAkqpdW8rO9str1sRbC4CqgUAcAAAAAAAAVdYNN3j7IoPYAigGAnEAAAAAAABUWQMGeHvBBfHWAaBqIBAHAAAAAABAldWqlVS3rrR0qbRhQ9zVAKjsCMQBAAAAAABQpf35z95++GG8dQCo/AjEAQAAAAAAUKUde6y3//1vvHUAqPwIxAEAAAAAAFCl7byzt++8E28dACo/AnEAAAAAAABUaTVqSL/5jfTFF3FXAqCyIxAHAAAAAABAlbfTTtKqVdKmTXFXAqAyIxAHAAAAAABAlbfTTt6OHBlvHQAqNwJxAAAAAAAAVHlXXuntuHHx1gGgciMQBwAAAAAAQJXXqJHUsKH04otxVwKgMiMQBwAAAAAAQLWw447e/vprvHUAqLwIxAEAAAAAAFAtXHihtxMnxlsHgMqLQBwAAAAAAADVwgEHePvvf8dbB4DKi0AcAAAAAAAA1UL79t6+8068dQCovAjEAQAAAAAAUG0ceKC0cqW0aVPclQCojAjEAQAAAAAAUG0ceqi3338fbx0AKicCcQAAAAAAAFQbOTne3ndfvHUAqJwIxAEAAAAAAFBt7LOPtx98EG8dAConAnEAAAAAAABUG1lZUufO0kcfxV0JgMqIQBwAAAAAAADVSu/e3s6eHW8dACofAnEAAAAAAABUK6ec4u2jj8ZbB4DKh0AcAAAAAAAA1crhh3s7dWq8dQCofAjEAQAAAAAAUK3UqCG1akUPcQD5EYgDAAAAAACg2tluO2+ffjreOgBULgTiAAAAAAAAqHZeeMHbm2+Otw4AlQuBOAAAAAAAAKqd9u19+uQTafPmuKsBUFkQiAMAAAAAAKBaOv54bx95JN46AFQeBOIAAAAAAAColoYP95ZAHEACgTgAAAAAAACqpexsb3/5Jd46AFQeBOIAAAAAAACoto49Vvr+e2nTprgrAVAZEIgDAAAAAACg2urUydsxY2ItA0AlQSAOAAAAAACAauuCC7x95pl46wBQORCIAwAAAAAAoNpK9BD//vt46wBQORCIAwAAAAAAoFrr3Vv68su4qwBQGRCIAwAAAAAAoFrbYw8/qebq1XFXAiBuBOIAAAAAAACo1vbYw9sPP4y3DgDxIxAHAAAAAABAtdavn7c33xxvHQDiRyAOAAAAAACAaq1zZ29ffTXeOgDEj0AcAAAAAAAA1d7xx3v7+efx1gEgXgTiAAAAAAAAqPb++EdvR46Mtw4A8SIQBwAAAAAAQLXXp4+399wTbx0A4kUgDgAAAAAAgIzQubMUgrRhQ9yVAIgLgTgAAAAAAAAywtlne/uf/8RbB4D4EIgDAAAAAAAgI5x7rrfPPBNvHQDiQyAOAAAAAACAjNC0qbc//hhvHQDiQyAOAAAAAACAjNGnj/Tll3FXASAuBOIAAAAAAADIGHvv7e2cOfHWASAeBOIAAAAAAADIGP36efvyy/HWASAeBOIAAAAAAADIGPvv7+2kSfHWASAeBOIAAAAAAADIGNnZUvPm0uOPx10JgDgQiAMAAAAAACCjtG8vbdggrV8fdyUAKhqBOAAAAAAAADLKmWd6+9578dYBoOIRiAMAAAAAACCjHHywt4MGxVsHgIpHIA4AAAAAAICMsuOO3s6YEW8dACoegTgAAAAAAAAyzllnebtsWaxlAKhgBOIAAAAAAADIOL17e3vPPfHWAaBiEYgDAAAAAAAg45xyirdDhsRbB4CKRSAOAAAAAACAjJOdLTVp4pcZNgXIHATiAAAAAAAAyEh33untuHHx1gGg4hCIAwAAAAAAICMdeaS3BOJA5iAQBwAAAAAAQEZKDJkyfnysZQCoQATiAAAAAAAAyFg77yzNnh13FQAqCoE4AAAAAAAAMtZee3k7Y0a8dQCoGATiAAAAAAAAyFhHH+3tTTfFWweAikEgDgAAAAAAgIx1+OHefvxxvHUAqBgE4gAAAAAAAMhYWVlS9+7Sp59KIcRdDYCtjUAcAAAAAAAAGa17d28/+yzWMgBUAAJxAAAAAAAAZLRzzvH2mWfirQPA1kcgDgAAAAAAgIzWt6+3Dz8cbx0Atj4CcQAAAAAAAGS81q2l+fPjrgLA1kYgDgAAAAAAgIx38snerlwZbx0Ati4CcQAAAAAAAGS8Pff09skn460DwNZFIA4AAAAAAICMd9RR3k6ZEm8dALYuAnEAAAAAAABkvOxsbydNircOAFsXgTgAAAAAAAAgaffdpa++khYvjrsSAFsLgTgAAAAAAAAg6YgjvH366XjrALD1EIgDAAAAAAAAkq65xtuJE+OtA8DWU2kCcTPbzsweMrM5ZrbOzGaa2e1m1rSE+znezCaY2XIzW2NmX5vZNWZWu4D165jZhWb2kZktMrNfzWyqmd1pZtunWX+gmYVCpvNKex8AAAAAAAAgPg0aSE2aSM88E3clALaWWnEXIElmtoOkSZJaSXpR0reSekm6VNJhZtY3hFDk6E1mdqOkayT9Kuk/kpZI2lfSjZIONLPDQwgbktavJektSX2jYz4paZ2kPSVdLOkPZtYnhPBNmsO9KOmzNPMnF+c2AwAAAAAAoPI59FAfMuWDD6S99oq7GgDlrVIE4pLulYfhl4QQ7krMNLPbJF0u6QZJhfa8NrOe8jB8maQ9QgjTo/kW7f88ech9W9Jmx8rD8LckHRJC2Jy0v+GS/irpSklnpTnkmBDCqJLcSAAAAAAAAFRuQ4d6IH777dJTT8VdDYDyFvuQKVHv8EMkzZR0T8rioZJWSTrdzBoUsatjonZkIgyXpBBCkPTn6OqFKdt0itqXk8PwyItR27KI4wIAAAAAAKCa2Gknb996K946AGwdsQfikvaP2tdTQ+kQwkpJ70uqL6moH6m0idrpqQtCCEslLZXUycw6Ji36OmoPN7PU++J3UftmAcfrYWaXmdnVZna6mW1XRH0AAAAAAACoAvbeW1q0SPrpp7grAVDeKkMg3i1qvy9g+Q9R27WI/SyK2o6pC8ysiaTEyTm7JS16WdLzkg6W9KWZ3WFm/zSztyUNkXSX8vdaT7hU0r8k/V3SI5Jmmtl9Zla3sCLN7Bwzm2xmkxcuXFjETQIAAAAAAEBFGzDA2xtuiLcOAOWvMgTijaN2eQHLE/ObFLGfl6N2sJl1SMyMxhBPfvtKBOOJ4VROkDRcHpRfIh8zfH9J70h6IoSwMeU4M+RjkXeT1EDSNpJOkg/5cq6khworMoQwIoSQE0LIadmS0VgAAAAAAAAqm4su8vaFF+KtA0D5qwyBeLkIIbwv6UF5cP6FmT1sZrdK+lDS2ZK+jVZNPnFmXUlPS7pCPr54W3lAf4Sk7SW9Y2ZHpxxnYgjh7hDC9yGE1SGEuSGEZ+Uh+lJJp5hZ9614UwEAAAAAALAV1awp9ezpw6YsL6gLJ4AqqTIE4om3lcYFLE/MX1aMfQ2W99L+Tt5r+1xJKyT1lzQtWmdB0vpXSzpR0rUhhPtDCPNCCCtCCOPkPcezJN1RnBsRQpgl6ZXoar/ibAMAAAAAAIDK6ayzvB05Mt46AJSvyhCIfxe1BY0R3iVqCxpjfIvgRoQQ9gwhNAghNAwhHBRC+EDSrvLe4Z8kbZI4ceb4NPv6XN7je3sza16cGyIpMSh4g2KuDwAAAAAAgEro3HO9/e67wtcDULVUhkA8EUYfYmZ56jGzbEl9Ja2W9EFpD2Bm/SW1l/RyCCH5hy51ojbfYN5mVkdSdnR1fTEP1Ttqp5e8SgAAAAAAAFQWtWpJbdtKDzwQdyUAylPsgXgIYZqk1yV1kI/jnWy4vLf1oyGEVYmZZrajme2Yui8za5Rm3vaSRspD7SEpi9+N2j9HAXiyYZJqSfo4hLAyaX85aY5Rw8yukbS3pEWSXs13QwEAAAAAAFClNIqSpjlz4q0DQPmpFXcBkQskTZJ0p5kdKGmqvLf1/vKhUq5NWX9q1FrK/AejAPwTSUskdZR0lHws8NNDCF+krH+DpCMlHSjpWzN7VdIaea/0XtHlS1O2+djMvpL0uaRf5GOc95W0i7wn+2khhBUluvUAAAAAAACodK6+WjrzTOnmm6Xbb4+7GgDlwUIIcdcgSTKzdpKuk3SYpOaS5kp6QdLwEMLSlHWDJIUQLGX+GZLOkbSjfLiT+ZLelnRTCGGq0jCzlpL+JOm38gC9RnTstyX9I4Twbcr6/5SH5V0kNZOPS/6zpDcl3RZCKPZwKTk5OWHy5MnFXR0AAAAAAAAVaNMmHzqlRQtp4cKi1wcymZlNCSHkG12jsqk0gXgmIhAHAAAAAACo3Nq3lxYskNaujbsSoHKrKoF47GOIAwAAAAAAAJXV+edL69ZJKxggF6gWCMQBAAAAAACAAjRv7u1jj8VbB4DyQSAOAAAAAAAAFODww73997/jrQNA+SAQBwAAAAAAAArQrp2UlSVNnx53JQDKA4E4AAAAAAAAUIhzz5VWr+bEmkB1QCAOAAAAAAAAFKJzZ2+/+y7eOgCUHYE4AAAAAAAAUIi99/b2rLPirQNA2RGIAwAAAAAAAIXo2dPbTz6RQoi3FgBlQyAOAAAAAAAAFKJWLemyy/zyu+/GWgqAMiIQBwAAAAAAAIrwhz94e+aZ8dYBoGwIxAEAAAAAAIAi7L67t9OnS+vWxVsLgNIjEAcAAAAAAACKYcgQbydOjLcOAKVHIA4AAAAAAAAUQ2K4lP/+N946AJQegTgAAAAAAABQDB07envXXdKmTfHWAqB0CMQBAAAAAACAYjCTcnL88pQp8dYCoHQIxAEAAAAAAIBiuu8+b485JtYyAJQSgTgAAAAAAABQTD16eDt3LsOmAFURgTgAAAAAAABQTDVrSjfe6Jdvvz3WUgCUAoE4AAAAAAAAUALnn+/tM8/EWweAkiMQBwAAAAAAAEqgSROpeXPpo4/irgRASRGIAwAAAAAAACV02GHevvFGvHUAKBkCcQAAAAAAAKCEEuOI//3v8dYBoGQIxAEAAAAAAIASat/e2/HjpRDirQVA8RGIAwAAAAAAAKVwwQXePv10vHUAKD4CcQAAAAAAAKAU/vpXb596Kt46ABQfgTgAAAAAAABQCq1bSy1bSi++GHclAIqLQBwAAAAAAAAopZ49vX355XjrAFA8BOIAAAAAAABAKT32mLf33htvHQCKh0AcAAAAAAAAKKUWLbydODHeOgAUD4E4AAAAAAAAUAZHHCGtWiWFEHclAIpCIA4AAAAAAACUwZ57evvFF/HWAaBoBOIAAAAAAABAGRx0kLejRsVaBoBiIBAHAAAAAAAAyqBvX2/ffTfeOgAUjUAcAAAAAAAAKAMzqWNHacoUxhEHKjsCcQAAAAAAAKCM9tvP208/jbcOAIUjEAcAAAAAAADK6IILvH3rrXjrAFA4AnEAAAAAAACgjHr08PbOO2MtA0ARCMQBAAAAAACAMsrKkho0kGbPltavj7saAAUpdiBuZp+b2flmlr01CwIAAAAAAACqoiuu8Hb8+HjrAFCwkvQQ31nS3ZLmmNkDZpazlWoCAAAAAAAAqpxBg7ydNCneOgAUrCSB+HaS/iJpoaSzJX1oZpPNbLCZNdgq1QEAAAAAAABVxDbbePvTT/HWAaBgxQ7EQwjzQwg3hhA6STpc0hhJu0m6T95r/F4z67FVqgQAAAAAAAAquZo1pdatpdGj464EQEFKdVLNEMJrIYTjJbWT9xpfJOlcSVPM7AMzG2hmdcuxTgAAAAAAAKDS69TJ23nz4q0DQHqlCsQTQgjzJf1d0h8lzZFkknpJelDSLDO7rKwFAgAAAAAAAFVF4sSa990Xbx0A0it1IG5m25rZUEk/SXpeUhtJL0k6RtLfJG2SdKuZ/a0c6gQAAAAAAAAqvaOP9vaNN+KtA0B6FkIo/spmJh8//NyorSVpvqSRkkaEEGYlrZst6S1J24UQtinPoquLnJycMHny5LjLAAAAAAAAQDmqWVMKQdq8Oe5KgIpjZlNCCDlx11GUYvcQN7O/SJohaaykIyVNknSypHYhhL8kh+GSFEJYGa3buvzKBQAAAAAAACq3ww/3QHzNmrgrAZCqJEOmDJfURNK9knYJIfQPITwTQthYyDZTJD1ShvoAAAAAAACAKuXww719//146wCQX0kC8fMkbRtCuDiE8E1xNgghvBJCOLN0pQEAAAAAAABVz2GHefvYY/HWASC/YgfiIYQRIYRVW7MYAAAAAAAAoKrr1Mnb0aPjrQNAfiUZQ7ynmf3VzNKOCW5mbaLlPcqtOgAAAAAAAKCKMZNac1Y9oFIqyZApV0oaJGlBAcvnSzpb0h/LWhQAAAAAAABQlQ0a5O2YMbGWASBFSQLxvSWNDyGEdAuj+W9L6lsehQEAAAAAAABV1eDB3g4fHm8dAPIqSSDeRtLsItaZI6lt6csBAAAAAAAAqr7tt5eysqTPPou7EgDJShKIr5bUsoh1WkpaV/pyAAAAAAAAgOrhpJO8/fXXeOsAkKskgfhnko42s4bpFppZI0lHR+sBAAAAAAAAGW2ffbw96qh46wCQqySB+Ah5D/A3zGy35AVm1l3S65JaROsBAAAAAAAAGe2cc7z95pt46wCQq9iBeAjhaUmPSOot6VMzm2NmH5vZHEmfSOol6dEQwpNbp1QAAAAAAACg6qhRQzrwQGn+fGnDhrirASCVrIe4QggDJZ0n6Rv5STb3iNqvJZ0TLQcAAAAAAAAgaa+9vH3iiXjrAOAshFC6Dc3qS2oiaVkIYXV5FpUpcnJywuTJk+MuAwAAAAAAAFvJ0qVSs2ZSTo708cdxVwNsPWY2JYSQE3cdRalV2g2jEJwgHAAAAAAAAChA06be0icSqBxKNGQKAAAAAAAAgJI59VRv6SEOxK9EgbiZNTCz/zOzN81sqplNTzNN21rFAgAAAAAAAFXNBRd4++KL8dYBoARjiJtZE0nvSdpZ0gpJjSQtl1RbUr1otTmSNoQQOpZ7pdUQY4gDAAAAAABUfyFINWrkXgaqo6oyhnhJeogPkYfhZ0uKRj/SvyQ1lNRH0ieSpknaqTwLBAAAAAAAAKoys9yxxBctircWINOVJBA/StI7IYSHQ1K38uA+kHSEpB0lXVvONQIAAAAAAABV2t13e/v44/HWAWS6kgTi7SRNSbq+WVKdxJUQwgJJ4ySdXD6lAQAAAAAAANXDMcd4O25crGUAGa8kgfhqeQiesFxSm5R15kvatqxFAQAAAAAAANVJ/fpSdrb02muMIw7EqSSB+Cx5L/GEbyT1M7PkfewjaV55FAYAAAAAAABUJ3vu6e2778ZbB5DJShKIT5S0n5lZdP1pSTtIesXMLjSzZyXtJemVcq4RAAAAAAAAqPLuucfb556Ltw4gk9UqwbqjJdWWtJ28t/h9kg6QdIykQ6J13pc0pBzrAwAAAAAAAKqFbt28XbYs1jKAjGahjIMWmdkekjpLminp4xDC5sK3QEJOTk6YPHly3GUAAAAAAACgguy8szR1KuOIo/oxsykhhJy46yhKsXuIm1k/SStCCJ8lzw8hTJE0pZzrAgAAAAAAAKqdBg28/f57qWvXeGsBMlFJxhAfL+mcrVUIAAAAAAAAUN3985/ennlmvHUAmaokgfgiSWu2ViEAAAAAAABAddevn7eTJsVbB5CpShKIT5DUZyvVAQAAAAAAAFR7NWpIJ5/sl1etircWIBOVJBAfIqmbmf3NzLK2VkEAAAAAAABAddazp7fDhsVaBpCRLBTzlLZm9pCkzpL6Spov6XNJ8ySl7iCEEM4uzyKrq5ycnDB58uS4ywAAAAAAAEAFWr9eqlNHOuAA6a234q4GKB9mNiWEkBN3HUWpVYJ1ByZdbhNN6QRJJQ7EzWw7SddJOkxSc0lzJY2RNDyEsLQE+zle0sWSdpdUW9J0SY9JujWEsD7N+nUkDZJ0hqROkupKmiXpjWibnwo4zhmSLpS0s6RNkj6VdEsI4b/FrRUAAAAAAACZp3ZtD8TffjvuSoDMU5JAvOPWKsLMdpA0SVIrSS9K+lZSL0mXSjrMzPqGEBYXYz83SrpG0q+S/iNpiaR9Jd0o6UAzOzyEsCFp/VqS3pL3ev9W0pOS1knaUx6q/8HM+oQQvkk5zi2SrpA0W9ID8uD9ZEljzeziEMLdpb0vAAAAAAAAUP117Sp9+aWPI96gQdzVAJmj2IF4QT2ly8m98jD8khDCXYmZZnabpMsl3SDpvMJ2YGY95WH4Mkl7hBCmR/Mt2v958pD7tqTNjpWH4W9JOiSEsDlpf8Ml/VXSlZLOSprfRx6GT5O0Z6L3upn9U9IUSbeY2X9DCDNLeicAAAAAAAAgM5x/vnTBBdI550iPPx53NUDmKMlJNbeKqHf4IZJmSronZfFQSasknW5mRX1XdkzUjkyE4ZIPaC7pz9HVC1O26RS1LyeH4ZEXo7ZlyvxEMH9D8lAuUQB+j6Q6ks4solYAAAAAAABksDOj9OiJJ+KtA8g0xQ7Ezax9cacS1rB/1L6eGkqHEFZKel9SfUl7FbGfxJjm01MXRMH1UkmdzCx56Jevo/b/27vz8NrK8n7cn4dJKCIgouKEoCLtV1T0OCAOiIoTKrUiTij+FLXOCnUea8UZtTgiKoJaUURRUKQOqMWpB7W0FFDB0wpoGWSQGeH9/bFWIIbkJDkn56ydnfu+rnU9O2vaT3Jews4nb9796Kqa+rXYva/fnrJ/174eN00P35xyDgAAAADcyIYbJve6V/f4tNOG7QWWkvmsIb4i3RtmzqbN87537euvZjj+63QzyLdLt7TJTM7v643WOq+qzZJsPun5fts/PjbJUUmemOQ/q+rbSa5Ocu8kD0xyUCbNWu9nqd82yaWttd/P0Gv6XgEAAABgRgcdlOy8c/KylyXf+tbQ3cDSMJ/g+rBMH4hvluSeSbZOckKS+a41vmlfL57h+MT+zWa5z7Hp1hDft6o+MrGGd7+G+NsnnTcRjKe11qrqSemWZnlDkr+ZdN53kny+tfbnhey1qp6X5HlJcoc7zHcyPQAAAADj4gEP6OrxxyetJVXD9gNLwXzeVHOfmY71y428Md362s9a/bbmr7V2YlV9MslzkpxcVV9O8sckD0py9ySnJdk+yeQ3ztwwXdD/6HTrix+d5PJ0b7T5z0l+UFV7ttaOzgJprR2c5OAkWbZs2Vxm3AMAAAAwpnbfPTnmmOSnP03uP9uCwcBqW5A31WytXddae2u6ZVXeOc/LJ2ZVbzrD8Yn9F83hXvsmeX6S05M8uX98SZJdkpzRn3PupPNfk2TPJK9vrX28tfaH1tolrbVvJnlSkvWTfHAN9QoAAADAEnfAAV096qhh+4ClYkEC8Ul+lG697/k4va8zrbt9l77OtMb49Vrn4NbafVprG7fWbtpae3hr7SdJdkg3O/znky6ZeOPM701zr/9I90acW1fVFv2+y5KcneSmVbXV6vQKAAAAAH/TL+D7kY8M2wcsFQsdiN88ycbzvGYijN6tX3rlelW1SbrlSy5P8pNVbaqqdklyhyTHttYmr/99k75uOc01N0mySf/h1ZMOfbevj5rmqR495RwAAAAAmNG66yb3vndy2WXJSScN3Q2MvwULxKvq4Un2SvJf87mutXZGkuOT3DHdOt6TvTVdwH54Pzt74rm2r6rtp+nhZtPs2zrJIelC7TdMOfzDvr6uD8Ane0u6Ndb/vbX2p0n7P9bX11fV9W/QWVUT/V+V5NNT+wAAAACA6bzpTV1dtmzYPmApqNbm9r6OVTXTrOf1ktw+3QzsJHlCa+2YeTVRdad0y63cMt0bW56a5H5JHppu+ZEHtNYumHR+S5LWWk25z5eSbJ1uWZQ/JtkmyePTrQW+d2vtiCnn3zbdzPPbpVv//LgkV6SblX7f/vHDWms/nnLd+5K8MslZSY5MskG6XwZskeQlrbUPzeXzXrZsWVu+fPlcTgUAAABgjK2/fvLnPydXXJFsuOHQ3cD8VdVJrbWR/7XOfGaI7zLDtnO6pUW+leQR8w3Dk+tniS9Lcmi6IHy/JHdK94aW958chs/imCTXpHujzP2TPDBdYH2PqWF4/7xnJ7lXkvcluTLJs5O8OMmt+17uNTUM76/brz/3D0mel+SZSU5J8ri5huEAAAAAMOEDH+jqRz86aBsw9uY8Q5yFZ4Y4AAAAAEly/vnJllsmt7td8rvfDd0NzN84zhAHAAAAANaAW9yiq2edlVx88bC9wDgTiAMAAADACHjZy7r65jcP2weMszkH4lX1hqq6pqpuM8Px21bV1VX16oVrDwAAAACWhve/v6uf+MSwfcA4m88M8cclOaG1ds50B/s3qPxekj0WoC8AAAAAWFKqki22SC6/PPG2f7BmzCcQv3OS/57lnP/uzwMAAAAA5ulpT+vqihWDtgFjaz6B+EZJLp/lnCuTbLLq7QAAAADA0vXYx3b1pz8dtg8YV/MJxM9Kcv9Zzrl/krNXvR0AAAAAWLrucY+ufvazw/YB42o+gfhxSR5cVXtNd7CqnpLkIUm+uRCNAQAAAMBSc+tbd9UMcVgz5hOIvyvJRUk+X1VHVdXzquqxff1Kks8l+WOSd66BPgEAAABgSXj4w5Pzz08uvHDoTmD8zDkQb62dneSRSf43yR5JPprka319QpL/SfLI1tpZC98mAAAAACwNj3pUV9/4xmH7gHFUrbX5XVC1fpLHpVsvfLN0s8Z/kuTrrbVrFri/sbZs2bK2fPnyodsAAAAAYIRcc02ywQbd43lGdzCYqjqptbZs6D5ms958L+hD76P6DQAAAABYQOuvn1R1YfjFFyebbjp0RzA+5rOGOAAAAACwFrznPV392MeG7QPGzZwD8ap6Q1VdU1W3meH4bavq6qp69cK1BwAAAABLzwtf2NVPfGLYPmDczGeG+OOSnNBaO2e6g/2bbn4v3RtuAgAAAACraKONunrGGcmllw7bC4yT+QTid07y37Oc89/9eQAAAADAanjGM7p6m2nXawBWxXwC8Y2SXD7LOVcm2WTV2wEAAAAAkuSww7r6pz8N2weMk/kE4mcluf8s59w/ydmr3g4AAAAAkCRVyT77dI/PlrjBgphPIH5ckgdX1V7THayqpyR5SJJvLkRjAAAAALDU7dUncR//+LB9wLio1trcTqy6bZKTk2yW5Oh0AfnZSW6b5NFJHp/kwiT3bK2dtSaaHTfLli1ry5cvH7oNAAAAAEbUddcl667bPZ5jjAeDqKqTWmvLhu5jNuvN9cTW2tlV9cgkX0qyR5InTDpcSVYk2VMYDgAAAAALY51J6zscdVTyxCcO1wuMg/ksmZLW2vIk2yV5UpL3JflkX5+U5K5JflFVT5j5DgAAAADAfKxY0dV3vWvQNmAszHmG+ITW2jVJjuq3JElVbZ3kTUmenWSrJOsuVIMAAAAAsJRtvXVXf/azYfuAcTCvGeKTVdW6VfXEqjouyRlJXp8uDP/2QjUHAAAAACQ77dTVCy8ctg9Y7OYdiFfVtlX1jiRnpVtP/BFJLkjyT0m2ba09cmFbBAAAAICl7UUv6uohhwzbByx2cwrEq2q9qtqzqv41ya+SvDrJ5umWTakkR7fW3tRa+5811yoAAAAALE177dXV73532D5gsVvpGuJVdZck+yZ5VpJbpAu/T0pyaJLPt9YurKrr1nSTAAAAALCUrbdesuGGyXHHJa0lVUN3BIvTbDPET0+yX5JrkxyYZIfW2n1aax9urVmxCAAAAADWkp137qpZ4rDq5rJkSkvyzSRfbq2dsob7AQAAAACm8alPdfVznxu2D1jMZgvE35jkf5M8O8mJVfXfVfWqqtpqzbcGAAAAAEy4/e27etppw/YBi9lKA/HW2ttba9smeXSSryS5U5J3Jvnfqjq2qp68FnoEAAAAgCWvKtlhh+THPx66E1i85rJkSlpr32qtPSnJ7ZO8Lsn/pAvJ/yXdkir3rKp7r7EuAQAAAIBsuWVXf/vbYfuAxWpOgfiE1tq5rbV3ttbunOQRSY5Mck2SZUl+VlW/qKoXrYE+AQAAAGDJe/nLu/rhDw/aBixa1VpbvRtU3SLJPkmem2S7JK21tu7qtzb+li1b1pYvXz50GwAAAAAsEq0l66yT3PSmyZ/+NHQ3cIOqOqm1tmzoPmYzrxni02mtnd9ae29rbfsku6ZbRgUAAAAAWGBVXb300i4cB+ZntQPxyVprJ7TWnrGQ9wQAAAAAbrD//l0944xh+4DFaEEDcQAAAABgzXrsY7v63vcO2wcsRgJxAAAAAFhEHvKQrh5yyLB9wGIkEAcAAACARaQquf3tk2uvtY44zJdAHAAAAAAWmV137er73z9sH7DYCMQBAAAAYJE5+OCu7rffsH3AYiMQBwAAAIBFZoMNbnh89tnD9QGLjUAcAAAAABahI4/s6mtfO2wfsJgIxAEAAABgEfrbv+3q4YcP2wcsJgJxAAAAAFiE1lknuetdu8eXXDJsL7BYCMQBAAAAYJF65Su7+rSnDdsHLBYCcQAAAABYpPbdt6vHHjtsH7BYCMQBAAAAYJGquuHxVVcN1wcsFgJxAAAAAFjE3vnOrj73ucP2AYuBQBwAAAAAFrH99+/qkUcO2wcsBgJxAAAAAFjE1l032Xrr5Mork2uvHbobGG0CcQAAAABY5J71rK4++9nD9gGjTiAOAAAAAIvc61/f1S9/edg+YNQJxAEAAABgkdtgg2THHZPLL0/OOmvobmB0CcQBAAAAYAy86lVd3XffYfuAUSYQBwAAAIAxsNdeXT3uuGH7gFEmEAcAAACAMVCV7LRT9/hPfxq2FxhVAnEAAAAAGBNPfWpXv/rVQduAkSUQBwAAAIAx8YxndPW97x22DxhVAnEAAAAAGBObb55stFFy8slJa0N3A6NHIA4AAAAAY+SJT+zql740bB8wigTiAAAAADBG3v/+rh522LB9wCgSiAMAAADAGNlyy+TmN0+OPXboTmD0CMQBAAAAYMz8zd909Yc/HLYPGDUCcQAAAAAYM5/5TFff8Y5h+4BRIxAHAAAAgDGz7bZd/eY3h+0DRo1AHAAAAADG0K67dvW884btA0aJQBwAAAAAxtAzn9nVE04YtA0YKQJxAAAAABhDu+/e1c9+dtg+YJQIxAEAAABgDG2xRVe/9rVh+4BRIhAHAAAAgDG1445dbW3YPmBUCMQBAAAAYEz93d919be/HbYPGBUCcQAAAAAYU/e5T1c/9rFh+4BRIRAHAAAAgDH18Id39dBDB20DRoZAHAAAAADG1Dp9+nfeeclFFw3aCowEgTgAAAAAjLG3vKWrH/jAkF3AaKjmLWYHs2zZsrZ8+fKh2wAAAABgjF15ZbLRRt1jUSBrSlWd1FpbNnQfszFDHAAAAADG2IYbJtts0z2+8sphe4GhCcQBAAAAYMy99KVdPeaYYfuAoY1MIF5Vt6uqT1XVOVV1VVWtqKoPVNXm87zP31XVCVV1cVVdUVWnVNVrq2qDac49tKraLNt3plyzzyznv2B1vxYAAAAAsJCe9rSuvulNw/YBQ1tv6AaSpKrulORHSW6Z5OgkpyW5b5KXJXlUVe3cWrtgDvc5IMlrk1ya5MtJ/pjkQUkOSPKwqnp0a+2aSZd8NcmKGW63d5Jtk3xzhuNHJ/nlNPstCg4AAADASLnlLbt66qnJn/+crDcSqSCsfaMy9D+SLgx/aWvtoImdVXVgklckeXuSlc68rqp7pQvDL0py79bamf3+6u//giQvSXLgxDWtta+mC8Wn3muzJK9KcnWSQ2d4yq+21mY6BgAAAAAjZa+9kiOOSI47Ltl996G7gWEMvmRKPzt8t3QztT885fCbk1yWZO+q2niWW+3R10MmwvAkaa21JK/rP3zRHNvaO8lGSY5qrZ0/x2sAAAAAYGQd2E8T/fKXh+0DhjR4IJ7koX09vrV23eQDrbU/JTkxyV8luf8s97l1X8+ceqC1dmGSC5NsW1XbzKGnfft68ErOuWdVvbyqXlNVe1fV7eZwXwAAAAAYxFZbdfXQQwdtAwY1CoH4Xfv6qxmO/7qv281yn4mZ3DcKvPslUCbenPOuU49POXenJDsk+VVr7XsrOfVlSd6f5B1JDkuyoqo+VlUbztInAAAAAKx1VcmtbtU9/sMfhu0FhjIKgfimfb14huMT+zeb5T7H9nXfqrrjxM5+DfG3Tzpv86zc8/r6iRmO/zbdWuR3TbJxktskeXK6JV+en+RTK7t5VT2vqpZX1fLzzjtvllYAAAAAYOEcckhXv/nNYfuAoYxCIL4gWmsnJvlkuuD85Kr6dFW9L8lPkzwnyWn9qddNf4ekqjZNF27P+GaarbXvt9Y+1Fr7VWvt8tba71trX0q39MuFSZ5aVfdYSZ8Ht9aWtdaWbbnllvP+PAEAAABgVT20X7z4ve8dtg8YyigE4hMzwDed4fjE/ovmcK99083SPj1dsP38JJck2SXJGf05567k+mekW6983m+m2Vr7XZJv9B8+eD7XAgAAAMDasPHGXf3v/05aG7YXGMIoBOKn93WmNcLv0teZ1hi/Xusc3Fq7T2tt49baTVtrD2+t/STduuDXJfn5Sm4x8WaaH59L49OYWANl41W8HgAAAADWqOf1CwZbNoWlaBQC8Yk3rtytqv6in6raJMnOSS5P8pNVfYKq2iXJHZIc21qbdq3yqrpfknukezPNE1bxqe7X1zNX8XoAAAAAWKPe/OauPvaxw/YBQxg8EG+tnZHk+CR3TPKiKYffmm629eGttcsmdlbV9lW1/dR7VdXNptm3dZJD0q0L/oaVtDLxZpoHr6zfqlo2zb51quq1SXZKcn6S41Z2DwAAAAAYym1uc8PjSy8drg8YwnpDN9B7YZIfJfnnqnpYklPTzbZ+aLqlUl4/5fxT+1pT9n+yD8B/nuSPSbZJ8vgk6yfZu7V28nRP3gfpeyW5KslnZun136vqv5L8R5Kz061xvnOSu6Wbyf701tols9wDAAAAAAbzkY8kL3xh8pWvJHvvPXQ3sPYMPkM8uX6W+LIkh6YLwvdLcqckH0xy/9baBXO81TFJrkmyZ5L9kzwwyZFJ7tFaO2Il1z093Uz0r8zhzTTfmy5s3zXJy5I8M13g/uEkO7TWjp9jrwAAAAAwiL326upx1jlgianm7WQHs2zZsrZ8+fKh2wAAAABgCapKNt7YsiksjKo6qbV2o+WmR81IzBAHAAAAANaue94zueyy5Oqrh+4E1h6BOAAAAAAsQRPLpvziF8P2AWuTQBwAAAAAlqDHPKar73//sH3A2iQQBwAAAIAl6O537+oRRwzbB6xNAnEAAAAAWKJuf/uuLl8+bB+wtgjEAQAAAGCJOvrorh522LB9wNoiEAcAAACAJWrHHbv6b/82bB+wtgjEAQAAAGAJu+Utk5NPHroLWDsE4gAAAACwhC1bllx7bXLFFUN3AmueQBwAAAAAlrAHP7ir73nPsH3A2iAQBwAAAIAlbP/9u3rooYO2AWuFQBwAAAAAlrB11+3qb3+bXHXVsL3AmiYQBwAAAIAl7lWv6uovfjFsH7CmCcQBAAAAYInbc8+uvvGNw/YBa5pAHAAAAACWuHvdq6vLlw/bB6xpAnEAAAAAWOLWWSd5yEOSiy5KWhu6G1hzBOIAAAAAQO53v65+9KPD9gFrkkAcAAAAAMg//mNXP/OZYfuANUkgDgAAAADkJjdJNt44+dnPhu4E1hyBOAAAAACQJHnEI7r62c8O2wesKQJxAAAAACBJcuCBXX3lK4ftA9YUgTgAAAAAkCTZZptk3XWT885LLrlk6G5g4QnEAQAAAIDrTby55vHHD9sHrAkCcQAAAADges97XlcngnEYJwJxAAAAAOB6t7hFV//zP4ftA9YEgTgAAAAA8Bee/eyunnzysH3AQhOIAwAAAAB/4YlP7Or73jdsH7DQBOIAAAAAwF947GO7ethhyWWXDdsLLCSBOAAAAADwF6qSpz2te3z00cP2AgtJIA4AAAAA3MiHPtTVpz992D5gIQnEAQAAAIAb2XzzGx5fd91wfcBCEogDAAAAANPaf/+uHnXUsH3AQhGIAwAAAADTeuUru/qKVwzbBywUgTgAAAAAMK2tturqWWcN2wcsFIE4AAAAADCjpzylq1deOWwfsBAE4gAAAADAjHbaqaunnjpsH7AQBOIAAAAAwIwe+tCu3utew/YBC0EgDgAAAADMaIcdbnjc2nB9wEIQiAMAAAAAK/WSl3T1l78ctA1YbQJxAAAAAGCl9t23q695zbB9wOoSiAMAAAAAK3W3u3X1+OOTX/962F5gdQjEAQAAAICVqkoOOKB7/P73D9sLrA6BOAAAAAAwq4nlUk48cdg+YHUIxAEAAACAWVUlW22VnHzy0J3AqhOIAwAAAABzsttuXT377GH7gFUlEAcAAAAA5uRJT+rqIYcM2wesKoE4AAAAADAnj3lMV9/5zmH7gFUlEAcAAAAA5mSdPk288srkkkuG7QVWhUAcAAAAAJizd72rq+94x7B9wKoQiAMAAAAAc/bKV3b1sMOG7QNWhUAcAAAAAJiz9dZLtt46OeecpLWhu4H5EYgDAAAAAPPy6Ed39Wc/G7YPmC+BOAAAAAAwLy9/eVdf97pB24B5E4gDAAAAAPNy17t29bvfHbYPmC+BOAAAAAAwb095SlfPPHPYPmA+BOIAAAAAwLw997ldfeMbh+0D5kMgDgAAAADM2y67dPXznx+0DZgXgTgAAAAAMG/rrpvc+97d43POGbYXmCuBOAAAAACwSl73uq6+5CXD9gFzJRAHAAAAAFbJHnt09Yc/HLQNmDOBOAAAAACwStZZJ7n73ZPzzktaG7obmJ1AHAAAAABYZbvu2tVvfWvYPmAuBOIAAAAAwCp74Qu7euyxw/YBcyEQBwAAAABW2V3u0tXly4ftA+ZCIA4AAAAArJa73CX5yU+SSy8duhNYOYE4AAAAALBa7n//rn75y8P2AbMRiAMAAAAAq+V97+vq2942bB8wG4E4AAAAALBattyyq2ecMWwfMBuBOAAAAACw2p74xK7+138N2wesjEAcAAAAAFht++3X1cMPH7YPWBmBOAAAAACw2nbaqavvfvewfcDKCMQBAAAAgNVWdcPj5cuH6wNWRiAOAAAAACyIU0/t6p57DtsHzEQgDgAAAAAsiO237+qKFcm55w7aCkxLIA4AAAAALJiPfKSr++8/bB8wHYE4AAAAALBg/v7vu3r44clVVw3bC0wlEAcAAAAAFtQ97tHVF7xg2D5gKoE4AAAAALCgfvazrh56aNLaoK3AXxCIAwAAAAALaoMNksc+tnt85pnD9gKTjUwgXlW3q6pPVdU5VXVVVa2oqg9U1ebzvM/fVdUJVXVxVV1RVadU1WuraoNpzj20qtos23dmeJ5nVdXPqurS/rlOqKrdV/XzBwAAAIBx8qIXdfXb3x62D5hsvaEbSJKqulOSHyW5ZZKjk5yW5L5JXpbkUVW1c2vtgjnc54Akr01yaZIvJ/ljkgclOSDJw6rq0a21ayZd8tUkK2a43d5Jtk3yzWme571J9ktyVpJPJNkgyVOSfL2qXtJa+9BsvQIAAADAOLvvfbt60EHJ858/bC8wodoILOJTVd9KsluSl7bWDpq0/8Akr0jy8dbaSpfgr6p7JTkpyUVJ7t1aO7PfX0k+kuQFSfZrrR04h342S3JOknWT3La1dv6kYw9IcmKSM5Lcp7V2Yb//jv3zb5xk+9baitmeZ9myZW358uWznQYAAAAAi1JVV6+5JllvJKbmsqZU1UmttWVD9zGbwZdM6WeH75ZupvaHpxx+c5LLkuxdVRvPcqs9+nrIRBieJK1L/F/Xf/iiOba1d5KNkhw1OQzvTQTzb58Iw/vnmej/JkmePcfnAQAAAICx9fSnd/VLXxq2D5gweCCe5KF9Pb61dt3kA621P6Wbjf1XSe4/y31u3dcbLdPfB9cXJtm2qraZQ0/79vXgaY7t2tfjpjn2zSnnAAAAAMCSdVC/FsTHPjZsHzBhFALxu/b1VzMc/3Vft5vlPhMzuW8UePdLoEy8Oeddpx6fcu5OSXZI8qvW2vemHNs4yW2TXNpa+/2q9FpVz6uq5VW1/LzzzltZKwAAAACwqG3eJ3I/+EEyAis3w0gE4pv29eIZjk/s32yW+xzb13379byTXL+G+Nsnnbd5Vu55ff3ENMdWu9fW2sGttWWttWVbbrnlLK0AAAAAwOL2//5fV884Y9g+IBmNQHxBtNZOTPLJdGH0yVX16ap6X5KfJnlOktP6U6+b/g5JVW2a5MlJrk5y6JrsFwAAAACWgne9q6t3ucuwfUAyGoH4xKzqTWc4PrH/ojnca98kz09yerpg+/lJLkmyS5KJ30Gdu5Lrn5FuvfLp3kxzoXsFAAAAgLH32Mfe8PjPfx6uD0hGIxA/va8zrbs98bujmdYYv17rHNxau09rbePW2k1baw9vrf0k3brg1yX5+UpuMfFmmh+f4f6XJTk7yU2raqvV6RUAAAAAloo3vrGre+01bB8wCoH4xBtX7lZVf9FPVW2SZOcklyf5yao+QVXtkuQOSY5trU27/ndV3S/JPdK9meYJK7ndd/v6qGmOPXrKOQAAAACw5L34xV39+teH7QMGD8Rba2ckOT7JHZO8aMrhtybZOMnh/ezsJElVbV9V20+9V1XdbJp9Wyc5JN264G9YSSsTb6Z58Cwtf6yvr6+q69+gs38jzxcluSrJp2e5BwAAAAAsGbe8ZfK4xyXXXJNcN+M7/MGat97QDfRemORHSf65qh6W5NQk90vy0HTLj7x+yvmn9rWm7P9kH4D/PMkfk2yT5PFJ1k+yd2vt5OmevA/S90oXZn9mZY221n5UVQcmeWW6N+88MskG/fU3T/KS1tqK2T5hAAAAAFhKdtqpmyF+3HHJYx4zdDcsVYPPEE+unyW+LMmh6YLw/ZLcKckHk9y/tXbBHG91TJJrkuyZZP8kD0xyZJJ7tNaOWMl1T083E/0rM7yZ5tR+90vy7CR/SDez/JlJTknyuNbah+bYKwAAAAAsGU97Wle/9rVh+2Bpq9ba0D0sWcuWLWvLly8fug0AAAAAWCuqX+9BJDl+quqk1tqyofuYzUjMEAcAAAAAxt/WW3f1iiuG7YOlSyAOAAAAAKwV//APXX3lK4ftg6VLIA4AAAAArBXPfnZXP/Yxy6YwDIE4AAAAALBW/NVfJbvt1j3+zneG7YWlSSAOAAAAAKw1n/hEV9/znmH7YGkSiAMAAAAAa80d7tDV448ftg+WJoE4AAAAALBWbbddV887b9g+WHoE4gAAAADAWnXggV3dZ59B22AJEogDAAAAAGvVYx7T1W98Y9g+WHoE4gAAAADAWlWV7Lxz9/jEE4fthaVFIA4AAAAArHXvfndXX/rSYftgaRGIAwAAAABr3QMe0NWf/zxpbdheWDoE4gAAAADAIO59766ecMKgbbCECMQBAAAAgEEcfnhXP/ShYftg6RCIAwAAAACD2H77rh511LB9sHQIxAEAAACAQVQlO+7YPf7Nb4bthaVBIA4AAAAADObd7+7qAQcM2wdLg0AcAAAAABjMwx7W1U9/Orn66mF7YfwJxAEAAACAwVQlt7xl9/gmNxm2F8afQBwAAAAAGNSKFTc8bm2wNlgCBOIAAAAAwKA22ijZf//u8WmnDdsL400gDgAAAAAM7glP6OrPfz5sH4w3gTgAAAAAMLi73a2rxxwzbB+MN4E4AAAAADC4zTbr6he+MGgbjDmBOAAAAAAwEm51q65ec82wfTC+BOIAAAAAwEh47Wu7+slPDtsH40sgDgAAAACMhEc+sqt///fD9sH4EogDAAAAACNh++2TTTbpHl977bC9MJ4E4gAAAADAyHjVq7p6yCHD9sF4EogDAAAAACNjv/26eswxw/bBeBKIAwAAAAAjY6ONunrMMckppwzbC+NHIA4AAAAAjJTXv76ru+46bB+MH4E4AAAAADBS/umfunruucP2wfgRiAMAAAAAI2fPPbt6+unD9sF4EYgDAAAAACPnH/6hq1/4wrB9MF4E4gAAAADAyFm2rKtvecugbTBmBOIAAAAAwMipuuHxkUcO1wfjRSAOAAAAAIyk3/ymqxPricPqEogDAAAAACPpTndK1l23e3zOOcP2wngQiAMAAAAAI+vTn+7qM585bB+MB4E4AAAAADCy9tqrq9/5TnLVVcP2wuInEAcAAAAARtYGGyRPfWr3+Oijh+2FxU8gDgAAAACMtIMO6uqLXzxsHyx+AnEAAAAAYKRtsUVXzzsvueaaYXthcROIAwAAAAAj7znP6epHPjJsHyxuAnEAAAAAYOR96ENd/exnh+2DxU0gDgAAAACMvA037Ory5cnFFw/bC4uXQBwAAAAAWBT23LOrO+44bB8sXgJxAAAAAGBROOKIrv72t0lrw/bC4iQQBwAAAAAWharkhS/sHv/wh8P2wuIkEAcAAAAAFo2Xvayr3/72sH2wOAnEAQAAAIBFY7vtuvq2tw3bB4uTQBwAAAAAWJQuvHDoDlhsBOIAAAAAwKLy0Y929fjjh+2DxUcgDgAAAAAsKs94Rlef8pRh+2DxEYgDAAAAAIvKTW96w+NTTx2uDxYfgTgAAAAAsOh897td/dznhu2DxUUgDgAAAAAsOg98YFe//OVh+2BxEYgDAAAAAIvO+usnm22WnHba0J2wmAjEAQAAAIBF6XGP6+ry5cP2weIhEAcAAAAAFqXXvKarX/rSsH2weAjEAQAAAIBF6a//uqvvfvewfbB4CMQBAAAAgEWpKtlii+7xqacO2wuLg0AcAAAAAFi0jjmmq4cdNmwfLA4CcQAAAABg0brvfbv6uc8N2weLg0AcAAAAAFi01lknucUtkt/9LrnkkqG7YdQJxAEAAACARe3pT+/qF784bB+MPoE4AAAAALCo/eM/dnXffYftg9EnEAcAAAAAFrWb3eyGx9deO1wfjD6BOAAAAACw6L35zV398Y+H7YPRJhAHAAAAABa9iXXEP/WpYftgtAnEAQAAAIBF78537uqnPz1sH4w2gTgAAAAAsOhVJVtt1T3+5S8HbYURJhAHAAAAAMbC17/e1Q9/eNg+GF0CcQAAAABgLOy4Y1cPOWTYPhhdAnEAAAAAYCyss06ywQbd45/8ZNheGE0CcQAAAABgbPziF1396EeH7YPRJBAHAAAAAMbGX/91Vw87bNg+GE0jE4hX1e2q6lNVdU5VXVVVK6rqA1W1+Tzv83dVdUJVXVxVV1TVKVX12qraYCXXrFtVz62qH1TVhf11Z1bVEVW13ZRz96mqtpLtBav6NQAAAAAAVk9Vcpe7dI+/8IVhe2H0rDd0A0lSVXdK8qMkt0xydJLTktw3ycuSPKqqdm6tXTCH+xyQ5LVJLk3y5SR/TPKgJAckeVhVPbq1ds2Ua27aP+euSX6Z5DNJrkxy2/7a7ZL8apqnO7o/f6rls/UJAAAAAKw5P/hBstVWyVOfmjzlKUN3wygZiUA8yUfSheEvba0dNLGzqg5M8ookb0+y0pnXVXWvdGH4RUnu3Vo7s99f/f1fkOQlSQ6ccunH04XhL2itfXya+64/w1N+tbV26GyfGAAAAACwdt361jc8vuaaZP2ZEj6WnMGXTOlnh++WZEWSD085/OYklyXZu6o2nuVWe/T1kIkwPElaay3J6/oPXzTlue+V5GlJjpguDO+vv2a6/QAAAADA6DrggK6eeOKwfTBaBg/Ekzy0r8e31q6bfKC19qckJyb5qyT3n+U+E7/3OXPqgdbahUkuTLJtVW0z6dDT+vovVbVpVT2jX2/8eVV151me755V9fKqek1V7V1Vt5vlfAAAAABgLdl9964eccSwfTBaRmHJlLv2dbp1upPk1+lmkG+X5Dsruc/5fd1m6oGq2izJxJtz3jXJb/vH9+nr1knOSLLFpMtaVX003TIu107zfC+b8vG1VXVIkpe31q5cSZ8AAAAAwBq2ww5dPeKI5KMfHbYXRscozBDftK8Xz3B8Yv9ms9zn2L7uW1V3nNjZryH+9knnbT7p8S37emCSE5L8dZJNkjw8XUD+wiRvnPI8v023Fvldk2yc5DZJnpxuyZfnJ/nUyprsZ58vr6rl55133iyfEgAAAACwqm5zm+TCC4fuglEyCoH4gmitnZjkk+mC85Or6tNV9b4kP03ynCSn9adOXpZl4vM/LclerbXTWmuXtta+k+RJ/bmvrKoNJj3P91trH2qt/aq1dnlr7fettS+lW/rlwiRPrap7rKTPg1try1pry7bccssF+dwBAAAAgBvbaaeunnXWsH0wOkYhEJ+YAb7pDMcn9l80h3vtm26W9unpZm0/P8klSXZJN+M7Sc6ddP7EPb8+dVmU1tp/pJsNvkm6meMr1Vr7XZJv9B8+eA69AgAAAABr0LOe1dXPf37YPhgdoxCIn97X7WY4fpe+zrTG+PVa5+DW2n1aaxu31m7aWnt4a+0nSXZIN+P759M890Uz3HLiDyo2mu25exNroGw8x/MBAAAAgDVkYob4q189bB+MjlEIxL/X192q6i/6qapNkuyc5PIkP1nVJ6iqXZLcIcmxrbXJa5V/u693m+aam+SGMH7FHJ/qfn09c95NAgAAAAAL6ha3uOHxJz85XB+MjsED8dbaGUmOT3LHJC+acvit6WZbH95au2xiZ1VtX1XbT71XVd1smn1bJzkkydVJ3jDl8JeTnJNkr6q675Rjb0y3XMv3Wmt/mHS/ZdM8xzpV9dokOyU5P8lx036yAAAAAMBadXq/RsRznztsH4yG9YZuoPfCJD9K8s9V9bAkp6abbf3QdEulvH7K+af2tabs/2QfgP88yR+TbJPk8UnWT7J3a+3kySe31i6rqn2SHJPkh1V1VJKz++d+YLr1xp8/5Tn+var+K8l/9Odumm4W+93SzWR/emvtkvl+AQAAAACAhbfdpIWaf/3r5C53mflcxt/gM8ST62eJL0tyaLower8kd0rywST3b61dMMdbHZPkmiR7Jtk/Xah9ZJJ7tNaOmOG5/zXJfZN8PcnDk7w0ydZJPpZkx9bar6dc8t50YfuuSV6W5JnpAvcPJ9mhtXb8HHsFAAAAANaCH/ygq1/84rB9MLxqrQ3dw5K1bNmytnz58qHbAAAAAICxdt11ybrrJne+czdLnIVXVSe11m603PSoGYkZ4gAAAAAAa8o6fQr6m98kf/7zsL0wLIE4AAAAADD2Jt5U88gjh+2DYQnEAQAAAICx97a3dfUd7xi2D4YlEAcAAAAAxt6tb93Vk0/u1hRnaRKIAwAAAABLwiMe0dW3vGXQNhiQQBwAAAAAWBKOPrqrRx01bB8MRyAOAAAAACwJG22UbLppcsopQ3fCUATiAAAAAMCS8aQndfVrXxu2D4YhEAcAAAAAloy3va2rxxwzbB8MQyAOAAAAACwZW23V1U98Ivnzn4fthbVPIA4AAAAALCkbbtjViTfZZOkQiAMAAAAAS8qKFV194QsHbYMBCMQBAAAAgCXlVrfq6rnnJpdfPmwvrF0CcQAAAABgyXn1q7t6hzsM2wdrl0AcAAAAAFhy3vGOrl5wQXL66cP2wtojEAcAAAAAlpyq5PDDu8fbbz9sL6w9AnEAAAAAYEl6xjNueHzyycP1wdojEAcAAAAAlqxjj+3qQQcN2wdrh0AcAAAAAFiyHvOYrh5yyLB9sHYIxAEAAACAJW3rrbv6f/83bB+seQJxAAAAAGBJm1guZWL5FMaXQBwAAAAAWNJ22aWrX/zioG2wFgjEAQAAAIAlbZNNuvqtbyXnnTdsL6xZAnEAAAAAYMn7+7/v6oMeNGwfrFkCcQAAAABgyfvwh7t6+ulJa8P2wpojEAcAAAAAlryq5OlP7x7/+78P2wtrjkAcAAAAACDJq1/d1X32GbQN1iCBOAAAAABAkh126OqppybXXTdsL6wZAnEAAAAAgN5TntLVD31o2D5YMwTiAAAAAAC9ww7r6sEHD9sHa4ZAHAAAAACgt/76XT3llKS1YXth4QnEAQAAAAAmefjDu/rJTw7bBwtPIA4AAAAAMMnhh3d1332H7YOFJxAHAAAAAJjk1re+4fH//u9wfbDwBOIAAAAAAFN85ztd/dSnhu2DhSUQBwAAAACYYtddu/rVrw7aBgtMIA4AAAAAMI2b3CT5j/8YugsWkkAcAAAAAGAae+3V1dNPH7YPFo5AHAAAAABgGi94QVeXLRu2DxaOQBwAAAAAYBo77dTVSy9NLrlk2F5YGAJxAAAAAIAZvO1tXT3wwGH7YGFUa23oHpasZcuWteXLlw/dBgAAAAAwg9aSdda54THTq6qTWmsjv7iMGeIAAAAAADOoStZdt3v8298O2wurTyAOAAAAALASX/1qVz/ykUHbYAEIxAEAAAAAVmL33bt65JHD9sHqE4gDAAAAAMziVrdKVqywjvhiJxAHAAAAAJjF4x/f1e99b9g+WD0CcQAAAACAWbz4xV192MOG7YPVIxAHAAAAAJjF3e9+w+M//Wm4Plg9AnEAAAAAgDl4z3u6+rWvDdsHq04gDgAAAAAwB895Tlff+tZh+2DVCcQBAAAAAOZg8827+utfJ9deO2wvrBqBOAAAAADAHD33uV21bMriJBAHAAAAAJijt72tq5/5zLB9sGoE4gAAAAAAc3TrW3f16KOH7YNVIxAHAAAAAJiHe9yjq60N2wfzJxAHAAAAAJiHRz+6q8cfP2wfzJ9AHAAAAABgHp71rK4ecMCwfTB/AnEAAAAAgHnYfvuu/uAHw/bB/AnEAQAAAADmaccdu3rKKcP2wfwIxAEAAAAA5umQQ7q6++7D9sH8CMQBAAAAAOZpYob4ihXJxRcP2grzIBAHAAAAAJinquT1r+8ev/e9w/bC3AnEAQAAAABWwT/8Q1c//vFh+2DuBOIAAAAAAKtg002Tm988Oe+85Fe/Grob5kIgDgAAAACwig4/vKuvfvWwfTA3AnEAAAAAgFX0mMd09Xe/G7YP5kYgDgAAAACwGnbfPTnppKS1oTthNgJxAAAAAIDVcJvbdPVb3xq2D2YnEAcAAAAAWA1vfGNX3/nOYftgdgJxAAAAAIDVcLvbdfX730+uvnrYXlg5gTgAAAAAwGp6/vO7+oMfDNsHKycQBwAAAABYTS96UVff9KZh+2DlBOIAAAAAAKtphx26+uMfD9sHKycQBwAAAABYAE9+clcvumjQNlgJgTgAAAAAwAJ4zGO6usceg7bBSgjEAQAAAAAWwDOf2dXvfz+54ophe2F66w3dAAAAAADAOKhKjjoqOf/8ZMMNh+6G6QjEAQAAAAAWyN/+7dAdsDIjs2RKVd2uqj5VVedU1VVVtaKqPlBVm8/zPn9XVSdU1cVVdUVVnVJVr62qDVZyzbpV9dyq+kFVXdhfd2ZVHVFV281wzbOq6mdVdWn/XCdU1e7z/bwBAAAAAFg7RiIQr6o7JTkpybOT/CzJ+5OcmeRlSX5cVVvM8T4HJDkyyb2TfCXJR5NcnuSAJN+oqvWnueamSY5P8okkmyT5TJIPJjkxyf2S3CgQr6r3Jjk0yVb9dZ9NskOSr1fVi+f4aQMAAAAAsBaNypIpH0lyyyQvba0dNLGzqg5M8ookb0/ygpXdoKruleS1SS5Kcu/W2pn9/urv/4IkL0ly4JRLP55k1yQvaK19fJr7rj/l4wck2S/JGUnu01q7sN//nnSh/nur6pjW2oq5fOIAAAAAAKwdg88Q72eH75ZkRZIPTzn85iSXJdm7qjae5VZ79PWQiTA8SVprLcnr+g9fNOW575XkaUmOmC4M76+/ZsquiWD+7RNheH/eRP83STfTHQAAAACAETJ4IJ7koX09vrV23eQDrbU/pVu65K+S3H+W+9y6r2dOPdAH1xcm2baqtpl06Gl9/Zeq2rSqntGvN/68qrrzDM+za1+Pm+bYN6ecAwAAAADAiBiFJVPu2tdfzXD81+lmkG+X5Dsruc/5fd1m6oGq2izJxJtz3jXJb/vH9+nr1umWQJm8Vnmrqo+mW8bl2v4+Gye5bZJLW2u/n6HXZJp1xwEAAAAAGNYozBDftK8Xz3B8Yv9ms9zn2L7uW1V3nNjZryH+9knnbT7p8S37emCSE5L8dbo31nx4uoD8hUneuJC99rPPl1fV8vPOO2+m0wAAAAAAWGCjEIgviNbaiUk+mS6MPrmqPl1V70vy0yTPSXJaf+rkZVkmPv/TkuzVWjuttXZpa+07SZ7Un/vKqtpgAfs8uLW2rLW2bMstt1yo2wIAAAAAMItRCMQnZlVvOsPxif0XzeFe+yZ5fpLTkzy5f3xJkl3SzfhOknMnnT9xz69PLIsyobX2H+mWVtkk3czxhe4VAAAAAIC1aBTWED+9rzOtu32Xvs60xvj1WmstycH99heqaod0M75/PuW575uZA+wL+7pRf//LqursJLetqq2mWUd8zr0CAAAAALB2jcIM8e/1dbeq+ot+qmqTJDsnuTzJT1b1CapqlyR3SHJsa23y+t/f7uvdprnmJrkh4F4x6dB3+/qoaZ7q0VPOAQAAAABgRAweiLfWzkhyfJI7JnnRlMNvTbJxksNba5dN7Kyq7atq+6n3qqqbTbNv6ySHJLk6yRumHP5yknOS7FVV951y7I3plkD5XmvtD5P2f6yvr6+q69+gs38jzxcluSrJp6f7XAEAAAAAGM4oLJmSJC9M8qMk/1xVD0tyapL7JXlouuVHXj/l/FP7WlP2f7IPwH+e5I9Jtkny+CTrJ9m7tXby5JP7JVD2SXJMkh9W1VFJzu6f+4Hp1ht//pRrflRVByZ5Zbo37zwyyQZJ9kpy8yQvaa2tWIWvAQAAAAAAa9DgM8ST62eJL0tyaLower8kd0rywST3b61dMMdbHZPkmiR7Jtk/Xah9ZJJ7tNaOmOG5/zXdOuJfT/LwJC9NsnW6meA7ttZ+Pc01+yV5dpI/JHlekmcmOSXJ41prH5pjrwAAAAAArEXVvQ8lQ1i2bFlbvnz50G0AAAAAAKyWqjqptbZs6D5mMxIzxAEAAAAAYE0TiAMAAAAAsCQIxAEAAAAAWBIE4gAAAAAALAkCcQAAAAAAlgSBOAAAAAAAS4JAHAAAAACAJUEgDgAAAADAkiAQBwAAAABgSRCIAwAAAACwJAjEAQAAAABYEgTiAAAAAAAsCQJxAAAAAACWBIE4AAAAAABLgkAcAAAAAIAlQSAOAAAAAMCSIBAHAAAAAGBJEIgDAAAAALAkCMQBAAAAAFgSBOIAAAAAACwJ1Vobuoclq6rOS/I/Q/exlt0iyflDNwELzLhmHBnXjCPjmnFkXDOujG3GkXHNOJo8rrdurW05ZDNzIRBnraqq5a21ZUP3AQvJuGYcGdeMI+OacWRcM66MbcaRcc04Wozj2pIpAAAAAAAsCQJxAAAAAACWBIE4a9vBQzcAa4BxzTgyrhlHxjXjyLhmXBnbjCPjmnG06Ma1NcQBAAAAAFgSzBAHAAAAAGBJEIgDAAAAALAkCMQBAAAAAFgSBOKscVV1u6r6VFWdU1VXVdWKqvpAVW0+dG+Mj6raoqqeW1VfqarfVNUVVXVxVf1bVT2nqqb9fldVD6iqb1TVH/trTq6ql1fVuit5rt2r6oT+/pdW1U+r6lmz9PesqvpZf/7F/fW7r+T8davqFX0/V/T9faOqHjD3rwrjqqqeUVWt3547wzkjOU6raqOqemtVnV5VV1bVuVX1xar667l/BRgXVfWw/vv2H/rXCOdU1beq6jHTnOv7NYtCVT22qo6vqrP6MXFmVX2pqnaa4Xxjm8FV1ZOq6qCq+mFVXdK/xvjsLNeMzdj1+mQ8zWdcV9VdqurVVfXdqvpdVV1dVf9XVUdX1UNneZ6RHKNVdfPqspcVdcPrrE9V1e1W9vkw+lble/aU6w+pG36evPMM54zsOK2FyBlbazbbGtuS3CnJ/yVpSb6a5J1Jvtt/fFqSLYbu0TYeW5IX9OPqnCSfS/KOJJ9KclG//8j0byQ86ZonJPlzkkuTfDLJe/px2ZJ8aYbneXF//PwkH07y/iS/6/e9d4Zr3tsf/11//oeTXNDve/E051eSL0367+Q9fX+X9v0+Yeivt224Lcnt+3H9p36MPHeac0ZynCa5SZJ/66/59yTvSvL5JNckuSzJ/Yb++trW3pbk3ZPG3MFJDkjyiSQ/T/LuKef6fm1bFFv/fW1i3B2S7rXvkUmuTnJdkmdMOd/Yto3EluSX/b/3n5Kc2j/+7ErOH5uxG69Pxnabz7hO8oX++ClJPp7u58mj+nHTkrx0hutGcowm2SLJ6f0130n3/6Ov9h//X5Jth/73sa2dsT3NtY+bdG1LcudpzhnZcZoFyhkH/0e0jfeW5Fv9oHzJlP0H9vs/NnSPtvHYkuzaf2NfZ8r+Wyf53368/d2k/TdLcm6Sq5Ism7R/wyQ/6s9/ypR73THJlf0LnDtO2r95kt/01+w05ZoH9Pt/k2TzKfe6oL/fHadc89T+mhOTbDhp/336fs9NssnQX3Pb2t/6FybfTnJG/6LkRoH4KI/TJK/tr/nS5P9W0/1QPfEDyDrz/brYFt+WZN/+3/zQJBtMc3z9SY99v7Ytii3da45rk/whyS2nHHtoP1bOnLTP2LaNzNaP0buke62xS1YeHI7V2I3XJ2O7zXNc75Nkx2n2PyTdLzWvSrLVlGMjO0bThfotyfum7H9pv/+4of99bKu+zWdsT7luy3SvU76Q5ITMHIiP7DjNAuWMg/8j2sZ3S/dbm5bkt9MM+k3S/WbpsiQbD92rbby3JK/rx+JBk/b9f/2+z0xz/q79se9P2f+P/f63TnPNtPdLcli//9nTXDPt/ZL8oN//0GmumfF+tvHfkrws3QzDByd5S6YPxEdynPYv1v6n37/NNNfMeD/beG3pZo+c24+HG4Xh05zv+7VtUWxJ7tf/mx89w/FLkvxp0sfGtm0kt8weHI7N2PX6ZOlss43rWa49PlMmWPX7R3KMJrlpksvTZS5TQ8t1kqzorzFLfAy2+YztJF9JF4hvkZUH4iM5TrOAOaM1xFmTJtbZOr61dt3kA621P6X7TdNfJbn/2m6MJeeavv550r5d+3rcNOf/IN035gdU1U3meM03p5yzStdU1YbpZhpcnuSH83gexly/7to7k3ywtfaDlZw6quP0TknukORXrbXfzqM3xs8j0s1OOSrJddWtt/zqqnpZTb/Gsu/XLBa/TjeL8L5VdYvJB6rqwel+UPv2pN3GNovVOI1dr0+Yi+l+nkxGd4zeP8lGSU7ss5fr9dnMt/oPV7o2OuOlqvZJskeS57fWLljJeaM8ThcsZxSIsybdta+/muH4r/u63VrohSWqqtZL8sz+w8kvVGYcn621P6f7jeN6Sbad4zW/T/ebyNtV1V/1z71xktsmubQ/PtV0/w3cKcm66f6keuoLrpmuYcz14/jwdMv/vG6W00d1nPp/AhPu09crk/wiyTHpftnzgSQ/qqrvV9WWk873/ZpFobX2xySvTnKrJP9dVQdX1Tuq6ovpZhf+a5LnT7rE2GaxGqex6/UJK1VVWyd5WLpw8AeT9o/yGDWu+Qv9OP5gulnkR89y+iiP0wUb2wJx1qRN+3rxDMcn9m+25lthCXtnkrsl+UZr7VuT9q/K+JzrNZtOqWviOTab4Tjj6U1JdkyyT2vtilnOHdVxamwz4ZZ9/Yd0f/L4oHQzZ++eLjR8cLq1Byf4fs2i0Vr7QJInpgsD903ymiR7pnuztUNba+dOOt3YZrEap7FrvDOj/q8cPpduube3tNYunHR4lMeocc31qmqdJJ9Jt5zIS+dwySiP0wUb2wJxYGxV1UuT7JfunYb3HrgdWCVVdb90s8Lf11r78dD9wAKYeP355ySPb639W2vt0tbafyb52yRnJXnIDMunwEirqlclOTLdG8beKcnGSe6d5Mwkn6uqdw/XHQBzVVXrpvsLzZ2THJHkvcN2BKvsFeneHHbfKb/UWdIE4qxJU3/LP9XE/ovWfCssNVX14nR/EvTf6d644Y9TTlmV8TnXay6eUtfEc1w0w3HGSL9UymHp/iTsjXO8bFTHqbHNhIv6+ovW2orJB1prl+eG9QLv21ffr1kUqmqXJO9K8rXW2itba2e21i5vrf083S97zk6yX1VNLCNhbLNYjdPYNd65kT4M/2y6v/D5YpJntP5d+yYZ5TFqXJMkqartkrw9yadba9+Y42WjPE4XbGwLxFmTTu/rTGv33KWvM639A6ukql6e5KAk/5UuDP/DNKfNOD77EHKbdLMXz5zjNVulmwV2Vh/opLV2Wboffm/aH59quv8GzkhybZJt+z7mcg3j66bpxttfJ7myqtrEluTN/Tmf6Pd9oP94VMep/ycwYWIsXDTD8YmZKxtNOd/3a0bd7n393tQD/Vj7Wbqfv3bsdxvbLFbjNHa9PuEvVNX6Sf4lyVOSfD7J06ZbR3nEx6hxzYS/Sbfkz7Mn/yzZ/zz5kP6cX/f79ug/HuVxumBjWyDOmjTxw8Bu/ZpF16uqTdL96dHlSX6ythtjfFXVq5O8P8kv04Xh585w6nf7+qhpjj043TsT/6i1dtUcr3n0lHNW6ZrW2pVJftQ//4Pm8TyMp6uSfHKG7Rf9Of/WfzyxnMqojtMz0r0p6HZVtc08emP8fCfd2uF/M/X1Qe9ufZ14h3rfr1ksbtLXLWc4PrH/6r4a2yxW4zR2vT7helW1Qbr3Mdkz3V9p7t1au3Yll4zqGP1JkiuS7NxnL9frX3vt1n94o1/gMnZWZOafJycmDn6p/3hFMvLjdOFyxtaazbbGtnR/9tySvGTK/gP7/R8bukfb+GzplpRoSZYnufks594syXnpAsdlk/ZvmO6bf0vylCnXbJPkyiQXJLnjpP2bJ/lNf81OU655QL//N0k2n7T/jv19rpx8r/7YU/trTkyy4aT99+n7PTfJzYb+etuG3ZK8pR8nz52yf2THaZLX9td8Kck6k/Y/od9/yuT9tvHdkhzd/5u/Ysr+3ZJcl26W+Kb9Pt+vbYtiS/Lkfjz8Icltpxx7dD+2r0iyRb/P2LaN5JZkl/7f/7MzHB+rsRuvT5bENodxfZMkx/bnHDKXf/NRHqNJPt4fe9+U/S/t9x839L+JbWG22cb2Sq47ob/uztMcG9lxmgXKGau/CNaIqrpTuhdFt0z3w++pSe6X5KHp/oThAa21C4brkHFRVc9K9wZW16ZbLmW6dx1e0Vo7dNI1e6R746srk3whyR+TPD7JXfv9T25TvklW1UuS/HO6FzhHpJvl9aQkt0v3TXz/aXp7X5JXpnujuCOTbJBkryRbpPsm/qEp51e6teqelO4NQb/en7tXuh80/q61dvRcvi6Mr6p6S7plU/ZtrR0y5dhIjtOqukm6WQEPSPeLq+8kuUO6WThXJ9m1tfbTVfqCsKhU1e3SvT64fbpx8It0YckeuSFI+fKk8/eI79eMuH6m0reSPDzJn5J8JV04/tfpllOpJC9vrX1w0jV7xNhmBPRjcY/+w1sneWS6JU9+2O87f/LYGqex6/XJ+JrPuK6qTyfZJ8n5ST6S7vXIVCe01k6Y8hwjOUaraot0r7W266/9Wbr/Hz0hXZj5gNbaGdN8jiwC8/2ePcM9Tki3bMpdWmu/mXJsZMfpguWMQ/8mwzb+W7ofdj+d5Pf9fwT/k+QDmfQbVJttdbfcMFt2ZdsJ01y3c5JvpJuNeEWS/0z3LszrruS5Hpfk++l+2L0syb8nedYs/e3Tn3dZf933k+y+kvPX6/v4z76vC/s+HzD019o2GltmmCE+6fhIjtN0f3r3j0l+nW52wXnpZhH8zdBfU9va3dItH3FQ/7rg6nQ/gH4lyX1nON/3a9vIb0nWT/LydH+qe0m6dZTPTXJMkt1muMbYtg2+ZfbX0iumuWZsxm68PhnLbT7jOjfMll3Z9pYZnmckx2iSmyf5YG54rfX7JJ9Kcruh/21sq7etyvfsae4xMeZvNEO8Pz6y4zQLkDOaIQ4AAAAAwJLgTTUBAAAAAFgSBOIAAAAAACwJAnEAAAAAAJYEgTgAAAAAAEuCQBwAAAAAgCVBIA4AAAAAwJIgEAcAAAAAYEkQiAMAwDxV1T5V1apqn6F7AQAA5k4gDgDAktYH2/PZ9hm659VVVW/pP5ddhu4FAADWpvWGbgAAAAb21mn2vTzJpkk+mOSiKcd+meS3SX6S5PdrsC8AAGCBCcQBAFjSWmtvmbqvnwW+aZIPtNZWzHDpxWuuKwAAYE2wZAoAAMzTTGuIV9WKfrtpVb2/qn5XVVdU1S+rao/+nPWq6vVV9euqurKqzqiqF6/kuR5ZVd+oqvOr6qr+/PdU1WbTnHv3qvqXvoerquq8qvp5VX2gqtaf6DHJm/tLvjd5OZhJ99muqt5ZVcv7e1xVVf9TVQdX1e2med5d+nu8paqWVdVxVXVxVV1YVV+uqtv3521bVV/o73lFVX2vqu4xzf0O7e+3bVW9sqpO679WZ/Vf15vN/q8EAAA3ZoY4AAAsrPWT/GuSmyc5OskGSZ6a5MtVtVuSFya5X5JvJrkqyZ5JDqqq81prR0y+UVW9OclbkvwxyTFJzk1y9yT7J3lMVe3UWrukP/fuSX6apCX5WrplXW6W5M79c74hyTVJPpBkjyQPSfKZJCum+RyemOQFSb6X5EdJrk7y/5I8N8njqmpZa+3saa67T5JXJ/l+kk8k2aG/192q6glJ/i3JaUkOS7J1f+xfq2rb1tql09zv/UkenOSL/dfykemWs3lQVT2wtXblNNcAAMCMBOIAALCwbpPk50l2aa1dlSRVdXiSHyT5UpIzktyttXZRf+zAdCHxa5JcH4hX1UPTheE/TvKYifP7Y/sk+XS69c9f0e9+VpINk+zRWjt6ckNVtXmSy5OktfaBfnb5Q5Ic2lo7YZrP4fAk75/of9J9dksX5L8hyd9Pc91jkjyjtfa5Sdd8Msn/ly5Yf19r7e2Tjr0xyT8meU669dqn2jnJPVtr/9Of/9p0X8MnJvmHJG+b5hoAAJiRJVMAAGDhvXxymNxa+2G6GdubJ3n15HC7tXZmkhPTzaJed9I9XtrXfSef319zaLo393z6NM99xdQdrbULW2vXzbX51trZU8Pwfv/xSU5JN1N7Ov82OQzvfaavFyd555Rjh/X1njPc74MTYXj//NelC8KvSxeyAwDAvJghDgAAC+ui1toZ0+w/J8k2SU6a5tjZ6V6b37p/nCQ7pVviZM+q2nOaazZIsmVVbdFauyDd7PKXJflqVR2Z5NtJTpyhl5WqqkoXtu+T5B7pgvzJYf3VM1y6fJp95/T1l621a6ccm/hcb7Quee/7U3e01s6sqt8luWNVbTb1lwUAALAyAnEAAFhYF8+w/89J0lqb7vif+7r+pH1bpHu9/uYbn/4Xbprkgtbaz6rqQUlen+RJSfZOkqo6PclbW2v/Mrf2kyQHplur+/dJvpUuuJ6Yeb5PuvW/p7Oyz+1Gx1prf+6y97/4vCf7vxn2/6HvYdMkF81wDgAA3IhAHAAARtPFSdZprd18rhe01n6cZPequkmSeyd5VJKXJPl8/6ad357tHlV1y3TLtfxXkge01v405fhT5/E5rK5bJTl9mv237utMv3wAAIBpWUMcAABG00+SbF5V/2++F7bWrmqt/ai19qbcsBb5EyadMrF0ybq5sW3T/Zxw/DRh+O3642vLQ6buqKptk9w+yQrLpQAAMF8CcQAAGE3v7+snquo2Uw9W1cZVdf9JHz+gqjaa5j636uvlk/Zd0Nc7THP+ir4+cPKbfFbVTZN8Imv3r0xfVlXXL89SVeskeU+6n2M+vRb7AABgTFgyBQAARlBr7TtV9Zok70jy66r6RpLfplszfOt0s6f/Ld2yKEnyqiS7VtUP+/MuTfL/kjw6yYVJDp50++8luS7JO6rqbv3xtNb+qbX2h6r6QpKnJPllVR2fbq3uRyS5Mskvk9xzDX3aU53Y93BEuuVRHpnuTT5PSvLutdQDAABjRCAOAAAjqrX2rqo6Md2yJw9Mt+zJxene5PLgJJ+fdPpH0gXb9+vPXS/JWf3+97XW/mfSfU+tqmcl2T/JC5Ns2B/6p74+J8mZSfZK8qIk5yX5WpI3Jfnygn+iM3tFkr9Nsm+SO6ab2f7BJG9qrV25FvsAAGBMVGtt6B4AAACuV1WHJnlWkm1aayuG7QYAgHFiDXEAAAAAAJYEgTgAAAAAAEuCQBwAAAAAgCXBGuIAAAAAACwJZogDAAAAALAkCMQBAAAAAFgSBOIAAAAAACwJAnEAAAAAAJYEgTgAAAAAAEvC/w8oufz3Vslw3QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1728x1080 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the accuracy changes\n",
    "plt.rcParams.update({'font.size': 20})\n",
    "plt.figure(1,figsize=(24,15)) \n",
    "plt.clf() \n",
    "plt.plot(acc,'-b',label='LightGBM Accuracy')\n",
    "\n",
    "plt.scatter(len(df1)-len(X_train),acc[len(X_test)-len(df2)],s=100,c='r')\n",
    "plt.text(len(df1)-len(X_train),acc[len(X_test)-len(df2)]+0.0005, 'test set starts', c='r')\n",
    "\n",
    "plt.legend(loc='best')\n",
    "plt.title('NSL-KDD Intrusion Detection Accuracy Change', fontsize=20)\n",
    "plt.xlabel('Timestamp')\n",
    "plt.ylabel('Accuracy')\n",
    "\n",
    "plt.draw()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The samples in the training set have similar statistical distribution. However, when the statistical distribution from the training set to the test set has changed. Hence, the model trained on the training set cannot accurately detect attacks in the test set.   \n",
    "* In the figure, it can be seen that when the test set starts, the current accuracy starts to drop significantly due to the statistical distribution change in the dataset.  \n",
    "* This kind of statistical distribution change is called **concept drift**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "tf36",
   "language": "python",
   "name": "tf36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}