Chapter 5.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chapter 5\n",
    "# Resampling Methods \n",
    "\n",
    "It involves repeatedly drawing samples from a training set and refitting a model of interest on each sample in order to obtain additional information about the fitted model. \n",
    "\n",
    "The process of evaluating a model's performance is known __model assessment.__\n",
    "\n",
    "The process of selecting the proper level of flexibility for a model is known as __model selection.__\n",
    "\n",
    "## Cross Validation :-\n",
    "\n",
    "In absence of large designated test dataset, we can use the cross validation approach to estimate the test error rate by holding out a subset from training data.\n",
    "\n",
    "### 1. Validation set approach :- \n",
    "It invloves randomly dividing the available set of observations into two parts, a training set, and a validation or hold-out set. The model is fit on the training set, and the fitted model is used to predict the responses for the observations in the validation set. \n",
    "\n",
    "The resulting validation set error rate, typically assessed using MSE in case of a quantitative response provides an estimate of the test error rate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 267,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import random\n",
    "from scipy import stats\n",
    "import matplotlib.pyplot as plt\n",
    "import statsmodels.api as sm\n",
    "from statsmodels.formula.api import ols\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "from sklearn.model_selection import LeaveOneOut\n",
    "from sklearn.model_selection import KFold\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.metrics import mean_squared_error \n",
    "from sklearn.metrics import make_scorer\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.linear_model import LogisticRegression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0MAAAENCAYAAADNDb7aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xd8VFX6+PHPmWTSewgQQiC00HtgsRfAXhCxYGd1WVl1rev63V236ve3rquuruWrrgWlKUqxK6uiKAiEntA7CSW9ZyZTnt8fE3qAIZnJTJLn/XrNa2bu3DP3ucYXZ557zn2OERGUUkoppZRSqq2xBDoApZRSSimllAoETYaUUkoppZRSbZImQ0oppZRSSqk2SZMhpZRSSimlVJukyZBSSimllFKqTdJkSCmllFJKKdUmNUsyZIyJMMYsM8asMcbkGmP+Ur99ujFmkzEmxxjzpjHG2hzxKKWUUkfSfkoppdom0xzrDBljDBAtIlX1HckPwP1AEvB5/W4zgO9F5BW/B6SUUkodQfsppZRqm0Kb4yDiybiq6t9a6x8iIp8d3McYswzo3BzxKKWUUkfSfkoppdqmZkmGAIwxIcAKoCfwkogsPeIzK3ArnqtwDbWdDEwGiI6OHt6nTx//B6yUUuqEVqxYUSQiKYGOw5e0n1JKqdbD236qWabJHXVAYxKAucB9IpJTv+11oFpEHjhV+6ysLMnOzvZzlEoppU7GGLNCRLICHYc/aD+llFItn7f9VLNXkxORMmAhcAmAMeZPQArwUHPHopRSSh1L+ymllGo7mquaXEr9lTaMMZHAGGCjMeYu4GJgooi4myMWpZRS6ljaTymlVNvUXPcMpQJT6+djW4D3ReQTY4wT2AUs8RTyYY6I/LWZYlJKKaUO0n5KKaXaoOaqJrcWGNrA9mYr4KCUUkqdiPZTSinVNjX7PUNKKaWUUkopFQw0GVJKKaWUUkq1SZoMKaWUUkoppdokTYaUUkoppZRSbZImQ0oppZRSSqk2SZMhpZRSSimlVJukyZBSSimllFKqTdJkSCmllFJKKdUmaTKklFJKKaWUapM0GVJKKaWUUkq1SZoMKaWUUkoppdokTYaUUkoppZRSbZImQ0oppZRSSqk2SZMhpZRSSimlVJukyZBSSimllFKqTdJkSCmlVJtnjIkwxiwzxqwxxuQaY/5Sv72bMWapMWaLMeY9Y0xYoGNVSinlO5oMKaWUUmAHLhSRwcAQ4BJjzCjgKeA5EekFlAJ3BjBGpZRSPqbJkFJKqTZPPKrq31rrHwJcCHxQv30qMC4A4SmllPKTZkmGdPqBUkqpYGeMCTHGrAYKgAXANqBMRJz1u+QBaYGKTymllO8118iQTj9QSikV1ETEJSJDgM7ASKBvQ7s11NYYM9kYk22MyS4sLPRnmEoppXyoWZIhnX6glFKqpRCRMmAhMApIMMaE1n/UGdh7gjaviUiWiGSlpKQ0T6BKKaWarNnuGdLpB0oppYKVMSbFGJNQ/zoSGANsAL4FJtTvdjswPzARKqWU8odmS4Z0+oFSSqkglgp8a4xZCywHFojIJ8BvgYeMMVuBZOCNAMaolFLKx0JPvYtviUiZMWYhR0w/qB8dOun0A+A1gKysrAYTJqWUUqqxRGQtMLSB7dvxXMBTSinVCjVXNTmdfqCUUkoppZQKKs01MpQKTDXGhOBJwN4XkU+MMeuBWcaYJ4BV6PQDpZRSSimlVDNplmRIpx8opZRSSimlgk2zFVBQSimllFJKqWCiyZBSSimllFKqTdJkSCmllFJKKdUmaTKklFJKKaWUapM0GVJKKaWUUkq1SZoMKaWUUkoppdokTYaUUkoppZRSbZImQ0oppZRSSqk2SZMhpZRSSimlVJukyZBSSimllFKqTdJkSCmllFJKKdUmaTKklFJKKaWUapM0GVJKKaWUUkq1SZoMKaWUavOMMenGmG+NMRuMMbnGmPvrtw8xxvxkjFltjMk2xowMdKxKKaV8JzTQASillFJBwAk8LCIrjTGxwApjzALgH8BfRORzY8xl9e/PD2CcSimlfEiTIaWUUm2eiOwD9tW/rjTGbADSAAHi6neLB/YGJkKllFL+oMmQUkopdQRjTAYwFFgKPAB8aYz5J56p5WeeoM1kYDJAly5dmiVOpZRSTXfKe4aMMSHGmL8aY8KbIyCllFLqdPiynzLGxAAfAg+ISAUwBXhQRNKBB4E3GmonIq+JSJaIZKWkpDQpBrdbmtReKaWU906ZDImIC7gHcPg/HKWUUur0+KqfMsZY8SRC00VkTv3m24GDr2cDfi2g8J9F25nwf4s1IVJKqWbibTW5qcDdjT2IVulRSinlZ03tpwyeUZ8NIvLsER/tBc6rf30hsKXREXohOSaMlbvL+CJ3vz8Po5RSqp639wyNBO4zxjwK7MFzQykAInKuF+21So9SSil/amo/dRZwK7DOGLO6ftvvgF8AzxtjQgEb9fcF+ctVg9N46dttPLtgMxf370iIxfjzcEop1eZ5mwy9Xv9oFK3So5RSys+a2k/9AJwo8xje2O89XSEWw4NjMrlnxko+XrOXcUPTmuvQSinVJnmVDInIVF8dUKv0KKWU8jVf9lOBdumAjvTpGMvzX2/hikGphIbo+uhKKeUvXv8La4yZZIz5xhizqf550ukeLBiq9CillGqdfNFPBQOLxfDQ2Ex2FFUzZ1V+oMNRSqlWzatkyBjze+AxYBbw6/rnR+u3eyUYqvQopZRqnXzRTwWTsf06MKhzPC98vYU6pzvQ4SilVKvl7cjQXcBF9SM0X4rIa8AleHkjabBU6VFKKdVqNamfCjbGGB4cm0leaS2zV+wJdDhKKdVqeVtAIRooPGZbMRDpZfugqNKjlFKq1WpqPxV0zs9MYXjXRP799VauHdaZCGtIoENSSqlWx9uRoS+A6caY3saYSGNMHzxrOnzpTWMR+UFEjIgMEpEh9Y/P6rcPF5HBIvIzEVnR2BNRSinVpjWpnwpGxhgeHpvJ/gobM5ftDnQ4SinVKnmbDN0LVAJrgCpgNVAN3OenuJRSSqnT0Sr7qTN7tmNU9yRe+nYbtXWuQIejlFKtzimTIWOMBcjCM6UtCkgFokTkNhEp83N8Siml1Em19n7q4Yt6U1Rl592fdgY6FKWUanVOmQyJiBuYLyJ2EXGLSEH9NqWUUirgWns/NSIjiXN6teP/vttOld0Z6HCUUqpV8Xaa3PfGmFF+jUQppZRqvFbdTz18UW9KquuYunhnoENRSqlWxdtqcruAz40x84E9gBz8QET+6I/AlFJKqdPQqvupIekJjOnbnle/28Yto7oSH2kNdEhKKdUqeDsyFAnMw9O5dAbS6x+d/RSXUkopdTpafT/14NhMKmxO3vhhR6BDUUqpVuOUI0P1N6a+C/woInb/h6SUUkp5r630U/07xXPpgI68+cMOJp2ZQWJ0WKBDUkqpFu+0Cig0QzxKKaXUaWlL/dSDYzOprnPy2qLtgQ5FKaVaBS2goJRSqjVoE/1UZodYrhzUibd/3ElRVavP/ZRSyu+0gIJSSqnWoEn9lDEmHXgH6Ai4gddE5Pn6z+7Ds6irE/hURB71ffjeu39MLz5Zu5dXFm7j8Sv6BTIUpZRq8bxNhg7emAqt6GZUpZRSrUZT+ykn8LCIrDTGxAIrjDELgA7A1cAgEbEbY9r7JtzG65ESw/hhnZn20y4mn9udDnERgQ5JKaVaLK+SIRGZ5O9AlFJKqcZqaj8lIvuAffWvK40xG4A04BfA3w/ejyQiBU2N1RfuH92Leavyeenbrfz16gGBDkcppVqsk94zZIy57pj3vY95/4A/glJKKaW84Y9+yhiTAQwFlgKZwDnGmKXGmO+MMSNO0GayMSbbGJNdWFh4uoc8belJUVyXlc6sZXvIL6v1+/GUUqq1OlUBhTeOeb/kmPd/9WEs/jV9OmRkgMXieZ4+PdARKaWUajqf9lPGmBjgQ+ABEanAM4MiERgF/AZ43xhjjm0nIq+JSJaIZKWkpJzOIRvtvgt7AvDiN1ua5XhKKdUanSoZOvYf/FO9D07Tp8PkybBrF4h4nidP1oRIKaVaPp/1U8YYK55EaLqIzKnfnAfMEY9leIortGtssL7UKSGSiSPTmZ2dx67i6kCHo5RSLdKpkiE5zffB6fe/h5qao7fV1Hi2K6WUasl80k/Vj/a8AWwQkWeP+GgecGH9PplAGFDUuFB9754LehJiMTz/tY4OKaVUY5xynSHjYTHGhDT0vkXYvfv0tiullGoxfNRPnQXcClxojFld/7gMeBPobozJAWYBt4uIXy8Ens7Xt4+L4LYzujJvVT5bC6r8GJVSSrVOp0qGYvCUG3UAdUDCEe8dQLRfo/OVLl1Ob7tSSqmWwif9lIj8ICJGRAaJyJD6x2ciUicit4jIABEZJiLf+OtEAPL3vsfqNZNwu51et7n7vB5EWEN0dEgppRrhVMlQN6D7EY9uDbw+JWNMujHmW2PMBmNMrjHm/iM+u88Ys6l++z8adRan8uSTEBV19LawMM92pZRSLZlP+qlgYTFWSkoWsXXbU163SY4J544zM/hk7V427a/0Y3RKKdX6nHSdIRHZ5aPjBHYxu5tv9jz//veeqXFhYRAeDldf7ZfDKaWUah4+7KeCQsf24yjdv5I9e94kLnYgHTte5VW7yed2590lu3huwWb+79bhfo5SKaVaj1PeM+QLIrJPRFbWv64EDi5mN4XmWszu5pth505wu2HhQqiogGefPVUrpZRSqtmUzd9G3McXEx8znA0b/4fKqo1etUuICuPnZ3fji9z95OSX+zlKpZRqPZolGTpSUCxmN2oUXHstPP00FATFYuJKKaUUMWengd2QvutBrKHxrFs7BYejzKu2d57TjfhIK88u2OznKJVSqvVo1mQoqBaz+9//hdpa+GvLWTdWKaVU62ZtH0XsuZ1xrHCSmfR3bPZ95OY+iIjrlG3jIqxMPrc732wsYOXu0maIVimlWr5mS4aCbjG7zEzPwquvvgpbtAKPUkqp4BB7QTohSRG4P48is8fjFJd8z/bt//Kq7R1nZpAcHcZzOjqklFJeOWEBBWPMIrxYrE5Ezj3VPl4sZrcwIIvZ/elP8M478LvfwezZzXZYpZRSTefLfiqYWMJCSLiqB8Vv5xK79Ww6pV7Pzl0vExc3kJSUi07aNjo8lLvP68GTn21g2Y4SRnZLaqaolVKqZTrZyNB/8CQwbwAL8ZQnXQRMA77HU7L0Wy+PEzSL2R2lQwd45BH44ANYurTZDquUUsonfNlPBZXIPklEDmxH5Td59Gj3W+JiB5G7/jdUV287ZdtbRnUlJTacZ77adFoLuCqlVFtkvPmH0hjzE3CniOQesa0f8KaIjPJjfMfJysqS7Oxs331hZSX07Al9+niqzB1/y5JSSqljGGNWiEhWoOM4qDX2U65yO/ufWUFYRhwxExNZnj0OqzWBEVlzCA2NPWnbt3/cwZ8/Xs/0u37GWT2bZ/a5UkoFE2/7KW/vGeoLHHs5agfQ53QDCzqxsfDnP8P338OnnwY6GqWUUo3T6vqpkPhw4i7qin1zKbIljIEDXqS2dhfr1/8GEfdJ2944sgup8RH8U0eHlFLqpLxNhr4D3jbG9DLGRNbf3/MGnukILd9dd3kKKjz2GLhOXbFHKaVU0GmV/VTMGZ2wdoqm7OPtxEcOp2fP/6GwaAE7d71y0nYR1hDuu7AXq3aXsXCTD5akUEqpVsrbZOiO+udcoBpYBxhgkh9ian5Wq6fUdm4uTJ0a6GiUUkqdvjvqn1tVP2VCDInX9MJdVUfFV7tI73wHHTpcxfbtz1FUvPCkba/L6kx6UiTPLtiso0NKKXUCXiVDIlIiIjcCEUAqECkiE0Wk+Sq/+dv48Z7FWP/4R6ipCXQ0SimlTkNr7qfC0mOJHpVK1ZK9OPKr6Nvnf4mJ6UNu7oPU1Ow6YTtriIVfX9iLdfnlfLX+QDNGrJRSLYfX6wwZY/oCvwceFxG3Maa3MWaQ/0JrZsbAP/4B+fnwwguBjkYppdRpas39VPzFGVhirJTO24rFRDBo4MuAYd26KbhcJ76Ad83QNLq3i+a5BZtxu3V0SCmljuVVMmSMuQ5PmdI04Lb6zbHAsyds1BKdcw5ceSX8/e9QXBzoaJRSSnmptfdTlohQEq7ojiOviuqf9hEZ2YUB/f9FVfVmNmz4nxNOgwsNsXD/mF5s3F/JZzn7mjlqpZQKft6ODP0VGCsidwMHKwysAQb7JapA+vvfPeW2n3wy0JEopZTyXpP6KWNMujHmW2PMBmNMrjHm/mM+f8QYI8aYgNWpjhyUQnivBMq/3Imrwk5y8rn06P4wBwo+Yc+eN0/Y7opBnejVPobnFmzGpaNDSil1FG+TofZ4OhU4vNq34MXK3y1Ov34waRK8+CLs2BHoaJRSSnmnqf2UE3hYRPoCo4B76tcpwhiTDowFdvsu3NNnjCHh6p6Iy03ZJ9sB6Nr1blJSLmbrtqcoKV3SYLsQi+GhsZlsK6xm/ur85gxZKaWCnrfJ0Arg1mO23Qgs8204QeIvf4HQUPjDHwIdiVJKKe80qZ8SkX0isrL+dSWwAc+UO4DngEcJgguA1naRxJ2fTu3aImybSzHG0K/vP4iM7EZOzq+x2fY22O7i/h3plxrH819vweE6+RpFSqmWw76rgurl+7ViZBN4mwz9GnjCGPMdEG2M+RL4G/Cg3yILpLQ0eOABmDEDVq4MdDRKKaVOzWf9lDEmAxgKLDXGXAXki8iaU7SZbIzJNsZkFxb6d12f2PPTCW0XSen8rYjDRWhoDIMGvoLbXcfadb/C5bIf18ZSPzq0q7iGOSvz/BqfUsr/3DUOSudsofCVNZR+uIXS9zYhTr3Q0RjeltbeiGcV75eAPwBvAQNFZIsfYwus3/4WkpM9z0oppYKar/opY0wM8CHwAJ6pc78H/ujF8V8TkSwRyUpJSTnd8I/idp/8B40JtZAwrgeuYhsV3+4BIDq6O/37PUNl5To2bf5jg1eJR/dtz+D0BF74eit1+qNJqRZJRKhZXcD+Z1dQnb2fmHPSiBvblZrVhRS9lYPb5gx0iC3OKZMhY0yIMWYb4BKR90XkaRGZJSJVzRBf4MTHw+OPw3//C199FeholFJKnYCv+iljjBVPIjRdROYAPYBuwBpjzE6gM7DSGNPRx6dwyIoVK5g2bRpO58l/0ET0TCRqSAqV3+XhKPCU1k5JGUNGxr3s2/cB+fkzjmtjjGd0KL+slvey9/glfqWU/ziLayl6M4eSWZsISYyg/b1DSbi8O3Gju5B4fSb2HRUUvLIGZ9nxo8PqxE6ZDImIC09lnkj/hxNk7r4bunXzjA6d4kqdUkqpwPBFP2WMMcAbwAYRebb+e9eJSHsRyRCRDCAPGCYi+30QdoNCQ0PZvn078+fPP+UIUfzl3TFWC2Xzth4aCere7X6Sk89n85a/UVa+4rg25/ZqR1bXRF78Zgs2h+u4z5VSwUecbiq+3c3+51ZSt7uShKt70H7KYMI6xRzaJ3pYB9r9vD+uMjuFL6+mbl91ACNuWby9Z+hfwHvGmPOMMT2MMd0PPvwZXMCFh3tKbK9e7bl/SCmlVLBqaj91Fp4CDBcaY1bXPy7zX7gNGzx4MKNHj2bdunV88803J903JDaM+Eu6Yd9eTs1qz31Kxljo3+9ZIiJSWbfuHuz2gqPaGGN4+KLeHKiwM31pQIvjKaW8YN9ZzoEXVlHx5S4i+ybR8aHhxJzRCWMxx+0b0TOR9lM8qwkU/t8abFtKmzvcFsl4U33CGHOiy1MiIiG+DenksrKyJDs7u/kO6HbDyJFQVAQbN0JERPMdWymlgpQxZoWIZAU6joNaUz/ldLj4/IvPWLFiBZdffjkjRow44b7iFgpfWYOzxEbHh4djibICUFW1ieXZ1xIb25dhQ6djsYQd1e6m139i84FKvn/0AqLCQhsdq1LKP9w1Dsq/2En1sv2EJISTcHUPIvsme9XWWW6n+K0cHAW1JF7bi+jhHfwcbXDytp/ytoCC5QSPZu1gAsJigaeegl274OWXAx2NUkqpBrSWfmr1f3cz95lVjD7/IjIzM/nss8/YtGnTCfc3FkPCNT1x1zoo/3Lnoe0xMb3p1/cpystXsnnL8YuIP3xRJkVVdbyzZJc/TkMp1UjHFUg4N40ODw33OhECCI0PJ+XuwYR3j6d09mYqvt6tpbdPwttpcm3b6NFw8cWeKXNlZYGORimlVCsVnxJJUV4lH7+whisuuZrU1FRmz55NXt6Jy2GHdYoh5sw0qpfux76r4tD2Dh0up0uXu8jPn8befR8c1WZ41yTOy0zh1e+2UWlz+O18lFLecxbVUvTGMQUSLuuOJez0r+lYIkJpd0d/ooa2p2LBLsrmbEVcmhA1xKtkyBgTaoz5tTHmQ2PMd8aY7w8+/B1g0HjqKSgthb//PdCRKKWUOkZr6ae6DU7hsrsHUbqvhs9eyuGaKycQExPDjBkzKCkpOWG7uLFdCIkPo2zu0T94enT/DYmJZ7Jp0+NUVKw9qs1DYzMprXHw1o87/XU6SikviNNNxTe72f+vFdTtabhAQmOYUAuJ12cSe0E61cv3U/xOLm67Fk45lrcjQ88BvwS+B4bjKT3aHjj53Z31jDHpxphvjTEbjDG5xpj7j/n8EWOMGGPanUbszWvwYLjlFvjXv2CPliRVSqkg06R+Kph0HZDM5fcMoryglgWvbubaq69HRJg2bRrV1Q1XiLKEh5JwZQ8c+6upWpx/eLsllAH9nyfM2o61635FXV3xoc8Gpycwpm8HXl+0nfIaHR1SKhAOFUj4aheRfZPp+PCJCyQ0hjGG+IszSLimJ7bNpRS+thZXZZ1Pvru18DYZGg9cKiLPA87653HABV62dwIPi0hfYBRwjzGmH3gSJWAsEPxlbf72NxCBP55y/T2llFLNq6n9VFBJ75vEFfcNprLUzvdv72bcFRMoLy9n5syZOBwNJy4R/ZOJ6JNExYJdOMtsh7aHhSUxcODLOBzF5OTej9t9eA2jh8ZmUmlz8p8ftvv9nJRSh7lrHJR+uIXC/1uL1LlIvqM/yTf3JSQu3C/Hi/lZKsm398dZUEPBy6sPrU+mvE+GooCDwyG1xpio+tW+h3rTWET2icjK+teVwAYgrf7j54BHgeCfyNi1K9x3H0ydCuvWBToapZRShzWpnwpGaZmJXHXfYKrL7SybeYBLL7qSvLw8PvzwwwbXIDLGkHBVDxAo++jo5CYubiC9e/+N0tIlbNv+9KHt/TrFcfnAVN78YQcl1Xq1WCl/ExFqVhWw/5kVVK/YT8y5nT0FEvok+f3YkX2SSPnlIMThpuCVNdh3lvv9mC2Bt8nQBuBgbc9s4M/GmD8A+Sdu0jBjTAaezmmpMeYqIF9E1pyizWRjTLYxJruwsPB0D+lbv/sdxMfDY48FNg6llFJH8lk/FVAi4Do88pPaM4Gr7x+KrdpJ7vxqzjt7NBs3buTLL79ssDpUaFIEsaO7YFtfTO364qM+65Q6gbS0W9i9+z8cOPDJoe0PjOlFjcPFq99v8995KaUOF0h4bxMhSQcLJHRrVIGExgrrHEv7KYMJibZS+J911KwN8O/qIOBtMnQ/nqluAA8Bw4ArgcmnczBjTAyeedwP1H/f74FTzjkTkddEJEtEslJSUk7nkL6XlORJiD77DBYuDGwsSimlDvJJPxVwy16Dty6D8sM5XIducYx7cCgOu4td/7UwdFAWS5cuZcmSJQ1+Rew5aYR2iKLso224646+WTqz1++Jjx/O+g2PUVXlKdndq0MsVw/uxNTFOymotDX0lUqpJvBXgYTGCk2OJGXKYMLSYimZuZHKRS3rmpGvebvO0PIjprltEZExIvIzEVnk7YGMMVY8idB0EZkD9AC6AWuMMTuBzsBKY0zH0z2JZnfffZCeDo8+6rmKp5RSKqB80U8FhZgOULAeXj0Xtn93aHNKl1jGPTgMt8tN8U/x9OyeyVdffUVOTs5xX2FCLCSO64mrzE7F10ffjmuxhDFwwIuEhsaydt3dOByeaTL3j8nE4RJeWaijQ0r5kn1HOQdeWOm3AgmNFRJtJeWuAUT2T6b80+2UfbwNcbfN37Telta+8EQPL9sb4A1gg4g8CyAi60SkvYhkiEgGkAcME5H9jTyX5hMR4SmmsHw5zJ4d6GiUUqrNa2o/FTT6j4NffAtRyfDuOFj0LNTfH9SucwzjHhyGwWBf14nUDmnMnTuXXbuOXzg1vFs8UVkdqFqUj2P/0RXowsPbM3Dgi9hs+8hd/xAibrq1i2b80DSmL93NvvLaZjlVpVqzQwUSXl2L1Ln9XiChMYw1hKSb+hJzdhpVP+6lZPoGxNH2Sm8bb1akNcbsOGZTChAG5IlIdy/anw0sAtYBB+/6/J2IfHbEPjuBLBEpOtl3ZWVlSXZ29ilj9juXC4YOhZoaWL8ewsICHZFSSjUbY8wKEckKdBwHNbWf8iWf9FP2Kvj415DzIfS+DMa9ApEJAJQdqGHec6uoc9ip6bwOm72WO++8k2OnkbuqHRx4JpvQlChSfjnouCvReXnT2bT5j2Rk3EuP7g+yp6SGC/65kBtHpvPEuIFNi1+pNkpEqFldSPkn23HXOog5uzNxY7o0631BjVH5Qz7ln24nrEscybf1IyTaGuiQmszbfsrbaXLdjnwA8cCTwItetv9BRIyIDBKRIfWPz47ZJ+NUiVBQCQnxLMS6bRu89lqgo1FKqTatqf1UsHDb7dTm5EJ4DFz7Blz6D9jyFbx2Puz3VDFN6BDFNQ8PJTw8nIi8PhgsTJs2jcrKyqO+KyTaSvxl3ajbVUHNigPHHSst7SZSUyewc+eLFBYuID0pihtGpPPe8j3klWrZ3bbMmwvl6niO+gIJpe9tIjRABRIaK/bsNJJu6ktdfiWFr6zBWdx2Roi9GhlqsKExoXiuuDXrPT5BMzIEnvuFRo+GnBzYuhXi4gIdkVJKNYtgGxlqSEvsp/b+4Q9UfvElGe+/T3j3bp6Nu5fC7DugtgSueA6G3ARARXEt859bRXltMWUJa0hpn8Idd9xBePjhaTjiFgpfW4uzoIYODw0nJOboWQwul50VK2+gpmYHI7LmUuF7mvAGAAAgAElEQVRM5bynF3LNkDSemjCoUeeggp+4BFeFHWeJDVeprf7ZjrPUhqvEhquqDmtqDBGZiURkJhLWJRYT4m3NrbZHnG4qv8uj4tvdmBAL8ZdkEP2z1IDfF9QY9p3lFL+zHiyGdrf3Jyw9NtAhNZq3/VRTkqFLgTdEpFOjvqCRgioZAsjOhhEj4PHH4a9/DXQ0SinVLFpIMtTi+ilHfj47JlxHSFISGe+9R0hMtOeDqkL4YBLsXATD74BLngJrBFWlNuY9t4ri6r2UxeXQo0cPJk6cSEjI4SvRjgPVHHh+FVFD25N0XeZxx7TZ9rJs+dVYrUmMyPqQJz7bxbs/7eLrh84jo110o86jLdi/fz8ffvghnTt35rLLLsNqDZ5pReIW3FWOQ8mNs8TmeV1qw1lqx1VmhyNvljcQEhdOSFI4oYkRWGLCqNtdQd3uCnCDCQ8homcC4b09yVFoQkTgTi7I2HeUUzp3C86CWiIHtSPhiu5BdV9QYzgKayh6Mwd3lYOkm/oQ2Tc50CE1ik+TIWPMHo5eFDUKiAB+JSLvNDrKRgi6ZAjgxhvh4489o0OpqYGORiml/C7YkqHW1E9V/7SU3XfeSeyFF5D2/PMYS/0VeZcTvn0SfngWOg2F66ZCYleqy+3Mf24VB6q2Ux6zmWHDhnHllVfiqV3kUf7FDioX5pEyeSDh3ROOO2ZJ6RJWr76ddu3G0KHrM5z39EIuHZDKczcMafR5tGYbNmxgzpw5hIaGUltbS1paGjfccANxzTRDRERw1ziPGNXxJDlHvsZ59MK8lhgroUkRhCRG1D97Ep/QpAhC4sMxoceP/LhtTuxby7BtLsW2qRRXuR2A0PaRRGQmEZGZSHi3eIy17Y0auWsclH22g5rsA4QkhJMwrmezLJzaXFyVdRS9nYtjbxUJ43oS87OW9/vW18nQecdsqgY2i0hFI+NrtKBMhrZtg759YdIkePXVQEejlFJ+F4TJUJP6KWNMOvAO0BFPoZ/XROR5Y8zTeNYrqgO2AZNEpOxk3+WLfqr4rbcpeOopUh54gHZ3//LoDzd+CnOngMUC4/8DvcZQU1HH/H+tIq8ql+qo3VxwwQWcd97h/yTuOhcHnluBsVro8OthDf7w3b37DbZs/V96dH+EGevP5/VF21nw4Ln0bN9yp8n4moiwaNEivvnmm0MJUH5+PnPnziUsLIwbbriB9PR0nxzLbXfiLLF7RnZKbcclPmI/uuqXiQwlNCmC0MRwQpIiCE08IvFJCG/yfSsigrOwFtumUmybS7DvKAenYKwWwrvHE14/pS60XeRRiXhr01ILJDSG2+6iZMYGbJtKiT0/nbiLu7aov63fp8kFSlAmQwC//jW8/LLn/qE+fQIdjVJK+VWwJUNNZYxJBVJFZKUxJhZYAYzDswbeNyLiNMY8BSAivz3Zd/minxIR9j7yGyo++4z0114l5pxzjt6heBu8fxscyIXzH4NzH6W2xsn8f61iZ9VKbBEHGDduHEOGHB7Zqd1YQvHbucRdnEHcBcf/YBcRcnMf4EDBp3TPfJXLX3NwQZ/2vHTTsCadS2tRV1fHRx99RE5ODgMHDuSqq646NDXuwIEDzJo1i4qKCq644gqGDh16yu8Th8szmnNwKlup/aiEx13jPGp/E2Y5lNwcTnTCD22zRIT65bxPxF3nwr6jHPumUmybS3EWeW64D0kMr7/XKInwnvFYwps3Ln9yFNVSNm8r9q1lhKXHknBNz4AtnNpcxCWUzd9K9bL9RA1JIXFCZoMXU4KRr0eG3uXo6QcNEpHbvAuv8YI2GSoshB49PAUV5s4NdDRKKeVXwZYM+bqfMsbMB14UkQVHbLsGmCAiN5+sra/6KXdtLTsn3oRj7166fTCbsC5djt6hrgY+fRjWzICeY2D869gklvnPr2Rb1RIc4RXccsvN9OjR41CT4mnrqd1YSscHhxGaHHncMV2uGrJXXIfNto81tc/y7LfVfH7/OfRNbdsFgioqKpg5cyb79u1jzJgxnHXWWcddIa+pqeGDDz5g+/btjBw5kovGjIVKZ32y40l6Dic+NtyVjqMPEmI8Sc7B0Z0jE5+kCCxRoUF9Vd5ZXIttSym2zWXYt5YhdS6wGMIz4g6NGllTo4P6HE7kuAIJl2YQPbJlFkhoDBGhcuEeKr7cRXiPeJJv7dfsyXdj+DoZ+jdwO/AxsAvogmfawFSg+OB+IvKXxgbsraBNhgCefBL+8Af44Qc466xAR6OUUn4ThMmQz/opY0wG8D0w4MhpdsaYj4H3RGRaA20mA5MBunTpMryhhVAbo27PHnZMuA5rhw5kzJqJJSrq6B1EYMXb8PmjENMBrp+KPWkw819YzqbKHzARddx115107OgpqOcst3PgmRWEd4sj+Y7+Df4wranZxfLscVjDOnHvV3eTlZHKa7cFzZ+62eXl5TFr1izq6uq49tpr6d27NwDOkhLyfv0I4ggnrFt/Qtt3RSKTWFK5nTU120h1JzK6bgAR1Ffws0BIfPjR09fqE5/QxAgssWGt5se1ON3Yd1Vg3+wZNXLs8yz8a4m1EtErkYjeiYT3TGwRa9nYt5dTOu/IAgk9CIlrm2tLVq88QOkHWwhNiaTdpAGEJgR3oQhfJ0NfAk+IyKIjtp0NPC4iFzcp0tMU1MlQdTX06gXdunkSohZ49UMppbwRhMmQT/opY0wM8B3wpIjMOWL774EsYLycouP0dT9V9cOP7Jk8mbhLLqbTM880fGU9fyW8fztU7YdL/0Fd/1uY++IyNlV8R3hUKHf/6pfEx8cD9YsrfrKdpJv7EDUw5fjvAoqLv2P1mjspk/N5eME4Pr73HAZ2jvfZObUUa9as4aOPPiIuLo6JEyfSvn17wDNqt2fKH7G0G4sJPTzC5q4txV1bzNboMpbElRMVEsa4gWeQfsZwrCmxmJC2+bvAVWHHtrkM25ZS7FtKPVMADYR1jj00ahSWHhtUyaCr2kH55623QEJj2baWUvzuBizhISRPGkBYavBWnPR1MlQOtBMRxxHbrECxiDTr2HlQJ0MAr78Okyd7psqNGxfoaJRSyi+CMBlqcj9Vv/8nwJci8uwR228H7gZGi8gpVyP1Rz9V9PrrFD7zLO1/8xuS7/x5wzvVlMCcX8DW/8Lgm3CM+QcfvLqSzRXfExsbx5R7JxMZGYm4hIIXV+GqdtDxoeEnnO6yY+dLbN/+LPO2T6Ay5FrenjTSp+cUzNxuN19//TU//vgjGRkZXHfddURHe370ictF/iP/RqxDCIkzJF43kJDEcKgrx75pA7U5Odhyc8nbvYdFgwZSFxbGz5Zn0zMykogB/YkcMICI/v0J790bS1jwjTC4XDWUli4lMXEUISHHT6VsKnELdXmVh0aN6vZUgngKQET0Sji0tlGgylOLCDWrCij/dDvuWicx53QmbnTrLJDQWI791Z7S23YXybf2JaJnYqBDapCvk6GFwHLgjyJSa4yJBP4CjBKRc5sa7OkI+mTI6YSBAz2v162D0OCfU6mUUqcrCJOhhTShnzKe4ZapQImIPHDE9kuAZ4HzRKTQm1j80U+JCPkPPEjlggV0+c/rRJ95ZsM7ut3w/T9g4d+hQ3+c17zNezN2sqV8MSmJHfnlvXcSGhqKfXcFha+sIebMTiRc2aPBrxJxs27drygo+pqnl9/D/954O8O7BuePHl+y2WzMmTOHzZs3k5WVxaWXXnpo3SZxC3v/PAup64wlpoaOvxl9wgIBIkLptm3Mnj+ffZWVDK6ooN/iJbjL6osRWq2E9+pJZH9PchQxYAARmb0wAUqQqqo2kb93Jvv2zcXlqiI+fhiDB72O1Xp8KXZfclU7Dpfv3lyKu7IOAGvH6EOjRuEZcc1y076jsMZTIGFbuadAwvheQT3yEUjOcjvFb+XgKKglcUIvood1CHRIx/F1MpQBzMAzRaAUSASygZtEZGdTAj1dQZ8MAcyf7xkVevVVzyiRUkq1MkGYDGXQhH6qfkrdImAdntLaAL8DXgDCOXzf0U8icvfJvstf/ZS7upqdN96Is7CIjA8+IKxz2ol33vJfmHMXuN24rnyFaV8KOyqzSU/pwc9/dQvGGErnbaV66T7a3zuUsLSGK2I5nZUsXT6eovJCPt33JK9Nutzn5xVMSkpKmDlzJkVFRVx66aWMHHl4NMxd5+LA0wtwVUZjCdtL6p+ux25zYrEYwiJPfOHT6XTy6aefsmrVKnr37s2Vo0bh3rwFW24utpwcanNzcZeXe3a2WonIzPQkRv37ETlgAOE9e/otQXK5bBQUfE7+3hmUl6/EmDA6tL+U2Nj+bN32T6KiMhg65G3Cw5vnh66I4Nhfg31zCbZNpdh3VYBLMGEWwnscHjVqqPhHk457ZIGEUAvxl3QjemTHoJq2F4zcNifF0zZg31pG3EVdib0gPagKZPiltHb9OgydgH0isrsJ8TVai0iGROCcczzrD23dCtF6VUEp1boEWzJ0UGvvp+p27mTHdddjTe9MxowZWCIiTrxz2W5P+e29q3Cd+RBvre1LXtV6eqUN4uZfjMdd62T/M9mEJITT/ldDTvjDr7p6G4uXjmNneTsGDJzOGT07+eXcAm3Hjh28//77iAjXX3893bt3P/SZq7KOAy8swVUBpm4NyX+ZzIov97Bu4R7cLohLiaRd5xjPIz2Wdp1jiEkMP/TDUERYtmwZX3zxBcnJyUycOJHk5ORDnzny87HVT6/zTLNbj7vCU7vDWK2E9+lzKDmKGDCA8B49MNbGFx+ort5WPwo0B6eznMjIDNLSJpLacTxhYZ77YkpKfmTtuilYrYkMHTKVqKiMRh+vsdx2J/Zt5YdGjVwlNgBC20US3iuBiN5JhHePb9IUNvv2ckrnbsFZqAUSGkOcbko/3ELNqgKiR3Yk4eqeQXNvnN/WGTLGjAUG4Lk6tqSR8TVai0iGABYv9lSU+9vfPBXmlFKqFQnWZAhafz9VuXAheVN+RdyVV9DpqadOfiXWYYMvfgsr3sbZ5XxeL7yaA7W7GZgximvvuISa1QWUzNpEwtU9iDnjxElO/r4v2LjhHjaUn8evrv4PFkvLWGfEW8uXL+fzzz8nKSmJm266iaSkwzfKO/ZXU/jaKlwVNuxFX7FszHkULwvF1IWyMWUp9uhKBpBFfFV7qosOl8sOjwqlXXoM7dJiPc/pMZTVFvDhnA8QESZMmEDPnj0bjEdEcOzZc2jkyJaTiy03F3dVFQAmLIzwvn2I7N+fiP4DiBjQ35MgnWRqvtttp6DwK/LzZ1JWthRjrKSkjCWt00QSE8849P9RUZWdWct2c8mAVNpHbGf1mjsBw9AhbxEb298H/7UbR0RwFtuwbyrBtrkU+/ZyxOGGEEN4t3jPqFHvRELbR3k1OnFUgYTE+gIJvbVAQmOICBVf7aLy2z1E9E4k6aa+WMIDf4+VT5IhY8xM4GsR+U/9+98CfwXWAv2Au0XkXd+E7J0WkwwBjB8PCxZ4RojqK9AopVRrECzJUFvtpwpffpmiF/5Nh9/9jqTbbj11g1XT4dOHcIYn81LdFErrihnW4wKuuOUcit/MpW5PJR0fySIk9sRXxOd+/0finNMJSfwt5w9tHVPAXS4Xn3/+OdnZ2fTq1Ytrr72WiCNG28o37Kd8+iakqoIte74lO20UsXXt2Je0BRl1gMG9+5B9IJuvdn6FIIxNvZirEq8jqTqVwrwqivZUUZJfhdPhmXlpCTFEdzQcsKyi2lHByCFnc8GYc4iMOfVIhLjdOHbvprY+MbLl5GBbvx53tadstYmIIKJ3b88Uu/ppduHdu1Nbl0f+3lns2/chDkcJERHppHW6kdROEwgPa3f4v4VbmLFsN09/sZEKm5OwEAv3XNCT20dCzrpJOJ2VDB70GomJP/PxX6FxxOHGvvPwqJHzgKe2SUh8GOH15bsjeiZiOWYKo4hQs7KA8s88BRJiz+lMbAsukDB3y1zmb5vPb0b8hv7JgUtWAaqW7qNs3lasnWJod0f/k/570hx8lQztBrJEpMAYYwEO4OlYPjTGXAr8XUQG+yxqL7SoZGjTJujfH6ZMgX//O9DRKKWUzwRRMtQm+ylxu8m79z6qvvuOLm+9SfRILyq97V8H792KrbSAFy33U+W087PMixh90SAKnl9J5IB2JE/sc8LmNkcdb308gR7xGxk5fCYJCcN9eEbNr6amhvfff5+dO3dy1llnMXr0aDCwuXQzi/cupm55EZduGkldTRHZ5dWURKZjku0MviqVM0YOwmIOj47tr97PjI0z+GDzB1TWVTIoZRC39buN0V1GYyGE8oIaivZUUZRXRVFeJQV7yjhgyaEuoojw2vZ0DBlA+84J9dPsPNPt4pIjT3nPirjd1O3cdTg5ys3Ftn49Lls1tkFCzflgz3SCGBKcA0jrNJH2fcdjCT16it2aPWU8Pj+HtXnlnNkjmQfHZvLOkl18vGYvmR1iePKq9jgLH8Bm282A/i+QkjLWH3+SJnGW2esr1JVg21KG2F1ggbD0uEOjRiY85HCBhC6xJFzTsgsk5Fflc838a7A5bViMhZ8P+Dl3D76bsJDAJSG1G0somb4BS4yVdpMGYG0fdepGfuKrZKjiYElSY8xwYCGQICKu+so7pSLi3zIjx2hRyRDA3XfDG2/Ahg1wguFwpZRqaYIoGWqz/ZSrspKd19+Aq7ycbh9+gDU19dSNastg3hQqN37HS2YKdpeFM3pfys86JFH59R7a3TmAiF4nrhg3a2kuUjiJ9jGGs8/4hPDwhtcpCnYFBQXMnDmTiooKzr/kfEoSS1i8dzE/7fuJ0tpS7iy4hmtLxlBsq+AnWyTWaMMZ1/ah96iOWE6SoNQ4api3dR7TNkxjT+UeOkV34qa+NzG+13hiw2KP2re63M7XC75lZc5PRFsTae8YTPUBNwd/llkjQjzJUVr9fUjpMSSlRhN6khGM2tp88vNnsjf/PRyuEqz2aGJykgj/pAzLATsAJiqKiL59iRzQH3fPPrxTEsmrOxy0i43kD1f048pBqYemmX2z8QB/mJvDvgobk0YlclHq01RX5dC3z/+jU6cJTfwr+I+43NTtqcS2qRTbllIceVWHPjMRIa2iQIKIcM/X95B9IJt3L32Xd9e/y/xt8+mZ0JMnznqC/u0CN0pUl1dJ0du5iEtod3s/wjMCs0aZr5Kh7cCFIrLTGPMIcJGIXFT/WQywS0SSfRW0N1pcMrRvnycJuuIKeO+9QEejlFI+EUTJUJvup+zbt7PzuusJ696drtPexRLuxdosbjf8+C8Kv/43r8rtiCuSM3pdQv+COgzQ4YHhGGvD9wQ5XG4mvjyNu/r+P9olDmLY0HexWFrWzebrNqxj3px5uCwutnTdwlrHWgCSIpI4p8NZXJ8zmqS8MLbbXWyorGHA4DBG/vICrKcxjcrldvFd3ne8s/4dVhxYQVRoFON7jefmvjfTObbzUftu3LiROXPmYLVamTD+OqJCkijOq6JoT2X9SFIVDrsL8KzlntAx+ohiDTEkp0VSU7eY/L0zKC7+HjC0Sz6ftLSbSE4+F2NCEJeLuh07PMUZcnKpzc2lJnc9ljpPguQIjyB6QH9iDk6xG9CfsIwMjDFU2Z3888tNTF2yk/R4+OPZ72LqltOz52N07fILn/xN/M1VVYd9SxnO4lqiR6a2igIJX+78kke+e4RHRzzKrf08U2W/z/uevyz+C8W2YiYNmMSUwVMCNkrkLK6l6K1cnGU2km7ofcIFnv3JV8nQ74CbgS+B24D7RGRm/WeXAb8TkbN9E7J3WlwyBPCnP8Ff/wrLlsGIEYGORimlmiyIkqE2309V/ve/5N17H/HXjif1iSe8L227/Tt2zfotU+0XY3HEc0Hn8+m6u5rY0V2IH9v1hM0+XJHHe4ve5JeDptK58630zvyzb07ET9ziZmPJRn7M/5Hc7Fzid8dTFlZGdmo2/dL6cWanMzmj0xl0daez/9UcQivs5NS6sO/4jhHjMkm786YmHT+3OJd317/Llzu+xI2b0V1Gc2u/WxmSMuTQ36qgoIBZs2ZRVlbG5ZdfzvDhh6cgiluoKK49YpqdJ1Gy2faR0P0H4rstwhpVhtuRRJjrMtonj6dj157Et49qcBRr4/4KHp+Xw4rtRVwca+O+NCdJeds8U+w2bkTsngQpJCmJqKwsz2NEFhsiUnhs3nq2FZTw+Nkf0DlyCV27TKZHj0eDqpxyW1BZV8lV864iJTKFGZfPINRy+L6oiroK/rn8n8zdOpce8T144uwnGNBuQEDidFU7KH5nPXW7K4i/rDux55xkOQA/8Fk1ufqVt7PwVOWZfsz2ChGZ60Uw6cA7QEc86ze8JiLPG2OeBq4E6oBtwCQRKTvZd7XIZKiy0jM61K8ffPON59KOUkq1YMGSDIFv+ilfCkQ/VfD88xS/8n90/POfSLzxRu8bVuwl9+2HmF3cmzB7ClfFZJFQ66TDA8OwpjQ819/lFsY+9x1j095jZPuv6Nf3aVJTx/voTHxjf/V+luxdwpJ9S/hp70+U15YzrHgYXau6EpoaynmXnMeItBFEhEYgbmHLf3dhvtmDVYQdpobYr/5O+vWX0uGx3/ospgPVB5i5cSazN8+moq6Cge0Gcmu/WxnTdQxWi5Xa2lo++OADtm3bxogRI7jkkksOLfZ6kIiL4pJF5OfPpKjoG0CwShZ1RWMp2tKP0r023G7P77pQq4WkNM/oUUrnGKI6RDF9417eXrab+Egrj13ShwnDOx+VMInDgX37dmrXrqU2ewU12dk48vMBsMTFETF0KGuSuvF6WSwjz1zKmWk/kJp6HX16P4HFEvyLzLvdLiyWllko4UhP/PQEszfPZsblM05YNOGH/B/48+I/U1hbyKT+k5gyZArhIV6MHPuYOFyUvLeJ2pxiYs7qRPzl3ZtteqLfSms3MphUIFVEVhpjYoEVwDigM/CNiDiNMU8BiMhJ/+VpkckQwEsvwb33wqefwmWXBToapZRqkmBKhoJNIPopcbnYM2UK1Ut+ouvUqUQNG+p9Y2cdS975M1/uDiOmOo3rQvsQ2T2elLsGnvCK/0dr9vLArGxeufQdwtzrGT78feJiA3P1GTz36mQfyPYkQHuXsK18GwDJEcmckXwGSRuTqC6q5oILLuDcc889dF55G0vIfW8zmTV1iMXg7l6L45n7ib3oItL+9RzGDyXEaxw1fLTtI6ZtmMauil10iOrAzX1v5trMa4kOiebrr79m8eLFdO3aleuvv57o6Gjs9gL27pvN3r3vYbPlY7Um06nT9aR1uoHIyPRD3+1yuCnZX10/zc5TrKFoTxX2WicAguCKCqV7r0RSu8bVF2uIJToh7IR/a8fevdRkZ1OzPJua7GzqduwAwGYNY/c1kcScX0iUDCPrzDewRsb5/L+XL9RWVvDdu2+yafH3DL3sKkaNv4GwCN8u3NpcVhes5rbPb+Pmvjfz25EnT9Yr6yr5Z/Y/mbNlDt3ju/O3s/7GoJRBzRTpYeIWyj/bQdUP+UQOSCbpht4Yq/+T0qBKho47qDHzgRdFZMER264BJojIzSdr22KTIYfDMzIUEQGrV0NIy78yoZRquzQZOrGm9FM7165i76b1ZF1xDWGRp1eFyVVezo7rrsddW0O3Dz7E2uH0lnT4Ytq/+WlrMenVPbg4JIOECb2IyerY4L5ut3Dp84uwmnIeG/EUxlgYkTXv0IKd/uYWNxuKN7Bk3xIW713MqoJVON1OwkPCGd5hOGeknsEZnc4gpjaGWbNmYbPZGD9+PH379gWgZG81i+duhU0lDIoKhfgwEs8y5N/7CyL696fLW2+efEFbH53D93nf8+76d1m2fxmRoZFc0/Mabul7CyU7Svjoo/l07FjK4CEVVFX9gIiTxMQzSUubSEq7MV7dq7WtsIo/zcthzZYSRsRHM65be6yVTor2VFJRZDu0X0S01XP/UWfPKFK79FiSUqMbvILvLCykZsUKqpctZ9+iJbh6bKXiehfWLRbSfsoibugZRGVlETlkCJaowFUSA0+RgQ2LvmXhO//BXlNN534D2b1uNTGJSZx7y8/pc9Z5LWqKn8Pt4IZPbqDCXsH8cfOJtnpXCe/H/B/50+I/UVhbyO39b+eeIfcEZJSo8od8yj/dTliXOJJv60dIdOMXDvZG0CZDxpgM4HtggIhUHLH9Y+A9EZnWQJvJwGSALl26DN+1a1fzBOtrs2fD9dfDW2/BHXcEOhqllGo0TYZOrCnJ0KKZU1k2bzaRsXGMGn8Dg8ZeRqjV+x8Mts2b2XnjRCIyM+n6zlRMmPc3T7vdbmZPe5MN2/P4WW1/+lja0/mxUVjjG/7R9EXOPu6etpLnrgknofY+EuJHMHjwm36bLrW/ej+L9y5myd4l/LTvJ8rsnln1vRN7c2anMxnVaRTD2g8jItSTxOTk5DBv3jyio6OZOHEiHf8/e+cdHlWZ9uH7zExmUibJpPeEBBIICYTeQTqIKE0QEEFRcYu7urrr2usWy1rX8lmwIEgzAgKigChKr6GEQHrvyUySyUymvt8fA4FIgDQgq3Nf11yTzDnnPe+clPf85nme3xMcTH2NiQObckjbVUyih4IYhYQqToN6mDsFi25HrtEQtWolCp9LO+pdDdKq0vj81Odsyd2CK1bmh0XSU1YF9gosFhXe3jeS1Pt+3N2jWzSe0WzjnR8yef+nLFxd5PxtUnduHxyF/AJxYzZaqSw6H0GqKtRTVVSPzeroieShURGd5E9MUgCh3TXI5c1HyUoLyti04V9Edd+MVKYk6C1Q1ApQKHBLSMB94ADcBgzAvX9/5J6ezY5xNdCWFLH9o3fJP3mMkLgeTLz3fvwju1CcnsaOT96nLDuT8PhExt51HwFRLbuu15ulJ5byxpE3eGvMW4yJHNOqY+vMdbx66FWSM5KJ9o7mheEvkBRwTbsOAGA4UUH16jMoNK7435WAwu/qReg6pRg66wb5m0YAACAASURBVOyzE/inEOKrC15/Ake+90xxhQn9z0aGAISAIUOguBjS08HtfzNE68SJEydOMXRp2rtOlWSeYdfKz8g/eRyvgECGzb6d+JGjW1zrULtlC0V/eQjN3NsIefbZVp3bYrGw7NNPKCoq5kZTPxRCTq/Hh6HwulgcCCGY+t9d1DVYWXF7BelnHiMqcgndunVMnY3BYuBg6UGHACrZS06NIz0rwC2AoaGOyM+QkCH4u/k3Oc5ut/PDDz/w888/ExkZyZw5c1C5uHF0Wz5Ht+UjWe3cEO6BR40Jj6EhqId6kzd/PnaDgS6rVqKMjOyQ+bcGIQQ63X6y8j9FW/U9MuxkNsgosEfjkz+WuvIGRo4cyZgxY5BdIXVv+6kynvk6lSKdkZl9w3hsSjwBni2LAthtdrRlBspz68g9Xkl+ahVWix2Vu4KoRD9i+gQQ0dMXpevFgnfrka+wVT2OrsEbQ/YCJpvNWI4ewXjihCM7RpJQxfc4a8gwEPcBA66K6LRZLRz8+iv2fbUKucKFUbffSe9xk5ukPNrtNk7u2MbPq5Zh0utJmjiF4XMW4KpWd/h8OoqCugJmbpjJ8LDhvDHmjTaPs6doD8/sfYZyQzkLey7kj33+2PgBwrXClFtD1bJTIJPwX5SAMuLqiOROJ4YkSXIBNgHfCSFeu+D1RcDvgHFCCMOVxvmfFkMAO3fC6NHw0kvwyCPXezZOnDhx0iacYujSdMQ6JYQg70QKP3/xKeU5WfiFRzJi3iK69h/UorSe8v/8h6qPlhLyjxfQ3Nq6fjAGg4GlS5dSV6Xj5oYBlFnLueGB/rhEXFwTtON0GYs/PcS/Z/air/dHFBWtIDHxvwQFtr421ma3kVad1hj9SalIwWq34ip3pX+wI/VtWOgwumm6XfIamEwm1q1bx+nTp+nbty9TptxExv5y9m/MxlBjpntvP3qarNjLDXjfFIN7Xw35d96FKSODqM8+xS3p2n5SbrHoKCn5iqLilRgM2SgUXoQEz8QvaAbfl57i81Ofk1+TzxDdEEJ0IUR3i+a2W2/DtZkUvoJqA89tTGV7WjmxgWpemJ7IkJj2ucpbzDYKTlWTc6yC3ONVNNRbkLvIiIj3JTrJn+je/rh5no8+Fpfv5/iJe9GbFCzP+AsPTJ7EDZGeGI8dP1t3dBBjSkqjY52yW9dGYeQ+YGCrUzt/SdHpU2z78G2qCvOJGzKCMXcuQe1z6dRNo76OPWuWc2zrFlzVakbMW0jimAmdzmRBCMHvt/+elIoUNkzbQJBHULvG05v1vHr4Vb5M/5IuXl14YfgL9Ans00GzbRmWCgOVH5/ErrfgO78HbvEd3wGhQ8WQJEm+wF+BPkAT2SyEGNWC4yXgM6BaCPHgBa9PBl4DbhBCVFxxIvwKxBA4eg7t2gVZWeB3TdtfOHHixEmH0NnEUHvXqY6kI9cpYbeTvn8Pu1cvQ1tSTGhcPCPnLyI8/vJmBcJmo+DeJRgOHiRqxXLcereuaFqr1fLRhx8h1dsYY+xHfkMGN90RgHLgnKbnEYIZ7+6hvLaB7x8eRuqJhej1pxnQ/0vU6u5XPE+xvpi9xY66n/2l+6kx1QAQ7xvfGP3pG9i3RfUNWq2WVatWUV5ezqRJkwj26MbedVlUF9cTHOPF0DERSN/nYa+34DuvB67dNRT+6c/of/iB8Lf/i+e4ca26Rm1FCEFNzWGKildSXv4NdrsZb6++hIXNIzDwJuTy80LHLuzsKtrFstRlVKRXkFSVhMxDxsw5M+kd5fiZmqw2Pvwpm//uyEQuk3hwfCx3DY/G5RJpbW3FbrNTkllD9rEKslMq0FebkCQI6aYhpk8A0Un+ePm7odef4cDhhRhMRl4/vITE6OE8NbUn/mrHz1CYzRhPpmI4eBDDoUMYDx/GbnB8Fu4SFdkojNwHDsAlLKxF4r+hXs/PX3zK8e3f4ukfwLjFv6dr/0Etfm/ludns+OR9ik6nEhTTjbF3/Y7QuB5tu1BXgS05W3jkp0d4dNCj3B7vKK2v15koStdSlK4DIRgyvWsTYdoS9hbv5dk9z1JSX8LCngu5v+/91zRKZKszU/lpKpZiPZrp3VAPbkHj6FbQ0WLoW0AFrAGaRG+EEJ+14PgRwM/ACRzW2gCPA2+dHbfq7Gv7hBC/u9xYvwoxdPIkJCXBgw/Cq69e79k4ceLESavphGKoXetUR3I11imb1Urqj9vZ++UX6LXVRPcdwIi5CwnsEnPJY6xaLbm3zkZYrUQnf4nC3/+S+zZHUVERn378CV4WN2Jqkmiwn2HquHxUU58HxXlx8nNGBXcsPcBztyQwt78bBw5OQy53Z+CA9bi4NHUX05v1HCw9yN4Sh+tbbm0uAIFugQwNdUR+BocMxs+tdR8U5uXlsXr1aux2OxNuuInC/VYKT2vxCnBj2IyuhHooqP7iNJJSjv+inriEqSl74QW0X6wk6Mkn8V1wWe+mDsFqraOkdB1FRSupr09HLlcTHDydsLB5eKqvfON9pvoMy3ctx5piRUKioWcDiV2m8PmPcnIqDUzpFcxTU3sS4n31U/CFEFQW6MlOqSDnWAVVRfUA+EeoiU4KICzBRG7JHzAYS3k7ZTF59b158qaezOp3sbgRVisNaacdkaOzD3uNQxQrQkIu6HU0EGV0lybHCyFI37eLHz79AENNDf2m3MKwObe3ySlOCMHp3Tv5afnH6LXVJNwwnpHzF+Ghubb1Y7+kxlTDLetvIUrelSci/0lxRg3F6Tp0ZY5/c0o3BVaLDTe1kkn3JhLS1btV49db6nnt0GusSV9DF68uPD/8efoGtsKNsp3YTTaqv0ij4YwWzzEReE2M6jBTi44WQ7VAgBDC1BGTaw+/CjEEsHgxrFgBZ85Aly7XezZOnDhx0io6oRj6TaxTFlMDR7/dxIENazEZDPQYNorht92BJqh517eGtDRy583HLTGRyE8+RmqFGQPAmTNnWLVyJWF2P2SVPfCUZ3Nzzw24znsfNA5LZyEEt72/j9yqen56ZAwN9Uc5cvR2/HxH0bv3+1Q3aNmUvYkd+Ts4XnEcq7DipnCjf1B/hoUOY1joMGK8Y9p8A3T48GE2b96Mt5eGGI/B5B+pR+WuYOBN0SSOCsN4uAzdhkxcAj3wuzMBhUZF1dKPKX/lFXwXLybokb+16bwtQQhBbd1xiopWUla2CbvdiKdnL8LC5hEUOBWFomVuYBeSVZLF8i9WYK+zcdLnJFnuBm7vuYAHh8zBRX513bkuha7cQM6xSnJSKijJrgEBmhATIUNfRyhy2Vq0hNWp8Yzo5s8/ZyQS5Xfp9y3sdkwZmRgOHWy087ZVVgIg9/NrFEbWbjHs+uE7clIOERTTjQn33k9QTLd2vxez0cC+dWs4vGk9CqWSYbPn02fSVOSKa9tHqb7GRHGGjg0/bcdUIENjdKTGKV3lhMZqCI3zISxOg3+EJ1WFer794AT6ahNDZ3YlaVxEq/+e9pXs45ndz1BSX8KCngv4U98/4aa4NrXtwibQbcik/kAp7n0D8ZkVi6Rof2Szo8XQLmCRECKr3TNrJ78aMVRYCLGxcOut8Pnn13s2Tpw4cdIqOqEYatc6dZnm4L7AaqALkAvMEUJoLzfWtVinGvR6Dn79JUe2bMRus9J7/GSGzJzb7KfYNRs3Uvy3R/C54w6Cn3i81efav3MvW374jjhVFLqCCPzk+dwS8jput70JXccCsC+7irkf7OPJm+K5Z2QM+QXLyMh4jnS680FhCVZhJd43vlH89Ansg1LeupSeX2Kz2di6dSv79+/H3zMUWV40MrsLvceG039yFEpXBTXf5qL/qRDX7j74zu+BTKVoNJjwnDyZsNdevSq9hKxWPaVlX1NctIo6fSpyuTtBQTcTFjoPL69ebR7XYrPz2Z5c3tqWRn8yiJLpqNZU85P3T/h6+DI/fj6z42bjrWpddKAjMdSayTlWQc6xSooyCwkd8jZuARlUFi7i87TB5Cps/HlCHHePiEbRglQ+IQTm3NzGtDr9wUNkWg1kBPmCJNFL7Uvv4aNRDx6Ea3w8UgeJluriIn747ANyUw7jFx7J2LvuIzLx6tWUGWrNFGfoKDqjpShdi7bUEfkxyxuQh5oYObA/Yd01+IerkTVz3UwGC99/lkbOsUpi+gQwdmEPVO6tE8f1lnpeP/w6q8+sJtIzkheGv0C/oH4d8v6uhBCCuh8LqPkul4YuLtT1UyJXKUhqRx1fR4uh54F5wCdA6YXbhBAft3WSbeFXI4YAHnvMYaRw5Aj0ubaFa06cOHHSHjqhGGrXOnWZ5uB34qh3fVGSpEcBn87UHFxfXcW+r1Zx/PvvkLu40H/KdAbeMhOVe9NP3sv+/W+qP1tG6Esv4j1tWqvP883H6ziQf4x+EX0pPuqJRlHKLZ6P4T7+fhj5MMhk3P7RPtLK81k0qYLNOesZrcpjsIeNLLfJjI9/mBjNpVP6WovRaGTt2rVkZ2fjaYlAVdWF7oOCGTwtBi8/N+xmG9rVZzCmVuExJATNzV2R5BKGQ4fIv2sxrr16OXoJqTq210pdXSpFRSspLfsam60etTqesLD5BAfdjELRPsesg7nVPLX+JKdL6xjTPYBnbu5JQdpRvv/+ezz9PcmMymRX9S5c5a7c0vUWFvRcQLT39bWMNhut5J4sJr/0USSP/VSm3kxZ2s1kyu0YA5X8cX4ifbq2PCWyNCuDbR+8TXluFpFhkfRxUSOlHMOSlw+AzN0dt379HNGjQQNxTUxE1gp7+V8ihCDr8AF+/OwDasrLiBs8nBsW3o2Xf/uMHgCMdWaK0nUUp2spTNehLXGkGrqo5IR00xAc68l/S1+izD2XdTPW4e5y5Z5NQghSthewd10Wnr4qJi/pRUBk63/vDpQc4Ok9T1OsL+b2+Nv5c78/X7Uokd1up7S0lPz8fPLz88nLyqXe5BCC4SFh3HPfvW0eu6PF0A+X2CSEEGNbO7n28KsSQzoddO0KAwbAd99d79k4ceLESYvphGKoQ9epc83Bzz5GCyFKzgqmH4UQl3UGuB7rlLakiN1rVnBmz0+4qj0ZPH02fSZNRXH2RlBYLOTffQ/GY8eI+mIFbgkJrRrfbrGx8uWPybAUMWrAeDK+s+Kp1DLN/S8ouw9k5+Db+fj0Jo5XHUCSBENDhjKr2y1oKj/AaMxj0MD1Le6RcyUqKipY9uly6vS1qGtj6Rreg+G3diMwylGfZKszU/lZKpYiPd43xaAeHookSZiys8mdNx+Fry9RX6zoMFtnm81AWdlmiopXUlt7DJnMlaDAmwgLm4+XV1K76x8q9SZe3HKaLw8XEurtyjO3JDCxZ1DjuGfOnCE5ORkXFxeG3jiUrTVb2ZS9CYvdwqjwUSzsuZBBwS1zIbxa2O1W0tIep7QsGeqnkvHDdGwGgQ2BPUDFqNFRdO8fiIemeXFqNhrYvXo5R7/dhLtGw9i77iN20LDG92QpK8d4+BD1Bw9iPHQIU0YmAJJKhVtSksOxbuAA3JKSkLWhrYnVbObgxmQOrP8SgMHTZzPg5pmNf18twag3U5yuoyhdR1G6lupih/hRqOSEdvUmrLsPoXEaAiI9kctlfHj8Q946+hbvjHuHUeGt84ApydTx3UepNOgtjLwtlp4jQlv98zdYDLxx5A1Wnl5JhGcEzw97ngHB7f+XbzabKSoqahQ/BQUFmM1mALy9vYmMjCTULQD1/gYCfPwI+nO/Zpv/toROZ63dUfyqxBDA66/DQw/Btm0wfvz1no0TJ06ctIjOJoY6kgubgwP5QgjNBdu0QoiL7qI7S3PwsuxMdq1aRu6xI6j9/Bl263wSbhiHTC7HWlVFzqxbkWQyuiR/2WoxYMioZvmyzylV6Jg8ZjpHvqzG4lLD5tiXKXKrI1Dlg7xhBKVFvdj18K14ubrQ0FDMgYPTUCr9GND/SxSK5vu4OO5F7Ahha/6BDWG3cfzQSXbu2Q52QbC8B4PGxxPS1RNw7GeprqdmSxZ2kxn16FBcIj0QwoqtRkf5m69jt1rwv/93yH00CGFFCPvZ5/PnEDQ9NxfNx9r4tc3eQFXVTmw2PR4esYSFziM4eMZFxhFtwWYXrDyQz8vfnsZosXHPyBj+NLYb7sqL08AqKipYuXIlOp2OKVOm0CWhC2vOrGH1mdVUN1TT3ac7d/S8gxujb2x3emJbEUKQmfUS+fkfEhg4FVfZYyR/nYM1vx4fuyPtKyjay9HotU8APsGO6Gbmof18//F76Kur6DNxCiPmLrwo8vlLrFqtw6nu0CEMBw/RcPo02O1I7u4EP/0UmunT2/QeaivK2fn5UtL378Y7KJjRC++9pN19g95CcYaOwnQtxenaRpMJhVJGSDcNYXEawuJ8CIjyvKihbX5tPjM2zGB0xGheHd02oy1jnZltH6dSkKal++BgbpjfHRdV6y3DD5Ye5OndT1OoL2R+j/k80O+BFkWpzmEwGBqFT35+PsXFxdjtDi+1wMBAIiMjiYqKIjIyEm/v8+mdltJ67GYbqsi2/y1dNTF01ia78acuhLBfZvcO51cnhkwm6NEDfHzg0CG4CrnLTpw4cdLRdGYx1J516pfNwSVJ0rVEDF1IZ1in8k8eZ9fKzyjJPINvaDgj5i6k26ChNJw8Sd7tC3Dr34/IDz9sdX1F4coUktO2Ua3Qk+J5mtGZi0BloVfYy4wxZFAy7AVGbAvnwfGxPDg+DoDq6j0cTVmEQqFGkhSNAsQhfs4Li86HhCQpkCTZL57lSMgdz5ICb+++hIXNx9u7f4dFX44X6nhy/UmOF9YwNMaPF6Yn0C3w8ulORqOR5ORkMjMzGTBgAJMnT8Ym2dicvZllqcvIqsnC382fud3nMqf7HHxcr49LWl7e+2RmvYyv70h693qXA7kGXlx9ArdyMwNdXHGtc/wuePlZsTb8SHXhcfwjujBhyf1ttru21dVhPHKEqo+WYjh4EM2cOQQ98Xib0yTzTqTww6cfUFWYT3Sf/oxetAR370BHzU+6lqIzOqqK9AAoXGQEn438hMX5EBjlifwy5gBCCJZsW8LJypNsmL6BQPe2p+TZ7YJD3+RycHMOviEeTLo3Ed+Q1pt2GCwG3jr6FivSVhCuDuf54c8zMHhgs3PX6XRNxE9FhaNzjlwuJzQ0tFH8RERE4NaGKF1r6Og0uTAcqQKjAM2F24QQ17QzVWdYZDqcFStgwQLH8/z513s2Tpw4cXJFOpsY6oh1qrnm4JIkneF/IE2uOYQQZB7cy65Vn1NdVEBwtzhGzluEV3o2JY8/3iontdPVp/ky/Ut+Tv+Rf565n60uJ7G5Cm6aMIsjKypxUUlMj/4Q7+IN7PKczIO1C9j+90lo3B1RiPKK76iq/BFJ5nJeVNBUZCDJkUkOkcHZZ5sZ8lK1ZOZlYFHU4enmz4gbhuPm7nZemEhyGtJrqN9TisLbHe/J3VB4uSIhBztUvPgfDIcOE/zEk6iHDGty3LlzcoG4ccxHjiRd+w8nawwWXtl6mhX78/FXq3jypnhuSWp5ipPdbuf7779n9+7dREZGMmfOHNRqNUII9hbvZdmpZewu3o1KruLmrjdzR/wdHVrL1VKKi9eSdvpxvLx60yfpI2x48faOTP5vZxYhCjl3KPLQn/4OYbOhcBuCV9AwYvoEEdMngNA4zUWRlJYirFYq3nyTqg8/wjUhgbA330QZHtamseprjOxek8ypH9djs5mRq/qjcB2MQulKSFfvxshPYBevy4qfX7IxayOP73qcJwc/yW09bmvT3H5Jwalqtn6citViZ8yC7sQNbN598kocKj3E03uepqCugHk95vFA3weoq65rIn5qa2sBUKlURERENEZ9QkNDcWmlm2V76WgxtBFH34Z/4/jEbBTwLPCNEOLD9k21dXSWRaZDsdsddUNaLZw+DR1c0OnEiRMnHU0nFEPtWqcu0xz8FaDqAgMFXyHEI5cbq7OtU3abjVM/7WDP2i+oq6ogqndf4vVmZOu+Juy1V/GaMqXZ4+rMdWzJ2UJyRjKnqk6hlCmZ0GUCC8zTsG3XstkjBY2vhlsmzObb/0tD4SJj+tB9aI6+wEl7F/b0f40l09rWyNRmtXNyZxH7vjlDheo4VmUtQwcNZ8LkccguyKAQdtHoGKeK88Fvfg9kro5olxCC0uefR7dyFUFPPYnv7Ve/l1BbEUKQfKSIf3+ThtZgZtGwLvxlQhxerm27eTxx4gQbNmzA3d2duXPnEhoa2rgtU5vJ8rTlbMzaiNluZkTYCBb2XMiQkCHXtK6oomIrJ1MfwM0tij5Jn+DqGsKBwyfZ/N5beNUVU+cXw8wlf0LZoCYnpZL81CqsFjsqdwVRvfyISQogoqcvStfWu8fV7dhB8d8fBZmMsJdfQn3DDVc8xmSwUJxZczbyo6WyUA8CJJkRhWwfdRVHcfPy4YYFi+k5anSbrqWuQcct628hwiuCz2/8HFkHCnK91sTWj05SklVD4qgwRsyORe7SuvGtVivZ+dms2b+GwvxC/E3+KOyO6+/p6dkk5S0wMLDJ3+r1oKPFUBUQKYSoP5cycNZudI8Q4pq26O1si0yHsW0bTJzoqCF68MEr7+/EiRMn15FOKIbatU5dpjn4fhyNXCOBfGC2EKL6cmN11nXKajaTsnUz+9evpaGulnDJha7ZhSR+vhzX7o5glxCClIoUktOT2Zq3FaPVSJxPHLNiZ3FTzE14q7wRdkHFB8fJKytgi3SEyMhIbhwzg81vH0eSJKbNqMd122IsNoE08328km5u8RyFEGQdqWDvukyqdZXoA9IQMgszZs4g4RemD3azDe2aMxhPNnWMO0fVRx9R/p9X8b17MUF/u3q9hNrLmdI6nlp/kgO51fSL1PDC9EQSQttvjV1cXMyqVaswGAxMmzaNXr2aWnpXGatYk76GVadXUd1QTaxPLHfE38FNMTdds7oirXYfx47fh0Lhia1gOoc3OAxAZEOm8VauJwKJhyfGcdfwaOxWOwWnqslJqSDnRCWmeityFxkR8b5EJ/kT3dsfN8+Wz9ucn0/hAw9iSkvD73f3EfCnPyHJzweRTUYrJefS3tJ1VBbUIQTIFTKCY7zOpr1pCOzihcJFTnF6Gjs+eZ+y7EzC4xMZe9d9BES1zjTk6d1PszFrI6tvXk2cT1yrjm0JNpudfeuzSdmWT0CkJ5OXJOLlf+lUNaPRSEFBQWPUp6ioCJvNkcao9lGTLWWTJ8tjSM8hPDTiITyUrU/Bu5p0tBgqByKEECZJknKBgUAtUCmEaJ9XZCvprItMhzBxIhw+DFlZoNFceX8nTpw4uU50QjHkXKdaiMlQz6FN6zi8cR1WUwNRJjsDX/k3P1iOkZyRTE5NDu4Kd6bETGFW7CwS/BIu+pTbUlpP2VtHyYuuZ1vRXnr37s0Ngyfy9ZspCCEYMNMT2ZZFJMpyYdTfYPRjILt8tmJJpo7dyZmU5dTiElpHuXQCd4+LIxtw1jFu2SkshXVNHOPOUbN5M8UP/xWvKTcS+p//XJVeQu1Fb7Ly5vZ0Pt6di5ergkdv7MHs/hHI2uic1ew59HrWrFlDfn4+w4cPZ9y4cRd9Wm+ymfgm+xuWnVpGpi4TX1df5vaYy10Jd+GqcO2wuVyKM4fWklfxFHa7Hal8NqNmP4qb2pMinZGn1p9kx+lyksK9+ffM3vQMdRTT2212ijNryEmpIPtYBfpqE5IEId00xPQJIDrJ/7I3+eewNzRQ+o9/UPNlMi5DRyJb8jilxVaK0rVU5DvEj0whERx9Pu0tKMYhfpodz27j5A/b+HnlMkx6PUkTpzB8zgJc1c0bh1zIwdKDLP5uMYsTF/OX/n9p3UVsJdkpFXz/WRqSBOMWxROdFABATU1Nk5S3srIyAGQyGSEhIU3qfTw8PDBajfz36H9Zfmo5oepQnhv2HINDBl/VubeGq5Em97EQYp0kSe8DsYARcBdCjGn3bFtBZ19k2sXRo9CvHzz6KPz739d7Nk6cOHFySTqhGHKuU62kTlvFxrdeoPRkBkiC4zF1iMERTE+czaQuk67oGFWzJYe6nYWkDzHyU8oeRo4cSf/Eoax//ShWi43c7gqSCl9mtuwHiBkDN74ErhpwcQUXd5A7UsB0ZQb2rs8i+2gF7t4uqBN0pGYfJjw8nNtuuw1Pz6Za1lJWT+UnqdjrLfjO7YFbQtM+NYaDB8lffDeuSb2JXLq0w3sJtRchBN+cKOX5TamU1ZqYNyiCRyb1wMfj6kRjrFYrW7Zs4fDhw3Tr1o1Zs2Y1W7guhGBfyT6WnVrGrqJdjI0Yy+tjXu/QVK0Lqddp+XHZR5zevZOAWB+ixmciJAO9e/8fvj5DG+e06XgJz21MRWuwsGRUDA+Mi8X1AjEihKCyQE92SgXZKRWNltX+EWqikwKI6ROAX5jHRYLe3GCl5GzaW/6+bKprJIQkRyaDoJjzhgfB0V4olK0rjzfq69izZjnHtm7BVa1mxLyFJI6ZgOwSHwiYbWZmfT0Li93CumnrrlpPnwvRlRv4+oO9lFeV4hllxYgWnU4HgFKpJDw8vDHlLSwsDOVlbMSPlh/lqd1PkVebx5y4OTw04CE8XK5/lKijxZAGkAkhqiVJcgMeBjyBN4QQJe2ebSv4X1lk2syCBZCcDBkZEB5+vWfjxIkTJ83SCcWQc51qIaX1pazPXM+6jHUU1xcz7agHCakqin09UXl4MGjabPpOnoqL6vJRAbvZRtlrh8FFxsFuxRw5eoSpU6cSG9WT9a8fpcFgZZnSwCM9jjGt6DWwmZocbxQ+HDTMJbVuHDLJSu/An8lzreW0yZ8kTy1TQ7W4qNzAxc0hnlxcadAFU3U0Dkkh8L+hGmWQwrFN4RBYpqIqcu9/AoWvD12WLUXuH9woujoD2RV6nvk6lZ8zKkkI9eKF6Yn0i7w2rm4HDx5ky5Yt+Pj4MHfuXAICAi6574q0Fbx44EXuSriLhwY81KHzEHY7J37Yxk8rPsZqMjFo0INlVwAAIABJREFU+hwGTZ+N1V5FSsqdGAx5JCa8QWDgpMZjdAYz/9ycxtrDhXTxc+dfM3sxrKt/s+Pryg3kpFSSc6yCkuwaEODl70p0nwCCo72pyK+lKF1HeV4dwi6QySWCor0I8rGi2PARHrmHCf3rX/C5Y0G7a6jKc7PZ8cn7FJ1OJSgmlnGLf0dI7MX+K+8de493U97lvfHvMSJsRLvOeSmsVislJSVNIj9GoxEAyeaCl9KffsMTiO0RQ1BQEHJ56wRgg7WBt4++zbJTywjxCOG54c8xJGTI1XgrLcbZZ+h/ldxc6N7dIYqWLr3es3HixImTZulsYqgz0RnXKYvdwk8FP5Gckczu4t3YhZ0hIUOYFTuLsZFjqXr2BfI3fk3e+JEUFOTi4ePL0FnzSBwzAfll7LeNaVVUfXYK9cRINpfsIjMzk7lz5xLiH8mG14+i05r40sPEmt+FEVR7EiwGrA0NHE/15PDJYCxWOT0jc+kefoINRa6UmlRM9ClgqGsOktUIFgNYjGAxojcOR2f5PS5SPn7K51FIFU3mYjXKyN3mj90u0WV8JUr1Wbtu2VnB5HKhsHK7QET94rXG/ZrZ/9xrigu2qQPhCjfNRrONd3/M5P2d2agUMv46qTsLhkQh78CUuJaQl5fH6tWrsdlszJo1i7i45utShBD8c/8/WX1mNc8OfZZZcbM65PxVhQVs+/Btik6nEt4zkfH3/BG/sIjG7RaLjmPH7qGm9hg9evyDsNCmbmq7Myt5fN0J8qoMzBkQzhNTeuLtfmmxW19jIvd4JdkplRSeqcZuFchkEoFdvAjr7kh7C+7qjcvZyI+ttpbiRx9Dv2MHnjdOJuSFfyBXty/CIYTg9O6d/LT8Y/TaahJuGM/I+Yvw0DhEcG5NLjO/nsn4yPG8fMPL7TrXhZhMpib1PoWFhVitVgB8fX2bmB1UZJjZ+cUZXFRyJixOICLet83nTSlP4andT5Fbm8vsuNk81P8h1MorpwleDTo6MqQCngbmAX5CCG9JkiYCcUKIt9s921bQGReZDufhh+GNN+D4cWhll3AnTpw4uRZ0NjHkXKeaJ682j68yvmJD5gaqGqoIdAtkWrdpzIidQYTn+ZtQu9lM3h13YM7IRPnyv9j30/cUnzmFT0gow2+7g7jBwy9Zd1P5+SlM6Vp8/tiLzzespLKykjvvvBNvdz++/M8RtFVGagZqeOaufqQfLGPfhiz01Sa69PJj6Ixu6K1VrF69GqvV2uwNurALar7LRb+zEFWsN36zwpHJzU2Ekr22mry/vYKpsIyopxfhFqlp3HZ+v7PPVuOlt1kawFIPrWmh6B0JiTMg8VYI7nWRMPo+rYxnvk6lUGtkRt8wHpvSg0DPq1+Lcyl0Oh2rVq2itLSUcePGMWLEiGYjIFa7lft33M/+4v28N+G9dn3KbzWb2b9+LQfWr0Xp6sqoOxaTOHpCs+e12QycOPFHqqp/omvXR4iKXNJkvwaLjTe2Z/Dhz9n4uCt59pae3NQr5IpRHLPRSnVJPX5h6ss2HxV2O1VLl1Lx+hsoo6IIf+tNVLGxbX7vjedvMLLvq9Uc3rQehVLJsNm3kzRxCvft+B1p1Wl8Pf1r/N2aj3ZdDiHsWK16jEaaiJ/S0lKEEEiSRHBwcJN6n1+mngJUF9fz7Qcn0JYZGDQ1mgE3dkFqo1hvsDbwTso7LDu1jCD3IJ4d9izDQoe1aaz20NFi6F0gDHgR2HLWpScM2CqEuKZ3651pkblqVFVB164wciRs3Hi9Z+PEiRMnF9EJxZBznTpLg7WB7fnbSU5P5lDZIeSSnFHho5gVO4vhYcNRyJqP9FjKysiZdStyDw+i1q4hL+M0u1Z+RmVBHoHRXRk5bxFRvftedNNp1Zkoe+0QqmhvVLdGsXTpUiwWC3fffTcquQcfPr8P6q34hXhQW2IgINKTYbO6Ed7dh5SUFDZu3IiXlxfz58+/KHXrSo5x4OgdU/jH+9H//DPh77yN55h2logJATZLM0LpQiF19rWGWsj6HrJ2gN0K/nEOUZQ4iwJZKM9tPMX2tDJiA9U8Py2RoV39rnz+a4DZbObrr7/m5MmTJCQkMG3atGZrQurMdSzcspAyQxnLpywnxrv1PYkKUo+z7cN30JYUET9iNKMX3oO79+VNoux2M6fSHqGsbCOREXfTrdtjF/3epRbX8GjyCU4U1TCuRyAvTE8kVNNxtTb1+w9Q9PDD2OvrCXn+ebxvntoh41YXF/HDZx+Qm3IYZaCGr7ucYcnNjzA7bnaLxxDCTm1tCgWF6ygr/QYkHXa7DLPZDYvFHbncF3f3UDSaKPz9u+LhEYpKGYhSFYjSxfeSPbTMDVZ2fnGG9ANlRPT0ZcJdPVvl0PdLjlUc46ndT5FTk8Os2Fn8dcBfr2mUqKPFUAnQ7axlabUQwvfs6006c18Lrvcic8146SWHkcLOnTBq1PWejRMnTpw0oROKod/8OnWm+gzJGclsyt5EnbmOcHU4s+JmcUvXW1rcxd5w6BB5d96FeuRIwt95G4Hg9K6d7F6zgtqKMiISejNy/iJCujWte6j7uZCazTn4LYhHHyRYunQpHh4e3H333ZRrrXz6rwO4I6OsiwoR4Y63uwuqslQaitLwDAij7w2TCfTxQuPugsZdiYdSjl1vOe8YNyUG9YiLm48KISh97jl0q1YT/Owz+Myd22HXs1XUV0HaBjiRjMjbjYQgVUSzWQwnZPh8bhs3FGUrGm9eC4QQ7N69m+3btxMcHMzcuXPRNONkW6QvYv7m+bgr3Pnipi/wcW1ZjZOxrpadyz8m9cfteAcFM/6eP9Kld99WzM9OesbzFBZ+TkjwTHr0+DeyXwh5q83Op3tyeXVrOjIJHpnco0PTDy1l5RQ99BDGw4fxmT+fwEf/juwyRgItRQjB8b072PDhq3gYZMQOHsbohffg5X/pv1MhBHV1Jygr20RJ6SYsljLsdhnV1aEoXeLx9VXi7m5BJq/DYqnEZCrHaq25aBxJkqN08UepCkSlCkSpDDgrlM4+KwPJTbGz58tK3DzcmHRvIiFd2271brKZeCflHT5L/YxA90CeHfosw8OGt3m81tDRYigP6C2EqDm3yEiSFADsE0J07YD5tpjfjBgyGiEuDsLCYO/eK+YjO3HixMm1pBOKod/kOqU369mSu4Xk9GRSq1JxkbkwPmo8t8beyoDgAW1yAqtevoKyf/wD//vvJ+D+PwJgtVg4vv1b9n21CmNtDbGDhjH8tjvwC3ek2gmboPzto9jrLQQ93J+C0iKWLVtGaGgoCxcuZNPxMr46WojWaKFOb6RHwylCJB1p1kAOWCMQNJ1nnEzBi8INbwErAuQUBqjwcVc2iiUfdxc07i4EbVqN66fv47rwTkIf+SsqReuKvjua3ZmVvLluJ710O7hDfZAupjOODZHDIHEmJMwAj9anQl1N0tPTSU5ORi6XM2fOHLp06XLRPscqjrH428Uk+ify4cQPL9uHSAhB2s8/8OOyjzAZ6hkwdQZDZs29oiHHpcbKyX2bnJw38PcfT2LCm8jlF49TUG3gifUn+Sm9gr6RGl6c2ZvuwR3jqC8sFspfe53qTz7BtXdvwt94HZdfWL23hSd2PcG3md/wvMu9ZH77PQCDp89mwM0zUZwVXEII9PpTlJVtpqz8GxoaChBCTnV1MNVVMYSFTWHo0LH4+zf/O2WzmTCbKzCbyzGZKjCZyzCbyjGZKxqfTaZyLJaqZo6WsJk9sRi88fQOwT8k0iGeVIGolIGoVAEozz7LZFd2bDxecZyndj9Fdk02M2Nn8tcBf8VTeXW7HnS0GPoP0A34C3AYSADeADKFEE+0c66t4jcjhgA++QQWL4a1a+HWW6/3bJw4ceKkkU4ohn4z65QQgmMVjp5A3+V+h9FqpJumG7fG3cpN0TehcW1fIEwIQcljj1Ozfj3h776L59jzaWdmo4HDmzdwaNNXWBpMJIwex9Bb5+PlH4Apv5aK946hHh6GZmoMqamprF27lvj4eGbPno1MJqOqqoqVK1dSXV3NxEk3EtOzFzUGC1qDBa3BTI3Bgiy/jn5HqrHIYFWEklPCRo3RsV1rsGC2Oup5Rhcc4e+Hv+CH8L680n8eQpLhrpSjcTsrmDwczxo3l2aElON7H3cl3m4urYok2KxWDDU69Noq6rVaXD09sXsH8cqPBWw8VkyUnzvP3pLAmO6BUJUFqV/BiWSoSANJDjGjIXEWxE8F1/Y3V+0IKisrWblyJVqtlhtvvJEBAwZcFIX7Nudb/vbT35gaM5V/jfhXszU62tJitn/0LvknUgiJ7c6EJX8iILJLu+dXWLicM+nPotEMJKn3BygUF99ECyFYn1LE8xtPoTdZ+f0NXfnDmG5NbLjbQ+3WrZQ89jiSiwuhr7yCemTbXd/2l+znnq33cG+ve/lzvz9TW1nOzmVLSd+/G++gIIYvmIyLXz7l5ZsxGvMAOQZDFIUFQej1sfTvP5JBgwahbkH/opZgt1swmysxnxVHJnM5ZlMFRkMpRdnZmM0VuHrVISlqANtFxysUGlSqX0SYVIGolAEoVUGolAGoVIFYkfNeynt8kvoJAW4BPDP0GUaGj+yQ99AcHS2GlMDLwD2AO2AAPgT+LoQwt3OureI3JYZsNujTB0wmSE0Fl85jD+rEiZPfNp1QDP3q1yltg5aNWRv5KuMrsmqycFO4MSV6CjNjZ9LLv1e7bYAvxN7QQN782zHn59Nl7RpU0dFNthtqaziwfg0p320GSaLPpKkMnj6bhu1l1B8oJfD+vijD1Ozdu5fvvvuOIUOGEBsby9q1a5Ekidtuu63ZCET9gVK06zNxCXTD784EFJqmUQAhBA0WOxW79qJ/4A9YuydQ8NiLaC0O+2XdWWGlM5jRnRVQurPf2y9zu+PlqsDXXUGAwoKf1IAGA542I27WepRmPfKGWqivxarXYa6vc9QV/XLucg88gsNI6tWdoKho/MIj8QuPROV+tl9TWSqc+BJOJoMuD+QqiJ0AvW6FuMkOd7rrSENDA8nJyWRkZNCvXz8mTpyIq2vT6//+sfd5O+Vt7u9zP/cl3df4us1q4dDGdexLXoVMoWDkvEUkTbixQ5vdlpVtIvXUX/Hw6EafpE9QqZq3Bq+uN/OPTaf46mgRMQEevDizN4Oi2+6OdiGmnByKHngQU0YG/n/8I/5/+H2r36PJZmLW17OwCztf3fJVY2NbfX0G6SeWUl7+DS7qeoSQkEk9KCoOJS/XD7U6iKFDh9K3b9/L9vzpaIQQHPu+gL1fZaH2dWHc4nA8AwyYTGUXiKfz0adzz83925XL1ahUAVgkD1J1BRQ11BOp6cWErrfi7X428qQMRKHw7JD/Z1fNWvts2kGlaMWBkiRFAMuAYMAOfCCEeFOSJF9gNdAFyAXmCCG0lxvrNyWGADZvhqlT4Z134A9/uN6zceLEiROg84mhC2nLOtWRdOQ6ZRd29pfsJzkjmR35O7DYLfT2782suFlM6jLpqjY2tBQVkXPrbOS+vnRZvbpZi+HainL2rP2CUz/twMXVlUE3ziI8PRKFrxuBv09Ckkls2bKF/fv3AxAYGMi8efPw8WladyLsgtrvcqnbWYgqzge/+T2QuTZv9GDKzCR3/u0oAgLo8sUK5N6Xj67Y7Tb0Wh0V5RWUl5ZTXVlJTWUlem01DTVaLHU12OtrkBr0SDT9lbEjYZC7US/3wCB3p17hQb3cnXq5Bw0uHlhVamjQM9DbxBAfCw0VxVQVFmA1n++p5OkfgH94JH4RUfhHROEXFoGfVIlL+teOqJG+DJRq6D7FIYy6jr1uvZHsdjs7duxg165dKBQK4uPj6dOnD9HR0chkMoQQPLHrCTZmb+SVUa8wOXoyRWfS2P7h21QW5BE7eBhj77wPte/VMYqoqvqZ4yd+j0oVQN8+y3Bzi7jkvj+lV/D4uhMUao3MGxTJozf2wNut/dfVbjRS+uxz1GzYgMeIEYS+8jIKn5b3inr76Nu8f/x9PpjwAUmaYMrKNlFW/g319emAhKdnf4qzfTh1Ro3ZrsZTpWTspEn0TurT6t4/HUlJpo7vPkqlQW9h5G2x9Gymhu8cQgisVt15odSYkneBgDKVY2goRob1ouNlMldUykA0PoPpGf9im+fcIWJIkqTIKw0ghMhvwWRCgBAhxBFJkjxxpDBMB+4EqoUQL0qS9CjgI4T4++XG+s2JISFgzBg4dQqysqAZO0QnTpw4udZ0FjHUUetUR9IR61RpfSkbMjewLnMdRfoivFXe3BxzMzNiZxDn03xvmKtB/b595C++G89x4wh7681L3vxUFuSxe/XnZB7cR2zAAPqpx+F1SzRew8Kx2+1s2rQJi8XC1KlTUama1hcIi43qNekYT1TiMTgYzS3dLnKMO4elvJzcuXMRZguRX3yBzUuNXltNvbYavbYKfXU19brqC16rxqDTIX5plS1JuHt5o/bxQ+3ri4fGBw8fP9Q+vqh9fVH7+OGh8UGp9qLObD8fbTJcEG0ymqkxWhjRLYBJCUGN18Zut1FbXk5lYT5VBXlUFuRRVZhPdVEBtrN9XpAkvAOD8AuLwN9Lhr85G7+qPfhShsJDAz2nOVLpooaD7NrfABcVFXH06FFOnDiByWTCy8uLPn36kJSUhKfGk3u33suZklQerL+Fot0H8fT1Z9zdv6Nr/8FXfW41NUdJOXYPMpmSvn0+Ra2+uInpOQxmK69vS2fprhz81SoenxJPQqhXY5qki7xtkSshBLo1ayn7xz+QB/gT/sYbuPXufcXjsnXZLPlmBrNDo+njbkOvPwWAt/cAvDzHkp2t4fDhTCwWC12ionCrqaRg7048ff0YtWAxPYaN6tAIcGsx1pnZ9skpCk5V031wMDfM735Zm/IrIYTgZNlBXjvwNDpDPmP8ezDJNxaZVYvJXImrMohug95r8/gdJYbs0PgxSXNXXwghWn0VJEnaALx99jFaCFFyVjD9KIS49G81v0ExBHDgAAweDE8/Dc89d71n48SJEyedSQxdlXWqPbRnndpbvJcVaSv4uehn7MLO4ODBzIpzNEZVya9cpHw1qPrkU8pfeomAv/wF//uWXHbf4vTT/PzFp3TTJuDrGoppvIzu4264ZCqRrc58gWNcNOoRYSAExrraJoJGr61CX15O+Y7tGM0mrAH+GOr1CPvF/YDcvLwdosbH1yFwfH3x0Pief83XF3cvzWWbyV4N7DYburISqgryqSzIaxRL2pIi7DZHHYYkSWjUMvykKvyVtfh5ueDfexQ+Q+cijxp0zc2ULBYLZ86cISUlhaysLIQQREZG4uOu4NS3a1EZ7fSYMJ6JC36H0vXapfnp6zNISbkTm81AUu8P0Wgu/6/oeKGOR5NPcKqktsnrnioFGo9zNWWOejKfC+rJLqwzO/e9WqVoFCTGEycpeuABLBUVBD32KD7z5jUrVozGQsrKN7M/4138ZXoAvL36Ehh0ExIDOHAgnZMnTyJJEomJiQwbNozg4GDA8Te145P/oyw7k/D4RMbedR8BUdEXneNaYbcLDm/J5cCmHHxDPJh0dzy+vjZoqLngUXv+a1PtL7b94mGqxSzsvK/xZqnGCz+bjWcqqxllbICIIXD3d22ea0eJoRTAFfgMWA4U/3IfIcTFlVSXn1gX4CcgEci/0PJUkiStEOKiWKMkSUuAJQCRkZH98/LyWnPKXwdz5sA330BmJpz9A3HixImT60UnEkMdvk61l/aIoXdS3iE5PZnp3aYzo9sMIrwunQZ0rRBCUPzXv1H7zTdEfPA+6pGXL3gWQpD38yFkm+spqD9NlttJRsxfRHRSf4z6uvMRm4JqPA4qkFlkZKtPUWTIoF6rpV5X3SgOLkQpyVAZGvCOi8O7azdHVMfHFw8fH0ckx8cXD40GueJ/q77WZrWgLSlujCBVFeRTmZ+DrszRNBNAhh0fNxt+IUH4d++PX/wA/MIj8QkORXaNUqdqa2s5sGcPB/fvwyRAEoIKt1Jqwg28M/sdvFRe12Qe5zAai0g5toiGhhJ6Jb6Nv//l+0tZbHYO5lRToTc1jfCdNeY496w1mKlruDh16xwucglvt/MCKUgycfPmD4hIP0rZoBsov/chvHy88FZWo7T8iEm/DWP9cQDyTDKCAqcwpuffKCk2sWfPHrKyslAqlfTv358hQ4bg3Uzap91u4+QP2/h55TJMej1JE6cwfM4CXDvCQMFqvkC06JoXNBeJmloKqoLZVrIQi3BhjNe7xLntuvQ5lJ4Os5DGh9f5r1Xnv0616XkqbwMZhhJm+o/kzxG349e77TbcHVYzJElSIrAImAOcxlH785UQwtiGSamBncA/hRBf/bL/w6XE0IX8JiND4BBB8fFwzz3wXttDhk6cOHHSEXQWMQQdu051BO1ZpwwWA0q58pKNUa8XdqOR3LnzsJSUEP3lWpSRV8xOpGZrLnU7CjjYsI3skiPI5PJGkRPkGsWwoBnY7BYO1G3B6mnDozGacy6K44eHjw8eGl/q3vwvdcnJBD/3HD63zbnab7dTYDWbqS4upCorjcoTP1KVfZrKagM1FlfOBUHlCgW+oeHn65HCI/GLiMQ7MAhZB6bX2W02jmz5mt1rlgMSCTfNwOjmybGTx7GarVhVVkYPGk2/vv3w9e0Ys4KWYDZXkXJsMXp9GvHxLxMSPL1DxrXa7OiMFwikenOT73UGM9r6CwRVfQPjj37LbQVbKB3qTuFwNVH+hQDk1YZzqDyBVNf91DZE0Kv2dsJM+agsdQiFK65h3QmM7om/Rt3U/dDDBc8LolAARn0de9Ys59jWLbiq1YyYt4heI29AatA2I1p0LYjS1DoaCV8OSXaRaDn30ItAth7uS0mlJ4k96xkxRkKubkbstOJ3sT4jnd3/fRK/H09QGeLGxG8Ptzk1sMMNFCRHu9oJOOp8bgTGCiGOtGJCLsAm4DshxGtnXzuDM02u5fzpTw4hlJoK3S97mZw4ceKkeVasgCeegPx8iIyEf/4Tbr+91cN0JjF0jvauUx3Fr3WdMhcUkHPrbFyCguiyaiWycy5pl0BY7JS+cRiAiqRqdJUlqH188a7xQXUMZL5K/Bb1RBV4+VrYyv97n4o33sBvyRICH/pLh72f/0n0FViOfUn1gfVU5mVTaXKnSgql0uxBXV1D424KpQrfsPBGgeR/Vix5+ge0+sayLDuTrR/8l/KcLGL6DWTc4t/jFeBoDmqxWPj8x885cOQAwUZH1kpkZCR9+vQhISHhovqwq4HVWsfxE79Hq91LbOyTREbcddXPeSEmcyUV5d9SVr4Zne4gIFAUy5C5jacmehFaUzDr817BXl5CQu0AFFYzDXIPcuVhnDb7oGu4ONVTwo4GPcHyWqJVeiKUdYS51BIkqyUQLZK+hpO5EmV6FUGudYwNyiLUva75CcqVzUZhmkZoNJfeplRfNj3TZrOzb302KdvyCYj0ZPKSRLz8W5cyKaxW9D/+SPWKFRj27kNycUGMHUbdzSMYNH5Bq8a6kKshhrrj+ORtPpADLBZC5LTwWAlHCkO1EOLBC15/Bai6wEDBVwjxyOXG+rUuMi2ivBy6doWJEyE5+XrPxklr6aCbUCdO2syKFbBkCRj+v737jo+qzho//jnplSSkUFNooUMAaSKoYEMR7Arouj678Mjay6M+6k8fV3dFV91lcS1rxQKsil2woYB0AUOH0AIJLb33me/vjztAgCQkZDKTTM779ZpXhsmde89NNGfO/X7v+ZaceC0oCP797wb/t9hMi6GzzlOO978NTAAyjDH9HK8lAa9hTcWrAv5kjFlb1348OU8VLV9B2vTptLnsMjq++MIZP1iXpeSS9fYW2lwUR+jYuHp3jDsm/8svOfTQw7SZMIGOzz/n1FbNLV7eAdj6mdWu+8gmym0+ZLcdSXbYYLJNNFlHMshO209Rbs7xt/gGBBLZOfZEkdQ5jsi4eEIiIk/7XVaUlbLiPx/w26KvCAoPZ+zvp9Nj+Kgaf+cvrX+J+cnzuSX8FjgE2dnZ+Pr6Hu9Gl5CQgFcT/u5stnK2bruPzMzvSEi4g65d7mvSRgMVFTlkZn7H0YyF5OauBuwEBXWnXbsriPQaRu7DsyhNTsb/lpv5pXdHdm7ZhZ/dj7jOnTg3KZHEaD+8ijOgKAN7wREqCo5gKzgChUfxLsnArywbL3P6VL0y/MgiggwTxhFbGOmFoRRm2sBmpyIolODE3vQeMoikHglER8VYBY1PgEvuNdubnMniOdsRgXG39qbLwJpbn1dXlZND3sefkDt/PlWHD+PToQMRN95I+PXX4RPZ+I6EzrpnqC0wGSu5hALvAx80tDOPiJwH/AJsxmqtDfAosAb4CIgDDgDXG2NyatyJgycnmXp5+mmrkcLKlTBypLujUfXlxA+hSjWYMXDwIAweDJmZp38/Ph5SUxu0y+ZSDDkrTzn2NQYoAt6rVgx9D/zdGLNIRC4HHjLGXFDXfjw9T2X9+w0yX3qJmIceIvK/znwVPnveDkq3ZBHQPZyynbln7Bh3TPHqNRyYNo2gQYOIffMNvFy4tkqLk5lirV+05RPI3g1ePtBtHPS/jrLOo8nKyCE77QDZ6QeO35tUkp93/O3+QcHHR5AiY+PwDQhg1cfzKMzJYuBF4xk95Vb8g2pv4243du5fcj8/p/3MrAtm0U26kZyczJYtWygvLycsLIyBAweSlJTUZNPojLGxY8fjHDr8EZ06TqZnz6cQcd5UwcrKPDIzf+Boxjfk5q7EGBuBgQm0ix5Pu9CRBFcFIcWZUHSUrEPpLF6axs6ASOxeXhT5HWKGWUe3qr2n71i8IDgaQtpVe8Sc8tXx3D/0eGFTWmEjt6SCzNwCfvt+ERmrvse7tIAs37ZsCEuCrkmM7tmeMYnRDOvS1mmLz9YlP7OU797YQuaBQgZdHMfwq7rifUrHPmMMZZs2kTt3LgULF2EqKwkaOYKIKVMIvfBCxImK3WSeAAAgAElEQVSNTZxVDJVhXV17H1hd0zbGmJ/ONsiz4elJ5oyKi6F7d+uxbJnLO8uoBqiosD54Hj0K48dbI3un6tTJGinSq53KWUpLram0GzfCpk0nHjl1XGcSgRq6ctWlGRVDTs1TjiY/X1crhr4D3jbG/EdEJgNXGmOm1LUPT89TxhgO3nsfhT/8QNxbbxJ8hgtztoIKjry4DlNhI2x8F0JGdzrjVfvyXbustYTaxZDw4ZnXElIOxsCRTY7FXT+FgnTwCYTES601jLpfDL7WIp8lBflW6+9jTRscRVJZkTXdKio2noun30nHxN71OnRJZQm3fXcb+/L38f749+nZtieVlZXs2LHjeDc6gPj4eJKSkujTp4/Tp9EZY9iz90X273+VmJjL6dvnBby8zuIYxkBZPlX5+6wpcAXLyalIwWAn0BZITEkI7bLthGRnI8XZHGtomUYHVjCEHXTHGxuhFQcYvGQ7keWlJNw6guDBfU8vcoIindI+3VZVyY4Vy1j+6ccUHUmnIqAN64L7symkF15+AQzvGsmYHlGcnxhN95iQJhs5q6q0sfzj3WxddpAO3cO45A/9CInwx15WRsHCReTOnUvZli14BQcTdtVVREyZjH+3bk0Si7OKoVSgrnl0xhjTteHhnT1PTzL18vrrcPvt8MUXMHGiu6NpXYqKrOImI8P6Wv35qa/l1rl+8AlBQdCjh/VITDz54YRhYuWhjIG0NKvQqV74pKScKGyCgqB/fxg4EAYMsEaWjx49fV8te2QoFSfmqRqKod7Ad1h3rHsB5xpjTmtp2tq6ntqLi0m96SaqMrPosuATfDt1qnP78v0FYLPj3zW8zu0AKo861hKqqqTL/Pln3Leqhd0OaWusEaOtn0FJlnVfSK8J0P9a6HIBeJ98Fd4YQ0l+HgWZGcR06drgznwZJRlM+ca6VjDvinlEB52YKpWfn8+mTZtITk4+Po2uT58+JCUlER8f79RpdAcOvMWu3X+lbcQo+vd/FR8fx6hWZRk4pqdRdBQKj5x47vhaVXKULP8cjrb1IrutH8ZLCCizEZNZTrscO6FEIdVGcezBMaSUtGFFahlpWUUE+Psx7JwhRPXuwK0/3cpNgaO5bs5eKvbuI/ruu4icPr1Jp3saY9iXvI5fv1xA+rYteAcEUtptBMt8e7E939qmQ1gAY3pEMyYxmvO6RxEW5PwOjClrj/Dzhzvx8YZh4Tvw/+YdbPn5+HXvRsSUKYRNnFTjQs7O5PR7hpoLLYaAqiro188aTdi0CVy8VoJHsdutouVMhc2xr9WnuVUXEQHt2kFMzMlfjz2//faaP4S2bQu33mp9gE1JgX37rN9v9e/XVCR17w7OaKmpWobi4ppHe/JOTHOhS5cTRc+AAdbzrl1PHnX08HuGnKGGYuifwFJjzAIRuQGYboy5qK59tJY8VZGayr7rb8A3tjMJc+fiFRDQ6H3aiorZf8stVOzfT/z77xHYt68TIlXYqmDfUqsw2v6V1VUsKAr6XmUt7ho7wmkzFHbk7OB3i35H17CuvHPZOwT6nHwzvTGG9PT006bRHVvU9fg0OrsNKoqgoth6lBeeeF5RdOJ75dWeV/t62PcI22OyCC0VBu6swq+kGGwVNf94QqLIahfG0QhDdmARdrHjTwgxgUm0C7+ANpEjkdB2VqMBx4hKVVUVmzZtYuXKlWRlZREWFsbIkSMZNGgQvn6+3LroVlILUvnyqi8Js/lx+Mn/o+Drrwk5/3w6Pv+cS0Y7D+/eybovPyVl7Uq8vb2JHz6GosTRrMzyYvnuLArLqvASGBgbzuge0ZyfGMXAzuH4nOVitMcYu53ilas48MGXrC0eQHFQO3r57GT4zUMIHjHcZQvHajHk6T77DK65Bt54w2q37Wka02ygstIqXOoqbI49z8w8ufg4xtsboqNrLmxOfS06Gs40l72+H0IrK60r9MeKo2OPXbusUYDqOnY8vUjq0cP6AKxz61smY2D//tNHe3btsr4HVhFcfbRn4EDr4kibeq7x4cHd5JyhhmIoHwg3xhhHM6B8Y0ydP+zWlKcKlywhfcafCJt4JR1mzmzUhxxTWUnajD9RvGoVsa++QsiYMU6MVB1XWQa7f7TuL9r5rdVauU1n6Hc19LsOOgxs+BR8Y6Cy5HhhsiR9GXcn/51xbfvxYsI1eFWWOoqUagVNeREVZcXsyPUhOb8Ne8vDACHe6whJbKePfSv+VNbv+D4BVtczv2Drq7/1PDOknC1tdhFgghhkLibAv/3xER1bUBjZtr0cLVhOVvYS7PYy/PyiiYkZT7uYKwgLG4zVoPJkpaWlrFu3jjVr1lBUVET79u0ZNWoUffr0wdux5tPHKR/z51V/5plRzzCp+yTHj8iQO28eR5+diW9MDJ1mzSKwn2uK/dwjh1j/9edsXfIjVVWVdBsynMETriYzqANLU7JYlpLJpvQ87AbaBPgwqnsUYxKtkaNO4fXvDGcrKCD/s8/InTuPiv378Y6MJOTa69kcMIrdmwqI7dOWi2/rQ2Coaz6jaDHk6YyB886zRhJ27YLgph1qdKmaCofAQOtD24gRZx69qe3eiICA04ua2p63bev8+3ga+yG0pMRab+rUIiklBbKyTmzn5WWNElQvkI49j43V+5Oai6Ii2LLl9MKnoNoK6d26nT7ak5DQLH6HragY2g7MMMYsEZFxwPPGmCF17aO15anMf/2LrNkv0+6xx2h7y9m1wTXGcOSJJ8j7+BPa//kpIm5oHWsJuV15IexcZN1jtGcx2Ksgsjv0mWTdrF9edPpITI2jNMWcOlv1/TahPB8ZwX/l5XNfbv6Jb3j5OoqVasWLXzD5Es7GkiiSc4PJKffG1wv6tPcnKS6c+I5RePlX397x3D8EfINPm+5XXW7uWjZumoaPTygDB7xBWVk6RzO+IStrMTZbCb6+bY8XQOHh59TadCEvL4/Vq1ezYcMGKioq6NatG6NGjaJLly4nXQTIKs1i4mcT6R3ZmzcvefO0CwSlGzeSfu992LKyaPf444TfcL3LRkpK8vP47buvSf7uG8qKCunYsw9DJ15Lt8FDyS+rYvluqzBalpLFkQKrVXu36ODjhdGILpEE+p3+8ynbuZPcD+eS/9VXmNJSApOSiJg6ldBLL8HLzw9jDNuWH+KX/+wiIMSXS6f1o0O3ph8Z02KoNVixwiqIwsKsD1DuaNdsjDWaUVp6+qOsrObXz7TNt99ar9dHeHjtU9NOfR5Sd6/8Fi0n50RhVL1ISkmxplgdExBgTbE7tUhKTLRGuDz15+NOdrs12nes2DlW+OzZc2K0JzT0RLFTfbSnGU+F9MRiSETmARcAUcBR4ElgJzAL8AHKsFprr69rP60tTxm7nfQ776Jo6VLi332HoKFDG7yPrNdeI/Mfs4i8/b+JuffeM79BOV9JDmz/0iqMUpcDxup05hfqKEKCayxiqo/EWA9re+MbxDOpn/PRwZ/5c9I9XN3jGmtbn7pHBYwxpKWlHZ9GV1FRQXh4+PFudBEREQ0+tcLCbSRvvI2KCuvCoa9vBNHRlzgKoOF41bHI8ZEjR1i5ciVbtmzBGEP//v0599xzad++fY3bP7T0IRYfWMyCiQtICEuocZuq3FwOPfg/FK9YQdhVV9H+ySfwCmzY2jyNUVlWxuaff2D9N59RkJlB246dOefKa+g9+kJ8fH0xxrA7o4ilKZks25XFmr3ZlFfZ8fPxYlhCW8YkRjG6SwQdN68md+5cStetR/z9aTPhCiKmTKl1emvmgUK+/fdminLKGXlNNwaOi23SQlCLodbgww+t+00cK3oD1tSrf/4TrrzSuQVKXds0sAvVSQIDTzwCAqyvW7bUvv0335wocGJiwAULurVoxsDhwzUXSXv2WIXsMWFhNRdJPXrUPQVL1086obAQNm8+uejZvNl6Haxis3v300d74uNbXCHqicWQs7TGPGUrLCT1hhuxFRRYDRVq+aBYk/wvvuDQw4/QZuKVdHzuOZddJVd1KC+yOpw1co2aSnsldy6+k7WH1/L6xa8zrMOwBr2/oqLieDe6vXutttTx8fEMGjSI3r17N6gbXWnpAQ4f/oywsEFERIzEy6v2pgHGGPbt28eKFSvYs2cPfn5+DB48mBEjRhAeXnsTkOUHlzPjxxnckXQHtw+8vc54jM1G1iuvkvXKK/gnJtJ51j/wS0io9/k4g91mI2X1cn798lMyUvcQHB7BoPETGXjxeAKCT1yMK6u0sXZfDstSMklO3kXPXxczfv8aIssKKIiIofzyq+jzh5uJ6njmtYXKSypZPGc7+zZm0TUpmrG/64V/EzRwAC2GWoeEBOv+Amfx9z+5OKmpUHHm9/39a/4jW9t5nUXHK1WHqirr53xqkZSSYhU21f82tG9fcyOHX3+FP/3JM9dPqqvIs9th797TR3v2VltDIizs5NGeAQOs0R4PmdKqxVDtWmueKt+zh9Trb8Cve3fi338Pr3p8UC1etYoD06YTNGQIcW/8G9H7HT1OYUUhtyy8hczSTD64/AO6hHU5q/3k5+ezceNGkpOTycnJwdfXl759+5KUlERcXJxTutHZbDa2bdvGihUrOHLkCCEhIQwfPpxzzjmHwDOM3JRWlXL1F1fj5+3HJ1d+gp93/f5bLvplOYcefBBjs9Hhr3+hzSWXNPo8GsoYw4HNG/n1qwXs3/QbvgGBDLjoMoZcPonQyCiMMZSuX0/Ohx9S+MOPUFVF4YChLOl7AXNNR/LKbIhA/05hx7vUDYoLx7eWRgzGGDYuTmPVp3sIaevPZdP7Ex0X6vTz0mKoNfDyOvkDa3Uvv9ywQiUgoFnchwDoAqXNQVmZNXJ0aiOHlJSa10s6VWgoTJtmNaJoiY/580//b9DX15qWWlpqjfYcm37o5WUViqeO9sTGtrjRnobQYqh2rTlPFfzwAwfvupuw666lw9NP1znKU5aSwv4pU/Ht0J74Dz/Eu75NQFSLk16YztSFUwn2DWbu5XMJDzhzi/Xa1DaN7lg3urOZRldRUcGGDRtYvXo1eXl5REZGMmrUKAYMGIBPPTv2vrT+Jd7Z8g7vXPoO57Rv2J/GykOHSL/3Pso2baLtbbcRc/99iG/TjJacSUbqXn79cgE7V/2CiNC1UzxxKakE7NyFV5s2hF9zDRGTb8IvPh4Am92wKT2PX3ZZ9xv9lpaHzW4I9fdhZLdIxiRGc35iNLFtg0471uHdeXz35lbKiioZfWMP+pzX0akjw1oMtQaePIKiU6+ar/z8EyNJdf1OgoKsKZzHHi3sb02NvLxg9OiTC5++fa1zbWW0GKpda89TGbNmkf3qa7T/v/8j4qYba9ym8uhRUm+8CWw2Ev4zH9+OHV0cpXK15Ixk/vDdH+gX1Y83Lnmj3iMndalpGl1CQsLxRV39zjDSWFRUxNq1a1m7di1lZWXExcVx7rnnkpiY2KCRpp05O7nx6xuZ1H0ST5371Fmdi72igoyZM8mdO4/Ac4bQ6aWX8I2JOat9NVZFaippc95h46pfOBASgM3bi87tOzHi1mnEDRpSZ8GSX1rJqj1Zx7vUHcwrBaBLVDCje0Qxpkc0I7tFEuxvFZmlhRX88M420rbl0HN4e86f0hNf/8YvQgtaDLUOOoKi3K0hBbkxJxdHtT3s9vpt15SPJ5+s+XxFGnePnAfRYqh2rT1PGZuNtBkzKF61mvg5cwgaPOik79uKith/8y1UHjhA/AfvE9Cnj5siVa62cO9CHv7lYSZ2m8gzo55x6ihAXl7e8UVdc3Jy8PPzO2lR1+rHys7OZuXKlSQnJ2Oz2ejVqxejRo0iNja2wce12W38btHvSC9Kt9YU8m9cl7T8r77m8BNP4BUcTKeXXiR4WMPuszpbxmajaOkycufOpXj5cvDxoc2llxJ4zVXsPJLOb99+RWlBPu279eCcK6+lx/CReHnVXbQYY9ibVezoUJfJ6r05lFba8PUWhsRHWF3qekTTq10ov323n7Vf76Nth2AundaPth0aP6Vci6HWQkdQlDt5akHuyaOuTqLFUO00T4EtP59919+AvbSELgsWHL/CbSorSbt9BsWrVxP72muEjD7PzZEqV3t146u8kvwKdw+6m2kDpjl9/zVNo4uIiGDgwIF07tyZdevWsWPHDry9vUlKSmLkyJFERUWd9fHm75jPX9b8hWdHP8uErhOccg7lu3aRfvc9VOzfT/R99xL5xz82WWORqtxc8hcsIHfefCoPHsQnJobwm24k4vrr8Yk+0RChsqKcbUt/Yt3Xn5J35DDh7TowZMLV9L1gHL5+9WtkUV5lY11qLstSMlmaksmOI1ZzoagQP0b3iGZ4UCBFSzKwVdm58OaeJA6tfyOWmmgxpJRyDU8syD21yHMiLYZqp3nKUpaSQuqNNxHQqxfxc94FX18OP/44+Qs+pcMzTxN+3XXuDlG5gTGG/13+v3yz9xv+dv7fuCzhsiY7VkVFBdu3byc5OZl9+/YBEBAQwLBhwxg2bBghjVy+IKMkg0mfT6J/VH9ev/h1pxYstqJiDv+/xylc9C0h48bR8dm/OvW+utLNW8idO5eChQsx5eUEDR1qrQ00bmyd9yvZ7Tb2/LqGtV9+wpHdKQS2CWPQpRNIuvQKAkMbFl9GQZl1r9GuTH7ZlUVOcQUhdrixMoi2pYbopEiunz4A8Tq7n6sWQ0op1RieWOQ5kRZDtdM8dULBokUcvO9+wiffhE90NFn/nE3Un2YQfffd7g5NuVG5rZxp309jW/Y23r70bQZED2jyY+bl5XHo0CG6devWoJbcdbl/yf0sS1/GZxM/I7ZNw6fYnYkxhtz33+fo83/Dt2NHOs/6BwG9e5/1/uwVFRR++y05H35I2cZNSFAQYZMmEjF5MgGJiQ2O7eD2rfz61QL2bvgVH39/+l94CUOumERYTMNHdOx2w9ZDBSzblcmynRkE7Siifag/zz59foP3dYwWQ0oppZqMFkO10zx1sowXXiD7zbcACJs0kQ4zZ+paQoqcshymfjOVkqoS5l0xj44hLauJxtK0pdz5051NNt2vupINv3Hw3nux5efT/oknCL/2mga9v/LQIXL/8xF5H3+MLScHv4QEIqZOJeyqSXiHNr6ldVbaftZ99Rnbly/B2O0kjjyPoVdeQ7uu3c96n4VllWQWltM1+uxH77QYUkop1WS0GKqd5qmTGZuNg/c/gLFV0fmll3QtIXXc3ry93LzwZtoFt+P98e8T4te4aWuuUlJZwlVfXEWwbzAfTfgIX++mb4NdlZ3NwQcfpGTVasKuu5b2jz+OV0BArdsbYyhZvZqcDz+k6KefAQi58EIipkwmeORIpAmWUynMyWLDwi/Z9OMiKkpLies3kKETryV+wCC3XADRYkgppVST0WKodpqnlKq/VYdWMePHGYzoOIKXx76Mj1f91vVxpxd+fYE52+bw3vj3GBQz6MxvcBJjs5E5ezbZr72Of5/edJ41C79TOuDZiorI//wLcufOpWLvXrwjIgi/7joibroR306dXBJneUkxm378lg0Lv6AoN4fo+C4MvfIaEkeOxrue6zY5gxZDSimlmowWQ7XTPKVUw3yS8glPrXqKyb0m8+jwR90dTp22Z29n8jeTubrH1Tw5spZlGJpY4ZIlHHr4EbDb6fjcTELHjqV8925y584l//MvsJeUEDBgABFTJtNm/Hi8nHSPVENVVVayY/kSfv3qU3IOphEaFc05V1xFv7GX4BcQ2OTH12JIKaVUk9FiqHaap5RquGOjLY8Me4SpvZtnsxqb3cbUhVM5UnyEL676otFrCjVGRXo6B+++h7Jt2/Dv05vybdsRPz/aXH45EVOnENi/v9tiO5Wx29n72zp+/XIBB3dsJSA4hIGXXMGgyyYQHB7RZMetb55yyViViLwNTAAyjDH9HK8lAa8BAUAV8CdjzFpXxKOUUkoppZqP+4bcx4HCAzz/6/PEhsYypvMYd4d0mvk757M1eyvPj3nerYUQgF/nzsTPm0vGc89Tsm4d0Q/cT/h11+ET0XTFxdkSLy+6DRlGtyHDOJSyg3Vffcqazz9i3def0nfMOIZMuJq2HV0zha/G+FwxMiQiY4Ai4L1qxdD3wN+NMYtE5HLgIWPMBWfal15xU0op99ORodppnlLq7JRUlvD7b3/P/oL9vDf+PXq27enukI47UnyESZ9PYlC7Qbw67lXtiNhIOYcOsv7rz9i6bDG2qip6DB3J0InX0qGH837n9c1Tzm8lUQNjzDIg59SXgWOrM4UBh1wRi1JKKaWUan6CfIOYPXY2Ib4h3PnTnWSVZrk7pONmrp2J3dh5fPjjWgg5QduOnbh4+p1Me/lthl91Awe2bmTu4w8w/8mH2bN+LcZud1ksLimGanEv8DcRSQNeAP7XjbEopZRqxUTkbRHJEJEtp7x+l4jsFJGtIvK8u+JTqrVoF9yO2eNmk1+ez12L76K0qtTdIfHTgZ9YfGAxM5Jm0Dm0s7vD8SjB4RGcd9MtTH/lXS68dRoFWRl8/vyfmfM/d7JlyY+4YgabO4uhGcB9xphY4D7grdo2FJHpIrJORNZlZma6LECllFKtxrvAZdVfEJELgUnAAGNMX6wLd0qpJtYnsg8zR89ka/ZWHlv+GHbjulGCUxVXFvPXNX+lR0QPbulzi9vi8HR+AYEMvnwSf5j1Bpff+QBe3t6krF7uklE4dxZDtwKfOp5/DAyrbUNjzL+NMecYY86Jjo52SXBKKaVaj1qmc88AZhpjyh3bZLg8MKVaqbFxY3ngnAf4Yf8PzP5tttviePm3l8koyeDJkU/i69X0i6u2dt4+PvQefSG3PPdPLr/rQZcc053F0CHgfMfzscAuN8ailFJKnSoRGC0ia0RkqYgMdXdASrUmv+vzO65LvI43N7/J57s/d/nxt2ZtZe6OudzQ8wYGRg90+fFbMxEhIDjEJcdyVWvtecAFQJSIpANPAtOAWSLiA5QB010Ri1JKKVVPPkAEMAIYCnwkIl1NDZPYRWQ6jjwWFxfn0iCV8lQiwqPDHyW9MJ2nVj1Fp5BODG3vmmsSVfYqnlr1FJEBkdwz+B6XHFO5h6u6yU02xnQwxvgaYzobY94yxiw3xgwxxgw0xgw3xqx3RSxKKaVUPaUDnxrLWsAORNW0oU7nVqpp+Hr58uIFLxIbGsu9P99Lan6qS447d/tctuds55FhjxDqF+qSYyr3cOc0OaWUUqo5+xxrGjcikgj4Ac2n169SrUQbvzb8a9y/8BZv7vzpTvLL85v0eIeLDvNy8suM6TyGi+MvbtJjKffTYkgppVSr55jOvQroKSLpIvIH4G2gq6Pd9nzg1pqmyCmlml5saCyzxs7iUNEh7v35XiptlU1yHGMMf1nzFwAeG/6YrinUCmgxpJRSqtWrZTp3hTHmZmNMP2PMYGPMT+6OU6nWbFDMIP486s+sO7qOp1Y91SRr0Cw+sJil6Uu5I+kOOoZ0dPr+VfPjkgYKSimllFJKNdaErhM4UHCAVze+SkJYAn/s/0en7buwopBn1zxLr7a9mNp7qtP2q5o3LYaUUkoppVSLMWPgDFILUpm1YRZxoXFcknCJU/Y7+7fZZJZmMmvsLHy89CNya6HT5JRSSimlVIshIjw96mkGRg/k0eWPsjlzc6P3uSlzE/N3zGdyr8n0i+rnhChVS6HFkFJKKaWUalH8vf2ZdeEsogKjuOunuzhcdPis91Vpr+SpVU8RHRTNXYPucmKUqiXQYkgppZRSSrU4kYGR/Gvcvyi3lXPHT3dQVFF0Vvv5YNsHpOSm8OiwRwnxC3FylKq502JIKaWUUkq1SN3Cu/HiBS+yN28vDy17iCp7VYPef7DoIK8kv8KFsRcyLn5cE0WpmjMthpRSSimlVIt1bsdzeXT4o/xy8BdeWPdCvd9njOGZ1c/gJV48OvzRJoxQNWfaKkMppZRSSrVoN/S8gdSCVN7f9j7xbeKZ3GvyGd/z3f7vWH5wOQ8PfZj2we1dEKVqjrQYUkoppZRSLd4DQx4grSCNmWtnEhsay3mdzqt124KKAp5b+xx9IvvUq3BSnkunySmllFJKqRbP28ub58Y8R2JEIg8ufZBdubtq3XbW+lnklOXw5Mgn8fbydmGUqrnRYkgppZRSSnmEIN8gZo+dTZBPEHcsvoOs0qzTtknOSOajlI+Y2nsqfSL7uCFK1ZxoMaSUUkoppTxG++D2zB43m7zyPO756R7KqsqOf+/YmkLtg9tzZ9KdboxSNRdaDCmllFJKKY/SN7Ivz45+ls1Zm3ls+WPYjR2AOVvnsDtvN48Nf4wg3yA3R6maAy2GlFJKKaWUxxkXN477htzH9/u/5+XfXiatII3XNr7GRXEXcUHsBe4OTzUT2k1OKaWUUkp5pN/3/T37C/bzxuY3+Db1W3y8fHhk2CPuDks1IzoypJRSSimlPJKI8NiIxxjefjhphWncPehu2gW3c3dYqhlxSTEkIm+LSIaIbDnl9btEZKeIbBWR510Ri1JKKVWT2nKV43sPiogRkSh3xKaUOnu+Xr7848J/8NIFL3FjzxvdHY5qZlw1MvQucFn1F0TkQmASMMAY0xd4wUWxKKWUUjV5l1NyFYCIxAIXAwdcHZBSyjlC/EK4OP5iXVNIncYlxZAxZhmQc8rLM4CZxphyxzYZrohFKaWUqkktuQrg78BDgHFtREoppZqaOxsoJAKjReQvQBnwoDHm15o2FJHpwHTHP4tEZGcjjhsFnL4CV8un59VyeOI5gZ5XS9PY84p3ViDNmYhMBA4aYzaKSF3baZ46Mz2vlsUTz8sTzwn0vGpTrzzlzmLIB4gARgBDgY9EpKsx5rQrb8aYfwP/dsZBRWSdMeYcZ+yrOdHzajk88ZxAz6ul8dTzciYRCQIeAy4507aap85Mz6tl8cTz8sRzAj2vxnJnN7l04FNjWQvYsSpApZRSqjnoBnQBNopIKtAZ2CAi7d0alVJKKadx550EULwAAAo4SURBVMjQ58BYYImIJAJ+eOYQn1JKqRbIGLMZiDn2b0dBdI4xRnOVUkp5CFe11p4HrAJ6iki6iPwBeBvo6mhhOh+4taYpck3AKdMYmiE9r5bDE88J9LxaGk89r7NWS65yB0/93eh5tSyeeF6eeE6g59Uo4pr6QymllFJKKaWaF3feM6SUUkoppZRSbqPFkFJKKaWUUqpVajXFkIi8LSIZjnuUPIKIxIrIzyKyXUS2isg97o7JGUQkQETWishGx3k95e6YnElEvEXkNxH52t2xOIuIpIrIZhFJFpF17o7HWUQkXEQ+EZEdjv/PRro7psYSkZ6O39OxR4GI3OvuuJRn5inwzFylearl0TzVcrg6T7Wae4ZEZAxQBLxnjOnn7nicQUQ6AB2MMRtEJBRYD1xljNnm5tAaRayVDYONMUUi4gssB+4xxqx2c2hOISL3A+cAbYwxE9wdjzN4apctEZkD/GKMeVNE/IAgY0yeu+NyFhHxBg4Cw40x+90dT2vniXkKPDNXaZ5qeTRPtUyuyFOtZmTIGLMMyHF3HM5kjDlsjNngeF4IbAc6uTeqxnOsPVXk+Kev4+ERVbuIdAauAN50dyyqbiLSBhgDvAVgjKnwpATjMA7Yo4VQ8+CJeQo8M1dpnlLNgeYp52g1xZCnE5EEYBCwxr2ROIdjiD4ZyAB+MMZ4xHkB/wAewlpk2JMY4HsRWS8i090djJN0BTKBdxzTRd4UkWB3B+VkNwHz3B2Eaj08KVdpnmpxNE+1TE2ep7QY8gAiEgIsAO41xhS4Ox5nMMbYjDFJWCu+DxORFj9lREQmABnGmPXujqUJjDLGDAbGA3c4pvu0dD7AYOBVY8wgoBh4xL0hOY9jOsVE4GN3x6JaB0/LVZqnWhzNUy2Mq/KUFkMtnGOu8gLgQ2PMp+6Ox9kcw71LgMvcHIozjAImOuYtzwfGisgH7g3JOYwxhxxfM4DPgGHujcgp0oH0ald7P8FKOp5iPLDBGHPU3YEoz+fJuUrzVMugeapFckme0mKoBXPcwPkWsN0Y85K743EWEYkWkXDH80DgImCHe6NqPGPM/xpjOhtjErCGfX8yxtzs5rAaTUSCHTdF4xievwRo8d2wjDFHgDQR6el4aRzQYm/4rsFkdIqccgFPzFWap1oWzVMtlkvylE9TH6C5EJF5wAVAlIikA08aY95yb1SNNgq4BdjsmLcM8KgxZqEbY3KGDsAcRwcRL+AjY4zHtPf0QO2Az6zPO/gAc40x37o3JKe5C/jQMVS/F7jNzfE4hYgEARcD/+3uWNQJHpqnwDNzleaplkXzVAvjyjzValprK6WUUkoppVR1Ok1OKaWUUkop1SppMaSUUkoppZRqlbQYUkoppZRSSrVKWgwppZRSSimlWiUthpRSSimllFKtkhZDSjmZiLQTkWUiUigiLzpxvxc42u02GyISJyJFjvayZ9q22cWvlFKtkeapWrdtdvGrpqfFkHILEUkVkVLHH+I8EVkpIreLiCf8NzkdyALaGGMeOPWbIvKuiFQ4/jjniMgPItLL9WE2njHmgDEmxBhjc3csSinlTJqnNE+p1sET/odWLdeVxphQIB6YCTyMtUq509XnipATxQPbTN2LeD1vjAkBOgMZwLuuCEwppVSDaJ7SPKU8nBZDyu2MMfnGmC+BG4FbRaQfgIj4i8gLInJARI6KyGsiEnjsfSLykIgcFpFDIvJHETEi0t3xvXdF5FURWSgixcCF9djfBBFJrnYFcEBtMYvIuSLyq4jkO76ee+y4wK3AQ44rahed4dxLgLlA9XP+h+OcDjme+9dw/P8RkQWnvDZbRP7heL5ERJ4WkRWOq5rfi0hUtW0nishWx7kuEZHe1b6X6tj/JhEpFpG3HFMqFjn29aOIRDi2TXD83H0c/75NRLY7ttsrIk2+crRSSjU1zVOap5Tn0mJINRvGmLVAOjDa8dJzQCKQBHQHOgFPAIjIZcD9wEWO751fwy6nAH8BQoHlZ9jfYOBt4L+BSOB14Mta/sC3Bb4B/unY9iXgGxGJNMb8HvgQxxU1Y8yPdZ2ziIQAU4HfHC89BoxwxDgQGAY8XsNbPwAuE5Fwx358sJL0+6ec/21ADOAHPOjYNhGYB9wLRAMLga9ExK/ae68FLnb8vK4EFgGPAlFYfzfuruWUMoAJQBvHsf/u+NkqpVSLp3kK0DylPIwWQ6q5OQS0FREBpgH3GWNyjDGFwF+Bmxzb3QC8Y4zZ6rhq9VQN+/rCGLPCGGMHys+wv2nA68aYNcYYmzFmjuM9I2rY7xXALmPM+8aYKmPMPGAH1h/j+npQRPKA3UAI8HvH61OBPxtjMowxmY7zuuXUNxtjDgPLgOsdL10GZBlj1lfb7B1jTIoxphT4CCtxgZWMvjHG/GCMqQReAAKBc6u9d7Yx5qgx5iDwC7DGGPObMaYc+AwYVNNJGWO+McbsMZalwPec+NCglFKeQPOU5inlQXzcHYBSp+gE5GBdCQoC1lv5BgABjs2p7gisq/a+tBr2Vf21M+0vHmvqw13V3uPnOM6pOgL7T3ltvyP2+nrBGFPTlbRT972/lhgA5gAzgDeAmzn5ahvAkWrPS7CS2WnHMMbYRSSNk+M/Wu15aQ3/DqEGIjIeeBLrSp0X1s98cy3xK6VUS6R5SvOU8iA6MqSaDREZivWHbjlWl5tSoK8xJtzxCHPczAlwGOumzmNia9hl9RtDz7S/NOAv1b4XbowJclxNO9UhrKRUXRxwsCHnW4tT9x3neK0mnwMDHHPXJ2BNe2jwMRxXN2NpZPyOqRoLsK7gtTPGhGNNbZA636iUUi2E5qka9615SrVoWgwptxORNiIyAZgPfGCM2eyYMvAG1lzeGMd2nUTkUsfbPgJuE5HeIhKEY051beqxvzeA20VkuFiCReQKEQmtYXcLgUQRmSIiPiJyI9AH+LpRPwjLPOBxEYl23Ej6BNa865rOqQz4BOvG1rXGmAP1PMZHwBUiMk5EfIEHsKZarGxk7H6AP5AJVDmuvl3SyH0qpZTbaZ46ieYp5VG0GFLu9JWIFGJd7XoM6wbP26p9/2GsucqrRaQA+BHoCWCMWYR1Y+jPjm1WOd5TXsfx6trfOqz52C8DuY7tfl/TTowx2VhXuB4AsoGHgAnGmKz6n3qtnsGaVrEJa9h+g+O12swB+nP61INaGWN2Yk1XmI11JfJKrPaxFWcZ87H9FmLdsPoR1s9wCvBlY/aplFJupnnqdJqnlEcRU2eLeaVaBkfLzS2AvzGmyt3xuIqIxGHdFNveGFPg7niUUkrVTPOU5inVPOnIkGqxRORqEfFzrCXwHPBVK0swXlhtW+drglFKqeZH85TmKdX8aTGkWrL/xpr3uwewYXWsaRVEJBgowFpj4Uk3h6OUUqpmmqc0T6lmTqfJKaWUUkoppVolHRlSSimllFJKtUpaDCmllFJKKaVaJS2GlFJKKaWUUq2SFkNKKaWUUkqpVkmLIaWUUkoppVSr9P8B8Xhvpb3cR3AAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1008x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Loading Auto dataset\n",
    "url = \"https://raw.githubusercontent.com/atystats/ISLR_Notes_codes/master/Datasets/Auto.csv\"\n",
    "Auto = pd.read_csv(url, error_bad_lines = False, na_values = '?')\n",
    "\n",
    "y = np.array(Auto.mpg)\n",
    "X = np.array(Auto.horsepower).reshape(-1,1)\n",
    "\n",
    "# Dividing the dataset in training and validation and getting validation MSE for different polynomial.\n",
    "MSE = []\n",
    "for i in range(7):\n",
    "    polynomial_features= PolynomialFeatures(degree=i+1)\n",
    "    xp = polynomial_features.fit_transform(X)\n",
    "    X_train, X_test, y_train, y_test = train_test_split(xp, y, test_size=0.5, random_state=20)\n",
    "    model = sm.OLS(y_train,X_train).fit()\n",
    "    MSE.append(mean_squared_error(y_test, model.predict(X_test)))\n",
    "    \n",
    "fig = plt.figure(figsize = (14,4))\n",
    "ax1 = fig.add_subplot(121)\n",
    "ax1.plot(list(range(1,8)), MSE, '-ok', color = \"r\")\n",
    "ax1.set_ylim(16,32)\n",
    "ax1.set_xlabel('Degree of Polynomial', fontsize = '12')\n",
    "ax1.set_ylabel('Mean Squared Error', fontsize = '12')\n",
    "\n",
    "# Dividing the dataset by setting different random state.\n",
    "\n",
    "ax2 = fig.add_subplot(122)\n",
    "\n",
    "for k in range(9):\n",
    "    MSE = []\n",
    "    for i in range(7):\n",
    "        polynomial_features= PolynomialFeatures(degree=i+1)\n",
    "        xp = polynomial_features.fit_transform(X)\n",
    "        X_train, X_test, y_train, y_test = train_test_split(xp, y, test_size=0.5, random_state=random.randint(1,100))\n",
    "        model = sm.OLS(y_train,X_train).fit()\n",
    "        MSE.append(mean_squared_error(y_test, model.predict(X_test)))\n",
    "        \n",
    "    ax2.plot(list(range(1,8)), MSE)\n",
    "    \n",
    "ax2.set_ylim(14,32)\n",
    "ax2.set_xlabel('Degree of Polynomial', fontsize = '12')\n",
    "ax2.set_ylabel('Mean Squared Error', fontsize = '12')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see the drawback of this approach in above plot. The validation error can be vary a lot depends on how we split the data. However all the plot shows that, MSE drops suddenly after adding polynomial of degree 2 and there is no use of adding polynomial terms of degree 3 or above. So it does help us in selecting the right model.\n",
    "\n",
    "__Drawback:-__\n",
    "1. The validation estimate of the test error can be highly variance , depending on precisely which observations are included in the training set and which are included in the validation set.\n",
    "2. Since only a subset of observations are used to fit the model. Since statistical method tnd to perform worse when trained on fewer observations. This suggest that validation error rate tends to overestimate the test error rate for the model fit on the entire data set.\n",
    "\n",
    "### Leave-One-Out-Cross-Validation (LOOCV) :-\n",
    "In LOOCV, a single observation is used for the validation set, and the remaining observation make up the training set.\n",
    "\n",
    "Since one observation ($i^{th}$) is not used in the fitting process, ${MSE}_i = (y_i - \\hat y_i)$ provides an approximately unbiased estimate for the test error, but it is highly variable since it is based on single observation.\n",
    "\n",
    "This process is repeated n times and produce n squared error ${MSE}_1$, ${MSE}_2$, .... , ${MSE}_n$. The LOOCV estimate for the test MSE is the average of these n test error estimates.\n",
    "$${CV}_{(n)} = \\frac{1}{n} \\sum_{i = 1}^{n} {MSE}_i$$\n",
    "\n",
    "__Advantage over validation approach__ :- \n",
    "1. LOOCV has a far less bias. It does not overestimate the test error rate as much as validation set approach.\n",
    "2. LOOCV performance is not random, it gives same result as we are using almost every observation. \n",
    "\n",
    "The following formula holds for least square methods or polynomial regression:-\n",
    "$${CV}_{(n)} = \\frac{1}{n} \\sum_{i = 1}^{n} (\\frac{y_i - \\hat y_i}{1 - h_i})^2$$\n",
    "where $\\hat y_i$ is the $i^{th}$ fitted value from the original least square fit, and $h_i$ is leverage. $h_i$ reflects the amount that an observation influence its own fit.\n",
    "$$1/n < h_i < 1.$$\n",
    "Hence, the residuals for high leverage points are inflated by exactly the right amount for this equality to hold.\n",
    "\n",
    "The drawback of LOOCV is that for a large value of n, it might take a lot of time to run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0MAAAEZCAYAAABVSv5XAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3XmcXHWV///XuffW1nsn6WyQEEA2wZWM4qjo4PJzmVFnXEZFUFzQcUVRcMTZHJkZHUW/jjM/fygqKi4oOsPM4I4KKOA3QVRCkD0bIUln6bW6tnt+f9zqpNLpTld3urs6Xe8nj/u4t+56upr0qVOfz/1cc3dERERERESaTdDoAERERERERBpBxZCIiIiIiDQlFUMiIiIiItKUVAyJiIiIiEhTUjEkIiIiIiJNScWQiIiIiIg0JRVDIiIiIkcxM3uDmd1ymO0/N7M3z9K195/bzM41sx/NRhxmttrMBs0snG6sIuNRMSQyBWbmZvaYCbYda2bXmNluMxsys1+b2Z+O2cfM7ANmdp+Z5c1ss5n9i5llxuz3FDO7wcz2mdme6rkuMLNjzKxsZieOc/3vmdknZvYnFhGRwzGzd5rZOjMrmNmXx9n+HDO7x8yGzexnZnbcYc71BjOrVD/0j06fncXYX2NmD5uZjVkfmdnOsTlsMu5+jbs/f4Zie9jMnltz7s3u3ubulZk4v8goFUMiM8DMFgG3AEXgdGAJ8Cng62b2ippdPwNcCJwPtAMvBM4Brq0519OAG4FfAI8BFgN/BbzQ3bcBPwXOG+f6LwKunoUfT0REJvYI8FHgi2M3mNkS4LvA3wCLgHXAtyY5363VD/2j0ztnOuAa3wO6gGeNWf8CwIEfzOK1ReYFFUMiM+O9wCDwJnd/1N3z7v4N4HLgk9UWoZOAtwPnuvut7l529w3Ay4EXmNk51XP9K3C1u3/M3Xs9sd7dX1XdfjVjiiHg1cAGd//9bP+gIiJygLt/193/E9g9zua/IPnb/G13HwH+HniCmZ061euYWaeZfcXMdpnZJjP7sJmN+znOzJ5XbY3qq7Ys2Xj7VWO6luQLulrnA9e4e9nMus3sf6rX3VtdPnaC6x7UXe9wcZjZiWZ2Y7U3RW+1Z0VXddtXgdXAf1dbxy4xszXV3hlRdZ+VZnZ9tffE/Wb2lppz/72ZXVt9vwbMbIOZrZ38XZZmpGJIZGY8D7jO3eMx668l+YN+MvAcYKu7/7p2B3ffAtwGPM/MWoCnAd85zLW+Bywxs2fUrDsP+MqR/QgiIjLDTgd+O/rC3YeAB6rrp+rfgE7gBJKWnPOBC8buVG2Nug74MEkvhQeApx/mvFcDrzCzXPX4TuDPOJBTAuBLwHEk+SwPTNp1r444DPhnYCVwGrCKpFjE3c8DNgN/Vm0d+/g4l/gGsLV6/CuAfzKz59RsfwnwTZKWr+vriVmak4ohkZmxBNg+zvrtNdsn2md0vyVAN8m/y4n2w93zwLepfpNXbXE6E/j6dAIXEZFZ0wb0jVnXR9JNeiJnVe8XHZ3Oqg4a8JfAX7v7gLs/DHySQ3sJQNJl+m53/467l4BPA49OdDF3/yWwA/jz6qpXAfe6+53V7bvd/Tp3H3b3AZIeD2O71Y3nsHG4+/3u/mN3L7j7LuCKOs+Lma0CngFc6u4j1Vi/wMHvxy3ufkP1HqOvAk+o59zSfFQMicyMXmDFOOtX1GyfaJ/R/XqBvUB8mP1GXQ28ysyyJH/8f+DuO6catIiIzKpBoGPMug5gwMyeWTNIwoaa7be5e1fNdBvJl2VpYFPNfpuAY8a55kpgy+gLd/fa1xP4Cge6yp1Hzf2nZtZiZv9ftWteP3AT0GWTj+p22DjMbKmZfdPMtlXP+7Xqz1mPlcCeanE2auz7UVsADgPZ0S52IrVUDInMjJ8ALx+n//arSP7430syKMIqM3tK7Q7Vb7jOAn7q7sPArST3EU3I3W8m6Z/+UuB1qIuciMh8tIGaFgkzawVOJLmP6OaaQRIm6zbXC5RIuqqNWg1sG2ff7SRdzkavabWvJ/AV4DnVAXzO4uCeBhcDpwBPdfcO4OzRU09yzsni+GeSQRoeXz3v68ac0w9z7keARWZW28I20fshclgqhkSmLm1m2ZopJBk5rgO4ysyWV9e/BrgM+EB1EIR7gc8B14x2ezCz00n6VP/E3X9SPf8lwBssGYJ7MYCZPcHMvjkmjq8AHyPpD/3fs/5Ti4jIIarDUGeBEAirf/9HWyC+B5xhZi+v7vO3wO/c/Z6pXKPa1eta4HIza7dkeO73kbSmjPW/wOlm9hfVON4NLJ/k/JtIRkT9BvBjd69tVWknuU9oX3Xk0r+rM+zJ4mgnaTnbZ2bHAB8Yc/wOkvujxot3C/Ar4J+r7/fjgTcB19QZm8h+KoZEpm4DSWIYnS5w990k/ZezwN0krTbvA85z99phVN9J0q/5ayRJ4AfAz6lpCXL3X5EMt30O8KCZ7QGuBG4YE8dXSL4J+5a7F2b2RxQRkTp9mCQXfJCkdSNfXUf1XpiXk9xnsxd4Ksnon9PxLmAIeJCkcPk64wzn7e69wCuBfyHJRScBv6zj/FeTtDyN7WnwaSBH0jp1G3UOt11HHP8APJnkHqr/JRmCvNY/Ax+u3jf1/nEu8RpgDUkr0feAv3P3H9cTm0gtS7pwioiIiIiINBe1DImIiIiISFOak2Ko2p/z12b22+qDr/6huv54M7vdzO4zs2+ZWXou4hEREamlPCUi0pzmqmWoAJzj7k8Angi8wMzOIrn5+1PufhJJX9o3zVE8IiIitZSnRESa0JwUQ9WRtAarL1PVyUluEP9Odf3VwMvmIh4REZFaylMiIs1pzh4+VR1+eD3wGODfgQeAfe5eru6ylfEfHoaZXQhcCNDa2nrmqaeeOvsBi4jIhNavX9/r7j2NjmMmKU+JiCwc9eapOSuGqmPkP9HMukiGQDxtvN0mOPZKkqGFWbt2ra9bt27W4hQRkcmZ2aZGxzDTlKdERBaOevPUnI8m5+77SJ6rchbQVfNgsmNJxooXERFpGOUpEZHmMVejyfVUv2nDzHLAc4GNwM+AV1R3ez3wX3MRj4iISC3lKRGR5jRX3eRWAFdX+2MHwLXu/j9mdjfwTTP7KPAb4Ko5ikdERKSW8pSISBOak2LI3X8HPGmc9Q8CT5mLGERERCaiPCUi0pzm/J4hERERERGR+UDFkIiIiIiINCUVQyIiIiIi0pRUDImIiIiISFNSMSQiIiIiIk1JxZCIiIiIiDQlFUMiIiIiItKUVAyJiIiIiEhTUjEkIiIiIiJNScWQiIiIiIg0JRVDIiIiIiLSlFQMiYiIiIhIU1IxJCIiIiIiTUnFkIiIiIiINCUVQyIiIiIi0pRUDImIiIiISFNSMSQiIiIiIk1JxZCIiIiIiDQlFUMiIiIiItKUVAyJiIiIiEhTUjEkIiIiIiJNScWQiIiIiIg0pTkphsxslZn9zMw2mtkGM3tPdf0Tzew2M7vTzNaZ2VPmIh4REZFaylMiIs0pmqPrlIGL3f0OM2sH1pvZj4GPA//g7t83sxdVXz97jmISEREZpTwlItKE5qQYcvftwPbq8oCZbQSOARzoqO7WCTwyF/GIiIjUUp4SEWlOc9UytJ+ZrQGeBNwOXAT80Mw+QdJl748nOOZC4EKA1atXz0mcIiLSnJSnRESax5wOoGBmbcB1wEXu3g/8FfBed18FvBe4arzj3P1Kd1/r7mt7enrmLmAREWkqylMiIs1lzoohM0uRJJhr3P271dWvB0aXvw3oxlQREWkI5SkRkeYzV6PJGcm3aRvd/YqaTY8Az6ounwPcNxfxiIiI1FKeEhFpTnN1z9DTgfOA35vZndV1HwLeAvwfM4uAEar9rUVEROaY8pSISBOaq9HkbgFsgs1nzkUMIiIiE1GeEhFpTnM6gIKIiIiIiMh8oWJIRERERESakoohERERERFpSiqGRERERESkKakYEhERERGRpqRiSEREREREmpKKIRERERERaUoqhkREREREpCmpGBIRERERkaakYkhERERERJqSiiEREREREWlKKoZERERERKQpqRgSEREREZGmNGkxZGahmX3EzDJzEZCIiMhUKE+JiMh0TVoMuXsFeAdQmv1wREREpkZ5SkREpqvebnJXA2+bzUBERESOgPKUiIhMWVTnfk8B3mVmlwBbAB/d4O5nz0ZgIiIiU6A8JSIiU1ZvMfT56iQiIjIfKU+JiMiU1VUMufvVsx2IiIjIdClPiYjIdNQ9tLaZXWBmN5rZH6rzC2YzMBERkalQnhIRkamqq2XIzC4Dzgc+CWwCjgMuMbOV7n75LMYnIiIyKeUpERGZjnrvGXoz8Gx33zS6wsx+CNwEKMmIiEijKU+JiMiU1dtNrhXYNWbdbiBXz8FmtsrMfmZmG81sg5m9p2bbu6pdGjaY2cfrjEdERKSW8pSIiExZvS1DPwCuMbMPAptJuh9cDvywzuPLwMXufoeZtQPrzezHwDLgpcDj3b1gZkunFr6IiAigPCUiItNQb8vQO4EB4LfAIHAnMAS8q56D3X27u99RXR4ANgLHAH8F/Iu7F6rbdk4pehERkYTylIiITNmkxZCZBcBa4C1AC7ACaHH3891931QvaGZrgCcBtwMnA880s9vN7Bdm9kcTHHOhma0zs3W7do3tBSEiIs1MeUpERKZr0mLI3WPgv9y94O6xu++srpsyM2sDrgMucvd+km563cBZwAeAa83MxonhSndf6+5re3p6pnNpERFZoJSnRERkuurtJneTmZ11JBcysxRJgrnG3b9bXb0V+K4nfg3EwJIjuY6IiDQl5SkREZmyegdQ2AR838z+C9gC+OgGd//byQ6ufot2FbDR3a+o2fSfwDnAz83sZCAN9NYZk4iIyCjlKRERmbJ6i6EcSUIAOLZmvY+z73ieDpwH/N7M7qyu+xDwReCLZnYXUARe7+71nlNERGSU8pSIiEzZpMVQ9cbUrwK/HB1NZ6rc/RbgkD7WVa+bzjlFRERAeUpERKZvSgMozEE8IiIiU6I8JSIi0zVnAyiIiIjMIuUpERGZsjkZQEFERGSWKU+JiMiUHekACiIiIvOB8pSIiExZXcWQu18w24GIiIhMl/KUiIhMx2HvGTKzV455fcqY1xfNRlAiIiL1UJ4SEZEjMdkACleNeX3rmNcfmcFYREREpkp5SkREpm2yYmjsMxcmey0iIjKXlKdERGTaJiuGxj5le7LXIiIic0l5SkREpm3SARTMzEi+WbPxXouIiDSS8pSIiEzXZMVQG1CueW01rw194yYiIo2lPCUiItM2WTF0/JxEISIiMj3KUyIiMm2HLYbcfdNcBSIiIjJVylMiInIkJhtAQUREREREZEFSMSQiIiIiIk1JxZCIiIiIiDQlFUMiIiIiItKUJhxAwcxupo4hSd397BmNSEREpA7KUyIicqQON5rcF2qWTwTeCFwNbAJWA68Hvjh7oYmIiBzWgstT7s7g4CDt7e2NDkVEpClMWAy5+9Wjy2Z2G/D/uPuGmnVfJ0kyfzerEc6Qa66Byy6DzZth9Wq4/HI499xGRyUiItO10PIUwA9/+EM23v1r3vXuDxFFkz0KUEREjlS99wydBjwwZt1DwKkzG87suOYauPBC2LQJ3JP5hRcm60VEZEE4qvPUKB/4Iqec9m1+e+e6RociItIU6i2GfgF82cxOMrOcmZ0MXAXcPHuhzZzLLoPh4YPXDQ8n60VEZEE4qvPUqNX5Z5HJDPO7O79AHMeNDkdEZMGrtxh6Q3W+ARgCfg8YcEE9B5vZKjP7mZltNLMNZvaeMdvfb2ZuZkvqjGdKNm+e2noRETnqvKE6Pyrz1KgdfwhgcBmLl/6Ge++9dzYvJSIi1FkMufsed381kAVWADl3f42799Z5nTJwsbufBpwFvMPMHgtJAgKeB8xaabJ69dTWi4jI0eVoz1OjerfcQ+q+c2ht28ftv7pyti8nItL06n7OkJmdBlwG/I27x2Z2ipk9vp5j3X27u99RXR4ANgLHVDd/CriEOoZHna7LL4eWloPX5XLJehERWRiO5jw1aterlrLzrpPwkQ7aum5j27Zts31JEZGmVlcxZGavBG4iSQznV1e3A1dM9YJmtgZ4EnC7mb0E2Obuv53qeabi3HPhyivhuOPALFl39tkaTU5EZKE42vPUqNc97530Fn5C5oGz6erawU03fmkuLisi0rTqbRn6CPA8d38bUKmu+y3whKlczMzagOuAi0i6JFwG/G0dx11oZuvMbN2uXbumcsn9zj0XHn4Y4hjOOw9uvhl2757WqUREZP456vMUQGemk+GXrmLnXWfi5QyWupE9e/ZM+3wiInJ49RZDS0mSChzoJuBMocuAmaVIEsw17v5dkgfkHQ/81sweBo4F7jCz5WOPdfcr3X2tu6/t6emp95IT+sAHktHk/uM/jvhUIiIyPyyYPPX6l1zM7v6bSW16Gkt6NvOLn157ROcTEZGJ1VsMrQfOG7Pu1cCv6znYzIxkiNON7n4FgLv/3t2Xuvsad18DbAWe7O6P1hnTtD3ucfCiF8FnPnPokNsiInJUWjB5anFuMbuf38beu54ObgyOXM/Q0NBsXlJEpGnVWwy9G/iomf0CaDWzHwL/CLy3zuOfTpKkzjGzO6vTi6Ye7sy59FLo7YUvf7mRUYiIyAxZUHnqglddyu4ddxA98mSWLb+fX958Q6NCERFZ0KJ6dnL3e8zsVOBPgf8BtgD/4+6DdR5/C8nzHg63z5p6zjVTnvlMOOss+MQn4MILIarrnRARkflooeWp5a3L2fbMMkvvejYtx65j69ZvUiy+lHQ6PVchiIg0hUlbhswsNLMHgIq7X+vu/+ru36w3wcxXZnDJJfDQQ/Cd7zQ6GhERma6FmqcuOO8yejc/hO08hWUrNrLu17c0OiQRkQVn0mLI3SskI/PkZj+cufXSl8Ipp8DHPw4+60+PEBGR2bBQ89SqjlVsOnMn+bufTSpd4A/3fIk4jhsdlojIglLvPUOfBr5lZs8ysxPN7ITRaTaDm21BkIws95vfwE9+0uhoRETkCCzIPHXumz7M7vsGoO9Yelb8ng13/a7RIYmILCj1FkOfBZ4H/Ay4D7i/Ot03S3HNmde9DlasSFqHRETkqLUg89RJix7DA6c9SGnjs8jlBlh325W4ujKIiMyYuoohdw8mmMLZDnC2ZTJw0UVJy9D69Y2ORkREpmMh56lXvPVD7Pp9Bh/upmvpOh5++OFGhyQismDU2zK0oL31rdDRodYhERGZfx637HQePP4u/N6z6ejYzS9/9vlGhyQismDUNaC0mUXA24FnAUuoGX7U3c+endDmTmcnvO1tyTDbDzwAJ57Y6IhERGQqFnqeesHbL+b+v/45Sx/bQrr9Fnbt2kVPT0+jwxIROerV2zL0KeCtwE3AmcB1wFLgxlmKa8695z3Js4Y++clGRyIiItOwoPPUU49dywPL1sEDZ7F48TZ+9sOrGx2SiMiCUG8x9BfAC939/wDl6vxlwJ/MWmRzbOVKOO88+NKXYOfORkcjIiJTtODz1DPf9nZ233EqXokoBz9iYGCg0SGJiBz16i2GWkie5g2QN7MWd78HeNLshNUYH/gAFArwb//W6EhERGSKFnyeevaJz+TBtvWw+cn0LH2IX/xITwwXETlS9RZDG4E/qi6vA/7ezD4MbJuVqBrklFPgZS+Df/93GDyqn1suItJ0FnyeMjMe+8Zz2fObMzFz9gz+J4VCodFhiYgc1eotht4DlKvL7wOeDPwZcOFsBNVIl1wCe/fC5zVYj4jI0aQp8tRLH/8iHox/h29/LMtW/IFf3fSjRockInJUq/c5Q//X3e+oLt/n7s9196e6+82zG97cO+ssOPtsuOIKKJUaHY2IiNSjWfKUmbHq/BfQ/9unEUUlNm2+hkql0uiwRESOWnUVQ2Z2zkTTbAfYCJdeClu3wje+0ehIRESkHs2Up17z1FfzUP8DxLvXsHTlBn6z7vZGhyQictSq6zlDwFVjXvcAaWArcMKMRjQPvPCFcMYZyUNYzzsPzCY/RkREGqpp8lRgAZ0vfzJDtyyn/dnf4Pd3fpEzn/I0TMlKRGTK6u0md3ztBHQClwOfndXoGsQsuXdowwa44YZGRyMiIpNptjx1wXMv5MHtu4kHeli88jfc+4c/NDokEZGjUr0DKBzE3SskSeaSmQ1n/nj1q2HVKvjYxxodiYiITNVCz1NREBG96Djydz+d1tZ93PaLzzU6JBGRo9K0iqGq5wHxTAUy36RS8L73wc03w623NjoaERGZhgWdp972Z+/joQdKxCPttPX8mm3bFswo4iIic6beARS2mNnmmqkX+DbwwdkNr7He/Gbo7k7uHRIRkfmrGfNUOkpTfFYbxT88ha6uHfziBj0TQkRkquodQOF1Y14PAfe6e/8MxzOvtLXBO94Bl18O99wDp57a6IhERGQCTZmn3vGXf8OX3/G3nHR6mqD9Jvbt20dXV1ejwxIROWrUO4DCL8ZM6xZ6ghn1rndBJgOf+ESjIxERkYk0a55qSefoP3OE0gNnsqRnMzde/5VGhyQiclSpq2XIzL4K+GT7ufv5RxzRPLN0KbzxjfCFL8BHPgIrVzY6IhERGauZ89Rbz/8o33n/P3H8Scaw/ZB8/i3kcrlGhyUiclSodwCFfcDLgJDkmQ0B8NLq+gdqpnGZ2Soz+5mZbTSzDWb2nur6fzWze8zsd2b2PTObl237F18M5TJ8+tONjkRERCbQtHmqu6WTXY/ZQ2nL6Sxdfh833fC9RockInLUqLcYOhl4sbuf6+4fcvfXAS8GTnH3fxidDnN8GbjY3U8DzgLeYWaPBX4MnOHujwfuBf56+j/K7DnhBHjlK+Fzn4O+vkZHIyIi42jqPHXBG/+RnXeeQhhWeLTvu5RKpUaHJCJyVKi3GDoLuG3MutuBp9VzsLtvd/c7qssDwEbgGHf/kbuXq7vdBhxbZzxz7pJLYGAgKYhERGTeaeo8taJrOVuX7aC04wSWrbyb239+Y6NDEhE5KtRbDP0G+CczywFU55cDd071gma2BngSSZKq9Ubg+1M931x58pPhuc9NusoVCo2ORkRExmj6PPWaC/6G3Xc+jlS6wP0PfY04XrCPWBIRmTH1FkNvAJ4O9JnZDqAPeAYwpRtRzawNuA64qHaUHzO7jKSLwjUTHHehma0zs3W7du2ayiVn1KWXwqOPwle/2rAQRERkfG+gyfPUCStOZFN2F+W9y1lyzO+5a90dDYlDRORoYu6TDr5zYGezVcBKYLu7b57ShcxSwP8AP3T3K2rWvx54G/Acdx+e7Dxr1671devWTeXSM8Yd1q6FwUG4+24Iw4aEISLScGa23t3XNjqOsZo9T931wHo2fPVLLDn7+2y95/m8/u3/b0PiEBFptHrzVL0tQwC4+xagA3i5mdXVD7sajAFXARvHJJgXAJcCL6knwTSaWXLv0L33wvXXNzoaEREZq9nz1BknnsmDhX4qQ510rFzPQ/dOOICeiIgwSTFkZt8wszfXvL6U5Fuz1wI/MbPz6rzO04HzgHPM7M7q9CLgs0A78OPqunk/PMHLX56MLvexjyUtRSIi0jjKU4f6k798HQMbnkxHx25++eMrGx2OiMi8NlnL0NOB6wHMLADeD7zW3f8IeEX19aTc/RZ3N3d/vLs/sTrd4O6PcfdVNevediQ/zFyIouS5Q7ffDjfd1OhoRESanvLUGGc98fk8sLtIXMyRXnoru3bsbHRIIiLz1mTFUJe7j/4VfRKQBf6z+voHwHGzFdh8dsEF0NMDH/94oyMREWl6ylPjOPPPns/QH57I4sXbuPG7VzU6HBGReWuyYqi3OsQowJ8At7p7pfq6FaiMd9BCl8vBu98NN9wAv/99o6MREWlqylPjeM7Zr+PBbSFxJaLS8TMGBwYbHZKIyLw0WTH0BeB/zewK4IPAl2q2nU3yULqm9Pa3Q2urWodERBpMeWoCpzzrTEYeeiw9yx7ixm9/rdHhiIjMS4cthtz9n4CPAyngPe7+jZrNPcAnZzG2eW3RInjLW+Ab34BNmxodjYhIc1KemtiLXvgOHr6/CzOnL/g+BT0xXETkEJMOre3uV7v7u9z9mnHWf2/2Qpv/3vveZLjtT32q0ZGIiDQv5anxWRCy/MzjGNn2GJau/AO3/KeeCSEiMtaUnjMkB1u9Gl7zGvj852H37kZHIyIicrC/+Iu/Zts9K4iiElv7vkul0pS3UImITEjF0BG65BIYHob/+I9GRyIiInKwMJWh7aQuCruOoWfVXaz70S8aHZKIyLyiYugInXEGvPjF8JnPJEWRiIjIfPLyV/4N2+9eQyYzzD0PXoPrieEiIvupGJoBl1wCvb3w5S83OhIREZGDZdsWES1NUepfRPfqO9l4652NDklEZN6oqxgys0Vm9k9mdoOZ3VQ7zXaAR4NnPhPOOgs+8QkolxsdjYhI81GeOryXv+rv2LXhZFpb97Hu/35p8gNERJpEVOd+XwcywLWAOoONYQaXXgp//ufwne/Aq1/d6IhERJqO8tRhtPWsptTilPOttK5ax9aND3Hsacc3OiwRkYartxj6Y6DH3fWQggm85CVwyinJQ1j/8i+TAklEROaM8tQkXvrnH+RH1/0jS590Bzd9/ypee9pHGx2SiEjD1XvP0O+AY2czkKNdEMAHPgC/+Q385CeNjkZEpOkoT01i8ZrHM1R24lKacOWv2LOtt9EhiYg0XL3F0I3AD8zsQ2b2xtppNoM72rzudbBiRdI6JCIic0p5qg4vfvHb2Xf/ySzu2cyN117V6HBERBqu3m5yzwS2As8bs96BL85oREexTAYuuii5f2j9ejjzzEZHJCLSNJSn6rDijHPov/4zdLtRWHIT+b53kutsbXRYIiINU1cx5O5/MtuBLBRvfStcfnnSOvStbzU6GhGR5qA8Vb9znv0a7tz4JXpW38dPr/4qf/rutzU6JBGRhpnyc4YsEYxOsxHU0ayzE972tmRUuQceaHQ0IiLNR3nq8I4/65Xs25EhDCvsbf0p5ZFio0MSEWmYep8zdIyZfc/MdgNloFQzyRgXXQRRBJ/8ZKMjERFpDspTUxAEPO3M5zOiOBO9AAAgAElEQVS0fRU9x2zg5q9e1+iIREQapt5vzD4HFIHnAIPAk4HrAbWtj2PFCjj/fPjSl2DnzkZHIyLSFJSnpuDU57yZPZvaSaULbC7/gLgcNzokEZGGqLcY+mPgje5+J+Du/lvgTcDFsxbZUe7974dCAf7t3xodiYhIU1Cemooow5NOfjIje3pYvPq3rP/2DxsdkYhIQ9RbDFVIuh0A7DOzHmAIOGZWoloATjkFXvYy+Pd/h8HBRkcjIrLgKU9N0eNf/F52P7SYXG6AjTuvx90bHZKIyJyrtxi6HXhRdfmHwLeA7wLrZiOoheLSS2HvXvj85xsdiYjIgqc8NUWW6+L0pSdSHGqn/bj1/OEHtzU6JBGROVdvMXQe8Ivq8kUkD7e7C3jtbAS1UDz1qXD22XDFFVDSLbwiIrNJeWoanvCSS9h930o6OnazbsN3Gx2OiMicq6sYcvd97r6nupx394+6+6Xuvr2e481slZn9zMw2mtkGM3tPdf0iM/uxmd1XnXdP/0eZny69FLZuhW98o9GRiIgsXMpT0xN1H8tJLYspF7Jkjrudrb/a0OiQRETmVL1Da2fM7HIze9DM+qrrnm9m76zzOmXgYnc/DTgLeIeZPRb4IPBTdz8J+Gn19YLywhfCGWckD2FVd2wRkdmhPDV9T3zJB9nzwCoWLd7GLT+/ttHhiIjMqXq7yX0KOAM4Fxj9SL8B+Kt6Dnb37e5+R3V5ANhIclPrS4Grq7tdDbyszniOGmZwySWwYQPccEOjoxERWbCUp6Yps/J0VlcyxJUQP+429t69udEhiYjMmXqLoT8HXuvutwIxgLtvYxqj9JjZGuBJJDe7LhvtwlCdL53gmAvNbJ2Zrdu1a9dUL9lwr341rFoFH/tYoyMREVmwlKeOwJNe9H72PbSaJcse5Mbrvt7ocERE5ky9xVARiGpXVIct3T2Vi5lZG3AdcJG799d7nLtf6e5r3X1tT0/PVC45L6RS8L73wc03w623NjoaEZEFSXnqCLQ85hksGzTMnPyq2xja1NvokERE5kS9xdC3gavN7HgAM1sBfBb4Zr0XMrMUSYK5xt1Hh6zZUT3X6Dl31nu+o82b3wzd3cm9QyIiMuOUp46EGU949tvp33oMPSvv4edfae5Rfzx24uES5d15ilsHGLlvL8O/28Xg7dsZ+MUW+n7wEHu/dx+7v76RvdfdR9+PNzH46+3k79lD8ZFBKkMlPbdJ5CgRTb4LAB8CPg78HmgB7gM+D/xDPQebmQFXARvd/YqaTdcDrwf+pTr/rzrjOeq0tcE73wkf/Sjccw+cemqjIxIRWVCUp45Q5xNfSteNn8JXldi5/FaKu15Puqej0WFNm5dj4nz5oMlHqsvDZeKRmvVj9ytUDn/y0AhyEUE2Ii6UiQdLB+5UGxUZYUeGsCNdnTKEnWPmHWksqvd7aRGZDTbVby6q3Q56fQoHmtkzgJtJklRcXf0hkv7Y1wKrgc3AK0eHRp3I2rVrfd26o/MZejt3wnHHwbnnwhe+0OhoRESmz8zWu/vaRscxHuWp6dtz8xe4edtVpDoHyd5xEedc9qZGh4S74yMVKn0FyvsKxEOlCYuYOJ8UOZ4v46X4sOe1VECQi7BclBQ21eLmkHU10+h6SwUk9XM1xkpMZaBEpb9Apa94yDzuL1DpL44bU9AaHSiaOjP7C6egpmAKWqKDricik6s3Tx22ZcjMVk+wadXoP0p3n3TYGXe/BZjoX/FzJjt+oVi6FN74xqQQ+shHYOXKRkckInJ0U56aWYuedh65z3yJcOkw2zpvozJwLmF7dlavGReTQqeyr7B/Xq4ul/uGGRnZTjHaQSm3k1LLTjwoE5QzWCVH6ClCyxIFWcJ0jqillTDdRirbQdTSRdS6hKitZfyCZwZbZCwMiLoyRF2ZCfdxdzxfptJfTKa+QnX5QOFU3DaYtDKNFQXV1qRDW5b2F1DtamUSmY7Jusk9zIGG3/GShAPhTAa00F18MXzuc/DpT+v+ocO55hq47DLYvBlWr4bLL09a1ERExngY5amZE2V4wml/yrq+b9N53B2s+9x1PPUD0//j6+U4+cA/WtzUFDyVvuq6wgDF3C5KLbuqBc8uSq2PUlq+g+JxeyE40JpicfKyEnry7IrJFMF6nagCoQeEcUhESEiK0NIHCqkwRxi2EkZtRFEHYbqDMNVJlO4mzHQTZpcQZpcQZXsI0p1YMPWiw8ywlhRBS4rU8tbDv2cDYwqmmnlxywCVDQUoH9rwGbSliBbniJYkU6qnurw4h6VUKImMZ7Ji6HdAluTZCl8DHpn1iBa4E06AV74yKYguuww6OxsdUWK+FB/lMnz5y/Dud0M+n6zbtAkuvDBZVkEkImMoT82wZc/+K+zK/6X1cdu5P1zH2pFXEWZTh+znsRMPFCmPFje1LTvVQiceLOEeU8n0Uay27JQ6dlNu20lpyQ6K6UcpR0MHnTcoQZiPYKidoP8kiuXlFMrHkC8uZ7jYhXtEFISkAieyEqmwSBgUCSkQWQGzPNgI2DBYHoIRyjaCBQU8GIGgAFEBgiIW9UO0GwtKWBwnYxIWgeGJ3x+PDS9HxOWIuBRR6ltCufdUrPBUMq3Hk2rvJEpHhOmQVDogTIVE6YBodF67fMi26rwjTdQ9cYucuxMPl/e3LMU1XfLKu/OM3LuXeP2OAwcYhF2ZaoHUkhRI1UIp7Mxgwcx1wXN39u7dy9atW3n00UcJw5BMJjPpFEX13sY+/bh2DRSIwoBsKiAbhQQz+HPL0euw/+e5+xPN7AySm0ZvAe4BvgJ8193zcxDfgnTJJfCtb8GaNdDX1/iWj2uuSYqN4eof//GKj0olKU6Gh8efDrftcNPY40rj9A6AZNs73pEMU37GGXDSScmyiDQ35alZkOviiSuewu/yPyK75nY2fvZ/Oe6pZyb3xPTVtPL0F6jEFUpUKIR5RnI7KLbtotK5h8rSXiqre/FUL6R2Y0F5/+ndjdJIjpF8O/l9K8iPtJMfaWdkpI2RkTbK5Ym6mu2CeGfS/ne4ViF3LA6wSg6L01BpxSplrFLCKpXqchkqFSwu718XeIkgKhOlYoJUTJCOCaKYcMw8SMeE1X3CdEzr8s20rn4Ij7/P0KOt7N2yhMFHjqc4uAqCJWCLsLAbs4m70I0nCI0oFRxUVCXzgFQ6JEzVFE+jxVQmouNxPXQtb6G7K0M4VKLcm6fcm6e0K5kPrduBF2sGiIgCUkuy1QLpQKGUWpIjaJk80RaLRR555BG2bt3Kli1b2Lp1K0NDSYEbhiFxHNc1sl69RdPolM1mJyyqRrvI7uwf4Zb7e7nlvl5uub+XnQOFg66ZDgMyqYBcKiSbCpMiKRWSjUIy1eVc7fpUSDYKyFSXD94WVI9Llg+cM1TxNc/VPYCCmQXA84A3AC8Ezhl9WvdcOppvTB11zTVw/vkQ19xH2dICV145tYLIPSkexhYVo8vjrRtv+7e/faAQqhWG0NGRbCsUDt0+GbPk56pnyuUOLP/t305+7lQKTj45KYxOPz2ZzjgDTjwxiVua13xp5Vzo5uMACspTM6hvG9d/7dW0nvYI969/OVH/sZRTA5DbQ5DdS6qlj3R2gGxugGxukHR6+KD6pFKJyOerBU6+jZHhVkaGWigMZykM5aDiWFzBKhWIK8lyHFcLlAMTcQWrxIQeE+CEgSXdzczwIMSDEIIAtxAPAjwIiavrR5fjMCIOIypRRCVKHbaQsjgmVSwSlUqkSiXSxSKpUpFUsUi6WCRTLJIuJMthuUBQKZAdLlBa7RQfF+OnlYiWJYVfoS9F36Z2+je3MbS9hXQ5phWjLZ2htbWL1kUraV1+Etme47CWDjyTI05lqRBSLsaUSzHlYoVyKaZSPLB80Lx2fSmmMmaAhpaONF3LWuhe3kL38la6lrfQtTRHayqksudAgVQene8ZgfjA58KgJTqoQIoWZxnKFtk+2MvW7dvYsmULO3bsIK5+oFm0aBGrVq3i2GOPZdWqVSxduhQzo1QqUSgUjmgaGRnZf53DMQvwIGLEA4bLRokQDyK621tZ3NFKmMoQhynKlqJkKcoWUfCIEQ/JxxEjFWekVCFfiimUKoyUKoyUYkbKB5br5tDq0BkHdMbGIg9YRLLcXjHSMZQiKKWMciqgkgnwTACZAMsGhNmAVDYglQvJpCATGWmDbJjMMyFkAkgFRiaAdAApg3Rg1WXfvy5lEJpjcYzHDnEFj+Pk9x1XiEsxQa6V7OmnELa21P8zzmP15qmpFEOnkHzz9lrgIeCN7v7QEUU5DQshyaxZk7S+jNXRkRRJ9RYxw8MHF1T1Gi1SRguQzYe5tfhd7zq4UJlKYZPJ1Nele6yJ3p9Vq+D66+Guu2DDhgPzh2r+L8xkkmHLR4uk0fmaNTCNLt5Sh/lUfIxt5YTpfdEw0zHNl/dnJs3TYkh5agZtufp87l5+O+4hZhWiVPmg7cV8mpGhFoqDWQoDOQoDWYr9GYr9GeKhgKBSIayUsLhEFFcIK0XCuEhULhLGFcLYieKYMHZSFSesxKQqyXIUV5fLTlRxUmNyXQwUU1AKkw+Txaj6oTKCUmjJh8sooBIFyYfMVECcCimnIuJ0mjiTpZLKEqczxFGWOEzhYQYPUrhFyUSIxyEeG3HF8MOMtm0Z8KhAKR4gYjuLWjezfNEAi3pKBCFUCsbw5ixDD7Wyb0snI+X0gYPdaSmWaS0UaS2UaC3HtKSy5LKtRG0dVNq6KLV1UGztoNDazkhLG/lcO8O5dgazbQzl2sh7QLESUyrFHJdJsyadprtiFPcW2fvoMHsfHaIwfOD3F6aCpEha1pK0Ii1voXtZK51Lsthoa9KuPPmdA2zfvp1tex9lR2kPO4N+8lYEICJkWWYRK7uWcezKY1h1wnF0HLtoxrvd1SqXywcVSPn8CPdt38tdW3Zz//a9PLJngMDLZC1meVtIT2tIZxpSVPYXVPl8/rBFVTqdJpfLHTS1tLQc9DpKZwhTGSzKUC6HDO0pMPhIP4M7Bhje1cdI3wCFoTzlYoHYRyDO4z6Mex6rDEA8BPEwToWADIG1QNCOh5141I0FrVjQhgVtYC2YhYTlPOliP+niQDIvVef71w2QLiXLYVzTzcZCLNuJZTsJct3V5S6C6nz/cvrAfWwelyCICTIhYVuGsKuNoCWFZcJkAJJsiGWTwUgsE2BZg0yMZR1PO6QrYDFxXMK9RBwXq/MSsZfw/esPvE7mxeq8TOwlspnlrFz5qmn//zIjxZCZLQJeQ5Jc2oGvAl+rZ2Se2bIQkkwQJK064+nuPrj4GF0eb91k2yc6Jp0+uEiZqPg47jh4+OHZeAcOb6ofaAcHYePGpDCqLZK2bDn4+Mc+9uBWpNNPTwosjVY6fUdSfBSLMDSUHDs0VP90uP23bBn/C4IggOOPn1pRX++UzU5caM/H4mymzJdiSHlq9viOu/nNt1/JlsXtFPJZorzTVhwmU8yTKQ2TYpgd1s1mX8pmX8omX85mX8qjvojKQWNWjPdHtmad17GfO4FDFFeoWEiZFJACAtxD8Ag8AA+p/3nyEBikwoB0FJAOg/3LqdBIhQGZKFmXCgPSoZGxChkrk6ZMijKRl/HCID4ygI0MEBYHCbymaAzK5BbvoHvxZlZ0baUlPULssGU4w5bdEbu3pgh2ZGkfTtFSSJEqpbCa+M1jcqUyrSMlOvJF2kYLpkKJVOXAH7t8lGEg28ZgppXt6Q52tCxiZ0s35Z5ldB+/mmNOPZ6TV63gmDAi7iux99Eh9u4YZu+jwwz05nEHx4nDEcLuPLQOMWJ9DBb27u/itqi7m5WLV7CidQlL6aIrnyXeXaC8Kz9+t7ueFqLF2WTkvnS4fwoyIZYOkuXR9ZkAS4V1FVFb9gzv7/r2ywd62TecfPA/dXk7zzxpCc84qYenrFlELj1+NxF3p1gsMjw8TD6f3z8NDwwwtG8fw/39DA8Mks8Pkx8ZYXikQL5Soeg+8XiTkLRqVrtgJtNoN8xkOSyXSVcqZDwmZ0ZLFJGOIgrlMvlykXypSL5YxA95YBVEYZYoaiEI2zBrJbY23NtIBYvIRW3kwjayQYZcGJA1Ixs4udDIGGTs4H8PcVigmN5LPruH4exeRjJ7KWb2UUrvw22AKC6QthKRlbHQ8RA8dAgdD2M8qOBWhurcgwrY7DxgOGp9LM966n9P+/iZKoZGSL5d+ypw23j7uPuN0w1yOhZCkjnai4+5iulIv03v64O77z60SNq+/cA+7e2HFkinnw4rVhxcJC3Ub/enwh36+2HfPti7N5le9Sro7T1039ZWeP7zD1/UlMuHHnc46XRy3sNNX/nKxMefe25997FN56HxE7Werl8PIyOH7r9kCXz969DTkyz39CStmkeTeVQMKU/NouGvvoaWB25gOLecgY6TGew8mcGOkxjsPJmRrsdAlCEIjMCMwCCsdmELg+R17XKyjxEEB/YLjOp2q56nZr/qtrH7xe4Uy3HSElJJlkuV+MC6cnV9pUKx7Pu3lSrJVKhuH3194Fyj+znFMcck+zilmn1HlwMzMqmkmMqEAS1BmXaGafFhspU8mfIgUWkQKw3T3rabRYu3sXjRVtrak8dVjZS62Fc8md7KGvZVjFL/Zor7dlHqH8YHIcqnaB1K0ZaPCPxAYqoEFSwVkwkD2iykowLdBadz9xDZnXsJC8WDfpdDUZYdLd30dy6BZSvIHXcM6VVLsY4Wdg4Ns713JyOF5IOAEZAqtRMWOkiVOkgV28mkUnR3x3R1lujuGKG7bZiutn46M/3YSInSQIryYJryUI5yvo3ySAflYidTGczRggoWVbAwuT/LIsdDZzgu0V8q0Vso0lcsMYJjaWNRZ5ZlS3Icu7SdtvYMlg0hFTLkI+zr30V//04G9+xkZO9uSrv3Eu/rJxyuYPky5EvEhTLlkjNCipEoQyGVphiFlEOnbBWcEqPVugMEIR6GeBhBmCFIZQlSGYJ0hjCTxjIpSKWoBBFlg2LsFEulw94zFYYh6XSadDpNKpUiCkOC2AkqjpWcoBQTliAsG1E5IB2nyHqGXJAjTYqIgMhDwqgA6QEqqT2UM3sppfdRzvRTTvdTyQxAZhDLDRCkDk1KHof4SAtWSmEeYBYQe0ixbBTzMeTLpPMF2vN5olKMVQzCLFHXUtLdSwk7ewg7luCpFipFp1JwyqWYuAAUDS8GBMWQoBgSFlOky2lS5QwWR8nkERaH++fEEeYhD3Xu4Oy/bnzL0MMc+kzlWu7uJ0w9vOlbCElmoRYfR4s9ew4tkO666+AP9d3dBwqjfD4Z8KL2vqlG/r6O5HdVqRxczNQuj/e6dt2+fVPrlnnGGUmB0tIyeREz2dTSAvUMNHSkXzS4J7/n6QwGMt7005/W/361tx8ojEan2tdjt7W3T69Vc6b+rc+jYuhhlKdmT6UEpTxkOxodyVGvWCyyZ88eent76e3tZffu+ykW15NKbaSj8xHCsEK5HNHffyzl8mPJZv6IxYvXsKQ9Q2vpEQoDd7Ft8108+uh29uwdZngkTWkkg+UzpIsH/4EsRBXymQqVsExAmUy5Qkc5RcbaId3JSMsi+rq68GqzdtvAAIv37Gbx4E6Wjexkpe8i11qm3NrOUG45A6kV9NtK9lWOYW/5GAbjnv3XMip0RDvpTu2gK9NLd24PXS19dLf1k804XjG8HOBlwyshcTnA4wivRMSVkLiSJo6TyeM0sWeIyeGexckSk6VCltizuGcwMoRTHIyiHBeTycs4yYAOyX+jyzGO4W64GY5TwYmDmFIQUwzLFMMShahIISxSCqutIQFghhlJq1ZgWBBgQUAQBFgYVu9vs+TWHJwYiGOn4jFx7FjRoORQcuJyTCWOKVOhTEyJEqSHsfQgQXqQVCZPOj1MOp0/ZArDQ/txViohxUKOYiFLqZChOJKhNJKmXMhQKWSpFHJUCq14qQ2jBeIsXmrByx0E3kYQtxLYaDFbJo4GqNgAlAdIj/TTMbCPnr37yBX6SRf6KVsfWxYN8ODKCg+sMB5YbuxpZ3+yaku10Z5upz3dTmfUweJwEYutm2466fR2OryNtriF1jhHSyVLrquNNc963JR+17Vm/J6h+WKhJJlmKj6OFjt3jl8k7ds38TGdnUmLRSZzYF67PNPzn/8cPvaxg1sbMpnkYb6nnz55YTMwcPj3IJ1OCsGurmQ+Ok30+txzD25pG6VWzsRExdnKlUmBvWvXgam3d/zX47UsQfK7mqhYGq+oWrwYvvnNmXt/5ksxNB8tlDwlcyOOY/b17WTb1p+yZ+/PKZXWY9aHOwwMLGHP7mPZvedY8sPddHV1s2TJEpYsXsySHCxhD0tGHibauYE9W+5l6+4+essZ9pRb2Ws9DIWLKKU6qWTb8Kh6n1JcIcgPQXGAcrkfCnuJynlaiiXah8ssHiizfF+ZRUOVg3qFlaOAkUU5ij0dlJcuo9RzIpX2NZQzKylVOskPhPTtHKFSPvCtWToXEQSWDAUeO16d4tiTVvg6P4K6V/B4EOIB3AfweJDAhwk9T0iRyItEViEMUkSWJhWkCC1FFLSQClqJwlaiIEdkGcwiojAgCo0ohCiMicIKgZUwK2OUkm6Do9NokeRGUv0cOplX56OTJ/OguhwQYIAHJTwsEQdFPCwShyMM5HbQn3uU4dwuCpk9VDJ9WGqQVDpPJioSjvOl10hsDJQDBsshQ+WIoXKKfClFvpRhpDoVClm8nCFTTpEpp0hXIlJxiigOCT0iIMQIMQsPf1N1pZxMcQWrOEFsBJUUYSVDVG4liHOE3op5G4HlwLKky0PJvUyFPjLkyXWm6DhmCZ0nHEPnY0+gY9USWjrTpLOzO5y6iiGRGeCejE430T+Td787ufelUDiy+UxqbZ1aQVO7nMtNrbVhvhUfozHNly8ajvT9cU+6FB6uWBr7uq9v/HOZJdN4rXvTKV5VDE1MeUqOhLszMLiB3t6fsWvXTxgcvKu6ZREjIyfTu+sYtm5tOehRFNlsliVLltDd2cHuvXt59NEDI7x1d3dzzMqVLO7soCOTJioXGdrTS3/vLvp7d9HXu4Oh3bupjHm2RWzw/7d390GS1PUdx9+fntnZh9t74vYOI3CgIShqEAwagmUCaBJSoOahjMaHQiqKWikfCAbjQ0lpcolSF4MxVUZUPErNWQSJomDiE4b4hPJgBERDRDiOOzj29p72bp+m55s/uu+c25uZXb276dntz6tqart7un/97V/tzne/3b/umeoTdaVUok5/fYqlU9Os3DvN6l0zrN5dp6/pyXP1BEaXJ4yNHMueVScwvfQE0v41xEwdTU2RTE1TnZ6hNp0yMN1gYLpBEg2IlLpmslcyzYzqTPWlzFRTZpIGqeqkcei46mptgKFlqxhcdgxDy/PXilUsWX4MS1aOsGTFKmoDgyjJrtokiahUE4ZX9lOtVfICbYpGY5K0MUkjnaTRmCJNJ5qWNb0/s5e0Pk5jZpxGfS9pfS+NdB9pui/fdpK0MZW10ZiiETM0mCGNOg3qNDT30IpK1KhqCdVkKZXqCirVY0hqI1T618DgE0j6n0DUVhL0kUZKIxqH/GxEg7TR+r00UtJGesjy+uQ007vGiV0zVCeFJho0JhrUJ1Pq0ykzMykz9ZSZyB75EK2Kp4js/qj6DElaJ6k3SFJQmqC0hqKfSqMfNQZJYpBEQ1QrfQwODjK8ZhXDq1eyZHmNoeU1hpb1s2R5jaWrBljZ4QuK5+JiyOwIOdr3eO1/RPp8iqbzzmtdmEnw6KNZUVOrHfr+0dRLxUcv6nb/TE/D9u2tC6X3vrf1Nu2KpE5cDLXnPGVH0tTUNka338Lo6NcZG/smjcYklWSI4aXPoZKczsTEyYyNpYyOjjI2NsbKlSsPPNr6+OOPZ3h4eM59RAQTe3azZ/to9hrdxu7tj7Fnxxb27HyUPTu3kU7vJKmmVPpSkr4g6WtALVAN+moptWpKf7VOn2aoJnWqlRT6sudaNCQaSf6SCCDyM29qcXkoK14qqFIhqST5dDb0bP/y7LuEYvaBHDx7SNtBmhcujUaby+7zkCQ1kmSAJBmgkgyQVPqpJIMklQGSpD9flr9faVrvwLoDJJVBqpUharU19PevoVZbRZIsjJtHp6am2DE6ytjo4+zcvp1du3ayZ/du9o6Ps29igsnpaSZn6sw0ovUZ1ojse7/qWfGk+gxJvYEakKQVkrSKGn2sWDbCa666/JeO08WQ2RHSS1c/eu3hG7awHMnfHxdD7TlP2dGSppPs2PGdA8XR1FQ2TnnZ0tMYGTmPVavOoVodpp5mVy7SdB/1dJw03ZfP71+WT9cPna/nVzsi5jdsIQLSmQqNGdGYSQ680pmERl3ZJaYmDSUkfTVqAwMMDg7S199PtX+Avlr/gelKpWn41CH/S2vW3OwVZs0f9M+4DhQvPy9kmguV/dP9VCqDTcVM/0GFjTT/JxaWWUQwNTXF+Pg4O3fv4d4HtvDAw1t47PExJnbvYLA+ziB1kkTZd4DNuuLUv2+ct1+5/pfe/3zz1NEdrGe2COwveHrh6se6da0Ls3Xruh+LLTz+/TFb2CqVAUZGzmVk5FzilPcwPv5jRke/xuj2W3jgZx/kgZ9dNWcbSVKjUlmSv4ay+2kqw/T3H5vPD1OpDFGpLjl4vjJEpTpMtTJ0YPtqdcmB4mDH3mn+58Ft3HP/Jh7Y8giPbt3K1O5dLFuxgl990nGccepJnH3ayaxc4YdxlIUkBgYGDgzjPPnJTzrwXkSwZdckdzy0gzsf2sFdD41x/9YdjExs5+R9W3hG43FWLx/sTpy+MmS2sHhYmh2OxfY0uV7kPGVFmJoeZcfYtwkaVPNCp1IdPlDw7C9+kqSvezHVU/qr83+0tpXbxHTK3Y/sygqkTTsYGe7n7//YT5M7hJOMmVnxXAy15zxlZla8+eYpD3o0MzMzM7NScjFkZmZmZmal5GLIzMzMzMxKycWQma8auqUAAAyBSURBVJmZmZmVkoshMzMzMzMrJRdDZmZmZmZWSi6GzMzMzMyslLpSDEm6RtI2Sfc0LTtd0ncl/UDS7ZKe041YzMzMWnGuMjMrn25dGdoAnD9r2ZXAeyLidODd+byZmVlRNuBcZWZWKl0phiLiVmBs9mJgWT69HNjSjVjMzMxaca4yMyufaoH7fgvwn5LWkxVlZ7dbUdIlwCUAa9eu7U50ZmZm88xVzlNmZgtTkQ9QeANwaUScAFwKfLzdihFxdUScGRFnrl69umsBmplZ6c0rVzlPmZktTEUWQxcBN+TT/wb4plQzM+s1zlVmZotYkcXQFuB38unzgPsLjMXMzKwV5yozs0WsK/cMSdoInAOMSNoMXAG8FvigpCowST7W2szMrAjOVWZm5dOVYigi/qzNW7/Rjf2bmZnNxbnKzKx8ihwmZ2ZmZmZmVhgXQ2ZmZmZmVkouhszMzMzMrJRcDJmZmZmZWSm5GDIzMzMzs1JyMWRmZmZmZqXkYsjMzMzMzErJxZCZmZmZmZWSiyEzMzMzMyslF0NmZmZmZlZKLobMzMzMzKyUXAyZmZmZmVkpuRgyMzMzM7NScjFkZmZmZmal5GLIzMzMzMxKycWQmZmZmZmVkoshMzMzMzMrJRdDZmZmZmZWSi6GzMzMzMyslFwMmZmZmZlZKbkYMjMzMzOzUnIxZGZmZmZmpdSVYkjSNZK2Sbpn1vI3SvqJpHslXdmNWMzMzGZznjIzK6duXRnaAJzfvEDSucCLgdMi4unA+i7FYmZmNtsGnKfMzEqnK8VQRNwKjM1a/AbgfRExla+zrRuxmJmZzeY8ZWZWTtUC930K8DxJ64BJ4K0R8f1WK0q6BLgknx2X9JMuxdgNI8Bo0UH0MPdPe+6bztw/nR1u/5x4pALpYc5TGf8tdeb+6cz90577prOu5Kkii6EqsBI4C3g2cJ2kJ0dEzF4xIq4Gru5yfF0h6faIOLPoOHqV+6c9901n7p/O3D/z4jyFf1fm4v7pzP3Tnvums271T5FPk9sM3BCZ7wENsgrQzMysFzhPmZktckUWQ58DzgOQdApQw5cKzcysdzhPmZktcl0ZJidpI3AOMCJpM3AFcA1wTf4Y02ngolZDD0pgUQ6rOILcP+25bzpz/3Tm/mniPNWRf1c6c/905v5pz33TWVf6R+X8XDczMzMzs7IrcpicmZmZmZlZYVwMmZmZmZlZKbkYKoikEyTdIuk+SfdKenPRMfUaSRVJd0n6YtGx9BpJKyRdL+nH+e/QbxUdUy+RdGn+d3WPpI2SBoqOqUiSrpG0Lb/3Zf+yYyR9RdL9+c+VRcZovcd5am7OU+05T3XmPHWwIvOUi6Hi1IHLIuJUsu+w+AtJTys4pl7zZuC+ooPoUR8E/iMingo8E/fTAZKOA94EnBkRzwAqwMuKjapwG4DzZy37a+BrEfFrwNfyebNmzlNzc55qz3mqDeepljZQUJ5yMVSQiNgaEXfm03vIPiSOKzaq3iHpeOAC4GNFx9JrJC0Dfhv4OEBETEfEzmKj6jlVYFBSFRgCthQcT6Ei4lZgbNbiFwPX5tPXAn/Y1aCs5zlPdeY81Z7z1Lw4TzUpMk+5GOoBkk4CzgBuKzaSnnIVcDnZlxzawZ4MPA58Ih+e8TFJS4oOqldExCPAemATsBXYFRFfLjaqnnRsRGyF7J9eYE3B8VgPc55qyXmqPeepDpyn5q0recrFUMEkDQOfBd4SEbuLjqcXSLoQ2BYRdxQdS4+qAs8CPhwRZwB78RCnA/IxxS8GngQ8EVgi6ZXFRmW2cDlPHcp5ak7OUx04T/UWF0MFktRHlmA+HRE3FB1PD3ku8CJJDwKfAc6T9KliQ+opm4HNEbH/DO31ZEnHMi8AfhYRj0fEDHADcHbBMfWixyT9CkD+c1vB8VgPcp5qy3mqM+epzpyn5qcrecrFUEEkiWws7X0R8YGi4+klEfH2iDg+Ik4iu6Hw6xHhMya5iHgUeFjSU/JFzwd+VGBIvWYTcJakofzv7Pn4xt1WbgQuyqcvAj5fYCzWg5yn2nOe6sx5ak7OU/PTlTxVPRqN2rw8F3gVcLekH+TL3hERNxcYky0cbwQ+LakGPABcXHA8PSMibpN0PXAn2dOw7gKuLjaqYknaCJwDjEjaDFwBvA+4TtKfkyXmlxQXofUo5yk7HM5TbThPHarIPKWIOBrtmpmZmZmZ9TQPkzMzMzMzs1JyMWRmZmZmZqXkYsjMzMzMzErJxZCZmZmZmZWSiyEzMzMzMyslF0NmR5ikYyXdKmmPpH84gu2ekz9usmdIWitpXFJlHuv2XPxmZmXkPNV23Z6L344+F0NWCEkPSprIP4h3Svq2pNdLWgy/k5cAo8CyiLhs9puSNkiazj+cxyR9RdJTux/m4YuITRExHBFp0bGYmR1JzlPOU1YOi+EP2hauF0bEUuBEsi/WehvZt50fcfM5I3QEnQj8KDp/ideVETEMHA9sAzZ0IzAzM/uFOE85T9ki52LIChcRuyLiRuClwEWSngEgqV/SekmbJD0m6V8kDe7fTtLlkrZK2iLpNZJC0sn5exskfVjSzZL2AufOo70LJf2g6Qzgae1ilnS2pO9L2pX/PHv/foGLgMvzM2ovmOPY9wH/CjQf81X5MW3Jp/tb7P+vJH121rIPSboqn/6GpL+R9K38rOaXJY00rfsiSffmx/oNSac2vfdg3v4PJe2V9PF8SMWX8ra+Kmllvu5Jeb9X8/mLJd2Xr/eApNd1On4zs4XAecp5yhYvF0PWMyLie8Bm4Hn5ovcDpwCnAycDxwHvBpB0PvCXwAvy936nRZMvB9YBS4FvztHes4BrgNcBq4CPADe2+YA/BrgJ+Kd83Q8AN0laFRGvBj5NfkYtIr7a6ZglDQOvAO7KF70TOCuP8ZnAc4B3tdj0U8D5klbk7VTJkvQnZx3/xcAaoAa8NV/3FGAj8BZgNXAz8AVJtaZt/wT43by/Xgh8CXgHMEL2ufGmNoe0DbgQWJbv+x/zvjUzW/CcpwDnKVtkXAxZr9kCHCNJwGuBSyNiLCL2AH8HvCxf70+BT0TEvflZq/e0aOvzEfGtiGgAU3O091rgIxFxW0SkEXFtvs1ZLdq9ALg/Ij4ZEfWI2Aj8mOzDeL7eKmkn8H/AMPDqfPkrgPdGxLaIeDw/rlfN3jgitgK3Ai/JF50PjEbEHU2rfSIi/jciJoDryBIXZMnopoj4SkTMAOuBQeDspm0/FBGPRcQjwH8Dt0XEXRExBfw7cEarg4qImyLip5H5L+DL/PyfBjOzxcB5ynnKFpFq0QGYzXIcMEZ2JmgIuCPLNwAI2D+m+onA7U3bPdyireZlc7V3ItnQhzc2bVPL9zPbE4GHZi17KI99vtZHRKszabPbfqhNDADXAm8APgq8koPPtgE82jS9jyyZHbKPiGhIepiD43+saXqixfwwLUj6A+AKsjN1CVmf390mfjOzhch5ynnKFhFfGbKeIenZZB903yR7ys0E8PSIWJG/luc3cwJsJbupc78TWjTZfGPoXO09DKxrem9FRAzlZ9Nm20KWlJqtBR75RY63jdltr82XtfI54LR87PqFZMMefuF95Gc3T+Aw48+HanyW7AzesRGxgmxogzpuaGa2QDhPtWzbecoWNBdDVjhJyyRdCHwG+FRE3J0PGfgo2VjeNfl6x0n6/Xyz64CLJZ0qaYh8THU782jvo8DrJf2mMkskXSBpaYvmbgZOkfRySVVJLwWeBnzxsDoisxF4l6TV+Y2k7yYbd93qmCaB68lubP1eRGya5z6uAy6Q9HxJfcBlZEMtvn2YsdeAfuBxoJ6fffu9w2zTzKxwzlMHcZ6yRcXFkBXpC5L2kJ3teifZDZ4XN73/NrKxyt+VtBv4KvAUgIj4EtmNobfk63wn32aqw/46tXc72XjsfwZ25Ou9ulUjEbGd7AzXZcB24HLgwogYnf+ht/W3ZMMqfkh22f7OfFk71wK/zqFDD9qKiJ+QDVf4ENmZyBeSPT52+peMeX+7e8huWL2OrA9fDtx4OG2amRXMeepQzlO2qCg6PmLebGHIH7l5D9AfEfWi4+kWSWvJbop9QkTsLjoeMzNrzXnKecp6k68M2YIl6Y8k1fLvEng/8IWSJZiE7LGtn3GCMTPrPc5TzlPW+1wM2UL2OrJxvz8FUrIn1pSCpCXAbrLvWLii4HDMzKw15ynnKetxHiZnZmZmZmal5CtDZmZmZmZWSi6GzMzMzMyslFwMmZmZmZlZKbkYMjMzMzOzUnIxZGZmZmZmpfT/bS3rXOpdF3AAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1008x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# LOOCV\n",
    "y = np.array(Auto.mpg)\n",
    "X = np.array(Auto.horsepower).reshape(-1,1)\n",
    "model = LinearRegression()\n",
    "loocv = LeaveOneOut()\n",
    "mse = make_scorer(mean_squared_error)\n",
    "\n",
    "# Dividing the dataset in training and validation and getting validation MSE for different polynomial.\n",
    "MSE = []\n",
    "for i in range(10):\n",
    "    polynomial_features= PolynomialFeatures(degree=i+1)\n",
    "    xp = polynomial_features.fit_transform(X)\n",
    "    results = cross_val_score(model, xp, y, cv=loocv, scoring = mse)\n",
    "    MSE.append(results.mean())\n",
    "\n",
    "fig = plt.figure(figsize = (14,4))\n",
    "ax1 = fig.add_subplot(121)\n",
    "ax1.plot(list(range(1,11)), MSE, '-ok', color = \"b\")\n",
    "ax1.set_ylim(16,30)\n",
    "ax1.set_xlabel('Degree of Polynomial', fontsize = '12')\n",
    "ax1.set_ylabel('Mean Squared Error', fontsize = '12')\n",
    "ax1.set_title(\" LOOCV\")\n",
    "\n",
    "# 10 - Fold validation for each polynomial curve.\n",
    "ax2 = fig.add_subplot(122)\n",
    "\n",
    "for k in range(9):\n",
    "    MSE = []\n",
    "    for i in range(10):\n",
    "        polynomial_features= PolynomialFeatures(degree=i+1)\n",
    "        xp = polynomial_features.fit_transform(X)\n",
    "        kfold = KFold(n_splits = 10, shuffle =  True, random_state = random.randint(1,100))\n",
    "        results = cross_val_score(model, xp, y, cv=kfold, scoring = mse)\n",
    "        MSE.append(results.mean())\n",
    "        \n",
    "    ax2.plot(list(range(1,11)), MSE)\n",
    "    \n",
    "ax2.set_ylim(16,30)\n",
    "ax2.set_xlabel('Degree of Polynomial', fontsize = '12')\n",
    "ax2.set_ylabel('Mean Squared Error', fontsize = '12')\n",
    "ax2.set_title(\"10-Fold Validation\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that even for 10 fold, validation error have less variance hence an improvement over the validation approach."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### K-Fold Cross Validation :-\n",
    "The approach involves randomly dividing the set of observations into k groups of approximately equal size. The first fold is treated as a validation set, and the method is fit on the remaining K-1 folds. The mean squared error is then computed on the observation in the held out fold. This procedure is repeated K times.\n",
    "\n",
    "The process results in k estimates of the test error. The K-fold CV estimates is computed by averaging these values.\n",
    "$${CV}_{(k)} = \\frac{1}{K} \\sum_{i = 1}^{K} {MSE}_i$$\n",
    "\n",
    "LOOCV is a special case of K-fold CV. Advantage of Kfold over LOOCV is that it is less computationally expensive. \n",
    "\n",
    "When we perform cross validation, our goal might be to determine how well a given statistical learning procedure can be expected to perform on independent data. In this case actual estimate of test MSE is of interest.\n",
    "\n",
    "But at other times we are interrested only in the location of the minimum point in the estimated test MSE curve to find out the suitable level of flexibility.\n",
    "\n",
    "### Bias - Variance trade off for K-fold Cross Validation :-\n",
    "KFold CV often gives more accurate estimate of the test error rate than LOOCV. This because of the bias-variance  trade off :-\n",
    "\n",
    "__Bias__:- The validation set approach can lead to overestimate the test error, since in this approach the training set used to fit the statistical learning method contains only half the observations of entire dataset.\n",
    "\n",
    "As LOOCV using almost all the observation, it provides approximately unbiased estimate. And KFold CV will lead to intermediate level of bias since each training set contains $\\frac{(k-1)}{k}n.$\n",
    "\n",
    "__Variance__:- When we perform LOOCV, we are in effect averaging the output of n fitted models each of which is trained on an almost identical set of observation, therefore these outputs are highly(positively) correlated with each other.\n",
    "\n",
    "In K-Fold CV with k<n, we are averaging the output of K fitted models that are somewhat less correlated with each other since the overlap between the training sets in each model is smaller. Since the mean of many highly correlated quantities has higher variance than does the mean of many quantities that are not as highly correlated, the test error estimate resulting from LOOCV tends to have high variance than does the test error estimate resulting from K-Fold CV.\n",
    "\n",
    "## Cross Validation on Classification Problem :-\n",
    "For classification, LOOCV error rate is\n",
    "$${CV}_{(n)} = \\frac{1}{n} \\sum_{i = 1}^{n} {Err}_i$$\n",
    "where ${Err}_i = I(y_i \\ne \\hat y_i)$\n",
    "Cross validation error can be used to decide on the flexibility of the learning method. We can plot CV error with increasing flexibility and select the one with minimum error rate."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Bootstrap\n",
    "\n",
    "Bootstrap can be used to quantify the uncertainity associated with a given estimator or statistical learning method.\n",
    "\n",
    "The idea is to find out the variability around an estimator. To do that we need a lot sample dataset and then calculating the same estimate and then calculating the variance. But it is not possible to generate new sample dataset from population as we mostly have just one sample available. \n",
    "\n",
    "The bootstrap obtain distinct datasets by repeatedly sampling observation from the original dataset. \n",
    "\n",
    "We continously sample observation (with replacement), n, and use them to calculate the value of estimator $\\alpha$. We continue this process till B times. We can compute standard error of these bootstrap estimates using\n",
    "$${SE}_B(\\hat \\alpha) = \\sqrt {\\frac{1}{B-1} \\sum_{r = 1}^{B} (\\hat \\alpha^{*r} - \\frac{1}{B} \\sum_{r^{'} = 1}^{B} \\hat \\alpha^{*r^{i}})^2}$$\n",
    "Bootstrap can be used to find SE of any estimate.\n",
    "\n",
    "Also, in case of linear regression, if the true relationship is not linear, bootstrap provides better estimates of standard error as it is non-paramteric. The standard formulas use $\\hat \\sigma^2$ estimate whch rely on assumption that linear model is correct."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## _Lab: Cross- Validation and Bootstrap_\n",
    "\n",
    "### Lab : The validation set approach\n",
    "\n",
    "We will see how to implement validation approach in Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "26.374851162407953\n"
     ]
    }
   ],
   "source": [
    "# Loading Auto dataset\n",
    "url = \"https://raw.githubusercontent.com/atystats/ISLR_Notes_codes/master/Datasets/Auto.csv\"\n",
    "Auto = pd.read_csv(url, error_bad_lines = False, na_values = '?')\n",
    "\n",
    "X = np.array(Auto['horsepower']).reshape(-1,1)\n",
    "y = np.array(Auto['mpg'])\n",
    "\n",
    "# Divinding the dataset in train and test.\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=20)\n",
    "\n",
    "# Linear Regression on training set.\n",
    "lm = LinearRegression()\n",
    "lm.fit(X_train,y_train)\n",
    "\n",
    "print(mean_squared_error(y_test, lm.predict(X_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19.08836013278956\n"
     ]
    }
   ],
   "source": [
    "# Now we will fit the model with polynomial terms degree 2.\n",
    "polynomial_features= PolynomialFeatures(degree=2)\n",
    "xp = polynomial_features.fit_transform(X)\n",
    "X_train, X_test, y_train, y_test = train_test_split(xp, y, test_size=0.5, random_state=20)\n",
    "\n",
    "lm = LinearRegression()\n",
    "lm.fit(X_train,y_train)\n",
    "\n",
    "print(mean_squared_error(y_test, lm.predict(X_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19.225822236707103\n"
     ]
    }
   ],
   "source": [
    "# Now we will fit the model with polynomial terms degree 3.\n",
    "\n",
    "polynomial_features= PolynomialFeatures(degree=3)\n",
    "xp = polynomial_features.fit_transform(X)\n",
    "X_train, X_test, y_train, y_test = train_test_split(xp, y, test_size=0.5, random_state=20)\n",
    "\n",
    "lm = LinearRegression()\n",
    "lm.fit(X_train,y_train)\n",
    "\n",
    "print(mean_squared_error(y_test, lm.predict(X_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that the error for degree 1,2 and 3 are 26.4, 19 and 19.1 respectively. Now we will select a different split of train and test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "22.854297270308564\n"
     ]
    }
   ],
   "source": [
    "# Divinding the dataset in train and test.\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=40)\n",
    "\n",
    "# Linear Regression on training set.\n",
    "lm = LinearRegression()\n",
    "lm.fit(X_train,y_train)\n",
    "\n",
    "print(mean_squared_error(y_test, lm.predict(X_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17.211724297391374\n"
     ]
    }
   ],
   "source": [
    "polynomial_features= PolynomialFeatures(degree=2)\n",
    "xp = polynomial_features.fit_transform(X)\n",
    "X_train, X_test, y_train, y_test = train_test_split(xp, y, test_size=0.5, random_state=40)\n",
    "\n",
    "lm = LinearRegression()\n",
    "lm.fit(X_train,y_train)\n",
    "\n",
    "print(mean_squared_error(y_test, lm.predict(X_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17.643970964506913\n"
     ]
    }
   ],
   "source": [
    "polynomial_features= PolynomialFeatures(degree=3)\n",
    "xp = polynomial_features.fit_transform(X)\n",
    "X_train, X_test, y_train, y_test = train_test_split(xp, y, test_size=0.5, random_state=40)\n",
    "\n",
    "lm = LinearRegression()\n",
    "lm.fit(X_train,y_train)\n",
    "\n",
    "print(mean_squared_error(y_test, lm.predict(X_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that using a different split results in different estimates of test error rate."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lab : Leave-One-Out Cross-Validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "24.231513517929226\n"
     ]
    }
   ],
   "source": [
    "# We can use cross_val_score to get the LOOCV.\n",
    "lm = LinearRegression()\n",
    "loocv = LeaveOneOut()\n",
    "mse = make_scorer(mean_squared_error)\n",
    "\n",
    "print(cross_val_score(lm, X,y, scoring = mse, cv = loocv).mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "cross_val_score will return n estimates of test error. We will take mean of all to get our LOOCV estimate of test error which is 24.23."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[24.231513517929226, 19.24821312448941, 19.33498406411397, 19.424430309374937, 19.03322024895203]\n"
     ]
    }
   ],
   "source": [
    "# We can calculate LOOCV estiamte for few polynomial models as well.\n",
    "cv_error = []\n",
    "lm = LinearRegression()\n",
    "loocv = LeaveOneOut()\n",
    "mse = make_scorer(mean_squared_error)\n",
    "for i in range(5):\n",
    "    polynomial_features= PolynomialFeatures(degree=i+1)\n",
    "    xp = polynomial_features.fit_transform(X)\n",
    "    results = cross_val_score(lm, xp, y, cv=loocv, scoring = mse)\n",
    "    cv_error.append(results.mean())\n",
    "print(cv_error)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that there is a sudden drop in MSE for degree 1 to 2 but no significant drop later on."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lab : K-Fold Cross Validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[24.248098354964892, 19.44593146195417, 19.572871855288646, 19.65952642053599, 19.12821822130557, 19.068743429073542, 19.208172513444264, 19.3147016080953, 19.26176031355881, 19.19136395196935]\n"
     ]
    }
   ],
   "source": [
    "# we will define K using KFold function and use it cross_val_score and compute the K Fold validation error.\n",
    "kfold = KFold(n_splits = 10, shuffle =  True, random_state = random.randint(1,100))\n",
    "\n",
    "cv_error = []\n",
    "lm = LinearRegression()\n",
    "kfold = KFold(n_splits = 10, shuffle =  True, random_state = 20)\n",
    "mse = make_scorer(mean_squared_error)\n",
    "for i in range(10):\n",
    "    polynomial_features= PolynomialFeatures(degree=i+1)\n",
    "    xp = polynomial_features.fit_transform(X)\n",
    "    results = cross_val_score(lm, xp, y, cv=kfold, scoring = mse)\n",
    "    cv_error.append(results.mean())\n",
    "print(cv_error)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lab : The Bootstrap"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.57583207459283"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# First we will create a function that will calculate an estimate based on some data.\n",
    "def alpha_fn(data, index):\n",
    "    X = data.X[index]\n",
    "    Y = data.Y[index]\n",
    "    return((Y.var() - np.cov(X,Y)[0,1])/(Y.var() + X.var() - 2 * np.cov(X,Y)[0,1]))\n",
    "\n",
    "url = \"https://raw.githubusercontent.com/atystats/ISLR_Notes_codes/master/Datasets/Portfolio.csv\"\n",
    "Portfolio = pd.read_csv(url, error_bad_lines = False, na_values = '?')\n",
    "alpha_fn(Portfolio, list(range(100)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.42576306047009127"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Now we will select a sample with replacement using np.random.choice and calculate the estimate again.\n",
    "alpha_fn(Portfolio, list(np.random.choice(100, 100)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "original :-  0.5758\n",
      "Bootsrap Standard Error :-  0.0937\n"
     ]
    }
   ],
   "source": [
    "def alpha_fn(data):\n",
    "    X = data.X\n",
    "    Y = data.Y\n",
    "    return((Y.var() - np.cov(X,Y)[0,1])/(Y.var() + X.var() - 2 * np.cov(X,Y)[0,1]))\n",
    "\n",
    "# boot function alternate is not in python. So we need to write our own function.\n",
    "def boot(data, statistic, R):\n",
    "    original = statistic(data)\n",
    "    stats_results = []\n",
    "    for i in range(R):\n",
    "        random_indices = list(np.random.choice(np.arange(1,len(data)+1,1), len(data)))\n",
    "        stats_results.append(statistic(data.loc[random_indices]))\n",
    "    std_error = np.std(stats_results, axis = 0)\n",
    "    print(\"original :- \", np.round(original,decimals = 4))\n",
    "    print(\"Bootsrap Standard Error :- \", np.round(std_error,decimals = 4))\n",
    "    \n",
    "boot(Portfolio, alpha_fn, R =1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Bootstrap for a linear regression model coefficients__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Build a model using statsmodel\n",
    "url = \"https://raw.githubusercontent.com/atystats/ISLR_Notes_codes/master/Datasets/Auto.csv\"\n",
    "Auto = pd.read_csv(url, error_bad_lines = False, na_values = '?')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "original :-  Intercept     39.9359\n",
      "horsepower    -0.1578\n",
      "dtype: float64\n",
      "Bootsrap Standard Error :-  [0.8674 0.0075]\n"
     ]
    }
   ],
   "source": [
    "def linear_reg_params(dataset):  \n",
    "    model = ols('mpg ~ horsepower', data = dataset).fit()\n",
    "    return(model.params)\n",
    "boot(Auto, linear_reg_params, 1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just like that we can use boot() function to calculate bootstrap error for any estimate."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Excercise : Conceptual"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2(a). Each datapoint has equal probability of being selected in the sample hence, 1/n.\n",
    "\n",
    "2(b). In bootstrap, since we are sampling with replacement, the observation has same probability of selection in each turn (1/n). So probability of not being selected in a sample is (1-1/n).\n",
    "\n",
    "2(c). Since in each of the trail is independent and the sample would have same probability of selection every time. The probability of not selecting an observation in n trial would be $(1-1/n)^n$.\n",
    "\n",
    "2(d). Since the probability of not being selected in the sample is $(1-1/5)^5$. The probability of getting selected is $1- (1-1/5)^5$ = 0.67.\n",
    "\n",
    "2(e). Since the probability of not being selected in the sample is $(1-1/100)^{100}$. The probability of getting selected is $1- (1-1/100)^{100}$ = 0.634.\n",
    "\n",
    "2(f). Since the probability of not being selected in the sample is $(1-1/10,000)^{10,000}$. The probability of getting selected is $1- (1-1/10,000)^{10,000}$ = 0.63."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD4CAYAAADrRI2NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAW9UlEQVR4nO3df5BdZ13H8feHbdOu8iMJWZh2N21SJ41E6zRwJxaqUsAmoToktI6ToBIUyaiU0SJxktGxGsZptYwgYwYaMCKMNFToxB0ssxP7Qx2lsDeT0jSBpdsUm91Fu5CmOrpjk/D1j/MsnNzczZ6b3ORm7/N5zdzJOd/znHufs2fzuWfPT0UEZmaWj5d0ugNmZnZhOfjNzDLj4Dczy4yD38wsMw5+M7PMXNLpDjRatGhRLFmypNPdMDObU/bt2/ediOir0vaiC/4lS5ZQr9c73Q0zszlF0r9XbetdPWZmmXHwm5llxsFvZpYZB7+ZWWYc/GZmmXHwm5llxsFvZpYZB7+ZWWYc/GZmmXHwm5llxsFvZpYZB7+ZWWZmDX5JuyQ9J+nJGaZL0kcljUp6QtJrS9M2SXoqvTa1s+ON9uwf58a7H2bp1n/gxrsfZs/+8fP5cWZmc1aVLf5PAWvPMP2twLL02gx8DEDSQuBO4CeBVcCdkhacS2dnsmf/ONseOMD4sSkCGD82xbYHDjj8zcyamDX4I+KfgaNnaLIO+HQUHgPmS7oCWAPsjYijEfE8sJczf4GctXuGRpg6fvKU2tTxk9wzNHI+Ps7MbE5rxz7+fuBIaXws1Waqn0bSZkl1SfXJycmWOzBxbKqluplZztoR/GpSizPUTy9G7IyIWkTU+voqPUDmFFfO722pbmaWs3YE/xiwuDQ+AEycod52W9Ysp/fSnlNqvZf2sGXN8vPxcWZmc1o7gn8QeGc6u+cG4IWI+DYwBKyWtCAd1F2dam23fmU/d916Hf3zexHQP7+Xu269jvUrm+5ZMjPL2qzP3JV0H3ATsEjSGMWZOpcCRMTHgQeBW4BR4H+BX03Tjkr6IDCc3mp7RJzpIPE5Wb+y30FvZlbBrMEfERtnmR7Ae2eYtgvYdXZdMzOz88FX7pqZZcbBb2aWGQe/mVlmHPxmZplx8JuZZcbBb2aWGQe/mVlmHPxmZplx8JuZZcbBb2aWGQe/mVlmHPxmZplx8JuZZcbBb2aWGQe/mVlmHPxmZpmpFPyS1koakTQqaWuT6VdLekjSE5IelTRQmnZS0uPpNdjOzpuZWeuqPHqxB9gB3EzxAPVhSYMRcajU7EPApyPibyS9GbgL+JU0bSoirm9zv83M7CxV2eJfBYxGxOGIeBHYDaxraLMCeCgNP9JkupmZXSSqBH8/cKQ0PpZqZV8DbkvDbwdeJumVafxySXVJj0la3+wDJG1ObeqTk5MtdN/MzFpVJfjVpBYN4x8A3ihpP/BGYBw4kaZdFRE14B3ARyT9yGlvFrEzImoRUevr66veezMza9ms+/gptvAXl8YHgIlyg4iYAG4FkPRS4LaIeKE0jYg4LOlRYCXw9Dn33MzMzkqVLf5hYJmkpZLmARuAU87OkbRI0vR7bQN2pfoCSZdNtwFuBMoHhc3M7AKbNfgj4gRwOzAEfB24PyIOStou6W2p2U3AiKRvAq8G/iTVXwPUJX2N4qDv3Q1nA5mZ2QWmiMbd9Z1Vq9WiXq93uhtmZnOKpH3peOqsfOWumVlmHPxmZplx8JuZZcbBb2aWGQe/mVlmHPxmZplx8JuZZcbBb2aWGQe/mVlmHPxmZplx8JuZZcbBb2aWGQe/mVlmHPxmZplx8JuZZcbBb2aWmUrBL2mtpBFJo5K2Npl+taSHJD0h6VFJA6VpmyQ9lV6b2tl5MzNr3azBL6kH2AG8FVgBbJS0oqHZh4BPR8RPANuBu9K8C4E7gZ8EVgF3SlrQvu6bmVmrqmzxrwJGI+JwRLwI7AbWNbRZATyUhh8pTV8D7I2IoxHxPLAXWHvu3TYzs7NVJfj7gSOl8bFUK/sacFsafjvwMkmvrDgvkjZLqkuqT05OVu27mZmdhSrBrya1xie0fwB4o6T9wBuBceBExXmJiJ0RUYuIWl9fX4UumZnZ2bqkQpsxYHFpfACYKDeIiAngVgBJLwVui4gXJI0BNzXM++g59NfMzM5RlS3+YWCZpKWS5gEbgMFyA0mLJE2/1zZgVxoeAlZLWpAO6q5ONTMz65BZgz8iTgC3UwT214H7I+KgpO2S3paa3QSMSPom8GrgT9K8R4EPUnx5DAPbU83MzDpEEaftcu+oWq0W9Xq9090wM5tTJO2LiFqVtr5y18wsMw5+M7PMOPjNzDLj4Dczy4yD38wsMw5+M7PMOPjNzDLj4Dczy4yD38wsMw5+M7PMOPjNzDLj4Dczy4yD38wsMw5+M7PMOPjNzDLj4Dczy0yl4Je0VtKIpFFJW5tMv0rSI5L2S3pC0i2pvkTSlKTH0+vj7V4AMzNrzawPW5fUA+wAbqZ48PqwpMGIOFRq9gcUj2T8mKQVwIPAkjTt6Yi4vr3dNjOzs1Vli38VMBoRhyPiRWA3sK6hTQAvT8OvACba10UzM2unKsHfDxwpjY+lWtkfAb8saYxia/99pWlL0y6gf5L0080+QNJmSXVJ9cnJyeq9NzOzllUJfjWpNT6hfSPwqYgYAG4BPiPpJcC3gasiYiXwfuCzkl7eMC8RsTMiahFR6+vra20JzMysJVWCfwxYXBof4PRdOe8G7geIiC8DlwOLIuL/IuK7qb4PeBq49lw7bWZmZ69K8A8DyyQtlTQP2AAMNrR5FngLgKTXUAT/pKS+dHAYSdcAy4DD7eq8mZm1btazeiLihKTbgSGgB9gVEQclbQfqETEI/C7wCUl3UOwGeldEhKSfAbZLOgGcBH4jIo6et6UxM7NZKaJxd31n1Wq1qNfrne6GmdmcImlfRNSqtPWVu2ZmmXHwm5llxsFvZpYZB7+ZWWYc/GZmmXHwm5llxsFvZpYZB7+ZWWYc/GZmmXHwm5llxsFvZpYZB7+ZWWYc/GZmmXHwm5llxsFvZpYZB7+ZWWYqBb+ktZJGJI1K2tpk+lWSHpG0X9ITkm4pTduW5huRtKadnTczs9bN+ujF9MzcHcDNFA9eH5Y0GBGHSs3+ALg/Ij4maQXwILAkDW8Afgy4EvhHSddGxMl2L4iZmVVTZYt/FTAaEYcj4kVgN7CuoU0AL0/DrwAm0vA6YHdE/F9EPAOMpvczM7MOqRL8/cCR0vhYqpX9EfDLksYotvbf18K8SNosqS6pPjk5WbHrZmZ2NqoEv5rUGp/QvhH4VEQMALcAn5H0korzEhE7I6IWEbW+vr4KXTIzs7M16z5+iq30xaXxAX6wK2fau4G1ABHxZUmXA4sqzmtmZhdQlS3+YWCZpKWS5lEcrB1saPMs8BYASa8BLgcmU7sNki6TtBRYBny1XZ03M7PWzbrFHxEnJN0ODAE9wK6IOChpO1CPiEHgd4FPSLqDYlfOuyIigIOS7gcOASeA9/qMHjOzzlKRzxePWq0W9Xq9090wM5tTJO2LiFqVtr5y18wsMw5+M7PMOPjNzDLj4Dczy4yD38wsMw5+M7PMOPjNzDLj4Dczy4yD38wsMw5+M7PMOPjNzDLj4Dczy4yD38wsMw5+M7PMOPjNzDLj4Dczy0yVZ+4iaS3wFxRP4PpkRNzdMP3DwJvS6A8Br4qI+WnaSeBAmvZsRLytHR1vtGf/OPcMjTBxbIor5/eyZc1y1q/sPx8fZWY2p80a/JJ6gB3AzRQPTx+WNBgRh6bbRMQdpfbvA1aW3mIqIq5vX5dPt2f/ONseOMDU8eKpjuPHptj2QPFd4/A3MztVlV09q4DRiDgcES8Cu4F1Z2i/EbivHZ2r6p6hke+H/rSp4ye5Z2jkQnbDzGxOqBL8/cCR0vhYqp1G0tXAUuDhUvlySXVJj0laP8N8m1Ob+uTkZMWu/8DEsamW6mZmOasS/GpSm+kJ7RuAz0dEefP7qvQA4HcAH5H0I6e9WcTOiKhFRK2vr69Cl0515fzelupmZjmrEvxjwOLS+AAwMUPbDTTs5omIifTvYeBRTt3/3xZb1iyn99KeU2q9l/awZc3ydn+UmdmcVyX4h4FlkpZKmkcR7oONjSQtBxYAXy7VFki6LA0vAm4EDjXOe67Wr+znrluvo39+LwL65/dy163X+cCumVkTs57VExEnJN0ODFGczrkrIg5K2g7UI2L6S2AjsDsiyruBXgPcK+l7FF8yd5fPBmqn9Sv7HfRmZhXo1JzuvFqtFvV6vdPdMDObUyTtS8dTZ+Urd83MMuPgNzPLjIPfzCwzDn4zs8w4+M3MMuPgNzPLjIPfzCwzDn4zs8w4+M3MMuPgNzPLjIPfzCwzDn4zs8w4+M3MMuPgNzPLzKz3458r9uwf556hESaOTXHl/F62rFnu+/ObmTXRFcG/Z/842x44wNTx4lG/48em2PbAAQCHv5lZg0q7eiStlTQiaVTS1ibTPyzp8fT6pqRjpWmbJD2VXpva2flp9wyNfD/0p00dP8k9QyPn4+PMzOa0Wbf4JfUAO4CbKR68PixpsPwIxYi4o9T+faQHqktaCNwJ1IAA9qV5n2/nQkwcm2qpbmaWsypb/KuA0Yg4HBEvAruBdWdovxG4Lw2vAfZGxNEU9nuBtefS4WaunN/bUt3MLGdVgr8fOFIaH0u100i6GlgKPNzKvJI2S6pLqk9OTlbp9ym2rFlO76U9p9R6L+1hy5rlLb+XmVm3qxL8alKb6QntG4DPR8T0DvdK80bEzoioRUStr6+vQpdOtX5lP3fdeh3983sR0D+/l7tuvc4Hds3MmqhyVs8YsLg0PgBMzNB2A/Dehnlvapj30erdq279yn4HvZlZBVW2+IeBZZKWSppHEe6DjY0kLQcWAF8ulYeA1ZIWSFoArE41MzPrkFm3+CPihKTbKQK7B9gVEQclbQfqETH9JbAR2B0RUZr3qKQPUnx5AGyPiKPtXQQzM2uFSjl9UajValGv11uez1fumlnOJO2LiFqVtr5y18wsM11xkzZfuWtmVl1XBL+v3DUzq64rgt9X7pqZVdcVwe8rd83MquuK4F+/sp/bXtdPj4oLhXskbnudL+gyM2umK4J/z/5xvrBvnJPp1NSTEXxh3zh79o93uGdmZhefrgh+n9VjZlZdVwS/z+oxM6uuK4LfZ/WYmVXXFcH/ph9tfivnmepmZjnriuB/5BvNH94yU93MLGddEfzjM+zLn6luZpazrgj+6fP3q9bNzHLWFcF/coZbS89UNzPLWVcE/0zb9d7eNzM7XaXgl7RW0oikUUlbZ2jzi5IOSToo6bOl+klJj6fXaY9sbIeZtuu9vW9mdrpZH8QiqQfYAdxM8fD0YUmDEXGo1GYZsA24MSKel/Sq0ltMRcT1be63mZmdpSpb/KuA0Yg4HBEvAruBdQ1t3gPsiIjnASLiufZ208zM2qVK8PcDR0rjY6lWdi1wraR/lfSYpLWlaZdLqqf6+mYfIGlzalOfnPS592Zm51OVZ+42O0bauPv8EmAZcBMwAPyLpB+PiGPAVRExIeka4GFJByLi6VPeLGInsBOKh623uAxmZtaCKlv8Y8Di0vgAMNGkzd9HxPGIeAYYofgiICIm0r+HgUeBlefYZzMzOwdVgn8YWCZpqaR5wAag8eycPcCbACQtotj1c1jSAkmXleo3AocwM7OOmXVXT0SckHQ7MAT0ALsi4qCk7UA9IgbTtNWSDgEngS0R8V1JbwDulfQ9ii+Zu8tnA5mZ2YVXZR8/EfEg8GBD7Q9LwwG8P73Kbf4NuO7cu2lmZu1SKfjnsiVb/6HTXTAza9m37v658/beXXHLBjOzbnM+N1od/GZmmXHwm5llxsFvZpYZB7+ZWWa6IvjP59FvM7NOOJ+51jWnczr8zcyq6YotfjMzq87Bb2aWGQe/mVlmHPxmZplx8JuZZcbBb2aWGQe/mVlmHPxmZpmpFPyS1koakTQqaesMbX5R0iFJByV9tlTfJOmp9NrUro6bmdnZmfXKXUk9wA7gZoqHqg9LGiw/QlHSMmAbcGNEPC/pVam+ELgTqAEB7EvzPt/+RTEzsyqqbPGvAkYj4nBEvAjsBtY1tHkPsGM60CPiuVRfA+yNiKNp2l5gbXu6bmZmZ6NK8PcDR0rjY6lWdi1wraR/lfSYpLUtzIukzZLqkuqTk5PVe29mZi2rcpM2NalFk/dZBtwEDAD/IunHK85LROwEdgJImpT07xX6NZNFwHfOYf65KLdlzm15wcuci3NZ5qurNqwS/GPA4tL4ADDRpM1jEXEceEbSCMUXwRjFl0F53kfP9GER0VehTzOSVI+I2rm8x1yT2zLntrzgZc7FhVrmKrt6hoFlkpZKmgdsAAYb2uwB3gQgaRHFrp/DwBCwWtICSQuA1almZmYdMusWf0SckHQ7RWD3ALsi4qCk7UA9Igb5QcAfAk4CWyLiuwCSPkjx5QGwPSKOno8FMTOzahRx2i73OU3S5nTMIBu5LXNuywte5lxcqGXuuuA3M7Mz8y0bzMwy4+A3M8tM1wR/lfsJXcwkLZb0iKSvp/sd/XaqL5S0N93raG86OwoVPpqW9wlJry29V9P7I0l6naQDaZ6PSmp2ncUFJalH0n5JX0zjSyV9JfX9c+lMMiRdlsZH0/QlpffYluojktaU6hfd74Sk+ZI+L+kbaV2/PoN1fEf6nX5S0n2SLu+29Sxpl6TnJD1Zqp339TrTZ8wqIub8i+Jso6eBa4B5wNeAFZ3uV4vLcAXw2jT8MuCbwArgz4Ctqb4V+NM0fAvwJYqL5G4AvpLqCylOpV0ILEjDC9K0rwKvT/N8CXjrRbDc7wc+C3wxjd8PbEjDHwd+Mw3/FvDxNLwB+FwaXpHW92XA0vR70HOx/k4AfwP8ehqeB8zv5nVMcaX+M0Bvaf2+q9vWM/AzwGuBJ0u1875eZ/qMWfvb6f8Ibfqhvx4YKo1vA7Z1ul/nuEx/T3FjvBHgilS7AhhJw/cCG0vtR9L0jcC9pfq9qXYF8I1S/ZR2HVrGAeAh4M3AF9Mv9XeASxrXK8Upw69Pw5ekdmpc19PtLsbfCeDlKQTVUO/mdTx925aFab19keIeXl23noElnBr85329zvQZs726ZVdPpXsCzRXpz9uVwFeAV0fEtwHSv69KzWZa5jPVx5rUO+kjwO8B30vjrwSORcSJNF7u4/eXK01/IbVv9efQSdcAk8Bfp91bn5T0w3TxOo6IceBDwLPAtynW2z66ez1PuxDrdabPOKNuCf5K9wSaCyS9FPgC8DsR8V9natqkFmdR7whJPw88FxH7yuUmTWOWaXNieZNLKHYHfCwiVgL/Q/Hn+Uzm/DKnfc7rKHbPXAn8MPDWJk27aT3PpuPL2C3BX+V+Qhc9SZdShP7fRsQDqfyfkq5I068Apm95PdMyn6k+0KTeKTcCb5P0LYpbfb+Z4i+A+ZKmrygv9/H7y5WmvwI4Sus/h04aA8Yi4itp/PMUXwTduo4BfhZ4JiImo7iX1wPAG+ju9TztQqzXmT7jjLol+KvcT+iilo7S/xXw9Yj489KkQWD66P4min3/0/V3pjMEbgBeSH/qNb0/Upr235JuSJ/1ztJ7XXARsS0iBiJiCcX6ejgifgl4BPiF1Kxxead/Dr+Q2keqb0hngyyluDngV7kIfyci4j+AI5KWp9JbgEN06TpOngVukPRDqU/Ty9y167nkQqzXmT7jzDp54KfNB1ZuoTgT5mng9zvdn7Po/09R/Pn2BPB4et1CsX/zIeCp9O/C1F4UT0Z7GjgA1Erv9WvAaHr9aqleA55M8/wlDQcZO7jsN/GDs3quofgPPQr8HXBZql+exkfT9GtK8/9+WqYRSmexXIy/E8D1QD2t5z0UZ2909ToG/hj4RurXZyjOzOmq9QzcR3EM4zjFFvq7L8R6nekzZnv5lg1mZpnpll09ZmZWkYPfzCwzDn4zs8w4+M3MMuPgNzPLjIPfzCwzDn4zs8z8P43m6oJiFdiyAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 2(g).\n",
    "x = np.arange(1,100001,1)\n",
    "y = [1-(1-1/i)**i for i in x]\n",
    "plt.scatter(x, y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Probability decrease sharply intially, as in the beginning we have less option to choose from. After a certain value of n, the probability stabilize because of increased number of n we have lot chances to include an observation but also we have a lot of option to choose from."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6253"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#2(h). \n",
    "store = []\n",
    "for i in range(10000):\n",
    "    sample = []\n",
    "    sample = list(np.random.choice(100, 100))\n",
    "    store.append(4 in sample)\n",
    "np.mean(store)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. (a). The approach involves randomly dividing the set of observations into k groups of approximately equal size. The first fold is treated as a validation set, and the method is fit on the remaining K-1 folds. The mean squared error is then computed on the observation in the held out fold. This procedure is repeated K times.\n",
    "\n",
    "3.(b). (i). The validation set approach can lead to overestimate the test error, since in this approach the training set used to fit the statistical learning method contains only half the observations of entire dataset.\n",
    "KFold CV will lead to intermediate level of bias since each training set contains $\\frac{(k-1)}{k}n.$\n",
    "\n",
    "3. (b). (ii). Advantage of Kfold over LOOCV is that it is less computationally expensive. KFold CV often gives more accurate estimate of the test error rate than LOOCV.\n",
    "\n",
    "4. We might estimate the standard deviation of our prediction by using the bootstrap approach. The bootstrap approach works by repeatedly sampling observations (with replacement) from the original data set B times, for some large value of B, each time fitting a new model and subsequently obtaining the RMSE of the estimates for all B models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Excercise : Applied"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    9667\n",
       "1     333\n",
       "Name: Default_0_1, dtype: int64"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "url = \"https://raw.githubusercontent.com/atystats/ISLR_Notes_codes/master/Datasets/Default.csv\"\n",
    "Default = pd.read_csv(url, error_bad_lines = False, index_col = 0)\n",
    "Default['Default_0_1'] = 0\n",
    "Default.loc[Default['default'] == 'Yes', 'Default_0_1'] = 1\n",
    "Default['Student_0_1'] = 0\n",
    "Default.loc[Default['student'] == 'Yes', 'Student_0_1'] = 1\n",
    "Default['Default_0_1'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "                   intercept_scaling=1, l1_ratio=None, max_iter=100,\n",
       "                   multi_class='auto', n_jobs=None, penalty='l2',\n",
       "                   random_state=None, solver='lbfgs', tol=0.0001, verbose=0,\n",
       "                   warm_start=False)"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 5.(a). Fitting a logistic regression.\n",
    "X = np.array(Default[['balance','income']])\n",
    "y = np.array(Default['Default_0_1'])\n",
    "\n",
    "clf = LogisticRegression()\n",
    "clf.fit(X,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 5.(b). Validation approach\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=20)\n",
    "clf = LogisticRegression()\n",
    "clf.fit(X_train,y_train)\n",
    "prediction = clf.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0322"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(prediction != y_test).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3% of the observations are misclassified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0284"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 5.(c). Repeating using different train and test split.\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=40)\n",
    "clf = LogisticRegression()\n",
    "clf.fit(X_train,y_train)\n",
    "prediction = clf.predict(X_test)\n",
    "(prediction != y_test).mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0262"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=60)\n",
    "clf = LogisticRegression()\n",
    "clf.fit(X_train,y_train)\n",
    "prediction = clf.predict(X_test)\n",
    "(prediction != y_test).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All the three validation approach gives different results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.0322"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = np.array(Default[['balance','income','Student_0_1']])\n",
    "y = np.array(Default['Default_0_1'])\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, random_state=20)\n",
    "clf = LogisticRegression()\n",
    "clf.fit(X_train,y_train)\n",
    "prediction = clf.predict(X_test)\n",
    "(prediction != y_test).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Adding the 3rd variable did not result in any improvement in the model. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078948\n",
      "         Iterations 10\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Coef.</th>\n",
       "      <th>Std.Err.</th>\n",
       "      <th>z</th>\n",
       "      <th>P&gt;|z|</th>\n",
       "      <th>[0.025</th>\n",
       "      <th>0.975]</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>const</th>\n",
       "      <td>-11.540468</td>\n",
       "      <td>0.434772</td>\n",
       "      <td>-26.543696</td>\n",
       "      <td>3.036732e-155</td>\n",
       "      <td>-12.392607</td>\n",
       "      <td>-10.688330</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>balance</th>\n",
       "      <td>0.005647</td>\n",
       "      <td>0.000227</td>\n",
       "      <td>24.835379</td>\n",
       "      <td>3.720520e-136</td>\n",
       "      <td>0.005201</td>\n",
       "      <td>0.006093</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>income</th>\n",
       "      <td>0.000021</td>\n",
       "      <td>0.000005</td>\n",
       "      <td>4.174113</td>\n",
       "      <td>2.991499e-05</td>\n",
       "      <td>0.000011</td>\n",
       "      <td>0.000031</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             Coef.  Std.Err.          z          P>|z|     [0.025     0.975]\n",
       "const   -11.540468  0.434772 -26.543696  3.036732e-155 -12.392607 -10.688330\n",
       "balance   0.005647  0.000227  24.835379  3.720520e-136   0.005201   0.006093\n",
       "income    0.000021  0.000005   4.174113   2.991499e-05   0.000011   0.000031"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 6.(a). Fit logistic regression and get standard error of each coefficient.\n",
    "X_train = Default[['balance','income']]\n",
    "y_train = Default['Default_0_1']\n",
    "X_train = sm.add_constant(X_train)\n",
    "\n",
    "clf = sm.Logit(y_train, X_train).fit()\n",
    "clf.summary2().tables[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078948\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.088082\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079917\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076110\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078156\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074271\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075125\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081930\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077785\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081634\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076574\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078227\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071789\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083341\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079934\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078079\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076682\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073617\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078182\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081350\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077539\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083891\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082203\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072536\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073988\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075215\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081379\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083768\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076858\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079461\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081009\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.087830\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078896\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076283\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085006\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081836\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072371\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078627\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077202\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082767\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.087361\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076321\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081006\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073328\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082397\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080914\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077797\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078533\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079270\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085009\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080859\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085743\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.089129\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076928\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075686\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081630\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077682\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073787\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074414\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080741\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082147\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084049\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079262\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081969\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078647\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077433\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081126\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071368\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078191\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080534\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080915\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078406\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076849\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083005\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081672\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081305\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080888\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075585\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078936\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081594\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075198\n",
      "         Iterations 10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078212\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074808\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078051\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078441\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076008\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085148\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075081\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080792\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081601\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077277\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072321\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078482\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083956\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076487\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.087255\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075584\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085275\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.088350\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082294\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076769\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080588\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080201\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078011\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075519\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075325\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075792\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079843\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075851\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078334\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074187\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077281\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082365\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076365\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081611\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083030\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076471\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078220\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080443\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077869\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076072\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078899\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078901\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080990\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075653\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080231\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082289\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079112\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083873\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081772\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.088919\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.087228\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076334\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081801\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076316\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077821\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071220\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082965\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080783\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079909\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.067432\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075528\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075366\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.089294\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078755\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077611\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077022\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080283\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084079\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078475\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079037\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075710\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074788\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075105\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085808\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078074\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085342\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080434\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084696\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075020\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075990\n",
      "         Iterations 10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085400\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075768\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081907\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072619\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083018\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083382\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081415\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.086980\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077620\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072191\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079983\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077444\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077199\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079796\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079909\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082331\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072746\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081213\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078818\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075102\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077178\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.087945\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073912\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085819\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077876\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080838\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082192\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076393\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080275\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.069952\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076665\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080875\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081506\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076532\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076953\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083886\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077182\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082682\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075907\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075285\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083168\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084706\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074841\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078106\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083912\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078427\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079987\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073802\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079701\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079436\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078088\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076703\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077299\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077454\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078076\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079315\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078809\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080841\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079281\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073797\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081736\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078747\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081549\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082427\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082293\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078694\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080266\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075819\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078145\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085893\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074576\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079209\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075461\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074170\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074589\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082076\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083499\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076455\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078524\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082277\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077224\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076882\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081263\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077417\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072031\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.088418\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077640\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079216\n",
      "         Iterations 10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077829\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075288\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072631\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079876\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081437\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076623\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.069559\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071262\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078353\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079221\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083538\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075014\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081007\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083757\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075027\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079019\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085419\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076404\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078677\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080410\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081855\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082517\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079772\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074397\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074863\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074453\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.087821\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075052\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.070343\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073745\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074228\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080303\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.067406\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078569\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080853\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074733\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083740\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074812\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074280\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072423\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080467\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079116\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.069618\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075514\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073175\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072376\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072361\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.088660\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077217\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074337\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078988\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081422\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079807\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071131\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082719\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076947\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078748\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.069246\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075058\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077675\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076274\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082877\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081887\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080385\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073316\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080338\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073064\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077070\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075567\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081243\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074893\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079405\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071397\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078957\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076992\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.068305\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075714\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082513\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085211\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079708\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075437\n",
      "         Iterations 10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083119\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076959\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084143\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083320\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079783\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071519\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077587\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078733\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076692\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073660\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073243\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080293\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078890\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080818\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075443\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.086262\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077515\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077540\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074405\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080805\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073437\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081539\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078116\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.087006\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080080\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074816\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082219\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073078\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079104\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.067909\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080374\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.091864\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085788\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080384\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074731\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076266\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080017\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076778\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.086776\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080366\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080724\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078539\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075504\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081497\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077348\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080866\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075019\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077157\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084515\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077718\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072256\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082330\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074752\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077578\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083215\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077945\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.069289\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080891\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078447\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083971\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079217\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076889\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084242\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081961\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074358\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076558\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074951\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.066108\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077095\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083847\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078229\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.089138\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081649\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078912\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077428\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084098\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083439\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077665\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075284\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084755\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078210\n",
      "         Iterations 10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076224\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074185\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083660\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075235\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077298\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074963\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078054\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081058\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085383\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078070\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072775\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083679\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082117\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077030\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075511\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079759\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084441\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080021\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074798\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073825\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076224\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074646\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078358\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075175\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072360\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075525\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078339\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077218\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073999\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080848\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077660\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082094\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072105\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085081\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.086810\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078501\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077202\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080155\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071853\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078490\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073335\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083888\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081079\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079038\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077312\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075529\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074661\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.070524\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085416\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079994\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077761\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079057\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081382\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080537\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084150\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080027\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080579\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081511\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076156\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077410\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072183\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071620\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077265\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071124\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082337\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082703\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076445\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.068208\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075577\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082287\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.086499\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073894\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077213\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081804\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083955\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083068\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082161\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078026\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079788\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080487\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074171\n",
      "         Iterations 10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079898\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081812\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077546\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083963\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076301\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079738\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078163\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080123\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076343\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075914\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075478\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083268\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082169\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081007\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074444\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080343\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075685\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079186\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071961\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081770\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077753\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082681\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076989\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079072\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076541\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078868\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083347\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080453\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081149\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085048\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080914\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.086157\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084038\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082050\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077564\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079942\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073705\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084174\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081430\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083367\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081560\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078356\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076778\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071676\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081267\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079078\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076340\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074777\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080375\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084024\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074351\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077849\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074540\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079477\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079620\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077344\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075495\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084138\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085756\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083197\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078981\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084951\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080480\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076313\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082243\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074797\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078689\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075963\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076655\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079624\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079040\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080797\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078777\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082095\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075615\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080220\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076829\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081539\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082307\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081027\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083350\n",
      "         Iterations 10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.069455\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078154\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081615\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081900\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083159\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085233\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081375\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085127\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077217\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078148\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079535\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076732\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078513\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077235\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079462\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078752\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082187\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077797\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072015\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075035\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077061\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075001\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080038\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083816\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080510\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078149\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082669\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084553\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076656\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.086075\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082833\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080801\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.087543\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082655\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076407\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083281\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075707\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081022\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.086837\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080740\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083215\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080955\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080190\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071905\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073437\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078406\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077111\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079021\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078921\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078059\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077317\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082284\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081808\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079221\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078180\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082019\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074622\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083744\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083056\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079472\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079653\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079540\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075964\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075569\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078939\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079853\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073827\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080094\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078492\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075181\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082981\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077540\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078210\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076078\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081566\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.069325\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082355\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083897\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076471\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081981\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081683\n",
      "         Iterations 10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077665\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078216\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078289\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075313\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078939\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077274\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080707\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080110\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082647\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071859\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077499\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075608\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074044\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082025\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073816\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075273\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078080\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078005\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077183\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081817\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075079\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074382\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076996\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082728\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080815\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084166\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075432\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080991\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085130\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076142\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076534\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081951\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082524\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.068259\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075404\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071815\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082223\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081223\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.069860\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071647\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075936\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077125\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080201\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074940\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071247\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080995\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072445\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081485\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080759\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075316\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078340\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076133\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078025\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076178\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077150\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079989\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.086475\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084845\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077987\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077287\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076528\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077493\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072092\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078326\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082161\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077334\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073884\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074884\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080264\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074848\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081057\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082452\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075794\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073227\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.070978\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077148\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072237\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075462\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076942\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074250\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073588\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.086310\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081699\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.088063\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080128\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085069\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080007\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075693\n",
      "         Iterations 10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.088043\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.089296\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072472\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082918\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079607\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080515\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082725\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079121\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082009\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079372\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076168\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083947\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083310\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074364\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080773\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079729\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075966\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078761\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.070718\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085810\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081092\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075347\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076750\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079720\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075531\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077300\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083795\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.086919\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082087\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076928\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077668\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084745\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080086\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074253\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075834\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079844\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085944\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072033\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081135\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073653\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075910\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077889\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081871\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.069841\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076769\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081756\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079636\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076071\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073266\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076122\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084007\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078891\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076117\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072167\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077661\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084262\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082724\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078823\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076435\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077306\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079577\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083030\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074930\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081185\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081403\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075798\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.069109\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082470\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085243\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.087231\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078874\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077013\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077731\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083291\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.086821\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083970\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083062\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080997\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077051\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075057\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071970\n",
      "         Iterations 10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081744\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082685\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083915\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077524\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079117\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075127\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076131\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078573\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077890\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077848\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079328\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075749\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073211\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074707\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076911\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083294\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083903\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075651\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084059\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079480\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078527\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075597\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078849\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083936\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081394\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080337\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077060\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079209\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082817\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082357\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077774\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078495\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078290\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079853\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078489\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077663\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083628\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080041\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077206\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078579\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074828\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077634\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082701\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080153\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084334\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082168\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077309\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079051\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075954\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080723\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076192\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076349\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079263\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.068894\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071930\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077753\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077777\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080731\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081818\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075754\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072594\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075690\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083323\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077575\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072298\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078751\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083587\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080653\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085663\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077493\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075086\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082231\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078717\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076457\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078985\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.086660\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074999\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076926\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079016\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075368\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084532\n",
      "         Iterations 10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081443\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.087948\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080979\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075529\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084709\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077111\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078756\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077489\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076675\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073114\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082589\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075689\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075148\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079592\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074401\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081880\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078218\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082427\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072568\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079146\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.086758\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082292\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076146\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084996\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085039\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076464\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080726\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075556\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077728\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078406\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081756\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.072289\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077204\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074864\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081064\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079253\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083502\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081128\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077785\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075422\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084568\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077095\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080469\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076223\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078944\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.083065\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075929\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.073560\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081572\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084090\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080500\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078283\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079239\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084940\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076177\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082580\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075369\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.069385\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075455\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071726\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085144\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.084461\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080633\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.085132\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075588\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076684\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.069663\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082520\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076528\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077461\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079104\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075149\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.081173\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079039\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080138\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076956\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082292\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.067203\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075471\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.076979\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.082236\n",
      "         Iterations 10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079774\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080545\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080268\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.071987\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.086327\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.077947\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.079774\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074234\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078522\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.087095\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.074427\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.080593\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078517\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075536\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.078517\n",
      "         Iterations 10\n",
      "Optimization terminated successfully.\n",
      "         Current function value: 0.075656\n",
      "         Iterations 10\n",
      "original :-  const     -11.5405\n",
      "balance     0.0056\n",
      "income      0.0000\n",
      "dtype: float64\n",
      "Bootsrap Standard Error :-  [4.352e-01 2.000e-04 0.000e+00]\n"
     ]
    }
   ],
   "source": [
    "# 6(b) and (c) and (d).\n",
    "# boot function alternate is not in python. So we need to write our own function.\n",
    "def boot(data, statistic, R):\n",
    "    original = statistic(data)\n",
    "    stats_results = []\n",
    "    for i in range(R):\n",
    "        random_indices = list(np.random.choice(np.arange(1,len(data)+1,1), len(data)))\n",
    "        stats_results.append(statistic(data.loc[random_indices]))\n",
    "    std_error = np.std(stats_results, axis = 0)\n",
    "    print(\"original :- \", np.round(original,decimals = 4))\n",
    "    print(\"Bootsrap Standard Error :- \", np.round(std_error,decimals = 4))\n",
    "    \n",
    "def logit_reg_params(dataset):  \n",
    "    X_train = dataset[['balance','income']]\n",
    "    y_train = dataset['Default_0_1']\n",
    "    X_train = sm.add_constant(X_train)\n",
    "    \n",
    "    clf = sm.Logit(y_train, X_train).fit()\n",
    "    return(clf.params)\n",
    "\n",
    "boot(Default, logit_reg_params, 1000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At the end of the above output we can see the bootstrap standard error of the logistic regression coefficients which are very much close to the one we got from 5.(a)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.683297\n",
      "         Iterations 4\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"simpletable\">\n",
       "<caption>Logit Regression Results</caption>\n",
       "<tr>\n",
       "  <th>Dep. Variable:</th>     <td>Direction_0_1</td>  <th>  No. Observations:  </th>  <td>  1089</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Model:</th>                 <td>Logit</td>      <th>  Df Residuals:      </th>  <td>  1086</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Method:</th>                 <td>MLE</td>       <th>  Df Model:          </th>  <td>     2</td> \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Date:</th>            <td>Mon, 13 Apr 2020</td> <th>  Pseudo R-squ.:     </th> <td>0.005335</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Time:</th>                <td>21:04:07</td>     <th>  Log-Likelihood:    </th> <td> -744.11</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>converged:</th>             <td>True</td>       <th>  LL-Null:           </th> <td> -748.10</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Covariance Type:</th>     <td>nonrobust</td>    <th>  LLR p-value:       </th>  <td>0.01848</td>\n",
       "</tr>\n",
       "</table>\n",
       "<table class=\"simpletable\">\n",
       "<tr>\n",
       "    <td></td>       <th>coef</th>     <th>std err</th>      <th>z</th>      <th>P>|z|</th>  <th>[0.025</th>    <th>0.975]</th>  \n",
       "</tr>\n",
       "<tr>\n",
       "  <th>const</th> <td>    0.2212</td> <td>    0.061</td> <td>    3.599</td> <td> 0.000</td> <td>    0.101</td> <td>    0.342</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Lag1</th>  <td>   -0.0387</td> <td>    0.026</td> <td>   -1.477</td> <td> 0.140</td> <td>   -0.090</td> <td>    0.013</td>\n",
       "</tr>\n",
       "<tr>\n",
       "  <th>Lag2</th>  <td>    0.0602</td> <td>    0.027</td> <td>    2.270</td> <td> 0.023</td> <td>    0.008</td> <td>    0.112</td>\n",
       "</tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                           Logit Regression Results                           \n",
       "==============================================================================\n",
       "Dep. Variable:          Direction_0_1   No. Observations:                 1089\n",
       "Model:                          Logit   Df Residuals:                     1086\n",
       "Method:                           MLE   Df Model:                            2\n",
       "Date:                Mon, 13 Apr 2020   Pseudo R-squ.:                0.005335\n",
       "Time:                        21:04:07   Log-Likelihood:                -744.11\n",
       "converged:                       True   LL-Null:                       -748.10\n",
       "Covariance Type:            nonrobust   LLR p-value:                   0.01848\n",
       "==============================================================================\n",
       "                 coef    std err          z      P>|z|      [0.025      0.975]\n",
       "------------------------------------------------------------------------------\n",
       "const          0.2212      0.061      3.599      0.000       0.101       0.342\n",
       "Lag1          -0.0387      0.026     -1.477      0.140      -0.090       0.013\n",
       "Lag2           0.0602      0.027      2.270      0.023       0.008       0.112\n",
       "==============================================================================\n",
       "\"\"\""
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#7.(a). Logistic Regression using all the observation.\n",
    "url = \"https://raw.githubusercontent.com/atystats/ISLR_Notes_codes/master/Datasets/Weekly.csv\"\n",
    "Weekly = pd.read_csv(url, error_bad_lines = False)\n",
    "Weekly[\"Direction_0_1\"] = 0\n",
    "Weekly.loc[Weekly['Direction'] == \"Up\", \"Direction_0_1\"] = 1\n",
    "\n",
    "X = Weekly[['Lag1', 'Lag2']]\n",
    "y = Weekly['Direction_0_1']\n",
    "X = sm.add_constant(X)\n",
    "\n",
    "clf = sm.Logit(y, X).fit()\n",
    "clf.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Optimization terminated successfully.\n",
      "         Current function value: 0.683147\n",
      "         Iterations 4\n",
      "0    0.545523\n",
      "dtype: float64\n",
      "Actual Direction :  0\n"
     ]
    }
   ],
   "source": [
    "# 7.(b) and (c).\n",
    "X_train = Weekly[['Lag1', 'Lag2']].iloc[1:]\n",
    "y_train = Weekly['Direction_0_1'].iloc[1:]\n",
    "X_train = sm.add_constant(X_train)\n",
    "\n",
    "X_test = Weekly[['Lag1', 'Lag2']].iloc[:1]\n",
    "y_test = Weekly['Direction_0_1'].iloc[1]\n",
    "X_test['const'] = 1\n",
    "clf = sm.Logit(y_train, X_train).fit()\n",
    "print(clf.predict(X_test))\n",
    "print(\"Actual Direction : \",y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the prediction is 0.54 which is greater than 0.5 which means the direction will be Up. However the actual direction is 'Down'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 241,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.44995408631772266"
      ]
     },
     "execution_count": 241,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 7(d).\n",
    "prediction = []\n",
    "for i in range(len(Weekly)):\n",
    "    X_train = np.array(Weekly[['Lag1', 'Lag2']].drop(index = i))\n",
    "    y_train = np.array(Weekly['Direction_0_1'].drop(index = i))\n",
    "    \n",
    "    X_test = np.array(Weekly[Weekly.index == i][['Lag1','Lag2']])\n",
    "    y_test = np.array(Weekly[Weekly.index == i]['Direction_0_1'])\n",
    "    \n",
    "    clf = LogisticRegression()\n",
    "    clf.fit(X_train,y_train)\n",
    "    if clf.predict(X_test) == 1:\n",
    "        prediction.append(\"Up\")\n",
    "    else:\n",
    "        prediction.append(\"Down\")\n",
    "(prediction != Weekly['Direction']).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "LOOCV Estimate of error is 44.95%."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 252,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcgAAAEHCAYAAAAqH9aLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAeqklEQVR4nO3df5Ac5X3n8c8HaeEWwnlxkA1aEHAxEQFjpGNLFxd1uUDAwtgGgc2Bc+Xjzj7rchVSZ86lMgq+BOecsmzl4FKHY1uxqTgxZUOBLEggFhC4IqYC9soSCBkUYzCgXRILw4INC6yW7/2xM2I02z3TO9sz3bP9flWp2OnpmX6mmZnPPE8/PxwRAgAABzqo6AIAAFBGBCQAAAkISAAAEhCQAAAkICABAEhAQAIAkGBx0QVIYvtYSX8p6ShJb0jaFBF/2rTPb0q6VdKTtU2bI+KPWj3vkUceGccff3zu5QUA9K9t27Y9FxFLmreXMiAl7ZP0yYj4ge3DJW2zfVdE/LBpv7+PiPdnfdLjjz9eo6OjuRYUANDfbD+VtL2UTawR8WxE/KD2988lPSppuNhSAQCqpJQB2cj28ZJWSnow4e53237I9t/aPiXl8Wttj9oe3bt3bxdLCgBYSEodkLZ/SdItkj4RES813f0DScdFxGmS/q+kLUnPERGbImIkIkaWLJnVxAwAQKLSBqTtAc2E4w0Rsbn5/oh4KSJ+Ufv7DkkDto/scTEBAAtUKQPStiV9TdKjEXFNyj5H1faT7VWaeS0/610pAQALWVl7sZ4h6SOSdtreUdv2+5KWSVJEfFnShyT9N9v7JE1KujRYmgToii3bx7Rx626NT0xq6dCg1q1erjUr6TeHha2UARkR35XkNvtcJ+m63pQIqK4t28e0fvNOTU5NS5LGJia1fvNOSSIksaCVsokVQHls3Lp7fzjWTU5Na+PW3QWVCOgNAhJAS+MTk3PaDiwUpWxiBZCu19cDlw4NaiwhDJcODXbtmEAZUIME+kj9euDYxKRCb14P3LJ9rGvHXLd6uQYHFh2wbXBgkdatXt61YxZhy/YxnbHhHp1w5e06Y8M9XT2n6A/UIIE+0up6YB61yFa104Xci5WOSEhCQAIl0yqkunk9sF1ILOSg6PYPD/QnmliBEmnXhJp23S+P64FV7q1KRyQkISCBEmkXUt28HljlkOjmDw/0LwISKJF2IbVm5bA+d9GpGh4alCUNDw3qcxedmqkZsF0nlCqHRFU6ImFuuAYJlEiWIRWdXA/M0gll3erlB+wj9SYkej1spaodkTB3BCRQIt0KqSydUIoIiV73Hq1yRyTMHQEJZNSLmk63QirP64t5node9x6d6/GYpL3aCEggg17WdLpRk8nSdJvlNSbtc8WNO/SJG3douIMAyRLceYbUXH4oMDYSdNIBMuj3IRBZOqFkeY1J+9TXmOtkVp92HYPynjko7XhDhw7M6sDU7//PMX8EJJBBEUMg8pz6LEvv1yyvsd3rbQyQLOVvF9x5h1TS8QYWWb94dd+sEE6qcUvVGPaCGTSxAhnMZ8LuTpoIu9G8167pNstrTNun0fjEZObyt7vmmvcPk6TjvfzaPk1MTh2w3+TUtBbZmk5Yg70Kw14wgxokkEGn4+Q6bSLspOY03xpnlteYtE+zpUODcyr/mpXDuv/Ks/Tkhvfp/ivPOiBA8xybWT8/V9y4Q5J07SUrdP+VZ+nFpnCsm45gbGTFUYMEMui0d2mnvTTnWnPKo8bZ6jU21oKHDh3QIYsP0sTklKw3r0FKbwZIPYSylj/NfIe91Ms9NjF5QFkbz09arbje6YherNVFQAIZddK7tNMmwrk26eY1XCLpNTaH7wuvTGlwYJH+zyUr9h+7OUDqodSq/Fmanucz7KW53M2NpfXzkxTCA4usl1/bpytu3KGlQ4O69pIVBGMF0cQKdFGnTYRzbdLtZieiVuE7+tTz+qcXX1VI+qcXX9XoU89nKn+Wpue0JtGsQZVU7mbjE5OzOjAdceiAFNLE5FTP1txEORGQQBd1eu0y65yr9RCZ3ZVkRh4dStJCdmxiUt944On9HVmmI/SNB57Wp7fsbFv+dtcokwL0iht36NNbds673I3q56fxOuihBy/W1BsHnlGGd1RTaZtYbZ8r6U8lLZL01YjY0HT/IZL+UtLpkn4m6ZKI+Emvywm0Mp8mwnZNus1NiM3y6lCSpedqo28++Iw+u+bUluVvFbpnbLhHL7+2L3G85Q0PPK2R494667po0nltV+6081PlVU1woFIGpO1Fkr4o6RxJeyR93/ZtEfHDht0+JumFiHiH7UslfV7SJb0vLdBat+b4bNWE2MmsNmmSrtE1d85plDQ0olmr8GoVaiHtr8l1Mvl6vdytzs98hvRgYSllQEpaJenxiHhCkmx/S9IFkhoD8gJJV9f+vlnSdbYdkeHTCZRQY43oLYMDsqWJV6ZSa51pNRpLuv/Ks3IrV1ItuF2N8owN97QM6KTwymp8YrKrk68XtaoJyqesATks6ZmG23sk/Zu0fSJin+0XJf2ypOcad7K9VtJaSVq2bFm3ygvMS3NzaePA9bQhG92u6bRqwjxjwz0tQ7Je5tGnnte9j+2d9RyN4TWX5ltJ+69JJmn+0dBJ7T0pWM88aYk2bt29v1crwz2qoayddJywrblmmGUfRcSmiBiJiJElS5bkUjggb+16XCZ1EunmIr/teplmmTBgcmpaNzzwdOpz1DvGDKcE+mEHL0r8kLeS14+Dxk4761Yv1y3bxnKbDxb9o6wBuUfSsQ23j5E0nraP7cWS3iLp+Z6UDshZlg4gSbWjLD1dO9Gul2nzsdOkjT1slBb0f3zhqbr2khWpAdqsW82gTFpeXWVtYv2+pBNtnyBpTNKlkn67aZ/bJF0m6R8kfUjSPVx/RFZ5LqGUx3MNHTqgF15JnvKsLql21K0OQFl6cjYeu12Ta6vnbnetcM3KYZ1w5e2pnYIsdbXZk16t1VXKgKxdU7xc0lbNDPO4PiJ22f4jSaMRcZukr0n6K9uPa6bmeGlxJUY/yXMi8Dyea8v2Mf3i1X0t9+l1J5G5Xt9ct3q51t38kKams/VgbdbpROrDQ4O5dkiay7Hp1brwlbWJVRFxR0T8akT8SkT8cW3bH9TCURHxakRcHBHviIhV9R6vQDt5Npnl8Vwbt+6eNTBdkmzNq+l0PpOXd3R9s+klHKSZKdvm9Bx5licnRR4bxSplDRLopryazLZsH8tlzcDUfUN6csP7MpXjM3+9a38T7dDggN5/2tG6ZdtYxzXbuQ6RSAr5NyT9y4MX67BDFmtsYlKL7FnXMbOaz4QL81XksVEsAhKVk0eTWb1ptdUx5lue0My1vTNPWpI4VKJejuamzYnJKX3jgadnPV/Wycubr6lmmag7LeRfnJzS1eefkkuTdreut5b92ChOaZtYgW7Jo8ms1bCMuT5XqyET9flO04YYbNy6O9N1v7p2NdtO169sNSk7vUDRr6hBonI6bTJrrFm1iqS5Xi+c66D5xprgXJuF29Vs08Ls6tt2tTxfrWafyWttyH6XZ89p9AYBiUqaa5NZu4nB64aHBjv60quXp9VwhkZjE5Na8Zk7ZUtpg5vSFjNuJS20Jian9s/uk9RE2upHR+P10UZDhw60LMtCkmfPafQOAQlkkGVtwTx6Ns5l5YzG6eiaDRxkXbLq2NRrl/M9ftL1zLQfHWkBPt9Ry/1UI7v6tl25LGiN3iIggQxaNQfmNVB9y/Yxvfxa6/GQWQwNDujq80/JVJbmkDnzpCUH9H5tJWsT6YspQZ62PYte1MjyCuAt28dSf8xUrZm53xCQQAbdHqie1oR7xKEDet+7jta9j+3NVLOzpB1/+J6Ojjk2Malbto3pg6cPH1DzfOX1fYlNpFl76nZjoH2W1TzmI88AbtUZickGyo1erOgr8xn8Ph9nnrRk1pyjeQ4WT2vCPfTgxfrsmlNbTurdaC5fuGkhc+9je/dP1H3/lWfpDz9wyrx6/XZjoH2rBZfzeE/k2fO21Q8bJhsoNwISfaPTIQh5HPeWbWMHdHixpA+ent/YuCyTF7RbQWOuoZN1woT5TorejUnVW/0QyOM9kef8q4ucPJ37QaaDTtnRxIq+0e1mtbkcNyR988FnNHLcW3M5dpZmyOaeolkWVZ7vMRuPPZ/XmfdA+1YLLufxnsizWXg6pTdSwuyCKBkCEn2jqFUV0p5/OkLrbn5IUnJNYC6dPLKuYp9n0GQ9ZhnVz8EnujTGMs9zM9zi+jXKjSZW9I1Ws7Xkqfk651sG08frTU2HPvPXuxKfYy7Nwd1c2zFNEcfM05qVw6khM9/3RJ7nhsnO+5ertITiyMhIjI6OFl0MdCipp+fgwKJcv9TTepMepJnJt9P8pGlS8bT1EXuxPFOV9OI9kYek1gSJCdDLwva2iBhp3k4TK/pC/Qtmcmpai2xNR2i4C18qab1JW4Vjkjybg/tpQHyvdWOljcbzPd/rvI3lbHwcM+v0BwISpdf8ZTIdsb+JKu8vk3YTAiS1twwlNMHm1cmDL9L28rwu23y+Gwf453nui+pwhrnhGiRKr5erQbQKsNDMFG6NBg6yrj7/lFn75nXdiZUweqvdlIJ5nfuiOpxhbghIlF4vv0zWrV4+a0KAuuGhQW28+LT9HUMW2Zp6I7Rx6+5ZnW/y6uTBF2lvZTmveZz7XnU4w/zQxIrS68ZUZWnWrBzW6FPP64YHnk5cCaMecFmaPfNo+uvla0e2ydrzOPf9PMSmSqhBovR63U3+s2tO1bWXrEit/aU1e37ypodyn9WHIQK9lfdsRWn6fYhNVTDMA32hTD05W63Z2I0hBmV67VXQjV6sKLe0YR6lC0jbGyV9QNLrkn4s6T9HxETCfj+R9HNJ05L2Jb24ZgQk8pA2xrGOsY5Af0kLyDI2sd4l6Z0R8S5J/yhpfYt9z4yIFVnCEchLu2Y4OtAAC0PpOulExJ0NNx+Q9KGiyoLyKFMzY/24n7zpocSJqOlAg2Zlev8iu9IFZJOPSrox5b6QdKftkPSViNiUtJPttZLWStKyZcu6Ukh0V16D5fP8kkrqzSrRgQazMdlD/yrkGqTtuyUdlXDXVRFxa22fqySNSLooEgppe2lEjNt+m2aaZX8vIu5rdVyuQfanTuc1be5s8fLr+zQ1/eZbyZL+w68v02fXnJr6uHZBSs0A7RQ9Ly/v0fZKNRdrRJzd6n7bl0l6v6TfSgrH2nOM1/77U9vflrRKUsuARH/qZLB8qynD6kLSDQ88fcCajnP9tZ/3OodYeIqc7IHa6/yUrpOO7XMlfUrS+RHxSso+h9k+vP63pPdIeqR3pUQvdTLrSLspw+qitm+rxzG1G+ajyFlzeD/PT+kCUtJ1kg6XdJftHba/LM00qdq+o7bP2yV91/ZDkr4n6faI+E4xxUW3dTJYfi6/zhv3ZWo35K2Xkz00r2WaNhyJ93M2peukExHvSNk+Lum82t9PSDqtl+VCcTpZ0ijLlGGN+7Z7HD1T0aluLMmVJKk5NW0FGt7P2ZQuIIEkc73WlzTX5UGW3mj6tmj+Jc8cmeiGXlyrTmpODc1epo33c3YEJBaktF/tSduaJxhvtw9QRmnNpqGZHrO8n+eudFPNdRPDPACU2XyGZBQ9nKSf9dNUcwBQOfVriGMTkwq9OSQj6woxrPySPwISAEpgvkMyWEIrf1yDBIASyGOIERNX5IsaJACUQJETCiAZAQkAJcA1xPKhiRUASoAhRuVDQAJASRRxDZHVPtIRkABQUaz20RrXIAGgoljtozUCEgAqitVrWiMgAaCiGFrSGgEJABXF0JLW6KQDABXF0JLWCEgAqDCmp0tHEysAAAmoQaJjvRpgzEBmAEUgINGRXg0wZiAzgKLQxIqO9GqAMQOZARSldAFp+2rbY7Z31P6dl7LfubZ3237c9pW9LmfV9WqAMQOZARSldAFZc21ErKj9u6P5TtuLJH1R0nslnSzpw7ZP7nUhq6xXA4wZyAygKGUNyHZWSXo8Ip6IiNclfUvSBQWXqVJ6NcCYgcwAilLWgLzc9sO2r7d9RML9w5Keabi9p7ZtFttrbY/aHt27d283ylpJa1YO63MXnarhoUFZ0vDQoD530am5d5zp1XEAoJkjovcHte+WdFTCXVdJekDSc5JC0v+SdHREfLTp8RdLWh0R/6V2+yOSVkXE77U67sjISIyOjubwCgAAC4XtbREx0ry9kGEeEXF2lv1s/7mkv0m4a4+kYxtuHyNpPIeiAUDlMfZ4RunGQdo+OiKerd28UNIjCbt9X9KJtk+QNCbpUkm/3aMiAsCCtGX7mK6+bZcmJqf2b6vy2OMyXoP8gu2dth+WdKakKyTJ9lLbd0hSROyTdLmkrZIelXRTROwqqsAA0O/qk3I0hmNdVccel64GGREfSdk+Lum8htt3SJo1BAQAMHdJk3I0quLY4zLWIAEAPdYuAKs49piABAC0DMCqjj0mIAEAiZNySNIRhw5Uduxx6a5Bon/RNRzoX/XPatJneMv2MZ2x4Z7KfbYJSOSCZamA/rdm5fCsz2uVP9s0sSIXLEsFLExV/mwTkMgFy1IBC1OVP9sEJHLBslTAwlTlz3ZqQNq+w/bxvSsK+hnLUgELU5U/26066fyFpDttf13SFyJi9vxDQE2rHnCdolcsULxufLb7RcvlrmwfJukPJJ0r6a8kvVG/LyKu6XrpcsZyV/2jueecNPOrtarjsQB0T9pyV+2uQU5JelnSIZIOb/oHdE2Ve84BKIfUJlbb50q6RtJtkv51RLzSs1Kh8qrccw5AObS6BnmVpItZRgpFWDo0qLGEMKxCzzkA5ZDaxBoR/5ZwRFGq3HMOQDkw1RxKqco95wCUAwGJ0kqaFxLAwtdqiFcvh38RkACA0mg1Obqknk6cTkACAEqj3RCvtPsISADAgtbJEK9uDf8qXUDavlFSvavikKSJiFiRsN9PJP1c0rSkfUmzIAAA+ku7IV69HP5VutU8IuKSiFhRC8VbJG1usfuZtX0JRwBYAFoN8er18K/S1SDrbFvSv5d0VtFlAQD0RpYhXr3qxdpysvIi2f4NSdek1Q5tPynpBUkh6SsRsSllv7WS1krSsmXLTn/qqae6VGIAQD9Km6y8kBqk7bslHZVw11URcWvt7w9L+maLpzkjIsZtv03SXbYfi4j7mneqBecmaWY1j3kWHQBQEYUEZESc3ep+24slXSTp9BbPMV77709tf1vSKkmzAhIAgE6UrpNOzdmSHouIPUl32j7M9uH1vyW9R9IjPSwfAGCBK2tAXqqm5lXbS23fUbv5dknftf2QpO9Juj0ivtPjMgIAFrBS9mKNiP+UsG1c0nm1v5+QdFqPiwUAqJCy1iABACgUAQkAQAICEgCABAQkAAAJCEgAABIQkAAAJCAgAQBIQEACAJCAgAQAIAEBCQBAAgISAIAEBCQAAAkISAAAEhCQAAAkICABAEhAQAIAkICABAAgAQEJAEACAhIAgAQEJAAACQhIAAASFBaQti+2vcv2G7ZHmu5bb/tx27ttr055/Am2H7T9I9s32j64NyUHAFRBkTXIRyRdJOm+xo22T5Z0qaRTJJ0r6c9sL0p4/OclXRsRJ0p6QdLHultcAECVFBaQEfFoROxOuOsCSd+KiNci4klJj0ta1biDbUs6S9LNtU1fl7Smm+UFAFRLGa9BDkt6puH2ntq2Rr8saSIi9rXYR5Jke63tUduje/fuzb2wAICFaXE3n9z23ZKOSrjrqoi4Ne1hCduig31mNkZskrRJkkZGRhL3AQCgWVcDMiLO7uBheyQd23D7GEnjTfs8J2nI9uJaLTJpHwAAOlbGJtbbJF1q+xDbJ0g6UdL3GneIiJB0r6QP1TZdJimtRgoAwJwVOczjQtt7JL1b0u22t0pSROySdJOkH0r6jqTfjYjp2mPusL209hSfkvQ/bD+umWuSX+v1awAALFyeqYxVw8jISIyOjhZdDABAidjeFhEjzdvL2MQKAEDhCEgAABIQkAAAJCAgAQBI0NVxkMhmy/Yxbdy6W+MTk1o6NKh1q5drzcrEiYEAAD1CQBZsy/Yxrd+8U5NT05KksYlJrd+8U5IISQAoEE2sBdu4dff+cKybnJrWxq1J87gDAHqFgCzY+MTknLYDAHqDgCzY0qHBOW0HAPQGAVmwdauXa3DgwPWgBwcWad3q5QWVCAAg0UmncPWOOPRiBYByISBLYM3KYQIRAEqGJlYAABIQkAAAJKCJtSDMngMA5UZAFoDZcwCg/GhiLQCz5wBA+RGQBWD2HAAoPwKyAMyeAwDlR0AWgNlzAKD86KRTAGbPAYDyKyQgbV8s6WpJvyZpVUSM1rafI2mDpIMlvS5pXUTck/D4qyV9XNLe2qbfj4g7ul/y/DB7DgCUW1E1yEckXSTpK03bn5P0gYgYt/1OSVslpaXItRHxJ10sIwCgwgoJyIh4VJJsN2/f3nBzl6R/YfuQiHith8UDAKDUnXQ+KGl7i3C83PbDtq+3fUTak9hea3vU9ujevXvTdgMA4ABdC0jbd9t+JOHfBRkee4qkz0v6rym7fEnSr0haIelZSf877bkiYlNEjETEyJIlSzp4JQCAKupaE2tEnN3J42wfI+nbkv5jRPw45bn/uWH/P5f0Nx0VEgCAFKVqYrU9JOl2Sesj4v4W+x3dcPNCzXT6AQAgN4UEpO0Lbe+R9G5Jt9veWrvrcknvkPQ/be+o/Xtb7TFftT1S2+8LtnfafljSmZKu6PVrAAAsbI6IosvQMyMjIzE6Olp0MQAAJWJ7W0SMNG8vVRMrAABlQUACAJCAgAQAIAEBCQBAAgISAIAEBCQAAAkISAAAEhCQAAAkICABAEhAQAIAkICABAAgAQEJAEACAhIAgAQEJAAACQhIAAASEJAAACQgIAEASLC46AL0ky3bx7Rx626NT0xq6dCg1q1erjUrh4suFgCgCwjIjLZsH9P6zTs1OTUtSRqbmNT6zTsliZAEgAWIJtaMNm7dvT8c6yanprVx6+6CSgQA6CYCMqPxick5bQcA9LdCAtL2xbZ32X7D9kjD9uNtT9reUfv35ZTHv9X2XbZ/VPvvEd0u89KhwTltBwD0t6JqkI9IukjSfQn3/TgiVtT+/U7K46+U9HcRcaKkv6vd7qp1q5drcGDRAdsGBxZp3erl3T40AKAAhXTSiYhHJcl2p09xgaTfrP39dUn/T9Kn5luuVuodcejFCgDVUMZerCfY3i7pJUmfjoi/T9jn7RHxrCRFxLO235b2ZLbXSlorScuWLZtXwdasHCYQAaAiuhaQtu+WdFTCXVdFxK0pD3tW0rKI+Jnt0yVtsX1KRLzUaTkiYpOkTZI0MjISnT4PAKBauhaQEXF2B495TdJrtb+32f6xpF+VNNq06z/bPrpWezxa0k/nXWAAABqUapiH7SW2F9X+/leSTpT0RMKut0m6rPb3ZZLSaqQAAHSkqGEeF9reI+ndkm63vbV2129Ietj2Q5JulvQ7EfF87TFfbRgSskHSObZ/JOmc2m0AAHLjiOpclhsZGYnR0ebWWgBAldneFhEjs7ZXKSBt75X0VNHlyOBISc8VXYg+xbnrHOeuc5y7zpXh3B0XEUuaN1YqIPuF7dGkXzNoj3PXOc5d5zh3nSvzuStVJx0AAMqCgAQAIAEBWU6bii5AH+PcdY5z1znOXedKe+64BgkAQAJqkAAAJCAgAQBIQECWlO2Nth+z/bDtb9seKrpM/SJtQW4ks32u7d22H7fd9bVVFxLb19v+qe1Hii5LP7F9rO17bT9a+6z+96LLlISALK+7JL0zIt4l6R8lrS+4PP2k1YLcaFCb+/iLkt4r6WRJH7Z9crGl6it/IencogvRh/ZJ+mRE/JqkX5f0u2V83xGQJRURd0bEvtrNByQdU2R5+klEPBoRu4suR59YJenxiHgiIl6X9C3NLEiODCLiPknPF12OfhMRz0bED2p//1zSo5JKt9guAdkfPirpb4suBBakYUnPNNzeoxJ+UWHhsn28pJWSHiy2JLN1bT1ItJdlUWnbV2mmOeKGXpat7DpckBuzOWEbY7/QE7Z/SdItkj4RES8VXZ5mBGSB2i0qbfsySe+X9FvBgNUDdLIgNxLtkXRsw+1jJI0XVBZUiO0BzYTjDRGxuejyJKGJtaRsnyvpU5LOj4hXii4PFqzvSzrR9gm2D5Z0qWYWJAe6xrYlfU3SoxFxTdHlSUNAltd1kg6XdJftHba/XHSB+kWLBbnRpNYR7HJJWzXTUeKmiNhVbKn6h+1vSvoHSctt77H9saLL1CfOkPQRSWfVvt922D6v6EI1Y6o5AAASUIMEACABAQkAQAICEgCABAQkAAAJCEgAABIQkEBF1FZQeNL2W2u3j6jdPq7osgFlREACFRERz0j6kqQNtU0bJG2KiKeKKxVQXoyDBCqkNr3XNknXS/q4pJW1VTwANGEuVqBCImLK9jpJ35H0HsIRSEcTK1A975X0rKR3Fl0QoMwISKBCbK+QdI5mVnG/wvbRBRcJKC0CEqiI2goKX9LM2ntPS9oo6U+KLRVQXgQkUB0fl/R0RNxVu/1nkk6y/e8KLBNQWvRiBQAgATVIAAASEJAAACQgIAEASEBAAgCQgIAEACABAQkAQAICEgCABP8fuKYv/Z2fY9UAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 504x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 8.(a) and (b).\n",
    "np.random.seed(1)\n",
    "x = np.random.normal(loc =0, scale = 1, size = 100)\n",
    "y = np.random.normal(loc =0, scale = 1, size = 100)\n",
    "e = np.random.normal(loc =0, scale = 1, size = 100)\n",
    "y = x - 2 * x**2 + e\n",
    "plt.figure(figsize = (7,4))\n",
    "plt.scatter(x,y)\n",
    "plt.xlabel(\"X\")\n",
    "plt.ylabel(\"Y\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "value of n is 100. value of p is 2 as there are 2 independent variables. Also, the relationship between X and Y is non linear (quadratic)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 254,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[7.974329481603451, 0.9824020475291425, 0.9817568247235399, 1.0161707331941572]"
      ]
     },
     "execution_count": 254,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 8(c). Fitting different polynomial\n",
    "model = LinearRegression()\n",
    "loocv = LeaveOneOut()\n",
    "mse = make_scorer(mean_squared_error)\n",
    "X = x.reshape(-1,1)\n",
    "MSE = []\n",
    "for i in range(4):\n",
    "    polynomial_features= PolynomialFeatures(degree=i+1)\n",
    "    xp = polynomial_features.fit_transform(X)\n",
    "    results = cross_val_score(model, xp, y, cv=loocv, scoring = mse)\n",
    "    MSE.append(results.mean())\n",
    "MSE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 257,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[8.661954812490729, 0.8587491068345234, 0.8753206150150602, 0.902724291180355]"
      ]
     },
     "execution_count": 257,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 8(d). Choosing a different random seed\n",
    "np.random.seed(5)\n",
    "x = np.random.normal(loc =0, scale = 1, size = 100)\n",
    "y = np.random.normal(loc =0, scale = 1, size = 100)\n",
    "e = np.random.normal(loc =0, scale = 1, size = 100)\n",
    "y = x - 2 * x**2 + e\n",
    "model = LinearRegression()\n",
    "loocv = LeaveOneOut()\n",
    "mse = make_scorer(mean_squared_error)\n",
    "X = x.reshape(-1,1)\n",
    "MSE = []\n",
    "for i in range(4):\n",
    "    polynomial_features= PolynomialFeatures(degree=i+1)\n",
    "    xp = polynomial_features.fit_transform(X)\n",
    "    results = cross_val_score(model, xp, y, cv=loocv, scoring = mse)\n",
    "    MSE.append(results.mean())\n",
    "MSE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "8(d). We have observed that the different random seed results in different datasets and hence the errors are different because we only have control over reducible error but not irreducible error.\n",
    "\n",
    "8(e). Also, we have the lowest error when degree of polynomial is 2, which is expected as in the actual model y has a quadratic relationship with x."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 258,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "22.532806324110698"
      ]
     },
     "execution_count": 258,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 9. Boston Dataset\n",
    "url = \"https://raw.githubusercontent.com/atystats/ISLR_Notes_codes/master/Datasets/Boston.csv\"\n",
    "Boston = pd.read_csv(url, error_bad_lines = False)\n",
    "# 9(a).\n",
    "Boston[\"medv\"].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 262,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.4088611474975351"
      ]
     },
     "execution_count": 262,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 9(b). standard error of mu.\n",
    "Boston['medv'].std()/np.sqrt(len(Boston))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 265,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "original :-  22.5328\n",
      "Bootsrap Standard Error :-  0.4031\n"
     ]
    }
   ],
   "source": [
    "# 9(c). Bootstrap Error\n",
    "def mean_medv(data):\n",
    "    return data['medv'].mean()\n",
    "\n",
    "def boot(data, statistic, R):\n",
    "    original = statistic(data)\n",
    "    stats_results = []\n",
    "    for i in range(R):\n",
    "        random_indices = list(np.random.choice(len(data), len(data)))\n",
    "        stats_results.append(statistic(data.iloc[random_indices]))\n",
    "    std_error = np.std(stats_results, axis = 0)\n",
    "    print(\"original :- \", np.round(original,decimals = 4))\n",
    "    print(\"Bootsrap Standard Error :- \", np.round(std_error,decimals = 4))\n",
    "    \n",
    "boot(Boston, mean_medv, 1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 266,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(21.726600,23.339000)\n"
     ]
    }
   ],
   "source": [
    "# 9(d). 95% CI\n",
    "print(\"(%f,%f)\"%(22.5328 - 2 * 0.4031, 22.5328 + 2 * 0.4031))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 272,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(21.729528014578616, 23.33608463364278)"
      ]
     },
     "execution_count": 272,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# CI based on t distribution\n",
    "def conf_interval(data_series):\n",
    "\n",
    "    mean = np.mean(data_series)\n",
    "\n",
    "    conf_int = stats.sem(data_series) * stats.t.ppf((1 + 0.95) / 2, len(data_series) - 1)\n",
    "\n",
    "    start = mean - conf_int\n",
    "    end = mean + conf_int\n",
    "\n",
    "    return start, end \n",
    "conf_interval(Boston.medv)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 273,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "21.2"
      ]
     },
     "execution_count": 273,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 9(e). Median\n",
    "Boston['medv'].median()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 274,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "original :-  21.2\n",
      "Bootsrap Standard Error :-  0.3789\n"
     ]
    }
   ],
   "source": [
    "# 9(f). Bootstrap standard error of median\n",
    "def median_medv(data):\n",
    "    return data['medv'].median()\n",
    "\n",
    "def boot(data, statistic, R):\n",
    "    original = statistic(data)\n",
    "    stats_results = []\n",
    "    for i in range(R):\n",
    "        random_indices = list(np.random.choice(len(data), len(data)))\n",
    "        stats_results.append(statistic(data.iloc[random_indices]))\n",
    "    std_error = np.std(stats_results, axis = 0)\n",
    "    print(\"original :- \", np.round(original,decimals = 4))\n",
    "    print(\"Bootsrap Standard Error :- \", np.round(std_error,decimals = 4))\n",
    "    \n",
    "boot(Boston, median_medv, 1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 276,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "12.75"
      ]
     },
     "execution_count": 276,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 9(g). 10th percentile \n",
    "np.percentile(Boston['medv'],10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 277,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "original :-  12.75\n",
      "Bootsrap Standard Error :-  0.5185\n"
     ]
    }
   ],
   "source": [
    "# 9(h). Bootstrap standard error of percentile.\n",
    "def perc_medv(data):\n",
    "    return np.percentile(data['medv'],10)\n",
    "\n",
    "def boot(data, statistic, R):\n",
    "    original = statistic(data)\n",
    "    stats_results = []\n",
    "    for i in range(R):\n",
    "        random_indices = list(np.random.choice(len(data), len(data)))\n",
    "        stats_results.append(statistic(data.iloc[random_indices]))\n",
    "    std_error = np.std(stats_results, axis = 0)\n",
    "    print(\"original :- \", np.round(original,decimals = 4))\n",
    "    print(\"Bootsrap Standard Error :- \", np.round(std_error,decimals = 4))\n",
    "    \n",
    "boot(Boston, perc_medv, 1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}