cs236756_tutorial_08_linear_regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# <img src=\"https://img.icons8.com/dusk/64/000000/mind-map.png\" style=\"height:50px;display:inline\"> CS 236756 - Technion - Intro to Machine Learning\n",
    "---\n",
    "#### Tal Daniel\n",
    "\n",
    "## Tutorial 08 - Linear Regression\n",
    "---\n",
    "<img src=\"./assets/tut_08_intro.jpg\" style=\"height:200px\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### <img src=\"https://img.icons8.com/bubbles/50/000000/checklist.png\" style=\"height:50px;display:inline\"> Agenda\n",
    "---\n",
    "* [The Regression Problem](#-The-Regression-Problem)\n",
    "    * [Classification vs. Regression](#-Classification-vs.-Regression)\n",
    "* [Linear Regression](#-Linear-Regression)\n",
    "    * [Residual Analysis](#-Residual-Analysis)\n",
    "    * [The Least Squares Criterion & Solution](#-Closed-Form-Leaset-Squares-Solution)   \n",
    "* [Examples](#-Example---Polynomial-Fitting-(Polynomial-Regression))\n",
    "    * [Polynomial Fitting](#-Example---Polynomial-Fitting-(Polynomial-Regression))\n",
    "    * [Cosine Fitting](#-Example---Cosine-Fitting)\n",
    "* [Regularized Least Squares](#-Regularization)\n",
    "    * [Bias-Variance Tradeoff](#-The-Bias-Variance-Tradeoff)\n",
    "    * [Lasso vs. Ridge Regression](#-Ridge-Regression)\n",
    "* [Recommended Videos](#-Recommended-Videos)\n",
    "* [Credits](#-Credits)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "# imports for the tutorial\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib notebook"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## <img src=\"https://img.icons8.com/nolan/96/000000/polyline.png\" style=\"height:50px;display:inline\"> The Regression Problem\n",
    "---\n",
    "### <img src=\"https://img.icons8.com/cotton/64/000000/fencing-swords.png\" style=\"height:50px;display:inline\"> Classification vs. Regression\n",
    "---\n",
    "* **Classification** is about predicting a label and **Regression** is about predicting a quantity."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* Predictive modeling is the problem of developing a model using historical data to make a prediction on new data.\n",
    "    * Classification predictive modeling is the task of approximating a mapping function (f) from input variables (X) to *discrete* (categories, labels) output variables (y).\n",
    "        * A classification algorithm may predict a continuous value, but the continuous value is in the form of a probability for a class label.\n",
    "    * Regression predictive modeling is the task of approximating a mapping function (f) from input variables (X) to a *continuous* output variable (y).\n",
    "        * A regression algorithm may predict a discrete value, but the discrete value in the form of an integer quantity.\n",
    "* In some cases, it is possible to convert a regression problem to a classification problem. For example, the quantity to be predicted could be converted into discrete buckets (e.g. class 1: 0-50, class 2: 51-100)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "|  | <center>Classification</center> | <center>Regression</center> \n",
    "| --- | --- | --- |\n",
    "|<center> Input </center>| <center> Feature vector $X$ </center>| <center>Feature vector $X$</center>\n",
    "|<center> Output </center>| <center> Label/Category - **Discrete** </center>| <center>Value - **Continuous**</center>\n",
    "|<center> MLE </center>| <center> $\\hat{\\theta} = \\underset{\\theta \\in \\mathcal{R}^{p}}{\\mathrm{argmax}} \\log p(D \\mid \\theta)$</center>| <center>$\\hat{\\theta} = \\underset{\\theta \\in \\mathcal{R}^{p}}{\\mathrm{argmax}} \\log p(D\\mid \\theta)$</center>\n",
    "|<center> Appropriate Models </center>| <center> Decision Trees, SVM, Neural Networks... </center>| <center>Decision Trees, Linear Regression Models, SVM, Neural Networks...</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### <img src=\"https://img.icons8.com/dusk/64/000000/classroom.png\" style=\"height:50px;display:inline\"> Example - House Price Model\n",
    "---\n",
    "Given the following dataset:\n",
    "\n",
    "| <center>Square Feet (X)</center> | <center>House Price in \\$1000 (y)</center> |\n",
    "| --- | --- |\n",
    "|<center> 1400 </center>| <center> 245 </center>|\n",
    "|<center> 1600 </center>| <center> 312 </center>|\n",
    "|<center> 1700 </center>| <center> 279 </center>|\n",
    "|<center> 1875 </center>| <center> 308 </center>|\n",
    "|<center> 1100 </center>| <center> 199 </center>|\n",
    "|<center> 1550 </center>| <center> 219 </center>|\n",
    "|<center> 2350 </center>| <center> 405 </center>|\n",
    "|<center> 2450 </center>| <center> 324 </center>|\n",
    "|<center> 1425 </center>| <center> 319 </center>|\n",
    "|<center> 1700 </center>| <center> 255 </center>|"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* We wish to predict the price of a house given its size\n",
    "* More formally:\n",
    "    * Given a house of size $x$, what is the price $y=f(x)$ of the house?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "def plot_possible_approx():\n",
    "    x = [1400, 1600, 1700, 1875, 1100, 1550, 2350, 2450, 1425, 1700]\n",
    "    y = [245, 312, 279, 308, 199, 219, 405, 324, 319, 255]\n",
    "    ap_x_1 = [1100, 2350]\n",
    "    ap_y_1 = [219, 405]\n",
    "    ap_x_2 = [1400, 1425, 1550, 1875, 2500]\n",
    "    ap_y_2 = [245, 310, 219, 308, 360]\n",
    "    fig = plt.figure(figsize=(8,5))\n",
    "    ax = fig.add_subplot(111)\n",
    "    ax.scatter(x, y, label=\"original data\", color='b')\n",
    "    ax.plot(ap_x_1, ap_y_1, label='approximation 1', color='k')\n",
    "    ax.plot(ap_x_2, ap_y_2, label='approximation 2', color='r')\n",
    "    ax.set_xlabel(\"Square Feet\")\n",
    "    ax.set_ylabel(\"House Price in $1000\")\n",
    "    ax.set_title(\"House Price vs. House Size\")\n",
    "    ax.grid()\n",
    "    ax.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfQAAAFNCAYAAAD2E503AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeZzM9R/A8dfHkTtEpISOX45F2G5yltxXRJGraBflTI7KUZISkisKlfsMOSqyjqicuZXKIrewdln2eP/++MyuxR6zx+zszLyfj8c8Zud7zfszs7z38/l+DiMiKKWUUsqzZXJ3AEoppZRKPU3oSimllBfQhK6UUkp5AU3oSimllBfQhK6UUkp5AU3oSimllBfQhK6UFzHGPG2MOejuOHyFMSbUGHO/u+NQCjShKx9mjDlsjHnmpm3tjTEb3RXTzRzxRDkSR4gxZqcxpkFCx4vIBhEpmZ4xppQxRowxD960bbAxZoa7YrqZMSafMWaqMeakMeaSMeYPY8xbMftFJLeI/O3OGJWKoQldqYxvs4jkBvIBXwLzjDF33HyQMSZLukfm/UYDuYHSQF6gEfCXWyNSKgGa0JVKhDGmtDEmyBhzwRiz1xjTKM6+IGPMq3Fex9bujTXaGHPaGHPRGLPLGFPWsS+bMWakMeaIMeaUMWaSMSZHUrGISDQwFcgB3G+MqW6MOWaMecsYcxKYFrMtTkz3GmMWGWPOGGPOGWPGxdnX0Riz3xhz3hjzvTGmeAKfwSpjTLebtv1ujGmWWDnTgjHmKWPMFse1txhjnoqz74YWlri1e2NMdmPMDEeZLzjOLezYl9cY86Ux5oQx5l9jzPvGmMwJhPAoMEtEzotItIgcEJEFcd5TjDEPGmPudrSixDwuG2MkznFOfdZKpYYmdKUSYIzJCiwDfgAKAa8DM40xzjRp1waqAg9ha9YtgXOOfSMc2ysADwL3AO86EU8W4FUgFPjTsfku4A6gOND5puMzA98BwUAJx/vMcexrAgwAmgF3AhuA2Qm89SzgxTjXLeN4v+VJlDNVHK0Qy4GxQAFgFLDcGFPAidPbYWvU9zrODQCuOPZ9BURiP/uKjjK8Gs81AH4BhhljOhhj/pfQm4nIcUfze25Ha8piUvZZK5VyIqIPffjkAziMTY4X4jwuAxsd+58GTgKZ4pwzGxjs+DkIeDXOvvZxzq0J/AE8cdP5BggDHoiz7UngnwRibI9NPheAs9gE84xjX3XgGpA9zvHVgWNxrnsGyBLPdVcCr8R5nclR9uLxHJvHEXNxx+thwNTEyunk5y9AyE2ffzgww7H/ZeC3m87ZDLSP8/09E2ff4DjndgQ2AeVvOr8wcBXIEWfbi8DaBGLMgU3G24AI4BBQ96YyPHjTOW85js+R3M9aH/pIzUNr6MrXNRGRfDEPoEucfXcDR8U2dccIxtZ0EyUiPwHjgPHAKWPMZGPM7dgaWk5gm6Mp+AKwyrE9Ib844isoIk+IyOo4+86ISHgC590LBItIZDz7igOfxonhP+wfG7eUTUQuYWvKrRybWgEzkyinsyrd9Pl/GGff3djPOy6nPn/gG+B7YI4x5rgx5iNHi0txICtwIk7ZP8e2wNxCRK6IyAci4o+t6c8D5sfXhwHAGFMX6I79vYppEXD6s1YqNTShK5Ww48C9xpi4/06KAf86fg7DJucYd8U9WUTGOhKBH7ZJ+k1sLfsK4BcnkeUV20ybEoktl3gUKJZAZ7mjwGtxk6mI5BCRTQlcazbwojHmSWytdW1sAPGXMy0cxybDuJz6/EUkQkSGiEgZ4CmgAdAWW+6rQME45b5dRPySCkZEQoAPgFzAfTfvd9yK+Qp4QUSOxtmV3M9aqRTRhK5Uwn7FJo2+xpisxpjqQEMc90aBnUAzY0xOY4dfvRJzojHmUWPM445aYRi2KTnKUdufAow2xhRyHHuPMeY5F8T/G3AC+NAYk8vRUayyY98koL8xxs8RQ15jTItErrUCm1yHAnNjWi0SKmcaxb8CeMgY85IxJosxpiVQBtsvAOzn38rx3TwCNI850RhTwxhTztGPIATbXB4lIiewfSI+McbcbozJZIx5wBhTLb4AjDHvOMp4mzEmO7b2fQE4eNNxtwNLgLdF5OZhj8n9rJVKEU3oSiVARK5hhynVxdasJwBtReSA45DR2HvYp7A1s5lxTr8dm7jPY5uJzwEjHfvewt6L/cUYEwKsBtJ87LiIRGH/AHkQOAIcw3ZaQ0QWYzvnzXHEsMdRzoSudRVYBDyD7SQXI8FyGmMGGGNWpiL+c9iadW/HdfsCDUTkrOOQd4AHHO895Ka47gIWYJP5fmAdEDO+vS1wG7DPce4CoEhCYQDTsN//ceBZoL6IhN50XCXsdzgqbm93RzmS9VkrlVJGJLEWO6WUUkp5Aq2hK6WUUl5AE7pSSinlBTShK6WUUl5AE7pSSinlBTShK6WUUl7Ao1dnKliwoJQoUcLdYaRaWFgYuXLlcncYLqfl9C6+UE5fKCNoOT3Ntm3bzorILbNLenRCL1GiBFu3bnV3GKkWFBRE9erV3R2Gy2k5vYsvlNMXyghaTk9jjLl5SmRAm9yVUkopr6AJXSmllPICmtCVUkopL+DR99DjExERwbFjxwgPT2hFyYwnb9687N+/391huFxKy5k9e3aKFi1K1qxZXRCVUkp5B69L6MeOHSNPnjyUKFECY4y7w3HKpUuXyJMnj7vDcLmUlFNEOHfuHMeOHeO++25ZsVIppZSD1zW5h4eHU6BAAY9J5ipxxhgKFCjgUS0uSinlDl6X0AFN5l5Gv0+llEqaVyZ0ZS1dupQPP/wwTa41ZswYLl++HPu6Xr16XLhwIdXXXb9+PZUqVSJLliwsWLAg1ddTSnmvmTOhRAnIlMk+z5zp7ogyFk3oHiIyMjLZ5zRq1Ih+/fqlyfvfnNBXrFhBvnz5Un3dYsWKMX36dF566aVUX0sp5b1mzoTOnSE4GETsc+fOmtTj0oTuAk2aNMHf3x8/Pz8mT54cuz137tz07t2bSpUqUatWLc6cOQPY2m6PHj146qmnKFu2LL/99hsAgwcPpnPnztSuXZu2bdsSHh5Ohw4dKFeuHBUrVmTt2rUAjBo1io4dOwKwe/duypYty+XLl5k+fTrdunUDoH379gQGBlKjRg3uv/9+1q1bR8eOHSldujTt27ePjTEwMJBHHnkEPz8/Bg0aBMDYsWM5fvw4NWrUoEaNGoCdpe/s2bOx71+2bFnKli3LmDFjADh8+DClS5emU6dO+Pn5Ubt2ba5cuXLLZ1WiRAnKly9Ppkz6q6iUStjAgWDrFOuBnwD7euBAd0aVsej/oi4wdepUtm3bxtatWxk7diznzp0D7DzClSpVYvv27VSrVo0hQ4bEnhMWFsamTZuYMGFCbHIG2LZtG0uWLGHWrFmMHz8esEl79uzZtGvXjvDwcHr06MGhQ4dYvHgxHTp04PPPPydnzpy3xHX+/Hl++uknRo8eTcOGDenZsyd79+5l9+7d7Ny5E4Bhw4axdetWdu3axbp169i1axdvvPEGd999N2vXro39IyJufNOmTePXX3/ll19+YcqUKezYsQOAP//8k65du7J3717y5cvHkiVL0vaDVkr5jODgM0AHoBowNHb7kSPuiijj8bpha3H16NEjNlGllQoVKsTWQhMyduxYFi9eDMDRo0f5888/KVCgAJkyZaJly5YAtGnThmbNmsWe8+KLLwJQtWpVQkJCYu9PN2rUiBw5cgCwceNGXn/9dQBKlSpF8eLF+eOPPyhfvjzTp0+nfPnyvPbaa1SuXDneuBo2bIgxhnLlylG4cGHKlSsHgJ+fH4cPH6ZChQrMmzePyZMnExkZyYkTJ9i3bx/ly5dPsKwbN26kadOmsQseNGvWjA0bNtCoUSPuu+8+KlSoAIC/vz9H9F+eUiqZoqOjmTp1KpkyvUV0dAjQD3gndn+xYm4LLcPx6oTuDkFBQaxevZrNmzeTM2dOqlevnuCQq7i9t2/uyR3zOu7KQCKS4Pv++eef5M6dm+PHjyd4TLZs2QDIlClT7M8xryMjI/nnn38YOXIkW7ZsIX/+/LRv3z7J4WKJxRT3PTJnzpyifgBKKd+1e/duAgIC2LRpE6VKVeXw4QmEh/vF7s+ZE4YNc2OAGYzLE7oxJjOwFfhXRBoYY+4A5gIlgMPACyJy3nFsf+AVIAp4Q0S+T817J1WTdoWLFy+SP39+cubMyYEDB/jll19i90VHR7NgwQJatWrFrFmzqFKlSuy+uXPnUqNGDTZu3EjevHnJmzfvLdeuWrUqM2fOpGbNmvzxxx8cOXKEkiVLcvHiRbp378769evp1q0bCxYsoHnz5smOPSQkhFy5cpE3b15OnTrFypUrY1cmypMnD5cuXaJgwYK3xNS+fXv69euHiLB48WK++eabZL+3UkrFCA0NZciQIYwePZr8+fMzffp02rZty6xZhoEDbTN7sWI2mbdu7e5oM470qKF3B/YDtzte9wPWiMiHxph+jtdvGWPKAK0AP+BuYLUx5iERiUqHGNNMnTp1mDRpEuXLl6dkyZI88cQTsfty5crF3r178ff3J2/evMydOzd2X/78+XnqqacICQlh6tSp8V67S5cuBAQEUK5cObJkycL06dPJli0bgYGBdOnShYceeogvv/ySGjVqULVq1WTH/vDDD1OxYkX8/Py4//77b2i679y5M3Xr1qVIkSI33EevVKkS7du357HHHgPg1VdfpWLFihw+fNip99yyZQtNmzbl/PnzLFu2jEGDBrF3795kx66U8nwiwpIlS3jjjTc4evQonTp1Yvjw4RQoUACwyVsTeCJExGUPoCiwBqgJfOfYdhAo4vi5CHDQ8XN/oH+cc78Hnkzs+v7+/nKzffv23bIto8iVK1e826tUqSJbtmxJ52jSX0hISIrPzcjf683Wrl3r7hDShS+U0xfKKJIxynn48GFp2LChAFKuXDn5+eef0/w9MkI50wKwVeLJia6uoY8B+gJxJ/AuLCInHH9MnDDGFHJsvwf4Jc5xxxzbbmCM6Qx0BihcuDBBQUE37M+bNy+XLl1Kq/jTXHyxiQhhYWEZOu60EBUVleIyhoeH3/JdZ1ShoaEeE2tq+EI5faGM4N5yRkZGMn/+fL7++mvADp1t1qwZ165dS/OYvP37dFlCN8Y0AE6LyDZjTHVnToln2y09rkRkMjAZ4JFHHpGYe7wx9u/fn2EXOgkNDY13+8qVKzNszGkpNYvQZM+enYoVK6ZxRK4RFBTEzb+X3sgXyukLZQT3lXPDhg1069aNvXv30qRJEz799FOKubDburd/n64ch14ZaGSMOQzMAWoaY2YAp4wxRQAcz6cdxx8D7o1zflEg4S7bSimlPNLZs2fp2LEjVatWJTQ0lKVLl7J48WKXJnNf4LKELiL9RaSoiJTAdnb7SUTaAEuBdo7D2gExs40sBVoZY7IZY+4D/gf85qr4lFJKpa/o6Gi+/PJLSpYsyTfffEO/fv3Yu3cvDRs2dHdoXsEd49A/BOYZY14BjgAtAERkrzFmHrAPiAS6iof1cFdKKRW/uGPKn376aSZOnIifn1/SJyqnpUtCF5EgIMjx8zmgVgLHDQN0mgCllPISYWFhDBkyhFGjRpEvXz6mTZtGu3btdFlkF9C53L2YJyyfOmrUKMqUKUP58uWpVasWwcHBqb6mUipjWLJkCWXKlOHjjz+mQ4cOHDx4kPbt22sydxFN6B7CW5dPrVixYuxiMM2bN6dv376pvqZSyr2Cg4Np3LgxTZo04fbbb2fjxo1MmTIldoIY5Rqa0F1Al091fvnUGjVqxK4M98QTT3Ds2LG0+RKUUukuIiKCjz76iDJlyrB69Wo+/vhjtm/fnuCCUSqNxTfbjKc8MupMcefOnRMRkcuXL4ufn5+cPXtWREQAmTFjhoiIDBkyRLp27Soidqa4V199VURE1q1bJ35+fiIiMmjQIKlUqZJcvnxZRERGjhwp7du3FxGR/fv3y7333itXrlyRqKgoefrpp2XRokXi7+8vGzduFBGRadOmxb5Hu3btpGXLlhIdHS3ffvut5MmTR3bt2iVRUVFSqVIl2bFjxw2xR0ZGSrVq1eT3338XEZHixYvLmTNnYssY83rr1q1StmxZCQ0NlUuXLkmZMmVk+/bt8s8//0jmzJljr9uiRQuZPHlyop9b165d5b333ot3X0b4Xp3lLbNRJcUXyukLZRRJm3KuX79e/Pz8BJAmTZpIcHBw6gNLY97yfeKmmeLcq0cPSOPlU6lQAXT51FhptXzqjBkz2Lp1K+vWrUv0s1VKZSxnz57lrbfeYurUqRQrVowlS5bQqFEjd4flk7w7obuBLp966/tB4sunrl69mmHDhrFu3bobzlFKZVzR0dFMnz6dvn37cvHiRd566y3eeeedG/7PUunLuxO6Lp+aLO5YPnXHjh289tprrFq1ikKFCiV9glLK7fbs2UNgYCAbN26kSpUqTJw4kbJly7o7LJ/n3QndDXT51OQtn/rmm28SGhpKixYtAChWrBhLly5NduxKKdcLCwtj6NChjBo1irx58zJ16lTatWtHpkzavzpDiO/Guqc8MmqnuITo8qm6fKo38YVy+kIZRZwr55IlS6RYsWICyCuvvBLb2deTeMv3SQKd4vTPKqWUUgk6cuQITZo0oXHjxrFjyr/44gsdU54BaZN7Okpo+dQVK1b4xPKpSinPERERwZgxYxg8eDAAH330ET169CBr1qzuDUwlSBO6UkqpG/z8888EBASwZ88eGjduzKeffkrx4sXdHZZKgja5K6WUAuDcuXO8+uqrVKlShZCQEJYsWcK3336rydxDaEJXSikfFx0dzbRp0yhZsiRfffUVffv2Zd++fTpBjIfRJnellPJhe/bsoUePHuzevZvKlSszadIkHVPuobSG7kbOLEH67rvvsnr16hRdPygoiAYNGiR5XPXq1dm6dWuix9y82ppSyrOFhYXRr18/KlasSHBwMF9++SXr16/XZO7BtIbuBjFjBlesWAHApUuXEjx26NCh6RVWosaMGUObNm1iV0ZTSnmuZcuW8frrrxMcHEzHjh1p1KgRjRs3dndYKpV8voY+cyaUKAGZMtnnmTNTf83ElhPt0qULlSpV4ujRozcsQfree+9RqlQpnn32WV588UVGjhwJ2GVPFyxYANglSwcNGkSlSpUoV64cBw4cAOC3337jqaeeomLFijz11FMcPHgw0fiuXLlCq1atKF++PC1btrxhWVNnl0+N7zilVMZ25MgRmjZtSqNGjcidOzcbNmzgyy+/jHeqaeV5fLqGPnMmdO4MMS3JwcH2NUDr1im75rZt25g2bRq//vorIsLjjz9OtWrVyJ8/PwcPHmTatGlMmDDhhnO2b9/OwoUL2bFjB5GRkVSqVAl/f/94r1+wYEG2b9/OhAkTGDlyJF988QWlSpVi/fr1ZMmShdWrVzNgwAAWLlyYYIwTJ04kZ86c7Nq1i127dlGpUqXYfcOGDeOOO+4gKiqKWrVqsWvXLt544w1GjRrF2rVrY+dyj++4xFZlU0q5T0REBJ9++mnsH98jRoygZ8+eOqbcy/h0DX3gwOvJPMbly3Z7SsVdTjR37tyxy4kCFC9e/Ia53WNs3ryZxo0bkyNHDvLkyUPDhg0TvH7Mkqv+/v6x86VfvHiRFi1aULZsWXr27MnevXsTjXH9+vW0adMGgPLly9+QiOfNm0elSpWoWLEie/fuZd++ffFew9njlFLu9fPPP1OpUiXefPNNatWqxb59++jbt68mcy/k0wk9oeW5E1m2O0mSyHKiCS0rmNg5N4tZXjTucqTvvPMONWrUYM+ePSxbtizJJU/h1uVagdjlU9esWcOuXbuoX79+vNdy9jillPucO3eOTp06UaVKFS5evMi3337L0qVLdUy5F/PphF6sWPK2O6Nq1ap8++23XL58mbCwMBYvXszTTz+d6DlPPvlkbCIODQ1l+fLlyXrPixcvcs899wAwffp0p2Kc6egssGfPHnbt2gXEv3xqjJjlU5M6TinlXiLC9OnTKVWqFNOmTePNN99k37592unNB/j0PfRhw268hw6QM6fdnlIpWU7U39+fRo0a8fDDD1O8eHEeeeSRZHVS6du3L+3atWPUqFHUrFkzyeMDAwPp0KED5cuXp0KFCrGxJmf51ISOU0q5z969ewkMDGTDhg1UrlyZiRMnUq5cOXeHpdJLfEuwecojLZZPnTFDpHhxEWPs84wZyTo9TYSEhMilS5dERCQsLEz8/f1l27Zt6R+Ii+nyqd7FF8rpKWUMCwuTfv36SZYsWeSOO+6QL774QqKiopw+31PKmVreUk4SWD7Vp2voYHuzp7RHe1rq3Lkz+/btIzw8nHbt2t3Q81wppRLy3Xff0a1bN4KDg+nQoQMfffRR7GgU5Vt8PqFnFLNmzXJ3CEopD3L06FG6d+/O4sWLKVOmDOvXr0+yv47ybj7dKU4ppTxNREQEn3zyCaVLl2bVqlV8+OGH7NixQ5O58s4auojEOyxLeSZJxrA+pbzZpk2bCAgIYPfu3TRo0IDPPvuMEiVKuDsslUF4XQ09e/bsnDt3TpOAlxARzp07R/bs2d0dilJuEzOmvHLlyly4cIHFixezdOlSTebqBl5XQy9atCjHjh3jzJkz7g7FaeHh4T6RsFJazuzZs1O0aFEXRKRUxiYifP311/Tp04fz58/Tp08fBg0aRO7cud0dmsqAvC6hZ82alfvuu8/dYSRLUFAQFStWdHcYLucr5VQqLezbt4/AwEDWr1/PU089xcSJE3W9BJUor2tyV0opT3b58mUGDBjAww8/zO7du5kyZQobNmzQZK6S5HU1dKWU8lTLly+nW7duHD58mPbt2/PRRx9x5513ujss5SG0hq6UUm529OhRmjVrRoMGDciZMyfr1q1j2rRpmsxVsmhCV0opN7l5TPnw4cPZsWMHVatWdXdoygNpk7tSSrnB5s2bCQgIiF2CeNy4cToMTaWK1tCVUiod/ffff3Tu3JmnnnqK//77j0WLFrFs2TJN5irVNKErpVQ6iBlTXqpUKaZOnUrv3r3Zv38/TZs21ZktVZrQhK6UUi62f/9+atSoQbt27XjwwQfZvn07I0eO1AliVJrShK6UUi4Sd0z5rl27mDx5Mhs3btQx5coltFOcUkq5QNwx5e3atePjjz/WYWjKpbSGrpRSaejYsWM8//zzNGjQgBw5chAUFMT06dM1mfuiv/6Cjz6CiIh0eTtN6EoplQYiIyMZPXo0pUuXZsWKFXzwwQfs3LmTatWquTs0lZ6OHYNRo+DRR+HBB+Gtt2DLlnR5a21yV0qpVPrll18ICAjg999/p169eowbN87jFolSqXD6NCxYAHPmwIYNdpu/P3z8MbzwAhQrli5haEJXSqkU+u+//+jfvz9TpkzhnnvuYeHChToMzVdcuACLFtkkvmYNREdDmTLw3nvQsiX873/pHpImdKWUSiYR4ZtvvqFPnz78999/9OzZk8GDB5MnTx53h6ZcKTQUli2zSXzlSntv/P77oX9/aNUKypZ1a3ia0JVSKhn2799Ply5dCAoK4oknnuDHH3/k4YcfdndYylXCw23ynjPHJvMrV+Cee+D1120Sf+QRyCAtMprQlVLKCZcvX2bYsGF8/PHH5M6dm88//5xXX32VTJm0b7HXiYiwzeizZ8O330JICNx5J3ToYJN45cqQAb93TehKKZWEFStW0K1bN/755x/atm3Lxx9/TKFChdwdlkpLUVG2Q9ucObaD27lzkDcvNG9uk3iNGpAlY6fMjB2dUkq50ZkzZ2jevDkLFy6kVKlSrF27lurVq7s7LJVWRODXX20SnzcPTpyAnDmhcWObxJ97DrJlc3eUTtOErpRSN4mMjOSzzz5j4MCBiAjDhg2jT58+3Hbbbe4OTaWWCOzaZZP4nDlw+LBN2vXq2SRevz7kyuXuKFPEZQndGJMdWA9kc7zPAhEZZIwZDHQCzjgOHSAiKxzn9AdeAaKAN0Tke1fFp5RS8Yk7pvzxxx9n1qxZ3H///e4OS6XWwYMU/+orCAyEAwcgc2Z49lkYPBiaNLHN6x7OlTX0q0BNEQk1xmQFNhpjVjr2jRaRkXEPNsaUAVoBfsDdwGpjzEMiEuXCGJVSCoDz58/Tv39/Jk+ezN13382CBQu44447NJl7ssOHYe5cWxPfuZMSxkC1atCjBzz/PBQs6O4I05TLErqICBDqeJnV8ZBETmkMzBGRq8A/xphDwGPAZlfFqJRSIsLMmTPp3bs3586do0ePHgwZMoQ8efIQFBTk7vBUcp04AfPn2yS+2ZE+nngCxoxh8z338FTz5u6Nz4WMzbsuurgxmYFtwIPAeBF5y9Hk3h4IAbYCvUXkvDFmHPCLiMxwnPslsFJEFtx0zc5AZ4DChQv7z5kzx2Xxp5fQ0FCfWBdZy+ldvKGcR44cYcyYMezYsYPSpUvTq1cvHnzwwdj93lBGZ3h6ObNcvMid69dTaO1a8u3ciREh9IEHOF2zJqdr1CC8SBHA88sZo0aNGttE5JFbdoiIyx9APmAtUBYoDGTGLgwzDJjqOGY80CbOOV8Czyd2XX9/f/EGa9eudXcI6ULL6V08uZyXL1+WgQMHStasWSVfvnwyadIkiYqKuuU4Ty5jcnhkOS9eFPn6a5G6dUWyZBEBkYceEhk0SGTfvnhP8chyxgPYKvHkxHTp5S4iF4wxQUAdiXPv3BgzBfjO8fIYcG+c04oCx9MjPqWU71i5ciXdunXj77//5uWXX2bkyJE6ptxTXL4My5fb5vTly+HqVSheHHr3tj3UH344w8za5g6u7OV+JxDhSOY5gGeAEcaYIiJywnFYU2CP4+elwCxjzChsp7j/Ab+5Kj6llG/5999/6dGjBwsWLKBkyZL89NNP1KhRw91hqaRcuwbff2+T+JIlEBYGd90Fr71mk/gTT/h0Eo/LlTX0IsBXjvvomYB5IvKdMeYbY0wFbAe5w8BrACKy1xgzD9gHRAJdRXu4K6VSKTIykvHjx/P2228TGRnJ+++/T58+fcjmQROG+JzISAgKskl84UK7stkdd0Dr1jaJV61qh52pG7iyl/suoGI8219O5AG3P2oAACAASURBVJxh2PvqSimVar/++isBAQHs3LmTunXrMm7cOB2GllFFR8OmTTaJz59v1xjPk8eOEX/xRXjmGcia1d1RZmhJJnRjzHNAE+AebK36OLBERFa5ODallEqR8+fPM2DAAD7//HOKFCnCggULaNasma5TntGIwPbtdhGUuXPh2DHInh0aNrQ18bp1IUcOd0fpMRJN6MaYMcBDwNfYTmtgO6u9YYypKyLdXRyfUko5TeKMKT979izdu3dn6NChuk55RrN37/WpVw8dsjXvOnVgxAibzPX7SpGkauj1ROShmzcaY+YCfwCa0JVSGcKBAwfo0qULa9eu5bHHHmPVqlVUrHjLXT/lLocOXZ+1bc8eu/xozZrQvz80bQr587s7Qo+XVEIPN8Y8JiI39zZ/FAh3UUxKKeW0K1eu8MEHHzBixAhy5crFxIkT6dSpE5m105T7HTtmVzGbPRu2brXbqlSBcePssqSFC7s3Pi+TVEJvD0w0xuThepP7vdhZ3tq7LiyllEraqlWr6Nq1K3///Tdt2rRh5MiRFNYk4V6nT9v1xOfMseuLAzzyCIwcCS+8APfem/j5KsUSTegish143BhzF7ZTnAGOicjJ9AhOKaXi8++//9KzZ0/mz59PyZIlWbNmDTVr1nR3WL7r/HlYvNgm8TVrbI/1MmXgvfds57Y40+kq13Gml7sBinO9l3tmY8wpx/RzSimVbmLGlL/zzjtERETomHJ3Cg2FZctsEl+5EiIi4IEH7D3xVq2gbFl3R+hzkurlXhuYAPwJ/OvYXBR40BjTRUR+cHF8SikFwG+//UZAQAA7duygTp06jBs3jgceeMDdYfmW8HCbvOfMscn8yhUoWhTeeMMmcX9/nbXNjZKqoX8KPCMih+NuNMbcB6wASrsoLqWUAuDChQsMGDCASZMmUaRIEebNm0fz5s11THl6iYiA1attEl+8GC5dgjvvhA4dbBKvXNn2WFdul1RCz8L1znBx/Ytd31wppVxCRJg1axa9e/fmzJkzvPHGGwwdOpTbb7/d3aF5v6go26Ftzhzbwe3cOcibF1q0sEm8Rg3Iki5re6lkSOobmQpsMcbMAY46tt0LtMIub6qUUmnu4MGDdOnShZ9++onHHnuMlStX6phyVxOBX3+1SXzePDhxAnLlgsaNbRKvXRu0r0KGllQv9+HGmCVAI+BJHL3cgdYisi8d4lNK+ZArV64wfPhwRowYQY4cOXRMuauJwK5d3D95sm1CP3zYJu169WwSr1/fJnXlEZJsM3Ekbk3eSimX+v777+natSt//fUXrVu35pNPPtEx5a5y8OD1qVcPHODeTJnguedgyBBbI8+b190RqhRIqpd7XqA/dnGWOx2bTwNLgA9F5IJrw1NKebvjx4/Ts2dP5s2bx0MPPcTq1aupVauWu8PyPocPX596dedO2xu9WjXo2ZNNhQtTuXFjd0eoUimpronzgPNAdREpICIFgBrABWC+q4NTSnmvqKgoxo4dS6lSpViyZAlDhw5l165dmszT0okT8Omn8OSTcN990K+fXc1szBg7LevatdC5MxFaI/cKSTW5lxCREXE3OGaJ+9AY08F1YSmlvNmWLVsICAhg+/btPPfcc4wbN44HdTaxtHHuHCxcaGviQUH2PnmFCvDhh3bq1fvuc3eEykWSSujBxpi+wFcicgrAGFMYO4/70cROVEqpm124cIGBAwcyceJE7rrrLh1TnlZCQuDbb20S//FHiIyEkiXh3Xdt57ZSpdwdoUoHSSX0lkA/YJ0xppBj2ylgKfCCKwNTSnkPEWH27Nn06tVLx5SnlcuXYflyu5LZihVw9SoULw69e9sk/vDDOmubj0lq2Np54C3HQymlku2PP/6gS5curFmzhkcffZQVK1ZQqVIld4flma5ehR9+sDXxJUsgLAzuugsCAmwSf/xxTeI+LMVT/RhjOojItLQMRinlPcLDwxk+fDgffvghOXLkYPz48bz22ms6pjy5IiNt57U5c2DRIrhwAe64A1q3tkm8alXQz1SRioQODAE0oSulbvHDDz/QtWtXDh06xEsvvcQnn3zCXXfd5e6wPEd0NGzaZJP4/Pl2jfE8eaBpU5vEn3kGsurs2+pGSY1D35XQLkBnfFBK3eD48eP06tWLuXPn6pjy5BKBbdtsEp871w4ry5EDGja0SbxuXTvkTKkEJFVDLww8hx2LHpcBNrkkIqWUx4mKimLChAkMHDiQa9euMXToUPr27avrlDtjz57rs7b99ZetedepAyNG2GSeJ4+7I1QeIqmE/h2QW0R23rzDGBPkkoiUUh5l69atBAQEsG3bNmrXrs348eN1THlSDh26Pmvbnj12+dFatWDAANusnj+/uyNUDjNnwsCBcOQIFCsGw4bZ7gsZUVK93F9JZN9LaR+OUspThIaG0q1bNyZMmMBdd93FnDlzeOGFF3RMeUKOHrWrmM2ZA1u32m1VqsD48fD886Dz1mc4M2dC5852hCBAcLB9DRkzqSe7U5wx5jYgq4iEuSAepVQGJyLMmTOHbt26ceHCBbp168Z7771HXp0+9FanT9tObXPmwMaNdtsjj8DIkXbWtnvvdW98KlEDB15P5jEuX7bbPTKhG2O6A6tE5KAxphp2fvcoY8z7IjLB5REqpTKMP//8ky5durB69WpKlizJDz/8gL+/v7vDyljOn4fFi20SX7PG9lgvWxbefx9atgS9HeExjhxJ3nZ3c6aG3g4Y5/j5beza6HuAXwBN6Er5gPDwcD788EOGDx9O9uzZGT9+PCVLltRkHiM0FJYutUl81SqIiIAHHoD+/W0P9bJl3R2hSoFixWwze3zbM6Kkhq0NAu4GBjqa2itge73XAfIYY94FgkRkvcsjVUq5RUJjyoOCgtwdmnuFh8PKlTaJL1sGV65A0aLwxhs2ifv766xtHm7YsBvvoQPkzGm3Z0SJLp8qIkOA7djEfwewUESGAu8BJ0RkqCZzlRozZ0KJEraTb4kS9rXKGE6cOMGLL77Ic889hzGGH3/8kZkzZ/r2BDERETaJt2sHhQpBs2Z2RbOOHWHDBludGznS3ifXZO7xWreGyZPtFPnG2OfJkzPm/XNwrsn9FaAHcA74yLHtIeALVwWlfIOn9SD1FVFRUUycOJGBAwdy9epVBg8ezFtvvUV2X53UJCoK1q+3NfGFC+3ypPnyQYsWtiZeowZkSc2kmyoja93ac/4/SvK3UEROcNPiLCJyADjgqqCUb/C0HqS+IO6Y8meffZbx48fzv//9z91hpT8R+PVXHhw3Dl56CU6cgFy5oHFjm8Rr1wadNEdlMPpnpXIbT+tB6s0uXrzI22+/zfjx4ylcuLBvjikXgd9/vz5rW3Awd2fNen3q1fr17Q1UpTIoTejKbTytB6k3EhHmzp1Lz549OX36tG+OKT9w4HoSP3jQNp8/+ywMHcrPBQrwdP367o5QKadoQldu42k9SL3Nn3/+SdeuXfnxxx/x9/dn2bJlPPLII+4OK30cPnx96tWdO22Pp+rVoVcv29GtYEEAony9J7/yKE4ldGNMNuB5oETccxw93pVKkZj75J4yT7K3CA8PZ8SIEQwfPpxs2bIxbtw4AgICvH+d8uPHr8/a9ssvdtuTT8Knn0Lz5nD33e6NT6lUcraGvgS4CGwDrrouHOVrPKkHqTdYvXo1Xbp04c8//6RVq1aMGjWKIkWKuDss1zl7FhYtskk8KMjeJ69QAT780M7aVqKEuyNUKs04m9CLikgdl0ailIeLWZXp9dehffuM1dpw8uRJevXqxezZs3nwwQf54YcfePbZZ90dlmuEhMC339ok/uOPEBkJJUvCoEE2iZcq5e4IlXIJZxP6JmNMORHZ7dJolPJQGXVMfVRUFJMmTWLAgAGEh4d775jyy5fhu+9sEl+xAq5etbXvPn1sD/Xy5XWiF+X1nE3oVYD2xph/sE3uBhARKe+yyJTyIBlxTP22bdsICAhg69atPPPMM0yYMMG7xpRfvQo//GCT+JIlEBYGRYpAQIBN4o8/rklc+RRnE3pdl0ahlIfLSGPqL168yDvvvMP48eMpVKgQs2fPpmXLlt4xpjwyEtautUl80SK4cAEKFIA2bWwSf/pp8PbOfUolIKnFWW4XkRDgUjrFo3zRunV2ZaqiRd0dSYplhDH1IsK8efPo2bMnJ0+epGvXrrz//vueP6Y8Oho2bbJJfP58u8Z4njzQtCm8+CLUqgVZs7o7SqXcLtHFWYBZjudtwFbH87Y4r5VKvcaN7QpVHmzYsFsnEUvPMfWHDh2iTp06tGrVirvvvpvffvuNzz77zGOS+S2L9MwQ2LrV3gMvXtzWvKdOtWPFFy2ySf2rr6BOHU3mSjkkWkMXkQaO5/vSJxzlc0JD4eJFu4JVaCjkzu3uiFIk7ph6sDkoPXq5X716lREjRvDBBx+QLVs2PvvsMwIDAz1qTHncDoV+7KFV8ByeaDsH5C+brOvWhY8+slOweujvh1LpQWeKU+516pR9Dg+3vZRbtXJvPKkQM6Y+KMhOROZq3jKmPKZD4cf0oQ+fEEUm1kgtJhUYwMd/NoX8+d0dolIeIakmd6VcKyahAyxY4L44PMjJkydp3bo1zz77LNHR0Xz//ffMnj3bI5M52I6DPRhNHz5hMp0owgme4wc++a+jJnOlkkETunKvkyftc5UqdvxwWJh748nAoqKimDBhAqVKlWLBggUMGjSIPXv2ULt2bXeHliqBBeYxml4s4HkCmcgZCgG6SI9SyeV0QjfGVDHGdHD8fKcxRu+rq9SLSeivvw5XrsDy5e6NJ4Pavn07Tz75JF27duXRRx9l9+7dDB482PMniNmwgbEXXmZTpsq8zDdEY+/96yI9SiWfUwndGDMIeAvo79iUFZjhqqCUDzl50k7+0aQJ3HWXHZakYoWEhNC9e3ceffRRjhw5wqxZs/jhhx946KGH3B1a6u3fD40bk/mB+/h3/BIKF8+BMbZD4eTJGWfaXKU8hbOd4poCFYHtACJy3BiTx2VRKd9x8iTceSfcdhs8/7wdmhQWBrlyuTsytxIR5s+fT48ePTh58iRdunTh/fffJ1++fO4OLW2cOGF7r992G6xcSYv7CtAiwN1BKeXZnG1yvyYiAgiAMSbJ/22NMdmNMb8ZY343xuw1xgxxbL/DGPOjMeZPx3P+OOf0N8YcMsYcNMY8l5ICKQ9z6pStmQO0aGGb3VescG9MbvbXX39Rt25dWrZsSZEiRfj1118ZN26c9yTzS5egfn27Etry5XCf3r1TKi04m9DnGWM+B/IZYzoBq4EpSZxzFagpIg8DFYA6xpgngH7AGhH5H7DG8RpjTBmgFeAH1AEmGGM8ZzCtSpmTJ68n9CpVoHBhn212v3r1Ku+99x5+fn5s2rSJsWPH8ttvv/Hoo4+6O7S0ExFh/3Dbtct+z/7+7o5IKa/hVJO7iIw0xjwLhAAlgXdF5MckzhEg1PEyq+MhQGOgumP7V0AQ9v58Y2COiFwF/jHGHAIeAzYnozzK05w8eX05y8yZbbP79Ol2YPLNU695sZ9++onAwED++OMPWrZsyahRo7j77rvdHVbaEoHXXoPvv4cvvrBN7kqpNONsp7j7gA0i8qaI9AE2GmNKOHFeZmPMTuA08KOI/AoUFpETAI7nQo7D7wGOxjn9mGOb8lYiNqEXLnx9W4sWNpn7SLP7qVOnaNOmDbVq1SIqKorvv/+eOXPmeF8yBxgyBKZNg3ffhVdecXc0SnkdZzvFzQeeivM6yrEt0bZAEYkCKhhj8gGLjTFlEzk8vqWg5JaDjOkMdAYoXLgwQUFBiUfuAUJDQ72iHEm5uZxZQkOpcu0ah0JDORazPSqKp/Ln58L48ewrWNAtcaaWM99nVFQU3333HVOmTOHatWu0bduWl156idtuu81jfheS83t71/LllBo5khN16nCwenU7nZ4H8NV/m97K68spIkk+gJ3xbPvdmXPjHD8I6AMcBIo4thUBDjp+7g/0j3P898CTiV3T399fvMHatWvdHUK6uKWc+/eLgMjMmTduDwwUyZlTJCws3WJLS0l9n9u2bZNHH31UAKlVq5YcPHgwfQJLY07/3q5YIZI5s0jt2iLXrrk0prTms/82vZS3lBPYKvHkRGc7xZ0xxjSKeWGMaQycTewEx+Qz+Rw/5wCeAQ4AS4F2jsPaAUscPy8FWhljsjma+P8H/OZkfMoTxUwqE9MpLkZMs/vKlekfkwuFhITQo0cPHn30UYKDg5k5cyY//vijd4wpT8i2bfb7LFfOTu2rK6Mp5TLONrkHADONMeOwTeNHgbZJnFME+MrRUz0TME9EvjPGbMb2mn8FOAK0ABCRvcaYecA+IBLoKrbJXnmrhBJ61apQqJDtBf388+kfVxoTERYsWECPHj04ceIEgYGBDBs2zHuGoSXkn3/s8LQCBWyfiDw6dYVSruRsL/e/gCeMMbkBIyKXnDhnF3Yympu3nwNqJXDOMEAnfPQVCSX0zJmhWTP45hs7Lj1HjvSPLY389ddfdOvWjVWrVlGxYkUWL17MY4895u6wXO/cOduL/epVWLsWPHThGKU8SaJN7saYNo7nXsaYXtjOaJ3ivFYq5U6dsk2w8a2o1aKFnTHOQ5vdr169yvvvv0/ZsmX5+eef+fTTT/ntt998I5lfuQKNGtka+tKlULq0uyNSyickVUOPmRFO28pU2osZsmbiGeBQtaqdEnb+fFtb9yDbt28nICCAgwcP8sILLzB69GjvHIYWn6goaNMGNm2CefPg6afdHZFSPiPRhC4inzvugYeIyOh0ikn5irizxN0sSxabyGfM8Jhm91OnTtGnTx9mzJjB/fffz8qVK6lTp467w0o/ItCrFyxaBKNG2VYWpVS6SbKXu6NjWqOkjlMq2RJL6HC92X3VqvSLKQWio6OZNGkSpUqVYu7cubz88svs2bPHt5I5wOjRMHYs9OgBPXu6OxqlfI6zw9Y2GWPGGWOeNsZUinm4NDLl/ZJK6NWqQcGCGXpu9x07dvDkk08SGBhIpUqV2L17Nx07diSHB7QopKl586B3b2jeHD75xN3RKOWTnB22FjNL3NA42wSombbhKJ8RFQVnztw47evNYprdZ83KcM3uly5d4t1332Xs2LEULFiQGTNm8NJLL2GM4cSJE+4OL32tXw8vv2wX1/nmG8jkbD1BKZWWnB22VsPVgSgfc+6cTeqJ1dDBNrtPnmwX9GjSJH1iS4SIsHDhQrp3786JEycICAhg2LBh5I+vp74v2LcPGjeG+++HJUsge3Z3R6SUz0pq2NrjjvXMQ40xm40xOv5EpY2ExqDfrHr1DNPs/vfff1OvXj1atGhBoUKF+OWXX5gwYYLPJvPbzp61Y82zZ7fDC++4w90hKeXTkmobG4+df70AMAoY4/KIlG9wNqFnyQJNm8KyZRAe7vq44nH16lWGDRuGn58fGzduZMyYMWzZssU3xpQn5NIlyvXvb1tali+HEiXcHZFSPi+phJ5JRH4UkasiMh+4Mz2CUj7A2YQOttn90iXb7J7O1q5dy8MPP8zbb79Nw4YNOXDgAN27dydLFme7n3ihiAho3pzcf/9t52evpP1jlcoIkkro+YwxzWIe8bxWKmVOnbLPiXWKi1G9up0PPB2b3U+dOsXLL79MzZo1iYiIYOXKlcybN4977rkn3WLIkESgc2f44QcO9u4NvjY0T6kMLKlqxjqgYQKvBVjkiqCUDzh5EnLmhNy5kz42a1bbIW7ePNvs7sKOV9HR0UyZMoV+/foRFhbG22+/zYABA3xvGFpCBg+G6dNh0CBOVq9OKXfHo5SKldRMcR3SKxDlY2LGoMc37Wt8WrSAL7+EH36w84S7wM6dOwkICODXX3+lRo0aTJgwgVKlNGXF+uILGDoUOnSAQYNg3Tp3R6SUikMHjCr3SGpSmZvVrGl7Ubug2f3SpUv06tULf39//v77b7755hvWrFmjyTyuFSsgIACeew4+/9z5P8SUUulGE7pyj+Qm9Jhm96VL7ZKcaSBmTHnp0qUZM2YMnTt35uDBg7Rp0wajCeu6rVttC0n58vYPqqxZ3R2RUioemtCVe5w65VyHuLhatICQENvsnkp///039evXp3nz5tx5551s3ryZiRMn+uyY8gT9/TfUr29Xvlu+HPLowotKZVROJXRjTE5jzDvGmCmO1/8zxjRwbWjKa127ZscvJ6eGDlCrll07PRXN7nHHlG/YsIHRo0ezZcsWHn/88RRf02udO2cnjomIsBPHFCni7oiUUolwtoY+DbgKPOl4fQx43yURKe93+rR9Tm5Cj2l2X7IkRc3uQUFBVKhQgbfffpsGDRpw4MABevTo4dFjymfOtHO6ZMpkn2fOTKMLX7liOx8GB9vbHKV1kkilMjpnE/oDIvIREAEgIlcAvcmoUiY5k8rcLKbZ/ccfnT7l9OnTtG3blho1anD16lWWL1/O/PnzPX5M+cyZdkh4cLAdHh4cbF+nOqlHRUGbNrB5s12PvkqVNIlXKeVazib0a8aYHNix5xhjHsDW2JVKvtQk9Fq1IF8+p5rdo6OjmTx5MqVKlWLOnDkMHDiQPXv2UK9eveS/bwY0cCBcvnzjtsuX7fYUE4FevWDRIhg1yi6HqpTyCM62NQ4CVgH3GmNmApWB9q4KSnm55MwSd7PbbrPN7osX22b3bNniPez3338nICCAX375herVqzNhwgRKe1mz8ZEjydvulNGjYexY6NkTevRIxYWUUunNqRq6iPwINMMm8dnAIyIS5LqwlFeLqaGnJKGDbXa/eBFWr75lV9wx5X/99Rdff/01P/30k9clc4BixZK3PUlz50Lv3vbzHTkyxXEppdzD2V7ulYFwEVkO5AMGGGOKuzQy5b1OnrTN5imdwvWZZ25pdr95TPmrr77KwYMHefnll712TPmwYXb23Lhy5rTbk239emjb1t4v//pr28tOKeVRnP1XOxG4bIx5GHgTCAa+dllUyrsld1KZm912GzRubHu7X7vGP//8Q4MGDWjevDkFCxZk06ZNTJo0yevHlLduDZMnQ/HiduK24sXt69atk3mhffvs53n//fYzdeFc+Uop13E2oUeKiACNgbEi8imgM0yolEltQgfbLHzhAvM6d8bPz4/169czatQotm7dyhNPPJE2cXqA1q3h8GGIjrbPyU7mx4/bsebZs9ux5nfc4YIolVLpwdmEfskY0x9oAyw3xmQGdP5HlTKnTqU6oa/Plo2QTJkI++or6tWrx/79++nZs6dHjylPdyEhUK8e/Pefnau9RAl3R6SUSgVnE3pL7DC1V0TkJHAP8LHLolLe7eTJFHeIO336NO3ataPas8/yY86ctMmViwWzZlG0aNE0DtLLRUTYIWl79sCCBVCxorsjUkqlkrO93E+KyCgR2eB4fURE9B66Sr6wMLh0Kdk19LhjymfPns2AAQOoP306WcPCYM0aFwXrpUSgUyc7Oc+UKXYFNaWUx3OqfdIYcwnHpDLAbdjm9lARyeuqwJSXihmDnoyE/vvvvxMYGMjmzZupVq0aEydOtMPQrl6F22+3vd3r1nVRwF5o0CD46isYPNiuba6U8grO1tDziMjtjkd24HlgvGtDU14pGbPEhYaG0rt3b/z9/Tl06BBfffUVa9euvT6mPFs22zv7229tE7JK2pQp8N570LEjvPuuu6NRSqWhFA02FZFvgZppHIvyBU7U0EWExYsXU7p0aUaNGsUrr7zCgQMHaNu27a1jylu0gPPntdndGcuXQ2Ag1KkDkybZsW5KKa/hbJN7szgvMwGPcL0JXinnJTFL3D///MPrr7/O8uXLKV++PPPmzePJJ5+M91gAate+3uxep44LAvYSW7fCCy/Aww/bzyqrDlJRyts4W0NvGOfxHHAJOyZdqeQ5edLWDO+884bN165dY/jw4fj5+REUFMQnn3zCtm3bEk/mYJvdGzXSZvfE/P031K8PhQrZWnru3O6OSCnlAk7V0EVEe86otHHypE3mccaLr1u3jsDAQPbv30+zZs0YM2YM9957r/PXbNHCLvP500/aY/tm587ZDoMREXbimNRO6KOUyrCcncu9qDFmsTHmtDHmlDFmoTFGB/6q5IszS9yZM2do37491atX58qVK3z33XcsXLgweckcbLN7njxOLanqU65csa0XwcGwdCmUKuXuiJRSLuRsk/s0YClwN3ZSmWWObUolz6lTSOHCfPHFF5QsWZJZs2bRv39/9u7dS/369VN2zezZbeJavFib3WNERdl5YDdvhpkz7aIrSimv5mxCv1NEpolIpOMxHbgzqZOUutm1o0dZtWMHnTp1oly5cuzcuZMPPviAnDcvG5ZcLVrYKUzXrk2bQD2ZiF3PfPFiu77588+7OyKlVDpwNqGfNca0McZkdjzaAOdcGZjyLqGhoUycMIHo48c5FBbG9OnTCQoKokyZMmnzBs89p83uMUaNgs8+g169oHt3d0ejlEonzib0jsALwEngBNDcsU2pRMUdU/79/PlkB14ZMIB27dql7Trl2bNDw4ba7D5nDvTpY4eofazLLSjlS5ydKe6IiDQSkTtFpJCINBGRYFcHpzzb4cOHadSoEc2aNSN//vyMGzgQgJz33++aN2zRwvbqDgpyzfUzunXroF07ePppO7VrphTNG6WU8lCJDlszxnxGIhPIiMgbaR6R8njXrl1j9OjRDBkyhEyZMjFy5EjeeOMN9ox3zBbsqqFTzz1nx1jPnw/PPuua98io9u6FJk3ggQfsmPzs2d0dkVIqnSU1Dn1rnJ+HAINcGIvyAuvXrycwMJB9+/bRtGlTPv3009hhaLf99589KIVLpyYpR47rze4TJtww1t2rHT9ux5pnz27Hmt9xh7sjUkq5QaL/44nIVzE/G2N6xH2tVFxnzpyhb9++TJ8+neLFi7Ns2TIaNGhwwzGxCd2Vk5u0aAGzZ9tm92eecd379V4+NwAAGHdJREFUZBQhIVCvnp3Pfv16KF7c3REppdwkOTfZdO52dYvo6Gi++OILSpUqxYwZM+jXrx/79u27JZkD3Hb+vJ1DPH9+1wVUp871Zndvd+2aHZK2dy8sXAgVK7o7IqWUG2mvGZViu3fv5umnn6ZTp074+fmxc+dOhg8fnuCY8tv++882t7uys1aOHNCgASxaBJGRrnsfdxOBTp1g9Wq7JGrt2u6OSCnlZon+z2qMuWSMCTHGhADlY36O2Z5OMaoMJjQ0lDfffJOKFSvyxx9/MG3aNNatW4efn1+i5912/nz6zCXeogWcPWt7fXurd9+Fr7+GIUOgfXt3R6OUygASTegikkdEbnc8ssT5OY+I3J5eQaqMY8mSJZQpU4aRI0fSoUMHDhw4QPv27Z0aU37bf/+lT0KvWxdy5fLeZvfJk+H99+GVV5h5/zuUKGEbPUqUsLO8KqV8kza5K6cEBwfTqFEjmjRpQr58+di4cSNTpkyhQIECTl8jtsnd1by52f277yAwEOrWZdbTE+n8miE42LbABwdD586a1JXyVZrQVaIiIiIYMWIEZcqU4aeffmLkyJFs27aNypUrJ+9CUVHp1+QOttn9zBnb89tbbNkCLVtChQowbx4DBmXl8uUbD7l8GRzz9yilfIwmdJWgDRs2ULFiRfr160ft2rXZt28fvXv3JmvWrMm/2LlzmOjo9EvodetCzpze0+z+99+21aFQIVi+HHLn5siR+A9NaLtSyrtpQle3OHv2LB07dqRq1aqEhoaydOlSFi9eTLFixVJ+0VOn7HN6JfScOa83u0dFpc97usrZs3Y4XmQkrFoV+xkm9HWk5mtSSnkulyV0Y8y9xpi1xpj9xpi9xpjuju2DjTH/GmN2Oh714pzT3xhzyBhz0BjznKtiU/GLjo7myy+/pGTJknzzzTe89dZb7N27l4YNG6b+4idP2uf0Suhgm91Pn/bsZvcrV+xa70eOwNKlULJk7K5hw+zfLXHlzGm3K6V8jytr6JFAbxEpDTwBdDXGxKyVOVpEKjgeKwAc+1oBfkAdYIIxJrML41Nx7N69m6r/b+/Oo6sqzz2Ofx9l0BgUqEhdFoGrzFIhUFtBCDgXGdXcehsLFhCvbR16mctlLVdZLENAq9jaWzQCKlXbokxWrLWMIqAGBJKAs4BClaEo0Mr03D/2jhzSBALJOSdnn99nrbOy8+69z34fEvaT/Z536N6doUOHfj2mPC8vj7POOqt6LlCa0BPRKa5Ur16p3ex++DDk5sLKlUFPtzL9FnJzgw7vTZuCWfB12rSgXETST9wSurtvc/fCcPtLoAS44Din9AOedfev3P1D4D3gsnjVTwL79u1j1KhRZGVlsXHjxkqPKT9pyXhCz8iAG25IzWZ3d7j33mBe+oceCmaEK0duLnz0ERw5EnxVMhdJXwn5DN3MmgEdgVVh0c/MbJ2ZPWFmpfOAXgBsiTltK8f/A0CqqHRM+eTJkxk0aBCbNm2q9Jjyk7Z9O4fPOCOYljWRcnKCz++XLUvsdavqgQfg17+G4cPhbi1qKCInFvflqMwsE5gN3OvuX5jZb4EJBHPDTwAeAAYD5WWRf5s/3syGAcMAGjduzOIIrH29d+/ehMaxfft2HnnkEVasWEHz5s2ZOnUq7du3Z/369XG7Zpt168isX583Ejx722mZmXStW5ftDz3Euwm6ZlV/nuf97W+0nTCBz3r0oLhXrxq7vnuif2+TIR1iBMUZGe4etxdQG3gZ+J8K9jcDNoTbY4GxMfteBi4/3vt36tTJo2DRokUJuc6BAwd80qRJnpGR4RkZGZ6fn+8HDhxIyLX9qqv8H+3aJeZaZd18s3vjxu6HDiXkclX6eS5a5F6njnv37u7//Gd1VSkuEvV7m0zpEKO74kw1wJteTk6MZy93AwqAEnd/MKb8/JjDBgAbwu15wC1mVtfMmgMtgNXxql+6Wb58OVlZWYwePZprrrmGkpISRo4ceWpjyk/F9u0cSNY63aXN7suXJ+f6lVVUBP37w0UXwZw5wfrmIiKVFM/P0LsCPwKuLDNELd/M1pvZOqAn8HMAdy8C/gAUAwuBn7p7ivVkqnl27NjBkCFD6NatG1988QVz585lzpw5VRtTfpJmzYJdJdv55FCD5Mw3fsMNwXSwNbm3+yefHJ0M56WX4rvErIhEUtw+Q3f35ZT/ufifj3PORECjaKvBkSNHmDFjBqNGjWLPnj2MHj2a8ePHV98wtEqaNQt+cvtB9hzZSUm9hl/PNw4J7JF91lnBELbZs+Hhh+H0GjYa8osvgvrt3h103mvaNNk1EpEUpJniImjDhg1kZ2czZMgQ2rRpw5o1a6p3TPlJGDcOMv/5GQD7zg6a3JMy33hOTjB07rXXEnzhEzhwIBiSVlwc/MHRoUOyayQiKUoJPUL27dvH6NGj6dixIyUlJRQUFLBkyRIuueSSpNVp82ZoxOcA7K/X8JjyhLrhhuAz6ZrU7O4OQ4fCX/8Kjz8O116b7BqJSApTQo+IefPm0bZtW/Lz8xk4cCAbN25k8ODBnHZacn/EF14Ib9OB2hzgw9bfPaY8oTIzjza7HzmS4ItXYPx4eOop+OUvYdCgZNdGRFKcEnqK27x5M/3796dfv37Uq1ePZcuWUVBQwLnnnpvsqgFH5xs/RG2OnB502UjafOM5ObBtW81odp82LfhHGDoU/vd/k10bEYkAJfQUdfDgQSZPnkybNm145ZVXyM/PZ82aNVxxxRXJrtoxYucbhyTPN967d81odl+wAO68M2gx+O1vg4nYRUSqSAk9Bb322mtkZWUxatQorr76aoqLixM7pvwklc433qlTkucbz8wMhoYls9n9jTfgBz+Ajh3hueegVtwnaxSRNKGEnkJ27tzJ0KFDueKKK9izZw9z5sxh7ty5NNUwp8rLyYFPP4UVKxJ/7fffDzrnNW4ML76Y+HntRSTSlNBTgLszffp0WrVqxcyZMxk5ciTFxcX069cv2VVLPb17Q926iW9237EjaB04fDiYOCaRy8iKSFpQQq/hioqKyM7OZvDgwbRu3ZrCwkLy8/PJ1NPdqalXL0isf/pT4prd9++HPn1gyxaYPx9atUrMdUUkrSih11D79u1jzJgxdOjQgaKiIgoKCli6dCnt27dPdtVSX2mz++uvx/9ahw8HnQZWrQqmzevSJf7XFJG0pIReA82fP5927doxadIkBg4cyKZNm2rEmPLI6NMnMc3u7nDPPcFCKw8/DDfeGN/riUhaU4aoQTZv3syAAQPo27cvmZmZLF26tEaNKY+MevXg+uvj3+w+ZQr85jcwYgTcdVf8riMighJ6jXDo0CGmTJlC27Ztefnll8nLy6OwsJBu3bolu2rRlZMTrHC2cmV83v+ZZ2DUqGCI2qRJ8bmGiEgMDYJNshUrVnDHHXfwwQcf0KdPH6ZOnUqzZs2SXa3oi212r+bPteuvWQOjR0N2NsycCfqoREQSQHeaJNm5cye33347Xbt2Ze/evcyZM4d58+YpmSfK2WfDdddVf7P7hg1cMn48tGgBL7wQ/NEgIpIASugJ5u7MmDGD1q1bM336dEaOHMmMGTM0pjwZcnJg69agB3p1+OQT+P73OXzGGcFY8wYNqud9RUQqQQk9gYqLi+nRowc//vGPadmyJWvWrCE/P58zzzwz2VVLT336QJ061dPbfc+eYG72PXtYn5eXhOXkRCTdKaEnwP79+xk7diyXXnopGzZs4PHHH2fZsmUaU55s55xTPc3uBw7ATTdBcTHMns3eiy+uvjqKiFSSEnqcLViwgHbt2pGXl8ett97Kxo0bGTJkiMaU1xQ5OcEMbqtXn9r57sESqK++CgUFcM011Vs/EZFKUlaJky1btnDjjTfSp08fMjIyWLJkCdOnT6dRo0bJrprE6tu3as3u48fDU0/BhAkwcGD11k1E5CQooVezgwcP8sADD9CmTRsWLlxIXl4ea9asoXv37smumpTnnHPg2muDZnf3kzv3d7+DiRPh9tth3Lj41E9EpJKU0KvRihUr6NSpEyNGjKBnz54UFxczevRo6tSpk+yqyfHk5MDmzSfX7L5gAfzkJ0FHuEcfBbP41U9EpBKU0KvBrl27GDZsGF27dmX37t288MILGlOeSvr2hdq1K9/svnp1MANcVhY89xzU0vxMIpJ8SuhV4O7MnDmTVq1a8cQTTzBixAhKSkro378/pie21FG/fuWb3d9/P1hTvXHj4Cldy9iKSA2hhH6KiouL6dmzJ7fddhstWrSgsLCQyZMna53yVJWTAx9/DG+8UfExn38eLOpy5AgsXBgkdRGRGkIJ/STt37+fX/ziF1x66aWsW7eOxx57jOXLl/Ptb3872VWTqjhRs/v+/cExW7fCvHnQsmVi6ycicgJK6CfhxRdfpF27dtx///3ceuutbNq0iaFDh2pMeRQ0aABXXx0k9LLN7ocPww9/GEwR+/vfV/tiLiIi1UGZqBK2bt3KTTfdRO/evTnzzDNZvHixxpRHUWmz+5tvHi1zh7vvhrlzYepUGDAgefUTETkOJfTjOHToEA8++CBt2rThpZde4v7772ft2rVkZ2cnu2oSD/36BT3WY5vdJ08OhqWNHAk/+1ny6iYicgJK6BV4/fXX6dy5M8OHDyc7O5uioiLGjBmjMeVR1rAhn7a9mi0P/pHTzLm70e+Ddc1vuQXy8pJdOxGR41JCL2PXrl3ccccddOnShZ07d/L8888zf/58mjdvnuyqSZzNmgUTNubQ5PBHDGcKU3bcxtLTsnnmuhmgfhIiUsPpLhVyd5588klat25NQUEBw4cPp6SkhAEDBmhMeZoYNw6eO9Cfg9RiMqN4h5b0PTKHsffVTXbVREROSAk9tHbtWgYNGsTFF19MYWEhU6ZM0ZjyNLN5M+ymIfPoy1YuoBd/Zg/12bw52TUTETkxzVkZ6tixI4sWLaJ79+4ahpamLrww6OR+K08D8C/O/LpcRKSmU+aK0aNHDyXzNDZxImRkBIm8NJlnZATlIiI1nbKXSCg3F6ZNg6ZNg8XTmjYNvs/NTXbNREROTE3uIjFyc5XARSQ16QldREQkApTQRUREIkAJXUREJAKU0EVERCJACV1ERCQClNBFREQiQAldREQkApTQRUREIkAJXUREJAKU0EVERCJACV1ERCQClNBFREQiQAldREQkApTQRUREIiBuCd3MmpjZIjMrMbMiM7snLG9oZq+Y2bvh1wYx54w1s/fMbJOZXRevuomIiERNPJ/QDwHD3b0N8D3gp2bWFhgDvOruLYBXw+8J990CtAOuBx41s9PjWD8REZHIiFtCd/dt7l4Ybn8JlAAXAP2AmeFhM4H+4XY/4Fl3/8rdPwTeAy6LV/3KmjULmjWD004Lvs6alagri4iIVF2tRFzEzJoBHYFVQGN33wZB0jez88LDLgBWxpy2NSyLu1mzYNgw2L8/+P7jj4PvAXJzE1EDERGRqjF3j+8FzDKBJcBEd3/ezP7h7vVj9u929wZm9hvgdXd/OiwvAP7s7rPLvN8wYBhA48aNOz377LNVruP69XDgwL+X16kD7dtX+e1PaO/evWRmZsb/QkmmOKMlHeJMhxhBcaaanj17vuXuncuWx/UJ3cxqA7OBWe7+fFj8dzM7P3w6Px/4LCzfCjSJOf1bwKdl39PdpwHTADp37uw9evSocj2vvBLK+7vGDI4cqfLbn9DixYupjjhqOsUZLekQZzrECIozKuLZy92AAqDE3R+M2TUPGBRuDwLmxpTfYmZ1zaw50AJYHa/6xbrwwpMrFxERqWni2cu9K/Aj4EozWxu+egF5wDVm9i5wTfg97l4E/AEoBhYCP3X3w3Gs39cmToSMjGPLMjKCchERkVQQtyZ3d18OWAW7r6rgnIlAwtNoace3ceNg8+bgyXziRHWIExGR1JGQXu6pIDdXCVxERFKXpn4VERGJACV0ERGRCFBCFxERiQAldBERkQhQQhcREYkAJXQREZEIUEIXERGJACV0ERGRCIj7amvxZGafAx8nux7V4FxgR7IrkQCKM1rSIc50iBEUZ6pp6u6NyhamdEKPCjN7s7yl8KJGcUZLOsSZDjGC4owKNbmLiIhEgBK6iIhIBCih1wzTkl2BBFGc0ZIOcaZDjKA4I0GfoYuIiESAntBFREQiQAk9DszsCTP7zMw2xJRNNrONZrbOzF4ws/ox+8aa2XtmtsnMrosp72Rm68N9U83MEh3L8ZQXZ8y+EWbmZnZuTFmk4jSzu8JYiswsP6Y8MnGaWQczW2lma83sTTO7LGZfysVpZk3MbJGZlYQ/t3vC8oZm9oqZvRt+bRBzTpTijNR9qKI4Y/ZH5j5UKe6uVzW/gO5AFrAhpuxaoFa4PQmYFG63Bd4G6gLNgfeB08N9q4HLAQNeAr6f7NhOFGdY3gR4mWCOgHOjGCfQE/grUDf8/ryIxvmX0noCvYDFqRwncD6QFW7XA94JY8kHxoTlY1L9/+dx4ozUfaiiOMPvI3UfqsxLT+hx4O5LgV1lyv7i7ofCb1cC3wq3+wHPuvtX7v4h8B5wmZmdD5zt7q978Nv2JNA/MRFUTnlxhn4FjAJiO2hELc47gTx3/yo85rOwPGpxOnB2uH0O8Gm4nZJxuvs2dy8Mt78ESoALCOKZGR42k6N1jlScUbsPHefnCRG7D1WGEnpyDCb4CxCCX74tMfu2hmUXhNtly2s0M+sLfOLub5fZFak4gZZANzNbZWZLzOw7YXnU4rwXmGxmW4ApwNiwPOXjNLNmQEdgFdDY3bdBkCSA88LDohZnrEjdh2LjTKP70DFqJbsC6cbMxgGHgFmlReUc5scpr7HMLAMYR9Cs92+7yylLyThDtYAGwPeA7wB/MLP/IHpx3gn83N1nm9l/AgXA1aR4nGaWCcwG7nX3L47zcWmk4owpj9R9KDZOgrjS5T50DD2hJ5CZDQJ6A7lhsw4Efwk2iTnsWwTNmls52hwWW16TXUTwudTbZvYRQZ0LzeybRCtOCOr9vAdWA0cI5omOWpyDgOfD7T8CpZ3iUjZOM6tNcPOf5e6lsf09bHYl/Fr6EUrU4ozcfaicONPpPnSsZH+IH9UX0IxjOxddDxQDjcoc145jO2l8wNFOGm8QPAGWdtLoley4ThRnmX0fcbQzSqTiBP4b+GW43ZKgGc8iGGcJ0CPcvgp4K5V/nmGdngQeKlM+mWM7xeVHNM5I3YcqirPMMZG5D53w3yPZFYjiC3gG2AYcJPjLbwhB54stwNrw9X8xx48j6G25iZielUBnYEO479eEEwHVlFd5cZbZ//V/pKjFCdQBng7rXQhcGdE4rwDeCm+Cq4BOqRxnGI8D62L+L/YCvgG8Crwbfm0Y0TgjdR+qKM4yx0TiPlSZl2aKExERiQB9hi4iIhIBSugiIiIRoIQuIiISAUroIiIiEaCELiIiEgFK6CIpzszGhStNrQtXRftususEYGb3mdknYZ3WmlneKb5PfzNrW931E4kaTf0qksLM7HKCWb+y3P2rcJnIOnG+5unufriSh//K3adU8ZL9gQUEE6KISAX0hC6S2s4HdvjRVd92uPunAGZ2fbj29fJwfecFYfl9Zjai9A3MbEO4sAVmNsfM3gqf+IfFHLPXzH5pZquAy8O1o5eEx75cOm1qZVR0rpldZGYLw/JlZtbazLoAfQkWiFlrZhdV9R9MJKqU0EVS21+AJmb2jpk9ambZAGZ2BvAY0AfoBnyzku832N07EcyadbeZfSMsP4tgStjvEswY9whwc3jsE8DECt7v5zFN7teF825XdO404K6wfATwqLuvAOYBI929g7u/X8k4RNKOmtxFUpi77zWzTgRJuyfwnJmNIZgC80N3fxfAzJ4GhlX8Tl+728wGhNtNgBbATuAwwQIYAK2AS4BXwlXKTieYMrY8xzS5m9kl5Z0brpbVBfhjzMpndStRXxEJKaGLpLjw8+zFwGIzW0+wQtpaKl7+8RDHts6dAWBmPQiWRr3c3feb2eLSfcC/Yj43N6DI3S8/heqWe66ZnQ38w907nMJ7ighqchdJaWbWysxaxBR1AD4GNgLNYz5z/q+YYz4CssLzswhWnQI4B9gdJvPWBCtPlWcT0CjskIeZ1TazdpWscrnnerBW94dmlhOWm5ldGp7zJVCvku8vkraU0EVSWyYw08yKzWwd0Ba4z93/RdDE/qKZLSdI8qVmAw3NbC1wJ/BOWL4QqBW+zwRgZXkXdPcDwM3AJDN7m6A1oEtlKnuCc3OBIWF5EdAvLH8WGGlma9QpTqRiWm1NJA2Ezekj3L13susiIvGhJ3QREZEI0BO6iIhIBOgJXUREJAKU0EVERCJACV1ERCQClNBFREQiQAldREQkApTQRUREIuD/AWmQ8ANbTUUmAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# let's plot the dataset\n",
    "plot_possible_approx()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* Which function $y=f(x)$ to choose?\n",
    "* Assuming a **linear** connection between dependent and independent variables **limits** the search space (constraining the problem)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## <img src=\"https://img.icons8.com/dusk/64/000000/line.png\" style=\"height:50px;display:inline\"> Linear Regression\n",
    "---\n",
    "A linear model makes a prediction by computing a weighted sum of the input features, plus a constant called the *bias term* (also called *intercept* sometimes).\n",
    "\n",
    "We denote:\n",
    "* $\\hat{y}$ - the predicted value\n",
    "* $n$ - number of features\n",
    "* $x_i$ is the $i^{th}$ feature value\n",
    "* $\\theta_j$ is the $j^{th}$ model parameter, where $\\theta_0$ is the bias weight ($\\theta_0 \\cdot 1$)\n",
    "\n",
    "The *Linear Regression* model prediction: $$ \\hat{y} = \\theta_0 + \\theta_1 x_1 + ... + \\theta_n x_n $$\n",
    "In *vector* form: $$ \\hat{y}=h_{\\theta} (x) =\\theta^T \\cdot x $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### <img src=\"https://img.icons8.com/dusk/64/000000/us-dollar.png\" style=\"height:50px;display:inline\"> Linear Regression Cost Function & Connection to MLE\n",
    "---\n",
    "How do we train a linear regression model?\n",
    "\n",
    "* Training a model means tuning its parameters so that the model best fits to the training data (train set)\n",
    "* The most common performance measure on a regression model is the Root Mean Square Error (RMSE), thus, we need to find the value of $\\theta$ that minimzes the RMSE. In practice, it is simpler to minimize the Mean Square Error - **MSE** (the same values that minimize the RMSE, also minimize the MSE).\n",
    "* The MSE Cost Function: $$ MSE(X, h_{\\theta}) = \\frac{1}{m}\\sum_{i=1}^m (\\theta^T \\cdot x^{(i)} - y^{(i)})^2 $$\n",
    "    * Pay attention to notations: $m$ is the number of samples, where $n$ is the number of features. That is, each sample has $n$ components."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### <img src=\"https://img.icons8.com/dusk/64/000000/broken-link.png\" style=\"height:30px;display:inline\"> Connection to MLE\n",
    "---\n",
    "* Maximum Likelihood Estimation (MLE) is the most common way to estimate parameters of a statistical model by calculating: $$\\hat{\\theta} = \\underset{\\theta}{\\mathrm{argmax}} \\log p(y|x, \\theta) $$\n",
    "* The Negative Log Likelhood (NLL) under **i.i.d** assumption: $$ NLL(\\theta) = -\\log p(D|\\theta) = -\\sum_{i=1}^n \\log p(y_i|x_i, \\theta) $$\n",
    "* If we assume that $$ P(y|x, \\theta) = \\mathcal{N}(\\theta^Tx, \\sigma^2) $$\n",
    "    * Note: Why would we assume that? When collecting data from the real world, it is safe to assume that it would not be deterministic, that is, we assume that there is some underlying distribution. More formally, in the case of linear regression, we assume that the $y$ is indeed a linear deterministic function of the input, but with added noise $\\epsilon \\sim \\mathcal{N}(0, \\sigma^2)$ which leads to $y = \\theta^T x + \\epsilon \\rightarrow p(y|x, \\theta) = \\mathcal{N}(\\theta^T x, \\sigma^2)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* We get: $$ NLL(\\theta) = -\\sum_{i=1}^n \\log \\big[ \\big( \\frac{1}{2\\pi \\sigma^2} \\big)^{\\frac{1}{2}} e^{-\\frac{1}{2\\sigma^2}(y_i - \\theta^Tx_i)^2} \\big] = -\\frac{N}{2} \\log (2\\pi \\sigma^2) + \\frac{1}{2\\sigma^2} \\sum_{i=1}^n (y_i - \\theta^Tx_i)^2 $$\n",
    "\n",
    "* Thus, **maximizing the log-likelihood** is equivalent to **minimizing the negative-log-likelihood** w.r.t to $\\theta$: $$ \\underset{\\theta}{\\mathrm{argmin}}\\frac{1}{2\\sigma^2} \\sum_{i=1}^n (y_i - \\theta^Tx_i)^2 =  \\underset{\\theta}{\\mathrm{argmin}} \\sum_{i=1}^n (y_i - \\theta^Tx_i)^2 = \\underset{\\theta}{\\mathrm{argmin}} \\frac{1}{n} \\sum_{i=1}^n (y_i - \\theta^Tx_i)^2 = MSE(\\theta) $$\n",
    "    * The equalities are since $\\sigma, n$ do not depend on $\\theta$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### <img src=\"https://img.icons8.com/bubbles/100/000000/home.png\" style=\"height:50px;display:inline\">  Example Cont. - A Linear Model for the House Prices\n",
    "---\n",
    "* One independent variable (house size) which \"explains\" the dependent variable (house price)\n",
    "\n",
    "<img src=\"./assets/tut_08_lr_1.jpg\" style=\"height:200px\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img src=\"./assets/tut_08_lr_2.jpg\" style=\"height:200px\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* **Estimated Regression Model**:\n",
    "    * The sample regression line provides an estimate of the population regression line ($\\hat{y} = \\mathbb{E}(y)$)\n",
    "\n",
    "<img src=\"./assets/tut_08_lr_3.jpg\" style=\"height:200px\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* **Linear Regression Assumptions**:\n",
    "    * The underlying relationship between the $x$ variable and the $y$ variable is *linear*\n",
    "    * Error values $\\epsilon$ are statistically independent\n",
    "    * The probabiity distribution of the errors is *normal* and independent of $x$ with mean 0 and an equal but *unknown* variance for all values of $x$: $p(\\epsilon) = \\mathcal{N}(0, \\sigma^2)$ "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "def plot_lls_sol(x, y, theta_ls):\n",
    "    x_with_bias = np.concatenate([x, np.ones(x.shape)], axis=1)\n",
    "    fig = plt.figure(figsize=(8,5))\n",
    "    ax = fig.add_subplot(111)\n",
    "    ax.scatter(x, y, label=\"original data\", color='b')\n",
    "    ax.plot(x, x_with_bias @ theta_ls, label='least-square solution', color='r')\n",
    "    ax.set_xlabel(\"Square Feet\")\n",
    "    ax.set_ylabel(\"House Price in $1000\")\n",
    "    ax.set_title(\"House Price vs. House Size\")\n",
    "    ax.grid()\n",
    "    ax.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "learned parameters:\n",
      "[[ 0.10976774]\n",
      " [98.24832962]]\n",
      "y_hat = 0.110x + 98.248330 (square feet)\n"
     ]
    }
   ],
   "source": [
    "# solve the regression problem for the house price example\n",
    "x = [1400, 1600, 1700, 1875, 1100, 1550, 2350, 2450, 1425, 1700]\n",
    "y = [245, 312, 279, 308, 199, 219, 405, 324, 319, 255]\n",
    "x = np.array(x).reshape(-1, 1)\n",
    "y = np.array(y).reshape(-1, 1)\n",
    "x_with_bias = np.concatenate([x, np.ones(x.shape)], axis=1)\n",
    "theta_ls = np.linalg.inv(x_with_bias.T @ x_with_bias) @ x_with_bias.T @ y # least-squares solution\n",
    "print(\"learned parameters:\")\n",
    "print(theta_ls)\n",
    "print(\"y_hat = {:.3f}x + {:3f} (square feet)\".format(theta_ls[0][0], theta_ls[1][0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfQAAAFNCAYAAAD2E503AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd5xV1fX38c+iSO8iQXrsgkhVsCBg76ixZUxEjTwaosbH/GzESIz8YuFBY9QkxB7HECyosYsyKlE0oIhSFFRQiigoZagyrOePfYa50y8wt535vl+v+5p71yl37Ttw15x99jnb3B0RERHJbXUynYCIiIjsPBV0ERGRGFBBFxERiQEVdBERkRhQQRcREYkBFXQREZEYUEEXiREzO9zMPsl0HrWFmRWa2Y8znYcIqKBLLWZmC83sqDKx4WY2NVM5lRXlUxQVjjVmNtPMTqpsfXd/y933SWeOO8rM3Mz2LBMbbWaPZiqnssyspZk9YGZfm9laM/vUzK4pXu7uTd3980zmKFJMBV0k+73j7k2BlsD9wEQza112JTOrl/bM4u8OoCmwH9ACOAX4LKMZiVRCBV2kCma2n5kVmNkqM5ttZqckLCsws18kvN52dG/BHWb2jZmtNrNZZtYjWtbAzMaa2ZdmttzM/mpmjarLxd23Ag8AjYAfm9lgM1tsZteY2dfAg8WxhJw6mdlTZvatma00s7sTll1oZnPN7Hsze9nMulTyGbxkZr8qE/vQzE6vqp01wcwOMbP/Rvv+r5kdkrCsVA9L4tG9mTU0s0ejNq+Ktm0XLWthZveb2TIzW2JmN5tZ3UpS6A885u7fu/tWd5/n7k8kvKeb2Z5mtnvUi1L8WG9mnrBeUp+1yM5QQRephJnVB/4NvALsBlwG5JtZMl3axwCDgL0JR9ZnAyujZbdG8V7AnkAH4HdJ5FMP+AVQCMyPwj8CWgNdgBFl1q8LPAcsArpG7zMhWjYMuB44HWgLvAX8s5K3fgw4N2G/+0fv93w17dwpUS/E88BdQBtgHPC8mbVJYvPzCUfUnaJtLwE2RMseBrYQPvveURt+UcE+AKYBY8zsAjPbq7I3c/elUfd706g3ZRI79lmL7Dh310OPWvkAFhKK46qEx3pgarT8cOBroE7CNv8ERkfPC4BfJCwbnrDtUOBTYECZ7Q1YB+yREBsIfFFJjsMJxWcVsIJQYI6Klg0GNgMNE9YfDCxO2O+3QL0K9vsicFHC6zpR27tUsG6zKOcu0esxwANVtTPJz9+BNWU+/43Ao9HynwHvldnmHWB4wu/vqIRloxO2vRB4G+hZZvt2wCagUULsXGBKJTk2IhTjGcAPwALg+DJt2LPMNtdE6zfa3s9aDz125qEjdKnthrl7y+IH8MuEZbsDX3no6i62iHCkWyV3fx24G7gHWG5m482sOeEIrTEwI+oKXgW8FMUrMy3Kb1d3H+DukxOWfevuGyvZrhOwyN23VLCsC/CnhBy+I/yxUa5t7r6WcKR8ThQ6B8ivpp3J6lPm878lYdnuhM87UVKfP/AP4GVggpktNbPboh6XLkB9YFlC2/9G6IEpx903uPv/untfwpH+RODxisYwAJjZ8cAVhH9XxT0CSX/WIjtDBV2kckuBTmaW+P+kM7Aker6OUJyL/ShxY3e/KyoE3Qld0v9DOMreAHRPKGQtPHTT7oiqpkv8CuhcyWC5r4D/k1hM3b2Ru79dyb7+CZxrZgMJR61TtiVQcTtrwlJCMUyU1Ofv7j+4++/dfX/gEOAk4OeEdm8Cdk1od3N3715dMu6+BvhfoAnQrezy6FTMw8BZ7v5VwqLt/axFdogKukjl3iUUjavNrL6ZDQZOJjo3CswETjezxhYuv7qoeEMz629mB0dHhesIXclF0dH+34E7zGy3aN0OZnZsCvJ/D1gG3GJmTaKBYodGy/4KXGdm3aMcWpjZmVXs6wVCcb0J+Fdxr0Vl7ayh/F8A9jazn5pZPTM7G9ifMC4Awud/TvS76Qf8pHhDMxtiZgdE4wjWELrLi9x9GWFMxP8zs+ZmVsfM9jCzIypKwMxuiNq4i5k1JBx9rwI+KbNec+AZ4LfuXvayx+39rEV2iAq6SCXcfTPhMqXjCUfW9wI/d/d50Sp3EM5hLyccmeUnbN6cULi/J3QTrwTGRsuuIZyLnWZma4DJQI1fO+7uRYQ/QPYEvgQWEwat4e6TCIPzJkQ5fBy1s7J9bQKeAo4iDJIrVmk7zex6M3txJ/JfSTiyvira79XASe6+IlrlBmCP6L1/XyavHwFPEIr5XOANoPj69p8DuwBzom2fANpXlgbwIOH3vxQ4GjjR3QvLrNeH8DsclzjaPWrHdn3WIjvK3KvqsRMREZFcoCN0ERGRGFBBFxERiQEVdBERkRhQQRcREYkBFXQREZEYyOnZmXbddVfv2rVrptOoEevWraNJkyaZTiNt1N54U3vjTe3NrBkzZqxw93J3l8zpgt61a1emT5+e6TRqREFBAYMHD850Gmmj9sab2htvam9mmVnZWyID6nIXERGJBRV0ERGRGFBBFxERiYGcPodekR9++IHFixezcWNlM0pmpxYtWjB37txMp5E22dLehg0b0rFjR+rXr5/pVEREdkrsCvrixYtp1qwZXbt2xcwynU7S1q5dS7NmzTKdRtpkQ3vdnZUrV7J48WK6dSs3G6aISE6JXZf7xo0badOmTU4Vc8kMM6NNmzY515sjIlKR2BV0QMVckqZ/KyISF7Es6JnWtGnTGt3fQw89xNKlS2t0n9nkoYce4le/+lWV6yxcuJDHHiuZ7nr69OlcfvnlqU5NRHJcfj507Qp16oSf+fmZzih1VNBzQLYUdHdn69atGXnvsgW9X79+3HXXXRnJRURyQ34+jBgBixaBe/g5YkR8i7oKeordfvvt9O/fn549e3LjjTduiw8bNoy+ffvSvXt3xo8fD0BRURHDhw+nR48eHHDAAdxxxx088cQTTJ8+nby8PHr16sWGDRtK7X/ZsmUMGjSIXr160aNHD9566y0AHnzwQfbee2+OOOIILr744m1HwMOHD+eJJ57Ytn1xb0JhYSFHHnkkffr04YADDuCZZ54BQiHdb7/9+OUvf0mfPn346quveOWVVxg4cCB9+vThzDPPpLCwsFy777rrLvbff3969uzJOeecA8B3333HsGHD6NmzJ0OHDmXWrFnltqssv2uvvZa33nqLXr16cccdd1BQUMBJJ51Ubr8DBgzYtt/Ro0dz4YUXMnjwYH784x/rDwCRWmbUKFi/vnRs/foQj6PYjXLPJq+88grz58/nvffew9055ZRTePPNNxk0aBAPPPAArVu3ZsOGDfTv359jjjmGTz/9lCVLlvDxxx8DsGrVKlq2bMndd9/N2LFj6devX7n3eOyxxzj22GMZNWoURUVFrF+/nmXLlnHjjTcyY8YMWrRowZAhQ+jdu3eVuTZs2JBJkybRvHlzVqxYwYABAzjllFMA+OSTT3jwwQe59957WbFiBTfffDOTJ0+mSZMm3HrrrYwbN47f/e53pfZ3yy238MUXX9CgQQNWrVoFwI033kjv3r15+umnee655/j5z3/OzJkzk/osb7nlFsaOHctzzz0HhFsxFkvc7+uvv15qv/PmzWPKlCmsXbuWffbZh0svvVSXqInUEl9+uX3xXBfvgv7rX0OSBSNpvXrBnXcmteorr7zCK6+8sq2YFhYWMn/+fAYNGsRdd93FpEmTAPjqq6/47LPP6NOnD59//jmXXXYZJ554Isccc0y179G/f38uvPBCfvjhB4YNG0avXr147bXXGDx4MG3bhnv3n3322Xz66adV7sfduf7663nzzTepU6cOS5YsYfny5QB06dKFAQMGADBt2jTmzJnDoYceCsDmzZsZOHBguf317NmTvLw8hg0bxrBhwwCYOnUqTz75JABHHHEEK1euZPXq1dW2sTqJ+x06dGip/Z544ok0aNCABg0asNtuu7F8+XI6duy40+8pItmvc+fQzV5RPI7U5Z5C7s51113HzJkzmTlzJgsWLOCiiy6ioKCAyZMn88477/Dhhx/Su3dvNm3aRKtWrfjwww8ZPHgw99xzD7/4xS/K7fPdd9+lV69e9OrVi2effZZBgwbx5ptv0qFDB372s5/xyCOPAJWP3q5Xr9628+DuzubNmwHIz8/n22+/ZcaMGcycOZN27dptu5wrcZYhd+foo4/e1qY5c+Zw//33l3uf559/npEjRzJjxgz69u3Lli1bcPdy65XNs7L8qvucK9tvgwYNtsXq1q3Lli1bqt2fiMTDmDHQuHHpWOPGIR5HKT9CN7O6wHRgibufZGatgX8BXYGFwFnu/n207nXARUARcLm7v7xTb57kkXSqHHvssdxwww3k5eXRtGlTlixZQv369Vm9ejWtWrWicePGzJs3j2nTpgGwYsUKdtllF8444wz22GMPhg8fDkCzZs1Yu3YtAAcffHCpbupFixbRoUMHLr74YtatW8f777/PNddcwxVXXMHKlStp3rw5jz/+OAceeCAQZqibMWMGZ511Fs888ww//PADAKtXr2a33Xajfv36TJkyhUUV/VkLDBgwgJEjR7JgwQL23HNP1q9fz+LFi9l77723rbN161a++uorhgwZwmGHHcZjjz1GYWEhgwYNIj8/nxtuuIG33nqLXXfdlebNm5faf2X5JX4GZSXut6CgoML9ikjtk5cXfo4aFbrZO3cOxbw4Hjfp6HK/ApgLFH/DXgu85u63mNm10etrzGx/4BygO7A7MNnM9nb3ojTkmBLHHHMMc+fO3dYl3bRpUx599FGOO+44/vrXv9KzZ0/22Wefbd3ZS5Ys4YILLth2hPrHP/4RCAPFLrnkEho1asQ777xDo0aNtr1HQUEBt99+O/Xr16dp06Y88sgjtG/fntGjRzNw4EDat29Pnz59KCoKH+PFF1/MqaeeykEHHcSRRx657eg7Ly+Pk08+mX79+tGrVy/23XffCtvUtm1bHnroIc4991w2bdoEwM0331yqoBcVFXHeeeexevVq3J0rr7ySli1bMnr0aC644AJ69uxJgwYNePjhh8vtv7L8evbsSb169TjwwAMZPnx4qTEBiftt3LhxhfsVkdopLy++Bbwcd0/ZA+gIvAYMBZ6LYp8A7aPn7YFPoufXAdclbPsyMLCq/fft29fLmjNnTrlYLlizZk3K9v3ggw/6yJEjU7b/HZHK9m6vdPybmTJlSsrfI5uovfGm9mYWMN0rqImpPkK/E7gaSLxpdzt3Xxb9MbHMzHaL4h2AaQnrLY5ipZjZCGAEQLt27UqNdoYw6UdlXbPZrKioKGV5b9y4kc2bN2fV55LK9m6vjRs3lvt3VNMKCwtT/h7ZRO2NN7U3O6WsoJvZScA37j7DzAYns0kFsXKjndx9PDAeoF+/fj54cOldz507N+OTfuyIVE5Wcskll6RkvzsjGyZnKdawYcNqL+vbWQUFBZT9txpnam+8qb3ZKZVH6IcCp5jZCUBDoLmZPQosN7P20dF5e+CbaP3FQKeE7TsCmb89moiISA5I2WVr7n6du3d0966EwW6vu/t5wLPA+dFq5wPPRM+fBc4xswZm1g3YC3gvVfmJiIjESSZuLHMLMNHMLgK+BM4EcPfZZjYRmANsAUZ6Do9wFxERSae0FHR3LwAKoucrgSMrWW8MENNL/kVERFJHd4rLoBNOOGHbfc4r87vf/Y7Jkyfv0P4TJzCpyuDBg5k+fXqV69x5552sLzvLgYiIZA0V9AzwaBrSF154gZYtW1a57k033cRRRx2Vpswqp4IuIpLdan1Bz88Pk97XqRN+1sQ8uePGjaNHjx706NGDO6Pbz1Y0DWnXrl1ZsWIFALfeeiv77rsvRx99NOeeey5jx44FSk8n2rVrV2688cZtU5zOmzcPgPfee49DDjmE3r17c8ghh/DJJ59Umd+GDRs455xz6NmzJ2effXapKVkvvfRS+vXrR/fu3bdN93rXXXexdOlShgwZwpAhQypdT0REMifes61VIz8/THZffOC5aFF4DTt+q8AZM2bw4IMP8u677+LuHHzwwRxxxBG0atWq1DSkiaZPn86zzz7LBx98wJYtW+jTpw99+/atcP+77ror77//Pvfeey9jx47lvvvuY9999+XNN9+kXr16TJ48meuvv37b7GMV+ctf/kLjxo2ZNWsWs2bNok+fPtuWjRkzhtatW1NUVMSRRx7JrFmzuPzyyxk3bhxTpkxh1113rXS9nj177tiHJiIiO61WH6GPGlVSzIutXx/iO2rq1KmcdtppNGnShKZNm3L66afz1ltvAaWnIS27zQknnECjRo1o1qwZJ598cqX7P/300wHo27cvCxcuBMLEKmeeeSY9evTgyiuvZPbs2VXm+Oabb3LeeecB4R7piYV44sSJ9OnTh969ezN79mzmzJlT4T6SXU9ERNKjVhf0yia5ryyeDK9gKs9iidOQJrtNWcXTgSZOBXrDDTcwZMgQPv74Y/79739vm/a0KhVNr/rFF18wduxYXnvtNWbNmsWJJ55Y4b6SXU9ERNKnVhf0yia5ryyejEGDBvH000+zfv161q1bx6RJkzj88MOr3Oawww7jpZdeYuPGjRQWFvL8889v13uuXr2aDh3Cbe8feuihpHLMjwYLfPzxx8yaNQuANWvW0KRJE1q0aMHy5ct58cUXt22TOH1pVeuJiEhm1Opz6GPGlD6HDtC4cYjvqD59+jB8+HAOOuggAH7xi1/Qu3fvbd3jFenfvz/HH388Bx54IF26dKFfv360aNEi6fe8+uqrOf/88xk3bhxDhw6tdv1LL71023SjvXr12pbrgQceSO/evenevTs//vGPOfTQQ7dtM2LECI4//njat2/PlClTKl1PREQypKIp2HLlURPTpz76qHuXLu5m4eejj27X5jVm6dKl7u6+bt0679u3r8+YMSMziaSJpk+NN7U33tTezCJD06dmvby8HR/RXpMuv/xy5s+fz8aNGzn//PNLjTwXERGpTq0v6NnigQceyJrpREVEJPfU6kFxIiIicRHLgu7bcRmY1G76tyIicRG7gt6wYUNWrlypL2qplruzcuVKGjZsmOlURER2WuzOoXfs2JHFixfz7bffZjqV7bJx48ZaVViypb0NGzakY8eOmU5DRGSnxa6g169fn27dumU6je1WUFBA7969M51G2tS29oqIpFrsutxFRERqIxV0ERGRGFBBFxERiQEVdBERkRhQQRcREYkBFXQREZEYUEEXERGJARV0ERGRGFBBFxERiQEVdBERkRhQQRcREYkBFXQREZEYUEEXERGJARV0ERGRGFBBFxERiQEVdBERkRhQQRcREYkBFXQREZGa5A6jR4NZeKxenZa3rZeWdxEREYm7NWtg2DCYMqUk1q0bNG+elrfXEbqIiMjO+OgjqFMHWrQoKeYHHwwrV8Lnn4ej9DRQQRcREdkRjzwSinXPnqGbHeCaa6CoCKZNg9at05qOutxFRESS9cMPcMkl8MADpePPPgsnn5yZnCIq6CIiItVZuhQGDYLPPiuJ7bZbOBLv1i1zeSVQQRcREanM668z+MgjS8fOPDN0tzdsmJmcKqFz6CIiIonc4eabw/nxxGJ+991h2cSJWVfMQUfoIiIiwdq1cPrpMHlyqfCMe++l76WXZiip5OkIXUREarfZs6F+/XC9eHEx79cPVqwAd9but19m80uSCrqIiNRO+fmhW71HD9iyJcSuuipcdvbf/0KbNpnNbzupy11ERGqPLVtg5EgYP750fNKkcJe3HKaCLiIi8bdsGQweDJ9+WhJr3RrefRf23DNjadUkdbmLiEh8vfFG6FbfffeSYn766bB+fbg1a0yKOaigi4hI3LjDLbeEQj54cEn8T38Ky558Eho1ylh6qaIudxERiYfCwnDTl5deKh1/+20YODAzOaWRCrqIiOS2efOgTx/YsKEk1qsXvPoq7Lpr5vJKM3W5i4hIbvrXv0K3+n77lRTzX/86jGT/4INaVcwhhQXdzBqa2Xtm9qGZzTaz30fx0Wa2xMxmRo8TEra5zswWmNknZnZsqnITEZEcVVQULjszg3POKYk/8UQ4P37HHVC3bubyy6BUdrlvAoa6e6GZ1QemmtmL0bI73H1s4spmtj9wDtAd2B2YbGZ7u3tRCnMUEZFcsHw5DB0Kc+aUxFq0CDeA2WuvzOWVRVJ2hO5BYfSyfvTwKjY5FZjg7pvc/QtgAXBQqvITEZEcMHVqOBr/0Y9Kivkpp8C6dbBqlYp5AnOvqsbu5M7N6gIzgD2Be9z9GjMbDQwH1gDTgavc/XszuxuY5u6PRtveD7zo7k+U2ecIYARAu3bt+k6YMCFl+adTYWEhTZs2zXQaaaP2xpvaG28pb687nSZOZI+//rVUeMEvf8nin/wkFPg0yrbf75AhQ2a4e79yC9w95Q+gJTAF6AG0A+oSegfGAA9E69wDnJewzf3AGVXtt2/fvh4XU6ZMyXQKaaX2xpvaG28pa++6de4nneQezoaXPKZOTc37JSnbfr/AdK+gJqZllLu7rwIKgOPcfbm7F7n7VuDvlHSrLwY6JWzWEViajvxERCSDPv0UmjWDJk3guedC7IADwnlzdzj00MzmlyNSOcq9rZm1jJ43Ao4C5plZ+4TVTgM+jp4/C5xjZg3MrBuwF/BeqvITEZEMe+KJ0H2+zz7hpjAAv/pVuOxs1izYbbfM5pdjUjnKvT3wcHQevQ4w0d2fM7N/mFkvwgC5hcD/AXD32WY2EZgDbAFGuka4i4jES1ERXHkl/PnPpeP/+hecdVZmcoqJlBV0d58F9K4g/rMqthlDOK8uIiJx8u23cNRR4ci7WJMmMH067Ltv5vKKkWoLenSDl2FAB8JR9VLgGXd/qcoNRURE3n67/DnwE08MR+RNmmQmp5iqsqCb2Z3A3sAjhEFrEAarXW5mx7v7FSnOT0REctGdd4au9US33Qa/+U3aLzurLao7Qj/B3fcuGzSzfwGfAiroIiISbNgA554LzzxTOv7GGzBoUGZyqkWqK+gbzewgdy872rw/sDFFOYmISC5ZsAD69w93biu2337w+uvhDm+SFtUV9OHAX8ysGSVd7p0Id3kbnrq0REQk602aBKefXjp26aVw111QT7Nzp1uVn7i7vw8cbGY/IgyKM2Cxu3+djuRERCTLbN3KHvfcA0OGlI4/9ljobpeMSWaUuwFdKBnlXtfMlke3nxMRkdpgxQo45hj44IOSW3o2aADvvw/775/JzCRS3Sj3Y4B7gfnAkijcEdjTzH7p7q+kOD8REcmkd9+FAQNKhb7r35/Wr78OWTRhiVR/hP4n4Ch3X5gYjG7N+gKwX4ryEhGRTPrzn+Hyy0vH/vd/4dprmfXGGwxWMc861RX0epQMhku0hDC/uYiIxMXGjXDeefDkk6Xjr79e/py5ZJ3qCvoDwH/NbALwVRTrBJxDmN5URERy3eefw0EHwcqVJbG99oKCAth994ylJdunytnW3P2PQB5hdPtA4JDoeV60TEREctWzz4a7tu2xR0kxv/hi2Lw5TGmqYp5Tqh3l7u5zCDOgiYhIrtu6Fa69Fm6/vXT8H/8I3e2Ss6ob5d4CuI4wOUvbKPwN8Axwi7uvqmxbERHJIt99B8cdB//9b0msbl2YORN69MhcXlJjquxyByYC3wOD3b2Nu7cBhgCrgMdTnZyIiOyk//43dKu3aVNSzI88EtasgS1bVMxjpLqC3tXdb028M5y7f+3utwCdU5uaiIjssL/8JRTygw4qif3hD6HLffJkaNYsc7lJSlR3Dn2RmV0NPOzuywHMrB3hPu5fVbWhiIik2caNcP75MHFi6firr8JRR2UmJ0mb6o7QzwbaAG+Y2Xdm9h1QALQGzkpxbiIikoyFC8OsZo0alRTzbt1g8WJwVzGvJaqbnOV74JroISIi2eT55+Gkk0rHLrgA/vY3qK97f9U21R2hV8rMLqjJREREJAlbt8J114Xz44nF/MEHw9H4Aw+omNdSOzNh7e+BB2sqERERqcL338OJJ8I775SOf/gh9OyZmZwkq1R3HfqsyhYB7Wo+HRERKeX996Fv39KxI46AZ56BFi0yk5NkpeqO0NsBxxKuRU9kwNspyUhERODvf4cRI0rHbrwxPMwyk5NkterOoT8HNHX3RWUeCwmj3UVEpKZs3gx5eaFgJxbzl18O58dHj1YxzxH5+dC1K9SpE37m56f+Pasb5X5RFct+WvPpiIjUQl9+CYccAkuWlMQ6d4apU6FTp8zlJTskPz/8PbZ+fXi9aFHJ32d5eal73+0e5W5mu5hZk1QkIyJSq7z0Ujji7tKlpJj/7GewaVOoAirmOWnUqJJiXmz9+hBPpWoLupldYWb7RM+PINwhbr6Z/TK1qYmIxJA73HBDKOTHH18Sv+++sOyRR2CXXTKXn+y0L7/cvnhNSeaytfOBu6PnvwVOAT4GpgH3pigvEZF4WbUKTj45dKMnev996N07MzlJSnTuHDpYKoqnUpVH6GZ2I7A7MMrMbgZ6EUa9/wZoZma/M7NBqU1RRCSHzZwZjsZbtSop5oceGq4rd1cxj6ExY6Bx49Kxxo1DPJWqLOju/nvgfcKRfGvgSXe/CfgDsMzdb3L3N1ObotQWmRgVKpIyDzwQCnliwR41KtzpbepUaNkyc7lJSuXlwfjxYWhE8RCJ8eNTOyAOkutyvwj4NbASuC2K7Q3cl6qkpPbJ1KhQkRq1eTNcfHE4D57ohRdKny+X2MvLS/93V7UF3d2XUWZyFnefB8xLVVJS+1Q1KlQFXbJdg2++CYdhiaOedt8d3n47xEXSYGfu5S5SYzI1KlRkp7z6KhxzDAMTY+eeGyZKadAgU1lJLbXDs62J1KTKRn+melSoyHZLvGPbMceUxP/617DsscdUzCUjdIQuWWHMmNLn0CE9o0JFkrZmDZx6KhQUlI5Pn07B2rUMHjw4E1mJbJNUQTezBsAZQNfEbaIR7yI7rfg8+ahRoZu9c+dQzHX+XDLuo4/gwAPD0XexAQPCQLdWrcLrskVeJAOS7XJ/BjgV2AKsS3iI1Ji8PFi4MFzVs3Chirlk2MMPh271nj1Livk110BRUZiTvLiYi2SJZLxGA3wAABzNSURBVAt6R3c/291vc/f/V/xIaWYiMVB8bf2MGbq2Pif88ANceGEo5MOHl8T//e9Q1G+5JdwoQSQLJXsO/W0zO8DdP0ppNiIxomvrc8jSpXD44fD55yWxdu3CkXi3bpnLS2Q7JPun5mHADDP7xMxmmdlHZjYrlYmJ5LpMzbgk2+H118PReIcOJcX8rLNgwwb4+msVc8kpyR6h6xZHIttJ19ZnKfcw4vKGG0rH774bRo7MTE4iNaDKgm5mzd19DbA2TfmIxEamZlySSqxdC6edBq+9Vjr+7rtw0EGZyUmkBlXX5f5Y9HMGMD36OSPhtYhUIlMzLsXVDk/eM3s21K8PzZuXFPP+/WHFinC0rmIuMVHlEbq7nxT91Ikkke2UeG09hFt669r6HbNDAwzz8+G880rHrroKbrtNI9UllvSvWiSFiq+t79tX19bvjKQHGG7ZEiq9WeliPmlSOBofO1bFXGJLt34VkaxX7QDDZcvgiCNg/vyShW3ahPPje+yR8vxEsoH+VBWRrFfZQMKzdisIR+O7715SzM84Ixy+r1ihYi61StIF3cwOM7MLoudtzUzn1UUkLUoPMHSu4RYcY8LyISUr3Xln6FZ/4glo1CgTaYpkVLKTs9wI9AP2AR4E6gOPAoemLjURkSAvDxquXs4ZI39UfuE774TJUkRquWSP0E8DTiGakMXdlwLNUpWUiMg2kyaBWeli3rs3fPNNOCJXMRcBki/om93dAQcwsybVbWBmDc3sPTP70Mxmm9nvo3hrM3vVzOZHP1slbHOdmS2IbjF77I40SERi4rTTwvnx008vHd+yBd5/H9q2zUxeIlkq2YI+0cz+BrQ0s4uBycDfq9lmEzDU3Q8EegHHmdkA4FrgNXffC3gteo2Z7Q+cA3QHjgPuNbO629sgEclhmzaFIm4GTz9dEv/pT8PRuDvU1deCSEWSKujuPhZ4AniScB79d+7+52q2cXcvjF7Wjx5OmFf94Sj+MDAsen4qMMHdN7n7F8ACQLdwEqkNZs0KRbxhw9Lx558PRVzzzopUK9lBcd2At9z91eh1IzPr6u4Lq9muLuE2sXsC97j7u2bWzt2XAbj7MjPbLVq9AzAtYfPFUUxE4uqPf4Trry8fX7EiXEcuIkmzcGq8mpXMpgOHuPvm6PUuwH/cvX9Sb2LWEpgEXAZMdfeWCcu+d/dWZnYP8I67PxrF7wdecPcny+xrBDACoF27dn0nTJiQTApZr7CwkKZNm2Y6jbRRe+Otyva6c8gZZ7DL99+XCm9s25Zp//pXOFLPMfr9xlu2tXfIkCEz3L1f2Xiyd4qrV1zMAdx9c1TUk+Luq8ysgHBufLmZtY+OztsD30SrLQY6JWzWEVhawb7GA+MB+vXr54MHD042jaxWUFBAXNqSDLU33ips77ffwm67lV/51lvh6qtpCAwuvzQn6Pcbb7nS3mQHxX1rZqcUvzCzU4EVVW0Q3XymZfS8EXAUMA94Fjg/Wu184Jno+bPAOWbWIOri3wt4L9mGiEiW+ve/w1F32WL+0Ufh/PjVV2cmL5GYSfYI/RIg38zuBgz4Cvh5Ndu0Bx6OzqPXASa6+3Nm9g5h1PxFwJfAmQDuPtvMJgJzgC3ASHcv2u4WiUh2OPtsmDixfHzTJtgl6Q4+EUlSUgXd3T8DBphZU8J597VJbDML6F1BfCVwZCXbjAE0W7RIrtq8GRo0KN91fsYZ4ZasIpIyVRZ0MzvP3R81s/9bJg6Au49LYW4ikitmz4YePcrHn34aTj01/fmI1ELVnUMvviNcs0oeIlKbjR0bzo+XKeb/KZ5/XMVcJG2qPEJ3979F58DXuPsdacpJRLJdly7lJylv2xaWLwczfigoyEhaIrVZtaPco4Fpp1S3nojE3HffldyWNbGY/+EP4Wj8m29y8hpykbhIdpT729EI938RzbgG4O7vpyQrEckeL74IJ5xQPj5zJhx4YPrzEZEKJVvQD4l+3pQQc2BozaYjIlnjZz+DRx8tH9+4ERo0SH8+IlKlZC9bG5LqREQkC2zZAvXrl4+fcgo880z5uIhkjSrPoZvZwdF85oVm9o6Z7ZeuxEQkjebNC+e/yxbzJ54I58dVzEWyXnWD4u4BfgO0AcYBd6Y8IxFJnz/9KRTy/cr8rb5sWSjkZ5yRmbxEZLtV1+Vep3jKVOBxM7su1QmJSBrsvTfMn1861rw5rFqlkeoiOaq6gt7SzE6v7LW7P5WatESkxq1aBa1alY//7nfw+9+nPx8RqVHVFfQ3gJMree2ACrpItnv1VTjmmPLx6dOhb9/05yMiKVHdneIuSFciIlLDDj8cpk4tH1+/Hho1Sn8+IpJSyc6HLiK5YMuWkru5JRbzY48Ng9zcVcxFYkoFXSQOPvqo4svO/vSnUMRfeikzeYlI2iR7pzgRyUa//S2MGVM+/vnn0K1b+vMRkYxJqqCbWWPgKqCzu19sZnsB+7j7cynNTkQqVtmlZVu36rIzkVoq2S73B4FNwMDo9WLg5pRkJCIVS5ztLNHZZ5ecH4+W5edD165Qp074mZ+f9mxFJM2SLeh7uPttwA8A7r4B0GGASDo8/XQo1G3alI6/+moo4hMmlArn58OIEbBoUVi8aFF4raIuEm/JFvTNZtaIcO05ZrYH4YhdRFLlqKNCIT/ttNLxtWtDpT7qqAo3GzUqXJmWaP36EBeR+Ep2UNyNwEtAJzPLBw4FhqcqKZFaq6gI6lXw37JlS/j++6R28eWX2xcXkXhI6gg9up/76YQi/k+gn7sXpC4tkVpmzpxwNF62mI8dG47GkyzmAJ07b19cROIhqYJuZocCG939eaAlcL2ZdUlpZiK1wU03hULevXvp+Pz5oZBfddV273LMGGjcuHSsceOKr24TkfhItsv9L8CBZnYg8D/AA8AjwBGpSkwk1iq7tKyoKAxN3wl5eeHnqFGhm71z51DMi+MiEk/JfnNscXcHTgXucvc/Ac1Sl5ZIDBVPTVq2mJ92WsllZztZzIvl5cHCheGy9IULVcxFaoNkj9DXRnOhnwcMMrO6QP1qthERgOefZ/BJJ5WPv/giHHdc+vMRkVhKtqCfDfwUuMjdvzazzsDtqUtLJAZOOgmef758fPVqaN48/fmISKwlVdDd/WtgXMLrLwnn0EUkUSWXnW2tX586mzdnICERqS2SHeW+1szWRI+NZlZkZqtTnZxIzvjkk4ovO/vjH8GdN195JTN5iUitkewReqkBcGY2DDgoJRmJ5JJbboHrrisfnzsX9t03/fmISK21Q9OnuvvTZnZtTScjkjMaN4YNG8rHt2yBunXTn4+I1HrJTp96esLLOkA/ovu6i9Qaa9ZAixbl48cfDy+8kP58REQSJHuEfnLC8y3AQsI16SLx9/LLFV9e9uyzcPLJ5eMiIhmQ7Dn0C1KdiEjWOeMMeOqp8vHvvoNWrdKfj4hIFZLtcu8I/Jkwy5oDU4Er3H1xCnMTSb+tWys/B+46yyQi2SvZ+0w+CDwL7A50AP4dxUTiYcGCcNlZ2WI+enTJbVlFRLJYsgW9rbs/6O5bosdDQNsU5iWSHuPGhUK+116l47NnhyJ+442ZyUtEZDslOyhuhZmdR5gLHeBcYGVqUhJJgzZtwrnwsnTZmYjkqGSP0C8EzgK+BpYBP4liIrmjsLBktrPEYj5kSEm3uoq5iOSoZEe5fwmckuJcRFLjtdfgqKPKx596KkxdKiISA1UWdDP7M1XcQMbdL6/xjERqyk9/Cv/8Z/n4ypXQunX68xERSaHqjtCnJzz/PaARQpLd3KFOJWeSNFJdRGKsyoLu7g8XPzezXye+FskqCxdCt27l49dfD2PGpD0dEZF0257JWXR4I9nn7rvhssvKxz/8EHr2TH8+IiIZskOzrYlk3O67w7Jl5eObN0P9+unPR0Qkw6obFLeWkiPzxma2pngR4O7ePJXJiZSyfj00aVI+PnAgvP12+vMREckiVV6H7u7N3L159KiX8LyZirmkzRtvhGvHyxbzCRPCQDcV83Ly86Fr1zA+sGvX8FpE4k1d7pK9LrgAHnqofPybb6Ct7jxcmfx8GDEidGgALFoUXgPk5WUuLxFJLRV0yS667GynjRpVUsyLrV8f4iroIvGV7K1fRVLru+9Ct3rZYn7VVZrtbDt9+eX2xUUkHlTQJbPeeisU8jZtSsdnzAhFfOzYzOSVwzp33r64iMRDygq6mXUysylmNtfMZpvZFVF8tJktMbOZ0eOEhG2uM7MFZvaJmR2bqtwk87rdd18o5IMGlQT33x9++CEU8j59MpdcjhszBho3Lh1r3Fj31xGJu1QeoW8BrnL3/YABwEgz2z9adoe794oeLwBEy84BugPHAfeamaa+ipMtW+C888CMLonDrm++ORTx2bOhnoZ17Ky8PBg/Hrp0CX8zdekSXuv8uUi8pezb092XEaZaxd3XmtlcoEMVm5wKTHD3TcAXZrYAOAh4J1U5Spp8/TUMHQpz55aOT5sGBx+cmZxiLi9PBVyktknLOXQz6wr0Bt6NQr8ys1lm9oCZtYpiHYCvEjZbTNV/AEi2Kz4/3r59STE/9VRYt46CKVNUzEVEapB5ikcPm1lT4A1gjLs/ZWbtgBWEO9D9AWjv7hea2T3AO+7+aLTd/cAL7v5kmf2NAEYAtGvXru+ECRNSmn+6FBYW0rRp00ynsfPc6TRhAnuMH18qvGDkSBb/5CfbXsemvUlSe+NN7Y23bGvvkCFDZrh7v3IL3D1lD6A+8DLwfytZ3hX4OHp+HXBdwrKXgYFV7b9v374eF1OmTMl0CjunsND9xBOLLzArefznPxWunvPt3U5qb7ypvfGWbe0FpnsFNTGVo9wNuB+Y6+7jEuLtE1Y7Dfg4ev4scI6ZNTCzbsBewHupyk9qyCefQNOm4fH88yF2wAGwfHko6Yccktn8RERqiVSeQz8U+BkwtMwlareZ2UdmNgsYAlwJ4O6zgYnAHOAlYKS7F6UwP9kZjz8ezo/vuy+sWxdil10WRrLPmgW77VbhZsX3GJ8xQ/cYFxGpSakc5T6VMCtbWS9Usc0YQFfLZquiIrjiCrjnntLxiRPhzDOr3Vz3GBcRSR3dKU6q98030LNnuEa8uJg3aQLz5oVu9SSKOVR9j3EREdk5KuhSuf/8J3Srt2sHH30UYieeCIWF4bHPPtu1O91jXEQkdVTQpTR3GDcuFPLDDiuJ3347bN0Kzz1Xfl7yJOke4yIiqaOCLsH69eGmL3XqhBnOir35Zijyv/lNKPI7QfcYFxFJHd04u7abPx/694fVq0ti++0Hr78OP/pRjb5V8cC34nPmXbqEYq4BcSIiO09H6LXVU0+FI+699y4p5pdeGmY7mzOnxot5sbw8WLgQ+vYNP1XMRURqhgp6bVJUBL/+dSjkZ5xREn/ssdCtfu+9mu1MRCRH6du7NlixAo4+GmbOLIk1ahTu7rLffpnLS0REaoyO0ONs2rRwNN62bUkxP+44WLs2DIJTMRcRiQ0V9Di6665QyAcOLIn98Y/hsrMXXwz3XRcRkVhRl3tcbNgQRphNmlQ6PmUKDB6ckZRERCR9VNBz3WefhcvOvv++JLbPPqGQt29f+XYiIhIr6nLPVc88E7rV99yzpJhffHG47GzePBVzEZFaRgU9l2zdWnLHtmHDSuL/+Ee47Gz8eF12JiJSS+nbPxesXAnHHhsuMytWvz588AF07565vEREJGvoCD2bvfdeOBrfddeSYn7kkbBmDWzerGIuIiLbqKBno3vvDYX84INLYn/4Q+hynzwZmjXLXG4iIpKV1OWeJeps3gxnnQWPP156weTJ4ahcRESkCiromfbFF3DwwQz69tuS2B57wBtvQIcOmctLRERyirrcM+W550K3+o9/DMXF/MILw7nxBQtUzEVEZLuooKfT1q1w7bWhkJ98ckn8oYcomDIF7r8/jF4XERHZTupyT4fvvoMTToB33y2JmYUJU3r2DK8LCjKSmoiIxIOO0FNp+vRQuNu0KSnmQ4bA6tXhaL24mIuIiOwkFfRUGD8+FPL+/Utio0eHIv7669C8ecZSExGReFKXe03ZtAkuuAD++c/S8ZdfhmOOyUxOIiJSa6ig76xFi8K848uWlcS6dIG33oJOnTKXl4iI1Crqct9RL7wQutW7di0p5uefH47UFy5UMRcRkbRSQd8e7vDb34ZCfuKJJfH77gvLHnoIdtklY+mJiEjtpS73ZKxaBSedBP/5T+n4Bx9Ar16ZyUlERCSBjtCr8sEH4Wi8VauSYn744fD99+GIXMW8VsjPD2dW6tQJP/PzM52RiEh5KugVue++UMj79CmJ3XBDuOzszTehZcvM5SZplZ8PI0aEsY/u4eeIESrqIpJ9VNATXXNNKOQXX1wSe+GF8E1+001hmdQqo0bB+vWlY+vXh7iISDbROfRiGzfCbbeF5x06hC72Ll0ym5Nk3Jdfbl9cRCRTVNCLNWwIS5aE27Q2aJDpbCRLdO4cutkriouIZBN1uSfafXcVcyllzBho3Lh0rHHjEBcRySYq6CJVyMsLt+bv0iUMoejSJbzOy8t0ZiIipanLXaQaeXkq4CKS/XSELiIiEgMq6CIiIjGggi4iIhIDKugiIiIxoIIuIiISAyroIiIiMaCCLiIiEgMq6CIiIjGggi4iIhIDKugiIiIxoIIuIiISAyroIiIiMaCCLiIiEgMq6CIiIjGQsoJuZp3MbIqZzTWz2WZ2RRRvbWavmtn86GerhG2uM7MFZvaJmR2bqtxERETiJpVH6FuAq9x9P2AAMNLM9geuBV5z972A16LXRMvOAboDxwH3mlndFOYnIiISGykr6O6+zN3fj56vBeYCHYBTgYej1R4GhkXPTwUmuPsmd/8CWAAclKr8ysrPh65doU6d8DM/P13vLCIisvPqpeNNzKwr0Bt4F2jn7ssgFH0z2y1arQMwLWGzxVEs5fLzYcQIWL8+vF60KLwGyMtLRwYiIiI7x9w9tW9g1hR4Axjj7k+Z2Sp3b5mw/Ht3b2Vm9wDvuPujUfx+4AV3f7LM/kYAIwDatWvXd8KECTud40cfwebN5eO77AIHHLDTu09KYWEhTZs2Tc+bZQG1N97U3nhTezNryJAhM9y9X9l4So/Qzaw+8CSQ7+5PReHlZtY+OjpvD3wTxRcDnRI27wgsLbtPdx8PjAfo16+fDx48eKfzHDoUKvq7xgy2bt3p3SeloKCAmmhLrlB7403tjTe1NzulcpS7AfcDc919XMKiZ4Hzo+fnA88kxM8xswZm1g3YC3gvVfkl6tx5++IiIiLZJpWj3A8FfgYMNbOZ0eME4BbgaDObDxwdvcbdZwMTgTnAS8BIdy9KYX7bjBkDjRuXjjVuHOIiIiK5IGVd7u4+FbBKFh9ZyTZjgLSX0eKBb6NGwZdfhiPzMWM0IE5ERHJHWka554K8PBVwERHJXbr1q4iISAyooIuIiMSACrqIiEgMqKCLiIjEgAq6iIhIDKigi4iIxIAKuoiISAyooIuIiMRAymdbSyUz+xZYlOk8asiuwIpMJ5FGam+8qb3xpvZmVhd3b1s2mNMFPU7MbHpF0+HFldobb2pvvKm92Uld7iIiIjGggi4iIhIDKujZY3ymE0gztTfe1N54U3uzkM6hi4iIxICO0EVERGJABT1FzOwBM/vGzD5OiN1uZvPMbJaZTTKzlgnLrjOzBWb2iZkdmxDva2YfRcvuMjNLd1uSUVF7E5b9xszczHZNiMWyvWZ2WdSm2WZ2W0I8du01s15mNs3MZprZdDM7KGFZrre3k5lNMbO50e/yiije2sxeNbP50c9WCdvkbJuraG8sv7Mqa2/C8tz8znJ3PVLwAAYBfYCPE2LHAPWi57cCt0bP9wc+BBoA3YDPgLrRsveAgYABLwLHZ7ptybY3incCXibcL2DXOLcXGAJMBhpEr3eLeXtfKc4XOAEoiFF72wN9oufNgE+jdt0GXBvFr43L/+Eq2hvL76zK2hu9ztnvLB2hp4i7vwl8Vyb2irtviV5OAzpGz08FJrj7Jnf/AlgAHGRm7YHm7v6Oh385jwDD0tOC7VNReyN3AFcDiYM14treS4Fb3H1TtM43UTyu7XWgefS8BbA0eh6H9i5z9/ej52uBuUAHQtsejlZ7mJL8c7rNlbU3rt9ZVfx+IYe/s1TQM+dCwl9zEP4hfZWwbHEU6xA9LxvPCWZ2CrDE3T8ssyiW7QX2Bg43s3fN7A0z6x/F49reXwO3m9lXwFjguigeq/aaWVegN/Au0M7dl0EoCsBu0WqxaXOZ9iaK5XdWYntz/TurXqbeuDYzs1HAFiC/OFTBal5FPOuZWWNgFKHLrtziCmI53d5IPaAVMADoD0w0sx8T3/ZeClzp7k+a2VnA/cBRxKi9ZtYUeBL4tbuvqeL0aCzaXLa9CfFYfmcltpfQvpz+ztIRepqZ2fnASUBe1EUD4a+6TgmrdSR0Xy6mpIsrMZ4L9iCca/rQzBYScn/fzH5EPNsLIf+nPHgP2Eq4B3Rc23s+8FT0/HGgeFBcLNprZvUJX/b57l7czuVRNyvRz+LTKjnf5kraG9vvrAram/vfWZk6eV8bHkBXSg8iOg6YA7Qts153Sg+4+JySARf/JRzxFQ+4OCHT7Uq2vWWWLaRkgEks2wtcAtwUPd+b0EVnMW7vXGBw9PxIYEZcfr9Rfo8Ad5aJ307pQXG3xaHNVbQ3lt9ZlbW3zDo5952V8Q82rg/gn8Ay4AfCX3EXEQZSfAXMjB5/TVh/FGHk5CckjJIE+gEfR8vuJroZULY9KmpvmeXb/nPEtb3ALsCjUf7vA0Nj3t7DgBnRF927QN8YtfcwQtfprIT/rycAbYDXgPnRz9ZxaHMV7Y3ld1Zl7S2zTs59Z+lOcSIiIjGgc+giIiIxoIIuIiISAyroIiIiMaCCLiIiEgMq6CIiIjGggi6S48xsVDRj1Kxo5rODM50TgJmNNrMlUU4zzeyWHdzPMDPbv6bzE4kb3fpVJIeZ2UDCXbz6uPumaLrHXVL8nnXdvSjJ1e9w97E7+ZbDgOcINzgRkUroCF0kt7UHVnjJDG8r3H0pgJkdF81lPTWap/m5KD7azH5TvAMz+ziaoAIze9rMZkRH/CMS1ik0s5vM7F1gYDQH9BvRui8X3w41GZVta2Z7mNlLUfwtM9vXzA4BTiFMAjPTzPbY2Q9MJK5U0EVy2ytAJzP71MzuNbMjAMysIfB34GTgcOBHSe7vQnfvS7j71eVm1iaKNyHc9vVgwl3h/gz8JFr3AWBMJfu7MqHL/djo/tmVbTseuCyK/wa4193fBp4F/sfde7n7Z0m2Q6TWUZe7SA5z90Iz60so2kOAf5nZtYRbWX7h7vMBzOxRYETle9rmcjM7LXreCdgLWAkUESayANgH6AG8Gs0+VpdwW9iKlOpyN7MeFW0bzXp1CPB4woxmDZLIV0QiKugiOS46n10AFJjZR4RZ0GZS+TSOWyjdO9cQwMwGE6Y/Heju682soHgZsDHhvLkBs9194A6kW+G2ZtYcWOXuvXZgnyKCutxFcpqZ7WNmeyWEegGLgHlAt4RzzucmrLMQ6BNt34cwexRAC+D7qJjvS5hBqiKfAG2jAXmYWX0z655kyhVu62Hu7S/M7MwobmZ2YLTNWqBZkvsXqbVU0EVyW1PgYTObY2azgP2B0e6+kdDF/ryZTSUU+WJPAq3NbCZwKfBpFH8JqBft5w/AtIre0N03Az8BbjWzDwm9AYckk2w12+YBF0Xx2cCpUXwC8D9m9oEGxYlUTrOtidQCUXf6b9z9pEznIiKpoSN0ERGRGNARuoiISAzoCF1ERCQGVNBFRERiQAVdREQkBlTQRUREYkAFXUREJAZU0EVERGLg/wO8FbzYXhiIAQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot\n",
    "plot_lls_sol(x, y, theta_ls)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### <img src=\"https://img.icons8.com/dusk/64/000000/timer.png\" style=\"height:50px;display:inline\"> Residual Analysis\n",
    "---\n",
    "* Purposes\n",
    "    * Examine for **linearity assumption**\n",
    "\n",
    "<img src=\"./assets/tut_08_res_1.jpg\" style=\"height:200px\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* Examine for constant variance for all levels of x\n",
    "\n",
    "<img src=\"./assets/tut_08_res_2.jpg\" style=\"height:200px\">\n",
    "\n",
    "* Evaluate normal distribution assumption"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    " * Graphical Analysis of Residuals\n",
    "    * Can plot residuals vs. x\n",
    "    * Can create histogram of residuals to check for normality"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x174bfec15c0>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfYAAAFNCAYAAADy5k0KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de3hdZZn38e+vKaGUlkIKlENLy1FpGUAIhIoOQVDAQYiOCkIVFO0LnnDmVaFmZqxoFZRXHEYcrCOItCOCQHVU5DSG0UsJUAYRKCeBQks5NWBbCg1t7/ePtdLuptnJTrIPydPf57r2lb2edbrv3TZ3n2c9ey1FBGZmZpaGEbUOwMzMzMrHhd3MzCwhLuxmZmYJcWE3MzNLiAu7mZlZQlzYzczMEuLCbpYgSW+X9Eit49hSSFolaa9ax2EGLuxmSHpK0rHd2s6U9PtaxdRdHs+6vICskHSfpBOLbR8Rv4uIN1UzxoGSFJL26dY2W9K8WsXUnaTtJV0h6TlJKyU9Kum8rvURMSYinqhljGZdXNjNho8/RsQYYHvgh8C1khq6byRpZNUjS98lwBhgf2AccBLwl5pGZFaEC7tZCSTtL6lN0iuSHpR0UsG6NkkfL1je0NtX5hJJL0j6q6T7JR2Qr9ta0sWSnpb0vKTLJW3TVywRsR64AtgG2EtSs6Qlks6T9BxwZVdbQUyTJN0g6UVJyyV9t2DdxyQtkvSypJslTS7yGfxG0qe7tf1J0vt6y7McJL1V0t35se+W9NaCdZuMuBT29iWNkjQvz/mVfN8J+bpxkn4oaZmkpZK+JqmuSAiHAf8ZES9HxPqIeDgiflZwzpC0j6Td8lGVrtdqSVGwXUmftdlguLCb9UHSVsB/AbcAOwOfAeZLKmWo+13A3wL7kfW0TwGW5+suytsPBvYBdgf+pYR4RgIfB1YBj+XNuwANwGRgZrft64BfAouBKfl5rsnXtQBfAt4H7AT8DvhJkVP/J/ChguNOzc/3qz7yHJR8VOJXwKXAeODbwK8kjS9h9zPIetiT8n3PBl7L110FrCX77N+S5/DxHo4BcCcwR9JHJe1b7GQR8Ww+LD8mH125kYF91mYDFxF++bVFv4CnyIrkKwWv1cDv8/VvB54DRhTs8xNgdv6+Dfh4wbozC/Z9B/AocES3/QW8Cuxd0DYdeLJIjGeSFaFXgJfICs2x+bpmoBMYVbB9M7Ck4LgvAiN7OO5NwFkFyyPy3Cf3sO3YPObJ+fIc4Ire8izx8w9gRbfP/3VgXr7+w8Bd3fb5I3BmwZ/fsQXrZhfs+zHgD8CB3fafAKwBtilo+xDw2yIxbkNWlBcCbwCPAyd0y2Gfbvucl2+/TX8/a7/8GszLPXazTEtEbN/1Aj5ZsG434JnIhsC7LCbr+fYqIv4b+C5wGfC8pLmStiPrsY0GFuZDxK8Av8nbi7kzj2/HiDgiIm4rWPdiRLxeZL9JwOKIWNvDusnAvxbE0EH2n47NcouIlWQ951PzplOB+X3kWapDun3+Fxas243s8y5U0ucPXA3cDFwj6VlJ38xHYCYDWwHLCnL/PtmIzGYi4rWI+HpEHErW878WuK6nOQ4Akk4AziX7e9U1QlDyZ202GC7sZn17FpgkqfDfyx7A0vz9q2RFussuhTtHxKV5QZhGNlT9BbJe92vAtIKCNi6y4duB6O0xjc8AexSZVPcM8H8Ki2pEbBMRfyhyrJ8AH5I0nawX+9sNAfScZzk8S1YUC5X0+UfEGxHxlYiYCrwVOBH4CFnea4AdC/LeLiKm9RVMRKwAvg5sC+zZfX1+ieYq4IMR8UzBqv5+1mYD4sJu1rd2suLxRUlbSWoG3kN+7RS4D3ifpNHKvrZ1VteOkg6T1JT3El8lG2Jel/f+fwBcImnnfNvdJR1XgfjvApYBF0raNp9QdmS+7nJglqRpeQzjJH2gl2P9mqzIXgD8tGsUo1ieZYr/18B+kk6TNFLSKcBUsnkDkH3+p+Z/No3A+7t2lHS0pL/J5xmsIBtGXxcRy8jmTPw/SdtJGiFpb0lH9RSApH/Oc6yXNIqsN/4K8Ei37bYDfg78U0R0/7pkfz9rswFxYTfrQ0R0kn296QSynvb3gI9ExMP5JpeQXeN+nqynNr9g9+3ICvjLZMPHy4GL83XnkV2rvVPSCuA2oOzfPY+IdWT/EdkHeBpYQja5jYi4kWwS3zV5DA/keRY71hrgBuBYssl0XYrmKelLkm4aRPzLyXra/zc/7heBEyPipXyTfwb2zs/9lW5x7QL8jKyoLwLuALq+H/8RoB54KN/3Z8CuxcIAriT7838WeCfwdxGxqtt2h5D9GX67cHZ8nke/PmuzgVJEbyN4ZmZmNpy4x25mZpYQF3YzM7OEuLCbmZklxIXdzMwsIS7sZmZmCUniKVA77rhjTJkypdZhlMWrr77KtttuW+swqsb5ps35ps351tbChQtfiojN7laZRGGfMmUK99xzT63DKIu2tjaam5trHUbVON+0Od+0Od/aktT9VsuAh+LNzMyS4sJuZmaWEBd2MzOzhCRxjd3MzIaXN954gyVLlvD668WeNjz0jBs3jkWLFlX9vKNGjWLixIlstdVWJW3vwm5mZlW3ZMkSxo4dy5QpU5BU63BKsnLlSsaOHVvVc0YEy5cvZ8mSJey552ZPCe6Rh+LNzKzqXn/9dcaPHz9sinqtSGL8+PH9GtlwYTczs5pwUS9Nfz8nF3azUs2fD1OmwIgR2c/58/vaw8wSdPnll/PjH/94s/annnqKAw44YMDHbW5uLss9WXyN3awU8+fDzJmwenW2vHhxtgxw+um1i8vMBi0iiAhGjCitr3v22WdXOKLBcY/drBStrRuLepfVq7N2M6u8Mo+YPfXUU+y///588pOf5JBDDuHqq69m+vTpHHLIIXzgAx9g1apVAJx//vlMnTqVAw88kNb83/vs2bO5+OKLAVi4cCEHHXQQ06dP57LLLttw/B/96Ed8+tOf3rB84okn0tbWBsA555xDY2Mj06ZN48tf/vKg8uiJe+xFfOfO7/Diqy9W/bzTVk+j9fYtp1gMm3z3Xgx797RiMfQj/mGTb5k437QNJt8TG05k6YqlJW27zbU3sv1nv8iI117LGhYvZv0nPsErr73Max9874DO/9zK53jkkUe46LsXcc4XzuETMz7B1TdczehtR3PZJZcx+xuzOfMTZ3Ld9ddxxz13IIk1L69h6YqlrFizgnUj17F0xVJmnDGDr37zq0x/23S++k9fZe36tSxdsZSXX3uZVZ2rNuS4Zu2aDeeeM2cODQ0NrFu3jmOOOYb777+fAw88cEB59MSFvYgXX32RydtPrvp569fU1+S8tTJs8q1rgI6OzdsbGqAf8Q+bfMvE+aZtMPnWjaijvq6+pG3HXXDRxqKeG/Haa4y74CLWfeiUAZ1/q7qtmLTHJKYfMZ1bbrqFxx5+jPcel/0n4Y3ON2hsamT8DuMZNWoU5332PI497lhOeOcJ1NXVUac66kbU8fqq11n515UcddRRAJx6+qnccdsd1NfVM3LESOq0MccgNpz72muvZe7cuaxdu5Zly5bx0EMPubCbVV1LC8ybB52dG9vq67N2M6uoEUt67tkXay/V6NGjgewa+1HvOIrv/+j7m21z8x0387u233Hjz27kisuv4IabbtiwLiKKzlivG1nH+vXrNyyveT3rsT/55JNcfPHF3H333eywww6ceeaZZb9Jj6+xm5WiqQlmzMh66JD9nDEjazezilo/cfd+tffXoYcdyl133sUTf3kCgNWrV/OXx/7CqlWrWPHXFRx73LF87aKv8cCfH9hkv3Hbj2PsdmO58w93AnD9T6/fsG7SHpN44M8PsH79epYuWcp9994HwIoVK9h2220ZN24czz//PDfddFNZcijkHrtZqZqaXMjNauDV2a2M/fQ/ooLh+NhmG16dXZ75DDvutCOXXn4pZ3/0bNasyXrWs/5lFmPGjuEjp3wk61EHfO3rX9ts30svv5RzzzmXbUZvw9HHHL2hvWl6E3tM3oOjDj+KN099MwcclH0N7qCDDuItb3kL06ZNY6+99uLII48sSw6FFBF9bzXENTY2Rrmfx956e2tNrpU1PN9Ax4QeruUmyvmmzfmmbTD5HjzyYPbcp7RbpAJs/dPr2Xb2HEYsWcr6ibvz6uxW1pzy9wM690DVra1j3ch1A9q3c10nu2838BGGRYsWsf/++2/SJmlhRDR239Y9djMzG/LWnPL3VS/kw5WvsZuZmSXEhd3MzCwhLuxmZlZ9kX1dzPrW38/Jhd3MzKpudaxm5V9Xurj3oet57KNGjSp5H0+eMzOzqnti/RPwAox+cTQMk6e31q2rY11dP2fFr+mE1atZx3pWvPEQ7LADbLttvw4xatQoJk6cWPL2LuxmZlZ1a1nLo+sfrXUY/dLvr/e1t2+4Y+XicTDnv4HRo2Hu3Io+FdJD8WZmZpWwYMGmt6GGqjwV0oXdzMysEnp6cBTA009X9LQu7GZmZpXQ9WyJ7vbYo6KndWE3MzOrhJaW7CmQhUaPhjlzKnramhd2SXWS/lfSL/PlBkm3Snos/7lDrWM0MzPrt+5PhZw8ueIT52BozIo/F1gEbJcvnw/cHhEXSjo/Xz6vVsGZmZkNWNdTIV9ZDMdUtqfepaY9dkkTgb8D/qOg+WTgqvz9VUBLteMyMzMbrmo9FP8d4IvA+oK2CRGxDCD/uXMtAjMzMxuOajYUL+lE4IWIWCipeQD7zwRmAkyYMIG2trayxjdt9TTq19T3vWGZ1a2to+H5IjMpE+R80+Z80+Z8Szdm3Ziy16lianmN/UjgJEnvBkYB20maBzwvadeIWCZpV+CFnnaOiLnAXIDGxsZobm4ua3Ctt7cyefvJZT1mKfp9Z6NhzvmmzfmmzfmWbvErizmt+bQyR9Szmg3FR8SsiJgYEVOAU4H/jogZwC+AM/LNzgB+XqMQzczMhp1aX2PvyYXAOyU9BrwzXzYzM7MSDIWvuxERbUBb/n45cEwt4zEzMxuuhmKP3czMzAbIhd3MzCwhLuxmZmYJcWE3MzNLiAu7mZlZQlzYzczMEuLCbmZmlhAXdjMzs4S4sJuZmSXEhd3MzCwhLuxmZmYJcWE3MzNLiAu7mZlZQlzYzczMEjIkHttqZhXS3g4LFkBHBzQ0QEsLNDXVOiozqyAXdrNUtbfDvHnQ2Zktd3Rky+DibpYwD8WbpWrBgo1FvUtnZ9ZuZslyYTdLVUdH/9rNLAku7GapamjoX7uZJcGF3SxVLS1QX79pW3191m5myfLkObNUdU2Q86x4sy2KC7tZypqaXMjNtjAeijczM0uIC7uZmVlCXNjNzMwS4sJuZmaWEBd2MzOzhLiwm5mZJcSF3czMLCEu7GZmZglxYTczM0uIC7uZmVlCXNjNzMwS4sJuZmaWkJoVdkmTJP1W0iJJD0o6N29vkHSrpMfynzvUKkYzM7PhppY99rXA/42I/YEjgE9JmgqcD9weEfsCt+fL1TN/PkyZAl//OsyaBe3tVT29mZnZYNSssEfEsoi4N3+/ElgE7A6cDFyVb3YV0FK1oObPh5kzYfHibLmjA+bNc3E3M7NhY0hcY5c0BXgL0A5MiIhlkBV/YOeqBdLaCqtXb9rW2QkLFlQtBDMzs8FQRNQ2AGkMcAcwJyJukPRKRGxfsP7liNjsOrukmcBMgAkTJhx6zTXXDD6YhQs3vH12LNSvK1g3efLgj1+CurV1rBu5ru8NE+F80+Z80+Z8S9e5rpPdxu5W1niOPvrohRHR2L29poVd0lbAL4GbI+LbedsjQHNELJO0K9AWEW/q7TiNjY1xzz33DD6gKVM2DMO3vgMm/zVvb2iAb3xj8McvQcPzDXRM6KjKuYYC55s255s251u6xa8sZs4xc8oaj6QeC3stZ8UL+CGwqKuo534BnJG/PwP4edWCmjMHRo/etK2+Hlqqd5nfzMxsMEbW8NxHAh8G/izpvrztS8CFwLWSzgKeBj5QtYhOPz372doKLM566i0t0NRUtRDMzMwGo2aFPSJ+D6jI6mOqGcsmTj89e93eCttX57q6mZlZuQyJWfFmZmZWHi7sZmZmCXFhNzMzS0if19glHUd297fdgQCeBX4eEb+pcGxmZmbWT70WdknfAfYDfgwsyZsnAp+VdEJEnFvh+MzMzKwf+uqxvzsi9uveKOmnwKOAC7uZmdkQ0tc19tclHd5D+2HA6xWIx8zMzAahrx77mcC/SxrLxqH4ScCKfJ2ZmZkNIb0W9vyxqk2SdiGbPCdgSUQ8V43gzMzMrH9KmRUvYDIbZ8XXSXo+av1YODMzM9tMX7Pi3wV8D3gMWJo3TwT2kfTJiLilwvGZmZlZP/TVY/9X4NiIeKqwUdKewK+B/SsUl5mZmQ1AX7PiR7Jx0lyhpcBW5Q/HzMzMBqOvHvsVwN2SrgGeydsmAaeSPUvdzMzMhpC+ZsV/Q9LPgZOA6eSz4oHTI+KhKsRnZmZm/dDnrPi8gLuIm5mZDQO9XmOXNE7ShZIelrQ8fy3K27avVpBmZmZWmr4mz10LvAw0R8T4iBgPHA28AlxX6eDMzMysf/oq7FMi4qLCO81FxHMRcSGwR2VDMzMzs/7qq7AvlvRFSRO6GiRNkHQeG2fJm5mZ2RDRV2E/BRgP3CGpQ1IH0AY0AB+scGxmZmbWT70W9oh4OSLOi4g3R0RD/to/b+uoVpCWoPZ2mDULFi/Ofra31zoiM7Mk9NVjL0rSR8sZiG1B2tth3jzoyP9v2NGRLbu4m5kNWp/fY+/FV4AryxWIbUEWLIDOzk3bOjuz9qam2sSUqvb27HPt6ICGBmhp8Wdslri+nu52f7FVwIQi68x611HkKk6xdhuYrpGRrv9EdY2MgIu7WcL66rFPAI4j+y57IQF/qEhElr6Ghp6LeEND9WNJmUdGzLZIfV1j/yUwJiIWd3s9RTY73qz/Wlqgvn7Ttvr6rN3KxyMjZlukvh4Cc1Yv604rfzi2RejqLS5YkP30td/K8MiI2Rap35PnJNUDW0XEqxWIx7YUTU3Z6/kG+MY3ah1NmlpaNr3GDh4ZMdsC9Pl1N0nnSnpT/v4osjvOPSbpk5UOzswGoakJZszY2ENvaMiWPTJilrRSeuxnAN/N3/8T2bPZHwDuBL5XobjMrBy6RkbMbIvR19fdvgzsBrTmQ/AHk82SPx4YK+lfgLaI+J+KR2pmZmZ96mvy3FckNeXbNQDXR8QFkkYAx0fEBdUI0szMzEpTyi1lzwK2BpYDX8jb9gP+o1JBmZmZ2cD0eY09IpYB53Vrexh4uFJBmZmZ2cAM+CEwlSbpeEmPSHpc0vm1jsfMzGw4GJKFXVIdcBlwAjAV+JCkqbWNyszMbOgbkoUdOBx4PCKeiIhO4Brg5BrHZGZmNuSVdOc5SVsDfw9MKdyngrPidye7EU6XJYC/jGtmZtaHUm8p+3Pgr8BCYE3lwtlAPbTFJhtIM4GZABMmTKCtra2sAez17HTWPh19b1hu260n7t+1+uetFeebNuebNudbsr1G7lb2OlVMqYV9YkQcX9FINrUEmFR4fuDZwg0iYi4wF6CxsTGam5vLGsCtrTB5clkPWZpoQ2801+DENeJ80+Z80+Z8S/bE43DWh8sbTjGlXmP/g6S/qWgkm7ob2FfSnvkd704FflHF85uZmQ1LpfbY3wacKelJsqF4ARERB1YiqIhYK+nTwM1AHXBFRDxYiXOZmZmlpNTCfkJFo+hBRPwa+HW1z2tmZjac9fUQmO0iYgWwskrxmJn1rr0dFiyAjo7sUbQtLX6CnVmBvnrs/wmcSDYbPth0tnoAe1UoLjOzzbW3w7yrobMzW+5Yni2Di7tZrq+nu52Y/9yzOuGYmfViwYKNRb1LZ2fW7sJuBgzdO8+ZmW2uo6N/7WZbIBd2Mxs+Ghr61262BXJhN7Pho6UF6us3bauvz9rNDCj9625Iehuwb0RcKWknYExEPFm50MzMuum6ju5Z8WZFlfoQmC8DjcCbgCuBrYB5wJGVC83MrAdNTS7kZr0odSj+vcBJwKsAEfEsMLZSQZmZmdnAlFrYOyMiyJ+wJmnbyoVkZmZmA1VqYb9W0veB7SV9ArgN+EHlwjIzM7OBKOkae0RcLOmdwAqy6+z/EhG3VjQyMzMz67dSJ8/tCfyuq5hL2kbSlIh4qpLBmZmZWf+UOhR/HbC+YHld3mZmZmZDSKmFfWREbLhBc/6+vpftzczMrAZKLewvSjqpa0HSycBLlQnJzMzMBqrUO8+dDcyX9F2yR7c+A3ykYlGZmZnZgJQ6K/4vwBGSxgCKiJWVDcvMzMwGotfCLmlGRMyT9I/d2gGIiG9XMDYzMzPrp7567F13mPPtY83MzIaBXgt7RHxfUh2wIiIuqVJMZmZmNkB9zoqPiHVkD4AxMzOzIa7UWfF/yGfE/5T8CW8AEXFvRaIyMzOzASm1sL81/3lBQVsA7yhvOGZmZjYYpX7d7ehKB2JmZmaD1+s1dklNkv4kaZWkP0rav1qBmZmZWf/1NXnuMuDzwHjg28B3Kh6RmZmZDVhfhX1ERNwaEWsi4jpgp2oEZWZmZgPT1zX27SW9r9hyRNxQmbDMzMxsIPoq7HcA7ymyHIALu5mZ2RDS153nPlqtQMzMzGzwSn0eu5mZmQ0DLuxmZmYJcWE3MzNLSEmFXdJoSf8s6Qf58r6SThzoSSV9S9LDku6XdKOk7QvWzZL0uKRHJB030HOYmZltiUrtsV8JrAGm58tLgK8N4ry3AgdExIHAo8AsAElTgVOBacDxwPfyx8aamZlZCUot7HtHxDeBNwAi4jVAAz1pRNwSEWvzxTuBifn7k4Fr8hviPAk8Dhw+0POYmZltaUot7J2StiH77jqS9ibrwZfDx4Cb8ve7A88UrFuSt5mZmVkJSn1s65eB3wCTJM0HjgTO7G0HSbcBu/SwqjUifp5v0wqsBeZ37dbD9lHk+DOBmQATJkygra2tzyT6Y9o0qK8v6yFLUle3ioaGtuqfuEacb9qcb9qcb+nGjIEyl6miSn1s662S7gWOICu+50bES33sc2xv6yWdAZwIHBMRXcV7CTCpYLOJwLNFjj8XmAvQ2NgYzc3NJWRSutZWmDy5rIcsSUNDGx0dzdU/cY0437Q537Q539ItXgynnVbeeIopdVb8kcDrEfErYHvgS5IGXPYkHQ+cB5wUEasLVv0COFXS1pL2BPYF7hroeczMzLY0pV5j/3dgtaSDgC8Ai4EfD+K83wXGArdKuk/S5QAR8SBwLfAQ2dD/pyJi3SDOY2ZmtkUp9Rr72ogISScDl0bED/Oh9AGJiH16WTcHmDPQY5uZmW3JSi3sKyXNAmYAf5t/t3yryoVlZmZmA1HqUPwpZF9vOysiniP7Ctq3KhaVmZmZDUips+KfA75dsPw0g7vGbmZmZhVQUmGXtJKN3yevJxuGXxUR4yoVmJmZmfVfqT32sYXLklrwrV7NzMyGnAE9tjUiFgDvKHMsZmZmNkilDsW/r2BxBNBIkVu9mpmZWe2U+nW39xS8Xws8RfYkNjMzMxtCSr3G/tFKB2JmZmaDV+q94idKulHSC5Kel3S9pIl972lmZmbVVOrkuSvJHtCyG9nNaf4rbzMzM7MhpNTCvlNEXBkRa/PXj4CdKhiXmZmZDUCphf0lSTMk1eWvGcDySgZmZmZm/VdqYf8Y8EHgOWAZ8P68zczMzIaQUmfFPw2cVOFYzMzMbJB6LeyS/o1ebkQTEZ8te0RmZmY2YH312O8peP8V4MsVjMXMzMwGqdfCHhFXdb2X9LnCZTMzMxt6+vMQGN8b3szMbIgb0NPdzMzMbGjqa/LcSjb21EdLWtG1CoiI2K6SwZmZmVn/9HWNfWy1AjEzM7PB81C8mZlZQlzYzczMEuLCbmZmlhAXdjMzs4S4sJuZmSXEhd3MzCwhLuxmZmYJcWE3MzNLiAu7mZlZQlzYzaqhvR1mzYLFT2c/29trHZGZJaqv57Gb2WC1t8O8q6GzEwjoWJ4tAzQ11TQ0M0uPe+xmlbZgQV7UC3R2Zu1mZmXmwm5WaR0d/Ws3MxuEmhZ2SZ+XFJJ2LGibJelxSY9IOq6W8ZmVRUND/9rNzAahZoVd0iTgncDTBW1TgVOBacDxwPck1dUmQrMyaWmB+vpN2+rrs3YzszKrZY/9EuCLQBS0nQxcExFrIuJJ4HHg8FoEZ1Y2TU0w48PQMB5Q9nPGhz1xzswqoiaz4iWdBCyNiD9JKly1O3BnwfKSvM1seGtqyl4NbfCNj9Q6GjNLWMUKu6TbgF16WNUKfAl4V0+79dAWPbQhaSYwE2DChAm0tbUNLNAipk3bfPS0GurqVtHQ0Fb9E9eI802b802b8y3dmDFQ5jJVVMUKe0Qc21O7pL8B9gS6eusTgXslHU7WQ59UsPlE4Nkix58LzAVobGyM5ubmssUO0NoKkyeX9ZAlaWhoo6OjufonrhHnmzbnmzbnW7rFi+G008obTzFVv8YeEX+OiJ0jYkpETCEr5odExHPAL4BTJW0taU9gX+CuasdoZmY2XA2pO89FxIOSrgUeAtYCn4qIdTUOy8zMbNioeWHPe+2Fy3OAObWJxszMbHjznefMzMwS4sJuZmaWEBd2MzOzhLiwm5mZJcSF3czMLCEu7GZmZglxYTczM0uIC7uZmVlCXNjNzMwS4sJuZmaWEBd2MzOzhNT8XvFmZmYV194OCxZARwc0NEBLCzQ11TqqinBhNzOztLW3w7yrobMzW+5Yni1DksXdQ/FmZpa2BQs2FvUunZ1Ze4Jc2M3MLG0dHf1rH+Zc2M3MLG0NDf1rH+Zc2M3MLG0tLVBfv2lbfX3WniBPnjMzs7R1TZDzrHgzM7NENDUlW8i781C8mZlZQlzYzczMEuLCbmZmlhAXdjMzs4S4sJuZmSXEhd3MzCwhLuxmZmYJcWE3MzNLiAu7mZlZQlzYzczMEuLCbmZmlhAXdjMzs4S4sJuZmSXEhd3MzCwhLuxmZmYJqVlhl/QZSY9IelDSNwvaZ0l6PF93XK3iMzMzG45G1uKkko4GTgYOjIg1knbO26cCpwLTgN2A2yTtFxHrahGnmZnZcFOrHvs5wIURsQYgIl7I208GromINRHxJFoPBPAAAAkGSURBVPA4cHiNYjQzMxt2alXY9wPeLqld0h2SDsvbdweeKdhuSd5mZmZmJajYULyk24BdeljVmp93B+AI4DDgWkl7Aeph+yhy/JnATIAJEybQ1tZWhqg3mjYN6uvLesiS1NWtoqGhrfonrhHnmzbnmzbnW7oxY6DMZaqoihX2iDi22DpJ5wA3REQAd0laD+xI1kOfVLDpRODZIsefC8wFaGxsjObm5jJFnmlthcmTy3rIkjQ0tNHR0Vz9E9eI802b802b8y3d4sVw2mnljaeYWg3FLwDeASBpP6AeeAn4BXCqpK0l7QnsC9xVoxjNzMyGnZrMigeuAK6Q9ADQCZyR994flHQt8BCwFviUZ8SbmZmVriaFPSI6gRlF1s0B5lQ3IjMzszT4znNmZmYJqdVQvFlx7e2wYAF0dEBDA7S0QFNTraMyMxsWXNhtaGlvh3lXQ2dnttyxPFsGF3czsxJ4KN6GlgULNhb1Lp2dWbuZmfXJPfYidtop+95htY0ZU5vz1spm+XaMAbbdfMMOQQKfyxb/55s455u2weS7007ljaU3LuxFfO5ztTlvW1v1bmIwFGyW7/wf9PwvZ/JkmNNarbAqZov/802c803bcMnXQ/E2tMyZA6NHb9o2enTWbmZmfXJht6Hl9NNh7tyshy5lP+fOzdrNzKxPHoq3oef0013IzcwGyD12MzOzhLiwm5mZJcSF3czMLCEu7GZmZglxYTczM0uIC7uZmVlCXNjNzMwS4sJuZmaWEEVErWMYNEkvksQjQgDYEXip1kFUkfNNm/NNm/OtrckRsdnjZZIo7CmRdE9ENNY6jmpxvmlzvmlzvkOTh+LNzMwS4sJuZmaWEBf2oWdurQOoMuebNuebNuc7BPkau5mZWULcYzczM0uIC3uFSbpC0guSHiho+5akhyXdL+lGSdsXrJsl6XFJj0g6rqD9UEl/ztddKknVzqUUPeVbsO7zkkLSjgVtSeYr6TN5Tg9K+mZBe3L5SjpY0p2S7pN0j6TDC9YN93wnSfqtpEX5n+W5eXuDpFslPZb/3KFgn2Gbcy/5Jvk7q1i+BeuH5++siPCrgi/gb4FDgAcK2t4FjMzfXwRclL+fCvwJ2BrYE/gLUJevuwuYDgi4CTih1rmVmm/ePgm4mex+AzumnC9wNHAbsHW+vHPi+d7SFS/wbqAtoXx3BQ7J348FHs3z+iZwft5+fir/hnvJN8nfWcXyzZeH7e8s99grLCL+B+jo1nZLRKzNF+8EJubvTwauiYg1EfEk8DhwuKRdge0i4o+R/Q36MdBSnQz6p6d8c5cAXwQKJ3Wkmu85wIURsSbf5oW8PdV8A9gufz8OeDZ/n0K+yyLi3vz9SmARsDtZblflm13FxviHdc7F8k31d1Yvf74wjH9nubDX3sfI/ncH2V+oZwrWLcnbds/fd28fFiSdBCyNiD91W5VkvsB+wNsltUu6Q9JheXuq+X4O+JakZ4CLgVl5e1L5SpoCvAVoByZExDLIigOwc75ZMjl3y7dQkr+zCvMd7r+zRtbqxAaSWoG1wPyuph42i17ahzxJo4FWsqG8zVb30Das882NBHYAjgAOA66VtBfp5nsO8A8Rcb2kDwI/BI4loXwljQGuBz4XESt6uXyaRM7d8y1oT/J3VmG+ZPkN699Z7rHXiKQzgBOB0/OhG8j+lzepYLOJZMOaS9g49FXYPhzsTXYt6k+SniKL/V5Ju5BmvpDFf0Nk7gLWk91jOtV8zwBuyN9fB3RNnksiX0lbkf3Snx8RXXk+nw+/kv/sutwy7HMukm+yv7N6yHf4/86q1cX9LekFTGHTyUbHAw8BO3XbbhqbTsx4go0TM+4m6wF2Tcx4d63zKjXfbuueYuNElCTzBc4GLsjf70c2dKeE810ENOfvjwEWpvLnm8f3Y+A73dq/xaaT576ZQs695Jvk76xi+XbbZtj9zqr5B5v6C/gJsAx4g+x/dWeRTbh4Brgvf11esH0r2UzLRyiYVQk0Ag/k675LfnOhofbqKd9u6zf8I0k1X6AemJfHfy/wjsTzfRuwMP+F1w4cmlC+byMbUr2/4N/ru4HxwO3AY/nPhhRy7iXfJH9nFcu32zbD7neW7zxnZmaWEF9jNzMzS4gLu5mZWUJc2M3MzBLiwm5mZpYQF3YzM7OEuLCbJUJSa/6EqvvzJ6011TomAEmzJS3NY7pP0oUDPE6LpKnljs8sNb6lrFkCJE0nuyvYIRGxJn/MZH2Fz1kXEetK3PySiLh4kKdsAX5JdqMUMyvCPXazNOwKvBQbnyj3UkQ8CyDp+PxZ2r/PnxP9y7x9tqTPdx1A0gP5gzCQtEDSwnwEYGbBNqskXSCpHZieP4P6jnzbm7tus1qKYvtK2lvSb/L230l6s6S3AieRPWzmPkl7D/YDM0uVC7tZGm4BJkl6VNL3JB0FIGkU8APgPcDbgV1KPN7HIuJQsrtpfVbS+Lx9W7LbyTaR3WXu34D359teAcwpcrx/KBiKPy6/P3exfecCn8nbPw98LyL+APwC+EJEHBwRfykxD7MtjofizRIQEaskHUpWvI8GfirpfLJbZD4ZEY8BSJoHzCx+pA0+K+m9+ftJwL7AcmAd2QMzAN4EHADcmj/trI7sdrM92WQoXtIBPe2bP2XrrcB1BU9Q27qEeM0s58Juloj8encb0Cbpz2RPXbuP4o+PXMumo3ajACQ1kz12dXpErJbU1rUOeL3gurqAByNi+gDC7XFfSdsBr0TEwQM4ppnhoXizJEh6k6R9C5oOBhYDDwN7FlyT/lDBNk8Bh+T7H0L2tCqAccDLeVF/M9kTq3ryCLBTPnEPSVtJmlZiyD3uG9mzv5+U9IG8XZIOyvdZCYwt8fhmWywXdrM0jAGukvSQpPuBqcDsiHidbOj9V5J+T1bsu1wPNEi6DzgHeDRv/w0wMj/OV4E7ezphRHQC7wcukvQnstGBt5YSbB/7ng6clbc/CJyct18DfEHS/3rynFlxfrqb2RYkH2b/fEScWOtYzKwy3GM3MzNLiHvsZmZmCXGP3czMLCEu7GZmZglxYTczM0uIC7uZmVlCXNjNzMwS4sJuZmaWkP8PGVwdqc5EEdwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "residuals = x_with_bias @ theta_ls - y\n",
    "# plot\n",
    "fig = plt.figure(figsize=(8,5))\n",
    "ax = fig.add_subplot(111)\n",
    "ax.scatter(x, residuals, label='residual', color='r')\n",
    "ax.fill_between(np.sort(x.reshape(-1,)), np.max(residuals), alpha=0.4, color=\"g\")\n",
    "ax.fill_between(np.sort(x.reshape(-1,)), np.min(residuals), alpha=0.4, color=\"b\")\n",
    "ax.set_xlabel(\"Square Feet\")\n",
    "ax.set_ylabel(\"House Price in $1000\")\n",
    "ax.set_title(\"House Price vs. House Size\")\n",
    "ax.grid()\n",
    "ax.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### <img src=\"https://img.icons8.com/doodle/96/000000/close-sign.png\" style=\"height:50px;display:inline\"> Closed-Form Leaset-Squares Solution\n",
    "---\n",
    "To find the value of $\\theta$ that minimizes the cost function, there is a *closed-form solution* - a mathemtical equation that gives the result directly. It is also called the **Normal Equation**. We will now derive it.\n",
    "\n",
    "* We wish to find a solution for $\\hat{y} = X \\theta$\n",
    "* The parameters $\\theta$ are obtained by minimzing the *sum of squared* errors or residuals (SSE): $$ SSE(\\theta) = \\sum_{i=1}^n (\\theta^T x_i - y_i)^2 = ||X \\theta -y ||_2^2 = (X\\theta -y)^T(X\\theta - y) = \\theta^TX^TX\\theta -\\theta^TX^Ty - y^TX\\theta +y^Ty$$\n",
    "* Minimizing w.r.t to $\\theta$: $$\\nabla_{\\theta}SSE(\\theta) = 2X^TX\\theta - 2X^Ty = 0 \\rightarrow \\theta^{*} = (X^TX)^{-1}X^Ty $$\n",
    "* The matrix $(X^TX)^{-1}X^T$ is the *Pseudo Inverse* of $X$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### <img src=\"https://img.icons8.com/dusk/64/000000/double-down.png\" style=\"height:50px;display:inline\"> Gradient Descent Solution\n",
    "---\n",
    "Iterative solution, as we have seen in tutorial 7\n",
    "\n",
    "#### <img src=\"https://img.icons8.com/dusk/64/000000/waypoint-map.png\" style=\"height:30px;display:inline\">Stochastic Gradient Descent (Mini-Batch Gradient Descent)\n",
    "---\n",
    "* **Pseudocode**:\n",
    "    * **Require**: Learning rate $\\alpha_k$\n",
    "    * **Require**: Initial parameter $w$\n",
    "    * **While** stopping criterion not met **do**\n",
    "        * Sample a minibatch of $m$ examples from the training set ($m=1$ for SGD)\n",
    "        * Set $\\tilde{X} = [x_1,...,x_m] $ with corresponding targets $\\tilde{Y} = [y_1,...,y_m]$\n",
    "        * Compute gradient: $g \\leftarrow 2\\tilde{X}^T\\tilde{X}w - 2\\tilde{X}^T \\tilde{Y}$\n",
    "        * Apply update: $w \\leftarrow w - \\alpha_k g$\n",
    "        * $k \\leftarrow k + 1$\n",
    "    * **end while**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### <img src=\"https://img.icons8.com/dusk/64/000000/graph.png\" style=\"height:50px;display:inline\"> Basis Function Expansion\n",
    "---\n",
    "* What if the data is actually more complex than a simple linear line? But your data has just one feature per sample.\n",
    "* Linear Regression can be made to model **non-linear** relationships by replacing $x$ with some non-linear functions of the inputs $\\phi(x)$: $$ y = \\theta_0 + \\theta \\phi(x) + \\epsilon $$ This is also called **feature extraction**.\n",
    "* The model is **still linear** in the parameters $\\theta$ - it is still called *linear regression*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### <img src=\"https://img.icons8.com/nolan/64/000000/fit-to-width.png\" style=\"height:50px;display:inline\"> Example - Polynomial Fitting (Polynomial Regression)\n",
    "---\n",
    "* Given $L$ measurements $\\{x_k, y_k\\}_{k=1}^L$ originating from the $n^{th}$ degree polynomial with additive white Gaussian noise $ y= h(x) + \\epsilon$ where: $h(x)=\\sum_{i=0}^n \\theta_ix^i$ and $\\epsilon \\sim \\mathcal{N}(0, \\sigma^2)$ i.i.d\n",
    "* **Goal**: find the coefficients $\\{\\theta_i\\}_{i=0}^n$ to best fit the polynomial $h(x)$\n",
    "* For each measurement $(x_k,y_k)$ we will fit the following model: $$ y_k = \\theta_0 x_k^0 + \\theta_1 x_k^1 + \\theta_2 x_k^2 +... + \\theta_n x_k^n , \\forall k = 1,2,...,L$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img src=\"./assets/tut_08_poly_1.jpg\" style=\"height:300px\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    " \n",
    "* **The Loss Function**: the optimization goal is to minimize the mean squared error $$ f_{\\theta}(x) = \\sum_{k=1}^L (y_k - (\\theta_0 x_k^0 + \\theta_1 x_k^1 + \\theta_2 x_k^2 +... + \\theta_n x_k^n))^2 $$\n",
    "* Denote: $$ \\Phi(X) = \\begin{bmatrix}\n",
    "  x_1^0 & \\cdots & x_1^n \\\\\n",
    "  \\vdots & \\vdots & \\vdots \\\\\n",
    "  x_L^0 & \\cdots & x_L^n\n",
    " \\end{bmatrix}, \\theta = \\begin{bmatrix}\n",
    "  \\theta_0 \\\\\n",
    "  \\vdots \\\\\n",
    "  \\theta_n \n",
    " \\end{bmatrix}, y =  \\begin{bmatrix}\n",
    "  y_1 \\\\\n",
    "  \\vdots \\\\\n",
    "  y_L\n",
    " \\end{bmatrix}$$\n",
    "* **The Least-Squares Solution**: $$ \\theta^{*} = (\\Phi(X)^T\\Phi(X))^{-1}\\Phi(X)^Ty $$\n",
    "* Example: we will fit $y=0.5x^2 + x + 2 + \\epsilon$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "# let's try this\n",
    "L = 100 # num of samples\n",
    "X = np.sort(6 * np.random.rand(L, 1) - 3, axis=0)\n",
    "epsilon = np.random.randn(L , 1)\n",
    "y = 0.5 * X**2 + X + 2 + epsilon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'y = 0.5*x^2 + x + 2 + epsilon')"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAe4AAAFNCAYAAADGn4wWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3df5hdV13v8c830wk9ZEqmCA7NoZCidfTSILkdUYn4TMqPFCwSg2gREUGNXq/8uhpN4EpRgcQb5YI+erUX0KqlU6RxrBQMXMOIVFubMMFQ0tg+1f44aQO1nZChUzOZfO8f55zOOSfn9+yz9157v1/Pkycz++zZe501c/Z3r7W+a21zdwEAgDCsSroAAACgewRuAAACQuAGACAgBG4AAAJC4AYAICAEbgAAAkLgBlLKzFaZ2TvM7CeSLguaM7N5M3te5es/NbP3JV0mZB+BG2hgZuvN7PNm9riZ3WVmL2uz73vNbLFyAZ+vvZA37LO+j6L8nqTtkj5sZj/TcMxvNbMbzOy4mZ00s1vN7Hv7OEfizOx3zOxuMztVqe+fSrpM3XL3EXe/N+lyIF8I3MC5bpA0K+lbJL1b0ifN7Jlt9r+xcgEfqb2Qm9m7zOwllX3OM7N3m9n3dVMAM3uPpBdL+kFJk5Lea2Y/XLPLiKQ7JF0u6emSrpN0i5mNdHHs9Wb2792Uo19mNmlmM13u/k1Jr5a0VtKbVL5ReXGX5/n3Pm+KgGARuBEMM9thZjc1bPt9M/tQhOf4Dkn/VdI17r7g7jdJOiLptX0c7sOSrpR0taQ/kvRVd7/NzH7czO41s6dVzvlKM3u4enNgZj8v6VWSrnD3R9z9TklXSPodM/sBSXL3e939g+7+kLsvufu1klZLGl9RBdQws9VmdtjM3lr5fqjSsn9PVOeQJHe/xt3vcvez7n67pH+Q9P1RnkOSzOwtZnbUzB4zs/1m9tya19zM3lb5vTxiZnvNbFXltW83s7+v9Gw8YmY3Nvzct7c438+Z2T1m9qiZ3Wxm6xp+7hcqPQ2PmdkfmJlF/Z6RTQRuhOQvJF1pZqOSZGbnSfpxSX/ebGcz+5SZzbX496kW53i+pHvd/VTNti9Xtrfy6srF+U4z+28Nr3nN/0uS5O43SvonSb9nZt8i6aOSftbdv155/Y/d/fvcfe7Jg7jf7e7f4e5fbPFeX6hy4L6nTTl74u6nJf2kpN80s++StFPSkKT3R3WORmZWkPQ9ku6M+LhbJb1L0jZJz1T55uCGht1+RNKEyjdur5H0lsr235L0WUkXSnq2pN/v4nxXSNot6cckXSTpPklTDbtdpfJ7/e7Kflt6fFvIqfOSLgDQLXd/yMy+IOl1kv6vyq3ZR9z9UIv9r+rjNCOSTjZsOymp2GL/T0i6VtIJSd8r6SYzm3P3GyS9XeUL/hmVbzquNrOvufttkv67pH+RNCPpb9y91Y1ER5WW+59L+g13byz7irj7VyoJV38laUzSi9x9KcpzNPgjlW+U9kd83J+XtNvdj0qSmX1A0rvM7Lnufl9ln99290clPVrpxXm9pI9IWpT0XEnr3P1BSU1vnhq8QdLH3P1LlfPtkvSYma1393+v7LOncnM2Z2afl/RCSX8bxZtFttHiRmiuU7kVqMr/TVvbKzAv6WkN254m6VSTfeXuX3X345Xu6n9UuXv8RyuvfcDdv1DZ9Yy7v68StFW5YP+lpMsk/W6/ha20UP9G0m3uvrvNfj9R7W1Q+YbhOQ09EM9pc5rrJK2X9Gl3v7vNOXbWnONTkn6g9hxdvJe9KtfHj3mLpx+Z2XMajvkcSf9Ss61VBv5zVR47r/7co5JM9TdkD9R8fZ+katf2r1b2/edKr8pb1Nm6yjEkSe4+L+k/Gs73cM3Xj6t80wh0ROBGaKYlvcDMLlO5q/H6Vjua2WesPtu79t9nWvzYnZKeZ2YX1Gz7bnXfdesqX+SXN7i/t6aVVS3bC1Xuir1B5ezxnpnZU1Suj5LKLcrWhXL/uLuPuvuopBdIur/6feXf/W1+/A9VDsRbqmPsLc6xp+YcV0n6Yu05OryX35D0SkmvcPdvtDnH/Q3HvF/SC2q2fbzFjz4g6ecb3nOhcrNVdXHN18+RdLxyzofd/efcfZ3K9fyHrca1axxX+Wah+v7WqJzsWOrwc0BHBG4Exd2fkPRJSR+X9M/tAo67v7Ih27v23ytb/My/Sjos6RozO9/MfkTlQHdTs/3N7DVmdqGVvUjS2yT9dbv3YGbnq9x1/i5Jb5ZUNLNf7Pjm648xrHI9LEj6KXc/28vP93CeN6qcuf7TKr+366yLzPUez7FL0k9Ierm7/0eUx67xR5J2mdnzK+dca2ava9hnR+V3ebHKwxw3VvZ9nZk9u7LPY6rJV2jj45LebGYvrNxgfUDS7Y03cEA/CNwI0XWSNij6bvKqq1VOUnpM0h5JP1pNHDOzl5jZfMO+96jclf5nKo+TXtfh+LslPeju/8fd/1PlLv/3mdmlPZTxxSq3al+h8hhptSfhJR1+rmuV7vMPqXxjMF9pzR6U9L+jOkfFB1Ru4d5d8z7eFeUJ3P2vJP22pCkz+4akr6jcwq/115IOqXzjdovKSYNSOYHs9srv/WZJb3f3f+twvr+T9Osq3/A9JOnbVP5bAVbMWgwlAalVCSh3SXpWu25VoFtm5pIudffIsvKBQaHFjaBYeW7t/5A0RdAGkEdMB0MwKgk+J1TO1r0y4eIAQCLoKgcAICB0lQMAEBACNwAAAQlijPsZz3iGr1+/PtJjfvOb39SaNWsiPWaoqIt61Mcy6qIe9bGMuqgXdX0cOnToEXdv+lTCIAL3+vXrdfDgwUiPOTMzo8nJyUiPGSrqoh71sYy6qEd9LKMu6kVdH2Z2X6vX6CoHACAgBG4AAAJC4AYAICAEbgAAAkLgBgAgIARuAAACQuAGACAgBG4AAAISxAIsAABEbXq2pL37j+n43ILWjRa0Y8u4tm4sJl2sjgjcAIDcmZ4tade+I1pYXJIkleYWtGvfEUlKffCmqxwAkDt79x97MmhXLSwuae/+YwmVqHu0uAEAuXN8bqHp9tLcgjbtOZDq7nNa3ACA3Fk3Wmi63VQO3q7l7vPp2VKsZeuEwA0AyJ0dW8ZVGB6q22aSvGG/NHafE7gBALmzdWNRu7dtUHG0IJNUHC2cE7SrWnWrJ4UxbgBALm3dWKwbv96054BKTYJ0q271pAysxW1mHzOzr5nZV2q2Pd3MPmdmd1f+v3BQ5wcAoBfNus8Lw0PasWU8oRI1N8iu8j+VdGXDtp2S/s7dL5X0d5XvAQBIXLPu893bNqQuq3xgXeXu/gUzW9+w+TWSJitfXydpRtKvDaoMAAD0orH7PI3iTk4bc/eHJKny/7fGfH4AAIJm7q3y6CI4eLnF/Sl3v6zy/Zy7j9a8/pi7Nx3nNrPtkrZL0tjY2OVTU1ORlm1+fl4jIyORHjNU1EU96mMZdVGP+lhGXdSLuj42b958yN0nmr0Wd1b5CTO7yN0fMrOLJH2t1Y7ufq2kayVpYmLCJycnIy3IzMyMoj5mqKiLetTHMuqiHvWxjLqoF2d9xN1VfrOkN1W+fpOkv475/AAABG2Q08FukPRPksbN7EEz+xlJeyS93MzulvTyyvcAAKBLg8wqf32Ll146qHMCAJB1LHkKAEBACNwAAASEwA0AQEAI3AAABITADQBAQAjcAAAEhMANAEBA4l7yFACAIEzPlrR3/zEdn1vQutGCdmwZT8WTwwjcAAA0mJ4tade+I1pYXJIkleYWtGvfEUlKPHgTuAEAQRpki3jv/mNPBu2qhcUl7d1/jMANAECvBt0iPj630NP2OJGcBgAITrsWcRTWjRZ62h4nAjcAIDiDbhHv2DKuwvBQ3bbC8JB2bBmP5PgrQeAGAARn0C3irRuL2r1tg4qjBZmk4mhBu7dtSHx8W2KMGwAQoB1bxuvGuKXoW8RbNxZTEagbEbgBAMGpBtQ0zrMeNAI3ACBIaW0RDxpj3AAABITADQBAQAjcAICgTc+WtGnPAV2y8xZt2nNA07Ol2M99pHQytnMzxg0ACFaSa4rXnfvi+M5NixsAEKxBr6CWxnMTuAEAwUpyTfGkzk3gBgAEK8k1xZM6N4EbABCsJNcUT+rcJKcBAIKV5ApqteeWTqkY07kJ3ACAoCW5glr13DMzM3rrGyZjOSdd5QAABITADQBAQAjcAAAEhMANAEBACNwAAASEwA0AQEAI3AAABITADQBAQAjcAAAEhJXTAADo0vRsKZHlVWsRuAEA6ML0bEm79h158hncpbkF7dp3RJI0GmM56CoHAKDG9GxJm/Yc0CU7b9GmPQc0PVuSVH6YSDVoVy0sLlUeMhIfWtwAAFS0a1Ufn1to+jPl7WviKmIygdvM3inpZyW5pCOS3uzuTyRRFgBAvrQbp27Xql43WlCpSfBeN1qIpdxVsXeVm1lR0tskTbj7ZZKGJF0ddzkAAPlTbVGX5hbkWm5RV7vD27Wqd2wZV2F4qG57YXhIO7aMD7rYdZIa4z5PUsHMzpP0VEnHEyoHACBHOo1Tt2o9rxstaOvGol57eVFDZpKkITO99vL4nwUee+B295Kk35F0v6SHJJ1098/GXQ4AQP60H6dW21b19GxJNx0qacldkrTkrpsOlZ5srcfFvFKA2E5odqGkmyT9uKQ5SX8p6ZPu/hcN+22XtF2SxsbGLp+amoq0HPPz8xoZGYn0mKGiLupRH8uoi3rUx7JQ6+LYw6d0eunsOdtXD63S+LMukCTNLSzqxMkndHrprFYPrdLY2vM1Whhu+7PFEYu0PjZv3nzI3SeavZZEctrLJP2bu39dksxsn6QXS6oL3O5+raRrJWliYsInJycjLcTMzIyiPmaoqIt61Mcy6qIe9bEs1LqYa8gal8ot6t3bNmiyQ5f3m3feIm/SUW2S/uTKNbHVRxJj3PdL+j4ze6qZmaSXSjqaQDkAADmzdWNRu7dtUHG0IJNUHC1o97YNXY1Ttxv/jlPsLW53v93MPinpS5LOSJpVpWUNAMCgbd3YX0LZji3j57TWJenx02c0t2BRFa+jRLLK3f0ad/9Od7/M3d/o7v+ZRDkAAOhWtbU+Whiu2/7Y44sqPbYQW5IaS54CANClrRuLWvOUczurz7rHtvQpgRsAgB50mlI2aARuAAB6kHSSGoEbAIAeNFukZZVZbEuf8nQwAAB6UPtAkuqDSooXLsW29CmBGwCAHjVOKZuZmYnt3HSVAwAQEAI3AAABoascAAIxPVuqG1fdsWU89kdKInkEbgAIwHTDwzFKcwvate+IJBG8c4aucgAIwN79x85ZI3thcSm21bqQHrS4ASAASa/WFZosDysQuAEgAOtGCyo1CdJxP1IyBN0OK4Qa3OkqB4AANFutqzA8FNtqXSHpZlihGtxLcwtyLQf3uJ7wtRIEbgAIQPWRksXRgkxScbSg3ds2BNFCjFs3wwoh5wzQVQ4AgWhcrQvNdTOsEHLOAC1uAECmdDOskPQTvlaCwA0AyJRuhhVCzhmgqxwAkDmdhhWaPeErlKxyAjcA4ByhTpXqRag5AwRuAECdXpZXzUOATxvGuAEAdbqdKhXyXOiQEbgBAHW6nSoV8lzokBG4AQB1up0qFfJc6JARuAEAdbqdKhXyXOiQEbgBAHW6XV415LnQISOrHABwjm6mSoU8FzpkBG4AQN9CnQsdMrrKAQAICIEbAICAELgBAAgIgRsAgIAQuAEACAiBGwCAgBC4AQAICIEbAICAELgBAAgIK6cBAGIxPVtiedQIELgBAAM3PVvSrn1Hnnx+d2luQbv2HZEkgnePEukqN7NRM/ukmd1lZkfN7PuTKAcAIB579x97MmhXLSwu6R03HtamPQc0PVtKqGThSarF/WFJf+vuP2pmqyU9NaFyAABWoNvu7+NzCy2PUZpb0I6//LIkWt/diL3FbWZPk/SDkj4qSe5+2t3n4i4HAGBlqt3fpbkFuZa7v5u1nteNFtoea/Gs67033zmgkmZLEl3lz5P0dUl/YmazZvYRM1uTQDkAAH2aW1jUL3/iy027v/fuP3bO/ju2jKswPNTxmOjM3D3eE5pNSLpN0iZ3v93MPizpG+7+6w37bZe0XZLGxsYun5qairQc8/PzGhkZifSYoaIu6lEfy6iLetRH2dzCohYXHtfDrXu/taG4tunPnTj5hE4vne3p50IQ9d/G5s2bD7n7RLPXkgjcz5J0m7uvr3z/Ekk73f2HWv3MxMSEHzx4MNJyzMzMaHJyMtJjhoq6qEd9LKMu6lEfZZv2HNDVF5/S7x5pniZVHC3o1p1XtPz5jb/5WT32+Lmt6wufOqzZ97wisnLGKeq/DTNrGbhj7yp394clPWBm45VNL5X01bjLAQDoT7tEs8LwkHZsGW/5uiRd8+rna3jI6rYND5muefXzIylf1iWVVf5WSddXMsrvlfTmhMoBAOhROdHs1Dnbh8y0e9uGjpnh1ddZjKU/iQRudz8sqWkXAAAg3XZsGVfp6KG6bYXhoa6CdtXWjUUCdZ9YqxwA0JOtG4sqXlhQcbQgU3lMu5egjZVhyVMAQM9GC8O6dedk0sXIJVrcAAAEhBY3AGQAT97KDwI3AASOJ2/lC13lABC4Vk/earb0KMJH4AaAwLVaEKXdQikIF4EbAALX6slbnZ7IhTARuAEgcM2evNXN0qMIE8lpABA4lhDNl46B28x+SdL17v5YDOUBAPSBJUTzo5uu8mdJusPMPmFmV5qZdfwJAAAwEB0Dt7v/T0mXSvqopJ+WdLeZfcDMvm3AZQMAAA26Sk5zd5f0cOXfGUkXSvqkmf2vAZYNAAA06GaM+22S3iTpEUkfkbTD3RfNbJWkuyX96mCLCAAAqrrJKn+GpG3ufl/tRnc/a2ZXDaZYAACgmY6B293f0+a1o9EWJ34szA8ACEmu53GzMD8AIDS5XjmNhfkBAKHJdeBmYX4AQGhyHbhZmB8AEJpcB24W5gcwSNOzJW3ac0CX7LxFm/Yc0PRsKekiIQNynZzGwvwABoXkVwxKrgO3xML8AAajXfIr1xysRK67ygFgUEh+xaDkvsXdLxZuAdDOutGCSk2CNMmvWCla3H2ojl2V5hbkWh67IvEEQNVKk19JbEMrBO4+sHALgE62bixq97YNKo4WZJKKowXt3rahq545Ggdoh67yPjB2BaAb/Sa/pi2xrdnQ4GjspUAVLe4+sHALgEFKU+OgVet/bmEx9rKgjMDdBxZuATBIaWoctGr9nzj5ROxlQRmBuw8rGbsCgE7S1Dho1co/vXQ25pKgijHuPrFwC4BBSdOqjq2mta0eot2XFAJ3xjHfHAhTWhoHO7aM1y3dKpVb/2NrVydYqnwjcGcYayUDWKlWrf/Rk3cnXLL8InBnWNqmlAAIU7PW/8wMgTspDFJkWJqmlAAAokHgzrA0TSkBkB3TsyUde/gUy7EmhMCdYWmaUgLkVdbWHK/mzpxeOstyrAkhcGcY882BZGVxzXGe1ZC8xJLTzGxI0kFJJXe/KqlyZF1appQAaTXIKZNZTBAldyZ5SWaVv13SUUlPS6oAcc5xZj41kD6DnjKZxSDHc8aTl0hXuZk9W9IPSfpIEueXpLmFxdi6sLLYXQZkwaC7fbOYIEruTPKSGuP+kKRflZTYYrcnTj4R2zgNY0JAOg26RZzFIFfNnVk9tIrcmYSYu8d7QrOrJL3K3X/RzCYl/UqzMW4z2y5puySNjY1dPjU1FWk5vvboSZ1o8dncUFwb6bmOlE62fC3qc/Vjfn5eIyMjSRcjNaiPZWmpi7mFRZ04+YROL53V6qFVGlt7vkYLwys+7rGHTzV9WMbqoVUaf9YF52zvpz4GUfZB1Ucv0vK3kRZR18fmzZsPuftEs9eSCNy7Jb1R0hlJ56s8xr3P3X+y1c9MTEz4wYMHIy3HH0/9jXYfPrfDoTha0K07r4j0XJv2HGg6JjSIc/VjZmZGk5OTSRcjNaiPZWmoi8ZxaKncao2ildfs2CbJVf58NuaiZL0+epGGukiTqOvDzFoG7ti7yt19l7s/293XS7pa0oF2QXtQxtaeH1sXVhq7y2rnlh57+BTj7UitQQ411U6ZlJaDtpTeXBSG3pDbedyjheHY5jinbT51Y7Lc6aWzqbxAAdLgx6G3bizq1p1XqDhaUGP/YxoDYhYz1dGbRB8y4u4zkmaSOn+7Oc5RT99K03zqLM4txcqkebpiXNOPQgmIo08d1mOPL56zPeRMdfQmdy3uahfxkdLJlssPZn36VigXKMQj7X/vcQ01hTB1a3q2pPknzpyzfXjIgs5UR29yFbhrL1BS6wtU1seQQrhAIT5p/3uPeqip1drhacxFabR3/zEtnj03oXjN6vNS00OCwcvV87i77SJu1fIszS1o054DqepG7MeOLeNNs1LTdIFCfNr3wKzp+XiD6HaPaqipm5XS0jpkILX+XZ1cOLfrHNmVq8DdbRdxqzE1KfolEZPQeIFaPbSKBRQSkJZx5SjHkAe9hOhKdbp5T1MuSjMsNwopZ13l3XYRN+syq5WmbsR+VTNp/23PD2n8WRek+mKVRWkaV46yizjt3e6tbshDye8IoTsfg5erwN3tH33j3M5mQvmgI53SFOCiHENOc+Lj9GxJ1uK1UFqsaZtaimTkqqu8totYOtV0ZaTafbduLLZc9SyUDzrSKW0BLqou4jR35e7df+ycedpSedGVkFqsae/Ox+DlqsUtLXcRbyiu1a07r+j4AaBrKhqtMnnzKquZ/Wn+vLS6KXKlY/wd6FbuAnev6JpauTSN56ZFmgPcSqT589LqpqjdkBiQRrnqKu8XXVMrw0pt5wph6lG/0vp5YRoksoLAjYFL23huWqQpwKVlatogZflmCflC4MbApTlhCe3nXo8mWbABSNPNEtAvxrhjlNcErayO52ZFmqamAeiMFndM0r6i1CDRRZluUS95CmCwCNwxyXuCFl2U6cVQRljykI+A9gjcMclighYXkGxom2198u4ES5Z9vX6G8txzh2WMccckawtuMDc7O9I897qVLOSL9PMZIh8BEi3u2GRtDmneu/6zJqShjKy0Ovv5DGWx5w69o8UdkxBbNe1wAUFSstLq7OczlLWeO/SHFneMQmrVdEJCE5KSlZvGfj5DWeu5Q39ocaMvzM1GUrLS6uznM5S1njv0hxY3+sLcbCRl83c+U9ffdn/dIzpDvGns9zOUpZ479IfAjb5xAcmXNEz/m54t6aZDpbqgbZJee3mYf4t8htAPAjeAjtKSyd0sMc0lff6ur8dWBiBpjHED6CgtmdxZSUwDVoLADaCjtATMrCSmAStB4AbQUVoCJrMZAAI3EJuQl+lMS8BkOhRAchoQiyiTu5LI7mb6H5AeBG5kWhqmMEnRre3e6QZgkO83DVOX0pLdDiSJwJ1BaQlWSUvTRT6q5K5O2d1peb+DwsNtAMa4M4fHbS5LyxQmqXNyV7fj3+1uANL0fgclLdntQJII3BnT78U75MSpVtJ0kW+X3NXLzVa7G4A0vd9BGUR2exb/9pFtBO6M6efiPbewmMlWelqmMEnts6F7udlqdwOQpvc7KFFnt9NDhRAxxp0x/Twq8MTJJ7SwWH8Pl4Vxw7Q9ArFVclcvN1udsrvjer9J5VFEnd3OmDlCRODOmE7BqtkF9/TSWTXrfAm9izWUKUy93my1ugGI6/0mnfQXZXZ7tzdN1c/N1Ref0rv3HEjl3xHyg8CdMe0u3s0uuO+88bB+5QXW9FhZ6GJNwxSmTqLsGYjj/Waplbq2MKy5hcWm26vqPjcXZzNbH2EhcGdQq4t3qycrnXXX8JBpcWn5YYksIxmfUHoGqrKUBGfN71nrtmfpRgXZQODOkXYX1jWrz9Oap5wXROCIWxzjuSH0DFT1k0cxSCv5/cw9fm5ru3F7lm5UkA0E7hxpdcGVpJMLizp8zStiLlH6JT2em0ZpSvpb6e+nm5uQtN2oALFPBzOzi83s82Z21MzuNLO3x12GvNqxZVwtega5CLWQh0VNerXSB31EOW96pb+fbqaXpeUBK0BVEi3uM5J+2d2/ZGYXSDpkZp9z968mUJZc2bqxqIP3Parrb7tfXrN9lRkXoRboJm2u3679qHswVvr76Sa/oHYf6ZSKDCUhYbEHbnd/SNJDla9PmdlRSUVJBO4YvG/rBk089+l1F6rihUtBXoTiGHummzRaUSd6RfH76eYmpLrPzMyM3vqGyV6LCUTK3L3zXoM6udl6SV+QdJm7f6Phte2StkvS2NjY5VNTU5Gee35+XiMjI5EeM1RR1cXcwqJOnHxCp5fOavXQKo2tPV+jNdNqojS3sKjSYws6W/P3u8pMxQsLKz5nbX0M8jwhiPpzcqR0suVrG4prez5e3L8frhvLqIt6UdfH5s2bD7n7RLPXEgvcZjYi6e8lvd/d97Xbd2Jiwg8ePBjp+WdmZjQ5ORnpMUMVRV00doFK5XHAXsY+e7Fpz4GmLa3iaEG37rxiRcdurI88P20t6s/JIH5vcf5+uG4soy7qRV0fZtYycCeSVW5mw5JuknR9p6CNMMQ91zXOseeQpmqlXbOMdFN5rHtTnyuS8ftB3iSRVW6SPirpqLt/MO7zYzDiTuLKwwM1sqg2I10qB+1qnx8P+AC6k8TTwTZJeqOkK8zscOXfqxIoByIUdyBlik4yopjKtXVjUbfuvELF0YIaB+ryPtUO6EYSWeVflFpOJ0ag4l6UI7RlQtOu1Tjx3MKiNu05oONzC1pbGNY3T595cmncpKdyAXnFymmIRBKBNMqxzdrAtfOFZzU3W8rNTUCrudUH73tUFz2+oNJcuWej2cM4kp7KBeQRgTtCec4+lroPpGmrp8bAdXrpbK6WNW2VWHjD7Q/oHZd1nnXSbws5TUunAiFJYow7k6oX/9Lcglwk2rSSxnrK+7KmrQLvUpdTRfttIa906VQgr2hxR4RH/3UnjfWU97HWVl3WQ62eeVljpS1kpnIBvaPFHZEsXvyjfBhEVav6aPXUsjjkfWpZqwz913/vxVrVELyHV5kufOowLWQgQbS4I5K1RJtBPc6yVT1Z5ZxJBIG8j7W2Syyc/szDKo4OpSYfAQCBOzJZu/gPqkt7x5ZxvfPGw+fM30XISW4AAAiQSURBVPXKOXs5dlRJbo2Ba/XQKr328qL27j+md954uK9jpy0Br5NWXdajhWHdunMy/gIBaInAHZGszSseVNf/1o1FvePGwys+dtQ9ArWBa/ozn9OH/rHU97EH1VvRr9BuIgC0R+COUJYSbQbZ9V9scWyXul6vepBJbidOPqGFxfr0j16OHVcCXjcBOW03EQBWjuQ0NDXIJUWbHbuq2+lhg0wGPL10dkXHjiNRsdtpdXmf6gZkEYEbTQ1yjm3jgyYadRNYBpkJvnqo+cei22PHkaXebUDO4mwHIO/oKkdLg+z6rx77kp23nJOoJnUOLINMBhxbe74Kw0t9HzuORMVuA3LWZjsAoMWNCKxkvne/rdNB9giMFoZXdOw4VgTrtt54ihqQPbS48aR+so9Xmvy0ktZpqx6Bdu+j2/e40t6GQScqdltvWZvtAIDAjYp+A/BKM6ijDizt3oekzGRY91JvWZrtAIDAjYp+A3AUyU9RBpZOSVtpWyd9JQjIQD4RuCGp/wCctuSnft4HGdbhYnEZ5BHJaZDUf5JY2pKf2r2PvD9MJGvS+IhYIA4EbkjqPwCn7ZnK7d5H2m4y0LvaGQy//Ikvs7gMcomuckhaWZJYmsZau3kfdK2GqTHxcMmbrQDA0Aeyj8CNJ6UpAK9Eu/eRlfeYR80SD5th6ANZR1c5gCB005Jm6AN5QIsbbZG1i7RoNYNhyExn3fn7RG4QuNESj4REmrRaLS7JZEggCXSVoyUeCYk0SdsMBiAptLjREo+ERNqQXAgQuNFG2lZFCwm5AQAGha5ytMSCJf1hRS8Ag0TgRkuMKfaH3AAAg0RXOdpiTLF35AYAGCRa3EDEeJgJgEEicAMRIzcAwCDRVQ5EbCUPbAGATgjcORLyFKXQyk5uAIBBIXDnRLvlS0eTLFgXWHo1H0K7OQOSwhh3ToQ8RSnksqM7zH0HukfgzomQpyiFXHZ0h5szoHsE7pwIeYpSyGVHd7g5A7pH4M6JkKcohVx2dIebM6B7iQRuM7vSzI6Z2T1mtjOJMuRNyMuXhlx2dIebM6B7sWeVm9mQpD+Q9HJJD0q6w8xudvevxl2WvAl5ilLIZUdnzH0HupfEdLAXSbrH3e+VJDObkvQaSQRuIMe4OQO6k0RXeVHSAzXfP1jZBgAAOjB3j/eEZq+TtMXdf7by/Rslvcjd39qw33ZJ2yVpbGzs8qmpqUjLMT8/r5GRkUiPGSrqoh71sYy6qEd9LKMu6kVdH5s3bz7k7hPNXkuiq/xBSRfXfP9sSccbd3L3ayVdK0kTExM+OTkZaSFmZmYU9TFDRV3Uoz6WURf1qI9l1EW9OOsjia7yOyRdamaXmNlqSVdLujmBcgAAEJzYW9zufsbMfknSfklDkj7m7nfGXQ4AAEKUyENG3P3Tkj6dxLkBAAgZK6cBABAQAjcAAAGJfTpYP8zs65Lui/iwz5D0SMTHDBV1UY/6WEZd1KM+llEX9aKuj+e6+zObvRBE4B4EMzvYao5c3lAX9aiPZdRFPepjGXVRL876oKscAICAELgBAAhIngP3tUkXIEWoi3rUxzLqoh71sYy6qBdbfeR2jBsAgBDlucUNAEBwchu4zey3zOxfzOywmX3WzNYlXaYkmdleM7urUid/ZWajSZcpKWb2OjO708zOmllus2bN7EozO2Zm95jZzqTLkyQz+5iZfc3MvpJ0WZJmZheb2efN7Gjlc/L2pMuUFDM738z+2cy+XKmL34jlvHntKjezp7n7Nypfv03Sf3H3X0i4WIkxs1dIOlBZS/63Jcndfy3hYiXCzL5L0llJfyzpV9z9YMJFip2ZDUn6V0kvV/mJfndIer27fzXRgiXEzH5Q0rykP3P3y5IuT5LM7CJJF7n7l8zsAkmHJG3N49+GmZmkNe4+b2bDkr4o6e3uftsgz5vbFnc1aFeskZTPO5gKd/+su5+pfHubyo9bzSV3P+rux5IuR8JeJOked7/X3U9LmpL0moTLlBh3/4KkR5MuRxq4+0Pu/qXK16ckHZVUTLZUyfCy+cq3w5V/A48luQ3ckmRm7zezByS9QdJ7ki5PirxF0meSLgQSVZT0QM33DyqnF2e0ZmbrJW2UdHuyJUmOmQ2Z2WFJX5P0OXcfeF1kOnCb2f8zs680+fcaSXL3d7v7xZKul/RLyZZ28DrVR2Wfd0s6o3KdZFY3dZFz1mRbrnulUM/MRiTdJOkdDT2YueLuS+7+QpV7KV9kZgMfSknksZ5xcfeXdbnrxyXdIumaARYncZ3qw8zeJOkqSS/1jCc/9PC3kVcPSrq45vtnSzqeUFmQMpXx3JskXe/u+5IuTxq4+5yZzUi6UtJAkxgz3eJux8wurfn2hyXdlVRZ0sDMrpT0a5J+2N0fT7o8SNwdki41s0vMbLWkqyXdnHCZkAKVhKyPSjrq7h9MujxJMrNnVmfgmFlB0ssUQyzJc1b5TZLGVc4evk/SL7h7KdlSJcfM7pH0FEn/Udl0W16z7M3sRyT9vqRnSpqTdNjdtyRbqviZ2askfUjSkKSPufv7Ey5SYszsBkmTKj8B6oSka9z9o4kWKiFm9gOS/kHSEZWvn5L0Lnf/dHKlSoaZvUDSdSp/RlZJ+oS7/+bAz5vXwA0AQIhy21UOAECICNwAAASEwA0AQEAI3AAABITADQBAQAjcAAAEhMANAEBACNwAzmFm31N5Nvv5Zram8qzhXD/OEkgLFmAB0JSZvU/S+ZIKkh50990JFwmACNwAWqisUX6HpCckvdjdlxIuEgDRVQ6gtadLGpF0gcotbwApQIsbQFNmdrOkKUmXSLrI3TP/zHogBJl+HjeA/pjZT0k64+4fN7MhSf9oZle4+4GkywbkHS1uAAACwhg3AAABIXADABAQAjcAAAEhcAMAEBACNwAAASFwAwAQEAI3AAABIXADABCQ/w9LB9Sh3SG11QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# let's see it\n",
    "fig = plt.figure(figsize=(8,5))\n",
    "ax = fig.add_subplot(111)\n",
    "ax.scatter(X, y)\n",
    "ax.grid()\n",
    "ax.set_xlabel(\"x\")\n",
    "ax.set_ylabel(\"y\")\n",
    "ax.set_title(\"y = 0.5*x^2 + x + 2 + epsilon\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "def plot_poly_lls(X, y, phi_x, theta_ls, title=\"\"):\n",
    "    fig = plt.figure(figsize=(8,5))\n",
    "    ax = fig.add_subplot(111)\n",
    "    ax.scatter(X, y, label=\"original\")\n",
    "    ax.plot(X, phi_x @ theta_ls, label=\"least-squares solution\", color='r')\n",
    "    ax.grid()\n",
    "    ax.legend()\n",
    "    ax.set_xlabel(\"x\")\n",
    "    ax.set_ylabel(\"y\")\n",
    "    ax.set_title(title)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(100, 3)\n",
      "theta:\n",
      "[[1.98949504]\n",
      " [0.86612449]\n",
      " [0.5144204 ]]\n"
     ]
    }
   ],
   "source": [
    "# 2 ways - manually or using scikit-learn\n",
    "# manually\n",
    "degree = 2\n",
    "phi_x = np.concatenate([X**d for d in range(degree + 1)], axis=1)  # feature extraction\n",
    "print(phi_x.shape)\n",
    "theta_ls = np.linalg.inv(phi_x.T @ phi_x) @ phi_x.T @ y\n",
    "print(\"theta:\")\n",
    "print(theta_ls)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAe4AAAFNCAYAAADGn4wWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3xUVfrH8c9DDCbSgg0lqLAWLICUgCCKYAMbIFYWCxbY1Z9YVlHQVSwoIK5tXVexgKwoKgKylhUVIxZQSkCUIuiCEitIWCJBQnJ+f9wkJGEmmSQzc6d8369XXpCZO/eeOUnmueec55xjzjlEREQkPtTzuwAiIiISOgVuERGROKLALSIiEkcUuEVEROKIAreIiEgcUeAWERGJIwrcIjHKzOqZ2fVm9ke/yyKBmVm+mf2h5P+TzGy032WSxKfALVKJmbU0s/fNbKuZrTSzk6s49k4zKyz5AM8v/0Fe6ZiWtSjKo8BQ4BEzu6LSOfc1sxfN7Hsz22xmH5vZMbW4hu/M7AEzW21mW0rq+xK/yxQq51xD59w3fpdDkosCt8iuXgRygL2A24BpZrZPFce/VPIB3rD8B7mZ3Wpmx5ccs5uZ3WZmXUMpgJndARwL9AB6AneaWd9yhzQEFgCdgD2B54A3zKxhCOduaWZrQylHbZlZTzPLDvHw34CzgCbApXg3KseGeJ21tbwpEolbCtwSN8xsuJm9Wumxv5vZw2G8xmFAR2CUc67AOfcqsAw4pxanewToA1wIPAEsd87NN7MLzOwbM2tccs3TzOzH0psDM/sTcDpwonNug3PuS+BE4AEzOw7AOfeNc+5B59wPzrki59wEoD7Quk4VUI6Z1TezJWY2rOT7lJKW/R3hugaAc26Uc26lc67YOfcp8CHQLZzXADCzy81shZltMrO3zeygcs85M7u25OeywczGm1m9kucOMbMPSno2NpjZS5Ved0iQ6w0xszVm9quZzTKz5pVe9+eSnoZNZvYPM7Nwv2dJTArcEk+eB/qYWQaAme0GXAD8K9DBZva6meUF+Xo9yDWOAr5xzm0p99jSkseDOavkw/lLM7uq0nOu3L9FAM65l4B5wKNmthfwDHClc+6XkuefdM51dc7llZ3EudXOucOccx8Fea/t8QL3mirKWSPOue3ARcDdZnYEMAJIAe4N1zUqM7N0oDPwZZjP2x+4FRgA7IN3c/BipcPOBrLwbtz6AZeXPH4PMBtoCrQA/h7C9U4ExgDnA/sD64CplQ47E++9Hl1yXO8avi1JUrv5XQCRUDnnfjCzucB5wFN4rdkNzrlFQY4/sxaXaQhsrvTYZiAzyPEvAxOAn4BjgFfNLM859yJwHd4H/g68m44Lzexn59x84P+Az4Fs4N/OuWA3EtUqabn/C7jLOVe57HXinPuiJOFqBtAM6OKcKwrnNSp5Au9G6e0wn/dPwBjn3AoAM7sPuNXMDnLOrSs5Zpxz7lfg15JenIHA00AhcBDQ3Dm3Hgh481TJIOBZ59zikuuNBDaZWUvn3NqSY8aW3Jzlmdn7QHvgP+F4s5LY1OKWePMcXiuQkn8DtrbrIB9oXOmxxsCWAMfinFvunPu+pLv6E7zu8XNLnrvPOTe35NAdzrnRJUGbkg/sV4A2wN9qW9iSFuq/gfnOuTFVHPfH0t4GvBuGAyv1QBxYxWWeA1oCbzrnVldxjRHlrvE6cFz5a4TwXsbj1cf5LsjuR2Z2YKVzHgh8Xu6xYBn4B+GNnZe+7lfAqHhD9l25/68DSru2by459rOSXpXLqV7zknMA4JzLBzZWut6P5f6/Fe+mUaRaCtwSb2YC7cysDV5X45RgB5rZW1Yx27v811tBXvYl8Acza1TusaMJvevW4X3I73zAuTvLtbJKy9Yeryv2Rbzs8Rozs93x6iMXr0UZvFDOveCcy3DOZQDtgG9Lvy/5+raKlz+OF4h7l46xB7nG2HLXOBP4qPw1qnkvdwGnAac65/5XxTW+rXTOb4F25R57IchLvwP+VOk9p5fcbJU6oNz/DwS+L7nmj865Ic655nj1/Hiwce1yvse7WSh9fw3wkh1zq3mdSLUUuCWuOOe2AdOAF4DPqgo4zrnTKmV7l/86LchrvgKWAKPMLM3MzsYLdK8GOt7M+plZU/N0Aa4FXqvqPZhZGl7X+a3AZUCmmV1d7ZuveI5UvHooAC5xzhXX5PU1uM7FeJnrg/He23MWQuZ6Da8xEvgjcIpzbmM4z13OE8BIMzuq5JpNzOy8SscML/lZHoA3zPFSybHnmVmLkmM2US5foQovAJeZWfuSG6z7gE8r38CJ1IYCt8Sj54C2hL+bvNSFeElKm4CxwLmliWNmdryZ5Vc6dg1eV/pkvHHS56o5/xhgvXPun8653/G6/Eeb2aE1KOOxeK3aU/HGSEt7Eo6v5nUhK+k+fxjvxiC/pDW7EHgoXNcocR9eC3d1ufdxazgv4JybAYwDpprZ/4Av8Fr45b0GLMK7cXsDL2kQvASyT0t+7rOA65xz/63meu8Bt+Pd8P0AHIz3uyJSZxZkKEkkZpUElJXAflV1q4qEyswccKhzLmxZ+SKRoha3xBXz5tb+BZiqoC0iyUjTwSRulCT4/ISXrdvH5+KIiPhCXeUiIiJxRF3lIiIicUSBW0REJI7ExRj33nvv7Vq2bBnWc/722280aNAgrOeMV6qLilQfO6kuKlJ97KS6qCjc9bFo0aINzrmAuxLGReBu2bIlCxcuDOs5s7Oz6dmzZ1jPGa9UFxWpPnZSXVSk+thJdVFRuOvDzNYFe05d5SIiInFEgVtERCSOKHCLiIjEkbgY4w6ksLCQ9evXs23btlq9vkmTJqxYsSLMpYpPyVoXaWlptGjRgtTUVL+LIiISsrgN3OvXr6dRo0a0bNkSM6v+BZVs2bKFRo0aVX9gEkjGunDOsXHjRtavX0+rVq38Lo6ISMjitqt827Zt7LXXXrUK2iJmxl577VXrHhsREb/EbeAGFLSlTvT7IyLxKK4Dt98aNmwY1vNNmjSJ77//PqznjHVr166lTZs21R533333Vfj+2GOPjVSRRERimgJ3DImVwO2co7i42O9iVFA5cH/yySc+lUREEsXMnFy6j51DqxFv0H3sHGbm5PpdpJAocIfJ+PHj6dy5M+3atWPUqFFlj/fv359OnTpx1FFHMWHCBACKiooYPHgwbdq0oW3btjz00ENMmzaNhQsXMmjQINq3b09BQUGF8//www/06NGD9u3b06ZNGz788EMAJk6cyGGHHcYJJ5zAkCFDuOaaawAYPHgw06ZNK3t9ae9Afn4+J510Eh07dqRt27a89tprgNfyPeKII7j66qvp2LEj3333HbNnz6Zbt2507NiR8847j/z8fABGjBjBkUceSbt27bjpppt2qYsPPviA9u3b0759ezp06MCWLVtwzjF8+PCy9/zSSy/t8rpJkyaVlR/gzDPPJDs7mxEjRlBQUED79u0ZNGhQhfcT7Lylqxide+65HH744QwaNAjthCcipWbm5DJy+jJy8wpwQG5eASOnL4uL4B23WeWxZPbs2axevZrPPvsM5xx9+/Zl7ty59OjRg2effZY999yTgoICOnfuzDnnnMPatWvJzc3liy++ACAvL4+MjAwee+wxHnjgAbKysna5xgsvvEDv3r257bbbKCoqYuvWrfzwww+MGjWKRYsW0aRJE3r16kWHDh2qLGtaWhozZsygcePGbNiwga5du7J48WIAVq1axcSJE3n88cfZsGEDo0eP5t1336VBgwaMGzeOBx98kGuuuYYZM2awcuVKzIy8vLxdrvHAAw/wj3/8g+7du5Ofn09aWhrTp09nyZIlLF26lA0bNtC5c2d69OgRUv2OHTuWxx57jCVLluzyXFXnzcnJ4csvv6R58+Z0796djz/+mOOOOy6ka4pIYhv/9ioKCosqPFZQWMT4t1fRv0OmT6UKTWIE7uuvhwAf6lVJLyqClJTgB7RvDw8/HNK5Zs+ezezZs8uCZn5+PqtXr6ZHjx48+uijzJgxA4DvvvuO1atX07p1a7755huGDRvGGWecwamnnlrtNTp37szll19OYWEh/fv3p3379rz33nv07NmTffbx1qG/4IIL+Oqrr6o8j3OOW2+9lblz51KvXj1yc3P5+eef2W233TjooIPo2rUrAPPnz2f58uV0794dgO3bt9OtWzcaN25MWloaV155JWeccQZnnnnmLtfo3r07f/nLXxg0aBADBgygRYsWfPTRRwwcOJCUlBSaNWvGCSecwIIFC2jXrl1IdRxMsPM2btyYLl260KJFCwDat2/P2rVrFbhFBIDv8woCPp6bV0D3sXP4Pq+A5hnpDO/dOuYCubrKw8A5x8iRI1myZAlLlixhzZo1XHHFFWRnZ/Puu+8yb948li5dSocOHdi2bRtNmzZl6dKl9OzZk3/84x9ceeWVu5zz008/LetunjVrFj169GDu3LlkZmZy8cUXM3nyZCB4ZvRuu+1WNk7tnGP79u0ATJkyhV9++YVFixaxZMkSmjVrVjYlqvzONs45TjnllLL3tHz5cp555hl22203PvvsM8455xxmzpxJnz59drn2iBEjePrppykoKKBr166sXLkypG7q8mUGQpqqVdV5d99997L/p6SksGPHjmrPJyLJoXlGesDHDWK++zwxWtwhtozLKwjjoiO9e/fm9ttvZ9CgQTRs2JDc3FxSU1PZvHkzTZs2ZY899mDlypXMnz8fgA0bNlC/fn3OOeccDj74YAYPHgxAo0aN2LJlCwDHHHNMha7hdevWkZmZyZAhQ/jtt99YvHgxt9xyC9dddx0bN26kcePGvPLKKxx99NGAt6PaokWLOP/883nttdcoLCwEYPPmzey7776kpqby/vvvs25d4A1ounbtyv/93/+xZs0aDjnkELZu3cr69etp3rw5W7du5fTTT6dr164ccsghu7z266+/pm3btrRt25Z58+axcuVKevTowZNPPsmll17Kr7/+yty5cxk/fnyF4NyyZUsef/xxiouLyc3N5bPPPit7LjU1lcLCwl1WOQt23pUrV9b0xygiSWR479aMnL6sQne5AZWbArHYfZ4Ygdtnp556KitWrKBbt26Alzj1/PPP06dPH5544gnatWtH69aty7qhc3Nzueyyy8pal2PGjAG8hLI///nPpKenM2/ePNLTd94RZmdnM378eFJTU2nYsCGTJ09m//33584776Rbt27sv//+dOzYkaIi75dwyJAh9OvXjy5dunDSSSeVtaYHDRrEWWedRVZWFu3bt+fwww8P+J722WcfJk2axMCBA/n9998BGD16NI0aNaJfv35s27YN5xwPPfTQLq99+OGHef/990lJSeHII4/ktNNOo379+sybN4+jjz4aM+P+++9nv/32Y+3atWWv6969O61ataJt27a0adOGjh07lj03dOhQ2rVrR8eOHZkyZUrZ42effXbA8ypwi0hVSgPx+LdXlXWL5wbpPg/Wre4Xi4dM26ysLFd5P+4VK1ZwxBFH1PqcibjM56RJk1i4cCGPPfZYjV6XiHURqkC/R9pneCfVRUWqj50SsS66j50TMHhnZqTz8YgTq3xtBPbjXuSc2zVTmQiOcZvZs2b2s5l9Ue6xPc3sHTNbXfJv00hdX0REpCaG925NemrFpOX01BSG927tU4kCi2Ry2iSgcubSCOA959yhwHsl30uYDB48uMatbRER8fTvkMmYAW3JzEjH8FraYwa0janxbYjgGLdzbq6Ztaz0cD+gZ8n/nwOygVsiVQYREZGa6N8hM+YCdWXRng7WzDn3A0DJv/tG+foiIiJxLaLJaSUt7tedc21Kvs9zzmWUe36Tcy7gOLeZDQWGAjRr1qzT1KlTKzzfpEmTgFORQlVUVERKVQuwJJFkros1a9awefPmCo/l5+eHfQOZeKW6qEj1sZPqoqJw10evXr2CJqdFezrYT2a2v3PuBzPbH/g52IHOuQnABPCyyitn661YsaJOmdDJnEldWTLXRVpa2i7LxCZitmxtqS4qUn3spLqoKJr1Ee2u8lnApSX/vxR4LcrXj7rTTz894Hre5d1xxx28++67tTp/dnZ2wGVHRUQkMUWsxW1mL+Ilou1tZuuBUcBY4GUzuwL4FjgvUtf3m3MO5xxvvvlmtcfefffdUSiRiIgkgoi1uJ1zA51z+zvnUp1zLZxzzzjnNjrnTnLOHVry76+Run5llfddfeOLn+p8zgcffJA2bdrQpk0bHn744YBbY7Zs2ZINGzYAcM8993D44YdzyimnMHDgQB544AGg4hacLVu2ZNSoUWXbbpauAPbZZ59x7LHH0qFDB4499lhWrVpV5/KLiEj8SYpNRgLtu3rnG6vrtHD8okWLmDhxIp9++inz58/nqaeeYtOmTaxatYpLLrmEnJwcDjrooLLjFy5cyKuvvkpOTg7Tp0+n8kpw5e29994sXryYq666qiy4H3744cydO5ecnBzuvvtubr311lqXXURE4ldSrFUeaN/VbTuK67Rw/EcffcTZZ59dtgb4gAED+PDDDytsjVn5+H79+pWtP37WWWcFPfeAAQMA6NSpE9OnTwe8zUEuvfRSVq9ejZmVbRoiIiLJJSla3MEWiK/LwvHBptGV3xozlOMDKd2OsvxWlLfffju9evXiiy++4N///ndIW16KiEjiSYrAHWzf1WCPh6JHjx7MnDmTrVu38ttvvzFjxgyOP/74oMcfd9xxZQE3Pz+fN954o0bX27x5M5mZXu/ApEmTal1uERGJb0kRuAMtHJ+2W706LRzfsWNHBg8eTJcuXTjmmGO48sorado0+J4pnTt3pm/fvhx99NEMGDCArKwsmjRpEvL1br75ZkaOHEn37t3Ltu4UEZHkkzTbes7Mya2w7+qwEw7kwm61X3mtNkpX1tm6dSs9evRgwoQJFfac9ksyL8CibT2rprqoSPWxk+qiomhu65kUyWmw68LxW7ZsiXoZhg4dyvLly9m2bRuXXnppTARtERGJL0kTuGPBCy+84HcRREQkRJV7aof3bh0TO4cpcIuIiFRSuv5H6VTi3LwCRk5fBuB78I7rwO2cw8z8LobEqXjI7xCR4CLZIg60/kdBYVGd1v8Il7jNKk9LS2Pjxo368JVacc6xceNG0tLS/C6KiNRCoBUxR05fVqcVMcuLxPof4RK3Le4WLVqwfv16fvnll1q9ftu2bfrQLpGsdZGWlkaLFi38LoaI1EKkW8TNM9LJDRCk67L+R7jEbeBOTU2lVatWtX59dnb2LvswJyvVhYjEm0i3iIf3bl1hjBsgPTWlTut/hEvcdpWLiEjyisSKmOX175DJmAFtycxIx4DMjHTGDGi7a2veOXjsMVIKoteFHrctbhERSV7RaBFXXv8joKeegmHD2Pcvf4HTTgvbtauiwC0iInGnNKD6Os/6q6/ghhvg5JP54YwziFYnugK3iIjEpZBaxJGyfTsMGgRpafDcc14QjxKNcYuIiNTUnXfCwoVeV3nz5lG9tAK3iIhITXzwAYwdC1deCQMGRP3yCtwiIhLXZubk0n3sHFqNeIPuY+eEbRGWgH79FS66CA45BB56qOzay3I3R/7aJTTGLSIicSuqa4o7B0OGwI8/wrx5zFy9eee1D4jeeuZqcYuISNyqagW1sHv6aZg+He67D7KyonvtchS4RUQkbkVtTfEVK+C66+Dkk+HGG6N77UoUuEVEJG5FegU1ALZtg4EDoUEDmDwZ6tWL3rUDUOAWEZG4Nbx3a9JTUyo8FvY1xUeMgKVLYeJE2H//6F47ACWniYhI3Ir4CmpvvgmPPALDhsGZZwa9NmwhM0qrtylwi4hIXIvYCmo//ACDB0O7dnD//VVeOzs7m2GDeoa/DAGoq1xERKSy4mK45BLIz4cXX/SWNo0RanGLiIhUdv/98O673pKmRx7pd2kqUItbRESkvHnz4K9/hfPPhyuu8Ls0u1DgFhERKfXrr3DhhXDggTBhApj5XaJdqKtcREQEvCVNL7vMS0r75BNo0sTvEgWkwC0iIgLw6KMwaxY8/DBkZfldmqDUVS4iIrJgAQwfDn37wrXX+l2aKilwi4hIctu8GS64wFsVbeLEmBzXLk9d5SIikrycgyuvhG+/hQ8/hD339LtE1VLgFhGR5PXEEzBtGowbB926VXv4zJzcyC2vGiIFbhERSU6LFsH118Npp8FNN1V7+MycXEZOX1a2B3duXgEjpy8DICOiBa1IY9wiIpJ88vLgvPOgWTP417/KtuoEL0B3HzuHViPeoPvYOczMyQW8zURKg3apgsKikk1GokctbhERSS7OeSuiffcdzJ0Le+1V9lRVrerv8woCns57vEHEi13Klxa3md1gZl+a2Rdm9qKZxc7q7SIiktCW3XQnTJ/OvcdfQvcPCspa1FB1q7p5RnrA8wV7PFKiHrjNLBO4FshyzrUBUoALo10OERFJPnOfncERD43mrcOO5anOZ5e1qEuDd1Wt6uG9W5OemlLh8fTUFIb3bh3xcpfn1xj3bkC6me0G7AF871M5REQkWfz0E0deP4TvMppx8+nXlc3XLj9OXVWrun+HTM7plElKyetSzDinU4T2Aq9C1AO3cy4XeAD4FvgB2Oycmx3tcoiISBLZsQMGDqRBQT5X9b+VLbtXHJMubWlX1aqemZPLq4tyKXIOgCLneHVRboWu9mgwV1KAqF3QrCnwKnABkAe8Akxzzj1f6bihwFCAZs2adZo6dWpYy5Gfn0/Dhg3Des54pbqoSPWxk+qiItXHTvFWF62eeoqDXniB9/98HV8c12uX5+un1KP1fo0AyCso5KfN29heVEz9lHo0a5JGRnoqq37cwvai4oCvzWxoYa2PXr16LXLOBVww3Y+s8pOB/zrnfgEws+nAsUCFwO2cmwBMAMjKynI9e/YMayGys7MJ9znjleqiItXHTqqLilQfO8VVXcyaBS+8AEOGsHnocB4vlzUOXot6zIC29Kymy/uyEW/gAnRUGzCxT4Oo1YcfgftboKuZ7QEUACcBC30oh4iIJLpvvoFLLoGOHeHRR+mf5k1iqs3qZ80z0skNkLwW7azyqAdu59ynZjYNWAzsAHIoaVmLiIiEzbZtcO65XhLatGlQErT7d6hdQtnw3q0rzPEutXX7DvIKorcxiS9Z5c65Uc65w51zbZxzFzvnfvejHCIikqCcg//7P8jJ8VZGa9Wqzqfs3yGTMQPakpGeWuHxTVsLyd1UELUkNS15KiIiiWfCBHj2WbjtNjjzzLCdtn+HTBrsvmtndbFzUVv6VIFbREQSy/z5MGwY9OkDd90V9tNXvfRp5Clwi4hI4vjpJzjnHDjgAJgyBVJSqn9NDfm99KkCt4iIJIbCQm/Hr02bYPp02HPPiFwm0CIt9cyitvSpdgcTEZHEMHw4fPghPP88HH10xC5TmpFefkpZZtOiqC19qsAtIiLxb8oUeOQRuO46GDQo4perPKUsOzs74tcspa5yERGJb0uWwJAh0KMHjB/vd2kiToFbRETi14YN0L+/N5798suQmlr9a+KcuspFROLEzJzcWi3VmbAKC+GCC+DHH72x7WbN/C5RVChwi4jEgZk5uRWW28zNK2Dk9GUAyRu8r78e5syBSZOgc2e/SxM16ioXEYkD499etcsa2QWFRVFbrSvmPPYYPP64l0l+6aV+lyaq1OIWEYkDfq/WFVNmz/ayx/v2hTFjAh6SyMMKCtwiInEgVraU9N2KFXD++dCmjTdfO8DKaKEOK8RrcFdXuYhIHAi0Wld6akrUVuuKCRs3wllnwe67w6xZ0KhRwMNCGVYoDe65eQU4dgb3aO3wVRcK3CIicaB0S8nMjHQMyMxIZ8yAtnHRQgyL7du9NcjXr4eZM+Ggg4IeGsqwQjznDKirXEQkTlRerStpOAdXXw0ffOB1j3frVuXhoQwrxHPOgFrcIiIS2x56CJ55xttbO4TlTEMZVvB7h6+6UOAWEZHY9frrcNNNXjf53XeH9JJQhhXiOWdAXeUiIhKbvvgCBg6EDh3gueegXuhtzeqGFQLt8BUvWeUK3CIisgvfp0r9/LOXQd6okZdB3qBB2C8RrzkDCtwiIlJBTZZXjUiA//13GDBg5xrkmfEXXCNJY9wiIlJBqFOlIjIX2jkYOhQ+/tjrHs/Kqv25EpQCt4iIVBDqVKmIzIW+/36YPBnuustbIU12ocAtIiIVhDpVKuxzoWfOhJEj4cIL4fbba3eOJKDALSIiFYQ6VSqsc6Fzcrw52p07w7PPglnNz5EkFLhFRKSCUJdXDdtc6B9/9Hb62nNPr9WdHvuLoPhJWeUiIrKLUKZKhWUudEEB9OsHv/7qJaTtv39dip0UFLhFRKTW6jQX2jm4/HJYsACmT4f27cNbuASlwC0iIv64+26YOhXGjoX+/f0uTdzQGLeIiETfpElw550weDDcfLPPhYkvCtwiIhJds2fDkCFw8snw5JPKIK8hBW4REYmepUvh3HPhyCPh1Vehfn2/SxR3FLhFRCQ61q2D00+HJk3gzTehcWO/SxSXlJwmIiKRt2ED9O4NW7dq45A6UuAWEZHIys+HM87wWtzvvANt2vhdorimwC0iIpFTWOiNaS9cCDNmwHHH+V2iuKfALSIikVFc7C2w8vbb8PTT3rKmUmcK3CIiEn7OwfDh8PzzcO+9cMUVzMzJrdvyqAIocIuISCQ88AA8+CAMGwYjRzIzJ5eR05eV7d+dm1fAyOnLABS8a8iX6WBmlmFm08xspZmtMLNufpRDREQi4LnnvNXQLrgAHn4YzBj/9qqyoF2qoLCI619aQvexc5iZk+tTYeOPXy3uR4D/OOfONbP6wB4+lUNEROqgcvf3A3usp9tfrvBWRXvuOajntQ+/zysIeo7cvAKGv7IUUOs7FFFvcZtZY6AH8AyAc267cy4v2uUQEZG6Ke3+zs0rwAH7fplD+xuHsumwo7zdvnbfvezY5hlV77FdWOy4c9aXES5xYvCjq/wPwC/ARDPLMbOnzayBD+UQEZFayiso5MaXl5Z1fx+84TuenXYXPzbak4sG3AGNGlU4fnjv1qSnplR7TqmeOeeie0GzLGA+0N0596mZPQL8zzl3e6XjhgJDAZo1a9Zp6lojqk4AACAASURBVNSpYS1Hfn4+DRs2DOs545XqoiLVx06qi4pUH568gkIKC7byY0nvd8ONv3DuXSOoV1TMtFFj+d++zWib2STg637avI3tRcVBzx3odfEg3L8bvXr1WuScywr0nB9j3OuB9c65T0u+nwaMqHyQc24CMAEgKyvL9ezZM6yFyM7OJtznjFeqi4pUHzupLipSfXi6j53DhQfA35btxp5bN/PKlLtw+QWc98exLP8pk8zf0xk2qGfQ13e4ezabtu7aum66Ryo5VbwulkXzdyPqXeXOuR+B78ysdclDJwHLo10OERGpndJEs0a//8bkl+8g838/c+W5d7C82R9IT01heO/WVb5+1FlHkZpScSvP1BRj1FlHRazMicSvrPJhwJSSjPJvgMt8KoeIiNRQ84x0dtv2C89Mu5vWv6xlyIDb+eyANqSYMWZA22ozw0uf12IsteNL4HbOLQEC9t2LiEhsu6VXS9oPuZHM3JUM63sz2QdnkZ6aElLQLtW/Q6YCdS1p5TQREQldYSF977sBli3hvnNu4q1DjiNTLeaoUuAWEZHQFBXBxRfDrFmsvvZabn1kPLf6XaYkpMAtIiLVKy6GIUPgpZdg3Dhyu3ThUL/LlKQUuEVEEkBEd95yztssZOJEGDXKW4c8Ozs855YaU+AWEYlzEd15yzm44QZ4/HFvm85Ro+paXKkjX3YHExGR8Am289b4t1fV7cTOwS23wCOPwHXXwbhxYFb96ySiFLhFROJcsJ23qtqRKyR33AHjx8NVV8FDDyloxwgFbhGROBds563qduQKyjm47TYYPRquvBIee0xBO4YocIuIxLlAO2+FsvRoQM7BX/4C993nZZE/+WTZntoSG5ScJiIS58K2hGhxMVx9tResr70WHn5YLe0YVG3gNrNrgCnOuU1RKI+IiNRCnZcQ3bEDrrgCJk+GkSPh3nsVtGNUKP0f+wELzOxlM+tjpp+kiEhCKSyEP/7RC9r33ON1k+ujPmZVG7idc38FDgWeAQYDq83sPjM7OMJlExGRSNu2Dc45B155BR54AP76V79LJNUIKePAOeeAH0u+dgBNgWlmdn8EyyYiIpG0dSv07Qv//jf84x9w441+l0hCEMoY97XApcAG4GlguHOu0MzqAauBmyNbRBERCbstW+DMM+Gjj7ylTAcP9rtEEqJQssr3BgY459aVf9A5V2xmZ0amWCIiEjGbNsFpp8HChTBlClx4od8lkhqoNnA75+6o4rkV4S1O9EV0YX4RkVjzyy9w6qmwfDm8+ir06+d3iaSGknoed0QX5hcRiTU//AAnnwzffAOvvQZ9+vhdIqmFpF4OJ2IL84uIxJpvv4UePWDdOnjrLQXtOJbULe6ILcwvIhJLvv4aTjrJG9t+5x3o1s3vEkkdJHWLO+wL84uIxJqVK72W9pYtMGeOgnYCSOrAHdaF+UVEKpmZk0v3sXNoNeINuo+dw8yc3OgW4PPPvaBdVAQffACdOkX3+hIRSd1VHraF+UVEKvE9+XXhQi97fI894L33oLUaJIkiqQM3hGFhfhGRAKpKfo34Z87HH8Ppp8Oee3rd461aRfZ6ElVJ3VUuIhIpviW/zpnjtbT32w8+/FBBOwElfYu7trRwi4hUpXlGOrkBgnREk1/feMPbMOTQQ73s8f32i9y1xDdqcddC6dhVbl4Bjp1jV1FPPBGRmFXX5NcaJ7ZNmuStgnbUUfD++wraCUyBuxa0cIuIVKd/h0zGDGhLZkY6BmRmpDNmQNuQeuZq1DhwDu69Fy67DE48EbKzYe+9w/12JIaoq7wWtHCLiISitsmvISe2FRXBtdfC44/DRRfBM89A/fp1LfYuAg0NZoT9KhIqtbhrQQu3iEgkhdQ4KCiA88/3gvbNN8Nzz0UsaAdq/ecVFIb9WhIaBe5a0MItIhJJ1TYONm6EU06BGTPgkUdg3DioF5mP82Ct/582b4vI9aR6Cty1UJexKxGR6lTZOPj6azj2WG+BlZde8rrKIyhY6397UXFEryvBaYy7lrRwi4hEStBVHQvWwan9oLgY3n0Xjjsu4mUJNq2tforafX5R4E5wmm8uEp92aRy8+KKXOX7AAd587cMOi0o5hvduXWHpVvBa/82ahH88XUKjW6YEpvnmIgnAORg9Gv74R+jSBebPj1rQhuBDgxnpqVErg1SkFncC83WtZBGpu99/h6FDYfJkb7rX00/D7rtHvRiBhgazs1dHvRziUYs7gWm+uUgc27DBW3N88mS4+27vXx+CtsQeBe4EpvnmInHq88+hc2f49FOYMgVuvx3M/C5VmZk5uaz6cYt/+4wnOQXuBKb55iL+q/Ga4zNnetO9fv8d5s71xrZjSGnuzPaiYuXO+ESBO4FpvrmIv2q85vjo0XD22XDkkd487S5dol7m6mivBv/5lpxmZinAQiDXOXemX+VIdJpvLlK1SE6ZDDlB9LffvKler7ziJaFNmADpsTmkpdwZ//mZVX4dsAJo7FcBojnHWfOpRWJPaYu4NLiWtoiBsPx9hhTkvv0W+veHJUvg/vvhpptiajy7Ml/2GZcKfOkqN7MWwBnA01G/uHNw//1sW/9j1OY4az61SGyKdLdvtQmiH3zgJaF9/TW8/joMHx7TQRuUOxML/Brjfhi4GYj+YrfffAN3383xw67mqP9+XuGpSI3TaExIJDZFuts3aJA79TB46CE46STIyPAWVTn99LBcM9JKc2fqp9RT7oxPzDkX3QuanQmc7py72sx6AjcFGuM2s6HAUIBmzZp1mjp1atjK0OC//+XwW29jj19+5qNBl/H5KWdUuMttm9kkbNcCWJa7Oehz4b5WbeTn59OwYUO/ixEzVB87xUpd5BUU8tPmbWwvKqZ+Sj2aNUkLy8pdq37cEnCzjPop9Wi9X6NdHq9NfVQue/P6jmP+/hD7Zmfzy/HHs/KWWyhq0KBO5wxXfdRErPxuxIpw10evXr0WOeeyAj3nR+AeA1wM7ADS8Ma4pzvnLgr2mqysLLdw4cKwlmPiMy9ywJiHOPnrBcw4sie39r6GgvppZGak8/GIE8N6re5j5wQcE4rEtWojOzubnj17+l2MmKH62CkW6qLyODR4rdZwtPICndsAh/f3WTkXpc71sWoVDBgAK1fCffd5+2jXsGs8kvVRE7HwuxFLwl0fZhY0cEe9q9w5N9I518I51xK4EJhTVdCOlKbN9+baC+/kb8cNot/yD5jxrxs5cvP3ERmnicUxofJzS1f9uEXj7RKzIjnUVH7KJOwM2hCBXJSZM73x7J9/hrffhltuqdV4tobeJGnncWekp3LfOUcz/YzLGXz+XTQryOO1yTfQf+XcsF8r1uZTV06W215UrGQ5iVmRHofu3yGTj0ecSGZGOpX7H8MSEHfsgBEjvPnZhx8OixbBySfX+nSajiW+bjLinMsGsv26/s45zidC7uVwwQXeKkUffsisS25kXPa6sE3fiqX51Np8RCqL5emK0Zp+FJGAuH49DBrkrYA2dCg88gikpdX+fEDGHqls2lq4y+OajpU8kq7FXdpFvCx3c8XlBzMz4f33vTGnf/6Tg8/uQ721/03I6Vu6Y5fyYn26YrSGmsK+tv+MGXD00V4L+1//giefrHPQnpmTS/62Hbs8nppimo6VRJIqcJf/gIIAH1CpqTBuHLdcfA8tNv3AG5Ouo/eqT4DEGkPS5iNSXqyPmYZ7qCnY2uFhu0HYuhX+/GcvCe0Pf4CcHG81tDAY//YqCot3TShuUH+3mOkhkchLqv24Q+0ifrl5Bz6+7FEee20sT868j8kdzuDeE68gN8/LEI+lbsTaGN67dcCsVN2xJ6eqe2BqNk0JItPtHq6hplBWSqtT2ZcsgYEDvezxW27xtuOsX7/O5S4V7Ge1uWDXrnNJXEkVuEPtIm6ekc56mnHeoPsZ/sFkhi6YQef1X3JN31v4mgPCuiSiHyp/QNVPqacFFHwQK+PK4RxDjvQSonVV3c17rW8Qiovh0Ue9YL333vDOO97iKmGm5UYFkqyrPNQu4tIus8KUVO478QoGn3sn+/y2idefu55BOW9SsH1HzHQj1lZpJu1/x55B6/0axcSHajKJpXHlcI4hx3q3e6CgB3XM7/jpJzjjDLjhBjjtNFi6NCJBG2JzaqlEX1IF7lB/6SvP7cw+OIvTLvs7C1ocyb2zH+ep6few7fsfo1ZuSTyxFODCOYYcy4mPM3NyCTZrutYt1rfegnbtIDsb/vlPLyFt771rW8RqxdrUUvFHUnWVl+8ihi0BV0Yqf2z/Dpllq5790nBPLj3/LgYv+jcjsify9qRhcMZe0Lt3lN+FJIJYC3DhGkOO5a7c8W+v2mWeNniLrtS4xbptmzc3+5FHvMD9/vveHtpREEtTS8UfSdXihp1dxG0zm/DxiBOr/QMo30p3Vo+JWf04//JHqb/vPtCnD1x3nZdFKlUKlsmbrBI1sz+Wu3KD3RQ5ajj+vnw5HHOMF7Svuw4+/TRqQVsEkjBw11SgrqnLrupL42U5MGyYl5DSsSN89pnfRY1ZsTSeGytiOcDVRSx35Qa7KcoM9WbJOZrPmgWdOsEPP8Abb8DDD9d5brZITSVVV3ltBe2aevRR6NcPLrsMjj0WRo6E228P6/SPRKCV2nYVlqlHMSpWu3LrNA3yp5/gT3/isNde84bHJk2C/faLXGFFqqDAXVcnnQTLlsH118Po0fD66zB5MrRt63fJYkasjefGilgKcLEyNS2SanWz5By89BJccw3k57Pm6qs55O9/h3rqrBT/KHCHQ5MmMHEi9O/vrUfcqRP89a9e8opa3zGdsCRVz73O8LNgEVCjm6WffoKrr4bp06FLF5g0ifU//cQhCtriM/0GhlO/fvDFF3DeeTBqFGRlwYIFZU8na4JWoo7nJopYmpoWE5yDqVPhqKO8cexx4+Djj+GII/wumQigwB1+++wDU6bArFnw66/QtSsMH86/P1mTtAlasZywJBrKqCA319t+c+BAOPhgb53xm2+G3dQ5KbFDv42RctZZ0KOH90f/wAO0n/giR5/6f8w/sF3ZIcmUoBVL47lSkYYy8JYsfeop7++1sBDGj/fyVmIwYCdDPoJUTS3uSGrSxNvKb84cioqKmfrirdz3n8dovC2/7JB4btUka9d/okn6oYzly6FnT29Hr6wsL9n0ppuiErRr+jekqZUCCtzR0asXl9/wNE917s8Fn8/m3aev4rzPZ1OvuChuWzX6AEkc8TiUEZabxoICuO02aN8evvwSnnkG3n3X6yKPgtr8DSkfQUBd5VFz7VlHM7LwT8w8sif3vPNPxr/1KFcumsXGO0b7XbRa0dzsxBJPQxlh2YHs7be9jPFvvoFLLoEHHvDyU6KoNn9DykcQUIs7akpbNXmHt+Wcix7gtj/eQYv6jmOvuchb0OHzz/0uYo3oA0T8UqdW57p13qyPPn28rvA5c+C556IetKF2f0OJulSu1IwCdxSVbaU57kzunXIXDb7+ylsyceFCr7vu8su9rNY4oA8Q8Uutbhp/+w3uuAMOP9yb4nXPPd7Ncq9eESpl9WrzN5T0+QgCKHD7q359b5OCNWvgxhu9aWSHHuotm7pli9+lq5I+QMQvNQp4znl/V61be8F6wABYtcpbIGn33SNc0qrV5m8oHvMRJPw0xh0Lmjb1pp9cfbWXLDN6NEyYAHfdBVdeGZNTUhJ5rW2Jbb0O34cp87+tsEVnwID32WfejfH8+V62+Msve3sKxIja/g3FUz6CREbsRYRk1qoVvPAC3HCDNx3lqqu8rQPvvx/OPBPM/C5hBfoASS6xMH94Zk4ury7KrRC0DTinU7nfxe+/9zb8mTzZ2whk0iS4+OKYXF9cf0NSG7H3myzQuTNkZ8PMmV5XX9++cOKJsGiR3yWTJBUr0/8CJaY54P2Vv8C2bXDffXDYYd6SpSNGwFdfwaWXxmTQFqkt/TbHKjNv7fNly+Af//DmmWZlwaBBsHat36WTJBMr84cDJqA5R4d5b8ORR3pDTaeeCitWwJgx0KhRVMsnEg0K3LEuNdUb+16zBm691dup6PDDvaUZ8/L8Lp0kiViZ/lc5Aa372iXMmnwDj8263wvS773n/Y384Q9RLZdINClwx4vGjeHee72uvwsv9BaMOPhgbwx8+3a/SycJLlam/5VmYrf74Sv+NfWvTHnpr+xV8D8W3f0wLF7sDSmJJDgF7nhzwAFess3ixdChg7cRwpFHwrRp3ni4xKx4Xts9Vqb/9U/ZyHsfP8ysyX/hyJ+/4ZEzrmbR2/PodPt1kJJS/QlEEoACd7xq3x7eeQfefBPS0rzVoLp3h3nz/C6ZBBDO5C4/bgB8nz+8dKn3O3700eyz4BOePvlSTvjT07x83DkU1/d3PrZItGk6WDwzg9NOg1NO8Vrht9/uzVM991wvMeeQQ/wuoe9iYQoThG9t9+rW6Y7k+/Vl6tLChd7CKbNmQePGrLryOi5p0p2fdtsDgPzarFMuEufU4k4Eu+3mLdSyejXceSc73niTwsOPYGJWX/rd+nJcdcmGU6xMYYLwJXdVdQMQS++3zj75xLsp7dwZPvzQW4xo3TouP7hvWdAupd2xJNkocCeShg2Z2fdKThz6FK+0OYlLFr/By/cPYtuVQ5k962O/Sxd1sTKFCapP7gq1+7uqG4BYer+14hy8/z6cdJI37LNwoddztHatt854RkbMZLeL+EmBO8GMf3sV36Y14dY+w+g15EmmtT2Zs5fO5qT+PeD882HBgoCvi+fEqWBi6UO+quSumrSUq7oBiKX3WyNFRV5y5THHeFnhy5fD3/7mBewRI7wZFSUikd2eiL/7ktgUuBNM+Q/pb5vuz229r+H4Pz3DhC4DYPZs6NIFevaEf/8biosByCsoTJwu1nJiZQoTVJ3cVZOWclU3ALH0fkOSnw9//7u30tl558GmTfDEE94e2X/5CzRosMtLwp3dnlDDC5I0lJyWYJpnpJNbqYX1c6O9eL7/VVx19TPw1FPe3O++fb2dyK69ll/32IeCwoYVXlObxKlYM7x36wqJXODvDmbBkrtq0lKubmOKaL3fOiXBffMN/POf3u/i5s1eQuW4cXD22dVO6Qr35jbhShoUiSYF7gRTZbBq3JiZJ17Ig9va0u6z97gq598cNWwYF6WlU+/Ik5jc8Uy+2atF2etivou1GvGyg1mgm63SxwMJdgMQrfdbXWZ7QEVF8J//wOOPw1tveWuHn3OO17I+5pgaXT+c2e2h3jSV3qhceMAWbhs7JyZ/jyR5KHAnmKo+vMt/4H57RA9eP6IHR//wFeO+mcXAef9h8OLX+eigo3m+w+m8e8gxNNsr/td5jofdl8LZMxCN91ujVuqGDfDss14X+H//C/vv7yWaDRkCmf7/XJqkp5JXUBjw8VIVblQOCPFGRSSCFLgTULAP70AfuEv3P4x3Tr2e5zsN4dzF/+GPS/7DEzPH8EuDpmw6byCsOVDzwSMsXnoGSlXbSi0u9qZwPfOMtwf27797eRXjxkH//t76+zEi2E655R9Xd7rEGgXuJFJV1/fvTfdm5umDeaLruZz98xfcuG4uh/3rSZj0OPTqBYMHw4AB0LBh0HMkqmgs4hIPPQOlgnXtdyrO8+ZbP/ec17pu1MhbX+Dqq71leSOkLj+fvK27trYrPx632fqSsBS4k0iwD1yAzQWFLBl1asl3fYFb4fvvYeJEr6vz0kvhqqu8BKKLLoKTT/YWfklwtRrPTXDlu/b32F7Aaas+4fwv3+OYdZ97TdWTTvJWOzv7bNhjj+pPWAd1/fmEkl9Q0xwEkUiL+nQwMzvAzN43sxVm9qWZXRftMiSr4b1bE6RnMPCHUPPm3v7Ga9Z4XZ8XXQRvvOGtaNWihZdYtHhxQm9uEveLmkRA/zb78kyzX3ji7YdZ8NjF/O3Nhziq+H8werQ39/qdd7x944ME7XDOm67rzyeU6WWxssGKSCk/mkw7gBudc4vNrBGwyMzecc4t96EsSaV/h0wWrvuVKfO/pXyorWdW9YeQGRx3nPf16KNe8H7+eXjsMXjoIW8e7gUXeHNx27QJPnAYh9RNWqKw0FvVbNo0mD6dYzduhIwMuPQiGDyYhsceG9LPPdw9GHX9+YSSX1D+GNhCZoznIEjii3rgds79APxQ8v8tZrYCyAQUuKNgdP+2ZB20Z4UPqsymRaF/CO2+uzfWPWAA/PorvPKKl4B0771e9+ihh3rPnXUWdO0a0a0WozH2nNTdpL/9xt4ffeSNWc+a5f28GzaEM8/09oTv08f7faiBcCd6hePnE0p+Qekx2dnZDBvUs6bFFAkrcz52c5pZS2Au0MY5979Kzw0FhgI0a9as09SpU8N67fz8fBomYaJVIOGoi9RNm9jj/Q/YO/sDmi9fRkpREb83bsKmbl3Z0K0bmzp3piiM4515BYXkbiqguNzvbz0zMpumk5Fet6zl8vURyevEovobNrDXvHns/cknNF20iHqFhexo0ICN3brx8wknsKlzZ4prGKzLW5a7OehzbTOb1Ph80f756HNjJ9VFReGuj169ei1yzmUFes63wG1mDYEPgHudc9OrOjYrK8stXLgwrNfPzs6mZ8+eYT1nvApHXZTvAm28LZ8e/13Mqd8soM+3i6n/v81Qv743Jei00+DUU+GII+rUpd597JyALa3MjHQ+HnFiHd7JrvURK1uDRsTWrfDRR/Dee/Duu17OAkDLltCvH0sOPJD2w4aFbQpXJH5u0fz56HNjJ9VFReGuDzMLGrh9SQs2s1TgVWBKdUFb4kP5LtD/pTXk9ZIFXg5sVJ+5x9X31kZ//XW44QbvBS1aeJnpJ57oBfQDDqjR9aI59hxPU7WqtWOHt9HMu+96wXrePNi+3QvMXbt6CWb9+sFRR4EZednZYZ13HWixGcMb6+5eyxXJEurnIxKCqAduMzPgGWCFc+7BaF9fIiNYwPxuy3Y44RQ44QR44AFYt87b7GT2bG/cdNIk78CDD/aS37p29b7atKlyullSjz3XxPbtkJPj7W89Zw588AFs2eI916EDXHutN33r+OMDbuoRbuUTvXLzCjAoS5TUVDuR0PjR4u4OXAwsM7MlJY/d6px704eySJiEHEgPOshb7nLIEG+Frc8/h+xs7+vNN71EKPCmEnXuvDOQd+0K++1XdppY20AkJjgH337r7WM9b573tWiRt3IZeCvg/fGPXqDu1Qv23rvGlwhHt3RpCzlQt7lWJBOpnh9Z5R9B0OnEEqdqFUjr1YP27b2v66/3As/atTB//s6vBx/0piKBF/Q7d4ajj6Z/u3bs0XVf7vr8N77fvC3xxp6rs327N7/+889hyRJvbHrxYti40Xu+fn3o1AmuuQa6dfO+mjcPerpgATmvoJDuY+fwfV4BTdJT+W37DgqLvDay31O5RJJV4i99JVERlvW2zaBVK+9r4EDvsW3bvK7e+fPh00+91uS0aQCcCpzauLG3nOZhh0HRYfB1a+//hxxSo1W7ygeuEe2LycvJ9f8mYPt2b+nQNWtg9eqK/65dW7afOqmp3tDC2WdDx47eV/v2IU/VCja3euG6X9l/awG5ed6UvkCbcfg9lUskGSlwh1FCZx+HINQkoRrVU1razhZjqfx8+OILr7W5dCksX+4lWk2eXPG1mZneTUDLll4yXIsW3mOZmd4uVXvvDWlpuwSu7UXF0Rlr/e03+Plnb2nZ77+H9evh668DB2eAxo29efKdO3td3ocdBu3aweGH13g+dXnB5la/+Ol3XN+m+lkntW0ha7hDpHYUuMNEa1qHJiz11LDhznHv8vLzvYD31VfeV2nwmzvXC4w7dux6rgYNOCa1IS/v3pBN6Y3ZlN6YPyxIY/cte7BhXmPofZR385Be0gosKvKCaVFRaP/fuhU2b4a8vJ1fv/7qBeytW3ctT2lw7tLFC86HHur1Hhx6qHejEYFV6YIF3qIQp4rWtoUcb7uiicQKBe4w0dZ/oYloPTVsuHPMvLLiYi9Y5uZ6Ldsff/TGgzdu5JPZS2i69X80LdjCgXk/sm/uVg7Zuo30Hb/De3UoT716XsDPyNj5tf/+Xtd+s2awzz6w777e2HPz5l5PQNOmUV8yNliXdUoI5ahrC1lTuURqToE7TBIx0SYSXf/B6iPYrmVhU6+el5W+335e0lY5D1bKbr6x7Q7+tmw3Dmhcnw+v7QYFBd4XeEu4pqR45yv9f6DvzeJmzfZgXdbndMqk3tb/Vjg2tZ7RMG038rYWqoUs4hMF7jBJtESbSHX9B6snK7mmH0EgWOC68bQjvVZ8gi/rWFWX9cy3fiQzI0Vd2SIxRIE7TBIt0SZSXdrDe7fmhpeWUHn01JVcsybnDlePQOXAVT+lHud0ymT826u44aUltTp3vCUqBuuyzkhP5eMRPaNfIBEJSoE7TBIt0SZSXf/9O2Ry/UtLAj5Xk3OHu0egfOCa+dY7PPxJbq3PHWuJivF2EyEiVVPgDqNESrSJZNd/ZpBzOwh5vepIJrn9tHkbBYX1an3uaCUqhhKQY+0mQkTqrl71h0gyGt67NempFffSDlfXf6BzlyoNLDNzcqs8RySTAbcXFQd8PNRzRyNRsTQg5+YV4Aheb1XdRIhIfFLgloD6d8hkzIC2ZGakY3it5DED2oallVb+3IGEEliCtfzD0SNQPyXwn0Wo545k2UqFGpATcbaDSLJTV7kEFcmu/9Jztxrxxi6JalB9YIlkMmCzJmmkpxbV+tzRSFQMNSAn2mwHEVGLW8JgZk4u3cfOodWIN+g+dk613dzl1bZ1GskegYz01DqdO5JlKxVqvUVyyENE/KEWt5SpTfZxXZOf6tI6DdYjUNX7CPU91rW3IdKJiqHWW6LNdhARBW4pUdsAXNcM6nAHlqreB5AwGdY1qbdEmu0gIgrcUqK2ATgcyU/hDCzVJW0l0nryCsgiyUmBW4DaB+BYS36qzftQhnX80uIykoyUhfMpnAAABx9JREFUnCZA7ZPEYi35qar3EY1pWhI9oc5lF0k0CtwC1D4ARyODuiaqeh+xdpMhNVd+BsONLy/V4jKSlNRVLkDdksRiaaw1lPehrtX4VDnxsMgFWgFAQx+S+BS4pUwsBeC6qOp9JMp7TEaBEg8D0dCHJDp1lYtIXAilJa2hD0kGanFLlZS1K7Ei2AyGFDOKndPvpyQNBW4JSltCSiwJtlqcn8mQIn5QV7kEpS0hJZbE2gwGEb+oxS1BaUtIiTVKLhRR4JYqxNqqaPFEuQEiEinqKpegtGBJ7WhFLxGJJAVuCUpjirWj3AARiSR1lUuVNKZYc8oNEJFIUotbJMy0mYmIRJICt0iYKTdARCJJXeUiYVaXDVtERKqjwJ1E4nmKUryVXbkBIhIpCtxJoqrlSzP8LFgItPRqcoi3mzMRv2iMO0nE8xSleC67hEZz30VCp8CdJOJ5ilI8l11Co5szkdApcCeJeJ6iFM9ll9Do5kwkdArcSSKepyjFc9klNLo5EwmdL4HbzPqY2SozW2NmI/woQ7KJ5+VL47nsEhrdnImELupZ5WaWAvwDOAVYDywws1nOueXRLkuyiecpSvFcdqme5r6LhM6P6WBdgDXOuW8AzGwq0A9Q4BZJYro5EwmNH13lmcB35b5fX/KYiIiIVMOcc9G9oNl5QG/n3JUl318MdHHODat03FBgKECzZs06TZ06NazlyM/Pp2HDhmE9Z7xSXVSk+thJdVGR6mMn1UVF4a6PXr16LXLOZQV6zo+u8vXAAeW+bwF8X/kg59wEYAJAVlaW69mzZ1gLkZ2dTbjPGa9UFxWpPnZSXVSk+thJdVFRNOvDj67yBcChZtbKzOoDFwKzfCiHiIhI3Il6i9s5t8PMrgHeBlKAZ51zX0a7HCIiIvHIl01GnHNvAm/6cW0REZF4ppXTRERE4ogCt4iISByJ+nSw2jCzX4B1YT7t3sCGMJ8zXqkuKlJ97KS6qEj1sZPqoqJw18dBzrl9Aj0RF4E7EsxsYbA5cslGdVGR6mMn1UVFqo+dVBcVRbM+1FUuIiISRxS4RURE4kgyB+4JfhcghqguKlJ97KS6qEj1sZPqoqKo1UfSjnGLiIjEo2RucYuIiMSdpA3cZnaPmX1uZkvMbLaZNfe7TH4ys/FmtrKkTmaYWYbfZfKLmZ1nZl+aWbGZJW3WrJn1MbNVZrbGzEb4XR4/mdmzZvazmX3hd1n8ZmYHmNn7Zrai5O/kOr/L5BczSzOzz8xsaUld3BWV6yZrV7mZNXbO/a/k/9cCRzrn/uxzsXxjZqcCc0rWkh8H4Jy7xedi+cLMjgCKgSeBm5xzC30uUtSZWQrwFXAK3o5+C4CBzrnlvhbMJ2bWA8gHJjvn2vhdHj+Z2f7A/s65xWbWCFgE9E/G3w0zM6CBcy7fzFKBj4DrnHPzI3ndpG1xlwbtEg2A5LyDKeGcm+2c21Hy7Xy87VaTknNuhXNuld/l8FkXYI1z7hvn3HZgKtDP5zL5xjk3F/jV73LEAufcD865xSX/3wKsADL9LZU/nCe/5NvUkq+Ix5KkDdwAZnavmX0HDALu8Ls8MeRy4C2/CyG+ygS+K/f9epL0w1mCM7OWQAfgU39L4h8zSzGzJcDPwDvOuYjXRUIHbjN718y+CPDVD8A5d5tz7gBgCnCNv6WNvOrqo+SY24AdeHWSsEKpiyRnAR5L6l4pqcjMGgKvAtdX6sFMKs65Iudce7xeyi5mFvGhFF+29YwW59zJIR76AvAGMCqCxfFddfVhZpcCZwInuQRPfqjB70ayWg8cUO77FsD3PpVFYkzJeO6rwBTn3HS/yxMLnHN5ZpYN9AEimsSY0C3uqpjZoeW+7Qus9KssscDM+gC3AH2dc1v9Lo/4bgFwqJm1MrP6wIXALJ/LJDGgJCHrGWCFc+5Bv8vjJzPbp3QGjpmlAycThViSzFnlrwKt8bKH1wF/ds7l+lsq/5jZGmB3YGPJQ/OTNcvezM4G/g7sA+QBS5xzvf0tVfSZ2enAw0AK8Kxz7l6fi+QbM3sR6Im3A9RPwCjn3DO+FsonZnYc8CGwDO/zE+BW59yb/pXKH2bWDngO72+kHvCyc+7uiF83WQO3iIhIPErarnIREZF4pMAtIiISRxS4RURE4ogCt4iISBxR4BYREYkjCtwiIiJxRIFbREQkjihwi8guzKxzyd7saWbWoGSv4aTezlIkVmgBFhEJyMxGA2lAOrDeOTfG5yKJCArcIhJEyRrlC4BtwLHOuSKfiyQiqKtcRILbE2gINMJreYtIDFCLW0QCMrNZwFSgFbC/cy7h96wXiQcJvR+3iNSOmV0C7HDOvWBmKcAnZnaic26O32UTSXZqcYuIiMQRjXGLiIjEEQVuERGROKLALSIiEkcUuEVEROKIAreIiEgcUeAWERGJIwrcIiIicUSBW0REJI78P9PDg5AgOGBvAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot\n",
    "plot_poly_lls(X, y, phi_x, theta_ls, title=\"y = 0.5*x^2 + x + 2 + epsilon\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(100, 3)\n",
      "theta:\n",
      "[[1.98949504]\n",
      " [0.86612449]\n",
      " [0.5144204 ]]\n"
     ]
    }
   ],
   "source": [
    "# using scikit-learn\n",
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "poly_features = PolynomialFeatures(degree=2, include_bias=True)\n",
    "phi_x = poly_features.fit_transform(X)\n",
    "print(phi_x.shape)\n",
    "# continue the same\n",
    "theta_ls = np.linalg.inv(phi_x.T @ phi_x) @ phi_x.T @ y\n",
    "print(\"theta:\")\n",
    "print(theta_ls)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAe4AAAFNCAYAAADGn4wWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3xUVfrH8c9DDCbSgg0lqLAWLICUgCCKYAMbIFYWCxbY1Z9YVlHQVSwoIK5tXVexgKwoKgKylhUVIxZQSkCUIuiCEitIWCJBQnJ+f9wkJGEmmSQzc6d8369XXpCZO/eeOUnmueec55xjzjlEREQkPtTzuwAiIiISOgVuERGROKLALSIiEkcUuEVEROKIAreIiEgcUeAWERGJIwrcIjHKzOqZ2fVm9ke/yyKBmVm+mf2h5P+TzGy032WSxKfALVKJmbU0s/fNbKuZrTSzk6s49k4zKyz5AM8v/0Fe6ZiWtSjKo8BQ4BEzu6LSOfc1sxfN7Hsz22xmH5vZMbW4hu/M7AEzW21mW0rq+xK/yxQq51xD59w3fpdDkosCt8iuXgRygL2A24BpZrZPFce/VPIB3rD8B7mZ3Wpmx5ccs5uZ3WZmXUMpgJndARwL9AB6AneaWd9yhzQEFgCdgD2B54A3zKxhCOduaWZrQylHbZlZTzPLDvHw34CzgCbApXg3KseGeJ21tbwpEolbCtwSN8xsuJm9Wumxv5vZw2G8xmFAR2CUc67AOfcqsAw4pxanewToA1wIPAEsd87NN7MLzOwbM2tccs3TzOzH0psDM/sTcDpwonNug3PuS+BE4AEzOw7AOfeNc+5B59wPzrki59wEoD7Quk4VUI6Z1TezJWY2rOT7lJKW/R3hugaAc26Uc26lc67YOfcp8CHQLZzXADCzy81shZltMrO3zeygcs85M7u25OeywczGm1m9kucOMbMPSno2NpjZS5Ved0iQ6w0xszVm9quZzTKz5pVe9+eSnoZNZvYPM7Nwv2dJTArcEk+eB/qYWQaAme0GXAD8K9DBZva6meUF+Xo9yDWOAr5xzm0p99jSkseDOavkw/lLM7uq0nOu3L9FAM65l4B5wKNmthfwDHClc+6XkuefdM51dc7llZ3EudXOucOccx8Fea/t8QL3mirKWSPOue3ARcDdZnYEMAJIAe4N1zUqM7N0oDPwZZjP2x+4FRgA7IN3c/BipcPOBrLwbtz6AZeXPH4PMBtoCrQA/h7C9U4ExgDnA/sD64CplQ47E++9Hl1yXO8avi1JUrv5XQCRUDnnfjCzucB5wFN4rdkNzrlFQY4/sxaXaQhsrvTYZiAzyPEvAxOAn4BjgFfNLM859yJwHd4H/g68m44Lzexn59x84P+Az4Fs4N/OuWA3EtUqabn/C7jLOVe57HXinPuiJOFqBtAM6OKcKwrnNSp5Au9G6e0wn/dPwBjn3AoAM7sPuNXMDnLOrSs5Zpxz7lfg15JenIHA00AhcBDQ3Dm3Hgh481TJIOBZ59zikuuNBDaZWUvn3NqSY8aW3Jzlmdn7QHvgP+F4s5LY1OKWePMcXiuQkn8DtrbrIB9oXOmxxsCWAMfinFvunPu+pLv6E7zu8XNLnrvPOTe35NAdzrnRJUGbkg/sV4A2wN9qW9iSFuq/gfnOuTFVHPfH0t4GvBuGAyv1QBxYxWWeA1oCbzrnVldxjRHlrvE6cFz5a4TwXsbj1cf5LsjuR2Z2YKVzHgh8Xu6xYBn4B+GNnZe+7lfAqHhD9l25/68DSru2by459rOSXpXLqV7zknMA4JzLBzZWut6P5f6/Fe+mUaRaCtwSb2YC7cysDV5X45RgB5rZW1Yx27v811tBXvYl8Acza1TusaMJvevW4X3I73zAuTvLtbJKy9Yeryv2Rbzs8Rozs93x6iMXr0UZvFDOveCcy3DOZQDtgG9Lvy/5+raKlz+OF4h7l46xB7nG2HLXOBP4qPw1qnkvdwGnAac65/5XxTW+rXTOb4F25R57IchLvwP+VOk9p5fcbJU6oNz/DwS+L7nmj865Ic655nj1/Hiwce1yvse7WSh9fw3wkh1zq3mdSLUUuCWuOOe2AdOAF4DPqgo4zrnTKmV7l/86LchrvgKWAKPMLM3MzsYLdK8GOt7M+plZU/N0Aa4FXqvqPZhZGl7X+a3AZUCmmV1d7ZuveI5UvHooAC5xzhXX5PU1uM7FeJnrg/He23MWQuZ6Da8xEvgjcIpzbmM4z13OE8BIMzuq5JpNzOy8SscML/lZHoA3zPFSybHnmVmLkmM2US5foQovAJeZWfuSG6z7gE8r38CJ1IYCt8Sj54C2hL+bvNSFeElKm4CxwLmliWNmdryZ5Vc6dg1eV/pkvHHS56o5/xhgvXPun8653/G6/Eeb2aE1KOOxeK3aU/HGSEt7Eo6v5nUhK+k+fxjvxiC/pDW7EHgoXNcocR9eC3d1ufdxazgv4JybAYwDpprZ/4Av8Fr45b0GLMK7cXsDL2kQvASyT0t+7rOA65xz/63meu8Bt+Pd8P0AHIz3uyJSZxZkKEkkZpUElJXAflV1q4qEyswccKhzLmxZ+SKRoha3xBXz5tb+BZiqoC0iyUjTwSRulCT4/ISXrdvH5+KIiPhCXeUiIiJxRF3lIiIicUSBW0REJI7ExRj33nvv7Vq2bBnWc/722280aNAgrOeMV6qLilQfO6kuKlJ97KS6qCjc9bFo0aINzrmAuxLGReBu2bIlCxcuDOs5s7Oz6dmzZ1jPGa9UFxWpPnZSXVSk+thJdVFRuOvDzNYFe05d5SIiInFEgVtERCSOKHCLiIjEkbgY4w6ksLCQ9evXs23btlq9vkmTJqxYsSLMpYpPyVoXaWlptGjRgtTUVL+LIiISsrgN3OvXr6dRo0a0bNkSM6v+BZVs2bKFRo0aVX9gEkjGunDOsXHjRtavX0+rVq38Lo6ISMjitqt827Zt7LXXXrUK2iJmxl577VXrHhsREb/EbeAGFLSlTvT7IyLxKK4Dt98aNmwY1vNNmjSJ77//PqznjHVr166lTZs21R533333Vfj+2GOPjVSRRERimgJ3DImVwO2co7i42O9iVFA5cH/yySc+lUREEsXMnFy6j51DqxFv0H3sHGbm5PpdpJAocIfJ+PHj6dy5M+3atWPUqFFlj/fv359OnTpx1FFHMWHCBACKiooYPHgwbdq0oW3btjz00ENMmzaNhQsXMmjQINq3b09BQUGF8//www/06NGD9u3b06ZNGz788EMAJk6cyGGHHcYJJ5zAkCFDuOaaawAYPHgw06ZNK3t9ae9Afn4+J510Eh07dqRt27a89tprgNfyPeKII7j66qvp2LEj3333HbNnz6Zbt2507NiR8847j/z8fABGjBjBkUceSbt27bjpppt2qYsPPviA9u3b0759ezp06MCWLVtwzjF8+PCy9/zSSy/t8rpJkyaVlR/gzDPPJDs7mxEjRlBQUED79u0ZNGhQhfcT7Lylqxide+65HH744QwaNAjthCcipWbm5DJy+jJy8wpwQG5eASOnL4uL4B23WeWxZPbs2axevZrPPvsM5xx9+/Zl7ty59OjRg2effZY999yTgoICOnfuzDnnnMPatWvJzc3liy++ACAvL4+MjAwee+wxHnjgAbKysna5xgsvvEDv3r257bbbKCoqYuvWrfzwww+MGjWKRYsW0aRJE3r16kWHDh2qLGtaWhozZsygcePGbNiwga5du7J48WIAVq1axcSJE3n88cfZsGEDo0eP5t1336VBgwaMGzeOBx98kGuuuYYZM2awcuVKzIy8vLxdrvHAAw/wj3/8g+7du5Ofn09aWhrTp09nyZIlLF26lA0bNtC5c2d69OgRUv2OHTuWxx57jCVLluzyXFXnzcnJ4csvv6R58+Z0796djz/+mOOOOy6ka4pIYhv/9ioKCosqPFZQWMT4t1fRv0OmT6UKTWIE7uuvhwAf6lVJLyqClJTgB7RvDw8/HNK5Zs+ezezZs8uCZn5+PqtXr6ZHjx48+uijzJgxA4DvvvuO1atX07p1a7755huGDRvGGWecwamnnlrtNTp37szll19OYWEh/fv3p3379rz33nv07NmTffbx1qG/4IIL+Oqrr6o8j3OOW2+9lblz51KvXj1yc3P5+eef2W233TjooIPo2rUrAPPnz2f58uV0794dgO3bt9OtWzcaN25MWloaV155JWeccQZnnnnmLtfo3r07f/nLXxg0aBADBgygRYsWfPTRRwwcOJCUlBSaNWvGCSecwIIFC2jXrl1IdRxMsPM2btyYLl260KJFCwDat2/P2rVrFbhFBIDv8woCPp6bV0D3sXP4Pq+A5hnpDO/dOuYCubrKw8A5x8iRI1myZAlLlixhzZo1XHHFFWRnZ/Puu+8yb948li5dSocOHdi2bRtNmzZl6dKl9OzZk3/84x9ceeWVu5zz008/LetunjVrFj169GDu3LlkZmZy8cUXM3nyZCB4ZvRuu+1WNk7tnGP79u0ATJkyhV9++YVFixaxZMkSmjVrVjYlqvzONs45TjnllLL3tHz5cp555hl22203PvvsM8455xxmzpxJnz59drn2iBEjePrppykoKKBr166sXLkypG7q8mUGQpqqVdV5d99997L/p6SksGPHjmrPJyLJoXlGesDHDWK++zwxWtwhtozLKwjjoiO9e/fm9ttvZ9CgQTRs2JDc3FxSU1PZvHkzTZs2ZY899mDlypXMnz8fgA0bNlC/fn3OOeccDj74YAYPHgxAo0aN2LJlCwDHHHNMha7hdevWkZmZyZAhQ/jtt99YvHgxt9xyC9dddx0bN26kcePGvPLKKxx99NGAt6PaokWLOP/883nttdcoLCwEYPPmzey7776kpqby/vvvs25d4A1ounbtyv/93/+xZs0aDjnkELZu3cr69etp3rw5W7du5fTTT6dr164ccsghu7z266+/pm3btrRt25Z58+axcuVKevTowZNPPsmll17Kr7/+yty5cxk/fnyF4NyyZUsef/xxiouLyc3N5bPPPit7LjU1lcLCwl1WOQt23pUrV9b0xygiSWR479aMnL6sQne5AZWbArHYfZ4Ygdtnp556KitWrKBbt26Alzj1/PPP06dPH5544gnatWtH69aty7qhc3Nzueyyy8pal2PGjAG8hLI///nPpKenM2/ePNLTd94RZmdnM378eFJTU2nYsCGTJ09m//33584776Rbt27sv//+dOzYkaIi75dwyJAh9OvXjy5dunDSSSeVtaYHDRrEWWedRVZWFu3bt+fwww8P+J722WcfJk2axMCBA/n9998BGD16NI0aNaJfv35s27YN5xwPPfTQLq99+OGHef/990lJSeHII4/ktNNOo379+sybN4+jjz4aM+P+++9nv/32Y+3atWWv6969O61ataJt27a0adOGjh07lj03dOhQ2rVrR8eOHZkyZUrZ42effXbA8ypwi0hVSgPx+LdXlXWL5wbpPg/Wre4Xi4dM26ysLFd5P+4VK1ZwxBFH1PqcibjM56RJk1i4cCGPPfZYjV6XiHURqkC/R9pneCfVRUWqj50SsS66j50TMHhnZqTz8YgTq3xtBPbjXuSc2zVTmQiOcZvZs2b2s5l9Ue6xPc3sHTNbXfJv00hdX0REpCaG925NemrFpOX01BSG927tU4kCi2Ry2iSgcubSCOA959yhwHsl30uYDB48uMatbRER8fTvkMmYAW3JzEjH8FraYwa0janxbYjgGLdzbq6Ztaz0cD+gZ8n/nwOygVsiVQYREZGa6N8hM+YCdWXRng7WzDn3A0DJv/tG+foiIiJxLaLJaSUt7tedc21Kvs9zzmWUe36Tcy7gOLeZDQWGAjRr1qzT1KlTKzzfpEmTgFORQlVUVERKVQuwJJFkros1a9awefPmCo/l5+eHfQOZeKW6qEj1sZPqoqJw10evXr2CJqdFezrYT2a2v3PuBzPbH/g52IHOuQnABPCyyitn661YsaJOmdDJnEldWTLXRVpa2i7LxCZitmxtqS4qUn3spLqoKJr1Ee2u8lnApSX/vxR4LcrXj7rTTz894Hre5d1xxx28++67tTp/dnZ2wGVHRUQkMUWsxW1mL+Ilou1tZuuBUcBY4GUzuwL4FjgvUtf3m3MO5xxvvvlmtcfefffdUSiRiIgkgoi1uJ1zA51z+zvnUp1zLZxzzzjnNjrnTnLOHVry76+Run5llfddfeOLn+p8zgcffJA2bdrQpk0bHn744YBbY7Zs2ZINGzYAcM8993D44YdzyimnMHDgQB544AGg4hacLVu2ZNSoUWXbbpauAPbZZ59x7LHH0qFDB4499lhWrVpV5/KLiEj8SYpNRgLtu3rnG6vrtHD8okWLmDhxIp9++inz58/nqaeeYtOmTaxatYpLLrmEnJwcDjrooLLjFy5cyKuvvkpOTg7Tp0+n8kpw5e29994sXryYq666qiy4H3744cydO5ecnBzuvvtubr311lqXXURE4ldSrFUeaN/VbTuK67Rw/EcffcTZZ59dtgb4gAED+PDDDytsjVn5+H79+pWtP37WWWcFPfeAAQMA6NSpE9OnTwe8zUEuvfRSVq9ejZmVbRoiIiLJJSla3MEWiK/LwvHBptGV3xozlOMDKd2OsvxWlLfffju9evXiiy++4N///ndIW16KiEjiSYrAHWzf1WCPh6JHjx7MnDmTrVu38ttvvzFjxgyOP/74oMcfd9xxZQE3Pz+fN954o0bX27x5M5mZXu/ApEmTal1uERGJb0kRuAMtHJ+2W706LRzfsWNHBg8eTJcuXTjmmGO48sorado0+J4pnTt3pm/fvhx99NEMGDCArKwsmjRpEvL1br75ZkaOHEn37t3Ltu4UEZHkkzTbes7Mya2w7+qwEw7kwm61X3mtNkpX1tm6dSs9evRgwoQJFfac9ksyL8CibT2rprqoSPWxk+qiomhu65kUyWmw68LxW7ZsiXoZhg4dyvLly9m2bRuXXnppTARtERGJL0kTuGPBCy+84HcRREQkRJV7aof3bh0TO4cpcIuIiFRSuv5H6VTi3LwCRk5fBuB78I7rwO2cw8z8LobEqXjI7xCR4CLZIg60/kdBYVGd1v8Il7jNKk9LS2Pjxo368JVacc6xceNG0tLS/C6KiNRCoBUxR05fVqcVMcuLxPof4RK3Le4WLVqwfv16fvnll1q9ftu2bfrQLpGsdZGWlkaLFi38LoaI1EKkW8TNM9LJDRCk67L+R7jEbeBOTU2lVatWtX59dnb2LvswJyvVhYjEm0i3iIf3bl1hjBsgPTWlTut/hEvcdpWLiEjyisSKmOX175DJmAFtycxIx4DMjHTGDGi7a2veOXjsMVIKoteFHrctbhERSV7RaBFXXv8joKeegmHD2Pcvf4HTTgvbtauiwC0iInGnNKD6Os/6q6/ghhvg5JP54YwziFYnugK3iIjEpZBaxJGyfTsMGgRpafDcc14QjxKNcYuIiNTUnXfCwoVeV3nz5lG9tAK3iIhITXzwAYwdC1deCQMGRP3yCtwiIhLXZubk0n3sHFqNeIPuY+eEbRGWgH79FS66CA45BB56qOzay3I3R/7aJTTGLSIicSuqa4o7B0OGwI8/wrx5zFy9eee1D4jeeuZqcYuISNyqagW1sHv6aZg+He67D7KyonvtchS4RUQkbkVtTfEVK+C66+Dkk+HGG6N77UoUuEVEJG5FegU1ALZtg4EDoUEDmDwZ6tWL3rUDUOAWEZG4Nbx3a9JTUyo8FvY1xUeMgKVLYeJE2H//6F47ACWniYhI3Ir4CmpvvgmPPALDhsGZZwa9NmwhM0qrtylwi4hIXIvYCmo//ACDB0O7dnD//VVeOzs7m2GDeoa/DAGoq1xERKSy4mK45BLIz4cXX/SWNo0RanGLiIhUdv/98O673pKmRx7pd2kqUItbRESkvHnz4K9/hfPPhyuu8Ls0u1DgFhERKfXrr3DhhXDggTBhApj5XaJdqKtcREQEvCVNL7vMS0r75BNo0sTvEgWkwC0iIgLw6KMwaxY8/DBkZfldmqDUVS4iIrJgAQwfDn37wrXX+l2aKilwi4hIctu8GS64wFsVbeLEmBzXLk9d5SIikrycgyuvhG+/hQ8/hD339LtE1VLgFhGR5PXEEzBtGowbB926VXv4zJzcyC2vGiIFbhERSU6LFsH118Npp8FNN1V7+MycXEZOX1a2B3duXgEjpy8DICOiBa1IY9wiIpJ88vLgvPOgWTP417/KtuoEL0B3HzuHViPeoPvYOczMyQW8zURKg3apgsKikk1GokctbhERSS7OeSuiffcdzJ0Le+1V9lRVrerv8woCns57vEHEi13Klxa3md1gZl+a2Rdm9qKZxc7q7SIiktCW3XQnTJ/OvcdfQvcPCspa1FB1q7p5RnrA8wV7PFKiHrjNLBO4FshyzrUBUoALo10OERFJPnOfncERD43mrcOO5anOZ5e1qEuDd1Wt6uG9W5OemlLh8fTUFIb3bh3xcpfn1xj3bkC6me0G7AF871M5REQkWfz0E0deP4TvMppx8+nXlc3XLj9OXVWrun+HTM7plElKyetSzDinU4T2Aq9C1AO3cy4XeAD4FvgB2Oycmx3tcoiISBLZsQMGDqRBQT5X9b+VLbtXHJMubWlX1aqemZPLq4tyKXIOgCLneHVRboWu9mgwV1KAqF3QrCnwKnABkAe8Akxzzj1f6bihwFCAZs2adZo6dWpYy5Gfn0/Dhg3Des54pbqoSPWxk+qiItXHTvFWF62eeoqDXniB9/98HV8c12uX5+un1KP1fo0AyCso5KfN29heVEz9lHo0a5JGRnoqq37cwvai4oCvzWxoYa2PXr16LXLOBVww3Y+s8pOB/zrnfgEws+nAsUCFwO2cmwBMAMjKynI9e/YMayGys7MJ9znjleqiItXHTqqLilQfO8VVXcyaBS+8AEOGsHnocB4vlzUOXot6zIC29Kymy/uyEW/gAnRUGzCxT4Oo1YcfgftboKuZ7QEUACcBC30oh4iIJLpvvoFLLoGOHeHRR+mf5k1iqs3qZ80z0skNkLwW7azyqAdu59ynZjYNWAzsAHIoaVmLiIiEzbZtcO65XhLatGlQErT7d6hdQtnw3q0rzPEutXX7DvIKorcxiS9Z5c65Uc65w51zbZxzFzvnfvejHCIikqCcg//7P8jJ8VZGa9Wqzqfs3yGTMQPakpGeWuHxTVsLyd1UELUkNS15KiIiiWfCBHj2WbjtNjjzzLCdtn+HTBrsvmtndbFzUVv6VIFbREQSy/z5MGwY9OkDd90V9tNXvfRp5Clwi4hI4vjpJzjnHDjgAJgyBVJSqn9NDfm99KkCt4iIJIbCQm/Hr02bYPp02HPPiFwm0CIt9cyitvSpdgcTEZHEMHw4fPghPP88HH10xC5TmpFefkpZZtOiqC19qsAtIiLxb8oUeOQRuO46GDQo4perPKUsOzs74tcspa5yERGJb0uWwJAh0KMHjB/vd2kiToFbRETi14YN0L+/N5798suQmlr9a+KcuspFROLEzJzcWi3VmbAKC+GCC+DHH72x7WbN/C5RVChwi4jEgZk5uRWW28zNK2Dk9GUAyRu8r78e5syBSZOgc2e/SxM16ioXEYkD499etcsa2QWFRVFbrSvmPPYYPP64l0l+6aV+lyaq1OIWEYkDfq/WFVNmz/ayx/v2hTFjAh6SyMMKCtwiInEgVraU9N2KFXD++dCmjTdfO8DKaKEOK8RrcFdXuYhIHAi0Wld6akrUVuuKCRs3wllnwe67w6xZ0KhRwMNCGVYoDe65eQU4dgb3aO3wVRcK3CIicaB0S8nMjHQMyMxIZ8yAtnHRQgyL7du9NcjXr4eZM+Ggg4IeGsqwQjznDKirXEQkTlRerStpOAdXXw0ffOB1j3frVuXhoQwrxHPOgFrcIiIS2x56CJ55xttbO4TlTEMZVvB7h6+6UOAWEZHY9frrcNNNXjf53XeH9JJQhhXiOWdAXeUiIhKbvvgCBg6EDh3gueegXuhtzeqGFQLt8BUvWeUK3CIisgvfp0r9/LOXQd6okZdB3qBB2C8RrzkDCtwiIlJBTZZXjUiA//13GDBg5xrkmfEXXCNJY9wiIlJBqFOlIjIX2jkYOhQ+/tjrHs/Kqv25EpQCt4iIVBDqVKmIzIW+/36YPBnuustbIU12ocAtIiIVhDpVKuxzoWfOhJEj4cIL4fbba3eOJKDALSIiFYQ6VSqsc6Fzcrw52p07w7PPglnNz5EkFLhFRKSCUJdXDdtc6B9/9Hb62nNPr9WdHvuLoPhJWeUiIrKLUKZKhWUudEEB9OsHv/7qJaTtv39dip0UFLhFRKTW6jQX2jm4/HJYsACmT4f27cNbuASlwC0iIv64+26YOhXGjoX+/f0uTdzQGLeIiETfpElw550weDDcfLPPhYkvCtwiIhJds2fDkCFw8snw5JPKIK8hBW4REYmepUvh3HPhyCPh1Vehfn2/SxR3FLhFRCQ61q2D00+HJk3gzTehcWO/SxSXlJwmIiKRt2ED9O4NW7dq45A6UuAWEZHIys+HM87wWtzvvANt2vhdorimwC0iIpFTWOiNaS9cCDNmwHHH+V2iuKfALSIikVFc7C2w8vbb8PTT3rKmUmcK3CIiEn7OwfDh8PzzcO+9cMUVzMzJrdvyqAIocIuISCQ88AA8+CAMGwYjRzIzJ5eR05eV7d+dm1fAyOnLABS8a8iX6WBmlmFm08xspZmtMLNufpRDREQi4LnnvNXQLrgAHn4YzBj/9qqyoF2qoLCI619aQvexc5iZk+tTYeOPXy3uR4D/OOfONbP6wB4+lUNEROqgcvf3A3usp9tfrvBWRXvuOajntQ+/zysIeo7cvAKGv7IUUOs7FFFvcZtZY6AH8AyAc267cy4v2uUQEZG6Ke3+zs0rwAH7fplD+xuHsumwo7zdvnbfvezY5hlV77FdWOy4c9aXES5xYvCjq/wPwC/ARDPLMbOnzayBD+UQEZFayiso5MaXl5Z1fx+84TuenXYXPzbak4sG3AGNGlU4fnjv1qSnplR7TqmeOeeie0GzLGA+0N0596mZPQL8zzl3e6XjhgJDAZo1a9Zp6lojqk4AACAASURBVNSpYS1Hfn4+DRs2DOs545XqoiLVx06qi4pUH568gkIKC7byY0nvd8ONv3DuXSOoV1TMtFFj+d++zWib2STg637avI3tRcVBzx3odfEg3L8bvXr1WuScywr0nB9j3OuB9c65T0u+nwaMqHyQc24CMAEgKyvL9ezZM6yFyM7OJtznjFeqi4pUHzupLipSfXi6j53DhQfA35btxp5bN/PKlLtw+QWc98exLP8pk8zf0xk2qGfQ13e4ezabtu7aum66Ryo5VbwulkXzdyPqXeXOuR+B78ysdclDJwHLo10OERGpndJEs0a//8bkl+8g838/c+W5d7C82R9IT01heO/WVb5+1FlHkZpScSvP1BRj1FlHRazMicSvrPJhwJSSjPJvgMt8KoeIiNRQ84x0dtv2C89Mu5vWv6xlyIDb+eyANqSYMWZA22ozw0uf12IsteNL4HbOLQEC9t2LiEhsu6VXS9oPuZHM3JUM63sz2QdnkZ6aElLQLtW/Q6YCdS1p5TQREQldYSF977sBli3hvnNu4q1DjiNTLeaoUuAWEZHQFBXBxRfDrFmsvvZabn1kPLf6XaYkpMAtIiLVKy6GIUPgpZdg3Dhyu3ThUL/LlKQUuEVEEkBEd95yztssZOJEGDXKW4c8Ozs855YaU+AWEYlzEd15yzm44QZ4/HFvm85Ro+paXKkjX3YHExGR8Am289b4t1fV7cTOwS23wCOPwHXXwbhxYFb96ySiFLhFROJcsJ23qtqRKyR33AHjx8NVV8FDDyloxwgFbhGROBds563qduQKyjm47TYYPRquvBIee0xBO4YocIuIxLlAO2+FsvRoQM7BX/4C993nZZE/+WTZntoSG5ScJiIS58K2hGhxMVx9tResr70WHn5YLe0YVG3gNrNrgCnOuU1RKI+IiNRCnZcQ3bEDrrgCJk+GkSPh3nsVtGNUKP0f+wELzOxlM+tjpp+kiEhCKSyEP/7RC9r33ON1k+ujPmZVG7idc38FDgWeAQYDq83sPjM7OMJlExGRSNu2Dc45B155BR54AP76V79LJNUIKePAOeeAH0u+dgBNgWlmdn8EyyYiIpG0dSv07Qv//jf84x9w441+l0hCEMoY97XApcAG4GlguHOu0MzqAauBmyNbRBERCbstW+DMM+Gjj7ylTAcP9rtEEqJQssr3BgY459aVf9A5V2xmZ0amWCIiEjGbNsFpp8HChTBlClx4od8lkhqoNnA75+6o4rkV4S1O9EV0YX4RkVjzyy9w6qmwfDm8+ir06+d3iaSGknoed0QX5hcRiTU//AAnnwzffAOvvQZ9+vhdIqmFpF4OJ2IL84uIxJpvv4UePWDdOnjrLQXtOJbULe6ILcwvIhJLvv4aTjrJG9t+5x3o1s3vEkkdJHWLO+wL84uIxJqVK72W9pYtMGeOgnYCSOrAHdaF+UVEKpmZk0v3sXNoNeINuo+dw8yc3OgW4PPPvaBdVAQffACdOkX3+hIRSd1VHraF+UVEKvE9+XXhQi97fI894L33oLUaJIkiqQM3hGFhfhGRAKpKfo34Z87HH8Ppp8Oee3rd461aRfZ6ElVJ3VUuIhIpviW/zpnjtbT32w8+/FBBOwElfYu7trRwi4hUpXlGOrkBgnREk1/feMPbMOTQQ73s8f32i9y1xDdqcddC6dhVbl4Bjp1jV1FPPBGRmFXX5NcaJ7ZNmuStgnbUUfD++wraCUyBuxa0cIuIVKd/h0zGDGhLZkY6BmRmpDNmQNuQeuZq1DhwDu69Fy67DE48EbKzYe+9w/12JIaoq7wWtHCLiISitsmvISe2FRXBtdfC44/DRRfBM89A/fp1LfYuAg0NZoT9KhIqtbhrQQu3iEgkhdQ4KCiA88/3gvbNN8Nzz0UsaAdq/ecVFIb9WhIaBe5a0MItIhJJ1TYONm6EU06BGTPgkUdg3DioF5mP82Ct/582b4vI9aR6Cty1UJexKxGR6lTZOPj6azj2WG+BlZde8rrKIyhY6397UXFEryvBaYy7lrRwi4hEStBVHQvWwan9oLgY3n0Xjjsu4mUJNq2tforafX5R4E5wmm8uEp92aRy8+KKXOX7AAd587cMOi0o5hvduXWHpVvBa/82ahH88XUKjW6YEpvnmIgnAORg9Gv74R+jSBebPj1rQhuBDgxnpqVErg1SkFncC83WtZBGpu99/h6FDYfJkb7rX00/D7rtHvRiBhgazs1dHvRziUYs7gWm+uUgc27DBW3N88mS4+27vXx+CtsQeBe4EpvnmInHq88+hc2f49FOYMgVuvx3M/C5VmZk5uaz6cYt/+4wnOQXuBKb55iL+q/Ga4zNnetO9fv8d5s71xrZjSGnuzPaiYuXO+ESBO4FpvrmIv2q85vjo0XD22XDkkd487S5dol7m6mivBv/5lpxmZinAQiDXOXemX+VIdJpvLlK1SE6ZDDlB9LffvKler7ziJaFNmADpsTmkpdwZ//mZVX4dsAJo7FcBojnHWfOpRWJPaYu4NLiWtoiBsPx9hhTkvv0W+veHJUvg/vvhpptiajy7Ml/2GZcKfOkqN7MWwBnA01G/uHNw//1sW/9j1OY4az61SGyKdLdvtQmiH3zgJaF9/TW8/joMHx7TQRuUOxML/Brjfhi4GYj+YrfffAN3383xw67mqP9+XuGpSI3TaExIJDZFuts3aJA79TB46CE46STIyPAWVTn99LBcM9JKc2fqp9RT7oxPzDkX3QuanQmc7py72sx6AjcFGuM2s6HAUIBmzZp1mjp1atjK0OC//+XwW29jj19+5qNBl/H5KWdUuMttm9kkbNcCWJa7Oehz4b5WbeTn59OwYUO/ixEzVB87xUpd5BUU8tPmbWwvKqZ+Sj2aNUkLy8pdq37cEnCzjPop9Wi9X6NdHq9NfVQue/P6jmP+/hD7Zmfzy/HHs/KWWyhq0KBO5wxXfdRErPxuxIpw10evXr0WOeeyAj3nR+AeA1wM7ADS8Ma4pzvnLgr2mqysLLdw4cKwlmPiMy9ywJiHOPnrBcw4sie39r6GgvppZGak8/GIE8N6re5j5wQcE4rEtWojOzubnj17+l2MmKH62CkW6qLyODR4rdZwtPICndsAh/f3WTkXpc71sWoVDBgAK1fCffd5+2jXsGs8kvVRE7HwuxFLwl0fZhY0cEe9q9w5N9I518I51xK4EJhTVdCOlKbN9+baC+/kb8cNot/yD5jxrxs5cvP3ERmnicUxofJzS1f9uEXj7RKzIjnUVH7KJOwM2hCBXJSZM73x7J9/hrffhltuqdV4tobeJGnncWekp3LfOUcz/YzLGXz+XTQryOO1yTfQf+XcsF8r1uZTV06W215UrGQ5iVmRHofu3yGTj0ecSGZGOpX7H8MSEHfsgBEjvPnZhx8OixbBySfX+nSajiW+bjLinMsGsv26/s45zidC7uVwwQXeKkUffsisS25kXPa6sE3fiqX51Np8RCqL5emK0Zp+FJGAuH49DBrkrYA2dCg88gikpdX+fEDGHqls2lq4y+OajpU8kq7FXdpFvCx3c8XlBzMz4f33vTGnf/6Tg8/uQ721/03I6Vu6Y5fyYn26YrSGmsK+tv+MGXD00V4L+1//giefrHPQnpmTS/62Hbs8nppimo6VRJIqcJf/gIIAH1CpqTBuHLdcfA8tNv3AG5Ouo/eqT4DEGkPS5iNSXqyPmYZ7qCnY2uFhu0HYuhX+/GcvCe0Pf4CcHG81tDAY//YqCot3TShuUH+3mOkhkchLqv24Q+0ifrl5Bz6+7FEee20sT868j8kdzuDeE68gN8/LEI+lbsTaGN67dcCsVN2xJ6eqe2BqNk0JItPtHq6hplBWSqtT2ZcsgYEDvezxW27xtuOsX7/O5S4V7Ge1uWDXrnNJXEkVuEPtIm6ekc56mnHeoPsZ/sFkhi6YQef1X3JN31v4mgPCuiSiHyp/QNVPqacFFHwQK+PK4RxDjvQSonVV3c17rW8Qiovh0Ue9YL333vDOO97iKmGm5UYFkqyrPNQu4tIus8KUVO478QoGn3sn+/y2idefu55BOW9SsH1HzHQj1lZpJu1/x55B6/0axcSHajKJpXHlcI4hx3q3e6CgB3XM7/jpJzjjDLjhBjjtNFi6NCJBG2JzaqlEX1IF7lB/6SvP7cw+OIvTLvs7C1ocyb2zH+ep6few7fsfo1ZuSTyxFODCOYYcy4mPM3NyCTZrutYt1rfegnbtIDsb/vlPLyFt771rW8RqxdrUUvFHUnWVl+8ihi0BV0Yqf2z/Dpllq5790nBPLj3/LgYv+jcjsify9qRhcMZe0Lt3lN+FJIJYC3DhGkOO5a7c8W+v2mWeNniLrtS4xbptmzc3+5FHvMD9/vveHtpREEtTS8UfSdXihp1dxG0zm/DxiBOr/QMo30p3Vo+JWf04//JHqb/vPtCnD1x3nZdFKlUKlsmbrBI1sz+Wu3KD3RQ5ajj+vnw5HHOMF7Svuw4+/TRqQVsEkjBw11SgrqnLrupL42U5MGyYl5DSsSN89pnfRY1ZsTSeGytiOcDVRSx35Qa7KcoM9WbJOZrPmgWdOsEPP8Abb8DDD9d5brZITSVVV3ltBe2aevRR6NcPLrsMjj0WRo6E228P6/SPRKCV2nYVlqlHMSpWu3LrNA3yp5/gT3/isNde84bHJk2C/faLXGFFqqDAXVcnnQTLlsH118Po0fD66zB5MrRt63fJYkasjefGilgKcLEyNS2SanWz5By89BJccw3k57Pm6qs55O9/h3rqrBT/KHCHQ5MmMHEi9O/vrUfcqRP89a9e8opa3zGdsCRVz73O8LNgEVCjm6WffoKrr4bp06FLF5g0ifU//cQhCtriM/0GhlO/fvDFF3DeeTBqFGRlwYIFZU8na4JWoo7nJopYmpoWE5yDqVPhqKO8cexx4+Djj+GII/wumQigwB1+++wDU6bArFnw66/QtSsMH86/P1mTtAlasZywJBrKqCA319t+c+BAOPhgb53xm2+G3dQ5KbFDv42RctZZ0KOH90f/wAO0n/giR5/6f8w/sF3ZIcmUoBVL47lSkYYy8JYsfeop7++1sBDGj/fyVmIwYCdDPoJUTS3uSGrSxNvKb84cioqKmfrirdz3n8dovC2/7JB4btUka9d/okn6oYzly6FnT29Hr6wsL9n0ppuiErRr+jekqZUCCtzR0asXl9/wNE917s8Fn8/m3aev4rzPZ1OvuChuWzX6AEkc8TiUEZabxoICuO02aN8evvwSnnkG3n3X6yKPgtr8DSkfQUBd5VFz7VlHM7LwT8w8sif3vPNPxr/1KFcumsXGO0b7XbRa0dzsxBJPQxlh2YHs7be9jPFvvoFLLoEHHvDyU6KoNn9DykcQUIs7akpbNXmHt+Wcix7gtj/eQYv6jmOvuchb0OHzz/0uYo3oA0T8UqdW57p13qyPPn28rvA5c+C556IetKF2f0OJulSu1IwCdxSVbaU57kzunXIXDb7+ylsyceFCr7vu8su9rNY4oA8Q8Uutbhp/+w3uuAMOP9yb4nXPPd7Ncq9eESpl9WrzN5T0+QgCKHD7q359b5OCNWvgxhu9aWSHHuotm7pli9+lq5I+QMQvNQp4znl/V61be8F6wABYtcpbIGn33SNc0qrV5m8oHvMRJPw0xh0Lmjb1pp9cfbWXLDN6NEyYAHfdBVdeGZNTUhJ5rW2Jbb0O34cp87+tsEVnwID32WfejfH8+V62+Msve3sKxIja/g3FUz6CREbsRYRk1qoVvPAC3HCDNx3lqqu8rQPvvx/OPBPM/C5hBfoASS6xMH94Zk4ury7KrRC0DTinU7nfxe+/9zb8mTzZ2whk0iS4+OKYXF9cf0NSG7H3myzQuTNkZ8PMmV5XX9++cOKJsGiR3yWTJBUr0/8CJaY54P2Vv8C2bXDffXDYYd6SpSNGwFdfwaWXxmTQFqkt/TbHKjNv7fNly+Af//DmmWZlwaBBsHat36WTJBMr84cDJqA5R4d5b8ORR3pDTaeeCitWwJgx0KhRVMsnEg0K3LEuNdUb+16zBm691dup6PDDvaUZ8/L8Lp0kiViZ/lc5Aa372iXMmnwDj8263wvS773n/Y384Q9RLZdINClwx4vGjeHee72uvwsv9BaMOPhgbwx8+3a/SycJLlam/5VmYrf74Sv+NfWvTHnpr+xV8D8W3f0wLF7sDSmJJDgF7nhzwAFess3ixdChg7cRwpFHwrRp3ni4xKx4Xts9Vqb/9U/ZyHsfP8ysyX/hyJ+/4ZEzrmbR2/PodPt1kJJS/QlEEoACd7xq3x7eeQfefBPS0rzVoLp3h3nz/C6ZBBDO5C4/bgB8nz+8dKn3O3700eyz4BOePvlSTvjT07x83DkU1/d3PrZItGk6WDwzg9NOg1NO8Vrht9/uzVM991wvMeeQQ/wuoe9iYQoThG9t9+rW6Y7k+/Vl6tLChd7CKbNmQePGrLryOi5p0p2fdtsDgPzarFMuEufU4k4Eu+3mLdSyejXceSc73niTwsOPYGJWX/rd+nJcdcmGU6xMYYLwJXdVdQMQS++3zj75xLsp7dwZPvzQW4xo3TouP7hvWdAupd2xJNkocCeShg2Z2fdKThz6FK+0OYlLFr/By/cPYtuVQ5k962O/Sxd1sTKFCapP7gq1+7uqG4BYer+14hy8/z6cdJI37LNwoddztHatt854RkbMZLeL+EmBO8GMf3sV36Y14dY+w+g15EmmtT2Zs5fO5qT+PeD882HBgoCvi+fEqWBi6UO+quSumrSUq7oBiKX3WyNFRV5y5THHeFnhy5fD3/7mBewRI7wZFSUikd2eiL/7ktgUuBNM+Q/pb5vuz229r+H4Pz3DhC4DYPZs6NIFevaEf/8biosByCsoTJwu1nJiZQoTVJ3cVZOWclU3ALH0fkOSnw9//7u30tl558GmTfDEE94e2X/5CzRosMtLwp3dnlDDC5I0lJyWYJpnpJNbqYX1c6O9eL7/VVx19TPw1FPe3O++fb2dyK69ll/32IeCwoYVXlObxKlYM7x36wqJXODvDmbBkrtq0lKubmOKaL3fOiXBffMN/POf3u/i5s1eQuW4cXD22dVO6Qr35jbhShoUiSYF7gRTZbBq3JiZJ17Ig9va0u6z97gq598cNWwYF6WlU+/Ik5jc8Uy+2atF2etivou1GvGyg1mgm63SxwMJdgMQrfdbXWZ7QEVF8J//wOOPw1tveWuHn3OO17I+5pgaXT+c2e2h3jSV3qhceMAWbhs7JyZ/jyR5KHAnmKo+vMt/4H57RA9eP6IHR//wFeO+mcXAef9h8OLX+eigo3m+w+m8e8gxNNsr/td5jofdl8LZMxCN91ujVuqGDfDss14X+H//C/vv7yWaDRkCmf7/XJqkp5JXUBjw8VIVblQOCPFGRSSCFLgTULAP70AfuEv3P4x3Tr2e5zsN4dzF/+GPS/7DEzPH8EuDpmw6byCsOVDzwSMsXnoGSlXbSi0u9qZwPfOMtwf27797eRXjxkH//t76+zEi2E655R9Xd7rEGgXuJFJV1/fvTfdm5umDeaLruZz98xfcuG4uh/3rSZj0OPTqBYMHw4AB0LBh0HMkqmgs4hIPPQOlgnXtdyrO8+ZbP/ec17pu1MhbX+Dqq71leSOkLj+fvK27trYrPx632fqSsBS4k0iwD1yAzQWFLBl1asl3fYFb4fvvYeJEr6vz0kvhqqu8BKKLLoKTT/YWfklwtRrPTXDlu/b32F7Aaas+4fwv3+OYdZ97TdWTTvJWOzv7bNhjj+pPWAd1/fmEkl9Q0xwEkUiL+nQwMzvAzN43sxVm9qWZXRftMiSr4b1bE6RnMPCHUPPm3v7Ga9Z4XZ8XXQRvvOGtaNWihZdYtHhxQm9uEveLmkRA/zb78kyzX3ji7YdZ8NjF/O3Nhziq+H8werQ39/qdd7x944ME7XDOm67rzyeU6WWxssGKSCk/mkw7gBudc4vNrBGwyMzecc4t96EsSaV/h0wWrvuVKfO/pXyorWdW9YeQGRx3nPf16KNe8H7+eXjsMXjoIW8e7gUXeHNx27QJPnAYh9RNWqKw0FvVbNo0mD6dYzduhIwMuPQiGDyYhsceG9LPPdw9GHX9+YSSX1D+GNhCZoznIEjii3rgds79APxQ8v8tZrYCyAQUuKNgdP+2ZB20Z4UPqsymRaF/CO2+uzfWPWAA/PorvPKKl4B0771e9+ihh3rPnXUWdO0a0a0WozH2nNTdpL/9xt4ffeSNWc+a5f28GzaEM8/09oTv08f7faiBcCd6hePnE0p+Qekx2dnZDBvUs6bFFAkrcz52c5pZS2Au0MY5979Kzw0FhgI0a9as09SpU8N67fz8fBomYaJVIOGoi9RNm9jj/Q/YO/sDmi9fRkpREb83bsKmbl3Z0K0bmzp3piiM4515BYXkbiqguNzvbz0zMpumk5Fet6zl8vURyevEovobNrDXvHns/cknNF20iHqFhexo0ICN3brx8wknsKlzZ4prGKzLW5a7OehzbTOb1Ph80f756HNjJ9VFReGuj169ei1yzmUFes63wG1mDYEPgHudc9OrOjYrK8stXLgwrNfPzs6mZ8+eYT1nvApHXZTvAm28LZ8e/13Mqd8soM+3i6n/v81Qv743Jei00+DUU+GII+rUpd597JyALa3MjHQ+HnFiHd7JrvURK1uDRsTWrfDRR/Dee/Duu17OAkDLltCvH0sOPJD2w4aFbQpXJH5u0fz56HNjJ9VFReGuDzMLGrh9SQs2s1TgVWBKdUFb4kP5LtD/pTXk9ZIFXg5sVJ+5x9X31kZ//XW44QbvBS1aeJnpJ57oBfQDDqjR9aI59hxPU7WqtWOHt9HMu+96wXrePNi+3QvMXbt6CWb9+sFRR4EZednZYZ13HWixGcMb6+5eyxXJEurnIxKCqAduMzPgGWCFc+7BaF9fIiNYwPxuy3Y44RQ44QR44AFYt87b7GT2bG/cdNIk78CDD/aS37p29b7atKlyullSjz3XxPbtkJPj7W89Zw588AFs2eI916EDXHutN33r+OMDbuoRbuUTvXLzCjAoS5TUVDuR0PjR4u4OXAwsM7MlJY/d6px704eySJiEHEgPOshb7nLIEG+Frc8/h+xs7+vNN71EKPCmEnXuvDOQd+0K++1XdppY20AkJjgH337r7WM9b573tWiRt3IZeCvg/fGPXqDu1Qv23rvGlwhHt3RpCzlQt7lWJBOpnh9Z5R9B0OnEEqdqFUjr1YP27b2v66/3As/atTB//s6vBx/0piKBF/Q7d4ajj6Z/u3bs0XVf7vr8N77fvC3xxp6rs327N7/+889hyRJvbHrxYti40Xu+fn3o1AmuuQa6dfO+mjcPerpgATmvoJDuY+fwfV4BTdJT+W37DgqLvDay31O5RJJV4i99JVERlvW2zaBVK+9r4EDvsW3bvK7e+fPh00+91uS0aQCcCpzauLG3nOZhh0HRYfB1a+//hxxSo1W7ygeuEe2LycvJ9f8mYPt2b+nQNWtg9eqK/65dW7afOqmp3tDC2WdDx47eV/v2IU/VCja3euG6X9l/awG5ed6UvkCbcfg9lUskGSlwh1FCZx+HINQkoRrVU1razhZjqfx8+OILr7W5dCksX+4lWk2eXPG1mZneTUDLll4yXIsW3mOZmd4uVXvvDWlpuwSu7UXF0Rlr/e03+Plnb2nZ77+H9evh668DB2eAxo29efKdO3td3ocdBu3aweGH13g+dXnB5la/+Ol3XN+m+lkntW0ha7hDpHYUuMNEa1qHJiz11LDhznHv8vLzvYD31VfeV2nwmzvXC4w7dux6rgYNOCa1IS/v3pBN6Y3ZlN6YPyxIY/cte7BhXmPofZR385Be0gosKvKCaVFRaP/fuhU2b4a8vJ1fv/7qBeytW3ctT2lw7tLFC86HHur1Hhx6qHejEYFV6YIF3qIQp4rWtoUcb7uiicQKBe4w0dZ/oYloPTVsuHPMvLLiYi9Y5uZ6Ldsff/TGgzdu5JPZS2i69X80LdjCgXk/sm/uVg7Zuo30Hb/De3UoT716XsDPyNj5tf/+Xtd+s2awzz6w777e2HPz5l5PQNOmUV8yNliXdUoI5ahrC1lTuURqToE7TBIx0SYSXf/B6iPYrmVhU6+el5W+335e0lY5D1bKbr6x7Q7+tmw3Dmhcnw+v7QYFBd4XeEu4pqR45yv9f6DvzeJmzfZgXdbndMqk3tb/Vjg2tZ7RMG038rYWqoUs4hMF7jBJtESbSHX9B6snK7mmH0EgWOC68bQjvVZ8gi/rWFWX9cy3fiQzI0Vd2SIxRIE7TBIt0SZSXdrDe7fmhpeWUHn01JVcsybnDlePQOXAVT+lHud0ymT826u44aUltTp3vCUqBuuyzkhP5eMRPaNfIBEJSoE7TBIt0SZSXf/9O2Ry/UtLAj5Xk3OHu0egfOCa+dY7PPxJbq3PHWuJivF2EyEiVVPgDqNESrSJZNd/ZpBzOwh5vepIJrn9tHkbBYX1an3uaCUqhhKQY+0mQkTqrl71h0gyGt67NempFffSDlfXf6BzlyoNLDNzcqs8RySTAbcXFQd8PNRzRyNRsTQg5+YV4Aheb1XdRIhIfFLgloD6d8hkzIC2ZGakY3it5DED2oallVb+3IGEEliCtfzD0SNQPyXwn0Wo545k2UqFGpATcbaDSLJTV7kEFcmu/9Jztxrxxi6JalB9YIlkMmCzJmmkpxbV+tzRSFQMNSAn2mwHEVGLW8JgZk4u3cfOodWIN+g+dk613dzl1bZ1GskegYz01DqdO5JlKxVqvUVyyENE/KEWt5SpTfZxXZOf6tI6DdYjUNX7CPU91rW3IdKJiqHWW6LNdhARBW4pUdsAXNcM6nAHlqreB5AwGdY1qbdEmu0gIgrcUqK2ATgcyU/hDCzVJW0l0nryCsgiyUmBW4DaB+BYS36qzftQhnX80uIykoyUhfMpnAAABx9JREFUnCZA7ZPEYi35qar3EY1pWhI9oc5lF0k0CtwC1D4ARyODuiaqeh+xdpMhNVd+BsONLy/V4jKSlNRVLkDdksRiaaw1lPehrtX4VDnxsMgFWgFAQx+S+BS4pUwsBeC6qOp9JMp7TEaBEg8D0dCHJDp1lYtIXAilJa2hD0kGanFLlZS1K7Ei2AyGFDOKndPvpyQNBW4JSltCSiwJtlqcn8mQIn5QV7kEpS0hJZbE2gwGEb+oxS1BaUtIiTVKLhRR4JYqxNqqaPFEuQEiEinqKpegtGBJ7WhFLxGJJAVuCUpjirWj3AARiSR1lUuVNKZYc8oNEJFIUotbJMy0mYmIRJICt0iYKTdARCJJXeUiYVaXDVtERKqjwJ1E4nmKUryVXbkBIhIpCtxJoqrlSzP8LFgItPRqcoi3mzMRv2iMO0nE8xSleC67hEZz30VCp8CdJOJ5ilI8l11Co5szkdApcCeJeJ6iFM9ll9Do5kwkdArcSSKepyjFc9klNLo5EwmdL4HbzPqY2SozW2NmI/woQ7KJ5+VL47nsEhrdnImELupZ5WaWAvwDOAVYDywws1nOueXRLkuyiecpSvFcdqme5r6LhM6P6WBdgDXOuW8AzGwq0A9Q4BZJYro5EwmNH13lmcB35b5fX/KYiIiIVMOcc9G9oNl5QG/n3JUl318MdHHODat03FBgKECzZs06TZ06NazlyM/Pp2HDhmE9Z7xSXVSk+thJdVGR6mMn1UVF4a6PXr16LXLOZQV6zo+u8vXAAeW+bwF8X/kg59wEYAJAVlaW69mzZ1gLkZ2dTbjPGa9UFxWpPnZSXVSk+thJdVFRNOvDj67yBcChZtbKzOoDFwKzfCiHiIhI3Il6i9s5t8PMrgHeBlKAZ51zX0a7HCIiIvHIl01GnHNvAm/6cW0REZF4ppXTRERE4ogCt4iISByJ+nSw2jCzX4B1YT7t3sCGMJ8zXqkuKlJ97KS6qEj1sZPqoqJw18dBzrl9Aj0RF4E7EsxsYbA5cslGdVGR6mMn1UVFqo+dVBcVRbM+1FUuIiISRxS4RURE4kgyB+4JfhcghqguKlJ97KS6qEj1sZPqoqKo1UfSjnGLiIjEo2RucYuIiMSdpA3cZnaPmX1uZkvMbLaZNfe7TH4ys/FmtrKkTmaYWYbfZfKLmZ1nZl+aWbGZJW3WrJn1MbNVZrbGzEb4XR4/mdmzZvazmX3hd1n8ZmYHmNn7Zrai5O/kOr/L5BczSzOzz8xsaUld3BWV6yZrV7mZNXbO/a/k/9cCRzrn/uxzsXxjZqcCc0rWkh8H4Jy7xedi+cLMjgCKgSeBm5xzC30uUtSZWQrwFXAK3o5+C4CBzrnlvhbMJ2bWA8gHJjvn2vhdHj+Z2f7A/s65xWbWCFgE9E/G3w0zM6CBcy7fzFKBj4DrnHPzI3ndpG1xlwbtEg2A5LyDKeGcm+2c21Hy7Xy87VaTknNuhXNuld/l8FkXYI1z7hvn3HZgKtDP5zL5xjk3F/jV73LEAufcD865xSX/3wKsADL9LZU/nCe/5NvUkq+Ix5KkDdwAZnavmX0HDALu8Ls8MeRy4C2/CyG+ygS+K/f9epL0w1mCM7OWQAfgU39L4h8zSzGzJcDPwDvOuYjXRUIHbjN718y+CPDVD8A5d5tz7gBgCnCNv6WNvOrqo+SY24AdeHWSsEKpiyRnAR5L6l4pqcjMGgKvAtdX6sFMKs65Iudce7xeyi5mFvGhFF+29YwW59zJIR76AvAGMCqCxfFddfVhZpcCZwInuQRPfqjB70ayWg8cUO77FsD3PpVFYkzJeO6rwBTn3HS/yxMLnHN5ZpYN9AEimsSY0C3uqpjZoeW+7Qus9KssscDM+gC3AH2dc1v9Lo/4bgFwqJm1MrP6wIXALJ/LJDGgJCHrGWCFc+5Bv8vjJzPbp3QGjpmlAycThViSzFnlrwKt8bKH1wF/ds7l+lsq/5jZGmB3YGPJQ/OTNcvezM4G/g7sA+QBS5xzvf0tVfSZ2enAw0AK8Kxz7l6fi+QbM3sR6Im3A9RPwCjn3DO+FsonZnYc8CGwDO/zE+BW59yb/pXKH2bWDngO72+kHvCyc+7uiF83WQO3iIhIPErarnIREZF4pMAtIiISRxS4RURE4ogCt4iISBxR4BYREYkjCtwiIiJxRIFbREQkjihwi8guzKxzyd7saWbWoGSv4aTezlIkVmgBFhEJyMxGA2lAOrDeOTfG5yKJCArcIhJEyRrlC4BtwLHOuSKfiyQiqKtcRILbE2gINMJreYtIDFCLW0QCMrNZwFSgFbC/cy7h96wXiQcJvR+3iNSOmV0C7HDOvWBmKcAnZnaic26O32UTSXZqcYuIiMQRjXGLiIjEEQVuERGROKLALSIiEkcUuEVEROKIAreIiEgcUeAWERGJIwrcIiIicUSBW0REJI78P9PDg5AgOGBvAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot\n",
    "plot_poly_lls(X, y, phi_x, theta_ls, title=\"y = 0.5*x^2 + x + 2 + epsilon\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### <img src=\"https://img.icons8.com/nolan/64/000000/fit-to-width.png\" style=\"height:50px;display:inline\"> Example - Cosine Fitting\n",
    "---\n",
    "* Given $L$ measurements $\\{x_k, y_k\\}_{k=1}^L$ originating from the sinusoidal signal $h(x)$ with additive white Gaussian noise $ y= h(x) + \\epsilon$ where: $h(x)=A\\cos(2\\pi f_0x_k + \\psi)$ and $\\epsilon \\sim \\mathcal{N}(0, \\sigma^2)$ i.i.d and the frequency $f_0$ is given.\n",
    "* **Goal**: find the amplitude $A$ and the phase $\\psi$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img src=\"./assets/tut_08_cos.jpg\" style=\"height:300px\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* **The Loss Function**: the optimization goal is to minimize the mean squared error $$ f_{\\theta}(x) = \\sum_{k=1}^L (y_k - A\\cos(2\\pi f_0x_k + \\psi))^2 $$\n",
    "    * The problem is not linear, not even in parameters\n",
    "    * We will use the following trigonometric identity: $$ \\cos(\\alpha +\\beta) = \\cos(\\alpha)\\cos(\\beta) - \\sin(\\alpha)\\sin(\\beta) \\rightarrow A\\cos(2\\pi f_0x_k + \\psi) = A\\cos(\\psi)\\cos(2\\pi f_0 x_k) - A\\sin(\\psi)sin(2\\pi f_0 x_k) $$\n",
    "    * We will also use the following transformation from polar coordinates: $$ \\theta_1 = A \\cos(\\psi) $$ $$ \\theta_2 = A \\sin(\\psi) $$\n",
    "    * This yields: $$ f_{\\theta}(x) = \\sum_{k=1}^L (y_k - (\\theta_1 \\cos(2\\pi f_0x_k) - \\theta_2 \\sin(2\\pi f_0x_k)))^2 $$\n",
    "        * The problem is now linear!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "\n",
    "    \n",
    "* Denote: $$ \\Phi(X) = \\begin{bmatrix}\n",
    "  \\cos(2\\pi f_0 x_1) & -\\sin(2\\pi f_0 x_1) \\\\\n",
    "  \\vdots & \\vdots \\\\\n",
    "  \\cos(2\\pi f_0 x_L) & -\\sin(2\\pi f_0 x_L)\n",
    " \\end{bmatrix}, \\theta = \\begin{bmatrix}\n",
    "  \\theta_1 \\\\\n",
    "  \\theta_2 \n",
    " \\end{bmatrix}, y =  \\begin{bmatrix}\n",
    "  y_1 \\\\\n",
    "  \\vdots \\\\\n",
    "  y_L\n",
    " \\end{bmatrix}$$\n",
    "* **The Least-Squares Solution**: $$ \\theta^{*} = (\\Phi(X)^T\\Phi(X))^{-1}\\Phi(X)^Ty $$\n",
    "* Finally: $$ A = \\sqrt{\\theta_1^2 +\\theta_2^2} $$ $$ \\psi = \\arctan\\frac{\\theta_2}{\\theta_1} $$\n",
    "* Example: we will fit $y=10\\cos(2\\pi \\cdot 0.8 \\cdot x + \\frac{\\pi}{3}) +\\epsilon$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "L = 100 # num of samples\n",
    "x = np.sort(6 * np.random.rand(L, 1) - 3, axis=0)\n",
    "epsilon = np.random.randn(L , 1)\n",
    "f_0 = 0.8\n",
    "psi = np.pi / 3\n",
    "A = 10\n",
    "y = A * np.cos(2 * np.pi * f_0 * x + psi) + epsilon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'y = 10cos(2*pi*0.8*x + pi/3 ) + epsilon')"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfYAAAFNCAYAAADy5k0KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dfZxdVX3v8e9vhok5EGRiwUAGNFyrUTGSvEiV21Q7oZagBU3T+lR8aL0tVaut1aYN0hZaHzJKW3pb9V5ptWIVowIdUwJFJYy03IuSEBAjpFILkhPkQTIpgYFMZn79Y++TnJzZ58zMmb3PXnufz/v1mldm9nlaZ2Wd/Tvrtx62ubsAAEA59ORdAAAAkB4COwAAJUJgBwCgRAjsAACUCIEdAIASIbADAFAiBHYgQ2Z2tpkNZ/waO81sMMvXKAIze4WZ7UrpuV5rZpvSeK5OMLP/a2Z/Ev8+aGa78y4T8kNgR6mY2YfM7C4zO2hmlyTc/mtmdr+ZPWFmw2b2rIyL9FFJQ/FrP9vMvmRme8xsn5ndYmYvTyjj52bzAu5+mruPtHoOi3zMzH4S/3zczKzZc5rZG8zsbjN73My+b2ZrG25fklS/eXL3f3X3pfXHzGxxLciZ2RfM7EEz+y8z+3cz+80Wz7VZ0kvM7KUZFzsV7v5Od/9Q3uVAGAjsKJt7Jf2hpC2NN5jZaZI+LemtkhZJelLSp7IqiJn9jKTj3P3W+NACSbdJOkPSsyRdIWmLmS0ws6PN7NNm9lPxY58b98KaBt+E13uOmV1mZkfHfy8zs0vjmy+QtFbS6ZJeKulcSb/d5HkGJH1B0vslPVPSeklXxl9MzjSziyQdFd/3lWb2wRlXSvLr/fpsv8zMwmsk/Uv8+0ZJS9z9mZJeK+nDZnZGi8d+SVG9TcvMLgntiw66F4EdHWFm683s6oZjf2tmf53m67j7Fe5+vaTHE24+X9I/u/vN7r5f0p9IWmdmx8bleZaZ/UPco95bn0I3s98ys3vN7DEz22xmi+PjFgfTh+Ne+HfN7CXxw14t6Vt1Zfuhu/+Vuz/o7hPufrmkeZKWuvuTkj6h6IvGKyT9uaSPubvHQeMqM/ty3IO+3cxOryvbfWb2Knf/kaSrJf2jpNWS3iXp4/Hd3i7pL919t7tXJf2lpF9vUo0nSxp19+s9skXSE5KeF39J+Z6k/yPpTfF7/Ju47nab2XlxmRbE9fW2lv9hsxS/1wvjLMLe+P9rfnxbUgr6NZKukyR33+nuT8fHPf55XouXG5H0S2mWPy7nC83sG3Fb2mVmb6i77XPxF7pvxP/X3zKz58a3NW1r8eM+3OT1XmRmI2Y2atGwzWsbXu+TZrYlfr1vm1mrOkEBENjRKV+QdI6Z9UuSmR0l6Y2KgtAUZnZtfCJK+rm2zTKcJunO2h/u/h+SDkh6QXzoHyUdHd/v2ZIui8tylqLe3hsknSTpfkm18dezJb0yfo7++D39JL5tmaSmY75mtlxRYL+3VqTaTZImJU3U3f11kr6qqKd/paRhM+ub5v1OxM8z5b3Hv5/W5HHbJN1t0Thzr0Vp+KclfbehnLXXcHd/TNI7JP2dmdXq7g53//w0ZWzH+ZLWKArKL5D0x0l3iuvnlZK+UXfsU2b2pKR7JD2oOOg3cbekJWb2zJTKLTM7Ji7PlYra2JslfSrOJtWcL+lDko6XdIekL8bHW7W1Zq/XJ+mfJX09fr33SvqimdUPWbxZ0p9JWqioLX6k/XeIEBDY0RHu/qCkmyW9Pj50jqRH3X17k/uf6+79TX7ObbMYCyTtazi2T9KxZnaSot7nO919r7uPu3utt32+pM+6++1xj+9CSf/TzJZIGpd0rKQXSjJ3vzt+r1J08k3KHCgOFv8o6c/cfV+cPv9dSe9WVE+XSPpgXSp+u7tf5e7jkv5K0nxJZzY853Mk/YqioYabJF0uaUOT975P0oK65z/E3SckfV5R8Hk6/ve33f0JMztTUSr/XYq+3Hxd0u/Fj/u6oi8fNyrq6Sam+lPwCXd/IP4y8RFFgSnJKyXd6e6H/g/c/d2K/r9eIekaRe+vmdrj+ude5EPOlXSfu/+Dux9099sVZVl+te4+W+Ks0tOSLlLU1k5R67bWzJmK/u+H3P2Au2+VdK2OrLNr3P077n5Q0ZeI5Wm8UeSHwI5OukLSW+Lf36ImvfUM7Vc0ZlzvmYpO4KdIeszd9yY8brGiXrokKU7j/0TSQHyi/ISkT0p6yMwur+vh7VV0Ij6CmVUU9aJudfeN8XM+6e4XuPtP4r/vjydE1XrHD9S9/qSk3XG5VHf8R+7++3FaX+5+l7uvb/Lenylpf93z15fvVYpS+IOKMgo/L+nvzWy5u9/q7h+WdDB+jZvd/aN1D79c0ksk/UPtvSSJe86jZjaqaPjh1+oyMt9t9rjGulD0/7K4yf0OpeHrxcMg/6ZoyOFdLV6n9n832uQ9XFv3HjZI2jCDrNJzJb28PgOl6IvjiXX3qf+/3i/pMUmLp2lrzSyW9EDcZmrulzRQ9/eP635/UtEXARQYgR2dNCzppfG44Lk6nGKcwsyuN7P9TX6ub/P1dyqaPFZ7jf8h6RmS/l3RyfRZtaGCBnsUnZBrjztG0k9JqkqSu/+Nu5+hKLX9AkWTzaQodf2C+icys2coqoeqmvRo3f3XEw6fUvccPYqC0p5mbzThOY547/HvO5s8fLmkm919m7tPuvttkr4t6VV1z3+fu19S/yAz61U0OfHzkt5lZj/donzvrmVgFGUprqzLyEw3E/2Uut+fo+b18BolTKKsc5Raj7G/SFHv+r+SbqzPKila+TA0g6zSA5K+1ZCBWuDu9V8w6v+vFygaftkTv2azttbMHkmnxG2m5jmK2y7KicCOjnH3pyRdpSi1+514slez+746PuEl/by62ePMrC+eTNUj6Sgzmx8HHCn6InGeReudj1E0Qe0ad388Tmler2i8c2H8PK+MH3elpN8ws+VxYP6opG+7+31m9jNm9vJ4LPMJSU/p8Nj4dYp6u4fKFr//MUlva+hFTecMM1sXz014n6IU8q3TPKbe5yW938wGLJr49wFJn2ty39skvSKeAyAzW6EodT1dT7o2O/4dkv5C0ufr6j5Nv2NmJ1u0VPGDkr7ceAczO1XSM9z9nvjvZ5vZmyya1NdrZmsUpaO3tnidn1fUJtJ0raQXmNlb4zbWF7ehF9Xd5zVm9nNmNk/RWPu33f2BadpaM9+O7/uH8WsNSjpPh+eIoIQI7Oi0KxRNKssqDf93igLnmxWNT44pGnOWu++U9E5FAf5hRanWd9c99q2KxjHviW9/X/y4GxXNoL9a0YSr5ymaES5FKe2/U5R2v19Riv4v4sfdLmmfHV6r/rOKMhVnSxqty0C8Ygbv62uKJkvtjcu5Lh5vn6lPK0r/36VoVvuW+JikQ5vcnB+X+1uKxvivMrPH4/f90XgMPZFFy8ber+gLy4SkjymaZLeh2WPm4EpFY/s/jH+SZoP/ko5Mw7uitPtuRXX4F5Le5+5fa/E6b1ZdHaUhHu8/W1H72aMoDf4xRZmjmislXawoBX+GolS91KKttXi9A4qW9r1a0qOKhj3eVvvCg3KyhCE2IDPxBK97JJ3YLMVZJmZ2tqR3u/vaae/c/DkukfTT7v6W6e5bdmZ2n6TfdPdvTnO/6xRNsms1673V48+T9FZ3f8O0d06RRev5d7t74kx/YCaOyrsA6B7xON/7JW3qhqAuHZop3rSni8yMKFoZ0BZ3/2dFGQ6gcAjs6Ih4TPshRSnEc3IuDkrO3T8+/b2AciIVDwBAiTB5DgCAEiGwAwBQIqUYYz/++ON9yZIlTW9/4okndMwxx3SuQAVBvSSjXpJRL8mol2TUS7I062X79u2PuvsJjcdLEdiXLFmibdu2Nb19ZGREg4ODnStQQVAvyaiXZNRLMuolGfWSLM16MbP7k46TigcAoEQI7AAAlAiBHQCAEiGwAwBQIgR2AABKhMAOAECJENgBACiRUqxjBwB0n+EdVV16wy7tGR3T4v6K1q9ZqrUrBvIuVu4I7ACAwhneUdWF19ylsfEJSVJ1dEwXXnOXJHV9cCcVDwAonEtv2HUoqNeMjU/o0ht25VSicNBjR8eQNgOQlj2jY7M63k3osaMjammz6uiYXIfTZsM7qnkXDUABLe6vzOp4NyGwoyOapc0u2bxTq4a26tQNW7RqaCuBHsCMrF+zVJW+3iOOVfp6tX7N0pxKFA5S8eiIZumx0bFxjY6NS2LyC4AjtRq+q/3L8N5UuQZ2M/uspHMlPezuL4mPPUvSlyUtkXSfpDe4+968yoh0LO6vqDqDsa/a5Bc+nEB3m8ms97UrBjhXJMg7Ff85Sec0HNsg6UZ3f76kG+O/UXBJabNmmPwCgFnv7cs1sLv7zZIeazj8OklXxL9fIWltRwuFTKxdMaCN65ZpoL8ikzTQX9HCo/sS78vkFwDMem9fiGPsi9z9QUly9wfN7Nl5FwjpaEybNabaJCa/AIg0G77ji//0zN3zLYDZEknX1o2xj7p7f93te919YcLjLpB0gSQtWrTojE2bNjV9jf3792vBggUpl7z4QqiX0bFxPbTvKR2YmNS83h4tOm6++ivJPflOCaFeQkS9JKNeks21XkbHxlXdO6bJuhjVY6aBhZXczxFzkWZ7Wb169XZ3X9l4PMQe+0NmdlLcWz9J0sNJd3L3yyVdLkkrV670wcHBpk84MjKiVrd3K+olGfWSjHpJRr0kS6NeyripVSfaS4iBfbOkt0saiv/9Wr7FQdrK+GEFkD5mvbcn7+VuX5I0KOl4M9st6WJFAf0rZva/JP1I0us7WSaCTra4cEPn0aaB7pJrYHf3Nze56Rc6WpAYQSd7rZawUMfpo00D3SfvdexBYd1k9ljC0lm0aZTF8I4q20/PUIhj7Lkh6GSPJSydRZvOF8Mg6SDzNDv02OtwtaDsceGGzqJN54crGqaHzNPsENjrEHSyl7QD3cZ1y/jWnZLGdOXqF55Am84JwSg9ZJ5mh1R8nXavFkS6bXZmuoSFep2dpHTl1dur+pUzBnTTPY9Qjx1GMEoPQ3izQ2BvMNt1k4z9ZIN6nb1mPcSb7nlEt2w4K6dSdS+CUXrWr1nK9tOzQCp+jki3ZYN6nT16iGFhaC8dtczd2PiEes0kHTmEx2z5qeixJ5hNCpiTaTao19mjhxiWdof2cFhj5m7C/dCXo1pQJ7M3FYG9wfCOqtZ/9U6NT0YXHqiOjmn9V++UlNxQOJlmg3qdPdKVnVP/5X/D8kmN7qgmnh/YEnVuptvQig2vkpGKb3DJ5p2HgnrN+KTrks07E+9Pui0b1OvsseKgMxqXsR2YmGQZW0amy9yR2UtGj73B6Nj4rI6TbssG9doeeojZo5fYOdNl7sjsJSOwp4CTaTaoV4SIXmLnTDe8xPBTMgJ7g4VH92nvk1N75wuP7suhNABCQy+xc6bL3JHZS0Zgb3Dxeadp/VV3anzi8Dh7X6/p4vNOy7FUAEJBL7GzmmXuGlcvXfbG5V0f0GsI7A34BgiglcZzxLzeHiYpzkE7O0yyzK01AnsCxnYBtFJ/jhgZGdEg54u2tBugmcDYGoEdwBRJvaj+vAuF0mk3QDOBsTXWsQM4QrPLjTZb8gm0q90AzeWIWyOwAzhCs17UQ/ueyqlEKKt2AzQbWLVGKh7AEZr1lg5MTHa4JPnj0sHZWv3CE/SFW3+UeLwVJjm3RmAHcIRm67Tn9XZXgo+Z19m76Z5HZnW8HpOcm+uuTyqAaTVLcy46bn5OJcoHlw7OHpPgskFgB3CEZheT6a901+6LBJ3sMQkuG6TiAUyRlOYcGflBTqXJB1vHZo9d/LJBjx0AEjDzOntcajgb9NgBIAEzrzuDSXDpI7ADQBMEHRQRqXgAAEqEwA4AQIkQ2AEAKBECOwAAJUJgBwCgRAjsAACUCIEdAIASYR17RrjcIwAgDwT2DHC5RwBAXkjFZ4DLPQIA8kJgzwCXewQA5IXAngGuMQwAyAuBPQNc7hEAkBcmz2Wg/nKP1dEx9ZodMcbOBDoAQFbosWdk7YqBQz33CXdJh2fHD++o5lw6AEBZ0WPPUKvZ8d3Qa2ctPwB0HoE9Q908O561/ADyMryjqks279To2LgkaeHRfbr4vNO65txDKj5D3Tw7nrX8APIwvKOq9V+981BQl6S9T45r/VV3ds0wKIE9Q908O76bsxUA8nPpDbs0PulTjo9PeNd0LIJNxZvZfZIelzQh6aC7r8y3RLNXPzu+28aZF/dXVE0I4t2QrQCQn1adh27pWAQb2GOr3f3RvAsxF2tXDHRFIG+0fs3SI8bYpe7JVgDIT7NORe22bkAqPgfDO6paNbRVp27YolVDW0s57rN2xYA2rlumgf6KTNJAf0Ub1y3ryi85ADpn/Zql6uuxKcf7eq1rOhbmPnUsIgRm9p+S9kpySZ9298sbbr9A0gWStGjRojM2bdrU9Ln279+vBQsWZFjamRsdG1d175gm6+q9x0wDCyvqr/R1tCwh1UtIqJdk1Esy6iVZnvUyOjauPaNjmojH2o/qMZ3U3/lzbJI062X16tXbk4apQw7si919j5k9W9I3JL3X3W9Ouu/KlSt927ZtTZ9rZGREg4OD2RR0llYNbU1MEw30V3TLhrM6WpaQ6iUk1Esy6iUZ9ZKMekmWZr2YWWJgDzYV7+574n8flvRPkl6Wb4nSwWxxAECWggzsZnaMmR1b+13S2ZK+l2+p0tHNa9sBANkLMrBLWiTp38zsTknfkbTF3f8l5zKlopvXtgMAshfkcjd3/6Gk0/MuRxa6eW07ACB7QQb2suvWte0AgOwR2JEKruQGAGEgsGPOuJIbAISDwJ6DsvVuu/2681krW3sBkC0Ce4eVsXfL2vzslLG9AMhWqMvdSquM1ylnbX52ythe0D264boYISKwd1gZe7eszc9OGdsLusPo2LguvOYuVUfH5DqcbSK4Z4/A3mFl7N1yJbfslLG9oDs8tO8psk05YYy9w8p6nXLW5mejrO0F5XdgYlJJfUeyTdkjsHcYO89hNmgvKKp5vckJYbJN2SOw56CsvVuWZWWjrO0F5bbouPmq9E2QbcoBY+xIRW1ZFhNlAEhSf6WPuTc5IbAjFSzLAoAwkIpHKliWBaDe6Ni4LryRzZXyQI8dqWBZFoqEjVOyx3K3/BDYkQo2qUFRMB+kM6LlblORxcsegR2pYJMaFAXzQTqD5W75YYwdqWFZFoqA+SCdwXK3/NBjB9BVmA/SGSx3yw89diBgbPqTPrbp7RyyePkgsAOB4lrs2WCbXpQdgR0IVKtJXgShuaEniTJjjB0IFJO8ALSDwA4EikleANpBYAcCxaY/ANrBGDsQKCZ5AWgHgR0IGJO8AMwWgR0AAsG+BUgDgR1A1wkxgLJvAdLC5DkAXSXUq7txcRqkhR47SinEHhnCEOrGP+xbgLTQY0fphNojQxhCDaDsW4C0ENhROqQ00UqoAZR9C5AWAjtKJ9QeGcIQagBdu2KAy5wiFYyxo3QW91dUTQjieffIEIaQN/5h3wKkgcCO0uF625gOARRlRmBH6YTcIwOArBHYUUr0yAB0KybPAQBQIvTYASAwbLCEuSCwA0BA2DMec0UqHgACUuQNloZ3VLVqaKtO3bBFu378OLs95oQeOwAEpKgbLDVmGg5MTJJpyAk9dgAISKhb3k6nyJmGsiGwA0BAQt3ydjpFzTSUUbCpeDM7R9L/ltQr6e/dfSjnIqFAGmcVr37hCbrpnkeYZYzgNdtgSZJWDW0Ntg2zlXM4ggzsZtYr6ZOSflHSbkm3mdlmd/9+viVDESTNKv7CrT86dHvos4xZ6oTGDZaKMFOerZzDEWoq/mWS7nX3H7r7AUmbJL0u5zKhIJLG+hqFOvbHteSRpAjj141Xp5vX28PV6XISZI9d0oCkB+r+3i3p5TmVBQUz0zG9EMf+Wp3AOUF2r6KMX9dnGkZGRjQYWJvtlmyYuXveZZjCzF4vaY27/2b891slvczd31t3nwskXSBJixYtOmPTpk1Nn2///v1asGBBtoWeg9GxcT207ykdmJjUvN4eLTpuvvorfZm/buj10q5dP35cByYmp73fvN4eLT3x2CnH86yXu6r7mt62bOC4DpZkqrK2l7nqRL00a9PN2nAIQmsvo2Pjqu4d02RdzOsx08DCSkfOtzVp1svq1au3u/vKxuOh9th3Szql7u+TJe2pv4O7Xy7pcklauXKlDw4ONn2ykZERtbo9T8M7qrrwxrs0Nt6j2shIpW9CG9e9OPNvkrOtl6J82x1tGI9MYpIue+PyxB5Fnu3loqGtiROQBvoreu/5g50vUJ2QP0d56kS9JLXpSl+vNq5bFlyvuCa09rJqaKuqo71Tjg/09+qWDYMdK0cn6iXUMfbbJD3fzE41s3mS3iRpc85lykQRxs6kYo391o/1SVEQr2eSzj/zOcF9KRneUdUTTx+ccpwJSGgcvx7orzB+PUtFGc5IQ5A9dnc/aGbvkXSDouVun3X3nTkXKxNFaWxFG/utH+srQqahcdZzzcKj+3TxeacFV150HpcinptuWo4XZGCXJHe/TtJ1eZcja0VpbEX5ApKkCCfEZjP5j553VPBlB4qgm5bjTZuKN7P3mNnCThSmGxVll6mibnNZFEX+4gQUQTcNZ8ykx36iog1ibpf0WUk3eIhT6Quq2S5ToTW2pG+7pmisfdXQ1iDLXCRFydwARVaE7F0apg3s7v7HZvYnks6W9BuSPmFmX5H0GXf/j6wL2A2K0Njqv4BUR8dkkmrf7kLcBatouilNCCBbM5oVH/fQfxz/HJS0UNJVZvbxDMuGwKxdMaBbNpylgf6KGlM2Ic7kL5JuShMCyNa0PXYz+11Jb5f0qKS/l7Te3cfNrEfSDyT9YbZFRGgYD85GETI3AMI3kzH24yWtc/f76w+6+6SZnZtNsRAyxoMBIFzTpuLd/U8bg3rdbXenXySErigz+QGgGwW7jh3hKspMfgDoRgR2tIXxYAAIE4E9QEXYAhUAECYCe2Aa9wxnjTgAYDZCvbpb1yrK1d4AAGEisAeGNeIAgLkgsAeGi60AAOaCwB4Y1ogDAOaCyXOBYY04iib0VRyhlw9IG4E9QKwRR1GEvooj9PIBWSAVD6Btoa/iCL18QBYI7ADaFvoqjtDLB2SBwA6gbaGv4gi9fEAWCOwA2hb6Ko7QywdkgclzANoW+iqOWjku2bxTo2PjkqT5ffRnUG4EdgBzUoRVHE8fnDz0+94nx5kZj1IjsAOBYL11NlrNjKd+UUYEdiAArLfODjPj0W0YbAICwHrr7DAzHt2GwA4EgF5ldpgZj25DYAcCQK8yO2tXDGjjumUa6K/IJA30V7Rx3TKGOFBajLEDAVi/ZukRY+wSvco0FWHmPpAWAjsQgNDXgwMoDgI7EAh6lQDSQGAPFGuaAQDtILAHKGlN8+9/+Q5tu/8xfXjtspxLB4SPL8boZsyKD1DSmmaX9MVbf6ThHdV8CgUURO2LcXV0TK7Dm/3w2UG3ILAHqNnaZZfYsASYBpv9oNsR2APUau0yG5YArbHZD7odgT1A69cslTW5jQ1LgNbY7AfdjsAeqKRrRrNhCTA9tpBFO4Z3VLVqaKtO3bBFq4a2FnpOBrPiA3N4RvzkEcd77MhxQmb4AsnY7AezVbarKxLYA5M08UeSJj36t+gNDugENvvBbLSacFnEdkQqPjAzmeDDDF8ASE/ZJlzSYw/M4v6KqjNoTEVtcCGaspnJ6VMzJgDKq9l5t6gTLumxByZp4k+Soja40CRtZlLdO1boiTMAZqdsEy7psQemceLPcZU+PXHgoMYn/NB9itzgQpM0tjbpnunYGtudAmEp24RLAnuAGif+EAiy0+mxtbLNvgXKokwTLoML7GZ2iaTfkvRIfOiD7n5dfiXKX5kaXGg6PbZWttm3AMIT6hj7Ze6+PP7p6qCObCWNrfWYZTbUUbbZtwDCE2pgBzpi7YoBbVy3TAP9FZmkgf6KBhZWMus9s90pgKwFl4qPvcfM3iZpm6QPuPvevAuE8moc6hgZGcnstdavWXrEGLvEZEgA6TJ3n/5eab+o2TclnZhw00WSbpX0qKKrlH5I0knu/o6E57hA0gWStGjRojM2bdrU9PX279+vBQsWpFDycqFekmVdL6Nj43po31M6MDGpeb09WnTcfPVX+jJ7vbTQXpJRL8mol2Rp1svq1au3u/vKxuO5BPaZMrMlkq5195e0ut/KlSt927ZtTW8fGRnR4OBgqmUrA+olGfWSjHpJRr0ko16SpVkvZpYY2INLxZvZSe7+YPznL0v6Xp7lQTmxhBBAWQUX2CV93MyWK0rF3yfpt/MtDsqGteQAyiy4wO7ub827DCg31pIDKDOWu6HrsJYcQJkR2NF1WEsOoMwI7Og6ZbuSEwDUC26MHcha2a7kFDJWHwCdR2BHV+LCOtlj9QGQD1LxADLRavUBgOzQYweQCVYfoIyKMLxEjx2oM7yjql0/flynbtiiVUNbNbyjmneRCovVByib2vBSdXRMrsPDS6GdJwjsQKz2oT0wMRn0h7YoWH2AsinK8BKBHYgV5UNbFEnXut+4bllwaUtgOsM7qlo1tFXVggwvMcYOxBgTTh+rD1B0jas7koQ2vESPHYgxJgygUVImr16Iw0sEdiDGmDCARq0ydqEOL5GKB2K1D+dDu26XScEuZQHQOYv7K4lj6wP9Fd2y4axpHz9ledzpzXv/aSGwA3XWrhjQyL4f6D+HBvMuCoAArF+zdMoYu0la/cITpn1s0u6L1b0TGt5RzbTDQCoeAIAm1q4Y0K+cMSCrO+aSrt5enXYpbNL4/KR75ittCOwAALRw0z2PyBuOzWQpbF4rbQjsAAC00G6AzmulDYEdAIAW2g3QSStteswyX2lDYC+p2k5J7HkOAHPT7lLYpN0XBxZWMl9pw6z4EuI62ACQntp5s52rujXuvjgyMpJVMQ8hsJdQqz3PCewAMHtF2h6ZwF5C7HkOoF1FuN44WmOMvYTY8xxAO4pyvXG0Ro+9hJJ2SmLP83zQ+0ER1Npp0tapY+MT+sBX7pTEHJ2iILCX0FwmekAaHRvXqqGtc647JjGiCGZyWXhToUIAAAr7SURBVNIJd9pugRDYC6xVb7BIEz1CMryjqureMVVHo6UtcwnGTGJEEUx3WdIa2m5xMMZeUIyFZePSG3Zp0o/cPHImW0cmYRIjimA27ZG2WwwE9oJq1RtE+9IMxkxiRBHMpj3SdouBwF5Q9AazkWYwbne3KqCTktppX4+pr9eOOEbbLQ4Ce0HRG8zG+jVL1WPpnNCStpPcuG4ZY5QISlI7vfT1p+vSXz2dtltQTJ4rKJa0ZWPtigEN//j7GujvTWVFAZMYUQTN2iltt5josRcUvcHs9Ff6tH7NUi3ur2jP6JguvWEXkxIBFAY99gKjN5iN0bFxXXgj688BFBM9dqDBQ/ueYsUBgMKixw40ODAxqaTvvKw4SAfb7ALZIrADdVqNpbPiYO7YZhfIHql4oE6zdLtJrDhIARsrhWt4R1Wrhrbq1A1btGpoKxNGC4weO1Bnz+iYdMrU4y56lGlgY6UwkUkpFwI7UCdKtz8+5fgAafhULO6vJF4atHGYg3H47CTVLRcsSl+ebZhUPFAnzZ3nMNVMttnlAkfZaVa3SV+2JDIp7cq7DdNjB+qkvfMcjlSrx1Y9GXqP2WlWt71mmmi4qqHEhNGZauydP3ngYNM2/JEzs+9PE9iBBv2VPt2yYXDWjyN9PDPTbazEOHx2mtXhhLsqfb1sUT0Ltc97dXRMpmgejqSm2Q+pVv/HZF42UvFACvJOvZUJFzjKTrM6rG1JzRbVM1P/eZcOB/XpdKoN02MHUkD6OD1c4Cg7reqWLapnLunzPp1DbXjfDzIq1WG59NjN7PVmttPMJs1sZcNtF5rZvWa2y8zW5FE+YKZqa3+bpd9apeWQjAscZYe6TcdMhoX6K3251XNePfbvSVon6dP1B83sxZLeJOk0SYslfdPMXuDus/tqBHRA49rfJBbfjxPn7NB7zA51O3fNlm3WVPp6dclrT8utnnPpsbv73e6etNXU6yRtcven3f0/Jd0r6WWdLR0wMzNJx7ma72YHoJiSlm3WFsmGkAUJbYx9QNKtdX/vjo8BwZnpLG1mcyNErOJo30yWbebJPGHtYipPbPZNSScm3HSRu38tvs+IpD9w923x35+U9P/d/Qvx35+RdJ27X53w/BdIukCSFi1adMamTZualmX//v1asGDB3N5QSYyOjeuhfU/pwMSkTjpaOmr+0eqv9OVdrKBM117q63Am5vX2aOmJx6ZVvNzwOUpWxHoZHRtXde+YJuvO/z1mGlhYSe18UMR66YQ062X16tXb3X1l4/HMeuzu/qo2HrZbR+7UfbKkPU2e/3JJl0vSypUrfXBwsOmTjoyMqNXt3WJ4R1UX3niXxsZ7JPXoA8sO6lO3T2jjuhcH800zBK3aS2MdTqfS16uN65ZpsAT1y+coWRHrJZrw2Tvl+EB/b1t7OCQpYr10QifqJbR17JslvcnMnmFmp0p6vqTv5Fym0uDKWnPXalx9oL+it5z5HGYcI3hsAlRuuYyxm9kvS/pbSSdI2mJmd7j7GnffaWZfkfR9SQcl/Q4z4tPDh3numtWVSbplw1mdLUxJMNbbeTO9GA+KKa9Z8f/k7ie7+zPcfZG7r6m77SPu/jx3X+ru1+dRvrJiR6+5ow7TxY59+ZjJxXhQXKGl4pEhPsxzRx2mi+GhfLBRTbmFttwNGWpcojGvt4cP8yw11uFxlT6ZSb//5Tt06Q27SCPPEsND+WGjmmzlOcREYO8y9R/mkZGRUszW7rRaHTbuPFdLI9fug+kx1osyanVu6O/A65OKB9pEGnnuGNpAGeV9bqDH3gWYdZwN0shzF/oOXkA7Wp8bsr8eO4G95PJOCZUZaeR0MNaLssn73EAqvuTyTgmVGWlkAEnyPjcQ2EuOdHF6atdeP3XDFq0a2ipJLBkCMEXeywlJxZdc3imhsmg2pLFx3TJ2nAMwRZ5DTPTYSy7vlFBZMKQBoCjosZdU/Uz44yp9mt/Xo9Enx4+YdTwy8oO8i1kYDGkAKAoCewk1po1Hx8ZV6evVZW9czvhvmxjSAFAUpOJLiLRx+hjSAFAU9NhLiLRx+thIBUBR0GMvIS4tmo21KwZ0y4azdNkbl0uKLvyyamgrlxgFEBQCewmRNs4O1w8HEDoCewnlvTlCmTF/AUDoGGMvKfbfzgbzFwCEjh47MAvMXwAQOgI7MAvMXwAQOlLxwCyw7A1A6AjswCwxfwFAyEjFAwBQIgR2AABKhMAOAECJENgBACgRAjsAACVCYAcAoEQI7AAAlAiBHQCAEjF3z7sMc2Zmj0i6v8Vdjpf0aIeKUyTUSzLqJRn1kox6SUa9JEuzXp7r7ic0HixFYJ+OmW1z95V5lyM01Esy6iUZ9ZKMeklGvSTrRL2QigcAoEQI7AAAlEi3BPbL8y5AoKiXZNRLMuolGfWSjHpJlnm9dMUYOwAA3aJbeuwAAHSFrgnsZvYhM/uumd1hZl83s8V5lykEZnapmd0T180/mVl/3mUKgZm93sx2mtmkmXX1zF4zO8fMdpnZvWa2Ie/yhMLMPmtmD5vZ9/IuSyjM7BQzu8nM7o4/P7+Xd5lCYGbzzew7ZnZnXC9/lunrdUsq3sye6e7/Ff/+u5Je7O7vzLlYuTOzsyVtdfeDZvYxSXL3P8q5WLkzsxdJmpT0aUl/4O7bci5SLsysV9K/S/pFSbsl3Sbpze7+/VwLFgAze6Wk/ZI+7+4vybs8ITCzkySd5O63m9mxkrZLWtvt7cXMTNIx7r7fzPok/Zuk33P3W7N4va7psdeCeuwYSd3xjWYa7v51dz8Y/3mrpJPzLE8o3P1ud9+VdzkC8DJJ97r7D939gKRNkl6Xc5mC4O43S3os73KExN0fdPfb498fl3S3pIF8S5U/j+yP/+yLfzKLQV0T2CXJzD5iZg9IOl/Sn+ZdngC9Q9L1eRcCQRmQ9EDd37vFiRozYGZLJK2Q9O18SxIGM+s1szskPSzpG+6eWb2UKrCb2TfN7HsJP6+TJHe/yN1PkfRFSe/Jt7SdM129xPe5SNJBRXXTFWZSL5AlHCPbhZbMbIGkqyW9ryFb2rXcfcLdlyvKir7MzDIbvjkqqyfOg7u/aoZ3vVLSFkkXZ1icYExXL2b2dknnSvoF75ZJF5pVe+lmuyWdUvf3yZL25FQWFEA8hny1pC+6+zV5lyc07j5qZiOSzpGUycTLUvXYWzGz59f9+VpJ9+RVlpCY2TmS/kjSa939ybzLg+DcJun5Znaqmc2T9CZJm3MuEwIVTxL7jKS73f2v8i5PKMzshNqKIzOrSHqVMoxB3TQr/mpJSxXNdL5f0jvdvZpvqfJnZvdKeoakn8SHbmW1gGRmvyzpbyWdIGlU0h3uvibfUuXDzF4j6a8l9Ur6rLt/JOciBcHMviRpUNHVuh6SdLG7fybXQuXMzH5O0r9KukvRuVaSPuju1+VXqvyZ2UslXaHoM9Qj6Svu/ueZvV63BHYAALpB16TiAQDoBgR2AABKhMAOAECJENgBACgRAjsAACVCYAcAoEQI7AAAlAiBHcCsmdnPmNl34+tMHxNfY5pLlwIBYIMaAG0xsw9Lmi+pImm3u2/MuUgARGAH0KZ47/jbJD0l6WfdfSLnIgEQqXgA7XuWpAWSjlXUcwcQAHrsANpiZpslbZJ0qqST3P09ORcJgEp2PXYAnWFmb5N00N2vNLNeSf/PzM5y9615lw3odvTYAQAoEcbYAQAoEQI7AAAlQmAHAKBECOwAAJQIgR0AgBIhsAMAUCIEdgAASoTADgBAifw3nvssMCjLFtgAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# let's see it\n",
    "fig = plt.figure(figsize=(8,5))\n",
    "ax = fig.add_subplot(111)\n",
    "ax.scatter(x, y)\n",
    "ax.grid()\n",
    "ax.set_xlabel(\"x\")\n",
    "ax.set_ylabel(\"y\")\n",
    "ax.set_title(\"y = 10cos(2*pi*0.8*x + pi/3 ) + epsilon\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "def plot_cos_lls_sol(x, y, y_pred):\n",
    "    fig = plt.figure(figsize=(8,5))\n",
    "    ax = fig.add_subplot(111)\n",
    "    ax.scatter(x, y, label=\"original\")\n",
    "    ax.plot(x, y_pred, label=\"least-squares solution\", color='r')\n",
    "    ax.grid()\n",
    "    ax.legend()\n",
    "    ax.set_xlabel(\"x\")\n",
    "    ax.set_ylabel(\"y\")\n",
    "    ax.set_title(\"y = 10cos(2*pi*0.8*x + pi/3 ) + epsilon\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(100, 2)\n",
      "theta:\n",
      "[[4.91360833]\n",
      " [8.62629552]]\n"
     ]
    }
   ],
   "source": [
    "phi_x = np.concatenate([np.cos(2 * np.pi * f_0 * x), -np.sin(2 * np.pi * f_0 * x)], axis=1)\n",
    "print(phi_x.shape)\n",
    "theta_ls = np.linalg.inv(phi_x.T @ phi_x) @ phi_x.T @ y\n",
    "print(\"theta:\")\n",
    "print(theta_ls)\n",
    "A_pred = np.sqrt(np.sum(np.square(theta_ls)))\n",
    "psi_pred = np.arctan(theta_ls[1] / theta_ls[0])\n",
    "y_pred = A_pred * np.cos(2 * np.pi * f_0 * x + psi_pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfYAAAFNCAYAAADy5k0KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOydeXxU1dn4vycbScgKgZCEQBLCagIJIIIgAi64i9RqrVbporbWtrZvUWxrtdWqrba1/fnaVtuivmJtq5KqqFCXVKWKgigQIEAWkgxrQhKSMNnP748zN0ySmawzc+/MnO/nk88k996595mTO/c5z3OeRUgp0Wg0Go1GExiEmC2ARqPRaDQaz6EVu0aj0Wg0AYRW7BqNRqPRBBBasWs0Go1GE0Boxa7RaDQaTQChFbtGo9FoNAGEVuwajRcRQlwohCjw8jWKhBBLvHkNf0AIcY4QothD57pCCPGCJ87lC4QQfxRC3OP4fYkQospsmTTmoRW7JqAQQtwvhNgphGgXQtznYv+XhRAHhRBNQogCIcQoL4v0IPCw49pjhRB/E0IcEkLUCyE2CyHOciHj04O5gJTyDCllYV/nEIpfCiFqHD+/EkIId+cUQlwjhNgjhGgQQuwWQqzosT/D1fiaiZTyfSnlVOdtQohUQ8kJIZ4TQhwWQpwUQuwTQnyjj3O9AuQIIWZ6WWyPIKX8ppTyfrPl0FgDrdg1gcYB4E5gQ88dQogzgD8BXwGSgVPAE94SRAhxJhAvpfzIsSkG+ASYA4wCngE2CCFihBDRQog/CSFGO9470WGFuVW+Lq43QQjxWyFEtOPvXCHEI47dtwArgFnATOAy4FY350kDngN+AMQBq4HnHROT+UKIHwNhjmMXCyF+NOBBcX29VYOdzAyCS4A3Hb8/BGRIKeOAK4AHhBBz+njv31Dj1i9CiPusNtHRBC9asWt8ghBitRDipR7b/p8Q4jFPXkdK+YyU8g2gwcXu64FXpZTvSSkbgXuAlUKIWIc8o4QQax0Wda2zC10IcbMQ4oAQ4oQQ4hUhRKpju3Ao02MOK3yHECLH8baLgf84yVYqpfyNlPKwlLJDSvkkEAFMlVKeAh5HTTTOAX4O/FJKKR1K40UhxN8dFvSnQohZTrKVCyHOl1JWAC8B/wcsBb4F/Mpx2E3Ar6WUVVJKG/BrYJWbYRwP1Ekp35CKDUATMMkxSdkF/AH4kuMz/t4xdlVCiMsdMsU4xuvGPv9hg8TxWe92eBFqHf+vSMc+Vy7oS4DXAaSURVLKFsd26fiZ1MflCoFLPSm/Q85pQoh/O+6lYiHENU77nnZM6P7t+F//Rwgx0bHP7b3meN8Dbq43XQhRKISoE2rZ5ooe1/tfIcQGx/W2CCH6GhONH6AVu8ZXPAdcJIRIABBChAHXopRQL4QQrzkeRK5+XhuiDGcAnxt/SClLgFZgimPT/wHRjuPGAr91yLIMZe1dA6QABwFj/fVCYLHjHAmOz1Tj2JcLuF3zFULkoRT7AUMkYxfQCXQ4HX4l8E+Upf88UCCECO/n83Y4ztPrszt+P8PN+7YCe4RaZw4Vyg3fAuzoIadxDSmlPAF8DXhKCGGM3WdSymf7kXEoXA8sRynlKcBPXB3kGJ/FwL+dtj0hhDgF7AUO41D6btgDZAgh4jwkN0KIkQ55nkfdY9cBTzi8SQbXA/cDScBnwDrH9r7uNXfXCwdeBTY5rvcdYJ0QwnnJ4jrgZ0Ai6l78xdA/ocYKaMWu8QlSysPAe8AXHZsuAqqllNvcHH+ZlDLBzc9lQxQjBqjvsa0eiBVCpKCsz29KKWullG1SSsPavh74q5TyU4fFdzewQAiRAbQBscA0QEgp9zg+K6iHryvPAQ5l8X/Az6SU9Q73+XeB21DjdB/wIydX/DYp5YtSyjbgN0AkML/HOScAX0AtNbwLPAmscfPZ64EYp/N3IaXsAJ5FKZ8Wx+utUsomIcR8lCv/W6jJzSbge473bUJNPt5GWbouXf0e4HEpZaVjMvELlGJyxWLgcyll1/9ASnkb6v91DvAy6vO5w3hfwvBF7uIyoFxKuVZK2S6l/BTlZbna6ZgNDq9SC/Bj1L2WTt/3mjvmo/73D0spW6WU7wCv0X3MXpZSfiylbEdNIvI88UE15qEVu8aXPAPc4Pj9BtxY616kEbVm7Ewc6gGeDpyQUta6eF8qykoHwOHGrwHSHA/Kx4H/BY4KIZ50svBqUQ/ibggholBW1EdSyocc5zwlpbxFSlnj+PugIyDKsI4rna7fCVQ55MJpe4WU8vsOtz5Syp1SytVuPnsc0Oh0fmf5zke58JegPArnAn8WQuRJKT+SUj4AtDuu8Z6U8kGntz8J5ABrjc/iCoflXCeEqEMtP3zZySOzw937eo4F6v+S6ua4Lje8M45lkA9QSw7f6uM6xv+uzs1neM3pM6wB1gzAqzQROMvZA4WaOI5zOsb5f90InABS+7nX3JEKVDruGYODQJrT30ecfj+Fmgho/Bit2DW+pACY6VgXvIzTLsZeCCHeEEI0uvl5Y4jXL0IFjxnXyAJGAPtQD9NRxlJBDw6hHsjG+0YCowEbgJTy91LKOSjX9hRUsBko1/UU5xMJIUagxsGGG4tWSrnKxeZ0p3OEoJTSIXcf1MU5un12x+9Fbt6eB7wnpdwqpeyUUn4CbAHOdzp/uZTyPuc3CSFCUcGJzwLfEkJk9yHfbYYHBuWleN7JI9NfJHq60+8TcD8Ol+AiiNKJMPpeY5+Osq5Putrp7FVCZT48PACvUiXwnx4eqBgppfMEw/l/HYNafjnkuKa7e80dh4B0xz1jMAHHvasJTLRi1/gMKWUz8CLKtfuxI9jL3bEXOx54rn4udvc+IUS4I5gqBAgTQkQ6FA6oicTlQuU7j0QFqL0spWxwuDTfQK13JjrOs9jxvueBrwoh8hyK+UFgi5SyXAhxphDiLMdaZhPQzOm18ddR1m6XbI7Pbwdu7GFF9cccIcRKR2zCHSgX8kf9vMeZZ4EfCCHShAr8+x/gaTfHfgKc44gBQAiRj3Jd92dJG9HxXwMeBZ51GntP8m0hxHihUhV/BPy95wFCiExghJRyr+PvsUKILwkV1BcqhFiOcke/08d1zkXdE57kNWCKEOIrjnss3HEPTXc65hIhxCIhRARqrX2LlLKyn3vNHVscx97puNYS4HJOx4hoAhCt2DW+5hlUUJm33PBPoRTndaj1STtqzRkpZRHwTZSCP4Zytd7m9N6voNYx9zr23+F439uoCPqXUAFXk1AR4aBc2k+h3O4HUS76Rx3v+xSoF6dz1c9GeSouBOqcPBDnDOBz/QsVLFXrkHOlY719oPwJ5f7fiYpq3+DYBnQVubneIfd/UGv8LwohGhyf+0HHGrpLhEob+wFqwtIB/BIVZLfG3XuGwfOotf1Sx4+raPBL6e6Glyi3exVqDB8F7pBS/quP61yH0xh5Asd6/4Wo++cQyg3+S5TnyOB54F6UC34OylUPfdxrfVyvFZXadzFQjVr2uNGY8GgCE+FiiU2j8RqOAK+9wDh3Ls5AQghxIXCblHJFvwe7P8d9QLaU8ob+jg10hBDlwDeklG/1c9zrqCC7vqLe+3r/5cBXpJTX9HuwBxEqn79KSuky0l+jGQhhZgugCR4c63w/AF4IBqUOXZHibi1djdcoRGUGDAkp5asoD4dG43doxa7xCY417aMoF+JFJoujCXCklL/q/yiNJjDRrniNRqPRaAIIHTyn0Wg0Gk0AoRW7RqPRaDQBRECssSclJcmMjAy3+5uamhg5cqTvBPIT9Li4Ro+La/S4uEaPi2v0uLjGk+Oybdu2ainlmJ7bA0KxZ2RksHXrVrf7CwsLWbJkie8E8hP0uLhGj4tr9Li4Ro+La/S4uMaT4yKEOOhqu3bFazQajUYTQGjFrtFoNBpNAKEVu0aj0Wg0AYRW7BqNRqPRBBBasWs0Go1GE0Boxa7RaDQaTQChFbtGo9FoNAFEQOSxazQajSb4KNhu45GNxRyqs5OaEMXq5VNZkZ9mtlimoxW7RqPRaPyOgu027n55J/a2DgBsdXbufnknQNArd+2K12g0Go3f8cjG4i6lbmBv6+CRjcUmSWQdtMWu8RnababRaDzFoTr7oLYHE9pi1/gEw21mq7MjOe02K9huM1s0jUbjh6QmRA1qezChFbvGJ7hzm933ShELH36HzDUbWPjwO1rRazSaAbF6+VSiwkO7bYsKD2X18qkmSWQdtCte4xPcucfq7G3U2dsAHfyi0Wi609fynfGql/d6Y6piF0L8FbgMOCalzHFsGwX8HcgAyoFrpJS1Zsmo8QypCVHYBrD2ZQS/6C+nRhPcDCTqfUV+mn5WuMBsV/zTwEU9tq0B3pZSTgbedvyt8XNcuc3coYNfNBqNjnofOqYqdinle8CJHpuvBJ5x/P4MsMKnQmm8wor8NB5amUtaQhQCSEuIIjE63OWxOvhFo9HoqPehY8U19mQp5WEAKeVhIcRYswXSeIaebrOerjbQwS8ajUbhbvlOT/z7R0gpzRVAiAzgNac19jopZYLT/lopZaKL990C3AKQnJw854UXXnB7jcbGRmJiYjwsuf9jhXGps7dxtL6Z1o5OIkJDSI6PJCHKtSXvK6wwLp4m9NQpOqKjh3WOQBwXT6DHxTXDHZc6exu2Wjtji/eQ+86bRNhP8fHV1xM5a7rpz4jh4Mn7ZenSpduklHN77ZBSmvqDCpLb5fR3MZDi+D0FKO7vHHPmzJF98e677/a5P1jR4+KagBuXbdukDA+X8rXXhnWagBsXD6HHxTWeGJd/v/iubBchsn7ESFkbHSc7QkOl/OlPhy+ciXjyfgG2Shc60Yqu+FeAm4CHHa//MlccjafpmcLyhxMfMLOlBi6+GM49F6K0q82j/OIX0NYGTz4Jl15qtjQazYA5f/MrEBZK3MESCAuDb38bfv5zuOwyOPNMs8WzLKYGzwkh/gZ8CEwVQlQJIb6OUugXCCH2Axc4/vYZBdttumCKF+lZgU5WHGTGoz+D3/9eKfbRo9WX9oknoKzMbHH9n717kevXUxcdR9trG7j0Jy/re9pTNDWBXQdyeY2WFnj2WbjySkhOVs+GJ5+EhAR46CGzpbM0ZkfFXyelTJFShkspx0sp/yKlrJFSnielnOx47Rk17zV02VPv0zOF5atbXwHgSz94Gt54A77xDdizR83Ms7IgJwfKy80RNgA4eNd9NIdG8K0r7iK8s4O5Wzbpe9pTrFwJmZkkbttmtiSByfr1UFMDN998eltcHNx+u9q3e7d5slkcs/PYLYXOm/Q+zqkqcc2NXPf5Rl6dvpgt4Ulw0UXKcj9wAIqL1ay8qAg2bTJRYj+mspK0117khVkX8uHEWRSNzWLlrnf0Pe0J6uvh7bfhxAlmrl4N99wD7e1mSxVY/PnPkJEB558PnPamzq6fjj08koq77jVXPgujFbsTOm/S+zinqnzp843EtNr587yruqewCAFTpsBdd6kZ+mefmSBpAPDrXyOBP595FQAv5yxj1pH9ZFdX6Ht6uLzzDnR0wCuvcGT5cnjgAVi2DKqquh2ml/aGSEmJmjh9/esQEtLNm3oiOp7nZy0ndcPLbNrwkdmSWhKt2J3Q3YK8j1GBLryjja9ufYUPJs6idPwU17nrQkBenlbsQ6G6Gp56irdmLcMWr0pBvDL9XNpFCCuL3tH39HDZtAliYuC88yi+6y61Fvzpp+p+ff11QC/tDYu//AVCQmDVKqC3N/WpM6+iU4TQ8IBPQ7D8Bq3YndDdgryPUYHumsOfkdJYw/ql1/LQylz39Z5nzYIdO6Cz07eC+imGhfi7K78Lp06x76u3dd3Tx2MSeS9zNlcVFbL6/GyTJfVzNm1SFnq4I5/6K1+BrVshLU1lHtx5J795vUgv7Q2FtjZYuxYuuQTGjwd6e02PxCXxcs4yLt36Bhw9aoaUlkYrdidclT3tU+k40O62wbEiP41fjKiE+Hh+/dRqt+NbsN3Gg4dGQFMT1975nB7XfjAsxLqjNdz06atsnDyfPx6L5Atz0rru6cKzLiKloZoV9fvNFtd/OXAASkvhwgu7b582DT76CG69FR55hMee+C5p9cd6vV0vg/TDhg1w5Ei3oDlXHqY/nfUFIttb4emnfSicf2DFPHZT6Sp7unatisz+9UiIj+/9ExcH8fHsaoRP950kLyyKlNhRbJUzdOvR/pASNm5UQTFhrm9BQ0llxagZ++gDe7j7ZeVS1uPqGsNd+Y3P3yShuZE/zP8i9rYO3t17nM1rlqmD7Mtgw++V6/i888wV2F8xgjmXL++9LyoK/vhHWLqUqTd9jdfXfofVl9zBpikLug7RyyD98NRTkJqqLHYHq5dP7VV++kjyBOqm5ZDw2msqHkfThVbs7njmGZWjetNNKgLW+DlwAE6eVL+fPEmOlOQ4vW3lDY/wadp03Xq0L3bvBpvN9YPRgaGkDiRNoC0klBnHSnm9bZEe1z4wLMHL97zPp6lT+Sx1arftgFI811wDf/ubqhUwcqQZovo3GzdCZiZMmuT+mGuv5b+RaaR862s8uf4XPLjkqzx51hf00l5/VFbCm2/C3Xd3TfqNglb2tg5ChaBDStIcvdcTWIF88EEu/mkBxa3huie7A+2Kd8Fr7++l/f0P+MNZV7Nw4hcpWP2Icg998AHs3AkHD0JdHbS3k3vHP1jwrbWs+MqvAZhfoax17W7rg40b1Wsfit0Yv5awCA6MTmf6sbJu2zW9SU2IIqSzg6nVB/k0dVq37d248UZVXGX9eh9LGAC0tamI+AsvVMGdfXDBlYsoXf8mn2Xkct3nbw54aS+oWbtWxdN87WtA9wBEgA4puyZHK/LT+E/2mYjOTqZ89l8doOiEVuw9KNhuY8Pv1hHW2cF/smZjq7Oz+p+fu75RQkKISx7N4bgxfJY6lQOjxjPHtgfQ7rY+2bgRpk+HCRPcHuI8fnvGZjLjaGmv7ZrurF4+lamNx4hsb2XP2CzATfDnwoXK4nz2WROk9HM++ggaG9kyeW5XXE3xkQa3iuSKs7LIu2klmXVH2Pzd+Vqp90VHh4qGP/98VZyK/muL/MQWRU1UHEtLPnG5P1jRir0H971SxKKSrTRERLEtbToAbZ2S+14pcnm8cyT91vEzmFu1m+gwod1t7rDb4b33+rTWofu47h6bSUpjDamtDXpc+2BFfho/y1APwb1jM9xbiCEhKor7rbfUkohm4GzcSGdoKLcfSexKY2vt6OzbSszJUXEle/b4VFS/49//hoqKbkFz/dUWqTrZSmHWHJaUbkPIzn7fFyxoxd6DOnsbO5Oz+evcK2kLDe+23RXOkfRbx88gvqWJ/5cXqWfm7njvPWhu7lexO4+rYX0+MkUHzvXHvJOVEBrKhj/cwuY1y9yP1w03KGWzbp1vBfR3Nm1i5/jpHA+N7La5TysxN1e97tzpZeH8nKeegqQkVRveQX+1RVITovggI5/E5gamHyvv933BglbsLngh7yJ+e84NAz5+RX4am9cs49HHbgPgvBMHvCWa//PmmzBiBCxe3O+hxriue/xWABY2aeuyX3bsgKlTITKy7+MmT4YFC1SQqGqRrOmPmhrYupW302e53O3WSpw0Sd3zu3Z5UTg/5+hReOUVFaw8YkTX5v5qi6xePpXtk/IAWFCxo9f+YEUr9h4kRocPans3srNh7FgVZKdxzcaNSqlHRw/8PUlJqvCHrkDXPzt2wMyZAzv2xhtVhsL27d6VKVB46y2Qkr25813udmslhoXBjBnaYu+LZ55Rtfa//vVum/urLbIiP407Vi2jcnQaZ1Xu0gGKDnS6Ww/uvfwMVr/4OW0dp62Y8FDBvZef0f+bhVCBSZs3e1FCP6ayUq0z9vjyDghdWrZ/6utVxsattw7s+Guuge99TwXRzZ7tXdkCgU2bICGBS752Be//a3e3oK5+rcScHFX7XOOaTZvUd3z69F67umqL9MBIgztUZ+dP46ey6Oje0/UaghxtsfdgRX4aj1w9q9sM8ZGrZw18BrhokapKdfiwV+X0SwxPxrIhfPny8tSkoLnZszIFEoZFOFCLfdQouPxyeP55lcalcY9TUaUVcyd0syIjQkP6txJzc+HQITjhsy7UfkPBdhsntmzjHx1JA67c2bMO/9bEiUQfOcSGd/VyB2iL3SXuZogDYuFC9bp5M1x9teeECgS2bVPrZzk5/R/bk7w8lQ6ze7e2Lt2xQ60xDlixg3LHv/SSUlqXXeYduQKBPXu6FVVyfkYUFhaypL/nhXHP79o1oPiSYKFgu41fPvcBKxrr2Jc0oSsPHfoOlO2ZBleUrIoFbVr3BpcuHcLzJcDQFrunyc9X1b20O74327YppRM+gHiFnsxyBCxpd7x7duyAhISuxhkD4qKLVAzD//1ft826/0EPjDKyF1wwtPfryHiXPLKxmAmHVfGp/UkTgYHlofcMVCxKVpkz40p0SiFoxe55IiJg3jwdQNeTzk7V1nLu3KG9f9IkVf5UK3b3GIFz/VRE60ZEBFx3HfzrX6qaIu7bjbpL+QwKNm5U2QYTJw7t/WlpqseEjozvxqE6O5OrKwDYlzSh2/a+6BmoWBcVhy12DHNqD3peSD9EK3ZvsHChijRuajJbEutg1NifM2do7w8JUVa7Vuyu6exU1uAs16lYfXLjjdDSAv/8J+C+2tfR+iCNb2huhv/8p9/aC30ihLLatcXejdSEKCbXVHAyIprDsUndtveFqzS4vSmTWFBf4RU5/Q2t2L3BokVqPfjjj82WxDps26Zeh6rYQa2zf/65zrt2RXk5NDYObn3dYM4cFY3sKDHrzlpq7eh0uT2QKdhu47u3Pw52Oz+sTx7ekkROjrLY9f3bxdJpY5hSXcGBpPRunqal08b0+T5XaXBpSxYQe7BEG1Roxe4dFixQN6l2x5/GCJw7YwBpg+6YNUtZ/eXlHhMrYBhK4JyBEMpq/+ADKClxay1FhAbX48JYkpix80NaQ8LYMGrq8BqM5OaqlMSqKs8K6se8u/c4k6sr2Jc0sdf2/jAKWJU9fCmb1yxj2iXnqkmT8V0IYoLrm+orEhLU7FwH0J1m2zalmIcSOGeQpypMaXe8C3bsUAp6qBOn669X73/uObfVvpLj+6lmF2AYSxKLy7ezdfwM7BGRw2sw4hwZrwGg+dARkk7Vsz+pe0OoIdV6z89Xr7rgklbsXmPhQvjwQ+WSD3Y6O5ViH44bHtSDMSREK3ZX7NihKh8Otb96ejosXQrPPsuKvFSX1b4SooYxKfNDDtXZGdNYy4xjZbyfmd9t+5AwFLteZ+9iQfMRAPaPTu+2fUi13tPTVW0Grdh1HrvXWLQI/vhHNTsfSkBTIHHgADQ0DF+xR0eryGSt2Hvz+efDv89uvBFWrYL//pcVCxf2yiMuLNw/vPP7GakJUUzfvwWALek53bYPiVGjIDVVW+xO3DxKTZKcXfFDrvUuhLLatWLXFrvXmDdPvX76qblyWAEjcG6oqW7OGAF0mtM0NkJJydDW151ZuVJNnnSfdkBFXk+pPwTAAYdFOewGIzoyvhuzTtpoi4klNH28y1rwgyY/X41vkFdS1Ba7t8jKUoVq9Jf4dODcjBnDP9esWfC3v0FtLSQmDv98gUBRkQoaGq5ij41Vyv3vf4ff/a7/DnEBzor8NMrjTlE7MoGGyBjSEqJYvXzq8BqM5ORAYaFqeBKmH78UFRGem8Pmu8/zzPny86G1VVUKHO73wY/RFru3CA1Viky73WDr1uEHzhkYAXTaaj/NcCLie3LjjSpy+9VXh3+uACCjuorE/JyuyOthdw3LzVU1Aw7o1s6AKhE9nEyZnugAOkArdu+i3W6nK84Nd33dQEfG92bHDoiJgYyM4Z9r2TK1DtyjxGzQsm8fTJniufPpyPjTHDsG1dWeVexTpqjlJK3YNV4jJweOHFE3b7BiBM55Yn0dIDkZxo3TFrszO3aoSWSIB77OoaFwww3wxhvqwRvM1NfD0aMqYNNTzJihgryCfcIPagkJPKvYQ0OV5yrIY5u0YvcmenbumYpzPdGlZU9jFOTwZObFV76i1oBfeMFz5/RHih356p5U7FFRKi0xmJ8JBt5Q7KDc8Z99pryFQYpW7N7E6OgUzF/iXbvULHr6dM+dMy9PPRRaWz13Tn+lqko1b/FkoFBOjmqNG+zR8fv2qVdPuuJBL9EZFBWpYl4pKZ49b36+8hKWlnr2vH6EVuzeJCVFRW4H85d4715loUREeO6ceXkqnWWPbtHo0cA5Z77yFeVtMayqYKS4WC1vTJrk2fPm5KglKvsQC90ECkVFylofTDfCgaAD6LRi9ypGR6dgttj37PGstQ46Mt4ZQ7Hn5PR93GC57jrlaQnmILp9+yAz07OTUlDPBClVRHiwIuVpxe5pcnLUvasVu8ZrBHNHp7Y22L8fpk3z7HknT1ZrlXqdXSn2jAzV69uTJCfDRRfBunXBee+Cstg9ub5uoGNvVFDiiROeqW3Rk8hIdV6t2DWepmC7jYUPv8OPDwAnT7LxjU/MFsn3lJaqICxPW+yhocrq0YpdKXZvFeJYtkyt4Z844Z3zW5nOTmWxe0OxZ2ergk3BrNi9FThnEOSlZbVi9wJGu0dbnZ3iMaoG8vpn3xxeL2d/xFgD97TFDsod/9lnwWtNAjQ3K6vSW4o9O1u9BmMxFZtNrYF7OnAOVMW56dODO/bGWIbwpmI/ehQOH/bO+S2OVuxewGj3CKebG2QcKRt6u0d/Ze9e9eotxV5bG9y9rXfvVt0Dva3YS0q8c34r441UN2eCPfamqEgFFo8b553zB3kAnVbsXsC5rePJyBgOxSYx9Xj50Ns9+it79kBaGsTFef7cRt52MLvjvRURb5CVpQJAg9Fi91aqm0FOjvIK1NZ65/xWx1sR8QZGgK1W7BpP0bOt476kiUytPjj0do/+yp493rHWQVk8QmjFHhl52rL2NJGRamIWjIq9uFiV6U1N9c75g7nGhTcj4g3i49XEVCt2jadYvXwqUeGhXX/vHTOR7JpK7lyWZaJUPkZK5Yr3dOCcQWysUmjBrtiN1B5vkZ0dvIp9yhTvWZRGZHwwrrMfOaI8Fd5U7KA8WUFah0Erdi+wIj+Nh1bmkuaw0K49uTsAACAASURBVEvGTCSio50XXigMngC6Q4dU9SdvWewQ3L3ZpVSf3ZOlZF2RnR2ca+yebv7Sk/HjlVUZjBa7tyPiDaZPV5PSIOzNrhW7l1iRn9ZluZckKndedEUpd7+8MziUuxE45y2LHZRiLymBkye9dw2rcvSoai7k7Z7T2dmqGUwwjXFzM5SXey9wDpQnICcnOC12Xyn2adNUum0QTky1YvciRnR86SjVwznjxCHsbR3BER3vSHW74q0aMtdsYOHD73h+QmNYq0YQWTDh7cA5A6OcajA9HA8cUB4Rb1rsELzFq4qKYPRoGDvWu9cxvIWGkRFEaMXuRYwo+LqoOOoiY8isPdRteyBT+t5WGkZEs6MjCgnY6uye91YEc292Q7EbQVjeIhhz2Y2IeG9a7KD+d3V1Kjo+mCgqOt2+1ksUbLexcL0a118+VkD+zzcFh6fUgVbsXsQ5Cr48MZXMWluv7YHKiU93cGBUercvr8e9FampkJQUvIo9LU1ZPt4kGC12I4fdFxY7BNc6u1Ej34tu+ILtNlb/83NsMoIjMaPIrqmk9lQbq1/8PGiUu1bsXsQ5Or4sMZWME4eJCg9l9XIvWwIWIP3oQUpGj++13aPeCiGCN4DOm6VknYmNVXXjg81iT0lRn92bBGNk/OHDykvhRcX+yMZi2jrV8kbpqPFknlDKvK1DBscyKBZW7EKIciHETiHEZ0KIrWbLMxSco+PLE1NJazjOLy/JZkV+mtmieZf6epIbT3BgdHqvXR73VsyapR6M7e2ePa+VaWtTVo8vFDsoqz2YFLu3mr/0ZPRoNYEIJovdB4FzzsZDeWIqE2sPu9wXyFhWsTtYKqXMk1LONVuQobIiP43Na5bx/W9fBsAVsc0mS+QDHMEqFckTu232irciLw9aWk67T4OB4mKl3H2l2IMtl93IYfcFubnBZbH7QLF3XwJNYbT9JHHNjb32BTJWV+yBw+TJ6nXfvq7Ob16LFjcbR0T8VdedR1pCFAJIS4jioZW5nvdWBGMAnbH04EvFbrMR0tLim+uZSU2N6mbnC4sdlDveqPkfDBQVqbgYL0bEr14+lfAQFdtzMDEFgAl1RwgPFUGxDAoQZrYAfSCBTUIICfxJSvmk804hxC3ALQDJyckUFha6PVFjY2Of+31BaGMj5wBFr72O7fwwvpQuIR2gAduebRQc2U1CVLhPZfLWuGRt2sT4sDAiYlv5xfxQYKTaUb+fwsL9Hr2WaG/nnPBwql59ldI0z0warHC/9EXWa68xPjyc948cQVZXe/16Y1tbmQF0HjhA4YgRXr+emcTt2sVsYEdLCycGeA8M534ZFx7OtJYWtqxbh33ChCGdw6q4Gpf8Dz9EpqXxmRe/XwnAI+eEc6jOTkLCWFgP34utpGnRDBK88AwaLD55vkgpLfkDpDpexwKfA4vdHTtnzhzZF++++26f+33GmDHylbkXy4l3vdbr5+yH3va5OF4blyuukHLGDO+c2xWzZ0t5wQUeO51l7hd3XHSRlHl5vrveli1Sgtxx//2+u6ZZ/PWvUoKU+/YN+C3Dul8++URd78UXh34Oi9JrXDo7pYyLk/K223wnRFOTGl8L3buefL4AW6ULnWhZV7yU8pDj9RiwHphnrkQeYMoUxh6tdLkroII69uzxbsW5ngRbb3ZfRcQbOHLZow4d8t01zWLfPtUvPTPTN9cz8rmDYZ3dZlMVDL1dcc6Z6OigbGRkScUuhBgphIg1fgcuBPw/dHTyZCbVHXa5K2CCOlpaoLTU94r9+HGVShPoVFerOvy+VOyjRkFiIlHBUEiluFhlAYT5aJUyOlpdLxgi43fvVq++VOwQfMGfWFSxA8nAB0KIz4GPgQ1SyjdNlmn4TJ5MUkMNozu7ByEFVG77gQMqEMibzV96YgTQBUM+u2HZ+VKxA2RnB4/F7qvAOYNgiYw3IuJnzPDtdbOzYb+56+q+xpKKXUpZKqWc5fg5Q0r5C7Nl8giOyPhH80d6P1rcLIy0M18qdkPJBUNkvK9qxPdk0qTAt9g7OtTE1NeKPSdHXdceQMtxrigqgjFj1I8vmTw56BoZWTkqPvBw5MYuDa1n85prTBbGSxguLyO9zxfEx6s10WBR7GPHqmpwviQ7m8h//ANaWyEiwrfX9hUHD6qlJF/lsBvk5kJnp4pNmT3bt9f2JUVFvnfDw+l+ByUlkJ/v++ubgCUt9oDFuMEC0C1k5Oa/sO5tTsQkUFDS4FsBjAC6QMfXgXMG2dmIzk6l/AIVXzV/6Ukw1Iz3QY14twRhIyOt2H3JyJGqcYnxAAkQCrbbuPvlndjq7EysO0xpfIrv+87n5akJU1OT767pazo61MPfJMUOBPbD0VfNX3oyebLyggTyOntVle8j4g2MRkYBaFC5Qyt2XzN5MjWfFQVU5Tmj7zzAxNrDHExM8X3f+bw8ZRUE4MPR8Iac980/Q3Mz2+J71+D3OsGg2PftU8s63u4T3pOwMJVFEsgWuw9KybolJkbV5A/ke7cHWrH7mPJRqYgD+7HV2b3Xp9zHGDn4I9paSG2opjwhpdt2nzBrlnoNMHe8szdk6vFyAH5RGeb7+2XsWDoiIwP74Wg0f/Fin3C35OZqxe5NgizlTSt2H7OhOZZRp043JQAv9Cn3MUYO/oS6IwAcTEzttt0nTJgACQkBp9idvSHTjpXRLkIoik/z/f0iBPa0tMDuy+7L5i89yclR7uraWnOu7yUMb9M/nnmTmphECipMaoKlFbvGm3weqVI9Mmq75wT7c+U5o+98hqP4Tnliiu9z8wO0N7vzfTH9eBklo8fTEhZhyv1iD+QKXk1NSrH6OnDOIDdXvRqWbQBQZ2/r8jZNrq5k7+h087yTkyerAlaBHIPjhFbsPubUxCwAMk90v7n9ufKc0Xd+ZvNxAFozsszJzc/LU1HjAdQpy/m+mH6snL1jMntt9xX2tDRVVTCAxrcLI7DKTIsdAipG5Gh9s/I2SUl2TQX7kiaa550MhhgRJ7Ri9zHXXHsunQgynSz2QKg8tyI/je9MEDBqFG/+7EpzCu7MmgWnTgXUl9fwhsS2NDH+5DH2js0w7X6xp6aqPvCVrvsd+DVGRLxZFnt6OsTFBdQ6e2tHJwCpDceJbbWzP0l1rzPFO6kVu8abXDF/Es0pacxoPBZ4lecOHDj9BTKDAOzNbnhDFtnVMsfxjKmm3S92oy1uIK6zGymoviys5IwQymoPIIs9IlSplynHKwDYP1plc5jindSKXeNtos+YxoURJyl7+FI2r1kWGEodaNpdzKaWWPPS+GbMgPDwgFtnX5Gfxh9mqWpvj95/g2n3S5diD8SHY3Gxspqjo82TwYiMD5AuhcnxkUSFhzK5Win2fUkTzfNOxsaqao2BeO+6QCt2M5g8WVkIAfIFBnhlSymRh23sGTnWvDS+iAil3APIYu9i/36ldMaPN02ElqQkGDEiMB+OZjR/6UlOjoqKD5BmOwlR4Ty0Mpe8BhvHRiYSkzLWXO9kEDWD0YrdDCZPhvp61YIzQPjbP94nVHZSnpjStc2UQJlZswJTsZeUQFaWOTnWBiEhSoZAc8VLaW6qm4ERGR9A6+wA6dVVlIw2b0LaRRClvGnFbgbGAySAZo8jKsoBOJiQ0m27zwNl8vJUWsuxY769rrcpKTldGtNMAvHhePSoKndqBYsdAmad3Uh3S6m2cTAhxfxiXNnZYLOpANsARyt2MzACdAJIsee21gBQmTCu23afB8oEYm92KVWamZUUu58vIxmFUzLXbOC2e19QG81W7KNHq9KnAWKxH61vRjQ1MqapjgrHc8HUYlzGc7e01Jzr+xCt2M0gMxNCQwOqGcyVcS3Yw0ZwfGRC1zZTAmUCsbTs4cOqV3dWltmSKMVutyuZ/BTnMr0SiDuolhY2tcWZKxgEVGR8a0cn6fVHAboUO5hYjCuAu2v2RCt2MwgPh4kTA2rmmN14jLaJGaQlRpubxjdqlCovG0iK3VjTtoLFbsjgx+vszmV6ATJrD9ESGs79Oxr7eJePyM1V7U0DoAhQRGgI6XVKsVfGJ3dtN60Yl3HvBtpSkgvCzBYgaJk0ya8fjr0oLSVuxhQ2r1lmtiSBF0BnTACtoNid84HPOcdcWYZIT4sx60QV5YkpVJ1sNUkiJ3JyoLlZPRvMDuYbJsnxkUxq6G6xm1qMKyEBkpKCQrFri90ssrICx2I31oCt4CoGtc6+d69yGQcCpaUqGn7iRLMlUTKEhfn1w7GnxZh1wkbpqPHWKOs8Y4Z63bPHXDk8QEJUONeMbqNpRDR1UXHWKMY1ebJf37sDRSt2s8jKgpoaFY3r71RXQ2OjtRR7Z2dANNQo2G7jjVc/xBaTxMLffGB+e9+wMMjI8OuHo1GmFyCso50JdUeoSEq3RllnwytTVmauHB5iUsMxRk6bTNkvL7NGMa4gyWXXit0sMlUzj4D4AhueB+MzmU2AlJY1grySjtmoTEg2P13IwM+XkYwyvWkJUUyoP0p4ZwdnXniW+UoHVGR8bGzgePPKyqzzXACl2CsrA8eb5wat2M3CsG4DSbFbxWLPyFAPRz9X7EaQ14T6I9ZIFzIIgJS3FflpbF6zjHcuV3UXZl8432SJHAgROMt0Vluig9MxIoHw3O0DrdjNwrjZA+ELbHxJrDIzDwkJiAC6Q3V2RrS1kNx4whrpQgbZ2apyYk2NuXJ4AiPl1OwcdmeysgJC8UScOKECAa2o2P14KWkgaMVuFomJEB8fEF9gSkth3DhzG2j0xOjN3tlptiRDJjUhivH1qoKeJdKFDALp4VhcrNzfo0aZLclpMjPVd8qPPSIAkUbNeysp9kBaAu0DrdjNJFBcblZzt4FS7A0Nfv0FXr18KpMbHYrdCulCBgGQy96FFZq/9CQrS1m6R46YLcmwiDKKGFnFkwcq3S0mxq+fCwNBK3YzMWbm/o4VFfvMmep1xw5z5RgGK/LTuG2iavpSmTDOGulCoO5bIQLHYrdavniALNNFGoo9I8NUObohROA8d/tAF6gxk6wseP115S4O8dM5VmurijK1mmIPkLrQM1tPQHQ0n/z+enM7uzkTGal6l/u7Yj95UlnFVrTYQd27CxeaK8swiDp8GNLS1P1iJQKxQ2EP/FSbBAiZmf7vcquoUBMTqyn2hAT14+eKnbIy89u1uiIQurxZMHCuYLuNpf8ooxPBn5992/zUxmEQefiw9Z4LoJ67ZWV+H8PQF1qxm0kgpLxZLdXNmUCILi4ttdYapYGf57IXbLfxs0deBuCG92otoUCNugVlTR0cjRlFwpEqa9QtGCJRhw5Z97nQ1ATHj5stidfQit1MjAe2P1uVVlfs/jy2VswDNsjOVg/G+nqzJRk0hgJNqCqnE8EnoYmWUKDOzWkqEsaRXnfEGnULhkJzMxE1NdaclAZBZLxW7GYycaJysfrzDVZaCiNGqD7SFqJgu411x0JoOVDKogffMv2hPSSOH1eWhVUVO/il1d5V+KfuMIfixtASFmEJBepcn6AyYRwT6o702u43HDyIkNKa924gGFT9oBW7mURGQmqqf99gpaUq6tVCwX+GRbY7MokRHW202WyWsMgGjTHhs+LD0Y9z2Q1FmV5/lMqE5F7bzcK5PkFFwjhSGmsY0d5qft2CoWBlT5622DVex9/dxRZ0FRsWmVHUZYK/ujStVoPfGeN/7ocWu6EoJ9QdsVThH+fmNIZcWadqzK9bMBSsrNijoyE52b+fu/2gFbvZ+HOAl5TqwW6xL69heRlFXdLrj3bb7jdYWbHHxKhqg35osa9ePpUE2hnbVGuNPuEOnJvTGPfuPTNGmF+3YCiUldEREaHuESviz8/dAaDz2M0mMxNsNmhpUWvV/kRtrcoFtphiT02IwlZnxxY3lk4E6XVHu7b7FVYs1euMn6a8rchPI6ZUte6sik8mLSGK1cunWkKBrshPU3Icng7r7uRs4adtnUtLaU5JYaTV0jQNMjPhv/81WwqvoS12s8nKUpbvwYNmSzJ4LLoGbLg0W8PCORI7mvT6o5awyAaNkcNuVfxUsQOcH3UKgMfuXmmNPuE9GTdOxeD4q7vYodgtS1aWKqzV3m62JF5BK3az8ecITYuuozm7NKvik5nUdNwapVgHi1Vz2A0mTYJDh+DUKbMlGTxW60jYE6N9qz+6ix1pmnYrK/bMTOjoUMo9ANGK3Wz8uUiNhdeAjX7b85bNIb/thP8pdauW6nXGiIz3x0lpWZmyiK26Bgzqe+WHwYnU1EBDg7Utdn82qAaAVuxmM26cWlv3xxustBTGjIHYWLMlcY9zDIM/YdVSvc74ccpblzfEqmvA4L+lTx1GiqUtdn82qAaADp4zm5AQ/+02ZMFUt15kZqoHY0XF6cYw/oBF4xe64c/tW8vKfOJpamtro6qqiubm5kG971RrO+1XX0vcpZdy7KNPiYmOIDrCTx7XQsAbb9AxejR79uwxWxrXSAlvvAFxceBjGePj4wc9LpGRkYwfP57w8PABHe8nd0qAY8zM/Y3SUpg3z2wp+sZ5Zu5Pit3CyxxdJCbCqFH+abGXlfmkc1pVVRWxsbFkZGQgBugdqD3Viq3WzuiRjWTUHSZs9HhaIiIZlxhFYnSElyX2AI52rQ2TJxMbH2+yMH3Q0QEjR/p88tzQ0EDsILycUkpqamqoqqoic4DPA+2KtwL+WKSmvV1F8ltZ8YD/VpkqK4PwcFWZ0Mr4Y2R8ba2qce+De7e5uZnRo0cPWKkDHK1vplNKWsOU3RXR0UanlBytH5zVbwa1p1qpq2ukLSSU5g5J7alWs0Vyz4gRfrFEJ4Rg9OjRg/L6aMVuBbKy1IOmttZsSQZOVZWa8VrZVQxKMUZE+N/EqaxM9RIIDTVbkr7xR8Xu42WOwSh1gNaOTgDaQpXbNaKjrdt2q2J4GkLbWmkLDadTgq3Wbl3l7ieKHQZ/D2nFbgX8MULT6ulCBiEhSkH6o8Vu9bEFpdgrKlQUv79g8WWOiFD1WO4QIbSHhHYpdmO7VTE8DREd7bQ6JiWW9jRERCjPY0eH2ZJ4HGvfKcGCP0Zo+otiB/+MYbB6DrvBpEkqer+83GxJBo7F793k+EhCHBZaa2gYEe3thAhBcnzkkM4XExPjSfF4+umnOXToUK/trR2dCCmJ6GijJTSs23ZLYlT6HIDVXl5eTk5OTr/HPfjgg93+Pvvss4ck2nCxrGIXQlwkhCgWQhwQQqwxWx6v4q8We0gIpKebLYlLCrbbWPjwO2Su2UBBXQRNxQe6/l748DvW7vTW0KBygS2qeLrhjylvZWUq8M+igV2J0RGkJUYRERpCa2g4IzrbSEtU5ZD3Hj7Jjqo69h4+aZqL251ijwgNIaKjHQG0hkV02+4NpJR0dg5j0mAodg96m3oq9v+aVLbWkopdCBEK/C9wMTADuE4IMcNcqbxIXByMHu1/ij09XQV4WQyjbautzo4E9kQnMbKhjvqjNUjAVme3dBvXd17/CIDbP6y1/iTEXxW7xSdNidERTEuJIyEhhoiOdpASW629y/pt7egc0vr1I488wplnnsnMmTO59957u7avWLGCOXPmcMYZZ/Dkk08C0NHRwapVq8jJySE3N5ff/va3vPjii2zdupXrr7+evLw87PbTjZWS4yM5ebiKxbfcwoVXXsy5ixaw/eMPSY6PZO3atUyZMoVzzz2Xm2++mdtvvx2AVatW8eKLL3adw/AuNDY2ct555zF79mxyc3P517/+BSjLefr06dx2223Mnj2byspKNm3axIIFC5g9ezZf/OIXaWxsBGDNmjXMmDGDmTNn8sMf/rDXWPzno4/I+/KXyVu0iPz8fBoaGpBSsnr16q7P/Pe//73X+55++uku+QEuu+wyCgsLWbNmDXa7nby8PK6//vpun8f5vPPnz+86b2FhIUuWLOHqq69m2rRpXH/99UgP1C2warrbPOCAlLIUQAjxAnAlsNtUqbyJv+WyW9hVbLRtNaiMN7q8HWHPWLXsYbRxtVpFuoLtNv790gcsQ/XkNiYhgOVkBU4XKPKnXPayMsjN9f1177gDPvtscO9pa4PmZiIjIsmgdwBVa85MeOqJAZ1q06ZN7N+/n48//hgpJVdccQXvvfceixcv5q9//SujRo3Cbrdz5pln8oUvfIHy8nJsNhu7du0CoK6ujoSEBB5//HEeffRR5s6d2+38idERvPX6epbPn89Va+5nVBS0dkJzfQ333nsv27ZtIz4+nqVLl5Kfn9+nrJGRkaxfv564uDiqq6uZP38+V1xxBQDFxcWsXbuWJ554gurqah544AHeeustRo4cyS9/+Ut+85vfcPvtt7N+/Xr27t2LEIK6urpe13j0scf43zVrWHj++TSOGkVkZCQvv/wyn332GZ9//jnV1dWceeaZLF68eEDj+/DDD/P444/zmYv/sfN5y8vLWbp0add5t2/fTlFREampqSxcuJDNmzezaNGiAV3THZa02IE0wLmIb5VjW+Dib3WhLdygpGd7VqO3tdHlzd1xVuCRjcUk16g8YENuS/eSF0Kts/uLxd7Z6RcWexchjke0GyuuYxDW3aZNm9i0aRP5+fnMnj2bvXv3sn+/6nL3+9//nlmzZjF//nwqKyvZv38/WVlZlJaW8p3vfIc333yTuLi4fq+xMDeHta++yj/++v8o2beXCcmj2bJlC0uWLGHMmDFERERw7bXX9nseKSU/+tGPmDlzJueffz42m42jR9X3d+LEicyfPx+Ajz76iN27d7Nw4ULy8vJ45plnOHjwIHFxcURGRvKNb3yDl19+mWhHh8TaU61dyxlTcufw/d8+xu+feoq6ujrCwsL44IMPuO666wgNDSU5OZlzzz2XTz75ZMBj7A7n844dO7bbeefNm8f48eMJCQkhLy+Pcg/Eq1jVYncV29/tDhZC3ALcApCcnExhYaHbkzU2Nva532zq7G1MbO5gZlk5f3m+gLGJI0mI8r6Le6jjEtLSwuIjRygDDlpwXNfkdXYL2InMGA3PwleiD5Obe7qbU0RoiMvPb+b98qX0Bs4JO0xrZCRfmxcNwpC3wfR72N24zIiPJ2bnTj624L3Qk4jjxzm7tZV9bW0c8pC8fd0v8fHxNDQ0qD/uv3/Q5xatrcSUldE2Ziz2mN6KNURw+vx90NDQQEtLC9///vf52te+1m3f66+/zsaNG9m0aRPR0dFccsklnDhxokvRvf322/zud79j3bp1PPHEE3R0dNDU1ERDQwOffPIJd9xxBwA//vGP+UJuLu8+8wwFRUXcfPPNfO973yMuLo729vYuOZubm2ltbe1yfRvnklJ2bV+3bh2HDx+msLCQ8PBwcnJyqK6uBiAqKqrrXKdOnWLJkiWsXbu222ey2+28/fbbFBYW8s9//pPf/e53FLzyKm3tnSQ6lv/v/J87+PK8fDYVvstZZ53FK6+8QktLC83NzV3nb2trw26309jYSGdnJw0NDbS1tXU7pqmpiVOnTnX93fP/YYy98Z6Ojo6u84aFhREaGtr1no6ODhobG13+T5ubmwf8DLCqYq8CnKOyxgPdojWklE8CTwLMnTtXLlmyxO3JjHUMK1Kw3cbdb+9kRUsK+R3tPP1BLbWjI3lo5Qyvu14HOy4F2208srGYqANlvAXUZOVZclzrHGvsXe54mciXIqI4sPc4vx6vbnkB/PbaPJa4GGMz75cfP/wO00qPURI7jl/vOj25S0uI4jvXmyOTgdtxWbAA/vtflpxzjvXz7j/4AIApy5czxUP/477ulz179gyqylgvHB6GcAkVzYJOJws9RAjSEqOIHUA1utjYWC6//HLuuecevv71rxMTE4PNZiM8PJy2tjaSkpJITk5m7969fPLJJ0RHR9PS0sLIkSO54YYbyMnJYdWqVcTGxpKQkEBnZyexsbEsW7aMHTt2dF3n4JtvkpaVxXeWLaOpqYk9e/Zw1113sWbNGlpbW4mLi+PVV19l1qxZxMbGMnnyZHbv3s1NN91EQUEBbW1txMbG0tLSQmpqKqNGjeLdd9+loqKia706JCSka0yXLl3KD3/4Q44ePUp2djanTp2iqqqK1NRUOjs7ufrqq1m2bBnZ2dkcPQWtTpltleVlzJtyBnelj2dLeTmVlZWcf/75/OlPf+LWW2/lxIkTfPjhhzz22GM0Nzd3XXf69OmsXbuWkSNHYrPZ2LZtG9HR0cTGxhIeHk5kZGS30q+xsbHdzltRUdF13r179xIWFtb1eSIiIoiMjHR5z0RGRva7hGFgVcX+CTBZCJEJ2IAvAV82VyTvYKwHO7uLD8WNtdz6b4GTslxSfwSAX+9v49rtNkvJCafXoh/ZWIytzo4Qgqr4ZCbUKbkFcP38CZaTu2C7jaaWdtLrjlKZcLrrmOV7yWdnq7XgykrIyDBbmr6xeKpbL0JCICKCqM520hKjOFrfTGtHJxGhISTHRw6qxOyFF17Inj17WLBgAaACu5577jkuuugi/vjHPzJz5kymTp3a5ea22Wx89atf7Yo8f+ihhwAV8PbNb36TqKgoPvzwQ6KiVMQ+HR0UfvQRj9xxB+FRUURFRbFu3TpSUlK47777WLBgASkpKcyePZsOR+74zTffzJVXXsm8efM477zzGDlyJADXX389l19+OXPnziUvL49p06a5/Exjxozh6aef5rrrrqPFkbb2wAMPEBsby5VXXklzczNSSn7729/2Srt77i9/4H82/4cRSGbk53PxxRcTERHBhx9+yKxZsxBC8Ktf/Ypx48Z1c48vXLiQzMxMcnNzycnJYfbs2V37brnlFmbOnMns2bNZt25d1/arrrqq67xSyq7z7t27d8D/v0EhpbTkD3AJsA8oAX7c17Fz5syRffHuu+/2ud9MMu56TU686zV5zi1PSQnyhxd/T0686zWZcddrXr/2YMbl7IfelhMdsv7kgm9KCXLut5+VZz/0tvcE9BDrP62ShTMWyuLRE+TZD70t139a1efxZtwv2Vm5HwAAIABJREFU6z+tktN+8oaceOersjE8Uv5lzhVy4l2vybyfbexXXl/hdlzefVdKkPLf//alOEPjZz+TUggp7XaPnbKv+2X37t3Dv8DevVLu2TP883ibpiYpP/lEypoaKaWUJ0+edHnY2rVr5be//W1fSiallHLPoXr5eWVtt5/SAzYlc0ODz+RwNy794epeArZKFzrRqhY7UsrXgdfNlsPbpCZEYauzcyhuDB0ihPH1R7u2WwnnQLMJdUdoDovg+MhEhAUD0HqyIj8Nls+DP21n811LLdmq0/DcjLKfZGRbM5UJyoMTHRFmOc9CL5xT3s4/31xZ+qOsTJUZjhxasRdTGDFClZy2OkahFyM/3GIkx0diq7V3W85oD3O4zFtawMOFfMyk36h4IcTtQohEXwgTjKxePpWo8FDaQ8M4HDua9PqjlnS9Ok800uuPqqUDISw3AXFLZiacOgXHjpktiUuMiZOxXGCk6Fkxcr8XhqL0h8h4C6dpuiUiQi11DKcYiy8wmpT0M2latWoVjz/+uA8E6o5z4R9QwbNjRjvWsv2pJPIAGEi62zjgEyHEPxzV4Kxn7vgxK/LTeGhlLmkJUVQmjCO78RgPrcy1nJVmTEBAxQFUxicjUMVeLF9EBSzf5c2YIKUbit1hsfvFxCkkRKU++kMuuz+luhkMovSpqbS0qIJVFg6gNAr/zByfoAoAxUQqma0+toOkX8UupfwJMBn4C7AK2C+EeFAIMcnLsgUNK/LT2LxmGQvOm8vM1hOWU+rQfQKSXn+UqoTkrvxDq1dyAyyv2I2JU7pjKaYyPtmSnhu3+EOXt9ZW1ZXQovUX3OKF0qdeobnZsm74PvGjLm8DZUAFahyL9EccP+1AIvCiEOJXXpQt+MjKgsOHwW5N9+uK/DQ235pPXEsTFY4ofgNLF1GB09HaFq3uZ0ycptmrqY6OJ3HsKEt6btwyaZIaWw+Uw/QaFRVKPn+z2CMcke9WVz7Nzf4Vu2AQEWH9sR0k/QbPCSG+C9wEVAN/BlZLKduEECHAfuBO74oYRBgPnPJymD7dVFHc4lCMxhqwM5ZeDx45EsaOtazFDo4gvxg7zJzO5jXLzBZncEyapGIYjhyBlBSzpXGNv6W6GYSHq4BPKyuf9nb1468W+4kTKoYhxKrFWAfHQD5FErBSSrlcSvlPKWUbgJSyE7jMq9IFG4aL0KJWJdD1cKxKSO61y/Lrwf5QtrekRClJf8MfmsFYvA+7W4Twibv4kksucVlT3Zmf/vSnvPXWW713GLL1YbEXFhZy2WUWVBnGZKStzVw5PEi/FruU8qd97NvjWXGCHIuvAwNdsh1L6u4i9ov14EmTuiqPWZKWFlXkxR8VuyFzSQmcc465srijrExZv6mpZksyeEaM8Noau5H7/Prr/WcX//znP3e9Y4AR8ZbEOTjRHz0OLggMv0OgkJwMUVHWt9gTE/nxl+eTlhCFQJU79Yv14KwspTitGoRUXq7WgP1RsU+cqKKhrRwZX1amYi0sHLVdsN3GwoffIXPNhu7ZJobFPsQYht/85jfk5OSQk5PDY4895rL9aUZGRlc99vvvv59p06ZxwQUXcN111/Hoo48C3dusZmRkcO+996rWqosWsbe8HEaM4OOPP+bss89m0aJFnH322RQXWzj2BvwnhmEQWLZATVAihLLarW6xZ2ayIj/N+oq8J1lZah2touK069hKGErRHxV7eDhMmGB9xW5hN3xBjx4H3Vr2pkRAR4f6CRvcY3vbtm2sXbuWLVu2IKXkrLPO4txzz+3W/tSZrVu38tJLL7F9+3ba29uZPXs2c+bMcXnupKQkPv30U574+c959Pnn+bOjr/h7772H3W5ny5Yt/OhHP+Kll14awoj4iIgI68cwDBKt2K1GVhb1RcVc8vA7HKqzk5oQxerlU62jRMvKICfHbCmGhnMMgxUVu+Gp8UfFDtZPeSsrA6e63lbDqD7ojJFtsuJWR/OPlpZBK/YPPviAq666qqsO+8qVK3n//fe7tT/tefyVV17ZVQP+8ssvd3vulStXAjBnyhRefvNNAOrr67npppsoLi4mNDSUNquvXQsRcJHx2hVvMUpixhB6sBxb7SkkFssRb21VFtnkyWZLMjSc14GtSEkJREerJRl/ZNIk645tQwNUV1vaYneXVXKozj6sIjXSjfveUPQDPd4VI0aMACkJ7eig3VEZ75577mHp0qVs2bKFV199lWZj/d3KeDGGwQy0YrcYrzVEEdNqJ9F+smubZXLE9+1TKS25uWZLMjRSUtQX2KoxDCUlyqvgr8UdJ01SaUO1tWZL0htjecvCxWncZZWkJkSdXgcegvJZvHgxBQUFnDp1iqamJtavX885fQQ4Llq0qEshNzY2smHDhr4v0N6ulggcqWL19fWkpSkP49NPPz1oeU0hwIrUaMVuMYpGjALoqkBmYIkc8V271Ku/uuJDQpTFZmXF7q9ueDi9vGFFq934n1tYsTuXbTboyjYJC1M/Q1A+s2fPZtWqVcybN4+zzjqLb3zjGyQmum//ceaZZ3LFFVcwa9YsVq5cydy5c4mPj3d/AUMmh2K/8847ufvuu7ngggu62rNanoiI0xOUAECvsVsMe/pEQDUD2ZEypWu7JXLEd+5UDxc3vZH9gqwsayr2zk4l10UXmS3J0HFe6pg711xZeuIHit2Io3lkY7Hr+JphrAP/4Ac/4Ac/+EG3bbuMiboD557jP/zhD7nvvvs4deoUixcv5n/+53+A7hZ41/HV1cydMYNCR377ggUL2LdvHw0NDcTGxnL//fcDsGTJEpYsWTIk+b2O81JHdLS5sngArdgtxjVfXAy/626xWyZHfNcumDLltFvQH5k0Cd5/X6UNWcnlffiwygX2Z4vdUJpWtNjLyiA+HvqwVK1An9kmI0ao6n4+4JZbbmH37t00Nzdz0003MbuvoENjsuHPzwXnevxasWs8zeULp9CSMIppp6oRYK2o+F27rGeJDZasLBVIVVMDSUlmS3MaP051K9hu67IyP4kdTdOWHUw0WygnCrbbGPvvT4iLTOLWh9+xzvdpsIwYAXV1PpmUPv/88wM/2Gj+4s/lWP2lg94A0YrdgoyYks2VcS1c+fClZotymqYm5c5ctcpsSYaHc8qbVuzDpmfudWl8MuKzIrZvt1lCeRryvXrcxr6kCd1zw30kn5QSj3S7johQSr2tzVrWsb82f3EmNFT9WFSxDyZTAXTwnDWxYoDX7t3q1V8D5wwMxWm18S0pUQ+WiVaydfunZ+51RUIK408ctkYWB0q+5tY2xp88RmWCalzkyyyTyMhIampqBv1gdokVrUopA6MUq4Vz2aWU1NTUEDmIyZO22K1IVha89JKK0hxkMQqv4e8R8QZGHrPV1oFLSlTltvBwsyUZFD2zNQ4mjOPqxhpqjvfdTMRXHKqzM6axlsj21i7Fbmz3BePHj6eqqorjx48P/2RtbSoXX0qIiRn++TxBezscO6Zem5q67Wpubh6UMjKd6urTXeq8yFDGJTIykvHjxw/4eItoDU03MjPVzVVVdbqPuNns2qXq2Fs4qnhAREfDuHHWtNj9zA0PKgbE5qQkDyaqBitzO62h2FMTokipOgJAZXxyt+2+IDw8nExPFcVpaYG8PLjnHvjZzzxzzuFSWAgXXwybNsG8eT12FZKfn2+OXEPhz3+GP/xBTVC8GMPgi3HRrngrYihPK9WM37kTZsywdAONAWPFlDc/Vew9c68POqzi2ydaI+Ng9fKpZJ9UGSaGbJbJMhksI0bA+PHWei7s369e/bUapTOZmWC3w9Gj/R9rcbRityJWbN+6a5f/u+ENJk2ylmKvq1MV2/xQsa/IT+Ohlbldnf5aJ6p7d760hsW+Ij+NVeM66BAh2OLHAhAZ7sePvenTYccOs6U4zb59asKRnm62JMPHigbVENGueCuSnq4sY6son5oalWcdKIo9Kwuee846QT9+GhFv0C33Wkp4PMFSzWBGVh3kcNwY2kJV/ELtqTafR8Z7jPnz4YEHVMpmbKzZ0iiLfdKkwPDkGd+/AwdgwQJzZRkmfjx1DWCMFphWUexFReo1kBS7lHDwoNmSKByK/Z22WNe9uP0JISzXDObkrmLKnQLnwEL9FwbL/PmqSuHWrWZLoti3TxWtCgQyM9UExer94weAVuxWxUrrwIESEW/gnMtuBRxKcPW2Bmx1dut19RssFlPsyTWHqEhI6bXdEv0XBovRZvXDD82VA1RddX/u9tiTiAj1bNCKXeM1rKbY4+Mhzc/clu6wWi57SQknYhKoCem+LOC3VmV2NpSXez1taEA0NJB0qp6KHhY7WKT/wmBJTFS9Gqyg2CsqVAnWQLHYAaZO1Ypd40UmTYLjx+Hkyf6P9TY7d6pWrVaqrT4cxo1TlbKsYlWWllIW11vxgJ9alZMmKaVeUWG2JF3/48NJqd02+21kPCir/aOP1HKSmQRSRLzB1Knqczl6y/srWrFbFatYlVIGVkQ8qAmKlTwiJSVUj3XtDfFLq9K5y5vZOGRY8YXFXZH7aQlRPLQy1/8C5wwWLFDFVMwe33371GugWezNzdaYlA4DHRVvVZwfjnl55slx6JBKxwokxQ7WSXlraYHKSiYuX0lUeGi38qx+a1UafdkPHIALLjBXFofyW3rpAjbHxZkri6cwIrY//PD0WJvB/v2qAt44194mv2Sq4/tWXGyd4mBDQFvsVsUqLTADLXDOwLDYzXZnlpeDlExblN8tH9yvrcqUFOssdZSUqGY/gaLUQRWKio01f5193z41sQiUJTrortj9GG2xW5X4ePVAMvvhaCj2M84wVw5Pk5UFjY0qjmHsWPPkcMph77MXtz8REqLG1+x7F5QM/l4GuSehoap860cfmSvH/v0wZ465MniasWPVs9fPFbu22C1KwXYbRVFj+GDjFnNzmnftUq42K7U49QRWSXnz8+I0brFKyltpaeCNLSh3/I4dvRqv+IzWVuVtCqTAOVDehwCIjNeK3YIYPaT3x44lo/Ywtjo73//7Z/ykYKfvhTEi4gMNqwQnlpTAyJGQnNz/sf7E/2/vzKPiKvO8//1RAVKBJAWBECgI2ckukA06Go1b4thjpz2vrbY9tuOM6XWmp6c73bF9e7Fbjzppt+6236P9tq/OUcdoq6ijMRmNhTExKySmspJAAhTZCVuAhOV5/3i4UBRVUMDd7+9zDqfCrap7H57ce7/3+a3Tpsm/zSBXR1FpAMse3YT2Eyfx0mmXNesB9Edhocwj37XLmONXVMjj2ylwToGFndECpcf1SU8G0hvPI669DQLAq9sr9b1BdXTIPux2868DPYExZhD2KVPs5acE5INTczNw+rTuh1YejKnyJEaITvjdKdYt9hOJJUvkq1HmeDumuink5MjOmkZZQ1SAhd2EKLnLJ5MmwCU6kVkvuw0JQN+CJRUVstuRHYXd7QYyMow3F1u0q9uAGJjypjwYZ188BUB2dbNssZ9IjBsnV8tGBdDZMdVNQQmgU/5GC8LCbkKU3OUTHllUY2Ldqe73dC1YYteIeAWjc9k7O+3rAw5OedMZ5RqZWCetBSe7yslasthPfxQWSmE3wt1RViar4I0bp/+xtcYGkfEs7CZkzYocEIDKJJkfOulij7DrWrBEEfbZs/U7pp4YHeBVUyPz2O0o7NnZMnrbgPlVrpGptdVoGRGPc4lJvbbbhoICmdVhxMOpnZq/hKKk8LGwM2ozMjYG50d50BTnxqSLNQAMKFji98uOR4mJ+h1TT6ZNAwIB6Qs2ArtGxAM9HQoNEPY1K3LgjnVhUfVBlGbkQFCMdYv99IdSqMYIP3tZme3860WlAdld8eHNqPGkoWp7qdFDGjIs7CZDCfxpaesEiFCRlIHJF2sQQz1NQXQLArJrRLyCsuIwqne4nYUdkA9OBsztqjwvfn/TRMw+W44dWXOtXeynP+bOlQ/devvZm5uBqipbrdiV+67SXbHMk4GGvQcsG3DJwm4ylMAfhYpkLybXBtDZ5UbTrZ3n5cvS3GZX/zrQc2MyKkjm+HFprp440Zjja42Bro5b648jRgj8+NHV2Lr2evuJOtBTqEZvYVf+T220Yg+975Yne5FdG8C6jw4bOKqhw8JuMkIDfCqSMuBtOIe49rbubbpE+B49Kjt02VnYlQAvI4U9O1uare3I1KlAba3sNaA3xcVAfHxPWphdKSgA9u3T151kw4j40Pvu8eRMJF5pQUd1tUEjGh4s7CYjNMCnItkLl+hEVl3vfGDNI3ztHhEPSDNmRgYqt++VvrW1H2Dp45tR19I28HfVwK6pbgpGdnnz+aTojRyp/7H1RClUs3u3fse0YQ576H23PFlaeBZdOW/EcIYNC7vJUAJ/FCqSZMrblIu9Te+aR/j6/cCIET2pHzblXEY2LpT6u31rgboWBC626ONbs7uwG5XyVl8PlJYC116r73GNoKBAvuppji8rk5USR4/W75gaE3rfLU/OBAD8U+oVo4Y0LFjYTcaqPG+vLl+1GdkAgMm1PUKjS4Sv3y9NbXFx2h7HYLbGJCP7Qm8R7xRCU1dHUWkAK3/9LnDxIp472WnZAJ0BMapD4eefyxoB112n73GNICVFPkDpGRlvw1S30PuuKysT7e5RyG0+Y/TQhgR3dzMhoV2+Lr8wDnMvnQFBrtTXrMjRPhho/34ZmGNzDiSkYVVLA8a2NKLe3bMC0crVoUTfTq06AQD4Mj4Ff3pb9gCwXYBXQoJsIKS3sBcXywdSZTVrdwoLgU2bZKEaPUoTl5UBt96q/XF0pk93xTetWzPedCt2IvoNEQWIaG/Xz98ZPSajiZ89E7eNuoSKx2/VJ8K3qUmWk7Wzf72L+qzJAIDJXbUCFLRydYSWO61MsmG502CMSHnz+WTQnNtmBWkiUVgInDkju61pTUODPJbNVuxhsXAzGNMJexdPCyFyu34+NHowhjNjhr6R2wcPylcHCPvNq5YB6O3qiCHSzNWhWAKyu8oEV46d0Gu77dA75a2hAdizxxlmeAU9/ew2DJyLSE6OfFhqbTV6JIPGrMLOBDNjBnDqFNDYqM/xnBAR38WNtxag0+XCVc1nQQC8Hje8SW7NrCKKJWBi3WmcG+XBpfhRvbbbjqlTZXW/Fp0eXLZulf51JwTOKcybJ90eevjZbZjqFpGcHOneMKqA1TAwq7D/kIi+JKIXiSjJ6MEYjvJ0rDwta43fL82YkyfrczwjiYtDzKRJuC/1Srerw+PWLq9cib6dfLEGJ5NkcxJbljtVUCLj9apn7vPJugBKuVUnMGIEsGiRvit2O2dzKFi4GYwhwXNE9DGACWHeegjA/wHwO8gupb8D8CSA+8PsYzWA1QCQlpYGn88X8XhNTU39vm92EurrsQjAwaIinG1oUG2/keZl/pYtiM3Kwp4tW1Q7lpmZl5KCuJIS7OmaCy3PFw+AxwpjMP+Zkzi+qAAP5nYibWwcPPVl8Pl0enAbIkOZl9F1dVgAYH9RES6cO6fJuILJf/99iJwclO7cqfmxFMxwf5mckYGs9evx+caN6IyP1+w4s7Zswdi0NGzfsWPAz5phXoaDq6UF1wAo37ABlSp2sdNlXoQQpv0BMAmAf6DPLViwQPTHp59+2u/7pqe5WQhAiN/+VtXdRpyXCROE+Pa3VT2WqfnRj4QYNUqIzk4hhA7ny6lT8v/z2We1PY7KDGlezp+Xf+tTT6k+nj40NAjhcgnx0EPaHysIU9xf3n1XzvOWLdoeZ/FiIW64IaqPmmJehovXK8S996q6SzXnBcBuEUYTTWeKJ6L0oF+/DsBv1FhMg9st64nrEUB3/jxw+rS9m78gqJPT2g/w+wohS3LW1Az8RTVwUAwDkpMBj0cfP+W2bbIKm5P86wp6BNAJYcsc9n6xaGS86YQdwH8Q0X4i+hLAcgA/NnpApkCvyPgDB+SrjUUntJNTqXs8AODzD7bqMwAnCTuRfpHxPp/0N3/lK9ofy2yMHy/nWUthv3BB1v13QkS8giLsQhg9kkFhOmEXQvyDEGKeEGK+EOI2IcQpo8dkChRh1/oEc4Do9O2gJ8v2btugU1lOvx9ITZU3Yyegl7AXF8sgsoQE7Y9lRgoKpLBrdY9wUqqbQk6OfJjRIT5ETUwn7EwEZsyQJ9iFC9oex++XptOMDG2PYyChOeOnRqegdUQckgMn9BmA32/rB6c+TJ8u84GvaFh3u6kJ2LXLWfnroRQWSjdaZaU2+3dSqpuCRSPjWditgnIxHTqk7XEU0dGjNKVBhOaMC4rBCU86ZjaejvANFenslO4OJwn73LmyBfBhDXtbb9smj+F0YQe0M8eXlcke8E5Ig1VgYWc05aqr5Ou+fdodQwhZI97mohPayQkATqRmIbdJB69PZaVcXdp8jnsxf758/fJL7Y5RXCxFx4n+dYV582SgrVbCfvSoFPVY7eo8mI6JE4H4eBZ2RiPS06Vfdu9e7Y4RCMiWlzaPiA/t5OT1uDHpmkVIDFTK6HgtcUAMQzBFpQEseyeAy64RePUv72vXyc7nk/71xERt9m8FYmPlHGhVga6szFn+dUA+LE6fbjlh5+5uVoEIyM3VVtgdJDp9Ojn9rRZ44WntXR37ZSc3zJmj7XFMgJJ90NLWgbKUbGRWHcN3tehkd+mS9K//+7+rt0+rUlgIPPWUrG8+cqR6+xVCCrsTUwlzcnquW4vAK3YrkZsrT7C2Nm32rwi7A0SnD8rDjJLupxV+vzTvjR2r7XFMQHD2weHUSZh57oQ2ney++EJeE072rysUFMi52LNH3f2eOiUfoJy2YgeksJeXa3ff1QAWdiuRmysji7UKQvL7pclfxfKJlmHaNNnD269xPSQHRcQHZx8cSp2EtKZaJDfXq9/JTvGvL12q7n6tiBJAp7Y53okR8Qo5OTIws6vfQXBxq6WPb9bOvTQMWNitRG6ufNXKHO8g0enDiBHAzJk4vW03jpxu1OaibWuTD2UOmePg7IPD42Ukdc65E+p3svP5gAULgNGj1d2vFUlLkwFuagfQOTGHXSEoMj60uFWgrgUPvr3fdOLOwm4lcnJk1KsWwt7R4bw0rBCqvFPQuf8ArnR0anPRHjsmLS4OmePg7IPDqZMAAPMvnFS3k11zM7BzpzN9v5EoLFS/UE1ZmYwOz8pSb59WIUjYQ4tbAdDGvTRMWNithMslI9a1EPbychlwY/OI+P74oHMcMhrOIjYoMl7Vi9ZBwYlA7+yD2gQPLiQm4ZvuenUD57Zvlw9L7F/voaBA9j2orlZvn0ePygqCLtfAn7UbHg9ak1Pw/hufIhDBjaS6e2mYcFS81cjNBf72N/k0rmYRGYeJTjj2JMhqe+MClQB65kG1i9bvB2JigJkz1dmfBeiVfVCyEOMCKjeD8fnknF59tbr7tTLBhWrUWmGXlTnTvw7pU89MmIC00ycjfkZ199Iw4RW71cjNBWpr1X0aB3qEffZsdfdrIeqmyhtXcnVVr+2qXbR+vwzSc5vrJqAb8+dLd097u3r7LC4G8vOBMWPU26fVueoqmeqmlp+9o0O6kZzoX4fM7ihLysCU2vAuOXesS133kgqwsFsNrQLo/H5gyhTnNtAAcM+d16I5Nh7jqnuezFW9aJ0cnAhIYW9tVa+Fa0uLNMWzGb43sbHAwoXqCXtVlXR3OFTYa+paUJ6ciZTmeoxpber1ntfjxmO3z1PXvaQCLOxWY948aYLXQtidLDoAVi3IwuXpM5ESqOquSKfaRdvSIgXNyXOsxG+oVVp2xw4pOBw415fCQqC0FLh8efj7cnKqG6TFrnycvAdMvdBjKfV63Ni69voB7w+h6XF1Ldrnw7OwW43ERPnkXFqq3j4vX5YlE50sOl0kLc5Dak0VKh6/NaqLNmoOH5YNYJw8x7NmyeArtYSd/euRKSyUDz0lJcPfl5NT3SCzO2rGTwSAbnM8AVg+M3XA74ZLjwtcbNE8PY6F3YqoXVr2yBHpR3NwRHw3c+civrZW/fa4HJwo/b45OeoJe3GxvBY8HnX2ZycKCuSrGub4o0eliy49ffj7siCr8rxYdH0+2mJcmFIrV+wCwFt7AgMKdLj0uE4hNE+PY2G3Irm5QEWF7M+uBiw6PSjldNUuLev3y8p206apu1+rMX++OsLe2ipFi/3r4UlPB7Kz1alApzR/sXEr54H45FgdKj3pvQLookmFjZRRo3V6HAu7FVEC6NRa+fj9svKaQ31ovVAebtQuLev3yzQ3J7W8DMf8+cDJk7KL4HDYuVO6kNi/HhmlUM1wOXrU8fcGGUDn7V6xB2/vj0gZNVqnx7GwW5G8PPmqljne75cm0rg4dfZnZbxetCckqN/NiYMTJUpv9uHOr88nV5DXXDPsIdmWggKZFjuc1Ni2NuDECcf61xUyPG4cT/Zi0sUaxHR29NreH8HVFxViiDRPj2NhtyITJsia0P0I+6AaFbDo9ECEpmnT1A1ObGgAKit5joEeYR+utcnnk/naSUnDHpJtUaMhTEWFjL9x+Ip9zYocnEibhPiOdkyurQEQXSpscPVFJdPGm+TWPD2Ohd2q9BNAN6hGBY2N8uJl0emmcfp0YN8+9QqpKP56nmMgM1MGuw1H2C9fZv96NOTmDr9QjZLq5vAV+6o8L26+ZyUAYO7Z44NKhV2V58XWtdd3Z9p43Nq741jYrUpurhSMK1f6vDWoRgUHD8pXjojvpmnGDBmcpVZ7XMVfz3Mszefz5w/PFL9zp/z/YWHvn7g42fVuOMKupLo5fMUOANfffh0wciSendahbiqsBrCwWxWlN/uhQ33eGlQkJkfE96FRWZ2okQMMyDlOTAQmTlRnf1ZHEfbOzqF9v7iY/evRUlAA7NkzqEI1wW68d9Z/iitjxgLjxmk4SIsQGysflHbsMHokA8LCblX6KS07qEhMv1/WLp88Wc3RWZrmrCxg1Ch5Q1QDv1+m0cXw5QZACntjo4yOHwo+n9xHcrKqw7LcMwBBAAAU2klEQVQlSqGaKANtQ914WdXH4B/jNV2/ccNQHpTCWErNBN9prMr06RF7s4eLxIwY6MGi0xeXSz44qSDsRaUB1O4swfqWsQMHMTqF4QTQXbkCbNvGaW7REtzprR+UVfq/rd/b7caLa2/DvNNl2JUxEz95Yx+fu4AU9suXZQyOieG7uVVxueQNMoywh4vEjBjowRHxfahracMbmICWHbux7NH/GfINrag0gGde2ozkpjocTs3uP4jRSShFgIYi7Lt2ybr77F+PjowM2bq1H2EPXqUHM/fMMcR3tKPEOxMdQvC5C/RU9FOj8I+GcD92C1OROR3jPijCVT//b2QkjcKaFTnd4t2rD3YEYuvrgdOnWdiDKCoNIHCxBUfGTcE32i8j8dhhPPi2XMEMNlhm3cYjWFgun+x3Zsk5VoIYzRx4ozmJicDUqUMT9uJi+cr+9egpLOxXiMIF2wJAfkDG75RkzALA5y4AmdXh9cr5/Jd/MXo0EeEVu0UpKg3g5WYPxrQ2IaPh3JBWgwkVFfIfHK3dzbqNR9ApBPamS7dFXs3hqEpHhqOmrgVLqvxoiE/AodRJvbY7nvx8eXMUYnDf8/nk+ZqSosmwbElhoayjUFMT9u1I5+OCwGGc9EzAucSkAT/rKAoKTL9iZ2G3KOs2HsHeFBnwNvtsOYDoahcH0y3svGLvRrlxVXom4PyoscirOdJr+2DI8LhRULkfO7LmoDPG1Wu747npJlkRTUm3jIa2NmDrVvavD5YBGsKEPR+FwILAIezxzhr4s06joAAoLwfOnjV6JBFhYbcoNXUtOJyajQ6Kwewz5b22R0tCRYWs3OXQrk3h6L5xEaHEOwtLqvyAEEO6of0yfyymXKzBjqyeB6doqlU5ghUr5OvGjdF/Z/duoLmZ/euDJS9P5rRHWGWGC7ad3HgW4y9dREmQsPO524XyoGTitDcWdouS4XGjNXYkKpIyMOdsea/t0ZKgVJxzcNemUNasyEFM13xsnrIQWfVnML+uakg3tJUXZNWu8jmLBg5idBoTJwKzZwMffRT9d3w++bpsmSZDsi3x8f0WqgkXbLvOewkAUDUzl8/dUPLzZdMsE5vjOXjOoqxZkYMH396Pg2lTsKD6ECAE3HEjohcgIaSwcxBSL1bleVF0+iC8Hhc+mbYEnZuewxMxxzFrKDe04mJgzBi8+MwDMouB6c2KFcBzzwGXLsl+3wNRXCwj6lNTtR+b3SgslHN95UrYZk99gm1/8BaQmIiXn7qfz91QRo2SfQpMLOy8YrcoylP2wVkL4W08hxUNFYN7oq6uxohLl9i/HgaPOxZrVuQgLjMDJRkzEfPeu0NL8/H55IMT3xjDs3KlFBol0r0/2tqAzz9nM/xQUfKvo+0IuW2b/A6fu+EpKJCljdXqJ6EyLOwWZlWeF2tffhhITMTzLXsGZybjUrIRqWtp687r3Ti9EDk1x/DHFz8enLifPg0cOcKBXv2xbJksshSNOb6kRK7seT6HxmA6vTU2ylTEr3xF2zFZmWuuAZqaZF0FE8LCbnUSE4F77gHWrwcuXoz+eyzsETlT39qd17tphgyUWXZo6+BS3pRVKK8wIzNypJyfaALoFP86C/vQyMyUP9E0hPnsM1nH/+qrtR+XVbnpJlmtc8MGo0cSFhZ2O/Cd78huV6+8Ev13/H5cTknhetthuNLR05zkZFIGDqdk4+ay7YNLeSsuBkaPlhHJTC+Cm4w87ZoiW4OWl/f/peJiYNYsYPx4fQZpRwoKohP2Dz+UfmSOv4lMcjKwZAkLO6MheXnAwoXACy9EX/DD78elSZM0HZYVCWdu3zS9AIurDmBW7CAaP/h8csUzguNTgwltMvJeuqwbv/f/ro/8pfZ2YMsWtn4Ml8JC2Xjn1KnInxFCCvsNN0iLChOZW26RKZgmzGdnYbcLq1dL83o0PrSODuDgQVzijm59CGdu3zijEC7RiUdGnIhuJ2fPyna6bDbuQ2j50oqkDFSOTUNj0X9H/lJJifRn8nwOjwH87EWlAXzzJy8BJ05gnWsq14UfiFtuka+bNhk7jjCwsNuFu++W/vYXXhj4s+XlQGsrC3sYwpnbD6RNRWB0KvJLfNHtRLnQly9Xb2A2oc/8EqF4ygLkHyuJ3ApTiVdgYR8eeXmyp3gYc7xiSZm993MAwDvp87npy0Dk58vUSxOa41nY7UJwEF1dXf+f3b8fAFjYwxC2wA8Rts29Wgr2pUsD72T9ehmotHCh+gO0OOHmt3jyAiS0tcpysV0E++G3/fUtNE6aCkyYoOdQ7cfIkUB+Ps5//Fn33CqthBVLyk1lO3A4JRs1Y8YPuUeCY4iJkSmbGzbIGKcQgs9hvVs2s7DbidWrZUvLgYLodu8GiHApO1ufcVmI4MpzCu5YF9Lvu0tevAOZ3erqZJT3N77BPe7DEK58aenUXHSOiO2Ojg/2w8d0dmBuxX58kJzDq0cVODZ1HhL278WZC40QQHfzqEBdCzIazmJJ9QG8P6unsh83fRmA++6T2Uivv95rc2gsid4tm/nOYyfy8wcOomtoAJ5/HrjlFnS6uaFDKKvyvPAmufv0sr/6/ttlJOw77/S/gzfflMVU7rxTl/FajXDlS3959xLEXL20O5892A8/+0w5xlxpxjbvHF49qsB/iglwt1/GrLMV3dta2jrgIsJtBz8DgF7Czk1fBmD5cmDOHNQ98SSWPvZJ9+r84fcP9GmFq6cFhEN27cbq1fJn+/aeYJlg/vAHoLYWePhhGZDE9MHjjsXWtdf1fePv/x547z0p3LGxfd5+b/txLP7Z/8bZCdPwvU8asSY2wLW1w9CnfCkgTZpr1wI7d/ZaJS6pkm6j7VlzcY5Xj8PmE88U/BZAfs1h7E+f3r09oaURD+x6B19MnIfKJNkUipu+9I/iwliWuRyPbfwT0g/sQSBzDgJhztPJtQGcThyHmjoAiKJ88jDhFbvduOuuyEF09fXAk09KgWL/7+BZtUqa3T77rM9bRaUBHPzVf2BC3Vk8fu19CNS3cvDRYHjgASA7G7j9dsxx9dwYCyr343iyF2dHj+PVoxpkTcTpxGTk1Rzutfmh3W8iuaUBz3/tB9z0JQqCTe1Fs69D3chE3Lf7/bCfjenswJ+LHsPLb/5Kt3OYhd1ujB4dOYjumWfktocfNmZsVufmm2UJ1Nde6/PWn9/dg+9s+S98NikP2yblAtDX9GZ5FDfHhQt46aPfY3SMQExnBxZXH8SOrLm8elSJNStnYl/mLOQHeoR9bl017tjxLuiBB/DSH76Disdvxda117Oo90Owu6glbiRen38zVh7dhvSGc30+e8f+jzHr3Am8umSVbuewIcJORHcQ0QEi6iSihSHvPUhEx4joCBGtMGJ8lkcJonv11Z5tFy8CTz8tV51cDW1ojBoF/PM/Ay++KIt4oCfy9bZNryKptRFPXHdfr6+EM8sxEcjLA/7yF6SU7EDRiSJc21KDMZcv4ejMBbx6VIlVeV54Vy7HxPozSL10Ed6xI/HivtcQk5gIPPKI0cOzDKFBha/k3woC8MSGP+COLzdh1tlypMQRprmBn2x5Bfuy52L5L76n2zlslI/dD+B2AM8HbySi2QDuAjAHQAaAj4lohhCio+8umIgoQXTPPw98//vAuXPAL38pTfG/+Y3Ro7M2Tzwh86rvvRcfvbIBD269gDG1Z/BPu95F0exrcSBtaq+PE6T4syhFybe+BezejanPPov/t6wMAPCbJ78PeHn+1GLu/1oJPPMIdt04GkAT8IvPgGef5Xa4gyDD4+710F49Ng1/KvwG/nH3e1h2ohQA0BEfD5fLBTQ3Y/wnG3BVfqZu4zNE2IUQhwCAQtKKAHwNwOtCiMsAKojoGIDFAKIocMz0QgmimzJFlpEUQqZmXHWV0SOzNm438MYbwMKFyL37q/jL2HRMvVANgsCT1/xDn48LSLMdC/sgWLdOthctLgamTWNRV5v8fBn8+bOfyfKyc+YA3/ue0aOyFGtW5ODBt/f3inx/5ppv4Zmrv4nFHRfx03GNWHS+HNizR9bcX7JE1/GZLSreCyC43mF11zZmsNx7r4yMP3wYuP9+4KtfBXJzjR6VPcjJAdavR9XqNRjZfgVHU7Px7NK7UeUJX0CFc4EHSWysfHhaskSet4y6uN3AzTej9Ysd2Dy9EE/mfx2tT27BmhU5/AAaJco8rdt4BDV1LcjwuE01fySibRoy2B0TfQwg3J3uISHEu12f8QH4qRBid9fvzwH4QgjxStfvfwXwoRDirTD7Xw1gNQCkpaUteD2kQEAwTU1NSExMHN4fZBPqWtpwpr4VVzo6kT4KGDFyFDzuvqlbTmag8yV4DqMhzhWDnAmj1RqeYeh9HVFbG8SIEUBfy56psOL9pa6lDYHaZgSfwTFE8Ca5VbsfWHFe9EDNeVm+fPkeIUSfFCfNVuxCiBuH8LVqAFlBv2cCqImw/xcAvAAACxcuFNf10/nJ5/Ohv/edQlFpAA9+sh8tbTEAYvCTee34c0kHHrt9tmmeNM1Af+dL6BwOhDvWhcdun4frbDC/fB2Fx4rzsvTxzQjU9b39ez2u8DUchoAV50UP9JgXs6W7vQfgLiKKJ6LJAKYD2GnwmGxDaGctgFOyBku4OVTwetz4VsHEPlXr+KGJMRuR3EPsNrIHhvjYiejrAP4IIBXAB0S0VwixQghxgIjeAHAQQDuAH3BEvHrwxTx8Is0VAdi69np9B2MTlApeZvRV2pXQqO7g7Yz1MWTFLoR4RwiRKYSIF0KkCSFWBL33qBBiqhAiRwhhvn54FibSRcsXc/TwHKqL0c0ynEq4ZjxcBMg+mM0Uz2gIX8zDh+dQXdg9ZAzhmvGw28g+mC3djdGQ0BSNOFcMX8yDJHQOx7pjQQT8eP1erNt4hM3Ig4TdQ8YRthkPoxpGuphY2B1G8MXs8/lsEa2tN8ocKmZkZcWpmJGVzzADw75exo70d2/w6HB8NsUzzBBhM/LwYdcGY0eMvjfwit0BcNSxNrAZefiYvYIXwwyF/u8N2vdjZ2G3OUabhOwMm5HVgX29jN0w+t7ApnibY7RJyM6wGZlhmHAYfW9gYbc5bC5WD6X3+uS1H2Dp45sBgFOGGIbpg9HphGyKtzlGm4TsQiSXxmO3z+OKcwzD9MFIFxOv2G2O0SYhu8AuDYZhrAKv2G1KcCT8WHcsRsbGoK65rVfUsc9XZvQwLQO7NBiGsQos7DYk1Gxc19IGd6wLT9+Zy/7fIcIuDYZhrAKb4m0Im43Vh10aDMNYBV6x2xA2G6sPF1JhGMYq8IrdhnBrUW1YlefF1rXX4+k7cwHIxi9LH9/MLUYZhjEVLOw2hM3G2sH9wxmGMTss7DbE6OIIdobjFxiGMTvsY7cpXH9bGzh+gWEYs8MrdoYZBBy/wDCM2WFhZ5hBwPELDMOYHTbFM8wg4LQ3hmHMDgs7wwwSjl9gGMbMsCmeYRiGYWwECzvDMAzD2AgWdoZhGIaxESzsDMMwDGMjWNgZhmEYxkawsDMMwzCMjWBhZxiGYRgbwcLOMAzDMDaChBBGj2HYENE5ACf7+UgKgPM6DcdK8LyEh+clPDwv4eF5CQ/PS3jUnJdsIURq6EZbCPtAENFuIcRCo8dhNnhewsPzEh6el/DwvISH5yU8eswLm+IZhmEYxkawsDMMwzCMjXCKsL9g9ABMCs9LeHhewsPzEh6el/DwvIRH83lxhI+dYRiGYZyCU1bsDMMwDOMIHCPsRPQ7IvqSiPYS0SYiyjB6TGaAiNYR0eGuuXmHiDxGj8kMENEdRHSAiDqJyNGRvUS0koiOENExIlpr9HjMAhG9SERnichv9FjMAhFlEdGnRHSo6/r5kdFjMgNENJKIdhLRvq55eVjT4znFFE9EY4QQDV3//lcAs4UQ3zV4WIZDRDcD2CyEaCeiJwBACPFzg4dlOEQ0C0AngOcB/FQIsdvgIRkCEbkAHAVwE4BqALsA3C2EOGjowEwAES0D0ATgP4UQc40ejxkgonQA6UKIEiIaDWAPgFVOP1+IiAAkCCGaiCgWwOcAfiSE2K7F8RyzYldEvYsEAM54ohkAIcQmIUR716/bAWQaOR6zIIQ4JIQ4YvQ4TMBiAMeEEOVCiCsAXgfwNYPHZAqEEJ8BqDV6HGZCCHFKCFHS9e9GAIcAeI0dlfEISVPXr7FdP5ppkGOEHQCI6FEiqgJwD4BfGT0eE3I/gA1GD4IxFV4AVUG/V4Nv1EwUENEkAHkAdhg7EnNARC4i2gvgLID/EUJoNi+2EnYi+piI/GF+vgYAQoiHhBBZAF4F8ENjR6sfA81L12ceAtAOOTeOIJp5YUBhtrG1i+kXIkoE8BaAfwuxljoWIUSHECIX0iq6mIg0c9+M0GrHRiCEuDHKj74G4AMAv9ZwOKZhoHkhom8D+CqAG4RTgi4wqPPFyVQDyAr6PRNAjUFjYSxAlw/5LQCvCiHeNno8ZkMIUUdEPgArAWgSeGmrFXt/ENH0oF9vA3DYqLGYCSJaCeDnAG4TQjQbPR7GdOwCMJ2IJhNRHIC7ALxn8JgYk9IVJPZXAIeEEE8ZPR6zQESpSsYREbkB3AgNNchJUfFvAciBjHQ+CeC7QoiAsaMyHiI6BiAewIWuTds5WwAgoq8D+COAVAB1APYKIVYYOypjIKK/A/AMABeAF4UQjxo8JFNARP8F4DrIbl1nAPxaCPFXQwdlMER0NYAtAPZD3msB4BdCiA+NG5XxENF8AC9DXkMxAN4QQvxWs+M5RdgZhmEYxgk4xhTPMAzDME6AhZ1hGIZhbAQLO8MwDMPYCBZ2hmEYhrERLOwMwzAMYyNY2BmGYRjGRrCwMwzDMIyNYGFnGGbQENEiIvqyq890QlePaW5dyjAmgAvUMAwzJIjoEQAjAbgBVAshHjN4SAzDgIWdYZgh0lU7fheAVgBfEUJ0GDwkhmHApniGYYZOMoBEAKMhV+4Mw5gAXrEzDDMkiOg9AK8DmAwgXQjxQ4OHxDAMbNaPnWEYfSCiewG0CyFeIyIXgG1EdL0QYrPRY2MYp8MrdoZhGIaxEexjZxiGYRgbwcLOMAzDMDaChZ1hGIZhbAQLO8MwDMPYCBZ2hmEYhrERLOwMwzAMYyNY2BmGYRjGRrCwMwzDMIyN+P/SpKL4TJvqoAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# let's see it\n",
    "plot_cos_lls_sol(x, y, y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## <img src=\"https://img.icons8.com/color/96/000000/infinity-large.png\" style=\"height:50px;display:inline\"> Regularization\n",
    "---\n",
    "* As we have already seen on previous tutorials, a good way to **reduce** overfitting is to regularize (i.e. constrain) the model. \n",
    "* The fewer degrees of freedom it has, the harder it will be to overfit the data. \n",
    "    * For example, in *Polynomial Fitting*, a simple regularization would be to reduce the number of polynomial degrees. \n",
    "* We will look at **Ridge & Lasso Regressions**, but first, we will review one of the most important concepts of ML."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### <img src=\"https://img.icons8.com/color/96/000000/wrestling.png\" style=\"height:50px;display:inline\"> The Bias-Variance Tradeoff\n",
    "---\n",
    "An important theoretical result of statistics and ML is the fact the **the model generalization error** can be expressed as a sum of 3 different errors: **bias, variance and irreducible error**.\n",
    "* Given a **true** (but unknown) function $F(x)$ with noise $F(x) = f + \\epsilon$, we seek to estimate it based on $n$ samples from a set $\\mathcal{D}$. We denote the **regression function** as $g(x; \\mathcal{D})$.\n",
    "* The error of the regression model is given by: $$ MSE = \\mathbb{E}_{\\mathcal{D}} \\big[(F(x) -  g(x; \\mathcal{D}) )^2 \\big] $$\n",
    "* The total error can be decomposed into 3 terms: $$ E[(F - g )^2] = E[(f + \\epsilon -g)^2] =  E[(f + \\epsilon -g +E[g] - E[g])^2] = ... = (f - E[g])^2 +E[\\epsilon^2] +E\\big[(E[g] - g)^2\\big]$$ $$ = Bias[g]^2 +\\sigma^2 +Var[g]$$\n",
    "\n",
    "\n",
    "* <a href=\"https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff\">Full derivation on Wikipedia </a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* **Bias** - $(f - E[g])$: It is the difference between the expected value and the true value. This part of the generalization error is due to wrong assumptions (e.g. assuming the data is linear when it is actually quadratic). A *high-bias* model is most likely to **underfit** the training data.\n",
    "* **Variance** - $E\\big[(E[g] - g)^2\\big]$: This part is due to the model's sensitivity to small variations in the training data. A model with many degrees of freedom (such as high-degree polynomial model) is likely to have **high variance**, and thus tend to **overfit** the training data. Low varaince - the estimate of $F$ does not change much as the training set varies.\n",
    "* **Irreducible Error** - $\\sigma^2$: This part is due to the noisiness of the data itself. The only way to reduce this part of the error is to clean up the data (e.g., fix the data sources, detect and remove outliers...)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* **Note** - if we do not separate the real function from the noise, then the bias is considered the **expected difference between the model and the true function (+noise)** : $Bias[g] = E[g - F]$\n",
    "* **Increasing** the model's *complexity* will typically **increase its variance** and **reduce its bias**. On the other hand, **reducing** the complexity **increases its bias** and **reduces its variance**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img src=\"./assets/tut_08_bias_var.jpg\" style=\"height:300px\">\n",
    "\n",
    "<a href=\"http://scott.fortmann-roe.com/docs/BiasVariance.html\">Image Source</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img src=\"./assets/tut_08_bias_var_2.jpg\" style=\"height:500px\">\n",
    "\n",
    "* Image from Duda, R. O., Hart, P. E., & Stork, D. G. (2012). Pattern classification. John Wiley & Sons"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### <img src=\"https://img.icons8.com/dusk/64/000000/2.png\" style=\"height:50px;display:inline\"> Ridge Regression\n",
    "---\n",
    "* *Ridge Regression* (also called *Tikhonov Regularization*) is a regularized version of Linear Regression where the *regularization term* is equal to $\\lambda \\sum_{i=1}^n \\theta_i^2$ ($l_2$) and is added to to the cost function.\n",
    "* This form of cost function **forces** the algorithm not only to fit the data but also **keep the model weights as small as possible**.\n",
    "* The regularization term is only added during **training**.\n",
    "* The **hyperparameter** $\\lambda$ controls how much to regularize the model.\n",
    "    * **The Shrinking Effect**\n",
    "        * When $\\lambda \\to 0$ we resort back to the (unconstrained) Least Squares solution.\n",
    "        * As $\\lambda$ increases, then all the weights end up very close to zero and the result is a **flat line** going through the data's mean."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* **Ridge Regression Cost Function**: $$ J(\\theta) = MSE(\\theta) +\\lambda \\sum_i\\theta_i^2 $$\n",
    "* **In Vector Form**: $$ J(\\theta) = ||X \\theta - y ||_2^2 +\\lambda ||\\theta ||_2^2 = (X \\theta - y)^T(X \\theta - y) + \\lambda\\theta^T\\theta$$\n",
    "* **The Gradient**: $$\\nabla_{\\theta}J(\\theta) = 2(X^TX\\theta - X^Ty) + 2\\lambda\\theta $$\n",
    "* **The Closed-Form Solution**: $$ \\theta^{*} = (X^TX +\\lambda I)^{-1}X^Ty $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* In Scikit-Learn:\n",
    "    * Closed-Form:\n",
    "        * `from sklearn.linear_model import Ridge`\n",
    "        * `ridge_reg = Ridge(alpha=1, solver='cholesky')`\n",
    "        * `ridge_reg.fit(X, y)`\n",
    "        * `ridge_reg.predict([values])`\n",
    "    * SGD:\n",
    "        * `sgd_reg = SGDRegressor(penalty='l2')`\n",
    "        * `sgd_reg.fit(X, y)`\n",
    "        * `sgd_reg.predict([values])`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* <a style=\"color:red\"> Important Notes </a>\n",
    "    * Sometimes the bias weight $\\theta_0$ is not regularized, and the sum starts from $i=1$, in vector form, the identity matrix $I$ is changed so the top-left value (first value on the diagonal) is zero.\n",
    "    * It is important to **scale** the data (using `StandardScaler` for example) before performing Ridge Regression, as it is very sensitive to the scale of the input features. This is actually true for most regularized models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* <br><img src=\"./assets/tut_08_alpha.jpg\" style=\"height:400px\">\n",
    "\n",
    "<a href=\"https://www.analyticsvidhya.com/blog/2016/01/complete-tutorial-ridge-lasso-regression-python/\">Image Source </a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### <img src=\"https://img.icons8.com/dusk/64/000000/1.png\" style=\"height:50px;display:inline\"> LASSO Regression\n",
    "---\n",
    "* *LASSO (Least Absolute Shrinkage and Selection Operator) Regression* is just like Ridge, but instead of $l_2$, it uses $l_1$ norm of the weight vector.\n",
    "* * **LASSO Regression Cost Function**: $$ J(\\theta) = MSE(\\theta) +\\lambda \\sum_i|\\theta_i| $$\n",
    "* **In Vector Form**: $$ J(\\theta) = ||X \\theta - y ||_2^2 +\\lambda ||\\theta ||_1 $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* The LASSO cost function is **not differentiable** at $\\theta_i=0$, but Gradient Descent still works if we use the *sub-gradient vector*: $$ g(\\theta,J) = \\nabla_{\\theta} MSE(\\theta) + \\lambda \\begin{bmatrix}\n",
    "  sign(\\theta_1) \\\\\n",
    "  \\vdots \\\\\n",
    "  sign(\\theta_n)\n",
    " \\end{bmatrix}, sign(\\theta_i)= \\begin{cases}\n",
    "    -1       & \\quad \\text{if } \\theta_i < 0 \\\\\n",
    "    0       & \\quad \\text{if } \\theta_i = 0 \\\\\n",
    "    +1  & \\quad \\text{if } \\theta_i > 0 \n",
    "  \\end{cases}$$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* LASSO Regression tends to **eliminate the weights of the least important features** (i.e., set to them zero). It automatically performs **Feature Selection** and outputs a *sparse* model (few non-zero weights).\n",
    "    * **Ridge** regression will tend to shrink the large weights while hardly shrinking the smaller weights at all. In **LASSO** regression, the shrinkage will be directly proportionate to the importance of the feature in the model.\n",
    "    * Since $\\lambda$ is an arbitrarily selected constant, some feature weights can reach zero, meaning that these features will not be included in the model at all. And that is the built-in feature selection of LASSO regression.\n",
    "    * In other words, ridge regression will try to find a good model with small-scale features possible while LASSO regression will try to find a model with as few features as possible.\n",
    "* $l_1$ regularization is much more **robust to outliers**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "* In Scikit-Learn:\n",
    "    * Algorithmic:\n",
    "        * `from sklearn.linear_model import Lasso`\n",
    "        * `ridge_reg = Lasso(alpha=0.1)`\n",
    "        * `ridge_reg.fit(X, y)`\n",
    "        * `ridge_reg.predict([values])`\n",
    "    * SGD:\n",
    "        * `sgd_reg = SGDRegressor(penalty='l1')`\n",
    "        * `sgd_reg.fit(X, y)`\n",
    "        * `sgd_reg.predict([values])`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img src=\"./assets/tut_08_alpha_2.jpg\" style=\"height:400px\">\n",
    "\n",
    "<a href=\"https://www.analyticsvidhya.com/blog/2016/01/complete-tutorial-ridge-lasso-regression-python/\">Image Source </a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img src=\"./assets/tut_08_lass_ridge.jpg\" style=\"height:300px\">\n",
    "\n",
    "* While LASSO is more robust to outliers and performs feature selection, Ridge provides a more balanced minimization of the weights."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### <img src=\"https://img.icons8.com/dusk/64/000000/plus-2-math.png\" style=\"height:50px;display:inline\"> Other Regularizations\n",
    "---\n",
    "* **ElasticNet** - a combination between Ridge and Lasso (both $l_1$ and $l_2$ regularizations)\n",
    "* A more generalized regularizer: $$ \\frac{1}{2}\\sum_{i=1}^n (\\theta^Tx_i - y_i)^2 + \\lambda \\sum_{j=0}^M |\\theta_j|^q$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img src=\"./assets/tut_08_reg.jpg\" style=\"height:200px\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### <img src=\"https://img.icons8.com/bubbles/50/000000/video-playlist.png\" style=\"height:50px;display:inline\"> Recommended Videos\n",
    "---\n",
    "#### <img src=\"https://img.icons8.com/cute-clipart/64/000000/warning-shield.png\" style=\"height:30px;display:inline\"> Warning!\n",
    "* These videos do not replace the lectures and tutorials.\n",
    "* Please use these to get a better understanding of the material, and not as an alternative to the written material.\n",
    "\n",
    "#### Video By Subject\n",
    "\n",
    "* Linear Regression - <a href=\"https://www.youtube.com/watch?v=HZ4cvaztQEs\">Lecture 3 | Machine Learning (Stanford)</a>\n",
    "    * <a href=\"https://www.youtube.com/watch?v=_21o_ylL0q4\">Machine Learning Lecture 13 \"Linear / Ridge Regression\" -Cornell CS4780</a>\n",
    "* Bias/Variance Trade Off - <a href=\"https://www.youtube.com/watch?v=SjQyLhQIXSM\">Bias/Variance (C2W1L02)</a>\n",
    "    * <a href=\"https://www.youtube.com/watch?v=zUJbRO0Wavo\">Machine Learning Lecture 19 \"Bias Variance Decomposition\" -Cornell CS4780</a>\n",
    "* Regularization - <a href=\"https://www.youtube.com/watch?v=sO4ZirJh9ds\">Linear regression (6): Regularization</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "## <img src=\"https://img.icons8.com/dusk/64/000000/prize.png\" style=\"height:50px;display:inline\"> Credits\n",
    "---\n",
    "* Icons from <a href=\"https://icons8.com/\">Icon8.com</a> - https://icons8.com\n",
    "* Datasets from <a href=\"https://www.kaggle.com/\">Kaggle</a> - https://www.kaggle.com/\n",
    "* Examples and code snippets were taken from <a href=\"http://shop.oreilly.com/product/0636920052289.do\">\"Hands-On Machine Learning with Scikit-Learn and TensorFlow\"</a>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}