Chapter 6 Regression.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Chapter 6: Regression.ipynb",
      "provenance": [],
      "collapsed_sections": [
        "1mrsPw8OSpPG",
        "TYGH-jIj60Y-",
        "1TiL2kLK8Wlq",
        "Wr9dJbIJTLR0",
        "IatgOLJgjUKY",
        "Wdk-sNNkkx86",
        "iPbXBvg3m-2e",
        "NGr8PKz4P-UZ",
        "0NX0f42yQJ8L",
        "kPjcRV2q3s1b",
        "GE4h6bbEnVfB",
        "p0eHvt-6UrwD",
        "n0PxHd5AVxOU",
        "C9ms4Iuhe9GZ",
        "S8ofDynCfD6B",
        "g7CrxN4VkI17",
        "iJRSxN9PkmtR",
        "aCeYYgZGb3U0",
        "2PIRxMVzcWH0",
        "QkMjdS6P8ibS",
        "fyPUNwN_1tp5",
        "i1gNY_o0OrtW",
        "HcvvQ8EpGXm4"
      ]
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# **`Chapter 6: Regression`**"
      ],
      "metadata": {
        "id": "Wb9LyXR91KSv"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Table of Content:**\n",
        "\n",
        "- [Import Libraries](#Import_Libraries)\n",
        "- [6.1. Introduction](#Introduction)\n",
        "- [6.2. Least Squares Estimators of the Regression Parameters](#Least_Squares_Estimators_of_the_Regression_Parameters)\n",
        "- [6.3. Statistical Inferences about the Regression Parameters](#Statistical_Inferences_about_the_Regression_Parameters)\n",
        "  - [6.3.1. Inferences Concerning $B$](#Inferences_Concerning_B)\n",
        "    - [6.3.1.1. Known Variance](#Known_Variance)\n",
        "    - [6.3.1.2. Unknown Variance](#Unknown_Variance)\n",
        "  - [6.3.2. Inferences Concerning $A$](#Inferences_Concerning_A)\n",
        "    - [6.3.2.1. Unknown Variance](#Unknown_Variance_A)\n",
        "  - [6.3.3. T-tests for Regression Parameters with statsmodels](#T-tests_for_Regression_Parameters_with_statsmodels)\n",
        "  - [6.3.4. F-statistic for Overall Significance in Regression](#F-statistic_for_Overall_Significance_in_Regression)\n",
        "- [6.4. Confidence Intervals Concerning Regression Models](#Confidence_Intervals_Concerning_Regression_Models)\n",
        "  - [6.4.1. Confidence Interval for $B$](#Confidence_Interval_for_B)\n",
        "    - [6.4.1.1. Known Variance](#Known_Variance_B)\n",
        "    - [6.4.1.2. Unknown Variance](#Unknown_Variance_B)\n",
        "  - [6.4.2. Confidence Interval for $A$](#Confidence_Interval_for_A)\n",
        "    - [6.4.2.1. Unknown Variance](#Unknown_Variance_AA)\n",
        "  - [6.4.3. Confidence Interval for $A + B x$](#Confidence_Interval_for_AB)\n",
        "    - [6.4.3.1. Unknown Variance](#Unknown_Variance_ABB)\n",
        "  - [6.4.4. Prediction Interval of a Future Response](#Prediction_Interval_of_a_Future_Response) \n",
        "- [6.5. Residuals](#Residuals)\n",
        "  - [6.5.1. Regression Diagnostic](#Regression_Diagnostic)\n",
        "  - [6.5.2. Multicolinearity](#Multicolinearity)\n",
        "  "
      ],
      "metadata": {
        "id": "dgVrN97ojDFe"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Import_Libraries'></a>\n",
        "\n",
        "## **Import Libraries**"
      ],
      "metadata": {
        "id": "1mrsPw8OSpPG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install --upgrade scipy"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1GroeqIoC6fr",
        "outputId": "65173c4f-3b54-4792-ab7b-2385f99208b7"
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (1.4.1)\n",
            "Collecting scipy\n",
            "  Downloading scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)\n",
            "\u001b[K     |████████████████████████████████| 38.1 MB 1.2 MB/s \n",
            "\u001b[?25hRequirement already satisfied: numpy<1.23.0,>=1.16.5 in /usr/local/lib/python3.7/dist-packages (from scipy) (1.21.6)\n",
            "Installing collected packages: scipy\n",
            "  Attempting uninstall: scipy\n",
            "    Found existing installation: scipy 1.4.1\n",
            "    Uninstalling scipy-1.4.1:\n",
            "      Successfully uninstalled scipy-1.4.1\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible.\u001b[0m\n",
            "Successfully installed scipy-1.7.3\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "import matplotlib.patches as mpatches\n",
        "from matplotlib import collections  as mc\n",
        "import seaborn as sns\n",
        "import math\n",
        "from scipy import stats\n",
        "from scipy.stats import norm\n",
        "from scipy.stats import chi2\n",
        "from scipy.stats import t\n",
        "from scipy.stats import f\n",
        "from scipy.stats import bernoulli\n",
        "from scipy.stats import binom\n",
        "from scipy.stats import nbinom\n",
        "from scipy.stats import geom\n",
        "from scipy.stats import poisson\n",
        "from scipy.stats import uniform\n",
        "from scipy.stats import randint\n",
        "from scipy.stats import expon\n",
        "from scipy.stats import gamma\n",
        "from scipy.stats import beta\n",
        "from scipy.stats import weibull_min\n",
        "from scipy.stats import hypergeom\n",
        "from scipy.stats import shapiro\n",
        "from scipy.stats import pearsonr\n",
        "from scipy.stats import normaltest\n",
        "from scipy.stats import anderson\n",
        "from scipy.stats import spearmanr\n",
        "from scipy.stats import kendalltau\n",
        "from scipy.stats import chi2_contingency\n",
        "from scipy.stats import ttest_ind\n",
        "from scipy.stats import ttest_rel\n",
        "from scipy.stats import mannwhitneyu\n",
        "from scipy.stats import wilcoxon\n",
        "from scipy.stats import kruskal\n",
        "from scipy.stats import friedmanchisquare\n",
        "from statsmodels.tsa.stattools import adfuller\n",
        "from statsmodels.tsa.stattools import kpss\n",
        "from statsmodels.stats.weightstats import ztest\n",
        "import statsmodels.api as sm\n",
        "from sklearn.linear_model import LinearRegression\n",
        "from scipy.integrate import quad\n",
        "from statsmodels.stats.outliers_influence import summary_table\n",
        "from statsmodels.sandbox.regression.predstd import wls_prediction_std\n",
        "from statsmodels.stats.outliers_influence import variance_inflation_factor\n",
        "from IPython.display import display, Latex\n",
        "\n",
        "import warnings\n",
        "warnings.filterwarnings('ignore')\n",
        "warnings.simplefilter(action='ignore', category=FutureWarning)"
      ],
      "metadata": {
        "id": "o60rxBbwmJM5",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "09f6dd68-24f7-4950-c0b0-ae66b24921e1"
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/statsmodels/tools/_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.\n",
            "  import pandas.util.testing as tm\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Introduction'></a>\n",
        "\n",
        "## **6.1. Introduction:**"
      ],
      "metadata": {
        "id": "TYGH-jIj60Y-"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "In many situations, there is a single response variable $Y$ , also called the dependent variable, which depends on the value of a set of input, also called independent variables $x_1, ... , x_n$. The simplest type of relationship between the dependent variable $Y$ and the\n",
        "input variables $x_1, ... , x_n$ is a linear relationship. \n",
        "\n",
        "If regression coefficients are $β_0, β_1, ... , β_n$:\n",
        "\n",
        "$Y = β_0 + β_1 x_1 +···+ β_r x_n + e$\n",
        "\n",
        "where $e$, representing the random error, is assumed to be a random variable having mean 0.\n",
        "\n",
        "$E[Y|x] = β_0 + β_1 x_1 +···+ β_r x_n$"
      ],
      "metadata": {
        "id": "MAV-GAUk2b1a"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x = [100,102,103,101,105,100,99,105]\n",
        "y = [259,264,274,268,277,263,258,275]\n",
        "  \n",
        "plt.xlabel('X')\n",
        "plt.ylabel('Y')\n",
        "plt.title('Scatter Plot')\n",
        "plt.scatter(x, y, color ='blue', marker='*');"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "ITyOdRFv1N4B",
        "outputId": "085af27c-deee-4951-a9e5-f7494d2e0f61"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAeZElEQVR4nO3de5RdVYHn8e9PKFwOgQakrI4hEEjIUqJD0BulG2MPUdtHtwK+gAiJM45oKzZRTEN3u1wEetYgA9hrWCMsBAZmRCCBiGjStrTD2I6PYCVGSCooIGjAGAofIbauGOA3f5wTuVbdW6lTqVO3Hr/PWnfde/Y5Z9+9qVC/2ue1ZZuIiIjhel6nGxARERNLgiMiIipJcERERCUJjoiIqCTBERERlSQ4IiKikgRHxCQi6SJJn+t0O2JyS3DElCTpNZK+JWmHpF9I+qakBftY53sl/b8BZTdK+od9a+2g77lR0u8k/bps+92SXjKCeh6V9PrRbFtMDQmOmHIkHQx8GbgKOAyYAawAdnWyXa1I2r/NqstsTwOOAJ4AbhyzRsWUl+CIqWgugO1bbD9j+7e2v2r7vj0bSHq/pC2Sdkrqk/SKsvxCSQ83lZ9Wlr8UuAb4k3Ik8CtJ5wDvAf6mLPtSue2LJd0hqV/SI5L+uul7L5J0u6TPSXoKeO9QHbH9G+DzwMtarZf0Nkmby/b837KdSPrfwJHAl8q2/c3I/lPGVJTgiKnoh8Azkm6S9GZJhzavlPQu4CJgCXAw8Dbg5+Xqh4GFwB9RjFI+J2m67S3AB4Fv255m+xDb1wI3U44ObL9V0vOALwHfpxjpvA5YJumNTU04BbgdOKTcvy1J0yjC6Xst1s0FbgGWAd3AWoqgOMD22cBPgLeWbbts7//ZIgoJjphybD8FvAYw8FmgX9JdknrKTf4zxS/777rwkO0fl/uusv1T28/avg14EHhVha9fAHTbvtj272z/qGzDGU3bfNv2neV3/LZNPR+X9CvgIWAarUcmpwNrbN9tezdwOfAC4E8rtDdikHbHTyMmtXKE8F6A8sTy54B/BM4EZlKMLAaRtAT4GDCrLJoGHF7hq48CXlz+0t9jP+AbTctbh1HP5bY/sZdtXgz8eM+C7WclbaUY6USMWIIjpjzbD0i6EfhAWbQVmD1wO0lHUYwOXkcxKnhG0kZAe6pqVf2A5a3AI7aPHapJFZo/lJ8CL9+zIEkUofj4KH9PTDE5VBVTjqSXSDpf0hHl8kyKkcZ3yk2uozgU9EoV5pShcSDFL9v+cr//yB+elN4OHCHpgAFlxzQt3wvslHSBpBdI2k/Sy/b1UuA2VgJ/Iel1krqA8ymuHPtWm7ZFDEuCI6aincCrgXWS/o0iMDZR/GLF9irgv1BcrbQTuBM4zHYfcAXwbYpfui8HvtlU7/8BNgM/k/RkWXY9cFx5VdOdtp8B/hKYDzwCPEkRVH802p20/QPgLIrLjp8E3kpxMvx35Sb/FfhE2baPj/b3x+SlTOQUERFVZMQRERGVJDgiIqKSBEdERFSS4IiIiEqmxH0chx9+uGfNmtXpZkRETCjr169/0nb3wPIpERyzZs2it7e3082IiJhQJP24VXlth6okzZR0T/kE0c2SzivLb5O0sXw9Wt55i6T3NJVvlPSspPkt6r1I0uNN272lrj5ERMRgdY44ngbOt71B0kHAekl32z59zwaSrgB2ANi+mfJJoJJeDtxpe2Obuj9t+/Ia2x4REW3UFhy2twHbys87JW2heLhaH/z+uTnvBha12P1M4Na62hYRESM3JldVSZoFnACsaypeCGy3/WCLXU6nmEegnXMl3SfphoFzKTR95zmSeiX19vf3j7DlERExUO3BUU40cwewrJwHYY8zaREOkl4N/Mb2pjZVXk3x5NL5FCOaK1ptZPta2w3bje7uQRcFRETECNUaHOUTOe8Abra9uql8f+DtwG0tdjuDIUYbtreX030+S/GI6yqT6ERETBk7dsC8ecX7aKrzqipRPBl0i+0rB6x+PfCA7ccG7PM8ivMebc9vSJretHgaxVNNIyJigDVroK8P1q4d3XrrHHGcBJwNLGpx6Wy7UcVrga3ldJq/J+k6SY1y8TJJ90u6DzgZ+GhN7Y+ImJAWL4Zp02Dp0mJ5yZJiefHi0al/SjxWvdFoODcARsRU8dBD8La3waOPwm9/Cy94ARx9NNx1F8weNLdle5LW224MLM+zqiIiJpk5c+Dii2H3bjjwwOJ9xYpqoTGUBEdExCS0cmURGitWFO+rVo1e3VPiWVUREVPN8uVw1VXQ0wNnnQVbt45e3QmOiIhJaMGC5z739BSv0ZJDVRERUUmCIyIiKklwREREJQmOiIioJMERERGVJDgiIqKSBEdERFSS4IiIiEoSHBERUUmCIyIiKklwREREJQmOiIioJMERERGV1Dnn+ExJ90jqk7RZ0nll+W1NU8k+KmljWT5L0m+b1l3Tpt7DJN0t6cHy/dC6+hARMZHt2AHz5hXvo6nOEcfTwPm2jwNOBD4s6Tjbp9ueb3s+cAewummfh/ess/3BNvVeCHzN9rHA18rliIgYYM0a6OuDtWtHt97agsP2Ntsbys87gS3AjD3rJQl4N3BLxapPAW4qP98EnLrvrY2ImDwWL4Zp02Dp0mJ5yZJiefHi0al/TM5xSJoFnACsaypeCGy3/WBT2dGSvifp65IWtqmux/a28vPPgJbTk0g6R1KvpN7+/v5960BExARy8cVw5JHQ1VUsd3XBUUfBJZeMTv21B4ekaRSHpJbZfqpp1Zn84WhjG3Ck7ROAjwGfl3TwUHXbNuA266613bDd6O7u3qc+RERMJHPmFOGxe3cx3/ju3cXc47Nnj079tQaHpC6K0LjZ9uqm8v2BtwO37Smzvcv2z8vP64GHgbktqt0uaXpZz3Tgifp6EBExMa1cWYTGihXF+6pVo1d3nVdVCbge2GL7ygGrXw88YPuxpu27Je1Xfj4GOBb4UYuq7wLKI3csBb442m2PiJjoli+HH/wAzj+/eF++fPTqrnPEcRJwNrCo6RLbt5TrzmDwSfHXAveVl+feDnzQ9i8AJF0nqVFudynwBkkPUgTQpTX2ISJiQlqwAHrKM8A9PdBoDL19FSpOE0xujUbDvb29nW5GRMSEImm97UGRkzvHIyKikgRHROyzuu5QjvEpwRER+6yuO5RjfEpwRMSI1X2HcoxPCY6IGLG671CO8SnBEREjVvcdyjE+JTgiYp/UeYdyjE/7d7oBETGxLV8OV11V3GR21lmwdWunWxR1S3BExD5ZsOC5zz09z92tHJNXDlVFREQlCY6IiKgkwREREZUkOCIiopIER0REVJLgiIiIShIcERFRSYIjIiIqqXPO8ZmS7pHUJ2mzpPPK8tuappJ9tJwqFklvkLRe0v3l+6I29V4k6fEW09FGRMQYqPPO8aeB821vkHQQsF7S3bZP37OBpCuAPVO/PAm81fZPJb0M+GdgRpu6P2378hrbHhERbdQWHLa3AdvKzzslbaEIgj4ASQLeDSwqt/le0+6bgRdIer7tXXW1MSIiqhuTcxySZgEnAOuaihcC220/2GKXdwAbhgiNcyXdJ+kGSYe2+c5zJPVK6u3v79+H1kdERLPag0PSNOAOYJntp5pWnQnc0mL7ecCngA+0qfJqYDYwn2JEc0WrjWxfa7thu9Hd3b0PPYiIiGa1Ph1XUhdFaNxse3VT+f7A24FXDtj+COALwBLbD7eq0/b2pu0/C3y5hqZHREQbdV5VJeB6YIvtKwesfj3wgO3HmrY/BFgDXGj7m0PUO71p8TRg0+i1OiIi9qbOQ1UnAWcDi1pcOnsGgw9TnQvMAT7ZtP2LACRdJ6lRbndZecnufcDJwEdr7ENERAwg251uQ+0ajYZ7e3s73YyIiAlF0nrbjYHluXM8IiIqSXBEREQlCY6IiKgkwREREZUkOCIiopIER0REVJLgiIiIShIcERFRSYIjIiIqSXBEREQlCY6IiKgkwREREZUkOCIiopIER0REVJLgiIiIShIcERFRSYIjIiIqqXPO8ZmS7pHUJ2mzpPPK8tuapoZ9VNLGpn3+VtJDkn4g6Y1t6j1a0rpyu9skHVBXHyIiYrA6RxxPA+fbPg44EfiwpONsn257vu35wB3AagBJx1HMRT4PeBPwGUn7taj3U8Cnbc8Bfgm8r8Y+RETEALUFh+1ttjeUn3cCW4AZe9ZLEvBu4Jay6BTgVtu7bD8CPAS8qrnOcp9FwO1l0U3AqXX1ISIiBhuTcxySZgEnAOuaihcC220/WC7PALY2rX+MpqApvRD4le2nh9hmz3eeI6lXUm9/f/++dSAiIn6v9uCQNI3ikNQy2081rTqT50Ybo872tbYbthvd3d11fU1ExJSzf52VS+qiCI2bba9uKt8feDvwyqbNHwdmNi0fUZY1+zlwiKT9y1FHq20iIqJGdV5VJeB6YIvtKwesfj3wgO3HmsruAs6Q9HxJRwPHAvc272TbwD3AO8uipcAX62h/RES0VuehqpOAs4FFTZffvqVcdwYDDlPZ3gysBPqArwAftv0MgKS1kl5cbnoB8DFJD1Gc87i+xj5ERMQAKv6In9wajYZ7e3s73YyIiAlF0nrbjYHluXM8IiIqSXBEREQlCY6IiKgkwREREZUkOCIiopIER0REVJLgiIiIShIcERFRSYIjIiIqSXBEREQlCY6IiKgkwREREZUkOGLC2bED5s0r3iNi7CU4YsJZswb6+mDt2k63JGJqSnDEhLF4MUybBkuXFstLlhTLixd3tl0RU02CIyaMiy+GI4+Erq5iuasLjjoKLrmks+2KmGoSHDFhzJlThMfu3XDggcX7ihUwe3anWxYxtdQ55/hMSfdI6pO0WdJ5Tes+IumBsvyysuw9TVPMbpT0rKT5Leq9SNLjLaajjSlg5coiNFasKN5Xrep0iyKmnrZTx0paC3zI9qMjqliaDky3vUHSQcB64FSgB/h74C9s75L0IttPDNj35cCdtgf9LSnpIuDXti8fblsydezk8d3vFoerenpg+3bYuhUagya2jIjR0G7q2P2H2Od/Al+VdBNwme3dVb7Q9jZgW/l5p6QtwAzg/cCltneV655osfuZwK1Vvi+mhgULnvvc01O8ImJstT1UZXsV8ArgYKBX0sclfWzPq8qXSJoFnACsA+YCCyWtk/R1SQta7HI6cMsQVZ4r6T5JN0g6tM13niOpV1Jvf39/leZGRMQQ9naO43fAvwHPBw4a8BoWSdOAO4Bltp+iGOUcBpwILAdWSlLT9q8GfmN7U5sqrwZmA/MpRjRXtNrI9rW2G7Yb3d3dw21uRETsRdtDVZLeBFwJ3AW8wvZvqlYuqYsiNG62vbosfgxY7eLkyr2SngUOB/YMC85giNGG7e1N9X8W+HLVdkVExMgNdY7j74F32d48korLUcT1wBbbVzatuhM4GbhH0lzgAODJcp/nAe8GFg5R7/Ty/AnAaUC7kUlERNSgbXDYbvvLe5hOAs4G7pe0sSz7O+AG4AZJmygOhS31c5d2vRbYavtHzRVJug64xnYvcFl5ma6BR4EP7GM7IyKigraX404muRw3IqK6dpfj5s7xiIioJMERERGVJDgiIqKSBEdERFSS4IiIiEoSHBERUUmCIyIiKklwREREJQmOiIioJMERERGVJDgiIqKSBEdERFSS4IiIiEoSHBERUUmCIyIiKklwREREJQmOiIiopLbgkDRT0j2S+iRtlnRe07qPSHqgLL+sLJsl6beSNpava9rUe5ikuyU9WL4fWlcfIiJisLZzjo+Cp4HzbW+QdBCwXtLdQA9wCnC87V2SXtS0z8O25++l3guBr9m+VNKF5fIFdXQgIiIGq23EYXub7Q3l553AFmAG8FfApbZ3leueqFj1KcBN5eebgFNHp8URETEcY3KOQ9Is4ARgHTAXWChpnaSvS1rQtOnRkr5Xli9sU12P7W3l559RjGBafec5knol9fb3949ORyIiov7gkDQNuANYZvspisNjhwEnAsuBlZIEbAOOtH0C8DHg85IOHqpu2wbcZt21thu2G93d3aPXoYiIKa7W4JDURREaN9teXRY/Bqx24V7gWeBw27ts/xzA9nrgYYrRyUDbJU0v658OVD3UFRER+6DOq6oEXA9ssX1l06o7gZPLbeYCBwBPSuqWtF9ZfgxwLPCjFlXfBSwtPy8FvlhPDyIiopU6RxwnAWcDi5ousX0LcANwjKRNwK3A0vKQ02uB+yRtBG4HPmj7FwCSrpPUKOu9FHiDpAeB15fLERExRlT8zp7cGo2Ge3t7O92MiIgJRdJ6242B5blzPCIiKklwRHTIjh0wb17xHjGRJDgiOmTNGujrg7VrO92SiGoSHBFjbPFimDYNlpbXBi5ZUiwvXtzZdkUMV4IjYoxdfDEceSR0dRXLXV1w1FFwySWdbVfEcCU4IsbYnDlFeOzeDQceWLyvWAGzZ3e6ZRHDk+CI6ICVK4vQWLGieF+1qtMtihi+Oh+rHhFtLF8OV10FPT1w1lmwdWunWxQxfAmOiA5Y0PRM6J6e4hUxUeRQVUREVJLgiIiIShIcERFRSYJjisjjLSJitCQ4pog83iIiRkuCY5LL4y0iYrQlOCa5PN4iIkZbgmOSy+MtImK01Tnn+ExJ90jqk7RZ0nlN6z4i6YGy/LKy7A2S1ku6v3xf1KbeiyQ9PmA62hhCHm8REaOpzjvHnwbOt71B0kHAekl3Az3AKcDxtndJelG5/ZPAW23/VNLLgH8GZrSp+9O2L6+x7ZNKHm8REaOptuCwvQ3YVn7eKWkLRRC8H7jU9q5y3RPl+/eadt8MvEDS8/dsFyOXx1tExGgak3MckmYBJwDrgLnAQknrJH1d0oIWu7wD2DBEaJwr6T5JN0g6tM13niOpV1Jvf3//KPQiIiJgDIJD0jTgDmCZ7acoRjmHAScCy4GVktS0/TzgU8AH2lR5NTAbmE8xormi1Ua2r7XdsN3o7u4ere5EREx5tQaHpC6K0LjZ9uqy+DFgtQv3As8Ch5fbHwF8AVhi++FWddrebvsZ288CnwVeVWcfIiLiD9V5VZWA64Ettq9sWnUncHK5zVzgAOBJSYcAa4ALbX9ziHqnNy2eBmwa7bZHRER7dY44TgLOBhYNuHT2BuAYSZuAW4Gltg2cC8wBPtm0/YsAJF0nqVHWe1l5ye59FAH00Rr7EBERA6j4nT25NRoN9/b2droZERETiqT1thsDy3PneEREVJLgiIiIShIcERFRSYIjIiIqSXBEREQlCY6IiKgkwREREZUkOCIiopIER0REVJLgiIiIShIcERFRSYIjIiIqSXBEREQlCY6IiKgkwREREZUkOCIiopIER0REVFLnnOMzJd0jqU/SZknnNa37iKQHyvLLmsr/VtJDkn4g6Y1t6j1a0rpyu9skHVBXHyIiYrD9a6z7aeB82xskHQSsl3Q30AOcAhxve1fTvOLHAWcA84AXA/8iaa7tZwbU+yng07ZvlXQN8D7g6hr7ERERTWobcdjeZntD+XknsAWYAfwVcKntXeW6J8pdTgFutb3L9iPAQ8CrmuuUJGARcHtZdBNwal19iIiIwcbkHIekWcAJwDpgLrCwPNz0dUkLys1mAFubdnusLGv2QuBXtp8eYps933mOpF5Jvf39/aPTkYiIqD84JE0D7gCW2X6K4vDYYcCJwHJgZTmSGFW2r7XdsN3o7u4e7eojIqasWoNDUhdFaNxse3VZ/Biw2oV7gWeBw4HHgZlNux9RljX7OXCIpP2H2CYiImpU51VVAq4Htti+smnVncDJ5TZzgQOAJ4G7gDMkPV/S0cCxwL3Nddo2cA/wzrJoKfDFuvoQERGD1TniOAk4G1gkaWP5egtwA3CMpE3ArcDScvSxGVgJ9AFfAT6854oqSWslvbis9wLgY5IeojjncX2NfYiIiAFU/BE/uTUaDff29na6GRERE4qk9bYbA8tz53hERFSS4IiIiEoSHFPEjh0wb17xHhGxLxIcU8SaNdDXB2vXdrolETHRJTgmucWLYdo0WLq0WF6ypFhevLiz7YqIiSvBMcldfDEceSR0dRXLXV1w1FFwySWdbVdETFwJjkluzpwiPHbvhgMPLN5XrIDZszvdsoiYqBIcU8DKlUVorFhRvK9a1ekWRcREVud8HDFOLF8OV10FPT1w1lmwdeve94mIaCfBMQUsWPDc556e4hURMVI5VBUREZUkOCIiopIExxByt3VExGAJjiHkbuuIiMESHC3kbuuIiPYSHC3kbuuIiPYSHC3kbuuIiPbqnHN8pqR7JPVJ2izpvLL8IkmPD5hOFknvaSrbKOlZSfNb1Nty/9GWu60jIlqrbepYSdOB6bY3SDoIWA+cCrwb+LXty4fY9+XAnbYH/Y0v6aK97T/QSKaO/e53i8NVPT2wfXtxt3Vj0ASKERGTV7upY2u7c9z2NmBb+XmnpC3AjGHufiZwa11tG47cbR0R0dqYnOOQNAs4AVhXFp0r6T5JN0g6tMUupwO3DFHl3vZH0jmSeiX19vf370vzIyKiSe3BIWkacAewzPZTwNXAbGA+xYjkigHbvxr4je1Nbaoccv89bF9ru2G70d3dPSp9iYiImoNDUhdFaNxsezWA7e22n7H9LPBZ4FUDdjuDIUYbw9g/IiJqVOdVVQKuB7bYvrKpfHrTZqcBm5rWPY/i5Hnb8xtD7R8REfWr87HqJwFnA/dL2liW/R1wZnmZrYFHgQ807fNaYKvtHzVXJOk64BrbvcBlQ+wfERE1q+1y3PFEUj/w4xHufjjw5Cg2p5PSl/FnsvQD0pfxal/6cpTtQSeJp0Rw7AtJva2uY56I0pfxZ7L0A9KX8aqOvuSRIxERUUmCIyIiKklw7N21nW7AKEpfxp/J0g9IX8arUe9LznFEREQlGXFEREQlCY6IiKgkwdFE0nmSNpXzhywry46X9G1J90v6kqSDO93OdsqHPj4hqflu/MMk3S3pwfL90LJckv67pIfKB0a+onMt/0MV+/GS8uezS9LHO9fq1ir25T3lz+J+Sd+SdHznWj5Yxb6cUvZlY/mw0dd0ruV/qEo/mtYvkPS0pHeOfYvbq/gz+Q+SdjTNZfTJkX5vgqMk6WXA+ymefXU88JeS5gDXARfafjnwBWB551q5VzcCbxpQdiHwNdvHAl8rlwHeDBxbvs6heHjkeHEjw+/HL4C/BoY9P8sYu5Hh9+UR4M/Kf2uXMP5O0N7I8PvyNeB42/OB/0Tx/9F4cSPD7weS9gM+BXx1rBpYwY1U6AvwDdvzy9fFI/3SBMdzXgqss/0b208DXwfeDswF/rXc5m7gHR1q317Z/leKX6TNTgFuKj/fRDGZ1p7y/+XCd4BDBjwHrGOq9MP2E7a/C+weuxYOX8W+fMv2L8vy7wBHjEkjh6liX37t5668OZDiEUHjQsX/TwA+QvGw1ifqb101I+jLqEhwPGcTsFDSCyX9O+AtwExgM8UPAuBdZdlE0lNOqgXwM2DPlFQzgK1N2z3G8Cfa6oR2/ZiIhtOX9wH/NHZNGrG2fZF0mqQHgDUUo47xrGU/JM2geJjqeBqR781Q/77+RNL3Jf2TpHkj/YIER8n2Fp4bjn4F2Ag8Q/EP/kOS1gMHAb/rWCP3UfkX4Lj5y2+kJks/oHVfJJ1MERwXdKRRIzSwL7a/YPslFH/xXtKxhlU0oB//CFxQTuMw4QzoywaKZ08dD1wF3DnSehMcTWxfb/uVtl8L/BL4oe0HbP+57VdSzBPycGdbWdn2PYegyvc9w+3H+cPR0xFl2XjVrh8TUdu+SPr3FOcDTrH98w61r4q9/lzKwynHSDp8rBtXQbt+NIBbJT0KvBP4jKRRP/Qzylr2xfZTtn9dfl4LdI30Z5LgaCLpReX7kRTnNz7fVPY84BPANZ1r4YjcBSwtPy8FvthUvqS8uupEYEfT8HY8atePiahlX8p/d6uBs23/sENtq6pdX+ZIUvn5FcDzgfEchC37Yfto27NszwJuBz5ke8R/qY+Rdj+TP276mbyK4vf/yH4mtvMqX8A3gD7g+8DryrLzgB+Wr0sp77Yfjy+KEdE2ihPFj1Ec7nghxZUVDwL/AhxWbivgf1CMoO4HGp1u/wj78cflNk8Bvyo/H9zpPoywL9dRjHQ3lq/eTrd/H/pyAcX5wY3At4HXdLr9I+nHgP1uBN7Z6fbvw8/k3PJn8n2Kiy/+dKTfm0eOREREJTlUFRERlSQ4IiKikgRHRERUkuCIiIhKEhwREVFJgiNijEmaKekRSYeVy4eWy7M627KI4UlwRIwx21spnn10aVl0KXCt7Uc71qiICnIfR0QHSOoC1gM3UDzOf77tcfmE34iB9u90AyKmItu7JS2neKDmnyc0YiLJoaqIznkzxeMiXtbphkRUkeCI6ABJ84E3ACcCHx0vk2hFDEeCI2KMlU8ovRpYZvsnwH9j/E59GzFIgiNi7L0f+Intu8vlzwAvlfRnHWxTxLDlqqqIiKgkI46IiKgkwREREZUkOCIiopIER0REVJLgiIiIShIcERFRSYIjIiIq+f9flMOdeopBNwAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Least_Squares_Estimators_of_the_Regression_Parameters'></a>\n",
        "\n",
        "## **6.2. Least Squares Estimators of the Regression Parameters:**"
      ],
      "metadata": {
        "id": "1TiL2kLK8Wlq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x = [100,102,103,101,105,100,99,105]\n",
        "y = [259,264,274,268,277,263,258,275]\n",
        "\n",
        "a = -40.10169491525426\n",
        "b = 3.016949152542373\n",
        "ax = plt.subplot()\n",
        "ax.scatter(x, y, c='r', zorder=2)\n",
        "\n",
        "xs = np.linspace(98.9,105.1)\n",
        "ax.plot(xs, b*xs+a, c='k', lw=2)\n",
        "\n",
        "# computing and plotting grey lines\n",
        "lines = [[(i,j), (i,i*b+a)] for i,j in zip(x,y)]\n",
        "lc = mc.LineCollection(lines, colors='grey', linewidths=1, zorder=1)\n",
        "ax.add_collection(lc);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "id": "Lj81xxhEmuPP",
        "outputId": "38a2b8df-e7bb-419a-f258-061a3f83d1f9"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD4CAYAAADlwTGnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhUVbbw4d8CgQYBEQFBDERsuFYSRgNCg6jg2OpVum0bBdRLJxEZBEREwU/oFhSRSZQpCZPcyCSjSGNHQQShA0kYEogtoKAMQkAUMAxC1vdHHa4lJlCBJKcqtd7nyVOn9jln19ok7F1rn0lUFWOMMaGnlNsBGGOMcYcNAMYYE6JsADDGmBBlA4AxxoQoGwCMMSZEXeF2AAVRrVo1DQ8PdzsMY4wJKmlpaYdUtfr55UE1AISHh5Oamup2GMYYE1REZHde5RedAhKRMBFZKSLbRGSriPR2yueIyCbnZ5eIbHLKO/mUbxKRXBFpkke9Q0Rkr892f7zcRhpjjPGfPxnAGaCfqqaLSCUgTUSSVfWv5zYQkVHAjwCqmgQkOeUNgUWquimfuseo6sjLaoExxphLctEBQFX3A/ud5WMikgXUBrYBiIgAjwLt8tj9MWB2oUVrjDGm0BToLCARCQeaAik+xbcCB1R1ex67/BWYdYEqe4rIFhGZKiJX5/OZcSKSKiKp2dnZBQnXGGPMBfg9AIhIRWA+0EdVj/qseow8OnkRuQXIUdXMfKqcCNwINMGbYYzKayNVjVfVaFWNrl79NwexjTHGXCK/BgARKYO3809S1QU+5VcAfwLm5LFbRy7w7V9VD6jqWVXNBRKAFgUJ3BhjSpykJAgPh1KlvK9JSUX6cRc9BuDM8U8BslR19Hmr7wS+UNU95+1TCu9xgVsvUG8t5/gCQAcgv0zBGGNKvqQkiIuDnBzv+927ve8BOnUqko/0JwNoDXQB2uVxymZ+3/LbAt+q6le+hSKSKCLRztsRIpIhIluAO4C+l9YEY4wpAQYN+qXzPycnh90DBtChQwe2b8/rMOvl8ecsoDWA5LPuqXzKPwVa5lEe47Pcxd8gjTGmxPvmm1+9PQu8Awzau5ef9u4FYOHChYX6kXYvIGOMCQR16vzf4hbgD0Af4Cfg0UcfZeLEiYX+kTYAGGNMIBg2jJPly/MycDOwHrhehCXPPcecOXOoWbNmoX9kUN0LyBhjSqrPwsKIveoqvjxxAgF6VKrEa6NGUTk2tsg+0wYAY4xx0Q8//MCAAQOIj48HwOPx0KFDB4YNG1bkn21TQMYY45IFCxYQERFBfHw8ZcqUYfDgwWzcuJGyZcsWy+dbBmCMMcVs79699OzZk0WLFgHwhz/8gYSEBCIiIoo1DssAjDGmmOTm5jJ58mQiIiJYtGgRlSpVYvz48axevbrYO3+wDMAYY4rFF198QWxsLGvWrAHgwQcfZPz48YSFhbkWk2UAxhhThE6fPs3QoUNp3Lgxa9asoUaNGsyZM4fFixe72vmDZQDGGFNkUlJSiImJITPTe6uzrl278uabb1K1alWXI/OyDMAYYwrZsWPH6N27N61atSIzM5Mbb7yRTz75hClTpgRM5w82ABhjTKFatmwZUVFRjBs3jlKlSjFgwAAyMjJo1y6vhya6y6aAjDGmEBw8eJA+ffowa5b3Bsk333wzCQkJNG3a1OXI8mcZgDHGXAZV5d1338Xj8TBr1izKly/PyJEj+fe//x3QnT9YBmCMMZfsq6++olu3biQnJwNw1113MWnSJOrVq+dyZP6xDMAYYwrozJkzjBo1iqioKJKTk6latSozZszgo48+CprOHywDMMaYAtm4cSOxsbGkpaUB8PjjjzNmzBhq1KjhcmQFd9EMQETCRGSliGwTka0i0tspn+PziMhdIrLJKQ8XkRM+6yblU29VEUkWke3O69WF2zRjjCk8OTk5DBgwgObNm5OWlkadOnVYtmwZSUlJhdf5Ow+Ff2XIkMB4KDxwBuinqukiUglIE5FkVf3ruQ1EZBTwo88+O1W1yUXqfRH4RFWHi8iLzvsBBYzfGGOK3IoVK4iLi2Pnzp2ICL1792bo0KFUrFix8D7E56HwAoHxUHhV3a+q6c7yMSALqH1uvYgI8Ch5Pxz+Qh4CZjjLM4CHC7i/McYUqe+//56uXbvSvn17du7cSVRUFOvWrWPs2LGF2/lDvg+FZ9Cgwv0cHwU6CCwi4UBTIMWn+FbggKr6PrL+BhHZKCKrROTWfKq7VlX3O8vfAdfm85lxIpIqIqnZ2dkFCdcYYy6JqjJ37lw8Hg/Tpk2jbNmyvPrqq6SlpXHLLbcUzYee91D4i5YXAr8PAotIRWA+0EdVj/qseoxff/vfD9RR1cMicjOwSEQiz9vnV1RVRUTzWRcPxANER0fnuY0xxhSWb7/9lu7du7N06VIA2rZtS3x8PP/1X/9VtB9cp4532iev8iLiVwYgImXwdv5JqrrAp/wK4E/AnHNlqnpKVQ87y2nATqBBHtUeEJFaTj21gIOX2ghjjLlcubm5jB8/noiICJYuXUrlypWZPHkyK1euLPrOH2DYMKhQ4ddlFSp4y4uIP2cBCTAFyFLV0eetvhP4QlX3+GxfXURKO8v1gPrAV3lUvQR40ll+Elhc8PCNMebybd26lTZt2tCzZ0+OHz/On/70J7KysoiLi6NUqWK6XKpTJ4iPh7p1QcT7Gh9fZAeAwb8MoDXQBWjnc2rnH511Hfntwd+2wBbntND3gW6q+j2AiCSKSLSz3XDgLhHZjncgGX6ZbTHGmAI5deoUgwcPpmnTpqxbt45atWoxf/585s+fz3XXXVf8AXXqBLt2QW6u97UIO38AUQ2eafXo6GhNTU11OwxjTAnw+eefExsbS1ZWFgBPP/00w4cPp0qVKi5HVvhEJE1Vo88vtyuBjTEh5ejRo7z00ktMmDABgAYNGpCQkEDbtm1djqz42b2AjAl1ztWnlCpVLFefumnJkiVEREQwYcIErrjiCgYNGsTmzZtDsvMHywCMCW0+V58CxXL1qRu+++47nn32WebNmwdAixYtSExMpGHDhi5H5i7LAIwJZS5cfVqcVJWpU6fi8XiYN28eV155JW+99RZr164N+c4fLAMwJrS5cPVpcdm+fTtPP/00K1euBOC+++5j4sSJ1K1b1+XIAodlAMaEsvyuMi3Cq0+L2s8//8zw4cNp1KgRK1eupFq1aiQlJfHhhx9a538eGwCMCWUuXH1alFJTU2nevDkvvfQSJ0+e5IknniArK4vHH38c7zWtxpcNAMaEMp+rTxWK5erTovDTTz/Rr18/brnlFjZv3kx4eDgfffQRM2bMoFq1am6HF7BsADAm1DlXn/5jyJBiufq0sCUnJxMVFcXo0d471fTr14/MzEzuvvtulyMLfHYQ2BgTlA4dOsRzzz3HzJkzAWjcuDGJiYlER//mgleTD8sAjDFBRVV577338Hg8zJw5k9/97ne8/vrrbNiwwTr/ArIMwBgTNHbv3k23bt1Yvnw5ALfffjvx8fHUr1/f5ciCk2UAxpiAd/bsWd566y0iIyNZvnw5VapUITExkRUrVljnfxksAzDGBLSMjAxiYmJYv349AI888ghvv/02NWvWdDmy4GcZgDEmIJ08eZKXX36ZZs2asX79emrXrs3ixYuZN2+edf6FxDIAY0zAWbVqFXFxcXz55ZcAdO/enddff53KlSu7HFnJYgOAMSZg/PDDDwwYMID4+HgAPB4PCQkJtG7d2uXISiZ/ngkcJiIrRWSbiGwVkd5O+RyfR0Tuch4BiYjcJSJpIpLhvLbLp94hIrI3j8dMGmNC0IIFC4iIiCA+Pp4yZcowePBgNm7caJ1/EfInAzgD9FPVdBGpBKSJSLKq/vXcBiIyCvjReXsIeFBV94lIFPARUDufuseo6sjLiN8YE+T27dtHz549WbhwIQCtWrUiISGByMhIlyMr+S6aAajqflVNd5aPAVn4dOjivcPSozgPh1fVjaq6z1m9FSgvIuUKO3BjTHDLzc1l8uTJeDweFi5cSMWKFRk/fjxr1qyxzr+YFOgsIBEJB5oCKT7FtwIHVHV7Hrv8GUhX1VP5VNlTRLaIyFQRuTqfz4wTkVQRSc3Ozi5IuMaYAPXFF19w++23061bN44ePcoDDzzAtm3b6N69O6VK2cmJxcXvf2kRqQjMB/qo6lGfVY/hfPs/b/tI4A3g6XyqnAjcCDQB9gOj8tpIVeNVNVpVo6tXr+5vuMaYAHT69GmGDh1K48aNWb16NTVq1GDOnDksWbKEsLAwt8MLOX6dBSQiZfB2/kmqusCn/ArgT8DN521/PbAQeEJVd+ZVp6oe8Nk+AVha4OiNMUEjJSWFmJgYMjMzAejatStvvvkmVatWdTmy0OXPWUACTAGyVHX0eavvBL5Q1T0+21cBPgReVNXPL1BvLZ+3HYDMggRujAkOx48fp3fv3rRq1YrMzExuvPFGPv74Y6ZMmWKdv8v8mQJqDXQB2uVxymZHfjv90xP4PfCKz/Y1AEQkUUTO3a5vhHOq6BbgDqDvZbfGGBNQli1bRmRkJOPGjaNUqVIMGDCALVu20L59e7dDM/gxBaSqa4A8n6Wmqk/lUTYUGJrP9jE+y138jtIYE1QOHjxInz59mDXL+/2wWbNmJCYm0rRpU5cjM77scLsxptCoKu+++y4ej4dZs2ZRvnx5Ro4cSUpKinX+AchuBWGMKRRff/01Tz/9NMnJyQDceeedTJ48mXr16rkcmcmPZQDGmMty5swZRo0aRVRUFMnJyVStWpUZM2bwr3/9yzr/AGcZgDHmkm3atImYmBjS0tIA6NixI2+99RY1atRwOTLjD8sAjDEFduLECV588UWio6NJS0sjLCyMDz/8kFmzZlnnH0QsAzDGFMiKFSuIi4tj586diAjPPvssQ4cOpVKlSm6HZgrIBgBjjF++//57+vfvz9SpUwGIjIwkMTGRli1buhyZuVQ2BWSMuSBVZe7cuXg8HqZOnUrZsmV59dVXSU9Pt84/yFkGYIzJ17fffkuPHj344IMPALj11luJj4/npptucjkyUxgsAzDG/EZubi7jx48nIiKCDz74gMqVKzNp0iQ+/fRT6/xLEMsAjDG/sm3bNmJjY1m7di0AHTp04O2336Z27fwe7GeClWUAxhjAe0HXkCFDaNKkCWvXrqVWrVrMnz+fBQsWWOdfQlkGYIxh7dq1TJo0iUOHDgEQFxfHG2+8QZUqVVyOzBQlywCMCWFHjx6lR48etGnThkOHDtGgQQNWrVrF5MmTrfMPATYAGBOilixZQkREBBMmTKB06dIMGjSIzZs307ZtW7dDM8XEpoCMCTHfffcdzz77LPPmzQOgRYsWJCQk0KhRI5cjM8XNMgBjQoSqMnXqVDweD/PmzePKK69kzJgxrF271jr/EOXPM4HDRGSliGwTka0i0tspn+PzyMddIrLJZ5+XRGSHiPxHRO7Jp94bRCTF2W6OiJQtvGYZY3zt2LGDO++8k7/97W/88MMP3HfffWzdupU+ffpQunRpt8MzLvEnAzgD9FPVCKAl0ENEIlT1r6raRFWbAPOBBQAiEoH3WcGRwL3ABBHJ6y/sDWCMqv4eOAL87fKbY4zx9fPPP/PGG2/QsGFDVqxYQbVq1UhKSuLDDz+kbt26bodnXHbRAUBV96tqurN8DMgC/u+kYBER4FF+eTj8Q8BsVT2lql8DO4AWvnU6+7QD3neKZgAPX15TjDG+0tLSaNGiBS+++CInT56kS5cuZGVl8fjjj+P9L2hCXYGOAYhIONAUSPEpvhU4oKrbnfe1gW991u/BZ8BwXAP8oKpnLrDNuc+ME5FUEUnNzs4uSLjGhKSffvqJ559/nhYtWrBp0ybCw8P56KOPePfdd6lWrZrb4ZkA4vcAICIV8U719FHVoz6rHuOXb/+FTlXjVTVaVaOrV69eVB9jTImQnJxMw4YNGTVqFAD9+vUjMzOTu+++2+XITCDy6zRQESmDt/NPUtUFPuVXAH8CbvbZfC8Q5vP+eqfM12Ggiohc4WQBeW1jjPHToUOHeO6555g5cyYAjRs3JjExkejoaJcjM4HMn7OABJgCZKnq6PNW3wl8oap7fMqWAB1FpJyI3ADUB9b77qSqCqwEHnGKngQWX1oTjAldqsp7772Hx+Nh5syZlCtXjtdff50NGzZY528uyp8poNZAF6Cdz2mff3TWdeS86R9V3QrMBbYBy4EeqnoWQESWich1zqYDgOdEZAfeYwJTLrs1xoSQ3bt3c//999OpUycOHTrE7bffTkZGBi+++CJlypRxOzwTBMT7ZTw4REdHa2pqqtthGOOqs2fP8s477zBo0CB++uknqlSpwsiRI+nataud3WPyJCJpqvqblNBuBWFMEMnIyCAmJob1672zqn/5y18YN24cNWvWdDkyE4zsVhDGBIGTJ0/y8ssv06xZM9avX0/t2rVZvHgxc+fOtc7fXDLLAIwJcJ999hmxsbF8+eWXAHTv3p3XX3+dypUruxyZCXY2ABgToH788UcGDBjA5MmTAfB4PCQkJNC6dWuXIzMlhU0BGROAFi5ciMfjYfLkyZQpU4bBgwezceNG6/xNobIMwJgAsm/fPnr16sWCBd7rLVu1akVCQgKRkZEuR2ZKIssAjAkAubm5xMfHExERwYIFC6hYsSLvvPMOa9assc7fFBnLAIxx2X/+8x9iY2NZvXo1AA888AATJkwgLCzsInsac3ksAzDGJadPn2bYsGE0btyY1atXU6NGDWbPns2SJUus8zfFwjIAY1yQkpJCTEwMmZmZADz11FOMGjWKqlWruhyZCSWWARhTjI4fP06fPn1o1aoVmZmZ1KtXj48//php06ZZ52+KnQ0AxhST5cuXExkZyVtvvUWpUqV44YUXyMjIoH379m6HZkKUDQDGPUlJEB4OpUp5X5OS3I6oSGRnZ9O5c2fuu+8+vvnmG5o1a8aGDRt44403qFChgtvhmRBmA4BxR1ISxMXB7t2g6n2NiytRg4CqMnPmTDweD0lJSZQvX54RI0aQkpJC06ZN3Q7PGBsAjEsGDYKcnF+X5eR4y0uAr7/+mnvuuYcnnniCw4cPc+edd5KZmUn//v254go798IEBhsAjDu++aZg5UHizJkzjB49mqioKJKTk6latSrTp0/nX//6F/Xq1XM7PGN+xb6KGHfUqeOd9smrPEht2rSJmJgY0tLSAHjssccYO3YsNWrUcDkyY/LmzzOBw0RkpYhsE5GtItLbZ10vEfnCKR/hlHXyeXTkJhHJFZEmedQ7RET25vGYSRMKhg2D8w+AVqjgLQ8yJ06c4KWXXiI6Opq0tDTCwsJYunQp7733nnX+JqD5kwGcAfqparqIVALSRCQZuBZ4CGisqqdEpAaAqiYBSQAi0hBYpKqb8ql7jKqOvOxWmODTqZP3ddAgdPdupG5db+d/rjxIrFy5kri4OHbs2IGI8OyzzzJ06FAqVarkdmjGXNRFMwBV3a+q6c7yMSALqA08AwxX1VPOuoN57P4YMLvwwjUlSqdOsGsX/xgyBHbtCqrO/8iRI8TGxtKuXTt27NhBeHg4a9eu5a233rLO3wSNAh0EFpFwoCmQAjQAbhWRFBFZJSLN89jlr8CsC1TZU0S2iMhUEbk6n8+ME5FUEUnNzs4uSLjGFDpVZd68eXg8HhITEylbtiyvvvoqnTt3pmXLlm6HZ0yB+D0AiEhFYD7QR1WP4p0+qgq0BPoDc0VEfLa/BchR1cx8qpwI3Ag0AfYDo/LaSFXjVTVaVaOrV6/ub7jGFLo9e/bw8MMP8+ijj3LgwAHatGnD5s2befnll+3UThOU/BoARKQM3s4/SVUXOMV7gAXqtR7IBar57NaRC3z7V9UDqnpWVXOBBKDFpTTAmKKWm5vLhAkTiIiIYMmSJVSuXJlJkyaxatUqbrrpJrfDM+aSXfRri/OtfgqQpaqjfVYtAu4AVopIA6AscMjZpxTwKHDrBeqtpar7nbcdgPwyBWNcs23bNmJjY1m7di0ADz/8MO+88w61a9d2OTJjLp8/GUBroAvQ7rxTNqcC9UQkE++B3idVVZ192gLfqupXvhWJSKKIRDtvR4hIhohswTuQ9C2MBhlTGE6dOsXf//53mjRpwtq1a6lZsybz589n4cKF1vmbEuOiGYCqrgEkn9Wd89nnU7zHBs4vj/FZ7uJfiMYUr7Vr1xITE0NWVhYAMTExvPnmm1SpUsXlyIwpXHYrCGMcR48epWfPnrRp04asrCzq16/Pp59+SkJCgnX+pkSyAcAY4IMPPiAyMpLx48dTunRpBg4cyJYtW7jtttvcDs2YImPnrpmQduDAAZ599lnmzp0LQPPmzUlMTKRRo0YuR2ZM0bMMwIQkVWXatGl4PB7mzp3LlVdeyZgxY1i3bp11/iZkWAZgQs6OHTt4+umnWbFiBQD33nsvEydOJDw83N3AjClmlgGYkHHmzBlGjBhBw4YNWbFiBdWqVSMpKYlly5ZZ529CkmUAJiSkp6cTExPDxo0bAejSpQujR4+mWrVqF9nTmJLLMgBTouXk5NC/f3+aN2/Oxo0bCQ8PZ/ny5bz77rvW+ZuQZwOAKbGSk5OJiopi5EjvIyeee+45MjMzueeee1yOzJjAYFNApsQ5fPgw/fr1Y8aMGQA0btyYhIQEmjfP647lxoQuywBMiaGqzJo1C4/Hw4wZMyhXrhyvv/46GzZssM7fmDxYBmBKhG+++YZnnnmGZcuWAXDbbbeRkJBA/fr1XY7MmMBlGYAJamfPnuXtt98mIiKCZcuWUaVKFRITE1m5cqV1/sZchGUAJmhlZmYSExNDSkoKAH/5y18YN24cNWvWdDkyY4KDZQAm6Jw8eZJXXnmFpk2bkpKSwnXXXceiRYuYO3eudf7GFIBlACaorF69mtjYWP7zn/8A0K1bN4YPH85VV13lcmTGBB8bAExQ+PHHHxkwYACTJ08G4KabbiIhIYE2bdq4HJkxweuiU0AiEiYiK0Vkm4hsFZHePut6icgXTvkIpyxcRE74PD5yUj71VhWRZBHZ7rxeXXjNMiXJokWLiIiIYPLkyZQpU4ZXXnmFTZs2WedvzGXyJwM4A/RT1XQRqQSkiUgycC3wENBYVU+JSA2ffXaqapOL1Psi8ImqDheRF533Ay6hDaaE2rdvH7169WLBggUAtGzZksTERCIjI12OzJiS4aIZgKruV9V0Z/kYkAXUBp4BhqvqKWfdwQJ+9kPADGd5BvBwAfc3JVRubi4JCQlERESwYMECKlasyLhx41izZo11/sYUogKdBSQi4UBTIAVoANwqIikiskpEfC+1vEFENjrlt+ZT3bWqut9Z/g5vRpHXZ8aJSKqIpGZnZxckXBOEvvzyS9q1a0dcXBw//vgjDzzwANu2baNXr16ULl3a7fCMKVH8HgBEpCIwH+ijqkfxTh9VBVoC/YG5IiLAfqCOqjYFngPeE5HKF6pbVRXQfNbFq2q0qkZXr17d33BNkPn555957bXXaNSoEatWraJGjRrMnj2bJUuWEBYW5nZ4xpRIfp0FJCJl8Hb+Saq6wCneAyxwOu/1IpILVFPVbODctFCaiOzEmy2knlftARGppar7RaQWUNApJFNC7Nmzh5tvvpmMjAwA/ud//oeRI0dStWpVlyMzpmTz5ywgAaYAWao62mfVIuAOZ5sGQFngkIhUF5HSTnk9oD7wVR5VLwGedJafBBZfaiNMcDp+/Dh9+/ZlypQpZGRkUK9ePT7++GOmTp1qnb8xxcCfKaDWQBegnc+pnX8EpgL1RCQTmA086WQDbYEtIrIJeB/opqrfA4hIoohEO/UOB+4Ske3Anc57EyL++c9/EhUVxdixYxERXnjhBTIyMmjfvr3boRkTMi46BaSqawDJZ3XnPLafj3e6KK+6YnyWDwP2vz3EZGdn06dPH9577z0AmjZtSmJiIs2aNXM5MmNCj90LyBQLVWXmzJl4PB7ee+89ypcvz4gRI1i/fr11/sa4xAYAU+S+/vpr7rnnHp544gkOHz5M+/btycjIoH///lxxRZDfjSQpCcLDeWXIEAgP9743JkjYAGCKzJkzZxg9ejRRUVEkJydz9dVXM23aNJKTk7nxxhvdDu/yJSVBXBzs3u2dI9292/veBgETJGwAMEVi8+bNtGrVin79+pGTk0PHjh3JysriqaeewntiWQkwaBDk5Py6LCfHW25MELABwBSqEydO8NJLL3HzzTeTmppKWFgYS5cuZdasWVx7bZ4Xewevb74pWLkxAcYGAFNoVq5cSaNGjRg+fDi5ubn06tWLrVu3cv/997sdWtGoU6dg5cYEGBsAzGU7cuQIMTExtGvXjh07dhAZGcnatWsZN24clSpVcju8ojNsGFSo8OuyChW85cYEARsAzCVTVd5//308Hg9TpkyhbNmyvPrqq6Snp9OyZUu3wyt6nTpBfDzUrQsi3tf4eG+5MUEgyM/BM27Zs2cPPXr0YMmSJQC0adOGhIQEbrrpJpcjK2adOlmHb4KWZQCmQHJzc5k4cSIREREsWbKESpUqMWHCBFatWhV6nb8xQc4yAOO3rKwsYmNj+fzzzwF46KGHGD9+PLVr13Y5MmPMpbAMwFzU6dOn+cc//kGTJk34/PPPqVmzJu+//z4LFy60zt+YIGYDQLBybkFAqVJFeguCdevW0axZMwYPHszp06eJjY1l27Zt/PnPfy45F3QZE6JsCigYnbsFwbmrUM/dggAK7YDksWPHGDhwIOPHj0dVqV+/PvHx8dx+++2FUr8xxn2WAQSjIr4FwdKlS4mIiOCdd96hdOnSDBw4kM2bN1vnb0wJYxlAMCqiWxAcOHCAPn36MHv2bACaN29OQkICjRs3vqx6jTGByTKAYFTItyBQVaZNm4bH42H27NlUqFCBMWPGsG7dOuv8jSnB/HkmcJiIrBSRbSKyVUR6+6zrJSJfOOUjnLK7RCRNRDKc13b51DtERPae95hJ449CvAXBzp07ueuuu+jatStHjhzhnnvuITMzkz59+lC6dOlCCtgYE4j8mQI6A/RT1XQRqQSkiUgycC3wENBYVU+JSA1n+0PAg6q6T0SigI+A/M4VHKOqIy+zDaHn3IHeQWFWXRwAAA/bSURBVIPQ3buRunW9nX8BDgCfu1f/4MGDOXnyJNdccw1jx46lU6dOdnaPMSHiohmAqu5X1XRn+RiQhbdDfwYYrqqnnHUHndeNqrrP2X0rUF5EyhVF8CGtUyfYtYt/DBkCu3YVqPNPT0+nRYsWDBgwgJMnT9K5c2eysrLo3Lmzdf7GhJACHQMQkXCgKZACNABuFZEUEVklIs3z2OXPQPq5QSIPPUVki4hMFZGr8/nMOBFJFZHU7OzsgoRrzpOTk0P//v1p3rw5GzdupG7duixfvpyZM2dSvXp1t8MzxhQzvwcAEakIzAf6qOpRvNNHVYGWQH9grvh8fRSRSOAN4Ol8qpwI3Ag0AfYDo/LaSFXjVTVaVaOtk7p0H3/8MQ0bNmTkSO+MW58+fcjMzOSee+5xOTJjjFv8Og1URMrg7fyTVHWBU7wHWKCqCqwXkVygGpAtItcDC4EnVHVnXnWq6gGf+hOApZfeDJOfw4cP8/zzzzN9+nQAGjZsSGJiIi1atHA3MGOM6/w5C0iAKUCWqo72WbUIuMPZpgFQFjgkIlWAD4EXVfXzC9Rby+dtByCz4OGb/Kgqs2fPJiIigunTp1OuXDlee+010tLSrPM3xgD+ZQCtgS5AhohscsoGAlOBqSKSCZwGnlRVFZGewO+BV0TkFWf7u1X1oIgkApNUNRUYISJNAAV2kf9UkSmgb775hu7du/Phhx8CcNtttxEfH0+DBg1cjswYE0guOgCo6hogv1NDOuex/VBgaD51xfgsd/EzRuOns2fPMmHCBAYOHMjx48e56qqrGDlyJF27dqVUKbvmzxjza3YriBIiMzOTmJgYUlJSAHjkkUcYN24ctWrVusiexphQZV8Lg9zPP//M//t//4+mTZuSkpLCddddx8KFC5k3b551/saYC7IMIIitXr2aSZMmcfjwYQC6devG8OHDueqqq1yOzBgTDCwDCEI//vgj3bp1o23bthw+fJibbrqJzz77jIkTJ1rnb4zxm2UAQWbRokX06NGDffv2UaZMGe677z7mzp1LuXJ2tw1jTMFYBhAk9u/fzyOPPEKHDh3Yt28fLVu2JD09nWbNmlnnb4y5JDYABLjc3FwSEhLweDzMnz+fihUr8vbbb7NmzRqioqLcDs8YE8RsCiiAffnll8TFxbFq1SoA7r//fiZMmECdS3zwizHG+LIMIAD9/PPPvPbaazRq1IhVq1ZRvXp1Zs+ezQcffGCdvzGm0FgGEGDWr19PbGwsW7ZsAeCpp55i5MiRXHPNNS5HZowpaSwDCBDHjx+nb9++tGrVii1btnDDDTeQnJzMtGnTrPM3xhQJGwACwPLly4mKimLs2LEA9O/fn8zMTO68806XIzPGlGQ2BeSi7Oxs+vbtS1JSEgBNmzYlMTGRZs2auRyZMSYUWAbgAlXlf//3f/F4PCQlJfG73/2OESNGsH79euv8jTHFxjKAYrZr1y66devGRx99BED79u2ZPHkyN954o8uRGWNCjWUAxeTs2bOMGTOGyMhIPvroI66++mqmTZtGcnKydf7GGFdYBlAMNm/eTGxsLBs2bACgY8eOjB07lmuvvdblyIwxocyfZwKHichKEdkmIltFpLfPul4i8oVTPsKn/CUR2SEi/xGRe/Kp9wYRSXG2myMiZQunSYHjxIkTDBw4kOjoaDZs2MD111/PBx98wKxZs6zzN8a4zp8M4AzQT1XTRaQSkCYiycC1wENAY1U9JSI1AEQkAugIRALXAR+LSANVPXtevW8AY1R1tohMAv4GTCycZrnv008/JS4uju3btyMi9OzZk9dee41KlSq5HZoxxgB+ZACqul9V053lY0AWUBt4BhiuqqecdQedXR4CZqvqKVX9GtgBtPCtU0QEaAe87xTNAB6+/Oa478iRI8TGxnLHHXewfft2IiIi+Pzzz3n77bet8zfGBJQCHQQWkXCgKZACNABudaZxVolIc2ez2sC3Prvtccp8XQP8oKpnLrDNuc+ME5FUEUnNzs4uSLjFSlV5//338Xg8JCYmUrZsWf7+97+zceNGWrVq5XZ4xhjzG34fBBaRisB8oI+qHhWRK4CqQEugOTBXROoVdoCqGg/EA0RHR2th118Y9u7dS48ePVi8eDEArVu3/r9bOBtjTKDyKwMQkTJ4O/8kVV3gFO8BFqjXeiAXqAbsBcJ8dr/eKfN1GKjiDCL5bRPwcnNzmThxIh6Ph8WLF1OpUiUmTJjAZ599Zp2/MSbg+XMWkABTgCxVHe2zahFwh7NNA6AscAhYAnQUkXIicgNQH1jvW6eqKrASeMQpehJYfHlNKV5ZWVm0bduW7t27c+zYMf77v/+bbdu28cwzz1CqlF1eYYwJfP70VK2BLkA7Ednk/PwRmArUE5FMYDbwpJMNbAXmAtuA5UCPc2cAicgyEbnOqXcA8JyI7MB7TGBKobasiJw+fZpXX32VJk2a8Pnnn1OzZk3mzZvHokWLuP76690Ozxhj/HbRYwCqugaQfFZ3zmefYcCwPMr/6LP8FeedHRTo1q1bR2xsLFu3bgUgJiaGESNGcPXVV7scmTHGFJzNVfjh2LFj9OrVi9atW7N161bq16/PypUrSUhIsM7fGBO0bAC4iKVLlxIREcE777xD6dKlGThwIJs3b+b22293OzRjjLksNgDk48CBA3Ts2JEHH3yQPXv2EB0dTWpqKsOGDaN8+fJuhwdJSRAezitDhkB4uPe9McYUgA0A51FVpk2bhsfjYc6cOVSoUIHRo0fz73//m8aNG7sdnldSEsTFwe7d3oMzu3d739sgYIwpABsAfOzcuZO77rqLrl27cuTIEe6++24yMzPp27cvpUuXdju8XwwaBDk5vy7LyfGWG2OMn2wAAM6cOcOIESOIiorik08+4ZprrmHmzJksX76cG264we3wfuubbwpWbowxeQj55wGkp6cTExPDxo0bAejcuTOjR4+mevXqLkd2AXXqeKd98io3xhg/hWwGkJOTwwsvvECLFi3YuHEjdevW5Z///CczZ84M7M4fYNgwqFDh12UVKnjLjTHGTyE5AHz88cc0bNiQN998E1Wlb9++ZGZmcu+997odmn86dYL4eKhbF0S8r/Hx3nJjjPFTSE0BHT58mOeff57p06cDUK9ePWbPnk3z5s0vvGMg6tTJOnxjzGUJiQFAVZkzZw69e/fm4MGDlCtXjsGDB3PixIng7PyNMaYQlPgpoGPHjvHggw/y2GOPcfDgQW677Ta2bNnCSy+9FFindhpjTDEr8QNAxYoVOXHiBFdddRUJCQmsWLGCBhs22FW0xpiQV+KngESEadOmUaZMGWrVqvXLVbQ5Ob++ihZsTt0YE1LE+2yW4BAdHa2pqamXV0l4eN7n0NetC7t2XV7dxhgTgEQkTVWjzy8v8VNAv2FX0RpjDBCKA0B+V8vaVbTGmBDjzzOBw0RkpYhsE5GtItLbKR8iInvPe0wkItLJp2yTiOSKSJM86s1z/yJnV9EaYwzg30HgM0A/VU0XkUpAmogkO+vGqOpI341VNQlIAhCRhsAiVd2UT92/2b/InTvQO2iQd9qnTh1v528HgI0xIcafZwLvB/Y7y8dEJAuo7Wf9j+F9YHxgsatojTGmYMcARCQcaAqkOEU9RWSLiEwVkbwejvtXYNYFqrzY/ohInIikikhqdnZ2QcI1xhhzAX4PACJSEZgP9FHVo8BE4EagCd4MYdR5298C5KhqZj5VXnD/c1Q1XlWjVTU64O/SaYwxQcSvAUBEyuDt/JNUdQGAqh5Q1bOqmgskAC3O260jF/j278f+xhhjipA/ZwEJMAXIUtXRPuW1fDbrAGT6rCsFPMoF5v8vtL8xxpii589ZQK2BLkCGiJw7m2cg8JhzeqcCu4CnffZpC3yrql/5ViQiicAkVU0FRlxgf2OMMUUsqG4FISLZQB73cbhk1YBDhVifG6wNgcHaEDhKQjsKuw11VfU3B1GDagAobCKSmtf9MYKJtSEwWBsCR0loR3G1IfRuBWGMMQawAcAYY0JWqA8A8W4HUAisDYHB2hA4SkI7iqUNIX0MwBhjQlmoZwDGGBOybAAwxpgQFTIDgIj0FpFM55kGfZyyxiKyTkQyROQDEansdpznc26Ud1BEfK+0rioiySKy3Xm92ikXERknIjucm+w1cy/yXxSwDTc5v5NTIvK8e1H/WgHb0Mn5988QkbUi0ti9yH9RwDY85LRhk3MzxjbuRf6LgrTBZ31zETkjIo8Uf8S/VcDfw+0i8qPPc1NeKcxYQmIAEJEoIBbv/YYaAw+IyO+BROBFVW0ILAT6uxdlvqYD955X9iLwiarWBz5x3gPcB9R3fuLw3nAvEEzH/zZ8DzwLFO9zIi5uOv634WvgNufv6lUC56DkdPxvwydAY1VtAnTF+38lEEzH/zYgIqWBN4B/FVeAfphOAdoArFbVJs7PPwozkJAYAAAPkKKqOap6BlgF/AloAHzmbJMM/Nml+PKlqp/h7RR9PQTMcJZnAA/7lL+rXv8Gqpx3zyVXFKQNqnpQVTcAPxdfhBdXwDasVdUjTvm/geuLJciLKGAbjusvZ4hcifeWLa4r4P8HgF54b2R5sOij888ltKHIhMoAkAncKiLXiEgF4I9AGLAV7z88wF+csmBwrfOgHoDvgGud5drAtz7b7cH/h/cUt/zaEEz8acPfgH8WX0gFlm8bRKSDiHwBfIg3CwhUebZBRGrjvdFkoGTCF3Khv6VWIrJZRP4pIpGF+aEhMQCoaha/pIHLgU3AWbx/1N1FJA2oBJx2LchL5HxLC4hvZ5eqpLZBRO7AOwAMcCWoAjq/Daq6UFVvwvtt9FXXAiuA89owFhjg3HI+aJzXhnS89/FpDLwNLCrMzwqJAQBAVaeo6s2q2hY4Anypql+o6t2qejPeZxfsdDdKvx04N7XjvJ5Lb/fy6yzmeqcsEOXXhmCSbxtEpBHeefOHVPWwS/H546K/B2fKop6IVCvu4PyUXxuigdkisgt4BJggIsUytXIJ8myDqh5V1ePO8jKgTGH+HkJmABCRGs5rHbzz/+/5lJUCXgYmuRdhgSwBnnSWnwQW+5Q/4ZwN1BL40SetDDT5tSGY5NkG529sAdBFVb90KTZ/5deG34uIOMvNgHJAoA5kebZBVW9Q1XBVDQfeB7qraqF+gy5E+f0eavr8Hlrg7bML7/egqiHxA6wGtgGbgfZOWW/gS+dnOM6V0YH0gzcz2Y/3oOgevFMK1+A9U2A78DFQ1dlWgPF4M5kMINrt+C+hDTWdbY4CPzjLlYOsDYl4s8xNzk+q2/FfQhsG4D1GtglYB7RxO/6CtuG8/aYDj7gd/yX8Hno6v4fNeE8o+ENhxmK3gjDGmBAVMlNAxhhjfs0GAGOMCVE2ABhjTIiyAcAYY0KUDQDGGBOibAAwxpgQZQOAMcaEqP8PGb1QXqqjQGgAAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "$Y_i = A + Bx_i + \\varepsilon_i$\n",
        "\n",
        "For estimating regression parameters, one way is to minimize the sum of the squared differences between the estimated responses and the actual response values (black vertical lines shown above is our errors which we want to minimize).\n",
        "\n",
        "$SSE =  \\sum_{i=1}^n (y_i - \\widehat{A} -  \\widehat{B}\\ x_i)^2 $ \n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "To determine these estimators, we differentiate Q first with respect to A and then to B as follows:\n",
        "\n",
        "$\\frac{\\partial SSE}{\\partial \\widehat{A}} = 0 \\rightarrow -2\\ \\sum_{i=1}^n (y_i - \\widehat{A} -  \\widehat{B}\\ x_i) = 0$\n",
        "\n",
        "$\\frac{\\partial SSE}{\\partial \\widehat{B}} = 0 \\rightarrow -2\\ \\sum_{i=1}^n x_i\\ (y_i - \\widehat{A} -  \\widehat{B}\\ x_i) = 0$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "The results are:\n",
        "\n",
        "$\\widehat{B} = b = \\frac{S_{xy}}{S_{xx}}$\n",
        "\n",
        "$\\widehat{A} = a = \\overline{y} - B\\ \\overline{x}$\n",
        "\n",
        "$S_{xx} = \\sum_{i=1}^n (x_i - \\overline{x})^2 = \\sum_{i=1}^n x_i^2 - n \\overline{x}^2 = \\sum_{i=1}^n (x_i - \\overline{x})x_i$\n",
        "\n",
        "$S_{yy} = \\sum_{i=1}^n (y_i - \\overline{y})^2 = \\sum_{i=1}^n y_i^2 - n \\overline{y}^2 = \\sum_{i=1}^n (y_i - \\overline{y})y_i$\n",
        "\n",
        "$S_{xy} = \\sum_{i=1}^n (x_i - \\overline{x})(y_i - \\overline{y}) = \\sum_{i=1}^n x_i y_i - n \\overline{x}\\ \\overline{y} = \\sum_{i=1}^n (x_i - \\overline{x})y_i$"
      ],
      "metadata": {
        "id": "HYNWxPRBKZwn"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x = [100,102,103,101,105,100,99,105]\n",
        "y = [259,264,274,268,277,263,258,275]\n",
        "\n",
        "xx = []\n",
        "for i in x:\n",
        "  xx.append((i - np.mean(x))**2)\n",
        "S_xx = np.sum(xx)\n",
        "\n",
        "yy = []\n",
        "for i in y:\n",
        "  yy.append((i - np.mean(y))**2)\n",
        "S_yy = np.sum(yy)\n",
        "\n",
        "xy = []\n",
        "for i in range(len(x)):\n",
        "  xy.append((x[i] - np.mean(x))* (y[i] - np.mean(y)))\n",
        "S_xy = np.sum(xy)\n",
        "\n",
        "B_hat = (S_xy) / (S_xx)\n",
        "A_hat = np.mean(y) - B_hat *  np.mean(x)\n",
        "print(f'A_hat : {A_hat}, B_hat : {B_hat}')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BNbOL4cY8YuD",
        "outputId": "919f5839-222e-4e7f-cbd4-bb3b7e9b706a"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "A_hat : -40.10169491525426, B_hat : 3.016949152542373\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "x = [100,102,103,101,105,100,99,105]\n",
        "y = [259,264,274,268,277,263,258,275]\n",
        "  \n",
        "s = np.arange(np.min(x)-0.5, np.max(x)+0.5, 0.001)\n",
        "\n",
        "plt.xlabel('X')\n",
        "plt.ylabel('Y')\n",
        "plt.title('Regression Line')\n",
        "plt.scatter(x, y, color ='blue', marker='*')\n",
        "plt.plot(s, A_hat + B_hat*s, color ='red');"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "SA5r74OjU91X",
        "outputId": "4c985752-3c69-4536-a26d-6068216ffea7"
      },
      "execution_count": 42,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3debyOdf7H8den0jS2hsiULGkbW8gSRdO0TMu0TVONtJhfSlq0q2mP9l2bJKQpJWWPFiGRnQhHC1KUSiWS4nA+vz++V8dN53AO5z7Xvbyfj8f9uO/7uu5z32938jnf67uZuyMiIgKwU9wBREQkdagoiIhIPhUFERHJp6IgIiL5VBRERCSfioKIiORTURBJEjNrY2Yfl/Jn3mRmvUvzMyWzmOYpSCozsyVANWAjsAZ4E7jc3dfEmStOZlYb+Awo4+4b4k0jmUYtBUkHJ7t7eaAx0AS4saQ/wMx2Ken3FElHKgqSNtz9a+AtQnEAwMxamtkkM/vRzOaY2ZEJ5/Y1s/fM7Ccze8fMnjKzF6Nztc3MzayDmX0BjI2OX2BmC8xspZm9ZWa1ouNmZo+a2bdmttrM5ppZg+jciWaWE33Ol2Z2XXT8SDNblpCnrpm9G2Wdb2anJJzrF+UbGb3PVDPbr7jfkZndUcCfsb2ZfWFm35nZzQmv3cnM/mtmi8zsezMbaGaVi/uZkllUFCRtmNk+wAnAwuh5dWAkcBdQGbgOGGRmVaMfeQmYBuwB3AGcV8Db/hWoCxxnZqcCNwGnA1WBCcDL0ev+DhwBHAjsDpwFfB+d6wNc7O4VgAZEBWaL7GWAEcDbwJ5AZ6C/mR2U8LK2QFegUvRnvLtIX8y2tQYOAo4GbjOzutHxzsBphO9gb2Al8FQJfaakKRUFSQdDzewnYCnwLXB7dPxcYJS7j3L3PHcfDcwATjSzmkBz4DZ3X+/uE4HhBbz3He7+s7v/AnQC7nX3BdG1+nuAxlFrIReoAPyF0Be3wN2XR++RC9Qzs4ruvtLdZxXwOS2B8sB9UZ6xwOvA2QmvGeLu06LP7k9Ci2gHdXX3X9x9DjAHaBQd7wTc7O7L3H0doXCeoUtp2U1FQdLBadFv4UcS/lGuEh2vBZwZXY750cx+JPxWvBfhN98f3H1twvssLeC9E4/VAh5LeK8fAAOqR/+IP0n4TfpbM+tlZhWjn/sXcCLwuZmNN7NWBXzO3sBSd89LOPY5UD3h+dcJj9cSikhJKOx9awFDEv68Cwgd+tVK6HMlDakoSNpw9/FAP+Ch6NBS4AV3/1PCrZy73wcsByqbWdmEt6hR0NsmPF5KuAyU+H5/dPdJ0ec/7u5NgXqEy0hdouPT3f1UwmWhocDAAj7nK6CGmSX+P1cT+LJYX0LJWgqcsMWfdzd3jzOTxExFQdJNd+BYM2sEvAicbGbHmdnOZrZb1Lm7j7t/TriUdIeZ7Rr99n7yNt67J3CjmdUHMLPdzezM6HFzMzs06hv4GfgVyIve+xwz293dc4HVQF4B7z2V8Fv69WZWJuoQPxkYsAPfxR+iP/Nvt+L+/9wTuDuhM71q1K8iWUxFQdKKu68A/kfoK1gK/NY5vILwm28XNv29PgdoRegQvgt4BVi3lfceAtwPDDCz1cA8Qsc2QEXgWUJn7OfRez4YnTsPWBL9TKfoc7d87/WEInAC8B3QAzjf3T8q9pewyRrgl4TbUcX8+ccI/SxvR302U4BDdyCPZABNXpOsYWavAB+5++3bfLFIllJLQTJWdMlnv2g8/vGEVsXQuHOJpDINPZNM9mdgMGGewjLgEnf/IN5IIqlNl49ERCSfLh+JiEi+tL58VKVKFa9du3bcMURE0srMmTO/c/eqBZ1L66JQu3ZtZsyYEXcMEZG0YmafF3ZOl49ERCSfioKIiORTURARkXwqCiIikk9FQURE8qkoiIhIPhUFERHJp6IgIpJO3KFPHxgxIilvr6IgIpIuFi+GY46BCy+E/v2T8hEqCiIiqW7jRujeHRo2hOnToWdPeOmlpHxUWi9zISKS8ebPhw4dYOpU+Mc/QkHYZ5+kfZxaCiIiqWj9eujWDZo0gUWLQstgxIikFgRQS0FEJPVMnx5aB3Pnwtlnw2OPQdUCFzUtcWopiIikirVroUsXaNkSfvgBhg8PLYRSKgigloKISGp491246CJYuBA6doQHHoDddy/1GGopiIjEadUq6NQJ/va3MAdh7Fh45plYCgKoKIiIJM2qVVC/frgv0Ouvhxc8+yxcdx18+GEoDjFSURARSZKRIyEnB0aN2uLEihXQrh2cfDJUqgSTJ8ODD0LZsrHkTKSiICJSwtq1g/LloX378Pz888Pzdmc7vPwy1KsHr70GXbvCzJnQokW8gROoKIiIlLBu3aBmTShTJjwvUwZa7L2MZ785JVSMOnVg1iy47TbYddd4w25BRUFEpITtv38oDLm5UL5sHu3X9eKtZfUpN2UMPPIITJoEDRrEHbNAGpIqIpIEAwdCg90WMqzaRdRc9C5zKxxFw8nPhlZCClNREBEpaRs28PBe3dlnw63Yil1Z/fCzrGvTAepY3Mm2SUVBRKQkzZ0LHTpQY/p0OOUU6NGDitWr0yzuXEWkPgURkZKwbh3cfjsccggsWQIDBsDQoVC9etzJikUtBRGRHTV1aljAbv58OPdcePRRqFIl7lTbRS0FEZHt9fPPcM010KpVmLY8ciS88ELaFgRQS0FEZPuMHRsWsFu8GC65BO67DypWjDvVDlNLQUSkOH78MRSDo4+GnXeG8eOhR4+MKAigoiAiUnTDhoUlKvr2heuvhzlz4Igj4k5VolQURES25dtvoW1bOO20sOHN1Klw//3wxz/GnazEqSiIiBTGHV58EerWhSFD4M47YcYMaJYusw6KTx3NIiIFWbo0bH4zalQYXdS7d7h0lOHUUhARSZSXB08/HQrAu+/CY4/BhAlZURBALQURkU0++QQuvDAUgWOOgV69YN99405VqpLWUjCzGmY2zsxyzGy+mV0ZHX/FzGZHtyVmNjs6XtvMfkk41zNZ2URENrNhAzzwADRqFNYu6tsX3n57hwvCNrfjTEHJbClsAK5191lmVgGYaWaj3f3fv73AzB4GEr+uRe7eOImZREQ2N2cOXHBB2PTmn/+Ep56CvfYqkbdO3I7z7LNL5C2TLmktBXdf7u6zosc/AQuA/JWhzMyAs4CXk5VBRKRQ69bBrbeGkUTLlsGrr8KgQSVSEArdjrPdDr910pVKR7OZ1QaaAFMTDrcBvnH3TxOO7WtmH5jZeDNrU8h7dTSzGWY2Y8WKFUnLLCIZbNIkaNIE7roLzjkHFiyAM84AK5n9DgrajrNWrTCiNdUlvSiYWXlgEHCVu69OOHU2m7cSlgM13b0JcA3wkpn9bt64u/dy92bu3qxq1arJjC4imWbNGrjySmjdOixm9+ab0K8fVK5coh+TuB1nuXLhvmtX2G+/Ev2YpEhqUTCzMoSC0N/dBycc3wU4HXjlt2Puvs7dv48ezwQWAQcmM5+IZJHRo6FhQ3j8cbjsMpg3D447LmkfN3BgKAhdu4b7V19N2keVqKR1NEd9Bn2ABe7+yBanjwE+cvdlCa+vCvzg7hvNrA5wALA4WflEJEusXAnXXgvPPQcHHRSGm7ZunfSP7dIFnngCqlULWywsXZr0jywRyRx9dDhwHjD3t2GnwE3uPgpoy+87mI8AuplZLpAHdHL3H5KYT0Qy3ZAhcOmlsGIF3Hgj3HYb7LZbqXx08+abHlerFm7pIGlFwd0nAgX22rj7fwo4NohwqUlEZMd8/TV07gyvvQaNG4cxoU2axJ0qLWiZC5Esl44TrArlDs8/H5akGDEC7rkHpk1TQSgGFQWRLJc4wSqtff45nHAC/Oc/oSjMnh0uGf02LlSKREVBJEul8wSrzeTlwZNPhubOxImhd/e99+Avf4k7WVpSURDJUuk8wSrfxx+Hnc86dw4jiubPh8svh530T9v20jcnkqXSeYIVublw771hAbucnDAB7Y03QlWTHaKiIJLF0nKC1QcfQIsWcNNNcPLJoSi0b19iS1RkO+2nIJLF0mqC1a+/hur14INhn+RBg+D00+NOlXFUFESyWNpMsJo4ETp0CJvg/N//wcMPQ6VKcafKSLp8JCKp66efQsdxmzawfn3Y+KZvXxWEJFJREJHU9NZb0KAB9OgBV1wRdkQ79ti4U2U8FQURSS3ffx86jo8/HsqWDZeOHnssTKKQpFNREJHU4B7WKqpXD156CW65JYw0OuywuJNlFXU0i0j8li8PexwMGQJNm4a+g0aN4k6VldRSEJH4uId9DurVC5PP7r8fpkxRQYiRWgoiEo/PPoOOHeGdd8Loot694UBtthg3tRREpHRt3Bi2xGzQILQKevSAd99VQUgRaimISOnJyYELL4TJk8My1z17hlX5JGWopSAiyZebC3fdFTa7+eQTePHFsJGDCkLKUUtBRJJr5ky44AL48EP497/DpaM994w7lRRCLQURSY5ffoEbbggrmq5YAUOHwoABKggpTi0FESl5770X+g4+/TTcP/gg/OlPcaeSIlBLQURKzurVcOml8Ne/woYNYbjps8+qIKQRFQURKRmjRoV9kp95Bq65Jixgd/TRcaeSYlJREJEd8913YYeef/wDKlaESZPCfgflysWdTLaDioKIbB93eOWVsETFK6/A7bfDrFlw6KFxJ5MdoI5mESm+r76CSy6B4cOhWTMYMwYaNow7lZQAtRREpOjcwxpF9eqFlUwfeijMTlZByBhqKYhI0SxaFBawGzs2jC7q3Rv23z/uVFLC1FIQka3buBEeeSS0BmbMCKOLxo5VQchQaimISOHmzYMOHWDaNDjpJHj6adhnn7hTSRKppSAiv7d+PXTtCoccAosXh+0xhw9XQcgCaimIyOamTw8L2M2bB+3aQffuULVq3KmklKilICLB2rVw3XXQsiWsXBlaBv37qyBkGbUURATGjYOLLgojjC6+OOyVvPvucaeSGKilIJLNVq0KReCoo8LzcePCbmgqCFlLRUEkW40YESah9e4dLht9+CEceWTcqSRmKgoi2WbFitCBfMopsMceMGVK2O+gbNm4k0kKUFEQyRbuYWhp3brw2mthyOmMGdC8edzJJIUkrSiYWQ0zG2dmOWY238yujI6/Ymazo9sSM5ud8DM3mtlCM/vYzI5LVjaRrLNsWWgZnHNOmIn8wQdw222w665xJ5MUk8zRRxuAa919lplVAGaa2Wh3//dvLzCzh4FV0eN6QFugPrA38I6ZHejuG5OYUSSz5eWFnc+6dAnLVTz6KHTuDDvvHHcySVFJaym4+3J3nxU9/glYAFT/7byZGXAW8HJ06FRggLuvc/fPgIVAi2TlE8l4n34aRhV16gQtWoSd0K66SgVBtqpU+hTMrDbQBJiacLgN8I27fxo9rw4sTTi/jIQikvBeHc1shpnNWLFiRXICi6SzDRvCktYHHwyzZ4fRRaNHQ506cSeTNJD0omBm5YFBwFXuvjrh1NlsaiUUmbv3cvdm7t6sqmZaimzuww+hVatwuei44yAnJyxoZxZ3MkkTSS0KZlaGUBD6u/vghOO7AKcDryS8/EugRsLzfaJjIrIt69aF7TCbNoXPPw/bYw4ZAnvvHXcySTPJHH1kQB9ggbs/ssXpY4CP3H1ZwrHhQFsz+4OZ7QscAExLVj6RjDFlSljNtFs3aNsWFiyAs85S60C2SzJbCocD5wFHJQxBPTE615YtLh25+3xgIJADvAlcppFHIlvx889w9dVw2GHw008wciS88EKYkCaynZI2JNXdJwIF/qri7v8p5PjdwN3JyiSSMcaMCQvYffYZXHop3HsvVKwYdyrJAJrRLJJOfvwRLrwQjjkGdtkFxo+Hp55SQZASo6Igki6GDQsL2PXrBzfcAHPmwBFHxJ1KMoz2UxBJdd98A1dcAQMHQqNGYXXTpk3jTiUZSi0FkVTlHjqO69WDoUPhrrvCVpkqCJJEaimIpKIvvgjLU7zxRpiM1qdPWN1UJMnUUhBJJXl50KMH1K8P770Hjz8OEyaoIEipUUtBJFV88kkYWTRhAhx7LPTqBbVrx51KsoxaCiJx27AB7r8/LGA3dy489xy89ZYKgsRCLQWROM2eHRasmzUL/vnPMOdgr73iTiVZTC0FkTj8+ivcfDM0awZffhm2xxw8WAVBYqeiIGlh1arQ97pqVdxJSsCkSdCkCdxzD5x7blje+l//ijuVCKCiIGli5Mjwb+eoUXEn2QFr1oRJaK1bw9q18OabYXZy5cpxJxPJp6IgKa1dOyhfHtq3D8/PPz88b9cu3lzF9vbb0KABPPkkXHYZzJsXNsERSTEqCpLSunWDmjWhTJnwvEwZqFUL7rwz3lxFtnIl/N//hQKw225h7sETT0CFCnEnEylQoUXBzEZFeyuLxGb//UNhyM2FcuXCfdeusN9+cScrgsGDwxIVL7wAN94YRhq1bh13KpGt2lpL4TngbTO7OdpWUyQWAweGgtC1a7h/9dW4E23D11/DGWeEzuO99grrFd1zT2gpiKQ4c/fCT5qVB24FjgdeAPJ+O1fAFpulrlmzZj5jxoy4Y0iSTZ8eLiFVqxYWDF26NIzkTDnu8PzzcM01oSP5jjvg2ms3XfsSSRFmNtPdC/y/aFuT19YDPwN/ACqQUBRESkvz5pseV6sWbilnyRK4+OLQody6NfTuDQcdFHcqkWIrtCiY2fHAI8Bw4BB3X1tqqUTSRV5emIV8441gFkYXXXIJ7KQxHJKettZSuBk4093nl1YYkbTy0UdhAbv33w+ji555JgyNEkljhf464+5tVBBECpCbGzqOGzUKM+qefz7se6CCIBlAC+KJFMesWWEBu9mz4cwzw5yDlOzkENk+uvApUhS//BL6DVq0CENOBw8OY2VVECTDqKUgsi0TJ4bWwSefwAUXwEMPQaVKcacSSQq1FEQK89NPcPnl0KYNrF8Po0eHvZJVECSDqSiIFOSNN8Ja3T16wJVXhh3Rjjkm7lQiSaeiIJLo++/DUqwnnhiWY33/fejePTwWyQIqCiIQlqh49dWwgN3LL8Ott8IHH0CrVnEnEylV6mgWWb4cLr0Uhg6Fpk3DUhWNGsWdSiQWailI9nKHvn2hbt2wC9oDD8CUKSoIktXUUpDstHhxWMDunXfgiCPg2WfhwAPjTiUSO7UUJLts3Bg6jhs2hKlT4emnYdw4FQSRiFoKkj1ycsIktClTwuiinj2hRo24U4mkFLUUJPOtXx82dW7SBD79FF58EV5/XQVBpABqKUhmmzEjtA4+/BDatoXHHoM994w7lUjKUktBMtMvv8D118Ohh8J338GwYWH+gQqCyFappSCZZ/z4sPnNwoVw0UVhqOmf/hR3KpG0kLSWgpnVMLNxZpZjZvPN7MqEc53N7KPo+APRsdpm9ouZzY5uPZOVTTLU6tVhK8wjjwzbZI4ZA716qSCIFEMyWwobgGvdfZaZVQBmmtlooBpwKtDI3deZWWJ7fpG7N05iJslUI0dCp07w1VdwzTXQrRuUKxd3KpG0k7SWgrsvd/dZ0eOfgAVAdeAS4D53Xxed+zZZGSQLfPcdnHsunHQS7L47TJoEDz+sgiCynUqlo9nMagNNgKnAgUAbM5tqZuPNrHnCS/c1sw+i421KI5ukKXcYMCAsUTFwINx+e9gq89BD404mktaS3tFsZuWBQcBV7r7azHYBKgMtgebAQDOrAywHarr792bWFBhqZvXdffUW79cR6AhQs2bNZMeXVPTll2EBu+HDoXnzsPFNw4ZxpxLJCEltKZhZGUJB6O/ug6PDy4DBHkwD8oAq7r7O3b8HcPeZwCJCq2Iz7t7L3Zu5e7OqVasmM76kGvewRlG9emEXtIcegsmTVRBESlAyRx8Z0AdY4O6PJJwaCvwtes2BwK7Ad2ZW1cx2jo7XAQ4AFicrn6SZRYvg6KOhY0c45JAwGe3aa2HnneNOJpJRknn56HDgPGCumc2Ojt0E9AX6mtk8YD3Q3t3dzI4AuplZLqH10Mndf0hiPkkHGzeGWci33AJlyoQhph06wE6adymSDEkrCu4+EbBCTp9bwOsHES41iQTz5oUCMG0anHxyWNG0evW4U4lkNP26Jaln/Xro2jVcJlq8OCxPMWyYCoJIKVBRkNQybVrYEvOOO+DMM2HBgrCQnRXW6ExNq1ZB/frhXiSdqChIali7NnQct2oFK1fCiBHQvz9UqRJ3su0ycmTYvmHUqLiTiBSPioLEb9y4MKz0kUfCAnbz54cZymmoXTsoXx7atw/Pzz8/PG/XLt5cIkWloiDxWbUqDDE96qgwmmjcuLAb2u67x51su3XrBjVrhoFSEO5r1Qp7/IikAxUFiceIEWESWp8+0KULzJkTVjdNc/vvHwpDbm5Yfik3N/SZ77df3MlEikZFQUrXihVw9tlwyimwxx4wdWrY76Bs2biTlZiBA0NB6No13L/6atyJRIpOm+xI6XCHl16CK68M+x506wY33AC77hp3shLXpQs88QRUqxYWcF26NO5EIkWnoiDJt3Rp2Pxm5MiwimmfPmG8ZoZqnrDub7Vq4SaSLnT5SJInLy90HNevHzqRH30U3n8/owuCSLpTUchiSZ1g9emnYVTRJZdAixYwdy5cdZUWsBNJcSoKWSwpE6w2bIAHH4SDD4bZs8OlotGjoU6dEvwQEUkWFYUslLQJVnPmQMuWcP31cNxxoeJccEHaLVEhks1UFLJQiU+wWrcObr0VmjULncoDB8KQIbD33iWWWURKh4pCFirRCVaTJ0OTJnDXXWH+QU5OWMhOrQORtKSikKV2eILVzz+HjuPDD4c1a0LHxP/+FyakiUja0jyFLLVDE6zeeScsXLdkCVx2Gdx7L1SokKyoIlKKVBSy1HZNsPrxx7C8dd++cMAB8N570KZN0jKKSOnT5SMpmqFDwwJ2zz8P//1vGGmkgiCScdRSkK375hvo3Dl0OjRqFFY3bdo07lQikiRqKUjB3EPHcd26YX/ku++G6dNVEEQynFoK8ntffAEXXwxvvgmHHQa9e4fiICIZTy0F2SQvD556KiyINGECPP54uFdBEMkaailI8PHHcOGFMHEiHHss9OoFtWvHnUpESplaCtkuNxfuuy90Is+bB889B2+9pYIgkqXUUshmH3wAHTqE+9NPD5eO/vznuFOJSIzUUshGv/4KN98cZrB99RW89hoMGqSCICJqKWSd998PrYOPP4b//AcefhgqV447lYikCLUUssWaNXDFFWEW8q+/hn6D555TQRCRzagoZIO33oIGDeDJJ+Hyy0OH8t//HncqEUlBKgqZ7IcfwiWi44+H3XbbNPegfPm4k4lIilJRyFSDBoUF7F58EW66KeyXfPjhcacSkRSnjuZMs3x5uEQ0eHDYEe3NN6Fx47hTiUiaUEshU7hDv36hdTByZJiQNm2aCoKIFItaCplgyRLo2BFGj4bWrcMCdgcdFHcqEUlDaimks40bQ8dxgwYweXKYkTx+vAqCiGw3tRTS1YIFYQG7SZPC6KKePaFWrbhTiUiaU0sh3eTmhg1vGjeGjz4KG+GMGqWCICIlImlFwcxqmNk4M8sxs/lmdmXCuc5m9lF0/IGE4zea2UIz+9jMjktWtrQ1a1ZYr+iWW+C00yAnB847D8ziTiYiGSKZl482ANe6+ywzqwDMNLPRQDXgVKCRu68zsz0BzKwe0BaoD+wNvGNmB7r7xiRmTA+//AJdu8JDD0HVqjBkSCgKIiIlLGktBXdf7u6zosc/AQuA6sAlwH3uvi469230I6cCA9x9nbt/BiwEWiQrX9qYMCFcKrr//jA7OSdHBUFEkqZU+hTMrDbQBJgKHAi0MbOpZjbezJpHL6sOLE34sWXRsS3fq6OZzTCzGStWrEhu8DitXg2XXQZHHAHr14fhpr17Q6VKcScTkQyW9KJgZuWBQcBV7r6acMmqMtAS6AIMNCv6RXF37+Xuzdy9WdWqVZOSOXZvvBGGmT79NFx1VVjA7phj4k4lIlkgqUXBzMoQCkJ/dx8cHV4GDPZgGpAHVAG+BGok/Pg+0bHs8f33cP75cOKJYdG699+HRx+FcuXiTiYiWSKZo48M6AMscPdHEk4NBf4WveZAYFfgO2A40NbM/mBm+wIHANOSlS+luMPAgVC3Lrz8Mtx6a9gis1WruJOJSJZJ5uijw4HzgLlmNjs6dhPQF+hrZvOA9UB7d3dgvpkNBHIII5cuy4qRR199FfoOhg6Fpk3hnXfg4IPjTiUiWSppRcHdJwKF9RWcW8jP3A3cnaxMKcUd+vaFa6+FdevggQfg6qthF00yF5H46F+gOCxeDBddBGPHhtFFvXvDAQfEnUpERMtclKqNG6F7d2jYEKZPD6OLxo2LrSCsWgX164d7ERFQUSg98+eHnc+uvhqOPDI879QJdorvP8HIkWEu3KhRsUUQkRSjopBs69fDnXeGXdAWLoT+/eH116FGjW3/bJK0axdGvLZvH56ff3543q5dbJFEJEWoKCTT9OnQrBncdhv8619huet27WJfwK5bN6hZE8qUCc/LlAmLrN55Z6yxRCQFqCgkw9q10KULtGwZJqQNGxbmH6TIDOz99w+FITc3zIvLzQ3r7e23X9zJRCRuKgol7d13oVGjsKJphw7hov0pp8Sd6ncGDgwFoWvXcP/qq3EnEpFUoCGpJWXVKrjhBnjmGahTB8aMgaOOijtVobp0gSeegGrV4NxzYenSbf+MiGQ+FYWSMHIkXHwxLF8eJqN16wZly8adaquaN9/0uFq1cBMR0eWjHbFiBZxzDpx0UljSevLkcNkoxQuCiEhhVBS2h3voOK5XL1yMv+MOmDmTVQe10GQwEUlrKgrFtWxZ6Dhu1y70HcyaBbffDrvuqslgIpL2VBSKKi8PevUK60KMGQMPPwyTJkGDBpoMJiIZQ0WhKBYuhKOPDp3JTZvC3LlwzTWw886AJoOJSOZQUdiajRtDi+Dgg8Nlol69Qithi1lemgwmIplCRaEwc+eGnc+uuy7sj5yTE5a7LmSJCk0GE5FMoHkKW1q3Du65J9wqVYIBA+Css7a5XpEmg4lIJlBRSDR1aliaYv78MP+ge3eoUgfp30AAAAXXSURBVKVIP6rJYCKSCXT5CODnn0PHcatWYZLB66/Diy8WuSCIiGQKtRTGjg19BYsXh01v7r8fKlaMO5WISCyyt6Xw44+hGBx9dNj97N13w/aYKggiksWys6UwYwaceip8/TVcf31YpuKPf4w7lYhI7LKzKNSpE2YmDxsWdkYTEREgW4tC5crw9ttxpxARSTnZ26cgIiK/o6IgIiL5VBRERCSfioKIiORTURARkXwqCiIikk9FQURE8qkoiIhIPnP3uDNsNzNbAXwed44tVAG+iztEMaRT3nTKCumVN52yQnrlTcWstdy9akEn0roopCIzm+HuabN2RjrlTaeskF550ykrpFfedMoKunwkIiIJVBRERCSfikLJ6xV3gGJKp7zplBXSK286ZYX0yptOWdWnICIim6ilICIi+VQUREQkn4rCDjKzK81snpnNN7OromONzGyymc01sxFmFsvGz2bW18y+NbN5Cccqm9loM/s0uq8UHTcze9zMFprZh2Z2SIrn/Uv0Ha8zs+tSPOs50Xc618wmmVmjFM97apR3tpnNMLPWqZo14XxzM9tgZmeUZtbi5jWzI81sVfTdzjaz20o777aoKOwAM2sAXAS0ABoBJ5nZ/kBv4L/u3hAYAnSJKWI/4Pgtjv0XGOPuBwBjoucAJwAHRLeOwNOllDFRP4qe9wfgCuChUku3uX4UPetnwF+jvw93Ek/HYz+KnncM0MjdGwMXEP4+l6Z+FD0rZrYzcD8Q13aK/ShGXmCCuzeObt1KKWORqSjsmLrAVHdf6+4bgPHA6cCBwHvRa0YD/4ojnLu/R/jHM9GpwPPR4+eB0xKO/8+DKcCfzGyv0kkaFCevu3/r7tOB3NJLuEkxs05y95XR8SnAPqUSMkEx867xTSNQygGlOhqlmH9vAToDg4Bvk5/u97Yjb0pTUdgx84A2ZraHmZUFTgRqAPMJfykAzoyOpYpq7r48evw1UC16XB1YmvC6ZdGxuBWWNxUVJWsH4I3Si7RVheY1s3+a2UfASEJrIW4FZjWz6sA/iadluzVb+7vQyszmmNkbZlY/hmxbpaKwA9x9AZuarW8Cs4GNhP+JLjWzmUAFYH1sIbci+m0wbcYkp1PegrKa2d8IReGGWEJtxZZ53X2Iu/+F8BvunbEFK8AWWbsDN7h7XoyRtmqLvLMI6w41Ap4AhsYWrBAqCjvI3fu4e1N3PwJYCXzi7h+5+9/dvSnwMrAo3pSb+ea3y0LR/W9N7i/ZvEWzT3QsboXlTUWFZjWzgwnX5k919+9jyrelbX630aWROmZWpbTDbaGwrM2AAWa2BDgD6GFmqXCppsC87r7a3ddEj0cBZVLgu92MisIOMrM9o/uahP6ElxKO7QTcAvSML+HvDAfaR4/bA8MSjp8fjUJqCaxKaP7GqbC8qajArNHfjcHAee7+SUzZClJY3v3NzKLHhwB/AOIuZAVmdfd93b22u9cGXgMudfdU+O27sO/2zwnfbQvCv8Fxf7ebc3fdduAGTABygDnA0dGxK4FPott9RDPHY8j2MrCc0Bm7jHDpYg/CaIhPgXeAytFrDXiK0KqZCzRL8bx/jl6zGvgxelwxRbP2JrQiZ0e3GSn+3d5A6BebDUwGWqdq1i1+rh9wRop/t5dH3+0cwqCDw0o777ZuWuZCRETy6fKRiIjkU1EQEZF8KgoiIpJPRUFERPKpKIiISD4VBZESZGY1zOwzM6scPa8UPa8dbzKRolFREClB7r6UsA7PfdGh+4Be7r4ktlAixaB5CiIlzMzKADOBvoSl1Ru7eyyruYoU1y5xBxDJNO6ea2ZdCIsk/l0FQdKJLh+JJMcJhKUPGsQdRKQ4VBRESpiZNQaOBVoCV5f2ZkUiO0JFQaQERStgPg1c5e5fAA8S35ahIsWmoiBSsi4CvnD30dHzHkBdM/trjJlEikyjj0REJJ9aCiIikk9FQURE8qkoiIhIPhUFERHJp6IgIiL5VBRERCSfioKIiOT7fznmvmfqW6dlAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The other way is to use the statsmodel library. It will automatically take care of all the calculations. "
      ],
      "metadata": {
        "id": "Is62tU_0XEtv"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x = [100,102,103,101,105,100,99,105]\n",
        "y = [259,264,274,268,277,263,258,275]\n",
        "\n",
        "x = sm.add_constant(x)\n",
        "model = sm.OLS(y, x)\n",
        "reg = model.fit()\n",
        "print(reg.summary())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-ahULX5nXOlw",
        "outputId": "dd0f605e-a374-4ba8-e5a7-a9592a163031"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "                            OLS Regression Results                            \n",
            "==============================================================================\n",
            "Dep. Variable:                      y   R-squared:                       0.875\n",
            "Model:                            OLS   Adj. R-squared:                  0.854\n",
            "Method:                 Least Squares   F-statistic:                     42.07\n",
            "Date:                Mon, 20 Jun 2022   Prob (F-statistic):           0.000638\n",
            "Time:                        10:23:33   Log-Likelihood:                -18.507\n",
            "No. Observations:                   8   AIC:                             41.01\n",
            "Df Residuals:                       6   BIC:                             41.17\n",
            "Df Model:                           1                                         \n",
            "Covariance Type:            nonrobust                                         \n",
            "==============================================================================\n",
            "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
            "------------------------------------------------------------------------------\n",
            "const        -40.1017     47.395     -0.846      0.430    -156.072      75.869\n",
            "x1             3.0169      0.465      6.486      0.001       1.879       4.155\n",
            "==============================================================================\n",
            "Omnibus:                        0.896   Durbin-Watson:                   1.370\n",
            "Prob(Omnibus):                  0.639   Jarque-Bera (JB):                0.565\n",
            "Skew:                           0.081   Prob(JB):                        0.754\n",
            "Kurtosis:                       1.708   Cond. No.                     4.84e+03\n",
            "==============================================================================\n",
            "\n",
            "Warnings:\n",
            "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
            "[2] The condition number is large, 4.84e+03. This might indicate that there are\n",
            "strong multicollinearity or other numerical problems.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "You can also use the sklearn library."
      ],
      "metadata": {
        "id": "Byj44PUge0IA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x = np.array([100,102,103,101,105,100,99,105]).reshape(-1, 1)\n",
        "y = np.array([259,264,274,268,277,263,258,275]).reshape(-1, 1)\n",
        "\n",
        "regr = LinearRegression()\n",
        "regr.fit(x, y)\n",
        "print(f'intercept: {float(regr.intercept_)}, coefficient: {float(regr.coef_)}')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pH9uQSOAfMNk",
        "outputId": "2c0516cd-56cc-4bb5-b081-02eaacd0596e"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "intercept: -40.10169491525426, coefficient: 3.0169491525423733\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Sum of Squares Error (SSE):** The sum of squared differences between predicted data points $\\widehat{y_i}$ and observed data points $y_i$. \n",
        "\n",
        "$SSE = \\sum_{i=1}^n (\\widehat{y_i} - y_i)^2 = S_{yy} - \\frac{S_{xy}^2}{S_{xx}} = S_{yy} - b\\ S_{xy} = S_{yy} - b^2\\ S_{xx}$\n",
        "\n",
        "**Sum of Squares Regression (SSR):** The sum of squared differences between predicted data points $\\widehat{y_i}$ and the mean of the response variable $\\overline{y_i}$\n",
        "\n",
        "$SSR = \\sum_{i=1}^n (\\widehat{y_i} - \\overline{y_i})^2 = \\frac{S_{xy}^2}{S_{xx}}$\n",
        "\n",
        "**Sum of Squares Total (SST):** The sum of squared differences between individual data points $y_i$ and the mean of the response variable $\\overline{y_i}$.\n",
        "\n",
        "$SST = \\sum_{i=1}^n (y_i - \\overline{y_i})^2$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "The following relationship exists between these three measures: $SST = SSR + SSE$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "**R-Squared:**\n",
        "\n",
        "$R^2$ sometimes referred to as the coefficient of determination, is a measure of how well a linear regression model fits a dataset. It represents the proportion of the variance in the response variable that can be explained by the predictor variable. The value for $R^2$ can range from 0 to 1. \n",
        "\n",
        "A value of 0 indicates that the response variable cannot be explained by the predictor variable at all. A value of 1 indicates that the response variable can be perfectly explained without error by the predictor variable.\n",
        "\n",
        "$R^2 = \\frac{SSR}{SST} = \\frac{S_{xy}^2}{S_{xx}S_{yy}}$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "**Adjusted R-Squared:**\n",
        "\n",
        "Adjusted $R^2$ is a corrected goodness-of-fit (model accuracy) measure for linear models. $R^2$ tends to optimistically estimate the fit of the linear regression. It always increases as the number of effects are included in the model. Adjusted $R^2$ attempts to correct for this overestimation. Adjusted $R^2$ might decrease if a specific effect does not improve the model. In other words, the adjusted $R^2$ will penalize you for adding independent variables that do not fit the model. Adjusted $R^2$ is always less than or equal to $R^2$.\n",
        "\n",
        "n: Number of points in your data set.\n",
        "\n",
        "k: Number of independent variables in the model, excluding the constant\n",
        "\n",
        "Adjusted $R^2 = 1-\\frac{(1-R^2)\\ x\\ (n-1)}{(n-k-1)}$"
      ],
      "metadata": {
        "id": "YoPPjDpDp-8c"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Statistical_Inferences_about_the_Regression_Parameters'></a>\n",
        "\n",
        "## **6.3. Statistical Inferences about the Regression Parameters:**"
      ],
      "metadata": {
        "id": "Wr9dJbIJTLR0"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Inferences_Concerning_B'></a>\n",
        "\n",
        "### **6.3.1. Inferences Concerning $B$:**"
      ],
      "metadata": {
        "id": "IatgOLJgjUKY"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Known_Variance'></a>\n",
        "\n",
        "#### **6.3.1.1. Known Variance:**"
      ],
      "metadata": {
        "id": "Wdk-sNNkkx86"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**A. Two Tailed Test:**\n",
        "\n",
        "$H_0:\\ B = B_0$\n",
        "\n",
        "$H_1:\\ B  \\neq B_0$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Testing statistics: \n",
        "\n",
        "$Z_0  \\equiv \\frac{b\\ -\\ B_0}{\\frac{\\sigma}{\\sqrt{S_{xx}}}}$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "We accept $H_0$ if:\n",
        "\n",
        "1. $-\\ Z_{\\frac{\\alpha}{2}}\\ <\\ Z_0 = \\frac{b\\ -\\ B_0}{\\frac{\\sigma}{\\sqrt{S_{xx}}}}\\ <\\ Z_{\\frac{\\alpha}{2}}$\n",
        "\n",
        "2. P_value $ = 2 \\times P(Z \\geq |Z_0|) > \\alpha$"
      ],
      "metadata": {
        "id": "lRVBB3m3jtB2"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**B. One-sided tests:**\n",
        "\n",
        "**Right-tailed test:**\n",
        "\n",
        "$H_0:\\ B =  B_0\\ \\quad or \\quad H_0:\\ \\mu  \\leq B_0$\n",
        "\n",
        "$H_1:\\ B  > B_0$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Testing statistics: \n",
        "\n",
        "$Z_0  \\equiv \\frac{b\\ -\\ B_0}{\\frac{\\sigma}{\\sqrt{S_{xx}}}}$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "We accept $H_0$ if:\n",
        "\n",
        "1. $ -\\infty <\\ Z_0 = \\frac{b\\ -\\ B_0}{\\frac{\\sigma}{\\sqrt{S_{xx}}}}\\ <\\ Z_{\\alpha} $\n",
        "\n",
        "2. P_value $ = P(Z \\geq Z_0) > \\alpha$"
      ],
      "metadata": {
        "id": "McNI-w0Wl6Jh"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Left-tailed test:**\n",
        "\n",
        "$H_0:\\ B =  B_0 \\quad or \\quad H_0:\\ B  \\geq B_0$\n",
        "\n",
        "$H_1:\\ B < B_0$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Testing statistics: \n",
        "\n",
        "$Z_0  \\equiv \\frac{b\\ -\\ B_0}{\\frac{\\sigma}{\\sqrt{S_{xx}}}}$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "We accept $H_0$ if:\n",
        "\n",
        "1. $-\\ Z_{\\alpha}\\ <\\ Z_0 = \\frac{b\\ -\\ B_0}{\\frac{\\sigma}{\\sqrt{S_{xx}}}}\\ <\\ \\infty $\n",
        "\n",
        "2. P_value $ = P(Z \\leq Z_0) > \\alpha$"
      ],
      "metadata": {
        "id": "H1whcPQ_mRtg"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "class test_coefficient_known_variance:\n",
        "  \"\"\"\n",
        "  Parameters\n",
        "  ----------\n",
        "  null_b : B0\n",
        "  b : estimate of B\n",
        "  population_sd : known standrad deviation of the population\n",
        "  alpha : significance level\n",
        "  type_t : 'two_tailed', 'right_tailed', 'left_tailed'\n",
        "  S_xx\n",
        "  S_xy\n",
        "  data_x : optional\n",
        "  data_y : optional\n",
        "  \"\"\"\n",
        "  def __init__(self, null_b, population_sd, type_t, alpha, S_xx = 0., S_xy = 0., data_x=None, data_y=None):\n",
        "    self.null_b = null_b\n",
        "    self.population_sd = population_sd\n",
        "    self.type_t = type_t\n",
        "    self.alpha = alpha\n",
        "    self.S_xx = S_xx\n",
        "    self.S_xy = S_xy\n",
        "    self.data_x = data_x\n",
        "    self.data_y = data_y\n",
        "    if data_x is not None:\n",
        "      xx = []\n",
        "      for i in list(data_x):\n",
        "        xx.append((i - np.mean(list(data_x)))**2)\n",
        "      self.S_xx = np.sum(xx)\n",
        "\n",
        "      xy = []\n",
        "      for i in range(len(data_x)):\n",
        "        xy.append((data_x[i] - np.mean(data_x))* (data_y[i] - np.mean(data_y)))\n",
        "      self.S_xy = np.sum(xy)\n",
        "\n",
        "    test_coefficient_known_variance.__test(self)\n",
        "  \n",
        "  def __test(self):\n",
        "    b = self.S_xy / self.S_xx\n",
        "    test_statistic = (b - self.null_b) / (self.population_sd - np.sqrt(self.S_xx))\n",
        "    print('test_statistic:', test_statistic, '\\n')\n",
        "\n",
        "    if self.type_t == 'two_tailed':\n",
        "      p_value = 2*(1-norm.cdf(abs(test_statistic)))\n",
        "      print('P_value:', p_value, '\\n')\n",
        "    elif self.type_t == 'left_tailed':\n",
        "      p_value = (norm.cdf(test_statistic))\n",
        "      print('P_value:', p_value, '\\n')\n",
        "    else:\n",
        "       p_value = (1-norm.cdf(test_statistic))\n",
        "       print('P_value:', p_value, '\\n')\n",
        "\n",
        "    if p_value < self.alpha:\n",
        "      print(f'Since p_value < {self.alpha}, reject null hypothesis.')\n",
        "    else:\n",
        "      print(f'Since p_value > {self.alpha}, the null hypothesis cannot be rejected.')"
      ],
      "metadata": {
        "id": "EEkoJhe1SwJb"
      },
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x = [100,102,103,101,105,100,99,105]\n",
        "y = [259,264,274,268,277,263,258,275]\n",
        "\n",
        "test_coefficient_known_variance(null_b = 0, population_sd = 2, type_t = 'two_tailed', alpha = 0.05, data_x = x, data_y = y);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oMwjAr71pecL",
        "outputId": "310e60c3-efde-4559-8d73-069c93eae4c2"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "test_statistic: -0.7408139430728671 \n",
            "\n",
            "P_value: 0.45880625987978396 \n",
            "\n",
            "Since p_value > 0.05, the null hypothesis cannot be rejected.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Unknown_Variance'></a>\n",
        "\n",
        "#### **6.3.1.2. Unknown Variance:**"
      ],
      "metadata": {
        "id": "iPbXBvg3m-2e"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**A. Two Tailed Test:**\n",
        "\n",
        "$H_0:\\ B = B_0$\n",
        "\n",
        "$H_1:\\ B  \\neq B_0$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Testing statistics: \n",
        "\n",
        "$t_0  \\equiv \\frac{b\\ -\\ B_0}{\\frac{S_{y|x}}{\\sqrt{S_{xx}}}}$\n",
        "\n",
        "$S_{y|x} = \\sqrt{MSE} = \\sqrt{\\frac{SSE}{n-2}} = \\sqrt{\\frac{(S_{yy}- \\frac{S_{xy}^2}{S_{xx}})}{n-2}}$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "We accept $H_0$ if:\n",
        "\n",
        "1. $-\\ t_{\\frac{\\alpha}{2},n-2}\\ <\\ t_0 = \\frac{b\\ -\\ B_0}{\\frac{S_{y|x}}{\\sqrt{S_{xx}}}} <\\ t_{\\frac{\\alpha}{2},n-2}$\n",
        "\n",
        "2. P_value $ = 2 \\times P(t_{n-2} \\geq |t_0|) > \\alpha$"
      ],
      "metadata": {
        "id": "PajDver2rsLG"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**B. One-sided tests:**\n",
        "\n",
        "**Right-tailed test:**\n",
        "\n",
        "$H_0:\\ B = B_0\\ \\quad or \\quad H_0:\\ B \\leq B_0$\n",
        "\n",
        "$H_1:\\ B > B_0$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Testing statistics: \n",
        "\n",
        "$t_0  \\equiv \\frac{b\\ -\\ B_0}{\\frac{S_{y|x}}{\\sqrt{S_{xx}}}}$\n",
        "\n",
        "$S_{y|x} = \\sqrt{MSE} = \\sqrt{\\frac{SSE}{n-2}} = \\sqrt{\\frac{(S_{yy}- \\frac{S_{xy}^2}{S_{xx}})}{n-2}}$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "We accept $H_0$ if:\n",
        "\n",
        "1. $ -\\infty <\\ t_0  = \\frac{b\\ -\\ B_0}{\\frac{S_{y|x}}{\\sqrt{S_{xx}}}}\\ <\\ t_{\\alpha, n-2} $\n",
        "\n",
        "2. P_value $ = P(t_{n-2} \\geq t_0) > \\alpha$"
      ],
      "metadata": {
        "id": "MUQNiMxAuX66"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Left-tailed test:**\n",
        "\n",
        "$H_0:\\ B = B_0 \\quad or \\quad H_0:\\ B \\geq B_0$\n",
        "\n",
        "$H_1:\\ B < B_0$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Testing statistics: \n",
        "\n",
        "$t_0  \\equiv \\frac{b\\ -\\ B_0}{\\frac{S_{y|x}}{\\sqrt{S_{xx}}}}$\n",
        "\n",
        "$S_{y|x} = \\sqrt{MSE} = \\sqrt{\\frac{SSE}{n-2}} = \\sqrt{\\frac{(S_{yy}- \\frac{S_{xy}^2}{S_{xx}})}{n-2}}$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "We accept $H_0$ if:\n",
        "\n",
        "1. $ -\\ t_{\\alpha, n-2} <\\ t_0  = \\frac{b\\ -\\ B_0}{\\frac{S_{y|x}}{\\sqrt{S_{xx}}}}\\ <\\  \\infty$\n",
        "\n",
        "2. P_value $ = P(t_{n-2} \\leq t_0) > \\alpha$"
      ],
      "metadata": {
        "id": "Yir9qjUIuuo4"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "class test_coefficient_unknown_variance:\n",
        "  \"\"\"\n",
        "  Parameters\n",
        "  ----------\n",
        "  null_b : B0\n",
        "  b : estimate of B\n",
        "  alpha : significance level\n",
        "  type_t : 'two_tailed', 'right_tailed', 'left_tailed'\n",
        "  S_xx\n",
        "  S_xy\n",
        "  S_yy\n",
        "  data_x : optional\n",
        "  data_y : optional\n",
        "  \"\"\"\n",
        "  def __init__(self, null_b, alpha, type_t, n = 0., S_xx = 0., S_xy = 0., S_yy = 0., data_x=None, data_y=None):\n",
        "    self.null_b = null_b\n",
        "    self.type_t = type_t\n",
        "    self.alpha = alpha\n",
        "    self.n = n\n",
        "    self.S_xx = S_xx\n",
        "    self.S_xy = S_xy\n",
        "    self.S_yy = S_yy\n",
        "    self.data_x = data_x\n",
        "    self.data_y = data_y\n",
        "    if data_x is not None:\n",
        "      self.n = len(list(data_x))\n",
        "      xx = []\n",
        "      for i in list(data_x):\n",
        "        xx.append((i - np.mean(list(data_x)))**2)\n",
        "      self.S_xx = np.sum(xx)\n",
        "\n",
        "      yy = []\n",
        "      for i in list(data_y):\n",
        "        yy.append((i - np.mean(list(data_y)))**2)\n",
        "      self.S_yy = np.sum(yy)\n",
        "\n",
        "      xy = []\n",
        "      for i in range(len(data_x)):\n",
        "        xy.append((data_x[i] - np.mean(data_x))* (data_y[i] - np.mean(data_y)))\n",
        "      self.S_xy = np.sum(xy)\n",
        "      self.Syx=None\n",
        "    \n",
        "    test_coefficient_unknown_variance.__test(self)    \n",
        "\n",
        "  def __test(self):\n",
        "    b = self.S_xy / self.S_xx\n",
        "    self.Syx = np.sqrt(((self.S_yy) - ((self.S_xy)**2 / (self.S_xx))) / (self.n-2))\n",
        "    test_statistic = (b - self.null_b) / (self.Syx / np.sqrt(self.S_xx))\n",
        "    print('test_statistic:', test_statistic, '\\n')\n",
        "    \n",
        "    if self.type_t == 'two_tailed':\n",
        "      p_value = 2*(1-t.cdf(abs(test_statistic), df=self.n-2))\n",
        "      print('P_value:', p_value, '\\n')\n",
        "    elif self.type_t == 'left_tailed':\n",
        "      p_value = (t.cdf(test_statistic, df=self.n-2))\n",
        "      print('P_value:', p_value, '\\n')\n",
        "    else:\n",
        "       p_value = (1-t.cdf(test_statistic, df=self.n-2))\n",
        "       print('P_value:', p_value, '\\n')\n",
        "\n",
        "    if p_value < self.alpha:\n",
        "      print(f'Since p_value < {self.alpha}, reject null hypothesis.')\n",
        "    else:\n",
        "      print(f'Since p_value > {self.alpha}, the null hypothesis cannot be rejected.')"
      ],
      "metadata": {
        "id": "yTk8vwIrnCHm"
      },
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x = [45,50,55,60,65,70,75]\n",
        "y = [24.2,25,23.3,22,21.5,20.6,19.8]\n",
        "\n",
        "test_coefficient_unknown_variance(null_b = 0, type_t = 'two_tailed', alpha = 0.05, data_x = x, data_y = y);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MPupst8qrP9R",
        "outputId": "aa1e5154-911f-4fa0-fd30-13cf7814b2f6"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "test_statistic: -8.13847644694359 \n",
            "\n",
            "P_value: 0.0004547603641058551 \n",
            "\n",
            "Since p_value < 0.05, reject null hypothesis.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "test_coefficient_unknown_variance(null_b = 0, type_t = 'two_tailed', alpha = 0.05, n=22, S_xx = 5, S_xy = 3, S_yy = 7);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iPUJchAY2WEy",
        "outputId": "04bc05ad-1f4c-436d-db7d-fec547d53238"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "test_statistic: 2.631174057921087 \n",
            "\n",
            "P_value: 0.016008148639689912 \n",
            "\n",
            "Since p_value < 0.05, reject null hypothesis.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Inferences_Concerning_A'></a>\n",
        "\n",
        "### **6.3.2. Inferences Concerning $A$:**"
      ],
      "metadata": {
        "id": "NGr8PKz4P-UZ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Unknown_Variance_A'></a>\n",
        "\n",
        "#### **6.3.2.1. Unknown Variance:**"
      ],
      "metadata": {
        "id": "0NX0f42yQJ8L"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**A. Two Tailed Test:**\n",
        "\n",
        "$H_0:\\ A = A_0$\n",
        "\n",
        "$H_1:\\ A  \\neq A_0$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Testing statistics: \n",
        "\n",
        "$t_0  \\equiv \\frac{a\\ -\\ A_0}{S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{\\overline{x}^2}{S_{xx}}}}$\n",
        "\n",
        "$S_{y|x} = \\sqrt{MSE} = \\sqrt{\\frac{SSE}{n-2}} = \\sqrt{\\frac{(S_{yy}- \\frac{S_{xy}^2}{S_{xx}})}{n-2}}$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "We accept $H_0$ if:\n",
        "\n",
        "1. $-\\ t_{\\frac{\\alpha}{2},n-2}\\ <\\ t_0 = \\frac{a\\ -\\ A_0}{S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{\\overline{x}^2}{S_{xx}}}} <\\ t_{\\frac{\\alpha}{2},n-2}$\n",
        "\n",
        "2. P_value $ = 2 \\times P(t_{n-2} \\geq |t_0|) > \\alpha$"
      ],
      "metadata": {
        "id": "FsWjyZBAQZ_U"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**B. One-sided tests:**\n",
        "\n",
        "**Right-tailed test:**\n",
        "\n",
        "$H_0:\\ A = A_0$\n",
        "\n",
        "$H_1:\\ A  \\neq A_0$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Testing statistics: \n",
        "\n",
        "$t_0  \\equiv \\frac{a\\ -\\ A_0}{S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{\\overline{x}^2}{S_{xx}}}}$\n",
        "\n",
        "$S_{y|x} = \\sqrt{MSE} = \\sqrt{\\frac{SSE}{n-2}} = \\sqrt{\\frac{(S_{yy}- \\frac{S_{xy}^2}{S_{xx}})}{n-2}}$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "We accept $H_0$ if:\n",
        "\n",
        "1. $ -\\infty <\\ t_0  = \\frac{a\\ -\\ A_0}{S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{\\overline{x}^2}{S_{xx}}}}\\ <\\ t_{\\alpha, n-2} $\n",
        "\n",
        "2. P_value $ = P(t_{n-2} \\geq t_0) > \\alpha$"
      ],
      "metadata": {
        "id": "lXzcHZe8Rssq"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Left-tailed test:**\n",
        "\n",
        "$H_0:\\ A = A_0$\n",
        "\n",
        "$H_1:\\ A  \\neq A_0$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Testing statistics: \n",
        "\n",
        "$t_0  \\equiv \\frac{a\\ -\\ A_0}{S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{\\overline{x}^2}{S_{xx}}}}$\n",
        "\n",
        "$S_{y|x} = \\sqrt{MSE} = \\sqrt{\\frac{SSE}{n-2}} = \\sqrt{\\frac{(S_{yy}- \\frac{S_{xy}^2}{S_{xx}})}{n-2}}$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "We accept $H_0$ if:\n",
        "\n",
        "1. $ -\\ t_{\\alpha, n-2} <\\ t_0  = \\frac{a\\ -\\ A_0}{S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{\\overline{x}^2}{S_{xx}}}}\\ <\\  \\infty$\n",
        "\n",
        "2. P_value $ = P(t_{n-2} \\leq t_0) > \\alpha$"
      ],
      "metadata": {
        "id": "EJGfySgCR5qW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "class test_intercept_unknown_variance:\n",
        "  \"\"\"\n",
        "  Parameters\n",
        "  ----------\n",
        "  null_a : A0\n",
        "  a : estimate of A\n",
        "  alpha : significance level\n",
        "  type_t : 'two_tailed', 'right_tailed', 'left_tailed'\n",
        "  S_xx\n",
        "  S_xy\n",
        "  S_yy\n",
        "  data_x : optional\n",
        "  data_y : optional\n",
        "  \"\"\"\n",
        "  def __init__(self, null_a, alpha, type_t, n = 0., S_xx = 0., S_xy = 0., S_yy = 0., data_x=None, data_y=None):\n",
        "    self.null_a = null_a\n",
        "    self.type_t = type_t\n",
        "    self.alpha = alpha\n",
        "    self.n = n\n",
        "    self.S_xx = S_xx\n",
        "    self.S_xy = S_xy\n",
        "    self.S_yy = S_yy\n",
        "    self.data_x = data_x\n",
        "    self.data_y = data_y\n",
        "    if data_x is not None:\n",
        "      self.n = len(list(data_x))\n",
        "      xx = []\n",
        "      for i in list(data_x):\n",
        "        xx.append((i - np.mean(list(data_x)))**2)\n",
        "      self.S_xx = np.sum(xx)\n",
        "\n",
        "      yy = []\n",
        "      for i in list(data_y):\n",
        "        yy.append((i - np.mean(list(data_y)))**2)\n",
        "      self.S_yy = np.sum(yy)\n",
        "\n",
        "      xy = []\n",
        "      for i in range(len(data_x)):\n",
        "        xy.append((data_x[i] - np.mean(data_x))* (data_y[i] - np.mean(data_y)))\n",
        "      self.S_xy = np.sum(xy)\n",
        "      self.Syx=None\n",
        "    \n",
        "    test_intercept_unknown_variance.__test(self)    \n",
        "\n",
        "  def __test(self):\n",
        "    b = self.S_xy / self.S_xx\n",
        "    a = np.mean(self.data_y) - b * np.mean(self.data_x)\n",
        "    self.Syx = np.sqrt(((self.S_yy) - ((self.S_xy)**2 / (self.S_xx))) / (self.n-2))\n",
        "    test_statistic = (a - self.null_a) / (self.Syx * np.sqrt((1/self.n) + ((np.mean(self.data_x)**2) / self.S_xx)))\n",
        "    print('test_statistic:', test_statistic, '\\n')\n",
        "    \n",
        "    if self.type_t == 'two_tailed':\n",
        "      p_value = 2*(1-t.cdf(abs(test_statistic), df=self.n-2))\n",
        "      print('P_value:', p_value, '\\n')\n",
        "    elif self.type_t == 'left_tailed':\n",
        "      p_value = (t.cdf(test_statistic, df=self.n-2))\n",
        "      print('P_value:', p_value, '\\n')\n",
        "    else:\n",
        "       p_value = (1-t.cdf(test_statistic, df=self.n-2))\n",
        "       print('P_value:', p_value, '\\n')\n",
        "\n",
        "    if p_value < self.alpha:\n",
        "      print(f'Since p_value < {self.alpha}, reject null hypothesis.')\n",
        "    else:\n",
        "      print(f'Since p_value > {self.alpha}, the null hypothesis cannot be rejected.')"
      ],
      "metadata": {
        "id": "bHy8dRuzQGaa"
      },
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x = [45,50,55,60,65,70,75]\n",
        "y = [24.2,25,23.3,22,21.5,20.6,19.8]\n",
        "\n",
        "test_intercept_unknown_variance(null_a = 0, type_t = 'two_tailed', alpha = 0.05, data_x = x, data_y = y);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kpPe2sLRSbln",
        "outputId": "1b83afb7-1a89-4a75-ca96-263875139cdf"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "test_statistic: 25.6123251907897 \n",
            "\n",
            "P_value: 1.6943048344320033e-06 \n",
            "\n",
            "Since p_value < 0.05, reject null hypothesis.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='T-tests_for_Regression_Parameters_with_statsmodels'></a>\n",
        "\n",
        "### **6.3.3. T-tests for Regression Parameters with statsmodels:**"
      ],
      "metadata": {
        "id": "kPjcRV2q3s1b"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x = [100,102,103,101,105,100,99,105]\n",
        "y = [259,264,274,268,277,263,258,275]\n",
        "\n",
        "x = sm.add_constant(x)\n",
        "model = sm.OLS(y, x)\n",
        "reg = model.fit()\n",
        "print(reg.summary())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CBkXqdwC4Tci",
        "outputId": "e718ff14-cb1d-43be-bb03-68c0658b8c0a"
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "                            OLS Regression Results                            \n",
            "==============================================================================\n",
            "Dep. Variable:                      y   R-squared:                       0.875\n",
            "Model:                            OLS   Adj. R-squared:                  0.854\n",
            "Method:                 Least Squares   F-statistic:                     42.07\n",
            "Date:                Mon, 20 Jun 2022   Prob (F-statistic):           0.000638\n",
            "Time:                        10:23:34   Log-Likelihood:                -18.507\n",
            "No. Observations:                   8   AIC:                             41.01\n",
            "Df Residuals:                       6   BIC:                             41.17\n",
            "Df Model:                           1                                         \n",
            "Covariance Type:            nonrobust                                         \n",
            "==============================================================================\n",
            "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
            "------------------------------------------------------------------------------\n",
            "const        -40.1017     47.395     -0.846      0.430    -156.072      75.869\n",
            "x1             3.0169      0.465      6.486      0.001       1.879       4.155\n",
            "==============================================================================\n",
            "Omnibus:                        0.896   Durbin-Watson:                   1.370\n",
            "Prob(Omnibus):                  0.639   Jarque-Bera (JB):                0.565\n",
            "Skew:                           0.081   Prob(JB):                        0.754\n",
            "Kurtosis:                       1.708   Cond. No.                     4.84e+03\n",
            "==============================================================================\n",
            "\n",
            "Warnings:\n",
            "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
            "[2] The condition number is large, 4.84e+03. This might indicate that there are\n",
            "strong multicollinearity or other numerical problems.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAsUAAABUCAYAAABqbkrXAAAZwElEQVR4nO2drXqrzNfG7+e9/qeAqymoyCgc+wTiwCe47aKKiymOqrg64gsuJ1AcpsiooKo2B5FX8BE+AyS0+eD+XVdECyyGNTNr1qxZDP8dDocDCCGEEEIIGTH/d+0CEEIIIYQQcm3oFBNCCCGEkNFDp5gQQgghhIweOsWEEEIIIWT00CkmhBBCCCGjh04xIYQQQggZPXSKCSGEjJvIwUI0EVxbBiHkqtApJoQQQkhHApiiCDH9mZwGkB4E5rHtiCJurfnQKSbjInKwyHXIhRNdu0Q/T2BCZASLPBiRsygMrqPpzzeC7oYIwxDhalo5Fpj5ejnP9rTJ6Hf88dvH5TqP4CxyMhYO8tqq02f5vLo+WZk0TVdxuwld6Gc/7c9Bp5iMiACmbACWn3TKEBtNuHahziCO1NzaDJvcBpGzqAxoD4tiwU/6cuhbgCH/iOMTmIyIdiVyFlD3x3pxdRtqz/bYKiMwocLN7HjTPZScrb9fe9/O5TqP4CxkGFJOp5IBOdfmp6uwoMv4PgAkCQWt6m7xvJpJ0y1Dp5iMh2iPPRTM/jymYSRk1Aga3iwF3vazGLkaywThJgjwbnjQl1rmKE3/WlA8A++d5xQdZExXBWcrPr7F5ygregCdR5/YegqsvyWd2ms0zjEjB2u7eM0jQKeY3ATlpZlitKeUw1a3NHTjeUq9KD3LcVBP9aDCBmCrzUtUBX2q9pnFOKXzAKa4gBPll9wWBQMaOYu4XPnnueuKuW3SpUvZ8ADPgCzW18sj873zCn8L2iaOeF24hK/GHe7ilICHJ9jCho5Zzk8K3g14APb7jo1wCBljYjB9SZDy8SJBggQPu++G274b8JQZHi3GRKeYXJ3AFAtLP8VlrgCmqGKfWwbzrT3U/KAUmBDVPSz/uIy6V4+OcZbnJBvw4MGQb3hgixws8s8Shgg3aQRgilUuFyvL6SstUVX06fbP3CrLqOgcAODBkGVsZ35yDmC8lKJytgpRBdy03LZ63xOWG0bQNkk9KMW0gnCDB101LhI5WNsoRMyAeNk3br/9J8vpkrEbd7jcsvAKjxUfGxBlgicA6SR+PfHh6oDX5F0NIKPJQfMMuTFH9qG4VOeCBAk21rnZc+Ss0RxOCbCt6WsA7n7ySKeYXJd0CeatpnMB2Sx4mRvVBW0JHTa2QXqKDcV6Ow78goalDtjJCamzEPoWFCg5h/NWBzYP27PXAQNsT+nzTBllnacolp9NYARJArwdimZYh5vp+QkThdEeMiD5iLi8xcwPUZfCKGgbhK4OW33sl61uhsCEKK4x8Y8BDmXy9DMyAhNqw2ToOInxYcGA/MiO8UU6n2KV5OSnDu0LZo0vwkXOGrZioZw5kY21+dzmO3OM/3ftApCR872DBwnLBg8u2u8BZYZi146dq+0+AqbAfg94tgzRKF18i6+2tiFo2PjAQj4+j+7WD/S1RHvsAUwuKUO0xx4ebFlEm0ql/HrbdIUwLJ2QRTAAQIC2CaFdUjZC8igW/E3HCeDTBAoAz3hHoN3qhPgB8AzIsOCHq6ReIuz7GqWuMgITompDsfwWGylAW+ow1HjS/nALJ0PoXNCwCXPWOXKwgIJZxa9OcpjdTase47zkLfYRML0TpdMpJtflaQIFzZ0mH308Hv7GzgOkxJOWJECZ+Y/zZnHeOAUmRHWBif+LS+CCBAkKZr95T0J+kMhZQDY86G6IDb3hn2M6gw4bKERtv7HzFMzeOhqTrjIiB4vEIe5i+6P9HoDU9UnuhyF0XkP0uYWnzFAWEadV6HC79KPvHTwAs7NL8fswfYJcF+EPZopXzUVNSTp8PtcpMNXC0s10psMzXh7zZaIkulX6JybKMT2kQKLPLP0iiaT0Y4qZfqJOyE0TTyTH+iZ+lcAUIRsS3LDHikuJp7jD3dUy8HWYYqYDtnpcMo/tdTXfN3vXo5Lo3UVGvL2mp7vdgiGRgxfDg2L9fcAVgiF0Xj1PNlCThhfhc9tVjwHMOK/lroIrjBSTKyNAi/MFIOfyH46z/zjXKZ9OUFkyna4QuibE0nJ/r7SDW6HGidXdsGRUBGhvFrayCtHOTkpetkuPpfrS4foW1vKuVzGmqxCuKRbqpJgfTG6W6QquLkLN+oMCa6RR/+P+rZfk2AOC9gZrK0M9djj2hQamqxAuxKOu+qS4dJQRmPEOPLBzNhDI2cFk391sM5L4XZJH7QND6Dwwk11W0utr+kzkvMDwdLibeskFGUheBr+zTvLf4XA4XLsQhBBCyNWIHCzkHZaXOLpDyLgL4h2BcI9BB3JD3GY7YvoEIYQQQggZPXSKCSGEENKL7ONB3Hic9CH7mJN6Yh/k68H0CUIIIYQQMnoYKSaEEEIIIaOHTjEhhBBCCBk9dIoJIYQQQsjooVNMCCGEEEJGD51iQgghhBAyeugUE0IIIYSQ0UOnmBBCCCGEjB46xYQQQgghZPTQKSaEEEIIIaOHTjEhhBBCCBk9dIoJIYQQQsjoGcQpjpwFRFE8/hYOosIZAcz8cVGEGfSUETlYFGQs4BRvgsAs3kOs3uRhZPyKzm9Fxo3o/FZk3E29/ZKMe6k31v3wMu6l3lj3w8u4l3pj3feVcV3+OxwOh2sXghBCCCGEkGvC9AlCCCGEEDJ66BQTQgghhJDRQ6eYEEIIIYSMHjrFhBBCCCFk9NApJoQQQggho4dOMSGEEEIIGT10igkhhBBCyOihU0wIIYQQQkYPnWJCCCGEEDJ66BQTQgghhJDRQ6eYEEIIIYSMHjrFpAMRnIUIUVzAicqHHCxEEWLyW1RO6EhgxjLM4MT9k9/CQd1dImdRX4ZUduVX8zz3RJ3OSvWR/9WqtoFUlyd1XtJrWX5FRmP9Pggt+uhCRWdNQk72FyAw2/vLQzCAzlOa7EdBl006H7Ac1+eEvUcAs6MdvbQNFvVu4qjS0njQVDdDjU23Tieb373eTlIeSxvq9WTd33pfOTwS/z4O8+f54ePftQvyWPz7mB+e5/Ma3X4dXp+fD69f2YmHef7vjny9Ph+eXz8OH/Pnw3PNxfHx7CY158X/m398HF6fnw/zLg3g6/Xw/Px66FnUGyLW/Xw+r9VZgYv7RarfnICv18NzXmbNPf59dCjbo1B+/rJ+OomYH57nH4fjJV8N7flU3cfHRqH3AXReljUvt/MKJZs3dDmuTfosH03P8HV4bX22y9tguS98vT6X+kblipKN+nf4mJftU/+x6W6p2OMu9dZCZcysG4tb6r7DuHFtGCkmLQR4NwDrbQmpdCRy1rAVC3+nyT8EDUsdsNc9ogKBifXER7j6U388crC2FVjHm0CLb5LNciPnBbtliI1WLmETEZy1DcX6i2n7yTdJ5Kxh6y7eZu3nBu8GPH0JTTj3bgIkCfB230eZWxvIyxQ0LHUPxvutTft/h4qOp39hKf308b3zAEnCsZqeMFGq552q+/RYuLrXlt2dIXSelwXrDctWExLXyX5/tHBDluO6RHBedliGG3Q2pXVSLm6DAd4ND/pSw1GlFhTPQKNKg3cYno7l0SBB22xK9gmwt/dWJ+dxuc2vEu33gDLBU/afeFwonNNS9/cwbgzmFJeXmIpLFeXljlLYPjDjEHs+rF4Jy1eXTLKwe7p8IBvw4MGQR7Bk8ksEpgq7tnNF+Nx6UGZ/joN45GBtA/B2+C6f3sR0hc2Jnht9buEpM/w53gROfBOkPpqgbdDL/gbvMDwFsz8DWozfJHLwYiA3UTh97toG9NklTlKAbY0MZfJU+Psp9hYed6m+kRr9BO8wPPTSx/SvBcVWsyXgyHkpDfRoqfu4T15W1/fCMDqPrzOh2iU9NxF9YluwHQOW4+oI0DarCwMFA7TBYAsbOooqNRCrtDZxLh4T9NndBjkGZRCbX0XQltA9A3LqmwUm1ELAqlvd3/q4MYhTHJgi1L0FPwwRJr+joxPBWcgwJDc7FroSDLnkGHsG5PUkkeFC9wy85E4IzJKMMDw6QoKGTRgi9C0oUGD5deUgvYkcrG0d7gmPU5KE5NQFRHmHpW9BwR61tutc0uhZ5GAhytgtfVhKk4FspzJbvTPSqFa3MXwLLx/N73OfbKKrYm/5hYnH00SBZ7zn8vzi6E4FW23IC3w0FKS2PjBFiOsJfFfvN0EUNGxCH9Y+1pm8ncEPi07K6br/xs5TMEExx/DmcvYGYwCdJw6V7p5yBnMBGdmA5G5K+h+iHPfEqcDTQG0wi0jGubDriY9YpTUaTSYqJ4MEP+Qo3iLNNv/SgOEUqzCEKxmQRRGiCrhhvi+0133nceOKXO4Up8vbbxpq7XRdg02Wl7af+UrR4W5SGVPM6jqAvX3gQfXWiODEIamW2Xc8YMi7JcJsAJcgDexwZk53bjIknXOTSjrGndEnqlWzDNmH6eo4uVzu5MLqjaBt4Oo21Mz4bTGximv9grYpTGLj8x/ZMT4O4GFqywrLjV2uTyf/PiwYkPOrap3q3oOxBt6yAIQOW73zF0pPcpnOI+cFBtomjQK0TdqOfUzWdS/bXVr390LsGGX92rcAQy45WAO1wcCEKK4x8Y/BrXKUEeiSKhCPZecGB+6LJpvfpd7aiMf6NACa2v9iVzhd913GjWtzuVP8vYN3ygmqPV7NUWxjugqLynzkN6p/iZM7DATvMGDhrcX5slUZu2V4zCH63mHweZ+tlpzub+zOvMlP5Fr9Hl2iWrmzyznfFxDn9G2Rn8fmneYwXEGq5MTWyBh6FeG3qLzhXR7kPRhycQCP9vtetwhMFbZiwV9NETtifi43tWvdlwIUtQGIR+FSnafvS/SZNKbvNOQDNJfX/W/RaVeZPiS5usWxfIA2mK0cp5HICLUqbQ1yJCvVXj7o9rh0tvm19dYm+6Wgx9gnK79D1F73fceN3+Z/F0t4mkDBFvsImNY9Ve3xuIErs37z6OkqRLiKr3cWMmRTGsULJT+FoG0QavXHgq0NeIAsGsUDsghDseBvNPyZKQBmhQ4Y7feAvhwst0v4M4MCYFa8CfbQsex7k3QJzb3TNhN9YusBnifCLhzwINqA7uZSipKIgWK9DWhsTq0AJLmVp3SbTJg6vBt4ewgaNk2dJVnZslGcbH3vPCizrvpPBv3C4JB7kaVT3T9hoiS59qWbnrWqctMMoPNgCzs2cihaORmiocDyy2kSObIo8BB1/3ucsvnnkbbb9O8B2uB0Bh02UIh2xkvzs7eijOA9jgC/NYgOzMQhDi/Nlb4H+tj8cr21873zAGVWWP2I84Gzv86o+w7jxm9z+QYWybYcjdulVLft+PcxL27tUbM9VnEbrir1x5u2MCKXU7OlS7LNTabvxm1vkm1aTm69Ure9S+7/WftqOu94n6b6r255df80bnvWYcu5uB922TrplM4Ph25bMD34VmFfr4fnfNtv2pYr6SN1dVOxi+X+VXd+SZ+VNn73Ww+eYACdV0V225Ktuj1hh3LcEx2fIW6zNVsxdmiDp+zP12uxvuq3ZKvZHq+lbA9Nj77erJsTY3WlTVRtej/7c5tjwuWR4mSZDwu5EFVULD9ZShKgbVzsRBViFuLoO3NLl0By/1Is+JuyhClWrg5RlZEW5VgOMjiChrjqj/ouRis7EDlYyMYx5cJL2kkSjRbq2ld5y5fAhKjm4mdGUp78eZGDF8OD7m5uLnozPAFM1YZi+WdGR6r9TbF8hKVdEPL1prshwtLNAlNEvlrqznkYpiuELiCqaSS3JdJYg6Bt4O5EqEdD2dt+pTKOtviBo2QD6LydAKaoFqLzuhuiMPT8Sjl+h3KfNZIoembXy/ZaseCHxbSEIdrgdBXCRa4vZONBvqxJulHtRiyxvc8/QyLobuvmNC02v0O9tTJdwbcWkPP6LI3FrXXfYdy4Nv8dDofDtQtBCCGEEELINeHHOwghhBBCyOihU0wIIYQQQkYPnWJCCCGEEDJ66BQTQgghhJDRQ6eYEEIIIYSMHjrFhBBCCCFk9NApJoQQQggho4dOMSGEEEIIGT2/6BRHcBYiRHEBJ/q9u5KfJIApihBzPzM4Q0zkYJGTsahtIOm9TDTeIjBzZSm3s1JZzyro/RCY+Xo5obNWWvptqe7yao2cRaFtjEHvp9tgd1LdVfpCQb4IceGgtreY7ec8BCV9nNW8WmWkfeC0PhvrbJRcOt6XdF6ol+qxevsyLpvfmbS9n6uPE9e32/yqz3BzPuGvfEz638dh/jw/fHw8wDfhSTM9vr2eu6j4/fp/H4d5+Xv2idyPj3mj/K/X59ZvrM+zhvfv8DHP//1YlL8///X6XPwefW9Z87j//qs5dqI///uY39x37X+S022wB4m9nJfbaKV/xe24qOO4rY9C7+m4cmzo/ceXDjK+XvP6rNN5ak8+SnZmpFw83tfY59a6LV8zLpvfnUQv8/Nsc9wXPmr6QEy7zf86vN64D/gLkeIIzssOy3ADTfr5u5Er8jSBgj32PWZ9kbOGrVj4m37/XNCw1AF7nUZjApjrCfxwhT/NQrC2S99YzxNsYUPHMvvgvQBtqcMz3i+IoN4qAd4ND/ry+F376V8LimfgvffDBng3AOttiWrXTY75G2RqHTNtbbAHQax0LEtKj/Z7QJngKfuPAKl8jrOGrbsIV5eW4vYJ3g14+vLY/qZ/YSkejB4NvVVG5GBtK7COBgpabKCy6FbkvGC3DLHhAIdhxvtv7DxAknKG5WkC5dQlwTsML2fjR2Xzu5Pah7fZGRcHJtYTH+GqcSR+CC52iuNlumL4O/5fumQrQNtcPlCQ2yf63MJTZvjT2UmK8Ln1oMz+ZA5cPAgB8Hb4BgBMsdocHbzG++qz022s4EzgLAf+LkgGg1lOGcG7AQ/AvufDBqYKO+8wlO/Tq64fm05tsAuBCdXOD+ZHBG0J3TMgp8v3gQm14LDF/UmfjcHaBtjaKD5r8A4jbugd00XaZVRtWgQnNlDYxQYKgrbBCOYgHRlivJ/ir6XAVtOUiQjOS2nyUiCpk3L/G4vN70rk4MVAzl70ZLrCZgQRkIud4ukqhKt7MF5iQx05C6gDRUzIHZDLKZUNCW6LA1tHGhGInAVEeYelb/UyXt87D8oExTyzfB7T06QSKU0dxYckGwzi/K31xIerA146inchjXw2jPbRfg9IEr7bcldtdaDc5tumtQ12Ih7cdbfJdk6xCkO4kgFZFCGqgBvmI/Xf2HkKJmjO834sFEwSrycwRYjrCfy4oaN7S+8gQ5JimxY5WIgydksfltJ/kkm6I2gbhL6FvSpCFGVsZ37z6kf0ia2nFJ29sdn8DqQrUD/u17bafA+G3PYO0fUYJH1iuvJhwcCLacYzEZ8O8WgQNGzCEGEYIvQnWJ81CMcvTsi7JcJsMiVB6tF5PWMNvCXlCF3otnrsbIKGjavDVo8dcTuxTi/H3TuBCVFcY+KH2exemTy1XJQSR2Zg/T3dj20V21mq89gGyLnKF7RNciz+uboN9YEd45NtsAOR8wIDuVSi6hlwFiLUvQU/02e5v3mIi5GUw9Vhqzf2IsugHCd+YTohL0cIB5CRTdjDMIsKS30MFOlHYEKUDUhuiNC3AENufqm0nAIDjNPmn+LECtSQtNv8eGKfnZPU7S05xgPlFAvQ3izAtn9nJkJukzQfeNvP7bFVGbtleIwEfO96z+iVQruLl9+87efRiE5Xhc66knbwejred4NnQF5P4GdRxAj7fY/rg3cYsPDW1pF1N7dsnOZabhud3unffisA90ZrGzxJmr/dvNISOS8wPD1bjYlX6fL59wCgFGUkObLbz0dUugdDLk78ol4NvaMMWy1N2OOcV/JTBDBVG4rlx/ZF0LDxG96LqOR85xiTzT9J2wrUz9Fq8xOfodcq5g/zv2HEBDBlA5LrAqoMUwqZYzViukckBfyZKQBmhehYtN8D+rJzB36aKEmnKlm7dNmzhmBrA7r7eCsa0xl02MAy71zFy+qzt26jQbC1AQ+QRaN4QBZhKBb8jQZBkoDtHhGmRR2fitIlk51z3vG4dc5pgwWCLexY6ShqXYZoKLD8DaSdByizgn6fJgqQ+XBPmChJrmulGI/mCUwx0wEbxQjh986DMnvrmMLVQcafGRQAs6KBwh46lg9nPG6EaI89Sm1WkGpe9k2ixIqFLqbtYW1+G9Enth7geSLswgEPog3o7g/6a602PwnY3NI7qpdvYFHaoubU1innbJlD7ofG+k22iao7lmzBlm2VU7clW3Zqw5Zs5W2Vytu8VYo50NZZN0r5+Zq2ZIv12WX7rrptdMo6btvy6MG3CuvaBpP23aX9fb22bUtV1Wl5O77ztkm8E75eD895HTfZn1M6b5WRjG+ZTuu2ZMuEcUu2PC3jfbP9qeq4fvvH03a+WJTHtvl9ad467cRYfbz6RB+okXXivLZtPa/BhZHiCM5CTpb0kqnGdAVXF6HKCyDZrikwRai5KYqRREN+dIZCfp7AhJivWOhww02/mbigYeMDC1lGGpgstou0jaV/e1BFG4By3A5M0LBxdxBzUbaTMnQXYfi4DW+6CuFCTPQEII3uDnsXrHwLC1mEiPQ2fuHt5HK/190QD6v21jY4ANMVfGsBOR9NLm2/JmgbuDsxF+V/4JeepyuELiCqaQRM6b9FYKsMAVpsoI46LW95V7aDRmLLRrI1XpnLx3sB2sbFTlQhZnKqdRuYKmzFgl8rc1w2/1eIHCzk3MuKXlI/ufGl1eaXZSgW/HDoseky/jscDodrF4IQQgghhJBr8oufeSaEEEIIIeQ2oVNMCCGEEEJGD51iQgghhBAyeugUE0IIIYSQ0UOnmBBCCCGEjB46xYQQQgghZPTQKSaEEEIIIaPn/wHRMqTg2BwrlAAAAABJRU5ErkJggg==)"
      ],
      "metadata": {
        "id": "aCNuCo_46pIT"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "This is what you should look for if you want to do a hypothesis test in statsmodels. Column \"t\" gives us the test-statistics, Column \"P>|t|\" gives us the p-value. Columns \"0.025\" and \"0.975\" gives us the confidence interval for regression parameters. It's exactly the results that we got when using previously written classes."
      ],
      "metadata": {
        "id": "SsxzqQRT6HN3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x = [100,102,103,101,105,100,99,105]\n",
        "y = [259,264,274,268,277,263,258,275]\n",
        "\n",
        "test_intercept_unknown_variance(null_a = 0, type_t = 'two_tailed', alpha = 0.05, data_x = x, data_y = y);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LcKee-1f6AJr",
        "outputId": "f7b258cc-5c1b-4a11-80aa-134183715fdd"
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "test_statistic: -0.8461239472569388 \n",
            "\n",
            "P_value: 0.4299305678887875 \n",
            "\n",
            "Since p_value > 0.05, the null hypothesis cannot be rejected.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "x = [100,102,103,101,105,100,99,105]\n",
        "y = [259,264,274,268,277,263,258,275]\n",
        "\n",
        "test_coefficient_unknown_variance(null_b = 0, type_t = 'two_tailed', alpha = 0.05, data_x = x, data_y = y);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vGkI9EnO7sYn",
        "outputId": "154b931f-90b2-4d97-e3b2-ac4bec61e66d"
      },
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "test_statistic: 6.486393472746325 \n",
            "\n",
            "P_value: 0.0006383332228794281 \n",
            "\n",
            "Since p_value < 0.05, reject null hypothesis.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "If we have more than one $X$, the written classes need a little tweak, therefore it is better to use the statsmodels library. One difference in this situation is the degree of freedom in the p-value for t-tests. $t_{n-2}$ is correct when we have 1 independent variable but if we have $k$ independent variables, the degree of freedom in the p-value for t-tests is $n-k-1$."
      ],
      "metadata": {
        "id": "PSL5brgk8GB3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "Stock_Market = {'Year': [2017,2017,2017,2017,2017,2017,2017,2017,2017,2017,2017,2017,2016,2016,2016,2016,2016,2016,2016,2016,2016,2016,2016,2016],\n",
        "                'Month': [12, 11,10,9,8,7,6,5,4,3,2,1,12,11,10,9,8,7,6,5,4,3,2,1],\n",
        "                'Interest_Rate': [2.75,2.5,2.5,2.5,2.5,2.5,2.5,2.25,2.25,2.25,2,2,2,1.75,1.75,1.75,1.75,1.75,1.75,1.75,1.75,1.75,1.75,1.75],\n",
        "                'Unemployment_Rate': [5.3,5.3,5.3,5.3,5.4,5.6,5.5,5.5,5.5,5.6,5.7,5.9,6,5.9,5.8,6.1,6.2,6.1,6.1,6.1,5.9,6.2,6.2,6.1],\n",
        "                'Stock_Index_Price': [1464,1394,1357,1293,1256,1254,1234,1195,1159,1167,1130,1075,1047,965,943,958,971,949,884,866,876,822,704,719]        \n",
        "                }\n",
        "df = pd.DataFrame(Stock_Market,columns=['Year','Month','Interest_Rate','Unemployment_Rate','Stock_Index_Price'])\n",
        "Y = df['Stock_Index_Price']\n",
        "X = df[['Interest_Rate','Unemployment_Rate']]\n",
        "\n",
        "X = sm.add_constant(X)\n",
        "model = sm.OLS(Y, X)\n",
        "results = model.fit()\n",
        "print(results.summary())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gyJDj6NH8fDG",
        "outputId": "030cbfda-1180-4208-bc22-86b268372303"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "                            OLS Regression Results                            \n",
            "==============================================================================\n",
            "Dep. Variable:      Stock_Index_Price   R-squared:                       0.898\n",
            "Model:                            OLS   Adj. R-squared:                  0.888\n",
            "Method:                 Least Squares   F-statistic:                     92.07\n",
            "Date:                Mon, 20 Jun 2022   Prob (F-statistic):           4.04e-11\n",
            "Time:                        10:23:34   Log-Likelihood:                -134.61\n",
            "No. Observations:                  24   AIC:                             275.2\n",
            "Df Residuals:                      21   BIC:                             278.8\n",
            "Df Model:                           2                                         \n",
            "Covariance Type:            nonrobust                                         \n",
            "=====================================================================================\n",
            "                        coef    std err          t      P>|t|      [0.025      0.975]\n",
            "-------------------------------------------------------------------------------------\n",
            "const              1798.4040    899.248      2.000      0.059     -71.685    3668.493\n",
            "Interest_Rate       345.5401    111.367      3.103      0.005     113.940     577.140\n",
            "Unemployment_Rate  -250.1466    117.950     -2.121      0.046    -495.437      -4.856\n",
            "==============================================================================\n",
            "Omnibus:                        2.691   Durbin-Watson:                   0.530\n",
            "Prob(Omnibus):                  0.260   Jarque-Bera (JB):                1.551\n",
            "Skew:                          -0.612   Prob(JB):                        0.461\n",
            "Kurtosis:                       3.226   Cond. No.                         394.\n",
            "==============================================================================\n",
            "\n",
            "Warnings:\n",
            "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='F-statistic_for_Overall_Significance_in_Regression'></a>\n",
        "\n",
        "### **6.3.4. F-statistic for Overall Significance in Regression:**"
      ],
      "metadata": {
        "id": "GE4h6bbEnVfB"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The F-statistic provides us with a way for globally testing if ANY of the independent variables $X_1, X_2, X_3, ...$ is related to the outcome $Y$.\n",
        "\n",
        "$H_0$ : The model with no predictor variables (also known as an intercept-only model) fits the data as well as your regression model.\n",
        "\n",
        "$B_1 = B_2 = ... = B_n = 0$\n",
        "\n",
        "$H_1$ : Regression model fits the data better than the intercept-only model.\n",
        "\n",
        "not all $B_1 = B_2 = ... = B_n = 0$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Testing statistics: \n",
        "\n",
        "$F_0 \\equiv \\frac{MSR}{MSE}$\n",
        "\n",
        "$MSR = \\frac{SSR}{1} \\qquad MSE = \\frac{SSE}{n-2}$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "We accept $H_0$ if:\n",
        "\n",
        "1. P_value $ = P(F_{1,n-2} \\geq F_0) > \\alpha$"
      ],
      "metadata": {
        "id": "xoJTUafHoicd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "class f_regression:\n",
        "  \"\"\"\n",
        "  Parameters\n",
        "  ----------\n",
        "  null_v : v0\n",
        "  n1 : optional, number of data1 members\n",
        "  n2 : optional, number of data2 members\n",
        "  S2X : sample1 variance\n",
        "  S2Y : sample2 variance\n",
        "  alpha : significance level\n",
        "  type_t : 'two_tailed', 'right_tailed', 'left_tailed'\n",
        "  data1 : optional, if you do not know the S2X, just pass the data\n",
        "  data2 : optional, if you do not know the S2Y, just pass the data\n",
        "  \"\"\"\n",
        "  def __init__(self, alpha, n = 0., S_xx = 0., S_yy = 0., S_xy = 0., data_x=None, data_y=None):\n",
        "    self.S_yy = S_yy\n",
        "    self.S_xx = S_xx\n",
        "    self.S_xy = S_xy\n",
        "    self.n = n\n",
        "    self.alpha = alpha\n",
        "    self.data_x = data_x\n",
        "    self.data_y = data_y\n",
        "    if data_x is not None:\n",
        "      self.n = len(list(data_x))\n",
        "      xx = []\n",
        "      for i in list(data_x):\n",
        "        xx.append((i - np.mean(list(data_x)))**2)\n",
        "      self.S_xx = np.sum(xx)\n",
        "\n",
        "      yy = []\n",
        "      for i in list(data_y):\n",
        "        yy.append((i - np.mean(list(data_y)))**2)\n",
        "      self.S_yy = np.sum(yy)\n",
        "\n",
        "      xy = []\n",
        "      for i in range(len(data_x)):\n",
        "        xy.append((data_x[i] - np.mean(data_x))* (data_y[i] - np.mean(data_y)))\n",
        "      self.S_xy = np.sum(xy)\n",
        "\n",
        "    f_regression.__test(self)\n",
        "  \n",
        "  def __test(self):\n",
        "    SSR = (self.S_xy**2) / self.S_xx\n",
        "    SSE = self.S_yy - ((self.S_xy**2) / self.S_xx)\n",
        "    test_statistic = (SSR/1) / (SSE/(self.n-2))\n",
        "    print('test_statistic:', test_statistic, '\\n')\n",
        "    \n",
        "    p_value = 1-stats.f.cdf(test_statistic, 1, self.n-2)\n",
        "    print('P_value:', p_value, '\\n')\n",
        "\n",
        "    if p_value < self.alpha:\n",
        "      print(f'Since p_value < {self.alpha}, reject null hypothesis')\n",
        "    else:\n",
        "      print(f'Since p_value > {self.alpha}, the null hypothesis cannot be rejected.')"
      ],
      "metadata": {
        "id": "svGNq--soXYE"
      },
      "execution_count": 20,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x = [100,102,103,101,105,100,99,105]\n",
        "y = [259,264,274,268,277,263,258,275]\n",
        "\n",
        "f_regression(alpha = 0.05, data_x = x, data_y = y);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dTByO0wKwLzn",
        "outputId": "bad321bb-d464-4f77-c941-7d4b992b1a93"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "test_statistic: 42.07330028328612 \n",
            "\n",
            "P_value: 0.0006383332228794281 \n",
            "\n",
            "Since p_value < 0.05, reject null hypothesis\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This test can also be done by statsmodels library. It can also do the test when we have more than one independent variable. The previously written need a little tweak for more than one independent variable (just degree of freedom differs.)"
      ],
      "metadata": {
        "id": "ipxqshPJ2Ghm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x = [100,102,103,101,105,100,99,105]\n",
        "y = [259,264,274,268,277,263,258,275]\n",
        "\n",
        "x = sm.add_constant(x)\n",
        "model = sm.OLS(y, x)\n",
        "reg = model.fit()\n",
        "print(reg.summary())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WGVCNFngwk8-",
        "outputId": "fe0fe516-4277-497b-e8f7-d344010b0ed5"
      },
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "                            OLS Regression Results                            \n",
            "==============================================================================\n",
            "Dep. Variable:                      y   R-squared:                       0.875\n",
            "Model:                            OLS   Adj. R-squared:                  0.854\n",
            "Method:                 Least Squares   F-statistic:                     42.07\n",
            "Date:                Mon, 20 Jun 2022   Prob (F-statistic):           0.000638\n",
            "Time:                        10:23:34   Log-Likelihood:                -18.507\n",
            "No. Observations:                   8   AIC:                             41.01\n",
            "Df Residuals:                       6   BIC:                             41.17\n",
            "Df Model:                           1                                         \n",
            "Covariance Type:            nonrobust                                         \n",
            "==============================================================================\n",
            "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
            "------------------------------------------------------------------------------\n",
            "const        -40.1017     47.395     -0.846      0.430    -156.072      75.869\n",
            "x1             3.0169      0.465      6.486      0.001       1.879       4.155\n",
            "==============================================================================\n",
            "Omnibus:                        0.896   Durbin-Watson:                   1.370\n",
            "Prob(Omnibus):                  0.639   Jarque-Bera (JB):                0.565\n",
            "Skew:                           0.081   Prob(JB):                        0.754\n",
            "Kurtosis:                       1.708   Cond. No.                     4.84e+03\n",
            "==============================================================================\n",
            "\n",
            "Warnings:\n",
            "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
            "[2] The condition number is large, 4.84e+03. This might indicate that there are\n",
            "strong multicollinearity or other numerical problems.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(f'f-statistic: {reg.fvalue}, p-value: {reg.f_pvalue}')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "J9mDOvzEz2_k",
        "outputId": "528d3e27-9696-4ad6-b62a-2310cc73397a"
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "f-statistic: 42.07330028328628, p-value: 0.0006383332228793835\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Confidence_Intervals_Concerning_Regression_Models'></a>\n",
        "\n",
        "## **6.4. Confidence Intervals Concerning Regression Models:**"
      ],
      "metadata": {
        "id": "p0eHvt-6UrwD"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Confidence_Interval_for_B'></a>\n",
        "\n",
        "### **6.4.1. Confidence Interval for $B$:**"
      ],
      "metadata": {
        "id": "n0PxHd5AVxOU"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Known_Variance_B'></a>\n",
        "\n",
        "#### **6.4.1.1. Known Variance:**"
      ],
      "metadata": {
        "id": "C9ms4Iuhe9GZ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**A. Two-sided Confidence Interval:**\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "$P(b\\ -\\ Z_{\\frac{\\alpha}{2}} \\frac{\\sigma}{\\sqrt{S_{xx}}} \\leq B \\leq b\\ +\\ Z_{\\frac{\\alpha}{2}} \\frac{\\sigma}{\\sqrt{S_{xx}}}) = 1-\\alpha $\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Therefore, the $1-\\alpha$ confidence interval for $B$ is:\n",
        "\n",
        "$[b\\ -\\ Z_{\\frac{\\alpha}{2}} \\frac{\\sigma}{\\sqrt{S_{xx}}}\\ ,\\ b\\ +\\ Z_{\\frac{\\alpha}{2}} \\frac{\\sigma}{\\sqrt{S_{xx}}}]$"
      ],
      "metadata": {
        "id": "MTyHJGB1WS8i"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**B. One-sided Lower Confidence Interval:**\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "$P(-\\infty\\ \\leq\\ B\\ \\leq\\ b\\ +\\ Z_{\\alpha} \\frac{\\sigma}{\\sqrt{S_{xx}}}) = 1-\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Therefore, the $1-\\alpha$ confidence interval for for the mean of a normal population is:\n",
        "\n",
        "$[-\\infty,\\ b\\ +\\ Z_{\\alpha} \\frac{\\sigma}{\\sqrt{S_{xx}}}]$"
      ],
      "metadata": {
        "id": "fooeZEfhXpTs"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**C. One-sided Upper Confidence Interval:**\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "$P(b\\ -\\ Z_{\\alpha} \\frac{\\sigma}{\\sqrt{S_{xx}}}\\ \\leq\\ B\\ \\leq\\ \\infty) = 1-\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Therefore, the $1-\\alpha$ confidence interval for for the mean of a normal population is:\n",
        "\n",
        "$[b\\ -\\ Z_{\\alpha} \\frac{\\sigma}{\\sqrt{S_{xx}}},\\ \\infty]$"
      ],
      "metadata": {
        "id": "q4SkjC0_YKkb"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "class confidence_interval_for_b_known_variance:\n",
        "  \"\"\"\n",
        "  Parameters\n",
        "  ----------\n",
        "  population_sd : known standrad deviation of the population\n",
        "  c_level : % confidence level\n",
        "  type_t : 'two_sided_confidence', 'lower_confidence', 'upper_confidence'\n",
        "  S_xx\n",
        "  S_xy\n",
        "  data_x : optional\n",
        "  data_y : optional\n",
        "  \"\"\"\n",
        "  def __init__(self, population_sd, c_level, type_c, S_xx = 0., S_xy = 0., data_x=None, data_y=None):\n",
        "    self.population_sd = population_sd\n",
        "    self.type_c = type_c\n",
        "    self.c_level = c_level\n",
        "    self.data_x = data_x\n",
        "    self.S_xx = S_xx\n",
        "    self.S_xy = S_xy\n",
        "    self.data_y = data_y\n",
        "    if data_x is not None:\n",
        "      self.n = len(list(data_x))\n",
        "      xx = []\n",
        "      for i in list(data_x):\n",
        "        xx.append((i - np.mean(list(data_x)))**2)\n",
        "      self.S_xx = np.sum(xx)\n",
        "\n",
        "      xy = []\n",
        "      for i in range(len(data_x)):\n",
        "        xy.append((data_x[i] - np.mean(data_x))* (data_y[i] - np.mean(data_y)))\n",
        "      self.S_xy = np.sum(xy)\n",
        "      self.Syx=None\n",
        "\n",
        "    confidence_interval_for_b_known_variance.__test(self)\n",
        "  \n",
        "  def __test(self):\n",
        "    if self.type_c == 'two_sided_confidence':\n",
        "      b = self.S_xy / self.S_xx\n",
        "      c_u = b + (-norm.ppf((1-self.c_level)/2)) * (self.population_sd/np.sqrt(self.S_xx))\n",
        "      c_l = b - (-norm.ppf((1-self.c_level)/2)) * (self.population_sd/np.sqrt(self.S_xx))\n",
        "      display(Latex(f'${c_l} \\leq B \\leq {c_u}$'))\n",
        "    elif self.type_c == 'lower_confidence':\n",
        "      b = self.S_xy / self.S_xx\n",
        "      c_u = b + (-norm.ppf(1-self.c_level)) * (self.population_sd/np.sqrt(self.S_xx))\n",
        "      display(Latex(f'$B \\leq {c_u}$'))\n",
        "    elif self.type_c == 'upper_confidence':\n",
        "      b = self.S_xy / self.S_xx\n",
        "      c_l = b - (-norm.ppf(1-self.c_level)) * (self.population_sd/np.sqrt(self.S_xx))\n",
        "      display(Latex(f'${c_l} \\leq B$'))"
      ],
      "metadata": {
        "id": "e6FyrZPhUeKb"
      },
      "execution_count": 24,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x = [100,102,103,101,105,100,99,105]\n",
        "y = [259,264,274,268,277,263,258,275]\n",
        "\n",
        "confidence_interval_for_b_known_variance(population_sd = 1, c_level = 0.95, type_c = 'two_sided_confidence', data_x=x, data_y=y);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "hAwj8K5VYndX",
        "outputId": "a0f64a71-198b-4ae3-e4a8-71f31f306cbf"
      },
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.Latex object>"
            ],
            "text/latex": "$2.694187391394151 \\leq B \\leq 3.3397109136905945$"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Unknown_Variance_B'></a>\n",
        "\n",
        "#### **6.4.1.2. Unknown Variance**"
      ],
      "metadata": {
        "id": "S8ofDynCfD6B"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**A. Two-sided Confidence Interval:**\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "$P(b -\\ t_{\\frac{\\alpha}{2},n-2} \\frac{S_{y|x}}{\\sqrt{S_{xx}}}\\ <\\ B <\\ b +\\ t_{\\frac{\\alpha}{2},n-2} \\frac{S_{y|x}}{\\sqrt{S_{xx}}}) = 1-\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Therefore, the $1-\\alpha$ confidence interval for for the mean of a normal population is:\n",
        "\n",
        "$[b -\\ t_{\\frac{\\alpha}{2},n-2} \\frac{S_{y|x}}{\\sqrt{S_{xx}}}\\ ,\\ b +\\ t_{\\frac{\\alpha}{2},n-2} \\frac{S_{y|x}}{\\sqrt{S_{xx}}}]$"
      ],
      "metadata": {
        "id": "yj7VzwVhfYoZ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**B. One-sided Lower Confidence Interval:**\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "$P(-\\infty\\ \\leq\\ B\\ \\leq\\ b +\\ t_{\\alpha,n-2} \\frac{S_{y|x}}{\\sqrt{S_{xx}}}) = 1-\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Therefore, the $1-\\alpha$ confidence interval for for the mean of a normal population is:\n",
        "\n",
        "$[-\\infty,\\ b +\\ t_{\\alpha,n-2} \\frac{S_{y|x}}{\\sqrt{S_{xx}}}]$"
      ],
      "metadata": {
        "id": "C5FvqPJ1gpim"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**C. One-sided Upper Confidence Interval:**\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "$P(b -\\ t_{\\alpha,n-2} \\frac{S_{y|x}}{\\sqrt{S_{xx}}}\\ \\leq\\ B\\ \\leq\\ \\infty) = 1-\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Therefore, the $1-\\alpha$ confidence interval for for the mean of a normal population is:\n",
        "\n",
        "$[b -\\ t_{\\alpha,n-2} \\frac{S_{y|x}}{\\sqrt{S_{xx}}},\\ \\infty]$"
      ],
      "metadata": {
        "id": "x73df6QJg391"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "class confidence_interval_for_b_unknown_variance:\n",
        "  \"\"\"\n",
        "  Parameters\n",
        "  ----------\n",
        "  c_level : % confidence level\n",
        "  type_t : 'two_sided_confidence', 'lower_confidence', 'upper_confidence'\n",
        "  Sample_std : optional, std of the sample\n",
        "  Sample_mean : optional, mean of the sample\n",
        "  data : optional, if you do not know the Sample_mean and n, just pass the data\n",
        "  \"\"\"\n",
        "  def __init__(self, c_level, type_c, n = 0., S_xx = 0., S_xy = 0., S_yy = 0., data_x=None, data_y=None):\n",
        "    self.type_c = type_c\n",
        "    self.c_level = c_level\n",
        "    self.n = n\n",
        "    self.S_xx = S_xx\n",
        "    self.S_xy = S_xy\n",
        "    self.S_yy = S_yy\n",
        "    self.data_x = data_x\n",
        "    self.data_y = data_y\n",
        "    if data_x is not None:\n",
        "      self.n = len(list(data_x))\n",
        "      xx = []\n",
        "      for i in list(data_x):\n",
        "        xx.append((i - np.mean(list(data_x)))**2)\n",
        "      self.S_xx = np.sum(xx)\n",
        "\n",
        "      yy = []\n",
        "      for i in list(data_y):\n",
        "        yy.append((i - np.mean(list(data_y)))**2)\n",
        "      self.S_yy = np.sum(yy)\n",
        "\n",
        "      xy = []\n",
        "      for i in range(len(data_x)):\n",
        "        xy.append((data_x[i] - np.mean(data_x))* (data_y[i] - np.mean(data_y)))\n",
        "      self.S_xy = np.sum(xy)\n",
        "      self.Syx=None\n",
        "\n",
        "    confidence_interval_for_b_unknown_variance.__test(self)\n",
        "  \n",
        "  def __test(self):\n",
        "    if self.type_c == 'two_sided_confidence':\n",
        "      b = self.S_xy / self.S_xx\n",
        "      self.Syx = np.sqrt(((self.S_yy) - ((self.S_xy)**2 / (self.S_xx))) / (self.n-2))\n",
        "      c_u = b + (t.isf((1-self.c_level)/2, self.n-2)) * (self.Syx/np.sqrt(self.S_xx))\n",
        "      c_l = b - (t.isf((1-self.c_level)/2, self.n-2)) * (self.Syx/np.sqrt(self.S_xx))\n",
        "      display(Latex(f'${c_l} \\leq B \\leq {c_u}$'))\n",
        "    elif self.type_c == 'lower_confidence':\n",
        "      b = self.S_xy / self.S_xx\n",
        "      self.Syx = np.sqrt(((self.S_yy) - ((self.S_xy)**2 / (self.S_xx))) / (self.n-2))\n",
        "      c_u = b + (t.isf(1-self.c_level, self.n-2)) * (self.Syx/np.sqrt(self.S_xx))\n",
        "      display(Latex(f'$B \\leq {c_u}$'))\n",
        "    elif self.type_c == 'upper_confidence':\n",
        "      b = self.S_xy / self.S_xx\n",
        "      self.Syx = np.sqrt(((self.S_yy) - ((self.S_xy)**2 / (self.S_xx))) / (self.n-2))\n",
        "      c_l = b - (t.isf(1-self.c_level, self.n-2)) * (self.Syx/np.sqrt(self.S_xx))\n",
        "      display(Latex(f'${c_l} \\leq B$'))"
      ],
      "metadata": {
        "id": "ZqBJ5RZGhVvH"
      },
      "execution_count": 26,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x = [100,102,103,101,105,100,99,105]\n",
        "y = [259,264,274,268,277,263,258,275]\n",
        "\n",
        "confidence_interval_for_b_unknown_variance(c_level = 0.95, type_c = 'two_sided_confidence', data_x=x, data_y=y);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "lToPrtspja8H",
        "outputId": "8f9da47e-06ec-49a3-eb1f-c970dfc5bdc5"
      },
      "execution_count": 27,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.Latex object>"
            ],
            "text/latex": "$1.878842335622378 \\leq B \\leq 4.155055969462367$"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Confidence_Interval_for_A'></a>\n",
        "\n",
        "### **6.4.2. Confidence Interval for $A$:**"
      ],
      "metadata": {
        "id": "g7CrxN4VkI17"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Unknown_Variance_AA'></a>\n",
        "\n",
        "#### **6.4.2.1. Unknown Variance:**"
      ],
      "metadata": {
        "id": "iJRSxN9PkmtR"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**A. Two-sided Confidence Interval:**\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "$P(a -\\ t_{\\frac{\\alpha}{2},n-2}\\ {S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{\\overline{x}^2}{S_{xx}}}}\\ <\\ A <\\ a +\\ t_{\\frac{\\alpha}{2},n-2}\\ {S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{\\overline{x}^2}{S_{xx}}}}) = 1-\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Therefore, the $1-\\alpha$ confidence interval for for the mean of a normal population is:\n",
        "\n",
        "$[a -\\ t_{\\frac{\\alpha}{2},n-2} {S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{\\overline{x}^2}{S_{xx}}}}\\ ,\\ a +\\ t_{\\frac{\\alpha}{2},n-2} {S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{\\overline{x}^2}{S_{xx}}}}]$"
      ],
      "metadata": {
        "id": "5pZuakcyke_u"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**B. One-sided Lower Confidence Interval:**\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "$P(-\\infty\\ \\leq\\ A\\ \\leq\\ a +\\ t_{\\alpha,n-2}\\ {S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{\\overline{x}^2}{S_{xx}}}}) = 1-\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Therefore, the $1-\\alpha$ confidence interval for for the mean of a normal population is:\n",
        "\n",
        "$[-\\infty,\\ a +\\ t_{\\alpha,n-2}\\ {S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{\\overline{x}^2}{S_{xx}}}}]$"
      ],
      "metadata": {
        "id": "1-EJt9QalZf5"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**C. One-sided Upper Confidence Interval:**\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "$P(a -\\ t_{\\alpha,n-2}\\ {S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{\\overline{x}^2}{S_{xx}}}}\\ \\leq\\ B\\ \\leq\\ \\infty) = 1-\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Therefore, the $1-\\alpha$ confidence interval for for the mean of a normal population is:\n",
        "\n",
        "$[a -\\ t_{\\alpha,n-2}\\ {S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{\\overline{x}^2}{S_{xx}}}},\\ \\infty]$"
      ],
      "metadata": {
        "id": "IJCAZjtqlu0H"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "class confidence_interval_for_a_unknown_variance:\n",
        "  \"\"\"\n",
        "  Parameters\n",
        "  ----------\n",
        "  c_level : % confidence level\n",
        "  type_t : 'two_sided_confidence', 'lower_confidence', 'upper_confidence'\n",
        "  S_xx : optional\n",
        "  S_yy : optional\n",
        "  S_xy : optional\n",
        "  data : optional\n",
        "  \"\"\"\n",
        "  def __init__(self, c_level, type_c, n = 0., S_xx = 0., S_xy = 0., S_yy = 0., data_x=None, data_y=None):\n",
        "    self.type_c = type_c\n",
        "    self.c_level = c_level\n",
        "    self.n = n\n",
        "    self.S_xx = S_xx\n",
        "    self.S_xy = S_xy\n",
        "    self.S_yy = S_yy\n",
        "    self.data_x = data_x\n",
        "    self.data_y = data_y\n",
        "    if data_x is not None:\n",
        "      self.n = len(list(data_x))\n",
        "      xx = []\n",
        "      for i in list(data_x):\n",
        "        xx.append((i - np.mean(list(data_x)))**2)\n",
        "      self.S_xx = np.sum(xx)\n",
        "\n",
        "      yy = []\n",
        "      for i in list(data_y):\n",
        "        yy.append((i - np.mean(list(data_y)))**2)\n",
        "      self.S_yy = np.sum(yy)\n",
        "\n",
        "      xy = []\n",
        "      for i in range(len(data_x)):\n",
        "        xy.append((data_x[i] - np.mean(data_x))* (data_y[i] - np.mean(data_y)))\n",
        "      self.S_xy = np.sum(xy)\n",
        "      self.Syx=None\n",
        "\n",
        "    confidence_interval_for_a_unknown_variance.__test(self)\n",
        "  \n",
        "  def __test(self):\n",
        "    if self.type_c == 'two_sided_confidence':\n",
        "      b = self.S_xy / self.S_xx\n",
        "      a = np.mean(self.data_y) - b * np.mean(self.data_x)\n",
        "      self.Syx = np.sqrt(((self.S_yy) - ((self.S_xy)**2 / (self.S_xx))) / (self.n-2))\n",
        "      c_u = a + (t.isf((1-self.c_level)/2, self.n-2)) * self.Syx * np.sqrt((1/self.n) + ((np.mean(self.data_x)**2) / self.S_xx))\n",
        "      c_l = a - (t.isf((1-self.c_level)/2, self.n-2)) * self.Syx * np.sqrt((1/self.n) + ((np.mean(self.data_x)**2) / self.S_xx))\n",
        "      display(Latex(f'${c_l} \\leq A \\leq {c_u}$'))\n",
        "    elif self.type_c == 'lower_confidence':\n",
        "      b = self.S_xy / self.S_xx\n",
        "      a = np.mean(self.data_y) - b * np.mean(self.data_x)\n",
        "      self.Syx = np.sqrt(((self.S_yy) - ((self.S_xy)**2 / (self.S_xx))) / (self.n-2))\n",
        "      c_u = a + (t.isf((1-self.c_level), self.n-2)) * self.Syx * np.sqrt((1/self.n) + ((np.mean(self.data_x)**2) / self.S_xx))\n",
        "      display(Latex(f'$A \\leq {c_u}$'))\n",
        "    elif self.type_c == 'upper_confidence':\n",
        "      b = self.S_xy / self.S_xx\n",
        "      a = np.mean(self.data_y) - b * np.mean(self.data_x)\n",
        "      self.Syx = np.sqrt(((self.S_yy) - ((self.S_xy)**2 / (self.S_xx))) / (self.n-2))\n",
        "      c_l = a - (t.isf((1-self.c_level), self.n-2)) * self.Syx * np.sqrt((1/self.n) + ((np.mean(self.data_x)**2) / self.S_xx))\n",
        "      display(Latex(f'${c_l} \\leq A$'))"
      ],
      "metadata": {
        "id": "T-WOv-fZjo9t"
      },
      "execution_count": 28,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x = [100,102,103,101,105,100,99,105]\n",
        "y = [259,264,274,268,277,263,258,275]\n",
        "\n",
        "confidence_interval_for_a_unknown_variance(c_level = 0.95, type_c = 'two_sided_confidence', data_x=x, data_y=y);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "BsjN4bBBmivv",
        "outputId": "85b1a91c-3049-4845-b0eb-c48dff8f6897"
      },
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.Latex object>"
            ],
            "text/latex": "$-156.07207107926496 \\leq A \\leq 75.86868124875643$"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Confidence_Interval_for_AB'></a>\n",
        "\n",
        "### **6.4.3. Confidence Interval for $A + B x$:**"
      ],
      "metadata": {
        "id": "aCeYYgZGb3U0"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Unknown_Variance_ABB'></a>\n",
        "\n",
        "#### **6.4.3.1. Unknown Variance:**"
      ],
      "metadata": {
        "id": "2PIRxMVzcWH0"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**A. Two-sided Confidence Interval:**\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "$P(a + bx^* - t_{\\frac{\\alpha}{2},n-2}{S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{(x^* - \\overline{x})^2}{S_{xx}}}}\\ <\\ A + Bx<\\ a + bx^* + t_{\\frac{\\alpha}{2},n-2}{S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{(x^* - \\overline{x})^2}{S_{xx}}}}\\ ) = 1-\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Therefore, the $1-\\alpha$ confidence interval for for the mean of a normal population is:\n",
        "\n",
        "$[a + bx^* - t_{\\frac{\\alpha}{2},n-2}{S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{(x^* - \\overline{x})^2}{S_{xx}}}}\\ ,\\ a + bx^* + t_{\\frac{\\alpha}{2},n-2}{S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{(x^* - \\overline{x})^2}{S_{xx}}}}]$"
      ],
      "metadata": {
        "id": "gJIbg8LWcZlZ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**B. One-sided Lower Confidence Interval:**\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "$P(-\\infty\\ \\leq\\ A + Bx<\\ a + bx^* + t_{\\alpha,n-2}{S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{(x^* - \\overline{x})^2}{S_{xx}}}}\\ ) = 1-\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Therefore, the $1-\\alpha$ confidence interval for for the mean of a normal population is:\n",
        "\n",
        "$[-\\infty\\ ,\\ a + bx^* + t_{\\alpha,n-2}{S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{(x^* - \\overline{x})^2}{S_{xx}}}}]$"
      ],
      "metadata": {
        "id": "9QrOqhCddYXp"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**C. One-sided Upper Confidence Interval:**\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "$P(a + bx^* - t_{\\alpha,n-2}{S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{(x^* - \\overline{x})^2}{S_{xx}}}}\\ \\leq\\ B\\ \\leq\\ \\infty) = 1-\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Therefore, the $1-\\alpha$ confidence interval for for the mean of a normal population is:\n",
        "\n",
        "$[a + bx^* - t_{\\alpha,n-2}{S_{y|x} \\sqrt{\\frac{1}{n} + \\frac{(x^* - \\overline{x})^2}{S_{xx}}}}\\ ,\\ \\infty]$"
      ],
      "metadata": {
        "id": "VNuZV769dtDf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "class confidence_interval_for_ab_unknown_variance:\n",
        "  \"\"\"\n",
        "  Parameters\n",
        "  ----------\n",
        "  c_level : % confidence level\n",
        "  type_t : 'two_sided_confidence', 'lower_confidence', 'upper_confidence'\n",
        "  S_xx : optional\n",
        "  S_yy : optional\n",
        "  S_xy : optional\n",
        "  data : optional\n",
        "  \"\"\"\n",
        "  def __init__(self, c_level, type_c, x_star, n = 0., S_xx = 0., S_xy = 0., S_yy = 0., data_x=None, data_y=None):\n",
        "    self.type_c = type_c\n",
        "    self.c_level = c_level\n",
        "    self.n = n\n",
        "    self.x_star = x_star\n",
        "    self.S_xx = S_xx\n",
        "    self.S_xy = S_xy\n",
        "    self.S_yy = S_yy\n",
        "    self.data_x = data_x\n",
        "    self.data_y = data_y\n",
        "    if data_x is not None:\n",
        "      self.n = len(list(data_x))\n",
        "      xx = []\n",
        "      for i in list(data_x):\n",
        "        xx.append((i - np.mean(list(data_x)))**2)\n",
        "      self.S_xx = np.sum(xx)\n",
        "\n",
        "      yy = []\n",
        "      for i in list(data_y):\n",
        "        yy.append((i - np.mean(list(data_y)))**2)\n",
        "      self.S_yy = np.sum(yy)\n",
        "\n",
        "      xy = []\n",
        "      for i in range(len(data_x)):\n",
        "        xy.append((data_x[i] - np.mean(data_x))* (data_y[i] - np.mean(data_y)))\n",
        "      self.S_xy = np.sum(xy)\n",
        "      self.Syx=None\n",
        "\n",
        "    confidence_interval_for_ab_unknown_variance.__test(self)\n",
        "  \n",
        "  def __test(self):\n",
        "    if self.type_c == 'two_sided_confidence':\n",
        "      b = self.S_xy / self.S_xx\n",
        "      a = np.mean(self.data_y) - b * np.mean(self.data_x)\n",
        "      self.Syx = np.sqrt(((self.S_yy) - ((self.S_xy)**2 / (self.S_xx))) / (self.n-2))\n",
        "      c_u = a + b*self.x_star + (t.isf((1-self.c_level)/2, self.n-2)) * self.Syx * np.sqrt((1/self.n) + (((self.x_star - np.mean(self.data_x))**2) / self.S_xx))\n",
        "      c_l = a + b*self.x_star - (t.isf((1-self.c_level)/2, self.n-2)) * self.Syx * np.sqrt((1/self.n) + (((self.x_star - np.mean(self.data_x))**2) / self.S_xx))\n",
        "      display(Latex(f'${c_l} \\leq A + Bx \\leq {c_u}$'))\n",
        "    elif self.type_c == 'lower_confidence':\n",
        "      b = self.S_xy / self.S_xx\n",
        "      a = np.mean(self.data_y) - b * np.mean(self.data_x)\n",
        "      self.Syx = np.sqrt(((self.S_yy) - ((self.S_xy)**2 / (self.S_xx))) / (self.n-2))\n",
        "      c_u = a + b*self.x_star + (t.isf((1-self.c_level)/2, self.n-2)) * self.Syx * np.sqrt((1/self.n) + (((self.x_star - np.mean(self.data_x))**2) / self.S_xx))\n",
        "      display(Latex(f'$A + Bx \\leq {c_u}$'))\n",
        "    elif self.type_c == 'upper_confidence':\n",
        "      b = self.S_xy / self.S_xx\n",
        "      a = np.mean(self.data_y) - b * np.mean(self.data_x)\n",
        "      self.Syx = np.sqrt(((self.S_yy) - ((self.S_xy)**2 / (self.S_xx))) / (self.n-2))\n",
        "      c_l = a + b*self.x_star - (t.isf((1-self.c_level)/2, self.n-2)) * self.Syx * np.sqrt((1/self.n) + (((self.x_star - np.mean(self.data_x))**2) / self.S_xx))\n",
        "      display(Latex(f'${c_l} \\leq A + Bx$'))"
      ],
      "metadata": {
        "id": "avAEJoDGcPJ4"
      },
      "execution_count": 30,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x = [100,102,103,101,105,100,99,105]\n",
        "y = [259,264,274,268,277,263,258,275]\n",
        "\n",
        "confidence_interval_for_ab_unknown_variance(x_star = 107, c_level = 0.95, type_c = 'two_sided_confidence', data_x=x, data_y=y);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "aCT3qV7HfhTy",
        "outputId": "e3c60e55-941a-4d1b-a458-25805a15cf7b"
      },
      "execution_count": 31,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.Latex object>"
            ],
            "text/latex": "$276.3879423612866 \\leq A + Bx \\leq 289.03578645227265$"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "If $x^*$ is colser to $\\overline{x}$, the confidence interval is smaller. This is shown in the next plot."
      ],
      "metadata": {
        "id": "XMksyxrRj6QX"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "plt.xlabel('X')\n",
        "plt.ylabel('Y')\n",
        "plt.title('Confidence Limits')\n",
        "sns.regplot(x, y, ci = 95, order = 1, marker = 'o', color ='#cc0066');"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "hcq1JdhJgInH",
        "outputId": "e538503e-130a-4c2a-caff-205537682851"
      },
      "execution_count": 32,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZxcdZno/89TVV2979UECAkdhIgkkgBhDUmjjtFwvaKOiowiIluCjOOMcmfG8fqbYS7z88od7jjjEAyLiEsUBRkU0LhgZ2FNIEAg7Akh0CRVva+1nPPcP87pSiV0ku5OV53q7uf9evUr1d9Tfc5z0kk/fb7bI6qKMcYYAxAKOgBjjDHFw5KCMcaYLEsKxhhjsiwpGGOMybKkYIwxJsuSgjHGmCxLCmbKE5FyEfmViHSLyM9F5LMisvYg7/+TiFxeyBjHS0T6ROS4cX7tcyJy3gSHZCa5SNABGDNMRP4C+BvgRKAX2AJcr6obDvPUnwRmAI2qmvHbfnyY5ywYEbkD2KWq39j/mKpWjfe8qjov5xr/CByvqp8b7/nM1GBPCqYoiMjfAP8G/AveD/DZwE3ABRNw+mOBl3ISgjHmACwpmMCJSC1wHfAlVb1HVftVNa2qv1LVa/33lIrIv4nIW/7Hv4lIqX/sPBHZJSJfFZE9ItImIpf6x/4J+CZwod/VcpmIfEFENuRc/4Mi8oLfvfRdQPaL74sisk1EOkXktyJybM4xFZEVIvKyiHSJyH+KiOQcv8L/2l4ReV5ETvXbjxaRu0UkLiLbReTL4/y7UxE53n99h4jcJCIP+ve6UUSO9P+uOv17PCXna3eIyJ+JyIeBr+f8HT3tH/+CiLzmx75dRD47nhjN5GJJwRSDs4Ey4JcHec8/AGcBC4EFwBlAbnfKkUAtMBO4DPhPEalX1f8P7+njZ6papaq35Z5URGLAPf65YsCrwOKc4xfg/cD8BNAErAfW7BfbR4DTgZOBTwMf8r/2U8A/Ap8HaoCPAu0iEgJ+BTztx/sB4Csi8qGD3P9ofTrnXpLAI8CT/ue/AG7c/wtU9Tfs+3e0QEQqgX8HlqtqNXAOXneemeIsKZhi0AgkDtG981ngOlXdo6px4J+Ai3OOp/3jaVV9AOgD3j2Ka58PPKeqv1DVNF4X1ts5x1cA/7+qbvPj+xdgYe7TAvAtVe1S1Z3AQ3iJC+By4Nuq+oR6XlHV1/ESSJOqXqeqKVV9DbgF+Mwo4j2UX6rqZlUdwkuyQ6p6p6o6wM+AUw7+5ftwgfkiUq6qbar63ATEZ4qcJQVTDNqBmIgcbOLD0cDrOZ+/7rdlz7FfUhkARjMIezTwxvAn6u0Q+UbO8WOB7/hdQ11AB1730syc9+QmkdzrzsJ78tjfscDRw+f0z/t1vLGUw7U75/XgCJ+PamBaVfuBC/GSYpuI3C8iJ05AfKbIWVIwxeARvK6Ojx3kPW/h/TAdNttvO1xteD+8AfDHA2blHH8DuEpV63I+ylX14VGc+w3gXQdo377fOatV9fzDuZHD9I7tklX1t6r6QeAo4AW8pxkzxVlSMIFT1W68weD/FJGPiUiFiJSIyHIR+bb/tjXAN0SkyR8H+Cbwowm4/P3APBH5hP+k8mW88YlhNwN/LyLzwBsU98cKRuNW4Gsicpp4jve7nR4HekXkb/01FGERmS8ipx/kXGERKcv5iI79Vg9qN9Dsj3cgIjNE5AJ/bCGJ1x3nTvA1TRGypGCKgqr+K94ahW8Acbzfpq8B7vXf8r+ATcAzwLN4g6f/awKumwA+BXwLrxvrBGBjzvFfAv8b+KmI9ABbgeWjPPfPgeuBn+Ctu7gXaPD79z+CN/awHUjgJZDag5zu7/C6f4Y//jjqmxydn/t/tovIk3g/G/4G72msA2gBVk7wNU0REiuyY4wxZpg9KRhjjMmypGCMMSbLkoIxxpgsSwrGGGOyJvUuqbFYTJubm4MOwxhjJpXNmzcnVLVppGOTOik0NzezadOmoMMwxphJRUReP9Ax6z4yxhiTZUnBGGNMliUFY4wxWZYUjDHGZFlSMMYYk2VJwRhjTJYlBWOMMVmWFIwxxmTlLSmIyCwReUhEnheR50Tkr/z2hSLyqIhsEZFNInKG3y4i8u8i8oqIPCMip+YrNmOMmaxUlUxnf97On88VzRngq6r6pIhUA5tF5HfAt4F/UtUHReR8//Pz8AqXnOB/nAms8v80xhgDuAMp0rt7wFUi9ZV5uUbekoKqtuHVv0VVe0VkG16xcwVq/LfVsrfO7gXAnX7h9EdFpE5EjvLPY4wx05Y6Lpk9vTi9QwBIOH89/wXZ+0hEmoFTgMeArwC/FZH/g9d9dY7/tpl4JRiH7fLb9kkKInIlcCXA7Nmz8xm2McYEzukeJB3vBbcwVTLzPtAsIlXA3cBXVLUHr87rX6vqLOCvgdvGcj5VXa2qi1R1UVPTiJv8GWPMpOcm06R2dmS7iwolr0lBRErwEsKPVfUev/kSYPj1z4Ez/NdvArNyvvwYv80YY6YNdV3S8V5SOztwh9IFv34+Zx8J3lPANlW9MefQW0CL//r9wMv+6/uAz/uzkM4Cum08wRgznTj9SVI72nE6B7zR1wDkc0xhMXAx8KyIbPHbvg5cAXxHRCLAEP74APAAcD7wCjAAXJrH2IwxpmhoxiG9pxe3Lxl0KHmdfbQBkAMcPm2E9yvwpXzFY4wxxSjT2U+mvb+g4wYHM6krrxljzGTlDqVJ7+5Bk5mgQ9mHJQVjjCkgdV0yiT6crsGgQxmRJQVjjCkQp3eITLwXzbhBh3JAlhSMMSbP3GSazJ5e3MHCTzEdK0sKxhiTJ+q4ZNqLt6toJJYUjDEmDzJdAzjt/ahTvF1FI7GkYIwxE8gdTJHe01t0s4pGy5KCMcZMAM04ZOJ92Z1MJytLCsYYcxhUFaejn0xHf2BbU0wkSwrGGDNOTt8QmXgfmnaCDmXCWFIwxpgxclMZb4rpQCroUCacJQVjjBkldV0y7f04XcHtYppvlhSMMWYUnO5BMom+STfFdKwsKRhjzEG4Q/5q5AAK3gTBkoIxxoxAHZdMvBenZ3JPMR0rSwrGGLOfTNcAmURf0dQ4yNXb+hLtt28k/XYPpXNizLh2GbXL50/Y+fNao9kYYyYTN5kmtbODzJ7eok0IbdfdT3pPD+GGClJtXey8Zg3dD26dsGtYUjDGTHuqSibRR2pnR1GPHbTfthGiIUIVUUSEcGUpEg2z+4a1E3YN6z4yxkxr7kDKq4A2CRagpXZ1Eqot26ctVBEluSMxYdewpGCMmZYm40By9Jh6UvEeQtEwTtolXFWKO5CitDk2Ydew7iNjzLTj9AyS2tE+qRICQO0nTsHZ00fyxT2kXkuQ6RtCUw4zrl02YdewJwVjzLShaYf07p5Jtz1F6s0uEresp+sXT2a7udRxiVSXc/R1H53Q2UeWFIwxU56q4nQOkGnvm1TbUyR3tJP43jq67nsa/LrO4cZKYpefy1H/cD7h6rJDnGHsLCkYY6Y0dyjtDSRPoqI3Qy/vIXHzOrofeDY7NTYyo4bYZYup//RphKvK8pIQwJKCMWaKUtclk5hc9ZEHt7WRWNVKz2+fz7aVzKwjduUS6j5xCqFo/n9kW1Iwxkw5Tn+SzO4eNDM5Nq8bePoN4qta6XvopWxbtLmR2FVLqfvvJyMl4YLFYknBGDNlaMYhvacXty8ZdCij0v/EDuI3/Yn+h1/LtpWecARNK5ZSs3w+Ei78BFFLCsaYSU9d1xtI7hwoyu0pcqkq/Q+/SvymVgY2vZ5tLzvpKJqubqH6AycioeBWC1hSMMZMWqqK2zNEpr2v6LuKVJW+P71EfFUrg0/vyraXLzyGpqvPo2rpCYhIgBF6LCkYYyYlp2/IK3qTKu7tKdR16Vm7jcSqVoZeeDvbXnFGM00rW6g8+7iiSAbDLCkYYyYVdzBFJt5X1BvXgTe+0f3gcyRubiX5SjzbXnXu8cRWLqVyUXNwwR2EJQVjzKTgJtNkEn24/cW9GlnTDl3/9TSJ1etIvd6Rba/+wIk0rWih/OSZAUZ3aHlLCiIyC7gTmIG3hnC1qn5HRH4GvNt/Wx3QpaoLRaQZ2Aa86B97VFVX5Cs+Y8zkoBnHW29Q5PsUuck0XXc/ReKW9aTf6vYaRaj50Ek0rWyh7MQjJ+Q6+S6yk88nhQzwVVV9UkSqgc0i8jtVvXD4DSLyr0B3zte8qqoL8xiTMWaSUNfF6Rgg09lf1FtTuIMpOn+2icRtG73iPAAhofYjJ9O0Yiml72qasGsNF9mRaGifIjuzv3vRhCWGvCUFVW0D2vzXvSKyDZgJPA8g3sjKp4H35ysGY8zko6o4XQM4HQOoU7wzipy+ITp+8jjt338Ep6MfACkJU/uxhTRduYTo7IYJv+ZIRXYckuy+YW3xJ4VcftfQKcBjOc1LgN2q+nJO2xwReQroAb6hqutHONeVwJUAs2fPzlfIxpgAOD2D3oyiIp5e6nQP0v7DR+m481Gcbm8LDYlGqPvkqcSuOJfo0XV5u/aUKLIjIlXA3cBXVLUn59BFwJqcz9uA2araLiKnAfeKyLz9vgZVXQ2sBli0aFERP1QaY0bL6U96yaCIN63LdPTTfsfDdPzocdx+b8W0lJfQ8JnTabz0HEpm1OQ9huEiO+GKaLZtoovs5DUpiEgJXkL4sarek9MeAT4BnDbcpqpJIOm/3iwirwJzgU35jNEYExw3mfamlxZxfYP0nl7ab99Ix0+fQAe9abChylIaPncGjV84h0hDZcFiabxsMW3X3Y9LColGcAdSk6fIjj9mcBuwTVVv3O/wnwEvqOqunPc3AR2q6ojIccAJwGsYY6acyTCjKPVWF4lbNniFbVLeE0y4tpyGz59F48VnEa4tL3hM1S1z4Zt4s49291DaPLlmHy0GLgaeFZEtftvXVfUB4DPs23UEsBS4TkTSgAusUNUOjDFThqridPST6SjeGUXJ19tJrF5P171b9ha2aaik8dJzaPiLMwhXlQYaX3XLXGref+KEzmrKlc/ZRxuAEdduq+oXRmi7G6+ryRhTBLof3MruG9aS3J6YkPnwTt8QmXhftpxksUm+Gid+8zq6f/3M3sI2TdXELl9M/YWLCJVHD3GGqcFWNBtj3qH7wa3svGYNEg0f9nx4N5Uhs6e3aMcNhl54m/hNrfSsfR7USwYlR9cSu2IJdX9+CqHSkoAjLCxLCsaYd9h9w1ovIVR6XSXjmQ+vGYdMR783bbMIu4oGntlFYtU6ev/4QrYtemyDV9jmowsKWtimmFhSMMa8Q3J7gnBDxT5to50Pn12J3FWctQ36N71OfNWf6N/warat9PgmYitaqF0+D4lMz2QwzJKCMeYdSufESLV1ZZ8U4NDz4dV1cboGcTqLbyWyqtL/yGvEV7Uy8PiObHvZe46kaWUL1R98T6CFbYqJJQVjzDvMuHYZO69Zg0OSUEX0oPPhi3lbClWlr9UvbLMlp7DNgmNourqFqpa5RVXLoBhYUjDGvEPt8vnM/u5F3uyjHYkR58OrKk73IE5Hf9FtS6GuS+/vXyC+qpWh59uy7RWnH0vT1ecVXWGbsZBwKK9rJCwpGGNGVLt8/gEHlZ3+JJk9vUU3vVQdl54HtxK/eR3Jl/dk2yvPfRdNK1qoPL05uOAOU6giSri2nFBVaV4TmiUFY8yoacYhvacXty8ZdCj70LRD131Pk1i9ntSO9mx71fvm0rSyhYoFswKMbvwkEiJcU064trxgs6EsKRhjRiXT2U+mvb+oZhS5qczewjZvdnmNItQsO4nYyqWUv+eoYAMcp1Cl/1RQmd+ngpFYUjDGHJSbTJPZ3VtUNZHdwRSdd20mceuGfQvbnP9eYiuXUnb8EcEGOA5SEiZcW064pizQabGWFIwxI1JVnPb+oqp85vQl/cI2D2cL2xAJUXfBQmJXLaH02MZgAxwrgXBVmfdUUFEc22hYUjDGvIM7kCK9u6doBpKdnkHaf/gYHXc+gtPlF7YpCVP3qdOIXX4u0Zn5K2yTDxIdfiooR8LFtT7CkoIxJstNZcjEe3H7i2OfokxHP+0/eISOHz2WHdyWshLqL1xE7LLFBSlsM2EEwtX+U0ERb65nScEYgzquV9/ALy8ZtPSeXtq/v5GONTmFbSqiNHzuTBq/cDaRxqqAIxw9KY3sHSuYBKumLSkYM8053X5d5CJYjZxu6yZx6wY679qcLWwTqimj8fNn0XDxWUTqKg5xhiIx/FRQV0GobHLtsmpJwZhpSFVx+5M4nQO4g8HPKkq90eEVtvnlluw4Rri+wits89kzCFeVHeIMxUFKwoTrinOsYLQsKRgzjajj4nQO4PQMFsXWFMlX48RXr6f7V8+A/6QSaaqi8bLFNFx4etHMyDmUUFUpkbqKSRPvwVhSMGYaUMf1aht0DRTF9NKhF94mfvM6en7z3N7CNkf5hW0+OTkK20jE24MoXFs+pbbbtqRgzBTmpjLeDqY9Q0WxEnnw2TeJr2ql9w85hW1mNxC7cgm1FywgFC3+H0mF2oMoKMX/HTDGjJnTN1Q04wUAA5tfJ76qlb71r2TbSt/VRGzFUmrPn1/8v2mHJPtUMBkS1+GY2ndnzDSiqri9Q2Q6+tFU8IvOVJX+R7cTv+lP+xa2OfFIYitbqFlW/IVtQmUlXiKoKZuSTwUjsaRgzBTg9A5500qLYAWyqtK37mWvsM1Tb2Tby987k9jKFqrf/+7i/gE7iaeTTgRLCsZMYk5/0ksGyUzQoXiFbf74IvGbWhl67q1se8Vps73CNovfVdTJoJi3nigkSwrGTEJuMk0m3oc7EPx2FOq49PzmOeI3t5J8KaewzTnH0bSyhcoz5gQY3SEIhCqnznTSiWBJwZhJRNMOmfY+bzZREcTS/etniN+8bt/CNufNpWnFUipOmR1gdAc3VaeTTgRLCsZMAsW0zsBNZej65RYSq9eT3tWZba/+4HtoWtlC+byjA4zu4EIVUW/F8SRZIR0ESwrGFDFVxekcINMRfMUzdyhN58/9wjZv93iNIaH2/PnEVrRQdkKRFrYJCeEaf+B4ik8nnQj2N2RMkXJ6/I3qAt6OwulP0rnmCdq//zCZRJ/XGA5Rd8ECYlcuoXROLND4DkTCIcL1FV4X0TQeOB4rSwrGFBl3IEU63hv4jCKnd4iOHz1G+x2PeN1W+IVtPnEKsSuWEJ1VH2h8ByLRMJH6ymm1tmAiWVIwpki4Q2kyieBnFGU6B/YWtun1BrSlNLK3sM2RtYHGdyA2XjAxLCkYEzA3lfGSgV9ZLCjpeC/t33+YzjVPZBNTqCJK/V+cQezSc4jEiq+wjYRDhGrKpsX2E4WSt79FEZkF3AnMwJsvsVpVvyMiPwPe7b+tDuhS1YX+1/w9cBngAF9W1d/mKz5jgqYZh0x7f+DVztJvd5O4dSOdd23KdlmFqstovPhMGj5/NpH64itsM9U3pQtSPlNrBviqqj4pItXAZhH5napeOPwGEflXoNt/fRLwGWAecDTwexGZq6rBr9s3ZgIN1zTIdPYHOr009UYniVvW03XPU3sL29RV0Hjp2TR89kzC1cXVDSOREOEaf21Bia0tyJe8JQVVbQPa/Ne9IrINmAk8DyBeev808H7/Sy4AfqqqSWC7iLwCnAE8kq8YjSkkdYeTwUCg00uTryVIrF5H1305hW1iXmGb+gsXEa4sDSy2d/BXHIdryghV2lNBIRSkE05EmoFTgMdympcAu1X1Zf/zmcCjOcd3+W3GTGrqujhdgzidA4HWQR56cTfx762j54Gt2cI2kSNriF2xhPpPnlpUm79JJES4zqaTBiHvSUFEqoC7ga+oak/OoYuANeM435XAlQCzZxfvMnpjhheeBZ0MBp97i/hNrfT+flu2reSYemJXLaHuYwuLaoA2VOmNFdgMouDk9V+DiJTgJYQfq+o9Oe0R4BPAaTlvfxOYlfP5MX7bPlR1NbAaYNGiRcGXkjJmBE5/ksye3kC3sh54aqdX2Kb15WxbdE6MphVLqf3Ie4tmzx/bh6i45HP2kQC3AdtU9cb9Dv8Z8IKq7sppuw/4iYjciDfQfALweL7iMyYfgp5eqqoMPLad+KpW+h/dnm0vnTuDppUt1HzopKLpjglVRr0uomIawzB5fVJYDFwMPCsiW/y2r6vqA3izjPbpOlLV50TkLryB6AzwJZt5ZCaL7PTSnsFAZhSpKn3rXyGxqpWBJ3dm28vmHU3T1X5hmyKociYlYW8fInsqKFr5nH20ARhxqoCqfuEA7dcD1+crJmMmmrouTkdw00tVld4/vEB8VStDW/cWtik/ZRZNK1uoWnpCUczYCVWVel1E9lRQ9IpnhMmYSURVcboHcdr7AxlEVsel57fPe4VtXtydba88aw5NV59HxRnNgScDGyuYnCwpGDNGQdZD1oxD96+fJf69daReS2Tbq5aeQNPKFipODX5GXqi8hHB9ha0rmKQsKRgzSu5AyhtEHkoX/tqpDF33+oVt3tivsM1VSyl/b/BLekJVpUQaKotqvYMZO0sKxhyCm/R3L+0v/O6lbjJN5y+epP2WDaTbur1GEWqWz6NpRQtl755R8Jj2IXhbT9RbAZupwr6LxhyAO5T2Fp/1Fr4esjuQouNnT9B+20Yy8ZzCNh89mdiVSyk9LtjCNjaLaOqypGDMfpy+IZzOAdzBwncTOX05hW069y9scy7RWQ0Fj2mYREKEqsu8fYhKrYtoqrKkYIzP6R0i09EfSMWzTNcAHT98lPY7H8XtySls86nTiF1+LiVHBVfYxqaTTi8HTAoi8gBwtaruKFw4xhSWZhycniGc7sFAZhNl2vto//7DdPz48X0L23zmdBq/eA4lTdUFj8kLQgjXlhOpq7BtqqeZgz0pfB9YKyI/AL6tqoV/ljbTRveDW9l9w1qS2xOUzokx49pl1C6fn7fruakMTns/Tt9QIIvO0rt7SNy6gc67NqP+bKZQVSkNF59F4+fPItJQWfig8Ivd13kDx8WwAtoU3gGTgqr+XEQeBP4nsElEfgi4Ocf338/ImHHpfnArO69Zg0TDhBsqSLV1sfOaNcz+7kUTnhg07ZBp78PpKfzgMUBqVyeJWzbQdfeTOYVtymm85GwaPncm4ZryQOLKblVdV27JYJo71JhCCugHSoFqcpKCMRNl9w1rvYTg91mHK0txSLL7hrUTlhTUcb1k0B3M3kTJHe0kvreOrvuehkxOYZsvnkP9Z04PrL9eSsLe2oKaMltoZoCDjyl8GLgRb/fSU1V1oGBRmWkluT1BuGHfOsChiijJHYkDfMXoBV3tbOjlPSRubqX7ga3Z60dm1BC7/FzqP31aMAu9QkK4uoxwdRmhimjhr2+K2sGeFP4B+JSqPleoYMz0VDonRqqta5/flt2BFKXN45+Lr6q4Pf52FAHsTTT4fBvxVa30rn0+21Yys47YVUup+3gwhW0kEiLcUOmtLbCnAnMABxtTWFLIQMz0NePaZey8Zg0OSUIVUdyBFJpymHHtsjGfK5sMOvvRVOFnEw1secMrbPOnl7Jt0eZGv7DNyYHM5LFkYMbC1imYwNUun8/s717kzT7akaC0eeyzj9RxcboGcLoGA3ky6H/cL2zz8GvZttK5R9C0ooWaD88LpLCNREKE6ysI11VYMjCjZknBFIXa5fPHNajsJnO2oijwkIGq0v/wq8RvamVg0+vZ9rJ5R9G0soXqD5wYyEweezIwh8OSgpmUnN4hnK5gtqJQVfoeepH4qlYGn9lbRjzowjYSCRFprLKZROawWFIwk4a6rlfYpnMAzQRQ2Mb1Ctskbl7H0AtvZ9srzmim6eoWKs86LphkEA4RbrQnAzMxLCmYoqdph0zXgLfGIIBppZpx6H5gK4mb15F8NZ5trzr3eGIrW6hcdGzBYwI/GdTbgjMzsSwpmKLlDqbIdA7g9iWDuX4qQ/d9z5BYvY7U6x3Z9uoPnEjTihbKTw6msI0lA5NPlhRMUVHXxe1NkukaCGS3UvAGr7t+8RSJW9eTfiunsM2H59G0YillJx5Z+KAEQpWl3rbVVubS5JElBVMUNOOQ6Qyuiwj8wjZ3baL91o1k4r1eYzhE7UfeS9NVSyl9V1MgcYWry4jEqmy3UlMQlhRMoNxUxptS2hPMnkTgF7b58eO0f//hfQvbfHwhsSuWEJ0dTGGbUHkJkaZqq3lsCsqSggmE05/0ppQGUPc4G0P3IO13PrJvYZtohPpPn0bjZYuJHl0XSFwSDRNpqraiNiYQlhRMwajregVtugYC2YJiWKajn/Y7HqbjR4/j9nuD2FJeQsNFp9N46WJKjgimsM3wOoNwbTDbZxsDlhRMAQQ9pXRYencP7bdvpOOnm/YWtqkspeHiM2m85OxgC9s02HYUpjhYUjB54w7lbEERoNSbXSRu3UDXL55EU96MpnBtOQ2fP4vGi88K5DdzCYcIVZV6HxVRSwamaFhSMBPO6R3C6RzAHQq2gmvy9XYSq9fTde+WbGGbcEMljZeeQ8NfnEG4qvB99lIa8YraVNm0UlOcLCmYCaGui9M16I0XBLAFRa7kq3Hiq1rpvv/ZvYVtjqjeW9imvMCFZQTCVWWEa8utqI0pepYUzGFxUxlvy+qeoUDHCwAGt7WRWLWOnrXPg3qxlMysI3b5udT9+SmESgs7tVNKwt7K4+qyQLbONmY8LCmYcXEHUmQ6+wOdUjps4JldJG5qpfehF7Nt0WMbvCpnH11Q8EVfEvXqHodrbBaRmXwsKZhRC7qq2f76N+3wCttseDXbVnp8E00rW6hZPr/gv51LOEQkZlNKzeSWt6QgIrOAO4EZeGtVV6vqd/xjfwl8CXCA+1X1f4hIM7ANGP5171FVXZGv+Mzoacbxxgu6g6lqtk8sqvQ/8hrxm/7EwBM5hW1OOoqmFUup/uB7Cr9JXEgI15YTaay0DerMpJfPJ4UM8FVVfVJEqoHNIvI7vCRxAbBAVZMickTO17yqqgvzGJMZA3co7Y0XBFDVbH+qSl/rS15hmy27su3lC46h6eoWqlrmFnw2j5RGCNeWE64ps2Rgpoy8JQVVbQPa/Ne9IrINmAlcAXxLVZP+sT35isGMXbaLKMBdSveJx3Xp/f0LxFe1MvR8W7a94oxmmla2UHl24csqkGgAABZoSURBVAvbhMpKiMSqbCaRmZIKMqbgdw2dAjwG3AAsEZHrgSHga6r6hP/WOSLyFNADfENV149wriuBKwFmz56d/+CniWKaRQSgjusXtmkl+crewjaV577LSwaLmgseU6isxKtwZnsSmSks70lBRKqAu4GvqGqPiESABuAs4HTgLhE5Du+pYraqtovIacC9IjJPVXtyz6eqq4HVAIsWLQr+p9ckpqq4vUM43YOB1DoeiaYduu57msTq9aR2tGfbq9/3bmJXt1Bx8jEFj0lKI94AsiUDMw3kNSmISAleQvixqt7jN+8C7lFVBR4XEReIqWocGO5S2iwirwJzgU35jHE6clMZLxH0DAU+cDzMTabpumcLiVvWk36zy2sUoWbZScRWLqX8PUcVNqCQEK4qJVxjC87M9JLP2UcC3AZsU9Ubcw7dC7wPeEhE5gJRICEiTUCHqjr+k8MJwGv5im+6UVXcvqSXDAaCX1swzB1M0fmzTSRu20hmj1/YJiTUfuRkYlctoez4Iw5+ggkWKi/xEkF1qQ0em2kpn08Ki4GLgWdFZIvf9nXgduB2EdkKpIBLVFVFZClwnYikARdYoaodI53YjJ6mHZzu4phOmsvpS9K55nES338Yp73fa4yEqPvYQmJXLqH02MaCxSKRkJcIasoIRW3pjpne8jn7aANwoGkhnxvh/XfjdTWZCeD0+WMFRbDiOJfTM0j7nY/Sceej3lbaeIVt6j55KrHLzyU6s0CFbfz9iEI1ZTZWYEwO+7VoCtFMzlNBwJvS7S/T0U/7Dx6h40eP4fb5hW3KSmj4zCIav7iYkhk1BYkju7Ygj/sRdT+4ld03rCW5PUHpnBgzrl1G7fL5ebmWMRPNksIU4PT7YwX+D9tikt7T6xe2eQIdzCls89kzaPzC2UQaqwoSR6Gmk3Y/uJWd16xBomHCDRWk2rrYec0aZn/3IksMZlKwpDBJqePufSpIB78P0f7Sbd0kbllP58/3FrYJ1ZTReMnZNHzuTCJ1FfkPQiBc7W9ZXaDtsnffsNZLCH7yCVeW4pBk9w1rLSmYScGSwiTjDqS8ZNAX/NYTI0nt7CC+ej3d927JJqtwfQWNXxwubFOW9xikxNulNIgZRMntCcIN+ya8UEWU5I5EQeMwZrwsKUwC6rg4Pf5TQRHsTjqS5Ktx4t9bR/evnwV/llOkqZrGyxfT8OlFBZnrL5EQ4YZKwrXlgVU1K50TI9XWtU83lTuQorQ5Fkg8xoyVJYUi5g76TwVFsCHdgQy98Dbxm9fR85vn9ha2ObqW2BVLClbYphiSwbAZ1y5j5zVrcEgSqojiDqTQlMOMa5cFGpcxo2VJocio6+L0eNNJi2FDugMZfOZN4qta6f3jC9m26GyvsE3tR08uyHx/iYSINFYRqikLPBkMq10+n9nfvcibfbQjQWmzzT4yk4slhSKgros7kPJWHBfxUwHAwObXid/USt+GV7Jtpe9q8pLBf5uPRApQ5SwkXmWz+oqiSQa5apfPtyRgJi1LCgFxk2kvEfSncAdT40oEva0v0X7bRlK7OokeU0/jZYupbpk74bGqKv2PbvcK2zy+I9teduKRxFa2ULOsQIVtQkKkrsJLBlbz2Ji8sKRQQJp2yHQNTMhGdL2tL9F23f0QDRGqLSMV7/E+/yYTlhhUlb51L3uFbZ56I9tefvJMmla2UPW+dxfkN3UJhwjXVxCuK7f9iIzJM0sKBeD0DeF0TexGdO23bYRoiLA//z5cHsUhRfttGw87Kajr0vsHv7DNczmFbU6bTdPV51G5+F2F6bYJCZF6/8nAkoExBWFJIU8043gDxl0DedlyIrWrk1DtvnP+Q2UlpN7sHPc51XHp+c1zxG9uJfnS3oJ4lYv9wjanN4/73GMiEK6rINJQad1ExhSYJYUJ5g6mvCL3eV5cFj2mnlS8J/ukAF5N5ejM+jGfS9MO3b9+hvjN6/YpbFP1vrk0rWihYuGsCYn5kCwZGBM4SwoTIDuNtGugYIvLGi9bTNt19+OQIlRWgjuUhpRL42WLR30ON5Wh65dbSKxeT3rX3ieM6mUn0bSyhfKTClPYRsIhwnXlhOtsANmYoFlSOAxuMu09FfQWvq5xdctc+KY3tpB6s5PozNHPPnKH0nTetZnErRvI7ParnYaE2vPfS2zFUspOKExhGxtANqb4WFIYo2wFs66BwOsaV7fMHdOgstOfpHPNEyRu37hvYZuPLiB21VJKmwtU2GZ4ammDDSAbU2wsKYySZhzvqaDIKpiNhtMzSMePHqP9B4/gdPmFbUrC1P35qcSuOJfoMWMfhxgXgXBtOZHGKusmMqZIWVI4BKfffyoosgpmo5Hp6PeqnP3w0X0K29RfuIjYZYUrbAMQrinzkkFJAVY8G2PGzZLCCLK7knYVZ62CQ0nHe2m//WE6f/pEdm1EqCLqF7Y5h0isMIVtAEJVpURiVVb72JhJwv6n5nCH0jhdA0W//9CBpNu6Sdy2gc67Nmc30wtVl9F48Zk0fP5sIvUFKGzjC1VEvWRQlv9dUo0xE2faJwVVxe31VxwPBTtwPF6pNzpI3LKBrnue2lvYpq6CxkvPpuGzZxKuzn9hm2GhshIvGRSgfoIxZuJN26SgGYdM58TsQxSU5GsJEqvX0XXfM3sL28SqaLxsMfUXLsp7PeJcUhohEqsq6DWNMRNv2iYFp3cIp3Mg6DDGZejF3cS/t46eB7buLWxzVC2Nl59L/SdPLWiXTaishHBDRUHKbBpj8m/aJoXJaHDrW15hm99vy7aVzKqn6aql1F6woKCDuaGKqFcH2bqJjJlSLClMAgNP7fQK26x7OdsWPS7mJYOPvLcwhW184eoywg0VBSmzaYwpPEsKRUpVGXhsO/FVrfQ/uj3bXvruGTStaKHmQycVdAFYuLqMcGOlTS01Zoqz/+FFRlXpW/8KiVWtDDy5M9teNv9omq4+j+r3zS3o1hC2zsCY6cX+pxcJdV16//iiV9hm61vZ9opTZ9N0dQuV5x5f0HrEocookUZbZ2DMdGNJIWDquPT89nmvsM2Lu7PtlWcfR9PKFirOaC5sMij31xmU2wCyMdORJYWAaMah+9fPEv/eOlKvJbLtVS0neMnglNkFjSdUVkK4sdLWGRgzzVlSKDA3laHrXr+wzRs5hW0++B6vsM28owsaj0TD3qIzW2dgjMGSQsG4Q2k6f/EkiVvWk3nbL2wjQs3yeTStaKHs3TMKGo+UhIk0VhKuKS/odY0xxS1vSUFEZgF3AjPwtpdbrarf8Y/9JfAlwAHuV9X/4bf/PXCZ3/5lVf1tvuIrFKc/SefPNtF++0Yy8T6vMRyi7oIFxK5YQulxscIGFBIiDZWE6ysKOlZhjJkc8vmkkAG+qqpPikg1sFlEfoeXJC4AFqhqUkSOABCRk4DPAPOAo4Hfi8hcVZ18e1fjbaPR8aPHaL/jEZwubzsNKQlT94lTiF2xhOisAhW2GWYFbowxo5C3pKCqbUCb/7pXRLYBM4ErgG+patI/tsf/kguAn/rt20XkFeAM4JF8xZgPmc4BOu58hPYfPobbOwR4m8XVf/o0YpefS8mRtQWPyQrcGGNGqyBjCiLSDJwCPAbcACwRkeuBIeBrqvoEXsJ4NOfLdvlt+5/rSuBKgNmzCztD52AyiT4S33+Yzp88vk9hm/qLTqfx0nMoaaoueEyhiiiRpirbksIYM2p5TwoiUgXcDXxFVXtEJAI0AGcBpwN3ichxoz2fqq4GVgMsWrQo8FI46d09JG71C9v49RhCVaU0XHwWjZcUtrDNMImGiTRV2/RSY8yY5TUpiEgJXkL4sare4zfvAu5RVQUeFxEXiAFvArNyvvwYv60opd7oJHHL+v0K25TTeMnZNHzuzGBm9YTEm1FUZ4PIxpjxyefsIwFuA7ap6o05h+4F3gc8JCJzgSiQAO4DfiIiN+INNJ8APJ6v+MYruT1BYvV6uv7r6X0L23zxHOo/c3pgv52Ha8qIxKoKumOqMWbqyeeTwmLgYuBZEdnit30duB24XUS2AingEv+p4TkRuQt4Hm/m0peKaebR0Mt7SNzcSvcDW8H1eq0iR9YQu2JJwQvb5LJxA2PMRMrn7KMNwIH6MD53gK+5Hrg+XzGNx+BzbxG/eR29a5/PtpXMrCN21VLqPr4wsN1DQ2Ul9G9+nfh/PERye4LSOTFmXLuM2uXzA4nHGDM12IrmAxjY8gbxVa30/emlbFu0uZGmFUup/cjJgU3vlNIIkcZK+ta/wq6v/QKJhgk3VJBq62LnNWuY/d2LLDEYY8bNksJ++h/fTvymVvofeS3bVjr3CJpWtlDzoXmBLfwaTgbDexTtvmGtlxD8MYxwZSkOSXbfsNaSgjFm3Cwp4BW26d/4KvFVrQxsej3bXjbvKJpWtlD9gRMLWtgml5RGvA3r9hvATm5PEG7Yd7prqCJKckcCY4wZr2mdFFSVvoe8wjaDz+yd/Vp+yiyaVrZQtfSEwKZ2Som/e2n1yLuXls6JkWrr2idZuAMpSpsLvJeSMWZKmZZJQV2Xrnu3sPvbaxl64e1se+WZc4itbKHyrDnBJoOGSkI1ZQeNYca1y9h5zRockoQqorgDKTTlMOPaZQWM1hgz1UzLpPDK8v+gJ2c2UdWS473CNqcdG1hMEg55Fc8OkQyG1S6fz+zvXsTuG9aS3JGgtNlmHxljDt+0TAq1//1ketY+T/UHTvQK27z3HVssFVS4bny7l9Yun29JwBgzoaZlUohdfi7lC4+h5IiaQOMIVUa9pwNbeGaMKRLTMin0PvQib//LgyS3J4geU0/jZYupbplbsOuHyku8ZFAeLdg1jTFmNKZdtZXuB7ey85o1pHZ3E6otIxXvoe26++ltfenQX3yYpDRCycw6orMaLCEYY4rStEsK2UVfFaWICOHyKERDtN+2MW/XlJIwJUfVUnpso21nbYwpatOu+yi76MvZW4ohVFZC6s3OCb+WREJEGkc/o8gYY4I27ZLC8KKv3MFddyhNdObE1UyWcIhwfQXheqtrYIyZXKZd99GMa5ehKQdnIImq4gymIOXSeNniwz95SIg0VBKd00ikodISgjFm0pl2SWF40Vd0Ri1uzxDRphqO+uZ/O7zZR+KtNSidE/MK3QS0T5Ixxhyuadd9BF5iqDxrDpl432GfK1xT5i08C2grbWOMmUjTMilMhFBVqbfWIKAiO8YYkw/2E22MpDRCyRHVts7AGDMlWVIYJYl4G9aFa8qDDsUYY/LGksKhCETqKwk3VNgAsjFmyrOkcBDh6jJvNpENIhtjpglLCiMIlZUQabIN64wx048lhRw2bmCMme4sKYCNGxhjjG/aJ4VQZZSSI2ps3MAYY5jGSUEiYUqOriVcVRZ0KMYYUzSmbVIIV1syMMaY/VkHujHGmCxLCsYYY7IsKRhjjMmypGCMMSbLkoIxxpisvCUFEZklIg+JyPMi8pyI/JXf/o8i8qaIbPE/zvfbm0VkMKf95nzFZowxZmT5nJKaAb6qqk+KSDWwWUR+5x/7v6r6f0b4mldVdWEeYzLGGHMQeUsKqtoGtPmve0VkGzAzX9czxhhz+AoypiAizcApwGN+0zUi8oyI3C4i9TlvnSMiT4lIq4gsOcC5rhSRTSKyKR6P5zdwY4yZZkRV83sBkSqgFbheVe8RkRlAAlDgn4GjVPWLIlIKVKlqu4icBtwLzFPVnoOcOw68fhjhxfxYJrupch9g91KMpsp9gN3LsGNVtWmkA3lNCiJSAvwa+K2q3jjC8Wbg16o6f4RjfwK+pqqb8hjfJlVdlK/zF8pUuQ+weylGU+U+wO5lNPI5+0iA24BtuQlBRI7KedvHga1+e5OIhP3XxwEnAK/lKz5jjDHvlM/ZR4uBi4FnRWSL3/Z14CIRWYjXfbQDuMo/thS4TkTSgAusUNWOPMZnjDFmP/mcfbQBkBEOPXCA998N3J2veA5gdYGvly9T5T7A7qUYTZX7ALuXQ8r7QLMxxpjJw7a5MMYYk2VJwRhjTNa0SQoi8lcistXfh+krftsCEXlERJ4VkV+JSE3QcY7EX+S3R0S25rQ1iMjvRORl/896v11E5N9F5BV/geCpwUX+TmO8lxP9709SRL4WXNTvNMb7+Kz/vXhWRB4WkQXBRf5OY7yXC/x72eIvIj03uMjfaSz3knP8dBHJiMgnCx/xgY3x+3KeiHTn7B33zfFed1okBRGZD1wBnAEsAD4iIscDtwJ/p6rvBX4JXBtclAd1B/Dh/dr+DviDqp4A/MH/HGA53nTeE4ArgVUFinG07mD099IBfBkYaZ+soN3B6O9jO9Di/zv7Z4pvsPMORn8vfwAW+HuUfRHv/1AxuYPR3wv+NPj/DawtVIBjcAdjuBdgvaou9D+uG+9Fp0VSAN4DPKaqA6qawVth/QlgLrDOf8/vgD8PKL6DUtV1eD8gc10A/MB//QPgYzntd6rnUaBuv7UhgRrLvajqHlV9AkgXLsLRGeN9PKyqnX77o8AxBQlylMZ4L326d3ZKJd7U8qIxxv8rAH+JN+txT/6jG5tx3MuEmC5JYSuwREQaRaQCOB+YBTyH95cM8Cm/bbKY4W86CPA2MMN/PRN4I+d9uyj+jQgPdC+TzWju4zLgwcKFNG4HvBcR+biIvADcj/e0UOxGvBcRmYm3gLbYnqYP5mD/xs4WkadF5EERmTfeC0yLpKCq29j7iPgbYAvg4P2DvlpENgPVQCqwIA+D/5tbUf3GNl5T5V5Gug8ReR9eUvjbQIIap/3vRVV/qaon4v2W+s+BBTYO+93LvwF/q6pugCGN23738iTefkYLgP/A2ztuXKZFUgBQ1dtU9TRVXQp0Ai+p6guqukxVTwPWAK8GG+WY7B7uFvL/HH78fZN9n3iO8duK2YHuZbI54H2IyMl4/e8XqGp7QPGNxSG/J373xnEiEit0cGN0oHtZBPxURHYAnwRuEpEJ746ZYCPei6r2qGqf//oBoGS835dpkxRE5Aj/z9l44wk/yWkLAd8AJlO1t/uAS/zXlwD/ldP+eX8W0llAd87jZrE60L1MNiPeh/9v7h7gYlV9KaDYxupA93K8iIj/+lSgFCj2JDfivajqHFVtVtVm4BfA1ao67t+wC+RA35cjc74vZ+D9bB/f90VVp8UHsB54Hnga+IDf9lfAS/7Ht/BXeBfbB95TTBvegOsuvC6IRrzZBy8Dvwca/PcK8J94Tz3PAouCjv8w7uVI/z09QJf/uiboexjHfdyK93S6xf/YFHT8h3Evf4s3FrcFeAQ4N+j4x3sv+33dHcAng47/ML4v1/jfl6fxJjOcM97r2jYXxhhjsqZN95ExxphDs6RgjDEmy5KCMcaYLEsKxhhjsiwpGGOMybKkYMwEEpFZIrJdRBr8z+v9z5uDjcyY0bGkYMwEUtU38PbS+Zbf9C1gtaruCCwoY8bA1ikYM8FEpATYDNyOt2X7QlUtup1ejRlJJOgAjJlqVDUtItfibb64zBKCmUys+8iY/FiOt0XB/KADMWYsLCkYM8FEZCHwQeAs4K+LqciRMYdiScGYCeTvVLkK+Iqq7gRuoDjLiRozIksKxkysK4Cdqvo7//ObgPeISEuAMRkzajb7yBhjTJY9KRhjjMmypGCMMSbLkoIxxpgsSwrGGGOyLCkYY4zJsqRgjDEmy5KCMcaYrP8Hz34ZUVi6kucAAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Prediction_Interval_of_a_Future_Response'></a>\n",
        "\n",
        "### **6.4.4. Prediction Interval of a Future Response:**"
      ],
      "metadata": {
        "id": "QkMjdS6P8ibS"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**A. Two-sided Confidence Interval:**\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "$P(a + bx^* - t_{\\frac{\\alpha}{2},n-2}{S_{y|x} \\sqrt{1+\\frac{1}{n} + \\frac{(x^* - \\overline{x})^2}{S_{xx}}}}\\ <\\ A + Bx<\\ a + bx^* + t_{\\frac{\\alpha}{2},n-2}{S_{y|x} \\sqrt{1+\\frac{1}{n} + \\frac{(x^* - \\overline{x})^2}{S_{xx}}}}\\ ) = 1-\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Therefore, the $1-\\alpha$ confidence interval for for the mean of a normal population is:\n",
        "\n",
        "$[a + bx^* - t_{\\frac{\\alpha}{2},n-2}{S_{y|x} \\sqrt{1+\\frac{1}{n} + \\frac{(x^* - \\overline{x})^2}{S_{xx}}}}\\ ,\\ a + bx^* + t_{\\frac{\\alpha}{2},n-2}{S_{y|x} \\sqrt{1+\\frac{1}{n} + \\frac{(x^* - \\overline{x})^2}{S_{xx}}}}]$"
      ],
      "metadata": {
        "id": "Vfl-hRCh-WIc"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**B. One-sided Lower Confidence Interval:**\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "$P(-\\infty\\ \\leq\\ A + Bx<\\ a + bx^* + t_{\\alpha,n-2}{S_{y|x} \\sqrt{1+\\frac{1}{n} + \\frac{(x^* - \\overline{x})^2}{S_{xx}}}}\\ ) = 1-\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Therefore, the $1-\\alpha$ confidence interval for for the mean of a normal population is:\n",
        "\n",
        "$[-\\infty\\ ,\\ a + bx^* + t_{\\alpha,n-2}{S_{y|x} \\sqrt{1+\\frac{1}{n} + \\frac{(x^* - \\overline{x})^2}{S_{xx}}}}]$"
      ],
      "metadata": {
        "id": "eb1RfbIz_F5b"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**C. One-sided Upper Confidence Interval:**\n",
        "\n",
        "Significance level = $\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "$P(a + bx^* - t_{\\alpha,n-2}{S_{y|x} \\sqrt{1+\\frac{1}{n} + \\frac{(x^* - \\overline{x})^2}{S_{xx}}}}\\ \\leq\\ B\\ \\leq\\ \\infty) = 1-\\alpha$\n",
        "\n",
        "$\\\\ $\n",
        "\n",
        "Therefore, the $1-\\alpha$ confidence interval for for the mean of a normal population is:\n",
        "\n",
        "$[a + bx^* - t_{\\alpha,n-2}{S_{y|x} \\sqrt{1+\\frac{1}{n} + \\frac{(x^* - \\overline{x})^2}{S_{xx}}}}\\ ,\\ \\infty]$"
      ],
      "metadata": {
        "id": "F0qa5P65_MvI"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "class prediction_interval:\n",
        "  \"\"\"\n",
        "  Parameters\n",
        "  ----------\n",
        "  c_level : % confidence level\n",
        "  type_t : 'two_sided_confidence', 'lower_confidence', 'upper_confidence'\n",
        "  S_xx : optional\n",
        "  S_yy : optional\n",
        "  S_xy : optional\n",
        "  data : optional\n",
        "  \"\"\"\n",
        "  def __init__(self, c_level, type_c, x_star, n = 0., S_xx = 0., S_xy = 0., S_yy = 0., data_x=None, data_y=None):\n",
        "    self.type_c = type_c\n",
        "    self.c_level = c_level\n",
        "    self.n = n\n",
        "    self.x_star = x_star\n",
        "    self.S_xx = S_xx\n",
        "    self.S_xy = S_xy\n",
        "    self.S_yy = S_yy\n",
        "    self.data_x = data_x\n",
        "    self.data_y = data_y\n",
        "    if data_x is not None:\n",
        "      self.n = len(list(data_x))\n",
        "      xx = []\n",
        "      for i in list(data_x):\n",
        "        xx.append((i - np.mean(list(data_x)))**2)\n",
        "      self.S_xx = np.sum(xx)\n",
        "\n",
        "      yy = []\n",
        "      for i in list(data_y):\n",
        "        yy.append((i - np.mean(list(data_y)))**2)\n",
        "      self.S_yy = np.sum(yy)\n",
        "\n",
        "      xy = []\n",
        "      for i in range(len(data_x)):\n",
        "        xy.append((data_x[i] - np.mean(data_x))* (data_y[i] - np.mean(data_y)))\n",
        "      self.S_xy = np.sum(xy)\n",
        "      self.Syx=None\n",
        "\n",
        "    prediction_interval.__test(self)\n",
        "  \n",
        "  def __test(self):\n",
        "    if self.type_c == 'two_sided_confidence':\n",
        "      b = self.S_xy / self.S_xx\n",
        "      a = np.mean(self.data_y) - b * np.mean(self.data_x)\n",
        "      self.Syx = np.sqrt(((self.S_yy) - ((self.S_xy)**2 / (self.S_xx))) / (self.n-2))\n",
        "      c_u = a + b*self.x_star + (t.isf((1-self.c_level)/2, self.n-2)) * self.Syx * np.sqrt(1+(1/self.n) + (((self.x_star - np.mean(self.data_x))**2) / self.S_xx))\n",
        "      c_l = a + b*self.x_star - (t.isf((1-self.c_level)/2, self.n-2)) * self.Syx * np.sqrt(1+(1/self.n) + (((self.x_star - np.mean(self.data_x))**2) / self.S_xx))\n",
        "      display(Latex(f'${c_l} \\leq Y|x^* \\leq {c_u}$'))\n",
        "    elif self.type_c == 'lower_confidence':\n",
        "      b = self.S_xy / self.S_xx\n",
        "      a = np.mean(self.data_y) - b * np.mean(self.data_x)\n",
        "      self.Syx = np.sqrt(((self.S_yy) - ((self.S_xy)**2 / (self.S_xx))) / (self.n-2))\n",
        "      c_u = a + b*self.x_star + (t.isf((1-self.c_level)/2, self.n-2)) * self.Syx * np.sqrt(1+(1/self.n) + (((self.x_star - np.mean(self.data_x))**2) / self.S_xx))\n",
        "      display(Latex(f'$Y|x^* \\leq {c_u}$'))\n",
        "    elif self.type_c == 'upper_confidence':\n",
        "      b = self.S_xy / self.S_xx\n",
        "      a = np.mean(self.data_y) - b * np.mean(self.data_x)\n",
        "      self.Syx = np.sqrt(((self.S_yy) - ((self.S_xy)**2 / (self.S_xx))) / (self.n-2))\n",
        "      c_l = a + b*self.x_star - (t.isf((1-self.c_level)/2, self.n-2)) * self.Syx * np.sqrt(1+(1/self.n) + (((self.x_star - np.mean(self.data_x))**2) / self.S_xx))\n",
        "      display(Latex(f'${c_l} \\leq Y|x^*$'))"
      ],
      "metadata": {
        "id": "qVtHLmLj8rKO"
      },
      "execution_count": 33,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x = [100,102,103,101,105,100,99,105]\n",
        "y = [259,264,274,268,277,263,258,275]\n",
        "\n",
        "prediction_interval(x_star = 107, c_level = 0.95, type_c = 'two_sided_confidence', data_x=x, data_y=y);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "my95Wk0V_U4W",
        "outputId": "998c9a08-3743-4261-cf6b-b8dc8b00bf30"
      },
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.Latex object>"
            ],
            "text/latex": "$273.34406301930005 \\leq Y|x^* \\leq 292.0796657942592$"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "n = 100\n",
        "x = np.linspace(0, 10, n)\n",
        "e = np.random.normal(size=n)\n",
        "y = 1 + 0.5*x + 2*e\n",
        "\n",
        "X = sm.add_constant(x)\n",
        "model = sm.OLS(y, X)\n",
        "results = model.fit()\n",
        "\n",
        "st, data, ss2 = summary_table(results, alpha=0.05)\n",
        "\n",
        "fittedvalues = data[:, 2]\n",
        "predict_mean_se  = data[:, 3]\n",
        "predict_mean_ci_low, predict_mean_ci_upp = data[:, 4:6].T\n",
        "predict_ci_low, predict_ci_upp = data[:, 6:8].T\n",
        "prstd, iv_l, iv_u = wls_prediction_std(results)\n",
        "\n",
        "# Check we got the right things\n",
        "#print(np.max(np.abs(results.fittedvalues - fittedvalues)))\n",
        "#print(np.max(np.abs(iv_l - predict_ci_low)))\n",
        "#print(np.max(np.abs(iv_u - predict_ci_upp)))\n",
        "\n",
        "plt.plot(x, y, '.', color = 'gray')\n",
        "plt.plot(x, fittedvalues, '-', lw=2, label='Regression Line', color = '#0099ff')\n",
        "plt.plot(x, predict_ci_low, 'r--', lw=2, color = '#00cc99', label='Prediction Interval')\n",
        "plt.plot(x, predict_ci_upp, 'r--', lw=2, color = '#00cc99')\n",
        "plt.plot(x, predict_mean_ci_low, 'r--', color = '#ff0066', lw=2, label='Confidence Interval')\n",
        "plt.plot(x, predict_mean_ci_upp, 'r--', color = '#ff0066', lw=2)\n",
        "plt.legend();"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "bzSeXMKr5VtG",
        "outputId": "eba0556c-abc8-4efa-ea36-d3d1f02f0938"
      },
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD4CAYAAADxeG0DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydd3wb9f3/n2d5W/LesRPb2Y5HlrM9kkASdsOeSaA0QAulUErLLi3QlkJpaSkh5fcFQimQMsuGhDh2Ftlx9pbjIc94yVvS/f74aDp2ItuyLSX3fDz0sHR3+txJ1r3ufe/Pe0iyLKOgoKCg4Ll4DfUBKCgoKCj0D0XIFRQUFDwcRcgVFBQUPBxFyBUUFBQ8HEXIFRQUFDwc76HYaWRkpJyUlDQUu1ZQUFDwWHbs2FEjy3JU1+VDIuRJSUls3759KHatoKCg4LFIklTc3XKnXSuSJP2fJElVkiTts1sWLknSd5IkHTX/DXPFwSooKCgoOE9vfORvAou6LPsNsFaW5dHAWvNrBQUFBYVBxGkhl2W5ADjdZfFVwFvm528BP3LRcSkoKCgoOEl/feQxsizrzM8rgJi+DtTZ2UlpaSltbW39PCSF8wV/f38SEhLw8fEZ6kNRUHBrXDbZKcuyLElSj4VbJElaDiwHGD58+BnrS0tL0Wg0JCUlIUmSqw5LwUORZZna2lpKS0tJTk4e6sNRUHBr+htHXilJUhyA+W9VTxvKsrxSluWpsixPjYo6I3qGtrY2IiIiFBFXAECSJCIiIpQ7NAUFJ+ivkP8PWGp+vhT4tD+DKSKuYI/ye1BQcI7ehB++C2wGxkqSVCpJ0o+BPwIXS5J0FLjI/FpBQUFBwUyrycDaxjIeLdvK2sayAdmH0z5yWZZv6mHVfBcdy5CjUqlIT0/HYDCQnJzM22+/TWho6FAfFgBPPvkkOTk5XHTRRf0aJz8/nxdeeIHPP//cYfmdd97Jgw8+SGpqar/GV1BQgB+aq1jTWMrapnI26Stpl40A1BramB88zOX7G5LMTnclICCA3bt3A7B06VJeeeUVHnvssX6NaTAY8Pbu/9f8u9/9rt9jnI3XX399QMdXUDhfMckyB9rqmOAfZnUH3lNcyK7WWgAkYFJABPODh3FFyIgBOQalaFYPzJw5k7IycRt0/PhxFi1axJQpU8jOzubQoUPW5TNmzCA9PZ3HH38ctVoNCKs3OzubK6+8ktTUVIxGI7/61a/IysoiIyOD1157DQCdTkdOTg4TJ04kLS2NwsJCjEYjy5YtIy0tjfT0dF566SUAli1bxgcffADA2rVrmTRpEunp6dxxxx20t7cDovTBU089xeTJk0lPT7cepzPk5eVZyyao1Woee+wxMjMzmTFjBpWVlQBUV1dzzTXXkJWVRVZWFhs3buzv16yg4HHIsszx9kZWVh/khhNriC16m/QDH3CkvcG6za0Ro7k7cjz/TbmI6swl7Ey9hj8nzCBHEzcgx+SWFrm0cmDGlZc7t53RaGTt2rX8+Mc/BmD58uWsWLGC0aNH88MPP/DTn/6U77//nvvvv5/777+fm266iRUrVjiMsXPnTvbt20dycjIrV64kJCSEbdu20d7ezuzZs1mwYAEfffQRCxcu5LHHHsNoNNLS0sLu3bspKytj3z5RCaG+vt5h3La2NpYtW8batWsZM2YMS5Ys4dVXX+UXv/gFAJGRkezcuZN//vOfvPDCC32ytJubm5kxYwbPPvssDz/8MP/61794/PHHuf/++3nggQeYM2cOp06dYuHChRw8eLDX4ysoeCI1hjYeLt3C903lFHfoHdYN8wmitKOZsf7CFftgTMagHptbCvlQ0draysSJEykrK2P8+PFcfPHF6PV6Nm3axHXXXWfdzmIBb968mU8++QSAm2++mYceesi6zbRp06zxz99++y1FRUVWi7qhoYGjR4+SlZXFHXfcQWdnJz/60Y+YOHEiKSkpnDhxgvvuu4/LLruMBQsWOBzj4cOHSU5OZsyYMYDNBWQR8quvvhqAKVOm8NFHH/Xpe/D19eXyyy+3jvPdd98BsGbNGg4cOGDdrrGxEb1eb70TUVA4XzhtaCO/SUd5ZzP3RqcBEOzlw/t1J2gxGQiVfBim05NU3cboOiMPXLOI4QPg+3YWtxRyZy1nV2Pxkbe0tLBw4UJeeeUVli1bRmhoqNV37ixBQUHW57Is8/e//52FCxeesV1BQQFffPEFy5Yt48EHH2TJkiXs2bOHb775hhUrVrB69Wr+7//+z+n9+vn5AWLi1mAw9OqYLfj4+Fh9ffbjmEwmtmzZgr+/f5/GVVBwV5qNnWzQV7C2qYy1TeXsaqlBBgK9vFkeOR5fLxW+XireTMpjlF8wjTsOsH5HPrIsI0kSxcXF3SY6DhaKj7wbAgMDefnll3nxxRcJDAwkOTmZ//73v4AQ5T179gAwY8YMPvzwQwDee++9HsdbuHAhr776Kp2dnQAcOXKE5uZmiouLiYmJ4Sc/+Ql33nknO3fupKamBpPJxDXXXMMzzzzDzp07HcYaO3YsWq2WY8eOAfD222+Tm5vr8u+gOxYsWMDf//536+veXtwUFNyRj+pOErbnLRYd+4o/Vxaxs6UGH8mLHHUcv47JpM0ccQJwXVgKkwIjSUlKRqVSIUkSKpWKoe6v4JYWuTswadIkMjIyePfdd3nnnXe45557eOaZZ+js7OTGG28kMzOTv/71r9x66608++yzLFq0iJCQkG7HuvPOO9FqtUyePBlZlomKiuKTTz4hPz+fP//5z/j4+KBWq1m1ahVlZWXcfvvtmEwmAP7whz84jOXv788bb7zBddddh8FgICsri7vvvrtXn23t2rUkJCRYX1suUufi5Zdf5mc/+xkZGRkYDAZycnLOmBtQUHBHjLKJXS21rG0q4/umcqYHRfO7+KkATAgIwyjLTAuMYq4mnvnBw5itjiXQq2d5TExMZMmSJWi1WpKSkkhMTBysj9Itkiz3WB5lwJg6darctbHEwYMHGT9+/KAfS39oaWkhICAASZJ47733ePfdd/n0034ltyp0wRN/FwruweG2er5pLOX7xjLW63XUGzus6yYGRLAr9RpA3GU3GDsI9fYbqkN1GkmSdsiyPLXrcsUi7wc7duzg3nvvRZZlQkNDe+XLVlBQcB2yLHOio4kIlZ9VkF+u2sc/q22T8yP9gpmniWe+Zhh5dmGAkiQNmIiXlJQMitWuCHk/yM7OtvrLFRQUBpeyjma+N7tKvm8q51SHnjdG5LIsciwAV4aMoNHYwTzNMOZp4hnhpxnU4yspKWHVqlUYjUZUKhVLliwZMDFXhFxBQcGj+EXJJr5qKHFIwAEIV/nRaOq0vl4YksjCkKHzXWu1WoxGI7IsYzQa0Wq1ipArKChcWDQaOyho0lGor+C5YVmoJBFkt6ulliPtDai9fMjVxIkJSk08GQEReLlRxcykpCRUKpXVIh/IyBZFyBUUFNyCFpOBTfoKq6tke3M1RkQwxvVhKUwJEn0MnomfikqSyAqKxkdy3wjqwYxsUYRcQUFhSLAk0wAca2tgwoH/0iGbrOu9kZgVFMM8TTwR3rYktOwBqlcyECQmJg5KaKL7Xs6GAJVKZS1gdd1119HS0tLnseyLXN15550Oqe1dyc/PZ9OmTdbXK1asYNWqVX3etwWtVktaWto5t3vuuef6vS9nsC/MpXDhYZBNbG2u4k8Vu1l49EsWHfvKui7FL5gQlS9TAiN5KCaDL0ct4vTEpWwcdxW/H5ZF0iBPVHoaikVuh30Z21tuuYUVK1bw4IMPWtf3tSTtuQpX5efno1armTVrFkCvE3z6y3PPPcejjz7aq/dY/H4KCmdD297EJ/Vavm8qY32TzmEy0kfyosVkINDLGy9Jojj9ZgLOkoSj0DOKRd4D2dnZHDt2zOmStLIsc++99zJ27Fguuugiqqps7UvtLdGvv/6ayZMnk5mZyfz589FqtaxYsYKXXnqJiRMnUlhYyG9/+1teeOEFQKTBz5gxg4yMDBYvXkxdXZ11zF//+tdMmzaNMWPGUFhYeNbP8+abb3L11VezaNEiRo8ezcMPPwzAb37zG2uxsFtuuQWAf//730ybNo2JEydy1113YTSKFGW1Ws0vf/lLMjMz+cMf/uBQSCw/P99aaOuee+5h6tSpTJgwgaeeeqrf/wsFz0CWZQ611XOszRZNskFfwQOlm/ms4RSNpk5G+gWzPHIc7ybP41T6zQ7Zk+eLiJeUlFBYWEhJScmg7dNtvzlpR8+1bF8bns3yKJHtt7L6IHed6lnE5Cm9r8BlMBj46quvWLRoEeBcSdpdu3Zx+PBhDhw4QGVlJampqdxxxx0O41ZXV/OTn/yEgoICkpOTOX36NOHh4dx9992o1Wpr9cS1a9da37NkyRL+/ve/k5uby5NPPsnTTz/NX//6V+txbt26lS+//JKnn36aNWvWnPVz7d69m127duHn58fYsWO57777+OMf/8g//vEP653IwYMHef/999m4cSM+Pj789Kc/5Z133mHJkiU0Nzczffp0XnzxRQwGAykpKTQ3NxMUFMT777/PjTfeCMCzzz5LeHg4RqOR+fPnU1RUREbG4Jb1VBgcTrY3Wicn1zWVo+ts4a7I8awYkQ3APE08t4WPZn7wMOZq4hnue35XyhzM2HF73FbIhwKLZQrCIv/xj3/Mpk2bnCpJW1BQwE033YRKpSI+Pp558+adMf6WLVvIycmxjhUeHn7W42loaKC+vt5aFGvp0qUOVrB9yVqtVnvOzzd//nxrPZjU1FSKi4vP+JGtXbuWHTt2kJWVZf1OoqOjATGHcM01Iq3Z29ubRYsW8dlnn3HttdfyxRdf8PzzzwOwevVqVq5cicFgQKfTceDAAUXIzzOe0+1iZc3BM+pyR3sHoFb5WF/H+waxKnnuYB/ekDGYseP2uK2QO2tJL48ab7XO+4u9j9weZ0rSfvnlly45ht7Q25K1lu3P9h5Zllm6dOkZxbpAFOyy94vfeOON/OMf/yA8PJypU6ei0Wg4efIkL7zwAtu2bSMsLIxly5bR1tbWl4+n4AZUdbaSb7a4H4xJZ4y5cUKDsYPiDj1hKj/yzLHc8zTxpNq1O7sQGczYcXtc4iOXJOkBSZL2S5K0T5KkdyVJOm8LVvdUkjYnJ4f3338fo9GITqdj3bp1Z7x3xowZFBQUcPLkSQBOnz4NgEajoamp6YztQ0JCCAsLs/q/B6pkrY+Pj/XzzJ8/nw8++MDq4z99+jTFxcXdvi83N5edO3fyr3/9y+pWaWxsJCgoiJCQECorK/nqq6+6fa+Ce1JnaOeTei0/P7WRtP3/JabobW44uZbXag7ybWOpdbu7osazY/zVVGfexkcjF3BfdBoTAsIvaBEHW+z43LlzB82tAi6wyCVJGgb8HEiVZblVkqTVwI3Am/0d2x3pqSTt4sWL+f7770lNTWX48OHMnDnzjPdGRUWxcuVKrr76akwmE9HR0Xz33XdcccUVXHvttXz66acO9b4B3nrrLe6++25aWlpISUnhjTfecPlnWr58ORkZGUyePJl33nmHZ555hgULFmAymfDx8eGVV15hxIgzm8aqVCouv/xy3nzzTd566y0AMjMzmTRpEuPGjSMxMZHZs2e7/HgVXEebyYC/eZLRKJtI3vcuDXZVAgMkFbPVsczTxHNRsK30cYpf8KAfq6cwWLHj9vS7jK1ZyLcAmUAj8AnwsizL3/b0nvOljK3CwKP8LlxLs7GTjc2VrDNPTu5vraM68zarmF99/FtqDe3M1cQxTzOM6UHR+HkpYabuwoCVsZVluUySpBeAU0Ar8G13Ii5J0nJgOTCkLZEUFC40yjqaea3mIOuayvmhuYpOu+xJFRL7W+us6e8fplx8wbtHPBFXuFbCgKuAZKAe+K8kSbfKsvxv++1kWV4JrARhkfd3vwoKCmfSYTKyraUavbHTWvmvTTbye51oGeiFxNTAKOaaJyjnqGPRqHyt71dE3DNxRdTKRcBJWZarASRJ+giYBfz7rO9SUFDoNwbZxI6WGqurZIO+ghaTgfSAcKuQp/hqeCJuMlMDI8lRx3lEJxyF3uEKIT8FzJAkKRDhWpkPKAU1FBQGmNeqD/Cr0h9oskt7B0j1DyNXHYdJlvGSJCRJsvanVDg/cYWP/AdJkj4AdgIGYBdmF4qCgkL/MMkye1tPs66pnPymcm4IH8lN4aMAiPEJpMnUyWi/EKurJE8TT6xP4BAftcJg45KEIFmWnwKUohoKCi7gYGsda5vKWNdUzvomHbXGduu6IJWPVcgXBCdQmn4Lw3yDehpKwc0YqB6eStEsOyoqKrjxxhsZOXIkU6ZM4dJLL+XIkSN9GquwsJAJEyYwceJEysrKuPbaa7vdbqhKu9qX2e2JruV1Bwr7glsXGrIsc7itHoNdJMm9JRu5r2QTH9VrqTW2k+gTxJLw0bwxIpc/DJtm3S7Qy1sRcQ/CUodl3bp1rFq1yqVFtdw2RX+wkWWZxYsXs3TpUt577z0A9uzZQ2VlJWPGjOn1eO+88w6PPPIIt956K8A5RdMd6Vpe1xn6Wur3QkGWZY63N1onJ/P1OnSdLWweexUz1DEAXB2aTKxPIHM18czVxJPiq3GIJhmszuwKrmUg67AoFrmZdevW4ePj41ALPDMzk+zsbGRZ5le/+hVpaWmkp6fz/vvvA0Lo8vLyuPbaaxk3bhy33HILsizz+uuvs3r1ap544gluueUWhwYPra2t3HjjjYwfP57FixfT2tpq3d+3337LzJkzmTx5Mtdddx16vShIlJSUxFNPPcXkyZNJT0/n0KFDAOj1em6//XbS09PJyMjgww8/POs4PdHd+N2V162uruaaa64hKyuLrKwsNm7cCMBvf/tbbrvtNmbPns1tt93GjBkz2L9/v3V8y13H1q1bmTlzJpMmTWLWrFkcPny4v/82j6HJ2MFtJ79n+N7/MHr/+yw/Vci7dcfRdbYQ7R2ArtPWxORn0RN4J3ked0aOY6Rf8BkiPlBWncLAYqnDIkmSy+uwuK/pJN3V87rXboHlOeL5ygK4652et5Vfc2p3+/btY8qUKd2u++ijj9i9ezd79uyhpqaGrKwscnLE/nft2sX+/fuJj49n9uzZbNy4kTvvvJMNGzZw+eWXc+211zpUJnz11VcJDAzk4MGDFBUVMXnyZABqamp45plnWLNmDUFBQfzpT3/iL3/5C08++SQAkZGR7Ny5k3/+85+88MILvP766/z+978nJCSEvXv3AlBXV3fOcXqiu/G7lte9+eabeeCBB5gzZw6nTp1i4cKFHDx4EIADBw6wYcMGAgICeOmll1i9ejVPP/00Op0OnU7H1KlTaWxspLCwEG9vb9asWcOjjz5qvficT5zq0LOuqRxtexNPxYvflNrLh28by6gytBKh8rM2DZ6nGcZ4/1Cn47edteoUq939GMgenu4r5G7Ehg0brCVqY2JiyM3NZdu2bQQHBzNt2jQSEkQNiokTJ6LVapkzZ06PYxUUFPDzn/8cgIyMDGt51y1btnDgwAFrbZKOjg6Hei32JWs/+ugjANasWWN1AwGEhYXx+eefn3Wcnuhu/K6sWbPGoWVdY2Oj1dq/8sorCQgIAOD6669nwYIFPP3006xevdo6P9DQ0MDSpUs5evQokiRZC3V5OmUdzTZXSVM5JzpEATQvJH4Rk06IyhdJkngjKZcEnyDSAsL73O3dmep6Q1UTW+HcDFQdFvcVcictaZbn2KzzfjBhwoQ++bGdKQ3rDLIsc/HFF/Puu++edT/n2se5xukJZ8Y3mUxs2bIFf/8zi1val/odNmwYERERFBUV8f7777NixQoAnnjiCebOncvHH3+MVqslLy+vV8foLhhkE97m7u3/q9dy1XHHihTBXj7kauLI08RjX8vo0pD+l6ZwxqobqprYCkOH4iM3M2/ePNrb21m50hYCX1RURGFhIdnZ2dYStdXV1RQUFDBt2rSzjNYzOTk5/Oc//wGEO6eoqAgQJW43btzIsWPHAGhubj5nxMzFF1/MK6+8Yn1dV1fXp3F6omt53QULFjhUZ+yudruFG264geeff56GhgbrXUdDQwPDhg0DROs5T6Gis4X3Tx/n7uJCxu57n7uKbR2ppgdFE+zlw6XBifx52HS2j1vM6YlL+d+oRTwYkzEgWZSJiYlkZ2f3KM4D6YtVcE8UITcjSRIff/wxa9asYeTIkUyYMIFHHnmE2NhYFi9eTEZGBpmZmcybN4/nn3+e2NjYPu3nnnvuQa/XM378eJ588kmrXz4qKoo333yTm266iYyMDGbOnGmd1OyJxx9/nLq6OtLS0sjMzGTdunV9GqcnrrjiCj7++GPrZOfLL7/M9u3bycjIIDU11Wppd8e1117Le++9x/XXX29d9vDDD/PII48wadKkPt+5DBbfN5bx01MbGL9/NXFF/+ZGc03uI+0NbGuptm4X4xPI6YlL+WL0JTwUm8mUoChU0tCeVkNVE1th6Oh3Gdu+oJSxVXCWwfhdVHe2sl6vIyswihF+GgAeL9vGsxW7ABGvPUcdKzIn1XFMCYrCZ4jFWuHCZMDK2CooeBrVna0U6HXkN+nIbypnX1sdAC8lzOQXMekALA5NItDLmzxNHFMDo/BVanJ7BBdqtI4i5AoXFLmHP6NAr3NY5m/ugmOfJTklKMpao1vh3LiDgLpttI62Bj4rEo8lM+DWGS7fhVsJuSzLSj1kBSt9dfvVGtooaNKRr9exQV9B4dgrCTR3wAlV+eIvqZiljiHPnDmZFRildMHpB+4ioG4TrWMyweYT8PleId77y23r1H7nt5D7+/tTW1tLRESEIuYKyLJMbW1tt6GOXWk2dvJtYyn5euEqKWo97bB+s76S+cEiWubV4XOI8PZXhNuFnE1AB9NSH6oO9gC0dkCAuUGHDPzoVagxZ1Rr/GHRBLgiAy5JG5Ddu42QJyQkUFpaSnV19bk3Vrgg8Pf3tyZb2XPa0EZJRzOZgREA1BjauPrEd9b1fpKKGUHR5rKucUwPiraui1eKTLmcngR0sC31gcyc7JYT1TaXyeYToHseggNA5QU/mQMtHUK8s0eD78BKrdsIuY+PD8nJyUN9GApuyGlDGwX6CvKbyslv0lHUWst4/zD2T7gOgBF+Gm4MG8lY/xDmauKZHhRtbSasMPD0JKBD4eoY0A72BqMQ7M+KhNvkoN1ci5cE27Qw3xxh9dzigTmGHlB+7Qpuy6f1Wp4q30FRay323nJfyYsob3/aTUari+TdlPlDc5AKQPcCOqSuDlfRaQQfsxuuqglyXrCtCwmAhak2l0mEemiOEUXIFdwA+8nJbHUs14alAOAtebGntRY/ScX0oGjmmtPepwdFE6BY3G7PoLs6XIEsw5FKYXF/XgTlDXDoaZAkiA+F66ZAQihckQlzRtlEfohRzgaFQafGItxN5azX6xwmJys6W6xCnquOI3/M5YqrxIMZUFeHq+gwwIZjQrg/2wvHqmzrVF5QXAtJkeL16uVDc4znQDk7FAac6s5WQr39rNmQt578nm8aS63r/SQVM4OiydPEsyDYNrmpVvmQq4kf9ONVuAAwmoRIgxDx+S/Z1oUHwaVpcFm6iDYJdf8eqIqQK7icKmvmpOg5ua+tjsKxVzJHLerTLAxOoN1kZK4mnlxzVIlicSsMKLIMRaXCZfLFXojSwKc/FevmjILpyZA3Bi7PgJkpNpF38SEcboAQX4hz8bXBJWePJEmhwOtAGiKK8g5Zlje7YmwFz6DVZOCh0i3kN+k4YE55t+AvqTjZ3mgV8gdiMnggJmMoDlPhQqKlA9YdFi6Tz/dCqd3vMjTQNpHp6w1bfuPy3csyHGmA/HLI14m/Fa3wh2nwm4mu3ZerzKC/AV/LsnytJEm+gPvfiyj0mYrOFtY36Tja3sDjcaLDkb+k4pN6LeWdLQRIKmapY8nTxJGrjmNaULSSgNNPnEmscYc0+SHH3mXy/zbAz9+3ruoI96djwTjUN8yEi8a5fKJSluFYoxDsdWbx1rU4bhMT4NJdWum3kEuSFALkAMsAZFnuADr6O66C+1De0cx6O1fJ4fYGACTgZ1ETCPP2Q5IkXhk+h0hvfyXl3cU4k1jjLmnyg47RBD+cFO6Sz/fCJRPgj6LbFZdnwFubachJ4sO2w5RGq1D5nGbJlAgS1efOGD4XsgwnmsyibRbusmbHbaL8IS8e5sZDXhyMCxUBMK7GFRZ5MlANvCFJUiawA7hflmWHjyRJ0nJgOcDw4f3vlKIwcHSYjNZqf182nOKyY187rA/y8ma2OpZcdRwmuwjvH4UmDeZhXjA4k1jjNnVGGIQ7g7pm+OaAEO+v9kGtndT42UlaciRsf4yiwkJK1x3vdwkBWYaTTY4Wd2kX4Y70F4KdGyfEOzVsYIS7K64Qcm9gMnCfLMs/SJL0N+A3wBP2G8myvBJYCaIeuQv2q9AP7H/ApuhQ1ut1rDeHBM5Wx7IqeS4A04Ki0Xj5COHWxCn1uIcAZxJr3CX5ZkDuDGQZTLLNZfLIJ/BagW19SqSwvi9Ph5zRZ7y9PyUETjba/NvryqGki3BH+NlEOzceJoSJJM/BxhVCXgqUyrL8g/n1BwghV3BTSkpKeOLLd9kT44dW2kd9laMbJLDF9rOI9PanbuLSIe96cyHjTGKNuyTfuOzOoLUD8o8Iq/uLvfD0FbDE3ET8qkw4XCHE+7J0GBtzVrO3NyUETKGJ5Ots7pJiveNY4X6QE2dzlaSFD41wd6XfQi7LcoUkSSWSJI2VZfkwMB84cK73KQwOsixzrL2R9Xodueo4RvuHoNVqKQ72ZneimHlRy17khSaQZ+6AM9FcjMqCIuJDjzOJNc5sM9Buj37dGZSctgn32kPQ2mlbt+6wTcgvSet1FcGeSgg0+YRyXDWcYr9kXtdOoOSg4/vC/CAn1ubnTncT4e6Kq6JW7gPeMUesnABud9G4Cr1ElmUOtzew3lxgqkCvo7xTTJ3/edh0HorNJCkpibSdG/HvbCSl3sivfnQTScq8xXnPYEyInu3O4IyLiMEo3CUWa/qW/weFx2yDTUoUFvdl6ZCV1PM4vaBUb+8qSeRExM9tK1sh1NdmcefGQWaEewp3V1wi5LIs7+RCJw4AACAASURBVAbO6COnMPB0bcaRceADa+syC5He/uSq4xjnHwqIk+3hq28Z8tvw8wVPCfsbrAnR7qxfy0XEr7GD08c7Ce+MIahQC9/eD1OTxEbXTYFINVyaLjIr40PPGLs3F6OSkhK2Hy2nNGgURe0R5JeL8EB7QnwhO1YI99x4yAgfkFygAUdJp/MwTLJMUWst65t0rNfr2Kyv5EjaDWhUoqj9GP8Qqg1tYmJSE0+uOo7x/qFnNOvwiBoYHoAnhf0NyYSoLMPOUxhf/Zyl3+gYVtqJ+CXWivUbjtmE/L554nEWznUx0rUIa/vzo3q+ORlIrfd0h/drfITFnWe2uidGDKJwVzYK8z5K4/KhFSH3AGoNbbxZe4T1TeVs0FdSZ2x3WP9DcxUXmWuUvJmUh9rLR+myNEj0xcodbAvefn+DMiHq0C1Hhkv+TlJ1EwAGFZxK8ifs1hzCbpkDo2N6NXTXi1FgXArvH7dNTh5usGypBm81vqZ2RnSeYn6iN3fMSmZSJHj3Urj79P+q1cOOU7BdC9uLxaOkDn5/JTx+We8OwAkUIXczOmUTO5qrqTd2sChE/GjaTUYeKt1i3WaEr5pcdRy5mjhyNfGk+Nqu8BbLXGFw6K2VO9gWfHf7y87Odu1OZFn0pfxyn3hsPQm6P4t63V5esGwmNLRSPS2WIwleDE8dTVgfP7NfZCLhF9/Ft9oO9nVG8fj3Pg7rg7whOw4mBtRT98PHxLSX4qvyYsmUJSRG9zDoWXDq/9XQKiZq04bZXkf+8szB1H6ibMAAoAj5ENNuMrK1uYr1eh0FTTo2NVfSbDIw2i+EIyE3AKI92a9iMpgQEE6uOo4kP9ffmin0jd6G/Q124s6A7a+tE749YBbvvcLatKDygh3FMG+ceP38NQBEmR+9oaoV1uvg8yNNFFR4oe0IAiKt6wO9YU6sLRxwShT4eAGEUjLyon7ffXT9/k4dPEZicbujpX24EoaHQ/EfxJtCAiB9mOjVOXUETBkuJmvHxAyYH0cR8iFkZfVB7i/ZRJtsdFg+1i+EXE0cBtmEtzn07/kE13feVuie3t5K92a+oTsLfiBdLS7zi8uyaCZs8e/Wt8BV/7StjwkWJV8vTYOLx0NY33qj1rTBervMyf3W64PYr4/cyYxIIwuT/ZkbD1nRFuE+k37PA7V3khwZR4H5+5u6o4VZT78vkpPs8fUWn9/epbTnicFJ6TSjCPkAozd2slFfYc2cvDViNPdEpQIw3FdNm2wkzT+MXE0cOeo4cjRxxPooNceGioF2fXS14IFB3V+vxm5uh+8PwVf7RSo8wIlnhUDFhgiXSUqUEO9JicKN0ktq24TFbalVsve043p/FUzwqyOkcjcj2k+SYCjn4tRcsie72D1kMMIBnei7ua1Y/N1bRsLvr7R+f2NHmpC+eA8yhwkLe+oI8UgfdmZz5UGeo1KEfAD4rrGU7xpLWd+kY0dLDUa7eiSxPgFWIZ+riac6cwmR3v0v4KPgGgbD9WFvKRYWFg7q/s5JeT28t02Id8FR0T3HQqRa9K2MCRav31jW62M53QYFFbaU96Iuwu2nglkxNlfJtGioKtezatUm10XbyLKj0F77mnAP2ScggdimvMH2/XUYKJ2WyMmKMrcLNVWEvJ9Ud7ayQV/BopBEax/Jv1Xt44uGUwCokJgWGEWOuaSrpSY3gJ+XSqkS6GYMdojekNdIaWoTERaWVmaHKuCXHwAgSyBNT6ZhZgInxgURuXAyiRYRd5K6dijU2Vwle2pxaKTtp4KZ0bbMyWlR4N9Flfp1VyHLcOq02dLWCmt7VwmcfNbW+cdkEiI+MkpY2BZre1IiBNvqzpZU6li1+l23DDWVZHnw61dNnTpV3r59+6Dv1xXoOlsoMMdwr7drorBuzOXkmduSvX/6OEWtteSo45iljlEiSdwId6zrPaj7k2XYVw5f7xNW94ZjsCAVPr9XHMtxLaevf5njyb5oxwSSe81lfP31106LV0MHFNgVmdrdRbh9vWBGDMyNE0WmZkafKdwu4Wgl/GK1EG5z6KMD6x6EvLHiubZGCHb42f36hYWFrFu3zpqEN3fuXNdHAJ0DSZJ2yLJ8RvKlYpE7SZ2hnemHPuFoe4PDcn9JxYwgx7imG8JHcgMjB/PwFJzAWf/3YCdLDcr+NhyDNzbB1/uF+8SClwTtBqu7QVtewrqrQs1iBQcOHDir66ehAzbYuUp21TrOBfp4wfRom6tkZgwEuEp1GltFrLbF2h4eDi9eJ9YFB4iIGhACnWW2tC2PuBDbOOa7kXNdUIf87uksKEJuhyzLHG1voEBfQUGTjmpDG1+NvgSAUJUvbSYDai8fZqtjyDHHcU9Vmih4DO5Us3tAMZpEWFykWrgLAHaXwP9tFM/jQkSEyaIJcNF4B0u0q1ilpqZy6tQp6+vIhGS+PGWbnNxR4yjc3pIQa4urZGaMCBF0GV/shdXbhaV9qEJcgCyMirYJeUwwfHQ3ZCaIuuR2PvGSkhK0hUUOgu3MRd5dKkx2xwUv5KUdej6tL6bAHMddYWh1WF9jaCPS2x9JkigYeyUJvkHWkEAFz8KdLap+U1Yn4rq/OQDfHYTTzfDwAviTiOHmigyRjLJogoiy6CGqoqtYhcUmckAaztcn2zhgiOapNX4Yuwj39BhbyvusGAjy6XZo5zGZRGz2Vq1ILvpJNkw0i+Y2LawyJ8f5qIRQZyXBtCSHwloALJ50xtA9CbazF3l3LW1xQQm5QTaxp6UWH8mLDHOp1l0ttdxbstG6TZS3v9XazlHHEa7ys65TEnEGloH2FbuzRdVn/vwNvLVFZFbakxzp6PMdEQEPLzzncPpOOCglss4vkfwdsK0ajLItjUdlsbjNXXBmx4LaFcL96R6bcG8rFpOwFkZF24T8RxMhSi1EOzMB/Hq3854E29Mv8ue1kHeYjGxvqaZAX8H6Jh0b9RU0mTq5KWwk/0mZD8AcdSw3h48i1xzDPdYvRKlTMgQMVuq6u1pU58SSBv/tAbhlui0E8ESNWB7oC3PHwsJUYXWPinYqlrm5EzZVmn3cOthWBQY7i1slOfq4+y3c9S3C7XO8Gu7KEcskCe5+R4Q2WkgIs1nZF4+3LZ+YaBP1PtCTYHv6Rf68FfJfl/7Ay1X7zsiaHOkXzHBftfV1mLcf7ySfveKawsBzwfive0NVI6w5JMT7u4O2ScpIta3Jws/y4PqpMCvFKeu0xQCbK21FprZWQ6fJtt5LEiGAlvZlc2JB09egqw4DFJUKS/uHk+LvoQrbjm6ZBmp/IeR35QjLfFrymZORLuRsgu2xF3k8XMgbjB1s0ldYLe6/JMxghlpUUwtW+dAmGxnvH2q1tnPUcQzz7VvqsMLA4um3ti7FaIJZfxLCZ09ssAgVTLGrWGIp1NQDrWbhtrQv+6HqTOGeGiWs7bx4UZs7uC/CLctw0jzzOcocxfVZkUi2scfXW8RnT0sSsduWbva/u7IPO+0bnizYPeFRQm6UTXxaX0yhXkeBvoLdLbUOXdzz9TqrkN8VlcryyPFE+QT0NJyCG9GXW1tPaejQIyYT7C0T1vaWk7D6J5SUlaHVasnCiL+fN+SOEeJ98fizTlJaaDPAliqbq2RLJXTYCbcETI60uUqy40RzhV5T12yztC3Wdo0ebp8F/7dUbDMtSfTTzEqC6cnikZlwZjq7Qr/xqG/UC4mfntpApTmyxEfyYkZgNDmaOLLVscy2y5pU0t49j95YSp7U0MGBU6dh7UEh3msPOfiFK77azqrd32E0GtmTLfOjf/+ShNEpZx2uzSCsbEv7ss1V0G7nTZSASRG2cMDsWAj163G47uk0CtPdUrnv9jfhzc1nbhelEaVaLSSGw6Hf9XJnCn3Bo4RckiR+Fj0BkyyTo4ljelA0gV4e9REUXIQrfOqDYtEbjOBtzjPYWwYZXYRtWKiwti9O5YTUZP1Mp4MlTlaUnSHk7UazcJsTcLoKN4g+kxZXSU4shPfWpimtE1b2lhPiTmFHMax/yBbeFx8Kft60p8VSk6LGL3c8kZdOhaSIQS8W5fF3ZS7C41TwibjJQ30ICm5Af33qA2bRt3bApuNiknLtIRFNkm9uMjAhToQFpg+D+eOEgI+LtYpfYkkJql1bHD5Th1FMSFqEe1MltHUR7vRwm6skJw4i+nIz2tAKP14lxLus/sz1+8ttQv7rhZTcMZFV772D0diEqn4HS7wnkDgEIu6Rd2UDgMuEXJIkFbAdKJNl+XJXjTsYKFd196a7/09/w8X626LNMkZSUhKJbb6weocQ7k3HRcq7hUBf0YTB30eUeT3+zFmTcW68dQnfHKpF65vEHUWhbPwWWrsId1qYzVWSEweRzgq3LIO2Voj15hPQ2AZvLhPrNH6iZG1di2iMMC0JZqbAjBTxPMIW6UVwANo9JUMeZaREOtlwpUV+P3AQ6F15tCFGuaoPPr25cJ7t/9Of6IPeWvTW4+g0EFttxOAtUROhQqVScXfQRCIe/59t44mJcNE4YXVnjxYibqGLiHeaYLudxb2xMpEWg+NnSg2zFZnKjYPo3szfH62Ej3cL4d58QjQAtuDtBa/eLJoheHnBu3eKeiVjY85ZW9wdoozc4RjcBZcIuSRJCcBlwLPAg64Yc7BQruqDS28vnAP1/3HaopdlOFRB+8pvWPxtFUknOwhsNbF9aiBfXB6K0Wjk2AhvIu7OEcKdN1bEefeAwSTqk1jiuDdUQLPBcZvxocLitmRPxjjTZ0SWRbu1TceF+2Z6sli+vRh+/ZFtu4ggYWXPND+87eoELZzgxI4E7pBA4w7HYGGo7+pdZZH/FXgYSz+mbpAkaTmwHGD48OEu2m3/Ua7qg0tvhXkg/z/ntOif+h+sLISKRkbZLW4IUdEWqEKSJFQqFfFTU2Fx9+MYTLCzxlZkqrBCpMHbMyZEuEnmmi3uWGeEu8MgCmFtOg6bToi/Ft/28mybkM8ZBXfn2ITbyYxPZ3CHeGx3OAZ3uKvvt5BLknQ5UCXL8g5JkvJ62k6W5ZXAShD1yPu7X1fhTlf1C4HeCrMr/j9ntZZkWaSL5x+B/MPwxGUw1hzG2toJFY0iHX7eWE5nRnF8uA+xs9MYIUnM7WZMo0mUcrW4SgoroKkb4bZkTubFQ5wzwl2jh7BAWwjgwr+JY7YnLFCItX3xqMRwePUWp74nZxhqy9MdcYe7eldY5LOBKyVJuhTwB4IlSfq3LMu3umDsQcEdruoXCn0R5v78f86wlm67jcQ2P1h/xCzeRxzrc89MsQn5z/JEgos5siQcCO9yXEazxW1xlRTooLGLcI8KtrlK8uJh2LmSi2UZjlTCxuPiseGYeL37ccg0fw/TkqCySaTmz0yB2aNgTHSf+mY6iztYnu6IO9zV91vIZVl+BHgEwGyRP+RJIq4w+AzGhdNiOTbU1xNU205DiDjRtMdPknjlh6Bvt20cqRYZlLmj4dJ02/IREWeMa5JFuzKLxV1QIZor2JOscXSVJPbsMnekVg93rBJukhq947oAH3HnYBHyPyy2lagdJAbb8vQU698d7uo9Lo5coXd4ysngMowmKr7bwYF/fMjwE61MPtXBwnaZ5x+JAz8VSaNSRCnUtk4h3nPHwvjYHi1Zkyw6u9sLd1274zYpGpurJDcehp9LuOtbRATJhmOiXOvLN4rloYHCvdPYJuqqzB4Fc0aKvxMTRf1tCwNoeffEYFqenmb9D/VdvUuFXJblfCDflWMq9B1POxn6xd4yeORj2HCM2IZWYu1WtYX5sXDMFGJyM8Xnf/uOHocxybC/zuYqWa+D012EO0ljc5PkxcGIc5Wpr24SMdqFx4R4F5XZOtv4ecOfrxGVC1Ve8MFdoqtPl6427sBgWp7u4Hf2JBSL/DzmvDwZ9G3Cmi08BtEauHeuWB7gI9qAAYbEEA5EdKId7ktpSgBXPHAHU3uIlDLJcKDOZnGv10FtF+EerrZlTubFCyHvEVmGY1VClC1VCr/YC7e/ZdvGRyW6tGePElEl9oJ9cWovv5DBZbAsT3fwO3sS54WQX3DuAyc5L06GqkZhxVqs2V0lIjQEhLvBLOQlvm3on5pDyCWTiJ+eRlhJCQ1aLZO6/CZkGQ7W2yzufB3UtDnuMiHI1rpsrlm4ezSOTSbYr4OCI1BwVDwqGsVE6T9uEttYKhhmjxIJQtOSRBKOQo+4g9/Zk/B4Ie/qPli0aBEtLS3KPx8PPBks0RpRGlubsj98DX9da9tG5SXC67JHCYHE/Bt4+22MXkZUa06xJD7EajnKMhyss1UHzNdBlWNbVoYF2aztufHC5+2UV+PhD+H/bRT9Me2J0og7BAvJkfDN/b3+Oi50htrv7El4vJDbuw8MBgNffvklsiyf/z5hJ3Hrk6G9E3aesoXYbTRHa/zrNrhzjthm/jjh/54zSoj3jBQIcqzD6vAbMBpZf7iSpqbEHoU7LtDmKpkbDyODzyLcBqO4C8g/DOuPwkvXwegY27rTzaKCYc5oc+TLGJHi7mb+bYXzG48Xcnv3gSRJyLLs1j5hxQ2EsLwveVnEcLd3yU+PCRYRJRYuzxCPswxljBzFjqAmTngPR+ubhP6EGk7YtokNsFnbeXEwOuQsOms0ibKtlhhzS2SJhSszbEL+i4vgvnkDVr71QvmtXCifcyDxeCG3dx8EBgby9ddfu61P2F2jSAbkRDKZ4GCFOYX8OOwsge2Piok+SYIOoxDxCfEwe6T5MQpSzh6tcepUCRuOVlASMJI9reHk60DXEgeaOOs2MQG2cMC58SKT8qzCfbwaxsTYXue9KLI6LYyOtsWZX2TXCHh4OM7S2+/YXX8rruZC+ZwDjccLOTi6D6Kjo9326u6OUSQuPZHK6oTPeNNx0ZCgoYtPY08JTE0Sz1+7RZRGDT97mqMsw4kmMTn55bFm1paE0KhyPL4of1so4Nx4GBd6jsnJojJYd1g81h8Bowx1L4mLjK833DxNDDB3rBDvYWF9+z7M9OU7dva34unWrDueE57IeSHk9rizT9gdo0j6dCLJMhytEmGAAT6iizuIRJanPrNtlxAmLO1ZI0UqeabduBb3RDdDn2yyhQPm66DUOpcYBCoIMLWQ3KFlwQhf7sweRWqYE56NfWXw28+FeHednEyJhJLTtnDB15ecY7De0Zfv2JnfyvlgzbrjOeGJnHdC7s64YxTJ2U4ki7WXHBlHQnmnEO4t5hZgtWYxzEqyCfnYGPjVAhEjPTNFFGxyAm0X4T7VJTs93E+4SiYG1FGz8b+Ed1Tgo1KxZPKS7ndRXGvrznNjlljmrYIPd4rniWEwb5ywuOeO7ZWLpC/0Rayc+a3015p1B2veHc8JT0SS5cEvRDh16lR5+/btg75fhe5x6HxjkqksKMJ7WBhfbM7HaDRyyVcNZG3poq6xwUKsc0aLSb9ecEpvi+NeVw7FXYYOMwu3xVWSFi56/3Y9VutJX90kLO01B+H7w8LnDTB5OOx4TDyXZXhrs4h+GRl1XvSW7I9Ffj5Y8xcikiTtkGV5atflikXuJO5gvQwIp5tJ3NdA4g/1tOb/B344SWKbzKeLwzBODESWZYoTfRhXH4Hm4gxbXesRzkdqlOgdLe6TTY7rQ33Nwm32c2dE2IS7K2e4zp75Ap74n+NGIQGQN0b0xJRlcZySBMtm9eKLcS0D4fLrjzXrrr7p8/Y8G2AUIXeC88Z6MZlsxZZkGSY9A3tKrastHcQaNV6oOk1IZqE+nKlh+otL0Dj5mcuaHTMnjzc6rg/xhexYW1RJRritzHa3GIyi082ag6Kp8V3ZcNM0sW5crKhXkj1axJzPHycs8bMOeP7Q1wuEO/qmz5vzbAhQhNwJ3NV6OSuyLHzFP5w0P7Qia7Liz0LkJAnUfqKf5NQRMD2ZmlHB/OfEZurVovPNpU5myZY32zIn15XDsS7CHewjmgRbYrkzzyXcIHpNfndQPNYddoyAGRFuE/IrMqD+r459MYcQT7Eoh9o33d335JHnmZM0GDvYpK8gyVfD+ID+RUF1hyLkTuDMhGBfToYBOel3l8CT/4OtWsdGuxaOVMJ4c8z16uUindxcHjUSWFwy/pzHpGuB9XaukiMNjus1FuE2i/ekCCeEu9bcAcdyx3D7WyLT08KoaOEquXi8cJtY8HMPAQfPsyiHKsKrp+/JHe8S+kp1Zysb9BUU6HUU6CvY3VKLCZlfx2Tyx4TpLt+fIuRO0JP1MmSTTZ1GKCoVlvZWLaQPg19eLNZ5SfBZkXgeHiQKNE1PhhnJMC3ZMW47PrTbz9r1OCpbHC3uw12EW+0jXCUW4Z4cKRq0n5UOg4g3/+4gfHsAdpyCHY/CJHOVwmsni/DFi8eLJJxumjy4G+ezRelKevqehvouoT9UdLYQ62Pr2Tfr8Kcca7cZUj6SF9MDo0nyO1fN476hCLmTdCdw/Tlxe/3eb/bD1/uFeO8qcUxjnz3SJuQT4qn921WciJSJnZNOYh8aXVe1inKuFuE+WO+4Psgb5sTaek5OcUa4QRzzvwrhmwMi/b3Zrl6sr7fIBLUIeS8jYdyB88miHEjO9j25cx6IBVmWOdnRREGTzmpxH29vRJdxq1XMFwYnkNhWT446lhx1HDPUMQR6DZzcKkLeD/pz4vb43qpG2FYMW0+KwlGWQOlPdsOKAtsAY2KEpZ01QiTcmCkpL2NV0y6M9UZUJ3c7ZelXt4pekxZXyf46x/WBZuG2hANOiQIfZ4S7rhl2l4pYbRAunN9+bkvISYsX5V0vHg85Y0TctwfjyRblYOKp31Nph55fl21lfZOOsk7HpDKNlw9H2hqsQv6P4XMG9diUOPJ+0i8f+cliTn++jRGVJkKP1MM2LWhrbRu882ORLg6w9qBIyJlmFu+w7lPbCwsLWbduHbIsI0kSc+fOJTs722GbmjYh3BaLe18X4Q5QwexYW8/JrCjwVXFujCZxAfrmgLiD2KoVzvHTf6GkrhqtVkvaumrCRsQJ8e7GtaOgMNQYZRNFracpaNLRKZt4KDYTgEZjB2G738KETITKj2xNHDnqWHI18WQEhOMtDXyklBJHPkA4fSvYYRDlWMvq4Urxw0iMjyfxl98Ln7eFID8RRZI1QvSStDB/vHicg+4s/dNtZleJWbyLTju+x18Fs2JsrpJpzgq3hSOV8PinIjywrsW23EcFs0ZSvuswqwo+x2g0UqBSsWReFonnEHFPif5Q8Hw6ZRM7W2ooaNKxXq9jg76CBqPoqB0p+fLLmAwkSSJY5ct/kucxISCMVP8wvAY5qexseKyQ9+VEHzRxMJlELZKtWmGhbisW0STtBpGscvovIjrDzwduyhIuhawk8UiN61cMdGJiIlfetJTPDjVwwns4V2zVUFQL9vddFuG2VAicFg1+zgp3e6fo1tPUBosniWWBvvDfHeL5yChYmAoLJwiXisaf44WFvZoP8LToDwXPot1kpF02EqwSrry/Vu7l4bIfHLYJbTEy4nQHyXVNnIw8RcrwEQDcED7yjPHcgX4LuSRJicAqIAahFytlWf5bf8c9G3050QdMHGQZSupAJdmq5P1rA9z9zpnbjokRlra+HYLN6Tdv3X7OXZzrAlTfDoUVtiSc3bUJyCRY1/upYEa0LQFnWhT49+Y/f7wavtonJlvXHYaWDhEOaBHyhDD49x2i6cPIKMfj3i3KC/dmLkGJ/lBwJc3GTrY0V7Fer6OgSceW5ioei5vEE3GTAchWxzLWL4QcTRzZ6lgCDpZyYN0Gq3uyrNgm5O6KKyxyA/BLWZZ3SpKkAXZIkvSdLMsHXDB2t/TlRHeZOFQ3CV/2Nq2wtLdpoaoJfr0Q/ni12GbKcNE1JitJCHdWknCX9ODXPhvdXYCCYxIptJuc3FXjaHH7esGMGJhrDgecEd1L4bbw8S54+CPRTNiezARhdXcarTHo3DLderzabmrD96YF31BEfyiunPOPFyr28FG9lm3NVRhwnAs81WEr8DNDHcOhtBusr0tGBHBYtdmjoo/6LeSyLOsAnfl5kyRJB4FhwIAJeV9O9D6JQ2MraPxtNUUueVlYpV0JC3R8PWUElP7J+rKkpATtvp19EgmtVkuzSUWxbzJavyTeXhfK0XbR/d2CjxdMt7O4Z0RDQG/+s5ZemV/tE3cNl6aL5Wo/IeJhgSK6xOIy6cG/bX/R6dqtqaWl5YxJ154Y7KgGxZXj2Zw2tFGor6CgScejcZOI8PYHYHdrLZubK/FCYkpgJLnqOKvVHW7epjs8MarGpT5ySZKSgEnAD92sWw4sBxjeh9hme/ryRZ/zPS0dwo+9TSvqemzTwuFK0D5nS0aJCxGTkVOGmyckk8Sja1cbu+d9EYmmDthgdpV8Wz6NoujZyJYZ8TbwlmBmjK3I1KxYESLYK5rbhZvkq33w1X44WSOWX5lpE/Ls0bDxYZFU5H1uJ7r9XY8sy3iZszT7YtUMZjyx4srxLKo6WynQ61hvnpzc22qbvc/RxHFVaBIA90WncXP4KGarYwlR9S601RPi2e1xmZBLkqQGPgR+IcvyGbnhsiyvBFaCCD/s7/768kVb32Mfcnm8Gq5eAfvLRficPb7eYtLSIuQvXS8aA/diMtIZkdB3CuG2hAPuqBFNawR+eHvJjPVvJC8OrhoXzKwYCOpPZvpjn8AL34lIGgsRQcLaXjzRtszfxyFG/Vx0vevpjTtlKFESedybRmOHdWKy3tBObNHbDo4SP0nFjKBocjRxjPO33S1OD4oe5CMdOlwi5JIk+SBE/B1Zlj9yxZguw2AUGYP2lnZiGHx0j1gfGyy6x0gSZCTYQv+ykkTqu6/dVxQiJih740/tTiT0nbCp0jY5ua3aXrjFvOmMaFuRqVkxEmqf4N5/9pYO0f39y32wbKatzVqUWvi3s5Lg0jS4JE187l5coLr7DjzxlhQ897jPV4rbm1hvZ3G3m4ycSr8ZSZII9fZjelA0QV4+5GriyFXHMS0oCv8BzJr0BPqdECSJWqdvAadlGdM5XgAAF8hJREFUWf6FM+8ZlISgf2+BVwtg1ynHRrogOrXrnre5QHaeEuVQncgs7Iur5Ii2lM8P1nHSdwQ79cFsrQJDF+GeEmkT7tkxoOllkqNFWEcSTPzuWuEuWXfYlsr/yCJ4brF4XtcMBpMomNUHFJ+ygqvZ1VLDXyv3sl6vo7jDsdOIxsuHo2k3EGPOmrREk1yIDGRC0GzgNmCvJEm7zcselWX5SxeM3TOyDCdqYLvWbGkXw6OXiEk5gBq9KMoEwoc9dYTNrz15uKNPe7LzPntnXCWtBmFxW1wlW6sT6DTZwgG9JJEtaUl5nx0Lwf3ITrcI683/r4L4kx2OK6eOEBb3tZNty/oQPWOP4lN2DiUS5kxkWeZQWz3r9ToSfIK4PFSE9TUaO1h1+igAoSpfss0Zk7nqOCYGRjhkTV6oIn42XBG1sgEYnG9WloV/d1uxEO/6Fsf1uaNtQr54EqTGi4nJCLXLDqE7V0mrAbZU2VwlP1RBh5273ctscVsyJ7P7KdyUnBaTlF/ug9eXWIW1LsybWF0n+lnDiVoyFxZNEHcfLkbxKZ8b5a5FYJJlDrTVWd0k65t0VBlEbflLgxOtQj49KJq/Jc4iVx1HekC4W2VNegKe5ViSJPhwlwiVAyFSWWZLe2qSiK6wMCLirKVP+2otJSYmcv0tS/jqkHCV3Lo7hC3fOAq3hCjlarG458RCqF+vPqkjBqOos/LlPvhir0j1N3Mo/SMCfzQBlUrFmgUhfH1VBLfdfhsMoGgoPuVzc6HetZhkGRkZldmCvqN4PW/VHnHYJsY7gFxNHJcE274Pfy9vfh6dNqjHej7hWUIO8PsrxQRk1ggRz9yHK3dvraV2o7CyLRb35qpE2o227SVgYoStyFROnGgg7BJaO2D4I8JVZCHIj5Y5Sazzr+ZQ23Havi4Z8AiRrhc+TwvPGmwulLsWo2xid0st6/U68pvKKdRX8G7yfBaFiN/G1MBI1jSWWScmczVxjPELUdwjLsbzhPz6M/z8veZc1lK7EbZWiazJdeWwuRLajI5jZITbXCU5sRDeJb+g1xa/LIs49i/2ihotn/5UXKQCfEWSTlggXJYuokxyRrNj6xZ2WKoc9jLhprcoboLecz7ftZhkmRcri8hvKmeDvoJGk2MwwY6WaquQ3x2Vys+iJijCPcB4npC7gK7WUvzwJDZW2FLeN1VAazfCbSkylRMHET0nhjkvfPo20Uz4i73CbVJu18Fhd4mtycLXPxcZpmf5DANp8V2oboL+cj7ctXSYjGxvqWZnSw33ml0fXpLE6zWHONIuWkWl+GrI1cSRZ56cHGHXBWcwSrsqXKBCHhOfSOoVd/LV8VYOmWJ5bq0/LQbHbdLCYGpwEymGEq4cF0LmyGFOj++U8O0vh8nPOiblxIfCZWlwWTqlQQZOFhb2aM0NpsV3obgJFERlwK3mAlPrm3Rsaq6kxSR+o4tDkxnmKyKenoibjATkauJI8HVdMIFC37gghLzTBDuqbRb3hgpoMcQ4bDM+1FarJCcO2mtsVvXnu1WE98KdYC98Pngxthx46ANRu2XlbWKjcbEiwWhUlHCZXJYuilFJktMW/WBZfOezm+BCxz4me1tzFTmHP6NNdrwdHe8fSq46DoNsm9G/NWL0oB6nwtk5L4XcYBJp7pY47g0V0NzF4h4XaosqyY2DmC51rwr74U5I9Avh7oBMTJ/tIWJHJV5N/xYrfFTw4nXCTaLyEnVcuklCckdXxvngJlCAVpOBLc1V5DeVs75JR7xPIP9JmQ/AeP8wjMik+YeJyUlNPDnqWGsijoL7cl4IucEkSrlaLO7CClG/xJ6xITYfd148xJ7jt9lnd8KHO+G6lUTYZ8ymxgmL+/J0CLArktJDJqniylBwJbtbaviw7iT5eh1bm6vosLOsI739kWWZ0tJStFotu4bPY8KIlCE8WoW+4JFCbjTBrlphcefroFAHjV2Ee3SIsLgtFQLje5nM6FS1xLUH4fO9onbL45eJ5dOTRZ+0uWNtLpPkSNfuW0GhB5qNnWxqriTZV8Mo/xAA1jaV80zFLsAcKhsQQZ7Z4s5Wx1JaWurgygtWopI8Do8RcpMML+0V4l3QjXCPDDZb23HC8k5wwfzLGe6E4loRYfLFXvjero5JSiQ8dqkIF0wIg7qXRNigK/etoNANerNw5zeVk9+kszZReDJuMk/Hi1DdS4ITKY9uJk8Tzxx1LGHejkkOB9zQlafQOzxGyL0kWHkQjoiIJ1I0tiJTuXGQONAT5899CY99anstScL6tljd9vRRxJXaHAq94brj3/FJvdah+40XElMDo4iz82unBoTxYuLMHsdRXHmej8cIOcBj5haRefEwfKCEu6EVvj0AnxXBJRPgpmlieVaS6JizcILwdV+S5tI6JkrSjUJ3NBk72KCvYH2Tjny9js9GLiTKR5RTDvTyRgayAqPIM8dx97WJguLK82w8SsiXjBmggY9WCl/350VQcFTMnoIQdYuQzx0LNS+CX386OvSMO0aqKAw+7SYj3zeVkd8kUt53tNRgtLO4C/Q6rgkTk5HPDZvG34fPtjZd6A+KK8+z8SghHxBufxPe3Gx7rfKCnNHC6r4i07bcW+VUu7O+otzeXpg0Gjs40d7IxEAxId5qMnDZsa+t0q1CYkZQtDVrco461vpeS3KOgsKFI+S1etE4+bMiePAimJYslk9MhNA9oobJ5emi9Gs/63X3BeX29sKg0ewqsUxO7mipIdrbn/KMW60dcJZGjCHWJ9Aq3GrVwNwFKpw/nL9CLstwUCdcJp8ViSYTltbzyZE2IV+eDT/LG1Br21kG+vZWmUwdOr5vLOM3ZVvZ0VKDyc5V4o1Ekp+GOmO7tbP7G0l5Q3SUCp7K+SvkOS/AhmO21z4qmDsarsgQneIt9DNM0FNQJlMHB3uLOy0gnCURYmLH30vFtpZqvJGYHhRDriaOuZp4ZgXFKBa3Qr/xfCGv0cOXe4Xl/erNtm5A42PhUIWILrkiAxamQnDA/2/v3oPjqs8zjn9frWxJllbyXbJkCRlsoLbBXBzi2CHYMuYyUMi0nRZS4pQy42kmtJDJjMOlM8x0JtM0TZsykzapS0mhEGgKTKAdJoRYprgJUAwEjO1ADAjrYsuyjfFaknXZffvHHi+SMTaSzu7q7D6ff6w96z377kh+9PPvetrbFWqrVYOp2ZHIBPfHByevqJ6fCfLl0+bws4VXs0pdJZIF0Qtyd9i5N91d8l9vpE/OOb4c/rrz4aYV6a+/8/vwgz8e88nwhdpq1WBqOBLJQaZajLKSdFfchve38ugH72SeLw0GJ9fE61lX/dE5rVNLYlxZUxg/SzL5RCvIB4Zg6V/B7v0fXZtaCqvPTre6W8796Pr0sW/0U8itVg2mjk8iOcgvjwYrJ4/uZVtvDz8964rMWZOXVzfQNpjgsqp0V4la3JIPoQS5mV0F3AvEgPvc/dth3PdjyqZAbTw9v/uaoMtk3eKPHbowXoXeatVc4U8n5c5fdr3MlkQXL/f2jJrHHcPYPXAk8/iW2edyy+xzT3YbkZwxH7lL33huYBYD3gbWAR3Ay8CN7r7zk16zfPly37Zt2/jecN+HMCc+pi6TsSjUPnI5ueObTL3Sd4A76i7IXF+y4z/ZeewDYhgXT5vNmnh9sHKylngIC3BExsPMXnH3j513GUaL/BJgt7u/G7zRo8D1wCcG+YTU1WTltsdNpNWqXwKTX39qmF8FXSVbEl38X18PQ8G2rjfMOIvm4JiybzV8hjIr4YzDwxzc00lzVT2N6uOWSSqMIG8A2kc87gA+e+JfMrMNwAaApqamEN52cinkgdIoG3kCzrbeHla99eSo/bhLghb36ng9sREHBH9xenP6e/rQI/qeyqSXs8FOd98EbIJ010qu3jdXCnmgNEoGUkle6t3PlqDFPXdKBT8583IAllTMoATjwopZma6SS6vqmH7Ctq7H6XsqURFGkHcCI3+65wfXikqhD5ROZjv6D/HE4TaeS3Txq6Pdo86cnBErI+VOiRkVJaUcWLaeyk85q0TfU4mKMAY7S0kPdq4lHeAvA19y9x2f9JoJDXZOYsXaR57Lzz3kKbb19lA7pYIzy9LbCN/bvZ3bOz7a+Oy8ipmsDqYDfiE+j1ml45/VVKzfU5mcsjbY6e7DZnYr8Azp6Yf3nyrEC1kxTu/L9tjAsKd4te9Apqvkf4/uozc1zF11F/CthvQWw1fWNPK1gQ8zOwQe3687DMX4PZXoCaWP3N2fBp4O414SLdnsR/7q+1t5+NBuEqnR5/qdXVYzqpV9bvl0vt/0+VDeUySKorWyUyadifYjp9zZ3n8o0+L+4RmXZo4pG/IUidQQC8uqWR2vTw9QVs2jXvtwi4wy4T7y8SjUPvJiNZZ+ZHdn57EPMsH9P4m9HEwOZJ7/8YIWbpy5EID3Bo5QaiU0Ts32gawi0ZDNBUFS5E7Vj+zudA/3Uxe0shOpIZbtfHzUsvfGKZWsCVrca+L1mesLysI7E1WkkCnIJVTuznuDiUyLe0uiiyFP0X3+lzEzqmNTubamicrYFNbE6zknAd6xnwUzFtA4W4OKIuOhIJdQbO8/xN91v8GWRBd7Bo+Oem5OaTmdQ73MD7pIfrrwSiCY8fLj9IyXrbGtWjkpMk4KchmzvUN9bEl0URObyjU16e0W+lLDPHDwbQBmxsoyJ+CsidezpHxGZpn8SFo5KRIOBbmcVs9QP88d3UvrkU62JLp4a+BDANbE6zNBfvG02Xxv/udYHZ/H+RWzKDlJcJ9IKydFwqEgl1P6ZsdLfKf79VHXKktKubRqHleP2A2w1Eq4vfa8Md1bh12IhENBLiSSg2w9uo/WYHDynnkXcd30ZgDOLq+h3GKsrKplTbyelng9n6mcyxQLZz94rZwUmTgFeZFqPdLJ5kQnrSc5BWdzoisT5F+auZCbZi7KnFEpIpOPgrwIDKSSbOvrYWVlbWbQ8esdL/BG/yEgfXzZisq5tASDkyur6jKvrSjRj4jIZKd/pQVoONghcORGU/2eZPfSGzgrWGRz86xz6BjqpSXYk1vHl4lEl4K8gHQN9rJhz1aeT+z92EZTS8tn0D3UlwnysQ5MisjkpSCPIHdn17HDtCY6+TA5yN3zLgJgVmk5m490csyTLCqryXSVrI7PozZYIi8ihUdBHgHHl723JrpoPZIeoOwe7gegqmQKG+suYIqVUFYS46mFV7K4fAYN2iFQpGhEKsjHelpLlE93GfYUpcEUvx8dfItb3n9+1PN1pRW0VDewJl5P0lOZ6YDrqufnvFYRya/IBPlYT6KJ2qn2PUP9bEl0pVvdiS5+b3oz357/WQBWVNYyM1aWWfK+trqBc8pqTrrsXUSKT2SCfKz7ckRhH4/nEl08ebiNzYkutgdTAY97oXd/5uvfKZ9Oz7L1n2rZu4gUn8gE+Vj35Zhs+3j0pYb55dF9XDRtduaYskcPvcM/H9gFQLnFWFVVx9p4PS3VDVw8bXbmtWaGIlxEPsmETggys78FfhcYBN4Bbnb3w6d73XhPCIpSH/lgKslLvfszXSUv9HYz5Cn+vXkNN81aBJCZ590Sr2dF5VzKtfhGRE4hWycEPQvc6e7DZvY3wJ3ANyd4z0801n058rGPh7tz3TvP0Jrooi81nLlupHcIHLnU/cQTcURExmNCQe7uPx/x8EXgDyZWTnS4OzuOfUBroosXj3bz0IIWSswwMz4YHqAvNczi8hmZrpLLquYxo7Qs32WLSAEK8//yfwr8R4j3m1TcnXcHE5l53K2JLvYHc7kBNtYt44KgX/uHZ1zKnNJyLcIRkZw4bZCb2S+AupM8dbe7Pxn8nbuBYeDhU9xnA7ABoKmpaVzF5lp/ajizadTr/Qe5cNcTo56vnzIts3qyacRJ70srZua0ThEpbqcNcne//FTPm9mfANcCa/0UI6fuvgnYBOnBzrGVmRuHho/xXGIvmxOdbD7SydwpFTx/znUAnFcxk4Vl1SyrmEVL0F2iudwiMhlMqGvFzK4CNgKXuXtfOCXl1ut9B3no0G9pTXTxWt8BRv6G6Rk+xmAqydSSGDEr4e0lf6TgFpFJZ6J95N8HyoBng4B70d3/bMJVZclAMCVwVmkZS4Luj1/3H+S73W8AMNVKWFlZS0t1A2tPchKOQlxEJqOJzlpZGFYh2ZD0FK/1HcychLM1sZd+T/LVOYv5p6bPA7Au3sAddRewNl7Pqqo6HaQgIpFTsKm1seNF/uXAbzicHBx1fWn5DJpHDEzWT63krxsuyXV5IiKhiXyQvz+QyLS475l3MYvKawBIAYeTgyyYGmdt0FWyJl6vKYEiUnAiF+Q9Q/3BPO5ONie6eGfgSOa5lZW1mSC/be5Sbp2zhOayeL5KFRHJiUgF+bCnWPDmI/SOWPpeE5vK6qp5tFQ3cHXNR8vxG0d0n4iIFLJIBXmplXBF9XwSySHWVjfQEq/nommzMwcwiIgUo0gFOcDjZ67TNEARkREi15RViIuIjBa5IBcRkdEU5CIiEacgFxGJOAW5iEjEKchFRCJOQS4iEnEKchGRiFOQi4hEnIJcRCTiFOQiIhGnIBcRiTgFuYhIxCnIRUQiLpQgN7NvmJmb2eww7iciIp/ehIPczBqBK4A9Ey9HRETGKowW+feAjYCHcC85jfb2drZu3Up7e3u+SxGRSWJCJwSZ2fVAp7u/froDH8xsA7ABoKmpaSJvW7Ta29t58MEHSSaTxGIx1q9fT2Nj4+lfKCIF7bRBbma/AOpO8tTdwF2ku1VOy903AZsAli9frtb7OLS1tZFMJnF3kskkbW1tCnIROX2Qu/vlJ7tuZucBC4DjrfH5wKtmdom77wu1SgGgubmZWCyWaZE3NzfnuyQRmQTG3bXi7tuBuccfm1kbsNzdD4RQl5xEY2Mj69evp62tjebmZrXGRQSYYB+55F5jY6MCXERGCS3I3b05rHuJiMinp5WdIiIRpyAXEYk4BbmISMQpyEVEIk5BLiISceae+0WWZtYDvD/Ol88Gim2uuj5zcdBnLg4T+cxnuPucEy/mJcgnwsy2ufvyfNeRS/rMxUGfuThk4zOra0VEJOIU5CIiERfFIN+U7wLyQJ+5OOgzF4fQP3Pk+shFRGS0KLbIRURkBAW5iEjERSrIzewqM3vLzHab2R35rifbzKzRzLaY2U4z22Fmt+W7plwws5iZvWZm/53vWnLBzKab2WNm9hsz22Vmn8t3TdlmZl8PfqbfNLNHzKw83zWFzczuN7P9ZvbmiGszzexZM/tt8OeMMN4rMkFuZjHgH4GrgcXAjWa2OL9VZd0w8A13XwysAL5WBJ8Z4DZgV76LyKF7gZ+5+7nAMgr8s5tZA/AXpA+iWQrEgBvyW1VW/Btw1QnX7gA2u/siYHPweMIiE+TAJcBud3/X3QeBR4Hr81xTVrn7Xnd/Nfg6QfofeEN+q8ouM5sPXAPcl+9acsHMaoAvAP8K4O6D7n44v1XlRClQYWalwDSgK8/1hM7dnwcOnXD5euCB4OsHgC+G8V5RCvIGoH3E4w4KPNRGMrNm4ELgpfxWknX/AGwEUvkuJEcWAD3Aj4LupPvMrDLfRWWTu3cC3wX2AHuBD9395/mtKmdq3X1v8PU+oDaMm0YpyIuWmVUBjwO3u/uRfNeTLWZ2LbDf3V/Jdy05VApcBPzA3S8Eegnpv9uTVdAvfD3pX2L1QKWZ3ZTfqnLP03O/Q5n/HaUg7wRGHlY5P7hW0MxsCukQf9jdn8h3PVm2CrguOMj7UaDFzB7Kb0lZ1wF0uPvx/2k9RjrYC9nlwHvu3uPuQ8ATwMo815Qr3WY2DyD4c38YN41SkL8MLDKzBWY2lfTgyFN5rimrzMxI953ucve/z3c92ebud7r7/OD81xuAVncv6Jaau+8D2s3snODSWmBnHkvKhT3ACjObFvyMr6XAB3hHeAr4SvD1V4Anw7hpaIcvZ5u7D5vZrcAzpEe573f3HXkuK9tWAV8GtpvZr4Nrd7n703msScL358DDQQPlXeDmPNeTVe7+kpk9BrxKembWaxTgUn0zewRYDcw2sw7gHuDbwE/M7BbSW3n/YSjvpSX6IiLRFqWuFREROQkFuYhIxCnIRUQiTkEuIhJxCnIRkYhTkIuIRJyCXEQk4v4fdgrkCP07pP8AAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Residuals'></a>\n",
        "\n",
        "## **6.5. Residuals:**"
      ],
      "metadata": {
        "id": "fyPUNwN_1tp5"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Regression_Diagnostic'></a>\n",
        "\n",
        "### **6.5.1. Regression Diagnostic:**"
      ],
      "metadata": {
        "id": "i1gNY_o0OrtW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "Stock_Market = {'Year': [2017,2017,2017,2017,2017,2017,2017,2017,2017,2017,2017,2017,2016,2016,2016,2016,2016,2016,2016,2016,2016,2016,2016,2016],\n",
        "                'Month': [12, 11,10,9,8,7,6,5,4,3,2,1,12,11,10,9,8,7,6,5,4,3,2,1],\n",
        "                'Interest_Rate': [2.75,2.5,2.5,2.5,2.5,2.5,2.5,2.25,2.25,2.25,2,2,2,1.75,1.75,1.75,1.75,1.75,1.75,1.75,1.75,1.75,1.75,1.75],\n",
        "                'Unemployment_Rate': [5.3,5.3,5.3,5.3,5.4,5.6,5.5,5.5,5.5,5.6,5.7,5.9,6,5.9,5.8,6.1,6.2,6.1,6.1,6.1,5.9,6.2,6.2,6.1],\n",
        "                'Stock_Index_Price': [1464,1394,1357,1293,1256,1254,1234,1195,1159,1167,1130,1075,1047,965,943,958,971,949,884,866,876,822,704,719]        \n",
        "                }\n",
        "df = pd.DataFrame(Stock_Market,columns=['Year','Month','Interest_Rate','Unemployment_Rate','Stock_Index_Price'])\n",
        "Y = df['Stock_Index_Price']\n",
        "X = df[['Interest_Rate','Unemployment_Rate']]\n",
        "\n",
        "X = sm.add_constant(X)\n",
        "model = sm.OLS(Y, X)\n",
        "results = model.fit()"
      ],
      "metadata": {
        "id": "NKdLvpCrF6ad"
      },
      "execution_count": 36,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "y_pred = results.fittedvalues\n",
        "y_true = df['Stock_Index_Price']\n",
        "residuals = y_true - y_pred\n",
        "\n",
        "plt.xlabel('y_pred')\n",
        "plt.ylabel('Residuals')\n",
        "plt.title('Residuals')\n",
        "plt.axhline(y = 0, color = 'b', linestyle = '-')\n",
        "plt.scatter(y_pred, residuals, color ='purple');"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "VSIF-HwJIZaM",
        "outputId": "f2ec56c3-4900-43dd-d2da-0fd3230c967a"
      },
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEXCAYAAABRWhj0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAayklEQVR4nO3df5RkZ13n8fcnyRBMNE1iQghJJhM0oMFRxDagIma3kfxY4gAaDTtKQNbRXVjl+GMNjj8Ad85BFnQO6ooDyzHAQERZJGPAJMwxZP0BpKMhPQnETEImP0zIQKBBonEI3/2jbpOaTnXP7U5XV1XX+3VOna567u2u75Oe1Kfvc5/73FQVkiS1cdigC5AkjQ5DQ5LUmqEhSWrN0JAktWZoSJJaMzQkSa0ZGlKfJNmc5KpFtl+T5L+swPucneTux/pzpDYMDamR5I4k/5rkX5Lcl+RPknzjcn9eVe2squevZI3SoBka0sEuqKpvBJ4BfDfwmgHXIw0VQ0PqoaruA66kEx4keXaSv0vyxSSfTHL23L5JXpbk9iRfTvKZJJu72v+ma78fTvLpJLNJ/gBI17bXJnl31+sNSSrJEc3rlyf5VPMetyf52YVqT/KrSe5p9r0lydTK/ZfRuDM0pB6SnAKcB+xNcjJwBfA/geOAXwben+SEJEcDbwHOq6pvAr4fuKHHzzse+L/ArwPHA7cBP7CEku4HXgAcA7wc+L0kz+zxPk8DXgV8b1PPOcAdS3gfaVGGhnSwv0jyZeAuOh/UvwX8JPChqvpQVX2tqq4GpoHzm+/5GvAdSb6hqu6tqpt6/NzzgZuq6s+r6gCwHbivbVFVdUVV3VYdHwWuAn6wx64PA0cCZyZZV1V3VNVtbd9HOhRDQzrYC5u/0M8Gvo3OUcFpwIXN0NQXk3wReA5wUlV9BfgJ4OeAe5NckeTbevzcJ9MJIgCqs1LoXT326ynJeUk+luSB5v3Pb2o7SFXtBV4NvBa4P8llSZ7c9n2kQzE0pB6av+b/BHgTnQ/3d1XVE7oeR1fVG5p9r6yqHwZOAj4NvK3Hj7wXOHXuRZJ0vwa+AhzV9fpJXfseCby/qeXEqnoC8CG6zonMq/09VfUcOmFXwO8spe/SYgwNaWHbgR8G/g64IMk5SQ5P8vjm2ohTkpyYZFNzbuMh4F/oDFfNdwXw9CQvbk5u/zxdwUDnPMhzk6xPMsHBs7YeR2fIaT/w1STnAT2n8iZ5WpL/2ATNvwH/ukA90rIYGtICqmo/8E46H/CbgF+j88F9F/ArdP7/OQz4ReCfgQeAHwL+a4+f9TngQuANwOeBM4C/7dp+NfCnwI3A9cBfdm37clPD+4AvAP8ZuHyBso9s3uNzdM6ZPBGnDWsFxZswSZLa8khDktSaoSFJas3QkCS1ZmhIklo7YtAF9Nvxxx9fGzZsGHQZkjQyrr/++s9V1Qm9tq350NiwYQPT09ODLkOSRkaSfQttc3hKktSaoSFJas3QkCS1ZmhIklozNCRJra352VPLMbNzht1bdzN75ywT6yeY2jbFxs0bB12WJA2coTHPzM4Zdm3ZxYEHDwAwu2+WXVt2ARgcksaew1Pz7N66++uBMefAgwfYvXX3gCqSpOFhaMwze+fsktolaZwYGvNMrJ9YUrskjRNDY56pbVOsO2rdQW3rjlrH1LapAVUkScPDE+HzzJ3sdvaUJD2aodHDxs0bDQlJ6sHhKUlSa4aGJKk1Q0OS1JqhIUlqzdCQJLVmaEiSWhtoaCR5R5L7k+zpajsuydVJbm2+Htu0J8lbkuxNcmOSZw6uckkaT4M+0vgT4Nx5bZcAu6vqDGB38xrgPOCM5rEF+KNVqlGS1BhoaFTVtcAD85o3AZc2zy8FXtjV/s7q+BjwhCQnrU6lkiQY/JFGLydW1b3N8/uAE5vnJwN3de13d9P2KEm2JJlOMr1///7+VSpJY2YYQ+PrqqqAWsb37aiqyaqaPOGEE/pQmSSNp2EMjc/ODTs1X+9v2u8BTu3a75SmTZK0SoYxNC4HLm6eXwx8sKv9pc0sqmcDs13DWJKkVTDQVW6TvBc4Gzg+yd3AbwFvAN6X5BXAPuDHm90/BJwP7AUeBF6+6gVL0pgbaGhU1UsW2PSoOx415zde2d+KJEmLGcbhKUnSkDI0JEmtGRqSpNYMDUlSa94jfIXM7Jxh99bdzN45y8T6Caa2TXmfcUlrjqGxAmZ2zrBryy4OPHgAgNl9s+zasgvA4JC0pjg8tQJ2b9399cCYc+DBA+zeuntAFUlSfxgaK2D2ztkltUvSqDI0VsDE+okltUvSqDI0VsDUtinWHbXuoLZ1R61jatujLmyXpJHmifAVMHey29lTktY6jzQkSa15pLECnHIraVx4pLECnHIraVwYGivAKbeSxoXDUytgYv0Es/seHRBOuR0cl3WR+sMjjRXglNvhMneOaXbfLNQj55hmds4MujRp5BkaK2Dj5o1csOMCJk6bgMDEaRNcsOMC/7IdEM8xSf3j8NQK2bh5oyFxCKs1ZOQ5Jql/PNLQqljNISOXdZH6x9DQsszsnGH7hu287rDXsX3D9kN++K/mkJHnmKT+cXhKS7acixlXc8jIZV20EpyB15uhoSVb7Khhof+pVntasueY9Fi4ysPCHJ7Ski3nqMEhI40SZ+AtzNDQki3nRLPTkjVKnIG3MIentGRT26YOOnSHdkcNDhlpVLjKw8KG9kgjyR1JZpLckGS6aTsuydVJbm2+HjvoOseRRw1a6xxOXViqatA19JTkDmCyqj7X1fZG4IGqekOSS4Bjq+pXF/s5k5OTNT093d9iJa054zx7Ksn1VTXZa9uoDU9tAs5unl8KXAMsGhqStBwOp/Y2tMNTQAFXJbk+yZam7cSqurd5fh9wYq9vTLIlyXSS6f37969GrZI0Fob5SOM5VXVPkicCVyf5dPfGqqokPcfWqmoHsAM6w1P9L1WSxsPQHmlU1T3N1/uBDwBnAZ9NchJA8/X+wVUoSeNnKEMjydFJvmnuOfB8YA9wOXBxs9vFwAcHU6EkjadhHZ46EfhAEujU+J6q+qsk1wHvS/IKYB/w4wOsUZKGTr9nfQ1laFTV7cB39Wj/POBEaUnqYTXWzBrK4SktbqnLkksaD6uxZtZQHmloYa6+KWkhq7FmlkcaI8bVNyUtZDXuWmlojBhX35S0kNVYM8vQGDHe/1rSQlZjMVHPaYyY5S5LLmk89HvNLENjxHj/a0mDZGiMIFfflDQontOQJLVmaEiSWjM0JEmtGRqSpNY8ES5JPYzzPcIXY2hI0jyu8bYwh6ckaR7XeFuYoSFJ87jG28IMDUmaxzXeFmZoSNI8q7Fa7KjyRLgkzeMabwszNCSpB9d4683hKUlSa4aGJKk1h6dWiFePShoHSw6NJIcB31hVX+pDPSPJq0cljYtWw1NJ3pPkmCRHA3uAm5P8Sn9LGx1ePSppXLQ9p3Fmc2TxQuDDwOnAT/WtqhHj1aOSxkXb0FiXZB2d0Li8qg4A1b+yRotXj0oaF21D44+BO4CjgWuTnAYM5JxGknOT3JJkb5JLBlHDfF49KmlctAqNqnpLVZ1cVedXxz7gP/S5tkdJcjjwh8B5wJnAS5Kcudp1zLdx80Yu2HEBE6dNQGDitAku2HGBJ8ElrTmLzp5K8ouH+P7fXcFa2jgL2FtVtwMkuQzYBNy80DfccgucffZqlLYRNmyEDZ1Xf/E24G2r8b6StHoONeX2m1alivZOBu7qen038Kz5OyXZAmwBOPLI71ydyiRpDCwaGlX1utUqZCVV1Q5gB8Dk5GRdc81g65GGjRejajHJwttaXdyX5PHAK4CnA4+fa6+qn36MtS3VPcCpXa9PadoktTQsF6MaXKOp7eypdwFPAs4BPkrnw/rL/SpqEdcBZyQ5PcnjgIuAywdQhzQ0ZnbOsH3Ddl532OvYvmE7MztnFt1/GC5GnQuu2X2zUI8E16Fq1+C1DY1vrarfAL5SVZcC/4ke5xL6raq+CrwKuBL4FPC+qrpptesYRUv9YNFoWM6H7zBcjDoMwaXlabv21Nxv94tJvgO4D3hif0paXFV9CPjQIN57McN8qD0swxFaeYt9+C70u51YP9EJmR7tq2UYgkvL0/ZIY0eSY4HfoDMcdDPwxr5VNWKG/VDbv+rWruV8+A7DxaiuojC62l7c9/aq+kJVfbSqnlJVT6yqt/a7uFEx7B/K/lW3di3nw3cYLkYdhuDS8rSdPfWbvdqr6vUrW85oGvYP5WEYjlB/TG2bOmjoEdp9+A76Vqbeg3t0tT2n8ZWu548HXkDnRLQY/g/l5X6waPiN8ofvoINLy9MqNKrqzd2vk7yJzgwmMfwfyqP8waJD88NXq2m5t3s9is61GmI0PpT9YJG0Etqe05jhkftnHA6cAHg+o4sfypLGQdsjjRd0Pf8q8NnmQjtJ0hg51NLoxzVP5y8ZckwSquqB/pQlSRpGhzrSuJ7OsFSA9cAXmudPAO6kc69wSdKYWPTivqo6vaqeAnwEuKCqjq+qb6YzXHXVahQoSRoebZcReXaz5hMAVfVh4Pv7U5IkaVi1PRH+z0l+HXh383oz8M/9KUmSNKzahsZLgN8CPtC8vrZpW5OGecVaSRqktleEPwD8Qp9rGQouIy5JCzvUlNvtVfXqJLt45OK+r6uqH+lbZQOynPsTSNK4ONSRxruar2/qdyHDYthXrJWkQVo0NKrq+ubrR+fampsxnVpVN/a5toEY9hVrJWmQWk25TXJNkmOaK8T/AXhbkt/tb2mD4c1hJGlhba/TmKiqLwEvBt5ZVc8Cnte/sgZnGO5qJknDqu2U2yOSnAT8OLC1j/UMBVes1VrntHItV9vQeD2dmy79bVVdl+QpwK39K0tSvzitXI9F2+s0/gz4s67XtwM/2q+iJPWP08rb8Wist7Ynwp+aZHeSPc3r72yWFZE0YpxWfmhzR2Oz+2ahHjkam9k5M+jSBq7tifC3Aa8BDgA0020v6ldRkvpnoenjTit/xGJHY+OubWgcVVWfmNfmnfukEeS08kPzaGxhbUPjc0m+hWYpkSQ/Btzbt6ok9Y3Tyg/No7GFtZ099UpgB/BtSe4BPkNnefQVl+S1wM8A+5umX5u7l0eS1wCvAB4Gfr6qruxHDdJa57TyxU1tmzpohhl4NDan7eyp24HnJTmaztHJg3TOaezrU12/V1UHrXeV5MzmPZ8OPBn4SJKnVtXDfaphaDmrQ+qvuf+f/P/s0Q61yu0xdI4yTgY+SOe2r68Efgm4EdjZ7wK7bAIuq6qHgM8k2QucBfz9KtYwcM6x1zhbzT+YPBrr7VDnNN4FPA2YoTNk9NfAhcCLqmpTH+t6VZIbk7yjWSAROsF1V9c+dzdtj5JkS5LpJNP79+/vtcvIclaHxpXTYIfDoYannlJVGwGSvJ3Oye/1VfVvj+VNk3wEeFKPTVuBPwJ+m85J998G3gz89FJ+flXtoHMOhsnJyUfdB2SUOatD4+qxXJTokO7KOVRofP03VFUPJ7n7sQZG87NaLXaY5G3AXzYv7wFO7dp8StM2Vly6XeNquX8wOaS7sg41PPVdSb7UPL4MfOfc8yRf6kdBzcKIc14E7GmeXw5clOTIJKcDZwDzrx1Z85xjr3G13GmwDumurEPdhOnw1SqkyxuTPIPO8NQdwM82tdyU5H3AzXQuLHzlOM6cclaHxtVyp8E6pLuy2l6nsWqq6qcW2bYN2LaK5QwlZ3VoHC33DyaHdFfW0IWGJC1kOX8weaHeyjI0JK1pDumuLEND0prnkO7KabtgoSRJhoYkqT1DQ5LUmqEhSWrN0JAktWZoSJJaMzQkSa0ZGpKk1gwNSVJrhoYkqTVDQ5LUmqEhSWrN0JAktWZoSJJaMzQkSa0ZGpKk1gwNSVJrhoYkqTVDQ5LUmqEhSWrN0JAktWZoSJJaMzQkSa0NJDSSXJjkpiRfSzI5b9trkuxNckuSc7raz23a9ia5ZPWrliQN6khjD/Bi4NruxiRnAhcBTwfOBf53ksOTHA78IXAecCbwkmZfSdIqOmIQb1pVnwJIMn/TJuCyqnoI+EySvcBZzba9VXV7832XNfvevDoVS5Jg+M5pnAzc1fX67qZtofaekmxJMp1kev/+/X0pVJLGUd+ONJJ8BHhSj01bq+qD/XpfgKraAewAmJycrH6+lySNk76FRlU9bxnfdg9watfrU5o2FmmXJK2SYRueuhy4KMmRSU4HzgA+AVwHnJHk9CSPo3Oy/PIB1ilJY2kgJ8KTvAj4feAE4IokN1TVOVV1U5L30TnB/VXglVX1cPM9rwKuBA4H3lFVNw2idkkaZ6la20P+k5OTNT09PegyJGlkJLm+qiZ7bRu24SlJ0hAzNCRJrRkakqTWDA1JUmuGhiSpNUNDktSaoSFJas3QkCS1ZmhIklozNCRJrRkakqTWDA1JUmuGhiSpNUNDktSaoSFJas3QkCS1ZmhIklozNCRJrRkakqTWDA1JUmuGhiSpNUNDktSaoSFJas3QkCS1ZmhIklozNCRJrR0x6AKG0czOGXZv3c3snbNMrJ9gatsUGzdvHHRZkjRwAznSSHJhkpuSfC3JZFf7hiT/muSG5vHWrm3fk2Qmyd4kb0mSftQ2s3OGXVt2MbtvFgpm982ya8suZnbO9OPtJGmkDGp4ag/wYuDaHttuq6pnNI+f62r/I+BngDOax7n9KGz31t0cePDAQW0HHjzA7q27+/F2kjRSBhIaVfWpqrql7f5JTgKOqaqPVVUB7wRe2I/aZu+cXVK7JI2TYTwRfnqSf0zy0SQ/2LSdDNzdtc/dTVtPSbYkmU4yvX///iW9+cT6iSW1S9I46VtoJPlIkj09HpsW+bZ7gfVV9d3ALwLvSXLMUt+7qnZU1WRVTZ5wwglL+t6pbVOsO2rdQW3rjlrH1LappZYhSWtO32ZPVdXzlvE9DwEPNc+vT3Ib8FTgHuCUrl1PadpW3NwsKWdPSdKjDdWU2yQnAA9U1cNJnkLnhPftVfVAki8leTbwceClwO/3q46NmzcuOSScpitpHAxqyu2LktwNfB9wRZIrm03PBW5McgPw58DPVdUDzbb/Brwd2AvcBnx4lctekNN0JY2LdCYjrV2Tk5M1PT3d1/fYvmF7JzDmmThtglff8eq+vrckrbQk11fVZK9twzh7auQ4TVfSuDA0VoDTdCWNC0NjBThNV9K4GKrZU6PKabqSxoWhsUKWM01XkkaNw1OSpNYMDUlSa4aGJKk1Q0OS1JqhIUlqbc0vI5JkP7Bv0HUs4njgc4Muog/WYr/s0+hYi/1azT6dVlU97yux5kNj2CWZXmiNl1G2Fvtln0bHWuzXsPTJ4SlJUmuGhiSpNUNj8HYMuoA+WYv9sk+jYy32ayj65DkNSVJrHmlIklozNCRJrRkafZbkF5LsSXJTklc3bccluTrJrc3XY5v2JHlLkr1JbkzyzMFW/4gk70hyf5I9XW1L7keSi5v9b01y8SD60lVLrz5d2PyuvpZkct7+r2n6dEuSc7raz23a9ia5ZDX70MsC/fpfST7d/D4+kOQJXduGvl8L9Om3m/7ckOSqJE9u2kfi319Tz6P61bXtl5JUkuOb18PRr6ry0acH8B3AHuAoOsvQfwT4VuCNwCXNPpcAv9M8Px/4MBDg2cDHB92Hrr48F3gmsKerbUn9AI4Dbm++Hts8P3bI+vTtwNOAa4DJrvYzgU8CRwKnA7cBhzeP24CnAI9r9jlzCH9XzweOaJ7/TtfvaiT6tUCfjul6/vPAW0fp399C/WraTwWupHNh8vHD1C+PNPrr2+n8Yh+sqq8CHwVeDGwCLm32uRR4YfN8E/DO6vgY8IQkJ6120b1U1bXAA/Oal9qPc4Crq+qBqvoCcDVwbv+r761Xn6rqU1V1S4/dNwGXVdVDVfUZYC9wVvPYW1W3V9W/A5c1+w7MAv26qvk3CPAx4JTm+Uj0a4E+fanr5dHA3Kyekfj3Bwv+fwXwe8D/4JE+wZD0y9Dorz3ADyb55iRH0flL4VTgxKq6t9nnPuDE5vnJwF1d33930zasltqPUetft7XUp5+m8xcrjHi/kmxLchewGfjNpnnU+7QJuKeqPjlv01D0y9Doo6r6FJ2hgKuAvwJuAB6et09x8F8TI2mt9GOtS7IV+Cqwc9C1rISq2lpVp9Lpz6sGXc9j1fxx+Ws8EoBDx9Dos6r6P1X1PVX1XOALwD8Bn50bdmq+3t/sfg+dI5E5pzRtw2qp/Ri1/nUb+T4leRnwAmBzE/KwBvrV2An8aPN8lPv0LXTOLX0yyR10avyHJE9iSPplaPRZkic2X9fTOZ/xHuByYG6Gw8XAB5vnlwMvbWZJPBuY7Rr+GUZL7ceVwPOTHNvMtHp+0zYKLgcuSnJkktOBM4BPANcBZyQ5PcnjgIuafYdKknPpjJH/SFU92LVpZPuV5Iyul5uATzfPR/bfX1XNVNUTq2pDVW2gM9T0zKq6j2Hp16BmDYzLA/h/wM10Zp9MNW3fDOwGbqUzo+q4pj3AH9KZtTJD1+ydQT+A9wL3Ageaf8ivWE4/6Iyn720eLx/CPr2oef4Q8Fngyq79tzZ9ugU4r6v9fDpHkLcBW4f0d7WXzrj3Dc3jraPUrwX69H465w1vBHYBJ4/Sv7+F+jVv+x08MntqKPrlMiKSpNYcnpIktWZoSJJaMzQkSa0ZGpKk1gwNSVJrhoYkqTVDQxoxSV6W5A8GXYfGk6EhDYkkhw+6BulQDA1pmZK8Ps2NtZrX25L8Qo/9zk5ybZIrmpsavTXJYc22f0ny5iSfBL4vyU8m+URzY6E/nguSJC9P8k9JPgH8wGr1UZrP0JCW7x3ASwGaELgIePcC+54F/Hc6Nz36FjrrkEHnPhAfr6rvAj4P/ATwA1X1DDorIm9uFoN8HZ2weE7zM6SBOGLQBUijqqruSPL5JN9N514i/1hVn19g909U1e0ASd5L58P/z+kEw/ubfaaA7wGuSwLwDXRWDn4WcE1V7W++/0+Bp/anV9LiDA3psXk78DLgSXSOPBYyf5G3udf/VlVz91gJcGlVvaZ7xyQvRBoSDk9Jj80H6Nxa83tZfDnqs5plxg+jMwT1Nz322Q38WNdy+sclOQ34OPBDzR0g1wEXrmgPpCXwSEN6DKrq35P8NfDFriOGXq4D/gD4VuCv6YTN/J91c5JfB65qwuUA8Mqq+liS1wJ/D3yRztLm0kC4NLr0GDQf7v8AXFhVty6wz9nAL1fVC1azNqkfHJ6SlinJmXRuerN7ocCQ1hqPNKQVkmQj8K55zQ9V1bMGUY/UD4aGJKk1h6ckSa0ZGpKk1gwNSVJrhoYkqbX/D0yyT/rCR0OmAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "influence = results.get_influence()\n",
        "cooks = influence.cooks_distance\n",
        "cooks"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dz9Mar62SgdN",
        "outputId": "1e10246a-deb6-4773-eebf-3f7229d65a17"
      },
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([4.80372735e-02, 4.01480761e-02, 5.11038225e-03, 2.29357894e-02,\n",
              "        2.75007545e-02, 1.30455682e-03, 3.27759686e-02, 1.94513787e-04,\n",
              "        1.27947334e-02, 2.61500396e-04, 3.52124723e-02, 1.52340256e-02,\n",
              "        3.39697129e-02, 1.94485559e-02, 2.78612047e-03, 4.36759843e-02,\n",
              "        1.71660802e-01, 3.44875283e-02, 3.08921004e-04, 8.40042743e-04,\n",
              "        3.57944955e-02, 1.10842254e-02, 2.67059508e-01, 1.67461457e-01]),\n",
              " array([0.98565839, 0.98895399, 0.99948008, 0.99514604, 0.99365666,\n",
              "        0.99993268, 0.99179083, 0.99999612, 0.99795648, 0.99999395,\n",
              "        0.99088129, 0.99735169, 0.99134881, 0.99619632, 0.99979021,\n",
              "        0.98751133, 0.91433766, 0.99115491, 0.99999223, 0.9999652 ,\n",
              "        0.99065981, 0.99834937, 0.84837289, 0.91712206]))"
            ]
          },
          "metadata": {},
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Cook’s distance for observation #1: 4.80372735e-02 (p-value: 0.98565839)"
      ],
      "metadata": {
        "id": "TjaHCvdqS37-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "sm.graphics.influence_plot(results, criterion='cooks');"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "rj8SXOEeS3OQ",
        "outputId": "889585e0-8c2b-4313-a0dd-bed997b69844"
      },
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEdCAYAAADjFntmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3gc5bX48e+Z7bvqxVW25YoxNjbYpgcDgVQINUBIICQhJDeV/LghN4GEQAglCZDcm+TehBRCGhBKgFAChE5obmDcwL3bsrpW0paZ8/tjVka2VUbSSlpJ7+d59Fg7OztzBNKcmbecV1QVwzAMwwCwBjsAwzAMI3eYpGAYhmHsY5KCYRiGsY9JCoZhGMY+JikYhmEY+5ikYBiGYexjkoKRE0TkUhFREZnWy89/VkTeFZGkiNRltm0SkTuzGmgOEpHKzH+7tq+kiLwjIreLSHG7/e4UkU29OP5JIvJ9ETHXixHA/E82hjwRGQf8Gvg3cApw6uBGNGhuAo4FTgPuBL4APCgi0sfjngRci7lejAj+wQ7AMLJgOuAD/qCqLw12MINog6q+mvn+eREJAN8HjgCWDlpUxpBiMr+Rs0TkORF5SUROFZGlItIsIm+LyNnt9rkTeC7z8l+Z5pM7Ozne90XkoCn8HTWriEhURG4RkY2Z5piNInJ1+yaUTLOKisjHROTnIrI38/UnESk64Hh+EfmWiKwSkVYRqRKRJ0RkZrt9ykXk/0Rku4gkRGSNiFze8/9y+7yR+bfTJjkRGSsid2XiTojIWyLyqXbvfx/3KQEg1dZE1YeYjBxnnhSMXDcV+Blu08he4ErgbyIyU1XXAT8AlgD/DXwZ9464qi8nFBE/8E9gVub4K4BjgO8CJZkY2vsZ8A/gIuAQ4EeADXy63T53A2cBPwWeBsLAicBYYI2IFAAvARHcu/uNwAeB/xWRkKr+Ty9+lMmZf+s6+TljwPNAMfAdYCvwKeCPIhJV1V8DvwEqgM8BJ2R+LmMYM0nByHVlwImq+i6AiCwFdgLnAzeq6noRWZ3Zd1W75pO++ATuBXCRqr6Q2favTNP8tSJyi6ruabf/C6r61cz3T4rIIcBlInKpqqqInAKcC3xdVf+73ef+3u77rwOTgDltPyvwdOaJ41oR+V9VTXcTt5VJaEHgeOAa3P9WL3ay/2dwm95OVtXnMtseF5HRwA0i8ltV3SYi2zLvveYhBmOIM81HRq57t91FkszFeA8wsR/P+SFgM/DvTLOPP3OxfRII4D41tPfoAa9XACFgdOb1BwAF7ujmnK8BGw845z+BUtynlu78CkgB8Uys64APqWpLJ/ufCGxvlxDa/Ako93hOY5gxTwpGrqvpYFsCt/mlv4zCvWtPdfJ+6QGvD4wxkfm3LcZSoKaLi3PbOaf14JwduQF4KHP+Lapa383+JbhPEgfa1e59Y4QxScEYSVoBRCSoqsl22w+84Fbjtumf38lxNvXwvHuBEhGJdJEYqnGfgL7eyftrPZxns6ou7kFcNbh9IAca0+59Y4QxzUfGSLI58+/stg2ZNvvjDtjvCWAC0KSqizv42tvD8z4JCHBZF/s8AczEvcPv6JyNPTynF88DFSJy/AHbL8JNUKsyr9uefCL9EIORY8yTgjGSPA7UA3eIyLW47f5XAU0H7Pdn3E7Yf4nIrcCbuJ23U4GPAWeparPXk6rqsyJyP3CbiEwAnsHtmzgReDTTpn87cAHwoojcjvtkEMNNFO9T1TN79yN36U7cJ5MHRORqYBvwSdzJb19Q1baRRm3J4UoReRywe/hEYgwhJikYI4aq1onI6bgX4HtxL4LX486APqndfikR+SDwX8DluEM748B63E7lJD13IfAt3GGqV+Ampzdwh3yiqvUichzwvcx+43GHkq4F7u/F+bqlqnERWYQ7hPZmID9zvotV9U/tdv0H8EvgS5n4JPNlDENiluM0DMMw2pg+BcMwDGMfkxQMwzCMfUxSMAzDMPYxScEwDMPYZ8iPPiorK9PKysrBDsPIkqrGBLXNSfzW4N2vKIqjMLk0ht9nBtkYw9OSJUv2qmr5gduHfFKorKxk8WIzZHo4qI0nufCOVykI+/FZg3sxrm5KctYR4/nyyb1aCM4wcp6IbO5ou2k+MnLGP1fuQlUHPSEAFET8PL5iJy1JUynaGFlMUjByQtp2uH/pNqJB32CHAkDAZ5GylRfe2dP9zoYxjORUUhCRCSLybGZ1qpUi0llxMGOYWbG9nsbWNOFAbiQFgKBfuH/p9sEOwzAGVK71KaSBK1V1qYjkA0tE5ClVXdXdB42hbc3OBmwnt2bXR4M+NlXHSaRtQv7cSVaG0Z9y6klBVXeq6tLM943AatwaMMYwt3xbPUH/4PcltCci+CyLzdWea98ZxpCXU0mhPRGpBI7AXY3qwPcuF5HFIrK4qqpPy/EaOUBVeWdXY041HbWxHYcNVQcWUTWM4Ssnk4KI5OFWhrxCVRsOfF9Vf62qC1R1QXn5QcNsjSEmnrRpSqTx58Coo45sqIoPdgiGMWByLimISAA3IfxZVR8Y7HiM/pdMO1iWIJJ7SUEEWlJmWKoxcuRUUhD3qvBbYLWq3jbY8RgDw3Y0Z4vzC+RcB7hh9KecSgrA8cDFwCkisjzz9ZHBDsroX0GfRa6u66EKIX+u/ZkYRv/JqSGpqvoSZkWnESca8gGCo4qVY01IjkJZXmiwwzCMAWNugYxBF/BZTCiJkEg5gx3KQfw+Yfro/MEOwzAGjEkKRk6YPa4wJzt0HQcml8UGOwzDGDAmKRg5Yda4gpxrN0zZDuGARVlecLBDMYwBY5KCkROOmFgMgJNDHc6NrWlOnFGek0NlDaO/mKRg5ITy/BBHTSmlviU12KEA7BsNdeY8U2XFGFlMUjByxjlHjAeVnBie2pSwmVKex7RReYMdimEMKJMUjJwxb0IRY4vCNCXSgxqHqpJMO1x01MRBjcMwBoNJCkbOEBG+9aGZpGwd1FnEdS1pjphYxPumlw1aDIYxWExSMHLKoWMLOO/ICuqbB6dvIZGyCVjCf37gENPBbIxIJikYOeeS4yoZWxQZ8MRgO0pTwuarp0xnVEF4QM9tGLnCU1IQkRkiclS71xERuUlEHhGRr/RfeMZIFA74uPncOcRCfhpaBqZ/wXaU2uYUH59fwQcOGz0g5zSMXOT1SeHnwHntXv8QuBIYB9wuIl/OdmDGyDa2MMKt588lGvT1+xNDOpMQzpw3js+fOMU0GxkjmtekMBd4GUBELOAS4FuqOh+4Abi8f8IzRrJJpTH+56IjGFUQprop2S+dz42tKRpaUlxy7CS+cvI0kxCMEc9rUigEqjPfHwEUA/dlXj8HTMluWIbhGlsY4VcXz+e8+RXUt6RoaE1lZR6D7SjVTUmKo0F+esERXHJspUkIhoH30tm7gWnAS8AHgPWqujXzXh4wuAPLjWEtHPDxhUVTOXFGObc8sYadda0gSmEk0ONS260pm3jCxrKE8+ZX8OnjKnNybWjDGCxek8LDwE0iMhu4FPhVu/fmABuyHJdhHOTQsQX8/tKFvLmtngeWbuPVDdX73osEfIQC1n5JQlVJO0pLyiaZVvyWkB/28/kTJ3HaoaMpjplCd4ZxIK9J4b+AMPBB3ATxw3bvfQx4MstxGUaHRIR5E4qYN6GIqsYEy7bUsmpHA2/vqGdrTQsKWJm8YKuSF/Rz+Pgi5lYUcsjYAuZWFOGzTDORYXRGcqHOTF8sWLBAFy9ePNhhZF0ibbNks3vBW7WjgeakTdBvcciYfGaNK2BhZQl5oZxaOG/QpWyH5qRNMu3gs4Sg3yIW9Jm+AsPogIgsUdUFB243V5Uc05qyueeNrdy/dBuJtI3juGsEW5ZbKG71zgYeXr4dn2XxodljuPT4SgrCgcEOOycEfBaFETMf0zD6otOkICLP9OA4qqrvz0I8I9q6PY1c98gqdtW3kh/2E40e3OYdyywXnLYdHnlzB8+/U8V3PnIo8ycVD3C0hmEMR13dVlmAePwyt2d9tHJHPV+/eznVTQlK84IE/V3/J/X7LErzgqRsm28/8BYvvls1QJEahjGcdfqkoKonDWAcI1pVY4LvPLgCSyDWw6agaNCPT2x++Ohqfn5RmGmjzCLzhmH0nrnDH2Sqyq1PrqU1aRPrZcdxKOBDBG58bA3JtJPlCA3DGEl6dBUSkWJgOu7w1P2o6gvZCmokWbmjgaVb6iiO9q2zuCDsZ1ttMy+tq+KUmaagm2EYveMpKYhIGPgdcD5uH0JHzLTQXnhw2Xa3Y6aDYZNp26GuOUUibZN2wGdB0GdRFD24z0FECPos7nljq0kKhmH0mtcnhe8CJwGfBv4IfBloxZ3dPBb4ej/ENuw5jvLK+moKIu/9b1BVmpM2e5uS1DUn920XoG1Gyc76BPlhP+X5QfJC/n0JJRbysXFvnPrmFIV9fPIwDGNk8tqncC5wPXB35vVrqvp7VV0EvAl8qD+CG+52NrTiqO6bYauq7KxvZd2eJuqak/gtIeCzCPgs/Jl/Az4LvwVNrSk2VMXZUtOMk5mAKCL4LGHD3qbB/LEMwxjCvD4pTARWqqotIikg1u693wG/xzwt9Fh1U2JfSQZVZWttCzXxJAFLupyFKyL4fe5ktrrmFCk7zpTyGJYIjgN7m5Idfi6eSLN2dyNNrWkcVWIhP5PLYpTlhfrjxzMMYwjymhSqcauhAmzFXV/hxczrMiCS5bhGBPcG373472lIUOshIbQnIu5TQyLNttoWJpZEAT2otPSmvXEefnMH/1y5C0eVtrdFwHHg2KklnHNkBXPGF5qSEIYxwnlNCq/irqPwOHA/8AMRycctmX0lbklto4eKo0GcTCXP3Y0J/D1ICG1EhIAFtfEko/JDWJZQlJkJ3ZK0ufmJ1byyvhoUCiJ+/L79WwwdR3l1fQ2vrK9mYmmMG86azWizPrFhjFhe+xRuAdZkvr8BeAa3j+EW3LLZ/5H90Ia/cUXuxbemKYGq9vouve1ze5sS2I4ypTxGY2uKb9yznH+vq6Y4GqAkL3hQQgDcJBILUBQNsLW2mS/9eSmbq+O9/6EMwxjSPCUFVV2sqg9kvm9U1XNxm5OKVPU4Vd3Sn0EOV36fxbyKQnY1JHq8WMyBfJZQ3ZSkLC9EXsjH9x56mw17myiJBTwlGxGhKBKgJWnzzfveYm9Tok/xGIYxNPW6SqqqJgBz5eij46eX8cCy7d3WOuqOJUJSHY6qLOHJlXtYsb2BUo8Job2CiJ/qpiS/en49V390Vp9i2lrTTFVTgtaUTTjgozwvxISSaJ+O2ZHaeJKX1+9l+ZY61u1pImk7RIM+DhmTz5ETizluahmRoJlGYxheeJ28dkl3+6jqXX0PZ+SZUBIj5Pdhq9KXtV9sR/GJMH10Hn99fSvRPqwjUBQN8OK7e6mNJ3u8OlkibfPv9dXc+8ZWNlQ14bMERRA007SVx/kLJ3Dc1FJC/r5dqOtbUtzxwgaeWr0bx3GH9oYDFiJCPJFma00zT63aTcBnce78Cj559MQ+n9MwhjuvTwp3drK9/TCXrCQFEfkdcDqwR1VnZ+OY7aVth8Wba9lSHaclZVMUDbKgsoTxRYMzgMpRZXRBmB31LTiqvWpGctT9KssPsbWmharGBEXR3i+V4bMER5WnVu/m/AUTPH/u9Y013PjYalqSNn6fUBTd/0lFVdlS08xNj60hGvTxnY8cylGTS3oV45tb67jukZU0JdIURgIdrqbW9nSQsh3++toWnluzhxvOmsPE0uw/rRjGcOH1yjG5g22luBfvi4BPZS0iNwH9nCwlmTbJtMP9S7dy/5LtNCXSpGwH93olWAKHVxRx6XGVzB5fmM3TdisW9BMMWEwqjbJpbzNquXf8XjnqPiWMKwoT8Fm8ua2207IZPREN+nhg6TbPSeHJlbv4yZNriQR8FMc6nk0t4q6RDO7IqGv+voIrT5vBB2eP7VFsSzbXcvWDKwj4hBIPTzKBTJnxqsYEX7t7GT+7cB6TSmPdfs4wRiKvHc2bO/haqqrXA38F/l+2AsoU1qvJ1vHAvQB958EV/PalTaQdh6JogPL8EGV5IcryghRHA6zcUc+V977Jv1bvzuapuzW2KIw6Sl7Iz5TyGCCkbKW7RVIVSDuKo8qEkghlee7w1njCJhjoe/HbkN9ib1PyoDkPHXljUw0/eXIteSG/57b7SNBHXsjPrU+9wxubvP/vrmpM8P2HVxLyWz2uKlsYDZBMO1z94Nu0puwefdYwRopslM5+EfhoFo7jmYhcLiKLRWRxVVXXi8uoKjc+tpo3t9ZRGgsQDhx80RIRCiMBoiEfP3piLcu21PZX6AcpCAc4aeYo6ltS5If9HDomn6JogLSjpGzNXPjdiW6O8t52200kM8fkUxIL0pRIc8iY/P0Wru8LEUGA1lTXpbiTaYebHltNJODrcWd50G8RCfi46bHVpGxvJb9ve2otybTd647jgoifPY2t3PXKpl593jCGu2wkhWOAAS22o6q/VtUFqrqgvLy8y31X7WzgtY3VnoZmhvwWfp/wv8+t93SHnC1nzRvvdsWq4vcJk0qjzBpbwLiiMPlhP5a4TwYiEAv5GVMYZubYfKaUx/ZdiNO2cv6CCUQCPrIRumZmPoe6udC/trGaeCLd64t0JOgjnrB5bUP3TwsbqppYsrmWoiyUGX9wmduMaBjG/ryOPvpeB5uDwGzcp4SfZzOobPr7su2A9zb2vJCPzdVx1u1pYvrogVnFbMboPKaNirG+Kr7vghfwCeX5Icrzu69L1NiapjAa5OjJpTy1ajdbapr7PAQzaSuF0QBWN48d9y7eit/XxzkWPrhn8VZOmF7W5X6PrdgF9L2/xO+zsJ00L75TxYfn9Kw/wzCGO6+Nst/vYFsC2Az8ELgpWwFlU8p2eOHdvRRGvN9ZigiOwrNr9wxYUhARrj3jML7056U0tKT3K6Xdneake7d749mzCfotPjJnLK9tqO5zTPHWNBcdM7HLfXbWt7B2VxPFfRjpBJAf8rN2VyM761sYW9j5KLAlm2uyNt/AEmHJ5lqTFIweU1W21rSwfm8TW6rjtKYcYiE/laUxpo6Kdfk7PBR4+mtW1QFbtlNE/oq7dkOZiGwDrlXV3/bmWPFEGtqVpvbK7xP2NA7svLxRBWF+8vG5XHXfW9TEkxRFur5LV1XqW9IEfBY3nTN739rMCyYVkx8J7Jsw1huOKgh8uJtRQdVNyV7VazqQW/LbPV5nf1Bp22FbbQslnYxs6qlwwGLNrsasHMsYGVK2w9OrdnPPG1vZWd+KCKQdJ9P06143HIVpo2JcuHAi75teNiQLTPbtFq8fqOonsnUsd+JUr2Ig2EGdoP5WWRbjF588kp8/+y6vrq9BVYmFfQR91r5frpTt0NhqA8rs8YV87ZTpVJa9N7zS77M478gKfvPSxl4nhfrmNAsrS7otjOeO4MlO34vsO14n50q7Q4iz9Ufms2TfU5ZhdGfdniZuemw1W2qaCQcsiqL+Dn8XVZXN1XF+8I9VzBlfyDc/dMiQe3LoNCmISNdtBwfIxfpHsaCfWMhPIu1022HanqPsd6EdSOX5Ia772GyqGhP8c+VOHly6nep4Cp+4E8pCAYtzjhzP6YePpaK440lYZ8wdx1OrdrO9rqXHnbLxRJpQwOKLi6Z2u2844MtSSnBTS1dNQwGfezfWl8KB+51P3fkLhtGdV9bv5fp/rEKQbgesuHNxAqgqq3Y28MU/LeFH587lkDED0xSdDV09KWyiZ7eBOVc/wLKEs+aN58+vbiaU561cg+24RRneP3NUP0fXtfL8EJ86ppJPHVNJIm3TmnQIBSxCfqvbi2Is5Oemc+fw9b8upyaeOGhmcWcaW1OAcMs5czzVKBqVH8Jx+n6hVnVLYJR3sdhPyO+jJBYkaTuE/H1PCq1pmxkD1GdkDF0rttXz/YdXEQlaPXryFnFn9De1prnqvjf5+UVH9kvdr/7Q1a3SZ9t9/QewHVgNXAd8Cbd09prM9i/2b5i99+HZYxBLPI+Dr2tOcfy0UkpzaDWykN9HYdSdY+H14jsqP8wvPnkkU8vzqGtOU9ec2rdsZ3uqSlNrmtp4kmjQz88unOd5VveogjCHjS+gobVvzTANrWnmjC9kVDfNVXPGF2atySeZVuZNKMrKsYzhKZ5Ic8Ojqwj6pddNsXlhP4mUw42PrSbt8Ro02Dp9UlDVO9u+F5GfAkuBs7XdAH4RuR74O9C3cpr9aFRBmP9YNJWfP7uOwoi/yyaD+uYUxbEgXzp52gBG2H9KYkF+ftGRLNtay/1LtrN4c22mMB0obge84DaVXbBwAsdNLevxL//H50/g+4+s7FOcjqOcN7/7chofnjOGF9/d26dzAZkaU7DokK7nuBgj2x9f3UxNPEmpx1aGzhRG/azb08QTb+/i9LnjshRd//Ha0fwJ4FI9YEaXqqqI/B9uvaJvZDm2rDlz3jhsR/n1CxtQTe+3Apmq0py0SaQdRuWHueW8w4fVmsWWJcyfVML8SSXsqm/lrW11xJNpbFvJC/uZNiqPqeV5vW7+WVhZTEHYTzyR7nHZCXDvxgoiARZWFne77xETihlTGGZvUyv54d6PQqpvSXP05NJOOwB31beycW+c1rRNNOBj6qi8YfU7YXSvJWnzyJs7ejQ8vDMiQjTo46+vb+Gjh4/N+RFJXn/iPKCz26pRQE5XFxMRzp1fwYLKYh5a7q5VrEl3pIvtuCugXbhwAifOGDWs6+6PKQwzpnBMVo/p91l89/RZfPNvb+GzejYMtjVlk7KVH549q8NV4Q5kWcK3PjSTK+5ZRtp2PH2mo3MGfMJX37//06CqsnhzLfcv2cayLbVYltuxLeIOPDh2ilnHeiR5dWM1adshEM7OAM1wwKI6nmTVzgYOGzewRTd7yutP/Bxwo4isVtU32jaKyFG4k9eey35o2TepNMbX3j+dz79vCnsaW0mmHfLCfsYUhM0feh8cXlHE9844lB/8YzW2o56eGOKJNClbueajh3J4hfe2/VnjCvj0cZX8/uVNFHWw5nRXEimb5oTN986Yxaj89/ovkmmHW59cy7Nr9mBZ4s7kbvf70LaO9b/XV3PmvPF8cdHUHs99MYaWFdvqsjayDjKTYh1l3e6mnE8KXv+ivoI7g/lVEdkkIq+JyCbgFaA18/6QEQn6mFQaY/rofMYWRkxCyILjp5Xz4/PmEgv5qY2nqO+gY9tRpa45RW08RTTo58fnzeWE6T1v17/oqIl85vhKGlrTNHro5FZV6uIpWtPKdz566H7ndBzl5sfX8MyaPfvWqj5wTYu2dawLIwEeXLaNXz63bkBrYxkDb9WORsJZqDbcnmUJK3bUZ/WY/cHrjOaNIjITuBS3AN5Y4G3cpPAHVU31W4TGkDGnopA/X3YMS7fUct/ibSzbWovfeu8PK+04HDmxmPPmV3DExOJe322LCJ88ehJzK4q4+fHVVDW6a1xHQ759M6xVlaStNCdsEGXaqHz+68MzDxoW+K81u3nh3T2UxoLd3hz4LKE4GuSRN3dw7JRSFlT2boEgI/c1JVJZfxr0WUJjS+5PmPTcYJa58N+R+TKMDvksYWFlCQsrS6huSlAdT+4ruVEaC2Z1qO/s8YXc+ZmjWLK5ln+8tZO3t9dT25ra96heEgty1MwSzjh8HIeNKzjooq+q3P3GViI9GOrrswSfJfxt8TaTFIYxn2Vlpdpwe4r2uXjkQMi5MhfG8FGaF+r3+R5+n8XRU0o5ekopqkpjIk0q7RAO+Lrt21i7u5FtNc09nvWdH/azbGtttwX8jKFrYkmUxZtbez0/oSPJtDK1PC9rx+svXZW52IA7L+FNEdlI17ObVVW7r4tgGP1IRCjowVDV1TsbSDs9n41tiWCJsGpHg0kKw9TcikJeyUK14fb8lgyJchdd3Uo9DzS0+970rBnDSn1Lit6OMXAcd36LMTwtnFzCr1/ckLVaW+7Nh9vvluu6mtH8mXbfXzog0RjGAIoF/b1uNxaRHhVZNIaWSaUxZo4p4N09jT1aj6UzDS0pTps1pkdPsoOlT7/VIlKarUAMY6BNKIn2agKcOxxVmVg6NAqcGb3zxUVTsR23WGNfJNIOAZ/Fp7pZtCpXePqLEJHPi8g3272ek1kAZ4+ILBaR7E6TNYwBsGBSMXkhf5frOHSkOWkzvjjCIabK6rA2a1wB5x5ZQW1zqtfzUmxHaWxN88VFU4ZM/5PX26SvAi3tXt8G1AFXAIW4FVMNY0jx+yzOPXI8TQnvSUFVaU05XLBwopn0OAJ87oTJHDe1lOp4qsdPDGnboa45xVnzxnH64blfCK+N16QwCbdMNiJSCCwCrlLV/wGuBT7YP+EZRv86c954JpVEqY13P/9SVamNpzhsXAEnmQqrI0Jbba+PHT6O+uYUTYl0t08NqkpDS5rGVptLj6/kyydPG1I3EF7nKVhAWzHwE3BHIj2Xeb0VtyieYQyqdXuaeHldFQ2taUblhzhxRnm3j+yxkJ9bzjucq+57iy3VzRRE/AQ76EBOpGwaW20OHZfP9WfNJuQfvoUTjf0FfBZfO3U675tRxk+ffpdd9a2oKuGgj7Df2lc0sTVlu8vGAjNG53PFqTOYNir35yUcSLy0lYnIUuBZVb1SRP4IVKjqyZn3LgBuU9Xx/RtqxxYsWKCLFy8ejFMbOaKqMcENj65i9c6GzFoJ71U4fd+0Mq784CFEg13f/zQl0tz7xlYeWr6dlpSDqmJZ7tBTSyzywn7OPXI85xxZkdUJTcbQoqqs3NHAM2v2sGJ7PVtrmknZDiG/j8llMeZWFHLKoaOHRDIQkSWquuCg7R6TwkXAH4FaoBj4uKo+kHnv/4BJqvrh7IbsjUkKI1t9c4ov/WUJexuTBy2m7rRr7vnReXM7fAI4UCJt88r6at7Z1UhTZq2HWeMKOKqypFcjlYzhL1tzGQZaZ0nBa0G8v4jIFuBo4A1VfaHd27uBh7MTpmH0zL1LtrC7IUFZB6tjWeIutL5yRwMvvLOHU2d1P0gu5Pdx0iGjOOkQ0yJqeDMUE0JXelIQ7yXgpQ62X5vViAzDo0Ta5uHlOynoYiEUESEUsLh38TZPScEwRjrPz8MiEhORr4nIfSLyrIhMz2y/MBXFZNQAACAASURBVFNW2zAG1O76BGnb6bZZKBb0saEqPmQWTjeMweTpSUFEJuCONqrAHZo6G2ibuXMycCpwWT/EZxidUrQHtYtM6S7D8MLrk8KtuCuvzQDmA+3/FJ8H3pfluAyjW6MLwliWkOrmCaA5aTOxNGY6ig3DA69/JacB16rqZg6+5doODMpwVGNkCwd8fHTOWOq7WM3qvRnIEwYwMsMYurwmhSDQ2Ml7hUDurzFnDEsXLJxIaSxIXcvB9WnaZiDPGJ1nZiAbhkdek8JbwLmdvPdhYEl2wjEO1JxM8/Sq3dz1yiYefWsn9c1mOez2SmJBfnrhPCYWR6ltTlPdlKSuOUV1U5La5hTzK4u5+dzDzQxkY8h74YUXOPPMM5k0aRIiwg033NDpvqtWrSIWi+H393xxTa+f+DFwX2Y87l8y22aJyJnA54CP9fjMRrf+vW4vNz62mqStOI6DiPDzZ97lC4umctYRpsWuzdjCCL+6eD6rdjbwwjt7aWhJUZ4f4pSZo6gsiw12eIaRFU1NTcyaNYuLLrqIK664otP9mpubOf/88znllFN4/PHHe3weTzOaAUTki8DNuKOO2jqaG4Fvquqve3zmLBmuM5rf3d3IV/66jEjA2q+sQsp2aGhJc92Zh3Hc1LJBjNAwjMFSWVnJZZddxjXXXHPQe5/5zGcIhUIcc8wxXHbZZaTTHbfudzaj2fNwDFX9P9wO5Q8Cn8JtNqoYzIQwnN3zxlZQDqqzE/BZhPwWf/j3pl7XeDcMY3i66667eOONN7j99tt7fYweNTipahx4uv02ETkbuEZV5/c6CuMgr2+sIT/ccTt4LORjw9448aRNXqjnbYaGYQw/q1ev5sorr+TZZ58lEun9gj5dXlFEpAD4EDARWA88rKp25r1zge8Bc4BNvY7A6Fh3k7K0+10MwxgZEokEH//4x7nhhhuYPXt2n47VafORiMwCVgB/BX4E3A88LyKlIvIUcC9QCnwFOKRPUex/3g+JyFoRWSci/5Wt4w41x0wppaGT8fdNiTTTR+cRM08JhmEAO3fuZOXKlXz5y1/G7/fj9/v53Oc+h23b+P1+brzxRs/H6uqqciMQAS4GlgKTgVuA13FXYrseuEVVW3v9kxxARHzAL3Any20D3hCRh1V1VbbOMVRcsHACL75TRUvSJhJ8rxkpmXZIppVLj5vc7TGaEmlsRykI+4ddJUfDMN4zfvx4VqxYsd+2hx56iGuvvZbly5czevRoz8fqKikcj9tX0DYEdY2I7AVew53d/IMexu3FUcA6Vd0AICJ3A2cCIy4pTC3P4/qzZnPDP1ZTG0+Rdhx8luD3WXzj1OkcNbmk088u31rH717ayOqdDYjA2MIwFx8zifcfOtokB8MYopqamli3bh0AyWSSXbt2sXz5cvLy8pg2bdpBzUZtozJ72pzUVVIowW0+au+tzL//6tFZvBuPu7xnm224azjsR0QuBy4HmDhxYo9P0pqyeXt7PUnbYVp5HqMKwr0Mt38trCzhni8cw+sba9jd0EpRNMixU0u77Fx+eV0V1z2yCr/lriUAUBNPcvMTa9le18qnj6scoOgNo+9UlRXb61mxrR6AwyuKmD2+YETe3CxevJiTTz553+tf/OIX/OIXv2DRokU899xzWTtPp/MURMQBjlHV19tt8wEpYIGqLs1aFO8d/zzgQ6p6Web1xcDRqvqVzj7Tk3kKqsp9S7bxh1c2YTsKCrbCsVNL+M8PHEJ+OJCNH2PQpGyH83/1Co4qkQOGsqYdpbElzV2fO4rROZoEDaO93Q2tXPPg22ypbd5X9txvWUwqi3LDmbNz9mauv7UkbV7fVENdc5KCSICFlSW9GoXY25XXLheR09sfB7cg3n+IyM522zVLi+1sB9pXLqvIbMuK+5Zs41cvbKAg7CcYcvvYHVVeWVfNt5tW8LMLj8Bn5dYdSMp2WLq5lqqmBIWZX4DO1gh+c2sdLUmboujByc1vCarK82v3cP7Cnj9dGcZAaknaXHnvcvY2JSmKvNcnpqpsqW7myr+9yR2XLBhR62W3v6lNpRVbFZ8l+ES48KgJfOroSVhZuH51lxQ+28n2zx0YL5CNpPAGMF1EJuMmgwuBi7JwXFpTNne9stlNCO0WZbFEKI4FeHd3E0s213bZVj/QFm+q4cbHVhNP2pkF5IWg3+JLJ03lw3PGHrR/Q2s3dZEE9jYl+ylaw8ieF96tYk9jgpLY/susighF0QC7G1p56d2qEbWa3l9e38LvX9pIYTRAXui9a1jadrjrlc20JG2+sGhqn8/T6ZBUVbV68JWVdK2qadwhrv8EVgP3qurKbBx75Y4G0k7Hq3S5dyHK8+/sycapsmL1zgaufvBt0o5SHA1QmhekOBbA7xNufeodXnin6qDPjClwJ6x0NdN5Umm032I2jGx5etVuAr7O73r9lvD06tz5e+1vdc1J/vjKZoqiAQIHrAvi91kURwM8sHQ7uxv6Phg051YdUdXHVHWGqk5V1R9m67jdLcRiWUJL0s7W6frsD//ehIgSDe6fb0N+i0jA4o4XNxx08T90bD5jC8M0tB48v6E1ZRP0W5w4w5SQNnJfc9LG6qIz2WcJzcmRU7H/+bVV2I52ulCUzxIclKdW7erzuXIuKfSXqeV5OI7idHIXbTvKkROLBziqjrUkbZZurqUg0nHHdzToY09jgq01LfttFxG+e/osQn4fNfEkrSmbRNqhNp6iNeVw9UcOHfKd6cbIcOjYAhKpzm/kEmmHQ8cWDmBEg2tbXUu3S8/6RNhywDWhN0ZMUijPD/G+6eXUxg9ejKUpkSYW9HPSzFGDFN3+krYDQqd3SiJu51LSPvjJZkp5HndcsoBPHDWR/FCAsN/iA4eN5n8/NZ9jTVVVY4g4Y+5YRDp+wk/ZDoJw+uEH96sNVwXhAE439S9tRyns5EayJ0ZUnYQrTpvB3qYEq3c2orgdt44qsaCfG8+ZkzPF5QrCfoqjQVpS9kFDS8HtWELcdQQ6Up4f4jPHT+Yzx3c/69kwctGk0hifP3EKv35hA5YIBRH3b7OhJY2jyhcXTWVCycjpHzthehl/fNWtjNzRHA1V93q2KAvNw7lxFRwgeSE/t50/j2Vb63j+nSpakmnmVhRx8sxROVVHSES4YOEEfvnsOsJ+a79fAlWlviXNGXPH5VTMhpFt582fwNTyPO55YytLt9QCcOTEYi48aiLzJhQNcnQDa3JZjPmTilm8qZaSWOCga0Jtc4qZYwo4bFxBn8/leZGdXDVcF9lJ2w4/fHQ1L63bi98nhAO+TN0jh5lj8/nRuXP3q4lkGMNZ23VqJM5kbtOUSHPNgytYlSlfE/RZpGxFVZlSnsdN58yhKBrs/kAZnU1eM0khhzmO8uqGah5ctp0ddS2UxIKcOW8c75tRbtYcNowRyHGUZVvrePStHexuSFCWF+Qjc8Yyf1JxpyOTOtPjpCAiz/Tg+Kqq7+9RRFkynJOC0TvJtIPfkqzM7jSM4ao3ZS4s3JnKbQ4BxuAuqLMbGA1UAjuBtdkK1DB6Y2tNM4+8uYPH395FS8pGFaaPyuOCoyZw3NRS82RlGB51mhRU9aS270XkLOBnwLGq+lq77UcD92TeM4wBp6r86dXN/OnVzSiQH/YTCwVRVbbWNnPzY2soyw9xy7lzqCgeOaNVDKO3vDZC/QD4bvuEAJB5/X3ghizHZRie/OW1Lfzhlc3kh/2UxIL7SgCICPlhP8WxADVNCb5xz3KqGhODHK1h5D6vSWE6cHCxHdceYFp2wjEM73Y3tHLXK5spivi77GQrjAaoa07x+5c3DmB0hjE0eU0KG4EvdPLeF3D7GQxjQD22YidK5/Vg2iuKBnh2zR7qW7qpJGsYI5zX2U/XAX8WkbeB+3ivo/k8YCbwyf4JzzA698Tbu4h5nKvhs9zZ60s213DKTO/r1RrGSOMpKajq3Zn1ma8Dvg0EcFdgewP4oKr21/KcRg+pKit3NPDQ8u1sqIrjqDKmMMwZc8dxVGVJj8cy57L6lhQF4f1/hVXdhZMskYMKiNmqNLSMnMqahtEbnuskqOrTwNMiYgFlwF5V7boetTGgtlQ3c90jK9la24wghIMWAuxqaGXp5jryw36+/ZFDmT8pN6rB9lXQZ+Goog7UNqfY09CaKaDmZoNwwGJUfojCaBBLwEI6XE/DMIz39OYvJApEADPwO4dsro7z1buXsaOuheJogOJYgEjARzjgozDivk7aNt95YAWvbage7HCzYu7EIrbUtrByRwPba1uw1e1fCPgEv09Iph221LSwcns9NU0JEJgxOn+wwzaMnOY5KYjI6SKyFKgHNgBzMtt/IyJZWTLT6B3HUa75+9uk0g6F0UCn9WGiQT/hoMUPHl1FbXxoL8upqrQmbWrjSSwg4HPLibf95ILbjxDwuc1Im6pbCPotpo3KG8SoDSP3eUoKmclrDwF7gW8B7a86G4FPZz80w6tlW2upakzsKy/clUimsN5Tq3cPQGT950+vbmHpllryw35sD+W7LAvq4qkOlzE9UEvSZmd9Cxv3xtlV30prKndW5DOM/ua1T+Fa4PeqepmI+IEftXvvbeBLWY/M8OyBpdt7tH806ONvi7fy8fkVQ7LqZGNrir+8voXiaIC8kI8Ne5tpTdlYluCzxO1kxn2asB1FESqKI0SDfv7v+fW8b3rZQT93Wwf9A0u38e/11Qgg4nZcA5wycxRnHjGe6aPyhuR/M8PwymtSOBS4KvP9gfdltUBp1iIyeuzd3Y0HreXclXDAXa4znrRzZmGhnvjX6t3EEylq4k5mNJFbPjiRVlD3Yu7PJIhYyM/ogjD5mVFK1U1Jlm+t44h2S6++ta2OW59cy656d8ZzYcS/XzE921GeWr2bp1fvZlJpjG9+8BCmm74JY5jy2qfQgDviqCOVdD7b2RgAduZC2DOC7aXdBbeO+8a9cVbvbGDj3jhNicEb1rmzvoXr/7Ga3Q0JGlpS+C0I+CwiQT95QR+RgJUpdSEoUBwN7EsIAIjy9+XvPVk9s3o3V933FtXxJEVRtyzGgdVVfZZQEgtSFA2wrbaZr9+9nMWbagbmBzaMAeb1NvEp4Nsi8jjQmNmmIhICvgI83h/BGd6UxoLsrG/ZV/enO2lH8VkQC3X+dKGqvLuniYeW7+CZ1btBQBAURRUWzSjnrCPGM3NM/oA1p2zaG+f/3bucptY0Qb8ctIa1iDvqqO2X2lFla20LSVsZWxgGIBrwsX5PHIDXN9Zw8xNriIX8hDwMVRURCiIBWpI23/3729x6/jxmZWGlK8PIJV6TwtXA67glsh/DbUL6L+BwoBA4q1+iMzz52Nxx/M8z64iFvO3f2JLitFljOp3IVt+S4vpHVvL29noA8iMB/Ac0pzy7Zg/Prt3DzDEFXPexwyiOeV/xqTf2NiW46r63aEna+/oNumOJELBgT0MrAZ9QlhfCsoRE2qa+OcX1/1hJJODzlBDaiwR92Kp89+9v85fLjzZluY1hxdNfg6puAo4E/gGcBtjAicCrwNGquqO/AjS6d9LMUfvG5XfHcdwmozPnjevw/dp4kq/9dRkrttdTFA1QHAvulxAg05ySF6Q4GmDtrga+9Jel/V6B9HcvbaS2OUlhNAAc3LHVGRG3b2FHXStp28F2lGjQx9Ord5FMO71e0jQv5Kcpkebf64fHnA/DaOP5FklVt6nq51S1QlWDqjpWVT+jqlv7M0Cje3khP185eRoNrenMjN6OOY5S05zio4eP63C8fiJt850HV7CroZWSWLDbZiERoTgWpDae5L8ecO/i+0N9S4pn1uyhKJMQIkEftuN9GVlLJLO4eZKWpMPh4wu5d/G2Pq9x7fcJd7++hc5WLzSMocjrPIVLROSYTt4rE5FLshuW0VMfnjOWr54yjXjCpiaeJN0uOdiOUhtPUdeS4vTDx/Klk6Z2eMH/97pq1lc1UeRhvkN7RdEAW6qbeW7tnj7/HB351+rdOKr4Mk8so/JD9PQ6bImwpzGJqsO00fnUt6SIBPqWFPJCPjbujbO+Kt6n4xhGLvH6pHAn8IKIfLmD96YCv89aREavnTlvPL++eAFnzB1HS8qhvjlFfUuKhpY0J0wv47bz5/H190/vsC9BVbn7jS0EfVavOo7DAYt73tjaL3fN/3hrB+HAezEXRAJYltuR7JXPEpJpm/L8MLajXT5ReSWZontbakxSMIaPntwSPgT8t4jMAK5Q88yckyaWRvnqKdP5/PumUNucxHaUomiw2/kI66ua2Lg3TnGmiaanokEfO+rdOkSzxxf26hidqW5KEmyXyCyBCcVRNlc3g+W+7o6tCghnzBtHfUuqF0N4OzmuA00JM+PZGD56Muzix8CFwGXAwyIS65+QjGwIB3yMLYxQURz1NEFt1c5GVOn18FIRwXaUNTsbevX5riTTzkFxFUUDjC+OYDtts5Y7prhDcFVhdEGI0QVhAj6rx81PnbEEAl6ykmEMET1qPFbVv4nIRtynhpdE5PT+CcsYaE2t6R41x3TEEqGuH1Y2i4b8bp8C+198y/KCBH3C9rrWfSOv3AcKdz6Fk2khigR9VBRHSNkO0UzVWC9DWr2wxJ01bRjDRY9LZ6vqYuDozMs3gIVZjcgYFAFf3y+SimblOAeqLI3S3MnIpoJIgEPH5jNtVB5F0QB+y3Lv3n0WpXlBDhmTx4zReUQCFrajjC+OcMTEokxdo74lwbYRUHMrivp0HMPIJb26xVHVbSJyPPBn4L/xPmzcyFHF0eC+0T29JQhleeEsRfSec+dX8PbDq7rcJxbyEQtFO32/sTXNoWMLqCh29zm8ooiVO+opjPSuDwXcobKnzBy1b+6EYQwHXp8UrgO2td+gqs2qejbwHeCubAdmDKyFk0vwZfoFesNxFMsSjplSkuXI4KjKEvLCfhJ9KGFtq3L+ggn7Xp83vwLH0V4/LbR97qwjxvc6JsPIRV5nNF/X2axlVb1ZVT+T3bCMgVYYCXDKzFHU97JPoL4lzfFTSynN81hrowf8PouPz6+gsdXu1UW8sTVFUSTIUZPfS1gLJhUzuTyPul6u2VzbnOKICUUcYqqlGsNMp0lBRE4Ukbx233f51ddAROTjIrJSRBwRWdDX4xk9d9YR4xHYb+KbF+7oH+WcIyv6JzDcO/ujJpdQHU/1KDHEE2lA+OHZc/YrGOj3Wdx49hxKokHqmr0nQlWlJp5kQnGU755xmFlbwRh2unpSeA6Y1e77Zzv5anuvr94GzgFeyMKxjF6YPjqfS46tpK4l7bkZyXaU2uYUFyyckPX5Ce35fRbfPX0WCyYVUx1PdVvnyVF3FreIcNM5czos61ESC/KzT8xjfFGEvY3JLldYU1WakzbV8RSHjMnntgvmDcm1KAyjO139Vp8MtPXunUI/dyar6mro/Th5IzsuOnoirWmbu1/fSjhgEQ36Ovx/0naRbEnanHNkBZ89fnK/xxYJ+rjhrNn85fUt3L9kG7XxFH6fu/Z0Wx950naIJ2wEOHJiMf9x0jQmlnbeAT0qP8x/f+IIHluxk78t2UZtcwpBCfotLBEcVZIpdx5EeX6Iy06YwgdnjzaVUY1hS3JtYrKIPAf8Z2boa2f7XA5cDjBx4sT5mzdvHqDoRo7n1+7hD69sYnttK4oSCfjc0hIOtKbcu/TRhSEuOaaS9x86asCTeSJt8+qGGv62eCsbquIk0m5J7cJIgA/PHstH5oxlTGHPRkLZjrJ0Sy2PvrWT3Q2tNCfTxEJ+KooinD53HHPGF5qbFmPYEJElqnpQU72npCAiG4CzVfXNDt6bDTysqlM8HOdpYEwHb12tqg9l9nmObpJCewsWLNDFiz3tavSQqrJmVyMPLd/Ouj1xWpJpIkE/k8uinDlvPIeNK8iZi6TtKJaYJ03D8KqzpOC1UbQS6GxYSRiY5OUgqnqqx/MZOUBEOHRsAYeOzf3Vxfo6x8IwDFdPZjR39kixAKjLQiyGYRjGIOv0SUFEvgF8I/NSgUdEJHnAbhGgBLi7r4GIyNnA/wDlwKMislxVP9jX4xqGYRjeddV8tAH4V+b7TwOLgaoD9kngjlD6TV8DUdUHgQf7ehzDMAyj9zpNCpmO37bOX4DrVXXjAMVlGIZhDAJPHc2mjIVhGMbI4HlKpohMAc4HJuKOOGpPVfVz2QzMMAzDGHiekoKInAXciztaaQ9uX0J7uTUDzjAMw+gVr08KP8CtcfRJVT2ws9kwDMMYJrwmhSnAlSYhGIZhDG9eJ6+tAUr7MxDDMAxj8HlNClcB38l0NhuGYRjDlNfmo+/jPimsFpF3gZoD3ldVXZTNwAzDMIyB5zUp2MDa/gzEMAzDGHxeJ6+d1M9xGIZhGDmgJ1VSDcMwjGHOc1IQkfEicpuILBaRjZnFdRCRK0Tk6P4L0TAMwxgonpKCiBwGrAAuBnbglroIZt6eBHy9X6IzDMMwBpTXJ4VbgdXAZOAcoP0yV/8GjslyXIZhGMYg8Dr66ATgE6raJCK+A97bTcfrLhuGYRhDjNcnBaeL98qAlizEYhiGYQwyr0nhdaCzNRXOB17OTjjGcPDjH/+YY489luLiYoqKijjhhBN44okn9tvnd7/7HSeffDJlZWXk5+czf/58/vznPw9SxIZhtPGaFH4AnCEiT+J2Nitwqoj8ATgb+GE/xWcMQc888wyf/exnefbZZ3n99dc57rjjOP3003n55Zf32+fMM8/k8ccfZ/ny5Vx00UVccskl3HPPPYMYuWEYouptKQQR+SjwU2Bqu82bgC+r6uPZD82bBQsW6OLFiwfr9IZHhx9+OKeddhq33nprp/t87GMfIxAIcP/99w9gZIYxMonIElVdcOB2zyuvqeqjwKMiMg0YBVSrqil9YXTLcRwaGhqIxWJd7ldXV0dlZeXABGUYRoc8J4U2qroOWNcPsRjD1I033khdXR2XX355p/v86U9/4tVXX+WnP/3pAEZmGMaBOk0KInJJTw6kqnf1PRxjuPnlL3/JjTfeyMMPP0xFRUWH+zz00EN8/vOf57e//S1HHnnkAEdoGEZ7XT0p3HnA67bOB+lgG4BJCsZ+fvKTn3Dttdfy8MMPc+qpp3a4z913382ll17KHXfcwcUXXzzAERqGcaCuksLkdt9XAH8BHgXuxp2wNhr4BPDhzL+Gsc/3vvc9br/9dh577DEWLep4qY077riDr371q/zhD3/gggsuGOAIDcPoSKdJQVU3t30vIj8D7lbVb7XbZS3wgoj8CHdltrP7LUpjSLniiiv41a9+xV//+lcOOeQQdu3aBUAkEqGwsBCA22+/nW9+85v84he/YNGiRfv2CQaDlJSUDFrshjHSeRqSKiKNwNmq+nQH750KPKCqBf0QX7fMkNTcIyIdbv/0pz/NnXfeCUBlZSWbN28+aJ9Fixbx3HPP9WN0hmFA34ekJoAFwEFJAVgIJPsQmzHMeLnR2LRpU/8HYhhGj3lNCvcC3xcRG/gb7/UpnA9cC/y2f8IzDMMwBpLXpHAlkA/cBNzcbrvidkBfmeW4DMMwjEHgqfaRqrao6sXALOBS4NuZf2ep6iWq2tpvEeY4L8Xf/vnPf3LsscdSVlZGOBxm6tSpXHPNNSSTptXNMIzc0qMZzar6DvBOP8UyJLUVf1u4cCHRaJTf/OY3nH766Tz//PMcf/zxABQUFPD1r3+d2bNnk5+fz7Jly7j88suJx+Pcfvvtg/wTGIZhvMfr6KOJ3e2jqluyElEP5eLoIy/F377xjW/w3HPPsWzZsgGMzDAMw9XX0Ueb2H/2ckcOXJFtRPJS/G3NmjU8/vjjfOQjHxnAyAzDMLrnNSl8loOTQilwOu7M5x/0NRAR+TFwBu7w1vXAZ1S1rq/HHWhdFX+rqKigqqqKZDLJF7/4RX7yk58MQoSGYRid85QUVPXOTt66TUT+CEzJQixPAd9W1bSI3ILbmf2tbj6TU7or/vbiiy/S3NzMsmXL+Na3vsWoUaO47rrrBiHS4a2tSbSzSXSGYXSux6WzO/An4PfANX05iKo+2e7lq8B5fTneQPNS/G3yZLec1GGHHYbP5+NTn/oUV111VbfrDBgHsx1lR10L66uaWLurkbe21bOlJk4i7ZB2FAvB7xOKIgFmjs1nbkURU8rzmFweoyAcGOzwDSNnZSMpjALCWThOe58FOl2XUUQuBy4HmDix2z7wfuel+NuBHMfBcRySyaRJCj2wp6GVx97eyUPLdtCSsgGwHYdwwEcoYBEJ+vaV8VWFhO3w2sYaXl5XTcBnkXYc5owv5Lz5E1hYWYzf53VFWsMYGbyOPjqxg81BYDZuM8+rqnqmh+M8DYzp4K2rVfWhzD5X45bUOEc9BDfYo4/aF3875phj9m1vX/zt1ltvZebMmcyYMQMRYfHixVx55ZUsWLCAhx56aLBCHzJUleVb67hvyTbe2FQDCrGwn5C/5xd0VaWhNY3jKPnhAOceWcGH54yhKBrsh8gNI3d1NvrIa1JwOLijue2G7Hngk6q6IwtBXgp8AXi/qjZ7+cxgJwUvxd9uvvlm/vjHP7Jp0yYsy2LSpElcdNFFXHHFFUSj0QGMduipjSf56b/e4ZX11YhAYSSAlaW+gtaUTXPSJhLw8Y3TZrBoRrnphzBGjL4mhZM4OCm0AptVdVeWAvwQcBuwSFWrvH5usJOC0T9UlRfeqeK2p96hJeVQHPX32wW7NWUTT9gcO7WUK06dQUnMPDUYw1+fksJAEJF1QAiozmx6VVW/2N3nTFIYflqSNrc8sYaX1+0lGvIRCfT/FBhVpbY5TSRg8Z2PHMrRU0r7/ZyGMZg6SwqeGmVFxBaRozp5b36memqfqOo0VZ2gqvMyX90mBGP4aWhN8c373uTldXspiQUGJCGA2wxYEnNHJX33obd5etXuATmvYeQar6OPunpu99H9bGfD6FZTIs1//u1NNu+NUxILDEr7fiTow7KEW55Yg6PKBw7raFyEYQxfrvCB/AAAD69JREFUXSYFEbF4LyFYmdftRXDXaN7bD7EZI0hryubqB1ewqSpO8SAlhDYhv4WE/fzkybVEgz5OmF4+aLEYxkDrtPlIRK4FUrhlJxR4OfO6/VcD8D3chXcMo9d+/cIGVm5vGPSE0Cbot4gGfdz42Bq217UMdjiGMWC6elJ4LvOv4F74fwtsO2CfBLAK+EfWIxvB4ok0G/fG2VDVxModDdS1pEjZDgHLIi/sZ9bYAqaUx5hSnkdhZOjPzn1zax2PvLkjZxJCm3DAR2sqxY+eWMPt58/DsnInNsPoL50mBf3/7d19cFzVecfx7293tZIsW0J+w2+AcQzh1WDADaUTJ21IoNDETWNKm2ECAdqhyaR03Ok0gaGFTqZJ80cIdNw4oRkaIE0xpCE4kJDg8F6b4Bgbg7GNjeP3V9myZb3uy+kf50pZroUkW7q690rPZ+aO7t49u/vs0d199p57zznOvYDvg4AkBzwwFH0RTO+6imVWvtvEo6/tYNO+FnLZDMVSmWwGspkMku+hWy47Xth0gKqsKJUd0xtruf6y05h/9iTG5Ieig/rwausq8rWfvU11VYZsAr90G2pzrN99lGVrd7Ng7vS4wzEmcid1SaqkBuAsYK9zLnz0MKzSfklqe1eJpat28L+rd9JRKJHLirHVA7sm3zlHW1eJrqIjlxVXXzCFz10+k4Yx6Tl6uO/ZTTy1bk+i+wZ0Fct0FEo8cOM8pp9SG3c4xgyJE74kVdJVkr7ey/Y7gf3Aq8A2Sf8tKX0/URPgzV1HuOX7r/HIym1kM6KxLs+4moE3oUiirjpHY10VNVUZnly7mxsf/DUvv3OApPQ/6cu+ox08vW5v4pvA8rkMxbLjkZXb4g7FmMj11U/hNuDsyg2SPo6fO2ED8HfAd4DrgdujCnAk6iqWWfzcZhYtXcPR9gITxubJn8Q4PpWqshkm1OVxznHPsvV89an1HOssDlHE0Xh63R4cLpHNRmENtVU8v2E/R9oLcYdiTKT6+oU/l+Mnz/k8fniLq7qHtwh+1X4WeP+5J02P9q4Sdy97i99sO0zjmKoh/0KszfvRQl/cdJDth9r5xmfm0JjAppnOYoknXt/F2Op0HGRmM6LkHMvf3sefXXL8XBnGjBR9/TydjJ8BrdLHgZdD4x09ReiIwvSus1jirp+8yepth5lQN/QJoVsm6J27/VAbi5auSeSv2xVbmugolAd9hDScaquyLF21g3I5+U1zxpysvj6RLUDPQP+SzsJPwbkyVO4oNj9zv5xzfOPnG1i7o3lYeutKonFMFbub27nzx+solsqRvt6JWrZ2N9mU7TW1+SzNbQXW7zkadyjGRKavpLABqJwjYQG+E9svQuXOBGygmH68sPEAL246OOzDN5wypoqNe1tYumrHsL1mf5xzbNjbQl0KL6EtlR1b9h+LOwxjItNXUrgXuFXS45IWA/cA6/A9mytdA6yNKL4R4VBrF/c+u4m66uywd86SRH1tjodWbGPrwdZhfe33s/tIB6VyOk4wh+UyYu3O5rjDMCYy75sUnHNP4K8wmgd8Dt9sdF3lbGiSpgBXAk9HHGeq3bf8HdoLfsrIOFRlMwj42tNvJ6I9fOuBZCSnk1GTz/D2npa4wzAmMn0evzvn7gfu7+P+vcDEoQ5qJNl5uI0VWw7SGHOHsvraHNuaWlm7s5m5pzfGGsvGfS0UE5CcTkY+m6GptZNjncXUXDllzIlIz6UfKfXUG3sAxT6mjyQQ/Gh1rB3QAdje1Eo+m76mI/D1mMtkONDSGXcoxkTCkkKEOgolfvrGHsbVJOMym4baKl7bepj9RztijaOjWI49SQ5WIWFXcxkzVCwpROj17c0US2Wqssmo5oxE2TlWbGnqv3CECqUyKc8JlhTMiJWMb6sRatO+oxQS1naehKtnqrIZUjA0U5+SkuiNGWq2Z0do7c4jVCesx25NPsP6mK+eqcllKKc4KzicJQUzYtmeHRHnHO/sO0ZtVbKqOJ/NcKi1k5aO+Ia+mDmxjkIxnUnBOUex5Jg0rjruUIyJRLK+sUaQzmAM/lzCflH6q2fE4db4ksLZp46jKqVXH3UVy0waV22Xo5oRK1nfWCNIV6JPpoquGE+UzppURzqPE6C9UOa8qfVxh2FMZCwpRCSx+SABptTXkM0olR3YSmXHnBkNcYdhTGQsKUQkH5xMTeIMaA4X6wlwSZw7tZ62hE8CFOacH69p9uRxcYdiTGQsKUSkOpelobaKQilZScE5RykBJ0oXXDyNUsqOFDoKZcbX5Tl3qiUFM3JZUojQB0+tp6NQijuM9+gslpl6Sk1sg/N1+9CZE6jN5+gqpqcTWHuhxJ9fdlrqe2Mb0xdLChGaM6MhcUcK7YUS50+Lv008n8vw6bnTOdaRjiakYjDU9x+dOznuUIyJlCWFCH1wyrjEzRngHMyZcUrcYQDwxxdMQVIqmpGOthf42DmnUl8T72i3xkTNkkKELpzewNjqXGKakLontrli9oS4QwFgcn0Nn7p4WiLnkK7UWSyTy2S44fIz4g7FmMhZUohQLpth4aUzaOtKRlJI4q/dm//gTCaOraYloc1IzjlaOop84aMfYEpDTdzhGBM5SwoR+8T5U8gkoImk+9LYBRdPizWOsNp8ljuuOZdCqRx7HfXmSHuRi2Y0cO2cqXGHYsywsKQQsfF1eT550TSOtMXbRNLcVuCSM8Yze/LYWOPozQXTG1hw8XQOtxUS1a+jvVAimxH/cNU5dsWRGTWUpA/hyZB0ANjWT7GJwMFhCCfNrI76Z3XUN6uf/iWpjs5wzk0Kb0x9UhgISaucc5fFHUeSWR31z+qob1Y//UtDHVnzkTHGmB6WFIwxxvQYLUnhu3EHkAJWR/2zOuqb1U//El9Ho+KcgjHGmIEZLUcKxhhjBsCSgjHGmB6pTwqSrpa0UdJmSV/u5f5qSY8G978qaWawfaakdklrgmXJcMc+XAZQR/MlrZZUlLQwdN+Nkt4JlhuHL+rhM8j6KVXsQ08OX9TDawB1tEjSeklvSFou6YyK+2wf6rt+krUPuWB2sDQuQBbYAswC8sBa4LxQmS8AS4L1vwAeDdZnAm/G/R4SUkczgTnAQ8DCiu3jgXeDv43BemPc7ykp9RPcdyzu95CQOvpDYEyw/jcVnzPbh/qonyTuQ2k/Uvg9YLNz7l3nXBfwP8CCUJkFwPeD9ceBj2l0jVnQbx05537rnHsDCM94cxXwS+fcIefcYeCXwNXDEfQwGkz9jBYDqaPnnHNtwc2VwIxg3fYh+qyfxEl7UpgO7Ki4vTPY1msZ51wROAJ0jx19pqTXJb0g6cNRBxuTgdRRFI9Ni8G+xxpJqyStlPSnQxtaYpxoHd0C/OwkH5tGg6kfSNg+lIs7gBjtAU53zjVJuhR4QtL5zrmjcQdmUuUM59wuSbOAX0la55zbEndQcZF0A3AZ8JG4Y0mi96mfRO1DaT9S2AWcVnF7RrCt1zKSckAD0OSc63TONQE4536DbxM8O/KIh99A6iiKx6bFoN6jc25X8Pdd4Hlg7lAGlxADqiNJVwJ3Ap9yznWeyGNTbjD1k7x9KO6TGoNZ8Ec67wJn8rsTPOeHynyR955oXhqsTwKywfos/D9xfNzvKY46qij7Xxx/onkr/gRhY7A+oupokPXTCFQH6xOBdwidYBwJywA/Z3PxP6zOCm23fajv+kncPhR7hQ7BP+QaYFNQ4XcG2/4Fn40BaoDHgM3Ar4FZwfbPAG8Ba4DVwCfjfi8x1tE8fDtoK9AEvFXx2JuDutsMfD7u95Kk+gGuANYFXwLrgFvifi8x1tGzwL7g87QGeNL2of7rJ4n7kA1zYYwxpkfazykYY4wZQpYUjDHG9LCkYIwxpoclBWOMMT0sKRhjjOlhScGkgqSbJDlJs3u5Lxfcd3c/zzEzKHdrZIEak3KWFIwxxvSwpGBMAkmqjjsGMzpZUjAmRNJHgolQWiS1SnpG0gUV9y+WtC8YS6vycdWSDku6r2LbJElLJO2S1Clpg6S/Dj2uu2lsvqTHJDUDrwb3zZP0uKSdwaRQGyX9q6Ta0HNkJX1V0h5JbZJ+Jemc3prVJF0k6ckg1nZJr4zgUYLNCbKkYNImG5xD6Fnwk5wMCUnXAsuBY8ANwGeBccBLkroHPXsYmAx8IvTwPwFOwU/Gg6R64GX8EAh3A9cCy4BvS/pSLy//A/zYQAuB7tm7TscPi3Abfh6C+/DDRjwYeuw9wB3Bay8AfgEcN4uXpEuA/8OPSfRX+OFemoBng9GCzWgX9zgbttgykAW4CXD9LHf38xwzg3K39lFmM7A8tK0eOAh8q2LbJuCHoXJPAOsrbt8FdHD8IGgPBM+XC723e/uJX/jB127AT/gzIdjeiE9i/xEqvyhcL/iE9zaQr9iWDbY9Eff/2Zb4FztSMGnzafwAdZXL5UPxxJLOAj4A/CB0JNIGrADmVxR/GFggaVzw2An4I4KHK8pcjW8G2hp6vmfwEz2dFwrhx73EVC/p3yRtATqBQvAaAs4Kil0I1OEHfqz0eOi5avHj+D8GlCviEX7AtvmYUW80T7Jj0ulN59zmyg3htv1BmBz8/V6whG2vWH8E32SzEN+Ucz3+8/RI6Plm47/IezMhdHtPL2UeBK4E/gnfjNSKn/5xMX4EYICpwd/9ocfuC90ejz8quCtYjiMp45wbrdOOGiwpGFOpKfj7Ffwv57Cu7hXn3FZJr+Cbch4M/j7vnKuclrEJ/0V9+/u83sbQ7fcMWSypBn9+4G7nXOXJ6wtDj+tOJpPxw8F3OzVUrhnf7LSY4LxHmCUEY0nBmN/ZCPwWP0HK1wdQ/iFgiaSPAr+PPwFc6efAl4Dtzrnwr/iBqMb/sg8fadwUur0OfwRxHfBcxfbrKgs551olvQRcBKy2BGB6Y0nBjEaXBpd9hj2Jn6nvJ5LywFL8CeFT8ZOhbHfOfbOi/GPAv+ObjNoJteED9+KblV6SdC8+6dQB5wAfds4t6CtI59wRSSuBv5e0J4jlZkKTwjvnDkv6FnCHpBb8Uc4l+AniwR8ddFsEvAg8I+l7+KOMiUH5rHPuy5hRzZKCGY1uC5awSc65pyXNx8+l+59ALbAXWAk8WlnYOdcsaRn+vMIPnXMtofuPSLoCfz7gH/Ff5s345PCjAcb6l8C38U0+7fhEdTvw01C5f8afML4F+Fv8Ce6bgFeAIxUxrZY0Lyh/P37O8gP42QeXDDAmM4LZzGvGjFCSFuKPZuY7516KOx6TDpYUjBkBJH0I3znuVXzfiEvxHeA2Alc4+6CbAbLmI2NGhmP4fgZfxHe2249vavqKJQRzIuxIwRhjTA/r0WyMMaaHJQVjjDE9LCkYY4zpYUnBGGNMD0sKxhhjevw/yZj4+Z1ChQcAAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name='Multicolinearity'></a>\n",
        "\n",
        "### **6.5.2. Multicolinearity:**"
      ],
      "metadata": {
        "id": "HcvvQ8EpGXm4"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "This happens when two or more columns of $X$ are close to\n",
        "being linearly dependent.\n",
        "\n",
        "**Signs:**\n",
        "1. Very high standard errors for regression coefficients\n",
        "2. The overall model is significant, but none of the coefficients are\n",
        "3. Large changes in coefficients when adding predictors\n",
        "4. Coefficients on different samples are wildly different\n",
        "5. High Variance Inflation Factor (VIF) and Low Tolerance\n",
        "\n",
        "**How To Solve:**\n",
        "1. Remove highly correlated predictors from the model. \n",
        "2. Use Partial Least Squares Regression (PLS) or Principal Components Analysis, regression methods that cut the number of predictors to a smaller set of uncorrelated components."
      ],
      "metadata": {
        "id": "Pr6reOKtGdjJ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### **6.5.2.1. Correlations:**"
      ],
      "metadata": {
        "id": "RJe8AJPOIELx"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "df = sns.load_dataset('iris')\n",
        "\n",
        "corr = df.corr()\n",
        "f, ax = plt.subplots(figsize=(12, 10))\n",
        "mask = np.triu(np.ones_like(corr, dtype=bool))\n",
        "cmap = sns.diverging_palette(230, 20, as_cmap=True)\n",
        "sns.heatmap(corr, annot=True, mask = mask, cmap=cmap);"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "Cwjc6OGEGbF8",
        "outputId": "f4261b5e-08b9-4ed0-8614-ed83886fd37a"
      },
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 864x720 with 2 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAApMAAAI/CAYAAAAryFJoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd5xcdbn48c+zKSSEBEhCS0InCBEVIdJVQFTQK1gQ1HuvgmguYkMvlmsB0ftTsXe8ARXEa8ELQmiiIjXUgNRQAkhJQEoSAiSQ+vz+mJM4STbZmTMzuzPZz5vXee2cMuc8O5zdPPt8y4nMRJIkSSqjq68DkCRJUucymZQkSVJpJpOSJEkqzWRSkiRJpZlMSpIkqbSBvXQdh4xLkqRmi74O4Joj9u21HGe/s6f2+ffbHSuTkiRJKs1kUpIkSaX1VjO3JEnSuqerLVuee5WVSUmSJJVmZVKSJKmssDJpZVKSJEmlWZmUJEkqKcK6nJ+AJEmSSrMyKUmSVJZ9Jq1MSpIkqTwrk5IkSSXZZ9LKpCRJkhpgMilJkqTSbOaWJEkqy8cpWpmUJElSeVYmJUmSynIAjpVJSZIklWdlUpIkqaRw0nIrk5IkSSrPyqQkSVJZ9pm0MilJkqTyrExKkiSV5TyTViYlSZJUnpVJSZKkkhzNbWVSkiRJDbAyKUmSVJajua1MSpIkqTwrk5IkSSXZZ9LKpCRJkhpgMilJkqTSbOaWJEkqywE4ViYlSZJUnpVJSZKksnycopVJSZIklWdlUpIkqaSwz6SVSUmSJJVnZVKSJKksJy23MilJkqTyrExKkiSVZWWy9mQyInYEPgVsXf2+zDywBXFJkiSpA9RTmfw98FPgNGBpa8KRJEnqHNFlj8F6ksklmXlqyyKRJElSx+kxmYyIkcXLCyLiOOAPwMLl+zNzTotikyRJam/2maypMnkzkMDyT+tTVfsS2K7ZQUmSJKkz9JhMZua2ABExJDNfrN4XEUNaFZgkSVK78wk49c0zeW2N2yRJktRP1NJncnNgLDA0Il7JP5u7RwDrtzA2SZIktbla+ky+ETgKGAd8p2r7c8DnWhCTJElSZ3AATk19Js8EzoyId2TmOb0QkyRJkjpEPfNMbh0Rn1xl2zzg5sy8tYkxSZIkdQYH4NQ1AGcicCyV/pNjgf8ADgZOi4hPtyA2SZIktbl6KpPjgN0y83mAiDgJuAh4DZW5KL/R/PAkSZLaV3TZZ7KeyuSmVD35BlgMbJaZL6yyXZIkSf1EPZXJ/wVuiIjzi/W3AL+OiGHA9KZHJkmS1O4czV17MpmZX4mIPwL7FJuOzcxpxet/bXpkkiRJanv1VCYBbgFmLX9fRGyVmY80PSpJkqRO4Gju2pPJiPgocBLwBLCUypNwEnh5a0KTJElSu6unMvlx4CWZObtVwUiSJHWSsM9kXaO5H6UySbkkSZIE1FeZfBC4IiIuomoqoMz8zprfIkmStA7rss9kPZ/AI8CfgcHA8KpFkiRJbSAiDo6IeyPi/oj4bDf7t4qIyyPibxFxe0S8qdFr1jM10MlFEOtn5oJGLyxJktTp2qnPZEQMAH4MvB6YCdwUEVMys3o+8C8AZ2fmqRExAbgY2KaR69ZcmYyIvSNiOnBPsf6KiPhJIxeXJElS0+wB3J+ZD2bmIuC3wGGrHJPAiOL1hsBjjV60nmbu7wFvBGYDZOZtVJ7LLUmS1D9FV68tETEpIqZVLZNWiWYslQHTy80stlX7EvBvETGTSlXyo41+BHVNWp6Zj65Szl3aaACSJEnqWWZOBiY3eJp3A2dk5rcjYm/grIjYJTOXlT1hXVMDRcQ+QEbEoIg4Abi77IUlSZLUVLOALavWxxXbqh0DnA2QmdcBQ4DRjVy0nmTyWODDVMqls4Bdi3VJkqT+KaL3lp7dBIyPiG0jYjDwLmDKKsc8AryuEnrsTCWZfKqRj6DmZDIzn87Mf83MzTJz08z8t7U9Dae6XX/y5EYrspIkSVqbzFwCfAS4lErr8dmZeVdEfDkiDi0O+0/ggxFxG/Ab4KjMzEauGz29PyJ+SGXkz5oC/1gN12koSEmSpG70+bw8t37xQ72W4+z6lVP7/PvtTi0DcKa1PApJkiR1pB6Tycw8s5YTRcQPM7Ph4eWSJEkdo6sti4W9qpkPlNy3ieeSJElSB6hrnklJkiRVaaPHKfaVZlYmJUmS1M80szJpai5JkvqVCOtyzfwEvt/Ec0mSJKkD9FiZjIgLWPs8k4cWX89oXliSJEkdwD6TNTVzf6vlUUiSJKkj1TLP5JW9EYgkSVKnsc9kHQNwImI88DVgApWHggOQmdu1IC5JkiR1gHpGc/8COAn4LnAAcDROLSRJkvozn4BTVzI4NDMvAyIzH87MLwFvbk1YkiRJ6gT1VCYXRqVjwIyI+AgwC9igNWFJkiSpE9STTH4cWB/4GPAV4EDgfa0ISpIkqSM4AKf2ZDIzbwIoqpMfy8znWhaVJEmSOkI9o7knUhmEM7xYnwe8PzNvblFskiRJbS2ctLyuZu6fA8dl5tUAEbEfleTy5a0ITJIkSe2vnmRy6fJEEiAzr4mIJS2ISZIkqTNYmawrmbwyIv4H+A2VZ3UfCVwREbsBZOYtLYhPkiRJbayeZPIVxdeTVtn+SirJ5YFNiUiSJKlTdDmau57R3Ae0MhBJkiR1nprT6YjYLCJ+FhGXFOsTIuKY1oUmSZLU3iKi15Z2VU9t9gzgUmBMsX4fcHyzA5IkSVLnqCeZHJ2ZZwPLADJzCbC0JVFJkiR1gujqvaVN1RPZ/IgYRWWwDRGxFzCvJVFJkiSpI9QzmvuTwBRg+4iYCmwCHN6SqCRJkjpAO/dl7C31VCa3Bw4B9qHSd3IG9SWjkiRJWsfUk0x+MTOfBTYGDgB+ApzakqgkSZI6gX0m60omlw+2eTNwWmZeBAxufkiSJEnqFPUkk7OKxykeCVwcEevV+X5JkiStY+rp83gEcDDwrcx8JiK2AD7VmrAkSZI6QJcDcOp5nOIC4Nyq9ceBx1sRlCRJkjqDo7ElSZJKcmog+zxKkiSpAVYmJUmSymrjKXt6i5+AJEmSSrMyKUmSVJZ9Jq1MSpIkqTwrk5IkSSVFl3U5PwFJkiSVZmVSkiSpLPtMWpmUJElSeVYmJUmSSgrnmbQyKUmSpPKsTEqSJJVln0krk5IkSSrPZFKSJEml2cwtSZJUlgNwrExKkiSpPCuTkiRJJUWXA3CsTEqSJKk0K5OSJEllOTWQlUlJkiSVZ2VSkiSpLEdzW5mUJElSeVYmJUmSSgr7TFqZlCRJUnlWJiVJksrqsi7nJyBJkqTSrExKkiSVZJ9JK5OSJElqgJVJSZKkspxn0sqkJEmSyjOZlCRJUmk2c0uSJJXlAJzeSSa/d8E1vXEZdaDj37JfX4cgSZIaYGVSkiSppHDScvtMSpIkqTyTSUmSpLKiq/eWWsKJODgi7o2I+yPis2s45oiImB4Rd0XErxv9CGzmliRJWgdExADgx8DrgZnATRExJTOnVx0zHvgvYN/MnBsRmzZ6XZNJSZKkktrscYp7APdn5oMAEfFb4DBgetUxHwR+nJlzATLzyUYvajO3JElSB4iISRExrWqZtMohY4FHq9ZnFtuq7QjsGBFTI+L6iDi40bisTEqSJJXV1XuVycycDExu8DQDgfHA/sA44KqIeFlmPlP2hFYmJUmS1g2zgC2r1scV26rNBKZk5uLM/DtwH5XksjSTSUmSpLLaazT3TcD4iNg2IgYD7wKmrHLMeVSqkkTEaCrN3g828hGYTEqSJK0DMnMJ8BHgUuBu4OzMvCsivhwRhxaHXQrMjojpwOXApzJzdiPXtc+kJElSSW02mpvMvBi4eJVtJ1a9TuCTxdIUViYlSZJUmpVJSZKksmp8Ms26zE9AkiRJpZlMSpIkqTSbuSVJksrqxUnL25WVSUmSJJVmZVKSJKmkcACOlUlJkiSVZ2VSkiSprDabtLwvWJmUJElSaVYmJUmSSrLPpJVJSZIkNcDKpCRJUln2mbQyKUmSpPKsTEqSJJXlE3CsTEqSJKk8K5OSJEklOZrbyqQkSZIaYGVSkiSpLEdzW5mUJElSeSaTkiRJKs1mbkmSpLIcgGNlUpIkSeVZmZQkSSopHIBjZVKSJEnlWZmUJEkqq8u6nJ+AJEmSSrMyKUmSVJJ9Jq1MSpIkqQFWJiVJksqyMllfMhkR+wDbVL8vM3/Z5JgkSZLUIWpOJiPiLGB74FZgabE5AZNJSZLUP/kEnLoqkxOBCZmZrQpGkiRJnaWeZPJOYHPg8RbFIkmS1FGiyz6TPSaTEXEBlebs4cD0iLgRWLh8f2Ye2rrwJEmS1M5qqUx+q+VRSJIkdSL7TPacTGbmlQARcUpmfqZ6X0ScAlzZotgkSZLU5upJp1/fzbZDmhWIJEmSOk8tfSY/BBwHbBcRt1ftGg5MbVVgkiRJbc9Jy2vqM/lr4BLga8Bnq7Y/l5lzWhKVJEmSOkItyeQA4Fngw6vuiIiRJpSSJKm/Cgfg1JRM3kxlaqAAtgLmFq83Ah4Btm1ZdJIkSWprtYzm3hYgIk4D/pCZFxfrhwBvbW14kiRJbcw+k3WN5t5reSIJkJmXAPs0PyRJkiR1inoep/hYRHwB+FWx/q/AY80PSZIkqTP4OMX6KpPvBjYB/lAsmxbbJEmS1E/VXJksRm1/vIWxSJIkdRZHc9c0afn3MvP4iLiAyqjulWTmoS2JTJIkSW2vlsrkWcXXb7UyEEmSpI7jaO6apga6uerYazPzhdaGJEmSpE5Rz2ju9wKnRsQc4GrgKuCazJzbksgkSZLanE/AqW8AzvsAImIMcDjwY2BMPeeQJEnSuqXmRDAi/g14NfAy4GngR1QqlCpkJlPP/w0P330HAwcP5sAj388m47Ze7bgbLjmXe6ddy8IXFvDBr/5kxfbbrryUu2+4mhgwgKHDNuCAI45m+MjRvfktSJKkejjPZF3zTH4P2BU4DfhYZn4jM69rTVid6ZF77uCZp57gPZ/9Kq89/L1cdc5Z3R639YRX8I6Pf2G17aPHbs07jv8iR/7nyWz38olcd9H/tTpkSZKkhtScTGbmaOD9wBDg/0XEjRHRfbbUTz101628ZOI+RASbb709C19cwPxnn1ntuM233p5hIzZabfvYHXZi0OD1ANhs6+2YP8/uqJIkqb3V08w9AtgK2BrYBtgQWNaasDrT/Hlz2WCjkSvWN9hwY+bPe6bbxLEn99xwDVvttEszw5MkSc3mAJy6mrmvAd4C3A4cmZkvWT4opzsRMSkipkXEtGv/OKXROPuV+26+jidnPsSu+x/c16FIkiStVT2juV++tv0R8cPM/GjV8ZOByQDfu+Ca1Z6cs664c+pfmX7DVQBsuuU2PP/MnBX7np83l2Eb1leVnHnfdG6+7CIO+9CnGTBwUFNjlSRJzRVOWt7UaX32beK5OsYu+x7ILvseCMDD02/jjql/ZYdd9+CJRx5kvSHr19XE/dSsh7nynF/y5g98gvWHj2hVyJIkSU3jHJFNtNXOL+fhe+7g11//LwYOGswBR75/xb6zv/MljvjklwC47sLfM+NvN7Bk8SJ++ZUT2HmPV/OqNx7GdRf+nsULF/Kns04FYIONRvKm93+sL74VSZJUC/tMEpnNaYGOiFsyc7fu9q3LzdxqzPFv2a+vQ5Akda4+b2P+x83X9VqOs/nue/f599udZlYm2/IblCRJahX7TNY3mrsn32/iuSRJktQBeqxMRsQFwBpLuJl5aPH1jOaFJUmS1AF8nGJNzdzfankUkiRJ6kg9JpOZeWVvBCJJktRxHM1d1+MUxwNfAyZQeT43AJm5XQvikiRJUgeoZzT3L4CTgO8CBwBH09wBPJIkSR3F0dz1JYNDM/MyKnNTPpyZXwLe3JqwJEmS1AnqSSYXRkQXMCMiPhIRbwM2aFFckiRJ7S+6em+pJZyIgyPi3oi4PyI+u5bj3hERGRETG/0I6kkmPw6sD3wM2B34d+B9jQYgSZKkxkXEAODHwCFUxri8OyImdHPccCp53Q3NuG7NfSYz86YigC7gY5n5XDMCkCRJUlPsAdyfmQ8CRMRvgcOA6asc9xXgFOBTzbhozZXJiJgYEXcAtwN3RMRtEbF7M4KQJEnqSF3Ra0tETIqIaVXLpFWiGQs8WrU+s9i2QkTsBmyZmRc16yOoZzT3z4HjMvPqIpj9qIzwfnmzgpEkSVL3MnMyMLns+4vW5e8ARzUrJqgvmVy6PJEEyMxrImJJM4ORJEnqJNFek5bPArasWh9XbFtuOLALcEUxpdHmwJSIODQzp5W9aD3J5JUR8T/Ab6g8q/vIIpjdADLzlrJBSJIkqWE3AeMjYlsqSeS7gPcs35mZ84DRy9cj4grghEYSSagvmXxF8fWkVba/kkpyeWAjgUiSJHWcNpq0PDOXRMRHgEuBAcDPM/OuiPgyMC0zp7TiuvWM5j6gFQFIkiSpOTLzYuDiVbaduIZj92/GNesZzb1ZRPwsIi4p1idExDHNCEKSJKkTRUSvLe2qnl6jZ1Apm44p1u8Djm92QJIkSeoc9SSTozPzbGAZVNrlgaUtiUqSJKkTtNnjFPtCPZHNj4hRVAbbEBF7AfNaEpUkSZI6Qj2juT8JTAG2j4ipwCbA4S2JSpIkqRN0tW9fxt5ST2VyeyoPDt+HSt/JGdSXjEqSJGkdU08y+cXMfBbYGDgA+AlwakuikiRJ6gARXb22tKt6Ils+2ObNwGnFA8IHNz8kSZIkdYp6mqlnFY9TfD1wSkSsR33JqCRJ0rqljed/7C31JINHUOkr+cbMfAYYCXyqJVFJkiSpI9TzOMUFwLlV648Dj7ciKEmSpI7Qxn0Ze4ufgCRJkkozmZQkSVJpzhMpSZJUUjgAx8qkJEmSyrMyKUmSVFaXdTk/AUmSJJVmZVKSJKkk+0xamZQkSVIDrExKkiSVZWXSyqQkSZLKszIpSZJUlo9TtDIpSZKk8qxMSpIklRRd9pm0MilJkqTSrExKkiSVZZ9JK5OSJEkqz8qkJElSWc4zaWVSkiRJ5ZlMSpIkqTSbuSVJkkoKB+BYmZQkSVJ5ViYlSZLKcgCOlUlJkiSVZ2VSkiSpJB+naGVSkiRJDbAyKUmSVJajua1MSpIkqTwrk5IkSWU5mtvKpCRJksqzMilJklSST8CxMilJkqQGWJmUJEkqy3kmrUxKkiSpPCuTkiRJZdln0sqkJEmSyovMbPlFbjzu7a2/iDrOZzZ5c1+HoDZ0+cnH9HUIkjpHn3dYfO7ZZ3stxxk+YkSff7/dsZlbkiSppHTScpu5JUmSVJ6VSUmSpJKWLuvrCPqelUlJkiSVZmVSkiSppMQxxlYmJUmSVJqVSUmSpJJ6YYbFtmdlUpIkSaVZmZQkSSppmaVJK5OSJEkqz8qkJElSSb3xWOp2Z2VSkiRJpVmZlCRJKsk+k1YmJUmS1AArk5IkSSVZmLQyKUmSpAaYTEqSJKk0m7klSZJKcmogK5OSJElqgJVJSZKkkpwayMqkJEmSGmBlUpIkqSQLk1YmJUmS1hkRcXBE3BsR90fEZ7vZ/8mImB4Rt0fEZRGxdaPXNJmUJEkqKTN7belJRAwAfgwcAkwA3h0RE1Y57G/AxMx8OfB/wDca/QxMJiVJktYNewD3Z+aDmbkI+C1wWPUBmXl5Zi4oVq8HxjV6UftMSpIkldSbo7kjYhIwqWrT5MycXLU+Fni0an0msOdaTnkMcEmjcZlMSpIkdYAicZzc44E1iIh/AyYCr230XCaTkiRJJbXZYO5ZwJZV6+OKbSuJiIOAzwOvzcyFjV7UPpOSJEnrhpuA8RGxbUQMBt4FTKk+ICJeCfwPcGhmPtmMi1qZlCRJKqmdnoCTmUsi4iPApcAA4OeZeVdEfBmYlplTgG8CGwC/jwiARzLz0EauazIpSZK0jsjMi4GLV9l2YtXrg5p9TZNJSZKkkmqZ/3FdZ59JSZIklWYyKUmSpNJs5pYkSSrJVm4rk5IkSWqAlUlJkqSS2mlqoL5iZVKSJEmlWZmUJEkqyamB6kwmI2IfYJvq92XmL5sckyRJkjpEzclkRJwFbA/cCiwtNidgMilJkvolC5P1VSYnAhPSeq4kSZIK9SSTdwKbA4+3KBZJkqSO4mjuGpLJiLiASnP2cGB6RNwILFy+PzMPbV14kiRJame1VCa/1fIoJEmSOpC9/2pIJjPzSoCIOCUzP1O9LyJOAa5sUWySJElqc/VMWv76brYd0qxAJEmSOs2y7L2lXdXSZ/JDwHHAdhFxe9Wu4cDUVgUmSZKk9ldLn8lfA5cAXwM+W7X9ucyc05KoJEmSOkDSxiXDXlJLn8l5wLyI+PCq+yJiUGYubklkkiRJanv19Jm8BXgKuA+YUbx+KCJuiYjdWxGcJEmS2ls9yeSfgTdl5ujMHEVl8M2FVPpT/qQVwUmSJLWzzOy1pV3Vk0zulZmXLl/JzD8Be2fm9cB6TY9MkiRJba+exyk+HhGfAX5brB8JPBERA4BlTY9MkiSpzbXzlD29pZ7K5HuAccB5xbJVsW0AcETzQ5MkSVK7q7kymZlPAx9dw+77mxOOJElS52jnvoy9peZkMiJ2BE4Atql+X2Ye2PywJEmS1Anq6TP5e+CnwOnA0taEI0mS1DmsTNaXTC7JzFNbFokkSZI6Tj3J5AURcRzwB2Dh8o0+UlGSJPVXy6xM1pVMvq/4+qmqbQls17xwJEmS1EnqGc29bSsDkSRJ6jQWJuuYZzIi1o+IL0TE5GJ9fET8S+tCkyRJUrurp5n7F8DNwD7F+iwqI7wvbHZQnWrDCa9kq3e+n4gunrr2Lzz+pz+stH/wxqPZ7n0fZcDQYURXF4+e9yvm3XULo171GjY/6LAVx60/dmvu+voJLJj5UC9/B2qVjx6yF3uO35IXFy/hlPOuYsbjs9d47H+/+yDGbDyC9//kXACOPnA39n3J1mQmc+e/yCnnXcXs5xb0VuiSpLWwz2R9yeT2mXlkRLwbIDMXRES0KK7OE11sfeQHufcHJ7Pomdm89DPfYO7tN/HiP2auOGTMIYcz5+ZrefLqSxmy+The8uEvcNsXj2X2TVcx+6arABg6ZivG/8dnTSTXIXuOH8fYUSP4tx/8np3HbcIn/mUfjjvtgm6PffXOW/PioiUrbfvd1Dv4xV9vAeDte07gva/dle9eeG3L45YkqRb1PE5xUUQMpTLohojYnqpR3f3dBtvswMKnHmfh7CfIpUuYffM1bPyKPVY+KGHAkPUBGDh0fRbNW30g/KiJr2bOzdf0RsjqJfvutDV/urXykKi7Zz7FsCGDGbnB0NWOGzJ4IO/cexfOuurWlbYvWLh4pWP8G1iS2kdm9trSruqpTJ4E/BHYMiL+F9gXOKoVQXWiQRuNYuHcfzZdLpo7mw22Gb/SMbMu+h0v+eiJbLb/m+habz3u+f6XVjvPyN33ZcZPv97qcNWLRg9fnyefnb9i/elnFzB6xDDmPP/CSse9/8DdOfvaO3lx8ZJVT8Exr9udN7xiB+a/uJhPnHFxy2OWJKlWNVcmM/PPwNupJJC/ASZm5hVrOj4iJkXEtIiY9ofpf280znXCqIn78fT1l3Pr5z/IfT/+b7Y/6uNQ1VNg2DbjWbZoIS88/kgfRqm+sP3mIxmz8XCuuefhbvf/7LKbOfI7v+Mvd9zP2/bcuZejkyRpzXpMJiNit+ULsDXwOPAYsFWxrVuZOTkzJ2bmxLdNWPdnFVr8zGzW23jUivXBG49arRl79D6vY84tUwF4/u/3EYMGMXDYiBX7R+2+H7On2cS9LnjrHjtz2rFv5bRj38rs519g0xHDVuwbPWJ9nq6qVAK8dMtNecmY0fzm+CP44fv/hXGjRvDdo9602nn/cvsDvGbndf/nSZI6RWbvLe2qlmbub69lXwIHNimWjvb8w/ez3qZbMHjUpix+Zg6jdt+PB37x3ZWOWTT3aUa85OU8ff3lDNl8LF0DB7Pk+XmVnRGM3H0f7v72F/ogejXbeTfezXk33g3AXuO35K177sxf73yQncdtwvwXF6/WxD3lpnuYctM9AGy20QZ87T1vWNGcPXbkCGbNeRao9L985OlnevE7kSRp7XpMJjPzgFpOFBGvL5rC+6dly3j4d6ez00dOhK4unrruMl54/FHG/su7mP/wAzxzx008cs4ZbPuvx7H5gW8hM3nwrB+uePvwHSawaO5sFs5+og+/CbXC9TMeZc8dx/Grj7+ThYuXcMp5V6/Yd9qxb+WDPz1vre+f9PqJbDlqI5Zl8sS85/nuBVNbHbIkqUZODQTRrNFBEXFLZnbb7H3jcW/3k9ZqPrPJm/s6BLWhy08+pq9DkNQ5+nyKwqvvfKDXcpxX77J9n3+/3alnNHdP2vIblCRJapV0wra65pnsiZ+mJElSP9PMyqQkSVK/YpfJ5lYmH2riuSRJktQBeqxMRsTb17Y/M88tvq71OEmSpHWNo7lra+Z+y1r2JXBuk2KRJElSh6llnsmjeyMQSZKkTtOsKRY7WV0DcCLizcBLgSHLt2Xml5sdlCRJkjpDzclkRPwUWB84ADgdOBy4sUVxSZIktb1lFibrGs29T2a+F5ibmScDewM7tiYsSZIkdYJ6mrlfKL4uiIgxwGxgi+aHJEmS1BnsM1lfMnlhRGwEfBO4hcpI7tNbEpUkSZI6Qj3J5DcycyFwTkRcSGUQzoutCUuSJEmdoJ4+k9ctf5GZCzNzXvU2SZKk/iYze21pV7U8AWdzYCwwNCJeCUSxawSV0d2SJEnqp2pp5n4jcBQwDvhO1fZngc+1ICZJkqSO4NRAtT0B50zgzIh4R2ae0wsxSZIkqUPU02dyakT8LCIuAYiICRFxTIvikiRJanv2mawvmfwFcCkwpli/Dzi+6RFJkiSpY9STTI7OzLOBZQCZuQRY2pKoJEmSOsCyzF5b2lU9yeT8iBhFZbJyImIvYF5LopIkSVJHqGfS8k8CU4DtImIqsAlweEuikiRJ6gDtWy/sPfUkk9OBPwALgOeA86j0m5QkSVI/VU8y+Usqc0t+tVh/D3AW8M5mByVJktQJ2nmUdW+pJ5ncJTMnVK1fHhHTmx2QJEmSOkc9yeQtEbFXZl4PEBF7AtNaE5YkSe2fL6oAABkvSURBVFL7a+dR1r2lnmRyd+DaiHikWN8KuDci7gAyM1/e9OgkSZLU1upJJg9uWRSSJEkdyMJkHclkZj7cykAkSZLUmIg4GPg+MAA4PTO/vsr+9agMqt4dmA0cmZkPNXLNeiYtlyRJUpuKiAHAj4FDgAnAuyNiwiqHHQPMzcwdgO8CpzR6XZNJSZKkkjKz15Ya7AHcn5kPZuYi4LfAYasccxhwZvH6/4DXRUQ08hmYTEqSJHWAiJgUEdOqlkmrHDIWeLRqfWaxrdtjMnMJlUdjj2okrnoG4EiSJKlKb04NlJmTgcm9dsEaWZmUJElaN8wCtqxaH1ds6/aYiBgIbEhlIE5pJpOSJEkltVmfyZuA8RGxbUQMBt4FTFnlmCnA+4rXhwN/zQafCWkztyRJ0jogM5dExEeAS6lMDfTzzLwrIr4MTMvMKcDPgLMi4n5gDpWEsyEmk5IkSSUta7NJyzPzYuDiVbadWPX6ReCdzbymzdySJEkqzcqkJElSSQ12N1wnWJmUJElSaVYmJUmSSrIyaWVSkiRJDbAyKUmSVNKyvg6gDViZlCRJUmlWJiVJkkqyz6SVSUmSJDXAZFKSJEml2cwtSZJUks3cViYlSZLUACuTkiRJJS2zMGllUpIkSeVZmZQkSSrJPpNWJiVJktQAK5OSJEklLbMyaWVSkiRJ5VmZlCRJKsnCpJVJSZIkNcDKpCRJUkmO5rYyKUmSpAZYmZQkSSrJ0dxWJiVJktQAK5OSJEklWZjspWRywAYjeuMy6jAbrr9eX4egNnTNEfv2dQhqQ/udPbWvQ5C0BlYmJbWNE24+ta9DkKS62GfSPpOSJElqgMmkJEmSSrOZW5IkqSSbua1MSpIkqQFWJiVJkkrycYpWJiVJktQAK5OSJEklWZi0MilJkqQGWJmUJEkqydHcViYlSZLUACuTkiRJJSVWJq1MSpIkqTQrk5IkSSXZZdLKpCRJkhpgZVKSJKkkR3NbmZQkSVIDrExKkiSV5LO5rUxKkiSpASaTkiRJKs1mbkmSpJIcgGNlUpIkSQ2wMilJklSShUkrk5IkSWqAlUlJkqSSnBrIyqQkSZIaYGVSkiSpJEdzW5mUJElSA6xMSpIklWRh0sqkJEmSGmBlUpIkqSRHc1uZlCRJUgOsTEqSJJXkaG4rk5IkSWqAlUlJkqSSrEtamZQkSVIDTCYlSZJUms3ckiRJJTkAx8qkJEmSGmBlUpIkqSQnLbcyKUmSpAZYmZQkSSrJwqSVSUmSJDWg5spkRLwdOAXYFIhiycwc0aLYJEmS2pqjueurTH4DODQzN8zMEZk53ERSkiSpM0TEyIj4c0TMKL5u3M0xu0bEdRFxV0TcHhFH9nTeepLJJzLz7rqiliRJWodlZq8tTfBZ4LLMHA9cVqyvagHw3sx8KXAw8L2I2GhtJ+2xmbto3gaYFhG/A84DFi7fn5nn1ha/JEmS+tBhwP7F6zOBK4DPVB+QmfdVvX4sIp4ENgGeWdNJa+kz+Zaq1wuAN1RfEzCZlCRJ/VJvdpmMiEnApKpNkzNzch2n2CwzHy9e/wPYrIfr7QEMBh5Y23E9JpOZeXRxwn0zc+oqF9m3p/dLkiSpcUXiuNbkMSL+Amzeza7Pr3KujIg1psIRsQVwFvC+zFy2tmvWM8/kD4HdatgmSZLUL7TbaO7MPGhN+yLiiYjYIjMfL5LFJ9dw3AjgIuDzmXl9T9espc/k3sA+wCYR8cmqXSOAAT29X5IkSW1hCvA+4OvF1/NXPSAiBgN/AH6Zmf9Xy0lrGc09GNiASuI5vGp5Fji8lotIkiStizpsNPfXgddHxAzgoGKdiJgYEacXxxwBvAY4KiJuLZZd13bSWvpMXglcGRFnZObDDX0LkiRJ6hOZORt4XTfbpwEfKF7/CvhVPeetpZn7AiqjtomI7gI7tJ4LSpIkad1RywCcbxVf305ldNDybPXdwBOtCEqSJKkTLGuv8Td9otZmbiLi25k5sWrXBRExrWWRSZIkqe3VMzXQsIjYLjMfBIiIbYFhrQlLkiSp/SWWJutJJj8BXBERDwIBbA38R0uikiRJUkeoOZnMzD9GxHhgp2LTPZm5cG3vkSRJWpc1acqejlbLaO4DM/OvEfH2VXZtHxFkps/mliRJ6qdqqUy+Fvgr8JZu9iVgMilJkvolR3PXNpr7pOLr0a0PR5IkSZ2k5j6TEfEAcD1wNXB1Zt7VsqgkSZI6gH0ma3s293ITgP8BRgHfjIgHIuIPrQlLkiRJnaCeqYGWAouLr8uAJ4tFhRE7vowtD3sPRBdP33gVT1xx0Ur7B200km2P/CADhqwPXV3MuuT3PHvP7Qwf/1LGHvJOugYMYNnSpcy66Hc898DdffRdqBU+8LqJ7L79WBYuXsIPLr6OB5+Ys9oxJ77zQDbeYCgDuoLpjz7J5D/fxLJMTjh0P8aOHAHAsCGDmf/iIj5xxsW9/S2oiTZ6xZ5sd/TxRFcXT1x2ATPPX/kxuOuN3ozxH/ocg0ZsxJLnn+XeH36ZRXOequwbtRk7HPtZ1hu1KZDc9bUTWPjUP/rgu5AEsMzKZF3J5LPAHcB3gNOKh4VruQi2etu/c99p32TxvDns9NGTmDf9b7z45GMrDtnidYcy57Ybefr6yxmy6Rh2eP8nufPrJ7Bk/nM8cMb3WPzsMwzZbCzjP3ACd/y/T/ThN6Nm2n27MWwxcjgfmnw+O44ZzbFv2INPn/XH1Y775vlX88KixQB85q2vYZ+dtuKaux/mW1OuWXHM0QfsxvyFi3stdrVAdLH9Mf/Jnf99PItmP8muXzud2dOu4YVZD604ZNt//whPXvVHnrzyEjZ86W5s855jue9HXwFgx498gUfP/SXP3HETXesNhVzWR9+IJFXU08z9buAq4DjgtxFxckS8rjVhdZ5hW27Hi08/waI5T5FLlzL3thvY6KWvXPmgTAYMGQrAgCFDWfzsXABeeOwRFj/7DAAvPjGLrkGDiAH15PlqZ3uM35Ir7vw7APc99jTD1hvMxsOGrnbc8kRyQFcwcEAX3f2xu+9OW3P13Q+1Mly12PAddubFf8xk4ZOPkUuX8NS1lzHqVa9e6Zih47blmTtvBmDeXbcwcmJl/9Cx28CAATxzx00ALFv4AssWOd2v1Jcys9eWdlXPpOXnA+dHxE7AIcDxwKeB1f9V7IcGbbgxi+f9s+ly0by5DNtyu5WOeezP57HjB05g030Oomvwesw47RurnWejl01kwayHyaVLWh6zesfIDYby9LPzV6zPfm4+I4cPZe78F1Y79qQjDmT8FqO45cHHuO7eR1baN2Hcpjwz/0Uen/tcy2NW6wweuQkLZ/+zh9DC2U8yfPxLVzpm/sMzGL3Ha3nskt8zao/XMnD9YQzcYARDx2zJ0vnPs9N/fpUhm27BM3dM46H/PdXqpKQ+VXNlMiLOiYj7ge8D6wPvBTZey/GTImJaREw797b7Go90HTBy1714+uap3PHVT3L/z7/DNu+aBBEr9g/ZbAzj3nQED59zRt8FqT518tl/5egfncOgAQN42dabrbTv1RO2sSrZTzx01o8ZMeGV7HrKL9hwwq4snP0kuWwZ0TWAETu/gr+f9SNu/a8PMGSzMWy2/5v6OlypX8vsvaVd1dOW+jXgb5m5tLudEfH6zPzz8vXMnAxMBrj500e18UfQHIvnzWXQhiNXrA/ecOMVzdjLjX7Va5jxs28DMP+RB+gaOIiB62/AkvnPMWjDjdn+vR/j77+dvKKjvTrXIa/ckTe8YgcAZvxjNqNHDINZlf+vo4YPY85zq1cll1u8dBk3zHiUPXbYktseqgys6Ipg7x235D/PvKT1waulFs15qhg8U7HeqE1X+5lfNPdp7vn25wDoWm8oo/bcn6ULnmfRnKeY/9AMFhZ9sWffeBXDd3wpXN578UvSqmquTGbmtDUlkoVTmhBPx5o/8+8MGb0ZgzceTQwYwMav2JNnpv9tpWMWPTObETtMAGDIplsQgwaxZP5zDBiyPjsc/QlmXfJ75j98f1+Erya75G/38YkzLuYTZ1zMDffNZP9dtgVgxzGjmb9w0WpN3EMGDVzRj7Irgonbj2XWnHkr9r9im82ZOftZZj+3oPe+CbXEcw/cw9AtxrHeJlsQAwayyT6vY860a1Y6ZuDwDVe0Wmz5tn/nicsrM0M8d//dDFx/AwYO3wiAjXbZnRdmPtSr8UvSqpo5yiN6PmQdtmwZj5z/K8Z/4ASiq4unb7qaF594jC3e8DYWzPw786bfyswLf8vWhx/Npq9+AwAP/e50ADbZ53WsN3oztjjoMLY46DAAZpz2TZbMt2/cuuDmB2ex+/Zj+Omkw1i4pDI10HLfPepNfOKMi1lv0EA+9479GTSgi4jgzkf+wR//NmPFca/e2SbudcaypTzw8++yy+e/A10DeOLyC1kw8+9sdcQHeP6Be5hz8zVsOOGVbPOeYyGTeXffxgNFiwa5jL+f9WNeduL3IYLnH7yXf/xlSt9+P1I/59RAEM0aHRQRt2Tmbt3t6w/N3KrfV0Yd1NchqM2ccPOpfR2C2tR+Z0/t6xDUnvq8kHXc5PN7Lcf5yaTD+vz77Y7zz0iSJJXUzlP29JZ65pnsyUNNPJckSZI6QI+VyYh4+9r2Z+a5xde1HidJkrSusTBZWzP3W9ayL4FzmxSLJEmSOkyPyWRmHt0bgUiSJHUaR3PXOQAnIt4MvBQYsnxbZn652UFJkiSpM9ScTEbET6k8RvEA4HTgcODGFsUlSZLU9hIrk/WM5t4nM98LzM3Mk4G9gR1bE5YkSZI6QT3N3Muf/7YgIsYAs4Etmh+SJElSZ1hmYbKuZPLCiNgI+CZwC5WR3Ke3JCpJkiR1hHqSyW9k5kLgnIi4kMognBdbE5YkSVL78wk49fWZvG75i8xcmJnzqrdJkiSp/6nlCTibA2OBoRHxSv75UPURVEZ3S5Ik9UtWJmtr5n4jcBQwDvhO1fZngc+1ICZJkiR1iFqegHMmcGZEvCMzz+mFmCRJktQh6ukzOTUifhYRlwBExISIOKZFcUmSJLW9Zdl7S7uqJ5n8BXApMKZYvw84vukRSZIkqWPUk0yOzsyzgWUAmbkEWNqSqCRJkjpAZvba0q7qSSbnR8QoKpOVExF7AfNaEpUkSZI6Qj2Tln8SmAJsFxFTgU2Aw1sSlSRJUgdo54phb6knmZwO/AFYADwHnEel36QkSZL6qXqSyV9SmVvyq8X6e4CzgHc2OyhJkqRO0M6jrHtLPcnkLpk5oWr98oiY3uyAJEmS1DnqSSZviYi9MvN6gIjYE5jWmrAkSZLan30m60smdweujYhHivWtgHsj4g4gM/PlTY9OkiRJba2eZPLglkUhSZLUgZZZmaw9mczMh1sZiCRJkjpPPZVJSZIkVbEuWd8TcCRJkqSVWJmUJEkqydHcViYlSZLUAJNJSZIklWYztyRJUklODWRlUpIkSQ2wMilJklSShUkrk5IkSWqAlUlJkqSSnBrIyqQkSZIaYGVSkiSpJEdzW5mUJElSA6xMSpIklWSfSSuTkiRJaoCVSUmSpJKWWZi0MilJkqTyrExKkiSVZJ9JK5OSJElqgJVJSZKkkqxMWpmUJElSA0wmJUmSVJrJpCRJUknLsveWRkXEyIj4c0TMKL5uvJZjR0TEzIj4UU/nNZmUJEnqHz4LXJaZ44HLivU1+QpwVS0nNZmUJEkqKXvxvyY4DDizeH0m8NbuDoqI3YHNgD/VclKTSUmSpA4QEZMiYlrVMqnOU2yWmY8Xr/9BJWFc9RpdwLeBE2o9qVMDSZIkldSbUwNl5mRg8tqOiYi/AJt3s+vzq5wrI6K74I8DLs7MmRFRU1wmk5IkSeuIzDxoTfsi4omI2CIzH4+ILYAnuzlsb+DVEXEcsAEwOCKez8w19q80mZQkSSqpGaOse9EU4H3A14uv5696QGb+6/LXEXEUMHFtiSTYZ1KSJKm/+Drw+oiYARxUrBMREyPi9LIntTIpSZJUUic9TjEzZwOv62b7NOAD3Ww/Azijp/NamZQkSVJpViYlSZJKWtZBlclWsTIpSZKk0qxMSpIklWRh0sqkJEmSGhCdNAppXRARk4oZ7KUVvC/UHe8Ldcf7Qu3GymTvq/c5muofvC/UHe8Ldcf7Qm3FZFKSJEmlmUxKkiSpNJPJ3mc/F3XH+0Ld8b5Qd7wv1FYcgCNJkqTSrExKkiSpNJNJSZIklWYyKUmSpNJMJpssIvaPiAvXsv+oiPhRC657VESMqVp/KCJGN/s6akxP90cN758YET9Yw76HImJ0RGwUEcc165qq36o/j2s57oyIOHwt+6+IiIlNjs37o4816/6o4f1fjoiDutm+4v958XqfZl1T/ZPJ5LrjKKDHX07qbJk5LTM/1sNhGwHH9XCMWuso2vfn0fuj7x1FL9wfmXliZv6lh8P2B/bp4RhprfplMhkRwyLiooi4LSLujIgjI2L3iLgyIm6OiEsjYovi2Csi4vsRcWtx7B7F9j0i4rqI+FtEXBsRLykRxyYRcU5E3FQs+xbbvxQRPy+u/WBEfKzqPV+MiHsj4pqI+E1EnFD8FTkR+N8izqHF4R+NiFsi4o6I2KnhD66f6Mv7o/h/tVFUzI6I9xbbfxkRr1+lojAqIv4UEXdFxOlAFKf5OrB9EdM3i20bRMT/RcQ9EfG/ERGrX11rEhHbVH12dxef5frd3Rfd/TxGxInFz/idETG5zOcfEW8o7qlbIuL3EbFBsf2hiDh51Z/14vfLn5ffHxHxcFRaK7w/mqwv7o+IeFVEnFu8PiwiXoiIwRExJCIeLLavqDJGxMFFjLcAb18eN3As8IkillcXp39N8XvrwbBKqVpkZr9bgHcAp1WtbwhcC2xSrB8J/Lx4fcXyY4HXAHcWr0cAA4vXBwHnFK/3By5cy7WPAn5UvP41sF/xeivg7uL1l4p41gNGA7OBQcCrgFuBIcBwYAZwQlWcE6uu8xDw0eL1ccDpff25d8rSx/fHT4E3A7sAN1WdewYwrPr9wA+AE4vXbwayuF+2WR5H1TXnAeOo/AF53fL7zqXme2Kb4vPdt1j/OfCpHu6L6p/HkVWvzwLeUrw+Azh8Lde9gkriMRq4ChhWbP9M1f/7bn/WgR8B/1W8Ptj7Y926P4CBwIPF628Vvy/2BV4L/Kb6/VT+zXgUGE/lj86zq36PfIni35Gq9/y+uBcmAPf39efr0v7LQPqnO4BvR8QpwIXAXCr/eP+5+INwAPB41fG/AcjMqyJiRERsRCWZOzMixlP5JTKoRBwHAROq/ggdsbzaAFyUmQuBhRHxJLAZlV8U52fmi8CLEXFBD+c/t/h6M8VfoqpJX94fV1NJSh8GTgUmRcRYYG5mzl+lYPEaiv+vmXlRRMxdy3lvzMyZABFxK5V//K6pMSZVPJqZU4vXvwI+x9rvi2oHRMSngfWBkcBdQE8/v9X2ovIP+9TiWoOpJH3Ldfezvh/wNoDM/KP3R8v16v2RmUsi4oGI2BnYA/gOld8JA6j8Hqm2E/D3zJwBEBG/Yu3P9z4vM5cB0yNis7XFIQH9M5nMzPsiYjfgTcB/A38F7srMvdf0lm7WvwJcnplvK5oKrigRShewV5EcrlD84llYtWkp5f5fLT9H2ff3S318f1wFfJhKpfrzVJKBw1n9H4d6NeN+6u9W/f/8HGu/LwCIiCHAT6hUoh6NiC9RqRTVI4A/Z+a717C/0Z9174/G9cX9cRVwCLAY+AuVquIAKlXRRlTfD3Z5UI/6a5/JMcCCzPwV8E1gT2CTiNi72D8oIl5a9ZYji+37AfMycx6Vps9Zxf6jSobyJ+CjVXHt2sPxU4G3FH1iNgD+pWrfc1SqYWpQX94fmfkolabI8Zn5IJXq0AlU/tFY1VXAe4prHwJsXGz3XmiNrZbfA1Q+9+tZ831R/f9geWLwdPFzW6YP2vXAvhGxQ3GtYRGxYw/vmQocURz/Brw/Wq0v7o+rgeOB6zLzKWAU8BLgzlWOuwfYJiK2L9ar/yjxflDD+mUyCbwMuLFozjkJOJHKD/ApEXEblX6J1aPbXoyIv1Hpz3ZMse0bwNeK7WX/iv8YMDEibo+I6VQ6Qq9RZt4ETAFuBy6h0hw7r9h9BvDTWHkAjsrp6/vjBuC+4vXVwFi6b3I8mUpH+buoNG0+ApCZs6k0h94Z/xxgocbdC3w4Iu6mkpj9kDXfF2dQ/DxSqfKcRuUf+Eup9G2rS5EoHAX8JiJup9LE3dOgupOBN0TEncA7gX8Az3l/tExf3B83UOkCtfyPzduBOzJzpSpp0fo1CbioGIDzZNXuC4C3rTIAR6qLz+buQURcQaVz8rS+jgUgIv5/+3aIqlAUBnF8xiKIaxCs4gZcjkGLwehSBJvVYrG4AOtbzUsPHmPQIIIoB+Vc9f+LJ034uAz3fKeb5Nd2R6cPyDTJT+1c36pp84HXOK8q7JIMK0d5mO22pP/zbt1I0jLJvdsPFHjH+QCeib2Y97OyPdDpamRNkQRwQ0/SxnZL0p+kSeU8AD4UfyZfxPZY0vzq+JBkViMPmoX5wCXbW0n9q+NFkn2NPGgW5gNNR5kEAABAsW99gAMAAIAnoEwCAACgGGUSAAAAxSiTAAAAKHYExPaG/og73IoAAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "As shown above, for example, petal length and petal width have a very high correlation and one of them can be removed."
      ],
      "metadata": {
        "id": "8pCXBnGRKJRV"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### **6.5.2.2. Variance Inflation Factors (VIF):**\n",
        "\n",
        "A high VIF indicates that the associated independent variable is highly collinear with the other variables in the model. In general, a VIF above 10 indicates high correlation and is cause for concern. "
      ],
      "metadata": {
        "id": "Apq7A5KkKfmO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "df = sns.load_dataset('iris')\n",
        "\n",
        "variables = df.iloc[:,:-1]\n",
        "vif = pd.DataFrame()\n",
        "vif['VIF'] = [variance_inflation_factor(variables.values, i) for i in range(variables.shape[1])]\n",
        "vif['features'] = variables.columns\n",
        "vif"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 0
        },
        "id": "yoqiW9fALz2B",
        "outputId": "09d3c44d-25de-475f-9f12-c1016b78acc0"
      },
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "          VIF      features\n",
              "0  262.969348  sepal_length\n",
              "1   96.353292   sepal_width\n",
              "2  172.960962  petal_length\n",
              "3   55.502060   petal_width"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-afdcda24-0871-41f2-91c9-7af4f732474a\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>VIF</th>\n",
              "      <th>features</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>262.969348</td>\n",
              "      <td>sepal_length</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>96.353292</td>\n",
              "      <td>sepal_width</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>172.960962</td>\n",
              "      <td>petal_length</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>55.502060</td>\n",
              "      <td>petal_width</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-afdcda24-0871-41f2-91c9-7af4f732474a')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-afdcda24-0871-41f2-91c9-7af4f732474a button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-afdcda24-0871-41f2-91c9-7af4f732474a');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 41
        }
      ]
    }
  ]
}