ml_classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "Classification is a machine learning algorithm whose output is within a set of discrete labels. This is in contrast to Regression, which involved a target variable that was on a continuous range of values.\n",
    "\n",
    "Throughout this tutorial we'll review different classification algorithms, but you'll notice that the workflow is consistent: split the dataset into training and testing datasets, fit the model on the training set, and classify each data point in the testing set. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## LDA vs QDA\n",
    "\n",
    "Discriminant Analysis is a statistical technique used to classify data into groups based on each data point's features. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Linear Discriminant Analysis\n",
    "\n",
    "Linear Discriminant Analysis (LDA) is a linear classification algorithm used for when it can be assumed that the covariance is the same for <b>all</b> classes. It's mostly used as a dimension reduction technique in the pre-processing portion so that the different classes are separable and therefore easier to classify and avoid overfitting. \n",
    "\n",
    "LDA is similar to Principal Component Analysis (PCA), but LDA also considers the axes that maximize the separation between multiple classes. It works by finding the linear combinations of the original variables that gives the best separation between the groups in the data set. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Quadratic Discriminant Analysis\n",
    "\n",
    "Quadratic Discriminant Analysis, on the other hand, is used for heterogeneous variance-covariance matrices. Because QDA has more parameters to estimate, it's typically less accurate than LDA."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Support Vector Machines\n",
    "\n",
    "Support Vector Machines (SVMs) is a machine learning algorithm used for classification tasks. Its primary goal is to find an optimal separating hyperplane that separates the data into its classes. This optimal separating hyperplane is the result of the maximization of the margin of the training data.\n",
    "\n",
    "### Separating Hyperplane\n",
    "\n",
    "Let's take a look at the scatterplot of the iris dataset that could easily be used by a SVM: \n",
    "\n",
    "![alt text](https://github.com/lesley2958/regression/blob/master/log-scatter.png?raw=true \"Logo Title Text 1\")\n",
    "\n",
    "Just by looking at it, it's fairly obvious how the two classes can be easily separated. The line which separates the two classes is called the <b>separating hyperplane</b>.\n",
    "\n",
    "In this example, the hyperplane is just two-dimensional, but SVMs can work in any number of dimensions, which is why we refer to it as <i>hyperplane</i>.\n",
    "\n",
    "#### Optimal Separating Hyperplane\n",
    "\n",
    "Going off the scatter plot above, there are a number of separating hyperplanes. The job of the SVM is find the <i>optimal</i> one. \n",
    "\n",
    "To accomplish this, we choose the separating hyperplane that maximizes the distance from the datapoints in each category. This is so we have a hyperplane that generalizes well.\n",
    "\n",
    "### Margins\n",
    "\n",
    "Given a hyperplane, we can compute the distance between the hyperplane and the closest data point. With this value, we can double the value to get the <b>margin</b>. Inside the margin, there are no datapoints. \n",
    "\n",
    "The larger the margin, the greater the distance between the hyperplane and datapoint, which means we need to <b>maximize</b> the margin. \n",
    "\n",
    "![alt text](https://github.com/lesley2958/ml-classification/blob/master/margin.png?raw=true \"Logo Title Text 1\")\n",
    "\n",
    "### Equation\n",
    "\n",
    "Recall the equation of a hyperplane: w<sup>T</sup>x = 0. Here, `w` and `x` are vectors. If we combine this equation with `y = ax + b`, we get:\n",
    "\n",
    "![alt text](https://github.com/lesley2958/ml-classification/blob/master/wt.png?raw=true \"Logo Title Text 1\")\n",
    "\n",
    "This is because we can rewrite `y - ax - b = 0`. This then becomes:\n",
    "\n",
    "w<sup>T</sup>x = -b &Chi; (1) + (-a) &Chi; x + 1 &Chi; y\n",
    "\n",
    "This is just another way of writing: w<sup>T</sup>x = y - ax - b. We use this equation instead of the traditional `y = ax + b` because it's easier to use when we have more than 2 dimensions.  \n",
    "\n",
    "#### Example\n",
    "\n",
    "Let's take a look at an example scatter plot with the hyperlane graphed: \n",
    "\n",
    "![alt text](https://github.com/lesley2958/ml-classification/blob/master/ex1.png?raw=true \"Logo Title Text 1\")\n",
    "\n",
    "Here, the hyperplane is x<sub>2</sub> = -2x<sub>1</sub>. Let's turn this into the vector equivalent:\n",
    "\n",
    "![alt text](https://github.com/lesley2958/ml-classification/blob/master/vectex1.png?raw=true \"Logo Title Text 1\")\n",
    "\n",
    "Let's calculate the distance between point A and the hyperplane. We begin this process by projecting point A onto the hyperplane.\n",
    "\n",
    "![alt text](https://github.com/lesley2958/ml-classification/blob/master/projex1.png?raw=true \"Logo Title Text 1\")\n",
    "\n",
    "Point A is a vector from the origin to A. So if we project it onto the normal vector w: \n",
    "\n",
    "![alt text](https://github.com/lesley2958/ml-classification/blob/master/normex1.png?raw=true \"Logo Title Text 1\")\n",
    "\n",
    "This will get us the projected vector! With the points (3,4) and (2,1) [this came from w = (2,1)], we can compute `||p||`. Now, it'll take a few steps before we get there.\n",
    "\n",
    "We begin by computing `||w||`: \n",
    "\n",
    "`||w||` = &#8730;(2<sup>2</sup> + 1<sup>2</sup>) = &#8730;5. If we divide the coordinates by the magnitude of `||w||`, we can get the direction of w. This makes the vector u = (2/&#8730;5, 1/&#8730;5).\n",
    "\n",
    "Now, p is the orthogonal prhoojection of a onto w, so:\n",
    "\n",
    "![alt text](https://github.com/lesley2958/ml-classification/blob/master/orthproj.png?raw=true \"Logo Title Text 1\")\n",
    "\n",
    "#### Margin Computation\n",
    "\n",
    "Now that we have `||p||`, the distance between A and the hyperplane, the margin is defined by:\n",
    "\n",
    "margin = 2||p|| = 4&#8730;5. This is the margin of the hyperplane!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Iris Classification\n",
    "\n",
    "Let's perform the iris classification from earlier with a support vector machine model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn import svm, datasets\n",
    "iris = datasets.load_iris()\n",
    "X = iris.data[:, :2] \n",
    "y = iris.target"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can create an instance of SVM and fit the data. For this to happen, we need to declare a regularization parameter, C. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "C = 1.0 \n",
    "svc = svm.SVC(kernel='linear', C=1, gamma=1).fit(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Based off of this classifier, we can create a mesh graph:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x11116e4a8>"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1\n",
    "y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1\n",
    "h = (x_max / x_min)/100\n",
    "xx, yy = np.meshgrid(np.arange(x_min, x_max, h),\n",
    "np.arange(y_min, y_max, h))\n",
    "plt.subplot(1, 1, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we pull the prediction method on our data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "Z = svc.predict(np.c_[xx.ravel(), yy.ravel()])\n",
    "Z = Z.reshape(xx.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lastly, we visualize this with matplotlib:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvXmUJHd15/u5EZFb7dXd1Xu3Wi21NiQkhMxiMRhkzwy2\nefAOlmc0YmAk80bAmGd7xvNsNO89sDlj7BGzeLB8jPSMBW0Dki0wCOQFvLANBiOB0IJoJKTe9641\nq7IyMyLu+yOiqnOJzMqsyqz1fs7J05mRv/rFzeyquPG739+9V1QVwzAMwwBwVtoAwzAMY/VgTsEw\nDMOYx5yCYRiGMY85BcMwDGMecwqGYRjGPOYUDMMwjHnMKRjrGhHJi8j+Ju8fFpGfanGuO0Tk663O\nvVyIyOtE5PhK2wHtfZ/G6sScgtERROQ1IvINEZkQkVER+V8i8mMi8ioRmRaRvoSf+a6IvCd+nhaR\n3xCR5+Lxh0Xkj0Rk31LsUtU+VX0hPsfHROQ/L2W+RnMbxnrBnIKxZERkAPgC8HvAJmAX8JtAUVW/\nCRwHbq35mWuBa4BPxYceBt4E3A4MAtcDjwM/uQwfYc0gIt5anNtYO5hTMDrBFQCq+ilVDVS1oKpf\nVNUn4/c/Dry95mfeDvyFql6Iww3/FHizqn5bVX1VnVDV31fVj9aeTETuFJHPV7x+TkT+rOL1MRG5\nIX6uInK5iNwFvBX4tTjs8/mKKW8QkSfjVc5DIpJt5UPPzR0//5iI/L6IPCoiUyLyLRG5rGLsVSLy\npXgVdUhE/kXFez8br5omY9t/o+K9ffF53iEiR4G/a8GuXxKR74vI7vj1G0XkCREZj1dzL60Ye1hE\nfl1EngSmRcSLj/3HRt9Js/mMdYCq2sMeS3oAA8AFoov/TwPDNe/vAXxgT/zaIVo9/O/x698BvtLG\n+fYD4/E8O4EjwPGK98YAJ36twOXx848B/7lmrsPAP8bzbAKeBd7V4Lx3AF+veF079wXgFYAHfAJ4\nMH6vFzgG3Bm/9zLgPHBN/P7rgOviz/NS4EzFd7MvPs/BeJ5cgl2vq/j87wO+A4zEr18GnAVeCbjA\nv4k/c6bi8z8R/x/lFvpOWpzvp1b6d9Iei3/YSsFYMqo6CbyG6OL1/wHnROQREdkWv38M+DLwtvhH\nfhLIAI/GrzcDp9o43wvAFHAD8Frgr4GTInIV8BPA11Q1bOMjfFhVT6rqKPD5eN7F8Oeq+o+q6hM5\nhbl53ggcVtUHNFoFfRf4NPDz8ef5sqo+paqhRqurT8Wfo5LfUNVpVS00OLeIyH8H/hnwelU9Fx+/\nC7hPVb+l0Sru40AReFXN5z9WM3ej76SV+Yw1jDkFoyOo6rOqeoeq7gauJbrL/N2KIR/nolN4G9Fd\ndDl+fQHY0eYpv0J0h/za+PmXiS6kPxG/bofTFc9ngDpRfInzXAK8Mg63jIvIOFEoazuAiLxSRP5e\nRM6JyATwLmBLzdzHFjj3ENEF+7dVdaLi+CXAr9acew/R/0+zuZt9loXmM9Yw5hSMjqOqPyAKp1xb\ncfgzwG4ReT3wFiInMcffAK+Yi4G3yJxT+Cfx86+wsFNYqZLAx4jCY0MVjz5VfXf8/ieBR4jCa4PA\nRwCpmWMh28eIViQPiMjNNef+rZpz96jqpyrGtPO9tDKfsYYxp2AsmVhE/dUKYXMP8K+Ab86NUdVp\noh1GDwBHVPWxivf+BvgS8Oci8vJY7OwXkXeJyC80OO1XgNcTxcGPA18D3kAUivpug585Q6Q5LDdf\nAK4QkbeJSCp+/JiIXB2/3w+MquqsiLyCaAdW26jql4lWIJ+J54EonPeueDUiItIbC9v9i/wsnZ7P\nWGWYUzA6wRSR8PgtEZkmcgZPA79aM+7jROGHgwlz3Ar8BfAQMBH//E1Eq4g6VPWHQJ7IGczpGi8A\n/0tVgwZ2fhS4Jg57fLblT7dEVHWKKNZ/G3CSKDTzX4h0FYB/B3xARKaIhOI/XcK5vgT8AvB5Ebkx\ndr7/FriXaDXxPJFgvtj5OzqfsfoQVWuyYxiGYUTYSsEwDMOYx5yCYRiGMY85BcMwDGMecwqGYRjG\nPGuuANbg8Gbdtqud7eyGYawlnMOHKKVTpDJr7vK0qvnh6fHzqjqy0Lg1961v27Wb33v4iytthmEY\nXeLVj/06j37hCDsv27zSpqwrXvfBTx9pZZyFjwzDWBWU04+y49lnePQLR0BqE7qN5WLNrRQMw1h/\n3PvM7dz+ZyHPAI7nsv2SoZU2acNiTsEwjBWl746buX3fHgALGa0CLHxkGMaK8erHfp2j+/aAmENY\nLdhKwTCMFaFSP9i5f9NKm2PEmFMwDGNZKacf5du/5nGV94DpB6sQcwqGYSwrw7d/kKv27bFw0SrF\nNAXDMJaV/l+/v76FkLFqsJWCYRjLQm3YyFidmFMwDKPrzOUhXOXZLqPVjjkFwzC6iuUhrC1MUzAM\no2tYHsLaw1YKhmF0BctDWJuYUzAMo6NYHsLaxpyCYRgd5bXf+CpTnpW+Xqt0VVMQkcMi8pSIPCEi\njyW8LyLyYRF5XkSeFJEbu2mPYRjd53Dv2y0PYQ2zHCuF16vq+Qbv/TRwIH68EviD+F/DMNYYloew\nPljp8NGbgYOqqsA3RWRIRHao6qkVtsswjDawPIT1Q7e3pCrwRRF5XETuSnh/F3Cs4vXx+FgVInKX\niDwmIo9NjI12yVTDMBZD3x03c/ufhYA5hPVAt53Ca1T1RqIw0S+KyGsXM4mq3q+qN6nqTYPDtrXN\nMFYLO559xvIQ1hldDR+p6on437Mi8ufAK4CvVgw5AeypeL07PmYYxiqmUj+wPIT1Rdecgoj0Ao6q\nTsXP/xnwgZphjwDvEZEHiQTmCdMTDGN1Y/rB+qabK4VtwJ+LyNx5PqmqfyUi7wJQ1Y8AfwH8DPA8\nMAPc2UV7DMPoAJ8o7OFRLA9hvdI1p6CqLwDXJxz/SMVzBX6xWzYYhtFZyulH+W+fvYWrUg+stClG\nl1jpLamGYawRqsJG+22VsF4xp2AYxoJY+euNg5XONgyjKVb+emNhKwXDMBpi5a83HuYUDMOow8pf\nb1zMKRiGUYXlIWxsTFMwDKOK33L+b8AcwkbFnIJhGPOU04/y6U8cB7GGCBsVCx8ZhgFYHoIRYU7B\nMAzLQzDmsfCRYWxwLA/BqMRWCoaxgbE8BKMWcwqGsQGxPASjEeYUDGODYXkIRjNMUzCMDYblIRjN\nMKdgGBuIe5+53fIQjKZ03SmIiCsi3xWRLyS8d4eInBORJ+LH/9Ftewxjo9J3x81x2OgBE5WNhiyH\npvDLwLPAQIP3H1LV9yyDHYaxYXn1Y7/Oo3PbTi0xzWhCV1cKIrIb+FngD7t5HsMwGlO97dQcgtGc\nbq8Ufhf4NaC/yZifE5HXAj8E/r2qHqsdICJ3AXcBbN25uxt2Gsa6w7adGouha05BRN4InFXVx0Xk\ndQ2GfR74lKoWReSdwMeBW2oHqer9wP0AV1x7vXbJZGOdoqrM+iHFIEQV0q6QS7k461hstW2nxmLp\nZvjoZuBNInIYeBC4RUT+pHKAql5Q1WL88g+Bl3fRHmODki8FFPyQUEGBYqBMzvqors/7izlBGcwh\nGO3TNaegqner6m5V3QfcBvydqv7ryjEisqPi5ZuIBGnD6BhBqJTD+ot/CBT9cPkN6jLl9KN8cff7\nrI6RsWiWPaNZRD4APKaqjwC/JCJvAnxgFLhjue0x1jd+gkOYf28drhTu++4nuN0LcVx3pU0x1ijL\n4hRU9cvAl+Pn76s4fjdw93LYYGxMnCaywXrTFCrLX5uobCwWy2g21jWeIw0dQ8ZdP7/+Vv7a6BTr\n56/CMBIQEQYyHl6FZ3AE+tMubrNlxBrC8hCMTmJVUo11jxM7hjDWEITIWax1LA/B6AbmFIwNw3rS\nECwPwegW5hQMY41h/ZSNbmKagrHmUFWmSz6jhTKjhTJTRZ+gydbT9YTlIRjdxlYKxppjqhRU5R+U\nQ2Wy6DOY9dZViCgJy0Mwuo05BWNN4YeamJCmRBnKudT6vVhaHoKxHFj4yFhTNAsTNcteXutYHoKx\nXNhKwVh25grRLWZbaLPcAm+d5B1UUk4/yt7v7avIQ7COaUZ3MadgLBuqynQ5oBRETsEVoTfttnUx\n9xzBE6mrWyRAxltfC9+5bafPgOUhGMuGOQVj2ZgqBlUX80CVqUUIxP0Zt8q5eI7Qu876I9i2U2Ol\nMKdgLAt+qIlVSRWY9UN62hCIRYS+tLekMNRqxvopGyvJ+lpvG6uWZgLxYnMMRGTdOYR7n7md//bZ\nW6yOkbFimFMwloXVLBCr6nxdpJXmvR86wlXeAyYoGytG152CiLgi8l0R+ULCexkReUhEnheRb4nI\nvm7bY6wMniOJF/+VFIjDWNMYm/UZn/WZmC2v6LbWym2nhrFSLMdf4y/TuM3mO4AxVb0c+B/Af1kG\ne4wVoj/tknVl/pqXcqLqpSshEGvsECpbdQYKk0V/RVYNVv7aWC10VWgWkd3AzwK/BfyHhCFvBn4j\nfv4wcK+IiK7XjuobHBGhJ+3Rs9KGEAnfQYPfsnaF76Vg5a+N1Ua3dx/9LvBrQH+D93cBxwBU1ReR\nCWAzcL5ykIjcBdwFsHXn7q4Za2wcmkWJlmulYOWvjdVI18JHIvJG4KyqPr7UuVT1flW9SVVvGhw2\nAW6j4Ych5SAkDMOOzdlM+HaXIZzVd8fN3P5n0ecxh2CsJrqpKdwMvElEDgMPAreIyJ/UjDkB7AEQ\nEQ8YBC500SZjDeGHIaOFMpPFgKlSwHgxoFAOOjK35wipBsJ3tsvCt9UxMlYzXfvtV9W7VXW3qu4D\nbgP+TlX/dc2wR4B/Ez+/NR5jeoJBGIZMFusdQMGPVg2doC/tkvWcqD0nkHaFwazX1dyHe5+53QRl\nY1Wz7HsBReQDIvKm+OVHgc0i8jyREP3e5bbHWJ0UG6nAwEypM6sFEaEn5TKcSzGcS9GX7v5OqPd+\n6AiA5SEYq5ZlKXOhql8Gvhw/f1/F8Vng55fDBmNt0Uzs7ZyysLxUlq8wjNWK1T4yukbRDwgV0g64\nbXYKSzkOxSB5ReAu8qIahIrGP7/c5TGq8xBsldANgnIJVHFS6QX/f1WV0C+3PH4jYU7B6DhFP2C6\nfPF+vgCkHKU/0/qvW9pzkHJA0nqhL92egwlCJV/y5/MSBOhNu6Td7kdPLQ+h+wSlIjNnj0YXeUAc\nl56tu/GyvR0Zv9Ewp2B0lCCodghzlENlthyQbSMpbDDjki8F+PHF3AH6Mi6O0/rFXFWZKvlVeQkK\n5EsBgxlpujV1qVgeQvdRDZk+9SIaXlxVauAzffoI/bsP4Hip1sfvuQLHtUuifQNGRyk0E4j9sC2n\n4DgOA9ml3c37oTZMVCv6AT3p7vwJWD+E5cGfyZO4YVGhlB8nOzTS+vipcbJDW7pk6drBqqQaHSVc\nZX2Sm2Yud+mcO559xvIQlokw8CExyKhoHB6qHh/pCInjg/rxGxFbKRgdJeM5+AnhI2h+B6Ia1SJS\nVTznYp+EueOguIvon9CsLHeqjTBUq8yFjExQXh68TINKWiJ4ub7k8UK9HxExTSHGVgpGR8l4jcND\n/enkX7cgVCaKPpNFP8pcnvUp+kHV8cnixePt4DpCOmG7kiMkHl8qnyjEISNzCMuCm8ni5fqh6mZB\ncFIZvJ76kmtuJhc5ixbHb0QWXCmISAb4OWBf5XhV/UD3zDLWKk0T0qXeKagqk0W/6sZNIRarq1cc\nc8c9x2lLIO5NuaQcZdaPdjOlXYec53R8G2LltlNj+ejZupvS1BilqTFQJdU3SGZgc8P/356te9oa\nv9FoJXz0OWACeBwodtccY63TrEnNrB/QWyPs+nHuQDskzdMMESHjSdea+di205VFRMgMbCIz0Nrq\nrN3xG41W/rJ2q+obum6JsS5oXpK6vfGNWE1Stm07NdYbrdw6fUNEruu6JUZXUFXKQUgpCJuHdjo0\nf6pJnD4pWazZ+EZ0QyBeDFb+uh5VxS9MUy7k0Q6WOjeWj4YrBRF5iuimzAPuFJEXiMJHAqiqvnR5\nTDQWix8q+aJPyMUNFz2e01auwELzT9XoAT2eQ9YVZmvyFdwGwq4j0nC8J1Csua40mme5qaxjZNVO\nI/ziDDOnj1bcfCi5LbtI9w2uqF1GezQLH71x2awwOs5cD+KLf54RM36I5zpNt2ouZv45ZvyQ/rSL\n50ZtLVWVtOtEJaobCHm5lIvnat14gFRYf3w5BMFy+lHu++4n5l+/5yWfnH9udYzq0TBk+vQRqFkd\nFM6fwM3kcFPpFbLMaJeGTkFVjwCIyB+r6tsq3xORPwbelviDxqqgmYA76wf0LTGTt9n8xSCkL+21\nXFtIJNo2mjS+0fFuUk4/yvDtH+RuwBGPQH3GPvkoqdLP0nfHzTyzb48JyjX4hakGOWRKaWqM3KZt\ny26TsTha+Wt7SeULEXGBl3fHHKNTNFMPOiEtdHv+leLeZ25n+PYP4orHZcNXcOnQfgS477ufoJx+\nlOve8xoAcwg1aBDQ8Lci7Ez/C2N5aKYp3A38JyAnIpNzh4EScP8y2GYsgWbhoU7ceS80fxiG8zpB\n2hW8WBxWVcrxliPPkUU3tenUPJXc+8ztvPdDR3DF49Kh/fPHHfF474eO8MXdHlOe5SEkkZQ9DESZ\nwpYUtqZoFj76beC3ReS3VfXudicWkSzwVSATn+dhVX1/zZg7gA8R9WoGuFdV/7Ddcxn1OCLkPIeC\nXx3jdSU5w3cx86ccSKpooRoyXrx41zjrgycBPWmXqWJ1Oeyc55BrU/j2w7Aj88wxFy56L3D58BV1\n7186tJ+ht+/l6CMPmKjcACeVJt2/idLU6MWloghepqexwzBWJc1WCjfGT/+s4vk8qvqdBeYuAreo\nal5EUsDXReQvVfWbNeMeUtX3tGW10RK5lIvnCMUgjJrduA4Zt/36QUmEYZjoEABm/Powgq807Lmc\nqlhJLEQkcNf3WYjmaV9An3MIAlyW4BCqTw4nXxg1cbkB2U3b8HK9VZnCqd5ByxReYzRTG/9b/G8W\nuAn4HlH46KXAY8Crm02s0b60fPwyFT/WcLR5bZJyHVJdEGprt5AuaS4/pK9BXaRays0Ebj/Aa0NA\nbxQuSmL84FGue+f7eerB3zTH0AARIdXTT8rCRWuahn+Jqvp6VX09cAq4UVVvUtWXAy/jYrinKSLi\nisgTwFngS6r6rYRhPyciT4rIwyKyp8E8d4nIYyLy2MTYaCunNtYpzUTsdtxUOw5hjpH77uG6296/\n8EDDWMO0clt1pao+NfdCVZ8WkatbmVxVA+AGERkC/lxErlXVpyuGfB74lKoWReSdwMeBWxLmuZ9Y\n3L7i2utttdEGc4KsaiTILlRILgxDCn6IApkmq4y0K8z6nbGxmfBdaU/Wc6IM6AZl71sV0PvuuLmh\nfrAQgs+e/YN4KUUdl1Jtdl0NGgaUZ/KgIV6ub74TWKPjhrHStOIUnhSRPwT+JH79VuDJdk6iquMi\n8vfAG4CnK45fqBj2h8A97cxrNKcchEyVquP4Wc+hp4EgWygHVcJ0KQhwJWAwW3/B8hwHIbmHclK5\neoC0A6WwfmyqgaNKsscTEgV0z5GG88zRln6QQGpvjit7HkF/6jJCv4gjwtEXJzl+OJ843i/kmT5z\nLH4VfSOZoRG8TI7pM0ep/KYyQyN1XcIMYyVo5dbqTuAZ4Jfjx/fjY00RkZF4hYCI5IB/CvygZsyO\nipdvAp5tzWxjIVSVfKle2J31Q8pB/d3t3B15LYFGF+ek8Y2WbI2O1zqEubFJlVUb2eNrdCkdyLhx\nYpvQl3bpT7tNBc2lOgQ8YeDnduGkHVzxSaVcXM9h7/4B+gbqnaaGYeQQNIwfCqoUx89Fmb+qdceD\nYqF9uwyjwyy4UlDVWeB/xI922AF8PE52c4A/VdUviMgHgMdU9RHgl0TkTYAPjAJ3tHkOowFNBdkg\nrAsLNROOi35Yt92zk0Jzu/bM+iFDqVTL4vRi9INa0pf2JgoaIsK2Hb3kJ8erjvuF5NVDQ1FElVJ+\nnFwmtyj7DKNTNNuS+qeq+i8qCuNVsVBBPFV9kkiUrj3+vorndwNt50AYS6PdjOPVJuJ0W1BOQrzk\nVYjjCE5C3odq+xVCraqosRpotlL45fhfK4y3BmkWX09qNpNpIhwnje+20Nxs/lab5SxFUK6ldHga\nSbj4l8sBzz11DtzqP6UoYSvBfYkke2VxSPV2rpqoX5qlNH4e1ZD0wDCpnG0TNVqj2ZbUU/HTnwLS\nqnqk8rE85hmLRUToSSX0L2ggyLqOQ8JwBMgmXAw9x6FRYnTSPhoHyCbo243s8RyHVML8jeyppJx+\nlL47bkbojEMA0EJI/m/PoeUQDRRVJSyFzOgeTh6e4PSR6vCR43pkN22vLokRl3zo3Vyf45DpyeDl\nOtM4fub8KaZP/Ijy9AT+zBQzp4+SP/liR+Y21j+t7D7aC9wnIvuIWnJ+Ffiaqj7RRbuMDpD1XFKO\nQzHuTZxyHVJO44zm/kyKkh9SKMe9jOPeCE5CtrGq0ijsn7RjNARSrkvGk9btyVbbk/GijOwke+bP\nvVRBuQmzT0xQPl4g85IBnLRD8bk85cPPc6jvTq72DtaNzwxswsv2UJoaBw1J9Q6QyuWYPPrDertn\nZxlI+ZT8pW1NDf0S5an6XJ6gOENxaoxM//CS5jfWP60Ize+H+R1E/xb4v4DfBTrTqcXoKq4j9LSR\n5Zv2HNIthGfKi+ijOVdSuxv2QOf0g2YE50vMfOV81bFb37qXpx4MEjOd3XSW3Obt868zXgFxBGo2\ndIWBorOT4C2tttLs+IWG75UmLphTMBZkwb82Efl/ROQvgS8ClwP/EdjdbcMMox2WwyE04mKmcyuO\nUpsM64Skv9q2BRhrjVZu2d5CtGX0UeArwD+oarGrVhkNCVXjfshRf2NXmhe4azTeD0MK5bnM5Siu\n3ywsU8tCiWJJdKtZTicF5Tmcfo/MVf1ISij9aBr/TPNf+QOFB3nYvxPnyMGmvRZKQSaxV7bruTjZ\nATwNcMMZ0BD1eubDSaFfojw9iYYhqZ5+3AZbV9MDmyhPjSW/1z9MWC5RnplEQyXV0zc/T6PjmazL\nlm05HEcYPTfLdL5BOnlMWI7tVI3tzDYdb6w+Wgkf3SgiA8DNRAlo94vIWVV9TdetM6ooBWFVQlrB\nj3bp9KaSE7eKfsB0RSnTufEaKuWK65IfKg4wkKFlxyAiDTOUsy4UasIjaXfhjON26ZZ+kLmqj/6f\n2R59GEfoedUmZp+aJP+lsw1/ZvzgUW59520Nw0hzhOqyZd8lnD98BFVFQ8X1HIa3b4GwzNjhH4FC\nqCGO47Bp11by0w6zo6fiRYBSnDhPum+Y7Obtdf/vXjqL1zuIPz1RddxJpVGEqRPPV8xzjnT/ME46\ny+yFU3XHL7n2Mi6/eggQRGDPpf2cOjbNi89Vzz1HcWosnif65Yrm2VQVPjNWPws6BRG5FvgnwE8Q\nVUs9Bnyty3YZNTTKUC4FStrVuh4JqlrlECrHJxESJYz1tHgzr6oNM5Rd12XQk3iFovMlrTtZQnku\nXNRphyBph/6f2Y5UbsVyhMy1AxQPTVE+2jjreOS+e6JKqg99oOk5imEPw/uuwA2no3yGVC9+4DJ6\n+BBhRbZ5GISMnjhbdQyIE93GSPUN4GXrdyz1bt2NXxhmduxcvCV1E6mePqaO/bB6O2zcKrNui2x8\nfPMmwa1a3Qk79vRy/myBqYlS1Y+EgV/lEC7OM0qqdwAv29P0OzFWD62Ej36HaMfRh4Fvq2rz9aPR\nFcqhNqwpVPTDutDMooRgP2xYFynJnkaUYkE553RnL0I39YPUpT1o/F1XIikhc01/U6cwjyqnj4w3\nDSP5oYvPQLQa8SHrzSQ6zTqHUHGOcn4i0SkAeLle+iq2uJby44njmmVYn/zRKbbsrBamHRFGtuXq\nnII/kyex6pUq5ekJcwpriFbCR5a8tkpodBneSD1MuqEftITSkoY7v1p48DcXdAzV8wt1nqjjNLqt\naDK8lWPzmMi9HuiO8md0nGbx+CQBdzHx+4znEITKTDlguuRTjsM/ALPlgLFCmdFCmXzRx2lSxiHT\nBUG5Gwlpied5cTraMlqLrxSfmWppjpH77uFQsGDNyCqKYQZNWH0l2gIgQqqvscPp7Q3pz0zRl5pg\nsD/AyzbJsG7Arst31B0LQ+Xs6Zm6415P4x7Nzew0Vh/mFNYIIlE10FoaCbgiQiYh81eISljX4gCi\nykTRZ9YPKQbKVCkgXwoYL5SZ8S9WRS2FymRZ6UmYP+NK2y0xF6KbCWm1aEmZfOQUWg4JyyEahGg5\npPDEOOXj7VUxDf2gLtO54XnVoWdTfQ5BOpclu2VXdPGeu4CLkI4T45IY6i1w+tlDHD90lBM/PM6J\n7z9HrzdBdvPOunkyA5vJ9NeHoFzP5fRpnyAI44cSBCEnjuTJT9ZHkB3XI7claf4teFbkb03RehaR\nseKkXYehrFRsMW3ck1hVKSaIylGmsktWqNiS6pB2hbGEYkPNtIOywlDWa8mexbIS+Qel56e58JEX\nyVzRh6QdSj+aJrhQWvgHK3h3/kHOvfP9PPXQb7Y0Pgx88ufqM5GLM7P09nv077mC8vRk3JSnHzed\nSZwnnVaOPXu4SosI/IBzx86x65oDjOUOUJ6ZBFW8XD+eB+NHztfNE/gBYyfO8O2JMlu25nDcaEtq\nYaZxwat03xBetrdq/kZ2GquXZlVSP0+zNBvVN3XFIqMpjghZb2EBt9nFvOiH9Gc8+jMXlwylRoLm\nAudo1Z7FsJIJaToTMPtE8tbLVjlQeJCnoCVtwS/kk0P+qpTyE/SM9JEZWLgvdG+2zGhCSCjwA/yZ\nSRxvmMzAxazpcPpMw7kK46Nk00OcOj694HnncLxU1fzG2qPZSuG/LpsVxrKyFoTpFROUO8j4waMc\n6ruTK/WBBR1DtPMoWQhuZyuvNhGsxXHqymvQZGXXTjKjsX5o6BRU9SvLaYhRjR/qfOG4dEXhuEbH\na0k50nCvSZIQ3Gx8I9Ju6/a0Sq1+4G3NkH3pIGSE0qE8pR9NNzUyc3U/Pa8ahpRD8dkpZr4W1QLy\ntmXIXlf4OmvLAAAgAElEQVQxz/PN735nR5TzwzMEGrKp2EPfUQdBmHaLnE1PEYqyqdzDULkHQarH\nz/bQdywafzGM1Dx3oVmp7VT/MIXRs5QmLwCK19NPdvNOXNelbyDFtp29uK5w7kyByXFNbgbkCJLp\nx88XKE+NoRqS6h0klR0CziV/l0ObG37VfnGG8tT4/Dxeri/6/ZydoZyvP94p/Nn4vCScN+G40T6t\nJK8dAH4buAaYz1lX1abreRHJEuU3ZOLzPDxXXK9iTAY4CLwcuAD8S1U93N5HWH/MlgNmansTO0LK\ngYKvdceTWlHOCdN1PZpdqetyNjfeERIrnzZyFq7AZPFijLmZPa1Q6xCyNw7S97oRcEBch8wV/ZSP\nzTD56ZOJBvW/eTuZK/vnP4/76k1krx9k5h8u0PcTI+AK4kg0z9F4ngTOX17ixfFThOdDUDjnuQxf\nPkDuGJzIjRMS9QQdTU8zUM4yvH2IIxNnqsZvunyQS58fQOZv25vnLojj0rN1DzNnj1UdzwyOMHPm\nOBpc1DT86Uny05Nc+U9uYv8Vw9H/nSNs2Zpj7MIsPyqNcO5IdViop7+Hs8fOU5o4P+80ytOTeLk+\nBndsZ+LU6arxvcODqJfcg2F27BzFiXM180T6QbFu/n56tu7uyAV6duxs3fypngHES1OarD+eG9ll\njmERtLI+fAD4A6L6R68nuoj/SQs/VwRuUdXrgRuAN4jIq2rGvAMYU9XLidp9/pdWDV+vhKpVDmEO\nP9Qqh1B5vFGWcsp1GM569KZcelIOg5nGFUr9MGxYCrvR3WIjexaTOHfvM7czfPsHccXjsuErkJxD\n3+tHkJSDxE7MSTuk9/SQPlC//dHZlCJzZT9SUQtKRHB6XPpu2RrNE4dKnLRDam8P6cvrd934PcqL\nY6cioTb+GIEfMDYxwbGeMULR+fBMKMpkapbD46frxo9OTJDfE/0/jtx3D4f8OwmD5ruRUj39DOy9\nktyWneQ2bad/9wGcTLbKIVQydeI4ruvgxJ/L9RxyObhwvP7Ov5AvUBo/R23GsV/IUw4zDFxyBQNb\nt9A/spmhSy7DG0queRn65SqHMD/PzCTFxPmnCGZb1yQaEZZLVQ5hbv7y9ESVo5s/PjNJMFu/ddZY\nmFacQk5V/xaQuMHObwA/u9APacRco9pU/Ki9WrwZ+Hj8/GHgJ2WDu/akJvYL0UwkFhEynkPWc3Gb\nxI8bOZbF0K5onSQop/b2oAk2Sdohc0W9U8jdkNy1TCQ5xu6knflVRSXTW4PE3IAgwVFD5BiSCtwF\nfsBo9uJF6e63fINDwTsS56iy13FJ9w2RHtiE46UojdfvDJrjzLH6i//o6bG2M6P9mUnESSG923D6\ntqNO4yJ2DXtPN0I12jW1RJqfN+F3N3YMRvu0siW1KCIO8JyIvAc4ATTIVKlGRFyixjyXA7+vqt+q\nGbKLqJYSquqLyASwGThfM89dwF0AW3eu76rdi/GInXCjK+WJGwrK5WQnpaGiCUWXtNRgvMalqms+\nYKN5nLBzmcWe1OzK0nB+tdBqprM0EXudhDCg6zjt2y9tCMri0HZmdCcEa2cR523ncxnztPKt/TLQ\nA/wSUez/bcC/aWVyVQ1U9Qai/guviIvrtY2q3q+qN6nqTYPDC2/LW8t4jrT9N92JDOKkRLfFkm2l\nSc8CGcqlIzPJf/+BMvtU/R3gzD8ml4ue+5nkeeq3nPaecEj6H3BcByfpuNYWjbs4fvNYjtSeHP1v\n3E6Y9/mVvY/yw+AOwiDEL0wzc/YY06ePUMqPz682BoczXHntMNdcv5mR7Tmym+uziufYf90ldcc2\n796S+L05rpN89yBCur/1jONUTwNBPJ4rcf6+YfxCnumzx5g+fZRSfiJxddX8vP2LOK9lUi+GVmof\nfRsgXi38kqq2lutfPce4iPw98Abg6Yq3TgB7gOMi4gGDRILzhkVE6M+4TBWDqj+BnOcQJiSkCXQk\nYcxxHHpT9ZVVPYGedLI9KVcSj3sL3Bm2lKEcKBN/doLBn981f+crrjD9tQv4p2brx5dCis/lycR6\ng4hEfZRnAorP5cldP1h1PJjyE3skOIFwtbOLZ1MnAEU1Wm1cOrSD/r4+nj75o+g40WrjwIF99DlZ\nvvfcD6uOX3HtfjaVB+l5xSZIRTpH6pIe/r0c44sPTDJ9ZmY+Du7PTlOaGufqH7+OPfv6cdxo/NCm\nDNt29vLtiQuUalpsSipD/9bt8xfXuc/l+7D50v1cOPxCNDC2f2TfHvLTwvSZoxWzKNlN23HTrfc8\nEMeld9teps9UCuLxPKl04vFSfozS5GjF581Tzo/Ts21vy0Jw4/PuwEmlmEk4bolzi6OV3Uc3EYnN\n/fHrCeAXVPXxBX5uBCjHDiFH1IuhVkh+hGjV8Q/ArcDfabu3EOsQz4kyl8uhzjfHARhPyDhWIj0g\n4y3dMWQ8l5QjzMaN6TMVF/hae5z4j7nR8Ua0k5Dmn5zlwr0vkN7XE2UWH5lBZ+rLhwNIr0tmf2/V\nRUZEkJRD7rrBuuNOn0f6ij5Kh+pj1T2nHV7m7mZ6V0joKL1nPNKzLpveuZcd7OHc8QsEfsCWXZtJ\nuR7iwtZX7Uw4LkjFCsxJO7hTx0gxWyeMBsUCGWcW17uojbieQ/9gml1XX8q5E5uZHTuNhiGZoRF6\nhwYY2d5T97myWZfeoQFKu68g7ZUQVUqaYTZ08HIwsPdK/EIeVcXL9eK47Rc18HJ9DeepPU4Yxj0c\nagTo2Wn8Qj5eAXT+vIv5XEZEK9/cHwH/TlW/BiAiryFyEi9d4Od2AB+PdQUH+FNV/YKIfAB4TFUf\nAT4K/LGIPA+MArct8nOsO0SkqkdCM/G2FIRkWuxjvBCO4yT2VKi1Z6HjSczpB21lKAca5SYsQDoW\npqXmN9pJO4mCtZN2yBxIdgoQrRj6j17UBFLX9KABuBmX7fu2Vo3VIMT1ko/XBvgvnBrDc5S6/04N\nOX3kLDv2b6s67HkOm0dyjJ6bpXfb3vnjQ5vmOrhVz+96DptHsoyen6Xo168AxHFI9Q4kfuZ2aDRP\n7fHSdH3pDiAWuKfacgrtnNdYPK04hWDOIQCo6tdFpHEBlIvjngRelnD8fRXPZ4Gfb9HWDU2zy+5q\n36+1HAXttJi8gkiqPDp3PKxtD9d0/pCkmLaGmrz5JUhQtwGvSUmQVMJ24TBU/IRmSQ13Q4VKucF7\nK4Lj0lAg7lK/DWNptOIUviIi9wGfIvqf/ZfAl0XkRgBV/U4X7TNivCYZx60IuyvFclU4LR2eQX3l\nzOmzHP7+MYKyz47929lz2U4ccRC35gIUKIXvjTOWmuFMepJQQjaVe9la7MdJ2H9ROjwTtaerJdDE\nFQqhEgYh546NcuTZyJ6dl+1g577tDR3Vjst38Ow//pCjzx5Hw5CRPVt4yc3XcPrkNJtGsuzY3Yfr\nCmdPz3D2VHJmt6py5mTj/fmbR7Js392H60TznDk53bDPTidI9fSTWFtWhFTfIOXpSYqx3pDqGyTd\nP4S0uWtI5/IkJqMucun+IVJ9Q5a4tkhkoRB+LBA3QlX1ls6a1Jwrrr1ef+/hLy7nKVcNfqhMFf2q\na0GP55BtsVvacrPcBe2Ob57gtIwTxLEZx3XocTK8+qdvIr0tN3+RUFUIlO/8wT/GDiH6Rh0VckGK\na6Z2Ju408rZlIuHbE9BI+J764llyrxzC25ypml8D5YmDj3GGifm7esd16HUypPLCaHqmaiEhXgrH\ndQiK1eK3OA4vff0r2HlJP27s/IMgZHqqzI9+OM61N2yJ8io02oH53PfHOZfQ7wBg/5WDbN/Ze3Ee\nPyQ/VebJx891tT+OPzvDzJmjFTuOlNzmnVGpjPz4Rb1BBDeTo3f7vrYu6DPnTlCenljyPOud133w\n04+r6k0LjWtl99HrO2OSsVQ8RxjKevhhtMvFcxYWdleK5XYIRfE5GYyiFV9HGITMBLOcmxpn9/aL\nvQdEhJnCLGeyk4QVN0WhKAW3zGhqmi3l+lQc/0yRC7//AqndOSTlUD4+g7slU+UQ5uefKXA6GK+e\nPwjJ+wVIU583EfgECRniGoZMnDnJnsuunj/mug49fSmyGY9vfvUUg0MZHFeYGCsSNkhCzOZctu+K\nVhrz83gOvf0pNo9kuXA2YUdXh/CyPfTvvZJgdjoSgrO9hH6ZwoWTiYK7PzPVsj4QlIrVDqFynjaF\nbCNiwXWaiGwTkY+KyF/Gr68RkYVTM42uIBLVLkq7zqp1CH133Mx7P3SEy4evWLaS11PebGJ+QYhy\n5sjZuuNjZ8bmy2dUjRdlPNWkPIJC+ViB0gvTaEnJXpt88Ro7PZ44v0qDm/ImK/YzR+ozlz3PYXhL\nFhQmxoqMnZ9t6BAgyn9IOofnOWza3PqW1MUiIni5PlI9/YjjNC59EZfeaBW/2TwzbWZfG0BryWsf\nA/4a2Bm//iHwK90yyFi7LFfLzCQ8dRqGQDI99fvVvXQqebBCSlsPx4X55D0XXqbBIlyTNw00i+Km\nMvW2hqFSKrUulJfLYeI5onmWX5iONJ7kmxppYzupOI3mkXodyWiJVr79Lar6pyJyN8yXo2j9t9HY\nELQqKGeu7id74xBOWij+IE/h8bGGJSraYdDP4eIQ1jQMcFTYe+VuVLUqxLNl1yYkIVwDsLXYOOSQ\nuaaf3I1DSEqYfXaKme+O0fOazXXzj+zejBPEydSV2gHgqENAWB9C0uSdZAduvKxuflXlzInWC82N\nnZ+NtI6keU4uvWBdu3i5/uRNSW1mIqd6+ikkzgPp/vr2psbCtLJSmBaRzcRfe1zpdGktqYx1RW2F\n00b0/uQIfW/YRnp3Dm9rltyrNzH0tr2RcLtEBOHG664h15fF9Vy8lIvruVz3iqsIvzIFIfMXRVVl\n9vEJUr31d+DiQMpLvlfq+2db6f/n20jtiuzv/fHNDP+rvUx+7uR85vD8/I9NcNX4dtKhG5XCCAVH\nhcumRxjwa1YuCtlcmkndm7DYcThxwqc4G+D7IX45xPdDDj01ymw7W2oVnnz8PMVi9Tw/aHOeTiGO\nQ9/2fdGqQJz5R8/WPTipdGfm8RqsBo2mtLJS+A9EmceXicj/AkaIso+NDUw5/SgA9333Ey0Jyk6/\nR+6GQaRi+6yTcmAgRfaafmafXFpFS2fAY9Mrt3PLj+9g8sIUfjlgaGQACYX8qbOc/6/Pkbm6H8k6\nzD4zxfSmMoVifZkLVTi5b4a9P6wWmp2hFNlrB5DURfsl5eAMphDP4fyHniNzTT+SieanFNJDmhsm\n9zDjlghE6fMzlByfH6XOV68SBMpln345Xh8IEWXi7Cjf/npIX38KxxWmJkqL2kY6ky/z7a+dpm8g\nheMsfp5O4WZy9O+5gqA0CxriZnJtb0etn0dxM9lFzWNEtLL76Dsi8hPAlUS/yodUtdx1y4xVy9zO\nIgHuprX8g9SubMOM49T+3iU7hdSuHBoojucwuKVa/E1f2kvxqUmKz14s2zU2UGjUcIyx6Un21hQC\nTu3KoqHWXbSdtBPN/8wUxe/XlwUThN7g4spgyptFlKpdUjBX2jphxRQLr+m+IfJTnfmzy0+unj9f\nEcHL5FbNPEYTpyAiPwYcU9XTsY7wcuDngCMi8huq2iB/3VjPLHaraTjdIOM4UMIOXKTC6WTBt9H8\n6bBxaCGdShB2G9nvhwRt2J8KG4ifGvdE1vrziGthEGP5aLZSuA/4KQAReS3wO8D/SdRF7X4shLTh\naNj7oAXKxwpoIUQrOqABECqz321fospeP0ju5ZHgWzyUZ+Zbo+hs8vyF702QvWHwokD8gzwj/3ie\nF3Mk7ljaU9pcb/+RGbSUML9C4XsTnE5PcDY7RYiyqdTDzuIQXsIupnlBXIOqhYGD4ASK79SopiJk\nTDBtiqpSnLwQ955W0r2DZIa2xDuTjHZpFnhzK1YD/xK4X1U/rar/L1HTHGOD0KmtpuMPHiM4X0TL\nIWExICwETH7uFMFYeyuFvp/ZRu8tI3gjGdyhNLmXDzH89r2MP3yc4ELp4vwzAZOfPUXPj2+i7/UX\nx/fcNMSmt+3jxhuvreuydvlL99N3NuFiojD+ydr5fSY/e5Lnyic51jNGwS1TdH1OZyd5uv8kYUJd\nDEG4ZmoH2TCFo5H47IUOB/JbecnUTlL926Jy3Ujct3lvW8LrRmTmzFGKY2cJyyXUL1OcvED+5Iuo\nrqIaUGuIZisFV0Q8VfWBnyTufNbCzxnriE7WLgonfMYeOIozlEJSQnC+1HZ5BWcoRfaq/mrB13OQ\nHo/0rh7G/uhI1fzuUIrslfXjnV6PHTfsZOfLd3Hu+HnKxTLbL9kGIUyPn2P2e/Wrl3C8zNgfHcEd\nToEXzV+QEqMDM6hc/CAqUHYCzqen2Vqq396aDVNcP7mbWadMICE9QXo+8e7GoylO3v6bHHr0w4hk\nGOhpqcnhhiUoFqIEtpqM5tAvUZ6eIt2X3KbVaEyzlcKniIrhfQ4oAHOlsy/HtqRuCFrdatou4XiZ\n4Fz7DgEgtSObWFDOSTukLumpm99rMn7OgJHdW9h52Q4cz4nm2dtcsAzGLs6f94pIUlKYKJNeYim4\nebJhit4gU5eJvfOTH+Pa2/4z4SL6dW80/GIh+fdItXHWtNGUhnf8qvpbIvK3RH0RvljR/MYh0haM\ndcyieh8sA+GUT9IuHfVDgvFS6+ODMLnKqB8SjLcezkqHXuKmIVHINBGzW0KVky+MsnP/+m5BuxQc\n14sy/mr31oognoXdFkPTMJCqfjPh2A9bmVhE9gAHgW1Ef373q+r/rBnzOuBzwIvxoc+o6gdamd/o\nDknhIkU5lZngTGaKUEKGyj3sKQyT1ga/PgK5Hxsm97JYCH4+z8zXLjTcwdOWfccLhNM+kkrVCb6z\nT06Qe2V8Xi867/RXL+DnSxx55gRHf3CcwA/Ytm8bV16/nxQeTh/VNYpCEkNHjRjws6RCl6Lj12Qu\nS8PM6NBRzl1W5PTMKGEQsmVgmB3He/BmKjKi77uH6975fp566Dc5fWSc7Ze0nuWrGlIcP09pKiol\nneodIDO8dV12I/PiWkpa17Wovd7TxkW6+VviA78a5zn0A4+LyJdU9fs1476mqm/soh1GizTSD57v\nOcd4ema+xPT5dJ6JVIGXTu5K3GHT/79tJ315X5ScBmSvHSS9v4+xPzyMdqDOTjAWaQVzqCphOaTv\nlhFS+3prztvLtz71LcadAmEYnfv4oROcOXSaG8qXMPymXXjbM6AQFgKmvnCacHLBHlLzCMLVUzt4\nvvcs014p6pkdOlw2M0KmgdN8cf8Eo2MTcW4CnLpwjtHhNNeWd+CWLzqGA4UHedi/k6vdg219PzNn\njlXF2UtTY5QLefp3XY4s0D97rSEi9O64lJmzxwjLUTKi43rkRnavSye4HHTtW1PVU8Cp+PmUiDwL\n7AJqnYKxCmiUfzDrlBlLVwupCASEnEtPsaNYfTfmDqfIXN5XLey6gmQcsi8doPDY+JLsdDelSO/t\nqVoliAhOyiG9v7fqrl9cYbpQYFxmquLzqoqvAWdKY8gnQqTXRTyHcGJx+RIZ9XhJficl8QlFyYRe\nYsVWgNnNyuj4RYcAUfe2cslnYo/PphcuOrvxg0e59Z238dSDQcthpEThlag0d3l6Yl3WA3JTafp3\nXUbol0EV8VLWR2EJLMttg4jsI2rN+a2Et18tIt8Tkb8UkZcshz1GNc0S0qbdYuLlLRRlyqsvE+Ft\nyzQUdr3dS8849bYlC8eVTqiSyYl83bZTiO13I/t1Oli0Q6gkrR7ZMNXQIQAUBoLEC1bgB0x59T0N\nRu67h+tue3/LNgSlBn0RVPFnm5QEXwc4XgonlTaHsES6vr4SkT7g08CvqGptLYPvAJeoal5Efgb4\nLHAgYY67iLfEbt25u8sWbywWSkhrJJaKQjaofy+Y8BNLfaofEozWC8HtEkyWk+cPotLQte/kerKJ\nbTRFIbdUIXgRpGeTnZfjOGS1iT2qLWkLURG45LKhlu9gtEJXVwoikiJyCJ9Q1c/Uvq+qk6qaj5//\nBZASkS0J4+5X1ZtU9abBYduJ0QlaTUjrDdLx3W81grAtYQ++f2qWcKyE1pSl1pBFZS7XzX9ilnCy\nXNHaMSaEcKKM1jSaGdoyGO0Qqt2c0kQI7iY9p4RsNl3n18QVNp2s7/sAF1cLoR9w+kjz8Jub7U2O\npYusy9CR0Xm65hQkWsN9FHhWVf97gzHb43GIyCtiey50yyYjop2ENEH4sdfewMjuqBewOELvYA+v\nfMPLyaaTL2LjDx2ndHgaDRT1Q/wLJSYeOh5vD1060lMtbqsqeMLk509Xn/d8kcmHTnD1xHYG/Ryi\nF1c4V09tb7x7qosIwpWj2xgeHpr/Pvv6e3gJu0lNNw57jNx3D4eCOxeeX4TeHftws71EKwbBSWXo\n27HPhFejJbr5W3Iz8DbgKRF5Ij72n4C9AKr6EaL6Se8WEZ8oQe42rbsFNDpJuwXt3E0p+q4Y5BUv\neTl+2ScMQtLZNGE5ZOaGkMI3x+p+Rgshk58+iaQFXEELnSs3kD7Qi5Nz63oiqyq9N29i8uH686Zw\nuSq/nYCQULStzmrdIDUtHPjRMGF6iNAB70LrMfC51UKzMJLjpejbsQ8NA1TVnIHRFt3cffR1GvXb\nuzjmXuDebtlgVLOYCqfe1sx8yWgv5UEc9nZSDqkdOQrUO4U5oo5qnfXxc1nLtYgI7rZs0/O6OLir\n6JbDKUlbS/V35x/kXJy70AriuM3/AA0jAbuF2CAstsJpMFpO3M2hfoh/rn73UbfxTyefU1UJOyBk\nr3YOFB7kKWg7oc0wWsWcwjpAVZkpBxRjkdVzhN6Ui+vIkgva+WeL+OeKeNsy813TVBUNlNmnJ+n7\np1vJXjsAnlA+OkP+S2cJRrvXxKUYn5MU885qLuI49XcNuuasMiQt9L5+hOxLBsARSkei7y1sobzG\n+MGjHOq7kyv1AXMMRldYX+mNG5Sp0kWHAOCHymTRp+h9qSMVTif+9ATFQ3nUVzRU/NNFJj55nP43\nbCN73QCSjnoMpPb2MPS2vXVCcKfxz9XvxddSiOY7I2R3m8F/sXu+tae4QnpfD8Nv24tkWvtzfHf+\nwWg3Ul1pB8NYOrZSWOMEoeInJHOVwxLug+13SEtCSyFTXzjN1KOnwREIFHckTWpntjpz2YkE3tz1\ng8z8Q3ca83lbM6RGsvVCsyNkrx9MFL5XE96OLO5IpqpXtTiCekL2unYzvlvLXTCMdrCVwhonaLBZ\ny3PSlPde2dkKpwrMhag2pxtmFnvbk7eqdgJ3c7o+R4FI+Pa2Zrt23k7hbk5OIHPSDu621r+3kfvu\n4ZB/J2GwcO6CYbSDOYU1jtsgpd8pl9l2tnsxdv98KbF8hJbDhmJwJwgulBKF77Ac4p9tUOKhw4SO\nErazjckB3Mjm4EIpuWxIKSQ40973dvdbvsGh4B1t/YxhLISFj9Y4riN4juCHPhDH8sMQ8cvs+tZX\nunbe4HyJ8okCqV25+RCShpEAXWij9HS7+GeLlE/PRqEr7+J58bWtkteLIcgoR/dOcW50FFVlcLCf\nfZObyTbIM5CsQ98/30bmQB9IlO099Vdn6oX7Ofufqq0C0wIazq8WLIxkdAJbKawDPvGjd9D3d59F\nZmcgDBh64RA3feR3yOQXcZFpg4lPn2T2yQnCUhhV+jwyw/gfH0Vnlt43oel5Hz7B7JOTkbgcKKW5\n83YwSa4WRTm0/XzkEEIFhYnxKZ5xj+P3JK8ahm7bTeZAb1Ql1hG8HVmG3rqHyUdOMfv0JFqO7X9x\nhrE/PooW27N//OBR3p1/MA4jmehsdAZbKaxxLiak/QGv+Ju/Xt6T+0r+b86R/5tl3gpaVvJfOkv+\nS2eX7ZSF7crMTKFORwnDkNFdJbY+V60HeLuyOMPp6lLesRCfubqf/F+fJf/XnbH/1rfu5amHVlFW\nnrGmMaewhllsQprRPqVcCPn642EQMkMRqHYK7nCapKxqSTl4I50V4uea8fDCAziua2EkY0lY+GgN\n0mqFU6NzZPNufR9gwHUd+sL6XU/BuWJiXwUthZRPdlYQrwwjGcZSMaewxlhqhrKxOLLnhMHBfpzK\nfs4Cbspj+Fh9HwT/TJHyyQJaru6wpqWQ4tPd03rCoLt6jrH+Maewhrj3mdsZvv2DuOKZQ1gBLjs8\nzO7hEVKZFK7nsm3zZq7N78QtJe8+mnj4JDOPjxPO+ISlkOIPphg7eLQjfaqTuPst3+CQfycnX7Dq\n88biMU1hjbCYCqdGZ3ECYcdzPewgrtS6kL4eKDNfOc/MV8533TaIwkh3vx0+88iynM5Yp5hTWAOY\noGy0w6FYdN65f/NKm2KsQSx8tIpZK4Ky9Lo4Q8vf79ioZ/zg0YowUnfqTxnrm26249wjIn8vIt8X\nkWdE5JcTxoiIfFhEnheRJ0Xkxm7Zs9ZYC4Ky0+cyePtuNr/rUjb9wiVs+neXNmyCYywf4wePcutb\n9660GcYapZsrBR/4VVW9BngV8Isick3NmJ8GDsSPu4A/6KI9a4a1IigP3raH1M4c4jlIysHtTzH4\ncztt1bAKOFB4EFRNdDbapmtOQVVPqep34udTwLPArpphbwYOasQ3gSER2dEtm9YCa0VQ9nZlcfo8\nxK3ZeSNC7mWDK2OUMc/4waO85c1Xcih4h1VRNdpiWYRmEdkHvAz4Vs1bu4BjFa+Px8dO1fz8XUQr\nCbbu3N0tM1ectSQoO30eiRm7nuAO2kphNTB+8Cj0hVYXyWiLrgvNItIHfBr4FVVdVNaOqt6vqjep\n6k2Dw5s6a+AqYK0IypX4p2frVwlEGbulIzMrYJGRxN1v+UYcRjLR2WiNrjoFEUkROYRPqOpnEoac\nAPZUvN4dH9swzOkHq1lQTiKc8Jl9ZpKwIhFL/ZBwJmC2ixm7RnvMhZGSVnWGkUQ3dx8J8FHgWVX9\n7w2GPQK8Pd6F9CpgQlVPNRi77qjUD9aSQ5gj/1dnmf7bs/hniwTjJQqPjzP28SNQtgvQqkOx1YLR\nEuPR25sAAAqBSURBVN3UFG4G3gY8JSJPxMf+E7AXQFU/AvwF8DPA88AMsGEqeq0l/aAZs09OMvuk\nrQxWM+MHj/KWt1/JZz53iJMvjLJz//oLwRqdo2tOQVW/DomdByvHKPCL3bJhNbIW8g+M9cf4waNc\n987389RDH1hpU4xVjmU0LyPmEIwVx3IXjAUwp7BMrJWENGP9MnLfPZa7YCyIOYVlYK0kpBkbBLXc\nBaMxViW1y6wXQdlYH8yX1/6cmuhsJGIrhS6xFhPSjI3B+MGjXHfb+1faDGOVYk6hC5igbKwJLNPZ\nSMCcQocxQdlYC8yJzqia6GxUYU6hg5igbKwlxg8e5VBwp4nORhUmNHcIE5SNtcitb93LUw+a6Gxc\nxFYKS8QEZWMtMx9GsoJ5Row5hSVggrKxbrCCeUaMOYVFYoKysV6Y36Jqu5EMzCksir47bjZB2VhX\njNx3j+UuGIA5hbao1Q/MIRjrDlstbHjMKbSI6QfGeufiasFyFzYy5hRawPQDY6NwoPAgh/wN0+vK\nSKCb7Tj/SETOisjTDd5/nYhMiMgT8eN93bJlKVhCmrGRGD94lFvfupfQDyyMtEHp5krhY8AbFhjz\nNVW9IX6supZQc4Ky6QfGRsJE541N15yCqn4VWJO3GpaQZhhYXaQNykprCq8Wke+JyF+KyEsaDRKR\nu0TkMRF5bGKsu37GBGXDuLhaCP3AHMMGYyWdwneAS1T1euD3gM82Gqiq96vqTap60+Bw9+qzmKBs\nGBcZue8eDgUmOm80VswpqOqkqubj538BpERky0rZY4KyYdQzJzrbamHjsGJOQUS2i4jEz18R23Jh\nJWwxQdkwkpkPIwXBSptiLBPd3JL6KeAfgCtF5LiIvENE3iUi74qH3Ao8LSLfAz4M3Kaqy1qq0QRl\nw1iYA4UHORS8w1YLG4Su9VNQ1X+1wPv3Avd26/wLYYKyYbTG+MGj0BfOh5G2XzK00iYZXWSldx+t\nCCYoG0Z7vDv/YBxGsi5t650N5xRMUDaMpWC5C+udDdWO01pmGsbiGbnvHg713cmVPGBhpHXMhlgp\nmKBsGJ3h3XkrmLfeWfdOYU4/MEHZMDpHGFjuwnplXTuFSv3AHIJhdIa51YLlLqxP1q1TsIQ0w+ge\nd7/lGwC2WliHrDunYPqBYSwPh/w7rQTGOmRdOQVLSDOM5WH84FHLXVinrBunYAlphrFSWO7CemJd\nOAVLSDOMlWHkvnvmRWdzDOuDNe8UTFA2jJXl7rd8g0PBO1baDKNDrFmnYIKyYawiNLTVwjphTToF\nE5QNY/UwJzpHYSQTndc6a84pnC28aIKyYaxCbn3rXmBZW6IYXWDNOYXtZ0omKBvGKuRAIVotnHzh\ngoWR1jDd7Lz2RyJyVkSebvC+iMiHReR5EXlSRG5sZd6MmzGHYBirkMowkrF26eZK4WPAG5q8/9PA\ngfhxF/AHXbTFMIxlxOoirV265hRU9avAaJMhbwYOasQ3gSER2dEtewzDWB7ufss35sNIxtpjJZvs\n7AKOVbw+Hh87VTtQRO4iWk0A5Lf/1V8e6r55S2YLcH6ljVhG7POub1r/vH8F/P/t3X/InWUdx/H3\nhz0T2/wJi9A2nYgpq2BuYy5zalpCNcxQWUGk4B9CIVhoGFIKkpH5R1GU6VoJ1SzHJiKik/KxH4a6\nuTnd/FHmtCmlEjzlj2bJpz/u6zmc1bPpY+c+l7vP5wVjz7nOfd/7nB2e8z33dc79vfhBm1mGoYvP\n75FvZqN9YuU129cD19fOMR2SNtpeUjvHsOTxdlse7+io+e2jZ4F5fbfnlrGIiKikZlG4Ffhs+RbS\nMmDC9v9MHUVExPC0Nn0kaQ1wKjBH0k7gCmAmgO3rgNuBjwF/BF4BuvY9tn1qumsA8ni7LY93RMjO\nFYgREdHY565ojoiI9qQoRERET4pCCyTNkLRZ0m21swyDpB2SHpa0RdLG2nnaJukQSWslPSbpUUkf\nqJ2pLZKOLc/r5J+/S7q4dq42SfqCpG2SHpG0RtL+tTMNUz5TaIGkLwJLgINsr6idp22SdgBLbHft\nYp8pSboR+I3tVZL2A2bZ7nwHOEkzaL42foLtp2vnaYOkdwO/BRbYflXSL4Dbbf+4brLhyZnCgEma\nC3wcWFU7SwyepIOBk4EfAth+bRQKQnE68GRXC0KfMeAdksaAWcBzlfMMVYrC4H0L+BIwSquNGNgg\naVNpSdJlRwEvAD8qU4SrJM2uHWpIPgWsqR2iTbafBa4FnqFpuTNhe0PdVMOVojBAklYAz9veVDvL\nkJ1kexFN59vPSzq5dqAWjQGLgO/bPh54GbisbqT2lWmyM4Gba2dpk6RDaZp1HgUcDsyW9Jm6qYYr\nRWGwPgicWebYbwJOk/STupHaV95dYft5YD2wtG6iVu0Edtq+r9xeS1Mkuu6jwIO2/1o7SMs+DDxl\n+wXb/wLWASdWzjRUKQoDZPvLtufank9zqv0r251+lyFptqQDJ38GzgCmXFipC2z/BfizpGPL0OnA\n9oqRhuXTdHzqqHgGWCZpliTRPL+PVs40VPtEl9R4W3sXsL75/WEM+JntO+pGat1FwE/LlMqf6F6L\nlt2UYv8R4MLaWdpm+z5Ja4EHgX8Dmxmxlhf5SmpERPRk+igiInpSFCIioidFISIielIUIiKiJ0Uh\nIiJ6UhSicyRdXrpcbi2dPU8Y8PFPnaoD7p7GB/DvnSVpQd/tcUkjuah8tC/XKUSnlDbWK4BFtndJ\nmgPsVznW/+ss4DZG4yK5qCxnCtE1hwEv2t4FYPtF288BSFos6Z7SuO9OSYeV8XFJ3y5nFY9IWlrG\nl0r6fWl8d2/fVcxvqFzpvVrS/WX/T5Tx8yWtk3SHpD9IuqZvnwskPVH2uUHSdyWdSNNz6Jsl39Fl\n83PLdk9IWj6I/7gISFGI7tkAzCsvlt+TdAqApJnAd4BzbC8GVgNf69tvlu2FwOfKfQCPActL47uv\nAldPI8flNG1OlgIfonlRn+ymuhBYCbwfWClpnqTDga8Ay2h6aB0HYPte4FbgUtsLbT9ZjjFWjn0x\ncMU0ckXsVaaPolNsvyRpMbCc5sX455IuAzYC7wPuKi05ZtC0Rp60puz/a0kHSToEOBC4UdIxNO3B\nZ04jyhk0zREvKbf3B44oP//S9gSApO3AkcAc4B7bfyvjNwPv2cvx15W/NwHzp5ErYq9SFKJzbL8O\njAPjkh4GzqN58dxme09LZ/53vxcDVwF32/6kpPnlmG+WgLNtP77bYPOh966+odd5a7+Hk8d4q/tH\nTCnTR9EpZU3hY/qGFgJPA48D75xcT1nSTEnv7dtuZRk/iWZhlQngYJrlJwHOn2aUO4GLSqdNJB3/\nBts/AJwi6dCy4tfZfff9g+asJaJ1KQrRNQfQTPlsl7QVWABcafs14BzgG5IeArawe5/8f0raDFwH\nXFDGrgG+Xsan+278Kprppq2StpXbe1TWpLgauB/4HbADmCh33wRcWj6wPnrqI0QMRrqkxsiTNA5c\nYntj5RwHlM9ExmgWK1pte33NTDF6cqYQ8fZxpaQtNIsUPQXcUjlPjKCcKURERE/OFCIioidFISIi\nelIUIiKiJ0UhIiJ6UhQiIqLnP2dWwukYY/Y2AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11115ad68>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.contourf(xx, yy, Z, cmap=plt.cm.Paired, alpha=0.8)\n",
    "plt.scatter(X[:, 0], X[:, 1], c=y, cmap=plt.cm.Paired)\n",
    "plt.xlabel('Sepal length')\n",
    "plt.ylabel('Sepal width')\n",
    "plt.xlim(xx.min(), xx.max())\n",
    "plt.title('SVC with linear kernel')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Kernels\n",
    "\n",
    "Classes aren't always easily separable. In these cases, we build a decision function that's polynomial instead of linear. This is done using the <b>kernel trick</b>.\n",
    "\n",
    "![alt text](https://github.com/lesley2958/regression/blob/master/svm.png?raw=true \"Logo Title Text 1\")\n",
    "\n",
    "If you modify the previous code with the kernel parameter, you'll get different results!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "svc = svm.SVC(kernel='rbf', C=1, gamma=1).fit(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Usually, linear kernels are the best choice if you have large number of features (>1000) because it is more likely that the data is linearly separable in high dimensional space. Also, you can RBF but do not forget to cross validate for its parameters as to avoid overfitting."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tuning\n",
    "\n",
    "Tuning parameters value for machine learning algorithms effectively improves the model performance. Let’s look at the list of parameters available with SVM.\n",
    "\n",
    "#### Gamma \n",
    "\n",
    "Notice our gamma value from earlier at 1. The higher the value of gamma the more our model will try to exact fit the as per training data set, leading to an overfit model.\n",
    "\n",
    "Let's see our visualizations with gamma values of 10 and 100: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "svc = svm.SVC(kernel='linear', C=1, gamma=10).fit(X, y)\n",
    "svc = svm.SVC(kernel='linear', C=1, gamma=100).fit(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluation\n",
    "\n",
    "As with any other machine learning model, there are pros and cons. In this section, we'll review the pros and cons of this particular model.\n",
    "\n",
    "#### Pros\n",
    "\n",
    "SVMs largest stength is its effectiveness in high dimensional spaces, even when the number of dimensions is greater than the number of samples. Lastly, it uses a subset of training points in the decision function (called support vectors), so it's also memory efficient.\n",
    "\n",
    "#### Cons\n",
    "\n",
    "Now, if we have a large dataset, the required time becomes high. SVMs also perform poorly when there's noise in the dataset. Lastly, SVM doesn’t directly provide probability estimates and must be calculated using an expensive five-fold cross-validation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Linear SVM\n",
    "\n",
    "SVM's implementation is based on the Sequential Minimal Optimization (SMO), which works as followS: \n",
    "\n",
    "1. recursively select a pair of elements ($\\alpha_1, \\alpha_2$).\n",
    "2. perform 1D optimization of the objective function on this pair.\n",
    "\n",
    "### Challenge\n",
    "\n",
    "Implement a function `linear_kernel()` that computes the kernel matrix K = $\\Phi \\Phi^T$\n",
    "\n",
    "Get a solution for the dual problem with [alphas, beta0] = SMO(K, ... y, C), where C = 0.1.\n",
    "\n",
    "### Kernel Matrix\n",
    "\n",
    "$K_{i,j} = \\langle \\Phi_i, \\Phi_j \\rangle$, the kernel matrix is obtained with the formula K = $\\Phi \\Phi^T$. Instead, however, you can use a different function to replace the scalar product, such as the Radial Basis Function (RBF). \n",
    "\n",
    "$ RBF(\\Phi_i, \\Phi_j) = exp(-\\gamma||\\Phi_i - \\Phi_j||^2)$\n",
    "\n",
    "#### Challenge \n",
    "\n",
    "Implement a function `rbf_kernel` and compute K with it. Set $\\gamma = 1$ and C = 1. \n",
    "\n",
    "How do $\\gamma$ and C influence results?\n",
    "\n",
    "How can you automatically find the parameters $\\gamma$ and C? Implement it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}