03-neural-networks_lsg.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Neuronale Netze\n",
    "\n",
    "In dieser Übung schauen wir uns zwei sehr einfache Beispiele für neuronale Netze an. Wenn ihr darüber hinausgehend Interesse an neuronalen Netzen habt, empfehle ich die Lehrveranstaltung \"AI 3: Artificial Neural Networks\" im Sommersemester.\n",
    "\n",
    "Falls ihr an der Theorie und den mathematischen Details interessiert seid, ist aus meiner Sicht folgendes Buch sehr empfehlenswert: https://www.deeplearningbook.org/ (u.a. von Yoshua Bengio, der zusammen mit Geoffrey Hinton und Yann LeCun den Turing-Award 2018 für Arbeiten an Deep Learning-Methoden erhalten hat).\n",
    "\n",
    "Es gibt natürlich eine vielzahl weiterführender praktischer Tutorials für Deep Learning. Aber Achtung: Viele Tutorials zeigen nur sehr knapp, wie man ein neuronales Netz trainiert und evaluiert, ignorieren aber das oft etwas mühsame, aber wichtige Drumherum, also das Vorbereiten des Datensatzes usw. Dieses Tutorial zeigt das hingegen sehr schön: https://www.pyimagesearch.com/2018/09/10/keras-tutorial-how-to-get-started-with-keras-deep-learning-and-python/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SepalLength</th>\n",
       "      <th>SepalWidth</th>\n",
       "      <th>PetalLength</th>\n",
       "      <th>PetalWidth</th>\n",
       "      <th>Name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5.1</td>\n",
       "      <td>3.5</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.7</td>\n",
       "      <td>3.2</td>\n",
       "      <td>1.3</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.6</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.0</td>\n",
       "      <td>3.6</td>\n",
       "      <td>1.4</td>\n",
       "      <td>0.2</td>\n",
       "      <td>Iris-setosa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>6.7</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.2</td>\n",
       "      <td>2.3</td>\n",
       "      <td>Iris-virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>146</th>\n",
       "      <td>6.3</td>\n",
       "      <td>2.5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1.9</td>\n",
       "      <td>Iris-virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>147</th>\n",
       "      <td>6.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.2</td>\n",
       "      <td>2.0</td>\n",
       "      <td>Iris-virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148</th>\n",
       "      <td>6.2</td>\n",
       "      <td>3.4</td>\n",
       "      <td>5.4</td>\n",
       "      <td>2.3</td>\n",
       "      <td>Iris-virginica</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>149</th>\n",
       "      <td>5.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5.1</td>\n",
       "      <td>1.8</td>\n",
       "      <td>Iris-virginica</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>150 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     SepalLength  SepalWidth  PetalLength  PetalWidth            Name\n",
       "0            5.1         3.5          1.4         0.2     Iris-setosa\n",
       "1            4.9         3.0          1.4         0.2     Iris-setosa\n",
       "2            4.7         3.2          1.3         0.2     Iris-setosa\n",
       "3            4.6         3.1          1.5         0.2     Iris-setosa\n",
       "4            5.0         3.6          1.4         0.2     Iris-setosa\n",
       "..           ...         ...          ...         ...             ...\n",
       "145          6.7         3.0          5.2         2.3  Iris-virginica\n",
       "146          6.3         2.5          5.0         1.9  Iris-virginica\n",
       "147          6.5         3.0          5.2         2.0  Iris-virginica\n",
       "148          6.2         3.4          5.4         2.3  Iris-virginica\n",
       "149          5.9         3.0          5.1         1.8  Iris-virginica\n",
       "\n",
       "[150 rows x 5 columns]"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "df = pd.read_csv(\"iris.csv\", index_col=0)\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fdd4aae0e90>]"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW8AAAD4CAYAAAAjKGdbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAZQUlEQVR4nO3dfYwd1XnH8d/DsiQLIWwAk4bFGysUgRS24LDiRZYqIGnJCyIrE6lYkDZpi6VKTYkSEZWKNmpFZVVIUdL+0cqEtkmhJOElqzRKQqlClAZhV2ubxk0MUkBgWJLiYNaEshCzPP1j9zre5c7ee87eOXNm5vuRVthn5/o+5479MPPsc+aYuwsAUC/HVB0AACAcyRsAaojkDQA1RPIGgBoieQNADR1bxh966qmn+oYNG8r4owGgkXbt2vVzd1/X7/GlJO8NGzZoZmamjD8aABrJzJ4KOZ6yCQDUEMkbAGqoZ/I2s7PN7JGjvl40s0+mCA4A0F3Pmre7PybpfEkysyFJs5K+XnJcAIBVhJZN3ivpcXcPKqwDAAYrtNvkGkl3dfuGmW2VtFWSxsfH1xgWepneM6tb739Mz87N6/TREd14xdma2jhWdVgAEun7ytvMjpN0laS7u33f3be7+6S7T65b13erIiJM75nVTfft1ezcvFzS7Ny8brpvr6b3zFYdGoBEQsomH5C0293/t6xg0J9b739M84cXlo3NH17Qrfc/VlFEAFILSd5bVFAyQVrPzs0HjQNonr6St5kdL+m3JN1Xbjjox+mjI0HjAJqnr+Tt7i+7+ynufqjsgNDbjVecrZHhoWVjI8NDuvGKsyuKCEBqpTzbBOXqdJXQbQK0F8m7pqY2jpWerGlHbC/Off5I3uiq047Y6WrptCNK4h9xw3Hu64EHU6Er2hHbi3NfD1x5o6uU7YjcoueFVtR64MobXaVqR2S1aH5oRa0Hkje6StWOyC16fmhFrQfKJugqVTsit+j5oRW1HkjeKJSiHfH00RHNdknU3KJXK8W5x9pQNkGluEUH4nDljUpxiw7EIXmjctyi54f2zfyRvAEswwrLeqDmDWAZ2jfrgStvDBS32/VH+2Y9cOWNgWG1ZDOwwrIeSN4YGG63m4H2zXqgbIKB4Xa7GWjfrAeSNwaG1ZLNQftm/iibYGC43QbS4cobA8PtNpAOyRuFYtr+Ym63aS8Mw+cFieSNAqlW2bGaLwyfFzqoeaOrVG1/tBeG4fNCR19X3mY2KumLks6V5JJ+390fLjMwVCtV21+q94kpNeRYnqAdEx39Xnl/QdJ33P0cSedJ2ldeSMhBqlV2Kd4nZuVnrqtFWf2Ijp7J28zeKuk3Jd0uSe7+S3efKzswVCtV21+K94kpNeRanqAdEx39lE3eJemApH8ys/Mk7ZJ0g7v/39EHmdlWSVslaXx8fNBxIrFUbX8p3iem1JBreYJ2THT0k7yPlfQeSZ9w951m9gVJfyrpz48+yN23S9ouSZOTkz7oQJFeU1bZxaz8zHm1aFPOC9amn5r3M5KecfedS7+/R4vJHFizFLXly85ZFzQuUZ5A/nomb3f/maSnzazzt/a9kn5calRojRS15QcfPRA0Li1e3W7bPKGx0RGZpLHREW3bPMEVL7LR7yKdT0i608yOk/SEpI+XF1L75NiSlkqK2nLse1CeQM76St7u/oikyZJjaaW2r5h78/Axmj/8etfxQcm5fg3EYoVlxXJtSUvl1dfemLhXG49B/RpNxLNNKpZrS1oqrxf0JRWNx5jaOKaZpw7qrp1Pa8FdQ2a6+oLeJZGbp/cue82Wi9brlqmJwQUGrAFX3hVr+4q5IbOg8RjTe2Z1765ZLfji/xEW3HXvrtlVO1punt6rO3bsX/aaO3bs183TewcWF7AWJO+Ktf2WfstF64PGY8SUpu7a+XTQOJAaZZOKtX3FXKcMUWZ5IqY01bni7nccSI3knYFcNzCIqfnGxHXL1ERQsg59j5hukyGzrol6tXJOm1s+kR5lkxpKsSoxpuabIq6Y94gpTYWWc3J9CiGai+RdQynaC2NqviniinmPmNWSt0xN6LqLx49caQ+Z6bqLxwvvENre8on0KJvUUIr2wpiab2xcIeWZlKslQ8o5bW/5RHpceddQivbCmBa+mLhCyzOjxw8HjafS9pZPpEfyrqEU7YUxLXwxcYWWZ4ou/KtuAml7yyfSI3nXUIon3oXWfGPjCi3PHJo/HDSeytTGMV19wdiyz6ufVZxALGreNZXiiXehLXxSeFyhLXknjQxrrkuiPmmk2rJJ0SrOyXeeTAJHKbjyRqVCyzNFJfcBrqaPQrcJUuPKG5UKXWE593L38kjReCp0myA1kneL5LoCMKQ8E/ts7rLnzjPDkRplk5ZoygrAmK6OFHOn2wSpkbxboik12ZiOlhRzZ89LpEbZpCVyrsmGPgArtKMl1dzZ8xIpceXdErmuAEyx6UGucwfWguTdErnWZFNsepDr3IG1oGzSErlu+pBi04Nc5w6sBcm7Re6e2X+knW12bl53z+xfNYGl2ICXTQ+AOJRNWuLa2x7WQ48fXDb20OMHde1tD3c9PtUGvBe/621B4zFtf01pkwSORvJuiZWJu9d4qg14n3y+e8dH0XhM219T2iSBo/VVNjGzJyX9QtKCpNfcfbLMoMqS6+12ivJEqFQb8Ia28cW0/eXcJgnECrnyvszdz69z4s7x1jlVeSJUzGYMMUI3V4hp+6NVEE3UmrJJrrfOqcoTm848OWg8ZjOGGKGbK8S0/dEqiCbqN3m7pH83s11mtrXbAWa21cxmzGzmwIEDg4twQHK9dU5Vnrjz+kt01mknLBs767QTdOf1l3Q9/papiTck9k1nnjzwck7o5goxmx6wdB1N1G+r4CZ3f9bMTpP0gJk96u7fP/oAd98uabskTU5OVrwp1Rvl+tS3Y0x6vcundcyAn089vWdWz7zwyrKxZ154RdN7Zrsmsek9s9q9/9Cysd37DxUeHyt0c4XYTQ9Yuo6m6evK292fXfrvc5K+LunCMoMqQ663zm86tvspKBqPFVo2SlVmCt1cIdfyF5BazwxhZieY2YmdX0v6bUn/U3Zgg5brrfMrh18PGo+VoqsjRujmCrmWv4DU+imbvF3S123xUuhYSf/q7t8pNaqS5HjrnKqcE/o+TYsr1zZRIFbPK293f8Ldz1v6ere7/3WKwNoiVTkn9H02nNI9GRaNx7rsnHVB47luxgCk1ppWwVylKueEvs+OJ14IGo/14KPdO5OKxnPdjAFIjQdTZSBVOSfkfWJbGEPLEylq2LHvQakFOSN5o6uYFsZOeaJzldspT0gqTHqhNeyY9xg9flgvdPkBaNEqztj3AVKibIKuYloYY8oToTXsmPcIXcUZ+z5ASiRvdBXTwhhTnghdMRnzHqGrOGPfB0iJ5I2uQh8YFfuaohWTRZ0gqR5MxcOskDuSN7qKKTWkKE+kejBVritygQ5+YImuYkoNKcoTMftRpnoNkFKrkneurV85xhWzkjHVa3JsrZTyPI9ortaUTXJdZZdrXKErH2Nfk2IlZ4rPONfziOZqTfLOtfUr17hCVz7GvibFSs4Un3Gu5xHN1ZqySa6r7JoUV8xrYlZyNmkVJxCrNVfeMa1fKW6FmxRXzGtCn+cdM/cUbX+0FiK11iTvmNavFLfCTYor5jUjBSs2i8ZTrOKMQWshUmtN2SSm9SvFrXCT4praOKa7Z/broccPHhl7z/hJq75mvmDFZtF47CpOqdy2P1oLkVprkrcU3vqVakOCpsR18/TeZYlbkh56/KBunt5buHFxqs0YUrQX5rjZB5qrNWWTGLneCuca1107nw4al8LnkuvcgdRadeUdKtdb4VzjiukcCZ1LrnMHUiN595DrrXBMXGW3Fw6ZdU3UQ0WtI0tmnjqonx16RS7pZ4de0cxTB3vW1nM8J0BKlE1aIkV74cXvelvQuLRYJ79jx/5lTxW8Y8d+3Ty9d2BxAU1E8m6JFO2FTz7fveOjaFyKq5MDoGzSGjEtdilWMsbulQm0HVfeLRG6UUKqlYyhKywBLCJ5t0ToRgmpVjKGrrAEsKjvsomZDUmakTTr7leWF1J5YrotUjyjOUVcoRslpFrJGLrCEsCikJr3DZL2SXprSbGUqlMG6FxNdsoAkgqTS8xrco0rdGXi6PHDeuHlNyb21faj7Lx/jqtFgabp697UzM6Q9CFJXyw3nPLElAFyfQ50ipJGzH6UMVgxCcTp98r785I+I+nEogPMbKukrZI0Pj6+9sgGLNXzqUOliiu0pBGzH2UMVkwCcXombzO7UtJz7r7LzC4tOs7dt0vaLkmTk5PZ9Xml2l8x57hCVjKmLGewYhII10/ZZJOkq8zsSUlfkXS5md1RalQlSPV86lAx+z7GxBW6kpFyBpC3nsnb3W9y9zPcfYOkayR9192vKz2yAZvaOKZtmyc0NjoikzQ2OqJtmyd6PkMj9DWhYvZ9jIkrdCVjirkDiNeqFZYxt+dlPwAqtq4eGlfsE/9I1kCeglZCuPv36trjnUroysRUex8WPdmv1xP/AOSJZWwDFtrGl6q2vOWi9UHjAPLWqrJJCqFlkFStcp1tyO7a+bQW3DVkpi0XrS/cngxA3kjeA3bSyLDmuvRCnzRSvDIxVW35lqkJkjXQEJRNBoyn5AFIgeQ9YHNdngey2jgAxKBsMmBtf9BSiqcwAuDKe+DavDIxxT6ZABaRvAeszSsTUzyFEcCibMomTbrdTtE9kuPnleIpjAAWZZG8U2x60CS5fl5tr/cDKWVRNuF2O0yun1eb6/1AallceXO7HSbXz4uNFYB0skje3G6Hyfnz4kmEQBpZlE243Q7D5wUgiytvbrfD8HkBMB/0duBa3MNyZmZm4H9uFWJa8nJs4wOQNzPb5e6T/R6fxZV3rmJa8nJt4wPQLFnUvHMV05KXaxsfgGYhea8ipiUv1zY+AM1C8l5FzP6SqfakBNBuJO9VxLTk0cYHIAV+YLmKmJY82vgApECrIABkILRVkLIJANQQyRsAaqhnzdvM3izp+5LetHT8Pe7+2bIDaxNWZAII1c8PLF+VdLm7v2Rmw5J+YGbfdvcdJcfWCqzIBBCjZ9nEF7209Nvhpa/B/5SzpViRCSBGXzVvMxsys0ckPSfpAXff2eWYrWY2Y2YzBw4cGHScjcWKTAAx+kre7r7g7udLOkPShWZ2bpdjtrv7pLtPrlu3btBxNhYrMgHECOo2cfc5Sd+T9P5SomkhVmQCiNEzeZvZOjMbXfr1iKT3SXq07MDaYmrjmLZtntDY6IhM0tjoiLZtnuCHlQBW1U+3yTskfcnMhrSY7L/m7t8sN6x2Yd9HAKF6Jm93/6GkjQliAQD0iRWWAFBDJG8AqCGSNwDUEMkbAGqI5A0ANUTyBoAaInkDQA2RvAGghkjeAFBDJG8AqCGSNwDUEMkbAGqI5A0ANUTyBoAaInkDQA2RvAGghkjeAFBDJG8AqCGSNwDUEMkbAGqI5A0ANUTyBoAaInkDQA2RvAGghnombzNbb2YPmtk+M/uRmd2QIjAAQLFj+zjmNUmfdvfdZnaipF1m9oC7/7jk2AAABXpeebv7T91999KvfyFpn6SxsgMDABQLqnmb2QZJGyXt7PK9rWY2Y2YzBw4cGEx0AICu+imbSJLM7C2S7pX0SXd/ceX33X27pO2SNDk56QOLEKiJ6T2zuvX+x/Ts3LxOHx3RjVecramN9bxJbdJcmqqv5G1mw1pM3He6+33lhgTUz/SeWd10317NH16QJM3Ozeum+/ZKUu2SXpPm0mT9dJuYpNsl7XP3z5UfElA/t97/2JFk1zF/eEG33v9YRRHFa9JcmqyfmvcmSR+VdLmZPbL09cGS4wJq5dm5+aDxnDVpLk3Ws2zi7j+QZAliAWrr9NERzXZJbqePjlQQzdo0aS5NxgpLYABuvOJsjQwPLRsbGR7SjVecXVFE8Zo0lybru9sEaJuQjoupjWOaeeqg7tr5tBbcNWSmqy8Yq+UP+GLnQodKWlx5A110Oi5m5+bl+lXHxfSe2cLj7901qwVf7JJdcNe9u2YLj89ZzFxCPy+sHckb6CK046JJHRoxc2nS/OuC5A10Edpx0aQOjZi5NGn+dUHNG5ULrZXG1FZDXxPacRHboZFjnThmLnSopMeVNyoVU1sOra3GvCa042LDKd2TVNF4bFwpXHbOuqBxiQ6VKpC8UakUteWY10xtHNO2zRMaGx2RSRobHdG2zROFV8U7nnghaDw2rhQefLT7g+WKxqXwzwtrR9kElUpRW46tx05t7L/Vr9OZ0e/4WuIKde1tD+uhxw8e+f2mM0/WnddfMvC4Qj6vWDmWmaqKiytvVGq1GnI3xxSs9S0aj3mPVEaPHw4aj7EycUvSQ48f1LW3PVxpXDFyLTNVFRfJG5UKrZUuFFzIFo3HvEcqRRflq1ysB1uZuHuNr/b+g4wrRq5lpqriomyCSnVuLcu85UzxHjEOzR8OGk8l17hybUesKi6SNwYqpvZ398z+I21ms3Pzuntmf+WJVSq/jplre2GubX98XstRNsHAxNT+QmuyZ512QtB4bFyhr3n7iccFjUtp2gs3nXly0LiUb5kpJq4U9eiqPi+SNwYmpvYXWpN94FOXviFRn3XaCXrgU5cONK7Q1/z8pe4lhaJxKU174Z3XX/KGRN2r2yTXtr+YuFLUo6v6vCib9JBra1KOUtX+VkvUIe8/yPbCmFbBVO2FqyXqIjFtfyn+rYTGlervZIo2yZW48l5Frq1Jucq1Je/444aCxqXwuQxZ917FovFYuX7Guf5byfXzGgSS9ypybU3KVUzt77ih7smtaDzGy79cCBqXwuey5aL1QeNS8fZUq80813p0rv9Wcv28BoGyySpybU3KVUxL3uGCBu2i8RhFf9Jq7xA6l1umJiRp2QYGWy5af2S8qrhSifm3kqrMIuX3eQ0CyXsVubZM5Sy09pfiMzbrvsCkV0UjdC63TE2smqxXGjLrWt/uVWqpor7aS+h57JRZOlfrnTKLpMrr5HVB2WQVTb7lykWKz3jk2O5/zYvGU4kpteQq9DzmWmapE668V9HkW65cpPiM5w+/HjSeSkypJVeh55GS5NqRvHto6i1Xm+Rc/gotteQs5N9KzuekLiiboFJNXgGHYpyTtSN5o1JNXgGHYpyTtetZNjGzf5R0paTn3P3c8kNCmzR5BVxZmrLqt0nnpAr9XHn/s6T3lxwHWqrJK+DKkOtKRqTXM3m7+/clFT+5HVgDap9haLFDx8C6Tcxsq6StkjQ+Pj6oPxYNRztmGFrs0DGw5O3u2yVtl6TJycmKN0xCnVD77B8tduig2wSoEcpM6GCRDlAjlJnQ0U+r4F2SLpV0qpk9I+mz7n572YEB6I4yE6Q+kre7b0kRCACgf9S8AaCGSN4AUEMkbwCoIZI3ANSQebf9odb6h5odkPRU5MtPlfTzAYZTJ22eu9Tu+TP39urM/53uvq7fF5WSvNfCzGbcfbLqOKrQ5rlL7Z4/c2/n3KX4+VM2AYAaInkDQA3lmLy3Vx1Ahdo8d6nd82fu7RU1/+xq3gCA3nK88gYA9EDyBoAaqiR5m9l6M3vQzPaZ2Y/M7IYux5iZ/a2Z/cTMfmhm76ki1kHrc+6XmtkhM3tk6esvqoh10MzszWb2X2b230tz/8sux7zJzL66dN53mtmG9JGWo8/5f8zMDhx17v+wiljLYmZDZrbHzL7Z5XuNPfdSz7kHn/eqnuf9mqRPu/tuMztR0i4ze8Ddf3zUMR+QdNbS10WS/n7pv3XXz9wl6T/d/coK4ivTq5Iud/eXzGxY0g/M7NvuvuOoY/5A0gvu/utmdo2kv5H0O1UEW4J+5i9JX3X3P64gvhRukLRP0lu7fK/J515afe5S4Hmv5Mrb3X/q7ruXfv0LLU5o5QOKPyzpy75oh6RRM3tH4lAHrs+5N9LSuXxp6bfDS18rf2L+YUlfWvr1PZLea2aWKMRS9Tn/xjKzMyR9SNIXCw5p7LnvY+7BKq95L90abZS0c8W3xiQ9fdTvn1HDktwqc5ekS5Zur79tZu9OGliJlm4dH5H0nKQH3L3wvLv7a5IOSTolbZTl6WP+knT1UqnwHjNbnzjEMn1e0mckvV7w/Saf+15zlwLPe6XJ28zeIuleSZ909xdXfrvLSxpzldJj7ru1+JyD8yT9naTp1PGVxd0X3P18SWdIutDMzl1xSKPPex/z/zdJG9z9NyT9h351JVprZnalpOfcfddqh3UZq/2573Puwee9suS9VPO7V9Kd7n5fl0OekXT0/33OkPRsitjK1mvu7v5i5/ba3b8ladjMTk0cZqncfU7S9yS9f8W3jpx3MztW0kmSDiYNLoGi+bv78+7+6tJvb5N0QeLQyrJJ0lVm9qSkr0i63MzuWHFMU899z7nHnPequk1M0u2S9rn75woO+4ak313qOrlY0iF3/2myIEvSz9zN7Nc6tT4zu1CL5+n5dFGWw8zWmdno0q9HJL1P0qMrDvuGpN9b+vVHJH3XG7KSrJ/5r/i5zlVa/JlI7bn7Te5+hrtvkHSNFs/rdSsOa+S572fuMee9qm6TTZI+KmnvUv1Pkv5M0rgkufs/SPqWpA9K+omklyV9vII4y9DP3D8i6Y/M7DVJ85KuacJfYknvkPQlMxvS4v+Qvubu3zSzv5I04+7f0OL/2P7FzH6ixauua6oLd+D6mf+fmNlVWuxKOijpY5VFm0CLzv0brPW8szweAGqo8m4TAEA4kjcA1BDJGwBqiOQNADVE8gaAGiJ5A0ANkbwBoIb+H+AMyjwOVduDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.plot(df.SepalWidth.values, df.PetalLength.values,'o')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Neuronale Netze für Regression\n",
    "\n",
    "\n",
    "\n",
    "In dieser Übung verwenden wir `keras` für das Training von neuronalen Netzen. Weitere Informationen zu diesem Paket finden Sie hier: https://keras.io/\n",
    "\n",
    "Die Beispiele, die wir uns in der Übung anschauen, sollten aber (hoffentlich) recht selbsterklärend sein. In dem Beispiel unten beschreiben wir die Netzwerkstruktur in Keras. Dieses Netzwerk besteht nur aus einem einzigen Knoten, die Aktivierungsfunktion ist die Identität. Zudem legen wir fest, dass wir den Mean Squared Error als Loss-Funktion nutzen wollen, und den Adam-Optimizer. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import keras\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Input, Flatten, Softmax\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "model = Sequential([ \n",
    "    Dense(1,                         # number of units in the layer\n",
    "          input_shape=(1,),          # shape of the inputs\n",
    "          activation='linear',       # the unit computes a simple linear (identity function) on its input\n",
    "          kernel_initializer='ones', # the weights (there is just one in this example) are initialised to 1\n",
    "          use_bias=False             # the unit does not use a bias term as offset\n",
    ")])\n",
    "\n",
    "model.compile(optimizer='adam', loss='mse')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nun trainieren und evaluieren wir das Modell. Das Array `x` wird später für den Plot der Netzwerkfunktion benötigt. Dann trainieren wir ein Modell mit SepalWidth als Input und PetalLength als Output. Alle 25 Trainingsepochen bestimmen wir die Prädiktion der Werte in `x` mit der Funktion `predict` (diese benötigen wir, um die Netzwerkfunktion zu plotten) und geben den Wert der Loss-Funktion aus. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loss after 25 epochs: 4.205177\n",
      "Loss after 50 epochs: 4.144640\n",
      "Loss after 75 epochs: 4.132361\n",
      "Loss after 100 epochs: 4.130443\n",
      "Loss after 125 epochs: 4.130252\n",
      "The network's coefficient is: 1.170187\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEWCAYAAABsY4yMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd1hUx/rA8e/QO3YCdmyACvaCPcYSa6IxJtFri9Ho1dxUyy+xpGpuNKZYEhM1tth7LyggNuwRFAUVRVAUkN535/fHWbkooFhWEOfzPD6yp8yZOQvvzs455x0hpURRFEUpeUyKugKKoiiKcagAryiKUkKpAK8oilJCqQCvKIpSQqkAryiKUkKpAK8oilJCqQCv5CGE8BVCDC/ktnWEEKeEEElCiA+MXbdcx60ihEgWQpgaoexWQohQQ/mvGaH8IUKIgFyvk4UQroafrYUQW4QQCUKINYZl3wghYoQQN592XZ5H958/pWAqwD8jQohwIUS0EMI217LhQgjfQu7/lxDiG6NV8PGNA3yllPZSyl+MdRDD+Xvl7msp5TUppZ2UUmeEw30FzDaUv9EI5d/DcJzLhpdvAE5AWSllPyFEZeATwENK+ZKx63K/R/mwV4ofFeCfLTPgP0VdiYIIzaP+TlQFgo1RnyL02G0SQpg9hWNflFJm53odK6W89Rh1eZz3UylJpJTq3zP4B4QDE4A4oJRh2XC03u/dbdyAPYZtLgBvGpaPALKATCAZ2AIMBbbk2jcMWJ3rdQTQwPCzN3AMSDD8751rO1/gW+AgkAbUNCwbbljvDPwDfJpPm/YBOiDdUK/aufc1bDMECMj1WgLvA6HAHWAOIHKtfw84DyQB54BGwFJAb6hfMtq3hmqGsswM+7kAmw3nLgx4L1eZU4HVwBJDucFAkwLep0v3HcuyEGWvBZYBibnbnmubsob9E4FA4Ot8zklN4EvDe5xlOPZIQz30htd/GbZvARwC4oEzQPuHvJ+OwALgBhAJfAOY5n5/gBmG9+MK8Kph3bf3vb+zCzhnD6vPNEO7E4BNQJlc63sZ3o94w7buudZVBtYDt4HYu8d/UJ1zrb9seK+vAAOK+u+/yOJOUVfgRfmHFuBfMfzCfmNYlhPgAVu0oDwUraffCIgB6hrW/3V3P8NrV8MfhQlaEL4KROZad8ewrozh538Zyn3b8LqsYVtf4BpQ17De3LBsOFoQvQiMeEC7fLk3oN//egh5g9lWoBRQxfDH29Wwrh9aAGoKCLTgVDX3+ctVTjXuDfB+wFzACmhgKLejYd1UtCDVDTBFCzhHHvZe5Xr9sLKzgNcM59s6n/JWon3A2AL1DG3ME+Bzlbcs17r2wPVcryuiBbtuhuN1Mrwu/4D3cyPwu+H4FdCC7chc708W2gerKTAKiMLwoXv/+5lP2wpTn0hDu22BdXfbh9YhSDHsY472wR0GWBjqcgaYZdjPCmj9sDobtk0E6hi2dcbwN/Qi/lNf3569ycBYIUT5+5b3AMKllIuklNlSypNofwxv5FeI1MZsk9ACTjtgFxAphHAzvD4gpdQD3YFQKeVSQ7krgBCgZ67i/pJSBhvWZxmWeaD9cU6RUs5/Cu3ObbqUMl5KeQ3Yb2gDaB8q/5VSHpOaMCnl1YcVZhinbg2Ml1KmSylPA3+ifajdFSCl3C61MfulgFdhKlrIsg9LKTdKKfVSyrT79jcF+gKTpZQpUsogYHFhjl2AgcB2Q1v0Uso9wHG0AHtXzvuJ9gH/KvCh4fi30ILmW7m2vyql/MNwbhajBUWnp1ifpVLKICllCjAJeNNwXvoD26SUewy/dzMAa7RvnM3Qvjl9Zqh3upQy94XVB9VZD9QTQlhLKW9IKUvaEGKhqQD/jBn+wLeiDdfkVhVoLoSIv/sPGAA86MKaH1oPr63hZ1+04N7O8Bq0P5L7g+RVtJ7XXRH5lD0Aree19sEteiy57wZJBewMP1dGGyJ5VC5AnJQyKdey+9t4/zGtCjleXpiy8zt/d5VH60nn3uahH1oPUBXod9/vSWu0AJdffaqi9Y5v5Nr+d7Se/F0550ZKmWr40Y7CedT6XDXUpxz3/W4aOiQRaOe2MloQzyZ/+dbZ8CHSH20Y8IYQYpuh0/NCUgG+aExB+3p5f5Dwk1KWyvXPTko5yrA+v7SfdwN8G8PPfuQN8FFof4S5VUEL3nflV/ZUtCGivx/xVsQUwCbX60e58yMCqFHAugelPY0Cyggh7HMtu7+Nj6swZT+obreBbLSAlXv/xxWB1iPO/XtiK6WcXkB9IoAMoFyu7R2klHULebyHpZstTH3ub3sW2u/WPb+bQghh2DbSUG6Vx7loLaXcJaXshPYhEwL88ahllBQqwBcBKWUYsArIfd/4VqC2EOJfQghzw7+mQgh3w/potLH13PyADmjjvteBA0BXtIt6pwzbbDeU+44QwkwI0R9t+GXrQ6qZhTYmbgssfYS7MU4DfYQQNkKImsC7hdwPtKGPT4UQjQ13gNQUQtwNAPm1HwApZQTaRb5pQggrIYSn4bjLH+HY+XrSsg1DCOuBqYZz4gEMfoIqLQN6CiG6CCFMDXVqL4SoVMDxbwC7gZlCCAchhIkQooYQol0hj1fgeX+E+gwUQngIIWzQbkFdazgvq4HuQoiOQghztNtBM9DOdyDaReHpQghbQ7mtHlZZIYSTEKKX4XbkDLSLw8a4lfa5oAJ80fkKLXgCYBgC6Iw2NhqF9hX0e7S7OEC7C8LD8DV4o2Gfi2i/wAcMrxPR7h44aPgDQkoZiza+/wnaxa9xQA8pZczDKiilzAT6oH2dX1jIID8L7U6QaLSx0UIHWSnlGrQ7N/5Gu76wEW0MGbQLo18Y2v9pPru/jXbhNQrYgHbtYE9hj/0QT1r2GLQhj5toF8sXPW5FDB84vYH/Q/t2EAF8xoP/lgehXbg8h3aBfS33DqE8yM/AG0KIO0KIPM85FLI+S9HafRPtYukHhn0voI3h/4rWo+8J9JRSZhp+f3uiXWi/BlxHG3p5GBO03/UotLue2gGjC9nWEufulXJFUZSnzvAg3zIp5Z9FXZcXkerBK4qilFAqwCuKopRQaohGURSlhFI9eEVRlBLqSRMjPVXlypWT1apVe6x9U1JSsLW1ffiGJYhqc8n3orUXXsw2P4kTJ07ESCnvfzIeKGYBvlq1ahw/fvyx9vX19aV9+/ZPt0LFnGpzyfeitRdezDY/CSFEgU9GG3WIRghRSgixVggRIoQ4L4RoaczjKYqiKP9j7B78z8BOKeUbQggL7n2EXVEURTEiowV4IYQDWhKsIZDzVGSmsY6nKIqi3Mtot0kKIRoA89Eej/YCTgD/MWR7y73dCLQJLXBycmq8cuXK+8vB1tYWU9MH57uSUqLlKnpxGKPNOp2OlJQUiuvts8nJydjZFTbR4fPvRWsvvJhtfhIdOnQ4IaVskt86Ywb4JsARoJWU8qgQ4mcgUUo5qaB9mjRpIu+/yHrlyhXs7e0pW7bsA4NZUlIS9vb2Ba4viZ52m6WUxMbGkpSURPXq1Z9auU/Ti3YB7kVrL7yYbX4SQogCA7wxL7JeR5uJ5qjh9Vq0WYoeSXp6+kODu/J0CCEoW7Ys6enpRV0VRVGeAqMFeCnlTSBCCFHHsKgj2nDNI1PB/dlR51pRSg5jP8k6FlguhPgHbVq274x8PEVRlOdK1IVYjq08bZSyjRrgpZSnpZRNpJSeUsrXpJR3jHk8Y4iIiKBDhw64u7tTt25dfv7555x1U6dOpWLFijRo0IAGDRqwffv2Z16/v/76izFjxjzz4yqK8mQy07PZN+8IG2ad4ezuMDLuJD18p0dUrJ5kLY7MzMyYOXMmjRo1IikpicaNG9OpUyc8PDwA+Oijj/j00/zmn1AURclf+Jlo9v9xitQsMyrFHMHqVYlFqb5P/Tgq2dhDODs706iRdm3Y3t4ed3d3IiMfbarPH374gaZNm+Lp6cmUKVMACA8Px83NjcGDB+Pp6ckbb7xBaqo2d7CPjw8NGzakfv36DBs2jIyMDACOHTuGt7c3Xl5eNGvWjKQk7RM/KiqKrl27UqtWLcaNGwdotzsOGTKEevXqUb9+fWbNmvVUzoeiKI8vPTmLnTMD2DYvGBLiqCrXMaPzWr532E9adtpTP95z1YP/cksw56IS812n0+keeq98fjxcHJjSs3DzD4eHh3Pq1CmaN2+es2z27NksWbKEJk2aMHPmTEqXLn3PPrt37yY0NJTAwECklPTq1Qt/f3+qVKnChQsXWLBgAa1atWLYsGHMnTuXMWPGMGTIEHx8fKhduzaDBg1i3rx5jB49mv79+7Nq1SqaNm1KYmIiOp021eTp06c5deoUlpaW1KlTh7Fjx3Lr1i0iIyMJCgoCID4+/pHPjaIoT4eUkrDDEfgtO0dmtilV4wLwb3yUH10i6e7anXFNx2Fj/vQf9Fc9+EJKTk6mb9++/PTTTzg4OAAwatQoLl26xOnTp3F2duaTTz7Js9/u3bvZvXs3DRs2pFGjRoSEhBAaGgpA5cqVadVKm0d44MCBBAQEcOHCBapXr07t2rUBGDx4MP7+/ly4cAFnZ2eaNm0KgIODA2Zm2udzx44dcXR0xMrKCg8PD65evYqrqyuXL19m7Nix7Ny5M6fOiqI8WykJGWz5zo/dS8KwSLjJS+ar+Krzek7V0jG341ymt5lOGasyDy/oMTxXPfgH9bSN+aBTVlYWffv2ZcCAAfTp0ydnuZOTU87P7733Hj169Mizr5SSiRMnMnLkyHuWh4eH57klUQhR4BOkD3pq1dLSMudnU1NTsrOzKV26NGfOnGHXrl3MmTOH1atXs3Dhwoc3VlGUp0JKybm9lzi4/hK6bEn1eH82NT/E4fJxvO32Dh80+gBbc1uICYXIE+D11lOvg+rBP4SUknfffRd3d3c+/vjje9bduHEj5+cNGzZQr169PPt36dKFhQsXkpycDEBkZCS3bt0C4Nq1axw+fBiAFStW0Lp1a9zc3AgPDycsLAyApUuX0q5dO9zc3IiKiuLYsWOA9oGWnZ1dYL1jYmLQ6/X07duXr7/+mpMnTz7BWVAU5VEkxqSxfrIPvuuuYRN/FQfbv/mi62Zu1nRkyatLmNh8IrbCDHy/h3nesPsLyEx5eMGP6LnqwReFgwcPsnTpUurXr0+DBg0A+O677+jWrRvjxo3j9OnTCCGoVq0av//+e579O3fuzPnz52nZUsuUbGdnx7JlyzA1NcXd3Z3FixczcuRIatWqxahRo7CysmLRokX069eP7OxsmjZtyvvvv4+FhQWrVq1i7NixpKWlYW1tzYYNGwqsd2RkJEOHDkWv1wMwbdo0I5wdRVFy0+slp7ecJ3DHdcjOwjXRjyUtAggpn8Z79UczvP5wLEwt4Oph2PIfiLkAdftA1+lg8fQnOSlWc7Lml4vm/PnzuLu7P3Tf5y0XTXh4OD169Mi5CPo4jNXmwp7zovCi5Sl50doLz2+bY6OS2fvzIWISzCh75xwJVQKY6xFEPScvvmz5JTVL14S0O7BnCpxcDI5VoMePUKvTEx33QbloVA9eURTlCeiy9Rxbc5ZTvrcwzU7HNXU/c70PEF3OhHGNJvJWnbcwFSYQtA52TIDUWPAeC+0nGqXXnpsK8EWkWrVqT9R7VxSl6EVfjmfv7CPEp1pQIe4s12r5M752GK0qtWF+i8k42znDnXDY9gmE7QWXhjBwLTh7PZP6qQCvKIryiLIydRxecoKzxxKxyEylavZWZrUPILOsPd81m0736t0Reh0c/AX2fwcmpto4e7MR2s/PiArwiqIoj+B68G18fj9BcqYFzrHHOePhyy+u1+leowfjmo7T7mmPPAGb/wPRZ6FON+j2AzhWeuZ1VQFeURSlEDLTsjnwZyAhwelYpSVQ0XQfP3Q8hE35l5jbYh5tKrWBjCTYMR6O/g72L8GbS8G9JzwgDbeUkjupWZSxtXjqdVYBXlEU5SGuHI9k/6KzpGWbUSn2EL6e+/GpGnvvA0sh22D7Z5AYBU2HQ8dJYOX44HJjUpi6OZiIuFR2fNgGS7OnO3yjAnwhVKtWDXt7e0xNTTEzM+PurZxxcXH079+f8PBwqlWrxurVq/PkojG2qVOnYmdnpzJaKooRpCVn4jv3MJcv67BNiaGczV6+7Xwclwo1WeI9iwYVGmgBff1IOL8FKtSFfouhctMHl5upY65vGL/7XcbCzISPO9XG1AiT7agAX0j79++nXLly9yybPn06HTt2ZMKECUyfPp3p06fz/fffF1ENFUV5WqSUXDxwFf8VIWTpTKh85wCbGvlwslIaI+r/m+H1h2MuTCDwD9j7JeizoOMU7fZHU/MHlrvnXDRfbjlHZHwarzesyMRX3ajgYGWUdqhUBU9g06ZNDB48GNCSgm3cuDHPNjqdjs8++ywnXfDdp119fX1p27Ytr7/+Oh4eHrz//vs5T52uWLGC+vXrU69ePcaPH59T1s6dO2nUqBFeXl507NgxZ/m5c+do3749rq6u/PLLLwCkpKTQvXt3vLy8qFevHqtWrTLaeVCUkiT5Tgabv9rH3r8vY5l4AxvzxUztsoEML1fW9lzLqAajML99ARZ0hu2fQqUmMPowtPn4gcH9amwKw/46xoilJ7C1NGXliBbM6t/AaMEdnrce/I4JcPNsvqusddlg+hjNeak+vDr9gZsIIejcuTNCCEaOHMmIESMAiI6OxtnZGdDyxt/NMZPbggULcHR05NixY2RkZNCqVSs6d+4MQGBgIOfOnaNq1ap07dqV9evX4+3tzfjx4zlx4gSlS5emc+fObNy4kVatWvHee+/h7+9P9erViYuLyzlGSEgI+/fvJykpiTp16jBq1Ch27tyJi4sL27ZtAyAhIeHRz42ivECklATtvMihTeHodZIqCb4sbbqXcBcTPm30f7zl9hYmWenak6iHZ4NVKejzB9Tv98CLqOlZOub6XuI3v0uYmwi+6O7OYO9qmJsav3/9fAX4InLw4EFcXFy4desWnTp1ws3NjbZt2xZq3927d/PPP/+wdu1aQAu0oaGhWFhY0KxZM1xdXQF4++23CQgIwNzcnPbt21O+fHkABgwYgL+/P6amprRt25bq1asDUKZMmZwJP7p3746lpSWWlpZUqFCB6Oho6tevz6effsr48ePp0aMHbdq0edqnRVFKjPhbqez9KYDoODNKJYSTUH43k1peoGWVNvx894GlMB/Y9rH24FLDgdDpa7B5cJpfn/PRTN0STERcGr28XPi8uztORuyx3+/5CvAP6GmnGTEXjYuLCwAVKlTg9ddfJzAwkLZt2+Lk5MSNGzdwdnbmxo0bVKhQIc++Ukp+/fVXunTpcs9yX19fo6YLrl27NidOnGD79u1MnDiRzp07M3ny5Edqt6KUdHq95NSGIAJ330DosqiSvJPfmu0jycWBb5t9T7fq3RApMbBuOJxdA2VrwZBtUK31A8uNiEvlyy3B7D1/i5oV7Pj7veZ41yj3wH2MQY3BP0RKSkpOTzklJYXdu3fnpAXu1asXixcvBmDx4sX07t07z/5dunRh3rx5ZGVlAXDx4kVSUrS0oIGBgVy5cgW9Xs+qVato3bo1zZs3x8/Pj5iYGHQ6HStWrKBdu3a0bNkSPz8/rly5AnDPEE1+oqKisLGxYeDAgXz66acqXbCi3CfmeiKrx+3iyJ7blI6/QKLjPD7vshvPpq+y6bVNdK/eDXFqGcxuAuc2QbsJMOrgA4N7epaOn/eG8sqPfhy6FMv/dXNjx3/aFElwh+etB18EoqOjef311wHIzs7mnXfeoWvXrgBMmDCBN998kwULFlClShXWrFmTZ//hw4cTHh5Oo0aNkFJSvnz5nIuxLVu2ZMKECZw9ezbngquJiQnTpk2jQ4cOSCnp1q1bzgfH/Pnz6dOnD3q9ngoVKrB+/foC63327Fk+++wzTExMMDc3Z968eU/71CjKc0mXrSfw71OcOhiHWVYGldK38rO3H2YVnZlz94GlmFBYMRCuBkAVb+j5E5Sv88By94VEM3XzOa7FpdLd05kvurvj7Gj9jFqVP5UuuIj4+voyY8YMtm7d+thlqHTBJd+L1l4wbptvhsaxd85REtItqRB7kjM1drDZ7RYDPAYytuFYbIQpBMyCAzPB3FobZ2/4LzApeLAjIi6Vr7aeY8+5aGqUt+Wr3vVoVfPZ9dhVumBFUV5oWZk6Di48RvCpZCwzU3HSbeCnDocoX7EWy7xn4VneE8IDYMuHEBsK9d6ArtPALu91tbvSs3T84X+Z2fvDMDURjO/qxrutq2NhVnxGvlWALyLt27d/4XpmilIUIs5G4zP/JClZljjHBuLnvp0DtVMZ4TmGd+u9i3lGEmwaA6eWQqmqMGAd1HrlgWX6XrjF1M3BhMem0r2+M593d8elVNEOx+THqAFeCBEOJAE6ILugrxGKoihPW0ZaNv6/HebihSysUxMoZbaT/75ynFpVGrLW+0tcHavD2bWwayKkxkGr/2gXUi1sCiwzMj6Nr7ecY2fwTVzL2bJkWDPa1i7/DFv1aJ5FD76DlDLmGRxHURQFgMuBEez/K4h0nTkucQFsqb+D4BrwYePP6V+nPyZ3rsKyvnDJByo2hn9t0B56LEBGto4/D1xh9r4wJJLPutRheJvqTz052NOmhmgURSkx0pIy2fdrAOHXwDYlFkubrXzbOYhm1duyscUknK3LwcGfwfd7beKNV/+rZX58wCQc/hdvM2VzMFdiUuha9yW+6OFOpdIF9/KLE6PeRSOEuALcASTwu5Ryfj7bjABGADg5OTVeuXLlPesdHR2pWbPmQ4+l0+kwNS3en6ZPm7HaHBYWVmxTGyQnJ2NnZ1fU1XhmXrT2wuO1WUpJ4sV0bp6U6KQZL8Xu4+9GO4msYk3fMn1pbNMYh6SL1LkwF7uUcG6Xa05YzRFkWBV8t0tsmp4VIZkcj9bhZCMY6G5B/fLFr0/coUOHAu+iQUpptH+Ai+H/CsAZoO2Dtm/cuLG837lz5/Isy09iYmKhtntUISEh0svLK+efvb29nDVrlpRSyilTpkgXF5ecddu2bTNKHQqSmJgoFy1aJP/9738/1XILe86Lwv79+4u6Cs/Ui9ZeKR+9zYmxqXLDpN1y9kgfueTtBfLn0a9JrwV15UT/iTIuLU7KtAQpt30q5RRHKWe4SXlu8wPLy8jSybn7w6TbFztknS+2y199Lsr0rOwnaJFxAcdlATHVqB9HUsoow/+3hBAbgGaAvzGP+bTVqVOH06dPA1qPuWLFijkPPgF89NFHKhe7ohQBqZf8sz2EI1uvodfpcUnczR9NdqKr7sKclr/TqmIrLUf79nGQdEObD/XlL8DKocAyA0JjmLI5iEu3U+js4cTknh7PzXBMfox2w6YQwlYIYX/3Z6AzEGSs4z0LPj4+1KhRg6pVqz7Sfj/88ENOuuApU6YAEB4ejpubG4MHD8bT05M33niD1NTUnOM0bNiQ+vXrM2zYMDIyMgA4duwY3t7eeHl50axZs5wUClFRUXTt2pVatWoxbtw4QPswGjJkCPXq1aN+/frMmjXraZ0GRSlyd24ms3biLgK23sAu4Qp3bH5hatedtGs3kA29N9DKrhqseAdWDdQSgg3fC93+W2Bwv5GQxr//PsnABUfJ1ksWDW3K/EFNnuvgDsa9yOoEbDAkyDID/pZS7nySAr8P/J6QuJB81z3ueLRbGTfGNxv/8A2BlStX8vbbb9+zbPbs2SxZsoQmTZowc+bMPDM67d69m9DQUAIDA5FS0qtXL/z9/alSpQoXLlxgwYIFtGrVimHDhjF37lzGjBnDkCFD8PHxoXbt2gwaNIh58+YxevRo+vfvz6pVq2jatCmJiYnodDoATp8+zalTp7C0tKROnTqMHTuWW7duERkZSVCQ9pkaHx//yOdGUYobvU7PybX/cGzfLUx02byUvJ25zffh4FqLpd7L8SxbF479CT5fgV4Hr3wJLf9dYJ72zGw9iw5e4WefUHR6ycedajOirStW5iXjep7RevBSystSSi/Dv7pSym+NdaxnITMzk82bN9OvX7+cZaNGjeLSpUucPn0aZ2dnPvnkkzz77d69m927d9OwYUMaNWpESEgIoaGhAFSuXJlWrVoBMHDgQAICArhw4QLVq1endu3agDaRiL+/PxcuXMDZ2ZmmTbWpwBwcHDAz0z6fO3bsiKOjI1ZWVnh4eHD16lVcXV25fPkyY8eOZefOnTg4FPy1VFGeBzFXE1j12U6O7o+jdHwIV0v9yLfdDtDv5TGs7rEaz2xgQSfYMQ4qN9cm4Wj9YYHB/dClGLr9coBpO0LwrlGOvR+344OOtUpMcIfn7DbJB/W0jZ2LZseOHTRq1AgnJ6ecZbl/fu+99+jRo0ee/aSUTJw4kZEjR96zPDw83KjpgkuXLs2ZM2fYtWsXc+bMYfXq1SxcuPDhDVWUYkaXpefIsuOcOZKAWVYm5TI2Maf1QarVbMTaln/iauMEPl/C4bnacEzfBVCvb4GTcNxMSOfb7efZciaKymWsWTC4CR3dnfLd9nn3XAX4orRixYo8wzN3c8EDbNiwISeNcG5dunRh0qRJDBgwADs7OyIjIzE313oU165d4/Dhw7Rs2ZIVK1bQunVr3NzcCA8PJywsjJo1a7J06VLatWuHm5sbUVFRHDt2jKZNm5KUlER2dnaB9Y2JicHCwoK+fftSo0YNhgwZ8vROhqI8I1EXYvCZE0hiphUV7pzGv8ZGAutm8VGTL3izzpuYhO2Dbb0h/ho0GqQNyRQwCUeWTs9fB8P5ae9FsvSSD1+pxfvtapSoHvv9VIAvhNTUVPbs2ZMzn+pd48aN4/Tp0wghqFatWp71AJ07d+b8+fO0bNkSADs7O5YtW4apqSnu7u4sXryYkSNHUqtWLUaNGoWVlRWLFi2iX79+ZGdn07RpU95//30sLCxYtWoVY8eOJS0tDWtrazZs2FBgnSMjIxk6dGjOPK/Tpk17imdEUYwrK0NHwB9HOHc2HcuMVOzlan7scJxGtduxqcUkXpIm2iQcQeugXG0YugOqehdY3uFLsUzeFETorWQ61CnP1F51qVrW9hm2qGiodMFFJDw8nB49euRcBH0cKl1wyfeitRdg25Jd3BpGAawAACAASURBVDqqI1VnhVPsYba5b+RyXRsmNJtA1yqdEaeXwZ5JkJUGbT6B1h+BmWW+Zd1K1IZjNp2OolJpa6b0rMsr7hUKHO58Hql0wYqiFHsZqVn4zTlI+CVzrFPvYGa2hOmdg+ji3ptZTT6jVFI0LO4B1w5B1VbQ4ycoXzvfsrJ1ev46FM5Pe0PJ1On54OWajO5Qs0QPx+RHBfgiUq1atSfqvStKSRJ2MBy/ZefJ0JtRIcaPNQ22kejhxC8t5uNdoREE/AgHfgQLW+g1W5v0uoBe+NHLsUzZHEzIzSTa1ynP1J51qVau5A/H5EcFeEVRikxKQgb7f/HnaqQptskxZNquY3q3S/T3/BdjGozB5voJ+K0VxIZB/X7QZRrY5Z+e91ZSOtO2h7DhVCQVS1nz28DGdKnrVKKGYx6VCvCKojxzUkrO7wklYP0lsvUmOMXvY3HjbZi51WCs1ScMqdsLtn8Gp5ZB6WowcD3U7JhvWdk6PUsOX2XWnotkZOsZ06Em/+5QE2uLF2s4Jj8qwCuK8kwlxqbh86MfUbEWOCRGca30GhZ2j2ZEwzEM9RhC2NrvYHZTSI/XLqC2HVfgJBzHwuOYtDGIkJtJtK1dnqk9PXAt/2Jl33wQFeAVRXkmpF5yZlMQR3ZGIfWS8klb+KPZXpzdG7Laey6uOuDvN3G/vB8qNoGeP8NLeZ8tAbidlMG0HedZf1INxzxI8ZkdthgbNmwYFSpUyPMgU1xcHJ06daJWrVp06tSJO3fuANrXzw8++ICaNWvi6enJyZMnn3mdfX19832yVlGKQlxUEqvH7+Dgrts4JF4mwva/zOp5iOHdvuCvTn/g+s9GmNsSrh/nYq0R8O7ufIN7tk7PXwev8PIMX7aciWJ0+xrs+bgtXeu9pIJ7PlSAL4QhQ4awc2fePGnTp0+nY8eOhIaG0rFjR6ZPnw5oaQ1CQ0MJDQ1l/vz5jBo16llXWVGKBb1Oz9Flx1n15WHi4/SUSljD7y3nkNizLhte20R/W1dM5nfQUg3U6gRjAomq2D3fGZaOh8fRc/ZBpm45h1flUuz8sC3jurphY6EGIgqiAnwhtG3bljJl8j7+vGnTJgYPHgxoScE2btyYs3zQoEEIIWjRogXx8fHcuHEjz/7Lli2jWbNmNGjQgJEjR+Zkh7Szs+OTTz6hUaNGdOzYkdu3bwNa1sgWLVrg6enJ66+/nvONISwsjFdeeQUvLy8aNWrEpUuXAG1mnDfeeAM3NzcGDBiQk+dmwoQJeHh44OnpqXLZK0Zz+3IcKz/ZwfGARErHnyO4zDTm9w5mYs8Z/Or9DS/5/qAlB0uPh7dWQP9l4OCSp5yY5Aw+XXOGN347THxqJnMHNGLpu82oocbaH+q5+ui7+d13ZJzPP11wtk5H3GOkC7Z0d+Ol//u/x6pPdHR0Ti4aZ2dnbt26BWhpAipXrpyzXaVKlYiMjMzZFrSnRVetWsXBgwcxNzdn9OjRLF++nEGDBpGSkkKjRo2YOXMmX331FV9++SWzZ89m0KBB/Prrr7Rr147Jkyczffp05s6dy4ABA5gwYQKvv/466enp6PV6IiIiOHXqFMHBwbi4uNCqVSsOHjyIh4cHGzZsICQkBCGESiOsPHXZWToOLwrk7IkUzLOysM1cyy/tj9Ghfm82Nf6UUlcOwNwWkHQTmr8PL38OlnmfyNbpJcuPXmXGrgukZuoY2c6VD16uha3lcxW2ipQ6U0aQX/qH+8cHfXx8OHHiRE7637S0NCpUqACAiYkJ/fv3B7Q0wn369CEhIYH4+HjatWsHaN8Y+vbtS1JSEpGRkTmzTFlZWeUco1mzZlSqVAmABg0aEB4eTosWLbCysmL48OF0795djdMrT1VUcDR75x0nKduacndOsKvmWqIal2VGy/l421aBjaPhwnZwqg/9l0OlxvmWc/LaHSZvCiIoMhHvGmX5qnddalZ4flKRFBfPVYB/UE+7KHLRODk55WSUvHHjRk6ArlSpEhERETnbXb9+HReXe796SikZPHhwoZKAPeji0YNyCeWXRtjMzIzAwEB8fHxYuXIls2fPZt++fQ+tg6I8SGZ6NgG/HeT8+WwsM9IwZRmzOgXTp8FAZnuOxubUUtjXD6QeOn0NLUaDad7wE5ucwcKgDPx3HsLJwZJf325ID09ndQH1Makx+CfQq1cvFi9eDMDixYvp3bt3zvIlS5YgpeTIkSM4OjreMzwD2iQda9euzRnWiYuL4+rVqwDo9XrWrl0LwN9//03r1q1xdHSkdOnSHDhwAIClS5fSqlUrHBwcqFSpUs74f0ZGRs7Uf/lJTk4mISGBbt268dNPP+XMN6sojyv8WATLP97F+fNZlI87yN7q37G7TwZ/vLaccZW6YLO4B+ycAFVawugj0OqDPMFdp5csO3KVl2f6cTAym5FtXfH5pD09vVxUcH8Cz1UPvqi8/fbb+Pr6EhMTQ6VKlfjyyy959913mTBhAm+++SYLFiygSpUqrFmzBoBu3bqxfft2atasiY2NDYsWLcpTpoeHB9988w2dO3dGr9djbm7OnDlzqFq1Kra2tgQHB9O4cWMcHR1ZtWoVoH2IvP/++6SmpuLq6sovv/wCaMF+5MiRTJ48GXNz85x65CcpKYnevXuTnp6OlFLN1ao8tvSULHx/8efSVYFNaiKZFmuY+eoVhjV+n2F13sLc779wZB7YlIU3FkLdPvnmjzkdEc+kjUGcjUygpWtZerikMqBb8cxm+rxR6YKLITs7O5KTkx+6nUoXXPIV1/Ze9LuE/4oLZEpzyt7xY0WDrTg29GKK9xRcb12CrR9DwjVoPARemQrWpfOUEZeSyQ+7Qlh5LILydpZ80cODnp7O+Pn5Fcs2F1cqXbCiKE9FSnw6PrP8iIg2xy45hpt2K1neM5YPmk6kn0tbTHb9HwSvh3J1YOhOqNoyTxk6vWTVsQj+uyuEpPRs3m1Vnf+8Ugt7q/znTlUenwrwxVBheu+K8ixJKQnecZ5Dm66hk4KyCdv5q9luajVox7pmC3jpwk6Y2xyy0qHD59DqP/lOwnEmIp7Jm4I4cz2B5tXL8FXvetR5qWR88y6OVIBXFOWBEm6nsHemPzfjLXFIiuBc2ZWsei2LCS3+SxebqojVQyDiCFRro03CUa5mnjLiUzP5764LrAi8Rjk7S37q34DeDdQFVGNTAV5RlHxJveTUujME7o0GncQxeQMLWvrSqnFvNjUYi2PgHxAwECztoPdcaPBOnouoer1kzYkIpu8IITE9m6He1fmokxqOeVZUgFcUJY/YiAT2/BRAbIo1pRLDOOa8gpCutnzr/TveGdmw8FWIuwSeb0GXb8G2XJ4ygiIT+GJjEKcj4mlarTRf9a6Hu7NDEbTmxaUCvKIoOXQ6PceWHufk4QRMs3VYp//N760D6dnsX3xX5x1s9n0Lp5dD6erwrw1Q4+U8ZcSnZjJj9wWWH71GWVsLfnzTi9cbVlTDMUXA6AFeCGEKHAcipZTP5XPxw4YNY+vWrVSoUOGeeVQ/++wztmzZgoWFBTVq1GDRokWUKlWK8PBw3N3dqVOnDgAtWrTgt99+e+b1LuztlooCEB0aw55fj5CQaUOZ+LPsd11NbMuKzG25nHpRwfBbG0hPgNYfQ7txYG59z/56vWTtyetM3xFCfGomg1tW46NOtXG0VsMxReVZ9OD/A5wHntvvZkOGDGHMmDEMGjTonuWdOnVi2rRpmJmZMX78eKZNm8b3338PQI0aNdRTospzITtTx8E/DhH8TwbmmVmYZC9k7ivnGNR8FEOc22K+Yxxc9oVKTbVJOJzq5ikjKDKByZuCOHktnsZVS/N17+Z4uDy3f/IlhlFTFQghKgHdgT+NeRxjKyhdcOfOnTEz0z4jW7RowfXr1x+p3BMnTtCuXTsaN25Mly5dclIKt2/fng8//BBvb2/q1atHYGAgoKUzeO211/D09KRFixY53yaSk5MZOnQo9evXx9PTk3Xr1uUc4/PPP8fLy4sWLVoQHR0NwJo1a6hXrx5eXl60bdv20U+IUmJcPxPF8o92EHQ2izJ3juFX5VuOvGPK4j4reC8+AfPf20LkSeg2A4btzhPcE9KymLIpiF6zA7gam8qMfl6sGdlSBfdiwtg9+J+AcUCBN7oKIUYAI0BL3uXr63vPekdHR5KSkgA4tukacVH551mRUj7WGF8ZFxua9q7y0O2Sk5PR6/U5dbnf/Pnz6dOnD0lJSSQnJ3PlyhW8vLywt7dn0qRJeHt737N9VlYWo0ePZuXKlZQrV45169Yxbtw45s6di06nIz4+nl27dnHw4EGGDBnC0aNHmThxIh4eHixduhQ/Pz9GjBjBoUOHmDx5MtbW1hw6dAiAO3fukJSUREpKCl5eXkyYMIFJkyYxe/Zsxo0bx9SpU1m/fj0uLi7Ex8fnaVN6enqe96G4SE5OLrZ1MwZjtVeXJYnZF0vMnTJYpaWSYbKIX7tc5dUKr9E1oyxOC96A1GvcLteS0FrvkZlaFvz9c/bXS8nByGxWX8wkORNermJGn1pm2CaF4e8f9kR1e9HeY2MyWoAXQvQAbkkpTwgh2he0nZRyPjAftFQF9z+ifP78+ZzH8c0tzDEtIOe7TqcrcN2DmFuYF+pxfzs7O0xMTPLd9ttvv81JwSuEoFatWly7do2yZcty4sQJXnvtNYKDg3Fw+F+vJigoiPPnz+ek+dXpdDg7O2Nvb4+pqSmDBg3C3t6erl27MmLECHQ6HYGBgaxbtw57e3t69OjB+++/j16vx9/fn5UrV+bU7e7/FhYW9OvXDyEELVu2ZM+ePdjb29OmTRvGjBnDm2++SZ8+ffK0ycrKioYNGz7yuXwWiuuj+8ZijPZePhyO76pg0vSlKHPHjzVeW6jYohXrGk7npUPz4PQMcKgIb6+kfJ1XKX/f/ueiEpm8KYjjV1NpVKUUX/WuR72Kjk+tfi/ae2xMxuzBtwJ6CSG6AVaAgxBimZRy4OMW2ObN2gWuK6pcNIsXL2br1q34+PjkfIOwtLTMSdXbuHFjatSowcWLF2nS5H/pIqSU1K1bl8OHD+db7v3fRoQQBeaZL+jbi7m5ec7yu+mCAX777TeOHj3Ktm3baNCgAadPn6Zs2bKP0XrleZKWlMn+n3y5EmmGTUo8CVYrWNc7kfHe0+ickopY2ANSbkGLUdrTqJb3zpiUmJ7Fj7svsuRwOKVsLPhvX0/eaFwJExN1d0xxZbQxeCnlRCllJSllNeAtYN+TBPfiaOfOnXz//fds3rwZGxubnOW3b9/OmX7v8uXLhIaG4urqes++derU4fbt2zkBPisri+Dg4Jz1dzNIBgQE4OjoiKOjI23btmX58uWA1sspW7YsDg4OdO7cmdmzZ+fse3cqv4JcunSJ5s2b89VXX1GuXLl7ctcrJY+UkpC9F1j26V7CIwSl43az1vMH0oY0ZEOn3+hyeCFi7RCwqwDv7YOu0+4J7lJK1p+8zssz/Fh8OJx3mldh3yfteLNpZRXcizl1H3whFJQueMyYMWRkZNCpUyfgf7dD+vv7M3nyZMzMzDA1NeW3337Lc5HWwsKCtWvX8sEHH5CQkEB2djYffvghdetqF7FKly6Nt7c3iYmJLFy4EICpU6cydOhQPD09sbGxybn18osvvuDf//439erVw9TUlClTptCnT58C2/PZZ58RGhqKlJKOHTvi5eVljNOmFAPJd9Lwmbmf6zFW2KXc4qrDcrb0M2Fyy9m0jPgH/ngFkND5W236vPvytIfcTGTyxmACw+PwqlyKRUOaUr/S0xuOUYxLpQsuhtq3b8+MGTPuGdLJj0oXXPI9bnullJzdfJbD2yPR6wW2iTtZ6u1LF++BjH6pNTbbx8GNM1Crs3aHTOmq9+yflJ7FT3tD+etQOA5WZozv6sabTZ5Nj/1Fe4+flEoXrCgvkPibSeyd6Ud0kg0OSRGccvqbqz3K8FOzP6j7z0bY/irYloc3FkHd1+/JHyOlZNPpKL7dfp6Y5AzeblaFzzrXobStRRG2SHlcKsAXQ+oWMeVx6PWSkytPcNwvFnQCq9TV/NXqGG+1eZ/vLSpivnIoJERA46GGSThK3bP/hZtJTNoUROCVOLwqOfLnoCZ4VS6V77GU58NzEeAf9x535dEVpyE7pfBirt5hz6wA4tJtKZV4kYAqK0l7oxZ/ev5G9YDZcG4jlHeDYbugSot79k3OyObnvRdZeDAceyszpvWpT/9nNByjGFexD/BWVlbExsZStmxZFeSNTEpJbGwsVlZWRV0VpZB02XqOLjrM6eMpmGVLTDKXsKjjeUa0/pg3EpMwWdwXstOhwxeGSTj+N9QipWTLPzf4dts5biVl0L9JZcZ1daOMGo4pMYp9gK9UqRLXr1/n9u3bD9wuPT39hQtMxmizlZUVlSpVeqplKsZx43w0PnMCSci2pVT8P+yuvQbHDs1ZXmMmTnu/goijUL2tNglH2Rr37BsancTkTcEcvhxL/YqO/DawMQ2r5J03VXm+FfsAb25uTvXq1R+6na+vb7F9+tJYXsQ2K5CVqePgPH+Cz+mwyMwiS85nSfcoPmk1iU5XTiAW9wZLB3jtN/B6656LqMkZ2fziE8rCgCvYWprx9Wv1eKdZFUzVcEyJVOwDvKIo/3PtxDX2LThDit6W0vFH2FBvI+4vd2N1uUE4bvsc4i6D1zvQ+Ruw/d/TyVJKtp29wTdbz3MzMZ3+TSoz/lU1HFPSqQCvKM+B9NQs/H/xJTTcFKu0VJLMFrCjTzpfNP2WFqfXw563oYwrDNoEru3v2TfsVjJTNgdxMCyWui4OzBnQiMZV1XDMi0AFeEUp5i4duITv8hDSpSWOd/axpvF22nUcwBqTcliveR8yEqHNp9D203sm4UjJyObXfWEsCLiMtbkpX/euyzvNq6rhmBeICvCKUkylJmZwY1M0wWl6bFLucMv2b/zetuY7z2nUDZgNV/yhUjPDJBweOftJKdkRdJOvt57jRkI6bzSuxIRX3ShnZ1mErVGKggrwilLMSCkJ2XmOgI1XyZKlsUvYweqW+3m9w3t8cSce87//BWZW0P1H7aElk//lDLx0O5mpm4M5EBqDu7MDv77dkCbV8k5Wo7wYVIBXlGIkKTaVvT/sIyreBruUaELLLOf2a5WYU/0Lqu2bBrdDwOM1ePV7sH8pZ7/UzGxm7wvjjwOXsTIzZWpPDwa2qIqZqVEnbVOKORXgFaUYkFJyZt0pju6ORi/NsE7ewKq2x2jr3Jnp2fGYrBoMjpXhndVQu8s9++0KvslXW84RlZBOn0YVmfiqO+Xt1XCMogK8ohS5O5EJ7J3px61UO+yTr3HM5W9M327AX6Xex3HnVEyyEqDlGGg/8Z487VdiUpiyORj/i7dxe8men95qSLPqajhG+R8V4BWliOj1khPLjnI8IAGhN8U0bQWrOl7gw5aj6HRqA/j/RZJdDSyHbASXBjn7pWXqmLM/jPn+l7EwM2FyDw8GtVTDMUpeKsArShG4fSmGPT8f4k6mHY5JIeyvvopK3Tryt6yD44aPAQFdvuNkmhvtDMFdSsmec9F8ueUckfFpvN6wIhNfdaOCw4uVokMpPBXgFeUZ0mXrOfJHAGdOZ2CWJcnOXsTa7reY6DWa5gd/h5v/QO2u2iQcpSojDamjr8amMHVzMPsv3Ka2kx2rRrSguauaR1d5MBXgFeUZiQqKwmfecRJ1djgmnGKH+3oad32NFbEvYb1+LNhWgH6LwaN3Tv6YTJ3kxz0X+c3vEuYmgi+6uzPYuxrmajhGKQQV4BXFyLIydRz4dT/nL4JlZhYpJvMIeFPPpGrv4uH7EyRGQpNh8MoUsPrffKd7z0XzfwFpxKSF0svLhc+7u+OkhmOUR6ACvKIY0dWjV9i3KIhUbHGID2BLo+106fgWS8KOY775U6jgAf0WQeVmOftci03lyy3B+ITcwsVOsOK9FrSsoYZjlEenAryiGEF6ahZ+P/oQdt0Cq7QUYq3+5NSgUvzg2Jeqe2ZCdgZ0nAwtx+ZMwpGepeM3v0vM9dWGY/6vmxuu2ddUcFcemwrwivKUhe67gN+qUDKwwi5+L1ta+vF267eYdHoj4thGqN4Oesy6ZxKOfSHRTN18jmtxqfT0cuHzbu685GiFr29EEbZEed6pAK8oT0lKfBr7ZuzjWow1Nql3uO64nLThNZmrb02F7VO18fXXfwfP/jkXUSPiUvlyyzn2no+mZgU7/h7eHO+a5Yq2IUqJoQK8ojwhKSXntv7DwS3XycYc68StbG1/ilENe/HKkb/gTjg0GACdvs6ZhCM9S8d8/8vM2R+GqYlg4qtuDG1VHQszdXeM8vQYLcALIawAf8DScJy1UsopxjqeohSFpFvJ7J2xj6hEO+xSbnLe6W8c3mnMwtjqOO6YCmVqwOAt2tyoBvsv3GLq5mCuxqbS3dOZL7q74+xoXfBBFOUxGbMHnwG8LKVMFkKYAwFCiB1SyiNGPKaiPBNSLzm96jhH98cgpTlmqWvZ1vkyn7i+TPNDf0BGMrT9TJuIw1y7tTEiLpWvt55j97loXMvbsuzd5rSupYZjFOMxWoCXUkog2fDS3PBPGut4ivKsxF2LY8+PB4hJt8c+OZzDVVdRq1sbFl8Kx9rnv1C5hTYJRwU3ADKydfzhf5nZ+8MQCMZ3dePd1mo4RjE+o47BCyFMgRNATWCOlPKoMY+nKMak1+k5tugQJwNTMNGbos9czt5ecUwo0xwP39+1STh6/ASNBudMwuF38TZTNgURHptK17ovMamnBxVLqeEY5dkQWkfbyAcRohSwARgrpQy6b90IYASAk5NT45UrVz7WMZKTk7Gzs3v4hiWIavOzk3EznRv7k0gRZbFPOMP+2uup4VGHsVcP45h6nVvlWxNWcziZltpk1rFpev4OyeREtA4nG8FAdwvql3/0/pR6j5WH6dChwwkpZZP81j00wAshxgDLpZR3nqQSQogpQIqUckZB2zRp0kQeP378scr39fWlffv2j1m755Nqs/HpsvQcmufL2WAdZlmpJMnVXOgFn2NN1TNroFQVbeq8Wp0AyMzW82fAZX71CUMiGdOhJu+1dcXSzPSxjq/eY+VhhBAFBvjCdCleAo4JIU4CC4FdshDdfiFEeSBLShkvhLAGXgG+f4R6K0qRijwdgc/vJ0mS9tgnHGdvg130bu7Nx8fXIFJjwXusNgmHhS0AB0JvM2VzMJdvp9ClrhOTenhQqbRNEbdCeZE9NMBLKb8QQkwCOgNDgdlCiNXAAinlpQfs6gwsNozDmwCrpZRbn0alFcWYMtOyOPDzPkKumGKZkUWC+TwuDyrFzNhSlPebAy4NYeA6cPYEICo+jW+2nWP72ZtULWvDoqFN6VCnQhG3QlEKeZFVSimFEDeBm0A2UBpYK4TYI6UcV8A+/wANn1pNFeUZuHIwjP1LzpEm7LBN8GNviwCGuNfnlWMrQZhA1++h2XtgYkpmtp6FB6/wi08oOr3kk061ea+tK1bmjzccoyhP20MDvBDiA2AwEAP8CXwmpcwSQpgAoUC+AV5RnidpyRn4ztjL5ZvWWKWncNPuT8yGVmLulXQcDi+COt2g2w/gWAmAg2ExTN4UxKXbKXTycGJyDw8ql1HDMUrxUpgefDmgj5Tyau6FUkq9EKKHcaqlKM/OhV1BHFgXTqawxCphN/tfPsUHTpVpdmQF2DnBm0vBvScIwc2EdL7Zdo6t/9ygShkbFg5pwstuTkXdBEXJV2HG4Cc/YN35p1sdRXl2UmJT2PuDD9fj7bBJi+NKuRW49KnBn8ERWEUdhabDoeMksHIkS6dn0cHL/LRXG4758JVavN+uhhqOUYo1lWxMeeFIKQnacJJDO6PRYYl58mYOdLnCZxbWuB9eapiEYzFUbgrAoUsxTN4UTNitZF5xd2JKTzUcozwfVIBXXigJNxLZ88M+olMdsE2N4lTF1TR43ZX5p05gps+CjlO02x9NzYlOTOfbbefZfCaKymWs+XNQE17xUMMxyvNDBXjlhSD1kpPLD3PsQAJIS0hfw7EetxmflEGVoyvAtQP0+BHKuJKl07P4wGVm7blIll7yn461GNVeDccozx8V4JUSL/bybfbMCiA2yxG75MsE1tpIx4YujP5nP8KqFPT5A+r3AyE4ejmWyZuCuRCdRIc65Znaqy5Vy9oWdRMU5bGoAK+UWDqdnsD5/pw+lYGJzoxM3VKC+qQxNfIm5U8HQsOB2iQcNmW4lZjOtB0hbDgVScVS1sz/V2M6eTghDDMvKcrzSAV4pUS6GRzJnjmBJOodsUsK4pDXLt6sbs9HQXuhbE0YvBWqtyFbp2dxwBVm7blIZraesS/XZHT7mlhbqOEY5fmnArxSomRlZnPo130EXRSYZwmSTP4ktp/kh9ALOFxIgXbjofXHYG5F4JU4Jm8KIuRmEu1qa8Mx1cup4Ril5FABXikxIo5dwWfBGVJwwDbhMIebH+DdUlk0++c4VPGGnj9B+TrcTspg2obTrD+pDcf8/q/GdFbDMUoJpAK88tzLTMvCb+ZuLkZYYpmRRaz1XLLeMOHnkBNYZVjz/+3deVSU59nH8e/NMjMgiCAKgoiKighxxSUaLW6pJkazq0lMY15rmyZpFtO82VOzaTSb1uyJWvdoXKIRRRtB4xpBxQ01KKjgjiDrADNzv39o39rWGLfhYWauzzmcMzDM3NcFM7/znIuH5+aOydBhBDYNMzfk8MGqA1htdv6UFMMTfVrgb5K3gXBP8soWLu1gWhZpcw5gVXXwK15Heq8MntQniNudDQn3woBxENCQ9NyzvLLk/DimZ8tQxg6Op3kD2VRCuDcJeOGSKoqtrJmwitwzAVisZRwLnkqLPprJWevxqdcEHlwILftxprSScfMzWbgtj0ZBFj55sCMDE8JlHCM8ggS8cDlZ3+9g/XdHqVZ+mEpXsrtvFs+e20fUvgLo/mdIegG7jz+zN+UyMWU/yVnSVgAAFIdJREFU1mo7jyXF8KSMY4SHkVe7cBklp0s5tiCfPXYH/tYCDkR8y809yvkgJwMV0RGGn9+EY9uRQl5dso09x4rp0aI+Ywcn0KKhjGOE55GAF7We1pqd87aweU0BDhWKV/kSDvQ/yPP522mQr2DgBOg8ioJyG+9+m8n89DzC61qY8kAHbr+pkYxjhMeSgBe1WlHeWVZNTOV0ZTB1yvPZ3vRbBkcX8VjOfmg9CAZOwB4YwZyfjjBx5T7Kq+z8oVdznuzbkgCzvLyFZ5N3gKiVtEOTPu1H0reUoRx+2Ku/If/WXN44vJWgknAYOgvi7mD7kUJenbGe3fnF3Ny8Pm8MiadlWKDR5QtRK0jAi1rnzIETrJq0gUJ7MHVKs8mMT+aBgFw65xwnL/I2gkZ8xlm7hQkLdzJv61EaBpqZPLwDd7SVcYwQF5OAF7WG3e5g88c/kLlb4233pYIZ2PsdZtzR7VhM8TBqJgd+LmFtZiETU/ZTYrUx6pZmPN2/lYxjhLgEeVeIWuF45hFWf7qVEoLxL81gV8cfGK330vpYFfT9K9z8OJnHynhz0yZyinfTrXkIbwxJoJWMY4T4RRLwwlDVlTbWf7iKvTm++FZ7UWz6msCkn5lwbD8+MX3h9vcpNEcycek+5v50hLomxaRh7RncLkLGMUL8Cgl4YZjDG39mzfRdlHvVw69kA/u7rOOJsh1EnQuCe77G0eZu5mfk8e7KNIqtNh7t0YxEy0kGto80unQhXIIEvKhxlaWVpE5cycGTgZirbJyp+ylte2Qx/vRRVMeHod9Ydp315tXPNrHjaBFdmobwxp3xtA6vS1raKaPLF8JlOC3glVJRwAwgHHAAX2itJzlrPeEaslftIm3BISq96mAuWcORm9fz56JMQh0x8EgyRQ07896q/czecoT6dcx8OLQdd7aPlHGMENfAmUfwNmCM1nqbUioQyFBKrdZa73XimqKWqigsY/X4lRw9F4ylqpQTDabTOzaTUUUl0PN/cXR/mm8zTzN+5lqKyqv43c1NeaZ/K4L8fI0uXQiX5bSA11ofB45fuF2ilMoCIgEJeA+itWbv4nQ2rDiBTdXFuzyZ01028HxhFoHB3WHER+yuCuPVrzLYfqSIxOhg3hjSlTYRdY0uXQiXp7TWzl9EqabAOiBBa138H/eNBkYDhIWFdZo3b941rVFaWkpAgGddUKq291x9roqTK05yjkj8yg7zc8QC7g7eTnubFwdjHuFg/T4syrax5oiNQBPcH2uiR4TPZccxtb3nG83T+gXP7Pl69O7dO0NrnXip+5we8EqpAGAt8LbWetHlvjcxMVGnp6df0zppaWkkJSVd02NdVW3tWWvNjlkb2LLuHBpvbFXL8er0I6MKczAn3Ifj1ndYuL+S8Sv2UVhexYhu0Tx7a+wVjWNqa8/O4mn9gmf2fD2UUr8Y8E49i0Yp5QssBGb/WrgL91CYe5qU91IpsIXiX5FHdsxiHjanE+uIgAcXsse/M6/N2kPG4UI6RQczY0gX4iOCjC5bCLfkzLNoFPA1kKW1/sBZ64jaweHQbP38B7Ztt6Ecdah2fIN3x7WMLTmFT+cnOdf1WT5My2PGpvUE+5uYeG9b7unYGC8vOTtGCGdx5hF8D2AEsEsptePC117SWic7cU1hgNNZ+aRM2sA5QvEvO0BO6yX8Xu0gKrA9evh8Fh4LYdzknzhbVsVD3aIZ0z+WIH85O0YIZ3PmWTTrATk8c2O2ajsbJ6Ww+4AP3nYL5d4zaNQ+lbHVdlTf8WQ1vo/Xvstia+5R2kfVY/rILiREyjhGiJoi/8kqrkn+1oOs/nI7ZV4h+JWmcyxhCX+0ZxEadRslfcfxweYSZny3iSA/Xybc05Z7O8k4RoiaJgEvrkqVtZq1E5dzIC8AU7UXxXW+Iq7tGh41haAHzmSxtT1vf76PgrJKHuzahOdujaWev8nosoXwSBLw4orlpO1lzewsrN7BmEs3UJywiCeqcglsN5oD8U/zyopcfsrJpF1UPaY+kkjbxvWMLlkIjyYBL35VZXEF/xi/nNyzIZirbRTV+ZgBTX6kU71WlA1YyZt7Apn+2XbqWnwYd/dNDE2MknGMELWABLy4rAPfb2PtkiNUedfDt+wHqtss5jnHWUy9Xmap/xDempPN6dIzDOvchOd/G0twHRnHCFFbSMCLSyorKGXVO8s5VtYAS1UJRQ2/5N7oLcQ2+Q2Hur7BS6nFbD60h7aNg/ji4UTaR8k4RojaRgJe/ButNbvmb2LT6gLsXsEoazKmuIW84ONDdb9PeDs/nmnTDhNg8eGdu25iaOcovGUcI0StJAEv/t+5/EJWvruSM1Vh+FWepiByDiMCttE4fgQpjf7I6yvyOVmcy7DOUTw/oDUhMo4RolaTgBdorcmYmkr6ZiuaYBy2xYTELWJk3Qjyb5nPg1sD2LjxEAmRdfnsoU50aBJsdMlCiCsgAe/hCrJPsPL9VIp0GH4VhyloMouRdbKp2/UZJloH8sX8fPxN53jzzgQe6NJExjFCuBAJeA9ltzvYNGUlu/Z4oRx1qWYucW2WMTKiE2tbLuKFtVZOFOdxX6fGvDCwNfUDzEaXLIS4ShLwHujkzsOsnLKJUq+G+JXtpLT5TEYFFlDW7W1G7I9n/bIC2jSqy8cPdqRTtIxjhHBVEvAexFZlZ+17S9mfWwdvux+VXtPoFpdCu9ZD+MT8KH9LKcTie46xg+N5qFu0jGOEcHES8B7i6MZ9rJ62kwrvUCzlP2GPmcmfQnzYlfApvX6qx7FzZ7n3wjgmVMYxQrgFCXg3V1VWyT/Gf0fOqRB8bd5YzZ/Qt82PhCWM4vFTA/hhdTGtw32ZPLwDiU1DjC5XCHEDScC7sYOrM0n9JptKn1BMZT/i33IWIyOimFX/a8Zt8cbsU87rd7RhRLdofLy9jC5XCHGDScC7oYqiclLeXkx+SSPMtmoq63zAHc12URr3FL33tCXvYCV3d2jEC7e1pmGgxehyhRBOIgHvZvYs2sSG5JNUe4fhXbGa0JZzGRbdnVcqP2XJRk3rcBPzh3WkSzMZxwjh7iTg3UTpiSKSxy3jdGUkluoSKoImc3/rE6xv9Aodd0Vi8vHi1UGt+N3NMo4RwlNIwLs4rTXb/p5K+voy7F4NUVXLaN5yAc2b38MDR8bw8w64s304L90WR8O6Mo4RwpNIwLuw6tPlzH18FoWOSCyVJ7E3msbQaM0Un3cZubMercICmDc6gW7N6xtdqhDCABLwLkg7NBs/SSY7UwGhaMcC2sUtpyxiJLdkd8bLy4dXbm/F77o3xVfGMUJ4LAl4F3Nm71GSP1pLiVcEFus+aDyVgU3DeapwAlv3BTK4XQQv3x5HmIxjhPB4EvAuwm6zs/b9xew/GIBy1MPBDDrHrWNj0GN0z4mjRcNA5oyKp3uLUKNLFULUEhLwLiB/6wFWfZ5OuU84lvId+EdPo01UJx45Mh6rtS4v3daSR7o3w+Qj4xghxL84LeCVUlOBQcAprXWCs9ZxZzZrNavGLSD3eAN8HP5ovuCWjvv5W9UfeTG3KV3CvZk08jc0CvIzulQhRC3kzCP46cAUYIYT13Bbh1IzSZ21D6tvOOaKLTSMmYk9uD+Djg+nSYMgZo9KoDpvt4S7EOIXOS3gtdbrlFJNnfX87qqqpILkNxeQfy4Ck8MbL59JdO14jucKxnDkTCTPDGjBqFuaY/LxIi3P6GqFELWZ0lo778nPB/z3lxvRKKVGA6MBwsLCOs2bN++a1iotLSUgIOCaHltblGw/yoldflSZQjCVrSU6Zj6b1GCmVfSkc7gPw2JN1Pf715zdHXq+Wp7Ws6f1C57Z8/Xo3bt3htY68VL3GR7wF0tMTNTp6enXtFZaWhpJSUnX9FijVRSUsOzNbzltjcZsPYEOmk5MkwDGnL6P4NBGjB0ST8+WDf7rca7c87XytJ49rV/wzJ6vh1LqFwNezqIxWMbs1WSsKcPm0xhfawotY1cyreJB3i+6iSd/25JRPZth9vE2ukwhhAuSgDdIcd4Zvn9nKYWOplhs5zDX+wjdpBmji/9K7/gm/GNQHI2D/Y0uUwjhwpx5muRcIAkIVUrlAa9rrb921nquQmvNxs+WsStD4fCKxKf6O5q23sK7xY9QZYrl85HxJMU2NLpMIYQbcOZZNMOd9dyu6szPeSRPXE2JVzSWyoP4R0xnn7krH5W/yuP9WzGqZ3MsvjKOEULcGDKiqQEOu4PUDxdwYH8gEI5P9TyCYnN4t/Qx2rdozapBbYgKkXGMEOLGkoB3suM7skn52wbKfKOwWPcSGDWHFQzgkHk4E+6Np3drGccIIZxDAt5JbNU2Ut6Zw+H8MLwJweSYjrVZBVMcf2Fk7wQ+7SXjGCGEc0nAO0HO+kxSp+6iwtQYS8V2ApssZoZjKOExN7P0DhnHCCFqhgT8DVRdXsn3b8zkWGE0vgRgUZ9xuEk91tV9nVcHt6VvXJjRJQohPIgE/A2yd8UGNi44SqWpOeaKjZii1jBVPcKdSd1Y+ZsYGccIIWqcBPx1qigsZenY2ZypiMGkvfH3nczmqHiqYsbx9yEJRNevY3SJQggPJQF/HdK/SWF7SglVvjGYrWmUN97J8oDf89yQLvSLa4hSyugShRAeTAL+GhSfKGDZ2PkU6VjMjir8LR+xokEfuvaayJKkFviZZBwjhDCeBPxV+vHLhezd7I3duwXmqhXkR5zgSIsXmTC4PU1DZRwjhKg9JOCvUEFOHsvfWUaJdyxm2xG8Ar8ipekwRg0Zw2ttwmQcI4SodSTgf4XD4eCHD2dxMCsErZphqVrE7sY+RPZ8jxl9Wso4RghRa0nAX0b+rv2s/nAdZaYYLFXZVDRYQUbrP/GXu7rTTMYxQohaTgL+Euw2O8lvf0leXjR4RWK2zSOjZXMG3/UJj8eHyzhGCOESJOD/Q/bGDNZ9mUmFuRWWyj0UhG/GO+l53usXj79JflxCCNchiXVBtbWSpa99ysmiOLy9QjHpmWR17ctj939OTAPZAFgI4Xok4IFdK9aw5Zt8Ki1tsVgzONYkh073v8WomyJkHCOEcFkeHfAVxaUsefkzzla1w9erDl5ef6fkrpE8e9sz1DF79I9GCOEGPDbFtsxdyM5V1VSZO2KxbuBEnGboo1No0TDQ6NKEEOKG8LiALz51hu9emUaxVydMnMHhO5PwUS/xaKcWMo4RQrgVjwr4NZ98SXZ6MNW+HTBXpFLZuzkjh39JgIxjhBBuyCOS7XRODsvfXESZqQNmx3EcQYvp9de3adWontGlCSGE07h1wDscDr4f9yHHD7XA7tMWc2UKDR76LYP7TpFxjBDC7bltwOduT2fNRxuo8OuA2XYYmm1l6Jg3CfQzGV2aEELUCLcLeLvdzvwX3+Lc2U5oUywmWzLdXv4DCa1GGl2aEELUKKcGvFJqADAJ8Aa+0lqPd+Z621KWs33OEax+PbFUHSCwRzX3jZ4o4xghhEdyWsArpbyBj4H+QB6wVSm1VGu990avZS0rI3vuSqodSSjfJpi9k7lz8ivUryd/RBVCeC5nHsF3AbK11ocAlFLzgCHADQ347N3bSZu4jUq/W7FU7qLl/VH0GvTejVxCCCFckjMDPhI4etHneUDX//wmpdRoYDRAWFgYaWlpV7WIvdqGogBf3700HTEQh6/PVT+HqyotLfWYXv/J03r2tH7BM3t2FmcG/KUG3/q/vqD1F8AXAImJiTopKenqV+rfj7S0NK7psS5MenZ/ntYveGbPzuLlxOfOA6Iu+rwxcMyJ6wkhhLiIMwN+K9BSKdVMKWUChgFLnbieEEKIizhtRKO1timlngBSOH+a5FSt9R5nrSeEEOLfOfU8eK11MpDszDWEEEJcmjNHNEIIIQwkAS+EEG5KAl4IIdyUBLwQQrgppfV//e+RYZRSp4HD1/jwUODMDSzHFUjP7s/T+gXP7Pl6RGutG1zqjloV8NdDKZWutU40uo6aJD27P0/rFzyzZ2eREY0QQrgpCXghhHBT7hTwXxhdgAGkZ/fnaf2CZ/bsFG4zgxdCCPHv3OkIXgghxEUk4IUQwk25fMArpQYopfYrpbKVUi8YXU9NUEpNVUqdUkrtNrqWmqCUilJKpSqlspRSe5RSTxldk7MppSxKqZ+UUpkXeh5rdE01RSnlrZTarpT63uhaXJ1LB/xFG3sPBNoAw5VSbYytqkZMBwYYXUQNsgFjtNZxQDfgcQ/4PVcCfbTW7YD2wAClVDeDa6opTwFZRhfhDlw64LloY2+tdRXwz4293ZrWeh1w1ug6aorW+rjWetuF2yWcf/NHGluVc+nzSi986nvhw+3PiFBKNQZuB74yuhZ34OoBf6mNvd36je/plFJNgQ7AFmMrcb4Lo4odwClgtdba7XsGPgKeBxxGF+IOXD3gr2hjb+EelFIBwELgaa11sdH1OJvW2q61bs/5/Yy7KKUSjK7JmZRSg4BTWusMo2txF64e8LKxt4dQSvlyPtxna60XGV1PTdJaFwFpuP/fXXoAg5VSuZwft/ZRSs0ytiTX5uoBLxt7ewCllAK+BrK01h8YXU9NUEo1UErVu3DbD+gH7DO2KufSWr+otW6stW7K+ffyGq31QwaX5dJcOuC11jbgnxt7ZwHzPWFjb6XUXGATEKuUylNK/Y/RNTlZD2AE54/odlz4uM3oopysEZCqlNrJ+QOZ1VprOW1QXBW5VIEQQrgplz6CF0II8csk4IUQwk1JwAshhJuSgBdCCDclAS+EEG5KAl4IIdyUBLwQQrgpCXghfoFSqrNSaueFa7PXuXBddre+HoxwL/KPTkJchlLqLcAC+AF5WutxBpckxBWTgBfiMi5c42grYAW6a63tBpckxBWTEY0QlxcCBACBnD+SF8JlyBG8EJehlFrK+UvXNgMaaa2fMLgkIa6Yj9EFCFFbKaUeBmxa6zkX9v/dqJTqo7VeY3RtQlwJOYIXQgg3JTN4IYRwUxLwQgjhpiTghRDCTUnACyGEm5KAF0IINyUBL4QQbkoCXggh3NT/ATjEGU3wai5HAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "\n",
    "x = np.linspace(0,5,num=100)\n",
    "# let's train the network for 5 iterations of 25 epochs each ...\n",
    "iterations = 5\n",
    "epochs_per_iter = 25\n",
    "number_of_ephochs_trained=0\n",
    "for i in range(iterations):\n",
    "    model.fit(df.SepalWidth, df.PetalLength, epochs=epochs_per_iter, verbose=0)\n",
    "    number_of_ephochs_trained = number_of_ephochs_trained + epochs_per_iter\n",
    "    y = model.predict(x)\n",
    "    l = model.evaluate(df.SepalWidth, df.PetalLength, verbose=0)\n",
    "    print('Loss after %i epochs: %f' % (number_of_ephochs_trained, l))\n",
    "    plt.plot(x,y, label=\"%i epochs\" % (number_of_ephochs_trained))\n",
    "\n",
    "plt.legend()\n",
    "plt.xticks(np.arange(0, 5, 1))\n",
    "plt.yticks(np.arange(0, 7, 1))\n",
    "\n",
    "def show(plt, title=None, xlabel=None, ylabel=None, legend=True, grid=True):\n",
    "    if title: plt.title(title)\n",
    "    if xlabel: plt.xlabel(xlabel)\n",
    "    if ylabel: plt.ylabel(ylabel)\n",
    "    if grid: plt.grid()\n",
    "    if legend: plt.legend()\n",
    "    plt.show()\n",
    "    \n",
    "def get_coefficient(m):\n",
    "    return model.layers[0].get_weights()[0][0][0]\n",
    "\n",
    "def show_coefficient(m):\n",
    "    print(\"The network's coefficient is: %5f\" % get_coefficient(m))\n",
    "\n",
    "    \n",
    "show_coefficient(model)\n",
    "\n",
    "show(plt, 'Network function for different epochs', 'x', 'y')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Aufgabe*: Was für eine Funktion stellt dieses Netzwerk dar?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Aufgabe*: Vergleichen Sie die Ergebnisse mit der linearen Regression von Übung 2. Fügen Sie weitere Neuronen hinzu und verändern Sie die Aktivierungsfunktion. Wie ändert sich die der Loss und die Netzwerkfunktion?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loss after 100 epochs: 3.765754\n",
      "Loss after 200 epochs: 3.481816\n",
      "Loss after 300 epochs: 3.086211\n",
      "Loss after 400 epochs: 2.719127\n",
      "Loss after 500 epochs: 2.572811\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEWCAYAAABsY4yMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOydd3hUVf6H3zM1vfeQBFLooYYepFcFBAGlCNhQ17Wsrrvr7m/7rrquuooNy0pVEVR0VcACBCmCBKS3FEhCEkivkDZzfn/cIQQIJZDJJOG8z5MnM3fuPeXeO5975nO+5xwhpUShUCgUrQ+dowugUCgUCvugBF6hUChaKUrgFQqFopWiBF6hUChaKUrgFQqFopWiBF6hUChaKUrgFZcghEgQQtx/jft2EEL8LIQoFUI8Zu+y1ck3XAhRJoTQ2yHtQUKIJFv6t9sh/XlCiC113pcJISJtr52FEF8KIYqFEKts2/4hhMgTQpxq7LK0RC4+f4rLowS+iRBCnBBCnBZCuNbZdr8QIuEaj18shPiH3Qp4/fwGSJBSukspF9grE9v5G3nuvZQyXUrpJqW02CG7vwGv29L/3A7pX4Atn1Tb26lAIOArpZwmhAgDngI6SymD7F2Wi2nIw17R/FAC37QYgMcdXYjLITQaek9EAAftUR4Hct11EkIYGiHvY1LKmjrv86WUOddRluu5norWhJRS/TXBH3AC+B1QAHjZtt2P1vo9t09H4DvbPkeB6bbt84FqoAooA74E7gG+rHNsMrCyzvsMoIft9UBgJ1Bs+z+wzn4JwD+BrcBZINq27X7b58HAPuDX9dRpA2ABKmzlal/3WNs+84Atdd5L4CEgCSgE3gBEnc8fAA4DpcAhoBewDLDayleG9quhrS0tg+24EOB/tnOXDDxQJ82/ACuBpbZ0DwJxl7lOKRflZb6GtD8BlgMldeteZx9f2/ElwE/A3+s5J9HAX23XuNqW94O2clht7xfb9u8PbAOKgL3A0KtcT0/gv0A2kAn8A9DXvT7Ai7brcRwYZ/vsnxdd39cvc86uVp7nbPUuBr4AfOp8PtF2PYps+3aq81kY8BmQC+Sfy/9KZa7zeartWh8HZjn6++8w3XF0AW6WPzSBH2m7Yf9h21Yr8IArmijfg9bS7wXkAV1sny8+d5ztfaTtS6FDE+E0ILPOZ4W2z3xsr++2pTvD9t7Xtm8CkA50sX1utG27H01EjwHzr1CvBC4U9Ivfz+NSMfsK8ALCbV/esbbPpqEJUB9AoIlTRN3zVyedtlwo8JuANwEnoIct3RG2z/6CJlLjAT2a4Gy/2rWq8/5qaVcDt9vOt3M96a1Ae8C4Al1tdbxE4Oukt7zOZ0OBk3Xeh6KJ3XhbfqNs7/2vcD0/B9625R+AJrYP1rk+1WgPVj3wMJCF7aF78fWsp27XUp5MW71dgU/P1Q+tQVBuO8aI9uBOBky2suwF/mM7zgmIv1qZbfuWAB1s+wZj+w7djH/q51vT8yfgUSGE/0XbbwNOSCkXSSlrpJS70b4MU+tLRGqebSma4AwBvgEyhRAdbe83SymtwK1AkpRymS3dj4AjwIQ6yS2WUh60fV5t29YZ7cv5ZynlO41Q77o8L6UsklKmAxttdQDtofKClHKn1EiWUqZdLTGbTx0P/FZKWSGl3AO8h/ZQO8cWKeUaqXn2y4Du11LQa0z7Rynl51JKq5Ty7EXH64E7gD9JKcullAeAJdeS92WYDayx1cUqpfwOSEQT2HPUXk+0B/w44Alb/jloonlXnf3TpJTv2s7NEjRRDGzE8iyTUh6QUpYDfwSm287LncDXUsrvbPfdi4Az2i/Ovmi/nJ62lbtCSlm3Y/VKZbYCXYUQzlLKbClla7MQrxkl8E2M7Qv+FZpdU5cIoJ8QoujcHzALuFLH2ia0Ft4tttcJaOI+xPYetC/JxSKZhtbyOkdGPWnPQmt5fXLlGl0XdaNBzgButtdhaBZJQwkBCqSUpXW2XVzHi/N0uka//FrSru/8ncMfrSVdd5+rPrSuQAQw7aL7JB5N4OorTwRa6zi7zv5vo7Xkz1F7bqSUZ2wv3bg2GlqeNFt5/Ljo3rQ1SDLQzm0YmojXUD/1ltn2ELkTzQbMFkJ8bWv03JQogXcMf0b7eXmxSGySUnrV+XOTUj5s+7y+aT/PCfxg2+tNXCrwWWhfwrqEo4n3OepL+y9oFtGHDQxFLAdc6rxvSORHBhB1mc+uNO1pFuAjhHCvs+3iOl4v15L2lcqWC9SgCVbd46+XDLQWcd37xFVK+fxlypMBVAJ+dfb3kFJ2ucb8rjbd7LWU5+K6V6PdWxfcm0IIYds305Zu+PV0Wkspv5FSjkJ7yBwB3m1oGq0FJfAOQEqZDHwM1I0b/wpoL4S4WwhhtP31EUJ0sn1+Gs1br8smYBia73sS2AyMRevU+9m2zxpbujOFEAYhxJ1o9stXVylmNZon7gosa0A0xh5gihDCRQgRDdx3jceBZn38WgjR2xYBEi2EOCcA9dUfACllBlon33NCCCchRDdbvh80IO96udG0bRbCZ8BfbOekMzD3Boq0HJgghBgjhNDbyjRUCNHmMvlnA98CLwkhPIQQOiFElBBiyDXmd9nz3oDyzBZCdBZCuKCFoH5iOy8rgVuFECOEEEa0cNBKtPP9E1qn8PNCCFdbuoOuVlghRKAQYqItHLkSrXPYHqG0LQIl8I7jb2jiCYDNAhiN5o1mof0E/RdaFAdoURCdbT+DP7cdcwztBt5se1+CFj2w1fYFQkqZj+bvP4XW+fUb4DYpZd7VCiilrAKmoP2cf/8aRf4/aJEgp9G80WsWWSnlKrTIjQ/R+hc+R/OQQesY/T9b/X9dz+Ez0Dpes4DVaH0H311r3lfhRtP+JZrlcQqts3zR9RbE9sCZBPwe7ddBBvA0V/4uz0HruDyE1sH+CRdaKFfiVWCqEKJQCHHJOIdrLM8ytHqfQussfcx27FE0D/81tBb9BGCClLLKdv9OQOtoTwdOolkvV0OHdq9noUU9DQF+cY11bXWc6ylXKBSKRsc2kG+5lPI9R5flZkS14BUKhaKVYleBF0J4CSE+EUIcEUIcFkIMsGd+CoVCoTiPXS0aIcQStHjs94QQJsBFSllktwwVCoVCUYvdBF4I4YE2Ei1SKqNfoVAompwbnRjpSkSi9aovEkJ0B3YBj9sGItQihJiPNtcKzs7OvcPCwi5J6FqwWq3odDdXl4Kqc+vnZqsv3Jx1vhGOHTuWJ6W8eGQ8YN8WfBywHRgkpdwhhHgVKJFS/vFyx8TFxcnExMTryi8hIYGhQ4de17EtFVXn1s/NVl+4Oet8Iwghdkkp4+r7zJ6PyZNokyTtsL3/BG0CLYVCoVA0AXYTeCnlKSBDCNHBtmkE2kALhUKhUDQB9vTgAR4FPrBF0KSiTYWrUCgUiibArgJvm1q1Xm9IoVAoFPZFdVUrFApFK0UJvEKhULRSlMArFApFK0UJvEKhULRSlMArFApFK0UJvEKhULRSlMArFApFK0UJvEKhULRSlMArFApFK0UJvEKhULRSlMArFApFK0UJvEKhULRSlMArFApFK0UJvEKhULRSlMArFApFK0UJvEKhULRSlMArFApFK0UJvEKhULRSlMArFApFK0UJvEKhULRSlMArFAqFAyk6U8W2lDy7pG2wS6oKxWWwVFupOFNN5Zka21/1hf/LtdfVVVY6xwcT3tnX0UVWKOxCQXkV721OZcm2Exj0Onb8fgRORn2j5qEEXtFgLDXWC0S5ovwiwbaJdMUFAq69rqmyXjFto5Mes4sBS40kZXcOHfsHMWhaDE6uxiaqnUJhX/LKKnl3cyrLfkzjbLWFW2ODeXR4TKOLOyiBv2mxWKy1QlyvSNcjzuf2u6pImzWRNrsYMbsY8PR3xuxqxKnONrOrAScX4wXvTc4G9HrNNayptpC45gQ/f5NO2sF8RsztTERX1ZpXtFxySit4Z1Mqy3ekUVVjZUL3EB4dHk10gLvd8lQC34KRVsnZ0ipNeGstjvPiXHHB+zoifaaGmkrLFdO+WKQ9/DSRNrsYLhTq2v8GnFyNmFzOi/SNYDDq6T8piujeAXy/6DBfvbGXfhPaIZ3lDaetUDQlp0sqWLgphQ93pFNtsXJ7j1B+MSya6AA3u+etBN7BWCxWqmwCXFG35XyBzVG/UFdXSg6t3HLZtA1mvU2MNSH28HO+QJSdXI2YnM+Lc93P9Ibm0f/u18adO37bm4TlR9jxv+O4h0JV/xpMzurWVTRvsovPsjAhhY92ZmCxSib3DOWRYdG083NtsjKob0kjYLVYqTx7roPwnDDbRPrsFbzp8hqqr9KSNhh1NgtDE153Xyf8w90wuxjJOn2Sjl1iLmhBN0eRvlGMJj0j7+lMQIQHWz5J4pN/JTLuoVi8g5rui6JQXCuZRWd5KyGZlTtPYpWSKb00YY/wbfr71a4CL4Q4AZQCFqBGShlnz/xuBKtVUnUlq+Mib7qijn9dXdFAkfZxwr+NJtJmV8NFVocRJ1fbe2cDeuPlRTohIZNuQ9s09qlolggh6D4ijJN5yZxOrGbV84mMnNeZyB7+ji6aQgFARsEZ3kxI4ZNdGQBMiwvj4SFRhPm4OKxMTdGCHyaltE+Q50VcItIXdxLWF91h21Z1FZHW20TaqY5I+7Vxu9DyqBXxOr60i/GKIq1oGK6BgmnP9GHd2/tZu3A/vcdF0HdCJDqdcHTRFDcp6flneGNjMp/uPolOCO7qE85DQ6MI9XJ2dNFavkUjpeSDP22ntMjKwRUbr7iv3qCztZi1iA43LzO+IW61YnxJpEedFrbBDiFMiuvD3ceJyb/uxQ8fHWPX2jRy08sYdW9nFUqpaFKO55XzxsZkVv+ciV4nmNVPE/ZgT8cL+znsLfAS+FYIIYG3pZTvNHYGQgjadPIhOzuTqPZt64jyha1os4sBg0mJdGvBYNQz7O6OBLT1YPPHx1j1fCLjH4rFN9T+kQmKm5uU3DJe35DMF3syMep1zB3QlgeHRBLo4eTool2CkNJ+YWdCiBApZZYQIgD4DnhUSvnDRfvMB+YDBAYG9l6xYsV15VVWVoab28315VZ11jiTJ8nYIrFUQ2g/gWd467Fr1DVuPmSWWfkypYod2RaMehgeZmBsOyNeZsdasMOGDdt1uf5Nuwr8BRkJ8RegTEr54uX2iYuLk4mJideVfkJCAkOHDr2+wrVQVJ3PU15cyTfvHCA7pZgeI8MYMDkKXSPE4zsadY0dz9FTpSzYkMSa/dk4G/XMGdCW+we3w8/N7OiiASCEuKzA282iEUK4AjopZant9Wjgb/bKT3Fz4+ppZtKverJ1VRJ7vs8gN6OMMfd3wdnd5OiiKVooh7JKeG1DEmsPnMLNbOAXQ6O4Lz4SH9eWc0/Z04MPBFYLIc7l86GUcp0d81Pc5OgNOm6Z0QH/CA82fXiUlc/tZNyDsQREeDi6aIoWxIHMYhasT+LbQ6dxNxt4dHg098W3w8ul5Qj7Oewm8FLKVKC7vdJXKC5Hp4HB+Ia6snbhfj77926GzupAxwHBji6WopmzN6OIBeuTWH8kBw8nA0+MjOGege3wdGm50VktPkxSoaiPgAgPpv++D9+8d5D1Sw6Tc6KEQdNiWs3oXkXjsTu9kAXrk0g4mouns5GnRrVn7qC2eDi1XGE/hxJ4RavF2d3ExMe68+Pnqez5Lp28zDLGPNAVV8/m0TmmcCyJJwp4dX0Sm5Py8HYx8vSYDswZEIF7KxD2cyiBV7RqdHodg+6IJiDcnQ3LDrPq2Z2MfTCWoEhPRxdN4SB2pOazYEMSW5Pz8XU18btxHbm7fwSu5tYnh62vRgpFPcT0CcQnxJU1b+1j9Uu7GXxne7oMDsEWBKBo5Ugp2Z5awKvrj7E9tQA/NzP/d2snZvYLx8XUemWw9dZMobgI31A3pj3Th+/eP8SmD4+Sk1bCLXe1V9NQtGKklGxLyefV75P46UQBAe5m/nhbZ2b2Dce5uYxsr66AglQI7NzoSSuBV9xUOLkaufWRbvz0ZSq71qaRn1nO2PldcfdpfsPMFdePlJIfkvJYsD6JXWmFBHk48deJXbizT5hdlsZrMGcL4di3cPRrSF4PJld48gjoGjcIQAm84qZDpxP0nxRFQIQH3y8+xKrndjLmga6Etvd2dNEUN4iUkoSjuby6Pok9GUWEeDrx99u7Mj2uDWaDg4W9KB2OrNFE/cRWkBZwC4TYadDxVrSpuxoXJfAKh2MpK6cqNYWq9Ax0TmZ07h7ovb0wR0cjGrlFU5fIHv5M/W0caxfu54tX9jBoajTdhrVRvnwLRErJ+sM5LNiQxL6TxYR6OfPPyV2Z2tuBwi4lnNoPR9fAka+01wB+HWDQY9DhVgjt3eit9roogVc0GZbiYipTUqhMTqYqJYXKlFQqU1Koyc6ud39T27b4zJuL56RJ6JztMwWrT7Ar034Xx/eLD7FlZRI5aSUMndURY3PxZxVXxGqVfHvoNAvWJ3Eou4RwHxdeuKMbk3uFYnTEXESWGkjfprXUj3wNxemAgLB+MOpvmqj7RTdZcZTAKxoVKSWWggIqk1OoTEmmKjmFytRUKlOSseSeX/dFODlhjozEJS4Oc1QU5ugojOHhUFODpaSU6syTFH60glN/+Su5/3kFrxl34T1zpl3KbHI2MO7BWHatO8GOL49TkFXOuAdj8fBrPvN6Ky7EapWsPXCK1zYkceRUKW19XXhxWncm9QhpemGvLIOU9ZqoH1sHFUVgcILIYTDkaWg/FtwCmrZMNpTAK64LKSU1p09TmZxCVWqKTdBTqEpJwVJUVLufzs0NU1QkbvGDMUdHY46OwhQVhTEk5Cr2Sz88p0zh7O7d5C9aRP7b75D/3/fx6N2LiqAgnDp2bNT6CJ0gbnw7/MM9+O79g6x8bidj7utKWGefRs1HcWNYrJI1+7N5bUMSx06XEenvysvTuzOxewiGphT20tNwbK0m6qkJYKkEZ2/oMB46joeo4VrHqYNRAq+4ItJqpTozs1a8NVtFa5lby8tr99N7emKKjsZ91CjMMdGYoqIwR0VhCAy8bk9bCIFL79649O5NVXo6BUuXYV21iuO3T8alf3985s3F7ZZbGtWnj+jqy7RnNF/+y9f20P/2KHqODle+vIOxWCVf7s3itQ1JpOSWExPgxoIZPbk1Nhh9Uy3XmHtM6yA9sgZO7gQkeEVAn/s1UQ/rD/rmJanNqzQKhyFraqhKz7igNV6ZkkxV6nFkRUXtfnp/P8yRUXhOmogpOhpzVDTmqEj0vr52FUFTeDhB//cHjvbsQddTpyhYtpyTDz2MqV07fObOxXPSxEbz6T39XbjjN3FsWHaYH1enkJNWwvA5nTA5qa9LU1NjsfLFnize2JhMal45HQLdeWNmL8Z1DbL/OrxWK2Qmah2kR9ZAfpK2Pbg7DPu91loP7ALN+OGv7tibDGtVFVUnTlCVmnqBT1514gSyurp2P0NIMObIKFz79MUUHaUJeWQ79F5eDiw9SFdXfO+7D585cyhZ9w0Fixdz6i9/IfeVV/C66068Z87EGHDjfqfRrGf0fV0IiPDgx8+SKcg+w/iHYvEKdGmEWiiuRrXFyuqfM3ljYzJp+WfoFOzBwtm9GN3ZzsJeXQHHN2kdpEfXQnkO6AzQNh76zocO48ArzH75NzJK4Fsp1rNnqTp+3NYSt9krySlUpaeDxaLtJATGsDDMkZG4DbkFk601boqMQu/meP/wSgijEc8Jt+Fx262c3bWL/EWLyX/7HQr++z4et92Gz7y5OHXocGN5CEHPUeH4hbnx7bsHWfV8IqPu6Uzbbn6NVAvFxVTVWNmUUc0fX0ogo+AsXUI8eOfu3ozsFGg/YT9TAEnfaqKevB6qy8HkBjGjtKiXmFHg7NiGzfWiBL6Fcy6G/IKolZQUqjMztThcAL0eU0QE5ugo3MeOwRwZZRPySHROLXsEpxACl7g4XOLiqEpLo2DJUopWr6Z49WpcBvTHd948XAcPviGfPqyjD9N+H8e6tw/w9Zv76HNbO/qMb4toKu/3JqCyxsInu07y5sYUMouq6N7Gk79M6MLwjgH2sf4K02zx6V9D2jbboKMg6H6nJurtBoOh5c86qgS+hWApKtLCDc/FkCen4HfoEMcKC2v3EUYjpnbtcIrtiufk2zFHRWGKjMTcti3C1PJWo2kopogIgv70R/wff4zClSspXLacjAcfwhQZed6nv84HmoevM1N+3YtNHx5l51fHyU0rYeS9XTA7q6/QjVBRbWFVYgZvJaSQVVxBjzAv7oyy8ujUQY0r7FLCqX2aoB9ZA6dtg478O0L8E5qoh/S066AjR6DuzmaElBJLfv751nidwUCWvDox5M7OmCMjqWofQ/jAQVroYWQkprAwhEFdUr2nJ34PPIDv3LmUrFtHweIlnPrzn8l95RW8Z9yF94wZGPz9G5yuwaRn+NxOBLT1YMvKJFY9t5NxD8XiG+Jmh1q0biqqLXz0UzoLN6VwuqSSuAhv/jW1G/HRfmzatKlxxN1SDWlbz/vpxRmAgPD+MPofWiepb9SN59OMUWrgAKSU1Jw6RWVKKlUpyRfGkBcX1+6nc3PDHBWF2y231A4GqhtDnpaQgF8zWn2+uSFMJjwnTsRjwgTO7NxJweIl5L21kPx338NjwgR85s7FqUP7hqUpBLFD2+Ab6sa6dw/wyb92MWJOJ6J7O2YgS0vjbJWFD3ak8fYPqeSWVtK3nQ8vT+/BwKhGisKqLIXk7zVRT/oWKorB4AxRw2Do77RBR643Tx+KEng7UhtDnpxcJ2pFE/ILYsi9vDBFR+E+ZswFg4EMAXbyH28yhBC49u2La9++VJ04QcHSpRSt/pzizz7DdeAAfObNwzU+vkE+fUiMF9Of6cO6d/bzzbsHyE0Pp9+kKPuH7rVQzlTVsHx7Gu/8kEpeWRUDIn15bUZP+kf63njipadsfvoaLQLGUgXOPtDxNm0Sr8hhYLo5o5+UwDcC52LIL7RVLo0hN/j7Y4qKwvP222tF3BwVhcG3EW5yxTVhatuWoD/9Cf/HHqNw5SoKly8nY/6DmKKi8Jk7B8+J1+7Tu3mbmfxkLzavPMbub9LJTS9l9H1dcXJrPUu+3ShllTUs+zGNdzenUlBexeAYPx4bEUOftjcwQlhKyD16ftBRZqK23budLZRxvDb3SzMbdOQI1BloALUx5Cl1h+YnU3kiDeqLIe/bz+aPa1Erek+1TFxzQe/lhd/8B/Cdp/n0+YsXc+pPfyb3PzaffuZMDH5X/ymvN+oYOqsjAREebFpxlJU2X94/zL0JatF8Ka2oZsm2E7y35ThFZ6oZ0t6fx0ZE0zviOoXdatFGjx75WvsrSNG2h/SC4f+ndZIGdGrWg44cgRL4erCePUtlauqFrfGU1EtjyMPDMEdG4TZ0aG1r3BwZic61eceQK85zRZ9+os2nb391n75zfAg+oa6se/sAn72wi6GzO9KhX1AT1KB5UXy2msVbT/DfLamUVNQwrIM/j42IoWf4dcy1X31Wm+flyNfaJF7luaAzQrtbYMAvtJa6R0ij16E1cVMLvKWs7ILW+Lk48uqsrPMx5AYDpvDw8zHk5wYDtWvX4mPIFeep69NXHj9O4bJlFH22muJPP8N10CCbT3/l0L2gdp5M/30fvnn3AN8vOkROWgkD74hG74hpa5uY4jPV/HfrcRZtPU5pRQ2jOgfy2PAYYts08FfrmQICT22AFe9CygaoPgNmD9ugo/Hafyf1S/hauSkEvqaw8JKh+ZUpKdScPl27jzCZMLVrh3P3bnjeMeX8YKCIiJsihlxxHnO7dud9+o9XUvjBB2Q88ACm6Cgtnn7iRHTm+gfBuHiYmPhED7Z9msy+DSfJyyhjzANdcfFonfdQYXkV7289zuKtJyitrGFslyB+OTyarqENEOHCE+fnT0/fRidpBfcQ6DFT6ySNiAdD6zx/9qb1CLyU1OTm1rFUzrfMLfn5tbsJZ2fMUVG49u+nDc2P1mwVY1gYQq8WeVCcR+/lhd+D8/G9Z57m0y9azKk//snm08/Ae+aMejvI9Xodg6e3JyDCg4TlR1j13E7Gzo8lsJ2HA2phH/LLKnlvy3GWbjvBmWoL47oG8ejwGDoFX0MdpYTsPedFPeegtj2gCwx+isTyIOJuu0/56Y1Aixd4WVND2tx5+B8+TNKZM7Xbde7uWgz50CGarWLr7DSGBNt1GThF6+MCn37HTxQsWULeG2+Q/+67eEycgO/cuZhjYi45rkO/IHxCXFm7cD+fvbSLITM60HlQy/aMc0sreXdzKst+TKOixsJt3UJ4dHg07QOv0qlcUwVpW2xrkq6BkkwQOggfAKP/qU236xMJQFlCghL3RqLFC7wwGDD4+VER15t28YMxRUVijo7BEOCvYsgVjYoQAtf+/XDt34/K1OMULFtK8erPKf7kU1zj4zWfftDAC+47/zB3pj/Th2//e4CNy46Qc6KEwdPboze2rEZGTmkF72xKZfmONKpqrEzsHsIvh0cTHXAFYa8ogeTvNFFP+g4qbYOOokfAsD/YBh2pEGF7YneBF0LogUQgU0p5mz3yaPPqKyQnJOCjRnUqmghzZDuC//xn/B97jKKPP6bggw/IuP9+zDHR+Mydi8eECbU+vZObkdse7cGOL1LY/U06eSfLGPdgLK5ezX8yq9MlFbyVkMJHP6VTY5VM6hHCL4dFE+l/mekZSrLqDDr6AazV4OILnSdooYyRQ2/aQUeOoCla8I8Dh4HWY0AqFDYM3t74PfQQPvfeS8maNRQsXkL2//2RnJf/g/fMmXjPuAuDry86nWDA5Gj8wz1Yv/QwHz+7k7HzuxIS3Tynoc0qOsvCTSms2JmBxSqZ0jOUR4ZF09bvohBgKSH3yPn49Kzd2nafSOj3oNZJGtYPdKp/yxHYVeCFEG2AW4F/Ak/aMy+FwpHoTCa8br8dz0mTOLNjBwWLFpP3+uvkv/MOnpMm4jN3LuboaKJ7B+Ad7MLat/bzxcs/Ez89hq5DQpuNnZhZdJY3NyazKvEkVimZ2rsNvxgaTbhvnVa31QIZO86LeuFxbXtobxj+R03U/TsqH70ZIOS5eG97JC7EJ8BzgDvw6/osGiHEfCO4o6kAACAASURBVGA+QGBgYO8VK1ZcV15lZWW4ud1cs/qpOjdv9KdO4bJhA84/bkdUV1PZpTNnRoykqlNHLNVwcrukLAu82kFwb4HOcKkgNlV9c89Y+Sq1mi2ZNQAMbmPgtkgjfs5aX4HOUol34R788nbgm78TU3UJVmGg0Lsb+b59yfPrS5W5cfz0lnSNmwPDhg3bJaWMq+8zuwm8EOI2YLyU8hdCiKFcRuDrEhcXJxMTE68rv4SEBIbeZB68qnPLoKawkKIVKyj48EMsuXmYY2LwmTcP91tvZdd3mez8+gT+4e6MfbArHr4Xritr7/qm5ZfzxsZkPtudiU4I7uobxkNDogjxcobyPG0E6ZE12qCjmrNg9oT2o7VBR9EjwanxndeWeI0diRDisgJvT4tmEDBRCDEecAI8hBDLpZSz7ZinQtHsMHh74/fww/jcdx8lX6+hYPFisv/wB3L+8x/azbgLn7vHs3FVOqueS2TM/V1o0/EGJuK6RlJzy3h9YzJf7MnCoBPM7h/BQ0OiCLJkwaH3NFHP2A7SCh6h0OtuTdQjBqlBRy0Iuwm8lPIZ4BmAOi14Je6KmxadyYTX5NvxvH0SZ7ZvJ3/xYvJeex1hfpch4+9kpy6e/726hwFToukxMswuvnxyThmvb0jif3uzMBl0zO0fwS87luKT8Rl8sAZyDmk7BnaFW57WRD24u/LTWygtPg5eoWhpCCFwHTAA1wEDqExJoWDJUoq/+JhuNR9zLP5XbPsUctJKGH53p0bLM+l0KQs2JPPVvizcDZJnu+UxyelnnJO+hd1ZIPQQMRDGPKcNOvJu22h5KxxHkwi8lDIBSGiKvBSKloQ5Korgv/0V/ycep3DFCkwfvoeLS0+S5STyjmTjP+jG0j9yqoTX1iez+UAKo437+DrkMB1Lt6M7WgpGF23QUYc/Qfsx4GJ/a0jRtKgWvELRDDD4+OD/i1/ge//9BH71NT4ffc6emtGUfa1j37EldH5gAgafaxfgg1nFLPvmRwzJ65hp2M0Cp4PoZQ1U+EPXybZBR0PA6Hz1xBQtFiXwCkUzQmcy4TVlMn0n307Et1tZtyqLzamhZM5+lu69XfGdNwdz1GUWipaSY/t/4uCGj4gs2MTzulQwgsU7Cn3nRzRRbxOnBh3dRCiBVzQ5UkpOnznNwfyDHC8+jsVqQSd0CCG0/5z/f/G25NJkco7moBO6y+5Td3ttWnVeXy6PC7bb9hdCoEP7DEHt63PbhRAXvK79b0sPuPT1RcddXL5z2wNHDyJcn0DVQWdSxHhKj+6n08SpeMb3xXfePFz690fYBh3l7PwUjq6hfU027YFsz65U9PojTl0novdv2MLiitaDEnhFk5B3No+tmVvZkrmFn079REFFwfUntr3xytUSEJ6Cbm2H0o8JZMc/TfcD71B+z72kBwi+jYOdHcFiEFhCTKCPwWRyRqfTo8tdg0hYe+FD5XIPvLqfIep9gF38EL7SA+pqD9P6Hszn0svOz2br9q31PiQvKFc9D/TLPkzr5lHPg/ziBsIFedbTWLjsg/kK5/pK59GgMxDpGdno944SeIVdqLHWsD9vP5tPbmZL5hYOFxwGwNfJl4EhA4n1i6WLXxdivGIw6U1IKbFICxKJlBKrtCKx/Zey9vWWrVsYMHDABdslEqv1/P7njpVSYuXC4+u+rrufRWpLMVqlVXstOX/MRWW6YlnrbKt9X6d8dctx8XEXl09KyfETxwmPCMfasQx5LAH9zv4c6P406db/0mHfIe5fI5ieoGf3gCgKJ8Zi9XS6pN4Wa52yntt+uX0uU4669b64/Jc7Hxdfl/rOoxXtPCGpfV1VVcW+E/su2V7f9Tl33Vo6vk6+JNyZ0OjpKoFXNBq5Z3LZkrmFLZlb+DH7R0qrStELPd39u/NYz8eID42ng0+H2hbSxRgxXjUPT4MnAS4BjV305kl+Cin7FhD181eQvh2Q5IdsYPWpJyit/CXvD6xhWGQJ/XatY+j6rYgtaXjefjs+c+dgjmz81mBT0dCRrHUfQHUflnUfCnUfbOceRkDtgw3Qtl/08LvcA7H2AX6F7bUPoMs0FuqW1aS3z+AxJfCK66baWs3enL21on608CgA/s7+jAwfyaDQQQwIGYCHSU0kek1YrZD1Mxz5SptyN/cIUYAMiuVk7KMsyO7AqpNeBPk7MdfkRrfUMlwtMQQvmITIOkHB0qUUr15N0ccf4zZkCD73zMOlX79mM5GZvThngSBAj+pArosSeEWDOFV+qtZL3569nbLqMgzCQI+AHjze63EGhw6mvXf7Vi8qjUZNJRzfbBP1tVB2Sht01HYQsvc8lqZ68mVxWxJ3FhLoYebPE6KY0Tccs0HHwR8y2bwyiVXPJzL+oViC//53/J94gsIVKyj88CPS592DuUMHfObNw+PW8ejU2sI3HUrgFVek2lLNzzk/syVzC5szN5NclAxAoEsgY9qOIT40nv7B/XEzqdn/rpmzRdoKR0e/hqTvoaoUjK7aoKOOtyJjRpOQUcOC9Un8nF5EsOdZ/j6pC9PiwnAynm+hdh3SBt827qx7Zz+f/CuR4XM6ERMXiP8jj+B7//2UfPWVNu/NM8+Q+/LLeM+ahded0zF4ezuw8oqmRAm84hKyy7LZnLmZrZlb2Z69nTM1ZzDoDPQK6MWTvZ8kPjSeaK9o1UpvCEUZWgv96NdwYgtYa8A1ALpO0eZPbzcEaTCz4UgOC94/xN6TxYR6OTOns4k/zByK2VC/9RAc5cn03/fhm3cO8O17B8lJK2XA7ZHozGa87rgDzylTKN+6jYIlS8h95RXyFi7Ec/Lt+MyZg7lduyY+CYqmRgm8gipLFbtO76q1XlKKUwAIcQ3h1shbiQ+Np19wP1yNrldJSVGLlHD6oLYgxtGvIXuvtt2vPQz4pSbqoXGg0yGl5LtDp1mwIYkDmSWE+Tjz/JRYpvRqw7YtP1xW3M/h6mlm0q96smVVEnu+Sycvo5TR93fB2c2EEAK3+EG4xQ+iMimJ/CVLKP70M4o+WoHb0KH4zJuHS7++6mHdSlECf5NysvRkraDvOLWDszVnMeqM9A7szeSYyQwOHUw7z3bqi98QLDWQ/qNtTdKvoCgdEBDWF0b+VRN1v5ja3a1WybcHsnl1fTKHs0uI8HXhhandmNwzFKO+YYty6w06hszoQECEB5s+PMrKZ3cy7sFYAiLOd3CbY2II+cc/CPjVryj88CMKP/qI9HnzMHfqhO+8uXiMG4dQPn2rQgn8TUKlpZLEU4m1ES8nSk4AEOoWysSoicSHxtM3qC8uRrUgcoOoKofk9ZqoH1sHZwtBb4aoYTD419B+LLgHXnCI1SpZe+AUr21I4sipUtr5ufLStO5M6hGCoYHCfjGdBgbjG+rK2oX7+ezfuxk6qwMdBwRfsI/B1xf/R3+J7wP3U/zllxQsXkLWb39Hzosv4T17Nt53Tkfv1TzXilU0DCXwrZj0kvRaL33nqZ1UWCow6Uz0CerDnR3uJD40ngiPCNVKbyhlOTY/fQ2kbARLJTh7a2LeYTxEDQfzpZ3OFqvkq31ZvL4hmaScMqL8XXn1rh7c1i0Eva7xrkFAhIfmy797gPVLDpNzooRB02PQX/Tw0Dk54T1tGl5Tp1K+ZQsFi5eQ+5//kPfWW8qnbyUogW9FnK05e0ErPb00HYAIjwimxExhUOgg+gT1wdmgZhBsMHnJ5+PTM34CJHiFQ9y92vzp4QNBX//XqcZi5at92by2IYmU3HLaB7qxYEZPbo0NblRhr4uzu4mJj/dg2+oU9n6fQV5mGWMe6Iqrp/mSfYUQuA0ejNvgwVQcPUbBkiUUf/IpRSs+Pu/T9+2jGgItECXwLRgpJceLj7Mlc0ttK73KWoWT3ok+QX2Y1WkWg0MHE+YR5uiitjysVsjcpXWQHvka8o5p24O6wdDfaS31oNgrrnRUY7Hy+Z4s3tiYzPG8cjoGufPmrF6M7RKEzk7CXhedXkf81BgCItzZuPQIq57dydgHYwmK9LzsMU4d2hPy7D8JeLKOTz93LubOnfCdNw+PsWOVT9+CUALfwjhTfYadp3ayOXMz32d9T356PgBtPdoyvcN0BocOpndQb8z6S1tqiqtQXQHHf9BE/ehaKDttG3QUD33uhw7jtFb71ZKxWFm9O5PXNyaTXnCGzsEeLJzdm9GdA5tE2C+mfZ8gfII1X371S7u55a72dBkcesVjDH5++D/2KL7zHzjv0//mt+d9+unTlE/fAlAC38yRUpJanFrbSk88nUi1tRpngzNRxige7v0wg0IH0ca9jaOL2jI5WwjHvtVEPXk9VJWByQ2iR2pRLzGjNH/9GqiqsfLp7pO8sTGZk4VniQ315N05cYzsFOBwe8OvjTvTnunDd+8fJOGDo+ScKGHwXe0xGK8cglnr099xx3mf/uWXyXvrLbwmT8Znzt2Y2rZtmkooGowS+GZIeXU5O7J31Hrp2eXZAER5RjGz40zi28TTK6AX2zZvY2jHoY4tbEukKB2OrLENOtoK0gJugRA7zTbo6BYwXPsvoMoaC6sST/JWQgqZRWfpHubF3yZ1YVgHxwt7XZxcjdz6SHd++l8qu9alkZdZzrgHu+Lm7XTVY4VOh9stt+B2yy2aT794MUWrVlH40Ue4DRuGz7y5uPRRPn1zQwl8M0BKSXJRcm0rfVfOLmqsNbgYXOgf3J/7Y+8nPjSeELcQRxe1ZSIlnNp3XtRP7de2+3WAQY9roh7SC3QNC1GsqLawMjGDtxJSyC6uoFe4F89OieWWGL9mK3Q6naD/7VEERHjw/eJDrHx2J2Me6Epo+2ufvsCpQ3tCnntW8+k/+kib92bDBpw6d8bnHptPb7z6zKAK+6ME3kGUVZWxPXt7bSv99JnTAMR4x3B357uJD4mnZ0BPjHr1RbkuLNWQts02knQNFGegDTrqB6P+rom672WWvrsKFdUWPtyRzsJNKeSUVhIX4c0LU7sRH918hf1iInv6MzUojrUL9/PFK3sYNDWabsPaNKj8Bn9//B97DN8HHqD4f19SsGQJWU//xubTz8J7+nT0npfv0FXYHyXwTYSUkmOFx9icqS2AsTdnLzWyBjejG/2D+/NwqOalB7kGObqoLZfK0gsHHVUUg8EJIofBkN9qcepu/ted/NkqCx/sSGPhplTyyirp186HV+7qwYBI3xYj7HXxCXZl6u/i+H7RIbasTCInrYShszpiNDVsyl2dszPed07Ha9pUyjdv1ua9eell8t60+fRz52CKiLBTLRRXQgm8HSmpKmF71vZa6yXnbA4AHbw7MLfLXOJD4+ke0B2jTrXSr5vSU+cHHaUmgKVK6xTtMF5rpUcNB9ONzaFTXlnD8u1pvLs5lbyyKgZG+fL6zJ70j/RtnDo4ELOzgfEPxbJr3Ql2fHmcgqxyxj0Yi4dfw8dKCJ0OtyFDcBsyhIojRyhYspTCcz798OH4zpuLc1xci3wYtlSUwDciVmnlSMGRWkHfm7sXi7TgbnRnQMgA4kPjiQ+Nx9/l+luRCiD32Pn49JM7tW1eEVooY8dbIaz/ZQcdNYSyyhqW/niC9zYfp6C8isExfjw2IoY+bX1uOO3mhNAJ4sa3wy/Mne8XHWLlczsZc19Xwjpffz2dOnYk5Lln8f/VExR+9BFFH60gbf16nLp21eanHzNa+fRNgBL4G6S4spgfs35kc+ZmtmVtI+9sHgCdfDpxb9d7iQ+Np5t/Nww6daqvG6tVE/KjX9N39yeQkKltD+4Bw/6giXpA5ysOOmoIJRXVLNl6gv9uPU7RmWqGdvDn0eEx9I5o3fOot431Y+rv4lj39n6+fG0P/W+Poufo8BtqcRsDAgh4/HH85s+n+Iv/aT79r39NzotB+Myehde0acqntyNKdRqIVVo5nH+41kvfn7cfq7TiafZkYPBA4tvEMzBkIH7Ofo4uasumugKOb7JND7AOynNAZ6DCsysuw36lWTCeVx6s01CKz1azaOtx3t9ynJKKGkZ0DODRETH0CLt5BvR4Bbhwx2/i2LDsMD+uTiEnrZThczpicroxqdA5O+N91514TZ9G+ebN5C9aTM6LL5H75lt4TZmixdOHX30QmaJhKIG/BgorCtmWtY0tmVvYlrWNgooCBIIuvl14IPYB4kPjifWLRa9T60HeEGcKIOlbTdSTN0B1OZjctcFGHW+F6JHs27GHoX2HNmq2RWeqeH/LcRZtPUFpZQ2jOgfy+IgYuobenC1Lo1nP6Pu6EBDuwY+rkyk8pfnyXoE3PtPoJT79osUUfvwxhR98gPvIEfjMm6eFtSoaBbsJvBDCCfgBMNvy+URK+Wd75deYWKwWDuQfqJ0v/UDeASQSb7M3A0MHMihkEANDBuLr3PI72RxOYZpt/vSvtbBGaQH3YOh+lybqbQeDwT5znxSUV/HfLaks2ZZGWWUN47oG8cvh0XQJuTmFvS5CCHqODscv3I1v3z3IqucTGXVPZ9p2a7xfpk4dOxLyr+fxf/JJCj/8kKIVKyj97nt8IiIoLj+jfPpGwJ4t+EpguJSyTAhhBLYIIdZKKbfbMc/rJv9sPtuytrE5czM/Zv1IUWURAkGsfywPd3+Y+NB4Ovt2Vq30G0VKbXWjo2u0gUenbYOO/DtB/BOaqAf3bPCgo4aQX1bJu5uPs/THE5yttjA+NphHh0fTMcjjqsfebIR19GHa77V4+a/f3EffCe2IG9cW0Yhz6hgDAwj41RP4PfQgxZ9/zsmFC8/79HfP1nx6D3VtroerCrwQ4pfAB1LKwoYkLKWUQJntrdH212x+e9VYaziQd6B2oNHB/IMA+Dj5cEubW2pb6V5ON4//ajcs1ZC2VWulH1kDJSdB6LRol9H/0Pz06xx01BBySyt554cUlm9Pp6LGwm3dQnh0eDTtA93tnndLxsPXmTue7k3Ch0f56cvj5KSVMvKezpidG7d9qHN2xnvGDPYGBtJbCAoWLSbn3y+S+8abeN1xh+bTh6mZURuCkFfxu4QQ/wDuAnYD7wPfyKsddP5YPbALiAbekFL+tp595gPzAQIDA3uvWLGiQRU4R1lZGW5uly6yUJcSSwmHzh7i0NlDHK04yhnrGQSCtua2dHbqTGfnzrQxtUEn7Nd6bEyupc6OQl9zBp+Cn/HL245PwS6MNeVYdCYKvXuS59eXfN8+VJsaboVcT52LKqysOV5NQkYN1VboH6JnQqSJELfmf52b0zWWUlKQBKd+lpjcICxe4OTZ+DHtdetsSM/AZf16nHbuBCmp7NGdMyNGUB0V1WhRUy2dYcOG7ZJSxtX32VUFHkBocVKjgXuAOGAl8F8pZcq1FEAI4QWsBh6VUh643H5xcXEyMTHxWpK8hISEBIYOHXrBthprDfty99W20g8XHAbAz9mPQSGDiG8Tz4DgAXiaW6bnWl+dHUrpqfNTAxz/QRt05OJ74UpHphvrqGtInU8VV7BwUwof/ZROjVUyuWcojwyLpp1fy1k8vNldYyArqYh17x6gptLCiLmdiOoV0Kjp11fn6tM5FH74IYUrVmAtLsYpNhafeXPxGDMGYbi5Y0WEEJcV+Gs6M1JKKYQ4BZwCagBv4BMhxHdSyt9cw/FFQogEYCxwWYFvDE6Xn2ZrltY5uj1rO6XVpeiFnh4BPXi81+PEh8bTwbuDGk3XWJRkwcHVcOh/kLEDkODdDvrO10Q9vD80cb9FZtFZFiak8PHODKxSMqWXJuwRvi1H2JszITFeTH+mD+ve2c+6dw7Qa0wE/SZF2nWu+1qf/sH5FH3+OYVLlpL11K/JefElfGbPxmvaVOXT18O1ePCPAXOBPOA94GkpZbUQQgckAfUKvBDCH6i2ibszMBL4V6OV3IbFamF3zm6+KPyC1/73GscKtZV3ApwDGNV2FPGh8fQP7o+7SfmsjYbVCikbYNcibZoAaYHAWBj6DHSaAAGdHPLzOaPgDG8mpPDJrgwApvZuwy+GRhPmoxYSb2zcvM1MfrIXm1ceY/c3aeRmlDL6vi44udo36kXn4oLPzJl433UXZQmbKFi0iJx//5u8N97Ac+od+MyZg6mNWhvhHNfSgvcDpkgp0+pulFJahRC3XeG4YGCJzYfXASullF9df1HrxyItPLL+ESprKukd1Jsnej1BfGg87b3bq1Z6Y1OWC3uWQ+IiKEoDFz8Y9Bj0vLtJOkkvR3r+Gd5MSOaTXSfRCcGdfcJ4eGg0oV5q7Vl7ojfqGDqrIwERHmxacZRVz+1k3EOx+LWxf2NK6HS4Dx+G+/BhnD14kIIlS7QlBpd/gPvIkfjMm4dzzx43vQZcVeCllH+6wmeHr/DZPqDndZbrmjHpTbwz6h2yD2Qzbvg4e2d38yGlFgGT+L5mw1irtdj0EX/SWusNWBijsTmRV84bG5P57OdM9DrBzH7hPDw0imBPJexNSef4EHxCXVn39gE+/dcuht3dkfZ9m25WVOcuXQh94QUCnnqKwuUfULhyJaXffotTt274zpuL++jRN61P3ypq3SOgB0W6IkcXo3VxpgD2rtBsmLxj4OQFfR+A3veAf3uHFi01t4zXNyTz+Z5MjHodcwZE8NCQKAI9rr4ykcI+BLXzZPrvNV/+u/cPkXOilAF3RKHXN12kkjEwkICnnsTv4YcoWr2agqVLyXzyKQwhwfjMvlvz6d1vLqu2VQi8opGQEk4maq31g59BTQWExsHtb0GXyWB0bMs4OaeUhXsr+OmbTZgMOu4d1I75t0QSoIS9WeDiYWLSr3qy7ZNk9m7IIDejlDEPdMXFwz4jkS+HzsUFn1mz8J4xg7KEBC2e/oUXyHv9dbymTcX77rtvGp9eCbxCWyhj30rNWz+9X1t0uscsiLsHgmIdXTqOnS5lwfokvt6fjUkHDwyO5IFbIvFzc5w9pKgfvV7H4DvbE9DWg43Lj7DquZ2MfTCWwLZNH+Gi+fTDcR8+nLMHNJ++4IMPKVi2HPdRo7R1ZHva3UV2KErgb2ay92mt9f2roKpME/PbXoHYqWB2/E/Zw9klvLYhiTX7T+Fq0vPwkCg66bKZMLqTo4umuAod+gXhE+zK2oX7Wf3ibm6Z0Z7Ogxy3prBz1y6E/vsFAp56ksIPPqTw448p/eYbnLt3x+eeebiPHNkqffrWVyPFlak6o8WtJ74PmYnaknZdp2qt9dDezWJ04IHMYl7bkMQ3B0/jbjbw6PBo7h3UDm9XEwkJpxxdPMU14h/uzvTf9+Gb9w6wcdkRck6UMHh6e/RGx40gNgYFaT79Qw9S9Pnnmk//xK8whoTgPeduvKZORd9MRg43BkrgbxZyjmgdpns/0tYq9WsPY5/XZm10bh4LWew7WcSC9Ul8fzgHdycDj42I4b5B7fB0UTMKtlSc3IxMeKwHO75IYfc36eRnljF2fiyuXo6113SurppPf9dd53365/9F3muv4zVtGt6zZ2Nq07jrDTgCJfCtmZpKOPyl1lpP2wo6I3SeBHH3QsTAZtFaB9iTUcSr3x9j49FcPJ2NPDmqPXMHtsXTWQl7a0CnEwyYHI1/uAfrlx5m5bM7GTO/KyHRjp/IT+j1uI8YgfuIEZzdf0Dz6Zcto2DpUtxHj9bWke3Rw9HFvG6UwLdGCo5rrfWfl8OZfPBuCyP/qnWcujWf9WB3pRXy6vokfjiWi7eLkafHdGDOgAjcnZSwt0aiewfgHeTC2oX7+eLln4mfHkPXIaHNZjCSc2xXQl/8NwFPPUnB8uUUrVxF6bp1OPfogc+8ebiPHNHifPqWVVrF5bHUwLG1WiRMynoQeugwTvPWI4fbdX71hvLT8QIWrE9iS3IePq4mfju2I3cPiMDNrG7H1o5vqBvTnonju0WH+GHFMXLSShgyswMGY/NZZ8EYHEzg00/j/4tfULT6cwqWLCHziScwhobifffsFuXTq29US6f4JOxeqv2VZoNHKAz9PfSaAx7Bji5dLVJKfkzNZ8H6JLanFuDnZuIP4zsxq384LiZ1G95MmF2M3PpwN376+jiJX58gP7OccQ/F4u7TvMYz6Fxd8Zk9C+8Zd1G2cSP5iy/06X3uno0xtHn79Oqb1RKxWiBlA133vwCbErUBStEj4NaXIWY06JvPZZVSsi0ln1e/T+KnEwX4u5v5422dmdk3HGdT82m1KZoWoRP0mxBJQIQH379/UPPl7+9Cm44+ji7aJQi9HveRI3EfOZKz+/dTsLiOTz9mNL7z5uHcvbuji1kvzUcJFFenLAd+Xga7FkNROh5GTxj0BPSeq/nszQgpJZuT8nh1fRK70goJ9DDzlwmduatvOE7N6Oe4wrG06+bH1N9pSwL+79U9DJgSjTQ0m4XfLsE5NpbQl14k4NdPnffp167DuWfP8z69vvnc30rgmztSwonNWiTM4S/BWgPtboGRf+XHHHeGDB/l6BJegJSShGO5vPp9EnsyigjxdOLvk7owvU8YZkPzufEVzQfvIFem/i6O9UsOs+3TZDzCoXqgBaO5+d4vF/j0n9nmvXn8cYxt2uAz5248p9yB3s3x6w8ogW+unCnQYtYTF0F+kjbZV7+HoPc88IsBQCYkOLSIdZFSsv5wDgs2JLHvZDGhXs48OzmWqb3bYDI0nw5eRfPE5GRg7Pyu7P4mje2fp/LpC4mMeygWT//mPZe/ztUVn7tn4z1zBqUbNlCweAmnn32O3AWv4TV9Oj6zZ2EMcdwIXiXwzQkp4eROrbV+4DOwVEJYPxi8ELrc7vDJvupDSsm3h06zYH0SB7NKCPNx5l93xDKlVxuMTTiToKLlI4Sg99i2ZBce59TOSlY9l8io+7oQ0cXX0UW7KkKvx2PUKDxGjeLsvn2aT79E+/MYM1qbn75btyYvlxL45kBFCew/N9nXATC5Q8/Z2oCkoK6OLl29WK2Sbw6eYsGGZA5nl9DW14V/T+3G7T1DlbArbgi3YMG0Z/qwduF+vnp9L/0mRtJ7bESziZe/Gs7duhH68ksEZD1FwbLlFK1ajBGE7AAAIABJREFURcmatTj36oXPvLm4j2g6n14JvCPJ2mOb7OsTqC6HoG4w4VVtbhhz84yztVglaw9k89r6ZI6eLiXSz5WXp3dnYvcQDErYFY2Ep78zd/ymNxuXH2HHF6nkppUyYm4nTM4tR7KMISEE/vY3+D3yCMWffUrB0mVkPnbOp5+D55QpdvfpW87Zai1UlWv2S+L7kLUbDM4Qe4fWWg/p1WymD7gYi1Xy1b4sXtuQTHJOGVH+rrx6Vw9u6xaC3o6LLStuXoxmPaPu7UxgWw+2fprMJ//SfHnvIMd3XjYEvZsrPnPm4D1rFqXfr6dg8WJOP/ssua+9htf0afjMno0x2D5jVpTANxU5hzULZu8KqCwG/44w7gXodic4O35OjstRY7Hyv71ZvL4xmdTcctoHuvH6zJ6M6xqshF1hd4QQdB8Rhl8bN7557wCrnk9k5LzORPZoPlNuXCtCr8djzGg8xozm7N69mke/WPvzGDuW4Gf/ic7cuJOwKYG3JzWV2jqmie9D+jbQm7TJvnrf06wm+6qPaouVz3/O5I2NyZzIP0PHIHfenNWLsV2C0ClhVzQxoR28mfZMH9a9vZ+1C/cTN74tfW5r12LvRefu3Ql9+WUCMjMpWP4BVampjS7uoATePuSnaIOR9nxgm+yrnTbZV8/Z4Orn6NJdkaoaK5/tPskbCcn8f3v3Hhdlmf9//HUxICgIggqioKig4hFExdIEJcjUzfDsllrpim5m2bar9e3X5mPb8tBBLa3cVfOYqK2HzFMipGGGB0ZF0VBDBckTQsMZhuv3x+ikKabAMMzM9Xw8fKTD3Pd9XdPMh2uu+7rf98XsQjo0deXzMSFEBnpV+cNUWlpKRkYGRUVFld6Hm5sbqakV3uvd6lS1v05OTvj4+ODgYPkBbvU9nIh+rSvfffkTh7alc+W8jsgX2uPkbLl9c2jWDK/p/0BK01zcpQp8ddGXwunthtH6uXhD2Fe7gYawr5bhtSrs615KysrZcDiDhfFnyMwppLOPG/8c1IGIQM9qW72QkZFB/fr18fPzq/Q+dTod9W3oxslV6a+UkuvXr5ORkUHLli2ruWXmYe+god+Ydni1qM++dWmsn3WIAZM60bBZ7VyU8KBMtUJIFfiqyrkIR5Ybwr7yLoOrD/R90zBar0VhXxUpLtOz7lAGn8af4VJuEUG+DXjn6Y6Et21c7W+6oqKiKhV35eEIIWjYsCFXr141d1OqlRCCjmE+NPSpz47Fx9kw+xD9xgYS0M3L3E2rdVSBr4xyPZzZbThpmrbTcIFSQKRhJUxAFNjV3kusbykq1bM26QKffXeOX34tIqSFO7OGduaxgEYmLcCquNcsa369vVu7MeKN7uz4PIVd/z3BlfM6Hnm6FXZqua6RKvAPQ3cZklfA4RWQewGcPaH3NOg6DtxbmLt1D6SwRM+apAt8/t1ZruiK6eHnwQcjuvBo64ZWXQwU6+Ts5sjTrwbz/fo0tN9e4OoFHU/8pQN1XeqYu2m1gvpV90fKy+FcAqwbCx+1hz3vgIcfDF8O005AxFsWUdwLSsr4z95zPDYnnn9tPUnLRs6s+UsosTE96eVv2lF7bfLCCy/g6elJx453XiGcnZ1NZGQkAQEBREZGcuPGDcAwjz116lT8/f3p3LkzR44cqfE2JyQkMGjQoBo/rqXQ2NsRNrot/cYG8svZXNa9e5CrF3TmblatoAp8RfKvw/6P4ZNusGIw/LzXEPY15TCM+9qQDWNf+0cJ+cVlfPbdWR6bHc+/t6XStokLsRN7EhvzCI+2tp3Cfstzzz3Hjh077np81qxZREREkJaWRkREBLNmzQJg+/btpKWlkZaWxuLFi5k8eXJNN1l5QIGPejPk711BwldzD3PqQJa5m2R2JpuiEUL4AiuAJkA5sFhKOd9Ux6sWUsLFHw0rYU5suhn21RPCphvWrzvUrjvO3E9ecRnL96fz333nuFFQSp82jXk5wp+QFrXjhgozvz7ByUu/PvR2er0eTQU5Hu2buvLPP3W47/Z9+vQhPT39rsc3b95Mws10znHjxhEeHs7s2bPZvHkzY8eORQhBz549ycnJISsrC+/fXXm4atUqFixYQElJCaGhoSxatAiNRoOLiwsxMTHEx8fj7u7O2rVrady4MVqtlkmTJlFQUEDr1q1ZunQp7u7unDlzhkmTJnH16lU0Gg3Lli0DIC8vj2HDhpGSkkJISAirVq1CCMGMGTPYsmUL9vb2REVF8f777z/0a2pNPFu4Mvz17uxakkLcF6lcSdfRa7g/Ghudlzdlr8uAv0kpA4GewItCiPYmPF7lFeVC0n/g00dh6ROG5Y4h42DyDzB+J3QZaTHFPa+4jI/j0ug9ew9zd56mi28D/vfXR1nxQo9aU9xro8uXLxuLtre3N1euXAEgMzMTX19f4/N8fHzIzMy8Y9vU1FRiY2NJTExEq9Wi0WhYvXo1APn5+XTt2pUjR44QFhbGzJkzARg7diyzZ8/m2LFjdOrUyfj4M888w4svvsjRo0fZv38/TZo0ASA5OZl58+Zx8uRJzp07R2JiItnZ2WzcuJETJ05w7Ngx3nzzTdO+SBainmsdnpoaRJfHfTmekMHmj5LJzy02d7PMwmQjeCllFpB18+86IUQq0Aw4aapjPrRLybeFfRWAd1CtD/uqSFGpnlUHzrMo4SzZ+SU8HujJS/0C6OJbO2MQ/mikXZGaXgd/rwtQfj+tFRcXx+HDh+nevTsAhYWFeHp6AmBnZ8fIkSMBePbZZxkyZAi5ubnk5OQQFhYGGL4xDB8+HJ1OR2ZmJtHR0YDhIqXS0lIAevTogY+PDwBBQUGkp6fTs2dPnJycmDBhAgMHDlTz9Lex09jRe1gAns3rE7/yFOvfPUj/mE40aeVm7qbVqBpZRSOE8AOCgR/v8bOJwEQALy8v49fkh5WXl/dA29rpi/C6vBfvrJ246s6gt3PkiudjXGraH51rAOiAHw5Vqg01LS8vj7g98Xx/qYzNZ0rJLpJ0aGjHlM5OtHLL58ZZLQlnzd3K37i5uaHTVe3kl16vr/I+8vLyKC8vv2M/jRs3Ji0tjSZNmvDLL7/QqFEjdDodXl5e/PTTT3S5ec/NCxcu4Orqese2hYWFjB49mrfffvuO49x6jk6nw97enry8PKSU6HQ6439vb8+vv/56x+O3+ltQUIBGozE+rtfrycvLo7CwkLi4OBISEli/fj3z589n69atd/W3qKio0p8rc3jQz/KDatFPcuH7Ev73/mGadBV4+NvOeSeTF3ghhAvwFfCKlPKuSVcp5WJgMUC3bt1keHh4pY6TkJDAfbe9fBIO3wr7+hUaB8Jj76PpPAJvJzdq/yVJd5JS8n5sHNtPC85dLaGLjxsf929HL//aG4WQmppa5dF3dYzgXVxcsLOzu2M/Tz/9NF999RUzZsxg4cKFREdHU79+fYYOHconn3zC888/z48//oi7uzsBAQF37G/gwIEMHjyY6dOn4+npSXZ2NjqdjhYtWlBeXs7OnTsZNWoUW7ZsoU+fPvj4+ODh4YFWq+Wxxx5j48aN9O3bl2bNmuHr60tcXBxPP/00xcXF5OTkUK9ePezt7Y3trVOnDk5OTgghKC8vZ9iwYfTr1w9/f/97vjZOTk4EBwdX6TWrSX/4Wa6EoqhSvl1ygguHsnF3bEKfUW3ROFj/vLxJC7wQwgFDcV8tpfyfKY91T6VFcHKzYRrm4gFD2FeHaMMFSb6htTrsqyK3bmY9d+dpjmcWE+DpwGfPhvBEBy+bWxFTGaNHjyYhIYFr167h4+PDzJkzGT9+PDNmzGDEiBEsWbKE5s2bs379egAGDBjAtm3b8Pf3p169esaTnrdr374977zzDlFRUZSXl+Pg4MDChQtp0aIFzs7OnDhxgpCQENzc3IiNjQVg+fLlxpOsrVq1Mu535cqVxMTE8NZbb+Hg4HDP492i0+kYPHgwRUVFSCn56KOPTPCKWQcnZwcGTulC0pZzHN5xnmuZ+TwZ0xEXd8s4t1ZZwlQhN8JQbZYD2VLKVx5km27duslDhyo3PXLHb/3rZw1FXbsaCm+AR2tDJkyXP4Nz7b/9V0WOXLjBnB2nOHAum2YN6vKkr57XRz9uMbG9qampBAYGVmkflpZF4+LiQl5eXqW3r47+VsfrXpNMMYK/3dnkK8R9kYp9HTv6T+xI0wB3kx2rJgghDkspu93rZ6YcwfcCxgDHhRDam4+9IaXcZoqDifIyw9LGQ0vh5+/Azt4Q9hXyPLQMq/VhX/dz+hcdc3eeZnfqZRq51OHtP7VndGhzfvh+n8UUd0WpLVoHe+Lu5cz2z4+z6SMtvYb507mvj1V+AzblKprvAdO/YiX5sO9Dev64BEpugJsv9HsTgsdA/SYmP7wpXcwu4KNvf2KjNhOXOva8FtWG53u1xNlRJUxYiqqM3hXT8WjqzLAZ3di97CTfr0vjyvlfCX+mHQ51an+O1MOw/Eph7wTH15Pn0hrH/v8A/8ctIuzrfq7oivhkzxm+TLqAnRBM7NOKyWGtaVCv9l85qyiWwrGuPQMmdeLQ9nSStv5M9qV8nozphGujuuZuWrWx/AJvp4EXf+R44o+Etwk3d2uqJLewlM+/O8uyxHRK9OWM7O7L1H4BNHGz7hNBimIuwk7QfWBLGjevz7dLT7LuvYM8Mb4jvu2t46JAyy/wAA6W/Ru3sETPF/vT+ey7s+QWlvJUl6a8GtkGv0aWdXNhRbFUfp0aMXxGN7Z/fpyvP9bS8+nWBEc1t/h5eeso8BaqpKyc2EMX+TgujSu6Yvq18+S1qLa0b+pq7qYpis1p4FWPof8IIX7lKX7YeJYr53X0G9uOOk6WWyYtd2mJBSsvl2xKzuTxD7/j/21KoUXDeqyf9AhLn+uuirsJXbx4kb59+xIYGEiHDh2YP/+37DsVF6wA1HGyJ2pCBx4Z0ppzyVf4as5hci4XmLtZlaYKfA2SUhKXepkBC/bxSqwWZ0d7lj3XnXUxj9Ddzzrm/Goze3t7PvjgA1JTUzlw4AALFy7k5ElDNJKKC1ZuEULQNaoFf5oaRH5uMetnHSL9+DVzN6tSLPe7h4X58dx15uw8zeHzN2jRsB7zRwXxp85NsbPVdezbZ8Avxx96s7r6MtBU8LZt0gmenFXhtt7e3sbEyPr16xMYGEhmZibt27dXccHKXXwDPRjxene2f36cbxYdo8eglnR70g9hQZ9ZNYI3sZTMXMYtTWLk4gNk3Cjg39Ed2f1qGIODmtluca8F0tPTSU5OJjQ0FFBxwcq9uTaqy9C/h9C2RxOSvv6ZbZ8dp7iwzNzNemBqBG8iP1/L54Ndp9l6LAu3ug68/mQ7xj3qh5ODZa/Rrzb3GWnfT2E1XLqfl5fH0KFDmTdvHq6u9z/noeKCFfs6GiKeC8TTrz6J68+wYdYhnozphEfT2r/KTRX4apaVW8iCuDTWHcrA0d6Ol/r585c+rXB1cjB30xSgtLSUoUOH8swzzzBkyBDj415eXsapl6ysLGOB9vHx4eLFi8bnZWRk0LRp0zv2KaVk3LhxvPfee394/Pstu7tfLpSjo6Px7xqNhrKyMuzt7UlKSiIuLo61a9fyySefsGfPnj9sg/LwhBB07utLIx8XdixOYcPsQ0SMC6R1V09zN+2+1BRNNbmRX8K/vzlJ2NwENhzOYEzPFnz39778LaqtKu61hJSS8ePHExgYyKuvvnrHz5566imWL18OGJIeBw8ebHx8xYoVSCk5cOAAbm5ud82/R0REsGHDBuO0TnZ2NufPnwegvLycDRs2ALBmzRp69+6Nm5sb7u7u7Nu3DzAkSIaFheHq6oqPjw+bNm0CoLi4mIKCildw5OXlkZuby4ABA5g3bx5arbbC5yrVo2mAOyPe6IFHU2d2LE7hh01nKS83TWBjdVAj+CrKKy5j6fc/85+958grKSM6uBnTHm+Dr0c9czdN+Z3ExERWrlxJp06dCAoKAuDdd99lwIABKi5YeWAu7o5Ev9qVvet+4siO81y7oCNyfAecnGvfQM5kccGVUW1xwTWguEzP6gMXWBh/huv5JUS19+K1J9rSxqvmomxrus9VpeKCH56KC67dTn5/ie/WnsalgSNPTupEI5+af2+aKy7YKunLJf87ksG83Wlk5hTySKuG/KN/W4KbW3amtKIoD69976Z4NHNmx+cpfDX7MH3HtKNNj9qTYqsK/AOSUrIj5Rc++PYnzlzJo4uPG7OHdqZ3QO29RZ5ifiou2Po1aenGiDe6s2Pxcb5depIr53U8OqQ1dhrzn+JUBf4BfJ92jbk7T3E0Ixd/Txc+e7YrT3RoYvFBRIqiVI96rnUYPC2Y/RvOcDTuItcu6oia0JF6ruaN+FYF/j60F3OYs+MU+89ep1mDuswZ1pkhwc2wrwW/mRVFqV00GjseG9kGzxb1iV99mvXvHaR/TCe8/MyXL6UK/D2kXdbx/q7T7DxxmYbOdXhrUHue6dkcR3t1kZKiKPfXtqc3Hk1d2P7ZcTa+f4Q+o9vQvlfTP97QBFSBv83F7ALm7U5jY3IGznXseTWyDS/0bomLukWeoigPoXHz+ox4ozs7/5tC/MpTXD2vo/eIADT2NfvtX801AFd1xby95QT9Pkjg62OXmPBYK/b+oy9TIwJUcbciRUVF9OjRgy5dutChQwf++c9/Gn/2888/ExoaSkBAACNHjqSkpAQwXGw0cuRI/P39CQ0NJT09vcbb/cUXXzBlypQaP65SNU4uDvzppS4ERzUnZW8mmz5MJj+nuEbbYNMF/teiUj7YdZqwufGsPHCeYSE+fPf3cN4YEIi7s7r/qbVxdHRkz549HD16FK1Wy44dOzhw4AAA06dPZ9q0aaSlpeHu7s6SJUsAWLJkiTHlcdq0aUyfPt2cXVAsjJ3GjkeH+BM1oQPXMvNY9+5Bss7k1NjxbXJ4WlSqZ/n+dBYlGG6RN6izN69GtqFVYxdzN81mzE6azansUw+9nV6vR6O597mQdh7tmN6j4gIshMDFxfD/uLS0lNLSUoQQSCnZs2cPa9asAQzhX2+//TaTJ09m8+bNvP322wAMGzaMKVOmIKW8awXV3LlzWbduHcXFxURHRzNz5kzS09Pp378/oaGhJCcn06ZNG1asWEG9evWIi4vjtddeo6ysjO7du/Ppp5/i6OjIwYMHefnll8nPz8fR0dEYW3Dp0iX69+/P2bNniY6OZs6cOej1esaPH8+hQ4cQQvDCCy8wbdq0h35NFdML6OaFh7cz2z47zqaPkuk9PICOYc1MvhLPpkbwpfpyVv94nrC58by3/RTBzRuw9aXefPLnrqq42wi9Xk9QUBCenp5ERkYSGhrK9evXadCgAfb2hvHO7ZHAt8cF29vb4+bmxvXr1+/Y565du0hLSyMpKQmtVsvhw4fZu3cvAKdPn2bixIkcO3YMV1dXFi1aRFFREc899xyxsbEcP36csrIyPv30U0pKShg5ciTz58/n6NGj7N69m7p1Dfcb1mq1xufHxsZy8eJFtFotmZmZpKSkcPz4cZ5//vmaehmVSmjYzIURr3fDN9CDvWt/Ys/KU5SV6k16TJsYwZeXS7Yez+LDXadJv15ASAt3FowKJrRVQ3M3zWbdb6R9P1W9dF+j0aDVasnJySE6OpqUlBS8vLzuet6tkdWDxAXv2rWLXbt2ERwcDBgubkpLS6N58+b4+vrSq1cvwBAXvGDBAiIjI2nZsiVt2rQBDN8YFi5cSEREBN7e3sbYYVdXV3Q6HWAINHNzcwMM2Tfnz5+nQ4cOnDt3jpdeeomBAwcSFRVV6ddFqRmO9RwY+NfOJG39mUPb0snOzKN/TCfqeziZ5HhWPYKXUrLn1GUGfvw9U79MxslBw5Jx3dgw6RFV3G1cgwYNCA8PZ8eOHTRq1IicnBzKygw3crg9Evj2uOCysjJyc3Px8Ljz9opSSl5//XW0Wi1arZYzZ84wfvx44O5fBremhO7lXlM/t9wrLtjd3Z2jR48SHh7OwoULmTBhQiVeCaWmCTtB6FOteHJSJ25cLmDduwfJOH3DJMey2gKf9HM2Iz7/gRe+OER+cRnzRwWxbepjRAR6qStQbdTVq1fJyTGc4CosLGT37t20a9cOIQR9+/Y1xvr+Pi74Vozwhg0b6Nev313vnyeeeIKlS5caYwkyMzON0cEXLlzghx9+AODLL7+kd+/etGvXjvT0dM6cOQP8Fhfcrl07Ll26xMGDBwHDt5Vbv3Tu5dq1a5SXlzN06FD+9a9/meWG4ErltQpqzPAZ3ajr4sDO/6RQUlT9d4qyuimak5d+Ze7OU8Sfvkrj+o786+mOjOzmS50aXn+q1D5ZWVmMGzcOvV5PeXk5I0aMMN4Fafbs2YwaNYo333yT4OBg4wh8/PjxjBkzBn9/fzw8PFi7du1d+42KiiI1NZVHHnkEMCRIrlq1Co1GQ2BgIMuXLycmJoaAgAAmT56Mk5MTy5YtY/jw4caTrJMmTaJOnTrExsby0ksvUVhYSN26ddm4cWOF/cnMzOT555+nvLwc4IFuOKLULu5NnBk2oxs3sgqo41T95dhkccFCiKXAIOCKlLLjg2xTlbjg2G/28P2v7nx99BKuTvZMDvfnuUf9qFvHeq8+taRYVbC9uOD09HQGDRpESkpKpfeh4oKVP2KuuOAvgE+AFSY8BnnFZby7LZXYpELq2Jfw1/DWxPRpjVu92he+ryiKUpNMVuCllHuFEH6m2v8tdR00JF/IIdzXnvfGhOFZ3zRnoxXlYfn5+VVp9K4oVWXxE9MaO8GWKb0Y095RFXdFUZTbmP0kqxBiIjARDHe2T0hIqNR+8vLyKr2tpbK0Pru5uRnXdVeWXq+v8j4sSXX0t6ioyKLeJ5b2vq7NzF7gpZSLgcVgOMla2ZMrtnhixtL6nJqaWuUThpZ0krU6VEd/nZycjBdhWQJLe1/XZhY/RaMoiqLcm8kKvBDiS+AHoK0QIkMIMd5Ux1KUh6HX6wkODjaugQcVF6xYJ5MVeCnlaCmlt5TSQUrpI6VcYqpjKcrDmD9//l3rwlVcsGKNzD4Hr9imX959l+LUh48LLtPrya4gLtgxsB1N3njjvttnZGTwzTff8H//9398+OGHACouWLFaag5esSmvvPIKc+bMwc7ut7e+igtWrJUawStm8Ucj7YpUZVXJ1q1b8fT0JCQk5I5lePeLBFZxwYolUyN4xWYkJiayZcsW/Pz8GDVqFHv27OHZZ59VccGK1VIFXrEZ7733HhkZGaSnp7N27Vr69evHqlWrVFywYrXUFI2ioOKCFetksrjgyqhKXLAtXv1maX1WccEPT8UFK3/kfnHBaopGURTFSqkCrygmouKCFXNTBV5RFMVKqQKvKIpipVSBVxRFsVKqwCuKolgpVeAVm+Ln50enTp0ICgqiW7ffVpZlZ2cTGRlJQEAAkZGR3LhxAzBcXTp16lT8/f3p3LmzWS4mSkhIuCPaWFEelCrwis2Jj49Hq9Vy+zUXs2bNIiIigrS0NCIiIpg1axYA27dvJy0tjbS0NBYvXszkyZPN1WxFeWjqSlbFLPat+4lrF/Meeju9Xo+mgrjgRr4uPDaiTaXas3nzZmMA2bhx4wgPD2f27Nls3ryZsWPHIoSgZ8+e5OTkkJWVhbe39x3br1q1igULFlBSUkJoaCiLFi1Co9Hg4uJCTEwM8fHxuLu7s3btWho3boxWq2XSpEkUFBTQunVrli5dasydnzRpElevXkWj0bBs2TLAEGA2bNgwUlJSCAkJMUYszJgxgy1btmBvb09UVBTvv/9+pfqvWCc1gldsihCCqKgoQkJCWLx4sfHxy5cvG4u2t7e3MUvm9rhguDNK+JbU1FRiY2NJTExEq9Wi0WhYvXo1APn5+XTt2pUjR44QFhbGzJkzARg7diyzZ8/m2LFjdOrUyfj4M888w4svvsjRo0fZv38/TZo0ASA5OZl58+Zx8uRJzp07R2JiItnZ2WzcuJETJ05w7Ngx3nzzTRO9aoqlUiN4xSwqO9Ku6qX7iYmJNG3alCtXrhAZGUm7du3o06dPhc9/kLjguLg4Dh8+bIz5LSwsxNPTEwA7OztGjhwJGOKChwwZQm5uLjk5OYSFhQGGbwzDhw9Hp9ORmZlJdHQ0YLhZdmlpKQA9evTAx8cHgKCgINLT0+nZsydOTk5MmDCBgQMHqnl65S5qBK/YlFsxwJ6enkRHR5OUlASAl5cXWVlZAGRlZRkL9O1xwXBnlPAtUkrGjRtnjAs+ffq08S5Qv1dRHPCt/VTkXnHB9vb2JCUlMXToUDZt2kT//v3v03PFFqkCr9iM/Px84w008vPz2bVrFx07dgTujAX+fVzwihUrkFJy4MAB3Nzc7pp/j4iIYMOGDcZpnezsbM6fPw9AeXm5MYZ4zZo19O7dGzc3N9zd3dm3bx/wW1ywq6srPj4+xtv0FRcXU1BQUGF/8vLyyM3NZcCAAcybNw+tVlstr5NiPdQUjWIzLl++bJz+KCsr489//rNx1DtjxgxGjBjBkiVLaN68OevXrwdgwIABbNu2DX9/f+rVq2c86Xm79u3b88477xAVFUV5eTkODg4sXLiQFi1a4OzszIkTJwgJCcHNzY3Y2FjA8Evk1knWVq1aGfe7cuVKYmJieOutt3BwcLjn8W7R6XQMHjyYoqIipJR89NFH1fp6KZZPxQVbMEvrs63FBYMhG/7WjUAqQ8UFK39ExQUriqLYIFXgFcWEqjJ6V5SqUgVeqVG1aUrQFqjX27apAq/UGCcnJ65fv66KTg2RUnL9+nWcnJzM3RTFTNQqGqXG+Pj4kJGRwdWrVyu9j6KiIpsqWFXtr5OTk/ECKcX2qAKv1BgHBwdatmxZpX0kJCQQHBxcTS2q/Wytv0r1MukUjRCivxDitBDijBBihimPpSiKotzJZAVeCKEBFgJPAu2B0UKI9qY6nqIoinInU47gewBnpJTnpJQlwFpgsAlsdwwOAAADjElEQVSPpyiKotzGlHPwzYCLt/07Awj9/ZOEEBOBiTf/mSeEOF3J4zUCrlVyW0ul+mz9bK2/YJt9rooWFf3AlAX+XrF5d62Pk1IuBhbf47kPdzAhDlV0ua61Un22frbWX7DNPpuKKadoMgDf2/7tA1wy4fEURVGU25iywB8EAoQQLYUQdYBRwBYTHk9RFEW5jcmmaKSUZUKIKcBOQAMslVKeMNXxqIZpHguk+mz9bK2/YJt9NolaFResKIqiVB+VRaMoimKlVIFXFEWxUhZf4G0xDkEIsVQIcUUIkWLuttQEIYSvECJeCJEqhDghhHjZ3G0yNSGEkxAiSQhx9GafZ5q7TTVFCKERQiQLIbaauy2WzqILvA3HIXwB9Dd3I2pQGfA3KWUg0BN40Qb+PxcD/aSUXYAgoL8QoqeZ21RTXgZSzd0Ia2DRBR4bjUOQUu4Fss3djpoipcySUh65+Xcdhg9/M/O2yrSkwa3bQTnc/GP1KyKEED7AQOC/5m6LNbD0An+vOASr/uDbOiGEHxAM/GjelpjezakKLXAF+FZKafV9BuYB/wDKzd0Qa2DpBf6B4hAU6yCEcAG+Al6RUv5q7vaYmpRSL6UMwnAVeA8hREdzt8mUhBCDgCtSysPmbou1sPQCr+IQbIQQwgFDcV8tpfyfudtTk6SUOUAC1n/epRfwlBAiHcN0az8hxCrzNsmyWXqBV3EINkAIIYAlQKqU8kNzt6cmCCEaCyEa3Px7XeBx4JR5W2VaUsrXpZQ+Uko/DJ/lPVLKZ83cLItm0QVeSlkG3IpDSAXWmTgOoVYQQnwJ/AC0FUJkCCHGm7tNJtYLGINhRKe9+WeAuRtlYt5AvBDiGIaBzLdSSrVsUHkoKqpAURTFSln0CF5RFEWpmCrwiqIoVkoVeEVRFCulCryiKIqVUgVeURTFSqkCryiKYqVUgVcURbFSqsArSgWEEN2FEMduZrM738xlt+o8GMW6qAudFOU+hBDvAE5AXSBDSvmemZukKA9MFXhFuY+bGUcHgSLgUSml3sxNUpQHpqZoFOX+PAAXoD6GkbyiWAw1gleU+xBCbMEQXdsS8JZSTjFzkxTlgdmbuwGKUlsJIcYCZVLKNTfv/7tfCNFPSrnH3G1TlAehRvCKoihWSs3BK4qiWClV4BVFUayUKvCKoihWShV4RVEUK6UKvKIoipVSBV5RFMVKqQKvKIpipf4/5OERIB10GGEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "model = Sequential([ \n",
    "    Dense(10,                         # number of units in the layer\n",
    "          input_shape=(1,),          # shape of the inputs\n",
    "          activation='relu'),      # the unit computes a simple linear (identity function) on its input\n",
    "    Dense(5),\n",
    "    Dense(1)    \n",
    "])\n",
    "\n",
    "model.compile(optimizer='adam', loss='mse')\n",
    "\n",
    "\n",
    "x = np.linspace(0,5,num=100)\n",
    "# let's train the network for 5 iterations of 25 epochs each ...\n",
    "iterations = 5\n",
    "epochs_per_iter = 100\n",
    "number_of_ephochs_trained=0\n",
    "for i in range(iterations):\n",
    "    model.fit(df.SepalWidth.values, df.PetalLength.values, epochs=epochs_per_iter, verbose=0)\n",
    "    number_of_ephochs_trained = number_of_ephochs_trained + epochs_per_iter\n",
    "    y = model.predict(x)\n",
    "    l = model.evaluate(df.SepalWidth.values, df.PetalLength.values, verbose=0)\n",
    "    print('Loss after %i epochs: %f' % (number_of_ephochs_trained, l))\n",
    "    plt.plot(x,y, label=\"%i epochs\" % (number_of_ephochs_trained))\n",
    "\n",
    "plt.legend()\n",
    "plt.xticks(np.arange(0, 5, 1))\n",
    "plt.yticks(np.arange(0, 7, 1))\n",
    "\n",
    "def show(plt, title=None, xlabel=None, ylabel=None, legend=True, grid=True):\n",
    "    if title: plt.title(title)\n",
    "    if xlabel: plt.xlabel(xlabel)\n",
    "    if ylabel: plt.ylabel(ylabel)\n",
    "    if grid: plt.grid()\n",
    "    if legend: plt.legend()\n",
    "    plt.show()\n",
    "    \n",
    "\n",
    "#print(model.get_weights())\n",
    "    \n",
    "show(plt, 'Network function for different epochs', 'x', 'y')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## MNIST\n",
    "Als nächstes schauen wir uns einen deutlich komplizierteren Datensatz an: MNIST enthält Bilder der Größe 28*28 von handgeschriebenen Ziffern. Die Aufgabe ist es, einen Klassifikator für diese Ziffern zu entwickeln. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Zunächst laden wir den Datensatz und konvertieren die Klassen-Labels zu Vektoren mit One-Hot-Encoding."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.utils import to_categorical\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import keras\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Input, Flatten, Softmax\n",
    "import matplotlib.pyplot as plt\n",
    "import tensorflow as tf\n",
    "\n",
    "dataset = keras.datasets.mnist\n",
    "(train_images, train_labels), (test_images, test_labels) = dataset.load_data()\n",
    "\n",
    "# reshape each image into a vector of length 28*28=784\n",
    "train_images = train_images.reshape((60000,28*28))\n",
    "test_images = test_images.reshape((10000,28*28))\n",
    "\n",
    "train_images, test_images = train_images / 255.0, test_images / 255.0\n",
    "# turn labels into categories\n",
    "train_labels_c = to_categorical(train_labels)\n",
    "test_labels_c  = to_categorical(test_labels)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dann erstellen wir ein sehr einfaches Netzwerk mit 10 Neuronen und wählen wieder mean squared error als Loss-Funktion (ist das in diesem Fall sinnvoll?)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = tf.keras.models.Sequential([\n",
    "  tf.keras.layers.Dense(10),\n",
    "])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(optimizer='adam',\n",
    "               loss=\"mse\"\n",
    "              ,metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Schließlich trainieren wir das Modell und erreichen eine Validation Accuracy von ca. 84%. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples, validate on 10000 samples\n",
      "60000/60000 [==============================] - 3s 57us/sample - loss: 0.0489 - accuracy: 0.8089 - val_loss: 0.0413 - val_accuracy: 0.8457\n"
     ]
    }
   ],
   "source": [
    "h = model.fit(train_images, train_labels_c, validation_data=(test_images, test_labels_c), verbose=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Aufgabe\n",
    "Verbessern Sie das Modell, indem Sie geeignetere Parameter (z.B. Netzwerkarchitektur, Loss-Funktion) wählen. Welche Validation Accuracy erreicht Ihr Modell?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples, validate on 10000 samples\n",
      "60000/60000 [==============================] - 14s 239us/sample - loss: 0.2614 - accuracy: 0.9253 - val_loss: 0.1373 - val_accuracy: 0.9587\n"
     ]
    }
   ],
   "source": [
    "model = tf.keras.models.Sequential([\n",
    "  #tf.keras.layers.Flatten(input_shape=(28, 28)),\n",
    "  tf.keras.layers.Dense(128, activation='relu'),\n",
    "  tf.keras.layers.Dense(10),\n",
    "  #tf.keras.layers.Softmax()\n",
    "])\n",
    "\n",
    "model.compile(optimizer='adam',\n",
    "              loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True)\n",
    "              ,metrics=['accuracy'])\n",
    "h = model.fit(train_images, train_labels_c, validation_data=(test_images, test_labels_c), verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples, validate on 10000 samples\n",
      "60000/60000 [==============================] - 21s 356us/sample - loss: 0.2252 - accuracy: 0.9330 - val_loss: 0.1267 - val_accuracy: 0.9606\n"
     ]
    }
   ],
   "source": [
    "#\"deep learning...\"\n",
    "model = tf.keras.models.Sequential([\n",
    "  tf.keras.layers.Dense(256, activation='relu'),\n",
    "    tf.keras.layers.Dense(256, activation='relu'),\n",
    "    tf.keras.layers.Dense(256, activation='relu'),\n",
    "  tf.keras.layers.Dense(128, activation='relu'),\n",
    "  tf.keras.layers.Dense(64, activation='relu'),\n",
    "  tf.keras.layers.Dense(10),\n",
    "  #tf.keras.layers.Softmax()\n",
    "])\n",
    "\n",
    "model.compile(optimizer='adam',\n",
    "              loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True)\n",
    "              ,metrics=['accuracy'])\n",
    "h = model.fit(train_images, train_labels_c, validation_data=(test_images, test_labels_c), verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples, validate on 10000 samples\n",
      "Epoch 1/5\n",
      "60000/60000 [==============================] - 6s 100us/sample - loss: 0.4706 - accuracy: 0.8777 - val_loss: 0.3103 - val_accuracy: 0.9143\n",
      "Epoch 2/5\n",
      "60000/60000 [==============================] - 5s 77us/sample - loss: 0.3034 - accuracy: 0.9159 - val_loss: 0.2825 - val_accuracy: 0.9219\n",
      "Epoch 3/5\n",
      "60000/60000 [==============================] - 7s 111us/sample - loss: 0.2833 - accuracy: 0.9208 - val_loss: 0.2739 - val_accuracy: 0.9231\n",
      "Epoch 4/5\n",
      "60000/60000 [==============================] - 7s 111us/sample - loss: 0.2729 - accuracy: 0.9237 - val_loss: 0.2682 - val_accuracy: 0.9257\n",
      "Epoch 5/5\n",
      "60000/60000 [==============================] - 7s 117us/sample - loss: 0.2667 - accuracy: 0.9259 - val_loss: 0.2676 - val_accuracy: 0.9272\n"
     ]
    }
   ],
   "source": [
    "#mehr epochen\n",
    "model = tf.keras.models.Sequential([\n",
    "  #tf.keras.layers.Flatten(input_shape=(28, 28)),\n",
    "  #tf.keras.layers.Dense(128, activation='relu'),\n",
    "  tf.keras.layers.Dense(10),\n",
    "  #tf.keras.layers.Softmax()\n",
    "])\n",
    "\n",
    "model.compile(optimizer='adam',\n",
    "              loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True)\n",
    "              ,metrics=['accuracy'])\n",
    "h = model.fit(train_images, train_labels_c, validation_data=(test_images, test_labels_c), verbose=1,epochs=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples, validate on 10000 samples\n",
      "Epoch 1/5\n",
      "60000/60000 [==============================] - 13s 214us/sample - loss: 0.2477 - accuracy: 0.9287 - val_loss: 0.1315 - val_accuracy: 0.9616\n",
      "Epoch 2/5\n",
      "60000/60000 [==============================] - 14s 229us/sample - loss: 0.1098 - accuracy: 0.9673 - val_loss: 0.0956 - val_accuracy: 0.9706\n",
      "Epoch 3/5\n",
      "60000/60000 [==============================] - 17s 284us/sample - loss: 0.0752 - accuracy: 0.9774 - val_loss: 0.0935 - val_accuracy: 0.9716\n",
      "Epoch 4/5\n",
      "60000/60000 [==============================] - 17s 287us/sample - loss: 0.0569 - accuracy: 0.9823 - val_loss: 0.0817 - val_accuracy: 0.9753\n",
      "Epoch 5/5\n",
      "60000/60000 [==============================] - 18s 296us/sample - loss: 0.0438 - accuracy: 0.9866 - val_loss: 0.0733 - val_accuracy: 0.9788\n"
     ]
    }
   ],
   "source": [
    "#kombination: mehr epochen, mehr hidden units\n",
    "model = tf.keras.models.Sequential([\n",
    "  #tf.keras.layers.Flatten(input_shape=(28, 28)),\n",
    "  tf.keras.layers.Dense(128, activation='relu'),\n",
    "  tf.keras.layers.Dense(10),\n",
    "  #tf.keras.layers.Softmax()\n",
    "])\n",
    "\n",
    "model.compile(optimizer='adam',\n",
    "              loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True)\n",
    "              ,metrics=['accuracy'])\n",
    "h = model.fit(train_images, train_labels_c, validation_data=(test_images, test_labels_c), verbose=1,epochs=5)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "C:/Users/Timon Felske/Desktop/test/20200317-PlayGround-SB-master-20_Wayfinding_HMM/20_Wayfinding_HMM (Python 3.6)",
   "language": "python",
   "name": "c_users_timon_felske_desktop_test_20200317-playground-sb-master-20_wayfinding_hmm_20_wayfinding_hmm"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}