Neural Network-手寫辨識.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*標準 NN(Fully Connected Neural Networks)\n",
    "\n",
    "輸入層 -> 隱藏層 ->  輸出層\n",
    "由神經元組成\n",
    "\n",
    "*個別神經元接收到總刺激進而產生輸出的方式：\n",
    "\n",
    "1.輸入做加權和，\n",
    "\n",
    "2.加上偏值(bias)，\n",
    "\n",
    "3.最後加上激活函數作用上去，\n",
    "即為輸出。\n",
    "\n",
    "*激活函數　activation function\n",
    "1. ReLU (冠軍)：小於0的時候輸出0，大於0的時候直接輸出\n",
    "2. Sigmoid (常用)： 模擬人類神經元，輸出 0/1\n",
    "3. Gaussian"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**神經網路學習法backpropagation**\n",
    "\n",
    "NN: Feedforward Fully-Connected Neural Network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**手寫辨識技巧**\n",
    "\n",
    "One-Hot encoding 將輸出值分類\n",
    "\n",
    "soft max 總輸入值機率加起來為1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Stochastic Gradient Descent (SGD )  \n",
    "打亂問題的順序，再次訓練的方法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**<font size = 5>主題：標準神經網路-手寫辨識</font>**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**1. 初始準備**\n",
    "\n",
    "Keras可以用各種不同的深度學習套件當底層，我們在此指定用Tensorflow以確保執行的一致性"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "env: KERAS_BACKEND=tensorflow\n"
     ]
    }
   ],
   "source": [
    "%env KERAS_BACKEND=tensorflow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**2.讀入MNIST數據庫**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n",
      "C:\\Users\\USER\\anaconda3\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n",
      "C:\\Users\\USER\\anaconda3\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n",
      "C:\\Users\\USER\\anaconda3\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n",
      "C:\\Users\\USER\\anaconda3\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n",
      "C:\\Users\\USER\\anaconda3\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n",
      "C:\\Users\\USER\\anaconda3\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n",
      "C:\\Users\\USER\\anaconda3\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n",
      "C:\\Users\\USER\\anaconda3\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n",
      "C:\\Users\\USER\\anaconda3\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n",
      "C:\\Users\\USER\\anaconda3\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n",
      "C:\\Users\\USER\\anaconda3\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n",
      "C:\\Users\\USER\\anaconda3\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import mnist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "60000"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(x_train) #共60000筆訓練資料"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(28, 28)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train[9487].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 240,\n",
       "        253, 165,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  32, 241,\n",
       "        252, 252,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   2, 141, 252,\n",
       "        252, 252,  98,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  30, 252, 252,\n",
       "        252, 252, 149,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  21, 217, 252,\n",
       "        252, 252, 149,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 134, 252,\n",
       "        252, 252, 223,  31,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 134, 252,\n",
       "        252, 252, 253,  44,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   8, 239,\n",
       "        252, 252, 253,  44,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 238,\n",
       "        252, 252, 253,  44,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 238,\n",
       "        252, 252, 253,  44,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0, 178,\n",
       "        253, 253, 255, 106,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  90,\n",
       "        252, 252, 253, 206,  21,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  90,\n",
       "        252, 252, 253, 252,  88,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  90,\n",
       "        252, 252, 253, 252,  88,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,  63,\n",
       "        234, 252, 253, 252, 220,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "        176, 252, 253, 252, 237,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "         45, 252, 253, 252, 245,  71,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "         29, 215, 253, 252, 237,   8,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0, 149, 253, 252, 237,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,  61, 165, 252, 237,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0],\n",
       "       [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,\n",
       "          0,   0]], dtype=uint8)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train[9487] #0-255 0-白色 255-黑色"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1837946cbc8>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAMVUlEQVR4nO3df+hddR3H8ddLt8CtlK191WHSKhQcSSsuI1DCiIUTcfZH0cBYKnz7Y4piSFJC/qEgkUVCS1aOptYiKHF/SDVGIEMYXsfa5kb5g5XTsX2Hui0YlO7dH99jfLd977l395xzz93ezwdczr3nfe89b+6+r517z+fc+3FECMD574K2GwAwGoQdSIKwA0kQdiAJwg4kMWeUG1u0aFEsWbJklJsEUtm/f7+OHDni2WqVwm77Rkk/k3ShpF9FxKNl91+yZIm63W6VTQIo0el0etaGfhtv+0JJP5e0UtJSSattLx32+QA0q8pn9uWSXouINyLiP5J+J2lVPW0BqFuVsF8h6c0Ztw8U605he9J213Z3amqqwuYAVFEl7LMdBDjj3NuIWB8RnYjoTExMVNgcgCqqhP2ApCtn3P6EpLertQOgKVXC/pKkq2x/yvZHJH1T0uZ62gJQt6GH3iLifdt3SfqzpofeNkTEK7V1BqBWlcbZI+J5Sc/X1AuABnG6LJAEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJjHTKZozewoULS+tHjx4tra9du7a0/vjjj591T2gHe3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIJx9vPAe++917MWEaWPtV1aX7duXWmdcfZzR6Ww294v6bikDyS9HxGdOpoCUL869uxfjogjNTwPgAbxmR1IomrYQ9JfbL9se3K2O9ietN213Z2amqq4OQDDqhr26yLiC5JWSlpr+0un3yEi1kdEJyI6ExMTFTcHYFiVwh4RbxfLw5KelbS8jqYA1G/osNueb/tjH16X9FVJe+pqDEC9qhyNv0zSs8U47RxJv42IP9XSFU7Rb6z8iSee6Fk7fvx43e2cYsuWLaX1FStWNLp9DG7osEfEG5I+V2MvABrE0BuQBGEHkiDsQBKEHUiCsANJ8BXXc8CxY8dK6w8++OCIOjnTtm3bSusMvY0P9uxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kATj7OeAuXPnltaXLl3as7Z3796628E5ij07kARhB5Ig7EAShB1IgrADSRB2IAnCDiTBOPs5YN68eaX122+/vWft/vvvr7udU2zatKm0ft999/WsXXLJJXW3gxLs2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcbZUcnrr79eWj9x4kTPGuPso9V3z257g+3DtvfMWLfQ9hbbrxbLBc22CaCqQd7G/1rSjaete0DS1oi4StLW4jaAMdY37BHxgqR3Tlu9StLG4vpGSbfW3BeAmg17gO6yiDgoScXy0l53tD1pu2u7OzU1NeTmAFTV+NH4iFgfEZ2I6ExMTDS9OQA9DBv2Q7YXS1KxPFxfSwCaMGzYN0taU1xfI+m5etoB0JS+4+y2N0m6QdIi2wck/VDSo5J+b/tOSf+S9PUmm0S5u+++u2ft4YcfLn3s0aNH624HY6pv2CNidY/SV2ruBUCDOF0WSIKwA0kQdiAJwg4kQdiBJPiK63lgzpze/4y2R9gJxhl7diAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiC77Of526++ebS+tNPP13p+U+ePFla3759e8/aqlWrKm0bZ4c9O5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kwTj7ee6OO+4orT/zzDOVnv+CC8r3F4899ljP2ooVK0ofO2/evKF6wuz67tltb7B92PaeGesesv2W7Z3F5aZm2wRQ1SBv438t6cZZ1v80IpYVl+frbQtA3fqGPSJekPTOCHoB0KAqB+jusr2reJu/oNedbE/a7truTk1NVdgcgCqGDfsvJH1G0jJJByX1PAoTEesjohMRnYmJiSE3B6CqocIeEYci4oOIOCnpl5KW19sWgLoNFXbbi2fc/JqkPb3uC2A89B1nt71J0g2SFtk+IOmHkm6wvUxSSNov6TsN9ohz2IsvvtiztmPHjtLHXn/99XW3k1rfsEfE6llWP9lALwAaxOmyQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kwU9Jn+cuv/zy0nq/Xw9q8qfEJicnS+t79+5tbNsZsWcHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQYZz/PXX311aX1lStXltafeuqpOts5BdOBjRZ7diAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgnH25B555JHSepPj7CdOnCit7969u7R+7bXX1tnOea/vnt32lbb/anuf7Vds31OsX2h7i+1Xi+WC5tsFMKxB3sa/L+m7EXGNpC9KWmt7qaQHJG2NiKskbS1uAxhTfcMeEQcjYkdx/bikfZKukLRK0sbibhsl3dpUkwCqO6sDdLaXSPq8pO2SLouIg9L0fwiSLu3xmEnbXdtdzoUG2jNw2G1/VNIfJN0bEccGfVxErI+ITkR0+v24IYDmDBR223M1HfTfRMQfi9WHbC8u6oslHW6mRQB16Dv0ZtuSnpS0LyJ+MqO0WdIaSY8Wy+ca6RCNmj9/fmm93/DWrl27ht72RRddVFq/5pprhn5unGmQcfbrJH1L0m7bO4t139d0yH9v+05J/5L09WZaBFCHvmGPiG2S3KP8lXrbAdAUTpcFkiDsQBKEHUiCsANJEHYgCb7imtzFF19cWr/llltK61XG2fuZM4c/zzqxZweSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJBjIRKnbbruttP7uu++W1tetW1dnO6iAPTuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJOGIGNnGOp1OdLvdkW0PyKbT6ajb7c76a9Ds2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgib5ht32l7b/a3mf7Fdv3FOsfsv2W7Z3F5abm2wUwrEF+vOJ9Sd+NiB22PybpZdtbitpPI+LHzbUHoC6DzM9+UNLB4vpx2/skXdF0YwDqdVaf2W0vkfR5SduLVXfZ3mV7g+0FPR4zabtruzs1NVWpWQDDGzjstj8q6Q+S7o2IY5J+IekzkpZpes//2GyPi4j1EdGJiM7ExEQNLQMYxkBhtz1X00H/TUT8UZIi4lBEfBARJyX9UtLy5toEUNUgR+Mt6UlJ+yLiJzPWL55xt69J2lN/ewDqMsjR+OskfUvSbts7i3Xfl7Ta9jJJIWm/pO800iGAWgxyNH6bpNm+H/t8/e0AaApn0AFJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5IY6ZTNtqck/XPGqkWSjoysgbMzrr2Na18SvQ2rzt4+GRGz/v7bSMN+xsbtbkR0WmugxLj2Nq59SfQ2rFH1xtt4IAnCDiTRdtjXt7z9MuPa27j2JdHbsEbSW6uf2QGMTtt7dgAjQtiBJFoJu+0bbf/d9mu2H2ijh15s77e9u5iGuttyLxtsH7a9Z8a6hba32H61WM46x15LvY3FNN4l04y3+tq1Pf35yD+z275Q0j8krZB0QNJLklZHxN6RNtKD7f2SOhHR+gkYtr8k6d+SnoqIzxbrfiTpnYh4tPiPckFEfG9MentI0r/bnsa7mK1o8cxpxiXdKunbavG1K+nrGxrB69bGnn25pNci4o2I+I+k30la1UIfYy8iXpD0zmmrV0naWFzfqOk/lpHr0dtYiIiDEbGjuH5c0ofTjLf62pX0NRJthP0KSW/OuH1A4zXfe0j6i+2XbU+23cwsLouIg9L0H4+kS1vu53R9p/EepdOmGR+b126Y6c+raiPss00lNU7jf9dFxBckrZS0tni7isEMNI33qMwyzfhYGHb686raCPsBSVfOuP0JSW+30MesIuLtYnlY0rMav6moD304g26xPNxyP/83TtN4zzbNuMbgtWtz+vM2wv6SpKtsf8r2RyR9U9LmFvo4g+35xYET2Z4v6asav6moN0taU1xfI+m5Fns5xbhM491rmnG1/Nq1Pv15RIz8IukmTR+Rf13SD9rooUdfn5b0t+LyStu9Sdqk6bd1/9X0O6I7JX1c0lZJrxbLhWPU29OSdkvapelgLW6pt+s1/dFwl6SdxeWmtl+7kr5G8rpxuiyQBGfQAUkQdiAJwg4kQdiBJAg7kARhB5Ig7EAS/wMNkrri4cnX+QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(x_train[9487],cmap='Greys')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train[9487] #答案為1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**3.輸入與輸出格式整理**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "原數據為28X28的矩陣，但標準神經網路只吃平平的數據，所以要調整成28*28=784長的向量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(60000, 28, 28)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train = x_train.reshape(60000,784)\n",
    "x_test = x_test.reshape(10000,784)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**1-hot encoding**\n",
    "\n",
    "1 -> [0,1,0,0,0,0,0,0,0]\n",
    "\n",
    "5 -> [0,0,0,0,0,1,0,0,0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.utils import np_utils"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train = np_utils.to_categorical(y_train, 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_test = np_utils.to_categorical(y_test, 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0., 1., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train[9487] #調整前答案直接為1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**4.神經網路建構**\n",
    "\n",
    "R(784) -> R(10)\n",
    "\n",
    "1.使用2個hidden layers\n",
    "\n",
    "2.每個hidden layers用500個神經元\n",
    "\n",
    "3.Activation Function 指名 sigmoid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Dense標準NN(Fully Connected Neural Networks)，SGD是標準Gradient descent\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Activation\n",
    "from keras.optimizers import SGD"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Sequential() #標準一層一層傳遞的神經網路為 Sequential，打開一個空的神經網路"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(Dense(500,input_dim = 784)) \n",
    "#加入隱藏層(500個神經元)，第一個隱藏層要告訴他input dimension\n",
    "model.add(Activation('sigmoid'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(Dense(500)) #第二層\n",
    "model.add(Activation('sigmoid'))   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(Dense(10)) #輸出10個數字，10個神經元\n",
    "model.add(Activation('softmax')) #輸出值機率加總為1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**5.組裝神經網路**\n",
    "\n",
    "1.決定使用的loss function是mse\n",
    "\n",
    "2.optimizer是標準的SGD，lr是學習率\n",
    "\n",
    "3.metrics = ['accuracy']則是看結果如何"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss = 'mse',optimizer = SGD(lr=0.1), metrics = ['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_1 (Dense)              (None, 500)               392500    \n",
      "_________________________________________________________________\n",
      "activation_1 (Activation)    (None, 500)               0         \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 500)               250500    \n",
      "_________________________________________________________________\n",
      "activation_2 (Activation)    (None, 500)               0         \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 10)                5010      \n",
      "_________________________________________________________________\n",
      "activation_3 (Activation)    (None, 10)                0         \n",
      "=================================================================\n",
      "Total params: 648,010\n",
      "Trainable params: 648,010\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary() #看結果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "392500"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#檢查第一隱藏層參數 392500，784與500個神經元相連，再加上Bias500(幾個神經元)\n",
    "784*500 + 500"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "250500"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#第二隱藏層 250500 參數\n",
    "500*500 + 500"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5010"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#輸出層參數 5010\n",
    "500*10 + 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "畫出結構圖"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.utils import plot_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaEAAALhCAYAAAD2GBszAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdf2gc550/8PfGcXLX0q5wi+RYV+VLSR0CLXt2OUdp0zNWzBmbzjrXRo5WquIW1mZ1NKmD9UeqrjDGQukfKxrcPyxW+49Y5F2iwCU7XEzB0p1MiORwhd32jsMiON0tdaOFg50r3F2Tus/3D90znpmdlWZ/SM+O9H7BkuzM7Mwzj9bz2XmeZ55PQAghQEREpMBDqgtARES7F4MQEREpwyBERETKMAgREZEyDzsXfPzxx3j11Vdx//59FeUhIqIdanh4GJqm2ZZV3QktLi4im81uW6GICJifn0epVFJdjLZXKpUwPz+vuhjUgPn5edfYUnUnJL355ptbWiAieiAQCOCVV17B4OCg6qK0tevXr2NoaIjXJx8aGhpyXc4+ISIiUoZBiIiIlGEQIiIiZRiEiIhIGQYhIiJShkGIaAcZHx/H+Pi46mK0rXK5jKmpKdXF8KWpqSkYhtHy/TIIEVHLGIaBQCCguhiuyuUyLl26hEOHDiEQCCAQCNQM2HK99dWOZH27vZzP5Oi6jnA4jEAggHA4vOnzoDMzM7bzPn78OIaHh1Eul1t7EsJhbm5OuCwmoi0EQMzNzakuRtNyudyWXj8avT5VKhWhaZpYXl4232cyGQFAxONx18+sra0JAGJtba2pMm+l5eVlAcD1ZS13IpEQAEQ+nxdCCJHP5wUAkUgkXPcr1zvrenl5WWiaJiqVSt1lHRwcFIODg1XLeSdERC1hGAZmZmZUF8NVKpVCKBRCb28vACAYDGJgYAAAMDEx4XpX0NnZaftvO/rNb36DYrEIIYT5WltbQzwet5V7dHQUABAKhWz/XVpaqtqnYRh46623XI/X29uL7u5upFKplp0DgxDRDlEul5HNZhEOh13f67puNsXIKYLK5bLZTAM8aIIZGRnB6uqquW+3ZinnskQiAV3XbesA9f1U5XIZo6OjOHbsmOv6RCKBSCTieboywzCQzWbNc5yZmbE1UXmpd+u2U1NT5vrFxcW6zq2vrw89PT22ZYuLi3jhhReqzhEAVlZWAMAsx5UrV6r2mUql8PLLL9c8Zn9/P0ZHR1vXLOe8NWJzHNH2Qwua4zRNszWhWN/LZqhisSgAiFgsZh7XuU2lUhGxWEwAEHfu3BFCPGiasl4b5L6sy5zvhRAiHo/XbPKqVyPXJ9lEWCwWq9bJfcXjcVtzlXO9laZpIplMCiHW60XTNFsTlZd6t342k8kIIYRYWFhwLUO9rMewkue4vLwsMpmMazPjwsKCWWa3v6X1XHK5XF3lqtUcxyBE1AZaEYTkfjYLCl62ceszaHRfrdTI9UlefN3I5bLPyBp4reslGSisF3DZLyODifzcZnUl+6Sc2zQTsPP5vK0cTvLHRTwer+rXWVtbM4NrrXMQYr2unN8NL9gnRESeyT4D2ZfgZxMTE5tuEwwGzX6OjZqa5Aze1v6Wp556CsD65Kr1kNs7mzW9lLeWt956C319fa7rpqamcPToUVQqFQDraRWsQ67feecdnDt3btNjBINBAK37bjAIERFhPbDk83nouo5oNOr6TMz09HTVMnlRlv1hXsnthWVQgXw1QgZOt4EU2WwWo6OjOHnyJILBIIaHh6Hrujkbua7rOHHiREPHbRaDEBHVFIvFVBdhW4VCIeRyOei6bnbmW8mEbG53So3WlXUASDPcBiRIkUgEwIOA2dXVBQA4f/48ACAcDuPxxx+vOQBlKzEIEVEVeWE8deqU4pI0TwYTr0/7a5qGTCbj2iwm8z3dvXvXXCb329/fX1e5kskkACCdTpv7aGZGh6WlJbMZ1cmZzVQGI7l8o7uxWndm8Xi8oXI6MQgR7RDOYcLW9/IiZ70QO3/NyyHKhmEgnU5D0zTbxUv+0pcBSg73BYCRkREA9jsFeTFVPUT74MGDAKqDkDx/t7uagYEB14vsyZMnoWkaJicnzc/duHEDsVjM7IvxWu+nT58GsN4H1NHRgUAggK6uLjOYyaHbhUJh03MsFAo4evRozfUXLlwA8OBvLP92cnk95PDuI0eO1P1ZNwxCRDuEbGKR/29939HRYfuvc3tgvYM9HA6jo6MDPT09SKfTtvU//vGPoWkannzySei6jt7eXvOu4fLlywAePHfy85//HMPDw609wQY9/fTTAIB79+6Zy+QFH1ivB7cmpytXrrjeQaRSKWiaZvvcT3/6U3Mbr/Xe2dmJYrFoBrtYLIZisWg+91OpVBCLxTwF8I0GJADrzxMtLCxgaWkJgUAAs7OzWFhY2PAztch6lPXarIBw3GvJ9LmNdo4RUf0CgQDm5uaUpPeWF1I//Jtv9Pok78ouXrxY1+cMwzCbrlQJh8PI5XJKy2A1Pj6Ojo6OuutSpveem5uzLeedEBHteNFoFEtLS7YmRC9UB6CVlRWMjY0pLYNVoVBAoVBANBpt2T4ZhIh2MWc/0k4lm9EmJyc99bG0g8XFRezbt8+c70611dVVTE9PI5VKtTQ4b1kQcs6ftJup7pglqsXZj7STdXZ2Ip1O4+bNm6qL4klfX585qKId6LqOy5cvt3xC14dbujeLS5cuuT7Y5ReGYeA//uM/8Otf/xq6rrdVm2y9DMNAR0dHXe3otZ4NUNFv4Cx/O5XN73ZbnQWDwbr7MmjdVtXblgWha9eu+ToIyWcLmplCQ3KbqXY73bp1q+7PCCHMiz+wPlJHVfu4s/xCCJTLZfOXu8qyEVFztiwI+Z0MHK0IQio1k+PFemFXdZGvVX5rkwADEJF/taxPyJpjIxwO15yKolb+jHpycMjPyzwezuaZZnN0tNJOy/HSLuWvhwxk1pTO1u+IfFmfVLeus55Xre+uPF/DMDAyMsI+QCKvnNNqN5rKQdM0EYvFzOnB5TTl1n1tlD/Daw6ORCJh5gWpVCpV07S3OkeH8xzq5fccL87Ptkv5N1ruJI+7trZWVVY5Db9bDhZN08wp++v57ubz+Zo5XWpBi1I57HRMNeNfW5pPSCaNsubhkDknrPvaLH+G20XF7YJkzeUhL2Rej1GvZoOQ2z68nqdzGxU5XrzsX1X5vZ5XPB63BQXn5xKJhADsSc+ceVm8fnedOVq8YhDyhkHIv7Y0CMlfmlU73+BXtPPltr3bMnmsTCbj+g9+s2PUq52CUKv31UjZ26n89Z5XsVg0A471czI4WhN6We+4hWjsu1uPWvvmi6+d9HILQi2ZtqfWtB/O5ZtND+K23rlsdXUVo6OjZj9BIpGwDR1s9RQkrdifl3rwWlet3FcjZW+n8tdzXjMzM+b0/E8++WTV50ZGRjA9PW0m/Hrttddw7do1z8dq9nsSCATwyiuv4Nlnn23o87vFe++9h6tXr5p5cMg/rl69ip6enqppe1pyJ4T/i3KbLZfvrc12m+2n1r5luzvg3rxT6xj1qnX8Zvbh5Tw3qtONmpbq2VcjZW+n8m92XvI4silN3tm4fU7eDWUyGZHL5cy+LOex6vnu1gNgc5wXbI7zry1N7y3zYmw2HUYr8mcEAgEYhoFQKIRr164hn8/b0sy2OkdHO/F7jpftLP/Kyoo5tb1M6CVnJ3YTCoUQi8UQiUQwMzNTNVXKTv5eESnljEqN/NKQI440TTN/bcrRQ7D88rWOhrK+isWibZ3s67EObpCDEYD1zmB5HNnOL210jHpZj99oh7O1PGtra3WdJ/7vl7ncJh6PC03TbPt3jjiTo72s9S77M9bW1sy68jI6zu3826X8biPrJLkPOSJSfr5YLIo7d+5UldX5OWvfkOT1u9so8E7IE94J+deWDkwQYj0YyAtKLBazDWm1/mMvFovmsOpYLFbVRGL9x1xrmbwYwdEUt9kx6uF2wWmkXmrtx8t5ygupvIgmk8mqYFgsFs31uVxOCCGq6l02NcXjcXPZZkFos3KrLL/XssljOT8vR8u5fS80TavZ5Oblu+sMsl4xCHnDIORftYIQ8wm1KT/leHHjx/IbhlE1IGG7qMwn5Ce8PvkX8wkRbeLNN980UysT0fZgEGpDfs/x4qfyj4+P26bnaSTdMfkHB5M0bmpqyhyU00q7KghZ5wnb6LXd+3Lye44XP5VfjphLJpPKZztXxTCMhr+r7bB/r8rlMi5duoRDhw7Z5hF006p/y1tN1q3bK5vN2raV8xvKuQ+d653kfIvS8ePHMTw83Pofls5OInb8EW0/KByYIKfd8sP+G70+VSoVoWmabS5D+fxYrQE6chCMcxRlO7GOJnW+rOWWA7nkiFG3KbSs5HpnXS8vLwtN0xoaLbylzwkRkT81k+qjHfbvVSqVQigUMp//CgaDGBgYALCersXtrkCmC2l1JtFW+s1vfoNisQixPtIZQgisra0hHo/byi2fpQyFQrb/Li0tVe3TMAy89dZbrsfr7e1Fd3c3UqlUy86BQYjIp6zpU6ypTaRGU2W0cyqRRpTLZYyOjuLYsWOu6xOJBCKRyKbNU9Jm9V5PWppm08709fVVPYS9uLiIF154oeocgfWHuAGY5XBrgk6lUnj55ZdrHrO/vx+jo6Ota5Zz3hqxOY5o+6GB5jhN08wHa+VzedamkkZTZcDSpKM6lYhTM7P8uz0XJvcln/9ypnxxO9Zm9e41LU2r085ItdKIyHNcXl6uen5TWlhYMMvs9neznot8rs+rLX9YlYgaV28Qkhcs64VE9g9YU1C4XUi8BAm3ZSpSiTg1cn1y5hyzkstln5E1yFrXS62s91annRGiOgWJk/whEY/Hq/p11tbWbLOF1Pq7yRlSavUn1cI+IaIdZH5+HoC9v+Kpp54CsP5A51aQ/QjWuRr9YGJiYtNtgsGg2c+xUVNTK+tdbu9swvRS3lreeuutmo8ZTE1N4ejRo+ZM8cPDw7Yh1++88w7OnTu36TGCwSCA1n0PGISIfGh6erpqmbw4yD4Yqk9nZyfy+Tx0XUc0GnV9JqaV9S63F5ZBBfLVCBk43QZSZLNZjI6O4uTJkwgGgxgeHoau62ZKDF3XceLEiYaO2ywGISIf0jQNgPvDwLFYbEuPvdX7VykUCiGXy5m5p5y2ot6tgz2a4TYgQZIzycuAKZ/fO3/+PAAgHA7j8ccfrznYZCsxCBH5kJxj7u7du+Yy+ct9q6Ye8msqERlMvD7tr2kaMpmMa7NYK+u91elBlpaWzCZTJxk8JRmM5PKN7sZq3ZnF4/GGyunEIETkQydPnoSmaZicnDR/ld+4cQOxWMzWJyB/ncsAIofoAuvZZAH7r3vnBVAOWzYMA+l0Gpqm2S5oje5/O4doHzx4EEB1EJL15nZXMzAw4HqR9VLv1v3JY1qPLdefPn0awHofUEdHBwKBALq6usxgJodub5anDVjP5SbzZ7m5cOECgAd/T/l3ksvrIYd3HzlypO7PunKOVODoOKLthwaGaMvRTMCD3E2tSvUh96kqlUgtjVyf5FBya7ZceX7Wlxu31Byb1bvbfmsda6O0MzLliJf0INa6rWVhYcGWbmdhYWHD7WvVixwNWO9MEkzlQNTG2i2VQ7um4mj0+iTvwC5evFjX5wzDMJuuVAmHw8jlckrLYDU+Po6Ojo6665KpHIho14pGo1haWrI1F3qhOgCtrKxgbGxMaRmsCoUCCoUCotFoy/bJIERENn5KxeGVfA5ocnLSUx9LO1hcXMS+ffvM+e5UW11dxfT0NFKpVEuDM4MQEdn4KRVHPTo7O5FOp3Hz5k3VRfGkr6/PHFTRDnRdx+XLl1s+oevDLd0bEfleu/UDtVIwGKy7L4PWbVW98U6IiIiUYRAiIiJlGISIiEgZBiEiIlKm5sAEOWU5EW2P27dvY+/evaqL0dZu374NgNcnP5qfn3efX885hcLt27ddp7Tgiy+++OKLr2ZeP/nJTzaftoeIvGu36XaI/IZ9QkREpAyDEBERKcMgREREyjAIERGRMgxCRESkDIMQEREpwyBERETKMAgREZEyDEJERKQMgxARESnDIERERMowCBERkTIMQkREpAyDEBERKcMgREREyjAIERGRMgxCRESkDIMQEREpwyBERETKMAgREZEyDEJERKQMgxARESnDIERERMowCBERkTIMQkREpAyDEBERKcMgREREyjAIERGRMgxCRESkDIMQEREpwyBERETKMAgREZEyDEJERKTMw6oLQOQX+Xwev/jFL6qW67qO3/72t+b7J554At/97ne3s2hEvhUQQgjVhSDygx/96Ee4evUqHn300Zrb/PGPfwQA8J8VkTdsjiPy6Dvf+Q6A9UBT6/XII4/ghz/8oeKSEvkH74SIPPrzn/+M7u5ufPzxxxtu99577+Gb3/zmNpWKyN94J0Tk0UMPPYShoSE88sgjNbc5cOAAvvGNb2xjqYj8jUGIqA6RSASffPKJ67q9e/fipZdeQiAQ2OZSEfkXm+OI6vTlL38ZH330keu6X/3qV/ja1762zSUi8i/eCRHV6fvf/z727t1btfwrX/kKAxBRnRiEiOoUiUTw6aef2pbt3bsXZ8+eVVQiIv9icxxRA0KhEH7961+bzwMFAgF8+OGH+PKXv6y4ZET+wjshogacPXsWe/bsAbAegA4fPswARNQABiGiBgwMDOD+/fsAgD179mB4eFhxiYj8iUGIqAEHDhzAt771LQDrD7G++OKLiktE5E8MQkQNGhoaAgB8/etfx/79+xWXhsiffDsw4dFHH6350CAR0W5z+/ZtHDlyRHUx6ubbVA6ffPIJnn/+eQwODqouCrWx9957D1evXsWbb765Jfs3DAOf//znd8QsCVevXgUAvPLKK4pLQvU6c+YMPvzwQwah7dbf34/+/n7VxaA2Jp/n4fdkc2+//TYA1hVtL/YJERGRMgxCRESkDIMQEREpwyBERETKMAgREZEyDEJEHo2Pj2N8fFx1MdpWuVzG1NSU6mL40tTUFAzDUF0MJRiEiHzCMIy2fR6pXC7j0qVLOHToEAKBAAKBQM2ALddbX+1I1rfbK5vN2rbVdR3hcBiBQADhcLhqvdPMzIztvI8fP47h4WGUy+UtOZe2JnwKgJibm1NdDGpzc3Nzwsdfc5tcLrel5zI4OCgGBwfr/lylUhGaponl5WXzfSaTEQBEPB53/cza2poAINbW1poq81ZaXl4WAFxf1nInEgkBQOTzeSGEEPl8XgAQiUTCdb9yvfNvuby8LDRNE5VKpe6y+vl6yDshIh8wDAMzMzOqi+EqlUohFAqht7cXABAMBjEwMAAAmJiYcL0r6OzstP23Hf3mN79BsViEEMJ8ra2tIR6P28o9OjoKYD3HlPW/S0tLVfs0DANvvfWW6/F6e3vR3d2NVCrV6lNpawxCRB6Uy2Vks1mEw2HX97qum00xpVLJ3EY20wAPmmBGRkawurpq7tutWcq5LJFIQNd12zpAfT9VuVzG6Ogojh075ro+kUggEols2jwlGYaBbDZrnuPMzIyticpLvVu3nZqaMtcvLi7WdW59fX3o6emxLVtcXMQLL7xQdY4AsLKyAgBmOa5cuVK1z1QqhZdffrnmMfv7+zE6Orq7muUU34k1DD6+/aTt06rmOE3TbE0o1veyGapYLAoAIhaLCSGErfnG2lQVi8UEAHHnzh0hxIOmKWs55b6sy5zvhRAiHo/XbPKqVyPNcbKJsFgsVq2TZY3H47bmKud6K03TRDKZFEKs14umabYmKi/1bv1sJpMRQgixsLDgWoZ6WY9hJc9xeXlZZDIZ12bGhYUFs8xuf0vrueRyubrK5efrIYMQ7Wit7BPyEhS8bOPWZ9DovlqpkSAkL75u5HLZZ2QNvNb1kgwU1gu47JeRwUR+brO6kn1Szm2aCdj5fN5WDif54yIej1f166ytrZnBtdY5CLFeV87vhhd+vh6yOY5om8k+A9mX4GcTExObbhMMBs1+jo2amubn5wHY+4meeuopAMD169frKpfc3tms6aW8tbz11lvo6+tzXTc1NYWjR4+iUqkAAIaHh21Drt955x2cO3du02MEg0EAO+O74RWDEBFtuc7OTuTzeei6jmg06vpMzPT0dNUyeVGW/WFeye2FZVCBfDVCBk63gRTZbBajo6M4efIkgsEghoeHoeu6mT5E13WcOHGioePuBgxCRIrEYjHVRdhWoVAIuVwOuq6bnflWmqYBgOudUqN1ZR0A0gy3AQlSJBIB8CBgdnV1AQDOnz8PAAiHw3j88cdrDkDZ7RiEiLaZvDCeOnVKcUmaJ4OJ16f9NU1DJpNxbRaTCSrv3r1rLpP7rTfHUTKZBACk02lzH83M6LC0tGQ2ozrJ4CnJYCSXb3Q3VuvOLB6PN1ROP2IQIvLAOUzY+l5e5KwXYueveTlE2TAMpNNpaJpmu3jJX/oyQMnhvgAwMjICwH6nIC+mqodoHzx4EEB1EJLn73ZXMzAw4HqRPXnyJDRNw+TkpPm5GzduIBaLmX0xXuv99OnTANb7gDo6OhAIBNDV1WUGMzl0u1AobHqOhUIBR48erbn+woULAB78jeXfTi6vhxze7ccMqY1iECLyQDaxyP+3vu/o6LD917k9sN7BHg6H0dHRgZ6eHqTTadv6H//4x9A0DU8++SR0XUdvb69513D58mUAD547+fnPf47h4eHWnmCDnn76aQDAvXv3zGXygg+s14Nbk9OVK1dc7yBSqRQ0TbN97qc//am5jdd67+zsRLFYNINdLBZDsVg0n/upVCqIxWKeAvhGAxKA9eeJFhYWsLS0hEAggNnZWSwsLGz4mVpkPcp63Q0CotGeOsUCgQDm5ubMW3giN9evX8fQ0FDDHdLNkhdSP/wzGxoaAgDMzc3V9Tl5V3bx4sW6PmcYhtl0pUo4HEYul1NaBqvx8XF0dHTUXZd+vh7yToiImhKNRrG0tGRrQvRCdQBaWVnB2NiY0jJYFQoFFAoFRKNR1UXZVrs6CDmnACFqJWc/0k4lm9EmJyc99bG0g8XFRezbt8+c70611dVVTE9PI5VKKQ/O221XB6FLly4hEonU/QxCuzAMAysrK5iZmWk4kNaaqj4QCGBqagq6ru/aPCfNcvYj7WSdnZ1Ip9O4efOm6qJ40tfXZw6qaAe6ruPy5cttPaHrVtnVQejatWuqi9CURCKBf/qnf8L58+cbDqTi/2YGliqVijmM9Pjx45iZmdm9eU6a1IqHJP0kGAzW3ZdB6y5evLgrAxCwy4OQ3125csV1pt56Wb/81qaAUChkTrdS6yl3IqJm7KogZJ0mPhwO13yautYU8PVMIy8/L6eidw5TbXaaea+afY6ks7MTFy5cgK7ruHXrlm3dTqonIlJkO2dLbSU0MGuspmkiFouZM9zKmXat1bDRFPBep5FPJBLm1PaVSqVqpuFWTzPvPAcrr1P9b7QPObOv16ny26medlJm1a3WaGZVUq+R62G78O2/znorXeY9sU4lLy+u1ovUZlPAu12sncvgmI5e5ovxeox6bRRAWrUPv9YTg5B3DEL+5ecg9HAr76ra2bvvvgsAthExbkMhrVPAW01MTHjuf4nFYujq6kImk8HJkyfR2dlp65huxTFU81s9yTQBVJtsKmVd0bZSHQUbhTojP2r80ncur7XdRuudy+7cuWNrknImqNrsGPVqxf422oe8Y7TegfilnuSdEF987fQX74R2mNXV1YafIzh48CByuRwKhQKmp6fNBFXO4avNHGM7/fKXvwQAHDt2rGqdX+pJ7IIh0s1qdNoeUs/PKSF2zeg4ObX7Zk90t2IK+EAgAMMwEAqFcO3aNeTzeVumxFZPM7+VyuUy3njjDWiaZpuQkfVERC2h+lasUajz9lOOztI0zRyRJUdbAQ9GbcnOceerWCza1skRdtbBDbKTHVhvupLHKRaLtqamjY5RL+vxnXnthfA2Oq7WPuRIN03TbAMI/FRPHJjgHQcm+Fe918N2smvuhHp6elAsFtHd3Y3HH38cIyMj+OpXv1o1Xf5GU8DXM33/yy+/jPn5eQQCAczPz9uamDabZt6rQCBgO77Mm9KKfQQCAdy8eRNjY2PI5XJVT3P7qZ6IqH0xlQPtaKpTOfgJ+4T8y8/Xw11zJ0RERO2HQYiIiJRhEGozG6VWsL6I2g1HLjZuampq104QzCDUZoRj+v9aL/IHwzC29EfDVu/fq3K5jEuXLuHQoUPmD6VaE+f67UdVoVCwlXVkZKRqG13XEQ6HEQ6Ha6ZV2Wib48eP79qUKQxCRFvIOfO43/bvhWEYiEajOHv2LPr6+lCpVJDJZDAxMeEaiIQlh9Xa2lrb/6j64IMPbO9PnTple5/NZjEzM4N0Oo10Oo13330XMzMzdW0TCoUwNja2K1OmMAgRbRHDMKouRn7av1epVAqhUMhMlR0MBjEwMABgfZ6/bDZb9Rk55N8Pidz2799va4XQNM1cVyqVEIlEMDY2hmAwiGAwiFgshvPnz5sPxnvZBgB6e3vR3d1t5vDaLRiEiFxYc09Z8x1Jbk1JzmWJRMJsdpHLy+Wy2SwDADMzM2YTjzW/VaP7B5rPIVWPcrmM0dFR1ymdZBkjkYhrIHKzWb3Xk6uqFbmoSqUSwuEwxsfHsbKyUrX+/fffBwAcOHDAXPbYY48BeHAH5WUbqb+/H6Ojo7uqWY5BiMjF8PAw/vCHP5hNR7qu25pKrCnRpWKxaHtvnelb/oru6uoy+wRWVlZw7tw5VCoVAMCTTz5pBqJG97/dbt++DQB44oknXNdfvHgR8XgckUhk0ymzgM3rPRqNIhKJmPWnaRqKxSJ0Xcfrr79u7qdcLiMajaK7uxtCCFy4cAHPPfecpzJYye0nJibwzDPPIBwO2wLE0tISANgeoJZ3d/IHgpdtJFmPsl53hW2bm6HF4ONpKmj7NDJtj5zOyTpV0fLysgBgJtgTwnvOpM22EWJ9iiTAPpN4o/tvVCPT9jgTEVrJ5ZVKxZwt3ZrPy/m5VtZ7K3N2VSoVkc/nzXNNJpMblsW53Ms21mM5vwde+Pl6yCBEO1ojQSgWi9W8OGiaZi5rZRBq9LOqg9BGx7cul6sHjuoAACAASURBVPMAWuchdH6ulfVuTRHifDUjmUxuWhbn8nqC0EbLN+Ln6yGb44gcpqenq5bJBIi1ht/Sxjo7O5HP56ua16xaWe9ye9HixxvOnDljK4t1kIJTLBbzvM1uxiBE5CAvGm6dw1t90djJF6VQKIRcLgdd15FIJKrWb0W9Wwd7tIIc2Sa5lVkOkDh8+LDnbXYzBiEiBzkJ5N27d81l8pd7f3//lhxTXiydz6C0OxlMvD7bImetn5iYqFrXynrfqlxUhmHYynLixImqMt+7d8+2zss2TnLm+N2AQYjI4eTJk9A0DZOTk+av1xs3biAWi9kS+8lfxDKAWIfwyqfqrb+CnRdAOWzZMAyk02lommZruml0/9s5RFtmvHUGIVlvbnc1AwMDrhdZL/Vu3Z88pvXYcv3p06cBrI9qk+lJurq6zAAih25vNFoum83ahnWXSiXcunXL9h3o6elBMpnE7OwsDMOAYRiYnZ1FMpk0R8N52cZ6DAA4cuRIzXLtOEp7pJoAH3fE0fZpNKnd2tqaSCaTZidxJpOpShpYLBbNDvBcLieEWO8Qz2QyZue7HPUWj8dtHfIAzKSB+L8RV63av5dEhm4aGZggBxwsLy+by+BxMIC1g9+6v43q3W2/tY5VLBbNEW2xWMyWDDEej4tYLOZaBimXy5n7jMfjIp/Pb7qtpmliYWGh4W3kaEBnEsnN+Pl6yHxCtKO1Yz4h+VBpO5UJaDyfkLwDsyYk9MIwDHPggSrhcBi5XE5pGazGx8fR0dFRd136+XrI5jgiako0GsXS0pLrjAIbUR2AVlZWMDY2prQMVoVCAYVCAdFoVHVRthWDENE2ck5BsxMEg0GkUilMTk7WPSOBKouLi9i3b585351qq6urmJ6eRiqVUh6ctxuDENE26urqcv1/v+vs7EQ6ncbNmzdVF8WTvr4+c1BFO9B1HZcvX/bFhK6t9rDqAhDtJu3WD9RKwWCw7r4MWreb6413QkREpAyDEBERKcMgREREyjAIERGRMr4emDA0NIS3335bdTGojclpUM6cOaO4JO1PJlJjXdF28u2MCWNjY/jwww9VF4N2uZs3b+KrX/0q9u/fr7ootIvt2bMHP/vZz3z5PfRtECJqB36eLoWoHbBPiIiIlGEQIiIiZRiEiIhIGQYhIiJShkGIiIiUYRAiIiJlGISIiEgZBiEiIlKGQYiIiJRhECIiImUYhIiISBkGISIiUoZBiIiIlGEQIiIiZRiEiIhIGQYhIiJShkGIiIiUYRAiIiJlGISIiEgZBiEiIlKGQYiIiJRhECIiImUYhIiISBkGISIiUoZBiIiIlGEQIiIiZRiEiIhIGQYhIiJShkGIiIiUYRAiIiJlGISIiEgZBiEiIlKGQYiIiJQJCCGE6kIQ+UEqlcI//MM/4MknnzSX/fa3v8UXvvAFfOYznwEA/P73v8c3v/lNvPPOO6qKSeQrD6suAJFfrK2t4dNPP8W//du/2ZYbhmF7r+v6dhaLyNfYHEfkUSQSQSAQ2HCbhx9+GD/96U+3qURE/sfmOKI6/M3f/A1++ctfotY/m0AggI8++giPP/74NpeMyJ94J0RUh+9973vYs2eP67qHHnoIR44cYQAiqgODEFEdXnzxRfz5z392XRcIBHD27NltLhGRvzEIEdVh//79OHr0aM27of7+/m0uEZG/MQgR1emll16q6hPas2cPjh07hi9+8YuKSkXkTwxCRHX6+7//+6o7ISEEXnrpJUUlIvIvBiGiOgWDQZw8eRIPP/zgMbu9e/fi+eefV1gqIn9iECJqwPDwMO7fvw9g/dmgb3/72/jc5z6nuFRE/sMgRNSAb3/72/jLv/xLAMD9+/cxNDSkuERE/sQgRNSAv/iLv8ALL7wAAPjsZz+LU6dOKS4RkT81PXfcn/70J+RyObNpgmi3+Ku/+isAwOOPP45cLqe4NETbr7e3F1/60pea24lo0j/+4z8KAHzxxRdffO2y1w9+8INmQ4ho+k7ov//7vwGg5lxaRLtRIBDA3NwcBgcHVRelrV2/fh1DQ0O8fvjQ0NAQ/vjHPza9H/YJERGRMgxCRESkDIMQEREpwyBERETKMAgREZEyDEJERKQMgxBRGxsfH8f4+LjqYrStcrmMqakp1cXwpampKRiGoboYDEJEVJthGAgEAqqL4apcLuPSpUs4dOgQAoEAAoFAzYAt11tf7axQKNjKOjIyUrWNrusIh8MIh8PQdd11Pxttc/z4cQwPD6NcLm/JOXjV9MOqRLR1rly5ovT4t27dUnr8WgzDQDQaxdjYGHp7e1GpVHDjxg1EIhEA1fUmhEC5XEZXVxfW1tbQ2dmpotieffDBB7b3zrkJs9ksrl+/jnQ6DQB47bXX8PHHH+PcuXOetwmFQhgbG0M0GkU6nUYwGNzKU6qJd0JE5MowDMzMzKguhqtUKoVQKITe3l4A6zmeBgYGAAATExPIZrNVn5GBp90DELCeRl4IYb40TTPXlUolRCIRjI2NIRgMIhgMIhaL4fz58ygUCp63Adbnfuvu7kYqldr2c5QYhIjaVLlcRjabRTgcdn2v6zoCgQDC4TBKpZK5jWyCAYCZmRmzOWd1ddXct1uzlHNZIpEwm3Csy1X3U5XLZYyOjuLYsWOu6xOJBCKRiGsgcmMYBrLZrHmOMzMztiYqL/Vu3XZqaspcv7i4WPf5lUolhMNhjI+PY2VlpWr9+++/DwA4cOCAueyxxx4D8OAOyss2Un9/P0ZHR9U1yzU7+dzc3JxowW6IdhQAYm5urql9aJpmThTpfL+8vCyEEKJYLAoAIhaLmcd1blOpVEQsFhMAxJ07d4QQQqytrdn2bd2XdZnzvRBCxONxEY/Hmzo3qZHrRy6XEwBEsVisWif3FY/HBQCRz+dd11tpmiaSyaQQYr1eNE0TmqaJSqVirt+s3q2fzWQyQgghFhYWXMvg9fzkS9M0sba2Zq6Xf0u3c9c0zfM2kjyXXC5XVzkHBwfF4OBgXZ9xwyBEtAVaEYTkfjYLCl62yefzAoBIJBJN76uVGrl+yADjRi6vVCpm8JCB17pekoHCepFfXl4WAMxgIj+3WV1lMhnXbRoJ2JVKReTzefNcZZCsVRbnci/bWI/l/G540aogxOY4ol0gFAoBAEZHRxWXpHkTExObbhMMBs1+jo2amubn5wHY+4meeuopAOszfNdDbu9s1vRSXqdgMIhQKIQrV64gmUzWHP3WCnJAgqrvBoMQEe1InZ2dyOfz0HUd0WjU9ZmY6enpqmXyolzvhV9uLywDCuSrGWfOnLGVxTpIwSkWi3nepl0wCBHtIu12AdpqoVAIuVwOuq4jkUhUrZcXa7c7pUbryjoApBXkyDbJrcxygMThw4c9b9MuGISIdgF5YXQ+b+JHMph4fdpf0zRkMhnXZjGZdPDu3bvmMrnf/v7+usqVTCYBAOl02txHK2Z0MAzDVpYTJ05UlfnevXu2dV62cYrH402Vs1EMQkRtyjlM2PpeXuSsF2Lnr3k5RNkwDKTTaWiaZmumkb+uZYCyDgeWT+hbf1HLi6nqIdoHDx4EUB2E5Pm73dUMDAy4XmRPnjwJTdMwOTlpfu7GjRuIxWLo6+ur2t9G9X769GkA631AHR0dCAQC6OrqMgOIHLptfU7HKZvN2oZ1l0ol3Lp1yywLAPT09CCZTGJ2dhaGYcAwDMzOziKZTKKnp8fzNtZjAMCRI0dqlmtLNTuygaPjiKqhBaPjYBmm6/Zy28a6LJ/PmyPEksmkOeRYKhaL5no5PFcOMZajxeSoung8bi5TPURbDi+Xw6WFcK8rN87hyXJ/yWTS/Fwmk7HVldd6F2K9TuWItlgsZhtGHo/HRSwWcy2DZB2eHY/HNxzeLbfVNE0sLCw0vI0cDWgdIehFq0bHBYRorteMOeKJqgUCAczNzZnNPdt9bAC++DfZ6PVD3pVdvHixrs8ZhqFsehopHA4jl8spLYPV+Pg4Ojo66q7LoaEhAMDc3FxTx2dzHBH5TjQaxdLSkuuMAhtRHYBWVlYwNjamtAxWhUIBhUIB0WhUWRl2VBDarrZq1W3ifsW/z9Zz9iPtVPI5oMnJyQ37WNrJ4uIi9u3bZ853p9rq6iqmp6eRSqWUBmffBqHtmmK+XaeyNwwDKysrmJmZMee0apWRkZGmz3m3/31U6erqcv3/naizsxPpdBo3b95UXRRP+vr6zEEV7UDXdVy+fFn5hK6+TeXgNsX8Vkx7v13HqZccptrI09gbKZVK5gN8hULBfNK+Xrv976OKH/qBWikYDNbdl0Hr2qXefHkntF1TzLfzVPZXrlzZkovt/Py82WnqnG3XK/59iMgrJUFIXjys2RCd7ddu06tLblPMW6dbX1lZqZlFUY7VDwQCKJVKG5Zls+NsVt5Gp4NvlXr7RgzDQKVSMZ8NOX/+/Ibb8u9DRE1rdox3I+P85TTja2trrlOiC7E+nt/6LEIsFrO9h2N8vnPaezk7rtvzDNbx95uVZbPjWJe3Yjr4ermVxXqe9TzPkclkzHqRz03Uek6Bf5+NoUWzaO90fM7Qv3ydykE+tGUWosaU6M7p1a0PebldaJzL5ENj1gfPKpWK7cK3WVm8HKeV08HXq9nPSzLnjCQfUrROIS/x77M5BiFvGIT8a0c8rFoqlTA/P29OIS73EQ6Hoev6hvt0eyDPuaxQKOCv//qvkclkzNS/i4uL+MIXvlDV4V6rLF6OMzIygunpads2hmGgo6MDmqaZfSxe9lWvVj2YKKcKsU4PEggEbOWX+PfZXCAQwNNPP101RQrZlUol3L59u+552ki927dv49lnn/Xvw6ozMzP44Q9/6DrleKtyZ4RCIWiaZssL8s///M9VF7iNyuJFK6eDV+WNN97Ac889V9VPo+t61azA/PsQUcs0eyvVyO20bM6R8yqhRrv+RvMmOT9Ta5k81vLysigWi1UpbDcri5fjyPI6517CJv0XtZbVo9nPC7HeNGVtlpJkk5xzHf8+mwOb4zxhc5x/+TqzaiQSAYCaTRXyF+/09LQ5W22pVDJn9q2HbF6anZ3F+++/j7/927+tqyxetHI6eBVmZ2dx8uTJquVudyoA/z5E1DpKgpC8iJVKJVtTj3VKdE3TMD09bU6J/vrrr+PVV1+t2oecYr7WdCWdnZ2Ix+OYnp7G7373u6rpKTYri5fjtHI6+HpYP++WW8XLEO1sNosvfvGLNaftCIVC0HXdTAsA8O9DRC3U7K1UI7fTzunh5Qgo67Tncrnc7s6dOxvuA6g9jbvc1rkPL2XxepxWTgfvhfOzbvvYbIi287PW+q91DLkN/z4bA5vjPGFznH/tiNFxRDuVylQOfsLrh38xlQMREfkegxAR+ZY17TjVZ2pqyrUvebsxCLUR53xqtV5EG9nq9Bbtkj6jXC7j0qVLOHTokG1uQTd++3dUKBRsZXUbearrOsLhsPnwuJuNtjl+/DiGh4eVD7phEGojYn0apU1fRBtxS2/hp/17YRgGotEozp49i76+PlQqFWQyGUxMTLgGIiEE1tbWAABra2tt/+/IOYP9qVOnbO+z2SxmZmaQTqeRTqfx7rvvVs0ov9k2oVAIY2NjiEajSu+IGISIdpCtTm/RLukzUqkUQqGQmaU0GAyaUz9NTEzYHimQZPI21UncvNi/f7/th6d1tpBSqYRIJIKxsTEEg0EEg0HEYjGcP3/ezDLrZRsA6O3tRXd3N1Kp1Lafo8QgRNQmNks34daU5FxWK72FbJYBYKbGGBkZsT171ej+ge1NqV4ulzE6Oopjx465rk8kEohEIq6ByE0r03zIPiq5Xs7JWI9SqYRwOIzx8XGsrKxUrX///fcBAAcOHDCXPfbYYwAe3EF52Ubq7+/H6OiosmY5BiGiNjE8PIw//OEPZtORruu2phLZnGRVLBZt762JDuWv6K6uLrNPYGVlBefOnUOlUgEAPPnkk2YganT/2+327dsAgCeeeMJ1/cWLFxGPxxGJRGy/+mvZrN6j0SgikYhZf5qmoVgsQtd1vP766+Z+yuUyotEouru7IYTAhQsX8Nxzz3kqg5XcfmJiAs888wzC4bAtQCwtLQGwzyIi7+7kDwQv20iyHmW9brtmHzTiw2ZE1VDnw6qtTDfhZRshHjzom0gkmt5/oxpNBVPrM3J5pVIx5wy0PgTt/Fwr613Oc+jcpp6cXlKlUhH5fN48V2tKlVr1b13uZRvrsZzfAy98nU+IaKerNwjJ5H1W8uJQb54mr0Go0c+qDkIbHd+6XM6goWmaGWScn2tlvVsTIzpfzUgmk5uWxbm8niC00fKN+HoCUyKyY7qJ1uvs7EQ+n69qXrNqZb3L7UWLR7SeOXPGVpaNUprEYjHP27QLBiGiNmCdiNVpqy8a7XZRaqVQKIRcLgdd15FIJKrWb0W9O/NvNUuObJPcyiwHSBw+fNjzNu2CQYioDahINyEvls5nUNqdDCZen23RNM18hsiplfWeTCYBAOl02txHK2Z0MAzDVpYTJ05UlfnevXu2dV62cYrH402Vs1EMQkRtwEu6CeDBr3MZQKxDeOVT9c70FlZy2LJhGEin09A0zdZ00+j+t3OI9sGDBwFUByFZb253NQMDA64X2Vam+Th9+jSA9VFtMsVJV1eXGUDk0O2NRstls1nbsO5SqYRbt27ZvgM9PT1IJpOYnZ2FYRgwDAOzs7NIJpPmaDgv21iPAQBHjhypWa4t1WynEgcmEFVDA6kcNks3IYQQxWLR7ACXWWg1TROZTMbsfHemt5DlAdaz4crPJ5PJlu1/s7QhtTRy/ZADDpaXl81l8DgYwNrBb91fq9J8FItFc0SbMz2NTEPiVgYpl8uZ+4zH4xtmL5bbapomFhYWGt5GjgZ0Zh7eDFM5ELWxdkvlIB8qbbd/p41eP+Qd2MWLF+v6nGEYNRM4bpdwOIxcLqe0DFbj4+Po6Oiouy6ZyoGIdq1oNIqlpSXXGQU2ojoAraysYGxsTGkZrAqFAgqFAqLRqLIyMAgR7XC1Uqv7WTAYRCqVwuTkZN0zEqiyuLiIffv2mfPdqba6uorp6WmkUimlwZlBiGiH6+rqcv1/v+vs7EQ6ncbNmzdVF8WTvr4+c1BFO9B1HZcvX1Y+oevDSo9ORFuu3fqBWikYDNbdl0Hr2qXeeCdERETKMAgREZEyDEJERKQMgxARESnDIERERMo0PTruM5/5DADYUgIT0foT5fKpctoYrx/+9IMf/KDpfTQ9bc+f/vQn5HI53L9/v+nCEPnNmTNn8Morr+DZZ59VXRSibdfb24svfelLTe2j6SBEtJu12xxxRH7DPiEiIlKGQYiIiJRhECIiImUYhIiISBkGISIiUoZBiIiIlGEQIiIiZRiEiIhIGQYhIiJShkGIiIiUYRAiIiJlGISIiEgZBiEiIlKGQYiIiJRhECIiImUYhIiISBkGISIiUoZBiIiIlGEQIiIiZRiEiIhIGQYhIiJShkGIiIiUYRAiIiJlGISIiEgZBiEiIlKGQYiIiJRhECIiImUYhIiISBkGISIiUoZBiIiIlGEQIiIiZRiEiIhImYdVF4DIL/7nf/4Hv//976uWl8tl3L1713wfDAbxhS98YTuLRuRbASGEUF0IIj949dVX8cYbb3jalv+siLzhnRCRR4cPH950m0AggGeeeWYbSkO0M7BPiMij559/Ho8++uim27388svbUBqinYFBiMijz33uc9A0DQ8/XLsB4dFHH4WmadtYKiJ/YxAiqsPg4CDu37/vum7v3r14/vnn8dnPfnabS0XkXwxCRHU4depUzSDz6aef4nvf+942l4jI3xiEiOrw6KOP4syZM9i7d2/Vus9//vP4u7/7OwWlIvIvBiGiOg0NDeHTTz+1Ldu7dy9efPFF1+BERLXxOSGiOt2/fx9dXV34z//8T9vyf/mXf8HRo0cVlYrIn3gnRFSnPXv24Hvf+x4eeeQRc9n+/fvxrW99S2GpiPyJQYioAYODg/jkk08AAI888ggGBwfx0EP850RULzbHETXo8ccfR6lUAgD867/+K77+9a8rLhGR//CnG1GDhoeHAQD/7//9PwYgogb5du64sbExfPjhh6qLQbvYf/3XfwEA/vd//xdnzpxRXBrazfbs2YOf/exn2L9/v+qi1M23zXGBQAAA0N/fr7gk1M5KpRJu3769Zd+Tjz76CN3d3bZBCn51+/ZtAMDTTz+tuCRUr/n5eczNzWFwcFB1Uerm2zshAL6tdNo+169fx9DQEN58803VRWl7Q0NDANb/XZG/yB/lfsQ+ISIiUoZBiIiIlGEQIiIiZRiEiIhIGQYhIiJShkGIyKPx8XGMj4+rLkbbKpfLmJqaUl0MX5qamoJhGKqLoQSDEJFPGIbRtkNxy+UyLl26hEOHDiEQCCAQCNQM2HK99dXOCoWCrawjIyNV2+i6jnA4jHA4DF3XXfez0TbHjx/H8PAwyuXylpxDO/P1c0JE2+nKlStKj3/r1i2lx6/FMAxEo1GMjY2ht7cXlUoFN27cQCQSAVBdb0IIlMtldHV1YW1tDZ2dnSqK7dkHH3xge3/q1Cnb+2w2i+vXryOdTgMAXnvtNXz88cc4d+6c521CoRDGxsYQjUaRTqcRDAa38pTaCu+EiHzAMAzMzMyoLoarVCqFUCiE3t5eAEAwGMTAwAAAYGJiAtlstuozMvC0ewAC1tN0CCHMl6Zp5rpSqYRIJIKxsTEEg0EEg0HEYjGcP38ehULB8zYA0Nvbi+7ubqRSqW0/R5UYhIg8KJfLyGazCIfDru91XUcgEEA4HDZn1i6Xy2YTDADMzMyYzTmrq6vmvt2apZzLEomE2YRjXa66n6pcLmN0dBTHjh1zXZ9IJBCJRFwDkRvDMJDNZs1znJmZsTVReal367ZTU1Pm+sXFxbrPr1QqIRwOY3x8HCsrK1Xr33//fQDAgQMHzGWPPfYYgAd3UF62kfr7+zE6Orq7muWETwEQc3NzqotBbW5ubk604muuaZoAYO7L+n55eVkIIUSxWBQARCwWE0IIc711m0qlImKxmAAg7ty5I4QQYm1tzbZv676sy5zvhRAiHo+LeDze9PkJIcTg4KAYHBys6zO5XE4AEMVisWqdLGs8HhcARD6fd11vpWmaSCaTQoj1etE0TWiaJiqVirl+s3q3fjaTyQghhFhYWHAtg9fzky9N08Ta2pq5Xv4t3c5d0zTP20jyXHK5XF3l9PP1kEGIdrRWBSEhqoOAW1Dwsk0+nxcARCKRaHpfrdRIEJIBxo1cXqlUzOAhA691vSQDhfUiv7y8LACYwUR+brO6ymQyrts0ErArlYrI5/PmucogWasszuVetrEey/nd8MLP10M2xxFts1AoBAAYHR1VXJLmTUxMbLpNMBg0+zk2amqan58HYO8neuqppwCsT0RbD7m9s1nTS3mdgsEgQqEQrly5gmQyWXP0WyvIAQk74bvhFYMQEW25zs5O5PN56LqOaDTq+kzM9PR01TJ5Ua73wi+3F5YBBfLVjDNnztjKYh2k4BSLxTxvs5sxCBEpstsuQKFQCLlcDrquI5FIVK2XF2u3O6VG68o6AKQV5Mg2ya3McoDE4cOHPW+zmzEIEW0zeWF0Pm/iRzKYeH3aX9M0ZDIZ12YxmRvs7t275jK533qTEiaTSQBAOp0299GKGR0Mw7CV5cSJE1Vlvnfvnm2dl22c4vF4U+X0EwYhIg+cw4St7+VFznohdv6al0OUDcNAOp2Gpmm2Zhr561oGKOtwYPmEvvUXtbyYqh6iffDgQQDVQUiev9tdzcDAgOtF9uTJk9A0DZOTk+bnbty4gVgshr6+vqr9bVTvp0+fBrDeB9TR0YFAIICuri4zgMih29bndJyy2axtWHepVMKtW7fMsgBAT08PkskkZmdnYRgGDMPA7Owskskkenp6PG9jPQYAHDlypGa5dhylwyKaAB+PBqHt06rRcbAM03V7uW1jXZbP580RYslk0hxyLBWLRXO9HJ4rhxjL0WJyVF08HjeXqR6iLYeXy+HSQrjXlRvn8GS5v2QyaX4uk8nY6sprvQuxXqdyRFssFrMNI4/H4yIWi7mWQbIOz47H4xsO75bbapomFhYWGt5Gjga0jhD0ws/Xw4AQTfbUKRIIBJjemzYl03ur+prLUVl++GfWaHpveVd28eLFuj5nGIby6WnC4TByuZzSMliNj4+jo6Oj7rr08/WQzXFE1JRoNIqlpSXXGQU2ojoAraysYGxsTGkZrAqFAgqFAqLRqOqibCsGIaIt4uxH2qnkc0CTk5Mb9rG0k8XFRezbt8+c70611dVVTE9PI5VKKQ/O221XByHnPFRErdTV1eX6/ztRZ2cn0uk0bt68qboonvT19ZmDKtqBruu4fPmyLyZ0bbVdHYQuXbqESCSypU9Ab6VSqYSRkRFzUsxGJmh0y+0iX1NTU9B1fdcm22qWaOFDkn4QDAbr7sugdRcvXtyVAQjY5UHo2rVrqovQMMMwUCgUcO3aNVQqFRw9ehTPPfdc3QFVCIG1tTXzfaVSMS+ax48fx8zMzK5NtkVEW29XByE/u3XrlvnciDV/SyNNi9ZfYNb26FAoZM75VWuqFSKiZuyqIGTNVRIOh2tO6VErD0k9uUzk52U+FGcK42ZzndSaj8o5vUmzDzN2dnbiwoUL0HW9KrOnH+qJiNqcoueTmoYGHs7SNE3EYjHz4Tc53bu1GjbKQ+I1l0kikTAfjKtUKlXT3bcq14mVnALemYfE68OMznpw27fXfC3tVE+tTOWw0zXysCq1h0auh+3Ct/866610+bSyNZ+JvLhaL1Kb5SFxu1g7l8HxxLN8qtzrMRqxsLBgS/5Vr42CkNt6v9QTg5B3DEL+xSCkQL2VvlF2Q+ty669458tte7dl8ljOKUe8HqMRmqbZpk6pV71ByC/1JIMQX3zt9Jdfg9DD2CXccpW4seYhadSrr76K3/3ud4hEIgDWZxq2YA8IBAAAIABJREFUDl1txTGsstksNE3bsgfv5IAE66STfqunN998s+l97HRXr14FALzyyiuKS0L1OnPmjOoiNGzXBKF6ra6uNvww28GDB5HL5VAoFDA9PW1mSXQ+Q9HMMaRCoYB///d/x5UrV5raz0Z++ctfAgCOHTtWtc4v9VRvKoDd6O233wbAuqLttWtGx8n8IptNK9KKPCSBQACGYSAUCuHatWvI5/O2dL2tynVSLpdx8+ZNWwAqFArm1P+tUC6X8cYbb0DTNNsU9n6qJyJqY2pbAxuHOttA5egsTdPMEVlytBXwYNSW7Bx3vorFom2d7MOwDm6QnezAeue5PE6xWBSJRMIsy0bH8EqOHHPbj3WEnJfRcdZzsPbNyJFumqZVTS3vl3riwATvODDBv+q9HraTXXMn1NPTg2KxiO7ubjz++OMYGRnBV7/6VTPT4+XLlwGsPxdTLBbN/o9YLIZisYienh7b/F8dHR22/wL2+cFefvllzM/PIxAIYH5+3tbEtNExvLp06VLN2RGefPJJz/sJBAK2c5AJwAKBAG7evImxsTHkcrmqKUX8Uk9E1N6YT4h2NNX5hPyk0XxCpJ6fr4e75k6IiIjaD4MQEbUEB400bmpqatfOzcgg1GY2Sq1gfZE/GIaxpX+vrd6/V+VyGZcuXcKhQ4fM72itOQv99n0uFAq2srqNPtV1HeFwGOFwuGZf7UbbHD9+fNfOVs8g1GaEIwdNrRf5g3PSV7/t3wvDMBCNRnH27Fn09fWhUqkgk8lgYmLCNRAJS/qQtbW1tv8+f/DBB7b3p06dsr3PZrOYmZlBOp1GOp3Gu+++i5mZmbq2CYVCGBsb25Wz1TMIEW0RwzCqLkZ+2r9XqVQKoVDInLHDmlpkYmIC2Wy26jNytKUfErnt37/f9gPQOoN9qVRCJBLB2NgYgsEggsEgYrEYzp8/bz6T6GUbAOjt7UV3d7eZPmW3YBAicmFN+2FNNSG5NSU5lyUSCbPZRS4vl8tmswwAzMzMmE081tQije4faD59Rz3K5TJGR0ddZ9OQZYxEIq6ByM1m9V5PmpBWpAEplUoIh8MYHx/HyspK1fr3338fAHDgwAFz2WOPPQbgwR2Ul22k/v5+jI6O7qpmOQYhIhfDw8P4wx/+YDYd6bpuayqxZqOVisWi7b11Jgv5K7qrq8vsE1hZWcG5c+dQqVQArD/fJQNRo/vfbrdv3wYAPPHEE67rL168iHg8jkgksulsJcDm9R6NRhGJRMz60zQNxWIRuq7j9ddfN/dTLpcRjUbR3d0NIQQuXLiA5557zlMZrOT2ExMTeOaZZxAOh20BYmlpCQBsz67Juzv5A8HLNpKsR1mvu8K2PRbbYvDxE8K0fRqZMUHOpGGdJWJ5eVkAMHMbCeE9XcVm2wixPjsFANuMEY3uv1GNzJjgzAFlJZdXKhVzdg9rKhXn51pZ761Ml1KpVEQ+nzfPNZlMblgW53Iv21iP5fweeOHn6yGDEO1ojQQht7Qf8uKgaZq5rJVBqNHPqg5CGx3fulxOwWSdAsr5uVbW+1akSxFCiGQyuWlZnMvrCUIbLd+In6+HbI4jcnBL+xEMBgFUN5+QN52dncjn81XNa1atrHdrGhDnqxlnzpyxlcU6SMEpFot53mY3YxAicpAXDbfO4a2+aOzki1IoFEIul4Ou60gkElXrt6LerYM9WkGObJPcyiwHSBw+fNjzNrsZgxCRg5x/6+7du+Yy+ct9q3LtyIul8xmUdieDiddnW+SEwRMTE1XrWlnvW5UGxDAMW1lOnDhRVeZ79+7Z1nnZxsmaQHKnYxAicjh58iQ0TcPk5KT56/XGjRuIxWK2nEryF7EMINYhvPKpeuuvYOcFUA5bNgwD6XQamqbZmm4a3f92DtGWyQadQUjWm9tdzcDAgOtF1ku9W/cnj2k9tlx/+vRpAOuj2uTM8F1dXWYAkUO3Nxotl81mbcO6S6USbt26ZfsO9PT0IJlMYnZ2FoZhwDAMzM7OIplMmqPhvGxjPQYAHDlypGa5dhylPVJNgI874mj7NJpPaG1tTSSTSbOTOJPJ2HItCbGe/0h2gMscTpqmiUwmY3a+y1Fv8Xjc1iEPwMzXhP8bcdWq/XvJIeWmkYEJcsDB8vKyuQweBwNYO/it+9uo3t32W+tYxWLRHNEWi8Vseaji8biIxWKuZZByuZy5z3g8LvL5/KbbapomFhYWGt5GjgZ05u/ajJ+vh0zlQDtaO6ZykA+VtlOZgMZTOcg7MGda9s0YhmEOPFAlHA4jl8spLYPV+Pg4Ojo66q5LP18P2RxHRE2JRqNYWlpynVFgI6oD0MrKCsbGxpSWwapQKKBQKCAajaouyrZiECLaRs4paHaCYDCIVCqFycnJumckUGVxcRH79u0z57tTbXV1FdPT00ilUsqD83ZjECLaRtbU5tb/97vOzk6k02ncvHlTdVE86evrMwdVtANd13H58mVfTOjaag+rLgDRbtJu/UCtFAwG6+7LoHW7ud54J0RERMowCBERkTIMQkREpAyDEBERKePrgQnz8/PYu3ev6mJQG5PJwebn5xWXpP3JKWNYV7SdfDtjwqOPPopPPvlEdTGIiNrC7du3fTnnnG+DEFE78PN0KUTtgH1CRESkDIMQEREpwyBERETKMAgREZEyDEJERKQMgxARESnDIERERMowCBERkTIMQkREpAyDEBERKcMgREREyjAIERGRMgxCRESkDIMQEREpwyBERETKMAgREZEyDEJERKQMgxARESnDIERERMowCBERkTIMQkREpAyDEBERKcMgREREyjAIERGRMgxCRESkDIMQEREpwyBERETKMAgREZEyDEJERKQMgxARESnDIERERMowCBERkTIMQkREpMzDqgtA5Bf5fB6/+MUvqpbruo7f/va35vsnnngC3/3ud7ezaES+FRBCCNWFIPKDH/3oR7h69SoeffTRmtv88Y9/BADwnxWRN2yOI/LoO9/5DoD1QFPr9cgjj+CHP/yh4pIS+QfvhIg8+vOf/4zu7m58/PHHG2733nvv4Zvf/OY2lYrI33gnROTRQw89hKGhITzyyCM1tzlw4AC+8Y1vbGOpiPyNQYioDpFIBJ988onrur179+Kll15CIBDY5lIR+Reb44jq9OUvfxkfffSR67pf/epX+NrXvrbNJSLyL94JEdXp+9//Pvbu3Vu1/Ctf+QoDEFGdGISI6hSJRPDpp5/alu3duxdnz55VVCIi/2JzHFEDQqEQfv3rX5vPAwUCAXz44Yf48pe/rLhkRP7COyGiBpw9exZ79uwBsB6ADh8+zABE1AAGIaIGDAwM4P79+wCAPXv2YHh4WHGJiPyJQYioAQcOHMC3vvUtAOsPsb744ouKS0TkTwxCRA0aGhoCAHz961/H/v37FZeGyKdEk27fvi0A8MUXX3zxtcteP/nJT5oNIaLpVA4ffvghAODNN99sdldEvmMYBj7/+c9XzZJw5swZvPLKK3j22WcVlcwf3nvvPVy9epXXDx8aGhqq+dB2PVqWT6i/v79VuyLaEZ5++mn+u9iEfN6K9eQ/b7/9dkv2wz4hIiJShkGIiIiUYRAiIiJlGISIiEgZBiEiIlKGQYiojY2Pj2N8fFx1MdpWuVzG1NSU6mL40tTUFAzDUF0MBiEiqs0wjLbNFFsul3Hp0iUcOnQIgUAAgUCgZsCW662vdlYoFGxlHRkZqdpG13WEw2GEw2Houu66n422OX78OIaHh1Eul7fkHLxq2XNCRNR6V65cUXr8W7duKT1+LYZhIBqNYmxsDL29vahUKrhx4wYikQiA6noTQqBcLqOrqwtra2vo7OxUUWzPPvjgA9v7U6dO2d5ns1lcv34d6XQaAPDaa6/h448/xrlz5zxvEwqFMDY2hmg0inQ6jWAwuJWnVBPvhIjIlWEYmJmZUV0MV6lUCqFQCL29vQCAYDCIgYEBAMDExASy2WzVZ2TgafcABAD79++HEMJ8aZpmriuVSohEIhgbG0MwGEQwGEQsFsP58+dRKBQ8bwMAvb296O7uRiqV2vZzlBiEiNpUuVxGNptFOBx2fa/rOgKBAMLhMEqlkrmNbIIBgJmZGbM5Z3V11dy3W7OUc1kikTCbcKzLVfdTlctljI6O4tixY67rE4kEIpGIayByYxgGstmseY4zMzO2Jiov9W7ddmpqyly/uLhY9/mVSiWEw2GMj49jZWWlav37778PYH0md+mxxx4D8OAOyss2Un9/P0ZHR9U1yzU7+dzc3JxowW6IdhQAYm5urql9aJpmThTpfL+8vCyEEKJYLAoAIhaLmcd1blOpVEQsFhMAxJ07d4QQQqytrdn2bd2XdZnzvRBCxONxEY/Hmzo3qZHrRy6XEwBEsVisWif3FY/HBQCRz+dd11tpmiaSyaQQYr1eNE0TmqaJSqVirt+s3q2fzWQyQgghFhYWXMvg9fzkS9M0sba2Zq6Xf0u3c9c0zfM2kjyXXC5XVzkHBwfF4OBgXZ9xwyBEtAVaEYTkfjYLCl62yefzAoBIJBJN76uVGrl+yADjRi6vVCpm8JCB17pekoHCepFfXl4WAMxgIj+3WV1lMhnXbRoJ2JVKReTzefNcZZCsVRbnci/bWI/l/G540aogxOY4ol0gFAoBAEZHRxWXpHkTExObbhMMBs1+jo2amubn5wHY+4meeuopAMD169frKpfc3tms6aW8TsFgEKFQCFeuXEEymaw5+q0V5IAEVd8NBiEi2pE6OzuRz+eh6zqi0ajrMzHT09NVy+RFud4Lv9xeWAYUyFczzpw5YyuLdZCCUywW87xNu2AQItpF2u0CtNVCoRByuRx0XUcikahaLy/WbndKjdaVdQBIK8iRbZJbmeUAicOHD3vepl0wCBHtAvLC6HzexI9kMPH6tL+machkMq7NYoODgwCAu3fvmsvkfuvNcZRMJgEA6XTa3EcrZnQwDMNWlhMnTlSV+d69e7Z1XrZxisfjTZWzUQxCRG3KOUzY+l5e5KwXYueveTlE2TAMpNNpaJpma6aRv65lgLIOB5ZP6Ft/UcuLqeoh2gcPHgRQHYTk+bvd1QwMDLheZE+ePAlN0zA5OWl+7saNG4jFYujr66va30b1fvr0aQDrfUAdHR0IBALo6uoyA4gcum19Tscpm83ahnWXSiXcunXLLAsA9PT0IJlMYnZ2FoZhwDAMzM7OIplMoqenx/M21mMAwJEjR2qWa0s1O7KBo+OIqqEFo+NgGabr9nLbxrosn8+bI8SSyaQ55FgqFovmejk8Vw4xlqPF5Ki6eDxuLlM9RFsOL5fDpYVwrys3zuHJcn/JZNL8XCaTsdWV13oXYr1O5Yi2WCxmG0Yej8dFLBZzLYNkHZ4dj8c3HN4tt9U0TSwsLDS8jRwNaB0h6EWrRscFhGiu1+z69esYGhpquvONaCcJBAKYm5szm3u2+9gAfPFvstHrh7wru3jxYl2fMwxD2fQ0UjgcRi6XU1oGq/HxcXR0dNRdl0NDQwCAubm5po7P5jgi8p1oNIqlpSXXGQU2ojoAraysYGxsTGkZrAqFAgqFAqLRqLIy7KggtF1t1arbxP2Kf5+t5+xH2qnkc0CTk5Mb9rG0k8XFRezbt8+c70611dVVTE9PI5VKKQ3Ovg1C2zXFfLtOZV8qlTAyMmLOC9bIHFW1yP02Y7f/fVTp6upy/f+dqLOzE+l0Gjdv3lRdFE/6+vrMQRXtQNd1XL58Wf2Ers12KqkamCA73HbKcepRqVTMjuRKpWJOF1Lv3E9urPOH1TvnldVu/vsI0bppe3Y6Dmzyr109bc92TTHfrlPZ37p1yxw6a53CXs7y24z5+Xmz09Q5265Xu/3vQ0TeKQlC8uJhzYbobL92m15dcpti3jrd+srKyv9v735D20jvPIB/1Wy6vZZdmbRYm+Tw9sWyYaEgkuulLtdrSTZccNhR9mCdRs66faME5aDLLvaLRScTQoy3L2RY9s0a2W+McGTWhW0lWL+JTRNKrCxdsKBv4hcpUntpNHCguYXe0SV97oX3mR2NRtJIHumZsb8fEEQzo5lHo3h+8+d5fr+WVRRlX/1QKIRqtdq2LZ2206m9vaaD76RVSg77CO9un40YhoF6vW6u/9q1a22X5e9DRHu210upXi6nZZrxWq3mmBJdiN3+/NaxCMlksuE9bP3z7WnvZXZcp/EM1v73ndrSaTvW6V6kg++FzIJrvx3X7XiOfD5v7hc5bqLVLTn+Pu2Bt+Nc4e244Ap0KQc5aMtsRIuU6Pb06tZBXk4HGvs0OWjMOvCsXq83HPg6tcXNdrxMB9+LjY2NhgNqL2TNGUkOUrSmkJf4+3TGIOQOg1BweRWEnunp8mmPZP33arVqplK3kinRrb02RkdHux7g9cYbb2B2dhbr6+vmc5PPPvsMb7zxhuu2uNEpHbzcdr+8//77ZhnfXn322WcN+alk6v9isdhQtx7g7+PWgwcPcPjw4b6se7948OABAPT825I61Wq1KQVQT/YaxXo9k8lms0LTNPHw4UNXZ7d2Tss4TZO3XSSn2z/dtsVte3tZV7fy+bzj1Uq3rLej7C9rUTAh+Pu40Wpf8sXXfnoF9nacvJ0j8yrJLyTJA2K7LsL2z7SaJre1tbUlKpVK03OTTm1xsx3ZXnvuJaD984tW09ySlRf3amtrq+G2lHX9AJrm8ffpDODtODd4Oy64At1FOx6PA0DLSznZO2thYcHMVisHZ3ZLZp9dXl7G/fv38eMf/7irtrjhZTp4t3Rdx507d8zbVcBuCo5e9tHy8jLGxsaapkejUWia1lRhkr8PEXlmr1GslzMZeWZaqVQabrHIM1XZe0lOx5dnrNbbQtaz20wmY2bWhcMZr3wA7lRDvVNb3GxH1rPXNM2cls/nG86yrZ+TD+JlrzanNrfjtH/ky3ol4aZ3XD6fb7uM3HfWqyH+Pp2BV0Ku8EoouALdO86eHl72gLKmPZfT5XL25xL2ddgPxk7L2tfhpi1ut+NlOvhOZLdlp5f1O3YKQvbPWve/03zrMvx92mMQcodBKLhYyoHIx1SWcggSHj+Ci6UciIgo8BiEiIhIGQYhH7HnU2v1IqJduq6bVVapO/Pz82YvUZUYhHxE7HYU6fgiaqffNZb8UsNJ13XcuHEDJ0+ebEhw6yRoJ3PlcrmhrU7DH4rFImKxGGKxmJnIt5tlzp07h8nJSeXFDxmEiPaZe/fuBXr9bhiGgUQigZ///Oc4e/Ys6vU68vk8ZmdnHQOREAK1Wg0AUKvVfH8yZy+jcuHChYb3q6urWFxcRC6XQy6XwyeffNJU1qTTMtFoFKlUColEQukVEYMQ0T7S7xpLfqnhtLS0hGg0apbKttbVmp2dxerqatNnZO5A5ZVEXXjhhRca7n5Yy7dUq1XE43EzX2Q4HEYymcS1a9fMUudulgF2cz4eP34cS0tLA/+OEoMQkU90qnnkdCvJPq1VjSV5WwaAWZ/p+vXr2NnZ2fP6ge5rV+2FruuYnp7GmTNnHOdnMhnE43HHQOTEy1pT8hmVnL+5udn196tWq4jFYpiZmUGpVGqaf//+fQDAsWPHzGlHjx4F8NUVlJtlpPHxcUxPTyu7LccgROQTk5OT+Pzzz81bR8ViseFWibydZFWpVBreW9M4ybPoSCRiPhMolUq4evUq6vU6AODEiRNmIOp1/YMmM2+/9NJLjvOnpqaQTqcRj8cbzvpb6bTfE4kE4vG4uf80TUOlUkGxWMR7771nrkfXdSQSCRw/fhxCCLz99tt49dVXXbXBSi4/OzuLH/7wh4jFYg0B4u7duwAaU1nJqzt5guBmGUnuR7lfB26vo1054pmoGbrMmOBlzSM3ywjxVbYJa7qkXtffq17rkbX6jJwuUzXBlonD/jkv97tMtmtfppckw/V63UxQDDTW9Wq1/63T3Sxj3Zb9/4EbgU7bQ7TfdRuEZComK3lw6LZYoNsg1OtnVQehdtu3TpdpnKw5A+2f83K/tyuHsheylEm7ttindxOE2k1vJ9BZtImo0cLCQtM0WaSwVfdbam94eBjb29tNt9esvNzvcnnh8bCKS5cuNbTF2knBLplMul7GLxiEiHxAHjScHg73+6Dht4OSl6LRKAqFAorFIjKZTNP8fux3a2cPL8iebZJTm2UHiVOnTrlexi8YhIh8QEXNI3mwtI9B8TsZTNyObdE0zRxDZOflfs9mswCAXC5nrsOLjA6GYTS05fz5801tfvz4ccM8N8vYpdPpPbWzVwxCRD4wNjYGTdMwNzdnnr2ur68jmUyahf+Ar87OZQCxduGVo+qtZ8H2A6DstmwYBnK5HDRNa7h10+v6B9lF++WXXwbQHITkfnO6qrl8+bLjQdbNfreuT27Tum05/+LFiwB2e7UNDQ0hFAohEomYAUR23W7XW251dbWhW3e1WsW9e/ca/g+MjIwgm81ieXkZhmHAMAwsLy8jm82aveHcLGPdBgCcPn26Zbv6aq8PldgxgagZeqgn1KnmkRBCVCoV8wG4LGCoaZrI5/Pmw3d7jSXZHmC3JLv8fDab9Wz9bgooOunl+CE7HGxtbZnT4LIzgPUBv3V9XtWaqlQqZo82e400WQvLqQ1SoVAw15lOp8X29nbHZTVNExsbGz0vI3sDdlu4kfWEiHzMb/WE5KBSv/2d9nr8kFdgU1NTXX3OMAyz44EqsVgMhUJBaRusZmZmMDQ01PW+ZD0hIjqwEokE7t6965hRoB3VAahUKiGVSiltg1W5XEa5XEYikVDWBgYhon3OnoJmPwiHw1haWsLc3FzXGQlU2dzcxJEjR8x8d6rt7OxgYWEBS0tLSoMzgxDRPheJRBz/HXTDw8PI5XK4c+eO6qa4cvbsWbNThR8Ui0XcvHlTeULXZ5RunYj6zm/PgbwUDoe7fpZBu/yy33glREREyjAIERGRMgxCRESkDIMQEREp41nHhEuXLnm1KqJ94YMPPsCvf/1r1c3wNZkyhseP4FlbW/NkMPaeMyY8efIE77zzDp4+fbrnxhAFzZ07d/C9730PL7zwguqmEA3c5ORk27IRbuw5CBEdZH5Lz0MUNHwmREREyjAIERGRMgxCRESkDIMQEREpwyBERETKMAgREZEyDEJERKQMgxARESnDIERERMowCBERkTIMQkREpAyDEBERKcMgREREyjAIERGRMgxCRESkDIMQEREpwyBERETKMAgREZEyDEJERKQMgxARESnDIERERMowCBERkTIMQkREpAyDEBERKcMgREREyjAIERGRMgxCRESkDIMQEREpwyBERETKMAgREZEyDEJERKQMgxARESnDIERERMqEhBBCdSOIgmBpaQn/8R//gRMnTpjT/vSnP+Hb3/42vvnNbwIA/vKXv+Bf/uVf8Jvf/EZVM4kC5RnVDSAKilqthi+++AJ/+MMfGqYbhtHwvlgsDrJZRIHG23FELsXjcYRCobbLPPPMM/jlL385oBYRBR9vxxF14Z//+Z/x2WefodWfTSgUwh//+Ee8+OKLA24ZUTDxSoioC2+++SYOHTrkOO9rX/saTp8+zQBE1AUGIaIu/PSnP8Xf//53x3mhUAg///nPB9wiomBjECLqwgsvvICf/OQnLa+GxsfHB9wiomBjECLq0s9+9rOmZ0KHDh3CmTNn8J3vfEdRq4iCiUGIqEv//u//3nQlJITAz372M0UtIgouBiGiLoXDYYyNjeGZZ74aZnf48GG8/vrrCltFFEwMQkQ9mJycxNOnTwHsjg167bXX8NxzzyluFVHwMAgR9eC1117DP/zDPwAAnj59iitXrihuEVEwMQgR9eAb3/gG3njjDQDAt771LVy4cEFxi4iCKbC547a2tvDnP/9ZdTPoAPvHf/xHAMCLL76IQqGguDV0kB06dAixWKzhOWVQBDZtT6ccXkREB8nHH38cyM4xwQubFisrK5iYmFDdDPKx27dv48qVKy1zvdFX5HOtlZUVxS2hboVCIfz1r39V3Yye8JkQEREpwyBERETKMAgREZEyDEJERKQMgxARESnDIERERMowCBG5NDMzg5mZGdXN8C1d1zE/P6+6GYE0Pz8PwzBUN0MJBiGigDAMw7eDtHVdx40bN3Dy5EmEQiGEQqGWAVvOt778rFwuN7T1+vXrTcsUi0XEYjHEYjEUi0XH9bRb5ty5c5icnISu6335Dn4W6MGqRIN069Ytpdu/d++e0u23YhgGEokEUqkURkdHUa/Xsb6+jng8DqB5vwkhoOs6IpEIarUahoeHVTTbtU8//bThvT1P4OrqKm7fvo1cLgcAePfdd/HkyRNcvXrV9TLRaBSpVAqJRAK5XA7hcLifX8lXeCVEFACGYWBxcVF1MxwtLS0hGo1idHQUwG69pcuXLwMAZmdnsbq62vQZGXj8HoCA3ZLuQgjzpWmaOa9arSIejyOVSiEcDiMcDiOZTOLatWsol8uulwGA0dFRHD9+HEtLSwP/jioxCBG5oOs6VldXEYvFHN8Xi0WEQiHEYjFUq1VzGXkLBgAWFxfN2zk7Ozvmup1uS9mnZTIZ8xaOdbrq51S6rmN6ehpnzpxxnJ/JZBCPxx0DkRPDMLC6ump+x8XFxYZbVG72u3XZ+fl5c/7m5mbX369arSIWi2FmZgalUqlp/v379wEAx44dM6cdPXoUwFdXUG6WkcbHxzE9PX2wbsuJgAIgVlZWVDeDfG5lZUV48d9c0zQBwFyX9f3W1pYQQohKpSIAiGQyKYQQ5nzrMvV6XSSTSQFAPHz4UAghRK1Wa1i3dV3Wafb3QgiRTqdFOp3e8/cTQoiJiQkxMTHR1WcKhYIAICqVStM82dZ0Oi0AiO3tbcf5VpqmiWw2K4TY3S+apglN00S9Xjfnd9rv1s/m83khhBAbGxuObXD7/eRL0zRRq9XM+fK3dPrumqa5XkaS36VQKHTVziAfDxmEaF/zKggJ0RwEnIKCm2W2t7cFAJHJZPa8Li/1EoQDjtTsAAAeIUlEQVRkgHEip9frdTN4yMBrnS/JQGE9yG9tbQkAZjCRn+u0r/L5vOMyvQTser0utre3ze8qg2Srttinu1nGui37/w03gnw85O04ogGLRqMAgOnpacUt2bvZ2dmOy4TDYfM5R7tbTWtrawAanxO98sorAHazoXdDLm+/remmvXbhcBjRaBS3bt1CNptt2fvNC7JDwn74v+EWgxAR9d3w8DC2t7dRLBaRSCQcx8QsLCw0TZMH5W4P/HJ5YelQIF97cenSpYa2WDsp2CWTSdfLHGQMQkSKHLQDUDQaRaFQQLFYRCaTaZovD9ZOV0q97itrBxAvyJ5tklObZQeJU6dOuV7mIGMQIhoweWC0jzcJIhlM3I721zQN+Xze8baYLFD56NEjc5pc7/j4eFftymazAIBcLmeuw4uMDoZhNLTl/PnzTW1+/Phxwzw3y9il0+k9tTNIGISIXLB3E7a+lwc564HYfjYvuygbhoFcLgdN0xpu08izaxmgrN2B5Qh96xm1PJiq7qL98ssvA2gOQvL7O13VXL582fEgOzY2Bk3TMDc3Z35ufX0dyWQSZ8+ebVpfu/1+8eJFALvPgIaGhhAKhRCJRMwAIrtuW8fp2K2urjZ0665Wq7h3757ZFgAYGRlBNpvF8vIyDMOAYRhYXl5GNpvFyMiI62Ws2wCA06dPt2zXvqO0W8QeIMC9QWhwvOodB0s3XaeX0zLWadvb22YPsWw2a3Y5liqVijlfds+VXYxlbzHZqy6dTpvTVHfRlt3LZXdpIZz3lRN792S5vmw2a34un8837Cu3+12I3X0qe7Qlk8mGbuTpdFokk0nHNkjW7tnpdLpt9265rKZpYmNjo+dlZG9Aaw9BN4J8PAwJsccndYqEQiGsrKyYl/BETm7fvo0rV67s+YF0r2SvrCD8mV25cgUAsLKy0tXn5FXZ1NRUV58zDEN5eppYLIZCoaC0DVYzMzMYGhrqel8G+XjI23FEtCeJRAJ37951zCjQjuoAVCqVkEqllLbBqlwuo1wuI5FIqG7KQB3oIGRPAULkJftzpP1KjgOam5tr+4zFTzY3N3HkyBEz351qOzs7WFhYwNLSkvLgPGgHOgjduHED8Xi8r4PP+knXdczMzJiD8dzm57JySqsvX/Pz8ygWiwe2zsleRSIRx3/vR8PDw8jlcrhz547qprhy9uxZs1OFHxSLRdy8eTMQCV29dqCD0Icffqi6CT3TdR2PHj3CrVu3IIRAPp9HPB7vuguqEAK1Ws18X6/XzUF9586dw+Li4oGtc7JXwsNBkkEQDoe7fpZBu6ampg5kAAIOeBAKskePHjXcSpCp83tJ92H9z2+9FRCNRs10K61GuRMR7cWBCkLWNPGxWKzlaOpWKeC7SSMvPy9T0durR+41zbz9XrYMEPbxF3sdRzI8PIy3334bxWKxqahaEPYTEfmcmp7he4ce+sVrmiaSyaQ57kBm2rXuhnYp4N2mkc9kMuaYhHq93pRp2Ks085J1PIQ1S7EQ7seR2PeDlczs6zZVvp/2k5dZtPe7XsYJkT/0cjz0i8D+dXa70+VAMetBWh5crQepTingnQ7W9mmwDTaTA/rcbqMb1roz6CEFvHX77Q7WQd1PDELuMQgFV5CD0DNeXlX52SeffAIADT1inLpCWlPAW83OzuLWrVuutpVMJhGJRJDP5zE2Nobh4eGGB9NebEMaGRmBEALlchm/+tWvMD09jeeff76hvn0/BG0/Xbp0qavlD6IHDx4A4L6iwTowz4Sc0sQ78SIF/DvvvANN0xCPxzE0NNTUY60faeaj0SgmJycBANeuXet5PU6cnjcFdT8Rkb8cmCuhbu3s7PQ8juDll19GoVBAuVzGwsKC2WPN3n11L9totd1++OyzzwAAZ86caZoXlP300Ucf7enzB0GvaXtIPfvdgiA5MFdCMrV7pxHdXqSAD4VCMAwD0WgUH374Iba3txu6TvczzTwA5PP5Pa3HStd1vP/++9A0rSF7cJD3ExH5yACfP3kKXT6Ikw/wNU0ze2TJ3law9NqSD8ftr0ql0jBP9rCzdm6QD9nx5cNzuZ1KpdLQYaDdNtzSNM2xd5n9ob2b3nHW72DNWCx7umma1pTVNyj7iR0T3GPHhODq9njoJ4H96+xlp1cqFZFMJs2gY+0CbD3ItkoBbz8YtptWq9VEJpNp2WOtXZp5N6xp5uU2rOn0pU5ByOkg32mdQdpPDELuMQgFV5CDEEs50L6mupRDkPCZUHAF+Xh4YJ4JERGR/zAIEVFfsBOJs/n5eeZhtGAQ8pl2pRWsLwoGwzD6+nv1e/290nUdN27cwMmTJ83/s61yGAbp/7dhGCiVSlhcXGxbh6xYLCIWiyEWizWVijl37hwz01twnJDP8NnF/mJP+hq09ffCMAwkEgmkUimMjo6iXq9jfX0d8XgcAJqyXQghoOs6IpEIarWar0saZDIZALtZO1pZXV3F7du3kcvlAADvvvsunjx5YmYxiUajSKVSSCQSyOVyB66InR2vhIj6xDAMLC4uBnb9vVpaWkI0GjUzvYfDYbPUyOzsrGPxRRl4/ByAgN0A2i5lVLVaRTweRyqVQjgcRjgcRjKZxLVr1xrGKI6OjuL48eNmqZSDjEGIyIG17Ie11ITkdOvIPi2TyZi3YuR0XdfNWzUAsLi4iFAohOvXrzeUFul1/cDey3fsha7rmJ6edsyuAey2OR6Pu64C3Ol36KZsyCDKgty/fx8AcOzYMXPa0aNHAQCffvppw7Lj4+OYnp4+8LflGISIHExOTuLzzz83K88Wi8WGwn7WarRSpVJpeG89YxZf5ryLRCLmc4JSqYSrV6+iXq8DAE6cOGEGol7Xr5pMgvrSSy85zp+amkI6nUY8Hu+YvQTo/DskEgnE43Fzf2qahkqlgmKxiPfee89cj67rSCQSOH78OIQQePvtt/Hqq6+6akM37t69C2A3sbAkr+7sz4bkPpL77MBSMzxp7xDgwVk0OL0MVpWZNKwDmLe2tgQAs7aREO7LVXRaRojd7BSwDdjtdf298mKwqr0mlJWcXq/XzZpT1tIq9s95+Tt4WT6l1Ta7nS6ziPRafsW+/qAeD3klRGSztrYGoPH5xCuvvALgq/ISXotGowB6K8/uJ+0e2EvhcNh8FtLudpSXv4O1LIj11qWb9vaL7JAQ9N98rxiEiGycyn7IA4b9lgr1Znh4GNvb202316y8/B0GVRZE07SW85LJpKfb2i8YhIhs5IHE6Qy93weSg3SgikajKBQKKBaLZtdnq378DtbOH/3g1GbZQeLUqVN93XZQMQgR2cj8W48ePTKnyTP18fHxvmxTHhwvXLjQl/UPigwmbjMCaJqGfD7veFvMy99hUGVBzp8/D6CxzY8fP26YZ2ctFnkQMQgR2YyNjUHTNMzNzZlntOvr60gmkw01leTZuAwgpVLJnHf9+nUAjWfG9gOe7KZsGAZyuRw0TWu4ndPr+lV20ZbFB+1BSO5Hp6uay5cvOx6I3fwO1vXJbVq3LedfvHgRwO4zoKGhIYRCIUQiETOYya7bbnrLWddv/54jIyPIZrNYXl6GYRgwDAPLy8vIZrMNPeaAr66QTp8+3XGb+5rSbhF7gAD3BqHB6bWUQ61WE9ls1uzVlM/nG2otCbFbZkL28ioUCkII0VQaRPZ6S6fTDXWUAJj1mgCIbDbr2frd1JBy4kXvOFkDyloCRH5f68uJpmmO62v3Ozitt9W22pUFSafTIplMOrbByum7OH0fWWpF0zSxsbHhuC7Z089eq6sXQT4espQD7Wt+LOUge2b5qU2Ad6Uc5BWZvUx7J4ZhKE9hE4vFUCgUBrKtmZkZDA0Ndb2fnAT5eMjbcUTkqUQigbt37zbcPnRDdQAqlUpIpVID2Va5XEa5XEYikRjI9vyMQYhogOwpZ/YjOQ5obm7O84wE/bK5uYkjR46Y+e76aWdnBwsLC1haWlIeeP2AQYhogCKRiOO/95vh4WHkcjncuXNHdVNcOXv2rNmpot+KxSJu3rzp+2Stg8JSDkQD5LfnQP0UDoc9ed6x33CfNOKVEBERKcMgREREyjAIERGRMgxCRESkDIMQEREpE+iMCUREtOvjjz/G66+/rroZXQtsF+379+/jz3/+s+pm0AF36dIlvPXWW/jRj36kuil0gB06dAivvfaa6mb0JLBXQkR+EOScXUR+wGdCRESkDIMQEREpwyBERETKMAgREZEyDEJERKQMgxARESnDIERERMowCBERkTIMQkREpAyDEBERKcMgREREyjAIERGRMgxCRESkDIMQEREpwyBERETKMAgREZEyDEJERKQMgxARESnDIERERMowCBERkTIMQkREpAyDEBERKcMgREREyjAIERGRMgxCRESkDIMQEREpwyBERETKMAgREZEyDEJERKQMgxARESnDIERERMowCBERkTLPqG4AUVD87//+L/7yl780Tdd1HY8ePTLfh8NhfPvb3x5k04gCKySEEKobQRQE77zzDt5//31Xy/LPisgdXgkRuXTq1KmOy4RCIfzwhz8cQGuI9gc+EyJy6fXXX8ezzz7bcblf/OIXA2gN0f7AIETk0nPPPQdN0/DMM61vIDz77LPQNG2ArSIKNgYhoi5MTEzg6dOnjvMOHz6M119/Hd/61rcG3Cqi4GIQIurChQsXWgaZL774Am+++eaAW0QUbAxCRF149tlncenSJRw+fLhp3vPPP49/+7d/U9AqouBiECLq0pUrV/DFF180TDt8+DB++tOfOgYnImqN44SIuvT06VNEIhH893//d8P03/72t/jJT36iqFVEwcQrIaIuHTp0CG+++Sa+/vWvm9NeeOEF/Ou//qvCVhEFE4MQUQ8mJibwt7/9DQDw9a9/HRMTE/ja1/jnRNQt3o4j6tGLL76IarUKAPj973+Pf/qnf1LcIqLg4akbUY8mJycBAN/97ncZgIh65HnuuCdPnuCdd95pOaCPaL/4n//5HwDA//3f/+HSpUuKW0PUf5OTk55nBPH8SmhzcxOrq6ter5bId55//nl8//vfxw9+8IOOy66trZm37qi1arWKtbU11c0gB2tra305tvcti/ZHH33Ur1UTBU4oFMJbb72FiYkJ1U3xtdu3b+PKlSs8fvjQlStX+rJePhMiIiJlGISIiEgZBiEiIlKGQYiIiJRhECIiImUYhIgCZGZmBjMzM6qb4Vu6rmN+fl51M3xnfn4ehmGoboYjBiEics0wDIRCIdXNcKTrOm7cuIGTJ08iFAohFAq1DNhyvvXlV4ZhoFQqYXFxEbFYrOVyxWIRsVgMsVgMxWKxYd65c+cwOTkJXdf73dyu9W2cEBF579atW0q3f+/ePaXbb8UwDCQSCaRSKYyOjqJer2N9fR3xeBxA834TQkDXdUQiEdRqNQwPD6totiuZTAYAMDs723KZ1dVV3L59G7lcDgDw7rvv4smTJ7h69SoAIBqNIpVKIZFIIJfLIRwO97/hLvFKiIhcMQwDi4uLqpvhaGlpCdFoFKOjowCAcDiMy5cvA9g9eDuN9JeBx88BCNgNoO1OPqrVKuLxOFKpFMLhMMLhMJLJJK5du4ZyuWwuNzo6iuPHj2NpaWkQzXaNQYgoIHRdx+rqqnlLxv6+WCwiFAohFouZKYJ0XTdv0wDA4uIiQqEQrl+/jp2dHXPdTrel7NMymYx5m8c6XfVzKl3XMT09jTNnzjjOz2QyiMfjrlPOGIaB1dVV8zsuLi423MZys9+ty87Pz5vzNzc3e/yWrd2/fx8AcOzYMXPa0aNHAQCffvppw7Lj4+OYnp7212054bGVlRXRh9USBRoAsbKysqd1aJomAJh/X9b3W1tbQgghKpWKACCSyaS5Xfsy9XpdJJNJAUA8fPhQCCFErVZrWLd1XdZp9vdCCJFOp0U6nd7Td5N6OX4UCgUBQFQqlaZ5cl3pdFoAENvb247zrTRNE9lsVgixu180TROapol6vW7O77TfrZ/N5/NCCCE2NjYc2+CW074XQpi/pdPymqY1TJPtLBQKXW9/YmJCTExMdP25ThiEiAbAiyAk19MpKLhZZnt7WwAQmUxmz+vyUi/HDxlgnMjp9XrdDB4y8FrnSzJQ1Go1c9rW1pYAYAYT+blO+yqfzzsu02vAbrXvu5ler9ebfne3+hWEeDuO6ACKRqMAgOnpacUt2bt2D+ylcDhsPgtpdztKZvC2Pid65ZVXAOwmV+2GXN5+W9NNe/tFdkjw0+/OIEREB8Lw8DC2t7dRLBaRSCQcx80sLCw0TZMHbnu3507k8mL3jlPDy0vt6vskk0lPt9UPDEJEB1gQDlJeikajKBQKKBaLZtdnK3lAd7pS6nVfWTuA9INTm2UHiVOnTvV1215gECI6gOSB8cKFC4pbsncymLjNCKBpGvL5vONtMVnv6dGjR+Y0ud7x8fGu2pXNZgEAuVzOXEc/MjqcP38eQGObHz9+3DDPLp1Oe9qGvWAQIgoIezdh63t5kLMeiO1n87KLsmEYyOVy0DSt4VaOPNOXAapUKpnzrl+/DqDxrFseTFV30X755ZcBNAch+f2drmouX77seCAeGxuDpmmYm5szP7e+vo5kMomzZ882ra/dfr948SKA3WdAQ0NDCIVCiEQiZjCTXbetY3lasa7f/j1HRkaQzWaxvLwMwzBgGAaWl5eRzWYxMjLSsKy8Qjp9+nTHbQ6M1z0d2DuOqBk86B0HS3drp5fTMtZp29vbZg+xbDZrdjmWKpWKOV924ZVdjGVvMdmrLp1Om9NUd9GW3ctld2khnPeVE3sXZrm+bDZrfi6fzzfsK7f7XYjdfSp77yWTyYZu5Ol0WiSTScc2WLX7va1kV3VN08TGxobjumRPP2vvP7f61TsuJIS3T8lkeV6PV0sUaKFQCCsrK0rKe8teWUH4m+z1+CGvyqamprr6nGEYylPYxGIxFAqFgWxrZmYGQ0NDXe8n4Kvy3isrK562ibfjiCjwEokE7t6923AL0Q3VAahUKiGVSg1kW+VyGeVyGYlEYiDbc4tBiGgfsz9H2q/kOKC5uTlXz1j8YHNzE0eOHDHz3fXTzs4OFhYWsLS0pDzw2u3rIDSoB6aqH8wGFX+f/otEIo7/3o+Gh4eRy+Vw584d1U1x5ezZs2anin4rFou4efOmL5O17psgNKg6J36tp6LrOmZmZsyR2W6TNbpx/fr1PX/ng/77qCL6OEjSj8LhcE/PO/a7qakpXwYgYB/VE3Kqc9KP2iuD2k43dF3Ho0ePzJTvq6uriMfj+K//+q89/0FWq1VzFHm5XDbTvXTrIP8+RNTavrgSGlSdE7/WU3n06FHDfWVZR8WL/FBra2tmzx17Wni3DvrvQ0St+SIIyYOHtSSv/SGqU40PyanOibXmR6lUalnKVw4YC4VCqFarbdvSaTud2ttrTZJO7A825WA2+2C8bp+NGIaBer1uDlC8du1a22X5+xBR17weeNTLYDNZD6NWqznW5RBid1CZdUBcMplseA/bAC577RWZot1pUF06nTZrfHRqS6ftWKd7UZOkW9bBcdaU9fJ7djOoMJ/Pm/tFDt5rVQuFv0978KiUw37Hwe7+ta/rCcmRw2ajWtTlsNf4sI40djrQ2KfJg7N19HO9Xm848HVqi5vteFmTpBvWImTosWaIJAufSXKkvDxwW/H36YxByB0GIf/a10FIqlQqIpPJtDybbcfNAUMeSK0Hmo2NDcez+1ZtcbMdp0qHsphUtwfmXmxvb5sHdKeg4cbGxkZT6g97+yX+Pp1ZTw744iuor32dtmdxcdFMr37ixAkAMNfhJu2I0zJO0+Q9fvmwfWZmpqn3VLdtsU9r1V43y3mVYmVnZ6ep7d2IxWIt66c8fPiwYXwDf5/OQqEQ3nrrLfzoRz/q6nMHze9+9zt88MEH+Oijj1Q3hWw++OADjIyMeJ62xxdXQvJ2jkzuhy+jriTPtNvVZrd/ptU0ua2trS1RqVSaaq13aoub7cj22pMEAu2fX7Sa1qte17W1tdVwNSI5XakIwd/HDYC349zg7Tj/2tflvePxOAA0pR2XZO+shYUFs+dXtVo108t3Q6ZjX15exv379/HjH/+4q7a44WVNkl7J7eXz+a4/u7y8jLGxsabp0WgUmqY1lTnm70NEPfM6qvVyJiPPTCuVinj48KF5tinPVGXvJTkdX56xWnt/Wc9uM5mMmd4dDme88nmJ04P7Tm1xs516vW72tpLT8vl8w1m29XPyQbx8LuHU5k77L5PJmFcH8mG+vaeZm95x+Xy+7TJy31mvhvj7dAZeCbnCKyH/2tcdE+w1SmQPKGvtDTldLmfvfmxfh/WAaG+PXNa+DjdtcbsdL2uSdCLriMhXJpNpqK0idQpC9u1b97/TfOsy/H3aYxByh0HIv1hPiCjAVNYTChIeP/yL9YSIiGjfYRAion1D13Wzyip9ZX5+3ux84zcMQj5mz6fW6kXUTr/LW/ilfIau67hx4wZOnjzZkFvQSZD+jgzDQKlUwuLiYlMORKtisYhYLOY4xu/cuXOYnJz0ZWHDfVPKYT/ifXHyglN5iyCt3w3DMJBIJJBKpTA6Oop6vY719XWzS799wLMQArquIxKJoFar+bbWDrCbmBcAZmdnWy6zurqK27dvI5fLAQDeffddPHnyBFevXgWwO7wilUohkUggl8v5qroqr4SI9rF+l7fwS/mMpaUlRKNRM6N8OBw2S5rMzs46FnmUgcfPAQiAWSeslWq1ing8jlQqhXA4jHA4jGQyiWvXrjWUOh8dHcXx48extLQ0iGa7xiBE5FOdyk043UqyT2tV3kLeugFglsa4fv06dnZ29rx+YLAl1XVdx/T0NM6cOeM4P5PJIB6Pu6427GWZD/mMSs7f3Nzs8Vu2dv/+fQDAsWPHzGlHjx4F0FwDbHx8HNPT0766LccgRORTk5OT+PzzzyGEQK1WQ7FYRCKRMB8w12q1ps9UKpWG99YzaPFlie9IJGI+NyiVSrh69Srq9ToA4MSJE2Yg6nX9g/bgwQMAwEsvveQ4f2pqCul0GvF4vOHKoJVO+z2RSCAej5v7T9M0VCoVFItFvPfee+Z6dF1HIpHA8ePHIYTA22+/jVdffdVVG7px9+5dAI1ZROTVnf3ZkNxHcp/5gtcDjzjYjKgZuhys6mW5CTfLCPHVQF9rpope19+rXkvBtPqMnC6zZMA2CNr+OS/3u8xzaF+mm5pe7dbfy3SZ9aOXMi/7OnccETVaW1sD0Pi84pVXXgGAptx9XolGowC8KQs/SO0e2EvhcNh8FtLudpSX+10ub7+F6aa9/SI7JPjpN2YQIvKhhYWFpmnyANKqxAa1Nzw8jO3t7abba1Ze7ne5vPjyNqX15SWZQNhJMpn0dFv9wCBE5EPywOJ0xt7vA0sQDly9ikajKBQKZj0qu37sd2tnj35warPsIHHq1Km+btsLDEJEPqSi3IQ8WF64cKEv6+8XGUzcZgTQNA35fN7xtpiX+z2bzQIAcrmcuY5+ZHQ4f/48gMY2P378uGGeXTqd9rQNe8EgRORDY2Nj0DQNc3Nz5hnu+vo6ksmkWXMJ+OrsXAaQUqlkzpP1nKxnyvYDoOy2bBgGcrkcNE1ruL3T6/oH2UVbVvm1ByG535yuai5fvux4IHaz363rk9u0blvOv3jxIoDdZ0BDQ0MIhUKIRCJmMJNdt930lrOu3/49R0ZGkM1msby8DMMwYBgGlpeXkc1mm+puySuk06dPd9zmwHjd04G944iaoYdSDp3KTQghRKVSMXt9ySq0mqaJfD5v9vCyl7eQ7QF2q+HKz2ezWc/W76Z2lZNejh+yZIe1fIn8ftaXE03THNfnVZmPSqVi9t6zl6eRZUic2mDl9F2cvo8s6aJpmtjY2HBcl+zp1209LCFYyoEo0PxWykH21PLb32mvxw95BTY1NdXV5wzDUJ7CJhaLoVAoDGRbMzMzGBoa6no/ASzlQETUUiKRwN27dxtuF7qhOgCVSiWkUqmBbKtcLqNcLiORSAxke24xCBEdMPYUNPuBHAc0NzfneUaCftnc3MSRI0fMfHf9tLOzg4WFBSwtLSkPvHYMQkQHTCQScfx30A0PDyOXy+HOnTuqm+LK2bNnzU4V/VYsFnHz5k1fJmtlKQeiA8Zvz4G8FA6He3resd/5eZ/wSoiIiJRhECIiImUYhIiISBkGISIiUqZvHRNkSnQi2vXgwQMcPnxYdTN8TRZb4/HDf9bW1vqTt9DrFAwPHjxomWaCL7744ouv4L7+8z//0+uQ4X3aHiIiIrf4TIiIiJRhECIiImUYhIiISBkGISIiUub/AZAdLKO+8N89AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pydot\n",
    "plot_model(model, show_shapes=True, to_file='model01.png')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**6.訓練神經網路**\n",
    "\n",
    "1.batch_size=100，100筆就調一次參數\n",
    "\n",
    "2.epochs = 20，60000筆共訓練20次"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\USER\\anaconda3\\lib\\site-packages\\keras\\backend\\tensorflow_backend.py:422: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.\n",
      "\n",
      "Epoch 1/20\n",
      "60000/60000 [==============================] - 6s 98us/step - loss: 0.0823 - accuracy: 0.3909\n",
      "Epoch 2/20\n",
      "60000/60000 [==============================] - 6s 92us/step - loss: 0.0618 - accuracy: 0.6344\n",
      "Epoch 3/20\n",
      "60000/60000 [==============================] - 5s 89us/step - loss: 0.0439 - accuracy: 0.7769\n",
      "Epoch 4/20\n",
      "60000/60000 [==============================] - 5s 88us/step - loss: 0.0320 - accuracy: 0.8505\n",
      "Epoch 5/20\n",
      "60000/60000 [==============================] - 6s 96us/step - loss: 0.0249 - accuracy: 0.8782\n",
      "Epoch 6/20\n",
      "60000/60000 [==============================] - 6s 92us/step - loss: 0.0208 - accuracy: 0.8914\n",
      "Epoch 7/20\n",
      "60000/60000 [==============================] - 5s 87us/step - loss: 0.0183 - accuracy: 0.9002\n",
      "Epoch 8/20\n",
      "60000/60000 [==============================] - 5s 90us/step - loss: 0.0165 - accuracy: 0.9064\n",
      "Epoch 9/20\n",
      "60000/60000 [==============================] - 5s 89us/step - loss: 0.0153 - accuracy: 0.9125\n",
      "Epoch 10/20\n",
      "60000/60000 [==============================] - 6s 92us/step - loss: 0.0143 - accuracy: 0.9167\n",
      "Epoch 11/20\n",
      "60000/60000 [==============================] - 5s 81us/step - loss: 0.0134 - accuracy: 0.9208\n",
      "Epoch 12/20\n",
      "60000/60000 [==============================] - 5s 83us/step - loss: 0.0128 - accuracy: 0.9248\n",
      "Epoch 13/20\n",
      "60000/60000 [==============================] - 5s 81us/step - loss: 0.0122 - accuracy: 0.9281\n",
      "Epoch 14/20\n",
      "60000/60000 [==============================] - 5s 81us/step - loss: 0.0116 - accuracy: 0.9308\n",
      "Epoch 15/20\n",
      "60000/60000 [==============================] - 5s 85us/step - loss: 0.0111 - accuracy: 0.9339\n",
      "Epoch 16/20\n",
      "60000/60000 [==============================] - 5s 86us/step - loss: 0.0107 - accuracy: 0.9369\n",
      "Epoch 17/20\n",
      "60000/60000 [==============================] - 5s 83us/step - loss: 0.0103 - accuracy: 0.9388\n",
      "Epoch 18/20\n",
      "60000/60000 [==============================] - 5s 89us/step - loss: 0.0100 - accuracy: 0.9409\n",
      "Epoch 19/20\n",
      "60000/60000 [==============================] - 5s 89us/step - loss: 0.0097 - accuracy: 0.9430\n",
      "Epoch 20/20\n",
      "60000/60000 [==============================] - 5s 89us/step - loss: 0.0094 - accuracy: 0.9450\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.callbacks.History at 0x18379942948>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(x_train, y_train, batch_size=100, epochs = 20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "from ipywidgets import interact_manual"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "predict 放的是神經網路的學習結果。這裡用 `predict_classes` 會讓 Keras 選 10 個輸出機率最大的那類"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "predict = model.predict_classes(x_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "#x_test每筆資料已經換成 784 維的向量, 要變回 28x28 的矩陣才能當成圖形顯示\n",
    "def test(測試編號):\n",
    "    plt.imshow(x_test[測試編號].reshape(28,28), cmap='viridis')\n",
    "    print(\"神經網路判斷為:\", predict[測試編號])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9c9bc361ad1644f1b69393e16d46b8bf",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "interactive(children=(IntSlider(value=4999, description='測試編號', max=9999), Button(description='Run Interact', …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<function __main__.test(測試編號)>"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "interact_manual(test, 測試編號 = (0,9999))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "#將神經網路存起來，以json格式\n",
    "model_json = model.to_json()\n",
    "open('NN_handwriting.json','w').write(model_json)\n",
    "model.save_weights('NN_handwriting_weights.h5') #存參數"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}