dlnd_image_classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Image Classification\n",
    "This project classify images from the [CIFAR-10 dataset](https://www.cs.toronto.edu/~kriz/cifar.html).  The dataset consists of airplanes, dogs, cats, and other objects. The images will be preprocess, then a convolutional neural network will be trained with all the samples. The images need to be normalized and the labels need to be one-hot encoded. \n",
    "## Get the Data\n",
    "Run the following cell to download the [CIFAR-10 dataset for python](https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "All files found!\n"
     ]
    }
   ],
   "source": [
    "from urllib.request import urlretrieve\n",
    "from os.path import isfile, isdir\n",
    "from tqdm import tqdm\n",
    "import problem_unittests as tests\n",
    "import tarfile\n",
    "\n",
    "cifar10_dataset_folder_path = 'cifar-10-batches-py'\n",
    "\n",
    "# Use Floyd's cifar-10 dataset if present\n",
    "floyd_cifar10_location = '/cifar/cifar-10-python.tar.gz'\n",
    "if isfile(floyd_cifar10_location):\n",
    "    tar_gz_path = floyd_cifar10_location\n",
    "else:\n",
    "    tar_gz_path = 'cifar-10-python.tar.gz'\n",
    "\n",
    "class DLProgress(tqdm):\n",
    "    last_block = 0\n",
    "\n",
    "    def hook(self, block_num=1, block_size=1, total_size=None):\n",
    "        self.total = total_size\n",
    "        self.update((block_num - self.last_block) * block_size)\n",
    "        self.last_block = block_num\n",
    "\n",
    "if not isfile(tar_gz_path):\n",
    "    with DLProgress(unit='B', unit_scale=True, miniters=1, desc='CIFAR-10 Dataset') as pbar:\n",
    "        urlretrieve(\n",
    "            'https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz',\n",
    "            tar_gz_path,\n",
    "            pbar.hook)\n",
    "\n",
    "if not isdir(cifar10_dataset_folder_path):\n",
    "    with tarfile.open(tar_gz_path) as tar:\n",
    "        tar.extractall()\n",
    "        tar.close()\n",
    "\n",
    "\n",
    "tests.test_folder_path(cifar10_dataset_folder_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Explore the Data\n",
    "The dataset is broken into batches to prevent your machine from running out of memory.  The CIFAR-10 dataset consists of 5 batches, named `data_batch_1`, `data_batch_2`, etc.. Each batch contains the labels and images that are one of the following:\n",
    "* airplane\n",
    "* automobile\n",
    "* bird\n",
    "* cat\n",
    "* deer\n",
    "* dog\n",
    "* frog\n",
    "* horse\n",
    "* ship\n",
    "* truck\n",
    "\n",
    "Understanding a dataset is part of making predictions on the data.  Play around with the code cell below by changing the `batch_id` and `sample_id`. The `batch_id` is the id for a batch (1-5). The `sample_id` is the id for a image and label pair in the batch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Stats of batch 2:\n",
      "Samples: 10000\n",
      "Label Counts: {0: 984, 1: 1007, 2: 1010, 3: 995, 4: 1010, 5: 988, 6: 1008, 7: 1026, 8: 987, 9: 985}\n",
      "First 20 Labels: [1, 6, 6, 8, 8, 3, 4, 6, 0, 6, 0, 3, 6, 6, 5, 4, 8, 3, 2, 6]\n",
      "\n",
      "Example of Image 8:\n",
      "Image - Min Value: 31 Max Value: 255\n",
      "Image - Shape: (32, 32, 3)\n",
      "Label - Label Id: 0 Name: airplane\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfoAAAH0CAYAAADVH+85AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAWJQAAFiUBSVIk8AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAFExJREFUeJzt3duPXWd5BvC19p4Zj+3ENnYSEpxAQoCKAq2gHApS76rSqupFpP6dve4NFaUHoFBAJUlJIOEQ2zlAiIOdSWzP7NULhCqoVPE9TvaMH36/+1fvmrXX/p69buaZl2WZAIBOq+O+AADgvSPoAaCYoAeAYoIeAIoJegAoJugBoJigB4Bigh4Aigl6ACgm6AGgmKAHgGKCHgCKCXoAKCboAaCYoAeAYoIeAIoJegAotnPcF/Be+dEbB8u2ds3zHM2tVuO/s7JN0zTPm3Ay2ZVeZbArnFsv27tG3h3LMv6VTmbuZu6k2+b92ITvkelJtWzGJzfx/Qh2hX/Yxx8+f9eHlTd6ACgm6AGgmKAHgGKCHgCKCXoAKCboAaCYoAeAYoIeAIoJegAoJugBoJigB4Bigh4Aigl6AChW216XNMNte1fS8raKe4zSwfF2p7S9bqutd2GTlM67e0va1rZJq8ZOuG3ej1X8fc7mkk8sTYllk1zj8TUieqMHgGKCHgCKCXoAKCboAaCYoAeAYoIeAIoJegAoJugBoJigB4Bigh4Aigl6ACgm6AGgWG2pDb9re4UK2yynSaXlHhyfk/5c3QvPVHqNydwSnjnpXbwX7v9x8UYPAMUEPQAUE/QAUEzQA0AxQQ8AxQQ9ABQT9ABQTNADQDFBDwDFBD0AFBP0AFBM0ANAMUEPAMVq2+u22XSV7ormwj9rTgcDcYtUcD/mcNVqFX5mW7yP/Lbk+7LNRrNtNsOlc+lZtVqvh2eW9LuyhHPBa+tms8l2BY6zfdEbPQAUE/QAUEzQA0AxQQ8AxQQ9ABQT9ABQTNADQDFBDwDFBD0AFBP0AFBM0ANAMUEPAMVqS222KS2lOM6SgxMpuI95ZYnfuPD7Sk6quNQmPRaDwyA9g7dZlvRucNoBQDFBDwDFBD0AFBP0AFBM0ANAMUEPAMUEPQAUE/QAUEzQA0AxQQ8AxQQ9ABQT9ABQTNADQDHtde8CLXTHx53npEiPgXusCO33t8UWulR6dt9rZ743egAoJugBoJigB4Bigh4Aigl6ACgm6AGgmKAHgGKCHgCKCXoAKCboAaCYoAeAYoIeAIoJegAoVtxel/6G2V51UtJ/tAovb06rpLZa0nSy7/3dzN0L2/hfUTlZWEM3z9t77tNNS/Asbvs7ts1vyyp4QI6zpNAbPQAUE/QAUEzQA0AxQQ8AxQQ9ABQT9ABQTNADQDFBDwDFBD0AFBP0AFBM0ANAMUEPAMVqS23meZ3NTZvhmXXYprAKSjCSMoVpmqb1lN2PJbjGbZZ0pOa4YiK5/9us6ch2pWVJiSV8PjbL+Hcz3ZWcA/NyJ9sVfqc38/h72lF8DozPpE/9eovlQOGq6I9bjrGjyhs9ABQT9ABQTNADQDFBDwDFBD0AFBP0AFBM0ANAMUEPAMUEPQAUE/QAUEzQA0AxQQ8AxQQ9ABQrbq/LfsMkrWZpE1oyl7aMhQVZYX/aNmua0vqpbTbsnfymvOQzm8M6rvFeuN8sTAdP9rL4/Nji/ViCz3qOq+HS83R7u+413ugBoJigB4Bigh4Aigl6ACgm6AGgmKAHgGKCHgCKCXoAKCboAaCYoAeAYoIeAIoJegAoVltqs9piictqi6U2ubhKJJC2bWyvxGWajrKxeZv3MRE+U8s6GNr2e0LwWYfFO4llSu7h3exLimayXckZt93zjf+PN3oAKCboAaCYoAeAYoIeAIoJegAoJugBoJigB4Bigh4Aigl6ACgm6AGgmKAHgGKCHgCKCXoAKFbcXhc2yi1BS9OSNZpttxEqnRv/LbiEjWHzvL12siVs84vuf9qkGDzD6d8VlwBuddn4s5hu2kRNdOGZk55VSXtdtClv6Iwk50BoCc77e5E3egAoJugBoJigB4Bigh4Aigl6ACgm6AGgmKAHgGKCHgCKCXoAKCboAaCYoAeAYoIeAIoJegAoVtteN4ctXqtgLt2VNKGlxU7znP2mS9rQ5vD34507h8Mzbx/cinZN01E0td4Z/wDOnDkV7ZqDArWoAXCapiltvQssyR82hc1r4f3YHI3fj7S1cb3a3v1YhU2bS3BWhbdj2maVYvp9yVrvtloR+Vu80QNAMUEPAMUEPQAUE/QAUEzQA0AxQQ8AxQQ9ABQT9ABQTNADQDFBDwDFBD0AFBP0AFCsttRmFZUOTNMczK2CwodpmqZlExSrhL0Iy5yVuBwGY5vD29Gu13/xq+GZZ555Ptq12WTlHpcunR+e+cQn/yjadfa+/WAqLKeJno+0zCktWApmsq/m9PK1V4dnDm69E+364GOXo7n9vd3hmbRWJTkX05qktAwnL3Tq540eAIoJegAoJugBoJigB4Bigh4Aigl6ACgm6AGgmKAHgGKCHgCKCXoAKCboAaCYoAeAYoIeAIrVttfNYaNcMreETWivvTrekHX9+uvRrnlnHc0dHY7P3Lx5K9r1xutvDc+k7XVv3cyaxi5dujg8s7OzF+366MceH565/9yZaNc8B8/9kjUiLul3M2gnu3njINr13e89PTxz/cZ4++I0TdP5CxeiudMPvG98KGzzS6Rdclu8xK06znI9b/QAUEzQA0AxQQ8AxQQ9ABQT9ABQTNADQDFBDwDFBD0AFBP0AFBM0ANAMUEPAMUEPQAUqy212WZxxhLWNxwcjBervPDCi9GuGzffjOY2m/Hfgoe3s3t/dDR+H/f3wgqMU0FbzzRNh++8MTzzn9/6j2jXa7/4+fDMh554PNp1+fKl4ZmzZ7PjYzNlJVDLNF6i8+JL16JdP74yXji1u5edA0vSHDVN02oZn1uW7N4fHIwXVb107Wq0675z90VzDz300PBM+qabnPnHWdbjjR4Aigl6ACgm6AGgmKAHgGKCHgCKCXoAKCboAaCYoAeAYoIeAIoJegAoJugBoJigB4Bigh4AitW2122CFrppmqZlM94xtFrtRrsefezx4Zn1OvttdvVnL0Rz166+PDzz9ju/inYdHo43a+3MWSfUmb2sze/wcHzfUdjmd+3qePvXy69lf9eHnnhkeOaJJ8dnpmmaLj30YDT3zp3xtscfXXkl2nVrGf9Onzt7Otp1Zj87P1ab28MzSWPmNE3TP33l68MzX/2Xr0a7Pv/5T0dzTz311PDMKsyJpLA06w18d3ijB4Bigh4Aigl6ACgm6AGgmKAHgGKCHgCKCXoAKCboAaCYoAeAYoIeAIoJegAoJugBoFhtqU0qqR/JKkumaffU/vDM4x96Itr1yIMXo7krD700PPOTn/402vX0008Pz/z8tay0ZH/nTja3P15cchT+nj61f2p45tz948/UNE3TzTfGy3C+992b0a6HH8lKj06duX945u0bWZXI/s74rvv378t27Waf2dtvHQzPPP/cc9Gur/3z14Znvv3tb0W7PvvZP4nm1uv18Mxms72qmbA+513hjR4Aigl6ACgm6AGgmKAHgGKCHgCKCXoAKCboAaCYoAeAYoIeAIoJegAoJugBoJigB4Bigh4Aimmv+11BFd28ynqJVvP43GqV/TY7c1/WrPXkxz42PPPwBy5Huy4+8ODwzNPf/69o14s/fDaaW6bxhqzb79yIdr165Y3hmXduXIl2PfDIJ4Znrh+ETXnXs9a79z/ygeGZ+3f3ol231rvju/ay79ibb7wVzb3w3Piz/41vfD3adfvOreGZL33pi9GudC44TqdlybpH52RZ3HN697zRA0AxQQ8AxQQ9ABQT9ABQTNADQDFBDwDFBD0AFBP0AFBM0ANAMUEPAMUEPQAUE/QAUEzQA0CxOW3vOemuHdzJ/rDN+NhO2Ci3nsfnVtNRtGteDqO5qBIqlDyL198Yb3ibpml67tlnorlfvfHzYOaVaNfVl54fnvnJCz+Idm12x5vhbty6P9r16c+MN+VN0zR9+a//anxoydrrXnn5+vDMqy//Mtr19jvju6Zpmq5c/dHwzAsvjD9T0zRN589fHJ556u+finZ9/gt/Fs1NwfkRHPexJWyv+8CFM3d9CHujB4Bigh4Aigl6ACgm6AGgmKAHgGKCHgCKCXoAKCboAaCYoAeAYoIeAIoJegAoJugBoNjOcV/ASbO9CpdQWDKzzOtsXbIr2pRNXrh4Kdr0uS98KZq7fXBjeOZX169Fu775b+MFRtde+nG069prV4dnvv989ndtDsd3TdM0Pfrw+DP8+c/9ebTrIx88Mzzzna8/G+165gf/Hc29fXQwPPO+C+ejXZ/77HjRzKc+kZUXzUHZ1zRlpTFzeFplZXDHly7e6AGgmKAHgGKCHgCKCXoAKCboAaCYoAeAYoIeAIoJegAoJugBoJigB4Bigh4Aigl6ACgm6AGgmPa6Y5R1GYW/zcLipKSjKW6vC65xDpft7GZtfnvnxr8y587fF+26c/vO8Mxrv7ge7bq1/Gh45v2v34x2Hd16M5r7yj/+w/DMT37wnWjXY5c/OjzzyAMXol23P3I5mvvXb39reObipYvRrs985jPDM2fPjjcATtM0baJmuEzWQnfv8UYPAMUEPQAUE/QAUEzQA0AxQQ8AxQQ9ABQT9ABQTNADQDFBDwDFBD0AFBP0AFBM0ANAsbn1n/pfO7gT/WHzZnxsvcp+L+3M43NzWk6Ttr8EwkuM/rh01zr4nKdpmpbN0fDMHN77O7duDc9cvXol2vXylZeHZ579/nejXa+8/MNo7tqV8eKd1199Jdq1szo1PPPJP/1itOvJT306mnv1zYPhmdVqP9r1d1/+m+GZBx9+f7Rrs9reWbVZNtlgcIlLeFh94Pzp+Ej9DW/0AFBM0ANAMUEPAMUEPQAUE/QAUEzQA0AxQQ8AxQQ9ABQT9ABQTNADQDFBDwDFBD0AFBP0AFBs57gv4KRJepPSBsAl2paZt9hSOMcVe8GubFP8mU1B42B653f2zgzPPPb4R6Ndly8/PjzzkY88Ge169ZWfRXM/fP6Z4Zlvfv3fo10/efHHwzNXr74U7br/4vuiuXcOx2d2d09Hu67/fPxvu3DhbLRrvZ9d4xKcBvMqO0HSJrrj4o0eAIoJegAoJugBoJigB4Bigh4Aigl6ACgm6AGgmKAHgGKCHgCKCXoAKCboAaCYoAeAYoIeAIrNcYvXCXft4E72h202wyProNHs13PjFUjpL7PVNtvrwkaoOe6i4zfij3k5Gh/ZBPVp0zQdHWVzb928MTxz9eq1aNeVn4231/00aNebpmn65euvR3OHR+Of2S9e/2W069HLjw7P/MVf/m2066FHn4jm9vb3h2fOnrsv2rWztzc8cxRkyzRN02PnT9/1weiNHgCKCXoAKCboAaCYoAeAYoIeAIoJegAoJugBoJigB4Bigh4Aigl6ACgm6AGgmKAHgGI7x30BFbbZC3Qv9L6k9+Ne+NtOuKAnKbfK3hN2VuOFINM0TRcuXhqeOXvuQrTrwx/+8PDML//449GuH7/4QjS3v396eOatt96Kdh0cHAzPnLvwvmjXKnyuDg/Hy5LSopnd6Ht2fAVy3ugBoJigB4Bigh4Aigl6ACgm6AGgmKAHgGKCHgCKCXoAKCboAaCYoAeAYoIeAIoJegAoJugBoNi8LMfXqPNeunZwJ/vDgjajdfh7aR1Uja3CdrLVFj/nOaxQS+d4FyxJi1fW/JVLnuHsmdpsjoZnkva0aZqm27duR3NzcBjM4f04Ohq/Hzu7p6Jdh+OrpmmaplvB/V/CV9317u74zE5WFvvY+dN3fTB6oweAYoIeAIoJegAoJugBoJigB4Bigh4Aigl6ACgm6AGgmKAHgGKCHgCKCXoAKCboAaBY9l/2i0WlD7pYuNdFj/12H/zNJim1Ccuc5vF3oNXemWjV3u7paC67H5nkjXAOr+/UOhqb1nvjRTN3ghKzaZqmTfRcHV+BnDd6ACgm6AGgmKAHgGKCHgCKCXoAKCboAaCYoAeAYoIeAIoJegAoJugBoJigB4Bigh4Aigl6ACimve53aaLjD1DvY582hgXvQHNYu5aax5vXlmWLjXerbNccXuMcfGbr8MFPxhbtdQDAe0HQA0AxQQ8AxQQ9ABQT9ABQTNADQDFBDwDFBD0AFBP0AFBM0ANAMUEPAMUEPQAUE/QAUEx7HXBPmOfxzrAtlrVNaVNe2hw4R/vGG+9+s+29n+C94o0eAIoJegAoJugBoJigB4Bigh4Aigl6ACgm6AGgmKAHgGKCHgCKCXoAKCboAaCYoAeAYkptjpPWB/j9bbGgJvlqrpasMCYu3gn2Lfmy4Yk5POC22kP0B8IbPQAUE/QAUEzQA0AxQQ8AxQQ9ABQT9ABQTNADQDFBDwDFBD0AFBP0AFBM0ANAMUEPAMUEPQAUq22vu/Hm9WhutRr/7bOzs852zeO77ts/He3KerUyc9o/dcLbye5m7qSLbv2Wa8aW6O5v8RPb5pdsCtvh4va6YFU8l01mc+nzMb5ri7f+//BGDwDFBD0AFBP0AFBM0ANAMUEPAMUEPQAUE/QAUEzQA0AxQQ8AxQQ9ABQT9ABQTNADQLHaUpvXXns1mjs6Ohqe2dvbi3bt7u6ODz3wYLRrb5191Ov1eGFPMvNr22t9WB1jwUSP1oqfzNZLS6J9J/8zy+9jMDif/PvxbvBGDwDFBD0AFBP0AFBM0ANAMUEPAMUEPQAUE/QAUEzQA0AxQQ8AxQQ9ABQT9ABQTNADQDFBDwDF5mXrlUsAwLZ4oweAYoIeAIoJegAoJugBoJigB4Bigh4Aigl6ACgm6AGgmKAHgGKCHgCKCXoAKCboAaCYoAeAYoIeAIoJegAoJugBoJigB4Bigh4Aigl6ACgm6AGgmKAHgGKCHgCKCXoAKCboAaCYoAeAYoIeAIoJegAoJugBoJigB4Bigh4Aigl6ACgm6AGgmKAHgGKCHgCKCXoAKCboAaCYoAeAYoIeAIoJegAoJugBoJigB4Bigh4Aigl6ACgm6AGgmKAHgGKCHgCK/Q/GHHQCiOxRVQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 250,
       "width": 253
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "%config InlineBackend.figure_format = 'retina'\n",
    "\n",
    "import helper\n",
    "import numpy as np\n",
    "\n",
    "batch_id = 2\n",
    "sample_id = 8\n",
    "helper.display_stats(cifar10_dataset_folder_path, batch_id, sample_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implement Preprocess Functions\n",
    "### Normalize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def normalize(x):\n",
    "    \"\"\"\n",
    "    Normalize a list of sample image data in the range of 0 to 1\n",
    "    : x: List of image data.  The image shape is (32, 32, 3)\n",
    "    : return: Numpy array of normalize data\n",
    "    \"\"\"\n",
    "    normalized_x = np.zeros(tuple(x.shape))\n",
    "    nr_images = x.shape[0]\n",
    "    max_value, min_value = x.max(), x.min()\n",
    "\n",
    "    for image_index in range(nr_images):\n",
    "        normalized_x[image_index,...] = (x[image_index, ...] - float(min_value)) / float(max_value - min_value)    \n",
    "    return normalized_x\n",
    "\n",
    "tests.test_normalize(normalize)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### One-hot encode\n",
    "The input, `x`, are a list of labels. The possible values for labels are 0 to 9. The one-hot encoding function should return the same encoding for each value between each call to `one_hot_encode`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def one_hot_encode(x):\n",
    "    \"\"\"\n",
    "    One hot encode a list of sample labels. Return a one-hot encoded vector for each label.\n",
    "    : x: List of sample Labels\n",
    "    : return: Numpy array of one-hot encoded labels\n",
    "    \"\"\"\n",
    "    from sklearn.preprocessing import OneHotEncoder\n",
    "    encoder = OneHotEncoder(10)    \n",
    "    one_hot_encoded_labels = encoder.fit_transform(np.array(x).reshape(-1, 1)).toarray()\n",
    "    return one_hot_encoded_labels\n",
    "\n",
    "\n",
    "tests.test_one_hot_encode(one_hot_encode)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Randomize Data\n",
    "As you saw from exploring the data above, the order of the samples is already randomized."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preprocess all the data and save it\n",
    "The code cell below will preprocess all the CIFAR-10 data and save it to file. The code below also uses 10% of the training data for validation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Preprocess Training, Validation, and Testing Data\n",
    "helper.preprocess_and_save_data(cifar10_dataset_folder_path, normalize, one_hot_encode)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Check Point\n",
    "The preprocessed data has been saved to disk."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle\n",
    "import problem_unittests as tests\n",
    "import helper\n",
    "\n",
    "# Load the Preprocessed Validation data\n",
    "valid_features, valid_labels = pickle.load(open('preprocess_validation.p', mode='rb'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build the network\n",
    "\n",
    "### Input\n",
    "The neural network needs to read the image data, one-hot encoded labels, and dropout keep probability. \n",
    "These names will be used at the end of the project to load the saved model.\n",
    "\n",
    "Note: `None` for shapes in TensorFlow allow for a dynamic size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Image Input Tests Passed.\n",
      "Label Input Tests Passed.\n",
      "Keep Prob Tests Passed.\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "def neural_net_image_input(image_shape):\n",
    "    \"\"\"\n",
    "    Return a Tensor for a batch of image input\n",
    "    : image_shape: Shape of the images\n",
    "    : return: Tensor for image input.\n",
    "    \"\"\"\n",
    "    return tf.placeholder(tf.float32, shape=((None,) + image_shape), name='x')\n",
    "\n",
    "\n",
    "def neural_net_label_input(n_classes):\n",
    "    \"\"\"\n",
    "    Return a Tensor for a batch of label input\n",
    "    : n_classes: Number of classes\n",
    "    : return: Tensor for label input.\n",
    "    \"\"\"\n",
    "    return tf.placeholder(tf.float32, shape=(None, n_classes), name='y')\n",
    "\n",
    "\n",
    "def neural_net_keep_prob_input():\n",
    "    \"\"\"\n",
    "    Return a Tensor for keep probability\n",
    "    : return: Tensor for keep probability.\n",
    "    \"\"\"\n",
    "    return tf.placeholder(tf.float32, shape=(None), name='keep_prob')\n",
    "\n",
    "\n",
    "\n",
    "tf.reset_default_graph()\n",
    "tests.test_nn_image_inputs(neural_net_image_input)\n",
    "tests.test_nn_label_inputs(neural_net_label_input)\n",
    "tests.test_nn_keep_prob_inputs(neural_net_keep_prob_input)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convolution and Max Pooling Layer\n",
    "Convolution layers have a lot of success with images. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides):\n",
    "    \"\"\"\n",
    "    Apply convolution then max pooling to x_tensor\n",
    "    :param x_tensor: TensorFlow Tensor\n",
    "    :param conv_num_outputs: Number of outputs for the convolutional layer\n",
    "    :param conv_ksize: kernal size 2-D Tuple for the convolutional layer\n",
    "    :param conv_strides: Stride 2-D Tuple for convolution\n",
    "    :param pool_ksize: kernal size 2-D Tuple for pool\n",
    "    :param pool_strides: Stride 2-D Tuple for pool\n",
    "    : return: A tensor that represents convolution and max pooling of x_tensor\n",
    "    \"\"\"\n",
    "    weights_shape = list(conv_ksize) + [x_tensor.get_shape().as_list()[3], conv_num_outputs]\n",
    "    weights = tf.Variable(tf.truncated_normal(weights_shape, stddev=5e-2))\n",
    "    bias = tf.Variable(tf.zeros(conv_num_outputs))\n",
    "    \n",
    "    output = tf.nn.conv2d(x_tensor, weights, \n",
    "                          strides=[1, conv_strides[0], conv_strides[1], 1],\n",
    "                          padding='SAME')\n",
    "    output = tf.nn.bias_add(output, bias)\n",
    "    output = tf.nn.relu(output)\n",
    "    \n",
    "    output = tf.nn.max_pool(output, \n",
    "                            ksize=[1, pool_ksize[0], pool_ksize[1], 1], \n",
    "                            strides=[1, pool_strides[0], pool_strides[1], 1],\n",
    "                            padding='SAME')                                  \n",
    "    return output\n",
    "\n",
    "\n",
    "tests.test_con_pool(conv2d_maxpool)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Flatten Layer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def flatten(x_tensor):\n",
    "    \"\"\"\n",
    "    Flatten x_tensor to (Batch Size, Flattened Image Size)\n",
    "    : x_tensor: A tensor of size (Batch Size, ...), where ... are the image dimensions.\n",
    "    : return: A tensor of size (Batch Size, Flattened Image Size).\n",
    "    \"\"\"\n",
    "    return tf.contrib.layers.flatten(x_tensor)\n",
    "\n",
    "\n",
    "tests.test_flatten(flatten)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fully-Connected Layer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def fully_conn(x_tensor, num_outputs):\n",
    "    \"\"\"\n",
    "    Apply a fully connected layer to x_tensor using weight and bias\n",
    "    : x_tensor: A 2-D tensor where the first dimension is batch size.\n",
    "    : num_outputs: The number of output that the new tensor should be.\n",
    "    : return: A 2-D tensor where the second dimension is num_outputs.\n",
    "    \"\"\"\n",
    "    return tf.layers.dense(x_tensor, num_outputs, activation=tf.nn.relu)  \n",
    "\n",
    "\n",
    "\n",
    "tests.test_fully_conn(fully_conn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Output Layer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def output(x_tensor, num_outputs):\n",
    "    \"\"\"\n",
    "    Apply a output layer to x_tensor using weight and bias\n",
    "    : x_tensor: A 2-D tensor where the first dimension is batch size.\n",
    "    : num_outputs: The number of output that the new tensor should be.\n",
    "    : return: A 2-D tensor where the second dimension is num_outputs.\n",
    "    \"\"\"\n",
    "    return tf.layers.dense(x_tensor, num_outputs)\n",
    "\n",
    "\n",
    "tests.test_output(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create Convolutional Model\n",
    "The function `conv_net` create a convolutional neural network model. The function takes in a batch of images, `x`, and outputs logits."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Neural Network Built!\n"
     ]
    }
   ],
   "source": [
    "def conv_net(x, keep_prob):\n",
    "    \"\"\"\n",
    "    Create a convolutional neural network model\n",
    "    : x: Placeholder tensor that holds image data.\n",
    "    : keep_prob: Placeholder tensor that hold dropout keep probability.\n",
    "    : return: Tensor that represents logits\n",
    "    \"\"\"\n",
    "    # Function Definition from Above:\n",
    "    #    conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides)\n",
    "    conv = conv2d_maxpool(x,\n",
    "                           conv_num_outputs=64,\n",
    "                           conv_ksize=[5,5],\n",
    "                           conv_strides=[1,1],\n",
    "                           pool_ksize=[3,3],\n",
    "                           pool_strides=[2,2]) # 14x14x64\n",
    "    conv = tf.layers.dropout(conv, rate=keep_prob)\n",
    "    conv = conv2d_maxpool(conv,\n",
    "                          conv_num_outputs=64,\n",
    "                          conv_ksize=[5,5],\n",
    "                          conv_strides=[1,1],\n",
    "                          pool_ksize=[3,3],\n",
    "                          pool_strides=[2,2]) # 7x7x64\n",
    "    conv = tf.layers.dropout(conv, rate=keep_prob)\n",
    "        \n",
    "    # Function Definition from Above:\n",
    "    #   flatten(x_tensor)\n",
    "    flattened_conv = flatten(conv)\n",
    "    \n",
    "    \n",
    "    # Function Definition from Above:\n",
    "    #   fully_conn(x_tensor, num_outputs)\n",
    "    fc = fully_conn(flattened_conv, 384)\n",
    "    fc = fully_conn(fc, 192)\n",
    "    \n",
    "    \n",
    "    # Function Definition from Above:\n",
    "    #   output(x_tensor, num_outputs)\n",
    "    fc = output(fc, 10)\n",
    "    \n",
    "    return fc      \n",
    "\n",
    "\n",
    "\n",
    "##############################\n",
    "## Build the Neural Network ##\n",
    "##############################\n",
    "\n",
    "# Remove previous weights, bias, inputs, etc..\n",
    "tf.reset_default_graph()\n",
    "\n",
    "# Inputs\n",
    "x = neural_net_image_input((32, 32, 3))\n",
    "y = neural_net_label_input(10)\n",
    "keep_prob = neural_net_keep_prob_input()\n",
    "\n",
    "# Model\n",
    "logits = conv_net(x, keep_prob)\n",
    "\n",
    "# Name logits Tensor, so that is can be loaded from disk after training\n",
    "logits = tf.identity(logits, name='logits')\n",
    "\n",
    "# Loss and Optimizer\n",
    "cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=y))\n",
    "optimizer = tf.train.AdamOptimizer().minimize(cost)\n",
    "\n",
    "# Accuracy\n",
    "correct_pred = tf.equal(tf.argmax(logits, 1), tf.argmax(y, 1))\n",
    "accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32), name='accuracy')\n",
    "\n",
    "tests.test_conv_net(conv_net)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train the Neural Network\n",
    "### Single Optimization\n",
    "The function `train_neural_network` to do a single optimization. \n",
    "This function will be called for each batch, so `tf.global_variables_initializer()` has already been called.\n",
    "\n",
    "Note: Nothing needs to be returned. This function is only optimizing the neural network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def train_neural_network(session, optimizer, keep_probability, feature_batch, label_batch):\n",
    "    \"\"\"\n",
    "    Optimize the session on a batch of images and labels\n",
    "    : session: Current TensorFlow session\n",
    "    : optimizer: TensorFlow optimizer function\n",
    "    : keep_probability: keep probability\n",
    "    : feature_batch: Batch of Numpy image data\n",
    "    : label_batch: Batch of Numpy label data\n",
    "    \"\"\"\n",
    "    session.run(optimizer, feed_dict={x: feature_batch,\n",
    "                                      y: label_batch,\n",
    "                                      keep_prob: keep_probability})\n",
    "\n",
    "\n",
    "tests.test_train_nn(train_neural_network)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Show Stats\n",
    "The function `print_stats` print loss and validation accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def print_stats(session, feature_batch, label_batch, cost, accuracy):\n",
    "    \"\"\"\n",
    "    Print information about loss and validation accuracy\n",
    "    : session: Current TensorFlow session\n",
    "    : feature_batch: Batch of Numpy image data\n",
    "    : label_batch: Batch of Numpy label data\n",
    "    : cost: TensorFlow cost function\n",
    "    : accuracy: TensorFlow accuracy function\n",
    "    \"\"\"\n",
    "    loss = sess.run(cost, feed_dict={x: feature_batch,\n",
    "                                     y: label_batch,\n",
    "                                     keep_prob: 1.})\n",
    "    valid_acc = sess.run(accuracy, feed_dict={x: valid_features,\n",
    "                                              y: valid_labels,\n",
    "                                              keep_prob: 1.})\n",
    "\n",
    "    print('Loss: {:>10.4f} Accuracy: {:.6f}'.format(loss, valid_acc))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hyperparameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "epochs = 50\n",
    "batch_size = 8192 # Set batch_size to the highest number that your machine has memory for: 64, 128, 256...\n",
    "keep_probability = 0.7"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train on a Single CIFAR-10 Batch\n",
    "Instead of training the neural network on all the CIFAR-10 batches of data, let's use a single batch. This should save time while you iterate on the model to get a better accuracy.  Once the final validation accuracy is 50% or greater, run the model on all the data in the next section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Checking the Training on a Single Batch...\n",
      "Epoch  1, CIFAR-10 Batch 1:  Loss:     2.3154 Accuracy: 0.105000\n",
      "Epoch  2, CIFAR-10 Batch 1:  Loss:     2.2545 Accuracy: 0.156000\n",
      "Epoch  3, CIFAR-10 Batch 1:  Loss:     2.1907 Accuracy: 0.243600\n",
      "Epoch  4, CIFAR-10 Batch 1:  Loss:     2.0756 Accuracy: 0.251800\n",
      "Epoch  5, CIFAR-10 Batch 1:  Loss:     2.0792 Accuracy: 0.264600\n",
      "Epoch  6, CIFAR-10 Batch 1:  Loss:     1.9137 Accuracy: 0.311000\n",
      "Epoch  7, CIFAR-10 Batch 1:  Loss:     1.8590 Accuracy: 0.341600\n",
      "Epoch  8, CIFAR-10 Batch 1:  Loss:     1.8131 Accuracy: 0.343200\n",
      "Epoch  9, CIFAR-10 Batch 1:  Loss:     1.7733 Accuracy: 0.342600\n",
      "Epoch 10, CIFAR-10 Batch 1:  Loss:     1.7241 Accuracy: 0.353400\n",
      "Epoch 11, CIFAR-10 Batch 1:  Loss:     1.6642 Accuracy: 0.387200\n",
      "Epoch 12, CIFAR-10 Batch 1:  Loss:     1.6094 Accuracy: 0.386200\n",
      "Epoch 13, CIFAR-10 Batch 1:  Loss:     1.5594 Accuracy: 0.404200\n",
      "Epoch 14, CIFAR-10 Batch 1:  Loss:     1.5724 Accuracy: 0.394600\n",
      "Epoch 15, CIFAR-10 Batch 1:  Loss:     1.5043 Accuracy: 0.401400\n",
      "Epoch 16, CIFAR-10 Batch 1:  Loss:     1.4634 Accuracy: 0.428600\n",
      "Epoch 17, CIFAR-10 Batch 1:  Loss:     1.4352 Accuracy: 0.431000\n",
      "Epoch 18, CIFAR-10 Batch 1:  Loss:     1.4104 Accuracy: 0.427400\n",
      "Epoch 19, CIFAR-10 Batch 1:  Loss:     1.3760 Accuracy: 0.437000\n",
      "Epoch 20, CIFAR-10 Batch 1:  Loss:     1.3397 Accuracy: 0.450000\n",
      "Epoch 21, CIFAR-10 Batch 1:  Loss:     1.2960 Accuracy: 0.456800\n",
      "Epoch 22, CIFAR-10 Batch 1:  Loss:     1.2534 Accuracy: 0.461000\n",
      "Epoch 23, CIFAR-10 Batch 1:  Loss:     1.2316 Accuracy: 0.453800\n",
      "Epoch 24, CIFAR-10 Batch 1:  Loss:     1.2012 Accuracy: 0.470800\n",
      "Epoch 25, CIFAR-10 Batch 1:  Loss:     1.1544 Accuracy: 0.470400\n",
      "Epoch 26, CIFAR-10 Batch 1:  Loss:     1.1524 Accuracy: 0.468800\n",
      "Epoch 27, CIFAR-10 Batch 1:  Loss:     1.1946 Accuracy: 0.452600\n",
      "Epoch 28, CIFAR-10 Batch 1:  Loss:     1.0968 Accuracy: 0.487800\n",
      "Epoch 29, CIFAR-10 Batch 1:  Loss:     1.0610 Accuracy: 0.487000\n",
      "Epoch 30, CIFAR-10 Batch 1:  Loss:     1.0380 Accuracy: 0.489000\n",
      "Epoch 31, CIFAR-10 Batch 1:  Loss:     1.0213 Accuracy: 0.487600\n",
      "Epoch 32, CIFAR-10 Batch 1:  Loss:     0.9634 Accuracy: 0.503200\n",
      "Epoch 33, CIFAR-10 Batch 1:  Loss:     0.9270 Accuracy: 0.498800\n",
      "Epoch 34, CIFAR-10 Batch 1:  Loss:     0.9266 Accuracy: 0.506600\n",
      "Epoch 35, CIFAR-10 Batch 1:  Loss:     0.8808 Accuracy: 0.512200\n",
      "Epoch 36, CIFAR-10 Batch 1:  Loss:     0.8508 Accuracy: 0.508000\n",
      "Epoch 37, CIFAR-10 Batch 1:  Loss:     0.8153 Accuracy: 0.513600\n",
      "Epoch 38, CIFAR-10 Batch 1:  Loss:     0.7998 Accuracy: 0.516200\n",
      "Epoch 39, CIFAR-10 Batch 1:  Loss:     0.7631 Accuracy: 0.525200\n",
      "Epoch 40, CIFAR-10 Batch 1:  Loss:     0.7445 Accuracy: 0.515200\n",
      "Epoch 41, CIFAR-10 Batch 1:  Loss:     0.7095 Accuracy: 0.532600\n",
      "Epoch 42, CIFAR-10 Batch 1:  Loss:     0.6917 Accuracy: 0.526000\n",
      "Epoch 43, CIFAR-10 Batch 1:  Loss:     0.6776 Accuracy: 0.526600\n",
      "Epoch 44, CIFAR-10 Batch 1:  Loss:     0.7156 Accuracy: 0.509200\n",
      "Epoch 45, CIFAR-10 Batch 1:  Loss:     0.6400 Accuracy: 0.527800\n",
      "Epoch 46, CIFAR-10 Batch 1:  Loss:     0.6634 Accuracy: 0.508800\n",
      "Epoch 47, CIFAR-10 Batch 1:  Loss:     0.6526 Accuracy: 0.521800\n",
      "Epoch 48, CIFAR-10 Batch 1:  Loss:     0.6344 Accuracy: 0.521400\n",
      "Epoch 49, CIFAR-10 Batch 1:  Loss:     0.5920 Accuracy: 0.523400\n",
      "Epoch 50, CIFAR-10 Batch 1:  Loss:     0.5607 Accuracy: 0.535200\n"
     ]
    }
   ],
   "source": [
    "print('Checking the Training on a Single Batch...')\n",
    "with tf.Session() as sess:\n",
    "    # Initializing the variables\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "    \n",
    "    # Training cycle\n",
    "    for epoch in range(epochs):\n",
    "        batch_i = 1\n",
    "        for batch_features, batch_labels in helper.load_preprocess_training_batch(batch_i, batch_size):\n",
    "            train_neural_network(sess, optimizer, keep_probability, batch_features, batch_labels)\n",
    "        print('Epoch {:>2}, CIFAR-10 Batch {}:  '.format(epoch + 1, batch_i), end='')\n",
    "        print_stats(sess, batch_features, batch_labels, cost, accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fully Train the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training...\n",
      "Epoch  1, CIFAR-10 Batch 1:  Loss:     2.2720 Accuracy: 0.162000\n",
      "Epoch  1, CIFAR-10 Batch 2:  Loss:     2.2380 Accuracy: 0.215600\n",
      "Epoch  1, CIFAR-10 Batch 3:  Loss:     2.1585 Accuracy: 0.228000\n",
      "Epoch  1, CIFAR-10 Batch 4:  Loss:     2.0087 Accuracy: 0.264800\n",
      "Epoch  1, CIFAR-10 Batch 5:  Loss:     2.2584 Accuracy: 0.188200\n",
      "Epoch  2, CIFAR-10 Batch 1:  Loss:     2.1457 Accuracy: 0.235800\n",
      "Epoch  2, CIFAR-10 Batch 2:  Loss:     1.9713 Accuracy: 0.282400\n",
      "Epoch  2, CIFAR-10 Batch 3:  Loss:     1.8977 Accuracy: 0.281600\n",
      "Epoch  2, CIFAR-10 Batch 4:  Loss:     1.8225 Accuracy: 0.310600\n",
      "Epoch  2, CIFAR-10 Batch 5:  Loss:     1.8463 Accuracy: 0.316000\n",
      "Epoch  3, CIFAR-10 Batch 1:  Loss:     1.8035 Accuracy: 0.349400\n",
      "Epoch  3, CIFAR-10 Batch 2:  Loss:     1.7478 Accuracy: 0.346200\n",
      "Epoch  3, CIFAR-10 Batch 3:  Loss:     1.6847 Accuracy: 0.363600\n",
      "Epoch  3, CIFAR-10 Batch 4:  Loss:     1.6249 Accuracy: 0.398200\n",
      "Epoch  3, CIFAR-10 Batch 5:  Loss:     1.6719 Accuracy: 0.402200\n",
      "Epoch  4, CIFAR-10 Batch 1:  Loss:     1.5978 Accuracy: 0.420800\n",
      "Epoch  4, CIFAR-10 Batch 2:  Loss:     1.5700 Accuracy: 0.411600\n",
      "Epoch  4, CIFAR-10 Batch 3:  Loss:     1.5021 Accuracy: 0.438400\n",
      "Epoch  4, CIFAR-10 Batch 4:  Loss:     1.4719 Accuracy: 0.438400\n",
      "Epoch  4, CIFAR-10 Batch 5:  Loss:     1.5076 Accuracy: 0.434800\n",
      "Epoch  5, CIFAR-10 Batch 1:  Loss:     1.4582 Accuracy: 0.459000\n",
      "Epoch  5, CIFAR-10 Batch 2:  Loss:     1.4577 Accuracy: 0.455200\n",
      "Epoch  5, CIFAR-10 Batch 3:  Loss:     1.3994 Accuracy: 0.452600\n",
      "Epoch  5, CIFAR-10 Batch 4:  Loss:     1.3792 Accuracy: 0.473000\n",
      "Epoch  5, CIFAR-10 Batch 5:  Loss:     1.4285 Accuracy: 0.467800\n",
      "Epoch  6, CIFAR-10 Batch 1:  Loss:     1.4016 Accuracy: 0.473800\n",
      "Epoch  6, CIFAR-10 Batch 2:  Loss:     1.3855 Accuracy: 0.464000\n",
      "Epoch  6, CIFAR-10 Batch 3:  Loss:     1.3703 Accuracy: 0.474400\n",
      "Epoch  6, CIFAR-10 Batch 4:  Loss:     1.3304 Accuracy: 0.485000\n",
      "Epoch  6, CIFAR-10 Batch 5:  Loss:     1.3374 Accuracy: 0.481800\n",
      "Epoch  7, CIFAR-10 Batch 1:  Loss:     1.3120 Accuracy: 0.500000\n",
      "Epoch  7, CIFAR-10 Batch 2:  Loss:     1.2850 Accuracy: 0.508200\n",
      "Epoch  7, CIFAR-10 Batch 3:  Loss:     1.2684 Accuracy: 0.504000\n",
      "Epoch  7, CIFAR-10 Batch 4:  Loss:     1.2516 Accuracy: 0.504000\n",
      "Epoch  7, CIFAR-10 Batch 5:  Loss:     1.2435 Accuracy: 0.519800\n",
      "Epoch  8, CIFAR-10 Batch 1:  Loss:     1.2148 Accuracy: 0.520200\n",
      "Epoch  8, CIFAR-10 Batch 2:  Loss:     1.2193 Accuracy: 0.523200\n",
      "Epoch  8, CIFAR-10 Batch 3:  Loss:     1.2104 Accuracy: 0.514800\n",
      "Epoch  8, CIFAR-10 Batch 4:  Loss:     1.1829 Accuracy: 0.519800\n",
      "Epoch  8, CIFAR-10 Batch 5:  Loss:     1.1917 Accuracy: 0.528600\n",
      "Epoch  9, CIFAR-10 Batch 1:  Loss:     1.1629 Accuracy: 0.533800\n",
      "Epoch  9, CIFAR-10 Batch 2:  Loss:     1.1417 Accuracy: 0.542000\n",
      "Epoch  9, CIFAR-10 Batch 3:  Loss:     1.1035 Accuracy: 0.542600\n",
      "Epoch  9, CIFAR-10 Batch 4:  Loss:     1.0830 Accuracy: 0.549200\n",
      "Epoch  9, CIFAR-10 Batch 5:  Loss:     1.1188 Accuracy: 0.542000\n",
      "Epoch 10, CIFAR-10 Batch 1:  Loss:     1.1086 Accuracy: 0.542400\n",
      "Epoch 10, CIFAR-10 Batch 2:  Loss:     1.0883 Accuracy: 0.552400\n",
      "Epoch 10, CIFAR-10 Batch 3:  Loss:     1.0254 Accuracy: 0.561800\n",
      "Epoch 10, CIFAR-10 Batch 4:  Loss:     1.0187 Accuracy: 0.556600\n",
      "Epoch 10, CIFAR-10 Batch 5:  Loss:     1.0713 Accuracy: 0.542800\n",
      "Epoch 11, CIFAR-10 Batch 1:  Loss:     1.0754 Accuracy: 0.539200\n",
      "Epoch 11, CIFAR-10 Batch 2:  Loss:     1.0836 Accuracy: 0.548400\n",
      "Epoch 11, CIFAR-10 Batch 3:  Loss:     1.0273 Accuracy: 0.541000\n",
      "Epoch 11, CIFAR-10 Batch 4:  Loss:     1.0136 Accuracy: 0.560200\n",
      "Epoch 11, CIFAR-10 Batch 5:  Loss:     0.9758 Accuracy: 0.576600\n",
      "Epoch 12, CIFAR-10 Batch 1:  Loss:     0.9689 Accuracy: 0.580200\n",
      "Epoch 12, CIFAR-10 Batch 2:  Loss:     0.9816 Accuracy: 0.574600\n",
      "Epoch 12, CIFAR-10 Batch 3:  Loss:     0.9295 Accuracy: 0.574600\n",
      "Epoch 12, CIFAR-10 Batch 4:  Loss:     0.9132 Accuracy: 0.580000\n",
      "Epoch 12, CIFAR-10 Batch 5:  Loss:     0.9096 Accuracy: 0.581400\n",
      "Epoch 13, CIFAR-10 Batch 1:  Loss:     0.9210 Accuracy: 0.580400\n",
      "Epoch 13, CIFAR-10 Batch 2:  Loss:     0.9238 Accuracy: 0.585000\n",
      "Epoch 13, CIFAR-10 Batch 3:  Loss:     0.8808 Accuracy: 0.571600\n",
      "Epoch 13, CIFAR-10 Batch 4:  Loss:     0.8938 Accuracy: 0.585800\n",
      "Epoch 13, CIFAR-10 Batch 5:  Loss:     0.8498 Accuracy: 0.594000\n",
      "Epoch 14, CIFAR-10 Batch 1:  Loss:     0.8542 Accuracy: 0.594600\n",
      "Epoch 14, CIFAR-10 Batch 2:  Loss:     0.8783 Accuracy: 0.588400\n",
      "Epoch 14, CIFAR-10 Batch 3:  Loss:     0.8400 Accuracy: 0.588800\n",
      "Epoch 14, CIFAR-10 Batch 4:  Loss:     0.8394 Accuracy: 0.585600\n",
      "Epoch 14, CIFAR-10 Batch 5:  Loss:     0.8156 Accuracy: 0.598200\n",
      "Epoch 15, CIFAR-10 Batch 1:  Loss:     0.8174 Accuracy: 0.598600\n",
      "Epoch 15, CIFAR-10 Batch 2:  Loss:     0.8217 Accuracy: 0.599200\n",
      "Epoch 15, CIFAR-10 Batch 3:  Loss:     0.7619 Accuracy: 0.603400\n",
      "Epoch 15, CIFAR-10 Batch 4:  Loss:     0.7629 Accuracy: 0.612000\n",
      "Epoch 15, CIFAR-10 Batch 5:  Loss:     0.7433 Accuracy: 0.605200\n",
      "Epoch 16, CIFAR-10 Batch 1:  Loss:     0.7621 Accuracy: 0.615800\n",
      "Epoch 16, CIFAR-10 Batch 2:  Loss:     0.7503 Accuracy: 0.616800\n",
      "Epoch 16, CIFAR-10 Batch 3:  Loss:     0.7238 Accuracy: 0.607000\n",
      "Epoch 16, CIFAR-10 Batch 4:  Loss:     0.7418 Accuracy: 0.606200\n",
      "Epoch 16, CIFAR-10 Batch 5:  Loss:     0.7285 Accuracy: 0.596800\n",
      "Epoch 17, CIFAR-10 Batch 1:  Loss:     0.7132 Accuracy: 0.626000\n",
      "Epoch 17, CIFAR-10 Batch 2:  Loss:     0.8047 Accuracy: 0.582600\n",
      "Epoch 17, CIFAR-10 Batch 3:  Loss:     0.7163 Accuracy: 0.608000\n",
      "Epoch 17, CIFAR-10 Batch 4:  Loss:     0.6995 Accuracy: 0.607800\n",
      "Epoch 17, CIFAR-10 Batch 5:  Loss:     0.7079 Accuracy: 0.608200\n",
      "Epoch 18, CIFAR-10 Batch 1:  Loss:     0.7083 Accuracy: 0.615400\n",
      "Epoch 18, CIFAR-10 Batch 2:  Loss:     0.6971 Accuracy: 0.622400\n",
      "Epoch 18, CIFAR-10 Batch 3:  Loss:     0.6508 Accuracy: 0.627600\n",
      "Epoch 18, CIFAR-10 Batch 4:  Loss:     0.6575 Accuracy: 0.617000\n",
      "Epoch 18, CIFAR-10 Batch 5:  Loss:     0.6430 Accuracy: 0.618800\n",
      "Epoch 19, CIFAR-10 Batch 1:  Loss:     0.6776 Accuracy: 0.618000\n",
      "Epoch 19, CIFAR-10 Batch 2:  Loss:     0.6702 Accuracy: 0.623800\n",
      "Epoch 19, CIFAR-10 Batch 3:  Loss:     0.6376 Accuracy: 0.609600\n",
      "Epoch 19, CIFAR-10 Batch 4:  Loss:     0.6052 Accuracy: 0.631200\n",
      "Epoch 19, CIFAR-10 Batch 5:  Loss:     0.5893 Accuracy: 0.618600\n",
      "Epoch 20, CIFAR-10 Batch 1:  Loss:     0.6066 Accuracy: 0.632000\n",
      "Epoch 20, CIFAR-10 Batch 2:  Loss:     0.6046 Accuracy: 0.631400\n",
      "Epoch 20, CIFAR-10 Batch 3:  Loss:     0.5870 Accuracy: 0.619000\n",
      "Epoch 20, CIFAR-10 Batch 4:  Loss:     0.5692 Accuracy: 0.628800\n",
      "Epoch 20, CIFAR-10 Batch 5:  Loss:     0.5539 Accuracy: 0.624000\n",
      "Epoch 21, CIFAR-10 Batch 1:  Loss:     0.5786 Accuracy: 0.626400\n",
      "Epoch 21, CIFAR-10 Batch 2:  Loss:     0.5805 Accuracy: 0.637400\n",
      "Epoch 21, CIFAR-10 Batch 3:  Loss:     0.5427 Accuracy: 0.627600\n",
      "Epoch 21, CIFAR-10 Batch 4:  Loss:     0.5313 Accuracy: 0.638800\n",
      "Epoch 21, CIFAR-10 Batch 5:  Loss:     0.4893 Accuracy: 0.640800\n",
      "Epoch 22, CIFAR-10 Batch 1:  Loss:     0.5400 Accuracy: 0.636600\n",
      "Epoch 22, CIFAR-10 Batch 2:  Loss:     0.5462 Accuracy: 0.637600\n",
      "Epoch 22, CIFAR-10 Batch 3:  Loss:     0.5572 Accuracy: 0.617200\n",
      "Epoch 22, CIFAR-10 Batch 4:  Loss:     0.4878 Accuracy: 0.648800\n",
      "Epoch 22, CIFAR-10 Batch 5:  Loss:     0.4632 Accuracy: 0.645200\n",
      "Epoch 23, CIFAR-10 Batch 1:  Loss:     0.4907 Accuracy: 0.636400\n",
      "Epoch 23, CIFAR-10 Batch 2:  Loss:     0.5219 Accuracy: 0.632400\n",
      "Epoch 23, CIFAR-10 Batch 3:  Loss:     0.4776 Accuracy: 0.636200\n",
      "Epoch 23, CIFAR-10 Batch 4:  Loss:     0.4844 Accuracy: 0.631800\n",
      "Epoch 23, CIFAR-10 Batch 5:  Loss:     0.4521 Accuracy: 0.638400\n",
      "Epoch 24, CIFAR-10 Batch 1:  Loss:     0.5183 Accuracy: 0.626400\n",
      "Epoch 24, CIFAR-10 Batch 2:  Loss:     0.4903 Accuracy: 0.644000\n",
      "Epoch 24, CIFAR-10 Batch 3:  Loss:     0.4539 Accuracy: 0.647800\n",
      "Epoch 24, CIFAR-10 Batch 4:  Loss:     0.4592 Accuracy: 0.624400\n",
      "Epoch 24, CIFAR-10 Batch 5:  Loss:     0.4385 Accuracy: 0.639800\n",
      "Epoch 25, CIFAR-10 Batch 1:  Loss:     0.4473 Accuracy: 0.645200\n",
      "Epoch 25, CIFAR-10 Batch 2:  Loss:     0.4358 Accuracy: 0.644400\n",
      "Epoch 25, CIFAR-10 Batch 3:  Loss:     0.4472 Accuracy: 0.639400\n",
      "Epoch 25, CIFAR-10 Batch 4:  Loss:     0.4376 Accuracy: 0.631600\n",
      "Epoch 25, CIFAR-10 Batch 5:  Loss:     0.4495 Accuracy: 0.614000\n",
      "Epoch 26, CIFAR-10 Batch 1:  Loss:     0.4656 Accuracy: 0.643600\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 26, CIFAR-10 Batch 2:  Loss:     0.4434 Accuracy: 0.638800\n",
      "Epoch 26, CIFAR-10 Batch 3:  Loss:     0.4035 Accuracy: 0.652400\n",
      "Epoch 26, CIFAR-10 Batch 4:  Loss:     0.3785 Accuracy: 0.651400\n",
      "Epoch 26, CIFAR-10 Batch 5:  Loss:     0.3944 Accuracy: 0.631000\n",
      "Epoch 27, CIFAR-10 Batch 1:  Loss:     0.4198 Accuracy: 0.652600\n",
      "Epoch 27, CIFAR-10 Batch 2:  Loss:     0.4237 Accuracy: 0.637400\n",
      "Epoch 27, CIFAR-10 Batch 3:  Loss:     0.4526 Accuracy: 0.622000\n",
      "Epoch 27, CIFAR-10 Batch 4:  Loss:     0.4441 Accuracy: 0.615800\n",
      "Epoch 27, CIFAR-10 Batch 5:  Loss:     0.4119 Accuracy: 0.636800\n",
      "Epoch 28, CIFAR-10 Batch 1:  Loss:     0.4361 Accuracy: 0.629000\n",
      "Epoch 28, CIFAR-10 Batch 2:  Loss:     0.4481 Accuracy: 0.635800\n",
      "Epoch 28, CIFAR-10 Batch 3:  Loss:     0.3964 Accuracy: 0.643000\n",
      "Epoch 28, CIFAR-10 Batch 4:  Loss:     0.3801 Accuracy: 0.648600\n",
      "Epoch 28, CIFAR-10 Batch 5:  Loss:     0.3633 Accuracy: 0.646800\n",
      "Epoch 29, CIFAR-10 Batch 1:  Loss:     0.3753 Accuracy: 0.645400\n",
      "Epoch 29, CIFAR-10 Batch 2:  Loss:     0.3446 Accuracy: 0.660000\n",
      "Epoch 29, CIFAR-10 Batch 3:  Loss:     0.3482 Accuracy: 0.653600\n",
      "Epoch 29, CIFAR-10 Batch 4:  Loss:     0.3321 Accuracy: 0.657000\n",
      "Epoch 29, CIFAR-10 Batch 5:  Loss:     0.3039 Accuracy: 0.655200\n",
      "Epoch 30, CIFAR-10 Batch 1:  Loss:     0.3363 Accuracy: 0.652800\n",
      "Epoch 30, CIFAR-10 Batch 2:  Loss:     0.3173 Accuracy: 0.661800\n",
      "Epoch 30, CIFAR-10 Batch 3:  Loss:     0.3113 Accuracy: 0.648400\n",
      "Epoch 30, CIFAR-10 Batch 4:  Loss:     0.3354 Accuracy: 0.647200\n",
      "Epoch 30, CIFAR-10 Batch 5:  Loss:     0.3182 Accuracy: 0.645600\n",
      "Epoch 31, CIFAR-10 Batch 1:  Loss:     0.3299 Accuracy: 0.654400\n",
      "Epoch 31, CIFAR-10 Batch 2:  Loss:     0.3135 Accuracy: 0.660000\n",
      "Epoch 31, CIFAR-10 Batch 3:  Loss:     0.2922 Accuracy: 0.661800\n",
      "Epoch 31, CIFAR-10 Batch 4:  Loss:     0.2757 Accuracy: 0.643800\n",
      "Epoch 31, CIFAR-10 Batch 5:  Loss:     0.2959 Accuracy: 0.658800\n",
      "Epoch 32, CIFAR-10 Batch 1:  Loss:     0.3560 Accuracy: 0.630200\n",
      "Epoch 32, CIFAR-10 Batch 2:  Loss:     0.3813 Accuracy: 0.645400\n",
      "Epoch 32, CIFAR-10 Batch 3:  Loss:     0.3588 Accuracy: 0.638800\n",
      "Epoch 32, CIFAR-10 Batch 4:  Loss:     0.2998 Accuracy: 0.652200\n",
      "Epoch 32, CIFAR-10 Batch 5:  Loss:     0.2787 Accuracy: 0.653400\n",
      "Epoch 33, CIFAR-10 Batch 1:  Loss:     0.3004 Accuracy: 0.653800\n",
      "Epoch 33, CIFAR-10 Batch 2:  Loss:     0.3006 Accuracy: 0.658600\n",
      "Epoch 33, CIFAR-10 Batch 3:  Loss:     0.3028 Accuracy: 0.654600\n",
      "Epoch 33, CIFAR-10 Batch 4:  Loss:     0.2905 Accuracy: 0.648000\n",
      "Epoch 33, CIFAR-10 Batch 5:  Loss:     0.2793 Accuracy: 0.647000\n",
      "Epoch 34, CIFAR-10 Batch 1:  Loss:     0.3125 Accuracy: 0.647600\n",
      "Epoch 34, CIFAR-10 Batch 2:  Loss:     0.3184 Accuracy: 0.645800\n",
      "Epoch 34, CIFAR-10 Batch 3:  Loss:     0.3267 Accuracy: 0.647800\n",
      "Epoch 34, CIFAR-10 Batch 4:  Loss:     0.3089 Accuracy: 0.632400\n",
      "Epoch 34, CIFAR-10 Batch 5:  Loss:     0.3720 Accuracy: 0.626200\n",
      "Epoch 35, CIFAR-10 Batch 1:  Loss:     0.3316 Accuracy: 0.640800\n",
      "Epoch 35, CIFAR-10 Batch 2:  Loss:     0.2880 Accuracy: 0.655800\n",
      "Epoch 35, CIFAR-10 Batch 3:  Loss:     0.3253 Accuracy: 0.649800\n",
      "Epoch 35, CIFAR-10 Batch 4:  Loss:     0.3019 Accuracy: 0.636000\n",
      "Epoch 35, CIFAR-10 Batch 5:  Loss:     0.2437 Accuracy: 0.669400\n",
      "Epoch 36, CIFAR-10 Batch 1:  Loss:     0.2967 Accuracy: 0.644200\n",
      "Epoch 36, CIFAR-10 Batch 2:  Loss:     0.3008 Accuracy: 0.643600\n",
      "Epoch 36, CIFAR-10 Batch 3:  Loss:     0.2441 Accuracy: 0.664000\n",
      "Epoch 36, CIFAR-10 Batch 4:  Loss:     0.2780 Accuracy: 0.654600\n",
      "Epoch 36, CIFAR-10 Batch 5:  Loss:     0.2895 Accuracy: 0.636200\n",
      "Epoch 37, CIFAR-10 Batch 1:  Loss:     0.3189 Accuracy: 0.641000\n",
      "Epoch 37, CIFAR-10 Batch 2:  Loss:     0.2999 Accuracy: 0.639400\n",
      "Epoch 37, CIFAR-10 Batch 3:  Loss:     0.2693 Accuracy: 0.660600\n",
      "Epoch 37, CIFAR-10 Batch 4:  Loss:     0.2689 Accuracy: 0.639000\n",
      "Epoch 37, CIFAR-10 Batch 5:  Loss:     0.2505 Accuracy: 0.659600\n",
      "Epoch 38, CIFAR-10 Batch 1:  Loss:     0.2365 Accuracy: 0.671800\n",
      "Epoch 38, CIFAR-10 Batch 2:  Loss:     0.2431 Accuracy: 0.656200\n",
      "Epoch 38, CIFAR-10 Batch 3:  Loss:     0.2340 Accuracy: 0.665800\n",
      "Epoch 38, CIFAR-10 Batch 4:  Loss:     0.2095 Accuracy: 0.660000\n",
      "Epoch 38, CIFAR-10 Batch 5:  Loss:     0.2004 Accuracy: 0.660000\n",
      "Epoch 39, CIFAR-10 Batch 1:  Loss:     0.2350 Accuracy: 0.662800\n",
      "Epoch 39, CIFAR-10 Batch 2:  Loss:     0.2294 Accuracy: 0.654200\n",
      "Epoch 39, CIFAR-10 Batch 3:  Loss:     0.2087 Accuracy: 0.671400\n",
      "Epoch 39, CIFAR-10 Batch 4:  Loss:     0.1886 Accuracy: 0.668600\n",
      "Epoch 39, CIFAR-10 Batch 5:  Loss:     0.2025 Accuracy: 0.659800\n",
      "Epoch 40, CIFAR-10 Batch 1:  Loss:     0.2388 Accuracy: 0.657000\n",
      "Epoch 40, CIFAR-10 Batch 2:  Loss:     0.2231 Accuracy: 0.653600\n",
      "Epoch 40, CIFAR-10 Batch 3:  Loss:     0.2248 Accuracy: 0.656600\n",
      "Epoch 40, CIFAR-10 Batch 4:  Loss:     0.2357 Accuracy: 0.648800\n",
      "Epoch 40, CIFAR-10 Batch 5:  Loss:     0.2998 Accuracy: 0.624000\n",
      "Epoch 41, CIFAR-10 Batch 1:  Loss:     0.3166 Accuracy: 0.645200\n",
      "Epoch 41, CIFAR-10 Batch 2:  Loss:     0.2685 Accuracy: 0.645000\n",
      "Epoch 41, CIFAR-10 Batch 3:  Loss:     0.2379 Accuracy: 0.658200\n",
      "Epoch 41, CIFAR-10 Batch 4:  Loss:     0.2624 Accuracy: 0.646800\n",
      "Epoch 41, CIFAR-10 Batch 5:  Loss:     0.2088 Accuracy: 0.652600\n",
      "Epoch 42, CIFAR-10 Batch 1:  Loss:     0.2066 Accuracy: 0.670200\n",
      "Epoch 42, CIFAR-10 Batch 2:  Loss:     0.2089 Accuracy: 0.674200\n",
      "Epoch 42, CIFAR-10 Batch 3:  Loss:     0.2019 Accuracy: 0.659600\n",
      "Epoch 42, CIFAR-10 Batch 4:  Loss:     0.1982 Accuracy: 0.647400\n",
      "Epoch 42, CIFAR-10 Batch 5:  Loss:     0.1717 Accuracy: 0.668400\n",
      "Epoch 43, CIFAR-10 Batch 1:  Loss:     0.1817 Accuracy: 0.675600\n",
      "Epoch 43, CIFAR-10 Batch 2:  Loss:     0.1988 Accuracy: 0.652200\n",
      "Epoch 43, CIFAR-10 Batch 3:  Loss:     0.1731 Accuracy: 0.667200\n",
      "Epoch 43, CIFAR-10 Batch 4:  Loss:     0.1434 Accuracy: 0.680600\n",
      "Epoch 43, CIFAR-10 Batch 5:  Loss:     0.1602 Accuracy: 0.670800\n",
      "Epoch 44, CIFAR-10 Batch 1:  Loss:     0.2028 Accuracy: 0.657600\n",
      "Epoch 44, CIFAR-10 Batch 2:  Loss:     0.1684 Accuracy: 0.674200\n",
      "Epoch 44, CIFAR-10 Batch 3:  Loss:     0.1767 Accuracy: 0.668800\n",
      "Epoch 44, CIFAR-10 Batch 4:  Loss:     0.1711 Accuracy: 0.667600\n",
      "Epoch 44, CIFAR-10 Batch 5:  Loss:     0.1413 Accuracy: 0.676400\n",
      "Epoch 45, CIFAR-10 Batch 1:  Loss:     0.1777 Accuracy: 0.667400\n",
      "Epoch 45, CIFAR-10 Batch 2:  Loss:     0.1690 Accuracy: 0.669600\n",
      "Epoch 45, CIFAR-10 Batch 3:  Loss:     0.1511 Accuracy: 0.674000\n",
      "Epoch 45, CIFAR-10 Batch 4:  Loss:     0.1377 Accuracy: 0.676200\n",
      "Epoch 45, CIFAR-10 Batch 5:  Loss:     0.1294 Accuracy: 0.677600\n",
      "Epoch 46, CIFAR-10 Batch 1:  Loss:     0.1631 Accuracy: 0.672000\n",
      "Epoch 46, CIFAR-10 Batch 2:  Loss:     0.1532 Accuracy: 0.671400\n",
      "Epoch 46, CIFAR-10 Batch 3:  Loss:     0.1338 Accuracy: 0.685800\n",
      "Epoch 46, CIFAR-10 Batch 4:  Loss:     0.1151 Accuracy: 0.690600\n",
      "Epoch 46, CIFAR-10 Batch 5:  Loss:     0.1118 Accuracy: 0.682000\n",
      "Epoch 47, CIFAR-10 Batch 1:  Loss:     0.1302 Accuracy: 0.680000\n",
      "Epoch 47, CIFAR-10 Batch 2:  Loss:     0.1255 Accuracy: 0.682400\n",
      "Epoch 47, CIFAR-10 Batch 3:  Loss:     0.1194 Accuracy: 0.680800\n",
      "Epoch 47, CIFAR-10 Batch 4:  Loss:     0.1514 Accuracy: 0.665400\n",
      "Epoch 47, CIFAR-10 Batch 5:  Loss:     0.1832 Accuracy: 0.650400\n",
      "Epoch 48, CIFAR-10 Batch 1:  Loss:     0.2481 Accuracy: 0.645000\n",
      "Epoch 48, CIFAR-10 Batch 2:  Loss:     0.1784 Accuracy: 0.663600\n",
      "Epoch 48, CIFAR-10 Batch 3:  Loss:     0.1449 Accuracy: 0.680800\n",
      "Epoch 48, CIFAR-10 Batch 4:  Loss:     0.1951 Accuracy: 0.648000\n",
      "Epoch 48, CIFAR-10 Batch 5:  Loss:     0.1533 Accuracy: 0.667800\n",
      "Epoch 49, CIFAR-10 Batch 1:  Loss:     0.1708 Accuracy: 0.674400\n",
      "Epoch 49, CIFAR-10 Batch 2:  Loss:     0.1711 Accuracy: 0.662200\n",
      "Epoch 49, CIFAR-10 Batch 3:  Loss:     0.1469 Accuracy: 0.670400\n",
      "Epoch 49, CIFAR-10 Batch 4:  Loss:     0.1446 Accuracy: 0.674400\n",
      "Epoch 49, CIFAR-10 Batch 5:  Loss:     0.1353 Accuracy: 0.674800\n",
      "Epoch 50, CIFAR-10 Batch 1:  Loss:     0.1905 Accuracy: 0.650200\n",
      "Epoch 50, CIFAR-10 Batch 2:  Loss:     0.2944 Accuracy: 0.615600\n",
      "Epoch 50, CIFAR-10 Batch 3:  Loss:     0.2690 Accuracy: 0.620600\n",
      "Epoch 50, CIFAR-10 Batch 4:  Loss:     0.1717 Accuracy: 0.666800\n",
      "Epoch 50, CIFAR-10 Batch 5:  Loss:     0.1642 Accuracy: 0.659400\n"
     ]
    }
   ],
   "source": [
    "save_model_path = './image_classification'\n",
    "\n",
    "print('Training...')\n",
    "with tf.Session() as sess:\n",
    "    # Initializing the variables\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "    \n",
    "    # Training cycle\n",
    "    for epoch in range(epochs):\n",
    "        # Loop over all batches\n",
    "        n_batches = 5\n",
    "        for batch_i in range(1, n_batches + 1):\n",
    "            for batch_features, batch_labels in helper.load_preprocess_training_batch(batch_i, batch_size):\n",
    "                train_neural_network(sess, optimizer, keep_probability, batch_features, batch_labels)\n",
    "            print('Epoch {:>2}, CIFAR-10 Batch {}:  '.format(epoch + 1, batch_i), end='')\n",
    "            print_stats(sess, batch_features, batch_labels, cost, accuracy)\n",
    "            \n",
    "    # Save Model\n",
    "    saver = tf.train.Saver()\n",
    "    save_path = saver.save(sess, save_model_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Checkpoint\n",
    "\n",
    "## Test Model\n",
    "The accuracy should have greater than 50%."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./image_classification\n",
      "Testing Accuracy: 0.6622287333011627\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAscAAAJ/CAYAAACUb342AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAWJQAAFiUBSVIk8AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3XecZFWZ//HPU50nBwYYQBiSMIIREdGVYFxFBRVh1VXB1TWiYFjTusL6M6wJFFxdVhFdZcGEvlaMIKMIIgoiWSSMhIGBydMznev5/fGcqnv7TnV19XR1mJ7v+/WqV3Xde+65p6ornHrqOeeYuyMiIiIiIlCa6gaIiIiIiEwX6hyLiIiIiCTqHIuIiIiIJOoci4iIiIgk6hyLiIiIiCTqHIuIiIiIJOoci4iIiIgk6hyLiIiIiCTqHIuIiIiIJOoci4iIiIgk6hyLiIiIiCTqHIuIiIiIJOoci4iIiIgk6hyLiIiIiCTqHE8xM9vHzF5uZm81sw+a2QfM7DQze6WZPdXM5kx1G0diZiUzO97MLjazu8xsk5l57vLDqW6jyHRjZssKr5Mzm1F2ujKzYwr34ZSpbpOISD2tU92AnZGZLQLeCrwJ2GeU4mUzuw24CrgMuMLdeye4iaNK9+F7wLFT3RaZfGZ2IfD6UYoNAhuANcANxHP4f91948S2TkREZPspcjzJzOzFwG3A/2P0jjHE/+hQojP9Y+DEiWvdmHyTMXSMFT3aKbUCuwAHA68Gvgw8aGZnmpm+mO9ACq/dC6e6PSIiE0kfUJPIzE4CLgJaCrs2ATcDDwN9wEJgb2A50/ALjJk9HTgut+lvwFnAH4HNue1bJ7NdskOYDXwUOMrMXujufVPdIBERkTx1jieJme1PRFvzHeNbgA8DP3H3wRrHzAGOBl4JvAyYNwlNbcTLC7ePd/c/T0lLZLp4H5Fmk9cK7Ab8HfA24gtfxbFEJPkNk9I6ERGRBqlzPHk+DnTkbl8OvNTde0Y6wN27iTzjy8zsNOCNRHR5qh2W+3ulOsYCrHH3lTW23wVcbWZfBL5NfMmrOMXMvujuN05GA3dE6TG1qW7HeLj7Cnbw+yAiO5dp95P9TGRmXcBLc5sGgNfX6xgXuftmdz/b3S9vegPHbtfc36umrBWyw0jP9dcAd+Y2G/CWqWmRiIhIbeocT46nAF2529e4+47cqcxPLzcwZa2QHUrqIJ9d2PycqWiLiIjISJRWMTl2L9x+cDJPbmbzgGcBewKLiUFzq4Hfu/t921NlE5vXFGa2H5HusRfQDqwErnT3R0Y5bi8iJ/YxxP16KB33wDjasidwCLAfsCBtXgfcB/xuJ5/K7IrC7f3NrMXdh8ZSiZkdCjwOWEoM8lvp7hc1cFwH8AxipphdgSHitXCTu980ljaMUP+BwNOAPYBe4AHgOnef1Nd8jXY9FngSsIR4Tm4lnuu3ALe5e3kKmzcqM3sM8HQih30u8XpaBVzl7huafK79iIDGY4gxIquBq939nnHUeRDx+O9OBBcGgW7gfuCvwB3u7uNsuog0i7vrMsEX4B8Az11+OknnfSrwU6C/cP785SZimi2rU88xdY4f6bIiHbtye48ttOHCfJnc9qOBK4FyjXr6gf8E5tSo73HAT0Y4rgx8H9izwce5lNrxZeDuUe7bEJFvfmyDdX+jcPz5Y/j/f7Jw7I/r/Z/H+Ny6sFD3KQ0e11XjMdm1Rrn882ZFbvupRIeuWMeGUc57KPBdYEud/839wOlA23Y8Hs8Efj9CvYPE2IHDUtllhf1n1qm34bI1jl0A/Dvxpazec/JR4ALg8FH+xw1dGnj/aOi5ko49CbixzvkGgF8CTx9DnStyx6/MbT+C+PJW6z3BgWuBI8dwnjbgPUTe/WiP2wbiPed5zXh96qKLLuO7THkDdoYL8OzCG+FmYMEEns+AT9d5k691WQEsHKG+4odbQ/WlY1du77GFNgz7oE7b3tngffwDuQ4yMdvG1gaOWwns3cDj/YbtuI8OfA5oGaXu2cDtheP+oYE2Pa/w2DwALG7ic+zCQptOafC4zhqPw5Ia5fLPmxXEYNbv1Hksa3aOiS8unyG+lDT6f/kzDX4xSuf4UIPPw34i73pZYfuZdepuuGzhuJcB68f4fLxxlP9xQ5cG3j9Gfa4QM/NcPsZznwOUGqh7Re6YlWnbadQPIuT/hyc1cI4lxMI3Y338ftis16guuuiy/RelVUyO64kP58o0bnOAb5rZqz1mpGi2/wb+qbCtn4h8rCIiSk8lFmioOBr4jZkd5e7rJ6BNTZXmjP5CuulEdOlu4ovBk4D9c8WfCpwLnGpmxwKXkKUU3ZEu/cS80o/PHbcPEbkdbbGTYu5+D3Ar8bP1JiJaujfwBCLlo+LdROTrAyNV7O5bzOxkIirZmTafb2Z/dPe7ah1jZrsD/0OW/jIEvNrd145yPybDXoXbTnTiRnMOMaVh5Zg/kXWg9wP2LR5gZi3E//oVhV1bidfkQ8Rrcn/giWSP1xOAa8zsae6+ul6jzOx0YiaavCHi/3U/kQLwZCL9o43ocBZfm02V2vR5tk1/epj4pWgNMIv4Xzye4bPoTDkzmwv8mngd560HrkvXS4k0i3zb30W8p/3jGM/3GuCLuU23ENHePuK5cRjZY9kGXGhmf3L3v45QnwE/IP7veauJ+ezXEF+m5qf6D0ApjiLTy1T3zneWC/GTdjFKsIpYEOHxNO/n7tcXzlEmOhYLCuVaiQ/pjYXy/1ujzk4iglW5PJArf21hX+Wyezp2r3S7mFry3hGOqx5baMOFheMrUbHLgP1rlD+J6KTmH4cj02PuwDXAk2ocdwywtnCuF43ymFem2PtkOkfN6BXxpeT9DP9pvwwc0cD/9S2FNv0RaK9RrkT8zJwv+5EJeD4X/x+nNHjcPxeOu2uEcitzZTbn/v4fYK8a5ZfV2PbxwrlWE2kZtR63/dn2NfqTUe7L49k22nhR8fmb/icnAY+kMusKx5xZ5xzLGi2byr+AbaPkvybyrLd5jyE6ly8hftK/vrBvF7LXZL6+7zHya7fW/+GYsTxXgK8Xym8C3kwh3YXoXH6ObaP2bx6l/hW5st1k7xOXAgfUKL+c+DUhf45L6tR/XKHsX4mBpzXf44lfh44HLga+2+zXqi666DL2y5Q3YGe5EJGp3sKbZv6ylujofYT4SXz2dpxjDtv+lHrGKMccwbZ5mHXz3hghH3SUY8b0AVnj+AtrPGbfps7PqMSS27U61JcDHXWOe3GjH4Sp/O716qtR/sjCc6Fu/bnjLim06ws1yny4UOZX9R6jcTyfi/+PUf+fxJesYopIzRxqaqfjfGoM7TuC4Z3Ev1DjS1fhmBLb5ni/sE75KwtlvzRK/Yewbce4aZ1jIhq8ulD+vEb//8Budfbl67xwjM+Vhl/7xODYfNmtwDNHqf8dhWO6GSFFLJVfUeN/cB71x13sxvD31r6RzkGMPaiUGwD2HcNj1TmWx1YXXXSZmIumcpskHgtlvJboFNWyCHgRMYDmF8B6M7vKzN6cZptoxOvJZkcA+Jm7F6fOKrbr98C/FTa/q8HzTaVVRISo3ij7rxGR8YrKKP3Xep1li939x0RnquKYeg1x94fr1Vej/O+AL+U2nZBmURjNm4jUkYp3mtnxlRtm9nfEMt4VjwKvGeUxmhRm1klEfQ8u7PqvBqu4kej4N+oDZOkug8AJ7l53AZ30OL2Z4bPJnF6rrJk9juHPizuBM0ap/1bgX+q2enzexPA5yK8ETmv0/++jpJBMkuJ7z1nufnW9A9z9PCLqXzGbsaWu3EIEEbzOOVYTnd6KdiKto5b8SpA3uvu9jTbE3Uf6fBCRSaTO8SRy9+8SP2/+toHibUQU5SvAPWb2tpTLVs9rCrc/2mDTvkh0pCpeZGaLGjx2qpzvo+Rru3s/UPxgvdjdH2qg/l/l/t415fE2049yf7ezbX7lNtx9E5Ge0p/b/HUz2zv9v/6XLK/dgdc1eF+bYRczW1a4HGBmzzCzfwFuA04sHPNtd7++wfrP9gane0tT6eUX3bnI3W9v5NjUOTk/t+lYM5tVo2gxr/XT6fk2mguItKSJ8KbC7bodvunGzGYDJ+Q2rSdSwhrxr4XbY8k7PtvdG5mv/SeF209s4JglY2iHiEwT6hxPMnf/k7s/CziKiGzWnYc3WUxEGi82s/ZaBVLk8Sm5Tfe4+3UNtmmAmOaqWh0jR0Wmi180WO7uwu1fNnhccbDbmD/kLMw1sz2KHUe2HSxVjKjW5O5/JPKWKxYSneJvMHyw22fc/WdjbfM4fAa4t3D5K/Hl5D/YdsDc1Wzbmavnx6MXqTqG4e9t3x/DsQC/yf3dBhxeo8yRub8rU/+NKkVxvzfG9ozKzJYQaRsVf/Adb1n3wxk+MO3SRn+RSff1ttymx6eBfY1o9HVyR+H2SO8J+V+d9jGztzdYv4hMExohO0Xc/SrgKqj+RPsMYlaFw4koYq0vLicRI51rvdkeyvCR278fY5OuBd6Wu30Y20ZKppPiB9VINhVu/6VmqdGPGzW1Jc2O8FxiVoXDiQ5vzS8zNSxssBzufo6ZHUMM4oF47uRdy9hSECZTDzHLyL81GK0DuM/d143hHM8s3F6fvpA0qqVwez9iUFte/ovoX31sC1H8YQxlG3VE4fZVE3COiXZY4fb2vIc9Lv1dIt5HR3scNnnjq5UWF+8Z6T3hYoan2JxnZicQAw1/6jvAbEAiOzt1jqcBd7+NiHp8FcDMFhA/L55BTCuV9zYzu6DGz9HFKEbNaYbqKHYap/vPgY2uMjfYpOPa6hU2syOJ/NnH1ytXR6N55RWnEnm4exe2bwBe5e7F9k+FIeLxXktMvXYVkeIwlo4uDE/5aURxurjf1CzVuGEpRulXmvz/q/jrxGhqTsE3TsW0n4bSSKaZqXgPa3i1SncfKGS21XxPcPfrzOw/GR5seG66lM3sZiK17jfEgOZGfj0UkUmktIppyN03uPuFROTj32sUOa3GtgWF28XI52iKHxINRzKnwjgGmTV9cJqZ/T0x+Gl7O8Ywxtdiij59osau97j7ynG0Y3ud6u5WuLS6+2J3f6y7n+zu521Hxxhi9oGxaHa+/JzC7eJrY7yvtWZYXLjd1CWVJ8lUvIdN1GDVdxC/3mwtbC8RucpvJ2afecjMrjSzExsYUyIik0Sd42nMw0eJN9G85zZy+BhPpzfm7ZAGwn2L4SktK4GPAS8EDiI+9DvzHUdqLFoxxvMuJqb9K/pHM9vZX9d1o/zbYbTXxnR8re0wA/HqmI6Pa0PSe/cniJSc9wO/Y9tfoyA+g48hxnz82syWTlojRWRESqvYMZwLnJy7vaeZdbl7T25bMVI0f4znKP6sr7y4xryN4VG7i4HXNzBzQaODhbaRIkzfAPassftYYuR+rV8cdhb56PQg0NXkNJPia2O8r7VmKEbki1HYHcGMew9LU8B9Gvi0mc0BngY8i3idPpPhn8HPAn6WVmZseGpIEWm+nT3CtKOoNeq8+JNhMS/zgDGe47Gj1Ce1HZf7eyPwxgan9BrP1HBnFM57HcNnPfk3M3vWOOrf0eXn621lnFH6otRxyf/kv/9IZUcw1tdmI4pzOC+fgHNMtBn9Hubu3e7+K3c/y92PIZbA/ldikGrFE4A3TEX7RCSjzvGOoVZeXDEf7xaGz39bHL0+muLUbY3OP9uomfAzby35D/DfuvuWBo/brqnyzOypwKdym9YTs2O8juwxbgEuSqkXO6NrC7efMwHnuCH394FpEG2jak0NN17XMvw1tiN+OSq+54znPaxMDFidttx9jbt/nG2nNHzJVLRHRDLqHO8YDirc7i4ugJGiWfkPl/3NrDg1Uk1m1kp0sKrVMfZplEZT/Jmw0SnOprv8T78NDSBKaRGvGuuJ0kqJlzA8p/YN7n6fu/+cmGu4Yi9i6qid0eWF26dMwDl+l/u7BLyikYNSPvgrRy04Ru7+KHBrbtPTzGw8A0SL8q/fiXrt/oHhebkvG2le96J0X/PzPN/i7pub2bgJdAnDV05dNkXtEJFEneNJYGa7mdlu46ii+DPbihHKXVS4XVwWeiTvYPiysz9197UNHtuo4kjyZq84N1XyeZLFn3VH8lq272fv84kBPhXnuvsPc7c/zPCo6UvMbEdYCryp3P0u4IrcpiPMrLh65Hh9u3D7X8yskYGAb6B2rngznF+4/fkmzoCQf/1OyGs3/eqSXzlyEbXndK/lY4Xb32pKoyZByofPz2rRSFqWiEwgdY4nx3JiCehPmdmuo5bOMbNXAG8tbC7OXlHxDYZ/iL3UzN42QtlK/Yez7QfLF8fSxgbdA+QXfXj2BJxjKtyc+/swMzu6XmEzexoxwHJMzOyfGT4o80/A+/Jl0ofsqxjeYf+0meUXrNhZnFm4/d9m9ryxVGBmS83sRbX2ufutDF8Y5LHA2aPU9zhicNZE+RrD862fC5zTaAd5lC/w+TmED0+DyyZC8b3nY+k9akRm9layBXEAthCPxZQws7emFQsbLf9Chk8/2OhCRSIyQdQ5njyziCl9HjCzS83sFfXeQM1suZmdD3yH4St23cC2EWIA0s+I7y5sPtfMPmNmw0Z+m1mrmZ1KLKec/6D7TvqJvqlS2kd+OeujzeyrZvYcMzuwsLzyjhRVLi4F/H0ze2mxkJl1mdkZRERzHrHSYUPM7FDgnNymbuDkWiPa0xzH+RzGduCSMSylOyO4+28ZPg90FzETwH+a2YEjHWdmC8zsJDO7hJiS73V1TnMaw7/wvd3Mvl18/ppZycxeSfzis5AJmoPY3bcS7c2PUXgncEVapGYbZtZhZi82s+9Rf0XM/EIqc4DLzOxl6X2quDT6eO7Db4D/yW2aDfzSzP6pGJk3s3lm9mngvEI179vO+bSb5f3Afem5cMJIr730Hvw6Yvn3vB0m6i0yU2kqt8nXRqx+dwKAmd0F3Ed0lsrEh+fjgMfUOPYB4JX1FsBw9wvM7Cjg9WlTCXgvcJqZ/Q54iJjm6XBgl8Lht7NtlLqZzmX40r7/lC5Fvybm/twRXEDMHlHpcC0GfmRmfyO+yPQSP0MfQXxBghid/lZibtO6zGwW8UtBV27zW9x9xNXD3P17ZvYV4C1p0wHAl4F/bPA+zRQfIVYQrNzvEvG4vzX9f24jBjS2Ea+JAxlDvqe732xm7wc+n9v8auBkM7sWuJ/oSB5GzEwAkVN7BhOUD+7uvzCz9wKfI5v391jgGjN7CLiJWLGwi8hLfwLZHN21ZsWp+CrwHqAz3T4qXWoZbyrHO4iFMiqrg85P5/8PM7uO+HKxO3Bkrj0VF7v7l8d5/mboJJ4LrwbczO4E7iWbXm4p8GS2na7uh+7+f5PWShGpSZ3jybGO6PwWO6MQHZdGpiy6HHhTg6ufnZrOeTrZB1UH9TucvwWOn8iIi7tfYmZHEJ2DGcHd+1Kk+FdkHSCAfdKlqJsYkHVHg6c4l/iyVPF1dy/mu9ZyBvFFpDIo6zVmdoW77zSD9NKXyNea2Z+B/8fwhVpG+v8U1Z0r193PTl9gPkb2Wmth+JfAikHiy+B4l7OuK7XpQaJDmY9aLmX4c3Qsda40s1OITn3XKMXHxd03pfSkHxAd+4rFxMI6I/kSESmfbowYVF0cWF10CVlQQ0SmkNIqJoG730REOp5NRJn+CAw1cGgv8QHxEnd/XqPLAqfVmd5NTG30C2qvzFRxK/GGfNRk/BSZ2nUE8UH2ByKKtUMPQHH3O4CnED+HjvRYdwPfBJ7g7j9rpF4zexXDB2PeQe2lw2u1qZfIUc4P9DnXzA5u5PiZxN0/SwxkPIdt5wOu5S/El5Ij3X3UX1LSdFxHMTxtKK9MvA6f6e7fbKjR4+Tu3yHmd/4sw/OQa1lNDOar2zFz90uI8RNnESkiDzF8jt6mcfcNxBR8ryai3SMZIlKVnunu7xjHsvLNdDzxGF3L6O9tZaL9x7n7P2jxD5Hpwdxn6vSz01uKNj02XXYli/BsIqK+twK3NWNlr5RvfBQxSn4R0VFbDfy+0Q63NCbNLXwU8fN8J/E4PwhclXJCZYqlgXFPIH7JWUB8Cd0A3A3c6u6P1Dl8tLoPJL6ULk31Pghc5+73j7fd42iTEWkKhwBLiFSP7tS2W4HbfZp/EJjZ3sTjuhvxXrkOWEW8rqZ8JbyRmFkncCjx6+DuxGM/QAycvgu4YYrzo0WkBnWORUREREQSpVWIiIiIiCTqHIuIiIiIJOoci4iIiIgk6hyLiIiIiCTqHIuIiIiIJOoci4iIiIgk6hyLiIiIiCTqHIuIiIiIJOoci4iIiIgk6hyLiIiIiCTqHIuIiIiIJOoci4iIiIgk6hyLiIiIiCTqHIuIiIiIJOoci4iIiIgk6hyLiIiIiCTqHIuIiIiIJOoci4iIiIgk6hyLiIiIiCTqHIuIiIiIJOoci4iIiIgk6hyLiIiIiCTqHIuIiIiIJOocj4GZebosm+q2iIiIiEjzqXMsIiIiIpKocywiIiIikqhzLCIiIiKSqHMsIiIiIpKoc5xjZiUzO83M/mxmPWb2qJn9n5kd2cCxS8zsk2Z2s5l1m9kWM7vFzD5uZotGOfZQM7vAzO41s14z22BmV5vZW8ysrUb5ZZXBgen2083se2b2kJkNmdk52/8oiIiIiOy8Wqe6AdOFmbUC3wOOT5sGicfnxcDfm9nJdY79O+BHQKUT3A8MAYeky2vN7Hnu/pcax74D+ALZF5UtwBzgGelyspkd5+5bRzj3ScC3U1s3pvOKiIiIyHZQ5DjzfqJjXAbeB8x394XAfsDlwAW1DjKzfYD/IzrGXwUOBrqA2cChwM+AxwA/MLOWwrHHA+cCPcCHgN3cfU46/vnAX4BjgLPrtPtrRMd8X3dfAMwCFDkWERER2Q7m7lPdhilnZrOBVcA84Cx3P7OwvwO4AXhc2rSvu69M+74FvAb4oru/q0bd7cB1wBOBV7r799L2FuBuYB/g5e5+aY1j9wVuBjqAvd39obR9GXBvKnY1cJS7l7fv3ouIiIhIhSLH4flEx7iPGlFad+8DPlvcbmZdwCvTzc/Xqtjd+4l0DYDn5XYdQ3SMV9bqGKdj7wWuJVImjhmh7Z9Tx1hERESkOZRzHJ6Srm90940jlPl1jW1PBdrT3783s5Hq70rXj8lte0a63sPMHq7Ttvk1js37XZ1jRURERGQM1DkOS9L1qjplHqyxbWnu790aOM+sGse2b8exeY82cKyIiIiINECd4/GppKWsd/e607XVOfZSd3/59jbA3TU7hYiIiEiTKOc4VKKve9QpU2vf6nS90Mx2H+M5K8c+rm4pEREREZk06hyHG9L1k8xs3ghljq6x7Y/EfMgAY43+VnKFDzKzQ8Z4rIiIiIhMAHWOw8+BTcSUaSNNx/ae4nZ33wx8P938VzMbMXfYzFrNbE5u0xXAfenvs4tzIBeOXTjqPRARERGRcVPnGEirz3063fyomb07TdNWmVP4UkaeLeIDwDpigN01ZvayNC8y6fgDzOx04HZidovKOQeA0wAnpnj7hZkdYWnKi9SZPszMPgXc07Q7KyIiIiIj0iIgyQjLR3cDC9LfJ5NFiauLgKRjDwd+SJaXPEgs5TyHiEZXHOPuw6aEM7NTga+QTQnXSywhvQCoRpPd3XLHLCMtApLfLiIiIiLjo8hx4u6DwCuAdwI3ER3cIeAy4Gh3/0GdY/9ALBv9fuAaYDPRue0h8pL/Azi82DFOx34dOIhY8vnWdN75wFrgSuC9wLJm3EcRERERqU+RYxERERGRRJFjEREREZFEnWMRERERkUSdYxERERGRRJ1jEREREZFEnWMRERERkUSdYxERERGRRJ1jEREREZFEnWMRERERkUSdYxERERGRpHWqGyAiMhOZ2b3APGDlFDdFRGRHtQzY5O77TuZJZ2zneMhxgHJ5sLqtxBAA5qXqliqLZbTdo3w6PHZZ5WHydLvGCRtahTs7sOay3TUrrh6QitQrs21TqqVtaNvy5XYASm2lOpWKyHaa19XVtWj58uWLprohIiI7ottvv52enp5JP++M7Rwz1A/AxjX3VjeZ9QLQap1xXWrL9vnAsOtSa0t1X7mUHiaLbYMDWfezXC6nvwbT7awTWuzIuperf7e0RMfcy7lOeNpWOcos67xX6qp0qku5ulsrnXwrtgnKQ9GecktflO1YUN3X3rY3ItOJmS0D7gW+4e6nNFD+FODrwKnufmGT2nAMcCVwlrufOY6qVi5fvnzR9ddf34xmiYjsdA477DBuuOGGlZN9XuUci4iIiIgkMzdyLCI7g0uBa4GHprohtdzy4EaWfeCyqW6GiCQrP3XcVDdBdgAztnP86EO3AHDnTb+sbpu7MHJsZ7VHWkWbZ4HzocFIp2hJGbst7VnKhacUi7bWDgB6e/ur+wYHI51icChSNvr6+6r7WlviuI6OOC6f99uW6synXlgp2lMqXOfLVc7XVsrSPtpbWoeVGRgYqO7bvHkTAP3eHWU696ju23PvFwIwv+MxiOyI3H0jsHGq2yEiIjOH0ipEZFoys4PN7Idmts7MtpjZb83s+YUyp5iZp9zj/PaV6TLPzD6f/h4wszNzZXYzs6+Z2Woz6zGzG83s9ZNz70REZLqasZHj9Y/cA0Df+vuq21o8IsZEAJmO3Hi5gXJEijtmLwRgVqUs0LdxQ1ynQXctLdkguo62qKQ9BXJLadAfgHnsm99ZGQSXRXRLLbGvtTWLUFcG8w0NxYC6oYHcTBspijzYF5Fpzx9n8Xc2A0a+fdEe740I8sb1WRvWz1sV7dtFkWOZdvYFfgfcAvwXsBQ4Gfipmb3a3S9poI524FfAIuAXwCZisB9mthi4BtgP+G26LAW+ksqKiMhOasZ2jkVkh3YU8Fl3f19lg5mdR3SYv2JmP3X3TaPUsRS4DTja3bcU9n2S6Bif4+5n1DhHw8xspOkoDh5LPSIiMj3M2M6xDcVdm9Uyp7ptaEtEVAd6Iv+2rT3LKukj8oIX7hHzTM+fNTvbt/amOL4cUdtSWxYdLg9GlLeUpoeb25ZFnFsqOccpgtzSkj3clanYSp6b+zjNp9w/FOcZyk1NbJU5jNO2smdR5f7K/MtpLrf+gSwnuiNFmGeV4nHYNJBFnAf7srxlkWldut38AAAgAElEQVRmI/Dv+Q3u/kcz+zbweuBlwDcaqOc9xY6xmbUBrwE2A2fWOYeIiOyElHMsItPRDe6+ucb2Fen6yQ3U0QvcVGP7wcAs4MY0oG+kczTE3Q+rdQHuGEs9IiIyPahzLCLT0eoRtj+cruc3UMcjXnMpyuqxo51DRER2QjM2raKjM9IiFi7ZrbptoC+lHwxEqmJLa1d135z5ywCYuyTSBHs2ZgGl7r5IRaikQsyakz1sXV2RVjGYVrrr7MjSFlpb0yp4HksfdrR3VPdVFmz2liytokyaYq4yWG8oS48Y6o/6u9pnRfsGs33dPTHIrjwU5+7ry1bI607Bt85KykZ/1lfwsr4bybS12wjbd0/XjUzfNtKi7pVjRzuHiIjshGZs51hEdmhPMbO5NVIrjknXfxpH3XcAW4Enmdn8GqkVx2x7yPY5dM/5XK9FB0REdigztnNcStO1lTqyKGpHmvKMthiIVrZs0N3C3faJMp3zALjjptur+26+9U4Alu4ekd3Fe2VTn1lHRG1npaiwl7OHtFyKc7e2RWMGB7dW97W0punacsGtwcE0EK8cg+1KuX2WFg0Z6I/jyoPZlGwMxii93i1x3Nat3dl5WtLAvc6Ikre2Z79GWyl7bESmmfnAvwH52SqeSgyk20isjLdd3H0gDbp7EzEgLz9bReUcIiKyk5qxnWMR2aH9BnijmR0BXE02z3EJeHMD07iN5kPAc4DTU4e4Ms/xycBPgJeOs34REdlBKelURKaje4FnAOuBtwAnATcAL2pwAZC63H0N8Ezg68TsFacDTwLeCpw93vpFRGTHNWMjx26RduD0VbfNnh2D2foHYsW6lvY9qvta22Ie4FUr7wfguuuyef3vf2gNAG3z9gfgoQ3Z3MmDAzGFakdnpGx0b87O19sX+xYtjrSKXXfN5kDutCg32J+lWlhKlWhPAwV7erPUCR+Iv3u3VI7L7UsDBct9sa2jlKVjdHVFXZ2z5sY5ylnbDc1zLNOLu68EcmtXcvwo5S8ELqyxfVkD53oYeMMIu22E7SIiMsMpciwiIiIikszYyHFrWwR+2juzqdXa04p1G7pjkFpXRy6SOysG4v3ipz8H4OZbsgF5LXMi+jrQsgsAj3TPre7r7Y3vFwOPRAS4rzcLOG3ZGgPeeu56EIBdd8nasv++UdeShbtm7UuR3IGBNFAutwpeX1+sytc/EFHhodyUbC3EgDwrx3Gdne3VfbPTlHbWEtuGBvKzW+m7kYiIiEieekciIiIiIsmMjRzj0e83z/JqW9tiurWerZEL/MiGe6r7Vm+Kh+KPN/4ZgDUbsunQHrMkrQnQGhHjNZuyiG7ZIxrcs7kS7c0e0qFSlO9JU7jddU9Pdd+jj64FIBfkhcEUhe6PqV3nL8h2dqap3GalAxZ2ZVHvrlIsCOIW12257zzdWyMPebAc+zb1Z/u6Fo+0RoKIiIjIzkmRYxERERGRRJ1jEREREZFkxqZV9PZEmsOm9Vuq27wvBtbNnRWpEG25wXrX3PRHAG7+a6yGN3vWwuq++fPi73Xr1wPQMdBf3dfX35PqjtSJyjRskE3lVhlMV8qlXPQPxLk3bc2mftuwIZUrRQpFx8bcKnhDkeYxK7V517lZysXei6N81+xIkyi3ZCvfbdqwDgBLg/Z6Lfs+tGRIK+SJiIiI5ClyLCIiIiKSzNjIsQ2lRTO611S39W6MyG9X10YA2uZk06jd87eHAXh03QYAZnUtqu57+JFH4o+1EYWdP39e7kwR+V0wPxbX8Fwwtrs7IsflNMVaS278W2vaVrbsX9DWFd9VWlPkuL83i3p3b4rzDJSj/MBgFr3e0hMR58fuE9O2tVgWcW5P9ZfSgL5ybuGPEhqQJyIiIpKnyLGIiIiISDJjI8ftrRGZ7WjJpk/rT9HkltaYBu32ux6q7vv9n+4GoFSKJaY3b+6t7tvywCoALEVhF83dWN03b3bkAHe0xb729mwRkM2bIg85BYmrS0wDlHsG0/my7yctFpFc82hnORcd3twdOceDQ5E7vKUry21e+UDcj0VzdgNg6T5ZvnRryq8esGjEwEB2PvMhRERERCSjyLGIiIiISKLOsYiIiIhIMmPTKlpbI52gpSXr//enKdjKfTGg7ldX3Vrdt3FzpDmUh6L8hk3ZFGsdKb2hJT1aG8hWyKNlPgC7l2Ln4EC2r7s70ioG+iN9ob0993Cn7Ith7euP85TSoLmWUjZ4bsuWSIt49NFHo+xglhLR1Zqmket4DADekrVhMOV0WCmmfiu1ZHW6ZSkgIiIiIqLIsYjsYMxspZmtnOp2iIjIzDRjI8eVoKjnZivr7OgAYM3miNDe98CjWfkUYZ03N6ZkG+zJoq/zOiIK7USEtrc321dujengenpiEN1DD66q7tuUBuS5W7rOGjM4GOX7+7Np1wbSthaL882aNae6rzIt3ObNm1MbssF6++4VU7gt2W0xAEMt2YBBKtO1pTnmSm3Zv9xaWxARERGRzIztHIuITLVbHtzIsg9cNtXNYOWnjpvqJoiI7DCUViEiIiIikszYyPFAGtw2MJClLcyaFXMYzylF2sKypXOr+xbOjwFuC5fEHME33XBHdd+xRxwJwNI9YvDdd3/2++q+waFIW3j00Vg9b82addV9GzdGKoSX4zvIwGBuoFwauDc4lG3zctRVSiP/Suu7q/uG0rFDaZ7jcm4lvo72uD9z58f9K3m2st5Amq7ZB+MAz2VSlE0r5Mn0ZGYGvB14K7A/sBa4FPjwCOU7gDOAVwMHAIPAn4Fz3f07I9T/TuDNwH6F+v8M4O7LmnmfRERkxzBjO8ciskM7h+i8PgScDwwAxwNHAO1ANenezNqBnwNHA3cAXwJmAScCl5jZk9z9Q4X6v0R0vFel+vuBlwJPA9rS+RpiZtePsOvgRusQEZHpY8Z2jisrz7Xmpi5ra4vpzHafEyvknfjCp1T3zZoTUdeNWyLye/AeWcbJkU+NKdJ22y2mgLvt7pXVfb+56WEAejrjoVy7dn1139BgZaq0aMNAfy5ynCLAQ0NZCLhSerCcppXLPv+rAwsrAw3zk7AtWBgD91rS6nxbNmaf6/09aXBfh6U6s2hxWYFjmYbM7BlEx/hu4Gnuvi5t/zBwJbAU+FvukPcQHeOfAi9198FU/izgOuCDZvZjd78mbX8W0TG+EzjC3Tek7R8CLgf2KNQvIiI7EeUci8h0c2q6/nilYwzg7r3AB2uUfwMxLcu7Kx3jVP4R4GPp5htz5V+fq39Drnz/CPXX5e6H1boQUWwREdnBzNjIcWVxjVJHllfs6d62eEyxtu+endV9KajM3ksjr3j5/rtU97Wm3NzywBoADtl3UXXfNdfdCcDq9ZHc29+bRW3nLIxyg72xrzyU7bP00Jcs+37iXokix+d7W0sWH+7wiDT3pSKDucVDFs6N+2FpKrhSbpGStraY5s3LaQEUcpFqLQIi01PlJ51f19h3FWRPcDObS+QYP+jutTqjv0rXT85tq/z92xrlr83XLyIiOx9FjkVkupmfrlcXd7j7EDF4rlj2oRHqqmxfsJ31i4jITkadYxGZbiqr2OxW3GFmLcDiGmV3H6GupYVyAJvGUL+IiOxkZmxaRWU1uo4Fe1W3lVojLWJo44Z0OxuR5haD5iqZBq0tQ7m6YmNXezxcTz4o+xx+ygHxOfvbm1cC0FLKUhW6+2I1uz3nx8p8rQs6qvvWbewDoK8nOw8Wfy/ZLYJcnV3zq7v2XRTpIZXBgI8OZOkRT3j8IVH/UJx7TmfWhr7++Ls0FPevPz8AMD8fnMj0cQORWnE0cE9h37PIvW+5+2YzuxvYz8wOdPe/Fsofm6uz4k9EasXf1aj/6TTxffHQPedzvRbgEBHZoShyLCLTzYXp+sNmVk3wN7NO4JM1yl9ATODymRT5rZTfBfhIrkzFN3P1z8+Vbwc+Me7Wi4jIDm0GR44rd60rt7Ez7at8fmYR1srCG63tsc9L2ZicclpAoxJVXrJkTnXfIYfsC8A1dzwIwIH7ZpHq9d0RVT7hOTH+58lPWFLd98DDjwLw8KosvbFUikj2nDnR5jsfziK7Byw7AICDHh+/+P7htizg1TU/Bg/2t8SAvNZyNtCw1aKOto6IWvf05lYB0VRuMg25+9Vmdi5wGnCLmX2PbJ7j9WybX/xZ4IVp/5/N7CfEPMevBHYFPu3uv83V/2szOx/4Z+BWM/t+qv8lRPrFKkA/q4iI7KQUORaR6ehdROd4I7GK3auIhT6eS24BEKhOwfY8stXzTiOma/sr8Gp3f3+N+t8KvBvoBt5CrKx3eapnHllesoiI7GRmbOTYiMUv2lpmZxtLPQD0pynZyp5NrdbeFZHVvrRQR0tbbrGMwcgF3twXx1t7tq+tI0WVU0D20EP3ru7bZXGMBTpgSRy358Kt1X27zovo88A+2VRzlfYMpPnalux/YHXflsH4V+2zS/QL2tqytg+lCHj7/FispPuRB6r7Wvsit3kwLfg15FlUWVO5yXTlMWjgvHQpWlajfC+REtFQWoTHvIlnp0uVmR0IzAFuH1uLRURkplDkWER2Oma2u5mVCttmEctWA1w6+a0SEZHpYMZGjkVE6jgdeJWZrSBymHcHngPsRSxD/d2pa5qIiEylGd85HhjMUiA6OmIZvFJrXA9mK80ykKY4GxyKtAXryy2SlaZI8zR9WltLbhW82fEQ7r5LpG/st8+86r6DHntolBmIX2iH+rJxRFu747qllKU5tHdEIGsoTbu2z977V/fdu+qRaOfGmB5u711mVfeV56TBhB0xsL9vKDcgbyhSOnpSekVpdrbyX6mkHw5kp/VL4InA84FFxKp4dwJfBM7xylyQIiKy05nxnWMRkSJ3vwK4YqrbISIi08+M7RwPDGxJ173Vbe3tEfkdSjGhIc8ipz19acBaOa5LllsgpDceJhuK8taR7SulqdKefHAMxDv4gGxxrV2WRCS399EY0FfO15mmjusfykW2W2Obt8fgwIH+nuq+1p6IHHekQYWViDXAxp416a/lAMxdkC1S4qtj0H0pLU5ipdxUbhqPJyIiIjKMflcXEREREUnUORYRERERSWZsWkWZGFA3mNIrAAb7I72hLw1OGxjKUi7K5XgoBgfTPMKt2UPjvXFcCyklIct2oLMtth1x2MEA2MDm6r4Na1dFXS1pdbrcGD86Iqdh65Zs7uP+njQYsC3mQO7dlM1XPK81VtQrtUeb20rt1X1zWqNBWzZFma7ObG7njWmO5sGyp3PkHo/BbGChiIiIiChyLCIiIiJSNWMjx4NpAFp/T7YKbEcpDXgjoqk+WK7uqwyW83J8XyiXs9Fqg4MR8vWWFDnuzR42K0fkt6UzotFDPVlYefO6GwFYuOeyOL5/QXVff986AHoGcivd9Ue9Le3p3IOrq/u6hqL8wEAawFfOBvL1Dm0EoNQTkea2Ult1X3eakq6UIuGbK3PIAUPlfChbRERERBQ5FhERERFJZmzk2NM8ZZ0dHdVt7a3xXWB9X4ru+lB131B1SrU4bnAoiyqTotBuKeJMts885QmnXGUrZ3V2WuQHr18dEd2FXdnCHd7bFX8M5HKbh1JbPaLJg31rq/sGLCLT/UMRFd7am51nYCj2dczaEPezlOUcD3m0vWdrtG+ILKpsWgREREREZBj1jkREREREEnWORWRaMbOVZrZyqtshIiI7pxmbVmEWd629vau6ra8vBqMNpunNOjuy1eIq21pa4vtCa1s2VVolw8KJ1Iu+cjbobtbsSFMop+Pd+qv7ujqjrqGBGBTYtyU3AG4o0h068mkOHicqDaxPdWbTrrV2dsa2NM3bUFtWVwfpnC1RZ19vNj1cR5rWrbs7Ui/a5nRW95VM341ERERE8mZs51hEZKrd8uBGln3gsiltw8pPHTel5xcR2dHM2M5xOQ2wGxjIBs8N9aVBbB532yyL2ralPz2Ny/NcRHdgKI7r6EjbhrLp10otUf/A1jSlW1cWje7qiqh1uSUiu4O5RUd6+yP63NWZRbY7u+LkliLAPVuyf8+sWfMAWLhobwA2bliXHZfqn7NoSTpPNs1badHiOG7B7gCs2ZK1PSslIiIiIqCcYxGZAhbeYWa3mlmvmT1oZueZ2fw6x7zKzK40s/XpmNvN7F/NrGOE8geb2YVmdr+Z9ZnZajO7yMwOqlH2QjNzM9vPzE4zs5vMrMfMVjTxbouIyA5gxkaOB1PEeO2abDq0zrbYZikq3EIWtR1KicXlFCUeHMwWARmiPZWJqHCpNYsOt1nk/ralaeJaWrKIc7kUn9mlUorRlrI6W1JUuT0XOe7qTJHtUmqnZVO/WVqcpCuFuIdmZ/u6N0ROc9vWuC5b1j5PU78N9EedfT1Z5NiUcyxT5xzgncBDwPnAAHA8cATQDvTnC5vZ14A3AA8APwA2AE8HPgY8x8ye5+6DufJ/n8q1Af8H3AXsBbwcOM7MjnX3G2q06wvAs4DLgJ8AQzXKiIjIDDZjO8ciMj2Z2TOIjvHdwNPcfV3a/mHgSmAp8Ldc+VOIjvGlwGvcvSe370zgo8DbiY4tZrYQ+F9gK3CUu9+WK38I8Hvgq8BTajTvKcCT3f3eMdyf60fYdXCjdYiIyPSh0KGITLZT0/XHKx1jAHfvBT5Yo/y7gEHgDfmOcfIxYC3wmty21wELgI/mO8bpHLcC/w082cweV+Ncnx5Lx1hERGaeGRs57koD2Nauz6ZDa0+D52Z1xXRmpb4sVbEylVu5nAbY5Uarldqj3GBvmmqtJRvkV0qrzLWnVIvNWdYCfVZJ34jj+0vZcd4W5Qdy/4JSOaZbK3sM3Ost5wb+pZX0Nt//UByfZU4w0JfOtybSKmjNpqEbSMf1bUp1t2ZTubW3ZX+LTKJKxPbXNfZdRXSEAbDILXoisAY43cxqHEIfsDx3+8h0/cQUWS56bLpeDtxW2HddvYbX4u6H1dqeIsq1otMiIjKNzdjOsYhMW5VBd6uLO9x9yMzW5jYtJNZ0X0KkTzRicbp+0yjl5tTY9nCD5xARkRlqxnaOd93rCQAceHgWYu3dGlHklpZK9CkXhUpzuFUiU+X8PGfVaFXaaPmdPqymllydpVI8vKW0rSV3OqtktHhWl1saiJcizB2zchFq4n60lqsbcnUNn5TNSrkBeZ5OmoYVWWv2L++atwCRKbAxXe8G3JPfYWYtROf2wULZP7l7o1HYyjFPdPebxtg2zXAoIrKTm7GdYxGZtm4g0g2OptA5JmaKqL4vuXu3md0KHGJmi/I5ynVcC7wi1TXWznFTHbrnfK7XIhwiIjsUDcgTkcl2Ybr+sJktqmw0s07gkzXKf56Y3u0CM9vm5w4zW2hm+ajy14mp3j5qZk+rUb5kZsdsf/NFRGQmm7GR485Z8Zm730HZZ2Zl0B01x/SMU+XHWKuxrYlqjUfyMZynlJtruaOj5toJIhPK3a82s3OB04BbzOx7ZPMcryfmPs6Xv8DMDgPeBtxtZj8H7gMWAfsCRxEd4rek8mvN7ERi6rdrzewK4FagDOxNDNhbDGhEqoiIbGPGdo5FZFp7F3AnMT/xm4np2C4FPgT8uVjY3d9uZj8lOsDPJaZqW0d0kj8DfKtQ/gozewLwXuAFRIpFP7AK+BXw/Qm5V8Mtu/322znssJqTWYiIyChuv/12gGWTfV7zsYQdRUSkIWbWB7RQo7MvMk1UFqq5Y0pbITKyJwJD7j6pP3UrciwiMjFugZHnQRaZapXVHfUclemqzgqkE0oD8kREREREEnWORUREREQSdY5FRERERBJ1jkVEREREEnWORUREREQSTeUmIiIiIpIociwiIiIikqhzLCIiIiKSqHMsIiIiIpKocywiIiIikqhzLCIiIiKSqHMsIiIiIpKocywiIiIikqhzLCIiIiKSqHMsItIAM9vLzC4ws1Vm1mdmK83sHDNbOMZ6FqXjVqZ6VqV695qotsvOoRnPUTNbYWZe59I5kfdBZi4zO9HMzjWzq8xsU3o+fWs762rK+/FIWptRiYjITGZm+wPXALsCPwLuAJ4GvAv4ezN7pruvbaCexamexwK/Ai4GDgZOBY4zsyPd/Z6JuRcykzXrOZpz1gjbB8fVUNmZ/SvwRKAbeIB47xuzCXiub0OdYxGR0f0n8Ub8Tnc/t7LRzD4PnAF8HHhLA/V8gugYn+3u787V807gC+k8f9/EdsvOo1nPUQDc/cxmN1B2emcQneK7gKOBK7eznqY+12sxdx/P8SIiM5qZ7QfcDawE9nf3cm7fXOAhwIBd3X1LnXpmA48CZWCpu2/O7SulcyxL51D0WBrWrOdoKr8CONrdbcIaLDs9MzuG6Bx/293/cQzHNe25Xo9yjkVE6nt2uv5F/o0YIHVwrwZmAU8fpZ4jgS7g6nzHONVTBn6Rbh477hbLzqZZz9EqMzvZzD5gZu82sxeaWUfzmiuy3Zr+XK9FnWMRkfoOStd3jrD/r+n6sZNUj0jRRDy3LgY+CXwO+Alwn5mduH3NE2maSXkfVedYRKS++el64wj7K9sXTFI9IkXNfG79CHgJsBfxS8fBRCd5AXCJmb1wHO0UGa9JeR/VgDwRkfGp5GaOdwBHs+oRKWr4ueXuZxc2/QX4kJmtAs4lBpX+tLnNE2mapryPKnIsIlJfJRIxf4T98wrlJroekaLJeG59lZjG7Ulp4JPIVJiU91F1jkVE6vtLuh4ph+3AdD1SDlyz6xEpmvDnlrv3ApWBpLO3tx6RcZqU91F1jkVE6qvMxfn8NOVaVYqgPRPoAa4dpZ5rU7lnFiNvqd7nF84n0qhmPUdHZGYHAQuJDvKa7a1HZJwm/LkO6hyLiNTl7ncT06wtA95e2H0WEUX7Zn5OTTM72MyGrf7k7t3A/6TyZxbqeUeq/+ea41jGqlnPUTPbz8z2LNZvZrsAX083L3Z3rZInE8rM2tJzdP/89u15rm/X+bUIiIhIfTWWK70dOIKYk/hO4Bn55UrNzAGKCynUWD76OmA5cDzwSKrn7om+PzLzNOM5amanELnFvyYWWlgH7A28iMjx/CPwPHffMPH3SGYaMzsBOCHd3B14AXAPcFXatsbd35vKLgPuBf7m7ssK9Yzpub5dbVXnWERkdGb2GODfieWdFxMrMf0QOMvd1xXK1uwcp32LgI8SHxJLgbXE6P9/c/cHJvI+yMw23ueomT0eeA9wGLAHMbhpM3Ar8B3gv9y9f+LvicxEZnYm8d43kmpHuF7nOO1v+Lm+XW1V51hEREREJCjnWEREREQkUedYRERERCRR53gGMrMVZuZpcMVYjz0lHbuimfWKiIiI7Ahm9PLRZnY6sb72he6+coqbIyIiIiLT3IzuHAOnA/sAK4CVU9qSHcdGYgWa+6a6ISIiIiKTbaZ3jmWM3P1S4NKpboeIiIjIVFDOsYiIiIhIMmmdYzNbZGavN7Pvm9kdZrbZzLaY2W1m9nkz26PGMcekAWAr69S7zQAyMzszTXC+T9p0ZSrjdQab7W9m/2Vm95hZr5mtN7PfmNkbzaxlhHNXB6iZ2Twz+7SZ3W1mPamefzezzlz555jZz81sTbrvvzGzZ43yuI25XYXjF5rZ2bnjHzCz881saaOPZ6PMrGRmrzWzX5rZo2bWb2arzOwSMztirPWJiIiITLbJTKv4ELHyTsUmoItYOnU58I9m9lx3v6kJ5+oGVgNLiC8A64H8qj7FlYJeDHwXqHRkNxLrcz8rXU42sxPqrNW9EPg9cDCwBWgB9gU+AjwJeKmZvQ04D/DUvlmp7svN7NnufnWx0ia0azHwB2B/oAcYBPYE3gScYGZHu/vtIxw7JmY2F/gB8Ny0yYmVlZYCJwEnmtm73P28ZpxPREREZCJMZlrFg8CngKcAc919PtABPBX4OdGRvcjMtlludazc/bPuvjtwf9r0cnffPXd5eaVsWqP7YqID+mvgYHdfAMwF3gz0ER2+L9Q55UcBA57l7nOAOUQHdBB4iZl9BDgn3f/F6b4vA34HtANnFytsUrs+ksq/BJiT2nYMsSTjEuC7ZtZW5/ix+GZqz03AccDsdD8XEl+MBoEvmNkzm3Q+ERERkaabtM6xu5/t7h909z+5e3faNuTu1wPHA7cBhwBHTVabkg8R0di7gRe5+19S2/rc/XzgnancG8zsgBHqmA282N1/m47td/evEh1GiPW/v+XuH3L3DanM34BXERHWw81s7wlo1zzgRHf/sbuX0/G/Bl5IRNIPAU4e5fEZlZk9FziBmBHkWHf/ibv3pPNtcPdPEh31EvDB8Z5PREREZKJMiwF57t4H/DLdnLTIYopSvyLdPNvdt9Yo9lUi6m3AiSNU9V13v6vG9stzf3+yuDN1kCvHHToB7brK3a+qcd6/AN9LN0c6dixen64vdPd1I5S5KF0f20iutIiIiMhUmNTOsZkdbGbnmdlNZrbJzMqVQXLAu1KxbQbmTaD9gPnp7ytrFUgR1xXp5lNGqOfmEbY/kq57yTrBRavT9cIJaNeKEbZDpGrUO3YsnpGuzzCzh2tdgD+mMrOIXGgRERGRaWfSBuSZ2T8QaQaVHNcyMcCsL92eQ6QRzJ6sNhF5txUP1in3QI3yeQ+NsH0oXa92dx+lTD73t1ntqndsZd9Ix45FZeaL+WSd+npmNeGcIiIiIk03KZFjM1sC/DfRAbyEGITX6e4LK4PkyAaljXtA3nbqmKLzjmai2tXMx7nyPDre3a2By8omnltERESkaSYrreKFRGT4NuDV7n69uw8UyuxW47jBdN1ZY19FI5HKkTya+3ufEUvBXjXKT6Rmtateikol2tuM+1RJDXlcE+oSERERmTKT1TmudOJuqsyakJcGoD27xnEb0vWuZtY+Qt2H1zlv5VwjRUnvyZ3j2FoFzKxETH8GcEOdczVTs9p1dJ1zVPY14z79Ll2/om4pERERkWlusjrHG9P1oSPMY/wmYqGKojuJnGQj5uodJk1hVq9Dtv/3RHIAACAASURBVCldL6i1M+UB/yDdfJeZ1cqFfSOxcIaTzfAwoZrYrqPN7BnFjWZ2INksFd8dZ3MBLkzXTzWz19UraGYL6+0XERERmUqT1Tm+nOjEHQp80cwWAKQll98HfAlYWzzI3fuBH6WbZ5vZ36Uliktm9nxi+reeOue9NV2/Kr+Mc8EniFXt9gAuM7ODUts6zOxNwBdTua+NMF3bRGlGuzYBPzCzF1W+lKTlqn9K5DLfCnxnvA1195+RdeYvMLOz8stTpyWsjzezHwGfH+/5RERERCbKpHSO07y656Sb7wDWm9k6YhnnTwNXAF8Z4fAPEh3nxwBXEUsSbyFW1dsAnFnn1F9L168ENprZ/Wa20swuzrXtbmIxjl4iTeEOM1ufznM+0Ym8Aji98Xs8fk1q18eIpaovA7aY2WbgN0SU/lHgpBq539vrdcAPiaWz/w1YZWYbzGwj8X/+IfDSJp1LREREZEJM5gp57wb+GfgTkSrRCtxIdO6OIxt8VzzuHuAI4H+JDl0LMYXZx4kFQzbVOi4d+yvgZcScvj1EGsI+wO6Fcv8HPJ6YUWMlMdXYVuC3qc0vcPctY77T49SEdq0lcrLPIQbNtQOrUn1PcvfbmtjWLe7+MuDFRBT5QaArnfMuYhGQE4G3NeucIiIiIs1mI0+/KyIiIiKyc5kWy0eLiIiIiEwH6hyLiIiIiCTqHIuIiIiIJOoci4iIiIgk6hyLiIiIiCTqHIuIiIiIJOoci4iIiIgk6hyLiIiIiCTqHIuIiIiIJK1T3QARkZnIzO4F5hFLv4uIyNgtAza5+76TedIZ2zk+/h+e7ACz5syqbtt9r/kA9PeUASgPDFX3tbW2ALD20W4AFi9cVN3Xu3ULAAsXdAGwz/6Lq/vKNgCAWTsAf7n7vuq+7i2xNPfj9js4NrRkgXrzvlRmc3XbvQ9uAOCBVVsBKLV2VvfNTvejq8MA2LRpY3ZnUxvKRJ1z5nRVd23Z2AtAz+aoc+kec6r7tm7uB+DSi24wRKTZ5nV1dS1avnz5otGLiohI0e23305PT8+kn3fGdo7nds0GYPas2dVtvZuj8/jg3x4GYP6cebkjoiPbQgcAJc/2dLTGw1TpmA719Vf3rd2wPuruGUwnyY7bdcFuUd7bAPjzTX+t7tt9yQIAZnW0V7dVqu2aG51bt6zP2tKWzj0UDXPPGtjWGvWbRwd/oDvr9HdviAb1bo0O9ODW7LjNa7sRkQmzcvny5Yuuv/76qW6HiMgO6bDDDuOGG25YOdnnVc6xiExLZuZmtmIM5Y9Jx5xZ2L7CzHyEw0RERIZR51hkhhhrZ1JERES2NWPTKqwcecVbN22pbiu1RrpBq8fd9v58akKkLcyaE3m+fVu2VvcNDaaUifmR7rBxbbavRKQ0VFIvWmZ3VPcNpn1rUhvKnqVQbNwQaQ5bsk10dkQu8+BgpDv0DfZldfXF/RksRztnz9o2JQSPMt3r11X39HdH23s2RM7G2pYslaKrM8vHFpkBrgOWA2umuiEVtzy4kWUfuGyqmyEiMiVWfuq4qW7CdpmxnWMR2bm4+1bgjqluh4iI7NhmbOe4b2sMRJszLxuQ194aUd1yGlg3OFCu7mtticFsQwMRae3dmo2sq4x9W7s6oq6eS19s6YxIc2tHzAIx5FnkeH0aANhbjihxR+v86r5yGjO3cUMWHS6X499x/30R+V27IQuAtbSkCHV7tPOAxy6r7lu8OAb3rV8X5bf2ZfeLFGnu6IjjF87J2rBocfbYyMQzs1OAlwBPBpYCA8DNwJfd/VuFsisB3H1ZjXrOBD4KHOvuK1K9X0+7jy7k157l7mfmjj0JeAfwRKAduAu4CPi8u/fljqu2ATgU+BhwIrAL8BfgTHf/oZm1Av8CnAo8BngQONvdz6vR7hLwz8A/ERFeA24DLgD+y93LxWPScXsA/wG8AJibjvmcu19UKHcMcGXxPtdjZi8A3gU8LdX9APAD4OPuvqGROkREZGaZsZ1jkWnoy0TH7jfAQ8Bi4EXA/5jZQe7+ke2s90bgLKLD/Dfgwty+FZU/zOwT/5+9O4+zsyzvP/65zjJrlskKgQABBMENFDdcY12ptdrW1rpVtLW12rrWn2j9VeiitlqX2qpt1doqFm2tWrfiryou4AoighEQE0gCScg2k8x2tuv3x3Wf8zw5OTOZJDOZ5Mz3/XrN68w89/Pcz/0Mh8k911z3dQNvJNIOPgHsBy4B3go81cye7O7Vtr7LwP8DlgOfIybUzwU+bWZPAV4OPAL4MjAJ/CbwPjO7190/2dbXx4DnAZuBDxH5QL8GvB94DPD8Ds+2DLgO2Ev8AjAE/BZwpZmd6u7vOOR3Zwpm9mfE92038AVgB/Ag4E+AXzazi919ZAb9TFWO4rwjHZuIiMyfrp0cW8rDHezNyqEV0rFKIdYh1snaRlOps8pEyu2dzMqhDS6N6HC5HFHhPSNZzb39w9Hn/skIuu3du73VNj4RxyZTiTXL3a+Wzp8YzXKACyk6XE+n7cvnS1dTnbd6jG+yWToOeODD7pueLy4s1LP/rOVUK3nVssUAnHvmmlbb4kW5hGc5Fh7g7nfkD1gUyP4ycJmZfdDdtx5up+5+I3Cjmb0F2NQpampmFxMT483Aw919Wzr+RuAzwK8ArycmynmnADcA65uRZTP7GDHB/w/gjvRce1Pbu4jUhsuA1uTYzJ5LTIx/BDzO3fen428GvgE8z8y+2B4NJiar/wH8djOybGZvB64H/srMPu3uvzi87xiY2ROIifF3gF/OR4lzkfgrgNccbt8iInJiU7UKkWOkfWKcjlWAfyB+UX3iHN7+Jen1L5sT43T/GvA6oAH83hTXvjqfcuHu3wI2ElHdN+Qnlmmiei3wQDMrdrj/Zc2JcTp/FHhD+rLT/evpHo3cNRuBvyOi2i+c8omn98r0+tL29Al3/ygRje8UyT6Iu1/U6QPlP4uInJC6NnIscrwxs9OJieATgdOB/rZTTp3D2z8kvX6tvcHdbzOzLcCZZjbUNlnc22lSD9wNnElEcNttBYrAyenz5v0b5NI8cr5BTIIf3KHtrjQZbncNkUbS6ZqZuJjI+f5NM/vNDu09wCozW+Huu47wHiIicgLq2snxqtXLAOix3KK7tOPc4jVRBm1XriTb8L5Ubi2lXtQL5VbbrntjoVt1PM6v2rJW289+EfOIzVtjp7xGI+tz7emnA3DeA2Ne4uUsjWHbls0A3H7Lza1jg+WYKxXKEdBfszIbw5JSHNsyGgsFJ4vZvOrWm6Ov+6w7GYCBYq6c3FDc8+ShKNs2NJiN4ZTB9rmZzBUzO4soNbYM+BbwFWCYmBSuA14E9E51/SxorsS8Z4r2e4gJ+1Iiv7dpuPPp1ADcvVN7M+ennDu2FNidIuUHcPeame0EVnfoa3uHYwDN6PfSKdoPZQXx8+8thzhvEaDJsYjIAtK1k2OR48xriQnZi9Of7VtSPu6L2s5vENHLToaO4P7NSezJRJ5wuzVt5822YWC5mZXbF/2lihcrgU6L306aor+Tc/0e6XgK7r78CK8XEZEu1bWT44H+CMJN7M0WvFXTYraR4R0AlHqyTTAKjYgY9xLnXOBZOvZEPeYod1bSor2ebGHdnuH077nFAr5CIbtu984IOO0bSZt6kKVg7k8R4MWLs808Iv0TxkYjCj2Yi/KW0rXLB+P1lPtmC+Hv3hhBtJGJ+M+5bvWKVtuplYiIn9kfUWIrZ+NbuXguA5XS5j7p9dMd2h7f4dge4EGdJpPAQ6e4RwMoTtH2IyK1YT1tk2Mzuw+wFtg4h+XLfkSkkzwO+Gpb2+OIcd/Q4brTzWydu29qO74+1++R+C7wdDO7v7vfcoR9HNIDTl3K9SdoEXwRkYVKC/JEjo1N6XV9/mCqs9tpIdr3iV9eX9x2/qXAo6e4xy6i1nAnH0mvbzazVbn+isA7iZ8FH55q8LOgef+3mVnrt9L0+dvTl53uXwT+OtVIbl5zJrGgrgZ8vMM1M/Hu9PrPqY7yAcxs0MweeYR9i4jICaxrI8cix5n3ExPd/zCzTxML1R4APA34FPCctvPfl87/gJk9kSjBdgHwKKIm7690uMdXgd82s88TC+VqwDfd/Zvufp2Z/Q2xYcfNZvafwChR5/gBwLeBI64ZfCju/gkzeyZRo/gWM/ssUef4WcTCvk+5+5UdLr2JqKN8vZl9hcgxfg6RWvJ/plgsOJPxfNXMLgPeBtxuZl8iKnAsAs4govnfJv77iIjIAtK1k+Ph3ZHuMDaS7XRXHU9/nfZIgejtz9YG9fVHHeChfZEC8ag7WtWu2D4ZQauNJ0eq5zbL+hyfiL5q9ViIV7Qs5aKR/hp+1523ATDQl60dGtsbu+BRyxbwjaYUDU+LAndVsw3LSksi/WLJigj6VXdmqZYPHIy27Xti7A968AWttodddy0Au38SVaV6npdVC1t2SpZ+IXPL3W9KtXX/ktj4owT8GPh1YgHcc9rO/6mZPYmoO/wMYqL7LaLKwq/TeXL8KmLC+cR0jwJRq/ebqc83mNmPiB3yfodYMHcH8GZix7mDFsvNsucSlSleAvxBOrYB+Ftig5RO9hAT+L8hfllYQmyk8s4ONZEPi7v/tZldS0ShHwM8k8hF3gr8E7FRioiILDBdOzkWOd64+3XAL03RbO0H3P3bRD5uu5uAyzucv4PYaGO6MVwFXHWosaZz103Ttn6atkuBSzscbxAR9PfP8P7578kLZnD+NXT+Pq6f5ppvExFiERERoIsnxyPDKSLr2fqkelqrVJmIiG6xmJV5W7lqEQBn7t0HwIpqtgteqRCl29aOxvUjlgXY7tcbC916+mIx3aK+bBHd0r5o88mICJ+9vJXqSf+SswEY25/tgseqiBjX6jG+XWP3tprGU3WsbbtjvVR1IqvIVShFxazetKlffec5rbaxcvwn3pbufUpfFi3eWctX2hIRERERLcgTEREREUm6NnK8duVKAAqWPWI9lXLbuSs29Vi+7ORW28Pu91gAHrT1agBOrW1qtS07Jf5S+9LeiApXcnsVFFZENLlciKhvMdvlluZfeEv1yGfuGczut3gw8o+LS7K8Yk9DLaYId2Myy20erkWEeTj1v69Wa7VVqhEyHi9HxHmo58JW24oLYwOxdTti3dLo3dlfnTf2LEZEREREMooci4iIiIgkmhyLiIiIiCRdm1bx3F+9BIDR0WzBW6UaqQj37o6d6yYqWWm1lcvuF+c0vgXA3mq26K5SifNPmYy+ejxLhRhrREpDJZWCWzJ4Rqut1H8qAA1rrpTL0iQKO2NhXX3f9taxyckoxbalGqkTo9m+B5xUjN3sTh2KhXWFpStbbb44SswVyvE8E5PZdYNbfgrA/tuvieeazBYaVh79BEREREQko8ixiIiIiEjStZHj1UNRRm0PWaS0vz82yzhlVZQzqzRObbXdOxw72t7VG6+FarbgbcuOiCLfU4hj69KGHACfX7MOgLrHgryHnpqVcjt1W2y8sWpXRIkLlSxyXK9HX8XcJiCVVPJt874oJ7fLs+epFOI/1Yp9ESUu7swW03mhL71GdLk2kJWv25JKy31zSSzEu2VHvdV2xvU/iDEjIiIiIqDIsYiIiIhIS9dGjvfti+2VR0dHWscmJyJnuFBcDkApF2GlHGHafSlCW2hkJdkaxGYZ10/EsVXnDLTaeh8ROcb1gZS//OTnttr+85P/BcBJ//s/AFyyf2urbSBFkeu50m8DaXORi/tiXOOFRdnw6pHnXBiPjUFqkztbbZZKxnn6z3n1YJaPPPbiP4xxronx2Zc/2Wr7wW13AfD7iIiIiAgociwiIiIi0qLJsYiIiIhI0rVpFbdujkVwnkuPqNVicV7DYzFcvZCVaxsvxuK8ctpZz4eyBW9Ly5FWMTEUaQ6lPXtbbQ+65WYAdqxcC8DalM4BcMaZ5wPw45NvAOAJ41laxaLJSJOwerZArpEW9ZUakSaxaSD73WVRLY6tm4zz3bLVes3PxsrxXD+p7Wu1lbffCsD9UkW7wq0/yZ55PyIiIiKSo8ixiCw4ZrbOzNzMPjrfYxERkeNL10aOv7/h7vgkt+BtciIW5w32xYK6umUbcCw7LSLFg/1RFm3/qsFW20Qjvk3D1Xj17Xe32s7dthmAM3tvAqD88x+32p5UjLJrT9wZi+eWVHObgBRi0V0usE0hgsM0iINnTWQR6mIq/WYpTmy5Z21GjgdS+bkX5xYhjn32UwCMpkjzrp1Zabs7Glm5OpHZZmbrgI3Av7r7pfM6GBERkRlS5FhEZI7cvHX40CeJiMhxRZNjEREREZGka9Mqbr/z5wAsXpRLjxiLVWnL+tOxQlbnuLwsIjzFcuwot7uWLZQbt0hi2ONVAPZblgthvZGOsWvdKXG/jXe02gb2xuc/8fgd5M4DaidHn82d9QCaSQ59jTh2djFrG0/HNqWFeW5ZYoU30zHSa3Gi2mrrGYvn2NsTjZXTsxrIbN+ByFwws8uBt6QvX2RmL8o1vxjYBHwduAL4Ujr3YmAZcKa7bzIzB77h7us79P9R4EXNc9vaHg68DngMsBLYDfwE+JC7f+oQ4y4A7wH+GPgM8Dx3n5juGhER6S5dOzkWkXl1DTAEvAr4MfDZXNuNqQ1iQvxG4NvAR4jJbIUjZGYvBT4A1IH/Bm4HVhO7pL8cmHJybGZ9wMeB3wD+AXile27RgoiILAhdOznuSYvNTl6alWQrLI9d4pb19wPQ3zeQa4uI6t5dsWvcQKOcdbYozhuvx7+TO2vZv5c9D7sQgBV/9LsA7Hj7e1tt3/729QBcnU7fnosSN9K/udXcyrrUPQPpvCf0ZpHtLSnqfG092qqNXAm41mfRWbGYdXpmikc/6kExzt96w6tbbZ9/z7sQmQvufo2ZbSImxze6++X5djNbnz59CvAyd//Ho72nmd0PeD8wAjzW3W9pa187zbXLgc8BjwYuc/e/Poz7Xj9F03kz7UNERI4fXTs5FpETwo2zMTFO/pD4mfYX7RNjAHff0ukiMzsD+B/gbOCF7n7lLI1HREROQF07OV7/4AcAUC5mj9hIebpDS5cA0Du4utU2seRMAMa37gJg8UAWVe5dHJt/FEYjL3lHLgK8cve2OP/GHwHgY1kZtU1ps5FKKXKcF+WixMVUgK2QK8rWKsmWospezqLDlhKKVzYiJ3qkmuUVj6cybaV6ijTn8pHrRLpkqRHnj27a3GorVPQXY5l335/Fvh6ZXr98GNfcF/gOMAhc4u5fPdybuvtFnY6niPJDDrc/ERGZX6pWISLzadss9tXMY9467VkHOhdYA/wCuGEWxyIiIicoTY5FZD75Idqm+uvWUIdjzV1zTj2M+38eeBNwIfBVM1t5iPNFRKTLdW1axfKV8e9jo5AtrOtftAyAcm/sgmc9y1ttPUOxVqe0ehMAP8wt5DtzMBbyuUVaxR0T2b/n636yEYDa5khTvHci24Fu1UknA7Bo734AdkxmFaEqqTRbfi38ZMqGmEjzhVpPltrRk3a/q6X+LZc6UUq/4lQ80jDqtVx5uHI03r7hdgA2XXFFq23JSachMoeaeUHFac+a2h7goDepmRWJyWy77xJVKS4BfjbTm7j728xsHHg38HUze5K7bz/UdTPxgFOXzkY3IiJyDClyLCJzZQ8R/T39CK//PnC6mT2l7fibgTM6nP8Bolz4/02VKw4wXbUKd38PsaDv/sA3zOyUIxyziIic4Lo2clxcFv921nIl2YYtPrd6PHax0tdq65+IyOz1G2KR+w9vuL3V9pDT4t/JjSMRAd7YyEVm++L3i8Hd98bX/dmmIwOD0f+FlVgMVyjkS7mlkmy5Qmyp+hyFFE4eaB4AyqU4/9GDzefJIsfN4exLm43szm02QrpnLdWJm6hnbb1LsgWJIrPN3feb2feAx5rZlcBtZPWHZ+KdwFOBz5nZJ4nNPB4FnEnUUV7fdr+fmtnLgQ8CPzKzzxF1jlcQEeV9wBOmGe8HzWwC+DDwTTP7JXe/a4ZjFRGRLqHIsYjMpRcCXwSeRuyC9xfMsIJDqhzxLOAW4LeJHfE2AQ8H7pzimn8mdsb7AjF5fj3wq8BOYmOPQ93zo8ALiMj0N83srJmMVUREukfXRo53j04CUPdcObRifF5KW0T3l7Iob6Maubz33LkJgFu37G213bE7Pu/ri9TJXIU1KqMRFV5m8XtGaTxrnByPtEVPkdyqZ7+LNMvKFdJYAEo98Z+jljI0C/m1Sr0RMZ5Iz7V/eN/BD93cPjqXj9yb9pb2avQ1Us/Gt2IkKzsnMhfc/efAM6ZotimO56//bzpHmi9NH52u+Q6xy910/W6a6v7u/u/Avx9qbCIi0p0UORYRERERSTQ5FhERERFJujatolqLxWzlUjb/X5TSIvr7ewHo6csev1SKdIMHP/A8AIZHsp3ktm29B4B9u2P3vIlcqsatExUA6j2pb6+02jwtuqu10hyyNIm0qR2FWtZXsZ4W6RUa6Zzs/MUp5WJ/StHYl2trfuYphaKcS6toplg0Unm3XJU3JsY7pGaIiIiILGCKHIuIiIiIJF0bOR7oj+L7vb25jTTKBz5udTLbsKNZR+2SpzwZgKf/8lNbTTu2xcK6Kz/xbwBc/f+uzvpIQdrqeFy/z7Pya4XU1gzW5gK6rZVA3mF/MG87B2Dv7rEDGvN9tfpI4ehJaxzU1mkM92zdcvDNRURERBYwRY5FRERERBJNjkVEREREkq5Nq7BCLJDLbWZHtZbSDVJ6RW9Pb6utt78/mspRT7hYLLbazjjrHABe8Du/C8ADL3hoq6256G7v8B4A7t29o9U2Ph5pG2NjkRJRSzvlQVb7uN7IFuQ1d82zQqqZXMr+87TG06xbnMvHKKT8jXq9lvrM0irqaZFfo3W/aq4tV7BZRERERBQ5FhERERFp6trI8XgqsVYu53aL6+0DoKcYu9IVSlnkeKISUdfJSkRY85HZ5ufLV6wB4ElPXntQm3NwZHZyMnazq1RiLMVC9ruIpZVx1Vw0ublaLh9NbmpGkZsL6vJR31oqW1etNcvI5Z65J565v38xAOXe/lZbvabIsYiIiEieIsciIiIiIknXRo4LxXi0nhQtBiinHONiOV5LubZmFLVg8fuC2cFR3mbgt1rLor3VanzejOQWi9m31BuRJ9zXuyj6yeUQeyqu1rBJcgdjXOk1Hx0uNG+eyrVZbiOSQspHLjbaascBAylivHRoeXwPegezZ653qCMnIiIisoApciwiIiIikmhyLCIiIiKSdG1axcDgEgB6cwvQSinloZ5KulUmst3sensj1aJQjFJuDc/KoTWXt1kxPisXsm9bT0pTaJZRG5sYbbW1L3cr5svKVdMiumqulFvqo7lwr1lWDrJFd95ckJevUZdGWErpIuVSdl1vX4zP0pir9UqrrVlqTuREYGbXAI93dzvUublrHPiGu6+fq3GJiEh3UeRYRERERCTp2sgxaWOMVu0zso0wiqmtXMo2+uhJG4MU0qK5yUrloOua6+OMfOAqIri1tLitGXkGKKbfPZrl3qqTWZ/NhXyFXF/NRYSlcjHdL4sqF9P4aulYp8V6zXEVc2NoLixsLr6r17OIeH5hoUiXOh8Ym+9BiIjIiaN7J8cisuC5+8/mewwiInJi6drJ8de//hUA8umJzShtc9voUi5y3CzB1jcQObr5fN+mxYPRtmbNmtaxoaVRIq2Z75svAdesvtZpm+ZiMRprtXwE+MCtofPXTUxMADCZIs7NHOS4Z1zX0yxRl9vcpOEH3qea26RkQjnHcpwws18FXgXcD1gO7AJuBz7p7u9vO7cE/B/gxcDpwA7gE8D/dfdK27kH5Ryb2eXAW4AnAGcArwbOA/YBXwDe5O7bZv0hRUTkhKCcYxGZV2b2+8DniInx54G/Bb4E9BMT4HafAP4Y+BbwAWCcmCz/42He+jXAB4EfA+8Bbk33u87MVh32g4iISFfo2sixiJww/gCoABe4+458g5mt7HD+2cD93X13OudPiQnu75jZGw8j6nsJ8Ah3/1Hufu8mIslvB353Jp2Y2fVTNJ03w3GIiMhxpGsnx5/81JUAeCO3eC5VP+tUB6q/P0q+rVx9cryuzP5NXrQodrhbc9JJcb1lZdQGBgYAKLYW0WUL3jyVg2s06mksWVszzaGRO9ZI6RSTk7V0TpY6UUkLBMdSekVec3z9aSxLhpa12vr64lg19VWsZfcrlrr2P7+ceGrAQStE3X1nh3Pf0JwYp3NGzexK4M+AhxKpETPxsfzEOLmciB4/z8xe7u6TB18mIiLdTGkVIjLfrgQGgFvM7N1m9qxDpDX8sMOxzel1WYe2qXyj/YC7DwM3An1EpYtDcveLOn0AWgwoInIC6trQYWUyIqyWK+VWaC6Wa0aQcyHksbGIrG7dGtft2rW91bZy5YrUZ1SEGh/LNvr46U9vyXdJT1+2GG7Fiog+r1oVr0sXD7Xaenv6ACjlorfNsVYqEVUeHh7OtcXYmxHuYjFbTNjXF32V00LDck9Pq62eotYTE7H4rlrNAmH5yLTIfHH3d5nZTuDlwCuJtAY3s28Ar3f3H7adv7dDN803c7FD21S2T3G8mZax9DD6EhGRLqHIsYjMO3f/N3d/JLACeDrwYeBxwNVmtnqObnvSFMdPTq/DU7SLiEgX0+RYRI4b7r7X3b/k7i8FPkqUdXvsHN3u8e0HzGwpcCEwAWyYo/uKiMhxrGvTKpoZE7m1c600Ck9JEJ5rc0+L5iqxYG20mpVLHds/AsCdGzelfrJ8jGZKw9CySHVcuSpLlVyxIj5vpkIsWbKk1bZ8eaRqlHK72ZVSbeVm/4sXZX/VXZn6ao69t7c/N4ZYdNdTjnSKSuXg1InJyTjWTLMAqNe1Q57MPzN7GvC/7t6e59OMGM/VDncvNLO/b1uUdzmRTvEvWownIrIwde3kWEROGFcBE2b2bWAT8bvtY4GHAdcD/ztH5SrQtwAAIABJREFU9/0ycK2ZfQq4B3hM+tgEXDYL/a/bsGEDF1100Sx0JSKy8GzYsAFg3bG+b9dOjsf2TXaq2CYix5/LgKcCDwF+mUhpuBN4A/ABd5+rP3G8G/gMsQDwOcB+IpXjTe31lo/QovHx8foNN9zw41noS+RINGttq3KKzKejeR+uA0ZmbygzY57PLRAR6XL57aPd/Zo5vM/1EKXe5uoeItPRe1COByfi+1AL8kREREREEk2ORUREREQSTY5FRERERBJNjkVkQXH3y93d5jLfWERETlyaHIuIiIiIJKpWISIiIiKSKHIsIiIiIpJociwiIiIikmhyLCIiIiKSaHIsIiIiIpJociwiIiIikmhyLCIiIiKSaHIsIiIiIpJociwiIiIikmhyLCIyA2a21sw+YmZ3m9mkmW0ys/eY2bLD7Gd5um5T6ufu1O/auRq7dI/ZeB+a2TVm5tN89M3lM8iJy8yebWbvM7NvmdlIer98/Aj7mpWfqXOhNN8DEBE53pnZ2cB1wGrgc8DPgIcDrwKeZmaPdvddM+hnRernXOBrwFXAecCLgaeb2cXu/ou5eQo50c3W+zDniimO145qoNLN3gxcAOwHthA/vw7bHLyXZ5UmxyIih/Z+4of4K939fc2DZvYu4DXAXwEvm0E/byUmxu9299fm+nkl8N50n6fN4rilu8zW+xAAd798tgcoXe81xKT458Djga8fYT+z+l6ebebu83VvEZHjnpmdBdwBbALOdvdGrm0xcA9gwGp3H52mn0HgXqABrHH3fbm2QrrHunQPRY/lALP1PkznXwM83t1tzgYsXc/M1hOT4yvd/QWHcd2svZfninKORUSm90vp9Sv5H+IAaYJ7LTAAPPIQ/VwM9APX5ifGqZ8G8JX05ROOesTSjWbrfdhiZs8xs8vM7LVmdomZ9c7ecEWmNOvv5dmmybGIyPTum15vm6L99vR67jHqRxamuXj/XAW8Dfhb4EvAXWb27CMbnsiMHfc/CzU5FhGZ3tL0OjxFe/P40DHqRxam2Xz/fA54BrCW+GvGecQkeQj4pJldchTjFDmU4/5noRbkiYgcnWbe5tEu4JitfmRhmvH7x93f3XboVuBNZnY38D5i4eiXZ3d4IjM27z8LFTkWEZleM4qxdIr2JW3nzXU/sjAdi/fPh4gybhemhVEic+G4/1moybGIyPRuTa9T5b+dk16nyp+b7X5kYZrz94+7TwDNxaKDR9qPyCEc9z8LNTkWEZles47nU1LJtZYUXXs0MA589xD9fDed9+j2qFzq9ylt9xPJm6334ZTM7L7AMmKCvPNI+xE5hDl/Lx8tTY5FRKbh7ncQZdbWAa9oa76CiLD9W74ep5mdZ2YH7Bzl7vuBj6XzL2/r549S/1erxrF0MlvvQzM7y8xObe/fzFYC/5K+vMrdtUueHBUzK6f34Nn540fyXj7WtAmIiMghdNjqdAPwCKIm8W3Ao/JbnZqZA7RvstBh++jvA+cDzwR2pH7umOvnkRPTbLwPzexSIrf4G8RGDLuB04FfJnJAfwg82d33zv0TyYnGzJ4FPCt9eTLwVOAXwLfSsZ3u/ifp3HXARuBOd1/X1s9hvZePNU2ORURmwMxOA/6c2N55BbGL02eBK9x9d9u5HSfHqW058BbiH5g1wC6iMsCfufuWuXwGOfEd7fvQzB4IvA64CDiFWPy0D7gF+BTwj+5emfsnkRORmV1O/PyaSmsiPN3kOLXP+L18rGlyLCIiIiKSKOdYRERERCTR5FhEREREJNHkWEREREQkWXCTYzPbZGZuZuvneywiIiIicnxZcJNjEREREZGpaHIsIiIiIpJociwiIiIikmhyLCIiIiKSLOjJsZktN7N3mdlGM5s0s61m9s9mtmaaa55gZv9lZtvMrJJeP2NmvzTNNZ4+1pnZ+Wb2r2a22cyqZvbZ3HmrzewdZnazmY2a2UQ67zoz+3MzO2OK/leZ2dvM7Cdmtj9de7OZ/VXajUtEREREZmDB7ZBnZpuAM4AXAn+ZPh8DikBvOm0T8BB339N27V8Cf5q+dGCY2Iu+uTXn2939jR3u2fwm/w7wQWCA2K6zDFzt7s9KE9/vENvJAtSBEWAo1/8fuvsH2/p+DLEveXMSXEnX9qevNwNPdvdbp/m2iIiIiAgLO3L8PmAP8Ch3HwQWAc8E9gLrgAMmuWb222QT478HVrv7MmBV6gvgMjN7wTT3fD/wA+CB7r6EmCS/LrW9hZgY/xx4HNDj7suJSe4DiYn8trYxnQF8npgYfwg4L50/CDwA+B/gNOC/zKw4k2+KiIiIyEK2kCPH24H7u/uutvbXAe8ENrr7WemYAbcB9wGucvfnduj3E8BzgTuBs9y9kWtrfpN/ATzA3cc7XP9T4Hzgt939kzN8lo8Dzwf+zt1f1aG9B/g+cAHwm+7+nzPpV0RERGShWsiR439qnxgnzRzgM81sMH1+ITExhojgdnJFej0DePgU5/x9p4lxMpJep8x3zjOzfuA305fv6nSOu1eA5oT4yTPpV0RERGQhK833AObRD6Y4vjX3+RAwCjwkfX2vu9/S6SJ3v9XMtgKnpvO/2+G070wzni8BjwD+2szOISa1351mMv1QoCd9/r0IbnfUzD0+bZp7i4iIiAgLO3K8r9NBd5/IfVlOr6vS61amt6Xt/Hb3TnPtXwP/TUx4Xw58DRhJlSpeb2ZDbefnI8wnTfOxJJ0zcIixi4iIiCx4C3lyfCR6D33KtOpTNbj7pLs/E7gY+Bsi8uy5r28zswtylzT/2+1xd5vBx/qjHLuIiIhI19PkeGaaEd/TD3He2rbzD5u7f9fd3+DuFwPLiEV+dxHR6A/lTt2eXpeZ2clHej8RERERyWhyPDM3pNdBM+u42M7MziXyjfPnHxV3H3X3q4DfT4cuyi0S/CFQS5//+mzcT0RERGSh0+R4Zm4k6g8DvGmKcy5Pr5uI8mmHJZVdm0pzUZ6RFuG5+z7g0+n4m83spGn6LpnZosMdk4iIiMhCo8nxDHgUg35z+vKZZvY+M1sBYGYrzOzviPQHgDfnaxwfhpvN7K1m9rDmRNnCw8k2GflB2659lwG7icV515nZr5lZKy/azO5jZq8GNhDVLURERERkGgt5E5AnuPs1U5zT/Kac6e6bcsfz20c3yLaPbv6Scajtow/or+2cvakviIV7w8BisooZO4EnuvtNbdc9jKjNfEo6VEvXLuLABYTr3f0bne4tIiIiIkGR48Pg7m8Gngh8jpisLgJ2ESXYntRpYnwYngm8DbgWuDv1XQFuAt5O7OZ3U/tF7v4DYtvoNwDXESXqhohUjB8SJeIepomxiIiIyKEtuMixiIiIiMhUFDkWEREREUk0ORYRERERSTQ5FhERERFJNDkWEREREUk0ORYRERERSTQ5FhERERFJNDkWEREREUk0ORYRERERSTQ5FhERERFJSvM9ABGRbmRmG4ElwKZ5HoqIyIlqHTDi7mcey5t27eT4le/4Jwdo1OsHtZlbfNJ8Beygk/JfxBbbHbfabp6XYvDFhh3UVqeRTsmub4bs830Wiumox6vn+vKDzs+PJfo3i/MLhewPAt5IfXsxvs4Nr2bR+HeX/cFBjy8iR21Jf3//8vPPP3/5fA9EROREtGHDBsbHx4/5fbt2ciwix5aZrQM2Av/q7pfO62COD5vOP//85ddff/18j0NE5IR00UUXccMNN2w61vft2slxMUVf89Ha5mcFi2iypchpXjMy24zCHnBlCsjmWxrePCOOljpEnFsRXcui2NaMABdy0evmeKx5o3xbfN5oNNI4s7E3vH7AMM2zyHGxGBFjmhH03DMbBz+/iIiIyELWtZNjEZH5dvPWYdZd9sX5HoaIHEOb3v70+R6CHCVVqxARERERSbp2ctxTrNFTrGE+2froKdTio9igp9igaLUpP0qFeuujaOmD+KBRbX2Y1zCvtdoKXsk+qFGgRrkA5UKkXDQ/iuYUzSkVso8iDYo0oFGZ8sOoYdQOOFavjlOvjlOtjFGtjLW+rlfH8UYFb1SoNyapNybx2ljro9iYoNiYmO//VNKFzGydmV1lZjvNbMLMfmhmv9LhvF4zu8zMbjKzMTMbMbNvmdlvTdGnm9lHzexcM/ukme0ws4aZrU/nnGVm/2RmPzezcTPbbWY/MbMPmtmKDn0+18y+bmZ70jg3mNmbzax3Tr4xIiJy3FNahYjMtjOA7wO/AD4GLAeeA3zOzJ7k7l8HMLMe4Grg8cDPgH8ABoBnA580swvd/U0d+j8b+B5wG3Al0A+MmNka4AdE+bQvAZ8G+oAzgRcCfw/sanZiZh8GXgJsAf4L2As8EvgL4Ilm9mR3r83S90RERE4QXTs5Htm3E4BatdI6ZqnEWWvBWq4aWiEteGuugWvky7Z5s+3Ac+KL4gF9V8aHW03lcg8Apd5F6dzc4sBG/JvbaHQoD8eB9zugpXUsu67ZRXMM+WV2lVpzsV5qmxjN+vKDy9yJzIL1wOXufkXzgJl9Avgf4PXA19Ph1xET4y8Dv9qciJrZFcTk+o1m9gV3v66t/8cAb2ufOJvZHxMT8Ve7+3vb2gbJ/a9hZpcSE+PPAM939/Fc2+XAW4BXAAf004mZTVWO4rxDXSsiIsefrk2rEJF5cyfwl/kD7n41cBfw8NzhlxC/5b02H6F19x1E9Bbg9zr0vx24osPxpoOKYrr7aH4CDLwKqAEvaTtOuvcu4PnT3ENERLpU10aOe/sjWtvTmy95lj5Pm2uUigc/fnMDjXzgOIvWpnJquY1Fao1m//Fa6i0fdF25ty/dP4sENxrRh+f6ykq+tbYIOaitWWrOc23lQuuk3Ciboyoe8Dylek+rrXRwYFpkNtzo3vHPEpuBiwHMbDFwH2Cru/+sw7lfS68P7tD2Y3ef7HD8v4G3Av9gZk8lUjauBX7qud12zGwAuADYCby6019ogEng/E4N7dz9ok7HU0T5ITPpQ0REjh9dOzkWkXmzd4rjNbK/Vi1Nr/dMcW7z+FCHtm2dLnD3O83s4cDlwNOAX09Nm83sne7+d+nrZcTvkKuI9AkREZEWpVWIyHxoJuefPEX7mrbz8jol6keD+wZ3fw6wAngocBnxc+69Zva7bX3+yN1tuo/DeiIREekKXRs5LhQiraKe++tuMe1GZ8VIgWhUxlptzcVp45VYwNeoZ+kY1VpKmShFykRfX3+rrXnM02I7K2VpC800h0b6NtdTigNAoRjnF/O79KW0j0Lh4IV/jXpzR720qDC/IC+ldBgHp1U0/5jsaUFevkJVvV5BZD64+z4zuwM4y8zOcffb2055Qnq94Qj7rwHXA9eb2XXAN4FnAR929/1mdgtwfzNb7u67j/AxDukBpy7lem0IICJyQlHkWETmy0eI3+XeYWat3xzNbCXwf3PnzIiZPdzMTurQ1Dw2ljv2LqAH+IiZHZS6YWbLzEz5wiIiC1DXRo4bKdrruVJpzUVslp66Oryj1Wb7I3hUa0Q0def2LK1x794of9ZIC+WWDq1utQ0Mxr4Cgyvi39dSKSuLWk6LAinEfeuF/FgiUl1qZOcX04L9+kT8Gz45WW219fRGimahbzDGmY84p99xiq0Sdbk+qaXXiHBbI/t9qFLptKZJ5Jh5J3AJ8Ezgx2b2JaLO8W8Cq4G/cfdvH0Z/zwNeYWbfAH4O7CFqIj+DWGD3nuaJ7v4RM7sIeDlwh5k1q2ksJ+oiPw74F+BlR/WEIiJywunaybGIHN/cvWJmTwZeS0xs/5hYtPdjolbxvx9ml/8O9AKPIqpE9ANbgauAv3X3m9vu/woz+zIxAX4SsfhvNzFJfgfw8SN8NBEROYF17eS4WIjoayG3JUYz/7ZQ6k9fZ5HTk4ai3Nppp58NwKafZ9ft3BFR5H0Tcb4Xso00vBrn7d24Mb7Obce86pTTAVh9VkR7J4tZPnJzE5CyZdFhr0S51ca+2MTrni1bWm2TAxGhHlp3HwB6BvpyzxUPVrKU/5yLHFsh3acYEeNmqbr280SOlrtv4sCU9/b29R2OTRDl1946C/1/j9g5b8bc/QvAFw7nGhER6W7KORYRERERSTQ5FhERERFJujatomSxsK6WSx2o19PiNI/H7u/JSqude9YpACxalFIgVi1utS3vi1SJ0ZRWUfGsHJqV4vztm/dE20SWcrEmXXdqfxzbX80Wy1fr0VcxtyBv/0ikU/Q1Ir2iXsr6Gq3GAr7a7hj7vt3Z7zVLlq+MtmKkWtQbucV6FtdVinG/nlyF2JpnKSAiIiIiosixiIiIiEhL10aOx0YjKprfzKOeNtmoVkYAGJzc32pbuTwW4lXT+WP7syhvsR5rgBaVI2I8VsnCr9V63GfFUCzyKxeyhXLLl0VU2ScjqlwczxYANpfmlTzra/++WPhXWBx9rT1rTautWI5I9o9/Fov0brzl56228x/6CABOP+f+AFRquZJxzfJwjXiG8cls4496Tb8biYiIiORpdiQiIiIiknRt5LhYXJResyhqT3M/5rTRx56d97TaxiaiRNrSlacCUC8vbbXddvtmANauWhb99JZbbfVaRGYHlqRNtmpZabZKKvNWHY3cYatmkePBvogw95SyvOdyc5OQtD31SWed3WrrL0bU+p7tEe0uVbJtse+8fRMAp9/nghjLYJYvTdr4ZKIa46rXszzjkn41EhERETmApkciIiIiIokmxyIiIiIiSdemVVCMR3PPL8iL1/5ipFecsfbkVls1lXnb8Is7Abhjx75W2+bhSEnYu3dTum5Vq600GOkOywYj5aIxni14I5VUq05GKkNPbne6QlrcZ6UsRaPcH+kQAz3x2utZysWSRQMAnHL6SXG/lYuy8e2KhXyb77kLgNWnnNVqs7ShWCMt/Gs0snSMZmk7EREREQmKHIuIiIiIJF0bOa57lGJzz5c1i9e+ckR0T1qVRYDv3R2R4p/viAVv5aFlrbZlZ6wDYOuGGwGw7dlCvnPPPj3OL6QIbU/2LR3dH316c6FdKYv2UkiRbcuiw16KY0P9sSCvVM+ivBMTUX6ukjYIoSf7vaZciv73bIsyb2tOW9tqq1YiOlwoxvmlchbZNlPkWERERCRPkWMRERERkaRrI8fFQiqb5tY6VklRVEtlzWq5fN+77rkXgI079wKw4vTTWm3LT4/o8GQt+hzZ/LNW26LeiPIWUuS4SpbjPDYZUd5iOe7TU87G0vAYw/7R8dax8Ym0KUkae9WyyPG9O2J8e3fH+BYPrc7uMxG/4zSGo61Qy7adznKMi2mcWbTYFTkWEREROYAixyJyXDGzTWa2ab7HISIiC5MmxyIiIiIiSdemVVQmImWgVsvSHGq1SDG4ayRSFOorsl3wxonSaqWB2OluPNvojloqB7dkZSx0W+xZukPvYPTRU05pC2SpGpWJSI+wNJaetMsdQD39XrJ9172tY1u3bQXglJOiz8pwloaxa2QYgH2jURZu+aqsDN3YWIxndDjSMvYNZ2XovCfu2bC4X62ePZg3ssWKIjL7bt46zLrLvjivY9j09qfP6/1FRE40ihyLiIiIiCRdGzkeGooybXt27W0ds1JEYndNRrR28+6xVtuS5bHAbVF/RICz2DB42kijuWFHoT8r87ZzJKLDQ4t7ABjsXdxqK6RNPEb3xaK7cimLBE8MRtS2Vs0WxVUnI6prKTI9kSLPAMP7owTbyGgcW7RmqNU2uDsW4N27bXeMN7cIsWDpP3Ehxl6yXLS4rt+NZH6YmQGvAP4QOBvYBXwG+NNprnku8PvAhUA/sBG4EniHu092OP884DLgicBqYC/wVeAKd7+17dyPAi9KY3k68FLgHOB77r7+yJ9URERONF07ORaR49p7gFcC9wD/BFSBZwKPAHqASv5kM/sw8BJgC/BfxET3kcBfAE80sye7ey13/tPSeWXg88DPgbXArwNPN7MnuPsNHcb1XuCxwBeBL3Hg78kdmdn1UzSdd6hrRUTk+NO1k+N9+yKaWs/l1TZ3kh5aGfm6+fzbsUZEWz3l5hYsi75aISLAxRT5tb4lrbYt2yMAtTodWnvq8lZbpR7nb9yyHYA11SzHuZE2Brl7y67WsYGUk9ybItR7J7Ko8m2/iA0++geij0XFbHwUY3xjExGhbm4ZDdDf05NOaUaJc9HrsVxitcgxYmaPIibGdwAPd/fd6fifAl8H1gB35s6/lJgYfwZ4vruP59ouB95CRKHfm44tA/4dGAMe5+4/zZ1/f+B7wIeAh3QY3kOAB7v7xtl5WhEROdHo7+oicqy9OL3+VXNiDODuE8AbO5z/KqAGvCQ/MU7+gkjJeH7u2O8AQ8Bb8hPjdI9bgH8GHmxm9+twr7853Imxu1/U6QP42SEvFhGR407XRo5F5LjVjNh+o0Pbt4iJMABmNgBcAOwEXm25v+jkTALn576+OL1ekCLL7c5Nr+cDP21r+/50AxcRke7XtZPjei1KnhVz6Qe1tNCtmU7RyP07W63E+X3lPgCskWtLa33Gq3F9fy4NsdibUiH6BqLPQlbK7e7hSO3YPBJ9jxWyQP291biBV7NjZ69OZeTSwry7d2VrjEbG456rVg0CUMjtnufF1FdK1ag3snSM5iLAciHOb+TaGsVDplOKzIVmftH29gZ3r5vZrtyhZUQu0CoifWImVqTXlx7ivEUdjm2b4T1ERKRLKa1CRI614fR6UnuDmRXJJrf5c3/k7jbdR4drLjjENf/aYWwq/i0issB1beS4WE6h39y/mffujEDVlrticZulhWwAzUBxTyq/Vrbs94bJFHGeTBuK9DaytMcHnxX/jg8tWwnADT/Z0GobSQvyznzwQwEYnRxtte0vRF8nLc3mAYuXxeK5/bU4r1Hqa7WtOvnUeF0RC/6qPdmzlvviP2NPb7xWUtQcYHQsNgYpluJ5Iq0z1GuKHMu8uIFIrXg88Iu2tseS+7nk7vvN7Bbg/ma2PJ+jPI3vAr+R+rppdoZ8ZB5w6lKu1yYcIiInFEWOReRY+2h6/VMza5V3MbM+4G0dzn8XUd7tI2Y21N5oZsvMLF954l+IUm9vMbOHdzi/YGbrj3z4IiLSzbo2ciwixyd3v9bM3gf8MXCzmf0nWZ3jPUTt4/z5HzGzi4CXA3eY2dXAXcBy4EzgccSE+GXp/F1m9myi9Nt3zeyrwC3EH4hOJxbsrQD6EBERadO1k+NKJfYQqOcyB+r1WIy2aDCtwylkaRW1RpxYSukUi3qzvIVaSrqoppSLQctSE045KdIiduyMv/Zu256tMTpt3Rlx/WDs1jdQyxbD1VuL+rIBjhVix7562gVvfDIL7PcPxQ5+hXL8J9u3c2errVSKY8vWRArn8P4sfcPSAsPmLn+VatZWKHRc+S9yLLwKuI2oT/wHZDvkvQn4cfvJ7v4KM/syMQF+ElGqbTcxSX4H8PG2879qZg8C/gR4KpFiUQHuBr4GfHpOnkpERE54XTs5FpHjl7s78Pfpo926Ka75AvCFw7jHJuCPZnjupcClM+1bRES6V9dOjnuK/QDUczvCnbxqDQDnrIuya7XcgrSxyYgGNyPIyxYPtNqq9Ygc79wbUdfFjLXa+nqidNud2+IvwQNLFucGEW0jkxExLhazaHTBY1F8o5hFocerUcJt19aoZDW56JRW28DySLWcaKTo8lh2XblnGQCnnXOfaCtk/1lL5Sg112jEM5TpRUREREQ604I8EREREZGkayPH9fGIrDbq2fy/VE75xD3pWE8WOV48EBHVWkpSLpdzpdxqEQEuWkSOa5Xsul17I7d5xFJEtpzlMe/cFXnIfasimrxkUbbngKcNRbwniyYPplJx28eiutWkVVttQ32x+cdkeq59o1nkeOeuiFqfcp+z49lzZejqaXOTeoocNzzbWASUcywiIiKSp8ixiIiIiEiiybGIiIiISNK1aRXl3pj3Fyi3jm3ddjcAvb2RCrFiKJfm4JF20NcT35J6vdFq27s7FshVxmMxXGV0pNW27Y5IaRg8ORb7DZSzb+nkeNynsvveeB3Jrmuk1IxaISvvVu2JRXqVWvRRy4ZAf3+UZKtW4nkmqlljOaVHDKaxeylL7SgU03iaqRaWlXatVPIpFiIiIiKiyLGIiIiISNK1keOBRRFhtdyCPEqxwG3S43V/tdJqqqYNQqy5oK6aLbq7Z+smAPaODANQGdnfalvSGyXf9u2JqHC9mEVt6+OjaQwREd5n2QI4r8WxesFbx+5Nr9t37gVg7apTW22FRozVitFHPddXLS26K1oq15bbwMTSRh+Wzm8uzAOoNrLnFxERERFFjkVEREREWro2crxqUURPy2RR1P37oxxaX3+K7ubyffH0uUUkt5j7ztz//mentoi63vSDbHfbtatPA2DdOWcBcPP3r2+1LV+9Mtrudw4AdbIocfPWXs7GN7w9Sr9tuS1Kua09ZUWrrbeUIsd9EQFeefLKVtvmO+6MZ00l6voGcoNPm40UU85xpZKNYTK3dbWIiIiIKHIsIiIiItKiybGIiIiISNK1aRWNvZGi4OR3oIvUhEKqkWaWLZ4rthaqpQVsZG3NMm89pfhd4qSTsnSHvcN7AJis7Yv7FrNUjQmPhXITjfHUUbYYrjd96wu530/GxmIB3+KlUWJu5Yqs1FyDtNNdKfofWrGk1bbjnijP1vCUJmHZfayZJuLxXEXL0irKRe2QJyIiIpKnyLGIHDfMbJ2ZuZl9dIbnX5rOv3QWx7A+9Xn5bPUpIiInjq6NHE+mBXb7J8ZbxxYNLQZgtBGbeVRrWSmzZhS54ClinIsq705l2nZtj2JrtbEs+rp3d0SMvXR73G9/trHGRIom725sAKCYi0aX61FqrlTqze6zIzYbGbUY+4233dpqW7psKQBLlqfzi9VWW60Yz7E3bTLSP5RFy2v15jOmY7nIcSMXyRYRERGRLp4ci8iC8Bngu8A98z0QERHpDl07OR4rRVS02pvb9GI8NvHwUkRPvZC35g1QAAAgAElEQVTLuW3EsXIrYpxdNzYR0ddt2yK/uFjPvm2eNvjYtmUnABPjWeTY087V947Fph77hsdabfXJuHdtMstRXtwf0d3+ZZHtct2Pvp+1LYrI8dnnngzAooEsAjxaiQj1tu0xP6gX97Xaao2IMPf3DBwwXoB6RZFjObG5+zAwPN/jmMrNW4dZd9kXj6qPTW9/+iyNRkREZkI5xyJyXDKz88zss2a228xGzezbZvaUtnM65hyb2ab0scTM3pU+r+bziM3sJDP7sJltN7NxM7vRzF50bJ5ORESOV10bORaRE9qZwHeAm4F/BNYAzwG+bGbPc/dPzqCPHuBrwHLgK8AIsBHAzFYA1wFnAd9OH2uAD6ZzRURkgerayfGekbsBqNaytAWzSGUopIpnE5NZCkQjpRsUiykXIvetaS5cW7E6dthbsSTbnW5if5RYKxbiur6ebIHdzpEdAAysiFJrd961udW2d3eUbfN6uXWsaDGwtWdE/+VdWQrEyPB+ADZv3gTA2WeenI2PSPsYm4z0jUpu47+JyYn0WZzT39vXahvozcYqcpx5HPBOd39984CZ/T0xYf6gmX3Z3UcO0cca4KfA4919tK3tbcTE+D3u/poO95gxM7t+iqbzDqcfERE5PiitQkSOR8PAn+cPuPsPgSuBIeDXZtjP69onxmZWBp4P7AMun+IeIiKyQHVt5Hj7rjsBKBWzRyyV43P3tCCvkS1Iq9fj80bzmOcix434fM9IbCwyMbGn1dZXjoVuVolz6n39rbbeckSCi2kDj1PXDrXaFi2OiHEhV96t3ojo7kDa32Pt4PJW28R4jKuSot3FUvZ7TbkvLSy0WHxXSPcFKKXffybrEXmmmpWvq+ai1iLHmRvcfV+H49cALwIeDPzrIfqYAG7qcPw8YAD4VlrQN9U9ZsTdL+p0PEWUHzLTfkRE5PigyLGIHI+2T3F8W3pdOoM+dnjzN+EDNa891D1ERGQB6trIcSk9WbmclWur1SLqammL6GKhmDu/mM6JqGsh/2uDxxdr1kYkt1bJNuAglUqrpU1Hxlo5vrBkKEWVU0R30aLs293bF+HhYimXc5zGWk/bTTc8G/vy5RGRrqfcaGtkEeDWFtSWoteetfX0RV5xrRHP1ciVqKt57jlEji8nTXG8mWw/k/JtnSbG+WsPdQ8REVmAFDkWkePRQ8xscYfj69Prj46i758BY8CFZtYpAr2+wzEREVkgujZyLCIntKXAnwH5ahUPJRbSDRM74x0Rd6+a2ZXAS4kFeflqFc17zIoHnLqU67WJh4jICaVrJ8crlq0AcgvsgMm0mK1QiMeuVbOaZ8VipDD0pEV7zQV6cX6kXJTLUQat4dmCt+Z5tjSC8KVGliZRLsfn47XYGc8s+ytvT28zBSI7VkkpGuWUE+K1bAyeUiB6Uqm4yYnsukIxxmdpkV61Pt5qazT/E6dzarkd8ryWPYfIceabwO+Z2SOAa8nqHBeAP5hBGbdDeRPwRODVaULcrHP8HOBLwK8eZf8iInKC6trJsYic0DYCLwPenl57gRuAP3f3q4+2c3ffaWaPBt4KPAN4KHAr8IfAJmZncrxuw4YNXHRRx2IWIiJyCBs2bABYd6zva50Xc4uIyNEws0mgCPx4vsciMoXmRjU/m9dRiEztAqDu7sd01zJFjkVE5sbNMHUdZJH51tzdUe9ROV5NswPpnFK1ChERERGRRJNjEREREZFEk2MRERERkUSTYxERERGRRJNjEREREZFEpdxERERERBJFjkVEREREEk2ORUREREQSTY5FRERERBJNjkVEREREEk2ORUREREQSTY5FRERERBJNjkVEREREEk2ORUREREQSTY5FRGbAzNaa2UfM7G4zmzSzTWb2HjNbdpj9LE/XbUr93J36XTtXY5eFYTbeo2Z2jZn5NB99c/kM0r3M7Nlm9j4z+5aZjaT308ePsK9Z+Xk8ldJsdCIi0s3M7GzgOmA18DngZ8DDgVcBTzOzR7v7rhn0syL1cy7wNeAq4DzgxcDTzexid//F3DyFdLPZeo/mXDHF8dpRDVQWsjcDFwD7gS3Ez77DNgfv9YNociwicmjvJ34Qv9Ld39c8aGbvAl4D/BXwshn081ZiYvxud39trp9XAu9N93naLI5bFo7Zeo8C4O6Xz/YAZcF7DTEp/jnweODrR9jPrL7XOzF3P5rrRUS6mpmdBdwBbALOdvdGrm0xcA9gwGp3H52mn0HgXqABrHH3fbm2QrrHunQPRY9lxmbrPZrOvwZ4vLvbnA1YFjwzW09Mjq909xccxnWz9l6fjnKORUSm90vp9Sv5H8QAaYJ7LTAAPPIQ/VwM9APX5ifGqZ8G8JX05ROOesSy0MzWe7TFzJ5jZpeZ2WvN7BIz65294YocsVl/r3eiybGIyPTum15vm6L99vR67jHqR6TdXLy3rgLeBvwt8CXgLjN79pENT2TWHJOfo5oci4hMb2l6HZ6ivXl86Bj1I9JuNt9bnwOeAawl/tJxHjFJHgI+aWaXHMU4RY7WMfk5qgV5IiJHp5mbebQLOGarH5F2M35vufu72w7dCrzJzO4G3kcsKv3y7A5PZNbMys9RRY5FRKbXjEQsnaJ9Sdt5c92PSLtj8d76EFHG7cK08ElkPhyTn6OaHIuITO/W9DpVDts56XWqHLjZ7kek3Zy/t9x9AmguJB080n5EjtIx+TmqybGIyPSatTifkkqutaQI2qOBceC7h+jnu+m8R7dH3lK/T2m7n8hMzdZ7dEpmdl9gGTFB3nmk/YgcpTl/r4MmxyIi03L3O4gya+uAV7Q1X0FE0f4tX1PTzM4zswN2f3L3/cDH0vmXt/XzR6n/q1XjWA7XbL1HzewsMzu1vX8zWwn8S/ryKnfXLnkyp8ysnN6jZ+ePH8l7/Yjur01ARESm12G70g3AI4iaxLcBj8pvV2pmDtC+kUKH7aO/D5wPPBPYkfq5Y66fR7rPbLxHzexSIrf4G8RGC7uB04FfJnI8fwg82d33zv0TSbcxs2cBz0pfngw8FfgF8K10bKe7/0k6dx2wEbjT3de19XNY7/UjGqsmxyIih2ZmpwF/TmzvvILYiemzwBXuvrvt3I6T49S2HHgL8Y/EGmAXsfr/z9x9y1w+g3S3o32PmtkDgdcBFwGnEIub9gG3AJ8C/tHdK3P/JNKNzOxy4mffVFoT4ekmx6l9xu/1IxqrJsciIiIiIkE5xyIiIiIiiSbHIiIiIiKJJsfTMLPFZvYuM7vDzCpm5ma2ab7HJSIiIiJzQ9tHT++/gCelz0eIlbv3zt9wRERERGQuaUHeFMzs/sDNQBV4nLsfVUFpERERETn+Ka1iavdPrzdpYiwiIiKyMGhyPLX+9Lp/XkchIiIiIseMJsdtzOzyVBz9o+nQ49NCvObH+uY5ZvZRMyuY2R+Z2ffNbG86fmFbnw82s4+b2WYzmzSznWZ2tZn9xiHGUjSzV5vZTWY2bmb3mtkXzOzRqb05pnVz8K0QERERWXC0IO9g+4HtROR4CZFznN9tJb87kBGL9p4J1ImdhA5gZr8PfIDsF5G9wBDwFOApZvZx4FJ3r7ddVya2RbwkHarx/9u78+i6q3L/4+/nZB6apHPSMW0ptIyFUmahgAIyCcgkoBb1LhH9oThc9TrhBDj8EC+I6I8foogMF1BRActUmUFaptJSStt0pnPTKfPZ94+9z/d7Gk6GtkmTnHxea3V9k/18h33Ss5KdJ8/e2/9/nQGcamYX7/5LFBEREZFMlDluxTn3c+dcJfDF0PS8c64y7d/zaaefh9+68EqgzDk3EBiO3yscMzuGeGB8PzA6nFMBfAtwwGXANzN05dv4gXEL8KW0+1cDjwK3dd2rFhERERHQ4HhPlQJXOed+7ZzbAeCcW+uc2xLiP8R/jZ8DLnbOrQjnbHPOXQtcH877upmVpW5qZqX4/e0Bvuuc+6Vzri5cuxQ/KF/aza9NREREpN/R4HjPbABuzxQws0HAieHT61qXTQQ/Aerxg+zT09pPBUpC7L9bX+ScawJu2P1ui4iIiEgmGhzvmVecc81txA7F1yQ74F+ZTnDO1QKzw6eHtboW4DXnXFurZTyzi30VERERkQ5ocLxn2tstb2g41rYzwAVY0ep8gCHhuLqd61Z10DcRERER2UUaHO+ZTKUSrRXsxn2tE+doa0MRERGRLqbBcfdJZZWLzGxoO+eNanV++sdV7Vw3Ync7JiIiIiKZaXDcfV4lzu6emOkEMysHpoZP57S6FmBKWLkikw/scQ9FREREZCcaHHcT59xG4Knw6dfNLNPX+utAIX7jkYfT2mcC20Ps860vMrNc4Oou7bCIiIiIaHDczb4DJPErUdxjZqPAr2NsZv8FfCOcd33a2sg457YCvwif/sjM/o+ZFYVrx+A3FBm3l16DiIiISL+hwXE3CrvpXYkfIF8ALDOzjfgtpH+Mn3h3F/FmIOl+iM8g5+LXOq4N1y7Fr4n8qbRzG7rrNYiIiIj0JxocdzPn3G+AacCf8EuzlQK1wGPABc65yzJtEOKcawTOwO+UNxc/wG4B/gYcT1yyAX6wLSIiIiJ7yJzTimB9kZmdDDwOLHXOVfdwd0RERESygjLHfdfXwvGxHu2FiIiISBbR4LiXMrMcM7vfzE4LS76l2g8ws/uBU4EmfD2yiIiIiHQBlVX0UmG5tqa0pi34yXnF4fMk8Dnn3G/3dt9EREREspUGx72UmRlwBT5DfBAwDMgD3gOeBm50zs1p+w4iIiIisqs0OBYRERERCVRzLCIiIiISaHAsIiIiIhJocCwiIiIiEmhwLCIiIiISaHAsIiIiIhLk9nQHRESykZktAcqAmh7uiohIX1UNbHHOjdubD83awfHsf/87rFGXlhxPJAGw3GYA1myuiUIrNywCYNjowQA0WTKKvfCqX054wsR9ACgf1Bzf0tUDUGLDAMitHxTFhhX585PbigDIt5K4L+YPLYnGqCmZU+ePCX9PZxafn8xv87Uabueji68zF58Vepx2pf946tRD0x4kIl2krKioaNDkyZMHdXyqiIi0Nn/+fOrq6vb6c7N2cJwM6zcn0gaYiUQOAC3JBgCam+IN6MoHlAGwcMF8AMaMr45iTQ07AGgIx61b4wHtqlVLASi2WgAmjTw4iu1o9G0DigpSN4piLhrAxoPw1KDWXO5On/sLUoPaDNe1+sg6vXS11riW3sfMrsJvgDMOKASuds7d2LO92i01kydPHjR79uye7oeISJ80depU5syZU7O3n5u1g2MR6XvM7GLgl8CrwI1AA/Bij3ZKRET6FQ2ORaQ3OTN1dM6t6tGedIG5K2up/sY/erobIiI9oub6M3q6C7slewfHqYoBe385rZkvUaivj+tYRk8YCUBL3nYANmxeG8U+ePLxACxatjTcIK7bnff2QgASjSsAmHbQ0VGsMJRxNGzZDEBeWs2xWV746P01wJZMxdJKQlLnZayEaLs8Iq7McK2O7V8n0kNGAGTDwFhERPomLeUmIj3OzK4xMwecGD53qX9pn88ys0ozu83MVppZi5nNSLtHlZn9ysxqzKzRzNaZ2YNmNrWNZ5ab2Y1mtsLM6s3sbTP7spmND8+7Yy+8dBER6WWyN3Ocyu6mZY6d85PYcnJ8RrepKV51YvmKZQCc9KHpANz30N1R7NVXfcnjmPF+9YmWvPi6KVP9z93ywgoAttRtimJlFb7NCvzzLBl/uZPNLb7NFaR1Oicc80IszuymJtml55Kj19UqA5w+jy91nYtm6bUg0gvNCscZwFjg+xnOGYSvP94GPIiflboGwMzGAc/iM89PAncDo4ELgDPM7KPOub+nbmRmheG8w/D1zXcB5cC3gA/sSsfNrK0Zd5N25T4iItI7ZO/gWET6DOfcLGCWmU0Hxjrnrslw2kHAncCnnHPNrWK34gfG33bO/TjVaGa3AE8Dvzezsc65bSH0NfzA+B7gEudcKkP9Y2BOV70uERHpe7J2cJxItF76DJIhaZpb4F92bm5eFFu1con/wPmT9p8cJ33+dP89AAwfUQnA+H0mR7EBA8sBWLvsPQAe/Nv9UeyEw08A4JjDfCJq2/rtcQddoe9d2o94w6+H7JK+z7mJnLRYWHYuZL9dWno4EbLkmSqIXdqSb7BzVlk1x9LHNAJfbT0wNrNRwCnAMuCn6THn3PNmdjdwGXAe8IcQ+iQ+8/zN1MA4nL/czG4EftTZTjnn2irbmI0fgIuISB+immMR6StqnHNrM7QfGo7POOeaMsSfTD/PzMqACcBK51xNhvOf3dOOiohI36XBsYj0Fe+10V4ejqvbiKfaK8KxLBzXtHF+W+0iItIPZG1ZhSV8/YBLvj/W1OhLJ0aPHB21zV3wAgCvz34dgNzC+PeGRx/2JYgtzm/hPOXoaVGsrsHvlldaPgCAkrLiKPbI448CMLJqFAADiwfHnUj6v+Tm5selHclG/9fiRI5/TmPajnp5of4iJ5RapF4fQHOLj5n5mCO9diL1OlpafZ7eJtIntFUHVBuOlW3Eq1qdtyUch7dxflvtIiLSD2Tt4FhE+o1Xw/E4M8vNMFnvxHCcA+Cc22Jmi4FqM6vOUFpxXFd17MCR5czuo4vgi4j0V1k7OI6n2MRZVDP/cpPNPttbUloaxVLZ3TWr/V9Uh44YFMVamnyG9YH7ZgFQPjyOXXTxxQDMe91nnMdUj4xi61b7Zd1enO2Xgrvo3POjWOPWMNmuMe7zjpApzk2EbHJOnNlNhgl5iYQLryWerNcclqTLyw+Z4wzZ8hTTHDzJMs65FWb2GPAh4EvAz1MxMzsSuATYBPw57bI/ANcA15lZ+moVo8M9RESkn8rawbGI9CtXAM8BPzOzU4BXiNc5TgKXO+e2pp3/U+Ac4GJgPzObia9dvhC/9Ns54ToREelnNCFPRPo859xi4HD8esf7AV8FPgw8ChzrnPtrq/Pr8OUWN+Frla8On18LXBdO24KIiPQ7WZw5TtUPpJVVhGMilFc0NcQT3kZW+sl5NavnAbDfgROj2FmnHQvAE8/8G4DXX30jip159pkANDf70oY3Xns1ik2ddggAY0eOA2DOm3GswPyE+f0nxkukloR1l2s3+wRXQUFa6URzHQDJsIuec2m77YU1nZ2FRJel7Z73vp31RHov59z0Nto7fAs751YCn9uFZ20Grgr/Imb2H+HD+Z29l4iIZA9ljkWkXzKzERnaRgPfAZqBv7/vIhERyXpZmzl2IXNs6as/ufC7QGrnORdnZocM8qtAzX3HL9u2pXZbFLv0YxcCUFLmd7VrzIvvuXa1X3r12GlHAfDugjjZtLHWrxx1+NRh/t6vxbGHwq57Z5/x0ajt+GP9pPrR431fmhrqotjmTT6b7MIScC1NcR9cyITXNfnz83KLoliC1jsFxteZ8snSvz1gZnnAbGAzUA2cCRTjd85b2YN9ExGRHpK1g2MRkQ7cCXwc+Ch+Mt424CXgZufcgz3ZMRER6TlZOzi2DDXHURVJyCAnLN6AoyBkW0ePqAagZsmKKDb9AycAUDnEL/fm4n0+WL/OZ473Hemv+8QlM6JYzYoaABYuXgzAwVMPjWJvvrkIgB/+9CdR222/GuOfM2IoABvr1kWx8ir/0A1rfTZ66/Z4DbjB5f6vw/Xbfc2xS8sOp5Z1s1CyqVyxiOecuwW4paf7ISIivYtqjkVEREREAg2ORURERESCrC2rcGErOHPv3xLOQlmFS8ZFBju2+WXdxo6eAMBzL8+KYus3+J3uJu6zH7BzucOG7esBmL/YT7Y7Y9pZUaxm2XIAfn3rbQAcNmVaFDvnvHMAGDlmVNS2sc7vzvf9n30PgLfffiuKffYzMwCYtO+BALz7+jtRrCnMuS8tHBJefPzfmpda1i0ZXnPailgJU5GFiIiISDpljkVEREREgqzNHGeeeRZtAxI+i19+SUk5ADmFfrm2nEQ8We+NuX5jkBM/6Jdae+Pdf0exee++CUCyxWeot03bEcXGVvrNP6ZM8RPxHn70iSi2YrnPRl9/7bejtsVL/cS9Wc88B8DCl1dFsUMP8xuPTJzsNyd59uV/RbFBpw0HoKpyLAD1W5rjlxwS56lMek7aBiGWfH9WXURERKQ/U+ZYRERERCTI2sxxsqUJgByLN/pI1ds6wpJnyZb4/Eb/cUGe/31h2NB486wXX/aZ3COn+prh8eMmR7FX5vm64NIBvt534fIlUWzUqGoAzvyw32L6jdcWRrF3Fvgs8dLVy6O26on+/IsvPxuAmVUvRLEJB/mM8eIVfom551+ZHcWOPO44AOoW+rrpHZvibbGnHXykf30N/nW11MXZ4qLcQkREREQkpsyxiIiIiEigwbGIiIiISJC1ZRWEJdwsJy4jSISPW8LkuZb0Zd7Mfynq6+oBGFU1Ngotq/kTAA/+2e8oe9Hll0exyqp9ACgs8CUKt992fxSbccnFABx1yFEAfOyC86NYcyjpKCopiNruud9f25j0ZRFXfnlGFJu4j3/O3x54BIDlK+Ll5J5+aRYAZcUDABhWVhXFphUcAkCO+f655vj3oQRxyYlIb2Bm1cAS4PfOuRmdOH8G8DvgcufcHV3Uh+nAU8D3nXPXdMU9RUSk71DmWEREREQkyNrMcX6ez5Q2NcWT7nLzfZY2ke9fdktD2pJn5pdua2r2WduiwrIodOjBfiLesmXLANi8cUsUO2T/wwCY/5afmPfiE69GscMm7w/AgftOAuCjp58dxerwS7798c57o7Z77noAgOZ83+emurh7E8f7CXmVI4b5e11yahTbd6JfMm5Q2UAAjjn4uPjCpH+N2+v80nGl+QOjUFNjPSJ93J+BF4HVPd0RERHJDlk7OBaR7OecqwVqe7ofbZm7spbqb/wj+rzm+jN6sDciItIZKqsQkV7JzCaZ2V/MbKOZbTezZ83slFbnzDAzF2qP09trwr8yM7shfNxkZteknTPczP6/ma0xszoze83MPrl3Xp2IiPRWWZs5NvNrGre0JKO2ujDZLj9MnsvLi9f5bWzxNQwF+UUANLXE5RiHHOLLKoat8xPdtu+Id8GrLB/tn9fkJ/cVl8aT3Arz8wFY+O4CAKpHVEexgjABMC/t15PDDz3A96/Ul380bd0exRKN/r6Dy31ZxPnnxxmoitD21mtz/bkufs3PPON35du6yb/2Uz9wehQrLY5LR0R6mXHAC8Bc4DdAFXAR8IiZXeKcu7e9i4N84ElgEDAT2IKf7IeZDQaeB8YDz4Z/VcCt4VwREemnsnZwLCJ92vHAz51zX0s1mNnN+AHzrWb2iHNuS5tXe1XAPOAE59z2VrHr8APjG51zV2d4RqeZ2ew2QpN25T4iItI7ZP3guKKiIvp461af8W1o8JPucvLiLG9+mKyXDKu75eUVRbF16/xktvycEgBGDh8TxbY1+HuOGzMegJNOiCfDrVm1FoD77r4PgIvO+2gUu/TCSwCYceEnorYHC/2EvEXLFvnzz/5YFFu2sAaAO++6C4CWxngXvE9/ZgYACZ8s56mXHotiy1f6HfWmH3USAAOHxa9r07oNiPRStcAP0hucc6+Y2V3AJ4Fzgd934j5faT0wNrM84FJgK3BNO88QEZF+SDXHItIbzXHObc3QPiscD+3EPeqBNzK0TwKKgdfChL62ntEpzrmpmf4Bb+/KfUREpHfI2syxmV/CrKQ03mQjJ8f/LlBb63/mOuKl3IpL/AYaDWF5t8YWi2KjR/kNQdas96tFlRaVR7GSgT4zvX2bT05d8rGPR7HHH/eli6+/5n9Gblh7WxRL1T1fdl6cHT77lHP8eZvW+3MKi6PYj396AwBPzZzj+7RP3IdtmzcCcOjBBwGweu2qKFZeVgrAtGl+M5DNG7ZFsc071iLSS61po/29cCxvI55urXPpO/1EUtd29AwREemHlDkWkd5oeBvtleHYmeXbMg2M06/t6BkiItIPaXAsIr3RYWY2IEP79HB8NUOss94GdgBTzCxTBnp6hjYREeknsrasorHJlzls2hQnj8rLfQlEeYUvadiwaWMUKyjwy5o1Nfsl3Brr4+Xahg4bBEALfhJcIieeyNfc7MsvHvzLQwCc+5GPRLGjjz4egMqRftWpdxatiGK/u9NP0pty+JSobeyoagBGVfoyjsWrlkSxYRUhmRUqQcaMGRnFNmzYDMCji/2ybcceF5djLlrgSyeeePafANSmlVWMGzMBkV6qHPgukL5axeH4iXS1+J3xdotzrilMuvsP/IS89NUqUs/oEgeOLGe2Nv4QEelTsnZwLCJ92tPAZ8zsSOA54nWOE8BnO7GMW0f+CzgZ+FIYEKfWOb4IeBg4u51rRUQki2Xt4Lio2L+0+oY4U9qczANg8NDBABSWxFUlJQMKwjk+K7y1Lt4EJOwnwqDB/q+8xWkT5bY2NgEwbsJEAMpKh0SxisE+G338KScDsODmeOWpsoE+Gz1wyLCo7Z9P+szvytUrAfjI6XEW+j+/9sXQT//sifuPimJr1/h5Rffd/3f/ugrjbPnDDz7r+z7EZ8svufDCKFZZGfdVpJdZAlwBXB+OBcAc4AfOuX/u6c2dc+vN7FjgWuAs4HBgAfA5oAYNjkVE+q2sHRyLSN/jnKsBLK3pI22cmjr/DuCODO3VnXjWe8Cn2ghbG+0iIpLlsnZwXFXlM7LJ9JWcQgo4N8+3FRTFP/+27vD1x2UVPoNcXhFv9FGzzNf+VgzymeDFqxZFsWFDqwE49+zzUw+JYqV5Pst7/PHTAXh7Ybzs6eVXfBqAgcVx5vihR/zmHU/NegaAsePHR7FjjjgCgG999au+Ib8xit3y+98AUBfqpBvSNgg54li/hNv6tX4ZulWr4mXemnb4Auaj9kFERERE0GoVIiIiIiIRDY5FRERERIKsLat4Z+G7wM4lBqWlJQAMHTYUgI2b10exhuZ6AI446HAA5i6aG8V+9/vfAnDBRRf5cxPxrnsVZVUALIJMhE8AAA/qSURBVFzsn7dw2cIodtapZwEw7dCjADjglwdEsTlvzgbglnvjXfP2O3g/AE467SQAmpLxDn533/cnAE770CkADBxQFsXOO9vPHRpR6V/XuPGjo1hJcREAm9f6XQHfnvsusTxEREREJKbMsYiIiIhIkLWZ47wwGS4/P152rbHRZ4ffmjsfgDXr34tio6tHALB+m88mv7NwQRQbMdJvuHHQpIMA2FqfjJ+T8L9f3HvPHwFYunJ5FPvgSScCUFHiNx/Ja4k3D3niKb9s2wvPPR+1/eT6HwFQWuIz3H/6451RbPI+fqm4B//xIACVI+KJfOed6if0D/mQXx7ujXfeimI//dGv/DnnngfACcd9MIrt2FyPiIiIiMSUORYRERERCTQ4FhEREREJsrasIuF8OUVLsilqKyzwE+mqqsJOd6XxpLat2zcD8NZbvpwimYx/b5gSJuktWbwUALP8KJaXWwvAmSf5coV3ly+JYo8//AgAxx93AgBFFUVRbNphhwFwxJRDo7Z9Rk8A4Gc3/ASANWFtYoAvXHEFAIvefQeAl2bH5Rh/K/AT64qL/C5469dtjr8OVgpAfsI/u7RwYBwrUlmFiIiISDpljkVEREREgqzNHDc3hexuIp4El2zxO+O1NPnl3QaUDopihcU+01xfv83HSuIM66baTQCsXPkmAAVpX7bRI0YBMGbIsHBdYRR77JmnAHg60QKAy41361u2vAaAj196adRWPdAvwTa2yt9zzZo4c1yc7+87cR+/nd3Lr78Yxeoa63z/wu53gyuGRLEvXvU5f06t31FvzaoNUWxweSUiIiIiElPmWEREREQkyNrMcQJfX2wWL7vmovpjn8FNNrekXeEzzE2N/vyqqngjjREhO7xp40Z/78bGKFZS6DO6FhLUFRVxHfNJJ08HoL7FZ6pnznosilWO8kvHDSyviNoa8RngRI6/WX1dvAnItm3bAfifv9/r+zRsRBQ782S/2cjdf7kbgJa0zUOmHDQFgJrFKwBYVbM2ipWXpr9+EREREVHmWEREREQk0OBYRHZiZrPMzHV85h4/p9rMnJnd0d3PEhER6aysLavADAC3049435YIk/SSybisIGH+94SyknIAGnfEy5w550stikMJRaI4/rJt3OJLLYYP92UOLTlxSUNeoV9ibejQ4QB8fL9PRrGGpL//S2/FE+umTvbLuo0d70s6JqyvjvtX6MtEBgz2kwgrR4+MYrk5vj+TDtkXgMXvLo5iK7f4peXKK325x9bt8XJy62vjCX8iIiIiosyxiLzfJ4DJPd2JbDB3ZS3V3/hHT3dDRER2QRZnjlNZ4WRaWziGbHLCLAo5t3OmOZmMr0skUo3+mAgZYYCK4YMBWLHeL6O2cFWctR021i+VVhI2HWlJxJnqH//oWgCaGhqitluvuxmAMWP9BMDjio6JYiVl/h7nnnseAGs2r4xiS7b57HBBqf/vXLx8YRQbEjLNB+3rNx0ZPS6eyLdq8XuItOacW9bTfRAREekpyhyL9ANmNsPMHjCzxWZWZ2ZbzOw5M7ssw7nvqzk2s+mhPvgaMzvCzP5hZhtDW3U4pyb8Kzezm81spZnVm9k8M7vKLO230fb7uq+ZXW9mr5jZOjNrMLOlZvZbMxuV4fz0vk0JfdtsZjvM7F9mdkwbz8k1syvN7MXw9dhhZq+a2RfMTN8bRUT6qezNHEcZ4/Sf8a5VLJb6WZiTCFnhtJ+NybA0WrLFH5sb4nvmFvvz8xO+JnjmwzOj2BnnngnAmjV++bS/P/pwFFuzZh0Ah005MGpbstpnfN9c8AYAA9M281hf67O8LUm/jFxqq2iARLPvz4P3PATAgOLSKFZW5Guo33nrXQAWzo23t544Vn8570d+DcwDngZWA4OB04E7zWw/59x3Onmfo4FvAs8CtwNDgMa0eD7wOFAB3BM+/yjwS2A/4POdeMZ5wBXAU8Dz4f4HAJ8BzjKzw51zKzNcdzjwn8ALwG3AmPDsJ8xsinNuQepEM8sD/gacCiwA/gTUAycCNwFHAh/vRF9FRCTLZPHgWETSHOicW5TeYGb5wCPAN8zs1jYGnK2dAlzhnPtNG/EqYHF4XkN4zveAfwNXmtm9zrmnO3jGncAvUten9feU0N9vA5/LcN0ZwOXOuTvSrvkscCvwReDKtHO/hR8Y3wx8yTnXEs7PAX4LfMrM7nfO/bWDvmJms9sITeroWhER6X30p0ORfqD1wDi0NQK/wv+SfHInb/VaOwPjlG+mD2ydcxuBH4ZPL+9EX1e2HhiH9pnAW/hBbSbPpQ+Mg9uBZuCIVEMomfgC8B5wdWpgHJ7RAnwF/2emSxERkX4nizPH4efdTlWOyZ3a0pd5S32cSPigWfylSS3zRsIfrSUuy6hZuBSAsqG+fOHoqXF5Y/12v1zb+i0bABhUOCiKXfB5X3JRVlIStc17y5dT3HW73wXv3PPPjGIjRvjJfY8/8TgA4ydUR7FhVX5S4NrlmwAYfkC8u9/SRX5nvHlzfFlFAXHJxUET4o8lu5nZGODr+EHwGKCo1Skj33dRZi93EG/Gl0K0NiscD+3oAaE2+VJgBnAIMJDUFpZeY4bLAF5p3eCcazKzNeEeKfviy0oWAt9uoxS6jk6u2OGcm5qpPWSUD+vMPUREpPfI4sGxiACY2Xj8oHYg8AwwE6jF/wZZDXwSwn7rHetoiZP16ZnYDNeVd+IZNwBfwtdG/xNYCWFvdT9gHtvGdZvbaG9m58H14HCcCHyvnX7ot0cRkX4oewfHlunns2vjCC6kjp0L2WEXZ5N8GSIkws9X5+KNPgZXVAHQkmwC4Jhpx0WxBtsBwMYd6wEYM2ZoFCvO8/da+HY0R4inHvOlmJMnVgOw7/h4DDDvzdcBePmF5wAYUBIn/p5/+iUAKof5CXz5ibhapnaNH1MMr/CJwWOnnRjFBpXF/ZGs9mX8gPDy1mUHZvYx/OC4szraOW+ImeVkGCBXhmNtexeb2TDgKmAucIxzbmuG/u6pVB/+7Jw7rwvuJyIiWUQ1xyLZb59wfCBD7IQuflYukGnptOnh+GoH14/Hf1+amWFgPCrE99Tb+CzzUWHVim5z4Mhyaq4/ozsfISIiXUyDY5HsVxOO09MbzexU/PJoXe06M4vKNMxsEH6FCYDfdXBtTTgeZ6k/2fh7lAL/jy74a5fzf/q5Cb+yxn+bWev6a8ysysz239NniYhI35PFZRXvX8s45v8ynL7Of2oiXkoyfWM9S03SC+UV6ZP1QjnFgLIyAJpy6qNYXrhuR/0W/9T8+Gdw3Tbf9t7KpVHbgfuPA+Dww/08oMWL45KLufP8GshnnfFhALZvq4tiSxb4yXYnHDcdgPEj941iQ8t92cfAYl9CYc358QtrSi/DlCx2C36ViP8xswfwNbwHAqcB9wEXdeGzVuPrl+ea2UNAHnA+fiB6S0fLuDnn3jOze4CLgdfMbCa+TvlD+HWIXwOmdEE/f4if7HcFfu3kJ/Ffl2H4WuRj8cu9zeuCZ4mISB+SvYNjEQHAOfeGmZ0I/Ai/8Ucu8Dp+s43NdO3guBH4IHAtfoA7BL/u8fX4bG1nfDpccxF+05B1wEPAd8lcGrLLwioW5wCX4Sf5nYmfgLcOWAJ8B7hrDx9TPX/+fKZOzbiYhYiIdGD+/PngJ47vVeZcR/NrREQ6ZmY1AM656p7tSe9gZg34VTJe7+m+iASpjWne7tFeiMQ6ek9WA1ucc+P2Tnc8ZY5FRLrHXGh7HWSRvS21m6Pek9Jb9Nb3pCbkiYiIiIgEGhyLiIiIiAQqqxCRLqFaYxERyQbKHIuIiIiIBBoci4iIiIgEWspNRERERCRQ5lhEREREJNDgWEREREQk0OBYRERERCTQ4FhEREREJNDgWEREREQk0OBYRERERCTQ4FhEREREJNDgWESkE8xslJndbmarzKzBzGrM7EYzG7iL9xkUrqsJ91kV7juqu/ou2akr3pNmNsvMXDv/CrvzNUj2MLPzzewmM3vGzLaE988fd/NeXfL9dnfl7o2HiIj0ZWY2AXgeGAb8FXgbOAL4InCamR3rnNvQifsMDvfZF3gSuAeYBFwOnGFmRzvnFnfPq5Bs0lXvyTTfb6O9eY86Kv3Jt4FDgG3ACvz3tl3WDe/tXabBsYhIx27Bf6O+yjl3U6rRzG4ArgZ+DFzRiftcix8Y/8I59+W0+1wF/DI857Qu7Ldkr656TwLgnLumqzso/c7V+EHxu8AJwFO7eZ8ufW/vDm0fLSLSDjMbDywCaoAJzrlkWmwAsBowYJhzbns79ykB1gFJoMo5tzUtlgjPqA7PUPZY2tRV78lw/izgBOecdVuHpd8xs+n4wfFdzrnLduG6Lntv7wnVHIuItO+kcJyZ/o0aIAxwnwOKgaM6uM/RQBHwXPrAONwnCcwMn564xz2WbNdV78mImV1kZt8wsy+b2YfNrKDruivSaV3+3t4dGhyLiLRvv3B8p434wnDcdy/dR6Q73kv3ANcB/xd4GFhmZufvXvdEdluv+D6pwbGISPvKw7G2jXiqvWIv3UekK99LfwXOAkbh/7IxCT9IrgDuNbMP70E/RXZVr/g+qQl5IiJ7JlWruacTOLrqPiKdfi85537RqmkB8F9mtgq4CT+J9JGu7Z7Ibtsr3yeVORYRaV8qU1HeRrys1XndfR+RvfFeug2/jNuUMBFKZG/oFd8nNTgWEWnfgnBsq8ZtYji2VSPX1fcR6fb3knOuHkhNHC3Z3fuI7KJe8X1Sg2MRkfal1uo8JSy5FgkZtWOBOuDFDu7zYjjv2NaZuHDfU1o9T6QtXfWebJOZ7QcMxA+Q1+/ufUR2Ube/tztDg2MRkXY45xbhl1mrBj7fKvx9fFbtD+lrbprZJDPbaXco59w24M5w/jWt7vOFcP9/ao1j6UhXvSfNbLyZjWx9fzMbAvwufHqPc0675EmXMrO88J6ckN6+O+/tbumfNgEREWlfhu1M5wNH4tckfgc4Jn07UzNzAK03VsiwffTLwGTgI8DacJ9F3f16pO/rivekmc3A1xb/C7/xwkZgDHA6vubzFeBDzrnN3f+KpK8zs3OAc8KnlcCpwGLgmdC23jn31XBuNbAEWOqcq251n116b3cHDY5FRDrBzEYDP8Bv7zwYv1PTX4DvO+c2tjo34+A4xAYB38P/EKkCNuBXA/iuc25Fd74GyS57+p40s4OArwBTgRH4yU5bgbeA+4DfOOcau/+VSDYws2vw39vaEg2E2xsch3in39vdQYNjEREREZFANcciIiIiIoEGxyIiIiIigQbHIiIiIiKBBsciIiIiIoEGxyIiIiIigQbHIiIiIiKBBsciIiIiIoEGxyIiIiIigQbHIiIiIiKBBsciIiIiIoEGxyIiIiIigQbHIiIiIiKBBsciIiIiIoEGxyIiIiIigQbHIiIiIiKBBsciIiIiIoEGxyIiIiIiwf8Cs75u0mI8w0oAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 8 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 319,
       "width": 355
      }
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "%config InlineBackend.figure_format = 'retina'\n",
    "\n",
    "import tensorflow as tf\n",
    "import pickle\n",
    "import helper\n",
    "import random\n",
    "\n",
    "# Set batch size if not already set\n",
    "try:\n",
    "    if batch_size:\n",
    "        pass\n",
    "except NameError:\n",
    "    batch_size = 64\n",
    "\n",
    "save_model_path = './image_classification'\n",
    "n_samples = 4\n",
    "top_n_predictions = 3\n",
    "\n",
    "def test_model():\n",
    "    \"\"\"\n",
    "    Test the saved model against the test dataset\n",
    "    \"\"\"\n",
    "\n",
    "    test_features, test_labels = pickle.load(open('preprocess_test.p', mode='rb'))\n",
    "    loaded_graph = tf.Graph()\n",
    "\n",
    "    with tf.Session(graph=loaded_graph) as sess:\n",
    "        # Load model\n",
    "        loader = tf.train.import_meta_graph(save_model_path + '.meta')\n",
    "        loader.restore(sess, save_model_path)\n",
    "\n",
    "        # Get Tensors from loaded model\n",
    "        loaded_x = loaded_graph.get_tensor_by_name('x:0')\n",
    "        loaded_y = loaded_graph.get_tensor_by_name('y:0')\n",
    "        loaded_keep_prob = loaded_graph.get_tensor_by_name('keep_prob:0')\n",
    "        loaded_logits = loaded_graph.get_tensor_by_name('logits:0')\n",
    "        loaded_acc = loaded_graph.get_tensor_by_name('accuracy:0')\n",
    "        \n",
    "        # Get accuracy in batches for memory limitations\n",
    "        test_batch_acc_total = 0\n",
    "        test_batch_count = 0\n",
    "        \n",
    "        for test_feature_batch, test_label_batch in helper.batch_features_labels(test_features, test_labels, batch_size):\n",
    "            test_batch_acc_total += sess.run(\n",
    "                loaded_acc,\n",
    "                feed_dict={loaded_x: test_feature_batch, loaded_y: test_label_batch, loaded_keep_prob: 1.0})\n",
    "            test_batch_count += 1\n",
    "\n",
    "        print('Testing Accuracy: {}\\n'.format(test_batch_acc_total/test_batch_count))\n",
    "\n",
    "        # Print Random Samples\n",
    "        random_test_features, random_test_labels = tuple(zip(*random.sample(list(zip(test_features, test_labels)), n_samples)))\n",
    "        random_test_predictions = sess.run(\n",
    "            tf.nn.top_k(tf.nn.softmax(loaded_logits), top_n_predictions),\n",
    "            feed_dict={loaded_x: random_test_features, loaded_y: random_test_labels, loaded_keep_prob: 1.0})\n",
    "        helper.display_image_predictions(random_test_features, random_test_labels, random_test_predictions)\n",
    "\n",
    "\n",
    "test_model()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###### Why 66% Accuracy?\n",
    "You might be wondering why can't get an accuracy any higher. First things first, 50% or 60% isn't bad for a simple CNN. Pure guessing would get you 10% accuracy. With more techniques the result will be better."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}