labs/day22-PCA.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CS 135 day22: Principal Components Analysis\n",
    "\n",
    "\n",
    "# Objectives\n",
    "\n",
    "* Understand PCA on both toy and \"real\" image datasets\n",
    "* Get familiar with how to perform transformations to low-dim. embeddings.\n",
    "* Get familiar with how to \"reconstruct\" images\n",
    "\n",
    "# Outline\n",
    "\n",
    "* [Part 1: PCA for toy data with F=2 features](#part1)\n",
    "* [Part 2: PCA with K=10 for image data with many features](#part2)\n",
    "* [Part 3: PCA where we select K for image data with many features](#part3)\n",
    "\n",
    "# Takeaways\n",
    "\n",
    "* Understanding how we add/remove the *mean* vector is important\n",
    "* Understanding that PCA is a nested model\n",
    "* * The component vectors for K=4 are the first four components of K=5, K=6, or K=10 or ...)\n",
    "* Selecting the number of components can be done in several ways:\n",
    "* * Using the fraction of explained variance criteria, as discussed here\n",
    "* * * Can be done very fast by reusing eigenvalues used at training time anyway\n",
    "* * Use an application specific performance metric"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sklearn.datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sklearn.decomposition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import plotting libraries\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.style.use('seaborn-v0_8') # pretty matplotlib plots\n",
    "\n",
    "import seaborn as sns\n",
    "sns.set('notebook', font_scale=1.25, style='whitegrid')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.colors as mcolors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setup: Create Small Toy Dataset with F=2\n",
    "\n",
    "We will have N=21 examples, with 2 features $x_1$ and $x_2$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhMAAAIPCAYAAADJt1zEAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAArtklEQVR4nO3de1SVdb7H8c8WQcFLZokailgEXsAcNRS1lrfspI1mo43keOs4NTPLy+iM47XOTJnmNVM7mjdU1EkcK80Zb5jmjDfSJsUO0kEPOW7PoSEyRSAQnvOHCyYCcetvw7O3+/1aq+VyPw/P/vJbrnjz7Gc/22FZliUAAIA7VMPuAQAAgHcjJgAAgBFiAgAAGCEmAACAEWICAAAYISYAAIARYgIAABghJgAAgBFiAgAAGKlp9wAAXLd06VItW7bM5f3nzJmjZ5991m3PHxkZqcaNG+vQoUN39PW9evWS0+nU559/rpo17fnfz/HjxzVixIhyj/v7+ysoKEgtWrTQk08+qeHDh6tWrVrGz5eamqqLFy/qiSeeMD4W4KmICcCLxMTEaOzYsWUeS0pK0tmzZ9W7d2+1bt26zLYf/t3U2LFjVbdu3Tv++hEjRujq1auqUcP+k6IhISEaNGhQ6d+/++47ffPNN0pOTtb8+fP1/vvva8OGDbrvvvvu+Dk+/vhj/eIXv9CLL75ITOCuRkwAXqRz587q3LlzmcecTqfOnj2rPn36uPUsREXGjRtn9PWjRo1yzyBuEBISUuH3U1hYqDfeeEMbN27UL37xCyUmJsrhcNzRc3z99dcqLi42HRXwePb/egAAHsTf318zZsxQTEyMTp8+rV27dtk9EuDxiAngLjd16lRFRkbq5MmTGjx4sKKiotSrVy9dunRJkvTll1/qlVde0RNPPKF27drpkUceUb9+/bR48WLl5+eXOVZkZKQef/zx0r+/9957ioyMVFJSkhITE/XjH/9Y0dHR6tq1q2bOnKmsrKwyX9+rVy9FRkbq+vXrkm5cvxAZGal169Zp3759GjJkiB555BHFxMTo17/+tS5cuFDu+0lPT9f48ePVtWtXtW/fXiNGjNCpU6c0atQoRUZGumXNatSooZ///OeSpB07dpTZdvXqVS1dulTPPPOMfvSjHykqKko9evTQtGnT5HQ6S/cbPny4pk2bJklasWKFIiMjdfz48dLte/fu1b//+78rNjZWbdu21aOPPqoRI0Zo//79bvkegOrEyxyAjxg/frwefPBBDR8+XJcuXdIDDzygs2fPatiwYbp+/br69OmjBx54QNnZ2UpKStLy5ct1/vx5LVmy5JbHXrFihc6ePau+ffvq8ccf16FDh7R161Z98cUXSkxMvOXX79y5U2fOnFHPnj0VExOjkydPateuXfrss8+0b98++fv7S5JSUlI0atQo5ebmqnfv3goNDdXf/vY3DR8+XPfcc4/xGn1fTEyMHA6HkpOTSx/Lzc3V0KFDde7cOXXr1k3dunVTfn6+jhw5ovfee09Hjx7Vrl27FBgYqEGDBqlevXrav3+/OnXqpC5duigkJESStGTJEr399tsKDQ1Vv379VLt2baWnp+vjjz/W8ePHtXz5cvXq1cut3w9QlYgJwEeEhoZq/fr1ZS5+XLx4sXJycrRx40Y9+uijpY9PmjRJffv21b59+5STk3PLiy7Pnj2rd999V1FRUZKkCRMmaODAgTp16pROnz6tdu3aVfr1KSkpZX6AWpal0aNH6+jRo0pKStJTTz0lSXr55ZeVk5Ojt99+W3369JEk/eY3v9G4cePc/ht97dq1de+99yo7O7t0Dd59912lp6dr3LhxZS6ELS4u1vPPP6+///3vOn78uHr06FF6/UpJTJRcn5GVlaV33nlHDz/8sLZu3arAwMDS4/zxj3/U73//e33wwQfEBLwKL3MAPuLJJ58s9y6K4cOHa+7cuWVCQpLuu+8+PfzwwyouLtbly5dveezu3buXhoQkBQQEqFu3bpJU4UsVPxQREVHmh6fD4VCPHj3KfH1qaqpSU1PVrVu30pCQJD8/P02bNk1+fn63fJ7bVXJG5Nq1a5Kkrl276tVXXy13IWmNGjUUExMjScrOzq70mDVr1tS8efP02muvlQkJSerSpYsk6ZtvvnHH+EC14cwE4COaN29e7rGSH/iXL19WWlqaLly4oAsXLujzzz/XmTNnJMmldyO0bNmy3GP169eXdOPdEXfy9fXq1Svz9adPn5YktW/fvty+zZs3V5MmTcpcs+AOJRFRp04dSVKrVq3UqlUrfffddzp9+rQyMjL0j3/8Q2lpaTp27JgkqaioqNJjNmjQQP3795ckZWRk6Ny5c7p48aLOnTunkydPunQMwNMQE4CP+OFvwZL01Vdf6Y033tCePXtKL4ps3LixOnTooMaNG+vixYuyLOuWxw4ICCj3WMnbKd319SW/rTdq1KjCYwQHB7s1Jr755hvl5OTo3nvvLX2Zp6CgQMuWLdPmzZt19epVSTeiKTo6Wg8//LBOnDjh0rEPHjyoRYsWKS0tTdKNsxXh4eGKjo5Wenq6S2sGeBJiAvBRlmXpxRdfVGpqqoYNG6b+/fsrPDy89ELG5557ThcvXrR5yn8pOTtQ8kP8h0rOIrjLJ598Iknq1KlT6WPz58/Xhg0b1LVrV40ePbr0jqCStGDBApdiIiUlRb/61a9Uv359zZo1S+3bt1eLFi0UEBCgc+fO6f3333fr9wFUB2IC8FFpaWlKTU3Vo48+qldeeaXMtsLCQmVkZEhy7cxCdYiOjpYkffbZZ+W2ffvtt/qf//kftz2XZVnasGGDJOmZZ54pffyDDz5QrVq1tHz5ctWuXbvM16Snp5d+bYmKbnb14YcfqqioSJMnT9ZPfvKTWx4D8AZcgAn4qJLPnfj2229LX+KQbrxeP2fOHH377beSVGabndq3b6+IiAgdPHiwzGeDFBUVae7cuS5dm+GKoqIizZ8/X5988ok6duyo3r17l26rVauWrl+/Xu6i1J07d+rgwYOSyl4jUvL5I99fw5II+frrr8sc43//93+1aNGicvsD3oAzE4CPCgsLU4cOHfTpp59q8ODBio2NVWFhof76178qIyND9913n77++muX3s1RXV577TWNHDlSL730Uul9MY4fP64vv/xStWrVuq2gcDqdWrp0aenfCwoKlJWVpePHj8vpdCoiIkJLliwpc3bhJz/5iVasWKHBgwfr3/7t3+Tv76/Tp0/rxIkTuv/++5WVlVVmvZo2bSpJ2r59uyRp4MCB6t+/v+Lj47V48WJ9/vnnCg0N1aVLl/TRRx/J399f/v7+HrXmgCs4MwH4KIfDobfffltxcXG6cuWKNm7cqP3796t58+ZauXKlpkyZIkk6cOCAzZP+S/v27bV582Y99thjOnLkiLZs2aKGDRvqj3/8owIDAyu8yPRmnE6nli1bVvrf2rVrdfDgQYWEhOjll1/Wn/70J91///1lvmbcuHGaPHmy6tevr61bt2rHjh0qKirSyy+/rC1btkhS6RkK6cb1FiNHjtR3332njRs36tSpU4qMjFR8fLw6deqkY8eOaePGjUpNTdWAAQO0Y8cOderUSf/4xz907tw5t6wZUB0cFi/OAfACBQUFyszM1AMPPFDunhL5+fnq0KGDWrZsqT//+c82TQj4Ls5MAPAKeXl5euKJJzRkyJBy1xSsXbtWRUVFio2NtWk6wLdxZgKA15g8ebJ27Nihhx56SN27d5efn5/OnDmj5ORkNWvWTNu2bVODBg3sHhPwOcQEAK9x/fp1bd26Ve+9954uXLig/Px8NWnSRL1799ZLL73k9g/7AuAaYgIAABjhmgkAAGCEmAAAAEa4aZUL/v73v8uyrNKPIwYAwBcUFhbK4XDoRz/6UaX7cWbCBZZl+dS98i3LUkFBgU99z+7E+plh/cywfmZYv7Jc/fnHmQkXlJyRKPmgobtdbm6uUlNTFR4erqCgILvH8TqsnxnWzwzrZ4b1KyslJcWl/TgzAQAAjBATAADACDEBAACMEBMAAMAIMQEAAIwQEwAAwAgxAQAAjBATAADACDEBAACMEBMAAMAIMQEAAIwQEwAAwAgxAQAAjPCpoQAAeKmQkJBKtzudzmqZgzMTAADACGcmAABez1N+Q/dVnJkAAABGiAkAAGCEmAAAAEaICQAAYISYAAAARng3BwAAXspT3qXCmQkAAGCEMxMAAK/nKb+h+yrOTAAAACPEBAAAMEJMAAAAI1wzAQDA9wwYMKDS7VyfUR5nJgAAgBFiAgAAGCEmAACAEWICAAAYISYAAIARYgIAABghJgAAgBHuMwEAwPfs2LFDrVu3VlBQkN2jeA3OTAAAACPEBAAAMEJMAAAAI8QEAAAwQkwAAAAjxAQAADBCTAAAACPEBAAAMMJNqwAAqAYhISGVbnc6ndU0iftxZgIAABghJgAAgBFiAgAAGCEmAACAEWICAAAYISYAAICRuyom8vPz9dRTT6lXr152jwIAgM+4q+4zMW/ePJ0/f/6W7+UFAKC6efN9JG7lrjkz8de//lWbN2+Wv7+/3aMAAOBT7oqYuHz5sqZPn65BgwapWbNmdo8DAIBPuSti4pVXXpG/v79mzJhh9ygAAPgcr79m4oMPPtDevXu1fv161a1bt8qex7Is5ebmVtnxPUleXl6ZP3F7WD8zrJ8Z1s8M61eWZVlyOBy33M+rY8LpdGrWrFkaOXKkOnfuXKXPVVhYqNTU1Cp9Dk+TkZFh9whejfUzw/qZYf3MsH7/EhAQcMt9vDYmiouLNXXqVAUHB2vSpElV/nz+/v4KDw+v8ufxBHl5ecrIyFBYWJgCAwPtHsfrsH5mWD8zrJ8Z1q+s9PR0l/bz2piIj4/XiRMntHLlSl27dk3Xrl2TdCMyiouLlZ2dLX9/f9WrV88tz+dwOBQUFOSWY3mLwMBAn/ue3Yn1M8P6mWH9zLB+N7jyEofkxTFx8OBBFRcXa8yYMRVuj42NVUxMjBISEqp5MgAAfIvXxsSUKVN05cqVco9Pnz5dhYWFmj9/vurXr2/DZAAA+BavjYmoqKgKH69du7Zq1Kihrl27VvNEAAD4prviPhMAAMA+xAQAADDitS9z3Mzu3bvtHgEAAJ/CmQkAAGCEmAAAAEaICQAAYISYAAAARogJAABghJgAAABGiAkAAGCEmAAAAEaICQAAYISYAAAARogJAABghJgAAABGiAkAAGCEmAAAAEaICQAAYISYAAAARogJAABghJgAAABGiAkAAGCEmAAAAEaICQAAYKSm3QMAALxbSEhIpdudTmc1TQK7cGYCAAAYISYAAIARYgIAABghJgAAgBFiAgAAGCEmAACAEWICAAAY4T4TAAAj3EcCnJkAAABGiAkAAGCEmAAAAEaICQAAYISYAAAARogJAABghJgAAABGiAkAAGCEmAAAAEaICQAAYITbaQOADwoJCal0O7fIxu3gzAQAADBCTAAAACPEBAAAMEJMAAAAI8QEAAAwQkwAAAAjxAQAADDCfSYAwAdxHwm4E2cmAACAEWICAAAYISYAAIARYgIAABghJgAAgBFiAgAAGCEmAACAEWICAAAYISYAAIARYgIAABghJgAAgBE+mwMAvEBISEil2/msDdiJMxMAAMAIMQEAAIwQEwAAwAgxAQAAjHj9BZhpaWlatmyZkpOTde3aNTVt2lT9+/fXr371KwUEBNg9HgAAdz2vjokLFy4oLi5O/v7+ev755xUcHKxjx45p+fLlOn36tNasWSOHw2H3mAAA3NW8OiZmz56tgoIC/elPf9KDDz4oSYqLi9Mbb7yh+Ph47d+/X3369LF5SgAA7m5eGxNFRUVKTk5Wp06dSkOixMCBAxUfH6+TJ08SEwDuCtxHAp7Ma2OiRo0a2r59uyzLKrctOztbkuTn51fdYwEA4HO8NiYcDoeaN29e4bb169dLkjp37uy257MsS7m5uW47nifLy8sr8yduD+tnhvUzw/qZYf3KsizLpWsPHVZFv9p7sfj4eL3xxhvq2LGjNm/e7JZjpqSkqKCgwC3HAgDAmwQEBCg6OrrSfbz2zERFNmzYoLlz56pRo0ZauHChW4/t7++v8PBwtx7TU+Xl5SkjI0NhYWEKDAy0exyvw/qZYf3MsH5mWL+y0tPTXdrvrogJy7K0YMECrV69Wo0aNdK6devUtGlTtz6Hw+FQUFCQW4/p6QIDA33ue3Yn1s8M62eG9TPD+t3g6u0VvD4mCgoKNG3aNO3cuVOhoaFas2aNQkND7R4LAACf4dUxUVRUpIkTJyopKUnR0dFauXKlGjZsaPdYAAD4FK+OiaVLlyopKUkdOnTQ6tWrVadOHbtHAgDA53htTPzzn/8svV12r169lJSUVG6fli1bql27djZMBwCA7/DamPjkk09K3665YMGCCvf56U9/SkwAAFDFvDYm+vXrp379+tk9BgAAPq+G3QMAAADvRkwAAAAjxAQAADBCTAAAACPEBAAAMEJMAAAAI8QEAAAwQkwAAAAjxAQAADBCTAAAACPEBAAAMEJMAAAAI8QEAAAwQkwAAAAjxAQAADBCTAAAACPEBAAAMEJMAAAAIzXtHgAAvEFISEil251OZzVNAngezkwAAAAjxAQAADBCTAAAACPEBAAAMEJMAAAAI8QEAAAwQkwAAAAj3GcCAFzAfSSAm+PMBAAAMEJMAAAAI8QEAAAwwjUTADzOgAEDKt3O9QuAZ+HMBAAAMEJMAAAAI8QEAAAwQkwAAAAjxAQAADBCTAAAACPEBAAAMMJ9JgB4nB07dqh169YKCgqyexQALuDMBAAAMEJMAAAAI8QEAAAwQkwAAAAjxAQAADBCTAAAACPEBAAAMEJMAAAAI8QEAAAwQkwAAAAjxAQAADBCTAAAACPEBAAAMMKnhgK464SEhFS63el0VtMkgG/gzAQAADBCTAAAACPEBAAAMOJyTOTn52vZsmV69tln9eSTT2rChAk6fvz4Tfdft26devTo4Y4ZAQCAB3PpAsy8vDz97Gc/03/913/JsixJ0pdffqm9e/dq6NChmjlzpvz8/Mp8TU5OjjIzM90/MQAA8CguxcSKFSv0+eefa9CgQZowYYICAgK0a9cuLVu2TO+++66ysrK0ePHickEBALg9vBMF3sillzn27Nmj1q1ba86cOWrSpIkaNmyoYcOGadu2bWrRooWSkpI0ffr0qp4VAAB4IJfOTDidTg0bNqzc4w888IA2bNiguLg47dixQ40bN9akSZPcPiQA3A5+eweql0tnJgICApSTk1PhtuDgYK1atUr16tXTqlWrlJiY6NYBAQCAZ3MpJtq0aaP9+/crOzu7wu0PPviglixZopo1a+oPf/iDPvzwQ7cOCQAAPJdLMfGzn/1M33zzjQYPHqxNmzbp/Pnz5fbp0qWLZs+eLcuy9Lvf/U7vv/++24cFAACex6WYePLJJzV27FhlZmZq1qxZ2rNnT4X7/fjHP9aCBQtUu3ZtXrMEAMBHuHzTqrFjx+rPf/6zJk6cqEcfffSm+/Xr10/vv/+++vfvr/r167tlyMpkZWVpxowZeuyxx9S+fXsNHTpUhw8frvLnBQAAN9zWp4aGhYXpxRdf1PTp09W2bVsFBgbedL+4uDidPn3aLUPezLVr1zR69GhduHBBI0eOVOPGjZWYmKgxY8Zo1apV6t69e5U+PwC4G2d14Y3u6LM53nvvPT377LNKTU0tt62wsFALFizQyJEjdfHiReMBK7Nx40Z98cUXWrx4sSZNmqRhw4Zp8+bNatasmV577bXSu3UCAICqc0cxMWLECGVkZOi5557TunXrSh9PS0vT4MGDtXr1at13331atmyZu+as0Pbt29WiRQv17Nmz9LE6deroueeeU0ZGhk6dOlWlzw8AAO4wJqZPn641a9bo3nvv1dy5czVmzBi9/fbbGjJkiNLS0jRkyBD95S9/Ue/evd09b6mrV6/q/PnzateuXblt0dHRkkRMAABQDW7rmonv69q1q3bu3KmZM2dq3759Onz4sBo0aKBVq1apc+fO7pyxQpmZmbIsS02bNi23LTg4WJJ7X3u0LEu5ubluO54ny8vLK/Mnbg/rZ4b1M8P6mWH9yrIsSw6H45b73XFMSNLRo0f12WefybIs+fn56fLly9q6dasiIyPVoEEDk0Pf0tWrVyWpwotAg4KCJLn3H0NhYWGF14jczTIyMuwewauxfmZYPzOsnxnW718CAgJuuc8dxURmZqb+8Ic/6MCBAwoICNDUqVPVt29fTZs2TTt37tTRo0c1Y8YM9evX704O75Li4mJJqrCYSi68dKWmXOXv76/w8HC3Hc+T5eXlKSMjQ2FhYTd9xw5ujvUzw/qZYf3MsH5lpaenu7TfHcXE008/ratXr6pNmzaaP3++HnroIUnS+vXrtX79er355pv6zW9+o507d+o///M/7+QpbqlOnTqSpPz8/HLbSs5I1KtXz23P53A4Ss94+IrAwECf+57difUzw/qZYf3MsH43uPpL+R1dgJmbm6tf/vKXSkxMLA2JEiNHjtS2bdvUpk0bHThw4E4O75JmzZpJunGW5Ie++uorSarwegoAAOBed3RmYvPmzXrkkUduuv2hhx5SYmKili9ffseD3UrdunUVFhamlJSUcttKHqvonR4AAMC97ujMRGUhUcLPz09jx469k8O77Omnn1Z6eroOHTpU+ti1a9dKz5iUvEUUAABUHaN3c9ht9OjR2r59uyZMmKBRo0apUaNGSkxM1KVLl7Rq1Sq3XoAJAAAq5tUxUbduXW3atEnz58/Xpk2bVFhYqFatWmnNmjXq0qWL3eMBAOATvDomJKlx48ZasGCB3WMAAOCz7uiaCQAAgBLEBAAAMEJMAAAAI8QEAAAwQkwAAAAjxAQAADBCTAAAACPEBAAAMEJMAAAAI8QEAAAwQkwAAAAjxAQAADBCTAAAACPEBAAAMEJMAAAAIzXtHgDwdiEhIZVudzqd1TQJANiDMxMAAMAIMQEAAIwQEwAAwAgxAQAAjBATAADACDEBAACMEBMAAMAI95kADP3wPhK5ublKTU1V69atFRQUZNNUAFB9ODMBAACMEBMAAMAIMQEAAIwQEwAAwAgxAQAAjBATAADACG8NBXwcH6EOwBRnJgAAgBFiAgAAGCEmAACAEWICAAAYISYAAIARYgIAABjhraEAXMbbSAFUhJgAvEhV/DAnAACY4mUOAABghJgAAABGiAkAAGCEmAAAAEaICQAAYISYAAAARnhrKACX8TZSABUhJgAvwg9zAJ6IlzkAAIARYgIAABghJgAAgBFiAgAAGCEmAACAEWICAAAYISYAAIARYgIAABghJgAAgBFiAgAAGCEmAACAEWICAAAYISYAAIARYgIAABghJgAAgBFiAgAAGCEmAACAEWICAAAYqWn3ACacTqeWLFmiw4cP6/Lly7r//vvVp08fjR8/XvXr17d7PAAAfILXxsTly5cVFxenK1euKC4uTi1atFBKSoo2bdqk5ORkbdmyRYGBgXaPCQDAXc9rY2LZsmXKzMzU+vXr1aVLF0nS0KFD1apVK82aNUtbtmzRqFGj7B0SAAAf4LXXTBw5ckShoaGlIVFi4MCBkqRPP/3UjrEAAPA5XntmYu3atbp27Vq5x7OzsyVJfn5+1T0SAAA+yWtjokmTJhU+vm7dOklS586d3fp8lmUpNzfXrcf0VHl5eWX+xO1h/cywfmZYPzOsX1mWZcnhcNxyP4dlWVY1zOOyRYsWKScnp9J9xo8frwYNGpR7fPfu3fr1r3+t5s2ba+fOnapVq5ZbZkpJSVFBQYFbjgUAgDcJCAhQdHR0pft43JmJbdu2KSsrq9J9XnjhhXIxsWfPHv32t79VYGCg3nrrLbeFRAl/f3+Fh4e79ZieKi8vTxkZGQoLC+MdMXeA9TPD+plh/cywfmWlp6e7tJ/HxcThw4dv+2sSEhI0e/Zs1a5dWytXrlSbNm3cPpfD4VBQUJDbj+vJAgMDfe57difWzwzrZ4b1M8P63eDKSxySB8bE7bAsS/PmzdPatWvVsGFDvfPOO2rXrp3dYwEA4FO8OiZef/11JSQkKDQ0VKtXr1aLFi3sHgkAAJ/jtTGxdetWJSQkqGXLlkpISFCjRo3sHgkAAJ/klTHx3XffadGiRZKk3r1768iRI+X2CQ4OVmxsbHWPBgCAz/HKmEhLSyu9OdXq1asr3Kd79+7EBAAA1cArY6Jdu3ZKS0uzewwAACAv/mwOAADgGYgJAABghJgAAABGiAkAAGCEmAAAAEaICQAAYISYAAAARogJAABghJgAAABGiAkAAGCEmAAAAEaICQAAYMQrP+gLviUkJKTS7U6ns5omAQBUhDMTAADACDEBAACMEBMAAMAIMQEAAIwQEwAAwAgxAQAAjBATAADACPeZgMfjPhIA4Nk4MwEAAIwQEwAAwAgxAQAAjBATAADACDEBAACMEBMAAMAIMQEAAIwQEwAAwAgxAQAAjBATAADACDEBAACM8NkcKCMkJKTS7XxOBgDghzgzAQAAjBATAADACDEBAACMEBMAAMAIMQEAAIwQEwAAwAgxAQAAjHCfCZThdDqVm5ur1NRUtW7dWkFBQXaPBADwcJyZAAAARogJAABghJgAAABGiAkAAGCEmAAAAEaICQAAYISYAAAARrjPBHxaSEhIpdudTmc1TQIA3oszEwAAwAgxAQAAjBATAADACDEBAACMEBMAAMAIMQEAAIwQEwAAwAj3mYBP4z4SAGCOMxMAAMAIMQEAAIwQEwAAwAgxAQAAjBATAADACDEBAACMEBMAAMAIMQEAAIzcNTFhWZaGDx+uyMhIu0cBAMCn3DUxsXbtWiUnJ9s9BgAAPueuiIm0tDQtXrxYAQEBdo8CAIDP8fqYKCgo0OTJk9WxY0fFxMTYPQ4AAD7H62PizTff1KVLlzRnzhy7RwEAwCd59aeGJicna926dZozZ46aNm1apc9lWZZyc3ONjvHwww9Xuv2///u/jY7vLnl5eWX+xO1h/cywfmZYPzOsX1mWZcnhcNxyP4+LiUWLFiknJ6fSfcaPHy8/Pz9NnTpVvXv31jPPPFPlcxUWFio1NbVKn6Oqj3+7MjIy7B7Bq7F+Zlg/M6yfGdbvX1y5HtHjYmLbtm3KysqqdJ8XXnhBb731lvLz8/Xqq69Wy1z+/v4KDw+v0udo3bp1lR7fVXl5ecrIyFBYWJgCAwPtHsfrsH5mWD8zrJ8Z1q+s9PR0l/bzuJg4fPjwLffZvXu3duzYoddff12SlJ2dLUm6fv166d/9/Px0zz33uG0uh8OhoKAgtx2vIlV9/NsVGBjocTN5E9bPDOtnhvUzw/rd4MpLHJIHxoQrDh48KEmaMWNGhdtjY2MVEhKijz76qBqnAgDAN3llTIwZM0YDBgwo9/jChQt15swZxcfHq1atWjZMBgCA7/HKmAgPD6/w+oU1a9ZIkrp27VrdIwEA4LO8/j4TAADAXl55ZsJbOZ1Ou0cAAMDt7qqYKHmZAwAAVB9e5gAAAEaICQAAYISYAAAARogJAABghJgAAABGiAkAAGCEmAAAAEaICQAAYOSuummVrwoJCal0O3feBABUJc5MAAAAI8QEAAAwQkwAAAAjXDOBKsO1HADgGzgzAQAAjBATAADACDEBAACMcM3EXYBrDwAAduLMBAAAMEJMAAAAI8QEAAAwwjUTqDJcywEAvoEzEwAAwAgxAQAAjBATAADACDEBAACMEBMAAMAIMQEAAIwQEwAAwAgxAQAAjBATAADACDEBAACMEBMAAMAIMQEAAIwQEwAAwIjDsizL7iE83aeffirLshQQEGD3KNXCsiwVFhbK399fDofD7nG8DutnhvUzw/qZYf3KKigokMPhUIcOHSrdj48gd4Gv/YNyOBw+E05VgfUzw/qZYf3MsH5lORwOl34GcmYCAAAY4ZoJAABghJgAAABGiAkAAGCEmAAAAEaICQAAYISYAAAARogJAABghJgAAABGiAkAAGCEmAAAAEaICQAAYISYAAAARogJAABghJgAAABGiAkAAGCEmAAAAEaICQAAYISYgEssy9Lw4cMVGRlp9yhew+l0asqUKerevbuioqLUo0cPzZo1S1euXLF7NI+UlZWlGTNm6LHHHlP79u01dOhQHT582O6xvEZaWprGjRunzp07KyoqSk888YQWL16sgoICu0fzOvn5+XrqqafUq1cvu0fxGjXtHgDeYe3atUpOTrZ7DK9x+fJlxcXF6cqVK4qLi1OLFi2UkpKiTZs2KTk5WVu2bFFgYKDdY3qMa9euafTo0bpw4YJGjhypxo0bKzExUWPGjNGqVavUvXt3u0f0aBcuXFBcXJz8/f31/PPPKzg4WMeOHdPy5ct1+vRprVmzRg6Hw+4xvca8efN0/vx5hYSE2D2K97CAWzh79qwVFRVlRUVFWREREXaP4xVee+01KyIiwjp69GiZxzds2GBFRERY8fHx9gzmoVasWGFFRERYH330UeljOTk5Vp8+fay+fftaxcXFNk7n+V566SWrbdu21rlz58o8PmfOHCsiIsLat2+fTZN5n0OHDlmRkZFW27ZtrZ49e9o9jtfgZQ5UqqCgQJMnT1bHjh0VExNj9zhe48iRIwoNDVWXLl3KPD5w4EBJ0qeffmrHWB5r+/btatGihXr27Fn6WJ06dfTcc88pIyNDp06dsnE6z1ZUVKTk5GR16tRJDz74YJltJf/eTp48acdoXufy5cuaPn26Bg0apGbNmtk9jlchJlCpN998U5cuXdKcOXPsHsWrrF27VitWrCj3eHZ2tiTJz8+vukfyWFevXtX58+fVrl27ctuio6MliZioRI0aNbR9+3a9+uqr5bbx7+32vPLKK/L399eMGTPsHsXrcM0Ebio5OVnr1q3TnDlz1LRpU7vH8SpNmjSp8PF169ZJkjp37lyN03i2zMxMWZZV4b+x4OBgSTcuZkXFHA6HmjdvXuG29evXS+Lfmys++OAD7d27V+vXr1fdunXtHsfrEBM+ZtGiRcrJyal0n/Hjx8vPz09Tp05V79699cwzz1TPcF7A1fVr0KBBucd3796td999V6GhoRo0aFAVTeh9rl69KkkVXpAaFBQkScrLy6vWme4G8fHx+vjjj9WxY0c99thjdo/j0ZxOp2bNmqWRI0cSXneImPAx27ZtU1ZWVqX7vPDCC3rrrbeUn59f4alTX+bq+v0wJvbs2aPf/va3CgwM1FtvvaVatWpV4ZTepbi4WJIqfLeBZVk33Yab27Bhg+bOnatGjRpp4cKFdo/j0YqLizV16lQFBwdr0qRJdo/jtYgJH+PK+/Z3796tHTt26PXXX5f0r9ddr1+/Xvp3Pz8/3XPPPVU3qIe6k/seJCQkaPbs2apdu7ZWrlypNm3aVMFk3qtOnTqSbry3/4dKzkjUq1evWmfyVpZlacGCBVq9erUaNWqkdevW8RLlLcTHx+vEiRNauXKlrl27pmvXrkm6ERnFxcXKzs6Wv78//wZvgZhAOQcPHpSkm16EFBsbq5CQEH300UfVOJX3sSxL8+bN09q1a9WwYUO98847FV5k6OtKrprPzMwst+2rr76SJH4guqCgoEDTpk3Tzp07FRoaqjVr1ig0NNTusTzewYMHVVxcrDFjxlS4PTY2VjExMUpISKjmybwLMYFyxowZowEDBpR7fOHChTpz5ozi4+M5Te+C119/XQkJCQoNDdXq1avVokULu0fySHXr1lVYWJhSUlLKbSt5jAirXFFRkSZOnKikpCRFR0dr5cqVatiwod1jeYUpU6ZUeFfa6dOnq7CwUPPnz1f9+vVtmMy7EBMoJzw8XOHh4eUeX7NmjSSpa9eu1T2S19m6dasSEhLUsmVLJSQkqFGjRnaP5NGefvppLVu2TIcOHdLjjz8u6cZdMRMTE/XQQw+VvkUUFVu6dKmSkpLUoUMHrV69uvSlI9xaVFRUhY/Xrl1bNWrU4P93LiImADf77rvvtGjRIklS7969deTIkXL7BAcHKzY2trpH81ijR4/W9u3bNWHCBI0aNUqNGjVSYmKiLl26pFWrVnEBZiX++c9/lt4uu1evXkpKSiq3T8uWLTm7gypFTABulpaWVnrR6urVqyvcp3v37sTE99StW1ebNm3S/PnztWnTJhUWFqpVq1Zas2ZNubuIoqxPPvmk9MO8FixYUOE+P/3pT4kJVCmHVfLeKwAAgDvA7bQBAIARYgIAABghJgAAgBFiAgAAGCEmAACAEWICAAAYISYAAIARYgIAABghJgB4nUOHDikyMrLCW5UDqH7EBACvcvbsWU2ePNnuMQB8DzEBwGvs27dPw4cP1+XLl+0eBcD3EBMAPF5mZqZ++ctfauzYsQoICFDbtm3tHgnA9xATAGzx2WefqU2bNurYsaP+7//+r8y22bNnKzIyUtOmTZMkpaSk6MCBAxowYIA+/PBDRURE2DEygJsgJgDYon379vr5z3+unJwc/f73vy99/MiRI9qwYYPCwsI0c+ZMSVJERIR27typ+fPnq2HDhjZNDOBmiAkAthk7dqzatm2rAwcOaNeuXfr22281depU1axZUwsXLlSdOnUkSaGhoQoPD7d5WgA3Q0wAsI2/v7/mzZunWrVqafbs2Zo5c6YyMzM1ceJERUVF2T0eABcREwBsFR4erokTJ+qrr77S3r171a1bN73wwgt2jwXgNhATAGzXt29f+fn5SZI6dOggh8Nh80QAbgcxAcBWlmVp+vTpKioqUoMGDbRixQqdPXvW7rEA3AZiAoCtNmzYoGPHjql///5avHixCgsLNXnyZBUUFNg9GgAXERMAbHPu3DktWrRI9957r2bOnKnY2FgNGTJEX3zxhRYvXmz3eABcREwAsMX169f1u9/9Tvn5+Zo5c2bp/SOmTJmi4OBgxcfH68SJEzZPCcAVxAQAWyxfvlxnzpxRz5499fTTT5c+Xq9ePf3Hf/yHiouLNWXKFOXk5Ng4JQBXOCzLsuweAgAAeC/OTAAAACPEBAAAMEJMAAAAI8QEAAAwQkwAAAAjxAQAADBCTAAAACPEBAAAMEJMAAAAI8QEAAAwQkwAAAAjxAQAADBCTAAAACP/D09lhA4SM66IAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 800x550 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "random_state = np.random.RandomState(0)\n",
    "\n",
    "# Create N training x values between -5 and 5\n",
    "N = 21\n",
    "F = 2\n",
    "\n",
    "x1_N = np.linspace(-5, 5, N) + random_state.uniform(low=-0.2, high=0.2, size=N)\n",
    "\n",
    "B = 1.5\n",
    "x2_N = 1.0 * x1_N + random_state.uniform(low=-B, high=B, size=N)\n",
    "\n",
    "x_NF = np.zeros((N,F))\n",
    "x_NF[:,0] = x1_N\n",
    "x_NF[:,1] = x2_N\n",
    "\n",
    "# Plot the training data\n",
    "plt.plot(x1_N, x2_N, 'ks')\n",
    "plt.title('Training Data')\n",
    "plt.xlabel('x1');\n",
    "plt.ylabel('x2');\n",
    "plt.gca().set_aspect('equal', 'box')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Review: Math of Principal Components Analysis (PCA)\n",
    "\n",
    "We'll assume feature vectors  $x \\in \\mathbf{R}^F$.\n",
    "\n",
    "We want to find a principal components analysis model with $K$ components, which has parameters:\n",
    "* $m \\in \\mathbb{R}^F$ : mean vector\n",
    "* $W \\in \\mathbb{R}^{F \\times K}$ : weight vectorss for each component\n",
    "\n",
    "This model makes projections to the latent \"codes\" $z_n \\in \\mathbb{R}^K$ like this:\n",
    "$$\n",
    "z_n = \\mathbf{W}^T (\\mathbf{x}_n - \\mathbf{m})\n",
    "$$\n",
    "\n",
    "And then makes reconstructions like this:\n",
    "$$\n",
    "\\hat{x}_n(\\mathbf{m}, \\mathbf{W}) = \\mathbf{W} z_n + \\mathbf{m}\n",
    "$$\n",
    "\n",
    "The training objective is:\n",
    "$$\n",
    "\\min_{\\mathbf{m} \\in \\mathbb{R}^F, \\mathbf{W} \\in \\mathbb{R}^{F\\times K}}\n",
    "\\quad \\sum_{n=1}^N \\sum_{f=1}^F (\\mathbf{x}_{nf} - \\hat{\\mathbf{x}}_{nf}(\\mathbf{m}, \\mathbf{W}) )^2\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 1: PCA for F=2 data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fit a pca model with K=1 component to our data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pca = sklearn.decomposition.PCA(n_components=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {color: black;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>PCA(n_components=1)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">PCA</label><div class=\"sk-toggleable__content\"><pre>PCA(n_components=1)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "PCA(n_components=1)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pca.fit(x_NF)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This fitting process estimates two parameters, $m$ and $W$, and stores them as attributes of `pca`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inspecting the learned Mean vector $m$\n",
    "\n",
    "This vector is accessible as the `mean_` attribute"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2,)\n",
      "\n",
      "[0.04018504 0.13658959]\n"
     ]
    }
   ],
   "source": [
    "m_F = pca.mean_\n",
    "print(m_F.shape)\n",
    "print(\"\")\n",
    "print(m_F)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1a: Can you convince yourself that this is the same as the training set's empirical mean vector?\n",
    "\n",
    "Compute the mean of each feature (each column) in the training set `x_NF`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inspecting the learned weights $W$\n",
    "\n",
    "This vector is accessible as the `components_` attribute of our learned `pca` model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 2)\n",
      "\n",
      "[[0.69601341 0.71802879]]\n"
     ]
    }
   ],
   "source": [
    "w_KF = pca.components_\n",
    "print(w_KF.shape)\n",
    "print(\"\")\n",
    "print(w_KF)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1b: Can you convince yourself that this weight vector is a unit vector?\n",
    "\n",
    "That is, is the sum-of-squares magnitude of each component vector (each row) equal to one?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO compute sum of squares of each row of w_KF"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Calling `transform`: Obtaining the low-dimensional embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z_NK = pca.transform(x_NF)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(21, 1)\n",
      "\n",
      "[[-6.52423243]\n",
      " [-6.4504965 ]\n",
      " [-5.11980459]\n",
      " [-5.87207543]\n",
      " [-4.10995134]\n",
      " [-4.34687772]\n",
      " [-2.03157546]\n",
      " [-1.97845129]\n",
      " [-1.46165685]\n",
      " [-1.40616067]\n",
      " [ 0.62968306]\n",
      " [ 0.50286399]\n",
      " [ 1.47389739]\n",
      " [ 1.19917502]\n",
      " [ 2.71280757]\n",
      " [ 3.41699812]\n",
      " [ 4.09659534]\n",
      " [ 5.96711016]\n",
      " [ 6.07911096]\n",
      " [ 6.14379828]\n",
      " [ 7.07924238]]\n"
     ]
    }
   ],
   "source": [
    "print(z_NK.shape)\n",
    "print(\"\")\n",
    "print(z_NK)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Compute the reconstructions\n",
    "\n",
    "Each training example (index $n$) can be reconstructed as:\n",
    "\n",
    "$$\n",
    "\\hat{x}_n(\\mathbf{m}, \\mathbf{W}) = \\mathbf{W} z_n + \\mathbf{m}\n",
    "$$\n",
    "\n",
    "If we convert this to matrix-vector arithmetic for the full dataset, we can do:\n",
    "\n",
    "$$\n",
    "\\hat{X} = \\mathbf{Z} \\mathbf{W} + \\mathbf{m}\n",
    "$$\n",
    "\n",
    "where $\\hat{X}$ is an $N \\times F$ matrix, where each row is equal to $\\hat{x}_n(\\mathbf{m}, \\mathbf{W})$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xhat_NF = np.dot(z_NK, w_KF) + m_F[np.newaxis,:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(21, 2)\n",
      "\n",
      "First 5 rows of this array\n",
      "[[-4.5007682  -4.54799709]\n",
      " [-4.449447   -4.49505257]\n",
      " [-3.52326759 -3.53957748]\n",
      " [-4.04685818 -4.07972959]\n",
      " [-2.82039619 -2.81447377]]\n"
     ]
    }
   ],
   "source": [
    "print(xhat_NF.shape)\n",
    "print()\n",
    "print(\"First 5 rows of this array\")\n",
    "print(xhat_NF[:5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualize the reconstructions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAAH4CAYAAABwss2HAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABeu0lEQVR4nO3dd3RUZeLG8edmZgKT0BNaIiRAIHQLCiSgPyl2mq6iqAh2BQEB6ShVUBQWlF2QLigqFgRxFxXBRrWsCAIBhAAGpYSeDGTK/f3BMBoTIMRJZib5fs7xKHdueXjlkDy5732vYZqmKQAAAACAwgIdAAAAAACCBQUJAAAAALwoSAAAAADgRUECAAAAAC8KEgAAAAB4UZAAAAAAwIuCBAAAAABeFCQAAAAA8LIGOkAo+N///ifTNGWz2QIdBQCAv83pdMowDF155ZWBjgIAQYc7SHlgmqZM0wx0jEJhmqaysrKKze/XXxi3/GHc8odxyx/G7Q/F6esaAFwq7iDlwbk7R40aNQpwkoKXmZmprVu3KiEhQREREYGOEzIYt/xh3PKHccsfxu0PmzZtCnQEAAha3EECAAAAAC8KEgAAAAB4UZAAAAAAwIuCBAAAAABeFCQAAAAA8KIgAQAAAIAXBQkAAAAAvChIAAAAAOBFQQIAAAAALwoSAAAAAHhRkAAAAADAi4IEAEAAmaYpj9Mp0zQDHQUAIMka6AAAABRHGal7tH/JR0pfu05uh0MWu11RSc0V07G9IuPjAh0PAIotChIAAIXs8Oo12j5pikyXy7fN7XDo4MpVOvTV16rTr4+iWyQHMCEAFF9MsQMAoBBlpO7JUY7+zHS5tH3SFGWk7inkZAAAiYIEAECh2r/ko/OWo3NMl0v7ly4rpEQAgD+jIAEAUEhM01T62nV52jd9zVoWbgCAAKAgAQBQSEyXS26HI0/7uh2Oi95pAgD4HwUJAIBCYlitstjtedrXYrfLsLKWEgAUNgoSAACFxDAMRSU1z9O+UclJMgyjgBMBAP6KggQAQCGK6dg+T3eGyjZqWAhpAAB/RUECAKAQRcbHqU6/PhctSb9Mn6HjP/9cSKkAAOcwuRkAgEIW3SJZ9thY7V+6TOlr1srtcMhitysqOUmVb7pBe998S8c3/qTvBw3ThJ0p2nTyRI5zpKWlBSA5ABR9FCQAQNCLjY294OehWBYi4+NUu3dPJfTqIdPlkmG1+p45qj98iP51w826qlx5DapdVxN/2a7/HT8W2MAAUEwwxQ4AgAAyDENhNlu2BRnCwsP18i/bteHoEYWHhWlArTq6ulz5AKYEgOKDggQAQBBymab+uWuH1hxJlzUsTA9Vjw90JAAoFphiBwAIKT3ia6mM1aqpu3/RKXfRfpGq2zT1yq4d2p2ZoRNOZ6DjAECxwB0kAEBIqVOqlK4qV14jEuuprNUW6DgFziNpye/7tSr9UKCjAECxQEECAISUl3am6EhWluIiIjUisb7K24p+SQIAFB4KEgAgpKSdPq0RKT/r8Jkzusxu16jEBjpziLsrAAD/4BkkAEDIOXDmjEakbNFzifVUpWRJbRr6nBqOHamSlSsHOprfhOLS5QBQFFCQAABB73xl4cyhw9r83Eid3v+bNg15Vg3HjJQ9NqaQ0wEAihKm2AEAQlaJitFq9PwY2S+7TFnp6do07Fll7t0X6FgAgBBGQQIAhLTwCuXV8PnRioiPk/PoMW0a9pwydqcGOhYAIERRkAAAIS+8XFk1HDNKkbVqyXXihDYPH6GTO3YGOhYAIARRkAAARYKtTGk1HD1CpRPryHXqlH5+bpRObEvJsV/t2rUVGxt73n8AAMUbBQkAUGRYS0Wq/sjnVKZ+PbkzM/XziNE6/vPPgY4FAAghFCQAQJFijbCr/ojhKnt5Y3lOn9aWkWN17MeN+TqXaZryOJ0yTdPPKQEAwYplvgEARY6lZEnVHz5E2154SUe//0Fbxo5Xjb69pZIlfPuUsVr1TEKiNh4/pvd/y76MeEbqHu1f8pHS166T2+GQxW5XVFJzxXRsr8j4uML+7QAAChF3kAAARVJYeLjqDhmoCs2aynQ6tXviZLn/9ExSGatNdUuV1t2x1fRgtXgZ3u2HV6/Rxv4DdXDlKrkdDkmS2+HQwZWrtLH/QB1evSYAvxsAQGGhIAEAiqwwm02JA/srqkWyTLdbznc/UFL5KEnSr6cdei11lzymqVsqV9GjcTV0TbnySnlpkkyXK9fzmS6Xtk+aoozUPYX52wAAFCIKEgCgSAuzWpXY/2mVv7aFZJrqUzNB11aIliR9fvigpqX+Io9pqm3FynqmVh3pIs8bmS6X9i9dVhjRAQABQEECABR5hsWiuCcek+XKKxRmGOpZo5ZaR1eUJH2Zflhv/rpXpmnKMIyLnOms9DVrWbgBAIooChIAoFgwwsJkbXeLom9sqzDD0BPxtfTdjFnavnqNHr2mWZ7LkXT2maTzTcMDAIQ2VrEDABQbhmHosu4PKNxu1/4lH2nXjFmSYVx0Wt1fWex2GVa+hAJAUcQdJABAsWIYhuIf7KbKN7Q9uyEfU+WikpMu6Y4TACB0UJAAAMWOYRgy3e78HRwWppgO7fwbCAAQNChIAIBixzRNpa9dl7+DPR4d/f4H/wYCAAQNChIAoNgxXS7fS2AvRVSLJEnSnvlvaO/bi1jJDgCKIAoSAKDYMaxWWez2vB9gsShxYH/VHfiM4rreJ0na99Y72rPgTUoSABQxFCQAQLFjGIaikprnad+SVavqikkvKbpFsiTpsjvvUI2HH5Qkpb2/WLtnz6MkAUARQkECABRLMR3bX3ypbotFdQcPUGR8XPZjO7RTzScelST99tEy7XptpkyPp6CiAgAKEQUJAFAsRcbHqU6/PuctSYbVqsT+T+coR+dUveVmJfTqIRmGfv/vJxrb8v90WWysYnP5BwAQOnjLHQCg2IpukSx7bKz2L12m9DVr5XY4ZLHbFZWcpJgO7c5bjs6p3LaNDKtNKZMmq3XFSrKFGfrX7l/EvSQACF0UJABAsRYZH6favXsqoVcPmS6XDKv1kl4CW+n66/TkUz3Vu0aCro2qKKsRpld275Sb55IAICQxxQ4AAJ1duCHMZrukcnTOuqNHNPGXHXJ6PEqqEKV7YqoVQEIAQGGgIAEA4AffHz+qCTtTlJ51RidczkDHAQDkE1PsAADwk40njuvJn/4X6BgAgL+hSN1BOn36tG655Ra1bt060FEAAAAAhKAiVZAmTJigXbt2BToGAAAAgBBVZKbYff3111q4cKFsNlugowAAipm0tLRARwAA+EmRuIN07NgxDR06VLfffrsuu+yyQMcBAAAAEKKKREF67rnnZLPZNGzYsEBHAQAAABDCQn6K3YcffqhPP/1Ur7/+ukqVKhXoOAAAAABCWEgXpLS0NI0dO1bdunVTs2bNCvRapmkqMzOzQK8RDBwOR7Z/I28Yt/xh3PKHccsfxu0Ppmnm64W4AFAcGKZpmoEOkR8ej0fdunVTenq6Fi9erBIlSkiSbr75ZmVlZWnlypV+u9amTZuUlZXlt/MBABBo4eHhatSoUaBjAEDQCdk7SHPnztV3332nGTNmKCMjQxkZGZLOFiePx6MjR47IZrOpdOnSfrmezWZTQkKCX84VzBwOh1JTUxUfHy+73R7oOCGDccsfxi1/GLf8Ydz+sHPnzkBHAICgFbIF6YsvvpDH49EjjzyS6+dJSUlq2rSpFixY4JfrGYahiIgIv5wrFNjt9mL1+/UXxi1/GLf8Ydzyh3ET0+sA4AJCtiANGjRIJ06cyLF96NChcjqdeumll1SmTJkAJAMAAAAQqkK2IDVs2DDX7SVLllRYWJiSk5MLOREAAACAUFck3oMEAAAAAP5AQQIAAAAAr5CdYnc+y5cvD3QEAAAAACGKO0gAAAAA4EVBAgAAAAAvChIAAAAAeFGQAAAAAMCLggQAAAAAXhQkAAAAAPCiIAEAAACAFwUJAAAAALwoSAAAAADgRUECAAAAAC8KEgAAAAB4UZAAAAAAwIuCBAAAAABeFCQAAAAA8KIgAQAAAIAXBQkAAAAAvChIAAAAAOBFQQIAAAAALwoSAAAAAHhRkAAAAADAi4IEAAAAAF7WQAcAAAS/2NjYC36elpZWSEkAAChY3EECAPwtFsOQaZqBjgEAgF9wBwkAcElqRETIZoSpTcVKala+giIsVq3v0lVRSc0V07G9IuPjAh0RAIB8oyABAPIspmRJvVi/sUzTlGEYvu1uh0MHV67Soa++Vp1+fRTdIjmAKQEAyD+m2AEA8qxEWFiOcvRnpsul7ZOmKCN1TyEnAwDAPyhIAIA8u7lSlfOWo3NMl0v7ly4rpEQAAPgXBQkAkGfNylfI037pa9aycAMAICRRkAAAeWIxDEVY8vboqtvhkOlyFXAiAAD8j4IEALiotLQ07dm3Txa7PU/7W+x2GVbWAQIAhB4KEgAgTwzDUFRS8zztG5WcdNFnlQAACEYUJABAnsV0bH/RO0OG1aqYDu0KKREAAP5FQQIA5FlkfJzq9Otz3pJkWK2q068PL4sFAIQsJogDAC5JdItk2WNjtX/pMqWvWSu3wyGL3a6o5CTFdGhHOQIAhDQKEgDgkkXGx6l2755K6NVDpsslw2rlmSMAQJFAQQIA5JthGDJstkDHAADAb3gGCQAAAAC8KEgAAAAA4EVBAgAAAAAvChIAAAAAeLFIAwAUUbGxsRf8PC0trZCSAAAQOriDBADFlMfplGmagY4BAEBQ4Q4SABQDVsNQVHi4wsPC1K5yVTUrX0Fr77zn7Atek5orpmN7XvAKAIAoSABQLDwZX1PXRlWU2+ORJeyPyQNuh0MHV67Soa++Vp1+fRTdIjmAKQEACDym2AFAMXDgzBlJylaO/sx0ubR90hRlpO4pzFgAAAQdChIAFANR4eEX3cd0ubR/6bJCSAMAQPCiIAFAMdCsfIU87Ze+Zi0LNwAAijUKEgAUcRbDUIQlb4+cuh0OmS5XAScCACB4UZAAoIhKS0tTWlqa9uzbJ4vdnqdjLHa7DCvr9wAAii8KEgAUcYZhKCqpeZ72jUpOkmEYBZwIAIDgRUECgGIgpmP7i94ZMqxWxXRoV0iJAAAIThQkACgGIuPjVKdfn/OWJMNqVZ1+fXhZLACg2GOiOQAUE9EtkmWPjdX+pcuUvmat3A6HLHa7opKTFNOhHeUIAABRkACgWImMj1Pt3j2V0KuHTJdLhtXKM0cAAPwJBQkAiiHDMGTYbIGOAQBA0OEZJAAAAADwoiABAAAAgBcFCQAAAAC8KEgAAAAA4MUiDQAQJGJjYy/4eVpaWiElAQCg+OIOEgAAAAB4UZAAIEhZ//J+Io/TKdM0A5QGAIDigSl2ABCErouK1hNxNfX54YMKDwtTs/IVtPbOe2Sx21W26dXy1E2U6tULdEwAAIqckC9IKSkpmjp1qjZs2KCMjAxVrVpVt912m3r06KHw8PBAxwOAfHG43bKGhemmSlWybXc7HDry5dfSN2t0tGRJRbRuFaCEAAAUTSFdkPbu3asuXbrIZrPp3nvvVaVKlbRu3TpNmzZNP/30k2bPni3jL1NUACAU/H7mtNwejyxh55kJ7XZrz7+mq1zNmoqMjyvccAAAFGEh/QzSuHHjlJWVpbfeekt9+vRRly5dNGXKFD344INavXq1Pv/880BHBIB8aVe56vnLkZfpcmn/0mWFlAgAgOIhZAuS2+3Whg0bdPXVV6tmzZrZPuvYsaMk6fvvvw9ENAD425qVr5Cn/dLXrGXhBgAA/Chkp9iFhYVpyZIluX5jcOTIEUmSxWIp7FgAkG/n3nPkcTq19s578nSM2+GQ6XLJsNkKMhoAAMVGyBYkwzBUrVq1XD97/fXXJUnNmjUrzEgA4BeG1SqL3S63w3HRfS12uwxryP5VDgBA0ClyX1Xnzp2rL7/8Uk2aNNG1117rt/OapqnMzEy/nS9YObzfkDny8I0Z/sC45Q/jdn5lm159drW6i+53DeOXR/x5+4NpmixiBADnYZhFaPL6/PnzNW7cOEVHR+vdd99V1apV/XLeTZs2KSsryy/nAoC88Bw4qKxZcyW3+/w7WSwKf+RBhVWuVHjBUGSEh4erUaNGgY4BAEGnSNxBMk1TL7/8smbNmqWKFStq3rx5fitH59hsNiUkJPj1nMHI4XAoNTVV8fHxstvtgY4TMhi3/GHcLqBePR0tWVJ7/jVdpsuV83OLRTGPPazK1/nvTnlRx5+3P+zcuTPQEQAgaIV8QcrKytKQIUO0bNkyVa9eXbNnz1b16tX9fh3DMBQREeH38wYru91erH6//sK45Q/jlruI1q1UrmZN7V+6TOlr1srtcMhit6ts02t0qm4dVb7uWsYtH/jzJqbXAcAFhHRBcrvd6tu3r1asWKFGjRppxowZqlAhb0vjAkAoiIyPU+3ePZXQq8fZ1eqsVjkcDm3dujXQ0QAAKJJCuiC9+uqrWrFiha666irNmjVLkZGRgY4EAAXCMAyW8gYAoBCEbEE6dOiQZs+eLcMw1Lp1a61YsSLHPjVq1FDjxo0DkA4AAABAKArZgvTtt9/6VpZ7+eWXc93n7rvvpiABAAAAyLOQLUi33nqrbr311kDHAAAAAFCEhAU6AAAAAAAECwoSAAAAAHhRkAAAAADAi4IEAAAAAF4UJAAAAADwoiABAAAAgBcFCQAAAAC8KEgAAAAA4EVBAgAAAAAvChIAAAAAeFGQAAAAAMCLggQAAAAAXhQkAAAAAPCiIAEAAACAFwUJAAAAALwoSAAAAADgRUECAAAAAC8KEgAAAAB4WQMdAACCSWxsbK7bLYYht2kqLS2tkBMBAIDCREECgPOoZrerXeWqala+giIsVmW6XdoxZapiOrZXZHxcoOMBAIACQEECgFw0L19BvWskyBr2x0zkCItVB1eu0qGvvladfn0U3SI5gAkBAEBB4BkkAPiLanZ7jnL0Z6bLpe2TpigjdU8hJwMAAAWNggQAf9GuctXzlqNzTJdL+5cuK6REAACgsFCQAOAvmpWvkKf90teslWmaBZwGAAAUJgoSAPyJxTAUYcnb45luh0Omy1XAiQAAQGGiIAEo9kzTlMfplGmacpumMt15Kz0Wu12GlbVuAAAoSvjKDqDYykjdo/1LPlL62nVyOxyy2O36YuAQuU6d0pEN3170+KjkJBmGUQhJAQBAYaEgASiWDq9eo+2TpmSbIud2OHRw5SoZFosUFiZ5POc93rBaFdOhXWFEBQAAhYgpdgCKnYzUPTnK0Z+ZbvfZO0MWS66fG1ar6vTrw8tiAQAogriDBCDgateufcHP09LS/Hq9/Us+uujiCqbbrQrNmspaqpTS16z1TcGLSk5STId2lCMAAIooChKAYsU0TaWvXZenfY//tEnN3lqghF49ZLpcMqxWnjkCAKCIY4odgGLFdLnkdjjytO+5ZbwNw1CYzUY5AgCgGKAgAQgakRaL2lWuqqjw8AK7hmG1ymK352lflvEGAKD4oSABCBpJ5aP0QLU4ja3bQFVLlCyQaxiGoaik5nnal2W8AQAofihIAILG+mNHtM+RqajwEhpVt74uK5m3Oz2XKqZj+zzdGarQ9OoCuT4AAAheFCQAQeOky6WRKVuUmpmhcrZwjUysrzh7hN+vExkfpzr9+ly0JO3813Rl7E71+/UBAEDwoiABCConXS6NStminRmnVMZm04jE+jq5Y6ffrxPdIlmXT5ygSm1a+55JstjtqtSmtRqOG6PIWrXkOnFCm4ePKJDrAwCA4MTTxwACbseOHYqIyH6nyJWRoS2jnpdSUvTzc6NU/7lhKlOvrl+vGxkfp9q9e+a6jHfD0SO0ZfRYnUzZfvb6I4arTN1Ev14fAAAEH+4gAQhK1shI1R/5rMo0bCB3ZqZ+HjlGxzf/XCDXym0Zb2upSNUf+ZzK1K939vojRuv4zwVzfQAAEDwoSACCljXCrvrPDVO5Ky6X5/RpbRk1Vsd+3Fi41x8xXGUvb3z2+iML9/oAAKDwUZAABDVLiRKqN2ywyl/dRJ6sLG0ZO15Hvvu+8K5fsqTqDx+i8k2uCsj1AQBA4aIgAQh6YeHhqjt4gCo0bybT6dS28ROUvnZ9tn1M05TH6ZRpmgVz/SEDVaFZ0z+uv279xQ8EAAAhh4IEICSE2WxKHNBP0de2kOlyaduEl3Xoq2+UkbpHO6ZM1fouXbX2znu0vktX7ZgyVRmpe/x//YH9FdUi+ez1X3xZh75e7ddrAACAwGMVOwAhI8xqVZ2+fRRms+ngyi+0fdJkyTAkj8e3j9vh0MGVq3Toq69Vp18fRbdI9uv1E/s/rR02mw598aW2T5os0+VUpVbX++0aAAAgsLiDBCCkGBaLEnr1VFRSc8k0s5WjPzNdLm2fNMXvd5IMi0W1e/dU5RvaSh6PdkyZqt8/XeHXawAAgMChIAEIOUZYmIwSJS66n+lyaf/SZf6/vsWiWj0eV5Vbb5ZMU7/8a5p++/i/fr8OAAAofBQkACElI3WPtk+ZqsNffJmn/dPXrC2QhRuMsDDVfOwRxXRsL0naNWOW0pYs9ft1AABA4eIZJAAh4/DqNdo+aYpMlyvPx7gdDpkulwybze95DMNQ/IPdFGaz6df3PlDqnNflyXKq2l3/8Pu1AABA4eAOEoCQkJG655LLkSRZ7HYZ1oL7WZBhGIrrep+q33uPJGnvGwu15823CuSuFQAAKHgUJAAhYf+Sjy65HElSVHKSDMMogETZVbv7LsV16ypJ+nXRe9rz+gJKEgAAIYiCBCDomaap9LXrLvk4w2pVTId2BZAod5fd0Uk1HnlIkpS2eIl2z5xDSQIAIMTkuSCdPn1aU6dO1R133KGbbrpJffr00fr153+T/Lx583T99df7IyOAYs50ueR2OC7pGKfHo4kpWxQZH1dAqXIX0/421erxuGQY+u3j/+iXaa/JPM9S5AAAIPjkaWK+w+HQ/fffry1btvh+Grpnzx59+umnuueeezR8+HBZLJZsx5w6dUoHDhzwf2IAxY5htcpit+epJJmmqS/SD2nZgd+07xJLlb9UuelGhdls2vHqv3Xgk89kOp1KeKqHjL/8PQkAAIJPnu4gTZ8+XT///LM6deqkL774QmvWrNGzzz6rcuXK6e2339bTTz8tt9td0FkBFFOGYZx9MWwefJF+SNNSdwWsHJ1TqXUr1enbWwoL08GVX2j7P6fIk49nqAAAQOHKU0H65JNPVK9ePY0fP15VqlRRhQoVdN999+n9999XXFycVqxYoaFDhxZ0VgDFWEzH9gW6Gl1BqHjdtao7sL8Mq1WHv16t7S9PksfpDHQsAABwAXkqSGlpaWrWrFmO7TExMZo/f75iYmK0dOlSTZo0ye8BAUCSIuPjVKdfn/OWJJfHI9M0dWXZcoUb7CKikpqr7uABMqxWpa9dr20vvCRPVlagYwEAgPPIU0EKDw/XqVOncv2sUqVKmjlzpkqXLq2ZM2dq0aJFfg0IAOdEt0jW5RMnqFKb1rLY7ZKkTLdLqw4f1KCtmzRk62Y9v31bgFPmVOGaq1Vv+BCFhYfr6Hffa+vzL8h95kygYwEAgFzkab5K/fr19fnnn6tfv36qUKFCjs9r1qypV155RY8++qhGjRolu/cbFwDwt8j4ONXu3VMJvXrIdLlULT4+0JHypPyVV6j+c8O0Zex4Hftxo2bdfJte3JmiM7mscJeWlhaAhAAAQMrjHaT7779fR48e1Z133qk333xTu3btyrFP8+bNNW7cOJmmqYEDB2rx4sV+DwsA5xiGoTCbLdAxLknZRg3VYOSzynS71LBMWQ2rU092VrYDACCo5Kkg3XTTTXrqqad04MABjR07Vp988kmu+7Vv314vv/yySpYsyU9AARSKtLS0C/4TbMrUq6sx27fqlMuluqVK69k69RRJSQIAIGjk+UWxTz31lD7++GP17dtX11xzzXn3u/XWW7V48WLddtttKlOmjF9CAkBR8ktGhkanbNEJp1MJkaX0XGJ92Qwj0LEAAIAuoSBJUnx8vB577DF98MEHclzgHSPx8fHq0qULBQkAziPVkalRKVt0zJmlGhGRqh4REehIAABAl1iQzvnggw90xx13aOvWrTk+czqdevnll9WtWzf9+uuvfzsgABRV+047NHDLJj2/fat+ycgIdBwAAKB8FqQHHnhAqamp6ty5s+bNm+fbnpKSojvvvFOzZs1SVFSUpk6d6q+c53X48GENGzZM1157ra644grdc889Wr16dYFfFwD84ZjTqY0njgc6BgAA8MrXa+mHDh2q66+/XoMHD9aLL76ob775RldeeaVee+01ZWVl6a677tKgQYNUqlQpf+fNJiMjQw8++KD27t2rbt26qXLlylq0aJEeeeQRzZw5Uy1btizQ6wMAAAAoWvJVkCQpOTlZy5Yt0/Dhw/XZZ59p9erVKleunGbOnKlmzZr5M+N5vfHGG9q+fbumT5+uVq1aSZI6deqkTp06acyYMVq+fLkMHnwGEGSCcXU9AABwVr6m2J2zdu1a/fjjjzJNU2FhYTp27JjeffddHTt2zE/xLmzJkiWKi4vzlSNJioyMVOfOnZWamqqNGzcWSg4AAAAARUO+CtKBAwfUo0cPPf300zp+/LgGDx6szz77TE2bNtWyZct022236T//+Y+/s2Zz8uRJ7dq1S40bN87xWaNGjSSJggQAAADgkuSrILVr104rV65UvXr19MEHH6h79+6KiYnR66+/riFDhigjI0P9+/dXjx49/J3X58CBAzJNU1WrVs3xWaVKlSQxjQUAAADApcnXM0iZmZl68skn1bNnT1mt2U/RrVs3tWzZUgMHDtSqVav8EjI3J0+elCTZ7fYcn0V43ydyoXc1XSrTNJWZmem38wWrc2Pmz7ErDhi3/GHc8odxyx/G7Q+mafKMLgCcR74K0sKFC3X55Zef9/NatWpp0aJFmjZtWr6DXYzH45GkXP+CN03zvJ/ll9PpzPW9T0VVampqoCOEJMYtfxi3/GHc8odxOys8PDzQEQAgKOWrIF2oHJ1jsVj01FNP5ef0eRIZGSlJOn36dI7Pzv10sHTp0n67ns1mU0JCgt/OF6wcDodSU1MVHx+f69055I5xyx/GLX8Yt/xh3P6wc+fOQEcAgKCV72W+A+2yyy6TdPZZpL86ePCgJOX6fFJ+GYbhm7pXHNjt9mL1+/UXxi1/GLf8Ydzyh3Hz7wwLAChq/tYy34FUqlQpxcfHa9OmTTk+O7cttxXuAAAAAOB8QrYgSWdX09u5c6e++uor37aMjAwtWrRItWrV8i33DQAAAAB5EbJT7CTpwQcf1JIlS9SnTx91795dFStW1KJFi7R//37NnDmTKQQAAAAALklIF6RSpUrpzTff1EsvvaQ333xTTqdTdevW1ezZs9W8efNAxwMAAChSXn31VU2dOjVP+44fP1533HHH377m4MGDtXjxYn3zzTeqWLHiJR3btWtX/fjjj7k+klHQunbtqg0bNmTbZrPZFBkZqcTERN1zzz269dZb//Z19u7dq+rVq//t8+APIV2QJKly5cp6+eWXAx0DAACgyLvhhhtyfDM+cOBA1axZU0888US27VdddZVfrnn33XcrKSlJZcqUueRjn3jiCR09etQvOfJrwoQJvv8+c+aMfv/9d/3nP/9R3759tXbtWo0ZMyZf5z116pQeeeQR1alTR6NHj/ZXXKgIFCQAAAAUjrp166pu3brZtg0cOFDR0dHq2LFjgVzzyiuv1JVXXpmvY1u0aOHnNJcut3F54okn1KtXLy1atEiNGjVS586dL/m8x44d0//+9z/VqVPHHzHxJxQkAACAIBIbG3vBz9PS0gopCQpKeHi4XnzxRbVu3Vr//ve/deeddyosLKTXTitS+D8BAACAAtG1a1fdcccdWrBggZo1a6Yrr7xS8+fPl3T2XZajR4/WDTfcoEaNGumKK65Qx44d9dZbb2U7x+DBg5WYmKhDhw5Jkj744AMlJibqhx9+0NixY9WyZUs1atRIHTp00NKlS3Nc/8+rGr/66qtKTEzUnj17NGjQIDVr1kyNGzfW3Xffra+//jpH/pUrV+ruu+/WlVdeqZYtW2rChAl69913lZiYqPXr1/+tsSlXrpxatWql3377TVu3bvVtP378uCZOnKjbbrtNl19+uRo3bqxbb71V//73v+V2u31j0KZNG0nSO++8ky3PmTNnNH36dN1+++268sor1bBhQ7Vt21YvvviiHA7H38pcXHAHCQAAAAVm9+7dmjZtmp544gllZGSoefPmOn78uO666y6dOXNG9957r2JiYnTo0CG9++67GjlypKKionTjjTde8LzPPPOMypUrp0ceeUQej0evv/66BgwYoCpVqqhp06YXPPbBBx9UtWrV1Lt3bx0/flxz587V448/ruXLl/uesfrwww81ePBg1apVS7169VJmZqYWLlzoKyn+kJiYqGXLlmnLli1q0KCBnE6nHnjgAe3Zs0ddunRRzZo1dfz4cS1evFhTpkyR3W7Xgw8+qGuuuUZDhgzR+PHj1bx5c91xxx2qVauWJKlXr1765ptv1LlzZ3Xp0kWZmZlavny55syZI6fTqeHDh/stf1FFQQIAAECByczM1Lhx43TLLbf4ts2bN08HDhzQ7Nmz1bJlS9/2G2+8Ubfddps+/vjjixak6OhovfXWW7JYLJKkyy+/XPfee68+/PDDixakhg0b6pVXXvH9+rLLLtOAAQP00UcfqWfPnnI4HBo3bpyqV6+ud999VxEREZKkTp06qV27dpc8BudTvnx5SfItJPHll19q27ZtGj16tO6++27ffp07d1ZycrI+/vhjX7lr27atxo8fr7i4ON9zTlu3btWXX36pxx9/XP369fMdf99996lNmzb6+OOPKUh5QEECAABAgUpKSsr26+7du6tdu3aKjo72bfN4PL67M5mZmRc956233uorR5J8U+kOHz580WPbt2+f7dcNGzbMduzatWt1/Phx9erVy1eOpLNFqkOHDnrnnXcueo28cDqdkuR7d2fbtm21bt06lSpVKtt+R48eVenSpS86LvXq1dP3338vqzX7t/iHDx9W2bJltXfvXr/kLuooSAAAACgwVqtV5cqVy7E9LCxMr732mjZu3Kh9+/Zp3759vmdkPB7PRc8bFRWV7dfh4eGSlKcpcBc7NjU1VZJUo0aNHMfWrFnzoufPqyNHjkiSKlSo4Ntms9n09ttv69tvv9W+ffu0d+9enTp1SpJUtmzZi54zPDxcS5cu1dq1a7Vnzx7t27dPx44dk2EYstlsfstelFGQAAAAUGByW51t27Zt6tq1q1wul5KTk9W6dWslJiaqSZMmuu666/J93r+T6c9cLpekP4rTn5UoUSLf1/2rLVu2SJIaNGggSfr999/VpUsXHT58WM2bN1dycrK6d++uq666Sl27dr3o+U6dOqWuXbtq27Ztatq0qZo0aaJ77rlHV1xxhUaMGKGffvrJb9mLMgoSAAAACtULL7ygzMxM/fe//8324tkDBw4EMNUf4uLiJJ1dYOKvzzPt3r3bL9c4cuSIvvnmG1WvXt33bqlXX31V+/fv1xtvvKFrrrnGt6/L5dKxY8dUpUqVC55z/vz52rJli1588UV16tQp22d5mXqIsyhIAAAAQaQ4vOfo6NGjKlWqlKpWrZpt+5w5cyTlbZpcQWrZsqVKlSqld955R506dfLdNTp8+LCWLVv2t8/vdDr17LPP6syZM+rZs6dv+7nFGhISErLtv3DhQjkcjmzjcu75K9M0cxxfu3btbMevWLFCqamp2Z7ZwvlRkAAAAFCoWrVqpWnTpunRRx/VzTffrDNnzmjFihX69ttvZbPZdPLkyYDmi4yM1ODBgzV8+HDddddduv3223XmzBktXLhQGRkZkv5YWOFilixZ4vvvrKws7d+/X8uXL9euXbt03333ZbvT06pVK33++ed6+OGH9Y9//EOStHr1an3++ecqUaJEtnEpX768LBaL1qxZo0WLFqlly5Zq3bq15s+fr759++ree++V3W7X999/r2XLlqlEiRI6c+aMMjMzsy08gZwoSAAAAChUPXv2VFhYmJYuXarnn39e5cqVU506dTR//nwtXLhQn332mdLT03MsplCY7rrrLkVGRmrmzJmaNGmSypYtqzvuuENOp1Nz5szJ9fmk3AwcOND331arVeXLl1eDBg3Uv39/tW3bNsc1HQ6HFi5cqBdffFGlS5dWjRo1NHXqVP3444+aNWuWtmzZovr166tkyZJ65plnNGPGDI0ZM0ajRo3SHXfcoYkTJ2rmzJmaPHmySpQooerVq2vUqFFyOp0aNWqUVq9erRtuuMGvY1XUGOaf78shV5s2bZKkbG9iLqoyMzO1detW1atXj58uXALGLX8Yt/xh3PKHcftDcfq6BuRHVlaWMjMzc119b8SIEXr77be1YsUKVatWrfDDocDlf/kPAAAAoAg6evSomjVrpjFjxmTbfurUKa1atUrR0dG67LLLApQOBY0pdgAAAMCfVK5cWS1bttTChQuVlZWlxo0b6/jx41q8eLEOHjyol156Kc/PICH0UJCAfIiNjb3g58VhBSIAAIqyV155RXPnztXHH3+spUuXqmTJkmrYsKGGDRum5OTkQMdDAaIgAQAAAH8RGRmpp556Sk899VSgo6CQ8QwSAAAAAHhRkIC/wWoYerpmbXWoUvXiOwMAACDoMcUO+BvKWm1qXr6CkitEKcoWrrn79gQ6EgAAAP4G7iABf0O6M0sz9+yWxzR1S+WqejSuhoJ1TRvTNOVxOsWrzwAAAM6PO0jA3/T54YNymR49GV9LN1SsLJsRJtPtlmGxBDqaJCkjdY/2L/lI6WvXye1wyGK3KyqpuWI6tldkfFyg4wEAAAQV7iABfvBl+mG9unun3Kap66Mravs/p8jjcgU6lg6vXqON/Qfq4MpVcjsckiS3w6GDK1dpY/+BOrx6TYATAgAABBfuIAH58Nf3HGVmZmrr1q2qevKUUl/9tw5/vVqmy6U6/fsqzGYLSMaM1D3aPnGyTLc7189Nl0vbJ02RPTaWO0kAAABe3EEC/Khc02tUd/AAGVar0teu17YXXpInKysgWXbNnH3ecnSO6XJp/9JlhZQIAAAg+FGQAD+rcM3Vqjd8iMLCw3X0u++19fkX5D5zplAznNqdqhObf87Tvulr1rJwAwAgz1599VUlJibm+Kdu3bq6+uqrdccdd2jevHlyX+SHdKHuwIEDOn36dJG6/sMPP6zExMSL7te6desc//8bNmyo5ORkPf744/rmm2/+dhaXy6Vff/31b58nP5hiBxSA8ldeofrPDdOWseN17MeN2jL6edUfPkQWu71Qrr//w6V53tftcMh0uWQEaCogACA0PfHEE6pZs6bv1x6PR7/++qveffddjR8/Xr///rsGDx4cwIQFZ/HixRo1apQ+++wzlSxZsthdX5LKly+vIUOG+H7tcDj066+/aunSpXr44Yf19NNP68knn8zXudPS0vTYY4+pY8eOeuyxx/wVOc8oSEABKduooRqMfFZbRo3Vic0/6+eRY1T/uWGyRkYW6HVN01T6uvV53t9it8uw8lcBAAQj0zTP/hDLapVhBNeLJJKTk9WsWbMc2++//37deuutWrBggR5++GFVrFgxAOkK1vr16+XwLn5UHK8vSREREerYsWOO7Y8//ri6deumyZMnq1GjRmrZsuUln/vXX3/Vzp07/REzX5hiBxSgMvXqqsHoEbJERurkthT9/NwoOU+eLNBrmi6XPJdwy71C82ZB90UXAIq7jNQ92jFlqtZ36aq1d96j9V26aseUqcpIDf4XkpcvX1433HCDXC6Xfvrpp0DHQSErXbq0Xn75ZYWFhelf//pXoOPkCwUJKGCl69RWw7EjZS1dWqd2/qKfnx0l5/HjBXY9w2q9pKl8UclJBZYFAHDpisIrGiK9syX++gO47777Tg8//LCaNGmixo0b64477tCSJUtyHJ+VlaWpU6fqlltuUePGjdWqVSuNGDFChw8fzrbfhg0bsp3v9ttv16JFi7Lts379eiUmJurTTz/VK6+8otatW6thw4a66aabNG/evGz7ut1uTZ48WTfffLMaN26spk2b6tFHH9W3337r26d169ZavHixJKlly5bq2rWrJGnw4MFq1qyZli9frmuvvVaXX365XnzxRd/1Z8yYkeP32bp1a918883Ztp04cUIvvPCC2rRpo8aNG6tt27aaOHGiTp06dcHrS9KRI0c0ZswYXX/99WrYsKFatWqlcePG6cSJEzmu/eabb+q2225T48aNdfPNN+uDDz7IsU9+1axZU5dffrn+97//6ejRo77tBw4c0OjRo3XDDTeoUaNGuuKKK9SxY0e99dZbvn1effVVPfDAA5KkiRMnKjEx0fcs0vHjxzVx4kTddtttuvzyy9W4cWPdeuut+ve//+3XZ96YVwMUglI1a6rh86P187MjlbF7tzYPH6EGo0covHx5v1/LMAxFJTXXwZWr8rT/mhGjNHr7Fu333nX66xLmAIDCk5G6R9snTZF5nnfphcIrGtxut7766itZLBbVq1fPt/0///mPnnnmGSUkJKhHjx6yWCxasWKFBg4cqN27d+vpp5/2Hf/ggw/qu+++00033aT7779f+/fv1xtvvKEffvhB77zzjiIiIrR06VINHDhQMTExevjhhxUREaHly5fr2Wef1ebNmzV69OhsuV544QVZrVZ16dJFdrtdCxcu1Pjx41W2bFndfvvtkqTRo0dr0aJFuvvuu9WgQQMdPXpUCxcu1IMPPqj33ntPdevW1dChQzV37lx99913eu655xQfH++7RkZGhp599lk99NBDstlsatiw4SWNXUZGhu666y7t3btXt99+uxo3bqyUlBTNnj1bP/30k+bOnXve6x85ckR33XWXjhw5onvuuUdxcXFKSUnRwoULtXr1ar3zzjsqVaqUJGn8+PGaN2+emjVrpi5duigtLU0jR46UxY8vuU9MTNT//vc/bdu2TUlJSTp+/LjuuusunTlzRvfee69iYmJ06NAhvfvuuxo5cqSioqJ04403+u4+Tp8+Xbfeequuv/56VahQQU6nUw888ID27NmjLl26qGbNmjp+/LgWL16sKVOmyG6368EHH/RLdgoSUEgi46qr4bjR+vnZUcrcu0+bhj6nhmNHqkRUlN+vFdOxvQ599fV5v8BKZ+e1H8o6o0olSmpkYn2N2b5V+wI8nxkAirv9Sz664N/d0h+vaKjdu2chpcrdyZMndeTIEd+vnU6nfv31V82ZM0c7d+5Ut27dVLVqVUlnv/EfOXKkGjdurDfeeENW77Ov3bp1U//+/TV9+nS1b99etWrV0ocffqjvvvtOTzzxhPr27es7f82aNTV06FB9/PHHuvnmmzV69GhVqVJFH374ocqUKSNJeuCBB9SjRw+98847uuWWW5SUlH2WxIcffqiIiAhJUtu2bXX99dfrww8/9BWkJUuW6Nprr9XIkSN9xyQnJ6t///7avHmz6tatq7Zt22rFihX67rvvdOONN2Z7xsrpdOrpp5/WI4884tu2fn3enwuePXu2UlNTNWbMGHXu3Nm3vXz58vrXv/6l9evXn/f6kyZN0oEDB7Ro0SLVr1/fd+z111+vxx57TK+99pr69++v3bt3a/78+WrTpo2mTp2qsLAw337dunXLc9aLKVeunCT57iAtXrxYBw4c0OzZs7M9l3TjjTfqtttu08cff6wbb7xRdevW1fHjxzV9+nTVq1fP95zTihUrtG3bNo0ePVp333237/jOnTsrOTlZH3/8sd8KElPsgEIUcdllajhujEpUjNbp/fu1eeizOn3woN+vExkfpzr9+px38QXTNGUYhgxJex2ZKmcL18jE+qrh/aIBACh8pmkqfe26PO0bDK9o6Nmzp5KSknz/XHfddbr33nu1ceNG9e3bV4MGDfLtu2bNGh0/flw33XSTTpw4oSNHjujIkSM6evSobr31VpmmqRUrVkiSPvvsMxmGoUcffTTb9dq3b6/33ntPt9xyi1avXq2TJ0+qW7duvnIkSWFhYerVq5ckafny5dmOb926ta8cSVKVKlUUHR2dbdpe5cqVtWHDBs2ePds3rathw4b65JNPdOedd+ZpXP5ayi7FZ599pgoVKuS41kMPPaQPP/xQV155Za7HmaapTz75RHXr1lWVKlV843vkyBE1atRIlStX1qeffipJWrVqlTwej+6//35fOZKkZs2aqXHjxvnO/leuvxT97t27a/Xq1dnKkcfj8U2Ny8zMvOD52rZtq3Xr1umOO+7Itv3o0aMqXbr0RY+/FNxBAgqZvWoVNRw3Rj8/O1Knfz+gzUOfVYMxo2SvWsWv14lukSx7bKz2L12m9DVr5XY4ZLHbtWLfHn2TfliPxNVQ1ZJ2HTpzWjszTikhspSerVNfJ1O2q3RiHb9mAQBcnOly+Z45uphgeEXDoEGDVLduXZmmqT179mj27NnKzMzU888/r//7v//Ltm9qaqqks9PcXnjhhVzPd26K96+//qro6GjfdLBzwsPD1ahRI0nSvn37JEm1atXKcZ5atWrJMIwc79CJjo7OsW94eHi2Z1eef/55Pf3005owYYImTJig+Ph4/d///Z86deqU7a7MheR2nbz69ddflZiYmK24SFKpUqWyTVf8qyNHjujEiRPatGnTeQtaeHi4pD/GLi4u5xTNWrVqaePGjfmNnyOTJFWoUMG3LSwsTK+99po2btyoffv2ad++fb7V+Dwez0XPabPZ9Pbbb+vbb7/Vvn37tHfvXt+zWWXLlvVLbomCBAREyUqV1HDcGG0ePtJ3J6nBmJGKuCzWr9eJjI9T7d49ldCrh2+Z2H9cdpkkaUTKFg2tXVflbeF6ccdm9a9VR3VLl9HPI0ar/nPDVKb++f8iBgD437lFdvJSkoLhFQ0NGjTwLfPdokUL3XTTTbr77rv15JNPasqUKbrhhht8+56729W/f//zPpdTqVIlSWfvPJQoUeKC177Q3TPTNGWapq8QnPPX0pGbq6++WitXrtTq1av19ddfa+3atXr99dc1f/58Pfvss7rvvvsueo68XOccl8uVLedff51X58pF8+bN9fjjj19w33MLZ5zJ5SX2/rwruWXLFoWFhfmK3bZt29S1a1e5XC4lJyf7XjbbpEkTXXfddRc93++//64uXbro8OHDat68uZKTk9W9e3ddddVV2Raq8AcKEhAgJaKi1GjcaP383NlnkjYPe04NRo9QZFx1v1/LMIwcP2U85nRq0JZNshqGnKap53ds06CERDWU9PPIMao3fIjKNW7k9ywAgNxdyiI7UclJQfeKhqioKL3yyivq3LmzBg4cqMWLF/sWEIiNPfsDwMjISCUnJ2c77rffftOmTZuy7XvuPT/2P63KmpWVpQEDBujaa6/VZd4f9v3yyy85vrk+9/6cmJiYS8qflZWlbdu2qVy5cmrVqpVatWolSUpJSdEDDzyg1157LU8F6a/OLXxw+i+v4HA6nTp27Fi2aX+xsbG+u21/dvDgQY0dO1YdO3ZUmzZtcnxeoUIFRURE6NSpUznGVzo73fBcAT1352jXrl3ZXvQrSXv2+GcZ+e3bt/sWZzh3Z+eFF15QZmam/vvf/6p69T++1zlw4ECezvnqq6/6Fuu45pprfNtdLpeOHTumKlX8NxOHZ5CAAAovX14Nnx+tyBo15Dx2TJuHPadTu3add3/TNOVxOv32Ex5TktN7rjMej8bv2KZyV14hz5kz2jpmnI7+8D+/XAcAkDcxHdtf9M6QYbUqpkO7Qkp0aerXr6/evXsrMzNTgwYN8k1fa9mypSIiIjRnzhzflCjp7Ne18ePHq1evXvrll18knX3WxOl06u2338527v/+979avny5srKyfOd7/fXXsy1h7fF49O9//1uSst3ByosTJ07onnvu0bhx47JtT0hIUOnSpbOt8HbuLlFepoVVrlxZkrR58+Zs2z/55JMcd3Hatm2rAwcO+J4XOue9997TJ5984lvc4q/Xt1gsat26tTZv3qyVK1dmO3blypXq06eP3njjDUlnx8Vms2nWrFlyOp2+/X788Uf98MMPF/39XExGRoaGDx8uwzDUo0cP3/ajR4+qVKlSvoU7zpkzZ44kZZvqeG6s/zy+5xZ7SEhIyHb8woUL5XA4WOYbKEpsZcqowZgR2jJqrE7t2KnNw0eqwchnVbpObd8+Gal7tH/JR0pfu873LFFUUnPFdGzv12VenaapekMHaduEiTr67XfaOGK0Jv2yQ98fP5rr/iwJDgD+dW6RnfMt9W1YrarTr0/QLvEtSQ8//LBWrFihH3/8UbNmzdLjjz+usmXLaujQoXr22WfVoUMH3XnnnSpfvrxWrFihb775RjfddJPv4f1//OMfWrp0qV544QVt3bpVV111lVJTU/XWW2+pcePGuvPOOxUeHq7hw4dr2LBh6tSpk+666y5FRETok08+0ffff69//OMfat68+SXljo6OVufOnfXWW2/piSee0LXXXivpbJHZt2+fhg4dmm1fSZoxY4aSk5NzvatzTrVq1dSkSRN9+eWXGjx4sJo0aaKUlBR98MEHvjth5zz22GP67LPP1LdvX3Xu3Fl169bV5s2b9f7776tt27a+u2W5XX/AgAFav369evXqpTvvvFP169fX7t279dZbbykqKsq3jHpMTIx69+6tiRMn6t5771X79u2Vnp6uBQsWqEKFCtlWJryQzMzMbO+wOnPmjPbs2aNly5bp999/14ABA9S0aVPf561atdK0adP06KOP6uabb9aZM2e0YsUKffvtt7LZbDp58qRv3yjvCr+ffvqpb/nvVq1a6fPPP9fDDz+sf/zjH5Kk1atX6/PPP1eJEiWyHf93GWagl0AJAZs2bZIk34OBRVlmZqa2bt2qevXqZbvliwvzx7i5MjK0ZfTzOrktRRa7XfVHDFe9tm3UvHwF9a6RIGsuc5rPfaGMbpHzdvrf4XE6tX3iZKWvXSeXx6Mpu3dq/dGcf2H+3YLEn7f8Ydzyh3H7Q3H6uhaqMlL35FhkJyo5STEd2gW8HL366quaOnWq5s+f73sG6a927dql22+/XW632/f+IEn65ptvfO/0cbvdqlatmm6//Xbdf//92Z69cTgcmj59uj7++GP9/vvvqlKlim644Qb16NFDpUuX9u23evVqzZgxQz/99JOks4sM3H333brrrrt8+6xfv14PPPCA+vfvr8ceeyxbztatWys8PNy34p3L5dKCBQv0wQcfKC0tTaZpKjExUffee686dOjgO+63335T3759tXnzZsXGxuqTTz7R4MGDtXjxYn3zzTfZlv6Wzk6RmzBhgr7++mudPn1ajRo1Ut++fTV79mzt2rUr24p7R44c0dSpU7VixQodPXpUl112mTp06KCHHnrI92xWbteXzk5X+9e//qUvv/xS6enpio6OVvPmzdWzZ09Vq1YtW6aPPvpIc+bM0S+//KLo6Gh1795dKSkpeu+995SSknKhPwJq3bp1ju8BbDaboqKidPnll+u+++7L8WfD6XRq2rRpWrp0qQ4cOKBy5cqpTp06evzxx7Vw4UJ99tln+uqrr3zl6IUXXtB7770nl8uladOmKSkpSfPnz9fChQu1f/9+lS5dWjVq1FC3bt18ZXzx4sV5XkzjQihIeVCcvpDwDUT++Gvc3A6HtowdrxObf1ZYyZJa+Mt2dY6pprALzDM3rFZdPnGC379gmm63Xvy/1ro2Kloe09TU3Tv1zZH0bPtQkAKDccsfxu0PxenrWqgzTdO3yE6wPXMEFFU8gwQEEYvdrvrPDVO5Ky6X5/Rp3X2RciT98cJAfzMsFk3dvVOrDh9UmGHoqRoJurJsOb9fBwBwfoZhKMxmoxwBhYiCBAQZS4kSqn7/vZKU5y+IBfXCQFPS9NRd+uTg7wozDCVElrroMQAAAKGMRRqAIPT7f5ZffKc/KcgXBpqSZu9N1ccHfteBM6cvuj8AAEAooyABQcY0TaWvXXdJxxTGCwN/pxwBAIBigCl2QJAxXa48vUX9z4LxhYEAAAChiDtIQJAxrFZZ7PY8l6SCfGEg7zkCAADFDXeQgCBjGIaikvL4cjvDCPoXBgIAAIQSChIQhGI6tr/4M0WGocSB/f3+klgAAIDijIIEBKHI+DjV6dfn/CXJYlHigH6KTk4q3GAAAABFHM8gAUEqukWy7LGx2r90mdLXrJXb4ZDFbldUcpJiOrRjWh0AAEABoCABQSwyPk61e/dUQq8eZ99zZLWyWh0AAEABoiABIcAwjAJ5CSwAAACy4xkkAAAAAPCiIAEAAACAFwUJAAAAALwoSAAAAADgRUECAAAAAC8KEgAAAAB4UZAAAAAAwIuCBAAAAABeFCQAAAAA8KIgAQAAAIAXBQkAAAAAvChIAAAAAOBlDXSAvyMtLU2vvPKKVq9erWPHjik6Olpt27ZV7969VaZMmUDHAwAAABBiQrYgHTt2TF26dNGJEyfUpUsXxcXFadOmTXrzzTe1YcMGvfPOO7Lb7YGOCQAAACCEhGxBmjp1qg4cOKDXX39dzZs3lyTdc889qlu3rsaOHat33nlH3bt3D2xIAAAAACElZJ9BWrNmjapXr+4rR+d07NhRkvTDDz8EIhYAAACAEBayd5DmzJmjjIyMHNuPHDkiSbJYLIUdCQAAAECIC9mCVKVKlVy3z5s3T5LUrFkzv17PNE1lZmb69ZzByOFwZPs38oZxyx/GLX8Yt/xh3P5gmqYMwwh0DAAISoZpmmagQ/zZpEmTdOrUqQvu07t3b5UrVy7H9uXLl+vpp59WtWrVtGzZMpUoUcIvmTZt2qSsrCy/nAsAgGAQHh6uRo0aBToGAASdoLuD9P777+vw4cMX3Oehhx7KUZA++eQTPfPMM7Lb7ZoyZYrfytE5NptNCQkJfj1nMHI4HEpNTVV8fDyrAF4Cxi1/GLf8Ydzyh3H7w86dOwMdAQCCVtAVpNWrV1/yMQsWLNC4ceNUsmRJzZgxQ/Xr1/d7LsMwFBER4ffzBiu73V6sfr/+wrjlD+OWP4xb/jBuYnodAFxA0BWkS2GapiZMmKA5c+aoQoUKeu2119S4ceNAxwIAAAAQokK6ID3//PNasGCBqlevrlmzZikuLi7QkQAAAACEsJAtSO+++64WLFigGjVqaMGCBapYsWKgIwEAAAAIcSFZkM6cOaNJkyZJktq0aaM1a9bk2KdSpUpKSkoq7GgAAAAAQlhIFqSUlBTfC2FnzZqV6z4tW7akIAEAAAC4JCFZkBo3bqyUlJRAxwAAAABQxIQFOgAAAAAABAsKEgAAAAB4UZAAAAAAwIuCBAAAAABeFCQAAAAA8KIgAQAAAIAXBQkAAAAAvChIAAAAAOBFQQIAAAAALwoSAAAAAHhRkAAAAADAyxroAEBuYmNjL/h5WlpaISUBAABAccIdJAAAAADwoiABAAAAgBcFCUEvtqRdvWrUUq3IyEBHAQAAQBFHQULQu7xMWV0bVVHP1amveqVKBzoOAAAAijAKEoLe54cP6qcTx2W3WDS0dl01Kl0m0JEAAABQRFGQEPTOeDx6ccc2/XD8qEpYLBpUu66OfPd9tn1M05TH6ZRpmgFKCQAAgKKAZb4REpymqZd3blffmrV1TfkK2jZ+ghIH9lfJypW1f8lHSl+7Tm6HQxa7XVFJzRXTsb0i4+MCHRsAAAAhhoKEoHS+9xx5XC5tnzRZ6avXatsLL3k3enyfux0OHVy5Soe++lp1+vVRdIvkwogLAACAIoIpdggpYVarEvv3Vfmrm5wtRn8qR39mulzaPmmKMlL3FHJCAAAAhDIKEkKOYbHIWvriq9mZLpf2L11WCIkAAABQVFCQEHJM09SRdevztG/6mrUs3AAAAIA8oyAh5Jgul9wOR572dTscMl2uAk4EAACAooKChJBxbilvWSyy2O15OsZit8uwshYJAAAA8obvHBH0MlL35FjK21a2bJ7uIkUlJ8kwjEJICQAAgKKAgoSgdnj1Gm2fNCXbNDm3w5GncmRYrYrp0K4g4wEAAKCIYYodglZG6p4c5SivDKtVdfr14WWxAAAAuCTcQULQ2r/kozyVo5JVq8p57Jhv+l1UcpJiOrSjHAEAAOCSUZAQlEzTVPradXna13nsmJounC+53TKsVp45AgAAQL5RkBCULnUpb7ndCrPZCjgVAAAAijoKEiRJsbGxF/w8LS2tkJKcZVitstjteSpJLOUNAAAAf2GRBgQlwzAUldQ8T/uylDcAAAD8hYKEoBXTsX2e7gxZS0UWQhoAAAAUBxQk5PCPqrEaX6+hYkvaA5ojMj5Odfr1OX9J8t412r/kI+1b9F4hJgMAAEBRRUFCDvEREaoVWUqjEusrzh4R0CzRLZJ1+cQJqtSmtSz2s4XNYrerUpvWumLyRFW/r4skae+bb2nPGwtlmmYg4wIAACDE8WQ7cngtdbeiw0uoVmQpjUisr7HbtwY0T2R8nGr37qmEXj1kulzZlvKOjI9TmM2m1Hnz9eu778vjdCq++wM8kwQAAIB84Q4ScjjldmnM9q3afuqkSlmteq5OPZ3YlhLoWDIMQ2E2W47yE3t7R9V87GFJ0v4Pl2r3zNkyPZ5ARAQAAECIoyAhV5lut8Zu36otJ08owmrVzyNG6/jPPwc61nlVve1W1erxuGQY+u3j/+qXaa9RkgAAAHDJKEiQdPY9R2lpadqxY4eWLl2qHTt26Jd9+/TQfz5S2caN5Dl9WltGjtWxHzcGOup5VbnpRtXu3VMKC9OBT1doxyv/kul2BzoWAAAAQggFCRdkKVlS9YYPUfkmV8qTlaUtY8fryHffBzrWeVVq3Up1+vaRwsJ0aNUX2v7PKfK4XIGOBQAAgBBBQcJFWUqUUN0hg1Sh2TUynU5tGz9B6es3BDrWeVW8rqXqDnxGhtWqw1+vVspLk+RxOgMdCwAAACGAgoQ8CbPZlDjwGUW1SJLpcinlxZd1+JvVgY51XlFJzVR3yEAZNpuOrFuvbeMnyJOVFehYAAAACHIUJORZmNWqxP59VfH662S63UqZOFkHv/gy0LHOq8LVTVR/+BCFhYfr6Pc/aMvY8XKfORPoWAAAAAhiFCRcEsNiUe3eT6lS2zaSx6Mdk1/Vgc9WBDrWeZW74nLVHzFcYSVL6vjGn7Rl1Fi5Mh2BjgUAAIAgRUHCJTMsFiX0fEJVbrlZMk3tnDpNv/1neaBjnVfZhg3UYOSzskRE6MTPW7Rl5Bi5MjICHQsAAABBiIKEfDHCwlTz8UcU06GdJGnXazOVtuQj3+emacrjdMo0zUBFzKZMvbpqMHqErKVK6WRKin5+bpScJ08GOhYAAACCjDXQARC6DMNQ/EPdFRYerl/f+0Cpc+bpzMGDcmc6lL52ndwOhyx2u6KSmiumY3tFxscFNG/p2glqMGakfh4xWqd2/qJFd9ylMdu36mQuy4CnpaUFICEAAAACjTtI+FsMw1D1++9VtS53S5J+W/YfHVy5Sm7H2ed83A6HDq5cpY39B+rw6jWBjCpJKlWzhho9P0rHnFmKj4jUyMT6Kmu1BToWAAAAggQFCX+bYRiKat5MCjv/HyfT5dL2SVOUkbqnEJPlLqJ6dY3YtkXpWWdUzR6hUXUpSQAAADiLggS/2L/kI8njueA+psul/UuXFVKiC/vtzGmN2LZFB8+cUUxJu66Nig50JAAAAAQBChL+NtM0lb52XZ72TV+zNmgWbjiYdUbPbtusBfv26Ov0w4GOAwAAgCDAIg3420yXy/fM0cW4HQ6ZLpcMW3BMaTvqdOqjA78FOgYAAACCBHeQ8LcZVqssdnue9rXY7TKs9HIAAAAEJwoS/jbDMBSV1DxP+0YlJ8kwjAJOBAAAAOQPP8qHX8R0bK9DX30tM5d3Cp1jWK2+F8sGGu85AgAAQG64gwS/iIyPU51+fc47fc6wWlWnX5+AvywWAAAAuBDuIMFvolskyx4bq/1Llyl9zVq5HQ5Z7HZFJScppkM7yhEAAACCHgUJfhUZH6favXsqoVePs6vVWa08cwQAAICQQUFCgTAMI2iW8gYAAADyimeQAAAAAMCLggQAAAAAXhQkAAAAAPAqMgXJNE117dpViYmJgY4CAAAAIEQVmYI0Z84cbdiwIdAxAAAAAISwIlGQUlJSNHnyZIWHhwc6CgAAAIAQFvIFKSsrSwMGDFCTJk3UtGnTQMcBAAAAEMJCviD985//1P79+zV+/PhARwlppmnK43TKNM1ARwEAAAACJqRfFLthwwbNmzdP48ePV9WqVQMdJyRlpO7R/iUfKX3tOrkdDoXZS8qoU0eOyFKKqMuCFwAAAChegq4gTZo0SadOnbrgPr1795bFYtHgwYPVpk0bderUqcBzmaapzMzMfB9fu3btC36+Y8eOfJ87v46uW689U6fJdLt92zyO09LGn5Ty8xbF9XxC5Zs3K/RcocjhcGT7N/KGccsfxi1/GLc/mKYpwzACHQMAglLQFaT3339fhw8fvuA+Dz30kKZMmaLTp09r9OjRhZLL6XRq69atBXb+gjx3bjwHDipr5hzJ48n1c9PlUurUadp/+rTCKlcq1GyhLDU1NdARQhLjlj+MW/4wbmexsBEA5C7oCtLq1asvus/y5cu1dOlSPf/885KkI0eOSJJcLpfv1xaLRWXLlvVbLpvNpoSEBL+cy5AUabHqlNvl21avXj2/nDuv9nz5tY6cpxz5uN0qtW274q7/v8IJFcIcDodSU1MVHx8vu90e6Dghg3HLH8Ytfxi3P+zcuTPQEQAgaAVdQcqLL774QpI0bNiwXD9PSkpSbGysVq5c6bdrGoahiIgIv5zryfhaalkhSjP37Naq9EOS5Ldz54Vpmjq+4bs87Xt8w7ey9+3NVIw8stvthfr/sqhg3PKHccsfxk38nQ4AFxCSBemRRx5Rhw4dcmyfOHGiNm/erLlz56pEiRIBSJY3v585LWtYmJ6sUUvWsDB9duhAoV7fdLnkzuMcfLfDIdPlkmGzFXAqAAAAIPBCsiAlJCTkOt1t9uzZkqTk5OTCjnRJPvgtTaWtVt1Wuaoejauh8LDC/UmeYbXKYrfnqSRZ7HYZ1pD8YwIAAABcspB/D1Koen3fHn34W5okqVu1eP363geFdm3DMBSV1DxP+0YlJzEVAwAAAMUGBSmAFqbt07v7f5Uk7Vnwpva+9U6hvag1pmP7PN0ZKnfF5YWQBgAAAAgORWru1LkpdsEoLS3tvJ/9+t4H2rPgTe17e5E8Tqfiut5X4HdtIuPjVKdfH22fNEWmy3Xe/X6Z9ppKVKqoMrw0FgAAAMUAd5CCwGV33qH4h7pLktLeX6zds+cVyp2k6BbJunziBFVq01oW75K3YfaSslzRWLVHj1CZBvXlzszUzyNG6/jmnws8DwAAABBoFKQgEduxvWo+8agk6bePlmnXazNlXuw9RX4QGR+n2r17qtlbC5T03ttqPHuGbB3aqVTtBNUfMVxlL28sz+nT2jJqrI79uLHA8wAAAACBREEKIlVvuVkJvXpIhqHf//uJdv5rmky3u1CubRiGwmy2bFP7LCVKqP7wISrf5Cp5srK0Zex4Hfnu+0LJAwAAAAQCBSnIVG7bRrWf7i2FhengipXaMWVqoZWk3ISFh6vukIGq0LyZTKdT28ZPUPra9QHLAwAAABQkClIQqnT9dUp8pq8Mi0WHvvxKKS//Ux6nM2B5wmw2JQ7op+iWLWS6XNo24WUd+np1wPIAAAAABYWCFKSiWyQrcdAAGVar0tes1bYXX75gSTJNUx6ns8AWdwizWlWnXx9VbHW95PFo+6TJOrjyiwK5FgAAABAoFKQgFtXsGtUbNlhh4eE6+u132vr8C3KfOZNtn4zUPdoxZarWd+mqtXfeo/VdumrHlKnKSN3j9zyGxaLavXuq8g1tJY9HO16Zqt8//czv1wEAAAAChYIU5MpfdaXqDR+isBIldOx/P2rrmHFynz4tSTq8eo029h+ogytXye1wSJLcDocOrlyljf0H6vDqNX7PY4SFqVaPx1X1tlsk09Qv/5qu3z7+j9+vAwAAAAQCBSkElLu8seqPGC6L3a7jmzZry6ixOrDqC6W8NOm8L3k1XS5tfuGlgrmTFBamGo8+rJhOHSRJu2bMVtriJX6/DgAAAFDYKEghomyD+mow6jlZIiN0YstW7Zz8qnSR541sYWHav3RZgeQxDEPx3R/QZZ3vlCSlzpuvfYveK5BrAQAAAIWFghRCSifWUa0nn7ikY9LXrC2whRsMw1DcfV1U/b4ukqS9b76lPW8sLLDrAQAAAAWNghRijv3wv0va3+1wnHcanr9U63yn4rs/IEn69d33lTpvPiUJAAAAIcka6ADIO9M0lb523SUdY7HbZVgL/n9z7O0dFRZu064Zs7X/w6Wa/doMzduXqtxqUlpaWoHnAQAAAPKDO0ghxHS5fKvV5VVUcpIMwyigRNlVve1W1erxuDymqVsqV9GjcTVUOFcGAAAA/IOCFEIMq1UWuz3P+zs9HsV0aFeAiXKqctONmpb6izymqbYVK6tHfC1KEgAAAEIGBSmEGIahqKTmedrXY5p6dfdORcbHFXCqnL5MP6xXdu2U2zT1f9EV9UC1ws8AAAAA5AcFKcTEdGx/8WeKDEP1Bg/Q+5s3FU6oXKw5mq5Jv2xXlsejKiVKBiwHAAAAcCkoSCEmMj5Odfr1OX9JsliUOKCfopOTCjdYLr49dlSPb/xeL+1MCXQUAAAAIE9YxS4ERbdIlj02VvuXLlP6mrVyOxyy2O2KSk5STId2AZlWdz4ZbnegIwAAAAB5RkEKUZHxcardu6cSevWQ6XLJsFoLbbU6AAAAoKiiIIU4wzBk2GyBjpEN7zkCAABAqOIZJAAAAADwoiABAAAAgBcFCQAAAAC8KEgAAAAA4EVBAgAAAAAvChIAAAAAeFGQAAAAAMCLggQAAAAAXhQkAAAAAPCiIAEAAACAFwUJAAAAALwoSAAAAADgRUECAAAAAC/DNE0z0CGC3Q8//CDTNBUeHh7oKAXONE05nU7ZbDYZhhHoOCGDccsfxi1/GLf8Ydz+kJWVJcMwdNVVVwU6CgAEHWugA4SC4vSF1DCMYlEE/Y1xyx/GLX8Yt/xh3P5gGEax+toGAJeCO0gAAAAA4MUzSAAAAADgRUECAAAAAC8KEgAAAAB4UZAAAAAAwIuCBAAAAABeFCQAAAAA8KIgAQAAAIAXBQkAAAAAvChIAAAAAOBFQQIAAAAALwoSAAAAAHhRkAAAAADAi4IEAAAAAF4UJAAAAADwoiABAAAAgBcFCQAAAAC8KEgAAAAA4EVBwkWZpqmuXbsqMTEx0FGCXlpamgYNGqSWLVuqYcOGuv766zV27FidOHEi0NGCyuHDhzVs2DBde+21uuKKK3TPPfdo9erVgY4V9FJSUtSrVy81a9ZMDRs21A033KDJkycrKysr0NFCxunTp3XLLbeodevWgY4CAAhS1kAHQPCbM2eONmzYEOgYQe/YsWPq0qWLTpw4oS5duiguLk6bNm3Sm2++qQ0bNuidd96R3W4PdMyAy8jI0IMPPqi9e/eqW7duqly5shYtWqRHHnlEM2fOVMuWLQMdMSjt3btXXbp0kc1m07333qtKlSpp3bp1mjZtmn766SfNnj1bhmEEOmbQmzBhgnbt2qXY2NhARwEABCkKEi4oJSVFkydPVnh4OD+lvoipU6fqwIEDev3119W8eXNJ0j333KO6detq7Nixeuedd9S9e/fAhgwCb7zxhrZv367p06erVatWkqROnTqpU6dOGjNmjJYvX843+rkYN26csrKy9N5776lmzZqSpC5duuiFF17Q3Llz9fnnn6tt27YBThncvv76ay1cuFA2my3QUQAAQYwpdjivrKwsDRgwQE2aNFHTpk0DHSforVmzRtWrV/eVo3M6duwoSfrhhx8CESvoLFmyRHFxcb5yJEmRkZHq3LmzUlNTtXHjxgCmC05ut1sbNmzQ1Vdf7StH55z78/X9998HIlrIOHbsmIYOHarbb79dl112WaDjAACCGAUJ5/XPf/5T+/fv1/jx4wMdJSTMmTNH06dPz7H9yJEjkiSLxVLYkYLOyZMntWvXLjVu3DjHZ40aNZIkClIuwsLCtGTJEo0ePTrHZ/z5ypvnnntONptNw4YNC3QUAECQY4odcrVhwwbNmzdP48ePV9WqVQMdJyRUqVIl1+3z5s2TJDVr1qwQ0wSnAwcOyDTNXP9MVapUSdLZhS6QnWEYqlatWq6fvf7665L483UhH374oT799FO9/vrrKlWqVKDjAACCHAWpGJk0aZJOnTp1wX169+4ti8WiwYMHq02bNurUqVPhhAtieR23cuXK5di+fPlyvf3226pevbpuv/32AkoYOk6ePClJuS5WERERIUlyOByFmimUzZ07V19++aWaNGmia6+9NtBxglJaWprGjh2rbt26USIBAHlCQSpG3n//fR0+fPiC+zz00EOaMmWKTp8+net0nuIor+P214L0ySef6JlnnpHdbteUKVNUokSJAkwZGjwejyTlugiDaZrn/Qw5zZ8/Xy+++KIqVqyoiRMnBjpOUPJ4PBo8eLAqVaqkfv36BToOACBEUJCKkby8Z2b58uVaunSpnn/+eUl/PN/gcrl8v7ZYLCpbtmzBBQ0y+Xk/z4IFCzRu3DiVLFlSM2bMUP369QsgWeiJjIyUdPZdNH917s5R6dKlCzVTqDFNUy+//LJmzZqlihUrat68eUyDPY+5c+fqu+++04wZM5SRkaGMjAxJZ4uTx+PRkSNHZLPZ+DMHAMiGgoRsvvjiC0k674PMSUlJio2N1cqVKwsxVegwTVMTJkzQnDlzVKFCBb322mu5LkhQXJ1bPezAgQM5Pjt48KAk8c3+BWRlZWnIkCFatmyZqlevrtmzZ6t69eqBjhW0vvjiC3k8Hj3yyCO5fp6UlKSmTZtqwYIFhZwMABDMKEjI5pFHHlGHDh1ybJ84caI2b96suXPnMlXsAp5//nktWLBA1atX16xZsxQXFxfoSEGlVKlSio+P16ZNm3J8dm4bhTJ3brdbffv21YoVK9SoUSPNmDFDFSpUCHSsoDZo0CCdOHEix/ahQ4fK6XTqpZdeUpkyZQKQDAAQzChIyCYhIUEJCQk5ts+ePVuSlJycXNiRQsa7776rBQsWqEaNGlqwYIEqVqwY6EhBqV27dpo6daq++uorXXfddZKkjIwMLVq0SLVq1fIt943sXn31Va1YsUJXXXWVZs2a5ZuuiPNr2LBhrttLliypsLAw/j4DAOSKggT4wZkzZzRp0iRJUps2bbRmzZoc+1SqVElJSUmFHS3oPPjgg1qyZIn69Omj7t27q2LFilq0aJH279+vmTNnskhDLg4dOqTZs2fLMAy1bt1aK1asyLFPjRo1uPsGAIAfUJAAP0hJSfEtaDFr1qxc92nZsiUFSWen2b355pt66aWX9Oabb8rpdKpu3bqaPXu2mjdvHuh4Qenbb79VVlaWJOnll1/OdZ+7776bggQAgB8Y5rm1dQEAAACgmAsLdAAAAAAACBYUJAAAAADwoiABAAAAgBcFCQAAAAC8KEgAAAAA4EVBAgAAAAAvChIAAAAAeFGQAAAAAMCLggQgqH311VdKTEzUmjVrAh0FAAAUAxQkAEFr27ZtGjBgQKBjAACAYoSCBCAoffbZZ+ratauOHTsW6CgAAKAYoSABCCoHDhzQk08+qaeeekrh4eFq0KBBoCMBAIBihIIEoMD9+OOPql+/vpo0aaLff/8922fjxo1TYmKihgwZIknatGmTVq1apQ4dOuijjz5SnTp1AhEZAAAUUxQkAAXuiiuu0KOPPqpTp05p5MiRvu1r1qzR/PnzFR8fr+HDh0uS6tSpo2XLlumll15ShQoVApQYAAAUVxQkAIXiqaeeUoMGDbRq1Sr997//1fHjxzV48GBZrVZNnDhRkZGRkqTq1asrISEhwGkBAEBxRUECUChsNpsmTJigEiVKaNy4cRo+fLgOHDigvn37qmHDhoGOBwAAIImCBKAQJSQkqG/fvjp48KA+/fRTtWjRQg899FCgYwEAAPhQkAAUqhtvvFEWi0WSdNVVV8kwjAAnAgAA+AMFCUChMU1TQ4cOldvtVrly5TR9+nRt27Yt0LEAAAB8KEgACs38+fO1bt063XbbbZo8ebKcTqcGDBigrKysQEcDAACQREECUEh++eUXTZo0SeXLl9fw4cOVlJSku+66S9u3b9fkyZMDHQ8AAEASBQlAIXC5XBo4cKBOnz6t4cOH+95vNGjQIFWqVElz587Vd999F+CUAAAAFCQAhWDatGnavHmzWrVqpXbt2vm2ly5dWiNGjJDH49GgQYN06tSpAKYEAACQDNM0zUCHAAAAAIBgwB0kAAAAAPCiIAEAAACAFwUJAAAAALwoSAAAAADgRUECAAAAAC8KEgAAAAB4UZAAAAAAwIuCBAAAAABeFCQAAAAA8KIgAQAAAIAXBQkAAAAAvChIAAAAAOBFQQIAAAAAr/8HJ2+S1AzmX7AAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 800x550 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the training data\n",
    "plt.plot(x1_N, x2_N, 'ks', label='Training Data')\n",
    "\n",
    "# Plot the reconstructed data\n",
    "plt.plot(xhat_NF[:,0], xhat_NF[:,1], 'r.', label='Reconstructed Data', mew=5)\n",
    "\n",
    "# Draw line between training example and reconstruction\n",
    "for n in range(N):\n",
    "    xy_22 = np.zeros((2,2))\n",
    "    xy_22[0] = x_NF[n,:]\n",
    "    xy_22[1] = xhat_NF[n,:]\n",
    "    plt.plot(xy_22[:,0], xy_22[:,1], 'r-')\n",
    "\n",
    "plt.legend(bbox_to_anchor=(1.75, 0.5))\n",
    "plt.xlabel('x1');\n",
    "plt.ylabel('x2');\n",
    "plt.gca().set_aspect('equal', 'box')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Show that we know how to perform the PCA transform ourselves"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def my_transform(x_NF, w_KF, m_F):\n",
    "    ''' Perform transformation to get low-dim embeddings\n",
    "    \n",
    "    Args\n",
    "    ----\n",
    "    x_NF : 2D array, shape (N,F)\n",
    "    w_KF : 2D array, shape (K,F)\n",
    "    m_F : 1D array, shape (F,)\n",
    "    \n",
    "    Returns\n",
    "    -------\n",
    "    z_NK : 2D array, shape (n_examples, n_components) = (N,K)\n",
    "        Each row is a low-dim. \"embedding\" of the corresponding row in x_NF\n",
    "    '''\n",
    "    \n",
    "    # Use the equation defining z in terms of x and m and W\n",
    "    z_NF = np.dot(x_NF - m_F[np.newaxis,:], w_KF.T)\n",
    "    return z_NF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z_NK = my_transform(x_NF, w_KF, m_F)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of z_NK\n",
      "(21, 1)\n",
      "\n",
      "First 5 rows of z_NK\n",
      "[[-6.52423243]\n",
      " [-6.4504965 ]\n",
      " [-5.11980459]\n",
      " [-5.87207543]\n",
      " [-4.10995134]]\n"
     ]
    }
   ],
   "source": [
    "print(\"Shape of z_NK\")\n",
    "print(z_NK.shape)\n",
    "print()\n",
    "print(\"First 5 rows of z_NK\")\n",
    "print(z_NK[:5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Discussion 1c: Are these z_NK values from `my_transform` the same as produced by `pca.transform`? \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO discuss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Now try with K=2 PCA for our F=2 dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-2 {color: black;}#sk-container-id-2 pre{padding: 0;}#sk-container-id-2 div.sk-toggleable {background-color: white;}#sk-container-id-2 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-2 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-2 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-2 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-2 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-2 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-2 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-2 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-2 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-2 div.sk-item {position: relative;z-index: 1;}#sk-container-id-2 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-2 div.sk-item::before, #sk-container-id-2 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-2 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-2 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-2 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-2 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-2 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-2 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-2 div.sk-label-container {text-align: center;}#sk-container-id-2 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-2 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>PCA(n_components=2)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" checked><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">PCA</label><div class=\"sk-toggleable__content\"><pre>PCA(n_components=2)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "PCA(n_components=2)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pca = sklearn.decomposition.PCA(n_components=2)\n",
    "pca.fit(x_NF)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Print out the transformed embedding features:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[-6.52423243, -0.66808798],\n",
       "       [-6.4504965 ,  0.0494726 ],\n",
       "       [-5.11980459, -0.6066986 ],\n",
       "       [-5.87207543,  0.78661395],\n",
       "       [-4.10995134, -0.29266499],\n",
       "       [-4.34687772,  0.75715317],\n",
       "       [-2.03157546, -0.90685279],\n",
       "       [-1.97845129, -0.00897906],\n",
       "       [-1.46165685,  0.22647118],\n",
       "       [-1.40616067,  0.5457974 ],\n",
       "       [ 0.62968306, -0.50382783],\n",
       "       [ 0.50286399,  0.16903618],\n",
       "       [ 1.47389739, -0.05406408],\n",
       "       [ 1.19917502,  1.10777136],\n",
       "       [ 2.71280757, -0.13916024],\n",
       "       [ 3.41699812, -0.11644356],\n",
       "       [ 4.09659534, -0.11612761],\n",
       "       [ 5.96711016, -0.78036674],\n",
       "       [ 6.07911096, -0.22292293],\n",
       "       [ 6.14379828,  0.46189491],\n",
       "       [ 7.07924238,  0.31198566]])"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pca.transform(x_NF)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Discussion 1d: What do you notice about the first column of z here when K=2?\n",
    "\n",
    "How does it compare with the old z?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO discuss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 1e: Show the learned weight vectors\n",
    "\n",
    "Print out the learned weight vectors for K=2 and show your self they are \"unit\" vectors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO access and print weights from the `pca` object"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Discussion 1f: How do the learned weight vectors for K=2 compare to the old weights for K=1?\n",
    "\n",
    "What is different? What is the same? Do any vectors point in the same direction?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO discuss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 2: PCA for 8x8 digit images\n",
    "\n",
    "Let us load a dataset of 8x8 pixel images of numerical digits (1, 2, ... 8, 9, 0) that comes from sklearn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_all_LF, y_all_L = sklearn.datasets.load_digits(n_class=10, return_X_y=True)\n",
    "x_all_LF = np.asarray(x_all_LF, dtype=np.float64) / 16.0 # Make each pixel intensity between 0 and 1\n",
    "\n",
    "F = 64 # number of features (flattened 8x8 pixel grayscale image)\n",
    "\n",
    "L = 1797 # all examples\n",
    "N = 1697 # train set size\n",
    "T =  100 # test set size\n",
    "\n",
    "shuffled_rows_L = np.random.RandomState(0).permutation(L)\n",
    "train_rows_N = shuffled_rows_L[:N]\n",
    "test_rows_T = shuffled_rows_L[N:]\n",
    "\n",
    "x_tr_NF = x_all_LF[train_rows_N].copy()\n",
    "y_tr_N = y_all_L[train_rows_N].copy()\n",
    "\n",
    "x_te_TF = x_all_LF[test_rows_T].copy()\n",
    "y_te_T = y_all_L[test_rows_T].copy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1697, 64)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_tr_NF.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(100, 64)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_te_TF.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Show the first image in training set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAS4AAAEPCAYAAAD8sCnQAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAkVklEQVR4nO3de1xUdf4/8BcEo8CgSciImYxgom0gXjYv6LpqmxdchX2kYVpCoalbostiJa1WamFQ9nA1SWK9wOhCF6V011aJNMy1cpGwTVAJx9CGFUqRi8Nlfn/wnfllDHLOMKeZD72ejwcPHp3z+Xzm/UB68TnnfM45LiaTyQQiIoG4OroAIiK5GFxEJBwGFxEJh8FFRMJhcBGRcBhcRCQcBhcRCYfBRUTCYXARkXAYXERkk1OnTmHIkCE4deqUpPZ6vR7x8fEYM2YMhg0bhsceewynT5+26bMZXEQkm16vx1NPPYWWlhZJ7Q0GA+bNm4cTJ05g/vz5iI+Px4ULFzB//nycOXNG9uczuIhIlvz8fMyePRuVlZWS+2zZsgVVVVXYsWMH/vjHPyImJgZ79uyBu7s7NmzYILsGBhcRSZaQkIDFixfDx8cHERERkvo0Nzfjgw8+wKhRozB48GDLdj8/P0REROD48eMwGAyy6mBwEZFk586dw1NPPYV9+/ZhwIABkvqcPXsWdXV1CA0NbbMvJCQEJpMJxcXFsupwk9WaiBympKQERqNRVp8zZ84gLS2t3f15eXmyxnv77behUqlk9THPpvz9/dvs8/PzAwBUVFTIGpMzLiJBGI1GWcFlNBrRs2dPu9YgN7QAoKamBgDg4eHRZp95W319vawxOeMiEohKpcK9994rqe3p06fh7+8ve1Zlb+Yrjy4uLu22cXWVN4dicBEJRrSHFnt5eQEAGhoa2uwzz7TUarWsMRlcRIIRLbj69esHAFavHJqXVFg7/3UrDC4iwYgWXAMGDICXl5fVK4fmbUOHDpU1Jk/OEwnGZDJJ+nIWKpUKDzzwAI4dO4bS0lLL9srKShw4cADjx4+Hj4+PrDE54yISiJxQMplMtzwhrpTCwkLo9XqEh4fD19cXALBs2TLk5+djwYIFiI2NRbdu3ZCZmYnm5mYkJibK/gzOuIgE4+wzruzsbKxcuRLnz5+3bOvbty92796NoUOHIi0tDZs3b0ZAQACysrIQHBws+zNc+F5FIjEUFxfDZDJh0KBBktqXlpbCxcUFISEhClf28+OhIpFgpD6RoStjcBEJhgdJDC4i4TC4GFxEwmFwMbiIhCLCcoifA4OLSDCccTG4iITD4GJwEQmHwcXgIhIOg4vBRSQcLkBlcBEJRe5Vxa6KwUUkmK4cSFIxuIgEw+By4uAqLCyEyWSCu7u7o0shUkxjYyNcXFwwbNgwyX0YXE4cXCaTCY2Njbh48WKHbd3c3KDRaGAwGNDU1PQzVHdrffr0UXR8T09PxcZW8sTv5cuXJbd1dXWFj48PqqurJdV048aNzpTmMH369JH9x5nB5cTB5e7ujosXLyIyMrLDtsHBwdDpdEhISEBJSYnyxXVg3759io4/a9Ysxca+evWqYmPHxMRIbtu/f3+sXr0aW7ZsgV6v77D9qVOnbC/Mgfbt24fAwEBZfRhcThxcRGQdg4vBRSQcBheDi0goXMfVyqbgunLlCjZu3IijR4+ipqYGgwcPxlNPPYXw8HB710dEP8GV8za85ae2thaxsbHYv38/oqKikJiYiPr6esTFxaGgoECJGonoR5z9LT8/B9kzrqysLJSWliItLQ0TJ04EAERGRiIyMhJr167FwYMHu+zDy4icQVcPJSlkz7hyc3MREBBgCS0A8PLywpw5c1BeXo6ioiK7FkhEN+OMS2Zw1dTUoKysDKGhoW32md/dxuAiUhaDS+ahosFggMlkgr+/f5t9fn5+AICKigr7VEZEbfCqYitZwVVTUwMA8PDwaLPPfBtKfX29Hcpq5ebmJun13Fqt9qbvjqb0/ZV1dXWKjd3Q0KDY2P3795fc1nzblNTbp+z5e/dzUqlUsvt05UCSSlZwmS/DWjv5bv5h2vPEvEajgU6nk9x+/fr1dvtsZ/b11187ugSbrF69WnafRYsWKVCJ2BhcMoPLy8sLgPW/yua/eN7e3nYoq5XBYEBCQkKH7bRaLdavX4+kpCSUl5fb7fNt9dprryk6/m9/+1vFxjbPqpUgJ4T69OmDRYsWYdu2bfjuu+86bO8M96jaYuPGjejXr5+sPgwumcFl/gEbDIY2+yorKwHA6vkvWzU1Ncn6hSwvL3eKX+DGxkZFx1fy6RBK1i7lZumf+u677yT1c4Z/d1sYjUbZfbgAVWZwqdVqaLVaFBcXt9ln3mbtiiMR2Q9nXDas45oxYwbOnTuHo0ePWrbV1tYiJycHQUFBlmURRKQMLoewYeV8bGwscnNzER8fj5iYGPTu3Rs5OTm4dOkS0tPTuWqeSEHOsByiM/cqFxcX4/XXX0dhYSGampowZMgQLFmyRPZ5W9kzLrVaDZ1Oh8mTJ0On0yElJQUeHh7IyMjA2LFj5Q5HRDI5csbVmXuV//vf/2L+/Pk4ffo0Hn/8cSQkJOD69etYvHgx/vGPf8iqw6anQ2g0GqSmptrSlYg6yZGHgZ25VzktLQ0NDQ3IzMy0nAuPiorC1KlTkZqaiunTp0uuQ/aMi4gcy5Ezrs7cq1xeXo5evXrddAGvR48eGDFiBCoqKnDt2jXJdTC4iATjqODq7L3KgYGBuHr1Kq5cuXLTdr1eD09PT6jVasm1/GKfgKrkCyeUHBsAjhw5otjYP/zwg2Jj79ixQ3Jbk8mE5uZmvPDCC5Iu+ISFhdlemGAcdajY2XuV4+PjcfLkScTHx+OZZ56Bt7c3MjMzcebMGfzpT3+Cq6v0edQvNriIRCVnAerly5exfPnydvfn5eVJHquz9yprtVosXrwYL730Eh588EHL9kceeQRPPPGE5DoABheRUBx5Yr6z9yqvWbMG2dnZGDZsGKKjo6FSqXDw4EFkZmbixo0bWLt2reRaGFxEgpEaXi4uLvD395c1q7qVztyr/M033yAnJwchISHIysqCm1tr9EyfPh3PP/889uzZg8mTJ0tez8WT80SCcdTJ+c7cq1xSUgKTyYTIyEhLaJlFR0cDAI4fPy65Fs64iATjqMPFztyrbH7uWHNzc5t95kNQa/vawxkXkWAcuY7L1nuVR44cCU9PT7z99tttTuCbn7k3btw4yXVwxkUkGEeeoJd6r3JhYSH0ej3Cw8Ph6+uLHj16YNWqVfjLX/6CqKgoPPjgg+jevTvy8/NRUFCAadOmybpfkcFFJBhHBpf5XuWUlBTodDo0NjZi8ODByMjIwOjRoy3tsrOzsXfvXuzatQu+vr4AgNmzZ6Nv377Ytm0btmzZgqamJmi1Wjz77LN49NFHZdXB4CISiDM8HULKvcrJyclITk5usz08PNwub7xncBEJhk9AZXARCaerPyRQCgYXkWAYXAwuIuEwuBhcRMJhcDG4iITD4GJwEQnFGZZDOAMGF5FgunIgScXgIhIMg4vBRSQcBheDi0g4XDnP4CISDmdcDC4iofCqYisGF5FgunIgSfWLDa4LFy4oNvatXoppD5GRkYqNPWHCBMXG3rlzp+S2DQ0N+Pbbb+Hv74/u3bsrVpOIGFy/4OAiEhWDi8FFJBwGF4OLSDgMLgYXkXAYXAwuIqGYTCbJC1C7csDZFFwlJSXYvHkzPvvsM9TW1sLf3x8RERFYunSp5cWPRKSMrhxIUskOLr1ej7lz58Ld3R0PP/ww/Pz88O9//xtbt27Fl19+iYyMDMu71YjI/hhcNgTXSy+9BKPRiHfeeQeBgYEAgLlz5yI5ORnbt29HXl4e7r//frsXSkStGFyAq5zGzc3N+OyzzzBy5EhLaJnNmjULAHDy5En7VUdEbZhv++noqyuTNeNydXVFbm6u1R9KdXU1AOC2226zT2VEZFVXDyUpZAWXi4sL7rrrLqv7zLdzjBo1qvNV/R83NzcEBwd32E6r1d70XYr+/fvbWFXHlP7FGjhwoGJj9+nTR7GxGxoaJLc1Go03fe+IlN8TZ2TLxSwGF+BissNPYfv27UhOTsaIESOwe/due9SF4uJi3Lhxgyf6qctTqVQICQnpsF1xcTGuX7+O8vJySeNqtVqo1WpJY4um0+u4du3ahQ0bNqB379549dVX7VGThcFgQEJCQofttFot1q9fj6SkJMn/qEr+hX7hhRcUGxsAFi5cqNjYI0aMUGxsOT8Xo9GIyspK+Pn5SZqVKHlzuJI2btyIfv36yerDGVcngstkMiE1NRVvvfUWevfujR07dsDf39+etaGpqQklJSWS25eXl0tu7+HhYWtZHVJ6lnju3DnFxr7zzjsVG9uWpzyoVCpJ/eT8njgTqYfCP8YnoNoYXEajEc8++yz279+P/v37IyMjQ9FzRkT0/3HGZUNwNTc3Y8WKFTh8+DBCQkKwbds2+Pj4KFEbEVnB4JK5jgsA/vrXv+Lw4cMYPnw4du7cydAi+pk5eh3XlStXkJSUhPHjxyMsLAzR0dE4duyYpL61tbVITU3FpEmTEBoaimnTpiEjIwNNTU2yapA14/rf//5nuaVn0qRJOHz4cJs2AwYMQGhoqKwiiEg6R864amtrERsbC71ejwULFkCj0SAnJwdxcXFIT0/HuHHj2u1rNBoRGxuL06dPIzo6GnfffTeOHDmCV155BRUVFVi9erXkOmQF1+eff245mZiammq1zUMPPcTgIlKIo1+WkZWVhdLSUqSlpWHixIkAWh8lHhkZibVr1+LgwYPtXpzasWMHioqKsG7dOsyePRtA6+2CS5YsgU6nwxNPPAGNRiOpDlnBNX36dEyfPl1OFyKyM0fOuHJzcxEQEGAJLQDw8vLCnDlzkJqaiqKiIoSFhVntm5OTg0GDBllCy2zJkiUYMmSIrAXKss9xEZFjOeocV01NDcrKyqweUZkXubb3opjLly/j4sWLGD9+vGVbbW0tWlpaEBoaimXLliEgIEByLQwuIsE4KrgMBgNMJpPV9Zp+fn4AgIqKCqt9y8rKALSuE9y+fTsmTJiA4cOHY+TIkVi3bh1u3Lghq5Zf7BNQT506pdjY7U2VRbBixQrFxv74448lt3V3d4dGo8HJkyfR2NioWE0ikrMA9fLly1i+fHm7+/Py8iSPVVNTA8D64m1PT08AQH19vdW+V69eBQBkZmbi2rVrWLRoEfz9/XH48GFkZmbi4sWLePPNNyXX8osNLiJROeoclzkwrZ18N9fU3ol58x+fb7/9Frm5uQgKCgIATJkyBe7u7nj33XdRUFBwy6uSP8bgIhKI3KuK/v7+smZVt+Ll5QXA+lM+zDMtb29vq33NM7LRo0dbQstszpw5ePfdd/Hpp58yuIi6KkfNuMw3gxsMhjb7KisrAaDd+5XNyxx8fX3b7LvjjjsAANevX5dcC4OLSDCOCi61Wg2tVovi4uI2+8zb2lvDOWjQIHTv3h2lpaVt9un1egCQ9ZQMXlUkEowjb/mZMWMGzp07h6NHj1q21dbWIicnB0FBQe0++6t79+6YOnUqvvrqK+Tn51u2t7S04G9/+xtuu+02TJkyRXIdnHERCcaRC1BjY2ORm5uL+Ph4xMTEoHfv3sjJycGlS5eQnp5uOTlfWFgIvV6P8PBwy+Hhn//8Z3z++eeIj4/Hww8/jLvuugsffvghTpw4gaVLl8pax8XgIhKMI4NLrVZDp9MhJSUFOp0OjY2NGDx4MDIyMjB69GhLu+zsbOzduxe7du2yBJc55DZt2oT9+/fj2rVrloeAPvjgg7LqYHARCcbRj7XRaDTt3qtslpycjOTk5DbbfX198eKLL+LFF1/sVA0MLiLBODq4nAGDi0ggJpNJ8sr5rhxwDC4iwXTlQJKKwUUkGAYXg4tIOAwuBheRcBhcDC4i4TC4GFxEQnH0M+edBYOLSDBdOZCkYnARCYbBxeAiEo6cRzd3VQwuIsFwxsXgIhIOg4vBRSQcBheDS0i33367YmNPmDBBsbHlvPrMx8cH06dPx8mTJ1FdXa1YTaLhcohWDC4iwXTlQJKKwUUkGAYXg4tIOAwuBheRcBhcDC4i4XABaiffq9jQ0IBp06Zh0qRJ9qqHiDrgyPcqOotOBdcrr7yCsrIye9VCRB2QGlpdPbxsPlT85JNPsHv3bri7u9uzHiLqQFcOJKlsmnH98MMPWLVqFaKiotCvXz9710REt/BLn20BNgbX6tWr4e7ujqSkJHvXQ0QdYHDZcKi4b98+/Otf/8LOnTuhVquVqImIbqGrh5IUsoKroqIC69atw4IFCzBq1CilarJwc3NDcHBwh+20Wu1N37s6b29vxcauq6tTbGwfHx/JbXv06HHT945I+T1xRiqVSnYfBhfgYpL4U2hpacGCBQtQVVWFvXv3olu3bgCAqVOnwmg04qOPPrJrYcXFxbhx4wZcXFzsOi6Rs1GpVAgJCemwXXFxMaqrq/H+++9LGnfmzJnw8fGRNLZoJM+4tm/fji+++ALbtm1DbW0tamtrAbQGWktLC6qrq+Hu7m7X2YDBYEBCQkKH7bRaLdavX4+kpCSUl5fb7fOdlZIzriNHjig29quvviq5bY8ePTBu3DgUFBTg2rVrHbbX6XSdKc1hNm7cKOsCl8lkkrwAtSvPzCQH18cff4yWlhbExcVZ3T9mzBjcd999yMzMtFtxTU1NKCkpkdy+vLxcVntRKflYG09PT8XGtuXxNNeuXZPUT9R/d6PRKLtPVw4kqSQH19NPP231L9+qVavQ2NiIlJQUyecjiMh2DC4ZwXXvvfda3d69e3e4urpi7NixdiuKiNrH4OrkLT9E9PNz9DquK1euICkpCePHj0dYWBiio6Nx7Ngx2eNUVVVhzJgxeOSRR2T3ZXARCcaRwVVbW4vY2Fjs378fUVFRSExMRH19PeLi4lBQUCBrrKSkJJsfy93px9ocPHiws0MQkQyOPFTMyspCaWkp0tLSMHHiRABAZGQkIiMjsXbtWhw8eFDSEqbs7Gx88skncHW1be7EGReRQBz9dIjc3FwEBARYQgsAvLy8MGfOHJSXl6OoqKjDMS5cuIDk5GQsWbIEbm62zZ0YXESCcVRo1dTUoKysDKGhoW32mRe5dhRczc3NWLlyJYKCgrB48WKba+ETUIkEIyeULl++jOXLl7e7Py8vT/JYBoMBJpMJ/v7+bfb5+fkBaL0t8Fa2bt2KM2fOYO/evTbPtgAGl5Bef/11xca+evWqYmPLqTs4OBjTp0+HTqcTdnGpUhz16OaamhoAgIeHR5t95oXL9fX17fYvLi7G1q1b8fTTTyMwMLBTtTC4iAQjZ8bl7+8va1Z1K+bAtHby3VxTeyfmGxoakJiYiF//+tc2LX/4KQYXkWAcdVXRy8sLQGsI/ZR5ptXefbSvvPIKvvvuO7z22mv4/vvvb9rX2NiI6upqdOvWzfIZHWFwEQlEzol3ewec+WZwg8HQZl9lZSUAWD3/BbTe61xfX4+oqKg2+woLCzFmzBhERUUhOTlZUi0MLiLBOGrGpVarodVqUVxc3GafeZu1K44AkJKSghs3brTZvmjRIgQGBuKZZ56xnOCXgsFFJBhHLkCdMWMGNm/ejKNHj+I3v/kNgNbV9Dk5OQgKCmr32V8jRoywut3FxQU9e/aUfa8zg4tIMI4MrtjYWOTm5iI+Ph4xMTHo3bs3cnJycOnSJaSnp1tOzhcWFkKv1yM8PBy+vr52r4MLUIkE48h7FdVqNXQ6HSZPngydToeUlBR4eHggIyPjpllTdnY2Vq5cifPnzytSB2dcRIJx9GNtNBoNUlNTb9kmOTlZ0ol2a+fLpGBwEQnGUQtQnQmDi0ggjlwO4UwYXESC6cqBJBWDi0gwDC4GF5FwGFwMLiLhMLgYXETCYXAxuIiEw+BicBEJhcshWjG4iATDBagMLiLhdOWZlFQMLiLBMLgYXETCYXAxuIiEw+BicCni448/VnT8CRMmKDq+Uvbt2ye5rbu7OwDgtddeQ2NjY4ftY2JibKxKmh9++EHR8eVgcDG4iITC5RCtGFxEgunKgSQVg4tIMAwuBheRcLgAlcFFJBzOuGwMrsbGRmzfvh3vvfceKioq0KdPH0ybNg2LFy+Gp6envWskoh9hcNkYXCtWrMChQ4cwc+ZMxMTEoLCwEG+++SZKS0uxdetWy7vViMi+eFWxlezgOnDgAA4dOoTFixdjxYoVAIDo6Gh4e3sjMzMThYWFGD58uN0LJaJWXTmQpJIdXNnZ2ejVqxeWLl160/ZHHnkEarUaKpXKbsURUVsMLpnB1dTUhMLCQkyaNAndunUDANTX10OlUiEgIADLly9XokYi+hEGl8zg+vbbb2E0GtGvXz/s27cPW7duRXl5OVQqFaZOnYrnnnsOPXv2tF9xbm4IDg7usJ1Wq73pu6O5uroqOn5dXZ2i4yvFfBuPFG5ubjd978jAgQNtqkmqmpoaRca15QiFwSUzuK5duwYA+Oijj7B7924sWrQIAwcOxIkTJ6DT6XD+/Hn8/e9/t9vhokajgU6nk9x+/fr1dvlcZ/f11187ugSbaDQa2X3uuOMOSe3S0tJkjy0qBpfM4DIajQCAsrIy7NixA2PGjAEA/O53v0OvXr2wadMm7N27Fw899JBdijMYDEhISOiwnVarxfr165GUlITy8nK7fHZnpKenKzr+iBEjFB1fKXJuPndzc8Mdd9yBqqoqNDU1ddh+zZo1naisY0rNuDZu3Ih+/frJ6sMFqDKDy7xGKzAw0BJaZnPmzMGmTZvw6aef2i24mpqaUFJSIrl9eXm5rPZKUfoXS9S1clKe8vBTTU1NkvqdO3fOlpIkU+rpEObJgFRcDtFKVnD16dMHAODr69tmn4+PD1xcXHD9+nX7VEZEVnXlQJJKVnD5+PjgzjvvxPnz59HS0nLTSeiLFy/CZDLJnvYSkTwMLkD25a/IyEhUVVVhz549N21/6623AADTpk2zT2VEZJX5cLGjL6VcuXIFSUlJGD9+PMLCwhAdHY1jx45J6vvFF1/g8ccfx8iRI3HvvfciIiICO3bskH16RfYC1IULFyI/Px/r1q1DSUkJ7rnnHhQUFODQoUOYNWsWRo8eLXdIIpLBkTOu2tpaxMbGQq/XY8GCBdBoNMjJyUFcXBzS09Mxbty4dvv+5z//waOPPgqNRoO4uDh4e3sjLy8PL7/8Ms6ePStrVYDs4PLw8EBmZibeeOMN/POf/8R7772HO++8E4mJiXjsscfkDkdEMjkyuLKyslBaWoq0tDRMnDgRQOtRWGRkJNauXYuDBw+2e6/ymjVr4O3tjXfffRc+Pj4AgHnz5mHZsmV45513MH/+fAwZMkRSHTatlFSr1Vi5ciXy8/Nx+vRpfPjhh4iLi1N84SUROfZQMTc3FwEBAZbQAgAvLy/MmTMH5eXlKCoqstqvqqoKpaWluP/++y2hZTZz5kwAwMmTJyXXwedxEQnGUTOumpoalJWVYcaMGW32hYSEAACKiooQFhbWZn/Pnj3x4YcfWm4V/LHq6moAwG233Sa5FgYXkUDkruO6fPnyLe8hzsvLk/zZBoMBJpMJ/v7+bfb5+fkBACoqKqz2dXNzs3pLXnNzMzIzM+Hi4oL77rtPci0MLiLBOGrlvPnuAQ8Pjzb7zIui6+vrZY358ssvo7S0FL///e8RFBQkuR+DSwG33367ouO3dx7BHuS8+1CuyMhIyW1NJhOam5vRt29fSQ+m/P777ztRWcdeeOEFRcbt1auX7D5yDhX9/f1lzapuxRyY1v49zDXJeYjohg0bkJmZiYEDB+L555+XVQuDi0gwjjrH5eXlBQBoaGhos8880/L29u5wHKPRiFWrVuGDDz5AYGAgtm/fDrVaLasWBheRYBwVXOa7YgwGQ5t9lZWVAGD1/NePXb9+HUuXLsWJEycQEhKCbdu2tbnKKAWDi0gwjgoutVoNrVaL4uLiNvvM20JDQ9vtX1dXh8cffxynTp3ChAkT8Prrr9v8wAAuvCISiNQ1XEqt5ZoxYwbOnTuHo0ePWrbV1tYiJycHQUFBlmUR1qxZswanTp3ClClT8MYbb3TqKSeccREJxpEr52NjY5Gbm4v4+HjExMSgd+/eyMnJwaVLl5Cenm45OV9YWAi9Xo/w8HD4+vriq6++wvvvvw+VSoWxY8fiwIEDbca+5557cPfdd0uqg8FFJBhHBpdarYZOp0NKSgp0Oh0aGxsxePBgZGRk3HSfcnZ2Nvbu3Ytdu3bB19cXn376KYDWE/PtPfQxISGBwUXUVTn6sTYajQapqam3bJOcnIzk5GTLfy9cuBALFy60Ww0MLiLB8NHNDC4i4Th6xuUMGFxEgmFwMbiIhMKXZbRicBEJpisHklQMLiLBMLgYXETCYXAxuIiEw+BicBEJh8HF4CISislkkrwAtSsHHIOLSDBdOZCkYnARCYbBxeAiEg6Di8FFJBwGlxMHV2NjI/r06SPprTMqlQoAsHHjRhiNRoUr61hTU5OjS7DZj99QbG+2/Fyam5sltbP2OGF7Uurn4uHhgcbGRll9GFxOHFwuLi5wd3dHYGCg5D7mh/mT7cx/BOhmtrxGTIrGxkZZr/QCGFyAEwfXsGHDHF0CkVNicDlxcBFRW3w6RCsGF5Fg+ARUBheRcLryTEoqBheRYBhcDC4i4TC4GFxEwmFwMbiIhMKriq0YXESC6cqBJBWDi0gwDC4GF5FwGFwMLiLhMLgYXETC4cp5BheRcDjjYnARCYXLIVq5OroAIpLHHF4dfSnlypUrSEpKwvjx4xEWFobo6GgcO3ZMUl+9Xo/4+HiMGTMGw4YNw2OPPYbTp0/LroHBRSQYRwZXbW0tYmNjsX//fkRFRSExMRH19fWIi4tDQUHBLfsaDAbMmzcPJ06cwPz58xEfH48LFy5g/vz5OHPmjKw6eKhIJBhHHgJmZWWhtLQUaWlplsdZR0ZGIjIyEmvXrsXBgwfbfaLrli1bUFVVhffeew+DBw8GAEyfPh0RERHYsGEDtm/fLrkO4WdcnZm2kvN55plnEBwcbPXr1KlTji7PKThyxpWbm4uAgICbnsHv5eWFOXPmoLy8HEVFRVb7NTc344MPPsCoUaMsoQUAfn5+iIiIwPHjx2EwGCTXIfSMyzxt1ev1WLBgATQaDXJychAXF4f09HSMGzfO0SWSTCUlJRgwYACWLFnSZl///v0dUJHzcdSMq6amBmVlZZgxY0abfSEhIQCAoqIihIWFtdl/9uxZ1NXVITQ01GrfPXv2oLi4GBqNRlItQgdXZ6at5Hyam5tx/vx5REREYNasWY4uxylpNBrs3btXctvLly9j+fLl7bbJy8uT/NkGgwEmkwn+/v5t9vn5+QEAKioq2u0LwKa+1ggdXLeatqamprab/uScvvnmG9y4cQODBg1ydClOyfwGpgEDBkjuo9fr7fb5NTU1AFpfqfZTnp6eAID6+nrZfc3b2utrjbDB1ZlpKzmnkpISAMDAgQMBAHV1dejevTtcXYU/FWsXwcHBsvuEhIRg9uzZdvl884p9a0cx5sPX9o5wbtXXTM6/s7C/EZ2ZtpJzMgfX4cOHMWHCBAwbNgzDhw9HYmIiqqurHVwdeXl5AQAaGhra7DPPlry9vW3uq1arJdci9IwLsG3aSs7JHFxfffUVli9fDk9PTxw/fhzZ2dn48ssv8c4777T7PwYpz/zCZWtX/yorKwFYP4fV2b7WCBtcnZm2knOKiIhASEgIFi1aZDmfM2XKFAwYMAAvvfQSdu7ciSeffNLBVf5yqdVqaLVaFBcXt9ln3mbtqiHQel7Oy8vrln2HDh0quRZhDxU7M20l5zRz5kw8+eSTltAymzt3Ltzc3Lg+zwnMmDED586dw9GjRy3bamtrkZOTg6CgIMv55Z9SqVR44IEHcOzYMZSWllq2V1ZW4sCBAxg/fjx8fHwk1yHsjMveU09yXiqVCt7e3qitrXV0Kb94sbGxyM3NRXx8PGJiYtC7d2/k5OTg0qVLSE9PtxzlFBYWQq/XIzw8HL6+vgCAZcuWIT8/HwsWLEBsbCy6deuGzMxMNDc3IzExUVYdws64OjNtJedTV1eHmTNnIj4+vs2+6upqfP/991yA6gTUajV0Oh0mT54MnU6HlJQUeHh4ICMjA2PHjrW0y87OxsqVK3H+/HnLtr59+2L37t0YOnQo0tLSsHnzZgQEBCArK0v+FVOTwDZt2mQaNGiQ6ciRI5Zt169fN91///2madOmmVpaWhxYHcn1hz/8wfSrX/3K9PXXX9+0/bnnnjMNGjTIdOjQIQdVRs7GxWQS96E9169fR2RkJKqqqm6atp49exbp6ek3/QUg53f69GnMnTsXnp6emDdvHnx8fJCfn4+CggJERkZiw4YNji6RnITQwQW0nuNKSUnB0aNH0djYiMGDByM+Ph6jR492dGlkgy+//BKbN2/GyZMnYTQaodVq8dBDD+Hhhx/mQlSyED64iOiXh3/CiEg4DC4iEg6Di4iEw+AiIuEwuIhIOAwuIhIOg4uIhMPgIiLhMLiISDgMLiISDoOLiITD4CIi4fw/O455Gw6CNaAAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 300x300 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(x_tr_NF[0].reshape((8,8)), interpolation='nearest', vmin=0, vmax=1, cmap='gray')\n",
    "plt.colorbar();\n",
    "plt.gcf().set_size_inches((3,3));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Show fifth row image in training set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAS4AAAEPCAYAAAD8sCnQAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAjxElEQVR4nO3df1xUVf4/8BcEKDDoSsCImoxg/mgD0Sx/oGtqmymuDftIxR+lFJm6JRqLlbRrpRYGu7auJUmsKIzG9EMp3bVVljLNtXKRsF1BpXEMbUitxBHk13z+4DvzzWaQe4e5zRx8PR8PHz26954z74cjL86995x7vSwWiwVERALxdncBRERyMbiISDgMLiISDoOLiITD4CIi4TC4iEg4DC4iEg6Di4iEw+AiIuEwuIjIKUePHsXgwYNx9OhRSccbjUakpKRg1KhRGDp0KB5++GEcO3bMqc9mcBGRbEajEU888QRaWlokHW8ymTBnzhwcPnwYc+fORUpKCk6fPo25c+fi+PHjsj+fwUVEspSUlGD69OmoqamR3ObVV1/FhQsXkJeXh9/97neYP38+tm/fDl9fX6xdu1Z2DQwuIpIsNTUVCxcuRHBwMOLj4yW1aW5uxvvvv48RI0Zg0KBBtu1hYWGIj4/HoUOHYDKZZNXB4CIiyU6ePIknnngCO3fuRL9+/SS1OXHiBK5cuYKYmBi7fdHR0bBYLCgvL5dVh4+so4nIbSoqKtDQ0CCrzfHjx5Gdnd3m/uLiYln9vfXWW/Dz85PVxjqaCg8Pt9sXFhYGAKiurpbVJ0dcRIJoaGiQFVwNDQ3o3r27S2uQG1oAUFtbCwDw9/e322fdVldXJ6tPjriIBOLn54fbb79d0rHHjh1DeHi47FGVq1nvPHp5ebV5jLe3vDEUg4tIMKI9tDgwMBAAUF9fb7fPOtJSqVSy+mRwEQlGtODq06cPADi8c2idUuHo+tf1MLiIBCNacPXr1w+BgYEO7xxatw0ZMkRWn7w4TyQYi8Ui6Y+n8PPzw7333ouDBw+isrLStr2mpga7d+/G2LFjERwcLKtPjriIBCInlCwWy3UviCultLQURqMRcXFxCAkJAQAsWbIEJSUlmDdvHpKSktClSxfk5+ejubkZaWlpsj+DIy4iwXj6iKuwsBDLly/HqVOnbNt69eqFbdu2YciQIcjOzsaGDRsQERGBgoICDBw4UPZnePG9ikRiKC8vh8ViwYABAyQdX1lZCS8vL0RHRytc2c+Pp4pEgpH6RIbOjMFFJBieJDG4iITD4GJwEQmHwcXgIhKKCNMhfg4MLiLBcMTF4CISDoOLwUUkHAYXg4tIOAwuBheRcDgBlcFFJBS5dxU7KwYXkWA6cyBJxeAiEgyDy4ODq7S0FBaLBb6+vu4uhUgxjY2N8PLywtChQyW3YXB5cHBZLBY0NjbizJkz7R7r4+MDtVoNk8mEpqamn6G66+vSpYui/YeGhirW9w8//KBY39bXVEnhad+pUnr27Cn7lzODy4ODy9fXF2fOnIFWq2332IEDB0Kn0yE1NRUVFRXKF9eO2NhYRfvPy8tTrO/8/HzF+pZTt6d9p0rZuXMnIiMjZbVhcHlwcBGRYwwuBheRcBhcDC4ioXAeVyunguv8+fNYt24d9u/fj9raWgwaNAhPPPEE4uLiXF0fEf0EZ8478ZYfs9mMpKQk7Nq1CwkJCUhLS0NdXR2Sk5Nx4MABJWokoh/x9Lf8/Bxkj7gKCgpQWVmJ7OxsjB8/HgCg1Wqh1WqxatUq7Nmzp9M+vIzIE3T2UJJC9oirqKgIERERttACgMDAQMyYMQMGgwFlZWUuLZCIrsURl8zgqq2tRVVVFWJiYuz2Wd/dxuAiUhaDS+aposlkgsViQXh4uN2+sLAwAEB1dbVrKiMiO7yr2EpWcFmXbPj7+9vtCwgIAADU1dW5oKxWPj4+kl7PrdForvmvu/Xt21fR/pX8B6nkciI5r1r3tO9UKX5+frLbdOZAkkpWcFlvwzq6+G79y3TlhXm1Wg2dTif5+DVr1rjssz1Zc3OzYn3PnDnTo/q+Ub5TORhcMoMrMDAQAFBfX2+3zzrSCgoKckFZrUwmE1JTU9s9TqPRYM2aNUhPT4fBYHDZ5ztLzsjCGc8//7xifW/btk2xvt977z3Jx3rad6qUdevWoU+fPrLaMLhkBpf1L9hkMtntq6mpAQCH17+c1dTUJGuBrcFg8IgFuY5OpV1Jyekm3377rWJ9O/PdeMp3qpSGhgbZbTgBVWZwqVQqaDQalJeX2+2zbnN0x5GIXIcjLifmcU2dOhUnT57E/v37bdvMZjP0ej2ioqJs0yKISBmcDuHEzPmkpCQUFRUhJSUF8+fPR2hoKPR6Pc6ePYucnBzOmidSkCdMh+jIWuXy8nK88sorKC0tRVNTEwYPHoxFixbh7rvvllWD7BGXSqWCTqfDxIkTodPpkJmZCX9/f+Tm5mL06NFyuyMimdw54urIWuX//ve/mDt3Lo4dO4ZHHnkEqampuHz5MhYuXIi///3vsupw6ukQarUaWVlZzjQlog5y52lgR9YqZ2dno76+Hvn5+bZr4QkJCbjvvvuQlZWFKVOmSK5D9oiLiNzLnSOujqxVNhgM6NGjxzU38Lp164Y77rgD1dXVuHTpkuQ6GFxEgnFXcHV0rXJkZCR++OEHnD9//prtRqMRAQEBUKlUkmvhE1AVsHTpUkX7V3IZzIcffqhY3+Qa7jpV7Oha5ZSUFBw5cgQpKSl4+umnERQUhPz8fBw/fhxPPvkkvL2lj6MYXESCkTMB9dy5c9f9RVpcXCy5r46uVdZoNFi4cCFefPFFPPDAA7btDz74IB577DHJdQAMLiKhuPPCfEfXKq9cuRKFhYUYOnQoEhMT4efnhz179iA/Px9Xr17FqlWrJNfC4CISjNTw8vLyQnh4uKxR1fV0ZK3yV199Bb1ej+joaBQUFMDHpzV6pkyZgueeew7bt2/HxIkTJc/n4sV5IsG46+J8R9YqV1RUwGKxQKvV2kLLKjExEQBw6NAhybVwxEUkGHedLnZkrbL1uWOOHslkPQWV87gmjriIBOPOeVzOrlUePnw4AgIC8NZbb9ldwLc+c2/MmDGS6+CIi0gw7rxAL3WtcmlpKYxGI+Li4hASEoJu3bphxYoV+MMf/oCEhAQ88MAD6Nq1K0pKSnDgwAFMnjxZ1npFBheRYNwZXNa1ypmZmdDpdGhsbMSgQYOQm5uLkSNH2o4rLCzEjh07sHXrVoSEhAAApk+fjl69emHTpk149dVX0dTUBI1Gg2eeeQYPPfSQrDoYXEQC8YSnQ0hZq5yRkYGMjAy77XFxcS554z2Di0gwfAIqg4tIOJ39IYFSMLiIBMPgYnARCYfBxeAiEg6Di8FFJBwGF4OLSCieMB3CEzC4iATTmQNJKgYXkWAYXAwuIuEwuBhcRMLhzHkGF5FwOOJicBEJhXcVWzG4iATTmQNJqhs2uJR8N6FWq1WsbwCyHrgml8FgUKxvcg0G1w0cXESiYnAxuIiEw+BicBEJh8HF4CISDoOLwUUkFIvFInkCamcOOKeCq6KiAhs2bMCnn34Ks9mM8PBwxMfHY/HixbYXPxKRMjpzIEklO7iMRiNmzZoFX19fzJ49G2FhYfj3v/+NjRs34osvvkBubq7t3WpE5HoMLieC68UXX0RDQwPefvttREZGAgBmzZqFjIwMbN68GcXFxbjnnntcXigRtWJwAd5yDm5ubsann36K4cOH20LL6v777wcAHDlyxHXVEZEd67Kf9v50ZrJGXN7e3igqKnL4l3Lx4kUAwE033eSayojIoc4eSlLICi4vLy/ccsstDvdt2bIFADBixIiOV/X/+Pj4YODAge0eZ12+I2cZT69evZysqn319fWK9Q0Affv2Vazvuro6xfqWw5nvVETO3MxicAFeFhf8LWzevBkZGRm44447sG3bNlfUhfLycly9epUX+qnT8/PzQ3R0dLvHlZeX4/Lly5LXk2o0GqhUKkl9i6bD87i2bt2KtWvXIjQ0FH/6059cUZONyWRCampqu8dpNBqsWbMG6enpkr9UJUdc27dvV6xvAFiwYIFifVdUVCjWtxzOfKciWrduHfr06SOrDUdcHQgui8WCrKwsvPHGGwgNDUVeXh7Cw8NdWRuamppk/SAZDAbJx1+9etXZstrVtWtXxfoGWqekKMVTgstKzncqooaGBtlt+ARUJ4OroaEBzzzzDHbt2oW+ffsiNzdX0esuRPT/ccTlRHA1Nzdj2bJl2LdvH6Kjo7Fp0yYEBwcrURsROcDgkjmPCwD++te/Yt++fRg2bBi2bNnC0CL6mbl7Htf58+eRnp6OsWPHIjY2FomJiTh48KCktmazGVlZWZgwYQJiYmIwefJk5ObmoqmpSVYNskZc3377rW1Jz4QJE7Bv3z67Y/r164eYmBhZRRCRdO4ccZnNZiQlJcFoNGLevHlQq9XQ6/VITk5GTk4OxowZ02bbhoYGJCUl4dixY0hMTMStt96Kjz76CC+//DKqq6vxxz/+UXIdsoLrs88+s11MzMrKcnjMzJkzGVxECnH3yzIKCgpQWVmJ7OxsjB8/HkDro8q1Wi1WrVqFPXv2tDmFKS8vD2VlZVi9ejWmT58OoHW54KJFi6DT6fDYY49BrVZLqkNWcE2ZMgVTpkyR04SIXMydI66ioiJERETYQgsAAgMDMWPGDGRlZaGsrAyxsbEO2+r1egwYMMAWWlaLFi3C4MGDZU3cln2Ni4jcy13XuGpra1FVVeXwjMo6ybWsrMxh23PnzuHMmTMYO3asbZvZbEZLSwtiYmKwZMkSRERESK6FwUUkGHcFl8lkgsVicThfMywsDABQXV3tsG1VVRUAoHfv3ti8eTPGjRuHYcOGYfjw4Vi9erXseZU37BNQlVwD9/333yvWNwAcPXpU0f7Js8mZgHru3DksXbq0zf3FxcWS+6qtrQUA+Pv72+0LCAgA0PZa1x9++AEAkJ+fj0uXLmHBggUIDw/Hvn37kJ+fjzNnzuD111+XXMsNG1xEonLXNS5rYDq6+G6tqa0L842NjQCAr7/+GkVFRYiKigIATJo0Cb6+vnjnnXdw4MCB696V/DEGF5FA5N5VDA8PlzWqup7AwEAAjp9+Yh1pBQUFOWxrHZGNHDnSFlpWM2bMwDvvvINPPvmEwUXUWblrxGVdDG4ymez21dTUAECb65Wt0xxCQkLs9t18880AgMuXL0uuhcFFJBh3BZdKpYJGo0F5ebndPuu2tuZwDhgwAF27dkVlZaXdPutDA+Q8JYN3FYkE484lP1OnTsXJkyexf/9+2zaz2Qy9Xo+oqKg2n/3VtWtX3Hffffjyyy9RUlJi297S0oK//e1vuOmmmzBp0iTJdXDERSQYd05ATUpKQlFREVJSUjB//nyEhoZCr9fj7NmzyMnJsV2cLy0thdFoRFxcnO308Pe//z0+++wzpKSkYPbs2bjlllvwwQcf4PDhw1i8eLGseVwMLiLBuDO4VCoVdDodMjMzodPp0NjYiEGDBiE3NxcjR460HVdYWIgdO3Zg69attuCyhtz69euxa9cuXLp0yfbAyAceeEBWHQwuIsG4+7E2arW6zbXKVhkZGcjIyLDbHhISghdeeAEvvPBCh2pgcBEJxt3B5QkYXEQCsVgskmfOd+aAY3ARCaYzB5JUDC4iwTC4GFxEwmFwMbiIhMPgYnARCYfBxeAiEoq7nznvKRhcRILpzIEkFYOLSDAMLgYXkXDkPLq5s2JwEQmGIy4GF5FwGFwMLiLhMLgYXEIyGAyK9S3nYW5ytfWyUEcsFguam5vx5ptvtvnmmB/TarUdqKx9Sv6dy8HpEK0YXESC6cyBJBWDi0gwDC4GF5FwGFwMLiLhMLgYXETC4QTUDr5Xsb6+HpMnT8aECRNcVQ8RtcOd71X0FB0KrpdffhlVVVWuqoWI2iE1tDp7eDl9qvjxxx9j27Zt8PX1dWU9RNSOzhxIUjk14vr++++xYsUKJCQkoE+fPq6uiYiu40YfbQFOBtcf//hH+Pr6Ij093dX1EFE7GFxOnCru3LkT//znP7FlyxaoVColaiKi6+jsoSSFrOCqrq7G6tWrMW/ePIwYMUKpmmx8fHwwcODAdo/TaDTX/FeK3r17O1lV+5qamhTrW2lXrlxRrG85P3DWY6W26devn1M1SdWlSxdF+vXz85PdhsElI7haWlrw9NNPIywsDE8++aSSNdmo1WrodDrJx69Zs0bBaqT7/vvv3V2C0zytdqlzljIzMxWuxHMwuGQE1+bNm/H5559j06ZNMJvNMJvNAFr/YbW0tODixYvw9fVFUFCQy4ozmUxITU1t9ziNRoM1a9YgPT1d8ir+4cOHd7C6tj3//POK9a208PBwxfqurKyUfKz1VfPe3t6Sng6h9C/Ts2fPKtLvunXrZN3gsv69SD22s5IcXB9++CFaWlqQnJzscP+oUaNw1113IT8/32XFNTU1oaKiQvLxBoNB8vFK/oD6+Ii7ICEgIECxvqUEkKM2Utp99dVXzpQkmVKPtWloaJDdpjMHklSSf8KeeuopXLp0yW77ihUr0NjYiMzMTHTr1s2lxRGRPQaXjOC6/fbbHW7v2rUrvL29MXr0aJcVRURtY3B1cMkPEf383D2P6/z580hPT8fYsWMRGxuLxMREHDx4UHY/Fy5cwKhRo/Dggw/KbsvgIhKMO4PLbDYjKSkJu3btQkJCAtLS0lBXV4fk5GQcOHBAVl/p6em4ePGiU3V0+Crynj17OtoFEcngzlPFgoICVFZWIjs7G+PHjwfQ+rx/rVaLVatWYc+ePZJuphQWFuLjjz+Gt7dzYyeOuIgE4u6nQxQVFSEiIsIWWgAQGBiIGTNmwGAwSHohyunTp5GRkYFFixY5fQeewUUkGHeFVm1tLaqqqhATE2O3Lzo6GkD7b3Jqbm7G8uXLERUVhYULFzpdi7gTjohuUHJC6dy5c1i6dGmb+4uLiyX3ZTKZYLFYHM6BDAsLA9C6LPB6Nm7ciOPHj2PHjh0dmu94wwaXkktblHw3IQD85S9/Uazv5557TrG+Y2NjJR/bu3dvLFu2DFlZWe3+MABAXl6e84VJcPfddyvavxzuenRzbW0tAMDf399un3Xicl1dXZvty8vLsXHjRjz11FOIjIzsUC03bHARiUrOiCs8PFzWqOp6rIHp6OK7taa2LszX19cjLS0Nd955p1PTH36KwUUkGHfdVQwMDATQGkI/ZR1ptbVW+eWXX8Y333yDP//5z/juu++u2dfY2IiLFy+iS5cuts9oD4OLSCByLry7OuCsi8FNJpPdvpqaGgBtrwH+8MMPUVdXh4SEBLt9paWlGDVqFBISEpCRkSGpFgYXkWDcNeJSqVTQaDQoLy+322fd5uiOI9D62KGrV6/abV+wYAEiIyNtj8ySisFFJBh3TkCdOnUqNmzYgP379+NXv/oVgNbZ9Hq9HlFRUbZpET91xx13ONzu5eWF7t27y17rzOAiEow7gyspKQlFRUVISUnB/PnzERoaCr1ej7NnzyInJ8d2cb60tBRGoxFxcXEICQlxeR2cgEokGHeuVVSpVNDpdJg4cSJ0Oh0yMzPh7++P3Nzca0ZNhYWFWL58OU6dOqVIHRxxEQnG3Y+1UavVyMrKuu4xGRkZki60O7peJgWDi0gw7pqA6kkYXEQCced0CE/C4CISTGcOJKkYXESCYXAxuIiEw+BicBEJh8HF4CISDoOLwUUkHAYXg4tIKJwO0YrBRSQYTkBlcBEJpzOPpKRicBEJhsHF4CISDoOLwUUkHAbXDRxcR48eVazv06dPK9Y3APziF79QrO9x48Yp1ndSUpLkY319fQEAs2fPRmNjY7vHK/m6OU/D4LqBg4tIRJwO0YrBRSSYzhxIUjG4iATD4GJwEQmHE1AZXETC4YjLyeBqbGzE5s2b8e6776K6uho9e/bE5MmTsXDhQgQEBLi6RiL6EQaXk8G1bNky7N27F9OmTcP8+fNRWlqK119/HZWVldi4caPt3WpE5Fq8q9hKdnDt3r0be/fuxcKFC7Fs2TIAQGJiIoKCgpCfn4/S0lIMGzbM5YUSUavOHEhSyQ6uwsJC9OjRA4sXL75m+4MPPgiVSgU/Pz+XFUdE9hhcMoOrqakJpaWlmDBhArp06QIAqKurg5+fHyIiIrB06VIlaiSiH2FwyQyur7/+Gg0NDejTpw927tyJjRs3wmAwwM/PD/fddx+effZZdO/e3XXF+fhg4MCB7R6n0Wiu+a+7NTU1Kdp/165dFeu7Z8+eivVtXcYjhY+PzzX/dTcp/w6d4cwZCoNLZnBdunQJAPCvf/0L27Ztw4IFC9C/f38cPnwYOp0Op06dwptvvumy00W1Wg2dTif5+DVr1rjkcztK6XVz/fv3F7JvZ9x8883uLgEAZP07VBqDS2ZwNTQ0AACqqqqQl5eHUaNGAQB+/etfo0ePHli/fj127NiBmTNnuqQ4k8mE1NTUdo/TaDRYs2YN0tPTYTAYXPLZHbF7925F+//8888V67ukpESxvqdNmyb5WB8fH9x88824cOGC4iNYKZ588klF+l23bh369Okjqw0noMoMLuscrcjISFtoWc2YMQPr16/HJ5984rLgampqQkVFheTjDQaDrOOVovTpTX19vWJ9f/PNN4r1LeUpDz/V1NTkVDtXU+rflXUwIBWnQ7SS9RNmvf4REhJity84OBheXl64fPmyayojIoc6cyBJJSu4goOD0bt3b5w6dQotLS3w9va27Ttz5gwsFovsYS8RycPgArzbP+RaWq0WFy5cwPbt26/Z/sYbbwAAJk+e7JrKiMgh6+lie3+Ucv78eaSnp2Ps2LGIjY1FYmIiDh48KKnt559/jkceeQTDhw/H7bffjvj4eOTl5cm+bif7Ysyjjz6KkpISrF69GhUVFbjttttw4MAB7N27F/fffz9Gjhwpt0siksGdIy6z2YykpCQYjUbMmzcParUaer0eycnJyMnJwZgxY9ps+5///AcPPfQQ1Go1kpOTERQUhOLiYrz00ks4ceKErFkBsoPL398f+fn5eO211/CPf/wD7777Lnr37o20tDQ8/PDDcrsjIpncGVwFBQWorKxEdnY2xo8fD6D1LEyr1WLVqlXYs2dPm2uVV65ciaCgILzzzjsIDg4GAMyZMwdLlizB22+/jblz52Lw4MGS6pB9qggAKpUKy5cvR0lJCY4dO4YPPvgAycnJ11zzIiJluPNUsaioCBEREbbQAoDAwEDMmDEDBoMBZWVlDttduHABlZWVuOeee2yhZWWdJnPkyBHJdXjGtGQiksxdI67a2lpUVVVh6tSpdvuio6MBAGVlZYiNjbXb3717d3zwwQe2pYI/dvHiRQDATTfdJLkWBheRQOTO4zp37tx11xAXFxdL/myTyQSLxYLw8HC7fWFhYQCA6upqh219fHwcLslrbm5Gfn4+vLy8cNddd0muhcFFJBh3zZyvra0F0Hqd+6esk9Pr6upk9fnSSy+hsrISv/nNbxAVFSW5HYNLAVqtVtH+8/LyFOtbydpfeeUVycd269YNarUaFRUVtjWy1/Pcc885X5hg5JwqhoeHyxpVXY81MB1dfLfWJOchomvXrkV+fj769+8v+/tjcBEJxl3XuAIDAwE4XnJmHWkFBQW1209DQwNWrFiB999/H5GRkdi8eTNUKpWsWhhcRIJxV3BZV8WYTCa7fTU1NQDg8PrXj12+fBmLFy/G4cOHER0djU2bNtndZZSCwUUkGHcFl0qlgkajQXl5ud0+67aYmJg221+5cgWPPPIIjh49inHjxuGVV15x+uU6nHhFJBCpc7iUmss1depUnDx5Evv377dtM5vN0Ov1iIqKsk2LcGTlypU4evQoJk2ahNdee61DbwTjiItIMO6cOZ+UlISioiKkpKRg/vz5CA0NhV6vx9mzZ5GTk2O7OF9aWgqj0Yi4uDiEhITgyy+/xHvvvQc/Pz+MHj3a4TPrbrvtNtx6662S6mBwEQnGncGlUqmg0+mQmZkJnU6HxsZGDBo0CLm5udesUy4sLMSOHTuwdetWhISE4JNPPgHQemF+5cqVDvtOTU1lcBF1Vu5+rI1arUZWVtZ1j8nIyEBGRobt/x999FE8+uijLquBwUUkGD66mcFFJBx3j7g8AYOLSDAMLgYXkVD4soxWDC4iwXTmQJKKwUUkGAYXg4tIOAwuBheRcBhcDC4i4TC4GFxEQrFYLJInoHbmgGNwEQmmMweSVAwuIsEwuBhcRMJhcDG4iITD4PLg4GpsbETPnj2xc+fOdo/18/MDAKxbtw4NDQ0KV9Y+Ry+9dKWmpibF+jYYDIr1/eO3H7fH+lb0O++8U9LFaEcvIRVBz5490djYKKsNg8uDg8vLywu+vr6IjIyU3Mb6MH/yTD169JDdxvpLSYm+PUFjY6OsV3oBDC7Ag4Nr6NCh7i6ByCMxuDw4uIjIHp8O0YrBRSQYPgGVwUUknM48kpKKwUUkGAYXg4tIOAwuBheRcBhcDC4iofCuYisGF5FgOnMgScXgIhIMg4vBRSQcBheDi0g4DC4GF5FwOHOewUUkHI64GFxEQuF0iFbe7i6AiOSxhld7f5Ry/vx5pKenY+zYsYiNjUViYiIOHjwoqa3RaERKSgpGjRqFoUOH4uGHH8axY8dk18DgIhKMO4PLbDYjKSkJu3btQkJCAtLS0lBXV4fk5GQcOHDgum1NJhPmzJmDw4cPY+7cuUhJScHp06cxd+5cHD9+XFYdPFUkEow7TwELCgpQWVmJ7Oxs26O4tVottFotVq1ahT179rT5RNdXX30VFy5cwLvvvotBgwYBAKZMmYL4+HisXbsWmzdvllyH8COujgxbyfM8/fTTGDhwoMM/R48edXd5HsGdI66ioiJERERc8/6AwMBAzJgxAwaDAWVlZQ7bNTc34/3338eIESNsoQUAYWFhiI+Px6FDh2AymSTXIfSIyzpsNRqNmDdvHtRqNfR6PZKTk5GTk4MxY8a4u0SSqaKiAv369cOiRYvs9vXt29cNFXked424amtrUVVVhalTp9rti46OBgCUlZU5fHHJiRMncOXKFcTExDhsu337dpSXl0OtVkuqRejg6siwlTxPc3MzTp06hfj4eNx///3uLscjqdVq7NixQ/Kx586dw9KlS9s8pri4WPJnm0wmWCwWhIeH2+0LCwsDAFRXV7fZFoBTbR0ROriuN2zNyspqM/3JM3311Ve4evUqBgwY4O5SPJL1jUf9+vWT3MZoNLrs82trawEA/v7+dvsCAgIAAHV1dbLbWre11dYRYYOrI8NW8kwVFRUAgP79+wMArly5gq5du9resXijGzhwoOw20dHRmD59uks+3zpj39FZjPX0ta0znOu1tZLzPQv7L6Ijw1byTNbg2rdvH8aNG4ehQ4di2LBhSEtLw8WLF91cHQUGBgIA6uvr7fZZR0tBQUFOt1WpVJJrEXrEBTg3bCXPZA2uL7/8EkuXLkVAQAAOHTqEwsJCfPHFF3j77bfb/MEg5VlfuOzo7l9NTQ0Ax9ewOtrWEWGDqyPDVvJM8fHxiI6OxoIFC2zXcyZNmoR+/frhxRdfxJYtW/D444+7ucobl0qlgkajQXl5ud0+6zZHdw2B1utygYGB1207ZMgQybUIe6rYkWEreaZp06bh8ccft4WW1axZs+Dj48P5eR5g6tSpOHnyJPbv32/bZjabodfrERUVZbu+/FN+fn649957cfDgQVRWVtq219TUYPfu3Rg7diyCg4Ml1yHsiMvVQ0/yXH5+fggKCoLZbHZ3KTe8pKQkFBUVISUlBfPnz0doaCj0ej3Onj2LnJwc21lOaWkpjEYj4uLiEBISAgBYsmQJSkpKMG/ePCQlJaFLly7Iz89Hc3Mz0tLSZNUh7IirI8NW8jxXrlzBtGnTkJKSYrfv4sWL+O677zgB1QOoVCrodDpMnDgROp0OmZmZ8Pf3R25uLkaPHm07rrCwEMuXL8epU6ds23r16oVt27ZhyJAhyM7OxoYNGxAREYGCggL5d0wtAlu/fr1lwIABlo8++si27fLly5Z77rnHMnnyZEtLS4sbqyO5fvvb31p++ctfWv73v/9ds/3ZZ5+1DBgwwLJ37143VUaexstiEfehPZcvX4ZWq8WFCxeuGbaeOHECOTk51/wGIM937NgxzJo1CwEBAZgzZw6Cg4NRUlKCAwcOQKvVYu3ate4ukTyE0MEFtF7jyszMxP79+9HY2IhBgwYhJSUFI0eOdHdp5IQvvvgCGzZswJEjR9DQ0ACNRoOZM2di9uzZnIhKNsIHFxHdePgrjIiEw+AiIuEwuIhIOAwuIhIOg4uIhMPgIiLhMLiISDgMLiISDoOLiITD4CIi4TC4iEg4DC4iEs7/AbK4TbOV5TlQAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 300x300 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(x_tr_NF[4].reshape((8,8)), interpolation='nearest', vmin=0, vmax=1, cmap='gray')\n",
    "plt.colorbar();\n",
    "plt.gcf().set_size_inches((3,3));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fit PCA with 10 components to this image dataset\n",
    "\n",
    "We'll fit this to the entire training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "K = 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "digit_pca = sklearn.decomposition.PCA(n_components=K)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-3 {color: black;}#sk-container-id-3 pre{padding: 0;}#sk-container-id-3 div.sk-toggleable {background-color: white;}#sk-container-id-3 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-3 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-3 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-3 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-3 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-3 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-3 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-3 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-3 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-3 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-3 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-3 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-3 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-3 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-3 div.sk-item {position: relative;z-index: 1;}#sk-container-id-3 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-3 div.sk-item::before, #sk-container-id-3 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-3 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-3 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-3 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-3 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-3 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-3 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-3 div.sk-label-container {text-align: center;}#sk-container-id-3 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-3 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-3\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>PCA(n_components=10)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" checked><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">PCA</label><div class=\"sk-toggleable__content\"><pre>PCA(n_components=10)</pre></div></div></div></div></div>"
      ],
      "text/plain": [
       "PCA(n_components=10)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "digit_pca.fit(x_tr_NF)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unpack the mean parameter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m_F = digit_pca.mean_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unpack the weights (K x F matrix)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "w_KF = digit_pca.components_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10, 64)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w_KF.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Show the mean image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAS4AAAEPCAYAAAD8sCnQAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAkl0lEQVR4nO3df1xUVf4/8BcKCDhoEjBiFoOUogX+yPIHuq5ameIqbGmYllJk6pZYLFbSrj1SC4PWHq4lSSwqjC7TDyW11VWWzTTXykVCW0ElHEMb8kcJA8iv+f7B5843Y5B7h7nNHHw9Hw8ePLr33MM7hBfn3nvOvW4Wi8UCIiKBdHF2AURESjG4iEg4DC4iEg6Di4iEw+AiIuEwuIhIOAwuIhIOg4uIhMPgIiLhMLiIyC5Hjx7FwIEDcfToUVntjUYjEhISMGrUKAwdOhRPPPEEjh07ZtfXZnARkWJGoxHPPvssmpubZbU3mUyYPXs2Dh8+jDlz5iAhIQFnzpzBnDlzcOLECcVfn8FFRIoUFBRgxowZqKyslH3M22+/jYsXL2Ljxo34wx/+gHnz5mHr1q3w8PDA6tWrFdfA4CIi2RITE7FgwQL4+fkhKipK1jFNTU3YsWMHRowYgbCwMOv2wMBAREVF4dChQzCZTIrqYHARkWynTp3Cs88+i+3btyMkJETWMSdPnkRNTQ0iIiJa7QsPD4fFYkFxcbGiOtwVtSYipykpKUF9fb2iY06cOIH09PQ29+fn5yvq7/3334enp6eiY6TRVFBQUKt9gYGBAICKigpFfXLERSSI+vp6RcFVX1+Pnj17OrQGpaEFAFVVVQAAb2/vVvukbbW1tYr65IiLSCCenp646667ZLU9duwYgoKCFI+qHE268+jm5tZmmy5dlI2hGFxEghHtocXdu3cHANTV1bXaJ420NBqNoj4ZXESCES24+vbtCwA27xxKUypsXf+6HgYXkWBEC66QkBB0797d5p1DadvgwYMV9cmL80SCsVgssj5chaenJx544AEcPHgQpaWl1u2VlZXYtWsXxo4dCz8/P0V9csRFJBAloWSxWK57QVwthYWFMBqNiIyMhL+/PwBg8eLFKCgowNy5cxEXF4du3bohOzsbTU1NSEpKUvw1OOIiEoyrj7hyc3OxdOlSnD592rqtT58+2LJlCwYPHoz09HSsW7cOwcHByMnJwYABAxR/DTe+V5FIDMXFxbBYLOjfv7+s9qWlpXBzc0N4eLjKlf36eKpIJBi5T2TozBhcRILhSRKDi0g4DC4GF5FwGFwMLiKhiDAd4tfA4CISDEdcDC4i4TC4GFxEwmFwMbiIhMPgYnARCYcTUBlcREJRelexs2JwEQmmMweSXAwuIsEwuFw4uAoLC2GxWODh4eHsUohU09DQADc3NwwdOlT2MQwuFw4ui8WChoYGnD17tt227u7u0Gq1MJlMaGxslNW/mjOKfXx8VOsbgMNfOfVrkV5TJUeXLl2g0WhQXV0t62J0TU1NR0prl1oXxLVareI/zgwuFw4uDw8PnD17FtHR0e22HTBgAPR6PRITE1FSUiKrf3d39f7Xp0+frlrfALBo0SJV+1dLVlaW7LYBAQGYPXs2Pv74Y/zwww/ttt+zZ09HSmvXxYsXVel327Ztst8ILWFwuXBwEZFtDC4GF5FwGFwMLiKhcB5XC7uC68KFC1izZg3279+PqqoqhIWF4dlnn0VkZKSj6yOiX+DMeTve8mM2mxEXF4edO3ciJiYGSUlJqK2tRXx8PA4cOKBGjUT0M67+lp9fg+IRV05ODkpLS5Geno7x48cDAKKjoxEdHY0VK1Zg9+7dnfbhZUSuoLOHkhyKR1x5eXkIDg62hhYAdO/eHTNnzkR5eTmKioocWiARXYsjLoXBVVVVhbKyMkRERLTaJ727jcFFpC4Gl8JTRZPJBIvFgqCgoFb7AgMDAQAVFRWOqYyIWuFdxRaKgktasuHt7d1qn7TMpba21gFltXB3d5f1em6dTnfNZ7l9q0Wr1arWN9CyHEZEAQEBstv26tXrms/tCQ0Ntasmufz9/VXp1561uJ05kORS9Nsr3Ya1dfFd+mY68sK8VquFXq+X3X7VqlUO+9rkeLNnz1Z8zJQpU1TrW1QMLoXB1b17dwBAXV1dq33SSMvX19cBZbUwmUxITExst51Op8OqVauQnJyM8vJyWX2rOeIaN26can0DwIwZM1TtXy0ff/yx7La9evXClClT8Mknn+Dy5cvttj906FBHSmvXTz/9pEq/b775Jvr27avoGAaXwuCSvsEmk6nVvsrKSgCwef3LXo2NjbIXTQNAeXm5SyyyDgsLU61vQNwJiHIWS//S5cuXZR13+vRpe0qSTa1F1g0NDYqPEfXf35EU/fZqNBrodDoUFxe32idts3XHkYgchyMuO+ZxTZ06FadOncL+/fut28xmMwwGA0JDQ63TIohIHZwOYcfM+bi4OOTl5SEhIQHz5s1DQEAADAYDzp07h4yMDM6aJ1KRK0yH6Mha5eLiYrz11lsoLCxEY2MjBg4ciIULF+K3v/2tohoUj7g0Gg30ej0mTpwIvV6P1NRUeHt7IzMzE6NHj1baHREp5MwRV0fWKn/zzTeYM2cOjh07hieffBKJiYmorq7GggUL8Mknnyiqw64r1FqtFmlpafYcSkQd5MzTwI6sVU5PT0ddXR2ys7Ot18JjYmLw4IMPIi0tTfbUF8COERcROZczR1wdWatcXl6OXr16XXMDr0ePHrj77rtRUVGBK1euyK6DwUUkGGcFV0fXKvfr1w8//fQTLly4cM12o9EIHx8faDQa2bXcsE9AVTrpT4mHHnpItb4Bdd9QpOZaUznLtyQ9evQA0DK52M/Pr932n376qd11yeFKd+mcVUtH1yonJCTgyJEjSEhIwIsvvghfX19kZ2fjxIkTeP755xUtZbthg4tIVEomoJ4/fx5Llixpc39+fr7svjq6Vlmn02HBggV47bXX8PDDD1u3P/bYY3j66adl1wEwuIiE4syRX0fXKi9fvhy5ubkYOnQoYmNj4enpid27dyM7OxtXr17FihUrZNfC4CISjNzwcnNzQ1BQkKJR1fV0ZK3yt99+C4PBgPDwcOTk5FiX3E2ZMgWvvPIKtm7diokTJ8qez8WL80SCcdbF+Y6sVS4pKYHFYkF0dHSrdcKxsbEAlC2U54iLSDDOOl3syFplT09PAEBTU1OrfdIpqK19beGIi0gwzpzHZe9a5eHDh8PHxwfvv/9+qwv40jP3xowZI7sOjriIBOPMC/Ry1yoXFhbCaDQiMjIS/v7+6NGjB5YtW4Y//elPiImJwcMPPwwvLy8UFBTgwIEDmDx5sqL1igwuIsE4M7iktcqpqanQ6/VoaGhAWFgYMjMzMXLkSGu73NxcbNu2DZs3b7Y+9nrGjBno06cPNmzYgLfffhuNjY3Q6XR46aWX8Pjjjyuqg8FFJBBXeDqEnLXKKSkpSElJabU9MjLSIW+8Z3ARCYZPQGVwEQnHlZYfOQuDi0gwDC4GF5FwGFwMLiLhMLgYXETCYXAxuIiE4grTIVwBg4tIMJ05kORicBEJhsHF4CISDoOLwUUkHM6cZ3ARCYcjLgYXkVB4V7EFg4tIMJ05kOS6YYPLy8tLtb5tvUzAkfLy8lTrW813Nip54Wd9fb3189WrV9ttr/Yvs1r929Mvg+sGDi4iUTG4GFxEwmFwMbiIhMPgYnARCYfBxeAiEorFYpE9AbUzB5xdwVVSUoJ169bhiy++gNlsRlBQEKKiorBo0SLrix+JSB2dOZDkUhxcRqMRs2bNgoeHBx599FEEBgbiP//5D9avX4+vv/4amZmZqt5SJ7rRMbjsCK7XXnsN9fX1+OCDD9CvXz8AwKxZs5CSkoKsrCzk5+fjvvvuc3ihRNSCwQV0UdK4qakJX3zxBYYPH24NLcn06dMBAEeOHHFcdUTUirTsp72PzkzRiKtLly7Iy8uz+U25dOkSAKBr166OqYyIbOrsoSSHouByc3PDrbfeanPfpk2bAAAjRozoeFX/x93dHQMGDGi3nU6nu+azHEraKuXt7a1a3wDg5+enWt9qXp9U8n2RlgfJXSb0yzMAR+vevbsq/dpzM4vBBbhZHPBdyMrKQkpKCu6++25s2bLFEXWhuLgYV69e5YV+6vQ8PT0RHh7ebrvi4mJUV1ejvLxcVr86nQ4ajUZW36Lp8DyuzZs3Y/Xq1QgICMCbb77piJqsTCYTEhMT222n0+mwatUqJCcny/5HDQkJ6WB1bZs7d65qfQNAaWmpan270ojrnnvuwZdffonq6up22+/YsaMjpbXLZDKp0u+aNWvQt29fRcdwxNWB4LJYLEhLS8N7772HgIAAbNy4EUFBQY6sDY2NjSgpKZHdvry8XHZ7NX9Ba2trVesb+P/XE9XgKk+HkFRXV+Onn35qt11ZWZk9Jcn23XffqdKv9BQMJfgEVDuDq76+Hi+99BJ27tyJ2267DZmZmbjtttscXRsR2cARlx3B1dTUhOeeew779u1DeHg4NmzYoOrFYiK6FoNL4TwuAPjrX/+Kffv2YdiwYdi0aRNDi+hX5ux5XBcuXEBycjLGjh2LIUOGIDY2FgcPHpR1rNlsRlpaGiZMmICIiAhMnjwZmZmZaGxsVFSDohHXDz/8YF3SM2HCBOzbt69Vm5CQEERERCgqgojkc+aIy2w2Iy4uDkajEXPnzoVWq4XBYEB8fDwyMjIwZsyYNo+tr69HXFwcjh07htjYWNxxxx349NNP8cYbb6CiogJ//vOfZdehKLi+/PJL68XEtLQ0m20eeeQRBheRSpz9soycnByUlpYiPT0d48ePBwBER0cjOjoaK1aswO7du9u8wbNx40YUFRVh5cqVmDFjBoCW5YILFy6EXq/H008/Da1WK6sORcE1ZcoUTJkyRckhRORgzhxx5eXlITg42BpaQMvk3JkzZyItLQ1FRUUYMmSIzWMNBgP69+9vDS3JwoULMXDgQEXvalB8jYuInMtZ17iqqqpQVlZm84xKmuRaVFRk89jz58/j7NmzGDt2rHWb2WxGc3MzIiIisHjxYgQHB8uuhcFFJBhnBZfJZILFYrE5XzMwMBAAUFFRYfNYaZ7dLbfcgqysLIwbNw7Dhg3D8OHDsXLlSllvcvq5G/YJqGoOt6uqqlTrG0Cb60UdQc27xEpm/Dc1NVk/y7njpObr5lyNkgmo58+fx5IlS9rcn5+fL7sv6efa1goIHx8fAG1PvpYmEWdnZ+PKlSuYP38+goKCsG/fPmRnZ+Ps2bN49913ZddywwYXkaicdY1LCkxbF9+lmtq6MN/Q0ACgZQVCXl4eQkNDAQCTJk2Ch4cHPvzwQxw4cOC6dyV/jsFFJBCldxWDgoIUjaquR3pChq2L6NJIy9fX1+ax0ohs5MiR1tCSzJw5Ex9++CE+//xzBhdRZ+WsEZe0GNzWgvPKykoAaHO9sjTNwd/fv9W+m2++GQBkLaaXMLiIBOOs4NJoNNDpdCguLm61T9rW1hzO/v37w8vLy+Z1TqPRCACKnpLBu4pEgnHmkp+pU6fi1KlT2L9/v3Wb2WyGwWBAaGhom8/+8vLywoMPPojjx4+joKDAur25uRl/+9vf0LVrV0yaNEl2HRxxEQnGmRNQ4+LikJeXh4SEBMybNw8BAQEwGAw4d+4cMjIyrBfnCwsLYTQaERkZaT09/OMf/4gvv/wSCQkJePTRR3Hrrbdiz549OHz4MBYtWqRoHheDi0gwzgwujUYDvV6P1NRU6PV6NDQ0ICwsDJmZmRg5cqS1XW5uLrZt24bNmzdbg0sKubVr12Lnzp24cuWK9SGgDz/8sKI6GFxEgnH2Y220Wm2ba5UlKSkpSElJabXd398fr776Kl599dUO1cDgIhKMs4PLFTC4iARisVhkz5zvzAHH4CISTGcOJLkYXESCYXAxuIiEw+BicBEJh8HF4CISDoOLwUUkFGc/c95VMLiIBNOZA0kuBheRYBhcDC4i4Sh5dHNnxeAiEgxHXAwuIuEwuBhcRMJhcN3AwSW9/koN33//vWp9A0Dv3r1V69vDw0O1vqVni8uh0WgAAD179kTXrl3bbS+9jKGz43SIFjdscBGJqjMHklwMLiLBMLgYXETCYXAxuIiEw+BicBEJhxNQO/hexbq6OkyePBkTJkxwVD1E1A5nvlfRVXQouN544w2UlZU5qhYiaofc0Ors4WX3qeJnn32GLVu2qDrvh4ha68yBJJddI64ff/wRy5YtQ0xMDPr27evomojoOm700RZgZ3D9+c9/hoeHB5KTkx1dDxG1g8Flx6ni9u3b8c9//hObNm2yLssgol9PZw8lORQFV0VFBVauXIm5c+dixIgRatVk5e7ujgEDBrTbTqfTXfNZjltvvdXOqtp30003qdY3AHTv3l21vr28vFTrW8kfOmntodw1iLfddptdNcl19epVVfr19PRUfAyDS0FwNTc348UXX0RgYCCef/55NWuy0mq10Ov1stuvWrVKxWqoo0JCQhQfc+edd8pqd8899yjuW1QMLgXBlZWVha+++gobNmyA2WyG2WwG0BJozc3NuHTpEjw8PODr6+uw4kwmExITE9ttp9PpsGrVKiQnJ6O8vFxW32qOuO6//37V+gaUPWVBqZ49e6rW94ULF2S39fHxwZ133onjx4+jpqam3fa7du3qSGntOnnypCr9rlmzRtENLovFInsCamcOONnB9e9//xvNzc2Ij4+3uX/UqFG49957kZ2d7bDiGhsbUVJSIrt9eXm57PZqPtbmxx9/VK1vQN3TuW7duqnWd3V1teJjampqZB1nNBrtKUk2JT+HStTX1ys+pjMHklyyg+uFF17AlStXWm1ftmwZGhoakJqaih49eji0OCJqjcGlILjuuusum9u9vLzQpUsXjB492mFFEVHbGFwdXPJDRL8+Z8/junDhApKTkzF27FgMGTIEsbGxOHjwoOJ+Ll68iFGjRuGxxx5TfCyDi0gwzgwus9mMuLg47Ny5EzExMUhKSkJtbS3i4+Nx4MABRX0lJyfj0qVLdtXR4cfa7N69u6NdEJECzjxVzMnJQWlpKdLT0zF+/HgAQHR0NKKjo7FixQrs3r0bbm5u7faTm5uLzz77DF262Dd24oiLSCDOfjpEXl4egoODraEFtEyInjlzJsrLy1FUVNRuH2fOnEFKSgoWLlwId3f7xk4MLiLBOCu0qqqqUFZWhoiIiFb7wsPDAaDd4GpqasLSpUsRGhqKBQsW2F0Ln4BKJBgloXT+/HksWbKkzf35+fmy+zKZTLBYLAgKCmq1LzAwEEDLssDrWb9+PU6cOIFt27bZPdoCbuDgsmcypCv0Dag72VLNd0IqefehtIavvr4edXV17bb39va2uy7ROOvRzVVVVQBsf6+lf9va2to2jy8uLsb69evxwgsvoF+/fh2q5YYNLiJRKRlxBQUFKRpVXY8UmLYuvks1tXVhvq6uDklJSbjnnnvsmv7wSwwuIsE4666i9FQSWyNgaaTV1lrlN954A99//z3+8pe/4PLly9fsa2howKVLl9CtWzfZTz5hcBEJRMmFd0cHnLQY3GQytdpXWVkJADavfwEta51ra2sRExPTal9hYSFGjRqFmJgYpKSkyKqFwUUkGGeNuDQaDXQ6HYqLi1vtk7bZuuMIAKmpqTafaTZ//nz069fP+sgsuRhcRIJx5gTUqVOnYt26ddi/fz9+85vfAGiZTW8wGBAaGmqdFvFLd999t83tbm5u6Nmzp+K1zgwuIsE4M7ji4uKQl5eHhIQEzJs3DwEBATAYDDh37hwyMjKsF+cLCwthNBoRGRkJf39/h9fBCahEgnHmWkWNRgO9Xo+JEydCr9cjNTUV3t7eyMzMvGbUlJubi6VLl+L06dOq1MERF5FgnP1YG61Wi7S0tOu2SUlJkXWh3db1MjkYXESCcdYEVFfC4CISiDOnQ7gSBheRYDpzIMnF4CISDIOLwUUkHAYXg4tIOAwuBheRcBhcDC4i4TC4GFxEQuF0iBYMLiLBcAIqg4tIOJ15JCUXg4tIMAwuBheRcBhcDC4i4TC4buDgkl61pAa5D/y3l06nU61vNS/81tTUyG7brVs362c5rx7ryDv65Ojatasq/cp5Xf0vMbhu4OAiEhGnQ7RgcBEJpjMHklwMLiLBMLgYXETC4QRUBheRcDjisjO4GhoakJWVhY8++ggVFRXo3bs3Jk+ejAULFsDHx8fRNRLRzzC47Ayu5557Dnv37sW0adMwb948FBYW4t1330VpaSnWr19v1y1eImof7yq2UBxcu3btwt69e7FgwQI899xzAIDY2Fj4+voiOzsbhYWFGDZsmMMLJaIWnTmQ5FIcXLm5uejVqxcWLVp0zfbHHnsMGo0Gnp6eDiuOiFpjcCkMrsbGRhQWFmLChAnWmc21tbXw9PREcHAwlixZokaNRPQzDC6FwfXdd9+hvr4effv2xfbt27F+/XqUl5fD09MTDz74IF5++WX07NnTccW5u2PAgAHttpOWwChZCiNnGYm9brrpJtX6BqDqDRA1b7V36dJFdlvp/1Hu/2vfvn3tqkkuOT+H9vDw8FB8DINLYXBduXIFAPCvf/0LW7Zswfz583H77bfj8OHD0Ov1OH36NP7+97877HRRq9VCr9fLbr9q1SqHfF1yHYMGDZLVbvjw4SpX4joYXAqDq76+HgBQVlaGjRs3YtSoUQCA+++/H7169cLatWuxbds2PPLIIw4pzmQyITExsd12Op0Oq1atQnJyMsrLy2X1reaIKzY2VrW+AaBPnz6q9a3miKuurk52Wx8fHwwaNAjffPONrMXZBQUFHSmtXcXFxar0m5aWpni0yAmoCoNLGrb369fPGlqSmTNnYu3atfj8888dFlyNjY0oKSmR3b68vFx2ezWf4PDjjz+q1jeg7qmoqzwd4ufHVFdXt9vuu+++s6ck2ZT8HCrR0NCgqD2nQ7RQFFy9e/cGAPj7+7fa5+fnBzc3N1k/ZERkv84cSHIpCi4/Pz/ccsstOH36NJqbm6+52Hr27FlYLBbVL5IS3egYXID82zz/Jzo6GhcvXsTWrVuv2f7ee+8BACZPnuyYyojIJul0sb0PtVy4cAHJyckYO3YshgwZgtjYWBw8eFDWsV999RWefPJJDB8+HHfddReioqKwceNGxZcoFE9Afeqpp1BQUICVK1eipKQEgwYNwoEDB7B3715Mnz4dI0eOVNolESngzBGX2WxGXFwcjEYj5s6dC61WC4PBgPj4eGRkZGDMmDFtHvvf//4Xjz/+OLRaLeLj4+Hr64v8/Hy8/vrrOHnypKJZAYqDy9vbG9nZ2XjnnXfwj3/8Ax999BFuueUWJCUl4YknnlDaHREp5MzgysnJQWlpKdLT0zF+/HgALWdh0dHRWLFiBXbv3t3mWuXly5fD19cXH374Ifz8/AAAs2fPxuLFi/HBBx9gzpw5GDhwoKw6FJ8qAoBGo8HSpUtRUFCAY8eOYc+ePYiPj1c0wZCI7OPMU8W8vDwEBwdbQwtouUM/c+ZMlJeXo6ioyOZxFy9eRGlpKe677z5raEmmTZsGADhy5IjsOvg8LiLBOGvEVVVVhbKyMkydOrXVvvDwcABAUVERhgwZ0mp/z549sWfPHutSwZ+7dOkSAGUvJGFwEQlE6Tyu8+fPX3cNcX5+vuyvbTKZYLFYEBQU1GpfYGAgAKCiosLmse7u7jaX5DU1NSE7Oxtubm649957ZdfC4CISjLNmzkuv9LO16kSanF5bW6uoz9dffx2lpaX43e9+h9DQUNnH3bDBdfXqVdX6PnPmjGp9A+pOOVE6k1uJ7du3y24rXQc5d+6c9VTietr6S+8ornT9VsmpYlBQkKJR1fVIgWnr4rtUk5KHiK5evRrZ2dm4/fbb8corryiq5YYNLiJROesal7RMztaaU2mk5evr224/9fX1WLZsGXbs2IF+/fohKysLGo1GUS0MLiLBOCu4pFUxJpOp1b7KykoAsHn96+eqq6uxaNEiHD58GOHh4diwYUOru4xyMLiIBOOs4NJoNNDpdDaflCFti4iIaPP4mpoaPPnkkzh69CjGjRuHt956y+5ny7nOiTsRtUvuHC615nJNnToVp06dwv79+63bzGYzDAYDQkNDrdMibFm+fDmOHj2KSZMm4Z133unQAzE54iISjDNnzsfFxSEvLw8JCQmYN28eAgICYDAYcO7cOWRkZFgvzhcWFsJoNCIyMhL+/v44fvw4Pv74Y3h6emL06NHYtWtXq74HDRqEO+64Q1YdDC4iwTgzuDQaDfR6PVJTU6HX69HQ0ICwsDBkZmZes045NzcX27Ztw+bNm+Hv74/PP/8cQMuF+eXLl9vsOzExkcFF1Fk5+7E2Wq0WaWlp122TkpKClJQU638/9dRTeOqppxxWA4OLSDB8dDODi0g4zh5xuQIGF5FgGFwMLiKh8GUZLRhcRILpzIEkF4OLSDAMLgYXkXAYXAwuIuEwuBhcRMJhcDG4iIRisVhkT0DtzAHH4CISTGcOJLkYXESCYXAxuIiEw+BicBEJh8HlwsHV0NCA3r17y3ozjKenJwBgzZo1qK+vl9W/kreRKCW9VEAttl4P5SheXl6q9f3QQw/Jbiu9VWfixImyLkar+eYjAGhsbFSlX61Wq/jNSgwuFw4uNzc3eHh4oF+/frKPkR7mT67JnsCV/ii15+abb1bctytoaGhQ/EeUweXCwTV06FBnl0DkkhhcLhxcRNQanw7RgsFFJBg+AZXBRSSczjySkovBRSQYBheDi0g4DC4GF5FwGFwMLiKh8K5iCwYXkWA6cyDJxeAiEgyDi8FFJBwGF4OLSDgMLgYXkXA4c57BRSQcjrgYXERC4XSIFl2cXQARKSOFV3sfarlw4QKSk5MxduxYDBkyBLGxsTh48KCsY41GIxISEjBq1CgMHToUTzzxBI4dO6a4BgYXkWCcGVxmsxlxcXHYuXMnYmJikJSUhNraWsTHx+PAgQPXPdZkMmH27Nk4fPgw5syZg4SEBJw5cwZz5szBiRMnFNXBU0UiwTjzFDAnJwelpaVIT0/H+PHjAQDR0dGIjo7GihUrsHv37jaf6Pr222/j4sWL+OijjxAWFgYAmDJlCqKiorB69WpkZWXJrkP4EVdHhq3kel588UUMGDDA5sfRo0edXZ5LcOaIKy8vD8HBwdbQAlresTBz5kyUl5ejqKjI5nFNTU3YsWMHRowYYQ0tAAgMDERUVBQOHToEk8kkuw6hR1zSsNVoNGLu3LnQarUwGAyIj49HRkYGxowZ4+wSSaGSkhKEhIRg4cKFrfbddtttTqjI9ThrxFVVVYWysjJMnTq11b7w8HAAQFFREYYMGdJq/8mTJ1FTU4OIiAibx27duhXFxcXQarWyahE6uDoybCXX09TUhNOnTyMqKgrTp093djkuSavVYtu2bbLbnj9/HkuWLGmzTX5+vuyvbTKZYLFYEBQU1GpfYGAgAKCioqLNYwHYdawtQgfX9YataWlpbaY/uaZvv/0WV69eRf/+/Z1dikuS3ngUEhIi+xij0eiwr19VVQXA9tuafHx8AAC1tbWKj5W2tXWsLcIGV0eGreSaSkpKAAC33347AKCmpgZeXl7Wdyze6AYMGKD4mPDwcMyYMcMhX1+asW/rLEY6fW3rDOd6x0qU/DsL+xPRkWEruSYpuPbt24dx48Zh6NChGDZsGJKSknDp0iUnV0fSi47r6upa7ZNGS76+vnYfq9FoZNci9IgLsG/YSq5JCq7jx49jyZIl8PHxwaFDh5Cbm4uvv/4aH3zwQZu/GKQ+6YXLtu7+VVZWArB9Daujx9oibHB1ZNhKrikqKgrh4eGYP3++9XrOpEmTEBISgtdeew2bNm3CM8884+Qqb1wajQY6nQ7FxcWt9knbbN01BFquy3Xv3v26xw4ePFh2LcKeKnZk2Equadq0aXjmmWesoSWZNWsW3N3dOT/PBUydOhWnTp3C/v37rdvMZjMMBgNCQ0Ot15d/ydPTEw888AAOHjyI0tJS6/bKykrs2rULY8eOhZ+fn+w6hB1xOXroSa7L09MTvr6+MJvNzi7lhhcXF4e8vDwkJCRg3rx5CAgIgMFgwLlz55CRkWE9yyksLITRaERkZCT8/f0BAIsXL0ZBQQHmzp2LuLg4dOvWDdnZ2WhqakJSUpKiOoQdcXVk2Equp6amBtOmTUNCQkKrfZcuXcLly5c5AdUFaDQa6PV6TJw4EXq9HqmpqfD29kZmZiZGjx5tbZebm4ulS5fi9OnT1m19+vTBli1bMHjwYKSnp2PdunUIDg5GTk6O8jumFoGtXbvW0r9/f8unn35q3VZdXW257777LJMnT7Y0Nzc7sTpS6ve//73lzjvvtPzvf/+7ZvvLL79s6d+/v2Xv3r1OqoxcjZvFIu5De6qrqxEdHY2LFy9eM2w9efIkMjIyrvkLQK7v2LFjmDVrFnx8fDB79mz4+fmhoKAABw4cQHR0NFavXu3sEslFCB1cQMs1rtTUVOzfvx8NDQ0ICwtDQkICRo4c6ezSyA5ff/011q1bhyNHjqC+vh46nQ6PPPIIHn30UU5EJSvhg4uIbjz8E0ZEwmFwEZFwGFxEJBwGFxEJh8FFRMJhcBGRcBhcRCQcBhcRCYfBRUTCYXARkXAYXEQkHAYXEQnn/wHQ/nA/O91RiQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 300x300 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(m_F.reshape((8,8)), interpolation='nearest', vmin=0, vmax=1, cmap='gray')\n",
    "plt.colorbar();\n",
    "plt.gcf().set_size_inches((3,3));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Show the weight vectors for the first 10 components (each one reshaped as an image)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmoAAAEJCAYAAAAkWgsWAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABB/0lEQVR4nO3dd3hUVfoH8O9NJY1epROcgKEEpNd1wSAGJPhzpbeIERUW0I0gLOiKCkQ0IChRk6VKUQQpBkEBUVEQESlSBULoIUTSQ6ac3x/ZGZjUc8IdZ4Lfz/Pw6Nx559wz885M3rnt1YQQAkRERETkctycPQEiIiIiKhoLNSIiIiIXxUKNiIiIyEWxUCMiIiJyUSzUiIiIiFwUCzUiIiIiF8VCjYiIiMhFsVAjIiIiclEs1IiIiIhcVJkLtXPnziE6Ohr9+/dHu3bt0KpVK/Tt2xdvvvkmrl27puccnWrfvn0ICgrCv/71rz9lfZMnT0aPHj0cvh7mTx9ZWVl455138Mgjj6Bly5Zo27Ythg8fjq+//toh67Ni/vSRnp6O6Oho9OnTB61atcJDDz2E119/HampqQ5Z352YQ8fYu3cvmjVr5vD1MX/6iImJQVBQUJH/2rRp45B1ljceZXlQfHw8YmJiYLFY0LlzZ3Tq1AkWiwW//vorli1bhs8++wxxcXF8kRXFxsYiISEBtWrVcuh6mD99ZGZmYujQoTh58iSCg4MxdOhQZGRkYPv27Xj++efxwgsv4JlnntF9vcyfPjIzMzFs2DCcOnUKnTp1Qq9evXD27FmsWLEC27dvx6effuqwzyJz6BiZmZmYNm0aHN0ZkfnTz/Hjx6FpGp577jlommZ3n6enp5Nm5WKEoiVLlgiDwSD69esnzp07V+j+9evXi2bNmom2bduKCxcuqA7vcvbu3SsMBoN48cUXHbaO3NxcMWPGDGEwGITBYBDdu3d32LqYP/3ExMQIg8EgZs6cKSwWi2351atXRdeuXUXz5s1FYmKirutk/vQzd+5cYTAYxMKFC+2Wr1ixQhgMBvHyyy/rvk4hmENHmjp1qu171FHrY/701a1bN9G7d2+HjH2vUNr1eeHCBcybNw8VK1bEsmXL0KhRo0IxAwcOxMiRI5GZmYn4+Hi96sl71s6dO9G3b1+sXbsWPXv2dOi6mD99bd26FZqm4cUXX7T7JVirVi0MGTIEZrMZu3fv1m19zJ++Ll68iOrVq+Opp56yWz5gwAAAwC+//KL7OplDx9m5cyfWr1+Pv//97w5bB/Onr9TUVCQnJ6N58+bOnopLU9r1+fnnn8NoNGLs2LGoWrVqsXERERGoVq0aOnXqZLc8NzcXcXFxSEhIwIULF+Dt7Y0WLVpgzJgxhYqUoKAg9OrVCxMmTMC8efNw8OBBuLu7o3Pnzvj3v/+NihUrYuHChdi8eTMyMjIQGBiISZMmoVu3brYxpk6dig0bNuDrr7/GsmXLkJCQgJycHBgMBkRERKBPnz4qT99OTEwMYmNjERISgvj4ePj7+yMoKKjUx9WtWxc7d+603V63bh2ysrLwyiuvYMiQIWjWrFmZ51Qa5u82PfI3cuRIZGRkoGLFioXivLy8AOQfw6YX5u82PfL37rvvFhlz5swZAECNGjXKPL/iMIe36fUdCuT/wZ8xYwbatWuHkSNHFrpfL8zfbXrk7/jx47bnSsVTKtS++eYbACh1y0+tWrUQGRlptywjIwMjRozA8ePHYTAYMGjQIKSlpWHnzp2IjIzE5MmTMW7cOLvHnD17FkOGDEHr1q0xePBg/PTTT9i2bRsuX74MPz8/XLhwAaGhoUhLS8OWLVswbtw4JCQkoEGDBnbjTJo0CefPn0e/fv1gMpmwfft2/POf/0RUVBTGjh2r8hIAABYvXozY2Fi0bdsWH330Efz9/QEA48ePL/WxAQEBdrdHjRqF6Oho2xiOxPzl0yt/w4YNKzJGCIGvvvoKgL5fQMxfPj0/f3dKS0vD3r17MWfOHHh4eOC5555TnltpmMN8eufw1VdfRXZ2NmbPno0rV64oz0cW85dPr/xZC7WsrCw888wzOHz4MHJzcxEcHIxnnnkG3bt3V57bPUllP2mXLl2EwWAQN2/eVN7HOnPmTGEwGMSMGTOE0Wi0LU9KShI9evQQQUFB4tdff7Uttx5nMHv2bNuyvLw80aNHD2EwGERoaKjIyMiw3ffOO+8Ig8EgYmNjbcumTJkiDAaDaN++vTh//rzdOrt16yaCg4NFUlJSifMuuH8+Pj5eGAwGMWzYMJGZman8OpTGkceoMX+Oz58QQqxcuVIYDAbRq1cvkZeXp9u4zJ/j8rdq1Srbc27evLnYsmWLbmPfiTnUP4cbN24UBoNBrFy5ssj16Yn50zd/L7zwgjAYDKJZs2YiMjJSREdHiwkTJojg4GARFBRky+lfndIxamlpaQAAX19fpWIwLy8PmzZtQsWKFTF9+nR4eNzekFe/fn1MmjQJQgisXbu20GOffvpp2/97enqidevWAPK3Zty5Fapt27YAgEuXLhUaIyIiwu4XRv369REZGQmj0YgtW7ZIP4+VK1di7ty56NChAz788EP4+flJP9YVMH+Oz19CQgLeeOMNeHh4YM6cObqetcT8OS5/VatWxdNPP43w8HBUqFABUVFRWLJkiW7jWzGH+ubw2rVreP3119GxY0cMHTr0rsaSwfzpmz8PDw/UrVsXH330ET744ANERUXh3Xffxdq1a1GhQgW8+eabSEpKuqt13AuUdn1WqVIFycnJSE9PR7Vq1aQfl5iYiOzsbHTr1g3e3t6F7m/Xrh2A25tBrQICAgqtx/oBKbhpt0KFCgDyPxAFde7cudAy65v62LFjUs9h3759tjd0586di/ygLly4sNRxAgICMHr0aKl16o35c2z+Vq1ahVmzZkHTNMydO9f2uuiF+XNc/vr06WM7XmfixIl48sknMWfOHHTo0AHBwcFSc5TBHOqbw2nTpsFkMuHNN98sdGkHR2D+9M3f3Llzi4wJDg7GqFGjbJesKrhL+K9GqVBr0KABkpOTkZiYWOqb9MyZM2jcuDHc3NyQkZEBoPjjQ6zXKsrOzrZbXtKvFuvB2jJq165daJn1l0h6errUGMnJyWjXrh3OnDmDxYsXIzQ0FE2bNrWLWbRoUanj1K1b12mFGvPnmPxZLBZER0djyZIl8Pb2xttvv42HH35Yal4qmL8/5/N33333ITIyEm+88Qa+/vprXQs15lC/HK5evRrff/89XnvtNdSrV09qDneL+fvz/ga2bNkSQP6Ztn91SoVaz5498fPPP+O7777Dgw8+WGzc5cuXERYWhipVqmDnzp22N0RxV2u2vlGqVKmiMh1pubm5hZZZN2GXdObOnR544AHExcVh27ZtmDJlCqZNm4Y1a9bAze323uOTJ0/qM2EHYf70z19eXh5efPFFbN++HZUrV8Z7772n+5Y0K+ZPv/zl5eVh//79MJvNRXYCqV+/PgDgxo0b0mPKYA71y2FCQgIAYObMmZg5c2ah+zdv3ozNmzdj4MCBmDNnjvS4JWH+9P0MnjhxAhaLBSEhIYXuz8nJAXB7S+FfmdIxav3794ePjw8+/vjjElusLFmyBEIItGvXDj4+PmjSpAl8fHxw6tSpIqv3ffv2AQAMBoPi9OUcOnSo0DLrNZKs+/tLExgYCB8fH4SHh6Nbt244dOgQli5dquc0HY750zd/FosFEydOxPbt21GvXj2sXr3aYUUawPzpmT+z2Ww7066oXUW//fYbAKBhw4ZlXkdRmEP9cjhw4ECMHz++0L+BAwcCyH8txo8fj969e5d5HQUxf/rlLysrC08++SSeeuopGI3GQvfv378fwO0ta39lSoVanTp18OyzzyI9PR2jRo0qdJCfxWLB0qVLsXz5cvj6+mLy5MkA8g+AfOyxx5CZmYnZs2fDZDLZHnPx4kXExMRA0zSEh4ff/TMqwvvvv4+UlBTb7fPnzyMuLg5+fn4ICwtTHu/VV1+Fj48PFixYgPPnz+s5VYdi/vLplb8PPvgAO3fuxH333YdVq1ahSZMmZR5LBvOXT4/8+fj44OGHH0ZmZibmz59vd9+xY8cQHx8PX19f9OvXr0zjF4c5zKdHDh9//HFMmDCh0D9roRYUFIQJEyboWqgxf/n0yF+VKlXQpUsXZGZmFtpl+sMPP2DdunWoXbs2HnnkkTKNfy9R7vUZGRmJtLQ0xMfH45FHHkHXrl3RtGlTZGRk4MCBAzh79iz8/f2xYMECuz9cUVFROHjwINavX4+jR4+iY8eOSE9Px86dO5GRkYGJEyfaDm7UW0pKCgYMGIBevXrBaDRi+/btyMnJwezZs8t0Ucv69etjwoQJiI6OxvTp07FixYo/5UBWPTB/+uQvLS0NH374IQCgefPm+OSTT4qMa9euXZEH8pYV86ff5+/ll1/G4cOHER8fj4MHDyIkJASXL1/Gjh07AADvvPOOQ3p9Mof8DmX+8s2cORNDhw5FbGwsfv75Z7Rq1Qrnz5/Hrl27UKFCBcTExHDXJ8pQqGmahpdeegmhoaFYvXo1Dh8+jAMHDsBkMqFevXqIiIjA6NGjC33BBQQEYM2aNYiPj8fWrVuxZs0a+Pn5oU2bNhg9ejS6du2q25MqaNasWdizZw+2bdsGs9mMkJAQjBs3Du3bty/zmKNHj8YXX3yB/fv3Y9WqVcVe/NTVMH/57jZ/R44csR34u2PHDtsf94LGjRuna6HG/OXT4/NXq1YtrFu3Du+//z527NiB5cuXo2LFiujVqxfGjRvnsLY2zGE+focyf40aNcKGDRvw3nvvYffu3fj1119RuXJl9OvXD88//3yRLbr+ijQhhHD2JBzF2j5jyZIl6NKli7OnQ4qYv/KN+Sv/mMPyjfm7Nygdo0ZEREREfx4WakREREQuioUaERERkYu6p49RIyIiIirPuEWNiIiIyEWxUCMiIiJyUdLXUTt48CCEEPD09HTkfKgAo9EITdPQpk2buxqH+XMOvfIHMIfOwPyVb8xf+adnDssr6UJNCGH7J/0YhYmk5ZpKD7pDBQ+1jYFe7mrxZoXn6enmuCtq63UIYVnyp8KiOKzqS6Y6vlnhAW6Kk3FXCNfz9S5bDuVjNWPhxs16ytG8leIzb8l/J1SsoPbH01MyiUIIpe+xUsdSzZ9KbG6m2oQsFrV43wClcM2i8J2uqX0/Cze5P13l6vNnVvsbaHJTe8+rfG8BAHIypEM1L7XPtnD3ko/lYfTyhZqnpyeEELj//vulB88zy7/A0d+p9Qv7W2A1pfhWtfyU4q9lyX9omlSWf9MBgIfCB+bU6dO6tFax5s+gkD+TwucjM0/tSz/AS+2L+eYts1L81czCTX6LU8VXrUFHDR/5+DO/65M/oGw51EzyxZfbqR/KMi1p3/irXQV92W9JpQf9z/ju9ZXGvr+6XFuaxLO/K41bkjLlLy9bOtb8zUql+Yhs+T/EAODxyNNq8X/I5094+SiNbapSTyru1JlzSuOWpCz5gzlPfvyUs0rzSfJvrBRfQ/F7zu2rD6VjvVt0UhrbWCdQOvbU72fKTXsxR+ExakREREQuioUaERERkYtioUZERETkolioEREREbkoFmpERERELoqFGhEREZGLYqFGRERE5KJYqBERERG5KBZqRERERC6KhRoRERGRi1LrKaHocLJ8+5OcPLU+Zz0ub1eKT64erhS/+fg16dgJndTa1wCKPfZ0pNI1TaVfZvWMRKV5ZFdtohSfpthCqmEl+bZe2Ua1XnIObO2qO/dU+TY+WtXaSmNbvNTasnWvqtYr8t1d8m3AktJylMZ+oIZayyJncU+/Kh17+hO170TDux8pxX9xSe17K+w+he9FhVZLAADJXp/O5p55XTrW4lNJaeyaii2hdiWmKcUfrxwmHTuxir/S2BDO+xtYHnGLGhEREZGLYqFGRERE5KJYqBERERG5KBZqRERERC6KhRoRERGRi2KhRkREROSiWKgRERERuSgWakREREQuioUaERERkYtioUZERETkohzah6Omn3wbn7dayLebAoCMXceV4r/07a4Uf391+fY43nnpSmOLChWV4p3FJzdVOvbWD5uUxk7tPV4p/tMj8q10AGBKZ/l2SAFuuUpjW1A+8gcAFr9q8rGKLWw8biQqxXtePqwUn5El31bI19NdaWx3yTZgzu4WJlLkW4DVeyhEaezhmy4qxbsr9k7rULe5dGx1T7UWYGpN35zHLU2+FeHNrz5XGjugXRe1yVTsrBS+/IsT0rFjH/y70tg+WnnJoGvgFjUiIiIiF8VCjYiIiMhFsVAjIiIiclEs1IiIiIhcFAs1IiIiIhfFQo2IiIjIRbFQIyIiInJRLNSIiIiIXBQLNSIiIiIXxUKNiIiIyEWxUCMiIiJyUcq9PlW6vQUaL0nHXoxbrDQPz5cWKcVf+Emtr13/jvWlY91uyfdzAwBzOen16Zb9h3ysb4DS2Meuq/V2vXpTrRege7p8TjSLWWns8tKrFQBS3CtLx+Zkq/Xfa6ip/c7Lqt1KKT79j91K8SoU21Y6jVul6tKxAd0fVRr7b6k1leLTc41q8bcs0rE1kaY0tsXbXyneWUSefB/hi7uPKI3tdeCkUvzDMd2U4tOGhEjHJt6U78sLAM2reSvF/9VxixoRERGRi2KhRkREROSiWKgRERERuSgWakREREQuioUaERERkYtioUZERETkolioEREREbkoFmpERERELoqFGhEREZGLYqFGRERE5KJYqBERERG5KOVenypMx/ZIx57edFRp7O59PleKnxn8oFK8xSTf59Lirdbn0plUWhxqFpN0rGfIQ0rz+PDLc0rxeSb5voEAYD78k3Sse7XaSmOjWiO1eCcyK7TvrOSt9rtNJJ5Riveq3kQpvmMr+bw0qeqjNHY5afUJi1816Vi3rBtKY0e0rqEUv/tCplL8L1fSpWMbB9VRGru85O9Gg07SsT7vfaI0doMAT6V49+RTSvFPNKknHXssQ2loUsQtakREREQuioUaERERkYtioUZERETkolioEREREbkoFmpERERELoqFGhEREZGLYqFGRERE5KJYqBERERG5KBZqRERERC6KhRoRERGRi2KhRkREROSiHNrr07Nhc+nYOg+q9VtMP3hAKd73SqJSvMejz0rHZrmp9Rn0VYp2HrNvFelYt2tqfR8BP6XonFvyfUcB4OahI9Kx3pXV+o76tHhYKV5vCu074e8l/1vM//RupXmYb1xViv/9Zp5SfHgr+f6PhrwkpbFNaKoUryeV/Jn9FfpxmtReX/ebl5Ti298n3/sRAFYduSYdezZd7fPduJKXUryeVPKXnCX/vAb852uleTz5mPzfVwB4o6lar1b3RPm/sfc16qo0tsprSNyiRkREROSyWKgRERERuSgWakREREQuioUaERERkYtioUZERETkolioEREREbkoFmpERERELoqFGhEREZGLYqFGRERE5KJYqBERERG5KIe2kMqp31Y6tuL8NUpjV715XG0yRrX2Kt+lyDe5aFLZojS2r4e7UryzWPyqSceK698qjT3p748qxVfz8VSKT3t1vnRs45GDlcY2K0U7l685Rzr2xvYEpbErt26pFF/JW+1939xTvi3UzU2fKI3tP2KqXKBwcrMbIf/doqWotdGyZKWrxbeoqxR/JS1XOjbxpnwsANT1l/s+cHaronoV5b+3+vQOVBq7cXW1ZoSW7PNK8aJesHSsj4faNh9N4X1N3KJGRERE5LJYqBERERG5KBZqRERERC6KhRoRERGRi2KhRkREROSiWKgRERERuSgWakREREQuioUaERERkYtioUZERETkolioEREREbkoFmpERERELsqhvT4tCo3WYr47pzR2n2YNlOJ7NKioFF8rzSgd6+OpKY1tcmIDOpVVmzX5t0dK64FK8+jsrtbbz7TtfaV4t/5/l44VrR9WGlvlfe10bvK/xaqFqvVfdatYVSm+5tHNSvEpP+2Vjq3UQr4vIQBoZtnPt3OTrZlN0rG5R35UGttilB8bAN7NaKEU/8PJ69Kxg1vfpzR2eekU6QP5vyMLO6n1Mzb7+yjFC2Nzpfg/vOR7PVdQ7ImrmRS+/4UANLW/sfcablEjIiIiclEs1IiIiIhcFAs1IiIiIhfFQo2IiIjIRbFQIyIiInJRLNSIiIiIXBQLNSIiIiIXxUKNiIiIyEWxUCMiIiJyUSzUiIiIiFyUdI8go9EIIQROnT4tPbhKU4nHasu32gAA36wrSvGnT19Tijcp9Am6pNjewk0h3Gg0QtOhfYY1f6cV8qfCrNhpJ121CUzDbmrxmsJvkHNJamNDPh965c86lnoO5ROjWeRbxgAA0tV+5wmfRkrxlk51pGOveXkpjY0LV6XCjCazQrZLGass+VNpzRP4kNJ8FLv+oLdHtlJ8t3byLY7ykhOVxk5KkcuKyWjUrf2Qwz9/ZrPSfESq2t9A1YRbtHTpWNWXWFOYi9Fk0u07tLySLtSsL5TKC6by0lbzVfyidTBPd9d4Y2iapsubtCz5U+GhPKy7WrhfZdUVuAS98mcd687/Sj5KPtRLrXegKs3DWynevYK/g2YiT9Mszs2fSqx/FbX5KEUDaqMD8FHrXekI5erz56H2w8fRf6EUv6HVqNQROuawvNKEUP1dle/cuXP49NNP8d133+HKlSvIy8tD3bp10b17dzz11FOoVauW3nN1in379mHkyJHo378/5s2b55B1fPvtt/jvf/+LI0eOQNM0BAYGYvTo0ejbt69D1gcwf3oICgoqNWbgwIGYM2eOrusFmD+9mEwmLF26FOvXr0dSUhJ8fHzQpk0bPP/882jdurXu67sTc6iP9PR0LF68GNu2bcP169dRu3Zt9OnTB2PHjkXlypV1Xx/Rn016i9qd4uPjERMTA4vFgs6dO6NTp06wWCz49ddfsWzZMnz22WeIi4tDmzZt9J7vPWf58uV44403UK1aNQwYMAAWiwXbtm3DpEmTcOXKFUREROi+TuZPH+PHjy9yuRACS5cuRVZWFjp16qT7epk//bzwwgvYtm0bGjVqhKFDh+LmzZtISEjADz/8gNjYWHTrprjLXRJzqI+bN29iyJAhOHv2LBo3boxBgwYhNTUVS5cuxVdffYWlS5eiTh35XehELkkoWrJkiTAYDKJfv37i3Llzhe5fv369aNasmWjbtq24cOGC6vAuZ+/evcJgMIgXX3xR97FPnTolgoODxaOPPipSUlJsy1NSUkSXLl1EcHCwyMjI0HWdzJ/jxcXFCYPBIGbMmKH72Myffn788UdhMBjEE088IW7dumVbvn//ftG8eXPRu3dv3dcpBHOop+nTpwuDwSDGjx9vl8MDBw6IBx54QIwdO1b3dRL92ZR2il+4cAHz5s1DxYoVsWzZMjRq1KhQzMCBAzFy5EhkZmYiPj5er3rynrRixQoYjUb85z//QbVqtw/krlatGiZPnozHH38cN27c0G19zJ/jnTp1CjExMahbty6mTp2q69jMn74OHz4MAOjfvz+87jgZoV27dmjatCmSkpJ0/fwBzKGezGYzEhIS4Onpif/85z92OWzbti0GDBiAb7/9FidPnnTiLInuntKuz88//xxGoxFjx45F1apVi42LiIhAtWrVCu32yc3NRVxcHBISEnDhwgV4e3ujRYsWGDNmDHr27GkXGxQUhF69emHChAmYN28eDh48CHd3d3Tu3Bn//ve/UbFiRSxcuBCbN29GRkYGAgMDMWnSJLtdFVOnTsWGDRvw9ddfY9myZUhISEBOTg4MBgMiIiLQp08fladvJyYmBrGxsQgJCUF8fDz8/f2ljlmqW7cudu7cCQDYtWsXatasiXbt2hWKe+KJJ/DEE0+UeX5FYf5u0yN/RZk9ezaMRiOmTZsGX1/fMs+vKMzfbXrkr0qV/MPjL126ZBdjNBqRmpoKT09PBAQElHmORWEOb7vbHN64cQNZWVlo2rRpka9l8+bNAQA///yz1LhErkqpUPvmm28AoNAXQkG1atVCZGSk3bKMjAyMGDECx48fh8FgwKBBg5CWloadO3ciMjISkydPxrhx4+wec/bsWQwZMgStW7fG4MGD8dNPP2Hbtm24fPky/Pz8cOHCBYSGhiItLQ1btmzBuHHjkJCQgAYNGtiNM2nSJJw/fx79+vWDyWTC9u3b8c9//hNRUVEYO3asyksAAFi8eDFiY2PRtm1bfPTRR/D3zz87rbhjlu5k/eJPTU1FcnIyunbtiuTkZCxYsADffPMNMjIyYDAYMG7cOPTu3Vt5biVh/vLpkb+ifPPNN/jhhx/Qtm1b3XNnHR9g/vTKX2hoKObPn49Vq1ahWbNmePjhh5Geno558+bh+vXriIiIsNtKowfmMJ8eObTmJi8vr8i4jIwMAMDFixel58UTPBxj8uTJOHDgAL799tsi709NTcV7772HXbt2ISUlBffddx/+7//+D2PGjIGHR5kOpb+3qOwn7dKlizAYDOLmzZvK+1hnzpxpO27HaDTaliclJYkePXqIoKAg8euvv9qWGwwGYTAYxOzZs23L8vLyRI8ePYTBYBChoaF2x2+98847wmAwiNjYWNuyKVOmCIPBINq3by/Onz9vt85u3bqJ4OBgkZSUVOK8Cx5fER8fLwwGgxg2bJjIzMxUfh2sjh8/LgwGg/jHP/4hunbtKnr16iVee+01ERUVJUJCQoTBYBDLly8v8/hFYf70y19RhgwZIgwGg9i1a5eu41oxf/rnLykpyZa3O/8tXrxYWCyWux6/IOZQ3xw+/PDDIigoSBw+fNhuudlsFgMHDhQGg0FMnz5daqy4uDgRHBwsmjdvLiIiIsTrr78uXnvtNfH4448Lg8Eg2rZtK3755Ze7mq+r+DOP3V28eLEwGAyie/fuRd5/8+ZN0bdvXxEUFCQmTJgg3nrrLREeHi4MBoOYMGGCw+dXHigdo5aWlgYAyrt08vLysGnTJlSsWBHTp0+3q5Dr16+PSZMmQQiBtWvXFnrs008/bft/T09P2ynzw4YNs/0KA/KPSQAK78YA8ncj3PkLsX79+oiMjITRaMSWLVukn8fKlSsxd+5cdOjQAR9++CH8/PykH1tQVlYWAODQoUMIDAzEpk2bMGPGDERHR2PdunXw9/fH3Llzi3w+ZcX86Ze/go4cOYIDBw7AYDDgb3/7m27j3on50zd/eXl5WLhwIQ4ePIjg4GCMGjUK4eHh8Pf3xwcffIDPP//8rsYvCnOobw4jIyMhhMCECROwY8cOZGZmIjExEVFRUTh37hyA/LOwS7N06VJER0ejcePGSEhIQHx8PKZPn44ZM2bgs88+w5w5c5CdnY2xY8cqbaH7K7t16xZmzpyJmJiYEuMWLVqEM2fOYObMmXj33Xfxr3/9C+vWrUNoaCi2bduG7du3/0kzdl1K2xSrVKmC5ORkpKen2x38XprExERkZ2ejW7du8PYufNFL6zFax48ft1seEBBQaD3WL7iCm+YrVKgAoOjN4J07dy60zPqldOzYMannsG/fPtsXUufOnYv8ol24cGGp4wQEBGD06NFwd799OcEZM2bYjRcYGIgRI0Zg8eLF+PLLL/HUU09JzbE0zJ9++Sto3bp1AIDBgwdLzacsmD998zd37lxs3LgRI0eOxLRp02wX1bx27RqGDh2Kl19+GYGBgWjVqpXUHGUwh/rm8IknnsDVq1fx/vvv47nnnrPFNG7cGK+++ipeeukl+PiUfCHngid4FHW828CBA3HixAksXboU8fHxeOWVV0qd51/Zzp078frrr+PSpUvo2bMndu/eXWRcXl4ePv30U9SpU8fuu9Pd3R1TpkzB9u3bsWbNGoSGhv5ZU3dJSoVagwYNkJycjMTExFK/ZM6cOYPGjRvDzc3NdqxAccf3WPf7Z2fbtygp6VenyrEjtWvXLrTM+ksyPV2uTUZycjLatWuHM2fOYPHixQgNDUXTpk3tYhYtWlTqOHXr1sXo0aNtr4Wvry8CAwMLxbVo0QIAkJSk2t6oeMyffvm7kxACO3bsgLu7+10dXF0a5k+//FksFnz66acICAhAVFSU3ZXPa9WqhRdeeAEvvPAC1q1bp2uhxhzq/xkcP3687QzPrKws3H///ejWrZvteKgaNWqUOB5P8LhNr5Os1q1bh6ysLLzyyisYMmQImjVrVuTjjh07hpycHPTp0wdubvY7+OrVq4cGDRpg//79MJvNdhs3/mqUCrWePXvi559/xnfffYcHH3yw2LjLly8jLCwMVapUwc6dO20f6GvXiu63af2gW8/C0ltubm6hZdZdECV9MO/0wAMPIC4uDtu2bcOUKVMwbdo0rFmzxu7NpXIaeP369eHh4QGTyQQhRKEWGUZjfu/T0n4NqmD+9MvfnQ4fPozr16+jY8eOqF69epnGkMH86Ze/Gzdu4NatW2jatGmRBcv9998PoOjdgHeDOXTMZ7B+/foYNmyY3TLr5VesuSwOT/DIp+dJVqNGjUJ0dLTdrvWinDlzBkB+/orSoEEDJCUl4eLFi2jYsKHM07gnKR2j1r9/f/j4+ODjjz9GampqsXFLliyBEALt2rWDj48PmjRpAh8fH5w6darIX1/79u0DABgMBsXpyzl06FChZb/88gsASLeJCQwMhI+PD8LDw9GtWzccOnQIS5cuLfOcvLy8EBISgry8POzfv7/Q/dYvmeJ+iZQF86df/u508OBBAEDHjh11Ga84zJ9++atUqRK8vLxw8eLFInf1JSYmAgBq1qxZ5nUUhTnU9zP40ksvoUOHDoVeE2uHF19fX3To0KHEMa5evQoAaNKkifL6582bh+PHj2PQoEHYsGED/v3vf+Ott97C559/jtq1a2P+/PmFXrtz585h8ODBWLZsGV566SWsXr0atWvXxpEjR3D16lVs2rTJNo71OMCtW7cWWveFCxewfv16vPrqq3j99dfx2WefoWbNmpg/fz4uXLig9Dz++9//Yv78+Wjfvj3i4uLsCqwJEyaU+q/gXoaOHTuWWqQBt8/MLa7Vl7UAlN1qe69SKtTq1KmDZ599Funp6Rg1alSh3XIWiwVLly7F8uXL4evri8mTJwPIP4D1scceQ2ZmJmbPng2TyWR7zMWLFxETEwNN0xAeHn73z6gI77//PlJSUmy3z58/j7i4OPj5+SEsLEx5vFdffRU+Pj5YsGABzp8/X+Z5DR06FAAwZ84c2xsWAE6cOIHVq1ejcuXKul7mgfnLp1f+rI4cOQIACAkJueuxSsL85dMjf15eXratFgsWLLC7LzU1FfPnzwcAPPbYY2UavzjMYT69PoNNmzZFWloaVq5cabc8Ojoa586dw4gRI0otGHiCh+NOsiqN9aS64nbDW5ffunXrT5uTK1K+QElkZCTS0tIQHx+PRx55BF27dkXTpk2RkZGBAwcO4OzZs/D398eCBQvsfqFERUXh4MGDWL9+PY4ePYqOHTsiPT0dO3fuREZGBiZOnGh7U+otJSUFAwYMQK9evWA0GrF9+3bk5ORg9uzZpR6/UJT69etjwoQJiI6OxvTp07FixYpCuy5lhIWF4fvvv8f69esRFhaG0NBQZGVlYevWrTCZTJg3b57UrxIVzJ9++bOy/rEt6irzemP+9Mvfyy+/jKNHjyIuLg579+5Fhw4dbLut/vjjD0RERBR5EP3dYg71y+GIESOwYcMGLFiwAEeOHEGTJk3w66+/4ueff0aHDh3w7LPPljoGT/Bw3ElWpbG+btZDfQqyPu8/s3h0RcqFmqZpeOmllxAaGorVq1fj8OHDOHDgAEwmE+rVq4eIiAiMHj260IUBAwICsGbNGsTHx2Pr1q1Ys2YN/Pz80KZNG4wePRpdu3bV7UkVNGvWLOzZswfbtm2D2WxGSEgIxo0bh/bt25d5zNGjR+OLL77A/v37sWrVqkLHR8h688030a5dO6xevRrr1q2Dp6cn2rdvj2effdYhX7rMXz698gfkb4Fxc3PTfTdZUZi/fHrkr3r16li3bh0++OADfPXVV1ixYgW8vLzwwAMPYPjw4XjkkUfKPL+SMIf59MihdTfyggUL8N133+HHH39EvXr18K9//QsjRoywFTol4QkejjnJSkalSpVKnK/1NdZ7g0W545Srt/1JrBdr3LNnj7OnQmXA/JVvzF/591fI4QcffCAMBoOIiYkpMe7SpUsiKChIdOrUSWRnZ4sTJ04Ig8EgBg8eXGT89evXhcFgEIMGDbItK+7Cr8W9ztYL006ZMqVQbGJiYqFxDh06JAwGg5g0aVKJz8U6bnh4uMjOzhYbNmywXYDdbDaX+NiyKO55HzhwQBgMBjF16tQiH9e7d28REhIiTCaT7nMqT5SOUSMiIrqX8AQPx5xkJSM4OBh+fn746aefYLFY7O67ePEikpKSEBISouulOVJSUjB9+nR0794dISEhGDx4MPbs2SP12BEjRiAoKKjIf9evX9dtjgWxUCMior8snuCRT++TrGR4e3ujX79+uHjxIpYvX25bbjabMXfuXAC4q8NSCsrKysKYMWOwZcsWDBw4EFFRUcjJycHYsWPx/fffl/r4U6dO4cEHH0R0dHShfxUrVtRtngWx2ykREf2l8QQP/U+ykjVp0iR8//33mD17Nvbu3YumTZvihx9+wG+//Ya+ffuiV69euq1r5cqVOHXqFGJjY/HQQw8BAMLDwxEeHo5Zs2bhyy+/LPY5X716FTdv3kSXLl0wYMAA3eYk457eojZnzhycPHkSXbp0cfZUqAyYv/KN+Sv//io5tJ7gsXbtWvTv3x8XL17E2rVrsWnTJmiahoiICCQkJNh1CABun+Dx/PPPw2QyYc2aNdi9ezfatGmD//73v3ZtrfQ2a9YsPPTQQ9i2bRu++uorhISEYNmyZXdVRIwePRrBwcG2Ezz+DFWrVsWaNWvwxBNP4MiRI1i+fDlyc3MRFRWF6OhoXYvFjRs3omHDhrYiDcg/o/TJJ59EYmJikbuTrawXY3bUruySaEJIdKwlIiIip7O2kFqyZMk9X0DrKSMjA+3bt0e/fv0wb948u/v27t2LUaNGYdq0aRg1alSRj//www/x9ttvY+vWrWjSpAmysrLg6+v7p2x1vKe3qBERERFdu3YNQgjUqVOn0H3WSyuV1DLu5MmT0DQNq1atQqdOndC2bVu0b98es2bNKnQJFr1JH6N28OBBCCHg6enpyPlQAUajEZqmoU2bNnc1DvPnHHrlD2AOnYH5K9/0zB851smTJ4u8sO+JEycQGxtb7ON27NghNb71mmxF9c+2XtsuJyenxPkJIXD27FnMmDEDmqZhx44dWLlyJY4fP47ly5fbdafQk/SoQgjbP0cwWdTG9VDcFiigtnlSE5bSg2yxanMXbvKT1+v1dnT+HL3/3GhWfI0VYr3dHbfpWs/Xu0w5VInNUeunp3mrtdzJFmqn2N/MKvpq5UWpESB/oVAA8HCTy7mz86ey9jzFz4jsa2CVazIrxWsK37k+nmpf6LIj88ie8iMvLw+52TnIunb7TFa/WtVtF8W9W9bLfxS1q9L6PilpN+aQIUOQm5uLp556yrbs0UcfRbVq1bBs2TJs3rwZAwcO1GWuBUkXap6enhBCwHD//dKDmxQ+I5cz5b+UAaChj9qXRrZWuMVHSXxNWdKxbrcySg+6g9lPvk3JqbOJ0LS730Ntzd/9CvlTkaOSbACqtdHJ1Fyl+Fsm+UK7ZR3F9iQWU+kx/3PqzDndjmEoy2dQy5PfJG/68kOl+VToqHbl/m25hXc5lOTNr36Tjv1oVDulsRtVktuqdfr0aaVxS1KW/N1SKL4OJ6vtfmlYqfSr9t9p9/k/lOI9FX6QhjauojS2r7vc63Lq9zNK45YHc+bMwZw5c5w9DYfIvpaCLf833na7/2eLUKdxfemtZiWxtqHKzS38t8S6Ja24LhNA8ZcJGTFiBJYtW4Y9e/Y4v1AjIiIichQNgM8dv+L13NdRr149APnHqhWUnJwMAEUev1Yaa9sxa4N5R+DJBEREROR0mgZ4aprtn54nVPr7+6NRo0Y4cuRIofusy1q1alXkY5OSkhAWFobZs2cXuu/Mmfyttg0aNNBvsgVwixoREVEpeDKIfoo7ySN/i5qb3W099evXD4sWLcK3336LHj16AMjfEvbJJ58gMDAQLVu2LPJx9erVwx9//IHPP/8cERERqFWrFoD8DgoLFiyApml47LHHdJ7tbSzUiIiISqFyMojKEbuaZLTIuCk9pvCXO+YvLVf+2HBfL7lywUviAGQhRJHPWoNWYNenvqXamDFjsHHjRkycOBGjR49GjRo18Mknn+Dy5cv46KOPbMcTHzx4EElJSejatSuqV68ONzc3vP7663juuecwaNAgDB06FF5eXkhISMChQ4fw/PPPIzg4WNe53omFGhERUSlUTgbJUTgJxAdyxVJm/GvSY5pHy8V+/O056TEfC64tFde0Zulngyed/b3I5ZpW4Bg1nTep+fv74+OPP8Zbb72Fjz/+GEajEc2aNUN8fDw6depki1u7di02bNiA5cuXo3r16gCAv//974iLi8MHH3yAxYsX207OmzdvHvr376/vRAtgoUZEREROpwHwcnPMyQRWtWrVKtSZoKDizqzt1q1boTZifwYWakREROR0jj5GrbxioUZEREROp2lAhTsLNVZqAFioERERkQvQNA0ePh52t6kMhZqjGnL4K7YQOZWhNpMZXxxWio/s3lg6tnd1tfY1UOo0UD7eqDkKnQAAoHaq/FXnAaBVrWZK8XuvKczHXLi/XIl06BRxN1Te+W558hdhTH14gtI8VLo/AECo23ml+BevyHf8SEorvkdfURpXVGtnpSeltlAKrfXa1FbrsOGVm6YU/2QTtU4GN0ThnorFUW1/5avaQ5DKBw1w93Kzu03cokZEREQuQNM0eFTgFrWCWKgRERGR82mwK9S4RS0fCzUiIiJJMjtpfSQu+mqV/MZkqbiaPbpIj+l267pU3Gu9A6XH/OVqtty6pUcsjMeoFY2FGhERETmfBrh7udvdJhZqRERE5AI0TYPnHbs+LdyiBoCFGhEREbkCDXa7PvNYpwFgoUZEREQuoOBZn0ZuUQPAQo2IiIhcgQa4efKsz4JYqBEREZHT5Z/16WV3m1ioERERkSvQNLhX8La7TQ4u1HIVWsycSFFrAfPa5mNK8S0aVlaKr1rBUzrW7FdRaWxnvvVU1q3lpkvHVqug9hqk12qpFJ+aY1aKb1FDvn2NW84NpbEt/jWU4p0pSVSWjl24+6zS2G91r6oUn/vteqX4wWGDpWM711N7/wFq7a/0pPIZrJZ8RDo28b13leZR7//CleI9atRViq96/rh0rNZhgNLYAv5K8VQ+5B+jxi1qBXGLGhERETkfj1ErEgs1IiIicr4CW9S46zMfCzUiIiIdeVz/XTr24nenpeKMExdIj9kwN0kqLm97tPSY9fu+KBXnJdE+q7gITdPgzl2fhbBQIyIiIufTNHj48GSCglioERERkdNpBY5RY52Wj4UaEREROZ+mwd2bW9QKYqFGREREzqdp0Lwr2N0mFmpERETkEjRoXhXsbhMLNSIiInIFmgZ48PIcBbFQIyIiIufTCmxRY6EGgIUaERERuQIeo1YkhxZqZiEfW8VHvrcmADz1tyZK8U829io96A5ut1KlY28JtR54EtcDdBiFlCgdHbDiSLLSPCa/qNaX8N+vjVOKn9S+pnSsuRz17gTU8lLbX/5z9VyXRkrzsPi6K8V7PzRIKf7l8/J9LsW+fUpji87/UIp3FkuFStKxd14oVGrsbPlevgCgZQcoxV/Z9o10bIOGzZXGNjZoqxRP5YOmaXC7Y4uaxmPUAHCLGhERkdM8+J9npOKyKsj/MDo78xWpuCqG+tJj/jkbtzTA444flyYWagALNSIiInIFmgbN2+f2bTMLNYCFGhEREbkCTYPmyQveFsRCjYiIiFyABuF+57GWZqfNxJWwUCMiIiLX4H7nsXgs1ADAzdkTICIiIoKWv0XN+s8Ruz5TUlIwffp0dO/eHSEhIRg8eDD27Nkj9dikpCRMnDgRnTt3Rps2bRAREYGjR4/qPseCWKgRERGRCyhQqOl8eY6srCyMGTMGW7ZswcCBAxEVFYWcnByMHTsW33//fYmPvXbtGoYNG4Z9+/Zh+PDhmDhxIs6fP4/hw4fjxIkTus6zIO76JCIiIqcTmgbh7ml3W89SbeXKlTh16hRiY2Px0EMPAQDCw8MRHh6OWbNm4csvv4RWzFa89957Dzdu3MD69evRrFkzAMCjjz6KsLAwzJ07F0uWLNFxpva4RY2IiIhcg7vH7X8627hxIxo2bGgr0gDAz88PTz75JBITE3Ho0KEiH2c2m7F582Z07NjRVqQBQM2aNREWFoYff/wR165d032+VizUiIiIyAU4btdnRkYGzp49i1atWhW6r2XLlgBQbKF2+vRpZGdnF/tYIQSOHJHvpKKKuz6JiIjI+TQNuGPXJzQNV65cwaRJk4p9yI4dO6SGvnbtGoQQqFOnTqH7atbMbzl46dKlYh8LoEyP1YNyoaZS37opBAf7ZCvNw3BmlVL8haUHleK9K/tLx9Z+bqrS2OZKar1B9aSSP+FZofSg/1m3/3eledRu0VUpfnJIRaX4bMj3uMzLsyiNXcmr/GyI9v7tK+nYwJoNlcb+Nv0+pXijpYpSfOMVa6Rj6z3SXWns8pLBK77yLX7SJixQGjsp16gUH/3VaaX4sRFvScfWC8hQGtuZZL5DLSr9g0P6SIX5HdwoPeSJY3K74TwU+sPWM8v1v7bg7nonW9zV+n7LysjIf4/5+PgUus/X1xcAkJOTo/xY67LiHqsHblEjIiIil2C2CLvbderUkd5qVhKLJf8HeVEnCwghir2vtMdaubk57ucfCzUiIiJyOgHAdEehJqDfUWp+fn4AgNzc3EL3WbeGBQQElPmx/v7ye+FUsVAjIiIipxMCMFrsb+tVqdWrVw8Aijw7Mzk5GUDRx6Dd7WP1wEKNiIiIXIKpwK5Pvfj7+6NRo0ZFnp1pXVbUWZ0A0LhxY/j5+ZX42NatW+s4W3vl5ZhaIiIiuseZLcL2T2/9+vXD77//jm+//da2LCsrC5988gkCAwNtl+koyMvLC6GhodizZw9OnTplW56cnIwvvvgC3bt3R9WqVXWfrxW3qBEREZHTCQDGAseo6WnMmDHYuHEjJk6ciNGjR6NGjRr45JNPcPnyZXz00Ue2kwUOHjyIpKQkdO3aFdWrVwcA/POf/8SuXbswatQojBkzBt7e3lixYgXMZjOioqJ0nqk9blEjIiIipxMCMFlu/xM6V2r+/v74+OOP0atXL3z88cd466234OPjg/j4eHTp0sUWt3btWrz00ks4c+aMbdl9992HVatWoXXr1oiNjcWiRYvQsGFDrFy5EkFBQfpOtABuUSMiIiIXIOy2qOl73me+WrVqYd68eSXGzJkzB3PmzCm0PDAwELGxsbrORwYLNSIiInI6AcAsHLfrs7xioUZERCRJpnj4w73o63EVpaJkpxPv+xpLj9l+/jSpuFu//SQ9pmYsfA0x3QkgzyTsbhMLNSIiInIBAkCe2Wy7Ld/g6t6mXKipFLhZRvkeikdS1fp7de86UCk+rVOkUrxbCa0iCqq2J05t7L7PKsWXB71b1FKK3zTMoBR/eOhgpfgGqzdJx17PNimNXdGr/Hx9uNWQ7xV5zkf+FzsABFVUOxfpaqZab8kGby+TjnXL/kNpbHPpIQ6j8h16M1d+pt+fV3sNnm6i1uN2Rl+1A6bDp3wqHXs2bqjS2BBqc6fyQQiBXNPt3HrqfTZBOcUtakREROR0AoDRzGPUCmKhRkRERE4nANwyW+xuEws1IiIicgFCwG7XJ/d85mOhRkRERE4nANwycYtaQSzUiIiIyOmEEDCKOws1lmoACzUiIiJyAYW2qLFCAcBCjYiIiFyAEECO8fYlaYS7EyfjQlioERERkdMJIZCTd0eh5s1dnwALNSIiIl2ZLfIFRtotuYv3Vvxtn/SYHrUaSMV5B3eQHhNuahe4LgsBII8nExTCQo2IiIicTgjYb1FjpQbAwYWaG+TbMG0/dV1p7Ad7NlKKHzxlm1L89DEPSsdW2XdYaew6oXkK0QJQeB31lAP5tl4NK/soja0d3aUU7+6ldrCCbKNjAKiWlqQ0tiW7mnywsACa8w60MFWuJx2bkanWlufMHzlK8Q/VUPvW/f6SfGsvf29fpbEfUAt3mhq+8l/RDzWuqjS2W9ZJpfhDKWrv426hbaRjPf64qDS2uVJtyUj+pS9PhBDItivUmD+AW9SIiIjIBVhgfzIBO7rmY6FGREREzicE8kzmO247byquhIUaEREROZ1FwG7Xp4XX5wDAQo2IiIhcQOGTCVioASzUiIiIyAUICNy6s1Djvk8ALNSIiIjIFQjAbLbY3SYWakREROQChABMeRa728RCjYiISFc+nvLXcXTX5K6TeevqVekxPToPkIpL9qwuPaaXu9w8A8TdXFRDwHTH5Tm4SS0fCzUiIiJyOiFgV6hxi1o+FmpERETkEswmXua2IBZqRERE5HRCCJiMFrvb5OBCraqP/DVQnulYX2ls1Zp7zdS/KcUHH/9MOtbtsX5KY1s0+eMXnMlH8pgEAOhUr5Li6N2UolvEq8VrqeelY7N3fKI0tl+XR+TnYfYC3PS9FpDKV5fmLv8RD/zqbaV5PPDEC0rxu68YleLf2HJMOnZ6v+ZKY1skX0RH/JlQ6dzr4yEf/WNKltI8RHWDUvxDjdR6DkdUkz+mynzxhNLYmm8VuUD+oS9fCuz65CFq+bhFjYiIiJxOADDZXUeNABZqRERE5AqEgMXuOmos1QAWakREROQChBAw5d2yu00s1IiIiMgVCAHzrRy728RCjYiIiFyCgDkvx+62s2RmZuK9997Dtm3bkJKSgiZNmuDpp59GWFiY1OOnTp2KDRs2FHnf2rVrERISIj0XFmpERESlMBqNEELg9OnTpcY6pLxoqXC2+eU/pOIsWnpZZ1MsN4mTg41GI7QiOjIIIWA25dnddgaz2Yznn38e+/fvx5AhQ3D//fdjy5YteOGFF5CdnY1//OMfpY5x8uRJNG7cGM8++2yh+xo0aKA0HxZqREREpbAWFkUVGIViHTGBgKq6D6nvhYPkaZpW9OvoIrs+t27dir1792LGjBkYPnw4AOCJJ57A8OHD8dZbb6Fv377w9/cv9vFmsxlnzpxBWFgYBgyQa+dVEhZqREREpWjTpo2zp3DPE8J+16eztqht3LgRvr6+ePLJJ23LPDw8MHLkSEyePBm7d+8ucRfouXPncOvWLRgMatcqLE75uPIqERER3eMEzHm5tn/OOkbt8OHDaNasGby8vOyWt2zZEgBw6NChEh9/8uRJAEDTpk0BANnZ2bBYyt4ai4UaEREROZ8QMBvzbP+cseszJycHN2/eRJ06dQrdV6tWLQDApUuXShzDWqh9/fXX6NmzJ9q0aYO2bdsiKioKqampynOS3vWpciBlWZgV85GuehCA4gpOVX5APli1JdTZROlQo9EkdUxE6ePk5++Ui+TvpnITMDWakB/fEvSw0thuWV6lB/1PntCgGdVaJxWnbJ9B+cQIxddBO5ekFF/JpPYmmdbBV37szMtKY587I/eZNRmNgA6fP6Bsn0GVV+w+o9pnKveq2vPKVYoGjgn5I6CEx31qg1+Qa09lNJkdc7wYOUTNKn5Y88pAu9tXrlzBpEmTin3Mjh07pMd/5513kJmZWWLMM888AwDw8fEpdJ+Xlxfc3d2Rk5NT6L47WQu13377DZMmTYKvry9+/PFHrF27FocPH8a6desQEBAgPW/pQk3lQMqyUGhpVyZeqivwkH8RHanYgy7LMM6d/9Wbev4cfRir/PhuAZJ9A8tAK+bspjKNVaYcysdqDnwdAMDHUzG+krMOdb5Nr8+fdaw7/yv1GIXxfb2c/3rZ85aO1Bx0tLSmWRz2nUf6su5mbFy3gt3ypPNpuq3js88+Q0pKSokx1mPPSnrflPaeCgsLQ8uWLREZGWl7Xn369EHjxo3x5ptvYtmyZRg/frz0vDXBS/8SERERITMzEw8++CD69euHt99+2+6+vLw8tGzZEn379sX8+fOVx87Ly0ObNm3QqlUrrF69WvpxPEaNiIiICIC/vz8qV66M5OTkQvddu3YNAIo8fk2Gl5cXAgICkJWVpfQ4FmpERERE/9OiRQscO3YMJpPJbvmRI0cAAK1atSr2sdnZ2XjssccwceLEQvelpqbijz/+UL7gLQs1IiIiov8JCwtDZmYmPv30U9sys9mM5cuXo3LlyujZs2exj/X19YWnpyd27NiBEydO2N0XExMDAAgPD1eaD49RIyIiIvofs9mMoUOH4ujRoxg+fDgCAwOxefNm/PTTT5g9ezYef/xxW+yJEydw8uRJtG3bFvXr1wcAHD16FEOGDIGvry+GDRuGqlWrYteuXfj+++8RHh6OuXPnKs2HhRoRERHRHdLT0xETE4Pt27cjKysLTZo0QWRkJB55xL7n6sKFC7Fo0aJCBdzhw4exaNEiHDhwAHl5eWjUqBEGDRqEoUOHws1NbWcmCzUiIiIiF8Vj1IiIiIhcFAs1IiIiIhfFQo2IiIjIRbFQIyIiInJRLNSIiIiIXBQLNSIiIiIXxUKNiIiIyEWxUCMiIiJyUSzUiIiIiFwUCzUiIiIiF8VCjYiIiMhFsVAjIiIiclEs1IiIiIhc1P8DT/iUfvKBZXEAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 700x280 with 11 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "W = 1.4\n",
    "H = 1.4\n",
    "K = 10\n",
    "fig, axgrid = plt.subplots(nrows=2, ncols=5, figsize=(5*W, 2*H))\n",
    "for k in range(10):\n",
    "    cur_ax = axgrid.flatten()[k]\n",
    "    im_handle = cur_ax.imshow(w_KF[k].reshape((8,8)), interpolation='nearest', vmin=-.5, vmax=.5, cmap='RdBu_r')\n",
    "    cur_ax.set_xticks([]);\n",
    "    cur_ax.set_yticks([]);\n",
    "    cur_ax.set_title(\"Comp k=%d\" % (k+1))\n",
    "    if k == K - 1:\n",
    "        plt.colorbar(im_handle, fraction=0.06, shrink=0.95);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Discussion 2a: Are these interpretable? Do you see evidence of the \"digits\" here?\n",
    "\n",
    "Look for evidence of strong vertical lines, or certain holes, etc. What are these components capturing?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO discuss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create reconstructed images"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1697, 64)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_tr_NF.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z_tr_NK = my_transform(x_tr_NF, w_KF, m_F)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1697, 10)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z_tr_NK.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xhat_tr_NF = np.dot(z_tr_NK, w_KF) + m_F[np.newaxis,:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1697, 64)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xhat_tr_NF.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Show the original and reconstructed images from training set\n",
    "\n",
    "We'll look at *first five* training examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAqAAAAEDCAYAAADqXLYYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA8qUlEQVR4nO3deXhM1/8H8PcksieS2JKI1FZbVRKKCFqE0qIEtdQe1E6oVuuLfrX80FKltdUeEkuoxNolUYLYSWJLxNZEqdiJiGTE/f3hyXxFJnHPbJnl/Xoez9OZ+dxzP8nJTD9z7j3nKCRJkkBEREREZCBWJZ0AEREREVkWFqBEREREZFAsQImIiIjIoFiAEhEREZFBsQAlIiIiIoNiAUpEREREBsUClIiIiIgMigUoERERERkUC1AiIiIiMigWoAJq1aqFWrVqlXQaGjlx4gRCQkIQEBCABg0aYMCAATh27FhJp1XiTLlPX7ZhwwbUqlULaWlpJZ1KiTPVPpUkCZGRkfj444/h7+8PX19ffPTRR1ixYgWePXtW0umVKFPt02fPnmHjxo3o0qUL6tevjyZNmmDs2LFISUkp6dSMgqn266vu3buHwMBAvPfeeyWdiklhAWoB9u7diwEDBiAlJQUfffQROnfujPPnz2PAgAGIjY0t6fRIS4cPH8bs2bNLOg3S0n/+8x9MnToVt2/fRqdOndC9e3fk5ORgzpw5GDFiBPLy8ko6RRI0efJk/Pe//0Vubi569OiBVq1aIS4uDh9//DHi4+NLOj3Ska+//hr37t0r6TRMTqmSToD0KycnB1OnToWzszO2bt0KLy8vAEBISAi6d++OadOmoVmzZnBwcCjhTEkTGzZswMyZM5Gbm1vSqZAWDh8+jK1bt6Ju3bpYu3YtnJ2dAQC5ubkYPnw49u/fj+joaHTr1q2EMyW54uLiEB0djaZNm2LFihWwtrYGAPTt2xc9e/bEtGnT8Oeff0KhUJRwpqSN6OhoxMTElHQaJokjoGbujz/+wO3bt9GrVy9V8QkAb7zxBvr164fbt29zFNQEXb58Gb1798a0adPg5eWFKlWqlHRKpIXffvsNADBq1ChV8QkAtra2GDlyJIAXVzLIdKSmpqJ8+fIYNmyYqvgEgLp16+LNN99Eeno67ty5U4IZkrZu3ryJGTNmICgoqKRTMUksQLXUr18/NGzYEPfu3cPkyZMRGBgIf39/9OrVC6dOnQIAbN68GR06dICvry/atWuHsLAwSJJUoJ2cnBz8/PPPaNu2rSpuzZo1iI6ORq1atbB161aN8jtx4gQAIDAwsNBrTZo0AQAcOXJEo7bNlbH3KQAcOHAAp0+fRv/+/REVFYUKFSpo9TObO2Pv01atWmH06NHw9fUt9JqtrS0AICsrS6O2zZWx9+mnn36KgwcPqj5n8z19+hQ3btyAnZ0dSpcurdkPb8aMvV/zSZKESZMmwdraGt9++61WbVkqXoLXgWfPnqF3795QKBQIDg5Geno6YmNjMXjwYHTp0gVRUVH48MMPERgYiG3btmHmzJlwc3ND586dVccPHjwYx48fR82aNfHJJ5/g5s2b+O6771CpUiWtcrty5QqAFyOer8p/Lj+G/seY+xR48eXhjz/+gLe3t9ZtWQpj7tNWrVqhVatWal/7/fffAcAsJmvomjH36auys7Nx/vx5zJ8/Hw8fPsSYMWNgZ2en03OYC1Po14iICBw6dAjz5s1D+fLlddKmpWEBqgPZ2dlwc3PD2rVrVaMV48ePx+7duxEZGYmtW7eiZs2aAICgoCCEhIRg+/btqjdLREQEjh8/jnbt2mHevHkoVepFt/zxxx8YO3asVrk9evQIAODq6lrotfxv35mZmVqdwxwZc58CQO3atbVuw9IYe5+qc/78eaxbtw6lSpVCjx499HIOU2YqfXr58mW0b99e9bhPnz4YPXq0zto3N8ber3///Tfmzp2Ldu3aoUOHDlq3Z6l4CV5H+vfvr3qjAMA777wDAGjZsqXqjQIADRo0AAD8888/que2bt0Ka2trTJo0SfVGAYB27dohICBAq7yePHkCAAVyy5f/XE5OjlbnMFfG2qekOVPq08uXL+PTTz9Fbm4uQkNDUa1aNZ2fwxyYQp8+f/4cAwcOxCeffAJvb29ERERg/PjxnDxYDGPt17y8PHz11VdwcHDAtGnTtGrL0rEA1ZGqVasWeOzo6AgA8PHxKfC8vb09AECpVAJ4Mcs1JSUFHh4eBSYJ5WvUqJFWeeVf4sk/38vyP/zyc6WCjLVPSXOm0qcJCQno3bs37ty5gz59+mDo0KE6bd+cmEKf1qhRA5MmTcK0adOwa9cuNG3aFLt378b69et1dg5zY6z9umLFCiQkJOCbb75BmTJltGrL0rEA1ZGiijh1I48vu3//PgAUeQ+Jh4eHVnnlX3pXd5k9//K8i4uLVucwV8bap6Q5U+jTXbt2YcCAAXjw4AGGDBmCr7/+WmdtmyNT6NOXOTg44MsvvwQALt9TDGPs15SUFPz888/o2LEj2rZtq3E79AIL0BLm5OQEoOj7MB8/fqxV+9WrVwcApKenF3ot/7n8GNINffcpGZ6h+nTFihWYMGEClEolpk6dii+++EIn7VJh+u7TpKQkbN++vdDsbOB/o3h3797V6hxUmD77NSYmBkqlEjt37lTt4vTybk4ZGRmoVasWl2WSiZOQSpizszOqVq2KtLQ03L9/H+7u7gVeT0xM1Kr9Ro0aITIyEkeOHCl06eHw4cMA/ndvDemGvvuUDM8Qfbpq1SrMmTMH9vb2+OGHH9CmTRut26Si6btPZ82ahYSEBPj4+KB+/foFXjt//jwAoHLlylqdgwrTZ782bty4yMljCxcuhJOTE0JCQnhVUSaOgBqB7t27Iy8vD999912BPZ/j4+O1XiS+devWcHd3R3h4OK5du6Z6Pj09HeHh4ShXrhzatWun1TmoMH32KZUMffbpiRMnMGfOHNjY2OCXX35h8Wkg+uzTTp06AQDmzJlTYLLRvXv3MGPGDNX5Sff01a8BAQEYM2aM2n/Ai+J3zJgxGDhwoLY/gkXgCKgR6NevH/78809ERUUhOTkZAQEBuHnzJmJjY+Hq6op79+7Byup/3xWSk5MRGxsLb29vdO3atdi2nZyc8PXXX2PChAno1q0bOnbsCEmSsGvXLjx+/Bg///wz16LTA332KZUMffbp3Llz8fz5c9SsWRPHjx/H8ePHC8V4eXnh448/1vnPZcn02ac9e/bE3r17sX//fnTo0AEtW7bEkydP8Ndff+HevXsYNGgQv2joCT9/TQNHQI2Ara0tVq5ciUGDBuHhw4dYv349Lly4gClTpqjWNXv5huzk5GQsXLgQUVFRstpv3749VqxYgZo1ayIqKgq7du1CnTp1sGbNGrRu3VovP5Ol03efkuHpq09zcnKQlJQE4MUkh4ULF6r99+uvv+rvh7NQ+nyfWltbY8mSJfjyyy9hb2+PDRs24LfffsObb76JhQsXqiYike7x89c0KCR1d0iTQf3zzz9wd3dX3Tz9sgkTJmDnzp3YvHmz2m36yDixT80P+9T8sE/NE/vVNHAE1AjMmjULDRo0wLlz5wo8f+XKFfz1119wd3fnzjcmhn1qftin5od9ap7Yr6aBI6BG4ODBgxg6dCjs7e3Rtm1bVKhQAf/++y9iY2ORk5ODefPm4YMPPijpNEkA+9T8sE/ND/vUPLFfTQMLUCORmJiIVatW4fTp07hz5w7c3NxQv359DBo0qNASHmQa2Kfmh31qftin5on9avxYgBIRERGRQfEeUCIiIiIyKBagRERERGRQZr8QfUJCAiRJgo2NTUmnYtaUSiUUCoVB7q1hnxoG+9T8sE/NjyH7FGC/Goqh+7UkmH0BKkkSlEplgW0odcnT01N27MsL377O8+fPhfL4999/Zcfm5OQItS2Hp6cn8vLydN6uOqJ9am1tLdS+j4+P7FiRfhLpI+DFB1BJMuY+Ff2fn4eHh+xYW1tb2bGPHz8WyuPWrVtC8bpmzH0qqly5crJjRd5LDx8+1CSdEmPIPgX0368iO/OVL19edqxov2ZmZgrF65qh+7UkmH0BamNjg2vXriE4OFgv7UdHR8uOzd+BQQ7RN4vI3rOJiYlCbcsh8nvQlmifurm5CbX/999/y4598OCB7FjRLd5E8tAHY+7TKlWqCLUv8rPUq1dPdmxYWJhQHiW9R7Qx96mo+fPny44VeS+JtGsMDNmngP771d/fX3bsmjVrZMeuW7dOKA+RtvXB0P1aEngPKBEREREZFAtQIiIiIjIoFqBEREREZFAsQImIiIjIoFiAEhEREZFBsQAlIiIiIoNiAUpEREREBsUClIiIiIgMyuwXohclsli8aHxcXJzsWJEFzgGxRXNFFvo1B6ILJov87kUXRBch0k+ify8lvci9tkQXiRbppwULFsiODQ0NFcpj2rRpsmNNvY+MScuWLWXHmtpC9OZm3LhxsmNF3tf79u0TzoX0iyOgRERERGRQskdADx8+rPFJAgMDNT6WiIiIiMyL7AI0JCQECoVCo5MkJydrdBwRERERmR/ZBWhwcLDGBSgRERERUT7ZBejs2bP1mQcRERERWQi9T0KSJEnfpyAiIiIiE6LVMky3b99GTEwM7t27h+fPn6uKTUmS8OzZMzx48AAHDhwQWn6IiIiIiMybxgVoamoqevfujaysLEiSpLo/9NX/dnBw0E2mRERERGQWNC5AlyxZgsePH6N9+/Zo0qQJfv75Z7z11lto1aoVLl68iC1btsDFxQW///67LvMlIiIiIhOncQF6/Phx1KtXD/PmzVM9vnnzJnr16gUAaNeuHUJCQhAREYFhw4bpJlsiIiIiMnkaT0J68OABGjRooHpco0YNnDt3TvW4cePGaNq0KWJiYrTLkIiIiIjMisYjoA4ODrCy+l/9WqlSJWRnZ+PGjRuoWLEiAKB27drYsmWL9lkaUFpamlB8UlKS7FiRPclbtGghlEdYWJhQvCUR3QO4cuXKsmP1uV+7yH7nAwcOFGrb1Inuky7yNyCyF7joXvAie5KL7ndvaUT+5kU+e0m3RPZrB8T6SuT9JPqZQfqn8Qho9erVcfr0adXjqlWrQpIkpKSkqJ57+PAhcnJytMuQiIiIiMyKxgVou3btcPLkSUyZMgUZGRmoUaMGypUrh8WLF+PGjRs4ffo0fv/9d6HRJCIiIiIyfxoXoH369EHjxo3x66+/4tChQyhVqhQGDRqEs2fPonXr1ujZsycePXqEfv366TJfIiIiIjJxGt8Damtri7CwMMTExKBWrVoAgEGDBsHKygrbtm2DnZ0dunXrhq5du+osWSIiIiIyfVrthKRQKNC2bdsCzw0cOFDjCRE3btx4bYy1tTUcHBxQunRpjc5BRERERCVLqwJU14KCglS7KL2Og4MDmjZtii+//BI+Pj56zoyIiIiIdEWrAnTTpk0IDw9HWloalEql2hiFQoHz58/Laq9bt244evQo/vnnH5QuXRq1atWCh4cHHj9+jLNnz+LOnTtwdXWFt7c37t69i9jYWJw6dQpRUVHw8PDQ5kchIiIiIgPRuACNiIjAjBkzIEkSXFxcUK5cOa2TadeuHaKiotC3b1+MGzcOzs7OqteeP3+OhQsXYtmyZZg0aRIaNWqEbdu2YdKkSfjll1/w9ddfa31+IiIiItI/jQvQjRs3wsnJCUuXLkXDhg11ksyiRYtQv359TJkypdBrVlZWGDt2LI4ePYoff/wR69evR+fOnbF7927s379fJ+cnIiIiIv3TeBmmtLQ0dOzYUWfFJwBcuHAB9evXLzbG19e3wJafNWvWxO3bt3WWAxERERHpl8YjoKVLl0apUrqdw1S6dGlcuHCh2JiLFy/CwcFB9fjJkycFLtVrKzExUSheZCtGEePHjxeKF91u0pKIbsG2bds22bHR0dGyY93c3ITyENmKUfTv1tSNGzdOb22L9hPphuiWja6urrJjLe39YUxE+1Vky2L2q2nTeAQ0KCgIe/fu1elWm82bN8fBgwexbt06ta9v3rwZBw8eRLNmzQC8KD737t2LatWq6SwHIiIiItIvjYcwP/vsMyQkJGDIkCEYMGAAqlSpAjs7O7WxcpdJGjt2LA4cOICZM2diw4YN8PPzQ/ny5ZGZmYnExESkpKTA3d0d48aNg1KpxAcffIDbt2/j888/1/THICIiIiID07gAbdu2LZRKJS5evIgTJ04UGSeyDJOnpyc2btyImTNnYt++fbhy5UqBdlq0aIHJkyfDx8cHaWlpePr0KUaMGIH27dtr+mMQERERkYFpXIDmb7+pa97e3li0aBEePHiAc+fO4f79+3ByckLdunVRoUIFVVzlypVx7NgxveRARERERPqjcQFa1H2auuLm5qa615OIiIiIzIdRbcUJANeuXcNvv/2G69evIzc3V22MQqHAzJkzDZwZEREREemC7AJ0wYIFaNKkCQICAlSP5VAoFBg7dqys2Pj4eIwYMQJKpRKSJBXbJgtQIiIiItMkuwBdsmQJrK2tVQXokiVLoFAoii0UAbECdP78+Xj27BmGDRuGBg0awN7eXm56RERERGQiZBegs2bNQp06dQo81rVLly7ho48+0usi00RERERUsmQXoF26dCn2sS6ULl1aaHcLIiIiIjI9RjUJqWPHjvj9998xfvz4AtttmguRLf5atGgh1Lbo1p1UNJFtTTt37iw7VmSLT0C/202aOpHt+kTNnz9fduzDhw+F2uaWuUXz8/PTW9si76WBAwcKtS2yHS/f07olss1y5cqVhdpOSkqSHRscHCzUtuj20OZK4wK0devWr42xtraGg4MDvLy80LRpU3zyySewsbEpMn7s2LE4d+4cunbtir59+6JSpUqwtbVVGxsYGKhp6kRERERUgjQuQK2srPDw4UM8evQIwItis2zZssjKykJWVlaB2AsXLiAuLg67d+/G2rVriywqHz16hKysLFy9ehUzZswo9vzJycmapk5EREREJUjjAnTZsmXo1asXfH19MXHiRNSvXx/W1tYAgNTUVHz//fdITk7G6tWrYWdnh+XLl2PLli1YvXo1hg0bprbN6dOn48yZM/D09ISvry+cnJw0TY+IiIiIjJTGBeicOXPg6uqKdevWwc7OrsBrNWvWxKJFi9CpUycsWrQICxYswIwZM5CcnIzdu3cXWYAeOXIEvr6+WL9+PUqVMqrbU4mIiIhIR6w0PfDo0aNo06ZNoeIzn52dHd59913Ex8ernmvUqBH++eefItvMy8tD48aNWXwSERERmTGNC9BSpUrhzp07xcY8ePAAz58/Vz1WKBSwsir6lP7+/jh37pymKRERERGRCdC4AK1Xrx7++OMPnD17Vu3rycnJiImJwdtvv6167sSJE6hUqVKRbX7xxRdITEzE999/j3v37mmaGhEREREZMY2vdY8aNQr9+/dH79690aVLF/j7+6N8+fLIzMxEYmIitmzZgmfPnmHkyJGQJAmjR4/G2bNnMWHChCLbXLhwIby8vLB69WqsXr0aLi4ucHR0LBSnUCiwd+9eTVMnIiIiohKkcQFav359/Pzzz5g6dSo2bdqEyMhI1WuSJKFMmTL47rvv0KRJE9y8eRN79uxBo0aN0Ldv3yLbjI2NLfD40aNHqmWeiIiIiMg8aDXbp2XLloiJicHBgwdx5swZ3L9/H05OTqhXrx6CgoJgb28PAHBxcUF0dDRq165dbHspKSnapENEREREJkDr6eb29vZo06YN2rRpU2SMk5PTa4tPIiIiIrIMsgvQLVu2oG7duqhTp47qsVwff/yxeGZmSJ97TIu0TcWbNm2a7FiR/YL9/f3FkyGdqFKliuxYkX2d16xZI5QH94AuGSJ7sIt+lrq5ucmOFf17Ed2X3hg9ePBAKF5kz/YFCxbIjhX5XAfEPq9F+7Vly5ZC8eZKdgE6ZcoUjB49WlWATpkyBQqFothjJEmCQqEosgDt37+/QKr/o1AoEBYWptGxRERERFSyZBego0ePRkBAQIHH2jp27JhGx72u8CUiIiIi4yVUgL7svffeQ506dWBjY6PxydeuXavxsURERERkmjSehDR69Gi89dZbWLp0qcYnb9y4scbHEhEREZFp0ngnpPv37+PNN9/UZS5EREREZAE0LkAbNmyIw4cPQ6lU6jIfIiIiIjJzGl+CHzBgAKZMmYL27dsjKCgIPj4+qoXnX8VlmIiIiIgon8YF6PDhw1X/HRYWpnZm+uuWYSIiIiIiy6NxATpq1Cguh0REREREwjQuQMeMGaPLPIiIiIjIQmi1F3xubi4yMjJw//59uLu7w8PDA7a2trrKzSTs27dPdmyLFi30lkd0dLTsWNHt3US3UjM2ItswihLZ4m/v3r1CbYtsBZeYmCjUtqkT7VOR34+rq6tYMgJEtgMU3bZTdDtAY5OWlqa3tkXeS6K/d5HPAEvcglH0s0nk70BkG1TR//+GhITIjjX1/0eWFI0K0KNHjyIiIgL79u0rMAu+VKlSaNmyJXr37o3AwECdJUlERERE5kOoAJUkCf/3f/+HiIgISJIEOzs7VKlSBW5ubsjKykJaWhpiYmIQGxuLTz75BJMnT4a1tbW+ciciIiIiEyRUgM6fPx/h4eHw9vbGZ599hqCgIDg4OKheVyqViIuLw7x587BhwwY4Ojri888/13nSRERERGS6ZC9En5qaimXLluHtt9/Gjh070KFDhwLFJwDY2NigTZs22Lp1K/z9/bFq1SqkpKToPGkiIiIiMl2yC9D169fD2toaCxYsgKOjY7Gx9vb2+OGHH2BjY4PNmzdrnSQRERERmQ/ZBejJkyfRrFkzeHt7y4qvWLEimjVrhqNHj2qcHBERERGZH9kF6PXr11GtWjWhxqtVq4YbN24IJ0VERERE5kt2AWplZQVJkoQaz8vLs7h1QYmIiIioeLILUG9vbyQlJQk1fvr0aVSqVEk4KSIiIiIyX7IL0GbNmiExMVF2EZqQkICTJ08iKChI4+SIiIiIyPzILkAHDBgAW1tbTJgwAf/880+xsVevXsW4cePg6OiInj17ap0kEREREZkP2QvRe3h4YOrUqZgyZQq6dOmC3r17o0WLFqhevTpcXFyQnZ2NK1eu4M8//0RERASePHmCBQsWoGzZsvrMv8SJ7EUrcguDyN7uABAcHCw79v79+0Jtf/PNN6+NcXd3F2rTkPS5T6/I/u5xcXFCbYvuSW1JRPtU5Hfp5+cnOzY0NFQoD32Ss+e2nZ2d/hPRkOie4ePHj5cdu2/fPtmxlStXFspD5H0t8jltqUR+R2vWrNFLu8CLjXfkmjZtmlDb9ILQTkgff/wxSpUqhW+++QbLli3DsmXLCsVIkgRXV1fMnz8f7dq101miRERERGQehApQ4MW3iPfeew8bN27EoUOHcPnyZWRmZsLNzQ0+Pj5o27YtOnfujDJlyugjXyIiIiIyccIFKACUKVMGI0eOxMiRI3WdDxERERGZOdmTkIiIiIiIdIEFKBEREREZFAtQIiIiIjIoFqBEREREZFAsQImIiIjIoFiAEhEREZFBKSRJkko6CX06deoUlEolbt68qZf2K1WqpJd2s7KyhOKdnJxkx9ra2gq1LWfnJAcHB0iShMaNGwu1rQnRPrW2thZq38fHR3aslZX873BPnz4VyiMjI0N2bF5enlDbcnh6eiIvLw9NmzbVeduv0nefenl5yY4VfX8Yi+vXr782pkyZMsjLy0NgYKDe89H3Z6+rq6teYkuVEludUOR9LfKeBuS9rw35PgX0368iu3WVL19edqxovz58+FB2rOjugnIYul9LgkbrgJoShUIBGxsbVKtWraRTEWJM/xOUs82mUqmEQqEwQDam26f29vZC8aJbAuqaUqk02N+hqfapMfH29n5tjFKphI2NjQGysZw+FXlf6+M9bcj3KWA5/SqyvbQ+tqI2dL+WBLMfASUiIiIi48J7QImIiIjIoFiAEhEREZFBsQAlIiIiIoNiAUpEREREBsUClIiIiIgMigUoERERERkUC1AiIiIiMigWoERERERkUCxAiYiIiMigWIASERERkUGxACUiIiIig2IBSkREREQGxQKUiIiIiAyKBSgRERERGRQLUCIiIiIyKBagRERERGRQLECJiIiILFBiYiLq1KmDxMREWfHp6ekIDQ1FYGAg6tevj0GDBuHs2bManZsFKBEREZGFSU9Px5gxY/D8+XNZ8RkZGejTpw+OHj2Kvn37IjQ0FGlpaejbty9SUlKEz88ClIiIiMiC7N27F927d8etW7dkH7No0SLcvXsXa9aswahRozBw4EBs2LABNjY2+O6774RzYAFKREREZCEmTJiA4cOHo0yZMujQoYOsY/Ly8rBjxw4EBASgdu3aqucrVKiADh064PDhw8jIyBDKgwUoERERkYW4dOkSxowZg+joaFStWlXWMRcvXsSTJ0/g6+tb6LV69epBkiScOXNGKI9SQtFEREREZLI2b94MW1tboWPyRze9vLwKvVahQgUAwPXr14XaNPsCNCEhAZIkwcbGpqRTMWtKpRIKhQL169fX+7nYp4bBPjU/7FPzY8g+Jf26cOECcnNzCz2fkpKCpUuXFnncnj17hM4jWnwCQGZmJgDAwcGh0Gv5z2VnZwu1afYFqCRJUCqVuHbtmqx4hUIh1L6bm5vsWHd3d9mxWVlZQnk8evRIdmxOTo5Q25IkvTbGw8MDeXl5Qu1qSrRPrazE7jQpV66c7FhnZ2fZsaJvzocPH8qOFe1TOX3l6elpNn2a/w1dDkdHR9mx9+/fF8pDNF7XjLlPRT977e3tZceWLVtWduzTp0+F8rh3757sWLmzjUUYsk9Jv3Jzc5Gbm1ugQMzNzYWrq2sJZvVC/t9uce9T0c9lsy9AbWxscO3aNQQHB8uKF/1m8J///Ed27H//+1/ZsVu2bBHKo7hvR686fvy4UNtyiuGtW7cKtakN0T4tXbq0UPsiv8tPPvlEduxvv/0mlMeCBQtkxx47dkyobTmFUHR0tFCb2hDtUxcXF6H2IyIiZMd+9NFHsmO//vproTymT58uFK9rxtynoiOlgYGBsmPnzZsnOzYhIUEoj4kTJ8qOFf0SKoch+pSj2fqhbvTa1tYWderUUT1OTk6Gl5eX8Cinrjk5OQFQ/wUt/+9aZEAGsIAClIiIiDSnr9FsOzs72TmIXMUQKfTv3LkjK07OlUBRnp6ehYp6SZIKjGhLkiR8dUAfKlWqBABqZ7rnL+Wk7v7Q4rAAJSIioiKJjmZbW1vLigsICJCdw8aNG2XHilxtGj58uKw4fRSg0dHRqFatWqHnjfGWiqpVq8LJyUntTPf85/z8/ITa5DJMREREREbi+fPnqn/GwtbWFm3btkV8fDxSU1NVz9+6dQu7du3Cu+++izJlygi1qfcR0Bs3buD69eto1KiRvk9FREREZLKM5RJ8QkIC0tPT0axZM9XE3LFjx2Lv3r0YMGAAQkJCYGdnh3Xr1iEvLw9ffPGF8Dn0PgK6detW9O/fX9+nISIiIjJ5eXl5qn8lZdOmTZg4cSIuX76seq5ixYpYv349/Pz8sHTpUixcuBCVK1dGeHg4atWqJXwOvY+AOjs7C9+YSkRERGRp1I2A6tOYMWMwZsyYQs/Pnj0bs2fPLvR89erVhVaKKY5WI6ARERE4f/58sTEDBw7EX3/9pc1piIiIiCyCMYyAGoJWBehPP/2ks0qYiIiIyNJZSgGq1SX43Nxc1dpQ5qJp06ZC8SKLy798L8XriOzcAQAzZsyQHduvXz+hti9duvTaGH1fJtBG48aNheJFFpc/e/as7Ni7d+8K5aFueY6i7N+/X6htU9exY0eheJHF5UUWIu/QoYNQHiJ/L1FRUUJtm7ry5csLxYvMLRBZIDs8PFwoD30sLk+WS5KkArPfjfn/rdrSagQ0ODgYO3bswNWrV3WVDxEREZHF4gioDF5eXpAkCR07dkTNmjXh4+Ojdn9ehUKB7777TptTEREREZk9cy8882lVgL68v25ycjKSk5PVxrEAJSIisgxyb6eYMmWKXs6/fPly2bHGdonb0LPgS5JWBWhYWJhR7FFKREREZA44AiqDnH1cc3Nzce3aNW1OQ0RERGT2LGkEVKtJSHXq1MGiRYuKjVm4cKHQrGIiIiIiS8VJSGqkpqYWWEpGkiRcu3YNhw8fVhv/7NkzHDp0CEqlUrssiYiIiCzAy8swmTOhAjQtLQ1jxoxR3fepUCiwbds2bNu2rchjJElCixYttMuSiIiIyMy9egnenAkVoO+//z6GDBmiGgWNiopC7dq1UadOHbXxNjY28PLyQu/evbXPlIiIiMjMvVyAWltbl2Am+iU8Cenzzz9X/XdUVBTatGmD0aNHa5XErVu3EB0djSNHjuDKlSt49OgRcnNz4ejoCGdnZ7z55pto2LAhunXrJrxDEBEREZGpYAEqQ0pKitYJREREYM6cOXj69CkAwM7ODs7OznB1dUVubi7u3r2LGzduYP/+/Vi8eDG++uor9OrVS+vzEhERERkTS5oFr1UBmi8pKQl+fn4AXtw8u3z5chw7dgxvvPEGhgwZAm9vb7XHxcTEYPr06XjjjTcwcuRIBAYGwsPDo1BcRkYG4uPjsWTJEnzzzTcoX748WrdurYvUC6lQoYJQ/Pbt22XHihTOogv3BwcHy459/PixUNumTt3uXMU5c+aM7NgZM2bIjvX39xfK4/bt27Jjc3Nzhdo2daJ/wxMnTpQde+DAAdmxW7duFcrD09NTKN6SODo6CsVXrVpVduwXX3whO/bo0aNCeYiws7MTis/JydFTJmTMeA+oDI8fP8bQoUORkJCAQ4cOwd3dHXPnzsXq1ashSRLi4+MRExODLVu2qP3gXblyJTw8PBAZGQk3N7ciz+Ph4YGuXbuiZcuWCA4OxsqVK/VWgBIREZHmypQpIyuuSpUqstsUGWQ5deqU7Fi5bGxsZMdqu/KPpcyC12od0GXLluHUqVMIDAwE8OLb2saNG+Hu7o7w8HDMnDkTDx48wOLFi9Uef+HCBXz44YfFFp8vK1OmDNq3b4+LFy9qkzYRERGR0cm/BJ//j5fgixATEwN/f3+sWrUKABAXF4cnT56gZ8+eaNiwIRo2bIi4uLgiL2k5OTkhOztb6JyWdvmYiIiILIelXILXagT0xo0baNiwoerxwYMHoVAo8N5776meq1y5coHF61/m5+eHnTt3IjU1Vdb5Tp8+jZ07d6JBgwbapE1ERERkdDgCKpOTk1OByQ/x8fGwsbEpUCDevn0b7u7uao8PDQ3FoUOH0K1bN7Rv3x5NmjSBj48P3NzcYGtrC6VSiUePHiE9PR1HjhzBrl27YGVlhbFjx2qTNhEREZFRspQRUK0K0OrVq2Pfvn0YN24ckpKScOXKFbz77ruqmX5///03/vjjD7zzzjtqj69ZsybWrl2LqVOnYtu2bcXOKJckCdWqVcP06dNRt25dbdImIiIiMkosQGXo378/xo4di+bNmyMnJwcKhQJ9+/YFAKxZswY//fQTnj59ikGDBhXZRr169RAdHY1Tp07hxIkTqoXonzx5Ajs7O7i6uqJ69epo1KgRL70TERGR2eI6oDK9//77mDdvHpYvXw4A6NOnj2rf9ydPnsDFxQUzZsxQzZIvToMGDVhgEhERkUWzlGWYtF6I/sMPP8SHH35Y6Pn+/ftjxIgRUCgU2p6CiIiIyOxxBFQHnJ2d9dU0ERERkVniPaAyXbp0CRs2bEBaWhqUSqXaal2hUCAsLEzbUxnEwYMHheIjIyP1kofINnMAsH79etmxt27dEk3HpP37779C8XKXBQOAwYMHy461trYWymPv3r2yY728vITa/ueff4Tijc2OHTv0Fq/uik5RRHZHAcS2+bQ0ZcuWFYp3cXGRHSuyra3oVTsrK/mrGZYvX16obVN/n5JmWIDKcPjwYXz66ad49uxZsXFFvaFF9md+tT3RvdKJiIhI/+Tubvj06VPZbb685OPrVKxYUXZsqVLyyiCR7TVFBz1exkvwMi1evBh5eXkYM2YMgoKC4OLiIvTt8dSpU7h+/ToAsV8yC1AiIiIyRxwBleHs2bNo164dRo0apdHxO3fuRGhoKOLi4tCiRQuhy5lERERE5oaz4OUcXKqU0FD3q+zt7bFw4UL0798fBw4cwJAhQ9CoUSNtUiIiIiIySZZ0CV6rveDfeecdnDx5UqsEbGxs8OOPP8LBwQHffvutWf+yiYiIiIrz8l7w+nLnzh1MnjwZ7777Lvz9/dGrVy/Ex8fLOrZfv36oVauW2n8iE/60GgH97LPP0KtXL8yZMwdDhw6Fq6urRu14enpi6NChWLFiBfbv369azJ6IiIjIUhhiBDQrKwshISFIT0/HgAED4OHhgcjISAwZMgTLly9H8+bNiz0+NTUV77zzDnr27FnotdKlS8vOQ6sC9KeffkLFihWxatUqrFq1Cm5ubqp94F+mUCheu6TMsGHDMGzYMG3SISIiIjJp+p6EFB4ejtTUVCxduhStWrUCAAQHByM4OBjTp0/H77//XuSE8ps3b+LBgwdo2rQpOnfurFUeWhWgsbGxBR7fv39fq2SIiIiILJm+C9Bt27ahcuXKquITAJycnNCjRw/MnTsXSUlJ8Pf3V3vshQsXAAA1a9bUOg+tCtCUlBStEyAiIiKiF5fcX54Fr+tL8JmZmbhy5Qo6duxY6LV69eoBgKwC9M033wTw4nK+o6OjRtuuazUJSdcmTZqEPXv2qB7HxsZi0qRJJZgRERERkeHocxJSRkYGJElSu3tehQoVAEC1Prs6Fy5cgEKhwPr169GkSRM0aNAAjRo1wvTp0/HkyROhXHSyF/zp06cRGRmJ8+fPIzs7G25ubqhZsya6dOlSZBWtTlRUFLy9vdG6dWsAL0ZYo6OjMWvWLF2kKcuNGzf01naDBg1kx4oOb8+bN092rOj2gTk5OULxxubu3btC8SJ/Ax999JHs2L/++ksoD9G8STeCgoJkx968eVOo7eI+2C3dG2+8IRRfv3592bHff/+97Nj8ER65RIqETZs2CbXNrTgt06t/U//++y/GjRtXZPzLA3evk5mZCQBwcHAo9JqjoyMAIDs7u8jjL1y4AEmScOXKFUydOhUKhQJ79uxBeHg4kpOTsXbtWtm7S2ldgC5btgzz588vtHBqQkICNm/ejNDQUE4uIiIishCVKlWSFSfyJeLXX3+VHSuyFWZ+QfY6ixYt0sv5X6XvWfD5tZq6S+b55yrucvonn3yCp0+fFtg4qH379ihbtizCwsKwY8cOdOnSRVYuWhWg+/fvx7x58+Dp6Ylx48ahYcOG8PDwwKNHj3DkyBH8+OOPmD9/Pnx9fREYGKjNqYiIiIjM3qsjoF5eXkKjnMVxcnICADx9+rTQa/kjny4uLkUe36dPH7XP9+vXD2FhYYiPjzdMAbp69Wq4uLggPDy8wDeesmXLokOHDvDz80OXLl0QFhbGApSIiIioGPoeAc2v1TIyMgq9duvWLQBQe3/o65QtWxbAi0lJcmk1Cens2bMICgoqcri9UqVKCAoKQlJSkjanISIiIrII+pyE5OzsjCpVquDMmTOFXst/ztfXV+2x6enp6NChg9p5OZcvXwYgdi+3VgVoTk7Oa1e9L126tFBFTERERGSpnj9/rvqnDx07dsSlS5ewf/9+1XNZWVmIjIxE9erVVcsxvapSpUq4f/8+oqOjC4yg5uXlYcGCBVAoFOjUqZPsPLS6BO/j44OjR4/i+fPnsLIqXMvm5eXhyJEj8PHx0eY0RERERGbPEFtxhoSEYNu2bQgNDcXAgQNRvnx5REZG4saNG1i+fLlqElJCQgLS09PRrFkzlCtXDlZWVpgxYwZGjhyJnj17onfv3rC1tcXu3buRlJSEUaNGoW7durLz0GoE9MMPP0Rqaiq+/fZb5ObmFnjt8ePH+Prrr3Hp0iV8+OGH2pyGiIiIyCLo8xI88OIyfEREBFq3bo2IiAjMmTMHDg4OWLlyJZo2baqK27RpEyZOnKi6vA68WKZuxYoV8PHxwZIlSzB//nxIkoS5c+di7NixQnloNQL66aefIjY2Fhs3bsRvv/0GX19fuLi4ICMjA6mpqcjMzEStWrUKTNcXIUmSXqp/IiIiImOk7604AcDDwwNz584tNmb27NmYPXt2oeebN2+O5s2ba52DVgWonZ0dwsPDMWfOHGzfvh0HDhxQvebg4IAePXpg4sSJahc8VWft2rXw9vZWPe7atSsCAgK0SZGIiIjIJBjiEryx0HohemdnZ3zzzTeYMmUK/v77b2RmZsLZ2RnVqlV77Wr4T548Ua28DwCNGzcu8Lq3t3eBgjTf8ePH0ahRI21TJyIiIjIqhhgBNQZa7wUvSRLi4uKQnJyMGjVqoEGDBqhZsyZmzJjx2q0Hhw0bpnYx1OLO9dNPP2HgwIFaZk1ERERkXCRJKjALniOgRcjJyUFoaCji4uIwYsQI1dpR2dnZ2LhxIzZt2oROnTph1qxZamfJHz9+HCNHjsTSpUtha2tb7LkyMjIwYcIEnDx5UpuUdc7V1VV27JgxY2THqvt9FefVSWDFsba2Fmrb1N2/f19v8ZcuXZIde/XqVaE8Hj16pJdYSyQyM/P999+XHfvHH38I5SHyt+Xv7y/Utpz1lovbYq+kpaWlCcWLDF5cuXJFdqzIlosA0KtXL9mxIkvUAEBcXNxrY4yxT1+etFIckWWGzp49KztW3TqVRRk0aJCsuM6dO8tu888//5QVV1TfcQRUhrCwMOzbtw8tWrRA+/btVc87ODggKioK77//PrZv3441a9aoPd7X1xeHDx/G6NGjoVQqizzPnj170KlTJ5w4cQIuLi6YM2eONmkTERERGSV9z4I3FloVoNu3b8fbb7+NpUuX4s033yzwWp06dbBgwQLUrl0bW7ZsUXv8qlWr4OvriwMHDiA0NLTQLzs3NxczZszA6NGj8fDhQwQEBGD79u3o2LGjNmkTERERGZ38SUj5/8z5ErxWBej169eLnaWuUCjQpEkTXLt2Te3rzs7OWLlyJXx9fbF371589tlnqiH5q1evokePHoiIiICNjQ2+/PJLhIWFwdPTU5uUiYiIiIwWR0BlcHJywvXr14uNuXXrVoGZ7q/KL0L9/Pzw559/YuLEidiyZQu6du2KlJQU1KhRA1u2bEFISIg2qRIREREZNY6AytSwYUPs2bMH586dU/t6SkoKYmNj8c477xTbzstF6K5duzB16lQ8ffoUISEh2LJlC2rWrKlNmkREREQmQd97wRsLrXdC2rNnD/r164eePXvCz88PLi4uyMzMRFJSErZs2YK8vDwMHz78tW05OTlh5cqVGDJkCBISEtCyZUt8+eWX2qRHREREZFLM/dJ7Pq0K0Lp162LevHmYMmUKVq9eXWBJAUmS4OzsjLlz56qWZ3qd/CJ08ODBiIuLw6xZszBp0iRtUiQiIiIyCdwJScD777+Ppk2bqhajf/DgAZycnFC7dm20adMGzs7ORR57/Phxtc8PGzYMU6ZMwdq1ayFJktp1+bgTEhEREZkbjoAKcHJyQvv27QusBSpHv379il1EV5IkrFu3DuvWrSvwvEKhwPnz5zXKlYiIiMhYsQAV8OjRI+zevRvnz5/Hw4cPsWDBApw8eRIKhQINGjQo8jiOYhIREZmXoiYmv0rujkEAUKlSJdmxdnZ2smOdnJxkxRW3ms+rXrezYz51A3C8BC8gJiYGkyZNQlZWFiRJUv1C4+LisHz5coSEhGDixIlqj311ZNMYiG5rNnLkSNmxInvYp6enC+URHBwsO7ZevXpCba9fv/61MaJbhxqSaG5vv/227NjKlSvLjq1YsaJQHh4eHrJj//33X6G2TZ3I/wwAYObMmbJj/fz8ZMceOnRIKI/p06fLjq1SpYpQ22PHjn1tjDG/T5OTk4XiDx48KDu2QoUKsmPd3d2F8ihbtqzsWHt7e6G25RQyxrgVJ2nHUkZAtfo0On36NMaPHw87OzuMGzcOHTp0UL0WEBCAihUrYvXq1cL7JRMRERFZIktZhkmrAnTx4sVwdHTEr7/+imHDhqFq1aqq15o1a4ZNmzbBzc0N4eHhWidKREREZM64EL1MCQkJaNeuXZHbY5YrVw5t27bFpUuXtDkNERERkUWwlK04tboHNDs7u9hlloAXNwM/efJEm9MQERERmT1OQpLpjTfewMmTJ4t8XZIkHD9+HD4+PtqchoiIiMgimPvIZz6tLsG3b98eZ86cwU8//VSoSn/27Bm+//57pKSk4IMPPtAqSSIiIiJLwEvwMgwePBh//fUXlixZgs2bN8PGxgbAi52MkpOTcevWLdSqVQuDBw/WSbJERERE5kqSpAKz3835ErxWI6B2dnZYt24d+vXrh+zsbNy4cQOSJCEuLg6PHj1Cjx49EBERAQcHB13lS0RERGS2OAIqk4ODA/7zn//gq6++wtWrV/Hw4UM4OTmhatWqqkV0N2/ejO7du2udLBEREZE5M/fCM59GBWh6ejoOHTqEzMxM1KlTB82bN4eVlRWqV69eIO7atWuYPHkyjh8/zgKUiIjIAiiVSllxs2fPlt3mqlWrZMdu2LBBdmxiYqKsuMWLF8tu89mzZ7Li1F1e5yz4IkiShFmzZiE8PLzAL8XPzw+//PILXF1dVXErV67EwoUL8fTp09cu1UREREREHAFVKzIyEmvXroW9vT0++OADlClTBgcPHkRiYiKmTZuGH3/8Effu3cO4ceNw/PhxSJKEli1b4ptvvtFX/jqXP5FKLjs7O73kcfr0aaF4Jycn2bHlypUTalvOMlqlSml9N4feyP02mk9kv+YaNWrIjg0LCxPKIyEhQXaspa21a21tLRQv+r6Wq0GDBkLxInkfO3ZMqG0PDw+dnt/QcnJyhOJ/+OEH2bH/93//JztWdOc+uSNoALBkyRKhtuVsxWjOI2SWiCOgRdixYwdsbGywadMm1KpVCwDwxRdfYPz48fjzzz9x9epVjB49GpcvX4a7uzsmT56Mjh076iVxIiIiInNj7nvA5xOaBX/58mUEBQWpik8AUCgUGD58OPLy8lTFZ8uWLbFz504Wn0REREQCOAtejczMTFSuXLnQ81WqVAEAXLlyBUOHDsVnn32mk+SIiIiILIUlXYIXGgF99uyZammll+XfM/fWW2+x+CQiIiLSkCFGQO/cuYPJkyfj3Xffhb+/P3r16oX4+HhZx6anpyM0NBSBgYGoX78+Bg0ahLNnzwrnoNVC9K9q3ry5LpsjIiIishj5I6D5//QxApqVlYWQkBDs3LkTXbp0wRdffIHs7GwMGTIEBw8eLPbYjIwM9OnTB0ePHkXfvn0RGhqKtLQ09O3bFykpKUJ56HTqsr5mmhIRERFZAn3f+xkeHo7U1FQsXboUrVq1AgAEBwcjODgY06dPx++//w6FQqH22EWLFuHu3bvYunUrateuDQBo3749OnTogO+++w6rV6+WnYdOR0CJiIiISHP6vgS/bds2VK5cWVV8Ai+WcuzRowf+/vtvJCUlFZnXjh07EBAQoCo+AaBChQro0KEDDh8+jIyMDNl5CI+A7tmzB9evXxd6TaFQYObMmaKnIiIiohKmVCrh6emJ6OhoWfFFjZ69SmTN5YcPH8qOFVGhQgVZcSLzW0aMGCErzsPDo9CuUZIkFViGSdeX4DMzM3HlyhW1qxTVq1cPAJCUlAR/f/9Cr1+8eBFPnjyBr6+v2mM3bNiAM2fOyFqTGNCgAE1OTkZycrLQayxAiYiITJNCoYCNjQ2qVatW0qnonNxNVBwdHXV+bqVSqbZY1+cl+IyMDEiSBC8vr0Kv5RfjRQ0y5o9uanKsOkIF6KxZs0TCjYK+vrnlc3Nzkx175swZ2bHly5cXysPd3V12bN26dYXabtq06WtjypYta7A1y0T71MpK7E6TsmXLyo4V6dO2bdsK5REQECA7Vh8LF3t6erJPX0OfOwv5+fkJxc+bN++1Mcb8PhX97BUZPRP5e7l165ZQHiJ/W6GhoUJtf/rpp6+N8fDw0Huf1q9fX6/t0/94eHjg119/LfD433//xbhx44o8Zs+ePbLbz8zMBAA4ODgUei2/yM7OzhY+Nv+5oo5VR6gA7dKli0i4UTDVb276+LalKTnFrVKpNNgkNFPtU5EvCZrE65pSqVS77Jo+mGqf6msrXk3aLl269Gtj+D7VPZHPan2NohnqfUr6ld+PVatWLfB8enq6zs6RP1ih7gtf/uX+or4MFndsPpEve8a7gbeO8Jub+WGfmh/2qflhnxKJeXmXyZfVq1cP3bt318k5nJycAABPnz4t9Fr+6KWLi4vGxzo7O8vOhbPgiYiIiCxApUqVAEDtbPX820/U3eOp7bHqsAAlIiIisgDOzs6oUqWK2nvd859TN8sdeHFrgJOTU7HHity7zgKUiIiIyEJ07NgRly5dwv79+1XPZWVlITIyEtWrV1ctx/QqW1tbtG3bFvHx8UhNTVU9f+vWLezatQvvvvsuypQpIzsPhWTOO90TERERkcrjx48RHByMu3fvYuDAgShfvjwiIyNx8eJFLF++XLXyTUJCAtLT09GsWTOUK1cOAHDjxg106dIFVlZWCAkJgZ2dHdatW4d79+5hw4YNRd7Hqg4LUCIiIiILkpGRgTlz5mD//v1QKpWoXbs2QkND0aRJE1XMV199haioKKxdu7bAsoCXL1/GnDlzcOzYMVhbW8PX1xcTJkzAW2+9JZQDC1AiIiIiMijeA0pEREREBsUClIiIiIgMigUoERERERkUC1AiIiIiMigWoERERERkUCxAiYiIiMigWIASERERkUGxACUiIiIig2IBSkREREQGxQKUiIiIiAyKBSgRERERGRQLUCIiIiIyKBagRERERGRQ/w/Whi4d3C9SawAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 700x280 with 11 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "row_ids = [0, 1, 2, 3, 4]\n",
    "\n",
    "W = 1.4\n",
    "H = 1.4\n",
    "fig, axgrid = plt.subplots(nrows=2, ncols=len(row_ids), figsize=(5*W, 2*H))\n",
    "\n",
    "# Top row: original images (rows of x_tr_NF)\n",
    "for col_id, row_id in enumerate(row_ids):\n",
    "    cur_ax = axgrid.flatten()[col_id]\n",
    "    im_handle = cur_ax.imshow(x_tr_NF[row_id].reshape((8,8)),\n",
    "                              interpolation='nearest', vmin=0.0, vmax=1.0, cmap='gray')\n",
    "    if col_id == 0:\n",
    "        cur_ax.set_ylabel(\"Original\\n Img\")\n",
    "    cur_ax.set_title(\"Img. %d\" % (row_id))\n",
    "    cur_ax.set_xticks([]);\n",
    "    cur_ax.set_yticks([]);\n",
    "\n",
    "\n",
    "\n",
    "# Bottom row: reconstructed images (rows of xhat_tr_NF)\n",
    "\n",
    "for col_id, row_id in enumerate(row_ids):\n",
    "    cur_ax = axgrid[1].flatten()[col_id]\n",
    "    im_handle = cur_ax.imshow(xhat_tr_NF[row_id].reshape((8,8)),\n",
    "                              interpolation='nearest', vmin=0.0, vmax=1.0, cmap='gray')\n",
    "    if col_id == 0:\n",
    "        cur_ax.set_ylabel(\"Reconstr.\\n K=10\")\n",
    "    cur_ax.set_xticks([]);\n",
    "    cur_ax.set_yticks([]);\n",
    "plt.colorbar(im_handle, fraction=0.05);\n",
    "plt.tight_layout()\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Show more original and reconstructed images from training set: All 9s\n",
    "\n",
    "We'll look at *five other* training examples, selected because they are all 9s"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAqAAAAEDCAYAAADqXLYYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABEe0lEQVR4nO3dd1gU1/s28HtBehFsqGjsqFFBFLtGg71EsZfYUGNXNEajX/UXWyzRWGKNRrFhQey9Yi+xgC1gQ8VYsKIIKIjz/uG7G5EF5mxj2b0/15XrCrPPnHmGw67Pnpk5RyFJkgQiIiIiIgOxyOoEiIiIiMi8sAAlIiIiIoNiAUpEREREBsUClIiIiIgMigUoERERERkUC1AiIiIiMigWoERERERkUCxAiYiIiMigWIASERERkUGZXQFaunRplC5dOqvTkC0yMhLly5fHpk2b0o25cuUK+vfvj2rVqsHb2xt+fn7YsGEDUlJS0sSmpKQgKCgIfn5+8PLyQu3atdG/f39cu3YtVdzo0aNVv6uM/jMGxpSLHLrq03Pnzsnqoy1btqRqe+fOnWjfvj28vLxQo0YNDBgwAJcvX9bb+WrKXPtV6dWrV5gwYQLq1q2L8uXLo3bt2vjf//6HmJgYtW3v2bMHnTp1go+PD6pWrYoffvgB586d09n56QL7VH6f3rt3L8P39b59+3R6rpoy9z6NiorC8OHDUb16dVSsWBF+fn4ICgpSG/ul48ePo3Tp0jh9+rRW55Rd5cjqBCh9T548weDBg5GcnJxuzIEDBzB8+HDY2tqiadOmsLW1xZEjR/DLL7/g1q1bGD9+vCpWkiQMHz4c+/fvh4eHB7p06YKnT59i//79OHnyJNasWQNvb28AQIMGDeDu7q72mGfOnMHFixdRrVo13Z6wGdBln7q7u2Pw4MFq24iNjcXatWthb28PT09P1fbffvsNy5cvh729PRo1agRnZ2eEhoaiS5cumDp1Klq1aqXbEzYTun6vvnnzBp06dcK9e/dQpUoVNGvWDLdu3cLmzZtx4sQJhISEwM3NTRU/b948LFq0CAULFoSfnx+Sk5Oxd+9e9OjRA7/++ivatm2r1/M3RVndpxEREQCA+vXro2zZsmmOXbJkSR2erXnQdZ9evnwZ/v7+iI+PR5UqVVC+fHlERkZi0qRJOHHiBObPnw8rKyu1x4mMjMTIkSN1fo7ZimRmPDw8JA8Pj6xOI1Pnz5+X6tSpo8o3ODg4TcyzZ88kHx8fqVatWtLdu3dV2+Pj46XmzZtLHh4e0p07d1TbN2zYIHl4eEjDhg2TkpOTVdvPnj0reXh4SB06dMg0r+joaKlSpUpS7dq1pRcvXmh3kjpizn2ann79+kkeHh7Sjh07Uh3fw8NDqlq1aqo24uLipPbt20teXl7SgwcPtDtJHTLnfp03b57k4eEhTZkyJVU7f/75p+Th4SH973//U227deuW5OHhITVv3lyKj49XbX/48KFUpUoVqXLlytLbt291eMaaY5/K61NJkqTZs2dLHh4e0vnz53V7cjpmrn2akpIiNWrUSPLw8JD++uuvVO0sW7ZM8vDwkJYuXao2lwMHDkg+Pj6qXE6dOqW7E81GzO4SvLF7+/YtRo8ejW7duiEuLg4+Pj7pxm7evBlv3rzByJEjUbRoUdV2e3t7DB8+HO3atcPbt29V2wMDA+Hi4oJJkyYhR47/Br+rVauG3r17o0KFChleNpAkCSNHjsTbt28xadIk5MqVS7uTNRP67FN1Nm7ciNDQUDRu3BjfffedavuuXbsAAAMGDEDx4sVV2x0dHTFq1CgkJiZi7dq1Gp6l+dFnvypvifhy5LJjx44AgLCwMNW269evo0CBAujVqxfs7e1V2wsWLIgqVaogLi4Ot27d0upczYWx9CnwaQRUoVCgTJky2p6WWdNXn167dg337t1D2bJl0atXr1Tt9OrVC0WKFMGKFStS/ZsaExODAQMGYPDgwbC2tka5cuV0e7LZDAtQAN26dYOPjw9evnyJsWPHokaNGqhYsSI6deqES5cuAQA2bdqE5s2bw9PTE40bN8aqVasgSVKqdt6/f4/58+ejUaNGqriVK1di27Ztau/FU+fBgwfYunUr6tSpgx07dqB69erpxoaGhsLKygoNGzZM81r9+vXx66+/qi6/3rt3D3fv3kXt2rXh5OSUJn7UqFEYN24cLC0t0z3ejh07EBYWhvr16+Pbb7/N9Fyykjn0qTpv3rzBnDlzYGdnh7Fjx6bJAwAqVaqUZj/lP3Lnz5/P9Hyykrn0q6urKwDg4cOHqWKV9wp+/uWvVatWOHr0KNq0aZMq9uPHj7h37x4AIE+ePJmeT1Zhn6btU+BTAVqoUCE4OjpmmrexMYc+jY6OBgB4e3tDoVCkirWwsEDp0qXx8uVLREVFqbZfvXoVoaGhaNmyJXbu3AkPD49M8zdlvAf0//vw4QO6dOkChUIBPz8/REdH49ChQ+jduzdat26NrVu3omnTpqhRowa2b9+OqVOnwsXFRXXP3IcPH9C7d2+cP38eHh4e6Ny5M548eYIZM2agUKFCsvPIly8fNm7ciIoVK2YYJ0kSbty4AXd3d6SkpGD69OnYt28fXrx4gaJFi6Jbt27o0KGDKj4yMhIAUKpUKVy5cgXz58/HpUuXIEkSqlatiuHDh2d4I3lSUhJ+//13WFhY4KeffpJ9PlnJ1PtUncWLF+PVq1fo379/qvvJAKjuRUpKSkqzn/Jb/b///iv7vLKKOfRrp06dsHfvXkydOhVOTk6oUKEC7ty5g3HjxsHCwgL+/v7pHi8pKQm3b9/G4sWLcfv2bbRp00bovLIC+zR1n758+RJPnz5F5cqVMX36dBw+fBhPnjxBwYIF8d1336Fv376wtraW/wvOAqbep8rfv7rPUwCIi4sD8OkztVSpUgAADw8P7Nq1i/fv/n8sQP+/xMREuLi4YPXq1ao/rOHDh2PPnj0IDg7Gli1bVN9WfH194e/vjx07dqjeLEFBQTh//jwaN26M2bNnqy5x79+/H0OHDpWdR+7cuZE7d+5M496+fYuEhAQAnz7YXr9+DV9fX3z8+BGHDx/G+PHjERUVhdGjRwP471t2eHg45s+fj/Lly6N9+/a4d+8ejh49inPnziEwMDDdN+n27dsRExODxo0bp7p8a8xMvU+/FBcXhw0bNsDW1hY9evRI87qXlxdCQ0Oxd+/eNJeh9u/fr8rB2JlDv/r4+GD58uUYNWoUunXrptru4OCAP//8E998843aY71//z7VqFuDBg0wadIk2eeUVdinqfv0n3/+AQBcvHgRL1++hK+vLxITE1UPtpw5cwaBgYFGXYSaep9WqFABCoUCx48fx9u3b1ONVMfExKhuuVAWogDw1Vdfyc7bHPAS/Ge6d++e6g1duXJlAEC9evVSDZUrL2F+Plq0ZcsWWFpaYsyYManur2zcuLFenhZXvlHu3bsHhUKBHTt2YNKkSZgyZQq2b98Od3d3BAYGqi53KONDQ0Ph7++PjRs3YvTo0ViyZAlmzZqFhIQEjB49Gh8/flR7vNWrVwMAfvjhB52fiz6Zcp9+aePGjUhISEC7du3U3p/bsWNHuLi4YN26dZg7dy6ePHmC2NhYBAcHY+7cubCzs0tzCcxYmXq/Pn78GL///jtiYmJQt25d9OrVC/Xr10diYiJ++eUXVYHypbdv36Jr167o1q0bSpUqhUOHDqFXr16p/hE0VuzT//r07du3KFq0KDp16oRdu3ZhzJgxmDRpEnbv3o2aNWviwoULWLp0qc7PS9dMuU8LFCiAli1b4unTp+jbty+uXr2KhIQEhIeHY8CAAap2s8tnalZgAfqZYsWKpfpZeUN/4cKFU223tbUFANVUDklJSYiMjISbmxsKFCiQpt0qVaroPFcLi/+6buTIkakKjnz58qneAMoHT5T3dubLlw8BAQGp2mrRogUqV66Mu3fv4vr162mOdfnyZdy8eRPly5dHhQoVdH4u+mTKffqlzZs3A/j07V2dXLlyYdmyZcibNy8WL16MunXrolq1apg8eTJGjRqFwoULq34Pxs6U+1WSJPTr1091q8zSpUvx888/Y9GiRVi1ahWePXuGPn36qP7B/Fzu3Lkxfvx4jBs3Dtu2bYOfnx/+/vtvzJs3T+fnpWvs0//6tEmTJti/fz8mTpyYqviyt7fHL7/8AuDTfL7GzpT7FAAmTJiABg0a4OLFi2jXrh28vb3RsWNH5M2bV/Vgkp2dnc5zNRUsQD/z+ROkn8vsMserV68AAHnz5lX7+pf34unC5w8SqSsKy5cvD+C/G6WVlwfKli2rdl4y5dN4ygdVPnfgwAEAQLNmzbTM2vBMuU8/d+fOHURFRaFUqVKq+43U8fT0xP79+zFnzhz8+OOPmDJlCo4cOYLOnTvj6dOn6Z6vsTHlfg0PD8eNGzdQp04dNGrUKFVs1apV0aZNG7x48QJHjhzJ8Lg5cuTA2LFjoVAocOjQIW1PQ+/Yp5n3KQAULVoUzs7Oaj+rjY0p9ynw6fwWLlyIjRs34ueff8aYMWOwceNG/Pnnn6rbmbLLZ2pW4D2gOuDg4AAA6V7m0sd9dba2tihQoAAeP36sdlJd5Tblty/lN9H0JuD98OGDqt0vKT8UmzZtqn3i2UR26NPPHT58GMCnkZPM2NnZpfkycf/+fcTGxqJq1aqapJ5tZId+ffToEYD0JxpXXrpUxkVGRuL27dv49ttvVeen5OzsjJw5c+LFixe6ORkjZIp9evfuXTx58gSenp5p+vTjx494//49bGxsdHMyRig79OnnKlasmOb5iStXrsDCwoIPHGWAI6A64OjoiGLFiuH+/fuqb26fCw8P18txlcXCqVOn0rx25coVAFA92e7p6QlbW1uEh4cjPj5ebbxCoUjzJPzr168RFRWFwoULo2DBgro+BaOVHfr0c8o5BDOaYuTIkSOoWbMm1q1bl+a1PXv2AADq1q0rnnQ2kh36VTll0t27d9W2pZxaKV++fACAZcuWYcSIEQgNDU0T++jRI8TGxqJIkSLanYARM8U+nTFjBnr27Iljx46pbfvLh81MTXbo0w8fPqBJkybo3LlzmthHjx7h8uXLqFSpktppD+kTFqA60r59e6SkpGDGjBmq0UTg0x+yvi5/de7cGQqFAvPnz0+1lvDjx4+xZMkSWFtbq54odHBwQKtWrZCQkICpU6emujF606ZNuHbtGmrVqpVm+U3lGvFeXl56OQdjZux9+rlr164hR44cqstE6pQrVw6vXr3CunXrUk0dcvHiRSxbtgzu7u5o2bKlbk/ICBl7v1aqVAkFCxbE8ePH0xSV165dQ3BwMJycnFCvXj0AUPXZ/PnzU40YJSQkqJYNbN++vV7Oy1iYWp82b94cALBgwYJUo32vXr1SzWqgbqYLU2LsfZojRw64u7sjLCwMFy5cUMXGx8dj1KhRSElJSfUwEqXFS/A60q1bNxw4cABbt25FREQEqlWrhidPnuDQoUPImTMnXr58meomZ13w9vbG4MGDMX/+fHz33Xeqy6/79+9HbGwsxo8fn+pm75EjR+LKlSsICQlR5Xj37l2EhoYiT548mDBhQppj3L9/H0Dam8nNQXboUwB49+4dnj59Cnd39wwfInJzc8OAAQOwcOFCtGrVCvXq1cOrV6+wd+9eWFpa4vfffzfqaV10xdj71crKCjNnzsQPP/yAgQMHom7duihZsiQePHigutVCOWci8GnUukOHDggODkbTpk3RsGFDSJKE48eP4+HDh2jatGmqaX9Mkan1aYsWLXDgwAEcOHBA1adJSUk4evQonj17hp49e8LX11en52NsjL1PAWD06NFo164d+vTpg+bNm8PZ2RmHDx/G/fv30bt3b9SuXVun+ZkaFqA6Ym1tjeXLl2PhwoXYu3cv1q1bB3d3d4wbNw7R0dEIDAxM94ZsbQwePBhly5bFypUrsXPnTigUCnz99dfo06eP6tu0kpOTE9atW4e//voLu3fvxpo1a+Di4oK2bdtiyJAhap82fPnyJQAgf/78Os/d2GWHPgX+6yN1/felIUOGoGDBgggKCsKGDRvg7OyMRo0aYeDAgWbzJSM79KuPjw+2bNmCJUuW4NSpUzhx4gScnZ3h6+uLfv36pVnCb/LkyahYsSLWrVuHzZs3Q6FQwMPDAwMGDEC7du3SrNRiakytTxUKBebNm4f169cjJCQEISEhsLS0RJkyZTBmzBjVCKkpyw59WqpUKQQHB2POnDkIDQ1FUlISSpUqheHDh5vVMxOaUkicpEon/v33X7i6uqa5YRwARowYgV27dmHTpk0mfd+OqWGfmib2q+lhn5oe9qnp4z2gOjJt2jRUqlQpzTyaUVFROHLkCFxdXVXrbVP2wD41TexX08M+NT3sU9PHEVAdOXnyJPr27QtbW1s0atQI+fLlw+PHj3Ho0CG8f/8es2fPljVFDhkP9qlpYr+aHvap6WGfmj4WoDoUHh6OFStW4MqVK3j+/DlcXFzg7e2NXr16wdvbO6vTIw2wT00T+9X0sE9ND/vUtLEAJSIiIiKD4j2gRERERGRQLECJiIiIyKBMfh7QsLAwSJIEKyurrE7FpCUnJ0OhUBjkvhz2qWGwT00P+9T0GLJPAfaroRi6X7OCyRegkiQhOTkZDx480Ev7Im9CV1dX2bEZrWijzrt372THPn36VKhtOfLnz4+UlBSdt6uOaJ9aWloKtS/STyITIX++TKIc6tZANiRj7lNRIgspiKyu8uzZM6E8kpOTheJ1zZT6VGSN7dy5c8uOffHihVAeou9rXTNknwL671d9EflcB6B2/tH0iH4OvH//PtMYQ/drVjD5AtTKygoPHjyAn5+fXtovWrSo7Fh1S12mR92KNxk5evSo7NiePXsKtS3Htm3bdN5mekT7VLm8nVwi/STyd7Vjxw695aEPxtynokTOReTvZdCgQUJ53Lt3Tyhe10ypT0U+x+bOnauXWABYuXKlULyuGbJPAf33q76Ifp6KnN+YMWOE2g4PD880xtD9mhV4DygRERERGRQLUCIiIiIyKBagRERERGRQLECJiIiIyKBYgBIRERGRQbEAJSIiIiKDYgFKRERERAbFApSIiIiIDMrkJ6IXJTppuZwJZZVEJiwWnYQ2ICBAduywYcOE2o6NjRWKNzai5yuyuIDIZMWiiwuI/A2ILi6Q3ftUlMjvXmQictH3qcjfS1ZPWm/sRCYWF1moQ/TzQqSfRPKgzIn8PkU/80T6tUiRIkJti9QNpowjoERERERkULJHQM+cOaPxQWrUqKHxvkRERERkWmQXoP7+/lAoFBodJCIiQqP9iIiIiMj0yC5A/fz8NC5AiYiIiIiUZBeg06dP12ceRERERGQm9P4QkiRJ+j4EEREREWUjWk3D9OzZMxw8eBAvX77Ex48fVcWmJEn48OEDYmNjceLECRw7dkwnyRIRERFR9qdxAXrz5k106dIF8fHxkCRJdX/ol/9vZ2enm0yJiIiIyCRoXIAuXrwYb9++RbNmzVC9enXMnz8fX3/9Nb799lvcunULISEhcHJywr59+3SZLxERERFlcxoXoOfPn0eFChUwe/Zs1c9PnjxBp06dAACNGzeGv78/goKC0K9fP91kS0RERETZnsYPIcXGxqJSpUqqn0uVKoXr16+rfq5atSpq1qyJgwcPapchEREREZkUjUdA7ezsYGHxX/1aqFAhJCYm4tGjRyhYsCAAoEyZMggJCdE+SwMSXVNbZH13kTWGRdajBiD0oBfXAc+YyPre+lzTd86cObJjK1asKNS2ua1JLbJuuMjfi+jvUeQzQHRNcnMj8t7T59+7yGe16PvUHIl8/uqzX3/55RfZsX5+fnrLw5RpPAJaokQJXLlyRfVzsWLFIEkSIiMjVdtev36N9+/fa5chEREREZkUjQvQxo0b4+LFixg3bhxiYmJQqlQp5MmTB4sWLcKjR49w5coV7Nu3D0WKFNFlvkRERESUzWlcgH7//feoWrUqNm/ejNOnTyNHjhzo1asXrl27hvr166Njx4548+YNunXrpst8iYiIiCib0/geUGtra6xatQoHDx5E6dKlAQC9evWChYUFtm/fDhsbG7Rt2xZt2rTRWbJERERElP1ptRKSQqFAo0aNUm3r2bOn8IM8So8ePco0xtLSEnZ2dnB2dtboGERERESUtbQqQHXN19dXtYpSZuzs7FCzZk38/PPPKFy4sJ4zIyIiIiJd0aoA3bhxI9auXYv79+8jOTlZbYxCocA///wjq722bdvi3Llz+Pfff+Hs7IzSpUvDzc0Nb9++xbVr1/D8+XPkzJkT7u7uePHiBQ4dOoRLly5h69atcHNz0+ZUiIiIiMhANC5Ag4KCMGXKFEiSBCcnJ+TJk0frZBo3boytW7eia9euGDZsGBwdHVWvffz4EQsWLMDSpUsxZswYVKlSBdu3b8eYMWPw559/4v/+7/+0Pj4RERER6Z/GBeiGDRvg4OCAJUuWwMfHRyfJLFy4EN7e3hg3blya1ywsLDB06FCcO3cOc+bMwbp169CqVSvs2bMHx48f18nxiYiIiEj/NJ6G6f79+2jRooXOik8AuHHjBry9vTOM8fT0TLXkp4eHB549e6azHIiIiIhIvzQeAXV2dkaOHLp9hsnZ2Rk3btzIMObWrVuws7NT/ZyQkJDqUr2h3bt3T3asyDJsms4kQGmJLmkosryqyBJsRYsWFcrj/v37smPNbSlO0eVV9bVUXkBAgFD88OHD9ZKHOQoMDJQdK7K8oz65uLgIxZvCssmin3utWrXSS6yo169f661t+kTjEVBfX1+EhobqdKnN2rVr4+TJk1izZo3a1zdt2oSTJ0+iVq1aAD4Vn6GhoShevLjOciAiIiIi/dJ4CPPHH39EWFgY+vTpgx49eqBo0aKwsbFRGyt3mqShQ4fixIkTmDp1KtavXw8vLy/kzZsXcXFxCA8PR2RkJFxdXTFs2DAkJyejSZMmePbsGX766SdNT4OIiIiIDEzjArRRo0ZITk7GrVu3cOHChXTjRKZhyp8/PzZs2ICpU6fi6NGjiIqKStVO3bp1MXbsWBQuXBj379/Hu3fvMGDAADRr1kzT0yAiIiIiA9O4AFUuv6lr7u7uWLhwIWJjY3H9+nW8evUKDg4OKFeuHPLly6eKK1KkCP7++2+95EBERERE+qNxAZrefZq64uLiorrXk4iIiIhMh1EtxQkADx48wN69e/Hw4UMkJSWpjVEoFJg6daqBMyMiIiIiXZBdgM6bNw/Vq1dHtWrVVD/LoVAoMHToUFmxp06dwoABA5CcnAxJkjJskwUoERERUfYkuwBdvHgxLC0tVQXo4sWLoVAoMiwUAbECdO7cufjw4QP69euHSpUqwdbWVm56RERERJRNyC5Ap02bhrJly6b6Wddu376N7777TnjicCIiIiLKPmQXoK1bt87wZ11wdnZGzpw5dd4uERERERkPo3oIqUWLFti3bx+GDx+earlNQxJdsi08PFwveUyYMEEofs6cObJjzW05ONE+El3WUl9ElmMV7dPsTmQJXNF4kaUD5d4LrySyJKjIkrBA9n+fitq+fbvsWIVCobc8RP62RJdYnjt3rlC8uRH5GxBdfthY/h0wZRoXoPXr1880xtLSEnZ2dihQoABq1qyJzp07w8rKKt34oUOH4vr162jTpg26du2KQoUKwdraWm1sjRo1NE2diIiIiLKQxgWohYUFXr9+jTdv3gD4VGzmzp0b8fHxiI+PTxV748YNHDt2DHv27MHq1avTLSrfvHmD+Ph43L17F1OmTMnw+BEREZqmTkRERERZSOMCdOnSpejUqRM8PT0xatQoeHt7w9LSEgBw8+ZN/Pbbb4iIiEBgYCBsbGywbNkyhISEIDAwEP369VPb5uTJk3H16lXkz58fnp6ecHBw0DQ9IiIiIjJSGhegM2fORM6cObFmzRrY2Nikes3DwwMLFy5Ey5YtsXDhQsybNw9TpkxBREQE9uzZk24BevbsWXh6emLdunXIkcOobk8lIiIiIh2x0HTHc+fOoUGDBmmKTyUbGxvUqVMHp06dUm2rUqUK/v3333TbTElJQdWqVVl8EhEREZkwjQvQHDly4Pnz5xnGxMbG4uPHj6qfFQoFLCzSP2TFihVx/fp1TVMiIiIiomxA4wK0QoUK2L9/P65du6b29YiICBw8eBDly5dXbbtw4QIKFSqUbpsjR45EeHg4fvvtN7x8+VLT1IiIiIjIiGl8rXvQoEHo3r07unTpgtatW6NixYrImzcv4uLiEB4ejpCQEHz48AEDBw6EJEkYPHgwrl27hhEjRqTb5oIFC1CgQAEEBgYiMDAQTk5OsLe3TxOnUCgQGhqqaepERERElIU0LkC9vb0xf/58jB8/Hhs3bkRwcLDqNUmSkCtXLsyYMQPVq1fHkydPcPjwYVSpUgVdu3ZNt81Dhw6l+vnNmzeqaZ6IiIiIyDRo9bRPvXr1cPDgQZw8eRJXr17Fq1ev4ODggAoVKsDX1xe2trYAACcnJ2zbtg1lypTJsL3IyEht0iEiIiKibEDrx81tbW3RoEEDNGjQIN0YBweHTItPIiIiIjIPsgvQkJAQlCtXDmXLllX9LFe7du3EM8siomtMi6wbLbJetz7zEF3jVnQNXdINrkWcPtH3h+ga3PoybNgw2bGia8GLrDNPuiPy+WiOa8GLvlcVCoVe8pgwYYJQfHh4uF7yoP/ILkDHjRuHwYMHqwrQcePGZfqHIkkSFApFugVo9+7dBVL9j0KhwKpVqzTal4iIiIiyluwCdPDgwahWrVqqn7X1999/a7Sfvr4hEREREZH+CRWgn/vmm29QtmxZWFlZaXzw1atXa7wvEREREWVPGj+ENHjwYHz99ddYsmSJxgevWrWqxvsSERERUfak8UpIr169QsmSJXWZCxERERGZAY0LUB8fH5w5cwbJycm6zIeIiIiITJzGl+B79OiBcePGoVmzZvD19UXhwoVVE89/KTtNw0RERERE+qVxAdq/f3/V/69atUrtk+mZTcNEREREROZH4wJ00KBBnA6JiIiIiIRpXIAOGTJEl3kQERERkZnQai34pKQkxMTE4NWrV3B1dYWbmxusra11lVu2ULduXdmxEydOlB0rsmQfAMTGxsqOFV2yj0tx6obIcqkAEBAQIDvW1dVVMBvKCiJLK86ZM0eobTl/X9rM22xsRN5PIr930c9HkSWWRT6nSbdEP39FlxAlcRoVoOfOnUNQUBCOHj2a6in4HDlyoF69eujSpQtq1KihsySJiIiIyHQIFaCSJOHXX39FUFAQJEmCjY0NihYtChcXF8THx+P+/fs4ePAgDh06hM6dO2Ps2LGwtLTUV+5ERERElA0JFaBz587F2rVr4e7ujh9//BG+vr6ws7NTvZ6cnIxjx45h9uzZWL9+Pezt7fHTTz/pPGkiIiIiyr5kT0R/8+ZNLF26FOXLl8fOnTvRvHnzVMUn8On+ogYNGmDLli2oWLEiVqxYgcjISJ0nTURERETZl+wCdN26dbC0tMS8efNgb2+fYaytrS1+//13WFlZYdOmTVonSURERESmQ3YBevHiRdSqVQvu7u6y4gsWLIhatWrh3LlzGidHRERERKZHdgH68OFDFC9eXKjx4sWL49GjR8JJEREREZHpkl2AWlhYQJIkocZTUlLMbl5QIiIiIsqY7ALU3d0dly9fFmr8ypUrKFSokHBSRERERGS6ZBegtWrVQnh4uOwiNCwsDBcvXoSvr6/GyRERERGR6ZFdgPbo0QPW1tYYMWIE/v333wxj7969i2HDhsHe3h4dO3bUOkkiIiIiMh2yJ6J3c3PD+PHjMW7cOLRu3RpdunRB3bp1UaJECTg5OSExMRFRUVE4cOAAgoKCkJCQgHnz5iF37tz6zD/LHTt2THZseHi47Nht27YJ5ZEzZ07Zsf7+/kJtm5sJEybIjhVZN1pkzWjRtrnGdMZ69uypl1hRIn8D27dvF2pbztrVny+dnN15eXnJjm3VqpXsWNE1wEU+q0U/10l3RD8j+Zmqf0IrIbVr1w45cuTAxIkTsXTpUixdujRNjCRJyJkzJ+bOnYvGjRvrLFEiIiIiMg1CBSjwaVTmm2++wYYNG3D69GncuXMHcXFxcHFxQeHChdGoUSO0atUKuXLl0ke+RERERJTNCRegAJArVy4MHDgQAwcO1HU+RERERGTiZD+ERERERESkCyxAiYiIiMigWIASERERkUGxACUiIiIig2IBSkREREQGxQKUiIiIiAxKIUmSlNVJ6NOlS5eQnJyMJ0+e6KV9S0tL2bEiq0LZ29sL5WFhIf+7xLNnz4TajouLyzQmf/78SElJQc2aNYXa1oS++9TV1VV2rIODg+xYkT4CgBcvXsiOjY+PF2pbDlPqUycnJ73EihL5G/jw4YNQ23J+d6bUpyLvPTc3N9mxor/3hIQE2bGi79PExMRMYwzZp4D++1Vf8uTJIxQvsmrY69evRdPJlKH7NStoNA9odqJQKGBlZYXixYtndSpGI2/evDqPT05OhrW1taYpCTGXPhX5R1Mf2KdZS/R3L+d3xz7NXI4cYv8sOjs76yVWLkP2KZB9+1Wf9LHkuKH7NSuY/AgoERERERkX3gNKRERERAbFApSIiIiIDIoFKBEREREZFAtQIiIiIjIoFqBEREREZFAsQImIiIjIoFiAEhEREZFBsQAlIiIiIoNiAUpEREREBsUClIiIiIgMigUoERERERkUC1AiIiIiMigWoERERERkUCxAiYiIiMigWIASERERkUGxACUiIiIig2IBSkRERGSGwsPDUbZsWYSHh8uKj46ORkBAAGrUqAFvb2/06tUL165d0+jYLECJiIiIzEx0dDSGDBmCjx8/yoqPiYnB999/j3PnzqFr164ICAjA/fv30bVrV0RGRgofnwUoERERkRkJDQ1F+/bt8fTpU9n7LFy4EC9evMDKlSsxaNAg9OzZE+vXr4eVlRVmzJghnAMLUCIiIiIzMWLECPTv3x+5cuVC8+bNZe2TkpKCnTt3olq1aihTpoxqe758+dC8eXOcOXMGMTExQnmwACUiIiIyE7dv38aQIUOwbds2FCtWTNY+t27dQkJCAjw9PdO8VqFCBUiShKtXrwrlkUMomoiIiIiyrU2bNsHa2lpoH+XoZoECBdK8li9fPgDAw4cPhdo0+QI0LCwMkiTBysoqq1MxacnJyVAoFPD29tb7sdinhsE+NT3sU9NjyD4l/bpx4waSkpLSbI+MjMSSJUvS3e/w4cNCxxEtPgEgLi4OAGBnZ5fmNeW2xMREoTZNvgCVJAnJycl48OCBXtoX+XDNmzev7FgLC7G7I+Lj42XHKv+Q5EpJSck0xs3NTVacLoj2qUKhEGrf1dVVdqy6N2N6Pnz4IJRHQkKC7FjRN7659amjo6PsWCcnJ9mxIn0EAK9fv5YdK0mSUNtyfiem1Kci/ZQzZ07ZsaKfj7GxsULxupY/f36D9SnpV1JSEpKSklIViElJSUJ/v/qifFI+o/epaN1i8gWolZUVHjx4AD8/P720X7RoUdmxK1eulB0r+gd34MAB2bHLly8XalvOU3Jbt24ValMbon0q+m1v4sSJsmMbNGggOzY6Olooj3379uklFpDXp1u2bBFqUxuifZojh9hHV5cuXWTH+vv7y47du3evUB5z5syRHZucnCzUtpwCbtu2bUJtakO0T0UL0H79+smOHTp0qOzY4OBgoTw0efpXlwzRpxzN1g91o9fW1tYoW7as6ueIiAgUKFBAeJRT1xwcHAAA7969S/OacgBE5Is+YAYFKBEREWlO31cS5bCxsZEd6+bmJjtW7hyYL1++lN2muiJNnXz58qUp6iVJSjWiLUmS8JczfShUqBAAqH3SXTmgoe7+0IywACUiIqJ06ftKohw+Pj6yY0VGseXeFjNr1izZbR4/flxWXHBwsNqrqMZ4S0WxYsXg4OCg9kl35TYvLy+hNjkNExEREZGR+Pjxo+o/Y2FtbY1GjRrh1KlTuHnzpmr706dPsXv3btSpUwe5cuUSalPvI6CPHj3Cw4cPUaVKFX0fioiIiCjbMpZL8GFhYYiOjkatWrWQJ08eAJ/upQ4NDUWPHj3g7+8PGxsbrFmzBikpKRg5cqTwMfQ+ArplyxZ0795d34chIiIiyvZSUlJU/2WVjRs3YtSoUbhz545qW8GCBbFu3Tp4eXlhyZIlWLBgAYoUKYK1a9eidOnSwsfQ+wioo6Oj8I2pREREROZG3QioPg0ZMgRDhgxJs3369OmYPn16mu0lSpTIcE5SEVqNgAYFBeGff/7JMKZnz544cuSINochIiIiMgvGMAJqCFoVoH/88YfOKmEiIiIic2cuBahWl+CTkpJUc0OZity5cwvFi0zqHhUVJTv28/su5OjVq5fs2F27dgm1/eTJk0xj9H2ZQBsi03cAYhORz5w5U3as6D0yTZo0kR176NAhobazu/LlywvF16lTR3bshQsXZMdaWloK5SG6UogIY5grUBuiU7j88MMPsmNFFuqoXr26UB49e/aUHSuyGAmZJ0mSUj39bsz/tmpLq09DPz8/7Ny5E3fv3tVVPkRERERmiyOgMhQoUACSJKFFixbw8PBA4cKFYWtrmyZOoVBk+XJlRERERMbO1AtPJa0K0NmzZ6v+PyIiAhEREWrjWIASERHR50qWLCk7tm/fvrJjr1y5Iju2YMGCsuJcXV1lt6kNQz8Fn5W0KkBXrVqV7e87IiIiIjIWHAGVoVq1apnGJCUl4cGDB9ochoiIiMjkmdMIqFYPIZUtWxYLFy7MMGbBggXo3LmzNochIiIiMgt8CEmNmzdv4sWLF6qfJUnCgwcPcObMGbXxHz58wOnTp5GcnKxdlkRERERm4PNpmEyZUAF6//59DBkyRHXfp0KhwPbt27F9+/Z095EkCXXr1tUuSyIiIiIT9+UleFMmVIA2bNgQffr0UY2Cbt26FWXKlEHZsmXVxltZWaFAgQLo0qWL9pkSERERmbjPC1DRxS6yE+GHkH766SfV/2/duhUNGjTA4MGDtUri6dOn2LZtG86ePYuoqCi8efMGSUlJsLe3h6OjI0qWLAkfHx+0bdtWeKUiIiIiouyCBagMkZGRWicQFBSEmTNn4t27dwAAGxsbODo6ImfOnEhKSsKLFy/w6NEjHD9+HIsWLcLo0aPRqVMnrY9LREREZEzM6Sl4rQpQpcuXL6vW8f348SOWLVuGv//+G1999RX69OkDd3d3tfsdPHgQkydPxldffYWBAweiRo0acHNzSxMXExODU6dOYfHixZg4cSLy5s2L+vXr6yL1NAoXLiwUn5CQIDt25MiRsmNr164tlEeePHlkx8bHxwu1nd2J/G4AICwsTHZsaGio7NicOXMK5SEySbPolYF///1XKN7YiI4KvH37Vnbsw4cPZceWK1dOKI8KFSrIjr148aJQ29md6N/wkydPZMeePXtWdqxIHwFAhw4dZMfu3r1bqO3nz59nGsO5uE0P7wGV4e3bt+jbty/CwsJw+vRpuLq6YtasWQgMDIQkSTh16hQOHjyIkJAQ5M+fP83+y5cvh5ubG4KDg+Hi4pLucdzc3NCmTRvUq1cPfn5+WL58ud4KUCIiItKcs7OzrLjRo0fLbrNevXqyY9evXy87Vu4KR69fv5bd5ocPH2TFpTe6aS5PwWs1D+jSpUtx6dIl1KhRAwDw/v17bNiwAa6urli7di2mTp2K2NhYLFq0SO3+N27cQNOmTTMsPj+XK1cuNGvWDLdu3dImbSIiIiKjo7wEr/yPl+DTcfDgQVSsWBErVqwAABw7dgwJCQno2LEjfHx84OPjg2PHjuHEiRNq93dwcEBiYqLQMUUupRERERFlJ+ZyCV6rEdBHjx7Bx8dH9fPJkyehUCjwzTffqLYVKVIk1eT1n/Py8sKuXbtw8+ZNWce7cuUKdu3ahUqVKmmTNhEREZHR4QioTA4ODkhKSlL9fOrUKVhZWaUqEJ89e5buPRYBAQE4ffo02rZti2bNmqF69eooXLgwXFxcYG1tjeTkZLx58wbR0dE4e/Ysdu/eDQsLCwwdOlSbtImIiIiMkrmMgGpVgJYoUQJHjx7FsGHDcPnyZURFRaFOnTqwsbEBANy7dw/79+9H5cqV1e7v4eGB1atXY/z48di+fTt27NiR7rEkSULx4sUxefJk4SdPiYiIiLIDFqAydO/eHUOHDkXt2rXx/v17KBQKdO3aFQCwcuVK/PHHH3j37h169eqVbhsVKlTAtm3bcOnSJVy4cEE1EX1CQgJsbGyQM2dOlChRAlWqVOGldyIiIjJZnAdUpoYNG2L27NlYtmwZAOD7779XrfuekJAAJycnTJkyRfWUfEYqVarEApOIiIjMmrlMw6T1RPRNmzZF06ZN02zv3r07BgwYwElyiYiIiGTgCKgOODo66qtpIiIiIpPEe0Blun37NtavX4/79+8jOTlZbbWuUCiwatUqbQ9lEHKWPvvc9evXZccOHDhQdqzyXlq5RJYPtLDQavatbOfRo0dC8c+ePZMdK7KSR4ECBYTy+Prrr2XHlilTRqjt8PDwTGOM+Zu3SB8BQMGCBWXHZnTP+pfkrvii5ODgIDt2woQJQm3Lmc7OmPtUdHnY27dvy45t27at7FiRFXdEKZesluvQoUOZxhhzn5JmWIDKcObMGfzwww+ZLjuV3mX4UaNGaXRchUKBGTNmaLQvERER6U/x4sVlxYl82RO5na9Tp06yY+V+KRQ5vjZLcfISvEyLFi1CSkoKhgwZAl9fXzg5OQl10qVLl1QjdyK/ZBagREREZIo4AirDtWvX0LhxYwwaNEij/Xft2oWAgAAcO3YMdevWRe/evbVJh4iIiChb41PwcnbOkUPo3qov2draYsGCBejevTtOnDiBPn36oEqVKtqkRERERJQtmdMleK2eRqlcuTIuXryoVQJWVlaYM2cO7OzsMGnSJJP+ZRMRERFl5PO14PXl+fPnGDt2LOrUqYOKFSuiU6dOOHXqlKx9u3XrhtKlS6v9T+QBUa1GQH/88Ud06tQJM2fORN++fZEzZ06N2smfPz/69u2Lv/76C8ePH1dNZk9ERERkLgwxAhofHw9/f39ER0ejR48ecHNzQ3BwMPr06YNly5ahdu3aGe5/8+ZNVK5cGR07dkzzmsjMIFoVoH/88QcKFiyIFStWYMWKFXBxcVGtA/85hUKB0NDQDNvq168f+vXrp006RERERNmavh9CWrt2LW7evIklS5bg22+/BQD4+fnBz88PkydPxr59+9J9oPzJkyeIjY1FzZo10apVK63y0KoA/XKOslevXmmVDBEREZE503cBun37dhQpUkRVfAKfpqPq0KEDZs2ahcuXL6NixYpq971x4wYAwMPDQ+s8tCpAIyMjtU6AiIiIiD5dcv/8KXhdX4KPi4tDVFQUWrRokea1ChUqAICsArRkyZIAPl3Ot7e312jZdaNaEmfMmDE4fPiw6udDhw5hzJgxWZgRERERkeHo8yGkmJgYSJKkdmW+fPnyAch4ZcUbN25AoVBg3bp1qF69OipVqoQqVapg8uTJSEhIEMpFJ2vBX7lyBcHBwfjnn3+QmJgIFxcXeHh4oHXr1ulW0eps3boV7u7uqF+/PoBPI6zbtm3DtGnTdJGmLDExMULxIsvBKc9LjtjYWKE8HB0dZceWKFFCqG05S/wZs1u3bgnFjx07VnbsgwcPZMdaW1sL5XHmzBnZsa6urkJtZ3ePHz8Wit+wYYPs2Ddv3siOTUpKEsrD19dXdqxyNEKu7P4+FXkvAcCvv/4qO1bdswnp+euvv4TyWLt2rezY8uXLC7UtZylOMj1fFp6PHz/GsGHD0o3/fOAuM3FxcQAAOzu7NK/Z29sDABITE9Pd/8aNG5AkCVFRURg/fjwUCgUOHz6MtWvXIiIiAqtXr0aOHPJKS60L0KVLl2Lu3LlpJk4NCwvDpk2bEBAQwIeLiIiIzITcqXgWLlwou00LC/kXbJVFlhxyB8kM9YyLvp+CV9Zq6i6ZK4+V0eX0zp074927d6kWDmrWrBly586NVatWYefOnWjdurWsXLQqQI8fP47Zs2cjf/78GDZsGHx8fODm5oY3b97g7NmzmDNnDubOnQtPT0/UqFFDm0MRERERmbwvR0ALFCggNMqZEQcHBwDAu3fv0rymHPl0cnJKd//vv/9e7fZu3bph1apVOHXqlGEK0MDAQDg5OWHt2rUoVKiQanvu3LnRvHlzeHl5oXXr1li1ahULUCIiIqIM6HsEVFmrqbvd8OnTpwCg9v7QzOTOnRvAp4eS5NLqIaRr167B19c3VfH5uUKFCsHX1xeXL1/W5jBEREREZkGfDyE5OjqiaNGiuHr1aprXlNs8PT3V7hsdHY3mzZurfS7nzp07AICvvvpKdi5aFaDv37/PdNZ7Z2dnoYqYiIiIyFx9/PhR9Z8+tGjRArdv38bx48dV2+Lj4xEcHIwSJUqk+wBkoUKF8OrVK2zbti3VCGpKSgrmzZsHhUKBli1bys5Dq0vwhQsXxrlz5/Dx40e1NwinpKTg7NmzKFy4sDaHISIiIjJ5hliK09/fH9u3b0dAQAB69uyJvHnzIjg4GI8ePcKyZctUDyGFhYUhOjoatWrVQp48eWBhYYEpU6Zg4MCB6NixI7p06QJra2vs2bMHly9fxqBBg1CuXDnZeWg1Atq0aVPcvHkTkyZNSjMdydu3b/F///d/uH37Npo2barNYYiIiIjMgj4vwQOfLsMHBQWhfv36CAoKwsyZM2FnZ4fly5ejZs2aqriNGzdi1KhRqsvrwKep5P766y8ULlwYixcvxty5cyFJEmbNmoWhQ4cK5aHVCOgPP/yAQ4cOYcOGDdi7dy88PT3h5OSEmJgY3Lx5E3FxcShdunSqx/VFSJKkl+qfiIiIyBjpeylOAHBzc8OsWbMyjJk+fTqmT5+eZnvt2rVRu3ZtrXPQqgC1sbHB2rVrMXPmTOzYsQMnTpxQvWZnZ4cOHTpg1KhRaic8VWf16tVwd3dX/dymTRtUq1ZNmxSJiIiIsgVDXII3FlpPRO/o6IiJEydi3LhxuHfvHuLi4uDo6IjixYtnOht+QkKCauZ9AKhatWqq193d3VMVpErnz59HlSpVtE2diIiIyKgYYgTUGGi9FrwkSTh27BgiIiJQqlQpVKpUCR4eHpgyZQqOHDmS4b79+vVTOxlqRsf6448/0LNnTy2zJiIiIjIukiSlegqeI6DpeP/+PQICAnDs2DEMGDBANXdUYmIiNmzYgI0bN6Jly5aYNm2a2qfkz58/j4EDB2LJkiWZrpMdExODESNG4OLFi9qknKnk5GSheJE120XWo1Y3R1dGBg4cKDv2y5HmzOzfvz/TmIyW7spqr1+/1mu8XKLrhu/cuVN2rMha16ZA9H26detW2bEi7+nQ0FChPBo3biw7Vu5qIkrbtm3LNMaY36cJCQl6i5e7NjUgvia9unvk0lOmTBmhtuUsP2mMffr8+XNZcb/99pvsNpUTncvRpk0b2bFXrlyRFSf3nHSBI6AyrFq1CkePHkXdunXRrFkz1XY7Ozts3boVDRs2xI4dO7By5Uq1+3t6euLMmTMYPHhwhv+gHD58GC1btsSFCxfg5OSEmTNnapM2ERERkVHS91PwxkKrAnTHjh0oX748lixZgpIlS6Z6rWzZspg3bx7KlCmDkJAQtfuvWLECnp6eOHHiBAICAtL8spOSkjBlyhQMHjwYr1+/RrVq1bBjxw60aNFCm7SJiIiIjI7yISTlf6Z8CV6rAvThw4cZPqWuUChQvXr1dC9pODo6Yvny5fD09ERoaCh+/PFH1cz/d+/eRYcOHRAUFAQrKyv8/PPPWLVqFfLnz69NykRERERGiyOgMjg4OODhw4cZxjx9+jTVk+5fUhahXl5eOHDgAEaNGoWQkBC0adMGkZGRKFWqFEJCQuDv769NqkRERERGjSOgMvn4+ODw4cO4fv262tcjIyNx6NAhVK5cOcN2Pi9Cd+/ejfHjx+Pdu3fw9/dHSEgIPDw8tEmTiIiIKFvQ91rwxkLrlZAOHz6Mbt26oWPHjvDy8oKTkxPi4uJw+fJlhISEICUlBf3798+0LQcHByxfvhx9+vRBWFgY6tWrh59//lmb9IiIiIiyFVO/9K6kVQFarlw5zJ49G+PGjUNgYGCq6SAkSYKjoyNmzZqlmp4pM8oitHfv3jh27BimTZuGMWPGaJMiERERUbbAlZAENGzYEDVr1lRNRh8bGwsHBweUKVMGDRo0gKOjY7r7nj9/Xu32fv36Ydy4cVi9ejUkSULDhg3TxHAlJCIiIjI1HAEV4ODggGbNmqWaC1SObt26ZTiJriRJWLNmDdasWZNqu0KhwD///KNRrkRERETGigWogDdv3mDPnj34559/8Pr1a8ybNw8XL16EQqFApUqV0t2Po5hERESm5f3797LiRFadcnJykh375s0b2bE3btyQFRcVFSW7TW3wEryAgwcPYsyYMYiPj4ckSaoRzWPHjmHZsmXw9/fHqFGj1O775chmduTq6io79svJ+jNy7NgxoTzy5csnO1b0D1rOcnDGLKNpwNTp0qWL7Ni///5bdqydnZ1QHiVKlJAdK7LMqznKbCaOz7m7u+stj/Lly8uOFZ3z2NbWNtMYY1y2UVPdunWTHSty3hcuXBDKo169erJjnzx5ItS2nCV2TalP6RNzGQHVqrK4cuUKhg8fDhsbGwwbNgzNmzdXvVatWjUULFgQgYGBstYSJyIiIjJ35jINk1YF6KJFi2Bvb4/NmzejX79+KFasmOq1WrVqYePGjXBxccHatWu1TpSIiIjIlHEiepnCwsLQuHHjdC8V5cmTB40aNcLt27e1OQwRERGRWTCXpTi1ugc0MTExw2mWgE/3sCQkJGhzGCIiIiKTx4eQZPrqq69w8eLFdF+XJAnnz59H4cKFtTkMERERkVkw9ZFPJa0uwTdr1gxXr17FH3/8kaZK//DhA3777TdERkaiSZMmWiVJREREZA54CV6G3r1748iRI1i8eDE2bdoEKysrAJ9WMoqIiMDTp09RunRp9O7dWyfJEhEREZkqSZJSPf1uypfgtRoBtbGxwZo1a9CtWzckJibi0aNHkCQJx44dw5s3b9ChQwcEBQUJz39IREREZI44AiqTnZ0d/ve//2H06NG4e/cuXr9+DQcHBxQrVgzW1tYAgE2bNqF9+/ZaJ0tERERkyky98FTSqACNjo7G6dOnERcXh7Jly6J27dqwsLBIs3LLgwcPMHbsWJw/f54FKBEREWnEzc1Ndmz16tVlxx46dEhWnKEmhedT8OmQJAnTpk3D2rVrU/1SvLy88OeffyJnzpyquOXLl2PBggV49+5dplM1ERERERFHQNUKDg7G6tWrYWtriyZNmiBXrlw4efIkwsPDMWHCBMyZMwcvX77EsGHDcP78eUiShHr16mHixIn6yj/LhYaGyo7t2LGj7Njdu3cL5SFnHWilwMBAobaz+5shOTlZKL5MmTKyY4cMGSI7Njo6WiiP48ePy47NaDo0dSwtLTONMeY1pi0sxG5fF1mDvWXLlrJjBw0aJJSHyP3ws2bNEmpbTp+aEjnrpCv17dtXdmzu3LmF8nj+/Lns2BUrVgi1Ledz3ZjfpySOI6Dp2LlzJ6ysrLBx40aULl0aADBy5EgMHz4cBw4cwN27dzF48GDcuXMHrq6uGDt2LFq0aKGXxImIiIhMjamvAa8kNIxw584d+Pr6qopP4NO3r/79+yMlJUVVfNarVw+7du1i8UlEREQkgE/BqxEXF4ciRYqk2V60aFEAQFRUFPr27Ysff/xRJ8kRERERmQtzugQvNAL64cMH1dRKn1Pep/L111+z+CQiIiLSkCFGQJ8/f46xY8eiTp06qFixIjp16oRTp07J2jc6OhoBAQGoUaMGvL290atXL1y7dk04B60mov9S7dq1ddkcERERkdlQjoAq/9PHCGh8fDz8/f2xa9cutG7dGiNHjkRiYiL69OmDkydPZrhvTEwMvv/+e5w7dw5du3ZFQEAA7t+/j65duyIyMlIoD60nov+ccilOIiIiIhKn73s/165di5s3b2LJkiX49ttvAQB+fn7w8/PD5MmTsW/fvnRnV1i4cCFevHiBLVu2qGaMadasGZo3b44ZM2YIzbKj0xFQIiIiItKcvi/Bb9++HUWKFFEVnwDg4OCADh064N69e7h8+XK6ee3cuRPVqlVLNV1hvnz50Lx5c5w5cwYxMTGy8xAeAT18+DAePnwo9JpCocDUqVNFD0VERERZLDk5Gfnz58e2bduyLAeROXSdnZ1lx3bo0EFWXJMmTWS3KVf+/PnTzFMtSVKqaZh0fQk+Li4OUVFRamcpqlChAgDg8uXLqFixYprXb926hYSEBHh6eqrdd/369bh69arsVauEC9CIiAhEREQIvcYClIiIKHtSKBSwsrJC8eLFszoVnXNwcJAVlydPHp0fOzk5We2lbn1ego+JiYEkSShQoECa1/LlywcA6Q4yKkc3NdlXHaECdNq0aSLhRkHf39xE7nt1cnKSHSu60ovIt6TRo0cLtT1s2LBMY9zc3Aw2Z5lon4quFJIrVy6hXOTKmzevUB7NmzeXHVu3bl2htuX0lSn1qchywPb29rJjc+QQ+w7/4cMH2bH+/v5CbcsZycmXL5/R9qkokVEukb+X169fC+Uh8lk9cOBAobZ79eqVaUzevHn13qfe3t56bZ/+4+bmhs2bN6f6+fHjxxn+O3z48GHZ7cfFxQFQP6Ks/OxLTEwU3le5Lb191RH69GzdurVIuFEw5W9umlL37UVbycnJaqfo0ofs2qeivx+RQkikaJaLfZq1RL+wyIlnn+qeyJcQuZcmRRiyT0m/lP1YrFixVNtFl3HOiPLyvrovZcqBrPS+sGW0r5LIFzKdPgVvjPjNzfSwT00P+9T0sE+JxHy+yuTnKlSogPbt2+vkGMpbDt69e5fmNeXoZXpXa+XsK3K1iU/BExEREZmBQoUKAYDap9WfPn0KIP2rpNrsqw4LUCIiIiIz4OjoiKJFi+Lq1atpXlNuU/eUO/Dp1gAHB4cM9/Xy8pKdCwtQIiIiIjPRokUL3L59G8ePH1dti4+PR3BwMEqUKKGajulL1tbWaNSoEU6dOoWbN2+qtj99+hS7d+9GnTp1hJ5HUEimvNI9EREREam8ffsWfn5+ePHiBXr27Im8efMiODgYt27dwrJly1CzZk0AQFhYGKKjo1GrVi3VNFSPHj1C69atYWFhAX9/f9jY2GDNmjV4+fIl1q9fn+59rOqwACUiIiIyIzExMZg5cyaOHz+O5ORklClTBgEBAahevboqZvTo0di6dStWr16NatWqqbbfuXMHM2fOxN9//w1LS0t4enpixIgR+Prrr4VyYAFKRERERAbFe0CJiIiIyKBYgBIRERGRQbEAJSIiIiKDYgFKRERERAbFApSIiIiIDIoFKBEREREZFAtQIiIiIjIoFqBEREREZFAsQImIiIjIoFiAEhEREZFBsQAlIiIiIoNiAUpEREREBsUClIiIiIgM6v8BF1HaoB9gqbMAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 700x280 with 11 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "row_ids = np.hstack(np.flatnonzero(y_tr_N== 9)[-5:])\n",
    "W = 1.4\n",
    "H = 1.4\n",
    "fig, axgrid = plt.subplots(nrows=2, ncols=len(row_ids), figsize=(5*W, 2*H))\n",
    "\n",
    "# Top row: original images (rows of x_tr_NF)\n",
    "for col_id, row_id in enumerate(row_ids):\n",
    "    cur_ax = axgrid.flatten()[col_id]\n",
    "    if col_id == 0:\n",
    "        cur_ax.set_ylabel(\"Original\\n Img\")\n",
    "    cur_ax.set_title(\"Img %d\" % (row_id))\n",
    "    im_handle = cur_ax.imshow(x_tr_NF[row_id].reshape((8,8)),\n",
    "                              interpolation='nearest', vmin=0.0, vmax=1.0, cmap='gray')\n",
    "    cur_ax.set_xticks([]);\n",
    "    cur_ax.set_yticks([]);\n",
    "\n",
    "\n",
    "\n",
    "# Bottom row: reconstructed images (rows of xhat_tr_NF)\n",
    "\n",
    "for col_id, row_id in enumerate(row_ids):\n",
    "    cur_ax = axgrid[1].flatten()[col_id]\n",
    "    if col_id == 0:\n",
    "        cur_ax.set_ylabel(\"Reconstr.\\n K=10\")\n",
    "    im_handle = cur_ax.imshow(xhat_tr_NF[row_id].reshape((8,8)),\n",
    "                              interpolation='nearest', vmin=0.0, vmax=1.0, cmap='gray')\n",
    "    cur_ax.set_xticks([]);\n",
    "    cur_ax.set_yticks([]);\n",
    "plt.colorbar(im_handle, fraction=0.05);\n",
    "plt.tight_layout()\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Discussion 2b: How good are these reconstructions?\n",
    "\n",
    "Across the two plots above, what do you notice?\n",
    "\n",
    "Remember that we are storing only 10 values for each image, not 64, which is a 6x savings!\n",
    "\n",
    "Given that, do you think the reconstructions are pretty good? What visual aspects aren't covered very well?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 2c: Show reconstructions for the first five *test* examples\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Step 1: Compute the low-dimensional embedding for the test set x_te_TF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z_te_TK = None # TODO transform x_te_TF to obtain z_te_TK by calling pca.transform"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xhat_te_TF = np.zeros((T,F)) # TODO reconstruct the data by multiplying weights times z, plus adding the mean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAqAAAAEDCAYAAADqXLYYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA2X0lEQVR4nO3de1hN+f4H8PculW7KrQtCzLgMKskkl2FyG5dnlBkjlyQcDFE4gzk4PzPM4HBMZtwOxyVqjBjlOjPUuIbGJbdRci0TYkYRlXa1fn942kfaZa19b+/363l6Hnut7/quz+7TzqfvWuv7lQmCIICIiIiISEfM9B0AEREREZkWFqBEREREpFMsQImIiIhIp1iAEhEREZFOsQAlIiIiIp1iAUpEREREOsUClIiIiIh0igUoEREREekUC1AiIiIi0ikWoCpq2bIlWrZsqe8wqvTHH39g5syZ6N69Ozw9PTFo0CDExsbqOyyDVR1y+qq0tDS0bdsWO3bs0HcoBqs65PT06dMYN24c3n33XbRt2xb+/v5YsGABHj9+rO/QDFJ1yOnJkycREhICX19feHt7Y+TIkTh48KC+wzJo1SGvrxIEAaNGjULLli2RkZGh73CqJRagRuqPP/5AUFAQDhw4AF9fX4wYMQIFBQWYN28eFi9erO/wSE0PHjxAWFgY5HK5vkMhNezatQujR4/G2bNn0b17dwQHB8PZ2RnR0dH4+OOP8ejRI32HSBL9+OOPCA0NRVpaGj744AMMHjwYd+/exZQpU/Ddd9/pOzzSkC1btiA5OVnfYVRrNfQdAGnH119/jUePHmHdunXo3r07AGDq1KkICQnB5s2bMXDgQLRt21bPUZIqzp49i+nTpyM7O1vfoZAanj59ioULF8LGxgY7d+5Es2bNFPtWrFiB1atXY9myZViyZIkeoyQpnjx5gvnz56N+/fqIi4tD/fr1AQDh4eEIDAzEmjVrEBgYiEaNGuk5UlLHrVu3sHz5cn2HUe1xBNQI3b9/H7/++iu8vb0VxScA1KxZEzNmzIAgCNi+fbseIyRVPHv2DLNnz0ZwcDDy8vLg4+Oj75BIDceOHcPz588xZMiQcsUnAEyaNAmWlpY4fPiwnqIjVVy9ehXOzs4ICgpSFJ8AYG9vD39/f5SUlODixYt6jJDUVVJSglmzZqF27dp455139B1OtcYCVIOCg4Ph4+ODx48fY86cOfDz84OXlxeCgoJw/vx5AMCOHTswYMAAeHh4oG/fvoiKioIgCOX6efHiBb777jv06dNH0W7z5s2Ij49Hy5YtsWvXrirjOH/+PARBgJ+fX4V93t7esLKywunTpzX3xo2YoeQUAO7evYu4uDh069YNe/bsQadOnbTyno2doeS0efPmmDZtGvr27Vthn7m5OWrUqIH8/HzNvXEjZig59fPzQ0JCAsLCwirsu3nzJgCUK0ypaoaS11etXbsWly5dwldffQU7OzuNvl9Tw0vwGlZcXIzhw4dDJpMhICAAmZmZSEhIwNixYxEYGIi4uDj069cPfn5+2L17N77++ms4Ojpi0KBBiuPHjh2LM2fOoEWLFhg2bBgePHiAJUuWiL5sU/aLrnHjxhX21ahRA66ursjMzERRUREsLS019+aNlCHkFACcnJywfft2eHl5aemdmg5DyGnr1q3RunVrpfuOHz+O/Px8tGnTRmPv2dgZQk5fJ5fLkZmZiaioKJw4cQKdOnVCx44dNfm2jZ4h5fXq1atYs2YNhg4dii5dumDt2rXaeMsmgwWohhUUFMDR0RFbtmxRFHfTpk3DgQMHEBsbi127dqFFixYAAH9/f4SGhmLPnj2KD0tMTAzOnDmDvn37Yvny5ahR42WKfvnlF0ydOlVUDE+fPgUAODg4KN1fq1YtlJaW4tmzZ6hTp45a79cUGEJOAaBu3bqoW7euht+daTKUnCqTl5eHr7/+GgAwbNgwtfoyJYaY0x49euDPP/8EALRv3x6rVq2CTCZT962aFEPJa1FREWbNmgUnJyfMmjVLw+/SNPESvBaMGjWq3Mhihw4dALz8ZVT2QQFeXg4HXj6xXmbXrl0wNzfH559/rvigAEDfvn3h6+sr6vxll+0qG90s215UVCSqP9J/TknzDDGnz549w/jx43Hnzh289957+Pjjj1XuyxQZUk5LS0vRq1cvjBkzBp6enkhJScGwYcP48KAKDCGvkZGRuH79OhYtWgRbW1uV3wv9DwtQLXB3dy/32sbGBgDg5uZWbnvNmjUBQDGVTlFREdLS0uDs7AxXV9cK/Yq9dGNlZVWu39eVFZ5lcdGb6TunpHmGltNHjx5h1KhROH/+PLy8vPDNN99wtEwiQ8qpmZkZvvjiC8yaNQuxsbGYOHEi0tPTMX/+fMl9mTp95/XcuXPYtGkTRo4cyUEDDWIBqgWVFXZvut8yJycHQOU3qTs7O4s6f9ml97JL8a97+vQpZDIZb6CWQN85Jc0zpJxeu3YNQ4YMwe+//w4/Pz9s2LCBn08VGFJOXzd16lTUrl0bR48e5fy9Eukzr/n5+Zg9ezbc3NwwY8aMN7Yn8XgPqAEpG9bPy8tTuv/Zs2ei+mnevDkAIDMzs8K+4uJi3L9/H+7u7jAz498f2qapnJLh0HROT506hbCwMDx79gwffvghvvrqKz4cqGOayumdO3eQmpqKDh06wMnJqdw+c3NzNGjQADk5OcjNzeXT8DqgibxevnxZ8X9pZQ+A9unTBwCQmJjIOV4lYAFqQOzs7ODu7o6MjAzk5OSgdu3a5fZfuHBBVD8dO3aETCZDcnJyhelAzp07hxcvXijuoSHt0lROyXBoMqdnz57FxIkTUVhYiIkTJ2LatGkajpbE0FROd+3ahf/85z+YPn06JkyYUG5ffn4+bt++DXt7ez78qSOayGvDhg2VTqsFAHFxccjKykJwcDAcHBxQq1YtTYRtMjgEZmCGDBmCkpISLFmyBMXFxYrtSUlJSEhIENWHi4sLunTpgt9++63cMYWFhYrVG4YPH67ZwKlSmsgpGRZN5DQnJwcREREoLCxEeHg4i08900ROBw4cCDMzM2zcuBH37t1TbJfL5ViwYAHy8/MxePBgmJubazx+Uk7dvDZq1AhTpkxR+tWwYUMAL+crnTJlCgtQiTgCamCCg4Nx8OBBxMXFITU1Fb6+vnjw4AESEhLg4OCAx48fi7p0PnfuXAQFBWHq1Kno168fnJ2dkZiYiDt37mDs2LFcwUGHNJVTMhyayOnmzZvx6NEj1KpVCyUlJZWuEz558mT+fOiAJnLaokULhIeH45tvvsGHH36IDz74ADVr1sTJkydx8+ZN+Pj4YPr06Tp6RwTw968hYwFqYCwtLbFhwwasWrUKP/30E77//ns0bNgQc+fORWZmJjZt2iTq6XV3d3ds374dkZGROHHiBF68eIGmTZti4cKFnNpFxzSVUzIcmshpcnIygJcPBa5cubLSdp9++in/g9QBTX1OJ06ciLfeegubN2/G/v37UVJSgqZNm2LmzJkIDg7m/b06xt+/hksmvL5mFenVH3/8gdq1ayudZ2zGjBnYt28fduzYAQ8PDz1ER6pgTo0Pc2p8mFPjxLwaLv5ZbWAWLVoEb29v/P777+W237p1C7/++itq166NVq1a6Sk6UgVzanyYU+PDnBon5tVwcQTUwJw4cQLjx49HzZo10adPHzg5OeH+/ftISEjAixcvsHz5cnzwwQf6DpMkYE6ND3NqfJhT48S8Gi4WoAbowoUL2LhxIy5duoQ///wTjo6OaN++PcaMGYP27dvrOzxSAXNqfJhT48OcGifm1TCxACUiIiIineI9oERERESkUyxAiYiIiEinjH4e0JSUFAiCAAsLC32HYtTkcjlkMplO7qdhTnWDOTU+zKnx0WVOAeZVV3SdV30w+gJUEATI5XLcvXtX36HA2tpadFtnZ2dJfRcVFYlu++oScZri4uKCkpISjferjLZzKmWZvNfXFq5KzZo1JcXx6rJxb/LgwQNJfYthTDlVNgdgZerXry+6bVZWlqQ45HK5pPaaZkw5bdq0qVb6zc/Pl9Q+JydHdFtt5F+XOQW0n1d7e3vRbevWrSu6bWlpqaQ4pHy2tfH913Ve9cHoC1ALCwvcvXsXAQEB+g4FPXr0EN02Pj5eUt8XLlwQ3VYb3wup8apD2zl1dHQU3Xb+/Pmi20rJP/ByAmWxmNOqDRo0SHTbqKgo0W2lxnvnzh1J7TXNmHKam5urlX6lfo+k/A7QRv51mVNA+3kdPXq06LaRkZGi20r9efH399da32LoOq/6wHtAiYiIiEinWIASERERkU6xACUiIiIinWIBSkREREQ6xQKUiIiIiHSKBSgRERER6RQLUCIiIiLSKRagRERERKRTRj8RvbZpa4JrBwcHSXFoa1JmUyRlcmMpk9ZLzZGUny0pcagSS3V39OhR0W2lfPakTJoNSJu0nKomJU9SJk2XuvShlN8XhrAgiqGLiIgQ3VbKz4DU/1O1+budXuIIKBERERHplOgR0FOnTql8Ej8/P5WPJSIiIiLjIroADQ0NhUwmU+kkqampKh1HRERERMZHdAEaEBCgcgFKRERERFRGdAG6ePFibcZBRERERCZC6w8hCYKg7VMQERERUTWi1jRMjx49wqFDh/D48WOUlpYqik1BEFBcXIzc3FwcP35c0hQoRERERGTcVC5A09PTMXz4cDx//hyCICjuD33939bW1pqJlIiIiIiMgsoF6Jo1a/Ds2TP0798fnTp1wnfffYd33nkH77//Pq5fv46dO3fC3t4eP//8sybjJSIiIqJqTuUC9MyZM2jXrh2WL1+ueP3gwQMEBQUBAPr27YvQ0FDExMRgwoQJmomWiIiIiKo9lR9Cys3Nhbe3t+L122+/jd9//13x+t1330Xnzp1x6NAh9SIkIiIiIqOi8giotbU1zMz+V782atQIBQUFuHfvHho0aAAAaNWqFXbu3Kl+lDokda3m//u//9NOIBJJWY+YqiZlvWYpbbkOuP5IWas5IyNDdNsePXpID4Y0YsWKFaLbTps2TXRbqeu1X7hwQXRbqT8vR44ckdTeGIwePVp0Wym5MpT/q+l/VB4Bbd68OS5duqR47e7uDkEQkJaWptj25MkTvHjxQr0IiYiIiMioqFyA9u3bF+fOncPcuXORnZ2Nt99+G/Xq1cPq1atx7949XLp0CT///DOaNGmiyXiJiIiIqJpTuQAdMWIE3n33Xfz44484efIkatSogTFjxuDKlSvo2bMnhg4diqdPnyI4OFiT8RIRERFRNafyPaCWlpaIiorCoUOH0LJlSwDAmDFjYGZmht27d8PKygofffQRBg8erLFgiYiIiKj6U2slJJlMhj59+pTbNnr0aEk3ERMRERGRadH6WvBERERERK9SawR0+/btiI6ORkZGBuRyudI2MpkMV69eVec0RERERGREVC5AY2JisHDhQgiCAHt7e9SrV0+TcRERERGRkVK5AP3hhx9ga2uLtWvXwsfHR5MxEREREZERU7kAzcjIwODBg7VSfMbHx7+xjZmZGWxsbODi4oJWrVqhRg217iYgIiIiIh1RuWqrVauW1oq+2bNnQyaTiW5fp04dzJ8/H71791b73FKf4N+9e7fa51Rm0KBBktrfuXNHK3GYIilL6x0+fFh0W6k/K0+ePJHUnirXtGlT0W2lLJ4hdaENLy8v0W2l/ByaIinfn/DwcNFtc3JyJMUh5XNqir+npXz2AGmfEW0uhStlmU8xg2avMsWfA2VUfgre398fhw8f1spSm4sXL4abmxsEQcC7776LyZMn48svv8TMmTPRu3dvmJmZoU6dOhg9ejT69++P/Px8TJs2rdzSoERERERkmFQewpw+fTpSUlIwbtw4hISEoGnTprCyslLa1s3NTVLfubm5yMrKwnfffVdhVDM0NBRJSUkYP348mjRpglmzZuHWrVv4+OOPsWHDBqxYsULVt0REREREOqByAdqnTx/I5XJcv34dZ8+erbSdKtMwbdu2DX369Kn0knqXLl3Qs2dPREVFISgoCM2aNUOvXr1w6tQpSechIiIiIt1TuQAtW35TGx48eAB/f/8q27i6upa7/87V1ZX3zBERERFVAyoXoFu3btVkHOW4uLjg5MmTKC0thZlZxdtUS0pKkJycjPr16yu2PXz4kHOREhEREVUDBrkU58CBA3Ht2jXMmDED2dnZ5fY9efIEc+fOxbVr19CvXz8AwNWrV3Hw4EG0a9dOH+ESERERkQSiR0BXrFiBTp06wdfXV/FaDJlMhqlTp0oKasKECUhOTsZPP/2EQ4cOwcnJCU5OTsjLy0NGRgaKi4vh5eWFyZMnIz8/H4MHD0aNGjUwduxYSechIiIiIt0TXYCuWbMG5ubmigJ0zZo1kMlkEAShyuNUKUAtLS0RFRWF6Oho7Nq1C9euXcO9e/cAAO7u7hgyZAiCg4NhYWGBrKws9OrVC8OGDYOHh4ek8xARERGR7okuQBctWoTWrVuXe61N5ubmCAkJQUhICIqKipCbmwtbW1vY2tqWa9ewYUOsXLlSq7EQERERkeaILkADAwOrfK1NlpaWcHJy0tn5iIiIiEh7DHYB9b1792L37t3IyspCUVGR0jYymQwJCQkaPa/UZcOkiIiIEN2WS3HqjzaXd5NCSk5zc3O1FoepkbKYhZTl+gAuxalJmzdvFt1WymdJyu9pQNryzab4OZW6TKWnp6d2ApHom2++Ed1Wat0g9WfMWKlcgPbs2fONbczNzWFtbQ1XV1d07twZw4YNg4WFxRuP27JlCxYtWvTG+0uJiIiIqPpRuQA1MzPDkydP8PTpUwAvi826devi+fPneP78ebm2165dw9GjR3HgwAFs2bIFlpaWVfb9/fffo1atWoiMjIS3t3elS3wSERERUfWj8jyg69atg0wmg4eHB6Kjo3Hp0iUcO3YM586dw549e9C1a1fUrVsXe/bswS+//IKPPvoIFy5cwKZNm97Y9/379zFo0CD4+fmx+CQiIiIyMioXoEuXLoWDgwO2bt0KHx8fmJubK/a1aNECq1atgp2dHVatWoUmTZpg4cKFaNOmDQ4cOPDGvt3c3JCfn69qaERERERkwFQuQJOTk9GrV69KRyitrKzQrVs3JCUlKbZ17NgRf/zxxxv7DgkJwYEDB3Dz5k1VwyMiIiIiA6XyPaA1atTAn3/+WWWb3NxclJaWKl7LZDKla7u/bsiQIUhLS0NAQADef/99uLm5Kb1vVJVJ7omIiIhIv1QuQNu1a4dffvkFo0aNQtu2bSvsT01NxaFDh8pNqXD27Fk0atTojX2npKQgLi4OcrkcBw8erLQdC1AiIiKi6kflAnTy5MkYNWoUhg8fjsDAQHh5eaF+/frIy8vDhQsXsHPnThQXF2PSpEkQBAFhYWG4cuUKZsyY8ca+//WvfyE/Px/9+/dHhw4dYGNjo2qYRERERGRgVC5A27dvj++++w7z5s3D9u3bERsbq9gnCALq1KmDJUuWoFOnTnjw4AESExPRsWNHjBw58o19p6WloXfv3li+fLmq4RERERGRgVJrJaQePXrg0KFDOHHiBC5fvoycnBzY2tqiXbt28Pf3R82aNQEA9vb2iI+PR6tWrUT1a21tjcaNG6sTGhEREREZKLWX4qxZsyZ69eqFXr16VdrG1tZWdPEJAO+//z6OHDmCiIgIUSsnEREREVH1IboA3blzJ9q0aYPWrVsrXov18ccfSwpqxowZGDFiBEaPHo3g4GA0bdoUtra2Stu6ublJ6luftLnOvJS+uW68fkhZB5w0S1trgUtdr11K31LWOqeqHTlyRHRbqZ9TKWvBR0ZGSurbGEj9jEghZd34adOmSepbyucvNzdXUt/0kugCdO7cuQgLC1MUoHPnzoVMJqvyGEEQIJPJJBegH3zwAeRyOW7fvo3z589X2k4mk+Hq1auS+iYiIiIi/RJdgIaFhcHX17fca21p2bKl1vomIiIiIv2SVIC+6r333kPr1q21co/m1q1bNd4nERERERkGlZfiDAsLw5QpUzQZCxERERGZAJUL0JycHLz11luajIWIiIiITIDK0zD5+Pjg1KlTkMvlal+GL3uwSSo+hERERERU/ahcgIaEhGDu3Lno378//P394ebmpph4/nVvegpeEASVYlD1OCIiIiLSH5UL0IkTJyr+HRUVpXRKJrHTMKWlpakaBhERERFVMyoXoJMnT37jPKBERERERK9TuQDlE/BEREREpAq11oIvKipCdnY2cnJyULt2bTg7O8PS0lJTsZEEjo6O+g7BYPXo0UNS+/nz54tuK2WJP6lLsWpzCTtTI+V7LyX/UnMqZelAqT+3YlhbW2u8T2MTHx8vqb2U5TVNcSlOKUuVSm0v5bNqit97Q6dSAZqcnIyYmBgcOXIEcrn8f53VqIEePXpg+PDh8PPz01iQRERERGQ8JBWggiDgq6++QkxMDARBgJWVFZo2bQpHR0c8f/4cGRkZOHToEBISEjBs2DDMmTMH5ubm2oqdiIiIiKohSQVoZGQkoqOj0bBhQ0yfPh3+/v7lLunI5XIcPXoUy5cvx7Zt22BjY4O///3vGg+aiIiIiKov0SshpaenY926dWjbti327t2LAQMGVLifyMLCAr169cKuXbvg5eWFjRs3coolIiIiIipHdAH6/fffw9zcHCtWrICNjU2VbWvWrIl///vfsLCwwI4dO9QOkoiIiIiMh+gC9Ny5c+jSpQsaNmwoqn2DBg3QpUsXJCcnqxwcERERERkf0QVoVlYWmjVrJqnzZs2a4d69e5KDIiIiIiLjJboANTMzk7z2eklJCecFJSIiIqJyRBegDRs2xMWLFyV1funSJTRq1EhyUERERERkvEQXoF26dMGFCxdEF6EpKSk4d+4c/P39VQ6OiIiIiIyP6AI0JCQElpaWmDFjBv74448q296+fRsRERGwsbHB0KFD1Q6SiIiIiIyH6InonZ2dMW/ePMydOxeBgYEYPnw4unfvjubNm8Pe3h4FBQW4desWDh48iJiYGOTn52PFihWoW7euNuOvVg4fPiy6bXh4uKS+mzRpIrqtqa0xLmW9dkDaWsQRERGi24aEhEiKIzQ0VFJ7qpyUNdsDAgJEt3VwcJAejEhSfl+IdfnyZY33qS+bN28W3VZK/h0dHSXFcefOHUntqWo9evQQ3VbKzwAZHkkrIX388ceoUaMGvvjiC6xbtw7r1q2r0EYQBDg4OCAyMhJ9+/bVWKBEREREZBwkFaDAy9GB9957Dz/88ANOnjyJmzdvIi8vD46OjnBzc0OfPn0waNAg1KlTRxvxEhEREVE1J7kABYA6depg0qRJmDRpkqbjISIiIiIjJ/ohJCIiIiIiTWABSkREREQ6xQKUiIiIiHSKBSgRERER6RQLUCIiIiLSKRagRERERKRTMkEQBH0HoU3nz5+HXC7HgwcP9B0KbG1tRbd1dnaW1Hd2drbots+fP5fUtxguLi4oKSlB586dNd7367SdUwsLC9FtpayEU6tWLUlxPHr0SHTbvLw8SX2LYUw5tba2Ft1WymfPzKx6/Q1fVFQEmUwGb29vrZ9L2zl1cnIS3bZGDfEzDkrNaXFxsei22vhe6PJzChhWXuVyuei2OTk5qoSjN7rOqz6oNA9odSKTyWBhYYFmzZrpOxStklqwappcLoelpaVOzmUqOa1fv75W2orFnBofmUwGmUyms3OZQk6lfEa08b3Q5ecUqL55rV27tr5DkETXedUHox8BJSIiIiLDUr2uHxERERFRtccClIiIiIh0igUoEREREekUC1AiIiIi0ikWoERERESkUyxAiYiIiEinWIASERERkU6xACUiIiIinWIBSkREREQ6xQKUiIiIiHSKBSgRERER6RQLUCIiIiLSKRagRERERKRTLECJiIiISKdYgBIRERGRTrEAJSIiIiKdYgFKREREZIIuXLiA1q1b48KFC6LaZ2ZmIjw8HH5+fmjfvj3GjBmDK1euqHRuFqBEREREJiYzMxNTpkxBaWmpqPbZ2dkYMWIEkpOTMXLkSISHhyMjIwMjR45EWlqa5POzACUiIiIyIYcPH8aQIUPw8OFD0cesWrUKf/31FzZv3ozJkydj9OjR2LZtGywsLLBkyRLJMbAAJSIiIjIRM2bMwMSJE1GnTh0MGDBA1DElJSXYu3cvfH190apVK8V2JycnDBgwAKdOnUJ2drakOFiAEhEREZmIGzduYMqUKYiPj4e7u7uoY65fv478/Hx4eHhU2NeuXTsIgoDLly9LiqOGpNZEREREVG3t2LEDlpaWko4pG910dXWtsM/JyQkAkJWVJalPoy9AU1JSIAgCLCws9B2KUZPL5ZDJZGjfvr3Wz8Wc6gZzanyYU+Ojy5ySdl27dg1FRUUVtqelpWHt2rWVHpeYmCjpPFKLTwDIy8sDAFhbW1fYV7atoKBAUp9GX4AKggC5XI67d+/qOxSj5uLigpKSEp2ciznVDebU+DCnxkeXOSXtKioqQlFRUbkCsaioCA4ODnqM6qWyJ+VlMlmlbczMpN3VafQFqIWFBe7evYuAgAB9h2LU4uPjdXYu5lQ3mFPjw5waH13klKPZ2qFs9NrS0hKtW7dWvE5NTYWrq6vkUU5Ns7W1BQAUFhZW2Fc28mlnZyepT6MvQImIiEh1HM3WDhcXlwpFvSAI5Ua0BUGoctRRVxo1agQASp90L5vKSdn9oVVhAUpERESV4mi2dsTHx6NZs2YVthviLRXu7u6wtbVV+qR72TZPT09JfXIaJiIiIiIDUVpaqvgyFJaWlujTpw+SkpKQnp6u2P7w4UPs378f3bp1Q506dST1qfUR0Hv37iErKwsdO3bU9qmIiIiIqi1DuQSfkpKCzMxMdOnSBfXq1QMATJ06FYcPH0ZISAhCQ0NhZWWFrVu3oqSkBJ999pnkc2h9BHTXrl0YNWqUtk9DREREVO2VlJQovvRl+/btmDlzJm7evKnY1qBBA3z//ffw9PTE2rVrsXLlSjRp0gTR0dFo2bKl5HNofQTUzs5O8o2pRERERKZG2QioNk2ZMgVTpkypsH3x4sVYvHhxhe3Nmzevck5SKdQaAY2JicHVq1erbDN69Gj8+uuv6pyGiIiIyCQYwgioLqhVgH777bcaq4SJiIiITJ2pFKBqXYIvKipSzA1FRERERKoTBKHc0+/avgSvT2qNgAYEBGDv3r24ffu2puIhIiIiMlkcARXB1dUVgiBg4MCBaNGiBdzc3FCzZs0K7WQyGZYsWaLOqYiIiIiMnrEXnmXUKkCXL1+u+HdqaipSU1OVtmMBSkRERFQ1XT8Fr09qFaBRUVEGsUYpERERkTHgCKgIvr6+b2xTVFSEu3fvqnMaIiIiIqNnSiOgaj2E1Lp1a6xatarKNitXrsSwYcPUOQ0RERGRSeBDSEqkp6fjr7/+UrwWBAF3797FqVOnlLYvLi7GyZMnIZfL1YuSiIiIyAS8Og2TMZNUgGZkZGDKlCmK+z5lMhl2796N3bt3V3qMIAjo3r27elESERERGbnXL8EbM0kFaO/evTFu3DjFKGhcXBxatWqF1q1bK21vYWEBV1dXDB8+XP1IiYiIiIzcqwWoubm5HiPRLskPIf39739X/DsuLg69evVCWFiYWkE8fPgQ8fHxOH36NG7duoWnT5+iqKgINjY2sLOzw1tvvQUfHx989NFHqFu3rlrnIiIiIjJULEBFSEtLUzuAmJgYLF26FIWFhQAAKysr2NnZwcHBAUVFRfjrr79w7949HDt2DKtXr8bs2bMRFBSk9nmJiIiIDIkpPQWvVgFa5uLFi/D09ATw8ubZ9evX47fffkPjxo0xbtw4NGzYUOlxhw4dwoIFC9C4cWNMmjQJfn5+cHZ2rtAuOzsbSUlJWLNmDb744gvUr18fPXv21EToRERERAbDVO4BVWsapmfPnmH48OEICgpCTk4OAGDZsmWIjIxEUlIStm3bhqFDh+LBgwdKj9+wYQOcnZ0RGxuLgIAApcUnADg7O2Pw4MHYvn076tevjw0bNqgTNhEREZFBKi0tVXwZM7UK0HXr1uH8+fPw8/MDALx48QI//PADateujejoaHz99dfIzc3F6tWrlR5/7do19OvXD46OjqLOV6dOHfTv3x/Xr19XJ2wiIiIig1N2Cb7si5fgK3Ho0CF4eXlh48aNAICjR48iPz8fQ4cOhY+PD3x8fHD06FEcP35c6fG2trYoKCiQdM5nz56pEzIRERGRweIleBHu3bsHHx8fxesTJ05AJpPhvffeU2xr0qRJucnrX+Xp6Yl9+/YhPT1d1PkuXbqEffv2wdvbW52wiYiIiAwOR0BFsrW1RVFRkeJ1UlISLCwsyhWIjx49Qu3atZUeHx4ejpMnT+Kjjz5C//790alTJ7i5ucHR0RGWlpaQy+V4+vQpMjMzcfr0aezfvx9mZmaYOnWqOmETERERGSRTGQFVqwBt3rw5jhw5goiICFy8eBG3bt1Ct27dYGVlBQC4c+cOfvnlF3To0EHp8S1atMCWLVswb9487N69G3v27Kn0XIIgoFmzZliwYAHatGmjTthEREREBokFqAijRo3C1KlT0bVrV7x48QIymQwjR44EAGzevBnffvstCgsLMWbMmEr7aNeuHeLj43H+/HmcPXtWMRF9fn4+rKys4ODggObNm6Njx4689E5ERERGi/OAitS7d28sX74c69evBwCMGDFCse57fn4+7O3tsXDhQsVT8lXx9vZmgUlEREQmzdinXyqj9kT0/fr1Q79+/SpsHzVqFD799FPIZDJ1T0FERERk9DgCqgF2dnba6pqIiIjIKPEeUJFu3LiBbdu2ISMjA3K5XGm1LpPJEBUVpe6piIiIiIwaC1ARTp06hb/97W8oLi6usl1ll+Fnzpyp0nllMhmWLFmi0rFEREREhoiX4EVavXo1SkpKMGXKFPj7+8Pe3l7SPZ/nz59HVlYWAGnfZBagREREZIw4AirClStX0LdvX0yePFml4/ft24fw8HAcPXoU3bt3x9ixY9UJh4iIiKha41PwYg6uUQMNGjRQ+fiaNWti5cqVGDVqFI4fP45x48ahY8eO6oREREREVC2Z0iV4tdaC79ChA86dO6dWABYWFvjmm29gbW2NL7/80qi/2URERERVeXUteG35888/MWfOHHTr1g1eXl4ICgpCUlKSqGODg4PRsmVLpV+PHj0SHYNaI6DTp09HUFAQli5divHjx8PBwUGlflxcXDB+/Hj897//xbFjxxST2RMRERGZCl2MgD5//hyhoaHIzMxESEgInJ2dERsbi3HjxmH9+vXo2rVrlcenp6ejQ4cOGDp0aIV9tWrVEh2HWgXot99+iwYNGmDjxo3YuHEjHB0dFevAv0omk+Hw4cNV9jVhwgRMmDBBnXCIiIiIqjVtP4QUHR2N9PR0rF27Fu+//z4AICAgAAEBAViwYAF+/vnnSh8of/DgAXJzc9G5c2cMGjRIrTjUKkATEhLKvc7JyVErGCIiIiJTpu0CdPfu3WjSpImi+AQAW1tbfPLJJ1i2bBkuXrwILy8vpcdeu3YNANCiRQu141CrAE1LS1M7ACIiIiJ6ecn91afgNX0JPi8vD7du3cLAgQMr7GvXrh0AiCpA33rrLQAvL+fb2NiotOy6Wg8hadrnn3+OxMRExeuEhAR8/vnneoyIiIiISHe0+RBSdnY2BEGAq6trhX1OTk4AoJifXZlr165BJpPh+++/R6dOneDt7Y2OHTtiwYIFyM/PlxSLRtaCv3TpEmJjY3H16lUUFBTA0dERLVq0QGBgYKVVtDJxcXFo2LAhevbsCeDlCGt8fDwWLVqkiTCJiIiIDNrrhef9+/cRERFRaftXB+7eJC8vDwBgbW1dYZ+NjQ0AoKCgoNLjr127BkEQcOvWLcybNw8ymQyJiYmIjo5GamoqtmzZgho1xJWWaheg69atQ2RkZIWJU1NSUrBjxw6Eh4fz4SIiIiKiN9D2U/BltZqyS+Zl56rqcvqwYcNQWFhYbuGg/v37o27duoiKisLevXsRGBgoKha1CtBjx45h+fLlcHFxQUREBHx8fODs7IynT5/i9OnT+OabbxAZGQkPDw/4+fmpcyoiIiIio/f6CKirq6ukUc6q2NraAgAKCwsr7Csb+bS3t6/0+BEjRijdHhwcjKioKCQlJemmAN20aRPs7e0RHR2NRo0aKbbXrVsXAwYMgKenJwIDAxEVFcUClIiIiKgK2h4BLavVsrOzK+x7+PAhACi9P/RN6tatC+DlQ0liqfUQ0pUrV+Dv71+u+HxVo0aN4O/vj4sXL6pzGiIiIiKToM2HkOzs7NC0aVNcvny5wr6ybR4eHkqPzczMxIABA5Q+l3Pz5k0AQOPGjUXHolYB+uLFizfOel+rVi1JFTERERGRqSotLVV8acPAgQNx48YNHDt2TLHt+fPniI2NRfPmzRXTMb2uUaNGyMnJQXx8fLkR1JKSEqxYsQIymQwffvih6DjUugTv5uaG5ORklJaWwsysYi1bUlKC06dPw83NTZ3TEBERERk9XSzFGRoait27dyM8PByjR49G/fr1ERsbi3v37mH9+vWKh5BSUlKQmZmJLl26oF69ejAzM8PChQsxadIkDB06FMOHD4elpSUOHDiAixcvYvLkyWjTpo3oONQaAe3Xrx/S09Px5ZdfoqioqNy+Z8+e4Z///Cdu3LiBfv36qXMaIiIiIpOgzUvwwMvL8DExMejZsydiYmKwdOlSWFtbY8OGDejcubOi3fbt2zFz5kzF5XUA8Pf3x3//+1+4ublhzZo1iIyMhCAIWLZsGaZOnSopDrVGQP/2t78hISEBP/zwA3766Sd4eHjA3t4e2dnZSE9PR15eHlq2bFnucX0pBEHQSvVPREREZIi0vRQnADg7O2PZsmVVtlm8eDEWL15cYXvXrl3RtWtXtWNQqwC1srJCdHQ0li5dij179uD48eOKfdbW1vjkk08wc+ZMpROeKrNlyxY0bNhQ8Xrw4MHw9fVVJ0QiIiKiakEXl+ANhdoT0dvZ2eGLL77A3LlzcefOHeTl5cHOzg7NmjV742z4+fn5ipn3AeDdd98tt79hw4blCtIyZ86cQceOHdUNnYiIiMig6GIE1BCovRa8IAg4evQoUlNT8fbbb8Pb2xstWrTAwoUL8euvv1Z57IQJE5ROhlrVub799luMHj1azaiJiIiIDIsgCOWegjfmEVC1p2H69NNPMXHiRBw5ckSxvaCgAD/88AMmT56MWbNmVTqVwJkzZzBp0qQKDzApk52djeDgYKxZs0ZrUxMQERER6ZO2H0IyFGoVoFFRUThy5Ai6d++O/v37K7ZbW1sjLi4OvXv3xp49e7B582alx3t4eODUqVMICwuDXC6v9DyJiYn48MMPcfbsWdjb22Pp0qXqhE1ERERkkFiAirBnzx60bdsWa9euxVtvvVVuX+vWrbFixQq0atUKO3fuVHr8xo0b4eHhgePHjyM8PLzCN7uoqAgLFy5EWFgYnjx5Al9fX+zZswcDBw5UJ2wiIiIig1P2EFLZFy/BVyIrK6vKp9RlMhk6deqEu3fvKt1vZ2eHDRs2wMPDA4cPH8b06dMVl9dv376NTz75BDExMbCwsMCsWbMQFRUFFxcXdUImIiIiMlgcARXB1tYWWVlZVbZ5+PBhuSfdX1dWhHp6euLgwYOYOXMmdu7cicGDByMtLQ1vv/02du7cidDQUHVCJSIiIjJoHAEVycfHB4mJifj999+V7k9LS0NCQgI6dOhQZT+vFqH79+/HvHnzUFhYiNDQUOzcuRMtWrRQJ0wiIiKiakHba8EbCrVXQkpMTERwcDCGDh0KT09P2NvbIy8vDxcvXsTOnTtRUlKCiRMnvrEvW1tbbNiwAePGjUNKSgp69OiBWbNmqRMeERERUbVi7Jfey6hVgLZp0wbLly/H3LlzsWnTJsUC9sDLYWQ7OzssW7YMHh4eovorK0LHjh2Lo0ePYtGiRfj888/VCZGIiIioWuBKSBL07t0bnTt3VkxGn5ubC1tbW7Rq1Qq9evWCnZ1dpceeOXNG6fYJEyZg7ty52LJlCwRBQO/evSu04UpIREREZGw4AiqBra0t+vfvX24uUDGCg4PLjZq+ThAEbN26FVu3bi23XSaT4erVqyrFSkRERGSoWIBK8PTpUxw4cABXr17FkydPsGLFCpw7dw4ymQze3t6VHsdRTCIiIqKXeAlegkOHDuHzzz/H8+fPIQiCYkTz6NGjWL9+PUJDQzFz5kylx74+sklERERkykxlBFStaZguXbqEadOmwcrKChERERgwYIBin6+vLxo0aIBNmzbhl19+UTtQIiIiImNnKtMwqVWArl69GjY2Nvjxxx8xYcIEuLu7K/Z16dIF27dvh6OjI6Kjo9UOlIiIiMiYcSJ6kVJSUtC3b99Kl8esV68e+vTpgxs3bqhzGiIiIiKTYCpLcap1D2hBQUGV0ywBgJWVFfLz89U5DREREZHR40NIIjVu3Bjnzp2rdL8gCDhz5gzc3NzUOQ0RERGRSTD2kc8yal2C79+/Py5fvoxvv/22QpVeXFyMf/3rX0hLS8MHH3ygVpBEREREpoCX4EUYO3Ysfv31V6xZswY7duyAhYUFgJcrGaWmpuLhw4do2bIlxo4dq5FgiYiIiIyVIAjlnn435kvwao2AWllZYevWrQgODkZBQQHu3bsHQRBw9OhRPH36FJ988gliYmJgbW2tqXiJiIiIjBZHQEWytrbGP/7xD8yePRu3b9/GkydPYGtrC3d3d1haWgIAduzYgSFDhqgdLBEREZExM/bCs4xKBWhmZiZOnjyJvLw8tG7dGl27doWZmRmaN29ert3du3cxZ84cnDlzhgUoERERURX4FHwlBEHAokWLEB0dXe6b4unpif/85z9wcHBQtNuwYQNWrlyJwsLCN07VRERERESmMwIq6R7Q2NhYbNmyBZaWlhg0aBBCQ0Px9ttv48KFC5g/fz4A4PHjxwgJCcG///1vFBYWokePHti/f782YiciIiIyGqa0EpKkEdC9e/fCwsIC27dvR8uWLQEAn332GaZNm4aDBw/i9u3bCAsLw82bN1G7dm3MmTMHAwcO1ErgRERERMbG2NeALyNpBPTmzZvw9/dXFJ8AIJPJMHHiRJSUlCiKzx49emDfvn0sPomIiIgk4FPwSuTl5aFJkyYVtjdt2hQAcOvWLYwfPx7Tp0/XSHBEREREpsKUHkKSNAJaXFysmFrpVTVr1gQAvPPOOyw+iYiIiFSkixHQP//8E3PmzEG3bt3g5eWFoKAgJCUliTo2MzMT4eHh8PPzQ/v27TFmzBhcuXJFcgxqTUT/uq5du2qyOyIiIiKToYuHkJ4/f47Q0FDs27cPgYGB+Oyzz1BQUIBx48bhxIkTVR6bnZ2NESNGIDk5GSNHjkR4eDgyMjIwcuRIpKWlSYpD7YnoX1W2FCcRERERSaftez+jo6ORnp6OtWvX4v333wcABAQEICAgAAsWLMDPP/8MmUym9NhVq1bhr7/+wq5du9CqVSsAQP/+/TFgwAAsWbIEmzZtEh2HRkdAiYiIiEh12r4Ev3v3bjRp0kRRfAKAra0tPvnkE9y5cwcXL16sNK69e/fC19dXUXwCgJOTEwYMGIBTp04hOztbdBySR0ATExORlZUlaZ9MJsPXX38t9VRERESkZ3K5HC4uLoiPj9d3KEbFxcUFcrm83DZBEMpNw6TpS/B5eXm4deuW0lmK2rVrBwC4ePEivLy8Kuy/fv068vPz4eHhofTYbdu24fLly3B2dhYVi+QCNDU1FampqZL2sQAlIiKqnmQyGSwsLNCsWTN9h2JU5HK50kvd2rwEn52dDUEQ4OrqWmGfk5MTAFQ6yFg2uqnKscpIKkAXLVokpblB4F9uuuHi4qKzOcuYU91gTo0Pc2p8dJHT9u3ba7V/+h9nZ2f8+OOP5V7fv38fERERlR6TmJgouv+8vDwAgLW1dYV9NjY2AICCggLJx5Ztq+xYZSQVoIGBgVKaGwT+5aYbcrlc6RRd2sCc6gZzanyYU+Ojy5ySdpXl0d3dvdz2zMxMjZ2j7PK+spHXssv9lT2AVNWxZczMxD9apNGn4A0R/3IzPsyp8WFOjQ9zSiTNq6tMvqpdu3YYMmSIRs5ha2sLACgsLKywr2z00t7eXuVj7ezsRMfCp+CJiIiITECjRo0AQOnT6g8fPgSg/B5PdY9VhgUoERERkQmws7ND06ZNcfny5Qr7yrYpe8odeHlrgK2tbZXHenp6io6FBSgRERGRiRg4cCBu3LiBY8eOKbY9f/4csbGxaN68uWI6ptdZWlqiT58+SEpKQnp6umL7w4cPsX//fnTr1g116tQRHYdMMOaV7omIiIhI4dmzZwgICMBff/2F0aNHo379+oiNjcX169exfv16dO7cGQCQkpKCzMxMdOnSBfXq1QMA3Lt3D4GBgTAzM0NoaCisrKywdetWPH78GNu2bav0PlZlWIASERERmZDs7GwsXboUx44dg1wuR6tWrRAeHo5OnTop2syePRtxcXHYsmULfH19Fdtv3ryJpUuX4rfffoO5uTk8PDwwY8YMvPPOO5JiYAFKRERERDrFe0CJiIiISKdYgBIRERGRTrEAJSIiIiKdYgFKRERERDrFApSIiIiIdIoFKBERERHpFAtQIiIiItIpFqBEREREpFMsQImIiIhIp1iAEhEREZFOsQAlIiIiIp1iAUpEREREOsUClIiIiIh06v8BRJ0BtIlSwqQAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 700x280 with 11 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "row_ids = [0, 1, 2, 3, 4] # row ids for our test set\n",
    "W = 1.4\n",
    "H = 1.4\n",
    "fig, axgrid = plt.subplots(nrows=2, ncols=len(row_ids), figsize=(5*W, 2*H))\n",
    "\n",
    "# Top row: original images (rows of x_tr_NF)\n",
    "for col_id, row_id in enumerate(row_ids):\n",
    "    cur_ax = axgrid.flatten()[col_id]\n",
    "    if col_id == 0:\n",
    "        cur_ax.set_ylabel(\"Original\\nImg\")\n",
    "    cur_ax.set_title(\"Img %d\" % (row_id))\n",
    "    im_handle = cur_ax.imshow(x_te_TF[row_id].reshape((8,8)),\n",
    "                              interpolation='nearest', vmin=0.0, vmax=1.0, cmap='gray')\n",
    "    cur_ax.set_xticks([]);\n",
    "    cur_ax.set_yticks([]);\n",
    "\n",
    "\n",
    "\n",
    "# Bottom row: reconstructed images (rows of xhat_tr_NF)\n",
    "for col_id, row_id in enumerate(row_ids):\n",
    "    cur_ax = axgrid[1].flatten()[col_id]\n",
    "    if col_id == 0:\n",
    "        cur_ax.set_ylabel(\"Reconstr.\\n K=10\")\n",
    "    im_handle = cur_ax.imshow(xhat_te_TF[row_id].reshape((8,8)),\n",
    "                              interpolation='nearest', vmin=0.0, vmax=1.0, cmap='gray')\n",
    "    cur_ax.set_xticks([]);\n",
    "    cur_ax.set_yticks([]);\n",
    "plt.colorbar(im_handle, fraction=0.05);\n",
    "plt.tight_layout()\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Discussion 2d: Is the PCA model good at reconstructing the test examples it has never seen before?\n",
    "\n",
    "Why or why not? Do you notice any differences compared to the training data?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 3: PCA with ALL components and selecting K"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## First, fit PCA that learns all possible components\n",
    "\n",
    "Remember that in general, PCA is a *nested* model.\n",
    "\n",
    "The weights for K=5 are just the first 5 rows of the weights for K=6.\n",
    "\n",
    "The weights for K=9 are just the first 9 rows of the weights for K=10 or K=15, etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "all_pca = sklearn.decomposition.PCA().fit(x_tr_NF)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Determine how each additional component explains observed variance better"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsUAAAH4CAYAAACix5D2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABoIUlEQVR4nO3dd3gUVd/G8XsJCaRRA0gPxYQAoUsv0pEiRAURpIOIiKgIotgrCjYERVBBigLSmygdEUThpRNAwACChk4gCaTN+wdPVmJCsplsSbLfz3V5PeycszO/PckDd07OnLEYhmEIAAAAcGN5XF0AAAAA4GqEYgAAALg9QjEAAADcHqEYAAAAbo9QDAAAALdHKAYAAIDbIxQDAADA7RGKAQAA4PbyurqAnGr37t0yDEOenp6uLgUAAABpiI+Pl8ViUe3atTPsy0yxSYZhyJkPAzQMQ3FxcU69pjtinB2PMXYOxtnxGGPnYJwdLzePcWbyGjPFJiXPEIeGhjrlejExMQoPD1flypXl4+PjlGu6I8bZ8Rhj52CcHY8xdg7G2fFy8xjv37/f5r7MFAMAAMDtEYoBAADg9gjFAAAAcHuEYgAAALg9QjEAAADcHqEYAAAAbo9QDAAAALdHKAYAAIDbIxQDAADA7RGKAQAA4PayfSjes2ePQkJCtGfPHpv6nzp1SiNHjlSjRo1Uu3ZtDRw4UAcOHHBskQAAAMjRsnUoPnXqlEaMGKGkpCSb+kdGRqp3797asWOHHn30UY0cOVInT57Uo48+qsOHDzu4WgAAAORU2TYUb9y4Ud27d9e5c+dsfs+UKVN08eJFzZw5U8OHD1f//v313XffydPTU++9954DqwUAAEBOltfVBaRl1KhRWrlypSpWrKgmTZpo1apVGb4nMTFRK1asUIMGDVSlShXr8eLFi6tTp06aN2+eIiMjVaJECUeWDgAA0tFl1LJ021d80PUOff6yoY8t57Gdva6Tc/rYf4xt7ZMdZMtQfOzYMY0YMUJDhgzRtGnTbHrPH3/8oZiYGNWoUSNVW2hoqL777jvt37+fUAwAwH+4W/gB0pItQ/H3338vLy+vTL0nMjJSklSyZMlUbcWLF5cknTlzJuvFAQCQTRBUAfvJlqE4s4FYkq5duyZJ8vb2TtWWfCw2NjZrhf2HYRiKiYmx6znvJLl2e38GpMQ4Ox5j7ByMs+NlZYwffnltuu3z32xrU5+M2PJvlC19zkZezrDP8s1HM+zz4dzfM+zz/KdbMuwz/P31GfZ57J30x88W/V9fk2Gfvq/9kGGfPq/SJz2OzFKGYchisdjUN1uGYjOSd6hI74PnyWPf+wrj4+MVHh5u13NmJCIiwqnXc1eMs+Mxxs7BODvef8f4tW//SrtjcnuvMhme05Z/W2zp88PmPRn2eXnqzxn2Gfp+xkF1+vKM69n4f2cz7HMoIuMAfiryeoZ9/r6Y9aB1Mepmhn0uX4vLsM+V6/RJj6OzlK2TrbkmFPv6+kqSbty4kaot+ad4Pz8/u17T09NTlStXtus57yQ2NlYREREKDAxMczYc9sE4Ox5j7ByMc+bZb/Y2/VAcEhKSYR//omUz7PP99uh02yVp5rrzGfY5eib1v5v/ZbFIhpF+n3pVimnn4fSv17NNZc1bdyzdPk8/XEMfz9+Xbp9x/ero7W/+L90+rw2qp9e+2plun4y8+3gDvTB1R7p93nuioZ7/7Ncs93l/eEONmeKefW79f8Ixjh1L//vtdrkmFJcpc+sn7+S1xbdL3tYtrfXGWWGxWOTj42PXc2bE29vb6dd0R4yz4zHGzsE432KPdbW2jKMtfZb9fCrDPqM+3Z5hn4N/ZjyjWrqYn86cT39W9cnutTT5+z3p9lk24X7d/9zydPu8OqRxhuPc+75qGYbi1vUrZBiKG9YoKyn9UFy3amlJWQvF1e++K8M+VStlfAO/LX1CKrpvH0f+HWXr0gkpF4XiChUqyNfXV/v370/VlnysZs2azi4LAOBgzrqRLCEx4wdJPfPRpgz7fPvTkQz7+Hp7Kjo2Pt0+z/Wuq4lzd6XbZ+rY1hmOT/uG5TMMxZkJFkBOlWtCsZeXl9q1a6cVK1bo6NGjCgoKknRrlnjVqlVq1qyZihQp4uIqAQCZ4azAe+FKxjfLhY1ZkWGfY39dzbBPh0aBWrM9It0+897qmOFnb1GnTIahODuy5Wt2e5+YmBiFh4crJCQkxYxiZs9jlr2uk537OHqMc8oOJzk2FO/evVunTp1SkyZNFBAQIEl66qmntHHjRvXr108DBgxQvnz5NHv2bCUmJmr06NEurhgA4AqRlzK+4WrAmz/Z5VqvDGqgN75Kfw3q8IdqZhiKnc3dwg+QlhwbiufPn68lS5Zo1qxZ1lBcqlQpffvtt5owYYKmTp0qDw8P1ahRQ5MmTVJwcLCLKwYA3M4es8AXr2Y8wzv47Yy35vLIY1FiUvp3ks15vYMefTX9LbruqZrxGlR7IqgC9pPtQ/GIESM0YsSIVMfHjx+v8ePHpzpeqVIlTZ061RmlAQDuwB6BN9GGNbz938h4hjevR54M1wMveq+Luo1O/0aygn75MryWrQizQPaT7UMxACD7ySj02sODY1dm2CdPHouSMpjhXTS+s7pmEHg98tjvRjLCLJAzEYoBAE738hfbMuyT0XIGSVo8vrO6ZXADXB4bA6+9bk4CkDMRigEAKThjx4c9RzN+qMSMl9tleAOch4dtTypl9hZARgjFAOBG7BF4r17P+NG3GRn5cC19Mn9Pun0CCtn2JD4CLwB7IBQDADIlox0YbNGmfvkMQ7FE4AXgPIRiAMgl0p4F/sv6J1sC5pSFe+1YEQDkHIRiAHATN+ISMuxjy0Mlvn+3k7q/sCrL9TALDCA7IRQDQA5gj7XAPV7MOMh2b323vl//R7p98ntl/E8HgRdATkMoBoBcYO2Okxn2MTLe4Ux9O1bNMBRLhF4AuQ+hGABczB6zwJMW7Mmwz6xX26vv6z/aWhYAuBVCMQBkc18s2ZdhnxqVA7Tv2IV0+xQukN+m6zELDMAdEYoBwIHsMQu8cuufGfZ5e1gTmx69TOAFgLTZ9iggAIBDfPfTkQz73N+sohMqAQD3xkwxAJhkj1ngb388nGGfId1CtfznExn2u/16MTExCg8PV0hIiHx8fDJ8LwC4O2aKAcBBjp2+kmGfpjVLOb4QAECGmCkGAAd55uPNGfZ5vu892spaYABwOUIxAKTBHksjvPPlVezNjJ8iBwBwPUIxAJjwyhfbMuzz3Zv3qduYFRn2YxYYAFyPUAzA7dhjFnj30fMZ9vHw4LYNAMgpCMUA8B9Xr9/MsM+g+6vpq+UHM+zHLDAA5AxMYwDAf/R5bU2Gfbq1qOyESgAAzsJMMYBcxR5LIwzDtmsxCwwAuQczxQDcysl/ojLsM/OVdk6oBACQnRCKAbiVJydszLBP0YLeTqgEAJCdsHwCgFvJ62FRQmLG6yNYGgEA7oVQDCDHyGi98PfvdMrwHLNf66BHXv7BXiUBAHIJQjGAXMOWXSP8fLyYBQYApMKaYgC5xo24RFeXAADIoZgpBpAt2GMrtQlPNdPoST/bqyQAgBshFAPIEd6fvTPDPlXKF2FpBADAFJZPAMgRft5zxtUlAAByMUIxgByhY+NAV5cAAMjFWD4BwOHSXi/8l/VPX73UNsNzDHuwplZvi7BfUQAA3IZQDMDlhry91qZ+rBcGADgKyycAuFxSxg+YAwDAoQjFAFxu6tjWri4BAODmWD4BIEsy2l/49SGNMjxH6WJ+LI0AALgUM8UAHOrV6dtdXQIAABkiFANwKO98Hq4uAQCADBGKATjU1y+3d3UJAABkiDXFAO4oo/XCHz7dPMNz+Hl7plgvHBMTo/DwcIWEhMjHxyfLNQIAYA/MFAMw7dmPt7i6BAAA7IJQDMA0jzwWV5cAAIBdEIoBmDb9xYwfzwwAQE7AmmLATWW0Xnjp+10yPEexwt7sLwwAyBWYKQaQpuETNrq6BAAAnIZQDCBNZ85fd3UJAAA4DaEYQJoe7VDF1SUAAOA0rCkGkKaH2wbr4bbBri4DAACnYKYYcEOGYbi6BAAAshVmioFcKKOdJWreHeCkSgAAyBlsCsVjxowxdXKLxaL33nvP1HsBOM7ePy64ugQAALIVm0Lx8uXLUx2zWG49ySqtX8NaLBYZhkEoBrKpWkHF9MSDNVUywNfVpQAAkC3YFIpnzZqV4nVMTIxefvllWSwWDR06VHXr1lXhwoUVHR2tPXv26IsvvpDFYtHHH3/siJoBZNEbjzWy/mALAABsDMX169dP8fqdd95RXFycFi1apDJlyqRoq1ixopo2baqwsDDNmTNHb731lv2qBWAXBGIAAFIytfvE6tWr1aZNm1SBOFnx4sXVpk0brV27NkvFAci8m/GJri4BAIAcx9TuE9HR0fLw8Ei3T1xcnBIT+ccZsLeMdpYoU9zPSZUAAJB7mJopDgoK0rp16xQZGZlme0REhNauXatq1aplqTgAmffXOR7PDABAZpmaKR4wYICefvpp9erVSwMHDlSNGjXk7++vqKgo7dq1S1999ZViY2P1+OOP27teABm4t04ZPRYWKn8fL1eXAgBAjmEqFHfo0EHPPfecPvnkk1Q30hmGIR8fH7399ttq1KiRXYoEYLtRveu6ugQAAHIc00+0Gzx4sNq3b68ffvhBhw4d0rVr11SoUCGFhoaqU6dOKlasmD3rBAAAABwmS495Llu2rB577DF71QIAAAC4RJZCcVRUlFavXq1Dhw7p6tWr+uSTT7Rr1y5ZLBbVqVPHXjUCbiWj3SUAAID9mQ7Fa9eu1QsvvKDo6GjrI50lafPmzZo+fboGDBigMWPG2K1QAAAAwFFMheJ9+/bpmWeeUcGCBTVkyBAdPXpUq1evliQ1aNBAq1at0owZM1SzZk21b9/ergUD7u79J5sppEIRV5cBAECuYmqf4s8++0w+Pj5atGiRhg4dqgoVKljbmjRpovnz56tQoUKaM2eO3QoFcAuBGAAA+zMVinfv3q327dvrrrvuSrM9ICBA7dq107Fjx7JUHAAAAOAMpkJxbGys/PzSf5Rsvnz5FBMTY6oowF3FxfNodAAAXMHUmuJy5cpp165dd2w3DEO///67ypYta7owwN2cOX9d78363dVlAADglkyF4o4dO+rTTz/VpEmTNGLEiBRtCQkJ+uCDD3T48GENHz7cLkUCuQXbrQEAkD2ZCsWDBg3Shg0b9Pnnn+v777+Xp6enJGno0KEKDw/XuXPnFBwcrEGDBtm1WCC3C60UoFG966hoQW9XlwIAgFsxtaY4X758mj17tvr06aPY2FidPXtWhmFo8+bNioqKUo8ePTR37lx5e/MPO5AZbz7emEAMAIALmH54h7e3t1588UWNHTtWf/75p65evSpfX19VqFBBXl5e9qwRcBseeSyuLgEAALeUpcc8S1KePHlUqVIle9QCAAAAuITpUHz58mUtWbJEJ0+eVHx8vAzDSNXHYrHonXfeyVKBQG6RlJT6/yMAACB7MBWKDx06pD59+igmJibNMJyMUAzcEp+QqI+/2+3qMgAAwB2YCsWffPKJoqOj9cADD6hVq1by9/eXxWK/tZAXLlzQRx99pC1btujatWuqUqWKRowYoSZNmmT43v379+vjjz/W7t27lZCQoJCQEA0bNkz33nuv3eoDMuN6bLzemfGb9h+/4OpSAADAHZgKxTt37lTz5s0dMgscHR2tAQMG6NSpU+rXr59KlCihBQsWaPDgwZo+fbqaNm16x/ceOnRIjz76qPLnz69BgwbJz89PCxYs0OOPP64PP/xQHTt2tHu9QLKM9iD2zpdXL/S7R7WDizupIgAAYCtTodgwDAUFBdm7FknSnDlzdPToUU2dOlUtW7aUJHXr1k3dunXTm2++qTVr1txxVnrq1Km6ceOGZs+erRo1akiSwsLC1KFDB02cOJFQDJcaP7ypKpYu6OoyAABAGkztU1ytWjUdOnTI3rVIkpYtW6by5ctbA7Ek+fr6qkePHoqIiNDevXvv+N6IiAgVLlzYGoglqUCBAqpbt67OnDmjqKgoh9QM2IJADABA9mUqFI8cOVK//fab5s6dm+6Ndpl17do1nThxIkWoTRYaGipJ6YbiihUr6urVq7pwIeXazVOnTsnHx0d+fn52qxUAAAC5h6nlE2vXrlVQUJDeeustTZo0SWXLllW+fPlS9bNYLJozZ47N542MjJRhGCpZsmSqtuLFb63DPHPmzB3fP3LkSO3atUsjR47U2LFj5e/vr9mzZ+vw4cN69tlnlSePqZ8BAAAAkMuZCsXffPON9c9Xr17V1atX0+yX2R0prl27JklpPh7ax8dHkhQbG3vH9wcGBurxxx/XO++8o4ceesh6vE+fPho6dGimarGFYRiKiYmx+3nTkvy50/v8yDpHjrOzvleyO76XnYNxdjzG2DkYZ8fLzWNsGIbNedRUKF6/fr2Zt2UoKSlJUtphOnmZRnof7NVXX9X8+fNVu3Zt9ezZU15eXlqzZo1mz56tmzdv6s0337RrvfHx8QoPD7frOTMSERHh1Ou5q8yMs2EYWrcn7R8Mb+fs75Xsju9l52CcHY8xdg7G2fFy6xh7eXnZ1M9UKC5durSZt2XI19dXknTjxo1Ubck/vfj7+6f53j///FMLFixQaGio5syZo7x5b320jh076rXXXtN3332n1q1b23W/Yk9PT1WuXNlu50tPbGysIiIiFBgYmOZMOuwjs+OclGRo+vJw/RJ+PcO+ISEh9igxx+N72TkYZ8djjJ2DcXa83DzGx44ds7mvTaH49OnTKliwoAoUKGB9bauyZcva3LdMmTKSbq0t/q9z585JUprrjSXpyJEjMgxD3bp1swbiZD179tR3332n7du32zUUWywW67IOZ/H29nb6Nd3Rf8c5oz2I81ikJ7vXUtsG5R1dWq7B97JzMM6Oxxg7B+PseLlxjDOzlNemUNyuXTsNHz5cTz75pCSpbdu2Nl3EYrFkaus2Pz8/BQYGav/+/anako+ltTOF9O/UeGJiYqq25GUZabUB9jC23z1qFFrK1WUAAACTbArF9erVs87iStI999zjsII6d+6syZMna8uWLWrevLmkW0+5W7BggSpVqmTdmi2tGn18fPT999+rR48eKab/586dK0npPg0PyAoCMQAAOZtNoXj27NnpvranAQMGaNmyZRo5cqT69++vYsWKacGCBTp79qymT59unaHevXu3Tp06pSZNmiggIEAFChTQiy++qJdffllhYWF66KGHlD9/fm3cuFFbt27VfffdZ9elEwAAAMg9TN1o50h+fn6aO3euJkyYoLlz5yo+Pl5VqlTRV199pYYNG1r7zZ8/X0uWLNGsWbMUEBAgSerevbtKlSqladOmacqUKUpISFBgYKBeeOEF9e3b11UfCQAAANmc6VB88+ZN/frrr7p8+bISExOtW6YZhqGEhARduXJFmzdv1rx58zJ97hIlSmjixInp9hk/frzGjx+f6niTJk3UpEmTTF8TAAAA7stUKD579qweffRR/f333/auBwAAAHA6U889njx5ss6ePatatWrpscceU4ECBVSvXj0NHjxYLVq0kCQVLVrUYQ/5AJxp+Zbjri4BAAA4mKmZ4u3bt6tSpUr67rvvJN16Asr169c1atQoSdLKlSv13HPPadOmTerVq5f9qgWcbPv+v/Xl8gOSpAGdq+mBls55WAsAAHAuUzPFFy5cUKNGjayvg4ODdeDAAevrzp07q169elqxYkXWKwRc5Oipy5o4d5cMQ7qvUaDC7q3k6pIAAICDmArFnp6eKfYBLlu2rKKioqxPnZNuPWQjM0++A7KTc5dj9eZXOxQXn6i6VYpraFhopp6KAwAAchZTyyfKly+v8PDwFK8Nw9Aff/yh4sWLS5Li4uIUHR1tnyoBB0r7Ec5/Wf80pk89eXiY+vkRAADkEKb+pW/VqpW2bdumzz//XNHR0apSpYr8/f319ddfKy4uTpGRkfrxxx9VunRpe9cLOJ1Pfk9XlwAAABzMVCgeMGCAKlSooEmTJmn16tXy8vLSI488ol9++UX169dX69atdeHCBT3wwAP2rhcAAACwO1PLJ/z8/LRw4ULNmzdPNWrUkCSNGDFCsbGxWrZsmfz9/fXAAw+of//+9qwVAAAAcAjTT7Tz9vbWgAEDrK89PT01btw4jRs3zi6FAQAAAM7C3UMAAABwezbNFPfu3dvUyS0Wi+bMmWPqvYAzXIq64eoSAABANmBTKN61a5epk7OvK7Kzm/GJeuvrHa4uAwAAZAM2heL169c7ug7AqQzD0KR5u/XH6Svy9/HUxJHNVSrATzExMQoPD1dISIh8fHxcXSYAAHASm0Ix+w0jt5m/7qi27DkjjzwWvdCvvkoF+Lm6JAAA4EKmd5+QpOjoaG3cuFEHDx7UtWvXVKRIEdWsWVPNmjWTl5eXvWoETEv7aXX/GvZgDYVWDnBSNQAAILsyHYpXr16tN954Q1evXpVhGNbjFotFJUuW1LvvvqsGDRrYpUjAUdo3DHR1CQAAIBswFYp37Nih0aNHy8vLS/3791edOnXk5+enq1evaufOnVq0aJEef/xxzZs3T8HBwfauGQAAALArU6H4888/l5eXl7799luFhISkaOvQoYPuv/9+Pfroo/rss8/0ySef2KVQAAAAwFFMPbzj4MGDat++fapAnKxGjRpq27atduxguysAAABkf6ZCcZ48eVSwYMF0+xQuXFgJCQmmigIAAACcyVQobtGihVavXq2rV6+m2R4dHa1NmzapWbNmWSoOAAAAcAZTofiFF15Q0aJF1b17d61evVqXLl2SJCUmJmr37t16/PHHFRUVpV69eun06dMp/gMAAACyG1M32t17771KSkpSfHy8Ro0aJUny9PRUQkKCDMOwbtHWt2/fFO+zWCw6dOhQFksGbHM68po88+ZRfEKSnnmktlrVK+fqkgAAQDZlKhTXqFHD3nUAdpWUZOjTBXsUn5CkOlWKq2Xdsq4uCQAAZGOmQvHs2bPtXQdgV6t++VPhEZfknc9Dwx+qKYvF4uqSAABANmZqTfGJEyds6vftt9+aOT2QJZGXYjRr9a1lOv06VVPxwj4urggAAGR3pkLxgw8+qO+///6O7efOndOgQYP05ptvmi4MMMMwDE3+fo9uxCWqWsWiuq9RoKtLAgAAOYCpUOzh4aFXXnlFI0eO1LVr11K0rVq1Sl26dNEvv/yiatWq2aVIwFbrfz+lPUfPyytvHo3oUUt58rBsAgAAZMzUmuIVK1ZozJgx+vHHH7V//35NmDBBlStX1muvvaY1a9YoX758Gj16tAYMGGDvegGrLqOW3bEtLiFJpYv5ObEaAACQk5kKxSVLltSsWbP05ZdfatKkSerbt68KFCigK1eu6J577tFbb72lcuXY/goAAAA5g6nlE9KtPYcHDhyosLAwJSYm6vLlyypUqJBeeuklAjEAAAByFNOh+MCBA9Yb7kqWLKmHHnpIV65c0YMPPqgpU6YoISHBnnUCAAAADmMqFL/33nvq2bOnDh8+rC5dumjFihV66623NHPmTBUrVkyTJ0/Wgw8+qAMHDti7XgAAAMDuTIXiGTNmyN/fX5MmTdL7778vP79bNzQ1aNBAy5cvV9euXXXkyBH17NnTrsUCAAAAjmAqFN97771auXKl2rVrl6rNz89P48eP1+TJk1WwYMEsFwgAAAA4mqndJ6ZOnZphnzZt2qhOnTpmTg8AAAA4lalQnCwqKkqrV6/WoUOHdPXqVX3yySfatWuXLBaL6tSpoyJFitirTiCFG3EJKlbYW+cvx6pn22D17lDF1SUBAIAczHQoXrt2rV544QVFR0fLMAxZLLeeHLZ582ZNnz5dAwYM0JgxY+xWKHC7JRuP6fzlWAUU8taDrSq7uhwAAJDDmVpTvG/fPj3zzDPKly+fnn76aXXq1Mna1qBBA5UqVUozZszQjz/+aLdCgWTnLsVo4YY/JEkDu1RTfq8s/cIDAADAXCj+7LPP5OPjo0WLFmno0KGqUKGCta1JkyaaP3++ChUqpDlz5titUCDZ1ysOKi4hSdUrFVXTmqVcXQ4AAMgFTIXi3bt3q3379rrrrrvSbA8ICFC7du107NixLBUH/Ne+Y+f1y76zymORHusWal22AwAAkBWmQnFsbKx1b+I7yZcvn2JiYkwVBaQlMTFJ05bslyTd17iCKpRiyz8AAGAfpkJxuXLltGvXrju2G4ah33//XWXLljVdGPBfP2yP0Ml/rsnfx5PdJgAAgF2ZukOpY8eO+vTTTzVp0iSNGDEiRVtCQoI++OADHT58WMOHD7dLkXA/XUYtu2PbtZh4+ft4ObEaAACQ25kKxYMGDdKGDRv0+eef6/vvv5enp6ckaejQoQoPD9e5c+cUHBysQYMG2bVYAAAAwBFMLZ/Ily+fZs+erT59+ig2NlZnz56VYRjavHmzoqKi1KNHD82dO1fe3t72rhcAAACwO9MbvHp7e+vFF1/U2LFj9eeff+rq1avy9fVVhQoV5OXFr7YBAACQc2T5qQd58uRRpUqV7FELAAAA4BKmlk8AAAAAuQmhGAAAAG6PUAwAAAC3l+U1xYAj9GpfRd/+eFgli/pqyphW8szLz28AAMBxSBrIdq5cu6klm/6QJPW5L4RADAAAHC7LM8UnTpzQoUOHdPXqVfXu3Vtnz55V4cKF2aMYps1be0SxNxNVuWwhNalZytXlAAAAN2B6Cu7kyZPq1auXOnXqpNGjR+vtt9+WJC1evFgtWrTQ+vXr7VYk3MfZC9e1ZnuEJGlA56rKk8fi2oIAAIBbMBWKIyMj1bt3b/3f//2fGjdurJo1a8owDElS4cKFFRMTo5EjR+rAgQN2LRa53+zV4UpMMlSnSnHVqFzM1eUAAAA3YSoUT5kyRZcuXdL06dP11VdfqWnTpta23r1765tvvpHFYtG0adPsVihyv6OnLmvr3rOyWKT+naq6uhwAAOBGTIXizZs3q02bNmrWrFma7XXr1lXbtm21b9++LBUH92EYhmauPCRJalm3rCqUKujiigAAgDsxFYovXryo8uXLp9unZMmSunjxoqmi4H52HT6n/ccvyDNvHvXuUMXV5QAAADdjKhQHBATo2LFj6fY5cuSIAgICTBUF95KYZGjmyoOSpM5NK6p4YR8XVwQAANyNqS3ZmjdvroULF+qXX35RkyZNUrWvW7dOW7duVY8ePbJcIHKfLqOW3bFtyaZjGtilmhOrAQAAMBmKhw8frrVr1+qxxx5Tu3btdOHCBUnSV199pb1792rdunUqWLCghg4datdiAQAAAEcwFYpLlCihWbNmacyYMfrhhx+sxydMmCBJqlixoiZMmKDSpUvbp0oAAADAgUw/0e7uu+/WkiVLtH//fu3fv19RUVHy9fVVSEiI6tatK4uFhy4AAAAgZ8jSY56joqJUvHhx9erVy3ps5cqVunz5sooUKZLl4gAAAABnMP2Y5+nTp6tp06Zatuzfm6YSEhL0/PPP695779WiRYvsUiAAAADgaKZC8Q8//KAPPvhAAQEBqlSpkvW4YRgaPXq0ihcvrpdeeknr16+3W6EAAACAo5gKxd98841KlCihJUuWqHXr1tbjnp6e6t+/vxYtWqSAgAB99dVXdisUAAAAcBRTa4pPnDihsLAwFSyY9qN4CxYsqPbt22vhwoVZKg650+tDGunV6duV38tDM15uJz8fL1eXBAAA3JypmWKLxaJr166l2ycuLo4dKJCmxZv+kCS1a1ieQAwAALIFU6E4KChImzdv1qVLl9Jsv3LlijZt2qQqVapkqTjkPsf+uqK9f1xQnjwWdW1WKeM3AAAAOIGpUNy7d29dvHhRAwcO1MaNG3Xp0iXFx8fr0qVL2rRpkwYOHKjz58+n2KoNkG49xlmSmtUsreJFfFxcDQAAwC2m1hR36NBBAwcO1Ndff60nnngiVbthGOrTp4+6dOmS5QKRe5y7FKOte89KksLuZZYYAABkH6Yf3jFmzBi1bdtWS5cu1eHDh3XlyhX5+voqODhYYWFhql+/vj3rRC6wbMtxJSUZqnl3gCqVKeTqcgAAAKyy9ES72rVrq3bt2vaqxerChQv66KOPtGXLFl27dk1VqlTRiBEj1KRJkwzfGx0drc8//1yrV6/WhQsXVLp0aT300EPq16+f8ubN0sdFFlyPidNPO05Kkh64924XVwMAAJCS6SfaOUp0dLQGDBiglStXKiwsTKNHj1ZsbKwGDx6srVu3pvveuLg4DRgwQF9//bXuvfdevfDCCypfvrzef/99vfPOO076BEjL6m0RuhGXqMCSBVQ7uJirywEAAEjB9NTp1q1bNWfOHJ08eVLx8fEyDCNVH4vFonXr1mXqvHPmzNHRo0c1depUtWzZUpLUrVs3devWTW+++abWrFlzx63eZs6cqb179+qtt95S9+7dJUmPPPKIhg0bprlz52ro0KEqUaJEJj8psiouPlErtp6QJIXdW5mt+gAAQLZjKhSvXr1ao0aNSjMIZ9WyZctUvnx5ayCWJF9fX/Xo0UMTJ07U3r17VatWrTTfu2DBAgUFBVkDcbJhw4YpJCREN27csHu9yNjGXX/pyrWbCiiYX81rl3Z1OQAAAKmYCsVff/218ubNqzfffFOtWrVSgQIF7FLMtWvXdOLECXXu3DlVW2hoqCTdMRT//fffOn36tAYNGmQ9Fh0dLW9vb9WoUUM1atSwS43InKQkw7oN2/3NKymvR7ZbsQMAAGAuFP/xxx/q3LmzunXrZtdiIiMjZRiGSpYsmaqtePHikqQzZ86k+d4TJ279er506dKaMWOGZs6cqX/++Ue+vr564IEHNHr0aOXLl8+u9SK1LqOW3bHt6xUHFXZvZSdWAwAAYBtTodjb21sFCxa0dy3WR0d7e3unavPxufWgh9jY2DTfe/XqVUnS7NmzFRUVpccee0wlS5bUunXrNHv2bJ0+fVpffPGFXes1DEMxMTF2PeedJH/uO33+nMJZ42VWbhnn7Iwxdg7G2fEYY+dgnB0vN4+xYRg238tkKhQ3adJEv/zyi5KSkpQnj/1+HZ6UlCRJaRafvH75Th8sPj5ekvTXX39p2bJlqlTp1sMh2rdvL09PTy1atEhbt25V06ZN7VZvfHy8wsPD7XY+W0RERDj1evbm7PEyK6ePc07AGDsH4+x4jLFzMM6Ol1vH2MvLy6Z+pkLxqFGj9PDDD2v06NEaPHiwAgMD77g0ITOh2dfXV5LSvCEu+acXf3//NN+bPJPcsGFDayBO1qNHDy1atEjbtm2zayj29PRU5crOWQ4QGxuriIgIBQYGpjmTnn38lW5rSEiIk+owJ+eMc87FGDsH4+x4jLFzMM6Ol5vH+NixYzb3NRWKhw8frrx582r16tVavXr1HftZLBYdOnTI5vOWKVNG0q21xf917tw5SUpzvbEk61ZrAQEBqdqKFi0qSbp+/brNtdjCYrFYw7izeHt7O/2a9pRTas/p45wTMMbOwTg7HmPsHIyz4+XGMc7MNrCmQvHVq1dlsVjuGFDN8vPzU2BgoPbv35+qLfnYnXaRCAoKUv78+XX06NFUbadOnZL0b+gGAAAAbmcqFG/YsMHedVh17txZkydP1pYtW9S8eXNJt7ZWW7BggSpVqmTdmu2/8ufPrw4dOmjp0qXauHGjdZ/jpKQkff311/Lw8FD79u0dVjcAAAByLtNPtHOUAQMGaNmyZRo5cqT69++vYsWKacGCBTp79qymT59unQbfvXu3Tp06pSZNmliXTDz33HP6/fffNXLkSPXq1Utly5bVjz/+qB07duiJJ55Q+fLlXfnRAAAAkE1lKRQfP35cly5dUmJiYoqn28XHx+vKlSvatGmTPvzww0yd08/PT3PnztWECRM0d+5cxcfHq0qVKvrqq6/UsGFDa7/58+dryZIlmjVrljUUJwfoSZMmaeXKlYqKilJgYKDefvttPfTQQ1n5qLDRig+66rXp27Xr8Dk9cG9lDehSzdUlAQAAZMhUKL5y5YoGDx6sgwcPZtg3s6FYunXT3MSJE9PtM378eI0fPz7V8YCAAL3xxht64403Mn1dZF3kpRj935FbN0W2b8TMPAAAyBlMbTI8ZcoUHThwQKVLl1bHjh2VP39+BQcH67777lPlypVlGIaKFi2quXPn2rteZHNrd5yUYUg17w5QqQA/V5cDAABgE1MzxZs3b1bJkiW1evVqeXl56fHHH1eePHmss8LTpk3TRx99pL///tuuxSJ7S0hM0trfTkqS2jcMdG0xAAAAmWBqpviff/7Rvffea31CSNWqVbV3715r+2OPPaaQkBAtWLDAPlUiR/j90D+6FHVThfzyqWF1+27XBwAA4EimQrGHh0eKJ8uVK1dOly5d0qVLl6zH6tevr5MnT2a9QuQYa3699fVufU9Zeea13+O/AQAAHM1UcilVqlSK52OXK1dOUupH6V25csV0YchZIi/FaPf/brBr15Ab7AAAQM5iKhQ3a9ZMGzZs0PLlyyVJwcHBypcvn+bPny/p1sM2Nm7caH30MnK/H3+NkGFIte4uxg12AAAgxzEVigcNGqRChQrp+eef14IFC+Tr66uuXbtq1apVat26tdq1a6dTp06pQ4cO9q4X2VBCYpLW/XbrUdodGgW6thgAAAATTO0+UaxYMS1atEjTp09XUFCQJGn06NE6f/68Nm7cqDx58qhTp04aNmyYXYtF9vTbwX90+dqtG+zqV7vL1eUAAABkmukn2pUoUUIvvfSS9bWfn58+++wzXbt2TV5eXsqXL59dCkT2t2Z7hCSpTf1y3GAHAABypCw95jktt+9Kgdzvn4vR2n30vCSpPTfYAQCAHMqmUNy7d289+OCDeuCBB6yvbWGxWDRnzhzz1SHb+2nHrW3YagUV011FfV1cDQAAgDk2heJdu3apUaNGKV7bwmKxmKsKOcKtJ9hxgx0AAMj5bArF69evV4ECBVK8BnYc/EdXrt1UIf98asANdgAAIAezKRSXLl06xev58+erVq1aatWqlUOKQvbUZdSyNI9fuXZTYWNWaMUHXZ1cEQAAgH2Y2ipg7ty5WrNmjb1rAQAAAFzCVCjOkyePChUqZOdSAAAAANcwFYr79eunpUuXavv27fauBwAAAHA6U/sUX758WV5eXho4cKAKFCig0qVLy9vbO1U/tmQDAABATmAqFM+dO9f656tXr+rq1atp9mNLNgAAAOQEpkIxW7IBAAAgNzEViv+7RRsAAACQk5kKxf9lGIYMw7D+OSEhQVeuXNHGjRvVs2dPe1wC2cCKD7pqwuyd2rLnjLo2r6TBXau7uiQAAAC7MBWKk5KSNHHiRC1dulSXL19Oty+hOPe4cTNBOw79I0lqXpvfFgAAgNzDVCj+5ptv9PXXX8tisSggIEAXL16Uv7+/PD09denSJSUlJalIkSLq06ePveuFC/1+KFI34xJ1V1Ef3V22kKvLAQAAsBtT+xQvX75cPj4+WrVqlX7++WfVqVNHrVq10tatW7V161a1bt1aUVFRatKkib3rhQtt2fOXJKlZrdLsLAIAAHIVU6E4IiJCbdu2VcWKFSVJoaGh2rVrlySpSJEi+uijjxQQEKAvv/zSfpXCpa7Hxmtn+DlJUvPaZVxcDQAAgH2ZCsUJCQm66667rK8DAwN1+vRpxcbGSpK8vLzUokULHT582D5VwuV+3f+3EhKTVLaEv8rf5e/qcgAAAOzKVCgOCAjQhQsXrK/Lli0rSTp+/Lj1WIECBXTu3Lkslofs4uc9ZyRJLWqzdAIAAOQ+pkJx3bp1tW7dOp06dUqSFBQUJEnatGmTtc/u3btVqFChLBcI17t6/ab2/HFe0q31xAAAALmNqVA8cOBARUdHq0uXLlq3bp0CAgLUrFkzTZ06VS+88IKGDBmiXbt26Z577rF3vXCBX/adVVKSocplCqpUMT9XlwMAAGB3pkJx1apVNXXqVJUrV876q/SxY8eqcOHCWrJkiX7++WeVK1dOzz77rF2LhWts2X1r6QQ32AEAgNzK9BPtmjZtqqZNm1pfV6xYUT/99JO2bdum/Pnzq27dusqfP79dioTrXLgSq0N/XpQkNa3J0gkAAJA7mZop/uKLLxQZGZnquLe3t1q3bq0mTZoQiHOJrXvPyDCkqhWKqFhhb1eXAwAA4BCmQvFHH32kli1bauDAgVq+fLlu3Lhh77qQTbB0AgAAuANTofitt95SvXr19Ouvv+r5559XkyZNNG7cOO3cudPe9cGFzl64rj9OX1Eei9SkRilXlwMAAOAwptYUP/TQQ3rooYcUGRmp5cuXa/ny5Vq0aJEWL16s0qVLq1u3buratat1/2LkTMl7E9e4u5gK+edzcTUAAACOY2qmOFmJEiU0ZMgQrVixQsuWLdPAgQOVlJSkKVOmqF27durTp4+96oQL/Lz73wd2AAAA5GZZCsW3Cw4O1siRI/Xyyy8rJCREhmGwnCIHO/l3lE7+c015PSxqGMrSCQAAkLuZ3pItWVJSkrZt26aVK1dq3bp1io6OloeHh1q3bq2wsDB71AgX2PK/pRN1q5SQn7eni6sBAABwLNOheM+ePVq5cqV++OEHXbp0SYZhKCQkRGFhYerSpYsKFy5szzrhYF1GLUvz+I6D/6jLqGVa8UFXJ1cEAADgPKZCcZs2bXTmzBkZhqGiRYuqX79+CgsLU3BwsL3rAwAAABzOVCj+559/1LZtW4WFhal58+by8PCwd10AAACA05gKxVu3blWhQoXsXAoAAADgGqZ2nyAQAwAAIDex25ZsAAAAQE5FKAYAAIDbIxQDAADA7dl0o9327dtVqVIlFS9e3NH1wEVWfNBV12Pi9MjLP0iSZr/WQYX887m4KgAAAOewaab42Wef1axZs6yvX3jhBa1fv95hRcE1wiMuSZJKF/MlEAMAALdiUyiOjo5WVFSU9fWSJUsUHh7usKLgGof+vBWKq1Yo6uJKAAAAnMum5RPly5fX4sWLdf78eet2bOvXr9eZM2fSfZ/FYtE777yT5SLhHIf+vChJqlqhiIsrAQAAcC6bQvGoUaM0cuRIbdy4UdKtsBseHp7hbDGhOOeIi0/U0VNXJDFTDAAA3I9Nofjee+/Vli1b9Oeff+rmzZvq16+fwsLCFBYW5uj64CTH/rqihMQkFfLLp5IBvq4uBwAAwKlsfsxzwYIFVatWLUlSqVKlFBISovr16zuqLjhZ8nrikApFZLFYXFwNAACAc9kcim+3YcOGFK+jo6MVGxurQoUKKW9eU6eEi/27npilEwAAwP2YTrBJSUmaMWOG5s+fr9OnT1uPBwYG6oEHHtCgQYOUJw/PBskJkpIMhVt3nuAmOwAA4H5MheKkpCQ98cQT2rx5sywWi0qWLKnixYsrKipKERER+vDDD7Vz50598cUX9q4XDnD63DVdj41XPi8PVSxd0NXlAAAAOJ2pUPztt99q06ZNatKkiV577TWVLVvW2nbu3Dm98sor2rx5sxYsWKAePXrYrVg4RvJ64uByhZXXg9l9AADgfkwloMWLF6tMmTKaMmVKikAsScWLF9fHH3+sMmXKaNGiRXYpEo7FemIAAODuTIXiEydOqFmzZsqfP3+a7fnz51fTpk11/PjxLBUH5zjEemIAAODmTIVii8Wi+Pj4dPvEx8crMTHRVFFwngtXYnXuUozyWKTg8oVdXQ4AAIBLmArFwcHB2rx5s65evZpm+5UrV7Rp0yYFBwdnqTg4XvKuExVKF5RPfk8XVwMAAOAapkLxI488ovPnz2vo0KHav3+/9XhCQoJ27NihgQMH6uLFi3r44YftVigcg/XEAAAAJnef6Nq1q3bs2KHFixerR48eyp8/v3x9fXX58mUlJSXJMAx169aNx0DnAKwnBgAAyMLDO9555x01bdpUCxcu1KFDh3TlyhX5+fmpWrVqevDBB9WpUyd71gkHiI6NV8Tft5bAhAQSigEAgPvK0jOZO3bsqI4dO9qrFjjZkZOXlWRIdxX1UdGC3q4uBwAAwGV4UoMbYz0xAADALYRiNxYewXpiAAAAiVDsthISk3T45GVJrCcGAAAgFLupE2euKi4+Uf4+nipT3N/V5QAAALgUodhNJa8nDgksqjx5LC6uBgAAwLUIxW6K/YkBAAD+laUt2RITE3X58mXFx8fLMIw0+5QqVSorl4ADGIbBzhMAAAC3MRWKExMT9cknn2j+/PmKioq6Yz+LxaJDhw6ZLg6OcfZCtK5ej5Nn3jyqXLagq8sBAABwOVOh+IsvvtC0adOUN29eBQUFqUCBAvauCw506MStWeKgcoXlmdfDxdUAAAC4nqlQvHTpUgUEBOi7775T2bJl7V0THIz1xAAAACmZutHu77//VseOHR0WiC9cuKBx48apWbNmqlWrlnr27Klffvkl0+e5ePGiGjVqpD59+jigypyL9cQAAAApmZopLlasmG7evGnvWiRJ0dHRGjBggE6dOqV+/fqpRIkSWrBggQYPHqzp06eradOmNp9r3LhxunTpkkPqzEm6jFqW5vHXv/xVkrTig67OLAcAACDbMTVT3LFjR61bt07Xrl2zdz2aM2eOjh49qo8//ljPPvusevfurW+//VZlypTRm2++ecddLv5r/vz5+vnnn5UnD7vOAQAAIH2mZor79u2rHTt2qHv37nrkkUcUGBgoLy+vNPs2atQoU+detmyZypcvr5YtW1qP+fr6qkePHpo4caL27t2rWrVqpXuOkydPavz48Ro2bJi++OKLTF0fAAAA7sdUKG7evLksFosMw9D48ePT7RseHm7zea9du6YTJ06oc+fOqdpCQ0MlKcNQnJiYqDFjxqhSpUp6/PHHCcUAAADIkKlQ3K1bN1ks9n80cGRkpAzDUMmSJVO1FS9eXJJ05syZdM/x+eef6/Dhw1qyZIny5s3Ss0kyZBiGYmJiHHqNZLGxsSn+156c9RlyAkeOM25hjJ2DcXY8xtg5GGfHy81jbBiGzZnVVGrMaHbYrOQ1yt7e3qnafHx8JKX/Bdu/f78+//xzPf/886pYsaJDarxdfHx8pmbC7SEiIsLu53T2Z8gJHDHOSIkxdg7G2fEYY+dgnB0vt47xnZb4/leWp1Lj4+P1559/KjY2VoUKFVKpUqXk6elp6lxJSUmSlGaiT77B7k5p/8aNGxo9erTuuecep23B5unpqcqVKzvlWrGxsYqIiFBgYGCaPzSk7690W0NCQswXlstkbZxhC8bYORhnx2OMnYNxdrzcPMbHjh2zua/pUBwTE6OJEydqyZIlunHjhvW4r6+vOnfurNGjR8vX1zdT50zuf/v5kiXPEPv7+6f53vfff1///POPPvzwQ12+fDlFW3x8vC5duqR8+fJluqb0WCwW6wy2s3h7e9v9ms7+DDmBI8YZKTHGzsE4Ox5j7ByMs+PlxjHOzHJfU6H4xo0b6tu3rw4cOKACBQqoevXqKl68uKKiorR3717NmzdPBw4c0HfffZepWeMyZcpIurW2+L/OnTsnSWmuN5akTZs2KTY2VmFhYanadu/erUaNGiksLMxhSz+ys+R9iCfO2aXNu/9S344h6t46yMVVAQAAZB+mQvHXX3+tAwcOKCwsTC+++GKK2du4uDi99dZb+v777zVr1iwNGjTI5vP6+fkpMDBQ+/fvT9WWfKxGjRppvnfChAlpPlDkscceU8WKFTV27FjrzXru6lRklCSpXIm0Z9sBAADclalQvHr1at199916++23Uz0cw8vLS6+++qp2796tFStWZCoUS1Lnzp01efJkbdmyRc2bN5d06yl3CxYsUKVKlaxbs/1X3bp10zxusVhUsGBBNW7cOFN15DaJSYb+OnddklTurgIurgYAACB7MfW4t1OnTqlhw4Z3fFqch4eHGjZsqJMnT2b63AMGDFDZsmU1cuRIffLJJ/r222/Vu3dvnT17Vi+99JJ1bcju3bu1bNkyXbhwwcxHcDuRF6MVn5AkL08PlSiSu9YLAQAAZJWpUOzl5aWoqKh0+0RFRcnDwyPT5/bz89PcuXPVunVrzZ07VxMmTJC3t7e++uqrFLO98+fP15gxY3T8+PFMX8Mdnfzn1nZ3ZUv4KU8e++8xDQAAkJOZWj4RGhqqjRs36u+//07zxrczZ85ow4YNql69uqmiSpQooYkTJ6bbZ/z48TbdNJfW+mR3xHpiAACAOzM1U9yvXz9FRUWpf//+WrNmjS5evKi4uDidPn1aixcvVp8+fXT9+nX17dvX3vXCpFP/mylmPTEAAEBqpmaK7733Xj355JOaPHmynnnmmVTthmFo+PDhatWqVZYLhH38G4qZKQYAAPgv0w/vePLJJ9W8eXN9//33OnTokK5fvy5fX19Vq1ZNDz74oGrVqmXHMpEViYlJ/+48wfIJAACAVLL0mOcaNWrccd9gZB9/X4xWQmKS8nl5qHhhdp4AAAD4L5tCcVJSUort15KSkmy+wJ22bYPzJC+dKFucnScAAADSYlMorlatmp588kkNHz7c+toWFotFhw4dMl8d7OJUJDfZAQAApMemUFyyZMkUj3JOaxs2ZF/Wm+xYTwwAAJAmm0Lxhg0b0n2N7O3UP//bo5idJwAAANJkasHv0qVLdfjw4XT77Nq1S5MnTzZVFOwnITFJZ87/b+cJlk8AAACkyVQoHjt2rNavX59un/Xr12v69OmmioL9/H0hWgmJhvJ7eahYIW9XlwMAAJAt2bR8YunSpdq2bVuKYz/99JNOnjyZZv+EhARt2bIlxTpkuEbyTXZlS/iz8wQAAMAd2BSK69atq1deeUVxcXGSbu0qceTIER05ciTd9z3++ONZrxBZwpPsAAAAMmZTKC5btqwWLFigqKgoGYahfv36KSwsTGFhYan6WiwW5c2bVyVLltRdd91l94KROdab7EqwnhgAAOBObH6iXZUqVax/DgsLU5s2bVS/fn2HFAX7+XePYmaKAQAA7sTUjXbvvvuuWrVqpc2bN2vfvn0p2l577TW2bMsmEhKTdNa68wShGAAA4E5MheKbN2/qiSee0OOPP65NmzZZj8fGxmrevHkaPny4nn/++Uw9Dhr2d/b8dSUkGvLOx84TAAAA6TEVir/55htt3LhRLVq0UMeOHa3Hvb29tWTJErVt21bLly/XzJkz7VUnTLh95wmLhZ0nAAAA7sRUKF6+fLmqV6+uqVOnqnLlyinaQkJC9Mknn6hKlSpauHChXYqEOf8+3pmb7AAAANJjKhSfOXNGDRo0uGO7xWJRw4YNdfr0adOFIevYjg0AAMA2pkKxr6+vzpw5k26fc+fOycfHx1RRsI9Tkf/bjo1QDAAAkC5TobhevXpav369Dh48mGb74cOHtW7dOtWtWzdLxcG8+IQknT0fLYnlEwAAABmxeZ/i2w0ZMkTr169Xnz599PDDD6tmzZry9/fXtWvXtHfvXi1cuFCJiYk80c6Fzp6/rsQkQz758yqgUH5XlwMAAJCtmQrF1apV04cffqiXXnpJM2bMSLGzgWEY8vPz08SJE1WjRg27FYrMSV5PzM4TAAAAGTMViiWpbdu2aty4sTZv3qzw8HBduXJFvr6+qlKlitq0aSM/Pz971olMOpm8nrgE64kBAAAyYjoUS7duuOvYsWOKvYqRPfy78wTriQEAADJi6ka7/zIMQ0lJSUpKSlJiYqJu3rypyMhIzZs3zx6nhwmnI9mODQAAwFamZoqTkpI0ceJELV26VJcvX063b8+ePU0VBvPiExJ19sKtnSfKE4oBAAAyZCoUf/PNN/r6669lsVgUEBCgixcvyt/fX56enrp06ZKSkpJUpEgR9enTx971wgZnzkcrKcmQb/68KlKAnScAAAAyYvoxzz4+Plq1apV+/vln1alTR61atdLWrVu1detWtW7dWlFRUWrSpIm964UNTv2T/NCOAuw8AQAAYANToTgiIkJt27ZVxYoVJUmhoaHatWuXJKlIkSL66KOPFBAQoC+//NJ+lcJmPN4ZAAAgc0yF4oSEBN11113W14GBgTp9+rRiY2MlSV5eXmrRooUOHz5snyqRKaci/92jGAAAABkzFYoDAgJ04cIF6+uyZctKko4fP249VqBAAZ07dy6L5cEM6/IJQjEAAIBNTIXiunXrat26dTp16pQkKSgoSJK0adMma5/du3erUKFCWS4QmRMXn6i//7fzBMsnAAAAbGMqFA8cOFDR0dHq0qWL1q1bp4CAADVr1kxTp07VCy+8oCFDhmjXrl2655577F0vMnDm/HUlGZKvtyc7TwAAANjIVCiuWrWqpk6dqnLlyll3Nxg7dqwKFy6sJUuW6Oeff1a5cuX07LPP2rVYZOxk8k12JfzZeQIAAMBGpvYpvnnzppo2baqmTZtaj1WsWFE//fSTtm3bpvz586tu3brKn5+ZSmf7dzs2lk4AAADYylQofuihh3TPPffolVdeSXHc29tbrVu3tkthMIft2AAAADLP1PKJkydPKl++fPauBXaQvB1b+RIFXFwJAABAzmEqFJcuXdq68wSyj5vxifrnIjtPAAAAZJap5RPvvvuuhg4dqtGjR6tDhw4qW7asvL290+ybvIcxHKPLqGVpHu/7+o+SpBUfdHVmOQAAADmSqVA8YMAAJSQkaOXKlVq5cuUd+1ksFh06dMh0cQAAAIAzmArF1atXt3cdAAAAgMvYFIqXLFmi6tWr6+6775YkzZ4926FFAQAAAM5k04127777rlasWGF93bdvXy1dutRRNQEAAABOZVMojouL09mzZ62vf/vtN/31118OKwoAAABwJpuWTwQFBWnVqlU6cuSIChcuLOnWkorffvst3fdZLBZ98803Wa8SAAAAcCCbQvG4ceM0fPhw/fHHH5Juhd0zZ87ozJkz6b7PYrFkvUIAAADAwWwKxTVr1tTPP/+sCxcu6ObNm2rTpo369eunvn37Oro+ZCB5H+LlPx/X9KUH1Ci0pF7sX9/FVQEAAOQsNm/JZrFYVKxYMUnSPffco5CQEJUuXdphhSFz/j5/60l2pQJ8XVwJAABAzmNqn2K2ZMt+zl64FYpLBvi5uBIAAICcx6bdJ5D9nb1wXZJUqhgzxQAAAJlFKM4F4hOSdO5SjCSWTwAAAJhBKM4FIi9FK8mQ8nt5qEiB/K4uBwAAIMchFOcC/64n9mUbPAAAABMIxbnAWevOE9xkBwAAYAahOBfgJjsAAICsIRTnAuxRDAAAkDWE4lwgeaaYPYoBAADMIRTncHHxiTp/JVYSyycAAADMIhTncP9cjJZhSN758qqQXz5XlwMAAJAjEYpzuOTt2EoVYzs2AAAAswjFORzbsQEAAGQdoTiHs27Hxs4TAAAAphGKc7i/b1s+AQAAAHMIxTnc2fPJM8UsnwAAADCLUJyD3YhL0IWrNyRJJVk+AQAAYBqhOAf752KMJMnX21MFfL1cXA0AAEDORSjOwf5dOsF2bAAAAFlBKM7BrHsUs54YAAAgSwjFOZh1ppidJwAAALKEUJyD/TtTTCgGAADICkJxDvZ38oM7irF8AgAAICsIxTlU7M0EXYq6KYmZYgAAgKwiFOdQyU+y8/fxkp8P27EBAABkBaE4hzp7gZvsAAAA7CVbhuILFy5o3LhxatasmWrVqqWePXvql19+sem9O3fu1KBBg1SvXj1Vr15dnTp10syZM5WUlOTgqp3r7HlusgMAALCXbBeKo6OjNWDAAK1cuVJhYWEaPXq0YmNjNXjwYG3dujXd9/7f//2f+vbtqxMnTmjw4MF64YUXVKJECb377rt6+eWXnfQJnOMsN9kBAADYTV5XF/Bfc+bM0dGjRzV16lS1bNlSktStWzd169ZNb775ptasWXPHp7e9+uqr8vf316JFi1SkSBFJUu/evfXUU09p4cKFevTRRxUSEuK0z+JIzBQDAADYT7abKV62bJnKly9vDcSS5Ovrqx49eigiIkJ79+5N830XL17U0aNH1aZNG2sgTnb//fdLknbt2uW4wp3sb55mBwAAYDfZaqb42rVrOnHihDp37pyqLTQ0VJK0d+9e1apVK1V7wYIF9eOPPypfvnyp2i5duiRJ8vDwsG/BLhIdG68r1/+3HRs32gEAAGRZtgrFkZGRMgxDJUuWTNVWvHhxSdKZM2fSfG/evHkVGBiY6nhiYqJmz54ti8Wi+vXr27VeV0leT1zIP5988nu6uBoAAICcL1uF4mvXrkmSvL29U7X5+PhIkmJjYzN1znfffVdHjx5Vly5dVKlSpawXeRvDMBQTE2PXc95J8ueOjY1VxJkrkqS7ing77fru4vZxhmMwxs7BODseY+wcjLPj5eYxNgzjjvei/Ve2CsXJ26alVbxhGHdsu5P33ntPs2fPVuXKlfXaa6/ZpcbbxcfHKzw83O7nTU9ERIT2HY6SJOX3iHP69d1FRESEq0vI9Rhj52CcHY8xdg7G2fFy6xh7edn2kLNsFYp9fW+tj71x40aqtuSfXvz9/TM8T1xcnF588UWtWLFCFStW1IwZM+TnZ/8b0jw9PVW5cmW7nzctsbGxioiIUGBgoBIPHpcUpSoVSykkpIJTru8ubh/ntH5jgaxjjJ2DcXY8xtg5GGfHy81jfOzYMZv7ZqtQXKZMGUm31hb/17lz5yQpzfXGt7t+/bqeeOIJ7dixQ6GhoZo2bVqq3SjsxWKxWJd1OIu3t7fOXb71Q0NgqcJOv7678Pb2ZmwdjDF2DsbZ8Rhj52CcHS83jnFmVhhkqy3Z/Pz8FBgYqP3796dqSz5Wo0aNO74/JiZGgwYN0o4dO9SiRQvNmjXLYYHYlXjEMwAAgH1lq1AsSZ07d9axY8e0ZcsW67Ho6GgtWLBAlSpVsm7NlpZXX31Ve/bsUfv27fXZZ5/lup92JOl6TLyuxcRLkkoWJRQDAADYQ7ZaPiFJAwYM0LJlyzRy5Ej1799fxYoV04IFC3T27FlNnz7dOg2+e/dunTp1Sk2aNFFAQIAOHjyo5cuXy8vLS40bN9aqVatSnbtq1aq6++67nf2R7Orvi7d2myhSIL/y58t2Xz4AAIAcKdulKj8/P82dO1cTJkzQ3LlzFR8frypVquirr75Sw4YNrf3mz5+vJUuWaNasWQoICNC2bdsk3brJ7tVXX03z3KNGjcrxofif/4Vilk4AAADYT7YLxZJUokQJTZw4Md0+48eP1/jx462vhwwZoiFDhji6NJdLninm8c4AAAD2k+3WFCN91pniAGaKAQAA7IVQnMP8fYnlEwAAAPZGKM5BDMO4baaY5RMAAAD2QijOQWJuJinmRoIk6S6WTwAAANgNoTgHuXjtViAOKOStfJ4eLq4GAAAg9yAU5yCX/heKuckOAADAvgjFOUjyTHGpYqwnBgAAsCdCcQ7CTDEAAIBjEIpzkIuEYgAAAIcgFOcQhmH8O1PM8gkAAAC7IhTnEFevxykuwZDFIt1V1MfV5QAAAOQqeV1dAO6sy6hlqY4ZhvTA8yslSSs+6OrskgAAAHIlZooBAADg9gjFAAAAcHuEYgAAALg9QjEAAADcHqEYAAAAbo9QDAAAALdHKAYAAIDbY5/ibOz2fYhjYmIUHh6ukJAQ+fjw8A4AAAB7YqYYAAAAbo9QDAAAALdHKAYAAIDbIxQDAADA7RGKAQAA4PYIxQAAAHB7hGIAAAC4PUIxAAAA3B6hGAAAAG6PUAwAAAC3RygGAACA2yMUAwAAwO0RigEAAOD2LIZhGK4uIif6v//7PxmGIS8vL6dczzAMxcfHy9PTUxaLxSnXdEeMs+Mxxs7BODseY+wcjLPj5eYxjouLk8ViUZ06dTLsm9cJ9eRKzv6msVgsTgvg7oxxdjzG2DkYZ8djjJ2DcXa83DzGFovF5szGTDEAAADcHmuKAQAA4PYIxQAAAHB7hGIAAAC4PUIxAAAA3B6hGAAAAG6PUAwAAAC3RygGAACA2yMUAwAAwO0RigEAAOD2CMUAAABwe4RiAAAAuD1CMQAAANweoRgAAABuj1AMAAAAt0coBgAAgNsjFAMAAMDtEYoBAADg9gjFOcCFCxc0btw4NWvWTLVq1VLPnj31yy+/uLqsHG/Pnj0KCQnRnj17UrX9+uuv6t27t+rUqaOGDRvqhRde0MWLF51fZA515MgRjRgxQg0aNFD16tXVtm1bffzxx4qLi0vRj3HOmlOnTunpp59W06ZNVbt2bfXp00dbt25N1Y9xto8bN27ovvvuU6tWrVK1McZZM3bsWAUHB6f53+1/RzPO5sXHx2vatGnq0KGDQkND1bZtW3344YeKiYlJ0c+dx9hiGIbh6iJwZ9HR0erZs6dOnTqlfv36qUSJElqwYIGOHj2q6dOnq2nTpq4uMUc6deqUevfurXPnzmn+/PmqVauWtW3btm0aMmSIKleurLCwMF29elUzZ87UXXfdpYULF8rX19d1hecAp06dUrdu3eTp6alevXqpePHi+vXXX7VmzRo1adJEX331lSwWC+OcRWfOnNGDDz4oSerbt6/8/f21ZMkSHTx4UJ988ok6dOggie9ne3rjjTc0d+5clS5dWhs2bLAeZ4yzLiwsTLGxsRo2bFiqtmbNmqlIkSKMcxY9+eSTWrt2re6//37VrVtXu3fv1tKlS9WyZUt9/vnn/L0sSQaytalTpxpBQUHGhg0brMeuX79utGnTxmjXrp2RlJTkwupypg0bNhj169c3goKCjKCgIGP37t3WtqSkJKNDhw5Gq1atjGvXrlmPb9q0yQgKCjI+++wzF1ScswwdOtSoVq2acfz48RTH3333XSMoKMhYu3Yt42wHzz77rBESEmIcOnTIeiw6Otpo3ry50apVK8Mw+H62py1bthjBwcFGtWrVjJYtW1qPM8ZZl5CQYISGhhpjx469Yx/GOWtWrlxpBAUFGR9++GGK42+++aYRFBRk7Nq1izE2DIPlE9ncsmXLVL58ebVs2dJ6zNfXVz169FBERIT27t3rwupynlGjRunxxx9XkSJF1KlTp1Tt+/fv14kTJ/TQQw/Jz8/PerxFixaqVKmSli1b5sxyc5zExET99ttvqlevnipWrJiirWvXrpKkXbt2Mc52YLFY1Lp1a4WEhFiP+fj4qGbNmvrrr790/fp1xtlOrly5ohdffFFhYWEqU6ZMijbGOOv+/PNP3bx5U0FBQXfswzhnzfz581W4cGE98cQTKY736dNHw4YNk5eXF2Ms1hRna9euXdOJEydUo0aNVG2hoaGSRCjOpGPHjmnEiBFaunSpKlSokKo9ee1azZo1U7WFhobqzz//1LVr1xxdZo6VJ08eLVu2TG+88UaqtkuXLkmSPDw8GGc7mDhxoj799NMUxxISEnTkyBEVLFhQvr6+jLOdvPLKK/L09NS4ceNStTHGWXfkyBFJUuXKlSVJMTExSkpKStGHcTYvISFBu3fvVoMGDZQvXz5JUmxsrBITE1W+fHk9/fTTql69OmMsQnG2FhkZKcMwVLJkyVRtxYsXl3RrXSFs9/333+vJJ5+0/sXwX//8848k6a677krVxphnzGKxqGzZsipXrlyqtm+++UaS1KBBA8bZzq5evapdu3bpiSeeUEREhJ566ilZLBbG2Q6WLl2qn376Se+++26K2bNkjHHWJYfidevWqUWLFqpdu7bq1Kmj0aNHW3+YZpzN++uvvxQXF6cyZcpo6dKlat++vWrVqqVatWpp9OjRunr1qiTGWJLyuroA3FnyT2Te3t6p2nx8fCTd+mkPtvPy8kq3/fr165L+Hd/bJX8d/nunLjI2Y8YMbd68WXXr1lWzZs20du1aSYyzvTzxxBPauXOnJKlNmzYKCwuTxPdzVp05c0ZvvfWW+vXrpwYNGqTZhzHOuuRQfPDgQT399NPy8fHR9u3bNX/+fO3bt08LFy5knLMgKipKkrRhwwZ9++23euyxx1S5cmXt2LFDc+fO1fHjxzVv3jzGWITibC3510cWiyVVm/G/TUPSaoN5tox5njz8giUzZs2apffee0/FihXTBx98IIlxtre+fftqwIAB2rlzp+bMmaMePXro22+/ZZyzICkpSWPHjlXx4sX17LPPpttPYoyzolOnTgoNDdVjjz1mnbho3769KlSooHfeeUfffPMN45wFyVthnjhxQjNnzlSjRo0kSW3btlXhwoU1adIkLVmyhDEWoThbS9765MaNG6nakmeI/f39nVpTbpc85mnNwCd/HRhz2xiGoYkTJ+rLL79UsWLFNHPmTOtSIMbZvtq3by/p1ixxxYoV9fLLL2v27NmMcxbMmDFDO3fu1LRp0xQdHa3o6GhJt0JwUlKSLl26JE9PT8bYDu6///40jz/yyCN6//339csvv1jvrWGcMy955rdixYrWQJysR48emjRpkrZt22ZdNuHOY5y7I38Ol3yXc2RkZKq2c+fOSVKa641hXkZjbrFYVKJECWeXlePExcXpueee05dffqly5crp22+/td5EIzHOjtS5c2dZLBYdOHCAcc6CTZs2KSkpSYMHD1ajRo2s/508eVJ///23GjVqpCeeeIIxdiAvLy/5+/srOjqacc6C5LAbEBCQqq1IkSKyWCy6fv06YyxmirM1Pz8/BQYGav/+/anako+ltTMFzKtevbqkW+P73zWE+/fvV6VKldK82Qb/SkxM1DPPPKN169YpNDRU06ZNU5EiRVL0YZyz5tKlS3rkkUdUs2ZNvf/++ynaYmJiZBiG8uXLxzhnwfPPP29di3m7F198UfHx8ZowYYIKFCig+Ph4SYyxWTExMerZs6cqVKigTz75JEXbpUuXdPnyZdWrV4/v5SwoUqSISpcurePHjyspKSnFEojTp0/LMAyVKVOGMRYzxdle586ddezYMW3ZssV6LDo6WgsWLFClSpWsW7PBPmrVqqUyZcpo/vz51l+XStLmzZt1/Phx6167uLNPP/1U69atU506dfTNN9+kCsQS45xVybM7P/74oyIiIlK0TZ06VdKt9YKMs3nVq1dX48aNU/2XP39+5cuXT40bN1b16tUZ4yzy8fGRp6en1q9fr8OHD6do++ijjyRJ3bp1Y5yzqFu3brp48aK+++67FMe//PJLSdJ9993HGIvHPGd7169ft34z9+/fX8WKFdOCBQv0xx9/aPr06WrcuLGrS8yxPv30U02ePDnVY543bNig4cOHKygoSA8//LAuXLigGTNmqEyZMpo3b17uf8xlFpw/f16tWrVSfHy8Ro0aZd3G53YVKlRQjRo1GOcs+vXXXzV48GAVKlRIvXr1UsGCBbVp0yZt2bJFHTp00McffyyLxcI421mHDh0UFxeX4jHPjHHWHDhwQI888oh8fHzUu3dvFSlSRBs3btTWrVvVrVs3vffee5IY56yIjY1Vr169dPjwYXXv3l1Vq1bV1q1btXbtWnXt2tX6Gyd3H2NCcQ4QGRmpCRMmaMuWLYqPj1eVKlU0cuRINWzY0NWl5Wh3CsXSrfWEU6ZM0ZEjR1SgQAE1b95czz77bJprsvCv1atX65lnnkm3z8MPP2x9uAfjnDX79u3T5MmTtXPnTsXFxalixYrq3r27evfuneJXpIyz/aQViiXGOKuSv5d37dqluLg4BQYG6uGHH1avXr34XraT69ev67PPPtMPP/yg8+fPq3Tp0urevbsGDhzIGP8PoRgAAABujzXFAAAAcHuEYgAAALg9QjEAAADcHqEYAAAAbo9QDAAAALdHKAYAAIDbIxQDAADA7RGKAQAA4PYIxQByvb/++kvBwcF65JFHXF2KaXv37lWPHj1Us2ZN3XPPPVq0aJGrS0IGwsPDtXbtWruec+zYsQoODtbixYvTbI+Pj9ewYcMUHBysjh076ty5c3a9PpCbEYoBIJtLSkrS8OHDtXfvXrVp00YPP/ywqlev7uqykI7NmzfrgQce0IEDB5x2zfj4eI0cOVIbNmxQUFCQZs+ereLFizvt+kBOl9fVBQAA0nfhwgWdP39ed999tz744ANXlwMbXLx4UUlJSU67Xnx8vJ5++mmtX79eVatW1ddff63ChQs77fpAbsBMMQBkc3FxcZJEyEGa4uPj9cwzz2jdunWqUaOGvvnmG75XABMIxQCyLHmdY2RkpCZOnKiWLVuqevXqatu2raZMmaKEhARr3/TW9+7YsUPBwcF67rnnrMc+/fRTBQcHa//+/fr888/VunVrhYaGqkOHDlq4cKEkaefOnXr00UdVq1YttWjRQq+//rquX7+eZq2//vqrevTooRo1aqhp06YaN26cIiMjU/VLTEzU7NmzFRYWppo1a6pu3brq16+ftmzZkqpvcHCwhg4dqvnz56tx48aqVatWis9wJ5s2bdKAAQNUt25dhYaGqlOnTpoyZYpiY2Otffr06aPWrVtLkn777TcFBwerT58+GZ775MmTGjdunFq0aKEaNWqoffv2mjBhgqKiolJ9zlmzZiksLEw1atRQ7dq11bNnTy1btizVOVu1aqWuXbvq9OnTGjlypO655x7Vrl1bgwYN0vHjx5WYmKhp06apTZs2qlmzprp06aIVK1akOEfy13Pnzp2aPHmyWrRooZo1a6pr166aP39+mp/l7NmzeuWVV9SiRQtVr15dTZo00ahRo3Ts2LEU/ZK/f2bOnKm1a9eqe/fuqlmzpurXr6+nn35ap06dSnXuK1eu6L333lObNm1UvXp1NW7cWM8++6yOHz9u+tx9+vTRCy+8IEmaOnWqgoODtWPHDknS5cuX9frrr6tDhw6qUaOGGjRooMGDB2vbtm13+lKmKyEhQc8++6zWrl2rOnXqaMaMGSpQoICpcwHujuUTAOxm+PDhOnPmjNq2bSsvLy+tXr1akyZNUmxsrE0hMT2vvPKKzp49q44dOyoxMVFLly7VuHHjdPz4cc2ePVvNmzdXr169tH79en377beKi4vT22+/neIcJ06c0ODBg1W7dm09+uij2rdvnxYuXKitW7fq+++/t66/TExM1PDhw7Vx40ZVrlxZ3bt3V2JiotauXashQ4boxRdfVL9+/VKce//+/dq+fbu6desmwzBUtWrVdD/P5MmT9emnn8rf31+tWrVSwYIF9euvv2rSpElav369Zs2aJT8/P4WFhalKlSqaNWuWSpcurbCwMJUuXTrdc+/evVuDBw9WdHS0WrRooYoVK+rAgQP68ssvtX37ds2ZM0c+Pj6Ki4vTsGHDtHXrVpUtW1YPPPCA4uLitHnzZo0ZM0a7du3SG2+8keLcly9f1sMPP6xSpUqpe/fuOnjwoLZu3arBgwerTp062rZtm9q1ayfDMLR06VI999xzKlmypOrVq5fiPO+++66OHj2qzp07K3/+/Fq/fr1eeeUVHT58WK+++qq1X3h4uPr166erV6+qUaNGuu+++xQREaFVq1Zp/fr1+uyzz9S4ceMU5165cqUOHDigli1bqn79+tq1a5d++OEH7dmzR2vXrpWnp6ck6dy5c+rVq5dOnz6tBg0aqF27djp//rzWrFmjjRs36ssvv1TdunUzfe6wsDD5+/tr/fr1qlevnho2bKjSpUsrLi5OgwcPVnh4uFq3bq22bdvqwoULWrNmjX755Rd98cUXat68ebpf29slJCTomWee0U8//aT69etr6tSp8vX1tfn9AP7DAIAsev75542goCCjXbt2xuXLl63HT548aVStWtWoV6+eERcXZxiGYZw+fdoICgoyevbsmeo8v/76qxEUFGSMGjXKemzSpElGUFCQUa9ePePMmTPW499++60RFBRkBAUFGfPmzbMev3LlilGrVi0jNDTUSEpKSnHNoKAgY/z48Smu+dFHHxlBQUHG2LFjrcdmzpxpBAUFGWPGjDHi4+NTnPu+++4zQkJCjGPHjlmPp1VHevbs2WMEBQUZbdq0MU6fPm09Hh8fbx3LV1991Xo8uf5HH300w3MnJSUZ7du3N0JCQoz169enaHv99deNoKAgY/bs2YZhGMYXX3xhBAUFGUOHDjViYmKs/S5evGjcf//9RlBQkLFq1Srr8ZYtWxpBQUHGiBEjrGNrGIbRvXt3IygoyKhfv74RGRlpPb5gwQIjKCjIePnll63Hkr+eVatWNfbu3Zvimp07dzaCgoKM33//3TAMw0hMTDQ6depkBAUFGUuWLEnxWTZv3mxUqVLFaNiwoREdHW0Yxr/fP0FBQSk+e1JSktGvXz8jKCjIWL16tfX4E088YQQFBRnz589Pce5Dhw4ZoaGhRsuWLa1f/8yee9GiRUZQUJDx4YcfWo9t2LDBCAoKMj766KMU1/v999+NoKAgY+DAgUZGkr8/FixYYIwYMcJa0+3XAWAOyycA2E3Pnj1VqFAh6+ty5cqpUqVKioqK0uXLl7N07k6dOqlUqVLW18kzeIUKFVKPHj2sxwsWLKjKlSvr5s2bOn/+fIpzFC5cWCNGjEhx7Mknn1RAQIBWr15tXbu7YMECeXh46OWXX1bevHlTnHvYsGFKTExMc0usDh062PRZkpd9PPPMMypTpoz1eN68efXiiy+qQIECWrJkibWezNi3b5/+/PNPtWzZUq1atUrR9vjjj2vIkCGqUKGCJGnRokXKkyePXn/9dXl7e1v7FSlSROPGjZOkNJc0DB48WBaLxfo6+WvRrVu3FLsd1KlTR5J05syZVOd44IEHVKNGjRTXfOaZZyRJS5culSTt2bNHf/zxh5o1a6Zu3bqleH/z5s3VrVs3Xbp0ST/99FOKtqCgoBSf3WKx6N5775Uk6zKHCxcuaP369QoNDU3x/SNJISEh6tq1q86cOaNffvkl0+e+E8MwJN2a/Y6JibEer1evnn766Sd9/vnn6b7/dp988ol+/PFHNW7cWPnz59e0adOsSzQAmMPyCQB2ExgYmOpY8vrG+Ph4u57bx8dHklSmTJkUAU2S8ufPL0mpQmW1atWs70uWN29ehYaGauPGjTpx4oTKlSunY8eOyc/PTzNmzEhVx8WLFyVJhw4dSnG8YMGCKliwoE2f5eDBg5Kke+65J1VbgQIFFBwcrN9//10nTpxQlSpVbDpnsuS6atWqlaqtePHi1mUs0dHRioiIUIUKFVSiRIlUfevUqSMPDw+Fh4enarvT16JcuXIpjufLl09S6q+DJDVq1CjNa97+GZL/t379+qn6SrfC5OLFixUeHp4iNCeH/tv5+/tL+vf78ODBgzIMQ3Fxcfr0009T9f/nn3+sNbRo0SJT576Txo0bKzAwUJs2bVKTJk3UoEEDNW7cWM2aNUvzvOk5f/682rVrpw8//FDz58/Xm2++qeeee07Lli1TkSJFMnUuALcQigHYTXIIul1yYE2eJTPrTmslvby8bD5HQEBAuueOiYnRtWvXJEnXr1/X5MmT73iuq1evpnidHMRtkXwTYHKY+q/kkHr7DXe2unLlSrrntrWGvHnzqkiRItbz3e6/P1gky8zX4q677kp1zM/PT5KsNwMmfy2Sj//XncYprTr++32Y/PU7cuSIjhw5csc6//t1tuXcd5I/f37Nnz9f06ZNs65b3rhxo6RbP7C99tprKWbP09O2bVt99NFHyps3r3r37q2NGzdq69atevHFFzV16lSbzgEgJUIxAKdKL0CYCYGZ8d+dF5Il7z5RsGBBa0CuVKmSVq9e7ZA6kkNeZGSkypcvn6o9OYjdvhTFVsn1JwfK/4qOjpavr6+1X1o7b0i3Hhhy/fp1UzXY4saNG6mOJX99kmc6bx+ntCT3z8o4PfLII3rttdcy/X6zChUqpDFjxmjMmDE6efKktm3bph9//FHbt2/X4MGDtWHDhjv+EHC7Vq1aWZf2WCwWvfvuu+rSpYs2btyomTNnqn///g7+JEDuw5piAE6VfOd/dHR0qraIiAiHXvvgwYOpHqhw/fp1HTlyRP7+/qpQoYL8/PxUtmxZnTp1Ks110AcPHtR7771nneEzI3lnit9//z1V282bN7V37175+vpmuMtEWpKXW+zbty9V25UrV1SvXj0NGjRIfn5+Kl++vM6dO5fmWth9+/YpNjZWwcHBma7BFnv37k11bNeuXZKkmjVrSvp3nHbu3JnmOX799VdJt9b5ZlZISIiktMdJklatWqWPP/5Yhw8fzvS5JaVa0iNJP//8s9566y2dPHlSklS+fHk98sgjmjlzpho0aKCrV6/q6NGjpq5XvHhx604hEydOtC7RAWA7QjEApypatKgKFSqk48ePW8OBJF26dElz58516LUjIyNTXMMwDE2cOFFRUVF66KGHlCfPrb8SH3roIcXHx+v1119PsR42JiZGr776qr7++us7zjrb4sEHH5QkTZkyRX/99Zf1eEJCgt5++21FRUWpc+fOmVqOkKxu3boqW7asNmzYkOomsc8//1xJSUlq1qyZtQ7DMPTGG2+kmKW/fPmydTu7sLCwTNdgi1mzZqX4IejChQv65JNP5OHhYR2f2rVrq1KlStq5c6f15sRk27Zt0+LFi1WoUCG1bNky09cvVaqUmjRpooMHD+qbb75J0XbixAm9/vrrmjZtmk2ztmlJnsW9fY/us2fPavbs2Zo+fXqKvsk3hebJk8fUD0LJ2rdvr7CwMMXHx+vZZ59N8wdPAHfG8gkATuXh4aGePXtq6tSpeuSRR9SxY0clJCToxx9/VOXKlTO8gz8rypUrp3feeUe//PKLKlSooJ07d2rfvn2qVq2annrqKWu/QYMG6ddff9UPP/yg8PBwNWnSRHny5NG6dev0999/q0OHDurcubPpOmrXrq3hw4drypQp6tatm1q1aqUCBQpox44dOnr0qKpVq6YxY8aYOreHh4fGjx+vIUOGaPDgwWrdurXKli2rvXv3ateuXapfv74effRRSdLAgQO1fft2/fzzz+rSpYuaNm2q+Ph4bd68WefPn1f37t2z9DnTEx8frwcffFDt2rWTp6enNmzYoPPnz+vZZ5+1znbnyZNHEyZMUP/+/TVu3DitXLlSISEhioiI0KZNm5QvXz598MEHpvfmffPNN9W7d2+98847+vHHH1WzZk1duXJFa9asUUxMjMaNG5did5DMKFmypCRZH4LStWtX3X///Zo/f76+//57HTlyRPXq1VN8fLy2bNmikydPauDAgWne9JgZL730kn777TdFRETo9ddf1/vvv5+l8wHuhFAMwOmeeuop+fj4aNGiRZo3b56KFy+unj17qm/fvmrYsKHDrlu3bl2NHTtWkyZN0tatW1W0aFENGjRIw4cPT3HzmKenp6ZPn665c+dq2bJlWrRokTw9PVW+fHkNGzZMDz74oDw8PLJUy1NPPaWqVatq1qxZWrdunZKSklSuXDmNGjVK/fv3NzVLnKxevXpauHChPvvsM23fvl0bN25UiRIl9Nhjj2nYsGHWWczkzzlnzhwtXbpUixcvlpeXl0JCQvTiiy+qY8eOWfqM6Xn66ad19uxZLVu2TDExMapSpYpee+01tWnTJkW/atWqafHixZo6dap+/vln7dy5U0WLFlW3bt00ZMgQVaxY0XQNpUuX1uLFi/XFF19o/fr1mj17tgoWLKg6depo4MCBatKkielz16tXT/369dOSJUs0Z84cBQYGKigoSF9//bVmzJihdevWad68eZJuPRFx2LBhqbadM8PPz0/vv/+++vTpo2XLlqlx48Z2OS/gDixGVm8JBwDARp9++qkmT56st956S927d3d1OQBgxZpiAAAAuD1CMQAAANweoRgAAABujzXFAAAAcHvMFAMAAMDtEYoBAADg9gjFAAAAcHuEYgAAALg9QjEAAADcHqEYAAAAbo9QDAAAALdHKAYAAIDbIxQDAADA7f0/VOwfJ2aFzogAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 800x550 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(np.cumsum(all_pca.explained_variance_ratio_), 'bs-');\n",
    "plt.xlabel('number of components K');\n",
    "plt.ylabel('fraction of variance explained');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3a: What K value will give you at least 80% of the variance? 95%?\n",
    "\n",
    "Specify a concrete K value, by carefully interpreting the plot above.\n",
    "\n",
    "Often, we use these heuristics to choose a K that strikes a balance between \"good\" reconstruction and small size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3b: Visualize reconstructions for the first 5 training set images with K=30.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO compute z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO compute xhat given z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO make plot of reconstructions xhat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise 3c: Visualize reconstructions for the first 5 *test* set images with K=30.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO compute z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO compute xhat given z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO make plot of reconstructions xhat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 4: Understanding what we mean by \"explained variance\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A goal of PCA is to find a low-dimensional reconstruction of our data that contains all the possible \"information\" of our original data.\n",
    "\n",
    "One way to define this precisely is to first look at squared error \"reconstruction cost\" of using just the empirical mean of our training data (this is like PCA with $K=0$)\n",
    "\n",
    "\\begin{align}\n",
    "\\text{cost}_{K=0}(X, m) &= \\frac{1}{N} \\sum_{n=1}^N \\sum_{f=1}^F (x_{nf} - m_f)^2 \\\\\n",
    "&= \\frac{1}{N} \\sum_{n=1}^N \\sum_{f=1}^F \\tilde{x}_{nf}^2, \\qquad \\tilde{x}_{nf} = x_{nf} - m_f\n",
    "\\end{align}\n",
    "\n",
    "Note that this cost is precisely equal to the *empirical variance* of the training set, since variance is the average squared error from the mean.\n",
    "\n",
    "This is just a property of the data itself. We can't change it. But it gives us a place to start when quantifying the quality of a reconstruction.\n",
    "\n",
    "Now, consider a reconstruction that uses the mean $m$ as well as a good $K$-dimensional weight component matrix $W$.\n",
    "\n",
    "We should be able to do better than the baseline above, right?\n",
    "\n",
    "Let's write the cost when we have $K>0$ principal components:\n",
    "\n",
    "\\begin{align}\n",
    "\\text{cost}_K(X, m, W) &= \\frac{1}{N}  \\sum_{n=1}^N \\sum_{f=1}^F (x_{nf} - m_f - \\sum_{k=1}^K W_{kf} \\tilde{x}_{nf} )^2 \\\\\n",
    "\\\\\n",
    "&= \\frac{1}{N}  \\sum_{n=1}^N \\sum_{f=1}^F (\\tilde{x}_{nf} - \\sum_{k=1}^K W_{kf} \\tilde{x}_{nf} )^2 \\\\\n",
    "\\end{align}\n",
    "\n",
    "Now, this cost will have units that depend on the scale of the original data. However, we can look at the *ratio* of this cost to the original mean-only cost above.\n",
    "\n",
    "\\begin{align}\n",
    "\\text{frac-unexplained-variance} = \\frac{\\text{cost}_K(X, m, W)}{\\text{cost}_{K=0}(X,m)}\n",
    "    &= \\frac\n",
    "        { \\frac{1}{N}  \\sum_{n=1}^N \\sum_{f=1}^F (\\tilde{x}_{nf} - \\sum_{k=1}^K W_{kf} \\tilde{x}_{nf} )^2 }\n",
    "        { \\frac{1}{N}  \\sum_{n=1}^N \\sum_{f=1}^F \\tilde{x}_{nf}^2 }\n",
    "\\end{align}\n",
    "\n",
    "Notice that as $K \\rightarrow 0$, this ratio approaches 1, while as $K \\rightarrow F$ this ratio approaches 0 on the training set. It turns out that this ratio will *monotonically decrease* as we increase $K$ and iteratively find the optimal weights $W$ for that K value. \n",
    "\n",
    "We can thus interpret this fraction as a measure of the *unexplained* variance of our original dataset. If the fraction is close to 1., most of our data is not well approximated by our PCA reconstructions. If the fraction is close to 0, we are doing a good job (most of the variance is accounted for).\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Surprising result: We can compute this fraction simply using sums of eigenvalues we already have!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can show a surprising result. The denominator can be computed as the sum of all eigenvalues of the covariance of $X$. And the numerator can be computed as a partial version of this sum.\n",
    "\n",
    "$$\n",
    "\\text{frac-unexplained-variance} = \\frac{\\text{cost}_K(X, m, W)}{\\text{cost}_{K=0}(X,m)}\n",
    "    = \\frac\n",
    "        { \\sum_{\\ell={K+1}}^F \\lambda_{\\ell} }\n",
    "        { \\sum_{f={1}}^F \\lambda_{f} }\n",
    "$$\n",
    "\n",
    "where $\\lambda_k$ is the $k$-th largest eigenvalue of the $F \\times F$ covariance matrix of the training data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Computing the denominator (PCA with K=0 reconstruction cost) in 3 ways"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Method 1: Directly compute reconstruction cost with K=0 model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m_1F = np.mean(x_tr_NF, axis=0)[np.newaxis,:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "costK0_v1 = np.sum(np.mean(np.square(x_tr_NF - m_1F), axis=0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4.686065871686287"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "costK0_v1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Method 2: Compute cost by measuring empirical variance of each feature and adding up"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cov_FF = np.cov(x_tr_NF.T, bias=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "var_F = np.diag(cov_FF)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "costK0_v2 = np.sum(var_F)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4.686065871686281"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "costK0_v2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Method 3: Compute cost as sum of eigenvalues of covariance matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lam_F, W_FF = np.linalg.eig(cov_FF)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get eigenvalues in sorted order from big to small"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sorted_ids = np.argsort(-1 * lam_F)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lam_F = lam_F[sorted_ids]\n",
    "W_FF = W_FF[:, sorted_ids]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.69601696, 0.64133098, 0.55282934, 0.39584765, 0.26867765,\n",
       "       0.2312219 , 0.20297662, 0.17096015, 0.15846922, 0.14244372])"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lam_F[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "costK0_v3 = np.sum(lam_F)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4.686065871686284"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "costK0_v3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Verify all three methods are the same!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4.686065871686287"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "costK0_v1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4.686065871686281"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "costK0_v2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4.686065871686284"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "costK0_v3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Computing the numerator (PCA with K-components reconstruction cost) in 2 ways"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "K = 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "xtilde_NF = x_tr_NF - m_1F"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Method 1: Direct reconstruction cost calculation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "W_KF = W_FF[:, :K].T.copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compute xhat given x in 2 steps:\n",
    "\n",
    "* project each x to z\n",
    "* then project each z back to xhat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Step 1: x to z\n",
    "z_tr_NK = np.dot(xtilde_NF, W_KF.T)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Step 2: z to xhat\n",
    "xhat_tr_NF = np.dot(z_tr_NK, W_KF) + m_1F"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now compute the total reconstruction cost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "costK_v1 = np.sum(np.mean(np.square(x_tr_NF - xhat_tr_NF), axis=0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.7958885969409755"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "costK_v1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Method 2: Calculation just summing all eigenvalues except the top-K"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "costK_v2 = np.sum(lam_F[K:])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.795888596940976"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "costK_v2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.allclose(costK_v1, costK_v2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bringing it all together: Fraction of variance explained"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thus, a measure of our reconstruction quality on the training set is given by the fraction of variance that is explained by our $K$-component PCA\n",
    "\n",
    "Note, we switch here to using fraction-of-explained-variance (rather than unexplained).\n",
    "\n",
    "$$\n",
    "\\text{frac-explained-variance} = 1 - \\frac{\\text{cost}_K(X, m, W)}{\\text{cost}_{K=0}(X,m)}\n",
    "    = \\frac\n",
    "        { \\sum_{\\ell={1}}^K \\lambda_{\\ell} }\n",
    "        { \\sum_{f={1}}^F \\lambda_{f} }\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "K"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Version 1: Ratio of direct reconstruction costs (PCA with K to PCA with 0 components)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "frac_explained_var = 1 - (costK_v1 / costK0_v1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.4033612259200122"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frac_explained_var"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Version 2: Ratio of sums of eigenvectors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "frac_explained_var_v2 = np.sum(lam_F[:K]) / np.sum(lam_F)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.40336122592001183"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "frac_explained_var_v2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Version 3: Check against sklearn implementation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.40336122592001167"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.sum(all_pca.explained_variance_ratio_[:K])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "python3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}