PLSA.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "PLSA.ipynb",
      "provenance": [],
      "authorship_tag": "ABX9TyOT3mBL3Aw1B0bK0bTD8npf",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/GiuliaLanzillotta/exercises/blob/master/PLSA.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fhh1qqPY1IVm",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "from sklearn.feature_extraction.text import CountVectorizer\n",
        "import scipy\n",
        "import math\n",
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "\n",
        "DEBUG = True"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0jo7-owh2u5k",
        "colab_type": "text"
      },
      "source": [
        "# PLSA\n",
        "\n",
        "The paper : https://arxiv.org/ftp/arxiv/papers/1301/1301.6705.pdf\n",
        "\n",
        "> #### We will use **Probabilistic latent semantic analysis** to extract the topics from a corpus of documents. We'll use  a preprocessed dataset of documents from the Associated Press.\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "Before diving into the code let's talk about the model we're using here. <br>The PLSA model is a probabilistic model over a corpus. It identifies *each document as a mixture of topics* (latent or unobserved variables), and *each topic as a multinomial distribution* over the words in the vocabulary. Below is a sketch of the model:\n",
        "$$ d \\rightarrow z \\rightarrow w $$\n",
        "where $d$ is a document in the corpus, $z$ is the latent variable topic and $w$ is the random variable word. \n",
        "<br>\n",
        "**The main drawback of this approach**? The inability to generalise to new documents, which derives from the fact that this model does not take into account a generative hypothesis for documents as well (look for LDA to see a complete generative model).\n",
        "<br>\n",
        "<br>\n",
        "Enough with the presentation, let's get started. \n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dV41aL4EcPsm",
        "colab_type": "text"
      },
      "source": [
        "## Data loading and pre-processing "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Loy172BLcUR_",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "c1c8195b-1826-4d2d-e18b-18c740824d21"
      },
      "source": [
        "# Note : I have uploaded the data on the hosted runtime, and un-packed it with the folliwing \n",
        "# commands. You need the data in your execution directory to be able to run these lines.\n",
        "!ls"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "associated-press  associated-press.tar\tsample_data\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "G6a32KkRcmjz",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "! gunzip associated-press.tar.gz && tar -xvf associated-press.tar"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "n-Z412c3coJB",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        },
        "outputId": "096f6942-535d-4ac1-c8e1-6834c8ac39c0"
      },
      "source": [
        "!ls && ls associated-press"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "associated-press  associated-press.tar\tsample_data\n",
            "doc.dat  test.dat  train.dat  voc.dat\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "n4A0C8P4cskj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# The data is organised it 4 different sets \n",
        "# We'll open the training set now\n",
        "train_docs = []\n",
        "with open(\"associated-press/train.dat\", 'r') as f:\n",
        "    for line in f.readlines():\n",
        "        train_docs.append(line)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bt57NMRsdGJQ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        },
        "outputId": "d3b79f35-071d-4f80-ca59-85a2f84e6b5a"
      },
      "source": [
        "train_docs[1]"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'work posit reach reach peopl year thought million million million presid govern year year first percent state appear abl time billion told resid complet day develop made made made made wednesday wednesday jerusalem investig compani compani compani compani parti parti group group anonym effect financ payment make spokesman isra isra think assur rappaport rappaport rappaport rappaport rappaport rappaport rappaport construct foreign foreign foreign foreign offici offici offici offici offici offici offici offici public take gener septemb septemb believ polit partner ask spoke earlier go oil oil oil oil oil oil war minist minist sell promi say came deni deni man specif mention attorney attorney person relay offer sale sale submit inc idea friend friend put acknowledg speak improp vice israel israel thing border border ministri ministri ministri mee mee mee secret come iii think refer close close shimon possibl relat focu handl handl repr wrongdo project pere pere pere pere pere pere pere pere pere pere pere pere pere labor engin lead condit al press iraq iraq franciscoba indic offer offer offer offer offer offer offer offer offer mark robert point tom san san san knowledg e seriou propo refer crazi arrang fund discount consent keep portion concern termin third post minist bruce dispo town plan import receiv edwin swiss swiss nativ see iraqi iraqi bomb bomb propo thing say senior port pipelin pipelin pipelin pipelin pipelin telephon francisco francisco conduct directli sourc wallach wallach flynn answer wage bechtel bechtel bechtel bechtel bechtel bechtel bechtel run memo memo memo memo memo knew comment associ comment jordan jordan quot denounc discuss behalf built arrang israel israel israel israel israel israel\\n'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oMXpL_vfdNMV",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# As we can see the text has already been pre-processed (tokenization, stemming)\n",
        "# Now we'll transform each dataset in a term-document matrix \n",
        "vectorizer = CountVectorizer(vocabulary=None)\n",
        "train_X = vectorizer.fit_transform(train_docs)\n",
        "# by defaul the Vectorizer builds a sparse matrix\n",
        "# being the dataset relatively small we can afford working \n",
        "# with the entire matrix in memory \n",
        "train_X = train_X.toarray()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PCvopEODdlwZ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 139
        },
        "outputId": "d088f3bd-5b68-4ece-a8f4-12e205956cc1"
      },
      "source": [
        "train_X"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[0, 0, 0, ..., 0, 0, 0],\n",
              "       [0, 0, 0, ..., 0, 0, 0],\n",
              "       [0, 0, 0, ..., 0, 0, 0],\n",
              "       ...,\n",
              "       [0, 0, 0, ..., 0, 0, 0],\n",
              "       [0, 0, 0, ..., 0, 0, 0],\n",
              "       [0, 0, 0, ..., 0, 0, 0]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Hv74dgpRdzxD",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "8f2a0535-2d3a-4d12-b1e7-cffacdc4e6ad"
      },
      "source": [
        "# as you can see the matrix is very sparse \n",
        "train_X.shape"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(2000, 6756)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pJQ_b828eZv2",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "4f1fdc8b-60d1-4c44-e5e8-c7f5943fdaa9"
      },
      "source": [
        "total_entries = train_X.shape[0]*train_X.shape[0]\n",
        "observed_entries = sum(sum(train_X))\n",
        "sparseness_factor = observed_entries/total_entries\n",
        "sparseness_factor"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.097353"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c24I9ABGece0",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "n_docs = train_X.shape[0]\n",
        "n_terms = train_X.shape[1]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yFsyWVojfKgs",
        "colab_type": "text"
      },
      "source": [
        "## Model building "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a220AhUvfuKQ",
        "colab_type": "text"
      },
      "source": [
        "\n",
        "\n",
        "> One mln dollar question: how many topics will be covered in 2k documents from the associated press?\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HQS16OzkfFKe",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# The first, and also the hardest modeling choice to make here is the \n",
        "# number of topics, which must be known before-hand.\n",
        "# Of course, we can use cross-validation to sort out the best option,\n",
        "# but for the sake of the implementation we'll now set it to an \n",
        "# arbitrary constant. \n",
        "n_topics = 20"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8RUTrlUlgY-s",
        "colab_type": "text"
      },
      "source": [
        "#### Here we initialise our model.\n",
        "\n",
        "<br> \n",
        "*Components*<br>\n",
        "The PLSA model is made of 3 matrices: $U$, $V$ and $Q$. \n",
        "<br>The matrix $U$ relates the documents to the topics: $u_{i}$ is the row vector that expresses the mixture components in the $i$-th document. Similarly, the matrix $V$ relates the words to the topics: $v_{j}$ is the row vector whose entries express the probability of the $j$-th word to have been generated by the relative topic. The underlying assumption is that each word has been generated by only one topic. Finally the matrix $Q$ contains the posterior probabilities for each topic: the entry $q_{ijz}$ expresses the probability that the $j$-th word in the $i$-th document belongs to the topic $z$.\n",
        "<br><br>\n",
        "*Constrants*<br>\n",
        "The initialisation of the matrices has to take into account the constraints that they have to satisfy. Each row of the above matrices must sum up to one and each entry must be non-negative, since the vectors in these matrices are probability vectors."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0ByBlxltfnE8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def init(n_topics, n_docs, n_words):\n",
        "  U = np.random.rand(n_docs,n_topics)\n",
        "  V = np.random.rand(n_words, n_topics)\n",
        "  Q = np.random.rand(n_docs,n_words,n_topics)\n",
        "  # Constraint \n",
        "  U = U/np.sum(U,axis=1).reshape(-1,1)\n",
        "  V = V/np.sum(V,axis=1).reshape(-1,1)\n",
        "  Q = Q/np.sum(Q,axis=2).reshape(Q.shape[0],-1,1)\n",
        "  return U,V,Q"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mqDnhlVvk1i0",
        "colab_type": "text"
      },
      "source": [
        "### EM Inference \n",
        "Implementing the **Expectation-Maximization** algorithm applied to PLSA. <br>\n",
        "We are using the Expectation-Maximization algo to maximize an otherwise intractable distribution. More specifically we are maximing the lower bound obtained via Jensen inequality."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Do8zNwALlNvB",
        "colab_type": "text"
      },
      "source": [
        "***Expectation step***<br>\n",
        "Here we update our posterior probabilities according to the current estimates for the matrices U and V. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "luOIiHXfktWs",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def compute_expectation(U,V,Q):\n",
        "  # a constant to avoid dividing by 0 \n",
        "  eps=1e-2\n",
        "  # update the matrix\n",
        "  Q = np.einsum('ik,jk -> ijk', U,V)\n",
        "  denominator = np.maximum(eps,np.einsum('ik,jk -> ij', U,V))\n",
        "  Q = Q/denominator.reshape(Q.shape[0],-1,1)\n",
        "  # check simplex constraint\n",
        "  assert np.all(np.einsum('ijk -> ij', Q) - 1 < eps), \"The entries do not sum up to 1\"\n",
        "  # check the non-negativity constraint\n",
        "  assert np.any(Q<0) == False, \"There are negative entries\"\n",
        "  return Q"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QhZSy68pnrMe",
        "colab_type": "text"
      },
      "source": [
        "***Maximization step***<br>\n",
        "Here we update new estimates for the matrices U and V, according to the responsibilities we've computed before"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7s7GCA0Em9kN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def compute_maximization(U,V,Q,X):\n",
        "  # a constant to avoid dividing by 0 \n",
        "  eps=1e-2\n",
        "\n",
        "  U = np.einsum('ij, ijz -> iz', X,Q)\n",
        "  denominator = np.maximum(eps,np.einsum('ij -> i', X))\n",
        "  U = U/denominator.reshape(-1,1)\n",
        "  # check simplex constraint\n",
        "  assert np.all(np.einsum('ik -> i', U) - 1 < eps), \"The entries do not sum up to 1\"\n",
        "  # check the non-negativity constraint\n",
        "  assert np.any(U<0) == False, \"There are negative entries\"\n",
        "\n",
        "  V = np.einsum('ij, ijz -> jz', X,Q)\n",
        "  denominator = np.maximum(eps,np.einsum('ij , ijz-> j', X, Q))\n",
        "  V = V/denominator.reshape(-1,1)\n",
        "  # check simplex constraint\n",
        "  assert np.all(np.einsum('jk -> j', V) - 1 < eps), \"The entries do not sum up to 1\"\n",
        "  # check the non-negativity constraint\n",
        "  assert np.any(V<0) == False, \"There are negative entries\"\n",
        "\n",
        "  return U,V\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4FwsBS44qiwg",
        "colab_type": "text"
      },
      "source": [
        "Here are some utility functions that we'll be used in the training loop"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yO4R5C59nMq2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "def compute_lower_bound(U, V, Q, X): \n",
        "  # a constant to avoid dividing by 0 \n",
        "  eps=1e-4\n",
        "  lb = np.einsum('ijk,ik -> ij', Q,np.log(np.maximum(U,eps))) \\\n",
        "      + np.einsum('ijk,jk -> ij', Q,np.log(np.maximum(V,eps))) \\\n",
        "      - np.einsum('ijk,ijk -> ij', Q,np.log(np.maximum(Q,eps)))\n",
        "  \n",
        "  lb = np.einsum('ij,ij -> ',X,lb)\n",
        "\n",
        "  return lb\n",
        "\n",
        "def compute_loglikelihood(U, V, X): \n",
        "  # a constant to avoid dividing by 0 \n",
        "  eps=1e-4\n",
        "  \n",
        "  ll = np.einsum('ik,jk -> ij', U,V)\n",
        "  ll = np.einsum('ij,ij -> ',X,np.log(np.maximum(ll,eps)))\n",
        "\n",
        "  return ll"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VNYkncjMtlEB",
        "colab_type": "text"
      },
      "source": [
        "### The training loop "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nfOBXDvAuX3l",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "training = True"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WPN7Uu2OvjkM",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from datetime import datetime"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "t7S3gA4KsyeV",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 634
        },
        "outputId": "919f2afd-4715-46c9-976c-a3606816961c"
      },
      "source": [
        "start = datetime.now()\n",
        "if training:\n",
        "  U,V,Q = init(n_topics, n_docs, n_terms)\n",
        "  n_itrs = 25 # number of iterations \n",
        "  lowerbounds = np.zeros(n_itrs)\n",
        "  loglikelihoods = np.zeros(n_itrs)\n",
        "  for i in range(n_itrs):\n",
        "      print(\"EM iteration {}\".format(i))\n",
        "      Q = compute_expectation(U, V, Q)\n",
        "      U, V = compute_maximization(U, V, Q, train_X)\n",
        "      lowerbounds[i] = compute_lower_bound(U,V,Q,train_X)\n",
        "      loglikelihoods[i] = compute_loglikelihood(U,V,train_X)\n",
        "end = datetime.now()\n",
        "duration = end-start\n",
        "print(\"Total time = \", duration)"
      ],
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "EM iteration 0\n",
            "EM iteration 1\n",
            "EM iteration 2\n",
            "EM iteration 3\n",
            "EM iteration 4\n",
            "EM iteration 5\n",
            "EM iteration 6\n",
            "EM iteration 7\n",
            "EM iteration 8\n",
            "EM iteration 9\n",
            "EM iteration 10\n",
            "EM iteration 11\n",
            "EM iteration 12\n",
            "EM iteration 13\n",
            "EM iteration 14\n",
            "EM iteration 15\n",
            "EM iteration 16\n",
            "EM iteration 17\n",
            "EM iteration 18\n",
            "EM iteration 19\n",
            "EM iteration 20\n",
            "EM iteration 21\n",
            "EM iteration 22\n",
            "EM iteration 23\n",
            "EM iteration 24\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "error",
          "ename": "AttributeError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-33-959659570b3b>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     13\u001b[0m \u001b[0mend\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdatetime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnow\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     14\u001b[0m \u001b[0mduration\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mend\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0mstart\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 15\u001b[0;31m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Total time =\"\u001b[0m\u001b[0;34m,\u001b[0m  \u001b[0mduration\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstrftime\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"%H:%M:%S\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;31mAttributeError\u001b[0m: 'datetime.timedelta' object has no attribute 'strftime'"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "57SvRox7Ex_N",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "680770c7-8ebf-4625-de2a-1e6e6e179bee"
      },
      "source": [
        "rint(\"Total time = \", duration)"
      ],
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Total time =  0:05:15.502507\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1pDiP-X0t6yw",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 385
        },
        "outputId": "60332178-9560-4111-fad3-2a49ccae73a3"
      },
      "source": [
        "# Now some plotting to make us all happy to be here \n",
        "iters = np.arange(n_itrs)\n",
        "plt.figure(figsize=(12,6))\n",
        "plt.plot(iters,lowerbounds,'b', label='lower bound')\n",
        "plt.plot(iters,loglikelihoods,'g', label=\"log likelihood\")\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "execution_count": 41,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAswAAAFwCAYAAACsMS2JAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjAsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8GearUAAAgAElEQVR4nOzdeZzN1R/H8dcxq2Vs2YoatNmXDNmV\n0EKkIlooCaVEIZJfaLO22NeQJCKyRYgsCUOSpUhZI4xlMGO2e35/3CExc+8wc+/cmXk/H4953Hu/\n9/P9ft94lE+n8z3HWGsREREREZGkZUvvACIiIiIivkwNs4iIiIiIC2qYRURERERcUMMsIiIiIuKC\nGmYRERERERfUMIuIiIiIuODzDbMx5lNjzDFjzPYU1rc0xuw0xuwwxnzh6XwiIiIikrkZX1+H2RhT\nFzgHfGatLeem9nZgFlDfWnvKGFPIWnvMGzlFREREJHPy+RFma+1q4OTlx4wxtxpjlhhjNhtj1hhj\nSiV+9QIwylp7KvFcNcsiIiIikio+3zAnYzzwirW2CtAdGJ14/A7gDmPMOmPMT8aYB9ItoYiIiIhk\nCv7pHeBaGWNyATWBr4wxFw8HJb76A7cD9wDFgNXGmPLW2tPezikiIiIimUOGa5hxjoqfttZWSuK7\nQ8AGa20c8JcxZjfOBnqTNwOKiIiISOaR4aZkWGsjcTbDLQCMU8XEr+fhHF3GGFMA5xSNP9Mjp4iI\niIhkDj7fMBtjZgDrgTuNMYeMMc8DTwHPG2N+AXYAzRLLlwIRxpidwEqgh7U2Ij1yi4iIiEjm4PPL\nyomIiIiIpCefH2EWEREREUlPaphFRERERFzw6VUyChQoYIsXL57eMUREREQkE9u8efMJa23B5L73\n6Ya5ePHihIeHp3cMEREREcnEjDH7XX2vKRkiIiIiIi6oYRYRERERcUENs4iIiIiICz49hzkpcXFx\nHDp0iAsXLqR3FEkUHBxMsWLFCAgISO8oIiIiImkuwzXMhw4dIiQkhOLFi2OMSe84WZ61loiICA4d\nOkSJEiXSO46IiIhImstwUzIuXLjADTfcoGbZRxhjuOGGGzTiLyIiIplWhmuYATXLPkZ/HiIiIpKZ\nZciGOb3lypUrXe57zz33eHVd6n379lGuXDmv3U9ERETEF6lh9mEJCQnpHUFEREQky0uThtkY84Ax\n5ndjzB/GmF5JfB9kjJmZ+P0GY0zxtLhverPW0qNHD8qVK0f58uWZOXMmAJ07d2b+/PkANG/enHbt\n2gHw6aef0qdPHwA+//xzqlWrRqVKlejYseOl5jhXrly8/vrrVKxYkfXr1191z2nTplGpUiXKlSvH\nxo0bATh58iSPPPIIFSpUoHr16mzbtg2Afv36MXTo0EvnlitXjn379rFv3z5Kly7NCy+8QNmyZWnU\nqBHR0dEAbN68mYoVK1KxYkVGjRrlid82ERERkQwl1atkGGP8gFFAQ+AQsMkYM99au/OysueBU9ba\n24wxrYBBwBOpvXfXrrB1a2qv8l+VKsHHH6es9uuvv2br1q388ssvnDhxgqpVq1K3bl3q1KnDmjVr\naNq0KYcPH+bIkSMArFmzhlatWrFr1y5mzpzJunXrCAgI4KWXXmL69Om0adOG8+fPc/fddzNs2LAk\n7xkVFcXWrVtZvXo17dq1Y/v27bz99ttUrlyZefPm8f3339OmTRu2uvmN2bNnDzNmzGDChAm0bNmS\nOXPm8PTTT/Pcc88xcuRI6tatS48ePa7p905EREQkM0qLZeWqAX9Ya/8EMMZ8CTQDLm+YmwH9Et/P\nBkYaY4y11qbB/dPN2rVrad26NX5+fhQuXJh69eqxadMm6tSpw8cff8zOnTspU6YMp06d4siRI6xf\nv57hw4czdepUNm/eTNWqVQGIjo6mUKFCAPj5+fHYY48le8/WrVsDULduXSIjIzl9+jRr165lzpw5\nANSvX5+IiAgiIyNdZi9RogSVKlUCoEqVKuzbt4/Tp09z+vRp6tatC8AzzzzDt99+m7rfJBERydKs\ntVgs1loc1pHke4slweHA4bA4rCUhwfnqsPbSMYfDYi2J51523DqPO6zFOiwJjn+P2SvqHNaC5dIx\n50fnMWeWf18diS2KddirjyW+XnUNa7E4r39lnfMzcNl9Lv8uqQxcek363pf/Hl+8/sXrXjw/Jccv\n78Yub82Suse/ny/Gvvzk//7ZO644J6lrXVlR887bua/ybUmel57SomEuChy87PMh4O7kaqy18caY\nM8ANwIkrL2aM6QB0ALjllltc3jilI8HeVrRoUU6fPs2SJUuoW7cuJ0+eZNasWeTKlYuQkBCstbRt\n25YPPvjgqnODg4Px8/NL9tpXrkjhaoUKf39/HA7Hpc+XL/0WFBR06b2fn9+lKRkiIvIvh3UQmxB7\n1U9MfMwVn2OJio3h/IXYSz9RMbFExcYSFRNDTFw88QkJxCXEE5eQ8N/3jgTiE5zfxzsSiHfEE+9I\nICHxJ94RT4JNfG/jcdiExM/O4w4SSLDxWOvAgQN78ediQ3r5sct+uPidufyY8/PF77j8u4vHE1+5\n9Hqx1jo/X/zO2MRjIilXf/cA7qvcN71jXMXnNi6x1o4HxgOEhYX59D9pderUYdy4cbRt25aTJ0+y\nevVqhgwZAkD16tX5+OOP+f7774mIiODxxx/n8ccfB+C+++6jWbNmdOvWjUKFCnHy5EnOnj1LaGio\n23vOnDmTe++9l7Vr15InTx7y5MlDnTp1mD59On379mXVqlUUKFCA3LlzU7x4cRYuXAjAli1b+Ouv\nv1xeO2/evOTNm5e1a9dSu3Ztpk+fnsrfIRGRtOGwDqLjoomKiyI6PvE1LjrJ91Fx0URGRREZHU1k\ndDRnL0Rx7kI052OjOR8bdek6FxKiiUmIJs4RS7yNJd7GEE8sCcTiIJYEE4s18Z79hVkDDn9w+IH1\nc746/F2+N/hhrD8m8X026+88RjYMgYmv2TCYy95nI9vF9yZb4jcX32e77L0h28Vjl71mS/zOmH+/\nN8YkHrt4XhKfzb912f5z3Fx9zcTjBoOfyQaGS98bk/grShwkuvKY8zwSr3X1sX+zOr8HLtVduo/z\nypjEzxgu1V48/+J5F4/9+53rz5ef95/7Xfzu4t3N1RlM4uekrnX5oNmV1098+5+aS5mSOH758FuS\n1wVMtisG7ZKov3IY7+qBvot15qpjABVKFMMXpUXDfBi4+bLPxRKPJVVzyBjjD+QBItLg3umqefPm\nrF+/nooVK2KMYfDgwRQpUgRwNtPfffcdt912G6GhoZw8eZI6deoAUKZMGd59910aNWqEw+EgICCA\nUaNGpahhDg4OpnLlysTFxfHpp58Czof72rVrR4UKFciRIwdTp04F4LHHHuOzzz6jbNmy3H333dxx\nxx1urz958mTatWuHMYZGjRpd72+NiGRxDuvgXOw5zsacJTImkrOxztfImMhLxy4ePxUVScS5s5w8\nH8npaOf3Z2MjiU44R6yNJs5GkWBirzOIH8TlgLjsztf47Je9D4G4QpAQBAmBZLOB+JtA/AgkyAQR\nYAIJyBaIf7ZAAv0CCcwWRJC/832QfyDBAYEEBwQlvgaSPTCQHIGB5AgKIntgIDmDnT85ggLIHhRA\noL8/QYF+BPr7ERzoT1CAHwEBhoAA8Pd3/rh6n03rWomkG5PaacSJDfBu4D6cjfEm4Elr7Y7LajoD\n5a21nRIf+nvUWtvS3bXDwsLslesO79q1i9KlS6cqs6Q9/bmIZF4X4i9w4MwB9p/ez/4z+zlw5gCn\nok8RGftv83s25iynoiOJvBDJubiznI8/m7KLxwdBTAjE5Hb+xF7+PuelJtff5iAwW3aCsmUn2C8H\nwX7ZyR6QnRwBOcgZmJ2cQdnJFZSDkODshGTPTp4cOciTIzu5cwWQIwfkzMl/Xi++z57d+RMYqIZU\nJCszxmy21oYl932qR5gT5yS/DCwF/IBPrbU7jDEDgHBr7XxgEjDNGPMHcBJoldr7iohI2jhz4Qz7\nz+y/1BBfek18/8/5f/57gjX4x+clW5yzuU2Izk1CVD6ICU1sdq9ugANtbkKCQsibPTf5c+TmhpAQ\nCuYOoWD+IPLlg/z5ueo1d+5/m1o1syKSntJkDrO1djGw+Ipj/7vs/QWgRVrcS0REUs5ay7Hzx1w2\nxGdizvznnCC/IAoF3UKO2FByn2qCY38oJ/4IxZ4KhTOhFMlZlNBiAf9tcEtc3fBe/hocnE6/ASIi\nacDnHvoTEZHrs/P4Tubumstfp/+61AwfOHOAmISY/9TlDspNaJ5QQvOGUqNoHQLOhxL1dyjHdofy\nx+ZQdv9ciIMJziHdwoWhWhiENYMqVZw/N92UHr86EZH0o4ZZRCQDS3AksHD3QkZsHMGKv1YAUDhn\nYULzhlKpSCWa3dmM0LyhhOYJ5cYcoZw7FMrv2/IQHg7h4bBkO8QnLgRRoACEhcHjvZ2vVapA0aL/\nfYJdRCQrUsMsIpIBnYw+yaQtkxgdPpp9p/dRLHcx3qv/Hi/c9QIFcxYkNhZ27HA2xeFzYMpm2LYN\n4uKc5+fP72yIe/RwNsdhYXDzzWqORUSSooZZRCQD2fbPNkZsGMH0X6cTHR9NvdB6DG04lGalmvHX\nXn/69YRNm5zNcUziTIw8eZwN8WuvOZvksDAoXlzNsYhISum54+uQK1euNLnOs88+y+zZswFo3749\nO3fuvObrT5kyhZdffhmAsWPH8tlnnwFwzz33cOWSfGnBU9cVkeTFO+KZvXM29abUo+LYikz/dTpP\nV3iaXzr9wqpnV/Fo6ceYNMGfSpVgyhTnyhKvvAJffgl79sCpU7B8OQwcCC1aQIkSapZFRK6FRph9\nxMSJE1N9jU6dOqVBEhHxFcfPH2f85vGM3TyWQ5GHKJ63OEMaDqFd5Xbkz54fgKNH4fnnYfFiaNgQ\nJk92zjsWEZG0oxHmVLDW0qNHD8qVK0f58uWZOXMmAA6Hg5deeolSpUrRsGFDHnrooUsjyclJauT2\nxIkT1KhRg0WLFnH8+HEee+wxqlatStWqVVm3bt1V1+jXrx9Dhw699Pmrr76iWrVq3HHHHaxZswaA\nCxcu8Nxzz1G+fHkqV67MypUrXR6Pjo6mVatWlC5dmubNmxMdHX39v2EikiLhf4fTdl5bin1UjLdW\nvkWpAqX4ptU3/PHKH3Sv2f1Sszx3LpQrB99/D8OHw5IlapZFRDwhQ48wd13Sla1Ht6bpNSsVqcTH\nD3ycotqvv/6arVu38ssvv3DixAmqVq1K3bp1WbduHfv27WPnzp0cO3aM0qVL065du2vK8c8//9C0\naVPeffddGjZsyJNPPkm3bt2oXbs2Bw4c4P7772fXrl0urxEfH8/GjRtZvHgx/fv3Z/ny5YwaNQpj\nDL/++iu//fYbjRo1Yvfu3ckeHzNmDDly5GDXrl1s27aNu+6665p+HSKSMrEJsczeOZsRG0fw06Gf\nyBWYixfueoHOVTtTuuB/d9GMjISuXZ2jyXfdBdOmQZky6RRcRCQLyNANc3pbu3YtrVu3xs/Pj8KF\nC1OvXj02bdrE2rVradGiBdmyZaNIkSLce++913TduLg47rvvPkaNGkW9evUAWL58+aU5zgCRkZGc\nO3fO5XUeffRRAKpUqcK+ffsuZX7llVcAKFWqFKGhoezevTvZ46tXr6ZLly4AVKhQgQoVKlzTr0VE\nXDty9gjjNo9j3OZxHD13lNvz384nD3xC24ptyROc56r6NWugTRs4cAD69IH//c+5rbOIiHhOhm6Y\nUzoSnNH4+/tTpUoVli5deqlhdjgc/PTTTwRfw3ZZQUFBAPj5+RF/caFVEUl31lp+OvQTIzaOYPbO\n2cQ54njo9od4pdorNLq1EdnM1bPlYmPh7bdh0CDnQ3tr1kDNmukQXkQkC9Ic5lSoU6cOM2fOJCEh\ngePHj7N69WqqVatGrVq1mDNnDg6Hg3/++YdVq1Zd03WNMXz66af89ttvDBo0CIBGjRoxYsSISzVb\nt17fVJQ6deowffp0AHbv3s2BAwe48847kz1et25dvvjiCwC2b9/Otm3bruu+IgIX4i8wdetUqk6o\nSs1Pa7JozyI6V+3Mnlf2sOjJRTxw2wNJNsvbt0O1as5VLp5/HrZuVbMsIuJNGXqEOb01b96c9evX\nU7FiRYwxDB48mCJFivDYY4+xYsUKypQpw80338xdd91FnjxX/69VV/z8/JgxYwZNmzYlJCSE4cOH\n07lzZypUqEB8fDx169Zl7Nix15z5pZde4sUXX6R8+fL4+/szZcoUgoKCkj3+4osv8txzz1G6dGlK\nly5NlSpVrvmeIgJjw8fSd2VfTkSdoEzBMox+aDTPVHyGXIHJLyPpcMAnn0Dv3pA7N3zzDTRt6sXQ\nIiICgLHWpneGZIWFhdkrV47YtWsXpUuXTuYM33Hu3Dly5cpFREQE1apVY926dRQpUiS9Y3lMRvlz\nEUkPU7ZO4blvnuPe4vfSp04f6peoj3GzEPLBg/Dss84VMB5+GCZOhEKFvJNXRCSrMcZsttaGJfe9\nRpg9pEmTJpw+fZrY2Fj69u2bqZtlEUnekj+W0H5+exqUbMCiJxcR6Of6CT1rYcYMeOkliI+HCROc\n0zC00YiISPpRw+wh1zpvWUQyn81/b+bxWY9TvnB55rSc47ZZPnnS2SjPnAk1ajiXi7v1Vi+FFRGR\nZOmhPxERD/jz1J889MVDFMhRgEVPLiJ3UG6X9cuWQYUKMGcOvPcerF6tZllExFdkyIbZl+ddZ0X6\n8xD5rxNRJ3jg8weId8Sz5Okl3BRyU7K10dHw6qvQqJHzwb4NG+DNN8Ff//9PRMRnZLiGOTg4mIiI\nCDVpPsJaS0RExDWtDy2SmUXFRdHkiyYcjDzI/FbzKVWgVLK1W7ZAlSrOba27dIHNm50794mIiG/J\ncGMYxYoV49ChQxw/fjy9o0ii4OBgihUrlt4xRNJdvCOeVrNbsfHwRua0nEOtW2olXRcPgwc7NyIp\nVAi++w4aNvRyWBERSbEM1zAHBARQokSJ9I4hIvIf1lo6L+rMgt0LGPngSJqXbp5k3d69zq2tf/wR\nnngCRo+G/Pm9HFZERK5JhpuSISLii95b8x7jt4ynd+3edK7W+arvrYVJk6BSJdixA6ZPdy4fp2ZZ\nRMT3qWEWEUmlyT9Ppu/KvjxT4Rneq/9ekjWdOkH79s4trn/9FZ58Umsri4hkFGqYRURS4ds93/LC\nghdoWLIhE5tOTHIHv2++gfHj4bXXnMvH3XxzOgQVEZHrpoZZROQ6hf8dTouvWlChcIVkNyY5edI5\nulypEgwcCNn0b10RkQwnwz30JyLiC/ae3EvjLxpf2pgkJCgkybquXeHECfj2WwgI8HJIERFJExrr\nEBG5RsfPH+eB6f9uTHJjyI1J1i1Y4Nze+s03nSPMIiKSMWmEWUTkGpyPPU+TGU04FHmIFW1WJLsx\nyalT0LEjlC8Pffp4OaSIiKQpNcwiIikU74in1ZxWhP8dzpyWc6h5c81ka197DY4dg4ULIfDqqc0i\nIpKBaEqGiEgKWGt5adFLLNy9kJEPjuSRUo8kW7t4MUyZAr16aatrEZHMQA2ziEgKvLP6HSZsmcCb\ntd/kxaovJlt3+jR06ABly0Lfvl4MKCIiHqMpGSIibnz686e8vept2lRsw7v133VZ+/rrcPQozJsH\nQUFeCigiIh6lEWYRERcW71lMhwUdaHRrIyY+nPTGJBctXQqffgo9e0JYmBdDioiIR6lhFhFJxqbD\nmy5tTDK7xWwC/JJfSPnMGefW16VLw//+58WQIiLicZqSISKShIsbkxTKWYjFTy1OdmOSi3r0gL//\nhh9/hOBgL4UUERGv0AiziMgVjp0/xv2f30+CTWDJU0sokquIy/ply2DCBOjeHe6+20shRUTEazTC\nLCJymfOx52nyRRMOnz3M922+584Cd7qsP3vWORWjVCno399LIUVExKvUMIuIJIp3xPPE7CfYfGQz\nX7f8mho313B7Ts+ecPAgrFunqRgiIpmVGmYREZwbk7y48EUW7VnEmMZjaFaqmdtzVqyAsWOdS8nV\ncN9bi4hIBqU5zCIiwIAfBjDx54n0qdOHTmGd3NafO+ecinHHHfDOO14IKCIi6UYjzCKS5U3cMpF+\nP/SjbcW2vHNvyrrfXr1g/35YswayZ/dwQBERSVcaYRaRLG3HsR10WtiJ+2+9nwkPT3C5MclFq1bB\nqFHw6qtQq5bnM4qISPpSwywiWVqf7/uQMzAnnz/6ucuNSS46fx7atYNbb4X33vNCQBERSXepapiN\nMfmNMcuMMXsSX/MlUVPJGLPeGLPDGLPNGPNEau4pIpJW1h9czze/f0OPmj0okKNAis7p3Rv27YPJ\nkyFHDs/mExER35DaEeZewApr7e3AisTPV4oC2lhrywIPAB8bY/Km8r4iIqliraXXil4UylmIrtW7\npuic1athxAh45RWoU8fDAUVExGektmFuBkxNfD8VeOTKAmvtbmvtnsT3fwPHgIKpvK+ISKos3buU\n1ftX07duX3IF5nJbHxXlnIpRsiS8/74XAoqIiM9I7SoZha21RxLfHwUKuyo2xlQDAoG9Lmo6AB0A\nbrnlllTGExG5msM66L2iN8XzFqdDlQ4pOqdPH9i7F1auhJw5PRxQRER8ituG2RizHCiSxFd9Lv9g\nrbXGGOviOjcC04C21lpHcnXW2vHAeICwsLBkrycicr1m7ZjF1qNbmdZ8GoF+gW7r162DTz6Bzp3h\nnns8n09ERHyL24bZWtsgue+MMf8YY2601h5JbIiPJVOXG1gE9LHW/nTdaUVEUikuIY63vn+L8oXK\n07pca7f10dHw3HMQGgoDB3ohoIiI+JzUTsmYD7QFBia+fnNlgTEmEJgLfGatnZ3K+4mIpMqknyex\n99ReFrRegF82P7f1ffvCnj3ObbBzuZ/qLCIimVBqH/obCDQ0xuwBGiR+xhgTZoyZmFjTEqgLPGuM\n2Zr4UymV9xURuWZRcVH0/6E/tW6uRePbG7ut//FH+PBD6NQJ6tf3QkAREfFJqRphttZGAPclcTwc\naJ/4/nPg89TcR0QkLQzfMJyj547yVYuv3O7oFx3tXBXj5pth8GAvBRQREZ+U2ikZIiIZwqnoUwxa\nN4jGtzem9i213db36we//w7LlkFIiOfziYiI79LW2CKSJQxaN4gzF87w/n3uF1HesAGGDoUXXoAG\nyT72LCIiWYUaZhHJ9A5HHuaTDZ/wZPknqVC4gsvaCxecq2IULepsmkVERDQlQ0QyvXdWv0O8I54B\n9w5wW9u/P+zaBUuWQO7cXggnIiI+TyPMIpKp7YnYw8QtE+lYpSMl85V0Wbtpk/MBv+efh/vv91JA\nERHxeWqYRSRT67uyL0H+QbxV9y2XdTExzqkYN94Iw4Z5KZyIiGQImpIhIpnWliNbmLljJn3q9KFI\nriIua995B3bsgEWLIE8eLwUUEZEMQSPMIpJpvbniTfJnz0+Pmj1c1m3e7Nz2+tln4aGHvJNNREQy\nDjXMIpIprfxrJUv3LqV37d7kCU5+yDguzjkVo3Bh565+IiIiV9KUDBHJdKy19F7Rm6IhRelctbPL\n2pkz4ddfYfZsyJfPSwFFRCRDUcMsIpnON79/w4bDG5jw8ASyB2RPts7hcE7FKFcOmjf3YkAREclQ\n1DCLSKaS4EjgzRVvcscNd/BspWdd1i5e7HzQb9o0yKYJaiIikgw1zCKSqUzbNo1dJ3bxVYuv8M/m\n+l9xAwdCaCg88YSXwomISIakhllEMo0L8Rd4e9XbVLmxCo+Vfsxl7dq1sG4djBgBAQFeCigiIhmS\nGmYRyTTGho/lwJkDTGo6CWOMy9qBA6FAAWjXzkvhREQkw9KsPRHJFCJjInlvzXvcV+I+GpRs4LJ2\n2zbnBiWvvgo5cngpoIiIZFhqmEUkU/hw/YeciDrBB/d94LZ28GDIlQs6u15xTkREBFDDLCKZwLHz\nxxi2fhiPlX6MqkWruqz96y/48kvo2FHrLouISMqoYRaRDO/9Ne8TFRfFu/XfdVs7bJhzCblu3bwQ\nTEREMgU1zCKSoe07vY8x4WN4rtJzlCpQymXtsWMwaRK0aQNFi3opoIiIZHhqmEUkQ+u3qh8Gw9v1\n3nZbO3w4xMRAjx5eCCYiIpmGGmYRybC2H9vOZ798xsvVXubmPDe7rD17FkaNcm6BfeedXgooIiKZ\nghpmEcmw3vr+LUKCQuhdu7fb2vHj4fRpeOMNLwQTEZFMRQ2ziGRI6w+u55vfv6FHzR7ckOMGl7Ux\nMfDhh1C/PlSr5qWAIiKSaWinPxHJcKy19FrRi0I5C9G1ele39Z9/Dn//DVOmeD6biIhkPmqYRSTD\nWbp3Kav3r2bEgyPIFZjLZW1CgnOjkrvuggauNwAUERFJkhpmEclQHNZB7xW9KZ63OB2qdHBbP28e\n7N4Ns2aBMV4IKCIimY4aZhHJUGbtmMXWo1uZ1nwagX6BLmuthYED4bbb4NFHvRRQREQyHTXMIpJh\nxCXE8db3b1G+UHlal2vttv777yE83LlChp+fFwKKiEimpIZZRDKMST9PYu+pvSxovQC/bO474IED\n4cYbnTv7iYiIXC8tKyciGUJUXBT9f+hP7Vtq0/j2xm7rw8Nh+XLo1g2CgrwQUEREMi2NMItIhjB8\nw3COnjvKVy2+wqTg6b1BgyBPHujY0QvhREQkU9MIs4j4vFPRpxi0bhBN7mhC7Vtqu63fvRvmzIHO\nnSF3bi8EFBGRTE0Ns4j4vEHrBnHmwhneq/9eiuqHDHFOw+jSxcPBREQkS1DDLCI+7XDkYT7Z8AlP\nVXiKCoUruK8/DFOnQrt2ULiwFwKKiEimp4ZZRHzaO6vfIcGRQP97+qeo/uOPweGA7t09HExERLIM\nNcwi4rP2ROxh4paJdKzSkZL5SrqtP3UKxo6FJ56AEiW8EFBERLIENcwi4rM+WPsBgX6B9KnbJ0X1\no0fDuXPwxhseDiYiIlmKGmYR8UlHzh7h822f81yl5yiSq4jb+qgo+OQTeOghqOB+qrOIiEiKqWEW\nEZ80cuNI4h3xdKvRLUX1kyfD8ePQq5eHg4mISJajhllEfM752POMCR/DI6Ue4bb8t7mtj4tzLiVX\nsybUdr9Ms4iIyDXRTn8i4kB4ZPsAACAASURBVHMmb53MqQun6F4zZUtdzJoF+/fDiBGQgk0ARURE\nrkmqR5iNMfmNMcuMMXsSX/O5qM1tjDlkjBmZ2vuKSOaU4Ejgo58+okaxGtS8uabbemth4EAoWxYa\nN/ZCQBERyXLSYkpGL2CFtfZ2YEXi5+S8A6xOg3uKSCY177d5/HnqT16v8XqK6hcvhu3bnStjZNMk\nMxER8YC0+OulGTA18f1U4JGkiowxVYDCwHdpcE8RyaSGrh9KyXwleaRUkv8qucrAgXDLLdCqlYeD\niYhIlpUWDXNha+2RxPdHcTbF/2GMyQYMA9xOSDTGdDDGhBtjwo8fP54G8UQko/jx4I/8dOgnulXv\nhl82P7f1a9c6f7p3h4AALwQUEZEsKUUP/RljlgNJLYT6n90ErLXWGGOTqHsJWGytPWTcPJFjrR0P\njAcICwtL6loikkkN/XEo+YLz8Vyl51JUP2gQFCgAzz/v4WAiIpKlpahhttY2SO47Y8w/xpgbrbVH\njDE3AseSKKsB1DHGvATkAgKNMeestVoxVUQA+OPkH8z7bR69a/cmZ2BOt/Xbt8PChTBgAOTI4YWA\nIiKSZaXFsnLzgbbAwMTXb64ssNY+dfG9MeZZIEzNsohc7qP1HxHgF8DL1V5OUf3gwZAzJ3Tu7OFg\nIiKS5aXFHOaBQENjzB6gQeJnjDFhxpiJaXB9EcnkIqIimLx1Mk+Vf4obQ250W79/P3zxBXToAPnz\neyGgiIhkaakeYbbWRgD3JXE8HGifxPEpwJTU3ldEMo8x4WOIjo9O8VJyw4Y5l5B77TUPBxMREUFb\nY4tIOrsQf4ERG0fwwG0PULZQWbf1x4/DxInw9NNQrJgXAoqISJanhllE0tX0bdM5dv4Y3WukbBvs\nESPgwgXo0cPDwURERBKpYRaRdOOwDoatH0bFwhWpX6K+2/qzZ2HkSHjkEShd2gsBRURESJtVMkRE\nrsuSP5aw68QupjWfhrs12gEmTIBTp5zbYIuIiHiLRphFJN0M/XEoRUOK8kTZJ9zWxsQ4H/a79164\n+24vhBMREUmkEWYRSRdbjmxh5b6VDG4wmAA/9/taT58Of/8Nkyd7IZyIiMhlNMIsIuli2PphhASG\n0KFKB7e1CQnOjUoqV4aGDb0QTkRE5DIaYRYRrzt45iAzt8+ky91dyBOcx239N9/A77/DzJmQgqnO\nIiIiaUojzCLidZ9s+ASAV+9+1W2ttTBwINx6Kzz2mKeTiYiIXE0jzCLiVWcunGH85vG0KNuC0Lyh\nbutXroRNm2DcOPDz80JAERGRK2iEWUS8auKWiZyNPZvibbAHDoQiRaBNGw8HExERSYYaZhHxmriE\nOD7Z8An1QusRdlOY2/rNm2HZMujWDYKDvRBQREQkCZqSISJe89XOrzgYeZDRjUenqH7wYMidGzp2\n9HAwERERFzTCLCJeYa1l6I9DufOGO3no9ofc1u/dC7Nnw4svQh73C2mIiIh4jEaYRcQrVu1bxc9H\nf2Z8k/FkM+7/W/3DD8HfH7p08UI4ERERFzTCLCJeMXT9UArmKMgzFZ9xW3v8OHz6KTzzDNx0kxfC\niYiIuKCGWUQ8bufxnSzes5iXq71MsL/7p/dGjoQLF6B7dy+EExERcUMNs4h43IfrPyTYP5gXw150\nW3v+vLNhbtYMSpXyQjgRERE31DCLiEcdPXeUadum8WzFZymYs6Db+k8/hZMnoWdPL4QTERFJATXM\nIuJRozaOIi4hjm41urmtjY+HYcOgVi2oWdML4URERFJAq2SIiMdExUUxOnw0Te9syh033OG2/quv\nYP9+GD7cC+FERERSSCPMIuIxU7ZO4WT0SbrXdP/0nrXOjUpKlYImTbwQTkREJIU0wiwiHpHgSOCj\nnz6iWtFq1Lq5ltv65cth61aYNAmy6T/lRUTEh6hhFhGPmP/7fP44+QezHp+FMcZt/eDBzjWXn3rK\nC+FERESugcZxRMQjhq0fRvG8xWleurnb2i1bnCPMXbtCUJAXwomIiFwDNcwikubWH1zPuoPr6Fa9\nG/7Z3P+PrCFDIHdu6NDBC+FERESukRpmEUlzw9YPI29wXtpVbue29q+/YNYs6NQJ8uTxQjgREZFr\npIZZRNLU3pN7mfvbXDpV6USuwFxu6z/8EPz84NVXvRBORETkOqhhFpE09fFPH+Nn/Hjl7lfc1p44\n4VwV45lnnA/8iYiI+CI1zCKSZk5Gn+TTrZ/yZPknuSnEfQc8ahRER0N398s0i4iIpBs1zCKSZsaG\njyUqLorXarzmtjYqCkaMgKZNoXRpL4QTERG5TmqYRSRNxMTHMGLjCBrd2ogKhSu4rZ88GSIioGdP\nL4QTERFJBW1cIiJp4otfv+DouaNMfWSq29r4eBg2DGrWhFruNwEUERFJV2qYRSTVrLUMWz+M8oXK\n07BkQ7f1c+Y4l5P76CMvhBMREUklNcwikmpL9y5lx/EdTGk2xe022NY6t8G+8054+GEvBRQREUkF\nNcwikmrD1g/jppCbaF2+tdva7793boU9cSJk01MUIiKSAeivKxFJla1Ht7L8z+V0qdaFQL9At/WD\nB0ORIvD0014IJyIikgbUMItIqny4/kNyBuSkQ5UObmu3boXvvoOuXSEoyAvhRERE0oAaZhG5boci\nDzFj+wza39WefNnzua0fMgRCQqBjRy+EExERSSNqmEXkug3fMByHdfDq3a+6rd23D2bOdDbLefN6\nPpuIiEhaUcMsItclMiaScZvH8XiZxymRr4Tb+g8/dD7k96r73lpERMSnqGEWkesyacskImMieb3G\n625rT5xwrorx1FNQrJgXwomIiKShVDXMxpj8xphlxpg9ia9JTmI0xtxijPnOGLPLGLPTGFM8NfcV\nkfQVEx/DRz99RO1balOtaDW39aNHQ3Q0dO/uhXAiIiJpLLUjzL2AFdba24EViZ+T8hkwxFpbGqgG\nHEvlfUUkHU36eRIHIw/St25ft7VRUTBiBDRpAmXLeiGciIhIGkttw9wMmJr4firwyJUFxpgygL+1\ndhmAtfactTYqlfcVkXRyIf4C7695n1o310rRNthTpjinZPTs6flsIiIinpDahrmwtfZI4vujQOEk\nau4AThtjvjbG/GyMGWKM8UvugsaYDsaYcGNM+PHjx1MZT0TS2oTNEzh89jD97+nvdhvs+HgYNgyq\nV4fatb0UUEREJI253RrbGLMcKJLEV30u/2CttcYYm8w96gCVgQPATOBZYFJS97PWjgfGA4SFhSV1\nPRFJJ9Fx0Xyw9gPqhtalfon6buu//hr+/BOGDgU3vbWIiIjPctswW2sbJPedMeYfY8yN1tojxpgb\nSXpu8iFgq7X2z8Rz5gHVSaZhFhHfNW7zOI6cO8KMx2a4HV221rkN9h13QNOmXgooIiLiAamdkjEf\naJv4vi3wTRI1m4C8xpiCiZ/rAztTeV8R8bKouCgGrh3IvcXvpV7xem7rV66EzZudK2P4JTsJS0RE\nxPeltmEeCDQ0xuwBGiR+xhgTZoyZCGCtTQC6AyuMMb8CBpiQyvuKiJeN2TSGf87/Q/97+qeofvBg\nKFwYnnnGw8FEREQ8zO2UDFestRHAfUkcDwfaX/Z5GVAhNfcSkfRzPvY8g9YNokHJBtQJreO2/pdf\nYOlSeP99CA72QkAREREP0k5/IuLWqE2jOB51PMWjy0OGQK5c0KmTh4OJiIh4gRpmEXHpbMxZBq8b\nzP233k/Nm2u6rd+/H778Ejp0gHxJ7v0pIiKSsahhFhGXRm4cSUR0RIpHlz/6yLmEXNeuHg4mIiLi\nJWqYRSRZkTGRDPlxCI1vb8zdxe52Wx8RARMmwFNPwc03eyGgiIiIF6hhFpFkDd8wnFMXTtHvnn4p\nqh8zBqKinEvJiYiIZBZqmEUkSacvnGbY+mE0vbMpYTeFua2Pjobhw6FxYyhXzgsBRUREvEQNs4gk\n6eOfPub0hdP0q9cvRfVTp8Lx49Czp2dziYiIeJsaZhG5yqnoU3z000c0L9WcyjdWdlufkABDh8Ld\nd0Md98s0i4iIZChqmEXkKh+u/5DImMgUz12eOxf27nWOLhvj2WwiIiLepoZZRP4jIiqCTzZ8wuNl\nHqdCYfcbdFrr3Ab79tuhWTMvBBQREfGyVG2NLSKZz7D1wzgXe463672dovoffoBNm2DcOPDz83A4\nERGRdKARZhG55ETUCYZvGE7Lsi0pVyhlS10MHgyFCkGbNh4OJyIikk7UMIvIJUPWDSEqLirFo8vb\ntsG338Krr0JwsIfDiYiIpBM1zCICwLHzxxi5aSSty7emdMHSKTpn6FDImRNefNHD4URERNKRGmYR\nAWDwusFciL/A/+r+L0X1Bw7AjBnQoQPky+fhcCIiIulIDbOIcPTcUUZvGs3TFZ7mzgJ3puicjz92\nvnbt6sFgIiIiPkANs4gwaO0gYhNi6Vu3b4rqDx6EMWPgySfhlls8HE5ERCSdqWEWyeL+Pvs3Y8LH\n0KZiG27Lf1uKznnzTefrO+94MJiIiIiPUMMsksUNXDuQBJvAW3XfSlH9pk3w+efw2msaXRYRkaxB\nDbNIFnYo8hDjNo/j2YrPUjJfSbf11job5UKFoFcvLwQUERHxAdrpTyQL+2DNBzisgz51+6So/uuv\nYe1aGD8eQkI8HE5ERMRHaIRZJIs6cOYAE7ZM4PnKz1M8b3G39TEx0LMnlC8P7dp5Pp+IiIiv0Aiz\nSBb1/pr3McbwZp03U1Q/ciT8+Sd89x34+Xk4nIiIiA/RCLNIFrTv9D4m/TyJ9pXbc0se90/unTjh\nXBHjoYegYUMvBBQREfEhaphFsqB3V7+Ln/Gjd53eKarv3x/OnYMhQzwcTERExAepYRbJYvae3MuU\nrVPoUKUDxXIXc1v/22/OTUo6doQyZbwQUERExMeoYRbJYt5d8y4BfgH0qp2ydeF69ICcOaFfP8/m\nEhER8VVqmEWykD0Re5j2yzQ6VenETSE3ua1fvhwWLoS33oKCBb0QUERExAepYRbJQt5Z/Q6BfoG8\nUfsNt7UJCfD661CiBLzyihfCiYiI+CgtKyeSRfx+4nem/zqd16q/RpFcRdzWT5kC27bBzJkQHOz5\nfCIiIr5KI8wiWcSA1QMI9g+mR60ebmvPnXNOw6hRA1q08EI4ERERH6YRZpEsYNfxXcz4dQY9a/Wk\nUM5CbusHD4ajR2HuXDDGCwFFRER8mEaYRbKA/j/0J2dgTrrX7O629uBBGDoUWreG6tW9EE5ERMTH\nqWEWyeS2H9vOrB2z6FKtCwVyFHBb36cPOBzwwQdeCCciIpIBqGEWyeT6/9CfXIG5eL3m625rw8Nh\n2jR47TUIDfVCOBERkQxADbNIJvbL0V+YvXM2Xat3JX/2/C5rrXU2yoUKQa+U7WkiIiKSJeihP5FM\nrP8P/ckTlIdu1bu5rZ07F9asgXHjIHduL4QTERHJIDTCLJJJ/XzkZ+b+Npdu1buRL3s+l7UxMdCz\nJ5QrB+3aeSmgiIhIBqERZpFMqt8P/cgbnJeu1bu6rR01CvbuhaVLwV//VhAREfkPjTCLZELhf4cz\n//f5vF7jdfIE53FZe+IEDBgADz4IjRp5KaCIiEgGooZZJBN6e9Xb5M+eny53d3FbO2CAc2e/oUO9\nEExERCQDUsMsksms2reKxXsW071Gd3IHuX5677ffYPRo6NABypTxUkAREZEMJtUNszEmvzFmmTFm\nT+Jrkk8XGWMGG2N2GGN2GWOGG6MNd0XSWnRcNC8seIGS+UryavVX3db37Ak5c0K/fp7PJiIiklGl\nxQhzL2CFtfZ2YEXi5/8wxtQEagEVgHJAVaBeGtxbRC7T/4f+/HHyDyY8PIEcATlc1q5YAQsWwJtv\nOtdeFhERkaSlRcPcDJia+H4q8EgSNRYIBgKBICAA+CcN7i0iibYc2cLQH4fyfOXnqV+ivsvahAR4\n/XXnbn6vuh+IFhERydLSYgGpwtbaI4nvjwKFryyw1q43xqwEjgAGGGmt3ZXUxYwxHYAOALfccksa\nxBPJ/OId8bSf356COQsypOEQt/VTp8Ivv8CXX0JwsBcCioiIZGApapiNMcuBIkl81efyD9Zaa4yx\nSZx/G1AaKJZ4aJkxpo61ds2Vtdba8cB4gLCwsKuuJSJXG/bjMH4++jNzWs5xu0nJuXPw1ltQvTq0\nbOmlgCIiIhlYihpma22D5L4zxvxjjLnRWnvEGHMjcCyJsubAT9bac4nnfAvUAK5qmEXk2uyJ2EO/\nH/rRvFRzHi39qNv6IUPgyBH4+mvQo7ciIiLupcUc5vlA28T3bYFvkqg5ANQzxvgbYwJwPvCX5JQM\nEUk5h3XwwoIXCPILYuRDI93WHzrkbJhbtXKOMIuIiIh7adEwDwQaGmP2AA0SP2OMCTPGTEysmQ3s\nBX4FfgF+sdYuSIN7i2Rpk7ZM4of9PzC00VBuCrnJbX2fPuBwwAcfeCGciIhIJpHqh/6stRHAfUkc\nDwfaJ75PADqm9l4i8q+/z/5Nj2U9uLf4vTxf+Xm39eHh8Nln0KsXFC/u+XwiIiKZhXb6E8mArLV0\nXtyZmIQYxj88Hnf7AFnrXEauYEHo3dtLIUVERDKJtFhWTkS8bM6uOcz7bR6DGwzmtvy3ua2fNw9W\nr4axYyG3692yRURE5ArGWt9duS0sLMyGh4endwwRn3Iy+iRlRpWhaO6ibGi/Af9srv+7NzYWypRx\nrre8dSv46z+TRURE/sMYs9laG5bc9/qrUySD6f5dd05EnWDJ00vcNssAo0bB3r2wZImaZRERkeuh\nOcwiGcjyP5czeetketTsQaUildzWR0TAgAHwwANw//1eCCgiIpIJqWEWySDOx56nw4IO3J7/dv5X\n738pOmfAAIiMhKFDPRxOREQkE9P/oBXJIP638n/8dfovfnj2B7IHZHdb//vvMHo0vPAClC3rhYAi\nIiKZlEaYRTKATYc38fGGj+lYpSN1Q+um6JyePSF7dujf38PhREREMjmNMIv4uLiEONovaE+RXEUY\n1GBQis75/nuYP9+5o1/hwh4OKCIiksmpYRbxcYPXDWbbP9uY98Q88gTncVufkODcpCQ0FLp29UJA\nERGRTE4Ns4gP++3EbwxYPYAWZVrQrFSzFJ0zeLBzveUZM5xrL4uIiEjqaA6ziI9yWAcvLHiBnAE5\nGfHgiBSds3Yt9O0LrVrBE094OKCIiEgWoRFmER81Nnwsaw+sZXKzyRTO5X4i8okTzka5eHEYNw6M\n8XxGERGRrEANs4gPOnjmIG8sf4OGJRvStmJbt/XWwrPPwvHjsH495M7t+YwiIiJZhRpmER9jreXF\nRS/isA7GNRmHScFQ8UcfwaJFMHw43HWXF0KKiIhkIWqYRXzMl9u/ZNGeRXzY6ENK5Cvhtn7jRnjj\nDWjeHF5+2QsBRUREshg99CfiQ05EnaDLki5UvakqXe7u4rb+9Gnnw31Fi8KkSZq3LCIi4gkaYRbx\nIa8tfY3TF04zselE/LL5uay1Fp5/Hg4dgjVrIF8+L4UUERHJYjTCLOIjlvyxhGnbptGrVi8qFK7g\ntn70aPj6a+duftWreyGgiIhIFmWstemdIVlhYWE2PDw8vWOIeNy52HOUHV2WHAE52NpxK0H+QS7r\nf/7Z2SQ3aAALFkA2/aeviIjIdTPGbLbWhiX3vaZkiPiAPiv6cPDMQdY8t8Zts3z2LLRsCQULwtSp\napZFREQ8TQ2zSDpbf3A9IzaO4KWqL1Hrlloua62Fjh3hzz9h1SooUMA7GUVERLIyNcwi6SgmPob2\nC9pTNHdRPrjvA7f1kybBjBnw7rtQp44XAoqIiIgaZpH09MHaD9h5fCcLWy8kJCjEZe327fDKK855\ny716eSmgiIiIaJUMkfSy49gO3l/zPq3LtabxHY1d1p4/75y3nCcPTJsGfq5XnBMREZE0pBFmkXSQ\n4Ejg+fnPkzsoN5888Inb+ldegd9+g+++gyJFvBBQRERELlHDLJIORm0axYbDG5jWfBoFcxZ0WTtt\nGkyeDG+95ZyOISIiIt6lKRkiXrb/9H7eXPEmD972IE+Vf8pl7W+/wYsvQt268PbbXgooIiIi/6GG\nWcSLrLV0XNgRgDGNx2CMSbY2OhqeeAKCg+GLL8Bf/z9IREQkXeivYBEvmvTzJJbuXcrwB4YTmjfU\nZe1rr8G2bbBoERQt6qWAIiIichWNMIt4yYLfF9BpYSfql6jPS1Vfclk7axaMHQs9esBDD3kpoIiI\niCRJDbOIF6z8ayUtvmpB5RsrM/eJufhlS35duL17oX17qF4d3nvPiyFFREQkSWqYRTxs4+GNNP2y\nKbfmv5Vvn/qW3EG5k62NiXHOW/bzgy+/hIAALwYVERGRJGkOs4gHbT+2nQenP0jBHAX57unvKJCj\ngMv6N96AzZth7lwIdT3FWURERLxEI8wiHrL35F4aTWtEkF8Qy9ssp2hu10/uzZsHn3wCXbrAI494\nKaSIiIi4pRFmEQ84HHmYhtMaEpMQww/P/kDJfCVd1u/fD889B3fdBYMHeymkiIiIpIgaZpE0diLq\nBI0+b8TxqOOsaLOCcoXKuayPi4NWrSAhAWbOhKAgLwUVERGRFFHDLJKGImMieXD6g+w9uZdvn/qW\nakWruT3nrbfgp5+cD/nddpsXQoqIiMg1UcMskkai46JpOqMpPx/5mblPzOXeEve6Pefbb51TMDp2\ndK6OISIiIr5HDbNIGohLiKPl7Jas3r+azx/9nIfvfNjtOYcPQ5s2UL48fPSRF0KKiIjIdVHDLJJK\nCY4E2s5ry8LdCxnTeAxPln/S7Tnx8fDkkxAV5dzVL3t2LwQVERGR66KGWSQVrLW8vPhlZmyfwcD7\nBtIprFOKzhswAFavhqlToVQpD4cUERGRVEnVOszGmBbGmB3GGIcxJsxF3QPGmN+NMX8YY3ql5p4i\nvuTNFW8ydvNYetXqxRu130jROStWwLvvQtu2zikZIiIi4ttSu3HJduBRYHVyBcYYP2AU8CBQBmht\njCmTyvuKpLuBawcycN1AOlXpxPv3vZ+ic/75B556Cu68E0aN8nBAERERSROpmpJhrd0FYIxxVVYN\n+MNa+2di7ZdAM2Bnau4tkp7Gho+l94rePFn+SUY1HuXunwEA9u6Fhx+GM2dg2TLImdMLQUVERCTV\nvLE1dlHg4GWfDyUeS5IxpoMxJtwYE378+HGPhxO5Vl/8+gUvLXqJJnc0YUqzKWQz7v8xWrkSqlVz\njjAvXuxcGUNEREQyBrd/0xtjlhtjtifx08wTgay14621YdbasIIFC3riFiLXbcHvC2gztw31itdj\n1uOzCPALcHvOuHHQqBEULgwbN8K97pdnFhERER/idkqGtbZBKu9xGLj5ss/FEo+JZCir9q2ixVct\nuOvGu5jfaj7ZA1yvBRcfD6+9BiNGwIMPwowZkCePl8KKiIhImvHGlIxNwO3GmBLGmECgFTDfC/cV\nSTObDm/i4RkPc2v+W/n2qW8JCQpxWX/qlLNJHjECXn8dFixQsywiIpJRpXZZuebGmENADWCRMWZp\n4vGbjDGLAay18cDLwFJgFzDLWrsjdbFFvGfHsR08MP0BCuYoyLJnlnFDjhtc1u/eDdWrww8/wKRJ\nMHQo+Pl5KayIiIikudSukjEXmJvE8b+Bhy77vBhYnJp7iaSHP0/9ScNpDQnyC2J5m+XcFHKTy/pl\ny6BlS/D3h++/h9q1vRRUREREPMYbUzJEMqS/z/5Ng88aEJMQw7JnllEyX8lka62FkSOd0zCKFYNN\nm9Qsi4iIZBZqmEWSEBEVQcNpDTkedZwlTy2hbKGyydbGxcGLL8Irr0DjxvDjj1C8uPeyioiIiGep\nYRa5QmRMJA9Mf4C9J/eyoPUCqhatmmxtRIRzybhx46BXL5g7F0JcPw8oIiIiGUyq5jCLZDbRcdE0\nndGUrUe3MveJudxT/J5ka3fudO7cd+gQfPYZPPOM93KKiIiI96hhFkkUlxBHy9ktWb1/NdMfnU6T\nO5okW/vtt9CqFWTP7lwNo3p1LwYVERERr9KUDBHg+PnjtJrTioW7FzK68Whal2+dZJ218OGH0KQJ\nlCzp3LlPzbKIiEjmphFmydLiEuIYEz6Gt1e9/f/27j446ure4/j7mycIQfLEIiGAlMQKTKlGUrz0\ninSES7x1BhVKuba1MtMptNURH2a8+NSrHW61jFQ70ykdHFItU67XsYrYyyVcmFbKWEEJVDRATYIg\nCQ/BPJCw2ZCHc//4LXnAzWoI2U12P6+Znf3tb89vf+fH4TAfzp49P5rON7Fm/hp+VPijkGVbWuAn\nP4HiYli40JuGkZYW4QqLiIhIxCkwS9zaXrmdFVtXUFZTxvy8+Txf9DxTfVNDlq2p8ULyrl3wxBPw\n5JOQoO9nRERE4oICs8SdyrpKHix5kDcOv8HkzMlsWrKJBdcswMxClj9wwPtx36lT8PLLsGRJhCss\nIiIiUaXALHGj6XwTT//1aZ7927MkJyTz85t/zgOzHmB40vBej9m8Gb77XW+puJ074Wu9rzAnIiIi\nMUqBWWKec46NBzby8PaHqW6s5ntf/R7PzH2G3FG5YY6B1avhkUdgxgzYtAlyey8uIiIiMUyBWWLa\n3uq93Lf1Pt7+5G1m5Mzg1cWvMmvCrLDHBAKwbBls2OBNvyguhhEjIlRhERERGXQUmCUmnT53mkd3\nPErxvmJ8aT7WL1jP0uuWkmDhf6l38iTccQe88w787Gfw+OPQy9RmERERiRMKzBJTzref59d7fs1T\nbz2Fv9XPg7Me5ImbniB9ePrnHltaCrff7t3u+tVXYdGiCFRYREREBj0FZokZW8u3cv/W+zn86WFu\nyb+F54qeY8roKWGPaW+HLVtg7VrYutWbp7xrFxQURKjSIiIiMugpMMuQV15bzgMlD/Cnf/yJ/Kx8\n3rzzTW69+tZel4kDb4m49eth3To4ehRycrz1le+9F3y+CFZeREREBj0FZhmyGlsaWbVzFc+98xzD\nkobxi3m/YMUNKxiWpbiOhgAACqtJREFUNCxkeee8peHWroXXXoPWVpg7F9asgQULIDk5whcgIiIi\nQ4ICsww5Ha6DDX/fwModKznZdJK7r72bp+c+Tc4VOSHLNzR4t7H+7W+hrAwyMryR5OXL4ZprIlx5\nERERGXIUmGVI2VO1h/v+9z52V+1mZu5MNi3ZxA3jbwhZtrTUG03euBH8fu+mI8XF3lJxWiZORERE\nvigFZhkSTjad5JEdj/Di/hcZO3IsL972Indde9dnlolrboZXXoHf/Ab27IHUVPjOd+DHP/ZuQCIi\nIiLSVwrMMmgdP3ucbRXbKKkoYctHW2hpa+Hhrz/MYzc9xqhho3qU/egjb8rF734HdXUwZQr86lfw\n/e97UzBERERELpUCswwaza3N7Dy6k5KKEkoqSiirKQMgZ2QOi6ctZuWNK/ly9pc7y7e1webN3rSL\n7dshKQkWLvRGk+fM0Q1HRERE5PJQYJaocc5RVlPWGZB3Ht1JoC1ASmIKsyfOZum1SynKL2L6mOk9\nloirqoIXXvAe1dUwYQKsWgU/+AGMHRvFCxIREZGYpMAsEVXbXMv2yu2UlJewrXIbx88eB2DK6Cks\nn7Gcorwi5kyaw4jknr/K6+iAHTu80eTNm73XRUXe61tvhcTEaFyNiIiIxAMFZhlQbR1t7KnaQ0m5\nN4r8bvW7dLgO0oelM2/yPH56008pyi9iYvrEHsf5/fD++95KF6Wl8Je/QEUFjB4NDz3kLQk3eXJ0\nrklERETiiwKzXHbHGo51BuQdR3ZQH6gnwRKYmTuTx2c/TlF+ETNzZ5KU4P31q6/3AnFpKezb5z0f\nOuSNIgNkZ0NhITz1FHzrWzAs9H1JRERERAaEArP0m7/Vz1sfv9U5F/nQmUMAjB81nkVTF1GUV8Tc\nyXPJSs3i1CkvFK/e0BWOKyu7Pis3F66/HhYvhoICb3v8eP2AT0RERKJHgVm+EOccJ5pOUFFbQXlt\nORV1Xc8HTh2gpb2F4UnDmXPVHJZdv4z5eUWk+aeyf79R+t/w+2A4rq7u+sy8PG9t5B/+0AvHBQUw\nZkz0rlFEREQkFAVm6dTW0caxhmNeEO4WjCvqKqioraC5rbmzbKIlMiljEnlZedzztXuZNmw+ydWz\n+WB/KlteglWlUFvrlU1IgKlTYe5cb8S4oACuuw7S06N0oSIiIiJ9oMAcZ5pbm6msq+wMwd1Hi482\nHKWto62zbGpSKpMzJzNxZD4FV8znirY8Upry6fg0D3/VRE7sSaaqCtaVQ1OTd0xKCkyfDosWdU2p\nmD5dt6IWERGRoUuBOYa0trdyxn+GGn8NNedqOH3uNEfqj1BRW9EZiqsaq3ocMyolg5xh+fgo5KrE\nJSQ15dN6Oo/GY/mcOZJDebXxYctnzzVmDIwb5805nj27KxxPmwbJyRG6YBEREZEIUGAexM63n/cC\ncDD8XgjCNf7Pvq45V0NdoC7k56Qn5DCqPY/Uc/PIr80ncCKPhiP5NB7N42xzFme7lR0xwgvBublw\nzde7QvGFx7hxkJPjjSSLiIiIxAMF5ghp72inPlBPXaCO+kA9tc21PcJuZwDu9rqhpSHkZxmJpHaM\nJqXVR2KLD9dUQEejj9Q6H4FPx+CafOD3wTkfNEykoTWNxgTvLngXQm/uzT1D8IXtUaO0IoWIiIhI\ndwrMfRBoC1DXXNcZenvdDtRR1xwMxn5vf1NrY6+fay6RlFYfCQEv5LadnUFrfTDw+n1wbky3bR8u\nkEnyqAQysyCr++NLkF3Y9Toz0xsNHjcOrrwSktTaIiIiIn2mCHWRNW+vYd/JfSFD7/mOEJN5u0ls\nTyPxfCa0ZOD8mbQ1XYXzXwuBTGjOhEBGt+1M8PtIT/KRnZZBdlYCWVneTTqysiHr6ovCcPCRnQ0Z\nGQq/IiIiIpGi2HWRtW/u5pO2vRDIpP1cBu1NuV64DWR0Bd1u24mtGaSnZJKVlk5WegoZGd7IbkYG\nZE7stp3Zczsjw3skJkb7ikVEREQkHAXmiyxsf4VDh3uG3lCB98J2Wprm/IqIiIjEMgXmi6xeHe0a\niIiIiMhgkhDtCoiIiIiIDGYKzCIiIiIiYSgwi4iIiIiEocAsIiIiIhJGvwKzmS02sw/NrMPMCnsp\nM8HM/mxmZcGyK/pzThERERGRSOrvCPMHwEJgZ5gybcBDzrlpwD8B95jZtH6eV0REREQkIvq1rJxz\n7iCAhVmI2Dl3AjgR3G40s4NALlDWn3OLiIiIiERCROcwm9kkoADYHabMMjN7z8zeq6mpiVTVRERE\nRERC+twRZjPbDowN8dZjzrk3vuiJzGwk8Efgfufc2d7KOefWAesACgsL3Rf9fBERERGRgfC5gdk5\nN6+/JzGzZLyw/Afn3Gv9/TwRERERkUgZ8CkZ5k1wXg8cdM79cqDPJyIiIiJyOfV3Wbk7zOw4MAv4\nHzMrCe4fZ2ZbgsX+GbgLuNnM9gcf3+xXrUVEREREIqS/q2S8DrweYn818M3g9i6g92U0REREREQG\nMXNu8P6uzsxqgKNROPVo4EwUzivRp7aPX2r7+KR2j19q+/gVqu2vcs75ejtgUAfmaDGz95xzIe9c\nKLFNbR+/1PbxSe0ev9T28etS2j6i6zCLiIiIiAw1CswiIiIiImEoMIe2LtoVkKhR28cvtX18UrvH\nL7V9/Opz22sOs4iIiIhIGBphFhEREREJQ4H5ImZ2i5kdNrNyM1sZ7fpIZJjZx2Z2IHhjnfeiXR8Z\nOGZWbGanzeyDbvuyzOz/zOyj4HNmNOsoA6OXtn/SzKp0Y63YZmYTzOzPZlZmZh+a2YrgfvX9GBam\n3fvc7zUloxszSwT+AfwLcBx4F7jTOVcW1YrJgDOzj4FC55zW5IxxZnYT0AT83jn3leC+1UCtc+6Z\n4H+UM51z/x7Nesrl10vbPwk0OeeejWbdZGCZWQ6Q45wrNbMrgL3A7cBS1PdjVph2/zZ97PcaYe5p\nJlDunKt0zp0HXgZui3KdROQycs7tBGov2n0b8FJw+yW8f1AlxvTS9hIHnHMnnHOlwe1G4CCQi/p+\nTAvT7n2mwNxTLvBJt9fHucQ/WBlyHLDNzPaa2bJoV0Yi7krn3Ing9kngymhWRiLuXjN7PzhlQ1/J\nxzgzmwQUALtR348bF7U79LHfKzCLeG50zl0P/CtwT/CrW4lDzpunprlq8WMtkAdcB5wA1kS3OjKQ\nzGwk8Efgfufc2e7vqe/HrhDt3ud+r8DcUxUwodvr8cF9EuOcc1XB59PA63jTcyR+nArOdbsw5+10\nlOsjEeKcO+Wca3fOdQAvoL4fs8wsGS80/cE591pwt/p+jAvV7pfS7xWYe3oXuNrMvmRmKcC/AZuj\nXCcZYGaWFvwxAGaWBswHPgh/lMSYzcDdwe27gTeiWBeJoAthKegO1PdjkpkZsB446Jz7Zbe31Pdj\nWG/tfin9XqtkXCS4tMjzQCJQ7Jz7zyhXSQaYmU3GG1UGSAI2qt1jl5n9F/ANYDRwCvgPYBPwCjAR\nOAp82zmnH4fFmF7a/ht4X8s64GNgebc5rRIjzOxG4K/AAaAjuPtRvPms6vsxKky730kf+70Cs4iI\niIhIGJqSISIiIiIShgKziIiIiEgYCswiIiIiImEoMIuIiIiIhKHALCIiIiIShgKziIiIiEgYCswi\nIiIiImEoMIuIiIiIhPH/9cs9aznaYZwAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 864x432 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nQJYFV-LwutI",
        "colab_type": "text"
      },
      "source": [
        "## Results analysis\n",
        "\n",
        "Now that we finally got our results for ```n_topics = 100``` we can analyse them, by looking, for example, at the most probable and the least probable words for each topic.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UGtG4phFvLPc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from operator import itemgetter \n",
        "\n",
        "n_top = 5 # 5 most probable\n",
        "n_low = 5 # 5 least probable\n",
        "names = vectorizer.get_feature_names() # get back the words associated with each index \n",
        "sorted_idx = np.argsort(V, axis=0)\n",
        "\n",
        "def show_topic(topic_idx):\n",
        "  words_top = sorted_idx[-6:-1,topic_idx]\n",
        "  words_bottom = sorted_idx[0:5,topic_idx]\n",
        "  print(\"TOPIC \", topic_idx)\n",
        "  print(\"5 most probable words:\")\n",
        "  print(itemgetter(*words_top)(names))\n",
        "  print(\"5 least probable words:\")\n",
        "  print(itemgetter(*words_bottom)(names))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DoOcFJyezxNy",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "d7a914d2-0013-4b48-8c0e-111e80fe8056"
      },
      "source": [
        "for i in range(n_topics):\n",
        "  print(\"--------\")\n",
        "  show_topic(i)"
      ],
      "execution_count": 77,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "--------\n",
            "TOPIC  0\n",
            "5 most probable words:\n",
            "('vega', 'tenor', 'intersect', 'chart', 'gunter')\n",
            "5 least probable words:\n",
            "('aaron', 'pet', 'pesticid', 'pest', 'pessim')\n",
            "--------\n",
            "TOPIC  1\n",
            "5 most probable words:\n",
            "('oat', 'schroeder', 'cholesterol', 'popey', 'quaker')\n",
            "5 least probable words:\n",
            "('aaron', 'peru', 'persuad', 'perspect', 'personnel')\n",
            "--------\n",
            "TOPIC  2\n",
            "5 most probable words:\n",
            "('edgar', 'literari', 'rambo', 'wagner', 'stallon')\n",
            "5 least probable words:\n",
            "('aaron', 'pig', 'pierr', 'piec', 'pie')\n",
            "--------\n",
            "TOPIC  3\n",
            "5 most probable words:\n",
            "('filipino', 'filibust', 'file', 'fijian', 'firstclass')\n",
            "5 least probable words:\n",
            "('aaron', 'platoon', 'platform', 'plate', 'plastic')\n",
            "--------\n",
            "TOPIC  4\n",
            "5 most probable words:\n",
            "('video', 'broaden', 'raton', 'boca', 'blockbust')\n",
            "5 least probable words:\n",
            "('aaron', 'plagu', 'placement', 'place', 'pizza')\n",
            "--------\n",
            "TOPIC  5\n",
            "5 most probable words:\n",
            "('absent', 'meager', 'appl', 'puett', 'kephart')\n",
            "5 least probable words:\n",
            "('aaron', 'packag', 'pack', 'pacif', 'pace')\n",
            "--------\n",
            "TOPIC  6\n",
            "5 most probable words:\n",
            "('filipino', 'filibust', 'file', 'fijian', 'firstclass')\n",
            "5 least probable words:\n",
            "('aaron', 'platoon', 'platform', 'plate', 'plastic')\n",
            "--------\n",
            "TOPIC  7\n",
            "5 most probable words:\n",
            "('filipino', 'filibust', 'file', 'fijian', 'firstclass')\n",
            "5 least probable words:\n",
            "('aaron', 'platoon', 'platform', 'plate', 'plastic')\n",
            "--------\n",
            "TOPIC  8\n",
            "5 most probable words:\n",
            "('filmmak', 'film', 'fill', 'filipino', 'firstquart')\n",
            "5 least probable words:\n",
            "('bcspehealth', 'chaney', 'merrel', 'hsn', 'kenner')\n",
            "--------\n",
            "TOPIC  9\n",
            "5 most probable words:\n",
            "('filipino', 'filibust', 'file', 'fijian', 'firstclass')\n",
            "5 least probable words:\n",
            "('aaron', 'platoon', 'platform', 'plate', 'plastic')\n",
            "--------\n",
            "TOPIC  10\n",
            "5 most probable words:\n",
            "('instant', 'ashar', 'unsolicit', 'kodak', 'shamrock')\n",
            "5 least probable words:\n",
            "('aaron', 'pilgrim', 'pile', 'pigeon', 'pig')\n",
            "--------\n",
            "TOPIC  11\n",
            "5 most probable words:\n",
            "('patriarca', 'twitchel', 'bendectin', 'tonka', 'kenner')\n",
            "5 least probable words:\n",
            "('aaron', 'fakhri', 'potato', 'postwar', 'portugu')\n",
            "--------\n",
            "TOPIC  12\n",
            "5 most probable words:\n",
            "('filipino', 'filibust', 'file', 'fijian', 'firstclass')\n",
            "5 least probable words:\n",
            "('aaron', 'platoon', 'platform', 'plate', 'plastic')\n",
            "--------\n",
            "TOPIC  13\n",
            "5 most probable words:\n",
            "('filipino', 'filibust', 'file', 'fijian', 'firstclass')\n",
            "5 least probable words:\n",
            "('aaron', 'platoon', 'platform', 'plate', 'plastic')\n",
            "--------\n",
            "TOPIC  14\n",
            "5 most probable words:\n",
            "('racist', 'longstand', 'vocal', 'racism', 'peoria')\n",
            "5 least probable words:\n",
            "('aaron', 'pit', 'pistol', 'pipelin', 'pipe')\n",
            "--------\n",
            "TOPIC  15\n",
            "5 most probable words:\n",
            "('timothi', 'oh', 'foremost', 'elliott', 'elliot')\n",
            "5 least probable words:\n",
            "('aaron', 'place', 'pizza', 'pittston', 'pittsburgh')\n",
            "--------\n",
            "TOPIC  16\n",
            "5 most probable words:\n",
            "('minu', 'invent', 'supercomput', 'fahrenheit', 'transistor')\n",
            "5 least probable words:\n",
            "('aaron', 'pittston', 'pittsburgh', 'pitch', 'pit')\n",
            "--------\n",
            "TOPIC  17\n",
            "5 most probable words:\n",
            "('bestknown', 'unsolicit', 'payless', 'sec', 'jeffer')\n",
            "5 least probable words:\n",
            "('aaron', 'parker', 'park', 'parish', 'pari')\n",
            "--------\n",
            "TOPIC  18\n",
            "5 most probable words:\n",
            "('highyield', 'induc', 'petersburg', 'quartet', 'barbershop')\n",
            "5 least probable words:\n",
            "('aaron', 'montana', 'monsignor', 'monro', 'monoxid')\n",
            "--------\n",
            "TOPIC  19\n",
            "5 most probable words:\n",
            "('basketb', 'stun', 'squarefoot', 'pamphlet', 'stein')\n",
            "5 least probable words:\n",
            "('aaron', 'physician', 'physic', 'phylli', 'phrase')\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Z7cjF-EnGPrv",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}