6.HMM.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Using a Tagger"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('And', 'CC'),\n",
       " ('now', 'RB'),\n",
       " ('for', 'IN'),\n",
       " ('something', 'NN'),\n",
       " ('completely', 'RB'),\n",
       " ('different', 'JJ')]"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import nltk\n",
    "text = nltk.word_tokenize(\"And now for something completely different\")\n",
    "nltk.pos_tag(text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we see that and is CC, a coordinating conjunction; now and completely are RB, or adverbs; for is IN, a preposition; something is NN, a noun; and different is JJ, an adjective.\n",
    "\n",
    "**Note-:** NLTK provides documentation for each tag, which can be queried using the tag, e.g. nltk.help.upenn_tagset('RB'), or a regular expression, e.g. nltk.help.upenn_tagset('NN.*'). Some corpora have README files with tagset documentation, see nltk.corpus.???.readme(), substituting in the name of the corpus.\n",
    "\n",
    "Let's look at another example, this time including some homonyms:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('They', 'PRP'),\n",
       " ('refuse', 'VBP'),\n",
       " ('to', 'TO'),\n",
       " ('permit', 'VB'),\n",
       " ('us', 'PRP'),\n",
       " ('to', 'TO'),\n",
       " ('obtain', 'VB'),\n",
       " ('the', 'DT'),\n",
       " ('refuse', 'NN'),\n",
       " ('permit', 'NN')]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text = nltk.word_tokenize(\"They refuse to permit us to obtain the refuse permit\")\n",
    "nltk.pos_tag(text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that refuse and permit both appear as a present tense verb (VBP) and a noun (NN). E.g. refUSE is a verb meaning \"deny,\" while REFuse is a noun meaning \"trash\" (i.e. they are not homophones). Thus, we need to know which word is being used in order to pronounce the text correctly. (For this reason, text-to-speech systems usually perform POS-tagging.)\n",
    "\n",
    "**Note-:** Your Turn: Many words, like ski and race, can be used as nouns or verbs with no difference in pronunciation. Can you think of others? Hint: think of a commonplace object and try to put the word to before it to see if it can also be a verb, or think of an action and try to put the before it to see if it can also be a noun. Now make up a sentence with both uses of this word, and run the POS-tagger on this sentence."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "man time day year car moment world family house country child boy\n",
      "state job way war girl place word work\n"
     ]
    }
   ],
   "source": [
    "text = nltk.Text(word.lower() for word in nltk.corpus.brown.words())\n",
    "text.similar('woman')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "made said put done seen had found left given heard brought got been\n",
      "was set told took in felt that\n"
     ]
    }
   ],
   "source": [
    "text.similar('bought')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "in on to of and for with from at by that into as up out down through\n",
      "is all about\n"
     ]
    }
   ],
   "source": [
    "text.similar('over')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a his this their its her an that our any all one these my in your no\n",
      "some other and\n"
     ]
    }
   ],
   "source": [
    "text.similar('the')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Observe that searching for woman finds nouns; searching for bought mostly finds verbs; searching for over generally finds prepositions; searching for the finds several determiners. A tagger can correctly identify the tags on these words in the context of a sentence, e.g. The woman bought over $150,000 worth of clothes.\n",
    "\n",
    "A tagger can also model our knowledge of unknown words, e.g. we can guess that scrobbling is probably a verb, with the root scrobble, and likely to occur in contexts like he was scrobbling."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Tagged Corpora"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1   Representing Tagged Tokens "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By convention in NLTK, a tagged token is represented using a tuple consisting of the token and the tag. We can create one of these special tuples from the standard string representation of a tagged token, using the function str2tuple():"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('fly', 'NN')"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tagged_token = nltk.tag.str2tuple('fly/NN')\n",
    "tagged_token"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'fly'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tagged_token[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'NN'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tagged_token[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can construct a list of tagged tokens directly from a string. The first step is to tokenize the string to access the individual word/tag strings, and then to convert each of these into a tuple (using str2tuple())."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('Doing', None),\n",
       " ('the', None),\n",
       " ('assignment', None),\n",
       " ('of', None),\n",
       " ('NLP', None),\n",
       " ('is', None),\n",
       " ('fun.', None),\n",
       " ('This', None),\n",
       " ('is', None),\n",
       " ('the', None),\n",
       " ('assignment', None),\n",
       " ('on', None),\n",
       " ('POS', None),\n",
       " ('tagging', None)]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sent = \"Doing the assignment of NLP is fun. This is the assignment on POS tagging\"\n",
    "[nltk.tag.str2tuple(t) for t in sent.split()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1   Reading Tagged Corpora\n",
    "\n",
    "Several of the corpora included with NLTK have been tagged for their part-of-speech. Here's an example of what you might see if you opened a file from the Brown Corpus with a text editor:\n",
    "\n",
    "The/at Fulton/np-tl County/nn-tl Grand/jj-tl Jury/nn-tl said/vbd Friday/nr an/at investigation/nn of/in Atlanta's/np$ recent/jj primary/nn election/nn produced/vbd / no/at evidence/nn ''/'' that/cs any/dti irregularities/nns took/vbd place/nn ./.\n",
    "\n",
    "Other corpora use a variety of formats for storing part-of-speech tags. NLTK's corpus readers provide a uniform interface so that you don't have to be concerned with the different file formats. In contrast with the file fragment shown above, the corpus reader for the Brown Corpus represents the data as shown below. Note that part-of-speech tags have been converted to uppercase, since this has become standard practice since the Brown Corpus was published."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(u'The', u'AT'), (u'Fulton', u'NP-TL'), ...]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nltk.corpus.brown.tagged_words()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(u'The', u'DET'), (u'Fulton', u'NOUN'), ...]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nltk.corpus.brown.tagged_words(tagset='universal')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Whenever a corpus contains tagged text, the NLTK corpus interface will have a tagged_words() method. Here are some more examples, again using the output format illustrated for the Brown Corpus:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(u'Confidence', u'NN'), (u'in', u'IN'), ...]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nltk.corpus.conll2000.tagged_words()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(u'Pierre', u'NNP'), (u'Vinken', u'NNP'), ...]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nltk.corpus.treebank.tagged_words()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Not all corpora employ the same set of tags; see the tagset help functionality and the readme() methods mentioned above for documentation. Initially we want to avoid the complications of these tagsets, so we use a built-in mapping to the \"Universal Tagset\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(u'Pierre', u'NOUN'), (u'Vinken', u'NOUN'), ...]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nltk.corpus.treebank.tagged_words(tagset='universal')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tagged corpora for several other languages are distributed with NLTK, including Chinese, Hindi, Portuguese, Spanish, Dutch and Catalan. These usually contain non-ASCII text, and Python always displays this in hexadecimal when printing a larger structure such as a list. \n",
    "\n",
    "TO check more tagged corpora, type `nltk.download()` and browse through the corpora section"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hidden Markov Models in python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we'll show how the Viterbi algorithm works for HMMs, assuming we have a trained model to start with. Further down we look at the forward and backward algorithms and Baum-Welch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Initialise the model parameters based on the example from the lecture slides<br>\n",
    "http://people.eng.unimelb.edu.au/tcohn/comp90042/l15.ppt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/jpeg": "/9j/4AAQSkZJRgABAQEAYABgAAD/4RD0RXhpZgAATU0AKgAAAAgABAE7AAIAAAAOAAAISodpAAQA\nAAABAAAIWJydAAEAAAAcAAAQ0OocAAcAAAgMAAAAPgAAAAAc6gAAAAgAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHByYW5hdiBzaHVrbGEA\nAAWQAwACAAAAFAAAEKaQBAACAAAAFAAAELqSkQACAAAAAzI4AACSkgACAAAAAzI4AADqHAAHAAAI\nDAAACJoAAAAAHOoAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAyMDE2OjA4OjIyIDEzOjQ4OjMyADIwMTY6MDg6MjIgMTM6NDg6MzIA\nAABwAHIAYQBuAGEAdgAgAHMAaAB1AGsAbABhAAAA/+ELIGh0dHA6Ly9ucy5hZG9iZS5jb20veGFw\nLzEuMC8APD94cGFja2V0IGJlZ2luPSfvu78nIGlkPSdXNU0wTXBDZWhpSHpyZVN6TlRjemtjOWQn\nPz4NCjx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iPjxyZGY6UkRGIHhtbG5zOnJk\nZj0iaHR0cDovL3d3dy53My5vcmcvMTk5OS8wMi8yMi1yZGYtc3ludGF4LW5zIyI+PHJkZjpEZXNj\ncmlwdGlvbiByZGY6YWJvdXQ9InV1aWQ6ZmFmNWJkZDUtYmEzZC0xMWRhLWFkMzEtZDMzZDc1MTgy\nZjFiIiB4bWxuczpkYz0iaHR0cDovL3B1cmwub3JnL2RjL2VsZW1lbnRzLzEuMS8iLz48cmRmOkRl\nc2NyaXB0aW9uIHJkZjphYm91dD0idXVpZDpmYWY1YmRkNS1iYTNkLTExZGEtYWQzMS1kMzNkNzUx\nODJmMWIiIHhtbG5zOnhtcD0iaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wLyI+PHhtcDpDcmVh\ndGVEYXRlPjIwMTYtMDgtMjJUMTM6NDg6MzIuMjg0PC94bXA6Q3JlYXRlRGF0ZT48L3JkZjpEZXNj\ncmlwdGlvbj48cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0idXVpZDpmYWY1YmRkNS1iYTNkLTEx\nZGEtYWQzMS1kMzNkNzUxODJmMWIiIHhtbG5zOmRjPSJodHRwOi8vcHVybC5vcmcvZGMvZWxlbWVu\ndHMvMS4xLyI+PGRjOmNyZWF0b3I+PHJkZjpTZXEgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9y\nZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj48cmRmOmxpPnByYW5hdiBzaHVrbGE8L3JkZjps\naT48L3JkZjpTZXE+DQoJCQk8L2RjOmNyZWF0b3I+PC9yZGY6RGVzY3JpcHRpb24+PC9yZGY6UkRG\nPjwveDp4bXBtZXRhPg0KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAog\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\nICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgCiAgICAgICAgICAgICAgICAg\nICAgICAgICAgICA8P3hwYWNrZXQgZW5kPSd3Jz8+/9sAQwAHBQUGBQQHBgUGCAcHCAoRCwoJCQoV\nDxAMERgVGhkYFRgXGx4nIRsdJR0XGCIuIiUoKSssKxogLzMvKjInKisq/9sAQwEHCAgKCQoUCwsU\nKhwYHCoqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq/8AA\nEQgBrwJCAwEiAAIRAQMRAf/EAB8AAAEFAQEBAQEBAAAAAAAAAAABAgMEBQYHCAkKC//EALUQAAIB\nAwMCBAMFBQQEAAABfQECAwAEEQUSITFBBhNRYQcicRQygZGhCCNCscEVUtHwJDNicoIJChYXGBka\nJSYnKCkqNDU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6g4SFhoeIiYqSk5SV\nlpeYmZqio6Slpqeoqaqys7S1tre4ubrCw8TFxsfIycrS09TV1tfY2drh4uPk5ebn6Onq8fLz9PX2\n9/j5+v/EAB8BAAMBAQEBAQEBAQEAAAAAAAABAgMEBQYHCAkKC//EALURAAIBAgQEAwQHBQQEAAEC\ndwABAgMRBAUhMQYSQVEHYXETIjKBCBRCkaGxwQkjM1LwFWJy0QoWJDThJfEXGBkaJicoKSo1Njc4\nOTpDREVGR0hJSlNUVVZXWFlaY2RlZmdoaWpzdHV2d3h5eoKDhIWGh4iJipKTlJWWl5iZmqKjpKWm\np6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uLj5OXm5+jp6vLz9PX29/j5+v/aAAwDAQAC\nEQMRAD8A+kaKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKK\nKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooo\noAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiig\nAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigArhfG3xb0LwF\nrUOl6xaajNPNbrcK1rGjKFLMuCWdTnKHt6V3VfM37RyNL8S9OjjGWfS4lUepM0tZVpOEbo9DL6EM\nRX5Km1j6Q07ULbVdNt7+wlE1tcxrLFIv8SkZFcn/AMLT0T/hYv8Awhn2XUP7R83yvN8tPJzs39d+\n7GP9nrXnf7P/AI0ktri58E60zRSxu72ay8FWB/eRfXOWA/3vasv/AJu4/wC3z/23qHVbjFrqzojg\nIxq1ac/sxbX6H0bRXG/EX4kad8PdMikuYmu765yLe0Rtu7HVmPO1RkfXt3x5b/wvjxzBAmqXnhKA\naO5G2b7POisD6Sklf0q5VYxdmclHA160OeK083a59CViReMNEuPFf/COWt6lxqSxNLLFF8wiC4+8\negPI461zkPxJbxH8Mb3xD4Lsmu9UtgqnTniaVkk3DKlUwWGCSCP6EV87+CvEXibRfHN3qfhzR/7Q\n1WRJRLafZZZdgZgW+RCGGDgcnipnWUWrdTow2XSqxm56OPTz8/I+xqK8t8dfGCTwZpOk2/8AZq3G\nv39pHcS2z7kS33DnI+9ndkBevByfXjJfjx450W4gfxN4Ut7a2m5QPbT27SL32s7EH8qqVaEXZmVP\nLsRUjzRXprufQcsscETSzOscaDczucBR6k1keHvFekeKlvH0K5+1w2c3kPMo+RmwD8p7jnr09K82\n+LXjHUtR+FdjeeG9Oa50vWrdjeSvA7taphSMlThTnIycjiuG+CHiXxTpl9DpWjaL9r0e81BPtt59\nklfyMgA/Op2rgYPIqXWtNRNaeXuWGlVb1XT03PZJPivocXxCHg5rXUDqJmWHzRGnk7iobrvzjB9K\n7evELnxns/aFGhf8I34fb/TUj/tBrHN3zGDu8zd17Zx0rtPiV8UtP+HttDEYDe6ncqWhtQ20KvTe\nxwcDIxjqcH6041FZuT2M6uEk5U4U46ySe/4+R3dFfPn/AAvjxxYRxajrHhGFNKmP7uT7PPCHH+zI\nxKn8q9i8N+MtO8W+EjrmitlQjb4pB80UijJRh+X1Bqo1Iy0RlWwVaglKS08tToaK+e9F/aI1+8t7\nm3m0C3vtUkZFsYLGKQA/e3lhuYt0XAGO/NX/AAx8etZbxdDovjXRYLHzplgZoo5IXt2YgDejknHI\nz045qVXgzaWV4mN9NvM91ooorY80KKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooA\nKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAo\noooAKKKKACiiigAooooAK+bP2gv+Ss6P/wBg+D/0fLX0nXiPxi+HvijxV8Q9N1PQdM+1WcFnFFJJ\n9oiTawlkYjDMD0YHp3rGum4aHp5XOEMRebsrPcx/jj4TufDPiiz8c+H90PmTK07IP9VcDlX+jY59\nwf71c/4R8QL4p/aK03Wli8k3kwdo/wC6wt8MB7ZBx7V9L67o1p4i0G80nUk3213EY3HcehHuDgj3\nArwLwD8I/F/hn4paffXunK2m2dxJm7W4iwybWAbZu3c5HGM81jUptTTW1zvwuLpzw0o1HaSi0vNM\nqfGfbdfHTTrfUzmz2WqEOcKIy53fhy1fRWp2tjNod1a6gkYsGt2SZSBtEe3B49AK4X4sfCtfH9vB\neafcR2urWiGNGlB2SpnOxiORgkkHB6n1rzybwf8AG3UdL/4R29um/ssr5TO93DtZMYwzL+8Ix2Oa\nv3oSlpe5j+7xVGklUUXDR30+aG/s0SzjxLrUS5+ztaIz+m8P8v6Fqr/BX/kuWq/9crr/ANGrXsXw\n1+Htr8PtAe2WUXN/csHu7kDAYjoqj+6Mn8yfYeTX/wALviL4Y+IF9q3gcRtHcSymK5jmiGyORs7W\nWQ9RnsD0yKjklCMdNjf6xRr1a6UkuZJJvTY7f4l/Eqx0LxJZ6BYeHYNe1tjG0YmUEQux+QDgktnB\n4xjI5rzz4vS/EO88L2d144g0yxsTeKIbO1IMgkKOQScscAAg/NjJHHpuePPhN4w1LVtN8UaJJHPr\nP2W2N6iTrG6XMaKDIjHCkZUdx09+KWufDP4o+MNBkvfFV8Lq9tQos9MEsQLEsAxJXEYwuTnJJx+Z\nPnldWZWG+rUlTkpR87738uy8zo7P/k01v+wfJ/6Oaj9mr/kTNW/7CH/tNa3/AAv4Q1a4+BreEtdt\nRp1+bea3G6RZFBLMyPlCeORkZzwa4T4feBvil4P8R29nEkdvoj3sct6RcQskqKQGI6uCVz0AzgZq\ntVKLt0MHKnUo1qfOk+a613XkZl5/ydkv/YRj/wDRQqPx4sV7+09b22rYez+2WUYWX7pQpGSPoWJ/\nM11Nz8PfFEn7Qo8TJpmdI+2pL9p+0RfdEYBO3du6j0re+LHwkbxxNDq+iXEdrq8EflsJchJ1HI5H\n3WGTg/n0qeSTi9OpssTRjVp3lo4Wv2Z23jO3srjwLrUOpBPshsZTJu6KAhIPsQQCPcCvFf2dZbg6\nd4si5NsIIm9g5Eg/UD9BUd34N+NXiKwXQdauyum8KzzXUO1wD/EUy7/jmvWvBfgK18D+C5dJsW+0\nXU6M9xORjzZCuOPRR0A/xNXrOala1jll7PDYaVLnUnJrbVKx49+zVbxP4s1ed0BlislVGI5UM4zj\n8hUPx9iSH4u6XJGNrSWMDsR3PnSDP5KPyrrfgd4A8TeD9b1S48R6b9jiuLdUjbz45NxDZI+Rjiov\njF8PfFHir4h6bqeg6Z9qs4LOKKST7REm1hLIxGGYHowPTvWfLL2NrHZ7en/aLnzK1t76bHt1FFFd\np8yFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRR\nQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFA\nBRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAF\nFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUU\nUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRR\nQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFA\nBRRRQAUUUUAFFFFABRWXqvifQdCmSHW9b07TpZF3Il5dxxMw6ZAYjIpNL8U+H9cuHt9E13TdRmRN\n7R2l5HKyrkDJCkkDJAz71fJO3NbQDVoooqACiiqen6xpmrGcaVqNpem3fy5hbTrJ5T/3W2k4Psad\nna4FyiiikAUUUUAFFFU9L1jTNbtTc6NqNpqFurlDLaTrKgYAEjKkjOCOPenZ2uBcopkM0VzBHPby\nJLFIodJEYMrqRkEEdQR3p9IAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAoooo\nAKKKzdV8SaHoLxLrms6fprTAmMXl0kJcDrjcRnqKai5OyA0qKQEMoKkEEZBHelpAFFFFABRRRQAU\nUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABXA\nfFb4q23wusNOnn0ubUpL+VkREk8tUVQCxLYPPzDAxzzyMV39VNS0nTtYt1g1fT7W/hRxIsd1Csqq\nw6MAwIyPWgBNI1KPWdEsdTt0eOK9t47hEkGGVXUMAR2ODVykVQqhVAAAwAB0paAPnf4+3GiWnxX8\nMT+KrOa90hLJzc28BId13NgAhl74PUVzvhu78E6v8ZfCTfDSzuvDiwTs94+oXOPPHGI0BkcksNyY\nB539ODXpPxU8NeNrj4l+HvE/gfRodSfS7ZlPnzxogclhghnUnhuxrHuvDPxU+IvirQD420XStDsN\nIuhdfaLaRGdsMpKjErtk7eOg7nOBX0dGrBYeKckvdafvev2epk07nZeOfFPj3RbrULrR7Dw9p2h2\nEIYX2uXRH2tyM7Ywjjac/KA+MnvzWFf/ABxurf4L6Z4ti0uBdRv7o2LJIzG3hkG7LnHzFSEyFznn\nGTjJy9c+HXiuf4m+IdQn8L6f4mg1ZPK0/UNQvVWLTVYEAmI5LbcjoueMg5JrQ8L+D/Geg/BaDRT4\nb0e/voL6SS40rV9kyXMRbcCjK+xWz03fXjAzzKGGVON7N3XW2lter697fJD1udP4B8XeKtd1q6td\nbi8Oalp8cYKar4cv1lijk/55ujOXyRnnAxjvnI5rwl8SrbS/C/jnXb3Q9LsY9H1Bo1j0y38g3bli\nq+YcnLFiAWxxknFVvh/8NdWsPinB4kj8Lx+C9MtLVo5LFdU+2teyOHGcgnaoyhwccquM5ODQvhPr\nd74K8eaHrUC6dJrOoG5sJXlV1ba29CdhJAJABzzg9KJRwylJNq3u7euuza27Bqbnh/xh8TLjVNAu\nNX8Mabc6NrimQHTWkEtkhAZWmZ22fdOcDrjAwcKfQPEmpTaP4U1bU7VUeayspriNZASpZELAHBBx\nkeorzrw8/wAWUfw3osmh2Gj2GlqsN/fy3Udwl7GihBtjU70OBkcjJxkgZB9I1/TW1nw1qelo4ja9\ntJbcOeil0K5/WuPEKCqR0jbyd+vX5FLY8zn+K2uR/s9W/jtbXT/7UlkKNCY38gD7Q0X3d+77oB+9\n1/KpPFfxP8SaPrXgvT9E0vT76XxDbLJLFKXTDsF+6wPyqN2TkNwK4u58FfE6X4NjwCnhW0WKznLi\n8/tKItdIZmfCJkbTls5Yj5RjGa7LWfBPiC78c/DnULfT99rotssd/J50Y8lgqjGC2W6H7ua7XTw0\nZO/K9Z9elvd6k3ZoeCvHviS7+JOq+CvGtnpcd/Z2q3UVxpZk8p1+TIO8k/8ALQc8dDx0ry3UfFGr\n6/8As1anf6ZpmhaJaf2n9nvoNMt3tt6fucFFBILEtht3BUYr1HT/AAlrcH7Rmq+KJbLbo1xpa28V\nz5qHdJiLjbncPutzjHFcdofws8VJ+zrrnha8sEtdYuL/AO1QW7zxsJFXyjjcrFQTsYDJ64zgc1VK\nWHjJT0XwP87/AHfgDuXbv4j+JvA3gHwfY3sOhLqGsKkdreStMtpbW6pEFab+Lf8APyQcDGa7n4ee\nJvEviBb4eIrXRpreF8W2q6HerNa3HTKbdzOrD1OM56DHPM3en+NtQ+H+j2V94A0a/WwC2tzpGo3U\nck0qLGqLNDMG2RkguOfmXHBOai+Evw71bw7441bX59EXwtplxai2h0ZdQ+2MW+QmRpMkYyGx3+Yj\ngdcqkaLoybspa9U769LO+3k0Cvc9F8XX3iGw0Iv4Q0iLVdTkkEaRzzLHHEDnMj5YEqMfdU5Oa4Xw\nh8S/EF18Up/BXipdBuZxA0iXOhyyMkTpnfG+8k7hggjggjvni98a/CWv+LvCtlbeHYxeC3vEmutO\nM/k/a4x/DuJA4POCR6jkCuY8IeA/EVh8YtK8SP4OsfDukfYXt2tLK7ikNsQjKDKRjezHByobhlyc\ng1nQhReHk5tXs/W/Tr+SG27l/wAD/ELx9441rUIrOz8PWum6VqHk3dzOJg7RbukahiN4VSckgcj0\nNY118etYnj1HW9GHhhdFsbny0069vimpXkYYAvGoOBkHIyDjB+9jnrfg94Q1nwxa+JYvENj9lF/q\nbzwDzkfzIyOvyMcfQ4NcVpfw28TeD1v9DsPh94b8TxS3nmWWuamYWEMTEDEkbfvGwBnCngk43dK6\nF9VdWasrK1tdPPW/9dBe9Y9z0TV7fX9BsdWst32e+gSeMMMEBgDg+/Ned+O/GXxB8PHVtSstO8Oa\nZoenriCTWboma/YBifKEbgAnbwjYY++cD0bSLL+ztGs7Mw2sLQwqjR2UXlQqwHOxOdq5zgc4FeD6\nl8MvF8virxg134W07xDLrhkSy16+vkC2UbA4xEctvUbVUgDaV4JWuPCRpOpJytZd/X1XT/gIcr2P\nZPAvif8A4TLwRpmv/ZxbNeRktEG3BWVipwfTKmuC8ZfFfWtP+JF14U0F/DmmfYrYTSXfiKeSOOdm\nVWCRlSADhu55wemOer+Euian4c+F+k6RrtmbO+tRKskRkR8ZldgcoSOQw71zXxF8Oa7rPiF/tfw9\n0XxhpTxKtvNFd/Yb232nO15mbJXJYgLgc8471SjRWJmmk4621XfTd2enmDvYvav8TtS8MfCWLxL4\nm0SG21aZkghtYrtJIbiRhlZFdGYBCAzYySAMZP3qzPCfxZ1Sfx1Y+G/FE/hq/OqQlrW78O3ZmSCQ\nBiY5QWbk4GCMD65O3Btvgnr1x8ELjQLyeGDVDqP9p2dk0peO3O3b5JfkEkFuRkZI56mul8G6Brk3\nim0utS+GfhXwrZ2kYaSeOKGa5llweYWiwIxnH3snHQnPG8oYVU52s3d9belru/3XF71xg+IvjPxL\n4t1608BaPpM+meHpPJuXv5JBLeSKW3LEU4BO0gZBHQk84GPF8avEMnwPuPGf2LTBqEep/ZFi8qTy\ntmF5I35zz1zj2q3beGvHXgLxd4qHhPSLPU9L8QSG7hvprxYRYSNuJ3p95wpY8KDkBec5A4/wN4Rv\n/G37MOpaVo5Q3y6q88McjbRKUCHZk8AkZwTxnGSByNY08PZSsuW8detnvf5iuz0/xl8QtV8Pax4D\ntbK3s3j8R3aQXZmRyUVmiBKYYYP7xuuegrB8TfGDVh4u1rSPCs/hiyi0Rds8viC7MTXcvOUhUMuS\nCCOeM4yRkZztV8O/EbxX4i8BXmqeFbfTLTw/fRG4RNRilkwrwl5jyBtIThVLNlWz1XMurfDzW/D/\nAMQPEOr6X4I0jxpY64fPiW/khVrKbJJ/1vVSWPC9QByMVFOnhopKVnKz6q1+bve2w22aPiD40Xcf\nwWsPG3h6xtkubi8FrNa3gaRYm+fcMqVJ+6CD6HpmpV+JHjLSPHnhvTPFukaRb6f4kGLeOzkkee2Y\n7cLIzYUkFgDhcc8Hjmr4+8AeI9Y+C1nommaHpEer/bkuZ7PRkS1t04fON7AEjKgnPJrV8eeENc1r\n4g+ANS02x8600ecvfSeci+SN0ZzgsC33T90HpURWFtay15uu2mn47dw94j1X4heLtZ8aa1oPw50z\nSZl0CLN7PqbSHzpD0jjVCMHIYc8EjqOMniD4sar4e+HOjalquhJpviHV5RbxWd9L5cMTd5XOcqgy\npKthhnBPGaqS+G/GfgT4j+Idd8H6Fb+IrHxCokaJr1Ld7WYc5bf95cs/C/mMcweLPh/418XfDrQb\nnXjpuoeKdIuftRtGULFcIcEwv/AXO0ZxhOo6fMSMcPzQvy8unXXbW/lfv8g1NTwJ8Ub7WPHE3hTx\nFLoN5dtb/abW/wDD9yZrZwOsZ3EkMOT26dOhPY+OPF1r4H8H3uvXsTTrbqBHChwZZGOFXPOASeTg\n4GTg9K5fwJoerDxNLqOo/D7w14Rs4YtsC20cUt47kYJEsWFCYzkYz2963viV4NPjvwHfaHFOtvcS\nbZLeRx8okU5UNjnB6EjkZzg9K5qiofWIp6R0vZ39dv0KV7HEy/Evxx4Wj0XWPHujaOmgavIsWdOe\nT7RZM43IZNxKn5d3C+h5HAPKeINM1/Wv2nfsd7ZeHNQf7JmGC/iklg+yBiRuUg/vcE9PlzW9qXhb\nx/8AETTdC8MeK9BtND0vTZUmvb8XqTm88sbAsca8oSGY/Nx7jGD0b+EtbP7RkfigWX/EmXS/s5uf\nNT/WYPG3O78cYrtjOlSu1yqXLLZ6eXXcnVnpVFFFeGaBRRRQAUUUUAFFFFABRRRQAUUUUAFFFFAB\nRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFF\nFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFACOgkRkcZVhgj2rJ8M+FdG8H6S\ndN8OWf2O0MplMfmvJ8xABOXJPYd616Krmklyp6AFFFFSAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQ\nAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFAB\nRRRQAUUUUAFFFFABRRWVLo97JrQvk8R6nFbhlb+z0jtvIIAAK5MJkwcZPz55OCOBQBq0UUUAcPd+\nK/EOr+Kb7R/BVlp7JphC3d7qTP5ZkP8AAoTnI5556Hpxm/4V8U3upatqGg+IbOKz1nTsM/2di0M8\nbdJEzyByODzyO+QMGFtU8BeMNbnbQ7/V9J1if7VFNpsXmyRSdWVkzwOTz7DrkgSaAmp/8JBrnjrW\nNIu7SJ7UQWmnqm65eNcEkpnhjtGAfU9gCeGM5cyu3e7uulv6sebGpPnV273d10S1/wCB6nodc0Pi\nJ4TOtf2V/bcH2vf5eNrbN3p5mNnt168da07a8TxD4YS7sTJAl/a74i4wyb14Jweoz2NeSLp2sN8O\nR4B/4RK9XU/tP/H8YR9lH7zf5vm9M7fl+nGf4a0rVZRtyf15fM2r15ws4Lddm79lp3Oyu/iRa6L4\n61fSvENxbWen2cURgkEbtK7uqkggZyOT0HFb914z8PWXh+31u51SFdOuceTMAzbyewUAtkYORjIw\nc4xXCA6h4Y+JusahceHNT1e1ksYLZbi1tTIWYRoDjPUEqQcZxxkVQg8K39n8OtNbUtD1N3XUWvQm\nlzhLuwRsBdqFTuPfaMFeM4wcYqtVXMvXo9NTmWIrLmW+/R6a6ep6bp3i7QdV0WfVrHU4XsbfPnSt\nlPLx/eDAEflz2qDSPHXhvXYrqTStUScWcZlmXy3RlQdW2sASB7A9vWvMx4f8Ua74K8SQCxuBHJND\nJam9tEtru6CElhIF5c4x8zEknpyTi1oOhXN7qF7qC6d4qDQaVLCJtcuAWZmRh5SR+XuYZOQQRz26\nAtYiq2tPwGsVWbiuXfyfmdmvxU8Gv9zWdwADMwtZiFBOAWOzA5IHNb82u6db65Do8tztv5oTOkWx\nj8gzli2MAcHqRXI+GPCv2/4LxaDqFq9nNcQSB0ljKMknmFlZgRnghT9BWX4I0jxBcWGua5rGnMur\nGwXTrK2u49ocRxAchuqu4Unt17VcatX3brfX/M1jWr+7zLdX2enfr6WOt074h+FdW1ddM0/WI5rt\n2KonluocjsGKhT7YPPauZtvi1b6k/iCO0ms7ZrJd1g9zFMySoDhnfauQOnygZ571yukafrl1rvhS\n7utL19fsl2q3Sz2awWsGTwYoUUbVx95yAM9a0b+x1W1m8f6b/YmpzNqr+dazw2peJwCWxuHUkMOB\nnoc4xWHt60lfb5eRzfWa8o3236Ptdde56Fc+NNH0bw/puoeIdStoGvYUdTEjkSEqCSiYL7eepHGR\nnk1I3jfw4nh5dc/tSNtOaQReeiM21z/CVA3A+xArh2sNU8P694W8RzaLfalaQaLFZywWsO+e1l8s\n8iMnPfBPGMnPYHG1Xw1rN34N16+XRbqAavqsU8GnJEWljjBbLMijK53c/StHXqq9l+fbc0liayvZ\nbeT7b/M9Q0jxx4b17VZNN0nVYrm7jDEoFYBgDztYgBvwJ456VFYfELwrqesjSrHWIZbtnKKoRwrs\nOyuRtb2wee2axtd0i4f4s+Hrm005ntIbCeJ5FjIiT5HCozgEKOcD68CuL03StastV0qy0PQtct3h\nv989pq0Ed1Y26hi26GcqNhxnlRls8MSBucq9WLs117Py/wAypYitB2avrbZ+W2vn3+R7jRRRXcek\nFFFFABRRRQAUUUUAFFFFABTJporaCSe4kSKKJS8kjsFVFAySSegA70+ue8eaRea94G1PTdNAa6mj\nHlqWC7iGDbcngZAxzxzUybUW0iJycYtpXY7QvHHhzxLePa6LqkdxOi7jGY3Qkeo3AZ/DNaGta5pv\nh7TWv9Zu0tbZWC72BJJPQADJJ9gOgJ7V5h4H0O4ufFWk3k1h4ribT7b95PrFwEiiYoVMcaFMsp9i\nuB1Hrc8fasL/AE3w9r8mm3MEVhqgMmnagiwzzjI5RCTu+gyec9M1yRxE/ZOUt/mcMcVP2LnJa/Py\n/I7XRfGnh/xFffY9G1FbqfyPtBRY3GE3BckkAA5I46+1bteV6Dqxu/jNrGqHSLy0/wCJH5n2aSIC\n4kw0fVAeGOMAE5wB06DtV8Ww+bo0c+l6pbNq7Mkfn24TyWAJ2yZPykgEgc5ANa0q3Mve7m1HEc0W\n5vrb8bGhf63p+mX1jZ31x5U+oSGO2XYxEjDGRkDA6jriqa+MdAZtWUagudH/AOP4eW/7rqPT5uVI\n+XP61mfEnSrrUPCyXWmQvNqGmXUV7bJEpLsytyBjnoSfwFeYyeC/ESWumCG0u/M8SxmLV28pv3BN\nwJN78fJ8pA59GrOtWqQnZRv/AF/n+BlXxFanNxjG6/pfn+B6vqPxD8LaVDaS3+rLELyITQr5MjOU\nIBDFQpKg543AZ5x0NWNQ8a+HdL0+zvr3VIktb3P2eZFZ1fHXlQcfj9K42S3vPBfj7WNQfw5f6xp+\npWqRWj2EHnGIIqgxMM5VenXrtGAecY9n4P1eysPB0F9psjgaw91PBHGZFtY2KEByBgDjPNQ69VNq\n34Pvb8SHia6bVtfR6apfO+56BcfEfwpa6Za39xqypb3m7yGMEm5wpwTt27sZBGSMVneH/iJbXlrr\n+oa1c2dvpum37W9vcQhiJY8naep3EjH3Rz6VT8Rx32hfFS08SnR77V7F9PNqBYw+bJA+4nO3sCD1\n4+8fx5m00G/uPCviX7b4W1BRNrP2qO0gfyJoU5O6IlSH2524AwQeO1KVaqp27X6PsKdesp27X6Pt\nv13PUtA8V6J4ojmfQr9LoQkCQbGRlz0+VgDj36cH0rYrzb4dQ6z/AMJTqU9zaagNMNuqRXWs2aQ3\nruCMKzj5pABnkkjgdK9Jrqozc4XkduHqSqU+aW4UUUVsbhRRRQAUUUUAFFFFABRRRQAUUUUAFFFF\nABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAVDeWkN/ZTWl0GaGdDHIqu\nUJUjBGVII/A1NRRuDV9GMhhjt4UhgjWKKNQiIigKqgYAAHQU+iigAooooAKKKKACiiigAooooAKK\nKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigArO1vQdL8R6ebHWrNLu33BgrEgqR3DDBB+h6\nEjoa0aKTSasxSipKz2MXQPCGheF/N/sLT0tWmx5j72diPTcxJx7dK0ZtOtbjULa+mi33FqHELlji\nPcMMQOmccZxnBI7mrNFJRilZLQlQjFcqWgUUUVRYUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUU\nAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAVlS+KvD1vrQ0efXd\nMi1QsqCxe8jE5ZgCo8sndkgggY5yK1aKACiiigDzKDRYvHvxC8QxeKmlnstHeOC108StGi7gT5h2\nkHJxnPfPoAKteD4JfDvxK1fwxYzzS6LHZrdwQyuXFqzMPkUnoDuY/gOpyTck8Da0fG8niKHxUYWc\nlPITT0GYckiNiG+fGcbiCeBUWieC9Z8MSaxqD+KJL2S9hkeUrpitKZQp2OPmJbbzhBwc4rgjCSkp\ncut3rpt955kac4zUuR3u7vTb7/6sd5XlOlfFLWr/AMSC3mt9HhgW9NvNp8szQXkS5K7t0hEbEH+E\nfMemB1HoXhz+038M2R15mOoNCDOSoQ7j6heAcY6VyN98MLzVJoYNT8VXd7pUFx50dvcW6yTr/si4\nJ3fp07VrW9rJRdP+v6+ZtX9tJRlSv+H4/wBP0MzSb7xGnxm8RNO1nPHa26+bEZHAWHYGjWIscKxJ\nUtnC5LGl0b4oavc+MNO0rUBok8d9IUaPT5Hke2yMrukyY2/4CT36V0Gp/D6W/wDFWp6rBrUlrbat\na/Zb21W3VjIvl7OHJ+Xsenb0OKoab8ML2yutEluvEzXSaLNutoBYrGgQ/eXhs7j/AHiT9Kw5K8Xa\nN9327/5HN7PExlaN7Xb3W1/Xt/w3U5rw/wCKdd8O3XjPWdZ+yXiWtz5dxGjybjPuKIsZbOIxk9eQ\nAMVseD/indaz4gXStRGnXj3EDSwPpiTJsdVLGNhKOSQOCvAx3zxqT/DIXN/rvm6xJ/ZmtOZZrNbZ\nQyS53K4kznhsnGOc4Oa0NB8Iarp90Zdb8VX2qKlt9mghjX7NGi4ILEKx3Pjox5HvxghTrxkl0+Xf\n1HTpYmEopbfLu/Pt5M5fwv8AE/Wdc122huItFWCa48mWzEzQ3VvliFJ80hZDxyEyfYE4r1SvPz8M\nru71TT5Na8UXOp2OnXBmt4Z7ZPPHOQrXGdxGcZ9ccY4x6BXRh1USaqHVhVWUWqv6fp/XoFFFFdJ1\nhRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAVleJtdi8NeGr3V54mmW1j3CNTjexIVRnsCSOec\nehrVqhrmjWviDRLrStQ3/Z7lNrlDhhzkEE55BAP4VMublfLuRPm5Xy79Dz3wd8VLrWvFFppOptpk\n4vUJjewSZDA4Bba/mDnOMZXjNdT438T3nh+HT7XR7WK51PVLkW9sLgkRqeMlsEE9QMZHXPbBj8N+\nENW0i+t59W8VXmqRWkAgt7ZYvIjAxjLgMfMOOhPPfmsHx5oWvr4SspfNutc1axvhPFfWcKxS26kj\npCufMHsCD0J4BrjvWhRfNe/9ebOC9eFB817/AC8vN+dv0NTw94o8Q3HxAuPDXiG106M2+n/aTLZl\nyJG3IMgseFwxGCM5HXFdtXlXgjS9f1DxpquuXh1OBpdM+yLeanZrBIZiVIZYQcBRt6cg9zkmutki\n8SWureHrVdVkvVBl/tKX7CiRyoBkMW52HJUBV5OSegNaUakuW7T3/U1w9WXJeSb18u9jnvjUsT6H\noi3EMlxEdVjDwxAlpF2tlRjuRwKyfD2iqvxA0q78G+Gta8P2cKv/AGi+oh40mQjhQGZt3PbPXBwM\nZrv/ABZ4W/4SiPTV+2fZfsF6l3nyt+/bn5eox168/SugqXQ56rm/L8CZYXnrucvK23Tz6HlHg3wv\np3xD0y68R+LxNqF1c3DxwxtO6LaxqeEUKR3J68dOM5JteG5NduPDni7wzp+oyzXWlTPbafdyviTa\nc4Uv6jBAPbPGABjWf4f6hp19eSeEPE8+iWt85lntDapcIJD1ZNxGzI9PQc4AAvWXw+0i08HXfh+R\np50viXu7p3/fSyE535xjIIGB7c5OSYhRmraWet33/rz2M4UJq3u2dnd3+L9fPXY888FwWGi+MNFt\npbXUvCuqhGguYp0aWDVSMLw5YbSTk8KV6YPTPXfGW41G28ATNp8yxQvIsd0dzK7IxxtXHGDnBB4I\nzUmnfDu/XUtJm1/xPNq1to5D2dt9kWLawxtLMCS2MDrz79c9B4u8Nx+LPDNzpEtw1t5xUrKE3bCr\nAjjjPT1FEKM1RlC1r7fd5aBToVFh507Wvtt28tDj9b8ZeIvDWj6ZFO/h+2m+yebcT3EzlG+baiRx\nKfNJxgk7SvXkYpsXxM1W/wDCmgS6bp9qda1m5e3RZWbyE2MAzHnOORxnjPfGDdb4b6o2p2upjxXI\nt+lqbS5uBYR5li3EgICcRnbhcjJ4z3IOD4k8E6j4Z8HaRb6Kt5qV1pt+08N5ZxDzbdXYE/uufM/M\ndBnjNRL28bvW1vLy/wCCZz+swvLW1vJ9vxtf/M2bbxz4jtda1zTtZ0yyln0rTftYjsS585/l6EnI\nUg9NuR71H4E+IOreJNWgt786LNBcRkn7FM0c0Dhd21o5Tuf0ygK57nBqj4M0HV9Y8Sa5qerNrVrF\nd2SWv2y6jFnctJ8pLIi/cAxxjI+vNbul/Du7i8T2GteIPEMmry6dGUt/9DSF+mBvcEs+AT15yc56\nguHtpNSV7X8u/wDkVTeIk4yV7X8tr9fl5M7miiivRPVCiiigAooooAKKKKACiiigAooooAKKKKAC\niiigAooooAKKKKACiiigAooooAKKKKACiiigAormPEnxH8KeENWt9M8RasLK7uIxJFGbeVwVLFQd\nyqQOQepq34p8Z6B4KsYbzxNfiygnk8qNvKeQs2CcYRSeg69K1VGo7Wi9dtN/QV0blFUtH1ex1/R7\nbVNJn+0WV0m+GXYy7l9cMAR+Iq7WbTTsxhRUV1dQ2VnNdXT7IYI2kkbBO1VGScDnoKo+HvEeleK9\nFj1bQLr7XZSsypL5bJkqcHhgD1HpT5ZW5raAadFFY3hrxboni+0uLrw7e/bIbac28reU8e2QAEjD\ngE8EcjihRk02logNmiiqup6laaPpV1qWpS+TaWkTTTSbS2xFGScAEnj0FJJt2QFqiqGia3p/iPRb\nfVtGuPtNjcqWil2Mm4AkHhgCOQeoq/Q007MAooopAFFFFABRVXU9TsdG02fUNVuorSzt13SzTNtV\nR06/UgAdyQKsRSLNCksZyjqGU46g07O1wHUUVDeXkGn2NxeXb+Xb28TSyvgnaqjJOByeB2pbgTUV\nm+H/ABDpfinRYdW0G6+1WMxYRy+WybirFTwwB6g9q0qcouLs9wCiiikAUVjReLdEn8XT+F4r3drN\nvALiW28pxtj+XndjafvLxnPNP8S+KNH8IaR/afiK8+x2fmLH5vlPJ8xzgYQE9vSr9nO6jZ3ewro1\nqKbFIs0KSxnKOoZTjqDTqgYUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUA\nFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAV5r8Z/iRrXw50jTLn\nQdHjv2vLgxyyzqzRxYAwuFIO5snHP8J4NelUUAUtGvZtS0Kwvrq2a0nuraOaS3f70LMoJQ+4Jx+F\nXaKKAPnH476MfEHxn0bTEGZJ9Dl8sY/jUXDL+qisLx34gb4h+FdG2ymUaJ4aOoXTA5zcGVIGB9/l\nLY969s1vwHqOqfG/w/4xjmtBp2mWbwTRO7CZnIlAKgLgjMi9WHQ/jyGhfAzUdG8O+OLBbrTzLrii\nHTtrPthjDswD/JkHlemcY719DRxVGMKd3rFK3zbT/AycXdlax8aavonwu+HvhvwqYYdY8QR+THdz\nruW1QMAX2ngnLDGQR8p4PFdFpHifxV4U+K1j4N8Y6vD4gttYt2ms9QW0S2kidQxKMicY+QjucsOe\noDLv4SanN4D8JwWWpW1n4n8L/Pa3IDPbu2QSrZAO0lV5wcYPBzWjoHgPxFffECHxl8Qb7TZb6xgN\nvYWelI/kRAgguTJ8xb5m49xzwAOac8O4yen2vW99LPtt+Nx6nIWniTx18RtO8X6xo+tWenaLp5mt\nbfS5LJZBdARtu3yHDodpU5Bxk9ABzz/h/wCIN34G/Z88Ox6ZNFa3mqX9xALyaIyLaxiQ75doB3EZ\nGBg9+D0rtIfhf418OzeI9L8G61pEGga7I8v+lxyG4tGZSCIwo285C5JPABABHMcHwV1UfCrRtGbU\nrO38RaHevfWV1FueHeXLBWyoOD8uTtOCOhrf2uGtytrlutPKz30777isxnwz+KV3qvxEfwvc+JYf\nFdncWxnttTXTzZOkijLRGPABGATnn69hL+zb/wAijr//AGGpf/RaV2nhXT/H39tT33jbWdL+zeXs\nh0zSID5W7/no0ki+Znr8oOOfwqj8JPAup+AtD1Sy1ie0mkvNQe6jNq7MAhVQAdyrz8prlrVKXs6k\nY2V+Xbra9+iX3KxSTuil8QfF2vr490DwL4RvIdMvtWRribUZYBMYIl3H5Ub5SSI36+3TOapajD45\ntPhh47tPHVzaahDDZ3AsL+NFiluE8s8tGg2qOmO4IbqMGtnx74C1TWfE2jeLPCV7aWmv6RlEW9Vj\nBPG2cqxX5hwz9BzuPTrVS28F+ONQ8M+LoPFXiCzub3XLd4bWyty/2O0JTaCGZS4B44AwME/MTwoS\npKnCzXS91re/f0+XzDW55jpmueO/BnwN0DxVpuvWkemwyiFdHNirCWMyv8zyklsk8YXbhccg5r07\nx94g1iD+zbi08a6L4O0uSzN1LLcotxeTMBnZHC4w6gFfuncScY6A52q/CrXL74Cad4IiutPGpWsi\ns8rSP5JAkZuDs3dGH8NL4h+Gfif/AIT7SfFnhi40O5urOwS1e21lJGjidBgPGUBPcn+Eg+ucDeVS\nhUnzNq95dFr26W76u4rNIw/DHxS8R6t8H/GOoS6nFc6jobEWmppaqhmUn5WMZG3oOm0cYyM5Nb/g\nu88f6h4B/wCEt1TxVp0jXGnNJBZXlolvbQMD/r5ZlG7G1WYgADJ7AVT0j4ReItN8I+OdHudT0+8n\n8QOJLa6+ePLEsWMiBSE6jhS3et6++G9/qXwJh8ES38NvfxWkcfnxFmiZ0YMAeAdpIxnGR1wcYM1Z\n4a7ULWcl0vpZX6d/QFc4Ow+JXiHR/iR4b0WXx9p/jCPUpkt7+G1sY0jtmcoAUmQAScscEH+E5UEi\numv/ABL4x8afE7WvDPg3W7fw9aaDCDJcPZrcPczNj5SHyFUHIyOeM85AGbF8KPHFze+DZ9Su/DUM\nPhe6jaKzsI5Y0ePcjSSFtvMjFORtC557mt/V/AHirSPiDqHir4d3+kxyavCI7601ZJPLVhtxIhTJ\nJ4JwcAZPXOBU5Yfm93lvZ9Fa9+ultvINTiPjFF40n+Buk3PjK5trW9hugl7aW8auLkkfu5GYcIy4\nbKplSW7YAGl4x8e674GsfDPhi78VxQXd9Ebi88QSaWGNvDn5FWBQQx4K5x6dM5G9rfwu8Sa/8GR4\nY1jxKupa2J1uRdXIPlqR/wAstwG4qMthiCfYDACav8PfGGq/8I74iW90S18X6EzRqI1lNncQk4Cs\nSN4O0tnA/iOMcEOFag4xjNqycundabp6X9el9EKzMbwX8VtT1fR/GmntrMOsXOjadNeafrUdn9nM\n6hGOWhIABVtoxjHrnqZ/Aes+Ptc+G83jTV/FamCKxvDDYx6bCDK6BwsrSY6hh90AAhFznJz11n4e\n8bah4V8RW3i/W9PubzVLSW3tLSyh2WtpuRlB3lfMbJIznOMd6Twj4I1LQPgr/wAIheT2r6h9kuoP\nNidjFuleQqclQcfOM8evWsalWgoy5Urtrz0trbRdey9CrM85uvir4utfgX4e103n+malfyW97q32\nNH+yxrKwBEYATJC45HOD3II6j4b+OL6/uNYkvfG2leK9Js7dp1kjtGs76Paqls2+xQY+SN2Tzjns\nJ9B8A+N/DHwu0nRdA1/TrTVrCeSSaN4fOtLxHlLbGZk3rweqjPUejCDwf8Jb608a3HiPxRD4fsw1\nk1nHpvh63eK3dWBDM+4A5wSMDrwcjGDpUlhXCa0WrtbffTpp5Wdu4veuecn47689jJ4hHi2yiuhc\n/J4U/spijQ5C/wDHztznGW+92/4DXo3jDx7r2q+J/CPhfwRdxaVceILVb+S+lhWZoYSpbAVvlJ2o\n598AZHWodB+H3xI8JWDeG/DHiXRrfQRdmWG/mtmkvYY2wWUIVMZ5z165JyOANvx58P8AWdW8SaD4\np8I6jawa9owMWdQU+VcxnOQ5QZH3nGABnecYwKdSeFdRcqVtbfdpdWXXvd9w96xxvgq11mz/AGod\nWg8R3sOoXqaKAbuGHyhOv7naxQEhWx1A4z04rc/aT/5JKf8Ar/h/k1WfCPw/8W2HxbvPGfirUtLu\nzeWHkNHZCRfKf938qqw+4NhGS2T1IyTWz8XPBWo+PfA50bR5rWG4+0xzbrp2VMLnPKqxzz6Vm60P\nrdKbaslG9tgs+VnF3fibx14C1zwnc+I9Zs9T0jXHS2nsIbNYhYswT7jjLPtz1Y84PHII1dB+IOp6\nF4r8daN41vvtf9ixHUbBjHHEZLXaW2/KACcNGO5ySO1Rf8K08Z6/4i8Pf8JvrWlXOjeHnEsAso5F\nmu3XbtMoYbRnbzg+oxzkcp8cdPsPE3xJ8Pab4d1OKTV9QJ0zUYLWQM0cQkVh5mD8uMsSDzhfatYR\no1pKm7ap3aW1ndfhow1Wp6h8I9S8Ra38PrbWfFl59pu9QkeeJRCkYihzhFAUDOcbsnJww5rt6hsr\nODT7C3srRBHBbRLFEg/hVRgD8hU1ePVmpzckrJlrYKKKKzGFFFFABRRRQAUUUUAFFFFABRRRQAUU\nUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRR\nQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFA\nBWfBoGjWury6rbaTYw6jMCJbyO2RZnz1y4GT0HftWhRTTa2AKKKKQBRRRQAUUUUAFFFFABRRRQAU\nUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRR\nRQAUUUUAFFFFABRRRQAUUVlS+FfD1xrQ1ifQtMl1QMri+ezjM4ZQAp8wjdkAAA54wKANWiiigDzd\nfGf9hL411Lyb69/sy8iTyLm/3Idz7f3Y2fuxznHzZwK0LD4h3kviHStP1bw3cabb6wheyuXuVkL/\nAChvmQD5eo4JyMjiqGpfDzVrzTPGNvFcWYfXbqGa2LO+EVJNx3/Lwcema2dW8KX1/rvhK9hlt1j0\nXf8AaAzNl9yoBs45+6euK4V7df0v5n+h5sfrC2/T+Z3/AAOB1Txdq8Hw0ludGk1FRJq0lvNd3N+J\npYgAMKrbVIDc4x93HXmuw0nXrqb4kXEGpWmoWdwmjfaJbIXwnhjO9eFjVeX/ANoHuRjms5/hpqsn\nw5vNCN1Zi9k1I3sT7nMeOOGO3IOM9j/ho2/g/X7vxPf6vq15Y28t9or6ez2JkJilLDDqGA4AGfvZ\nzWcI1k03fp+plCNdSTd+na3W5HY/E64l8TWOlar4dk00X8hSEyXiNMv90yQ4DJnjr+GasfDHULy/\nh8QG+u57kw6vNHH50hfYgAwoyeB7Vzui/C7XrC+0Gef+wYhpNyHc2qSCW4QnLM8hXLMOw4HPUV2X\ngjwzeeGotXW+lgkN9qMl1H5LE4RsYByBzx71dL2zmnP+tDSh9YlOLqX6/l5eZmeIfiRNpXim40PS\ntDOp3FtGryA3iQuxZd22NCCZDgjhefasvxf4u8QxeLfDEWj6bdLbXY89bSSZbd7p8cxvkZTbnkHg\nk+wq74z8Ea5r2pXclp/YV7bXUIjX+07YrNaY/wCecsa7jzz8x9sHnLL/AOH+tW9n4VbQ76ynvtAV\n1LX+8JKGxj7uTgYwB6Y54pT9u3Ja2+Xf/IVT6zJyWtrq23f/AC/roTeJviZdeG76cS+HJTYQS+V9\nquLtbdpiCAxiiYbpAMjleO/A5qjceLNek+MNlZQWsh0s2iyCE3SIpicjdcN67R/Acn5TjGaztU+F\nOv30mtJ5+iSrqNwbhL24jka6U5BEYbBCL1BIySPbgdFeeDdb/wCEw0nW9Pk00rFpyWF9FcNJyuTv\nKFRycEgZx096T9vJ63tddiX9ZlLW6V127v8A4BRt/jLYzalFu09U0qa48hbv7dEZhngO1v8AfC57\n+nPXiuz8U67/AMI14YvNX+z/AGn7Kobyt+zdlgvXBx19K4LSvhbqunXEFgT4fk0qG5803kmnrLey\nR9fLO9CvPTPUdRx8tdP8Uv8AkmOs/wDXNP8A0YtaQlWVKUp7pGtOeIVGcqm6Wm3Yo2PxHum1XSoN\na8N3Gl2esELZXTXCyb2ONoZQAVznuc9OOuG6T8SLzVbi9kHh0w6Zp00kd7qDXq7YlQE5C7QWJx90\ndMjnmqej+FPEWtP4ZuNf1DT20rSUjubYW0biaZtqlBJn5RjHUH8OcjX8N+CZ9P8AC+vaPq8sLrqt\n5cShrdmO2ORFUZyB8wwfalF15Na6fLt/mKDxMmtXb5dl+Fyjb/FCfZZ6hqXhq6stBvpvKt9Ra4Vz\nySFZ4gMqDgnqfbPFZVl4s8Qaf4v8YJZaVea/FbXCuI2u9kdrGAxO3dnk8YVRk4PpVmP4f+JrzTNP\n8Oa1qmmv4fsJg6yW8bi5nRSSqsD8q8HHBOMfxVLJ4Q8ZWGveIr7QL3SFi1l/uXXmEouGAYELww3d\nOQc89Kh+3dm7/hfZ/gZv6y7N309L7O/la5Y1L4r2dvoui3mm2H2qXVg5SO4uktkiCcPukb5Qd3AH\nf1HAPU+G9auNd0n7VeaXc6XcLI0clvcA9R3VsDcpzwwGOvpXJSfD7U9P8MaNpujz6TffYN5ntdXs\nxJBOz5JcHBdSCTgA88Z6HOt8PPCFz4Q0e6gvbiFpLq4af7PbFzDB2CoX+Y8AcnnoOcZOtN1vaLn2\nt+n+ZvSliPapT2t5W2X6/wDDHW0UUV2HeFFFFABUV1JLFaTSW0PnzIjNHFvC+YwHC5PAyeM1LRQB\n5qnxP1+R9XVPBWToy7r8f2rH+5GGP9z5uFb7ueldT4O8RX/ifSf7RvtG/syCUK1sftSzecpzzwAV\nx6GseDwVqMU/jh2mtSPEMRS1w7fIdki/P8vHLjpnvXReFNJn0Lwnp2mXbxvNawiN2iJKkj0yAf0r\nkpKtze+3bXt306djhoKvz/vG7a9u+nTtqZd/4g8X2+oTw2Pgj7ZbJIViuP7Whj81c8NtIyM+hrc0\n+71G50Rbm+0wWV+UYmyNwrhWBO0eYBjnAOccZrDv/hj4Q1PUJ76+0jzbm4kMkr/aZl3MTknAfA/C\nt/SdJstD0uHTtLh8i0gBEce9m25JJ5Yk9Se9aQjUUnzPT+vJfma041lN8709V/8AIr82cjH8TEn8\nL6dqNvpZe/vr/wCwDTzPgpJnBy23oBg9O9V/EHxYt9H1q8srPTo7yLTyFupZL+KBg3cRxt80mPbv\nx6GprL4eSWvxNl18zwnS1aS5gtQTuS4kVVdsYxjgnOc5xWfrHw31VvEmqXmi/wBhTW+qOsrvqtmJ\nprV8/MY8qVOevPHbHGTzyeJ5f+G+/wCbOWTxfJpve3Tot/mzZv8A4hbrzTrHwvpEmt3t9aLeiITr\nAscJ6FmbIDZ7f/WrnvG/jXxD/wAI9ok+n6Ve6U93erHcxzOIpN6nIhBIB2tjO8YGBg9SK2b7wVrG\nna5p+s+EbnTlvLexFjPFewmOGVRzvCxD5TnsAB/Is17wb4l13wjYQXurWd1rVrfre+ZLGY4QACPL\nGxckDOckZPt2J+2lGS1/DyCp9YlGSd7+VrdPn3LF5491EaoNJ0bwzLqWqQWq3GoQLeJGloWAPl7y\nCHYZ7Yz2zzitd/FWBdD0nUNM0ie8k1G6a0a1aTy5IpRj5ehBOSMdOo+lJJ4V8W2HiCbX9BvNHW/1\nK2SPUYLlZfJWRQBvjIyx6Hg9PfPEMHw0vLK18Ox297DPLYap/aN9LLlfMYld2wAHsuOcfhQ3iNbf\np30/DcHLFa2v9y7q1vludrpuoXkuifbdesF0mZQzSwfaBN5arnksoA6DPFcavxUlFqusTeGruPw0\n8xiGp+epbqQG8nGcbhjOfzPFd9dW0d5ZzW0+THNG0b4ODgjB/nXmq/DzxRJoKeErnV9O/wCEcWbc\nZ0jf7W0YbeEwfk+9jvx156VrW9qrKH6b+fkbV/bxsqd3p5b9L+XoVpNR8Qa58azYE6hBYWapIIbP\nUljjMQYESuAMOrZGV+9hgM8V1nje8urS/wDDC2tzNAs+sxRyiOQqJEIbKtjqPY0uneFLqx+JV7r4\ne3FhNp6WsUSs3mKV2dRjGPlPeoPH/wDyEfCX/Ych/wDQWqOWUacnLe/6mahOFObk9W/1Mm2TVvHn\njDXIZ9b1DSNL0ec2sMOmzeTJI+eXZsHI+Xp7jGMHMFnqutaPJ4v8M3+pT3/9n6ZJdWN87YmRShID\nMOSwyOevB9gL9t4c8a2Xjm/1qym0OG1vcrLbq02JQuRG7DHD4xkggdao23hzxPo+k+Lr/wASzaVd\nNqGnzvJcW7SGbcI8InICiMDdwBnOKztLezvd3fl/VjK01rZ3u7vy1/4B2/hW4ebwTo1zdzNJI+nQ\nSSyytksTGpLEnqe+TXGW/wAZbGbUot2nqmlTXHkLd/bojMM8B2t/vhc9/TnrxXXeEY1l+H+hxyDK\nPplurD1BiWuH0r4W6rp1xBYE+H5NKhufNN5Jp6y3skfXyzvQrz0z1HUcfLWtR1uWHszapKvyw9l8\n/wAC1pvi46HrHju+1m6ubiz0+5hWCAyFtu7cAqAnAycdPr2q74f+KEer6hNY3unQ2lyLVrmAW+oR\n3SShQSVLJ9xsDOD2z04zDd/De61JfF8V5dQRprc8U1q8ZZmjKEn5wQOuQOCe9O8PeBtXtbqa41aL\nw9bFLNreCPSrFULuykGR5CgYHBxheDnoMYOcfbqSS21/N/oZx+sxkkttfzf6WsU4Pi/cNp2nande\nFbmPTb2Y24njukkbzMn5UTALdBydvOR251rL4kCOXWoPEujy6PcaRAtxLEs6z70bGMMuBkllGOnP\nUc1mH4c6ufAXhzRPtFl9p0q/+0zvvfYy73bCnbknDDqB3rQ1T4f3Gs+JfEd1dXMUVlq9jHbRFCWk\njddpDFcAEZXpnn2pxeJSv6du3+YReLSTvd6dF2f5MZ4Z+KMWua9baXe6dFZPeoXtXh1CK53YG7a4\nTmM47HnPFZ0PxdvpvDg14eEZ/wCy45vKubgXq/uzkD5V25b7w5wBnjNXPCXgXV9L1izudYi8OxQW\nMWxP7OsF824fGA7yMgZSOuVIyait/h5q0XweuvCjXFmb+aUOsgd/KA81X5O3PRT260k8Q479+i8r\nfqJPFuG7vZ9F5W/UibxXr8/xjt7CC0lk0z7IHSAXCqDE5XNwRjLEAcIcnrjGTVrQfilL4g1SJLTw\n7O+nSz+V9pguVmliBJCvJCo3IpxyScDsTVifwfrkHjnTdd0q5sFjjsY7K7WbfvCg/M0eBgtjpu4y\nOQaxJvhlr97rFk95PoUaWl59o/tSztWgvJRu3fMqARk5A56jrk85X7+L0vv5eQf7TFu19/Ly/r9U\neq0UUV6J6oUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRVKDWNP\nudWuNNt7qOW8tUV54kOfLDEgZPQHjp16HuKu0k09hJp7BRRRTGFFFZx1/TF15tFa6Av0g+0tEUYA\nR5xu3Y2/rmk2luJyS3NGisSTxloEemwagdRRrS4u/scUyI7q8uSNowDkfKfm6e9bdJSi9mKMoy2d\nwoooqigooooAKKKqanqdpo2mTahqU3k2sC7pJNpbaM46AE96TaSuxNpK7LdFRWtzFeWkN1bNvhmR\nZI2wRuUjIODz0NS0x7hRRRQAUUUUAFFFFABRRRQAUVi6j4w0LStdttG1DUFhv7nb5UJRjnccLlgM\nDJHcitqpUk3ZMlSjJtJ7BRRRVFBRRRQAUUVT03V7DWIZZdLukuooZTC7x8rvGMgHoeo5HFK6vYV1\nexcorPv9c07TNQsbK+uPKuNQcpbJsZvMYYyMgYHUdcVoUXTdkCabsgooopjCobiztrtoWuraGdoJ\nBJEZIwxjcdGXPQ+4rJk8Z6BH4lXw+2oKdUZgot1iduSM4LAbRx71oXOrWVpqllp1xNsu74SG3j2M\nd+wAtyBgYBHUio5oPqRzwkt12+Zcpk0MdxC8M8ayxSKUdHUFWUjBBB6iuf1rx/4X8PX5stW1aOG5\nAy0SRvIU/wB7YDg+x5q6/ijRU8Otro1CJ9MRQzXEQLgZIGCFBOckAjGR3o9pBtq60F7Wm21zLTfU\n04YYraCOC3jSKKJQkcaKFVFAwAAOgA7U+kVg6BlOVYZBpas0CiqWq6vY6JY/bNUn8iDeqb9jN8zH\nAGACepqrr3irRPDEMcmu6hHaCQ4RSCzt7hVBOPfGKlyjHdkSnGN+Z2sa9FZOg+KdF8TQvJoeoRXY\njOHUAq6+5VgCB74xVrSdWstc0yLUNLm8+1mLBJNjLnaxU8MAeoIoUoy2YRnGVuV3uXKKKKosKKKK\nACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigArKl1i9j1oWKeHNTltyyr/\nAGgklt5ABAJbBmEmBnB+TPBwDwa1aKACiiigDxS202Wx8YfEG5g1TUkm0+xlljb7QQZGaFyC+Bk7\nc5XpjAq3C91pXwug17UPFWuteatHHANjGcpubOIkLLhyq4Lls/eI5Neg3vgfw5qGszard6ar3s8T\nwySiV13K0ZjYEBgMlWIzjNT3HhPRLvw3HoNzYLJpsQURws7fJjphs7gffOeSK4Fhpq9vO2/U8yOE\nnHmtbrbV9WeYaNqev6feeM9LvbzVIvsmiy3EUd7qC3M0LhMqwkXG04bOB0464zTC+uaV4V8J+Jv+\nEm1a4ub28hgmt5p90DI27A29zhcEnJJOeK9Hsvh/4Y05ZlsdKWAXFo9nLsmk/eRN95T83JP97r05\n4q1N4S0S40ex0uay3WWnyrLbRea48tlztOc5OMnqTSWGqW1f4+YlhKvLZv01fe/ZHmHii88Qw+Id\na1J9Y1Q6dZ3AWK40a+iljtMH7ktruBJ4wdxXnOdwrXm1C617x7NZxarfLYXnhoXCC3laMbmwQ6rk\nhWIPuecV1mqfDzwrrOqNqOo6PFLdMQWcSOgcjuVUgE+5HNaUfh3SYtaGrRWapei2FqHVmCiIfwbM\n7ccelUsPU5nd6X7+v9fqUsLV5nd6N935/wBfqeK6IbzSPhdol9YapqELX2uxRPElwVjRB5uVUDGA\n3BYc5IHpXXyfb/F/jjxJZXfiS/0O30ZFW3isp/JyCCTK/wDeUYB/4EOR36aD4b+E7ZHSDSRGr3Ed\nyVWeXHmJu2H73GN7cdOenAqxrvgXw34kvFutZ0uO4uFXb5qyPGzD3KEZ/HOKmOHqRil6aXfn/wAO\nRDCVYwUdNLaXdnv5fM80i8SeIPEeneCA+rXVlPfXlxbz3Fs2zzkUoNxXoTgkcjg84rqvBb6hpnxE\n8QeHrjV77U7O1himha+l82RSyqT8x/3vpxXWS+GdHmm0yRrGNTpRJshGSiwk4zhVIB6DqDUsGh6d\nba5daxBb7b+7RUnm3sd6qAAME4HQdBWkKE4yUm77flb8zWnhqkZKTldprvtaz/E4bxY15f8Axc0v\nRE16/wBKs7rTizraXBjLsGc/L2DHA5xnAxXK6nr+u2PhDxFp0WuXlx/ZWrRQW2oecRKysX3Kzjls\nEDr646AAd/4h8BJ4l8eWup6pFbXGlR2BtpYHdhIX3MQRgcdRzuB61q/8IN4cHh4aGumIunCXzvJW\nRwWf+8WB3E/U9AB0ArOVGpOUmtN+r7GcsPVnKbWm/V9vy6nG6/a6louq6D4Yj8S6sIdauWa71Ka4\nzN8oXEUTY/dgnsOcsAcjg5Gt3t/Y6d418MS6rcavZWdpFNFcXT+ZLGzOmUZ+/X9O1er634f0rxHY\n/Y9bso7uHIYBshlPqrDBXp2IyOOlUrbwR4ds9BudGtdMSOwuv9fGrvuk5yMvnd+tVPDzbfK9Pn2t\nb9S6mFm5Plenq+1rW7X1uee2j3/hybwPfWfiO+v11h4reezmlDQ+WwQfu0H3dmcE8nOORzmhq+pe\nIrLVNT1281nUzp8OoFIb3SbyK6tYgCNsUlsGGOMAlmHJGQScn0zS/h/4X0XVU1LTdIihu0GEk3u2\n3jGQCSAffGevrTLz4deE9Q1htUu9Fhku3kErtvcK7ZzlkB2nJ65HPOc5qXh6nLZP8X2/r/MzeErO\nNk7fN9vT+u50cMyXEEc0Lbo5FDo3qCMg0+gAKoCjAHAA7UV6B6oUUUUAFFFFABQSFUljgDkk9qKj\nubeO7tZbacFopkMbgMVJUjB5HI47igD5+1W8bxKfE+sJpGs3c1zcIdNvbWzLwwxwngl88ZUc46da\n9t8Ka2niPwpp+qKQWuIQZQO0g4cf99A1b0rR7DRNKj03S7ZYLSIELFkt1JJySSTyT1qPRNB03w7Y\nNZaNb/ZrdpDJ5fmM4DHGcbicdOg4rko0Z05czd77+v8AVzhw+HqUp8zd7rX1vf8AVnl93/x+z/8A\nJVf9Y3+p+51/h9vT2r0PzHh+HryRPfpImmsyvfHFypEZwXPZ/X3reqO4t47u1lt7hd8UyFHXJGVI\nwRke1XCjyX13NKdDkvZ7nh0Nz4htvDvhHWI/FOrG61a7Nq6yTeZEiFmUHYwwzAHOWzzj0Fbltq99\n4W1Lxrplz4gv57TTraKS3urv/SZomkVegJAJy4ABIGQD613p8GaCdP02yNh/o+ly+daJ50n7p85z\nndk8+uakn8J6JdXepXNzYLLLqkSxXhd2IlVQAvGcAjA5GDxnrWCw9SOz/F9rP8TmjhKsbNPX1f8A\nLZ/jY8w8MXGtr47sdHutS16Oy1XTZHDX9+ssrqyMVlRRnyTkcAkkY6msHT0vtH+E+oanpur6lb3D\naqLcIlyVRcY+YAYIY55PfAr2LSvh/wCGNEvba80vSlt7i2Z2ilWaQsNwwckt8wx2OQOcdaif4beE\nna7J0hR9skEkwWeVQzBtwwA2FGT0GBUfValt9deve3kZ/UqvLur69XpdLyOb1WzvvDHirwhZRa/q\n16L2+la5a6u2YSfLH8u0YG3OSBzjJ5rC1rVL+0+JD+FofFNzFpN5eRSTXH2iQy2hIJNuspPy7uPp\n8uejA+s3+h6dqeoWN7fW/m3GnuXtn3svlscZOAcHoOuazW8BeGpNLutOk0tXt7uf7TPulkLvJ/e3\nlt3r37n1Nazw83flel/Ptt8zaphajbUHpe+77Wt8y34m1CfRPCGpX1ihkmtLV3jDZbBC8E9zjqfp\nXmRvNW0LQPDfiiDxTf6jd6rcRrcWM8weGQSZLKifwlSNvHQ9MdK9gEEYtRbspki2bCJSX3LjHJbJ\nP49awNN+H3hbSdX/ALT0/R4YrsMWV97sEJOcqpJVfbAGO1XVpTnJOL/4Hma16NSpJOL/ABenn5nG\neAdG8j4r+KG/tHUJPsUiDEk+RPvDf6zj5sdvSum8R/8AJUPBf+7f/wDopa37HQdN03Vb7UrK28u7\n1AqbmTzGPmEdOCcDr2ArM1rSb278eeGNRt4d9pYrdi4k3qNm+NQvBOTkg9AaSpOFPl87/wDk3+RK\nounS5f7yf/kyf5GB8L4rfTG8Q22pMkeuR38kl60pw7xnBV8nqpyTn3z3rlGtY4/h54+u9LUJotze\nL9hCjCHbIA7IP7vIA/3cdq9Ag+Fng22MnkaPt8yMxv8A6VMcqeo5eqGvfDrTrDwRrVj4O0vy7y/j\njUp9oY+ZtcEcyNgcZ9KxlRqKFmlon66r0MJYeqqdmlopbN31XodoIRc6UIGeRFlg2FonKMuVxkMO\nQfcdK8i8D6zqOsePYtFvvE9xcWOltM9o6SOrajh+N7fxgDnBzwMcjJr1yWyjvNJayvFYxSw+VKqu\nVJBGCMqQR+BrLXwR4dSLTY4tNWIaWxa0Mcro0RJyfmDZbJ65JzXRVpznKLj0/E6q1Kc5Rcenrr5G\nV8VP+RHP/X5b/wDo1aoJbw2nx2mudZ2hrqwUaU8vQEAB1U9m+9x1wx9a3PH+kX2t+FTZ6XB58/2m\nF9m9V+VXBJySB0FLf/D3wvqmtPq19pnm3zurtL9olXLKAAcBgOw7VNSnKVS8Vtbf5kVac5VbxW3K\n9fK/kYUdvDc/HX7Vom3bbaeU1WSL7pdiQiMe7cKcf7PtWl8KP+SZ6X/vT/8Ao+SrmmfD7wxo+tJq\n2naZ5N9GzMsv2iVsFgQeCxHRj2o+H2k3uh+BbDT9Uh8i6haUvHvVsbpXYcqSOhBop05RqXkt77fI\nKVOcKt5Lfm2215fI6Siiius7gooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooo\noAKKKKACiiigAooooAKKpXWtaXY3sVne6lZ291NgxwTTqjvk4GFJyckEfWku9c0mwmeK+1SytpI0\nEjpNcIjKhIAYgngZIGfU1PNHuTzRXUvUVQGvaO0ttEuq2Jku1DW6C5TMwJwCgz8wyO1VLLXnufEO\nq6dNBbww6eqMJxexuzgjJ3Rj5o8f7XWjniL2kdNTaoqjp+t6VqzOulanZ3rRgFxbXCSFQemdpOKS\nHXdIudRawt9UspbxGZWto7hGkUr94FQc5GOfSjmj3Hzx7l+isjR9bl1TVNVtJLaCFbCYRo8d4kzS\nA55ZF5jPHRufyqzp+uaTqzumlanZ3rRjLrbXCSFR6naTihSixKcXsy9RWW3ijQFCFtc00B3KJm7j\n+ZhjIHPJGRx71pSSJDE0krqkaKWZ2OAoHUk9hTUk9mNSi9mOorNg8R6HdXMVvbazp808wzFFHdIz\nSDn7oByeh6elE/iPRLW4e3udY0+GaNwjxyXSKyseQCCcgn0pc0d7i542vc0qKoy65pMGpLp8+p2c\nd65AW2e4QSMT0AUnPNY1142tbHxvLoN+kNrbxWQu3v5rgIi5cKFIIwOSOc/hSlUjHdilVhHd+R09\nFVDqunDTP7SN/a/YcbvtXnL5WM4zvzjrx1p1hqVjqluZ9Mvbe8hDbDJbyrIob0yCRnkfnVcy2K5l\ne1yzRRRTKCiiigAooooAKKKKACiuFHxC1LVLi7PhHwrPrNjaOY3vDdpAruM5CBgdwxjpzz06Z6Lw\nz4lsvFOk/brASRlHMU0Ey7ZIZB1VhWUasJOyZjCvTm7Rf9eXf5GxRWVceKvD1pcPb3Wu6ZBNG214\n5LyNWU+hBOQav2d7a6hapdWFzDdW8mdksMgdGwcHBHB5BFWpRbsmaKUW7Jk1FZkfiXQpZYoota09\n5J32RIt0hMjZxtUZ5OeMCrFnq2nahNNDYX9rdS25xMkEyu0ZyRhgDx0PX0oUovZgpxezLdFZ9tr+\nj3l8bK01axnuwSDBFco0gI6/KDnisTRviJoesahq0Au7a2h059onmukAnXozrzjZnHOecjpSdSCa\nV9yXVpppN7nV0VzeqeM7K2a1XSJ9N1OSW5SCZBqcMRhVs/NyfmPHCjk9q1jrelA3QOp2ebL/AI+h\n9oX9x/v8/L070KcW7XBVIN2TL1FR29xDd26XFrNHPDIu5JI2DKw9QRwRVVdc0l9U/s1NTs2v8kfZ\nRcIZcgZI2Zz056dKq6L5kupeorir34iqnj2PwzpVjBfPuRZ7hr+OIRkthgFIO9lHJUHdwRjitzX/\nABB/YdxpEX2bz/7Svks8+Zt8vcCd3Q56dOPrWaqwabT2MlWptNp7aGzRXHaj46uT4iuNF8LaFLrt\n3ZruvGFwtvHDnGF3tkFvbjoeuDh2l+P4dQ0rWnn06ay1TRYXkutPmcZ+VSRtcDlTj72O44wRle2p\n3tcX1ilzct/z6eex19FUtH1IaroFjqbR+QLu2juChbOzcobGeM4z1qMeIdFNit6NXsDaNJ5S3H2l\nPLL/AN0NnGfbrWnMrXua88bXuaNFZviLV/7A8OX2q+R9o+yRGTyt+3fjtnBx+VYms+O/7PbTLLTd\nKm1PWNTgWeKxikCBEP8AEzkYA6847HOKmVSMN2ROrCHxM62iuU0TxrNd+IBoXiLRZtE1OVDJbxtM\ns0c6AZO11wMjnj269q0vC3iD/hJdIe++zfZttxJBs8zfnY2M5wOvpRGpCWiYRrQm7J/0jZooorQ1\nCiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKypbfxCdaEsGqaYml\n7lJtn02Rp9uBuHmicLknOD5fGRwcZOrRQAUUUUAfPPiOyQeLPEtv4g1CS0urq83WluNGW7muo2Yh\nPKkYgrgYGAw6YGTxXWWWh2198W7Gx12Bb/yPD0e4XcIyzjC5ZcsA3J7nB7161RXFHCJO7fW/9ann\nRwMVK7fW/wCfmfOtpplknwYtNTW1iF9/bKr9p2/vAOm0N1A9umeetdbdWyXfiz4j280V5Kkltbgr\nZIHl+6OVUkbsdSM8gGvXaKI4RJJX/Dya/UI4FRSV+3Tya/U8O8A3FqPGllb6LHb6zH9ikEt9aWT2\nVxbrsOEdQRGx3ADcdxJP3uBWVoosdK13RrbR411a8TUBnT7vTHtb6z2sSSZIzh+ASQ5IAA+XG4D6\nGopfVNEr7eXp5+XW4vqOiXNs+3p5+XW54re2epX9n8SbfRkkkuHv4sxRAl5E3NuUAcnjt3Gal0uX\nQdZ8XeFG+H9kYpbJd2pSwwGIJHtAKyNgBmPzDPOfU17LRVfVdd/w876di/qeqd/w876djwew0PTJ\nvg54n1OaxgkvlvZAly6AyIAyYCt1A5PTrnmvVIyW+FqliSTowJJ7/ua6Soby1S+sZ7SUsI54miYq\neQGGDj35q6dBU1p2sXSwqpLR9LfmfPUDaHffDrSdJ0jTmHi2S6V43S2YSsN7fvPMxym3tngjOOM1\n3vhfR7DUPi54tm1K0gvJbVoPKaaINsYrksAehyo57V3ugaJbeHdCttJsnlkgtlKo0xBY5JPJAA7+\nlaNZU8LblculunZP/Pcxo4Ll5XJ7W0t2TX67nz+w0e28L69oviLTZZvGVxdv5W61Z5pnZhtaN8fd\nzk8EbhnGc1p3b6dpPxJ01vHsCXMdvoMK3LSwmdI5cbdzABs85GcEZYGvbaKPqluu3l+fcPqNre9t\nbp2vvrqeDxWEFx8OtSnmj1Cy0OXXTc6fJFaidYIuQGeNmGY8HB5PI79+l+E91HLr2tR2FvaT2eyP\n/iaWVvJbJcMOgaJvkVsHOEUY565zXqdFVDC8klK+xdPB8k4yT28vXz8+3zCiiiuw7wooooAKKKKA\nCmTR+dbyRbiu9Su4dsin0UAeUeDvF2nfD7QH8OeL1n06+spJHjBgdlukZiQyEDBGQRk4Hv1x0Hw0\n0++jt9a1jULSSy/tjUJLqG2lGHRCTgsOxOT+We9dvRXPCi4tXei2OSnh3BxvK6jtp+f9I8/1jwr4\nhu9Zuri10LwPPDJKWSS9s5GmYerkDBaut8OWd1p+gW1rf22nWtxHu3xaZGUt1yxI2A8jggn3zWnR\nVxpRjLmRrCjGEnJHzpPpdivwZvNTFpF9uOsFftJQeYF/uhuoHtXZ3Fha+H/izeQ6Fpqoi+HJHFpa\ngoZmBOF+XncdoGeueetesUVzxwijbXt07X/M5I4FRtZ7W6dr/nc+ddAuLOfxN4Ru7JrSIfbVjmt7\nPT2iS2cn5VadyWlZhzgk45xV26XSdMi+IWmPbw2+oGYtZxi32lYN4JCnGFXBXjIzxXvtFSsHZWv+\nHlbuQsBaNub8PK3c8e8RaRp+leC/BLadZw27z39k8zxxgNK2wnLHqxyT19azviOtvq3jW8vdKsbi\n5stKjiXXngk2rMBIP3fuQByf9nttzXuVFXLCqSavpp07Gk8EpJxTsnbp2X9fkZcVzFqnhXz/AA9I\ngSe0P2NkwoUlcKPbBwMdsV4eF0ebwHD4ftdKn/4TkXf3RbMLgOHLFzJj7uztnjrjjNfQlFXVoe0t\nr0tt+XY1rYb2trvpbbv27M8x0XTLUfHjUxcWls8semJPuEK4ExMe6QccMSTz15Nbnj//AJCPhL/s\nOQ/+gtXZVja/4f8A7cuNIl+0+R/Zt8l5jy93mbQRt6jHXrz9KHRcYOMerv8AiDoONOUY63d/xOA8\nOeLtC8P/ABH8R6e12ZYdQuWuFuFif93Ku7zI2GM8HOCMiqWm69pfijVvHOsWs5jml0iWKC1ZG3NC\nkYBlY42jLbQBnPXjvXstVNWsf7U0W90/zPK+128kHmbd2zcpXOOM4z0qPYTta+iu9v8Agmbw07W5\nlZNu1u/z8zP8IOsfgHQnkYKq6ZblmY4AHlLzXj1tNpsfxCj8TyWMy+EZNTdIZGb90txtH77ZxhS3\nPtg/3dte36Np/wDZGg2Gm+b532O2jg8zbt37FC5xk4zjpmrtVOi6kYpu1i6mHdSME3bl8r6/5HNf\nEMhvhxrRU5BtGII79K5Cec+EfGOk+KNRglfR7rRorOa5jjLi1f5SCwHIBwPzNeheItI/t/w5faV5\n/wBn+1xGPzdm7ZnvjIz+dW7G2+x6fb2u/f5ESx7sY3YAGcfhTnSc5322+8dSjKdTmWlrW9U2edDU\n4vH/AMSNCvfD0csul6J5r3GoNEyI7uoxGNwBJ4HHuT067Pwt/wCRRm/7CFz/AOjDXZ1jeFvD/wDw\njWkPY/aftO64kn3+XsxvbOMZPT1ojSlGpzPXe/4f5BCjKNVTbve9/wALfkbNFFFdJ1hRRRQAUUUU\nAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRWH4o8Z+HvBdnBc+J9TjsIriTyoi6sxdvYKC\ncDucYHegDcoqO3uIru1iuLaRZYZkEkciHIdSMgg+hFSUAFFec/E7wVr+vsdX0TxvqXh+Kxsn32lp\n5gWZl3NuJWRcHHHQ9K4f4M+GvFfiaw0vxdffELWZLeO6bzNMmlllSUI2MFjLjB/3TXbHDQlRdVzW\nnSz37E3d7WPfqK81ufjt4VguZ2hs9Zu9Ktp1t59btrEvZRMcdXznjcOinPbORnf8V/Efw/4S0Ow1\nO8lmvU1IqLCCwQSyXe7BBQZAIwwOcjqO5AOTw1ZNJxeo+ZHV0VwmgfFzRPEHjCHwzBp2r2epyQGa\nSK+tRCYMAnY4Lbg2MHgEYI5qndfHHwvbX1wotNYn0u2ufss+twWJexjk4yDJnPBIHCnOQRkEEv6r\nXvy8ruHMj0eivHfiv8TNT8P+MfCul6Nbai1lc3UdxdSWlsHOoRh0IhgbPzHk7gMZ3IM4JBztQ+IV\n3pXx8R7n+3pNPuNFjlh0OGOSSVpWQNt8gHAkAznPTB5rWGBqzipd039wuZHudFch4Y+J3h7xRoep\n6nC9xp8ekMwv4dQi8uS2wCcsAT/dPQ54I61m6L8ZvD2s6xY2LWGs6bFqbmPT77ULLyre8bIAWN8n\nJOeMgehwSBWP1atdrlem47o9BorgNc+MGjaH4yu/C50nXNQ1S2jWQQ6fZiYzblVsIA2chWycgDCn\nnpnubW5+02EN08Mtt5sSyNFOArxZGdrAEgEdDyaidKcEnJWvsF0yaivF/G/xq0rUvBniGHw0muJ5\nKGCHXLe2ZbUTZXCrMpypOeMgfqK29C+Itn4Z+E/hS88QS6hqmp6pbolvbW6G4uryTjdgE/MRuGST\n3HcgVu8HWUFJrVu1vlcXMj02iuJ034o6XrOgave6fpurG/0kf6Vo8lni9Qk4H7vPOevB6DnFeN+B\nvE1zq3g7xF4j8X6748zErtLcafchLVQ0seFgz8qygt90YAUnAqqeCqTUnLSzS+8HJH01RXnEnxU8\nN+EfD3hFNRfWriDWrRGtru4QTzBQE+aYqcljvGdobJzWp4S+KGkeLfEN5oSafq2karaRCZrPVrUQ\nSMhxyAGPTKnnHDAjIzjGWGqqLly6f0h8yOzorzib45eF4b6UC01iXSobkW0muRWRaxSTAyPMzngn\nH3eeoyCCfRUdZY1eNldGAKspyCD3BqKlGpTtzq1wTTHUV4x4H+KWr+IPjFrOnX+mayLH5ILW2+xb\nRYYyS9wM/KWx1OfQYFc98LPjJ/YPw7up/FCeJPEE0V9I091FE1ytrEVjCCSWRgFDNuwMnoemRnre\nX1kn3VvxFzI+iKK5TXPiT4e0Pw3pusmWe/j1YounW9jCZJ7wvjARDjnkZBxg8dSAX+EPH+leMpb2\n2s4L7T9QsGAudP1KDybiIH7rFcng/XjjOMjPL7Gooc/LoO6OoornfG/jXTvAXh3+2dYhuprfzlh2\n2qKz5bOOGZRjj1rA0v4zeHNT8Q2GlSWesaeNT/5B95f2XkwXmcbfLYnJzkYJAHI9Rlxw9WceeMbo\nLo9BorzzUfjNoth4q1Hw7Bouv6lqdgwDwafZCYyDAJZQHzgZGScdaueI/iro/h7XF0RNO1fWNWWH\nz7ix0m0FxJapgEGT5gB94dCe2cAjL+rVrpcu+vyDmR29FeU/EDx9a+IP2f8AVvEngzVLiBlMCCaF\n2hmt38+LchIOVbDYODgg8Eg5J4U+M2iLp/hrTNXi1iJ721ggTWLy1ZbW5nEahwJWO5jvJUtjGec4\n5q1g6rp86Wzat6K4uZXPVqK4rxP8UdJ8Na+dETTNY1rUY4PtE8GkWfntbx8YZ8sMA59/fGRnkPiT\n8Q4vEHwHm8SeDNRvbItdRRGSN2gmibcNyEqfQjoSDnrSp4WrNx0spNK/qDkj2SivNPDnxj0G5udE\n0e/g1e1k1CBEtdRvrQx295JtUHY7Hc2WOMkYz35GZH1vTj8Ydb0yDU/EX9pw6SZWtHuF/s5F2phk\nQHcJORzju1H1Wom1JW0uHMj0eivnjw14s/4x3/tXxhr/AIn/AOQu0X2zS7z/AEvoMLvkP3OuRmtr\nxf8AFPVdI+LPh7RrLT9cl0qCEyz29vZh59SLRHaU5/eKvGcY+YPnOBjZ4CpzuC6X/AXMj22iuX8L\nfEDSfFf9srbQ3djLos5gvYr5FRoyM8/KzDHyt+RqfwV4ysfHegHWNJtryC0894UN2iq0m3GWUKx+\nXJI5wcg8VySo1I35lt+uxV0dDRXKeL/iLo/g28s7C6hvtR1S+ybfTdMt/OuHUZy23IGOD3ycHAOD\njNT4raXqngnWtX0Sz1OTUNJBjudKNp/pltJuKDfHnGAQWOCcBWzyCKqOHqyipKOjC6O9orxP9n7U\ntR8QRXer6zqPim8u2R0Zr6UtpzZkB/cj++NuD2AJA616D4u+Iel+EL+y06az1LVdTvgzQ6fpVt58\nxRQSX25HAwe+evGAcXVw04VnRjq0JS0udXRXM+CvHukeO7O7m0mO7t5rKbybq0vYfLlhbnhgCR2P\nftzineMvHWleCLezbUo7u6ub6XybSysYfNnuHyAQq5A43Dv34yeKy9jU5/Z217DurXOkorkfCXxJ\n0Xxdc6hZwwX2lahp3zXNjqsHkTRpgHeRkjbz65HcAEE40Pxy8LzX0QNprEWlTXJto9clsiti8mDg\neZnPJGPu8dTgAkX9WrNtcr0DmR6PRXJ+MviNo3gq6sbK8hvtQ1K/P+j6fpsHnTuvPzbcgYyMdcns\nDg4oeHfi5oXiXVNU0+zstVt7nSrRrq6S7tliKbcbo8Fs71JwQQBkHnvSWHquHOo6BdbHd0VxFv8A\nFbQ7n4ZT+OY7XUBpkDbWhaNPOJ8wJwN+3qw/i6VLq/xQ0PR9A0TUpLfULqbXYY5rDTbS3Et1KrKG\n+4DjgMM8/TNH1ere3L1t80F0dlRXJaD8SNF8QaDq2pWsV7by6Mrtf6fdweXc2+0McMhOMkKcc+xw\nQccwn7QfhaTSIdUTSvEB095fJmuxYAxWr5+7I+7bnBDYUscEd+KqOFrybSi9A5keqUVy3ir4h6H4\nTsLCe7NzfTamVFhZ2EXmz3WcfcXIB+8OpHXAyeKr+Gvij4e8S2eqzA3WlS6OC1/a6nD5M0CgHLFQ\nTxwR1yMcgZGY9hVcOfldgujsaK890j40eHdW1aytGsNZ0+31GQxWGo31l5VtdtnAEb5Oc9sge+Dx\nXoVTUpTpO01YE0wooorMYUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAVyPj/4aaB8SLOyg8Qi5U2Uh\neGW2kCOA2Ny8gjB2rnjPAwRXXUUAVtOsLfStLtdPsY/KtbSFIIUz91FUKo/ICrNFFAGd4i/5FfVP\n+vOb/wBANeY/A6zk1H9n97KBxHLc/a4Uc9FZiQD+tev0V0RrctJ07btP7ritrc+atG8W2Ph34J6p\n8OtW0++i8Wnz7OPSxaO0k7ytlHUgEEYcHrk7eM5GbfiDQtR8Baf8Ltb1+3mlsdC+TUvLXf8AZGd1\nYZxxxnGR3QDPIr6Krl/G/hPUfFFvZnRfE+oeHbyzl3pNaMWjlBxlZI9yhx8oxk4GTwcmu6GOi6mq\nsm2313VtNNPxJ5dDyaz1q08ZftMQ3uhiZLS40SSOC6khaLzh5bjzVDANtycZI/h+lcVoVhoGleHb\nzwr441fxra6yl55B8OaU5Md0GZdrJGyFGz15POARnIr3rwr8N7rSfGE/irxP4il8Q609v9mimNol\nskUfoEUkZ9+Op45zXeVcsdCm+WnqrLr1V+68+y+QuVvc8H+KlvB4VvPhPe3Au4dI0W5WKee5UO8K\nr5BUSeWCC+2N/u5ztOM1c0y9tdW/aoi1GycTW1zoCT28u0jcjIpVgDyMg/XmvbKK5VjP3fK462a3\n7u/Yrl1Pm7T9FvvEB+N+m6SjSXc19G0cadZNs87lR7kKR+NUfC9p4R8Qx+GNHuNc8d6prdu8fmaT\nHIGh02WPClisiBUjBHG0kheDzX0/RWv9oOz07dfJLt5eQuQ8b0BFf9rvxOzDJTRUZT6HbbD+RNes\n6taSX+i3tnBIIpbi3kiRz/CWUgHj61boriq1nUcXa1kl9xSVj5d0zxfp2g/APXfAuqWV5F4jgM0c\nlkbViQGYN5rMBtCqD1JzwMcEGna3pUkPhr4YeI7+61Ox0O3sRbXl/pbFZrEseJNwB2g7sZAJO0jq\nQD9QUV3f2glLmjDdtvXurO2mn4k8h478LLDw3da54g1/wzf+KtX3232abVdaZWhuuBjYxUSMyhQO\nRwOO4rh/Cv8AyaF4m/6/G/8AQ4a+mqKz+vPmbt1i9/5fkHKfOWoqrX/wLDAMPKiOCO48giuhvbZ7\nz9qbULaF/Lkm8PNGr/3SUAB/WvbKKHjf7vRrfu7hynyToVj4f0/wzceFfG2reN7fWo7zyH8NaW2U\nuMsrIyRshQ5yDy2TgEA5FfVGiWKaX4f0+wiE4S1tYoV+0MpkwqBRvK/KW45xxnpV6is8Vi3iOluu\n/Xy0/wAxxjY8P8HatZ+Hv2jvGFhrMjWlzrEkYsUeJj5/VuCBgDHckDtXOfDtVH7KfjFgoDNNcZOO\nT+6ir6TorT66v5f5ev8AL8uouU+X9b0mdfAPwx8RXE+pW2jWEJivrzTGKzWQdxiQEA7fTOOwHUiu\n/wDhLY+GL3xtquveGNS8Wa0yWwtJdW1hla3uMlG2IzKshZdo6gYH1GfYaKKmOc6bha2/Xo3ft+vy\nBRszyX9pP/kkp/6/4f5NXLeLvEdr8TNX8EeHvC9tdDVbG7jub+OS1eI6cqhNwfIHT1HHAGckV6t8\nSfA3/CwvCR0T+0f7O/0hJvO8jzfu5427l9fWuotofs9pDBu3eWipuxjOBjNOlioUqMdLyTl8rpeW\noOLbPH/h6in9o7x+5HzCOMA+gJXP8hVGy1yz+Ffxm8XXfjYT2ljr22fT9QEDyRyBSSYwVBO4bwCM\ncbeeoJ9zorP62nJ80dGkt+1utvLsPlPmSbSb+H4BfEHX7yzmsLbXtThu7K2nXa4iNyhD47A7sD12\n5HBBJ4m8R2njv4b+CvAvhy0uv+Eije1823NsyG1VISvmbiANrBg4YHG3k4r6bordZgr8zjqnda+S\nXbXYXIeF+N/HOpx/FfUPDfiLXtV8NaCtqhso9ItN1zqLnGPLmCsysXJHHy/JtODknz+wBi/ZU1y2\nlBSeHxAqyxOMMhxFwQeR0P5GvrSilTx8acYxjDaz37fLr8w5bnzh4n8R2nxDtfAfhXwxaXn9sWVx\nBLeQvbNGbFURVJYkDjndkcYA7kCuiT/k6TxP/wBi8f8A0CGvbaKh41JcsY6Wa37u/YOU+S/+bSv+\n49/Su88fala+GfjR4B8Q6472ukw6ayPdeUzqG2SDGFBJPzrwB3r3iirlmClK7jo+br/Mrdugcp84\nfF06j4O8c6lceHYmlg8daWLQKnAM+5FYj3KHj3kNe7+EvD8PhXwhpmh2+CtlbrGzDo79Wb8WJP41\nzN38NLvWfiNZ+JPEniSXUrLTJnn03SxZpEtsxIK5cHL4wDkgEkLzxg99WWJrxnShTi723/Jfchpa\n3PF/Ft0vgT9oC28Y+Iopl0C9077IL6OFpEtpf7rbQSM449dxx0OK3hNJfEfjT4g+OtPtp4dAv9Oa\n1spZYzGLspGqtIoPOMxk5x/Fg8gge40UfXPcty62Svfonfbv8w5dTy39nX/kjtn/ANfM/wD6HWb8\nVfHGraF8QtJ0a91q48MeGLq1d7jU7azE0kzc7kVtrFGACgFRlS+45GMeyUVH1mLryrSje99O1/l0\n9AtpY8M/Z52xeJvHkJe+LNeQyJ/aQIunQtMVeTIBLkEEn1PvXQfG618NT2WjzeK49cs47a4ZrfWt\nKjDLYOcD97yWCk7D8q5JQYI6H1OinLFc2I9va3z8rb2/QOXSx89/D+88T+Iz4q8N6V4nvPFXh6XS\nZ4oNXvbd4il3JGAqBpCX43HgnHGRjPPI6DZeGLjwpZ+F/E+reO311bloJfDFg+URt7MGWORNgGME\n5bIJJxjmvrKiuhZhZu0bXttZarvp+SQuQ8Q8avF4C+N3hjxZrouP7AGm/wBnG8ZDIbeULJwwUHkh\ngeOvz4BwazvBOor4s+N3j+7022lhS+0gpbpNGYmcbYlV8HkBwAwz2YGvoCuN0bwB/ZHxS1zxj/aX\nnf2tAsP2PyNvlbQgzv3Hd9z0HWpp4uDptSXvctk++t9v+CHK7ngVr4u0+y/Zv1fwZNFeDX4bk/aL\nT7LIPs6C4Ql5GxhQDhOTncQMc5rZ8UaW9ovwx8S6pe6tpugR6BbWlzqOkuyTWbGHhtwBKht4B45A\nYdcV9LUVf9opSvGG7bevdW00/wAw5Dw3wfp3hyXw94513wxd+KNSS402aCXU9bZDHeFY25jOA7EY\nwSwGM4rnAoH7FpIABa5ycDqftlfStFZrHPmvb7Se/bpt/XYOU8C8ReZ4d8QfDHxrqMUz6DY6XDb3\nc8cbOLUtHtDsBnAPmDtztx1wK2dQ8RW/xS8F+OdO8EaFOsbIrRaokPlpqUihCV5Ct5mF24Ofl25I\nyBXslFQ8Wmk3H3ltrpvfb/g/IfKfLnhiz8Ja/Z+G9Dvdb8d6jrcMqLJo0MgaLTpYztLFZUCog56E\nkA4OK+o6KKzxOJ9u07W+d9/kgirBRRRXIUFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQ\nAUUUUAFFFFABRRWD4n8X2PhX7Et3bXl3PfSGOC3sohJI5A54JHqPfmplJRV5EynGEeaT0N6iorWf\n7VZw3HlSw+bGr+VMu10yM7WHYjoRUtUVuFFZHhvxLZ+KdOlvNPjnjjiuHt2E6gEsuMkYJ45rXpRk\npK6JjJSXNHYKKKKZQUUUUAFFR3FxHaWstxcNsihQu7YJwoGScD2rDtPGFrqGl6XqGn6fqV3balcG\nGN4bfd5QDlPMk5+VMjOfTtUuUU7MiU4xdmzoKKKKosKKKKACisvWfEVhoU2nw3zSebqN0lrbpGmd\nzswGSegAzk/pmtSkpJuyJUk20ugUUUUygoork7j4j6JD4eudajS7ns7e9+xExxgMz4ByAxHy898H\n2qJTjH4mROpCHxOx1lFFFWWFFFFABRRWM/iW2t5dV+32l9ZW2loHlu57ciKVSMkxkZLY74FJyS3J\nlJR3NmiubTxzprt4eUQXefECs1r8i/JtAJ3/ADcfeHTNdJSjKMthRnGfwsKKyrvxFYWfiSw0OVpD\nfXyO8aqnyqqqSST/AMBI4yab4c8S2fie1u57COeNbS7e0cTKAS6gEkYJ4+YUueN7X1D2kHLlvqa9\nFFZ+pat/Z1zZQ/YL67+1zCLfaw71hz/FIc/KvPWqbSV2U2krs0KKKyte8R6f4chtX1FpN13cJbwJ\nGm4u7ED6ADqcnoO54oclFXYpSUVeRq0UUUygooooAKKKKACiiigAooooAKKKKACiiigAooooAKKK\nKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKypbjxCNaEUGl6Y+l7lBuX1KRZ9uBuP\nlCArkHOB5nOByM4GrRQAUUUUAeLahHoWr+KPGDePb4w3liWXTY5bgxeVEFJRolBG5j8pxzknOOaz\n7jSxrsfw8ufEsEk93qTSQXMksjh54FceXnnurZyOTnOa9rvtD0nU5km1LTLO8ljGEe4t0kZRnOAS\nOKmuNPs7u4gnu7SCea2YtBJLEGaInupI4PA6elcDwt73f9Xvr+R5jwTle7X+et9fyOa+JF1f6T8O\nb+XQ2khliRF8yJiHjj3AEg9Qcd+3X3rh9Lg0LR/F3hRfh/etNLeLjUooZzIHj2jLSLkhWGWOOMHs\nK9lqjp+h6TpLu+laXZ2TSDDtbW6Rlh6HaBmtqlFzmpJ9vl6epvVw7nUU0+3ys76euzPArS38NP4E\n128v78xa9aXrtZJ9qZXQ7lIMaA85OcnBxjPGM1s+LLf+1tS0ufW7/T7i5j0WGSfTNTuJbRg5Cl2h\nk4j3scjkkZHIOOPXh4Y0ASI40PTQ8b70b7JHlWzncOODkDmrGoaPpmrCMarp1pfCLPl/aYFk2Z64\n3A4zgflXOsI+W2hzLAS5OW66fP1PE9a1a41jwX4PtonaHSrmWaKb+2LljE7IwCCWWMKSo5wQB056\nZqpexzW/wr161XUrG8sYdRhWCOweZ4bdtxLqrSryv3TwzDknPPPvM+l6fdaethc2NtNZqFVbeSFW\njAXoApGOMcVG2h6S2lrpraXZmwU5W1NunlA5zkJjHUk9OppvCSbbb6W/C3/BCWBm225bq34W+7qe\nd+JfCun+Af7H8TaHBIg0+7H9oyeYzvPHIAjMSSefbgZY1ltKtl8KNT8Qasl15/ia+3vHazrCxjZi\nFQuysFTaGPTkNjjJr2G5tYL22e3vII7iCQYeKVAysPQg8GmS6dZT6f8AYJrO3ks9gT7O8SmPaOg2\n4xgYHHtWksNq+XRW/Ha/3G0sJ7z5HZW289r/AHHhvhnT7WTXfFejp9gexk0ZpjBpl3JLAXVlZDvY\n5Zlzyc4ySPUVDolvpcXh3wNJYNEb2bXomvQku5gwlIXcufl+UDsM9a9ztdF0qxmjlstMs7eSKMxR\nvFbqhRCSxUEDgZJOOmSahh8NaFbsht9F06IpKJkKWiLtcdHGBww9etZLBtW2/p3MVgGktV1/NM8m\nv00LV/EHjKbxzftDqNjK6aaklwYzFEAxjMSg/MT8pxzyc45pttbXXibWvAFv4ke4Ml1YXInbzGR5\nYgHwGYYOHQAE5yQx9a9gvdD0nUrhJ9R0yzu5oxhJJ7dHZRnOASMjmppLCzmvob2a0gkuoARFO0YL\nxgjBCt1GQe1X9Vd9X/wdb6/kX9Sber6/frfX8jxyC1h0jTPiLolnqMulafby24gY+ZIsO8tuB2gt\nhgApODx1zU/w1ktNH8dLpdstuz3VmS8uk6iZ7SXbyGaNwXVxyMlh7Lg163HplhFLcyxWVukl5j7S\n6xKDPgEfOcfN1PX1qLT9E0rSWdtK0yzsmkADm2t0jLAdM7QM044ZqUWnt/mxxwcozjJNafldu342\nPPPi9pmmT6z4WutTRBFJfpbXMjylFEBYFgTkYGCxzwR61kyW3hjVvGWu23je+WKysLeJdHje6KRr\nAU+/GQfnbAQ9856Ht69fadZapbiDU7O3vIQwYR3ESyLn1wQRnmoLvw/o1+kKX+k2NytuuyFZrZHE\na+i5HA4HApzw7lNyVtf8v6Y6mEcpuatrbdeVtfzPFxfXmq+G/BVv4xup10W6u5hczTSFPOVSPL8x\nuuOvPpznjIUTxaTqPjiTwXeSSQW+nQrbTxSmQom5A+xzzhRuAOeABg8V6j4z8HnxZYWsEWpSWBtZ\nRIq+UJYZMY4eIkK4GBjPHXg5qPwj4K/4Rq8vb+81E6jf3axxGVbdYI44kUBUWNeBwB7YAwBznD6t\nU57fj8rHO8JV9py9P5uvw27331PNfAmlqPFGnwW2o6IbfULJxe2Wn3F1I9xE0Zz5uQVRwT0yuD0H\nNYC2OlW/w11doGiTVRq4hZfOy6wqRt+TPTdnnHPrX0Lp+iaVpLO2laZZ2TSABzbW6RlgOmdoGaik\n8NaFNLNJLounO87b5ma0QmRs5yxxyc881X1N8ttOv4lfUHyKN11/E88u9Hj8KfFfTY/DpuFlv9Pu\nXuPMmeVrmVUdgzbicksAfrXE+HRdzXmk6tHrGkQaxNf4d2nunvpm3EGOSNQyhSOM7QMAc9a+hJLC\nzlvor2W0ge7hUrFO0YMkYPUBuoBz2qGLRNKg1JtQh0yzjvWJLXKW6CQk9ctjPNXLCtyunpf/AC/y\nLngW5Xi7K97fd+On4nnHhTwzpmt/EjxVe6pC1w9hqSPboZGCI+WO4gHBPyjrkVtfE1Tpi6J4pjUk\n6NfKZto5MMhCuP5D8a7O3sLO0nnmtbSCCW4bfM8cYVpW9WI6nnvT7q1t762e2vYIriCQYeKVA6sP\ncHg1qqCVNxW7/wA7o3WGSpOC3et/O919x88xz6vDZ3Nm+Wm8cJHMhAJCObhgR/3yc/Rvatu50WzE\nnxFsRDuhsbW2MC7iNpjjO08Y6Y6dK9kOkaaz2jtp9qWshttSYFzAMAYTj5eg6Y6U5NMsI57maOyt\nlluxi4kWJQ0wxjDnHzcetYLBtbv+rP8AVnPHANaN3/4Z/q7niNnpek3MPw7tNPlAS5mna++zXB3e\naUh3qSDlCQACBjAPaux8CWMOgfEvxVoWl74tNhS3ljt2cuEZkUkgnJ7/AKD0Fdtb+HdEtGhNro9h\nAYHaSIx2qL5bEAFlwOCQoBI9B6VZjsLOG+mvYbSCO6nAEs6xgPIAMAM3U4A71dPDcrUuqf6WNKWD\n5JKWl0193La36nl/jPSdEPxq0OXWFijtrq3Z55JZzGrOgYJzkY5CjAxn3rnrTw5Y6h8PfF2rXnnS\nXNjqVwbXEzKsLAISwUHBJyASQeFFe3X2k6dqbRHUtPtbwwkmI3EKyeWTjJGRx0HT0pE0fTI7Oe0j\n060S2uWLzwrAoSVj1LLjBJwOT6UpYXmk3prf8UKWCUpyelnf8V/TPEfEM99rmq6Bbazd6cbVtCgu\nY/7ZuJYoJJSo3vujIJkPPU9M98VctmuTo/gAXWoR6ii63tguIxJgxhlAH7xVY4OR0xgAdq9gutD0\nm9tILW80uyuLe3AWGGW3RkiAGAFBGBwMcVJJpenzC1EtjbOLMhrYNCp8gjoU4+XGB0pfVZXbvv8A\n8An6lLmcnLf/AIH+R5p4J8M6Zq/jzxRqepQtPPp+su1qDIwWJ95O4AHBPC9c/dFSfGfTbCb/AIR+\n8vkUD7elvLI0hUCJuWB5wBx17etek21hZ2ck8lnaQQPcP5kzRRhTK395iOp9zRe6fZ6nbfZ9StIL\nuDIbyp4hIuR0OCMVp9XXsnDq/wDM1eFXsHT0u/8AO54R43fRdRv9Vi0/7K0el2kMVrJf6k2zy/L+\nVbWJMF2PB3MzDuRyK0ZpbTWLzwTaeMr1v7Ek0nzWaecxxyzgMPnfI5A285HXrzz663h3RWmjlbR7\nAyRRiKNzapuRANoUHHAxxjpiue8VfD2PX009dNvl0qOwJ8u1+yJNbHOefJOFzyeffpWEsNNXktb9\nPmc08HUXNJWd+nfW55vbLZP4b8dQ2OtNa2Ul7BHFeSeZIrplgFZlBbaRxuwc8Zzmt/4YzWmm+Nrv\nR7WKEPJaea76VqJubF8MMMFbLo2Dj5nPPYAiu28LeDIvD9nqC311/alzqche8lkhVEkzkbRGMgLg\nkY6c+nFbOn6PpmkiQaVp1pYiXHmfZoFj346Z2gZxk/nV08PJOMnpb/N/1/mXRwk1KE3o1/m/8/8A\nhy5RRRXeeoFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFF\nFFABRRRQAUUUUAFFFFABRXHaz8TNJ0bUbu1ax1S8SxKrd3VpbB4YGPZmLDB//V1rqHv4F0ptRQmS\n3EPngqOWXbu4B9qiNSEm0nsZxqwk2k9izRXBQfGDw9KlpNLaatb2l0Sq3ctp+6Vhn5cqTluOig9R\n741NI+IOj6qupefFeaVJpiCS6h1GHynRMZ3YBP5deRxyKhV6UtpGccTRk7KSOporj9K+Jej6pqVp\naNaalYC+JFlcXtr5cV0cgYRgTnOe+PzIFUPE3xNs7W11q00eLUHubGJ4zqEVr5ltBNtO0M3OPmG3\nkYz7c0OvTUea4PE0VHm5jv6K4aw+IEdl4Z0I6jDf6tq9/aLO1vp9sJJSuOZCowAM8f061X1rx095\nH4Uu/Dd20VtqWrR2t1HJEu8DcA8bBgdpGccfgcYNL6xTte4niqSje/y9f+HPQaKq6nqdno2mzahq\nc629rAu6SRgTtGcdByeTjArmdL+JelandR2z6fqthNcQma0S8tdn2sAZxGQSGJHTpnIxWkqkIuzZ\nrKrCElGT1Z2FFeZ/D3xXr/ivxdqb31xdw6fbvIEtGsEWNOQFRpfvCQZzt56HntR4k8Q6uPiVPosX\ni638OWEditwstxbwurPuA25fHJBJ6/w9Kx+sRcOdJ6u3T/Mw+twdNVEnZu3T/M9MoryvS/iBqllo\nnix7+/g1tNH2C11KGJUSZnO1QQvynDY6e/J4NdTovig2raVoviCSd9VudNF7JdPGiRN3ZflxyvP8\nPQZqo4iEv687Dhiqc7f11sdXRXGW/wAUdDuvDJ1uCC9aL7YtkIPLTzWkIBGBvxgg9c9jXZ1pGpGf\nws2hUhU+F3CivP4fHOoXHxYu/Dxsr1LGOExJ5VqCyvlf37kk4j64PTkZBrX8Aa/ea3otzBq8gk1P\nTbuWzunChd7KeG2jgccfUGojWhKXKvP8DOGIhOXKvP8AA6miiitjoCiiigAooooAKKKKACiiigAo\noooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACiiigAooooAKKKKACii\nigAooooAKKKKACiiigAooooAKKKKACiiigArKl8OWU2tDVXn1MXAZW2JqtykGVAA/ciQR445G3B5\nJzk1q0UAFFFFAHjfivwX4t1zUdciudNbU1knMumXcmqbYraPrsSHI+YjCknAyM+59Lgsrr/hBo7F\n4Sl3/ZwhMRYHD+XjbkHHXjOcVs0VhChGEnJPc5qeGjTlKSb1PJz4K1//AIQLwhpw0/8A0rTdR8+6\nj86P92nmM2c7sHgjpk1o6v4I1TWvFHi9mRbe01SxhitbhnUhpECHBAJYDK4Jx9K9Hoqfq0LW/rRW\nI+p07W16fgrfkeYR6B4s8Qv4d0zXNIt9KstDmjmku1ulkNyYwAoRV5XI9fr2wa8nhzxho+l+JfDu\nl6Nb39lq0008N+btI9gkXBUoeScKAOgBOScV6vRS+rR7u/f8OwfU4v7Tv307WtseN6h8OdUKeH72\n40CPWhb6alnd6ab4W7I65IYSA4I5x1P9RoReBNWttL8LJb6Ta20lvrKX97BazsUgT5c8yuSxAUZ2\nkjI4z1PqlFJYSmn/AMN/kSsDSTb/AMvLy8jnvHXhuXxX4Qu9KtpUhnkKtE8hO3crA4OATjjtXM2+\nkeKvE2v+H5/EGjwaLbaIfNZluUma4kwBhQv3Rleh9epr0eitZ0YzlzP+rG08PGcuZvt87ao4z4ea\nDqWhf29/att5H2vUpJ4f3itvQ9D8pOPoeahm8INqPxZutV1fSre80ltMEUb3CxyL5wdf4Dkg43c4\n/Gu5oo9jHlUeiBYeHJGHRO5xXxE8PwH4XapY6NZw2yRKLhYbeMIvyuGb5VGOgNZPjTQtR8V+ENB1\nbwpGJr6OAqAsioTDNDtcbmIGRwMe5r0plDqVcBlYYIIyCKjtbW3sbZLaygit4IxhIokCKo9gOBUz\noKbfZq33E1MNGbd9mkvud0eXWPw21DTviBp3kRD/AIR+3EF07B1x9oii2D5c7slgGzjvXe6Frs2r\n6hq9rPYi1Om3Zt1YTiQSjG4NwPlJBB28kZwa2ajht4bZWW3hjiV3aRhGoXczHLMcdyTknvTp0VTf\nubDp4dUn7j0vc4p9H1vT/jBJrdppi3mnX1oltLMLlENvyuWKnlsbOg9evao/hchuU8Q60v8AqNT1\naaSD/aQE4b8yR+Fd46LIjJIoZWGGVhkEelR21rBZWyW9nBHbwRjCRRIFVR6ADgUKjafMn3f3gsOo\n1FJPTV/NktFFFbnSFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAU\nUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUUUAFFFFABRRRQAUUVwHxW+Ktt8L\nrDTp59Lm1KS/lZERJPLVFUAsS2Dz8wwMc88jFAHf0VT0jUo9Z0Sx1O3R44r23juESQYZVdQwBHY4\nNXKAKN1relWOpW+n3up2dve3X+otpbhEkl5x8qk5bn0p13rGmafeWtpf6jaWtzeNttoJp1R5zxwi\nk5Y8jp6ivAP2gtO1PVfil4atdB3f2l9ieW22HDF42ZxtP975ePfFR6j41h8eeK/hLrCFVuvtrw3k\nS/8ALOZXh3cdgeGHswr1YYDmhCaejTb8t7ffYjm1sfR9Fea63498S6l8Qrnwh8PLDTJLnTYRNf3u\nrtIIUJAIRQnzZ+Zeee4wMZrm9a+Iev8Aij4SeLbVLW10rxBoRNtrEZkfYIyGDNCynOSVIAJIwDyc\niuaODqStfrb5X2Y+ZHtcM8VxEJLeVJYySA6MGBIODyPcEU+vHfh5fePLD4KaW2jaRpeqXEipHpyr\nMYxDCc7pLjew3EHsnXNWPCvxM8TX3xB1HwZra+HbnUEs3ltbvSJpGgSVQcxybiW69fuldp4ORhyw\nc058rT5b9ewcx6pJe2sPn+dcwx/Z4/Nm3SAeUnPzN6D5TyfQ+lM0/UrHVrJLzSr23vrWQkJPbSrI\njYODhlJBwRivn34W3/imOb4h316ukXlrC9y+pJcCWRpJlSTaqA8GLIIIbnbgV0mj/FQaH8DtJ1uD\nQbCLUdSupLSx0vTYfs9uZfNYZxk4GBk+pOOM5GlTAyi+WOrul96uJSPZ6K8us/H3i/w7400fQviP\npukrFrvyWV3pDSbY5QcGOQOSScsgyOBnqecZ1v8AEfx/rvjTxZoPhrStCddDlby7m8MqLsVmGxgr\nHc7Y4PygbWz1FZLB1Hrpa17303t+Y+ZHsVFea+F/iD4j8a/B6bxB4d0m0l8RRyeQto7YgkdWXcRu\ndcDYxOC3Xue/e6NLqE+g2EutQR2+pSW0bXcMRyscxUF1HJ4DZA5P1NY1KMqTalunYady7RXlU3xX\n1DSLzx/Za9BZR3PhyIT6eI43UXCP/q9+WPdogSMfeNYlv8cdYuPhHHrkdjp58RS6uNLS08txFvI3\nA7N+77vH3uv5VusDWeqXb8VcXMj3CisvWJtZtvC91Lo9vb3esx2xMELfLFJNjgcsMKT6sPrXln/C\nTfH7/oSNB/7/AKf/ACTWVLDuqm1JL1aQ27Hs9FVNJkvptFspNXhSC/e3ja6ijOVjlKjeo5PAbI6n\n6mvMn+IXjbxVrmuw/DbStHk0/Q3aCSfVWk33cwBysYQgDkEDdweCSM4Cp0ZVG7Wst30Bux6Zqmsa\nZolqLnWdRtNPt2cIJbudYkLEEgZYgZwDx7Vcr50+LPje3+IH7P8Ap+s29s1pINZSC4t2bd5UixyZ\nGcDIwQc4712x+I3i/wAPeN9CsPHWjaXa6V4iYx2jWMrtNaSErtSVmwrEblU4AHJIJwRXS8DNQT+1\nrdenbuTzK56rRXieufGLxHJ4y1/SPDreFtOTRW8pY9fuXilvXBYHyjuVOwGGI6g5weOm8bfEzUPB\n/grSru+0eGx1/Vpfs8dld3StDbP3eSRTgoODwRwwyRzjN4OsnFdZefzHzI9Gory7wJ8Ub7WPHE3h\nTxFLoN5dtb/abW/8P3JmtnA6xncSQw5Pbp06E5ngL4gfEb4gm8k02z8NWlpYXhgnnuEnJcYyAiK/\n3hjkkgHcMDg5Hg6sb81klb8dg5keyUV47F8aL6P4M6j4jv7O0i8RWN82nPZBG8vz94wNu7djYcn5\nuSpwew6fWdY+IUOj6MukaToy3k1t5+q6hqExjs7MhCWTYH8w88buQMc9SVmWFqRdpWWrW/YOZHd1\nSu9Z0vT9Pe+v9StLWzjcxvcTTqkasG2lSxOAdwIx6jFeQad8TPEvjH4d+OILZdHi1jQo3V7y0mkN\nvLCRJl4iCTu2xsVOcElTxXN+G9b1rw9+y+dR1HTPD+qaVHKo062u7d5i2bqQSmdSQpIbBTb2610R\nwE/tPXmSt6i5j6NiljnhSaCRZIpFDI6NlWB5BBHUU+vMdd+ImqWJ8K+GvCOl2M3iDXLOKdEn3Ja2\nkW3JYqpztAV8AHgL34Bm8M/ELWz4o1bwl41sLG212ytTeW82ns7W9zHjsG+YEcdTzzwMc87wtTl5\nvn52va4+ZHpFFeB6f8X/AIj6r8NbzxhaaN4eSy02YpcGUzbpxlB+7Tdxt3clm57DIweh8XfGO607\nSfCL6FbafaXHiWITC51iRhbWi4XIcpgnlvvcAAZI540eArKXLpu1v21/IOZHrdFcV4A8S+JvEFjf\ntr9lpEiwuRaalo16k9rd9QQBvLKQQPvYznoMc6Xga98V3/h9pvHmmWumap57KILVgyGPA2nId+c5\n79ulc86Lhe7Wnn/X/AHc6CWaKBA88iRqWVAzsACzEKo57kkADuSKfXhfx0m8SL8QvAkOnyWAtn1K\nN9PjmaTDXQkjGZgONg3Lgr82C/qK9ngbU00CNrtLZ9WFqDKsGfJafbyF3c7d3TPOOtXUoclOE7/E\nCeti7RXjH/CTfH7/AKEjQf8Av+n/AMk16j4WuNbu/DNnP4rs4bLV3Um5t7cgoh3HABDN2weppVcO\n6Su5J+jTBO5pmaJZ0gaRBK6s6xlhuZQQCQO4BZcn3HrT68D8OzeMZf2pNeRpNIeeK2iS7V2mMaWW\nYWAh7iTDKcH5dxf1rY8Z/Ef4h+EbS51rUdK8N6dpi3QjttOvLtnvrmMFQWTY+w/eyccqOoIGTu8F\nLnjCMk20nv3FzHslFeZeLPihqVsvhTTvB+m21xrXieJLiBb1j5VvGwDEvtIJ4J6f3SeelV/D3xE8\nWy/FqTwb4o0zTIPs9gbiWSwEjmVgAQ0ZY/dOfulcg8ZrNYSq4c3k3vrZBzI9VorwnQvjR4w8USC8\n0C08KTQte+RFoU+oGHUZV4OVLsEPB+8Ae/Bwa9P8Q6h4vg1rw6nhrSLa4065mI1h7l1ElrHlMFcS\nAE4MnQP90fiVMLUpSUZtJ+vb+vmCkmdMSFUliAAMkntVTTNX03WrQ3Wjaha6hbhihmtZ1lTcOoyp\nIzyOKsXDbLWVsBtqE4YZB4rz34HeIP8AhJfh899/ZGlaTi9kj+z6VbeREcKnzbcn5jnk+wrKNO9K\nVTtb8bjvrY9GorzjxrrXxXsfEjw+B/DOlalpQjUrPdSqrl8fMMGZOB9KueAdW+JOoaldJ8QfD+na\nVaLCDBJZyBmd89DiV+MZ7CreHap+05l6XV/uC+tjuJporaCSe4kSKKNS7yOwVUUDJJJ6ADvVafWN\nMtdTt9OutRtIb65BMFrJOqyygdSqE5boeg7V47+0pNrUeh6JHaSWyaZLfosilnEjzAMVBA+Ux4GT\n3yBWrqPiLV7H4seCdG8R6P4cudTvbeRri+gtXd4CDJgQSOdyjAGcg8k1rDCc1ONS+/Np6C5tbHrN\nFeHQ/Ff4g6rbeLbjRdJ0D7N4ZnmM01yZQZIk3fKqBuXwhJOQO2KZd/Gjxla+ENK8bS+H9Jj8N3c6\n20kHnyPdOwLBnU8KqkowAIJ454Oaf9n1r20+/rukHMj3SivL/HnxSvNH8Z2vhbw3NoNtdtb/AGq5\nv9euTFaxKc7UG0glzwe/BHHUjY+F/wAQW8eaVfi9htotS0u5NtdGzl8yCU84kjbJypwccnp1Oaxl\nhasaXtWtB8yvY6y91jTNOurW21DUbS0uLx9lrFPOqPO2QMICcscsOB6j1q5XmvxL8Rf2N4+8A2X9\nj6Tf/wBpag8X2i+tfNltcPCN0LZGxvmznnlV9KzT8Q/Her/EzxH4T8L6Zocn9mANFdXxlRUX5c79\npO4nJAA246npVxwspwUl2vq/OwubU9cqnqOsaZo8cT6vqNpYJM4jia6nWIO/90biMn2FeY6F8ann\n+D2qeLdb06NL/S7k2clrbsVSaUldu0nJUfOM8n7pPpXB/E3VfHGq+GPCV34z07SYLa81GG4t3055\nA0JK5CSq5PJDZBVjjaQe1a0sBOVTlnprb8L6CclbQ+lqK8n8Y/FfUbTx7ceFvDFx4csZNPgEt3e+\nIrsxRM7AERR7SCWAYHPPccYybGlfErXfFvwhu/EfhTT7L+3NOkaO6spg00UhQAv5ZVgTlSGXk85X\nnrWP1Oqoqb2dvx2uPmR6hRXkvgb4s6x8RfFdlaeHtOtoNKtrKObWbm4idmjmYH91FhgOSCATnIDH\nHGD61WVajOjLlnuNO+wUUUViMKKKKACiiigAooooAKqalpOnaxbrBq+n2t/CjiRY7qFZVVh0YBgR\nketW6KAEVQqhVAAAwAB0paKKAPNPFnhLW9T+OPhLxBZWXm6Xp0LrdXHmoPLJD4+Unceo6A1yOsfB\n7WLH486Z4i8OWQl0J76O+ucTIgtn3ZkwrMCQcbvlB647CveaK7aeNq00krbW+X+epLimeU6n4Z8V\n+EPipqPi7wdo8PiG01qBUu7FrtLaSGRQPmDPwV+UHueSMcA1V0b4Ya8/gjxtPrDWsXiLxcrStbxt\nmO2PzMse7nu5BIyOnJ617BRS+uVOVJJX016u2wcqPn3UfA/xA1n4S6J4em8OC3OiXMfnaedUjxqk\nXXO5SAmD2Ld8g5AFavg3wJ4i074x2PiSXwhY+HtIeweA2lldRSfZSFIHmEY3sxGcqDwy5OQa9tor\nR4+o4uPKrO/frv1/r0Fyo8V8OeD/ABh4d1Lx3pLaAt1p2vtdS22ppfRKAWSTy18sndklwCTgA+o5\nqtbfCbxFdfA/QtMaOKx8SaHfSXsEE8ivG581m2FlJAyCCOeoAOMkj3Oik8dVvdJXun9yt37D5UeS\nf8I54y+IHjzw9q/jHQrfw3p/hxxcJALxLmS7nJByCnCqDGnB59N2flueA/CGuaL8QfH+palY+Taa\nxOHsZPORvOG6Q5wGJX7w+8B1r0+is5Yubi4JJK1rdtb9+4cp5B8OfD3jfwH8Fr+0tNGgbxGt40tr\nZXE8bJIrGMEllkA+6GP3h0r1LRpdQn0Gwl1qCO31KS2ja7hiOVjmKguo5PAbIHJ+pq7RWdau6zcp\nJXbv/XkNKx4n8XfhZrfirx9pmo+H4N1jfwx2esMsyJtiSZW3EMQW4wflyf3Y46ZafhPrH/C/11ZL\nbb4UW7GqKRKm0XIjHGwHdnzBnOMYr26iuiOPrRgoK1kmvv8A17C5Ve5n6/otv4i8PX2j3ryx299A\n0EjQkBwrDBIJBGfwNeUf8MveC/8AoJ69/wCBEP8A8ar2eisKWJrUVanKw2k9yppOmw6Potlplqzv\nDZW8dvG0hBYqihQTgAZwPQV5NY+HvHnw11vxHF4O8P23iLTdYma7tZWvY7d7SVgeHV8bwCegPIA5\nBJA9kopU68oXuk090/8AgWBq54F4h+DWv2PwMtfDGjRx6rq76uNQu/JlWNMmNlO0yFcgDYOxPpW1\ne+GfHfjvx34Yfxdo1npOm+HJjcTXcF4swv5VK/NGowyKxQEKwGFLZJOBXsdFdH1+q1qlfXXrrv5C\n5UeKeP8Awj4i13WtSXUPhtoviRJfmsdVsb4WE8fBVRNufdKyjbxkLx74FfU/g3r5+EvhzT0ktdS1\nvQLlrtbS4bdBMjNuMGW4I4Uc4B5HAOa9zoojj6sVFRtp69rd9PlYOVHm/gTQ9WHiaXUdR+H3hrwj\nZwxbYFto4pbx3IwSJYsKExnIxnt70z4I+Etb8IeHdYtfEVl9jmudUe4iXzUk3RlEAOUJA5B4PNel\n0VlPEzlGUbKzt36erDlPDtV+Ees3fx1XUIoN3hO6vYtUusyoFFwitwUzuJLE8gYw/J4rS+KXgvxF\nrPxE0LXbXQYvFei2MJV9GlvVt1WXJO87ztIJ2ZGGyEwRivX6K0WOq80ZO2it/XW/mg5UeLeAPhz4\nk0sfEKz1bS7LS4/EVvts2tJVNvGWWUbFUfMFXzQPuj7px2rJj8FeOrn9n658DXPhb7Pe2csbWso1\nGBxdhrh5H43YTaCOp5z+Fe/0VX1+o5czS3T67rTuHKjyXxD4H8S2WseEPGPhmzhvtW0TT47K70ua\nZYzNHtKsEkPygje/JOOhGehm8OeD/Emr+O9W8c+LrCDSrqWwaxsdLiuFmaNcY3PIPlJ9Mf3jnGOf\nVKKz+tz5eWyva1+tr3t/WocqPEvDvw+8T2P7OOs+F7rTPL1m5lkaK2+0RHcCyEfMG2joeprRvvC/\niiD4beF9JPhHRfEcNpZLBqOl30ipcRvs27oZ92xTyQSORgYJzx65RTeNqOTbS3v13tbuHKjyD4S/\nD7WPDHibXtdm0dPDdjeQLBa6Gt/9rKsoU+Y0mSDyGxzn526ADPX/AAz8Sa74n8MXF34ps7S0v4L6\na1aOz5jxHgHB3sD824ZBxxXYVFa2lvY2kVrZQRW1vCoSOGFAiIo6AAcAe1RVxDrXc1q7fh/XUErH\nm/xe8Ka/reqeEdb8M6empzaDqP2mSzNwkLSruRuHfgD93jufmHB5rv7cT6r4fjGqWjWFxdWwFxbC\nVZDAzL8yhxw2CSM9DjNXqKzlWcoRg18Ow7a3PGP+GXvBf/QT17/wIh/+NV6j4W8OWnhLwzZ6Fpsk\n0trZqVje4YFyCxbkgAdT6VrUVVXFVqy5akroFFLY8nk8M+LNE+P+p+JtL0ddR0vW7WG1e6W7jjNk\nMRqzlG5cr5W7aByGHOcivPG+EnjY+F9e0e58HadfaveXJuD4kuNRR5ZwrhtsStlkZzuJY7chiG5x\nX03RXRDMKkLWS6d+m3X/AIBPKmeF+KtA1XSJPhvrNi2nJ4j0i0hs20S91CGGS6+VUZYmLbWOSw4z\n94Yz0MXhmXXdS/aevLjWbW20/UBopdrWKX7QtoCECq7DG4/MCcYHPB6GvXfFHgvw940s4bXxNpkd\n9HA5eIlmRkJ64ZSGAPGRnBwM9BUXhbwF4Z8FLOPDOkxWLXB/evveR29tzknHtnFaLGQ9k0171mtu\n7vvf9PmHLqeMeK/hx4o8ULcWl38NNDt9XlmCL4i0zUhb26qZAzSNb53OcFslgW54BwBXpGtat4l8\nGt4E0ewjtdStrqaLTtTvLnJlJCoN6DeCSQJWJw2NvOO/oFRPaW8t1FcyQRPcQhlilZAXjDY3BT1G\ncDOOuBWMsY6llOKsr6a9V5t/hp5D5bC3CGS1lRBlmQgD3xXnvwO8K6z4P+Hz6b4js/sd2b2SUR+a\nknylUAOUJHY969GormjVlGm6a2dvw/4cdtbnnHjX4IeG/HfiR9a1e91WG5eNYylrLGqYUccNGxz+\nNXPAPwi0H4daldX2iXeozy3UIhcXksbKBnPG1F54ru6K0eKrOn7Ny93sHKr3POPjb4P1nxh4RsYv\nDtul1eWOoR3X2ZpVjMqhWUgMxABG4Hk9Ae/BzNQ8PeK/EXxX8FeKbzw9/Z0NlBKt/Eb2KX7MxMgA\nyCN+QVPyjv7V61RVQxU4QUElpf8AHfqLl1PG/C3gPxJp3hn4m2l7p3lz6610dOTz4z5+9JQvIbC5\nLL97HWszWvhx4ru/2btE8LW+lb9atbvzJrX7RENq75TneW2nh16HvXu9FafXqvNzWW6f3K3cOVHj\nvjX4e6qvxEsfF+leGdP8VwSWQtL7SL2SNMMFIWRWkBQYwo6E8H+9kdl8OdG1HTNIuZ9Z8O6D4eur\nqXIs9Gt1TZGM7RK6kh35PTgfiQOworKeKnOmqb6ev+dvwDlSdzzX4l+E9a8QePvAOpaRZfaLTSNQ\nea9k81E8pC8JBwxBbhG6A9KPBvhLW9K+NHjDXb+y8rTdSVBaz+ajeZjGflBLDp3Ar0qihYqap+zs\nrWt+Nw5Ve54T4d+Eev3Xwe8T+GtZt006/vtTN3ZGSZHVsBCpJQtgHaR6jriqvibw98UfGHhrw1pd\n/wCE7ayTR7mFp2XUYXe4KrtEgG7CqBnIySSwxwK+gKK2WYVObmaTd79dNLdxcqPGvFPw+1XTvipe\n+KdL8H6b4zsNWt1WewvZYo2tpVCjeplBGCFHTJ5YYAAJ1v7O8aaJ8J7iHQvDGjWfibUG/wCPfQkj\ntYrQMoG9i74aRQCMqSM4xkDJ9PorN4ubUVJJ2t31ttfUfKeL+B/h14p+FvjSxGkeZrWhanbImrnz\no0+zXGeZVVipZR2xk7SwIJC17RRRWVevKvLmnv8AmNKwUUUVgMKKKKACiiigD//Z\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Image(filename='Data/parameters.JPG')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "A = np.array([[0.6, 0.2, 0.2], [0.5, 0.3, 0.2], [0.4, 0.1, 0.5]])\n",
    "pi = np.array([0.5, 0.2, 0.3])\n",
    "O = np.array([[0.7, 0.1, 0.2], [0.1, 0.6, 0.3], [0.3, 0.3, 0.4]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's consider the example observation sequence UP, UP, DOWN, for which we'll try to discover the hidden state sequence."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "states = UP, DOWN, UNCHANGED = 0, 1, 2\n",
    "observations = [UP, UP, DOWN]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we'll code the Viterbi algorithm. It keeps a store of two components, the best scores to reach a state at a give time, and the last step of the path to get there. Scores alpha are initialised to -inf to denote that we haven't set them yet. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "alpha = np.zeros((len(observations), len(states))) # time steps x states\n",
    "alpha[:,:] = float('-inf')\n",
    "backpointers = np.zeros((len(observations), len(states)), 'int')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The base case for the recursion sets the starting state probs based on pi and generating the observation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.35,  0.02,  0.09],\n",
       "       [ -inf,  -inf,  -inf],\n",
       "       [ -inf,  -inf,  -inf]])"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# base case, time step 0\n",
    "alpha[0, :] = pi * O[:,UP]\n",
    "alpha"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now for the recursive step, where we maximise over incoming transitions reusing the best incoming score, computed above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.35 ,  0.02 ,  0.09 ],\n",
       "       [ 0.147,  0.007,  0.021],\n",
       "       [  -inf,   -inf,   -inf]])"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# time step 1\n",
    "for t1 in states:\n",
    "    for t0 in states:\n",
    "        score = alpha[0, t0] * A[t0, t1] * O[t1,UP]\n",
    "        if score > alpha[1, t1]:\n",
    "            alpha[1, t1] = score\n",
    "            backpointers[1, t1] = t0\n",
    "alpha"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Repeat with the next observation. (We'd do this as a loop in practice.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.35   ,  0.02   ,  0.09   ],\n",
       "       [ 0.147  ,  0.007  ,  0.021  ],\n",
       "       [ 0.00882,  0.01764,  0.00882]])"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# time step 2\n",
    "for t2 in states:\n",
    "    for t1 in states:\n",
    "        score = alpha[1, t1] * A[t1, t2] * O[t2,DOWN]\n",
    "        if score > alpha[2, t2]:\n",
    "            alpha[2, t2] = score\n",
    "            backpointers[2, t2] = t1\n",
    "alpha"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now read of the best final state, and follow the backpointers to recover the full path."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.argmax(alpha[2,:])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "backpointers[2,1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "backpointers[1,0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Phew. The best state sequence is [0, 0, 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Formalising things"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can put this all into a function to handle arbitrary length inputs "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def viterbi(params, observations):\n",
    "    pi, A, O = params\n",
    "    M = len(observations)\n",
    "    S = pi.shape[0]\n",
    "    \n",
    "    alpha = np.zeros((M, S))\n",
    "    alpha[:,:] = float('-inf')\n",
    "    backpointers = np.zeros((M, S), 'int')\n",
    "    \n",
    "    # base case\n",
    "    alpha[0, :] = pi * O[:,observations[0]]\n",
    "    \n",
    "    # recursive case\n",
    "    for t in range(1, M):\n",
    "        for s2 in range(S):\n",
    "            for s1 in range(S):\n",
    "                score = alpha[t-1, s1] * A[s1, s2] * O[s2, observations[t]]\n",
    "                if score > alpha[t, s2]:\n",
    "                    alpha[t, s2] = score\n",
    "                    backpointers[t, s2] = s1\n",
    "    \n",
    "    # now follow backpointers to resolve the state sequence\n",
    "    ss = []\n",
    "    ss.append(np.argmax(alpha[M-1,:]))\n",
    "    for i in range(M-1, 0, -1):\n",
    "        ss.append(backpointers[i, ss[-1]])\n",
    "        \n",
    "    return list(reversed(ss)), np.max(alpha[M-1,:])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([0, 0, 1], 0.017639999999999999)"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "viterbi((pi, A, O), [UP, UP, DOWN])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "([0, 0, 2, 2, 2, 2, 0, 0], 6.2233919999999994e-06)"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "viterbi((pi, A, O), [UP, UP, DOWN, UNCHANGED, UNCHANGED, DOWN, UP, UP])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exhaustive method"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's verify that we've done the above algorithm correctly by implementing exhaustive search."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from itertools import product\n",
    "\n",
    "def exhaustive(params, observations):\n",
    "    pi, A, O = params\n",
    "    M = len(observations)\n",
    "    S = pi.shape[0]\n",
    "    \n",
    "    # track the running best sequence and its score\n",
    "    best = (None, float('-inf'))\n",
    "    # loop over the cartesian product of |states|^M\n",
    "    for ss in product(range(S), repeat=M):\n",
    "        # score the state sequence\n",
    "        score = pi[ss[0]] * O[ss[0],observations[0]]\n",
    "        for i in range(1, M):\n",
    "            score *= A[ss[i-1], ss[i]] * O[ss[i], observations[i]]\n",
    "        # update the running best\n",
    "        if score > best[1]:\n",
    "            best = (ss, score)\n",
    "            \n",
    "    return best"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((0, 0, 1), 0.017639999999999999)"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exhaustive((pi, A, O), [UP, UP, DOWN])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((0, 0, 2, 2, 2, 2, 0, 0), 6.2233919999999994e-06)"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exhaustive((pi, A, O), [UP, UP, DOWN, UNCHANGED, UNCHANGED, DOWN, UP, UP])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Yay, it got the same results as before. Note that the exhaustive method is practical on anything beyond toy data due to the nasty cartesian product. But it is worth doing to verify the Viterbi code above is getting the right results. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Supervised training, aka \"visible\" Markov model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's train the HMM parameters on the Penn Treebank, using the sample from NLTK. Note that this is a small fraction of the treebank, so we shouldn't expect great performance of our method trained only on this data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from nltk.corpus import treebank"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[(u'Pierre', u'NNP'), (u'Vinken', u'NNP'), (u',', u','), (u'61', u'CD'), (u'years', u'NNS'), (u'old', u'JJ'), (u',', u','), (u'will', u'MD'), (u'join', u'VB'), (u'the', u'DT'), (u'board', u'NN'), (u'as', u'IN'), (u'a', u'DT'), (u'nonexecutive', u'JJ'), (u'director', u'NN'), (u'Nov.', u'NNP'), (u'29', u'CD'), (u'.', u'.')], [(u'Mr.', u'NNP'), (u'Vinken', u'NNP'), (u'is', u'VBZ'), (u'chairman', u'NN'), (u'of', u'IN'), (u'Elsevier', u'NNP'), (u'N.V.', u'NNP'), (u',', u','), (u'the', u'DT'), (u'Dutch', u'NNP'), (u'publishing', u'VBG'), (u'group', u'NN'), (u'.', u'.')], ...]\n"
     ]
    }
   ],
   "source": [
    "corpus = treebank.tagged_sents()\n",
    "print corpus"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "news_text = nltk.corpus.brown.words(categories='news')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "brownSent = \"\"\n",
    "for word in news_text[0:20]:\n",
    "    brownSent = brownSent + str(word)\n",
    "    brownSent = brownSent + \" \""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have to first map words and tags to numbers for compatibility with the above methods."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0:00:02.033333\n"
     ]
    }
   ],
   "source": [
    "from datetime import datetime\n",
    "start1 = datetime.now()\n",
    "\n",
    "word_numbers = {}\n",
    "tag_numbers = {}\n",
    "\n",
    "num_corpus = []\n",
    "for sent in corpus:\n",
    "    num_sent = []\n",
    "    for word, tag in sent:\n",
    "        wi = word_numbers.setdefault(word.lower(), len(word_numbers))\n",
    "        ti = tag_numbers.setdefault(tag, len(tag_numbers))\n",
    "        num_sent.append((wi, ti))\n",
    "    num_corpus.append(num_sent)\n",
    "    \n",
    "word_names = [None] * len(word_numbers)\n",
    "for word, index in word_numbers.items():\n",
    "    word_names[index] = word\n",
    "tag_names = [None] * len(tag_numbers)\n",
    "for tag, index in tag_numbers.items():\n",
    "    tag_names[index] = tag\n",
    "#Now let's hold out the last few sentences for testing, so that they are unseen \n",
    "#during training and give a more reasonable estimate of accuracy on fresh text.\n",
    "\n",
    "training = num_corpus[:-10]\n",
    "testing = num_corpus[-10:]\n",
    "\n",
    "#Next we compute relative frequency estimates based on the observed tag\n",
    "#and word counts in the training set. Note that smoothing is important, here we add a small constant to all counts. \n",
    "\n",
    "S = len(tag_numbers)\n",
    "V = len(word_numbers)\n",
    "\n",
    "# initalise\n",
    "eps = 0.1\n",
    "pi = eps * np.ones(S)\n",
    "A = eps * np.ones((S, S))\n",
    "O = eps * np.ones((S, V))\n",
    "\n",
    "# count\n",
    "for sent in training:\n",
    "    last_tag = None\n",
    "    for word, tag in sent:\n",
    "        O[tag, word] += 1\n",
    "        if last_tag != None:\n",
    "            pi[tag] += 1\n",
    "        else:\n",
    "            A[last_tag, tag] += 1\n",
    "        last_tag = tag\n",
    "        \n",
    "# normalise\n",
    "pi /= np.sum(pi)\n",
    "for s in range(S):\n",
    "    O[s,:] /= np.sum(O[s,:])\n",
    "    A[s,:] /= np.sum(A[s,:])\n",
    "#Now we're ready to use our Viterbi method defined above\n",
    "\n",
    "predicted, score = viterbi((pi, A, O), map(lambda (w,t): w, testing[0]))\n",
    "print datetime.now()-start1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "               TOKEN\t TRUE\t PRED\n",
      "                   a\t   DT\t   DT\n",
      "               white\t  NNP\t  NNP\n",
      "               house\t  NNP\t  NNP\n",
      "           spokesman\t   NN\t   NN\n",
      "                said\t  VBD\t  VBD\n",
      "                last\t   JJ\t   JJ\n",
      "                week\t   NN\t   NN\n",
      "                that\t   IN\t  WDT\n",
      "                 the\t   DT\t   DT\n",
      "           president\t   NN\t   NN\n",
      "                  is\t  VBZ\t  VBZ\n",
      "         considering\t  VBG\t  VBG\n",
      "                 *-1\t-NONE-\t-NONE-\n",
      "           declaring\t  VBG\t  VBG\n",
      "                that\t   IN\t  WDT\n",
      "                 the\t   DT\t   DT\n",
      "        constitution\t  NNP\t  NNP\n",
      "          implicitly\t   RB\t  SYM\n",
      "               gives\t  VBZ\t  VBZ\n",
      "                 him\t  PRP\t  PRP\n",
      "                 the\t   DT\t   DT\n",
      "           authority\t   NN\t   NN\n",
      "                 for\t   IN\t   IN\n",
      "                   a\t   DT\t   DT\n",
      "           line-item\t   JJ\t   JJ\n",
      "                veto\t   NN\t   VB\n",
      "                 *-2\t-NONE-\t-NONE-\n",
      "                  to\t   TO\t   TO\n",
      "             provoke\t   VB\t   VB\n",
      "                   a\t   DT\t   DT\n",
      "                test\t   NN\t   NN\n",
      "                case\t   NN\t   NN\n",
      "                   .\t    .\t    .\n"
     ]
    }
   ],
   "source": [
    "print '%20s\\t%5s\\t%5s' % ('TOKEN', 'TRUE', 'PRED')\n",
    "for (wi, ti), pi in zip(testing[0], predicted):\n",
    "    print '%20s\\t%5s\\t%5s' % (word_names[wi], tag_names[ti], tag_names[pi])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hey, not bad, only three errors. Can you explain why these might have occurred?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Marginalisation in Hidden Markov Models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A related problem is marginalisation, when we wish to find the probability of an observation sequence *under any hidden state sequence*. This allows hidden Markov models to function as language models, but also is key to unsupervised training and the central algorithm for training."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As with the Viterbi algorithm, we'll need to start with the mathematical definition and attempt to factorise it (to follow a recursion, thus allowing for dynamic programming). The quantity we wish to compute is $$p(\\vec{w}) = \\sum_{\\vec{t}} p(\\vec{t}, \\vec{w})$$\n",
    "where $w$ are the observations (words) and $t$ are the states (tags)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start by expanding the summation \n",
    "$$\n",
    "p(\\vec{w})  = \\sum_{t_1} \\sum_{t_2} \\cdots \\sum_{t_{N-1}} \\sum_{t_N} p(\\vec{t}, \\vec{w})\n",
    "$$\n",
    "and expand the HMM probability \n",
    "$$\n",
    "p(\\vec{w})  = \\sum_{t_1} \\sum_{t_2} \\cdots \\sum_{t_{N-1}} \\sum_{t_N} p(t_1) p(w_1 | t_1) p(t_2 | t_1) p(w_2| t_2) \\cdots p(t_{N-1} | t_{N-2}) p(w_{N-1}| t_{N-1}) p(t_{N} | t_{N-1}) p(w_{N}| t_{N})\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's compare the full marginal probability $p(\\vec{w})$ and the probability up to position $N-1$, finishing with tag $t_{N-1}$\n",
    "$$p(w_1, w_2, \\ldots, w_{N-1}, t_{N-1}) = \\sum_{t_1} \\sum_{t_2} \\cdots \\sum_{t_{N-1}} p(t_1) p(w_1 | t_1) p(t_2 | t_1) p(w_2| t_2) \\cdots p(t_{N-1} | t_{N-2}) p(w_{N-1}| t_{N-1})\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "They look rather similar, and in fact we can express $p(\\vec{w})$ more simply as\n",
    "$$\n",
    "p(\\vec{w})  = \\sum_{t_N} p(w_1, w_2, \\ldots, w_{N-1}, t_{N-1}) p(t_{N} | t_{N-1}) p(w_{N}| t_{N})\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can continue further by defining $p(w_1, w_2, \\ldots, w_{N-1}, t_{N-1})$ in terms of $p(w_1, w_2, \\ldots, w_{N-2}, t_{N-2})$ and so forth. (This is the same process used in the Viterbi algorithm, albeit swapping a max for a sum.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Formally we store a matrix of partial marginals, $\\alpha$ defined as follows\n",
    "$$\\alpha[i, t_i] = p(w_1, w_2, \\ldots, w_i, t_i)$$\n",
    "computed using the recursion\n",
    "$$  \n",
    "\\alpha[i, t_i] = \\sum_{t_{i-1}} \\alpha[i-1, t_i] p(t_i | t_{i-1}) p(w_i| t_i) \n",
    "$$\n",
    "and the base case for $i=1$,\n",
    "$$\n",
    "\\alpha[1, t_1] = p(t_1) p(w_1 | t_1)\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we have computed the formulation, we can put this into an iterative algorithm: compute the vector of alpha[1] values, then alpha[2] etc until we reach the end of our input"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def forward(params, observations):\n",
    "    pi, A, O = params\n",
    "    N = len(observations)\n",
    "    S = pi.shape[0]\n",
    "    \n",
    "    alpha = np.zeros((N, S))\n",
    "    \n",
    "    # base case\n",
    "    alpha[0, :] = pi * O[:,observations[0]]\n",
    "    \n",
    "    # recursive case\n",
    "    for i in range(1, N):\n",
    "        for s2 in range(S):\n",
    "            for s1 in range(S):\n",
    "                alpha[i, s2] += alpha[i-1, s1] * A[s1, s2] * O[s2, observations[i]]\n",
    "    \n",
    "    return (alpha, np.sum(alpha[N-1,:]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[ 0.35    ,  0.02    ,  0.09    ],\n",
       "        [ 0.1792  ,  0.0085  ,  0.0357  ],\n",
       "        [ 0.012605,  0.025176,  0.016617]]), 0.054398000000000002)"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A = np.array([[0.6, 0.2, 0.2], [0.5, 0.3, 0.2], [0.4, 0.1, 0.5]])\n",
    "pi = np.array([0.5, 0.2, 0.3])\n",
    "O = np.array([[0.7, 0.1, 0.2], [0.1, 0.6, 0.3], [0.3, 0.3, 0.4]])\n",
    "\n",
    "forward((pi, A, O), [UP, UP, DOWN])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[  3.50000000e-01,   2.00000000e-02,   9.00000000e-02],\n",
       "        [  1.79200000e-01,   8.50000000e-03,   3.57000000e-02],\n",
       "        [  1.26050000e-02,   2.51760000e-02,   1.66170000e-02],\n",
       "        [  5.35956000e-03,   3.52065000e-03,   6.34588000e-03],\n",
       "        [  1.50288260e-03,   8.28808500e-04,   1.97959280e-03],\n",
       "        [  2.10797093e-04,   4.48307010e-04,   4.36840386e-04],\n",
       "        [  3.67757541e-04,   2.20335560e-05,   1.05072304e-04],\n",
       "        [  1.91590157e-04,   9.06688053e-06,   3.91483114e-05]]),\n",
       " 0.00023980534876399999)"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "forward((pi, A, O), [UP, UP, DOWN, UNCHANGED, UNCHANGED, DOWN, UP, UP])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's confirm we did this correctly by implementing an exhaustive equivalent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def exhaustive_forward(params, observations):\n",
    "    pi, A, O = params\n",
    "    N = len(observations)\n",
    "    S = pi.shape[0]\n",
    "\n",
    "    total = 0.0\n",
    "    # loop over the cartesian product of |states|^N\n",
    "    for ss in product(range(S), repeat=N):\n",
    "        # score the state sequence\n",
    "        score = pi[ss[0]] * O[ss[0],observations[0]]\n",
    "        for i in range(1, N):\n",
    "            score *= A[ss[i-1], ss[i]] * O[ss[i], observations[i]]\n",
    "        total += score\n",
    "            \n",
    "    return total"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.054397999999999988"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exhaustive_forward((pi, A, O), [UP, UP, DOWN])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.00023980534876400081"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "exhaustive_forward((pi, A, O), [UP, UP, DOWN, UNCHANGED, UNCHANGED, DOWN, UP, UP])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Backward algorithm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The same process but working from left to right rather than right to left give us the backward algorithm."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def backward(params, observations):\n",
    "    pi, A, O = params\n",
    "    N = len(observations)\n",
    "    S = pi.shape[0]\n",
    "    \n",
    "    beta = np.zeros((N, S))\n",
    "    \n",
    "    # base case\n",
    "    beta[N-1, :] = 1\n",
    "    \n",
    "    # recursive case\n",
    "    for i in range(N-2, -1, -1):\n",
    "        for s1 in range(S):\n",
    "            for s2 in range(S):\n",
    "                beta[i, s1] += beta[i+1, s2] * A[s1, s2] * O[s2, observations[i+1]]\n",
    "    \n",
    "    return (beta, np.sum(pi * O[:, observations[0]] * beta[0,:]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's confirm the it gets the same marginal probability as the forward algorithm "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[ 0.1216,  0.1077,  0.1076],\n",
       "        [ 0.24  ,  0.29  ,  0.25  ],\n",
       "        [ 1.    ,  1.    ,  1.    ]]), 0.054397999999999995)"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "backward((pi, A, O), [UP, UP, DOWN])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Unsupervised training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unsupervised training of a HMM involves running forward and backward to estimate the *expected* probability of taking various state sequences, then updates the model to match these *expectations*. This repeats many times until things stabilise (covergence). Note that it's non-convex, so the starting point often affects the converged solution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def baum_welch(training, pi, A, O, iterations):\n",
    "    pi, A, O = np.copy(pi), np.copy(A), np.copy(O) # take copies, as we modify them\n",
    "    S = pi.shape[0]\n",
    "    \n",
    "    # do several steps of EM hill climbing\n",
    "    for it in range(iterations):\n",
    "        pi1 = np.zeros_like(pi)\n",
    "        A1 = np.zeros_like(A)\n",
    "        O1 = np.zeros_like(O)\n",
    "        \n",
    "        for observations in training:\n",
    "            # compute forward-backward matrices\n",
    "            alpha, za = forward((pi, A, O), observations)\n",
    "            beta, zb = backward((pi, A, O), observations)\n",
    "            assert abs(za - zb) < 1e-6, \"it's badness 10000 if the marginals don't agree\"\n",
    "            \n",
    "            # M-step here, calculating the frequency of starting state, transitions and (state, obs) pairs\n",
    "            pi1 += alpha[0,:] * beta[0,:] / za\n",
    "            for i in range(0, len(observations)):\n",
    "                O1[:, observations[i]] += alpha[i,:] * beta[i,:] / za\n",
    "            for i in range(1, len(observations)):\n",
    "                for s1 in range(S):\n",
    "                    for s2 in range(S):\n",
    "                        A1[s1, s2] += alpha[i-1,s1] * A[s1, s2] * O[s2, observations[i]] * beta[i,s2] / za\n",
    "                                                                    \n",
    "        # normalise pi1, A1, O1\n",
    "        pi = pi1 / np.sum(pi1)\n",
    "        for s in range(S):\n",
    "            A[s, :] = A1[s, :] / np.sum(A1[s, :])\n",
    "            O[s, :] = O1[s, :] / np.sum(O1[s, :])\n",
    "    \n",
    "    return pi, A, O"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's test it out by training on our example from above"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "pi2, A2, O2 = baum_welch([[UP, UP, DOWN]], pi, A, O, 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[  9.99894603e-01,   8.36665388e-12,   8.22616418e-05],\n",
       "        [  2.22272332e-01,   3.12183957e-02,   4.43718595e-01],\n",
       "        [  4.20221660e-07,   4.52046779e-01,   2.00391873e-02]]),\n",
       " 0.47208638604110348)"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "forward((pi2, A2, O2), [UP, UP, DOWN])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looks like it memorised the sequence, and has assigned it a very high probability. The downside is that it won't be very accepting of other sequences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[  9.99894603e-01,   8.36665388e-12,   8.22616418e-05],\n",
       "        [  2.22272332e-01,   3.12183957e-02,   4.43718595e-01],\n",
       "        [  4.20221660e-07,   4.52046779e-01,   2.00391873e-02],\n",
       "        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00],\n",
       "        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00],\n",
       "        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00],\n",
       "        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00],\n",
       "        [  0.00000000e+00,   0.00000000e+00,   0.00000000e+00]]), 0.0)"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "forward((pi2, A2, O2), [UP, UP, DOWN, UNCHANGED, UNCHANGED, DOWN, UP, UP])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This looks strangely reminiscent of many other learning problems.... Can you think how we might deal with this?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Incidentally training on both sequences leads to perhaps a better model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "pi3, A3, O3 = baum_welch([[UP, UP, DOWN], [UP, UP, DOWN, UNCHANGED, UNCHANGED, DOWN, UP, UP]], pi, A, O, 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[  9.97616199e-01,   6.82049129e-15,   6.29715798e-14],\n",
       "        [  5.90761154e-01,   7.47740892e-03,   8.78089899e-06],\n",
       "        [  8.43266173e-04,   2.34964056e-01,   6.52309204e-04]]),\n",
       " 0.23645963152993088)"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "forward((pi3, A3, O3), [UP, UP, DOWN])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[  9.97616199e-01,   6.82049129e-15,   6.29715798e-14],\n",
       "        [  5.90761154e-01,   7.47740892e-03,   8.78089899e-06],\n",
       "        [  8.43266173e-04,   2.34964056e-01,   6.52309204e-04],\n",
       "        [  5.82459529e-09,   2.71931576e-08,   1.19461094e-01],\n",
       "        [  8.75785986e-10,   1.60469759e-06,   5.79399172e-02],\n",
       "        [  1.69690262e-05,   1.79695695e-02,   4.16867527e-03],\n",
       "        [  7.92352114e-03,   2.46221157e-05,   9.62381867e-05],\n",
       "        [  4.71403139e-03,   5.99516484e-05,   5.81125813e-07]]),\n",
       " 0.004774564161046658)"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "forward((pi3, A3, O3), [UP, UP, DOWN, UNCHANGED, UNCHANGED, DOWN, UP, UP])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Keeping (making?) in real"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a toy implementation of a HMM for pedagogical purposes. In reality we use several tricks to make things faster (e.g., matrix-vector operations) and to avoid floating point issues of underflow. These tricks complicate the code a fair bit. See the [Rabiner tutorial](http://www.ece.ucsb.edu/Faculty/Rabiner/ece259/Reprints/tutorial%20on%20hmm%20and%20applications.pdf) for details, especifically the section about scaling factors. Another trick is to [work in log-space](http://machineintelligence.tumblr.com/post/4998477107/the-log-sum-exp-trick), which is easy for Viterbi but a bit more painful (and slower) for forward-backward."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercises?\n",
    "\n",
    " - NLTK, currently has HMM implemeneted. Can you train the HMM with brown corpus with different training sizes:\n",
    "    - 5000 sentences.\n",
    "    - 10000 sentences\n",
    "    - 15000 sentences\n",
    "    - 20000 sentences\n",
    "    \n",
    " - Plot the difference in accuracy for a set of held out dataset, for training set of varying sizes.\n",
    " - Plot the difference in time taken for training with varying sizes of training set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Referrences"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Trevor Cohn course page-: http://people.eng.unimelb.edu.au/tcohn/comp90042.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Developer - Pranav Shukla<br>\n",
    "Email id - pranavdynamic@gmail.com"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [Root]",
   "language": "python",
   "name": "Python [Root]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}