geo871_similarity_wiki.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e3a6e9b9",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "This notebook illustrates how we can compare documents using cosine similarity. It does so in two ways. \n",
    "\n",
    "The first example builds simple binary vectors for every document, with 1 if a word is present and 0 otherwise. We then calcuate cosine similarty using these vectors. We don't do this for all parts of speech, at the moment the code looks for nouns, verbs and adjectives.\n",
    "\n",
    "The second example calculates cosine similarity using Spacy's inbuilt function, which uses Word2Vec to compare similarity. This means that words with similar **context** (trained empirically) are more similar.\n",
    "\n",
    "**The program reads in from Wikipedia places  to build a dataset. We can then extract a sample for comparisons**\n",
    "\n",
    "**The first block of our code initialises libraries and reads in data. We only need to do this once.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "d5a2314b",
   "metadata": {},
   "outputs": [],
   "source": [
    "import OSGridConverter #To convert from =SGB36 to WGS84\n",
    "import pandas as pd #To use pandas for elegant data handling\n",
    "import spacy #Our NLP tools\n",
    "import matplotlib.pyplot as plt #To plot results\n",
    "from matplotlib.figure import Figure #To plot results\n",
    "from matplotlib.ticker import MaxNLocator #To plot results\n",
    "import numpy as np #To do some numerical things\n",
    "import scipy\n",
    "import sklearn\n",
    "from sklearn.metrics.pairwise import cosine_similarity\n",
    "from sklearn.neighbors import DistanceMetric\n",
    "from math import radians\n",
    "import math # To calculate square root\n",
    "\n",
    "#import os #We import os to stop a weird kernel crash...\n",
    "#os.environ['KMP_DUPLICATE_LIB_OK']='TRUE'\n",
    "\n",
    "#Load a language model to do NLP\n",
    "nlp = spacy.load(\"en_core_web_md\") # We can't use the sm model here because it doesn't calculate token2vec"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "c78ff881",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "296695\n",
      "10000\n"
     ]
    }
   ],
   "source": [
    "#Read in the Wikipedia data\n",
    "wiki = pd.read_csv('./data/place-text.csv',sep='\\t')\n",
    "print(len(wiki))\n",
    "wiki.head()\n",
    "wiki = wiki.groupby(['id','lat','lon'])['text'].apply(lambda x: ','.join(x)).reset_index() # This groups individual paragraphs as articles\n",
    "\n",
    "print(len(wiki))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f300b16f",
   "metadata": {},
   "source": [
    "Here we **draw a random sample of documents** from our corpus and analyse those data. We can rerun this block to build a new sample. The size of this sample can also be changed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "e796bdd4",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Now I am going to load a random sample of m documents and run the NER pipeline\n",
    "m = 25\n",
    "sample = wiki.sample(n = m)\n",
    "docs = list(nlp.pipe(sample.text, batch_size=100))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1115495",
   "metadata": {},
   "source": [
    "This code block only makes sense if documents have locations. It calculates all the pairwise distances, to allow us to test the hypothesis that document similarity is related to distance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "42872640",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[    0.          1085.34630695  1018.41641251 10436.28846506\n",
      "   8433.28295112   689.24497848   826.57487062 12654.41017795\n",
      "   1826.52685972 10656.3753179    448.18390967  8131.48391067\n",
      "    506.6827664    563.76494135   128.88158786   171.10323512\n",
      "   6621.54612835   762.59854643   250.53134746 10678.18450858\n",
      "    382.70056811  6930.28567753  9582.00324592 15944.36038006\n",
      "    611.85621981]\n",
      " [ 1085.34630695     0.           643.22608683 11402.11908008\n",
      "   9380.18037017  1711.33765463   729.81265826 13263.4428575\n",
      "   1660.86323087 10703.38230113  1393.79867121  8238.6079615\n",
      "   1462.72974528  1605.95772064   980.77112076  1191.88857921\n",
      "   7661.49029431   703.07604084   876.71506464 10713.48687015\n",
      "   1446.31811602  7908.21742262  9826.17033948 15938.51994523\n",
      "   1607.69093934]\n",
      " [ 1018.41641251   643.22608683     0.         11442.76575262\n",
      "   8873.83007232  1434.44620929   220.33051238 13616.72926405\n",
      "   1048.46756368 11284.50324362  1111.44363055  7621.64548826\n",
      "   1170.70349462  1367.48592651   980.28737506  1031.14836232\n",
      "   7592.87779082   265.78229127   769.57309602 11297.61079561\n",
      "   1262.99744595  7937.3333025  10343.58621544 16551.60901433\n",
      "   1319.96343054]\n",
      " [10436.28846506 11402.11908008 11442.76575262     0.\n",
      "   6555.89596655 10072.70404982 11232.41033011  3653.53741644\n",
      "  11875.88351589  6665.52166844 10368.03273945 12544.96196528\n",
      "  10320.47342621 10108.3559456  10499.43949412 10411.68975143\n",
      "   3946.64240094 11177.85976039 10686.49556502  6731.96786507\n",
      "  10186.66602332  3506.06048414  5664.32732836  7531.41219762\n",
      "  10183.86752708]\n",
      " [ 8433.28295112  9380.18037017  8873.83007232  6555.89596655\n",
      "      0.          7744.34091238  8698.71281545 10016.5121492\n",
      "   8377.16985799 13048.85045772  8027.03722999  5994.32610724\n",
      "   7958.77434284  7869.90005571  8561.85089018  8278.53209257\n",
      "   5130.56472073  8703.41453577  8561.23021241 13118.88897647\n",
      "   8053.70044455  6000.27778607 11728.54440982 12822.23502109\n",
      "   7829.43453057]\n",
      " [  689.24497848  1711.33765463  1434.44620929 10072.70404982\n",
      "   7744.34091238     0.          1214.11668281 12582.32916685\n",
      "   1885.45412996 10980.9499013    327.76741136  7738.17107839\n",
      "    264.26793444   131.94065933   818.03627009   536.66384643\n",
      "   6180.7587933   1176.78532582   839.05671443 11009.06331622\n",
      "    316.78339282  6581.13192065  9804.49064179 16193.09439998\n",
      "    114.64449915]\n",
      " [  826.57487062   729.81265826   220.33051238 11232.41033011\n",
      "   8698.71281545  1214.11668281     0.         13463.10677759\n",
      "   1101.18609195 11240.98804735   891.3508045   7632.0617194\n",
      "    950.38702666  1147.97423176   805.37293227   822.50541083\n",
      "   7375.55680117    70.19996694   584.5978236  11256.47784311\n",
      "   1047.80183291  7727.88545285 10260.57528099 16527.8314053\n",
      "   1099.63978397]\n",
      " [12654.41017795 13263.4428575  13616.72926405  3653.53741644\n",
      "  10016.5121492  12582.32916685 13463.10677759     0.\n",
      "  14446.87208666  4061.52929977 12789.61390906 15765.16546746\n",
      "  12768.23553231 12561.35738963 12659.05816855 12708.74222123\n",
      "   7184.14013471 13394.17220384 12882.64836975  4106.07584936\n",
      "  12559.79463916  6426.98780945  3978.48740917  4127.79345789\n",
      "  12671.58408458]\n",
      " [ 1826.52685972  1660.86323087  1048.46756368 11875.88351589\n",
      "   8377.16985799  1885.45412996  1101.18609195 14446.87208666\n",
      "      0.         12331.38699401  1657.85226834  6578.95744011\n",
      "   1681.93626493  1892.32646934  1848.49415062  1749.69372197\n",
      "   7937.9718991   1169.34485337  1625.66760901 12345.07851401\n",
      "   1887.10642703  8419.68207348 11360.4807729  17597.29417173\n",
      "   1788.00202786]\n",
      " [10656.3753179  10703.38230113 11284.50324362  6665.52166844\n",
      "  13048.85045772 10980.9499013  11240.98804735  4061.52929977\n",
      "  12331.38699401     0.         11006.37194481 18559.53349094\n",
      "  11026.11222405 10889.27283191 10590.3758041  10790.9534811\n",
      "   8420.53021164 11175.94750666 10791.63969516    70.32028388\n",
      "  10782.93703005  7462.93927188  1525.52480538  5296.40356939\n",
      "  11012.81096312]\n",
      " [  448.18390967  1393.79867121  1111.44363055 10368.03273945\n",
      "   8027.03722999   327.76741136   891.3508045  12789.61390906\n",
      "   1657.85226834 11006.37194481     0.          7749.92245916\n",
      "     69.17614411   260.1216683    570.53791941   277.08803279\n",
      "   6490.5039518    851.26941111   534.72061632 11031.10184555\n",
      "    237.43097459  6869.72791914  9880.87294619 16268.29171018\n",
      "    216.65461939]\n",
      " [ 8131.48391067  8238.6079615   7621.64548826 12544.96196528\n",
      "   5994.32610724  7738.17107839  7632.0617194  15765.16546746\n",
      "   6578.95744011 18559.53349094  7749.92245916     0.\n",
      "   7722.51390528  7839.64584773  8211.00377772  7984.82443961\n",
      "  10266.92027084  7692.51728302  8024.58009275 18612.6580577\n",
      "   7962.4569776  11221.36016045 17055.36297682 15865.57647069\n",
      "   7718.57896231]\n",
      " [  506.6827664   1462.72974528  1170.70349462 10320.47342621\n",
      "   7958.77434284   264.26793444   950.38702666 12768.23553231\n",
      "   1681.93626493 11026.11222405    69.17614411  7722.51390528\n",
      "      0.           214.99638212   631.4201865    336.50734858\n",
      "   6437.31938835   912.5245922    603.47475877 11051.5333135\n",
      "    243.56788249  6824.23826119  9889.71301114 16279.41906887\n",
      "    150.79909261]\n",
      " [  563.76494135  1605.95772064  1367.48592651 10108.3559456\n",
      "   7869.90005571   131.94065933  1147.97423176 12561.35738963\n",
      "   1892.32646934 10889.27283191   260.1216683   7839.64584773\n",
      "    214.99638212     0.           692.05372466   419.02206759\n",
      "   6230.90807274  1105.03205071   729.7187363  10916.41776392\n",
      "    185.85137062  6610.84582623  9728.48546231 16119.37558808\n",
      "    124.49372168]\n",
      " [  128.88158786   980.77112076   980.28737506 10499.43949412\n",
      "   8561.85089018   818.03627009   805.37293227 12659.05816855\n",
      "   1848.49415062 10590.3758041    570.53791941  8211.00377772\n",
      "    631.4201865    692.05372466     0.           294.9860778\n",
      "   6702.92495959   737.56114787   224.92363714 10611.01862255\n",
      "    509.71991406  6994.38504895  9535.88559182 15883.54947505\n",
      "    740.30849019]\n",
      " [  171.10323512  1191.88857921  1031.14836232 10411.68975143\n",
      "   8278.53209257   536.66384643   822.50541083 12708.74222123\n",
      "   1749.69372197 10790.9534811    277.08803279  7984.82443961\n",
      "    336.50734858   419.02206759   294.9860778      0.\n",
      "   6570.66699413   766.2299781    316.15526014 10813.86776493\n",
      "    255.30991237  6906.46542766  9696.93540301 16071.37437578\n",
      "    449.86916462]\n",
      " [ 6621.54612835  7661.49029431  7592.87779082  3946.64240094\n",
      "   5130.56472073  6180.7587933   7375.55680117  7184.14013471\n",
      "   7937.9718991   8420.53021164  6490.5039518  10266.92027084\n",
      "   6437.31938835  6230.90807274  6702.92495959  6570.66699413\n",
      "      0.          7327.53910074  6866.33478566  8488.33309789\n",
      "   6330.62153754   957.67965418  6925.72809864 11297.32730952\n",
      "   6294.33599646]\n",
      " [  762.59854643   703.07604084   265.78229127 11177.85976039\n",
      "   8703.41453577  1176.78532582    70.19996694 13394.17220384\n",
      "   1169.34485337 11175.94750666   851.26941111  7692.51728302\n",
      "    912.5245922   1105.03205071   737.56114787   766.2299781\n",
      "   7327.53910074     0.           518.25648177 11191.73053211\n",
      "    997.31705006  7672.69540523 10191.79298959 16464.56250744\n",
      "   1062.79399297]\n",
      " [  250.53134746   876.71506464   769.57309602 10686.49556502\n",
      "   8561.23021241   839.05671443   584.5978236  12882.64836975\n",
      "   1625.66760901 10791.63969516   534.72061632  8024.58009275\n",
      "    603.47475877   729.7187363    224.92363714   316.15526014\n",
      "   6866.33478566   518.25648177     0.         10811.20309291\n",
      "    571.42539604  7180.45802921  9751.51427934 16087.35984753\n",
      "    740.56074445]\n",
      " [10678.18450858 10713.48687015 11297.61079561  6731.96786507\n",
      "  13118.88897647 11009.06331622 11256.47784311  4106.07584936\n",
      "  12345.07851401    70.32028388 11031.10184555 18612.6580577\n",
      "  11051.5333135  10916.41776392 10611.01862255 10813.86776493\n",
      "   8488.33309789 11191.73053211 10811.20309291     0.\n",
      "  10808.51486571  7530.78672232  1588.78663334  5278.00627858\n",
      "  11039.79077953]\n",
      " [  382.70056811  1446.31811602  1262.99744595 10186.66602332\n",
      "   8053.70044455   316.78339282  1047.80183291 12559.79463916\n",
      "   1887.10642703 10782.93703005   237.43097459  7962.4569776\n",
      "    243.56788249   185.85137062   509.71991406   255.30991237\n",
      "   6330.62153754   997.31705006   571.42539604 10808.51486571\n",
      "      0.          6683.68368479  9647.44649893 16036.55344415\n",
      "    267.50280495]\n",
      " [ 6930.28567753  7908.21742262  7937.3333025   3506.06048414\n",
      "   6000.27778607  6581.13192065  7727.88545285  6426.98780945\n",
      "   8419.68207348  7462.93927188  6869.72791914 11221.36016045\n",
      "   6824.23826119  6610.84582623  6994.38504895  6906.46542766\n",
      "    957.67965418  7672.69540523  7180.45802921  7530.78672232\n",
      "   6683.68368479     0.          5970.23985947 10554.44505892\n",
      "   6690.72222216]\n",
      " [ 9582.00324592  9826.17033948 10343.58621544  5664.32732836\n",
      "  11728.54440982  9804.49064179 10260.57528099  3978.48740917\n",
      "  11360.4807729   1525.52480538  9880.87294619 17055.36297682\n",
      "   9889.71301114  9728.48546231  9535.88559182  9696.93540301\n",
      "   6925.72809864 10191.79298959  9751.51427934  1588.78663334\n",
      "   9647.44649893  5970.23985947     0.          6391.04307807\n",
      "   9852.91588931]\n",
      " [15944.36038006 15938.51994523 16551.60901433  7531.41219762\n",
      "  12822.23502109 16193.09439998 16527.8314053   4127.79345789\n",
      "  17597.29417173  5296.40356939 16268.29171018 15865.57647069\n",
      "  16279.41906887 16119.37558808 15883.54947505 16071.37437578\n",
      "  11297.32730952 16464.56250744 16087.35984753  5278.00627858\n",
      "  16036.55344415 10554.44505892  6391.04307807     0.\n",
      "  16243.76534595]\n",
      " [  611.85621981  1607.69093934  1319.96343054 10183.86752708\n",
      "   7829.43453057   114.64449915  1099.63978397 12671.58408458\n",
      "   1788.00202786 11012.81096312   216.65461939  7718.57896231\n",
      "    150.79909261   124.49372168   740.30849019   449.86916462\n",
      "   6294.33599646  1062.79399297   740.56074445 11039.79077953\n",
      "    267.50280495  6690.72222216  9852.91588931 16243.76534595\n",
      "      0.        ]]\n"
     ]
    }
   ],
   "source": [
    "# We use this block to calculate the distances between all the documents\n",
    "# We are interested in finding out later if there is a relationship between distance and cosine similarity\n",
    "dist = DistanceMetric.get_metric('haversine')\n",
    "\n",
    "sample['lat'] = np.radians(sample['lat'])\n",
    "sample['lon'] = np.radians(sample['lon'])\n",
    "dists = dist.pairwise(sample[['lat','lon']].to_numpy())*6373\n",
    "\n",
    "print(dists)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f4b65fb",
   "metadata": {},
   "source": [
    "Build a dictionary of terms in all our documents which we then use for building vectors for similarity comparisons."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "fcd50e29",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['#', '%', '+', ',same', '-', '.', '13th', '15m', '1760', '1891.,effort', '18th', '1900', '1939–45', '1940', '1950', '1960', '1970', '1980', '1988.,voter', '1990', '2009.,<w', '20th', '21st', '28th', '295th', '312th', '31ff', '33rd', '3check_isbn', '41st', '68th', '7check_isbn', '95.5', 'BSk', 'People', 'a.', 'abidjan.svg\">location', 'ablution', 'abolitionism', 'abolitionist', 'abundance', 'accelerator', 'access', 'accolade', 'accommodation', 'accordance', 'acquisition', 'acre', 'acronym', 'act', 'activation', 'activist', 'activity', 'actor', 'actress', 'addition', 'address', 'adherent', 'administration', 'administrator', 'adolescent', 'adult', 'advantage', 'advent', 'adventure', 'advice', 'adviser', 'advisor', 'advocate', 'aegis', 'affiliate', 'affiliation', 'affordability', 'aftermath', 'afternoon', 'age', 'agency', 'agent', 'agreement', 'agriculture', 'aid', 'air', 'aircraft', 'airline', 'airport', 'album', 'alignment', 'allegiance', 'altoid', 'aluminum', 'alumnus', 'am', 'amateur', 'ambition', 'ambulance', 'amount', 'amusement', 'analysis', 'anarchist', 'ancestry', 'animal', 'anniversary', 'announcer', 'anthem', 'anthology', 'antique', 'apartment', 'apostle', 'application', 'aqueduct', 'arboretum', 'arch', 'architect', 'architecture', 'archive', 'are:,surakarta', 'area', 'arena', 'arm', 'army', 'arrangement', 'array', 'arrest', 'arrival', 'art', 'artefact', 'article', 'artifact', 'artillery', 'artist', 'asherie', 'assassin', 'assassination', 'assembly', 'assessment', 'assistance', 'associate', 'association', 'athlete', 'athletic', 'attack', 'attempt', 'attendance', 'attendee', 'attention', 'attorney', 'attraction', 'auction', 'audience', 'auditorium', 'aunt', 'auspex', 'author', 'authority', 'auto', 'automobile', 'autumn', 'average', 'aviator', 'award', 'axis', 'b', 'bachelor', 'back', 'backdrop', 'backer', 'background', 'baking', 'ball', 'ballfield', 'balloon', 'ballot', 'ballpark', 'ballplayer', 'banana', 'band', 'banditry', 'bank', 'bar', 'barbecue', 'barber', 'barn', 'barrier', 'bascule', 'base', 'baseball', 'basement', 'basin', 'basis', 'basketball', 'bat', 'bath', 'batik', 'batiks', 'baton', 'battle', 'beach', 'beachgoer', 'bean', 'bear\"', 'beat', 'beauty', 'bed', 'bedroom', 'beer', 'beginning', 'being', 'benefit', 'bergamot', 'beverage', 'bicycle', 'bid', 'bike', 'biking', 'biographer', 'birth', 'birthplace', 'black', 'blacksmith', 'blend', 'bleue', 'block', 'blow', 'blue', 'bluegrass', 'board', 'boat', 'bocce', 'body', 'bond', 'book', 'boom', 'border', 'borough', 'bot', 'bottling', 'bottom', 'boulevard', 'boundary', 'boutique', 'box', 'boxer', 'boxing', 'brain', 'branch', 'brand', 'break', 'breath', 'brick', 'bridge', 'bridge.jpg\">bridge', 'bridgehead', 'broadband', 'broadcasting', 'brochure', 'bronze', 'brook', 'brother', 'brunch', 'brutto', 'buckle', 'budget', 'builder', 'building', 'bulk', 'burial', 'burner', 'bus', 'business', 'buyer', 'bypass', 'c.', 'c.1942', 'cabin', 'cable', 'cafe', 'cafés', 'cake', 'camat', 'camera', 'camp', 'campaign', 'camping', 'campus', 'canal', 'cancer', 'candidate', 'candlelight', 'candy', 'cannonball', 'canopy', 'canvas', 'canyon', 'cap', 'capability', 'capacity', 'capita', 'capital', 'capitalism', 'capitalist', 'captain', 'capture', 'car', 'care', 'career', 'cargo', 'cascade', 'case', 'cast', 'castor', 'catamaran', 'cataract', 'category', 'cathedral', 'cause', 'cavern', 'celadon', 'celebration', 'celebrity', 'celsius', 'cement', 'cemetery', 'census', 'center', 'centerpiece', 'centre', 'century', 'ceo', 'ceramic', 'ceramics', 'ceremony', 'chain', 'chairman', 'challenge', 'champion', 'championship', 'chan', 'change', 'channel', 'characteristic', 'charge', 'charity', 'charm', 'charter', 'chattanooga', 'cheese', 'chemical', 'chief', 'child', 'childhood', 'chimney', 'chipboard', 'chloral', 'chlorine', 'choice', 'church', 'circle', 'circus', 'citation', 'citico', 'citizen', 'city', 'claim', 'class', 'classification', 'classroom', 'cleaning', 'clergyman', 'clerk', 'clientele', 'climate', 'climbing', 'clinic', 'clock', 'closure', 'cloth', 'clothe', 'clothing', 'cloverleaf', 'club', 'co', 'coach', 'coast', 'coat', 'coauthor', 'cocaine', 'cocoa', 'coconut', 'code', 'codivoire.jpg\">seat', 'coffee', 'collaboration', 'collaborator', 'collar', 'collection', 'college', 'colonial', 'colonist', 'colonized', 'colony', 'color', 'colour', 'comedy', 'command', 'commerce', 'commission', 'committee', 'commune', 'communication', 'community', 'commute', 'commuter', 'company', 'competition', 'competitor', 'completion', 'complex', 'composer', 'compound', 'concentration', 'concert', 'concession', 'condition', 'conductor', 'conference', 'conflict', 'congregation', 'connection', 'consensus', 'consent', 'consequence', 'consolidation', 'constraint', 'construction', 'container', 'content', 'context', 'contract', 'contradiction', 'contrast', 'contribution', 'contributor', 'control', 'controversy', 'convention', 'conversion', 'convert', 'conviction', 'cooper', 'cooperative', 'coord', 'cops', 'core', 'corn', 'corncob', 'corner', 'cornerback', 'cornerstone', 'corporation', 'corruption', 'cost', 'cotton', 'council', 'councilor', 'count', 'counter', 'counterterrorism', 'country', 'countryside', 'county', 'county).,in', 'countywide', 'countywide).,in', 'couple', 'court', 'courthouse', 'cow', 'cr', 'cradle', 'crash', 'creation', 'creator', 'creek', 'crew', 'crime', 'crisis', 'cropped).jpg\">part', 'crossroad', 'crowd', 'culture', 'culvert', 'currency', 'curve', 'custom', 'customer', 'cut', 'cycle', 'cycling', 'cyclist', 'daily', 'dairy', 'dam', 'damage', 'dance', 'database', 'date', 'datum', 'daughter', 'day', 'day,(tzom', 'daytime', 'de', 'deal', 'dealer', 'dean', 'death', 'debt', 'debut', 'decade', 'decision', 'decline', 'decoration', 'decree', 'defeat', 'defense', 'defenseman', 'deficit', 'defile', 'definition', 'degradation', 'degree', 'delegation', 'demand', 'demonstration', 'demonstrator', 'den', 'denomination', 'density', 'department', 'deployment', 'depot', 'depth', 'dermatology', 'descent', 'design', 'designation', 'designer', 'desire', 'destination', 'destruction', 'detail', 'detection', 'detector', 'development', 'diagnosis', 'diagnostic', 'dialect', 'diamond', 'difficulty', 'digit', 'diocese', 'diorama', 'diploma', 'diplomat', 'direction', 'director', 'directory', 'disability', 'disaster', 'disbandment', 'disc', 'discipline', 'discothèque', 'discussion', 'disease', 'disinfection', 'displacement', 'display', 'disrepair', 'disruption', 'dissolution', 'distance', 'distribution', 'distributor', 'district', 'diva', 'diver', 'diversion', 'diversity', 'divinity', 'division', 'djan', 'docking', 'doctor', 'document', 'documentary', 'documentation', 'dog', 'dollar', 'dolphin', 'domain', 'domestication', 'donation', 'door', 'downtown', 'downturn', 'dozen', 'drain', 'drama', 'draught', 'dressage', 'drinking', 'drive', 'driver', 'drug', 'drummer', 'dryness', 'duchy', 'dulcimer', 'duo', 'duty', 'dweller', 'dwelling', 'dynamo', 'dynasty', 'd’ivoire', 'earth', 'east', 'east:,<w', 'economist', 'economy', 'ed', 'edge', 'edition', 'editor', 'editorial', 'education', 'educator', 'effect', 'effort', 'eh’häusl', 'eighteen', 'election', 'electorate', 'electricity', 'electrification', 'element', 'elephant', 'elevation', 'embassy', 'emergency', 'emigrant', 'emigration', 'employee', 'employer', 'employment', 'enclave', 'end', 'energy', 'engagement', 'engine', 'engineer', 'enroll', 'enrollment', 'entertainment', 'entity', 'entrance', 'entrepreneur', 'entry', 'enumeration', 'environ', 'epb.,beginning', 'epidemic', 'episode', 'equipment', 'era', 'error', 'esplanade', 'essayist', 'establishment', 'estimate', 'ethnicity', 'evening', 'event', 'ever', 'exam', 'examination', 'example', 'excellence', 'exception', 'exchange', 'excitement', 'execution', 'executive', 'exhibit', 'exhibition', 'exile', 'existence', 'exit', 'expanse', 'expansion', 'expatriate', 'expedition', 'expenditure', 'experience', 'experiment', 'explorer', 'explosion', 'export', 'exportation', 'expressway', 'expulsion', 'extension', 'extent', 'eye', 'face', 'facility', 'fact', 'factory', 'fall', 'fame', 'family', 'fan', 'fanico', 'fantasy', 'fare', 'farm', 'farmer', 'farming', 'fashion', 'fatality', 'father', 'fault', 'favor', 'favorite', 'feature', 'fee', 'feeder', 'feedstock', 'feel', 'fellowship', 'female', 'fence', 'ferry', 'festival', 'fever', 'fiber', 'fiction', 'fiddle', 'field', 'fight', 'fighting', 'figure', 'file', 'film', 'filmmaker', 'final', 'financing', 'fire', 'firearm', 'firefighter', 'firewood', 'firework', 'firm', 'fishing', 'flag', 'flame', 'fledgling', 'flight', 'flood', 'flooding', 'floor', 'flower', 'folk', 'folklore', 'follower', 'following', 'food', 'foot', 'footage', 'football', 'footballer', 'foothill', 'foothold', 'force', 'foreground', 'foreigner', 'forest', 'form', 'formation', 'fort', 'fortification', 'fortune', 'founder', 'founding', 'fourth', 'framebuilder', 'franc', 'franchise', 'fraud', 'freelance', 'freight', 'frequency', 'freshwater', 'friday', 'friend', 'friendship', 'fruit', 'fuel', 'fun', 'function', 'fund', 'funding', 'funicular', 'funk', 'furnace', 'future', 'gala', 'gallery', 'game', 'gap', 'garbage', 'garden', 'garret', 'gas', 'gate', 'gateway', 'gather', 'gauge', 'gendarmerie', 'gender', 'general', 'generation', 'genre', 'gentleman', 'gentrification', 'gigabit', 'glamour', 'glioblastoma', 'glove', 'go', 'goal', 'goat', 'gold', 'golf', 'good', 'gospel', 'government', 'governor', 'grade', 'graduate', 'graffiti', 'grain', 'grandeur', 'grandson', 'grant', 'grassland', 'gravel', 'green', 'greenway', 'grid', 'grist', 'gristmill', 'grocery', 'ground', 'groundbreaking', 'group', 'growth', 'guaguas', 'guard', 'guest', 'guide', 'guitarist', 'gunman', 'gutter', 'half', 'hall', 'hammer', 'hammerhead', 'hand', 'handball', 'handcycling', 'handful', 'handle', 'handling', 'harbour', 'harnessing', 'harp', 'hat', 'haven', 'hazard', 'head', 'headline', 'headquarter', 'headquarters', 'health', 'healthcare', 'heart', 'heat', 'hectare', 'height', 'heir', 'help', 'heritage', 'heyday', 'hierarchy', 'high', 'highway', 'hiking', 'hill', 'hinterland', 'hip', 'hispanic', 'historian', 'history', 'hitter', 'hockey', 'holder', 'holiday', 'home', 'hometown', 'honey', 'honor', 'honour', 'hooch', 'hop', 'hope', 'horror', 'horse', 'hospital', 'host', 'hotel', 'hour', 'house', 'household', 'householder', 'housing', 'hub', 'humidity', 'hundred', 'hunting', 'husband', 'hymn', 'i', 'i-95', 'ice', 'icon', 'icon.svg\">class', 'idea', 'idealism', 'identity', 'idr', 'image', 'imagination', 'imitation', 'immigrant', 'immigration', 'immortality', 'impact', 'implementation', 'import', 'importance', 'impressiveness', 'improvement', 'in', 'inception', 'inch', 'income', 'incorporation', 'increase', 'incubator', 'incumbent', 'independence', 'index', 'indication', 'indicator', 'individual', 'industrialisation', 'industrialist', 'industry', 'inflation', 'influx', 'info', 'information', 'infrastructure', 'inhabitant', 'initial', 'initiative', 'injury', 'inmate', 'innovation', 'inroad', 'instability', 'installation', 'institution', 'instrument', 'insurance', 'insurgency', 'interchange', 'interconnection', 'interest', 'interior', 'internet', 'internment', 'intersect', 'intersection', 'interstate', 'interview', 'introduction', 'inventor', 'investigation', 'investment', 'investor', 'invitation', 'iron', 'issue', 'item', 'ivory', 'jail', 'jazz', 'jersey.jpg\">paterson', 'jet', 'jetty', 'jewel', 'jewellery', 'jewelry', 'jitney', 'job', 'jobs\"', 'joint', 'journalist', 'journey', 'judge', 'jumping', 'junction', 'junior', 'jurisdiction', 'justice', 'katedra', 'kayaking', 'kelurahan', 'keraton', 'kid', 'kiln', 'kilometre', 'kind', 'kindergarten', 'king', 'km', 'km²', 'knot', 'l', 'lab', 'labor', 'laborer', 'lack', 'lagoon', 'lake', 'lamb', 'lamp', 'land', 'landfill', 'landmark', 'landowner', 'landscape', 'lane', 'language', 'laser', 'launch', 'laundry', 'law', 'laxity', 'layoff', 'layout', 'leader', 'leadership', 'league', 'learning', 'leash', 'leave', 'lecture', 'ledge', 'legend', 'legislation', 'legislature', 'leisure', 'lesson', 'level', 'library', 'licence', 'life', 'lift', 'light', 'lighthouse', 'lighting', 'lightning', 'limelight', 'limestone', 'limit', 'line', 'linebacker', 'link', 'list', 'literacy', 'literate', 'literature', 'livability', 'living', 'load', 'local', 'locale', 'locality', 'location', 'lock', 'locomotive', 'log', 'look', 'loop', 'looting', 'loss', 'lot', 'low', 'luggage', 'lumber', 'lurah', 'luxury', 'lyric', 'm', \"m'bidjan\", 'machine', 'machinery', 'magazine', 'magnate', 'magnet', 'mahogany', 'main', 'maincheck.</w>,train', 'mainland', 'maintenance', 'majority', 'makeover', 'maker', 'makeup', 'male', 'mall', 'man', 'management', 'manager', 'mandate', 'manganese', 'mango', 'mansion', 'manufacture', 'manufacturer', 'manufacturing', 'manuscript', 'map', 'maquis', 'marathon', 'marble', 'marcory', 'margin', 'mark', 'market', 'marriage', 'martyr', 'maser', 'mask', 'massacre', 'master', 'match', 'material', 'mathematic', 'mathematician', 'maximum', 'mayor', 'mean', 'meat', 'mecca', 'medal', 'medallion', 'media', 'median', 'medicine', 'medium', 'meet', 'meeting', 'megabit', 'melting', 'member', 'memorabilia', 'memorial', 'memory', 'merchant', 'merger', 'merging', 'merit', 'metal', 'meter', 'method', 'metre', 'metro', 'mezzo', 'mid', 'mid-1950', 'mid-1970', 'middle', 'midfielder', 'migrant', 'migration', 'mile', 'militant', 'mill', 'milling', 'million', 'millworkers-1910.jpg\">child', 'mime', 'min', 'mining', 'minister', 'minority', 'mint', 'mission', 'misunderstanding', 'mix', 'mm', 'mob', 'model', 'modernity', 'moisture', 'moment', 'monarch', 'monastery', 'monsoon', 'monster', 'month', 'monument', 'mood', 'morning', 'mosque', 'mother', 'motherhood', 'motherland', 'motorcycle', 'motorist', 'mound', 'mountain', 'mouth', 'movement', 'movie', 'municipality', 'municipally', 'murder', 'museum', 'music', 'musical', 'musician', 'mystery', 'name', 'namesake', 'namespace=\"file', 'naming', 'nation', 'native', 'navy', 'ndombolo', 'need', 'neglect', 'negligence', 'negotiation', 'neighbor', 'neighborhood', 'neighboring', 'nerve', 'network', 'neutrino', 'newcomer', 'news', 'newspaper', 'nickname', 'night', 'nightlife', 'nighttime', 'no', 'node', 'nomination', 'non', 'none', 'north', 'northeast', 'northwest', 'notice', 'novel', 'ns', 'nucleus', 'number', 'nys&w', 'observation', 'observer', 'occasion', 'occupation', 'ocean', 'offender', 'offensive', 'offering', 'office', 'officer', 'official', 'oil', 'old', 'one', 'onepage&q=&f', 'online', 'onstage', 'opening', 'opera', 'operating', 'operation', 'operator', 'opportunity', 'optic', 'optometry', 'orange', 'orchestra', 'order', 'ordinance', 'ore', 'organ', 'organisation', 'organist', 'organization', 'organizer', 'origin', 'other', 'ouster', 'outbreak', 'outcry', 'outflow', 'outlet', 'outskirt', 'over', 'overhaul', 'owner', 'ownership', 'p.', 'p.r.', 'p69', 'packaging', 'pact', 'pad', 'paddler', 'page', 'page=11', 'page=56', 'page=59', 'page=62', 'pair', 'palace', 'palm', 'paper', 'par', 'parade', 'parcel', 'parent', 'park', 'parking', 'parliament', 'part', 'participant', 'participation', 'partnership', 'party', 'passenger', 'past', 'patch', 'patent', 'pathway', 'patient', 'patrol', 'pattern', 'pavement', 'pdci', 'peacekeeper', 'peacekeeping', 'peach', 'peak', 'pear', 'pedestrian', 'pediatric', 'peninsula', 'people', 'percent', 'percentage', 'performance', 'performer', 'period', 'permission', 'person', 'personality', 'petition', 'petroleum', 'pharmacopoeia', 'pharmacy', 'phase', 'phenomenon', 'philanthropist', 'phone', 'photo', 'photograph', 'physician', 'physicist', 'picture', 'pier', 'pile', 'pilgrimage', 'pineapple', 'pioneer', 'pipe', 'pipeline', 'pirate', 'pitcher', 'place', 'placement', 'plague', 'plain', 'plan', 'plan,*working', 'planner', 'planning', 'plant', 'plantation', 'planting', 'plaque', 'platform', 'play', 'player', 'playground', 'playing', 'playoff', 'playwright', 'plot', 'pluralism', 'plywood', 'poem', 'poet', 'poetry', 'point', 'pointing', 'pole', 'police', 'policy', 'politic', 'politician', 'pollutant', 'polluter', 'polo', 'polygon', 'pond', 'pony', 'pool', 'pop', 'popularity', 'population', 'porcelain', 'port', 'portal', 'portion', 'portrayal', 'position', 'possession', 'possibility', 'post', 'postmaster', 'pot', 'pottery', 'poverty', 'pow', 'power', 'pp', 'practice', 'pre', 'precipitation', 'prefecture', 'prek-12).,as', 'prelude', 'preparation', 'preschool', 'presence', 'present', 'preserve', 'presidency', 'president', 'press', 'price', 'principal', 'principle', 'printing', 'priority', 'prison', 'prisoner', 'privilege', 'prize', 'problem', 'processing', 'produce', 'producer', 'product', 'production', 'profession', 'professional', 'professor', 'profile', 'program', 'project', 'proliferation', 'promoter', 'pronunciation', 'propaganda', 'property', 'prophesy', 'proportion', 'prosperity', 'prostitute', 'prostitution', 'protagonist', 'protest', 'provider', 'province', 'proximity', 'psychology', 'public', 'publication', 'publicity', 'publishe', 'publisher', 'purpose', 'pursuit', 'purview', 'quality', 'quarry', 'quarter', 'quarterback', 'question', 'r', 'r&b', 'race', 'racer', 'racetrack', 'raceway', 'racing', 'radio', 'rafting', 'raider', 'rail', 'railbus', 'railbuse', 'railfan', 'railroad', 'railway', 'rain', 'rainfall', 'rainforest', 'raising', 'rally', 'ramp', 'range', 'rape', 'rapid', 'rapper', 'rate', 'ratio', 'rda', 're', 'readiness', 'reality', 'reapportionment', 'rear', 'reason', 'rebellion', 'rebirth', 'receiver', 'recipient', 'recital', 'reclaiming', 'reconstruction', 'record', 'recording', 'recreation', 'red', 'redevelopment', 'reference', 'referendum', 'refinery', 'refining', 'reform', 'reformer', 'refuge', 'refugee', 'regatta', 'regency', 'reggae', 'region', 'regulating', 'regulation', 'rehabilitation', 'relationship', 'relic', 'religion', 'relocation', 'reluctance', 'remainder', 'rematch', 'remnant', 'renewal', 'renovation', 'reorganization', 'repair', 'repaving', 'repeal', 'report', 'representation', 'representative', 'reprisal', 'republic', 'reputation', 'requirement', 'rescue', 'research', 'reservation', 'reserve', 'reservoir', 'residence', 'residency', 'resident', 'resistance', 'resort', 'resource', 'responsibility', 'rest', 'restaurant', 'resting', 'restoration', 'result', 'resurgence', 'resurrection', 'retailer', 'retaliation', 'return', 'revenue', 'revitalization', 'revolution', 'revolver', 'rice', 'ride', 'ridge', 'right', 'rink', 'riot', 'rip', 'rise', 'risk', 'rival', 'rivalry', 'river', 'riverbed', 'riverfront', 'road', 'roadblock', 'roadway', 'robotic', 'rock', 'rod', 'rodeo', 'role', 'roll', 'roof', 'room', 'root', 'round', 'route', 'routine', 'routing', 'rowing', 'rubber', 'rug', 'rugby', 'ruin', 'rule', 'ruler', 'rumba', 'run', 'running', 'rush', 's', 'safety', 'sailor', 'sale', 'salon', 'sample', 'sanctuary', 'sand', 'sanitary', 'sanitation', 'satellite', 'sawmill', 'scene', 'schedule', 'scheme', 'school', 'science', 'scope', 'score', 'scow', 'screenplay', 'sea', 'seaport', 'search', 'season', 'seat', 'secessionist', 'second', 'second-', 'section', 'sector', 'security', 'segregation', 'selectman', 'self', 'seller', 'senator', 'sentence', 'sequel', 'sequence', 'series', 'service', 'session', 'setting', 'settlement', 'settler', 'sewer', 'sex', 'shack', 'shape', 'share', 'sheep', 'shelf', 'shelter', 'ship', 'shipping', 'shoe', 'shoemaker', 'shooting', 'shop', 'shopper', 'shopping', 'shore', 'shot', 'show', 'showcase', 'shuttle', 'sibling', 'side', 'sidewalk', 'sight', 'sign', 'signaling', 'signer', 'silk', 'simulcast', 'sin', 'singer', 'siren', 'sister', 'site', 'situation', 'size', 'skating', 'skill', 'skirmish', 'skyline', 'skyscraper', 'slang', 'slaveowner', 'slope', 'slum', 'smartgrid', 'snack', 'snake', 'snow', 'snowfall', 'soap', 'soccer', 'society', 'softball', 'soldier', 'solicitor', 'son', 'song', 'songwriter', 'source', 'south', 'southeast', 'southwest', 'soy', 'space', 'span', 'speaker', 'specialist', 'speciman', 'spectacle', 'spectator', 'spectrum', 'speed', 'spelling', 'spike', 'splash', 'sport', 'sporting', 'sportspeople', 'spot', 'spouse', 'spring', 'stadium', 'staff', 'stage', 'stake', 'stampede', 'stand', 'standard', 'staple', 'star', 'start', 'startup', 'state', 'statesman', 'station', 'statistic', 'statue', 'status', 'statute', 'steam', 'steamboat', 'steamer', 'stick', 'stilt', 'stock', 'stone', 'stop', 'storage', 'store', 'storey', 'storm', 'story', 'storyline', 'strawberry', 'streaking', 'stream', 'street', 'street.jpg\">overlooking', 'strength', 'stretch', 'strike', 'strip', 'stronghold', 'structure', 'stub', 'student', 'studio', 'study', 'style', 'sub', 'subdistrict', 'subject', 'submarine', 'subsidiary', 'suburb', 'success', 'suffix', 'suicide', 'sum', 'summer', 'summit', 'sunrise', 'sunshine', 'supervision', 'support', 'supporter', 'surface', 'surgeon', 'survey', 'surveyor', 'survivor', 'suspect', 'swampland', 'swim', 'swimming', 'swing', 'sympathy', 'symphony', 'synagogue', 'system', 'sœur', 'tackle', 'tailor', 'target', 'tarring', 'tavern', 'tax', 'taxi', 'taxicab', 'teacher', 'teaching', 'team', 'tear', 'technique', 'technology', 'teen', 'telephone', 'telescope', 'television', 'temperature', 'temple', 'tenement', 'tennis', 'tenor', 'tension', 'term', 'terminal', 'terminology', 'terminus', 'territory', 'text', 'textile', 'thank', 'theater', 'theatre', 'theft', 'theme', 'thermostat', 'thief', 'thing', 'third', 'thousand', 'threat', 'ticket', 'tide', 'timber', 'time', 'title', 'title=\"asnat', 'title=\"bardwell', 'title=\"battle', 'title=\"beezie', 'title=\"bundesarchiv', 'title=\"chattanooga', 'title=\"confederate', 'title=\"diamond', 'title=\"harlan', 'title=\"loudspeaker.svg\">about', 'title=\"manzhoulitracks.jpg\">train', 'title=\"market', 'title=\"paterson', 'title=\"race', 'title=\"softredirarrow.svg\">soft', 'title=\"stade', 'title=\"typeface', 'title=\"vilkaviskis.ogg', 'title=\"vilkaviškis', 'title=\"wikisource', 'title=\"wikivoyage', 'today', 'toll', 'tom', 'tomb', 'ton', 'tongue', 'tonne', 'tool', 'top', 'topography', 'tornado', 'total', 'tourism', 'tourist', 'tournament', 'tow', 'tower', 'town', 'townlet', 'township', 'toy', 'track', 'trackage', 'tract', 'trade', 'trading', 'tradition', 'traffic', 'trafficking', 'trail', 'train', 'trainer', 'training', 'transformation', 'transit', 'transition', 'translator', 'transmitter', 'transport', 'transportation', 'transshipment', 'trauma', 'travel', 'traveler', 'treasury', 'treatment', 'treaty', 'tree', 'trend', 'trestle', 'trial', 'triangle', 'tribe', 'tributary', 'tribute', 'trip', 'troop', 'tropic', 'troupe', 'truck', 'truss', 'tryout', 'tuna', 'tunnel', 'turn', 'turnout', 'turnover', 'turnpike', 'turntable', 'tv', 'tvrm),*ccky', 'type', 'typeface', 'typewriter', 'u19', 'u21s', 'unemployment', 'uniform', 'union', 'unit', 'university', 'unrest', 'unseat', 'update', 'upgrade', 'uranium', 'urbanisation', 'urbanization', 'use', 'user', 'utility', 'vacation', 'vagrant', 'valley', 'value', 'van', 'vantage', 'variant', 'variation', 'variety', 'vegetation', 'vehicle', 'vendor', 'venom', 'venture', 'venue', 'verge', 'version', 'vicinity', 'victim', 'victory', 'video', 'vie', 'view', 'vigil', 'vigor', 'vilkaviškis</w>,<w', 'villa', 'village', 'visit', 'visitation', 'visitor', 'vitality', 'vocabulary', 'vocalist', 'voice', 'voip', 'vol', 'volleyball', 'volunteer', 'vote', 'voter', 'voting', 'wage', 'wakeboard', 'walk', 'walking', 'walkway', 'wall', 'war', 'ward', 'warning', 'warp', 'wash', 'waste', 'wastewater', 'watch', 'water', 'waterfall', 'waterfront', 'waterway', 'wave', 'way', 'wealth', 'weather', 'weaver', 'web', 'website', 'wedding', 'week', 'weekday', 'weekend', 'well', 'welterweight', 'west', 'westbound', 'wharf', 'wheel', 'while', 'whole', 'wife', 'will', 'window', 'wing', 'winner', 'winter', 'wireless', 'wit', 'woman', 'wood', 'word', 'work', 'worker', 'working', 'workshop', 'world', 'woro', 'worshiper', 'wound', 'wrestler', 'wrestling', 'writer', 'writing', 'yard', 'year', 'youth', 'zip', 'zone', 'zouglouphile', 'zydeco', '°', 'хот', 'ᠬᠣᠲᠠ', 'ᠮᠠᠨᠵᠤᠤᠷ', '−4.03333).,abidjan']\n"
     ]
    }
   ],
   "source": [
    "#Create a list of all tokens we want to keep in our collection\n",
    "pos = set()\n",
    "for doc in docs:\n",
    "    for token in doc:\n",
    "        if token.pos_ == 'NOUN': # or token.pos_ == 'ADJ' or token.pos_ == 'VERB': #Here we choose which parts of speech we keep\n",
    "            pos.add(token.lemma_)\n",
    "pos = (sorted(list(pos)))\n",
    "print(pos)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "0d6bb7ad",
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "#Create an empty data frame to store the term vectors we are going to build\n",
    "vectors = pd.DataFrame(0, sample.id, pos)\n",
    "\n",
    "#Build the binary vectors, iterating through each document\n",
    "i=0\n",
    "for doc in docs:\n",
    "    doc = list(map(str, doc))\n",
    "    for n in pos:\n",
    "        a = vectors.columns.get_loc(n)\n",
    "        if n in [x.lower() for x in list(doc)]:\n",
    "            vectors.iloc[i, a] = 1\n",
    "    i=i+1\n",
    "\n",
    "#print(vectors) #This is a term document indcidence matrix as we say earlier in the course"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "634a6dcc",
   "metadata": {},
   "source": [
    "Compare pairwise all documents using their binary term incidence matrices. Similar documents have cosine ~1, documents with no overlap have cosine = 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "2b09b985",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1.         0.03806935 0.10232344 0.14433757 0.1117464  0.12532409\n",
      "  0.09547033 0.125      0.13608276 0.04845016 0.09128709 0.16984156\n",
      "  0.13868983 0.09056544 0.14153463 0.13363062 0.14147911 0.11705751\n",
      "  0.12780193 0.14580296 0.11023964 0.15231795 0.15319084 0.14286919\n",
      "  0.0410305 ]\n",
      " [0.03806935 1.         0.30384016 0.21665358 0.24066124 0.33830773\n",
      "  0.43613919 0.20938142 0.33414756 0.22133592 0.33958062 0.05172606\n",
      "  0.27677432 0.46337996 0.24569879 0.45785077 0.19389664 0.466002\n",
      "  0.43370998 0.17317946 0.42447142 0.2286323  0.24993752 0.24326883\n",
      "  0.47797332]\n",
      " [0.10232344 0.30384016 1.         0.21339865 0.42423496 0.38370849\n",
      "  0.36842527 0.23022773 0.30733263 0.22096672 0.33398158 0.07944581\n",
      "  0.47887363 0.33043358 0.30288716 0.28909779 0.20917252 0.33733189\n",
      "  0.31064862 0.14833808 0.30663451 0.20229597 0.23032672 0.31440056\n",
      "  0.29868485]\n",
      " [0.14433757 0.21665358 0.21339865 1.         0.23700054 0.23116299\n",
      "  0.22272906 0.17526705 0.21886638 0.2397661  0.16779432 0.14008295\n",
      "  0.1878395  0.18823674 0.23113687 0.23145502 0.14002801 0.22343829\n",
      "  0.1716665  0.21646126 0.21821789 0.16152393 0.24908936 0.26995276\n",
      "  0.19289589]\n",
      " [0.1117464  0.24066124 0.42423496 0.23700054 1.         0.32519576\n",
      "  0.27694465 0.23546563 0.28023884 0.20883076 0.27480256 0.09760722\n",
      "  0.39292899 0.25329821 0.26028593 0.21332487 0.22069165 0.26588638\n",
      "  0.24832158 0.15641238 0.27151823 0.17924137 0.24315292 0.33837756\n",
      "  0.21222048]\n",
      " [0.12532409 0.33830773 0.38370849 0.23116299 0.32519576 1.\n",
      "  0.3953023  0.23925507 0.3209337  0.24508686 0.37798129 0.07740086\n",
      "  0.40317675 0.38136106 0.2863832  0.39584138 0.24316445 0.37951806\n",
      "  0.37940764 0.17541718 0.34880111 0.27369318 0.2094382  0.2947658\n",
      "  0.38145104]\n",
      " [0.09547033 0.43613919 0.36842527 0.22272906 0.27694465 0.3953023\n",
      "  1.         0.20457927 0.31737272 0.214097   0.3713746  0.07412493\n",
      "  0.33928211 0.4031652  0.25480445 0.43011873 0.23154955 0.44884533\n",
      "  0.35856858 0.15272071 0.35568901 0.23504369 0.26504407 0.27633817\n",
      "  0.42305706]\n",
      " [0.125      0.20938142 0.23022773 0.17526705 0.23546563 0.23925507\n",
      "  0.20457927 1.         0.29768105 0.19380063 0.22821773 0.08492078\n",
      "  0.20073527 0.2445267  0.21230194 0.26726124 0.13137346 0.22575377\n",
      "  0.26473257 0.11664237 0.25985058 0.21759707 0.26261287 0.25326811\n",
      "  0.25644061]\n",
      " [0.13608276 0.33414756 0.30733263 0.21886638 0.28023884 0.3209337\n",
      "  0.31737272 0.29768105 1.         0.22746648 0.30346637 0.06933752\n",
      "  0.27713943 0.34754797 0.25423759 0.40006613 0.17327582 0.34817357\n",
      "  0.38013156 0.19047619 0.31503601 0.22208408 0.24122532 0.25981628\n",
      "  0.36851387]\n",
      " [0.04845016 0.22133592 0.22096672 0.2397661  0.20883076 0.24508686\n",
      "  0.214097   0.19380063 0.22746648 1.         0.18955175 0.0987462\n",
      "  0.2079522  0.21061969 0.21395011 0.20718159 0.25851957 0.23333986\n",
      "  0.18045326 0.25431036 0.18312439 0.17711589 0.24174906 0.22653933\n",
      "  0.16698646]\n",
      " [0.09128709 0.33958062 0.33398158 0.16779432 0.27480256 0.37798129\n",
      "  0.3713746  0.22821773 0.30346637 0.18955175 1.         0.07087699\n",
      "  0.42722355 0.38549967 0.23478003 0.38338931 0.1834488  0.36113777\n",
      "  0.36       0.13690212 0.3598199  0.2315552  0.19178532 0.22357908\n",
      "  0.42378277]\n",
      " [0.16984156 0.05172606 0.07944581 0.14008295 0.09760722 0.07740086\n",
      "  0.07412493 0.08492078 0.06933752 0.0987462  0.07087699 1.\n",
      "  0.06942613 0.07383252 0.09615385 0.12104551 0.13730876 0.09088562\n",
      "  0.12403473 0.11886433 0.14978617 0.1478281  0.14867526 0.10588418\n",
      "  0.0836242 ]\n",
      " [0.13868983 0.27677432 0.47887363 0.1878395  0.39292899 0.40317675\n",
      "  0.33928211 0.20073527 0.27713943 0.2079522  0.42722355 0.06942613\n",
      "  1.         0.31414486 0.28762256 0.26141577 0.21952714 0.32225241\n",
      "  0.30065572 0.13793115 0.30072427 0.22109722 0.1840256  0.30944755\n",
      "  0.28033262]\n",
      " [0.09056544 0.46337996 0.33043358 0.18823674 0.25329821 0.38136106\n",
      "  0.4031652  0.2445267  0.34754797 0.21061969 0.38549967 0.07383252\n",
      "  0.31414486 1.         0.25226113 0.54218378 0.19329505 0.4652485\n",
      "  0.46826872 0.20282435 0.41076625 0.2175628  0.30443073 0.30489017\n",
      "  0.49942303]\n",
      " [0.14153463 0.24569879 0.30288716 0.23113687 0.26028593 0.2863832\n",
      "  0.25480445 0.21230194 0.25423759 0.21395011 0.23478003 0.09615385\n",
      "  0.28762256 0.25226113 1.         0.23452567 0.24029033 0.24993546\n",
      "  0.25427121 0.17829649 0.24072778 0.18478512 0.23788041 0.24265126\n",
      "  0.23693524]\n",
      " [0.13363062 0.45785077 0.28909779 0.23145502 0.21332487 0.39584138\n",
      "  0.43011873 0.26726124 0.40006613 0.20718159 0.38338931 0.12104551\n",
      "  0.26141577 0.54218378 0.23452567 1.         0.21606791 0.50949752\n",
      "  0.48795004 0.26497887 0.47982246 0.24425211 0.28074496 0.27769744\n",
      "  0.59215653]\n",
      " [0.14147911 0.19389664 0.20917252 0.14002801 0.22069165 0.24316445\n",
      "  0.23154955 0.13137346 0.17327582 0.25851957 0.1834488  0.13730876\n",
      "  0.21952714 0.19329505 0.24029033 0.21606791 1.         0.24334814\n",
      "  0.19483527 0.15559461 0.22917462 0.28498522 0.24415641 0.17640442\n",
      "  0.19902714]\n",
      " [0.11705751 0.466002   0.33733189 0.22343829 0.26588638 0.37951806\n",
      "  0.44884533 0.22575377 0.34817357 0.23333986 0.36113777 0.09088562\n",
      "  0.32225241 0.4652485  0.24993546 0.50949752 0.24334814 1.\n",
      "  0.42499112 0.1755497  0.41715376 0.23579212 0.26349264 0.28148299\n",
      "  0.46108107]\n",
      " [0.12780193 0.43370998 0.31064862 0.1716665  0.24832158 0.37940764\n",
      "  0.35856858 0.26473257 0.38013156 0.18045326 0.36       0.12403473\n",
      "  0.30065572 0.46826872 0.25427121 0.48795004 0.19483527 0.42499112\n",
      "  1.         0.17888544 0.42093999 0.19069252 0.23014238 0.26179078\n",
      "  0.45845591]\n",
      " [0.14580296 0.17317946 0.14833808 0.21646126 0.15641238 0.17541718\n",
      "  0.15272071 0.11664237 0.19047619 0.25431036 0.13690212 0.11886433\n",
      "  0.13793115 0.20282435 0.17829649 0.26497887 0.15559461 0.1755497\n",
      "  0.17888544 1.         0.17634669 0.18274347 0.24505429 0.21815478\n",
      "  0.1722922 ]\n",
      " [0.11023964 0.42447142 0.30663451 0.21821789 0.27151823 0.34880111\n",
      "  0.35568901 0.25985058 0.31503601 0.18312439 0.3598199  0.14978617\n",
      "  0.30072427 0.41076625 0.24072778 0.47982246 0.22917462 0.41715376\n",
      "  0.42093999 0.17634669 1.         0.2220587  0.21505972 0.25036122\n",
      "  0.44973404]\n",
      " [0.15231795 0.2286323  0.20229597 0.16152393 0.17924137 0.27369318\n",
      "  0.23504369 0.21759707 0.22208408 0.17711589 0.2315552  0.1478281\n",
      "  0.22109722 0.2175628  0.18478512 0.24425211 0.28498522 0.23579212\n",
      "  0.19069252 0.18274347 0.2220587  1.         0.27429019 0.16278764\n",
      "  0.20356104]\n",
      " [0.15319084 0.24993752 0.23032672 0.24908936 0.24315292 0.2094382\n",
      "  0.26504407 0.26261287 0.24122532 0.24174906 0.19178532 0.14867526\n",
      "  0.1840256  0.30443073 0.23788041 0.28074496 0.24415641 0.26349264\n",
      "  0.23014238 0.24505429 0.21505972 0.27429019 1.         0.24558079\n",
      "  0.25860327]\n",
      " [0.14286919 0.24326883 0.31440056 0.26995276 0.33837756 0.2947658\n",
      "  0.27633817 0.25326811 0.25981628 0.22653933 0.22357908 0.10588418\n",
      "  0.30944755 0.30489017 0.24265126 0.27769744 0.17640442 0.28148299\n",
      "  0.26179078 0.21815478 0.25036122 0.16278764 0.24558079 1.\n",
      "  0.2494012 ]\n",
      " [0.0410305  0.47797332 0.29868485 0.19289589 0.21222048 0.38145104\n",
      "  0.42305706 0.25644061 0.36851387 0.16698646 0.42378277 0.0836242\n",
      "  0.28033262 0.49942303 0.23693524 0.59215653 0.19902714 0.46108107\n",
      "  0.45845591 0.1722922  0.44973404 0.20356104 0.25860327 0.2494012\n",
      "  1.        ]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABGEAAARuCAYAAABzxirnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABEuUlEQVR4nO3deZimd13n+++3q6rXpLN0ZycQlhwG5YKALYILIogGhtFhRmcGlYs9guMZQRxx1MttzozHFT0ziicQwHEYFBXmODOAxBUXFiMmEAgIsmXtJJCk093p7uqq3/mjnoxt252+76ru78+78npdV650rfenq+566ql3P89T2VoLAAAAAE6tDb0HAAAAADwYiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAYAJy8yPZubTCo/3NZn5iVW+7UMzc29mzs2e/qPMfOkatpy0v3tmnpeZ783MezPz507G+zzq/V+SmS0z50/2+wYApsMVAQAokpnfFhHfGxH/KCLujYhrI+I/tNb+dLXvs7X2pSdn3eDj/UlEPHqVb/v5iDjtJG7533/3zPyxiHhUa+07VvnuroiIOyNie2utnYR561pmvjkibmqt/XDvLQAwJW4JAwAFMvN7I+IXIuI/RsR5EfHQiPjliPjmjrMm6RTdmuRhEfGx1QQYt24BAIYSYQDgFMvMMyLiJyLiX7fW3t5a29daW2yt/Y/W2r+dvc6mzPyFzLxl9t8vZOam2ct2Zub/zMy7M/OLmfknmblh9rLPZubXz/78Y5n5tsz8L7O71Xw0M3cdsePCzPztzLwjMz+Tmf/mATY/OzM/Nns/N2fm982e/7TMvOmI1/tsZv7bzPxwZu7LzKtmd+151+xtfy8zz5q97nHvkpOZj8zMP8jML2TmnZn5lsw886jjvCYzPxwR+zJz/v6/e2ZeHhE/GBH/cnZ3p+sy81sz8y+POsarM/O/H+PYb46IF0TE98/e/utP8Pl4WmbeNNtzW0S86Rjvcy4zf3b2d/l0RPzjo15+YWb+zuzz+anMfNlRb/uDmfk3s4/hX2bmxcf6+B15l67MfGFm/llmvnZ2rnw6M79y9vwbM/P2zHzBEW+7abbx85m5OzN/JTO3HPV3fPXs7W7NzBfNXnZFRHz7ER+v/zF7/mtm58q9mfmJzHzG0R8XAHiwE2EA4NR7SkRsjoh3PMDr/FBEPDkiLouIx0fEkyLi/rt6vDoiboqIc2LlVjQ/GBHHu8XGN0XEr0fEmRHxOxHxnyMiZtHmf0TEdRFxUUQ8IyJemZnfeJz3c1VEfGdr7fSIeGxE/MEDbP/nEfHMiPg/IuKfRMS7Zht3xsp1jePGniNkRPxkRFwYEY+JiIsj4seOep3nxUrMOLO1dvj+Z7bW3h0rtzD6jdbaaa21x8/+7g/PzMcc8fbfERG/dvSBW2svjIi3RMRPz97+9+KBPx8REedHxNmxcguaK47x93lZRDwnIp4QEbsi4luOevlbY+VzeuHsZf/xiGjxvbO/67MjYntEvDgi9h/jGMfyFRHx4YjYERH/LVbOhS+PiEfFyt//P2fm/XcJ+6lY+ZxdNnv5RRHxI0f9Hc+YPf8lEfFLmXlWa+3K+Lsfr3+SmY+OiO+OiC+fnTPfGBGfHbgZAB40RBgAOPV2RMSdR4aDY/j2iPiJ1trtrbU7IuLHI+L5s5ctRsQFEfGw2S1o/uQB7jbzp621d7bWlmIlODx+9vwvj4hzWms/0Vo71Fr7dES8PiL+1XHez2JEfElmbm+t3dVa+9ADbP9PrbXdrbWbI+JPIuIDrbW/aq0djJXw9IQHeNuIiGitfaq1dnVr7eDs7//zEfG1R73a/9Nau7G1dt+A93cwIn4jVsJDZOaXRsQlEfE/T/S2Mw/0+YiIWI6IH53tPdaefxERvzDb+8VYCUwx23JxRHx1RLymtXagtXZtRLzhiPf/0oj44dbaJ9qK61prXxi4+zOttTfNPv+/ESsx6ydmO98TEYci4lGZmbESil7VWvtia+3eWAlZR54Pi7O3XWytvTMi9sbxHw9oKSI2xco5s9Ba+2xr7W8GbgaABw0RBgBOvS9ExM5j3Q3nCBdGxOeOePpzs+dFRPxMRHwqIt4zu4vJDzzA+7ntiD/vj4jNs+M+LCIunN1N5e7MvDtWbq1y3nHezz+PlVtifC4z/zgzn/IAx9x9xJ/vO8bTJ3ww3sw8NzN/fXZ3lj0R8V9j5ZY0R7rxRO/nKL8aEd82Cw7Pj4i3zeLMEA/0+YiIuKO1duAEb3/k3s8d9bL7w8eRL79o9ueLI2K1AePoj3201o71+TgnIrZGxF8ecT68e/b8+33hqHC4P47zuWytfSoiXhkrt166ffa5vPBYrwsAD2YiDACceu+LiAMR8U8f4HVuiZVQcr+Hzp4XrbV7W2uvbq09Ilbu7vO9q3i8jRtj5VYSZx7x3+mttWcf65Vba3/RWvvmiDg3Iv57RLxt5PHG+slYuYvV41pr22PlFix59KwHePu/97LW2vtj5ZYfXxMR3xbHuCvSAzju52PAloiIW2Mlphz59ke+77Mz8/SjXn7z7M83RsQjj/E+983+v/WI551/gh3Hc2esBJkvPeJ8OKO1NvS3Vx3r4/3fWmtfHSsftxYrd3cCAI4gwgDAKdZauydWHmvjlzLzn2bm1sxcyMxnZeZPz17trRHxw5l5TmbunL3+f42IyMznZOb9dyHZEyt3/VgaOeODEbFn9uCpW2YP/vrYzPzyo18xMzdm5rdn5hmttcUjjnkqnR4rd3e5OzMvioh/O/Ltd0fEJbPHvjnSf4mVx8U5PPJXgR/38zHQ2yLi32TmQ3LlgYn/962XWms3RsSfR8RPZubmzHxcrDzmyltmr/KGiPj3mXlprnhcZu6Y3S3q5oj4jtnn78Vx7FhzQq215Vi5O9prM/PciIjMvOgBHiPoaLsj4hH3P5GZj87Mp+fKgxcfiJXAc6rPGQCYHBEGAAq01n4+Vh5w9Ycj4o5YubXDd8fKrUwiIv6viLgmVh5U9SMR8aHZ8yIiLo2I34uVSPG+iPjl1tofjTz+UqzciuayiPhMrNwS4g2x8sCrx/L8iPjs7K5BL4/ZY6ucQj8eEU+MiHsi4n9FxNtHvv1vzv7/hcw88vFrfi1WHlh4zK1gIh748zHE6yPid2PlgZA/FH//7/O8WHmMmlti5XFzfrS1dvXsZT8fKxHnPbESwK6KiC2zl70sVgLVFyLiS2Ml5qzWa2Llbm7vn32efy+O/5gvR7sqVh7/5e5c+Y1TmyLi/46V8+q2WLkF1Q+uYRsArEt5/Mf1AwCYttmvXL49Ip7YWvtk7z0AwIObW8IAAOvZKyLiLwQYAOAfggf6LQ0AAJOVmZ+NlQf3/ad9lwAArHB3JAAAAIAC7o4EAAAAUECEAQAAAChQ+pgwO8+ea5dcvFB5yFX76w9v7T1hlFyY0MP7bJjrvWCc5eXeC4bL7L1gnCl9bDdMrFlP7FRoBw/1njBYzk/sMmxhGt93I2JalwkREUtLvRcMNzex85ZTZ1IPRTCxb2ZTuh42tcvbuYldD5uSCV0mTOn64oHYF4fawWNeKJT+5H7JxQvxwd+9uPKQq/aNF17We8Io8zvP6z1hsLb9tN4TRsl99/WeMNyUYlxEtAl9bHPblt4TRmkTCwVLn/587wmDzZ11Zu8Jo7SLzu09YbDcd6D3hHHuuqf3guHOPrP3glFyaTo/ILaJ/XCYB6bzQ8zUfvBuG6cTvSd1/TYi2vZtvSesX4en8w8KS5/8dO8Jg32g/f5xXzatSzYAAACAiRJhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACiwpgiTmZdn5icy81OZ+QMnaxQAAADAerPqCJOZcxHxSxHxrIj4koh4XmZ+yckaBgAAALCerOWWME+KiE+11j7dWjsUEb8eEd98cmYBAAAArC9riTAXRcSNRzx90+x5f0dmXpGZ12TmNXd8YWkNhwMAAACYrrVEmDzG89rfe0ZrV7bWdrXWdp2zY24NhwMAAACYrrVEmJsi4uIjnn5IRNyytjkAAAAA69NaIsxfRMSlmfnwzNwYEf8qIn7n5MwCAAAAWF/mV/uGrbXDmfndEfG7ETEXEW9srX30pC0DAAAAWEdWHWEiIlpr74yId56kLQAAAADr1lrujgQAAADAQCIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACmRrrexg2/Ps9hX5jLLjrcXv3nJt7wmjfOWrXt57wmDbbjnYe8IoS5um0yr3PmRj7wmj7Lsge08Y7KL37u89YZS7H7Wl94RRzvjsgd4TBrv7EZt7Txhlx6/9Re8Jg+1/zhN7Txjl9ifM954w2AV/fqj3hFFue/J0vp899F339p4wyuf+8em9Jww2N51vDRER8dB33tV7wmA3Xn5W7wmjnP/B6ZwMS5vmek8YJZfresBa3fy1C70nDHbjL702Dtx04zF/2JnOT5cAAAAAEybCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAArMVx4sF+Zjfud5lYdcta981ct7Txjlz1/7K70nDPbMf/HC3hNGuf3LNvWeMNjCvt4Lxtl0V+s9YbClTXO9J4wyf3A6H9upyeXeC8bZcNq23hMG2/3l0/o6O/MT0/k6O3hm6VW+Ndtx/VLvCYO1hWmdt9s/M53zdm5i38vywGLvCYNt/9y0vpltODChy4TM3hNG2X/+Qu8Jg+28bjqXCbfuP/7L3BIGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUGC+9Ggb5qJtP630kKu17ZaDvSeM8sx/8cLeEwa7+m1v7j1hlGc8/yW9JwzXeg8YJ5enM3jh7gO9J4yyYWm594RRlham828CO/7qrt4TxjlnR+8Fgz38Hff2nrBuLW7f1HvCujV/+57eE0bZPqHL24W77us9YZTFc0/vPWGwbTdP62ed+T3TuR62eMb23hNG2XjvdK4zbt49ncuEDYeO/3GdzqUwAAAAwISJMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVWHWEyc3NmfjAzr8vMj2bmj5/MYQAAAADryfwa3vZgRDy9tbY3Mxci4k8z812ttfefpG0AAAAA68aqI0xrrUXE3tmTC7P/2skYBQAAALDerOkxYTJzLjOvjYjbI+Lq1toHTsoqAAAAgHVmTRGmtbbUWrssIh4SEU/KzMce/TqZeUVmXpOZ1xxa2r+WwwEAAABM1kn57Uittbsj4o8i4vJjvOzK1tqu1tqujXNbT8bhAAAAACZnLb8d6ZzMPHP25y0R8fUR8fGTtAsAAABgXVnLb0e6ICJ+NTPnYiXmvK219j9PziwAAACA9WUtvx3pwxHxhJO4BQAAAGDdOimPCQMAAADAAxNhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACgwX3q05eXIffeVHnK1ljad2XvCKLd/2abeEwZ7xvNf0nvCKL//a1f1njDYV7zmFb0njLLp3qXeEwZrc1t6Txjl0Pbai/e12nL7wd4TBtv3iO29J4xy2p99pveEwW599vm9J4xy1l8v9p4w3MT+2W3hnul8bBcvPKP3hFEWT5vO94elTdt6Txhly417ek8YbO+lZ/aeMMrc3umct7ncek8Y5b4d0/nYbrxrrveE4fL4L5rYt2QAAACAaRJhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAXmS4+WGbFQe8jV2vuQjb0njLKwr/eCEVrvAeN8xWte0XvCYB/4qdf1njDK43/6u3pPGGzH9Uu9J4yyuC17Txhl/rRpfG+IiLj3oulsjYg47eDB3hMGa3O9F4xzzyMXek8YbNut07oMu+vSrb0nDLbjY4d6Txhlz8OmdRk2JZt3T+dju3/nxC5wY1vvAYNtvnNalwkL+6ZzLtx52XS+Nxz+xPFv7+KWMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIAC86VHW16Otu++0kOu1r4LsveEUTbd1XpPGCyXp7M1ImLTvUu9Jwz2+J/+rt4TRrnu+3+594TBvuFbXtB7wjhtofeCUeb3He49YbBcntbHtrXpXOZO6XtZRMTBs6ZzXWF+/3LvCaMc3jLXe8JgeXhi5+3ZvRcMt2X3tD62eXCx94ThJvZP8Qt7p3N9/OBZG3tPGGVucTpfZ4vbei8Yrj3A19jEvvwAAAAApkmEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKLCmCJOZZ2bmb2XmxzPzhsx8yskaBgAAALCezK/x7X8xIt7dWvuWzNwYEVtPwiYAAACAdWfVESYzt0fEUyPihRERrbVDEXHo5MwCAAAAWF/WcnekR0TEHRHxpsz8q8x8Q2ZuO/qVMvOKzLwmM685tHzfGg4HAAAAMF1riTDzEfHEiHhda+0JEbEvIn7g6FdqrV3ZWtvVWtu1ccOWNRwOAAAAYLrWEmFuioibWmsfmD39W7ESZQAAAAA4yqojTGvttoi4MTMfPXvWMyLiYydlFQAAAMA6s9bfjvR/RsRbZr8Z6dMR8aK1TwIAAABYf9YUYVpr10bErpMzBQAAAGD9WstjwgAAAAAwkAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQIH50qNt2BC5bUvpIVfrovfu7z1hlKVNc70nDLZw94HeE0Zpc9M4ZyMidly/1HvCKN/wLS/oPWGw9/zWr/aeMMpXvfLlvSeMsuHQdM7dsz9+sPeEUXK+9lv9Wpz18Wl97z18+kLvCYPN7T/ce8IoOz+SvScMtvkzd/aeMMqFyzt6Txhs8bTpXH5FROS907kMO+2WaV0mbDi03HvCYLl5WrdzyAldrdn5kcXeEwa78b523JdN6wwBAAAAmCgRBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoMB86dEyos3PlR5yte5+1JbeE0aZP9h6Txhsw9Jy7wmjHNpe+2WyFovbsveEcdpC7wWDfdUrX957wih/9gu/0nvCKF//7S/uPWGw/edu7D1hlIVDh3pPGOzAeZt6Txhl/45pXKeJiDjtlml9f7j70ul8791649beE0a5+5HT+Tpb2jKt83bL7rN7Txjs4BnTufyKiFjYe7j3hME2LE7n57KIiOWF6Xyd7bl4Ot8bljYe/+PqljAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQIFsrZUdbHue3b5i7hvKjrcWy1/zuN4T1q2lhWm1v/n7lnpPGGzxtPneE0aZ33e494TBNhyaznkQEbG0baH3hFF+7y1v7D1hsMu/+fm9J4yy4fO7e08Y7PCjLuw9Yd1q2XvBOFl39XTNDp0xrcvbuQPLvScMtnDXgd4TRjlw/tbeEwbbfPv+3hPGmc5pG4fO2dJ7wigbFqfzwV34wnTO2/f/9VVxz/5bjvndd1o/DQMAAABMlAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKDAmiJMZn5PZl6fmR/NzFeepE0AAAAA686qI0xmPjYiXhYRT4qIx0fEczLz0pM1DAAAAGA9WcstYR4TEe9vre1vrR2OiD+OiOeenFkAAAAA68taIsz1EfHUzNyRmVsj4tkRcfHJmQUAAACwvsyv9g1bazdk5k9FxNURsTcirouIw0e/XmZeERFXRERsjq2rPRwAAADApK3pgXlba1e11p7YWntqRHwxIj55jNe5srW2q7W2ayE2reVwAAAAAJO16lvCRERk5rmttdsz86ER8c8i4iknZxYAAADA+rKmCBMRv52ZOyJiMSL+dWvtrpOwCQAAAGDdWVOEaa19zckaAgAAALCerekxYQAAAAAYRoQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoMB85cFyfi7mzjqz8pCr9sVHbO49YZRc7r1guB1/dVfvCaPse8T23hMGu/ei0i/pNcvlhd4TBjv74wd7Txhl/7kbe08Y5fJvfn7vCYO9+//7td4TRnnWpV/Ve8Jgt3zN1t4TRpnf13vBcDs+Nq3LsC8+ZlPvCYOd9757ek8Y5Y4vm871mohpfS877w9u6z1hsN1PP7/3hFHO/NSh3hMG23jXtC5vD2+bzvXxG591du8Jgx3aPXfcl7klDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQIFtrZQc7Y+uF7cmPfmnZ8daiffSTvSeMsuG0bb0nDHfOjt4Lxrn73t4Lhjt4sPeCUSovf9Yq5+d7TxilHTrUe8IoOaHLsLZ3X+8Jo7zrk3/We8Jgz7r0q3pPGGfDdP4tKzdv7j1hlNw6nb1tYVrfH+LuPb0XDJbbtvaeMM7ydK7XtAl9jXFq5Z69vScM1g5M52ed993zjrjn8B15rJdN59oDAAAAwISJMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAXmS4+2vBy570DpIVdr33Oe2HvCKLu/fK73hMEe/o57e08Y5dZnn997wmBtOqdBRERsuqv1njDYWR/f33vCKAfO29R7wiib7zjUe8Jgt3zN1t4TRnnWpb0XDPeuT/5Z7wmjPOX7Xt57wmBzB6dzeRsRMX9gufeEwRb2Hu49YZR7n3xe7wmDzU3nW0NERJzxod29Jwx2z+N39p4wytbdB3tPGGxu77RO3HueOJ1zYf7AdL6XLf3B5uO+zC1hAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVOGGEy842ZeXtmXn/E834mMz+emR/OzHdk5pmndCUAAADAxA25JcybI+Lyo553dUQ8trX2uIj464j4dyd5FwAAAMC6csII01p7b0R88ajnvae1dnj25Psj4iGnYBsAAADAunEyHhPmxRHxruO9MDOvyMxrMvOaQ0v3nYTDAQAAAEzPmiJMZv5QRByOiLcc73Vaa1e21na11nZtnNuylsMBAAAATNb8at8wM18QEc+JiGe01trJmwQAAACw/qwqwmTm5RHxmoj42tba/pM7CQAAAGD9GfIrqt8aEe+LiEdn5k2Z+ZKI+M8RcXpEXJ2Z12bmr5zinQAAAACTdsJbwrTWnneMZ191CrYAAAAArFsn47cjAQAAAHACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABeZLj7a0FHHXPaWHXK3bn3BB7wmjnPmJ1nvCunXWXy/2njDYPY9c6D1hlINnZe8Jgx0+fVof2/075npPGGXzHb0XDDe/r/eCkTZM599bnvJ9L+89YZT3/eyv9J4w2NNe8rLeE0b5/HOmc95e+quHe08Y5e5Lt/SeMFxO6/rtthu3954w2J6HTudrLCJi6629Fwy351Gn954wypS+zG578nTO28UPHP9l0/lbAAAAAEyYCAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBgvvRoc3MRZ59ZesjVuuDPD/WeMMrBM2s/lWuxuH1T7wnjTChVbrt1qfeEUeb3L/eeMNjc/sO9J4xy2i3Ze8IobUJzd3zsYO8Jo+Tmzb0nDDZ3sPWeMMrTXvKy3hMG+6OrXt97wijPeP5Lek8YbHH7xt4TRrngfYu9Jww2pe8NEREbDkznusI5107re9ncfdM5bzd/YUI/PEREbJjOF9pD3zOd6wm37zn+1omdIQAAAADTJMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACsxXHzCXlqsPuSq3PXlj7wmj7Lh+qfeEdWvhnsXeEwa769KtvSeMcnjLXO8Jg+38SPaeMMrdl5ZfvK/JeR+cztfZFx+zqfeEUS781ObeEwabPzCN6wj3+/xzpvNvWc94/kt6Txjl93/tqt4TBnvmv3xR7wmjfObyhd4TBtuwOK3vvRf82bbeEwa77SnTuQ4WEXHRH0/nXFjcNq2PbbbWe8JgNz33cO8Jgx361PFfNp1rDwAAAAATJsIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAAChwwgiTmW/MzNsz8/ojnvdjmXlzZl47++/Zp3YmAAAAwLQNuSXMmyPi8mM8/7Wttctm/73z5M4CAAAAWF9OGGFaa++NiC8WbAEAAABYt9bymDDfnZkfnt1d6ayTtggAAABgHVpthHldRDwyIi6LiFsj4ueO94qZeUVmXpOZ1xxa2r/KwwEAAABM26oiTGttd2ttqbW2HBGvj4gnPcDrXtla29Va27VxbutqdwIAAABM2qoiTGZecMSTz42I64/3ugAAAABEzJ/oFTLzrRHxtIjYmZk3RcSPRsTTMvOyiGgR8dmI+M5TNxEAAABg+k4YYVprzzvGs686BVsAAAAA1q21/HYkAAAAAAYSYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAokK21soOdsfn89pSHvaDseGuxdNa23hNGaQtzvScMNn/7nt4TRlm88IzeEwZb2jSd8yAiIg/XXf6s1ebP3Nl7wijL27f2njDKfRed1nvCYJt37+89YZQN90xn7+JFZ/aeMMqG+w73njDY4vaNvSeMMndwufeEwa7+jTf1njDK01/40t4TBlu4+2DvCaPM7Z3O3kPnTef7bkTEwbMWek8YbGHvUu8Jo2y6fTrXE5Y3z/eeMNgHPvwrsWfvzXmsl7klDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQYL70aK1FHjhUesjV+tw/Pr/3hFG2f6b1njDY9oVptb/F02q/TNZiz8OmszUi4uDZvRcMd+Hyjt4TRrn7kZt6TxjljM9M43tDRMQdX7a994RRznv7bb0nDHbvk8/rPWGUuy/d0nvCYBe8b7H3hFE+c/lC7wmDPf2FL+09YZQ/ePMbek8Y7LG/+F29J4xy8e8u9Z4w2O5dm3tPGOXcDx3sPWGwpc3T+lln78NP7z1hsDsfP9d7wmCHPnf882BaZwgAAADARIkwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABeZrD5cRc9PoPnMHei8YZ+5g6z1hsIW77us9YZSlTdt6T1i3tuyeznm7eFrxxeUaLW3J3hNGWbhrShe6G3sPGCW3be09YbC5Q70XjJTTuQxr07pIiA2L0xm8cPfB3hNGeewvflfvCYNd/z2/3HvCKJe/59t7TxhuOl9iERGxsGdC3yDatK4nTEqb671guAe4ijCNIgIAAAAwcSIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAgRNGmMx8Y2benpnXH/G8yzLz/Zl5bWZek5lPOrUzAQAAAKZtyC1h3hwRlx/1vJ+OiB9vrV0WET8yexoAAACA4zhhhGmtvTcivnj0syNi++zPZ0TELSd5FwAAAMC6Mr/Kt3tlRPxuZv5srIScrzzeK2bmFRFxRUTE5rnTV3k4AAAAgGlb7QPzviIiXtVauzgiXhURVx3vFVtrV7bWdrXWdm2c27rKwwEAAABM22ojzAsi4u2zP/9mRHhgXgAAAIAHsNoIc0tEfO3sz0+PiE+enDkAAAAA69MJHxMmM98aEU+LiJ2ZeVNE/GhEvCwifjEz5yPiQMwe8wUAAACAYzthhGmtPe84L/qyk7wFAAAAYN1a7d2RAAAAABhBhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACgQLbWyg52xuYL2lMueUHZ8daibdnYe8IoeWCx94TBFs89vfeEURa+sK/3hMHaxvneE0bJg9M5b/Pe/b0njLJ0wdm9J4xyYOfm3hMG2/rJO3tPGGVKX2dt00LvCaMcPmd77wmDbThwuPeEUe67aFvvCYNt+/TdvSeMMqXrCi2z94RR3v2/3tJ7wmDPeva39Z4wSpubzrlwcOeW3hNGmd+/1HvCYPN7DvSeMNj7P/GGuGf/Lcc8cd0SBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBgvvRoy8uR++4rPeRqff655/WeMMr2zy33njDYtpsP9p4wyt5Lz+w9YbD9O+d6TxhnQhn4tFsO954wysEzpnUunPGJe3tPGGz308/vPWGU8/74jt4TBrvn8Tt7Txhlz0OncyF2zrXT+t5721Omcxl28b7Tek8YZfeuzb0nDJe9B4zzrGd/W+8Jg73rnf+t94RRnvm8F/WeMNj8/qXeE0Y5vK02CazFzU87q/eEwQ6+/vjfx6Zz7QEAAABgwkQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQIH50qPNbYi2fVvpIVfr/A8e6D1hlA0HlnpPGGx+z7Q+tnN7a79M1mYaX1/3W9g7nfN2w6Hl3hNGWdh7uPeEcSb04T3zU4d6T1i3tu4+2HvCKFtv7b1guLn7FntPGOWiP87eEwY7eNZC7wmjnPuh6XydLeyZ1uVtm5vOefvM572o94RRrn7rm3pPGOypr7ii94RRtv3Nnt4TBnvInk29Jwx2y57jX7l1SxgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACJ4wwmXlxZv5hZt6QmR/NzO+ZPf9bZ08vZ+auUz8VAAAAYLrmB7zO4Yh4dWvtQ5l5ekT8ZWZeHRHXR8Q/i4j/91QOBAAAAFgPThhhWmu3RsStsz/fm5k3RMRFrbWrIyIy89QuBAAAAFgHRj0mTGZeEhFPiIgPnJI1AAAAAOvU4AiTmadFxG9HxCtba3tGvN0VmXlNZl5zaGn/ajYCAAAATN6gCJOZC7ESYN7SWnv7mAO01q5sre1qre3aOLd1NRsBAAAAJm/Ib0fKiLgqIm5orf38qZ8EAAAAsP4M+e1IXxURz4+Ij2TmtbPn/WBEbIqI/xQR50TE/8rMa1tr33hKVgIAAABM3JDfjvSnEXG8X4H0jpM7BwAAAGB9GvXbkQAAAABYHREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgALzvQf8Q7W0aa73hFFaZu8Jgy2esb33hFFyufWeMNjmOw/1njDKwbM29p4wWG6eVrPesDid8zYiYumcLb0nDLbxroO9J6xbc3undRm251Gn954w2OYvTOsybHHbdK6HLexd6j1hlKUpfT9r07meEBGxvDCd6+Pz+6d13j71FVf0njDYe193Ze8Jo3zdi17ae8JgbW46X2Nt/vhbJ3QpDAAAADBdIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACgwHzp0VqLOLxUesjVyuXWe8Io+89f6D1hsI33LveeMMp9O2q/TNZiYd9c7wmjzC1O5+ssD/ZeMM7yQvaeMMr8/ml8b4iIOLxtOpe3ERGbbrqj94TB7nnizt4TRsnpXIRFbJjWZUK26XxwN92+v/eEUfY+/PTeE9athT2He08Y7PC26Vy/jYjY9jd7ek8Y7Ote9NLeE0b5wze9ofeEwb7uxS/rPWG4B/g25pYwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgALzlQdrBw/F0ic/XXnIVbv5hef1njDKzuta7wmDbd59X+8Jo2y8a673hMHuvGxr7wmjLG7rvWC4nR9Z7D1hlD0Xl168r9l5f35v7wmD3fiss3tPGOWh1x3sPWGw+QPT+V4WEXHbk6fzb1kPfc+0PrY3Pfdw7wmDXfq6aV3e3vn46VyviTahrRFxyX/f23vCYDc/7azeE0Z5yJ5NvScM1uay94RRvu7FL+s9YbA/fOPre08Y7EnfeOdxXzadaw8AAAAAEybCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFDhhhMnMizPzDzPzhsz8aGZ+z1Ev/77MbJm589TNBAAAAJi2+QGvczgiXt1a+1Bmnh4Rf5mZV7fWPpaZF0fEMyPi86d0JQAAAMDEnfCWMK21W1trH5r9+d6IuCEiLpq9+LUR8f0R0U7ZQgAAAIB1YNRjwmTmJRHxhIj4QGZ+U0Tc3Fq77gRvc0VmXpOZ1yzGwdUvBQAAAJiwIXdHioiIzDwtIn47Il4ZK3dR+qGI+IYTvV1r7cqIuDIiYnue7RYzAAAAwIPSoFvCZOZCrASYt7TW3h4Rj4yIh0fEdZn52Yh4SER8KDPPP1VDAQAAAKbshLeEycyMiKsi4obW2s9HRLTWPhIR5x7xOp+NiF2ttTtP0U4AAACASRtyS5iviojnR8TTM/Pa2X/PPsW7AAAAANaVE94SprX2pxGRJ3idS07WIAAAAID1aNRvRwIAAABgdUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAApka63uYJl3RMTnTsG73hkRd56C98u0OA+4n3OBCOcBf8u5QITzgBXOA+7nXCDi1J0HD2utnXOsF5RGmFMlM69pre3qvYO+nAfcz7lAhPOAv+VcIMJ5wArnAfdzLhDR5zxwdyQAAACAAiIMAAAAQIH1EmGu7D2AfxCcB9zPuUCE84C/5VwgwnnACucB93MuENHhPFgXjwkDAAAA8A/derklDAAAAMA/aJOOMJl5eWZ+IjM/lZk/0HsPfWTm5sz8YGZel5kfzcwf772JPjLzzMz8rcz8eGbekJlP6b2JPjLzezLz+tllwit776FGZr4xM2/PzOuPeN7PzC4TPpyZ78jMMztOpMhxzoUfy8ybM/Pa2X/P7rmRU+8458Flmfn+2TlwTWY+qedGTr3MvDgz/3B23fCjmfk9s+d/6+zp5cz0W5IeBI53Lhzx8u/LzJaZO0/ljslGmMyci4hfiohnRcSXRMTzMvNL+q6ik4MR8fTW2uMj4rKIuDwzn9x3Ep38YkS8u7X2jyLi8RFxQ+c9dJCZj42Il0XEk2LlPHhOZl7adxVF3hwRlx/1vKsj4rGttcdFxF9HxL+rHkUXb46/fy5ERLy2tXbZ7L93Fm+i3pvj758HPx0RP95auywifmT2NOvb4Yh4dWvtMRHx5Ij417OfG6+PiH8WEe/tOY5SxzsXIjMvjohnRsTnT/WIyUaYWLly/anW2qdba4ci4tcj4ps7b6KDtmLv7MmF2X8e7OhBJjO3R8RTI+KqiIjW2qHW2t1dR9HLYyLi/a21/a21wxHxxxHx3M6bKNBae29EfPGo571ndh5ERLw/Ih5SPoxyxzoXePA5znnQImL77M9nRMQtpaMo11q7tbX2odmf742Vf6S7qLV2Q2vtE33XUel458Lsxa+NiO+Pgp8jpxxhLoqIG494+qb42w8gDzKZOZeZ10bE7RFxdWvtA50nUe8REXFHRLwpM/8qM9+Qmdt6j6KL6yPiqZm5IzO3RsSzI+Lizpv4h+HFEfGu3iPo6rtnd017Y2ae1XsMXbwyIn4mM2+MiJ8Nt457UMnMSyLiCRHhZ4UHuSPPhcz8poi4ubV2XcWxpxxh8hjPc+uHB6nW2tLsZqUPiYgnze6OwIPLfEQ8MSJe11p7QkTsiwiPFfUg1Fq7ISJ+KlbuhvLuiLguVm5+yoNYZv5QrJwHb+m9hW5eFxGPjJW7Lt8aET/XdQ29vCIiXtVauzgiXhWzW9Cy/mXmaRHx2xHxytbant576OfIcyFWrhv8UKzcPbHElCPMTfF3/2XzIeHmhA96s7uf/FEc+37grG83RcRNR9wK6rdiJcrwINRau6q19sTW2lNj5abon+y9iX4y8wUR8ZyI+PbWmn+weZBqre2e/aPNckS8Plbu2s6Dzwsi4u2zP/9mOA8eFDJzIVZ+6H5La+3tJ3p91q9jnAuPjIiHR8R1mfnZWOkKH8rM80/VhilHmL+IiEsz8+GZuTEi/lVE/E7nTXSQmefc/9suMnNLRHx9RHy86yjKtdZui4gbM/PRs2c9IyI+1nESHWXmubP/PzRWHnTvrX0X0UtmXh4Rr4mIb2qt7e+9h34y84IjnnxurNx1kQefWyLia2d/fnqI9OteZmas3OLphtbaz/feQz/HOhdaax9prZ3bWruktXZJrPzD7hNnP1ucmh1T/geh2a8W/IWImIuIN7bW/kPfRfSQmY+LiF+NlfNgQ0S8rbX2E31X0UNmXhYRb4iIjRHx6Yh4UWvtrq6j6CIz/yQidkTEYkR8b2vt9ztPokBmvjUinhYROyNid0T8aKw83sOmiPjC7NXe31p7eZeBlDnOufC0WLkrUouIz0bEd7bWbu0ykBLHOQ8+ESu/TXE+Ig5ExHe11v6y10ZOvcz86oj4k4j4SEQsz579g7HyveE/RcQ5EXF3RFzbWvvGHhupcbxz4cjflje7Ncyu1tqdp2zHlCMMAAAAwFRM+e5IAAAAAJMhwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFDg/we/bcIIO4A4fAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1440x1440 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cosine = cosine_similarity(vectors) # We can calculate cosine using these binary vectors\n",
    "\n",
    "print(cosine)\n",
    "\n",
    "fig2 = plt.figure(figsize = (20,20)) # create a 5 x 5 figure \n",
    "ax = fig2.add_subplot(111)\n",
    "ax.imshow(cosine, interpolation='none')\n",
    "ax.set_title('Cosine similarity for documents')\n",
    "ax.yaxis.set_major_locator(MaxNLocator(integer=True))\n",
    "ax.xaxis.set_major_locator(MaxNLocator(integer=True))\n",
    "\n",
    "plt.show() #Plot the cosine similarity. Remember, the diagonal is 1 and the matrix is symmetric"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "47178105",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Conway is a town in Franklin County, Massachusetts, United States. The population was 1,897 at the 2010 census. It is part of the Springfield, Massachusetts Metropolitan Statistical Area.,Conway was first settled in 1762 as the southwest portion of Deerfield. The town was eventually separated and was officially incorporated in 1767. The town was named for General Henry Seymour Conway,  a leader in the British House of Commons during repeal of the Stamp Act. (Conway, New Hampshire, as well as other towns across the country, were also named for him.) The town was known for its sheep farming and other agrarian pursuits in its early years, with some industry along the South River which was washed out in a dam break in 1869.  Today the town is still mostly a farming community.,On February 25, 2017, an EF1 tornado touched down in Conway. The tornado caused significant damage to localized areas. It was the first tornado in the month of February in Massachusetts state history.,<w namespace=\"File\" title=\"Bardwell's Ferry Bridge viewed from north in 2014.jpg\">Historic Bardwell's Ferry Bridge</w>,According to the United States Census Bureau, the town has a total area of convert, of which convert is land and convert, or 0.47%, is water.  Conway is in the central part of the county, and is bordered by Shelburne to the north, Deerfield to the east, Whately to the southeast, Williamsburg to the south, Goshen to the southwest, Ashfield to the west, and Buckland to the northwest. Conway is convert southwest of Greenfield, convert north-northwest of Springfield, and convert west-northwest of Boston.,Conway lies south of the Deerfield River, which comprises most of its northern border. The Bardwell's Ferry Bridge connects Conway to Shelburne across the Deerfield River. The town lies along the South River, which flows from Ashfield to the Deerfield River through the town, with many brooks feeding it as well as the nearby Mill River through Deerfield.  The town is dotted with forests and hills, and is home to several state forests, including Conway State Forest, South River State Forest, and the Poland Brook State Wildlife Management Area.,Massachusetts Route 116 passes through the center of town, from Ashfield in the east towards Deerfield, along its way towards Springfield.  Deerfield also has the nearest exits of Interstate 91, the nearest interstate highway to the town. A stretch of freight rail passes through the northern part of town, along the banks of the Deerfield River, between Deerfield and the Hoosac Tunnel to the west.  The nearest passenger rail service can be reached at the Amtrak station in Amherst, with the nearest bus service in Deerfield. The nearest small airport is in Turners Falls, with the nearest national air service being at Bradley International Airport in Windsor Locks, Connecticut.,As of the census of 2000, there were 1,809 people, 692 households, and 512 families residing in the town.  By population, Conway ranked twelfth of the twenty-six towns in Franklin County, and 295th of the 351 cities and towns in Massachusetts.  The population density was 48.0 people per square mile (18.5/km²), which ranked sixteenth in the county and 312th in the Commonwealth.  There were 749 housing units at an average density of 19.9 per square mile (7.7/km²).  The racial makeup of the town was 98.73% White, 0.17% African American, 0.22% Native American, 0.50% Asian, 0.11% from other races, and 0.28% from two or more races. Hispanic or Latino of any race were 0.94% of the population.,There were 692 households out of which 36.3% had children under the age of 18 living with them, 61.4% were married couples living together, 9.0% had a female householder with no husband present, and 25.9% were non-families. 18.8% of all households were made up of individuals and 5.2% had someone living alone who was 65 years of age or older.  The average household size was 2.61 and the average family size was 3.01.,In the town, the population was spread out with 25.2% under the age of 18, 5.3% from 18 to 24, 28.9% from 25 to 44, 31.2% from 45 to 64, and 9.5% who were 65 years of age or older.  The median age was 40 years. For every 100 females there were 97.9 males.  For every 100 females age 18 and over, there were 96.2 males.,The median income for a household in the town was $56,094, and the median income for a family was $62,917. Males had a median income of $41,146 versus $32,273 for females. The per capita income for the town was $25,605.  About 2.5% of families and 3.5% of the population were below the poverty line, including 2.6% of those under age 18 and 4.0% of those age 65 or over.,Conway employs the open town meeting form of government, and is led by a board of selectmen and a Town Administrator.  The town offices, police station, fire station, post office and Field Memorial Library (donated to the town by Marshall Field and named for his family) are all located in the center of town.  The nearest hospital, Franklin Medical Center, is located in Greenfield, as are most of the nearest state offices.,On the state level, Conway is represented in the Massachusetts House of Representatives by the First Franklin district, which includes the southeastern third of Franklin County and towns in north central Hampshire County. In the Massachusetts Senate, the town is represented by the Berkshire, Hampshire and Franklin district, which includes all of Berkshire County, as well as much of western Franklin and Hampshire Counties.   The town is patrolled by the Second (Shelburne Falls) Barracks of Troop \"B\" of the Massachusetts State Police.,On the national level, Conway is represented in the United States House of Representatives as part of Massachusetts's 1st congressional district, and is represented by House Rep Richard E. Neal (D) since 2012. Massachusetts is currently represented in the United States Senate by Senators Ed Markey and Elizabeth Warren.,Conway is a member of Frontier Regional and Union 38 School Districts, which also includes Deerfield, Whately and Sunderland. Each town operates its own elementary school, with Conway Grammar School serving the town's students from kindergarten through sixth grades. All four towns send seventh through twelfth grade students to Frontier Regional High School in Deerfield. There are several private, religious and charter schools located in the Greenfield and Deerfield area, with the nearest being the Deerfield Academy.,The nearest community college, Greenfield Community College, is located in Greenfield.  The nearest state colleges are Massachusetts College of Liberal Arts in North Adams, and the nearest state universities are Westfield State University and the University of Massachusetts Amherst. The nearest private colleges, including members of the Five Colleges and Seven Sisters, are located southeast in the Northampton area.\n",
      "\n",
      "Harlan is a home rule-class city in and the county seat of Harlan County, Kentucky, United States. The population was 1,745 at the 2010 census, down from 2,081 at the 2000 census.,Harlan is one of three Kentucky county seats to share its name with its county, the others being Greenup and Henderson.,<w namespace=\"File\" title=\"Harlan County Kentucky Courthouse.jpg\">The fifth and present courthouse of Harlan, Kentucky, built from 1918 to 1922</w>,Harlan was first settled by Samuel and Chloe Howard in 1796. Upon the founding of Harlan County (named for Kentucky pioneer Silas Harlan) in 1819, the Howards donated convert of land to serve as the county seat. Greene, James III. The Kentucky Encyclopedia, p. 408. \"Harlan\". University Press of Kentucky (Lexington), 1992. Accessed 30 July 2013.  The community there was already known as \"Mount Pleasant\", apparently owing to a nearby Indian mound. A post office was established on September 19, 1828, but called \"Harlan Court House\" due to another Mt. Pleasant preempting that name. During the Civil War, Confederate raiders under Gen. Humphrey Marshall occupied the town; the local postmaster renamed the community \"Spurlock\" after himself;  and, in October 1863, the courthouse was burnt down in reprisal for the Union destruction of the courthouse in Lee County, Virginia.  In 1865, the post office was renamed \"Harlan\" and, although the community was formally incorporated by the state assembly as \"Mount Pleasant\" on April 15, 1884, Commonwealth of Kentucky. Office of the Secretary of State. Land Office. \"Harlan, Kentucky\". Accessed 29 July 2013.  the town was already usually called \"Harlan Court House\" or \"Harlan Town\" by its inhabitants.  The city's terms of incorporation were amended to change the name to \"Harlan\" on March 13, 1912. One year before, the Louisville and Nashville Railroad had arrived in Harlan and prompted massive growth. The city had initially expanded east along the Clover Fork; after World War II, it also expanded south along Martin's Fork.,Harlan is the site of a criminal case in which a man, Condy Dabney, was convicted in 1924 of murdering a person who was later found alive.,A flood in 1977 prompted federal aid that diverted the Clover Fork into man-made tunnels under Ivy Hill in 1989.  In the 1990s, a flood wall was completed on the city's west side along the four-lane bypass U.S. Route 421.,<w namespace=\"File\" title=\"Overlooking Flood Wall and Highway 421.jpg\">A view of U.S. Route 421 from downtown; the flood wall and Harlan Independent Schools' athletic fields are also visible.</w>,Harlan is located in west-central Harlan County at coord (36.841487, -83.320066), at the junction of the Clover Fork and Martin's Fork rivers. The Clover Fork continues north convert to join the Poor Fork, forming the Cumberland River, a major tributary of the Ohio River. Harlan is in a narrow mountain valley, constrained to the north by the western end of Black Mountain, to the south by Little Black Mountain, and to the west by Ewing Spur. The elevation at the Harlan Courthouse is convert above sea level, while the surrounding ridges rise outside the city limits to convert (Black Mountain), convert (Little Black Mountain), and convert (Ewing Spur).,U.S. Route 421 passes through the city as four-lane highway; it leads north convert to Hyden and southeast convert to Pennington Gap, Virginia. The closest city with a population greater than 10,000 is Middlesboro, Kentucky, convert to the southeast via U.S. Routes 119 and 25E.,According to the United States Census Bureau, the city of Harlan has a total area of convert, of which convert, or 2.45%, are water.,The climate in this area is characterized by hot, humid summers and generally mild to cool winters.  According to the Köppen Climate Classification system, Harlan has a humid subtropical climate, abbreviated \"Cfa\" on climate maps.,As of the census of 2000, there were 2,081 people, 926 households, and 550 families residing in the city. The population density was 1,187.4 people per square mile (459.1/km²). There were 1,060 housing units at an average density of 604.8 per square mile (2.339/km²). The racial makeup of the city was 91.01% White, 7.02% Black or African American, 0.29% Native American, 0.86% Asian, and 0.82% from two or more races. Hispanics or Latinos of any race were 0.62% of the population.,<w namespace=\"File\" title=\"Harlan Poke Sallet 2015.jpg\">Downtown Harlan during the annual Poke Sallet Festival</w>,There were 926 households out of which 24.0% had children under the age of 18 living with them, 41.3% were married couples living together, 14.4% had a female householder with no husband present, and 40.6% were non-families. 39.0% of all households were made up of individuals and 19.4% had someone living alone who was 65 years of age or older. The average household size was 2.17 and the average family size was 2.91.,In the city, the population was spread out with 21.3% under the age of 18, 8.1% from 18 to 24, 25.8% from 25 to 44, 25.1% from 45 to 64, and 19.7% who were 65 years of age or older. The median age was 42 years. For every 100 females there were 89.4 males. For every 100 females age 18 and over, there were 81.8 males.,The median income for a household in the city was $17,270, and the median income for a family was $29,135. Males had a median income of $37,500 versus $20,852 for females. The per capita income for the city was $15,572. About 23.8% of families and 32.4% of the population were below the poverty line, including 40.0% of those under age 18 and 20.6% of those age 65 or over.,<w namespace=\"File\" title=\"Overlooking East Central Street.jpg\">Overlooking East Central Street. Harlan High School and Middle School are visible on the left.</w>,Two school districts, the Harlan County Public Schools and the Harlan Independent Schools, are based in the city. The independent schools, whose district roughly coincides with the city limits of Harlan, feature Harlan Elementary, Harlan Middle, and Harlan High.,Harlan County High School, which opened in 2008 as the consolidation of the county district's three previous high schools (James A. Cawood, Evarts, and Cumberland), serves all other public high school students in the county.  Harlan also features a campus of Southeast Kentucky Community and Technical College.,The Harlan Daily Enterprise newspaper is published Monday through Saturday. Radio stations serving Harlan are WHLN (adult contemporary, 1410 AM), WFSR (gospel, 970 AM) and WTUK (country, 105.1 FM).,Some storylines of the FX Networks drama Justified take place in Harlan, although no scenes have been filmed there. The 2012–13 National Geographic series Kentucky Justice which followed the Harlan County Sheriff's Office in their daily duties was filmed in Harlan and Harlan County.,Harlan was also featured in the Darrell Scott song \"You'll Never Leave Harlan Alive\" (later performed by Brad Paisley and Patty Loveless), the Steve Earle song \"Harlan Man\", the Anna McGarrigle song \"Goin' Back to Harlan\" (notably covered by Emmylou Harris) and the song \"Harlan County Line\" from Dave Alvin.,* Karl Spillman Forester, federal judge for the United States District Court for the Eastern District of Kentucky,* James E. Keller, former justice of the Kentucky Supreme Court,* Cawood Ledford, University of Kentucky basketball and football announcer,* Jordan Smith, musician and winner of Season 9 of The Voice\n",
      "\n",
      "Streetsboro is a city in Portage County, Ohio, United States. It is formed from the former township of Streetsboro, which was formed from the Connecticut Western Reserve. It is nearly co-extant with the former Streetsboro Township; the village of Sugar Bush Knolls was also formed in part from a small portion of the former township. The population was 12,311 at the 2000 census, and 16,028 at the 2010 census.  It is part of the Akron Metropolitan Statistical Area.,According to the United States Census Bureau, the city has a total area of convert, of which convert is land and convert is water. citation,Long before settlers moved into the Connecticut Western Reserve, Seneca Indians traversed the area now called Streetsboro. They used Old Portage Trail, which crosses the southwest corner of the city, to go from Lake Erie to the Ohio River basin.,The founder of Streetsboro Township was Titus Street from Connecticut, who purchased the land in 1798. Streetsboro Township contained convert. The land was surveyed by Streets' agents Ralph Cowles and Lemuel Punderson in the summer of 1822 and divided into lots of convert.,In 1825, a turnpike from Cleveland to Wellsville, Ohio was laid by Frederick Wadsworth, Samuel Cowles, John Strauyhen and Titus Street. Street agreed to give enough land to make it through the township. In 1827, the turnpike was completed.  It followed much of the route of the present Ohio State Route 14.,Major growth occurred in Streetsboro with the opening of the Ohio Turnpike on October 1, 1955, with Exit 13 (now known as Exit 187) initially being the only interchange in Portage County. By 1957, with the growth of the automobile industry, Streetsboro experienced a population explosion. Proximity to the Akron and Cleveland areas, along with direct access to Interstate 80 (Ohio Turnpike), Interstate 480 and State Routes 303, 43 and 14, have greatly contributed to the rapid growth of manufacturing, distribution, commercial and residential development in the past fifty years.,In 1968, voters decided to merge the township and the village to become one city consisting of convert. Streetsboro was primarily a farming community until 1970.,Streetsboro was home to a small amusement park, Shady Lake Park, located on Route 14, which is now an apartment complex of the same name.,Due to the steady growth of Streetsboro and its surrounding area within the past ten years, the city has become a retail hub for Portage County with the arrival of many national big-box retailers including Wal-Mart Supercenter, Target, Lowe's, The Home Depot and Staples.  The first Northeast Ohio location for Sonic Drive-In opened in Streetsboro in September 2008.,As of the census of 2010, there were 16,028 people, 6,562 households, and 4,316 families residing in the city. The population density was convert. There were 7,104 housing units at an average density of convert. The racial makeup of the city was 87.7% White, 7.9% African American, 0.2% Native American, 2.2% Asian, 0.3% from other races, and 1.8% from two or more races. Hispanic or Latino of any race were 1.7% of the population.,There were 6,562 households of which 31.4% had children under the age of 18 living with them, 50.3% were married couples living together, 11.1% had a female householder with no husband present, 4.4% had a male householder with no wife present, and 34.2% were non-families. 26.5% of all households were made up of individuals and 6.8% had someone living alone who was 65 years of age or older. The average household size was 2.43 and the average family size was 2.97.,The median age in the city was 37.9 years. 22.3% of residents were under the age of 18; 8.3% were between the ages of 18 and 24; 30.5% were from 25 to 44; 27% were from 45 to 64; and 11.7% were 65 years of age or older. The gender makeup of the city was 48.5% male and 51.5% female.,As of the census  of 2000, there were 12,311 people, 4,908 households, and 3,381 families residing in the city. The population density was 512.6 people per square mile (197.9/km²). There were 5,244 housing units at an average density of 218.3 per square mile (84.3/km²). The racial makeup of the city was 95.39% White, 1.96% African American, 0.09% Native American, 1.37% Asian, 0.02% Pacific Islander, 0.15% from other races, and 1.02% from two or more races. Hispanic or Latino of any race were 0.78% of the population.,There were 4,908 households out of which 31.1% had children under the age of 18 living with them, 55.3% were married couples living together, 9.9% had a female householder with no husband present, and 31.1% were non-families. 25.1% of all households were made up of individuals and 5.7% had someone living alone who was 65 years of age or older. The average household size was 2.49 and the average family size was 3.00.,In the city the population was spread out with 24.3% under the age of 18, 8.3% from 18 to 24, 35.8% from 25 to 44, 21.8% from 45 to 64, and 9.8% who were 65 years of age or older. The median age was 34 years. For every 100 females there were 96.7 males. For every 100 females age 18 and over, there were 93.6 males.,The median income for a household in the city was $48,661, and the median income for a family was $55,814. Males had a median income of $36,672 versus $27,835 for females. The per capita income for the city was $21,764. About 3.9% of families and 5.3% of the population were below the poverty line, including 7.5% of those under age 18 and 6.1% of those age 65 or over.,Primary and secondary education is provided mainly by the Streetsboro City School District, which includes five schools, while a small portion of the city is part of the neighboring Kent City School District.  Students in preschool through third grade attend Streetsboro Elementary School, formerly known as Campus Elementary School. Henry Defer Intermediate School houses students in grades four through six with grades seven and eight at Streetsboro Middle School. Streetsboro High School includes grades nine through 12. All schools and the district offices, with the exception of Streetsboro High  School, are located on a convert central campus with the Streetsboro City Park. The campus also includes the Pierce Streetsboro Library, a branch of the Portage County District Library, which opened in 1988.,Voters in the district approved a $38.7 million bond issue in 2013 to fund expansion and renovations of the district's facilities.  Included in the plans were the new building for Streetsboro High School along State Route 14, and an expansion and renovation of the former Campus Elementary to become Streetsboro Elementary School for grades preschool through three. Both of those projects were completed in December 2016 and the first classes held in January 2017.  Additional plans include renovation of the former high school building to house Streetsboro Middle School, which will enable grade six to be moved to the middle school from Henry Defer Intermediate School, and repurposing the current middle school building.,* WSTB 88.9 FM (Streetsboro High School-Alternative Rock Monday through Saturday, Oldies on Sunday),* Gateway News (Weekly newspaper serving Streetsboro, Mantua Village and Township, and Shalersville Township)\n"
     ]
    }
   ],
   "source": [
    "#Look at content of potentially interesting documents\n",
    "a = sample.columns.get_loc('text')\n",
    "\n",
    "print(sample.iloc[13, a])\n",
    "print()\n",
    "print(sample.iloc[17, a])\n",
    "print()\n",
    "print(sample.iloc[18, a])\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "56fbff61",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAleklEQVR4nO3de4xc130f8O/v3nnt7Jv7EMklKVISZUVyJdmlmQhp0TSIYdoJygQNIDltiiIJJBVVGxctHPWPBij6l5OgSFErkAVXMNAWlQzESYiUqRI0DzeI0pKyqQclUWZoiVxS5C65r9mdmZ3H/fWP2aFmZ+dx78y99zzu7wNQ2pm5u3N2duY7Z84953eImSGEEMJ8juoGCCGECIcEuhBCWEICXQghLCGBLoQQlpBAF0IIS6RU3fHs7CwfPXpU1d0LIYSR3njjjdvMPNfpNmWBfvToUZw/f17V3QshhJGI6KNut8mQixBCWEICXQghLOEr0InoFBFdIqLLRPR8l2N+goguENFFIvqLcJsphBCin75j6ETkAngBwOcBLAI4R0RnmPndlmOmAPwOgFPMfJWI5iNqrxBCiC789NBPArjMzFeYuQLgFQCn2475BQDfYearAMDMS+E2UwghRD9+An0BwLWWy4s717V6EMA0Ef05Eb1BRP+k0w8ioqeJ6DwRnV9eXh6sxUIIITryE+jU4br2Eo0pAH8bwE8D+AKAf0dED+75JuaXmPkEM5+Ym+s4jVIIIcSA/MxDXwRwuOXyIQA3Ohxzm5m3AGwR0XcBPAbgg1BaOYAnv/H60D/j1WeeCKElQggRDz899HMAjhPRMSLKAHgKwJm2Y/4AwN8lohQR5QH8KID3wm1qeC7eWMfb19chpeCFEDbp20Nn5hoRPQfgNQAugJeZ+SIRPbtz+4vM/B4R/S8AbwHwAHyTmd+JsuH99Opdn/76X6JUreNbv/Q55DPKFssKIUSofKUZM58FcLbtuhfbLv8mgN8Mr2nRaXbM65500YUQ9kjkStHmUIsEuhDCJskM9J0+ek0CXQhhkUQGOqSHLoSwUCIDvRnj0kMXQtgkmYG+M4heq3uKWyKEEOFJaKA3/l+tSw9dCGGPRAZ6s19elR66EMIiiQt0z+O7Qy4S6EIImyQu0CstIV6pSaALIeyRuEDfrkmgCyHslMBAr9/9utzytRBCmC55gV79pFfuedJLF0LYI3GBXqrWe14WQghTJS/QK7sDvCyBLoSwROICvdgW6O2XhRDCVIkKdM9jlKq1Xddtbde6HC2EEGZJVKCXqnV4bedANyXQhRCWSFSgF8p7w7tYqcGTqotCCAskLNCre67zPGCzIr10IYT5EhXoGx0CHQA2Sp2vF0IIkyQm0D2PsVHq3BNfl0AXQlggMYFe2K513XJuvSiBLoQwX2ICfa1Y6XpbsVKXBUZCCOMlJtBXtroHOgCs9gh8IYQwQSIC3fMYa32GVe5sSqALIcyWiEBfLVa6jp83rWxV7u5kJIQQJkpEoN/pM9wCNMrobnRYeCSEEKbwFehEdIqILhHRZSJ6vsPtP0FE60R0Yeffr4ff1MEtF7ZDPU4IIXSU6ncAEbkAXgDweQCLAM4R0Rlmfrft0P/DzD8TQRuHUihX95TM7Wa5sI0H5scibpEQQkTDTw/9JIDLzHyFmSsAXgFwOtpmhSdIr3tru4ailAEQQhjKT6AvALjWcnlx57p2TxDRm0T0R0T0SKcfRERPE9F5Ijq/vLw8QHODWwo4jLK0IcMuQggz+Ql06nBd+3SQ7wG4l5kfA/CfAfx+px/EzC8x8wlmPjE3NxeooYMoVmrYDHii89ZGOaLWCCFEtPwE+iKAwy2XDwG40XoAM28w8+bO12cBpIloNrRWDujWAL3tQlmGXYQQZvIT6OcAHCeiY0SUAfAUgDOtBxDRfiKina9P7vzcO2E3NqhBe9uDvBEIIYRqfWe5MHONiJ4D8BoAF8DLzHyRiJ7duf1FAD8P4J8RUQ1ACcBTrHiVztZ28OGWplsbZRybHQ25RUIIEa2+gQ7cHUY523bdiy1ffx3A18Nt2nCGGQvf3Bl2yWd8PTxCCKEFa1eKDjtsIsMuQgjTWBnom9s1bA25+bPMdhFCmMbKQF8KIYw3ZbaLEMIwdgZ6SDVZZNhFCGES6wJ9kMVE3UixLiGESawL9DCX7m+UqrI1nRDCGNYF+u3NcHvV0ksXQpjCqkCv1Ly+W80FtRzyG4QQQkTFqkC/sxV++K752L5OCCF0YFegR7DRs+c19hsVQgjdWRPozOxr79BBRNHzF0KIsFkT6BvlGqo1L5KfvRJBz18IIcJmTaCvRjgsUqzUZfqiEEJ71gT6SjHaXrSMowshdGdFoHseYz3k6YrtViN+wxBCiGFZEeiF7VrkUwujfsMQQohhWRHocYRtsVLHdk3G0YUQ+rIi0NdK8QyHSC9dCKEzKwJ9vRRP0MZ1P0IIMQjjA71crWO7Gs3883YS6EIInRm/C/JGOb6QLZRrYGYQUWz3GZYnv/H6UN//6jNPhNQSIURUjO+hb5Ti2yau7jE2h9yrVEdvX1/HO9fXVTdDCDEk6aEHVCjXMJ5Lx3qfYejVw/7Sf/ouiEh64UIYzvgeeiGk7eZ0vb84SHFgIexgdKCXq/XICnJ1U4j5E0FcWFJdCOMZHegqessFC8fQpYsuhB2MDnQVJyjrdUapIitGhRD68RXoRHSKiC4R0WUier7HcZ8jojoR/Xx4TexuS1FvubBt57CLEMJsfQOdiFwALwD4IoCHAXyZiB7uctzXALwWdiO7UTWFcGvbsh66edPqhRAd+OmhnwRwmZmvMHMFwCsATnc47l8A+F0ASyG2rytmRrGiKtAtHEcXQhjPT6AvALjWcnlx57q7iGgBwM8BeLHXDyKip4noPBGdX15eDtrWXUrVOrx4J7jcVbRwDN3Axa9CiDZ+Ar3TS719XsRvA/g1Zu6ZdMz8EjOfYOYTc3NzPpvYmcphjy1FnwyiQjLmIoQV/KwUXQRwuOXyIQA32o45AeCVnRonswC+REQ1Zv79MBrZicqZJvU6Y7tWRzblKmtD2CTShTCfn0A/B+A4ER0DcB3AUwB+ofUAZj7W/JqIvgXgD6MMc6Ax5KJSueJZE+gy3CKEHfoGOjPXiOg5NGavuABeZuaLRPTszu09x82jojrQS9U6JmFeTZduJNSFMJ+v4lzMfBbA2bbrOgY5M//T4ZvVn+rFParfUMJELf8VQpjL2JWiqvf3VH3/oSLpoQthAyMDve4xanW1BUjKMe2SFAeZ5SKEHYwMdB16x5WYqzxGiSADLkLYwMhAr9bUlwe0KdAlzYWwg5GBXqmrD9OqBm0Ik4n7pAohdjMy0HUI07rH8Dz1nxTCIFEuhB2MDPS6JkFa06QdQggBGBrougSpLm8sQggBGBrougRpXTbiFEJoxMhA9zQJUl3aIYQQgKGBrkuOsvpzs6Hglv8KIczlq5aLbliT8NGlHUNjiXMhbGBkD12Ei+15axIi0YzsoYtw6TKEJYQYjpGB7miyqtGW1ZUMSKoLYQFDA73/Mb/x2vtdb7u2Uux7DAB89QsPDd0OE3jMiaq4+OQ3Xu96m8cM5kY54V4dh1efeSKKpgkxFCPH0HXpGevySWFYHgMsPXQAQLXOuHhjHRdvbKhuihCBGdlDT/noGvfqXTd75v164P24FnTR6x6DmeElqIfeq3f94e0tPP1fzyPjOtILF8YxsoeuS5Dq0o5hlHe20mOwFnXmhRCDMzLQU44ezfbzSUF3xZa9WVXv0yqEGI4eyRhQ2lUfpCmXtBnLH8bWdu2TryXQhTCamYGeUt/sjAZtCMNmS6Bvlms9jkwGOTUsTGZkKmVc9c3OWhLoG6Xq3a8L5WqPI4UQujNylks25YBI7VqYjOuqu/OQVGrerjH0jXIVnsdwLDg3MKhmBU3pqSdTrzUKfqmcHWVkoBMRMikH21V15Q5zafN76KvFyq7LngeslarYN5pR1CL1bNlWUPi3uV3DD5e3AOw+p9TJlduN4+6bHe16zNuL6wAai9MePjARawfJyEAHgFzaVRzo5vfQb29ud7wuyYHe3A1LFlolx6WbBaxuNTo3X/mpB3se21zD0uu4Wxvlu1/nMy7umxsLoZX++OpmEtEpIrpERJeJ6PkOt58moreI6AIRnSeivxN+U3cbURyopge65zGWC3sDfWlj73VJ0tyAXPI8GZY2ynfDPAof3SneXesRh76BTkQugBcAfBHAwwC+TEQPtx32vwE8xsyPA/glAN8MuZ17jGTUBqrq+x/W7a1t1Op7U6tcrWOtGN0TXHeVWiPQZeTFfp7H+MHSZqT3UfcYlyO+j1Z+eugnAVxm5ivMXAHwCoDTrQcw8yZ/8hl1FDGcU8orDFQiIG94D/36aqnrbYs9brNdeWcYj8F3w13YaXG1FMtiupvrZWzENIPMT6AvALjWcnlx57pdiOjniOh9AP8TjV76HkT09M6QzPnl5eVB2ntXPqNu+H8k7Ro9E2Rru4Y7m9174UuFcqwfE3XhebvLH5QS+BgkRa3u4Yd3tmK7vyvL8dyXn0DvlFx7euDM/HvM/BCAnwXwHzr9IGZ+iZlPMPOJubm5QA1tN5ZVF+ijCu87DB/2eSJ73iclhpNkq1LbNXbeb8aDMNf1tRKqMX4Cu13YjqWX7ifQFwEcbrl8CMCNbgcz83cB3E9Es0O2rSfXIWXDLmM5cwO9WKnh5nq573GLq6XEFesqtK2Ubb8s7OB5jKsKOiwf3Y7+Pv0E+jkAx4noGBFlADwF4EzrAUT0AO0UNiGizwLIALgTdmPbjefSUd9Fl/s1N9CvLG/5msFR9xgfxvAE1MlacXcPqn2evrDD8ua2kinPcQxl9g10Zq4BeA7AawDeA/BtZr5IRM8S0bM7h/1DAO8Q0QU0ZsQ8yTFM5FUVrBOK3kiGtV6s+uqdNy2uFhM17LDSNn1ts1xL3KeUJLi+puakPzMCvf4G4SsRmfksgLNt173Y8vXXAHwt3Kb1NzkSf7BmUo6Rc9CZGR8sFQJ+D/DBrQI+c2Q6olbpY71U7dh7Wi5s49B0XkGLRBQqNS/Seef93Noo42iPVabDMnr9+sRIGnFXsFXxJhKGj9fLWC8GPylzZ7PScQGSbbr1nKLuUYl4rWxVlC4aK5RrkQ67GB3orkOxj6NP581bFl+te0MtoPjgVgF1i1fa1OoePl7v/DF8rViVKpQW0eG8yHopuueT0YEOANP5eAN9atS8HvrfLG8ONUWrVKn3nepossXVUsdVs01JOzlsMx1mLkXZQTB3usaO6dEMProTzwsu5RLGDZuDvlGu9lwV6tdHd7ZwcHLE+JIH7ap1Dx/1mcJ2a6OMo+W8sllVUWstGVute76WeaccgtMy3mnKhtrFivpAL0a4OtWCHnoGcW0xOjOaNW7buQ9uFkIZM/S8xtCLbT68veXr04uNv3u7Ss3D5nYNWy3/3v94A+9/vLHruq3t2q6drkzBzD0/icWlWQAuCmZ1NztwHcJUPoOVHkvZwzIzZtb4+a2N8p651cNYLmxjZatiTXndQrnqe4HJ6lYVH6+XcGByJOJWxe/VZ57AeqmKNz5agdeWNc1ysV/9wkN7vm//ZA6fXpiMo4mh0OU0UJTtML6HDgBzY9lY7sekQPciqvL2g1sFK2qFex7j3RsbgT69XLpZsLLGTblax5vX1vaEeT8318u4shxfJcFh6VJ+Kcp2WBHoszEE+mQ+jWzKnPHj62vRVJIrlGu4ZUHN9Cu3NwOfIKvVGe9+vGHFG1pTpebhe1dXB64seWV5S9lCnaCISI8N5iPcvlL9bxeCkYwb+arRuD4FhMHzONJZKVdubxodarc3tweeubKyWcGHMZ2Ej1q17uH7V1dR3B7ujf+9Gxu7dunRmcqy201RTiywItABYH4iF+nPvyfinx+mG+ulSGtVFLfrxi42KlXqeOf6+lA/48ryJu502L7PJLW6hwvX1kKbxvfO9XUsFfQPdR3qME1E2AZrAv2eieh60BMjaWOm6zEzrsbQg+w31U9HtbqHNxfXhp7pwAy8fX1diylwg6h7jAvX1gZaOdwNcyPUdX+j36fBwsCpCNug/u0qJPlMChMjaWxEsAprv0G98ztblUjnuTatF6tYL1WNKYXAzHjnxgY2Q+qR1uqNUPzc0X1Iu+b0izyP8ebiWqiznz752cDb19fw+OFpbWdC7RttTHPudgK4Oaunm+Y+Ab2O6zQjqGkqn0YmwnF8c56JPkQRvETAfIS9/7DdiPEEVRgLluLywa1N3A6591jcruOtxTV4usyH64OZ8fb19Uin+Hoe8Oa1tUiXtw8j5TqxTKLoJuqhW2t66ABwz2QWP1gKZyFN0/RoxpjqitW6h9sxju0uFcp4yBvXfju+q3eKke3AtLpVxbsfbxgxH/uDW5uxDIk0h3ROHt2n5VDlwtQIlrrM1OrVuwZ6z8vvx3UI+yejDXSreujZlIuZkN99Dxq0kGSpsB14LvEwanXG7S29x0yXCuXIV3neXC/jbzSfj31jrRTrtoLVWuN8hY6fXmbGskq2kTwwlYt8eM6qQAeAgyG+A7ouYW7cnOGWJQVTx7r1dHRQKFdx8fpGLPf1w+UtbafulSp1XLoZf+mCzXJN2ze6o7Px1rgnAu7dF10d9CbrAn12LBva4oH9Ezm4mg8nNNU9VlIa9M5WRcs56dW6h7cW12Mt+/vujQ0ta5z8YEld+eOrK3ruerV/Iod8Nr7hoINT8RS2sy7QHYdCOzlq0nDLWrES63BLU7XmoaDhC/bSzUIkK2V7qXuMd66vazXMUKrUlX6KYoaSDZn7ISI8MD8Wy325DuFYhLsUtbIu0IHGWNWw8lkXkzHXWh/GagTT0Pxa29JrRsPKVkXZTkOb5ZpWAabDYp8lTeemz4/nMBXDa/zITD62iRVWBvpELo2xIVdjmdQ7B6LdBaWftZL6XWBaqS4Y9eGdLW12eNJhQ4dqzdO2qNmD+8cj/fnZtIOjM/H0zgFLAx0YPpCjnl4UJmbGhsJt0nQIjaZytR7JopkganXGisKNiFt5mpzf0KUd7SZyaRyciq7zdnx+PNbzcNYG+vxEduANpE2aew4A5aqHusLC/aVKPdKi/UHoclJSl3boUiE0o/Fq2vvnR+G64YfuVD4de8fQqoVFrXJpF1P5DFYH6CmZ1DsH9AiP4nYdk3l9X7Stol7erZPJkTSu9Tmm1+/p57EAej8eY7kUUhoHejbl4v7ZsdDXK0Q9nNOJvo9yCAYJZiJg3qC55wBin83RSbGq/k0FAMY02fNVl3bMjmWUT701oVLpoemRUEvrHpjKYULBHrR6POsiMjeWxXsBv2d6NGNUsSUAKGlwwkmHNxWg8cls31jvLQmjXN4NAJmUgxlNilOlXAcHpnJYXOled6fX7znsY+E6hIMhzDqLmuM0pjG+tThcaWWg8TvfPxfPlMh2ZiVXQJmUE3hakkkbWTTpMIOgHGH99aAemB8b+PxJGO6fH9Oqvs3RmVFlvfRD0yPajOP3Mz+RC2Wq8uF98U1TbGd1oAMIXNtFZSW2QelwQlKHNjRN5NI4GtNCjnaz41ksRDhrYhC5tIt7Z+Jd6g40OlSq/g6Dum/I9rouKXmsm3wFOhGdIqJLRHSZiJ7vcPs/IqK3dv79FRE9Fn5TBxOkLnM+42pZHa6fQfeDDJNOgQ40Xphx1+EZzabwyMGJWO/Tr6Mzo7EudQeAB+8ZN274cmYsi4khavwfnh5R+jv3HUMnIhfACwA+D2ARwDkiOsPM77Yc9kMAf4+ZV4noiwBeAvCjUTQ4qIlcCq5Lvqb1RbmTSJRqPhaxDDuTod8YalXhtMlOiAifXpjE96+uxjIvPZd28ZkjU9oGmOMQHj4wgfMfrsZyf3PjWeNmizXdO5PH2wOMpRMBh6bV9c4Bfz30kwAuM/MVZq4AeAXA6dYDmPmvmLn5TPlrAIfCbebgiMj32WaTlvq30mHRho4FulyH8Njhqcj3kcykHHz23int1y5M5TOxDAekUw4eOhD/lL2wzI1lB9pVaG48q/w54OeZvgDsmsq6iN69718G8EedbiCipwE8DQBHjhzx2cThTY6kfM1Hj3Lz1ij5idIoZzL4bYMKadfBZ45M442PViOp+pdOOfjsvdPIZ8x47tw/N4bbm5VIKyD+yP5xY06EduLsbEQRdG/eAxqUC/HzNtTp9HjH1y8R/X00Av3XOt3OzC8x8wlmPjE3N+e/lUPyU8yeCBg15EXZTof5FDq0oZtmDzrMecZA4wTY44entJlz7ofjED69MAEnopGh/ZM5zBsw77yfoGtRXJe0mKrq58+6COBwy+VDAG60H0REjwL4JoDTzHwnnOaFw0/vKZd2tZpqFkQqqldnAKoXr/STTbl4/MhUaLXyiYBHFyaN2SS71XguHck86ZGMi4cUrI6MwuRIGqkA5QD25TNa5IefZ/c5AMeJ6BgRZQA8BeBM6wFEdATAdwD8IjN/EH4zh5NL9/81VY99DSPKXcRNakM/+UwKjy5MhjJH/fj8eOjbHcbpyL48pkPsURIBjxyc0HqJfxBEFGiSxLQmEyr6PvrMXAPwHIDXALwH4NvMfJGIniWiZ3cO+3UAMwB+h4guENH5yFo8gIzr9H0RZw0IpG5GNHgzMmW65/RoZui50bPjWRxRONc4DES0E8Dh9CrvnRk1dpZYN0FOpkd94t0vX61g5rMAzrZd92LL178C4FfCbVp4iAgp10G1x3xtXaeb+TGWS+FWPFtndmXS+YdjM6O4tV5GcYByBY4Da4YVcmkXD+2fwDvXh1vuPp5LDb0gR0dBntO6dGjMTbGAUn3Gt3QfA+5Fh3HcOHZ+CYvjEI7NDRZAC1PqlnVHoXESc/ChI8cBHlmY1GL8OGx+P7UT6fMJX49WxKDfkIvJz8epkbTSN6RMyjFqpgcA3DOeG2i44dC0+qlpYfvU/vGBh16Ozowa97f3y+9jknIdkMriQS2SE+h9Jtbp8gcZhOMQZsbUjV/OjWeNe/wchzAzGqxnms+4vqbAmiabcgfaMDmfdWPdXi1ufjtJOnUGExPo/VYy6rDachgqFzWYtv9q08RIsHAepsaH7hamRgLvw3t8ftzKoZYmv5GgU3QkJtD7lTvRcel6ELNjGSUnZsZzKWNLJgSdHWTT2Hk7IsJ9Ac4rTObTsRc/i5ufGkkAUNcoOxIT6DWvdzVAv388XRERjuyLfyqdaeVRWwWdM52OYN9JncyNZX2vplXxXIub3wqi9TrD0yQ/EhHozIx6nwe8plm1wEEsTI3E2osczaaM266vVdDRgn7nYUxHRL4qJKZcMnIjmKC2A5SlDnJslBIR6DWP+45z6VbPexCOE+xj87AaOwOZG3IafVLWhp9hlJnRrNVj501BtlXUYRtIICGB7iesddgkIgwHJnOxrFrbN5Yxfgw1aJ6ztjUlwzOWbewf0ItJaw6GUaz4r0gZZfXKIBIR6H7CumJBDx1ofGz+VMQrGYkau9GYLuiJ8CT06Imo7wpJG6dudrIZIKS3AoR/lCTQd9gw5NI0lc9EulvM4X15KxaTOAGHi4Ieb6p+xez8FLsznedxoCGXzbIEemz8nLDwPHuGXQDg+D1jfT86DyKTcnDM4JktrYKujozi8dRRv3LMOpRrjlphuxboE1lBhlzi4/cMtC3DLkBj9d/9s+HXvH5gfszoQmatgu6qo0u9jqj1WyFpct0jv4IMtwCNqYtBxtyjkohnqN+et009dKBRdyTMXXomRtI4YOjGv51kUk6gXrpJFSWH0W9kyf44Bwrl4BuL6zDskoxA99nzti3QHYfwwD3h9dIfvMfsaYqd+K1UmUk52pRIjVq/DngSpiwWBgjnDQn0ePg94WnTidGm+fFcKNPMZsez1m1gAAD7fO7a4/c4G/R6007A8DmYOfCQCxB8mCYKCfjzoOfGFq1sGkNvdV8I+0fGuWApTn7n0s8mYGVkk9sr0C37hNZJueqhPsDKcR3moici0P3WabFh+X8n+0YzQ1UKnBnLYCJn52KSfCbVdyGW6xBmFZYnjluv8wpJmOEy6JzyUqXet8RI1Oz/6wC+H2TVf4woDVNMyfZCTAenepf/nZ/IWrP5sR+ZHr+rCZuBDyvI/PM936u4BID1fx0/hbmabA70+fEs0gO8GEcyrvXjx/sncz3Hhhf6BL5tek3nTMLUzWFCeZg3gzBYPw8ryOIAm2t1OA5hfjyL66ulQN93z0TOupktT37j9T3XFSv1uyfFr600HqPf+uNLcIg6Dsm8+swT0TZSoVyme2gnYabPMLPdVJ+Hsz/Qgxxrb54DwECBPswGwiZpzNdvhNWjhybVNkaxbMqF61LHE4NBNwUx0TChrHrqs/WBblffcjjT+Qwcp1HmwI90ysG4BTVb2tncuw5LPu2iUN97cjDMhWq6GmZyRN3viysi1g+IBRktsH1KluMQxgPMVpkcSVs33CL86VZRMQmVFofZX1j1abgEBDr5XgyRgBlZgaok2lBRUQymU3C7Llm9r6oNEhBh/nveSSg6FKQeSRI+XovORjv87ZNSy2aYT+qqIyQRge63OmASFk0EmUechClqorN8hx56Ut7gg5ZVbuUqzhBf905Ep4joEhFdJqLnO9z+EBG9TkTbRPRvwm/mcPwGeq8FFbYI8mRN0mIasVun2SxJmLIIDJcDqhde9f0MRUQugBcAfB7AIoBzRHSGmd9tOWwFwL8E8LNRNHJYfnuaqv8YcehVp2PPsao/PwplXIeQbduZKCk99GF2ZMopzhA/934SwGVmvsLMFQCvADjdegAzLzHzOQDBiwjHwG9QJ2GIIcjwoMR5srWfAM0F3BDEVMOc+FX9KcZPgi0AuNZyeXHnusCI6GkiOk9E55eXlwf5EQPx+wdKwhn8YCtnRZK1B3gSXh/A4LO7HEf9wis/gd6pozbQa52ZX2LmE8x8Ym5ubpAfMRA/DzJRMnroQebYDjMfV5ivfcglCUOSwOCBPpZVv27Dz19oEcDhlsuHANyIpjnR8BPo2ZSbiJ1Y6kECXfUqCaFU68lB16XEnFNJuc5AC6gmRtRP6/QT6OcAHCeiY0SUAfAUgDPRNitcvYoNNake+4pLkE635HmytVbnzCZsxpPfrQlbTY2or0ra9y2FmWtE9ByA19CoXvQyM18komd3bn+RiPYDOA9gAoBHRF8B8DAzb0TXdP+yKRcpl3rWaEjKGfxggS6JnmTplimug5ReNlGzEmel7nXcgejaShEA8Buvvb/ntsmRNBwipbWCfH1GYOazAM62Xfdiy9c30RiK0VY+k8JGqfsknKSsgrO5RLAIV7plkUwqIcMtTRnXgduh7tEjB7tX4tShFlQyUgyNHnivQM9nk9FDpwCTEdU/PYVKrYvQ/C7OM53plTiT8VdC/ypxSemhB1mZnJSTYKKz1hAfZjm8iE9yAr3HGLnjDLc6zCTpAIkuS/+TbVegJ6DOkQ0S81fqVGzo7m2ZlPL5o3Fpn1vc89iEnAgTnbnOJwN0aemhGyEZ4wxo7MBC1HmWR1KGW4DG6r9uj0OrlEuJGTdNuk57rDZd29my8F+9eqHn88H0sWdbJCbJHIcwknZR7LArd1JOiAI7j0PGRXG79+7ksrmFAIC/tZDs/VVNk6hXbT6b6hjoSeqhA8BELt030INsVSfMJr1reyTqM3W3xUNJWSXa5GcV3CAr5YQQaiUq0LvVdEnKKtGmqXz/sPZzjBBCL4kK9E7BnU45iTv5N5ZN9VzKnc+4iSmVKoRNEpVk+Q5j5UnrnQMAEWFmtHshoZmxbIytEUKEJVFnA3NpZ8+OPaoL0selfWpaa/Gh9oJDY9nUnk8tcuJMCP0lKtCJaO+2WgkJ9HatxYfaCw7Jkn8hzJSoQAf2LvFPypJ/6WELYb9kpFmLbEL3SRRC2M/aHnq35czlah3XVhrLmX/rjy9hPJfqWsdYerVCCJNYG+jd5NIuHj0ky5mFEPaxNtCldy2ESJrEjaELIYStJNCFEMISEuhCCGEJCXQhhLCEBLoQQlhCAl0IISwhgS6EEJaQQBdCCEsQ99v+Pao7JloG8JGSO2+YBXBb4f3rRh6P3eTx+IQ8FrupfjzuZea5TjcoC3TViOg8M59Q3Q5dyOOxmzwen5DHYjedHw8ZchFCCEtIoAshhCWSHOgvqW6AZuTx2E0ej0/IY7Gbto9HYsfQhRDCNknuoQshhFUk0IUQwhKJDHQiOkVEl4joMhE9r7o9KhHRy0S0RETvqG6LakR0mIj+jIjeI6KLRPSrqtukEhHliOj/EdGbO4/Hv1fdJtWIyCWi7xPRH6puSyeJC3QicgG8AOCLAB4G8GUielhtq5T6FoBTqhuhiRqAf83MPwLgxwD884Q/N7YB/CQzPwbgcQCniOjH1DZJuV8F8J7qRnSTuEAHcBLAZWa+wswVAK8AOK24Tcow83cBrKhuhw6Y+WNm/t7O1wU0XrgLalulDjds7lxM7/xL7CwKIjoE4KcBfFN1W7pJYqAvALjWcnkRCX7Ris6I6CiAzwD4v4qbotTOEMMFAEsA/oSZk/x4/DaArwLwFLejqyQGOnW4LrG9DrEXEY0B+F0AX2HmDdXtUYmZ68z8OIBDAE4S0acVN0kJIvoZAEvM/IbqtvSSxEBfBHC45fIhADcUtUVohojSaIT5f2fm76hujy6YeQ3AnyO551t+HMA/IKIP0Rim/Uki+m9qm7RXEgP9HIDjRHSMiDIAngJwRnGbhAaIiAD8FwDvMfN/VN0e1Yhojoimdr4eAfBTAN5X2ihFmPnfMvMhZj6KRmb8KTP/Y8XN2iNxgc7MNQDPAXgNjZNe32bmi2pbpQ4R/Q8ArwP4FBEtEtEvq26TQj8O4BfR6H1d2Pn3JdWNUugAgD8jorfQ6Aj9CTNrOV1PNMjSfyGEsETieuhCCGErCXQhhLCEBLoQQlhCAl0IISwhgS6EEJaQQBdCCEtIoAshhCX+P0xYnmUtL5vXAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#This block plots violin plots of the cosine similarity as a function of distance\n",
    "classes = 5 #Change this value to modify the number of classes we plot\n",
    "\n",
    "m1 = np.min(dists)\n",
    "m2 = np.max(dists)\n",
    "step = (m2 - m1)/classes+1\n",
    "boxes = {}\n",
    "\n",
    "rows = len(cosine)\n",
    "cols = len(cosine[0])\n",
    "for i in range(rows):\n",
    "    for j in range(cols):\n",
    "        k = (cols * i) + j\n",
    "        if i is not j:\n",
    "            dist = dists[i][j]\n",
    "            #print(dist)\n",
    "            cos = cosine[i][j]\n",
    "            box = int(dist/step)            \n",
    "            if box in boxes.keys():\n",
    "                boxes[box].append(cos)\n",
    "            else:\n",
    "                boxes[box] = [cos]\n",
    "   \n",
    "labels, data = sorted(boxes.keys()), boxes.values()\n",
    "\n",
    "plt.violinplot(data, showmedians=True)\n",
    "plt.xticks(range(1, len(labels) + 1), labels)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07c9b229",
   "metadata": {},
   "source": [
    "Compare documents using word2vec. We can filter the documents so only some terms are used. Important to remember that this is still a bag of words approach."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "11cb825e",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABGEAAARuCAYAAABzxirnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABGjUlEQVR4nO3de7idd13n/e+32UnapmnTNj03PXEobQMUCBXkKDBDrYIHRgZEQH2kOCMDCOOo6DUizzPjoygHZxgcoBxUwBGpPDAgClosOBSbYgutpfRAMT0lPaWHNGlOv+ePvTpmYtLc907y/bl2X6/rytXstfbe9yfZd3d23rnX2tlaCwAAAAD2rwN6DwAAAAB4JBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAUywzr8rM5xYe71mZec0c3/akzLw/MxdMXv5SZv7MXmzZZ7/2zDwmMy/OzPsy83f2xfvc6f2fkpktM2f29fsGAKaHLwQAoEhm/nhEvCkiHhcR90XE5RHxn1prX5nr+2ytnbVv1g0+3pcj4vQ5vu0/RMQh+3DL//61Z+ZbI+LRrbWfmOO7Oz8i7oiIQ1trbR/Mm9cy88MRcVNr7Vd7bwGAaeJKGAAokJlvioh3RcR/johjIuKkiPhvEfFDHWdNpf10NcnJEfH3cwkwrm4BAIYSYQBgP8vMwyLibRHxc621C1trG1prW1prn2mt/cLkdRZn5rsy85bJj3dl5uLJfcsz839m5vrMvCszv5yZB0zuuzEzXzD5+Vsz848z8/cnD6u5KjNX7bDj+Mz8ZGbenpnfyczXP8zm8zLz7yfv5+bM/PeT25+bmTft8Ho3ZuYvZOY3MnNDZl4weWjPn03e9ouZefjkdXf7kJzMfFRm/lVm3pmZd2TmRzNz2U7H+cXM/EZEbMjMmYd+7Zl5bkS8JSL+9eThTldk5o9l5mU7HePNmfmpXRz7wxHx6oj4D5O3f8EePh7PzcybJntui4gP7eJ9LsjM3578Wm6IiB/Y6f7jM/PTk4/ndZn5mp3e9i2Zef3k9/CyzFyxq9+/HR/SlZk/mZl/k5nvnJwrN2Tm905uX5OZ6zLz1Tu87eLJxn/IzLWZ+XuZedBOv8Y3T97u1sz8qcl950fEK3b4/frM5PZfnJwr92XmNZn5/J1/XwDgkU6EAYD97+kRcWBE/OnDvM6vRMTTIuLsiHhiRJwTEQ891OPNEXFTRBwVs1fRvCUidnfFxosj4o8iYllEfDoi/mtExCTafCYiroiIEyLi+RHxxsx84W7ezwUR8drW2tKIWBkRf/Uw218SEf8iIh4bES+KiD+bbFwes19r7Db27CAj4jci4viIOCMiVkTEW3d6nZfHbMxY1lrb+tCNrbXPx+wVRv+jtXZIa+2Jk1/7qZl5xg5v/xMR8Qc7H7i19pMR8dGI+K3J238xHv7jERFxbEQcEbNX0Jy/i1/PayLiByPiSRGxKiL+1U73fzxmP6bHT+77zztEizdNfq3nRcShEfHTEfHALo6xK98TEd+IiCMj4mMxey48NSIeHbO//v+amQ89JOw3Y/Zjdvbk/hMi4j/u9Gs8bHL7/xUR78nMw1tr74v/8/frRZl5ekS8LiKeOjlnXhgRNw7cDACPGCIMAOx/R0bEHTuGg114RUS8rbW2rrV2e0T8ekS8cnLflog4LiJOnlxB8+WHedjMV1prn2utbYvZ4PDEye1PjYijWmtva61tbq3dEBHvj4iX7eb9bImIMzPz0Nba3a21rz/M9v/SWlvbWrs5Ir4cEV9rrf1da+3BmA1PT3qYt42IiNbada21L7TWHpz8+t8REc/Z6dV+t7W2prW2ccD7ezAi/kfMhofIzLMi4pSI+J97etuJh/t4RERsj4hfm+zd1Z6XRsS7JnvvitnAFJMtKyLimRHxi621Ta21yyPiAzu8/5+JiF9trV3TZl3RWrtz4O7vtNY+NPn4/4+YjVlvm+z8i4jYHBGPzsyM2VD08621u1pr98VsyNrxfNgyedstrbXPRcT9sfvnA9oWEYtj9pxZ2Fq7sbV2/cDNAPCIIcIAwP53Z0Qs39XDcHZwfER8d4eXvzu5LSLi7RFxXUT8xeQhJr/0MO/nth1+/kBEHDg57skRcfzkYSrrM3N9zF6tcsxu3s9LYvZKjO9m5l9n5tMf5phrd/j5xl28vMcn483MozPzjyYPZ7k3Iv4wZq+k2dGaPb2fnXwkIn58EhxeGRF/PIkzQzzcxyMi4vbW2qY9vP2Oe7+7030PhY8d7z9h8vMVETHXgLHz73201nb18TgqIg6OiMt2OB8+P7n9IXfuFA4fiN18LFtr10XEG2P26qV1k4/l8bt6XQB4JBNhAGD/+2pEbIqIH36Y17klZkPJQ06a3Battftaa29urZ0Wsw/3edMcnm9jTcxeJbFshx9LW2vn7eqVW2uXttZ+KCKOjohPRcQfjzzeWL8Rsw+xekJr7dCYvYIld571MG//T+5rrV0Ss1d+PCsifjx28VCkh7Hbj8eALRERt8ZsTNnx7Xd830dk5tKd7r958vM1EfGoXbzPDZP/HrzDbcfuYcfu3BGzQeasHc6Hw1prQ7971a5+vz/WWntmzP6+tZh9uBMAsAMRBgD2s9baPTH7XBvvycwfzsyDM3NhZn5/Zv7W5NU+HhG/mplHZebyyev/YUREZv5gZj70EJJ7Y/ahH9tGzvjbiLh38uSpB02e/HVlZj5151fMzEWZ+YrMPKy1tmWHY+5PS2P24S7rM/OEiPiFkW+/NiJOmTz3zY5+P2afF2fryG8FvtuPx0B/HBGvz8wTc/aJif/31UuttTUR8b8i4jcy88DMfELMPufKRyev8oGI+L8z8zE56wmZeeTkYVE3R8RPTD5+Px27jjV71FrbHrMPR3tnZh4dEZGZJzzMcwTtbG1EnPbQC5l5emY+L2efvHhTzAae/X3OAMDUEWEAoEBr7R0x+4SrvxoRt8fs1Q6vi9mrTCIi/p+IWB2zT6r6zYj4+uS2iIjHRMQXYzZSfDUi/ltr7Usjj78tZq+iOTsivhOzV0J8IGafeHVXXhkRN04eGvSzMXlulf3o1yPiyRFxT0R8NiIuHPn2n5j8987M3PH5a/4gZp9YeMxVMBEP//EY4v0R8ecx+0TIX49/+ut5ecw+R80tMfu8Ob/WWvvC5L53xGzE+YuYDWAXRMRBk/teE7OB6s6IOCtmY85c/WLMPsztksnH+Yux++d82dkFMfv8L+tz9jtOLY6I/zdmz6vbYvYKqrfsxTYAmJdy98/rBwAw3SbfcnldRDy5tXZt7z0AwCObK2EAgPns30TEpQIMAPDPwcN9lwYAgKmVmTfG7JP7/nDfJQAAszwcCQAAAKCAhyMBAAAAFBBhAAAAAAqUPifM8iMWtFNWLKw85Jx98+6jek8YZcmSTb0nDLZxy3ScAw85aOGW3hMG27p9urrqdO3N3gNGmTlgW+8Jozy4aXo+L+TWKTsXNvZeMFweNT2fbyMitk3R57CDZzb3njDKfQ8c2HvCvHXAwu29Jwy2fft0fb494ABP87C/bN+8oPeEwRYsnq6vwRYcMD2fE7bcPz1fL25Zf1ds27Bhl5/ESiPMKSsWxt/++YrKQ87ZaZ/42d4TRjln1bd7TxjsyrXH9Z4wyspjbu09YbD1Dx7Ue8Iot29Y0nvCYNun6C9bERHLD9nQe8Io1107PZ8XDrxtup7T/vCrp+eLq8U/Mz2fbyMi1j8wPZ9zzz7m5t4TRvnS353Re8Jw09UJYsnR0/Pnw6aNi3pPGOXAg6Yndk7b04Ju+oelvScMduhp63tPGOWIJQ/0njDYTZec0HvCYGve887d3jddf6sAAAAAmFIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAF9irCZOa5mXlNZl6Xmb+0r0YBAAAAzDdzjjCZuSAi3hMR3x8RZ0bEyzPzzH01DAAAAGA+2ZsrYc6JiOtaaze01jZHxB9FxA/tm1kAAAAA88veRJgTImLNDi/fNLnt/5CZ52fm6sxcffud2/bicAAAAADTa28iTO7itvZPbmjtfa21Va21VUcduWAvDgcAAAAwvfYmwtwUESt2ePnEiLhl7+YAAAAAzE97E2EujYjHZOapmbkoIl4WEZ/eN7MAAAAA5peZub5ha21rZr4uIv48IhZExAdba1fts2UAAAAA88icI0xERGvtcxHxuX20BQAAAGDe2puHIwEAAAAwkAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAjOVB/vm3UfFaZ/42cpDztkNP/Z7vSeM8vpbntp7wmDbtx/fe8IoRyx6oPeEwZ6x7PreE0Y588Cbek8Y7O3fPbf3hFFedOw3ek8Y5d3rjug9YbBXfO/FvSeM8udvfU7vCYP9zElf6T1hlLVbDus9YbDP3ray94RRfuN5n+g9YbCL7z2994RRfvjwy3pPGGzFzD29J4zyok+8ufeEwT7ykvf0njDKLx/ykt4TBtvesveEUZ6+/Du9Jwz2U6/8WO8Jg/3oH92x2/tcCQMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFJipPNiSJZvinFXfrjzknL3+lqf2njDK7x5/ae8Jg/3E5iW9J4zy7476q94T5q3btx/ce8Jg33Pkjb0njHLW4pt7Txjl5Weu7j1hsC/f/ujeE0bJba33hMGeduB3e08Y5Z33Pr/3hMHedtqnek8Y5fVXvaz3hMFecvIVvSeMcte2Q3pPGOyTd63qPWGUI868o/eEwdZsObL3hFG+56gbe08YbOO2Rb0njPLSZdPz98hXXPWTvScMdsOmD+/2PlfCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAArMVB5s45aFceXa4yoPOWfbtx/fe8IoP7F5Se8Jg/3hKV/qPWGUf3vz83pPGOy2jUt7Txjl8EUbe08YbHtk7wmjvHXti3tPGOX2+6bnc1isPqz3glGO2rql94TBfu76f917wijXfGc6vqaJiHhwZemXfHvtrjum58+zT2x/Uu8Joxyx5IHeEwa78bYje08Y5Uknr+k9YbC/XH9m7wmjXHT9Y3pPGGzlCbf2njDKB+54Vu8Jg627dnnvCYNt3bT7P3ddCQMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoMOcIk5kHZubfZuYVmXlVZv76vhwGAAAAMJ/M7MXbPhgRz2ut3Z+ZCyPiK5n5Z621S/bRNgAAAIB5Y84RprXWIuL+yYsLJz/avhgFAAAAMN/s1XPCZOaCzLw8ItZFxBdaa1/bJ6sAAAAA5pm9ijCttW2ttbMj4sSIOCczV+78Opl5fmauzszVW+99YG8OBwAAADC19sl3R2qtrY+IL0XEubu4732ttVWttVUzhx68Lw4HAAAAMHX25rsjHZWZyyY/PygiXhAR39pHuwAAAADmlb357kjHRcRHMnNBzMacP26t/c99MwsAAABgftmb7470jYh40j7cAgAAADBv7ZPnhAEAAADg4YkwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABSYqTzYQQu3xMpjbq085JwdseiB3hNG+XdH/VXvCYP925uf13vCKP/thEt6Txjssgc3954wym3bDu09YbA1m4/sPWGU5yy7pveEUT5/58reEwZb85yNvSeMcue2Y3tPGOwPTr2w94RR3nvIc3tPGOwHDr+i94RRrj7umN4TBnv5SZf2njBvXb70pN4TRrn2nqN6TxjspcdM13l78so7e08Y7OiF9/aeMMqzDrqu94TBvnLqab0nDLZu8dbd3udKGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACgwEzlwbZuPyDWP3hQ5SHn7BnLru89Yd66bePS3hNGuezBzb0nDPaUxYt6Txjl6s139Z4w2MX3Pa73hFGesuQ7vSeMMpPbe08YbOURt/aeMMoVNx/de8JgJ85s7T1hlJVLbuk9YbBFua33hFFOOXR6/nw4ZdHtvSeMcuQBG3pPGGx7m65/L15924reEwY7duae3hNGuX3m0N4TBrt24zG9J4xy1uKbek8YbOVR0/M12JqZLbu9b7o+swEAAABMKREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACgwEzlwbZuPyBu37Ck8pBzduaBN/WeMMrt2w/uPWGwwxdt7D1hlNu2Hdp7wmBXb76r94RRzlg0Peft0gWbek8Y5YHti3tPGOWm+5f1njDY2UdO158PCx9ovScMds/26dkaEXHYgg29Jwx229bDek8Y5cxDbu09YbD126bja9uHPH7RHb0nDHb8wrt7Txjl3vsO6j1hsBMW3N97wihf2To9XzMeveje3hNGuW/79Jy3j12yrveEwS46YOtu73MlDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKDAXkWYzFyWmX+Smd/KzKsz8+n7ahgAAADAfDKzl2//7oj4fGvtX2Xmoog4eB9sAgAAAJh35hxhMvPQiHh2RPxkRERrbXNEbN43swAAAADml715ONJpEXF7RHwoM/8uMz+QmUt2fqXMPD8zV2fm6q33PLAXhwMAAACYXnsTYWYi4skR8d7W2pMiYkNE/NLOr9Rae19rbVVrbdXMYR6tBAAAADwy7U2EuSkibmqtfW3y8p/EbJQBAAAAYCdzjjCttdsiYk1mnj656fkR8ff7ZBUAAADAPLO33x3p30XERyffGemGiPipvZ8EAAAAMP/sVYRprV0eEav2zRQAAACA+WtvnhMGAAAAgIFEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKzNQeLmP79unoPm//7rm9J4zyPUfe2HvCYNsje08YZc3mI3tPGOzi+x7Xe8IoSxds6j1hsF9d/q3eE0Z5zZpn9J4wytp7lvaeMNinr3lK7wmjrNjSek8Y7N23f1/vCaNs3Lao94TBjli0ofeEUa6595jeEwa7dvHRvSeMcvmGk3pPGOx/rTu194RRDjxoc+8Jg31+wxm9J4yyev30nLcvPvqK3hNG+eamE3tPGOzj107P12B3PXjZbu+bjiICAAAAMOVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIzpQc7YFssP2RD5SHn7EXHfqP3hFHOWnxz7wmDvXXti3tPGOU5y67pPWGwpyz5Tu8JozywfXHvCYO9Zs0zek8Y5f0r/qb3hFHevuS23hMG+9yylb0njLLtL4/pPWGwFy77Zu8Jo3z6rif1njDYow5c13vCKNffv7z3hMGOXnxf7wmjLF84PXtfcNz0fA0WEfHpGx/fe8Jg29t0/Vv8Adl6TxjsAzc+s/eEUV536kW9Jwz2zBU39J4w2B2LHtztfdP1fx8AAADAlBJhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVmKg/24KaFcd21x1Uecs7eve6I3hNGefmZq3tPGOz2+5b0njDK5+9c2XvCYDO5vfeEUW66f1nvCYOtvWdp7wmjvH3Jbb0njPILR1zfe8Jg7//Mv+w9YZRTP/XV3hMGu/CXVvWeMMpFl57Ve8Jgd549XX/2XnnRY3pPGGzTsxf2njDKmvVn9J4w2H03H9p7wiiHnHBv7wmDfej6p/WeMMrihVt7TxjstjXT9ffIi454XO8Jg33pC2f3njDYffdetNv7XAkDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQYK8iTGa+ITOvzMyrMvON+2gTAAAAwLwz5wiTmSsj4jURcU5EPDEifjAzH7OvhgEAAADMJ3tzJcwZEXFJa+2B1trWiPjriPiRfTMLAAAAYH7ZmwhzZUQ8OzOPzMyDI+K8iFixb2YBAAAAzC8zc33D1trVmfmbEfGFiLg/Iq6IiK07v15mnh8R50dELDhi2VwPBwAAADDV9uqJeVtrF7TWntxae3ZE3BUR1+7idd7XWlvVWlu14JAle3M4AAAAgKk15ythIiIy8+jW2rrMPCkifjQinr5vZgEAAADML3sVYSLik5l5ZERsiYifa63dvQ82AQAAAMw7exVhWmvP2ldDAAAAAOazvXpOGAAAAACGEWEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKDBTebDcmnHgbaWHnLNXfO/FvSeM8uXbH917wnCrD+u9YJQ1z9nYe8JgK4+4tfeEUc4+8qbeEwb79DVP6T1hlM8tW9l7wijv/8y/7D1hsG+/+r29J4zyA7/3ot4TBjt76Td6TxjlwadMx9c0ERFfveHU3hNGefQz1vSeMNja+5f2njDKqx/9td4TBjvgMdt7Txjl47/5/b0nDHbqa6/pPWGUr685sfeE4bZn7wWj3Lpxev5u9pLz/qb3hME+8of37/Y+V8IAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAWytVZ2sEOOWNFWvvCNZcfbGwu21P2+7Au5bXr2HrB1erZGRNx51sLeEwZbcvP23hNGWfjA9JwLB0zZ54Q203vBOAd96m97Txhs5uQVvSeM8tmvfqb3hMGe8YbX9p4wypKbN/WeMNg9jzq494RRDrpja+8Jgy28b0vvCfPWxmMX954wzhR9qfDAUdP1b/HHfHV97wmD3Xv6Yb0njLJ9Qe8Fwy397sbeEwb728vfG/fef3Pu6r7p+r8PAAAAYEqJMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVmKg+WR22JxT9za+Uh5+xnTvpK7wmjPO3A7/aeMNjPXf+ve08Y5Q9OvbD3hMFOnNnae8Io92xvvScM9u7bv6/3hFFeuOybvSeMcuEvreo9YbCzl36j94RRnvGG1/aeMNjfvPu/954wyrvuPqX3hMFOW7Su94RRPrbuab0nDPZjR63uPWGUFQvv7D1hsMs3ndx7wii/840X9J4w2Puf+vu9J4zyzU0rek8Y7JoHju09YZQfP+KS3hMG+9Adz+w9YbArXrVpt/e5EgYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQYI8RJjM/mJnrMvPKHW57e2Z+KzO/kZl/mpnL9utKAAAAgCk35EqYD0fEuTvd9oWIWNlae0JEfDsifnkf7wIAAACYV/YYYVprF0fEXTvd9hetta2TFy+JiBP3wzYAAACAeWNfPCfMT0fEn+3uzsw8PzNXZ+bqLesf2AeHAwAAAJg+exVhMvNXImJrRHx0d6/TWntfa21Va23VwmUH783hAAAAAKbWzFzfMDNfHRE/GBHPb621fTcJAAAAYP6ZU4TJzHMj4hcj4jmtNY8xAgAAANiDId+i+uMR8dWIOD0zb8rM/ysi/mtELI2IL2Tm5Zn5e/t5JwAAAMBU2+OVMK21l+/i5gv2wxYAAACAeWtffHckAAAAAPZAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACgwEzlwbZtPyDWP3BQ5SHnbO2Ww3pPGOWd9z6/94TBrvnOcb0njPLeQ57be8JgK5fc0nvCKIct2NB7wmAbty3qPWGUT9/1pN4TRrno0rN6TxjswaeU/tG515bcvKn3hMHedfcpvSeM8sbDb+w9YbDzrjmv94RRfvbEL/WeMNjn1z+h94RRXnbkJb0nDLYgtveeMM51S3ovGOzGJy7vPWGUv77rsb0nDHbW0lt7TxjlT+95Su8Jg5120B29Jwy2OLfu9j5XwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKzFQe7OCZzXH2MTdXHnLOPnvbyt4TRnnbaZ/qPWGwB1eWnnZ77QcOv6L3hMEW5bbeE0a5bethvScMdsSiDb0njPKoA9f1njDKnWcv6T1hsK/ecGrvCaMsf9SBvScMdtqi6Tpvz7vmvN4TBvvc6Z/rPWGU07/8qt4TBnvZ4y7rPWGUC+9e1XvCYFetP673hFHOeOYNvScMduuWw3tPGGXJzObeEwa74p4Tek8Y5fnLv9V7wmDvuOwFvScMdvvG3f8d0pUwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAECBmcqD3ffAgfGlvzuj8pBz9hvP+0TvCaO8/qqX9Z4w2F13LO09YZSrjzum94TBTjn0rt4TRjnzkFt7Txjsmnun5zyIiLj+/uW9J4xy5UWP6T1hsEc/Y03vCaM8eMdxvScM9rF1T+s9YZSfPfFLvScMdvqXX9V7wijXPOv3e08Y7Puu+qHeE0Z59Yqv9p4w2FMP+U7vCaO87U9e2nvCYC/60St6Txjl0lvO6T1hsMcuX9d7wiifXfv43hMG+92nf7z3hMHetGT3fy9zJQwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAECBPUaYzPxgZq7LzCt3uO2tmXlzZl4++XHe/p0JAAAAMN2GXAnz4Yg4dxe3v7O1dvbkx+f27SwAAACA+WWPEaa1dnFE3FWwBQAAAGDe2pvnhHldZn5j8nClw/fZIgAAAIB5aK4R5r0R8aiIODsibo2I39ndK2bm+Zm5OjNXb7t/wxwPBwAAADDd5hRhWmtrW2vbWmvbI+L9EXHOw7zu+1prq1prqxYcsmSuOwEAAACm2pwiTGYet8OLPxIRV+7udQEAAACImNnTK2TmxyPiuRGxPDNviohfi4jnZubZEdEi4saIeO3+mwgAAAAw/fYYYVprL9/FzRfshy0AAAAA89befHckAAAAAAYSYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoMFN+xCw/4pxcfO/pvSeM8pKTr+g9YbBPbH9S7wmjvPykS3tPGOyURbf3njDK+m1Lek8Y7NrFR/eeMMrRi+/rPWGUTc9e2HvCYGvvX9p7wijL7tvSe8JgP3bU6t4TRvn8+if0njDYyx53We8Jo3zfVT/Ue8JgF531//WeMMrb73pU7wmDfXHtGb0njLL5mOn5fDttPvbkC3pPGOy3bjm394RRHrd0be8Jg731Wy/uPWGwWzZ9cLf3uRIGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACgwU3mwAxZujyVHb6g85Jz98OGX9Z4wyl3bDuk9YbAjljzQe8K8deQB0/H/10Mev+iO3hMGu3zDSb0njLJ84X29J4yyZv0ZvScM9upHf633hFH+LJ7de8JgKxbe2XvCKC878pLeEwa78O5VvSeM8uoVX+09YbC33/Wo3hNG+YUjru89YbAzD7y594RRXnfNq3pPGOwpB36394RRPnLn9/aeMNjjl07XeXvyFH09fsJpd/eeMNjbF+/+72WuhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUmKk82PbtGZs2Lqo85JytmLmn94RRPnnXqt4TBrvxtiN7Txjl8qUn9Z4w2PY2XV31+IV3954w2P9ad2rvCaO84Lhrek8Y5b6bD+09YbADHrO994RRNh67uPeEwS7fdHLvCaMsiOk5F65af1zvCaM89ZDv9J4w2BfXntF7wihnHnhz7wmD/cDBm3pPGOUNG6fn67BvbT6294RRPnPtyt4TBjtq2f29J4zyfcde23vCYK86/JLeEwb77wfs/vPX9HymAAAAAJhiIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAECBPUaYzPxgZq7LzCt3uO3szLwkMy/PzNWZec7+nQkAAAAw3YZcCfPhiDh3p9t+KyJ+vbV2dkT8x8nLAAAAAOzGHiNMa+3iiLhr55sj4tDJzw+LiFv28S4AAACAeWVmjm/3xoj488z87ZgNOd+7u1fMzPMj4vyIiAXLD5vj4QAAAACm21yfmPffRMTPt9ZWRMTPR8QFu3vF1tr7WmurWmurFixdMsfDAQAAAEy3uUaYV0fEhZOffyIiPDEvAAAAwMOYa4S5JSKeM/n58yLi2n0zBwAAAGB+2uNzwmTmxyPiuRGxPDNviohfi4jXRMS7M3MmIjbF5DlfAAAAANi1PUaY1trLd3PXU/bxFgAAAIB5a64PRwIAAABgBBEGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgALZWis72EGPPr6d+tvnlx1vb2y6/tDeE0Y54sw7ek8Y7KRD7+49YZQ7Nh7Se8Jg92w8sPeEUe6976DeEwY78KDNvSeMsmhmW+8Jo2xr2XvCYAdduKz3hFEWbtjee8Jgt790Y+8J41y3pPeCwc545g29J4xy9VdO6z1hsM3HbOk9YZzN0/NvsAs2Ts/WiIjrXv57vScM9tiLX9V7wihb103P14ztwOn5czciYvnx9/SeMNjGLy/vPWGw73z4HbHx1jW7/AJ3uj6zAQAAAEwpEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKDATPUBW6s+4tx85CXv6T1hlDVbjuw9YbC/XH9m7wmjvPSYS3tPGOzYmXt6TxjlhAX3954w2Oc3nNF7wijb23Q19g9d/7TeEwY79bXX9J4wyrf/4PTeEwZ7/1N/v/eEUW584vLeEwa7dcvhvSeM8qIfvaL3hHnrKQd+t/eEwb61+djeE0Z57MWv6j1hsG8/e7o+375l7RN6Txhs+cLp+fo2IuLERXf2njDY2lOX9Z4w2O98Zvd/L5uur9IBAAAAppQIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACgwU3mw7ZsXxKZ/WFp5yDn75UNe0nvCKN9z1I29Jwx20fWP6T1hlJNX3tl7wmC3zxzae8IoX9l6cO8Jg61ef1LvCaMckK33hFEWL9zae8JgX19zYu8Jozzqq+t7Txjsm5tW9J4wyl/f9djeEwZbMrO594RRLr3lnN4TBvvYky/oPWGUj9z5vb0nDPaZa1f2njDK1nUH9Z4w2FvWPqH3hFH+8zHf6D1hsNff8tTeE0a5esNxvScM9sUrz+g9YbC1939rt/e5EgYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKDAHiNMZq7IzIsy8+rMvCoz3zC5/ccmL2/PzFX7fyoAAADA9JoZ8DpbI+LNrbWvZ+bSiLgsM78QEVdGxI9GxH/fnwMBAAAA5oM9RpjW2q0Rcevk5/dl5tURcUJr7QsREZm5fxcCAAAAzAOjnhMmM0+JiCdFxNf2yxoAAACAeWpwhMnMQyLikxHxxtbavSPe7vzMXJ2Zq7fdv2EuGwEAAACm3qAIk5kLYzbAfLS1duGYA7TW3tdaW9VaW7XgkCVz2QgAAAAw9YZ8d6SMiAsi4urW2jv2/yQAAACA+WfId0d6RkS8MiK+mZmXT257S0Qsjoj/EhFHRcRnM/Py1toL98tKAAAAgCk35LsjfSUidvctkP50384BAAAAmJ9GfXckAAAAAOZGhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACgwEzlwRYs3haHnra+8pBztr1l7wmjbNy2qPeEwVaecGvvCaMcvfDe3hMGu3bjMb0njHL0oun5vX3x0Vf0njDKB258Zu8Jo9y25ojeE4bbPl1/Ptx7+sG9Jwx2zQPH9p4wyllLp+fPsyvuOaH3hFEeu3xd7wmD/dYt5/aeMMrjl97ce8JgRy27v/eEUW65d3HvCYMtXzhdv7evv+WpvScM9rvHX9p7wijvuOu03hMG+8uFp/eeMNzDXO7iShgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoMBM5cEWHLA9jljyQOUh5+zpy7/Te8IoL112ae8Jg33gjmf1njDKsw66rveEwc5afFPvCaPct/2g3hMG++amE3tPGOV1p17Ue8IoFx3xuN4TBrt142G9J4xy+1+f0nvCYD9+xCW9J4zyp/c8pfeEwZ6//Fu9J4zy2bWP7z1hsMctXdt7wignL7qj94TBvu/Ya3tPGOXPt5b+1WqvnLjozt4TRrl6w3G9Jwz2jrtO6z1hlDcdcUPvCYN96eTH9p4w2PpFm3d7nythAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAVmKg+25f6FcdMlJ1Qecs5+6pUf6z1hlFdc9ZO9Jwy27trlvSeM8pVTT+s9YbCVR93ae8Ioj12yrveEwT5+7VN6TxjlmStu6D1hlC994ezeEwZ7yXl/03vCKKu/e0zvCYN96I5n9p4wymkH3dF7wmDvuOwFvSeM8rtP/3jvCYO99Vsv7j1hlBNOu7v3hMFedfglvSeM8qmPP6v3hMHWnrqs94RRvnjlGb0nDPaXC0/vPWGUL5382N4TBvv0Yz7fe8Jg5yy+d7f3uRIGAAAAoIAIAwAAAFBAhAEAAAAoIMIAAAAAFBBhAAAAAAqIMAAAAAAFRBgAAACAAiIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUGCPESYzV2TmRZl5dWZelZlv2On+f5+ZLTOX77+ZAAAAANNtZsDrbI2IN7fWvp6ZSyPissz8Qmvt7zNzRUT8i4j4h/26EgAAAGDK7fFKmNbara21r09+fl9EXB0RJ0zufmdE/IeIaPttIQAAAMA8MOo5YTLzlIh4UkR8LTNfHBE3t9au2MPbnJ+ZqzNz9bYNG+a+FAAAAGCKDXk4UkREZOYhEfHJiHhjzD5E6Vci4l/u6e1aa++LiPdFRBx4wgpXzAAAAACPSIOuhMnMhTEbYD7aWrswIh4VEadGxBWZeWNEnBgRX8/MY/fXUAAAAIBptscrYTIzI+KCiLi6tfaOiIjW2jcj4ugdXufGiFjVWrtjP+0EAAAAmGpDroR5RkS8MiKel5mXT36ct593AQAAAMwre7wSprX2lYjIPbzOKftqEAAAAMB8NOq7IwEAAAAwNyIMAAAAQAERBgAAAKCACAMAAABQQIQBAAAAKCDCAAAAABQQYQAAAAAKiDAAAAAABUQYAAAAgAIiDAAAAEABEQYAAACggAgDAAAAUECEAQAAACggwgAAAAAUEGEAAAAACogwAAAAAAWytVZ3sMzbI+K7++FdL4+IO/bD+2W6OA94iHOBCOcB/8i5QITzgFnOAx7iXCBi/50HJ7fWjtrVHaURZn/JzNWttVW9d9CX84CHOBeIcB7wj5wLRDgPmOU84CHOBSL6nAcejgQAAABQQIQBAAAAKDBfIsz7eg/gnwXnAQ9xLhDhPOAfOReIcB4wy3nAQ5wLRHQ4D+bFc8IAAAAA/HM3X66EAQAAAPhnbaojTGaem5nXZOZ1mflLvffQR2YemJl/m5lXZOZVmfnrvTfRR2Yuy8w/ycxvZebVmfn03pvoIzPfkJlXTj4nvLH3Hmpk5gczc11mXrnDbW+ffE74Rmb+aWYu6ziRIrs5F96amTdn5uWTH+f13Mj+t5vz4OzMvGRyDqzOzHN6bmT/y8wVmXnR5GvDqzLzDZPbf2zy8vbM9F2SHgF2dy7scP+/z8yWmcv3546pjTCZuSAi3hMR3x8RZ0bEyzPzzL6r6OTBiHhea+2JEXF2RJybmU/rO4lO3h0Rn2+tPS4inhgRV3feQweZuTIiXhMR58TsefCDmfmYvqso8uGIOHen274QEStba0+IiG9HxC9Xj6KLD8c/PRciIt7ZWjt78uNzxZuo9+H4p+fBb0XEr7fWzo6I/zh5mflta0S8ubV2RkQ8LSJ+bvL3xisj4kcj4uKe4yi1u3MhMnNFRPyLiPiH/T1iaiNMzH5xfV1r7YbW2uaI+KOI+KHOm+igzbp/8uLCyQ9PdvQIk5mHRsSzI+KCiIjW2ubW2vquo+jljIi4pLX2QGtta0T8dUT8SOdNFGitXRwRd+10219MzoOIiEsi4sTyYZTb1bnAI89uzoMWEYdOfn5YRNxSOopyrbVbW2tfn/z8vpj9R7oTWmtXt9au6buOSrs7FyZ3vzMi/kMU/D1ymiPMCRGxZoeXb4p//A3kESYzF2Tm5RGxLiK+0Fr7WudJ1DstIm6PiA9l5t9l5gcyc0nvUXRxZUQ8OzOPzMyDI+K8iFjReRP/PPx0RPxZ7xF09brJQ9M+mJmH9x5DF2+MiLdn5pqI+O1wddwjSmaeEhFPigh/V3iE2/FcyMwXR8TNrbUrKo49zREmd3Gbqx8eoVpr2yaXlZ4YEedMHo7AI8tMRDw5It7bWntSRGyICM8V9QjUWrs6In4zZh+G8vmIuCJmLz/lESwzfyVmz4OP9t5CN++NiEfF7EOXb42I3+m6hl7+TUT8fGttRUT8fEyuoGX+y8xDIuKTEfHG1tq9vffQz47nQsx+bfArMfvwxBLTHGFuiv/zXzZPDJcTPuJNHn7ypdj148CZ326KiJt2uArqT2I2yvAI1Fq7oLX25Nbas2P2UvRre2+in8x8dUT8YES8orXmH2weoVprayf/aLM9It4fsw9t55Hn1RFx4eTnnwjnwSNCZi6M2b90f7S1duGeXp/5axfnwqMi4tSIuCIzb4zZrvD1zDx2f22Y5ghzaUQ8JjNPzcxFEfGyiPh05010kJlHPfTdLjLzoIh4QUR8q+soyrXWbouINZl5+uSm50fE33ecREeZefTkvyfF7JPufbzvInrJzHMj4hcj4sWttQd676GfzDxuhxd/JGYfusgjzy0R8ZzJz58XIv28l5kZs1c8Xd1ae0fvPfSzq3OhtfbN1trRrbVTWmunxOw/7D558neL/bNjmv9BaPKtBd8VEQsi4oOttf/UdxE9ZOYTIuIjMXseHBARf9xae1vfVfSQmWdHxAciYlFE3BARP9Vau7vrKLrIzC9HxJERsSUi3tRa+8vOkyiQmR+PiOdGxPKIWBsRvxazz/ewOCLunLzaJa21n+0ykDK7OReeG7MPRWoRcWNEvLa1dmuXgZTYzXlwTcx+N8WZiNgUEf+2tXZZr43sf5n5zIj4ckR8MyK2T25+S8z+2fBfIuKoiFgfEZe31l7YYyM1dncu7Pjd8iZXw6xqrd2x33ZMc4QBAAAAmBbT/HAkAAAAgKkhwgAAAAAUEGEAAAAACogwAAAAAAVEGAAAAIACIgwAAABAAREGAAAAoIAIAwAAAFDg/wdtHhsk+KkHsQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1440x1440 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Calculate similarity between documents using cosine similarity in Spacy (based on Word2Vec)\n",
    "cosine = [] \n",
    "#In this code we don't use all the words; we filter out entities, stop words and punctuation\n",
    "for doc1 in docs:\n",
    "    s1 = \" \".join([ent.text for ent in doc1 if not ent.ent_type_ if not ent.is_stop if not ent.is_punct]) \n",
    "    d1 = nlp(s1)    \n",
    "    sim = []\n",
    "    for doc2 in docs:\n",
    "        s2 = \" \".join([ent.text for ent in doc2 if not ent.ent_type_ if not ent.is_stop if not ent.is_punct])\n",
    "        d2 = nlp(s2)\n",
    "        value = d1.similarity(d2)\n",
    "        #value = doc1.similarity(doc2)\n",
    "        sim.append(value)\n",
    "    cosine.append(sim)\n",
    "    \n",
    "#Plot the similarity matrix again\n",
    "fig2 = plt.figure(figsize = (20,20)) # create a 5 x 5 figure \n",
    "ax = fig2.add_subplot(111)\n",
    "ax.imshow(cosine, interpolation='none')\n",
    "ax.set_title('Cosine similarity for documents')\n",
    "ax.yaxis.set_major_locator(MaxNLocator(integer=True))\n",
    "ax.xaxis.set_major_locator(MaxNLocator(integer=True))\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "c56d209a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "One of the towns of Castro is Castrolanda. Castrolanda is a colony that was founded by Dutch immigrants between 1951 and 1954. Its economy is based on agricultural production and dairy farming which is one of the most important in this region that is considered the largest dairy in Brazil due to their genetic quality. It has a considerable production of grain, mostly soy and beans. Also having one of the largest agricultural cooperatives in Brazil, which bears the name of the colony.\n",
      "\n",
      "Alfred is a village located in the town of Alfred in Allegany County, New York, United States. The population was 4,174 at the 2010 census. The village lies in the north-central part of the town of Alfred on the eastern edge of Allegany County. It is named after Alfred the Great.,Alfred is located in the hills of western New York along the Southern Tier. The layout of the valley is such that the lowest part contains the village, and upon the hills to the east and west are Alfred University and Alfred State College, respectively. The valley was originally called Kanakadea (or Canakadea, or other spellings) by Native Americans. Local folklore states that Kanakadea means \"where the earth meets the sky\".,Alfred is located in the 607 (telephone) area code, 14802 zip (postal) code, and is in the second poorest county in the state (Allegany). The closest cities are Hornell and Olean.,The elevation of Alfred is about convert but rises to a high point of convert at the summit of Jericho Hill just south of the village. The hills greatly affect the weather in the region, which results in quick changes as well as different conditions in neighboring valleys.,The village is home to one of the largest optical telescopes in New York state, at the Stull Observatory at Alfred University.   Alfred State College is also located in the village.,The village was first settled around 1807 and was incorporated in 1881. The village was previously called \"Alfred Centre\".,The following are listed on the National Register of Historic Places: Alfred Village Historic District, Alumni Hall, Fireman's Hall, Allen Steinheim Museum and the Terra Cotta Building. citation,According to the United States Census Bureau, the village has a total area of convert, all of it land. A small stream, Canacadea Creek, flows through the center of the village, then to the Canisteo River, part of the Susquehanna River watershed. The topography is steep hill country, part of the Allegheny Plateau, a physiographic region that is part of the Appalachian Mountains.,As of the census of 2000, there were 3,954 people, 530 households, and 169 families residing in the village. The population density was 3,352.5 people per square mile (1,293.8/km²). There were 576 housing units at an average density of 488.4 per square mile (188.5/km²). The racial makeup of the village was 90.21% White, 4.40% Black or African American, 0.33% Native American, 2.71% Asian, 0.03% Pacific Islander, 1.21% from other races, and 1.11% from two or more races. Hispanic or Latino of any race were 2.73% of the population.,There were 530 households out of which 12.3% had children under the age of 18 living with them, 26.0% were married couples living together, 4.2% had a female householder with no husband present, and 68.1% were non-families. 38.9% of all households were made up of individuals and 10.6% had someone living alone who was 65 years of age or older. The average household size was 2.05 and the average family size was 2.73.,In the village, the population was spread out with 3.0% under the age of 18, 83.1% from 18 to 24, 5.9% from 25 to 44, 4.6% from 45 to 64, and 3.3% who were 65 years of age or older. The median age was 20 years. For every 100 females there were 144.8 males. For every 100 females age 18 and over, there were 147.2 males.,The median income for a household in the village was $21,313, and the median income for a family was $70,694. Males had a median income of $15,750 versus $39,375 for females. The per capita income for the village was $8,224. About 2.0% of families and 37.7% of the population were below the poverty line, including none of those under age 18 and 2.8% of those age 65 or over.\n",
      "id                                                   2475\n",
      "lat                                              0.543859\n",
      "lon                                              0.918926\n",
      "text    Abadeh (Persian   آباده  main , also Romanized...\n",
      "Name: 10, dtype: object\n"
     ]
    }
   ],
   "source": [
    "#Look at content of potentially interesting documents\n",
    "a = sample.columns.get_loc('text')\n",
    "\n",
    "print(sample.iloc[11, a])\n",
    "print()\n",
    "print(sample.iloc[15, a])\n",
    "\n",
    "print(sample.iloc[3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "2baaa4ed",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 -> 0.9419445378236924\n",
      "0 -> 111\n",
      "1 -> 0.9191101901704861\n",
      "1 -> 42\n",
      "2 -> 0.8898547469512932\n",
      "2 -> 91\n",
      "3 -> 0.913729627552191\n",
      "3 -> 36\n",
      "4 -> 0.9066650709532309\n",
      "4 -> 20\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD6CAYAAACvZ4z8AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAoyklEQVR4nO2dW6wk13We/7+qu8/9Mpw5Q5EztIYJaImMLNHKgJYgQLFsS6KUKET8IjJIBDAWKAKkoDhBbMYvQeAXI04CKxABikgIxrAlyoBFhE4GogTFgZCAiTgMSZHDizQZXuZwyJkz9zPn3t0rD93NqdPTl13dVbVXVa0PIOd0V1XXqttfa6+99toUERiGYRjFJfBtgGEYhpEuJvSGYRgFx4TeMAyj4JjQG4ZhFBwTesMwjIJjQm8YhlFwhgo9ycdIniH5cp/lJPkfSB4n+TOSH48su5Pk6+1lDyVpuGEYhuEGh+XRk/w0gCsA/lREPtJj+RcBfB3AFwH8GoBvisivkQwB/BzAZwEsA3gWwD0i8sowo/bt2yeHDh2KeSiGYRjl5bnnnjsrIku9llWGbSwiPyF5aMAqd6H1EhAA/5vkIskbABwCcFxETgAAySfa6w4V+kOHDuHo0aPDVjMMwzDakHyr37IkYvQHAJyMfF5uf9fve8MwDCNDkhB69vhOBnzf+0fI+0geJXl0ZWUlAbMMwzAMIBmhXwZwU+TzQQCnBnzfExF5VEQOi8jhpaWeYSbDMAxjBJIQ+qcAfKWdffMJAJdE5F20Ol9vIXkzyRqAu9vrGoZhGBkytDOW5HcB/DqAfSSXAfwrAFUAEJFHABxBK+PmOIB1APe2l9VJPgjgaQAhgMdE5FgKx2AYhmEMwCXr5p4hywXAA32WHUHrRWAYhmF4wkbGGoZhFJyhHn3R+PK3nxn7N773tU8mYIlhGEY2lE7oO2zsNFBvXJvt+ea5NQDAob0zu74ngZlaCLJX1qhhGIZeSif03/vaJ3Huyhaef/tiz+X/5unXAAD/7LO/fM2y/fMT+OjBxRStMwzDSJ7SxejrjSZefXd1pG3PXN7C6cubCVtkGIaRLqUT+p+fvoLNncbI27/23iq26qNvbxiGkTWlEvqV1S2curgx1m/s1EdvERiGYfigNDH67XoTr757OZHfOru6hXcubuDA4lQiv+eTaBaSvP+/wXT3RxcpC2ncrKwinQujOJRG6F9/bxXb9WZiv/fz06vYO1PDZDVM7Dd9c2l9BxJR+pPn1wEAN103vWu9WhhgZqIct06jKdhoh/reONvKyLp5Xysja7oWIrAsLCMHlOJpPX15M/FO1EZDcOzUZfztD+5J9HezpuOBnlndxM9OXtq1rJOB9Huf//Cu78OA+PQvLyEMiidy3R75G2fX8P/OXAFw9Xz87m+1MrI+9IG5a16CRv45dXEDzR4TMj34nefH/u1v/cNfvea7xekaZlN2nAov9Nv1Jl57L52Y+oW17cKEcM5c3nJet9EUnLuyhf3zkylapINzV/qfl7NXtkzoC8ZWvYFXTvUO8a5v1/tu16/1281rPfr3Dl43hQ9/YD6GlfEpvND//PQqdhIM2XTzi9Or2Ddbw0QlvyGcRlOwMkDQenH6cvGFfrvexMX1nb7LL6xvo95oohKWKqeh0Fza6H+9u1u2Ufq1fl0YdI8lRaHv0Atr23jvUrp57/WG4Benr6S6j7Q5e2ULjR6jhIdtU2+k9wLVwLCXX7MJnFvbzsgaIwsuDxD6tFjbqqPRjPf8xaWwQi8ieP10NmmQ713axMX1/D7wo7wMR2kF5A2X85K2I2Fky6WN/uGZtBABVjfTfcEUVuhPXdrElc3sLtovzuTTq9+uN3F2RMF+t8Ait7nTwAUHb/3c2hZ2Ct6yKRNpC27//aarVYUU+mZT8MbKWqb7vLS+g5XV/Hm4py9vokeCgRPnr2yPNcpYM66eerNpXn1R2NjuXegwCy6bRx+f9y5vehGgTuXLPDHuSOGievVxzsu459DQweqWH28eQOrRh0IKfSfVKWsure8M7LXXxurmzthNxncLKHLn17axvu3uKKxu1nN13Y3erG35a52ubdchozatHShcemUS4jUO717awMJU1dv+43Dq4vje+Pp2K5a9Z6aWgEU6GMVROHl+HQsHFlKwxg8X1rbx1f/87Ni/8+hXDmNxupaLwXVrW/50o9kENneamKqlk6ZdOKE/4zlOvrK6hQ9/wKsJTjSagncvJeONv3NxozBCv7HdGKlz+szqJrbqs7keT9FhZXULL568iNUhwucySOj5ty/i5qUZ/M2l2URtTIM4rbg0WNuum9C74pIpkSZbO01c2aqnPqR5XM6sbibW8XRmdRM7jTlUCzBw6OSF9ZE6p5tNYPnCRi4EbRDNpuAX7bTkYYN/XAcJvXVuDTcuTKUmYkkxaORrFmyk+KLJ/5MZQUS8hm06+Bh0EZckOxCLknmy02jinTHOy/KFjdQHvqTNyQvriXu2zSbwizO6S3vvNJreMm46bKSYQFIood+qN1U8aL6bgMNY26rjwlqyL6PlC/nvlF2+sBF7hHCUnXoz1xk42/UmTpxNJ3PszOUt763tQWylWCbF2Yad9GwolNBrGbiixY5+pCFGa1t1XMqgZkdaNJqCtxPI1nrr3DqaCpyNUXjj7NpYL7phHF/RO6hwS8F4kDRnriuU0Gt5vFLMkhqbZlNwKqUwyzhhD9+curiRSPG7zZ0G3svhvMKbOw28czHdtORL6zs4s6rz3Gjw6JOcL6ObQgl9RUkKVzXUYUcvVq5spVbN8/TlzVwWOms2BW+dS07k3jy3lmpOdBqcWFlDM4NLd2JF57nR0ArfTtGGQgm9ltQ2LXb0Is0YcqMpOJ3DMhBJj6Re32rkqhxGqxWSTWvsymYdZ6/oi9XveO6IBVqVcNN6CRZK6MOAmFaQwjU7qTO1cnOngfMpd4jlbaSsiKRSuuKNlDo102D5wnom3nyHt8/rOzcakjgAoJ6SHU5CT/JOkq+TPE7yoR7L95B8kuTPSP6U5Eciy94k+RLJF0geTdL4Xsx7HpVKAvNKhf69S6MXMHPl4vqO93zkOKysbmE9haHvq5v1gbNTaaHZlMwzpi6s7XirEtkPLUKflh1DhZ5kCOBhAF8AcBuAe0je1rXaHwB4QUQ+CuArAL7ZtfwzInK7iBxOwOaB7JudSHsXA1mcrqqdcSirAmR5KnT2Vop1kdL87aRYubLlJX9c2z3Sa45YH6Rlhosi3QHguIicEJFtAE8AuKtrndsA/BgAROQ1AIdIXp+opY7sna0h8KizS7M6p9db3dzJrJbHaWUPcT8urm+nmhJ6/sq2Os+1G1+C22pd6hBXQE+mXFovHBdJPADgZOTzcvu7KC8C+G0AIHkHgA8CONheJgB+SPI5kveNZ+5wqmGAvTN+vHoS2D/vt0XRj9MxJv8el/XtRur1tZMgibx5DfsYlUZTcH7NT3hpu960ip8Z4iL0vXIFu187fwRgD8kXAHwdwPMAOu7jp0Tk42iFfh4g+emeOyHvI3mU5NGVlRUn4/tx4+LUWNuPyt7ZCUxW/XcG9+JMxrndWe8vLps72WTGnL68mWp+9DicX9vOtBO2G03ZN1SSEZ2WHS5CvwzgpsjngwBORVcQkcsicq+I3I5WjH4JwBvtZafa/54B8CRaoaBrEJFHReSwiBxeWlqKexy72Ddbw0Q1+/jNjYs6wzZXtuqZl2XwXUV0GO9c3Mikud5sIrEqoUnje55j3/vXCHv61ePjoobPAriF5M0kawDuBvBUdAWSi+1lAPBVAD8RkcskZ0jOtdeZAfA5AC8nZ35vSOJAxl79RDXAkueO4H6c9SC661sNtdk3IpJpTZp3lNYB8h1eW91Md7KNOARKXPq0+heH/qyI1AE8COBpAK8C+AsROUbyfpL3t1e7FcAxkq+hFaL5Rvv76wH8T5IvAvgpgP8mIj9I+iB6cePiVKbNsQOLU6CSm6Wbc57isOcUNc2jnF/bTrWAVDfr2w2VdYCueJxRCWj1EaRZsTEOFSWj2SspKb1TwreIHAFwpOu7RyJ/PwPglh7bnQDwsTFtHInJaoiluQmcyaATkvTXLzCMRlO8dXqdX9seOCmFL3xkmrx7eQML03pmHttpNFMrhRGHje0Gpmv+x51oKJ9CIrWZuHQmfCdEVuK7T3En7KWNHW8dbhfWt9U0zTs0mzLSDFLjsrK6pepcaCjiBQCbSuzQMGlOmuNv/B9diuydyaZTVqs3D8BrClu9Iepq81/c2PEyQGhrp4nLCibF6aDBmwf02KFB6NMshuj/6FKEJG5YSDcTploJsFfxfKm+Z7vy3eHXjc+yBJpKIjSUtC602DFR8S+FaRZD9H90KbN/Pl2h3z83gUBBfK8fVzzObA+0qhVqIu2iboO4oCidUIm+qrHDRzr2NTak+LLxf3QpMz9ZTXVS4v1zOlMqgXZWg+fQie8XTZR6o+nVnlZ/iQ5l05IgpsWOSQWlxdPs5yu80APAUkpiHIbEnmm9YRsNeey+XzRRrmzVvXqQzSawpuCaAECoRGH15K/Tu1efpkNaCqFPK4Z+3XRNddhmM8Nc8b42pDgPZlxWFYSRtLRwQjV54zrsAIApz5lzae6/FEK/OJ1ORcvrFHfCAulONuxKs6ljmjYAKjKA1jwPUupQU5BlAgA1BZ2gHXzn86c5aZKes5wiYUAspDAhiXah95FG2AstQp/kdIF5tgHQI/Qasl06zEz48+jDkBajT4LFhGPptUqAmQn/I/oGkda0ZHHRMB8noGOQkAYbgFZMWoM3rWmgoc/neTblffu/0hmxmLBHv6hoOHs/9Myao8MODS0LDTZ0SLPzzwVSl0efttgOYiblsJGes5wyC1PVRFO5Fqd0h20APTnKWuzQMC+olvRKQEfno6ZCgJPV0Ftxs7mU55kujdBXwmRDLWnE/JNGyzOkxQ4NLRxFOu899DitMPQ5N+nnuTahT5CkxDkI0r8wSaAlc02T1+YbTadixnPoxvf+e7Ewlf1zTab/gimV0CcVV5+frKrOn+8Q+pwlPYKWXOm0SsDGwb8FV5n17Kz4blH0Yt6DRz8zUUn93tShBBmRVFw9Dx2xgB6B1TKpgwahT7MUbVymqmFqMxq54PtF04t5DyHZLMLAeu66DJiqhYmklPm4GUZBQ/ocoCdnW4MdaZaijQvJ1LM9+u87/UyTUZishpmXQjChT4EkvPE8ZNwAOlLXapVATYxew4svzVK0o+DLq56qhSpaWL3I+vnOIkLg/87PmHEv4vREMq2CLNAwGEWDDR18pxMCwKSCcrhR5iY8ZZl42q8LWYZmq5Ugk9ILuu66DBh33s68ePNAy6P33R+rQVw7aEjn811PpRtfHr3G+HyHLIV+T0b7Kp3Qz0+O18Odl45YoBWD9V1n2/foyyga0vl81lPpha/RoJrTk2cnKpklEGTlOJZO6EmO1ZmaJ6EH/HuxmoRtdqLiNY89CPwOs+9FrRJ4Ca9pOw9RSCZeG6sfe2bMo0+NUcW6llE8LUlmPQutpvNVCf1ev/nJqpqO6ShZh1Gqnl4ucbguA6GvhMzshVdOoR/Ro89D2YNufA9K0ea5+WyRab1/sg6jaA7bdFjMwNPeM13L7MVfSqEfNXSj9UEdhE+hnVaYQudzDoE9SucvyFp4fYw+jctcBnH6LO/FUgp9NQwwPUJII49CP1OreMu80ZhZ4WuO3yDwt+9hZC28efDoSaYuxFm++Esp9MBoN3cebtBugsDf6EdflQAHUasEY6fYjsLidE1d66bDZDXbsSF58OiBdF/MtUqQaWvb6eqSvJPk6ySPk3yox/I9JJ8k+TOSPyX5EddtfRH3ZpueCFXVKYmDL8GdV/piXJqdKMU+45BVa7VWCVSl3A4iTY8+6xDiUOUiGQJ4GMAXANwG4B6St3Wt9gcAXhCRjwL4CoBvxtjWC3G987x4Ib2Y91B6FdDp0QPA/nkPQj+nW+izqt+UlzpRQCuRIa26N1n317gcxR0AjovICRHZBvAEgLu61rkNwI8BQEReA3CI5PWO23ohbvxYW/ZIHHwIbtbhgDhM1yqZhuH2zFTVpxMmPdWm7/0kRVrhmyzSN6O4PIkHAJyMfF5ufxflRQC/DQAk7wDwQQAHHbf1QjUMYr2tNXYsujI3kX2HrK9WhCsfWJjMbF/Xz2e3r1GZn6pmco/kbcBhGiGWqVqYefjK5dL26kHqnhDtjwDsIfkCgK8DeB5A3XHb1k7I+0geJXl0ZWXFwazxieOl59mjDwJiNuMiUtpDXdfPT2YySjYI8iH0YcDUr1kW+0iaNITeR/aVi9AvA7gp8vkggFPRFUTksojcKyK3oxWjXwLwhsu2kd94VEQOi8jhpaUl9yMYA9fBRGFA9U3vYWTtYWuPxU5Ww0w6xPbPTaKak078tM/H4nQ+ZmaLMlkNMZ2w9+1jLIfLHfgsgFtI3kyyBuBuAE9FVyC52F4GAF8F8BMRueyyrU9cL2DSF9oHWcfp85CKemBxKvV93JBhiGhc0hagvTO6O6T7kXTdGx/hq6FPo4jUST4I4GkAIYDHROQYyfvbyx8BcCuAPyXZAPAKgN8ZtG06hxIf1/xy32UEkiDLVMfpWpgLL3bf7ARqlQDb9WYqvz9Vy6bVkBQLU1VUQqLe6BldHZu9s/k5F1Gum6nh1MWNRH5ruhZ6iQ44Pf0icgTAka7vHon8/QyAW1y31YJrh0he8n4H0Rkh20xH03ahNa2ymyAgblycwptn11L5/RsXp1QWMesHSeybncB7lzYT/+2pWphbhylJD9xXGQz9bleKTFbdarEUIXSTZYes9oybKGmFb4IAuHExP2GbDmnl+2sfRzCIJOP03kpweNmrIly89elqfoRrEFnFzfOUoTRVC1MJKSzNTqqbH9aFvTO1VNIs9+dY6IHk4vS+0ktN6B3iZUUI3QDZCb32jJtuDu6ZTuE30+/oTYNKGCTeaTpRDXJZEDBKEgI95Sk+D5jQD22ShSHVjvCMSxax88lqPjpio+ybrSX6AM5MVNSWJHYh6cFkrTEL+emr6EUSQu/zZZevJzIFhnnrmia3HpcsptLL4whikjiQoAeeV2++w77ZCYQJ1mLPw4CxYUzXKmM7fD5f/ib0Q4S8CB2xHcKAqYeh8hSfj3LjYjIjZcOAmZZXSIMwYGLVNqdrYe7DNh3G9erNo/fIsDlEiyT0ADCXcuZNHgZK9WKiEiaSGXL9fH5Gwg4iqYFeeX/pRRlHqMOQmPGoJfm/I8dkshoM9OSmFE1unQRph1by6tEDyaRaZjHaNguum6klUqL3hoVinA9gvPpNvieGL73QkxwYvilSjB4AZkaYQtGVIMh3C+i6mfE6ZWcmKl5mr0oDkvjAmLH1xelqYTLWgPFaqwuex5aUXugBYHLAzZhn4epFmh73dK2S6+wKkmMNciqKN99h3LBLkcI2QCv1dJS5pgH/1VxN6NFfzIMAmChIamWHKcfRwKOQ57BNh1FDDSRw/UK+BwV1MzdZHTnUl5fyzHEZVbB9lwUploqNSL+Rr5PVMNceai9IptZKyWstkyhTtXCk7IrrZmq5HAk7jFHDN3tnJgrRKd3NKOGbSph+ttswinclRmCy1vs0FC0+3yEtQU4z/p8lo3iiRQtTdBj1uPJUnjkOo7RaNbR0TejRP8VyWOplXklN6AtyvvbPT8TKqQ8CJJZ3ro3JavwWThgQewt6PkYJZWkYRGhCj/6ee3E9+uSPK+8ZN1EmKmGsIlZ7ZyZQKWCYosP+uXje+dLcRGr9QL6ZqISoxuy30+AAFffujEEY9K5n0y+kk3fSuPGmqvnOuOkmTrXFPJfgdWH/fLzjy3ulymHEHfikoe+qmEo2Ar06S4rq0U9Vw8Rr3hQlPt/BVbzJVm2YIjNZDZ0rkgaBnzlRsyRuSFdDS9eEvs1kj4yJvE8I3o8ghZo3RevPmKy6zYi0MFUtTHXTQexzrNm/Z7pW6DAWEE+4w4AqdKTYVyQGU11hmkrIQqaHdUg6fKMhsyBpXMSt6N5rB9fO1aK3boB4Qu87rbJDcZUsJt050BrewmmSdNxw1BGDmnER8aQn6dDK/GTFqRMyrxOAx2HQSPputIR/TejbdBdwKtqI2G6SjqlryCxImsXpwdPqhSFzNT/uOJDEdUMykSaqQeFCeL3oFebtu64JvS66PfoijnKMkuQD6TrJet4IAw4c8r445bciYdYMy6f3NfF11tQqgfP9PplABdAk0GGFAro9+CRKtGomydrYRcu4iTJI3JKaMDovDBN6XxNf+8C1xW8evTK6L1ytwB2xQKsSX1I3oYY84bRYmOov5osFmTnJldmJysApBosyk5QLro6gFh3RYYUCSKISuYmLHqMHkutALbLQ94vBk/mdTWtUyP6hrDBgITOv+lEL3Z4dLam3OqxQQvSiFDm1skNSD6bPKdLSZqIS9mz5TNcqhc8X78V8n5fb3GSxRkYPw1XATegVEm1mxa1nkUeS8sSL7NEDvT33snnzHfoV6NJQuCtLKgNCWB1IoKIkSaH4ahaDqIem5QKlSRKeeK0SFL7100vEfM8Y5It+rcAyhW0At9h7JQzUtHKcnlCSd5J8neRxkg/1WL5A8q9IvkjyGMl7I8veJPkSyRdIHk3S+KSJinvRxQtIxhMvujcPAHM9jrHImUaD6JeWW8RxFINwSa/U5CwOvTokQwAPA/gsgGUAz5J8SkReiaz2AIBXRORLJJcAvE7yz0Vku738MyJyNmnjk6Yj7oTbhcw71TBArRJgu94c+TfKIHi9XmZleMH1olO7ZXOnsev7Io6MHoRL6EaThri4rXcAOC4iJ9rC/QSAu7rWEQBzbLVTZgGcB1BP1NIM6FwYLc2tLBhXsMrgyXVX+wxDHYWqfNFdvyUMWPgBht2EDhqhyaN3EfoDAE5GPi+3v4vyLQC3AjgF4CUA3xCRjpsoAH5I8jmS941pb6p0Loyey5M+43rkZfBsg4C7apZMl1jkgWvrt5TxpefirQc5E/pe1krX588DeAHAjQBuB/AtkvPtZZ8SkY8D+AKAB0h+uudOyPtIHiV5dGVlxcX2xHm/Oabn+qTOuB55GUI3wG4vtgz1XAbRPaxfyzD/LHERcRevPytcrtAygJsinw+i5blHuRfA96XFcQBvAPgwAIjIqfa/ZwA8iVYo6BpE5FEROSwih5eWluIdRUJUgk6MXs8FSptxPPJKWJ4me1TctZSe9cVEtVx1oXoROIi4yzpZ4SL0zwK4heTNJGsA7gbwVNc6bwP4TQAgeT2ADwE4QXKG5Fz7+xkAnwPwclLGJ02nUqGi65M648x+UybPNuq1ltGDjVK2ulC9cInKaNKRoU+qiNRJPgjgaQAhgMdE5BjJ+9vLHwHwhwAeJ/kSWoGP3xeRsyT/BoAn252bFQDfEZEfpHQsY3PVoy8PncqTjWZ3NG44GqZIy4poHLqMMeko3aM9tdRzyRKXVn+uhB4AROQIgCNd3z0S+fsUWt5693YnAHxsTBsT5cvffqbvsqYIli9sgBy83ve+9sk0TPPGdC3E6mb8JKkydMR2iNYgL73Qdwm7lmH+WaJJxF0oz5PqQEDiVw4s+DYjc6ZrlZGEvkwefTQ8UYaCd4PoHkxYhsGFo6Cpr690Ql80bzwJRu1cLFOnZC0ymK7swhYG3DXzVtVh8FAZkWuSE/1ROqE3rmVkoS9RCCMICFKTj+aXSkTpK4PmWywookfDnTChN0YS7ErI0nm2AfIXm02L6KhPl3IARcPFW9f0MjChN0ZKFyxjhyRJE/o20ZGhmob6Z4UmEXehXC6Z0ZM4s9q/v00phb5cqbeD6HjxRLlqQ3VoOii9yzpZYUJvIAgYe6KVMmaelFXUehF2xpyU9Hy4SLginTehN1rEFe5SCr2Fbt5HUx0XHzQdBhg2FCl9+Z5WoydxB72UdZBMueXtKldLens2xBMuA8nFhN7QRtxh7GUc9m5c5X2hL+mrz6VkSGP0+XwSx55WA0D8QUBlS60ELEYfpYSJNrtw8datM9ZQR9xpz8qYO20af5Wg5KEbl/j7KIUC08KE3gAQfxh7GUdDljdQcS2aaq37wEXEzaM31BH3wS2lzgPWG9umk3VT1tPhIvR18+gNbcQN3ZQ9va7svH/5S3obOHn0JvSGNmJ79CUUeqK0unYNQcmzblzCMiJ6xN6E3gAQP4tC0wz3RvaU/fK7hmW0hG9M6I0WMR7cEjrz76PjsfVPp0VX1nuh3nC7E7R0yJrQG4YjOh5ZHZRV4Du4Crh59IZh5JayxuY7uAq4llx6E3rDiIOO59Y7nRh9WeXetZPVhN7QRYz7UUnY0QOlPXCjC/PojVwS93bUVJkvK0RM6juUveaPefRGLombHVBCnTcilFvm3WvNa6lJb0JvAIgv3FrSxrJEUM6WjHEtrp66DZgyVBFXuJXcv4YnSh65cX5eLHRjqMI8+uFYjP4qJbz87yMiaDpOKqLlOXESepJ3knyd5HGSD/VYvkDyr0i+SPIYyXtdtzV0YDF6F0zpjXit2dwIPckQwMMAvgDgNgD3kLyta7UHALwiIh8D8OsA/h3JmuO2hgLitjC13MBZYh69AcS795VEbpw8+jsAHBeREyKyDeAJAHd1rSMA5tjKuZoFcB5A3XFbQwHxY/RK7uAMEQBiUg+g3C+8OHH3PMXoDwA4Gfm83P4uyrcA3ArgFICXAHxDRJqO2xoKiKvbOm5fD5T2wHdT5uyjOIeuxSFyEfpe/evd1n8ewAsAbgRwO4BvkZx33La1E/I+kkdJHl1ZWXEwy0gWi9EPw0I3BhCvVaflOXER+mUAN0U+H0TLc49yL4DvS4vjAN4A8GHHbQEAIvKoiBwWkcNLS0uu9hsJEfuGVHIDZ4lA1Dy4hj/iRGO03C8uQv8sgFtI3kyyBuBuAE91rfM2gN8EAJLXA/gQgBOO2xoKiK/zSu7gLJGSHncPynwW4oSttNwvlWEriEid5IMAngYQAnhMRI6RvL+9/BEAfwjgcZIvoRWu+X0ROQsAvbZN51AMI10EAHU8t2oo4+mIc8xaPPqhQg8AInIEwJGu7x6J/H0KwOdctzXyTxnrkSt5ZlWgRcB8EOfYtZwmGxlrALAiVU6Iloa44ZVYMXodd4wJvQEgftlZluzOaTbbIq/kwTWMODiFboziE8R06YOSVbXqlJvtVLAsQz32L3/7mb7LdhpNnDy/joAcuN73vvbJNEwzYmJCbwAAwphKH5ZA6KJERzg2BQjLdfjXUAmIW2+YRxDXQygCMQ5Zi0NgQm8AiO+hByUL3UQjNo2mxH4x5hHzxotDyR5Xox+VmC5qpWRKH50pSMuwdsMPcXwiLe5AuZ5Woy9xPNQgiB/qyTtNE3qjTZw7X0nkxoTeaFGN4aGHJfPmAUAiE00oKUhoeCJOmFNL0kL5nlijJ9WK+61QLWFPpHn0Rgcl2h0LE3oDAFAL3W+FOOsWhai4i+M0ckYxMY/eyC3VkM6eykQlTNcYhciuv82jLzNxxFtLX5YJvQGgle9bcwzfuK5XJKLRGovclJs42q1E503ojau4euoTZRT6iBdvOl9uYoVulCh9+Z5Yoy+TVbfbYbJavtANdnn0JvVlJgjcw5wWozfU4Srgri8EwygqrrH3inn0hjYmHUM3pfToI5g/b7gKvYVuDHVM1obfDkFQzhh9FB2PruET8+iN3DLl4KlPVkI1FfkMwxeutZ4svdJQx3RteDHTqVq5wzaGAZhHb+SYMCAmhnS0llboI8+rtWgMVwE3j95QybDwzXS1nFMYRCdD1/HoGj5xFfCqknIhOqww1DDMY5+eKKdHH32uteRGG/5wFXDz6A2VzAyJ00+XNHQTDdeUbWJ041pcJ+qxGL2hkkFCTrrn2heN6POq49E1fOIyf0MlpJr+nHIGXI2+DArdTFVDNQNA0uLL336m5/eNpuDk+XUAwD95/Nm+4RubZ7UcuHj0WuLzgAm90cWgFMvSZtygFWv9WzcuALAYveEm9Fri84AJvdFFJ8Vya+fa2TVmJop/u5hHbrjgErrR5NHrscRQQz+v3mXkrGGUAbfQjR6P3knoSd5J8nWSx0k+1GP5vyD5Qvu/l0k2SF7XXvYmyZfay44mfQBG8vTrkC1rxo1hdOPirWvy6Ie2xUmGAB4G8FkAywCeJfmUiLzSWUdE/hjAH7fX/xKA3xWR85Gf+YyInE3UciM1+gt98UM3huFC3oTexZI7ABwXkRMisg3gCQB3DVj/HgDfTcI4ww+9Ol1Jq0NvGB3CgEM7W/MWujkA4GTk83L7u2sgOQ3gTgB/GflaAPyQ5HMk7xvVUCM7esXip6pWtdIwogyL02vy6F3a4r2Opt/cC18C8L+6wjafEpFTJPcD+BHJ10TkJ9fspPUSuA8AfumXfsnBLCMtegn9pMXnDWMX1bB3dloH19GzWeDyylkGcFPk80EAp/qseze6wjYicqr97xkAT6IVCroGEXlURA6LyOGlpSUHs4y0qIQBql2Ti1jGjWHsZpjHXlPk0btY8iyAW0jeTLKGlpg/1b0SyQUAfwfAf4l8N0NyrvM3gM8BeDkJw4106RZ2E3rD2M2wGHyuQjciUif5IICnAYQAHhORYyTvby9/pL3qPwDwQxFZi2x+PYAn27HdCoDviMgPkjwAYzz6DfnfqjewfGEDAPAnP/45Jiph3wJNNsjIKCPDhDxXQg8AInIEwJGu7x7p+vw4gMe7vjsB4GNjWWh4YaIS4lcOLPg2wzDUMlzo9cToLTG65Jg3bhijMSgGr6lyJWAlEAzDMEaiWukv5Jo6YgETesMwjJGoDChs1p215htd1hiGYeSEgaEbRSWKARN6wzCMkRgUutGUcQOY0BuGYYzEIDGvWejGMAwj/1QCol9ijYVuDMMwCgBJVPp49Ra6MQzDKAjVPp67Cb1hGEZB6JdGqWlULGBCbxiGMTL9YvGWR28YhlEQ+oVoqgMGU/lAlzWGYRg5op/Qa5p0BDChNwzDGJl+gm7plYZhGAWhV4hGW+VKwITeMAxjZHqVQdBWuRKwevSGYRhO9JqNbafRxPp2AyfPt2Zj+7c/fB1hQMxO9JZWX/M/mNAbhmGMSDUMsDAV4KMHdc/GZkJvGIbhQJ5nY9MXTDIMwzASxYTeMAyj4JjQG4ZhFBwTesMwjIJjQm8YhlFwTOgNwzAKjgm9YRhGwTGhNwzDKDgUEd82XAPJFQBveTRhH4CzHvevCTsXu7HzsRs7H1fxfS4+KCJLvRaoFHrfkDwqIod926EBOxe7sfOxGzsfV9F8Lix0YxiGUXBM6A3DMAqOCX1vHvVtgCLsXOzGzsdu7HxcRe25sBi9YRhGwTGP3jAMo+CY0EcgeSfJ10keJ/mQb3t8QvIxkmdIvuzbFg2QvInkX5N8leQxkt/wbZMvSE6S/CnJF9vn4l/7tsk3JEOSz5P8r75t6YUJfRuSIYCHAXwBwG0A7iF5m1+rvPI4gDt9G6GIOoB/LiK3AvgEgAdKfH9sAfgNEfkYgNsB3EnyE35N8s43ALzq24h+mNBf5Q4Ax0XkhIhsA3gCwF2ebfKGiPwEwHnfdmhBRN4Vkf/b/nsVrYf6gF+r/CAtrrQ/Vtv/lbazj+RBAH8XwH/0bUs/TOivcgDAycjnZZT0QTYGQ/IQgF8F8H88m+KNdqjiBQBnAPxIREp7LgD8CYDfA9D0bEdfTOivwh7fldZLMXpDchbAXwL4pyJy2bc9vhCRhojcDuAggDtIfsSzSV4g+fcAnBGR53zbMggT+qssA7gp8vkggFOebDEUQrKKlsj/uYh837c9GhCRiwD+B8rbn/MpAH+f5JtohXt/g+Sf+TXpWkzor/IsgFtI3kyyBuBuAE95tslQAkkC+E8AXhWRf+/bHp+QXCK52P57CsBvAXjNq1GeEJF/KSIHReQQWprx30XkH3k26xpM6NuISB3AgwCeRquj7S9E5Jhfq/xB8rsAngHwIZLLJH/Ht02e+RSAf4yWx/ZC+78v+jbKEzcA+GuSP0PLQfqRiKhMKzRa2MhYwzCMgmMevWEYRsExoTcMwyg4JvSGYRgFx4TeMAyj4JjQG4ZhFBwTesMwjIJjQm8YhlFwTOgNwzAKzv8HHLsSNgBESoAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#This block plots violin plots of the cosine similarity as a function of distance\n",
    "classes = 5 #Change this value to modify the number of classes we plot\n",
    "m1 = np.min(dists)\n",
    "m2 = np.max(dists)\n",
    "step = (m2 - m1)/classes+1\n",
    "boxes = {}\n",
    "\n",
    "rows = len(cosine)\n",
    "cols = len(cosine[0])\n",
    "for i in range(rows):\n",
    "    for j in range(cols):\n",
    "        k = (cols * i) + j\n",
    "        if i < j: #Only need values below the diagonal\n",
    "            dist = dists[i][j]\n",
    "            cos = cosine[i][j]\n",
    "            box = int(dist/step)            \n",
    "            if box in boxes.keys():\n",
    "                boxes[box].append(cos)\n",
    "            else:\n",
    "                boxes[box] = [cos]\n",
    "   \n",
    "labels, data = sorted(boxes.keys()), boxes.values()\n",
    "\n",
    "for key in sorted(boxes.keys()):\n",
    "    print(key, '->', np.mean(boxes[key]))    \n",
    "    print(key, '->', len(boxes[key]))    \n",
    "\n",
    "    \n",
    "plt.violinplot(data)\n",
    "plt.xticks(range(1, len(labels) + 1), labels)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "f27380bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "#output all dependencies so that we can reproduce the notebook (we only need this to set things up for Binder)\n",
    "#%load_ext watermark\n",
    "#%watermark --iversions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fc96fe10",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}