Topic Modelling with Gensim_v4.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Topic modelling with Gensim : the LDA algorithm\n",
    "\n",
    "LDA is a generative statistical model that allows sets of observations to be explained by unobserved groups that explain why some parts of the data are similar.\n",
    "\n",
    "### 1. Loading of the required libraries and vocabularies.\n",
    "***\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: en-core-web-sm==3.0.0 from https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.0.0/en_core_web_sm-3.0.0-py3-none-any.whl#egg=en_core_web_sm==3.0.0 in d:\\programdata\\anaconda3\\lib\\site-packages (3.0.0)\n",
      "Requirement already satisfied: spacy<3.1.0,>=3.0.0 in d:\\programdata\\anaconda3\\lib\\site-packages (from en-core-web-sm==3.0.0) (3.0.3)\n",
      "Requirement already satisfied: setuptools in d:\\programdata\\anaconda3\\lib\\site-packages (from spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (50.3.1.post20201107)\n",
      "Requirement already satisfied: blis<0.8.0,>=0.4.0 in d:\\programdata\\anaconda3\\lib\\site-packages (from spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (0.7.4)\n",
      "Requirement already satisfied: wasabi<1.1.0,>=0.8.1 in d:\\programdata\\anaconda3\\lib\\site-packages (from spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (0.8.2)\n",
      "Requirement already satisfied: typer<0.4.0,>=0.3.0 in d:\\programdata\\anaconda3\\lib\\site-packages (from spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (0.3.2)\n",
      "Requirement already satisfied: numpy>=1.15.0 in d:\\programdata\\anaconda3\\lib\\site-packages (from spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (1.19.2)\n",
      "Requirement already satisfied: srsly<3.0.0,>=2.4.0 in d:\\programdata\\anaconda3\\lib\\site-packages (from spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (2.4.0)\n",
      "Requirement already satisfied: requests<3.0.0,>=2.13.0 in d:\\programdata\\anaconda3\\lib\\site-packages (from spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (2.24.0)\n",
      "Requirement already satisfied: preshed<3.1.0,>=3.0.2 in d:\\programdata\\anaconda3\\lib\\site-packages (from spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (3.0.5)\n",
      "Requirement already satisfied: pydantic<1.8.0,>=1.7.1 in d:\\programdata\\anaconda3\\lib\\site-packages (from spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (1.7.3)\n",
      "Requirement already satisfied: jinja2 in d:\\programdata\\anaconda3\\lib\\site-packages (from spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (2.11.2)\n",
      "Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in d:\\programdata\\anaconda3\\lib\\site-packages (from spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (1.0.5)\n",
      "Requirement already satisfied: pathy in d:\\programdata\\anaconda3\\lib\\site-packages (from spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (0.3.5)\n",
      "Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in d:\\programdata\\anaconda3\\lib\\site-packages (from spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (4.50.2)\n",
      "Requirement already satisfied: catalogue<2.1.0,>=2.0.1 in d:\\programdata\\anaconda3\\lib\\site-packages (from spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (2.0.1)\n",
      "Requirement already satisfied: packaging>=20.0 in d:\\programdata\\anaconda3\\lib\\site-packages (from spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (20.4)\n",
      "Requirement already satisfied: cymem<2.1.0,>=2.0.2 in d:\\programdata\\anaconda3\\lib\\site-packages (from spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (2.0.5)\n",
      "Requirement already satisfied: thinc<8.1.0,>=8.0.0 in d:\\programdata\\anaconda3\\lib\\site-packages (from spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (8.0.1)\n",
      "Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.0 in d:\\programdata\\anaconda3\\lib\\site-packages (from spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (3.0.1)\n",
      "Requirement already satisfied: click<7.2.0,>=7.1.1 in d:\\programdata\\anaconda3\\lib\\site-packages (from typer<0.4.0,>=0.3.0->spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (7.1.2)\n",
      "Requirement already satisfied: idna<3,>=2.5 in d:\\programdata\\anaconda3\\lib\\site-packages (from requests<3.0.0,>=2.13.0->spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (2.10)\n",
      "Requirement already satisfied: chardet<4,>=3.0.2 in d:\\programdata\\anaconda3\\lib\\site-packages (from requests<3.0.0,>=2.13.0->spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (3.0.4)\n",
      "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in d:\\programdata\\anaconda3\\lib\\site-packages (from requests<3.0.0,>=2.13.0->spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (1.25.11)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in d:\\programdata\\anaconda3\\lib\\site-packages (from requests<3.0.0,>=2.13.0->spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (2020.6.20)\n",
      "Requirement already satisfied: MarkupSafe>=0.23 in d:\\programdata\\anaconda3\\lib\\site-packages (from jinja2->spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (1.1.1)\n",
      "Requirement already satisfied: smart-open<4.0.0,>=2.2.0 in d:\\programdata\\anaconda3\\lib\\site-packages (from pathy->spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (2.2.1)\n",
      "Requirement already satisfied: six in d:\\programdata\\anaconda3\\lib\\site-packages (from packaging>=20.0->spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (1.15.0)\n",
      "Requirement already satisfied: pyparsing>=2.0.2 in d:\\programdata\\anaconda3\\lib\\site-packages (from packaging>=20.0->spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (2.4.7)\n",
      "Requirement already satisfied: boto3 in d:\\programdata\\anaconda3\\lib\\site-packages (from smart-open<4.0.0,>=2.2.0->pathy->spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (1.17.7)\n",
      "Requirement already satisfied: botocore<1.21.0,>=1.20.7 in d:\\programdata\\anaconda3\\lib\\site-packages (from boto3->smart-open<4.0.0,>=2.2.0->pathy->spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (1.20.7)\n",
      "Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in d:\\programdata\\anaconda3\\lib\\site-packages (from boto3->smart-open<4.0.0,>=2.2.0->pathy->spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (0.10.0)\n",
      "Requirement already satisfied: s3transfer<0.4.0,>=0.3.0 in d:\\programdata\\anaconda3\\lib\\site-packages (from boto3->smart-open<4.0.0,>=2.2.0->pathy->spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (0.3.4)\n",
      "Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in d:\\programdata\\anaconda3\\lib\\site-packages (from botocore<1.21.0,>=1.20.7->boto3->smart-open<4.0.0,>=2.2.0->pathy->spacy<3.1.0,>=3.0.0->en-core-web-sm==3.0.0) (2.8.1)\n",
      "[!] As of spaCy v3.0, shortcuts like 'en' are deprecated. Pleaseuse the full\n",
      "pipeline package name 'en_core_web_sm' instead.\n",
      "[+] Download and installation successful\n",
      "You can now load the package via spacy.load('en_core_web_sm')\n"
     ]
    }
   ],
   "source": [
    "import gensim\n",
    "import pandas as pd\n",
    "\n",
    "import spacy\n",
    "import sys\n",
    "\n",
    "## Run once this chunk with the command below active. Then comment it out and run the notebook\n",
    "!{sys.executable} -m spacy download en\n",
    "\n",
    "nlp = spacy.load(\"en_core_web_sm\", disable=['parser', 'ner'])\n",
    "\n",
    "import nltk\n",
    "import re\n",
    "import pprint\n",
    "import os\n",
    "import sys\n",
    "\n",
    "\n",
    "from gensim import corpora\n",
    "\n",
    "import nltk\n",
    "from nltk import sent_tokenize\n",
    "from nltk.corpus import stopwords\n",
    "\n",
    "##nltk.download('stopwords')\n",
    "##stop_words = stopwords.words('english')\n",
    "\n",
    "import pyLDAvis\n",
    "import pyLDAvis.gensim_models as gensimvis # don't skip this\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "import logging\n",
    "logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.ERROR)\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\",category=DeprecationWarning)\n",
    "\n",
    "from pprint import pprint\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2. Pre-processing\n",
    "\n",
    "* Read two files exported from the database which contain the SE Glossary articles definitions and their titles.\n",
    "* In later versions, the **corresponding tables will be directly exported from the database**.\n",
    "* Merge by_id_ and discard records with duplicate titles and/or definitions.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 161,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>definition</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Arrival of tourist at a tourist accommodation ...</td>\n",
       "      <td>See      arrival disambiguation page     fo...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>Nights spent</td>\n",
       "      <td>A     night spent    or     tourism night   ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4</td>\n",
       "      <td>Supply side tourism statistics</td>\n",
       "      <td>Regulation (EU) No 692/2011 of 6 July 2011 e...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5</td>\n",
       "      <td>Tourist</td>\n",
       "      <td>For     tourism    statistics, a     travell...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7</td>\n",
       "      <td>Asia-Europe Meeting (ASEM)</td>\n",
       "      <td>The     Asia-Europe Meeting    , abbreviated...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1279</th>\n",
       "      <td>2879</td>\n",
       "      <td>Statistical territory</td>\n",
       "      <td>The     statistical territory    of the     ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1280</th>\n",
       "      <td>2881</td>\n",
       "      <td>Statistical indicator</td>\n",
       "      <td>A     statistical indicator    is the repres...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1281</th>\n",
       "      <td>2885</td>\n",
       "      <td>Tourism industries</td>\n",
       "      <td>For the article     Tourism industries - eco...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1282</th>\n",
       "      <td>2891</td>\n",
       "      <td>Sawnwood</td>\n",
       "      <td>Sawnwood    is wood that has been produced ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1283</th>\n",
       "      <td>2896</td>\n",
       "      <td>LeaMickatest</td>\n",
       "      <td>test</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1284 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        id                                              title  \\\n",
       "0        1  Arrival of tourist at a tourist accommodation ...   \n",
       "1        3                                    Nights spent      \n",
       "2        4                  Supply side tourism statistics      \n",
       "3        5                                         Tourist      \n",
       "4        7                         Asia-Europe Meeting (ASEM)   \n",
       "...    ...                                                ...   \n",
       "1279  2879                              Statistical territory   \n",
       "1280  2881                              Statistical indicator   \n",
       "1281  2885                                 Tourism industries   \n",
       "1282  2891                                           Sawnwood   \n",
       "1283  2896                                       LeaMickatest   \n",
       "\n",
       "                                             definition  \n",
       "0        See      arrival disambiguation page     fo...  \n",
       "1       A     night spent    or     tourism night   ...  \n",
       "2       Regulation (EU) No 692/2011 of 6 July 2011 e...  \n",
       "3       For     tourism    statistics, a     travell...  \n",
       "4       The     Asia-Europe Meeting    , abbreviated...  \n",
       "...                                                 ...  \n",
       "1279    The     statistical territory    of the     ...  \n",
       "1280    A     statistical indicator    is the repres...  \n",
       "1281    For the article     Tourism industries - eco...  \n",
       "1282     Sawnwood    is wood that has been produced ...  \n",
       "1283                                               test  \n",
       "\n",
       "[1284 rows x 3 columns]"
      ]
     },
     "execution_count": 161,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dat1= pd.read_csv(\"ESTAT_dat_concepts_2021_04_08.csv\",sep=\";\")\n",
    "dat2= pd.read_csv(\"ESTAT_dat_link_info_2021_04_08.csv\",sep=\";\")\n",
    "Gloss_concepts = pd.merge(dat1,dat2,on=['id'])\n",
    "del(dat1,dat2)\n",
    "\n",
    "Gloss_concepts = Gloss_concepts[['id','title','definition']]\n",
    "\n",
    "Gloss_concepts = Gloss_concepts.drop_duplicates(subset=[\"definition\"])\n",
    "Gloss_concepts = Gloss_concepts.dropna(axis=0,subset=[\"definition\"])\n",
    "Gloss_concepts = Gloss_concepts.drop_duplicates(subset=[\"title\"])\n",
    "Gloss_concepts = Gloss_concepts.dropna(axis=0,subset=[\"title\"])\n",
    "\n",
    "Gloss_concepts.reset_index(drop=True, inplace=True)\n",
    "Gloss_concepts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Pre-processing input data (cont).\n",
    "***\n",
    "\n",
    "Next we tokenize the texts - definitions in the articles, select tokens with minimum length 5, delete stop words and apply a simple pre-processing (convert to lowercase, drop accents). \n",
    "\n",
    "The result, _texts_ is a list with 1286 elements corresponding to the records in the dataframe _df_."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "First 10 texts: \n",
      " [['arrival', 'disambiguation', 'meanings', 'arrival', 'within', 'context', 'european', 'union', 'tourism', 'statistics', 'arrival', 'defined', 'tourist', 'arrives', 'tourist', 'accommodation', 'establishment', 'rented', 'accommodation', 'rented', 'accommodation', 'scope', 'tourism', 'regulation', 'however', 'collected', 'tourist', 'resident', 'resident', 'country', 'establishment', 'located', 'limit', 'applied', 'children', 'counted', 'adults', 'overnight', 'stays', 'children', 'charge'], ['night', 'spent', 'tourism', 'night', 'overnight', 'night', 'guest', 'tourist', 'resident', 'resident', 'actually', 'spends', 'sleeps', 'stays', 'tourist', 'accommodation', 'establishment', 'rented', 'accommodation'], ['regulation', 'establishes', 'common', 'framework', 'systematic', 'development', 'production', 'dissemination', 'european', 'statistics', 'tourism', 'purpose', 'member', 'states', 'transmit', 'harmonised', 'statistics', 'tourism', 'supply', 'demand', 'tourism', 'supply', 'statistics', 'accommodation', 'statistics', 'rented', 'accommodation', 'capacity', 'occupancy', 'tourist', 'accommodation', 'establishments', 'reporting', 'country', 'resident', 'resident', 'guests', 'tourism', 'demand', 'statistics', 'participation', 'tourism', 'residents', 'reporting', 'country', 'characteristics', 'tourism', 'trips', 'residents', 'reporting', 'country', 'characteristics', 'visits', 'residents', 'reporting', 'country', 'first', 'reference'], ['tourism', 'statistics', 'traveller', 'moves', 'different', 'geographic', 'locations', 'purpose', 'duration', 'visitor', 'traveller', 'taking', 'destination', 'outside', 'usual', 'environment', 'purpose', 'business', 'leisure', 'personal', 'purpose', 'employed', 'resident', 'entity', 'country', 'place', 'visited', 'these', 'trips', 'taken', 'visitors', 'qualify', 'tourism', 'trips', 'visitor', 'classified', 'tourist', 'overnight', 'visitor', 'includes', 'overnight', 'visitor', 'excursionist', 'include', 'overnight'], ['europe', 'meeting', 'abbreviated', 'informal', 'process', 'dialogue', 'operation', 'brings', 'countries', 'europe', 'oceania', 'following', 'partners', 'european', 'union', 'member', 'states', 'members', 'norway', 'switzerland', 'asean', 'members', 'brunei', 'darussalam', 'cambodia', 'indonesia', 'malaysia', 'myanmar', 'philippines', 'singapore', 'thailand', 'vietnam', 'asean', 'secretariat', 'partners', 'australia', 'bangladesh', 'china', 'india', 'japan', 'kazakhstan', 'republic', 'korea', 'mongolia', 'zealand', 'pakistan', 'russian', 'federation'], ['association', 'south', 'asian', 'nations', 'abbreviated', 'asean', 'established', 'august', 'bangkok', 'thailand', 'accelerate', 'economic', 'growth', 'social', 'progress', 'cultural', 'development', 'promote', 'peace', 'stability', 'region', 'world', 'economic', 'regions', 'outside', 'asean', 'orange', 'member', 'states', 'asean', 'brunei', 'brunei', 'darussalam', 'cambodia', 'kingdom', 'cambodia', 'indonesia', 'republic', 'indonesia', 'people', 'democratic', 'republic', 'malaysia', 'myanmar', 'burma', 'myanmar', 'philippines', 'republic', 'philippines', 'singapore', 'republic', 'singapore', 'thailand', 'kingdom', 'thailand', 'vietnam', 'socialist', 'republic', 'vietnam'], ['european', 'trade', 'association', 'abbreviated', 'organisation', 'established', 'seven', 'european', 'countries', 'promote', 'trade', 'economic', 'integration', 'benefit', 'member', 'states', 'original', 'signatories', 'norway', 'switzerland', 'withdrew', 'joining', 'european', 'union', 'finland', 'associate', 'member', 'member', 'iceland', 'joined', 'liechtenstein', 'currently', 'member', 'states', 'iceland', 'liechtenstein', 'norway', 'switzerland', 'association', 'responsible', 'management', 'trade', 'countries', 'participation', 'european', 'economic', 'includes', 'european', 'union', 'countries', 'iceland', 'liechtentstein', 'norway', 'switzerland', 'worldwide', 'network', 'trade', 'agreements'], ['european', 'union', 'abbreviated', 'economic', 'political', 'union', 'european', 'countries', 'established', 'november', 'treaty', 'european', 'union', 'maastricht', 'treaty', 'december', 'member', 'states', 'belgium', 'denmark', 'germany', 'ireland', 'greece', 'spain', 'france', 'italy', 'luxembourg', 'netherlands', 'portugal', 'united', 'kingdom', 'january', 'added', 'member', 'states', 'austria', 'finland', 'sweden', 'countries', 'joined', 'cyprus', 'czechia', 'estonia', 'hungary', 'latvia', 'lithuania', 'malta', 'poland', 'slovenia', 'slovakia', 'january', 'bulgaria', 'romania', 'members', 'latest', 'country', 'accede', 'croatia', 'january', 'united', 'kingdom', 'member', 'states', 'follows', 'alphabetical', 'order', 'second', 'austria', 'estonia', 'italy', 'portugal', 'belgium', 'finland', 'latvia', 'romania', 'bulgaria', 'france', 'lithuania', 'slovakia', 'croatia', 'germany', 'luxembourg', 'slovenia', 'cyprus', 'greece', 'malta', 'spain', 'czechia', 'hungary', 'netherlands', 'sweden', 'denmark', 'ireland', 'poland'], ['acronym', 'northeast', 'south', 'geographical', 'grouping', 'countries', 'partners', 'europe', 'meeting', 'partners', 'australia', 'bangladesh', 'china', 'india', 'japan', 'kazakhstan', 'republic', 'korea', 'mongolia', 'zealand', 'pakistan', 'russian', 'federation'], ['arrears', 'housing', 'housing', 'bills', 'loans', 'credit', 'repayment', 'statistical', 'indicator', 'referring', 'arrears', 'payment', 'housing', 'housing', 'bills', 'repayment', 'loans', 'credit', 'household', 'credit', 'loans', 'encompass', 'commercial', 'credit', 'loans', 'planned', 'scheduled', 'repayments', 'exception', 'mortgage', 'loans', 'dwelling', 'included', 'overdraft', 'facilities', 'credit', 'store', 'cards', 'repayments', 'planned', 'borrowing', 'friends', 'relatives', 'informal', 'credit', 'these', 'arrears', 'account', 'bills', 'credit', 'mortgage', 'repayment', 'schedule', 'months', 'financial', 'reasons', 'total', 'currently', 'arrears', 'includes', 'household', 'currently', 'schedule', 'housing', 'related', 'bills', 'payments', 'comprise', 'mortgage', 'repayment', 'dwelling', 'utility', 'bills', 'water', 'electricity', 'heating', 'other', 'loans', 'credit', 'repayment', 'account', 'loans', 'other', 'mortgage', 'repayment', 'dwelling', 'purchase', 'instalments', 'similar', 'order', 'catalogues', 'finance', 'minimum', 'credit', 'store', 'repayments', 'included', 'other', 'housing', 'household', 'bills', 'education', 'health', 'bills', 'covered', 'housing', 'related', 'bills']]\n",
      "\n",
      "Total texts:  1284 \n",
      "\n"
     ]
    }
   ],
   "source": [
    "#Tokenize texts and clean-up text.\n",
    "from gensim.parsing.preprocessing import remove_stopwords\n",
    "def sent_to_words(sentences):\n",
    "    for sentence in sentences:\n",
    "        sentence = remove_stopwords(sentence) ## remove stop words\n",
    "        tokens = gensim.utils.tokenize(sentence)\n",
    "        sentence = [token for token in tokens if len(token) >= 5] ##minimum length = 5 \n",
    "        yield(gensim.utils.simple_preprocess(str(sentence), deacc=True))  # to lower + deacc=True removes punctuations\n",
    "        \n",
    "texts = list(sent_to_words(Gloss_concepts['definition']))\n",
    "print('\\nFirst 10 texts: \\n',texts[:10])\n",
    "print('\\nTotal texts: ',len(texts),'\\n')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Creation of corpus and terms frequencies.\n",
    "***\n",
    "\n",
    "Next we create:\n",
    "* a dictionary from _texts_ with name _id2word_. This has 7258 unique tokens. \n",
    "* a mapping with name _corpus_ of the texts into lists with tuples: (word id, frequency in each text)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dictionary(7304 unique tokens: ['accommodation', 'adults', 'applied', 'arrival', 'arrives']...) \n",
      "\n",
      "First 10 texts:\n",
      "\n",
      "[[(0, 3), (1, 1), (2, 1), (3, 3), (4, 1), (5, 1), (6, 2), (7, 1), (8, 1), (9, 1), (10, 1), (11, 1), (12, 1), (13, 2), (14, 1), (15, 1), (16, 1), (17, 1), (18, 1), (19, 1), (20, 1), (21, 2), (22, 2), (23, 1), (24, 1), (25, 1), (26, 2), (27, 3), (28, 1), (29, 1)], [(0, 2), (13, 1), (19, 1), (21, 1), (22, 2), (25, 1), (26, 1), (27, 2), (30, 1), (31, 1), (32, 3), (33, 1), (34, 1), (35, 1)], [(0, 3), (10, 4), (14, 1), (20, 1), (21, 1), (22, 2), (24, 5), (26, 6), (27, 1), (36, 1), (37, 2), (38, 1), (39, 2), (40, 1), (41, 1), (42, 1), (43, 1), (44, 1), (45, 1), (46, 1), (47, 1), (48, 1), (49, 1), (50, 1), (51, 1), (52, 1), (53, 1), (54, 4), (55, 3), (56, 1), (57, 2), (58, 1), (59, 1), (60, 1), (61, 1)], [(10, 1), (19, 3), (22, 1), (24, 1), (26, 2), (27, 1), (52, 3), (60, 2), (62, 1), (63, 1), (64, 1), (65, 1), (66, 1), (67, 1), (68, 1), (69, 1), (70, 1), (71, 1), (72, 1), (73, 1), (74, 1), (75, 1), (76, 1), (77, 1), (78, 1), (79, 1), (80, 1), (81, 1), (82, 1), (83, 1), (84, 2), (85, 1), (86, 1), (87, 4), (88, 1)], [(14, 1), (28, 1), (48, 1), (56, 1), (89, 1), (90, 2), (91, 1), (92, 1), (93, 1), (94, 1), (95, 1), (96, 1), (97, 1), (98, 1), (99, 1), (100, 2), (101, 1), (102, 1), (103, 1), (104, 1), (105, 1), (106, 1), (107, 1), (108, 1), (109, 1), (110, 1), (111, 2), (112, 1), (113, 1), (114, 1), (115, 1), (116, 1), (117, 1), (118, 2), (119, 1), (120, 1), (121, 1), (122, 1), (123, 1), (124, 1), (125, 1), (126, 1), (127, 1), (128, 1)], [(40, 1), (48, 1), (56, 1), (77, 1), (89, 1), (90, 3), (94, 2), (95, 2), (98, 1), (104, 2), (109, 1), (113, 2), (119, 2), (121, 5), (124, 2), (126, 3), (127, 2), (129, 1), (130, 1), (131, 1), (132, 1), (133, 1), (134, 1), (135, 1), (136, 1), (137, 2), (138, 1), (139, 1), (140, 2), (141, 1), (142, 1), (143, 1), (144, 1), (145, 1), (146, 1), (147, 1), (148, 1), (149, 1), (150, 1), (151, 1), (152, 1), (153, 1)], [(14, 5), (28, 2), (48, 4), (50, 1), (56, 2), (73, 1), (89, 1), (97, 3), (114, 3), (125, 3), (131, 2), (137, 2), (138, 1), (146, 1), (154, 1), (155, 1), (156, 1), (157, 1), (158, 1), (159, 3), (160, 1), (161, 1), (162, 1), (163, 2), (164, 1), (165, 1), (166, 1), (167, 1), (168, 1), (169, 1), (170, 1), (171, 1), (172, 4), (173, 1), (174, 1)], [(10, 1), (14, 3), (28, 3), (48, 3), (56, 3), (89, 1), (97, 2), (111, 1), (137, 1), (138, 1), (140, 2), (158, 2), (161, 1), (175, 1), (176, 1), (177, 1), (178, 2), (179, 2), (180, 2), (181, 2), (182, 2), (183, 2), (184, 1), (185, 2), (186, 2), (187, 1), (188, 2), (189, 2), (190, 2), (191, 2), (192, 2), (193, 2), (194, 3), (195, 1), (196, 2), (197, 2), (198, 2), (199, 1), (200, 2), (201, 2), (202, 1), (203, 1), (204, 2), (205, 1), (206, 2), (207, 2), (208, 1), (209, 2), (210, 2), (211, 2), (212, 2), (213, 2), (214, 2)], [(91, 1), (92, 1), (96, 1), (97, 1), (100, 1), (101, 1), (103, 1), (106, 1), (107, 1), (108, 1), (110, 1), (112, 1), (117, 1), (118, 2), (121, 1), (122, 1), (128, 1), (151, 1), (215, 1), (216, 1), (217, 1), (218, 1)], [(73, 1), (83, 1), (105, 1), (157, 2), (203, 1), (219, 2), (220, 4), (221, 8), (222, 1), (223, 1), (224, 1), (225, 1), (226, 1), (227, 1), (228, 9), (229, 3), (230, 1), (231, 1), (232, 1), (233, 1), (234, 1), (235, 1), (236, 1), (237, 1), (238, 1), (239, 1), (240, 3), (241, 7), (242, 2), (243, 1), (244, 1), (245, 7), (246, 1), (247, 1), (248, 4), (249, 3), (250, 1), (251, 1), (252, 1), (253, 2), (254, 1), (255, 1), (256, 1), (257, 2), (258, 1), (259, 6), (260, 3), (261, 2), (262, 1), (263, 1), (264, 1), (265, 2), (266, 1), (267, 1), (268, 1)]]\n",
      "\n",
      "Total texts:  1284\n"
     ]
    }
   ],
   "source": [
    "#Create Dictionary\n",
    "id2word = corpora.Dictionary(texts) #Gensim creates a unique id for each word in the document. \n",
    "print(id2word,'\\n')\n",
    "\n",
    "## The produced corpus shown above is a mapping of (word_id, word_frequency).\n",
    "\n",
    "#Alternatively:\n",
    "corpus = [id2word.doc2bow(text) for text in texts] #corpus package automatically creates a set of corpus reader instances that can be used to access the corpora in the NLTK data package.\n",
    "\n",
    "print('First 10 texts:\\n')\n",
    "print(corpus[:10])\n",
    "print('\\nTotal texts: ',len(corpus))\n",
    "\n",
    "#Human readable format of corpus (term-frequency)\n",
    "#[[(id2word[id], freq) for id, freq in cp] for cp in corpus[:1]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. Dominant topic in each document.\n",
    "***\n",
    "Αt this stage we build the  _lda_model_, through which the dominant topic for each document will be extracted and we show the weight of the topic and the keywords. We define a dataframe _df_topic_sents_keywords_  in order to store these dominant topics. \n",
    "\n",
    "In function _format_topics_sentences()_ below, the list _ldamodel[corpus]_ has one nested list element per text. Each nested list contains tuples (topic, contribution). We sort each nested list by descending contribution to find the dominant topic and then, we retrieve for this topic, the list _wp_ of tuples (word, probability) for the most probable words, using the function _ldamodel.show_topic()_. We join these words into a list and put the result in column 'Topic_Keywords' of the dataframe.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 164,
   "metadata": {},
   "outputs": [],
   "source": [
    "#What is the Dominant topic and its percentage contribution in each document.\n",
    "\n",
    "def format_topics_sentences(ldamodel=None, corpus=corpus):\n",
    "    # Init output\n",
    "    sent_topics_df = pd.DataFrame()\n",
    "\n",
    "    # Get main topic in each document\n",
    "    for i, row_list in enumerate(ldamodel[corpus]): \n",
    "        row = row_list[0] if ldamodel.per_word_topics else row_list ## ldamodel.per_word_topics is False          \n",
    "        #print(row)\n",
    "        row = sorted(row, key=lambda x: (x[1]), reverse=True) ## sort the nested list by descending contribution\n",
    "        # Get the Dominant topic, Perc Contribution and Keywords for each document\n",
    "        topic_num, prop_topic = row[0]\n",
    "        wp = ldamodel.show_topic(topic_num)\n",
    "        topic_keywords = \", \".join([word for word, prop in wp])\n",
    "        sent_topics_df = sent_topics_df.append(pd.Series([int(topic_num), round(prop_topic,4), topic_keywords]), ignore_index=True)\n",
    "    sent_topics_df.columns = ['Dominant_Topic', 'Perc_Contribution', 'Topic_Keywords']\n",
    "\n",
    "    # Add original tokenized text and title to the end of the output\n",
    "    contents = pd.Series(texts)\n",
    "    sent_topics_df = pd.concat([sent_topics_df, contents], axis=1)\n",
    "    sent_topics_df = pd.concat([sent_topics_df, Gloss_concepts['id'],Gloss_concepts['title']], axis=1)\n",
    "    return(sent_topics_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 165,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Build LDA model\n",
    "lda_model = gensim.models.ldamodel.LdaModel(corpus=corpus,\n",
    "                                           id2word=id2word,\n",
    "                                           num_topics=20, \n",
    "                                           random_state=100,\n",
    "                                           update_every=1,\n",
    "                                           chunksize=100,\n",
    "                                           passes=10,\n",
    "                                           alpha='auto',\n",
    "                                           per_word_topics=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 166,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Text id</th>\n",
       "      <th>Text title</th>\n",
       "      <th>Text tokenized definition</th>\n",
       "      <th>Dominant_Topic</th>\n",
       "      <th>Perc_Contribution</th>\n",
       "      <th>Topic_Keywords</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Arrival of tourist at a tourist accommodation ...</td>\n",
       "      <td>[arrival, disambiguation, meanings, arrival, w...</td>\n",
       "      <td>17</td>\n",
       "      <td>0.3693</td>\n",
       "      <td>employed, place, stock, private, similar, acco...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>Nights spent</td>\n",
       "      <td>[night, spent, tourism, night, overnight, nigh...</td>\n",
       "      <td>17</td>\n",
       "      <td>0.4157</td>\n",
       "      <td>employed, place, stock, private, similar, acco...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4</td>\n",
       "      <td>Supply side tourism statistics</td>\n",
       "      <td>[regulation, establishes, common, framework, s...</td>\n",
       "      <td>17</td>\n",
       "      <td>0.5458</td>\n",
       "      <td>employed, place, stock, private, similar, acco...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5</td>\n",
       "      <td>Tourist</td>\n",
       "      <td>[tourism, statistics, traveller, moves, differ...</td>\n",
       "      <td>17</td>\n",
       "      <td>0.7203</td>\n",
       "      <td>employed, place, stock, private, similar, acco...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7</td>\n",
       "      <td>Asia-Europe Meeting (ASEM)</td>\n",
       "      <td>[europe, meeting, abbreviated, informal, proce...</td>\n",
       "      <td>7</td>\n",
       "      <td>0.7317</td>\n",
       "      <td>european, member, countries, states, union, ec...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1279</th>\n",
       "      <td>2879</td>\n",
       "      <td>Statistical territory</td>\n",
       "      <td>[statistical, territory, european, union, cont...</td>\n",
       "      <td>3</td>\n",
       "      <td>0.2045</td>\n",
       "      <td>statistics, level, statistical, international,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1280</th>\n",
       "      <td>2881</td>\n",
       "      <td>Statistical indicator</td>\n",
       "      <td>[statistical, indicator, representation, stati...</td>\n",
       "      <td>3</td>\n",
       "      <td>0.2126</td>\n",
       "      <td>statistics, level, statistical, international,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1281</th>\n",
       "      <td>2885</td>\n",
       "      <td>Tourism industries</td>\n",
       "      <td>[article, tourism, industries, economic, analy...</td>\n",
       "      <td>10</td>\n",
       "      <td>0.2471</td>\n",
       "      <td>activities, water, households, energy, prices,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1282</th>\n",
       "      <td>2891</td>\n",
       "      <td>Sawnwood</td>\n",
       "      <td>[sawnwood, produced, sawing, lengthways, profi...</td>\n",
       "      <td>0</td>\n",
       "      <td>0.1992</td>\n",
       "      <td>activities, activity, market, product, product...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1283</th>\n",
       "      <td>2896</td>\n",
       "      <td>LeaMickatest</td>\n",
       "      <td>[]</td>\n",
       "      <td>3</td>\n",
       "      <td>0.1382</td>\n",
       "      <td>statistics, level, statistical, international,...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1284 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      Text id                                         Text title  \\\n",
       "0           1  Arrival of tourist at a tourist accommodation ...   \n",
       "1           3                                    Nights spent      \n",
       "2           4                  Supply side tourism statistics      \n",
       "3           5                                         Tourist      \n",
       "4           7                         Asia-Europe Meeting (ASEM)   \n",
       "...       ...                                                ...   \n",
       "1279     2879                              Statistical territory   \n",
       "1280     2881                              Statistical indicator   \n",
       "1281     2885                                 Tourism industries   \n",
       "1282     2891                                           Sawnwood   \n",
       "1283     2896                                       LeaMickatest   \n",
       "\n",
       "                              Text tokenized definition  Dominant_Topic  \\\n",
       "0     [arrival, disambiguation, meanings, arrival, w...              17   \n",
       "1     [night, spent, tourism, night, overnight, nigh...              17   \n",
       "2     [regulation, establishes, common, framework, s...              17   \n",
       "3     [tourism, statistics, traveller, moves, differ...              17   \n",
       "4     [europe, meeting, abbreviated, informal, proce...               7   \n",
       "...                                                 ...             ...   \n",
       "1279  [statistical, territory, european, union, cont...               3   \n",
       "1280  [statistical, indicator, representation, stati...               3   \n",
       "1281  [article, tourism, industries, economic, analy...              10   \n",
       "1282  [sawnwood, produced, sawing, lengthways, profi...               0   \n",
       "1283                                                 []               3   \n",
       "\n",
       "      Perc_Contribution                                     Topic_Keywords  \n",
       "0                0.3693  employed, place, stock, private, similar, acco...  \n",
       "1                0.4157  employed, place, stock, private, similar, acco...  \n",
       "2                0.5458  employed, place, stock, private, similar, acco...  \n",
       "3                0.7203  employed, place, stock, private, similar, acco...  \n",
       "4                0.7317  european, member, countries, states, union, ec...  \n",
       "...                 ...                                                ...  \n",
       "1279             0.2045  statistics, level, statistical, international,...  \n",
       "1280             0.2126  statistics, level, statistical, international,...  \n",
       "1281             0.2471  activities, water, households, energy, prices,...  \n",
       "1282             0.1992  activities, activity, market, product, product...  \n",
       "1283             0.1382  statistics, level, statistical, international,...  \n",
       "\n",
       "[1284 rows x 6 columns]"
      ]
     },
     "execution_count": 166,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_topic_sents_keywords = format_topics_sentences(ldamodel=lda_model, corpus=corpus)\n",
    "df_topic_sents_keywords.rename(columns = {0:'Text tokenized definition'}, inplace = True)\n",
    "df_topic_sents_keywords.rename(columns = {'id':'Text id','title':'Text title'}, inplace = True)\n",
    "df_topic_sents_keywords = df_topic_sents_keywords[['Text id','Text title','Text tokenized definition','Dominant_Topic', 'Perc_Contribution', 'Topic_Keywords']]\n",
    "df_topic_sents_keywords['Dominant_Topic'] = df_topic_sents_keywords['Dominant_Topic'].astype(int)\n",
    "df_topic_sents_keywords\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. Most representative document for each topic.\n",
    "***\n",
    "\n",
    "**Need to add the document id**\n",
    "\n",
    "We want to take samples of documents that best represent a given topic, to have a complete analysis.This code receives the most exemplary document for each topic. We create the _sent_topics_sorteddf_mallet_ and these Topic Keywords are presented below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 167,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Group top 5 sentences under each topic\n",
    "sent_topics_sorteddf_mallet = pd.DataFrame()\n",
    "\n",
    "sent_topics_outdf_grpd = df_topic_sents_keywords[['Text id','Text title','Dominant_Topic', 'Perc_Contribution', 'Topic_Keywords']].groupby('Dominant_Topic')\n",
    "\n",
    "for i, grp in sent_topics_outdf_grpd:\n",
    "    sent_topics_sorteddf_mallet = pd.concat([sent_topics_sorteddf_mallet, \n",
    "                                             grp.sort_values(['Perc_Contribution'], ascending=[0]).head(1)], \n",
    "                                            axis=0)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 168,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Topic_Num</th>\n",
       "      <th>Topic_Perc_Contrib</th>\n",
       "      <th>Topic Keywords</th>\n",
       "      <th>Text id</th>\n",
       "      <th>Text title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0.5951</td>\n",
       "      <td>activities, activity, market, product, product...</td>\n",
       "      <td>133</td>\n",
       "      <td>Capital innovation expenditures</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0.5631</td>\n",
       "      <td>force, nitrogen, parties, condition, fruit, ag...</td>\n",
       "      <td>119</td>\n",
       "      <td>Greenhouse gas (GHG)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>0.5355</td>\n",
       "      <td>output, short, environmental, trade, changes, ...</td>\n",
       "      <td>58</td>\n",
       "      <td>Agri-environmental indicators (AEI)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>0.7068</td>\n",
       "      <td>statistics, level, statistical, international,...</td>\n",
       "      <td>1699</td>\n",
       "      <td>Urban-rural typology</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>0.4864</td>\n",
       "      <td>spent, treatment, company, month, monthly, cre...</td>\n",
       "      <td>15</td>\n",
       "      <td>Credit or store card unclear balance</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5</td>\n",
       "      <td>0.6679</td>\n",
       "      <td>protection, property, release, power, employee...</td>\n",
       "      <td>11</td>\n",
       "      <td>European Union (EU)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>6</td>\n",
       "      <td>0.4089</td>\n",
       "      <td>mainly, housing, consumer, building, large, or...</td>\n",
       "      <td>146</td>\n",
       "      <td>Retail trade</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7</td>\n",
       "      <td>0.7964</td>\n",
       "      <td>european, member, countries, states, union, ec...</td>\n",
       "      <td>10</td>\n",
       "      <td>European Free Trade Association (EFTA)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>8</td>\n",
       "      <td>0.7691</td>\n",
       "      <td>number, average, death, values, total, point, ...</td>\n",
       "      <td>163</td>\n",
       "      <td>Life expectancy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>9</td>\n",
       "      <td>0.4809</td>\n",
       "      <td>education, sector, public, groups, electricity...</td>\n",
       "      <td>24</td>\n",
       "      <td>Built-up land</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>10</td>\n",
       "      <td>0.8855</td>\n",
       "      <td>activities, water, households, energy, prices,...</td>\n",
       "      <td>155</td>\n",
       "      <td>Recovery of waste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>11</td>\n",
       "      <td>0.7517</td>\n",
       "      <td>value, period, reference, available, income, i...</td>\n",
       "      <td>165</td>\n",
       "      <td>Equivalised disposable income</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>12</td>\n",
       "      <td>0.4430</td>\n",
       "      <td>surface, cover, placed, processed, covered, fo...</td>\n",
       "      <td>113</td>\n",
       "      <td>Cover of storage facilities for manure</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>13</td>\n",
       "      <td>0.8661</td>\n",
       "      <td>agricultural, holding, production, manure, liv...</td>\n",
       "      <td>50</td>\n",
       "      <td>Arable land</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>14</td>\n",
       "      <td>0.6554</td>\n",
       "      <td>financial, gross, assets, units, capital, inco...</td>\n",
       "      <td>2821</td>\n",
       "      <td>Financial instruments</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>15</td>\n",
       "      <td>0.2866</td>\n",
       "      <td>hours, growth, quality, necessary, require, ob...</td>\n",
       "      <td>157</td>\n",
       "      <td>Wastewater</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>16</td>\n",
       "      <td>0.7515</td>\n",
       "      <td>persons, health, people, employment, human, li...</td>\n",
       "      <td>138</td>\n",
       "      <td>Activity limitation</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>17</td>\n",
       "      <td>0.7203</td>\n",
       "      <td>employed, place, stock, private, similar, acco...</td>\n",
       "      <td>5</td>\n",
       "      <td>Tourist</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>18</td>\n",
       "      <td>0.5439</td>\n",
       "      <td>social, labour, taxes, knowledge, costs, contr...</td>\n",
       "      <td>722</td>\n",
       "      <td>Personnel costs</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>19</td>\n",
       "      <td>0.7513</td>\n",
       "      <td>goods, services, accounts, transport, national...</td>\n",
       "      <td>20</td>\n",
       "      <td>Export</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Topic_Num  Topic_Perc_Contrib  \\\n",
       "0           0              0.5951   \n",
       "1           1              0.5631   \n",
       "2           2              0.5355   \n",
       "3           3              0.7068   \n",
       "4           4              0.4864   \n",
       "5           5              0.6679   \n",
       "6           6              0.4089   \n",
       "7           7              0.7964   \n",
       "8           8              0.7691   \n",
       "9           9              0.4809   \n",
       "10         10              0.8855   \n",
       "11         11              0.7517   \n",
       "12         12              0.4430   \n",
       "13         13              0.8661   \n",
       "14         14              0.6554   \n",
       "15         15              0.2866   \n",
       "16         16              0.7515   \n",
       "17         17              0.7203   \n",
       "18         18              0.5439   \n",
       "19         19              0.7513   \n",
       "\n",
       "                                       Topic Keywords  Text id  \\\n",
       "0   activities, activity, market, product, product...      133   \n",
       "1   force, nitrogen, parties, condition, fruit, ag...      119   \n",
       "2   output, short, environmental, trade, changes, ...       58   \n",
       "3   statistics, level, statistical, international,...     1699   \n",
       "4   spent, treatment, company, month, monthly, cre...       15   \n",
       "5   protection, property, release, power, employee...       11   \n",
       "6   mainly, housing, consumer, building, large, or...      146   \n",
       "7   european, member, countries, states, union, ec...       10   \n",
       "8   number, average, death, values, total, point, ...      163   \n",
       "9   education, sector, public, groups, electricity...       24   \n",
       "10  activities, water, households, energy, prices,...      155   \n",
       "11  value, period, reference, available, income, i...      165   \n",
       "12  surface, cover, placed, processed, covered, fo...      113   \n",
       "13  agricultural, holding, production, manure, liv...       50   \n",
       "14  financial, gross, assets, units, capital, inco...     2821   \n",
       "15  hours, growth, quality, necessary, require, ob...      157   \n",
       "16  persons, health, people, employment, human, li...      138   \n",
       "17  employed, place, stock, private, similar, acco...        5   \n",
       "18  social, labour, taxes, knowledge, costs, contr...      722   \n",
       "19  goods, services, accounts, transport, national...       20   \n",
       "\n",
       "                                      Text title  \n",
       "0                Capital innovation expenditures  \n",
       "1                        Greenhouse gas (GHG)     \n",
       "2         Agri-environmental indicators (AEI)     \n",
       "3                           Urban-rural typology  \n",
       "4        Credit or store card unclear balance     \n",
       "5                         European Union (EU)     \n",
       "6                                Retail trade     \n",
       "7      European Free Trade Association (EFTA)     \n",
       "8                             Life expectancy     \n",
       "9                               Built-up land     \n",
       "10                          Recovery of waste     \n",
       "11              Equivalised disposable income     \n",
       "12     Cover of storage facilities for manure     \n",
       "13                                   Arable land  \n",
       "14                         Financial instruments  \n",
       "15                                 Wastewater     \n",
       "16                        Activity limitation     \n",
       "17                                    Tourist     \n",
       "18                            Personnel costs     \n",
       "19                                     Export     "
      ]
     },
     "execution_count": 168,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Find the most representative document for each topic\n",
    "# Reset Index    \n",
    "sent_topics_sorteddf_mallet.reset_index(drop=True, inplace=True)\n",
    "\n",
    "# Format\n",
    "sent_topics_sorteddf_mallet.columns = ['Text id','Text title','Topic_Num', 'Topic_Perc_Contrib', 'Topic Keywords' ]\n",
    "sent_topics_sorteddf_mallet = sent_topics_sorteddf_mallet[['Topic_Num', 'Topic_Perc_Contrib', 'Topic Keywords','Text id','Text title']]\n",
    "\n",
    "# Show\n",
    "sent_topics_sorteddf_mallet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7. The topics as a mix of keywords.\n",
    "***\n",
    "\n",
    "The above LDA model is built with 20 different topics where each topic is a combination of keywords and each keyword contributes with a certain weight to the topic.\n",
    "Finally,we want to understand the volume and distribution of topics in order to judge how widely it was discussed,so we define the _df_dominant_topics_.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 169,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(0,\n",
      "  '0.106*\"activities\" + 0.089*\"activity\" + 0.076*\"market\" + 0.070*\"product\" + '\n",
      "  '0.062*\"products\" + 0.059*\"innovation\" + 0.040*\"process\" + '\n",
      "  '0.027*\"classification\" + 0.025*\"production\" + 0.023*\"categories\"'),\n",
      " (1,\n",
      "  '0.081*\"force\" + 0.045*\"nitrogen\" + 0.031*\"parties\" + 0.031*\"condition\" + '\n",
      "  '0.029*\"fruit\" + 0.028*\"agreement\" + 0.022*\"nurseries\" + 0.020*\"young\" + '\n",
      "  '0.016*\"accident\" + 0.016*\"gases\"'),\n",
      " (2,\n",
      "  '0.079*\"output\" + 0.047*\"short\" + 0.044*\"environmental\" + 0.042*\"trade\" + '\n",
      "  '0.039*\"changes\" + 0.039*\"indicators\" + 0.033*\"indicator\" + 0.029*\"measure\" '\n",
      "  '+ 0.025*\"secondary\" + 0.023*\"producers\"'),\n",
      " (3,\n",
      "  '0.042*\"statistics\" + 0.042*\"level\" + 0.036*\"statistical\" + '\n",
      "  '0.030*\"international\" + 0.029*\"country\" + 0.028*\"defined\" + 0.027*\"based\" + '\n",
      "  '0.027*\"population\" + 0.024*\"regions\" + 0.024*\"national\"'),\n",
      " (4,\n",
      "  '0.060*\"spent\" + 0.057*\"treatment\" + 0.045*\"company\" + 0.043*\"month\" + '\n",
      "  '0.040*\"monthly\" + 0.039*\"credit\" + 0.033*\"issued\" + 0.026*\"money\" + '\n",
      "  '0.021*\"roads\" + 0.018*\"vegetation\"'),\n",
      " (5,\n",
      "  '0.062*\"protection\" + 0.028*\"property\" + 0.025*\"release\" + 0.024*\"power\" + '\n",
      "  '0.024*\"employee\" + 0.023*\"greater\" + 0.023*\"january\" + 0.020*\"identified\" + '\n",
      "  '0.019*\"motor\" + 0.019*\"required\"'),\n",
      " (6,\n",
      "  '0.061*\"mainly\" + 0.055*\"housing\" + 0.039*\"consumer\" + 0.035*\"building\" + '\n",
      "  '0.033*\"large\" + 0.027*\"organisms\" + 0.025*\"floor\" + 0.021*\"measurement\" + '\n",
      "  '0.021*\"section\" + 0.020*\"litter\"'),\n",
      " (7,\n",
      "  '0.102*\"european\" + 0.063*\"member\" + 0.062*\"countries\" + 0.053*\"states\" + '\n",
      "  '0.049*\"union\" + 0.033*\"economic\" + 0.022*\"tables\" + 0.022*\"abbreviated\" + '\n",
      "  '0.021*\"commission\" + 0.018*\"state\"'),\n",
      " (8,\n",
      "  '0.148*\"number\" + 0.055*\"average\" + 0.042*\"death\" + 0.037*\"values\" + '\n",
      "  '0.034*\"total\" + 0.034*\"point\" + 0.029*\"rates\" + 0.028*\"years\" + '\n",
      "  '0.027*\"change\" + 0.024*\"period\"'),\n",
      " (9,\n",
      "  '0.076*\"education\" + 0.066*\"sector\" + 0.064*\"public\" + 0.036*\"groups\" + '\n",
      "  '0.030*\"electricity\" + 0.029*\"refer\" + 0.027*\"providing\" + '\n",
      "  '0.024*\"information\" + 0.023*\"technical\" + 0.022*\"distribution\"'),\n",
      " (10,\n",
      "  '0.038*\"activities\" + 0.036*\"water\" + 0.035*\"households\" + 0.033*\"energy\" + '\n",
      "  '0.026*\"prices\" + 0.026*\"development\" + 0.024*\"resources\" + 0.022*\"products\" '\n",
      "  '+ 0.021*\"domestic\" + 0.021*\"waste\"'),\n",
      " (11,\n",
      "  '0.042*\"value\" + 0.039*\"period\" + 0.035*\"reference\" + 0.031*\"available\" + '\n",
      "  '0.029*\"income\" + 0.026*\"information\" + 0.026*\"household\" + 0.024*\"survey\" + '\n",
      "  '0.023*\"group\" + 0.018*\"collected\"'),\n",
      " (12,\n",
      "  '0.080*\"surface\" + 0.075*\"cover\" + 0.042*\"placed\" + 0.032*\"processed\" + '\n",
      "  '0.027*\"covered\" + 0.024*\"forms\" + 0.023*\"holiday\" + 0.022*\"ground\" + '\n",
      "  '0.020*\"store\" + 0.020*\"partially\"'),\n",
      " (13,\n",
      "  '0.047*\"agricultural\" + 0.047*\"holding\" + 0.034*\"production\" + '\n",
      "  '0.029*\"manure\" + 0.027*\"livestock\" + 0.026*\"crops\" + 0.019*\"vehicle\" + '\n",
      "  '0.018*\"plants\" + 0.018*\"holdings\" + 0.016*\"includes\"'),\n",
      " (14,\n",
      "  '0.121*\"financial\" + 0.071*\"gross\" + 0.064*\"assets\" + 0.048*\"units\" + '\n",
      "  '0.047*\"capital\" + 0.041*\"income\" + 0.034*\"total\" + 0.032*\"production\" + '\n",
      "  '0.031*\"consumption\" + 0.029*\"expenditure\"'),\n",
      " (15,\n",
      "  '0.118*\"hours\" + 0.057*\"growth\" + 0.036*\"quality\" + 0.026*\"necessary\" + '\n",
      "  '0.026*\"require\" + 0.023*\"observation\" + 0.022*\"progress\" + 0.021*\"round\" + '\n",
      "  '0.018*\"initial\" + 0.018*\"quantity\"'),\n",
      " (16,\n",
      "  '0.072*\"persons\" + 0.062*\"health\" + 0.056*\"people\" + 0.053*\"employment\" + '\n",
      "  '0.035*\"human\" + 0.026*\"living\" + 0.022*\"conditions\" + 0.019*\"according\" + '\n",
      "  '0.017*\"indicator\" + 0.017*\"needs\"'),\n",
      " (17,\n",
      "  '0.067*\"employed\" + 0.043*\"place\" + 0.041*\"stock\" + 0.041*\"private\" + '\n",
      "  '0.037*\"similar\" + 0.031*\"accommodation\" + 0.030*\"tourism\" + 0.024*\"outside\" '\n",
      "  '+ 0.019*\"relative\" + 0.016*\"loans\"'),\n",
      " (18,\n",
      "  '0.099*\"social\" + 0.075*\"labour\" + 0.065*\"taxes\" + 0.036*\"knowledge\" + '\n",
      "  '0.031*\"costs\" + 0.031*\"contributions\" + 0.027*\"benefits\" + 0.026*\"payments\" '\n",
      "  '+ 0.025*\"profit\" + 0.024*\"market\"'),\n",
      " (19,\n",
      "  '0.090*\"goods\" + 0.087*\"services\" + 0.051*\"accounts\" + 0.050*\"transport\" + '\n",
      "  '0.036*\"national\" + 0.034*\"enterprise\" + 0.031*\"economic\" + 0.028*\"service\" '\n",
      "  '+ 0.024*\"production\" + 0.022*\"business\"')]\n"
     ]
    }
   ],
   "source": [
    "#Print the Keyword in the 20 topics\n",
    "\n",
    "from pprint import pprint\n",
    "pprint(lda_model.print_topics())\n",
    "doc_lda = lda_model[corpus]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 8. Visualization of the topics.\n",
    "***\n",
    "\n",
    "To visualize the fitted LDA model we use the _pyLDAvis_ package. This is the Python porting of the R package _LDAvis_, see [LDAvis vignette](https://cran.r-project.org/web/packages/LDAvis/vignettes/details.pdf) for details and Chuang, Jason, Manning, Christopher D., and Heer, Jeffrey (2012). Termite: Visualization Techniques for Assessing Textual Topic Models, *Advanced Visual Interfaces* for the theory behind the visualization algorithm. The paper is available [here](https://dl.acm.org/doi/pdf/10.1145/2254556.2254572?casa_token=q2BavKP415QAAAAA:MhcYHzz4PJpC7dNkkm12GL-ohQRUXBgumPJ9l1t_5n3M4qVE1kdDqKGfPmtnR7qbale_ukS-2nJs).\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 170,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "<link rel=\"stylesheet\" type=\"text/css\" href=\"https://cdn.jsdelivr.net/gh/bmabey/pyLDAvis@3.3.1/pyLDAvis/js/ldavis.v1.0.0.css\">\n",
       "\n",
       "\n",
       "<div id=\"ldavis_el1005220031555744489202571526\"></div>\n",
       "<script type=\"text/javascript\">\n",
       "\n",
       "var ldavis_el1005220031555744489202571526_data = {\"mdsDat\": {\"x\": [-0.2263901679390172, -0.18702153101762523, -0.1459660502763586, -0.08771722475401025, -0.142750144934078, -0.1556245410580669, -0.12926269402496132, -0.0046032863509576, 0.028830919321525685, 0.049497580598196464, 0.11429964109867274, 0.03120978791722217, 0.05560601552457041, 0.0587180533610461, -0.010615077964956175, 0.13789379224308668, 0.14680249151272418, 0.15413847671199826, 0.15334858262632164, 0.1596053774046672], \"y\": [-0.08002468505777197, -0.2481885405429819, 0.05059039377686846, 0.1331277153012114, 0.232981350817987, 0.062444497281538114, -0.1022020810002827, -0.07220675805409094, 0.07836565915143329, 0.06187674822200108, -0.01843582448823447, -0.13117840753433233, -0.023241069142440722, 0.0056768666457604105, 0.13661086752411783, -0.027805843215952926, -0.019942783543616616, -0.023173393203752543, -0.0034241472951543886, -0.011850565642306719], \"topics\": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20], \"cluster\": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], \"Freq\": [12.857073835385686, 10.646288145557383, 9.788811592086827, 9.048593526869029, 8.212530220183268, 7.73224218632142, 6.017547381585915, 5.0008710469130335, 4.35304424182982, 3.9576736114428197, 3.7289634744413998, 3.421500940488833, 3.3436065574169076, 3.0425049912166284, 2.8858356218158043, 1.761422927295805, 1.3581225604177114, 1.1344044674782132, 0.8777542339929346, 0.8312084372605676]}, \"tinfo\": {\"Term\": [\"european\", \"financial\", \"goods\", \"number\", \"services\", \"activities\", \"social\", \"countries\", \"member\", \"gross\", \"states\", \"labour\", \"production\", \"assets\", \"level\", \"agricultural\", \"union\", \"holding\", \"market\", \"output\", \"income\", \"accounts\", \"statistics\", \"period\", \"persons\", \"activity\", \"taxes\", \"transport\", \"products\", \"national\", \"international\", \"regions\", \"level\", \"however\", \"fertilisers\", \"levels\", \"method\", \"collection\", \"intensive\", \"regional\", \"equal\", \"instance\", \"difference\", \"given\", \"organization\", \"phosphorus\", \"definition\", \"deficit\", \"rural\", \"removal\", \"positive\", \"proportion\", \"rules\", \"border\", \"comparable\", \"harmonised\", \"classifications\", \"cross\", \"having\", \"spending\", \"statistics\", \"based\", \"particular\", \"statistical\", \"means\", \"directly\", \"specific\", \"population\", \"country\", \"usually\", \"measures\", \"different\", \"defined\", \"context\", \"areas\", \"national\", \"eurostat\", \"classification\", \"person\", \"refers\", \"common\", \"example\", \"standard\", \"reference\", \"available\", \"household\", \"collected\", \"investment\", \"methods\", \"group\", \"disposable\", \"characteristics\", \"surveys\", \"irrigation\", \"input\", \"equivalised\", \"results\", \"class\", \"previous\", \"obtained\", \"accidents\", \"value\", \"major\", \"volume\", \"there\", \"survey\", \"second\", \"variables\", \"engaged\", \"practices\", \"frequency\", \"sample\", \"applicable\", \"months\", \"period\", \"basic\", \"monetary\", \"annual\", \"considered\", \"information\", \"carried\", \"provide\", \"individual\", \"income\", \"calculated\", \"structure\", \"order\", \"refers\", \"basis\", \"single\", \"certain\", \"including\", \"according\", \"agricultural\", \"holding\", \"manure\", \"livestock\", \"crops\", \"vehicle\", \"plants\", \"holdings\", \"permanent\", \"farming\", \"excludes\", \"animals\", \"animal\", \"arable\", \"worked\", \"vehicles\", \"seeds\", \"trees\", \"fodder\", \"construction\", \"harvested\", \"items\", \"removed\", \"actually\", \"poultry\", \"harvest\", \"grassland\", \"grown\", \"grazing\", \"house\", \"normally\", \"plant\", \"designed\", \"production\", \"types\", \"includes\", \"areas\", \"other\", \"including\", \"countries\", \"states\", \"union\", \"tables\", \"commission\", \"european\", \"geographical\", \"united\", \"region\", \"council\", \"actual\", \"operating\", \"inland\", \"organisation\", \"europe\", \"central\", \"nations\", \"network\", \"member\", \"norway\", \"multi\", \"customs\", \"policies\", \"effect\", \"responsible\", \"south\", \"iceland\", \"promote\", \"switzerland\", \"assessed\", \"additional\", \"world\", \"policy\", \"state\", \"possible\", \"global\", \"economic\", \"abbreviated\", \"trade\", \"established\", \"access\", \"members\", \"development\", \"energy\", \"prices\", \"households\", \"resources\", \"domestic\", \"waste\", \"research\", \"institutions\", \"organic\", \"directive\", \"industry\", \"science\", \"materials\", \"industries\", \"principal\", \"operations\", \"specified\", \"markets\", \"useful\", \"mineral\", \"residues\", \"paper\", \"comprises\", \"consumers\", \"sustainable\", \"consumed\", \"distinguish\", \"fuels\", \"matter\", \"fulfil\", \"application\", \"water\", \"material\", \"development\", \"excluding\", \"natural\", \"systems\", \"management\", \"industrial\", \"these\", \"activities\", \"related\", \"products\", \"economy\", \"terms\", \"following\", \"include\", \"consumption\", \"goods\", \"accounts\", \"transport\", \"enterprise\", \"service\", \"subsidies\", \"primary\", \"recorded\", \"services\", \"weight\", \"imports\", \"flows\", \"territory\", \"significant\", \"exports\", \"sales\", \"processing\", \"import\", \"intra\", \"ownership\", \"applies\", \"nomenclature\", \"record\", \"extra\", \"takes\", \"foreign\", \"abroad\", \"export\", \"payment\", \"interest\", \"control\", \"final\", \"legal\", \"resident\", \"business\", \"direct\", \"national\", \"economic\", \"production\", \"include\", \"change\", \"including\", \"financial\", \"gross\", \"assets\", \"units\", \"capital\", \"fixed\", \"liabilities\", \"expenditure\", \"transfers\", \"stocks\", \"account\", \"composed\", \"purchases\", \"balance\", \"coverage\", \"acquisition\", \"makes\", \"registered\", \"reported\", \"patents\", \"satellite\", \"transactions\", \"involving\", \"securities\", \"current\", \"determine\", \"programmes\", \"higher\", \"bodies\", \"respective\", \"measured\", \"consumption\", \"instruments\", \"produced\", \"income\", \"total\", \"added\", \"production\", \"physical\", \"general\", \"defined\", \"economy\", \"categories\", \"social\", \"labour\", \"taxes\", \"knowledge\", \"costs\", \"contributions\", \"benefits\", \"profit\", \"local\", \"calendar\", \"ratio\", \"security\", \"received\", \"pension\", \"workers\", \"wages\", \"depending\", \"schemes\", \"personnel\", \"functions\", \"salaries\", \"farmer\", \"inhabitants\", \"staff\", \"especially\", \"developing\", \"crude\", \"performed\", \"involved\", \"resource\", \"family\", \"payments\", \"funds\", \"support\", \"market\", \"total\", \"income\", \"other\", \"defined\", \"health\", \"people\", \"employment\", \"living\", \"needs\", \"share\", \"children\", \"implementation\", \"present\", \"developed\", \"degree\", \"intensity\", \"persons\", \"limited\", \"participation\", \"under\", \"instrument\", \"disability\", \"intellectual\", \"problems\", \"strategy\", \"defines\", \"illness\", \"project\", \"target\", \"objectives\", \"capable\", \"society\", \"human\", \"conditions\", \"examples\", \"referred\", \"classified\", \"status\", \"according\", \"indicator\", \"cases\", \"activities\", \"activity\", \"related\", \"output\", \"short\", \"environmental\", \"changes\", \"measure\", \"producers\", \"applied\", \"improved\", \"increase\", \"leading\", \"potential\", \"traffic\", \"presented\", \"combined\", \"million\", \"leave\", \"amounts\", \"impact\", \"reducing\", \"decisions\", \"safety\", \"secondary\", \"commonly\", \"tonnes\", \"alcohol\", \"track\", \"limit\", \"imported\", \"equivalents\", \"meaning\", \"environment\", \"associated\", \"indicators\", \"external\", \"indicator\", \"equivalent\", \"trade\", \"abbreviated\", \"these\", \"protection\", \"property\", \"release\", \"employee\", \"greater\", \"january\", \"identified\", \"motor\", \"required\", \"france\", \"created\", \"increasing\", \"treaty\", \"artificial\", \"better\", \"germany\", \"denmark\", \"risks\", \"internal\", \"spain\", \"earnings\", \"italy\", \"combustion\", \"ireland\", \"disposal\", \"chemical\", \"carrying\", \"sweden\", \"earlier\", \"substance\", \"power\", \"december\", \"rights\", \"temporary\", \"cover\", \"number\", \"average\", \"death\", \"values\", \"point\", \"rates\", \"expressed\", \"deaths\", \"causes\", \"cause\", \"disease\", \"birth\", \"mortality\", \"early\", \"chosen\", \"expectancy\", \"patterns\", \"immediate\", \"later\", \"starting\", \"abuse\", \"child\", \"largely\", \"medical\", \"factors\", \"causing\", \"geometric\", \"fatal\", \"subjected\", \"places\", \"effects\", \"observed\", \"three\", \"change\", \"years\", \"growth\", \"total\", \"calculated\", \"population\", \"period\", \"result\", \"related\", \"education\", \"sector\", \"public\", \"groups\", \"electricity\", \"refer\", \"distribution\", \"mixed\", \"generation\", \"field\", \"compiling\", \"processes\", \"users\", \"professional\", \"communication\", \"commercial\", \"division\", \"infrastructure\", \"involves\", \"generated\", \"residential\", \"technical\", \"recreational\", \"purchase\", \"specialised\", \"classify\", \"essential\", \"providing\", \"study\", \"diseases\", \"source\", \"distinguished\", \"purposes\", \"information\", \"funds\", \"includes\", \"power\", \"employed\", \"place\", \"private\", \"similar\", \"accommodation\", \"tourism\", \"relative\", \"loans\", \"reduce\", \"finance\", \"reasons\", \"dissemination\", \"active\", \"entity\", \"establishment\", \"freshwater\", \"structures\", \"destination\", \"stock\", \"supply\", \"borrowing\", \"located\", \"arrears\", \"duration\", \"dwelling\", \"bills\", \"capacity\", \"referring\", \"controlled\", \"first\", \"taking\", \"outside\", \"residents\", \"other\", \"includes\", \"country\", \"product\", \"innovation\", \"machinery\", \"performance\", \"marketing\", \"innovations\", \"software\", \"organisational\", \"comprise\", \"solely\", \"remaining\", \"specifically\", \"frequent\", \"acquired\", \"times\", \"activity\", \"latin\", \"numerical\", \"advantage\", \"process\", \"occurs\", \"categories\", \"conceived\", \"market\", \"preparations\", \"expenditures\", \"intended\", \"products\", \"activities\", \"modus\", \"equipment\", \"classification\", \"produced\", \"surplus\", \"production\", \"consist\", \"variable\", \"mainly\", \"housing\", \"consumer\", \"building\", \"organisms\", \"floor\", \"measurement\", \"section\", \"litter\", \"above\", \"assigned\", \"battery\", \"mentioned\", \"laying\", \"beginning\", \"quantities\", \"slatted\", \"purchased\", \"bought\", \"classes\", \"subdivided\", \"allowing\", \"large\", \"stable\", \"retail\", \"mixtures\", \"latest\", \"locations\", \"civil\", \"minor\", \"straw\", \"small\", \"range\", \"chemicals\", \"different\", \"nitrogen\", \"parties\", \"condition\", \"fruit\", \"agreement\", \"nurseries\", \"young\", \"accident\", \"gases\", \"carbon\", \"constitute\", \"greenhouse\", \"atmosphere\", \"climate\", \"citrus\", \"sequence\", \"instead\", \"kyoto\", \"force\", \"protocol\", \"grant\", \"methane\", \"dioxide\", \"berry\", \"fossil\", \"injury\", \"violence\", \"sulphur\", \"headings\", \"warming\", \"emissions\", \"convention\", \"directly\", \"change\", \"hours\", \"necessary\", \"require\", \"observation\", \"progress\", \"round\", \"initial\", \"quantity\", \"wastewater\", \"august\", \"caused\", \"maintain\", \"interaction\", \"retirement\", \"collecting\", \"provisions\", \"partners\", \"pasture\", \"defining\", \"moving\", \"completely\", \"republic\", \"object\", \"distinguishes\", \"periodic\", \"transformation\", \"descriptions\", \"ability\", \"sorting\", \"quality\", \"growth\", \"measured\", \"surface\", \"placed\", \"processed\", \"forms\", \"holiday\", \"ground\", \"store\", \"protected\", \"metal\", \"layer\", \"precipitation\", \"vertical\", \"sheets\", \"stores\", \"incorporating\", \"walls\", \"attached\", \"plastic\", \"adding\", \"cover\", \"tents\", \"partially\", \"impermeable\", \"fitted\", \"entering\", \"permeable\", \"floating\", \"slurry\", \"covered\", \"material\", \"spent\", \"treatment\", \"company\", \"month\", \"monthly\", \"credit\", \"issued\", \"money\", \"roads\", \"vegetation\", \"immediately\", \"debit\", \"difficulties\", \"lucas\", \"partly\", \"statements\", \"accurate\", \"crime\", \"hospital\", \"amount\", \"cards\", \"surgical\", \"orange\", \"various\", \"cropland\", \"minimum\", \"permits\", \"residence\", \"permit\", \"credits\", \"interest\", \"index\", \"months\"], \"Freq\": [625.0, 487.0, 469.0, 341.0, 457.0, 460.0, 334.0, 380.0, 395.0, 288.0, 322.0, 251.0, 535.0, 261.0, 367.0, 312.0, 300.0, 308.0, 261.0, 211.0, 420.0, 265.0, 382.0, 333.0, 218.0, 208.0, 219.0, 259.0, 240.0, 394.0, 261.5827126749504, 207.22273701631005, 364.5782983363497, 104.36227521124053, 93.67390268736648, 77.37161818168447, 77.15235711848631, 72.85573810739889, 68.11724818540965, 67.25128222572334, 54.34531569193823, 54.3152649686299, 51.40484599169092, 170.7322127666588, 48.044866748540734, 46.42884132021816, 125.53092984146707, 44.229211892894135, 42.80078264214578, 38.676553757166964, 38.10984224787131, 37.7656642950256, 43.43521669277205, 35.634705026710016, 35.18045887138861, 34.78101794354387, 31.01267571119259, 30.58451313398595, 64.81547515352052, 28.195388278402934, 365.5403069538142, 235.4430163200722, 103.23826412416109, 307.85975046157756, 144.93396394080887, 120.65558947161645, 119.95729161096624, 230.4773711977891, 248.60865418955348, 139.2346218161484, 92.67094574329876, 136.88422481015235, 243.41137275965264, 78.50150771062967, 150.84266532700016, 206.6297371471133, 107.17082930314353, 109.91588497997466, 91.04329192177724, 101.7537086026479, 80.32654708217675, 85.8767920050888, 77.43816021624697, 250.7495281498629, 218.98495598134102, 184.96997883672705, 127.48076244599417, 94.77315184073282, 89.58964230734887, 167.11433752703425, 72.15670751183451, 61.32933753085726, 57.192583040339244, 56.65337239463961, 54.25494998341458, 53.57906310100146, 53.31646332053507, 52.06280372286231, 51.54094900086533, 49.83200229584129, 49.746785375439245, 301.3153269302493, 45.947759870748705, 43.956177600539476, 43.51362149545973, 173.61348592509114, 40.4121944022586, 39.02712186810631, 36.18096317380753, 34.33468522881915, 33.55902505934303, 31.40567028573304, 31.305281894094374, 48.17688555341223, 276.9633301297621, 68.36288878425476, 68.6152146558627, 74.11432270990845, 112.57534887502403, 188.71430990752708, 66.86178757779385, 73.96009438100603, 67.81044507795427, 207.26807163637622, 100.81469568338846, 68.488908749806, 71.14591447486279, 103.17609212404787, 76.73763400377702, 69.94238423766221, 60.021891194154314, 63.149934028589094, 58.528736473833106, 311.4818432422969, 307.09640329687363, 191.398661515789, 174.4580000475496, 172.95370125948472, 125.4456326563481, 120.03628588404526, 117.44286631138151, 85.61523276589878, 80.98163616362653, 78.0742834708526, 75.75130066585064, 74.65412137794463, 72.84776322805672, 66.16638354297864, 62.2453041507958, 56.216033471956926, 54.94903253810564, 52.652071728608384, 52.46928799843613, 47.862777260845704, 46.41052552088557, 42.84222270215913, 42.57052335412335, 40.02761616770541, 38.41639547024474, 38.07495154920095, 36.20569671985888, 31.841959094229797, 30.960681050276875, 87.62641630333546, 49.99861997482083, 62.123778496977316, 226.35433335220043, 67.60178151634837, 104.89346549808614, 94.90998874199761, 60.5232851470526, 52.241185886994884, 378.9371951147804, 321.44153898616895, 299.7954324938545, 135.97223074103945, 128.01799647659305, 618.5038306410556, 96.99863619743006, 88.53547607938698, 78.61910844071413, 78.3245795989818, 77.47672402514786, 73.29143821203186, 68.57040289850347, 63.29291538370172, 61.322321200523916, 59.003223075089075, 52.87444366591737, 46.173830564783515, 384.58322800707066, 40.95525220943403, 37.419368807353614, 36.922407905360174, 33.89695092118945, 33.475880140902355, 29.84117991603932, 28.702768746092833, 27.101027965846338, 26.416245017712875, 24.79070150072713, 24.149163629207788, 39.40376479110538, 84.48753005085219, 94.88514081617197, 107.46696847404468, 71.95521818938754, 51.20135119388299, 201.74863940713544, 132.04043573772805, 99.42322032037205, 47.81416212894443, 41.85887169209802, 43.71179733021046, 52.62145587013861, 182.75763582608033, 145.45745361377703, 194.80598623552518, 133.92721629164046, 114.83535124282156, 114.00992205852071, 113.33629912650493, 97.70396899282423, 80.1346805407227, 71.40080425678403, 71.23870796793467, 63.029177271017936, 56.698106598913235, 52.334038484589826, 43.04235091146297, 42.65139107757236, 33.687792035668956, 31.991285095090348, 31.412744973521143, 31.361779540677382, 31.18643854873279, 30.660200128464478, 28.834457304934254, 28.696444626089566, 27.79088037139616, 25.075098116533372, 23.667196949732844, 23.32282697481294, 22.73306065931504, 22.628259765981593, 98.07992979036688, 199.21198907014428, 55.61890951886343, 143.30634352741686, 50.185713786990576, 74.36093384301375, 73.56219412044972, 57.93919478263664, 74.65499497765232, 74.45036380186818, 211.14510748488306, 92.43160253134043, 120.06302837302512, 74.61099301671086, 62.46802659309428, 76.20457907312893, 75.69544796688949, 60.11272134944846, 468.0828187196524, 264.706279188763, 257.9430492335096, 174.71457796925742, 145.64503371290147, 93.93444269968961, 88.28865543146445, 86.96579863916551, 451.47779387096034, 85.60860313040664, 76.60244162646629, 64.79208684145597, 63.213539426537835, 57.55347092278918, 52.97729845001664, 44.57261989233815, 42.92574760797432, 42.55327154193487, 36.97885905868909, 32.270958246754084, 27.507888495561424, 27.074155733288535, 23.422252556319673, 22.7588271054225, 21.21309707142495, 48.52678346315899, 18.55167388626941, 16.965304874824287, 45.95434029591892, 25.192773197759106, 42.90947656215807, 94.94295776209935, 100.20311786373365, 76.12366042849999, 112.6966633875613, 79.97405925626289, 184.6977073491214, 159.87117274169034, 126.28361932653189, 77.40480364920053, 49.45725642721764, 50.47504922159919, 486.6451589503028, 287.2806885658484, 260.10110838530113, 192.9360559139342, 189.4393356955095, 102.98298091565164, 87.11326399330588, 117.73275659562583, 79.27589288305018, 51.310542929281446, 94.50621576545582, 30.625902930306584, 25.32145142654708, 112.72965628310419, 24.481912701691, 23.598587922375664, 23.336286923232255, 22.060866567700295, 19.725443697678788, 19.62574063972664, 15.914724528605472, 88.75538292346802, 15.000933307731858, 14.229466966438306, 92.26337900466604, 11.094112779207236, 10.638176086933775, 50.77551827517116, 9.451407504180883, 9.277451010555966, 53.06272179735303, 124.9980204977307, 27.98967950701263, 65.93304418751225, 164.7306127727541, 138.8577096393465, 50.90032328296718, 130.65242967852294, 26.404937804572526, 45.7248150286415, 50.76992920585652, 32.7806921166309, 26.846111573510015, 333.42845632403254, 250.27899256540647, 218.44199191173698, 119.61314524074399, 105.15702444894721, 105.05016003020214, 89.0442115487086, 84.04987114046446, 64.63127060542998, 62.77393752090749, 60.72305402006559, 60.67980472911667, 49.18245582908747, 48.30315145242393, 38.96540901077375, 37.24377424682715, 36.91277004976189, 34.06749709168778, 33.46981288132954, 33.02043381938698, 25.195359224173867, 24.711416026091253, 24.44095958691663, 21.084071506123443, 20.976354346050258, 18.615527484790725, 17.948999249261743, 17.86574920575672, 17.529297459694284, 15.96232435919161, 64.81897989983155, 88.2188953812579, 52.10896723573875, 25.20129561572727, 79.57550927950359, 59.72308518367673, 47.51410646556682, 29.975513188317716, 25.44248709701511, 181.0184931678922, 164.31805982599832, 153.57784869820242, 74.94267747642455, 49.49312951137796, 48.29130226689498, 43.707649125324195, 43.67032602403642, 39.34912277500614, 39.105314040943746, 38.385840366858936, 36.404216702959275, 210.80215942029622, 30.534851103754036, 29.73283809945921, 27.701242753577482, 26.64388992521069, 23.87183056926822, 22.864422233329133, 19.309700678939077, 19.246971658045204, 18.60377827875375, 18.542524291184616, 17.50000336472828, 16.591770287842305, 16.218081130973413, 13.466389470211729, 13.443676675966271, 102.11942013137032, 65.63715581258344, 36.50161512463412, 29.07140868604981, 47.37881538422735, 47.70181641435644, 55.84365777849984, 50.98454182366566, 31.774115645653154, 43.75209317090224, 35.23809750023431, 23.198443189845587, 210.04541783934167, 124.1843606426031, 116.22179195693774, 104.21762700059116, 77.6178172967146, 61.524949581569736, 53.53992027714744, 46.925759981816825, 33.89174129462397, 32.86916761245027, 31.125562583324847, 29.18126799512155, 28.908237741865502, 28.66152443196036, 24.793101855577877, 23.102187569221993, 22.625137968921354, 20.894488707229907, 20.68753262068683, 19.96618485735817, 18.071633148412673, 65.39952250127901, 16.073132479634648, 15.37257055306778, 13.860796140118017, 13.751031793357717, 13.721954858718886, 13.174187434173371, 11.642281029240728, 13.037395443215525, 50.9351411101207, 32.27630961818917, 102.20769084619025, 46.52881069293825, 88.6099145834031, 43.918251983026444, 110.91295798461897, 38.486297432036345, 18.787483177191984, 155.73016147877198, 68.98019524809672, 63.590401717957974, 59.65579947612476, 57.736804890562475, 56.4472766029132, 49.37461753997109, 46.95754391104435, 46.3491890131531, 39.7221845240598, 38.19341311239947, 36.36545561656036, 36.00711382433293, 29.91275293909869, 27.831551808929593, 27.655465869714472, 26.347041202058946, 26.059410022503734, 25.78632903226185, 25.76358024099832, 25.662278594799417, 21.462120840821225, 21.186487104063026, 20.491709159193874, 19.712241053668105, 19.640074723320076, 18.790876576548524, 17.990939915926624, 16.85505273007965, 16.60569993795358, 60.57535653417805, 25.790859225668992, 37.543245344562195, 25.32484789584157, 20.496053020738515, 340.61144223783737, 126.61806510775976, 97.24544170640752, 85.28384794239591, 78.5864458705324, 67.37023126563808, 54.369056047192025, 31.086736877728974, 29.980080274811083, 29.389517209061466, 28.500095098216267, 27.084496593179125, 25.107416734638676, 17.117609522114755, 16.662164861477788, 13.707811886348077, 13.52450926421886, 13.385616733310577, 10.726481177933438, 8.850915900037608, 7.510188699147821, 7.215661564011325, 7.0797999809567544, 34.72572076010519, 29.00939264442751, 5.9470071294832865, 5.619000121915082, 5.336997485207414, 4.859088120814926, 16.25024766807897, 18.49949307327199, 18.365229509789152, 16.092571060907535, 61.83054976144947, 63.60274758946805, 30.979172155449337, 78.92494605910659, 43.113967519181855, 54.2129021266737, 55.18090341723558, 21.582699532919843, 24.768279689050278, 170.7865445916752, 147.65797496132288, 142.75867068406694, 81.32304943419041, 68.12819439521427, 65.85266472485948, 49.874598594311784, 43.435717879600105, 42.30012294444594, 36.634428357353165, 36.104387985560265, 39.62527319320616, 31.80160363499531, 27.521036220375493, 28.28260638656115, 26.05714105052456, 25.766602088512894, 24.203626652154217, 23.89763315677218, 22.20016151108214, 20.471807948507273, 50.63260168711952, 17.866816865002498, 17.218998329774163, 15.76860394789252, 14.662067712375563, 13.361487279690829, 59.469165539325594, 9.763112206345697, 8.80334078236068, 45.904270494849946, 18.86376375131986, 39.567264393355416, 53.32691579660415, 30.235006825546872, 19.19059789379689, 17.919172229470323, 137.30947914273148, 87.78991793418673, 84.13516153379814, 75.97388349480754, 63.9329277543048, 61.782534470488415, 39.5828081308074, 33.55330345853216, 30.805088831417187, 29.155863595965943, 28.66023325309503, 28.52483692387513, 28.24926861206987, 24.61113483006081, 23.868139705665943, 22.041774407677693, 21.53432180472457, 20.780030825333938, 84.65518134554786, 20.188342006273455, 20.044202031556335, 18.499628787821383, 18.265344317524857, 17.40024791800672, 16.664384917400394, 15.990206330876745, 15.84366997776971, 12.554339427480393, 11.395584054711476, 11.17120246132998, 15.70604830430351, 48.08935089512438, 25.62838130335514, 28.530076269807722, 20.85894597919565, 16.640951339346433, 134.51748020319977, 114.18522556969546, 38.82588985781616, 38.19207261627963, 36.64307524828161, 25.30981306823035, 22.76318821284428, 22.349275062657064, 16.504420501098156, 11.244186945357182, 8.675430723607837, 8.615711363860187, 8.464490574669723, 7.258653759327461, 7.149778269977983, 172.07095168428526, 4.620888429549418, 3.8547259596397176, 3.6361989951760276, 78.11659969501252, 7.52612550767657, 43.60295195666067, 1.8648535411742169, 146.19320896489768, 1.52585839177996, 10.05074967955173, 33.218834827031095, 119.71732452140816, 204.56583955965635, 0.8494567345799157, 29.455489527853388, 51.73724031006294, 28.85059555595892, 17.201417944817372, 48.002490221504786, 13.296676503163894, 8.564376435668066, 71.48525246209593, 64.47057213903427, 46.35152531148131, 40.86014585425807, 32.09604663314671, 29.57089470933072, 24.29680745933288, 24.244842691727133, 23.289037311549265, 20.652547088441132, 19.4614985959681, 17.58641676121462, 17.0610623344464, 16.232602100696, 15.387458395232064, 13.61101852215702, 12.93438184172899, 12.29432579642383, 12.27609323949712, 11.953838406850595, 11.890807801834226, 11.568021571348238, 39.24087211816676, 9.292731151324219, 8.99134551105425, 8.852670478600277, 8.285412086592736, 7.674959191735743, 7.500567733274632, 5.768590346506561, 13.677150598207445, 20.03094584098105, 12.006814260614659, 9.473850898578794, 13.51966430227695, 41.296828995171744, 28.44166717862093, 28.20230319151144, 26.294040247149855, 25.50301170286901, 19.678135690484183, 18.0365315400834, 14.64870615357369, 14.60509689727669, 14.43869663862615, 13.275216474923264, 12.190213122403302, 11.809125838705205, 11.573994005728649, 11.410776557722878, 11.409767932494566, 10.91267517478715, 9.145145564321624, 73.62514229916209, 7.995162655461204, 7.0561987350737185, 6.868225433377563, 6.473776198221284, 6.283120157009114, 6.132064320093096, 6.084180070660616, 5.442994298471157, 5.202659435353423, 5.118621188512952, 3.6564755578661896, 12.332223991702087, 8.593906003987426, 12.893933234599649, 11.917665450112013, 89.48081754665868, 19.930680191968488, 19.500181507440143, 17.44264227710882, 16.533729212948796, 16.04028343091675, 13.594896606661758, 13.499763857105915, 10.277453909660887, 10.26500166747252, 10.133190992308473, 8.418236511301256, 8.07732467331246, 7.984187938678925, 7.918681520657164, 7.107750327814642, 6.79894336598124, 6.659184059152855, 6.628093133069816, 6.548057682040633, 6.176050666605814, 6.066663096955198, 5.641722715251859, 5.518541244110289, 5.48889790712426, 4.406326939080272, 4.1941020873009665, 3.9368999408027063, 3.2544151778806696, 27.423297908253048, 43.73022824282351, 7.1199042660334335, 47.104076993539636, 24.696646266503308, 19.045555908752146, 14.049979067385566, 13.35769304747488, 12.979595043363718, 12.010280578621904, 9.649158215555357, 8.94413296180741, 6.3198135100487605, 6.1729011302946235, 6.155820271704513, 5.9022258547571, 4.054050364980102, 4.052743875298824, 3.12462056854651, 3.0693379078154837, 2.900659015228246, 2.6577237558063955, 43.860631654496885, 2.0050210014753165, 11.982230798901565, 1.3994469977683341, 1.3908281681273191, 0.7384807510285856, 0.7202901574715779, 0.5105598431757963, 6.605514239302238, 16.105126262082532, 8.026288598315462, 33.18555094352258, 31.803440767975637, 25.06090938444821, 24.12202834629745, 22.04251453535645, 21.680136446689378, 18.40842686365434, 14.669217460841095, 11.52911511826598, 9.88293900464367, 7.922290170157668, 7.198322629054773, 6.008274709553673, 5.83423894368894, 5.4392604295890745, 4.376013690249905, 4.286202352911864, 3.529685525723624, 2.8647499245582972, 2.0844895136936072, 1.6558774035880819, 1.2606605853209072, 0.8487339203765357, 0.7792411932863716, 0.7089850102527923, 2.24118631507126, 0.03950157247893896, 0.039503572137750284, 0.039488816034797716, 0.0394825493555614, 0.8123677439600093, 0.03948972460188441, 0.04002586436119545], \"Total\": [625.0, 487.0, 469.0, 341.0, 457.0, 460.0, 334.0, 380.0, 395.0, 288.0, 322.0, 251.0, 535.0, 261.0, 367.0, 312.0, 300.0, 308.0, 261.0, 211.0, 420.0, 265.0, 382.0, 333.0, 218.0, 208.0, 219.0, 259.0, 240.0, 394.0, 262.786062759906, 208.42604269135586, 367.4789826026533, 105.56706846172801, 94.8772065730239, 78.57493638875982, 78.35566097431263, 74.05908102046371, 69.32055211180668, 68.4545861323004, 55.54861955073792, 55.5297724805216, 52.60814985555068, 174.8434584416705, 49.24817060771122, 47.63214525303777, 128.78997984094823, 45.432515755513606, 44.00408649027751, 39.879857659646845, 39.31314743536361, 38.96915402657808, 44.866137885098134, 36.83800887484174, 36.38377726591233, 35.984321798169525, 32.21597956894565, 31.787816982117683, 67.47881818436588, 29.398692149286237, 382.9146516420783, 248.5168539528431, 109.66717463903925, 347.5398280391792, 162.21702148926047, 134.69995360643057, 134.67338766323087, 289.32650890832076, 318.23158010440824, 167.00938073224583, 108.3331376387385, 171.90213008276774, 369.25183456906166, 92.19389737651049, 246.88834128458294, 394.98906142074594, 154.75882909247304, 169.2216332449998, 138.49302495655445, 238.1397379768188, 120.58388070626597, 205.81403207631018, 124.3274304122154, 251.95612342108464, 220.19155153577415, 186.17657389557638, 128.68735892258343, 95.97976395263471, 90.79935309260028, 169.6771919789805, 73.36330257435439, 62.535932712964474, 58.39917811487346, 57.85996751662975, 55.46154505673347, 54.7856581598508, 54.52332085799983, 53.2693987951059, 52.747544063888704, 51.03859736440029, 50.95338046264521, 308.8422338759066, 47.154354950816575, 45.162772659388814, 44.72021656391719, 178.57270089202333, 41.61878947010762, 40.233716948402524, 37.38755919376728, 35.54128032020882, 34.76562013009091, 32.61226534458239, 32.51187695891461, 51.23718852506098, 333.8634729320299, 74.61357611625205, 75.49872221018813, 82.69466058922603, 133.19408197998297, 249.6508036249261, 75.08610946769423, 88.57161354345116, 81.34543531508295, 420.59746637098715, 163.85376377821854, 87.92847114465059, 101.54562186849925, 238.1397379768188, 138.1226620710171, 113.1405688390565, 81.77923804288581, 216.84615614089222, 144.2802191966095, 312.69639823590893, 308.31098240242915, 192.61321565869687, 175.67255419045748, 174.1682554023926, 126.66018681567898, 121.25084002695311, 118.72891823081672, 86.82978693176686, 82.19619030653439, 79.28883761876986, 76.9658548087585, 75.86867552085249, 74.06231737096458, 67.38093772803913, 63.45985829370366, 57.43058761817844, 56.16359868309171, 53.86662587151624, 53.683842146060435, 49.07733140375356, 47.62507968885385, 44.056776849364645, 43.785077552061786, 41.24217031491093, 39.630949613152595, 39.289505692108804, 37.420250862766736, 33.05651323713765, 32.175235206063306, 91.24771332170236, 55.51040885080092, 71.60088406617726, 535.5577089940163, 94.0413282130196, 231.78581068736932, 246.88834128458294, 120.10680384929378, 216.84615614089222, 380.13548935667325, 322.6396968350942, 300.993703600846, 137.1703881634771, 129.37435707237907, 625.7616088182401, 98.1967936018608, 89.73363348381773, 79.81726585298732, 79.5228385422785, 78.67488148853273, 74.48959563814144, 69.76856032763062, 64.49126844894292, 62.52047861522321, 60.20138566102296, 54.072601070348114, 47.37398993239754, 395.59248777802753, 42.153409613864774, 38.617526286316725, 38.12056530979092, 35.095108335639004, 34.67404168208392, 31.068256609573908, 29.900926150523578, 28.299185370277083, 27.61440242631765, 25.988858905157876, 25.34732108479413, 42.11209239942953, 95.89117191843238, 108.71206301444988, 128.6293777205434, 83.42793719015212, 59.815383478377285, 426.5343194458971, 255.93623905734555, 211.46542906948793, 60.49896649304201, 56.14622142446701, 69.18370924969733, 197.05246420849184, 184.10114048502334, 146.66613180739625, 196.52436170650577, 135.1358944940174, 116.04402963864209, 115.21860023892484, 114.54497730690905, 98.91265067533774, 81.34335873041137, 72.60948244124425, 72.4473864077431, 64.23785545673249, 57.90678477931735, 53.54271668244421, 44.251029115096664, 43.86006925797648, 34.89647027496261, 33.199963309408126, 32.62142317599936, 32.5704577210815, 32.400967882534744, 31.868878312389146, 30.043152401534627, 29.905122806493683, 29.004715565162485, 26.28377630532446, 24.87587520551121, 24.531505155217058, 23.941738839719157, 23.83693800969202, 103.95967131007036, 215.29174262311014, 64.81413334609829, 197.05246420849184, 59.19013812753009, 94.2688125898538, 93.37000547777436, 71.57788300381802, 101.78490393443998, 105.12876040460517, 460.6327847455948, 150.8365657855549, 240.9369380030329, 123.46632935192757, 93.46421717579229, 178.4320879855675, 199.46583830626767, 186.2631391959222, 469.29382639508424, 265.91728686786536, 259.15405690894147, 175.9255856446893, 146.85604138833335, 95.145450378792, 89.50273927109795, 88.17680634783429, 457.7868385920884, 86.8196108058385, 77.81344930556868, 66.00309455461571, 64.4245471019697, 58.76447861862023, 54.188306125448506, 45.783627571290566, 44.13675530393638, 43.764279235331216, 38.18986674515971, 33.48196592585648, 28.718896178416646, 28.291541571386738, 24.633260309758928, 23.969834803490507, 22.424104806659887, 51.354794699524255, 19.762681561701275, 18.176312616064045, 49.56827420326668, 27.176677753042778, 46.7754302728203, 111.51023003036015, 119.42375313618828, 89.37468807066952, 146.85651031227803, 105.57227366373476, 394.98906142074594, 426.5343194458971, 535.5577089940163, 199.46583830626767, 124.30654428791617, 216.84615614089222, 487.87105147835007, 288.50657417201353, 261.3269938858498, 194.1619414204538, 190.66522119605818, 104.20917362721933, 88.33914949385456, 119.49071493854666, 80.50177838359886, 52.53642844975059, 97.29513378832343, 31.851788438697703, 26.54733909334699, 118.24423130612237, 25.707798202239683, 24.824473422924346, 24.56217247308331, 23.286752080982794, 20.951329240510436, 20.851626170753622, 17.14061008180265, 95.90952159396467, 16.22681883823523, 15.455352466986984, 102.03928099602221, 12.319998296488826, 11.864061610442652, 57.18623958081029, 10.677293012572001, 10.503336526360084, 61.36445556810624, 186.2631391959222, 36.361857942178894, 103.92361865406939, 420.59746637098715, 353.52938601378145, 103.40572217623374, 535.5577089940163, 38.44266180799874, 121.9040104230776, 369.25183456906166, 123.46632935192757, 71.6228559708134, 334.89933984864115, 251.48813103369505, 219.65109024107517, 120.82224359083463, 106.36612277828539, 106.25925835954031, 90.25330987804678, 85.25896949079858, 65.84036893476815, 63.98303586740755, 61.932152353577806, 61.88890305845485, 50.3915547428789, 49.53128861039646, 40.17450734011193, 38.45287257616533, 38.12186840141206, 35.2765954288684, 34.67891121066772, 34.22953215811463, 26.404457553512042, 25.92051444952411, 25.65005796389888, 22.29316983546162, 22.185452719852428, 19.824625822753024, 19.158097587129873, 19.074847535094897, 18.738395833024377, 17.171422688529788, 81.13975923840965, 114.40788008346006, 83.48723033889779, 34.28301855372813, 261.1835862390415, 353.52938601378145, 420.59746637098715, 120.10680384929378, 369.25183456906166, 182.22917296152286, 165.52874063818473, 154.83005148797793, 76.15335727336887, 50.7038093133956, 49.50198208285583, 44.91832891895487, 44.881021682627576, 40.55980258632872, 40.31600677590293, 39.596520174825976, 37.61489649658995, 218.4421931428742, 31.745530897384704, 30.94351789308988, 28.91192254720815, 27.854569734569385, 25.082510362898887, 24.075102120627328, 20.520386338714665, 20.457651451675872, 19.814458077100863, 19.753204094524943, 18.71068317347671, 17.802450088960665, 17.42876092460408, 14.677069263842395, 14.654356469596937, 112.3314727882339, 72.90635712452818, 42.055746303172626, 35.73832527692895, 72.90839689317541, 82.24353580983194, 144.2802191966095, 140.73632665141423, 65.86430963717011, 460.6327847455948, 208.46764238100513, 150.8365657855549, 211.25867720692807, 125.40270014377919, 117.43505132682449, 105.43088637554578, 78.83107666063047, 62.738208957951585, 54.75317965736302, 48.13901935707146, 35.10500066855431, 34.08242697636614, 32.33882199270854, 30.394527365008305, 30.121497105781366, 29.874783822149034, 26.00636121949374, 24.315446972089223, 23.83839735880442, 22.10774808063721, 21.90079200757923, 21.17944423231279, 19.284892529468063, 70.12955885392036, 17.28639186155417, 16.585829916983645, 15.074055508207913, 14.964291167323474, 14.935214232888876, 14.387446822374388, 12.855540412280682, 14.427754292474278, 56.41294626759678, 36.58297926978397, 120.16144392365709, 56.23750692341945, 140.73632665141423, 64.21693523502654, 211.46542906948793, 255.93623905734555, 105.12876040460517, 156.93368992918388, 70.18372371666692, 64.7939301683699, 60.859327950230586, 58.94033335717799, 57.65080506436389, 50.57814603965219, 48.16107236145628, 47.55271747140747, 40.92571297447173, 39.39694160265746, 37.5689841205189, 37.21070715524878, 31.116281413369475, 29.035080304824067, 28.858994320126396, 27.55056965247087, 27.26293853602879, 26.989857523550924, 26.967108691410242, 26.86580704521134, 22.665649291233148, 22.39001556687702, 21.695237609605798, 20.915769525053058, 20.84360318631806, 19.994405034151526, 19.194468366338548, 18.058581206936655, 17.809228392648738, 79.63221515866375, 32.82166486116264, 58.22066656038434, 51.43879231813361, 70.04821589928903, 341.82684008967504, 127.83345951876277, 98.46083611741052, 86.49924235706943, 79.80300267142212, 68.58562567664109, 55.59742319423986, 32.302131288731985, 31.195474685814087, 30.60491162006447, 29.71548951452973, 28.29989100418213, 26.32281114564168, 18.33300393311776, 17.87755932788438, 14.92320630266154, 14.739903758289383, 14.601011154567706, 11.94187558893644, 10.06631033402038, 8.725583113464484, 8.4310559811151, 8.295194401048166, 40.76336645950667, 34.07496442739626, 7.162401554091027, 6.834394532918086, 6.552391899524078, 6.074482566357208, 21.822229739347595, 26.03692634577386, 25.901468176892326, 23.06844559888328, 124.30654428791617, 145.0108907715299, 75.88073881428231, 353.52938601378145, 163.85376377821854, 289.32650890832076, 333.8634729320299, 60.35072529557197, 150.8365657855549, 172.00287333629595, 148.87430369609004, 143.9752797992007, 82.53937817178097, 69.3445231263109, 67.07024927504455, 51.09092732540843, 44.65204666744267, 43.51645167921311, 37.85075712599538, 37.32071674025455, 41.00368481831184, 33.017932366091955, 28.73736499794803, 29.552911609651435, 27.273469798329106, 26.982930864408143, 25.41995539428961, 25.113961931666683, 23.416490250858708, 21.68813667960391, 53.99536905780208, 19.083145655569307, 18.435327101092067, 16.984932695998804, 15.878396459457564, 14.577816103594722, 66.56077534735257, 10.979440949919807, 10.019669517631346, 74.63303062784202, 29.078312378951466, 89.67185422057865, 249.6508036249261, 83.48723033889779, 231.78581068736932, 79.63221515866375, 138.5280912140214, 89.00853006006608, 85.35467882065821, 77.19249557878213, 65.15153982559474, 63.00114654177837, 40.801420206925485, 34.77191554121928, 32.0237009408138, 30.374475688065942, 29.878845324384972, 29.7434490246192, 29.467880683359812, 25.829746901350752, 25.086753052348566, 23.260386478967636, 22.752933894185887, 21.99864289662388, 89.72133007131069, 21.40695408687047, 21.262814106516807, 19.718240859111326, 19.4839564240597, 18.61886005540223, 17.882996988690337, 17.208818406222793, 17.062285481659444, 13.772951561772242, 12.614196126001419, 12.389814532619923, 18.742506061599517, 80.15996787251983, 37.42909828926465, 120.10680384929378, 231.78581068736932, 318.23158010440824, 135.74755791788846, 115.41530328438415, 40.055967572504855, 39.42215033096833, 37.87315296297031, 26.539890782919045, 23.99326595058503, 23.579352777345758, 17.734499537223854, 12.474264671026168, 9.905508472371512, 9.845789089216836, 9.69456839166619, 8.488731474016154, 8.379856052739859, 208.46764238100513, 5.850966173466814, 5.084803674328409, 4.866276731365682, 106.23058391059534, 10.43626967477011, 71.6228559708134, 3.094931300919461, 261.1835862390415, 2.7559361469330304, 19.070839774190382, 65.90060341427841, 240.9369380030329, 460.6327847455948, 2.0795344492686088, 80.88786471560164, 169.2216332449998, 103.92361865406939, 75.98517795630241, 535.5577089940163, 52.3211280177479, 24.35616250392306, 72.70879641016678, 65.69411605748653, 47.575069229933575, 42.083689772710336, 33.31959057884801, 30.79443952938727, 25.520351386465073, 25.468386627617225, 24.512679433218324, 21.876091015783867, 20.68504251442036, 18.809960689921002, 18.284609555421323, 17.456146029402383, 16.611002371888407, 14.83456244060928, 14.157925760181252, 13.51786971854662, 13.49963716633635, 13.177382331273746, 13.114351732809201, 12.791565501313467, 44.232909683971464, 10.516275097289038, 10.214889429506512, 10.076214402061941, 9.508956042436589, 8.898503124105286, 8.724111651726894, 6.992134268479371, 22.058557097613146, 50.35972510483386, 32.66048168402128, 20.172410980067962, 171.90213008276774, 42.526126825031994, 29.670965001454643, 29.431601012079053, 27.523338067717468, 26.732309553421157, 20.907433505080906, 19.265829354680122, 15.878003968170411, 15.834394711873411, 15.667994453222871, 14.504514293040536, 13.419510937000023, 13.038423653301926, 12.80329182032537, 12.6400743723196, 12.639065805577347, 12.141973029362342, 10.374443378918345, 83.8122345424737, 9.224460481096681, 8.28549660714384, 8.097523247974284, 7.703074012818005, 7.5124179775767255, 7.361362134689817, 7.313477889431368, 6.672292113067878, 6.431957249950145, 6.3479191363457845, 4.885773372462911, 18.099172808912684, 21.10107100399181, 134.69995360643057, 124.30654428791617, 90.71893233314893, 21.168794999306208, 20.738296314823174, 18.680757063599057, 17.771844050936206, 17.278398342900026, 14.833011393151994, 14.737878656842547, 11.515568696151123, 11.503116468522062, 11.371305811503797, 9.656351316640949, 9.315439505136256, 9.222302725169161, 9.156796321589987, 8.345865123393287, 8.037058152471477, 7.897299200805582, 7.866207943751708, 7.786172468530869, 7.414165457393716, 7.304777883445434, 6.879837501742095, 6.756656121085083, 6.727012693614496, 5.644441730579913, 5.4322169904932025, 5.175014727292942, 4.492529998185172, 38.179898956002674, 75.88073881428231, 61.36445556810624, 48.346505699941694, 25.939074132886685, 20.287983769567525, 15.292406900502138, 14.600120925571158, 14.222022872401284, 13.252708489601924, 10.891586044592923, 10.18656079908197, 7.5622413433839935, 7.41532895933219, 7.39824810074208, 7.144653771943372, 5.296478378960191, 5.295171762893935, 4.3670484018817435, 4.311765736853051, 4.143086844265813, 3.9001516166872365, 70.04821589928903, 3.2474488448604446, 22.590820366866822, 2.641874831103568, 2.6332559971648863, 1.980908580066153, 1.9627179865091453, 1.7529876722133637, 22.68380428826751, 67.78200662314813, 64.81413334609829, 34.428250875196014, 33.04614068669009, 26.30360930316266, 25.364728314245884, 23.2852144540709, 22.922836369074357, 19.651126804705125, 15.911917379555545, 12.771815058069079, 11.125638938621648, 9.164990100153467, 8.441022551439751, 7.250974628268122, 7.076938866686626, 6.681960367152982, 5.618713652321297, 5.528902306015542, 4.772385444438074, 4.107449843272747, 3.3271895264552027, 2.8985773223025317, 2.503360504035357, 2.0914338683741662, 2.021941112000821, 1.951684928967242, 36.629344541832616, 1.2823885228966259, 1.2824683015967808, 1.2823357745068826, 1.2822474538964008, 27.176677753042778, 1.2828265285441998, 51.23718852506098], \"Category\": [\"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic11\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic12\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic13\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic14\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic15\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic16\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic17\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic18\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic19\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\", \"Topic20\"], \"logprob\": [30.0, 29.0, 28.0, 27.0, 26.0, 25.0, 24.0, 23.0, 22.0, 21.0, 20.0, 19.0, 18.0, 17.0, 16.0, 15.0, 14.0, 13.0, 12.0, 11.0, 10.0, 9.0, 8.0, 7.0, 6.0, 5.0, 4.0, 3.0, 2.0, 1.0, -3.4954, -3.7284, -3.1634, -4.4143, -4.5224, -4.7136, -4.7164, -4.7737, -4.8409, -4.8537, -5.0668, -5.0674, -5.1224, -3.9221, -5.19, -5.2243, -4.2296, -5.2728, -5.3056, -5.4069, -5.4217, -5.4308, -5.2909, -5.4889, -5.5017, -5.5131, -5.6278, -5.6417, -4.8906, -5.723, -3.1608, -3.6007, -4.4251, -3.3325, -4.0859, -4.2692, -4.275, -3.622, -3.5463, -4.126, -4.5331, -4.143, -3.5674, -4.6991, -4.0459, -3.7312, -4.3877, -4.3625, -4.5508, -4.4396, -4.6761, -4.6093, -4.7127, -3.349, -3.4845, -3.6533, -4.0255, -4.322, -4.3782, -3.7548, -4.5946, -4.7572, -4.8271, -4.8365, -4.8798, -4.8923, -4.8972, -4.921, -4.9311, -4.9648, -4.9665, -3.1653, -5.046, -5.0903, -5.1004, -3.7167, -5.1744, -5.2092, -5.285, -5.3373, -5.3602, -5.4265, -5.4297, -4.9986, -3.2496, -4.6487, -4.645, -4.5679, -4.1499, -3.6333, -4.6709, -4.57, -4.6568, -3.5395, -4.2602, -4.6468, -4.6088, -4.2371, -4.5331, -4.6258, -4.7788, -4.728, -4.804, -3.0482, -3.0624, -3.5352, -3.6278, -3.6365, -3.9576, -4.0017, -4.0236, -4.3397, -4.3953, -4.4319, -4.4621, -4.4767, -4.5011, -4.5973, -4.6584, -4.7603, -4.7831, -4.8258, -4.8293, -4.9212, -4.952, -5.032, -5.0384, -5.0999, -5.141, -5.15, -5.2003, -5.3287, -5.3568, -4.3164, -4.8775, -4.6604, -3.3674, -4.5759, -4.1366, -4.2366, -4.6865, -4.8336, -2.7735, -2.9381, -3.0078, -3.7984, -3.8587, -2.2836, -4.1362, -4.2275, -4.3463, -4.35, -4.3609, -4.4164, -4.483, -4.5631, -4.5947, -4.6333, -4.743, -4.8785, -2.7587, -4.9984, -5.0887, -5.1021, -5.1876, -5.2001, -5.315, -5.3539, -5.4113, -5.4369, -5.5004, -5.5266, -5.037, -4.2743, -4.1582, -4.0337, -4.4348, -4.7751, -3.4039, -3.8278, -4.1115, -4.8436, -4.9766, -4.9333, -4.7478, -3.4058, -3.6341, -3.3419, -3.7166, -3.8704, -3.8777, -3.8836, -4.032, -4.2302, -4.3456, -4.3479, -4.4703, -4.5762, -4.6563, -4.8518, -4.8609, -5.0968, -5.1485, -5.1667, -5.1683, -5.174, -5.191, -5.2524, -5.2572, -5.2892, -5.3921, -5.4498, -5.4645, -5.4901, -5.4947, -4.0282, -3.3196, -4.5954, -3.649, -4.6982, -4.305, -4.3158, -4.5545, -4.3011, -4.3038, -3.2614, -4.0875, -3.8259, -4.3017, -4.4793, -4.2805, -4.2872, -4.5177, -2.405, -2.9751, -3.0009, -3.3905, -3.5725, -4.0111, -4.0731, -4.0882, -2.4411, -4.1039, -4.215, -4.3825, -4.4072, -4.501, -4.5838, -4.7566, -4.7942, -4.8029, -4.9433, -5.0795, -5.2392, -5.2551, -5.4, -5.4287, -5.4991, -4.6716, -5.6331, -5.7225, -4.726, -5.3271, -4.7946, -4.0004, -3.9465, -4.2213, -3.829, -4.172, -3.335, -3.4793, -3.7151, -4.2046, -4.6526, -4.6322, -2.1154, -2.6425, -2.7419, -3.0406, -3.0589, -3.6684, -3.8357, -3.5345, -3.93, -4.3651, -3.7543, -4.8811, -5.0713, -3.578, -5.105, -5.1418, -5.1529, -5.2092, -5.321, -5.3261, -5.5357, -3.8171, -5.5948, -5.6476, -3.7783, -5.8965, -5.9385, -4.3755, -6.0568, -6.0754, -4.3315, -3.4747, -4.9711, -4.1143, -3.1986, -3.3695, -4.3731, -3.4304, -5.0294, -4.4803, -4.3757, -4.8131, -5.0128, -2.3085, -2.5953, -2.7314, -3.3336, -3.4624, -3.4635, -3.6288, -3.6865, -3.9492, -3.9783, -4.0116, -4.0123, -4.2224, -4.2404, -4.4552, -4.5004, -4.5093, -4.5895, -4.6072, -4.6208, -4.8912, -4.9106, -4.9216, -5.0694, -5.0745, -5.1939, -5.2304, -5.235, -5.254, -5.3477, -3.9463, -3.6381, -4.1646, -4.891, -3.7412, -4.0282, -4.2569, -4.7175, -4.8815, -2.7806, -2.8773, -2.9449, -3.6624, -4.0773, -4.1019, -4.2016, -4.2025, -4.3067, -4.3129, -4.3315, -4.3845, -2.6282, -4.5603, -4.5869, -4.6577, -4.6966, -4.8065, -4.8496, -5.0185, -5.0218, -5.0558, -5.0591, -5.117, -5.1702, -5.193, -5.379, -5.3806, -3.353, -3.795, -4.3818, -4.6094, -4.121, -4.1142, -3.9566, -4.0476, -4.5205, -4.2006, -4.417, -4.8351, -2.5366, -3.0622, -3.1284, -3.2375, -3.5321, -3.7645, -3.9035, -4.0354, -4.3608, -4.3914, -4.4459, -4.5104, -4.5198, -4.5284, -4.6734, -4.744, -4.7649, -4.8444, -4.8544, -4.8899, -4.9896, -3.7034, -5.1068, -5.1513, -5.2549, -5.2628, -5.2649, -5.3057, -5.4293, -5.3161, -3.9534, -4.4096, -3.2569, -4.0439, -3.3997, -4.1016, -3.1752, -4.2336, -4.9507, -2.7763, -3.5906, -3.6719, -3.7358, -3.7685, -3.7911, -3.925, -3.9752, -3.9882, -4.1425, -4.1817, -4.2308, -4.2407, -4.4261, -4.4982, -4.5046, -4.5531, -4.564, -4.5746, -4.5754, -4.5794, -4.7581, -4.771, -4.8044, -4.8432, -4.8468, -4.891, -4.9345, -4.9998, -5.0147, -3.7205, -4.5744, -4.1989, -4.5926, -4.8042, -1.9076, -2.8972, -3.1611, -3.2924, -3.3742, -3.5282, -3.7426, -4.3016, -4.3378, -4.3577, -4.3884, -4.4394, -4.5152, -4.8982, -4.9252, -5.1204, -5.1339, -5.1442, -5.3656, -5.5578, -5.7221, -5.7621, -5.7811, -4.1909, -4.3707, -5.9555, -6.0122, -6.0637, -6.1575, -4.9502, -4.8206, -4.8279, -4.96, -3.614, -3.5857, -4.305, -3.3699, -3.9745, -3.7454, -3.7277, -4.6665, -4.5288, -2.5749, -2.7204, -2.7542, -3.3169, -3.4939, -3.5279, -3.8058, -3.944, -3.9705, -4.1143, -4.1289, -4.0359, -4.2558, -4.4004, -4.3731, -4.455, -4.4662, -4.5288, -4.5415, -4.6152, -4.6963, -3.7907, -4.8324, -4.8693, -4.9573, -5.0301, -5.123, -3.6299, -5.4367, -5.5402, -3.8888, -4.7781, -4.0373, -3.7389, -4.3063, -4.7609, -4.8295, -2.6987, -3.146, -3.1885, -3.2906, -3.4631, -3.4973, -3.9426, -4.1078, -4.1933, -4.2483, -4.2654, -4.2702, -4.2799, -4.4178, -4.4484, -4.528, -4.5513, -4.587, -3.1824, -4.6159, -4.623, -4.7032, -4.716, -4.7645, -4.8077, -4.849, -4.8582, -5.0909, -5.1877, -5.2076, -4.8669, -3.7479, -4.3773, -4.27, -4.5832, -4.8091, -2.6664, -2.8303, -3.909, -3.9255, -3.9669, -4.3369, -4.4429, -4.4613, -4.7645, -5.1482, -5.4076, -5.4145, -5.4322, -5.5859, -5.601, -2.4202, -6.0375, -6.2188, -6.2772, -3.2099, -5.5497, -3.793, -6.9449, -2.5832, -7.1455, -5.2604, -4.065, -2.783, -2.2472, -7.7312, -4.1852, -3.6219, -4.206, -4.7231, -3.6968, -4.9806, -5.4205, -2.8049, -2.9082, -3.2381, -3.3642, -3.6057, -3.6876, -3.8841, -3.8862, -3.9264, -4.0466, -4.106, -4.2073, -4.2376, -4.2874, -4.3408, -4.4635, -4.5145, -4.5653, -4.5667, -4.5933, -4.5986, -4.6262, -3.4047, -4.8452, -4.8781, -4.8937, -4.9599, -5.0364, -5.0594, -5.322, -4.4587, -4.0771, -4.5889, -4.8259, -4.4703, -3.0936, -3.4665, -3.475, -3.545, -3.5756, -3.8349, -3.922, -4.13, -4.133, -4.1445, -4.2285, -4.3137, -4.3455, -4.3656, -4.3798, -4.3799, -4.4245, -4.6012, -2.5154, -4.7355, -4.8605, -4.8875, -4.9466, -4.9765, -5.0008, -5.0087, -5.1201, -5.1652, -5.1815, -5.5179, -4.3022, -4.6633, -4.2576, -4.3364, -2.1404, -3.6421, -3.664, -3.7755, -3.829, -3.8593, -4.0247, -4.0317, -4.3044, -4.3056, -4.3186, -4.504, -4.5453, -4.5569, -4.5652, -4.6732, -4.7176, -4.7384, -4.7431, -4.7552, -4.8137, -4.8316, -4.9042, -4.9263, -4.9317, -5.1513, -5.2007, -5.264, -5.4544, -3.323, -2.8563, -4.6715, -2.5255, -3.1712, -3.4311, -3.7353, -3.7858, -3.8145, -3.8921, -4.111, -4.1869, -4.5342, -4.5577, -4.5605, -4.6026, -4.9782, -4.9785, -5.2386, -5.2564, -5.313, -5.4004, -2.5969, -5.6822, -3.8945, -6.0418, -6.048, -6.681, -6.706, -7.0501, -4.49, -3.5988, -4.2952, -2.8213, -2.8638, -3.1021, -3.1403, -3.2304, -3.247, -3.4106, -3.6377, -3.8785, -4.0326, -4.2537, -4.3496, -4.5303, -4.5597, -4.6298, -4.8473, -4.868, -5.0622, -5.2709, -5.5889, -5.8191, -6.0918, -6.4874, -6.5728, -6.6673, -5.5164, -9.5548, -9.5548, -9.5551, -9.5553, -6.5312, -9.5551, -9.5416], \"loglift\": [30.0, 29.0, 28.0, 27.0, 26.0, 25.0, 24.0, 23.0, 22.0, 21.0, 20.0, 19.0, 18.0, 17.0, 16.0, 15.0, 14.0, 13.0, 12.0, 11.0, 10.0, 9.0, 8.0, 7.0, 6.0, 5.0, 4.0, 3.0, 2.0, 1.0, 2.0467, 2.0455, 2.0434, 2.0398, 2.0385, 2.0358, 2.0358, 2.0349, 2.0338, 2.0335, 2.0294, 2.0292, 2.0281, 2.0275, 2.0265, 2.0257, 2.0256, 2.0244, 2.0235, 2.0206, 2.0202, 2.0199, 2.0189, 2.0181, 2.0176, 2.0173, 2.0132, 2.0127, 2.011, 2.0095, 2.0048, 1.9972, 1.9909, 1.93, 1.9386, 1.9412, 1.9356, 1.8239, 1.8044, 1.8694, 1.8951, 1.8235, 1.6346, 1.8905, 1.5586, 1.4033, 1.6838, 1.6198, 1.6318, 1.201, 1.645, 1.1772, 1.5778, 2.2352, 2.2345, 2.2335, 2.2305, 2.2273, 2.2265, 2.2247, 2.2234, 2.2205, 2.2191, 2.2189, 2.218, 2.2177, 2.2176, 2.217, 2.2168, 2.216, 2.216, 2.2153, 2.214, 2.2129, 2.2126, 2.2118, 2.2105, 2.2095, 2.2072, 2.2054, 2.2046, 2.2023, 2.2021, 2.1784, 2.0531, 2.1525, 2.1444, 2.1304, 2.0718, 1.9601, 2.124, 2.0597, 2.058, 1.5323, 1.7543, 1.9901, 1.8842, 1.4035, 1.6522, 1.759, 1.9306, 1.0063, 1.3377, 2.32, 2.32, 2.3176, 2.317, 2.3169, 2.3143, 2.3139, 2.313, 2.3098, 2.309, 2.3085, 2.308, 2.3078, 2.3074, 2.3057, 2.3046, 2.3026, 2.3021, 2.3011, 2.301, 2.2989, 2.2981, 2.296, 2.2958, 2.294, 2.2928, 2.2925, 2.2909, 2.2865, 2.2855, 2.2834, 2.2194, 2.182, 1.4627, 1.9938, 1.5311, 1.3679, 1.6386, 0.9006, 2.3994, 2.3988, 2.3986, 2.3938, 2.392, 2.3909, 2.3903, 2.3891, 2.3874, 2.3874, 2.3872, 2.3863, 2.3852, 2.3838, 2.3832, 2.3825, 2.3802, 2.3769, 2.3743, 2.3737, 2.371, 2.3706, 2.3678, 2.3674, 2.3623, 2.3617, 2.3593, 2.3582, 2.3554, 2.3541, 2.3361, 2.276, 2.2665, 2.2228, 2.2546, 2.2471, 1.6539, 1.7407, 1.6479, 2.1673, 2.1089, 1.9434, 1.0822, 2.4922, 2.4912, 2.4907, 2.4905, 2.489, 2.489, 2.4889, 2.4872, 2.4845, 2.4827, 2.4827, 2.4805, 2.4784, 2.4767, 2.4718, 2.4716, 2.4643, 2.4624, 2.4618, 2.4617, 2.4613, 2.4608, 2.4584, 2.4583, 2.4568, 2.4524, 2.4497, 2.449, 2.4477, 2.4475, 2.4413, 2.4219, 2.3465, 2.181, 2.3345, 2.2623, 2.2611, 2.2881, 2.1895, 2.1545, 1.7195, 2.0098, 1.803, 1.9958, 2.0966, 1.6487, 1.5306, 1.3686, 2.5572, 2.5552, 2.5551, 2.5529, 2.5515, 2.547, 2.5461, 2.5459, 2.5459, 2.5457, 2.5441, 2.5413, 2.5408, 2.5389, 2.5372, 2.533, 2.532, 2.5317, 2.5275, 2.5229, 2.5167, 2.5158, 2.5094, 2.5079, 2.5043, 2.5031, 2.4965, 2.4908, 2.4841, 2.484, 2.4735, 2.3989, 2.3843, 2.3993, 2.295, 2.2821, 1.7996, 1.5784, 1.115, 1.6132, 1.6381, 1.1021, 2.808, 2.8062, 2.8058, 2.8042, 2.804, 2.7987, 2.7965, 2.7957, 2.7951, 2.7869, 2.7814, 2.7712, 2.7632, 2.7627, 2.7616, 2.7598, 2.7593, 2.7564, 2.7502, 2.7499, 2.7363, 2.733, 2.7319, 2.7278, 2.7098, 2.7057, 2.7014, 2.6916, 2.6885, 2.6864, 2.6651, 2.4116, 2.5488, 2.3555, 1.8731, 1.876, 2.1017, 1.3997, 2.4349, 1.8299, 0.8263, 1.4844, 1.8292, 2.9912, 2.9907, 2.99, 2.9855, 2.9841, 2.9841, 2.9821, 2.9813, 2.977, 2.9765, 2.9758, 2.9758, 2.9713, 2.9705, 2.965, 2.9636, 2.9633, 2.9607, 2.9601, 2.9596, 2.9487, 2.9478, 2.9473, 2.9398, 2.9395, 2.9326, 2.9304, 2.9301, 2.9289, 2.9225, 2.771, 2.7356, 2.5242, 2.6878, 1.807, 1.2173, 0.8149, 1.6076, 0.3205, 3.1276, 3.127, 3.1262, 3.1183, 3.1101, 3.1095, 3.107, 3.1069, 3.104, 3.1038, 3.1032, 3.1016, 3.0987, 3.0954, 3.0944, 3.0915, 3.0899, 3.0848, 3.0827, 3.0735, 3.0733, 3.0712, 3.071, 3.0674, 3.0639, 3.0623, 3.0482, 3.0481, 3.039, 3.0293, 2.9927, 2.9278, 2.7033, 2.5896, 2.1851, 2.1189, 2.4053, 0.7802, 1.3566, 1.2622, 3.2238, 3.2198, 3.2191, 3.2179, 3.214, 3.21, 3.2071, 3.204, 3.1943, 3.1933, 3.1913, 3.1888, 3.1884, 3.1881, 3.1817, 3.1783, 3.1773, 3.1731, 3.1725, 3.1705, 3.1645, 3.1597, 3.1567, 3.1535, 3.1456, 3.145, 3.1448, 3.1414, 3.1304, 3.1282, 3.1274, 3.1043, 3.0677, 3.04, 2.7669, 2.8496, 2.5842, 1.3349, 1.5075, 3.2813, 3.2717, 3.2703, 3.2691, 3.2684, 3.2679, 3.265, 3.2637, 3.2634, 3.2592, 3.258, 3.2565, 3.2562, 3.2496, 3.2467, 3.2464, 3.2444, 3.2439, 3.2434, 3.2434, 3.2432, 3.2345, 3.2338, 3.232, 3.2298, 3.2296, 3.227, 3.2243, 3.2201, 3.2191, 3.0155, 3.048, 2.8503, 2.5804, 2.0601, 3.3715, 3.3655, 3.3627, 3.3609, 3.3597, 3.3572, 3.3527, 3.3367, 3.3354, 3.3346, 3.3333, 3.3312, 3.3278, 3.3065, 3.3047, 3.2901, 3.289, 3.2882, 3.2678, 3.2464, 3.2251, 3.2194, 3.2167, 3.2148, 3.2141, 3.1891, 3.1793, 3.1699, 3.1518, 3.0803, 3.0333, 3.0313, 3.015, 2.6767, 2.5509, 2.4792, 1.8756, 2.04, 1.7005, 1.575, 2.3468, 1.5685, 3.391, 3.3899, 3.3896, 3.3833, 3.3804, 3.3798, 3.374, 3.3705, 3.3698, 3.3655, 3.365, 3.3639, 3.3606, 3.3549, 3.3542, 3.3525, 3.352, 3.3491, 3.3485, 3.3448, 3.3404, 3.3338, 3.3323, 3.3299, 3.3238, 3.3184, 3.311, 3.2855, 3.2807, 3.2687, 2.9121, 2.9654, 2.58, 1.8545, 2.3824, 0.9067, 1.9066, 3.4837, 3.4787, 3.4781, 3.4766, 3.4736, 3.473, 3.4622, 3.4568, 3.4537, 3.4515, 3.4508, 3.4507, 3.4503, 3.4442, 3.4427, 3.4387, 3.4374, 3.4355, 3.4344, 3.4339, 3.4335, 3.4287, 3.4279, 3.4248, 3.4219, 3.419, 3.4184, 3.3998, 3.3909, 3.389, 3.3157, 2.9815, 3.1137, 2.0551, 1.0845, 0.5416, 3.5363, 3.5346, 3.5142, 3.5137, 3.5123, 3.4979, 3.4927, 3.4918, 3.4735, 3.4415, 3.4128, 3.4119, 3.4097, 3.3888, 3.3866, 3.3535, 3.3093, 3.2684, 3.254, 3.2379, 3.2184, 3.0491, 3.0388, 2.9651, 2.9542, 2.9048, 2.8603, 2.846, 2.7336, 2.6501, 2.5352, 2.3603, 2.2638, 2.0598, 1.1333, 2.1755, 2.5002, 4.0221, 4.0202, 4.013, 4.0095, 4.0016, 3.9985, 3.9899, 3.9898, 3.9878, 3.9815, 3.9781, 3.9718, 3.9698, 3.9664, 3.9625, 3.953, 3.9487, 3.9442, 3.944, 3.9416, 3.9411, 3.9385, 3.9193, 3.9154, 3.9115, 3.9096, 3.9013, 3.8911, 3.8879, 3.8467, 3.5611, 3.1171, 3.0384, 3.2833, 1.4963, 4.2697, 4.2568, 4.2564, 4.2534, 4.252, 4.2385, 4.2331, 4.2185, 4.2183, 4.2174, 4.2105, 4.203, 4.2, 4.1981, 4.1968, 4.1967, 4.1923, 4.1729, 4.1695, 4.156, 4.1385, 4.1344, 4.1252, 4.1204, 4.1164, 4.115, 4.0954, 4.087, 4.0838, 4.0092, 3.9154, 3.4008, 1.9528, 1.9543, 4.4653, 4.4188, 4.4175, 4.4105, 4.4068, 4.4047, 4.3919, 4.3913, 4.3653, 4.3652, 4.3638, 4.3418, 4.3364, 4.3349, 4.3338, 4.3185, 4.3118, 4.3085, 4.3078, 4.3059, 4.2963, 4.2933, 4.2807, 4.2766, 4.2757, 4.2314, 4.2204, 4.2056, 4.1567, 4.1481, 3.9279, 2.3251, 4.7095, 4.6865, 4.6724, 4.6508, 4.6466, 4.6441, 4.6371, 4.6144, 4.6055, 4.5561, 4.5522, 4.5517, 4.5445, 4.4682, 4.4682, 4.4008, 4.3957, 4.3791, 4.352, 4.2674, 4.2533, 4.1014, 4.1001, 4.0972, 3.7488, 3.7331, 3.502, 3.5018, 3.2984, 2.6468, 4.7533, 4.7517, 4.7416, 4.7398, 4.7352, 4.7343, 4.7247, 4.7087, 4.6877, 4.6716, 4.6443, 4.6308, 4.602, 4.5969, 4.5843, 4.5401, 4.5355, 4.4884, 4.4297, 4.3224, 4.2302, 4.104, 3.8882, 3.8366, 3.7774, 1.9962, 1.3099, 1.3099, 1.3096, 1.3095, 1.2799, 1.3093, -2.3647]}, \"token.table\": {\"Topic\": [1, 4, 5, 10, 18, 16, 6, 12, 3, 4, 17, 2, 14, 2, 4, 9, 14, 2, 7, 6, 20, 15, 7, 14, 5, 9, 15, 9, 15, 4, 3, 2, 7, 19, 4, 12, 15, 17, 3, 10, 16, 20, 10, 3, 3, 2, 3, 2, 4, 5, 10, 6, 3, 1, 3, 14, 11, 4, 7, 16, 8, 10, 17, 19, 18, 2, 12, 1, 7, 1, 9, 1, 2, 1, 2, 10, 11, 16, 16, 8, 17, 11, 14, 12, 7, 1, 14, 16, 16, 2, 6, 14, 1, 2, 12, 8, 9, 14, 7, 17, 20, 2, 6, 11, 3, 9, 12, 7, 15, 12, 18, 12, 12, 4, 2, 4, 7, 6, 12, 17, 10, 2, 11, 11, 16, 12, 9, 12, 17, 16, 2, 16, 1, 13, 15, 1, 3, 9, 14, 16, 13, 17, 2, 18, 1, 10, 11, 13, 4, 1, 3, 4, 10, 13, 20, 1, 13, 18, 7, 15, 5, 15, 17, 9, 12, 2, 6, 6, 15, 17, 3, 5, 16, 5, 5, 7, 1, 10, 14, 8, 6, 13, 14, 1, 17, 8, 4, 4, 1, 4, 6, 11, 14, 7, 11, 19, 7, 2, 3, 5, 19, 11, 20, 20, 20, 3, 1, 8, 7, 12, 4, 12, 12, 20, 1, 11, 10, 1, 1, 2, 5, 7, 8, 9, 18, 1, 9, 9, 11, 8, 18, 3, 5, 14, 7, 9, 8, 4, 5, 1, 1, 2, 16, 20, 17, 5, 6, 5, 1, 17, 9, 12, 13, 2, 11, 14, 5, 10, 13, 18, 13, 13, 5, 14, 14, 11, 12, 11, 2, 4, 5, 6, 5, 7, 10, 13, 4, 2, 12, 13, 3, 17, 14, 11, 9, 5, 2, 19, 6, 14, 10, 11, 10, 1, 5, 15, 2, 10, 10, 2, 8, 13, 1, 4, 18, 14, 4, 4, 9, 1, 2, 1, 2, 3, 5, 9, 10, 13, 8, 9, 3, 3, 5, 12, 6, 7, 8, 15, 6, 6, 12, 6, 10, 6, 6, 12, 3, 8, 8, 3, 12, 1, 13, 6, 10, 14, 7, 14, 19, 7, 19, 16, 6, 3, 1, 4, 5, 6, 11, 11, 17, 1, 6, 19, 17, 11, 2, 15, 14, 17, 5, 5, 8, 8, 13, 17, 1, 5, 7, 9, 13, 13, 4, 12, 11, 1, 6, 4, 9, 17, 6, 17, 3, 3, 11, 17, 7, 19, 2, 17, 13, 3, 12, 18, 1, 3, 3, 1, 9, 17, 9, 3, 7, 3, 3, 19, 20, 18, 3, 2, 5, 11, 16, 1, 3, 9, 4, 11, 9, 12, 20, 10, 19, 9, 6, 10, 6, 10, 1, 5, 6, 8, 14, 1, 3, 5, 6, 13, 14, 2, 3, 5, 6, 10, 2, 7, 8, 19, 10, 11, 9, 10, 9, 10, 2, 8, 16, 3, 5, 5, 5, 1, 2, 13, 13, 8, 18, 17, 4, 15, 15, 2, 1, 17, 5, 9, 7, 15, 9, 3, 9, 15, 9, 1, 18, 6, 20, 11, 1, 6, 2, 8, 13, 7, 11, 2, 20, 11, 3, 11, 8, 17, 8, 8, 16, 12, 12, 16, 15, 19, 16, 10, 10, 2, 6, 1, 2, 1, 7, 10, 9, 16, 3, 9, 14, 8, 14, 16, 20, 15, 16, 18, 2, 7, 2, 5, 10, 3, 4, 8, 15, 15, 5, 5, 19, 5, 5, 10, 1, 3, 5, 10, 7, 18, 16, 1, 6, 6, 12, 2, 4, 2, 4, 16, 19, 17, 1, 2, 10, 5, 1, 18, 20, 16, 13, 16, 15, 2, 11, 20, 20, 20, 2, 9, 12, 11, 18, 4, 1, 4, 6, 4, 2, 3, 5, 18, 9, 4, 17, 6, 3, 12, 4, 12, 15, 17, 18, 9, 18, 7, 12, 2, 6, 15, 4, 5, 20, 2, 4, 6, 5, 4, 15, 16, 1, 3, 8, 14, 10, 3, 4, 14, 6, 5, 5, 19, 9, 1, 5, 17, 20, 18, 18, 7, 12, 6, 13, 1, 7, 8, 8, 9, 15, 8, 2, 7, 12, 18, 3, 19, 1, 2, 12, 8, 8, 9, 1, 7, 9, 14, 19, 3, 12, 3, 5, 3, 19, 12, 4, 1, 4, 1, 9, 12, 1, 4, 7, 10, 3, 11, 13, 2, 19, 15, 9, 10, 2, 5, 6, 5, 14, 9, 4, 5, 13, 15, 19, 13, 6, 1, 7, 15, 10, 15, 3, 5, 6, 7, 15, 5, 15, 13, 8, 7, 18, 9, 4, 11, 1, 19, 11, 17, 2, 5, 10, 13, 18, 13, 13, 16, 7, 3, 5, 13, 13, 18, 16, 18, 2, 16, 12, 8, 14, 8, 6, 6, 13, 14, 10, 13, 2, 9, 11, 14, 1, 2, 3, 6, 13, 4, 1, 1, 7, 5, 8, 9, 12, 13, 14, 11, 15, 1, 3, 7, 18, 18, 11, 5, 6, 14, 13, 6, 14, 5, 8, 5, 7, 4, 4, 5, 10, 12, 2, 16, 18, 1, 11, 11, 20, 18, 1, 1, 10, 8, 6, 2, 7, 8, 5, 2, 6, 10, 16, 13, 7, 8, 3, 17, 6, 6, 13, 9, 19, 10, 6, 14, 2, 4, 16, 3, 19, 3, 16, 8, 9, 15, 15, 18, 2, 13, 4, 11, 13, 1, 5, 15, 5, 1, 20, 16, 8, 1, 2, 12, 1, 4, 20, 4, 1, 2, 6, 10, 1, 2, 14, 1, 2, 9, 10, 14, 7, 19, 19, 9, 3, 16, 1, 2, 14, 13, 16, 12, 6, 11, 17, 14, 5, 8, 13, 19, 20, 1, 15, 2, 9, 2, 5, 11, 4, 3, 5, 4, 6, 12, 14, 9, 8, 2, 13, 3, 11, 19, 5, 6, 6, 2, 5, 9, 10, 12, 15, 15, 10, 2, 3, 7, 8, 12, 14, 10, 4, 10, 10, 6, 7, 7, 18, 6, 20, 11, 3, 3, 10, 13, 9, 4, 4, 7, 5, 13, 1, 3, 2, 15, 12, 9, 15, 2, 20, 20, 3, 3, 19, 17, 2, 8, 19, 17, 5, 18, 3, 5, 6, 3, 8, 4, 7, 2, 3, 12, 17], \"Freq\": [0.20708282732921118, 0.5157534567444505, 0.12112391787180277, 0.14847447997188726, 0.7729446602159538, 0.9599521223809246, 0.9614079921634067, 0.9168441691484396, 0.23153828824418363, 0.7480467774042856, 0.9447031270472984, 0.9812891617005834, 0.9823252093706869, 0.40892646496191676, 0.1871358398978263, 0.3881335938621583, 0.013861914066505653, 0.020556012640377538, 0.976410600417933, 0.9965504804946316, 0.7234709131409195, 0.8246226213453526, 0.9667878786841465, 0.9501870969571046, 0.458065528524059, 0.09552077372065686, 0.4450399684712422, 0.16789176296258188, 0.8250680922732595, 0.9787113567018609, 0.9820697462251082, 0.49320288013733915, 0.49320288013733915, 0.7692008657212615, 0.9260997917198793, 0.047492297011275865, 0.821983668585455, 0.9726058254728149, 0.9945749351592176, 0.9287480726322732, 0.938118168473422, 0.6011079273054835, 0.9648299612517882, 0.9885502743406436, 0.9874508662164747, 0.8948582589580297, 0.08464875422575957, 0.9534977029832766, 0.04809557337947894, 0.9426732382377871, 0.9862440928165943, 0.9749678339323875, 0.9856564389466289, 0.6116125176844439, 0.3847893323180276, 0.9238370076507029, 0.9641254879225429, 0.9468456220565893, 0.9949220940932362, 0.9185381169390563, 0.08200534947895499, 0.8747237277755199, 0.9203566565319459, 0.6957706385480846, 0.8693296314407234, 0.9945885683285148, 0.9934801145028823, 0.033828288752999744, 0.9556491572722429, 0.9456099104030669, 0.04828646350994384, 0.06701193348794494, 0.9113622954360513, 0.38371690210220205, 0.5574754992805577, 0.04343964929458891, 0.007239941549098152, 0.9569397989037263, 0.9030159447442628, 0.986113419222627, 0.798677605254256, 0.9643506994312637, 0.9297558741287154, 0.9540672787753836, 0.8429102759850209, 0.977251515474441, 0.9406092674191339, 0.8889127798133751, 0.9742491740015375, 0.2110904033745668, 0.76945856713955, 0.013618735701584955, 0.11595705562013897, 0.6164032956649493, 0.2624291258771566, 0.9846359921175872, 0.8857354125885385, 0.9377407274774934, 0.9912662561865655, 0.8935412915671688, 0.6899936684839816, 0.8923088501319505, 0.0932262977749799, 0.9502658352447583, 0.2732891318402556, 0.4858473454937878, 0.22774094320021301, 0.37697463517794666, 0.614329035104802, 0.947560324957374, 0.8794064785315585, 0.9616779453477039, 0.8377078490625696, 0.9800438869000853, 0.733682551169472, 0.2201047653508416, 0.036684127558473605, 0.3941868087532644, 0.49876698250413043, 0.09653554500079944, 0.9864282050095932, 0.975439197173019, 0.9595269983420233, 0.495726564855378, 0.4461539083698402, 0.8302637315751963, 0.9795555858586861, 0.9509128001317487, 0.8702480441166403, 0.9169988096629231, 0.9761702060879552, 0.9106512733959705, 0.6500350923852717, 0.035456459584651184, 0.30728931640031026, 0.9622553904858512, 0.19202177796492562, 0.6446445403108216, 0.12344257154888075, 0.013715841283208971, 0.9446797753349728, 0.9372589618671242, 0.9868879201756054, 0.873668007787562, 0.9856995117159087, 0.9707183212652916, 0.9379180616143313, 0.9533073786450478, 0.9893768973737957, 0.6634385917208494, 0.20732455991276544, 0.12439473594765926, 0.9255835531291425, 0.9474531771974616, 0.9504399077655887, 0.9619671906025882, 0.9646116994631562, 0.8092616808297081, 0.9732577515910271, 0.9585835768479299, 0.9652781975875028, 0.6462178980857598, 0.9513583711775819, 0.9052708515838785, 0.08229735014398895, 0.8483860417836144, 0.14264898047689092, 0.7262840355259539, 0.24846559110098423, 0.8962726870652661, 0.9686340977331854, 0.9511570829696798, 0.9668929703008701, 0.9697335198270063, 0.3221249263757365, 0.6710935966161177, 0.8568896884505494, 0.03254011475128669, 0.10846704917095562, 0.9881491892661299, 0.9192860386147195, 0.06413623525218973, 0.8720333733614538, 0.5213005537926994, 0.42651863492129943, 0.9871564108702825, 0.9808502994838535, 0.9970129351547921, 0.7824490577531805, 0.056562582488181724, 0.10369806789499982, 0.0031423656937878733, 0.05342021679439385, 0.07137940539682965, 0.2855176215873186, 0.628138767492101, 0.9335688654156739, 0.2655572016342881, 0.25080402376571653, 0.2212976680285734, 0.23605084589714498, 0.9645418769622657, 0.9597416151205714, 0.8381552677522638, 0.5123777845275274, 0.9932923746655584, 0.9752163861217374, 0.9395504912811454, 0.9016135658931821, 0.08820132709824607, 0.9706047037685689, 0.9851632773494983, 0.9596889976982351, 0.8292834141055622, 0.18280608328006256, 0.7921596942136044, 0.9443118422100354, 0.9684693719532853, 0.6580874548222492, 0.04333086122286414, 0.08666172244572828, 0.13811712014787947, 0.06770447066072523, 0.9588957682349075, 0.8898823995061362, 0.9783369805291237, 0.0155291584210972, 0.959680290899881, 0.943719143668167, 0.9705715263061319, 0.7363476103771825, 0.865911096051501, 0.11173046400664528, 0.9546043407624412, 0.8928572663143125, 0.967357710221204, 0.9584039653446277, 0.2689639036633575, 0.7256950608275494, 0.9694315451129479, 0.7969651099380618, 0.11052800794761442, 0.08144169006666326, 0.8274749682075616, 0.7789098209384889, 0.2273324156723572, 0.7577747189078573, 0.9778337155543472, 0.8982928112472895, 0.09651079790260136, 0.9568420246922296, 0.9759220014134419, 0.8982332185869942, 0.9814171046488444, 0.9562163121009656, 0.9750046128139399, 0.9647901752893036, 0.3095090211120612, 0.6534079334587959, 0.8880132261395053, 0.9786473375505578, 0.9635721238234843, 0.9910031593879223, 0.9130526761259736, 0.9506236572511438, 0.9413807101008559, 0.9272893881449649, 0.967772900186683, 0.02813372676690818, 0.4735844005762877, 0.11956833875935977, 0.3751163568921091, 0.6074530634681826, 0.2672793479260004, 0.12149061269363653, 0.9941694384701635, 0.9517206070918206, 0.2304419469609891, 0.6913258408829673, 0.9806109687442531, 0.2762557191308666, 0.6630137259140798, 0.9889690877811884, 0.9858800946514998, 0.9946389510304957, 0.9940188285519452, 0.9628871415067236, 0.5048188543696472, 0.9947387661590129, 0.9678763053885223, 0.9040478006250501, 0.07090570985294511, 0.987780042580041, 0.9721213674927887, 0.6181396946970713, 0.3585210229243013, 0.2958721080422509, 0.6851775133610021, 0.9334496734603706, 0.9856594191575021, 0.9465662145901743, 0.8917659481789147, 0.16529207984321684, 0.7934019832474408, 0.03305841596864337, 0.95668020289111, 0.9756803106933831, 0.9891945930799279, 0.009588315926461336, 0.6913983559287871, 0.29723667638060003, 0.4178529477917889, 0.24779651555094456, 0.07288132810321898, 0.2283614947234195, 0.014576265620643798, 0.004858755206881266, 0.009717510413762532, 0.09511185394653776, 0.8797846490054743, 0.9837450307322356, 0.13515765046473338, 0.8447353154045836, 0.9381361964756268, 0.008368851090348683, 0.9875244286611448, 0.41948860641296143, 0.5243607580162019, 0.9352832094764681, 0.978070801425357, 0.9712680354148975, 0.16003554375651127, 0.8357411729506701, 0.959539362225839, 0.11738823700088741, 0.8510647182564337, 0.18486621282577523, 0.801086922245026, 0.9644870300966959, 0.9854471320133763, 0.763080120461532, 0.9907542959504321, 0.9775233789072322, 0.8519397724687231, 0.13451680617927206, 0.9547489904951357, 0.9982145866705749, 0.8878260421928983, 0.37975798823838514, 0.988396668113454, 0.5704546676802218, 0.974201851323544, 0.9848023102343227, 0.9839116362405298, 0.4315367312533391, 0.005604373133160248, 0.4259323581201788, 0.07846122386424347, 0.05043935819844223, 0.10738289044708682, 0.8829259881204917, 0.03894475699303161, 0.9541465463292745, 0.9154870185634609, 0.8150665447805496, 0.9773806512534269, 0.9779776650833207, 0.8252043491566986, 0.9458140353726584, 0.9446528591855574, 0.9375698659529109, 0.9648890302373684, 0.9640797848934914, 0.6228497434747517, 0.3593363904662029, 0.9473049190034564, 0.29531432046459444, 0.13125080909537531, 0.377346076149204, 0.188673038074602, 0.9395088574041636, 0.965152221270433, 0.9878122944959566, 0.8779124428800243, 0.9702347798194988, 0.9780177166710946, 0.01715820555563324, 0.8526234729973107, 0.033436214627345515, 0.10030864388203656, 0.9972430355518996, 0.844849781721536, 0.9671793862153935, 0.9680391809759687, 0.9840460122361443, 0.8942203673692627, 0.9947780248116105, 0.9140753123964738, 0.9842218512237512, 0.005893544019303899, 0.9813497726069947, 0.9620459288748412, 0.4085358219280433, 0.5798572956398034, 0.9726458149276668, 0.9588465674158028, 0.9780482888344014, 0.9632652400995934, 0.02963893046460287, 0.7876596869944185, 0.9932547959168843, 0.08743362103630654, 0.8918229345703267, 0.995747856945563, 0.985438103399076, 0.8904035840710983, 0.7303801907437659, 0.9810521101941935, 0.9634739202825832, 0.9936803332935092, 0.9922433957130349, 0.005088427670323256, 0.9742120579565442, 0.9851557073189341, 0.08012002136717912, 0.90802690882803, 0.9540910682311855, 0.968797866999416, 0.9618692698703138, 0.8903492958385383, 0.8728869221436519, 0.9498932194904366, 0.3785190684383327, 0.9803698389743075, 0.9825364601294699, 0.9035654595632129, 0.9895461605567141, 0.9763389580368728, 0.12032135529468188, 0.38101762509982595, 0.38603101490377106, 0.10026779607890157, 0.010026779607890156, 0.15963013391663253, 0.45300443408774094, 0.07765790298646988, 0.1337441662544759, 0.08197223093016265, 0.09060088681754819, 0.2905285531511418, 0.2398013454580853, 0.22596665245088807, 0.2305782167866205, 0.004611564335732409, 0.4921570302979811, 0.3922990821215791, 0.11412336934445938, 0.7554051462560879, 0.9685229839763505, 0.9582372492296918, 0.36237978646636443, 0.6323882548138516, 0.14147632921921874, 0.8488579753153124, 0.8359411900201797, 0.06146626397207204, 0.08605276956090085, 0.2554406301424295, 0.7368479715647005, 0.9711871795449998, 0.9800215510936858, 0.02803916469869153, 0.7570574468646714, 0.21229653271866444, 0.9441401303714093, 0.9356704001908591, 0.9438407096797233, 0.8204031092608536, 0.9889841452364576, 0.9877372996118475, 0.9419782547142164, 0.9736475957307282, 0.9724513101317279, 0.9059483144460324, 0.9907731653220643, 0.9693203038958161, 0.77003765991618, 0.192509414979045, 0.9553438188863926, 0.21244115037900668, 0.25796425403165096, 0.5007541401790871, 0.9570676341822089, 0.9809500635586865, 0.8587893244960733, 0.9199064075152059, 0.03679625630060824, 0.9633248333123956, 0.9970087349699966, 0.9688433910204591, 0.9897919737214793, 0.9605945012793979, 0.9556437198281301, 0.9243956039402821, 0.9218613024613654, 0.9851370895363434, 0.915978008736385, 0.9265121740025597, 0.9658776489305448, 0.9713654464578447, 0.9931946008748259, 0.8675164219690746, 0.9940826987437603, 0.0904304064231494, 0.8816964626257066, 0.8438620798465546, 0.9211283368410703, 0.8413121234652451, 0.8545597174487509, 0.7934155665699882, 0.9165826164062953, 0.9682409067547704, 0.9459007694327324, 0.15072378423305108, 0.8373543568502838, 0.9932540833081228, 0.005442488127715741, 0.9799562499043255, 0.9848408151818608, 0.9373819338440128, 0.9765154062222307, 0.93828991900541, 0.9904791377448514, 0.9848548072643909, 0.9778006034696525, 0.9872362663155677, 0.9128603372182986, 0.8990276104223285, 0.8478241953231904, 0.973637696540635, 0.9764980787121399, 0.8284702718109964, 0.9755196534440859, 0.9363992547973827, 0.13970795978230585, 0.8103061667373739, 0.02794159195646117, 0.9916245847763872, 0.13017663356870512, 0.3062979613381297, 0.5589937794420867, 0.9769453321242091, 0.9638564868814754, 0.8640090842682094, 0.12342986918117278, 0.98434061254181, 0.9606653950231543, 0.9010411278476641, 0.8938642731126689, 0.09246871790820713, 0.006164581193880475, 0.9894574995568274, 0.8636921734142524, 0.11407255120565599, 0.9404259226904139, 0.8584630891992592, 0.13846178858052569, 0.12265915291777674, 0.8586140704244372, 0.025278538670358015, 0.9732237388087837, 0.3469024754567492, 0.6359878716707068, 0.9297436704061072, 0.8835170355838935, 0.8644618589704147, 0.9826986211659033, 0.9911964891227245, 0.9613032668815121, 0.9517827555716231, 0.8190154744849021, 0.08190154744849021, 0.05460103163232681, 0.8581070914281602, 0.9630017705627986, 0.893192586112329, 0.48087686181477257, 0.9139227523335333, 0.07947154368117681, 0.9426896609752874, 0.9461958236911453, 0.9448055564785143, 0.9368195520041539, 0.039034148000173075, 0.9497465852593521, 0.9758918914275361, 0.8990296616587524, 0.9581141921332786, 0.5240651456408352, 0.007595147038272973, 0.4683674006935, 0.9801636864305332, 0.03182388658115857, 0.16972739509951235, 0.7849892023352447, 0.9447868903570319, 0.9663968183758244, 0.9709969556214662, 0.9641131948999014, 0.9543488442251247, 0.9644077292079392, 0.021918357481998618, 0.9726378097423132, 0.9975811141996395, 0.7866577071981667, 0.956597565891558, 0.8721136216488682, 0.9180228054774044, 0.9100273582126848, 0.23164710042779837, 0.694941301283395, 0.9796507463364439, 0.19163935604644636, 0.7665574241857854, 0.9800026349266594, 0.9803906087581004, 0.4781408655189167, 0.6991931182611144, 0.21665138875696502, 0.06893453278630705, 0.9834853299472974, 0.9768764286265583, 0.9330196722421005, 0.9603959545743724, 0.9746554929389443, 0.5078813026824098, 0.2497776898438081, 0.2414517668490145, 0.994042009428587, 0.13722560390110863, 0.2495010980020157, 0.5988026352048377, 0.9557383837873143, 0.9727358363895925, 0.3983919067056099, 0.5311892089408132, 0.9695083831014385, 0.9392053760755329, 0.04559249398424917, 0.9436834965976765, 0.7482833966778489, 0.8709654536775286, 0.8863789786875428, 0.959157805545732, 0.949802673720086, 0.9280129425399377, 0.04034838880608425, 0.139850506698735, 0.07866591001803844, 0.7691777868430424, 0.969084418084874, 0.9907645002777719, 0.9639250949268704, 0.9436510549760706, 0.8296804606007123, 0.0029952363198581674, 0.1647379975921992, 0.7432719734193702, 0.9904435221933813, 0.5094975472144023, 0.6570728022479608, 0.28160262953484033, 0.0505440617113816, 0.9515869687929746, 0.03204509119454584, 0.9659306060070246, 0.9657343744572647, 0.6763319389759374, 0.2861404357205889, 0.9886692875459747, 0.9637969293708873, 0.18329932586071224, 0.733197303442849, 0.9007319714468034, 0.07205855771574428, 0.9896838650629137, 0.724097783311521, 0.9899376885011661, 0.9687959836121399, 0.11958194554979658, 0.8738680636331289, 0.7949496258321783, 0.013825210884037884, 0.18664034693451143, 0.9665977536516859, 0.8630202594593088, 0.119863924924904, 0.9586001619659985, 0.969881063352725, 0.7660216393385533, 0.22603917226383538, 0.9566340799677819, 0.8091347036531671, 0.7257062186385265, 0.9615431415621714, 0.9627675509672422, 0.9858278887262834, 0.9886399689767217, 0.9832101309598329, 0.9717288130894596, 0.9841288276240302, 0.9259084934552992, 0.06589439446074462, 0.15061575876741626, 0.02824045476889055, 0.7342518239911543, 0.9365149447970511, 0.9755220823991994, 0.9742447015846903, 0.07697961352394402, 0.6350818115725381, 0.27905109902429703, 0.988233502833236, 0.9944930286087309, 0.42199000444698126, 0.005601637227172317, 0.23526876354123732, 0.24460482558652452, 0.08962619563475707, 0.49805563644412815, 0.49805563644412815, 0.9743412453437995, 0.9852335830667711, 0.927169831140955, 0.9565692761694279, 0.962017251487421, 0.9415376656936433, 0.9831339282958876, 0.9751302266937308, 0.9181399255404545, 0.9940504175387375, 0.8672593932614358, 0.8354821261520468, 0.14677388702671093, 0.09014318070498029, 0.8864079435989728, 0.8387386923350996, 0.9932260607476443, 0.9221425747847435, 0.887713837302035, 0.9417139665897902, 0.36800844910405434, 0.1784283389595415, 0.4460708473988538, 0.2619179273241055, 0.7071784037750849, 0.9437420251557481, 0.8820808138465925, 0.6123608400357654, 0.36741650402145926, 0.9768810787829392, 0.9849488138526812, 0.97058636922399, 0.9723851595772494, 0.9336969491971032, 0.9866540148529297, 0.9432407174834303, 0.968033022082432, 0.9588694323352556, 0.9840428612296397, 0.9962051987143543, 0.8114538041524036, 0.1678869939625663, 0.9438790183567027, 0.4283199472148952, 0.43251916238366866, 0.08398430337546965, 0.025195291012640893, 0.029394506181414377, 0.9897607886682988, 0.9787510784231583, 0.9931580397874387, 0.9447431708592103, 0.6099316801656494, 0.0066296921757135795, 0.15248292004141234, 0.1657423043928395, 0.05966722958142222, 0.9803580315866197, 0.9877468434727322, 0.9085853618825162, 0.9779372918741095, 0.9760132963657807, 0.9545933706835653, 0.8213802111078001, 0.9643993747791394, 0.9673474502831286, 0.9865120466804104, 0.8503526181809551, 0.13426620287067711, 0.9221631298002895, 0.29388899286294057, 0.694646710403314, 0.9567615421979442, 0.9317806852828637, 0.9915944279773299, 0.8568705741659157, 0.9656158173598735, 0.1656980914649209, 0.2982565646368576, 0.1656980914649209, 0.364535801222826, 0.9720611137761188, 0.8810668056770925, 0.8674623072355564, 0.34352062904083613, 0.6526891951775886, 0.9536756269189478, 0.9395688823742046, 0.9260117565569762, 0.9584065405879753, 0.9771819717130281, 0.9333731039722053, 0.9468098312314984, 0.9828841091704591, 0.9505626080388119, 0.9334556893623303, 0.9638118301001433, 0.9807301248161024, 0.9611043595760923, 0.05703729020072673, 0.9268559657618093, 0.9423447331357477, 0.9941272356989502, 0.905835051636923, 0.9856371172451502, 0.975090144859921, 0.8703174877961263, 0.9941708806785162, 0.9851746751545739, 0.010922113915239179, 0.9696581425700928, 0.8397887695498473, 0.9888144342811523, 0.9869908040266693, 0.9845516643834235, 0.6186993818245305, 0.37121962909471834, 0.9182136013569245, 0.6612647424293941, 0.30859021313371726, 0.5758569956374997, 0.3971427556120688, 0.9943286246861532, 0.8871082143368634, 0.9586023031366094, 0.8818155049691686, 0.6677751737243596, 0.3751690071333448, 0.6163490831476378, 0.9698696239043485, 0.9641374719671637, 0.9420113865843679, 0.8910446383072825, 0.09652983581662228, 0.9140963632723811, 0.9743105744535485, 0.9524233206639365, 0.9585151484932102, 0.8558163338956477, 0.9419925544457756, 0.6193323528420209, 0.36999075624328526, 0.8940713827968676, 0.15548547582537048, 0.8318472956657321, 0.7119067187820567, 0.9949178701468584, 0.8862293617906671, 0.10358525007942863, 0.0028773680577619063, 0.005754736115523813, 0.9558265750094909, 0.023503932172364527, 0.0182808361340613, 0.1945441649906211, 0.20670317530253493, 0.5836324949718633, 0.044582486648612904, 0.9473778412830242, 0.9707549885843471, 0.9054752852532145, 0.7552187913934775, 0.9287478596886319, 0.31733716620827557, 0.6346743324165511, 0.20471185004898254, 0.773355877962823, 0.9669082722391996, 0.9107931857015943, 0.9150280734028705, 0.8231153757345357, 0.9879610598906017, 0.9545612884058037, 0.7773683508295637, 0.9342758394696891, 0.11667584036485076, 0.7292240022803174, 0.11667584036485076, 0.972148851701948, 0.39946304113531556, 0.7633067600809551, 0.22372784347200408, 0.9743930574540154, 0.022399840401241735, 0.9760411334536041, 0.9653602683017086, 0.9377701771395089, 0.9619506609056382, 0.2034914735495301, 0.7925457390876435, 0.99146763248871, 0.9364922337396081, 0.10670931589547085, 0.8536745271637668, 0.9549247387325486, 0.9924831229416479, 0.03704021353866475, 0.944525445235951, 0.48601452081888796, 0.48601452081888796, 0.6158680538310213, 0.6633554730725153, 0.3209784547125074, 0.9778881316819356, 0.9838950564363255, 0.7038987211035205, 0.10463359367755033, 0.18073075271576877, 0.6935881280520497, 0.2600955480195187, 0.835336544678628, 0.9043864597116253, 0.1584026737675973, 0.05374376431400623, 0.3931780652445719, 0.16971715046528282, 0.22346091477928906, 0.9841090742513, 0.9355605182670375, 0.4681616301805456, 0.5249084944448541, 0.954119129793946, 0.06255896078182048, 0.9279579182636705, 0.9813447800315325, 0.7086617580493719, 0.9955468306276719, 0.968343029928713, 0.9674634736126481, 0.9792819778223716, 0.7230863418471551, 0.1382370947648973, 0.12760347209067444, 0.9684585988455406, 0.996698590073619, 0.9918243198749995, 0.9940156067046239, 0.9502957560357975, 0.9691703176684276, 0.8322885779862195, 0.16166756550811456, 0.9746076377654437, 0.019427394772733097, 0.982667566602716, 0.6158605649631358, 0.3695163389778815, 0.9693362422869183, 0.49457424554291074, 0.8988247825736916, 0.9868925914494745, 0.9769955632906211, 0.8110028101650404, 0.749367670849925, 0.9742537361875835, 0.9622167999728092, 0.6869628462801814, 0.8187035490726428, 0.9894235805989843, 0.8683895918524914, 0.06967289974636422, 0.9243271366350986, 0.9905596120711546, 0.9795055133602796, 0.9707648601594859, 0.87599304836375, 0.10428488670997024, 0.31721755349034253, 0.2413611820035215, 0.44134616137786786, 0.9342966590549281], \"Term\": [\"abbreviated\", \"abbreviated\", \"abbreviated\", \"abbreviated\", \"ability\", \"above\", \"abroad\", \"abuse\", \"access\", \"access\", \"accident\", \"accidents\", \"accommodation\", \"according\", \"according\", \"according\", \"according\", \"account\", \"account\", \"accounts\", \"accurate\", \"acquired\", \"acquisition\", \"active\", \"activities\", \"activities\", \"activities\", \"activity\", \"activity\", \"actual\", \"actually\", \"added\", \"added\", \"adding\", \"additional\", \"additional\", \"advantage\", \"agreement\", \"agricultural\", \"alcohol\", \"allowing\", \"amount\", \"amounts\", \"animal\", \"animals\", \"annual\", \"annual\", \"applicable\", \"application\", \"application\", \"applied\", \"applies\", \"arable\", \"areas\", \"areas\", \"arrears\", \"artificial\", \"assessed\", \"assets\", \"assigned\", \"associated\", \"associated\", \"atmosphere\", \"attached\", \"august\", \"available\", \"average\", \"balance\", \"balance\", \"based\", \"based\", \"basic\", \"basic\", \"basis\", \"basis\", \"basis\", \"basis\", \"battery\", \"beginning\", \"benefits\", \"berry\", \"better\", \"bills\", \"birth\", \"bodies\", \"border\", \"borrowing\", \"bought\", \"building\", \"business\", \"business\", \"business\", \"calculated\", \"calculated\", \"calculated\", \"calendar\", \"capable\", \"capacity\", \"capital\", \"carbon\", \"cards\", \"carried\", \"carried\", \"carrying\", \"cases\", \"cases\", \"cases\", \"categories\", \"categories\", \"cause\", \"caused\", \"causes\", \"causing\", \"central\", \"certain\", \"certain\", \"certain\", \"change\", \"change\", \"change\", \"changes\", \"characteristics\", \"chemical\", \"chemicals\", \"chemicals\", \"child\", \"children\", \"chosen\", \"citrus\", \"civil\", \"class\", \"classes\", \"classification\", \"classification\", \"classification\", \"classifications\", \"classified\", \"classified\", \"classified\", \"classified\", \"classify\", \"climate\", \"collected\", \"collecting\", \"collection\", \"combined\", \"combustion\", \"commercial\", \"commission\", \"common\", \"common\", \"common\", \"commonly\", \"communication\", \"company\", \"comparable\", \"compiling\", \"completely\", \"composed\", \"comprise\", \"comprises\", \"conceived\", \"condition\", \"conditions\", \"conditions\", \"considered\", \"considered\", \"consist\", \"consist\", \"constitute\", \"construction\", \"consumed\", \"consumer\", \"consumers\", \"consumption\", \"consumption\", \"context\", \"context\", \"context\", \"contributions\", \"control\", \"control\", \"controlled\", \"convention\", \"convention\", \"costs\", \"council\", \"countries\", \"country\", \"country\", \"country\", \"country\", \"country\", \"cover\", \"cover\", \"cover\", \"coverage\", \"covered\", \"covered\", \"covered\", \"covered\", \"created\", \"credit\", \"crime\", \"cropland\", \"crops\", \"cross\", \"crude\", \"current\", \"current\", \"customs\", \"death\", \"deaths\", \"debit\", \"december\", \"december\", \"decisions\", \"deficit\", \"defined\", \"defined\", \"defined\", \"defined\", \"defined\", \"defines\", \"defining\", \"definition\", \"definition\", \"degree\", \"denmark\", \"depending\", \"descriptions\", \"designed\", \"designed\", \"destination\", \"determine\", \"developed\", \"developing\", \"development\", \"development\", \"difference\", \"different\", \"different\", \"different\", \"difficulties\", \"dioxide\", \"direct\", \"direct\", \"directive\", \"directly\", \"directly\", \"disability\", \"disease\", \"diseases\", \"disposable\", \"disposal\", \"dissemination\", \"distinguish\", \"distinguished\", \"distinguished\", \"distinguishes\", \"distribution\", \"division\", \"domestic\", \"duration\", \"dwelling\", \"earlier\", \"early\", \"earnings\", \"economic\", \"economic\", \"economic\", \"economic\", \"economy\", \"economy\", \"economy\", \"education\", \"effect\", \"effects\", \"effects\", \"electricity\", \"emissions\", \"emissions\", \"employed\", \"employee\", \"employment\", \"energy\", \"engaged\", \"entering\", \"enterprise\", \"entity\", \"environment\", \"environment\", \"environmental\", \"equal\", \"equipment\", \"equipment\", \"equivalent\", \"equivalent\", \"equivalents\", \"equivalised\", \"especially\", \"essential\", \"established\", \"established\", \"established\", \"establishment\", \"europe\", \"european\", \"european\", \"eurostat\", \"eurostat\", \"example\", \"example\", \"example\", \"example\", \"example\", \"example\", \"example\", \"examples\", \"examples\", \"excludes\", \"excluding\", \"excluding\", \"expectancy\", \"expenditure\", \"expenditure\", \"expenditures\", \"expenditures\", \"export\", \"exports\", \"expressed\", \"external\", \"external\", \"extra\", \"factors\", \"factors\", \"family\", \"family\", \"farmer\", \"farming\", \"fatal\", \"fertilisers\", \"field\", \"final\", \"final\", \"finance\", \"financial\", \"first\", \"fitted\", \"fixed\", \"floating\", \"floor\", \"flows\", \"fodder\", \"following\", \"following\", \"following\", \"following\", \"following\", \"force\", \"force\", \"foreign\", \"foreign\", \"forms\", \"fossil\", \"france\", \"frequency\", \"frequent\", \"freshwater\", \"fruit\", \"fuels\", \"fulfil\", \"functions\", \"funds\", \"funds\", \"gases\", \"general\", \"general\", \"general\", \"general\", \"generated\", \"generation\", \"geographical\", \"geometric\", \"germany\", \"given\", \"given\", \"global\", \"global\", \"global\", \"goods\", \"grant\", \"grassland\", \"grazing\", \"greater\", \"greenhouse\", \"gross\", \"ground\", \"group\", \"group\", \"groups\", \"grown\", \"growth\", \"growth\", \"harmonised\", \"harvest\", \"harvested\", \"having\", \"having\", \"headings\", \"health\", \"higher\", \"higher\", \"holding\", \"holdings\", \"holiday\", \"hospital\", \"hours\", \"house\", \"household\", \"households\", \"households\", \"housing\", \"however\", \"human\", \"human\", \"iceland\", \"identified\", \"illness\", \"immediate\", \"immediately\", \"impact\", \"impermeable\", \"implementation\", \"import\", \"imported\", \"imports\", \"improved\", \"include\", \"include\", \"include\", \"include\", \"include\", \"includes\", \"includes\", \"includes\", \"includes\", \"includes\", \"includes\", \"including\", \"including\", \"including\", \"including\", \"including\", \"income\", \"income\", \"income\", \"incorporating\", \"increase\", \"increasing\", \"indicator\", \"indicator\", \"indicators\", \"indicators\", \"individual\", \"individual\", \"individual\", \"industrial\", \"industrial\", \"industries\", \"industry\", \"information\", \"information\", \"information\", \"infrastructure\", \"inhabitants\", \"initial\", \"injury\", \"inland\", \"innovation\", \"innovations\", \"input\", \"instance\", \"instead\", \"institutions\", \"instrument\", \"instruments\", \"instruments\", \"intellectual\", \"intended\", \"intended\", \"intended\", \"intensity\", \"intensive\", \"interaction\", \"interest\", \"interest\", \"internal\", \"international\", \"intra\", \"investment\", \"involved\", \"involves\", \"involving\", \"ireland\", \"irrigation\", \"issued\", \"italy\", \"items\", \"january\", \"knowledge\", \"kyoto\", \"labour\", \"large\", \"large\", \"largely\", \"later\", \"latest\", \"latin\", \"layer\", \"laying\", \"leading\", \"leave\", \"legal\", \"legal\", \"level\", \"level\", \"levels\", \"liabilities\", \"limit\", \"limited\", \"litter\", \"livestock\", \"living\", \"loans\", \"local\", \"located\", \"locations\", \"lucas\", \"machinery\", \"mainly\", \"maintain\", \"major\", \"makes\", \"management\", \"management\", \"management\", \"manure\", \"market\", \"market\", \"market\", \"marketing\", \"markets\", \"material\", \"material\", \"materials\", \"matter\", \"meaning\", \"means\", \"means\", \"means\", \"measure\", \"measured\", \"measured\", \"measurement\", \"measures\", \"measures\", \"medical\", \"medical\", \"member\", \"member\", \"members\", \"members\", \"mentioned\", \"metal\", \"methane\", \"method\", \"methods\", \"million\", \"mineral\", \"minimum\", \"minimum\", \"minimum\", \"minor\", \"mixed\", \"mixtures\", \"modus\", \"monetary\", \"monetary\", \"money\", \"month\", \"monthly\", \"months\", \"months\", \"mortality\", \"motor\", \"moving\", \"multi\", \"national\", \"national\", \"national\", \"nations\", \"natural\", \"natural\", \"natural\", \"necessary\", \"needs\", \"network\", \"nitrogen\", \"nomenclature\", \"normally\", \"normally\", \"norway\", \"number\", \"numerical\", \"nurseries\", \"object\", \"objectives\", \"observation\", \"observed\", \"observed\", \"obtained\", \"occurs\", \"occurs\", \"operating\", \"operations\", \"orange\", \"order\", \"order\", \"order\", \"organic\", \"organisation\", \"organisational\", \"organisms\", \"organization\", \"other\", \"other\", \"other\", \"output\", \"outside\", \"outside\", \"outside\", \"ownership\", \"paper\", \"partially\", \"partially\", \"participation\", \"particular\", \"particular\", \"parties\", \"partly\", \"partners\", \"pasture\", \"patents\", \"patterns\", \"payment\", \"payment\", \"payments\", \"payments\", \"payments\", \"pension\", \"people\", \"performance\", \"performed\", \"period\", \"period\", \"period\", \"periodic\", \"permanent\", \"permeable\", \"person\", \"person\", \"person\", \"personnel\", \"persons\", \"persons\", \"phosphorus\", \"physical\", \"physical\", \"place\", \"placed\", \"places\", \"places\", \"plant\", \"plant\", \"plants\", \"plastic\", \"point\", \"policies\", \"policy\", \"policy\", \"population\", \"population\", \"population\", \"positive\", \"possible\", \"possible\", \"potential\", \"poultry\", \"power\", \"power\", \"practices\", \"precipitation\", \"preparations\", \"present\", \"presented\", \"previous\", \"prices\", \"primary\", \"principal\", \"private\", \"problems\", \"process\", \"process\", \"process\", \"process\", \"processed\", \"processes\", \"processing\", \"produced\", \"produced\", \"produced\", \"producers\", \"product\", \"production\", \"production\", \"production\", \"production\", \"production\", \"products\", \"products\", \"professional\", \"profit\", \"programmes\", \"progress\", \"project\", \"promote\", \"property\", \"proportion\", \"protected\", \"protection\", \"protocol\", \"provide\", \"provide\", \"providing\", \"providing\", \"provisions\", \"public\", \"purchase\", \"purchased\", \"purchases\", \"purposes\", \"purposes\", \"purposes\", \"quality\", \"quality\", \"quantities\", \"quantity\", \"range\", \"range\", \"rates\", \"ratio\", \"reasons\", \"received\", \"record\", \"recorded\", \"recreational\", \"reduce\", \"reducing\", \"refer\", \"reference\", \"referred\", \"referred\", \"referring\", \"refers\", \"refers\", \"refers\", \"refers\", \"refers\", \"region\", \"regional\", \"regions\", \"registered\", \"related\", \"related\", \"related\", \"related\", \"related\", \"relative\", \"release\", \"remaining\", \"removal\", \"removed\", \"reported\", \"republic\", \"require\", \"required\", \"research\", \"resident\", \"resident\", \"residential\", \"residents\", \"residents\", \"residues\", \"resource\", \"resources\", \"respective\", \"responsible\", \"result\", \"result\", \"result\", \"result\", \"results\", \"retail\", \"retirement\", \"rights\", \"rights\", \"risks\", \"roads\", \"round\", \"rules\", \"rural\", \"safety\", \"salaries\", \"sales\", \"sample\", \"satellite\", \"schemes\", \"science\", \"second\", \"secondary\", \"secondary\", \"section\", \"sector\", \"securities\", \"security\", \"seeds\", \"sequence\", \"service\", \"services\", \"services\", \"share\", \"sheets\", \"short\", \"significant\", \"similar\", \"single\", \"single\", \"slatted\", \"slurry\", \"slurry\", \"small\", \"small\", \"social\", \"society\", \"software\", \"solely\", \"sorting\", \"source\", \"source\", \"south\", \"spain\", \"specialised\", \"specific\", \"specific\", \"specifically\", \"specified\", \"spending\", \"spent\", \"stable\", \"staff\", \"standard\", \"standard\", \"starting\", \"state\", \"state\", \"statements\", \"states\", \"statistical\", \"statistical\", \"statistical\", \"statistical\", \"statistics\", \"statistics\", \"statistics\", \"status\", \"status\", \"status\", \"stock\", \"stock\", \"stocks\", \"store\", \"stores\", \"strategy\", \"straw\", \"straw\", \"structure\", \"structure\", \"structures\", \"study\", \"subdivided\", \"subjected\", \"subsidies\", \"substance\", \"sulphur\", \"supply\", \"support\", \"support\", \"support\", \"surface\", \"surgical\", \"surplus\", \"surplus\", \"survey\", \"survey\", \"surveys\", \"sustainable\", \"sweden\", \"switzerland\", \"systems\", \"systems\", \"tables\", \"takes\", \"taking\", \"taking\", \"target\", \"taxes\", \"technical\", \"technical\", \"temporary\", \"temporary\", \"tents\", \"terms\", \"terms\", \"territory\", \"there\", \"these\", \"these\", \"these\", \"three\", \"three\", \"times\", \"tonnes\", \"total\", \"total\", \"total\", \"total\", \"total\", \"tourism\", \"track\", \"trade\", \"trade\", \"traffic\", \"transactions\", \"transactions\", \"transfers\", \"transformation\", \"transport\", \"treatment\", \"treaty\", \"trees\", \"types\", \"types\", \"types\", \"under\", \"union\", \"united\", \"units\", \"useful\", \"users\", \"usually\", \"usually\", \"value\", \"value\", \"values\", \"variable\", \"variable\", \"variables\", \"various\", \"vegetation\", \"vehicle\", \"vehicles\", \"vertical\", \"violence\", \"volume\", \"wages\", \"walls\", \"warming\", \"waste\", \"wastewater\", \"water\", \"water\", \"weight\", \"worked\", \"workers\", \"world\", \"world\", \"years\", \"years\", \"years\", \"young\"]}, \"R\": 30, \"lambda.step\": 0.01, \"plot.opts\": {\"xlab\": \"PC1\", \"ylab\": \"PC2\"}, \"topic.order\": [4, 12, 14, 8, 11, 20, 15, 19, 17, 3, 6, 9, 10, 18, 1, 7, 2, 16, 13, 5]};\n",
       "\n",
       "function LDAvis_load_lib(url, callback){\n",
       "  var s = document.createElement('script');\n",
       "  s.src = url;\n",
       "  s.async = true;\n",
       "  s.onreadystatechange = s.onload = callback;\n",
       "  s.onerror = function(){console.warn(\"failed to load library \" + url);};\n",
       "  document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
       "}\n",
       "\n",
       "if(typeof(LDAvis) !== \"undefined\"){\n",
       "   // already loaded: just create the visualization\n",
       "   !function(LDAvis){\n",
       "       new LDAvis(\"#\" + \"ldavis_el1005220031555744489202571526\", ldavis_el1005220031555744489202571526_data);\n",
       "   }(LDAvis);\n",
       "}else if(typeof define === \"function\" && define.amd){\n",
       "   // require.js is available: use it to load d3/LDAvis\n",
       "   require.config({paths: {d3: \"https://d3js.org/d3.v5\"}});\n",
       "   require([\"d3\"], function(d3){\n",
       "      window.d3 = d3;\n",
       "      LDAvis_load_lib(\"https://cdn.jsdelivr.net/gh/bmabey/pyLDAvis@3.3.1/pyLDAvis/js/ldavis.v3.0.0.js\", function(){\n",
       "        new LDAvis(\"#\" + \"ldavis_el1005220031555744489202571526\", ldavis_el1005220031555744489202571526_data);\n",
       "      });\n",
       "    });\n",
       "}else{\n",
       "    // require.js not available: dynamically load d3 & LDAvis\n",
       "    LDAvis_load_lib(\"https://d3js.org/d3.v5.js\", function(){\n",
       "         LDAvis_load_lib(\"https://cdn.jsdelivr.net/gh/bmabey/pyLDAvis@3.3.1/pyLDAvis/js/ldavis.v3.0.0.js\", function(){\n",
       "                 new LDAvis(\"#\" + \"ldavis_el1005220031555744489202571526\", ldavis_el1005220031555744489202571526_data);\n",
       "            })\n",
       "         });\n",
       "}\n",
       "</script>"
      ],
      "text/plain": [
       "PreparedData(topic_coordinates=              x         y  topics  cluster       Freq\n",
       "topic                                                \n",
       "3     -0.226390 -0.080025       1        1  12.857074\n",
       "11    -0.187022 -0.248189       2        1  10.646288\n",
       "13    -0.145966  0.050590       3        1   9.788812\n",
       "7     -0.087717  0.133128       4        1   9.048594\n",
       "10    -0.142750  0.232981       5        1   8.212530\n",
       "19    -0.155625  0.062444       6        1   7.732242\n",
       "14    -0.129263 -0.102202       7        1   6.017547\n",
       "18    -0.004603 -0.072207       8        1   5.000871\n",
       "16     0.028831  0.078366       9        1   4.353044\n",
       "2      0.049498  0.061877      10        1   3.957674\n",
       "5      0.114300 -0.018436      11        1   3.728963\n",
       "8      0.031210 -0.131178      12        1   3.421501\n",
       "9      0.055606 -0.023241      13        1   3.343607\n",
       "17     0.058718  0.005677      14        1   3.042505\n",
       "0     -0.010615  0.136611      15        1   2.885836\n",
       "6      0.137894 -0.027806      16        1   1.761423\n",
       "1      0.146802 -0.019943      17        1   1.358123\n",
       "15     0.154138 -0.023173      18        1   1.134404\n",
       "12     0.153349 -0.003424      19        1   0.877754\n",
       "4      0.159605 -0.011851      20        1   0.831208, topic_info=           Term        Freq       Total Category  logprob  loglift\n",
       "14     european  625.000000  625.000000  Default  30.0000  30.0000\n",
       "236   financial  487.000000  487.000000  Default  29.0000  29.0000\n",
       "279       goods  469.000000  469.000000  Default  28.0000  28.0000\n",
       "497      number  341.000000  341.000000  Default  27.0000  27.0000\n",
       "374    services  457.000000  457.000000  Default  26.0000  26.0000\n",
       "...         ...         ...         ...      ...      ...      ...\n",
       "2120     permit    0.039489    1.282336  Topic20  -9.5551   1.3096\n",
       "3937    credits    0.039483    1.282247  Topic20  -9.5553   1.3095\n",
       "282    interest    0.812368   27.176678  Topic20  -6.5312   1.2799\n",
       "2383      index    0.039490    1.282827  Topic20  -9.5551   1.3093\n",
       "247      months    0.040026   51.237189  Topic20  -9.5416  -2.3647\n",
       "\n",
       "[817 rows x 6 columns], token_table=      Topic      Freq         Term\n",
       "term                              \n",
       "89        1  0.207083  abbreviated\n",
       "89        4  0.515753  abbreviated\n",
       "89        5  0.121124  abbreviated\n",
       "89       10  0.148474  abbreviated\n",
       "1645     18  0.772945      ability\n",
       "...     ...       ...          ...\n",
       "153       7  0.104285        world\n",
       "668       2  0.317218        years\n",
       "668       3  0.241361        years\n",
       "668      12  0.441346        years\n",
       "779      17  0.934297        young\n",
       "\n",
       "[1001 rows x 3 columns], R=30, lambda_step=0.01, plot_opts={'xlab': 'PC1', 'ylab': 'PC2'}, topic_order=[4, 12, 14, 8, 11, 20, 15, 19, 17, 3, 6, 9, 10, 18, 1, 7, 2, 16, 13, 5])"
      ]
     },
     "execution_count": 170,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Visualize the topics\n",
    "\n",
    "pyLDAvis.enable_notebook()\n",
    "vis = pyLDAvis.gensim_models.prepare(lda_model, corpus, id2word)\n",
    "vis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 171,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Dominant_Topic</th>\n",
       "      <th>Topic_Keywords</th>\n",
       "      <th>Num_Documents</th>\n",
       "      <th>Perc_Documents</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>17</td>\n",
       "      <td>employed, place, stock, private, similar, acco...</td>\n",
       "      <td>31.0</td>\n",
       "      <td>0.0241</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>17</td>\n",
       "      <td>employed, place, stock, private, similar, acco...</td>\n",
       "      <td>12.0</td>\n",
       "      <td>0.0093</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>17</td>\n",
       "      <td>employed, place, stock, private, similar, acco...</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0.0273</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>17</td>\n",
       "      <td>employed, place, stock, private, similar, acco...</td>\n",
       "      <td>246.0</td>\n",
       "      <td>0.1916</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7</td>\n",
       "      <td>european, member, countries, states, union, ec...</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0023</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1279</th>\n",
       "      <td>3</td>\n",
       "      <td>statistics, level, statistical, international,...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1280</th>\n",
       "      <td>3</td>\n",
       "      <td>statistics, level, statistical, international,...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1281</th>\n",
       "      <td>10</td>\n",
       "      <td>activities, water, households, energy, prices,...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1282</th>\n",
       "      <td>0</td>\n",
       "      <td>activities, activity, market, product, product...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1283</th>\n",
       "      <td>3</td>\n",
       "      <td>statistics, level, statistical, international,...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1284 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      Dominant_Topic                                     Topic_Keywords  \\\n",
       "0                 17  employed, place, stock, private, similar, acco...   \n",
       "1                 17  employed, place, stock, private, similar, acco...   \n",
       "2                 17  employed, place, stock, private, similar, acco...   \n",
       "3                 17  employed, place, stock, private, similar, acco...   \n",
       "4                  7  european, member, countries, states, union, ec...   \n",
       "...              ...                                                ...   \n",
       "1279               3  statistics, level, statistical, international,...   \n",
       "1280               3  statistics, level, statistical, international,...   \n",
       "1281              10  activities, water, households, energy, prices,...   \n",
       "1282               0  activities, activity, market, product, product...   \n",
       "1283               3  statistics, level, statistical, international,...   \n",
       "\n",
       "      Num_Documents  Perc_Documents  \n",
       "0              31.0          0.0241  \n",
       "1              12.0          0.0093  \n",
       "2              35.0          0.0273  \n",
       "3             246.0          0.1916  \n",
       "4               3.0          0.0023  \n",
       "...             ...             ...  \n",
       "1279            NaN             NaN  \n",
       "1280            NaN             NaN  \n",
       "1281            NaN             NaN  \n",
       "1282            NaN             NaN  \n",
       "1283            NaN             NaN  \n",
       "\n",
       "[1284 rows x 4 columns]"
      ]
     },
     "execution_count": 171,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Topic distribution across documents\n",
    "\n",
    "# Number of Documents for Each Topic\n",
    "topic_counts = df_topic_sents_keywords['Dominant_Topic'].value_counts()\n",
    "\n",
    "# Percentage of Documents for Each Topic\n",
    "topic_contribution = round(topic_counts/topic_counts.sum(), 4)\n",
    "\n",
    "# Topic Number and Keywords\n",
    "topic_num_keywords = df_topic_sents_keywords[['Dominant_Topic', 'Topic_Keywords']]\n",
    "\n",
    "# Concatenate Column wise\n",
    "df_dominant_topics = pd.concat([topic_num_keywords, topic_counts, topic_contribution], axis=1)\n",
    "\n",
    "# Change Column names\n",
    "df_dominant_topics.columns = ['Dominant_Topic', 'Topic_Keywords', 'Num_Documents', 'Perc_Documents']\n",
    "\n",
    "# Show\n",
    "df_dominant_topics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 9. The model's coherence score.\n",
    "***\n",
    "\n",
    "This is based on the work in Röder, M., Both, A., & Hinneburg, A. (2015, February). Exploring the space of topic coherence measures. In *Proceedings of the eighth ACM international conference on Web search and data mining* (pp. 399-408), available [here](http://svn.aksw.org/papers/2015/WSDM_Topic_Evaluation/public.pdf)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 172,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Coherence Score:  0.4240461305157298\n"
     ]
    }
   ],
   "source": [
    "from gensim.models import CoherenceModel\n",
    "\n",
    "coherence_model_lda = CoherenceModel(model=lda_model, texts=texts, dictionary=id2word, coherence='c_v')\n",
    "coherence_lda = coherence_model_lda.get_coherence()\n",
    "print('\\nCoherence Score: ', coherence_lda)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1284"
      ]
     },
     "execution_count": 173,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "docs_topics = lda_model.get_document_topics(corpus,minimum_probability=0.3)\n",
    "res  = [d for d in docs_topics]\n",
    "res\n",
    "len(res)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}