[tutorial] How-to-extract-main-content-from-web-pages-using-Machine-Learning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# > \"Knowledge is of no value unless you put it into practice\"\n",
    "\n",
    "> -- Anton Chekhov\n",
    "\n",
    "## Motivation\n",
    "\n",
    "I have read many articles on machine learning, heard everyone else talking about it, but still don't know where to start. Until one day, when I was tunning the accuracy of my Hacker News crawler, it suddenly occurred to me - what I'm doing is to figure out the best parameters of a formula that can tell the possibility of a DOM being the main content, isn't this process called **training** in the terminology of machine learning? isn't it a good idea to feed all those features into a classifier model, and let it do the tunning to find out the best parameters (instead of me doing it manually)?\n",
    "\n",
    "This is the “aha!” moment for me on machine learning, also the first time to solve a real-world problem using machine learning. So here comes the note I take when falling down the rabbit hole, I'll try to explain things from a newbie's view (actually my view), hoping others may get inspired.\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem to Solve\n",
    "\n",
    "It's super easy to get the main content of a HTML page, if you have done some crawler related works before. \n",
    "\n",
    "Let's take a page of `The Times` as an example: [![genesis-block.png](https://user-images.githubusercontent.com/2657334/45598094-8e9da180-ba08-11e8-9b2e-2ee9024e8b5d.png)](https://www.thetimes.co.uk/article/chancellor-alistair-darling-on-brink-of-second-bailout-for-banks-n9l382mn62h)\n",
    "You just need to look at where the DOM is in the page using your favorite DOM inspector, write down its `XPATH` or `CSS selector`, then apply the rule to [Beautiful Soup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/) or whatever your library of choices, and you are done - using this rule you can scrap the entire `The Times` and get their main contents, pretty straightforward right?\n",
    "\n",
    "However, what if you have hundreds of pages to extract content from? writing down their `XPATHes` and using hundreds of `if...else` to apply those rules? Nah, you will become crazy maintaining all those rules. So **extract the main content of \"any\" page is hard**, how does [Hacker News Summary](http://hackernews.betacat.io/) solve this?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A Brilliant Way\n",
    "\n",
    "This method is based on an assumption that the DOM element with the most text is more likely to be the main content. Using this rule only, however, will lead to the `html` DOM been recognized as the main content, because it contains everything, so we also need to take the DOM depth into consideration. Besides, those who have more intensive `<a>` links, look like ADs, comments, sidebars, etc. should be reduced of weight. Combining all those rules, we can score every DOM in a page, and the one with the highest score is the winner to be the main content. Following are the factors currently used to score a DOM:\n",
    "\n",
    "* The number of words it contains (including descendants)\n",
    "* The depth it resides in the whole page\n",
    "* Whether it contains a `h` element that has a high similarity to the title of this page, most blogs will put the title of its article into the `<title>` element for SEO purpose\n",
    "* Whether its `class` or `id` contains string like `article`, `content`, or `ad-`, `comment`, etc.\n",
    "* Whether it contains a `<img>`, and the area ratio of that image\n",
    "* The intensive of its `<a>` links, if all children of an element are the `<a>` links, it's more likely to be a menu or ad, and less likely to be the main content.\n",
    "\n",
    "After quantifying those factors, I use a linear formula to calculate the score, so the problem becomes - how can I pick the right weight for those factors. The most naive approach is to set them from experience, then test against some pages, and adjust them accordingly. After many trials and errors, I finally found a set of weights that works for most of the pages posted on [Hacker News](https://news.ycombinator.com/), including news websites, blogs, Github readmes, etc.\n",
    "\n",
    "These weights work so well that my crawler keeps sending me summaries of [Hacker News](https://news.ycombinator.com/) posts with high accuracy, but if I want to improve it, or add more rules, I need be careful and add more test cases to stabilize my changes, as it's becoming harder to maintain the balance between the old working weights and a new one. As you may have guessed, this is where **Machine Learning** kicks in!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A More Sexy Way\n",
    "\n",
    "Now that I have known the factors contributing to the possibility of a DOM being the main content, why not let the machine learning find the best weights to those factors? as it has a wide variety of models, optimized algorithms to figure out the best weights.\n",
    "\n",
    "Time to get our hands dirty with some code\n",
    "![exciting](https://i.giphy.com/media/11sBLVxNs7v6WA/giphy.webp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn import preprocessing  \n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.naive_bayes import GaussianNB, MultinomialNB\n",
    "from sklearn.svm import SVC"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Here are the extracted features of a DOM:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>target</th>\n",
       "      <th>depth</th>\n",
       "      <th>text_ratio</th>\n",
       "      <th>alink_text_ratio</th>\n",
       "      <th>has_img</th>\n",
       "      <th>contain_title</th>\n",
       "      <th>attr</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>False</td>\n",
       "      <td>0.047619</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.267532</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>html,story</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>False</td>\n",
       "      <td>0.095238</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.267532</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>body</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>False</td>\n",
       "      <td>0.142857</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.267532</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>div,app</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>False</td>\n",
       "      <td>0.190476</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.267532</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>div,css,v89234,e1dd99ql0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>False</td>\n",
       "      <td>0.238095</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.267532</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>div</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   target     depth  text_ratio  alink_text_ratio  has_img  contain_title  \\\n",
       "0   False  0.047619         1.0          0.267532    False           True   \n",
       "1   False  0.095238         1.0          0.267532    False           True   \n",
       "2   False  0.142857         1.0          0.267532    False           True   \n",
       "3   False  0.190476         1.0          0.267532    False           True   \n",
       "4   False  0.238095         1.0          0.267532    False           True   \n",
       "\n",
       "                       attr  \n",
       "0                html,story  \n",
       "1                      body  \n",
       "2                   div,app  \n",
       "3  div,css,v89234,e1dd99ql0  \n",
       "4                       div  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataframe = pd.read_csv('/tmp/features.csv')\n",
    "dataframe.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The `target` column means whether this DOM is the main content or not, tagged by me manually.\n",
    "* The `depth`, `text_ratio` and `alink_text_ratio` columns are three key features I mentioned early, note that they have been normalized so they are comparable across all kinds of web pages. Let me take the `text_ratio` as an example, if DOM A in a page has 500 words, and DOM B in another page has only 100 words, you can not get the conclusion that DOM A has a higher possibility than DOM B to be the main content, because they are from different pages!\n",
    "* The `attr` column is a set of strings, namely `tag name`, `class` and `id`, as some of them like `article`, `content` are more likely to have a positive contributions, and others like `comment`, `ad` may have negative contributions, which I'll leave to the machine learning algorithm figure out.\n",
    "\n",
    "In order to convert those `attr` strings into numbers, I'm using the [CountVectorizer](http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html) method, which tokenizes and counts the word occurrences a text document, so most frequent attributes will have a higher value(count).\n",
    "\n",
    "#### Now the features become:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style>\n",
       "    .dataframe thead tr:only-child th {\n",
       "        text-align: right;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>depth</th>\n",
       "      <th>text_ratio</th>\n",
       "      <th>alink_text_ratio</th>\n",
       "      <th>has_img</th>\n",
       "      <th>contain_title</th>\n",
       "      <th>blogsearch2_form</th>\n",
       "      <th>postarticle</th>\n",
       "      <th>9b33</th>\n",
       "      <th>clicktalediv</th>\n",
       "      <th>unblock</th>\n",
       "      <th>...</th>\n",
       "      <th>star</th>\n",
       "      <th>70rfte</th>\n",
       "      <th>1k2cjfc</th>\n",
       "      <th>breadcrumb</th>\n",
       "      <th>headerarea</th>\n",
       "      <th>e304</th>\n",
       "      <th>blogs</th>\n",
       "      <th>1e0ws</th>\n",
       "      <th>screencasts_section</th>\n",
       "      <th>directory</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.047619</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.267532</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.095238</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.267532</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.142857</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.267532</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.190476</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.267532</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.238095</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.267532</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 1437 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      depth  text_ratio  alink_text_ratio  has_img  contain_title  \\\n",
       "0  0.047619         1.0          0.267532    False           True   \n",
       "1  0.095238         1.0          0.267532    False           True   \n",
       "2  0.142857         1.0          0.267532    False           True   \n",
       "3  0.190476         1.0          0.267532    False           True   \n",
       "4  0.238095         1.0          0.267532    False           True   \n",
       "\n",
       "   blogsearch2_form  postarticle  9b33  clicktalediv  unblock    ...      \\\n",
       "0                 0            0     0             0        0    ...       \n",
       "1                 0            0     0             0        0    ...       \n",
       "2                 0            0     0             0        0    ...       \n",
       "3                 0            0     0             0        0    ...       \n",
       "4                 0            0     0             0        0    ...       \n",
       "\n",
       "   star  70rfte  1k2cjfc  breadcrumb  headerarea  e304  blogs  1e0ws  \\\n",
       "0     0       0        0           0           0     0      0      0   \n",
       "1     0       0        0           0           0     0      0      0   \n",
       "2     0       0        0           0           0     0      0      0   \n",
       "3     0       0        0           0           0     0      0      0   \n",
       "4     0       0        0           0           0     0      0      0   \n",
       "\n",
       "   screencasts_section  directory  \n",
       "0                    0          0  \n",
       "1                    0          0  \n",
       "2                    0          0  \n",
       "3                    0          0  \n",
       "4                    0          0  \n",
       "\n",
       "[5 rows x 1437 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = dataframe.target\n",
    "X = dataframe.drop(['target'], axis=1)\n",
    "\n",
    "corpus = X['attr']\n",
    "vc = CountVectorizer()  \n",
    "vc.fit(corpus)\n",
    "\n",
    "numeric_features = pd.concat([X.drop(['attr'], axis=1), pd.DataFrame(vc.transform(corpus).toarray(), columns=vc.vocabulary_)], axis=1)\n",
    "\n",
    "numeric_features.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "#### I also draw a chart trying to visualize the relationship between `target` and `text_ratio`, `depth`, `alink_text_ratio`, it seems the `text_ratio`(in blue) has the most significant impact on `target`(in red). Whatever, I do not need to worry about their importances any more!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeQAAAD8CAYAAACmXNe7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xt4VNW9N/Dvb3IhDJBgICchgcwgEiAhUkyEWi+oQblU\n6WPRekmL7VONl9oW9OjR5i0Cdo7ayim16MH0aGvraNWKR0AUKSL6qvUliDGGBAOahEsi4ZYQhpDL\nrPePmcBkMslcM7P3zPfzPHlgr7323r9ZezK/7L3XrCVKKRAREVFkGSIdABERETEhExERaQITMhER\nkQYwIRMREWkAEzIREZEGMCETERFpABMyERGRBjAhExERaQATMhERkQbER+rAo0ePVmazOVKHJyLS\npR07dhxWSqVFOg4KvYglZLPZjPLy8kgdnohIl0SkPtIx0ODgLWsiIiINYEImIiLSACZkIiIiDWBC\nJiIi0gAmZCIiIg1gQiYiItIAJmQiIiINYEImIiLSAK8Dg4jIcwCuAXBIKTXVw3oB8AcA8wHYAPxY\nKfVpqAN1Z7UCpaVAQwOQnQ3Mnw9s3Hh22WIBiovP1q862o5tB21o7bQjOcGACckJ2Nva2e/yrEwj\n8lKTfN7eW329rffa4O4NTFHLWmlF6ZZSNLQ0IDslG5YiC4rzee6JQk2UUgNXELkMQBuAv/aTkOcD\n+DkcCXkmgD8opWZ6O3BhYaEKdKQuqxUoKQFstv7rJCQAycnA0aPAlcXtuGpxG5Qf9wPiBZiXPRx5\nqUmoOtqOtxra0DVAU3mrr6f1fXhqcKMRKCtjUo5y1korStaXwNZ59twbE4wou7aMSTlCRGSHUqow\n0nFQ6HlNUUqp9wEcHaDK9+BI1kop9S8AI0VkTKgC9KS0dOBkDACdncCRI4BSQMFNNr+SMQB0KWDb\nQcdBth20DZiMfamvp/V9eGpwm81RTlGtdEtpr2QMALZOG0q38NwThVooniFnAdjnsrzfWdaHiJSI\nSLmIlDc3Nwd8wIYG/+qPzLAHdJzWTnuvf4Otr5f1gOOi2GwGDAbAXP8erLi57wb+ngjSnYYWz+e4\nv3IiClxYO3UppcqUUoVKqcK0tMAnK8nO9q/+8abAXmZygqHXv8HW18v6njvU9fWOOwz1MKMEf+qb\nlP09EaQ72Smez3F/5UQUuFAk5AMAxrksj3WWDRqLxfEI01ebVhvRccq/Y8QLMCvTcZBZmUbES3D1\n9bTe4x1qDEMp/vNsgdHoOBEU1SxFFhgTev+yGROMsBTx3BOFWigS8joAi8Th2wBalFKNIdhvv4qL\nHf2JTCZAxPHvXXedXR41CkhMPFu/4u0kbHh8OOI6zl4hTh81pNcVo/uyawenvNQkzMseHlR9Pa3v\n7050PUwwoBvmuH2w3rqJHbpiQHF+McquLYMpxQSBwJRiYocuokHiSy/rlwBcDmA0gG8APAwgAQCU\nUmucX3taDWAuHF97+olSymv36WB6WfuC39IJnNnsuF09EHayJooM9rKOXl4T8mAZ7IRMgfPla2WA\n445EXV1YQiIiJybk6MWRuqgP90cC/WEnayKi0GFCJo+Kix1Xv3a7IzF7EspO1r2+ZmV2LBMRxRIm\nZPLKU6/2UHay7vM1q3rHMpMyEcUSJmTyylOv9lB26OJAYERE7NRFGmAwOK6M3Yk4bpkT0Vns1BW9\neIVMEdffs2gOBEZEsYQJmSJusJ9RExHpARMyRdxgP6MmItKD+EgHQAQ4ki8TMBHFMl4hExERaQAT\nMhERkQYwIRMREWkAEzIREZEGMCFTWFgrrTCvMsOw3ADzKjPufvPuXsvWSv/GyXTfn7ftB6rvbV/+\nHsvbsd1fu7flYF5bKOIL9bnxd32o24tIqzhSFw06a6UVJetLYOvsfz5HY4LR54nvPe1voO0Hqg9g\nwH35eyxfju2vQF9bqOIL5bkJZL2//IlXjzhSV/RiQqZBZ15lRn1Lvdd6phQT6hbXBby//rYfqD6A\nAffl77F8Pba/AnltoYwvVOcm0PX+8jVePWJCjl78HjINuoYW3yZODrZeqMpd1wWybSD1At1PuOIL\n1bkJdL2/QrUfonDiM2QadNkpvg1KHWy9QMq9bePvPgOtF+h+whVfqM5NoOv9Far9EIUTEzINOkuR\nBcYE44B1jAlGWIp8G7za0/4G2n6g+t725e+xfDm2vwJ9baGKL5TnJpD1/vInXiItiVu2bFlEDlxW\nVraspKQkIsem8Do//XyYR5qx4+AOtJ5uhSnFhFvyb0HzyeYzy6vmrvK5E46n/Q20/UD1ve3L32MF\n8tq9LQf62kIVXyjPTSDrQ9le0WD58uWNy5YtK4t0HBR67NRFRKQj7NQVvXjLmoiISAOYkImIiDSA\nCZmIiEgDmJCJiIg0gAmZiIhIA5iQiYiINIAJmYiISAOYkImIiDTAp4QsInNFZLeI7BGRBz2szxaR\nrSKyU0Q+F5H5oQ+ViIgoenlNyCISB+ApAPMA5AK4WURy3ar9HwCvKKWmA7gJwNOhDpSIiCia+XKF\nPAPAHqXUV0qpDgB/B/A9tzoKQLLz/ykADoYuRCIioujny3zIWQD2uSzvBzDTrc4yAO+IyM8BDAMw\nOyTRERERxYhQdeq6GcBflFJjAcwH8DcR6bNvESkRkXIRKW9ubg7RoYmIiPTPl4R8AMA4l+WxzjJX\nPwXwCgAopT4GkARgtPuOlFJlSqlCpVRhWlpaYBGHibXSCvMqMwzLDTCvMsNaaQ2qvt7Xe+O+/d1v\n3h3W/QVb33W9v23lfqxgX/tgt0Wkz02o33tE0cLr9IsiEg/gSwBFcCTi7QBuUUpVudR5C8DLSqm/\niMgUAFsAZKkBdq7l6RetlVaUrC+BrdN2psyYYETZtWUe51n1Vl/v6wNpL3eDub9Q1O9ZD8DvtvLG\nn9fubjDaIpj4QtnWoXjvxSJOvxi9fJoP2fk1plUA4gA8p5SyiMgKAOVKqXXOXtd/AjAcjg5eDyil\n3hlon1pOyOZVZtS31PcpN6WYULe4zu/6el/vTX/bh2t/oapvSjEBQEBt5eux/TVYbRFofKFs61C8\n92IRE3L08qVTF5RSGwFsdCtb6vL/XQAuDm1okdPQ0hDScr2v9ybS9UJVf6D9+Hssf/Ydiu3CFV+o\n2zrY9x5RNOFIXR5kp2SHtFzv672JdL1Q1c9OyQ64rXw99mBtF674QtnWvqwniiVMyB5YiiwwJhh7\nlRkTjLAUWQKqr/f13nja3t1g7i8U9XvWB9JW3vjz2t0NRlsEE18o29qX9USxJG7ZsmUROXBZWdmy\nkpKSiBzbm/PTz4d5pBk7Du5A6+lWmFJMWDV3Vb+dTLzV1/v6QNrrlvxb0HyyOSz7C0X9nvWBtJX7\nsYJ57eFoi0ifm1C+92LR8uXLG5ctW1YW6Tgo9Hzq1DUYtNypi4hIq9ipK3rxljUREZEGMCETERFp\nABMyERGRBjAhExERaQATMhERkQYwIRMREWkAEzIREZEGMCETERFpABMyERGRBjAhExERaQATMhER\nkQYwIRMREWkAEzIREZEGMCE7WSutMK8yw7DcAPMqM6yV1qDq6329N8FuP9j7I+rB9xbpBadfhOMX\ntmR9CWydtjNlxgQjyq4t8zgvq7f6el8f6vYK9/6IekTje4vTL0YvJmQA5lVm1LfU9yk3pZhQt7jO\n7/p6X+9NsNsP9v6IekTje4sJOXrxljWAhpaGkJbrfb03wW4/2Psj6sH3FukJEzKA7JTskJbrfb03\nwW4/2Psj6sH3FukJEzIAS5EFxgRjrzJjghGWIktA9fW+3ptgtx/s/RH14HuLdEUpFZGfgoICpSUv\nfP6CMv3epGSZKNPvTeqFz18Iqr7e13sT7PaDvT+iHtH23gJQriL0uc2fwf1hpy4iIh1hp67oxVvW\nREREGsCETEREpAFMyERERBrAhExERKQBPiVkEZkrIrtFZI+IPNhPnR+IyC4RqRKRF0MbJhERUXSL\n91ZBROIAPAXgKgD7AWwXkXVKqV0udSYCeAjAxUqpYyLyb4MVMBERUTTy5Qp5BoA9SqmvlFIdAP4O\n4HtudW4H8JRS6hgAKKUOhTZMIiKi6OZLQs4CsM9leb+zzFUOgBwR+VBE/iUic0MVIBERUSzwesva\nj/1MBHA5gLEA3heRfKXUcddKIlICoAQAsrM5liwREVEPX66QDwAY57I81lnmaj+AdUqpTqXU1wC+\nhCNB96KUKlNKFSqlCtPS0gKNmYiIKOr4kpC3A5goIuNFJBHATQDWudX5XziujiEio+G4hf1VCOMk\nIiKKal4TslKqC8A9ADYBqAbwilKqSkRWiMgCZ7VNAI6IyC4AWwHcr5Q6MlhBExERRRtOLkFEpCOc\nXCJ6xexIXdZKK8yrzDAsN8C8ygxrpTWk9aNNrL9+0g++V0mvQtXLWleslVaUrC+BrdMGAKhvqUfJ\n+hIAQHF+cdD1o02sv37SD75XSc9i8pa1eZUZ9S31fcpNKSbULa4Lun60ifXXT/oRC+9V3rKOXjF5\ny7qhpWFQy6NNrL9+0g++V0nPYjIhZ6d4HpQkVOXRJtZfP+kH36ukZzGZkC1FFhgTjL3KjAlGWIos\nIakfbWL99ZN+8L1KehaTCbk4vxhl15bBlGKCQGBKMaHs2rJ+O334Wz/axPrrJ/3ge5X0LCY7dRER\n6RU7dUWvmLxCJiIi0homZCIiIg1gQiYiItIAJmQiIiINYEImIiLSACZkIiIiDWBCJiIi0gAmZCIi\nIg1gQiYiItIAJmQiIiINYEImIiLSACZkIiIiDWBCJiIi0gAmZCIiIg2ImYRsrbTCvMoMw3IDzKvM\nsFZaIx2SprG9iIjCKz7SAYSDtdKKkvUlsHXaAAD1LfUoWV8CAJy43AO2FxFR+MXEFXLpltIzyaWH\nrdOG0i2lEYpI29heREThFxMJuaGlwa/yWMf2IiIKv5hIyNkp2X6Vxzq2FxFR+MVEQrYUWWBMMPYq\nMyYYYSmyRCgibWN7ERGFX0wk5OL8YpRdWwZTigkCgSnFhLJry9hBqR9sLyKi8BOlVEQOXFhYqMrL\nyyNybCIivRKRHUqpwkjHQaHn0xWyiMwVkd0iskdEHhyg3kIRUSLCNwsREZEfvCZkEYkD8BSAeQBy\nAdwsIrke6o0A8EsAn4Q6SCIiomjnyxXyDAB7lFJfKaU6APwdwPc81HsEwOMA2kMYHxERUUzwJSFn\nAdjnsrzfWXaGiFwAYJxS6s2BdiQiJSJSLiLlzc3NfgdLREQUrYLuZS0iBgD/BeA+b3WVUmVKqUKl\nVGFaWlqwhyYiIooaviTkAwDGuSyPdZb1GAFgKoD3RKQOwLcBrGPHLiIiIt/5kpC3A5goIuNFJBHA\nTQDW9axUSrUopUYrpcxKKTOAfwFYoJTid5qIiIh85DUhK6W6ANwDYBOAagCvKKWqRGSFiCwY7ACJ\niIhigU/TLyqlNgLY6Fa2tJ+6lwcfFhERUWyJiaEziYiItI4JmYiISAOYkImIiDSACZmIiEgDfOrU\npUfWSitKt5SioaUB2SnZsBRZOH0gEfX5bJg/cT421m4M2zI/i6g/UTn9orXSipL1JbB12s6UGROM\nnNOXKMZ5+mwIt2A/izj9YvSKylvWpVtK+/zC2TptKN1SGqGIiEgLPH02hBs/i6g/UZmQG1oa/Con\notiglc8ArcRB2hKVCTk7JduvciKKDVr5DNBKHKQtUZmQLUUWGBOMvcqMCUZYiiwRioiItMDTZ0O4\n8bOI+hOVCbk4vxhl15bBlGKCQGBKMbFDFxF5/Gy4q/CusC7zs4j6E5W9rImIohV7WUevqLxCJiIi\n0hsmZCIiIg2I2pG6iKJV1dF2bDtoQ2unHckJBszKNCIvNSnSYRFRkJiQiXSk6mg73mpoQ5ez60dr\npx1vNbQBAJMykc7xljWRjmw7aDuTjHt0KUc5EekbEzKRjrR22v0qJyL9YEIm0pHkBM+/sv2VE5F+\n8BkykY7MyjT2eoYMAPHiKKfYkJaWFl9RUbEOwFTwokpv7AC+6Orquq2goOCQ+0omZCId6em4xV7W\nsctisZgzMjKGpaWlHTMYDJEZ2YkCYrfbpbm5Obepqel/ACxwX8+ETKQzealJTMAxzGw2D01LS2tl\nMtYfg8Gg0tLSWpqamqZ6XB/ugIiIKHAiAiZj/XKeO4+5lwmZiIh8dvjw4bjHHnssbbCPs2HDhhGb\nN28eNtjH0RImZCIi8tmRI0finn322X/ztb7dbkd3d7ffx3n33XdHfPDBB8P93lDHoiYhWyutMK8y\nw7DcAPMqM6yV1kiHREQUeWvWpCIzMx8GQwEyM/OxZk1qMLu77777xu7bt2/I5MmTc3/605+Ou+ii\ni3Jyc3On5OTk5L7wwgsjAWD37t2JZrN56nXXXWfOycnJ27t3b+Lvf//70WazeWp+fv6Um266ybRo\n0aJsADh48GD8nDlzJkydOnXK1KlTp7zzzjvDdu/enfjXv/41bc2aNemTJ0/Offvtt2MiMUdFpy5r\npRUl60tg63SMVlTfUo+S9SUAwHlHiSh2rVmTiiVLTGhvd1x8NTYmYskSEwDgzjuPBrLLlStX7r/m\nmmuG1tTU7Ors7MSJEycMqamp9sbGxviZM2dOvuWWW44DQENDw5Bnn33266Kiorq6urqEJ554Ysyn\nn366a+TIkfbvfOc7OXl5eacA4I477hh37733fjNnzpy22traxDlz5kz86quvqhYtWtQ8fPjw7hUr\nVnwTmsbQvqhIyKVbSs8k4x62ThtKt5QyIRNR7FqxIutMMu7R3m7AihVZgSZkV3a7XRYvXjz2X//6\n13CDwYBDhw4l7t+/Px4AxowZ01FUVHQSAD744INhM2fOPJGent4NANddd92xL7/8MgkAPvzww+Ta\n2tqhPftsa2uLa2lpiZq7t/6IioTc0NLgVzkRUUxoakr0q9xPzzzzTOqRI0fiKysrq4cMGaKysrLy\nT506ZQAAo9Ho03iuSil8+umn1UajMeZ7jvv0V4iIzBWR3SKyR0Qe9LD+XhHZJSKfi8gWETGFPtT+\nZadk+1VORBQTMjI6/Cr3QUpKSvfJkycNANDS0hI3evToziFDhqj169ePOHjwoMdEf8kll5z85JNP\nRjQ3N8d1dnbijTfeOMdlXeujjz56ppPYRx99NBQARowY0X3ixIm4QOPUI68JWUTiADwFYB6AXAA3\ni0iuW7WdAAqVUucD+AeA34Y60IFYiiwwJvQeOtCYYISlyBLOMIiItGXp0gNISup9pZqUZMfSpQcC\n3WVGRkZ3QUFB28SJE/MqKiqMFRUVw3JycnKff/75UePHj2/3tM348eM7lyxZ0lhYWDiloKBg8rhx\n406npKR0A0BZWdm+Tz/9dFhOTk7uhAkT8lavXp0GAAsXLjz+5ptvjoylTl2i1MB3CUTkIgDLlFJz\nnMsPAYBS6tF+6k8HsFopdfFA+y0sLFTl5eUBBe2JtdKK0i2laGhpQHZKNixFFj4/JqKo889//rNj\n9uzZlT5vsGZNKlasyEJTUyIyMjqwdOmBUDw/9ldLS4shJSXF3tnZiTlz5pz34x//+PCiRYuOhzsO\nLaioqBg9bdo0s3u5L8+QswDsc1neD2DmAPV/CuAtv6ILgeL8YiZgIiJ3d955NBIJ2N3999+f+f77\n7yefPn1aZs2a1frDH/4wJpPxQELaqUtEfgigEMCsftaXACgBgOxsPt8lIooVZWVl+yMdg9b50qnr\nAIBxLstjnWW9iMhsAKUAFiilTnvakVKqTClVqJQqTEsb9JHXiIiIdMOXhLwdwEQRGS8iiQBuArDO\ntYLzufEzcCTjPnM8EhER0cC8JmSlVBeAewBsAlAN4BWlVJWIrBCRnvkcfwdgOIBXReQzEVnXz+6I\niIjIA5++h6yU2qiUylFKTVBKWZxlS5VS65z/n62USldKfcv502fiZSLSBqsVMJsBg8Hxr5XDvhNp\nQlSM1EXkr6qj7dh20IbWTjuSEwyYlWlEXmpSpMMadFYrUFIC2JwjzdbXO5YBoJhfUqAA3HvvvZmB\njDn90UcfDd23b1/ijTfe2BLMfqJJTI4XSrGt6mg73mpoQ2unY7yE1k473mpoQ9VRj2MaRJXS0rPJ\nuIfN5ignCqfy8nLjm2++mRLpOLSECZlizraDNnS5jYfTpRzl0a6hn+Hd+ysn/VuzfU1q5srMfMNy\nQ0Hmysz8NduDm34RAP7jP/4jw2w2Ty0oKJhUW1s7BACqqqqGXHrppRPz8vKmFBQUTNq5c2cSACxc\nuNB8yy23ZE+dOnWK2Wye+tJLL6W0t7fLo48+mrl+/fpzJk+enPunP/3pHACorq4eOmPGjEljx47N\n/81vfuPznMvRggmZYk7PlbGv5dGkv6//c1iA6LRm+5rUJe8sMTW2NSYqKDS2NSYueWeJKZik/MEH\nHxhff/311MrKyl2bN2+uraioGAYAt912m+npp59uqKqqqv7d7363/6677jrzrtq3b9+QioqK6vXr\n19cuXrzYZLfb8dBDDx289tprj9XU1Oy6/fbbjwHAnj17krZt2/bl9u3bq5944onM06dPS/CtoB9M\nyBRzkhM8v+37K/eXljtNWSyAsfew7zAaHeUUfVa8vyKrvav39IvtXe2GFe+vyAp0n1u3bh0+f/78\n4yNGjLCnpqbar7766uPt7e2GnTt3Dr/hhhsmTJ48Offuu+82HTp0KKFnm4ULFx6Ni4tDfn7+6XHj\nxp3+7LPPPHbYuPrqq48PHTpUjRkzpis1NbWzZyrHWBFTL5YIAGZlGvFWQ1uv29bx4igHguvwpfVO\nUz0xlJY6blNnZzuScSRji9UOduHQ1OZ5msX+ygNlt9sxYsSIrpqaml2e1ovIgMs9hgwZcua3Mi4u\nDl1dXbxCJopmealJmJc9/MwVcXKCAfOyhyMvNSnoDl966DRVXAzU1QF2u+PfSCfjWO1gFw4Zwz1P\ns9hfuS+uvPLKto0bN45sa2uTY8eOGTZv3jzSaDTax44d2/Hcc8+dAzgS9Mcffzy0Z5u1a9ee093d\njaqqqiH79u0bMm3atPbk5OTutrY25iAXbAyKSXmpSbh7aioenD4ad09NPXNFFmyHL3aa8k8sd7AL\nh6WXLT2QFN97+sWk+CT70ssCn37xkksusV133XVHp06dmjd79uyJ559//kkAeOmll77685//PHrS\npEm5EydOzHvttddG9myTlZXVMW3atCnf/e53J65atareaDSqefPmnfjyyy+HunbqinW8ZU3kItgO\nX9nZjtvUnsqpr1juYBcOd17omOVpxfsrspramhIzhmd0LL1s6YGe8kA9/vjjTY8//niTe/kHH3xQ\n66n+VVdddeLFF1/s9Wdpenp69xdffFHtUnTMdX1tbW1VMDHqkW4TMuc/psGQnGDwmAx87fBlsfR+\nhgyw09RAgm1v8u7OC+88GmwCpvDQZUK2VlpRsr4Etk7Hp159Sz1K1jt6zjApUzC8dfjyRoudprQs\n2Pb2hJ3EtOW1116ri3QMeqHLP0NLt5SeScY9bJ02lG7RUM8Z0qWBOnz5SkudprQuFO3tip3ESM90\neYXc0OK5h0x/5UT+yEtN8ishWK36viKOdPz+tvdABuokxqtk0jpdJuTslGzUt/TtOZOdwp4zFF5a\n/96xN3qP3x07iZGe6fKWtaXIAmNC72dMxgQjLEXsOUPhpYfvHQ9E7/G7G+xR2IgGky7fpcX5xSi7\ntgymFBMEAlOKCWXXlrFDF4VdOL53PJhDcUbb96ZnZRoR7za2U7CdxIjCRZcJGXAk5brFdbA/bEfd\n4jomY+olXONJD/ZkDT23lOvrAaXO3lIO1euJtskmQt1JjPo6fPhw3GOPPZYWyLa7d+9OXLMm+Nmm\nPMVRV1eXMHfu3HNDse9I0W1CpthSdbQdT39xFI/tPIynvzg6YK/ZwU5irgZ7sobBvqXsKf6EBKCt\nTZuTY/iiv1HYYtWaNUjNzES+wYCCzEzkr1mDoBLikSNH4p599tmApkasra0d8vLLL/t8/M7OTp/j\nMJvNnW+//fZXgcSlFUzIpHn+fpUlnM9Fi4uBsjLAZAJEHP+WlYWuQ9Rg31J2j3/UKMe/R46E7o8Z\nb3crtDw7lt6tWYPUJUtgamxEolJAYyMSlyyBKZikfN99943dt2/fkMmTJ+fecccdY3/961+nT506\ndUpOTk7ukiVLMgFg27ZtxpycnFybzSatra2G8847L2/79u1JpaWlWeXl5cMnT56cu3z5co9J/ckn\nnxx15ZVXnvftb3875zvf+c6klpYWw0UXXZSTm5s7JScnJ/eFF14Y6SmO3bt3J06cODEPAGw2m1x/\n/fXmnJyc3ClTpuSuX79+RKCvN5xEKeW91iAoLCxU5eXlETk26cvTXxztdzSnu6f2/VwxGBzJxJ2I\n47vBemI2ex6K02RyfMdZ68dz78UNOK7Ie/5o8bae+vrnP//ZMXv27Epf6mZmIr+xEX1mdhozBh0H\nD8KnfbjbvXt34jXXXDOxtra2au3atcmvvvrqOVartV4phdmzZ5/3wAMPNM2bN6/tF7/4RWZ7e7vh\n1KlThrFjx3Y8+uijTRs2bBixcuXK9K1bt+7pb/9PPvnkKIvFkvX5559Xpaend3d2duLEiROG1NRU\ne2NjY/zMmTMn19XVfVFbW3smDve4Hn744fRdu3YNffXVV+t27tyZNH/+/Il79+79wmg0Ribhuamo\nqBg9bdo0s3s5r5BJ81o7+vkqSz/l0fRcNNzzF4f6itzb3Ypo6+WtNU1NfZPxQOX+evvtt5Pff//9\n5Nzc3Ny8vLzcvXv3JtXU1CQBwG9/+9vGbdu2JVdUVBgfeeSRPuNeD+TSSy9tTU9P7wYAu90uixcv\nHpuTk5N7xRVX5Bw6dCjR2zzJH3300fAf/ehHRwBg+vTp7ZmZmR2VlZWaf3bBhEyad6LZ89u0v/Jw\nJ7HBNNi3xN0F8sfMQLecvSX4aOvlrTUZGfA8/WI/5f5SSmHx4sWNNTU1u2pqanY1NDR8sWTJksMA\n8M0338TbbDbDyZMn42w2m1+5xmg0nvlr+5lnnkk9cuRIfGVlZXVNTc2uUaNGdZ46dSoqc1dUviiK\nLhv/YEQOOX7+AAASpElEQVTHqd5lHacc5Z6EO4kNtnAOxenvHzPeOtB5S/DRdDdDi5YuxYGkJPSe\nfjEJ9qVLEfD0iykpKd0nT540AMC8efNa//a3v41uaWkxAMDXX3+dcODAgXgA+MlPfmIqLS09eP31\n1x+55557xvZs29bWFufP8VpaWuJGjx7dOWTIELV+/foRBw8eTHSPw93FF1/c9sILL6QCwOeffz6k\nsbEx8fzzz9f8+Km6HKkL6DuA/ITkBOxt7ex3mQPM69exXUlY+wgw5x4bRmbYcbzJgE2rjThe3f/5\nLC7WbwKOJH8nxxjolnNxsWPbP7zSjitKzp67rWVG/PIHjnPH2bEG1513wjH94gpkNTUhMSMDHUuX\n4kBPeSAyMjK6CwoK2iZOnJh35ZVXttxwww1HL7zwwsmA48rWarV+/frrr6ckJCSoO++882hXVxcu\nuOCCyevWrRsxZ86ctri4ODVp0qTcW2655fDDDz98yNvxbrvttqPz5s07LycnJ/f888+3jR8/vt1T\nHPfee++ZfT3wwAOHFi1aZMrJycmNi4vDM888Uzd06FBNPD8eiC47dfX0unUfs3Yg8QJ+H1Gnwt3x\nJ9JjO+uJtw50VUfbseHrNiiX6xixA9eMP/u7yPb2jz+dukib+uvUpcsrZE8DyHvDAeb94+0ORPzX\nRvznz5NC9iE60JR54ZzSMNrGdh5s2dmee2X33HLedtDWKxkDgDL0/l3k3QwiB10+Qw50oHgOMO8b\nT9/73XnkdK/lplFtGDmlPSTfVfXle8bheo4ajb1+B/N7vt6eOXOyB+rPa6+9ljx58uRc15+rrrpq\nQqTjiiRdJuRAB4rnAPO+8eUOROJQxzPdHu5Jy58kMNCUeYHyNrJXf/HpodevP2072KOWeetAF4nJ\nHjjQiD4sXLiwtad3ds/P5s2b90Y6rkjSZYbyNIC8N2IH1j5uDPiXNJZ+yX29ehmZ0bteT9LylAR+\n8hNg9GjP7RfqqyhvV9wDJSmt9/r1N8GG44p/oLsX4Z7sIZzDphKFmk8JWUTmishuEdkjIg96WD9E\nRF52rv9ERMyhDtSVpwHkp48a0u9yXIcBr60Yji0vJAX0Sxprv+S+Xr0cb+pdz2Bw/Nx6a98k0NnZ\nezhG1wR94pD/V1HufyDdfffZ5Re3D3zFPVCS8mVsZ9dj+fLHWSj/mOsv9ltvDex7wIHw5/V4+l1N\n3jcc370gyeehNP1pb1/+AImlP65JX7z2shaROABfArgKwH4A2wHcrJTa5VLnbgDnK6XuFJGbAFyn\nlLpxoP0GPXSme9fM+fOBjRs9LpsNDajvHttnF3Hogh0GZMcdxPxJe7Bx93lo6M5EdtxBXP/jZgz5\nfhaGpxtwvEmwafUwVLyd1O/2rvXbvrHj9NoD+Mdf0jzuT+vr21vsGDpCAfEuXxdUynFP0qmrAzh9\nUmBMUTjeZED1B/GYcmnXma+2VH+QgCmXdvq0fLIFSBoGxLuOHdTVjVMnDEhKEbR9Y4dp606UvHoH\n0NAAa+o9+MO3LbjiLtf9nT0+pFeoLi9BAUrOfG2q53xOm9t+5itV7m2TKsdgnpuEop+d9nis400G\n1H4ouOiyI+hOS0by4W8wobkee9NMaB2djrjmVnz8/ihMvFidqb/7/8ZhyqWdSE4H2r6xQ1Ueg+Sf\nc+bcuC9nfFWHuIlD0To6Hcea4nvF7h6/+/6PNwmqP0js0/b5l53CiHT0idfTcnftKTSda8bwdANa\nm4Ca/5uAnEu6+339rvV9OXdb/zsBv/xXKYqPrvZ6br21n/u5dW8fT/H7ez6CXTZVN6Lkvml+fuCd\nxV7W+tdfL2tfEvJFAJYppeY4lx8CAKXUoy51NjnrfCwi8QCaAKSpAXYeVEL29D2YARjQDeX1ZoAC\n4PgUnza3Hd//dRsSh55d23EKWPvI8D5J2Zf6elzf1QEknmyDPcWI5KYDmPDxu9h77Q/QmjgUSV0d\naLcbgMSznfTd8rXfy10dQOdJO4amANJiQ+ew4b0SdMcpIOOR51Hy9r9j9tx9uOzXSb3idd+fNz2v\nH4DHtvHnWH2O7VYQbNu4L3s7d/7uz1uFUMQ7UHt2nALef6Qd/3x7XEDtPdjtMxjnL+PjAwEnZSZk\n/QtmLOssAPtclvc7yzzWUUp1AWgBMCqgSH3h6b7UALLhy/25s78xc+6x9foFBvp2YnLlrb4e18cn\nAi2njHiwMB13X3MB5lj+HXffeCkenD4aiUeP9ErGQN9k6O9yfCLQfioeDxamo/WUsffVsjPe+nt+\nBAAouMfYJ15/knHP/ubcY+u3bfw5Vp9juxUE2zbuy97Onb/781YhFPEO1J6JQx3l/a2PdPsMxvmr\nnzIG0SgrKyu/sbExHgCmT58+2Z/6AwlmDmZAP/Mwh7VTl4iUiEi5iJQ3NzcHviM/H4BZ8CsYcdLn\n+u6dlYIt1+v64Rlubw9nu7eOTvdYP1gjM7o9H9ctnp563iilALvd48AVjv3Yvb52X48VbiMzuiHo\nP36t8daePeWhau+RGXZNt8/w9PB99H7afCr1j5VH8h/bebjgj5VH8j9tPhWSxOTNzp07a0K1r2Dm\nYAb0Mw+zL++KAwDGuSyPdZZ5rOO8ZZ0C4Ij7jpRSZUqpQqVUYVpawH/s+N3ltRgvoQy3w4Q6COyI\nQ9eA9d07K50tF4/b91/foOv1bU1uH2bOdk8+/I3H+sGKbzrm+bhu8fTU87q/xmN4sDAdbY397y9U\nxwq3lKb9sCMO8U0Bj4AYVt7as6c8VO3d1mSHHXH9nt9Ia/smPHF92nwqdcuBk6aTXSoRAE52qcQt\nB06aQpGUZ8+ePSEvL2/Keeedl/fEE0+Mdl9vNBqnA8CGDRtGzJgxY9LcuXPPHT9+fN6CBQvG293m\nQW1ra5PLLrts4sqVK/vsB+g79zEAROM8zL4k5O0AJorIeBFJBHATgHVuddYBuNX5/+sBvDvQ8+Og\neeoK60UxXkIdxsOOODyPRR6umM+Gu2m158kMJqz+i8ft+6u/abVR1+tNq/92tsBltIdZHYcQf8rt\n9r376fZzWU51YP7qhwA4jjtQPPNXPwQ51XuyGve3m6/7C8Wx/H2tQW6O+FM2zFr9mwHiG3j7UMfr\nrb639nQ9V760t7d4XI/n6fwG2/7BLnecAkzVjQiHD5tsWd1uHWi6FQwfNtncHzv6zWq11lVVVVV/\n9tlnu5555pn0pqamfieNqK6uHvrUU0/t27NnT1VDQ8OQzZs3D+9Z19raarj66qsn/uAHPzh63333\nHfa0/cqVK/ePGzfudE1Nza5nnnlm/9q1a5P37NmT9Pnnn1dXV1fv+uyzz4xvvfXW8FmzZtnmzp17\nfPHixVk/+9nPxt5www1HLrzwwnaLxXKgsLCwraamZtdA42dXVVUZ33jjjb3bt2/fbTQa7W+++eae\nXbt2VW/btu3LX/3qV2PtdnufWFy3f/zxx/9NRPDll1/uevHFF78qKSkx22w2nx+oeU3IzmfC9wDY\nBKAawCtKqSoRWSEiC5zVngUwSkT2ALgXQJ+vRoWUp9EI7rrL5+Vi00coK3oZprj9ENhhituPu3K3\nnVk+vvkwWp/bhxONdig7cKLRjozXd6Ck+o8et/dUv/W5fTi++bBu17u+XvfRHvKuLcK8I7VIPtQI\n2O1IPtSI6bs+CWr5mo/fQF71DkAEJdV/RMbrO/qNJ696B675+I1e218Q4P5KNt2PjN++HNSx/H2t\nwz/6qtfxTmw+MuDy8I++6rX9vI/fPPPaPMXnvn/37YON19v+3Ot7a0/Xc+VLe3trP9fjeTr37vH7\nez6CXQ6mQ5e/eq6MfS33x+OPP54+adKk3IKCgilNTU0JVVVV/Y5LnJ+ff3LChAmdcXFxyMvLs+3d\nu/fM8RcsWHDej370o8P33HNPn7uq/YnWeZh1ObkEEVGs8qeX9R8rj+R7Sr7D4qXj5/mjAu6pvWHD\nhhEPP/xw5nvvvVc7YsQI+4wZMyYtXbr04B133GEuLy+vHjNmTJfRaJxus9l2btiwYcTKlSvTt27d\nugcAFi1alF1YWHjyF7/4xZGsrKz8K664oqWtrS1u7dq1XxsMnq8Rd+/enXjNNddMrK2trQKA22+/\nfWxOTk77/fff3+eKur6+PuGSSy6ZlJiYqHbu3FmdnJxsd4/BkyeffHJUeXn5sL/+9a8NPcubNm1K\nWbt27ddDhgxRWVlZ+e++++5uAHCNxTW2q666asLPf/7zQwsWLDgBAAUFBZOefvrphpkzZ/a6TxNM\nL2siItKhizOMB+Kk93zIcQL7xRnGgOdDBoDjx4/HpaSkdI8YMcK+c+fOpIqKimGB7ut3v/vdwZEj\nR3YtWrSo385B7nMfR+s8zEzIRERR6oK0oUeLsobVD4uXDsBxZVyUNaz+grShQfUGXLhwYUtXV5ec\ne+65effff3/WtGnTfP8aiwfPPffcvvb2dsOdd97ZdwQn9J77+I477hj7/e9/v7VnHuacnJzc6667\nbsLx48fjVq9ePapnHmaLxdL02WefGdetWzdixowZp3rmYe6vU5e722677WhFRcWwnJyc3Oeff36U\np3mYezqY9XjggQcO2e12ycnJyb3xxhsn+DsPM29ZExHpCAcG0T/esiYiItIwryOkEBERhUNTU1Pc\n5ZdfPsm9/L333tudkRGaUWNee+215NLS0l63mseNG3daC1M/MiETEZEmZGRkdNfU1OzyXjNwCxcu\nbF24cOGgHiNQvGVNRKQjSinY7XY/R28nrXCeO49DtTEhExHpSF1d3anm5uYUJmX9sdvt0tzcnALg\nC0/rI3bLeseOHYdFpD4EuxoNwONwaxrB+IKj5fi0HBvA+IKlyfhSU1PNM2bM2NXU1DQVvKjSGzuA\nL7q6um7ztDJiX3sKFREpV0oVRjqO/jC+4Gg5Pi3HBjC+YGk9Poo+/OuKiIhIA5iQiYiINCAaEnJZ\npAPwgvEFR8vxaTk2gPEFS+vxUZTR/TNkIiKiaBANV8hERES6p+uELCJzRWS3iOwRkQc1EM9zInJI\nRL5wKUsVkc0iUuv895wIxTZORLaKyC4RqRKRX2osviQR+X8iUuGMb7mzfLyIfOI8xy+LSNATqwcZ\nZ5yI7BSRDVqLT0TqRKRSRD4TkXJnmVbO70gR+YeI1IhItYhcpKHYJjnbrOenVUQWayU+ih26Tcgi\nEgfgKQDzAOQCuFlEciMbFf4CYK5b2YMAtiilJgLY4lyOhC4A9ymlcgF8G8DPnO2llfhOA7hSKTUN\nwLcAzBWRbwN4HMDvlVLnATgG4KcRiq/HLwFUuyxrLb4rlFLfcvm6jlbO7x8AvK2UmgxgGhxtqInY\nlFK7nW32LQAFAGwAXtdKfBRDlFK6/AFwEYBNLssPAXhIA3GZAXzhsrwbwBjn/8cA2B3pGJ2xvAHg\nKi3GB8AI4FMAM+EYmCHe0zmPQFxj4fhgvhLABgCisfjqAIx2K4v4+QWQAuBrOPusaCk2D7FeDeBD\nrcbHn+j+0e0VMoAsAPtclvc7y7QmXSnV6Px/E4D0SAYDACJiBjAdwCfQUHzO28GfATgEYDOAvQCO\nK6W6nFUifY5XAXgAZ8ehHQVtxacAvCMiO0SkxFmmhfM7HkAzgD87b/f/j4gM00hs7m4C8JLz/1qM\nj6KYnhOy7iilFBwfmhEjIsMBvAZgsVKq1XVdpONTSnUrx23DsQBmAJgcqVjcicg1AA4ppXZEOpYB\nXKKUugCOxzg/E5HLXFdG8PzGA7gAwH8rpaYDOAm327+Rfu8BgPP5/wIAr7qv00J8FP30nJAPABjn\nsjzWWaY134jIGABw/nsoUoGISAIcydiqlFqrtfh6KKWOA9gKxy3gkSLSM+Z6JM/xxQAWiEgdgL/D\ncdv6D9BOfFBKHXD+ewiOZ6AzoI3zux/AfqXUJ87lf8CRoLUQm6t5AD5VSn3jXNZafBTl9JyQtwOY\n6OzlmgjHraZ1EY7Jk3UAbnX+/1Y4nt2GnYgIgGcBVCul/stllVbiSxORkc7/D4Xj+XY1HIn5+kjH\np5R6SCk1VillhuO99q5Sqlgr8YnIMBEZ0fN/OJ6FfgENnF+lVBOAfSLSM/F8EYBdWojNzc04e7sa\n0F58FOV0PTCIiMyH47leHIDnlFKWCMfzEoDL4Zgl5hsADwP4XwCvAMgGUA/gB0qpoxGI7RIAHwCo\nxNlnoL+C4zmyFuI7H8DzcJxLA4BXlFIrRORcOK5IUwHsBPBDpdTpcMfnSkQuB/DvSqlrtBKfM47X\nnYvxAF5USllEZBS0cX6/BeB/ACQC+ArAT+A8z5GOzRnfMAANAM5VSrU4yzTRdhQ7dJ2QiYiIooWe\nb1kTERFFDSZkIiIiDWBCJiIi0gAmZCIiIg1gQiYiItIAJmQiIiINYEImIiLSACZkIiIiDfj/ucqF\n1L/PR4sAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10a2fb150>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(dataframe.index, dataframe.target, color='red', label='target')\n",
    "plt.scatter(numeric_features.index, numeric_features.depth, color='green', label='depth')\n",
    "plt.scatter(numeric_features.index, numeric_features.text_ratio, color='blue', label='text_ratio')\n",
    "plt.scatter(numeric_features.index, numeric_features.alink_text_ratio, color='skyblue', label='alink_text_ratio')\n",
    "plt.legend(loc=(1, 0))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Next is the `train` and `predict` routine\n",
    "\n",
    "Here for simplicity reason, I'll just verify against the training dataset itself. But in the real world, you should split your data into training and testing dataset, and use [cross validation](http://scikit-learn.org/stable/modules/cross_validation.html) to evaluate the model's performance.\n",
    "\n",
    "As shown in the code, I also tried several classifiers, the Naive Bayes, Random Forest, and finally chose SVM with a polynomial kernel. I cannot explain why it works better then others, maybe a polynomial model is more suitable in my case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        False         True \n",
      "0   -0.002231 -6.106299e+00\n",
      "1   -0.002263 -6.092221e+00\n",
      "2   -0.002258 -6.094217e+00\n",
      "3   -0.002263 -6.092019e+00\n",
      "4   -0.002263 -6.091994e+00\n",
      "5   -0.002263 -6.091983e+00\n",
      "6   -0.002263 -6.091982e+00\n",
      "7   -0.002263 -6.091984e+00\n",
      "8   -0.002263 -6.091983e+00\n",
      "9   -0.002263 -6.091982e+00\n",
      "10  -0.002263 -6.091983e+00\n",
      "11  -0.002263 -6.091982e+00\n",
      "12  -0.002263 -6.091983e+00\n",
      "13  -0.002263 -6.091982e+00\n",
      "14  -0.002263 -6.091983e+00\n",
      "15  -0.002263 -6.091982e+00\n",
      "16  -0.002263 -6.091983e+00\n",
      "17  -0.002263 -6.091982e+00\n",
      "18  -0.002263 -6.091983e+00\n",
      "19  -0.002263 -6.091982e+00\n",
      "20  -0.002263 -6.091983e+00\n",
      "21  -0.002263 -6.091982e+00\n",
      "22  -0.002263 -6.092320e+00\n",
      "23 -31.137579 -3.008704e-14\n",
      "24  -0.002261 -6.092956e+00\n",
      "25  -0.002263 -6.091982e+00\n",
      "26  -0.002263 -6.091983e+00\n",
      "27  -0.002263 -6.091983e+00\n",
      "28  -0.002263 -6.091988e+00\n",
      "29  -0.002263 -6.092045e+00\n",
      "..        ...           ...\n",
      "46  -0.002263 -6.091982e+00\n",
      "47  -0.002263 -6.091982e+00\n",
      "48  -0.002263 -6.091982e+00\n",
      "49  -0.002263 -6.091982e+00\n",
      "50  -0.002263 -6.091982e+00\n",
      "51  -0.002263 -6.091982e+00\n",
      "52  -0.002263 -6.091982e+00\n",
      "53  -0.002263 -6.091982e+00\n",
      "54  -0.002263 -6.091982e+00\n",
      "55  -0.002263 -6.091980e+00\n",
      "56  -0.002263 -6.091982e+00\n",
      "57  -0.002263 -6.091982e+00\n",
      "58  -0.002263 -6.091981e+00\n",
      "59  -0.002263 -6.091982e+00\n",
      "60  -0.002263 -6.091982e+00\n",
      "61  -0.002263 -6.091982e+00\n",
      "62  -0.002263 -6.091982e+00\n",
      "63  -0.002263 -6.091982e+00\n",
      "64  -0.002263 -6.091981e+00\n",
      "65  -0.002263 -6.091982e+00\n",
      "66  -0.002263 -6.091983e+00\n",
      "67  -0.002263 -6.091982e+00\n",
      "68  -0.002263 -6.092000e+00\n",
      "69  -0.002263 -6.091982e+00\n",
      "70  -0.002263 -6.091982e+00\n",
      "71  -0.002263 -6.091982e+00\n",
      "72  -0.002263 -6.091982e+00\n",
      "73  -0.002263 -6.091982e+00\n",
      "74  -0.002263 -6.091982e+00\n",
      "75  -0.002263 -6.091982e+00\n",
      "\n",
      "[76 rows x 2 columns]\n",
      "Number of mispredicted out of 76 is 0 (0.00%)\n",
      "\n",
      "Predicted rows:\n",
      "    target     depth  has_img                 attr      False          True\n",
      "23    True  0.555556    False  div,content,writing -31.137579 -3.008704e-14\n",
      "\n"
     ]
    }
   ],
   "source": [
    "scaler = preprocessing.StandardScaler()\n",
    "scaler.fit(numeric_features)\n",
    "scaled_X = scaler.transform(numeric_features)\n",
    "\n",
    "# clf = MultinomialNB()\n",
    "# clf = RandomForestClassifier()\n",
    "clf = SVC(C=1, kernel='poly', probability=True)\n",
    "clf.fit(scaled_X, y)\n",
    "predicted_index = clf.predict(scaled_X).tolist().index(True)\n",
    "\n",
    "scaled_X = scaler.transform(numeric_features)\n",
    "pred_y = clf.predict(scaled_X)\n",
    "\n",
    "print pd.DataFrame(clf.predict_log_proba(scaled_X), \n",
    "                                           columns=clf.classes_)\n",
    "\n",
    "print 'Number of mispredicted out of %d is %d (%.2f%%)' % (y.shape[0], (y!=pred_y).sum(), (y!=pred_y).sum()*100.0/y.shape[0])\n",
    "print \n",
    "\n",
    "print 'Predicted rows:'\n",
    "print dataframe[pred_y].drop(['text_ratio', 'alink_text_ratio', 'contain_title'], axis=1).merge(pd.DataFrame(clf.predict_log_proba(scaled_X)[pred_y], \n",
    "                                           columns=clf.classes_, index=dataframe[pred_y].index), left_index=True, right_index=True)\n",
    "print \n",
    "\n",
    "# print 'Acutual rows:'\n",
    "# print dataframe[dataframe.target]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## And That’s It 🎉\n",
    "\n",
    "So we now have a working classifier that can tell us what DOM in a page is most likely to be the main content, and it is just under 20 lines of code! Along the way, I also learnt machine learning is not that obvious, you need to convert your problem into a model-solving one, and that's where machine learning can do the best.\n",
    "\n",
    "This probably shouldn’t be used in production, because it’s missing loads of features (like evaluating the performance of the model, error handling when the model thinks none of the DOM is the main content etc…) but has hopefully given you a idea that machine learning is not that intimidating, and how it can be used to solve a real-world problem."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}