Visualization_of_Football_Matches/visualization_football.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Visualization of Football Matches using the Lean Startup Method\n",
    "\n",
    "[Lean Startup](https://en.wikipedia.org/wiki/Lean_startup) has been a breakthrough during the last years in the entrepreneurial landscape. But this movement is not only a methodology that can be used to create successful businesses, it can also be applied to other domains like Data Science. In this [post](https://miguelgfierro.com/blog/2016/how-to-develop-a-data-science-project-using-the-lean-startup-method/), I discuss how to apply the Lean Startup method to a Data Science project. As an example, I created a project that visualizes all football matches that took place in the UEFA Champions League since 1955.\n",
    "\n",
    "There are several methodologies to develop data science projects. At Microsoft, we developed and intensively use the [Team Data Science Process](https://azure.microsoft.com/en-us/documentation/learning-paths/data-science-process/) (TDSP). This methodology enables us to efectively implement projects while collaborating between teams inside and outside the company.\n",
    "\n",
    "In some situations, when starting a Data Science project, we have a clear view of the business case, access to all customer data and a clear roadmap of what the customer wants. In that situation, the Lean Startup process has little value. However, when there is uncertainty, when the customer doesn't know what he wants or when we don't know if a product is going to be sucessful, then the Lean Startup method can prove its benefit.\n",
    "\n",
    "Implementing the Lean Startup method in the TDSP is easy, we can use all the tools TDSP proposes. In the Lean Startup method the priority is to reduce (or eliminate) the uncertainty to understand what the customer really wants.\n",
    "\n",
    "So following the Lean Startup method, first we have to set the hyphotesis. The next step is to build a [Minimun Viable Product](https://en.wikipedia.org/wiki/Minimum_viable_product) (MVP). The MVP has two important features, it helps us to validate the hypothesis or ideas we proposed and it is a complete, end to end product with the minimum number of features. The final step is to show the MVP to the customer and measure its impact."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <div class=\"bk-root\">\n",
       "        <a href=\"http://bokeh.pydata.org\" target=\"_blank\" class=\"bk-logo bk-logo-small bk-logo-notebook\"></a>\n",
       "        <span id=\"76c440ce-2298-48ae-a60f-7d5e47bbe87c\">Loading BokehJS ...</span>\n",
       "    </div>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": [
       "\n",
       "(function(global) {\n",
       "  function now() {\n",
       "    return new Date();\n",
       "  }\n",
       "\n",
       "  var force = \"1\";\n",
       "\n",
       "  if (typeof (window._bokeh_onload_callbacks) === \"undefined\" || force !== \"\") {\n",
       "    window._bokeh_onload_callbacks = [];\n",
       "    window._bokeh_is_loading = undefined;\n",
       "  }\n",
       "\n",
       "\n",
       "  \n",
       "  if (typeof (window._bokeh_timeout) === \"undefined\" || force !== \"\") {\n",
       "    window._bokeh_timeout = Date.now() + 5000;\n",
       "    window._bokeh_failed_load = false;\n",
       "  }\n",
       "\n",
       "  var NB_LOAD_WARNING = {'data': {'text/html':\n",
       "     \"<div style='background-color: #fdd'>\\n\"+\n",
       "     \"<p>\\n\"+\n",
       "     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n",
       "     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n",
       "     \"</p>\\n\"+\n",
       "     \"<ul>\\n\"+\n",
       "     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n",
       "     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n",
       "     \"</ul>\\n\"+\n",
       "     \"<code>\\n\"+\n",
       "     \"from bokeh.resources import INLINE\\n\"+\n",
       "     \"output_notebook(resources=INLINE)\\n\"+\n",
       "     \"</code>\\n\"+\n",
       "     \"</div>\"}};\n",
       "\n",
       "  function display_loaded() {\n",
       "    if (window.Bokeh !== undefined) {\n",
       "      Bokeh.$(\"#76c440ce-2298-48ae-a60f-7d5e47bbe87c\").text(\"BokehJS successfully loaded.\");\n",
       "    } else if (Date.now() < window._bokeh_timeout) {\n",
       "      setTimeout(display_loaded, 100)\n",
       "    }\n",
       "  }\n",
       "\n",
       "  function run_callbacks() {\n",
       "    window._bokeh_onload_callbacks.forEach(function(callback) { callback() });\n",
       "    delete window._bokeh_onload_callbacks\n",
       "    console.info(\"Bokeh: all callbacks have finished\");\n",
       "  }\n",
       "\n",
       "  function load_libs(js_urls, callback) {\n",
       "    window._bokeh_onload_callbacks.push(callback);\n",
       "    if (window._bokeh_is_loading > 0) {\n",
       "      console.log(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n",
       "      return null;\n",
       "    }\n",
       "    if (js_urls == null || js_urls.length === 0) {\n",
       "      run_callbacks();\n",
       "      return null;\n",
       "    }\n",
       "    console.log(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n",
       "    window._bokeh_is_loading = js_urls.length;\n",
       "    for (var i = 0; i < js_urls.length; i++) {\n",
       "      var url = js_urls[i];\n",
       "      var s = document.createElement('script');\n",
       "      s.src = url;\n",
       "      s.async = false;\n",
       "      s.onreadystatechange = s.onload = function() {\n",
       "        window._bokeh_is_loading--;\n",
       "        if (window._bokeh_is_loading === 0) {\n",
       "          console.log(\"Bokeh: all BokehJS libraries loaded\");\n",
       "          run_callbacks()\n",
       "        }\n",
       "      };\n",
       "      s.onerror = function() {\n",
       "        console.warn(\"failed to load library \" + url);\n",
       "      };\n",
       "      console.log(\"Bokeh: injecting script tag for BokehJS library: \", url);\n",
       "      document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
       "    }\n",
       "  };var element = document.getElementById(\"76c440ce-2298-48ae-a60f-7d5e47bbe87c\");\n",
       "  if (element == null) {\n",
       "    console.log(\"Bokeh: ERROR: autoload.js configured with elementid '76c440ce-2298-48ae-a60f-7d5e47bbe87c' but no matching script tag was found. \")\n",
       "    return false;\n",
       "  }\n",
       "\n",
       "  var js_urls = ['https://cdn.pydata.org/bokeh/release/bokeh-0.12.2.min.js', 'https://cdn.pydata.org/bokeh/release/bokeh-widgets-0.12.2.min.js', 'https://cdn.pydata.org/bokeh/release/bokeh-compiler-0.12.2.min.js'];\n",
       "\n",
       "  var inline_js = [\n",
       "    function(Bokeh) {\n",
       "      Bokeh.set_log_level(\"info\");\n",
       "    },\n",
       "    \n",
       "    function(Bokeh) {\n",
       "      \n",
       "      Bokeh.$(\"#76c440ce-2298-48ae-a60f-7d5e47bbe87c\").text(\"BokehJS is loading...\");\n",
       "    },\n",
       "    function(Bokeh) {\n",
       "      console.log(\"Bokeh: injecting CSS: https://cdn.pydata.org/bokeh/release/bokeh-0.12.2.min.css\");\n",
       "      Bokeh.embed.inject_css(\"https://cdn.pydata.org/bokeh/release/bokeh-0.12.2.min.css\");\n",
       "      console.log(\"Bokeh: injecting CSS: https://cdn.pydata.org/bokeh/release/bokeh-widgets-0.12.2.min.css\");\n",
       "      Bokeh.embed.inject_css(\"https://cdn.pydata.org/bokeh/release/bokeh-widgets-0.12.2.min.css\");\n",
       "    }\n",
       "  ];\n",
       "\n",
       "  function run_inline_js() {\n",
       "    \n",
       "    if ((window.Bokeh !== undefined) || (force === \"1\")) {\n",
       "      for (var i = 0; i < inline_js.length; i++) {\n",
       "        inline_js[i](window.Bokeh);\n",
       "      }if (force === \"1\") {\n",
       "        display_loaded();\n",
       "      }} else if (Date.now() < window._bokeh_timeout) {\n",
       "      setTimeout(run_inline_js, 100);\n",
       "    } else if (!window._bokeh_failed_load) {\n",
       "      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n",
       "      window._bokeh_failed_load = true;\n",
       "    } else if (!force) {\n",
       "      var cell = $(\"#76c440ce-2298-48ae-a60f-7d5e47bbe87c\").parents('.cell').data().cell;\n",
       "      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n",
       "    }\n",
       "\n",
       "  }\n",
       "\n",
       "  if (window._bokeh_is_loading === 0) {\n",
       "    console.log(\"Bokeh: BokehJS loaded, going straight to plotting\");\n",
       "    run_inline_js();\n",
       "  } else {\n",
       "    load_libs(js_urls, function() {\n",
       "      console.log(\"Bokeh: BokehJS plotting callback run at\", now());\n",
       "      run_inline_js();\n",
       "    });\n",
       "  }\n",
       "}(this));"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.5.2 |Anaconda custom (64-bit)| (default, Jul  2 2016, 17:53:06) \n",
      "[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]\n"
     ]
    }
   ],
   "source": [
    "#Load all libraries\n",
    "import os,sys  \n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import xarray as xr\n",
    "import datashader as ds\n",
    "import datashader.transfer_functions as tf\n",
    "from datashader import reductions\n",
    "from datashader.colors import colormap_select, Hot, inferno\n",
    "from datashader.bokeh_ext import InteractiveImage\n",
    "from bokeh.palettes import Greens3, Blues3, Blues4, Blues9, Greys9\n",
    "from bokeh.plotting import figure, output_notebook\n",
    "from bokeh.tile_providers import WMTSTileSource, STAMEN_TONER, STAMEN_TERRAIN\n",
    "from functools import partial\n",
    "import wget\n",
    "import zipfile\n",
    "import math\n",
    "from difflib import SequenceMatcher\n",
    "\n",
    "output_notebook()\n",
    "#print(sys.path)\n",
    "print(sys.version)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualization of Champions League matches\n",
    "Visualization is one of the key parts in a Data Science project. It allows us to get a global sense of our data and to understand better our results. \n",
    "\n",
    "There are many free and non-free tools in the market to make data visualization. One of my favourites is [datashader](https://github.com/bokeh/datashader), an open source python library that allows to visualize big amounts of data with a clean and nice API. \n",
    "\n",
    "### MVP1: Initial match visualization\n",
    "\n",
    "We can easily create a visualization of the Champion League matches from 1955 to 2016 using datashader. For that we need a dataset of the matches, such as [this one](https://github.com/jalapic/engsoccerdata/blob/master/data-raw/champs.csv) and the coordinates of the stadiums of the teams, that you can find [here](http://opisthokonta.net/?cat=34). This last dataset has the stadiums coordinates only of teams form England, Spain, France, Germany and Scotland. We will still need the data from other countries such as Italy, Portugal, Netherlands and many others, but for our first MVP we don't care about it, our objective is to reach a minimum product as fast as possible to reduce uncertainty, and 5 countries is enough for now. \n",
    "\n",
    "The first step is to treat the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of rows: 164\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Team</th>\n",
       "      <th>Stadium</th>\n",
       "      <th>Latitude</th>\n",
       "      <th>Longitude</th>\n",
       "      <th>Country</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Arsenal</td>\n",
       "      <td>Emirates Stadium</td>\n",
       "      <td>51.555000</td>\n",
       "      <td>-0.108611</td>\n",
       "      <td>England</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>Real Madrid</td>\n",
       "      <td>Santiago Bernabéu</td>\n",
       "      <td>40.453060</td>\n",
       "      <td>-3.688350</td>\n",
       "      <td>Spain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>64</th>\n",
       "      <td>Bayern Munich</td>\n",
       "      <td>Allianz Arena</td>\n",
       "      <td>48.218775</td>\n",
       "      <td>11.624753</td>\n",
       "      <td>Germany</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>121</th>\n",
       "      <td>Paris Saint-Germain</td>\n",
       "      <td>Parc des Princes</td>\n",
       "      <td>48.841389</td>\n",
       "      <td>2.253056</td>\n",
       "      <td>France</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     Team             Stadium   Latitude  Longitude  Country\n",
       "0                Arsenal    Emirates Stadium   51.555000  -0.108611  England\n",
       "99           Real Madrid   Santiago Bernabéu   40.453060  -3.688350    Spain\n",
       "64         Bayern Munich       Allianz Arena   48.218775  11.624753  Germany\n",
       "121  Paris Saint-Germain    Parc des Princes   48.841389   2.253056   France"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_stadium = pd.read_csv(\"stadiums.csv\", usecols=['Team','Stadium','Latitude','Longitude','Country'])\n",
    "print(\"Number of rows: %d\" % df_stadium.shape[0])\n",
    "dd1 = df_stadium.take([0,99, 64, 121])\n",
    "dd1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next step is to match the club names in the dataset of coordinates with the those in the dataset of matches. They are similar but not always exactly the same, for example, in the dataset of coordinates we have `Real Madrid FC` and in the dataset of matches we have `Real Madrid`. Furthermore, in the first one there are several entries for some teams, like `Atletico Madrid`, `Atletico Madrid B` or `Atletico Madrid C` meaning they are the teams from the first division and from other divisions. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of teams that have participated in the Champions League: 502\n",
      "Number of matches in the dataset: 6554\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>home</th>\n",
       "      <th>visitor</th>\n",
       "      <th>home_country</th>\n",
       "      <th>visitor_country</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1955-09-04</td>\n",
       "      <td>Sporting CP</td>\n",
       "      <td>Partizan Belgrade</td>\n",
       "      <td>POR</td>\n",
       "      <td>SRB</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1955-09-07</td>\n",
       "      <td>Budapesti Voros Lobogo</td>\n",
       "      <td>RSC Anderlecht</td>\n",
       "      <td>HUN</td>\n",
       "      <td>BEL</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1955-09-08</td>\n",
       "      <td>Servette Geneve</td>\n",
       "      <td>Real Madrid</td>\n",
       "      <td>SUI</td>\n",
       "      <td>ESP</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1955-09-14</td>\n",
       "      <td>Rot-Weiss Essen</td>\n",
       "      <td>Hibernian FC</td>\n",
       "      <td>GER</td>\n",
       "      <td>SCO</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1955-09-20</td>\n",
       "      <td>Djurgardens IF</td>\n",
       "      <td>Gwardia Warszawa</td>\n",
       "      <td>SWE</td>\n",
       "      <td>POL</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Date                    home            visitor home_country  \\\n",
       "0  1955-09-04             Sporting CP  Partizan Belgrade          POR   \n",
       "1  1955-09-07  Budapesti Voros Lobogo     RSC Anderlecht          HUN   \n",
       "2  1955-09-08         Servette Geneve        Real Madrid          SUI   \n",
       "3  1955-09-14         Rot-Weiss Essen       Hibernian FC          GER   \n",
       "4  1955-09-20          Djurgardens IF   Gwardia Warszawa          SWE   \n",
       "\n",
       "  visitor_country  \n",
       "0             SRB  \n",
       "1             BEL  \n",
       "2             ESP  \n",
       "3             SCO  \n",
       "4             POL  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_match = pd.read_csv('champions.csv', usecols=['Date','home','visitor','hcountry','vcountry'])\n",
    "df_match = df_match.rename(columns = {'hcountry':'home_country', 'vcountry':'visitor_country'})\n",
    "df_teams_champions = pd.concat([df_match['home'], df_match['visitor']])\n",
    "teams_champions = set(df_teams_champions)\n",
    "print(\"Number of teams that have participated in the Champions League: %d\" % len(teams_champions))\n",
    "print(\"Number of matches in the dataset: %d\" % df_match.shape[0])\n",
    "df_match.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To find the string similarity you can use different methods. Here we will use a simple method to calculate it with `difflib`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Real Madrid FC and Real Madrid (Idx=99) are 0.923077 similar.\n",
      "New maximum rank: 0.923077\n",
      "('Santiago Bernabéu ', 40.453060000000001, -3.6883499999999998)\n",
      "Atletico de Madrid FC and Atlético Madrid (Idx=87) are 0.810811 similar.\n",
      "New maximum rank: 0.810811\n",
      "('Vicente Calderón ', 40.401719, -3.7206059999999996)\n",
      "('Unknown', nan, nan)\n"
     ]
    }
   ],
   "source": [
    "def similar(a, b):\n",
    "    return SequenceMatcher(None, a, b).ratio()\n",
    "\n",
    "def get_info_similar_team(team, df_stadium, threshold=0.6, verbose=False):\n",
    "    max_rank = 0\n",
    "    max_idx = -1\n",
    "    stadium = \"Unknown\"\n",
    "    latitude = np.NaN\n",
    "    longitude = np.NaN\n",
    "    for idx, val in enumerate(df_stadium['Team']):\n",
    "        rank = similar(team, val)\n",
    "        if rank > threshold:\n",
    "            if(verbose): print(\"%s and %s(Idx=%d) are %f similar.\" % (team, val, idx, rank))\n",
    "            if rank > max_rank:\n",
    "                if(verbose): print(\"New maximum rank: %f\" %rank)\n",
    "                max_rank = rank\n",
    "                max_idx = idx\n",
    "                stadium = df_stadium['Stadium'].iloc[max_idx]\n",
    "                latitude = df_stadium['Latitude'].iloc[max_idx]\n",
    "                longitude = df_stadium['Longitude'].iloc[max_idx]\n",
    "    return stadium, latitude, longitude\n",
    "print(get_info_similar_team(\"Real Madrid FC\", df_stadium, verbose=True))\n",
    "print(get_info_similar_team(\"Atletico de Madrid FC\", df_stadium, verbose=True))\n",
    "print(get_info_similar_team(\"Inter Milan\", df_stadium, verbose=True))\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next step is to create a dataframe relating each match with the stadium coordinates of each team"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 1min 50s, sys: 460 ms, total: 1min 51s\n",
      "Wall time: 1min 51s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "df_match_stadium = df_match\n",
    "home_stadium_index = df_match_stadium['home'].map(lambda x: get_info_similar_team(x, df_stadium))\n",
    "visitor_stadium_index = df_match_stadium['visitor'].map(lambda x: get_info_similar_team(x, df_stadium))\n",
    "df_home = pd.DataFrame(home_stadium_index.tolist(), columns=['home_stadium', 'home_latitude', 'home_longitude'])\n",
    "df_visitor = pd.DataFrame(visitor_stadium_index.tolist(), columns=['visitor_stadium', 'visitor_latitude', 'visitor_longitude'])\n",
    "df_match_stadium = pd.concat([df_match_stadium, df_home, df_visitor], axis=1, ignore_index=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of missing values for home teams: 4499 out of 6554\n",
      "Number of matches with complete data: 728 out of 6554\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>home</th>\n",
       "      <th>visitor</th>\n",
       "      <th>home_country</th>\n",
       "      <th>visitor_country</th>\n",
       "      <th>home_stadium</th>\n",
       "      <th>home_latitude</th>\n",
       "      <th>home_longitude</th>\n",
       "      <th>visitor_stadium</th>\n",
       "      <th>visitor_latitude</th>\n",
       "      <th>visitor_longitude</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1955-09-04</td>\n",
       "      <td>Sporting CP</td>\n",
       "      <td>Partizan Belgrade</td>\n",
       "      <td>POR</td>\n",
       "      <td>SRB</td>\n",
       "      <td>El Molinón</td>\n",
       "      <td>43.536111</td>\n",
       "      <td>-5.637222</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1955-09-07</td>\n",
       "      <td>Budapesti Voros Lobogo</td>\n",
       "      <td>RSC Anderlecht</td>\n",
       "      <td>HUN</td>\n",
       "      <td>BEL</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1955-09-08</td>\n",
       "      <td>Servette Geneve</td>\n",
       "      <td>Real Madrid</td>\n",
       "      <td>SUI</td>\n",
       "      <td>ESP</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Santiago Bernabéu</td>\n",
       "      <td>40.453060</td>\n",
       "      <td>-3.688350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1955-09-14</td>\n",
       "      <td>Rot-Weiss Essen</td>\n",
       "      <td>Hibernian FC</td>\n",
       "      <td>GER</td>\n",
       "      <td>SCO</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Easter Road</td>\n",
       "      <td>55.961667</td>\n",
       "      <td>-3.165556</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1955-09-20</td>\n",
       "      <td>Djurgardens IF</td>\n",
       "      <td>Gwardia Warszawa</td>\n",
       "      <td>SWE</td>\n",
       "      <td>POL</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Date                    home            visitor home_country  \\\n",
       "0  1955-09-04             Sporting CP  Partizan Belgrade          POR   \n",
       "1  1955-09-07  Budapesti Voros Lobogo     RSC Anderlecht          HUN   \n",
       "2  1955-09-08         Servette Geneve        Real Madrid          SUI   \n",
       "3  1955-09-14         Rot-Weiss Essen       Hibernian FC          GER   \n",
       "4  1955-09-20          Djurgardens IF   Gwardia Warszawa          SWE   \n",
       "\n",
       "  visitor_country home_stadium  home_latitude  home_longitude  \\\n",
       "0             SRB   El Molinón      43.536111       -5.637222   \n",
       "1             BEL      Unknown            NaN             NaN   \n",
       "2             ESP      Unknown            NaN             NaN   \n",
       "3             SCO      Unknown            NaN             NaN   \n",
       "4             POL      Unknown            NaN             NaN   \n",
       "\n",
       "      visitor_stadium  visitor_latitude  visitor_longitude  \n",
       "0             Unknown               NaN                NaN  \n",
       "1             Unknown               NaN                NaN  \n",
       "2  Santiago Bernabéu          40.453060          -3.688350  \n",
       "3        Easter Road          55.961667          -3.165556  \n",
       "4             Unknown               NaN                NaN  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(\"Number of missing values for home teams: %d out of %d\" % (df_match_stadium['home_stadium'].value_counts()['Unknown'], df_match_stadium.shape[0]))\n",
    "df1 = df_match_stadium['home_stadium'] == 'Unknown'\n",
    "df2 = df_match_stadium['visitor_stadium'] == 'Unknown'\n",
    "n_complete_matches = df_match_stadium.shape[0] - df_match_stadium[df1 | df2].shape[0]\n",
    "print(\"Number of matches with complete data: %d out of %d\" % (n_complete_matches, df_match_stadium.shape[0]))\n",
    "df_match_stadium.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Now, even though there are many entries in the dataset that don't have any value, we are going to create a dataframe with the teams that do have values and advance in the project. This dataframe finds the combination of teams (home and visitor) that have values and concatenate each other to create the map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Latitude</th>\n",
       "      <th>Longitude</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>53.4631</td>\n",
       "      <td>-2.29139</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>51.4926</td>\n",
       "      <td>7.45184</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>55.8532</td>\n",
       "      <td>-4.30926</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>43.7233</td>\n",
       "      <td>7.25876</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Latitude Longitude\n",
       "0  53.4631  -2.29139\n",
       "1  51.4926   7.45184\n",
       "2      NaN       NaN\n",
       "3  55.8532  -4.30926\n",
       "4  43.7233   7.25876"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def aggregate_dataframe_coordinates(dataframe):\n",
    "    df = pd.DataFrame(index=np.arange(0, n_complete_matches*3), columns=['Latitude','Longitude'])\n",
    "    count = 0\n",
    "    for ii in range(dataframe.shape[0]):\n",
    "        if dataframe['home_stadium'].loc[ii]!= 'Unknown' and dataframe['visitor_stadium'].loc[ii]!= 'Unknown':\n",
    "            df.loc[count] = [dataframe['home_latitude'].loc[ii], dataframe['home_longitude'].loc[ii]]\n",
    "            df.loc[count+1] = [dataframe['visitor_latitude'].loc[ii], dataframe['visitor_longitude'].loc[ii]]\n",
    "            df.loc[count+2] = [np.NaN, np.NaN]\n",
    "            count += 3\n",
    "    return df\n",
    "df_agg = aggregate_dataframe_coordinates(df_match_stadium)\n",
    "df_agg.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have to transform the latitude and longitude coordinates to [web mercator](https://en.wikipedia.org/wiki/Web_Mercator) format in order to be able to represent it in a map using bokeh. Mercator coordinates are a cilindrical projection of the World coordinates. It was invented in 1569 by [Gerardus Mercator](https://en.wikipedia.org/wiki/Mercator_projection) and became the standard format for nautical purposes. The web mercator format is an adaptation of the original mercator format and it is currently used by most modern map systems such as Google Maps, Bing Maps or OpenStreetMaps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Latitude</th>\n",
       "      <th>Longitude</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-255076.256689</td>\n",
       "      <td>7.069114e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>829535.256912</td>\n",
       "      <td>6.708890e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-479704.406257</td>\n",
       "      <td>7.529248e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>808041.021713</td>\n",
       "      <td>5.422726e+06</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Latitude     Longitude\n",
       "0 -255076.256689  7.069114e+06\n",
       "1  829535.256912  6.708890e+06\n",
       "2            NaN           NaN\n",
       "3 -479704.406257  7.529248e+06\n",
       "4  808041.021713  5.422726e+06"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def to_web_mercator(yLat, xLon):\n",
    "    # Check if coordinate out of range for Latitude/Longitude\n",
    "    if (abs(xLon) > 180) and (abs(yLat) > 90):  \n",
    "        return\n",
    " \n",
    "    semimajorAxis = 6378137.0  # WGS84 spheriod semimajor axis\n",
    "    east = xLon * 0.017453292519943295\n",
    "    north = yLat * 0.017453292519943295\n",
    " \n",
    "    northing = 3189068.5 * math.log((1.0 + math.sin(north)) / (1.0 - math.sin(north)))\n",
    "    easting = semimajorAxis * east\n",
    " \n",
    "    return [easting, northing]\n",
    "df_agg_mercator = df_agg.apply(lambda row: to_web_mercator(row['Latitude'], row['Longitude']), axis=1)\n",
    "df_agg_mercator.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next step is to plot the trayectories in the map using datashader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "plot_width  = 850\n",
    "plot_height = 600\n",
    "x_range = (-1.9e6, 5.9e6)\n",
    "y_range = (3.7e6, 9.0e6)\n",
    "def create_image(x_range=x_range, y_range=y_range, w=plot_width, h=plot_height, cmap=None):\n",
    "    cvs = ds.Canvas(plot_width=w, plot_height=h, x_range=x_range, y_range=y_range)\n",
    "    agg = cvs.line(df_agg_mercator, 'Latitude', 'Longitude',  ds.count())\n",
    "    #img = tf.shade(agg, cmap=reversed(Blues3), how='eq_hist')\n",
    "    #img = tf.shade(agg, cmap=reversed(Greens3), how='eq_hist')    \n",
    "    img = tf.shade(agg, cmap=cmap, how='eq_hist')\n",
    "    return img\n",
    "\n",
    "def base_plot(tools='pan,wheel_zoom,reset',plot_width=plot_width, plot_height=plot_height,**plot_args):\n",
    "    p = figure(tools=tools, plot_width=plot_width, plot_height=plot_height,\n",
    "        x_range=x_range, y_range=y_range, outline_line_color=None,\n",
    "        min_border=0, min_border_left=0, min_border_right=0,\n",
    "        min_border_top=0, min_border_bottom=0, **plot_args)\n",
    "    \n",
    "    p.axis.visible = False\n",
    "    p.xgrid.grid_line_color = None\n",
    "    p.ygrid.grid_line_color = None\n",
    "    \n",
    "    return p"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<bokeh.models.renderers.TileRenderer at 0x7fd69392f5f8>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ArcGIS=WMTSTileSource(url='http://server.arcgisonline.com/ArcGIS/rest/services/World_Street_Map/MapServer/tile/{Z}/{Y}/{X}.png')\n",
    "p = base_plot()\n",
    "p.add_tile(ArcGIS)\n",
    "#InteractiveImage(p, create_image, cmap=inferno)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function `InteractiveImage` is not rendered in github. I made a snapshot and show it here:\n",
    "<p align=\"center\">\n",
    "<img src=\"map.JPG\" alt=\"Matches between teams in the Champions League\" width=\"60%\"/>\n",
    "</p>\n",
    "\n",
    "This is the first MVP which provides a final product. It is imperfect, it doesn't have all the teams in Europe and some of the matches are wronly represented. However, we got some data, transform it and plot it in a map."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### MVP2: Improving the map to show all teams in Europe\n",
    "\n",
    "Now that we have the map, we can start to improve it. If you are into football, you will notice that there are several points in the north of Spain, that corresponds to Sporting de Gijon. Sadly for Sporting supporters, they have never reached to the Champions. Instead, Sporting Clube de Portugal has participated several times in the championship, but since the current dataset doesn't have teams from Portugal, the system mistakenly thinks that `Sporting CP` from `champions.csv` is the Sporting de Gijon from `stadiums.csv`. So lets fix this issue by getting the stadiums coordinates from the rest of the countries in Europe.  \n",
    "We can get that info from [wikidata](https://query.wikidata.org/). Using `SPARQL` language we can get the information we need:\n",
    "```SQL\n",
    "SELECT ?clubLabel ?venueLabel ?coordinates ?countryLabel  WHERE {\n",
    "  ?club wdt:P31 wd:Q476028.\n",
    "  ?club wdt:P115 ?venue.\n",
    "  ?venue wdt:P625 ?coordinates.\n",
    "  ?club wdt:P17 ?country.\n",
    "  SERVICE wikibase:label { bd:serviceParam wikibase:language \"en\". }\n",
    "}\n",
    "ORDER BY ?clubLabel\n",
    "```\n",
    "This generates 4435 Results in 10111 ms that can be saved to a csv file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>clubLabel</th>\n",
       "      <th>venueLabel</th>\n",
       "      <th>coordinates</th>\n",
       "      <th>countryLabel</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4430</th>\n",
       "      <td>ŠK SFM Senec</td>\n",
       "      <td>NTC Senec</td>\n",
       "      <td>Point(17.410412 48.217703)</td>\n",
       "      <td>Slovakia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4431</th>\n",
       "      <td>ŠK Slovan Bratislava</td>\n",
       "      <td>Štadión Pasienky</td>\n",
       "      <td>Point(17.141111 48.166111)</td>\n",
       "      <td>Slovakia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4432</th>\n",
       "      <td>Żurrieq F.C.</td>\n",
       "      <td>Ta' Qali National Stadium</td>\n",
       "      <td>Point(14.415372222 35.894863888)</td>\n",
       "      <td>Malta</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4433</th>\n",
       "      <td>ŽFK Crvena zvezda</td>\n",
       "      <td>Red Star Stadium</td>\n",
       "      <td>Point(20.464914 44.7832)</td>\n",
       "      <td>Serbia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4434</th>\n",
       "      <td>ŽP Šport Podbrezová</td>\n",
       "      <td>Stadium Kolkáreň</td>\n",
       "      <td>Point(19.535 48.8167)</td>\n",
       "      <td>Slovakia</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 clubLabel                 venueLabel  \\\n",
       "4430          ŠK SFM Senec                  NTC Senec   \n",
       "4431  ŠK Slovan Bratislava           Štadión Pasienky   \n",
       "4432          Żurrieq F.C.  Ta' Qali National Stadium   \n",
       "4433     ŽFK Crvena zvezda           Red Star Stadium   \n",
       "4434   ŽP Šport Podbrezová           Stadium Kolkáreň   \n",
       "\n",
       "                           coordinates countryLabel  \n",
       "4430        Point(17.410412 48.217703)     Slovakia  \n",
       "4431        Point(17.141111 48.166111)     Slovakia  \n",
       "4432  Point(14.415372222 35.894863888)        Malta  \n",
       "4433          Point(20.464914 44.7832)       Serbia  \n",
       "4434             Point(19.535 48.8167)     Slovakia  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_stadium_read = pd.read_csv('stadiums_wikidata.csv', usecols=['clubLabel','venueLabel','coordinates','countryLabel'])\n",
    "df_stadium_read.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first step is to clean the column coordinates. For that we will use a regex pattern. The pattern `[-+]?[0-9]*\\.?[0-9]+` finds any signed float in a string. Then we create two patterns separated by a space and name the columns using this format: `(?P<Longitude>)`. Finally we have to concatente the club information with the coordinates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of rows: 4435\n",
      "Unique team's name number: 4314\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Team</th>\n",
       "      <th>Stadium</th>\n",
       "      <th>Longitude</th>\n",
       "      <th>Latitude</th>\n",
       "      <th>Country</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3388</th>\n",
       "      <td>Real Madrid</td>\n",
       "      <td>Juan Ramón Loubriel Stadium</td>\n",
       "      <td>-66.150833</td>\n",
       "      <td>18.393333</td>\n",
       "      <td>United States of America</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3389</th>\n",
       "      <td>Real Madrid</td>\n",
       "      <td>Galeshewe Stadium</td>\n",
       "      <td>24.740857</td>\n",
       "      <td>-28.716786</td>\n",
       "      <td>South Africa</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3390</th>\n",
       "      <td>Real Madrid C</td>\n",
       "      <td>Ciudad Real Madrid</td>\n",
       "      <td>-3.61166667</td>\n",
       "      <td>40.47916667</td>\n",
       "      <td>Spain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3391</th>\n",
       "      <td>Real Madrid C.F.</td>\n",
       "      <td>Ciudad Real Madrid</td>\n",
       "      <td>-3.61166667</td>\n",
       "      <td>40.47916667</td>\n",
       "      <td>Spain</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3392</th>\n",
       "      <td>Real Madrid FC</td>\n",
       "      <td>Estadio Santiago Bernabéu</td>\n",
       "      <td>-3.68835</td>\n",
       "      <td>40.45306</td>\n",
       "      <td>Spain</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  Team                      Stadium    Longitude     Latitude  \\\n",
       "3388       Real Madrid  Juan Ramón Loubriel Stadium   -66.150833    18.393333   \n",
       "3389       Real Madrid            Galeshewe Stadium    24.740857   -28.716786   \n",
       "3390     Real Madrid C           Ciudad Real Madrid  -3.61166667  40.47916667   \n",
       "3391  Real Madrid C.F.           Ciudad Real Madrid  -3.61166667  40.47916667   \n",
       "3392    Real Madrid FC    Estadio Santiago Bernabéu     -3.68835     40.45306   \n",
       "\n",
       "                       Country  \n",
       "3388  United States of America  \n",
       "3389              South Africa  \n",
       "3390                     Spain  \n",
       "3391                     Spain  \n",
       "3392                     Spain  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_temp = df_stadium_read['coordinates'].str.extract('(?P<Longitude>[-+]?[0-9]*\\.?[0-9]+) (?P<Latitude>[-+]?[0-9]*\\.?[0-9]+)', expand=True)\n",
    "df_stadium_new = pd.concat([df_stadium_read['clubLabel'],df_stadium_read['venueLabel'], df_temp, df_stadium_read['countryLabel']], axis=1) \n",
    "df_stadium_new = df_stadium_new.rename(columns = {'clubLabel':'Team', 'venueLabel':'Stadium','countryLabel':'Country'})\n",
    "print(\"Number of rows: %d\" % df_stadium_new.shape[0])\n",
    "unique_teams_stadium = list(set(df_stadium_new['Team']))\n",
    "print(\"Unique team's name number: %d\" % len(unique_teams_stadium))\n",
    "df_stadium_new.take(list(range(3388,3393)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As it can be seen in the previous dataframe, we came into another problem. The new dataset contains all team instances, all over the world. There are some teams that have the same name in different countries, there is a Real Madrid team from USA and South Africa, and a similar name within a country. In our case, we are only interested in the instance `Real Madrid, Estadio Santiago Bernabeu, -3.68835, 40.45306, Spain`. So how can we find an automated way to filter the correct teams that have participated in the Champions League?\n",
    "\n",
    "A practical approach is to combine an automated and manual way. With the data we have so far we can automatically filter the two first entries using the country. We can get the country info from `champions.csv` dataset. To distinguish teams from the same country we will filter them manually. \n",
    "\n",
    "The first step then is to get a dataframe with all the teams that have participated in Champions and their country of origin. Then we have to remove the repeated entries and rename the country code to the country name that can be found in wikidata. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of unique teams: 502\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Team</th>\n",
       "      <th>Country</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>AC Chievo Verona</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>AC Milan</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>AC Sparta Praha</td>\n",
       "      <td>Czech Republic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ACF Fiorentina</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>AE Larisa</td>\n",
       "      <td>Greece</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               Team         Country\n",
       "0  AC Chievo Verona           Italy\n",
       "1          AC Milan           Italy\n",
       "2   AC Sparta Praha  Czech Republic\n",
       "3    ACF Fiorentina           Italy\n",
       "4         AE Larisa          Greece"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_match_home = df_match[['home','home_country']]\n",
    "df_match_home = df_match_home.rename(columns={'home':'Team','home_country':'Country'})\n",
    "df_match_visitor = df_match[['visitor','visitor_country']]\n",
    "df_match_visitor = df_match_visitor.rename(columns={'visitor':'Team','visitor_country':'Country'})\n",
    "df_champions_teams = pd.concat([df_match_home,df_match_visitor], axis=0, ignore_index=True)\n",
    "df_champions_teams = df_champions_teams.drop_duplicates()\n",
    "print(\"Number of unique teams: %d\" % df_champions_teams.shape[0])\n",
    "country_dict = {'ALB':'Albania',\n",
    "                'AND':'Andorra',\n",
    "                'ARM':'Armenia',\n",
    "                'AUT':'Austria',\n",
    "                'AZE':'Azerbaijan',\n",
    "                'BEL':'Belgium',\n",
    "                'BIH':'Bosnia and Herzegovina',\n",
    "                'BLR':'Belarus',\n",
    "                'BUL':'Bulgaria',\n",
    "                'CRO':'Croatia',\n",
    "                'CYP':'Cyprus',\n",
    "                'CZE':'Czech Republic',\n",
    "                'DEN':'Denmark',\n",
    "                #'ENG':'England',\n",
    "                'ENG':'United Kingdom',\n",
    "                'ESP':'Spain',\n",
    "                'EST':'Estonia',\n",
    "                'FIN':'Finland',\n",
    "                'FRA':'France',\n",
    "                'FRO':'Feroe Islands',\n",
    "                'GEO':'Georgia',\n",
    "                'GER':'Germany',\n",
    "                'GIB':'Gibraltar',\n",
    "                'GRE':'Greece',\n",
    "                'HUN':'Hungary',\n",
    "                'ITA':'Italy',\n",
    "                'IRL':'Ireland',\n",
    "                'ISL':'Iceland',\n",
    "                'ISR':'Israel',\n",
    "                'KAZ':'Kazakhstan',\n",
    "                'LTU':'Lithuania',\n",
    "                'LUX':'Luxembourg',\n",
    "                'LVA':'Latvia',\n",
    "                'MDA':'Moldova',\n",
    "                'MKD':'Macedonia',\n",
    "                'MLT':'Malta',\n",
    "                'MNE':'Montenegro',\n",
    "                'NED':'Netherlands',\n",
    "                #'NIR':'Northern Ireland',\n",
    "                'NIR':'United Kingdom',\n",
    "                'NOR':'Norwey',\n",
    "                'POL':'Poland',\n",
    "                'POR':'Portugal',\n",
    "                'ROU':'Romania',\n",
    "                'RUS':'Russia',\n",
    "                #'SCO':'Scotland',\n",
    "                'SCO':'United Kingdom',\n",
    "                'SMR':'San Marino',\n",
    "                'SRB':'Serbia',\n",
    "                'SUI':'Switzerland',\n",
    "                'SVK':'Slovakia',\n",
    "                'SVN':'Slovenia',\n",
    "                'SWE':'Sweden',\n",
    "                'TUR':'Turkey',\n",
    "                'UKR':'Ukrania',\n",
    "                #'WAL':'Wales',\n",
    "                'WAL':'United Kingdom'}\n",
    "df_champions_teams['Country'].replace(country_dict, inplace=True)\n",
    "#df_champions_teams.to_csv('match_unique.csv')# To check that the mapping is correct\n",
    "df_champions_teams.sort_values(by='Team',inplace=True)\n",
    "df_champions_teams = df_champions_teams.reset_index(drop=True)\n",
    "df_champions_teams.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we have the list of all teams that have participated in the Champions League, we have to generate a new dataset relating each Champions League matches with the coordinates of the team stadiums. For that we will use the function `similar` to match a the name of the team in the different datasets similarly as we did before. \n",
    "\n",
    "Once the csv has been generated, let's manually erase the combinations that are not correct and save everything in a new file. We won't correct those entries that are not matched, a Data Science project is better out than perfect!! "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 2min 9s, sys: 240 ms, total: 2min 9s\n",
      "Wall time: 2min 9s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "def get_info_similar_team_country(team, country, df_stadium, df, threshold, verbose):\n",
    "    team2 = \"Unknown\"\n",
    "    stadium = \"Unknown\"\n",
    "    latitude = np.NaN\n",
    "    longitude = np.NaN\n",
    "    cols = list(df)\n",
    "    for idx, val in enumerate(df_stadium['Team']):\n",
    "        rank = similar(team, val)\n",
    "        if rank > threshold and country == df_stadium['Country'].iloc[idx]:\n",
    "            if(verbose): print(\"%s and %s(Idx=%d) are %f similar and from the same country %s.\" \n",
    "                               % (team, val, idx, rank, country))\n",
    "            team2 = df_stadium['Team'].iloc[idx]\n",
    "            stadium = df_stadium['Stadium'].iloc[idx]\n",
    "            latitude = df_stadium['Latitude'].iloc[idx]\n",
    "            longitude = df_stadium['Longitude'].iloc[idx]\n",
    "            dtemp = pd.DataFrame([[team, team2, stadium, latitude, longitude, country]], columns=cols)\n",
    "            df = df.append(dtemp, ignore_index=True)\n",
    "    #if there is no match, register it\n",
    "    if(team2 == \"Unknown\"):\n",
    "        df_nomatch = pd.DataFrame([[team, team2, stadium, latitude, longitude, country]], columns=cols)\n",
    "        df = df.append(df_nomatch, ignore_index=True)\n",
    "    return df\n",
    "\n",
    "def generate_new_stadium_dataset(df_champions_teams, df_stadium_new, threshold=0.6, verbose=False):\n",
    "    df = pd.DataFrame(columns=['Team', 'Team2', 'Stadium', 'Latitude','Longitude','Country'])\n",
    "    for idx, row in df_champions_teams.iterrows():\n",
    "        df = get_info_similar_team_country(row['Team'],row['Country'], df_stadium_new, df, \n",
    "                                           threshold=threshold, verbose=verbose)\n",
    "    return df\n",
    "\n",
    "verbose = False # You can change this to True to see all the combinations\n",
    "threshold = 0.5\n",
    "df_stadiums_champions = generate_new_stadium_dataset(df_champions_teams, df_stadium_new, threshold, verbose)\n",
    "df_stadiums_champions.to_csv('stadiums_champions.csv', index=False)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After we filtered the entries in the csv, let's load again the data and repeat the process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Team</th>\n",
       "      <th>Stadium</th>\n",
       "      <th>Latitude</th>\n",
       "      <th>Longitude</th>\n",
       "      <th>Country</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>AC Chievo Verona</td>\n",
       "      <td>Stadio Marc'Antonio Bentegodi</td>\n",
       "      <td>45.435356</td>\n",
       "      <td>10.968647</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>AC Milan</td>\n",
       "      <td>Stadio Giuseppe Meazza</td>\n",
       "      <td>45.478080</td>\n",
       "      <td>9.124000</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>AC Sparta Praha</td>\n",
       "      <td>Generali Arena</td>\n",
       "      <td>50.099716</td>\n",
       "      <td>14.416122</td>\n",
       "      <td>Czech Republic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ACF Fiorentina</td>\n",
       "      <td>Stadio Artemio Franchi</td>\n",
       "      <td>43.780822</td>\n",
       "      <td>11.282258</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>AE Larisa</td>\n",
       "      <td>AEL FC Arena</td>\n",
       "      <td>39.615610</td>\n",
       "      <td>22.399490</td>\n",
       "      <td>Greece</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               Team                        Stadium   Latitude  Longitude  \\\n",
       "0  AC Chievo Verona  Stadio Marc'Antonio Bentegodi  45.435356  10.968647   \n",
       "1          AC Milan         Stadio Giuseppe Meazza  45.478080   9.124000   \n",
       "2   AC Sparta Praha                 Generali Arena  50.099716  14.416122   \n",
       "3    ACF Fiorentina         Stadio Artemio Franchi  43.780822  11.282258   \n",
       "4         AE Larisa                   AEL FC Arena  39.615610  22.399490   \n",
       "\n",
       "          Country  \n",
       "0           Italy  \n",
       "1           Italy  \n",
       "2  Czech Republic  \n",
       "3           Italy  \n",
       "4          Greece  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_stadiums_champions = pd.read_csv('stadiums_champions_filtered.csv', usecols=['Team','Stadium','Latitude','Longitude','Country'])\n",
    "df_stadiums_champions.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As previously, we create a dataframe that relates each match with the coordinates of its stadium "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of matches with complete data: 5110 out of 6554\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Date</th>\n",
       "      <th>home</th>\n",
       "      <th>visitor</th>\n",
       "      <th>home_country</th>\n",
       "      <th>visitor_country</th>\n",
       "      <th>home_stadium</th>\n",
       "      <th>home_latitude</th>\n",
       "      <th>home_longitude</th>\n",
       "      <th>visitor_stadium</th>\n",
       "      <th>visitor_latitude</th>\n",
       "      <th>visitor_longitude</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1955-09-04</td>\n",
       "      <td>Sporting CP</td>\n",
       "      <td>Partizan Belgrade</td>\n",
       "      <td>POR</td>\n",
       "      <td>SRB</td>\n",
       "      <td>Estádio José Alvalade</td>\n",
       "      <td>38.761194</td>\n",
       "      <td>-9.160783</td>\n",
       "      <td>Partizan Stadium</td>\n",
       "      <td>44.788611</td>\n",
       "      <td>20.459167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1955-09-07</td>\n",
       "      <td>Budapesti Voros Lobogo</td>\n",
       "      <td>RSC Anderlecht</td>\n",
       "      <td>HUN</td>\n",
       "      <td>BEL</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Constant Vanden Stock Stadium</td>\n",
       "      <td>50.834167</td>\n",
       "      <td>4.298333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1955-09-08</td>\n",
       "      <td>Servette Geneve</td>\n",
       "      <td>Real Madrid</td>\n",
       "      <td>SUI</td>\n",
       "      <td>ESP</td>\n",
       "      <td>Stade de Genève</td>\n",
       "      <td>46.177778</td>\n",
       "      <td>6.127500</td>\n",
       "      <td>Estadio Santiago Bernabéu</td>\n",
       "      <td>40.453060</td>\n",
       "      <td>-3.688350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1955-09-14</td>\n",
       "      <td>Rot-Weiss Essen</td>\n",
       "      <td>Hibernian FC</td>\n",
       "      <td>GER</td>\n",
       "      <td>SCO</td>\n",
       "      <td>Stadion Essen</td>\n",
       "      <td>51.486667</td>\n",
       "      <td>6.976389</td>\n",
       "      <td>Easter Road</td>\n",
       "      <td>55.961667</td>\n",
       "      <td>-3.165556</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1955-09-20</td>\n",
       "      <td>Djurgardens IF</td>\n",
       "      <td>Gwardia Warszawa</td>\n",
       "      <td>SWE</td>\n",
       "      <td>POL</td>\n",
       "      <td>Stockholm Olympic Stadium</td>\n",
       "      <td>59.345278</td>\n",
       "      <td>18.078889</td>\n",
       "      <td>Stadion Polonii Warszawa</td>\n",
       "      <td>52.255517</td>\n",
       "      <td>21.001322</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Date                    home            visitor home_country  \\\n",
       "0  1955-09-04             Sporting CP  Partizan Belgrade          POR   \n",
       "1  1955-09-07  Budapesti Voros Lobogo     RSC Anderlecht          HUN   \n",
       "2  1955-09-08         Servette Geneve        Real Madrid          SUI   \n",
       "3  1955-09-14         Rot-Weiss Essen       Hibernian FC          GER   \n",
       "4  1955-09-20          Djurgardens IF   Gwardia Warszawa          SWE   \n",
       "\n",
       "  visitor_country               home_stadium  home_latitude  home_longitude  \\\n",
       "0             SRB      Estádio José Alvalade      38.761194       -9.160783   \n",
       "1             BEL                    Unknown            NaN             NaN   \n",
       "2             ESP            Stade de Genève      46.177778        6.127500   \n",
       "3             SCO              Stadion Essen      51.486667        6.976389   \n",
       "4             POL  Stockholm Olympic Stadium      59.345278       18.078889   \n",
       "\n",
       "                 visitor_stadium  visitor_latitude  visitor_longitude  \n",
       "0               Partizan Stadium         44.788611          20.459167  \n",
       "1  Constant Vanden Stock Stadium         50.834167           4.298333  \n",
       "2      Estadio Santiago Bernabéu         40.453060          -3.688350  \n",
       "3                    Easter Road         55.961667          -3.165556  \n",
       "4       Stadion Polonii Warszawa         52.255517          21.001322  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_match_stadium_new= df_match\n",
    "home_stadium_index = df_match_stadium_new['home'].map(lambda x: get_info_similar_team(x, df_stadiums_champions))\n",
    "visitor_stadium_index = df_match_stadium_new['visitor'].map(lambda x: get_info_similar_team(x, df_stadiums_champions))\n",
    "df_home = pd.DataFrame(home_stadium_index.tolist(), columns=['home_stadium', 'home_latitude', 'home_longitude'])\n",
    "df_visitor = pd.DataFrame(visitor_stadium_index.tolist(), columns=['visitor_stadium', 'visitor_latitude', 'visitor_longitude'])\n",
    "df_match_stadium_new = pd.concat([df_match_stadium_new, df_home, df_visitor], axis=1, ignore_index=False)\n",
    "df1 = df_match_stadium_new['home_stadium'] == 'Unknown'\n",
    "df2 = df_match_stadium_new['visitor_stadium'] == 'Unknown'\n",
    "n_complete_matches = df_match_stadium_new.shape[0] - df_match_stadium_new[df1 | df2].shape[0]\n",
    "print(\"Number of matches with complete data: %d out of %d\" % (n_complete_matches, df_match_stadium_new.shape[0]))\n",
    "df_match_stadium_new.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next step is to aggregate the coordinates and transform them to mercator format. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of rows: 15330\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Latitude</th>\n",
       "      <th>Longitude</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-1.019774e+06</td>\n",
       "      <td>4.687522e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2.277504e+06</td>\n",
       "      <td>5.588304e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>6.821102e+05</td>\n",
       "      <td>5.808884e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-4.105852e+05</td>\n",
       "      <td>4.932000e+06</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Latitude     Longitude\n",
       "0 -1.019774e+06  4.687522e+06\n",
       "1  2.277504e+06  5.588304e+06\n",
       "2           NaN           NaN\n",
       "3  6.821102e+05  5.808884e+06\n",
       "4 -4.105852e+05  4.932000e+06"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_agg = aggregate_dataframe_coordinates(df_match_stadium_new)\n",
    "df_agg_mercator = df_agg.apply(lambda row: to_web_mercator(row['Latitude'], row['Longitude']), axis=1)\n",
    "print(\"Number of rows: %d\" % df_agg_mercator.shape[0])\n",
    "df_agg_mercator.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we plot the coordinates in the map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 0 ns, sys: 0 ns, total: 0 ns\n",
      "Wall time: 7.63 µs\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "#InteractiveImage(p, create_image, cmap=inferno)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, the image for github:\n",
    "<p align=\"center\">\n",
    "<img src=\"map2.JPG\" alt=\"Matches between teams in the Champions League\" width=\"60%\"/>\n",
    "</p>\n",
    "\n",
    "This is the second MVP. In this case we have the correct connections between teams in Europe. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### MVP3: improving design\n",
    "\n",
    "If you take a look at the map you will notice that it is fairly ugly. The final quality of a product will drop if we don't show a good visualization. The next step is to improve the map changing the background and colors.  \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "ArcGIS2 = WMTSTileSource(url='http://tile.stamen.com/toner-background/{Z}/{X}/{Y}.png')\n",
    "p2 = base_plot()\n",
    "p2.add_tile(ArcGIS2)\n",
    "cmap = reversed(Blues3)\n",
    "#cmap = reversed(Greens3)\n",
    "#InteractiveImage(p2, create_image, cmap=cmap)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, the image for github:\n",
    "<p align=\"center\">\n",
    "<img src=\"map3.JPG\" alt=\"Matches between teams in the Champions League\" width=\"40%\"/>\n",
    "</p>\n",
    "\n",
    "We developed three different MVPs iterating on the project. The Lean Startup method provides a very easy way to remove uncertainty and get a minimum product very quickly that we can show to the customer. \n",
    "\n",
    "Happy data sciencing!"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}