covid-ontario.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Visualizing COVID-19 cases in Ontario\n",
    "By Kendra Oudyk\n",
    "\n",
    "### How to run this code\n",
    "<font color='red'>In the above ribbon, click **cell** and then click **Run All**</font> "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd \n",
    "import numpy as np\n",
    "import datetime\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib\n",
    "from matplotlib.widgets import CheckButtons\n",
    "import requests\n",
    "from io import StringIO \n",
    "from pandas.plotting import register_matplotlib_converters\n",
    "register_matplotlib_converters() \n",
    "\n",
    "%matplotlib inline\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the data from the data.ontario.ca website\n",
    "Read it into a pandas dataframe, which is like a table)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "data_url = 'https://data.ontario.ca/dataset/f4112442-bdc8-45d2-be3c-12efae72fb27/resource/455fd63b-603d-4608-8216-7d8647f43350/download/conposcovidloc.csv'\n",
    "response = requests.get(data_url)\n",
    "csv_string = response.content.decode('utf-8')\n",
    "cases = pd.read_csv(StringIO(csv_string))\n",
    "cases.head(4) # see the first few columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define the population of each phu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [],
   "source": [
    "phu_populations =   {'Algoma Public Health Unit': 114434,\n",
    "                     'Brant County Health Unit': 155203,\n",
    "                     'Chatham-Kent Health Unit': 106317,\n",
    "                     'Durham Region Health Department': 712402,\n",
    "                     'Eastern Ontario Health Unit': 208711,\n",
    "                     'Grey Bruce Health Unit': 169884,\n",
    "                     'Haldimand-Norfolk Health Unit': 114081,\n",
    "                     'Haliburton, Kawartha, Pine Ridge District Health Unit': 188937,\n",
    "                     'Halton Region Health Department': 619087,\n",
    "                     'Hamilton Public Health Services': 592163,\n",
    "                     'Hastings and Prince Edward Counties Health Unit': 168493,\n",
    "                     'Huron Perth District Health Unit': 139757,\n",
    "                     'Kingston, Frontenac and Lennox & Addington Public Health': 212719,\n",
    "                     'Lambton Public Health': 130964,\n",
    "                     'Leeds, Grenville and Lanark District Health Unit': 173170,\n",
    "                     'Middlesex-London Health Unit': 507524,\n",
    "                     'Niagara Region Public Health Department': 472485,\n",
    "                     'North Bay Parry Sound District Health Unit': 129752,\n",
    "                     'Northwestern Health Unit': 87675,\n",
    "                     'Ottawa Public Health': 1054656,\n",
    "                     'Peel Public Health': 1605952,\n",
    "                     'Peterborough Public Health': 147977,\n",
    "                     'Porcupine Health Unit': 83441,\n",
    "                     'Region of Waterloo, Public Health': 584361,\n",
    "                     'Renfrew County and District Health Unit': 108631,\n",
    "                     'Simcoe Muskoka District Health Unit': 599589,\n",
    "                     'Southwestern Public Health': 211498,\n",
    "                     'Sudbury & District Health Unit': 199023,\n",
    "                     'Thunder Bay District Health Unit': 149960,\n",
    "                     'Timiskaming Health Unit': 32689,\n",
    "                     'Toronto Public Health': 3120358,\n",
    "                     'Wellington-Dufferin-Guelph Public Health': 311908,\n",
    "                     'Windsor-Essex County Health Unit': 424830,\n",
    "                     'York Region Public Health Services': 1225797}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Timeline\n",
    "### turn the dates into integers, which are easier to deal with\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_next_cell": 2,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "cases['Date'] = cases['Accurate_Episode_Date']\n",
    "cases = cases.dropna(subset=['Date'])\n",
    "cases['Date'] = [int(str(s).replace('-', '')) for s in cases['Date']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### get a list of all dates until today\n",
    "We want to begin at Feb 15, 2020, but we're looking at cases in 14-day periods, so we'll start the timeline 14 days before (inclusive of) Feb 15.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "d1 = datetime.date(2020,2,15)\n",
    "d2 = datetime.date.today()\n",
    "datetimes = [(d1 + datetime.timedelta(days=x)) for x in range((d2-d1).days + 1)]\n",
    "dates = [int(str(s).replace('-', '')) for s in datetimes]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get the relevant data \n",
    "I.e., get the data for each PHU and day from the *cases* dataframe, and store it in a new dataframe called *data*. *data* has a row for each day, and a column for each PHU\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# initialize dataframe\n",
    "data = pd.DataFrame()\n",
    "data['Date'] = dates\n",
    "\n",
    "days_in_range = 14\n",
    "overall_begin_date = 20200215 # february 15 2020\n",
    "phus = np.unique(cases['Reporting_PHU'])  \n",
    "for phu in phus:\n",
    "    population = int(phu_populations[phu])\n",
    "    \n",
    "    # select the cases for the given PHU\n",
    "    phu_cases = cases[cases['Reporting_PHU'] == phu]\n",
    "    \n",
    "    daily_rate = []\n",
    "    for n, date in enumerate(dates):\n",
    "        if date >= overall_begin_date: # if it's after Feb 15\n",
    "            # select the cases for the given date, in the given PHU\n",
    "            i_begin_date = n - days_in_range + 1 # add 1 because it's 14 days inclusive\n",
    "            begin_date = dates[i_begin_date]\n",
    "            end_date = date\n",
    "            phu_cases_in_current_dates = phu_cases[phu_cases['Date'].\\\n",
    "                                                   between(begin_date, end_date)]\n",
    "            \n",
    "\n",
    "            # calculate the infection rate\n",
    "            n_cases = len(phu_cases_in_current_dates)\n",
    "            rate = n_cases / population * 10000\n",
    "            daily_rate.append(rate)\n",
    "    data[phu] = daily_rate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Inspect the first 3 rows\n",
    "to see if the dates start as expected"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Inspect the last 3 rows\n",
    "to see if the dates end as expected"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.tail(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualize the timelines for all PHUs on the same interactive plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_next_cell": 0,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# This code was based on https://stackoverflow.com/questions/31410043/hiding-lines-after-showing-a-pyplot-figure/31417070\n",
    "\n",
    "%matplotlib notebook\n",
    "\n",
    "matplotlib.rc('font', size = 14)\n",
    "\n",
    "linewidth=4\n",
    "alpha=.1\n",
    "ymax = np.max(data.loc[:, data.columns != 'Date'].values) + 2\n",
    "ymin = - .2\n",
    "days = 14\n",
    "\n",
    "def main():\n",
    "    \n",
    "    NUM_COLORS = 34\n",
    "    LINE_STYLES = ['solid', 'dashed', 'dashdot', 'dotted']\n",
    "    COLORS = ['DarkRed', 'red', 'DarkKhaki', 'Lime', 'Green', 'DeepSkyBlue', 'Blue', 'Violet', 'black']\n",
    "    LINE_COLORS = []\n",
    "    for color in COLORS:\n",
    "        for style in LINE_STYLES:\n",
    "            LINE_COLORS.append(color)\n",
    "    NUM_STYLES = len(LINE_STYLES)\n",
    "    cm = plt.get_cmap('gist_rainbow')\n",
    "    cm = plt.get_cmap('nipy_spectral')\n",
    "    \n",
    "    x = datetimes\n",
    "            \n",
    "    fig, ax = plt.subplots(figsize=(14,10))\n",
    "    ax.set_xlim(datetimes[0], datetimes[-1])\n",
    "    ax.axhspan(ymin, 0.05*days, facecolor='blue', alpha=alpha)\n",
    "    ax.axhspan(0.05*days, 0.1*days, facecolor='green', alpha=alpha)\n",
    "    ax.axhspan(0.1*days, 0.25*days, facecolor='yellow', alpha=alpha)\n",
    "    ax.axhspan(0.25*days, 0.5*days, facecolor='darkorange', alpha=alpha)\n",
    "    ax.axhspan(0.5*days, ymax, facecolor='red', alpha=alpha)\n",
    "    \n",
    "    for i, phu in enumerate(phus[0:]):\n",
    "        y = data[phu]\n",
    "        #color = cm(i//NUM_STYLES*float(NUM_STYLES)/NUM_COLORS)\n",
    "        color = LINE_COLORS[i]\n",
    "        linestyle = LINE_STYLES[i%NUM_STYLES]\n",
    "        ax.plot(x, y, label=phu, color=color, linestyle=linestyle)\n",
    "        #ax.plot(x, y, label=phu) # default color scheme\n",
    "\n",
    "    ax.legend(loc='best', bbox_to_anchor=(0.01, 1),\n",
    "              ncol=2, borderaxespad=0, title=' ',#title='Regional public health unit',\n",
    "              frameon=False)\n",
    "    ax.spines['right'].set_visible(False)\n",
    "    ax.spines['top'].set_visible(False)\n",
    "    ax.set_ylim(ymin, ymax)\n",
    "    ax.set_ylabel('infection rate in the last 14 days\\nper 10,000 people\\n(case_count / population * 10,000)')\n",
    "    ax.set_xlabel('Date')\n",
    "\n",
    "    fig.subplots_adjust(top=0.4)\n",
    "    fig.suptitle('Left-click on a PHU name to show/hide it   |   Right-click to hide all   |   Middle-click to show all',\n",
    "                 va='top', size='large', y=.99)\n",
    "\n",
    "    leg = interactive_legend()\n",
    "    def line_hover(event):\n",
    "        for line in ax.get_lines():\n",
    "            if line.contains(event)[0]:\n",
    "                print(line.get_label())\n",
    "    fig.canvas.mpl_connect('motion_notify_event', line_hover) \n",
    "    return fig, ax, leg\n",
    "\n",
    "def interactive_legend(ax=None):\n",
    "    if ax is None:\n",
    "        ax = plt.gca()\n",
    "    if ax.legend_ is None:\n",
    "        ax.legend()\n",
    "\n",
    "    return InteractiveLegend(ax.get_legend())\n",
    "\n",
    "class InteractiveLegend(object):\n",
    "    def __init__(self, legend):\n",
    "        self.legend = legend\n",
    "        self.fig = legend.axes.figure\n",
    "\n",
    "        self.lookup_artist, self.lookup_handle = self._build_lookups(legend)\n",
    "        self._setup_connections()\n",
    "\n",
    "        self.update()\n",
    "\n",
    "    def _setup_connections(self):\n",
    "        for artist in self.legend.texts + self.legend.legendHandles:\n",
    "            artist.set_picker(10) # 10 points tolerance\n",
    "\n",
    "        self.fig.canvas.mpl_connect('pick_event', self.on_pick)\n",
    "        self.fig.canvas.mpl_connect('button_press_event', self.on_click)\n",
    "\n",
    "    def _build_lookups(self, legend):\n",
    "        labels = [t.get_text() for t in legend.texts]\n",
    "        handles = legend.legendHandles\n",
    "        label2handle = dict(zip(labels, handles))\n",
    "        handle2text = dict(zip(handles, legend.texts))\n",
    "\n",
    "        lookup_artist = {}\n",
    "        lookup_handle = {}\n",
    "        for artist in legend.axes.get_children():\n",
    "            if artist.get_label() in labels:\n",
    "                handle = label2handle[artist.get_label()]\n",
    "                lookup_handle[artist] = handle\n",
    "                lookup_artist[handle] = artist\n",
    "                lookup_artist[handle2text[handle]] = artist\n",
    "\n",
    "        lookup_handle.update(zip(handles, handles))\n",
    "        lookup_handle.update(zip(legend.texts, handles))\n",
    "\n",
    "        return lookup_artist, lookup_handle\n",
    "\n",
    "    def on_pick(self, event):\n",
    "        handle = event.artist\n",
    "        if handle in self.lookup_artist:\n",
    "\n",
    "            artist = self.lookup_artist[handle]\n",
    "            artist.set_visible(not artist.get_visible())\n",
    "            self.update()\n",
    "\n",
    "    def on_click(self, event):\n",
    "        if event.button == 3:\n",
    "            visible = False\n",
    "        elif event.button == 2:\n",
    "            visible = True\n",
    "        else:\n",
    "            return\n",
    "\n",
    "        for artist in self.lookup_artist.values():\n",
    "            artist.set_visible(visible)\n",
    "        self.update()\n",
    "\n",
    "    def update(self):\n",
    "        for artist in self.lookup_artist.values():\n",
    "            handle = self.lookup_handle[artist]\n",
    "            if artist.get_visible():\n",
    "                handle.set_visible(True)\n",
    "            else:\n",
    "                handle.set_visible(False)\n",
    "        self.fig.canvas.draw()\n",
    "\n",
    "    def show(self):\n",
    "        plt.show()\n",
    "        \n",
    "\n",
    "if __name__ == '__main__':\n",
    "    fig, ax, leg = main()\n",
    "    plt.tight_layout()\n",
    "    \n",
    "    plt.show()\n",
    "    \n"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "formats": "ipynb,py:light"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}