COVID-19 Reports Data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Intro"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "the dataset that will be used in this notebook is the [**Mapping 2019-nCoV**](https://systems.jhu.edu/research/public-health/ncov/) which contains several datasources (including the **WHO** and the **ECDC**) and it can be found [here](https://github.com/CSSEGISandData/COVID-19)\n",
    "\n",
    "Particually, we will use the **CSSE COVID-19 Dataset** folder, which contains two *types* of data:\n",
    "\n",
    "- Daily reports (csse_covid_19_daily_reports): <br>\n",
    "    this folder contains daily aggregated case reports, wich contains information such the number of confirmed cases, the number of deaths, and the number of recovered cases.\n",
    "    each report contains accumulated till the date of the report.\n",
    "    we can use the last report to show statistic measures such as total number of cases for each country, most affected countries, etc ..., but we can perform time analysis on the data.\n",
    "- Time series summary (csse_covid_19_time_series):"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Import libraries and load data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import glob\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "import plotly\n",
    "from plotly.offline import init_notebook_mode, iplot\n",
    "import plotly.express as px\n",
    "\n",
    "import plotly.graph_objects as go\n",
    "from plotly.graph_objects import Bar\n",
    "from plotly.subplots import make_subplots\n",
    "\n",
    "init_notebook_mode(connected=True)\n",
    "plotly.offline.init_notebook_mode(connected=True)\n",
    "\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "%matplotlib inline\n",
    "import seaborn as sns\n",
    "\n",
    "sns.set()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_last_report_date(reports_path):\n",
    "\n",
    "    # this is a buggy code, that just works!\n",
    "    # this function is used to get the date of the last report data in csse_covid_19_daily_reports folder\n",
    "\n",
    "    files_names_list = glob.glob(f\"{reports_path}/*.csv\")\n",
    "\n",
    "    files_names_list = sorted(files_names_list)\n",
    "\n",
    "    last_report_name = files_names_list[-1].split(\"/\")[-1]\n",
    "\n",
    "    last_report_date = last_report_name.split(\".\")[0]\n",
    "\n",
    "    return last_report_date"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "last_report_date = get_last_report_date(\n",
    "    \"COVID-19/csse_covid_19_data/csse_covid_19_daily_reports\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "last_report_date"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "last_report_path = (\n",
    "    f\"COVID-19/csse_covid_19_data/csse_covid_19_daily_reports/{last_report_date}.csv\"\n",
    ")\n",
    "world_pop_path = \"other-data/World Population Data/total_population_clean.csv\"\n",
    "pop_ages_1_path = \"other-data/World Population Data/population_ages_0-14_clean.csv\"\n",
    "pop_ages_2_path = \"other-data/World Population Data/population_ages_15-64_clean.csv\"\n",
    "pop_ages_3_path = \"other-data/World Population Data/population_ages_65_clean.csv\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(last_report_path)\n",
    "world_pop_df = pd.read_csv(world_pop_path)\n",
    "pop_ages_group_1_df = pd.read_csv(pop_ages_1_path)\n",
    "pop_ages_group_2_df = pd.read_csv(pop_ages_2_path)\n",
    "pop_ages_group_3_df = pd.read_csv(pop_ages_3_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Helper functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def save_fig_as_div(fig_obj, file_name):\n",
    "    with open(f\"charts/{file_name}\", \"w\") as fig_file:\n",
    "        fig_div_string = plotly.offline.plot(\n",
    "            figure_or_data=fig_obj, output_type=\"div\", include_plotlyjs=\"cdn\"\n",
    "        )\n",
    "        fig_file.write(fig_div_string)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_n_color_palette(palette_name, n_colors, as_hex=False):\n",
    "    palette = sns.color_palette(palette=palette_name, n_colors=n_colors)\n",
    "    if as_hex:\n",
    "        palette = palette.as_hex()\n",
    "    palette.reverse()\n",
    "    return palette"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_most_n_affected_countries(df, n):\n",
    "    most_affected_countries_names = df[\"Country\"][:n].values\n",
    "\n",
    "    most_affected_df = df[df[\"Country\"].isin(most_affected_countries_names)]\n",
    "\n",
    "    return most_affected_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_countries_fatalities(df):\n",
    "    countries_fatalities_df = pd.DataFrame(data=df[[\"Country\", \"Confirmed\", \"Deaths\"]])\n",
    "\n",
    "    countries_fatalities_df[\"Fatality rate\"] = (\n",
    "        countries_fatalities_df[\"Deaths\"] * 100.0\n",
    "    ) / countries_fatalities_df[\"Confirmed\"]\n",
    "\n",
    "    countries_fatalities_df = countries_fatalities_df.round({\"Fatality rate\": 2})\n",
    "\n",
    "    countries_fatalities_df.sort_values(by=\"Confirmed\", ascending=False, inplace=True)\n",
    "\n",
    "    return countries_fatalities_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Get to know the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(df[\"Country_Region\"].unique())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Field description\n",
    "- Province/State: China - province name; US/Canada/Australia/ - city name, state/province name; Others - name of the event (e.g., \"Diamond Princess\" cruise ship); other countries - blank.\n",
    "- Country/Region: country/region name conforming to WHO (will be updated).\n",
    "- Last Update: MM/DD/YYYY HH:mm (24 hour format, in UTC).\n",
    "- Confirmed: the number of confirmed cases.\n",
    "- Deaths: the number of deaths.\n",
    "- Recovered: the number of recovered cases.\n",
    "- Active: Active cases = total confirmed - total recovered - total deaths."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Populations Ages groups"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pop_ages_group_1_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pop_ages_group_2_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pop_ages_group_3_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Countries age structure statistics:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This data contains the percentages of age groups of population in each country, the age groups are:\n",
    "\n",
    "- Ages 0 to 14 years: children and adolescents.\n",
    "- Ages 15 to 64 years: working population or population in education\n",
    "- Over the age of 65: retirees; elderly\n",
    "\n",
    "will use this data later to study the relation between death rate and percentage of elderly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "countries_ages_groups_df = (\n",
    "    pop_ages_group_1_df.merge(\n",
    "        pop_ages_group_2_df,\n",
    "        left_on=\"Country\",\n",
    "        right_on=\"Country\",\n",
    "        how=\"inner\",\n",
    "        suffixes=(\"_group_1\", \"_group_2\"),\n",
    "    )\n",
    "    .merge(pop_ages_group_3_df, left_on=\"Country\", right_on=\"Country\", how=\"inner\")\n",
    "    .drop(columns=[\"Country Code_group_1\", \"Country Code_group_2\", \"Country Code\"])\n",
    "    .rename(\n",
    "        columns={\n",
    "            \"percentage_group_1\": \"0 to 14 years\",\n",
    "            \"percentage_group_2\": \"15 to 64 years\",\n",
    "            \"percentage\": \"over 65 years\",\n",
    "        }\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "countries_ages_groups_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1 - Drop irrelevant columns:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`FIPS`: the **FIPS** represents the Federal Information Processing Standard state code, and it's related only to the `U.S.` states, we won't need it now (unless we want to draw a map specific to the `U.S.`)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Admin2`: it's not clear what does this column means, will be dropped :)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Province_State`: the city on which the case was recorded, but not all cases are associated with the city, so we will drop it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"Province_State\"].isna().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Last_update`: the date on which this row was last updated, irrelvant to our analysis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Lat` and `Long_` columns specify the coordinates of the case, we won't be needing this inofrmation right now."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Combined_Key`: this column is the concatenation of `Admin2`, `Province_State`, and ` \tCountry_Region`, irrelvant too."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def drop_irrelevant_columns(df, irrelevant_columns):\n",
    "    new_df = df.drop(columns=irrelevant_columns)\n",
    "    return new_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "irrelevant_columns = [\n",
    "    \"FIPS\",\n",
    "    \"Admin2\",\n",
    "    \"Province_State\",\n",
    "    \"Last_Update\",\n",
    "    \"Lat\",\n",
    "    \"Long_\",\n",
    "    \"Combined_Key\",\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = drop_irrelevant_columns(df, irrelevant_columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2 - Normalize columns' names:\n",
    "\n",
    "rename column `Country_Region` to `Country`, for easier manipulation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def rename_columns(df, columns_mapping):\n",
    "    new_df = df.rename(columns=columns_mapping)\n",
    "    return new_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "columns_mapping = {\"Country_Region\": \"Country\"}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = rename_columns(df, columns_mapping)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3 - Aggregate data by countries:\n",
    "aggregate the number of confirmed cases, number of deaths, number of reovered cases, and number of active cases for each country."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def apply_aggregation(df, group_by_column, sort_by_column):\n",
    "    aggregate_df = (\n",
    "        df.groupby(by=group_by_column)\n",
    "        .sum()\n",
    "        .reset_index()\n",
    "        .sort_values(by=sort_by_column, ascending=False)\n",
    "    )\n",
    "    return aggregate_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "aggregated_df = apply_aggregation(\n",
    "    df, group_by_column=\"Country\", sort_by_column=\"Deaths\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "aggregated_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "the number of `Confirmed` cases in each country should be equal to the sum of: `Deaths`, `Recovered`, and `Active`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "aggregated_df[\"Confirmed\"].eq(\n",
    "    aggregated_df[\"Deaths\"] + aggregated_df[\"Recovered\"] + aggregated_df[\"Active\"]\n",
    ").sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "aggregated_df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4 - Focus on *Outbreaks*:\n",
    "We will focus in this notebook only on countries with high **death tolls**, countires that are *nearly* affected by the pandemic will be discarded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "most_n_countries = 15"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "most_affected_countries_names = aggregated_df[\"Country\"][:most_n_countries].values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(most_affected_countries_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "most_affected_df = aggregated_df[\n",
    "    aggregated_df[\"Country\"].isin(most_affected_countries_names)\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "most_affected_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Visualization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 15 shades of red\n",
    "reds = [\n",
    "    \"#fce8e9\",\n",
    "    \"#f9d0d4\",\n",
    "    \"#f6b9be\",\n",
    "    \"#f3a1a9\",\n",
    "    \"#f08a93\",\n",
    "    \"#ec727d\",\n",
    "    \"#e95b68\",\n",
    "    \"#e64352\",\n",
    "    \"#e32b3d\",\n",
    "    \"#e01427\",\n",
    "    \"#ca1223\",\n",
    "    \"#b3101f\",\n",
    "    \"#9d0e1b\",\n",
    "    \"#860c17\",\n",
    "    \"#700a14\",\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "white_color_hex = \"#FFFFFF\"\n",
    "blue_color_hex = \"#04668C\"\n",
    "green_color_hex = \"#158A24\"\n",
    "orange_color_hex = \"#F77300\"\n",
    "coquelicot_color_hex = \"#EC4414\"\n",
    "red_color_hex = \"#E01427\"\n",
    "purple_color_hex = \"#613266\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overall Statistics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "overall_stats_df = (\n",
    "    pd.DataFrame(df[[\"Deaths\", \"Recovered\", \"Active\"]].sum(), columns=[\"total\"])\n",
    "    .reset_index()\n",
    "    .rename(columns={\"index\": \"measure\"})\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "overall_stats_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "color_map = {\n",
    "    \"Active\": orange_color_hex,\n",
    "    \"Recovered\": green_color_hex,\n",
    "    \"Deaths\": red_color_hex,\n",
    "}\n",
    "\n",
    "fig = px.pie(\n",
    "    data_frame=overall_stats_df,\n",
    "    values=\"total\",\n",
    "    names=\"measure\",\n",
    "    color=\"measure\",\n",
    "    color_discrete_map=color_map,\n",
    ")\n",
    "\n",
    "fig.update_layout(\n",
    "    autosize=False,\n",
    "    width=800,\n",
    "    height=600,\n",
    "    title_text=\"<i><b>Overall statistics</b></i>\",\n",
    "    xaxis_title=\"Measures\",\n",
    "    yaxis_title=\"Class\",\n",
    "    plot_bgcolor=\"rgba(0, 0, 0, 0)\",\n",
    "    paper_bgcolor=\"rgba(0, 0, 0, 0)\",\n",
    "    font={\"family\": \"Courier New, monospace\", \"size\": 14, \"color\": \"#eaeaea\"},\n",
    ")\n",
    "\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "save_fig_as_div(fig, \"overall_stats_pie_chart.html\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Pie Chart"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.pie(\n",
    "    most_affected_df,\n",
    "    names=\"Country\",\n",
    "    values=\"Deaths\",\n",
    "    color=\"Deaths\",\n",
    "    color_discrete_sequence=reds[::-1],\n",
    ")\n",
    "\n",
    "fig.update_layout(\n",
    "    autosize=False,\n",
    "    width=800,\n",
    "    height=700,\n",
    "    title_text=\"<i><b>Most affected 15 countries by deaths</b></i>\",\n",
    "    plot_bgcolor=\"rgba(0, 0, 0, 0)\",\n",
    "    paper_bgcolor=\"rgba(0, 0, 0, 0)\",\n",
    "    font={\"family\": \"Courier New, monospace\", \"size\": 14, \"color\": \"#eaeaea\"},\n",
    ")\n",
    "\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "save_fig_as_div(fig_obj=fig, file_name=\"most_affected_countries_pie_chart.html\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Treemap Chart"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "country_continent_df = pd.read_csv(\"other-data/Countries-Continents.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "country_continent_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "most_n = 50"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "treemap_df = get_most_n_affected_countries(aggregated_df, most_n).merge(\n",
    "    right=country_continent_df, left_on=\"Country\", right_on=\"Country\", how=\"inner\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "treemap_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "treemap_df[\"world\"] = \"world\"  # in order to have a single root node"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.treemap(\n",
    "    treemap_df,\n",
    "    path=[\"world\", \"Continent\", \"Country\"],\n",
    "    values=\"Deaths\",\n",
    "    color=\"Deaths\",\n",
    "    color_continuous_scale=[orange_color_hex, coquelicot_color_hex, red_color_hex],\n",
    ")\n",
    "\n",
    "fig.update_layout(\n",
    "    autosize=False,\n",
    "    width=800,\n",
    "    height=600,\n",
    "    title_text=\"<i><b>Treemap of Country-Continent</b></i>\",\n",
    "    plot_bgcolor=\"rgba(0, 0, 0, 0)\",\n",
    "    paper_bgcolor=\"rgba(0, 0, 0, 0)\",\n",
    "    font={\"family\": \"Courier New, monospace\", \"size\": 14, \"color\": \"#eaeaea\"},\n",
    ")\n",
    "\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "save_fig_as_div(fig_obj=fig, file_name=\"most_affected_countries_treemap_chart.html\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stacked Bar Chart:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = go.Figure(\n",
    "    data=[\n",
    "        go.Bar(\n",
    "            name=\"Deaths\",\n",
    "            x=most_affected_df[\"Country\"],\n",
    "            y=most_affected_df[\"Deaths\"],\n",
    "            marker_color=red_color_hex,\n",
    "        ),\n",
    "        go.Bar(\n",
    "            name=\"Recovered\",\n",
    "            x=most_affected_df[\"Country\"],\n",
    "            y=most_affected_df[\"Recovered\"],\n",
    "            marker_color=green_color_hex,\n",
    "        ),\n",
    "        go.Bar(\n",
    "            name=\"Active\",\n",
    "            x=most_affected_df[\"Country\"],\n",
    "            y=most_affected_df[\"Active\"],\n",
    "            marker_color=orange_color_hex,\n",
    "        ),\n",
    "    ]\n",
    ")\n",
    "\n",
    "fig.update_layout(\n",
    "    barmode=\"stack\",\n",
    "    autosize=False,\n",
    "    width=800,\n",
    "    height=600,\n",
    "    title_text=\"<i><b>Most affected 15 countries by deaths</b></i>\",\n",
    "    xaxis_title=\"Country\",\n",
    "    yaxis_title=\"Cases\",\n",
    "    plot_bgcolor=\"rgba(0, 0, 0, 0)\",\n",
    "    paper_bgcolor=\"rgba(0, 0, 0, 0)\",\n",
    "    font={\"family\": \"Courier New, monospace\", \"size\": 14, \"color\": \"#eaeaea\"},\n",
    ")\n",
    "\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "save_fig_as_div(fig_obj=fig, file_name=\"most_affected_countries_stacked_bar_chart.html\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Countries fatalities rate:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "countries_fatalities_df = pd.DataFrame(\n",
    "    data=most_affected_df[[\"Country\", \"Confirmed\", \"Deaths\"]]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "countries_fatalities_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "let's compute the fatality rate:\n",
    "\n",
    "The fatality rate of a country, is: (deaths * 100) / confirmed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "countries_fatalities_df[\"Fatality rate\"] = (\n",
    "    countries_fatalities_df[\"Deaths\"] * 100\n",
    ") / countries_fatalities_df[\"Confirmed\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "countries_fatalities_df = countries_fatalities_df.round({\"Fatality rate\": 2})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "countries_fatalities_df.sort_values(by=\"Deaths\", ascending=False, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "countries_fatalities_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The fatality rate is effected by both factors:\n",
    "- The quality of health care.\n",
    "- The average age of the population."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = make_subplots(rows=1, cols=3, shared_yaxes=True)\n",
    "\n",
    "fig.add_trace(\n",
    "    go.Bar(\n",
    "        name=\"Deaths\",\n",
    "        x=countries_fatalities_df[\"Deaths\"],\n",
    "        y=countries_fatalities_df[\"Country\"],\n",
    "        orientation=\"h\",\n",
    "        marker_color=red_color_hex,\n",
    "    ),\n",
    "    row=1,\n",
    "    col=1,\n",
    ")\n",
    "\n",
    "fig.add_trace(\n",
    "    go.Bar(\n",
    "        name=\"Confirmed\",\n",
    "        x=countries_fatalities_df[\"Confirmed\"],\n",
    "        y=countries_fatalities_df[\"Country\"],\n",
    "        orientation=\"h\",\n",
    "        marker_color=blue_color_hex,\n",
    "    ),\n",
    "    row=1,\n",
    "    col=2,\n",
    ")\n",
    "\n",
    "fig.add_trace(\n",
    "    go.Bar(\n",
    "        name=\"Fatality rate\",\n",
    "        x=countries_fatalities_df[\"Fatality rate\"],\n",
    "        y=countries_fatalities_df[\"Country\"],\n",
    "        orientation=\"h\",\n",
    "        marker_color=purple_color_hex,\n",
    "    ),\n",
    "    row=1,\n",
    "    col=3,\n",
    ")\n",
    "\n",
    "fig.update_layout(\n",
    "    barmode=\"stack\",\n",
    "    autosize=False,\n",
    "    width=800,\n",
    "    height=600,\n",
    "    title_text=\"<i><b>Case fatality rate</b></i>\",\n",
    "    yaxis_title=\"Country\",\n",
    "    plot_bgcolor=\"rgba(0, 0, 0, 0)\",\n",
    "    paper_bgcolor=\"rgba(0, 0, 0, 0)\",\n",
    "    font={\"family\": \"Courier New, monospace\", \"size\": 14, \"color\": \"#eaeaea\"},\n",
    ")\n",
    "\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "save_fig_as_div(\n",
    "    fig_obj=fig, file_name=\"most_affected_countries_fatality_rate_bar_chart.html\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cases per million people:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cases_statistics_df = most_affected_df.merge(\n",
    "    world_pop_df, left_on=\"Country\", right_on=\"Country\", how=\"inner\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cases_statistics_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cases_statistics_df[\"active cases per million\"] = (\n",
    "    1000000.0 * cases_statistics_df[\"Active\"]\n",
    ") / cases_statistics_df[\"population_size\"]\n",
    "cases_statistics_df[\"deaths per million\"] = (\n",
    "    1000000.0 * cases_statistics_df[\"Deaths\"]\n",
    ") / cases_statistics_df[\"population_size\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cases_statistics_df = cases_statistics_df.round(\n",
    "    {\"active cases per million\": 0, \"deaths per million\": 0}\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Deaths per million people"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cases_statistics_df.sort_values(by=\"deaths per million\", ascending=False, inplace=True)\n",
    "\n",
    "fig = px.bar(\n",
    "    data_frame=cases_statistics_df,\n",
    "    x=\"Country\",\n",
    "    y=\"deaths per million\",\n",
    "    color=\"deaths per million\",\n",
    "    color_continuous_scale=[white_color_hex, red_color_hex],\n",
    ")\n",
    "\n",
    "fig.update_layout(\n",
    "    autosize=False,\n",
    "    width=800,\n",
    "    height=600,\n",
    "    title_text=\"<i><b>Deaths per million people</b></i>\",\n",
    "    xaxis_title=\"Country\",\n",
    "    yaxis_title=\"Deaths per million\",\n",
    "    plot_bgcolor=\"rgba(0, 0, 0, 0)\",\n",
    "    paper_bgcolor=\"rgba(0, 0, 0, 0)\",\n",
    "    font={\"family\": \"Courier New, monospace\", \"size\": 14, \"color\": \"#eaeaea\"},\n",
    ")\n",
    "\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "save_fig_as_div(fig_obj=fig, file_name=\"deaths_per_million_people_bar_chart.html\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Active cases per million people"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cases_statistics_df.sort_values(\n",
    "    by=\"active cases per million\", inplace=True, ascending=False\n",
    ")\n",
    "\n",
    "fig = px.bar(\n",
    "    data_frame=cases_statistics_df,\n",
    "    x=\"Country\",\n",
    "    y=\"active cases per million\",\n",
    "    color=\"active cases per million\",\n",
    "    color_continuous_scale=[white_color_hex, orange_color_hex],\n",
    ")\n",
    "\n",
    "fig.update_layout(\n",
    "    autosize=False,\n",
    "    width=800,\n",
    "    height=600,\n",
    "    title_text=\"<i><b>Active cases per million people</b></i>\",\n",
    "    xaxis_title=\"Country\",\n",
    "    yaxis_title=\"Active cases per million\",\n",
    "    plot_bgcolor=\"rgba(0, 0, 0, 0)\",\n",
    "    paper_bgcolor=\"rgba(0, 0, 0, 0)\",\n",
    "    font={\"family\": \"Courier New, monospace\", \"size\": 14, \"color\": \"#eaeaea\"},\n",
    ")\n",
    "\n",
    "# rotate x-axis ticks\n",
    "fig.update_xaxes(tickangle=45)\n",
    "\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "save_fig_as_div(fig_obj=fig, file_name=\"active_cases_per_million_people_bar_chart.html\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Aging vs fatality-rate relation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "most_n = 40"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "most_affected_countries_df = get_most_n_affected_countries(aggregated_df, most_n)\n",
    "\n",
    "countries_fatalities_df = get_countries_fatalities(most_affected_countries_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "most_affected_countries_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "countries_fatalities_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "countries_ages_groups_df = countries_ages_groups_df.round(\n",
    "    {\"0 to 14 years\": 1, \"15 to 64 years\": 1, \"over 65 years\": 1}\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "countries_ages_groups_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "age_fatality_correlation_df = countries_fatalities_df.merge(\n",
    "    countries_ages_groups_df, left_on=\"Country\", right_on=\"Country\", how=\"inner\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "age_fatality_correlation_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.scatter(\n",
    "    data_frame=age_fatality_correlation_df,\n",
    "    x=\"over 65 years\",\n",
    "    y=\"Fatality rate\",\n",
    "    color=\"Deaths\",\n",
    "    size=\"Deaths\",\n",
    "    hover_data=[\"Country\", \"Confirmed\"],\n",
    "    color_continuous_scale=[orange_color_hex, red_color_hex],\n",
    "    trendline=\"ols\",\n",
    ")\n",
    "\n",
    "fig.update_layout(\n",
    "    autosize=False,\n",
    "    width=1000,\n",
    "    height=600,\n",
    "    title_text=\"<i><b>The correlation between fatality rate and aging</b></i>\",\n",
    "    xaxis_title=\"Percentage of elderly (age over 65 years)\",\n",
    "    yaxis_title=\"Fatality rate\",\n",
    "    plot_bgcolor=\"rgba(0, 0, 0, 0)\",\n",
    "    paper_bgcolor=\"rgba(0, 0, 0, 0)\",\n",
    "    font={\"family\": \"Courier New, monospace\", \"size\": 14, \"color\": \"#eaeaea\"},\n",
    ")\n",
    "\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "save_fig_as_div(fig_obj=fig, file_name=\"age_vs_fatality_rate_scatter_plot.html\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.7.3 64-bit ('base': conda)",
   "language": "python",
   "name": "python37364bitbasecondaec3359f57daf47e5be8d0ffe5590fea3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  },
  "toc-showcode": false,
  "toc-showmarkdowntxt": false,
  "toc-showtags": false
 },
 "nbformat": 4,
 "nbformat_minor": 4
}