4-Plotting_Data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is a visualization?\n",
    "\n",
    "Visualizations help us reveal patterns in data by mapping some property of the data onto a visual property our brain can interpret, e.g.\n",
    "\n",
    "* A geometry or glyph type (scatter, bar, pie etc.)\n",
    "* Scales/axes (x and y)\n",
    "* Colors\n",
    "* Shapes\n",
    "* Size\n",
    "* Opacity\n",
    "\n",
    "In this section we will explore how to use pandas `.plot` (which uses matplotlib) and `.hvplot` to plot data directly from a DataFrame. The difference between the two is that .plot uses `Matplotlib` to plot the data which gives you a static image, while ``.hvplot`` uses HoloViews and Bokeh in the background to give you interactive plots.\n",
    "\n",
    "<img src=\"https://hvplot.pyviz.org/assets/hvplot-wm.png\" width=150px></img>\n",
    "\n",
    "hvPlot is a library developed at Anaconda by us to quickly visually explore datasets using an API users are already familiar with."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will start by importing pandas and make a simple dataset containing a few timeseries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "idx = pd.date_range('1/1/2000', periods=1000)\n",
    "df  = pd.DataFrame(np.random.randn(1000, 4), index=idx, columns=list('ABCD')).cumsum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we will initialize both matplotlib and hvplot plotting backends:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import hvplot.pandas\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotting\n",
    "\n",
    "To plot this dataframe we can now simple call `.plot`, which will give us a static Matplotlib plot:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.plot();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively we can call `.hvplot`, giving us a fully interactive plot including hover, interactive legends, zooming panning etc.:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.hvplot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are several plot types available using the `.hvplot.<type>()` methods. See [hvPlot reference gallery](https://hvplot.pyviz.org/reference/index.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Customizing the plot\n",
    "\n",
    "hvPlot tries to chose sensible defaults for plots however you can adjust the `width` and `height` of a plot by setting these as options. You can also make the plot responsive to fit the entire screen:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.hvplot(responsive=True, min_height=400)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition to options you can see when typing `df.hvplot.<type>(Shift+Tab)` you can see the full list of options here: https://hvplot.pyviz.org/user_guide/Customization.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotting single variables\n",
    "\n",
    "To plot just a single column we can declare both the x- and the y-column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.hvplot.line('index', 'A')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now plot column 'C' in the same way:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotting histograms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let us return to the mpg dataset and explore generating a few different plot types for this data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mpg = pd.read_csv('auto-mpg.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A histogram is a very useful tool for understanding the distribution of a dataset, by selecting a column we can generate a histogram for that column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mpg.mpg.hvplot.hist()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now plot a histogram of the horsepower (hp) column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotting scatter plots\n",
    "\n",
    "Scatter plots allow us to see the relationship between two variables:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mpg.hvplot.scatter('hp', 'mpg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now plot the relationship between 'weight' and 'mpg' values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also color by the origin to see how this third variable interacts with the other two:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mpg.hvplot.scatter('hp', 'mpg', color='origin')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotting bars\n",
    "\n",
    "Bar graphs can be very useful for visualizing statistics for a (relatively small) number of categorical values, e.g. by computing the mean 'mpg' for each 'origin':"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mpg.groupby('origin').mean().hvplot.bar('origin', 'mpg')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot a bar graph of the mean miles per gallon (mpg) values per year:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotting subsets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To explore subsetting we will load a second dataset containing the population by year and by country:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pop = pd.read_csv('world_ind_pop_data.csv')\n",
    "\n",
    "pop.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we will select the 5 most populous countries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "most_populous = ['CHN', 'IND', 'USA', 'IDN', 'BRA']\n",
    "populous_df = pop[pop.CountryCode.isin(most_populous)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And plot the 'Total Population' ``by`` country:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "populous_df.hvplot('Year', 'Total Population', by='CountryName')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here the `by` variable overlays the line plot for each country. In a bar chart on the other hand it will group each bar:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mpg.groupby(['yr', 'origin']).mean().hvplot.bar('yr', 'mpg', by='origin', rot=90)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we have multiple `y` variables we want to plot we can also generate a separate plot for each of these by leaving ``y`` unspecified and using the subplots option:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mpg.groupby('origin').mean().hvplot.bar('origin', subplots=True, shared_axes=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another option for subsetting data is to use the ``groupby`` argument, instead of overlaying or subplotting this will generate widgets to select between the different values of the specified column(s):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "income_groups = ['Low income', 'Middle income', 'High income']\n",
    "income_df = pop[pop.CountryName.isin(income_groups)]\n",
    "\n",
    "income_df.hvplot.bar('CountryName', 'Urban population (% of total)', groupby='Year',\n",
    "                     ylim=(0, 100)).redim.values(CountryName=income_groups)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a bar plot of the 'Urban population (% of total)' by 'Year' with a widget to select between 'CountryName':"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adding widgets\n",
    "\n",
    "Another major strength when comparing hvPlot to the standard .plot API is the integration with Panel widgets. By creating a set of widgets and passing those in instead of fixed variables we can quickly create a GUI for exploring some dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import panel as pn\n",
    "\n",
    "x = pn.widgets.Select(options=list(mpg.columns), value='mpg', name='x')\n",
    "y = pn.widgets.Select(options=list(mpg.columns), value='hp', name='y')\n",
    "color = pn.widgets.Select(options=list(mpg.columns), value='origin', name='color')\n",
    "\n",
    "pn.Row(\n",
    "    pn.Column(x, y, color),\n",
    "    mpg.hvplot.scatter(x, y, color=color)\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}