4. Pandas Tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T13:15:18.240575Z",
     "start_time": "2020-06-24T13:15:18.229186Z"
    }
   },
   "outputs": [],
   "source": [
    "# # This line imports the NumPy package\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction\n",
    "\n",
    "**Pandas** is desgined to make data pre-processing and data analysis fast and easy in Python. Pandas adopts many coding idioms from NumPy, such as avoiding the `for` loops, but pandas is designed for working with heterogenous data represented in tabular format.\n",
    "\n",
    "To use Pandas, you need to import the `pandas` module, using for example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T12:57:09.909716Z",
     "start_time": "2020-06-24T12:57:02.830486Z"
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This import style is quite standard; all objects and functions the `pandas` package will now be invoked with the `pd.` prefix."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Series\n",
    "\n",
    "Pandas has two main data structures, **Series** and **DataFrame**. Series are a the Pandas version of 1-D Numpy arrays. A Series is a single dimension array-like object containing a *sequence of values*, with an array of *data labels*, called its **index**.\n",
    "A Series can be created easily from a Python list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T14:06:29.883282Z",
     "start_time": "2020-06-24T14:06:29.873731Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0    4\n",
      "1    8\n",
      "2    1\n",
      "3    3\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "ts = pd.Series([4, 8, 1, 3])\n",
    "print(ts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The underlying structure can be recovered with the `values` attribute:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T14:06:30.975916Z",
     "start_time": "2020-06-24T14:06:30.966676Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[4 8 1 3]\n"
     ]
    }
   ],
   "source": [
    "print(ts.values)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The string representation of a Series display two columns: the first column represents the index array, the second column represents the values array. Since no index was specified, the default indexing consists of increasing integers starting from 0. To create a Series with its own index, you can write:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T13:10:32.009532Z",
     "start_time": "2020-06-24T13:10:31.999642Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "first     4\n",
      "second    8\n",
      "third     1\n",
      "fourth    3\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "ts = pd.Series([4, 8, 1, 3], index=['first', 'second', 'third', 'fourth'])\n",
    "print(ts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The labels in the index can be used to select values in the Series (note the list in the second line):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T13:13:05.940761Z",
     "start_time": "2020-06-24T13:13:05.922266Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4\n",
      "second    8\n",
      "fourth    3\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "print(ts['first'])\n",
    "print(ts[['second', 'fourth']])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using NumPy functions or NumPy-like operations, such as boolean indexing, universal functions, and so on, will preserve the indexes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T13:15:28.579927Z",
     "start_time": "2020-06-24T13:15:28.567266Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "first     4\n",
      "second    8\n",
      "dtype: int64\n",
      "first       54.598150\n",
      "second    2980.957987\n",
      "third        2.718282\n",
      "fourth      20.085537\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "print(ts[ts > 3])\n",
    "print(np.exp(ts))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can think about a Series as a kind of fixed-length, ordered Python's `dict`, mapping index values to data values. In fact, it is possible to create a Series directlty from a Python's `dict`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T13:18:31.486140Z",
     "start_time": "2020-06-24T13:18:31.465527Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pisa       80\n",
      "London    300\n",
      "Paris       1\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "my_dict = {'Pisa': 80, 'London': 300, 'Paris': 1}\n",
    "ts = pd.Series(my_dict)\n",
    "print(ts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Arithmetic operations on Series are automatically aligned on the index labels:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T13:29:55.852232Z",
     "start_time": "2020-06-24T13:29:55.833699Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "first      8.0\n",
      "fourth     NaN\n",
      "pisa       NaN\n",
      "second    16.0\n",
      "third      NaN\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "ts1 = pd.Series([4, 8, 1, 3], index=['first', 'second', 'third', 'fourth'])\n",
    "ts2 = pd.Series([4, 8, 1], index=['first', 'second', 'pisa'])\n",
    "ts_sum = ts1 + ts2\n",
    "print(ts_sum)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here two index values are correctly computed (corresponding to the label `first` and `second`). The two other index labels `third` and `fourth` in `ts1` are missing in `ts2`, as well as the `pisa` index label in `ts2`. Hence, for each of these index label, a `NaN` value (*not a number*) appears, which Pnadas considers as a **missing value**.\n",
    "\n",
    "The `pd.isnull` and `pd.notnull` functions detects missing data, as well as the corresponding instance methods:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T13:31:20.930884Z",
     "start_time": "2020-06-24T13:31:20.911850Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "first     False\n",
      "fourth     True\n",
      "pisa       True\n",
      "second    False\n",
      "third      True\n",
      "dtype: bool\n",
      "first      True\n",
      "fourth    False\n",
      "pisa      False\n",
      "second     True\n",
      "third     False\n",
      "dtype: bool\n"
     ]
    }
   ],
   "source": [
    "print(pd.isnull(ts_sum))\n",
    "print(ts_sum.notnull())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Both Series and its index have a `name` attribute:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T13:33:04.624972Z",
     "start_time": "2020-06-24T13:33:04.613753Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "new name\n",
      "first      8.0\n",
      "fourth     NaN\n",
      "pisa       NaN\n",
      "second    16.0\n",
      "third      NaN\n",
      "Name: sum, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "ts_sum.name = 'sum'\n",
    "ts_sum.index.name = 'new name'\n",
    "print(ts_sum)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# DataFrame\n",
    "\n",
    "A DataFrame is a rectangular table of data. It contains an ordered list of columns. Every column can be of a different value type. A dataFrame has both a *row index* and a *column index*. It can be thought as a dict of Series (one per column) all sharing the same index labels.\n",
    "\n",
    "There are many way to construct a DataFrame, but the most common is using a dictionary of equally-sized Python's lists (or NumPy's arrays):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T13:46:58.794613Z",
     "start_time": "2020-06-24T13:46:58.779817Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "            Brand  Price\n",
      "0     Honda Civic  22000\n",
      "1  Toyota Corolla  25000\n",
      "2      Ford Focus  27000\n",
      "3         Audi A4  35000\n"
     ]
    }
   ],
   "source": [
    "cars = {'Brand': ['Honda Civic', 'Toyota Corolla', 'Ford Focus', 'Audi A4'],\n",
    "        'Price': [22000, 25000, 27000, 35000]}\n",
    "\n",
    "df = pd.DataFrame(cars)\n",
    "print(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The resulting DataFrame will received its index automatically as with Series."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To pretty-print a DataFrame in a Jupyter notebooks, it is enough to write its name (or using the `head()` instance method for very long DataFrames):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T13:48:19.586597Z",
     "start_time": "2020-06-24T13:48:19.564159Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Brand</th>\n",
       "      <th>Price</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Honda Civic</td>\n",
       "      <td>22000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Toyota Corolla</td>\n",
       "      <td>25000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Ford Focus</td>\n",
       "      <td>27000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Audi A4</td>\n",
       "      <td>35000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Brand  Price\n",
       "0     Honda Civic  22000\n",
       "1  Toyota Corolla  25000\n",
       "2      Ford Focus  27000\n",
       "3         Audi A4  35000"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is possible to change the order of the colums at DataFrame construction time. If you provide a column's name not included in the dictionary, a column with missing values will appear:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T13:50:34.030261Z",
     "start_time": "2020-06-24T13:50:34.008485Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Color  Price           Brand\n",
      "0   NaN  22000     Honda Civic\n",
      "1   NaN  25000  Toyota Corolla\n",
      "2   NaN  27000      Ford Focus\n",
      "3   NaN  35000         Audi A4\n"
     ]
    }
   ],
   "source": [
    "df = pd.DataFrame(cars, columns=['Color', 'Price', 'Brand'])\n",
    "print(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If working with a large table, it might be useful to sometimes have a list of all the columns' names. This is given by the `keys()` methods:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T14:19:07.147448Z",
     "start_time": "2020-06-24T14:19:07.137275Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['Color', 'Price', 'Brand'], dtype='object')\n"
     ]
    }
   ],
   "source": [
    "print(df.keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Many feature from the NumPy package can be directly used with Pandas DataFrames"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T08:48:02.257248Z",
     "start_time": "2020-06-25T08:48:02.236802Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[['Mahler' 1860 1911 'Kaliste' 51]\n",
      " ['Beethoven' 1770 1827 'Bonn' 57]\n",
      " ['Puccini' 1858 1924 'Lucques' 66]\n",
      " ['Shostakovich' 1906 1975 'Saint-Petersburg' 69]]\n"
     ]
    }
   ],
   "source": [
    "print(df.values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T08:48:03.529820Z",
     "start_time": "2020-06-25T08:48:03.499697Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(4, 5)\n"
     ]
    }
   ],
   "source": [
    "print(df.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another common way to create a DataFrame is to use a *nested dict of dicts*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T14:27:29.650782Z",
     "start_time": "2020-06-24T14:27:29.643762Z"
    }
   },
   "outputs": [],
   "source": [
    "pop = {'Nevada': {2001: 2.4, 2002: 2.9},\n",
    "       'Ohio': {2000: 1.5, 2001: 1.7, 2002: 3.6}}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If this nested dict is passed to the DataFrame, the outer dict keys are interpreted as column labels, and the inner keys are interpreted as row labels:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T14:28:37.554310Z",
     "start_time": "2020-06-24T14:28:37.535827Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Nevada</th>\n",
       "      <th>Ohio</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2001</th>\n",
       "      <td>2.4</td>\n",
       "      <td>1.7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002</th>\n",
       "      <td>2.9</td>\n",
       "      <td>3.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2000</th>\n",
       "      <td>NaN</td>\n",
       "      <td>1.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Nevada  Ohio\n",
       "2001     2.4   1.7\n",
       "2002     2.9   3.6\n",
       "2000     NaN   1.5"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(pop)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Accessing a DataFrame"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now create a more complex DataFrame:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T14:10:32.187393Z",
     "start_time": "2020-06-24T14:10:32.162017Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              birth  death              city\n",
       "Mahler         1860   1911           Kaliste\n",
       "Beethoven      1770   1827              Bonn\n",
       "Puccini        1858   1924           Lucques\n",
       "Shostakovich   1906   1975  Saint-Petersburg"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dict_of_list = {'birth': [1860, 1770, 1858, 1906], \n",
    "                'death':[1911, 1827, 1924, 1975], \n",
    "                'city':['Kaliste', 'Bonn', 'Lucques', 'Saint-Petersburg']}\n",
    "composers_df = pd.DataFrame(dict_of_list, index=['Mahler', 'Beethoven', 'Puccini', 'Shostakovich'])\n",
    "composers_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are multiple ways of accessing values or series of values in a Dataframe. Unlike in Series, a simple bracket gives access to a column and not an index, for example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T14:11:08.259631Z",
     "start_time": "2020-06-24T14:11:08.241496Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Mahler                   Kaliste\n",
       "Beethoven                   Bonn\n",
       "Puccini                  Lucques\n",
       "Shostakovich    Saint-Petersburg\n",
       "Name: city, dtype: object"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers_df['city']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "returns a Series. Alternatively one can also use the attributes syntax and access columns by using:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T14:11:37.321286Z",
     "start_time": "2020-06-24T14:11:37.309806Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Mahler                   Kaliste\n",
       "Beethoven                   Bonn\n",
       "Puccini                  Lucques\n",
       "Shostakovich    Saint-Petersburg\n",
       "Name: city, dtype: object"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers_df.city"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The attributes syntax has some limitations, so in case something does not work as expected, revert to the brackets notation.\n",
    "\n",
    "When specifiying multiple columns, a DataFrame is returned:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T14:12:06.558313Z",
     "start_time": "2020-06-24T14:12:06.540600Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>city</th>\n",
       "      <th>birth</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>Kaliste</td>\n",
       "      <td>1860</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>Bonn</td>\n",
       "      <td>1770</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>Lucques</td>\n",
       "      <td>1858</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>Saint-Petersburg</td>\n",
       "      <td>1906</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          city  birth\n",
       "Mahler                 Kaliste   1860\n",
       "Beethoven                 Bonn   1770\n",
       "Puccini                Lucques   1858\n",
       "Shostakovich  Saint-Petersburg   1906"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers_df[['city', 'birth']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One of the important differences with a regular Numpy array is that **with Pandas' DataFrame regular indexing doesn't work**. Instead one has to use either the `iloc` or the `loc` attributes.\n",
    "\n",
    "**Remember that `loc` and `iloc` are attributes, not methods, hence they use brackets `[]` and not parenthesis `()`.**\n",
    "\n",
    "The `loc` attribute allows to recover elements by using the index labels, while the `iloc` attribute can be used to recover the regular indexing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T14:16:36.990555Z",
     "start_time": "2020-06-24T14:16:36.979449Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1911\n",
      "1911\n"
     ]
    }
   ],
   "source": [
    "print(composers_df.iloc[0,1])\n",
    "print(composers_df.loc['Mahler', 'death'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adding columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is very simple to add a column to a Dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T14:25:29.267510Z",
     "start_time": "2020-06-24T14:25:29.253143Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "      <th>country2</th>\n",
       "      <th>country</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "      <td>Austria</td>\n",
       "      <td>???</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "      <td>Germany</td>\n",
       "      <td>???</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "      <td>Italy</td>\n",
       "      <td>???</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "      <td>Russia</td>\n",
       "      <td>???</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              birth  death              city country2 country\n",
       "Mahler         1860   1911           Kaliste  Austria     ???\n",
       "Beethoven      1770   1827              Bonn  Germany     ???\n",
       "Puccini        1858   1924           Lucques    Italy     ???\n",
       "Shostakovich   1906   1975  Saint-Petersburg   Russia     ???"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers_df['country'] = '???'\n",
    "composers_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, an existing list can be used:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T14:25:30.513553Z",
     "start_time": "2020-06-24T14:25:30.500562Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "      <th>country2</th>\n",
       "      <th>country</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "      <td>Austria</td>\n",
       "      <td>???</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "      <td>Germany</td>\n",
       "      <td>???</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "      <td>Italy</td>\n",
       "      <td>???</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "      <td>Russia</td>\n",
       "      <td>???</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              birth  death              city country2 country\n",
       "Mahler         1860   1911           Kaliste  Austria     ???\n",
       "Beethoven      1770   1827              Bonn  Germany     ???\n",
       "Puccini        1858   1924           Lucques    Italy     ???\n",
       "Shostakovich   1906   1975  Saint-Petersburg   Russia     ???"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers_df['country2'] = ['Austria','Germany','Italy','Russia']\n",
    "composers_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deleting columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `del`keyword is used to delete columns:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T14:25:32.996744Z",
     "start_time": "2020-06-24T14:25:32.983364Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "      <th>country2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "      <td>Austria</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "      <td>Germany</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "      <td>Russia</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              birth  death              city country2\n",
       "Mahler         1860   1911           Kaliste  Austria\n",
       "Beethoven      1770   1827              Bonn  Germany\n",
       "Puccini        1858   1924           Lucques    Italy\n",
       "Shostakovich   1906   1975  Saint-Petersburg   Russia"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "del composers_df['country']\n",
    "composers_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Importing Excel files as DataFrames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another very common way of \"creating\" a Pandas Dataframe is by importing a table from another format like CSV or Excel. In order to import Excel files, you need to install the `xlrd` package:\n",
    "\n",
    "```shell\n",
    "> pip install xlrd\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An Excel table is provided in the [composers.xlsx](data/composers.xlsx) file and can be read with the `pd.read_excel` function. There are many more readers for other types of data (csv, json, html etc.) but we focus here on Excel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T14:46:22.696917Z",
     "start_time": "2020-06-24T14:46:22.651062Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>composer</th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Mahler</td>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Beethoven</td>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Puccini</td>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Shostakovich</td>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       composer  birth  death              city\n",
       "0        Mahler   1860   1911           Kaliste\n",
       "1     Beethoven   1770   1827              Bonn\n",
       "2       Puccini   1858   1924           Lucques\n",
       "3  Shostakovich   1906   1975  Saint-Petersburg"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers_df = pd.read_excel('data/composers.xlsx')\n",
    "composers_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The reader automatically recognized the heaers of the file. However it created a new index. If needed we can specify which column to use as header:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T14:47:02.153798Z",
     "start_time": "2020-06-24T14:47:02.086758Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              birth  death              city\n",
       "composer                                    \n",
       "Mahler         1860   1911           Kaliste\n",
       "Beethoven      1770   1827              Bonn\n",
       "Puccini        1858   1924           Lucques\n",
       "Shostakovich   1906   1975  Saint-Petersburg"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers_df = pd.read_excel('data/composers.xlsx', index_col = 'composer')\n",
    "composers_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we open the file in Excel, we see that it is composed of more than one sheet. Clearly, when not specifying anything, the reader only reads the first sheet. However we can specify a sheet:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T14:51:26.183305Z",
     "start_time": "2020-06-24T14:51:26.141838Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sibelius</th>\n",
       "      <td>unknown</td>\n",
       "      <td>unknown</td>\n",
       "      <td>unknown</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Haydn</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Rohrau</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                birth    death              city\n",
       "composer                                        \n",
       "Mahler           1860     1911           Kaliste\n",
       "Beethoven        1770     1827              Bonn\n",
       "Puccini          1858     1924           Lucques\n",
       "Shostakovich     1906     1975  Saint-Petersburg\n",
       "Sibelius      unknown  unknown           unknown\n",
       "Haydn             NaN      NaN            Rohrau"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers_df = pd.read_excel('data/composers.xlsx', index_col = 'composer', sheet_name='Sheet2')\n",
    "composers_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see above, some information is missing. Some missing values are marked as \"`unknown`\" while other are `NaN`. `NaN` is the standard symbol for unknown/missing values and is understood by Pandas while \"`unknown`\" is just seen as text. \n",
    "This is impractical as now we have columns with a mix of numbers and text which will make later computations difficult. What we would like to do is to replace all \"irrelevant\" values with the standard `NaN` symbol that says \"*no information*\".\n",
    "For this we can use the `na_values` argument to specify what should be a `NaN`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T14:58:59.168497Z",
     "start_time": "2020-06-24T14:58:59.139749Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860.0</td>\n",
       "      <td>1911.0</td>\n",
       "      <td>Kaliste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770.0</td>\n",
       "      <td>1827.0</td>\n",
       "      <td>Bonn</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858.0</td>\n",
       "      <td>1924.0</td>\n",
       "      <td>Lucques</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>1906.0</td>\n",
       "      <td>1975.0</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sibelius</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Haydn</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Rohrau</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               birth   death              city\n",
       "composer                                      \n",
       "Mahler        1860.0  1911.0           Kaliste\n",
       "Beethoven     1770.0  1827.0              Bonn\n",
       "Puccini       1858.0  1924.0           Lucques\n",
       "Shostakovich  1906.0  1975.0  Saint-Petersburg\n",
       "Sibelius         NaN     NaN               NaN\n",
       "Haydn            NaN     NaN            Rohrau"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers_df = pd.read_excel('data/composers.xlsx', index_col = 'composer', sheet_name='Sheet2', \n",
    "                             na_values=['unknown'])\n",
    "composers_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Plotting DataFrames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will learn more about plotting later, but let's see here some possibilities offered by Pandas. Pandas builds on top of Matplotlib but exploits the knowledge included in Dataframes to improve the default output. \n",
    "\n",
    "We can pass Series to Matplotlib which manages to understand them. Here's a default scatter plot:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T15:01:47.227488Z",
     "start_time": "2020-06-24T15:01:47.005823Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAD4CAYAAAAAczaOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAZF0lEQVR4nO3df5BV533f8fdHy8pZTWMvGtapWKFCMogMGsZgbSTNqG4kpRVImQ6UNA5K2lBLUzWu5Ik1Li0kja3YmRFjEnviieyOUlNFdQMiMaG4cUzIyLGmGf1aDPqBBNZakgMrVeBBuLW1woC+/eM+C5fL/XH27v15zuc1s6Oz33P27nm07Pc++zzf5zmKCMzMrBgu6fYNmJlZ5zjpm5kViJO+mVmBOOmbmRWIk76ZWYHM6fYN1DNv3rxYuHBht2/DzKyv7Nu37/sRMVLtXE8n/YULFzI+Pt7t2zAz6yuSvlfrnId3zMwKxEnfzKxAnPTNzArESd/MrECc9M3MCqSnq3fMzIpm1/5Jtuw5zOsnp5g/PMSGlUtYs2K0Za/vpG9m1iN27Z9k087nmTp9FoDJk1Ns2vk8QMsSv4d3zMx6xJY9h88l/GlTp8+yZc/hln0PJ30zsx7x+smpGcWb4aRvZtYj5g8PzSjeDCd9M7MesWHlEoYGBy6IDQ0OsGHlkpZ9j4ZJX9ICSd+U9KKkg5J+M8Uvl7RX0svpv3NTXJK+IGlC0nOSPlj2WuvT9S9LWt+yVpiZ5cCaFaM8sHYZo8NDCBgdHuKBtctaWr2jRs/IlXQFcEVEfFvSTwL7gDXAvwFORMRmSRuBuRHxnyTdDnwMuB24HvjDiLhe0uXAODAGRHqdayPirVrfe2xsLLzhmpnZzEjaFxFj1c417OlHxBsR8e10/P+Al4BRYDXwJ+myP6H0RkCKPxIlTwLD6Y1jJbA3Ik6kRL8XWDWLdpmZ2QzNqE5f0kJgBfAU8FMR8UY69X+An0rHo8CRsi87mmK14pXf427gboCrrrpqJrdnZtaz2r3oKqvMSV/SPwC+Cnw8Iv6vpHPnIiIk1R8nyigiHgIegtLwTite08ys08qT/PBlg/zwnTOcfreU0soXXQEdfTPIlPQlDVJK+P8jInam8JuSroiIN9LwzbEUnwQWlH35lSk2CdxUEf/b5m/dzKw3Va6sfevt0xddM3X6LPfvPsipM++2dQVupSzVOwK+DLwUEZ8rO7UbmK7AWQ/8z7L4r6cqnhuAH6RhoD3ArZLmpkqfW1PMzCxXqq2srebk1Om2r8CtlKWnfyPwr4HnJR1Isd8CNgM7JN0FfA/4cDr3dUqVOxPA28BHACLihKTPAM+k6z4dESda0gozsx4wPaQzOcsVtK1cgVupYdKPiP8NqMbpX6hyfQD31HitrcDWmdygmVk/qBzSaWRocICfGLyk6tBPK1fgVvKKXDOzFsg6pAMw97JBHli7jE/982vavgK3krdWNjNrgZkMybxz+l3g/GRtz1XvmJlZ/Vr7+cNDmcfypydr16wYPffRKR7eMTPLYHrMfvLkFMH58spd+yeB6pulDQ7Umg5t72RtPU76ZmYZNHrASbXN0rb8yw8w2oHtkmfCwztmZhlkecBJraGayqqedk/W1uOevplZBs0+4KQT2yXPhHv6ZlZ4WTZD27BySdM99k5P1tbjpG9mhVa5qKrW/je1yisBbtz8WNd3z8zKSd/MCq3eBG1l8q7ssWd9w+glHtM3s0LLMkFbS6OKnl7kpG9mhfa+ocEZxcvN5g2jW5z0zazQVGP9VK14uWYrerrJSd/MCu1klV0u68XLVVuF280a/Cyc9M2s0GbTW++1GvwsXL1jZoXWTP19rzzkvBlO+mZWaDPd3rgfyzTLOembWWHU6qHPZMXsTOr6e5GTvpkVQqt66P1YplnOE7lmVgi1euif2PHsuT3xs+jHMs1yTvpmVgi1euJnIy54GEoj/VimWc5J38wKoV5PfCZbJ/RjmWY5j+mbWSFUK80sN3lyKnMpZi9tlTxTTvpmliv1KnQAPv7ogapfJ+jrUsysPLxjZrnR6OHl9ZJ3QN/tmNkMJ30zy40sWx3XelB5Lf1SipmVk76Z5UaWGvpq1Tf19EspZlYNk76krZKOSXqhLPYBSU9Iel7S1yS9t+zcJkkTkg5LWlkWX5ViE5I2tr4pZtbvdu2f5MbNj7Fo419y4+bHZlQ/DzB8WfU98Mvj09U3WfRTKWZWWXr6DwOrKmL/FdgYEcuAvwA2AEhaCqwDrklf80VJA5IGgAeB24ClwB3pWjMzoPF4fBYR2eJrVowyUGfD/H4sxcyqYfVORDwuaWFF+Grg8XS8F9gD/A6wGtgeEaeAVyVNANel6yYi4hUASdvTtS/OtgFmlg+N9rSpVpUz/XXTsZNT1ffA/0GV+B3XL+ArT/79RfF/dcNV/N6abH8J9KNmSzYPUkrau4BfBhak+CjwZNl1R1MM4EhF/Pomv7eZ5VCt8fjJk1Os+PRf88N3znD63TgX2/Dnz0JwQUyUqnAqVRuXn07s2546wtkIBiTuuH5BrhM+NJ/07wS+IOl3gN3Aj1t1Q5LuBu4GuOqqq1r1smbW4+YPDzFZI/G/VeUpVqfPXpzeAy5K/PXG5X9vzbLcJ/lKTVXvRMShiLg1Iq4FtgHfTacmOd/rB7gyxWrFq732QxExFhFjIyMjzdyemfWhmVbV1BLQt1skdEJTPX1J74+IY5IuAf4z8F/Sqd3An0r6HDAfWAw8TenNd7GkRZSS/TrgV2d782aWH+UPM6nV489ieGiQv9t4S6tuK3eylGxuA54Alkg6KukuStU33wEOAa8D/w0gIg4COyhN0H4DuCcizkbEGeBeShO+LwE70rVmZi11cup0U+WeRaGoVePUA8bGxmJ8fLzbt2FmHVD5kJPZGhocKOzQjqR9ETFW7ZxX5JpZT6hWsjkbedw3pxWc9M2sJ9Qq2ZyekK11rpnXLDInfTPrCfUeQ1jraVW/dsNVdTdQy9u+Oa3g/fTNrCsqV9je/LMjfHXf5AVDPNM19uWVPdUecFJtPiCP++a0gidyzazjaiXpX7p2lG8eOt7wyVW1XjPLU6+KoN5Ernv6ZtZxtfbZ+eah403X2PfzIww7yWP6ZtZxWfa9t/Zw0jezjqs3aWvt5aRvZh1XrRpncED86NSZph+gYtl4TN/MOq6yGmf4skF++M6Zc/vhTz9Apfxaaw339M2sK9asGOXvNt7Cq5t/kcsunXNuX/xpXlHbHu7pm1lH1Cup9MRu57inb2Zt1+j5t57Y7RwnfTNru3rPv4XqE7teUdseHt4xs7ZrNHzTaJsFax0nfTNru1rPvw3gxs2PnUvwTvLt5+EdM2u7es+/rRzft/Zy0jeztluzYpQH1i6ruQ2yyzM7x8M7ZpbZbHaynB6+WbTxL6m2t6/LMzvDPX0zy6RR2WVWLs/sLid9M8ukUdllVi7P7C4P75hZJq1aNevyzO5y0jezTGqVXTYzLOPyzO7x8I6ZZeJhmXxwT9/MMvGwTD446ZvlULseEu5hmf7npG+WM9OlldOVNlkeSNKuNwnrPR7TN8uZmZZWtqr+3vpDw6QvaaukY5JeKIstl/SkpAOSxiVdl+KS9AVJE5Kek/TBsq9ZL+nl9LG+Pc0xs2oVNlC9tHLX/kk+sePZltTfW3/I0tN/GFhVEfss8LsRsRz4ZPoc4DZgcfq4G/gSgKTLgU8B1wPXAZ+SNHe2N29mF9q1fxLVOFdZWjndwz8b1TZF8LYIedUw6UfE48CJyjDw3nT8PuD1dLwaeCRKngSGJV0BrAT2RsSJiHgL2MvFbyRmNktb9hyuuq+N4KLSymrDQOW8LUI+NTum/3Fgi6QjwO8Dm1J8FDhSdt3RFKsVv4iku9OQ0fjx48ebvD2zYqrVOw8unsRt1JP/0akzHtfPoWaT/keB+yJiAXAf8OVW3VBEPBQRYxExNjIy0qqXNSuEWr3zalsaN+rJn5w67QndHGo26a8HdqbjP6M0Tg8wCSwou+7KFKsVN7MWmsmq2XoPNpnmCd38aTbpvw78fDq+BXg5He8Gfj1V8dwA/CAi3gD2ALdKmpsmcG9NMTNrofKHlYhSD/+Btcuq1txXXluLJ3TzpeHiLEnbgJuAeZKOUqrC+bfAH0qaA7xDqVIH4OvA7cAE8DbwEYCIOCHpM8Az6bpPR0Tl5LCZtcBMVs2WX3vj5sdatqGa9a6GST8i7qhx6toq1wZwT43X2QpsndHdmVnHbFi55IKVvOAN1fLI2zCYGeAN1YrCSd/MzvGGavnnvXfMzArESd/MrEA8vGPWAd662HqFk75ZmzWzv71Zu3h4x6zNau1v/7tfO9ilO7Iic9I3a7NaK1rfevu097WxjnPSN2uzeita79/t3r51lpO+WZvVW9F6cup0B+/EzEnfrO08WWu9xEnfrAMuqbGNZa24Wbs46Zt1wLvVH0NbM27WLk76Zh1Q7clV9eJm7eLFWWYt0GjF7Wy2LfZqXmslJ32zWcqy4rbZbYu9mtdazUnfbJZqrbjdsufwBYm5mW2Ls762WVYe0zebpVorblvxbNl2vrYVk5O+2SzVWnHbimfLtvO1rZic9M1macPKJQwNDlwQa9WzZdv52lZMHtM3m6V2PlvWz621VlNE764OGRsbi/Hx8W7fhplZX5G0LyLGqp3z8I6ZWYE46ZuZFYiTvplZgTjpm5kViKt3zPD+NlYcDXv6krZKOibphbLYo5IOpI/XJB0oO7dJ0oSkw5JWlsVXpdiEpI2tb4pZc6b3t5k8OUVwfn8bP7/W8ijL8M7DwKryQET8SkQsj4jlwFeBnQCSlgLrgGvS13xR0oCkAeBB4DZgKXBHutas6+rtb2OWNw2HdyLicUkLq52TJODDwC0ptBrYHhGngFclTQDXpXMTEfFK+rrt6doXZ3X3Zi0wk/1tPAxk/W62E7kfAt6MiJfT56PAkbLzR1OsVvwiku6WNC5p/Pjx47O8PbPGsu5v42Egy4PZJv07gG2tuJFpEfFQRIxFxNjIyEgrX9qsqqz723gYyPKg6eodSXOAtcC1ZeFJYEHZ51emGHXiZl2VdX8bb3NseTCbks1/ChyKiKNlsd3An0r6HDAfWAw8DQhYLGkRpWS/DvjVWXxvs5bK8oCT+cNDTFZJ8N7m2PpJlpLNbcATwBJJRyXdlU6to2JoJyIOAjsoTdB+A7gnIs5GxBngXmAP8BKwI11r1je8zbHlgXfZNJsBV+9YP6i3y6ZX5JrNQDPPuTXrJd57x8ysQJz0zcwKxEnfzKxAnPTNzArESd/MrEBcvWO55zJLs/Oc9C3XpjdJm94zZ3qTNMCJ3wrJwzuWa94kzexCTvqWa94kzexCTvqWa1n3yjcrCid9yzVvkmZ2IU/kWtv0QtVM1r3yzYrCSd/aYqZVM+18g/AmaWbneXjH2mImVTN+9qxZ5zjpW1vMpGrGZZVmneOkb20xk6oZl1WadY6TvrXFTKpmXFZp1jlO+tZSu/ZPcuPmx7jv0QP8xOAlDA8NImB0eIgH1i6rOqHqskqzznH1jrVMZcXOW2+fZmhwgM//yvK61TMuqzTrHCd9q2mmZZT1JmQbJXCXVZp1hpO+VdXM7pSekDXrfR7Tt6qaKaP0hKxZ73PSt6qa6bVXm5AF+NGpM15oZdYjnPStqmZ67WtWjPLA2mXMvWzwgvjJqdNeYWvWI5z0rapmyyjXrBjlsksvniryCluz3uCJXKtqNmWUntA1611O+lZTs2WU84eHmKyS4D2ha9Z9DYd3JG2VdEzSCxXxj0k6JOmgpM+WxTdJmpB0WNLKsviqFJuQtLG1zbBe4hW2Zr0rS0//YeCPgEemA5JuBlYDH4iIU5Len+JLgXXANcB84G8kXZ2+7EHgnwFHgWck7Y6IF1vVEOsdXmFr1rsaJv2IeFzSworwR4HNEXEqXXMsxVcD21P8VUkTwHXp3EREvAIgaXu61km/SypX2978syN889DxliVpr7A1603NVu9cDXxI0lOSviXp51J8FDhSdt3RFKsVv4ikuyWNSxo/fvx4k7dn9VR7aMlXnvx7P8TErACaTfpzgMuBG4ANwA5JasUNRcRDETEWEWMjIyOteEmrUG21bSWXWJrlU7PVO0eBnRERwNOS3gXmAZPAgrLrrkwx6sStw7KWTrrE0ix/mu3p7wJuBkgTtZcC3wd2A+skvUfSImAx8DTwDLBY0iJJl1Ka7N0925u35mQtnXSJpVn+ZCnZ3AY8ASyRdFTSXcBW4KdTGed2YH2UHAR2UJqg/QZwT0ScjYgzwL3AHuAlYEe61rqg1h455VxiaZZPKo3Q9KaxsbEYHx/v9m3kUrurd8yseyTti4ixaue8IregXFJpVkzecM3MrECc9M3MCsRJ38ysQJz0zcwKxEnfzKxAnPTNzArESd/MrECc9M3MCsRJ38ysQJz0zcwKxEnfzKxAnPTNzArESd/MrECc9M3MCsRJ38ysQJz0zcwKxEnfzKxAnPTNzArESd/MrECc9M3MCsRJ38ysQJz0zcwKxEnfzKxAnPTNzArESd/MrECc9M3MCqRh0pe0VdIxSS+Uxe6XNCnpQPq4vezcJkkTkg5LWlkWX5ViE5I2tr4pZmbWSJae/sPAqirxz0fE8vTxdQBJS4F1wDXpa74oaUDSAPAgcBuwFLgjXWtmZh00p9EFEfG4pIUZX281sD0iTgGvSpoArkvnJiLiFQBJ29O1L874js3MrGmzGdO/V9JzafhnboqNAkfKrjmaYrXiF5F0t6RxSePHjx+fxe2ZmVmlZpP+l4CfAZYDbwB/0KobioiHImIsIsZGRkZa9bJmZkaG4Z1qIuLN6WNJfwz8r/TpJLCg7NIrU4w6cTMz65CmevqSrij79F8A05U9u4F1kt4jaRGwGHgaeAZYLGmRpEspTfbubv62zcysGQ17+pK2ATcB8yQdBT4F3CRpORDAa8C/A4iIg5J2UJqgPQPcExFn0+vcC+wBBoCtEXGw5a0xM7O6FBHdvoeaxsbGYnx8vNu3YWbWVyTti4ixaue8ItfMrECamsjtdbv2T7Jlz2FePznF/OEhNqxcwpoVVStEzcwKJXdJf9f+STbtfJ6p02cBmDw5xaadzwM48ZtZ4eVueGfLnsPnEv60qdNn2bLncJfuyMysd+Qu6b9+cmpGcTOzIsld0p8/PDSjuJlZkeQu6W9YuYShwYELYkODA2xYuaRLd2Rm1jtyN5E7PVnr6h0zs4vlLulDKfE7yZuZXSx3wztmZlabk76ZWYE46ZuZFYiTvplZgTjpm5kVSE9vrSzpOPC9Wb7MPOD7LbidXud25kcR2ghuZzv9o4io+rzZnk76rSBpvNa+0nniduZHEdoIbme3eHjHzKxAnPTNzAqkCEn/oW7fQIe4nflRhDaC29kVuR/TNzOz84rQ0zczs8RJ38ysQPoy6UvaKumYpBeqnPuEpJA0L30uSV+QNCHpOUkfLLt2vaSX08f6TrYhi2rtlHS/pElJB9LH7WXnNqV2Hpa0siy+KsUmJG3sdDvqqfWzlPQxSYckHZT02bJ437URav4sHy37Ob4m6UDZuTy1c7mkJ1M7xyVdl+J9+btZo40fkPSEpOclfU3Se8vO9dbPMiL67gP4J8AHgRcq4guAPZQWdM1LsduBvwIE3AA8leKXA6+k/85Nx3O73bZG7QTuB/5DlWuXAs8C7wEWAd8FBtLHd4GfBi5N1yztdtsatPFm4G+A96TP39/Pbaz3b7bs/B8An8xjO4G/Bm5Lx7cDf1t23He/mzXa+Azw8+n4TuAzvfqz7MuefkQ8DpyocurzwH8EymenVwOPRMmTwLCkK4CVwN6IOBERbwF7gVVtvvUZqdPOalYD2yPiVES8CkwA16WPiYh4JSJ+DGxP1/aEGm38KLA5Ik6la46leF+2Eer/LCUJ+DCwLYXy1s4Apnu+7wNeT8d9+btZo41XA4+n473AL6XjnvtZ9mXSr0bSamAyIp6tODUKHCn7/GiK1Yr3g3vTn8NbJc1NsTy182rgQ5KekvQtST+X4nlqY7kPAW9GxMvp87y18+PAFklHgN8HNqV4ntp5kPNJ+5cpjTpAD7YxF0lf0mXAbwGf7Pa9dMCXgJ8BlgNvUBoWyJs5lP60vwHYAOxIveG8uoPzvfw8+ihwX0QsAO4Dvtzl+2mHO4F/L2kf8JPAj7t8PzXlIulTSoKLgGclvQZcCXxb0j8EJjn/rks6N1kn3tMi4s2IOBsR7wJ/TOnPRMhXO48CO9Of/U8D71LatCpPbQRA0hxgLfBoWThv7VwP7EzHf0YO/81GxKGIuDUirqX0Bv7ddKr32tjtSZFZTKYspPak2Gucn8j9RS6cLHo6zk8WvUppomhuOr682+1q1E7girLj+yiNFwJcw4UTRq9Qmiyak44XcX7C6Jput6tBG38D+HQ6vprSn8Hq5zbW+jdLaaz6WxWxXLUTeAm4KR3/ArAvHfft72aVNk4XG1wCPALc2as/y67/z2vyf/g2SkMbpyn1Cu+qOP8a55O+gAcpvfM+D4yVXXcnpYmVCeAj3W5XlnYC/z214zlgNxe+Cfx2audhUrVEit8OfCed++1utytDGy8FvgK8AHwbuKWf21jv3yzwMPAbVa7PTTuBfwzsS4ntKeDadG1f/m7WaONvpp/Ld4DNpN0OevFn6W0YzMwKJC9j+mZmloGTvplZgTjpm5kViJO+mVmBOOmbmRWIk76ZWYE46ZuZFcj/B9QKzThq7+DDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "composers_df = pd.read_excel('data/composers.xlsx', index_col = 'composer', sheet_name='Sheet5')\n",
    "plt.plot(composers_df.birth, composers_df.death, 'o')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Different types of plots are accessible when using the `plot` function of DataFrame instances via the `kind` option. The variables to plot are column names passed as keywords instead of whole series like in Matplotlib:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T15:03:52.577393Z",
     "start_time": "2020-06-24T15:03:52.310044Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEGCAYAAACUzrmNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAeR0lEQVR4nO3de5zddX3n8dd7LpkEgyEkkYVcmmgS2+BCxJHLplrAVRBX0jXqolZYsbK1wUdllZs3UGpXYrWrK9JGZClbS6SGSryyKAqVh1wmNMSEi46AzQwslxCBYDLXz/5xfpOcOXPO/M7MnN+5zfv5eMyDcz6/3/nN98sk88n3rojAzMxsPC21LoCZmdU/JwszM0vlZGFmZqmcLMzMLJWThZmZpWqrdQGyMH/+/Fi6dGmti2Fm1lC2bt36TEQsKHatKZPF0qVL6erqqnUxzMwaiqTflLrmbigzM0vlZGFmZqmcLMzMLJWThZmZpXKyMDOzVE4WZmZNYvfePu7f9Vt27+2r+LObcuqsmdl0c/O2Xi7evJ32lhYGhofZsO4Yzly9sGLPd8vCzKzB7d7bx8Wbt7N/YJgX+gbZPzDMRZu3V7SF4WRhZtbgevbso71l9K/z9pYWevbsq9j3cLIwM2twi+bOYmB4eFRsYHiYRXNnVex7OFmYmTW4ebM72LDuGGa2t3BoRxsz21vYsO4Y5s3uqNj3yGyAW9Ji4HrgCCCAjRHxJUmHA98ElgKPAe+MiD2SBHwJOAP4HfBfI+K+5FnnAJ9IHv2XEfH3WZXbzKwRnbl6IWuWz6dnzz4WzZ1V0UQB2bYsBoGPRMQq4ERgvaRVwCXAjyNiBfDj5D3Am4EVydd5wNUASXK5DDgBOB64TNLcDMttZtaQ5s3u4NjFh1U8UUCGySIinhhpGUTEC8CDwEJgLTDSMvh74I+T12uB6yPnLuAwSUcCpwG3RsSzEbEHuBU4Patym5nZWFUZs5C0FHg1cDdwREQ8kVz6f+S6qSCXSHblfawniZWKF36P8yR1Sep6+umnK1p+M7NaynKxXbkyX5QnaTawGfhwRDyfG5rIiYiQFJX4PhGxEdgI0NnZWZFnmpnVwu69fQfGHn7W/UzJxXb592XR9ZQv02QhqZ1covhGRNyUhJ+UdGREPJF0Mz2VxHuBxXkfX5TEeoGTC+I/zbLcZma1kr8Su39omKHhYQaHYT+5qbEXbd7OmuXzx00iWcisGyqZ3fR14MGI+GLepS3AOcnrc4Cb8+JnK+dE4Lmku+oW4E2S5iYD229KYmZmTaVwJXbfYC5R5GtvaWHn489nvmK7UJYtizXAe4FfSNqWxD4GfA64UdL7gd8A70yufZ/ctNluclNn3wcQEc9KugK4N7nvMxHxbIblNjOrut17+/jJQ0/R1qJx78stvgvaW1oOtDbg4IrtrLqjMksWEfEzoFSt31Dk/gDWl3jWtcC1lSudmVn9GOl6apV4sX+o6D0vmdHKUAQb1h3D0UfNyXzFdiHvOmtmVkP5XU+lzGpv5cLTVvLWYxceaDlsWHcMFxWMWWQ5yO1kYWZWQyObAOZ3KRXaNzDE//jBQxz+ko4Dg9hZr9gu5L2hzMyqoNRaiWKbALa3ihmto3vx+wZjzCB2liu2CzlZmJll7OZtvay58jb+5Jq7WXPlbWzZ1nvgWrFNAL/wjmO55pzXckh766jnVHrb8YlwN5SZWYbyxyQK10qMtAiKdSnt3tvHMKPXF2c9iD0etyzMzDJU7sFEhV1K1dh2fCLcsjAzy9BUDiaq9iD2eNyyMDObgrRN/qbaQqjmIPZ43LIwM5uk/H2cxtufqVQLoZobAU6Vk4WZ2SSUM3Cdb97sjlHxchNNvXA3lJnZJJQ7cF1M4YaB1dgIcKqcLMzMJmHR3FnsGxgcFds3MFjWwPVUEk2tOFmYmU1S/mFuxd6XMpUZUrXiZGFmNgk9e/Yxs230CuuZba1ltQ7qbQ1FOTzAbWY2CVNtHdTTGopyuGVhZjYJk20d5K/LqJc1FOVwy8LMbJIm2jpotOmy+dyyMDMrQ6mV2uW2Dhpxumw+tyzMzFJUokVQ7JCjrM/NriS3LMzMxlGsRfDRb22n+8kXJvScRpwum8/JwsxsHMUW0PUPDnPGl/9l1CFGaRpxumw+d0OZmY2jWIsAoH8oxt0LqphGmy6bzy0LM7NxjLQIZrSN/XUZw3FgEV7aVuX5z2uU6bL53LIwM0uU2jL8zNULOWrOTN7+d3eNur9vKHjJjNaGnhJbLicLMzPSZzy1t7Uys72F/QMHu6TaWsS9jz3Lp7/7QNlblTcqd0OZ2bRXzhqIYrOWBoeDS/95B0PDMSpe7zvIToaThZlNe+VsGX5g7KJ17M6yA0Ojk0UjTYktV2bJQtK1kp6StCMvdqykn0v6haTvSHpp3rVLJXVLeljSaXnx05NYt6RLsiqvmTW2cgeYi1k0dxb7B4dGxfYPDo35hX/m6oV87Iw/KPqM9lY15JTYcmU5ZnEd8BXg+rzYNcBHI+J2SecCFwKflLQKOAs4GjgK+JGklclnrgLeCPQA90raEhEPZFhuM2swlRhgjohx34/4w+Xzi8Zv+NMTaG9rbbgpseXKrGUREXcAzxaEVwJ3JK9vBdYlr9cCmyKiLyIeBbqB45Ov7oh4JCL6gU3JvWZmQHnjDcVaHfmxnj37mNU++t/Os9rbio47LD/iUM4+acmo2NknLaFz2byGnBJbrmrPhtpJ7pf9t4F3AIuT+EIgf05aTxID2FUQPyHjMppZAym251KrxE8eeopTfv9l/Kz7mTGtjoBRsU/+p1UT2orjM2v/PWefuJRtu37L6sWHsfyIQ7OsYl2odrI4F/iypE8CW4D+Sj1Y0nnAeQBLlixJudvMmkWxFdYv9g9x+Xd28vFv72BoeJjBYQ4kkwu/tR0I+gbjQOyK7z7AJ9+yiiu+98CopDJeK2H5EYdOiyQxoqrJIiIeAt4EkIxJvCW51MvBVgbAoiTGOPHCZ28ENgJ0dnYW72w0s6YzMkvpos3baZV4sT83UL23b6jo/a0tghBw8Hp7SwuvWjiHOy8+tSG34qiGqk6dlfSy5L8twCeAv00ubQHOktQhaRmwArgHuBdYIWmZpBnkBsG3VLPMZlb/zly9kDsvPpVPn3k0sztax713aDjoK5j5tG9g8ECCaOZxh6nIcursDcDPgVdK6pH0fuBdkn4JPAQ8DvxvgIjYCdwIPAD8EFgfEUMRMQicD9wCPAjcmNxrZjbGy17awcDQ6C6ploJlEWtXH0VLYRDY+fhzDXMQUS2o1PSwRtbZ2RldXV21LoaZVUn+1Nn9g0NEBLPa2+gfGmI4Ri+am9HWQnvLwe6qEYfMaGU4oin3dSqXpK0R0Vnsmldwm1lDK5w6OzAUtLa0cNV7juNrZ3cys210t1R7q8a0PgB+1z/UcEedVpOThZk1tGJbdcxobWHOrHaOPmrOmJlSQ8PBZW89mpntLRzSPnZ8oxn3daoEJwsza2jjHVda6nS695z4e9x58an87XtfQ0fBORXNuK9TJXiLcjNrOIXnToxMnS22RqLU6XTzZnfw+pUL+PzbS3/WDnKyMLOGUmofqPGOK503u6NkAmjko06rycnCzBpG/mB2sYOGJvuLfiqfnS48ZmFmDaOccycsG04WZtYwxhvMtmw5WZhZwyg1uwmY9MFHVh6PWZhZQykckP5Z9zOsufK2KR18ZOncsjCzhjOy4R+QevCRVYaThZnVvVLna3vAu3rcDWVmdW2887U94F09blmYWd1KO1+71IC310xUnlsWZla3ip2vPdLNlLadh1WWk4WZ1a1i3Uz9Q0M8t6+f3Xv7Ru3z5CSRLXdDmVndKuxmamuB4YD13/hX1lx5G1u29da6iNOGk4WZ1bWR87Wves9xtLa0MDAUniZbA04WZlYVpaa/lmPe7A7mzGpnRqunydaKxyzMLHPjTX8tl6fJ1pZbFmaWqbTpr+XyNNnacsvCzDJVzvTXcnmabO04WZhZpirdfeRpsrXhbigzy5S7j5qDWxZmljl3HzU+JwszG2X33r5Mfqm7+6ixOVmY2QGTmeKaVXKx+uJkYWbA6CmuIzOXLtq8nTXL55dMApVYP2GNIbMBbknXSnpK0o682GpJd0naJqlL0vFJXJK+LKlb0nZJx+V95hxJv0q+zsmqvGbTXc+efbRKo2LjrZDufvIFLvyWT6mbLrKcDXUdcHpBbAPw6YhYDXwqeQ/wZmBF8nUecDWApMOBy4ATgOOByyTNzbDMZtPWjt7neLF/aFSs1BTXm7f1csaX/4X+wdFTYr39RvPKLFlExB3As4Vh4KXJ6znA48nrtcD1kXMXcJikI4HTgFsj4tmI2APcytgEZGZTtHtvH1d874Ex8U++ZdWYLqiR7qr+oRhzv7ffaF7VHrP4MHCLpL8ml6j+QxJfCOzKu68niZWKjyHpPHKtEpYsWVLZUps1uWKrrF/S0cqrFs4p616AGa1i/cnLMy+r1Ua1F+V9ELggIhYDFwBfr9SDI2JjRHRGROeCBQsq9VizaaHYKuuh4SjaSih2b24z2GDjHY/4nIkmVe1kcQ5wU/L6n8iNQwD0Aovz7luUxErFzayCJrLKuvDejrYWBPQP4YHuJlbtbqjHgT8CfgqcCvwqiW8Bzpe0idxg9nMR8YSkW4C/yhvUfhNwaXWLbDY9TGSVdf69z+0bYP037uOFvsED1ye7UaDVr8yShaQbgJOB+ZJ6yM1q+gDwJUltwH6SMQbg+8AZQDfwO+B9ABHxrKQrgHuT+z4TEYWD5mZWIRNZZT1y7+69fT5nYhpQxNgZDY2us7Mzurq6al0Ms2ljy7ZeLvLivIYnaWtEdBa75hXcZjZl3iiw+TlZmFlFeKPA5ubzLMzMLFVZyULS25K9mZ6T9LykFyQ9n3XhzMysPpTbDbUBeGtEPJhlYcxsLG8BbvWg3GTxpBOFWfV5C3CrF+MmC0lvS152Sfom8G3gwLLMiLip6AfNbMqKnS9x4bfGP1/CLCtpLYu35r3+HbkV1COCg1t3mFmFFduwr29wmH+8+9/40BtW1LBkNh2Nmywi4n0AktZExJ351yStybJgZtPdormz6B8aHhP/nz/6Je8+YYlbF1ZV5U6d/V9lxsysQubN7uDcNUvHxIcCdj7uyYhWXWljFieRO3NigaT/nnfppUBrlgUzMzjpFfO4+vZHilxpvm16rL6ljVnMAGYn9x2aF38eeHtWhTKznKPmFN+Mr1TcLCtpYxa3A7dLui4iflOlMplZ4sX+IWa2t7B/4ODYxcz2ljFnZZtlrdx1Fr+T9HngaGDmSDAiTs2kVGYGUHKbb2//bdVW7gD3N4CHgGXAp4HHOHjGhJlNwe69fdy/67dFT5abyAl2E3222USU27KYFxFfl/QXeV1TThZmU1TOCu3Jbv/t1d9WSeW2LAaS/z4h6S2SXg0cnlGZzKaF/BXaaWdXz5vdwbGLD5tQi6LcZ5uVo9xk8ZeS5gAfAT4KXANckFmpzKaBkRXa+UbOrq7nZ9v0VFY3VER8N3n5HHBKdsUxmz4WzZ2V2dnVWT7bpqdyz7NYKenHknYk74+R9Ilsi2bW3KY6eF2rZ9v0pIj0laCSbgcuBP4uIl6dxHZExKsyLt+kdHZ2RldXV62LYVaWLM+r8FkYNhGStkZEZ7Fr5c6GOiQi7pGUHxuccsnMLNOzq30utlVKuQPcz0h6BcmGNJLeDjyRWanMzKyulNuyWA9sBH5fUi/wKPCezEplZmZ1JW3X2fydZr8P/IRca+RFYB3wxeyKZmZm9SKtZTGy0+wrgdcCNwMC3gvck2G5zMysjqTtOvtpAEl3AMdFxAvJ+8uB72VeOrMG4BlHNh2UO8B9BNCf974/iZUk6VpJT42szUhi35S0Lfl6TNK2vGuXSuqW9LCk0/LipyexbkmXlFles6q4eVsva668jT+55m7WXHkbW7b11rpIZpkod4D7euAeSf+cvP9j4LqUz1wHfCX5LAAR8V9GXkv6ArkV4UhaBZxFbgv0o4AfSVqZ3HoV8EagB7hX0paIeKDMcptlJn//pf3kVktftHk7a5bPdwvDmk652318VtIPgNclofdFxL+mfOYOSUuLXVNuwcY7gZHzMNYCmyKiD3hUUjdwfHKtOyIeST63KbnXycJqbmT/pZFEAQf3XyqWLNxdZY2s3JYFEXEfcF+Fvu/rgCcj4lfJ+4XAXXnXe5IYwK6C+AnFHijpPOA8gCVLllSomGalTWT/JW8Xbo2u3DGLSnsXcEMlHxgRGyOiMyI6FyxYUMlHmxVV7v5L3i7cmkHZLYtKkdQGvA14TV64F1ic935REmOcuFnNlXMw0US7q8zqUdWTBfAfgYcioicvtgX4R0lfJDfAvYLcOg4BKyQtI5ckzgLeXeXymo0rbf8lbxduzSCzbihJNwA/B14pqUfS+5NLZ1HQBRURO4EbyQ1c/xBYHxFDETEInA/cAjwI3Jjca9YwvF24NYOytihvNN6i3OqRZ0NZvavEFuVmNkXeLtwaWa1mQ5mZWQNxsjAzs1ROFmZmlsrJwszMUjlZmJlZKicLs3Hs3tvH/bt+6605bNrz1FmzErz5n9lBblmYFeHN/8xGc7IwK2Jk8798I5v/mU1HThZmRXjzP7PRnCzMivDmf2ajeYDb6lI9bLpXzlkVZtOFk4XVnYnOQsoysXjzP7McJwurK/mzkEZOlrto83bWLJ9f9Je2p7eaVYfHLKyuTGQWkqe3mlWPk4XVlYnMQvL0VrPqcbKwujKRWUie3mpWPR6zsLoxMlC9Zvl87rz41NRB65HEclHBmIUHpM0qz8nC6sJkB6o9vdWsOtwNZZmYyG6tUx2onje7g2MXH+ZEYZYhtyys4ibaShgZqB6ZKgsHB6qdAMzqg1sWVlGTaSV4oNqs/jlZWEVNZjpr4QyojrYW1p+8POuimtkEOFlYRU22lXDm6oXcefGpfOD1LweCjXc8wporb2PLtt4MS2tm5XKysIqa6m6tX/1pN32D4RXZZnXGA9xWcZOdzuqBbrP65WRhmZjMbq0e6DarX5l1Q0m6VtJTknYUxD8k6SFJOyVtyItfKqlb0sOSTsuLn57EuiVdklV5rfZ84JBZ/cqyZXEd8BXg+pGApFOAtcCxEdEn6WVJfBVwFnA0cBTwI0krk49dBbwR6AHulbQlIh7IsNxWQ16RbVafMksWEXGHpKUF4Q8Cn4uIvuSep5L4WmBTEn9UUjdwfHKtOyIeAZC0KbnXyaKGCg8bqvThQz5wyKz+VHvMYiXwOkmfBfYDH42Ie4GFwF159/UkMYBdBfETij1Y0nnAeQBLliypcLFtROHq7He+ZhE3bu3x4UNmTa7aU2fbgMOBE4ELgRslqRIPjoiNEdEZEZ0LFiyoxCOtQLHV2dff9W8+fMhsGqh2sugBboqce4BhYD7QCyzOu29REisVtxootjq7kA8fMmtO1U4W3wZOAUgGsGcAzwBbgLMkdUhaBqwA7gHuBVZIWiZpBrlB8C1VLrMlik1tLeSprmbNKcupszcAPwdeKalH0vuBa4GXJ9NpNwHnJK2MncCN5Aaufwisj4ihiBgEzgduAR4EbkzutRooNrX17JOWeKqr2TSgiKh1GSqus7Mzurq6al2MppX1bCgzqw1JWyOis9g1r+C2CSuc2uqprmbNzxsJmplZKicLMzNL5WRhZmapnCzMzCyVk4WZmaVysjAzs1ROFmZmlsrJwszMUjlZmJlZKicLMzNL5WRhZmapnCzMzCyVk4WZmaVysjAzs1ROFmZmlsrJwszMUjlZmJlZKicLMzNL5WRhZmapnCzMzCyVk4WZmaVysjAzs1ROFmZmlsrJwszMUjlZmJlZKicLMzNLlVmykHStpKck7ciLXS6pV9K25OuMvGuXSuqW9LCk0/LipyexbkmXZFVeMzMrLcuWxXXA6UXifxMRq5Ov7wNIWgWcBRydfOarkloltQJXAW8GVgHvSu41M7MqasvqwRFxh6SlZd6+FtgUEX3Ao5K6geOTa90R8QiApE3JvQ9UuLhmZjaOWoxZnC9pe9JNNTeJLQR25d3Tk8RKxceQdJ6kLkldTz/9dBblNjObtqqdLK4GXgGsBp4AvlCpB0fExojojIjOBQsWVOqxZmZGht1QxUTEkyOvJX0N+G7ythdYnHfroiTGOHEzM6uSqrYsJB2Z9/Y/AyMzpbYAZ0nqkLQMWAHcA9wLrJC0TNIMcoPgW6pZZjMzy7BlIekG4GRgvqQe4DLgZEmrgQAeA/4bQETslHQjuYHrQWB9RAwlzzkfuAVoBa6NiJ1ZldnMzIpTRNS6DBXX2dkZXV1dtS6GmVlDkbQ1IjqLXfMKbjMzS+VkUWD33j7u3/Vbdu/tq3VRzMzqRlVnQ9W7m7f1cvHm7bS3tDAwPMyGdcdw5uqiyzrMzKYVtywSu/f2cfHm7ewfGOaFvkH2Dwxz0ebtbmGYmeFkcUDPnn20t4z+39He0kLPnn01KpGZWf1wskgsmjuLgeHhUbGB4WEWzZ1VoxKZmdUPJ4vEvNkdbFh3DDPbWzi0o42Z7S1sWHcM82Z31LpoZmY15wHuPGeuXsia5fPp2bOPRXNnOVGYmSWcLArMm93hJGFmVsDdUGZmlsrJwszMUjlZmJlZKicLMzNL5WRhZmapmnKLcklPA7+Z4mPmA89UoDj1zvVsHtOhjuB6Zun3IqLoudRNmSwqQVJXqX3dm4nr2TymQx3B9awVd0OZmVkqJwszM0vlZFHaxloXoEpcz+YxHeoIrmdNeMzCzMxSuWVhZmapnCzMzCzVtEoWkq6V9JSkHUWufURSSJqfvJekL0vqlrRd0nF5954j6VfJ1znVrEM5itVT0uWSeiVtS77OyLt2aVLPhyWdlhc/PYl1S7qk2vUYT6mfpaQPSXpI0k5JG/LiDVdHKPmz/Gbez/ExSdvyrjVTPVdLuiupZ5ek45N4Q/7dLFHHYyX9XNIvJH1H0kvzrtXXzzIips0X8HrgOGBHQXwxcAu5hXzzk9gZwA8AAScCdyfxw4FHkv/OTV7PrXXd0uoJXA58tMi9q4D7gQ5gGfBroDX5+jXwcmBGcs+qWtctpY6nAD8COpL3L2vkOo73Zzbv+heATzVjPYH/C7w5eX0G8NO81w33d7NEHe8F/ih5fS5wRb3+LKdVyyIi7gCeLXLpb4CLgPzR/rXA9ZFzF3CYpCOB04BbI+LZiNgD3AqcnnHRJ2ScehazFtgUEX0R8SjQDRyffHVHxCMR0Q9sSu6tCyXq+EHgcxHRl9zzVBJvyDrC+D9LSQLeCdyQhJqtngGM/Et7DvB48roh/26WqONK4I7k9a3AuuR13f0sp1WyKEbSWqA3Iu4vuLQQ2JX3vieJlYo3gvOTZvu1kuYmsWaq50rgdZLulnS7pNcm8WaqY77XAU9GxK+S981Wzw8Dn5e0C/hr4NIk3kz13MnBX/bvINfLAXVYx2mdLCQdAnwM+FSty1IFVwOvAFYDT5Drvmg2beS6IE4ELgRuTP713azexcFWRTP6IHBBRCwGLgC+XuPyZOFc4M8lbQUOBfprXJ6SpnWyIPfLcxlwv6THgEXAfZL+HdDLwSxPcq13nHhdi4gnI2IoIoaBr5FrzkJz1bMHuCnpnrgHGCa3GVsz1REASW3A24Bv5oWbrZ7nADclr/+JJvwzGxEPRcSbIuI15BL/r5NL9VfHWg/6VPsLWErpwcLHODjA/RZGD6Ldk8QPBx4lN4A2N3l9eK3rlVZP4Mi81xeQ6w8FOJrRA2mPkBtEa0teL+PgQNrRta5XSh3/DPhM8noluea6GrmOpf7MkuuLv70g1lT1BB4ETk5evwHYmrxu2L+bReo4MgmjBbgeOLdef5Y1/59X5R/UDeS6YAbI/Sv0/QXXH+NgshBwFblM/wugM+++c8kNOHUD76t1vcqpJ/B/knpsB7YwOnl8PKnnwySzT5L4GcAvk2sfr3W9yqjjDOAfgB3AfcCpjVzH8f7MAtcBf1bk/qapJ/CHwNbkF+LdwGuSexvy72aJOv5F8nP5JfA5kl016vFn6e0+zMws1XQfszAzszI4WZiZWSonCzMzS+VkYWZmqZwszMwslZOFWQVIWlpiN+NrJK0q8ZkPJ7sIjLzfm2UZzabCycIsQxHxpxHxQGFcUiu5vY8OGfsps/rjZGFWOW2SviHpQUnfknSIpJ9K6oRcy0HSFyTdT27B1VHATyT9ZOQBkj4r6f7kHIcjalQPszGcLMwq55XAVyPiD4DngT8vuP4ScmcvHBsRnyG35fYpEXFK3vW7IuJYcttWf6BK5TZL5WRhVjm7IuLO5PU/kNuuIt8QsHmcz/cD301ebyW3j5BZXXCyMKucwr1zCt/vj4ihcT4/EAf33xkit2mcWV1wsjCrnCWSTkpevxv4Wcr9L5A7w8Cs7jlZmFXOw8B6SQ+S2yL76pT7NwI/zB/gNqtX3nXWzMxSuWVhZmapnCzMzCyVk4WZmaVysjAzs1ROFmZmlsrJwszMUjlZmJlZqv8PjjhbWW7D1yAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "composers_df.plot(x = 'birth', y = 'death', kind = 'scatter')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T15:04:46.851228Z",
     "start_time": "2020-06-24T15:04:46.568575Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEbCAYAAADnH5IjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3hddZ3v8fcnaXqjF0KBOkAVsCI3oWrnQayjBXQQ5OLDZWBEHZlxGGbgKMNIUUbmVJQjdIbLHJkZQDzj8aggtxFBkOEWkLsgpQItNy30Agg1vaQNbZp8zx9r7XZlZSfZabKy904+r+fZT7LW+q7fXr/uNN/81u+yFBGYmZkNtYZqX4CZmY1MTjBmZlYIJxgzMyuEE4yZmRXCCcbMzArhBGNmZoVwgjEbRpJC0sxejp0i6b8HWN5cScuH5uoGRtIXJD04gPilkj5e0LUUVrZtOycYG1aSPiPpCUltkl6TdIekj1T7umpBRPwoIv60r5i+EtRoIen7kr5V7euw/jnB2LCRdDZwOfC/gOnAO4F/B46t5nUVQdKYWi7PbDg4wdiwkDQVuAA4IyJujoj1EdEREbdGxDlpzDhJl0tamb4ulzQuPTZX0nJJ8yT9Pm39fFrSkZJekPQHSedl3m++pBsl/UTSOkm/lnRg5vg+klokrZb0rKRjMseOlPRcet4KSV/JHDtK0sL0vIclHZA5tlTSuZIWAev7SApHSvqtpLck/bOkhvT8brec0tbKGZJeBF6U9EB66Om0BXhSJvYfMv8up/bxOZwqaXFat99K+pvMsdK/cdmyJE2T9DNJayU9Dry7t/dJ4z8n6RVJqyT9Y+5Yg6SvSno5PX69pB0yx2+Q9LqkNZIekLRfuv804BRgXvpvcGum2FmSFqXn/ETS+L6uz4ZBRPjlV+Ev4JPAZmBMHzEXAI8COwM7AQ8D30yPzU3P/yegCfhr4E3gx8BkYD+gHdgjjZ8PdAAnpPFfAX6Xft8EvAScB4wFDgXWAe9Nz30N+JP0+2bgA+n37wd+DxwENAJ/ASwFxqXHlwILgRnAhF7qGMB9wA4kLbgXgC+mx74APJiLvSuNnZDZNzMTU/p3uSCt15HABqC5l/f/FEliEPCxNPYDlZQFXAdcD2wH7A+syF5v7n32BdqAjwLjgEvTsj+eHv9y+lnvlh6/Crg2c/5fpp/rOJJW78LMse8D38q931LgcWCX9N9rMXB6tX/uR/ur6hfg1+h4kfzV+Xo/MS8DR2a2DweWpt/PJUkgjen25PSX7UGZ+CeBT6ffzwcezRxrIE0c6et1oCFz/Fpgfvr9q8DfAFNy1/cfpAkvs+954GPp90uBv+ynjgF8MrP9d8A96fdfoGeCObTM+fkE004mcZMkwQ9V+Ln8FPhyf2WRJNQOYO/Msf/VR4L5J+C6zPZ2wKZMglkMHJY5/kdp+T3+AAG2T+s9Nd3uLcF8NrO9ALiy2j/3o/3lW2Q2XFYBO/bTl7AL8Epm+5V035YyIqIz/b49/fpG5ng7MCmzvaz0TUR0AcvT8nYBlqX7su+1a/r98SR/vb8i6X5JB6f73wX8Q3p7bLWk1SStlew1LqN/2Zh8HfuK7c2qiNic2d5A93+HLSQdIenR9JbiapJ67lhBWTsBY8pce292ofu//3qSn4GSdwH/lfl3XAx0AtMlNUq6KL19tpYkeZC7znJeL3PdVkVOMDZcHgE2Ap/uI2YlyS+eknem+7bVjNI3aT/Hbml5K4EZpb6PzHutAIiIX0XEsSS36n5KclsIkl+YF0bE9pnXxIi4NlNOJcuTz8h8318dh2y587Q/6ybgX4DpEbE9cDvJ7bL+vElyiyt/7b15je7//hOBaZnjy4Ajcv+W4yNiBfAZkoEfHwemAruXikm/egn4OuEEY8MiItaQ3Db5t7RzfqKkpvQv6gVp2LXA1yXtJGnHNP6Hg3jbD0o6Lm01nUWS4B4FHiP5C3deeg1zgaOB6ySNVTIfZWpEdABrgVJL57vA6ZIOUmI7SZ+SNHmA13WOpGZJM0j6In4ygHPfAPYc4PuVjCXp03gT2CzpCKDPYdElacvxZmB++tntS9IH1ZsbgaMkfUTSWJJ+nezvmyuBCyW9CyD9zEujCSeTfFargIkkt+KyBvNvYMPICcaGTURcApwNfJ3kl9wy4EySVgLAt4AngEXAb4Bfp/u21S3ASUAr8DnguEhGrm0iSShHAG+RDJX+fEQsSc/7HLA0vT1zOkn/ERHxBMnggivSMl8i6TfZlut6kmRAwM+B7w3g3PnA/01vLf3ZQN40ItYBXyJpkbWStBR+NoAiziS57fQ6ST/If/bxXs8CZ5AMwngtfb/shNB/Td/7vyWtI0n8B6XHfkBy+20F8Fx6LOt7wL7pv8FPsZqltEPMbESRNJ+kM/yz1b4Ws9HKLRgzMyuEE4yZmRXCt8jMzKwQbsGYmVkhvIBeascdd4zdd999UGWsX7+e7bbbbmguqIa5niPHaKgjjI56VquOTz755FsRsVO5Y04wqd13350nnnhiUGW0tLQwd+7cobmgGuZ6jhyjoY4wOupZrTpK6nVFB98iMzOzQjjBmJlZIZxgzMysEE4wZmZWCCcYMzMrhBOMmdkotqptI08vW82qto1DXraHKZuZjVK3LFzBuTctoqmhgY6uLhYcfwDHzNq1/xMr5BaMmdkotKptI+fetIi3O7pYt3Ezb3d0Me+mRUPaknGCMTMbhZa3ttPU0D0FNDU0sLy1vZczBq6wBCPpREk/k7RCUpukJyX9eZm4v5b0oqS305jDysTsKum/JK2T9JakK9JHsA64LDMzg92aJ9DR1dVtX0dXF7s1Txiy9yiyBXM20Ab8PXAMcB/wY0n/oxSQJpwrSZ5gdwTwLHCbpP0zMU3AnSTPaj+Z5BGzJwJXZ9+skrLMzCwxbdI4Fhx/AOObGpg8bgzjmxpYcPwBTJs0bsjeo8hO/qMj4q3M9r2SdiFJPN9J980H/m9EfBNA0v3A+4GvAqUnEZ4A7EPydMLfpXEdJM9P/0ZEvDiAsszMLHXMrF2ZM3NHlre2s1vzhCFNLlBgCyaXXEqeAnYBkLQnsBfJ88FL53QBN5C0QEqOAH5VSi6pnwKbgE8OsCwzM8uYNmkcB87YfsiTCwx/J//BwAvp93unX5fkYhYDO0jaKRPXLSYiNgEvZ8qotCwzMxsmwzYPJu1w/zTwl+mu5vTr6lxoa+b4m+nXfEwprjkTW0lZ+Ws6DTgNYPr06bS0tPRXjT61tbUNuox64HqOHKOhjjA66pmvY2dXsKmzi7GNDTQ2qCrXNCwJRtLuwI+BWyLi+8PxnpWIiKtJBwvMnj07BvsshdHwzAlwPUeS0VBHGPn1XNW2kccfeZD3zT6YaZPG9TmBclXbxsL6XPIKv0UmaQfgDuAV4JTMoVLrYmrulObc8dYyMaW41lxsf2WZmY0otyxcwZyL7+V3b65nzsX38qPHXul1AmUp9rPXPMaci+/lZwtXFHpthSaYdK7KbcBY4KiI2JA5XOov2Tt32t7AHyLizUxctxhJY4E9M2VUWpaZ2YiRnY3fGcHbHV3M/9mzPX6xNzU08OzKtYXP3M8rcqLlGJJRXO8BPhkRv88ej4jfknT4n5g5pyHdviMTegfwx5Leldl3DDAO+MUAyzIzGzGeXbmGBnXvX+noDDZ09JxACVH4zP28Ivtg/h04kmRi5DRJ0zLHnoqIjSRzV34oaSnwEPAXJAnpM5nYG4F/BG6WdD7JbbDLgB9n5sBQYVlmZiPCLQtXMO/Gp9m4OXqN2W5sI50RLDj+APbbZWrhM/fzikwwf5p+/dcyx/YAlkbEtZImAecC55PMvj8qIp4pBUZEh6RPAleQzHPZCFwHnJMtsJKyzMxGgtKtsb6Sy4SmRs45fC+OPnDXLZ35C44/gHm5zv8iO/oLSzARsXuFcd8FvttPzHKSIc6DLsvMrN6VFqp8m65eY9o7Ovn2HUvYYbtxW0aQFT1zP8+rKZuZ1ajeHgZWbqFKSYxt7N4fs3Fz9OjIL3Lmfp4TjJlZDeprSHG5hSpnNE/gmr/4YyY2NXYrp+iO/L74iZZmZjUmO/y4dBts3k2LmDNzxy0tj/ztrt888Qj77TKFLrr3yxTdkd8Xt2DMzGpMpQ8Dy9/uGo4l+AfCLRgzsxozmIeBDXdHfl/cgjEzG2a9dd6XDLYlMpwd+X1xC8bMbBj1tRBlVm8tkeFcrHKwnGDMzIZJJZ33WdMmjeu2v9LkVCt8i8zMbJhU2nlfTjY5DddilYPlBGNmNkx2a55Ae8fmbvvaOzZX1Hk/mORULU4wZmbDSLnVj/PbvRnMyLJqcYIxMxsmy1vbGT+m+0z78WMaK2qF1Nocl0q4k9/MbJgMthVSS3NcKuEWjJnZMNnWVkh23kytzHGphFswZmbDaKCtkHobmpzlFoyZWUF6m7FfaSukHocmZ7kFY2ZWgKFoeZR7sFhpaHI93CJzC8bMbIiVa3l85cZFvPTGugGVU49Dk7OcYMzMhli5SZGbNndx5P/+ZbcHh/WnHocmZ/kWmZnZECvX8gDY1Bl9rj1WTr0NTc5yC8bMbIiVWh5jx/T8FRtdwfLW9n6X7M+XVy9Dk7PcgjEzG4Tels8/Ztau7DJ1PCdc9Wi3+I2dwaO/XcVld79Ql0OPB8IJxsxsG/U3UqxpTCPjmxp4uyM7CgwuuesFNm2ubMn+elboLTJJMyVdJWmRpE5JLWVitpf0fyT9QVKbpDskzSwTt6+keyRtkLRS0gWSGnMxknSepGWS2iU9IGlWgVU0s1Gqkjkq5UZ7dXRBV1d021frqyJvq6L7YPYDjgSeB17oJeYnwOHAl4HPANOAeyRNKQVIagbuBgI4FrgA+AfgG7myvgqcD1wMHA20AXdLescQ1cfMDEhGio1p6L4Scj5RTJs0jrM/sVePczfnEkw9DT0eiKJvkd0aEbcASLoR2DF7UNLBwJ8CH4+Ie9J9jwG/A04D/iUNPR2YABwXEWuBu9IENF/SgohYK2k8SYL5dkRckZb1CLAUOBP4eqE1NbO6M5jHDz+zYg1tGzu77SuXKHaYOLbs+U2NYvyYxi231kba7TEoOMFERM9xet3NAjqAlsw5b0h6GvgUWxPMEcCdaXIpuY6kpfIx4Fbgw8AU4PpMWesl3Zqe7wRjZlsMZqb9qraNfPPnz/XYf/5R+/ZIFLNmbF+2jGu/eBBNYxrrbujxQFR7mPJ4oDMiOnP7NwH7ZLb3BpZkAyLiVWBDeqwU0wm8mCtrcSbGzKyi/pNyw4hL+55dubbHRMrtxjay/y5Te7zXzOmT+fzB7+y27/MHv5PZe0yry6HHA1HtUWQvAeMlvS8ifgMgaQKwPzA5E9cMrC5zfmt6rBTTViZZtQITJY2NiE1DevVmVpfKrfHVKHHfkt9zyN478+BLb/Vo3QRs2beps4vO3ETKzohe+1EuOPZ9fP5Du7Nw2WpmzdiemdMnl40baRQR/UcNxRulfTARMTezbyxJy+R14FRgLXARcAqwOSLGp3EdwDkRcXmuzOXADyLiPEn/mMZsn4v5IvBdYFw+wUg6jaSvh+nTp3/wuuuuG1Qd29ramDRp0qDKqAeu58gxGuoIPevZ2RUseX0dXbnff40SAUQE2SMN6WONs/GSECCS0Ue7NU9g6oSmwurQn2p9locccsiTETG73LGqtmAiYpOkk4Fr2XoL7EHgB8ChmdBWoGfbM2m1tGZiJklqzLVimoEN5VovEXE1cDXA7NmzY+7cuYOoDbS0tDDYMuqB6zlyjIY6Qvl6rl+4gnk3LaJRYv2m/I2P7iaObYSADR1b4yaPG8O/nfJ+pk4YWxP9KLX4WVb7FhkR8Xg672UvklbLy5JuA7LTX5eQ60eRNAOYyNbEtARoBGaSDIsu6dF/Y2ZWWuPrviW/Z/6tz/YYEZbV2RVs7ux+S6y9YzP77TK16omlllW7kx+ASDyfJpf3AB8HvpcJuQM4XFL2xuVJQDtwf7r9MMktthNLAZImksyHuaPI6zez+rXzlHF05JJHbnoLx87ahYb8TuDZlWvq5uFf1VBoCyb9BX9kurkrMEXSCen27RGxQdL5JC2Mt4D3kUyUvC4i7soUdSXwJeBmSRcDewLzgUtLQ5cj4m1JFwHnS2pNyzybJIl+p8Bqmlkdyg5T7goY0wATmsawqbOTroCuzq39LT9duJKxjQ10dG5t5WzugtN/+Gu6IkbsWmKDVfQtsp2BG3L7Stt7kEyCnAZcTjIJcxnJ3JdLsidERKukw4ArSOa8rAYuI0kyWReRJJSvpeU+AXwiIt4YktqY2YiQHaZcGkk2bkwD/3bKB4DgjB89RUfn5i3xTY2iY3PPaX0b0r6bkbqW2GAVPdFyKckgi75izgLOqqCs5+je8V8uJoAL05eZWVnlhimPbWxg6oSmss9y6ewK/ufR+/HNnz9HA+rW2Q/19Rjj4VQTfTBmZsOpr0cR9/YUyVM+9C4eOvdQrvzcBxmXe87LSF1LbLCqPorMzGw45NcdW3D8AczLTaYstUB6e4rktEnj+OheO/HPJ/R+rm3lBGNmI96a9g7mXHxvj3XH+noU8bRJ43pNGvX8GOPh5ARjZiNaqeXydkdj2Qd8bWtyGMy5o4X7YMxsRFve2t5jpNFIfcBXrXGCMbMRbbfmCeRXXHSn/PBwgjGzEW3apHHs1jyhx6gwoMdy/Da03AdjZiPe1AlNPHTuR7d0yj/40ltlO/1taLkFY2ajwrRJ4zgwfbpkfw8bs6HhBGNmI1K5J1LC1ln8We70L4ZvkZnZiJNdyLKjq4uLP7z1V11fs/htaLkFY2YjSnYhy9ItsOWt7VtaMr0tBeM5LUPPLRgzG1HKLWSpdH9/S8HY0HKCMbMRpdwtsC5gu7GN3fZ5Jn7xfIvMzEaU7C2wcY3JHH4BR13xID9buKK6FzfKOMGY2YhzzKxdue3MjxBKEkxXhIcjV4ETjJnVrN6GGldi/aZOxjV6OHI1uQ/GzGpSfqjxQGfbezhy9bkFY2Y1p9xQ44He3sr2xTRKHo5cBW7BmFnNKTfUeFuee18ajvz4Iw/y0DEfcXIZZm7BmFnNGcrbW9MmjWNCU6OTSxU4wZhZzfFs+5HBt8jMrCZ5tn39c4Ixs0ErPfd+qBOBZ9vXt0JvkUmaKekqSYskdUpqKRPzR5L+U9IKSW2SnpJ0Spm4fSXdI2mDpJWSLpDUmIuRpPMkLZPULukBSbMKrKLZqHfLwhXMufhePnvNY8y5+N6KZssPZn6L1Y+iWzD7AUcCjwJN+YOSGoCfAdOAecDrwAnADyW1R8TNaVwzcDfwHHAs8G7gEpIE+fVMkV8FzgfOAZYAZwN3S9o/Il4vooJmo1l2OHFpxNe8mxYxZ+aOvbY8Bju/xepH0Z38t0bEjIg4EXi2zPG9gNnAlyPiRxFxT0T8LfAUcFIm7nRgAnBcRNwVEVcC3wDOljQFQNJ4kgTz7Yi4IiLuBk4EAjizqAqajWbLW9tpTJdjKelttvyqto088MKbzLvRT5McLQpNMBHR1U9IqVWzJrd/Ncn6dCVHAHdGxNrMvutIks7H0u0PA1OA6zPvvx64NT3fzIbYMyvWsH5TZ7d95YYTl26jnf7/nmTj5u6/Frx8y8hV7WHKzwCPARdIeo+kKZK+AMwBrszE7U1yy2uLiHgV2JAeK8V0Ai/m3mNxJsbMhsiqto188+fP9dh//qf27XZ7LHsbbUNHZ494L98yclV1FFlEhKQjgFuAF9LdHcCpEXFvJrSZpFWT15oeK8W0RUT+J7gVmChpbERsyh6QdBpwGsD06dNpaWkZTHVoa2sbdBn1wPUcOQZTx/aOTr60TwedEVv2NUjssO4lWlp+12dcKTaAXaY28ZsnHtmma6iUP8vqqGqCSTv5f0DSyX8S8HuSQQHfk7QqIn5R5PtHxNXA1QCzZ8+OuXPnDqq8lpYWBltGPXA9R47B1HFV20b+/uJ7ebtj6y2v8U0NPZZkKRc3pgEkGNvYSGd0suD49xTa0e/PsjqqfYvsqPT16Yi4PiJaImIe8F/AgkxcKzC1zPnN6bFSzKT80OU0ZkO+9WJmg1PpbPt83Lgxya+djs5kSX139I9c1Z5ouTfJL/98v8lTwDGZ7SXk+lEkzQAmsrVvZgnQCMwEns+9R7f+GzMbGpXOts/GrWnv4Iwf/Zp1GzdvOb4tC1la7at2C+YVkv6R9+b2fxBYmtm+Azhc0uTMvpOAduD+dPthYC3J0GQAJE0Ejk7PN7MCTJs0jgNnbN9vcijF7bfLFD+nZZQoeib/REknSDoB2BXYqbSd/vK/HXgV+KmkP5f0cUmXAX8G/FumqCuBjcDNacxpwHzg0tLQ5Yh4G7gIOE/SGZIOA25I6/idIutpZpXzQpajR9G3yHYm+SWfVdreIyKWpong2yQz86cAL5NMrLy6dEJEtKZxV5DMa1kNXEaSZLIuIkkoXyMZOPAE8ImIeGMI62Rmg+SFLEeHQhNMRCyl+4TJcjEvkbmt1Ufcc8Ch/cQEcGH6MrMa5oUsR75q98GYmdkIVVGCkXScpBclrZG0VtI6SWv7P9PMzEarSm+RLQCOjojFRV6MmQ1eUc9mMRuoShPMG04uZrXPS+FbLekzwUg6Lv32CUk/AX5KMlwYgNLzWsys+so9m+UrNy5i3z+awszpk/s522zo9dcHc3T6mkKycvGfZvYdVeylmdlALG9tp6mh+3/pTZu7OOJfH6joKZNmQ63PFkxEnAogaU5EPJQ9JmlOkRdmZgOzW/OEHjPkATq64OzrF/b5lEmzIlQ6TLncTHjPjjerIaUZ8k2NPaeebe6CZ1d64KcNr/76YA4meVLkTpLOzhyaQrKwpJnVkGNm7UqDxJnXPlXmaJTZZ1ac/kaRjQUmpXHZXsK1wAlFXZSZbbu931G+Q3+XqV5M0oZXf30w9wP3S/p+RLwyTNdkZoOwflMn45saejwIbP2mno8rNitSpfNgNkj6Z2A/YHxpZ0T0uTaYmQ2/3pa993L4Ntwq7eT/EclDu/YAvkHyrJZfFXRNZtaPVW0beXrZ6rJPgRzscvh9lW02EJW2YKZFxPckfTlz28wJxqwKKpmtv63L4XslABtKlbZgOtKvr0n6lKT3AzsUdE1m1ovsbP11Gzf3+Tz7Sp80uS1lm1Wi0gTzLUlTgX8AvgJcA/x9YVdlZmWVm61fep59LZdto1NFt8gi4rb02zXAIcVdjpn1pdxs/aF6nn2RZdvoVOnzYPaSdI+kZ9LtAyR9vdhLM7O8Ip9nX2TZNjpV2sn/XeAc4CqAiFgk6cfAt4q6MDMrr8jn2RdZto0+lSaYiRHxuNRtjaPNBVyPmVWgyOfZF1m2jS6VdvK/JendpIsZSToBeK2wqzIzs7pXaQvmDOBqYG9JK4DfAacUdlVmZlb3+ltNObuC8u3AfSStnvXA8cClxV2amZnVs/5ukU1OX7OBvwWage2B04EP9Fe4pJmSrpK0SFKnpJbc8bmSopfXnbnYfdORbBskrZR0gaTGXIwknSdpmaR2SQ9ImtXvv4KZmQ25/lZT/gaApAeAD0TEunR7PvDzCsrfDzgSeBRoKnP818DBuX3vBH4C3FHaIakZuBt4DjgWeDdwCUmCzA6X/ipwPsmItyXA2cDdkvaPiNcruF6zYbGqbaNHatmIV2kfzHRgU2Z7U7qvP7dGxC0Akm4EdswejIi1JMlnC0l/AnQB12d2nw5MAI5Lz7lL0hRgvqQFEbFW0niSBPPtiLgiLesRkoU5z6R7IjKrGq/3ZaNFpaPIfgA8Lml+2np5DPh+fydFRM8HhPfvz4H7I2JlZt8RwJ1pcim5jiTpfCzd/jDJkza3JKaIWA/cmp5vVnVe78tGk4oSTERcCJwKtKavUyPi20N9MZL2At4PXJs7tDfJLa/sNb0KbEiPlWI6gRdz5y7OxJhV1UDX+/LS+VbPKr1FRkT8mqTPpEgnk6zcfFNufzOwukx8a3qsFNMWEfnH9rUCEyWNjYjsbT4knQacBjB9+nRaWloGdfFtbW2DLqMeuJ7brrMrOP29G+mK2LKvQZ289eJTtLzcbSIza9o7WN7ajkgmoO3WPIGpE8p1ZW47f5YjRy3WseIEM0xOBv47Iv4wHG8WEVeTzO9h9uzZMXfu3EGV19LSwmDLqAeu5+CsX7iCebk+mMNyfTCr2jYy5+J7ebtj60DJ8U2beejcjw7poAB/liNHLdaxZhKMpAOBfYALyxxuBaaW2d+cHivFTJLUmGvFNAMb8q0Xs2qpZL2v0q20t9najVm6leZRZ1YvaibBkLRe2oFbyhxbQq4fRdIMYCJb+2aWAI3ATOD5TGiP/huzautvvS8vnW8jQaWjyIbDySTDmtvKHLsDOFzS5My+k0gS0v3p9sPAWuDEUoCkicDRZObUmNUDL51vI0GhLZj0F/yR6eauwJR0oUyA2yNiQxr3IWB3en9K5pXAl4CbJV0M7AnMBy4tDV2OiLclXQScL6mVrRMtG4DvDHHVzArnpfOt3hV9i2xn4IbcvtL2HiSTICFpvayhl5ZGRLRKOgy4gmRey2rgMpIkk3URSUL5GjANeAL4RES8MZhKmFWLl863elZogomIpYAqiDsLOKufmOeAQ/uJCZJBAuUGCpiZ2TCqpT4YMzMbQZxgzMysEE4wZmZWCCcYMzMrhBOM2RDzApVmiVqayW9W9/ysF7Ot3IIxGyJ+1otZd04wZkNkoM96MRvpnGDMhogXqDTrzgnGbIh4gUqz7tzJbyPGqraNVV8Y0gtUmm3lBGMjwkBHbxWZjLxApVnCCcbqXnb0VukJkPNuWsScmTuW/UXvocRmw8N9MFb3BjJ6y0OJzYaPE4zVvYGM3vJQYrPh4wRjdW8go7c8lNhs+LgPxupaqbN+zswdeejcQ/vtuC8lo3m5Phh3ypsNPScYq1vb2lnvocRmw8O3yKxmDGQV4sF21k+bNI4DZ2zv5GJWILdgrCYMtDVS6qwvDUuGrZ31ThpmtcEtGKu6bWmNuLPerPY5wVjVbcvQ4fzIsXFjGjhj7syiL9XMBsAJxqpuW1sjx8zalYfOPZS//uieQHD1A79lzsX38rOFKwq8WjOrlBOMVd1gVyH+95aX2Lg5PDPfrPD+HvgAAA9YSURBVMYUmmAkzZR0laRFkjoltfQS9z5Jt0laI2mdpMclfTAXs6+keyRtkLRS0gWSGnMxknSepGWS2iU9IGlWgVW0IVJqjfzwiwfx0LmHVrw2mGfmm9WuokeR7QccCTwKNJULSBPAL4FbgJPS3X8MTMjENAN3A88BxwLvBi4hSZBfzxT3VeB84BxgCXA2cLek/SPi9SGrlRViW1Yhdme/We0qOsHcGhG3AEi6EdixTMyVadxnM/t+kYs5nSThHBcRa4G7JE0B5ktaEBFrJY0nSTDfjogr0vd8BFgKnEn3RGQjhGfmm9WuQhNMRHT1dVzSvsBBwN/3U9QRwJ1pcim5DrgY+BhwK/BhYApwfeb910u6NT3fCWaE8sx8s9pU7U7+g9KvzZKelrRZ0suS/ioXtzfJLa8tIuJVYEN6rBTTCbyYO3dxJsaqJD9LfyCz9ivhmflmtafaM/nfkX79AbAA+BVwAnCNpNci4vb0eDOwusz5remxUkxbRHSWiZkoaWxEbMoekHQacBrA9OnTaWlpGVRl2traBl1GPRhoPde0d7C8tR0BATRPbKJ1Q8eW7d2aJzB1QtkuuqoaDZ/naKgjjI561mIdq51glH69JiIWpN/fJ2kf4GvA7eVPGxoRcTVwNcDs2bNj7ty5gyqvpaWFwZZRDwZSz1VtG5lz8b283ZEd8NcFbN0e37SZh879aM21PkbD5zka6gijo561WMdq3yJrTb/el9t/L7BvLm5qmfObM2W0ApPyQ5fTmA351osNj3LDiPM8rNhsZKp2glmcflVuv4DsAIEl5PpRJM0AJrK1b2YJyZ/F+fVCevTf2PApN4w4z8OKzUamaieYh0laHofm9h8GPJ3ZvgM4XNLkzL6TgHbg/kxZa4ETSwGSJgJHp+dbFZSbpf/5g9+5zbP2zax+FNoHk/6CPzLd3BWYIumEdPv2iNgg6QJggaTVJJ38xwMfJRl+XHIl8CXgZkkXA3sC84FLS0OXI+JtSRcB50tqZetEywbgOwVW0/pRbhjxlw/by8OKzUa4ojv5dwZuyO0rbe8BLI2IyyU1AP+DJGk8D5wQEb8snRARrZIOA64gmfOyGrgsjc+6iCShfA2YBjwBfCIi3hjCOtk2yM/S35ZZ+2ZWX4qeaLmUnv0r5eIuBS7tJ+Y5et5Ky8cEcGH6MjOzKqp2H4yZmY1QTjBmZlYIJxgzMyuEE4yZmRXCCcbMzArhBGNmZoVwgjEzs0I4wZiZWSGcYMzMrBBOMGZmVggnGDMzK4QTjJmZFcIJxszMCuEEY2ZmhXCCMTOzQjjBmJlZIZxgzMysEE4wZmZWCCcYMzMrhBOMmZkVwgnGzMwK4QRjZmaFcIIxM7NCFJpgJM2UdJWkRZI6JbWUiVkqKXKv18vE7SvpHkkbJK2UdIGkxlyMJJ0naZmkdkkPSJpVYBXNzKwXYwoufz/gSOBRoKmPuB8D38lsb8oelNQM3A08BxwLvBu4hCRBfj0T+lXgfOAcYAlwNnC3pP0jokfSMjOz4hSdYG6NiFsAJN0I7NhL3GsR8Wgf5ZwOTACOi4i1wF2SpgDzJS2IiLWSxpMkmG9HxBXpez4CLAXOpHsiMjOzghV6iywiuoaoqCOAO9PkUnIdSdL5WLr9YWAKcH3m/dcDt6bnm5nZMKqVTv6/krRJ0hpJN0p6V+743iS3vLaIiFeBDemxUkwn8GLu3MWZGDMzGyZF3yKrxC0kfTTLgX2A/wn8UtL7ImJNGtMMrC5zbmt6rBTTFhGdZWImShobEfm+ndOA0wCmT59OS0vLoCrS1tY26DLqges5coyGOsLoqGct1rHqCSYivpzZ/KWkh4GFwKnA5QW/99XA1QCzZ8+OuXPnDqq8lpYWBltGPXA9R47RUEcYHfWsxTrWyi2yLSLiGeB54AOZ3a3A1DLhzemxUsyk/NDlNGZDvvViZmbFqrkEk4r0VbKEXD+KpBnARLb2zSwBGoGZubJ69N+YmVnxai7BSNqfJCk8mdl9B3C4pMmZfScB7cD96fbDwFrgxExZE4Gj0/PNzGwYFdoHk/6CPzLd3BWYIumEdPt24BDgs8BtwEqSxPJ14FXg+5mirgS+BNws6WJgT2A+cGlp6HJEvC3pIuB8Sa1snWjZQPdJnGZmNgyK7uTfGbght6+0vQewLI25HNgeWAX8AjgvO+clIlolHQZcQTKvZTVwGUmSybqIJKF8DZgGPAF8IiLeGLoqmZlZJQpNMBGxFFA/YYdVWNZzwKH9xARwYfoaNqvaNrK8tZ3Orug/2MxslKj6MOV6d8vCFZx70yKaGho4/b0bWb9wBcfM2rXal2VmVnU118lfT1a1beTcmxbxdkcX6zZupiuCeTctYlXbxmpfmplZ1TnBDMLy1naaGrr/EzY1NLC8tb1KV2RmVjucYAZht+YJdHR1X8+zo6uL3ZonVOmKzMxqhxPMIEybNI4Fxx/A+KYGJo8bQ4PEguMPYNqkcdW+NDOzqnMn/yAdM2tX5szckeWt7bz14lMc5g5+MzPALZghMW3SOA6csT2NDf2NyDYzGz2cYMzMrBBOMGZmVggnGDMzK4QTjJmZFcIJxszMCqFkfUiT9CbwyiCL2RF4awgup9a5niPHaKgjjI56VquO74qIncodcIIZQpKeiIjZ1b6OormeI8doqCOMjnrWYh19i8zMzArhBGNmZoVwghlaV1f7AoaJ6zlyjIY6wuioZ83V0X0wZmZWCLdgzMysEE4wZmZWCCeYlKSZkq6StEhSp6SWfuIvkxSS/qXMsX0l3SNpg6SVki6Q1JiLkaTzJC2T1C7pAUmzhrha5a6733pKWprWLft6vUxcTdaz0s9S0vsk3SZpjaR1kh6X9MFcTE3WMX3fPuspaW6Zz7H0unOk1DON+SNJ/ylphaQ2SU9JOqVMXE3Ws8I6bi/p/0j6Q1rHOyTNLBNXO3WMCL+SfqhjgWXADcBioKWP2H2BtcAa4F9yx5qBlcDdwCeA04H1wLdycV8D2oEzgY8Dt5NMknpHtesJLAV+BHwo8/pAvdSzwjrOAtYBPwQ+mb7OBz5SD3WspJ7AlNxn+CHgz4AAzhpB9WwAfgX8FjgFOAz4j7Sex9VDPSv8mb0TWAF8DjgGeJxkcviUWq1jYT8U9fYCGjLf31juA84cvwf4Jskv4nyC+RrQmvvQ5wEbSvuA8STJ6Z8yMdsBb+Z/EKpRz3L1KhNTs/WssI6PAj+u1zoO9Gc2E3cO0AnsMlLqCexNkkyOzu3/NfCTeqhnBXU8OK3jYZl909Nr/0qt1tG3yFIR0VVJnKQTSH6gL+ol5AjgzohYm9l3HTAB+Fi6/WGSvy6vz7z/euDW9PzCVFrPCtRsPfuro6R9gYOA7/RTVM3WMX2fbfks/xy4PyJWZvbVez2b0q9rcvtXA9mnANZsPSuo4yygA2jJnPMG8DTwqUxcTdXRCWYAJE0ALgG+mn4g5ewNLMnuiIhXSf6C2DsT0wm8mDt3cSam2v5K0qa0f+JGSe/KHa/neh6Ufm2W9LSkzZJelvRXubh6rmMPkvYC3g9cmztU7/V8BngMuEDSeyRNkfQFYA5wZSaunus5HuiMiM7c/k3APpntmqqjE8zAfA14jeS+fW+aSf5yymtNj5Vi2sr8sLQCEyWNHeyFDtItwN+R3Ms+h6R5/ktJUzMx9VzPd6Rff0DS1/QJ4BfANZKOzMTVcx3LOZnkr+Cbcvvrup6R3OM5guT32QskLZmrgb+MiHszofVcz5eA8ZLeV9qR/sG7P7BDJq6m6jhmqAoa6STtAXwFOCT9gR6xIuLLmc1fSnoYWAicClxenasaUqXbJtdExIL0+/sk7UPyR8Tt1bmswp0M/HdE/KHaFzKUJDWQ/LEwDTgJ+D1wJPA9Sasi4hfVvL4hcifwO+AqSaeSDDK6CJgKbK7mhfXFLZjKXQTcATyfDhfcnuTfb1y6Xfql1Uryoec1p8dKMZPyQwfTmA0RsWnoL3/bRcQzwPPABzK767mepeu7L7f/XpIRgtm4eq1jN5IOJLmVkr89BvVfz6PS16cj4vqIaImIecB/AQsycXVbz/R9Tybp2F9CMlJsT5LEmp1CUFN1dIKp3HuB40g+nNJrBskwv1Zg1zRuCbn7mJJmABPZem90CdAI5Mew97h/WkMifZXUcz0Xp1+V2y8g29laz3XMO5lkWOotZY7Vez33JvnFmO9TeAp4d2a7rusZEY+TXNfewMyI+BNgZ5IRkSU1VUcnmMp9ETgk93qDZCTGISRD/CBp5RwuaXLm3JNI/nPfn24/TNLEPbEUIGkicHR6fk2RtD/JD9+Tmd31XM+HSf4oODS3/zCSUTkl9VzHvJOBWyOircyxeq/nKyR9B+/N7f8gyZD7knqvJ5F4PiJelvQekjks38uE1FYdixrXXW8vkgx/Qvp6BHg2sz2xl3OWUn6i5WvAXSQf/mlAG+UnOm0AziD5xfZzkolO06tZT5Ihj9eSTFg7BPhbksldv6XnhK6arGclnyVwFskInPNIOvmvJGm9/Ek91HEgP7MkEyyD5BZSuXLqup7AZJIks5hkGPbHgcvSOv9dPdSzwp/Z80mSwiHAl0j+qP1+LX+Whf1Q1NsL2J2tt4Hyr917OWcpZSYkktzHv5fkr4bXSCZlNuZiBPwjsDyN+yXw/mrXEziAZCLpmyQjjl4Hvk9mYl6t17PSzxI4m6TjdBPwGzKzvmu9jgOs5+UkI4vG9VFWXdeT5HbPDSR9E20kLdG/IV0xvtbrWWEdLyf5Y28jyaiyc4ExtfxZerl+MzMrhPtgzMysEE4wZmZWCCcYMzMrhBOMmZkVwgnGzMwK4QRjZmaFcIIxqxJJu0t6psz+a9Jn1pQ756x01nVpu9zMfLOa4ARjVmMi4osR8Vx+f7o44Vkks77Nap4TjFl1jZH0I0mL0we7TZTUImk2JC0USZdIeppk5vUuJI8W2LIStKQL0wenPSppepXqYdaDE4xZdb0X+PeI2IdkAcK/yx3fDngsIg6MiAtIlkI5JCIOyRx/NCIOBB4A/nqYrtusX04wZtW1LCIeSr//IfCR3PFOej6BMmsTcFv6/ZMka1qZ1QQnGLPqyi8GmN9+O3o+2jarI7YuKNiJn1JrNcQJxqy63inp4PT7zwAP9hO/jmR5erOa5wRjVl3PA2dIWkzyLI//6Cf+auAX2U5+s1rl5frNzKwQbsGYmVkhnGDMzKwQTjBmZlYIJxgzMyuEE4yZmRXCCcbMzArhBGNmZoX4/07MjGUh/BwjAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "composers_df.plot(x = 'birth', y = 'death', kind = 'scatter',\n",
    "                  title = 'Composer birth and death', grid = True, fontsize = 15)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some additional plotting options are available in the plot() module. For example histograms:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T15:05:37.947613Z",
     "start_time": "2020-06-24T15:05:37.641676Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAD4CAYAAADrRI2NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAXCUlEQVR4nO3dfZRV9X3v8fdXHqRcND6A1ogUzIpWkgbFMVIFI2obavAhFaOu0KUhdq7aeI0PTTXepax740quscXWmzRFY9XEaHyqCanplaQQdYkPgBAfEI1KzChRgvdGoyKi3/vH2eA4mYEzMOfsObPfr7Vmsc9v73N+39+cmQ/7/PaevSMzkSRVx3ZlFyBJai6DX5IqxuCXpIox+CWpYgx+SaqYwWUXUI+RI0fm2LFjyy5DklrKkiVLfpOZo7q2t0Twjx07lsWLF5ddhiS1lIj4ZXftTvVIUsUY/JJUMQa/JFVMS8zxd+ftt9+mo6ODdevWlV1KKYYNG8bo0aMZMmRI2aVIajEtG/wdHR3ssMMOjB07logou5ymykzWrl1LR0cH48aNK7scSS2mZad61q1bx6677lq50AeICHbdddfKftqRtG0aFvwRcW1EvBwRj3VpPzsinoyIxyPi8m3sY9uKbGFVHrukbdPIPf7rgGmdGyJiKnAcMCEzPwJc0cD+JUndaNgcf2beExFjuzSfCXwtM98qtnm5r/qbM/+pvnopAM79s322uM2qVauYPn06jz32vg81nH766Zx33nmMHz/+955z5ZVX0t7ezvDhwwEYMWIEv/vd7/qmaEmqQ7MP7u4DTImIy4B1wAWZ+XB3G0ZEO9AOMGbMmOZV2AeuueaabtvfeecdrrzySmbOnLkp+KVW09c7Wb1Rzw6ZtqzZB3cHA7sAk4C/BW6JHiarM3NuZrZlZtuoUb93qYl+Y8OGDXz2s59lv/32Y8aMGbzxxhscfvjhmy4xMWLECM4//3wmTJjAZZddxosvvsjUqVOZOnXqpte4+OKLmTBhApMmTeKll14qayiSKqLZwd8B3JE1DwHvAiObXEOfWrlyJWeddRYrVqxgxx135Jvf/Ob71r/++uscfPDBLF++nEsuuYQPfvCDLFiwgAULFmxaP2nSJJYvX85hhx3G1VdfXcYwJFVIs4P/TmAqQETsAwwFftPkGvrUXnvtxaGHHgrAzJkzue+++963ftCgQZxwwgk9Pn/o0KFMnz4dgAMPPJBVq1Y1rFZJggbO8UfETcDhwMiI6AAuBa4Fri1O8VwPnJotfrf3rjNVXR8PGzaMQYMG9fj8IUOGbHrOoEGD2LBhQ98XKUmdNGyPPzNPycw9MnNIZo7OzG9n5vrMnJmZH83MiZn5n43qv1mef/55Fi1aBMD3vvc9Jk+evNntd9hhB1577bVmlCZJ3WrZSzZ0VdbR/n333ZdvfOMbzJo1i/Hjx3PmmWcyb968Hrdvb29n2rRpm+b6JanZohVmWtra2rLrjVhWrFjBfvvtV1JF/YPfA5XB0zlbR0Qsycy2ru0te60eSdLWMfglqWIMfkmqGINfkirG4Jekihkwp3NKqoAFXy2n36kXldNvgwyc4O/rH4hevtGzZ89mxIgRXHDBBb163sKFCxk6dCiHHHIIAKeddhrTp09nxowZvXodSaqXUz0lW7hwIffff3/ZZUiqEIN/G1x22WXss88+TJ48mZUrVwLwzDPPMG3aNA488ECmTJnCk08+CcC8efM4+OCDOeCAAzjqqKN46aWXWLVqFd/61reYM2cO+++/P/feey8A99xzD4cccgh77703t912W2njkzQwGfxbacmSJdx8880sW7aMu+66i4cfrt1Ppr29nauuuoolS5ZwxRVXcNZZZwEwefJkHnjgAR555BFOPvlkLr/8csaOHcsZZ5zBueeey7Jly5gyZQoAq1ev5r777uNHP/oRF154YWljlDQwDZw5/ia79957+fSnP73pTlrHHnss69at4/777+fEE0/ctN1bb70FQEdHByeddBKrV69m/fr1jBs3rsfXPv7449luu+0YP368N2aR1OcM/j707rvvstNOO7Fs2bLfW3f22Wdz3nnnceyxx7Jw4UJmz57d4+tsv/32m5Zb4VpKklqLUz1b6bDDDuPOO+/kzTff5LXXXmPevHkMHz6ccePGceuttwK10F6+fDkAv/3tb9lzzz0BuP766ze9jpdpltRsA2ePv8nn2U6cOJGTTjqJCRMmsNtuu3HQQQcBcOONN3LmmWfyla98hbfffpuTTz6ZCRMmMHv2bE488UR23nlnjjjiCJ577jkAjjnmGGbMmMEPfvADrrrqqqaOQVI1NeyyzBFxLTAdeDkzP9pl3fnAFcCozNzirRe9LHP3/B6oDKVelnnw7eV03KJ/wFXGZZmvA6Z1U8hewJ8Dzzewb0lSDxp568V7gFe6WTUH+BLgUUtJKkFTD+5GxHHAC5m5vC9er8pnvFR57JK2TdOCPyKGA18GLqlz+/aIWBwRi9esWfN764cNG8batWsrGYCZydq1axk2bFjZpUhqQc08q+dDwDhgeUQAjAaWRsTHM/PXXTfOzLnAXKgd3O26fvTo0XR0dNDdfwpVMGzYMEaPHl12GZJaUNOCPzMfBXbb+DgiVgFt9ZzV050hQ4Zs9q9fJUnda9hUT0TcBCwC9o2Ijoj4fKP6kiTVr2F7/Jl5yhbWj21U35KknnnJBkmqGINfkirG4JekijH4JaliDH5JqhiDX5IqxuCXpIox+CWpYgx+SaoYg1+SKsbgl6SKMfglqWIMfkmqGINfkirG4JekijH4JaliDH5JqphG3nrx2oh4OSIe69T29Yh4MiJ+HhH/FhE7Nap/SVL3GrnHfx0wrUvbfOCjmfkx4Cngogb2L0nqRsOCPzPvAV7p0nZ3Zm4oHj4AjG5U/5Kk7pU5xz8L+HFPKyOiPSIWR8TiNWvWNLEsSRrYSgn+iLgY2ADc2NM2mTk3M9sys23UqFHNK06SBrjBze4wIk4DpgNHZmY2u39JqrqmBn9ETAO+BHwiM99oZt+SpJpGns55E7AI2DciOiLi88D/BnYA5kfEsoj4VqP6lyR1r2F7/Jl5SjfN325Uf5Kk+viXu5JUMQa/JFWMwS9JFWPwS1LFGPySVDEGvyRVjMEvSRVj8EtSxRj8klQxBr8kVYzBL0kVY/BLUsUY/JJUMQa/JFWMwS9JFWPwS1LFGPySVDGNvPXitRHxckQ81qltl4iYHxFPF//u3Kj+JUndqyv4I+JPtuK1rwOmdWm7EPhpZn4Y+GnxWJLURPXu8X8zIh6KiLMi4gP1PCEz7wFe6dJ8HHB9sXw9cHyd/UuS+khdN1vPzCkR8WFgFrAkIh4C/jUz5/eyv90zc3Wx/Gtg9542jIh2oB1gzJgxvexGGvjmzH+q7BLUouqe48/Mp4H/Dvwd8AngnyLiyYj4y63pODMTyM2sn5uZbZnZNmrUqK3pQpLUjXrn+D8WEXOAFcARwDGZuV+xPKcX/b0UEXsUr7kH8HIv65UkbaN69/ivApYCEzLzbzJzKUBmvkjtU0C9fgicWiyfCvygF8+VJPWBuub4gU8Bb2bmOwARsR0wLDPfyMzvdPeEiLgJOBwYGREdwKXA14BbIuLzwC+Bz2xj/ZKkXqo3+H8CHAX8rng8HLgbOKSnJ2TmKT2sOrLu6iRJfa7eqZ5hmbkx9CmWhzemJElSI9Ub/K9HxMSNDyLiQODNxpQkSWqkeqd6vgjcGhEvAgH8IXBSw6qSJDVMvX/A9XBE/DGwb9G0MjPfblxZkqRGqXePH+AgYGzxnIkRQWbe0JCqJEkNU1fwR8R3gA8By4B3iuYEDH5JajH17vG3AeOLyyxIklpYvcH/GLUDuqu3tKEkDTgLvlpe31Mv6vOXrDf4RwJPFFflfGtjY2Ye2+cVSZIaqt7gn93IIiRJzVPv6Zw/i4g/Aj6cmT+JiOHAoMaWJklqhHovy/zXwG3AvxRNewJ3NqooSVLj1HvJhr8BDgVehU03ZdmtUUVJkhqn3uB/KzPXb3wQEYPZzN2zJEn9V73B/7OI+DLwBxHxZ8CtwLzGlSVJapR6g/9CYA3wKPBfgbvo3Z23JEn9RL1n9bwLXF18SZJaWL3X6nmObub0M3Pvrek0Is4FTi9e81Hgc5m5bmteS5LUO725Vs9Gw4ATgV22psOI2BP4b9Su/fNmRNwCnAxctzWvJ0nqnbrm+DNzbaevFzLzSmo3YN9ag6kdKB5M7RaOL27Da0mSeqHeqZ6JnR5uR+0TQG+u5b9JZr4QEVcAz1O7fePdmXl3N322A+0AY8aM2ZquJDXApOfnltb3opL6/dO9dy2p58aoN7z/vtPyBmAV8Jmt6TAidgaOA8YB/4/aLR1nZuZ3O2+XmXOBuQBtbW3+zYAk9ZF6z+qZ2od9HgU8l5lrACLiDuAQ4LubfZYkqU/UO9Vz3ubWZ+Y/9KLP54FJxYXe3gSOBBb34vmSpG3Qm7N6DgJ+WDw+BngIeLq3HWbmgxFxG7CU2rTRIxRTOpKkxqs3+EcDEzPzNYCImA38e2bO3JpOM/NS4NKtea4kadvUe8mG3YH1nR6vL9okSS2m3j3+G4CHIuLfisfHA9c3piRJUiPVe1bPZRHxY2BK0fS5zHykcWVJkhql3qkeqP2F7auZ+Y9AR0SMa1BNkqQGqvfWi5cCfwdcVDQNwfPuJakl1bvH/2ngWOB1gMx8EdihUUVJkhqn3uBfn5lJcWnmiPgvjStJktRI9Z7Vc0tE/AuwU0T8NTALb8qirhZ8tby+p1605W0kAXUEf0QE8H3gj4FXgX2BSzJzfoNrkyQ1wBaDPzMzIu7KzD8BDHtJanH1zvEvjYiDGlqJJKkp6p3jPxiYGRGrqJ3ZE9Q+DHysUYVJkhpjs8EfEWMy83ngk02qR5LUYFva47+T2lU5fxkRt2fmCc0oSpLUOFua449Oy3s3shBJUnNsKfizh2VJUova0lTPhIh4ldqe/x8Uy/Dewd0dG1qdJKnPbTb4M3NQIzqNiJ2Aa4CPUvskMSszFzWiL0nS+9V7Omdf+0fgPzJzRkQMpXbJZ0lSEzQ9+CPiA8BhwGkAmbme99/WUZLUQGXs8Y8D1gD/GhETgCXAOZn5eueNIqIdaAcYM2ZM04tsZXPmP1VKv+eW9fmR8sY86fm5pfQLMKm0ntXqenMHrr4yGJgI/HNmHkDtL4Ev7LpRZs7NzLbMbBs1alSza5SkAauM4O8AOjLzweLxbdT+I5AkNUHTgz8zfw38KiL2LZqOBJ5odh2SVFVlzcqeDdxYnNHzLPC5kuqQpMopJfgzcxnQVkbfklR1ZczxS5JKZPBLUsUY/JJUMQa/JFWMwS9JFWPwS1LFGPySVDEGvyRVjMEvSRVj8EtSxRj8klQxBr8kVYzBL0kVY/BLUsUY/JJUMQa/JFWMwS9JFVNa8EfEoIh4JCJ+VFYNklRFZe7xnwOsKLF/SaqkUoI/IkYDnwKuKaN/Saqysvb4rwS+BLzb0wYR0R4RiyNi8Zo1a5pXmSQNcE0P/oiYDrycmUs2t11mzs3MtsxsGzVqVJOqk6SBr4w9/kOBYyNiFXAzcEREfLeEOiSpkpoe/Jl5UWaOzsyxwMnAf2bmzGbXIUlV5Xn8klQxg8vsPDMXAgvLrEGSqsY9fkmqGINfkirG4JekijH4JaliDH5JqhiDX5IqxuCXpIop9Tx+DSyLnl1bWt+TmFta31KrcY9fkirG4JekijH4JaliDH5JqhiDX5IqxuCXpIox+CWpYgx+SaoYg1+SKqbpwR8Re0XEgoh4IiIej4hzml2DJFVZGZds2ACcn5lLI2IHYElEzM/MJ0qoRZIqp+l7/Jm5OjOXFsuvASuAPZtdhyRVValz/BExFjgAeLCbde0RsTgiFq9Zs6bZpUnSgFVa8EfECOB24IuZ+WrX9Zk5NzPbMrNt1KhRzS9QkgaoUoI/IoZQC/0bM/OOMmqQpKoq46yeAL4NrMjMf2h2/5JUdWXs8R8K/BVwREQsK76OLqEOSaqkpp/OmZn3AdHsfiVJNf7lriRVjMEvSRVj8EtSxRj8klQxBr8kVYzBL0kVY/BLUsWUcVnmppoz/6nS+j538O0l9XxCSf1KA9OiZ9eW1vefTu3713SPX5IqxuCXpIox+CWpYgx+SaoYg1+SKsbgl6SKMfglqWIMfkmqGINfkiqmrJutT4uIlRHxi4i4sIwaJKmqyrjZ+iDgG8BfAOOBUyJifLPrkKSqKmOP/+PALzLz2cxcD9wMHFdCHZJUSZGZze0wYgYwLTNPLx7/FXBwZn6hy3btQHvxcF9g5TZ2PRL4zTa+RtkcQ//gGPqHgTAGaOw4/igzR3Vt7LdX58zMucDcvnq9iFicmW199XplcAz9g2PoHwbCGKCccZQx1fMCsFenx6OLNklSE5QR/A8DH46IcRExFDgZ+GEJdUhSJTV9qiczN0TEF4D/AwwCrs3Mx5vQdZ9NG5XIMfQPjqF/GAhjgBLG0fSDu5KkcvmXu5JUMQa/JFVMywZ/RFwbES9HxGPdrDs/IjIiRhaPIyL+qbhExM8jYmKnbU+NiKeLr1PLHkNEzI6IFyJiWfF1dKd1FxVjWBkRn+zUXtolMHp6HyLi7Ih4MiIej4jL+/MYiv67ey++3+l9WBURy/rzOHoYw/4R8UAxhsUR8fGivZV+JyZExKKIeDQi5kXEjp3W9cf3Ya+IWBARTxQ//+cU7btExPzi+zo/InYu2pv/XmRmS34BhwETgce6tO9F7cDxL4GRRdvRwI+BACYBDxbtuwDPFv/uXCzvXOYYgNnABd1sOx5YDmwPjAOeoXZwfFCxvDcwtNhmfMljmAr8BNi+eLxbfx7D5n6eOq3/e+CS/jyOHt6Lu4G/6PR7sLAFfyceBj5RLM8C/mc/fx/2ACYWyzsATxW1Xg5cWLRfCPyvst6Llt3jz8x7gFe6WTUH+BLQ+aj1ccANWfMAsFNE7AF8Epifma9k5v8F5gPTGlz6JpsZQ3eOA27OzLcy8zngF9Quf1HqJTB6GMOZwNcy861im5eL9n45hqLGHt+LiAjgM8BNRVO/HEcPY0hg4x7yB4AXi+VW+p3YB7inWJ4PnFAs99f3YXVmLi2WXwNWAHsWNVxfbHY9cHyncTT1vWjZ4O9ORBwHvJCZy7us2hP4VafHHUVbT+1l+0Lxke/ajR8Haa0x7ANMiYgHI+JnEXFQ0d5KY+hsCvBSZj5dPG6lcXwR+HpE/Aq4ArioaG+lMTzOe8F9Iu/9AWi/H0NEjAUOAB4Eds/M1cWqXwO7F8tNH8eACf6IGA58Gbik7Fq20T8DHwL2B1ZTm2JoNYOpfTydBPwtcEux19yqTuG9vf1WcyZwbmbuBZwLfLvkerbGLOCsiFhCbepkfcn11CUiRgC3A1/MzFc7r8vaXE5p59IPmOCnFpbjgOURsYrapSCWRsQf0vNlIvrd5SMy86XMfCcz3wWupvaxFVpoDNT2TO4oPro+BLxL7UJUrTQGACJiMPCXwPc7NbfSOE4F7iiWb6UFf54y88nM/PPMPJDaf8DPFKv67RgiYgi10L8xMzd+/18qpnAo/t04Bdr8cTTrgEcjvoCx9HwwbhXvHdz9FO8/ePJQvnfw5DlqB052LpZ3KXMMwB6dls+lNocJ8BHefyDrWWoHsQYXy+N470DWR0oewxnA/yiW96H2cTX68xh6+nmiNqf6sy5t/XYc3bwXK4DDi+UjgSUt+Dux8eSA7YAbgFn9+X0ovqc3AFd2af867z+4e3lZ70XT3swGfHNvojYV8ja1PczPd1m/iveCP6jd/OUZ4FGgrdN2s6gdFPoF8LmyxwB8p6jx59SuYdT5P4KLizGspDhTo2g/mtqZA88AF/eDMQwFvgs8BiwFjujPY9jczxNwHXBGN9v3u3H08F5MBpYU4fcgcGAL/k6cU3xPnwK+RnHFgX78PkymNo3zc2BZ8XU0sCvwU+Bpame97VLWe+ElGySpYgbSHL8kqQ4GvyRVjMEvSRVj8EtSxRj8klQxBr8kVYzBL0kV8/8B7DtnixgTj6oAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "composers_df.plot.hist(alpha = 0.5)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here you see again the gain from using Pandas: without specifying anything, Pandas made a histogram of the two columns containing numbers, labelled the axis and even added a legend to the plot."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# DataFrame Operations\n",
    "\n",
    "One of the great advantages of using Pandas to handle tabular data is how simple it is to extract valuable information from them. Here we are going to see various types of operations that are available for this.\n",
    "\n",
    "## Matrix operations\n",
    "\n",
    "The strength of Numpy is its natural way of handling matrix operations, and Pandas reuses a lot of these features. For example one can use simple mathematical operations to opereate at the cell level:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T15:09:22.426144Z",
     "start_time": "2020-06-24T15:09:22.385407Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>composer</th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Mahler</td>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Beethoven</td>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Puccini</td>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Shostakovich</td>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       composer  birth  death              city\n",
       "0        Mahler   1860   1911           Kaliste\n",
       "1     Beethoven   1770   1827              Bonn\n",
       "2       Puccini   1858   1924           Lucques\n",
       "3  Shostakovich   1906   1975  Saint-Petersburg"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_excel('data/composers.xlsx')\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T15:09:39.792603Z",
     "start_time": "2020-06-24T15:09:39.775993Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    3720\n",
       "1    3540\n",
       "2    3716\n",
       "3    3812\n",
       "Name: birth, dtype: int64"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['birth'] * 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T15:10:00.912276Z",
     "start_time": "2020-06-24T15:10:00.899426Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    7.528332\n",
       "1    7.478735\n",
       "2    7.527256\n",
       "3    7.552762\n",
       "Name: birth, dtype: float64"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.log(df['birth'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can directly use an operation's output to create a new column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T15:19:14.832808Z",
     "start_time": "2020-06-24T15:19:14.807838Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>composer</th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "      <th>age</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Mahler</td>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "      <td>51</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Beethoven</td>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "      <td>57</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Puccini</td>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "      <td>66</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Shostakovich</td>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "      <td>69</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       composer  birth  death              city  age\n",
       "0        Mahler   1860   1911           Kaliste   51\n",
       "1     Beethoven   1770   1827              Bonn   57\n",
       "2       Puccini   1858   1924           Lucques   66\n",
       "3  Shostakovich   1906   1975  Saint-Petersburg   69"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['age'] = df['death'] - df['birth']\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we applied functions only to series. Indeed, since our Dataframe contains e.g. strings, no operation can be done on it. If however we have a homogenous Dataframe, this is possible:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T15:10:52.552872Z",
     "start_time": "2020-06-24T15:10:52.534293Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3720</td>\n",
       "      <td>3822</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3540</td>\n",
       "      <td>3654</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3716</td>\n",
       "      <td>3848</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3812</td>\n",
       "      <td>3950</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   birth  death\n",
       "0   3720   3822\n",
       "1   3540   3654\n",
       "2   3716   3848\n",
       "3   3812   3950"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[['birth', 'death']] * 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Column operations\n",
    "\n",
    "There are other types of functions whose purpose is to summarize the data. For example the mean or standard deviation. Pandas by default applies such functions column-wise and returns a series containing e.g. the mean of each column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T15:11:31.318490Z",
     "start_time": "2020-06-24T15:11:31.299730Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "birth    1848.50\n",
       "death    1909.25\n",
       "dtype: float64"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that columns for which a mean does not make sense, like the city are discarded. \n",
    "\n",
    "Sometimes one needs to apply to a column a very specific function that is not provided by default. In that case we can use one of the different `apply` methods of Pandas.\n",
    "\n",
    "The simplest case is to apply a function to a column, or Series of a DataFrame. Let's say for example that we want to define the the age >60 as 'old' and <60 as 'young'. We can define the following general function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T15:16:11.585908Z",
     "start_time": "2020-06-24T15:16:11.579767Z"
    }
   },
   "outputs": [],
   "source": [
    "define_age = lambda x: 'old' if x > 60 else 'young'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now apply this function on an entire Series:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T15:19:19.576840Z",
     "start_time": "2020-06-24T15:19:19.558825Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    young\n",
       "1    young\n",
       "2      old\n",
       "3      old\n",
       "Name: age, dtype: object"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.age.apply(define_age)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also apply a function to an entire DataFrame. For example we can ask how many composers have birth and death dates within the XIXth century:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T15:21:10.779069Z",
     "start_time": "2020-06-24T15:21:10.759181Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "birth    2\n",
       "death    1\n",
       "dtype: int64"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nineteen_century_count = lambda x: np.sum( (x >= 1800) & (x < 1900) )\n",
    "df[['birth','death']].apply(nineteen_century_count)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Boolean Indexing\n",
    "\n",
    "Just like with Numpy, it is possible to subselect parts of a Dataframe using boolean indexing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T15:22:52.386996Z",
     "start_time": "2020-06-24T15:22:52.375984Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0     True\n",
      "1    False\n",
      "2    False\n",
      "3     True\n",
      "Name: birth, dtype: bool\n"
     ]
    }
   ],
   "source": [
    "mask = df['birth'] > 1859\n",
    "print(mask)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just like in Numpy we can use this logical Series as an index to select elements in the Dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-24T15:23:15.607297Z",
     "start_time": "2020-06-24T15:23:15.591321Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>composer</th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "      <th>age</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Mahler</td>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "      <td>51</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Shostakovich</td>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "      <td>69</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       composer  birth  death              city  age\n",
       "0        Mahler   1860   1911           Kaliste   51\n",
       "3  Shostakovich   1906   1975  Saint-Petersburg   69"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[mask]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Merging\n",
    "\n",
    "Often information is comming from different sources and it is necessary to combine it into one object. We are going to see the different ways in which information contained within separate Dataframes can be combined in a meaningful way.\n",
    "\n",
    "## Concatenation\n",
    "\n",
    "The simplest way we can combine two Dataframes is simply to \"paste\" them together:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T09:10:31.191085Z",
     "start_time": "2020-06-25T09:10:31.070211Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              birth  death              city\n",
       "composer                                    \n",
       "Mahler         1860   1911           Kaliste\n",
       "Beethoven      1770   1827              Bonn\n",
       "Puccini        1858   1924           Lucques\n",
       "Shostakovich   1906   1975  Saint-Petersburg"
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers1 = pd.read_excel('data/composers.xlsx', index_col='composer',sheet_name='Sheet1')\n",
    "composers1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T09:10:50.431994Z",
     "start_time": "2020-06-25T09:10:50.378518Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Verdi</th>\n",
       "      <td>1813</td>\n",
       "      <td>1901</td>\n",
       "      <td>Roncole</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Dvorak</th>\n",
       "      <td>1841</td>\n",
       "      <td>1904</td>\n",
       "      <td>Nelahozeves</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Schumann</th>\n",
       "      <td>1810</td>\n",
       "      <td>1856</td>\n",
       "      <td>Zwickau</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Stravinsky</th>\n",
       "      <td>1882</td>\n",
       "      <td>1971</td>\n",
       "      <td>Oranienbaum</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            birth  death         city\n",
       "composer                             \n",
       "Verdi        1813   1901      Roncole\n",
       "Dvorak       1841   1904  Nelahozeves\n",
       "Schumann     1810   1856      Zwickau\n",
       "Stravinsky   1882   1971  Oranienbaum\n",
       "Mahler       1860   1911      Kaliste"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers2 = pd.read_excel('data/composers.xlsx', index_col='composer',sheet_name='Sheet3')\n",
    "composers2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To be concatenated, Dataframes need to be provided as a list to the `pd.concat` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T09:11:09.559547Z",
     "start_time": "2020-06-25T09:11:09.536473Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Verdi</th>\n",
       "      <td>1813</td>\n",
       "      <td>1901</td>\n",
       "      <td>Roncole</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Dvorak</th>\n",
       "      <td>1841</td>\n",
       "      <td>1904</td>\n",
       "      <td>Nelahozeves</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Schumann</th>\n",
       "      <td>1810</td>\n",
       "      <td>1856</td>\n",
       "      <td>Zwickau</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Stravinsky</th>\n",
       "      <td>1882</td>\n",
       "      <td>1971</td>\n",
       "      <td>Oranienbaum</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              birth  death              city\n",
       "composer                                    \n",
       "Mahler         1860   1911           Kaliste\n",
       "Beethoven      1770   1827              Bonn\n",
       "Puccini        1858   1924           Lucques\n",
       "Shostakovich   1906   1975  Saint-Petersburg\n",
       "Verdi          1813   1901           Roncole\n",
       "Dvorak         1841   1904       Nelahozeves\n",
       "Schumann       1810   1856           Zwickau\n",
       "Stravinsky     1882   1971       Oranienbaum\n",
       "Mahler         1860   1911           Kaliste"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_composers = pd.concat([composers1,composers2])\n",
    "all_composers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One potential problem is that two tables contain duplicated information:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T09:12:01.291916Z",
     "start_time": "2020-06-25T09:12:01.254007Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          birth  death     city\n",
       "composer                       \n",
       "Mahler     1860   1911  Kaliste\n",
       "Mahler     1860   1911  Kaliste"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_composers.loc['Mahler']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is very easy to get rid of it using:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T09:12:18.690762Z",
     "start_time": "2020-06-25T09:12:18.656734Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Verdi</th>\n",
       "      <td>1813</td>\n",
       "      <td>1901</td>\n",
       "      <td>Roncole</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Dvorak</th>\n",
       "      <td>1841</td>\n",
       "      <td>1904</td>\n",
       "      <td>Nelahozeves</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Schumann</th>\n",
       "      <td>1810</td>\n",
       "      <td>1856</td>\n",
       "      <td>Zwickau</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Stravinsky</th>\n",
       "      <td>1882</td>\n",
       "      <td>1971</td>\n",
       "      <td>Oranienbaum</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              birth  death              city\n",
       "composer                                    \n",
       "Mahler         1860   1911           Kaliste\n",
       "Beethoven      1770   1827              Bonn\n",
       "Puccini        1858   1924           Lucques\n",
       "Shostakovich   1906   1975  Saint-Petersburg\n",
       "Verdi          1813   1901           Roncole\n",
       "Dvorak         1841   1904       Nelahozeves\n",
       "Schumann       1810   1856           Zwickau\n",
       "Stravinsky     1882   1971       Oranienbaum"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_composers.drop_duplicates()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Joining\n",
    "\n",
    "An other classical case is that of two list with similar index but containing different information:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T09:42:42.326225Z",
     "start_time": "2020-06-25T09:42:42.235023Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              birth  death              city\n",
       "composer                                    \n",
       "Mahler         1860   1911           Kaliste\n",
       "Beethoven      1770   1827              Bonn\n",
       "Puccini        1858   1924           Lucques\n",
       "Shostakovich   1906   1975  Saint-Petersburg"
      ]
     },
     "execution_count": 129,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers1 = pd.read_excel('data/composers.xlsx', index_col='composer',sheet_name='Sheet1')\n",
    "composers1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T09:42:56.133368Z",
     "start_time": "2020-06-25T09:42:56.067953Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>first name</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>Gustav</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>Ludwig van</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>Giacomo</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Brahms</th>\n",
       "      <td>Johannes</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           first name\n",
       "composer             \n",
       "Mahler         Gustav\n",
       "Beethoven  Ludwig van\n",
       "Puccini       Giacomo\n",
       "Brahms       Johannes"
      ]
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers2 = pd.read_excel('data/composers.xlsx', index_col='composer',sheet_name='Sheet4')\n",
    "composers2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we use simple concatenation, this doesn't help us much. We just end up with a large matrix with lots of `NaN`'s:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T09:44:18.040530Z",
     "start_time": "2020-06-25T09:44:17.960321Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "      <th>first name</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860.0</td>\n",
       "      <td>1911.0</td>\n",
       "      <td>Kaliste</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770.0</td>\n",
       "      <td>1827.0</td>\n",
       "      <td>Bonn</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858.0</td>\n",
       "      <td>1924.0</td>\n",
       "      <td>Lucques</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>1906.0</td>\n",
       "      <td>1975.0</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Gustav</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Ludwig van</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Giacomo</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Brahms</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Johannes</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               birth   death              city  first name\n",
       "composer                                                  \n",
       "Mahler        1860.0  1911.0           Kaliste         NaN\n",
       "Beethoven     1770.0  1827.0              Bonn         NaN\n",
       "Puccini       1858.0  1924.0           Lucques         NaN\n",
       "Shostakovich  1906.0  1975.0  Saint-Petersburg         NaN\n",
       "Mahler           NaN     NaN               NaN      Gustav\n",
       "Beethoven        NaN     NaN               NaN  Ludwig van\n",
       "Puccini          NaN     NaN               NaN     Giacomo\n",
       "Brahms           NaN     NaN               NaN    Johannes"
      ]
     },
     "execution_count": 131,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.concat([composers1, composers2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The better way of doing this is to **join** the tables. This is a classical database concept avaialble in Pandas. `join()` operates on two tables: the first one is the \"left\" table which uses `join()` as a method. The other table is the \"right\" one.\n",
    "Let's try the default join settings:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T09:45:06.835088Z",
     "start_time": "2020-06-25T09:45:06.757433Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "      <th>first name</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "      <td>Gustav</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "      <td>Ludwig van</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "      <td>Giacomo</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              birth  death              city  first name\n",
       "composer                                                \n",
       "Mahler         1860   1911           Kaliste      Gustav\n",
       "Beethoven      1770   1827              Bonn  Ludwig van\n",
       "Puccini        1858   1924           Lucques     Giacomo\n",
       "Shostakovich   1906   1975  Saint-Petersburg         NaN"
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers1.join(composers2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that Pandas was smart enough to notice that the two tables had a index name and used it to combine the tables. We also see that one element from the second table (Brahms) is missing. The reason for this is the way indices not present in both tables are handled. There are four ways of doing this with two tables called here the \"left\" and \"right\" table."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Join left\n",
    "\n",
    "The two Dataframes that should be merged have a common index, but not necessarily the same items. For example here Shostakovich is missing in the second table, while Brahms is missing in the first one."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T09:51:15.654892Z",
     "start_time": "2020-06-25T09:51:15.578862Z"
    }
   },
   "source": [
    "When using the \"left\" join, we use the first Dataframe as basis and only use the indices that appear there."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T09:48:10.588614Z",
     "start_time": "2020-06-25T09:48:10.541269Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "      <th>first name</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "      <td>Gustav</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "      <td>Ludwig van</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "      <td>Giacomo</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              birth  death              city  first name\n",
       "composer                                                \n",
       "Mahler         1860   1911           Kaliste      Gustav\n",
       "Beethoven      1770   1827              Bonn  Ludwig van\n",
       "Puccini        1858   1924           Lucques     Giacomo\n",
       "Shostakovich   1906   1975  Saint-Petersburg         NaN"
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers1.join(composers2, how = 'left')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Join right\n",
    "\n",
    "When using the \"right\" join, we use the second Dataframe as basis and only use the indices that appear there."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T09:49:12.876448Z",
     "start_time": "2020-06-25T09:49:12.836603Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "      <th>first name</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860.0</td>\n",
       "      <td>1911.0</td>\n",
       "      <td>Kaliste</td>\n",
       "      <td>Gustav</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770.0</td>\n",
       "      <td>1827.0</td>\n",
       "      <td>Bonn</td>\n",
       "      <td>Ludwig van</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858.0</td>\n",
       "      <td>1924.0</td>\n",
       "      <td>Lucques</td>\n",
       "      <td>Giacomo</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Brahms</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Johannes</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            birth   death     city  first name\n",
       "composer                                      \n",
       "Mahler     1860.0  1911.0  Kaliste      Gustav\n",
       "Beethoven  1770.0  1827.0     Bonn  Ludwig van\n",
       "Puccini    1858.0  1924.0  Lucques     Giacomo\n",
       "Brahms        NaN     NaN      NaN    Johannes"
      ]
     },
     "execution_count": 134,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers1.join(composers2, how = 'right')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Inner join\n",
    "\n",
    "When using the \"inner\" join, we return only the items that appear in both Dataframes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T09:49:25.070369Z",
     "start_time": "2020-06-25T09:49:25.034314Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "      <th>first name</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "      <td>Gustav</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "      <td>Ludwig van</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "      <td>Giacomo</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           birth  death     city  first name\n",
       "composer                                    \n",
       "Mahler      1860   1911  Kaliste      Gustav\n",
       "Beethoven   1770   1827     Bonn  Ludwig van\n",
       "Puccini     1858   1924  Lucques     Giacomo"
      ]
     },
     "execution_count": 135,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers1.join(composers2, how = 'inner')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Outer join\n",
    "\n",
    "When using the \"inner\" join, we return all the items that appaer in both Dataframes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T09:49:35.747756Z",
     "start_time": "2020-06-25T09:49:35.701517Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "      <th>first name</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770.0</td>\n",
       "      <td>1827.0</td>\n",
       "      <td>Bonn</td>\n",
       "      <td>Ludwig van</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Brahms</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Johannes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860.0</td>\n",
       "      <td>1911.0</td>\n",
       "      <td>Kaliste</td>\n",
       "      <td>Gustav</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858.0</td>\n",
       "      <td>1924.0</td>\n",
       "      <td>Lucques</td>\n",
       "      <td>Giacomo</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>1906.0</td>\n",
       "      <td>1975.0</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               birth   death              city  first name\n",
       "composer                                                  \n",
       "Beethoven     1770.0  1827.0              Bonn  Ludwig van\n",
       "Brahms           NaN     NaN               NaN    Johannes\n",
       "Mahler        1860.0  1911.0           Kaliste      Gustav\n",
       "Puccini       1858.0  1924.0           Lucques     Giacomo\n",
       "Shostakovich  1906.0  1975.0  Saint-Petersburg         NaN"
      ]
     },
     "execution_count": 136,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers1.join(composers2, how = 'outer')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Merging\n",
    "\n",
    "Sometimes tables don't have the same indices but similar contents that we want to merge. For example let's imagine whe have the two Dataframes below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T10:02:35.530321Z",
     "start_time": "2020-06-25T10:02:35.378339Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>composer</th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Mahler</td>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Beethoven</td>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Puccini</td>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Shostakovich</td>\n",
       "      <td>1906</td>\n",
       "      <td>1975</td>\n",
       "      <td>Saint-Petersburg</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       composer  birth  death              city\n",
       "0        Mahler   1860   1911           Kaliste\n",
       "1     Beethoven   1770   1827              Bonn\n",
       "2       Puccini   1858   1924           Lucques\n",
       "3  Shostakovich   1906   1975  Saint-Petersburg"
      ]
     },
     "execution_count": 138,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers1 = pd.read_excel('data/composers.xlsx', sheet_name='Sheet1')\n",
    "composers1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T10:02:47.472207Z",
     "start_time": "2020-06-25T10:02:47.394103Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>last name</th>\n",
       "      <th>first name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Puccini</td>\n",
       "      <td>Giacomo</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Beethoven</td>\n",
       "      <td>Ludwig van</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Brahms</td>\n",
       "      <td>Johannes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Mahler</td>\n",
       "      <td>Gustav</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   last name  first name\n",
       "0    Puccini     Giacomo\n",
       "1  Beethoven  Ludwig van\n",
       "2     Brahms    Johannes\n",
       "3     Mahler      Gustav"
      ]
     },
     "execution_count": 139,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers2 = pd.read_excel('data/composers.xlsx', sheet_name='Sheet6')\n",
    "composers2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The indices don't match and are not the composer name. In addition the columns containing the composer names have different labels. Here we can use `merge()` and specify which columns we want to use for merging, and what type of merging we need (inner, left etc.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T10:03:56.561292Z",
     "start_time": "2020-06-25T10:03:56.508980Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>composer</th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>city</th>\n",
       "      <th>last name</th>\n",
       "      <th>first name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Mahler</td>\n",
       "      <td>1860</td>\n",
       "      <td>1911</td>\n",
       "      <td>Kaliste</td>\n",
       "      <td>Mahler</td>\n",
       "      <td>Gustav</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Beethoven</td>\n",
       "      <td>1770</td>\n",
       "      <td>1827</td>\n",
       "      <td>Bonn</td>\n",
       "      <td>Beethoven</td>\n",
       "      <td>Ludwig van</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Puccini</td>\n",
       "      <td>1858</td>\n",
       "      <td>1924</td>\n",
       "      <td>Lucques</td>\n",
       "      <td>Puccini</td>\n",
       "      <td>Giacomo</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    composer  birth  death     city  last name  first name\n",
       "0     Mahler   1860   1911  Kaliste     Mahler      Gustav\n",
       "1  Beethoven   1770   1827     Bonn  Beethoven  Ludwig van\n",
       "2    Puccini   1858   1924  Lucques    Puccini     Giacomo"
      ]
     },
     "execution_count": 140,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.merge(composers1, composers2, left_on='composer', right_on='last name')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again we can use another variety of join than the default inner join."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Splitting\n",
    "\n",
    "Often Pandas tables mix regular variables (e.g. the size of cells in microscopy images) with categorical variables (e.g. the type of cell to which they belong). In that case, it is quite usual to split the data using the category to do computations. Pandas allows to do this very easily."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Grouping"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T08:56:13.601372Z",
     "start_time": "2020-06-25T08:56:13.518195Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>period</th>\n",
       "      <th>country</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Mahler</th>\n",
       "      <td>1860</td>\n",
       "      <td>1911.0</td>\n",
       "      <td>post-romantic</td>\n",
       "      <td>Austria</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beethoven</th>\n",
       "      <td>1770</td>\n",
       "      <td>1827.0</td>\n",
       "      <td>romantic</td>\n",
       "      <td>Germany</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Puccini</th>\n",
       "      <td>1858</td>\n",
       "      <td>1924.0</td>\n",
       "      <td>post-romantic</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Shostakovich</th>\n",
       "      <td>1906</td>\n",
       "      <td>1975.0</td>\n",
       "      <td>modern</td>\n",
       "      <td>Russia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Verdi</th>\n",
       "      <td>1813</td>\n",
       "      <td>1901.0</td>\n",
       "      <td>romantic</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              birth   death         period  country\n",
       "composer                                           \n",
       "Mahler         1860  1911.0  post-romantic  Austria\n",
       "Beethoven      1770  1827.0       romantic  Germany\n",
       "Puccini        1858  1924.0  post-romantic    Italy\n",
       "Shostakovich   1906  1975.0         modern   Russia\n",
       "Verdi          1813  1901.0       romantic    Italy"
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What if we want now to count how many composers we have in each category? In classical computing we would maybe do a for loop to count occurrences. Pandas simplifies this with the `groupby()` function, which actually groups elements by a certain criteria, e.g. a categorical variable like the period:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T08:57:05.723610Z",
     "start_time": "2020-06-25T08:57:05.693209Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<pandas.core.groupby.generic.DataFrameGroupBy object at 0x117068390>"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composer_grouped = composers_df.groupby('period')\n",
    "composer_grouped"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The output is a bit cryptic. What we actually have is a new object called *group* which has a lot of handy properties. First let's see what the groups actually are. As for the Dataframe, let's look at a summary of the object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T08:57:44.681494Z",
     "start_time": "2020-06-25T08:57:44.486278Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr:last-of-type th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"8\" halign=\"left\">birth</th>\n",
       "      <th colspan=\"8\" halign=\"left\">death</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>period</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>baroque</th>\n",
       "      <td>10.0</td>\n",
       "      <td>1663.300000</td>\n",
       "      <td>36.009412</td>\n",
       "      <td>1587.0</td>\n",
       "      <td>1647.0</td>\n",
       "      <td>1676.5</td>\n",
       "      <td>1685.0</td>\n",
       "      <td>1710.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>1720.200000</td>\n",
       "      <td>43.460838</td>\n",
       "      <td>1640.0</td>\n",
       "      <td>1697.25</td>\n",
       "      <td>1736.0</td>\n",
       "      <td>1755.25</td>\n",
       "      <td>1764.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>classic</th>\n",
       "      <td>5.0</td>\n",
       "      <td>1744.400000</td>\n",
       "      <td>12.054045</td>\n",
       "      <td>1731.0</td>\n",
       "      <td>1732.0</td>\n",
       "      <td>1749.0</td>\n",
       "      <td>1754.0</td>\n",
       "      <td>1756.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1801.200000</td>\n",
       "      <td>6.942622</td>\n",
       "      <td>1791.0</td>\n",
       "      <td>1799.00</td>\n",
       "      <td>1801.0</td>\n",
       "      <td>1806.00</td>\n",
       "      <td>1809.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>modern</th>\n",
       "      <td>13.0</td>\n",
       "      <td>1905.692308</td>\n",
       "      <td>28.595992</td>\n",
       "      <td>1854.0</td>\n",
       "      <td>1891.0</td>\n",
       "      <td>1902.0</td>\n",
       "      <td>1918.0</td>\n",
       "      <td>1971.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>1974.090909</td>\n",
       "      <td>26.139834</td>\n",
       "      <td>1928.0</td>\n",
       "      <td>1962.00</td>\n",
       "      <td>1982.0</td>\n",
       "      <td>1990.00</td>\n",
       "      <td>2016.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>post-romantic</th>\n",
       "      <td>5.0</td>\n",
       "      <td>1854.200000</td>\n",
       "      <td>17.123084</td>\n",
       "      <td>1824.0</td>\n",
       "      <td>1858.0</td>\n",
       "      <td>1860.0</td>\n",
       "      <td>1864.0</td>\n",
       "      <td>1865.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1927.400000</td>\n",
       "      <td>25.540164</td>\n",
       "      <td>1896.0</td>\n",
       "      <td>1911.00</td>\n",
       "      <td>1924.0</td>\n",
       "      <td>1949.00</td>\n",
       "      <td>1957.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>renaissance</th>\n",
       "      <td>7.0</td>\n",
       "      <td>1527.142857</td>\n",
       "      <td>59.881629</td>\n",
       "      <td>1397.0</td>\n",
       "      <td>1528.5</td>\n",
       "      <td>1540.0</td>\n",
       "      <td>1564.5</td>\n",
       "      <td>1567.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>1595.285714</td>\n",
       "      <td>56.295986</td>\n",
       "      <td>1474.0</td>\n",
       "      <td>1594.00</td>\n",
       "      <td>1613.0</td>\n",
       "      <td>1624.50</td>\n",
       "      <td>1643.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>romantic</th>\n",
       "      <td>17.0</td>\n",
       "      <td>1824.823529</td>\n",
       "      <td>25.468695</td>\n",
       "      <td>1770.0</td>\n",
       "      <td>1810.0</td>\n",
       "      <td>1824.0</td>\n",
       "      <td>1841.0</td>\n",
       "      <td>1867.0</td>\n",
       "      <td>17.0</td>\n",
       "      <td>1883.588235</td>\n",
       "      <td>28.026904</td>\n",
       "      <td>1827.0</td>\n",
       "      <td>1869.00</td>\n",
       "      <td>1887.0</td>\n",
       "      <td>1904.00</td>\n",
       "      <td>1919.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              birth                                                          \\\n",
       "              count         mean        std     min     25%     50%     75%   \n",
       "period                                                                        \n",
       "baroque        10.0  1663.300000  36.009412  1587.0  1647.0  1676.5  1685.0   \n",
       "classic         5.0  1744.400000  12.054045  1731.0  1732.0  1749.0  1754.0   \n",
       "modern         13.0  1905.692308  28.595992  1854.0  1891.0  1902.0  1918.0   \n",
       "post-romantic   5.0  1854.200000  17.123084  1824.0  1858.0  1860.0  1864.0   \n",
       "renaissance     7.0  1527.142857  59.881629  1397.0  1528.5  1540.0  1564.5   \n",
       "romantic       17.0  1824.823529  25.468695  1770.0  1810.0  1824.0  1841.0   \n",
       "\n",
       "                      death                                                   \\\n",
       "                  max count         mean        std     min      25%     50%   \n",
       "period                                                                         \n",
       "baroque        1710.0  10.0  1720.200000  43.460838  1640.0  1697.25  1736.0   \n",
       "classic        1756.0   5.0  1801.200000   6.942622  1791.0  1799.00  1801.0   \n",
       "modern         1971.0  11.0  1974.090909  26.139834  1928.0  1962.00  1982.0   \n",
       "post-romantic  1865.0   5.0  1927.400000  25.540164  1896.0  1911.00  1924.0   \n",
       "renaissance    1567.0   7.0  1595.285714  56.295986  1474.0  1594.00  1613.0   \n",
       "romantic       1867.0  17.0  1883.588235  28.026904  1827.0  1869.00  1887.0   \n",
       "\n",
       "                                \n",
       "                   75%     max  \n",
       "period                          \n",
       "baroque        1755.25  1764.0  \n",
       "classic        1806.00  1809.0  \n",
       "modern         1990.00  2016.0  \n",
       "post-romantic  1949.00  1957.0  \n",
       "renaissance    1624.50  1643.0  \n",
       "romantic       1904.00  1919.0  "
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composer_grouped.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we have a dataframe with a statistical summary of the the contents. The \"names\" of the groups are here the indices of the Dataframe. These names are simply all the different categories that were present in the column we used for grouping. Now we can recover a single group:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T08:58:36.933456Z",
     "start_time": "2020-06-25T08:58:36.907796Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>period</th>\n",
       "      <th>country</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Haendel</th>\n",
       "      <td>1685</td>\n",
       "      <td>1759.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>Germany</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Purcell</th>\n",
       "      <td>1659</td>\n",
       "      <td>1695.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>England</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Charpentier</th>\n",
       "      <td>1643</td>\n",
       "      <td>1704.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>France</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Couperin</th>\n",
       "      <td>1626</td>\n",
       "      <td>1661.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>France</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Rameau</th>\n",
       "      <td>1683</td>\n",
       "      <td>1764.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>France</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Caldara</th>\n",
       "      <td>1670</td>\n",
       "      <td>1736.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pergolesi</th>\n",
       "      <td>1710</td>\n",
       "      <td>1736.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Scarlatti</th>\n",
       "      <td>1685</td>\n",
       "      <td>1757.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Caccini</th>\n",
       "      <td>1587</td>\n",
       "      <td>1640.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>Italy</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Bach</th>\n",
       "      <td>1685</td>\n",
       "      <td>1750.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>Germany</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             birth   death   period  country\n",
       "composer                                    \n",
       "Haendel       1685  1759.0  baroque  Germany\n",
       "Purcell       1659  1695.0  baroque  England\n",
       "Charpentier   1643  1704.0  baroque   France\n",
       "Couperin      1626  1661.0  baroque   France\n",
       "Rameau        1683  1764.0  baroque   France\n",
       "Caldara       1670  1736.0  baroque    Italy\n",
       "Pergolesi     1710  1736.0  baroque    Italy\n",
       "Scarlatti     1685  1757.0  baroque    Italy\n",
       "Caccini       1587  1640.0  baroque    Italy\n",
       "Bach          1685  1750.0  baroque  Germany"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composer_grouped.get_group('baroque')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If one has multiple categorical variables, one can also do a grouping on several levels. For example here we want to classify composers both by period and country. For this we just give two column names to the `groupby()` function:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T08:59:41.133661Z",
     "start_time": "2020-06-25T08:59:41.051149Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>birth</th>\n",
       "      <th>death</th>\n",
       "      <th>period</th>\n",
       "      <th>country</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>composer</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Haendel</th>\n",
       "      <td>1685</td>\n",
       "      <td>1759.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>Germany</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Bach</th>\n",
       "      <td>1685</td>\n",
       "      <td>1750.0</td>\n",
       "      <td>baroque</td>\n",
       "      <td>Germany</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          birth   death   period  country\n",
       "composer                                 \n",
       "Haendel    1685  1759.0  baroque  Germany\n",
       "Bach       1685  1750.0  baroque  Germany"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composer_grouped = composers_df.groupby(['period','country'])\n",
    "composer_grouped.get_group(('baroque','Germany'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The main advantage of this Group object is that it allows us to do very quickly both computations and plotting without having to loop through different categories. Indeed Pandas makes all the work for us: it applies functions on each group and then reassembles the results into a Dataframe (or Series depending on output).\n",
    "For example we can apply most functions we used for Dataframes (mean, sum etc.) on groups as well and Pandas seamlessly does the work for us."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Unstacking"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's have a look again at one of our grouped Dataframe on which we applied some summary function like a mean on the age column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T09:08:04.942179Z",
     "start_time": "2020-06-25T09:08:04.852600Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "country  period       \n",
       "Austria  classic          56.000000\n",
       "         modern           50.000000\n",
       "         post-romantic    61.500000\n",
       "Belgium  renaissance      69.500000\n",
       "Czechia  classic          68.000000\n",
       "         modern           74.000000\n",
       "         romantic         61.500000\n",
       "England  baroque          36.000000\n",
       "         modern           81.000000\n",
       "         renaissance      73.000000\n",
       "Finland  post-romantic    92.000000\n",
       "France   baroque          59.000000\n",
       "         modern           87.500000\n",
       "         romantic         70.333333\n",
       "Germany  baroque          69.500000\n",
       "         modern           87.000000\n",
       "         post-romantic    85.000000\n",
       "         romantic         59.250000\n",
       "Italy    baroque          54.250000\n",
       "         classic          52.000000\n",
       "         post-romantic    66.000000\n",
       "         renaissance      64.000000\n",
       "         romantic         58.500000\n",
       "RUssia   modern           62.000000\n",
       "Russia   modern           79.000000\n",
       "         romantic         48.000000\n",
       "Spain    classic          52.000000\n",
       "         romantic         49.000000\n",
       "USA      modern           81.000000\n",
       "Name: age, dtype: float64"
      ]
     },
     "execution_count": 121,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composers_df['age'] = composers_df['death'] - composers_df['birth']\n",
    "composers_df.groupby(['country','period']).age.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we have two level of indices, with the main one being the country which contains all periods. Often for plotting we however need to have the information in another format. In particular we would like each of these values to be one observation in a regular table. For example we could have a country vs period table where all elements are the mean age. To do that we need to **unstack** our multi-level Dataframe:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-25T09:08:39.090317Z",
     "start_time": "2020-06-25T09:08:38.994496Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>period</th>\n",
       "      <th>baroque</th>\n",
       "      <th>classic</th>\n",
       "      <th>modern</th>\n",
       "      <th>post-romantic</th>\n",
       "      <th>renaissance</th>\n",
       "      <th>romantic</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>country</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Austria</th>\n",
       "      <td>NaN</td>\n",
       "      <td>56.0</td>\n",
       "      <td>50.0</td>\n",
       "      <td>61.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Belgium</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>69.5</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Czechia</th>\n",
       "      <td>NaN</td>\n",
       "      <td>68.0</td>\n",
       "      <td>74.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>61.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>England</th>\n",
       "      <td>36.00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>81.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>73.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Finland</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>92.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>France</th>\n",
       "      <td>59.00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>87.5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>70.333333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Germany</th>\n",
       "      <td>69.50</td>\n",
       "      <td>NaN</td>\n",
       "      <td>87.0</td>\n",
       "      <td>85.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>59.250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Italy</th>\n",
       "      <td>54.25</td>\n",
       "      <td>52.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>66.0</td>\n",
       "      <td>64.0</td>\n",
       "      <td>58.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>RUssia</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>62.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Russia</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>79.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>48.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Spain</th>\n",
       "      <td>NaN</td>\n",
       "      <td>52.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>49.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>USA</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>81.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "period   baroque  classic  modern  post-romantic  renaissance   romantic\n",
       "country                                                                 \n",
       "Austria      NaN     56.0    50.0           61.5          NaN        NaN\n",
       "Belgium      NaN      NaN     NaN            NaN         69.5        NaN\n",
       "Czechia      NaN     68.0    74.0            NaN          NaN  61.500000\n",
       "England    36.00      NaN    81.0            NaN         73.0        NaN\n",
       "Finland      NaN      NaN     NaN           92.0          NaN        NaN\n",
       "France     59.00      NaN    87.5            NaN          NaN  70.333333\n",
       "Germany    69.50      NaN    87.0           85.0          NaN  59.250000\n",
       "Italy      54.25     52.0     NaN           66.0         64.0  58.500000\n",
       "RUssia       NaN      NaN    62.0            NaN          NaN        NaN\n",
       "Russia       NaN      NaN    79.0            NaN          NaN  48.000000\n",
       "Spain        NaN     52.0     NaN            NaN          NaN  49.000000\n",
       "USA          NaN      NaN    81.0            NaN          NaN        NaN"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "composer_unstacked = composers_df.groupby(['country','period']).age.mean().unstack()\n",
    "composer_unstacked"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}