Data_exploration_student.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Carpentry Reproducible Research Workshop - Data Exploration"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Authors: Brian Avery, Nima Hejazi, Kellie Ottoboni, Clara Sorensen, Niek Veldhuis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Things You'll Need to Complete This Tutorial\n",
    "\n",
    "This lesson uses packages that you've already downloaded along with the Anaconda distribution of Python.\n",
    "\n",
    "### Download Data\n",
    "\n",
    "In this lesson, you will use the Gapminder dataset. Please download the data from the web [here](https://raw.githubusercontent.com/Reproducible-Science-Curriculum/data-exploration-RR-Jupyter/gh-pages/data/gapminderDataFiveYear_superDirty.txt) and [here](https://raw.githubusercontent.com/Reproducible-Science-Curriculum/data-exploration-RR-Jupyter/gh-pages/data/PRB_data.txt) on GitHub."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview\n",
    "\n",
    "- Teaching: 40 min\n",
    "- Exercises: 20 min\n",
    "\n",
    "### Questions\n",
    "\n",
    "- Why is the Jupyter notebook a useful tool for conducting and sharing data exploration?\n",
    "- What is \"tidy data\"?\n",
    "- What steps does one take to explore data?\n",
    "\n",
    "### Objectives\n",
    "Use the Python Pandas library in the Jupyter Notebook to:\n",
    "* Assess the structure and cleanliness of a dataset, including the size and shape of the data, and the number of variables of each type.\n",
    "* Describe findings, translate results from code to text using Markdown comments in the Jupyter Notebook, summarizing your thought process in a narrative.\n",
    "* Modify raw data to prepare a clean data set -- including copying data, removing or replacing missing and incoherent data, dropping columns, removing duplicates.\n",
    "* Assess whether data is “Tidy” and identify appropriate steps and write and  execute code to arrange it into a tidy format - including merging, reshaping, subsetting, grouping, sorting, and making appropriate new columns.\n",
    "* Identify several relevant summary measures.\n",
    "* Illustrate data in plots and determine the need for repeated or further analysis.\n",
    "* Justify these decisions in Markdown in the Jupyter Notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setting up the notebook\n",
    "\n",
    "## About Libraries in Python\n",
    "\n",
    "A library in Python contains a set of tools (called functions) that perform tasks on our data. Importing a library is like getting a piece of lab equipment out of a storage locker and setting it up on the bench for use in a project. Once a library is imported, it can be used or called to perform many tasks.\n",
    "\n",
    "Python doesn’t load all of the libraries available to it by default. We have to add an import statement to our code in order to use library functions. To import a library, we use the syntax `import libraryName`. If we want to give the library a nickname to shorten the command, we can add `as nickNameHere`. An example of importing the Pandas library using the common nickname `pd` is below.\n",
    "\n",
    "**`import`** `pandas` **`as`** `pd`\n",
    "\n",
    "## matplotlib and other plotting libraries\n",
    "\n",
    "matplotlib is the most widely used Python library for plotting.  We can run it in the notebook using the magic command `%matplotlib inline`. If you do not use `%matplotlib inline`, your plots will be generated outside of the notebook and may be difficult to find.  See [the IPython docs](http://ipython.readthedocs.io/en/stable/interactive/plotting.html) for other IPython magics commands.\n",
    "\n",
    "In this lesson, we will only use matplotlib and Seaborn, another package that works in tandem with matplotlib to make nice graphics.  There is a whole range of graphics packages in Python, ranging from basic visualizations to fancy, interactive graphics like [Bokeh](http://bokeh.pydata.org/en/latest/) and [Plotly](https://plot.ly/python/).  \n",
    "\n",
    "We encourage you to explore on your own!  Chances are, if you can imagine a plot you'd like to make, somebody else has written a package to do it.\n",
    "\n",
    "## Markdown\n",
    "Text can be added to Jupyter Notebooks using Markdown cells. Markdown is a popular markup language that is a superset of HTML. To learn more, see [Jupyter's Markdown guide](http://jupyter-notebook.readthedocs.io/en/latest/examples/Notebook/Working%20With%20Markdown%20Cells.html) or revisit the [Reproducible Research lesson on Markdown](https://github.com/Reproducible-Science-Curriculum/introduction-RR-Jupyter/blob/master/notebooks/Navigating%20the%20notebook%20-%20instructor%20script.ipynb). \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Pandas Library\n",
    "\n",
    "One of the best options for working with tabular data in Python is the Python Data Analysis Library (a.k.a. Pandas). The Pandas library is built on top of the NumPy package (another Python library). Pandas provides data structures, produces high quality plots with matplotlib, and integrates nicely with other libraries that use NumPy arrays. Those familiar with spreadsheets should become comfortable with Pandas data structures."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each time we call a function that’s in a library, we use the syntax `LibraryName.FunctionName`. Adding the library name with a `.` before the function name tells Python where to find the function. In the example above, we have imported Pandas as `pd`. This means we don’t have to type out `pandas` each time we call a Pandas function.\n",
    "\n",
    "See this free [Pandas cheat sheet](https://www.datacamp.com/community/blog/python-pandas-cheat-sheet) from DataCamp for the most common Pandas commands. \n",
    "\n",
    "# Getting data into the notebook\n",
    "\n",
    "We will begin by locating and reading our data which are in a table format as a tab-delimited file. We will use Pandas’ `read_table` function to pull the file directly into a `DataFrame`.\n",
    "\n",
    "## What’s a `DataFrame`?\n",
    "A `DataFrame` is a 2-dimensional data structure that can store in columns data of different types (including characters, integers, floating point values, factors and more). It is similar to a spreadsheet or a SQL table or data.frame in R. A `DataFrame` always has an index (0-based). An index refers to the position of an element in the data structure.\n",
    "\n",
    "Note that we use `pd.read_table`, not just `read_table` or `pandas.read_table`, because we imported Pandas as `pd`.\n",
    "\n",
    "In our original file, the columns in the data set are separated by a TAB. We need to tell the `read_table` function in Pandas that that is the delimiter with `sep = ‘\\t’`.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\telmo\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:1: FutureWarning: read_table is deprecated, use read_csv instead.\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n"
     ]
    }
   ],
   "source": [
    "gapminder = pd.read_table(\"gapminderDataFiveYear_superDirty.txt\", sep = \"\\t\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first thing to do when loading data into the notebook is to actually \"look\" at it.  How many rows and columns are there?  What types of variables are in it and what values can they take?\n",
    "\n",
    "There are usually too many rows to print to the screen.  By default, when you type the name of the `DataFrame` and run a cell, Pandas knows to not print the whole thing.  Instead, you will see the first and last few rows with dots in between.  A neater way to see a preview of the dataset is the `head()` method.  Calling `dataset.head()` will display the first 5 rows of the data.  You can specify how many rows you want to see as an argument, like `dataset.head(10)`.  The `tail()` method does the same with the last rows of the `DataFrame`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>pop</th>\n",
       "      <th>life Exp</th>\n",
       "      <th>gdpPercap</th>\n",
       "      <th>region</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1952.0</td>\n",
       "      <td>8425333.0</td>\n",
       "      <td>28.801</td>\n",
       "      <td>779.445314</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1957.0</td>\n",
       "      <td>9240934.0</td>\n",
       "      <td>30.332</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1957.0</td>\n",
       "      <td>9240934.0</td>\n",
       "      <td>30.332</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1962.0</td>\n",
       "      <td>10267083.0</td>\n",
       "      <td>31.997</td>\n",
       "      <td>853.100710</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1962.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>31.997</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     year         pop  life Exp   gdpPercap            region\n",
       "0  1952.0   8425333.0    28.801  779.445314  Asia_Afghanistan\n",
       "1  1957.0   9240934.0    30.332  820.853030  Asia_Afghanistan\n",
       "2  1957.0   9240934.0    30.332  820.853030  Asia_Afghanistan\n",
       "3  1962.0  10267083.0    31.997  853.100710  Asia_Afghanistan\n",
       "4  1962.0         NaN    31.997         NaN  Asia_Afghanistan"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>pop</th>\n",
       "      <th>life Exp</th>\n",
       "      <th>gdpPercap</th>\n",
       "      <th>region</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1715</th>\n",
       "      <td>1987.0</td>\n",
       "      <td>9216418.0</td>\n",
       "      <td>62.351</td>\n",
       "      <td>706.157306</td>\n",
       "      <td>Africa_Zimbabwe</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1716</th>\n",
       "      <td>1992.0</td>\n",
       "      <td>10704340.0</td>\n",
       "      <td>60.377</td>\n",
       "      <td>693.420786</td>\n",
       "      <td>Africa_Zimbabwe</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1717</th>\n",
       "      <td>1997.0</td>\n",
       "      <td>11404948.0</td>\n",
       "      <td>46.809</td>\n",
       "      <td>792.449960</td>\n",
       "      <td>Africa_Zimbabwe</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1718</th>\n",
       "      <td>2002.0</td>\n",
       "      <td>11926563.0</td>\n",
       "      <td>39.989</td>\n",
       "      <td>672.038623</td>\n",
       "      <td>Africa_Zimbabwe</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1719</th>\n",
       "      <td>2007.0</td>\n",
       "      <td>12311143.0</td>\n",
       "      <td>43.487</td>\n",
       "      <td>469.709298</td>\n",
       "      <td>Africa_Zimbabwe</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        year         pop  life Exp   gdpPercap           region\n",
       "1715  1987.0   9216418.0    62.351  706.157306  Africa_Zimbabwe\n",
       "1716  1992.0  10704340.0    60.377  693.420786  Africa_Zimbabwe\n",
       "1717  1997.0  11404948.0    46.809  792.449960  Africa_Zimbabwe\n",
       "1718  2002.0  11926563.0    39.989  672.038623  Africa_Zimbabwe\n",
       "1719  2007.0  12311143.0    43.487  469.709298  Africa_Zimbabwe"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>pop</th>\n",
       "      <th>life Exp</th>\n",
       "      <th>gdpPercap</th>\n",
       "      <th>region</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1952.0</td>\n",
       "      <td>8425333.0</td>\n",
       "      <td>28.801</td>\n",
       "      <td>779.445314</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1957.0</td>\n",
       "      <td>9240934.0</td>\n",
       "      <td>30.332</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1957.0</td>\n",
       "      <td>9240934.0</td>\n",
       "      <td>30.332</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1962.0</td>\n",
       "      <td>10267083.0</td>\n",
       "      <td>31.997</td>\n",
       "      <td>853.100710</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1962.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>31.997</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1967.0</td>\n",
       "      <td>11537966.0</td>\n",
       "      <td>34.020</td>\n",
       "      <td>836.197138</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>1972.0</td>\n",
       "      <td>13079460.0</td>\n",
       "      <td>36.088</td>\n",
       "      <td>739.981106</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>1977.0</td>\n",
       "      <td>14880372.0</td>\n",
       "      <td>38.438</td>\n",
       "      <td>786.113360</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>1982.0</td>\n",
       "      <td>12881816.0</td>\n",
       "      <td>39.854</td>\n",
       "      <td>978.011439</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>1987.0</td>\n",
       "      <td>13867957.0</td>\n",
       "      <td>40.822</td>\n",
       "      <td>852.395945</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>1992.0</td>\n",
       "      <td>16317921.0</td>\n",
       "      <td>41.674</td>\n",
       "      <td>649.341395</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>1997.0</td>\n",
       "      <td>22227415.0</td>\n",
       "      <td>41.763</td>\n",
       "      <td>635.341351</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>2002.0</td>\n",
       "      <td>25268405.0</td>\n",
       "      <td>42.129</td>\n",
       "      <td>726.734055</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>2007.0</td>\n",
       "      <td>31889923.0</td>\n",
       "      <td>43.828</td>\n",
       "      <td>974.580338</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>1952.0</td>\n",
       "      <td>1282697.0</td>\n",
       "      <td>55.230</td>\n",
       "      <td>1601.056136</td>\n",
       "      <td>Europe_Albania</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>1957.0</td>\n",
       "      <td>1476505.0</td>\n",
       "      <td>59.280</td>\n",
       "      <td>1942.284244</td>\n",
       "      <td>Europe_Albania</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>1962.0</td>\n",
       "      <td>1728137.0</td>\n",
       "      <td>64.820</td>\n",
       "      <td>2312.888958</td>\n",
       "      <td>Europe_Albania</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>1967.0</td>\n",
       "      <td>1984060.0</td>\n",
       "      <td>66.220</td>\n",
       "      <td>2760.196931</td>\n",
       "      <td>Europe_Albania</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>1972.0</td>\n",
       "      <td>2263554.0</td>\n",
       "      <td>67.690</td>\n",
       "      <td>3313.422188</td>\n",
       "      <td>Europe_Albania</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>1977.0</td>\n",
       "      <td>2509048.0</td>\n",
       "      <td>68.930</td>\n",
       "      <td>3533.003910</td>\n",
       "      <td>Europe_Albania</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>1982.0</td>\n",
       "      <td>2780097.0</td>\n",
       "      <td>70.420</td>\n",
       "      <td>3630.880722</td>\n",
       "      <td>Europe_Albania</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>1987.0</td>\n",
       "      <td>3075321.0</td>\n",
       "      <td>72.000</td>\n",
       "      <td>3738.932735</td>\n",
       "      <td>Europe_Albania</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>1992.0</td>\n",
       "      <td>3326498.0</td>\n",
       "      <td>71.581</td>\n",
       "      <td>2497.437901</td>\n",
       "      <td>Europe_Albania</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>1997.0</td>\n",
       "      <td>3428038.0</td>\n",
       "      <td>72.950</td>\n",
       "      <td>3193.054604</td>\n",
       "      <td>Europe_Albania</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>2002.0</td>\n",
       "      <td>3508512.0</td>\n",
       "      <td>75.651</td>\n",
       "      <td>4604.211737</td>\n",
       "      <td>Europe_Albania</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>2007.0</td>\n",
       "      <td>3600523.0</td>\n",
       "      <td>76.423</td>\n",
       "      <td>5937.029526</td>\n",
       "      <td>Europe_Albania</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>1952.0</td>\n",
       "      <td>9279525.0</td>\n",
       "      <td>43.077</td>\n",
       "      <td>2449.008185</td>\n",
       "      <td>Africa_Algeria</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>1957.0</td>\n",
       "      <td>10270856.0</td>\n",
       "      <td>45.685</td>\n",
       "      <td>3013.976023</td>\n",
       "      <td>Africa_Algeria</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>1962.0</td>\n",
       "      <td>11000948.0</td>\n",
       "      <td>48.303</td>\n",
       "      <td>2550.816880</td>\n",
       "      <td>Africa_Algeria</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>1967.0</td>\n",
       "      <td>12760499.0</td>\n",
       "      <td>51.407</td>\n",
       "      <td>3246.991771</td>\n",
       "      <td>Africa_Algeria</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1690</th>\n",
       "      <td>1982.0</td>\n",
       "      <td>9657618.0</td>\n",
       "      <td>49.113</td>\n",
       "      <td>1977.557010</td>\n",
       "      <td>Asia_Yemen, Rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1691</th>\n",
       "      <td>1987.0</td>\n",
       "      <td>11219340.0</td>\n",
       "      <td>52.922</td>\n",
       "      <td>1971.741538</td>\n",
       "      <td>Asia_Yemen, Rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1692</th>\n",
       "      <td>1992.0</td>\n",
       "      <td>13367997.0</td>\n",
       "      <td>55.599</td>\n",
       "      <td>1879.496673</td>\n",
       "      <td>Asia_Yemen, Rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1693</th>\n",
       "      <td>1997.0</td>\n",
       "      <td>15826497.0</td>\n",
       "      <td>58.020</td>\n",
       "      <td>2117.484526</td>\n",
       "      <td>Asia_Yemen, Rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1694</th>\n",
       "      <td>2002.0</td>\n",
       "      <td>18701257.0</td>\n",
       "      <td>60.308</td>\n",
       "      <td>2234.820827</td>\n",
       "      <td>Asia_Yemen, Rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1695</th>\n",
       "      <td>2007.0</td>\n",
       "      <td>22211743.0</td>\n",
       "      <td>62.698</td>\n",
       "      <td>2280.769906</td>\n",
       "      <td>Asia_Yemen, Rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1696</th>\n",
       "      <td>1952.0</td>\n",
       "      <td>2672000.0</td>\n",
       "      <td>42.038</td>\n",
       "      <td>1147.388831</td>\n",
       "      <td>Africa_Zambia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1697</th>\n",
       "      <td>1957.0</td>\n",
       "      <td>3016000.0</td>\n",
       "      <td>44.077</td>\n",
       "      <td>1311.956766</td>\n",
       "      <td>Africa_Zambia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1698</th>\n",
       "      <td>1962.0</td>\n",
       "      <td>3421000.0</td>\n",
       "      <td>46.023</td>\n",
       "      <td>1452.725766</td>\n",
       "      <td>Africa_Zambia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1699</th>\n",
       "      <td>1967.0</td>\n",
       "      <td>3900000.0</td>\n",
       "      <td>47.768</td>\n",
       "      <td>1777.077318</td>\n",
       "      <td>Africa_Zambia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1700</th>\n",
       "      <td>1972.0</td>\n",
       "      <td>4506497.0</td>\n",
       "      <td>50.107</td>\n",
       "      <td>1773.498265</td>\n",
       "      <td>Africa_Zambia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1701</th>\n",
       "      <td>1977.0</td>\n",
       "      <td>5216550.0</td>\n",
       "      <td>51.386</td>\n",
       "      <td>1588.688299</td>\n",
       "      <td>Africa_Zambia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1702</th>\n",
       "      <td>1982.0</td>\n",
       "      <td>6100407.0</td>\n",
       "      <td>51.821</td>\n",
       "      <td>1408.678565</td>\n",
       "      <td>Africa_Zambia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1703</th>\n",
       "      <td>1987.0</td>\n",
       "      <td>7272406.0</td>\n",
       "      <td>50.821</td>\n",
       "      <td>1213.315116</td>\n",
       "      <td>Africa_Zambia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1704</th>\n",
       "      <td>1992.0</td>\n",
       "      <td>8381163.0</td>\n",
       "      <td>46.100</td>\n",
       "      <td>1210.884633</td>\n",
       "      <td>Africa_Zambia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1705</th>\n",
       "      <td>1997.0</td>\n",
       "      <td>9417789.0</td>\n",
       "      <td>40.238</td>\n",
       "      <td>1071.353818</td>\n",
       "      <td>Africa_Zambia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1706</th>\n",
       "      <td>2002.0</td>\n",
       "      <td>10595811.0</td>\n",
       "      <td>39.193</td>\n",
       "      <td>1071.613938</td>\n",
       "      <td>Africa_Zambia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1707</th>\n",
       "      <td>2007.0</td>\n",
       "      <td>11746035.0</td>\n",
       "      <td>42.384</td>\n",
       "      <td>1271.211593</td>\n",
       "      <td>Africa_Zambia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1708</th>\n",
       "      <td>1952.0</td>\n",
       "      <td>3080907.0</td>\n",
       "      <td>48.451</td>\n",
       "      <td>406.884115</td>\n",
       "      <td>Africa_Zimbabwe</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1709</th>\n",
       "      <td>1957.0</td>\n",
       "      <td>3646340.0</td>\n",
       "      <td>50.469</td>\n",
       "      <td>518.764268</td>\n",
       "      <td>Africa_Zimbabwe</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1710</th>\n",
       "      <td>1962.0</td>\n",
       "      <td>4277736.0</td>\n",
       "      <td>52.358</td>\n",
       "      <td>527.272182</td>\n",
       "      <td>Africa_Zimbabwe</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1711</th>\n",
       "      <td>1967.0</td>\n",
       "      <td>4995432.0</td>\n",
       "      <td>53.995</td>\n",
       "      <td>569.795071</td>\n",
       "      <td>Africa_Zimbabwe</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1712</th>\n",
       "      <td>1972.0</td>\n",
       "      <td>5861135.0</td>\n",
       "      <td>55.635</td>\n",
       "      <td>799.362176</td>\n",
       "      <td>Africa_Zimbabwe</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1713</th>\n",
       "      <td>1977.0</td>\n",
       "      <td>6642107.0</td>\n",
       "      <td>57.674</td>\n",
       "      <td>685.587682</td>\n",
       "      <td>Africa_Zimbabwe</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1714</th>\n",
       "      <td>1982.0</td>\n",
       "      <td>7636524.0</td>\n",
       "      <td>60.363</td>\n",
       "      <td>788.855041</td>\n",
       "      <td>Africa_Zimbabwe</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1715</th>\n",
       "      <td>1987.0</td>\n",
       "      <td>9216418.0</td>\n",
       "      <td>62.351</td>\n",
       "      <td>706.157306</td>\n",
       "      <td>Africa_Zimbabwe</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1716</th>\n",
       "      <td>1992.0</td>\n",
       "      <td>10704340.0</td>\n",
       "      <td>60.377</td>\n",
       "      <td>693.420786</td>\n",
       "      <td>Africa_Zimbabwe</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1717</th>\n",
       "      <td>1997.0</td>\n",
       "      <td>11404948.0</td>\n",
       "      <td>46.809</td>\n",
       "      <td>792.449960</td>\n",
       "      <td>Africa_Zimbabwe</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1718</th>\n",
       "      <td>2002.0</td>\n",
       "      <td>11926563.0</td>\n",
       "      <td>39.989</td>\n",
       "      <td>672.038623</td>\n",
       "      <td>Africa_Zimbabwe</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1719</th>\n",
       "      <td>2007.0</td>\n",
       "      <td>12311143.0</td>\n",
       "      <td>43.487</td>\n",
       "      <td>469.709298</td>\n",
       "      <td>Africa_Zimbabwe</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1720 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        year         pop  life Exp    gdpPercap            region\n",
       "0     1952.0   8425333.0    28.801   779.445314  Asia_Afghanistan\n",
       "1     1957.0   9240934.0    30.332   820.853030  Asia_Afghanistan\n",
       "2     1957.0   9240934.0    30.332   820.853030  Asia_Afghanistan\n",
       "3     1962.0  10267083.0    31.997   853.100710  Asia_Afghanistan\n",
       "4     1962.0         NaN    31.997          NaN  Asia_Afghanistan\n",
       "5     1967.0  11537966.0    34.020   836.197138  Asia_Afghanistan\n",
       "6     1972.0  13079460.0    36.088   739.981106  Asia_Afghanistan\n",
       "7     1977.0  14880372.0    38.438   786.113360  Asia_Afghanistan\n",
       "8     1982.0  12881816.0    39.854   978.011439  Asia_Afghanistan\n",
       "9     1987.0  13867957.0    40.822   852.395945  Asia_Afghanistan\n",
       "10    1992.0  16317921.0    41.674   649.341395  Asia_Afghanistan\n",
       "11    1997.0  22227415.0    41.763   635.341351  Asia_Afghanistan\n",
       "12    2002.0  25268405.0    42.129   726.734055  Asia_Afghanistan\n",
       "13    2007.0  31889923.0    43.828   974.580338  Asia_Afghanistan\n",
       "14    1952.0   1282697.0    55.230  1601.056136    Europe_Albania\n",
       "15    1957.0   1476505.0    59.280  1942.284244    Europe_Albania\n",
       "16    1962.0   1728137.0    64.820  2312.888958    Europe_Albania\n",
       "17    1967.0   1984060.0    66.220  2760.196931    Europe_Albania\n",
       "18    1972.0   2263554.0    67.690  3313.422188    Europe_Albania\n",
       "19    1977.0   2509048.0    68.930  3533.003910    Europe_Albania\n",
       "20    1982.0   2780097.0    70.420  3630.880722    Europe_Albania\n",
       "21    1987.0   3075321.0    72.000  3738.932735    Europe_Albania\n",
       "22    1992.0   3326498.0    71.581  2497.437901    Europe_Albania\n",
       "23    1997.0   3428038.0    72.950  3193.054604    Europe_Albania\n",
       "24    2002.0   3508512.0    75.651  4604.211737    Europe_Albania\n",
       "25    2007.0   3600523.0    76.423  5937.029526    Europe_Albania\n",
       "26    1952.0   9279525.0    43.077  2449.008185    Africa_Algeria\n",
       "27    1957.0  10270856.0    45.685  3013.976023    Africa_Algeria\n",
       "28    1962.0  11000948.0    48.303  2550.816880    Africa_Algeria\n",
       "29    1967.0  12760499.0    51.407  3246.991771    Africa_Algeria\n",
       "...      ...         ...       ...          ...               ...\n",
       "1690  1982.0   9657618.0    49.113  1977.557010  Asia_Yemen, Rep.\n",
       "1691  1987.0  11219340.0    52.922  1971.741538  Asia_Yemen, Rep.\n",
       "1692  1992.0  13367997.0    55.599  1879.496673  Asia_Yemen, Rep.\n",
       "1693  1997.0  15826497.0    58.020  2117.484526  Asia_Yemen, Rep.\n",
       "1694  2002.0  18701257.0    60.308  2234.820827  Asia_Yemen, Rep.\n",
       "1695  2007.0  22211743.0    62.698  2280.769906  Asia_Yemen, Rep.\n",
       "1696  1952.0   2672000.0    42.038  1147.388831     Africa_Zambia\n",
       "1697  1957.0   3016000.0    44.077  1311.956766     Africa_Zambia\n",
       "1698  1962.0   3421000.0    46.023  1452.725766     Africa_Zambia\n",
       "1699  1967.0   3900000.0    47.768  1777.077318     Africa_Zambia\n",
       "1700  1972.0   4506497.0    50.107  1773.498265     Africa_Zambia\n",
       "1701  1977.0   5216550.0    51.386  1588.688299     Africa_Zambia\n",
       "1702  1982.0   6100407.0    51.821  1408.678565     Africa_Zambia\n",
       "1703  1987.0   7272406.0    50.821  1213.315116     Africa_Zambia\n",
       "1704  1992.0   8381163.0    46.100  1210.884633     Africa_Zambia\n",
       "1705  1997.0   9417789.0    40.238  1071.353818     Africa_Zambia\n",
       "1706  2002.0  10595811.0    39.193  1071.613938     Africa_Zambia\n",
       "1707  2007.0  11746035.0    42.384  1271.211593     Africa_Zambia\n",
       "1708  1952.0   3080907.0    48.451   406.884115   Africa_Zimbabwe\n",
       "1709  1957.0   3646340.0    50.469   518.764268   Africa_Zimbabwe\n",
       "1710  1962.0   4277736.0    52.358   527.272182   Africa_Zimbabwe\n",
       "1711  1967.0   4995432.0    53.995   569.795071   Africa_Zimbabwe\n",
       "1712  1972.0   5861135.0    55.635   799.362176   Africa_Zimbabwe\n",
       "1713  1977.0   6642107.0    57.674   685.587682   Africa_Zimbabwe\n",
       "1714  1982.0   7636524.0    60.363   788.855041   Africa_Zimbabwe\n",
       "1715  1987.0   9216418.0    62.351   706.157306   Africa_Zimbabwe\n",
       "1716  1992.0  10704340.0    60.377   693.420786   Africa_Zimbabwe\n",
       "1717  1997.0  11404948.0    46.809   792.449960   Africa_Zimbabwe\n",
       "1718  2002.0  11926563.0    39.989   672.038623   Africa_Zimbabwe\n",
       "1719  2007.0  12311143.0    43.487   469.709298   Africa_Zimbabwe\n",
       "\n",
       "[1720 rows x 5 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes the table has too many columns to print on screen. Calling `df.columns.values` will print all the column names in an array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['year', 'pop', 'life Exp', 'gdpPercap', 'region'], dtype=object)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder.columns.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Assess the structure and cleanliness\n",
    "\n",
    "\n",
    "## How many rows and columns are in the data?\n",
    "We often want to know how many rows and columns are in the data -- what is the \"shape\" of the `DataFrame`. Shape is an attribute of the `DataFrame`. Pandas has a convenient way for getting that information by using `DataFrame.shape`  (using `DataFrame` here as a generic name for your `DataFrame`). This returns a tuple (immutable values separated by commas) representing the dimensions of the `DataFrame` (rows, columns).<p>\n",
    "To get the shape of the gapminder `DataFrame`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1720, 5)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can learn even more about our `DataFrame`. The `info()` method gives a few useful pieces of information, including the shape of the `DataFrame`, the variable type of each column, and the amount of memory stored.\n",
    "\n",
    "The output from `info()` displayed below shows that the fields ‘year’ and ‘pop’ (population) are represented as ‘float’ (that is: numbers with a decimal point). This is not appropriate: year and population should be integers or whole numbers. We can change the data-type with the function `astype()`. The code for `astype()` is shown below; however, we will change the data types later in this lesson."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 1720 entries, 0 to 1719\n",
      "Data columns (total 5 columns):\n",
      "year         1717 non-null float64\n",
      "pop          1715 non-null float64\n",
      "life Exp     1719 non-null float64\n",
      "gdpPercap    1718 non-null float64\n",
      "region       1720 non-null object\n",
      "dtypes: float64(4), object(1)\n",
      "memory usage: 67.3+ KB\n"
     ]
    }
   ],
   "source": [
    "gapminder.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `describe()` method will take the numeric columns and provide a summary of their values. This is useful for getting a sense of the ranges of values and seeing if there are any unusual or suspicious numbers.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>pop</th>\n",
       "      <th>life Exp</th>\n",
       "      <th>gdpPercap</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1717.000000</td>\n",
       "      <td>1.715000e+03</td>\n",
       "      <td>1719.000000</td>\n",
       "      <td>1718.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>1979.449039</td>\n",
       "      <td>2.954973e+07</td>\n",
       "      <td>59.436493</td>\n",
       "      <td>7208.452200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>17.228189</td>\n",
       "      <td>1.058366e+08</td>\n",
       "      <td>12.931041</td>\n",
       "      <td>9838.590984</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>1952.000000</td>\n",
       "      <td>6.001100e+04</td>\n",
       "      <td>23.599000</td>\n",
       "      <td>241.165877</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>1967.000000</td>\n",
       "      <td>2.807799e+06</td>\n",
       "      <td>48.143000</td>\n",
       "      <td>1203.161887</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1977.000000</td>\n",
       "      <td>7.026113e+06</td>\n",
       "      <td>60.523000</td>\n",
       "      <td>3529.585686</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1992.000000</td>\n",
       "      <td>1.959366e+07</td>\n",
       "      <td>70.840500</td>\n",
       "      <td>9322.285886</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>2007.000000</td>\n",
       "      <td>1.318683e+09</td>\n",
       "      <td>82.603000</td>\n",
       "      <td>113523.132900</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              year           pop     life Exp      gdpPercap\n",
       "count  1717.000000  1.715000e+03  1719.000000    1718.000000\n",
       "mean   1979.449039  2.954973e+07    59.436493    7208.452200\n",
       "std      17.228189  1.058366e+08    12.931041    9838.590984\n",
       "min    1952.000000  6.001100e+04    23.599000     241.165877\n",
       "25%    1967.000000  2.807799e+06    48.143000    1203.161887\n",
       "50%    1977.000000  7.026113e+06    60.523000    3529.585686\n",
       "75%    1992.000000  1.959366e+07    70.840500    9322.285886\n",
       "max    2007.000000  1.318683e+09    82.603000  113523.132900"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `DataFrame` function `describe()` just blindly looks at all numeric variables. We wouldn't actually want to take the mean year. Additionally, we obtain ‘NaN’ values for our quartiles. This suggests we might have missing data which we can (and will) deal with shortly when we begin to clean our data.\n",
    "\n",
    "For now, let's pull out only the columns that are truly continuous numbers (i.e. ignore the description for ‘year’). This is a preview of selecting columns from the data; we'll talk more about how to do it later in the lesson."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pop</th>\n",
       "      <th>life Exp</th>\n",
       "      <th>gdpPercap</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1.715000e+03</td>\n",
       "      <td>1719.000000</td>\n",
       "      <td>1718.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>2.954973e+07</td>\n",
       "      <td>59.436493</td>\n",
       "      <td>7208.452200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.058366e+08</td>\n",
       "      <td>12.931041</td>\n",
       "      <td>9838.590984</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>6.001100e+04</td>\n",
       "      <td>23.599000</td>\n",
       "      <td>241.165877</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>2.807799e+06</td>\n",
       "      <td>48.143000</td>\n",
       "      <td>1203.161887</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>7.026113e+06</td>\n",
       "      <td>60.523000</td>\n",
       "      <td>3529.585686</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.959366e+07</td>\n",
       "      <td>70.840500</td>\n",
       "      <td>9322.285886</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.318683e+09</td>\n",
       "      <td>82.603000</td>\n",
       "      <td>113523.132900</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                pop     life Exp      gdpPercap\n",
       "count  1.715000e+03  1719.000000    1718.000000\n",
       "mean   2.954973e+07    59.436493    7208.452200\n",
       "std    1.058366e+08    12.931041    9838.590984\n",
       "min    6.001100e+04    23.599000     241.165877\n",
       "25%    2.807799e+06    48.143000    1203.161887\n",
       "50%    7.026113e+06    60.523000    3529.585686\n",
       "75%    1.959366e+07    70.840500    9322.285886\n",
       "max    1.318683e+09    82.603000  113523.132900"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder[['pop','life Exp', 'gdpPercap']].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also extract one specific variable metric at a time if we wish:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "59.43649312390918\n"
     ]
    }
   ],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Values in columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's say you want to see all the unique values for the `region` column. One way to do this is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Europe_Netherlands                         15\n",
      "Asia_Afghanistan                           14\n",
      "Americas_Dominican Republic                14\n",
      "Americas_Bolivia                           14\n",
      "Africa_Nigeria                             13\n",
      "Americas_Argentina                         13\n",
      "Asia_Bangladesh                            13\n",
      "Asia_West Bank and Gaza                    13\n",
      "Africa_Burundi                             13\n",
      "Africa_Algeria                             13\n",
      "Asia_Korea, Rep.                           12\n",
      "Asia_Mongolia                              12\n",
      "Oceania_New Zealand                        12\n",
      "Africa_Burkina Faso                        12\n",
      "Asia_Saudi Arabia                          12\n",
      "Asia_Sri Lanka                             12\n",
      "Americas_El Salvador                       12\n",
      "Africa_South Africa                        12\n",
      "Europe_Croatia                             12\n",
      "Asia_Cambodia                              12\n",
      "Americas_Mexico                            12\n",
      "Europe_Poland                              12\n",
      "Americas_Peru                              12\n",
      "Asia_Oman                                  12\n",
      "Africa_Lesotho                             12\n",
      "Africa_Eritrea                             12\n",
      "Africa_Tanzania                            12\n",
      "Europe_Montenegro                          12\n",
      "Europe_Hungary                             12\n",
      "Europe_France                              12\n",
      "                                           ..\n",
      "Africa_Rwanda                              12\n",
      "Americas_Venezuela                         12\n",
      "Africa_Uganda                              12\n",
      "Africa_Gambia                              12\n",
      "Americas_Nicaragua                         12\n",
      "Africa_Ghana                               12\n",
      "Americas_United States                     12\n",
      "Asia_Indonesia                             12\n",
      "Asia_Thailand                              12\n",
      "Americas_Puerto Rico                       12\n",
      "Americas_Chile                             12\n",
      "Africa_Togo                                12\n",
      "Americas_Ecuador                           12\n",
      "Africa_Cote d'Ivoire                       11\n",
      "Asia_Jordan                                11\n",
      "Americas_Colombia                          11\n",
      "Asia_Korea, Dem. Rep.                      10\n",
      "Africa_Congo, Dem. Rep.                    10\n",
      "Americas_Canada                             9\n",
      "Africa_Central African Republic             8\n",
      "Asia_China                                  8\n",
      "Africa_Central african republic             4\n",
      "Asia_china                                  4\n",
      "_Canada                                     3\n",
      "Americas_Colombia                           2\n",
      "    Asia_Korea, Dem. Rep.                   2\n",
      "Africa_Cote d'Ivore                         1\n",
      "Africa_Democratic Republic of the Congo     1\n",
      "Africa_Congo, Democratic Republic           1\n",
      "    Asia_Jordan                             1\n",
      "Name: region, Length: 151, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "gapminder2 = gapminder.region.value_counts()\n",
    "print (gapminder2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This output is useful, but it looks like there may be some formatting issues causing the same region to be counted more than once. Let's take it a step further and find out to be sure. \n",
    "\n",
    "As mentioned previously, the command `value_counts()` gives you a first global idea of your categorical data such as strings. In this case that is the column `region`. Run the code below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "151\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Europe_Netherlands                         15\n",
       "Asia_Afghanistan                           14\n",
       "Americas_Dominican Republic                14\n",
       "Americas_Bolivia                           14\n",
       "Africa_Nigeria                             13\n",
       "Americas_Argentina                         13\n",
       "Asia_Bangladesh                            13\n",
       "Asia_West Bank and Gaza                    13\n",
       "Africa_Burundi                             13\n",
       "Africa_Algeria                             13\n",
       "Asia_Korea, Rep.                           12\n",
       "Asia_Mongolia                              12\n",
       "Oceania_New Zealand                        12\n",
       "Africa_Burkina Faso                        12\n",
       "Asia_Saudi Arabia                          12\n",
       "Asia_Sri Lanka                             12\n",
       "Americas_El Salvador                       12\n",
       "Africa_South Africa                        12\n",
       "Europe_Croatia                             12\n",
       "Asia_Cambodia                              12\n",
       "Americas_Mexico                            12\n",
       "Europe_Poland                              12\n",
       "Americas_Peru                              12\n",
       "Asia_Oman                                  12\n",
       "Africa_Lesotho                             12\n",
       "Africa_Eritrea                             12\n",
       "Africa_Tanzania                            12\n",
       "Europe_Montenegro                          12\n",
       "Europe_Hungary                             12\n",
       "Europe_France                              12\n",
       "                                           ..\n",
       "Africa_Rwanda                              12\n",
       "Americas_Venezuela                         12\n",
       "Africa_Uganda                              12\n",
       "Africa_Gambia                              12\n",
       "Americas_Nicaragua                         12\n",
       "Africa_Ghana                               12\n",
       "Americas_United States                     12\n",
       "Asia_Indonesia                             12\n",
       "Asia_Thailand                              12\n",
       "Americas_Puerto Rico                       12\n",
       "Americas_Chile                             12\n",
       "Africa_Togo                                12\n",
       "Americas_Ecuador                           12\n",
       "Africa_Cote d'Ivoire                       11\n",
       "Asia_Jordan                                11\n",
       "Americas_Colombia                          11\n",
       "Asia_Korea, Dem. Rep.                      10\n",
       "Africa_Congo, Dem. Rep.                    10\n",
       "Americas_Canada                             9\n",
       "Africa_Central African Republic             8\n",
       "Asia_China                                  8\n",
       "Africa_Central african republic             4\n",
       "Asia_china                                  4\n",
       "_Canada                                     3\n",
       "Americas_Colombia                           2\n",
       "    Asia_Korea, Dem. Rep.                   2\n",
       "Africa_Cote d'Ivore                         1\n",
       "Africa_Democratic Republic of the Congo     1\n",
       "Africa_Congo, Democratic Republic           1\n",
       "    Asia_Jordan                             1\n",
       "Name: region, Length: 151, dtype: int64"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# How many unique regions are in the data?\n",
    "print(len(gapminder.region.unique()))\n",
    "\n",
    "# How many times does each unique region occur?\n",
    "gapminder.region.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The table reveals some problems in our data set. The data set covers 12 years, so each ‘region’ should appear 12 times, but some regions appear more than 12 times and others fewer than 12 times. We also see inconsistencies in the region names (string variables are very susceptible to those), for instance:\n",
    "\n",
    "Asia_china\tvs. Asia_China\n",
    "\n",
    "Another type of problem we see is the various names of 'Congo'. In order to analyze this dataset appropriately we need to take care of these issues. We will fix them in the next section on data cleaning."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercises\n",
    "Are there other columns in our `DataFrame` that have categorical variables? If so, run some code to list the categories below. Save your list to a variable and count the number of unique categories using `len`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "13"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Years = pd.unique(gapminder.year)\n",
    "len(Years)\n",
    "#can make print(len(gapminder...))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is the outcome when you run `value_counts()`?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1967.0    146\n",
       "1977.0    145\n",
       "1982.0    144\n",
       "1972.0    143\n",
       "1962.0    143\n",
       "1992.0    143\n",
       "1957.0    143\n",
       "1987.0    142\n",
       "1997.0    142\n",
       "2007.0    142\n",
       "2002.0    142\n",
       "1952.0    142\n",
       "Name: year, dtype: int64"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder.year.value_counts()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data cleaning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Referencing objects vs copying objects\n",
    "Before we get started with cleaning our data, let's practice good data hygiene by first creating a copy of our original data set. Often, you want to leave the original data untouched.  To protect your original, you can make a copy of your data (and save it to a new `DataFrame` variable) before operating on the data or a subset of the data.  This will ensure that a new version of the original data is created and your original is preserved.\n",
    "\n",
    "###### Why this is important\n",
    "Suppose you take a subset of your `DataFrame` and store it in a new variable, like `gapminder_early = gapminder[gapminder['year'] < 1970]`.  Doing this does not actually create a new object. Instead, you have just given a name to that subset of the original data: `gapminder_early`. This subset still points to the original rows of `gapminder`.  Any changes you make to the new `DataFrame` `gapminder_early` will appear in the corresponding rows of your original `gapminder` `DataFrame` too.  \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>pop</th>\n",
       "      <th>life Exp</th>\n",
       "      <th>gdpPercap</th>\n",
       "      <th>region</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1952.0</td>\n",
       "      <td>8425333.0</td>\n",
       "      <td>28.801</td>\n",
       "      <td>779.445314</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1957.0</td>\n",
       "      <td>9240934.0</td>\n",
       "      <td>30.332</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1957.0</td>\n",
       "      <td>9240934.0</td>\n",
       "      <td>30.332</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1962.0</td>\n",
       "      <td>10267083.0</td>\n",
       "      <td>31.997</td>\n",
       "      <td>853.100710</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1962.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>31.997</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     year         pop  life Exp   gdpPercap            region\n",
       "0  1952.0   8425333.0    28.801  779.445314  Asia_Afghanistan\n",
       "1  1957.0   9240934.0    30.332  820.853030  Asia_Afghanistan\n",
       "2  1957.0   9240934.0    30.332  820.853030  Asia_Afghanistan\n",
       "3  1962.0  10267083.0    31.997  853.100710  Asia_Afghanistan\n",
       "4  1962.0         NaN    31.997         NaN  Asia_Afghanistan"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder = pd.read_csv(\"gapminderDataFiveYear_superDirty.txt\", sep = \"\\t\")\n",
    "gapminder_copy = gapminder.copy()\n",
    "gapminder_copy.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Handling Missing Data\n",
    "\n",
    "Missing data (often denoted as 'NaN'- not a number- in Pandas, or as 'null') is an important issue to handle because Pandas cannot compute on rows or columns with missing data. 'NaN' or 'null' does not mean the value at that position is zero, it means that there is no information at that position. Ignoring missing data doesn't make it go away. There are different ways of dealing with it which include:\n",
    "\n",
    "* analyzing only the available data (i.e. ignore the missing data)\n",
    "* input the missing data with replacement values and treating these as though they were observed\n",
    "* input the missing data and account for the fact that these were inputed with uncertainty (ex: create a new boolean variable so you know that these values were not actually observed)\n",
    "* use statistical models to allow for missing data--make assumptions about their relationships with the available data as necessary\n",
    "\n",
    "For our purposes with the dirty gapminder data set, we know our missing data is excess (and unnecessary) and we are going to choose to analyze only the available data. To do this, we will simply remove rows with missing values.\n",
    "\n",
    "This is incredibly easy to do because Pandas allows you to either remove all instances with null data or replace them with a particular value.\n",
    "\n",
    "`df = df.dropna()` drops rows with any column having NA/null data.  `df = df.fillna(value)` replaces all NA/null data with the argument `value`.\n",
    "\n",
    "For more fine-grained control of which rows (or columns) to drop, you can use `how` or `thresh`. These are more advanced topics and are not covered in this lesson; you are encouraged to explore them on your own."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>pop</th>\n",
       "      <th>life Exp</th>\n",
       "      <th>gdpPercap</th>\n",
       "      <th>region</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1952.0</td>\n",
       "      <td>8425333.0</td>\n",
       "      <td>28.801</td>\n",
       "      <td>779.445314</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1957.0</td>\n",
       "      <td>9240934.0</td>\n",
       "      <td>30.332</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1957.0</td>\n",
       "      <td>9240934.0</td>\n",
       "      <td>30.332</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1962.0</td>\n",
       "      <td>10267083.0</td>\n",
       "      <td>31.997</td>\n",
       "      <td>853.100710</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1967.0</td>\n",
       "      <td>11537966.0</td>\n",
       "      <td>34.020</td>\n",
       "      <td>836.197138</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     year         pop  life Exp   gdpPercap            region\n",
       "0  1952.0   8425333.0    28.801  779.445314  Asia_Afghanistan\n",
       "1  1957.0   9240934.0    30.332  820.853030  Asia_Afghanistan\n",
       "2  1957.0   9240934.0    30.332  820.853030  Asia_Afghanistan\n",
       "3  1962.0  10267083.0    31.997  853.100710  Asia_Afghanistan\n",
       "5  1967.0  11537966.0    34.020  836.197138  Asia_Afghanistan"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_copy = gapminder_copy.dropna()\n",
    "gapminder_copy.head()\n",
    "\n",
    "#note how indexes don't get reorganized. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Changing Data Types\n",
    "We can change the data-type with the function `astype()`. The code for `astype()` is shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 1712 entries, 0 to 1719\n",
      "Data columns (total 5 columns):\n",
      "year         1712 non-null int32\n",
      "pop          1712 non-null int32\n",
      "life Exp     1712 non-null float64\n",
      "gdpPercap    1712 non-null float64\n",
      "region       1712 non-null object\n",
      "dtypes: float64(2), int32(2), object(1)\n",
      "memory usage: 66.9+ KB\n"
     ]
    }
   ],
   "source": [
    "gapminder_copy['year'] = gapminder_copy['year'].astype(int)\n",
    "gapminder_copy['pop'] = gapminder_copy['pop'].astype(int)\n",
    "gapminder_copy.info()\n",
    "\n",
    "#the astype(int) specifies for integer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Handling (Unwanted) Repetitive Data\n",
    "You can identify which observations are duplicates.\n",
    "The call `df.duplicated()` will return boolean values for each row in the `DataFrame` telling you whether or not a row is repeated.\n",
    "\n",
    "In cases where you don’t want repeated values (we wouldn’t--we only want each country to be represented once for every relevant year), you can easily drop such duplicate rows with the call `df.drop_duplicates()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       False\n",
       "1       False\n",
       "2        True\n",
       "3       False\n",
       "5       False\n",
       "6       False\n",
       "7       False\n",
       "8       False\n",
       "9       False\n",
       "10      False\n",
       "11      False\n",
       "12      False\n",
       "13      False\n",
       "14      False\n",
       "15      False\n",
       "16      False\n",
       "17      False\n",
       "18      False\n",
       "19      False\n",
       "20      False\n",
       "21      False\n",
       "22      False\n",
       "23      False\n",
       "24      False\n",
       "25      False\n",
       "26      False\n",
       "27      False\n",
       "28      False\n",
       "29      False\n",
       "30      False\n",
       "        ...  \n",
       "1690    False\n",
       "1691    False\n",
       "1692    False\n",
       "1693    False\n",
       "1694    False\n",
       "1695    False\n",
       "1696    False\n",
       "1697    False\n",
       "1698    False\n",
       "1699    False\n",
       "1700    False\n",
       "1701    False\n",
       "1702    False\n",
       "1703    False\n",
       "1704    False\n",
       "1705    False\n",
       "1706    False\n",
       "1707    False\n",
       "1708    False\n",
       "1709    False\n",
       "1710    False\n",
       "1711    False\n",
       "1712    False\n",
       "1713    False\n",
       "1714    False\n",
       "1715    False\n",
       "1716    False\n",
       "1717    False\n",
       "1718    False\n",
       "1719    False\n",
       "Length: 1712, dtype: bool"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_copy.duplicated()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at the first five rows of our data set again (remember we removed the NaNs):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>pop</th>\n",
       "      <th>life Exp</th>\n",
       "      <th>gdpPercap</th>\n",
       "      <th>region</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1952</td>\n",
       "      <td>8425333</td>\n",
       "      <td>28.801</td>\n",
       "      <td>779.445314</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1957</td>\n",
       "      <td>9240934</td>\n",
       "      <td>30.332</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1957</td>\n",
       "      <td>9240934</td>\n",
       "      <td>30.332</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1962</td>\n",
       "      <td>10267083</td>\n",
       "      <td>31.997</td>\n",
       "      <td>853.100710</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1967</td>\n",
       "      <td>11537966</td>\n",
       "      <td>34.020</td>\n",
       "      <td>836.197138</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   year       pop  life Exp   gdpPercap            region\n",
       "0  1952   8425333    28.801  779.445314  Asia_Afghanistan\n",
       "1  1957   9240934    30.332  820.853030  Asia_Afghanistan\n",
       "2  1957   9240934    30.332  820.853030  Asia_Afghanistan\n",
       "3  1962  10267083    31.997  853.100710  Asia_Afghanistan\n",
       "5  1967  11537966    34.020  836.197138  Asia_Afghanistan"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_copy.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our statement from above is correct, rows 1 & 2 are duplicated. Let's fix that:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "drop_duplicates() got an unexpected keyword argument 'drop'",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-52-9980d67930cd>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mgapminder_copy\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mgapminder_copy\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdrop_duplicates\u001b[0m \u001b[1;33m(\u001b[0m\u001b[0mdrop\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mTrue\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      2\u001b[0m \u001b[0mgapminder_copy\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mhead\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mTypeError\u001b[0m: drop_duplicates() got an unexpected keyword argument 'drop'"
     ]
    }
   ],
   "source": [
    "gapminder_copy = gapminder_copy.drop_duplicates(drop=True)\n",
    "gapminder_copy.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reindexing with `reset_index()`\n",
    "Now we have 1704 rows, but our indexes are off because we removed duplicate rows. We can reset our indices easily with the call `reset_index(drop=True)`. Remember, Python is 0-indexed so our indices will be valued 0-1703.\n",
    "\n",
    "The concept of reindexing is important. When we removed some of the messier, unwanted data, we had \"gaps\" in our index values. By correcting this, we can improve our search functionality and our ability to perform iterative functions on our cleaned data set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>pop</th>\n",
       "      <th>life Exp</th>\n",
       "      <th>gdpPercap</th>\n",
       "      <th>region</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1952</td>\n",
       "      <td>8425333</td>\n",
       "      <td>28.801</td>\n",
       "      <td>779.445314</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1957</td>\n",
       "      <td>9240934</td>\n",
       "      <td>30.332</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1957</td>\n",
       "      <td>9240934</td>\n",
       "      <td>30.332</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1962</td>\n",
       "      <td>10267083</td>\n",
       "      <td>31.997</td>\n",
       "      <td>853.100710</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1967</td>\n",
       "      <td>11537966</td>\n",
       "      <td>34.020</td>\n",
       "      <td>836.197138</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   year       pop  life Exp   gdpPercap            region\n",
       "0  1952   8425333    28.801  779.445314  Asia_Afghanistan\n",
       "1  1957   9240934    30.332  820.853030  Asia_Afghanistan\n",
       "2  1957   9240934    30.332  820.853030  Asia_Afghanistan\n",
       "3  1962  10267083    31.997  853.100710  Asia_Afghanistan\n",
       "4  1967  11537966    34.020  836.197138  Asia_Afghanistan"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_copy = gapminder_copy.reset_index(drop = True)\n",
    "gapminder_copy.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Handling Inconsistent Data\n",
    "\n",
    "The `region` column is a bit too messy for what we'd like to do.\n",
    "The `value_counts()` operation above revealed some issues that we can solve with several different techniques.\n",
    "\n",
    "### String manipulations\n",
    "\n",
    "Common problems with string variables are leading and trailing white space and upper case vs. lower case in the same data set.\n",
    "\n",
    "The following three commands remove all such lingering spaces (left and right) and put everything in lowercase. If you prefer, the three commands can be written in one single line (which is a concept called chaining). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "americas_dominican republic                14\n",
       "europe_netherlands                         14\n",
       "americas_bolivia                           13\n",
       "africa_burundi                             13\n",
       "asia_bangladesh                            13\n",
       "asia_afghanistan                           13\n",
       "africa_congo, rep.                         12\n",
       "africa_sao tome and principe               12\n",
       "africa_guinea-bissau                       12\n",
       "asia_china                                 12\n",
       "africa_mozambique                          12\n",
       "asia_jordan                                12\n",
       "asia_cambodia                              12\n",
       "africa_tanzania                            12\n",
       "asia_nepal                                 12\n",
       "africa_guinea                              12\n",
       "asia_philippines                           12\n",
       "asia_singapore                             12\n",
       "africa_gambia                              12\n",
       "americas_paraguay                          12\n",
       "africa_eritrea                             12\n",
       "americas_argentina                         12\n",
       "asia_mongolia                              12\n",
       "europe_bosnia and herzegovina              12\n",
       "africa_benin                               12\n",
       "americas_panama                            12\n",
       "americas_ecuador                           12\n",
       "asia_korea, rep.                           12\n",
       "europe_montenegro                          12\n",
       "europe_denmark                             12\n",
       "                                           ..\n",
       "asia_saudi arabia                          12\n",
       "africa_djibouti                            12\n",
       "africa_niger                               12\n",
       "asia_myanmar                               12\n",
       "africa_swaziland                           12\n",
       "europe_austria                             12\n",
       "americas_venezuela                         12\n",
       "americas_brazil                            12\n",
       "europe_france                              12\n",
       "americas_peru                              12\n",
       "americas_trinidad and tobago               12\n",
       "asia_syria                                 12\n",
       "europe_czech republic                      12\n",
       "africa_egypt                               12\n",
       "europe_poland                              12\n",
       "asia_lebanon                               12\n",
       "asia_west bank and gaza                    12\n",
       "europe_albania                             12\n",
       "africa_libya                               12\n",
       "americas_puerto rico                       12\n",
       "europe_italy                               12\n",
       "asia_india                                 12\n",
       "europe_switzerland                         12\n",
       "africa_cote d'ivoire                       11\n",
       "africa_congo, dem. rep.                    10\n",
       "americas_canada                             9\n",
       "_canada                                     3\n",
       "africa_congo, democratic republic           1\n",
       "africa_democratic republic of the congo     1\n",
       "africa_cote d'ivore                         1\n",
       "Name: region, Length: 146, dtype: int64"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_copy['region'] = gapminder_copy['region'].str.lstrip() # Strip white space on left\n",
    "gapminder_copy['region'] = gapminder_copy['region'].str.rstrip() # Strip white space on right\n",
    "gapminder_copy['region'] = gapminder_copy['region'].str.lower() # Convert to lowercase\n",
    "gapminder_copy['region'].value_counts() # How many times does each unique region occur?\n",
    "\n",
    "# We could have done this in one line!\n",
    "# gapminder_copy['region'] = gapminder_copy['region'].str.lstrip().str.rstrip().lower()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### regex + `replace()`\n",
    "\n",
    "A regular expression, a.k.a. regex, is a sequence of characters that define a search pattern. In a regular expression, the symbol “*” matches the preceding character 0 or more times, whereas “+” matches the preceding character 1 or more times. “.” matches any single character. Writing “x|y” means to match either ‘x’ or ‘y’.\n",
    "\n",
    "For more regex shortcuts (cheatsheet): https://www.shortcutfoo.com/app/dojos/regex/cheatsheet\n",
    "\n",
    "To play \"regex golf,\" check out this [tutorial by Peter Norvig](https://www.oreilly.com/learning/regex-golf-with-peter-norvig) (you may need an O'Reilly or social media account to play).\n",
    "\n",
    "Pandas allows you to use `regex` in its `replace()` function -- when a regex term is found in an element, the element is then replaced with the specified replacement term. In order for it to appropriately correct elements, both regex and inplace variables need to be set to `True` (as their defaults are False). This ensures that the initial input string is read as a regular expression and that the elements will be modified in place.\n",
    "\n",
    "For more documentation on the replace method: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.replace.html\n",
    "\n",
    "Here's an incorrect regex example: we create a temporary `DataFrame` in which a regex pulls all values that contain the term “congo”. Unfortunately, this creates 24 instances of the Democratic Republic of the Congo -- this is an error in our cleaning! We can revert back to the non-temporary `DataFrame` and correctly modify our regex to isolate only the Democratic Republic instances (as opposed to including the Republic as well)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "africa_dem rep congo             24\n",
       "europe_netherlands               14\n",
       "americas_dominican republic      14\n",
       "africa_burundi                   13\n",
       "americas_bolivia                 13\n",
       "asia_afghanistan                 13\n",
       "asia_bangladesh                  13\n",
       "africa_guinea                    12\n",
       "asia_nepal                       12\n",
       "asia_philippines                 12\n",
       "africa_sao tome and principe     12\n",
       "asia_jordan                      12\n",
       "americas_nicaragua               12\n",
       "asia_cambodia                    12\n",
       "asia_singapore                   12\n",
       "africa_mozambique                12\n",
       "asia_china                       12\n",
       "africa_guinea-bissau             12\n",
       "africa_gambia                    12\n",
       "africa_eritrea                   12\n",
       "africa_tanzania                  12\n",
       "asia_mongolia                    12\n",
       "europe_bosnia and herzegovina    12\n",
       "africa_benin                     12\n",
       "americas_panama                  12\n",
       "americas_ecuador                 12\n",
       "asia_korea, rep.                 12\n",
       "americas_argentina               12\n",
       "europe_montenegro                12\n",
       "africa_zambia                    12\n",
       "                                 ..\n",
       "europe_belgium                   12\n",
       "africa_somalia                   12\n",
       "asia_saudi arabia                12\n",
       "africa_djibouti                  12\n",
       "africa_niger                     12\n",
       "asia_myanmar                     12\n",
       "africa_swaziland                 12\n",
       "europe_austria                   12\n",
       "americas_venezuela               12\n",
       "americas_brazil                  12\n",
       "europe_france                    12\n",
       "americas_peru                    12\n",
       "americas_trinidad and tobago     12\n",
       "americas_puerto rico             12\n",
       "africa_egypt                     12\n",
       "europe_poland                    12\n",
       "asia_lebanon                     12\n",
       "europe_albania                   12\n",
       "asia_syria                       12\n",
       "asia_west bank and gaza          12\n",
       "europe_czech republic            12\n",
       "europe_italy                     12\n",
       "asia_india                       12\n",
       "europe_switzerland               12\n",
       "asia_iran                        12\n",
       "africa_libya                     12\n",
       "africa_cote d'ivoire             11\n",
       "americas_canada                   9\n",
       "_canada                           3\n",
       "africa_cote d'ivore               1\n",
       "Name: region, Length: 143, dtype: int64"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# This gives a problem -- 24 values of the congo!\n",
    "temp = gapminder_copy['region'].replace(\".*congo.*\", \"africa_dem rep congo\", regex=True)\n",
    "temp.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>pop</th>\n",
       "      <th>life Exp</th>\n",
       "      <th>gdpPercap</th>\n",
       "      <th>region</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>328</th>\n",
       "      <td>1952</td>\n",
       "      <td>14100005</td>\n",
       "      <td>39.143</td>\n",
       "      <td>780.542326</td>\n",
       "      <td>africa_congo, dem. rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>329</th>\n",
       "      <td>1957</td>\n",
       "      <td>15577932</td>\n",
       "      <td>40.652</td>\n",
       "      <td>905.860230</td>\n",
       "      <td>africa_congo, dem. rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>330</th>\n",
       "      <td>1962</td>\n",
       "      <td>17486434</td>\n",
       "      <td>42.122</td>\n",
       "      <td>896.314634</td>\n",
       "      <td>africa_congo, dem. rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>331</th>\n",
       "      <td>1967</td>\n",
       "      <td>19941073</td>\n",
       "      <td>44.056</td>\n",
       "      <td>861.593242</td>\n",
       "      <td>africa_congo, dem. rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>332</th>\n",
       "      <td>1972</td>\n",
       "      <td>23007669</td>\n",
       "      <td>45.989</td>\n",
       "      <td>904.896069</td>\n",
       "      <td>africa_democratic republic of the congo</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>333</th>\n",
       "      <td>1977</td>\n",
       "      <td>26480870</td>\n",
       "      <td>47.804</td>\n",
       "      <td>795.757282</td>\n",
       "      <td>africa_congo, dem. rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>334</th>\n",
       "      <td>1982</td>\n",
       "      <td>30646495</td>\n",
       "      <td>47.784</td>\n",
       "      <td>673.747818</td>\n",
       "      <td>africa_congo, dem. rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>335</th>\n",
       "      <td>1987</td>\n",
       "      <td>35481645</td>\n",
       "      <td>47.412</td>\n",
       "      <td>672.774812</td>\n",
       "      <td>africa_congo, dem. rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>336</th>\n",
       "      <td>1992</td>\n",
       "      <td>41672143</td>\n",
       "      <td>45.548</td>\n",
       "      <td>457.719181</td>\n",
       "      <td>africa_congo, dem. rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>337</th>\n",
       "      <td>1997</td>\n",
       "      <td>47798986</td>\n",
       "      <td>42.587</td>\n",
       "      <td>312.188423</td>\n",
       "      <td>africa_congo, democratic republic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>338</th>\n",
       "      <td>2002</td>\n",
       "      <td>55379852</td>\n",
       "      <td>44.966</td>\n",
       "      <td>241.165877</td>\n",
       "      <td>africa_congo, dem. rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>339</th>\n",
       "      <td>2007</td>\n",
       "      <td>64606759</td>\n",
       "      <td>46.462</td>\n",
       "      <td>277.551859</td>\n",
       "      <td>africa_congo, dem. rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>340</th>\n",
       "      <td>1952</td>\n",
       "      <td>854885</td>\n",
       "      <td>42.111</td>\n",
       "      <td>2125.621418</td>\n",
       "      <td>africa_congo, rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>341</th>\n",
       "      <td>1957</td>\n",
       "      <td>940458</td>\n",
       "      <td>45.053</td>\n",
       "      <td>2315.056572</td>\n",
       "      <td>africa_congo, rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>342</th>\n",
       "      <td>1962</td>\n",
       "      <td>1047924</td>\n",
       "      <td>48.435</td>\n",
       "      <td>2464.783157</td>\n",
       "      <td>africa_congo, rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>343</th>\n",
       "      <td>1967</td>\n",
       "      <td>1179760</td>\n",
       "      <td>52.040</td>\n",
       "      <td>2677.939642</td>\n",
       "      <td>africa_congo, rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>344</th>\n",
       "      <td>1972</td>\n",
       "      <td>1340458</td>\n",
       "      <td>54.907</td>\n",
       "      <td>3213.152683</td>\n",
       "      <td>africa_congo, rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>345</th>\n",
       "      <td>1977</td>\n",
       "      <td>1536769</td>\n",
       "      <td>55.625</td>\n",
       "      <td>3259.178978</td>\n",
       "      <td>africa_congo, rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>346</th>\n",
       "      <td>1982</td>\n",
       "      <td>1774735</td>\n",
       "      <td>56.695</td>\n",
       "      <td>4879.507522</td>\n",
       "      <td>africa_congo, rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>347</th>\n",
       "      <td>1987</td>\n",
       "      <td>2064095</td>\n",
       "      <td>57.470</td>\n",
       "      <td>4201.194937</td>\n",
       "      <td>africa_congo, rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>348</th>\n",
       "      <td>1992</td>\n",
       "      <td>2409073</td>\n",
       "      <td>56.433</td>\n",
       "      <td>4016.239529</td>\n",
       "      <td>africa_congo, rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>349</th>\n",
       "      <td>1997</td>\n",
       "      <td>2800947</td>\n",
       "      <td>52.962</td>\n",
       "      <td>3484.164376</td>\n",
       "      <td>africa_congo, rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>350</th>\n",
       "      <td>2002</td>\n",
       "      <td>3328795</td>\n",
       "      <td>52.970</td>\n",
       "      <td>3484.061970</td>\n",
       "      <td>africa_congo, rep.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>351</th>\n",
       "      <td>2007</td>\n",
       "      <td>3800610</td>\n",
       "      <td>55.322</td>\n",
       "      <td>3632.557798</td>\n",
       "      <td>africa_congo, rep.</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     year       pop  life Exp    gdpPercap  \\\n",
       "328  1952  14100005    39.143   780.542326   \n",
       "329  1957  15577932    40.652   905.860230   \n",
       "330  1962  17486434    42.122   896.314634   \n",
       "331  1967  19941073    44.056   861.593242   \n",
       "332  1972  23007669    45.989   904.896069   \n",
       "333  1977  26480870    47.804   795.757282   \n",
       "334  1982  30646495    47.784   673.747818   \n",
       "335  1987  35481645    47.412   672.774812   \n",
       "336  1992  41672143    45.548   457.719181   \n",
       "337  1997  47798986    42.587   312.188423   \n",
       "338  2002  55379852    44.966   241.165877   \n",
       "339  2007  64606759    46.462   277.551859   \n",
       "340  1952    854885    42.111  2125.621418   \n",
       "341  1957    940458    45.053  2315.056572   \n",
       "342  1962   1047924    48.435  2464.783157   \n",
       "343  1967   1179760    52.040  2677.939642   \n",
       "344  1972   1340458    54.907  3213.152683   \n",
       "345  1977   1536769    55.625  3259.178978   \n",
       "346  1982   1774735    56.695  4879.507522   \n",
       "347  1987   2064095    57.470  4201.194937   \n",
       "348  1992   2409073    56.433  4016.239529   \n",
       "349  1997   2800947    52.962  3484.164376   \n",
       "350  2002   3328795    52.970  3484.061970   \n",
       "351  2007   3800610    55.322  3632.557798   \n",
       "\n",
       "                                      region  \n",
       "328                  africa_congo, dem. rep.  \n",
       "329                  africa_congo, dem. rep.  \n",
       "330                  africa_congo, dem. rep.  \n",
       "331                  africa_congo, dem. rep.  \n",
       "332  africa_democratic republic of the congo  \n",
       "333                  africa_congo, dem. rep.  \n",
       "334                  africa_congo, dem. rep.  \n",
       "335                  africa_congo, dem. rep.  \n",
       "336                  africa_congo, dem. rep.  \n",
       "337        africa_congo, democratic republic  \n",
       "338                  africa_congo, dem. rep.  \n",
       "339                  africa_congo, dem. rep.  \n",
       "340                       africa_congo, rep.  \n",
       "341                       africa_congo, rep.  \n",
       "342                       africa_congo, rep.  \n",
       "343                       africa_congo, rep.  \n",
       "344                       africa_congo, rep.  \n",
       "345                       africa_congo, rep.  \n",
       "346                       africa_congo, rep.  \n",
       "347                       africa_congo, rep.  \n",
       "348                       africa_congo, rep.  \n",
       "349                       africa_congo, rep.  \n",
       "350                       africa_congo, rep.  \n",
       "351                       africa_congo, rep.  "
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# What happened? This shows all the rows that have congo in the name.\n",
    "gapminder_copy[gapminder_copy[\"region\"].str.contains('congo')]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using regex to correctly consolidate the Congo regions...\n",
    "\n",
    "As noted above, regular expressions (often simply \"regex\") provide a powerful \n",
    "tool for fixing errors that arise in strings. In order to correctly label the \n",
    "two different countries that include the word \"congo\", we need to design and\n",
    "use (via `pd.df.replace()`) a regex that correctly differentiates between the\n",
    "two countries.\n",
    "\n",
    "Recall that the \".\" is the wildcard (matching any single character); combining \n",
    "this with \"*\" allows us to match any number of single characters an unspecified \n",
    "number of times. By combining these characters with substrings corresponding to\n",
    "variations in the naming of the Democratic Republic of the Congo, we can\n",
    "correctly normalize the name.\n",
    "\n",
    "If you feel that the use of regex is not particularly straightforward, you are\n",
    "correct -- appropriately using these tools takes a great deal of time to master.\n",
    "When designing regex for these sorts of tasks, you might find the following\n",
    "prototyper helpful: https://regex101.com/  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "americas_dominican republic      14\n",
       "europe_netherlands               14\n",
       "africa_burundi                   13\n",
       "asia_bangladesh                  13\n",
       "americas_bolivia                 13\n",
       "asia_afghanistan                 13\n",
       "africa_congo, rep.               12\n",
       "asia_cambodia                    12\n",
       "africa_guinea-bissau             12\n",
       "asia_china                       12\n",
       "africa_mozambique                12\n",
       "asia_jordan                      12\n",
       "americas_nicaragua               12\n",
       "africa_gambia                    12\n",
       "asia_nepal                       12\n",
       "africa_guinea                    12\n",
       "asia_philippines                 12\n",
       "asia_singapore                   12\n",
       "africa_sao tome and principe     12\n",
       "africa_tanzania                  12\n",
       "africa_eritrea                   12\n",
       "americas_argentina               12\n",
       "asia_mongolia                    12\n",
       "europe_bosnia and herzegovina    12\n",
       "africa_benin                     12\n",
       "americas_panama                  12\n",
       "americas_ecuador                 12\n",
       "asia_korea, rep.                 12\n",
       "europe_montenegro                12\n",
       "americas_paraguay                12\n",
       "                                 ..\n",
       "africa_somalia                   12\n",
       "asia_saudi arabia                12\n",
       "africa_djibouti                  12\n",
       "africa_niger                     12\n",
       "asia_myanmar                     12\n",
       "africa_swaziland                 12\n",
       "europe_austria                   12\n",
       "americas_venezuela               12\n",
       "americas_brazil                  12\n",
       "europe_france                    12\n",
       "americas_peru                    12\n",
       "americas_trinidad and tobago     12\n",
       "asia_west bank and gaza          12\n",
       "europe_czech republic            12\n",
       "europe_romania                   12\n",
       "africa_egypt                     12\n",
       "europe_poland                    12\n",
       "asia_lebanon                     12\n",
       "europe_albania                   12\n",
       "asia_iran                        12\n",
       "asia_syria                       12\n",
       "americas_puerto rico             12\n",
       "europe_italy                     12\n",
       "asia_india                       12\n",
       "europe_switzerland               12\n",
       "africa_libya                     12\n",
       "africa_cote d'ivoire             11\n",
       "americas_canada                   9\n",
       "_canada                           3\n",
       "africa_cote d'ivore               1\n",
       "Name: region, Length: 144, dtype: int64"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_copy['region'].replace(\".*congo, dem.*\", \"africa_dem rep congo\", regex=True, inplace=True)\n",
    "gapminder_copy['region'].replace(\".*_democratic republic of the congo\", \"africa_dem rep congo\", regex=True, inplace=True)\n",
    "gapminder_copy['region'].value_counts() # Now it's fixed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise (regex):\n",
    "\n",
    "Now that we've taken a close look at how to properly design and use regex to\n",
    "clean string entries in our data, let's try to normalize the naming of a few\n",
    "other countries. Using the pandas code we constructed above as a template,\n",
    "construct similar code (using `pd.df.replace()`) to set the naming of the Ivory\n",
    "Coast and Canada to \"africa_cote d'ivoire\" and \"americas_canada\", respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (<ipython-input-57-03da694de8f5>, line 1)",
     "output_type": "error",
     "traceback": [
      "\u001b[1;36m  File \u001b[1;32m\"<ipython-input-57-03da694de8f5>\"\u001b[1;36m, line \u001b[1;32m1\u001b[0m\n\u001b[1;33m    gapminder_copy['region'].replace(\".*ivore_cote d' ivore'\", regex=True inplace =True )\u001b[0m\n\u001b[1;37m                                                                                ^\u001b[0m\n\u001b[1;31mSyntaxError\u001b[0m\u001b[1;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    "gapminder_copy['region'].replace(\".*ivore_cote d' ivore'\", regex=True inplace =True )\n",
    "gapminder_copy['region'].replace(\"^_canada\",\"americas_canada\", regex=True inplace=True\n",
    "Gapminder_copy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tidy data\n",
    "\n",
    "Having what is called a \"_Tidy_ data set\" can make cleaning, analyzing, and visualizing your data much easier. You should aim for having Tidy data when cleaning and preparing your data set for analysis. Two of the important aspects of Tidy data are:\n",
    "* every variable has its own column\n",
    "* every observation has its own row\n",
    "\n",
    "(There are other aspects of Tidy data, here is a good blog post about Tidy data in Python: http://www.jeannicholashould.com/tidy-data-in-python.html)\n",
    "\n",
    "Currently the gapminder dataset has a single column for continent and country (the ‘region’ column). We can split that column into two, by using the underscore that separates continent from country.\n",
    "We can create a new column in the `DataFrame` by naming it before the = sign:\n",
    "`gapminder['country'] = `\n",
    "\n",
    "The following commands use the function `split()` to split the string at the underscore (the first argument), which results in a list of two elements: before and after the \\_. The second argument tells `split()` that the split should take place only at the first occurrence of the underscore."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>pop</th>\n",
       "      <th>life Exp</th>\n",
       "      <th>gdpPercap</th>\n",
       "      <th>region</th>\n",
       "      <th>country</th>\n",
       "      <th>continent</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1952</td>\n",
       "      <td>8425333</td>\n",
       "      <td>28.801</td>\n",
       "      <td>779.445314</td>\n",
       "      <td>asia_afghanistan</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>asia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1957</td>\n",
       "      <td>9240934</td>\n",
       "      <td>30.332</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>asia_afghanistan</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>asia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1957</td>\n",
       "      <td>9240934</td>\n",
       "      <td>30.332</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>asia_afghanistan</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>asia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1962</td>\n",
       "      <td>10267083</td>\n",
       "      <td>31.997</td>\n",
       "      <td>853.100710</td>\n",
       "      <td>asia_afghanistan</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>asia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1967</td>\n",
       "      <td>11537966</td>\n",
       "      <td>34.020</td>\n",
       "      <td>836.197138</td>\n",
       "      <td>asia_afghanistan</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>asia</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   year       pop  life Exp   gdpPercap            region      country  \\\n",
       "0  1952   8425333    28.801  779.445314  asia_afghanistan  afghanistan   \n",
       "1  1957   9240934    30.332  820.853030  asia_afghanistan  afghanistan   \n",
       "2  1957   9240934    30.332  820.853030  asia_afghanistan  afghanistan   \n",
       "3  1962  10267083    31.997  853.100710  asia_afghanistan  afghanistan   \n",
       "4  1967  11537966    34.020  836.197138  asia_afghanistan  afghanistan   \n",
       "\n",
       "  continent  \n",
       "0      asia  \n",
       "1      asia  \n",
       "2      asia  \n",
       "3      asia  \n",
       "4      asia  "
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_copy['country']=gapminder_copy['region'].str.split('_', 1).str[1]\n",
    "gapminder_copy['continent']=gapminder_copy['region'].str.split('_', 1).str[0]\n",
    "gapminder_copy.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Removing and renaming columns\n",
    "\n",
    "We have now added the columns `country` and `continent`, but we still have the old `region` column as well. In order to remove that column we use the `drop()` command. The first argument of the `drop()` command is the name of the element to be dropped. The second argument is the *axis* number:  \n",
    "*0 for row, 1 for column*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "ename": "KeyError",
     "evalue": "\"['region'] not found in axis\"",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-62-6f74eeed4c53>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mgapminder_copy\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mgapminder_copy\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdrop\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'region'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      2\u001b[0m \u001b[0mgapminder_copy\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mhead\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\pandas\\core\\frame.py\u001b[0m in \u001b[0;36mdrop\u001b[1;34m(self, labels, axis, index, columns, level, inplace, errors)\u001b[0m\n\u001b[0;32m   3938\u001b[0m                                            \u001b[0mindex\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mindex\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcolumns\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mcolumns\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   3939\u001b[0m                                            \u001b[0mlevel\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mlevel\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0minplace\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0minplace\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 3940\u001b[1;33m                                            errors=errors)\n\u001b[0m\u001b[0;32m   3941\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   3942\u001b[0m     @rewrite_axis_style_signature('mapper', [('copy', True),\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\pandas\\core\\generic.py\u001b[0m in \u001b[0;36mdrop\u001b[1;34m(self, labels, axis, index, columns, level, inplace, errors)\u001b[0m\n\u001b[0;32m   3778\u001b[0m         \u001b[1;32mfor\u001b[0m \u001b[0maxis\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlabels\u001b[0m \u001b[1;32min\u001b[0m \u001b[0maxes\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mitems\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   3779\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mlabels\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mnot\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 3780\u001b[1;33m                 \u001b[0mobj\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mobj\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_drop_axis\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlabels\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0maxis\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlevel\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mlevel\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0merrors\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0merrors\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   3781\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   3782\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0minplace\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\pandas\\core\\generic.py\u001b[0m in \u001b[0;36m_drop_axis\u001b[1;34m(self, labels, axis, level, errors)\u001b[0m\n\u001b[0;32m   3810\u001b[0m                 \u001b[0mnew_axis\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0maxis\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdrop\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlabels\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mlevel\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mlevel\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0merrors\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0merrors\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   3811\u001b[0m             \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 3812\u001b[1;33m                 \u001b[0mnew_axis\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0maxis\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdrop\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlabels\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0merrors\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0merrors\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   3813\u001b[0m             \u001b[0mresult\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mreindex\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m**\u001b[0m\u001b[1;33m{\u001b[0m\u001b[0maxis_name\u001b[0m\u001b[1;33m:\u001b[0m \u001b[0mnew_axis\u001b[0m\u001b[1;33m}\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   3814\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\pandas\\core\\indexes\\base.py\u001b[0m in \u001b[0;36mdrop\u001b[1;34m(self, labels, errors)\u001b[0m\n\u001b[0;32m   4963\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0merrors\u001b[0m \u001b[1;33m!=\u001b[0m \u001b[1;34m'ignore'\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   4964\u001b[0m                 raise KeyError(\n\u001b[1;32m-> 4965\u001b[1;33m                     '{} not found in axis'.format(labels[mask]))\n\u001b[0m\u001b[0;32m   4966\u001b[0m             \u001b[0mindexer\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mindexer\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;33m~\u001b[0m\u001b[0mmask\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   4967\u001b[0m         \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mdelete\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mindexer\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mKeyError\u001b[0m: \"['region'] not found in axis\""
     ]
    }
   ],
   "source": [
    "gapminder_copy = gapminder_copy.drop('region', 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, it is a good idea to look critically at your column names. Use lowercase for all column names to avoid confusing `gdppercap` with `gdpPercap` or `GDPpercap`. Avoid spaces in column names to simplify manipulating your data - look out for lingering white space at the beginning or end of your column names. The following code turns all column names to lowercase. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>pop</th>\n",
       "      <th>life exp</th>\n",
       "      <th>gdppercap</th>\n",
       "      <th>country</th>\n",
       "      <th>continent</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1952</td>\n",
       "      <td>8425333</td>\n",
       "      <td>28.801</td>\n",
       "      <td>779.445314</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>asia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1957</td>\n",
       "      <td>9240934</td>\n",
       "      <td>30.332</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>asia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1957</td>\n",
       "      <td>9240934</td>\n",
       "      <td>30.332</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>asia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1962</td>\n",
       "      <td>10267083</td>\n",
       "      <td>31.997</td>\n",
       "      <td>853.100710</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>asia</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1967</td>\n",
       "      <td>11537966</td>\n",
       "      <td>34.020</td>\n",
       "      <td>836.197138</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>asia</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   year       pop  life exp   gdppercap      country continent\n",
       "0  1952   8425333    28.801  779.445314  afghanistan      asia\n",
       "1  1957   9240934    30.332  820.853030  afghanistan      asia\n",
       "2  1957   9240934    30.332  820.853030  afghanistan      asia\n",
       "3  1962  10267083    31.997  853.100710  afghanistan      asia\n",
       "4  1967  11537966    34.020  836.197138  afghanistan      asia"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_copy.columns = gapminder_copy.columns.str.lower()\n",
    "gapminder_copy.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also want to remove the space from the `life exp` column name. We can do that with Pandas `rename` method. It takes a dictionary as its argument, with the old column names as keys and new column names as values.\n",
    "\n",
    "If you're unfamiliar with dictionaries, they are a very useful data structure in Python.  You can read more about them [here](https://docs.python.org/3/tutorial/datastructures.html#dictionaries)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>pop</th>\n",
       "      <th>life Exp</th>\n",
       "      <th>gdpPercap</th>\n",
       "      <th>region</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1952.0</td>\n",
       "      <td>8425333.0</td>\n",
       "      <td>28.801</td>\n",
       "      <td>779.445314</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1957.0</td>\n",
       "      <td>9240934.0</td>\n",
       "      <td>30.332</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1957.0</td>\n",
       "      <td>9240934.0</td>\n",
       "      <td>30.332</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1962.0</td>\n",
       "      <td>10267083.0</td>\n",
       "      <td>31.997</td>\n",
       "      <td>853.100710</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1962.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>31.997</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Asia_Afghanistan</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     year         pop  life Exp   gdpPercap            region\n",
       "0  1952.0   8425333.0    28.801  779.445314  Asia_Afghanistan\n",
       "1  1957.0   9240934.0    30.332  820.853030  Asia_Afghanistan\n",
       "2  1957.0   9240934.0    30.332  820.853030  Asia_Afghanistan\n",
       "3  1962.0  10267083.0    31.997  853.100710  Asia_Afghanistan\n",
       "4  1962.0         NaN    31.997         NaN  Asia_Afghanistan"
      ]
     },
     "execution_count": 134,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_copy = gapminder_copy.rename(columns = {'life exp':'lifeexp'})\n",
    "gapminder_copy.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Merging data\n",
    "\n",
    "Often we have more than one `DataFrame` that contains parts of our data set and we want to put them together. This is known as merging the data.\n",
    "\n",
    "Our advisor now wants us to add a new country called The People's Republic of Berkeley to the gapminder data set that we have cleaned up. Our goal is to get this new data into the same `DataFrame` in the same format as the gapminder data and, in this case, we want to concatenate (add) it onto the end of the gapminder data.\n",
    "\n",
    "Concatentating is a simple form of merging, there are many useful (and more complicated) ways to merge data.  If you are interested in more information, the [Pandas Documentation](http://pandas.pydata.org/pandas-docs/stable/merging.html) is useful."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>year</th>\n",
       "      <th>pop</th>\n",
       "      <th>lifeExp</th>\n",
       "      <th>gdpPercap</th>\n",
       "      <th>region</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1952</td>\n",
       "      <td>113805</td>\n",
       "      <td>66.75</td>\n",
       "      <td>779.6780</td>\n",
       "      <td>Americas_Peoples Republic of Berkeley</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1957</td>\n",
       "      <td>114077</td>\n",
       "      <td>67.96</td>\n",
       "      <td>821.2960</td>\n",
       "      <td>Americas_Peoples Republic of Berkeley</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1962</td>\n",
       "      <td>111268</td>\n",
       "      <td>71.11</td>\n",
       "      <td>857.2354</td>\n",
       "      <td>Americas_Peoples Republic of Berkeley</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1967</td>\n",
       "      <td>114785</td>\n",
       "      <td>72.03</td>\n",
       "      <td>838.1382</td>\n",
       "      <td>Americas_Peoples Republic of Berkeley</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1972</td>\n",
       "      <td>116716</td>\n",
       "      <td>72.67</td>\n",
       "      <td>737.1058</td>\n",
       "      <td>Americas_Peoples Republic of Berkeley</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   year     pop  lifeExp  gdpPercap                                 region\n",
       "0  1952  113805    66.75   779.6780  Americas_Peoples Republic of Berkeley\n",
       "1  1957  114077    67.96   821.2960  Americas_Peoples Republic of Berkeley\n",
       "2  1962  111268    71.11   857.2354  Americas_Peoples Republic of Berkeley\n",
       "3  1967  114785    72.03   838.1382  Americas_Peoples Republic of Berkeley\n",
       "4  1972  116716    72.67   737.1058  Americas_Peoples Republic of Berkeley"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "PRB = pd.read_csv(\"PRB_data.txt\", sep = \"\\t\")\n",
    "PRB.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "ename": "KeyError",
     "evalue": "'region'",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\pandas\\core\\indexes\\base.py\u001b[0m in \u001b[0;36mget_loc\u001b[1;34m(self, key, method, tolerance)\u001b[0m\n\u001b[0;32m   2656\u001b[0m             \u001b[1;32mtry\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 2657\u001b[1;33m                 \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_engine\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mget_loc\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   2658\u001b[0m             \u001b[1;32mexcept\u001b[0m \u001b[0mKeyError\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mpandas/_libs/index.pyx\u001b[0m in \u001b[0;36mpandas._libs.index.IndexEngine.get_loc\u001b[1;34m()\u001b[0m\n",
      "\u001b[1;32mpandas/_libs/index.pyx\u001b[0m in \u001b[0;36mpandas._libs.index.IndexEngine.get_loc\u001b[1;34m()\u001b[0m\n",
      "\u001b[1;32mpandas/_libs/hashtable_class_helper.pxi\u001b[0m in \u001b[0;36mpandas._libs.hashtable.PyObjectHashTable.get_item\u001b[1;34m()\u001b[0m\n",
      "\u001b[1;32mpandas/_libs/hashtable_class_helper.pxi\u001b[0m in \u001b[0;36mpandas._libs.hashtable.PyObjectHashTable.get_item\u001b[1;34m()\u001b[0m\n",
      "\u001b[1;31mKeyError\u001b[0m: 'region'",
      "\nDuring handling of the above exception, another exception occurred:\n",
      "\u001b[1;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-73-5400096f8da3>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      8\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      9\u001b[0m \u001b[1;31m#gapminder_copy['country']=gapminder_copy['region'].str.split('_', 1).str[1]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m---> 10\u001b[1;33m \u001b[0mgapminder_copy\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'continent'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mgapminder_copy\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'region'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstr\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msplit\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'_'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mstr\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;36m0\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m     11\u001b[0m \u001b[0mgapminder_copy\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mhead\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\pandas\\core\\frame.py\u001b[0m in \u001b[0;36m__getitem__\u001b[1;34m(self, key)\u001b[0m\n\u001b[0;32m   2925\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mcolumns\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mnlevels\u001b[0m \u001b[1;33m>\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   2926\u001b[0m                 \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_getitem_multilevel\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 2927\u001b[1;33m             \u001b[0mindexer\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mcolumns\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mget_loc\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   2928\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mis_integer\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mindexer\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   2929\u001b[0m                 \u001b[0mindexer\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;33m[\u001b[0m\u001b[0mindexer\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\pandas\\core\\indexes\\base.py\u001b[0m in \u001b[0;36mget_loc\u001b[1;34m(self, key, method, tolerance)\u001b[0m\n\u001b[0;32m   2657\u001b[0m                 \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_engine\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mget_loc\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   2658\u001b[0m             \u001b[1;32mexcept\u001b[0m \u001b[0mKeyError\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 2659\u001b[1;33m                 \u001b[1;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_engine\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mget_loc\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_maybe_cast_indexer\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mkey\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   2660\u001b[0m         \u001b[0mindexer\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mget_indexer\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mkey\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mmethod\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mmethod\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mtolerance\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mtolerance\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   2661\u001b[0m         \u001b[1;32mif\u001b[0m \u001b[0mindexer\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mndim\u001b[0m \u001b[1;33m>\u001b[0m \u001b[1;36m1\u001b[0m \u001b[1;32mor\u001b[0m \u001b[0mindexer\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msize\u001b[0m \u001b[1;33m>\u001b[0m \u001b[1;36m1\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32mpandas/_libs/index.pyx\u001b[0m in \u001b[0;36mpandas._libs.index.IndexEngine.get_loc\u001b[1;34m()\u001b[0m\n",
      "\u001b[1;32mpandas/_libs/index.pyx\u001b[0m in \u001b[0;36mpandas._libs.index.IndexEngine.get_loc\u001b[1;34m()\u001b[0m\n",
      "\u001b[1;32mpandas/_libs/hashtable_class_helper.pxi\u001b[0m in \u001b[0;36mpandas._libs.hashtable.PyObjectHashTable.get_item\u001b[1;34m()\u001b[0m\n",
      "\u001b[1;32mpandas/_libs/hashtable_class_helper.pxi\u001b[0m in \u001b[0;36mpandas._libs.hashtable.PyObjectHashTable.get_item\u001b[1;34m()\u001b[0m\n",
      "\u001b[1;31mKeyError\u001b[0m: 'region'"
     ]
    }
   ],
   "source": [
    "## bring in PRB data (no major problems) and make it conform to the gapminder at this point\n",
    "# clean the data to look like the current gapminder\n",
    "PRB['country'] = PRB ['region'].str.split('_', 1).str[1].str.lower()\n",
    "PRB['continent'] = PRB ['region'].str.split('_', 1).str[0].str.lower()\n",
    "PRB = PRB.drop ('region', 1)\n",
    "PRB.columns = PRB.columns.str.lower()\n",
    "PRB.head()\n",
    "\n",
    "#gapminder_copy['country']=gapminder_copy['region'].str.split('_', 1).str[1]\n",
    "gapminder_copy['continent']=gapminder_copy['region'].str.split('_', 1).str[0]\n",
    "gapminder_copy.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\telmo\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:1: FutureWarning: Sorting because non-concatenation axis is not aligned. A future version\n",
      "of pandas will change to not sort by default.\n",
      "\n",
      "To accept the future behavior, pass 'sort=False'.\n",
      "\n",
      "To retain the current behavior and silence the warning, pass 'sort=True'.\n",
      "\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>continent</th>\n",
       "      <th>country</th>\n",
       "      <th>gdpPercap</th>\n",
       "      <th>gdppercap</th>\n",
       "      <th>life exp</th>\n",
       "      <th>lifeExp</th>\n",
       "      <th>pop</th>\n",
       "      <th>region</th>\n",
       "      <th>year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>asia</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>NaN</td>\n",
       "      <td>779.445314</td>\n",
       "      <td>28.801</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8425333</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>asia</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>NaN</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>30.332</td>\n",
       "      <td>NaN</td>\n",
       "      <td>9240934</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>asia</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>NaN</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>30.332</td>\n",
       "      <td>NaN</td>\n",
       "      <td>9240934</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>asia</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>NaN</td>\n",
       "      <td>853.100710</td>\n",
       "      <td>31.997</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10267083</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1962</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>asia</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>NaN</td>\n",
       "      <td>836.197138</td>\n",
       "      <td>34.020</td>\n",
       "      <td>NaN</td>\n",
       "      <td>11537966</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1967</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  continent      country  gdpPercap   gdppercap  life exp  lifeExp       pop  \\\n",
       "0      asia  afghanistan        NaN  779.445314    28.801      NaN   8425333   \n",
       "1      asia  afghanistan        NaN  820.853030    30.332      NaN   9240934   \n",
       "2      asia  afghanistan        NaN  820.853030    30.332      NaN   9240934   \n",
       "3      asia  afghanistan        NaN  853.100710    31.997      NaN  10267083   \n",
       "4      asia  afghanistan        NaN  836.197138    34.020      NaN  11537966   \n",
       "\n",
       "  region  year  \n",
       "0    NaN  1952  \n",
       "1    NaN  1957  \n",
       "2    NaN  1957  \n",
       "3    NaN  1962  \n",
       "4    NaN  1967  "
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_comb = pd.concat([gapminder_copy,PRB])\n",
    "gapminder_comb.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# combine the data sets with concat\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that the `DataFrames` have been concatenated, notice that the index is funky. It repeats the numbers 0 - 11 in the `peoples republic of berkeley data`. <p>\n",
    "#### **Exercise:** fix the index.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# our code for fixing index\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Subsetting and sorting\n",
    "\n",
    "There are many ways in which you can manipulate a Pandas `DataFrame` - here we will discuss two approaches: subsetting and sorting.\n",
    "\n",
    "##### Subsetting\n",
    "We can subset (or slice) by giving the numbers of the rows you want to see between square brackets.\n",
    "\n",
    "*REMINDER:* Python uses 0-based indexing. This means that the first element in an object is located at position 0. this is different from other tools like R and Matlab that index elements within objects starting at 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>continent</th>\n",
       "      <th>country</th>\n",
       "      <th>gdpPercap</th>\n",
       "      <th>gdppercap</th>\n",
       "      <th>life exp</th>\n",
       "      <th>lifeExp</th>\n",
       "      <th>pop</th>\n",
       "      <th>region</th>\n",
       "      <th>year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>asia</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>NaN</td>\n",
       "      <td>779.445314</td>\n",
       "      <td>28.801</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8425333</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>asia</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>NaN</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>30.332</td>\n",
       "      <td>NaN</td>\n",
       "      <td>9240934</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>asia</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>NaN</td>\n",
       "      <td>820.853030</td>\n",
       "      <td>30.332</td>\n",
       "      <td>NaN</td>\n",
       "      <td>9240934</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>asia</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>NaN</td>\n",
       "      <td>853.100710</td>\n",
       "      <td>31.997</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10267083</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1962</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>asia</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>NaN</td>\n",
       "      <td>836.197138</td>\n",
       "      <td>34.020</td>\n",
       "      <td>NaN</td>\n",
       "      <td>11537966</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1967</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>asia</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>NaN</td>\n",
       "      <td>739.981106</td>\n",
       "      <td>36.088</td>\n",
       "      <td>NaN</td>\n",
       "      <td>13079460</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>asia</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>NaN</td>\n",
       "      <td>786.113360</td>\n",
       "      <td>38.438</td>\n",
       "      <td>NaN</td>\n",
       "      <td>14880372</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1977</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>asia</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>NaN</td>\n",
       "      <td>978.011439</td>\n",
       "      <td>39.854</td>\n",
       "      <td>NaN</td>\n",
       "      <td>12881816</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1982</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>asia</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>NaN</td>\n",
       "      <td>852.395945</td>\n",
       "      <td>40.822</td>\n",
       "      <td>NaN</td>\n",
       "      <td>13867957</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1987</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>asia</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>NaN</td>\n",
       "      <td>649.341395</td>\n",
       "      <td>41.674</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16317921</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1992</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>asia</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>NaN</td>\n",
       "      <td>635.341351</td>\n",
       "      <td>41.763</td>\n",
       "      <td>NaN</td>\n",
       "      <td>22227415</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1997</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>asia</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>NaN</td>\n",
       "      <td>726.734055</td>\n",
       "      <td>42.129</td>\n",
       "      <td>NaN</td>\n",
       "      <td>25268405</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>asia</td>\n",
       "      <td>afghanistan</td>\n",
       "      <td>NaN</td>\n",
       "      <td>974.580338</td>\n",
       "      <td>43.828</td>\n",
       "      <td>NaN</td>\n",
       "      <td>31889923</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>europe</td>\n",
       "      <td>albania</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1601.056136</td>\n",
       "      <td>55.230</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1282697</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>europe</td>\n",
       "      <td>albania</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1942.284244</td>\n",
       "      <td>59.280</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1476505</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1957</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   continent      country  gdpPercap    gdppercap  life exp  lifeExp  \\\n",
       "0       asia  afghanistan        NaN   779.445314    28.801      NaN   \n",
       "1       asia  afghanistan        NaN   820.853030    30.332      NaN   \n",
       "2       asia  afghanistan        NaN   820.853030    30.332      NaN   \n",
       "3       asia  afghanistan        NaN   853.100710    31.997      NaN   \n",
       "4       asia  afghanistan        NaN   836.197138    34.020      NaN   \n",
       "5       asia  afghanistan        NaN   739.981106    36.088      NaN   \n",
       "6       asia  afghanistan        NaN   786.113360    38.438      NaN   \n",
       "7       asia  afghanistan        NaN   978.011439    39.854      NaN   \n",
       "8       asia  afghanistan        NaN   852.395945    40.822      NaN   \n",
       "9       asia  afghanistan        NaN   649.341395    41.674      NaN   \n",
       "10      asia  afghanistan        NaN   635.341351    41.763      NaN   \n",
       "11      asia  afghanistan        NaN   726.734055    42.129      NaN   \n",
       "12      asia  afghanistan        NaN   974.580338    43.828      NaN   \n",
       "13    europe      albania        NaN  1601.056136    55.230      NaN   \n",
       "14    europe      albania        NaN  1942.284244    59.280      NaN   \n",
       "\n",
       "         pop region  year  \n",
       "0    8425333    NaN  1952  \n",
       "1    9240934    NaN  1957  \n",
       "2    9240934    NaN  1957  \n",
       "3   10267083    NaN  1962  \n",
       "4   11537966    NaN  1967  \n",
       "5   13079460    NaN  1972  \n",
       "6   14880372    NaN  1977  \n",
       "7   12881816    NaN  1982  \n",
       "8   13867957    NaN  1987  \n",
       "9   16317921    NaN  1992  \n",
       "10  22227415    NaN  1997  \n",
       "11  25268405    NaN  2002  \n",
       "12  31889923    NaN  2007  \n",
       "13   1282697    NaN  1952  \n",
       "14   1476505    NaN  1957  "
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_comb[0:15]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use a different way to select the first 15 rows\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Select the last 10 rows\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise\n",
    "\n",
    "*What does the negative number (in the third cell) mean?*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Answer:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*What happens when you leave the space before or after the colon empty?*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Answer:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Subsetting can also be done by selecting for a particular column or for a particular value in a column; for instance select the rows that have ‘africa’ in the column ‘continent. Note the double equal sign: single equal signs are used in Python to assign something to a variable. The double equal sign is a comparison: the variable to the left has to be exactly equal to the string to the right.\n",
    "\n",
    "**There other ways of subsetting that are worth knowing about. Do an independent reading of using .loc/.iloc with `DataFrames`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>continent</th>\n",
       "      <th>country</th>\n",
       "      <th>gdpPercap</th>\n",
       "      <th>gdppercap</th>\n",
       "      <th>life exp</th>\n",
       "      <th>lifeExp</th>\n",
       "      <th>pop</th>\n",
       "      <th>region</th>\n",
       "      <th>year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>africa</td>\n",
       "      <td>algeria</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2449.008185</td>\n",
       "      <td>43.077</td>\n",
       "      <td>NaN</td>\n",
       "      <td>9279525</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>africa</td>\n",
       "      <td>algeria</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3013.976023</td>\n",
       "      <td>45.685</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10270856</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>africa</td>\n",
       "      <td>algeria</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2550.816880</td>\n",
       "      <td>48.303</td>\n",
       "      <td>NaN</td>\n",
       "      <td>11000948</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1962</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>africa</td>\n",
       "      <td>algeria</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3246.991771</td>\n",
       "      <td>51.407</td>\n",
       "      <td>NaN</td>\n",
       "      <td>12760499</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1967</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>africa</td>\n",
       "      <td>algeria</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4182.663766</td>\n",
       "      <td>54.518</td>\n",
       "      <td>NaN</td>\n",
       "      <td>14760787</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>africa</td>\n",
       "      <td>algeria</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4910.416756</td>\n",
       "      <td>58.014</td>\n",
       "      <td>NaN</td>\n",
       "      <td>17152804</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1977</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>africa</td>\n",
       "      <td>algeria</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5745.160213</td>\n",
       "      <td>61.368</td>\n",
       "      <td>NaN</td>\n",
       "      <td>20033753</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1982</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>africa</td>\n",
       "      <td>algeria</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5681.358539</td>\n",
       "      <td>65.799</td>\n",
       "      <td>NaN</td>\n",
       "      <td>23254956</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1987</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>africa</td>\n",
       "      <td>algeria</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5023.216647</td>\n",
       "      <td>67.744</td>\n",
       "      <td>NaN</td>\n",
       "      <td>26298373</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1992</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>africa</td>\n",
       "      <td>algeria</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4797.295051</td>\n",
       "      <td>69.152</td>\n",
       "      <td>NaN</td>\n",
       "      <td>29072015</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1997</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>africa</td>\n",
       "      <td>algeria</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5288.040382</td>\n",
       "      <td>70.994</td>\n",
       "      <td>NaN</td>\n",
       "      <td>31287142</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>africa</td>\n",
       "      <td>algeria</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6223.367465</td>\n",
       "      <td>72.301</td>\n",
       "      <td>NaN</td>\n",
       "      <td>33333216</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>africa</td>\n",
       "      <td>angola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3520.610273</td>\n",
       "      <td>30.015</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4232095</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>africa</td>\n",
       "      <td>angola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3827.940465</td>\n",
       "      <td>31.999</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4561361</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>africa</td>\n",
       "      <td>angola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4269.276742</td>\n",
       "      <td>34.000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4826015</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1962</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>africa</td>\n",
       "      <td>angola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5522.776375</td>\n",
       "      <td>35.985</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5247469</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1967</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>africa</td>\n",
       "      <td>angola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5473.288005</td>\n",
       "      <td>37.928</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5894858</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>africa</td>\n",
       "      <td>angola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3008.647355</td>\n",
       "      <td>39.483</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6162675</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1977</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>africa</td>\n",
       "      <td>angola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2756.953672</td>\n",
       "      <td>39.942</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7016384</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1982</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>africa</td>\n",
       "      <td>angola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2430.208311</td>\n",
       "      <td>39.906</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7874230</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1987</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>africa</td>\n",
       "      <td>angola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2627.845685</td>\n",
       "      <td>40.647</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8735988</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1992</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>africa</td>\n",
       "      <td>angola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2277.140884</td>\n",
       "      <td>40.963</td>\n",
       "      <td>NaN</td>\n",
       "      <td>9875024</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1997</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>africa</td>\n",
       "      <td>angola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2773.287312</td>\n",
       "      <td>41.003</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10866106</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>africa</td>\n",
       "      <td>angola</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4797.231267</td>\n",
       "      <td>42.731</td>\n",
       "      <td>NaN</td>\n",
       "      <td>12420476</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>122</th>\n",
       "      <td>africa</td>\n",
       "      <td>benin</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1062.752200</td>\n",
       "      <td>38.223</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1738315</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>123</th>\n",
       "      <td>africa</td>\n",
       "      <td>benin</td>\n",
       "      <td>NaN</td>\n",
       "      <td>959.601080</td>\n",
       "      <td>40.358</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1925173</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>124</th>\n",
       "      <td>africa</td>\n",
       "      <td>benin</td>\n",
       "      <td>NaN</td>\n",
       "      <td>949.499064</td>\n",
       "      <td>42.618</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2151895</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1962</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125</th>\n",
       "      <td>africa</td>\n",
       "      <td>benin</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1035.831411</td>\n",
       "      <td>44.885</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2427334</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1967</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>126</th>\n",
       "      <td>africa</td>\n",
       "      <td>benin</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1085.796879</td>\n",
       "      <td>47.014</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2761407</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>127</th>\n",
       "      <td>africa</td>\n",
       "      <td>benin</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1029.161251</td>\n",
       "      <td>49.190</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3168267</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1977</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1598</th>\n",
       "      <td>africa</td>\n",
       "      <td>uganda</td>\n",
       "      <td>NaN</td>\n",
       "      <td>682.266227</td>\n",
       "      <td>49.849</td>\n",
       "      <td>NaN</td>\n",
       "      <td>12939400</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1982</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1599</th>\n",
       "      <td>africa</td>\n",
       "      <td>uganda</td>\n",
       "      <td>NaN</td>\n",
       "      <td>617.724406</td>\n",
       "      <td>51.509</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15283050</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1987</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1600</th>\n",
       "      <td>africa</td>\n",
       "      <td>uganda</td>\n",
       "      <td>NaN</td>\n",
       "      <td>644.170797</td>\n",
       "      <td>48.825</td>\n",
       "      <td>NaN</td>\n",
       "      <td>18252190</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1992</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1601</th>\n",
       "      <td>africa</td>\n",
       "      <td>uganda</td>\n",
       "      <td>NaN</td>\n",
       "      <td>816.559081</td>\n",
       "      <td>44.578</td>\n",
       "      <td>NaN</td>\n",
       "      <td>21210254</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1997</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1602</th>\n",
       "      <td>africa</td>\n",
       "      <td>uganda</td>\n",
       "      <td>NaN</td>\n",
       "      <td>927.721002</td>\n",
       "      <td>47.813</td>\n",
       "      <td>NaN</td>\n",
       "      <td>24739869</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1603</th>\n",
       "      <td>africa</td>\n",
       "      <td>uganda</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1056.380121</td>\n",
       "      <td>51.542</td>\n",
       "      <td>NaN</td>\n",
       "      <td>29170398</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1688</th>\n",
       "      <td>africa</td>\n",
       "      <td>zambia</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1147.388831</td>\n",
       "      <td>42.038</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2672000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1689</th>\n",
       "      <td>africa</td>\n",
       "      <td>zambia</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1311.956766</td>\n",
       "      <td>44.077</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3016000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1690</th>\n",
       "      <td>africa</td>\n",
       "      <td>zambia</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1452.725766</td>\n",
       "      <td>46.023</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3421000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1962</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1691</th>\n",
       "      <td>africa</td>\n",
       "      <td>zambia</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1777.077318</td>\n",
       "      <td>47.768</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3900000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1967</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1692</th>\n",
       "      <td>africa</td>\n",
       "      <td>zambia</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1773.498265</td>\n",
       "      <td>50.107</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4506497</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1693</th>\n",
       "      <td>africa</td>\n",
       "      <td>zambia</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1588.688299</td>\n",
       "      <td>51.386</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5216550</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1977</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1694</th>\n",
       "      <td>africa</td>\n",
       "      <td>zambia</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1408.678565</td>\n",
       "      <td>51.821</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6100407</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1982</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1695</th>\n",
       "      <td>africa</td>\n",
       "      <td>zambia</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1213.315116</td>\n",
       "      <td>50.821</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7272406</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1987</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1696</th>\n",
       "      <td>africa</td>\n",
       "      <td>zambia</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1210.884633</td>\n",
       "      <td>46.100</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8381163</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1992</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1697</th>\n",
       "      <td>africa</td>\n",
       "      <td>zambia</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1071.353818</td>\n",
       "      <td>40.238</td>\n",
       "      <td>NaN</td>\n",
       "      <td>9417789</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1997</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1698</th>\n",
       "      <td>africa</td>\n",
       "      <td>zambia</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1071.613938</td>\n",
       "      <td>39.193</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10595811</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1699</th>\n",
       "      <td>africa</td>\n",
       "      <td>zambia</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1271.211593</td>\n",
       "      <td>42.384</td>\n",
       "      <td>NaN</td>\n",
       "      <td>11746035</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1700</th>\n",
       "      <td>africa</td>\n",
       "      <td>zimbabwe</td>\n",
       "      <td>NaN</td>\n",
       "      <td>406.884115</td>\n",
       "      <td>48.451</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3080907</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1701</th>\n",
       "      <td>africa</td>\n",
       "      <td>zimbabwe</td>\n",
       "      <td>NaN</td>\n",
       "      <td>518.764268</td>\n",
       "      <td>50.469</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3646340</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1702</th>\n",
       "      <td>africa</td>\n",
       "      <td>zimbabwe</td>\n",
       "      <td>NaN</td>\n",
       "      <td>527.272182</td>\n",
       "      <td>52.358</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4277736</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1962</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1703</th>\n",
       "      <td>africa</td>\n",
       "      <td>zimbabwe</td>\n",
       "      <td>NaN</td>\n",
       "      <td>569.795071</td>\n",
       "      <td>53.995</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4995432</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1967</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1704</th>\n",
       "      <td>africa</td>\n",
       "      <td>zimbabwe</td>\n",
       "      <td>NaN</td>\n",
       "      <td>799.362176</td>\n",
       "      <td>55.635</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5861135</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1972</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1705</th>\n",
       "      <td>africa</td>\n",
       "      <td>zimbabwe</td>\n",
       "      <td>NaN</td>\n",
       "      <td>685.587682</td>\n",
       "      <td>57.674</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6642107</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1977</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1706</th>\n",
       "      <td>africa</td>\n",
       "      <td>zimbabwe</td>\n",
       "      <td>NaN</td>\n",
       "      <td>788.855041</td>\n",
       "      <td>60.363</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7636524</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1982</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1707</th>\n",
       "      <td>africa</td>\n",
       "      <td>zimbabwe</td>\n",
       "      <td>NaN</td>\n",
       "      <td>706.157306</td>\n",
       "      <td>62.351</td>\n",
       "      <td>NaN</td>\n",
       "      <td>9216418</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1987</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1708</th>\n",
       "      <td>africa</td>\n",
       "      <td>zimbabwe</td>\n",
       "      <td>NaN</td>\n",
       "      <td>693.420786</td>\n",
       "      <td>60.377</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10704340</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1992</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1709</th>\n",
       "      <td>africa</td>\n",
       "      <td>zimbabwe</td>\n",
       "      <td>NaN</td>\n",
       "      <td>792.449960</td>\n",
       "      <td>46.809</td>\n",
       "      <td>NaN</td>\n",
       "      <td>11404948</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1997</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1710</th>\n",
       "      <td>africa</td>\n",
       "      <td>zimbabwe</td>\n",
       "      <td>NaN</td>\n",
       "      <td>672.038623</td>\n",
       "      <td>39.989</td>\n",
       "      <td>NaN</td>\n",
       "      <td>11926563</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1711</th>\n",
       "      <td>africa</td>\n",
       "      <td>zimbabwe</td>\n",
       "      <td>NaN</td>\n",
       "      <td>469.709298</td>\n",
       "      <td>43.487</td>\n",
       "      <td>NaN</td>\n",
       "      <td>12311143</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2007</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>625 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     continent   country  gdpPercap    gdppercap  life exp  lifeExp       pop  \\\n",
       "25      africa   algeria        NaN  2449.008185    43.077      NaN   9279525   \n",
       "26      africa   algeria        NaN  3013.976023    45.685      NaN  10270856   \n",
       "27      africa   algeria        NaN  2550.816880    48.303      NaN  11000948   \n",
       "28      africa   algeria        NaN  3246.991771    51.407      NaN  12760499   \n",
       "29      africa   algeria        NaN  4182.663766    54.518      NaN  14760787   \n",
       "30      africa   algeria        NaN  4910.416756    58.014      NaN  17152804   \n",
       "31      africa   algeria        NaN  5745.160213    61.368      NaN  20033753   \n",
       "32      africa   algeria        NaN  5681.358539    65.799      NaN  23254956   \n",
       "33      africa   algeria        NaN  5023.216647    67.744      NaN  26298373   \n",
       "34      africa   algeria        NaN  4797.295051    69.152      NaN  29072015   \n",
       "35      africa   algeria        NaN  5288.040382    70.994      NaN  31287142   \n",
       "36      africa   algeria        NaN  6223.367465    72.301      NaN  33333216   \n",
       "37      africa    angola        NaN  3520.610273    30.015      NaN   4232095   \n",
       "38      africa    angola        NaN  3827.940465    31.999      NaN   4561361   \n",
       "39      africa    angola        NaN  4269.276742    34.000      NaN   4826015   \n",
       "40      africa    angola        NaN  5522.776375    35.985      NaN   5247469   \n",
       "41      africa    angola        NaN  5473.288005    37.928      NaN   5894858   \n",
       "42      africa    angola        NaN  3008.647355    39.483      NaN   6162675   \n",
       "43      africa    angola        NaN  2756.953672    39.942      NaN   7016384   \n",
       "44      africa    angola        NaN  2430.208311    39.906      NaN   7874230   \n",
       "45      africa    angola        NaN  2627.845685    40.647      NaN   8735988   \n",
       "46      africa    angola        NaN  2277.140884    40.963      NaN   9875024   \n",
       "47      africa    angola        NaN  2773.287312    41.003      NaN  10866106   \n",
       "48      africa    angola        NaN  4797.231267    42.731      NaN  12420476   \n",
       "122     africa     benin        NaN  1062.752200    38.223      NaN   1738315   \n",
       "123     africa     benin        NaN   959.601080    40.358      NaN   1925173   \n",
       "124     africa     benin        NaN   949.499064    42.618      NaN   2151895   \n",
       "125     africa     benin        NaN  1035.831411    44.885      NaN   2427334   \n",
       "126     africa     benin        NaN  1085.796879    47.014      NaN   2761407   \n",
       "127     africa     benin        NaN  1029.161251    49.190      NaN   3168267   \n",
       "...        ...       ...        ...          ...       ...      ...       ...   \n",
       "1598    africa    uganda        NaN   682.266227    49.849      NaN  12939400   \n",
       "1599    africa    uganda        NaN   617.724406    51.509      NaN  15283050   \n",
       "1600    africa    uganda        NaN   644.170797    48.825      NaN  18252190   \n",
       "1601    africa    uganda        NaN   816.559081    44.578      NaN  21210254   \n",
       "1602    africa    uganda        NaN   927.721002    47.813      NaN  24739869   \n",
       "1603    africa    uganda        NaN  1056.380121    51.542      NaN  29170398   \n",
       "1688    africa    zambia        NaN  1147.388831    42.038      NaN   2672000   \n",
       "1689    africa    zambia        NaN  1311.956766    44.077      NaN   3016000   \n",
       "1690    africa    zambia        NaN  1452.725766    46.023      NaN   3421000   \n",
       "1691    africa    zambia        NaN  1777.077318    47.768      NaN   3900000   \n",
       "1692    africa    zambia        NaN  1773.498265    50.107      NaN   4506497   \n",
       "1693    africa    zambia        NaN  1588.688299    51.386      NaN   5216550   \n",
       "1694    africa    zambia        NaN  1408.678565    51.821      NaN   6100407   \n",
       "1695    africa    zambia        NaN  1213.315116    50.821      NaN   7272406   \n",
       "1696    africa    zambia        NaN  1210.884633    46.100      NaN   8381163   \n",
       "1697    africa    zambia        NaN  1071.353818    40.238      NaN   9417789   \n",
       "1698    africa    zambia        NaN  1071.613938    39.193      NaN  10595811   \n",
       "1699    africa    zambia        NaN  1271.211593    42.384      NaN  11746035   \n",
       "1700    africa  zimbabwe        NaN   406.884115    48.451      NaN   3080907   \n",
       "1701    africa  zimbabwe        NaN   518.764268    50.469      NaN   3646340   \n",
       "1702    africa  zimbabwe        NaN   527.272182    52.358      NaN   4277736   \n",
       "1703    africa  zimbabwe        NaN   569.795071    53.995      NaN   4995432   \n",
       "1704    africa  zimbabwe        NaN   799.362176    55.635      NaN   5861135   \n",
       "1705    africa  zimbabwe        NaN   685.587682    57.674      NaN   6642107   \n",
       "1706    africa  zimbabwe        NaN   788.855041    60.363      NaN   7636524   \n",
       "1707    africa  zimbabwe        NaN   706.157306    62.351      NaN   9216418   \n",
       "1708    africa  zimbabwe        NaN   693.420786    60.377      NaN  10704340   \n",
       "1709    africa  zimbabwe        NaN   792.449960    46.809      NaN  11404948   \n",
       "1710    africa  zimbabwe        NaN   672.038623    39.989      NaN  11926563   \n",
       "1711    africa  zimbabwe        NaN   469.709298    43.487      NaN  12311143   \n",
       "\n",
       "     region  year  \n",
       "25      NaN  1952  \n",
       "26      NaN  1957  \n",
       "27      NaN  1962  \n",
       "28      NaN  1967  \n",
       "29      NaN  1972  \n",
       "30      NaN  1977  \n",
       "31      NaN  1982  \n",
       "32      NaN  1987  \n",
       "33      NaN  1992  \n",
       "34      NaN  1997  \n",
       "35      NaN  2002  \n",
       "36      NaN  2007  \n",
       "37      NaN  1952  \n",
       "38      NaN  1957  \n",
       "39      NaN  1962  \n",
       "40      NaN  1967  \n",
       "41      NaN  1972  \n",
       "42      NaN  1977  \n",
       "43      NaN  1982  \n",
       "44      NaN  1987  \n",
       "45      NaN  1992  \n",
       "46      NaN  1997  \n",
       "47      NaN  2002  \n",
       "48      NaN  2007  \n",
       "122     NaN  1952  \n",
       "123     NaN  1957  \n",
       "124     NaN  1962  \n",
       "125     NaN  1967  \n",
       "126     NaN  1972  \n",
       "127     NaN  1977  \n",
       "...     ...   ...  \n",
       "1598    NaN  1982  \n",
       "1599    NaN  1987  \n",
       "1600    NaN  1992  \n",
       "1601    NaN  1997  \n",
       "1602    NaN  2002  \n",
       "1603    NaN  2007  \n",
       "1688    NaN  1952  \n",
       "1689    NaN  1957  \n",
       "1690    NaN  1962  \n",
       "1691    NaN  1967  \n",
       "1692    NaN  1972  \n",
       "1693    NaN  1977  \n",
       "1694    NaN  1982  \n",
       "1695    NaN  1987  \n",
       "1696    NaN  1992  \n",
       "1697    NaN  1997  \n",
       "1698    NaN  2002  \n",
       "1699    NaN  2007  \n",
       "1700    NaN  1952  \n",
       "1701    NaN  1957  \n",
       "1702    NaN  1962  \n",
       "1703    NaN  1967  \n",
       "1704    NaN  1972  \n",
       "1705    NaN  1977  \n",
       "1706    NaN  1982  \n",
       "1707    NaN  1987  \n",
       "1708    NaN  1992  \n",
       "1709    NaN  1997  \n",
       "1710    NaN  2002  \n",
       "1711    NaN  2007  \n",
       "\n",
       "[625 rows x 9 columns]"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_comb[gapminder_comb['continent'] == 'africa'] "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       1952\n",
       "1       1957\n",
       "2       1957\n",
       "3       1962\n",
       "4       1967\n",
       "5       1972\n",
       "6       1977\n",
       "7       1982\n",
       "8       1987\n",
       "9       1992\n",
       "10      1997\n",
       "11      2002\n",
       "12      2007\n",
       "13      1952\n",
       "14      1957\n",
       "15      1962\n",
       "16      1967\n",
       "17      1972\n",
       "18      1977\n",
       "19      1982\n",
       "20      1987\n",
       "21      1992\n",
       "22      1997\n",
       "23      2002\n",
       "24      2007\n",
       "25      1952\n",
       "26      1957\n",
       "27      1962\n",
       "28      1967\n",
       "29      1972\n",
       "        ... \n",
       "1694    1982\n",
       "1695    1987\n",
       "1696    1992\n",
       "1697    1997\n",
       "1698    2002\n",
       "1699    2007\n",
       "1700    1952\n",
       "1701    1957\n",
       "1702    1962\n",
       "1703    1967\n",
       "1704    1972\n",
       "1705    1977\n",
       "1706    1982\n",
       "1707    1987\n",
       "1708    1992\n",
       "1709    1997\n",
       "1710    2002\n",
       "1711    2007\n",
       "0       1952\n",
       "1       1957\n",
       "2       1962\n",
       "3       1967\n",
       "4       1972\n",
       "5       1977\n",
       "6       1982\n",
       "7       1987\n",
       "8       1992\n",
       "9       1997\n",
       "10      2002\n",
       "11      2007\n",
       "Name: year, Length: 1724, dtype: int64"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_comb['year']\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also create a new object that contains the data within the `continent` column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Sorting\n",
    "Sorting may help to further organize and inspect your data. The command `sort_values()` takes a number of arguments; the most important ones are `by` and `ascending.` The following command will sort your `DataFrame` by year, beginning with the most recent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>continent</th>\n",
       "      <th>country</th>\n",
       "      <th>gdpPercap</th>\n",
       "      <th>gdppercap</th>\n",
       "      <th>life exp</th>\n",
       "      <th>lifeExp</th>\n",
       "      <th>pop</th>\n",
       "      <th>region</th>\n",
       "      <th>year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>979.3384</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>80.03</td>\n",
       "      <td>112580</td>\n",
       "      <td>Americas_Peoples Republic of Berkeley</td>\n",
       "      <td>2007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>133</th>\n",
       "      <td>africa</td>\n",
       "      <td>benin</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1441.284873</td>\n",
       "      <td>56.728</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8078314</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>617</th>\n",
       "      <td>americas</td>\n",
       "      <td>guatemala</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5186.050003</td>\n",
       "      <td>70.259</td>\n",
       "      <td>NaN</td>\n",
       "      <td>12572928</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1339</th>\n",
       "      <td>africa</td>\n",
       "      <td>senegal</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1712.472136</td>\n",
       "      <td>63.062</td>\n",
       "      <td>NaN</td>\n",
       "      <td>12267493</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2007</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>605</th>\n",
       "      <td>europe</td>\n",
       "      <td>greece</td>\n",
       "      <td>NaN</td>\n",
       "      <td>27538.411880</td>\n",
       "      <td>79.483</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10706290</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2007</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     continent    country  gdpPercap     gdppercap  life exp  lifeExp  \\\n",
       "11         NaN        NaN   979.3384           NaN       NaN    80.03   \n",
       "133     africa      benin        NaN   1441.284873    56.728      NaN   \n",
       "617   americas  guatemala        NaN   5186.050003    70.259      NaN   \n",
       "1339    africa    senegal        NaN   1712.472136    63.062      NaN   \n",
       "605     europe     greece        NaN  27538.411880    79.483      NaN   \n",
       "\n",
       "           pop                                 region  year  \n",
       "11      112580  Americas_Peoples Republic of Berkeley  2007  \n",
       "133    8078314                                    NaN  2007  \n",
       "617   12572928                                    NaN  2007  \n",
       "1339  12267493                                    NaN  2007  \n",
       "605   10706290                                    NaN  2007  "
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_comb.sort_values(by='year', ascending=False).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise\n",
    "\n",
    "Organize your data set by country, from ‘Afganistan’ to ‘Zimbabwe’."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summarize and plot\n",
    "\n",
    "Summaries (but can’t *say* statistics…)\n",
    "* Sort data\n",
    "* Basic summaries\n",
    "\n",
    "\n",
    "Plots \n",
    "* of subsets \n",
    "* single variables\n",
    "* pairs of variables\n",
    "* Matplotlib syntax (w/ Seaborn for defaults (prettier, package also good for more analysis later...))\n",
    "\n",
    "Exploring is often iterative - summarize, plot, summarize, plot, etc. - sometimes it branches…\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Summarizing data\n",
    "\n",
    "Remember that the `info()` method gives a few useful pieces of information, including the shape of the `DataFrame`, the variable type of each column, and the amount of memory stored. We can see many of our changes (continent and country columns instead of region, higher number of rows, etc.) reflected in the output of the `info()` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 1724 entries, 0 to 11\n",
      "Data columns (total 9 columns):\n",
      "continent    1712 non-null object\n",
      "country      1712 non-null object\n",
      "gdpPercap    12 non-null float64\n",
      "gdppercap    1712 non-null float64\n",
      "life exp     1712 non-null float64\n",
      "lifeExp      12 non-null float64\n",
      "pop          1724 non-null int64\n",
      "region       12 non-null object\n",
      "year         1724 non-null int64\n",
      "dtypes: float64(4), int64(2), object(3)\n",
      "memory usage: 134.7+ KB\n"
     ]
    }
   ],
   "source": [
    "gapminder_comb.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also saw above that the `describe()` method will take the numeric columns and give a summary of their values. We have to remember that we changed the column names and this time it shouldn't have NaNs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pop</th>\n",
       "      <th>gdppercap</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1.724000e+03</td>\n",
       "      <td>1712.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>2.933800e+07</td>\n",
       "      <td>7211.397109</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.055847e+08</td>\n",
       "      <td>9852.111363</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>6.001100e+04</td>\n",
       "      <td>241.165877</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>2.748356e+06</td>\n",
       "      <td>1200.430507</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>6.985493e+06</td>\n",
       "      <td>3525.296011</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.919694e+07</td>\n",
       "      <td>9325.462346</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.318683e+09</td>\n",
       "      <td>113523.132900</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                pop      gdppercap\n",
       "count  1.724000e+03    1712.000000\n",
       "mean   2.933800e+07    7211.397109\n",
       "std    1.055847e+08    9852.111363\n",
       "min    6.001100e+04     241.165877\n",
       "25%    2.748356e+06    1200.430507\n",
       "50%    6.985493e+06    3525.296011\n",
       "75%    1.919694e+07    9325.462346\n",
       "max    1.318683e+09  113523.132900"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_comb[['pop', 'gdppercap']].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### More summaries\n",
    "\n",
    "What if we just want a single value, like the mean of the population? We can call mean on a single column this way:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "29337996.48897912"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_comb['pop'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What if we want to know the mean population by _continent_? Then we need to use the Pandas `groupby()` method and tell it which column we want to group by.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pop</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>continent</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <td>1.692720e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>africa</th>\n",
       "      <td>9.909432e+06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>americas</th>\n",
       "      <td>2.438132e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>asia</th>\n",
       "      <td>7.687689e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>europe</th>\n",
       "      <td>1.715144e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>oceania</th>\n",
       "      <td>8.874672e+06</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    pop\n",
       "continent              \n",
       "           1.692720e+07\n",
       "africa     9.909432e+06\n",
       "americas   2.438132e+07\n",
       "asia       7.687689e+07\n",
       "europe     1.715144e+07\n",
       "oceania    8.874672e+06"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_comb[['continent','pop']].groupby(by ='continent').mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What if we want to know the median population by continent?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pop</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>continent</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <td>17010154.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>africa</th>\n",
       "      <td>4580410.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>americas</th>\n",
       "      <td>5926420.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>asia</th>\n",
       "      <td>14530830.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>europe</th>\n",
       "      <td>8617353.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>oceania</th>\n",
       "      <td>6403491.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  pop\n",
       "continent            \n",
       "           17010154.0\n",
       "africa      4580410.0\n",
       "americas    5926420.0\n",
       "asia       14530830.5\n",
       "europe      8617353.0\n",
       "oceania     6403491.5"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_comb[['continent','pop']].groupby(by ='continent').median()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or the number of entries (rows) per continent?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pop</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>continent</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>oceania</th>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>americas</th>\n",
       "      <td>300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>europe</th>\n",
       "      <td>362</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>asia</th>\n",
       "      <td>398</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>africa</th>\n",
       "      <td>625</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           pop\n",
       "continent     \n",
       "             3\n",
       "oceania     24\n",
       "americas   300\n",
       "europe     362\n",
       "asia       398\n",
       "africa     625"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_comb[['continent','pop']].groupby(by ='continent').count().sort_values(by='pop')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sometimes we don't want a whole `DataFrame`. Here is another way to do this that produces a `Series` that tells us number of entries (rows) as opposed to a `DataFrame`. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "continent\n",
       "              3\n",
       "africa      625\n",
       "americas    300\n",
       "asia        398\n",
       "europe      362\n",
       "oceania      24\n",
       "dtype: int64"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gdpcap = gapminder_comb[['continent','country']].groupby(by = 'continent').size()\n",
    "gdpcap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also look at the mean GDP per capita of each country: \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>gdppercap</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>country</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>afghanistan</th>\n",
       "      <td>804.072939</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>albania</th>\n",
       "      <td>3255.366633</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>algeria</th>\n",
       "      <td>4426.025973</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>angola</th>\n",
       "      <td>3607.100529</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>argentina</th>\n",
       "      <td>8955.553783</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>australia</th>\n",
       "      <td>19980.595634</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>austria</th>\n",
       "      <td>20411.916279</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bahrain</th>\n",
       "      <td>18077.663945</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bangladesh</th>\n",
       "      <td>805.429465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>belgium</th>\n",
       "      <td>19900.758072</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>benin</th>\n",
       "      <td>1155.395107</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>bolivia</th>\n",
       "      <td>2976.250423</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                gdppercap\n",
       "country                  \n",
       "afghanistan    804.072939\n",
       "albania       3255.366633\n",
       "algeria       4426.025973\n",
       "angola        3607.100529\n",
       "argentina     8955.553783\n",
       "australia    19980.595634\n",
       "austria      20411.916279\n",
       "bahrain      18077.663945\n",
       "bangladesh     805.429465\n",
       "belgium      19900.758072\n",
       "benin         1155.395107\n",
       "bolivia       2976.250423"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gapminder_comb[['country','gdppercap']].groupby(by='country').mean().head(12)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What if we wanted a new `DataFrame` that just contained these summaries? This could be a table in a report, for example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>nrows</th>\n",
       "      <th>meanpop</th>\n",
       "      <th>medianpop</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>continent</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <td>3</td>\n",
       "      <td>1.692720e+07</td>\n",
       "      <td>17010154.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>africa</th>\n",
       "      <td>625</td>\n",
       "      <td>9.909432e+06</td>\n",
       "      <td>4580410.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>americas</th>\n",
       "      <td>300</td>\n",
       "      <td>2.438132e+07</td>\n",
       "      <td>5926420.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>asia</th>\n",
       "      <td>398</td>\n",
       "      <td>7.687689e+07</td>\n",
       "      <td>14530830.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>europe</th>\n",
       "      <td>362</td>\n",
       "      <td>1.715144e+07</td>\n",
       "      <td>8617353.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>oceania</th>\n",
       "      <td>24</td>\n",
       "      <td>8.874672e+06</td>\n",
       "      <td>6403491.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           nrows       meanpop   medianpop\n",
       "continent                                 \n",
       "               3  1.692720e+07  17010154.0\n",
       "africa       625  9.909432e+06   4580410.0\n",
       "americas     300  2.438132e+07   5926420.0\n",
       "asia         398  7.687689e+07  14530830.5\n",
       "europe       362  1.715144e+07   8617353.0\n",
       "oceania       24  8.874672e+06   6403491.5"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "continent_mean_pop = gapminder_comb[['continent', 'pop']].groupby(by='continent').mean()\n",
    "continent_mean_pop = continent_mean_pop.rename(columns = {'pop':'meanpop'})\n",
    "continent_row_ct = gapminder_comb[['continent', 'country']].groupby(by='continent').count()\n",
    "continent_row_ct = continent_row_ct.rename(columns = {'country':'nrows'})\n",
    "continent_median_pop = gapminder_comb[['continent', 'pop']].groupby(by='continent').median()\n",
    "continent_median_pop = continent_median_pop.rename(columns = {'pop':'medianpop'})\n",
    "gapminder_summs = pd.concat([continent_row_ct,continent_mean_pop,continent_median_pop], axis=1)\n",
    "gapminder_summs = gapminder_summs.rename(columns = {'y':'year'})\n",
    "gapminder_summs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualization with `matplotlib`\n",
    "\n",
    "Recall that [matplotlib](http://matplotlib.org) is Python's main visualization \n",
    "library. It provides a range of tools for constructing plots and numerous \n",
    "high-level plotting libraries (e.g., [Seaborn](http://seaborn.pydata.org)) are \n",
    "built with matplotlib in mind. When we were in the early stages of setting up \n",
    "our analysis, we loaded these libraries like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "import seaborn as sns\n",
    "sns.set()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Consider the above three commands to be essential practice for plotting (as\n",
    "essential as **`import`** `pandas` **`as`** `pd` is for data munging).*\n",
    "\n",
    "Now, let's turn to data visualization. In order to get a feel for the properties\n",
    "of the data set we are working with, data visualization is key. While, we will\n",
    "focus only on the essentials of how to properly construct plots in univariate\n",
    "and bivariate settings here, it's worth noting that both matplotlib and Seaborn\n",
    "support a diversity of plots: [matplotlib \n",
    "gallery](http://matplotlib.org/gallery.html), [Seaborn\n",
    "gallery](http://seaborn.pydata.org/examples/). \n",
    "\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Single variables\n",
    "\n",
    "* __Histograms__ - provide a quick way of visualizing the distribution of numerical\n",
    "  data, or the frequencies of observations for categorical variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\telmo\\Anaconda3\\lib\\site-packages\\numpy\\lib\\histograms.py:824: RuntimeWarning: invalid value encountered in greater_equal\n",
      "  keep = (tmp_a >= first_edge)\n",
      "C:\\Users\\telmo\\Anaconda3\\lib\\site-packages\\numpy\\lib\\histograms.py:825: RuntimeWarning: invalid value encountered in less_equal\n",
      "  keep &= (tmp_a <= last_edge)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'count')"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEJCAYAAABohnsfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3dfXAU9eHH8fflgSgNDoJ3gCnDTOtDKlhiJ5biwylWScgDyGFVEgSrGIn1AbWRh6RJoTJBRbGdTnSYwYeiVk+qCWA8UBEqxGrIKA4afCQoAY8LREkCgeRuf3+g9yOCySZwuxf5vGaYZL+3e98Pubn53O3e7jkMwzAQERExIcbuACIi0nuoNERExDSVhoiImKbSEBER01QaIiJimkpDRERMU2mIiIhpcXYHiLTGxhZCoZ6dijJwYCJ79jSf4ESRp9zWUm5r9dbc0Duyx8Q4OP30n/3o7T/50giFjB6Xxvfb90bKbS3ltlZvzQ29Ozto95SIiHSDSkNERExTaYiIiGkqDRERMU2lISIipqk0RETENJWGiIiY9pM/T+N4HGoL4nT2s3ze1oPtNO07YPm8IiJdUWl0ok98LNn3Vlg+78qHJ9Bk+awiIl3T7ikRETFNpSEiIqZFvDSam5vJyspix44dALz33ntce+21ZGZmcs8993Do0CEAamtr8Xg8pKWlUVhYSHt7OwA7d+4kNzeX9PR08vPzaWlpiXRkERH5EREtjc2bNzN58mTq6uqAwwVyxx13MH/+fF555RUAli9fDkBBQQHFxcWsXr0awzDwer0AzJs3j5ycHHw+HyNGjKCsrCySkUVEpBMRLQ2v10tJSQkulwuAjRs3kpKSQnJyMgBFRUVcddVV1NfX09raSkpKCgAejwefz0dbWxvV1dWkpaV1GBcREXtE9NNTCxYs6LC8fft2+vbty913380XX3zBb37zG2bPns1HH32E0+kMr+d0OvH7/TQ2NpKYmEhcXFyHcRERsYelH7kNBoNs2LCBF154gTPPPJPCwkKWLFnCRRddhMPhCK9nGAYOhyP880g/XO7KwIGJJyS71Y73/BA7zi85EZTbWsptvd6cHSwujTPOOIORI0cydOhQAMaNG8czzzyDx+MhEAiE12toaMDlcjFgwACampoIBoPExsYSCATCu7rM2rOnucdfemLngxsI9PxMDaez33FtbxfltpZyW683ZI+JcXT6YtvSj9xecsklfPjhh+zatQuAN998k+HDh5OUlERCQgI1NTUAVFRU4Ha7iY+PJzU1lcrKSgDKy8txu91WRhYRkSNY+k5jyJAhzJ8/nxkzZnDw4EF+9atfMWvWLAAWLVpEUVERzc3NDB8+nKlTpwJQUlLC7NmzeeyxxxgyZAiPPPKIlZFFROQIlpTG2rVrw79ffvnlXH755Uetk5ycHP747ZGSkpJYtmxZJOOJiIhJOiNcRERMU2mIiIhpKg0RETFNpSEiIqapNERExDSVhoiImKbSEBER01QaIiJimkpDRERMU2mIiIhpKg0RETFNpSEiIqapNERExDSVhoiImKbSEBER01QaIiJiWkRLo7m5maysLHbs2NFh/JlnnuGGG24IL+/cuZPc3FzS09PJz8+npaUFgH379pGXl8e4cePIzc3t8D3iIiJivYiVxubNm5k8eTJ1dXUdxj/77DOWLFnSYWzevHnk5OTg8/kYMWIEZWVlADz66KOkpqby6quv8oc//IEFCxZEKq6IiJgQsdLwer2UlJTgcrnCY4cOHaK4uJg777wzPNbW1kZ1dTVpaWkAeDwefD4fAOvWrSM7OxuArKws/vvf/9LW1hapyCIi0oWIfUf4sd4VPPzww0yaNImf//zn4bHGxkYSExOJizscxel04vf7Adi9ezdOp/Nw0Lg4EhMT2bt3L4MGDYpUbBER6UTESuOHNm7cyK5du5gzZw7vvPNOeNwwDBwOR4d1f7h85LoxMd17czRwYGL3w0YBp7OfrdvbRbmtpdzW683ZwcLSWLVqFZ9++ikTJkxg//79NDQ0MHPmTB566CGampoIBoPExsYSCATCu7RcLhcNDQ0MHjyY9vZ2Wlpa6N+/f7fm3bOnmVDI6FFmOx/cQKCpx9s6nf2Oa3u7KLe1lNt6vSF7TIyj0xfbln3ktrS0lFdffZWKigruv/9+RowYwaOPPkp8fDypqalUVlYCUF5ejtvtBuCyyy6jvLwcgMrKSlJTU4mPj7cqsoiI/EBUnKdRUlKC1+slIyODTZs2MXPmTADuuusu3n//fTIzM3nuuecoLi62OamIyMkt4run1q5de9TYqFGjGDVqVHg5KSmJZcuWHbVe//79efzxxyOaT0REzIuKdxoiItI7qDRERMQ0lYaIiJim0hAREdNUGiIiYppKQ0RETFNpiIiIaSoNERExTaUhIiKmqTRERMQ0lYaIiJim0hAREdNUGiIiYppKQ0RETFNpiIiIaSoNERExLeKl0dzcTFZWFjt27ADghRdeICsri+zsbObMmcOhQ4cAqK2txePxkJaWRmFhIe3t7QDs3LmT3Nxc0tPTyc/Pp6WlJdKRRUTkR0S0NDZv3szkyZOpq6sDYNu2bSxdupTnn3+eFStWEAqFeO655wAoKCiguLiY1atXYxgGXq8XgHnz5pGTk4PP52PEiBGUlZVFMrKIiHQioqXh9XopKSnB5XIB0KdPH0pKSkhMTMThcHDOOeewc+dO6uvraW1tJSUlBQCPx4PP56OtrY3q6mrS0tI6jIuIiD0i+h3hCxYs6LCclJREUlISAHv37uXZZ5+ltLSU3bt343Q6w+s5nU78fj+NjY0kJiYSFxfXYVxEROwR0dL4MX6/n+nTpzNp0iRGjRpFTU0NDocjfLthGDgcjvDPI/1wuSsDByaekMxWczr72bq9XZTbWsptvd6cHWwojc8//5zp06dzww03cNNNNwEwePBgAoFAeJ2GhgZcLhcDBgygqamJYDBIbGwsgUAgvKvLrD17mgmFjB5ltfPBDQSaeryt09nvuLa3i3JbS7mt1xuyx8Q4On2xbelHbpubm7n55pu56667woUBh3dbJSQkUFNTA0BFRQVut5v4+HhSU1OprKwEoLy8HLfbbWVkERE5gqWlsXz5choaGnjyySeZMGECEyZM4O9//zsAixYtorS0lPT0dPbv38/UqVMBKCkpwev1kpGRwaZNm5g5c6aVkUVE5AgOwzB6tu+mlzje3VPZ91ac4ERdW/nwBO2e6kWU21q9NTf0juxRtXtKRER6N5WGiIiYptIQERHTVBoiImKaSkNERExTaYiIiGkqDRERMU2lISIipqk0RETENJWGiIiYptIQERHTVBoiImKaSkNERExTaYiIiGkqDRERMU2lISIipkW8NJqbm8nKymLHjh0AVFVVkZ2dzdixY1m8eHF4vdraWjweD2lpaRQWFtLe3g7Azp07yc3NJT09nfz8fFpaWiIdWUREfkRES2Pz5s1MnjyZuro6AFpbW5k7dy5lZWVUVlayZcsW1q9fD0BBQQHFxcWsXr0awzDwer0AzJs3j5ycHHw+HyNGjKCsrCySkUVEpBMRLQ2v10tJSQkulwuADz74gGHDhjF06FDi4uLIzs7G5/NRX19Pa2srKSkpAHg8Hnw+H21tbVRXV5OWltZhXERE7BEXyTtfsGBBh+Xdu3fjdDrDyy6XC7/ff9S40+nE7/fT2NhIYmIicXFxHcZFRMQeES2NHwqFQjgcjvCyYRg4HI4fHf/+55F+uNyVzr4gPZo5nf1s3d4uym0t5bZeb84OFpfG4MGDCQQC4eVAIIDL5TpqvKGhAZfLxYABA2hqaiIYDBIbGxtevzv27GkmFDJ6lNfOBzcQaOrxtk5nv+Pa3i7KbS3ltl5vyB4T4+j0xbalH7kdOXIk27ZtY/v27QSDQVatWoXb7SYpKYmEhARqamoAqKiowO12Ex8fT2pqKpWVlQCUl5fjdrutjCwiIkcwVRrHOo7w2WefdXuyhIQEFi5cyB133EFGRga/+MUvSE9PB2DRokWUlpaSnp7O/v37mTp1KgAlJSV4vV4yMjLYtGkTM2fO7Pa8IiJyYnS6e+qbb74B4JZbbmHZsmUYxuHdPO3t7dx+++2mP8m0du3a8O+jR49mxYoVR62TnJzM8uXLjxpPSkpi2bJlpuYREZHI6rQ07r33XjZu3AjAqFGj/n+juLjwx2BFROTk0WlpLF26FIA5c+ZQWlpqSSAREYlepj49VVpaSn19Pd9++214FxXA8OHDIxZMRESij6nS+Mc//sHSpUsZOHBgeMzhcPDGG29ELJiIiEQfU6VRXl7OmjVrGDRoUKTziIhIFDP1kdshQ4aoMERExNw7jdGjR/Pggw/y+9//nlNOOSU8rmMaIiInF1Ol8dJLLwF0OC9DxzRERE4+pkrjyJPzRETk5GWqNJ588sljjv/xj388oWFERCS6mSqNTz75JPz7oUOHqK6uZvTo0RELJSIi0cn0yX1H8vv9FBYWRiSQiIhErx5dGn3QoEHU19ef6CwiIhLlun1MwzAMtmzZ0uHscBEROTl0+5gGHD7Z77777otIIBERiV7dOqZRX19Pe3s7w4YNi2goERGJTqaOaWzfvp3MzEyuvvpqPB4PV155JZ9//nmPJ62oqCAzM5PMzEweeOABAGpra/F4PKSlpVFYWEh7ezsAO3fuJDc3l/T0dPLz82lpaenxvCIicnxMlcb8+fOZPn061dXV1NTUkJ+fz7x583o04YEDB1iwYAHLli2joqKCTZs2UVVVRUFBAcXFxaxevRrDMPB6vQDMmzePnJwcfD4fI0aMoKysrEfziojI8TNVGnv27GHixInh5UmTJtHY2NijCYPBIKFQiAMHDtDe3k57eztxcXG0traSkpICgMfjwefz0dbWRnV1dfhbAr8fFxERe5g6phEMBvnmm2/o378/AHv37u3xhImJidx1112MGzeOU089lQsvvJD4+HicTmd4HafTid/vp7GxkcTEROLi4jqMi4iIPUyVxpQpU7juuusYN24cDoeDyspKpk2b1qMJt27dyn/+8x/efPNN+vXrx5///Gc2btyIw+EIr2MYBg6HI/zzSD9c7srAgYk9ymk3p7OfrdvbRbmtpdzW683ZwWRpXHbZZTzxxBO0tbXx1Vdf4ff7ueqqq3o04YYNGxg9enT4PA+Px8PSpUsJBALhdRoaGnC5XAwYMICmpiaCwSCxsbEEAgFcLle35tuzp5lQyOh6xWOw88ENBJp6vK3T2e+4treLcltLua3XG7LHxDg6fbFt6pjG7Nmzyc3NpaCggIceeoiZM2cyd+7cHgVKTk6mqqqK/fv3YxgGa9eu5be//S0JCQnU1NQAhz9d5Xa7iY+PJzU1lcrKSuDwNwi63e4ezSsiIsfP1DuNxsZGpk6dCkBCQgI33ngj5eXlPZrwkksu4aOPPsLj8RAfH8/5559PXl4eV111FUVFRTQ3NzN8+PDwfCUlJcyePZvHHnuMIUOG8Mgjj/RoXhEROX6mD4T7/f7wV742NDRgGD3b5QOQl5dHXl5eh7Hk5GSWL19+1LpJSUksW7asx3OJiMiJY6o0brzxRq6++mouvfRSHA4HVVVVuoyIiMhJyFRpXHPNNYwYMYL//e9/xMbGcvPNN3POOedEOpuIiEQZU6UBh3cfJScnRzKLiIhEuR59n4aIiJycVBoiImKaSkNERExTaYiIiGkqDRERMU2lISIipqk0RETENJWGiIiYptIQERHTVBoiImKaSkNERExTaYiIiGkqDRERMc2W0li7di0ej4dx48Zx//33A1BVVUV2djZjx45l8eLF4XVra2vxeDykpaVRWFhIe3u7HZFFRAQbSuOrr76ipKSEsrIyVqxYwUcffcT69euZO3cuZWVlVFZWsmXLFtavXw9AQUEBxcXFrF69GsMw8Hq9VkcWEZHvWF4ar732GhkZGQwePJj4+HgWL17MqaeeyrBhwxg6dChxcXFkZ2fj8/mor6+ntbWVlJQUADweDz6fz+rIIiLyHdNfwnSibN++nfj4eGbMmMGuXbu4/PLLOfvss3E6neF1XC4Xfr+f3bt3dxh3Op34/f5uzTdwYOIJy24lp7OfrdvbRbmtpdzW683ZwYbSCAaDbNq0iWXLltG3b1/y8/M55ZRTcDgc4XUMw8DhcBAKhY453h179jQTChk9ymrngxsINPV4W6ez33FtbxfltpZyW683ZI+JcXT6Ytvy0jjjjDMYPXo0AwYMAODKK6/E5/MRGxsbXicQCOByuRg8eDCBQCA83tDQgMvlsjqyiIh8x/JjGmPGjGHDhg3s27ePYDDIW2+9RXp6Otu2bWP79u0Eg0FWrVqF2+0mKSmJhIQEampqAKioqMDtdlsdWUREvmP5O42RI0cyffp0cnJyaGtr4+KLL2by5Mn84he/4I477uDgwYNcdtllpKenA7Bo0SKKiopobm5m+PDhTJ061erIIiLyHctLA+Caa67hmmuu6TA2evRoVqxYcdS6ycnJLF++3KpoIiLSCZ0RLiIipqk0RETENJWGiIiYptIQERHTVBoiImKaSkNERExTaYiIiGkqDRERMU2lISIipqk0RETENJWGiIiYptIQERHTVBoiImKaSkNERExTaYiIiGkqDRERMc3W0njggQeYPXs2ALW1tXg8HtLS0igsLKS9vR2AnTt3kpubS3p6Ovn5+bS0tNgZWUTkpGZbabz99tu8/PLL4eWCggKKi4tZvXo1hmHg9XoBmDdvHjk5Ofh8PkaMGEFZWZldkUVETnq2lMY333zD4sWLmTFjBgD19fW0traSkpICgMfjwefz0dbWRnV1NWlpaR3GRUTEHrZ8R3hxcTF33303u3btAmD37t04nc7w7U6nE7/fT2NjI4mJicTFxXUY746BAxNPXHALOZ39bN3eLsptLeW2Xm/ODjaUxosvvsiQIUMYPXo0L730EgChUAiHwxFexzAMHA5H+OeRfrjclT17mgmFjB5ltfPBDQSaeryt09nvuLa3i3JbS7mt1xuyx8Q4On2xbXlpVFZWEggEmDBhAt9++y379+/H4XAQCATC6zQ0NOByuRgwYABNTU0Eg0FiY2MJBAK4XC6rI4uIyHcsP6bx5JNPsmrVKioqKrjzzju54oorKC0tJSEhgZqaGgAqKipwu93Ex8eTmppKZWUlAOXl5bjdbqsji4jId6LmPI1FixZRWlpKeno6+/fvZ+rUqQCUlJTg9XrJyMhg06ZNzJw50+akIiInL1sOhH/P4/Hg8XgASE5OZvny5Uetk5SUxLJly6yOJiIixxA17zRERCT6qTRERMQ0lYaIiJim0hAREdNUGiIiYppKQ0RETFNpiIiIaSoNERExTaUhIiKmqTRERMQ0lYaIiJim0hAREdNUGiIiYppKQ0RETFNpiIiIaSoNERExzZbS+Oc//0lmZiaZmZk8+OCDAFRVVZGdnc3YsWNZvHhxeN3a2lo8Hg9paWkUFhbS3t5uR2QREcGG0qiqqmLDhg28/PLLlJeX8+GHH7Jq1Srmzp1LWVkZlZWVbNmyhfXr1wNQUFBAcXExq1evxjAMvF6v1ZFFROQ7lpeG0+lk9uzZ9OnTh/j4eH75y19SV1fHsGHDGDp0KHFxcWRnZ+Pz+aivr6e1tZWUlBTg8NfD+nw+qyOLiMh3LP+O8LPPPjv8e11dHa+++ipTpkzB6XSGx10uF36/n927d3cYdzqd+P3+bs03cGDi8Ye2gdPZz9bt7aLc1lJu6/Xm7GBDaXzv008/5dZbb+W+++4jNjaWurq68G2GYeBwOAiFQjgcjqPGu2PPnmZCIaNHGe18cAOBph5v63T2O67t7aLc1lJu6/WG7DExjk5fbNtyILympoYbb7yRe++9l4kTJzJ48GACgUD49kAggMvlOmq8oaEBl8tlR2QREcGG0ti1axd/+tOfWLRoEZmZmQCMHDmSbdu2sX37doLBIKtWrcLtdpOUlERCQgI1NTUAVFRU4Ha7rY4sIiLfsXz31NKlSzl48CALFy4Mj11//fUsXLiQO+64g4MHD3LZZZeRnp4OwKJFiygqKqK5uZnhw4czdepUqyOLiMh3LC+NoqIiioqKjnnbihUrjhpLTk5m+fLlkY4lIiIm6IxwERExTaUhIiKmqTRERMQ0lYaIiJim0hAREdNUGiIiYppKQ0RETFNpiIiIaSoNERExTaUhIiKmqTRERMQ0lYaIiJim0hAREdNUGiIiYppKQ0RETFNpiIiIaZZ/CVNPrFy5kscee4z29namTZtGbm6u3ZEi6lBbEKez33HdR0+2bz3YTtO+A8c1r4j8tEV9afj9fhYvXsxLL71Enz59uP766xk1ahRnnXWW3dEipk98LNn3Vlg+78qHJ9Bk+awi0ptEfWlUVVXxu9/9jv79+wOQlpaGz+fj9ttvN7V9TIzjuOZ3nX7qcW3f2+Y93r9Xb5+/p5TbWr01N0R/9q7yRX1p7N69G6fTGV52uVx88MEHprc//fSfHdf8S4vGHtf2vW3egQMTbZk3WubvKeW2Vm/NDb07O/SCA+GhUAiH4/+bzzCMDssiImKdqC+NwYMHEwgEwsuBQACXy2VjIhGRk1fUl8ZFF13E22+/zd69ezlw4ABr1qzB7XbbHUtE5KQU9cc0Bg0axN13383UqVNpa2vjmmuu4de//rXdsURETkoOwzAMu0OIiEjvEPW7p0REJHqoNERExDSVhoiImKbSEBER00760li5ciUZGRmMHTuWZ5999qjba2tr8Xg8pKWlUVhYSHt7uw0pj62r7K+//joTJkxg/Pjx3HbbbXz77bc2pDxaV7m/t27dOq644goLk3Wuq9xffPEFN9xwA+PHj+fmm2/uNX/vDz/8kEmTJjF+/HhuvfVW9u3bZ0PKY2tubiYrK4sdO3YcdVs0Pzc7yx2tz0vTjJPY119/bYwZM8ZobGw0WlpajOzsbOPTTz/tsE5mZqbx3nvvGYZhGHPmzDGeffZZO6IepavsTU1NxsUXX2x8/fXXhmEYxqOPPmr87W9/sytumJm/uWEYRiAQMNLT040xY8bYkPJoXeUOhULG2LFjjfXr1xuGYRgPPfSQ8eCDD9oVN8zM33vy5MnGunXrDMMwjNLSUuORRx6xI+pR3n//fSMrK8sYPny48dVXXx11e7Q+NzvLHa3Py+44qd9pHHkxxL59+4Yvhvi9+vp6WltbSUlJAcDj8XS43U5dZW9ra6OkpIRBgwYBcO6557Jr1y674oZ1lft7RUVFpi9KaYWucn/44Yf07ds3fOLpjBkzouIS/mb+3qFQiJaWFgAOHDjAKaecYkfUo3i9XkpKSo55BYhofm52ljtan5fdcVKXxrEuhuj3+3/0dqfT2eF2O3WV/fTTT+eqq64CoLW1lSVLlnDllVdanvOHusoN8K9//YvzzjuPkSNHWh3vR3WV+8svv+SMM85g7ty5TJw4kZKSEvr27WtH1A7M/L1nz55NUVERl1xyCVVVVVx//fVWxzymBQsWkJqaeszbovm52VnuaH1edsdJXRpdXQwxmi+WaDZbU1MTeXl5JCcnM3HiRCsjHlNXuT/55BPWrFnDbbfdZke8H9VV7vb2dt59910mT57Myy+/zNChQ1m4cKEdUTvoKndrayuFhYU89dRTbNiwgZycHGbNmmVH1G6J5uemGdH2vOyOk7o0uroY4g9vb2hoiJqLJZq5kOPu3bvJycnh3HPPZcGCBVZHPKaucvt8PgKBAJMmTSIvLy/8f7BbV7mdTifDhg3j/PPPByArK6tbl/CPlK5yf/LJJyQkJIQvzXPdddfx7rvvWp6zu6L5udmVaHxedsdJXRpdXQwxKSmJhIQEampqAKioqIiaiyV2lT0YDDJjxgzGjRtHYWFh1LwK6yr3nXfeyerVq6moqGDJkiW4XC6ee+45GxMf1lXuCy64gL1797J161YA1q5dy/Dhw+2KG9ZV7mHDhvH111/zxRdfAPDGG2+Eiy+aRfNzszPR+rzsFhsPwkeFFStWGJmZmcbYsWONJUuWGIZhGNOnTzc++OADwzAMo7a21pg0aZKRlpZm3HPPPcbBgwftjNtBZ9nXrFljnHvuucb48ePD/+bOnWtz4sO6+pt/76uvvoqaT08ZRte533//fWPSpElGRkaGcdNNNxkNDQ12xg3rKve6deuM7OxsIysry5g2bZrx5Zdf2hn3KGPGjAl/Cqm3PDcN49i5o/l5aZYuWCgiIqad1LunRESke1QaIiJimkpDRERMU2mIiIhpKg0RkZ+Yzi6YeKT169eTnZ1NdnY29957b/hyMp1RaYiI/IRs3ryZyZMnU1dX1+l6+/btY/bs2SxevJiVK1eSnJzM4sWLu7x/lYaIyE/IsS6YWF5ezsSJE5kwYQJz587l4MGD1NXVceaZZ3LWWWcBMGbMGF5//fUu7z8uYslFTmLvvPMOixYt4swzz+SLL77glFNOYeHChbhcLubNm8fWrVtxOBxceuml3HPPPcTFxXHeeedxyy238NZbb7F//37uuecexo4da/d/RXqZH16a5NNPP8Xr9fL888+TkJDAww8/zNKlS5kyZQpff/01W7duJTk5mVdffZWGhoYu71+lIRIhW7ZsYdasWaSmpvLvf/+bgoICzj77bPr378/KlStpa2sjPz+fJ554gry8PILBIKeeeiovvfQSW7duZcqUKaSmpjJgwAC7/yvSi73zzjts376da6+9Fjh8efbzzjuP0047jQceeIC//OUvhEIhrr32WuLj47u8P5WGSIQkJyeHL5E9adIk5s+fz8cff8wrr7yCw+GgT58+XH/99Tz99NPk5eUBMGXKlPC255xzDtXV1aSlpdn2f5DeLxgMMm7cOIqKigBoaWkhGAwSDAYZPHgwL774IgAffPABQ4cO7fL+dExDJEJiY2OPGjOOcfn9I7+m9MhtQqHQMe9DpDtGjRrFa6+9xp49ezAMg7/+9a88/fTTOBwObrrpJvx+P4Zh8NRTT5GRkdHl/ak0RCJk69at4avevvDCC1xwwQWMGzeOZ555BsMwOHToEF6vl4suuii8TXl5OXD4mwC3bdvGhRdeaEt2+elITk7m9ttvZ9q0aWRmZhIKhcjLy5ifRC4AAAC3SURBVCMmJob58+czffp00tPTOe2007j55pu7vD9dsFAkAt555x1mzZpFcnIy9fX1DBgwgAULFvCzn/2M+++/n48//pi2tjYuvfRS7rvvPvr06cO5557L+PHj+eyzzwiFQsyaNatDoYhEAx3TEImQxMREHn/88aPGH3744R/dZs6cOTrwLVFNu6dERMQ07Z4SERHT9E5DRERMU2mIiIhpKg0RETFNpSEiIqapNERExDSVhoiImPZ/5BoY3Oxki0QAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#import numpy as np\n",
    "plt.hist(gapminder_copy['pop'])\n",
    "plt.xlabel('pop')\n",
    "plt.ylabel('count')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* __Boxplots__ - provide a way of comparing the summary measures (e.g., max, min,\n",
    "  quartiles) across variables in a data set. Boxplots can be particularly useful with larger data sets.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "Could not interpret input 'life_exp'",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-140-3795a9edee66>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0msns\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mboxplot\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mx\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'year'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'life_exp'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdata\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mgapminder_copy\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      2\u001b[0m \u001b[0mplt\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mxlabel\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'year'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      3\u001b[0m \u001b[0mplt\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mylabel\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'life_exp'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\seaborn\\categorical.py\u001b[0m in \u001b[0;36mboxplot\u001b[1;34m(x, y, hue, data, order, hue_order, orient, color, palette, saturation, width, dodge, fliersize, linewidth, whis, notch, ax, **kwargs)\u001b[0m\n\u001b[0;32m   2229\u001b[0m     plotter = _BoxPlotter(x, y, hue, data, order, hue_order,\n\u001b[0;32m   2230\u001b[0m                           \u001b[0morient\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcolor\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mpalette\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0msaturation\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 2231\u001b[1;33m                           width, dodge, fliersize, linewidth)\n\u001b[0m\u001b[0;32m   2232\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   2233\u001b[0m     \u001b[1;32mif\u001b[0m \u001b[0max\u001b[0m \u001b[1;32mis\u001b[0m \u001b[1;32mNone\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\seaborn\\categorical.py\u001b[0m in \u001b[0;36m__init__\u001b[1;34m(self, x, y, hue, data, order, hue_order, orient, color, palette, saturation, width, dodge, fliersize, linewidth)\u001b[0m\n\u001b[0;32m    444\u001b[0m                  width, dodge, fliersize, linewidth):\n\u001b[0;32m    445\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 446\u001b[1;33m         \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mestablish_variables\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mx\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0my\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mhue\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0morient\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0morder\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mhue_order\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    447\u001b[0m         \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mestablish_colors\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mcolor\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mpalette\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0msaturation\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    448\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\Anaconda3\\lib\\site-packages\\seaborn\\categorical.py\u001b[0m in \u001b[0;36mestablish_variables\u001b[1;34m(self, x, y, hue, data, orient, order, hue_order, units)\u001b[0m\n\u001b[0;32m    153\u001b[0m                 \u001b[1;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0minput\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mstring_types\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    154\u001b[0m                     \u001b[0merr\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;34m\"Could not interpret input '{}'\"\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0minput\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 155\u001b[1;33m                     \u001b[1;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0merr\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    156\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    157\u001b[0m             \u001b[1;31m# Figure out the plotting orientation\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mValueError\u001b[0m: Could not interpret input 'life_exp'"
     ]
    }
   ],
   "source": [
    "sns.boxplot(x='year', y='life_exp', data = gapminder_copy)\n",
    "plt.xlabel('year')\n",
    "plt.ylabel('life_exp')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pairs of variables\n",
    "\n",
    "* __Scatterplots__ - visualization of relationships across two variables..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'pop')"
      ]
     },
     "execution_count": 138,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEUCAYAAAAr20GQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3de1hUdf4H8PcgiPdF2RnYzNKslSLQWrfQTdR2ZQIZRHC95K1HQ01dyswyJSldu7jtQ2Vtm3ahUn7JZqJYolmrWbhbuqu0CKaWeUlguKyAAg4z398fwKwDA84Mc87MOfN+Pc8+z55zhjnfj2c6n+/53o5GCCFAREQ+z8/TBSAiIu/AhEBERACYEIiIqBkTAhERAWBCICKiZkwIREQEQAUJoba2FvHx8Th37lyHn9u/fz8MBgMMBgOWLl2KS5cuyVRCIiJlUHRCOHr0KKZNm4bTp093+Lnq6mosX74cGRkZyM3NRVhYGDIyMuQpJBGRQig6IWRnZyM9PR06nc66LycnBxMnTsSECROwYsUKNDQ04PTp07juuutw8803AwDGjh2LvXv3eqrYREReSdEJYe3atRg+fLh1+8SJE8jOzsYHH3yA7du3Izg4GG+99RYGDhyIkpISFBcXAwB27dqF8vJyTxWbiMgr+Xu6AO70z3/+Ez/++CMmT54MADCZTLjtttvQp08fvPDCC3jqqadgsVgwefJkBAQEeLi0RETeRVUJwWw2IzY2FmlpaQCAS5cuwWw2w2w2IzQ0FH/7298AAAUFBRgwYIAni0pE5HUU3WTU2t13341PP/0UFRUVEELg6aefxrvvvguNRoM5c+agtLQUQghkZmYiLi7O08UlIvIqqnpCCAsLw+LFizF79mxYLBbceuutmDdvHvz8/LB69Wo8+OCDuHLlCkaMGIG5c+d6urhERF5Fw+WviYgIUFmTERERuY4JgYiIADAhEBFRM0V3KldVXYLF0tQFEhzcCxUVtR4ukXTUHB9jUy41x6fG2Pz8NOjbt2e7xxWdECwWYU0ILdtqpub4GJtyqTk+NcdmD5uMiIgIABMCERE1Y0IgIiIATAhERNRM0Z3KRES+5GBhCT7afwoV1Q0I7hOIpNGDMSI81G3fz4RARKQABwtL8O6uYlxptAAAKqob8O6upne8uCspsMmIiEgBPtp/ypoMWlxptOCj/afcdg4mBCIiBaiobnBqvyuYEIiIFCC4T6BT+13BPgQiIhdJ3cl7taTRg236EACgq78fkkYPdts5mBCIiFwgRyfv1Vq+k6OMiIi8TEedvFI9JYwID5XsuwH2IRARuUSOTl65MSEQEblAjk5euTEhEBG5IGn0YHT1t72FuruTV27sQyAicoEcnbxyY0IgInKR1J28cmNCICLVkHNegBoxIRCRKsg9L0CN2KlMRKogx+JvaseEQESqoMZ5AXKTPCHU1tYiPj4e586da3Ns7969mDBhAhISErBw4UJcvHhR6uIQkUqpcV6A3CRNCEePHsW0adNw+vTpNsdqa2vx9NNPY8OGDdixYweGDBmC9evXS1kcIlIxNc4LkJukCSE7Oxvp6enQ6XRtjplMJqSnpyMkJAQAMGTIEFy4cEHK4hCRio0ID8Xs2DDrE0Fwn0DMjg1jh7ITJB1ltHbt2naP9e3bF+PGjQMA1NfXY8OGDZg5c6aUxSEilVPbvAC5eXzYaU1NDRYtWoSwsDBMnDjRqb8NDu5ls63V9nZn0byOmuNjbMql5vjUHJs9Hk0IZWVlmDt3LqKiorBixQqn/76iohYWiwDQdOGMxhp3F9FrqDk+xqZcao5PjbH5+WnaVKSv5rGEYDabsWDBAsTGxmLhwoWeKgYRSYyzh5VD9oSQkpKC1NRUlJSU4NixYzCbzdi9ezcA4Pbbb++w34GIlIWzh5VFloTw+eefW///xo0bAQAREREoLi6W4/RE5CGeeKsYuY4zlYlIMpw9rCxMCEQkGc4eVhYmBCKSDGcPK4vH5yEQkXqp8a1iasaEQESS4uxh5WBCICJSCKnndDAhEBEpgBxzOtipTESkAHK8EY4JgYhIAeSY08EmIyIfw7WFlCm4T6Ddm78753TwCYHIh7S0Q7fcWFraoQ8Wlni4ZHQtcszpYEIg8iFytEOTNOR4IxybjIh8CNcWUjap53TwCYHIh3BtIeoIEwKRD+HaQtQRNhkR+RCuLUQdYUIg8jFcW8h91DaElwmBiMgFanw9KPsQiIhcoMYhvEwIREQuUOMQXiYEIiIXqHEIr+QJoba2FvHx8Th37lybY0VFRUhKSoJer8fKlSvR2NgodXGIvM7BwhIs+8tXmPP851j2l6+4jIRCqHEIr6QJ4ejRo5g2bRpOnz5t9/iyZcuwatUq7N69G0IIZGdnS1kcIq/DtYWUS46lJOQm6Sij7OxspKen4/HHH29z7Pz586ivr8ewYcMAAElJSXjllVdw//33S1kkIq/SUcekkm8svkJtQ3glTQhr165t91hZWRm0Wq11W6vVorS0VMriEHkdNXZMknJ5bB6CxWKBRqOxbgshbLYdERzcy2Zbq+3tlrJ5KzXH56uxaft2h7Gqzu5+pfybKKWcrlBzbPZ4LCGEhobCaDRat8vLy6HT6Zz6joqKWlgsAkDThTMaa9xaRm+i5vh8ObbEewbZTG4CmjomE+8ZpIh/E1++dkrk56dpU5G2OS5jWWz0798fgYGBOHz4MABg+/btiI6O9lRxiDxCjR2TpFyyPyGkpKQgNTUVERERePHFF5GWloba2lqEh4dj1qxZcheHyOPU1jHpSWpbW0huGiGE8HQhXMUmI3VgbMrlTfG1XlsIaGp+c/WJy5tic5drNRlxcTsiO1jTVB4O4e08JgSiVtS4iqUv4BDezuNaRkStqHEVS1+gxrWF5MaEQNQKa5rKpMa1heTGJiOiVoL7BNq9+bOm6d34etDOY0IgaiVp9GC7o1VY0/R+HMLbOUwIRK2wpkm+igmByA7WNMkXsVOZiIgAMCEQEVEzJgQiIgLAPgQikhiXAVEOJgRSBN5UlInLgCgLm4zI6/FF9MrFZUCUhQmBvB5vKsrFZUCUhQmBvB5vKsrFBeeUhQmBvB5vKsrFBeeUhQmBvB5vKsrFd0YrC0cZkdfj2kLKxmVAlIMJgRSBNxUi6bHJiIiIAEicEHJzcxEXF4eYmBhs3ry5zfHCwkIkJycjISEB8+fPR3V1tZTFISKiDkiWEEpLS5GRkYGsrCzk5ORgy5YtOHnypM1n1q5di9TUVOzYsQODBg3CW2+9JVVxiIjoGiTrQ8jPz0dUVBSCgoIAAHq9Hnl5eVi8eLH1MxaLBZcuXQIA1NXV4Wc/+5lUxSGiZlwGhNojWUIoKyuDVqu1but0OhQUFNh8Zvny5ZgzZw6effZZdO/eHdnZ2U6dIzi4l822Vtvb9QIrgJrjY2zy2Hf4LN7LO44GkxlA0+S+9/KOo0/vbhjzqwEufac3xeduao7NHskSgsVigUajsW4LIWy26+vrsXLlSmRmZiIyMhLvvPMOnnjiCWzYsMHhc1RU1MJiEQCaLpzRWOO+ALyMmuNjbPLJ3FloTQYtGkxmZO4sRPgNQU5/n7fF505qjM3PT9OmIm1zXKoTh4aGwmg0WreNRiN0Op11+7vvvkNgYCAiIyMBAFOmTMHXX38tVXGICFwGhDomWUIYOXIkDh48iMrKStTV1WHPnj2Ijo62Hr/xxhtRUlKC77//HgDw2WefISIiQqriEBG4DAh1TLImo5CQECxZsgSzZs2CyWTCpEmTEBkZiZSUFKSmpiIiIgLPPfccHnnkEQghEBwcjGeffVaq4pCbsWNSmZJGD7Z5PwHAZUDofzRCCOHpQriKfQie0fqlJ0DTTcXVNWq8KTZ388bY3JnMvTE+d1FjbNfqQ+DSFeS0jt5PwKcE78dlQKg9XLqCnMaOSSJ1YkIgp7FjkkidmBDIaXw/AZE6sQ+BnMb3ExCpExMCuYQdk+7DIbzkLZgQiDyo9RDeiuoGvLurGACYFEh27EMg8qCOhvASyc3hJ4SqqiocOnQIfn5+uOuuu9C7t2+tAkgkBQ7hJW/i0BPCp59+ipiYGGRmZuLNN9/EuHHj8I9//EPqshGpHofwkjdx6AkhIyMDmzZtwpAhQwA0vfoyLS0N27Ztk7RwRGrHtYXImziUELp162ZNBgAQHh5u824DInINh/CSN3EoIURHR2PDhg2YMWMGunTpgpycHNxyyy24ePEihBDW12SSZ3H4ojJxCC95C4dWOw0PD4fZbLZ7TKPRoKioyO0FcwRXO/0fd69AKic1Xzs1xwaoOz41xuaW1U4LCwvdViCSBlcgJaLOcighWCwWvPXWW/jiiy/Q2NiI3/zmN1iwYAH8/TmvzVtw+CIRdZZDd/Q///nPKC4uxuzZs2GxWLBlyxa88MILWLlypdTlUzQ52/SD+wTavflz+KJr2B9DvsihhHDgwAFs3boVAQEBAIAxY8YgISFB0oIpndxLEnD4ovtwOQnyVQ5NTBNCWJMBAHTt2tVmm9qSe0mCEeGhmB0bZn0iCO4TqIgOZW/E5STIVzn0hBAWFoZnn30WM2bMgEajwfvvv49f/vKXUpdN0TzRps/hi+7B/hjyVQ49IaSnp6O6uhpxcXGIjY1FVVUVnnrqKanLpmhckkC5eO3IVzmUEMrKynDixAlYLBZYLBZcuHABly9fvubf5ebmIi4uDjExMdi8eXOb499//z1mzpyJhIQEzJ07FxcvXnQ+Ai/Ft4opF68d+SqHEsKTTz6JyZMn4+jRozhy5Aj0ev01RxiVlpYiIyMDWVlZyMnJwZYtW3Dy5EnrcSEEHnroIaSkpGDHjh249dZbsWHDhs5F40XYpq9cvHbkqxzqQ6irq8OUKVOs2zNnzkR2dnaHf5Ofn4+oqCjrshZ6vR55eXlYvHgxgKbJbj169EB0dDQAYMGCBaiurnYpCG/FNn3l4rUjX+RQQrjpppvwr3/9C3feeScA4LvvvsP111/f4d+UlZVBq9Vat3U6HQoKCqzbZ86cwc9//nOsWLECRUVFuOmmm5zul2g9BVurVfc7GtQcH2NTLjXHp+bY7HEoIfz000+YOXMmhgwZAn9/fxw7dgxarRYGgwFAU19BaxaLxWZFVCGEzXZjYyO+/vprbNq0CREREXjppZfw/PPP4/nnn3e48FzLSB0Ym3KpOT41xuaWtYwee+wxp08cGhqKQ4cOWbeNRiN0Op11W6vV4sYbb0RERAQAID4+HqmpqU6fh3wDZw4TSc+hhHDXXXc5/cUjR47E+vXrUVlZie7du2PPnj1Ys2aN9fgdd9yByspKFBcXIywsDJ9//jnCw8OdPg+pH2cOE8lDstXpQkJCsGTJEsyaNQsmkwmTJk1CZGQkUlJSkJqaioiICLz22mtIS0tDXV0dQkNDsW7dOqmKA4C1TKXiSq5E8pB0uVKDwWDtZ2ixceNG6/8fOnQoPvzwQymLYMVapnJx5jCRPByah6AGXJ9GuThzmEgePpMQWMtULs4cJpKHz7zhhu8LUC6+iJ5IHj6TEPi+AGXjzGEi6flMQmAtk4ioYz6TEADWMomIOuJTCYHch3M6iNSHCYGcxjkdROrkM8NOyX04p4NInZgQyGmc00GkTkwI5DTOHCZSJyYEchpnDhOpEzuVyWmc00GkTkwIKiLnUFDO6SBSHyYEleBQUCLqLPYhqASHghJRZzEhqASHghJRZzEhqASHghJRZzEhqASHghJRZ7FTWSU4FJSIOosJQUJyrwjKoaBE1BmSNhnl5uYiLi4OMTEx2Lx5c7uf27dvH+69914piyK7lmGgLZ26LcNADxaWeLhkRET2SZYQSktLkZGRgaysLOTk5GDLli04efJkm8+Vl5fjhRdekKoYHsNhoESkNJIlhPz8fERFRSEoKAg9evSAXq9HXl5em8+lpaVh8eLFUhXDYzgMlIiURrI+hLKyMmi1Wuu2TqdDQUGBzWfee+893HbbbRg6dKhL5wgO7mWzrdX2dul7pKDt2x3Gqjq7+10tpzfF526MTbnUHJ+aY7NHsoRgsVig0Wis20IIm+3vvvsOe/bsQWZmJkpKXGtXr6iohcUiADRdOKOxpnOFdqPEewbZLCUBNA0DTbxnkEvl9Lb43ImxKZea41NjbH5+mjYV6atJlhBCQ0Nx6NAh67bRaIROp7Nu5+XlwWg0Ijk5GSaTCWVlZbj//vuRlZUlVZFkX/wN4DBQIlIOjRBCSPHFpaWlmDZtGj788EN0794dU6dOxZo1axAZGdnms+fOncOsWbPw+eefO3UOZ54QWi/+BjTV2GfHhiniJq3G2koLxqZcao5PjbFd6wlBsk7lkJAQLFmyBLNmzUJiYiLi4+MRGRmJlJQUfPvtt1Kdtl0c9UNE1DFJJ6YZDAYYDAabfRs3bmzzueuvv97ppwNncdQPEVHHfGYtIy7+RkTUMZ9JCFz8jYioYz6zlhFH/RARdcxnEgLAxd+IiDriM01GRETUMSYEIiIC4GNNRkRESib1agtMCERECtB6tYWWd6wAcFtSYEIgInKRnOujdbTaAhMCEZEHyVFjv5ocqy2wU5mIyAVyr48mx2oLfEIgItWQswlH7vXRkkYPtrtisztXW2BCICJVkLsJJ7hPoN2bv1Tro8mx2gITAhFJSq5auxydrleTo8bemtSrLTAhEJFk5Ky1y92Eo8b10ZgQiHyM2oZKtpC7CQdQ3/poHGVE5ENaauwtN86WGvvBwhJJzidnrZ1L3HcenxCIPEytNXZA3lq7Gptw5MaEQGSHXDdpNU5uuprcHa9qa8JpTerfJZuMiFqRs1lFjZObrjYiPBSzY8Os3x/cJxCzY8NUfdOWihy/Sz4hkCKotVlF7TV2QN21drX9LiV9QsjNzUVcXBxiYmKwefPmNsf37t2LCRMmICEhAQsXLsTFixelLA4plJo7QlljVy41/i4le0IoLS1FRkYGPvroI3Tt2hVTp07F3XffjZtvvhkAUFtbi6effhpbt25FSEgIXn75Zaxfvx5paWlSFYncSG01o6vJ2RHKGrt78XfZOZI9IeTn5yMqKgpBQUHo0aMH9Ho98vLyrMdNJhPS09MREhICABgyZAguXLggVXF8wsHCEiz7y1eY8/znWPaXrySrqaixZnQ1OYcvssbuPr7wu/TvorHZ599Fo4y1jMrKyqDVaq3bOp0OBQUF1u2+ffti3LhxAID6+nps2LABM2fOdOocwcG9bLa12t6dKLH36yi+fYfP4r2842gwmQE0/SjfyzuOPr27YcyvBri1HDlfHrRbM8r58gckjLnFpe/sKDZt3+4wVtXZ3S/FNU8Y0xt9enfDe7uKUF5Vh5/37Y5Zsbe6/O94rTImjOnt8r+bN7hWfPsOn3Xbv2VH1P677NP7vxDCdp8QQJ/e3dx2PskSgsVigUbzv2wmhLDZblFTU4NFixYhLCwMEydOdOocFRW1sFia/oW02t4wGms6V2gvdq34MncWWpNBiwaTGZk7CxF+Q5Bby2LvP4KW/a5cg2vFlnjPILvNKon3DJLsmoffEIQX5o+w2SdFbEp3rfhaD6s1VtVhffYRVNfUu/0pSO7f5e0D++Lvds55+8C+klzzzJ2FMFtsM4LZIpz6b9zPT9OmIn01yRJCaGgoDh06ZN02Go3Q6XQ2nykrK8PcuXMRFRWFFStWSFUUj1HrUrxqXOXRl6i1nV3u32XBqQqn9neWojuVR44cifXr16OyshLdu3fHnj17sGbNGutxs9mMBQsWIDY2FgsXLpSqGB6j5qV42RGqXGqeCBc5OBh///dPdvdLQe4+hJ7duuBSvdnufneRLCGEhIRgyZIlmDVrFkwmEyZNmoTIyEikpKQgNTUVJSUlOHbsGMxmM3bv3g0AuP3227F27VqpiqTamhEg702aNXb3UvPvUs6Kitw1drmfSOw1uXe03xWSTkwzGAwwGAw2+zZu3AgAiIiIQHFxsZSnt6HmmhEg/02aNXb3UPvvUs5au5pjA4Daukan9rvCZ2Yqq7lm1II3aeVR++9Szlq7mmMDgF7d/e3e/Ht1d99t3GfWMlLzWHZyP7nmdHiiVuvM/s6Se/nrVsP00UUDyf6bk/vaidZjTq+x3xU+kxC4RAA5Ss4JTnL/Lj3Rzu7M/s7S+Gk63Han9mrm7qyxX81eh3JH+13hM01GHBmjbGrteFX7yJik0YPxZu4xXF2H1UCaWvtH+0+h0WxbW240C8ma366Y7N+I29vfWRoA9p4F3JnyfOYJgTV25VLzkgRy19jbG6LozqGLVzt57r9tbmKieb+7yZ3srjTab6ppb39ntfet7jybzzwhAKyxu5Naa+wAEBigQYOp7X9mgQHub36Q+yYmx9DFq+070vbpp2X/TH2YJOck1/lUQiD3OFhYgrd3HkPL03lFdQPe3nkMgDqGSl6xkww62t8ZfhrAYudrpWr6lmPo4tXa6+90Yz+oz+jqr7H79NHVXyHzEEhe7+8uxv4jP8Eimm4oo4ddJ0ktLOvT42jVVAuzaNovRULQaOzfQCSq1MryaN7CXjLoaH9nyfn0o3aBAV3arB/Wsl8KXQO64Epj28Td1Y3n86mEIGczByDfDbrlXFd3TloErNvuPqccox2uJnctU85auxxjy68m59MP0DTss3XloWW/u/l30bTpVG7ZLwX/LkCDyf5+KcjxdOczncoHC0vwzidFNh2T73xSJFnHZMsNuuXG0nKDfn+3NLOz97fTVtvefmrf6GHXObW/M+QeqSLn0w8A+Ldzd2xvf2fYSwYd7e8suStGcgwI8JmE8H97v7M7JO3/9n4nyfnkvkHL2fTQXn1LqkYHuUfGyEnukSpys9ek0tF+JenSzt2zvf2ddbnB/r9Ze/td4TMJQe7ONLnbhuUkdy3zrltDnNrfWfbmBXS0n3yT2eLc/s6So+nUZxKC3Nrr8JSqI1RO7bWlSzUyJv8/9l+t2t5+InINE4JE5L5pyknupx97o2I62k9ErmFCkIjcj5NERJ3FhEBERACYEIiIqBkTAhERAWBCICKiZkwIREQEgAmBiIiaSZoQcnNzERcXh5iYGGzevLnN8aKiIiQlJUGv12PlypVotLOSHxERyUOyhFBaWoqMjAxkZWUhJycHW7ZswcmTJ20+s2zZMqxatQq7d++GEALZ2dlSFYeIiK5BsoSQn5+PqKgoBAUFoUePHtDr9cjLy7MeP3/+POrr6zFs2DAAQFJSks1xIiKSl2TvQygrK4NWq7Vu63Q6FBQUtHtcq9WitLTUqXMEB/ey2dZqe7tUVlf/zlVyl1PO+NQcm6vnU3Nsnvg7Oc+lhNjceT7JEoLFYrF5T6sQwmb7WscdUVFRC0vzAjpabW8YjTUuldXVv3OVK+dTSnxqjs2V86k5NkA58ak5NmfO5+enaVORtjnurgK1FhoaCqPRaN02Go3Q6XTtHi8vL7c5TkRE8pIsIYwcORIHDx5EZWUl6urqsGfPHkRHR1uP9+/fH4GBgTh8+DAAYPv27TbH3e3t5fc6tZ/n845zqf18ao5N7vOpOTa5zqcRQqo30zYNO33jjTdgMpkwadIkpKSkICUlBampqYiIiEBxcTHS0tJQW1uL8PBwPPfcc+jatavD3++uJiMlUHN8jE251ByfGmO7VpORpAlBakwI6sDYlEvN8akxNo/1IRARkbIwIRAREQAmBCIiaibZPAQ5+LV6QXHrbbVRc3yMTbnUHJ/aYrtWPIruVCYiIvdhkxEREQFgQiAiomZMCEREBIAJgYiImjEhEBERACYEIiJqxoRAREQAmBCIiKgZEwIREQFQQEKora1FfHw8zp07BwD46KOPEBcXB4PBgD/+8Y9obGwEAGzbtg333HMPJkyYgAkTJiAjIwMA8NNPP2H69Om477778NBDD+HSpUsei8UeR+MrKyvDvHnzkJiYiKlTp1o/X11djXnz5iE2NhbTp0+3eQudpzkSW0VFhfWaTZgwAffeey/uuOMOAMqPDQDOnTuH6dOnY8KECZg5cybOnz8PALhy5QqWLVuG2NhYTJw4EadOnfJYLK05GltBQQGSk5NhMBgwf/586/Xx5theffVVjB8/HuPHj8e6desAAPn5+TAYDIiJibHeNwCgqKgISUlJ0Ov1WLlypTVub7+ndIrwYkeOHBHx8fEiPDxcnD17Vpw6dUqMGjVKlJaWCiGESE9PF2+//bYQQojVq1eL3NzcNt8xb948sXPnTiGEEK+++qpYt26dfAFcgzPxzZ49W2RlZQkhhMjKyhIPP/ywEEKIZ555RrzxxhtCCCG2bdtm3e9pzsTWwmw2ixkzZogdO3YIIdQR22OPPSY2b94shBDivffeE0uXLhVCCPHmm2+Kp556SgghxNdffy1+//vfeyCSthyNzWKxiNGjR4uDBw8KIYT4+OOPxfz584UQ3hvbV199JaZMmSIaGhrElStXxKxZs0Rubq4YPXq0OHPmjDCZTGLOnDli3759Qgghxo8fL/79738LIYR48sknrdfRm+8pneXVTwjZ2dlIT0+3vmv5+PHjGDZsmHV77Nix2Lt3LwDg22+/xbZt22AwGPDYY4/h4sWLMJlM+Oabb6DX6wEASUlJyMvL80wwdjgaX2VlJYqLizF16lQAQHJyMh555BEAwL59+2AwGAAA8fHx+OKLL2AymTwQjS1nrl2LrVu3onv37tZ41BCbxWJBbW0tAKCurg7dunUD0BRbQkICAODXv/41Kisr8dNPP8kdShuOxlZVVYX6+npERUVZ93/55Ze4cuWK18am1WqxfPlydO3aFQEBARg8eDBOnz6NG2+8EQMGDIC/vz8MBgPy8vJw/vx51NfXY9iwYQD+d+/w9ntKZ3l1Qli7di2GDx9u3Q4LC8PRo0dx4cIFmM1m5OXloby8HEDTxV64cCF27NiBX/ziF1i9ejWqqqrQq1cv+Pv7Wz9TWlrqkVjscTS+s2fP4rrrrsPzzz+P5ORkpKamIiAgAEBTU5JWqwUA+Pv7o1evXqisrPRIPFdz5toBgNlsxl//+lcsXbrUuk8NsT388MPIzMzEqFGj8PbbbyMlJQWAbWxA02+zpKRE3kDscDS2vn37okePHvjyyy8BAB9//DFMJhOqqj7hR7YAAAYPSURBVKq8NrZbbrnFeoM/ffo0du3aBY1GY1NWnU6H0tJSuzGUlpZ6/T2ls7w6IbQ2aNAgLF26FA899BCmT5+OIUOGWG+Mr732Gn71q19Bo9HgwQcfxIEDByCEgEZju9xr621v0l58jY2NOHbsGKKiorB161b89re/xfLly+1+hxACfn7ed1k7unYAcODAAQwcOBBDhgxp9zuUGNsTTzyB1atX48CBA3jmmWewePFiCCHa/DaVFptGo8Err7yCN954A4mJiaipqUFQUBACAgK8PrYTJ05gzpw5ePzxxzFgwIA2ZdVoNLBYLHb3K+2e4izvuUoOaGhoQGRkJHJycvDBBx8gJCQEAwYMQE1NDTIzM62fE0KgS5cu6NevH2pqamA2mwEARqPR+ujrjdqLT6vVomfPnhg7diyApuaTgoICAE01mpbaaGNjIy5duoSgoCCPxdCe9mJrsXfvXsTFxdn8jdJjq6ysxPfff4/f/e53AAC9Xg+j0YiqqiqEhISgrKzM+h3l5eVe+dvs6Lr5+/vj/fffR05ODhISEmCxWBAUFOTVsR0+fBgPPPAAli5diokTJyI0NNRmsELLPaL1/pYYlHZPcZaiEsLly5fxwAMPoLa2FleuXMGmTZsQFxeHHj164M0338TRo0cBAJs2bcK4ceMQEBCA4cOH45NPPgEA5OTkIDo62pMhdKi9+G644QaEhoZi//79AIC///3vCA8PBwCMHj0aOTk5AIBPPvkEw4cPt6l5e4v2Ymtx5MgRm6YKQPmx9e3bF4GBgTh06BCApptRz5490a9fP4wePRrbt28HABw6dAiBgYG47rrrPBmGXR1dtxUrVlgrJu+88w7uu+8++Pn5eW1sFy5cwKJFi/Diiy9i/PjxAIChQ4fihx9+wI8//giz2YydO3ciOjoa/fv3R2BgIA4fPgwA2L59O6KjoxV3T3GaZ/qynTN27Fhx9uxZIYQQ2dnZIi4uTsTExIhXXnnF+plvvvlGJCYmivvuu08sWLBAVFdXCyGEOHfunJgxY4aIjY0Vc+bMEf/97389EkNHHInv1KlTYsaMGWL8+PFiypQp4ocffhBCCFFVVSXmz58v4uLixJQpU6zf4y0ciU0IISIjI0V9fb3NPjXEdvToUTFp0iQRHx8vpkyZIgoLC4UQQtTX14vHH39cxMXFicTERPGf//zHIzG0x9HYEhMThV6vF6mpqaKmpkYI4b2xrVmzRgwbNkwkJCRY/5eVlSXy8/OFwWAQMTExYu3atcJisQghhCgqKhLJyclCr9eLRx99VDQ0NAghlHFPcRXfmEZERAAU1mRERETSYUIgIiIATAhERNSMCYGIiAAwIRARUTMmBCIiAsCEQEREzfw9XQAiJUhLS0NwcDCWLFkCoGnm6p49e5CcnIzXX38dJpMJ3bp1wxNPPIE77rgD5eXlWLVqFSoqKmA0GtG/f3+89NJLCA4Oxr333ovIyEgcP34cjz76KMaNG+fh6Iia8AmByAHTp0/H1q1brS9Jyc7OxqhRo5CRkYENGzYgJycHa9aswR/+8AdcvnwZH3/8MYYNG4YtW7bgs88+Q7du3azLOQBNK2/u2rWLyYC8Cp8QiBxw66234vrrr8e+ffswaNAglJWVwWw2o6ysDA888ID1cxqNBmfOnMHs2bNx6NAhvPPOOzh9+jROnDiBoUOHWj/Xet0mIm/AhEDkoJanhIEDB2Ly5MmwWCwYMWIEXnrpJetnLly4AJ1Ohz/96U/WV0zefffdaGxsxNWrxPTo0cMTIRB1iE1GRA7S6/UoKirC7t27kZycjBEjRuCrr76yvjN4//79SEhIQH19Pb788kvMnj0biYmJCA4ORn5+vnXJZCJvxScEIgd17doVer0e5eXl6NevH/r164fVq1fj0UcfhRAC/v7+eP3119GzZ08sWrQI69atw8svv4yAgADceeedOHPmjKdDIOoQVzslctDly5cxY8YMrFq1yvoqRiI1YZMRkQMOHDiAMWPGYNSoUUwGpFp8QiAiIgB8QiAiomZMCEREBIAJgYiImjEhEBERACYEIiJqxoRAREQAgP8HACfMbhewv2QAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# scatter plot goes here\n",
    "\n",
    "plt.scatter(gapminder_copy['year'], gapminder_copy['pop'])\n",
    "plt.xlabel('year')\n",
    "plt.ylabel('pop')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'pop')"
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEZCAYAAACXRVJOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3df1RUZf4H8DcqoKRE0gxz0jbLWiiC1NoUf5ZnhYUFVOSoUGJZlJnRUllukvjNo1ntOVRsdmLXMlNW2SyUdgU00y1wt8VW2FVIrWNqCjOgK6KAw8zz/UOYBZmBYZh779w779df3ecO93meZryf5z6/ro8QQoCIiLzeAKULQEREnoEBgYiIADAgEBFROwYEIiICwIBARETtGBCIiAiABgJCU1MT4uPjcfr06R4/t3//fiQkJCAhIQHPP/88Ll26JFMJiYjUQdUBobKyEikpKThx4kSPn2tsbMTy5cuRk5ODoqIihIWFIScnR55CEhGphKoDQkFBAbKzs6HX621phYWFmD17NmbOnImXX34Zra2tOHHiBG666SbcfvvtAIAHH3wQe/bsUarYREQeSdUBYc2aNbjvvvtsx8eOHUNBQQG2bt2KHTt2IDg4GBs2bMCoUaNQW1uLmpoaAMCuXbtQX1+vVLGJiDzSIKUL4E7/+Mc/8OOPP2Lu3LkAALPZjLvuuguBgYF4/fXX8corr8BqtWLu3Lnw9fVVuLRERJ5FUwHBYrEgNjYWWVlZAIBLly7BYrHAYrHAYDDgz3/+MwCgqqoKN998s5JFJSLyOKruMrrW+PHjsXv3bjQ0NEAIgVWrVuGjjz6Cj48PFi1ahLq6OgghsHHjRsTFxSldXCIij6KpJ4SwsDAsXboUCxcuhNVqxZ133oknnngCAwYMwKuvvorHH38cV65cQVRUFB577DGli0tE5FF8uP01EREBGusyIiIi1zEgEBERAAYEIiJqp+pB5fPnL8Fq5RAIEZEzBgzwwQ03XOfwvKoDgtUqGBCIiNyEXUZERASAAYGIiNoxIBAREQAGBCIiaqfqQWUiIm9y4HAtPt3/PRoaWxEc6I+kaaMRFW5w2/UZEIiIVODA4Vp8tKsGV9qsAICGxlZ8tOvqO17cFRTYZUREpAKf7v/eFgw6XGmz4tP937stDwYEIiIVaGhs7VO6KxgQiIhUIDjQv0/pruAYAhGRi6Qe5O0sadroLmMIAOA3aACSpo12Wx4MCERELpBjkLezjmtylhERkYfpaZBXqqeEqHCDZNcGOIZAROQSOQZ55caAQETkAjkGeeXGgEBE5IKkaaPhN6jrLdTdg7xy4xgCEZEL5BjklZuPEEK1b5hpaGjiC3KIiJw0YIAPgoOHOjzPJwQi0hQ51wZoDQMCEWmG3GsDtIaDykSkGXJsAKdlDAhEpBlaXBsgJ8kDQlNTE+Lj43H69Olu5/bs2YOZM2ciMTERS5YswYULF6QuDhFpmBbXBshJ0oBQWVmJlJQUnDhxotu5pqYmrFq1Cnl5edi5cydCQ0ORm5srZXGISOO0uDZATpIGhIKCAmRnZ0Ov13c7ZzabkZ2djZCQEABAaGgozp49K2VxiEjjosINWBgbZnsiCA70x8LYMA4oO0mWdQjTp0/Hpk2bMHLkSLvnW1pakJqaigULFmD27NlSF4eIiOxQfNrpxYsX8fTTTyMsLKzPwYAL04iInNfbwjRFZxkZjUakpqYiNDQUa9asUbIoREReT7EnBIvFgsWLFyM2NhZLlixRqhhEJAOuHlYH2QNCeno6MjIyUFtbiyNHjsBisaCkpAQAcPfdd/NJgUhjuHpYPbi5HRFJatn6MrsLw4ID/fHmkkkKlMh7efQYAhFpH1cPqwcDAhFJiquH1YMBgYgkxdXD6qH4OgQi0jYtvllMqzioTETkJfjGNCIijZB6PQcDAhGRCsixnoODykREKiDH2+AYEIiIVECO9RzsMiLyQtxbSH2CA/0drvh2Fz4hEHmZjr7ojptLR1/0gcO1CpeMeiLHeg4GBCIvI0dfNLmfHG+DY5cRkZfh3kLqFRVukLRrj08IRF6GewuRIwwIRF6GewuRI+wyIvIy3FuIHOFeRkRELlLb9F3uZUREJAEtvhqUYwhERC7Q4vRdBgQiIhdocfouAwIRkQu0OH1X8oDQ1NSE+Ph4nD59utu56upqJCUlISYmBitWrEBbW5vUxSHySAcO12LZ+jIsWrcXy9aXcRsJFdDi9F1JA0JlZSVSUlJw4sQJu+eXLVuGlStXoqSkBEIIFBQUSFkcIo/EvYXUSY6tJOQm6SyjgoICZGdn48UXX+x27qeffkJLSwvGjBkDAEhKSsI777yD1NRUKYtE5HF6GpxU883FG0i9lYTcJA0Ia9ascXjOaDRCp9PZjnU6Herq6vp0/Z7m0xKpxTkHg5DnGluh0w2TuTTkzRRbh2C1WuHj42M7FkJ0OXYGF6aRFgx3sM/98EB/mEwXFSgRaVVvC9MUm2VkMBhgMplsx/X19dDr9UoVh0gxWhycJHVS7AlhxIgR8Pf3x8GDB3Hvvfdix44dmDp1qlLFIVIM9xZyL7VtJ+FJZA8I6enpyMjIQEREBH73u98hKysLTU1NCA8PR1pamtzFIfIIWhucVIoWt5OQEze3IyLNWLa+zOF7h99cMkmBEnkWbm5H5CJ2PaiPFreTkBO3riCyg4vF1EmL20nIiQGByA4t7mTpDThjq3/YZURkB7se1IkztvqHAYHIjmAHi8XY9eD5OGPLdewyIrKDXQ/kjfiEQGQHux7IG3EdAhGRl/DYvYyIiMizMCAQEREABgQiImrHQWUikhy3AVEHBgRSDd5U1Ik7kKoHu4xIFbi3kHpxGxD1YEAgVeBNRb24DYh6MCCQKvCmol7cgVQ9GBBIFXhTUS9uA6IeDAikCrypqFdUuAELY8NswTs40B8LY8M4oOyBuHUFqQZnGRH1T29bVzAgEBF5Ce5lRERETpE0IBQVFSEuLg7R0dHYsmVLt/OHDx/GnDlzkJiYiCeffBKNjY1SFoeIiHogWUCoq6tDTk4O8vPzUVhYiG3btuH48eNdPrNmzRpkZGRg586duPXWW7FhwwapikNERL2QbOuK8vJyTJgwAUFBQQCAmJgYFBcXY+nSpbbPWK1WXLp0CQDQ3NyM66+/XqriEFEnHKAneyQLCEajETqdznas1+tRVVXV5TPLly/HokWLsHbtWgwZMgQFBQV9yqOnwREism/fwVPYVPwdWs0WAFcX920q/g6BwwbjgXtvVrh0pCTJAoLVaoWPj4/tWAjR5bilpQUrVqzAxo0bERkZiQ8//BAvvfQS8vLynM6Ds4yI+m7j54dtwaBDq9mCjZ8fRvjPghQqFclBsVlGBoMBJpPJdmwymaDX623HR48ehb+/PyIjIwEA8+bNwzfffCNVcYioHbcBIUckCwgTJ07EgQMHcO7cOTQ3N6O0tBRTp061nb/llltQW1uLH374AQDwxRdfICIiQqriEFE7bgNCjkjWZRQSEoLMzEykpaXBbDYjOTkZkZGRSE9PR0ZGBiIiIvDaa6/hN7/5DYQQCA4Oxtq1a6UqDkmAA5PqlDRtdJf3EwDcBoSu4kplcsm1Lz0Brt5UuEeNOjCYe6fexhD4xjRySU/vJ+CNxfNFhRv4PVE33LqCXMKBSSLtYUAgl3Bgkkh7GBDIJXw/AZH2cAyBXNLR/8yBSSLt4CwjIg/AWT8kB84yIvJw107hbWhsxUe7agCAQYFkxTEEIoX1NIWXSE5OPyGcP38eFRUVGDBgAO6//34MGzZMynIReQ1O4SVP4dQTwu7duxEdHY2NGzfij3/8I2bMmIG///3vUpeNyCtwCi95CqeeEHJycrB582aEhoYCuPrqy6ysLHz22WeSFo7IG3BvIfIUTgWEwYMH24IBAISHh3d5twF5Bs5UUSdO4SVP4VRAmDp1KvLy8vDwww9j4MCBKCwsxB133IELFy5ACGF7TSYphzNV1I17C5EncGodQnh4OCwWi91zPj4+qK6udnvBnMF1CP+zbH2Z3UHI4EB/vLlkkgIlIiJP45Z1CIcPH3ZbgUganKlCRP3lVECwWq3YsGED/va3v6GtrQ2TJk3C4sWLMWgQ17V5iuBAf4dPCEREznCqy+jNN99ETU0NUlJSYLVasW3bNtx2221YsWKFHGV0yNO7jOQc5OULa9yPg/SkNW7pMvrqq6+wfft2+Pr6AgAeeOABJCYmuqeEGiX3IC9nqrgXB+nJGzkVEIQQtmAAAH5+fl2OqTsl3ijGmSruwzfCkTdyaqVyWFgY1q5di5MnT+LUqVNYu3Ytfv7zn0tdNlXjIK+68fsjb+RUQMjOzkZjYyPi4uIQGxuL8+fP45VXXpG6bKrG7QjUjd8feSOnAoLRaMSxY8dgtVphtVpx9uxZXL58ude/KyoqQlxcHKKjo7Fly5Zu53/44QcsWLAAiYmJeOyxx3DhwoW+18BD8Y1i6sbvj7yRUwHht7/9LebOnYvKykocOnQIMTExvc4wqqurQ05ODvLz81FYWIht27bh+PHjtvNCCDz11FNIT0/Hzp07ceeddyIvL69/tfEgUeEGLIwNs7UogwP9OeNHRfj9kTdyalC5ubkZ8+bNsx0vWLAABQUFPf5NeXk5JkyYYNvWIiYmBsXFxVi6dCmAq4vdAgICMHXqVADA4sWL0djY6FIlPBUHedWN3x95G6cCwm233YZvv/0W48aNAwAcPXoUI0eO7PFvjEYjdDqd7Viv16Oqqsp2fPLkSdx44414+eWXUV1djdtuu63P4xI9zaclIqK+cSognDlzBgsWLEBoaCgGDRqEI0eOQKfTISEhAcDVsYJrWa3WLjuiCiG6HLe1teGbb77B5s2bERERgbfeegvr1q3DunXrnC68py9MIyLyJG5ZmPbCCy/0OWODwYCKigrbsclkgl6vtx3rdDrccsstiIiIAADEx8cjIyOjz/mQ9+DKYSJpORUQ7r///j5feOLEicjNzcW5c+cwZMgQlJaWYvXq1bbzY8eOxblz51BTU4OwsDDs3bsX4eHhfc6HvANXDhNJT7Ld6UJCQpCZmYm0tDSYzWYkJycjMjIS6enpyMjIQEREBN59911kZWWhubkZBoMBb7zxhlTFAcAWpppx5TCR9Jza3M5T9WUMgZu/qduidXsdnvtg+XQZS0KkXr2NITi1DkELemphkufjymEi6XlNQODeNOrGlcNE0vOaN9zwBTLqxu29iaTHMQSOIRCRl3DLOgQtYAuTiKhnXvOEQO7HabxE6sInBJIEF4oRaY/XzDIi9+I0XiLtYUAgl3AaL5H2MCCQS7hQjEh7GBDIJVwoRqQ9HFQml3AaL5H2cNopEZGX4LRTL8J1AUTUHwwIGsF1AUTUXxxU1giuCyCi/mJA0AiuCyCi/mJA0AiuCyCi/mJA0AiuCyCi/uKgskZwXQAR9RfXIUiI00CJyJP0tg5B0i6joqIixMXFITo6Glu2bHH4uX379mH69OlSFkV2HdNAOwZ1O6aBHjhcq3DJiIjskywg1NXVIScnB/n5+SgsLMS2bdtw/Pjxbp+rr6/H66+/LlUxFMNpoESkNpIFhPLyckyYMAFBQUEICAhATEwMiouLu30uKysLS5culaoYiuE0UCJSG8kGlY1GI3Q6ne1Yr9ejqqqqy2c2bdqEu+66C/fcc49LefTUF6Y03Q1DYDrfbDddpxumQImIiHomWUCwWq3w8fGxHQshuhwfPXoUpaWl2LhxI2prXetX7+ugspyDvLMm39plKwng6jTQWZNvhcl0UZI8iYh6otjmdgaDARUVFbZjk8kEvV5vOy4uLobJZMKcOXNgNpthNBqRmpqK/Px8Scoj914/nAZKRGoj2bTTuro6pKSk4JNPPsGQIUMwf/58rF69GpGRkd0+e/r0aaSlpWHv3r19yqMvTwjL1pfZ7b8PDvTHm0sm9SlfIiI1UmzaaUhICDIzM5GWloZZs2YhPj4ekZGRSE9Px7///W+psnWIg7xERD3zmoVpfEIgIm+n6MI0T8K9foiIeuY1exlxkJeIqGde02VEROTt2GVEREROYUAgIiIADAhERNTOawaViYjUTurtdxgQiIhUQI7tdxgQiIhcJOeGmT29Y4UBgYhIQXJvmCnH9jsMCESkKXK12uVosXcWHOjvcPsdd+EsIyLSDDnfZS73hplybL/DgEBEmiHnu8wdtczd2WLvLCrcgIWxYbbrBwf6Y2FsGGcZEZG6yNWNI2erPWnaaLtvRZRyw8yocIOk+68xIBCRpOQcfJWjn72DFjfMZEAg8kJamy7ZQe5Wu9QtdrkxIBB5ADlv0FqcLtlBi612OTEgEClM7hu0FqdLdqa1VrucGBCIHNDqfHYlpkvKPfiqVdzLiEgBcrba5b5BK9FiB9iN01/cy4ioE60OhMp9g9bidEmlaO03KWlAKCoqwnvvvYe2tjYsXLgQDz30UJfze/bsQW5uLoQQGDlyJF577TVcf/31UhaJ3IgDoe6hxMwYQLstdrl+l1r8TUoWEOrq6pCTk4NPP/0Ufn5+mD9/PsaPH4/bb78dANDU1IRVq1Zh+/btCAkJwdtvv43c3FxkZWVJVSRyIw6Euo8SN2gtt9jl+l1q8TcpWUAoLy/HhAkTEBQUBACIiYlBcXExli5dCgAwm83Izs5GSEgIACA0NBRFRUVSFccraO3xtTOtD4Rq9QYtNzl/l0r8Jj/8azXaLMKWNmigj1t/k5IFBKPRCJ1OZzvW6/WoqqqyHd9www2YMWMGAKClpQV5eXlYsGBBn/IIDh7qnsJqwL6Dp7Cp+Du0mi0Arv4oNxV/h8Bhg/HAvTe7Pb9zDn705xpbodMNc3t+uhuGwHS+2W66FPklPjAMgcMGY9OuatSfb8aNNwxBWuydkvy/9Ab7Dp6S5f+lnL9LuX+TgcP+CyG6pgkBBA4b7Lb8JAsIVqsVPj4+tmMhRJfjDhcvXsTTTz+NsLAwzJ49u095NDQ0wWoVvX9QIXK22Dd+ftgWDDq0mi3Y+PlhhP8syO35DXfw+Do80B8m00W35zdr8q12W+yzJt8qSX4AEP6zILz+ZFSXNKny0rJru3FM55uRW3AIjRdb3P7vQc7f5d2jbsCXdgLC3aNukOR3svHzw7Bcc7+zWEWf/o0PGODTY0Nast1ODQYDTCaT7dhkMkGv13f5jNFoRGpqKkJDQ7FmzRqpiqIIObfh7bh+X9L7S46teDuTY6dHb3LgcC2WrS/DonV7sWx9mWS/S0DeHUiTpo3GoIFdG57u7lbpUPV9Q5/S+0vVg8oTJ05Ebm4uzp07hyFDhqC0tBSrV6+2nbdYLFi8eDFiY2OxZMkSqYqhGC0OOHXGgVD10uLsmM7ENa3oa4/dRe56XTd4IC61WOymu4tkASEkJASZmZlIS0uD2WxGcnIyIiMjkZ6ejoyMDNTW1uLIkSOwWCwoKSkBANx9992SPinI2YWj9UFQgDdod9LyhAA5Gyuf7v8elmvu/xYBTawfsdfl3lO6KyRdh5CQkICEhIQuaX/4wx8AABEREaipqZEy+y7kbhV5Q4ud3EPrLfbI0cH48l9n7Ka7m5x1k7NeANDU3NandFd4zUpluVtFbLGTs7TcYgfk7WuXs25yjyEMHTLI7s1/6BD33ca9JiDI3Spii139tPiWL0D+lq3cq74/+PxIl26jgT6QpCEm+9jItXNOe0l3hdcEBLlbRQBb7Gqm1bd8AfK3bOWun4APAHHNsfvJ0WLvzN6Ack/prpBs2qmnkXuaJLmfVqdKOmqZa6HFDlz9t3ftLdkH0rTa/7TnKKzXtJitQuBPe466Pa8rZvs3Ykfp/eUorLkz3HlNQOA8dnXT8roOuVvsjqYpunP6YmfHT/8X13ZqiPZ0d5Nj4LXDlTb7XTWO0vvL0VXdmZvXdBkB7MJxNy1PlfT39UGrufs/NX9f93c/yN1il2P6Ymf7DnUfr+hIXxATJkme5BqvCghaJ/d21J0H7xoaW/HB50cAaGOq5BU7waCn9P4Y4APYWzs1QJr7s6ytaADd9t/pLZ3s8xvkY/fpw2+Q+34oXtNlpHUdN+jOXSoffH5Esi6V/N3f2V0AlL/7O0nyc9R4lahRK8vjeQdHC2ml2qZL7v+XcpKjn72Dv6/9LjZH6f3l5+C6jtJd4VVPCHK2oAHg45Ia7D90BlZxtbU3bcxNkj0i93SDlqKOcsx46EzuVqacrXa5Z6vI/f9yoA+6/TY70t1NzkA+aCDQarafLgU5nuy85gnhwOFafPjX6i4t6A//Wi1ZC/rjkhp8+a8ztpuKVQBf/usMPi6RZnW23DdorZs25qY+pfeH3LNV5DbIwR3SUXp/OArYUgRyuf/NOeoaYpeRC/6052iXF0sAQJtFmuloALDfwUCao3S1kfPRHJDnH4NS5J6t4qhlLkWLHUC3bdl7S+8PObvfBjq4ezpK7y85fideExDkHkiTu1/42i1/e0vvLzkfzQF5+k87s7eSt6d0NbHXfdNTOtlnsfYtXQ28JiBo3bVPP72l95ecj+aA/AGdyBsxIJBL5H4CIiLpMSAQEREABgQiImrHgEBERAAYEIiIqB0DAhERAWBAICKidgwIREQEQOKAUFRUhLi4OERHR2PLli3dzldXVyMpKQkxMTFYsWIF2tq4yIiISCmSBYS6ujrk5OQgPz8fhYWF2LZtG44fP97lM8uWLcPKlStRUlICIQQKCgqkKg4REfVCsoBQXl6OCRMmICgoCAEBAYiJiUFxcbHt/E8//YSWlhaMGTMGAJCUlNTlPBERyUuy9yEYjUbodDrbsV6vR1VVlcPzOp0OdXV1fcojOHho/wsKQKcb5pbrMD/mp+a8tJ6fVvNyZ36SBQSr1drlHa1CiC7HvZ13RkNDE6xu2DzHZLrY72swP+an9ry0np9W8+pLfgMG+PTYkJasy8hgMMBkMtmOTSYT9Hq9w/P19fVdzhMRkbwkCwgTJ07EgQMHcO7cOTQ3N6O0tBRTp061nR8xYgT8/f1x8OBBAMCOHTu6nHe3D5ZP71M682N+cuWn5brJnZ9W85IrPx8hpHqT6tVpp++//z7MZjOSk5ORnp6O9PR0ZGRkICIiAjU1NcjKykJTUxPCw8Px2muvwc/Pz+nru6vLiIjIG/TWZSRpQJAaAwIRkfMUG0MgIiJ1YUAgIiIADAhERNROsnUIchgg1RvdiYg0qLd7pqoHlYmIyH3YZURERAAYEIiIqB0DAhERAWBAICKidgwIREQEgAGBiIjaMSAQEREABgQiImqn6pXKUjl27Bhyc3MREBCAhIQETJo0SekiuRXrp15arhug7vqpuew2gro5dOiQOHv2rDAajWL58uVKF8ftWD/10nLdhFB3/dRc9g7sMrLjnnvuQUtLC5555hlMmTJF6eK4HeunXlquG6Du+qm57B0YEOz4z3/+gxtvvBFbt27FJ598onRx3I71Uy8t1w1Qd/3UXPYOHEOwo7W1FStWrMDQoUMxbdo0pYvjdqyfemm5boC666fmstvI3Ud18eJF8etf/1qcOnXK7vn3339fREdHi/j4eLF+/Xpb+sMPPyzi4uJEYmKiSExMFIcOHXJL3jt37hSxsbFixowZYvPmzX2vkBN5dOaofl988YWYPXu2+NWvfiVWr17ttrw9oX4FBQW27y0xMVGMGzdO/N///Z9b8nZn/Vz97goLC0VcXJyIi4sT69atc1venvDd9ZTe37zdXT97cnNzbd/N66+/7tI1lCq7EmQNCIcOHRLx8fEiPDzc7o+yrKxMxMfHi4sXL4q2tjbx5JNPipKSEmG1WsXkyZOF2Wx2eO2jR4/a/vunn34Sly9f7jXv2tpa8eCDD4rz58+LS5cuiYSEBHHs2DHZ63fy5EkxefJkcfbsWXHlyhWRkpIi9u3bp5n6XVuPGTNmiIaGBo+qn6t1u3z5svjFL34hGhoahNlsFsnJyaKsrMyj6taf+jn7nSpdP3vKysrEvHnzRGtrq7hy5YpIS0sTpaWlTpdbybIrRdYxhIKCAmRnZ0Ov19s9f+TIEUyePBlDhw7FwIEDMWXKFOzZswc//PADAGDRokVITEzE5s2bu/xdW1sbli1bhi1btqC2thaPPvooqqqqes27vLwcEyZMQFBQEAICAhATE4Pi4mLZ67d7927ExcXBYDDA19cXOTk5uOeeezRTv85WrVqFzMxMDB8+3KPq52rdLBYLrFYrmpub0dbWhra2Nvj7+3tU3fpTv96+U0+pnz06nQ7Lly+Hn58ffH19MXr0aJw5c8bpcitZdqXIGhDWrFmD++67z+H58PBwfP311/jvf/+L1tZW7N27F/X19WhsbERUVBTeffddbNy4EVu3bkVZWZnt7wYNGoQNGzYgPz8f8+bNw7PPPovx48f3mrfRaIROp7Md6/V61NXVyV6/H3/8ERaLBYsXL8bMmTORn5+P66+/XjP161BeXo6WlhbExsZ2+TtPqJ+rdRs6dCieffZZxMbGYtq0aRgxYgTGjRvnUXXrT/16+049pX723HHHHRgzZgwA4MSJE9i1a5etb9+ZcitZdqV41KByVFQUkpKSsGDBAgQFBSEqKgqVlZUYO3Ysxo4da/tccnIy9u/fb3fhhxACZrPZqfysVit8fP73SjkhRJdjd3NUP4vFgoqKCnz88ccICAjAU089hc8++wxJSUndrqHG+nXYunUrHn300R6v4an1c1S3mpoabN++HV9++SWGDRuGF154ARs2bMDjjz/e7RqeWjfAcf16+04789T6HTt2DE8++SRefPFFjBo1qtv5vpQbkP+7kZNHTTttampCdHQ0ioqK8PHHH8PPzw8333wzKioqcODAAdvnhBAYNOh/scxsNuPRRx9FWloatm7dit///vddPu+IwWCAyWSyHZtMJoeP1O7gqH433ngjoqKiMHz4cAwePBi//OUvuzy+qr1+AHDlyhX885//xPTp07v9nRrq56huX3/9NaKiohAcHAw/Pz8kJSXhm2++UVXdAMf16+k7BTy/fgcPHsQjjzyC559/HrNnz+53ueUsuyKUGLh48MEH7Q5sVVdXi8TERGE2m0VjY6OIiYkRFRUVYu/evWLWrFmipaVFXLx4USQkJIhvv4J1LUQAAAOPSURBVP22y98eP37c9t+OBoiuzbtjcKihoUFcvnxZJCYmisrKStnrd+jQIRETEyMuXLhgG7grKCjQTP2EEKKqqkrMnz/f4TU9pX59rdtXX30lEhMTxaVLl4TVahWvvPKKeOeddzyybq7Ur6fv1BPr19mZM2fE+PHjRXl5ud3zzpZbibIrxSO6jGbOnIm8vDyEhYUhOjoaiYmJsFgseOSRR3DvvfcCACorKzFr1ixYrVakpqZ26UICgNGjR9v++6abbnIq35CQEGRmZiItLQ1msxnJycmIjIx0X8XaOVO/xx9/HKmpqTCbzZg0aRLmzJnT5Rpqr9+pU6dgMBgcXsNT6+dM3Y4cOYKkpCT4+voiIiICTzzxRJdreGrdAOfq5yi9g6fWb8OGDWhtbcW6detsafPnz0dKSorL5Qbk+26U4COEEEoXgoiIlOdRYwhERKQcBgQiIgLAgEBERO0YEIiICAADAhERtWNAICIiAAwIRETUjgGBiIgAeNjmdkSeKisrC8HBwcjMzAQA7NixA6WlpZgzZw7ee+89mM1mDB48GC+99BLGjh2L+vp6rFy5Eg0NDTCZTBgxYgTeeustBAcHY/r06YiMjMR3332H5557DjNmzFC4dkRX8QmByAkPPfQQtm/fjra2NgBX98mfMmUKcnJykJeXh8LCQqxevRrPPPMMLl++jL/85S8YM2YMtm3bhi+++AKDBw/Gjh07bNe74447sGvXLgYD8ih8QiBywp133omRI0di3759uPXWW2E0GmGxWGA0GvHII4/YPufj44OTJ09i4cKFqKiowIcffogTJ07g2LFjXV561NO7CYiUwoBA5KSOp4RRo0Zh7ty5sFqtiIqKwltvvWX7zNmzZ6HX6/Hmm2+iqqoKc+bMwfjx49HW1obO24YFBAQoUQWiHrHLiMhJMTExqK6uRklJCebMmYOoqCiUlZXh+++/BwDs378fiYmJaGlpwddff42FCxdi1qxZCA4ORnl5OSwWi8I1IOoZnxCInOTn54eYmBjU19dj+PDhGD58OF599VU899xztpc2vffee7juuuvw9NNP44033sDbb78NX19fjBs3DidPnlS6CkQ94vbXRE66fPkyHn74YaxcudL2rl4iLWGXEZETvvrqKzzwwAOYMmUKgwFpFp8QiIgIAJ8QiIioHQMCEREBYEAgIqJ2DAhERASAAYGIiNoxIBAREQDg/wE0vEkvAYvUgwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# let's try plotting the log of x\n",
    "plt.scatter(gapminder_copy['year'], gapminder_copy['pop'])\n",
    "plt.xscale('log')\n",
    "plt.xlabel('year')\n",
    "plt.ylabel('pop')\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's try plotting the log of x\n",
    "plt.scatter(gapminder_copy['year'], gapminder_copy['pop'])\n",
    "plt.xscale('')\n",
    "plt.xlabel('year')\n",
    "plt.ylabel('pop')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "### Why use `seaborn`?\n",
    "\n",
    "As noted above, Seaborn is a high-level plotting library for statistical data \n",
    "visualization. In addition to simplifying plotting, it also provides facilities \n",
    "for customizing matplotlib plots (accessible via `sns.set()`)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Saving your plots as image files  \n",
    "If you'd like to save your plots as an image file, you can run `fig.savefig('my_figure.png')` where `\"my_figure\"` is the file name.    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Interpret plots and summaries\n",
    "\n",
    "### Exploration is an iterative process\n",
    "\n",
    "In this lesson, we've taken the raw data and worked through steps to prepare it for analysis, but we have not yet done any \"data analysis\".  This part of the data workflow can be thought of as \"exploratory data analysis\", or EDA.  Many of the steps we've shown are aimed at uncovering interesting or problematic things in the dataset that are not immediately obvious.  We want to stress that when you're doing EDA, it will not necessarily be a linear workflow like what we have shown.  When you plot or summarize your data, you may uncover new issues: for example, we saw this when we made a mistake fixing the naming conventions for the Democratic Republic of Congo.  You might discover outliers, unusually large values, or points that don't make sense in your plots.  Clearly, the work here isn't done: you'll have to investigate these points, decide how to fix any potential problems, document the reasoning for your actions, and check that your fix actually worked.\n",
    "\n",
    "On the other hand, plots and summaries might reveal interesting questions about your data.  You may return to the cleaning and prepping steps in order to dig deeper into these questions.  You should continuously refine your plots to give the clearest picture of your hypotheses.\n",
    "\n",
    "### Interesting findings\n",
    "\n",
    "What's interesting about this data set? What would you like to investigate further?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Answer:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Putting it all together\n",
    "\n",
    "On your own or with a partner, using the techniques you've learned in this lesson, explore one or both of the data sets provided in Data Carpentry's Lessons [Introduction to Genomics](http://www.datacarpentry.org/introduction-genomics/02-examining-sra-runtable.html) and [Python for Ecologists](http://www.datacarpentry.org/python-ecology-lesson/01-starting-with-data/). We've provided headers to guide you through the process.\n",
    "\n",
    "* The Genomics Lesson uses the [Lenski dataset](http://www.ncbi.nlm.nih.gov/sra?term=SRA026813). Follow section A. in [these directions](http://www.datacarpentry.org/introduction-genomics/02-examining-sra-runtable.html) to download the .csv file.\n",
    "* The Ecology Lesson uses [surveys.csv](https://ndownloader.figshare.com/files/2292172) from the Portal Teaching data, a subset of the data from [Ernst et al Long-term monitoring and experimental manipulation of a Chihuahuan Desert ecosystem near Portal, Arizona, USA](http://www.esapubs.org/archive/ecol/E090/118/default.htm). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Import your data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Describe your data set here using *Markdown*\n",
    "What is the general shape of your `DataFrame`? What are the datatypes? Are there missing values? What questions do you have about your data set and how will you answer those questions?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Answers:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Calculate summary statistics for your data set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Is this a tidy data set? Why or why not?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Create bar, box, and scatter plots of your data set. What insights do these plots provide?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Key points\n",
    "\n",
    "* Data exploration involves inspecting, summarizing, and plotting data in various ways.\n",
    "* Successful data exploration may reveal that the data needs to be modified or cleaned.\n",
    "* Data exploration is an iterative process."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}