00 Data Wrangling with Pandas.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img align=\"center\" src=\"http://sydney.edu.au/images/content/about/logo-mono.jpg\">\n",
    "<h1 align=\"center\" style=\"margin-top:10px\">Data Wrangling with Pandas (Short Guide)</h1>\n",
    "<h3 align=\"center\" style=\"margin-top:20px\">Marcel Scharth (The University of Sydney Business School)</h3>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first step in any data project is to get the data ready for analysis. In a traditional curriculum, the dataset was usually taken for granted and ready for analysis from the start. However, this no longer reflects the reality in industry: the trend towards [big data](http://www.forbes.com/sites/bernardmarr/2016/04/28/big-data-overload-most-companies-cant-deal-with-the-data-explosion/#52c807d83920) brought with it an increased demand for professionals that have the computational skills to work with possibly large, complex and unstructured data. It is [commonly stated](https://www.r-bloggers.com/the-real-prerequisite-for-machine-learning-isnt-math-its-data-analysis/) that up to 80% of a data scientist's job can be data preparation, exploratory data analysis (EDA), and visualisation. \n",
    "\n",
    "In data science terminology, the process of processing data is called data wrangling or munging. Common tasks include loading the data, merging datasets, identifying and handling errors (data cleaning), dealing with missing values, etc.\n",
    "\n",
    "In this lesson we explore the basics of how to use the pandas package to work efficiently with data in Python. As a complement to this guide, the <a href=\"http://pandas.pydata.org/pandas-docs/stable/10min.html\" target=\"_blank\">10 minutes to pandas</a> tutorial in the official pandas documentation is also a useful starting reference.\n",
    "\n",
    "<a href=\"#1.-Dataset\">Dataset</a> <br>\n",
    "<a href=\"#2.-Importing-the-data\">Importing the data</a> <br>\n",
    "<a href=\"#3.-Viewing-the-Data\">Viewing the data</a> <br>\n",
    "<a href=\"#4.-Data-selection\">Data selection</a> <br>\n",
    "<a href=\"#5.-Exporting-data\">Exporting data</a> <br>\n",
    "<a href=\"#6.-Descriptive-statistics\">Descriptive statistics</a> <br>\n",
    "<a href=\"#7.-Viewing-column-types\">Data column types</a> <br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Dataset\n",
    "\n",
    "We will use the `Credit` dataset from the  <a href=\"http://www-bcf.usc.edu/~gareth/ISL/index.html\" target=\"_blank\">Introduction to Statistical Learning</a> texbook by James, Witten, Hastie and Tibshirani. It records the average credit card balace at end of the month for customers of a financial services company, as well as other individual characteristics such age, education, gender, marital status, number of cards, and credit rating.\n",
    "\n",
    "### 2. Importing the data\n",
    "\n",
    "We start by loading the data. First, we need to load the pandas package. Since the dataset is in the `csv` format, we use the <a href=\"http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html\" target=\"_blank\">read_csv</a> function to import the file. The package will automatically read the column names and infer the variable types.  We assign the data variable to a variable called `data`, which will store the dataset as an object called a `DataFrame`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We will always assume that the data file is in a subdirectory called \"Data\"\n",
    "\n",
    "data=pd.read_csv('Data/Credit.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Note: To get help on any function or object, append a question mark to it and run the cell.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "#pd.read_csv? # data?, data.head?, etc (the hash starts a comment, everything after it is ignored when running the cell)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Viewing the Data\n",
    "A pandas `DataFrame` has some similarities to a spreadsheet. However, unlike a spreadsheet, you are not able to click through it or manually make modifications in the current setup. We do everything through coding. This may initially feel restrictive, but it is ultimately more efficient and scalable to work in this way. Some Python environments such as Spyder have GUIs (graphical user interfaces) for viewing and modifying data frames.  \n",
    "\n",
    "Two basic methods to have a first view of the data are the `head` and `tail` methods. The head method displays the first rows of the data (by default five), while the tail displays the last rows. You can specify the number of rows within the parentheses. \n",
    "\n",
    "Running a cell with only the name of the `DataFrame` will provide a full view, but pandas limits number of rows that can be shown. See [here](http://stackoverflow.com/questions/19124601/is-there-a-way-to-pretty-print-the-entire-pandas-series-dataframe) if you want to change this setting.\n",
    "\n",
    "Note that only the last line of a Jupyter cell will generate an output on the screen. We can use the `print` function from thestandard Python library to output multiple variables (see the next section for some examples). However, `print` does not include the table formatting that you can see below (the technical term is \"pretty printing\").  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Obs</th>\n",
       "      <th>Income</th>\n",
       "      <th>Limit</th>\n",
       "      <th>Rating</th>\n",
       "      <th>Cards</th>\n",
       "      <th>Age</th>\n",
       "      <th>Education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Student</th>\n",
       "      <th>Married</th>\n",
       "      <th>Ethnicity</th>\n",
       "      <th>Balance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>14.891</td>\n",
       "      <td>3606</td>\n",
       "      <td>283</td>\n",
       "      <td>2</td>\n",
       "      <td>34</td>\n",
       "      <td>11</td>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Caucasian</td>\n",
       "      <td>333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>106.025</td>\n",
       "      <td>6645</td>\n",
       "      <td>483</td>\n",
       "      <td>3</td>\n",
       "      <td>82</td>\n",
       "      <td>15</td>\n",
       "      <td>Female</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Asian</td>\n",
       "      <td>903</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>104.593</td>\n",
       "      <td>7075</td>\n",
       "      <td>514</td>\n",
       "      <td>4</td>\n",
       "      <td>71</td>\n",
       "      <td>11</td>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Asian</td>\n",
       "      <td>580</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>148.924</td>\n",
       "      <td>9504</td>\n",
       "      <td>681</td>\n",
       "      <td>3</td>\n",
       "      <td>36</td>\n",
       "      <td>11</td>\n",
       "      <td>Female</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Asian</td>\n",
       "      <td>964</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>55.882</td>\n",
       "      <td>4897</td>\n",
       "      <td>357</td>\n",
       "      <td>2</td>\n",
       "      <td>68</td>\n",
       "      <td>16</td>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Caucasian</td>\n",
       "      <td>331</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Obs   Income  Limit  Rating  Cards  Age  Education  Gender Student Married  \\\n",
       "0    1   14.891   3606     283      2   34         11    Male      No     Yes   \n",
       "1    2  106.025   6645     483      3   82         15  Female     Yes     Yes   \n",
       "2    3  104.593   7075     514      4   71         11    Male      No      No   \n",
       "3    4  148.924   9504     681      3   36         11  Female      No      No   \n",
       "4    5   55.882   4897     357      2   68         16    Male      No     Yes   \n",
       "\n",
       "   Ethnicity  Balance  \n",
       "0  Caucasian      333  \n",
       "1      Asian      903  \n",
       "2      Asian      580  \n",
       "3      Asian      964  \n",
       "4  Caucasian      331  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Obs</th>\n",
       "      <th>Income</th>\n",
       "      <th>Limit</th>\n",
       "      <th>Rating</th>\n",
       "      <th>Cards</th>\n",
       "      <th>Age</th>\n",
       "      <th>Education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Student</th>\n",
       "      <th>Married</th>\n",
       "      <th>Ethnicity</th>\n",
       "      <th>Balance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>395</th>\n",
       "      <td>396</td>\n",
       "      <td>12.096</td>\n",
       "      <td>4100</td>\n",
       "      <td>307</td>\n",
       "      <td>3</td>\n",
       "      <td>32</td>\n",
       "      <td>13</td>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Caucasian</td>\n",
       "      <td>560</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>396</th>\n",
       "      <td>397</td>\n",
       "      <td>13.364</td>\n",
       "      <td>3838</td>\n",
       "      <td>296</td>\n",
       "      <td>5</td>\n",
       "      <td>65</td>\n",
       "      <td>17</td>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>African American</td>\n",
       "      <td>480</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>397</th>\n",
       "      <td>398</td>\n",
       "      <td>57.872</td>\n",
       "      <td>4171</td>\n",
       "      <td>321</td>\n",
       "      <td>5</td>\n",
       "      <td>67</td>\n",
       "      <td>12</td>\n",
       "      <td>Female</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Caucasian</td>\n",
       "      <td>138</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>398</th>\n",
       "      <td>399</td>\n",
       "      <td>37.728</td>\n",
       "      <td>2525</td>\n",
       "      <td>192</td>\n",
       "      <td>1</td>\n",
       "      <td>44</td>\n",
       "      <td>13</td>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Caucasian</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>399</th>\n",
       "      <td>400</td>\n",
       "      <td>18.701</td>\n",
       "      <td>5524</td>\n",
       "      <td>415</td>\n",
       "      <td>5</td>\n",
       "      <td>64</td>\n",
       "      <td>7</td>\n",
       "      <td>Female</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Asian</td>\n",
       "      <td>966</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Obs  Income  Limit  Rating  Cards  Age  Education  Gender Student  \\\n",
       "395  396  12.096   4100     307      3   32         13    Male      No   \n",
       "396  397  13.364   3838     296      5   65         17    Male      No   \n",
       "397  398  57.872   4171     321      5   67         12  Female      No   \n",
       "398  399  37.728   2525     192      1   44         13    Male      No   \n",
       "399  400  18.701   5524     415      5   64          7  Female      No   \n",
       "\n",
       "    Married         Ethnicity  Balance  \n",
       "395     Yes         Caucasian      560  \n",
       "396      No  African American      480  \n",
       "397     Yes         Caucasian      138  \n",
       "398     Yes         Caucasian        0  \n",
       "399      No             Asian      966  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Obs</th>\n",
       "      <th>Income</th>\n",
       "      <th>Limit</th>\n",
       "      <th>Rating</th>\n",
       "      <th>Cards</th>\n",
       "      <th>Age</th>\n",
       "      <th>Education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Student</th>\n",
       "      <th>Married</th>\n",
       "      <th>Ethnicity</th>\n",
       "      <th>Balance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>14.891</td>\n",
       "      <td>3606</td>\n",
       "      <td>283</td>\n",
       "      <td>2</td>\n",
       "      <td>34</td>\n",
       "      <td>11</td>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Caucasian</td>\n",
       "      <td>333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>106.025</td>\n",
       "      <td>6645</td>\n",
       "      <td>483</td>\n",
       "      <td>3</td>\n",
       "      <td>82</td>\n",
       "      <td>15</td>\n",
       "      <td>Female</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Asian</td>\n",
       "      <td>903</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>104.593</td>\n",
       "      <td>7075</td>\n",
       "      <td>514</td>\n",
       "      <td>4</td>\n",
       "      <td>71</td>\n",
       "      <td>11</td>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Asian</td>\n",
       "      <td>580</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Obs   Income  Limit  Rating  Cards  Age  Education  Gender Student Married  \\\n",
       "0    1   14.891   3606     283      2   34         11    Male      No     Yes   \n",
       "1    2  106.025   6645     483      3   82         15  Female     Yes     Yes   \n",
       "2    3  104.593   7075     514      4   71         11    Male      No      No   \n",
       "\n",
       "   Ethnicity  Balance  \n",
       "0  Caucasian      333  \n",
       "1      Asian      903  \n",
       "2      Asian      580  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The rows of our `DataFrame` have a numerical index (in bold above), which is the default behaviour. An important detail, if you are not used to Python or some other programming languages, is that the index starts from zero instead of one. **Numerical indexes start from zero in Python**. This does not need to be the case in the `DataFrame`, but pandas follows the Python convention by default.\n",
    "\n",
    "Alternatively, we can specify the `DataFrame` index (that is, a label for each row), which does not need to be a number. For example, if you have time series data, it can be the date.  \n",
    "\n",
    "In our case we can see that the first column is an observation index, so that we could specify that this is the case when reading the data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Income</th>\n",
       "      <th>Limit</th>\n",
       "      <th>Rating</th>\n",
       "      <th>Cards</th>\n",
       "      <th>Age</th>\n",
       "      <th>Education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Student</th>\n",
       "      <th>Married</th>\n",
       "      <th>Ethnicity</th>\n",
       "      <th>Balance</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Obs</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>14.891</td>\n",
       "      <td>3606</td>\n",
       "      <td>283</td>\n",
       "      <td>2</td>\n",
       "      <td>34</td>\n",
       "      <td>11</td>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Caucasian</td>\n",
       "      <td>333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>106.025</td>\n",
       "      <td>6645</td>\n",
       "      <td>483</td>\n",
       "      <td>3</td>\n",
       "      <td>82</td>\n",
       "      <td>15</td>\n",
       "      <td>Female</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Asian</td>\n",
       "      <td>903</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>104.593</td>\n",
       "      <td>7075</td>\n",
       "      <td>514</td>\n",
       "      <td>4</td>\n",
       "      <td>71</td>\n",
       "      <td>11</td>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Asian</td>\n",
       "      <td>580</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>148.924</td>\n",
       "      <td>9504</td>\n",
       "      <td>681</td>\n",
       "      <td>3</td>\n",
       "      <td>36</td>\n",
       "      <td>11</td>\n",
       "      <td>Female</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Asian</td>\n",
       "      <td>964</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>55.882</td>\n",
       "      <td>4897</td>\n",
       "      <td>357</td>\n",
       "      <td>2</td>\n",
       "      <td>68</td>\n",
       "      <td>16</td>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Caucasian</td>\n",
       "      <td>331</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Income  Limit  Rating  Cards  Age  Education  Gender Student Married  \\\n",
       "Obs                                                                          \n",
       "1     14.891   3606     283      2   34         11    Male      No     Yes   \n",
       "2    106.025   6645     483      3   82         15  Female     Yes     Yes   \n",
       "3    104.593   7075     514      4   71         11    Male      No      No   \n",
       "4    148.924   9504     681      3   36         11  Female      No      No   \n",
       "5     55.882   4897     357      2   68         16    Male      No     Yes   \n",
       "\n",
       "     Ethnicity  Balance  \n",
       "Obs                      \n",
       "1    Caucasian      333  \n",
       "2        Asian      903  \n",
       "3        Asian      580  \n",
       "4        Asian      964  \n",
       "5    Caucasian      331  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data=pd.read_csv('Data/Credit.csv', index_col='Obs')\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pandas has specialised functions for reading other types of input, such as Excel files. You can see a list of available functions <a href=\"http://pandas.pydata.org/pandas-docs/stable/io.html\" target=\"_blank\">here</a>. The pandas  <a href=\"http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_table.html#pandas.read_table\" target=\"_blank\">read table</a> function reads data stored as general delimited text files (for example, where the columns are separated by space rather than commas).  In practical business situations, you may often need to obtain data from a [relational database](https://en.wikipedia.org/wiki/Relational_database) rather thah having to load a flat file stored in your computer. You can read database queries and tables input into a `DataFrame` by using use the `read_sql_table`, `read_sql_query`, or `read_sql`  functions.  \n",
    "\n",
    "Our dataset here is simple to work with, but others may require customising the function call. Refer to the documentation for finding the appropriate options for other data that you come across. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Data selection\n",
    "\n",
    "There are two ways to select data in pandas: by providing the column and index labels or by providing a numerical index. \n",
    "\n",
    "**Selecting a column by label**\n",
    "\n",
    "The output will now look different because selecting only one column returns a `Series` (a specialised object for when there is only one column of data) rather than a `DataFrame`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Obs\n",
       "1     14.891\n",
       "2    106.025\n",
       "Name: Income, dtype: float64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['Income'].head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Selecting multiple columns by label**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Income</th>\n",
       "      <th>Education</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Obs</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>14.891</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>106.025</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Income  Education\n",
       "Obs                    \n",
       "1     14.891         11\n",
       "2    106.025         15"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[['Income','Education']].head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, the inner brackets are used to indicate that we are passing a list of column names. The example will make this clear, and is a useful template for some of what we will do. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Income', 'Education']\n"
     ]
    }
   ],
   "source": [
    "names=['Income','Education']\n",
    "print(names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Income</th>\n",
       "      <th>Education</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Obs</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>14.891</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>106.025</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Income  Education\n",
       "Obs                    \n",
       "1     14.891         11\n",
       "2    106.025         15"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[names].head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Selecting a column by a numerical index**\n",
    "\n",
    "The `iloc` method allows us to select data by numerical indexes. We just have to be careful not be confused by zero indexing. If want the first column then the index needs to be zero. The following is equivalent to what we did above. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Obs\n",
       "1     14.891\n",
       "2    106.025\n",
       "Name: Income, dtype: float64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.iloc[:,0].head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `:` syntax indicates that we want to include all rows in the selection. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Selecting multiple columns by numerical indexes**\n",
    "\n",
    "Here, we pass a list of column numbers for indexing. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Income</th>\n",
       "      <th>Education</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Obs</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>14.891</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>106.025</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Income  Education\n",
       "Obs                    \n",
       "1     14.891         11\n",
       "2    106.025         15"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.iloc[:,[0,5]].head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another method is *slicing*. Suppose that we want to select the data from the 1st to the 6th column.  When specifying a range of integer indexes, the last one does not count. This may be initially confusing, but it's the standard Python syntax with logic behind it. What the cell below does is to request indexes 0, 1, 2, 3, 4, 5, which correspond to columns 1-6. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Income</th>\n",
       "      <th>Limit</th>\n",
       "      <th>Rating</th>\n",
       "      <th>Cards</th>\n",
       "      <th>Age</th>\n",
       "      <th>Education</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Obs</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>14.891</td>\n",
       "      <td>3606</td>\n",
       "      <td>283</td>\n",
       "      <td>2</td>\n",
       "      <td>34</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>106.025</td>\n",
       "      <td>6645</td>\n",
       "      <td>483</td>\n",
       "      <td>3</td>\n",
       "      <td>82</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Income  Limit  Rating  Cards  Age  Education\n",
       "Obs                                               \n",
       "1     14.891   3606     283      2   34         11\n",
       "2    106.025   6645     483      3   82         15"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.iloc[:,0:6].head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Selecting rows by labels**\n",
    "\n",
    "The `loc` method alows one to select rows by the designated index labels (`Obs`). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Income</th>\n",
       "      <th>Limit</th>\n",
       "      <th>Rating</th>\n",
       "      <th>Cards</th>\n",
       "      <th>Age</th>\n",
       "      <th>Education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Student</th>\n",
       "      <th>Married</th>\n",
       "      <th>Ethnicity</th>\n",
       "      <th>Balance</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Obs</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>14.891</td>\n",
       "      <td>3606</td>\n",
       "      <td>283</td>\n",
       "      <td>2</td>\n",
       "      <td>34</td>\n",
       "      <td>11</td>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Caucasian</td>\n",
       "      <td>333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>106.025</td>\n",
       "      <td>6645</td>\n",
       "      <td>483</td>\n",
       "      <td>3</td>\n",
       "      <td>82</td>\n",
       "      <td>15</td>\n",
       "      <td>Female</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Asian</td>\n",
       "      <td>903</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>55.882</td>\n",
       "      <td>4897</td>\n",
       "      <td>357</td>\n",
       "      <td>2</td>\n",
       "      <td>68</td>\n",
       "      <td>16</td>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Caucasian</td>\n",
       "      <td>331</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Income  Limit  Rating  Cards  Age  Education  Gender Student Married  \\\n",
       "Obs                                                                          \n",
       "1     14.891   3606     283      2   34         11    Male      No     Yes   \n",
       "2    106.025   6645     483      3   82         15  Female     Yes     Yes   \n",
       "5     55.882   4897     357      2   68         16    Male      No     Yes   \n",
       "\n",
       "     Ethnicity  Balance  \n",
       "Obs                      \n",
       "1    Caucasian      333  \n",
       "2        Asian      903  \n",
       "5    Caucasian      331  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.loc[[1,2,5],:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unlike slicing with numerical indexing, slicing with label indexes includes the last item. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Income</th>\n",
       "      <th>Limit</th>\n",
       "      <th>Rating</th>\n",
       "      <th>Cards</th>\n",
       "      <th>Age</th>\n",
       "      <th>Education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Student</th>\n",
       "      <th>Married</th>\n",
       "      <th>Ethnicity</th>\n",
       "      <th>Balance</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Obs</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>14.891</td>\n",
       "      <td>3606</td>\n",
       "      <td>283</td>\n",
       "      <td>2</td>\n",
       "      <td>34</td>\n",
       "      <td>11</td>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Caucasian</td>\n",
       "      <td>333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>106.025</td>\n",
       "      <td>6645</td>\n",
       "      <td>483</td>\n",
       "      <td>3</td>\n",
       "      <td>82</td>\n",
       "      <td>15</td>\n",
       "      <td>Female</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Asian</td>\n",
       "      <td>903</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Income  Limit  Rating  Cards  Age  Education  Gender Student Married  \\\n",
       "Obs                                                                          \n",
       "1     14.891   3606     283      2   34         11    Male      No     Yes   \n",
       "2    106.025   6645     483      3   82         15  Female     Yes     Yes   \n",
       "\n",
       "     Ethnicity  Balance  \n",
       "Obs                      \n",
       "1    Caucasian      333  \n",
       "2        Asian      903  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.loc[1:2,:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Selecting rows by numerical index**\n",
    "\n",
    "This is useful when the index variable is a string or date. Here we are back to zero indexing. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Income</th>\n",
       "      <th>Limit</th>\n",
       "      <th>Rating</th>\n",
       "      <th>Cards</th>\n",
       "      <th>Age</th>\n",
       "      <th>Education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Student</th>\n",
       "      <th>Married</th>\n",
       "      <th>Ethnicity</th>\n",
       "      <th>Balance</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Obs</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>14.891</td>\n",
       "      <td>3606</td>\n",
       "      <td>283</td>\n",
       "      <td>2</td>\n",
       "      <td>34</td>\n",
       "      <td>11</td>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Caucasian</td>\n",
       "      <td>333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>106.025</td>\n",
       "      <td>6645</td>\n",
       "      <td>483</td>\n",
       "      <td>3</td>\n",
       "      <td>82</td>\n",
       "      <td>15</td>\n",
       "      <td>Female</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Asian</td>\n",
       "      <td>903</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Income  Limit  Rating  Cards  Age  Education  Gender Student Married  \\\n",
       "Obs                                                                          \n",
       "1     14.891   3606     283      2   34         11    Male      No     Yes   \n",
       "2    106.025   6645     483      3   82         15  Female     Yes     Yes   \n",
       "\n",
       "     Ethnicity  Balance  \n",
       "Obs                      \n",
       "1    Caucasian      333  \n",
       "2        Asian      903  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.iloc[0:2,:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Jointly selecting rows and columns**\n",
    "\n",
    "We can combine the previous examples to simultaneously select specific rows and columns. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Income</th>\n",
       "      <th>Education</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Obs</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>14.891</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>106.025</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Income  Education\n",
       "Obs                    \n",
       "1     14.891         11\n",
       "2    106.025         15"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.loc[1:2,['Income', 'Education']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Income</th>\n",
       "      <th>Education</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Obs</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>14.891</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>106.025</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Income  Education\n",
       "Obs                    \n",
       "1     14.891         11\n",
       "2    106.025         15"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.iloc[0:2,[0,5]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Combining labels and numerical indexes**\n",
    "\n",
    "As a more advanced concept, a slice of a `DataFrame` is itself a pandas object (a `DataFrame` if the slice has multiple columns, or a `Series` if it has only one). That means that we can chain the operations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Income</th>\n",
       "      <th>Education</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Obs</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>14.891</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>106.025</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Income  Education\n",
       "Obs                    \n",
       "1     14.891         11\n",
       "2    106.025         15"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[['Income', 'Education']].iloc[0:2,:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Conditional selection** \n",
    "\n",
    "Suppose that we want to know the sample average credit card balance only for females. Below, we select the balance column and the rows such that the value of the gender column is female. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "529.536231884058"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.loc[data['Gender']=='Female','Balance'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is called Boolean indexing in Python, because it involves the creation of binary variables indicating whether the condition is true of false for each row. The next cell will help you to understand this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Obs\n",
      "1      Male\n",
      "2    Female\n",
      "Name: Gender, dtype: object\n",
      "Obs\n",
      "1    False\n",
      "2     True\n",
      "Name: Gender, dtype: bool\n"
     ]
    }
   ],
   "source": [
    "print(data['Gender'].head(2))\n",
    "print(data['Gender'].head(2)=='Female')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also specify multiple conditions. The following selects males with age equal or lower than 30.  You can look at this <a href=\"http://www.tutorialspoint.com/python/python_basic_operators.htm\" target=\"_blank\">reference</a> for a list of Python comparison operators. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "499.59090909090907"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.loc[(data['Gender']=='Female') & (data['Age']<=30),'Balance'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Assigning new values to a data selection** \n",
    "\n",
    "You may have noted that the first character in \"Male\" above is a space. This is an unintentional error in the data. The <a href=\"http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.unique.html\" target=\"_blank\">`unique`</a> method in pandas allows us to view all the unique values in a column. In this case, it confirms that all entries are like this.   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([' Male', 'Female'], dtype=object)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['Gender'].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use our data selection knowledge to fix this. Below, we joinly select the rows in which the gender is \" Male\" and the Gender column.  We then replace the values in those locations with the correct label. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Income</th>\n",
       "      <th>Limit</th>\n",
       "      <th>Rating</th>\n",
       "      <th>Cards</th>\n",
       "      <th>Age</th>\n",
       "      <th>Education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>Student</th>\n",
       "      <th>Married</th>\n",
       "      <th>Ethnicity</th>\n",
       "      <th>Balance</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Obs</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>14.891</td>\n",
       "      <td>3606</td>\n",
       "      <td>283</td>\n",
       "      <td>2</td>\n",
       "      <td>34</td>\n",
       "      <td>11</td>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Caucasian</td>\n",
       "      <td>333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>106.025</td>\n",
       "      <td>6645</td>\n",
       "      <td>483</td>\n",
       "      <td>3</td>\n",
       "      <td>82</td>\n",
       "      <td>15</td>\n",
       "      <td>Female</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Asian</td>\n",
       "      <td>903</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Income  Limit  Rating  Cards  Age  Education  Gender Student Married  \\\n",
       "Obs                                                                          \n",
       "1     14.891   3606     283      2   34         11    Male      No     Yes   \n",
       "2    106.025   6645     483      3   82         15  Female     Yes     Yes   \n",
       "\n",
       "     Ethnicity  Balance  \n",
       "Obs                      \n",
       "1    Caucasian      333  \n",
       "2        Asian      903  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.loc[data['Gender']==' Male','Gender']='Male'\n",
    "data.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. Exporting data\n",
    "\n",
    "Once you have made the necessary modications to the dataset, you may want to save it to continue working on it later. More generally, you may wish to save the results of your analysis or export tables so that they insert them in a report or webpage (after formatting). Pandas has methods to export data as <a href=\"http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.to_csv.html\" target=\"_blank\">csv</a> and <a href=\"http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.to_excel.html\" target=\"_blank\">Excel</a> files, <a href=\"http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.to_latex.html\" target=\"_blank\">LaTex</a> and <a href=\"http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.to_html.html\" target=\"_blank\">HTML</a> tables, among <a href='http://pandas.pydata.org/pandas-docs/stable/api.html#id12' target=\"_blank\">other options</a>.\n",
    "\n",
    "In the following example I export our <TT>DataFrame</TT> as an Excel file. You can run the cell and try to open the file in Excel to check that it worked. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.to_excel('new_data.xlsx')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6. Descriptive statistics\n",
    "\n",
    "After loading and preparing the data, we can start exploring it by looking at the basic descriptive statitistics. The `describe` method provides a table with basic summary statitics for the data.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Income</th>\n",
       "      <th>Limit</th>\n",
       "      <th>Rating</th>\n",
       "      <th>Cards</th>\n",
       "      <th>Age</th>\n",
       "      <th>Education</th>\n",
       "      <th>Balance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>400.0</td>\n",
       "      <td>400.0</td>\n",
       "      <td>400.0</td>\n",
       "      <td>400.0</td>\n",
       "      <td>400.0</td>\n",
       "      <td>400.0</td>\n",
       "      <td>400.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>45.2</td>\n",
       "      <td>4735.6</td>\n",
       "      <td>354.9</td>\n",
       "      <td>3.0</td>\n",
       "      <td>55.7</td>\n",
       "      <td>13.4</td>\n",
       "      <td>520.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>35.2</td>\n",
       "      <td>2308.2</td>\n",
       "      <td>154.7</td>\n",
       "      <td>1.4</td>\n",
       "      <td>17.2</td>\n",
       "      <td>3.1</td>\n",
       "      <td>459.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>10.4</td>\n",
       "      <td>855.0</td>\n",
       "      <td>93.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>23.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>21.0</td>\n",
       "      <td>3088.0</td>\n",
       "      <td>247.2</td>\n",
       "      <td>2.0</td>\n",
       "      <td>41.8</td>\n",
       "      <td>11.0</td>\n",
       "      <td>68.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>33.1</td>\n",
       "      <td>4622.5</td>\n",
       "      <td>344.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>459.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>57.5</td>\n",
       "      <td>5872.8</td>\n",
       "      <td>437.2</td>\n",
       "      <td>4.0</td>\n",
       "      <td>70.0</td>\n",
       "      <td>16.0</td>\n",
       "      <td>863.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>186.6</td>\n",
       "      <td>13913.0</td>\n",
       "      <td>982.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>98.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>1999.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Income    Limit  Rating  Cards    Age  Education  Balance\n",
       "count   400.0    400.0   400.0  400.0  400.0      400.0    400.0\n",
       "mean     45.2   4735.6   354.9    3.0   55.7       13.4    520.0\n",
       "std      35.2   2308.2   154.7    1.4   17.2        3.1    459.8\n",
       "min      10.4    855.0    93.0    1.0   23.0        5.0      0.0\n",
       "25%      21.0   3088.0   247.2    2.0   41.8       11.0     68.8\n",
       "50%      33.1   4622.5   344.0    3.0   56.0       14.0    459.5\n",
       "75%      57.5   5872.8   437.2    4.0   70.0       16.0    863.0\n",
       "max     186.6  13913.0   982.0    9.0   98.0       20.0   1999.0"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.describe().round(1) # here I appended round to limit the number of decimal places in the display, try without it"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are also individual functions for a range of summary statistics. Refer to the [pandas documentation](http://pandas.pydata.org/pandas-docs/stable/api.html#api-dataframe-stats) for a full list of available functions. As examples, we calculate the means of the dataset and the correlation between income and credit card limit. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Income         45.22\n",
       "Limit        4735.60\n",
       "Rating        354.94\n",
       "Cards           2.96\n",
       "Age            55.67\n",
       "Education      13.45\n",
       "Balance       520.02\n",
       "dtype: float64"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.mean().round(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Income</th>\n",
       "      <th>Limit</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Income</th>\n",
       "      <td>1.00</td>\n",
       "      <td>0.79</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Limit</th>\n",
       "      <td>0.79</td>\n",
       "      <td>1.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Income  Limit\n",
       "Income    1.00   0.79\n",
       "Limit     0.79   1.00"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[['Income','Limit']].corr().round(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7. Viewing column types\n",
    "\n",
    "When preparing complex datasets for analysis, it is often useful to work with different types of variables (say, numerical or categorical) separately. You can use the dataframe `info` method to view the list of columns and their variable types, which `pandas` calls `dtypes`. The `object` dtype is the most general type: a column with this dtype will typically contain text data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 400 entries, 1 to 400\n",
      "Data columns (total 11 columns):\n",
      "Income       400 non-null float64\n",
      "Limit        400 non-null int64\n",
      "Rating       400 non-null int64\n",
      "Cards        400 non-null int64\n",
      "Age          400 non-null int64\n",
      "Education    400 non-null int64\n",
      "Gender       400 non-null object\n",
      "Student      400 non-null object\n",
      "Married      400 non-null object\n",
      "Ethnicity    400 non-null object\n",
      "Balance      400 non-null int64\n",
      "dtypes: float64(1), int64(6), object(4)\n",
      "memory usage: 57.5+ KB\n"
     ]
    }
   ],
   "source": [
    "data.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To select columns of a particular type, you can do as follows: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Gender</th>\n",
       "      <th>Student</th>\n",
       "      <th>Married</th>\n",
       "      <th>Ethnicity</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Obs</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Caucasian</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Female</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Asian</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Asian</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Female</td>\n",
       "      <td>No</td>\n",
       "      <td>No</td>\n",
       "      <td>Asian</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Male</td>\n",
       "      <td>No</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Caucasian</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Gender Student Married  Ethnicity\n",
       "Obs                                   \n",
       "1      Male      No     Yes  Caucasian\n",
       "2    Female     Yes     Yes      Asian\n",
       "3      Male      No      No      Asian\n",
       "4    Female      No      No      Asian\n",
       "5      Male      No     Yes  Caucasian"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text_variables=data.select_dtypes(['object'])\n",
    "text_variables.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>.CodeMirror pre {font-family: monospace; font-size: 12pt; line-height: 22.4px;}\n",
       "\n",
       "pre {font-family: monospace; font-size: 12pt; line-height: 22.4px;}\n",
       "\n",
       ".output_area pre {font-family: monospace; font-size: 12pt; line-height: 22.4px;}\n",
       "\n",
       ".rendered_html table {font-size: 10.5pt;}\n",
       "\n",
       "h1 {\n",
       "  font-family: 'Open Sans' !important;\n",
       "  color: #222;\n",
       "  font-size: 2.5em !important;\n",
       "  padding-bottom: 0px;  \n",
       "  margin-top: 14px;\n",
       "  margin-bottom: 6px;\n",
       "  font-weight: 800 !important;\n",
       "}\n",
       "\n",
       "\n",
       "h2 {\n",
       "  font-family: 'Open Sans' !important;\n",
       "  font-size: 1.6em !important;\n",
       "  font-weight: 800 !important;\n",
       "  color: #222;\n",
       "}\n",
       "\n",
       "\n",
       "h3 {\n",
       "  font-family: 'Open Sans' !important;\n",
       "  font-size: 25px !important;\n",
       "  color: #222 !important;\n",
       "  margin-bottom: 30px !important;\n",
       "  margin-top: 25px !important;\n",
       "}\n",
       "\n",
       "\n",
       "p {\n",
       "  font-family: 'Source Sans Pro', sans-serif;\n",
       "  color: #222;\n",
       "  font-size: 19px;\n",
       "  margin-bottom: 10px;\n",
       "  margin-top:0;\n",
       "  line-height: 1.4;\n",
       "}\n",
       "\n",
       "li {\n",
       "  font-family: 'Source Sans Pro', sans-serif;\n",
       "  color: #222;\n",
       "  font-size: 19px;\n",
       "  line-height: 1.4;\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "/*code {\n",
       "     padding: 2px 4px !important; \n",
       "     font-size: 90% !important;\n",
       "     color: #c7254e !important; \n",
       "     background-color: #f9f2f4 !important; \n",
       "     border-radius: 2px !important;\n",
       "     text-decoration: none !important;\n",
       "      font-size: 90% !important;\n",
       "}\n",
       "*/\n",
       "\n",
       "\n",
       "code {\n",
       "      font-size: 90% !important;\n",
       "/*      border-style: solid !important;\n",
       "      border-width: 1px !important;\n",
       "      border-color :#cccccc !important;*/\n",
       "}\n",
       "\n",
       "TT {\n",
       "      font-family: monospace;\n",
       "      font-size: 95% !important;\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "a.neutral {\n",
       "  text-decoration: none !important;\n",
       "  border-style: dotted !important;\n",
       "  border-width: 1px !important;\n",
       "  border-color :#cccccc !important;\n",
       "\n",
       "}\n",
       "\n",
       "</style>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# This cells formats the notebook for display online.  Please omit it from your work.  \n",
    "from IPython.core.display import HTML\n",
    "style = open('jstyle.css', \"r\").read()\n",
    "HTML('<style>'+ style +'</style>')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}