3_IntroToPython.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "A6Megjh0hi-e"
   },
   "source": [
    "# An Introduction to Python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "T_0_ySXWhi-k"
   },
   "source": [
    "The following jupyter notebook serves as a basic introduction to python for use in analytics (statistics, machine learning, data science, data analytics, business analytics, etc.) applications. Python is a very general language, so this introduction does nothing to even scratch the surface of the capabilities of the language. We introduce python datatypes, defining objects, subsetting and slicing, and then introduce `pandas` and `numpy` as libraries which allow us to conveniently interface with and operate on data frames.\n",
    "\n",
    "We'll typically make use of these libraries as well as a few others -- in QSO370/QSO570 the following Python libraries will be used quite often:\n",
    "+ `pandas` for reading and manipulating data frames\n",
    "+ `numpy` for numerical operations on vectors/columns\n",
    "+ `matplotlib` and `seaborn` for data visualization (I'll provide a short primer)\n",
    "+ `sklearn` for predictive modeling"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WC2aitL9hi-l"
   },
   "source": [
    "### Python Basics: Objects\r\n",
    "\r\n",
    "We define objects/variables with the `=` operator in Python. Python interprets type via the value stored in the object. We can run a block of code in a jupyter notebook using `shift+Enter`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "executionInfo": {
     "elapsed": 946,
     "status": "ok",
     "timestamp": 1611083083013,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "xw-V1URwhi-n"
   },
   "outputs": [],
   "source": [
    "w = True\n",
    "x = 2\n",
    "y = 2.0\n",
    "z = \"Hello\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 1066,
     "status": "ok",
     "timestamp": 1611083164873,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "wUNMGuO8hi-s",
    "outputId": "329cee10-c308-48a2-b6fd-0df752077596"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "str"
      ]
     },
     "execution_count": 4,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(w)\n",
    "type(x)\n",
    "type(y)\n",
    "type(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TSSkxqi_hi-w"
   },
   "source": [
    "By default, Python only prints the result of the last statement from a code block. We use the `print()` function to force Python to print additional items. Try removing the calls to `print()` in the block above and re-executing the cell. What happens?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MCSZRCuAhi-x"
   },
   "source": [
    "### Special Objects: Lists and Dictionaries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HwYjclTmhi-y"
   },
   "source": [
    "Beyond the basic boolean (true/false), numeric, and string datatypes Python also has lists (ordered containers of objects), dictionaries, arrays, and data frames. We will most often be working with data frames (tables) and lists, but it is worth seeing those other datatypes too since they can be useful."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 581,
     "status": "ok",
     "timestamp": 1611083283463,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "03-ufvtshi-y",
    "outputId": "298baf6c-4905-4c2a-addf-ba21595da3ee"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'list'>\n",
      "<class 'list'>\n"
     ]
    }
   ],
   "source": [
    "#A list\n",
    "x = [1, 2, 4, 99]\n",
    "#Another (more complicated) list\n",
    "y = [\"a\", True, 45, 22, [1, 9, 3]]\n",
    "print(type(x))\n",
    "print(type(y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dKFtcX-7hi-1"
   },
   "source": [
    "Note that both `x` and `y` above are lists, but they are quite different from one another. We can easily see that `x` is a list of four numeric values but `y` is a list of disparate objects. The first entry is a string, the second is a boolean, the third and fourth are numerics, and the fifth is a list itself! We can use subsetting to confirm this. There are a few things to keep in mind when referencing indices in python.\n",
    "\n",
    "+ Indexing in python begins from 0 -- that is, the first item in these lists occupies slot 0, the second item occupies slot 1, etc.\n",
    "+ We can reference a range of indices using the colon (`:`) operator -- for example, `1:4` but the colon operator is right endpoint exclusive. That is, if we request `x[1:4]` we will get the objects in slots 1, 2, and 3 of the list `x`. Remember, these are the second, third, and fourth items. This is indeed confusing, but it will be natural soon."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 1182,
     "status": "ok",
     "timestamp": 1611083364051,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "JD5Mm6liDL6W",
    "outputId": "7b58a235-ec43-49e6-84a9-1c400cf97060"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['a', True, 45, 22, [1, 9, 3]]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 1069,
     "status": "ok",
     "timestamp": 1611083357765,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "gnOQO1vmpdZW",
    "outputId": "c1a148d8-d433-4671-e237-1e83617ece26"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "99"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 603,
     "status": "ok",
     "timestamp": 1611083428010,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "GK7yiMxBp27i",
    "outputId": "c903ed9a-340c-48c8-8bde-a63e56ec3831"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[True, 45, 22]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y[1:4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 657,
     "status": "ok",
     "timestamp": 1611083505929,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "1bGsHeWehi-2",
    "outputId": "8676ff94-96a2-4715-c936-1a4a1340fac0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[True, 45, 22]\n",
      "<class 'str'>\n",
      "<class 'list'>\n"
     ]
    }
   ],
   "source": [
    "print(y[1:4])\n",
    "print(type(y[0]))\n",
    "print(type(y[4]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 362,
     "status": "ok",
     "timestamp": 1611083521705,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "HxmZQNqwD5PG",
    "outputId": "2d4379e1-b34e-421c-d4dd-3a45824f9b05"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['a', True, 45, 22, [1, 9, 3]]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 617,
     "status": "ok",
     "timestamp": 1611083687102,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "xEAcw8vxD6TR",
    "outputId": "be4b17c8-e3b0-40e9-ad9f-166468bbbd3b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y[4][2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8xaWiUFGhi-7"
   },
   "source": [
    "Notice that, indeed, the last item of the list `y` is indeed a list! Now, what would happen if we requested `y[4][1]`? Would this work if we replaced the `4` by a `3`? Try it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "Pg1fhPFjhi-9"
   },
   "outputs": [],
   "source": [
    "#Use this cell to explore...\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cqI0f9cDsGxj"
   },
   "source": [
    "We can also subset beginning at the start of a list using ` :n`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 691,
     "status": "ok",
     "timestamp": 1611083776142,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "hW8Ysoh2rgSj",
    "outputId": "6db1fc3a-667f-4cf6-fa75-c4b154bc907b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['a', True, 45]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y[:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5A8aWMKNs9tq"
   },
   "source": [
    "Adding list does not necessarily work in the way we would expect."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 629,
     "status": "ok",
     "timestamp": 1611083809374,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "YTP2MfOctCh6",
    "outputId": "a855ca41-61e5-43bc-dc69-783b426e444f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 2, 4, 5, 9, 10]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[1, 2, 4] + [5, 9, 10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "executionInfo": {
     "elapsed": 481,
     "status": "ok",
     "timestamp": 1579268164090,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AAuE7mDQ8K_t2bmWgnSB3UWY2INXq568pNYg09t4sKd8=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "3z_RTUz-tnbY",
    "outputId": "6aa1fe77-b103-4210-cf99-8b3b3285e83e"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 2, 4, 5, 9, 10, 18, 20]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = [1, 2, 4] + [5, 9, 10, 18, 20]\n",
    "sorted(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-T2NTT0Phi_B"
   },
   "source": [
    "Dictionaries are similar to lists in that they are ordered containers for objects. They get us closer to data frames, however, because each object has a name with which we can reference it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 879,
     "status": "ok",
     "timestamp": 1611083960026,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "rkiKvpyQhi_C",
    "outputId": "e45890aa-8d28-4576-82df-15621fd9e53d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'State': 'NH', 'School': ['Plymouth', 'UNH', 'Keene', 'SNHU', 'Franklin Pierce', \"St. Anselm's\", 'Dartmouth', 'Colby-Sawyer', 'New England College', 'Rivier'], 'Accredited': [True, True, True, True, True, True, True, True, True, True]}\n"
     ]
    }
   ],
   "source": [
    "#A simple dictionary\n",
    "colleges = {\"State\" : \"NH\",\n",
    "            \"School\" : [\"Plymouth\", \"UNH\", \"Keene\", \"SNHU\", \"Franklin Pierce\", \"St. Anselm's\", \"Dartmouth\", \"Colby-Sawyer\", \"New England College\", \"Rivier\"],\n",
    "            \"Accredited\" : 10*[True]}\n",
    "print(colleges)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ILrr0e7lhi_G"
   },
   "source": [
    "The code above creates a dictionary with slots for `State`, `Schoool` name, and whether or not the school is `Accredited`. Note that dictionaries can have slots storing objects of different lengths (which is not allowed for data frames), and we can define what goes into those slots in a variety of ways. A nice shortcut was used to create the `Accredited` slot though -- multiplying a list by a number extends that list by copying it that number of times!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GLsZughrhi_H"
   },
   "source": [
    "### Data Frames and `pandas`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Fk3Tlhuqhi_J"
   },
   "source": [
    "This section begins our discussion of Python for analytics. You'll have a fourth notebook which re-covers some of this material and takes a more purposeful look at troubleshooting common errors we'll encounter.\r\n",
    "\r\n",
    "The `pandas` library is really useful for working with and creating data frames. It will be one of the libraries that we use most often in QSO370. In order to make use of a library in python we must first `import` it. Furthermore, anytime we wish to use a feature of that library, we must tell python the library name -- since this is the case, we often provide a shortened reference name with `as`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "executionInfo": {
     "elapsed": 733,
     "status": "ok",
     "timestamp": 1611084008626,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "WCj_1Hnzhi_L"
   },
   "outputs": [],
   "source": [
    "#import the pandas library with shortened reference name pd\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "phlgj_2hhi_N"
   },
   "source": [
    "Using `pd` in reference to the `pandas` library is standard in the Python community. Now let's use `pandas` to create a data frame from our `colleges` dictionary!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 359
    },
    "executionInfo": {
     "elapsed": 659,
     "status": "ok",
     "timestamp": 1611084051112,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "Wr6TzncZhi_O",
    "outputId": "5e86c42e-876d-4e1c-a19d-4731b1ca07b6"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>State</th>\n",
       "      <th>School</th>\n",
       "      <th>Accredited</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NH</td>\n",
       "      <td>Plymouth</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NH</td>\n",
       "      <td>UNH</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NH</td>\n",
       "      <td>Keene</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NH</td>\n",
       "      <td>SNHU</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NH</td>\n",
       "      <td>Franklin Pierce</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>NH</td>\n",
       "      <td>St. Anselm's</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NH</td>\n",
       "      <td>Dartmouth</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>NH</td>\n",
       "      <td>Colby-Sawyer</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>NH</td>\n",
       "      <td>New England College</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>NH</td>\n",
       "      <td>Rivier</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  State               School  Accredited\n",
       "0    NH             Plymouth        True\n",
       "1    NH                  UNH        True\n",
       "2    NH                Keene        True\n",
       "3    NH                 SNHU        True\n",
       "4    NH      Franklin Pierce        True\n",
       "5    NH         St. Anselm's        True\n",
       "6    NH            Dartmouth        True\n",
       "7    NH         Colby-Sawyer        True\n",
       "8    NH  New England College        True\n",
       "9    NH               Rivier        True"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "colleges_df = pd.DataFrame(colleges)\n",
    "colleges_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EQNupQLqhi_R"
   },
   "source": [
    "Note that `pandas` automatically replicated \"NH\" down the `State` column when creating the data frame. This behavior seems convenient, but what if in our original dictionary we had set `\"State\" : [\"NH\", \"VT\"]`? Go back and try it! Python also added row numbers to our data frame -- these are more than just a superficial artifact of printing so that we know row 6 belongs to Dartmouth. These row numbers are called `indices` and are a referenceable part of our data frame."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tY4VeCb2hi_S"
   },
   "source": [
    "### Importing a Data Frame"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "L7wSg7KGhi_T"
   },
   "source": [
    "It will often be the case that we wish to import a data frame from a `*.csv` file stored on our local machine or out on the web. With `pandas` we have some simple functionality that allows for this. Below you can see how to read in both types of files (local and remote), but know that the local file exists on my machine. The line to read in the housing data **will not** run for you unless you download the `housing.csv` file from BrightSpace and adjust the path to point to the file's location on your machine. In addition, if you are running Python through Google Colab there are a few additional steps to reading local data into a Python session."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 1361,
     "status": "ok",
     "timestamp": 1611084306572,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "_scMOrA3hi_U",
    "outputId": "71279162-090e-43e3-8671-024ebdd2d434"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "(103, 8)\n"
     ]
    }
   ],
   "source": [
    "#Read in the housing.csv file which exists in the \"Grad Datasets\" folder on my desktop.\n",
    "#homes = pd.read_csv(\"C:/Users/agilb/Desktop/Grad Datasets/housing.csv\")\n",
    "\n",
    "#Read in the heart_transplant.csv file which is hosted at openintro.org\n",
    "heart_transplant_df = pd.read_csv(\"http://www.openintro.org/data/csv/heart_transplant.csv\")\n",
    "\n",
    "#print(type(homes))\n",
    "print(type(heart_transplant_df))\n",
    "\n",
    "#print(homes.shape)\n",
    "print(heart_transplant_df.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DT_X8Rq6hi_X"
   },
   "source": [
    "We did a bit more than just load the heart transplant data in that last chunk. We asked python to print the object type of `heart_transplant_df`, verifying that it is a `pandas` data frame. We also asked for the shape of the data frame -- this returns a vector of length two, of the format (*number of rows*, *number of columns*). Now that we have the data, we can view it. Calling print on the name of the data frame will print the entire data frame to our notebook -- this is more than we need. In order to get a feel for the dataset we likely need only see a few rows. There are many ways to achieve this -- we can use subsetting to request a specific set of rows or also use the `.head()` or `.tail()` methods to get the first five and last five rows respectively. I'll show a few examples and then ask you to write and run code to pull specific rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 765,
     "status": "ok",
     "timestamp": 1611084381709,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "9HPG2FkChi_Y",
    "outputId": "cb1fff61-6ce3-4e99-968f-c586e139a47d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   id  acceptyear  age survived  survtime prior transplant  wait\n",
      "0  15          68   53     dead         1    no    control   NaN\n",
      "1  43          70   43     dead         2    no    control   NaN\n",
      "2  61          71   52     dead         2    no    control   NaN\n",
      "3  75          72   52     dead         2    no    control   NaN\n",
      "   id  acceptyear  age survived  survtime prior transplant  wait\n",
      "0  15          68   53     dead         1    no    control   NaN\n",
      "1  43          70   43     dead         2    no    control   NaN\n",
      "2  61          71   52     dead         2    no    control   NaN\n",
      "3  75          72   52     dead         2    no    control   NaN\n"
     ]
    }
   ],
   "source": [
    "#Two ways to print the first four rows of the data frame.\n",
    "print(heart_transplant_df[0:4])\n",
    "print(heart_transplant_df[:4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 204
    },
    "executionInfo": {
     "elapsed": 761,
     "status": "ok",
     "timestamp": 1611084416136,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "LbEmezLBhi_c",
    "outputId": "4b6118fa-f856-4733-85f3-bca1be74b440"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>acceptyear</th>\n",
       "      <th>age</th>\n",
       "      <th>survived</th>\n",
       "      <th>survtime</th>\n",
       "      <th>prior</th>\n",
       "      <th>transplant</th>\n",
       "      <th>wait</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>15</td>\n",
       "      <td>68</td>\n",
       "      <td>53</td>\n",
       "      <td>dead</td>\n",
       "      <td>1</td>\n",
       "      <td>no</td>\n",
       "      <td>control</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>43</td>\n",
       "      <td>70</td>\n",
       "      <td>43</td>\n",
       "      <td>dead</td>\n",
       "      <td>2</td>\n",
       "      <td>no</td>\n",
       "      <td>control</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>61</td>\n",
       "      <td>71</td>\n",
       "      <td>52</td>\n",
       "      <td>dead</td>\n",
       "      <td>2</td>\n",
       "      <td>no</td>\n",
       "      <td>control</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>75</td>\n",
       "      <td>72</td>\n",
       "      <td>52</td>\n",
       "      <td>dead</td>\n",
       "      <td>2</td>\n",
       "      <td>no</td>\n",
       "      <td>control</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>68</td>\n",
       "      <td>54</td>\n",
       "      <td>dead</td>\n",
       "      <td>3</td>\n",
       "      <td>no</td>\n",
       "      <td>control</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  acceptyear  age survived  survtime prior transplant  wait\n",
       "0  15          68   53     dead         1    no    control   NaN\n",
       "1  43          70   43     dead         2    no    control   NaN\n",
       "2  61          71   52     dead         2    no    control   NaN\n",
       "3  75          72   52     dead         2    no    control   NaN\n",
       "4   6          68   54     dead         3    no    control   NaN"
      ]
     },
     "execution_count": 23,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Use .head() to get the first five rows\n",
    "heart_transplant_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 824
    },
    "executionInfo": {
     "elapsed": 746,
     "status": "ok",
     "timestamp": 1611084462278,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "QYbNiYnChi_f",
    "outputId": "e77ef1b7-8662-450d-e5b4-b441cb0be70a"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>acceptyear</th>\n",
       "      <th>age</th>\n",
       "      <th>survived</th>\n",
       "      <th>survtime</th>\n",
       "      <th>prior</th>\n",
       "      <th>transplant</th>\n",
       "      <th>wait</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>78</th>\n",
       "      <td>81</td>\n",
       "      <td>73</td>\n",
       "      <td>52</td>\n",
       "      <td>alive</td>\n",
       "      <td>445</td>\n",
       "      <td>no</td>\n",
       "      <td>treatment</td>\n",
       "      <td>6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79</th>\n",
       "      <td>80</td>\n",
       "      <td>72</td>\n",
       "      <td>46</td>\n",
       "      <td>alive</td>\n",
       "      <td>482</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>26.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>80</th>\n",
       "      <td>78</td>\n",
       "      <td>72</td>\n",
       "      <td>48</td>\n",
       "      <td>alive</td>\n",
       "      <td>515</td>\n",
       "      <td>no</td>\n",
       "      <td>treatment</td>\n",
       "      <td>210.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81</th>\n",
       "      <td>76</td>\n",
       "      <td>72</td>\n",
       "      <td>52</td>\n",
       "      <td>alive</td>\n",
       "      <td>545</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>46.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82</th>\n",
       "      <td>64</td>\n",
       "      <td>72</td>\n",
       "      <td>48</td>\n",
       "      <td>dead</td>\n",
       "      <td>583</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>32.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>83</th>\n",
       "      <td>72</td>\n",
       "      <td>72</td>\n",
       "      <td>26</td>\n",
       "      <td>alive</td>\n",
       "      <td>596</td>\n",
       "      <td>no</td>\n",
       "      <td>treatment</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>84</th>\n",
       "      <td>71</td>\n",
       "      <td>72</td>\n",
       "      <td>47</td>\n",
       "      <td>alive</td>\n",
       "      <td>630</td>\n",
       "      <td>no</td>\n",
       "      <td>treatment</td>\n",
       "      <td>31.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>85</th>\n",
       "      <td>69</td>\n",
       "      <td>72</td>\n",
       "      <td>47</td>\n",
       "      <td>alive</td>\n",
       "      <td>670</td>\n",
       "      <td>no</td>\n",
       "      <td>treatment</td>\n",
       "      <td>10.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>86</th>\n",
       "      <td>7</td>\n",
       "      <td>68</td>\n",
       "      <td>50</td>\n",
       "      <td>dead</td>\n",
       "      <td>675</td>\n",
       "      <td>no</td>\n",
       "      <td>treatment</td>\n",
       "      <td>51.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87</th>\n",
       "      <td>23</td>\n",
       "      <td>69</td>\n",
       "      <td>58</td>\n",
       "      <td>dead</td>\n",
       "      <td>733</td>\n",
       "      <td>no</td>\n",
       "      <td>treatment</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>88</th>\n",
       "      <td>63</td>\n",
       "      <td>71</td>\n",
       "      <td>32</td>\n",
       "      <td>alive</td>\n",
       "      <td>841</td>\n",
       "      <td>no</td>\n",
       "      <td>treatment</td>\n",
       "      <td>27.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>89</th>\n",
       "      <td>30</td>\n",
       "      <td>69</td>\n",
       "      <td>44</td>\n",
       "      <td>dead</td>\n",
       "      <td>852</td>\n",
       "      <td>no</td>\n",
       "      <td>treatment</td>\n",
       "      <td>16.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>90</th>\n",
       "      <td>59</td>\n",
       "      <td>71</td>\n",
       "      <td>41</td>\n",
       "      <td>alive</td>\n",
       "      <td>915</td>\n",
       "      <td>no</td>\n",
       "      <td>treatment</td>\n",
       "      <td>78.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>91</th>\n",
       "      <td>56</td>\n",
       "      <td>71</td>\n",
       "      <td>38</td>\n",
       "      <td>alive</td>\n",
       "      <td>941</td>\n",
       "      <td>no</td>\n",
       "      <td>treatment</td>\n",
       "      <td>67.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>92</th>\n",
       "      <td>50</td>\n",
       "      <td>71</td>\n",
       "      <td>45</td>\n",
       "      <td>dead</td>\n",
       "      <td>979</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>83.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>93</th>\n",
       "      <td>46</td>\n",
       "      <td>71</td>\n",
       "      <td>48</td>\n",
       "      <td>dead</td>\n",
       "      <td>995</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>94</th>\n",
       "      <td>21</td>\n",
       "      <td>69</td>\n",
       "      <td>43</td>\n",
       "      <td>dead</td>\n",
       "      <td>1032</td>\n",
       "      <td>no</td>\n",
       "      <td>treatment</td>\n",
       "      <td>8.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95</th>\n",
       "      <td>49</td>\n",
       "      <td>71</td>\n",
       "      <td>36</td>\n",
       "      <td>alive</td>\n",
       "      <td>1141</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>36.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>41</td>\n",
       "      <td>70</td>\n",
       "      <td>45</td>\n",
       "      <td>alive</td>\n",
       "      <td>1321</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>58.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>14</td>\n",
       "      <td>68</td>\n",
       "      <td>53</td>\n",
       "      <td>dead</td>\n",
       "      <td>1386</td>\n",
       "      <td>no</td>\n",
       "      <td>treatment</td>\n",
       "      <td>37.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>26</td>\n",
       "      <td>69</td>\n",
       "      <td>30</td>\n",
       "      <td>alive</td>\n",
       "      <td>1400</td>\n",
       "      <td>no</td>\n",
       "      <td>control</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>40</td>\n",
       "      <td>70</td>\n",
       "      <td>48</td>\n",
       "      <td>alive</td>\n",
       "      <td>1407</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>41.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>34</td>\n",
       "      <td>69</td>\n",
       "      <td>40</td>\n",
       "      <td>alive</td>\n",
       "      <td>1571</td>\n",
       "      <td>no</td>\n",
       "      <td>treatment</td>\n",
       "      <td>23.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>101</th>\n",
       "      <td>33</td>\n",
       "      <td>69</td>\n",
       "      <td>48</td>\n",
       "      <td>alive</td>\n",
       "      <td>1586</td>\n",
       "      <td>no</td>\n",
       "      <td>treatment</td>\n",
       "      <td>51.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>102</th>\n",
       "      <td>25</td>\n",
       "      <td>69</td>\n",
       "      <td>33</td>\n",
       "      <td>alive</td>\n",
       "      <td>1799</td>\n",
       "      <td>no</td>\n",
       "      <td>treatment</td>\n",
       "      <td>25.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     id  acceptyear  age survived  survtime prior transplant   wait\n",
       "78   81          73   52    alive       445    no  treatment    6.0\n",
       "79   80          72   46    alive       482   yes  treatment   26.0\n",
       "80   78          72   48    alive       515    no  treatment  210.0\n",
       "81   76          72   52    alive       545   yes  treatment   46.0\n",
       "82   64          72   48     dead       583   yes  treatment   32.0\n",
       "83   72          72   26    alive       596    no  treatment    4.0\n",
       "84   71          72   47    alive       630    no  treatment   31.0\n",
       "85   69          72   47    alive       670    no  treatment   10.0\n",
       "86    7          68   50     dead       675    no  treatment   51.0\n",
       "87   23          69   58     dead       733    no  treatment    3.0\n",
       "88   63          71   32    alive       841    no  treatment   27.0\n",
       "89   30          69   44     dead       852    no  treatment   16.0\n",
       "90   59          71   41    alive       915    no  treatment   78.0\n",
       "91   56          71   38    alive       941    no  treatment   67.0\n",
       "92   50          71   45     dead       979   yes  treatment   83.0\n",
       "93   46          71   48     dead       995   yes  treatment    2.0\n",
       "94   21          69   43     dead      1032    no  treatment    8.0\n",
       "95   49          71   36    alive      1141   yes  treatment   36.0\n",
       "96   41          70   45    alive      1321   yes  treatment   58.0\n",
       "97   14          68   53     dead      1386    no  treatment   37.0\n",
       "98   26          69   30    alive      1400    no    control    NaN\n",
       "99   40          70   48    alive      1407   yes  treatment   41.0\n",
       "100  34          69   40    alive      1571    no  treatment   23.0\n",
       "101  33          69   48    alive      1586    no  treatment   51.0\n",
       "102  25          69   33    alive      1799    no  treatment   25.0"
      ]
     },
     "execution_count": 25,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Use .tail() to get the last five rows\n",
    "heart_transplant_df.tail(n = 25)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 483
    },
    "executionInfo": {
     "elapsed": 640,
     "status": "ok",
     "timestamp": 1611084553907,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "F-ddw6Sohi_h",
    "outputId": "8684cf38-4649-4088-abce-4e3100ac957a"
   },
   "outputs": [],
   "source": [
    "#Use this block to print rows 10 through (and including 23). \n",
    "#How does this change if you want to print the rows corresponding to indices 10 through (and including 23)?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "w46y15nBhi_k"
   },
   "outputs": [],
   "source": [
    "#Use this block to edit your call to the .head() method so that python prints the first 8 rows instead of the first 5.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OEXJRdmohi_m"
   },
   "source": [
    "In addition to subsetting a data frame by rows (observations) we can also subset by columns (variables)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 419
    },
    "executionInfo": {
     "elapsed": 740,
     "status": "ok",
     "timestamp": 1611084689394,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "c23YJZh-hi_n",
    "outputId": "98450b07-a714-45a5-d0f0-1e1da4f869d0"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>survived</th>\n",
       "      <th>prior</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>53</td>\n",
       "      <td>dead</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>43</td>\n",
       "      <td>dead</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>52</td>\n",
       "      <td>dead</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>52</td>\n",
       "      <td>dead</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>54</td>\n",
       "      <td>dead</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>30</td>\n",
       "      <td>alive</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>48</td>\n",
       "      <td>alive</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>40</td>\n",
       "      <td>alive</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>101</th>\n",
       "      <td>48</td>\n",
       "      <td>alive</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>102</th>\n",
       "      <td>33</td>\n",
       "      <td>alive</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>103 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     age survived prior\n",
       "0     53     dead    no\n",
       "1     43     dead    no\n",
       "2     52     dead    no\n",
       "3     52     dead    no\n",
       "4     54     dead    no\n",
       "..   ...      ...   ...\n",
       "98    30    alive    no\n",
       "99    48    alive   yes\n",
       "100   40    alive    no\n",
       "101   48    alive    no\n",
       "102   33    alive    no\n",
       "\n",
       "[103 rows x 3 columns]"
      ]
     },
     "execution_count": 31,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "heart_transplant_df[[\"age\", \"survived\", \"prior\"]]\n",
    "#print(heart_transplant_df[\"age\"].head())\n",
    "#print(heart_transplant_df[[\"id\", \"age\", \"survtime\"]].head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8nByqoylhi_t"
   },
   "source": [
    "### Subsetting with `.loc[]` and `.iloc[]`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lImsCYuvhi_v"
   },
   "source": [
    "With `pandas` we have two methods that are very useful for subsetting. They are much more convenient and flexible than the aproaches we have used above. For example, it is quite easy to subset by both *rows* and *columns* with these methods. The difference between `.loc[]` and `.iloc[]` is that `.iloc[]` subsets by row and column **indices** while `.loc[]` subsets by **name or condition**. We will most often use `.loc[]`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 142
    },
    "executionInfo": {
     "elapsed": 671,
     "status": "ok",
     "timestamp": 1611084818154,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "IsNtTEMAhi_v",
    "outputId": "97f8ff25-22cf-4e07-d6bf-86e96cafb8e0"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>survived</th>\n",
       "      <th>survtime</th>\n",
       "      <th>prior</th>\n",
       "      <th>transplant</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>43</td>\n",
       "      <td>dead</td>\n",
       "      <td>2</td>\n",
       "      <td>no</td>\n",
       "      <td>control</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>52</td>\n",
       "      <td>dead</td>\n",
       "      <td>2</td>\n",
       "      <td>no</td>\n",
       "      <td>control</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>52</td>\n",
       "      <td>dead</td>\n",
       "      <td>2</td>\n",
       "      <td>no</td>\n",
       "      <td>control</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age survived  survtime prior transplant\n",
       "1   43     dead         2    no    control\n",
       "2   52     dead         2    no    control\n",
       "3   52     dead         2    no    control"
      ]
     },
     "execution_count": 32,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Get rows indexed 1, 2, and 3 and columns indexed 2, 3, 4, 5, and 6.\n",
    "heart_transplant_df.iloc[1:4, 2:7]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 142
    },
    "executionInfo": {
     "elapsed": 685,
     "status": "ok",
     "timestamp": 1611084865250,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "m06__MUnI4l1",
    "outputId": "ee21af44-7d0f-41ef-cf68-b3407d52891a"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>survived</th>\n",
       "      <th>survtime</th>\n",
       "      <th>prior</th>\n",
       "      <th>transplant</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>43</td>\n",
       "      <td>dead</td>\n",
       "      <td>2</td>\n",
       "      <td>no</td>\n",
       "      <td>control</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>52</td>\n",
       "      <td>dead</td>\n",
       "      <td>2</td>\n",
       "      <td>no</td>\n",
       "      <td>control</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>52</td>\n",
       "      <td>dead</td>\n",
       "      <td>2</td>\n",
       "      <td>no</td>\n",
       "      <td>control</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age survived  survtime prior transplant\n",
       "1   43     dead         2    no    control\n",
       "2   52     dead         2    no    control\n",
       "3   52     dead         2    no    control"
      ]
     },
     "execution_count": 33,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "heart_transplant_df[[\"age\", \"survived\", \"survtime\", \"prior\", \"transplant\"]][1:4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 266
    },
    "executionInfo": {
     "elapsed": 945,
     "status": "ok",
     "timestamp": 1611084920678,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "Nc4yO2Achi_y",
    "outputId": "10fbb494-2a26-4a94-dcdd-5227914ff1af"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>survived</th>\n",
       "      <th>prior</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>45</td>\n",
       "      <td>alive</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>53</td>\n",
       "      <td>dead</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>30</td>\n",
       "      <td>alive</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>48</td>\n",
       "      <td>alive</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>40</td>\n",
       "      <td>alive</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>101</th>\n",
       "      <td>48</td>\n",
       "      <td>alive</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>102</th>\n",
       "      <td>33</td>\n",
       "      <td>alive</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     age survived prior\n",
       "96    45    alive   yes\n",
       "97    53     dead    no\n",
       "98    30    alive    no\n",
       "99    48    alive   yes\n",
       "100   40    alive    no\n",
       "101   48    alive    no\n",
       "102   33    alive    no"
      ]
     },
     "execution_count": 34,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Get the \"age\", \"survived\", and \"prior\" columns, and print only the last 7 rows of the dataset.\n",
    "heart_transplant_df.loc[(len(heart_transplant_df) - 7):  , [\"age\", \"survived\", \"prior\"]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 266
    },
    "executionInfo": {
     "elapsed": 886,
     "status": "ok",
     "timestamp": 1611084946508,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "uo6B9iCkJQbO",
    "outputId": "7b9bf2a9-434a-4b6c-8517-b989d6a19186"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>survived</th>\n",
       "      <th>prior</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>45</td>\n",
       "      <td>alive</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>53</td>\n",
       "      <td>dead</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>30</td>\n",
       "      <td>alive</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>48</td>\n",
       "      <td>alive</td>\n",
       "      <td>yes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>40</td>\n",
       "      <td>alive</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>101</th>\n",
       "      <td>48</td>\n",
       "      <td>alive</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>102</th>\n",
       "      <td>33</td>\n",
       "      <td>alive</td>\n",
       "      <td>no</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     age survived prior\n",
       "96    45    alive   yes\n",
       "97    53     dead    no\n",
       "98    30    alive    no\n",
       "99    48    alive   yes\n",
       "100   40    alive    no\n",
       "101   48    alive    no\n",
       "102   33    alive    no"
      ]
     },
     "execution_count": 35,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#A second method for achieving the same as the previous code block.\n",
    "heart_transplant_df[[\"age\", \"survived\", \"prior\"]].tail(n = 7)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "i-rzUUXqhi_0"
   },
   "source": [
    "Note that in the above code chunk we requested rows from \"7 less than the length of the data frame, all the way to the end of the data frame\". Also, since we wanted multiple columns returned, we needed to use the list notation -- for convenience, if we wanted every column between `acceptyear` and `survtime` we could conveniently use `\"acceptyear\":\"survtime\"` in place of the list notation. \n",
    "\n",
    "In addition to this prescribed subsetting we can use `.loc[]` to subset based on a condition. For example, we can obtain all of those transplant recipients who had a prior transplant by requesting rows for which `heart_transplant_df[\"prior\"] == \"yes\"` (see below, and note that this condition is case sensitive)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 421
    },
    "executionInfo": {
     "elapsed": 696,
     "status": "ok",
     "timestamp": 1611085097960,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "c7Q4uNcJhi_0",
    "outputId": "075974fc-0d73-4ede-c795-89d6018b3cfe"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>acceptyear</th>\n",
       "      <th>age</th>\n",
       "      <th>survived</th>\n",
       "      <th>survtime</th>\n",
       "      <th>prior</th>\n",
       "      <th>transplant</th>\n",
       "      <th>wait</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>100</td>\n",
       "      <td>74</td>\n",
       "      <td>35</td>\n",
       "      <td>alive</td>\n",
       "      <td>39</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>38.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>60</th>\n",
       "      <td>94</td>\n",
       "      <td>73</td>\n",
       "      <td>43</td>\n",
       "      <td>dead</td>\n",
       "      <td>165</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>62</th>\n",
       "      <td>90</td>\n",
       "      <td>73</td>\n",
       "      <td>52</td>\n",
       "      <td>dead</td>\n",
       "      <td>186</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>160.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>74</th>\n",
       "      <td>58</td>\n",
       "      <td>71</td>\n",
       "      <td>47</td>\n",
       "      <td>dead</td>\n",
       "      <td>342</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>21.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79</th>\n",
       "      <td>80</td>\n",
       "      <td>72</td>\n",
       "      <td>46</td>\n",
       "      <td>alive</td>\n",
       "      <td>482</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>26.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81</th>\n",
       "      <td>76</td>\n",
       "      <td>72</td>\n",
       "      <td>52</td>\n",
       "      <td>alive</td>\n",
       "      <td>545</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>46.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82</th>\n",
       "      <td>64</td>\n",
       "      <td>72</td>\n",
       "      <td>48</td>\n",
       "      <td>dead</td>\n",
       "      <td>583</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>32.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>92</th>\n",
       "      <td>50</td>\n",
       "      <td>71</td>\n",
       "      <td>45</td>\n",
       "      <td>dead</td>\n",
       "      <td>979</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>83.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>93</th>\n",
       "      <td>46</td>\n",
       "      <td>71</td>\n",
       "      <td>48</td>\n",
       "      <td>dead</td>\n",
       "      <td>995</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>95</th>\n",
       "      <td>49</td>\n",
       "      <td>71</td>\n",
       "      <td>36</td>\n",
       "      <td>alive</td>\n",
       "      <td>1141</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>36.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>96</th>\n",
       "      <td>41</td>\n",
       "      <td>70</td>\n",
       "      <td>45</td>\n",
       "      <td>alive</td>\n",
       "      <td>1321</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>58.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>40</td>\n",
       "      <td>70</td>\n",
       "      <td>48</td>\n",
       "      <td>alive</td>\n",
       "      <td>1407</td>\n",
       "      <td>yes</td>\n",
       "      <td>treatment</td>\n",
       "      <td>41.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     id  acceptyear  age survived  survtime prior transplant   wait\n",
       "30  100          74   35    alive        39   yes  treatment   38.0\n",
       "60   94          73   43     dead       165   yes  treatment    4.0\n",
       "62   90          73   52     dead       186   yes  treatment  160.0\n",
       "74   58          71   47     dead       342   yes  treatment   21.0\n",
       "79   80          72   46    alive       482   yes  treatment   26.0\n",
       "81   76          72   52    alive       545   yes  treatment   46.0\n",
       "82   64          72   48     dead       583   yes  treatment   32.0\n",
       "92   50          71   45     dead       979   yes  treatment   83.0\n",
       "93   46          71   48     dead       995   yes  treatment    2.0\n",
       "95   49          71   36    alive      1141   yes  treatment   36.0\n",
       "96   41          70   45    alive      1321   yes  treatment   58.0\n",
       "99   40          70   48    alive      1407   yes  treatment   41.0"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "heart_transplant_df.loc[heart_transplant_df[\"prior\"] == \"yes\", :]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "P8z7PlUmhi_3"
   },
   "source": [
    "There are many boolean (T/F) operators which can be used for conditioning (subsetting or flow control). Some of the most common are below:\n",
    "+ `a == b` a **test** for exact equality between `a` and `b`\n",
    "+ `a > b`, `a < b`, `a >= b`, and `a <= b` behave as expected\n",
    "+ `a in list` or `a in [\"red\", \"green\", \"blue\", \"yellow\"]` tests whether the value stored in the object `a` is contained in the corresponding list objects.\n",
    "\n",
    "Use the code blow below to subset the `heart_transplant_df` data frame so that you achieve each of the following (individually).\n",
    "1. You obtain information on all recipients over 50 years old.\n",
    "2. You obtain `age`, `survtime`, `prior`, and `wait` on all recipients who are still alive.\n",
    "3. You obtain information on all recipients who are dead or (`|`) are atleast 57 years old.\n",
    "4. You obtain information on all recipients who are under 55 years old and (`&`) who had a prior heart transplant."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 421
    },
    "executionInfo": {
     "elapsed": 739,
     "status": "ok",
     "timestamp": 1611085955910,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "5-0rxjUWhi_4",
    "outputId": "a033a0f4-e665-43f0-9ca0-59db5d4d9f5a"
   },
   "outputs": [],
   "source": [
    "# Write and execute your code here.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6r3Su72zhi_6"
   },
   "source": [
    "### Missing Values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "P8rTmQtVhi_7"
   },
   "source": [
    "We've covered a lot in this crash course so far, but we've left out a significant chunk -- how do we identify missing values? We can do that quite easily with the `.isna()` method. We'll combine `.isna()` with `.sum()` to identify where `NA` (missing) values occur in our dataset. There are lots of questions to consider when dealing with missing data.\n",
    "+ Why is the data missing?\n",
    "+ Does the missing data tell us something (ie. is the missing data actually data itself)?\n",
    "+ Does the missing data corrupt (make useless) the variable or possibly even dataset?\n",
    "+ How should we treat the missing data? Leave it as missing? Remove rows containing missing data? Try to fill it in somehow?\n",
    "\n",
    "We won't address any of those items here but you will certainly encounter missing data and need to answer these questions throughout our course. Learning how to deal with missing data is critical, especially since `sklearn` won't let us build models involving variables with missing values!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 986,
     "status": "ok",
     "timestamp": 1611086103362,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "XTut7Cauhi_8",
    "outputId": "5859abc0-2795-4f24-e42c-c4a78bbe2b0a"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "id             0\n",
       "acceptyear     0\n",
       "age            0\n",
       "survived       0\n",
       "survtime       0\n",
       "prior          0\n",
       "transplant     0\n",
       "wait          34\n",
       "dtype: int64"
      ]
     },
     "execution_count": 51,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "heart_transplant_df.isna().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zAGRU5n5hi__"
   },
   "source": [
    "Now that we know *the basics of* how to work with data frames in python, we should learn how to gain insight from the information in these data frames."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "b031jdlYhjAA"
   },
   "source": [
    "## Analyzing Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "eHWbA20mhjAB"
   },
   "source": [
    "The `pandas` library comes with methods for computing useful information on data frames:\n",
    "+ `.info()` provides information on the column structure of a data frame\n",
    "+ `.describe()` computes summary statistics for numerical columns of a data frame\n",
    "\n",
    "and also for computing summary statistics for individual columns as well.\n",
    "+ `.mean()` for computing the standard mean\n",
    "+ `.median()` for computing the median (a better -- more robust -- measure of center in the case of outliers)\n",
    "+ `.std()` for computing standard deviation\n",
    "+ `.min()` and `.max()` for identifying the minimum and maximum values in a column\n",
    "+ `.quantile([0.1, 0.25, 0.5, 0.75, 0.9])` computes quantile boundaries -- for example this call would compute the boundaries for the 10th, 25th, 50th, 75th, and 90th percentiles\n",
    "+ `.value_counts()` for constructing a frequency table for categorical columns.\n",
    "\n",
    "See the `pandas` cheatsheet [here](http://datacamp-community-prod.s3.amazonaws.com/dbed353d-2757-4617-8206-8767ab379ab3) for more."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 842,
     "status": "ok",
     "timestamp": 1611086408069,
     "user": {
      "displayName": "Adam Gilbert",
      "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj1KWqLKG2SAQxx8KvqzqhM94EvqxkXs2iOw7gQWQ=s64",
      "userId": "02074648007699947871"
     },
     "user_tz": 300
    },
    "id": "-HgvniVBhjAC",
    "outputId": "0b51ca2b-8b19-4c28-c9cc-9488a02beb8b"
   },
   "outputs": [],
   "source": [
    "#Get standard info on columns including how many non-null (not missing) \n",
    "#values are present and also the type of object (numeric or non-numeric object).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "axq9JU3NhjAE"
   },
   "outputs": [],
   "source": [
    "#Compute general summary statistics for all numerical columns.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KRhcr09EhjAG"
   },
   "source": [
    "Use the code block below to find:\n",
    "1. The average survival time across all recipients\n",
    "2. The average survival time for those recipients who have died\n",
    "3. The minimum and maximum wait times as well as the cutoffs for the 10th and 90th percentiles of wait times.\n",
    "4. A frequency table for the survived and transplant columns.\n",
    "5. Execute `pd.crosstab(heart_transplant_df[\"survived\"], heart_transplant_df[\"transplant\"])` and explain its output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "AITPeQbIhjAH"
   },
   "outputs": [],
   "source": [
    "#Experiment here\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kSEKQ14A8T-4"
   },
   "source": [
    "## Summary\r\n",
    "\r\n",
    "Okay -- that's plenty for this notebook. You were exposed to some Python basics, including atomic data types (booleans, strings, numerics, etc.), lists, arrays, dictionaries, and data frames. In analytics we'll most often be working with data frames, lists, and series but it is useful to know about these other structures since dealing with them is sometimes required. The most important items you learned in this notebook were how to subset both rows and columns of data frames -- in particular, subsetting based on conditions. Your next notebook will give you more opportunity to work with data frames and will discuss some of this material in more detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "moOCcc-coJ90"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "3_IntroToPython.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}