L3_practice_with_pandas.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lesson 3: Practice with Pandas"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr/>\n",
    "\n",
    "Pandas can be a bit frustrating during your first experiences with it. In this lesson, we will practice using Pandas. The more and more you use it, the more distant the memory of life without it will become."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The data set\n",
    "\n",
    "The data set comes from [Kleinteich and Gorb, *Sci. Rep.*, **4**, 5355, 2014](https://doi.org/10.1038/srep05225), and was [featured in the New York Times](http://www.nytimes.com/2014/08/25/science/a-frog-thats-a-living-breathing-pac-man.html).  They measured several properties about the tongue strikes of horned frogs.  Let's take a look at the data set, which is in the file `~/Dataset/frog_tongue_adhesion.csv`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# These data are from the paper,\n",
      "#   Kleinteich and Gorb, Sci. Rep., 4, 5225, 2014.\n",
      "# It was featured in the New York Times.\n",
      "#    http://www.nytimes.com/2014/08/25/science/a-frog-thats-a-living-breathing-pac-man.html\n",
      "#\n",
      "# The authors included the data in their supplemental information.\n",
      "#\n",
      "# Importantly, the ID refers to the identifites of the frogs they tested.\n",
      "#   I:   adult, 63 mm snout-vent-length (SVL) and 63.1 g body weight,\n",
      "#        Ceratophrys cranwelli crossed with Ceratophrys cornuta\n",
      "#   II:  adult, 70 mm SVL and 72.7 g body weight,\n",
      "#        Ceratophrys cranwelli crossed with Ceratophrys cornuta\n",
      "#   III: juvenile, 28 mm SVL and 12.7 g body weight, Ceratophrys cranwelli\n",
      "#   IV:  juvenile, 31 mm SVL and 12.7 g body weight, Ceratophrys cranwelli\n",
      "date,ID,trial number,impact force (mN),impact time (ms),impact force / body weight,adhesive force (mN),time frog pulls on target (ms),adhesive force / body weight,adhesive impulse (N-s),total contact area (mm2),contact area without mucus (mm2),contact area with mucus / contact area without mucus,contact pressure (Pa),adhesive strength (Pa)\n",
      "2013_02_26,I,3,1205,46,1.95,-785,884,1.27,-0.290,387,70,0.82,3117,-2030\n",
      "2013_02_26,I,4,2527,44,4.08,-983,248,1.59,-0.181,101,94,0.07,24923,-9695\n",
      "2013_03_01,I,1,1745,34,2.82,-850,211,1.37,-0.157,83,79,0.05,21020,-10239\n",
      "2013_03_01,I,2,1556,41,2.51,-455,1025,0.74,-0.170,330,158,0.52,4718,-1381\n",
      "2013_03_01,I,3,493,36,0.80,-974,499,1.57,-0.423,245,216,0.12,2012,-3975\n"
     ]
    }
   ],
   "source": [
    "!head -20 Dataset/frog_tongue_adhesion.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first lines all begin with `#` signs, signifying that they are comments and not data. They do give important information, though, such as the meaning of the ID data. The ID refers to which specific frog was tested.\n",
    "\n",
    "Immediately after the comments, we have a row of comma-separated headers. This row sets the number of columns in this data set and labels the meaning of the columns. So, we see that the first column is the date of the experiment, the second column is the ID of the frog, the third is the trial number, and so on.\n",
    "\n",
    "After this row, each row represents a single experiment where the frog struck the target. So, these data are already in tidy format. Let's go ahead and load the data into a `DataFrame`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date</th>\n",
       "      <th>ID</th>\n",
       "      <th>trial number</th>\n",
       "      <th>impact force (mN)</th>\n",
       "      <th>impact time (ms)</th>\n",
       "      <th>impact force / body weight</th>\n",
       "      <th>adhesive force (mN)</th>\n",
       "      <th>time frog pulls on target (ms)</th>\n",
       "      <th>adhesive force / body weight</th>\n",
       "      <th>adhesive impulse (N-s)</th>\n",
       "      <th>total contact area (mm2)</th>\n",
       "      <th>contact area without mucus (mm2)</th>\n",
       "      <th>contact area with mucus / contact area without mucus</th>\n",
       "      <th>contact pressure (Pa)</th>\n",
       "      <th>adhesive strength (Pa)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2013_02_26</td>\n",
       "      <td>I</td>\n",
       "      <td>3</td>\n",
       "      <td>1205</td>\n",
       "      <td>46</td>\n",
       "      <td>1.95</td>\n",
       "      <td>-785</td>\n",
       "      <td>884</td>\n",
       "      <td>1.27</td>\n",
       "      <td>-0.290</td>\n",
       "      <td>387</td>\n",
       "      <td>70</td>\n",
       "      <td>0.82</td>\n",
       "      <td>3117</td>\n",
       "      <td>-2030</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2013_02_26</td>\n",
       "      <td>I</td>\n",
       "      <td>4</td>\n",
       "      <td>2527</td>\n",
       "      <td>44</td>\n",
       "      <td>4.08</td>\n",
       "      <td>-983</td>\n",
       "      <td>248</td>\n",
       "      <td>1.59</td>\n",
       "      <td>-0.181</td>\n",
       "      <td>101</td>\n",
       "      <td>94</td>\n",
       "      <td>0.07</td>\n",
       "      <td>24923</td>\n",
       "      <td>-9695</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2013_03_01</td>\n",
       "      <td>I</td>\n",
       "      <td>1</td>\n",
       "      <td>1745</td>\n",
       "      <td>34</td>\n",
       "      <td>2.82</td>\n",
       "      <td>-850</td>\n",
       "      <td>211</td>\n",
       "      <td>1.37</td>\n",
       "      <td>-0.157</td>\n",
       "      <td>83</td>\n",
       "      <td>79</td>\n",
       "      <td>0.05</td>\n",
       "      <td>21020</td>\n",
       "      <td>-10239</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2013_03_01</td>\n",
       "      <td>I</td>\n",
       "      <td>2</td>\n",
       "      <td>1556</td>\n",
       "      <td>41</td>\n",
       "      <td>2.51</td>\n",
       "      <td>-455</td>\n",
       "      <td>1025</td>\n",
       "      <td>0.74</td>\n",
       "      <td>-0.170</td>\n",
       "      <td>330</td>\n",
       "      <td>158</td>\n",
       "      <td>0.52</td>\n",
       "      <td>4718</td>\n",
       "      <td>-1381</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2013_03_01</td>\n",
       "      <td>I</td>\n",
       "      <td>3</td>\n",
       "      <td>493</td>\n",
       "      <td>36</td>\n",
       "      <td>0.80</td>\n",
       "      <td>-974</td>\n",
       "      <td>499</td>\n",
       "      <td>1.57</td>\n",
       "      <td>-0.423</td>\n",
       "      <td>245</td>\n",
       "      <td>216</td>\n",
       "      <td>0.12</td>\n",
       "      <td>2012</td>\n",
       "      <td>-3975</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         date ID  trial number  impact force (mN)  impact time (ms)  \\\n",
       "0  2013_02_26  I             3               1205                46   \n",
       "1  2013_02_26  I             4               2527                44   \n",
       "2  2013_03_01  I             1               1745                34   \n",
       "3  2013_03_01  I             2               1556                41   \n",
       "4  2013_03_01  I             3                493                36   \n",
       "\n",
       "   impact force / body weight  adhesive force (mN)  \\\n",
       "0                        1.95                 -785   \n",
       "1                        4.08                 -983   \n",
       "2                        2.82                 -850   \n",
       "3                        2.51                 -455   \n",
       "4                        0.80                 -974   \n",
       "\n",
       "   time frog pulls on target (ms)  adhesive force / body weight  \\\n",
       "0                             884                          1.27   \n",
       "1                             248                          1.59   \n",
       "2                             211                          1.37   \n",
       "3                            1025                          0.74   \n",
       "4                             499                          1.57   \n",
       "\n",
       "   adhesive impulse (N-s)  total contact area (mm2)  \\\n",
       "0                  -0.290                       387   \n",
       "1                  -0.181                       101   \n",
       "2                  -0.157                        83   \n",
       "3                  -0.170                       330   \n",
       "4                  -0.423                       245   \n",
       "\n",
       "   contact area without mucus (mm2)  \\\n",
       "0                                70   \n",
       "1                                94   \n",
       "2                                79   \n",
       "3                               158   \n",
       "4                               216   \n",
       "\n",
       "   contact area with mucus / contact area without mucus  \\\n",
       "0                                               0.82      \n",
       "1                                               0.07      \n",
       "2                                               0.05      \n",
       "3                                               0.52      \n",
       "4                                               0.12      \n",
       "\n",
       "   contact pressure (Pa)  adhesive strength (Pa)  \n",
       "0                   3117                   -2030  \n",
       "1                  24923                   -9695  \n",
       "2                  21020                  -10239  \n",
       "3                   4718                   -1381  \n",
       "4                   2012                   -3975  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load the data\n",
    "df = pd.read_csv('Dataset/frog_tongue_adhesion.csv', comment='#')\n",
    "\n",
    "# Take a look\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Practice 1: Mastering .loc\n",
    "\n",
    "Your goal here is to extract certain entries out of the `DataFrame`.\n",
    "\n",
    "**a)** Extract the impact time of all impacts that had an adhesive strength of magnitude greater than 2000 Pa. *Note*: The data in the `'adhesive strength (Pa)'` column is all negative. This is because the adhesive force is defined to be negative in the measurement. Without changing the data in the data frame, how can you check that the magnitude (the absolute value) is greater than 2000?\n",
    "\n",
    "**b)** Extract the impact force and adhesive force for all of Frog II's strikes.\n",
    "\n",
    "**c)** Extract the adhesive force and the time the frog pulls on the target for juvenile frogs (Frogs III and IV). *Hint*: We saw the `&` operator for Boolean indexing across more than one column. The `|` operator signifies OR, and works analogously. For technical reasons that we can discuss if you like, the Python operators `and` and `or` will **not** work for Boolean indexing of data frames. You could also approach this using the `isin()` method of a Pandas `Series`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Practice 2: Split-Apply-Combine of the frog data set\n",
    "\n",
    "You'll now practice your split-apply-combine skills.\n",
    "\n",
    "**a)** Compute standard deviation of the impact forces for each frog.\n",
    "\n",
    "**b)** Compute the coefficient of variation of the impact forces *and* adhesive forces for each frog.\n",
    "\n",
    "**c)** And now, finally.... Compute a `DataFrame` that has the mean, median, standard deviation, and coefficient of variation of the impact forces and adhesive forces for each frog. After you make this `DataFrame`, you might want to explore using the `pd.melt()` function to make it tidy. You can read [the documentation](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.melt.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Computing environment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPython 3.6.8\n",
      "IPython 7.1.1\n",
      "\n",
      "pandas 0.24.2\n",
      "jupyterlab 1.0.2\n"
     ]
    }
   ],
   "source": [
    "%load_ext watermark\n",
    "%watermark -v -p pandas,jupyterlab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}