618_02_Pandas_Intro.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SI 618: Data Manipulation and Analysis\n",
    "## 02 - Introduction to pandas\n",
    "### Dr. Chris Teplovs, School of Information, University of Michigan\n",
    "<small><a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\"><img alt=\"Creative Commons License\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png\" /></a>This work is licensed under a <a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc-sa/4.0/\">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "6xlxDwAkVZ6g"
   },
   "source": [
    "\n",
    "## Objectives:\n",
    "* Know how to manipulate Series and DataFrame\n",
    "* Draw a random sample of data\n",
    "* Select subset of data using boolean masking\n",
    "* Compute descriptive and summary statistics\n",
    "* Discretize continuous variables\n",
    "* Sort Series/DataFrame by index or column\n",
    "* Group data and calculate aggregate statistics\n",
    "* Use vectorized string methods\n",
    "* Make basic plots (scatter plot, histogram, bar chart, etc)\n",
    "\n",
    "## Submission Instructions:\n",
    "Please turn in this Jupyter notebook file (both .ipynb and .html formats) on Canvas before you leave the classroom. Try to get through as much as you can."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "yzYmZ_ZRVZ6h"
   },
   "source": [
    "### IMPORTANT: Replace ```?``` in the following code with your uniqname."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "NJ-VtIuoVZ6i"
   },
   "outputs": [],
   "source": [
    "MY_UNIQNAME = '?'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-YzZjtKXVZ8D"
   },
   "source": [
    "## Preface\n",
    "This lab consists of Part 0, Part 1 and Part 2. Part 0 is an introduction to some Pandas basics. There are no points assocaited with Part 0 but we will be covering it in class. Part 1 is based on a survey dataset and Part 2 uses movies and cast data. We will guide you through Part 1, and let you work on Part 2 on your own. \n",
    "\n",
    "**Question 0 is worth 2 points.  Questions 1-12 are worth 1 point each.  Questions 13-23 are worth 2 points each.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Question 0 (2 points, 0.5 points each): \n",
    "What is the output of pandas.DataFrame.describe()? (Chapter 5)\n",
    "\n",
    "\n",
    "What is the output of value_counts() work? (Chapter 5)\n",
    "\n",
    "\n",
    "What’s the difference between a series & a dataframe? (Chapter 5)\n",
    "\n",
    "\n",
    "Explain what foo_bar.groupby(’A’).sum() returns (Data Wrangling with Python and Pandas 1.9)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "4LttaTgGVZ8E"
   },
   "source": [
    "## Part 0: Background"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](assets/02.002.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](assets/02.003.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](assets/02.004.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "tGLSanHxVZ8H"
   },
   "source": [
    "* less looping over elements\n",
    "* lots of built-in functionality\n",
    "* a \"paradigm shift\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](assets/02.005.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](assets/02.006.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](assets/02.007.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](assets/02.008.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](assets/02.009.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](assets/02.010.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](assets/02.011.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "YjYixiP1VZ8H",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "# Data structures\n",
    "\n",
    "We're all familiar with lists:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "xhegqV6SVZ8I"
   },
   "outputs": [],
   "source": [
    "names = [\"Charlotte\", \"Ingrid\", \"Ian\", \"Eric\"]\n",
    "scores = [80, 95, 85, 70]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "zu3jCo_KVZ8N",
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "Now let's say that we wanted to divide each of those scores by two and assign the results to another variable. Go ahead and write some code that does that... There are lots of ways to do this, so go ahead and write one way to do it (without importing any additional python packages) and assign the results to a \n",
    "variable called ```half```:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8mhhgNVAVZ8P"
   },
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "QAnFgL-YVZ8Y"
   },
   "source": [
    "If you followed the above instructions, the following cell block should print\n",
    "a list of floats that looks like ```\n",
    "[40.0, 47.5, 42.5, 35.0]```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "niLrWWpCVZ8Z",
    "outputId": "a0bde7db-3edd-43ae-b5fe-05f4cd0367c4"
   },
   "outputs": [],
   "source": [
    "half"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "PmZ3WH5NVZ8e",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "We can put data into an array structure that allows us to apply more powerful\n",
    "functions.  The data structure that we're interested in is called an ```ndarray``` and is from the ```numpy``` package:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Hp5a1NU_VZ8f",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "ascores = np.array(scores)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8uGqlkmZVZ8j",
    "outputId": "0a568c45-e6d4-4e49-947d-cf52e23e4762",
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "ascores "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "oMPWnT1RVZ8m"
   },
   "outputs": [],
   "source": [
    "ahalf = ascores / 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "dE9-NV3xVZ8q",
    "outputId": "cc580497-7fd1-475e-86b7-911cc9b9bbca"
   },
   "outputs": [],
   "source": [
    "ahalf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "CJqs59pPVZ8w",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Numpy arrays are powerful, but they have some limitations:  they can only \n",
    "consist of one type of data (e.g. int), etc.  pandas provides two additional\n",
    "data structures that are built on numpy ndarrays.\n",
    "\n",
    "The first are Series.  Let's create a simple pandas Series and examine it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "zCY2O1qNVZ8x",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "XxeUVoa4VZ83"
   },
   "outputs": [],
   "source": [
    "sscores = pd.Series(scores,name='scores')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "mLurEmmeVZ86",
    "outputId": "82a6e021-1b0a-44ae-992b-46062e40c66c"
   },
   "outputs": [],
   "source": [
    "sscores"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iH8UUuMEVZ89",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "So you see a couple of useful things: an index (0 to 3) and a data type (dtype), which in this case is an int64.\n",
    "\n",
    "**A Series is a one-dimensional ndarray with axis labels**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "0eDglQ3VVZ8-",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "data = dict(zip(names,scores))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "zip(names,scores)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "w84Dm6g0VZ9B"
   },
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "xN7msMHZVZ9H",
    "outputId": "86fab839-41ef-40d2-88a3-417db686a732"
   },
   "outputs": [],
   "source": [
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "THyCYrt9VZ9Q"
   },
   "outputs": [],
   "source": [
    "sData = pd.Series(data=data,name='score')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "yxxn9GEkVZ9W",
    "outputId": "d9c958a9-44eb-4c6c-f48a-b8bc4e95d7f0"
   },
   "outputs": [],
   "source": [
    "sData"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iRq0P-5uVZ9b",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "So Series are a bit friendlier than numpy arrays, but they're still only one-dimensional.  Keep in mind that our basic data abstraction is a table, which can\n",
    "be thought of as a two-dimensional array.  Let's go ahead and create a simple DataFrame with just one column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "rAyN0TCDVZ9f",
    "outputId": "af63b6af-d63a-454b-b3a4-044a5d1bcd6d"
   },
   "outputs": [],
   "source": [
    "pd.DataFrame(scores,index=names,columns=['score'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "zaNqugmcVZ9i",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Let's return to the code we ran last time and walk though it just to make sure we understand it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_names = pd.read_csv('data/names.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Actually, that CSV file was created from a number of smaller CSV files.  Here's\n",
    "the code that will load the set of smaller CSV files:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WyLINje5VZ9j"
   },
   "outputs": [],
   "source": [
    "years = range(1880, 2015)\n",
    "pieces = []\n",
    "for year in years:\n",
    "    path = 'data/names/yob%d.csv' % year\n",
    "    frame = pd.read_csv(path)\n",
    "    frame['year'] = year\n",
    "    pieces.append(frame)\n",
    "df_names = pd.concat(pieces, ignore_index=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_names.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's discuss what just happened there."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "rE4uriL0VZ9l"
   },
   "source": [
    "# Part 1 (as a group): Mental Health Disorders In the Tech Workplace\n",
    "From https://www.kaggle.com/osmi/mental-health-in-tech-survey"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "N1djnH6yVZ9m"
   },
   "source": [
    "## Data Description\n",
    "\n",
    "This dataset is from a 2014 survey that measures attitudes towards mental health and frequency of mental health disorders in the tech workplace."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "PhOC6r2AVZ9n"
   },
   "source": [
    "## Metadata\n",
    "\n",
    "**Timestamp**\n",
    "\n",
    "**Age**\n",
    "\n",
    "**Gender**\n",
    "\n",
    "**Country**\n",
    "\n",
    "**state**: If you live in the United States, which state or territory do you live in?\n",
    "\n",
    "**self_employed**: Are you self-employed?\n",
    "\n",
    "**family_history**: Do you have a family history of mental illness?\n",
    "\n",
    "**treatment**: Have you sought treatment for a mental health condition?\n",
    "\n",
    "**work_interfere**: If you have a mental health condition, do you feel that it interferes with your work?\n",
    "\n",
    "**no_employees**: How many employees does your company or organization have?\n",
    "\n",
    "**remote_work**: Do you work remotely (outside of an office) at least 50% of the time?\n",
    "\n",
    "**tech_company**: Is your employer primarily a tech company/organization?\n",
    "\n",
    "**benefits**: Does your employer provide mental health benefits?\n",
    "\n",
    "**care_options**: Do you know the options for mental health care your employer provides?\n",
    "\n",
    "**wellness_program**: Has your employer ever discussed mental health as part of an employee wellness program?\n",
    "\n",
    "**seek_help**: Does your employer provide resources to learn more about mental health issues and how to seek help?\n",
    "\n",
    "**anonymity**: Is your anonymity protected if you choose to take advantage of mental health or substance abuse treatment resources?\n",
    "\n",
    "**leave**: How easy is it for you to take medical leave for a mental health condition?\n",
    "\n",
    "**mental_health_consequence**: Do you think that discussing a mental health issue with your employer would have negative consequences?\n",
    "\n",
    "**phys_health_consequence**: Do you think that discussing a physical health issue with your employer would have negative consequences?\n",
    "\n",
    "**coworkers**: Would you be willing to discuss a mental health issue with your coworkers?\n",
    "\n",
    "**supervisor**: Would you be willing to discuss a mental health issue with your direct supervisor(s)?\n",
    "\n",
    "**mental_health_interview**: Would you bring up a mental health issue with a potential employer in an interview?\n",
    "\n",
    "**phys_health_interview**: Would you bring up a physical health issue with a potential employer in an interview?\n",
    "\n",
    "**mental_vs_physical**: Do you feel that your employer takes mental health as seriously as physical health?\n",
    "\n",
    "**obs_consequence**: Have you heard of or observed negative consequences for coworkers with mental health conditions in your workplace?\n",
    "\n",
    "**comments**: Any additional notes or comments\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's load the usual libraries and also ask for plots to be rendered inside the notebook:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "7WzcvvbBVZ9p"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "JSRo9nBsVZ9p"
   },
   "source": [
    "Then read the CSV file into a DataFrame:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "7WzcvvbBVZ9p"
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"data/survey.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Sviw9MgTVZ9s"
   },
   "source": [
    "It's common to look at the resulting DataFrame using .head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "vfZrtu7xVZ9t",
    "outputId": "9981f8bc-789d-4243-c701-4f518e781373"
   },
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "PZLAWq-AVZ9y"
   },
   "source": [
    "If you want to look at a random sample, you can use .sample()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ZTZNPV6vVZ9z",
    "outputId": "0c41c5a3-117a-4194-bdfa-2ac568d4a071"
   },
   "outputs": [],
   "source": [
    "df.sample(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "lVXzJKBwVZ93"
   },
   "source": [
    "Finally, you can get some basic information about the size and shape of the DataFrame:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "35wmK1eeVZ94",
    "outputId": "6d530143-54a5-49ef-bd13-74fbb6cbb257"
   },
   "outputs": [],
   "source": [
    "print(\"The number of rows of the dataset is: \", len(df))\n",
    "print(\"The number of columns of the dataset is: \", len(df.columns))\n",
    "print(\"The shape of the dataset is: \", df.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "WM6QabC9VZ98"
   },
   "source": [
    "You can list the columns:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Oz5UXsevVZ98",
    "outputId": "753c7815-7f15-47c5-887b-f2df74868924"
   },
   "outputs": [],
   "source": [
    "df.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "MLVrG97_VZ-B"
   },
   "source": [
    "And you can extract one or more columns.  The following pair of \n",
    "commands do exactly the same thing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "2HuKMyd3VZ-B",
    "outputId": "7c05d6db-55e9-4b0d-aea5-fe6282365fda"
   },
   "outputs": [],
   "source": [
    "print(df['Country'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "sc5m2sFKVZ-H",
    "outputId": "fe857af6-702d-47f3-a0d3-e408bb9ac0dc"
   },
   "outputs": [],
   "source": [
    "country_state = df[['Country','state']]\n",
    "country_state.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "sCdBNks5VZ-K"
   },
   "source": [
    "## Extracting rows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "yvJlk3mRVZ-K",
    "outputId": "15aedfc7-b423-4694-a4f7-f631479ba7cf"
   },
   "outputs": [],
   "source": [
    "df.iloc[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "t2ixMmwDVZ-P",
    "outputId": "b2faeb7e-e480-42d0-d38c-f5dc3b72cb37"
   },
   "outputs": [],
   "source": [
    "df.loc[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "DjG0tSoNVZ-S"
   },
   "outputs": [],
   "source": [
    "df_gender = df.set_index('Gender')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "UpkWJyHGVZ-U",
    "outputId": "f17d6766-c2fe-46d0-c746-e4c657792cf3"
   },
   "outputs": [],
   "source": [
    "df_gender.loc['Male']\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "vQx4rzahVZ-a",
    "outputId": "95d2d95a-2888-4284-93f6-dd1673f1b429"
   },
   "outputs": [],
   "source": [
    "df.iloc['Gender'] # generates error"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Y5gxfBo5VZ-c",
    "outputId": "fe806a10-ebfc-49a8-eb38-3b17ed407ca2"
   },
   "outputs": [],
   "source": [
    "df.iloc[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "zBjjcPHZVZ-i"
   },
   "source": [
    "## Sorting\n",
    "You can use either sort_values() or sort_index():\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "2TQQb4fkVZ-i",
    "outputId": "d52d4824-f7f5-41ea-ff1e-2f460b2d1277"
   },
   "outputs": [],
   "source": [
    "df_sorted = df.sort_values('Age')\n",
    "df_sorted.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "1Kx8cgRcVZ-n"
   },
   "source": [
    "## Filtering using Boolean Masking"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.Age"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['Age'] > 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[df['Age'] > 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['Age'] > 40"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df[  df['Age'] > 40  ]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "LiI5pI5HVZ-o"
   },
   "source": [
    "### Example: Find people who reported a family history of mental health conditions. </font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Y9ny0e_TVZ-q"
   },
   "source": [
    "Solution:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "T_Op5bxLVZ-q",
    "outputId": "58dd3e60-d095-497d-e7f3-887cda3bf94b"
   },
   "outputs": [],
   "source": [
    "family_history_mask = df.family_history=='Yes'\n",
    "df[family_history_mask].tail(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "UwMZ0M4pVZ-t"
   },
   "source": [
    "You can use a simple expression like ```df[df['family_history'] == 'Yes']``` or you can make more complex boolean expressions using parentheses: \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Z6ma1OzsVZ-v",
    "outputId": "5f7fe8c7-4133-4d9f-cb61-bbd7779f485c"
   },
   "outputs": [],
   "source": [
    "df_filtered = df[(df['family_history'] == 'Yes') & (df['treatment'] == 'Yes')]\n",
    "df_filtered.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "3UNhgIHjVZ-y"
   },
   "source": [
    "### <font color=\"red\">Q1: How many people are willing to discuss a mental health issue with their supervisor or their coworkers? </font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "2uanzG8zVZ-z",
    "outputId": "30f233de-79f0-402d-f4c4-24659b7e137b"
   },
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "s5Wu-JcYVZ-3"
   },
   "source": [
    "### <font color=\"red\">Q2: Make a new DataFrame ```df_millenials``` with only millennials (born between 1976 and 1996). Make appropriate assumptions when constructing your filter. </font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "dgNzrGCnVZ-4"
   },
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "n8onap2fVZ-8"
   },
   "source": [
    "**NOTE: We will still use df for the following analysis**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2NM77kaZVZ-9"
   },
   "source": [
    "## Descriptive and Summary Statistics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "vGZSCd5tVZ_A"
   },
   "source": [
    "Example: What is the mean age of the survey sample?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9HI-vM2kVZ_B"
   },
   "source": [
    "Solution:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "2zrN3A_LVZ_B",
    "outputId": "92a53bd1-dc67-456c-f534-f696c81320ab"
   },
   "outputs": [],
   "source": [
    "df['Age'].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Does that look right?  What should we do?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "1niH0x8fVZ_E"
   },
   "source": [
    "### <font color=\"red\">Q3: What is the _median_ age of the survey sample?  </font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "kbbISqYpVZ_F",
    "outputId": "1582ffd1-5bc4-4fb1-bc28-60108a5c3b1e"
   },
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "1neFC7WFVZ_H"
   },
   "source": [
    "### <font color=\"red\">Q4: Write one line of code to compute basic statistics (mean, standard deviation, min, 25% percentile, etc) about Age  </font>\n",
    "\n",
    "Hint: see the readings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "1iIWxtHqVZ_I",
    "outputId": "29f693a3-4dbf-4111-cbe2-f8fa61718798"
   },
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gW_LQGwxVZ_K"
   },
   "source": [
    "## Unique Values, Counts, Membership"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "3TTfX0gdVZ_K"
   },
   "source": [
    "Example: Write one line of code to check unique values of Gender</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "KBPIDc2WVZ_L"
   },
   "source": [
    "Solution:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.coworkers.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "y2DMcfrsVZ_L",
    "outputId": "3ede4799-6d99-4223-b9b9-8da7e4be8bdb"
   },
   "outputs": [],
   "source": [
    "df.Gender.unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9MszMiO-VZ_O"
   },
   "source": [
    "Example: Write one line of code to count the occurrences of the countries and show the top 5 countries.  </font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "7vNBUsIMVZ_P"
   },
   "source": [
    "Solution:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "D9SvONy5VZ_R",
    "outputId": "907cfc2e-a455-4f62-dd8f-2a2ba28aa944"
   },
   "outputs": [],
   "source": [
    "df.Country.value_counts().head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Are you sure that's correct?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "s4X2vNrgVZ_Y"
   },
   "source": [
    "### <font color=\"red\">Q5: Find the unique categories of no_employees. What is the frequency of each category? </font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "4e4yFxHaVZ_Z",
    "outputId": "5e397ead-7a42-4a61-87fa-20df4b0e5045"
   },
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "SRRfpLm4VZ_c"
   },
   "source": [
    "### <font color=\"red\">Q6: How many Michiganders aged between 20 and 30 sought treatment for a mental health condition? How many did not? </font>\n",
    "Show your code and also include a markdown block with a complete sentence or two that answers these questions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "zOfjr2vHVZ_d",
    "outputId": "9f5eec08-53e3-4473-83e4-4c795864324b"
   },
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "BDXeB5sdVZ_j"
   },
   "source": [
    "Replace this with you answer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "UK51a2wpVZ_k"
   },
   "source": [
    "### <font color=\"red\">Q7: Among the people from United States, how many repondents were there from each state?  </font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ewBvXnp2VZ_l",
    "outputId": "acba1da4-1310-4317-a401-b7f725c69c12"
   },
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Chh5GIJlVZ_n"
   },
   "source": [
    "## Basic Plots"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "7QwUQ7xdVZ_n"
   },
   "source": [
    "Example: Investigate the proportion (%) of people receiving health benefits from their employers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gXHPzifTVZ_o"
   },
   "source": [
    "Solution:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "K5dDt8VTVZ_r",
    "outputId": "59d6a47b-e6d2-4125-d648-80db845840fe"
   },
   "outputs": [],
   "source": [
    "df.benefits.value_counts(normalize=True).plot.bar()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example: Create a histogram of the distribution of Age values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "3aYfJAfpVZ_w",
    "outputId": "5f6e7a2c-2353-4f98-b4ae-b13d184a9cc5"
   },
   "outputs": [],
   "source": [
    "df.Age.value_counts().plot.hist()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <font color=\"red\">Q8: Experiment with the number of bins in the histogram of the Age distribution:</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fM3RDXkeVZ_v"
   },
   "source": [
    "Hint: use the bins= option to plot()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# insert your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "G4W5ERXPVZ_y"
   },
   "source": [
    "## Discretization and Binning\n",
    "Discretization of continuous variables can potentially enhance the goodness-of-fit of a statistical model. Age, for example, is a common varaible to be categorized.\n",
    "\n",
    "### <font color=\"red\">Q9: </font> Discretize the age into bins (16,21], (21,25], (25,30], (30, 35], (35,40], (40,50], (50,60], (60,).  \n",
    "\n",
    "How many respondents are in each range? \n",
    "Hint: create a list of bin values\n",
    "Hint: use pd.cut()\n",
    "Hint: use value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Y4mlZZtSVZ_y"
   },
   "outputs": [],
   "source": [
    "bins = [15,20,25,30,35,40,50,60,200]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "3-PEDPaJVZ_0"
   },
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "GDbfYtuEVZ_8"
   },
   "source": [
    "## Aggregation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "90l0AuGfVZ_9"
   },
   "source": [
    "Example: Find the size of each Age group."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "LurKzxlWVZ_-"
   },
   "source": [
    "Solution:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "5GQRlKaeVZ_-",
    "outputId": "f04c450c-e9af-4989-af28-3a0df2639a6a"
   },
   "outputs": [],
   "source": [
    "df.groupby('Age_group').size()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9Jh8LMd5VaAA"
   },
   "source": [
    "### <font color=\"red\">Q10: Find the median age of each state. </font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Bk-Oeau1VaAB",
    "outputId": "84888e1e-42f5-4f45-e76c-d5defbcab20c"
   },
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Ala4108AVaAE"
   },
   "source": [
    "# Part 2 (on your own): Exploration of Movie Titles and Movie Cast"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "3cJ_P6-OVaAI"
   },
   "source": [
    "## Time to load some data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8h74mSPfVaAL"
   },
   "outputs": [],
   "source": [
    "titles = pd.read_csv('data/titles.csv', index_col=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iltVCIzzVaAP"
   },
   "source": [
    "The titles DataFrame contains a list of movie titles and release year"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "sDtWZcdDVaAP"
   },
   "outputs": [],
   "source": [
    "cast = pd.read_csv('data/cast.zip', index_col=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "QmRKuZ2lVaAQ"
   },
   "source": [
    "The ```cast``` DataFrame contains the following columns \n",
    "\n",
    "**title** = name of movie\n",
    "\n",
    "**year** = year of movie\n",
    "\n",
    "**name** = name of actor/actress\n",
    "\n",
    "**type** = actor or actress\n",
    "\n",
    "**character** = character name\n",
    "\n",
    "**n** = number in the credits (NaN when not available)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "vUJbzigkVaAR",
    "outputId": "581af96c-82f1-461c-a08d-5a74b4875a6e"
   },
   "outputs": [],
   "source": [
    "titles.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "cTwkedBiVaAU",
    "outputId": "a3597a7a-f598-4fdd-f6b6-39be1c2d0953"
   },
   "outputs": [],
   "source": [
    "cast.sample(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "20egAUt2VaAW"
   },
   "source": [
    "### Q11: How many entries are there in the cast table?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fDBtA5EPVaAa"
   },
   "source": [
    "### Q12: How many entries are there in the titles table?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "JfdCBERDVaAb",
    "outputId": "a4e81b19-3449-48f9-8108-2e2cbecb954c"
   },
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ZV5eEDrtVaAf"
   },
   "source": [
    "## Some basic analyses"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "cGC68wHGVaAg"
   },
   "source": [
    "### Q13: What are the two earliest movies?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "nZf39Ow4VaAg",
    "outputId": "a1a6285e-93af-46a3-eaf2-e0c61361b524"
   },
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "vdacpNIqVaAj"
   },
   "source": [
    "### Q14: How many movies have the title \"Hamlet\"?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "KRs9FLM9VaAj",
    "outputId": "7ce5b3b6-4848-402c-cca5-6b2d0861431e"
   },
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "4hNG1A9CVaAp"
   },
   "source": [
    "### Q15: List all of the \"Treasure Island\" movies from earliest to most recent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Rhf67jPPVaAp",
    "outputId": "ff83ba46-a80b-43ee-dbe1-4b2226f61f5f"
   },
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "n6ekJFVmVaAu"
   },
   "source": [
    "### Q16: List the supporting roles (having n=2) played by Cary Grant in the 1940s,\n",
    "in order by year."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6MgR83iQVaAu",
    "outputId": "b2c79b39-d14d-4d37-fd5e-7d1514885d15"
   },
   "outputs": [],
   "source": [
    "#(cast.year >= 1940) & (cast.year < 1950)\n",
    "#(cast.year == 1940) | (cast.year == 1941)... \n",
    "# cast.year // 10 == 194\n",
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "0nigWnfaVaAw"
   },
   "source": [
    "### Q17: What are the ten most common movie names of all time?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "7vo-EPlZVaAw",
    "outputId": "30cd67a7-1f47-4a1f-ff9a-3060ed5ca7a4"
   },
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "puDsuQMgVaAy"
   },
   "source": [
    "### Q18: Plot the number of \"Hamlet\" films made each decade\n",
    "\n",
    "Make sure your decades are in chronological order on the x-axis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "VW5tJBjCVaAz",
    "outputId": "bb560e2d-c531-4805-8563-a17af893f2ea"
   },
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "0e8ojF-uVaA0"
   },
   "source": [
    "### Q19: Who are the 10 people most often credited as \"Herself\" in film history?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "xZ3Ar2hPVaA1",
    "outputId": "2441de5e-b683-405d-9c7e-cd79ee07c138"
   },
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "nTo4QMqZVaA3"
   },
   "source": [
    "### Q20: What are the 10 most frequent roles that start with the word \"Science\"?\n",
    "Hint: read docs on str.startswith()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WP5VMIAqVaA4",
    "outputId": "aa267acd-6e2e-47de-c0c4-4e60084a5ca1"
   },
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "eKpdayrYVaA7"
   },
   "source": [
    "### Q21: How many leading (n=1) roles were available to actors, and how many to actresses, in the 1950s?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "1mYaTynCVaA8",
    "outputId": "7bf60f15-94e1-4a12-bcea-6f6e1e19e8b7"
   },
   "outputs": [],
   "source": [
    "# insert code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bfTRaAmFVaBB"
   },
   "source": [
    "### Q22: How many supporting (n=2) roles were available to actors,and how many to actresses, in the 1950s?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "eT2piDNlVaBC",
    "outputId": "142cebf9-f2d8-42b1-d7f1-4215cb40e7d1"
   },
   "outputs": [],
   "source": [
    "# insert code here\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "wHN-cI-dVaBE"
   },
   "source": [
    "### Q23: Comment on the differences in gender ratios for leading vs. supporting roles in the 1950s.  Does there appear to be a bias?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [
    "N1djnH6yVZ9m",
    "PhOC6r2AVZ9n",
    "sCdBNks5VZ-K",
    "zBjjcPHZVZ-i",
    "1Kx8cgRcVZ-n",
    "3UNhgIHjVZ-y",
    "s5Wu-JcYVZ-3",
    "2NM77kaZVZ-9",
    "1niH0x8fVZ_E",
    "1neFC7WFVZ_H",
    "gW_LQGwxVZ_K",
    "s4X2vNrgVZ_Y",
    "SRRfpLm4VZ_c",
    "UK51a2wpVZ_k",
    "Chh5GIJlVZ_n",
    "9mJUVrJLVZ_v",
    "G4W5ERXPVZ_y",
    "GDbfYtuEVZ_8",
    "9Jh8LMd5VaAA",
    "3cJ_P6-OVaAI",
    "20egAUt2VaAW",
    "fDBtA5EPVaAa",
    "ZV5eEDrtVaAf",
    "cGC68wHGVaAg",
    "vdacpNIqVaAj",
    "4hNG1A9CVaAp",
    "n6ekJFVmVaAu",
    "0nigWnfaVaAw",
    "puDsuQMgVaAy",
    "0e8ojF-uVaA0",
    "nTo4QMqZVaA3",
    "eKpdayrYVaA7",
    "bfTRaAmFVaBB"
   ],
   "name": "618_02_Pandas_I.ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}