01_statistics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Welcome to the Dark Art of Coding:\n",
    "## Introduction to Data Science Fundamentals\n",
    "Basic Statistics\n",
    "\n",
    "From count, min, max to variance/standard deviation\n",
    "\n",
    "<img src='images/logos.3.600.wide.png' height='250' width='300' style=\"float:right\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Main objectives\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At the end of this lesson, you will be able to:\n",
    "\n",
    "* Understand how to calculate counts, minimums and maximums\n",
    "* Understand the measures of central tendencies and the differences between: \n",
    "   * mean\n",
    "   * median\n",
    "   * mode\n",
    "* Understand measures of dispersion and the differences between:\n",
    "   * simple spread\n",
    "   * quantiles\n",
    "   * interquartile ranges\n",
    "   * variance\n",
    "   * standard deviation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A broad overview\n",
    "---\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To help us understand some of the concepts we are gonna explore, we will explore most of these measures in several ways:\n",
    "\n",
    "* through small, simple datasets\n",
    "* through a larger more sophisticated dataset\n",
    "* through hand-developed code\n",
    "* through related functions found in Python's Standard Library and in major data science libraries such as numpy, scipy, and pandas "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get us started, let's take a look at our large dataset. To do so, we will plot the data using the `matplotlib` library. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defects in batches of parts\n",
    "\n",
    "Let's simulate the number of defects within a series of part batches.\n",
    "\n",
    "Given any batch of parts, some of the parts will be defective.\n",
    "\n",
    "Each number below represents the number of defects within a given batch.\n",
    "\n",
    "**NOTE**: I manually organized these by roughly by size to make it easier to discuss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "num_defects =  [99, 52, 42, 44, 27, 22, 22, 24, 22, 21,\n",
    "                21, 18, 18, 15, 19, 19, 18, 17, 18, 16,\n",
    "                16, 17, 18, 15, 15, 13, 11, 14, 14, 13,\n",
    "                14, 14, 14, 13, 14, 14, 14, 14, 12, 14,\n",
    "                13, 12, 11, 12, 13, 13, 12, 13, 11, 13,\n",
    "                11, 12, 12, 10, 10, 11, 12, 13, 10, 12,\n",
    "                11, 10, 12, 12, 11, 11, 9, 9, 9, 11,\n",
    "                12, 8, 8, 10, 8, 10, 11, 10, 9, 10,\n",
    "                10, 8, 8, 11, 10, 9, 8, 8, 10, 10,\n",
    "                7, 7, 7, 8, 8, 7, 7, 8, 7, 7,\n",
    "                9, 7, 7, 10, 7, 8, 9, 9, 6, 6,\n",
    "                8, 8, 9, 7, 7, 9, 6, 8, 9, 8,\n",
    "                9, 9, 6, 8, 9, 7, 8, 6, 8, 6,\n",
    "                6, 6, 6, 6, 6, 8, 7, 8, 7, 5,\n",
    "                6, 8, 8, 8, 8, 5, 4, 6, 6, 4,\n",
    "                6, 6, 4, 5, 7, 5, 7, 6, 5, 4,\n",
    "                6, 4, 7, 5, 6, 4, 4, 6, 6, 3,\n",
    "                5, 6, 6, 4, 6, 3, 3, 3, 5, 5,\n",
    "                5, 3, 2, 5, 2, 3, 2, 4, 3, 2,\n",
    "                5, 2, 2, 2, 3, 4, 4, 4, 5, 5,\n",
    "                3, 5, 4, 2]         "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# The following enables inline plots\n",
    "# and import matplotlib.pyplot uses plt as an alias\n",
    "\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# importing the Counter class helps us simplify the counting process\n",
    "\n",
    "from collections import Counter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Simple Histogram\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will start off with a simple display of the most frequent values in the dataset using `Counter`'s `.most_common()` method.\n",
    "\n",
    "From there, we will use a histogram to help visualize our defects data and then we will use various statistical evaluations to examine that data in greater detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's count how many times each defect count appears\n",
    "#     and look at the five most frequent counts\n",
    "\n",
    "defect_counts = Counter(num_defects)\n",
    "\n",
    "for defect_count, freq in defect_counts.most_common(5):\n",
    "    print(defect_count, '>  ', freq)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Counters behave a lot like dictionaries, but...\n",
    "\n",
    "For those who have NOT used Counters before should know that while they might look a lot like dictionaries they not only have the addition `.most_common()` method, they also have another interesting feature\n",
    "\n",
    "If you look up a key that does not exist, you get back a `0` value (versus a `KeyError` that you would get back from a dictionary).\n",
    "\n",
    "For example...\n",
    "\n",
    "```\n",
    ">>> defect_count[77]     # does NOT exist in the Counter\n",
    "0\n",
    "\n",
    ">>> defect_count[8]      # does exist\n",
    "24\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's create a series of x values and y values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# We will start off by creating a series of x values\n",
    "#     and then parse the defect counts for each x value.\n",
    "# In this case, if we try to extract the value associated with a key\n",
    "#     that does not exist, we simply get back a 0.\n",
    "\n",
    "x_values = list(range(101))\n",
    "y_values = [defect_counts[x] for x in x_values]\n",
    "\n",
    "print(x_values)\n",
    "print()\n",
    "print(y_values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's craft a bar chart using our x and y values...\n",
    "#     For now, I don't want to focus too much on the mechanics of creating the \n",
    "#     graph... just on what the graph can tell us at a glance\n",
    "\n",
    "#     We will use this picture to help us get a sense for what the data looks like\n",
    "\n",
    "plt.figure(figsize=(12, 10))\n",
    "plt.bar(x_values, y_values)\n",
    "plt.axis([0, 105, 0, 25])\n",
    "plt.title('Frequency of defects detected')\n",
    "plt.xlabel('# of defects')\n",
    "plt.ylabel('# of batchs')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Simple statistics: "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* count\n",
    "* maximum\n",
    "* minimum"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# len() returns the total number of elements in a Python sequence\n",
    "\n",
    "num_points = len(num_defects)\n",
    "\n",
    "print(\"Number of data points:\", num_points)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# max() and min() return the maximum (or minimum) value from a \n",
    "#     Python sequence\n",
    "\n",
    "largest_value = max(num_defects)\n",
    "smallest_value = min(num_defects)\n",
    "\n",
    "print(\"Largest value:\", largest_value)\n",
    "print(\"Smallest value:\", smallest_value)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Central Tendencies"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Measures of central tendency (also called measures of location) give us an idea of what constitutes a typical or common value for a given data set. The three most common measures of central tendency are: \n",
    "\n",
    "* mean (also referred to as the arithmetic mean)\n",
    "* median\n",
    "* mode\n",
    "\n",
    "To start off, we will create our own code to calculate these values, but we will shortly turn to more efficient functions offered by Python Libraries. The purpose of the dive into handmade functions to give ourselves a chance to discuss the principles involved."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## mean"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The arithmetic mean is what we commonly refer to as the average. It is generally calculated as the total OR sum of all the values divided by the number of values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def mean(values):\n",
    "    '''Return the mean (or average) of a sequence of values.\n",
    "    \n",
    "    >>> mean([1, 2, 3, 4])\n",
    "    2.5\n",
    "    \n",
    "    >>> mean([1, 2, 3, 4, 5])\n",
    "    3.0\n",
    "    \n",
    "    '''\n",
    "    return sum(values) / len(values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# temporarily, let's use a small number of values\n",
    "\n",
    "mean([1, 2, 3, 4])         # even number of values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean([1, 2, 3, 4, 5])      # odd number of values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's apply this to our larger dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean(num_defects)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## median"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The median is the center-most value in a sorted sequence of values.\n",
    "\n",
    "**NOTE**: If there are an even number of items, the median is the average of the two center-most values.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def median(values):\n",
    "    \"\"\"Return the median value from a sequence of values\n",
    "    \n",
    "    >>> median([1, 3, 5])          # odd number of values\n",
    "    3\n",
    "    \n",
    "    >>> median([1, 3, 5, 7])       # even number of values\n",
    "    4.0\n",
    "    \"\"\"\n",
    "    \n",
    "    sorted_values = sorted(values)\n",
    "    num = len(values)\n",
    "    centerpoint = num // 2         # find the index for the midpoint, truncating\n",
    "                                   #     any floats...\n",
    "    \n",
    "    if num % 2 == 1: \n",
    "        # return the center value \n",
    "        #     if n is ODD\n",
    "        return sorted_values[centerpoint]\n",
    "    \n",
    "    else:\n",
    "        # return the average of the two center-most values\n",
    "        #     if num is EVEN\n",
    "        c1 = centerpoint - 1\n",
    "        c2 = centerpoint\n",
    "        return (sorted_values[c1] + sorted_values[c2]) / 2\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# median of an odd number of values\n",
    "\n",
    "odd = [1, 2, 3]\n",
    "\n",
    "median(odd)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# median of an even number of values\n",
    "# REMINDER: median sorts the values in the sequence before \n",
    "#     returning any values\n",
    "\n",
    "even = [42, 1, 5, 6]\n",
    "\n",
    "median(even)    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# median num of tweets recieved\n",
    "\n",
    "median(num_defects)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## mode"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The mode is the most frequently found value in a sequence of values.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from collections import Counter\n",
    "\n",
    "def mode(values):\n",
    "    \"\"\"Returns a list of the most common (frequent) value(s) \n",
    "    \n",
    "    If there is more than one element with the SAME maximum frequency, then\n",
    "    return all such elements as a list\n",
    "    \"\"\"\n",
    "    \n",
    "    mode_values = {}\n",
    "    counts = Counter(values)\n",
    "    max_count = max(counts.values())\n",
    "    \n",
    "    return [value for value, count in counts.items() if count == max_count]        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mode([1, 2, 3, 4, 4, 4])     # single number (4) is the most frequent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mode([1, 1, 1, 2, 3, 4, 5, 5, 5])     # two numbers (1 and 5) are most frequent\n",
    "                                      # this dataset has more than one trend >>> it is called bimodal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mode(num_defects)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## mean vs median vs mode"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For a quick review of the differences between mean, median and mode:\n",
    "\n",
    "||Calculation|Application|\n",
    "|:--|:---|:---|\n",
    "|mean|sum of values/number of values|symmetrical data with only one trend|\n",
    "|median|middle value of a sorted sequence of values|skewed data with outlier(s)|\n",
    "|mode|most frequent value(s)|categorical data OR data with more than one trend/cluster|\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## frequency table\n",
    "\n",
    "Along with `mean`, `mode`, `median` it is sometimes useful to see more broadly how frequently the various values appear. A frequency table is useful for this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from collections import Counter\n",
    "\n",
    "def freq_table(values):\n",
    "    \"\"\"Returns a series of values and counts in a frequency table \n",
    "    \"\"\"\n",
    "    \n",
    "    frequencies = Counter(values)\n",
    "    print('Value\\tFrequency')\n",
    "    \n",
    "    for value, count in frequencies.most_common():\n",
    "        print('{}\\t{}'.format(value, count))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "freq_table([7, 8, 9, 9, 9, 6, 6])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "freq_table(defect_counts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Experience Points\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Complete the following exercises:\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**A**. Given a list of values:\n",
    "`values = [1, 2, 3, 4, 5, 11, 12, 13, 14, 15]`\n",
    "\n",
    "Calculate the following results:\n",
    "\n",
    "* mean\n",
    "* median\n",
    "* mode\n",
    "* number of values (count)\n",
    "* maximum value\n",
    "* minimum value\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**B**. Given a sequence of values available from the Python `range()` object starting at 100 up to, but not including 1000 with a step of 3 (i.e. `100, 103, 106 ... 994, 997`):\n",
    "\n",
    "`nums = range(100, 1000, 3)`\n",
    "\n",
    "Calculate the following results:\n",
    "\n",
    "* mean\n",
    "* median\n",
    "* mode\n",
    "* number of values (count)\n",
    "* maximum value\n",
    "* minimum value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you (and your partner, if you're working in pairs) are done, then you can put your green sticky up! This is how we know you're done.\n",
    "\n",
    "<img src='images/green_sticky.300px.png' width='200' style='float:left'>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Measures of Dispersion\n",
    "\n",
    "Dispersion is a representation of how varied OR spread out our data is. Dispersion may also be refered to as measures of variability OR measures of spread."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simple spread\n",
    "\n",
    "Using the maximum and minimum values, you can calculate the range of spread."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def data_range(x):\n",
    "    '''Returns the range (i.e. the difference) between the \n",
    "    highest and lowest values\n",
    "    '''\n",
    "    \n",
    "    return max(x) - min(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_range([5, 3, 4, 2, 1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_range(num_defects)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Downsides of using simple spread:\n",
    "\n",
    "A simple spread is very susceptible to outliers which may not take into account where the **majority** of the datapoints lie...\n",
    "\n",
    "In the first example, below, the majority of the values are near 7. In the second example, with one exception, the majority of the values are still near 7, but the ranges are vastly different.\n",
    "\n",
    "```\n",
    "Example 1: [7, 6, 7, 5, 7, 8]    # range of 3 \n",
    "Example 2: [7, 6, 7, 5, 7, 99]   # range of 94\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Quantiles and interquartile ranges\n",
    "\n",
    "A method to overcome the weakness of simple spread is to use a narrower range. One way to create a narrower range is to calculate quantiles and interquartile ranges.\n",
    "\n",
    "|Term|Definition|\n",
    "|:--- |:--- |\n",
    "|quantile|a cutpoint OR dividing point in a range of values creating contiguous intervals with equal probabilities|\n",
    "|quartile|a value that cuts your data into quarters|\n",
    "|interquartile range|the range between the lower quartile and upper quartile|"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are gonna construct some basic functions. Please note: these are for example only and lack some of the sophistication needed to be used in production. This lack of sophistication yields a lack of precision, etc. But they should be good enough for government work."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Quantiles"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Quantiles are cut points that divide sequence of observations in a sample space into contiguous intervals/groups with equal probabilities.\n",
    "\n",
    "In the following example we choose a simple cut point that breaks the sample values into two groups. The cut point effectively marks the median.\n",
    "\n",
    "<img src='images/quantile_halvsies.jpg' height='600' width='600'>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this next example we choose nine cut points that break the sample values into ten equally-sized groups. \n",
    "\n",
    "The cut points mark the 10%, 20%, 30% through 90% points of the data. This more refined set of cut points allows us to fine-tune our approach to filtering out out-liers. As shown in this image, our range would be all values between the 10% and 90% cut points.\n",
    "\n",
    "<img src='images/quantile_percentile.jpg' height='600' width='600'>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With this in mind, let's create our first function to help calculate where a given quantile falls."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def quantile(values, percentile):\n",
    "    \"\"\"Returns the pth-percentile value in a sequence of values\n",
    "    \"\"\"\n",
    "    \n",
    "    p_index = int(percentile * len(values))\n",
    "    return sorted(values)[p_index]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Given a sequence of values, we can calculate a given quantile.\n",
    "# In this case, we calculate the quantile at the halfway mark (50% OR 0.5)\n",
    "#     or roughly the median.\n",
    "\n",
    "grades1 = [85, 87, 89, 55, 65, 67, 75, 80, 91, 96]        # ten values\n",
    "\n",
    "quantile(grades1, .5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grades2 = [85, 87, 89, 55, 65, 67, 75, 80, 91, 96, 99]    # eleven values\n",
    "\n",
    "quantile(grades2, .5)\n",
    "\n",
    "# NOTE: As described above, this function lacks some details\n",
    "#     it is less detailed than even our median function\n",
    "#     and does NOT account for the length of the dataset in\n",
    "#     terms of odd/even num of values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# While not as sophisticated as our median calc,\n",
    "#     we can generally calculate other quantiles, such as the \n",
    "#     the 25th percentile\n",
    "#     or the 75th percentile\n",
    "#     i.e. the quartiles\n",
    "\n",
    "print('25%: ', quantile(grades2, .25))\n",
    "print('75%: ', quantile(grades2, .75))\n",
    "\n",
    "# sorted for comparison: [55, 65, 67, 75, 80, 85, 87, 89, 91, 96, 99]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To see the percentile breakdown of defects, we can calculate which defect counts fall into which percentiles"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is important to note that the percentile calc simply finds the **NEAREST** index in the defect list to the given percentile and returns the value stored at that index."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for percent in [0.10, 0.25, 0.75, 0.90, 0.95, 0.999]:\n",
    "    q = quantile(num_defects, percent)\n",
    "    \n",
    "    print('Percent: {}\\t{}'.format(percent, q))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Interquartile ranges"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To show the range between two quantile values, we can use an interquantile range.\n",
    "\n",
    "It is very common to use quantiles of 25% and 75%, but other quantiles may be used as well:\n",
    "* 10% <-> 90%\n",
    "* 20% <-> 80%\n",
    "* 30% <-> 70%\n",
    "\n",
    "It really depends on how much you want to filter out outliers AND your definition of outliers.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This image is focused on the 25% and 75% cut points which are typically called quartiles because the values are divided into quarters:\n",
    "\n",
    "<img src='images/quantile_quartile.jpg' height='600' width='600'>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def interquartile_range(values):\n",
    "    '''Return the difference between the 75% and 25% percentiles.\n",
    "    '''\n",
    "    \n",
    "    return quantile(values, 0.75) - quantile(values, 0.25)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "values = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]\n",
    "\n",
    "# Let's first just confirm the quantile VALUES at the 25% and 75% cut points\n",
    "print(quantile(values, 0.25), '<->', quantile(values, 0.75))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now, let's calculate the interquartile range\n",
    "\n",
    "print(interquartile_range([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We can calculate the interquartile range for our defects data (presuming 75% and 25%)\n",
    "\n",
    "interquartile_range(num_defects)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def interquartile_range(values, upper_bound=0.75, lower_bound=0.25):\n",
    "    '''Return the difference between the 75% and 25% percentiles.\n",
    "    '''\n",
    "    \n",
    "    return quantile(values, upper_bound) - quantile(values, lower_bound)\n",
    "\n",
    "interquartile_range(num_defects, 0.9, 0.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Downsides of using interquartile range\n",
    "\n",
    "The interquartile range helps eliminate some of the risks associated with outliers, but still has some weaknesses:\n",
    "\n",
    "* it can only tell you the difference between a 'high' value and a 'low' value\n",
    "* it does NOT tell you how often the high values OR low values occur versus how often values occur near the center\n",
    "\n",
    "We need a method to more accurately measure variability."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Experience Points\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Complete the following exercises:\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**A**. Given a list of values:\n",
    "`values = [1, 2, 3, 4, 5, 11, 12, 13, 14, 15]`\n",
    "\n",
    "Calculate the following results:\n",
    "\n",
    "* calculate a simple spread\n",
    "* calculate the 0.30 and 0.70 quantiles\n",
    "* calculate the interquartile range for 0.15 and 0.85\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**B**. Given a sequence of values available from the Python `range()` object starting at 200 up to, but not including 300 with a step of 5 (i.e. `200, 205, 210 ... 290, 295`):\n",
    "\n",
    "`nums = range(200, 300, 5)`\n",
    "\n",
    "\n",
    "Calculate the following results:\n",
    "\n",
    "* calculate a simple spread\n",
    "* calculate the 0.15 and 0.85 quantiles\n",
    "* calculate the interquartile range for 0.25 and 0.75\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you (and your partner, if you're working in pairs) are done, then you can put your green sticky up! This is how we know you're done.\n",
    "\n",
    "<img src='images/green_sticky.300px.png' width='200' style='float:left'>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Variance and standard deviation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are several techniques to help evaluate the spread of the data and how much data varies. These techniques build upon each other."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**DON'T DO THIS:  Average difference from the mean**\n",
    "\n",
    "`Average difference from the mean`: One method that **seems**, at first look, to be a reasonable solution is to simply calculate the average difference of each value from the mean.\n",
    "\n",
    "The weaknesss with calculating the average distance from the mean is that positive values and negative values cancel each other out and you lose the measure of spread:\n",
    "\n",
    "```\n",
    "nums = [1, 2, 9]\n",
    "mean: 4\n",
    "\n",
    "* 1 - 4 = -3\n",
    "* 2 - 4 = -2\n",
    "* 9 - 4 =  5\n",
    "------------\n",
    "total   =  0\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def diff_mean(values):\n",
    "    \"\"\"Calculate the difference from the mean for every value in a \n",
    "    sequence of values\n",
    "    \n",
    "    This has the effect of translating the values so that the new mean\n",
    "    is zero (0).\n",
    "    \"\"\"\n",
    "    \n",
    "    x_bar = mean(values)\n",
    "    return [value - x_bar for value in values]    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "diffs = diff_mean([1, 2, 9])\n",
    "\n",
    "mean(diffs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mean(diff_mean(num_defects))\n",
    "\n",
    "# 1.8634331550571255e-14\n",
    "# which is effectively ZERO... "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Variance**\n",
    "\n",
    "`Variance`: To calculate variation from the mean, a popular method is to calculate the variance, which is defined as:\n",
    "\n",
    "`The average of the 'difference OR distance of values from the mean' squared.`\n",
    "\n",
    "The benefit of **squaring** each of the \"difference from the mean\" values before using them to calculate the average is that squared numbers are **always positive**.\n",
    "\n",
    "```\n",
    "nums = [1, 2, 9]\n",
    "mean: 4\n",
    "\n",
    "* (1 - 4)^2 =  9\n",
    "* (2 - 4)^2 =  4\n",
    "* (9 - 4)^2 = 25\n",
    "----------------\n",
    "total       = 38\n",
    "\n",
    "variance    = 38/3 = 12.6666\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def variance(values):\n",
    "    \"\"\"Return the variance of a sequence of values.\n",
    "    \n",
    "    NOTE: this functions presumes that values has a minimum of TWO elements.\n",
    "    \"\"\"\n",
    "    \n",
    "    num = len(values)\n",
    "    deviations = diff_mean(values)\n",
    "    squared_diffs = [d ** 2 for d in deviations]\n",
    "    sum_squared_diffs = sum(squared_diffs)\n",
    "    \n",
    "    return sum_squared_diffs / num"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "variance([1, 2, 9])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "variance(num_defects)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Downsides of using variance\n",
    "\n",
    "One of the weaknesses of using the variance is that no one really knows what it means when we say **`distance squared`** or **`difference squared`**.\n",
    "\n",
    "Because we are using squared values, most folks don't have a good sense of how the variance relates to the actual dataset. In our example above, the variance is ~80 which is really close to the biggest value in our dataset... 99. This can be somewhat confusing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Standard Deviation**\n",
    "\n",
    "`Standard Deviation`: To overcome this weakness we often calculate the standard deviation, which is defined as:\n",
    "\n",
    "`The square root of the variance`\n",
    "\n",
    "The benefit of the square root is that we are simply using a distance OR a difference.\n",
    "\n",
    "```\n",
    "nums = [1, 2, 9]\n",
    "mean: 4\n",
    "\n",
    "* (1 - 4)^2 =  9\n",
    "* (2 - 4)^2 =  4\n",
    "* (9 - 4)^2 = 25\n",
    "----------------\n",
    "total       = 38\n",
    "\n",
    "std dev     = sqrt(38/3) = 3.5590\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using the standard deviation\n",
    "\n",
    "So... what does that really mean?\n",
    "\n",
    "For datasets where standard deviation is used, the standard deviation from the mean can tell you whether the dispersion in the data is narrow OR wide.\n",
    "\n",
    "In the following image, the dispersion in the red dataset is fairly narrow, which means the standard deviation will be a relatively small number.\n",
    "\n",
    "One nuance to note... in a typical normal (bell-shaped) distribution:\n",
    "\n",
    "* 65% of all values are within one standard deviation of the mean\n",
    "* 95% of all values are within two standard deviations\n",
    "* 99% of all values are within three standard deviations\n",
    "\n",
    "\n",
    "<img src='images/std_narrow.jpg' height='600' width='600'>\n",
    "\n",
    "In the following image, the dispersion in the red dataset is fairly wide, which means the standard deviation will be larger than in the previous example.\n",
    "\n",
    "<img src='images/std_wide.jpg' height='600' width='600'>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def standard_deviation(values):\n",
    "    \"\"\"Return the standard deviation of a sequence of values\n",
    "    \"\"\"\n",
    "    \n",
    "    from math import sqrt\n",
    "    return sqrt(variance(values))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Narrow data set with a mean of 5\n",
    "\n",
    "l = [3, 4, 5, 6, 7]\n",
    "print(\"mean:\" , mean(l))\n",
    "\n",
    "print(\"std:\", standard_deviation(l))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Wider data set with a mean of 5\n",
    "\n",
    "l = [1, 3, 5, 7, 9]\n",
    "print(\"mean:\" , mean(l))\n",
    "\n",
    "print(\"std:\", standard_deviation(l))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Out of curiousity, let's look at our number of defects\n",
    "\n",
    "print('mean:', mean(num_defects))\n",
    "print('std:', standard_deviation(num_defects))\n",
    "\n",
    "# 65% of the batches will be have a defect count within 9 defects of the average."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pro tip... using functions from libraries...\n",
    "\n",
    "All of the items we have covered here are fairly well represented in statistics literature and in various Python and Data Science libraries, so let's take a look at where you can find these tools"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are several places you might want to start, but these are certainly not the only options:\n",
    "\n",
    "|Library|Purpose|\n",
    "|:---|:---|\n",
    "|**statistics**|Mathematical statistics functions including averages, measures of central location, measures of spread. |\n",
    "||Built into the Python Standard Library as of version 3.4: [statistics module](https://docs.python.org/3/library/statistics.html#module-statistics)|\n",
    "|||\n",
    "|**numpy**|a powerful N-dimensional array object|\n",
    "||sophisticated (broadcasting) functions|\n",
    "||tools for integrating C/C++ and Fortran code|\n",
    "||useful linear algebra, Fourier transform, and random number capabilities|\n",
    "|||\n",
    "|**scipy**|SciPy is a collection of mathematical algorithms and convenience functions built on numpy|\n",
    "||adds power to interactive Python sessions|\n",
    "||provides the user with high-level commands and classes for manipulating and visualizing data|\n",
    "| ||\n",
    "|**pandas**|pandas is a Python package providing fast, flexible, and expressive data structures|\n",
    "||designed to make working with “relational” or “labeled” data easy and intuitive|\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import scipy\n",
    "import statistics\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## numpy\n",
    "\n",
    "We will create a `numpy array` from our list of defects"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "defects = np.array(num_defects)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Let's use \"tab complete\" to examine the methods that are available to us on a numpy array\n",
    "\n",
    "defects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "defects.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NOTE**: there is no `.median()` method available on a `numpy array`.\n",
    "\n",
    "But, the `numpy` library has a `.median()` method available that can take an array as **an argument**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.median(defects)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(defects.var())\n",
    "print(defects.std())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## scipy\n",
    "The scipy library gives you access to these same types of functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(scipy.mean(num_defects))\n",
    "print(scipy.median(num_defects))\n",
    "print(scipy.var(num_defects))\n",
    "print(scipy.std(num_defects))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "np.mean?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# in some ways, it appears that scipy essentially repackages some of the numpy methods\n",
    "\n",
    "scipy.mean?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## statistics\n",
    "\n",
    "The Python statistics library (new in version 3.4 of Python) has many of the same functions, but with slightly **different names**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(statistics.mean(num_defects))\n",
    "print(statistics.median(num_defects))\n",
    "print(statistics.variance(num_defects))\n",
    "print(statistics.stdev(num_defects))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NOTE**: The astute viewer will notice that the values provided by `scipy.std()` differ from the value given by the standard deviation formula in `scipy/numpy`.\n",
    "\n",
    "### Population Standard Deviation\n",
    "The standard deviation of a population gives identifies the amount of dispersion of data for an entire population. Because the denominator (`n`) is larger in this formula the population standard deviation tends to be smaller. \n",
    "\n",
    "**Formula**:\n",
    "\n",
    "---\n",
    "\n",
    "$\\large \\sigma = \\sqrt\\frac{\\sum (X - \\mu)^2 }{n}$\n",
    "\n",
    "Where:\n",
    "\n",
    "$\\sigma = population \\ standard \\ deviation$\n",
    "\n",
    "$\\sum = sum \\ of \\ ...$\n",
    "\n",
    "$\\mu = population \\ mean$\n",
    "\n",
    "$n = number \\ of \\ values \\ in \\ the \\ population $\n",
    "\n",
    "---\n",
    "\n",
    "\n",
    "### Sample Standard Deviation\n",
    "A standard deviation of a sample **estimates** the standard deviation of a population based on a random **sample** of that larger population. Because the denominator (`n - 1`) is comparatively smaller in this formula the sample standard deviation tends to be larger.  \n",
    "\n",
    "**Formula**:\n",
    "\n",
    "---\n",
    "\n",
    "$\\large s = \\sqrt\\frac{\\sum (X - \\bar{X})^2 }{n - 1}$\n",
    "\n",
    "Where:\n",
    "\n",
    "$s = sample \\ standard \\ deviation$\n",
    "\n",
    "$\\sum = sum \\ of \\ ...$\n",
    "\n",
    "$\\bar X = sample \\ mean$\n",
    "\n",
    "$n = number \\ of \\ values \\ in \\ the \\ sample $\n",
    "\n",
    "---\n",
    "\n",
    "### When to Use Population Standard Deviation versus Sample Standard Deviation\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "Statisticians use the population and sample standard deviations in different situations. \n",
    "\n",
    "* If a teacher wants to summarize the grades for his students (his entire population), she would use the population standard deviation\n",
    "* If a researcher investigates the relationship between women, exercise and blood pressure, she would use the sample standard deviation because this allows her to apply (or estimate) the results for the entire population.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## pandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lastly, the `pandas` library allows you to perform similar calculations. We start by reading in the number of defects as a `pandas Series`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "defects_pd = pd.Series(num_defects)\n",
    "\n",
    "print(defects_pd.mean())\n",
    "print(defects_pd.median())\n",
    "print(defects_pd.var())\n",
    "print(defects_pd.std())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Experience Points\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Complete the following exercises:\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**A**. Using any **two** of these four libraries:\n",
    "\n",
    "* statistics \n",
    "* numpy\n",
    "* scipy\n",
    "* pandas\n",
    "\n",
    "Calculate the variance and standard deviation of the following numbers:\n",
    "\n",
    "`numbers = [83, 76, 56, 29, 78, 67, 78, 17, 83, 100]`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you (and your partner, if you're working in pairs) are done, then you can put your green sticky up! This is how we know you're done.\n",
    "\n",
    "<img src='images/green_sticky.300px.png' width='200' style='float:left'>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Resources\n",
    "\n",
    "* [Cartoon Guide to Statistics, Larry Gonick/Woollcott Smith](http://www.larrygonick.com/titles/science/the-cartoon-guide-to-statistics/)\n",
    "* [Manga Guide to Statistics, Shin Takahashi](https://nostarch.com/mg_statistics.htm)\n",
    "* [Head First Statistics, Dawn Griffiths](http://shop.oreilly.com/product/9780596527587.do)\n",
    "* [Naked Statistics, Charles Wheelan](https://www.amazon.com/Naked-Statistics-Stripping-Dread-Data/dp/1480590185)\n",
    "* [Statistics in a Nutshell, Sarah Boslaugh](http://shop.oreilly.com/product/0636920023074.do)\n",
    "* [Think Stats, Allen Downey](http://greenteapress.com/wp/think-stats-2e/)\n",
    "* [Think Python, Allen Downey](http://greenteapress.com/wp/think-python/)\n",
    "* [Think Bayes, Allen Downey](http://greenteapress.com/wp/think-bayes/)\n",
    "* [Bayes' Theorem Examples: A Visual Introduction For Beginners, Dan Morris](https://www.amazon.com/Bayes-Theorem-Examples-Introduction-Beginners-ebook/dp/B01LZ1T9IX)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "    \n",
    "    \n",
    "    \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}