6_MatplotlibAndSeabornPlottingTutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "3g26ZcCXkltv"
   },
   "source": [
    "# Plotting Tutorial\n",
    "\n",
    "Python has lots of great plotting packages. The most basic plotting library is `matplotlib`, while the `seaborn`, `plotly`, and `ggplot` libraries offer simplified, more easily customizable plots. For those who have used `ggplot2` or `plotly` in R, they behave similarly here. You are welcome to utilize any of the libraries (and to mix and match libraries that you prefer for different plot types or objectives), but we will likely focus on `matplotlib` and `seaborn`. Let's start by importing those libraries. Since we will be working with data frames we will also import `numpy` and `pandas`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "nR9U5g9iklty"
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Mh6P3tJLklt7"
   },
   "source": [
    "## Basic Plotting Introduction with `matplotlib`\n",
    "\n",
    "We'll start with some fundamentals of plotting. We'll work with small lists of values and not with any data frames yet. The next few cells should help you become familiar with `pyplot`'s plotting syntax. We'll do some more advanced plotting later in this workbook.\n",
    "\n",
    "We'll start by plotting a line graph passing through several ordered pairs: $(1, 1)$, $(2, 4)$, $(3,9)$, and $(4, 16)$. We do this by defining a list of $x$-values using `[1, 2, 3, 4]` and the corresponding array of $y$-values `[1, 4, 9, 16]`. Then we pass these lists as arguments to the `plot()` method from `plt`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Q2EF_3svklt8"
   },
   "outputs": [],
   "source": [
    "plt.plot([1, 2, 3, 4], [1, 4, 9, 16]) #plot(x, y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "p8f1HcnwkluH"
   },
   "source": [
    "Notice that we could obtain the same result by first defining `x` and `y`, then passing these list objects to the `plot` method from `plt`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "esEkVJiYkluK"
   },
   "outputs": [],
   "source": [
    "x = [1, 2, 3, 4]\n",
    "y = [1, 4, 9, 16]\n",
    "\n",
    "plt.plot(x, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "x3ij09YTkluQ"
   },
   "source": [
    "We can customize our plots by editing the plot attributes. See the [`matplotlib.pyplot` documentation](https://matplotlib.org/3.3.0/api/_as_gen/matplotlib.pyplot.html) for a comprehensive list of plot attributes which can be defined or altered. We change the size of the plot as well as add a title and axis labels to our plot below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8TiHNlvrkluQ"
   },
   "outputs": [],
   "source": [
    "plt.figure(figsize = (3,3))\n",
    "plt.plot([1,2,3,4], [1,4,9,16])\n",
    "plt.title(\"My First Plot\")\n",
    "plt.xlabel(\"X label\")\n",
    "plt.ylabel(\"Y label\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Gym3v7yhkluU"
   },
   "source": [
    "We can also edit colors and linetypes. Again, you can see more information if you visit `matplotlib`'s documentation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "bJgYGymUkluX"
   },
   "outputs": [],
   "source": [
    "plt.figure(figsize = (8, 10))\n",
    "plt.plot([1,2,3,4], [1,4,9,16], \"bo\")\n",
    "plt.plot([1,2,3,4], [2,4,6,8], \"g--\")\n",
    "plt.plot([1,2,3,4], [1,2,6,24], \"r:\")\n",
    "plt.plot([1,2,3,4], [20,12,23,8], \"m.\")\n",
    "plt.title(\"My First Plot (blue circles, green dashes, red dotted, magenta points)\")\n",
    "plt.xlabel(\"X Axis\")\n",
    "plt.ylabel(\"Y Axis\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8mT_gvD5klud"
   },
   "source": [
    "More compact plotting allows for multiple sets of data to be passed to one call of `plot`. In the code block below, we pre-define the obects `x` and `y` -- each is a list of four values, and the values in the list `y` are computed using the values from the list `x`. In the line which maps the values to the plot (`plt.plot()`), we first create four green circles (`\"go\"`) at the points $(1,1)$, $(2,4)$, $(3,9)$, and $(4,16)$, then create four red triangles (`\"r^\"`) at the points whose coordinates are defined by the lists `x` and `y` (joined elementwise). Finally, we add a title and axis labels and then tell Python to show the plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "61gsRJSHklue"
   },
   "outputs": [],
   "source": [
    "x = np.arange(1,5) #int range from 1 to 5 (exclusive)\n",
    "y = x**3\n",
    "\n",
    "plt.plot([1,2,3,4],[1,4,9,16], \"go\", x, y, \"r^\")\n",
    "plt.title(\"My Plot\")\n",
    "plt.xlabel(\"X Axis\")\n",
    "plt.ylabel(\"Y Axis\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fzHzoC-ukluo"
   },
   "source": [
    "## Multiple plots in one figure (using subplot)\n",
    "\n",
    "Sometimes we would like to see several plots within a single figure (side-by-side plots, for example). With `matplotlib` it is possible to achieve this with the `.subplot()` method. Calling `plt.subplot(n,m,k)` accesses the `k`$^{th}$ plot (counting from top left to bottom right) in an `n`-row by `m`-column array of plots. In the code block below we create a 1 by 2 array of plots with green circles in the plot on the left and red triangles in the plot on the right. Beyond the `plt.subplot()` lines, most of the code below should seem familiar to you from earlier on -- the call to `plt.suptitle()` allows us to create a \"super title\" -- a title for the entire array of plots. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Dh8c08ioklup"
   },
   "outputs": [],
   "source": [
    "plt.figure(figsize = (8, 4))\n",
    "\n",
    "plt.subplot(1,2,1)\n",
    "plt.plot([1,2,3,4], [1,4,9,16], \"go\")\n",
    "plt.title(\"My First Plot\")\n",
    "plt.xlabel(\"X Axis\")\n",
    "plt.ylabel(\"Y Axis\")\n",
    "\n",
    "plt.subplot(1,2,2)\n",
    "plt.plot([1,2,3,4], [2,4,6,8], \"r^\")\n",
    "plt.title(\"My Second Plot\")\n",
    "plt.xlabel(\"X Axis\")\n",
    "plt.ylabel(\"Y Axis\")\n",
    "\n",
    "plt.suptitle(\"A two-plot figure\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "UDgdwKenklut"
   },
   "source": [
    "If we want to create subplots more efficiently (with less typing) we can do this with the `subplots()` method. It works slightly differently. You can choose which method you like best -- for now, I would suggest sticking with the method which defines each plot individually with a call to `plt.subplot()`, even though it requires more typing. In the plot array below, notice that we left the top left plot (location `[0, 0]`) and bottom right plot (location `[1, 1]`) empty since we did not define them to contain anything. Try filling them in with plots of your own!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "rMWFV4Uxkluu"
   },
   "outputs": [],
   "source": [
    "x = np.arange(1,5)\n",
    "y = x**3\n",
    "fig, ax = plt.subplots(nrows = 2, ncols = 2, figsize = (6,6))\n",
    "\n",
    "ax[0,1].plot([1,2,3,4], [1,4,9,16], \"go\")\n",
    "ax[1,0].plot(x, y, \"r^\")\n",
    "ax[0,1].set_title(\"Squares\")\n",
    "ax[1,0].set_title(\"Cubes\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "lICFg2Fmkluy"
   },
   "source": [
    "## A variety of plot types\n",
    "\n",
    "`matplotlib` is useful for much more than simple linegraphs and point plots. Below are a few examples of bargraphs, histograms, and scatterplots.\n",
    "\n",
    "### Bar Graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "tRAjqnjKklu0"
   },
   "outputs": [],
   "source": [
    "classYear = [\"F\", \"So\", \"J\", \"Sr\"]\n",
    "numStudents = [12, 6, 3, 2]\n",
    "\n",
    "plt.figure(figsize = (8, 4))\n",
    "plt.subplot(1,2,1)\n",
    "plt.bar(classYear, numStudents, color = \"green\")\n",
    "plt.title(\"A Bar Graph\")\n",
    "plt.xlabel(\"Class Year\")\n",
    "plt.ylabel(\"Number of Students\")\n",
    "\n",
    "plt.subplot(1,2,2)\n",
    "plt.barh(classYear, numStudents, color = \"green\")\n",
    "plt.title(\"A [Horizontal] Bar Graph\")\n",
    "plt.xlabel(\"Number of Students\")\n",
    "plt.ylabel(\"Class Year\")\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6mgjcjT6klu7"
   },
   "outputs": [],
   "source": [
    "#Stacked Bar Graphs\n",
    "classYear = [\"F\", \"So\", \"J\", \"Sr\"]\n",
    "numStudents = [12, 6, 3, 2]\n",
    "minors = [2, 1, 2, 2]\n",
    "#nonMinors = list(pd.Series(numStudents) - pd.Series(minors))\n",
    "nonMinors = [10, 5, 1, 0]\n",
    "index = np.arange(4)\n",
    "width = 0.30\n",
    "\n",
    "plt.figure(figsize = (8, 4))\n",
    "plt.subplot(1,2,1)\n",
    "plt.bar(index, nonMinors, width, color = \"green\", label = \"Total Students\")\n",
    "plt.bar(index, minors, width, color = \"blue\", label = \"minors\")\n",
    "plt.title(\"A Horizontally Stacked Bar Graph\")\n",
    "plt.xlabel(\"Class Year\")\n",
    "plt.ylabel(\"Number of Students\")\n",
    "plt.xticks(index, classYear)\n",
    "\n",
    "plt.legend(loc = \"best\")\n",
    "\n",
    "plt.subplot(1,2,2)\n",
    "plt.bar(index, nonMinors, width, color = \"green\", label = \"Total Students\")\n",
    "plt.bar(index, minors, width, color = \"blue\", label = \"minors\", bottom = nonMinors)\n",
    "plt.title(\"A Stacked Bar Graph\")\n",
    "plt.xlabel(\"Class Year\")\n",
    "plt.ylabel(\"Number of Students\")\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wPpxK7z1klu_"
   },
   "source": [
    "### Histograms"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6PsJVMDgklvA"
   },
   "outputs": [],
   "source": [
    "x = np.random.randn(1000) # Get 1000 random values with mean 0 and sd 1\n",
    "\n",
    "plt.title(\"Histogram\")\n",
    "plt.xlabel(\"Random Values\")\n",
    "plt.ylabel(\"Frequency\")\n",
    "plt.hist(x, 10) #histogram with 10 bins\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "VI9YjH4RklvF"
   },
   "source": [
    "### Scatterplots"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Tj5hEUD7klvG"
   },
   "outputs": [],
   "source": [
    "height = np.array([167, 170, 149, 165, 155, 180, 166, 146, 159, 185, 145, 168, 172, 181, 169])\n",
    "weight = np.array([86, 74, 66, 78, 68, 79, 90, 73, 70, 88, 66, 84, 67, 84, 77])\n",
    "\n",
    "plt.xlim(140,200)\n",
    "plt.ylim(60,100)\n",
    "plt.scatter(height, weight)\n",
    "plt.title(\"Scatterplot\")\n",
    "plt.xlabel(\"Height\")\n",
    "plt.ylabel(\"Weight\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "532flHVrklvL"
   },
   "source": [
    "Now that we've had exposure to some fundamental plotting techniques with `matplotlib`, let's explore more interesting applications. We'll use `seaborn` for the plots in this second section, but you'll notice that the syntax is quite similar to the syntax for `matplotlib` and even overlaps significantly. In addition to the plotting, I'll ask you to apply what you learned in our earlier notebooks about exploring data frames. \n",
    "\n",
    "Through the remainder of this notebook we'll work with a pretty famous dataset on wines. The version of this dataset we will use has just under 6500 wines and includes variables about their chemical composition, characteristics like sugar content, alcohol content, and a quality rating. We'll read the wine data into our jupyter notebook from a web url. We will also read in Python's `math` library so that we can utilize the ceiling (round-up) and floor (round-down) functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "jGvzt7K3klvM"
   },
   "outputs": [],
   "source": [
    "import math\n",
    "\n",
    "#Read in the data\n",
    "wineData = pd.read_csv(\"https://raw.githubusercontent.com/rajeevratan84/datascienceforbusiness/master/winequalityN.csv\")\n",
    "wineData.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "nBNZ--RRklvP"
   },
   "source": [
    "Use what you learned in our first jupyter notebook about finding the shape (number of observations and variables in a dataset) and also finding number of NA (missing values) in the various columns of our dataset. Report on the exact number of observations and variables in the dataset, describe how python is thinking about each of the columns in our dataset, and discuss whether any potential *problem columns* exist. Use the code block below to experiment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Ozmw1iDGklvQ"
   },
   "outputs": [],
   "source": [
    "#Experiment here\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "SdaNUWzgklvT"
   },
   "source": [
    "## A very important first step: train/test/safe split\n",
    "\n",
    "Data is an extremely valuable resource -- we should treat it with respect and never use it all up at once. I always recommend breaking off one or two subsets of your data and moving them to a *safe* for later. This is a very important step and **should never be ignored if your goal is to say anything beyond describing the observations appearing in your dataset**. We will typically break up into the following three sets:\n",
    "+ `train`: a relatively large subset of your data (usually around 75% of the observations) which will be used in exploratory analysis, data visualization, hypothesis development, and model construction. The *training* data is the data you'll make the most use out of.\n",
    "+ `test`: a subset of the remaining observations (typically between 50% and 75%) which will be used to test claims and to compare models. Notice that, since we don't actually look at the *test* data or use it to inform any of our claims, it simulates *new* data. It is important that the *test* data is not used to influence any of our hypotheses or models.\n",
    "+ `safe`: all of the remaining observations which are not contained in the *training* and *test* sets. The *safe* data will be used to run one last confirmation on our final findings before passing any findings/recommendations/models \"up the chain\" for operationalizing.\n",
    "\n",
    "While we won't get so far as working with the `test` or `safe` sets in this discussion, we will still create the `train`, `test`, and `safe` sets here for practice."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "X0nSKcIaklvT"
   },
   "outputs": [],
   "source": [
    "#Set a seed so that your results are reproduced identically each time you run this notebook\n",
    "np.random.seed(370)\n",
    "#We randomly shuffle the indices of the original data frame\n",
    "shuffledIndices = np.random.permutation(list(range(0, len(wineData))))\n",
    "#Choose the first 75% of entries as the training indices\n",
    "train_ind = shuffledIndices[ : math.floor(0.75*len(wineData))]\n",
    "#split the remaining indices so that the test data contains 15% of the observations and the safe has the remaining 10%\n",
    "test_ind = shuffledIndices[math.floor(0.75*len(wineData)) : math.floor(0.9*len(wineData))]\n",
    "safe_ind = shuffledIndices[math.floor(0.9*len(wineData)) : ]\n",
    "###\n",
    "#Now create the training, test, and safe sets\n",
    "train = wineData.iloc[train_ind, ]\n",
    "test = wineData.iloc[test_ind, ]\n",
    "safe = wineData.iloc[safe_ind, ]\n",
    "#print the head() of the training data to see what we did!\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_7LacuAFklvY"
   },
   "source": [
    "Use the code block below to run the code `len(train) + len(test) + len(safe) == len(wineData)`. What does it check?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "nXz6KpuVklva"
   },
   "outputs": [],
   "source": [
    "#Experiment here\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-pLbDAmqklve"
   },
   "source": [
    "## Initial Exploration\n",
    "\n",
    "Use what you learned in our original jupyter notebook to explore the training data set. Use the code block below to help you answer the following questions.\n",
    "+ How many observations are in the training data?\n",
    "+ Are there more missing values than expected?\n",
    "+ Get summary statistics on the columns\n",
    "\n",
    "Be sure not to touch the `test` or `safe` sets from this point along -- we don't want to know what they contain."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "-riSFCUhklve"
   },
   "outputs": [],
   "source": [
    "#Experiment here\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "znqdUvQBklvh"
   },
   "source": [
    "## Data Viz\n",
    "\n",
    "Data visualization, when done right, can be one of the most effective tools in your analytics toolkit. We use techniques in data viz to gain insight into a dataset, to identify relationships, to develop conjectures, but also to tell a compelling data story. A strong visual gives us great insight while a poor visual is at best confusing and at worst misleading. Keep the following tips in mind.\n",
    "+ **Visualizing a Single Numerical Variable**: histograms and boxplots\n",
    "+ **Visualizing a Single Categorical Variable**: barplots\n",
    "+ **Visualizing a relationship between Two Numerical Variables**: scatterplots and heatmaps\n",
    "+ **Visualizing a relationship between Two Categorical Variables**: a barplot with fill color or a mosaic plot\n",
    "+ **Visualizing a relationship between One Numerical Variable and One Categorical Variable**: Side-by-side boxplots or overlayed histograms using fill color and transparency to show multiple groups.\n",
    "\n",
    "You should also keep in mind these, possibly not so obvious, suggestions.\n",
    "+ **Never a pie chart**: While pie charts are somewhat pervasive, they can be very difficult to read and are very easy to mislead with. There is nothing a pie chart can do that a barplot with fill cannot.\n",
    "+ **Avoid 3D Plots**: Adding a third spatial dimension for a plot displayed on a 2D screen is ineffective. Try using alternative features to visualize a third dimension (color, size, shape, or transparency are all good options). \n",
    "+ **More is not more**: Keep in mind that the more information you try to pack into a plot, the harder it is to read. You are better off producing multiple plots, each showing one interesting aspect of the data than you are in producing a single plot which is too *busy* to offer any real insights.\n",
    "+ **Never a pie chart, and never ever a 3D-Exploded pie chart**\n",
    "\n",
    "Now let's get to work visualizing our wine data.\n",
    "\n",
    "### One Variable at a Time\n",
    "\n",
    "Let's start by visualizing one variable at a time. We'll begin by exploring the distribution of wine `type` in our dataset. Since this is a categorical variable, a simple barplot will do."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "p0edN0FNklvi"
   },
   "outputs": [],
   "source": [
    "sns.countplot(x = \"type\", data = train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wzqRRpC0klvk"
   },
   "source": [
    "The plot above is super simple, but it clearly shows that the majority of wines contained in our training data are *white* wines. This leads us to believe that the same is true about the distribution of all wines in our full dataset (and even beyond that, about all wines for which our dataset is representative. \n",
    "\n",
    "The `quality` variable in our dataset could be considered categorical -- use the code block below to create a barplot to visualize the distribution of wine `quality` ratings. What can you say? Add a third argument `hue = \"type\"` to your plot -- what additional information does this plot provide?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "raUJQwEAklvl"
   },
   "outputs": [],
   "source": [
    "#Experiment here\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "r8g9xFp2klvn"
   },
   "source": [
    "Another variable we may care to visualize is the `alcohol` content of the wine. Since `alcohol` is a numerical variable we can use a histogram or boxplot (or both) to visualize it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "uDRD-jd3klvo"
   },
   "outputs": [],
   "source": [
    "#Create a figure object so that we can add multiple plots to it\n",
    "fig = plt.figure(figsize = (10, 4))\n",
    "\n",
    "ax1 = fig.add_subplot(1, 2, 1) #add a plot to a 1x2 array of plots, in position 1 (top-left)\n",
    "sns.distplot(train[\"alcohol\"], color = \"green\", kde = False, bins = 10)\n",
    "ax1 = fig.add_subplot(1, 2, 2)\n",
    "sns.boxplot(train[\"alcohol\"], orient = \"h\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "251dFYUUklvy"
   },
   "source": [
    "The curve laid over the top of the histogram is present by default (its purpose is to show an estimated distribution curve), but it can be turned off by setting the argument `kde = False`. Try it! We also can control the number of bins present in a histogram. This one might be showing too much detail -- reduce the number of bins to 10 by setting the argument `bins = 10`.\n",
    "\n",
    "### Multiple variables, looking for associations\n",
    "\n",
    "Visualizing one variable at a time can be quite helpful in learning about our data, indentifying claims about these individual variables, and for identifying whether assumptions (like approximate normality, absence of extreme skew, no extreme outliers) are satisfied. These are all interesting and important, but the real magic of data visualization comes when we start looking for associations between two or more variables. We'll take a look at a few scenarios (numerical versus numerical, numerical versus categorical, and categorical versus categorical) as well as adding some extra dimension to these scenarios by making use of additional plot attributes like color, shape, or size. After a few examples, you'll be on your own to answer some questions about the wine data.\n",
    "\n",
    "If your dataset is small enough (in number of columns), sometimes it is a great idea to begin with a panel of plots showing potential relationships between all of the variables in your dataset. We'll start there. The plot may take a minute or two to generate, so don't get impatient."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "xuKFTueKklvy"
   },
   "outputs": [],
   "source": [
    "sns.pairplot(train, diag_kind = \"kde\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-OEtXDarklv4"
   },
   "source": [
    "The plot panel took a while to generate -- that's because python was busy making $13\\times 13 = 169$ plots. Once the plot renders though, we can quickly see potential relationships between variables. For example, looking at the plot in the intersection between `chlorides` and `density` we can see a positive association between these two characteristics. The plot panel is indeed somewhat difficult to read though because of the number of variables in our dataset -- everything (including the labels) is quite tiny. One last note is that we can control the type of plot shown along the diagonal with the `diag_kind` argument. Here we included a density plot (with `\"kde\"`), but we could also show a histogram or other plot type.\n",
    "\n",
    "Let's take a closer look at that plot between `chlorides` and `density`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "bWx-rzJBklv6"
   },
   "outputs": [],
   "source": [
    "#Build a scatterplot\n",
    "sns.scatterplot(x = \"chlorides\", y = \"density\", data = train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "yTmcFi-gklwB"
   },
   "source": [
    "I wonder if the relationship between chlorides and density is depentent on the type of wine (red or white)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "snMAqV4BklwC"
   },
   "outputs": [],
   "source": [
    "#Color points by wine type\n",
    "sns.scatterplot(x = \"chlorides\", y = \"density\", hue = \"type\", data = train, alpha = 0.4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "sB24CUPvklwF"
   },
   "source": [
    "It seems as though the majority of red wines have higher chloride levels and higher densities than whites do. There are some white wines with elevated levels of chlorides, but those have typically lower densities than their red counterparts. I suspect that rather than a true association between chloride levels and density, there is a confounding variable at play here (possibly just wine type).\n",
    "\n",
    "It looks like there is quite a bit of difference in the chloride levels of red and white wines. Let's get a better picture of that by using side-by-side boxplots."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "gvCzSFOXklwF"
   },
   "outputs": [],
   "source": [
    "#Build the boxplot\n",
    "sns.boxplot(y = \"chlorides\", x = \"type\", data = train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "dkIq-EpHklwJ"
   },
   "source": [
    "The outliers are really distoting this plot -- let's change the chlorides scale so that we have something more readable. We'll use two methods here -- the first just cuts the plot (essentially zooming in) and ignoring all of the plotted values above 0.2 while the second transforms the chlorides scale to a log-scale, a common transformation which pulls outliers closer to the rest of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "fV0yX5hZklwL"
   },
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize = (18,7))\n",
    "\n",
    "#Change plotting limits\n",
    "ax1 = fig.add_subplot(1, 3, 1)\n",
    "sns.boxplot(y = \"chlorides\", x = \"type\", data = train)\n",
    "\n",
    "ax1 = fig.add_subplot(1, 3, 2)\n",
    "sns.boxplot(y = \"chlorides\", x = \"type\", data = train)\n",
    "ax1.set(ylim = (0, 0.2))\n",
    "\n",
    "#Change plotted scale to log-scale\n",
    "ax2 = fig.add_subplot(1, 3, 3)\n",
    "sns.boxplot(y = \"chlorides\", x = \"type\", data = train)\n",
    "ax2.set(yscale = \"log\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "U0Z8Cx0hklwT"
   },
   "source": [
    "Both plots show that the white wines in our training set have lower chloride levels than the red wines do. The advantage of the plot on the left is that the y-axis scale still measures chloride levels in the same units that they are measured in the dataset but the disadvantage of not showing all of the data. The second plot does show all the data but at a cost of potentially distorting the data with a non-linear y-axis scale. For example, we can see in the plot on the left that the middle 50% of red wines in terms of chloride levels occupies a wider range than the middle 50% of white wines -- this is masked (and even flipped) in the log-scale plot. Failing to understand our plots and the impacts of decisions like scaling can lead to erroneous conclusions.\n",
    "\n",
    "Finally, let's take a look at comparing two categorical features of the wines by using filled barplots. Unfortunately our wines data only has a single categorical column (wine type), so we'll need to engineer ourselves another categorical feature. Feature engineering is the act of creating new features (columns) from the existing columns in your data set (or from external sets, or combinations of the two). Engineering new features is a necessary skill and can be extremely fruitful when done correctly.\n",
    "\n",
    "Let's create a new categorical variable called `rating` which takes the levels *good* (rating at least 7), *moderate* (rating 5 to 7), and *poor* (rating less than 5). We'll engineer this feature in the original wine dataset and then resplit the training, test, and safe data. It is good practice to do this since otherwise you would need to re-engineer the feature separately for each of the sets -- not so bad for just a single engineered feature, but pretty terrible if you've engineered many."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "TBIbvFFTklwU"
   },
   "outputs": [],
   "source": [
    "#Engineer the feature for the wineData dataset\n",
    "\n",
    "#Initialize the rating column with just 0's\n",
    "wineData[\"rating\"] = 0*wineData.size\n",
    "\n",
    "wineData.loc[wineData[\"quality\"] >= 7, \"rating\"] = \"good\"\n",
    "wineData.loc[(wineData[\"quality\"] >= 5) & (wineData[\"quality\"] < 7), \"rating\"] = \"moderate\"\n",
    "wineData.loc[wineData[\"quality\"] < 5, \"rating\"] = \"poor\"\n",
    "\n",
    "#Print column to verify\n",
    "#wineData[\"rating\"]\n",
    "\n",
    "train = wineData.iloc[train_ind, ]\n",
    "test = wineData.iloc[test_ind, ]\n",
    "safe = wineData.iloc[safe_ind, ]\n",
    "\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "tvnGShJwklwY"
   },
   "source": [
    "As you can see, we now have a rating column which lists a *good*, *moderate*, or *poor* rating for each of the wines. Now let's visualize whether the distributions of ratings is different across the red and white wines. We'll do this by creating a barplot with a set of bars for each wine type, and those bars will be filled (colored) according to their wine ratings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "451Wn9rlklwZ"
   },
   "outputs": [],
   "source": [
    "sns.countplot(x = \"type\", hue = \"rating\", data = train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "UAVhWfoqklwc"
   },
   "source": [
    "We can see from the plot that the distributions are similar. Most wines are *moderately* rated.\n",
    "\n",
    "## On Your Own\n",
    "\n",
    "That's it -- you've got everything you need from me. You now know how to build some basic plots to explore data in all sorts of scenarios. You can go beyond these basic plots by looking up `matplotlib`'s or `seaborn`'s documentation or by doing a quick google search for how to build a particular plot type. Now it's time for you to have an opportunity to practice. There is very famous dataset on `iris` flowers available at the following web url (https://gist.githubusercontent.com/curran/a08a1080b88344b0c8a7/raw/639388c2cbc2120a14dcf466e85730eb8be498bb/iris.csv) -- use `pd.read_csv()` to read in this dataset just as we read in the `wine` dataset, and then use visualization techniques to explore relationships between petal and sepal widths and lengths across three varieties of iris. Don't forget to split your new data into training, test, and safe sets like we did with the `wine` data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "2PBs_2fQklwd"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "name": "5_MatplotlibAndSeabornPlottingTutorial.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}