python-exercises6.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "python-exercises6.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2iiNvtpD1n72",
        "colab_type": "text"
      },
      "source": [
        "Let's begin by loading the `plotnine` module; call it `p9` for convenience."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qiisrphy1tHt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ASArtf6r2Scs",
        "colab_type": "text"
      },
      "source": [
        "We'll also need `pandas` (call it `pd`):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ifk6Dj6y2XCA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zkwpr8wLzl5h",
        "colab_type": "text"
      },
      "source": [
        "We'll want to connect to the shared Google Drive to get access to our data. Remember, this is always done the same way:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hVlGlyQPzrwi",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# mount google drive \n",
        "from google.colab import drive\n",
        "drive.mount('/content/gdrive')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U53QQLlx18-d",
        "colab_type": "text"
      },
      "source": [
        "Now load the LABEVENTS.csv and PATIENTS.csv data from MIMIC:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "J__DChom1-kB",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "labevents = pd.read_csv(\"/content/gdrive/My Drive/[YCMI_CBDS Summer Course] Data/mimic_iii/LABEVENTS.csv\")\n",
        "patients = pd.read_csv(\"/content/gdrive/My Drive/[YCMI_CBDS Summer Course] Data/mimic_iii/PATIENTS.csv\")"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H8fBZKS19paR",
        "colab_type": "text"
      },
      "source": [
        "Take a look at both datasets to get a sense of how it is structured:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yI-s7dv89vWN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "x8KoCWhlGmcv",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XsZ8xXbi9jnr",
        "colab_type": "text"
      },
      "source": [
        "In the `labevents` dataframe, the `charttime` column represents date and time. Use `pd.to_datetime` to tell the computer this:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2LsQhj2i961O",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DO0ZjP-mJQ3Q",
        "colab_type": "text"
      },
      "source": [
        "### Bar chart\n",
        "\n",
        "Let's start simply: Make a bar chart showing the `gender` data from `patients`. You should end up with two bars: one for `M` one for `F`. Which gender is more common in our dataset?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "h8_VqVc_Gy_x",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8xR9F9QOC7vV",
        "colab_type": "text"
      },
      "source": [
        "### Time series"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pM1eBvrf4iGU",
        "colab_type": "text"
      },
      "source": [
        "Select blood potassium concentration in mEq/L (`itemid == 50971`) for patient `40310`.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ExbTz4Xy3t04",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gKgVzxBzHcPj",
        "colab_type": "text"
      },
      "source": [
        "Make a line graph of the measurements (`valuenum`) vs date (`charttime`). Note that ggplot automatically handles timestamps; we do not need to do anything special for dates."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "civ5YdIp3pts",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DVp7SZx0HrRJ",
        "colab_type": "text"
      },
      "source": [
        "Did you give your graph meaningful x- and y-axis labels? If not, redo it but with labels \"Date\" for the x-axis and \"[K] (mEq/L)\" for the y-axis (you'll want to use `p9.labs`)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F-ihnhBmH_6Q",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AIUxz10QBd_K",
        "colab_type": "text"
      },
      "source": [
        "How do you interpret the graph between mid-November and late December? Are there any time periods that stand out as not being like the rest?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ykr3WXEcTm5Q",
        "colab_type": "text"
      },
      "source": [
        "**PUT YOUR ANSWER HERE**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_llJvfQ4T2W8",
        "colab_type": "text"
      },
      "source": [
        "### Box plot\n",
        "\n",
        "So far we've seen how one particular person's blood potassium concentration changed over time.\n",
        "\n",
        "Let's instead try to get a sense of the variability of blood sodium concentrations (`labevents['itemid'] == 50983`) within individuals but also how individuals compare. To do this, we'll use a boxplot (`geom_boxplot`). Specifically, make a boxplot that `group`s by `subject_id`, plots the concentrations (`valuenum`) along the y-axis, and positions the box across the x-axis beased on `subject_id`. To keep the graph readable, only include data for `subject_id < 20000`. Store the selected data in the variable `sodium` as we'll need it later."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0w2Ktin8WLZF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yiLlPO67yRfK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C2t_Wwdaz7Sx",
        "colab_type": "text"
      },
      "source": [
        "Is our visualization telling the right story?\n",
        "\n",
        "Does the x-axis location mean anything? The `subject_id` only exists to distinguish unique patients; its numeric value has no meaning.\n",
        "\n",
        "To get a better picture: make a new variable `sodium2` based on `sodium` (set it equal to `pd.DataFrame(sodium)`), then set the `subject_id` column to represent categorical values (`astype('category')`). Now regraph.\n",
        "\n",
        "How does your graph compare to the previous one?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Y4Pxflsu1jA3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QdiDuwZoDUm1",
        "colab_type": "text"
      },
      "source": [
        "Did you see any patients whose blood sodium levels seem unusual relative to the rest? (e.g. unusually high, unusually low, or unusually variable)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lOL70Ou_Dzo4",
        "colab_type": "text"
      },
      "source": [
        "- high mean: 10036, 10067\n",
        "- high variability: 10098, 100104\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UIHk7gc75Bwq",
        "colab_type": "text"
      },
      "source": [
        "### Scatter plot\n",
        "\n",
        "Let's load the population data from the Pima Indian diabetes dataset:\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "seBGusgv5F5Z",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "diabetes = pd.read_csv('/content/gdrive/My Drive/[YCMI_CBDS Summer Course] Data/diabetes.csv')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gG5brrsxKbzQ",
        "colab_type": "text"
      },
      "source": [
        "As always, start by looking at the data to get a sense of what it contains:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gb14w4zg1Bk0",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GQNgYq4_1FrV",
        "colab_type": "text"
      },
      "source": [
        "Make a scatter plot showing the relationship of `Glucose` and `Insulin` in the `diabetes` data.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "i0i3xGvi1GQR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M03XtPMdnxYj",
        "colab_type": "text"
      },
      "source": [
        "You can see that there are a lot of times where insulin has a zero value. Let's filter those out and continue."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k6v_q6sanwhJ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "73tCY6iF5GYN",
        "colab_type": "text"
      },
      "source": [
        "There are so many points, it's hard to see where the most likely values are in the basic scatter plot. To get a better sense, add a `p9.geom_density_2d` to overlay a contour map of the point density. Specify a `color` so that it stands out:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5fp2Riz15Jy3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m5tZCsSKLfZq",
        "colab_type": "text"
      },
      "source": [
        "Finally, let's add a smoothed curve `p9.geom_smooth` showing the overall trend. Again, specify a color so that you can see the trend curve:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-cIA6JQkLdeG",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}