W142 Jupyter Notebook Assignment-Revised060719.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [W142] Jupyter Notebook Assignment "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Estimated Time: 120 Minutes\n",
    "\n",
    "Created by: Yidan Zhang\n",
    "\n",
    "In this assignment you will be examining a dataset from The World Inequality Database on Education (WIDE) to explore the powerful influence of circumstances(such as wealth, gender, ethnicity and location) have on their opportunities for education and life. \n",
    "\n",
    "Please use this international datasets to begin to identify, through analysis and data visualizations who are the most marginalized children with regards to education in the country your group has chosen.\n",
    "\n",
    "You can refer to the Jupyter Notebook Tutorial - Out of School Children Rate in Uganda as an example and tutorial of some data visualization techniques while you complete this Notebook assignment. You may want to be in touch with your group members to each run different analyses that look at different markers of marginalization. \n",
    "\n",
    "To complete this assignment, each member of your group will need to select:\n",
    "1. A country of your group's interest that is in the WIDE dataset\n",
    "2. One set of unique indicator (AKA outcome variable) \n",
    "    Depending on the avaliability of data in your country of choice, examples of indicators include: \n",
    "         1) Out of School Children Rate at primary, lower secondary and upper secondary level:\n",
    "               'eduout_prim_m', 'eduout_lowsec_m', 'eduout_upsec_m'\n",
    "         2) Completion rate at primary, lower secondary and upper secondary level:\n",
    "               'comp_prim_v2_m','comp_lowsec_v2_m', 'comp_upsec_v2_m'\n",
    "         3) Percent of people aged 18-22 years attending higher education\n",
    "               'attend_higher_1822_m'\n",
    "         4) Over-age primary school attendance\n",
    "               'overage2plus_m'\n",
    "         5) Transition rate to lower secondary school and transition rate to upper secondary school\n",
    "             'trans_prim_m', 'trans_lowsec_m'\n",
    "3. Three characteristics : Sex, Location, and Wealth\n",
    "\n",
    "If your group is having trouble identifing indicators for your country of choice, reach out to the GSI. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Student Name:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double click and type your name here:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color = #254117> 1. Tell us which country you are looking at and why </font> "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double click and type your response here:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color = #254117>2. Import dependencies and read in the WIDE data file</font> "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Just run this cell\n",
    "\n",
    "%matplotlib inline\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import numpy as np\n",
    "from datascience import *\n",
    "import matplotlib.pyplot as plots\n",
    "import math as m\n",
    "import qgrid as q\n",
    "import pandas as pd\n",
    "import sys\n",
    "sys.path.insert(0, \"development\")\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "from IPython.display import HTML"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Just run this code block! This code reads in our data.\n",
    "\n",
    "wide = pd.read_csv('WIDE.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# type in a code block below to display the first few rows of your dataset:\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color = #254117> 3. Manipulate the dataset and select a subset of data</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is the marker of marginalization you want to look at? \n",
    "For example, in the tutorial, we the marker of marginalization we used was out of school children rate, and the variables you need to included are gender, location, and wealth."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Type your response here:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# First print the list of country names, so you can see if the country you are interested in is here. \n",
    "# If the country your group selected is not here, pick anoher country from this list. \n",
    "# type your code below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Print the list of column names, so you can select the indicator and variables you want to include in the analysis.\n",
    "# type your code below:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now, use the .loc function to select your country of interest and the list of the variables you are interested in\n",
    "# Assign the new dataframe a name, and print the table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### You have successfully created a smaller and more specific datasetto work with, now let's check for missing values and remove them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the number of missing values in each column of your new dataset\n",
    "# type  your code below:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# check the shape of your current dataset\n",
    "# type your code below:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# drop all rows that contains a missing value, and check the shape again \n",
    "# type your code below:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# give your new dataset a name, after you removing the missing value, and print it \n",
    "# type your code below:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Does this dataset look like what you are expecting? Does it include all the variables and indicators you need? Does it contain any missing values? \n",
    "\n",
    "If your dataset contains all the variables and indicators, and does not contain missing values, you can move on to the next part. Otherwise, check your data cleaning process. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color = #254117> 4. Summary Statistics and Visualization</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you have data from multiple years, you can start with plotting the overall marker of marginalization over time. Modify the following code to plot your graph. If you don't have multiple years in your dataframe, skip this part"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Just run this code block. It allows the plot to appear in the Notebook.\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code for graph over time.\n",
    "#note,replace 'yourcountry' with the name of your dataframe created in part 3. \n",
    "yourcountry.groupby('year').mean().plot(kind='line')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, create a more specific dataset with data from the most recent year, and give it a name."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# type your code below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# plot the mean of your marker by groups\n",
    "# for example : uganda16.groupby('Location').mean().plot(kind='bar').legend(bbox_to_anchor=(1, 0.5))\n",
    "# the above code plots all 2016 data from Uganda by location.\n",
    "\n",
    "# type your code below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# you can add more cell to produce a graph for each variable of interest\n",
    "# for example: uganda16.groupby('Sex').mean().plot(kind='bar').legend(bbox_to_anchor=(1, 0.5))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, try plotting some side by side bar graphes to see the difference in the marker of marginalization by a combination of two variables. Add more code cells if needed. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# type your code below:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Insert a markdown cell below each plot you produced to give an interpration of the graph."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Looking across all the graphes you produced, what stood out the most for you? Why?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "type your response here. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## <font color = #254117> Submiting your assignment</font>\n",
    "\n",
    "When you are ready to submit this assignment, click File on top of this page, download as a .pdf file, and upload it to bcourses. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}