Python Foundations for Data Science | CLICK HERE .ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Python Foundations for Data Science 🤓"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Agenda for our workshop ⏰\n",
    "\n",
    "### Basics first 🐣 Python Data Types (40 min)\n",
    "*  Strings\n",
    "*  Practice Round ✍️\n",
    "*  Integers and Floats\n",
    "*  Practice Round ✍️\n",
    "\n",
    "### Data Analytics - Reading Data in Python 🧮(30 min)\n",
    "*  Tools we will use\n",
    "*  Importing data from CSV\n",
    "*  Previewing the DataFrame\n",
    "*  Practice Round ✍️\n",
    "\n",
    "### Data Analytics - Our first analysis 🔍(50 min)\n",
    "*  Boolean analysis\n",
    "*  Our first analysis - let's check out the largest countries\n",
    "*  Combining data points for more insight\n",
    "*  Practice Round ✍️\n",
    "*  Data visualization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Welcome to the Jupyter Notebook! How to use this file? 🤔\n",
    "\n",
    "* Type inside the empty cells to write code. These empty cells will have a `In [ ]:` prefix before\n",
    "* Press the `return/enter ⏎` key to add a new line inside the cell\n",
    "* To display your results use the Python built in `print(STUFF_YOU_WANT_TO_PRINT)` method or simply put the stuff you want to print as the last line inside the cell. The result of the last line will appear as the `Out[]:` or the output of the cell :)\n",
    "* Press `shift` + `return/enter ⏎` to run your code 🤓 this will run the code inside your currently selected cell and print anything inside `print()` method and the last line of your cell\n",
    "* To add a new cell, select any cell and press the `b` key (make sure you are not just typing the letter `b` in the cell). This will add a new cell below\n",
    "* To delete a cell, double press the `d` key (make sure you are not just typing the letter `d` in the cell)\n",
    "\n",
    "#### Try to run the code below! 😃"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Welcome to the Python workshop of Le Wagon!\")\n",
    "print(\"Are you ready to do your first data analysis?\")\n",
    "\"Yes we are! 🚀\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Python Basics ⚙️"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "    Everything in Python is an object. Objects have attributes and methods. \n",
    "Real life example: a car as an object, has attributes such as color, make, year of manufacture, transmission mode etc., and methods such as accelerating, reverse, braking."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Python Data Types and Variables 🧱\n",
    "- Strings\n",
    "- Integers\n",
    "- Floats\n",
    "- Booleans & Logicals\n",
    "- Variables\n",
    "- Lists & Dictionaries*\n",
    "\n",
    "*Only FYI"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STRINGS   str() 📝\n",
    "To represent a text and defined with \"double\" or 'single' quotes\n",
    "\n",
    "    \"Python for Beginners\"\n",
    "    'Python for Beginners'\n",
    "    '2019'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Built-in Methods for Strings\n",
    "    type(\"python\")\n",
    "    int('2019')\n",
    "    len(\"python\")\n",
    "    'python'.capitalize()\n",
    "    \"python is fun\".count('n')\n",
    "    \"PYTHON\".lower()\n",
    "    'python'.index('o')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  🚀 Try out built-in methods for `strings` !\n",
    "\n",
    "How? Type in a line of code on the empty line below and press `Shift` + `enter/return ⏎` keys to run it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### INTEGERS   int() 🔢\n",
    "To represent integers and can do standard arithmetics.\n",
    "    \n",
    "    10\n",
    "    -24\n",
    "    2019"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Built-in Methods for Integers\n",
    "    type(10)\n",
    "    abs(-24)\n",
    "    min(10, -24, 2019)\n",
    "    max(10, -24, 2019)\n",
    "    float(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Arithmetic Operators\n",
    "    1 + 2    => 3  (Addition)\n",
    "    1 - 2    => -1 (Subtraction)\n",
    "    2 * 4    => 8  (Multiplication)\n",
    "    5 / 2    => 2.5(Division)\n",
    "    5 // 2   => 2  (Floor division)\n",
    "    5 % 2    => 1  (Modulus)\n",
    "    2 ** 3   => 8  (Exponent)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### FLOATS   float()\n",
    "To represent decimal numbers and can do standard arithmetics.\n",
    "\n",
    "    10.23132\n",
    "    -24.1\n",
    "    2019.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Built-in Methods for Floats\n",
    "    all built-functions for integers\n",
    "    round(10.23132, 2)\n",
    "    import math # first import math package - a collection of advanced mathematical functions\n",
    "    math.floor(10.23132)   \n",
    "    math.ceil(-24.1)\n",
    "    math.ceil(24.1)\n",
    "    math.factorial(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###   🚀Try out built-in methods for `integers` and `floats` below!\n",
    "\n",
    "How? Type in a line of code on the empty line below and press `Shift` + `enter/return ⏎` keys to run it\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### BOOLEANS & LOGICALS   bool() 🔀\n",
    "Booleans represent something that is True or False\n",
    "\n",
    "    True\n",
    "    False\n",
    "    \n",
    "Comparison Operators always yield Boolean values\n",
    "\n",
    "     ==       # different from assignment operator '='\n",
    "     !=\n",
    "     > or >=\n",
    "     < or <=\n",
    "     \n",
    "Logicals include:\n",
    "\n",
    "    or\n",
    "    and\n",
    "    not"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### VARIABLES 📦\n",
    "- Allows you to store values to reuse later\n",
    "- You assign a value to a variable\n",
    "- Variables can be overwritten and incremented\n",
    "- By convention, variable names should be in snake_case\n",
    "\n",
    "Variable assignment statement - *putting things in a box*\n",
    "\n",
    "    putting the `string` \"John\" in the box `first_name`\n",
    "        \n",
    "        first_name = \"John\"\n",
    "        \n",
    "    putting the `integer` 29 in the box `age`\n",
    "    \n",
    "        age = 29\n",
    "       \n",
    "    \n",
    "Variable reading - *opening the box to see what is inside*\n",
    "\n",
    "    print(age)\n",
    "    print(first_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  🚀Try out `booleans` and `variables`\n",
    "\n",
    "How? Type in a line of code on the empty line below and press `Shift` + `enter/return ⏎` keys to run it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Practice! ✍️ Try out `strings`, `integers`, `floats`,  `booleans` and `variables` below!\n",
    "\n",
    "Some ideas:\n",
    "- Make two variables and check if they are equal (*pssst*, ==)\n",
    "- Make two variables - your first name and last name \n",
    "- Create your own full name out of first_name and last_name variables (make sure it’s always capitalized!)\n",
    "- Check the length of your full name\n",
    "- Make todays date out of a year, month and day variables 🤔\n",
    "- Create a variable with your age in years - now calculate your age in minutes (approximately)\n",
    "- A class of students got their exams graded 15, 12, 11, 16, 18. Calculate the grade average.\n",
    "\n",
    "How? Type in a line of code on the empty line below and press `Shift` + `enter/return ⏎` keys to run it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Science - Analysis 🧮"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### First of all - what are we using? 🔨"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Jupyter Notebook](https://jupyter.org) is an open-source web application which allows you to create and share documents with code, visualizations and narrative. Which is what we are doing right now! "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Numpy](https://www.numpy.org) fundamental package for scientific computing in Python."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Pandas](https://pandas.pydata.org) is an open-source library providing easy to use data structuring and analytics for Python. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### So this is how every Jupyter notebook starts..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "print(\"You are good to go!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Python and Pandas is great for reading data files, like CSV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "file = \"countries of the world.csv\"\n",
    "countries_data = pd.read_csv(file, decimal=\",\")\n",
    "# no need to use print() when we want to see DataFrames, simply put what you want to see on the last line\n",
    "countries_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### We can then have a quick look at our `DataFrame` 🗺"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can run `countries_data.shape` to check how many rows and columns we have. The result will be printed as (rows, columns)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"rows and columns -->\", countries_data.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can run `countries_data.columns` to check all the columns that our DataFrame has\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"all column names -->\", countries_data.columns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can run `countries_data.head(YOUR_NUMBER)` to check the first `YOUR_NUMBER` of records of the DataFrame\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "countries_data.head(5) # will show the first 5 records"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can run `countries_data.tail(YOUR_NUMBER)` to check the last `YOUR_NUMBER` records of the DataFrame\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "countries_data.tail(5) # will show the last 5 records"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can run `countries_data[COLUMN_NAME]` to see only the values of `COLUMN_NAME`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "countries_data[\"Population\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can run `countries_data[[COLUMN_NAME_1, COLUMN_NAME_2,...etc]]` to see a few values together using the double square bracket notation\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "countries_data[[\"Country\", \"Population\"]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Bonus: What if you want to show data for just one country?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "countries_data['Country'] = countries_data['Country'].map(str.strip) # Clean the text in Country column\n",
    "countries_data.set_index('Country', inplace=True) # Set Country names as the unique index\n",
    "countries_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can see instead of numbers, we have country names as our index\n",
    "\n",
    "⚠️**Note** that when you do this you remove the `Country` column, and make it the index of the whole DataFrame instead (for example, `country_data['Country']` will not work any more)\n",
    "\n",
    "We can simply use `countries_data.loc['COUTRY_NAME']` syntax to get data for one country"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "countries_data.loc['Algeria']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Your turn! 🚀Run the code above to import `numpy` and `pandas` and read the CSV to get `countrie_data`. No need to change anything there ;)\n",
    "\n",
    "#### Now play around with your `DataFrame` 🤓\n",
    "\n",
    "Some ideas:\n",
    "* check the amount of rows and columns in your DataFrame\n",
    "* check what are all the columns that you have\n",
    "* view only the \"Climate\" column of your DataFrame\n",
    "* view the first 10 rows of your DataFrame\n",
    "* view the last 10 rows of your DataFrame, but only viewing the \"Country\", \"Population\" and \"Climate\" 🤔\n",
    "\n",
    "How? Type in your code on the empty line below and press `Shift` + `enter/return ⏎` keys to run it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Now let's do some data analytics! 🔍"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Let's say we want to see countries with over 50,000,000 people. We want to create a `boolean` - `True` or `False` - with this condition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "large_population = countries_data[\"Population\"] > 50_000_000\n",
    "large_population"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### We can see that `large_population` is now a list of `True` and `False` values, matching our countries DataFrame 🧐"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Now we can apply this condition to our `countries_data` using the square brackets `[condition]`, same syntax as we used for checking columns "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "large_countries = countries_data[large_population]\n",
    "large_countries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Boom 💥we have our first data analytics solution!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also sort our large_countries by Population, having largest first, using the syntax of `large_countries.sort_values(\"COLUMN_TO_SORT_BY\", ascending=False)`, like below\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "large_countries.sort_values(\"Population\", ascending=False) # change False to True if you wan't smallest countries first :)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### We can also combine data points! Let's see which of our large countries have the biggest Service industry 🤔\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "large_service_countries = large_countries.sort_values(\"Service\", ascending=False)\n",
    "large_service_countries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's make our view a little cleaner - I only want to see the Country, Population and the ratio of Service industry. We can use the `large_service_countries[[COLUMN_NAME_1, COLUMN_NAME_2,...etc]]` syntax for that 🔍\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "large_service_countries[[\"Country\", \"Population\", \"Service\"]].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Your turn! 🚀 Explore your `DataFrame` and run some analysis 🤓\n",
    "\n",
    "Some ideas:\n",
    "* check which countries have the warmest climate ☀️\n",
    "* let's check the tiny countries! make a variable to store countries with Population < 3_000_000\n",
    "* Beach life! 😎check the countries with longest Coastline. Remember to type the full column name!\n",
    "* Are these coastal countries always online? 📱Let's sort them by amount of Phone usage. Again, check the full column name!\n",
    "#### ...and anything else you want to analyze! 🚀\n",
    "\n",
    "How? Type in your code on the empty line below and press `Shift` + `enter/return ⏎` keys to run it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Finally, let's make our findings visual 🎨"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once again, we already have Python libraries available for us with all we need.\n",
    "\n",
    "We will be using [matplotlib](https://matplotlib.org/) - a very popular data visualization library for Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib # same as with pandas and numpy, we need to first import the library"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start by creating our first [plot](https://pandas.pydata.org/pandas-docs/version/0.23.4/generated/pandas.DataFrame.plot.html) out of our `large_service_countries` data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "large_service_countries.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "😰 Seems quite messy... \n",
    "\n",
    "Let's use the documentation of [DataFrame.plot](https://pandas.pydata.org/pandas-docs/version/0.23.4/generated/pandas.DataFrame.plot.html) and [matplotlib](https://matplotlib.org/api/_as_gen/matplotlib.pyplot.plot.html?highlight=plot#matplotlib.pyplot.plot) to see how we can clean up our plot."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### First of all, let's reduce the amount of data points we are trying to display"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "large_service_countries.head().plot(y='Service')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### What is this strange zig-zag? Our horizontal axis (or the x-axis)! 💡"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "large_service_countries.head().plot(y='Service', x='Country')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### This line chart is not very fitting... let's see that other [kinds](https://pandas.pydata.org/pandas-docs/version/0.23.4/generated/pandas.DataFrame.plot.html) are available to us"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "large_service_countries.head().plot(kind='bar', y='Service', x='Country')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Already much better! Now let's make it more interesting 🤓"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "large_service_countries.plot(\n",
    "kind='bar',\n",
    "y=['Service', 'Industry', 'Agriculture'],\n",
    "x='Country',\n",
    "stacked=True,\n",
    "color=['#809BCE', '#B8E0D2', '#EAC4D5'],\n",
    "figsize=(8, 6)).legend(loc='lower left')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Your turn! 🚀 Visualize the `DataFrame` you've made in the previous challenges or make new ones 📊\n",
    "\n",
    "Remember to use online resources to see what are the different attributes you can play with:\n",
    "\n",
    "* Matplotlib documentation on [plot attributes](https://matplotlib.org/api/_as_gen/matplotlib.pyplot.plot.html?highlight=plot#matplotlib.pyplot.plot)\n",
    "* Pandas documentation on the [DataFrame.plot() function we are using](https://pandas.pydata.org/pandas-docs/version/0.23.4/generated/pandas.DataFrame.plot.html)\n",
    "* 👉 [This neat article](http://queirozf.com/entries/pandas-dataframe-plot-examples-with-matplotlib-pyplot) with a bit more visual support"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}