intro.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bY4wGo3hKv1i",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![IE](img/ie.png)\n",
    "\n",
    "# Python for Data Analysis II\n",
    "\n",
    "##  Master in Business Analytics and Big Data, October 2021-2022\n",
    "\n",
    "### Professor: Juan Luis Cano Rodríguez <jcano@faculty.ie.edu>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "civOrU5RKv1o",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Outline\n",
    "\n",
    "- Who am I?\n",
    "- What is Python? Why Python?\n",
    "- PyData Ecosystem\n",
    "- About this course\n",
    "    - Calendar, sessions\n",
    "    - Material\n",
    "    - Software\n",
    "    - Learning objectives\n",
    "    - Evaluation method"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "w3UGLILTKv1s",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Who am I?\n",
    "\n",
    "![Me](img/juanlu_esa.jpg)\n",
    "\n",
    "- **Aerospace Engineer** from TU Madrid + 1 year at Politecnico di Milano\n",
    "- **Developer Advocate** at **Read the Docs**, an open source hosting service for documentation \n",
    "  - Previously Planning & Execution Engineer at Satellogic, a satellite imagery company\n",
    "- **Contributor** to the PyData ecosystem: NumPy, SciPy, conda, Dask, ...\n",
    "- **Instructor** of Python courses for Data Scientists at Airbus, Boeing, Telefónica and others\n",
    "- Experience as **Data Scientist** with Python for telco and aerospace industries\n",
    "- Honorific member of the **Python España** non-profit and former co-organizer of **PyCon Spain**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "3O_aLqNxKv1y",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# What is Python?\n",
    "\n",
    "![Major languages](img/growth_new.png)\n",
    "\n",
    "* **Python is** a dynamic, interpreted programming language\n",
    "* It is easy to learn, but powerful\n",
    "* Features a huge ecosystem of contributed packages\n",
    "* Most rapidly growing language in Stack Overflow\n",
    "\n",
    "<small>Source: https://stackoverflow.blog/2017/09/06/incredible-growth-python/ (data updated as of 2020)</small>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9QUsIoogKv11",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Why is Python growing so quickly?\n",
    "\n",
    "![Related tags](img/related_tags_over_time.png)\n",
    "\n",
    "* Quick answer: **Data Science**\n",
    "* Boundless growth since the creation of pandas\n",
    "* Already popular for scripting and web development\n",
    "\n",
    "<small>Source: https://stackoverflow.blog/2017/09/14/python-growing-quickly/</small>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fIKTIyntKv16",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "More interesting surveys:\n",
    "\n",
    "* Kaggle, 2020: 74 % of data scientists use Jupyter, 83 % use scikit-learn https://www.kaggle.com/kaggle-survey-2020\n",
    "  - Comparatively, 32 % use RStudio\n",
    "* KDnuggets, 2019: Most used programming language in Data Science for third year in a row https://www.kdnuggets.com/2019/05/poll-top-data-science-machine-learning-platforms.html\n",
    "  - 66 % of data scientists use Python\n",
    "  - Comparatively, 47 % use R and 33 % use SQL or Excel\n",
    "  - 34 % use specifically Anaconda"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "VmwymI3pKv19",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Disadvantages\n",
    "\n",
    "* Python itself is _slow_ - the trick is that the libraries wrap faster languages (C, C++, FORTRAN)\n",
    "* Weaker time series and statistical analysis (use R for that!)\n",
    "* Packaging, installation and distributing can be tricky (if interested, choose my elective \"Advanced Python\")\n",
    "* ~~Python 2 vs 3 split~~ Python 2 is officially dead since January 2020 🎉 focus on Python 3.7 onwards"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# PyData ecosystem\n",
    "\n",
    "![PyData ecosystem](img/ecosystem/1.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "![PyData ecosystem](img/ecosystem/2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "![PyData ecosystem](img/ecosystem/3.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "![PyData ecosystem](img/ecosystem/4.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "![PyData ecosystem](img/ecosystem/5.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "sPJolsQMKv2R",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# About this course\n",
    "\n",
    "## Learning objectives\n",
    "\n",
    "1. Understand the different pieces of the PyData ecosystem and the relationship between them\n",
    "2. Learn how to use pandas to perform advanced data manipulations\n",
    "3. Understand how pandas and seaborn leverage matplotlib objects for static visualization\n",
    "4. Understand how to the API of scikit-learn helps solve classical machine learning problems\n",
    "5. Learn how to create transformation pipelines with scikit-learn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Calendar\n",
    "\n",
    "* **From January 10th to March 28th**\n",
    "  - January: 5 F2F sessions, 1 Forum - Advanced pandas, visualization, NumPy\n",
    "  - February: 6 F2F session, 1 Forum - NumPy, SciPy stats, scikit-learn pipelines\n",
    "  - March: 4 F2F sessions, 2 Forums - More visualization, interactive dashboarding, final presentations\n",
    "* **All F2F sessions will be hands on**, have your Jupyter notebooks ready!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "V2JI6cAgKv2S",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Individual assignment\n",
    "\n",
    "* Analyze a medium sized dataset, and write a report\n",
    "* You will have to apply all the things we see in the first half of the subject\n",
    "* Details coming soon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "V2JI6cAgKv2S",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Group project\n",
    "\n",
    "* Analyze a moderately big dataset, write a report, and present the results\n",
    "* You will have to apply all the things we see in the subject\n",
    "* Details coming soon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "t6t9tmi8Kv2V",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Evaluation method\n",
    "\n",
    "> The evaluation be based on the daily progress of the student, as well as a big group project that will be presented during the last session.\n",
    "\n",
    "| Criteria                   | Score % |\n",
    "|----------------------------|---------|\n",
    "| Class Participation        |  10 %   |\n",
    "| Individual Assignments     |  45 %   |\n",
    "| Group Project Report       |  35 %   |\n",
    "| Group Project Presentation |  10 %   |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "oZkXQ455Kv2X",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Material\n",
    "\n",
    "- Main content: \"Python Data Science Handbook\" by Jake Vanderplas https://github.com/jakevdp/PythonDataScienceHandbook\n",
    "- scikit-learn 0.24 official documentation https://scikit-learn.org/0.24/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "kPp2i65xKv2Z",
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "...yes, everything is freely available on the Internet :)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "CaQFquQbKv2f",
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Software\n",
    "\n",
    "> We will use the **Anaconda Individual Edition 2021.05** or newest with **Python 3.8**\n",
    "\n",
    "- Compatible with Windows, Mac and Linux\n",
    "- Contains all the packages that we will need\n",
    "- Easy to install additional packages\n",
    "- Backed up by a for-profit company, Anaconda http://anaconda.com/ (formerly Continuum Analytics)\n",
    "- For a cloud option, we will use Binder https://mybinder.org/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "tIdHggZ9Kv2m",
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Shall we begin?\n",
    "\n",
    "![Talk is cheap](img/quote-talk-is-cheap-show-me-the-code-linus-torvalds-273528.jpg)"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "colab": {
   "collapsed_sections": [],
   "name": "Course Introduction.ipynb",
   "provenance": [],
   "toc_visible": true,
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "rise": {
   "autolaunch": true,
   "enable_chalkboard": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}