1_Introduction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Table of Contents\n",
    "- **What is Python**\n",
    "- **Python environments**\n",
    "- **Setup Guide**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# What is Python?\n",
    "\n",
    "Python is a modern, general-purpose, object-oriented, high-level programming language.\n",
    "\n",
    "General characteristics of Python:\n",
    "\n",
    "* *clean and simple language*: \n",
    "    * easy to read code, \n",
    "    * easy to learn, natural, syntax, \n",
    "    * maintainability scales well with size of projects.\n",
    "* *expressive language*: fewer lines of code, fewer bugs, easier to maintain.\n",
    "\n",
    "Technical details:\n",
    "\n",
    "* *dynamically typed*: no need to define the type of variables, function arguments or return types,\n",
    "* *automatic memory management*: no need to explicitly allocate and deallocate memory for variables and data arrays,\n",
    "* *interpreted**: source code is not compiled to binary machine code but pre-compiled in an intermediate representation (named bytecode); the Python interpreter reads and executes the bytecode,\n",
    "* *object-oriented*: effective in tackling complexity for large programs.\n",
    "\n",
    "Advantages:\n",
    "\n",
    "* Ease of programming, minimizing the time required to develop, debug and maintain the code,\n",
    "* Well designed language that encourage many good programming practices:\n",
    "  * “Blocks by Indentation” forces proper code structuring & readability,\n",
    "  * Modular and object-oriented programming, good system for packaging and re-use of code,\n",
    "  * Documentation tightly integrated with the code,\n",
    "* A large standard library, and a large collection of add-on packages.\n",
    "\n",
    "Disadvantages:\n",
    "\n",
    "* Since Python is an interpreted and dynamically typed programming language, the execution of python code can be slow compared to compiled statically typed programming languages, such as C/C++,\n",
    "* Somewhat decentralized, with different environment, packages and documentation spread out at different places."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Python environments\n",
    "\n",
    "There are many different *environments* through which the Python *interpreter* can be used. Each environment has different advantages and is suitable for different workflows. One strength of Python is that it is versatile and can be used in complementary ways, but it can be confusing for beginners so we will start with a brief survey of python environments."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "## Python interpreter\n",
    "\n",
    "The standard way to use the Python programming language is to use the Python interpreter to run Python code. The **Python interpreter is a program that reads and execute the python code in files passed to it as arguments**. At the command prompt, the command `python` is used to invoke the Python interpreter.\n",
    "For example, to run a file `my-program.py` that contains Python code from the command prompt, use:\n",
    "\n",
    "```shell\n",
    "$ python my-program.py\n",
    "```\n",
    "\n",
    "We can also start the interpreter by simply typing `python` at the command line, and interactively type Python code into the interpreter (press `Ctrl+D` or type `exit()` to exit):\n",
    "\n",
    "```python\n",
    "while True:\n",
    "  code = input(\">>> \")  # prompt the user for some code\n",
    "  exec(code)        # execute it\n",
    "```\n",
    "\n",
    "This model is often called a REPL, or Read-Eval-Print-Loop: \n",
    "\n",
    "The `>>>` symbols represent the *prompt* where you will type code expressions. The interpreter awaits our instructions step by step. This is often how we want to work when quick testing **small portions of code**, or when doing small calculations. \n",
    "\n",
    "\n",
    "While some Python programmers execute all of their Python code in this way, those doing data analysis or scientific computing make use of IPython (an enhanced Python interpreter) or Jupyter notebooks (web-based code notebooks originally created within the IPython project).\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## IPython\n",
    "IPython (*Interactive Python*) is a **command shell for interactive computing in Python**.\n",
    "\n",
    "IPython offers a fully compatible replacement for the standard Python interpreter, with convenient shell features, special commands, command history mechanism and output results caching.\n",
    "\n",
    "We can start IPython by simply typing `ipython` at the command (press `Ctrl+D` or type `exit()` to exit).\n",
    "\n",
    "The original IPython interface runs an interactive shell built with Python. The name shell indicates that it is the outermost layer around the kernel (i.e. the engine capable of executing code) and allows user to access it through a user interface. \n",
    "\n",
    "The default IPython prompt adopts the numbered `In [1]:` style compared with the standard `>>>` prompt. Just as with the standard interpreter, you can execute arbitrary Python statements by typing them in and pressing Return (or Enter)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Jupyter Notebook\n",
    "\n",
    "The \"Project **Jupyter**\" denotes an organization created with the aim of **supporting interactive data science and scientific computing** via the development of open-source software.\n",
    "\n",
    "The **Notebook** term may refer both to the notebook document and the web-based environment used to create it. Jupyter Notebook represents another interface to the IPython kernel and can connect to it to allow interactive programming in Python. It uses an internal library for converting the document in HTML and allows visualization and editing in the browser. Although using a web browser as graphical interface, Jupyter notebooks are usually run locally, from the same computer that runs the browser. \n",
    "\n",
    "![notebook image](https://jupyter.readthedocs.io/en/latest/_images/notebook_components.png)\n",
    "\n",
    "The document you are reading, with extension `.ipynb`, is a notebook and consists in an ordered list of cells which can contatin code, text, mathematics, visualization of output and plots.\n",
    "\n",
    "Jupyter notebooks are particularly useful as scientific lab books when you are doing lots of data analysis using computational tools. This is because, with Jupyter notebooks, you can:\n",
    "* **Record the code you write in a notebook as you manipulate your data**. This is useful to remember what you've done, repeat it if necessary, etc.\n",
    "* **Graphs and other figures are rendered directly in the notebook**.\n",
    "* You can **update the notebook (or parts thereof) with new data by re-running cells**. You could also copy the cell and re-run the copy only if you want to retain a record of the previous attempt."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " There are two types of cell:\n",
    "- **Markdown cells**: contain explanatory text.\n",
    "- **Code cells**:  contain executable code.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Text cells \n",
    "\n",
    "Text cells (like this) use *markdown syntax*:  it consists in plain text formatting syntax that enables the creation of rich text that can be converted to HTML!\n",
    "\n",
    "You can include well-formatted text, formulas, and images too."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Code cells\n",
    "A code cell contains executable code and displays its output just below.\n",
    "The subsequent cells are examples of code cell: execute them clicking the play button or using Ctrl+Enter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# this is an executable code cell. \n",
    "a = 2\n",
    "b = 4\n",
    "a*b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In *code cells* you can use question mark for accessing the documentation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "a?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In *code cells* you can also use system aliases. Use exclamation mark for terminal operation: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Python 3.9.7\n"
     ]
    }
   ],
   "source": [
    "!python --version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1_Introduction.ipynb\n",
      "2_Python_Basics.ipynb\n",
      "3_Pandas_Tutorial.ipynb\n",
      "4_ScikitLearn_Tutorial.ipynb\n",
      "A.Y. 2020-2021\n",
      "Binder_Practice.ipynb\n",
      "data\n",
      "images\n",
      "img\n",
      "my-program.py\n",
      "README.md\n"
     ]
    }
   ],
   "source": [
    "!ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setup Guide: Installation of Python\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Versions of Python\n",
    "\n",
    "from [python wiki](https://wiki.python.org/moin/Python2orPython3):\n",
    "\n",
    "*Python 2.x is legacy, Python 3.x is the present and future of the language*\n",
    "\n",
    ">*Python 3.0 was released in 2008. The final 2.x version 2.7 release came out in mid-2010, with a statement of extended support for this end-of-life release. The 2.x branch will see no new major releases after that.*\n",
    "\n",
    ">*As of January 2020, Python 2 has reached End Of Life (EOL) status, meaning it will receive no further updates or bugfixes, including for security issues. Many frameworks and other add on projects are following a similar policy.*\n",
    "\n",
    "\n",
    ">*As such, we can only recommend learning and teaching Python 3.*\n",
    "\n",
    "To see which version of Python you have, run:\n",
    "```shell\n",
    "$ python --version\n",
    "Python 3.9.7\n",
    "```\n",
    "\n",
    "Several versions of Python can be installed in parallel.\n",
    "\n",
    "### IMPORTANT: In this course we will use Python 3\n",
    "\n",
    "There are several differences between Python 2.7.x and Python 3.x; the most relevant ones are reported [here](https://sebastianraschka.com/Articles/2014_python_2_3_key_diff.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "New users are recommended to download and install **Anaconda**.\n",
    "It is a package manager, an environment manager, a Python distribution, and a collection of 1,000+ open source packages. Among the available packages:\n",
    "- **jupyter**;\n",
    "- **numpy**: fundamental package for scientific computing with Python;\n",
    "- **matplotlib**: a Python plotting library;\n",
    "- **pandas**: a Python library for data pre-processing and data analysis;\n",
    "- **scikit-learn**: a Machine Learning library in Python;\n",
    "- **NLTK** (Natural Language Toolkit): platform for building Python programs to work with human language data.\n",
    "\n",
    "Download Anaconda from the [official website](https://www.anaconda.com/products/individual) and follow the [instructions](https://docs.anaconda.com/anaconda/install/) for your OS.\n",
    "\n",
    "**conda** is the package and environment manager provided with Anaconda. From the [conda website](https://conda.io/en/latest/):\n",
    "> Conda is an open source **package management system** and **environment management system** that runs on Windows, macOS and Linux. Conda quickly installs, runs and updates packages and their dependencies. Conda easily creates, saves, loads and switches between environments on your local computer. It was created for Python programs, but it can package and distribute software for any language.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Managing packages\n",
    "\n",
    "In programming, a module is a piece of software that has a specific functionality. In Java, the term *package* is often used as a synonym of module. In Python, a *package* is a collection of modules.\n",
    "\n",
    "-  `conda` provides many commands for installing packages from the Anaconda repository and cloud.\n",
    "-  `pip` is the Python Packaging Authority’s recommended tool for installing packages from the Python Package Index, PyPI.\n",
    "\n",
    "For example, you can install a given package by typing the following commands in the terminal: \n",
    "> ```conda install [package name]```\n",
    "\n",
    "or\n",
    "\n",
    "> ```pip install [package name]``` \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Managing Environments\n",
    "\n",
    "Python has its own unique way of downloading, storing, and locating packages (or modules). There are a few different locations where these packages can be installed on your system. \n",
    "\n",
    "Third party packages, or **site packages**, installed using `pip` or `conda` are typically placed in one of the directories pointed to by `site.getsitepackages`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-07-14T09:38:08.140606Z",
     "start_time": "2020-07-14T09:38:08.113511Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['C:\\\\Users\\\\Alessandro\\\\anaconda3\\\\envs\\\\aide', 'C:\\\\Users\\\\Alessandro\\\\anaconda3\\\\envs\\\\aide\\\\lib\\\\site-packages']\n"
     ]
    }
   ],
   "source": [
    "import site\n",
    "\n",
    "print(site.getsitepackages())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By default, every project on your system will use these same directories to store and retrieve site packages. \n",
    "Site packages are stored according to just their name, there is no differentiation between versions.\n",
    "\n",
    "Consider the following scenario\n",
    "- you have two projects: *ProjectA* and *ProjectB*.\n",
    "- Both projects have a dependency on the same package, *ProjectC*. \n",
    "- *ProjectA* needs *ProjectC* `v1.0.0`.\n",
    "- *ProjectB* needs *ProjectC* `v2.0.0`.\n",
    "\n",
    "This is a real problem for Python since it can’t differentiate between versions in the `site-packages` directory. So both `v1.0.0` and `v2.0.0` would reside in the same directory with the same name.\n",
    "\n",
    "An **environment manager** allows to create lightweight \"virtual environments\" with their own site directories, optionally isolated from system site directories. This means that each project can have its own dependencies, regardless of what dependencies every other project has.\n",
    "\n",
    "In our example, we would just need to create a separate virtual environment for both *ProjectA* and *ProjectB*.\n",
    "Each environment, in turn, would be able to depend on whatever version of *ProjectC* they choose, independent of the other.\n",
    "\n",
    "Practically, **virtual environments are just directories containing a few scripts**.\n",
    "\n",
    "Availble tools:\n",
    "-  `conda` provides many commands for managing environments\n",
    "-  `venv` is the Python tool for managing virtual environments\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Differences between `conda`, `pip`, `virtualenv`\n",
    "\n",
    "Look at [the documentation](https://docs.conda.io/projects/conda/en/latest/commands.html#conda-vs-pip-vs-virtualenv-commands) for the full-size table, and at [this blog post](https://www.anaconda.com/blog/understanding-conda-and-pip).\n",
    "![jupyter-home.png](images/conda-pip-venv.PNG)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating a conda environment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Create the environment named \"aideLAB\" using the following command (Unix shell or Anaconda prompt)\n",
    "\n",
    "```bash\n",
    "$ conda create --name aideLAB python=3.9\n",
    "```\n",
    "\n",
    "2. Get the list of available environments\n",
    "\n",
    "```bash\n",
    "$ conda env list\n",
    "```\n",
    "3. Activate your newly created environment\n",
    "\n",
    "```bash\n",
    "$ conda activate aideLAB\n",
    "```\n",
    "\n",
    "4. Install `jupyter` (or any other needed package)\n",
    "\n",
    "```bash\n",
    "(cyberLab)$ conda install jupyter\n",
    "```\n",
    "or\n",
    "\n",
    "```bash\n",
    "(cyberLab)$ pip install jupyter \n",
    "```\n",
    " \n",
    "5. If you need to go back to the \"system context\":\n",
    "\n",
    "```bash\n",
    "(cyberLab)$ conda deactivate\n",
    "```\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  Jupyter Notebook Usage\n",
    "\n",
    "After installing jupyter using `conda` or `pip`, to start a Jupyter notebook server, navigate to a suitable working directory and type the following command (Unix shell or Anaconda prompt)\n",
    "```bash\n",
    "(aideLAB)$ jupyter notebook\n",
    "```\n",
    "\n",
    "This starts a Jupyter notebook server and automatically opens it in the browser. You should get something like this:\n",
    "\n",
    "![jupyter-home.png](images/jupyter-home.png)\n",
    "\n",
    "Here's a quick list of the main functionalities in Jupyter notebooks (have a look at `Help > Keyboard Shortcuts`)\n",
    "* Start a new notebook clicking `New` in the top-right corner\n",
    "* Type in some Python code in a **cell** and press `Shift` + `Enter` to execute.\n",
    "* Change the cell type from Code to Markdown using the drop-down box (top-middle) to write explanatory text. Markdown guidance is available [here](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet%20%22Markdown%20cheat-sheet%20on%20GitHub). \n",
    "* Add new cells using the `+` button (top-left).\n",
    "* Save the notebook using the disk button (top-left).\n",
    "* Run cells in various ways (run all, run selection, run all above, run all below, etc.) using the options in the `Cell` menu.\n",
    "* Interrupt the kernel, restart it, clear all output, etc. using the options in the `Kernel` menu.\n",
    "* Download as Python script, HTML, LaTeX, etc. using the options under `File > Download as`.\n",
    "\n",
    "An open notebook has exactly one interactive session connected to a kernel which will execute code sent by the user and communicate back results. \n",
    "\n",
    "This kernel remains active if the web browser window is closed, and reopening the same notebook from the dashboard will reconnect the web application to the same kernel."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**To sum up**:\n",
    "\n",
    "You may want to use an integrated development environment (IDE), e.g. Spyder (comes with Anaconda installation), PyCharm, JupyterLab to name but a few, for the following reasons:\n",
    "- more complex projects (many files)\n",
    "- debug utilities\n",
    "- powerful code editing tools\n",
    "\n",
    "You may prefer Jupyter notebook:\n",
    "- for scripts and prototypes\n",
    "- for interactive devoloping\n",
    "- for visualization purpose\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}