Week 1 Day 2 - Intro to Jupyter.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "u6cuwnN0VQ0z"
   },
   "source": [
    "# Introduction to Jupyter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dOopGhfIVQ06"
   },
   "source": [
    "**Acknowledgement**:\n",
    "This notebook was adapted from a [chapter](https://gureckislab.org/courses/spring21/labincp/chapters/02/00-jupyter.html) authored by [Todd M. Gureckis](http://gureckislab.org/~gureckis) is released under the CC BY-SA 4.0 license. Several of the sections on how to use the Jupyter Interface were developed and shared by [Jessica Hamrick](http://www.jesshamrick.com) as part of a course developed at UC Berkeley.\n",
    "\n",
    "For extra orientation, I recommend checking out that chapter, as well as the [accompanying video lecture](https://vimeo.com/501114991) by Todd. It's quite excellent!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IfZNkVhxVQ0-"
   },
   "source": [
    "## A bit more about Jupyter notebooks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TCZq1O0fVQ0-"
   },
   "source": [
    "The \"notebook\" interface is an approach to a programming and data analysis which combines programming/analysis code with text, images, figure, animations, and other multimedia in the same document.   The idea for computational notebooks has been around for a long time (e.g., [Mathematica](https://www.wolfram.com/mathematica/) is a commercial product that has long provided a notebook interface for data analysis and programming).  However, Jupyter is now one of the leading tools that data scientists use to analyze their data and write code.\n",
    "\n",
    "\n",
    "There are several reasons for the popularity of Jupyter:\n",
    "\n",
    "1. Jupyter is open source and free.  This means it is not owned by any one person or company.  Scientists and other programmers devote their time to improving Jupyter.  The code is readable by anyone which means it is easier to catch bugs.\n",
    "2. Jupyter works with several different programming languages including R, Python, Matlab and others which are popular for data analysis.\n",
    "3. Jupyter runs in a standard webbrowser.  This means you can use Jupyter often on any computing device you like including low-cost laptops like Chromebooks/Netbooks or even a tablet like an iPad.\n",
    "4. Jupyter has many extensions which provide interactive widgets, animations, beautiful graphics, etc...\n",
    "5. Jupyter is fun!  Once you get the hang of the notebook concept then it makes develping and debugging code a pretty fun experience.\n",
    "6. Many companies use Jupyter internally and are looking to hire people with this skill."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rqzoH9dnVQ1D"
   },
   "source": [
    "## The \"notebook\" and \"kernel\" concepts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PUUpzTD1VQ1E"
   },
   "source": [
    "The key to Jupyter is the concept of a **computational notebook**.  A computational notebook is a document-based approach to structuring data analysis and programming.  A notebook appears and acts similar to other programs you are familiar with like Excel or Google Sheets (spreadsheets) or Word or Google Docs (word processor).  These programs provide ways for you to enter information into documents and to save those file, send them to others, copy-paste, text, alter the formatting, etc...\n",
    "\n",
    "We will primarily use the \"classic\" Jupyter Notebook interface which consists of a single document which you interact with not unlike a word processor.  There is a newer interface called [JupyterLab](https://blog.jupyter.org/jupyterlab-is-ready-for-users-5a6f039b8906) which looks more like Matlab or RStudio's interface allowing multiple windows and views of the running code.  The traditional notebook interface is prefered here primarily because it simple and faciliates reading along.\n",
    "\n",
    "### Cells\n",
    "\n",
    "A computational notebook typically is made up of a set of elements called \"cells\" that group little bits of information together.  A notebook document (usually ending with the .ipynb extension on the file name) is a list of cells arranged from the top of the document to the bottom.  Cells can contain several types of information including code, text, images and so forth.  You develop your project by creating new cells, entering information, reordering cells, and saving the results.\n",
    "\n",
    "### The Kernel\n",
    "\n",
    "The key element which makes a notebook \"computational\" is that it can be linked to a computing engine known as a \"kernel\" which can run on the same computer or another computer using the Internet.  The kernel is separate from the notebook and can for instance be stopped and started independently from the program displaying the notebook itself. The magic happens when a cell is \"executed.\"  This sounds scary but refers simply to making the code in the cell \"go\".  When this happens the code from the cell that was executed is sent to the computational kernel and this kernel will run the sequence of commands contained within the cell.  Any results such as figures or printouts from the results of that code running will be sent to the notebook from the kernel and will be inserted beneath the cell which was just executed.  \n",
    "\n",
    "A diagram of the basic conceptual idea is shown in the following figure that sketches out the flow between the notebook and the kernel.  Cells when they are executed are sent to the computational kernel which is an independent piece of software which can be started and stopped independently from the notebook.\n",
    "\n",
    "\n",
    "<img src=\"https://gureckislab.org/courses/spring21/labincp/_images/jupyterflow.png\" width=\"400\" height=\"200\" />\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dua5W2zDVQ1F"
   },
   "source": [
    "### Caution: The Notebook State and Kernel State are not the Same!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7PvIbyhXVQ1G"
   },
   "source": [
    "One somewhat confusing aspect of Jupyter for beginners is that the notebook is simply a way to organize information and the order of information in a notebook has nothing to do with the order in which code has been executed on the server.\n",
    "\n",
    "Instead, each time you execute a cell, the commands from that cell are sent to the kernel, run, and then returned.  Usually it make sense to organize your code so that multiple cells are arranged in order from the top of the document to the bottom.  You would then execute the first cell, then the second cell, and so forth. \n",
    "\n",
    "However, **and this is really important**, you don't have to execute the cells in order.  You can execute the third cell before the second or first cell if you want.  This just means that the code in the third cell would be sent to the kernel first and then the other cells would be sent.  This means that the order of the cells in the notebook document are really just for presentation, and it is up to you to \"run/execute\" the cells in whatever order you want.\n",
    "\n",
    "This is mostly helpful because when you are doing and analysis you might want to go back up to the top and change something and then continue where you left off.  To do that you could scroll back to a earlier cell, change it, then execute it and then go back down to the beginning of the script.  \n",
    "\n",
    "However, things can get confusing.  If you delete a cell it doesn't mean the code was \"undone\".  The kernel doesn't know anything about any edits you make to the notebook interface.  All it does is receive code when you send it via the \"execute cell\" command and then it waits.  This can mean that the \"state\" or current status of the kernel can at times be different from what your cells or notebook look like.  One easy option to deal with this is that if things get too far out of wack you can simply restart the kernel.  This will erase it's current state and set it back to a fresh start.  However, then you need to re-execute all of the cells in your notebook to get back to where you were.\n",
    "\n",
    "The best analogy is like telling your friend how to cook a meal for you over the phone.  You have the instructions in front of you (your notebook) and you tell your friend (the kernel) what to do.  If you read the instructions out of order your friend can't tell the difference.  They just know each step you told them to do and follow it.  It is up to you to direct the instructions so they arrive to your friend (the kernel) in the right order.  And of course if things get confused you can just tell your friend to start over from stratch and begin again.\n",
    "\n",
    "At first this can be confusing but eventually you get the hang of it.  It is one source of bugs/errors in Jupyter programming though and some people don't like using Jupyter for that reason.  In our case the upsides overweight this awkwardness.\n",
    "\n",
    "<img src=\"https://gureckislab.org/courses/spring21/labincp/_images/jupyterwarning.png\" width=400 height=200></img>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "riqYnXQ0VQ1H"
   },
   "source": [
    "## Overview of the Jupyter Notebook"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ezo_sTYpVQ1I"
   },
   "source": [
    "In the following section we will go over the basics of the Jupyter interface.  While you can easily read this file on the web as a reminder it can be helpful to actually open this file in Jupyter so you can follow along and see the interface as it will actually appear.  Refer to the earlier section for several methods for opening this file in a Jupyter instance.\n",
    "\n",
    "First, as described, notebooks are made up of a  series of *cells*. For example, the text you are reading is in what is called a \"Markdown cell\". The following cell is a \"code cell\":"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "id": "IVWpatXaVQ1J"
   },
   "outputs": [],
   "source": [
    "# this is a code cell"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bJfWPMIpVQ1L"
   },
   "source": [
    "You can tell what the type of a cell is by selecting the cell, and looking at the toolbar at the top of the page. For example, try clicking on this cell. You should see the cell type menu displaying \"Markdown\", like this:\n",
    "\n",
    "<img src=\"https://gureckislab.org/courses/spring21/labincp/_images/cell-type-menu.png\" width=400 height=200>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ct6hKP4GVQ1L"
   },
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lCHvL0NyVQ1M"
   },
   "source": [
    "## Command mode and edit mode"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OG81rOoeVQ1N"
   },
   "source": [
    "In the notebook, there are two modes: *edit mode* and *command mode*. By default the notebook begins in *command mode*. In order to edit a cell, you need to be in *edit mode*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Y9JYOxRhVQ1O"
   },
   "source": [
    "**When you are in command mode**, you can press **enter** to switch to edit mode. The outline of the cell you currently have selected will turn green, and a cursor will appear."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Cpqa-xFxVQ1O"
   },
   "source": [
    "**When you are in edit mode**, you can press **escape** to switch to command mode. The outline of the cell you currently have selected will turn gray, and the cursor will disappear."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Xq8W9mrlVQ1P"
   },
   "source": [
    "### Code cells\n",
    "\n",
    "This is what a code cell looks like in command mode (Note: the following few cells are not actually cells -- they are images and just look like cells! This is for demonstration purposes only.):\n",
    "\n",
    "<img src=\"https://gureckislab.org/courses/spring21/labincp/_images/command-mode-outline.png\" width=400 height=200></img>\n",
    "\n",
    "If we press enter, it will change to **edit mode**:\n",
    "\n",
    "\n",
    "<img src=\"https://gureckislab.org/courses/spring21/labincp/_images/edit-mode-outline.png\" width=400 height=200></img>\n",
    "\n",
    "And pressing escape will also go back to **command mode**:\n",
    "\n",
    "<img src=\"https://gureckislab.org/courses/spring21/labincp/_images/command-mode-outline.png\" width=400 height=200></img>\n",
    "\n",
    "If we were to press **`Ctrl-Enter`**, this would actually *run* the code in the code cell:\n",
    "\n",
    "<img src=\"https://gureckislab.org/courses/spring21/labincp/_images/code-cell-run.png\" width=400 height=200></img>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TLFZAZewVQ1Q"
   },
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HP1NlwvNVQ1Q"
   },
   "source": [
    "## Executing cells"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oAqkRRCJVQ1Q"
   },
   "source": [
    "Code cells can contain any valid Python code in them. When you run the cell, the code is executed and any output is displayed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HWkuV3_0VQ1Q"
   },
   "source": [
    "**HINT**: \n",
    "You can execute cells with **`Ctrl-Enter`** (which will keep the cell selected), or **`Shift-Enter`** (which will select the next cell)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7hGve56PVQ1R"
   },
   "source": [
    "Try running the following cell and see what it prints out:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "id": "fFHE6Fe2VQ1R"
   },
   "outputs": [],
   "source": [
    "print(\"Printing cumulative sum from 1-10:\")\n",
    "total = 0\n",
    "for i in range(1, 11):\n",
    "    total += i\n",
    "    print(\"Sum of 1 to \" + str(i) + \" is: \" + str(total))\n",
    "print(\"Done printing numbers.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "egvBx3GWVQ1S"
   },
   "source": [
    "You'll notice that the output beneath the cell corresponds to the `print` statements in the code. Here is another example, which only prints out the final sum:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "id": "frBcOi-bVQ1S"
   },
   "outputs": [],
   "source": [
    "total = 0\n",
    "for i in range(1, 11):\n",
    "    total += i\n",
    "print(total)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "X0uIO1PTVQ1T"
   },
   "source": [
    "Another way to print something out is to have that thing be the last line in the cell. For example, we could rewrite our example above to be:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "id": "FU7Z7knZVQ1T"
   },
   "outputs": [],
   "source": [
    "total = 0\n",
    "for i in range(1, 11):\n",
    "    total += i\n",
    "total"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "07gOJUPMVQ1U"
   },
   "source": [
    "However, this *will not work* unless the thing to be displayed is on the last line. For example, if we wanted to print the total sum and then a message after that, this will not do what we want (it will only print \"Done computing total.\", and not the total sum itself)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "id": "j6U14MbWVQ1U"
   },
   "outputs": [],
   "source": [
    "total = 0\n",
    "for i in range(1, 11):\n",
    "    total += i\n",
    "total\n",
    "print(\"Done computing total.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "w9-geWJpVQ1V"
   },
   "source": [
    "If you are accustomed to Python 2, note that the parentheses are obligatory for the `print` function in Python 3."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-MPmys_iVQ1V"
   },
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oOZlsy3bVQ1V"
   },
   "source": [
    "## The IPython kernel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "f-LTG0qHVQ1W"
   },
   "source": [
    "When you first start a notebook, you are also starting what is called a *kernel*. This is a special program that runs in the background and executes code (by default, this is Python, but it could be other languages too, like R!). Whenever you run a code cell, you are telling the kernel to execute the code that is in the cell, and to print the output (if any).\n",
    "\n",
    "Just like if you were typing code at the Python interpreter, you need to make sure your variables are declared before you can use them. What will happen when you run the following cell? Try it and see:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "id": "tnPV_uUqVQ1W"
   },
   "outputs": [],
   "source": [
    "a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "g0AcUcWlVQ1W"
   },
   "source": [
    "The issue is that the variable `a` does not exist. Modify the cell above so that `a` is declared first (for example, you could set the value of `a` to 1 -- or pick whatever value you want). Once you have modified the above cell, you should be able to run the following cell (if you haven't modified the above cell, you'll get the same error!):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "id": "VMgPF6bKVQ1W"
   },
   "outputs": [],
   "source": [
    "print(\"The value of 'a' is: \" + str(a))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HpI_go8mVQ1X"
   },
   "source": [
    "Running the above cell should work, because `a` has now been declared. To see what variables have been declared, you can use the `%whos` command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "id": "DN6BpqoeVQ1X"
   },
   "outputs": [],
   "source": [
    "%whos"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zvAlkgfQVQ1X"
   },
   "source": [
    "If you ran the summing examples from the previous section, you'll notice that `total` and `i` are listed under the `%whos` command. That is because when we ran the code for those examples, they also modified the kernel state."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "q-B1Aw5MVQ1X"
   },
   "source": [
    "(Note that commands beginning with a percent (%) or double percent (%%) are special IPython commands called *magics*. They will **only** work in IPython.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "b4tQJp1SVQ1X"
   },
   "source": [
    "### Restarting the kernel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SgDp_v2WVQ1X"
   },
   "source": [
    "It is generally a good idea to periodically restart the kernel and start fresh, because you may be using some variables that you declared at some point, but at a later point deleted that declaration. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WDY-dqVHVQ1Y"
   },
   "source": [
    "Your code should <b>always</b> be able to work if you run every cell in the notebook, in order, starting from a new kernel."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rYAW0sqPVQ1Y"
   },
   "source": [
    "To test that your code can do this, first restart the kernel by clicking the restart button:\n",
    "\n",
    "<img src=\"https://gureckislab.org/courses/spring21/labincp/_images/restart-kernel-button.png\" width=400 height=200></img>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RFP-NoPsVQ1Y"
   },
   "source": [
    "Then, run all cells in the notebook in order by choosing **Cell$\\rightarrow$Run All** from the menu above."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VNjAgtQQVQ1Y"
   },
   "source": [
    "**HINTS**: \n",
    "- There are many keyboard shortcuts for the notebook. To see a full list of these, go to <b>Help$\\rightarrow$Keyboard Shortcuts</b>.\n",
    "- To learn a little more about what things are what in the IPython Notebook, check out the user interface tour, which you can access by going to <b>Help$\\rightarrow$User Interface Tour</b>."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FjbcsKYbVQ1Z"
   },
   "source": [
    "## Side note: the words in this notebook are also code! They're in a markup language called Markdown"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DHbjVc1kVQ1Z"
   },
   "source": [
    "Markdown is a special way of writing text in order to specify formatting, like whether text should be bold, italicized, etc. If you're curious, you can read more here: [https://help.github.com/articles/markdown-basics](https://help.github.com/articles/markdown-basics)\n",
    "\n",
    "- Hint #1: after editing the Markdown, you will need to run the cell so that the formatting appears.\n",
    "- Hint #2: try selecting *this* cell so you can see what the Markdown looks like when you're editing it. Remember to run the cell again to see what it looks like when it is formatted.\n",
    "\n",
    "Markdown is a useful tool in Jupyter for making things look nice but you don't have to be a Markdown expert to succeed at this class."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6VvBQevyVQ1a"
   },
   "source": [
    "## Additional Resources\n",
    "\n",
    "This chapter gives you a basic introduction to the Jupyter Notebook system we will be using more or less daily in this class.  It is something you will get more familiar with through experience.  However, if you want additional resources here are some useful guides and videos introducing Jupyter:\n",
    "\n",
    "- [What is Jupyter Notebook? (youtube)](https://www.youtube.com/watch?v=q_BzsPxwLOE)\n",
    "- [Jupyter Notebook: An Introduction (Mike Driscoll, RealPython.com)](https://realpython.com/jupyter-notebook-introduction/)\n",
    "- [Project Jupyter homepage](https://jupyter.org)\n",
    "- [List of other programming languages/kernels you can use with Jupyter](https://github.com/jupyter/jupyter/wiki/Jupyter-kernels)\n"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "name": "00-jupyter.ipynb",
   "provenance": []
  },
  "kernel_info": {
   "name": "python3"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  },
  "nteract": {
   "version": "0.15.0"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}