3-Introduction to NumPy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to NumPy\n",
    "Prepared by Jen Nunnelley with influence from Nickolas Freeman, Ph.D. and Irem Sengul orgut, Ph.D.\n",
    "\n",
    "This notebook provides an introduction to the `numpy` package. The content borrows heavily from the book *Data Science Handbook*, which was written by Jake VanderPlas and is available at https://jakevdp.github.io/PythonDataScienceHandbook/ (accessed 12/17/2019).\n",
    "\n",
    "The following table of contents lists the topics discussed in this notebook. Clicking on any topic will advance the notebook to the associated area.\n",
    "\n",
    "\n",
    "# Table of Contents\n",
    "<a id=\"Table_of_Contents\"> </a>\n",
    "\n",
    "1. [NumPy](#numpy)<br>\n",
    "    1.1 [Importing NumPy](#Importing_NumPy)<br>\n",
    "    1.2 [NumPy Arrays](#NumPy_Arrays)<br>\n",
    "    1.3 [Basic Computations using NumPy Arrays](#Array_Computation)<br>\n",
    "    1.4 [Aggregations](#Array_Aggregations)<br>   \n",
    "    \n",
    "#### Disclaimer\n",
    "\n",
    "This notebook is by no means a comprehensive resource for the `numpy` package. Also, it is important to realize that the Python language and the available packages will continue to evolve. That being said, the objects, functions, and methods described in this notebook may one day change. If changes occur, areas of this notebook that use deprecated features may cease to work and will need to be revised or omitted."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## NumPy\n",
    "<a id=\"numpy\"> </a>\n",
    "\n",
    "This section of the notebook will introduce the `numpy` package and demonstrate several features of the `numpy` multi-dimensional array. \n",
    "\n",
    "From https://en.wikipedia.org/wiki/NumPy (accessed on 1/6/2018):\n",
    "\n",
    ">NumPy is a library for the Python programming language, adding support for large, multi-dimensional arrays and matrices, along with a large collection of high-level mathematical functions to operate on these arrays. The ancestor of NumPy, Numeric, was originally created by Jim Hugunin with contributions from several other developers. In 2005, Travis Oliphant created NumPy by incorporating features of the competing Numarray into Numeric, with extensive modifications. NumPy is open-source software and has many contributors.\n",
    "\n",
    "[Back to Table of Contents](#Table_of_Contents)<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importing NumPy\n",
    "<a id=\"Importing_NumPy\"> </a>\n",
    "\n",
    "Users can import available packages and modules using Python's `import` statement. Two forms of import expressions are commonly used.\n",
    "1. The first common import expression takes the form **import mypackage as mp**. This statement imports a package named *mypackage*, and assigns it to the alias *mp*. Suppose that *mypackage* contains the definition for a function named *myfunction*. If this were true, we would call *myfunction* using the syntax `mp.myfunction(*args)`, where `*args` is a placeholder for any function arguments.<br>\n",
    "\n",
    "2. The second common import expression takes the form **from mypackage import mysubmodule**. This statement imports a specific submodule from a package named *mypackage*. Since there is no alias, this type of import will bring in the functions specified in the submodule as they are writtin. For example, if the submodule *mysubmodule* includes a function called *myfunction*, we would call *myfunction* using the syntax `myfunction(*args)`, where `*args` is a placeholder for any function arguments.\n",
    "\n",
    "<div class=\"alert alert-block alert-danger\">\n",
    "    <b>Name conflicts with <i>from - import</i> approach:</b> When using the <i>from - import</i> approach sepcified in bullet 2, it is important to make sure that method names in package or module you are importing do not conflict with names defined in the importing code. For example, if we import a submodule named *mysubmodule* that includes a function called *myfunction*, but we also have a function named *myfunction* in the importing code, there will be a naming conflict.\n",
    "</div>\n",
    "\n",
    "The following code block uses the *import - as* approach to import NumPy. The alias *np* is a standard convention.\n",
    "\n",
    "[Back to Table of Contents](#Table_of_Contents)<br>\n",
    "\n",
    "<b> Using the first way to import packages, import NumPy. <b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Motivation for NumPy\n",
    "\n",
    "Before we look at specific details of the NumPy package, it is important to understand its motivation. NumPy was developed to support scientific computations via the efficient implementation of a multi-dimensional array. In addition to an efficient array implementation, NumPy also includes functions for performing operations on NumPy arrays that are optimized for computational efficeincy. The following code block illustrate the substantial increas in efficiency that NumPy provides in comparison to a standard python list. Specifically, the example considers the task of adding two vectors of a specified size using both standard Python lists and NumPy arrays. The time of the addition, and the size of the resulting objects are reported for comparison purposes.\n",
    "\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "    <b>The <i>del()</i> function:</b> The <i>del()</i> function is a python method that deletes a created object from memory. For example, <i>del(my_var)</i> deletes a Python variable named <i>my_var</i>, freeing any computer memory that was being used to store the variable. The <i>del(my_var)</i> can take multiple arguments. For example, <i>del(my_var1, my_var2)</i> deletes the Python variables named <i>my_var1</i> and <i>my_var2</i>. If you pass an argument to <i>del(my_var)</i> that does not correspond to an existing Python object, an error will be raised.\n",
    "</div>\n",
    "\n",
    "[Back to Table of Contents](#Table_of_Contents)<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "import sys\n",
    "\n",
    "SIZE = 100000\n",
    "\n",
    "# Using Python List\n",
    "list1 = range(SIZE)\n",
    "list2 = range(SIZE)\n",
    "\n",
    "start = time.time()\n",
    "result = [(x+y) for x,y in zip(list1,list2)]\n",
    "print(\"Using Python lists, the addition took\",(time.time() - start)*1000,\"milliseconds.\")\n",
    "print(\"The size of the result object based on Python lists is\",sys.getsizeof(result),\"bytes.\\n\")\n",
    "\n",
    "del(list1, list2, result)\n",
    "\n",
    "#Using NumPy List\n",
    "nparray1 = np.arange(SIZE)\n",
    "nparray2 = np.arange(SIZE)\n",
    "start = time.time()\n",
    "result = nparray1 + nparray2\n",
    "print(\"Using NumPy arrays, the addition took\",(time.time() - start)*1000,\"milliseconds.\")\n",
    "print(\"The size of the result object based on NumPy arrays is\",sys.getsizeof(result),\"bytes.\\n\")\n",
    "\n",
    "del(nparray1, nparray2, result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition to demonstrating the substantial performance gains offered by NumPy, the previous code block also illsutrates some of the subtle differences of working with Python lists and NumPy arrays, and a method for checking the computation time that is required to execute a portion of code. Specifically:\n",
    "\n",
    "- The `time.time()` function, from the `time` module, returns the current system time. Saving the value of the current time in a variable `start` and then computing the difference `time.time() - start` returns the seconds elapsing between the two calls to `time.time()` in seconds. Multiplying by 1000 converts the elapsed time to milliseconds.\n",
    "\n",
    "\n",
    "- When working with Python lists, the `range()` function returns a sequence of integers starting at zero and ending at the argument passed to `range()`. In our example, we pass a variable `SIZE` to the `range()` function. Thus, the sequence stored in the list is 0, 1, ..., `SIZE`-2, `SIZE`-1.\n",
    "\n",
    "\n",
    "- When working with NumPy arrays, the `np.range()` function returns a sequence of integers starting at zero and ending at the argument passed to `np.arange()`. In our example, we pass a variable `SIZE` to the `np.arange()` function. Thus, the sequence stored in the NumPy array is 0, 1, ..., `SIZE`-2, `SIZE`-1.\n",
    "\n",
    "\n",
    "- The `sys.getsizeof()` function, from the `sys` library, returns the size of an object in bytes.\n",
    "\n",
    "\n",
    "- When working with Python lists, the `zip()` function essentially combines two or more list objects and allows element-wise operations to be performed.\n",
    "\n",
    "\n",
    "- When working with NumPy arrays, there is no need to *zip* arrays. Instead, element-wise operations are performed using standard mathematical operators.\n",
    "\n",
    "[Back to Table of Contents](#Table_of_Contents)<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Searching for Relevant NumPy Methods\n",
    "\n",
    "A final interesting thing to note is that the NumPy package includes a method to search for other methods that may be used to accomplish a desired task. This is extremely helpful given the large size of the package. As an example, suppose that we are interested in methods that work with the *standard deviation* for a set of numbers. The following code block shows how NumPy's `lookfor()` method allows us to search for such methods.\n",
    "\n",
    "[Back to Table of Contents](#Table_of_Contents)<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### NumPy Arrays\n",
    "<a id=\"NumPy_Arrays\"> </a>\n",
    "\n",
    "The key data structure used by NumPy is the multi-dimensional array or `ndarray`. The following code blocks demonstrate. several methods for creating numpy arrays.\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "    <b>NumPy's <i>random</i> module:</b> The final two examples utilize random number generators from NumPy's <i>random</i> module. The methods available in this module can be accessed via the prefix <i>np.random.</i> (this assumes NumPy was imported as np).\n",
    "</div>\n",
    "\n",
    "[Back to Table of Contents](#Table_of_Contents)<br>\n",
    "\n",
    "<b>Create a one dimensional (1-d) array with a list<b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Create a two-dimensional (2-d) array with a list.  Use a variable called my_list to store thbis list in. <b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Create a two-dimensional array of size 4 x 5 (4 rows and 5 columns) that is filled with all zeros. <b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b> Create a two-dimensional array that is 5 x 3 matrix filled with values randomly drawn from a continuous uniform distribution that ranges from 0 to 100.<b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b> Create a two-dimensional array that is 5 x 3 matrix filled with values randomly drawn from a continuous uniform distribution that ranges from 0 to 10 in integer form.<b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Create a two-dimensional 3 x 3 matrix filled with values randomly drawn from a normal distribution with a mean of 10.0 and standard deviation of 2.0<b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following code blocks demonstrate several attributes of NumPy arrays that we can use to gain information regarding their structure.\n",
    "\n",
    "<b>Create a two-dimensional 3 x 5 matrix filled with values randomly drawn from a normal distribution with a mean of 10.0 and standard deviation of 2.0 and store the array in a variable named my_array.  We will use this variable to sho the different attributes NumPy has available.<b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Use the shape attribute to get the shape of the array<b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Use the size attribute to get the size of the array.  (aka, how many entries are there)<b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Use the dtype attribute to determine the data type for my_array<b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Basic Computations with NumPy Arrays\n",
    "<a id=\"Array_Computation\"> </a>\n",
    "\n",
    "NumPy is able to achieve fast computation times through the use of *vectorized* operations. Essentially, instead of performing operations on NumPy array elements one at a time, NumPy operations can be performed on entire arrays or slices of arrays. The following code block shows some examples.\n",
    "\n",
    "[Back to Table of Contents](#Table_of_Contents)<br>\n",
    "\n",
    "<b>Create two two-dimensional 3 x 5 arrays, the first with values randomly drawn from a uniform distribution with low and high values of 0.0 and 1.0, respectively, and the second with values drawn from discrete uniform distribution ranging from 1 to 10.<b>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Add the arrays together<b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Subtract array1 from array2<b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Add 1 to all elements of array1<b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Compute 20*(array1^2)-array2<b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Compute the absolute value for 20*(array1^2)-array2<b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b> Delete the variables array1 and array2 from the computer. <b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Aggregations\n",
    "<a id=\"Array_Aggregations\"> </a>\n",
    "\n",
    "In addition to performing computations on each element of an array, NumPy also allows users to perform aggregations accross all elements and to perform computations along a particular array *axis*. Before continuing, it is important to understand exactly what is meant by an *array axis*.\n",
    "\n",
    "In NumPy, *axes* are defined for arrays with more than one dimension. For example, a 2-dimensional array has two axes: the first running vertically downwards across rows (axis 0), and the second running horizontally across columns (axis 1). Another way to think about this is if I have an $mxn$ array, where $m$ is the number of rows and $n$ the number of columns, the first axis, axis 0, runs across the $m$ rows and the second axis, axis 1, runs across the $n$ columns. \n",
    "\n",
    "Using this concept of array axes, we can do computations across rows and columns of arrays. The following code blocks provide examples.\n",
    "\n",
    "NumPy provides many other aggregation functions, but we won't discuss them in detail here.\n",
    "\n",
    "The following table provides a list of useful aggregation functions available in NumPy:\n",
    "\n",
    "|Function Name      | Description                                   |\n",
    "|-------------------|-----------------------------------------------|\n",
    "| ``np.sum``        | Compute sum of elements                       |\n",
    "| ``np.prod``       | Compute product of elements                   |\n",
    "| ``np.mean``       | Compute mean of elements                      |\n",
    "| ``np.std``        | Compute standard deviation                    |\n",
    "| ``np.var``        | Compute variance                              |\n",
    "| ``np.min``        | Find minimum value                            |\n",
    "| ``np.max``        | Find maximum value                            |\n",
    "| ``np.argmin``     | Find index of minimum value                   |\n",
    "| ``np.argmax``     | Find index of maximum value                   |\n",
    "| ``np.median``     | Compute median of elements                    |\n",
    "| ``np.percentile`` | Compute rank-based statistics of elements     |\n",
    "| ``np.any``        | Evaluate whether any elements are true        |\n",
    "| ``np.all``        | Evaluate whether all elements are true        |\n",
    "\n",
    "[Back to Table of Contents](#Table_of_Contents)<br>\n",
    "\n",
    "<b>Create a two-dimensional 3 x 5 array with values drawn from discrete uniform distribution ranging from 1 to 10 in whole numbers.<b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Sum all values in my_array1<B>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Sum the values in each row.<Br>\n",
    "Note that we need to sum the values across each column, or axis 1.<b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Sum the values in each column.<Br>\n",
    "Note that we need to sum the values across each row, or axis 1.<b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Find the maximum value in each row<B>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}