getting_started.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting Started\n",
    "\n",
    "###1. Install python and the key packages\n",
    " - The easiest way to do this is by downloading [Anaconda](https://store.continuum.io/cshop/anaconda/) (or Enthought, but I prefer Anaconda): \n",
    " - Many key packages will be already included in Anaconda (you don't need to do anything beyond the above), but fyi they include:\n",
    "    - [python](https://www.python.org/), the language\n",
    "    - [ipython](http://ipython.org/), the UI and the notebook that makes everything easier\n",
    "    - [numpy](http://www.numpy.org/), the numerical package for using python (upon which the the following are built)\n",
    "    - [scipy](http://www.scipy.org/), scientific computing atop numpy (think Matlab or R)\n",
    "    - [scikit-learn](http://scikit-learn.org/stable/), a great library of machine learning tools \n",
    "    - [statsmodels](http://statsmodels.sourceforge.net/), a library of statistical modeling tools \n",
    "    - [matplotlib](http://matplotlib.org/), the 2d plotting library\n",
    "    - [pandas](http://pandas.pydata.org/), data analysis tools and data structures (here's a [tour](http://nbviewer.ipython.org/urls/gist.github.com/wesm/4757075/raw/a72d3450ad4924d0e74fb57c9f62d1d895ea4574/PandasTour.ipynb))\n",
    "    - [numba](http://numba.pydata.org/), high-performance code option to allow inline compilation of python code (advanced)\n",
    "\n",
    " - A few advanced packages that you may want to use in your project (not included in anaconda, nor needed for the class at present):\n",
    "    - [xkcd style](http://jakevdp.github.io/blog/2013/07/10/XKCD-plots-in-matplotlib/) (for fun)\n",
    "    - [autograd](https://github.com/HIPS/autograd), automatic differentiation of numpy code\n",
    "    - [theano](http://deeplearning.net/software/theano/), automatic differentiation and other optimizations\n",
    "    - [spearmint](https://github.com/HIPS/Spearmint), bayesian optimization\n",
    " - Some other, possibly useful links:\n",
    "    - [deep learning](http://deeplearning.net/software_links/), many great links to other python and non-python packages for machine learning and deep learning.\n",
    "    - [stan](http://mc-stan.org/), a probabilistic programming language, including this [python interface](http://mc-stan.org/pystan.html).  Note that there are many other probabilistic programming languages.\n",
    "    - [databricks/spark](https://databricks.com/spark/about), for large-scale distributed computing, including particularly atop [AWS](http://aws.amazon.com/free/)\n",
    "\n",
    "###2. Get accustomed to doing machine learning with this python stack (not necessary, but helpful)\n",
    " - Basic python tutorials are [here](https://docs.python.org/2/tutorial/introduction.html) and [here](http://www.codecademy.com/en/tracks/python)\n",
    " - Machine learning in python tutorial [here](http://scikit-learn.org/stable/tutorial/) \n",
    " - Plotting tutorial [here](http://matplotlib.org/users/pyplot_tutorial.html)\n",
    " - Handling data with pandas tutorial [here](http://pandas.pydata.org/pandas-docs/stable/10min.html#min), or the lengthier [tour](http://nbviewer.ipython.org/urls/gist.github.com/wesm/4757075/raw/a72d3450ad4924d0e74fb57c9f62d1d895ea4574/PandasTour.ipynb)\n",
    " \n",
    "    \n",
    "###3. Get tooled up on git, which is how everything will be submitted and available in this class\n",
    " - Get a [github](https://github.com/) account (free!)\n",
    " - I prefer the command line, but you may want to use [GitHub for mac](https://mac.github.com/) or similar.\n",
    " - Here are some [tutorials](https://help.github.com/articles/good-resources-for-learning-git-and-github/)\n",
    " - Here is a great [cheat sheet](https://education.github.com/git-cheat-sheet-education.pdf)\n",
    "    \n",
    "###4. Install and test the Gaussian Process python package GPy \n",
    " - Install [GPy](http://sheffieldml.github.io/GPy/), a nice python package for Gaussian Processes.  This will be the key interface used throughout the course.\n",
    "    - if your installation of anaconda was successful, you should be able to run\n",
    "       - ```pip install GPy``` (from the command line, with which I will assume basic familiarity)\n",
    "       - documentation for GPy is available [here](https://gpy.readthedocs.org/en/latest/)\n",
    "       - you can test GPy with ```import GPy``` followed by ```GPy.tests()``` in ipython.  I recommend doing this step in python or ipython itself rather than in a notebook, since there is a lot of printed output.\n",
    "       \n",
    "###5. Verify your basics with the following commands:\n",
    "  - Note, this is an extended and altered version of the [scikit-learn tutorial](http://scikit-learn.org/stable/tutorial/); see that link for full description)\n",
    "  - Also, the markdown syntax is very helpful; nice tutorials are [here](http://jupyter.cs.brynmawr.edu/hub/dblank/public/Jupyter%20Notebook%20Users%20Manual.ipynb) and [here](http://nbviewer.ipython.org/github/Carreau/ipython/blob/2.x/examples/Notebook/Markdown%20Cells.ipynb) and [here](http://www.notebooksapp.com/ios/markdown/). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Populating the interactive namespace from numpy and matplotlib\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "%pylab inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Interactive namespace is empty.\n"
     ]
    }
   ],
   "source": [
    "%whos\n",
    "# also %doctest_mode is helpful for cutting and pasting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import sklearn\n",
    "from sklearn import datasets\n",
    "iris = datasets.load_iris()\n",
    "digits = datasets.load_digits()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Variable   Type      Data/Info\n",
      "------------------------------\n",
      "datasets   module    <module 'sklearn.datasets<...>n/datasets/__init__.pyc'>\n",
      "digits     Bunch     {'images': array([[[  0.,<...>[0, 1, 2, ..., 8, 9, 8])}\n",
      "iris       Bunch     {'target_names': array(['<...>m)', 'petal width (cm)']}\n",
      "sklearn    module    <module 'sklearn' from '/<...>es/sklearn/__init__.pyc'>\n"
     ]
    }
   ],
   "source": [
    "%whos"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['images', 'data', 'target_names', 'DESCR', 'target']"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#note iris and digist are of type Bunch\n",
    "digits.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "numpy.ndarray"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(digits.data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1797, 64)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# the input values\n",
    "digits.data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1797, 8, 8)"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# the input values in their actual image shape\n",
    "digits.images.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1797,)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# the outputs/targets\n",
    "digits.target.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  0.,   0.,   5.,  13.,   9.,   1.,   0.,   0.],\n",
       "       [  0.,   0.,  13.,  15.,  10.,  15.,   5.,   0.],\n",
       "       [  0.,   3.,  15.,   2.,   0.,  11.,   8.,   0.],\n",
       "       [  0.,   4.,  12.,   0.,   0.,   8.,   8.,   0.],\n",
       "       [  0.,   5.,   8.,   0.,   0.,   9.,   8.,   0.],\n",
       "       [  0.,   4.,  11.,   0.,   1.,  12.,   7.,   0.],\n",
       "       [  0.,   2.,  14.,   5.,  10.,  12.,   0.,   0.],\n",
       "       [  0.,   0.,   6.,  13.,  10.,   0.,   0.,   0.]])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check out a digit; look, it's the image of a 0\n",
    "digits.images[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAPYAAAD7CAYAAABZjGkWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAADDFJREFUeJzt3V2MXWUVxvHnaQtpC0iLGKUwSSEBg9xAWxsDRUfDECSI\n",
       "N1IgMRhMvJKPaEKoXJByRe/AxHgBAkGpkGkRAkEEVEogRmRqK5WWAkNJWgTahAlGgQRkeTG7ZJgW\n",
       "Zs85+32ns/r/JZM532t1ep7Ze87Z512OCAHIZc5MNwCgewQbSIhgAwkRbCAhgg0kRLCBhOb1+wC2\n",
       "eb8MmEER4cmX9R3s5oGnfZ+1a9dq7dq1XZQvWm/Dhg091RseHtbq1aunfb/rr7++p3pjY2NavHjx\n",
       "tO83NDTUU72RkRGtWLGip/uuW7eup/usWbOmp3q9/Fxmy/PTPiDTktgVB1Ii2EBCMxbswcHB1PXO\n",
       "OOOMqvXmz59ftd6SJUuq1lu1alXVerP9+UmwC6kd7AULFlStR7AP7XrsigMJEWwgIYINJDRlsG1f\n",
       "YPtF2y/b7u1NVgBVfWawbc+V9AtJF0j6iqTLbZ9eozEAvZtqi71S0isR8VpEfCDpPknfLd8WgH5M\n",
       "FewTJe2ecH5PcxmAQ9hUweYDHsAsNNWHQF6XNDDh/IDGt9qfMPHg9cHBwepv7gOHi02bNmnTpk1T\n",
       "3m6qYI9IOtX2Ukn/knSppMsn36jmp2CAw9nkDedNN9100Nt9ZrAj4kPbV0l6TNJcSXdExI7u2gRQ\n",
       "wpSfx46IRyU9WqEXAB3hyDMgIYINJESwgYQINpAQwQYSIthAQgQbSIhgAwkRbCChTiaBZNbrZI5e\n",
       "7dq1q2q9sbGxqvUk6bjjjqtab3h4uGq9Sy65pGq9g2GLDSREsIGECDaQEMEGEiLYQEIEG0iIYAMJ\n",
       "EWwgIYINJNRmdtedtt+yva1GQwD612aLfZfGZ3cBmCWmDHZEPC2p/gHFAHrG39hAQp18uosRP0Ad\n",
       "XY34aYURP0AdbUf8sCsOJNTm7a57Jf1F0mm2d9u+snxbAPrRZnbXAdM1ARza2BUHEiLYQEIEG0iI\n",
       "YAMJEWwgIYINJESwgYQINpAQwQYSmnWzuzZv3ly1Xu1ZWqOjo1XrnXLKKVXrSdLQ0FDVerWfM8zu\n",
       "AlAEwQYSIthAQgQbSIhgAwkRbCAhgg0kRLCBhAg2kFCbxQwHbD9p+wXb/7R9TY3GAPSuzSGlH0j6\n",
       "SURstX20pM22n4iIHYV7A9CjNrO73oyIrc3p/0jaIWlJ6cYA9G5af2PbXirpLEnPlmgGQDdaf7qr\n",
       "2Q3fKOnaZsv9MWZ3AXV0OrvL9hGS7pd0T0Q8OPl6ZncBdXQ2u8u2Jd0haXtE3NpRfwAKavM39jmS\n",
       "vi/pm7a3NF8XFO4LQB/azO56RhzIAswqBBZIiGADCRFsICGCDSREsIGECDaQEMEGEiLYQEIEG0ho\n",
       "1s3uGhsbq1pv2bJlVevNxCyt2pYvXz7TLaTHFhtIiGADCRFsICGCDSREsIGECDaQEMEGEiLYQEIE\n",
       "G0iozSql820/a3ur7e22b67RGIDetVnM8H3b34yId23Pk/SM7VXNIocADkGtdsUj4t3m5JGS5kp6\n",
       "u1hHAPrWKti259jeKuktSU9GxPaybQHoR6tPd0XER5LOtH2spMdsD0bEpv3XM7sLqKPT2V37RcQ7\n",
       "th+RtELSx4/O7C6gji5ndx1ve1FzeoGkIUlbOukSQBFtttgnSLrb9hyN/yL4TUT8qWxbAPrR5u2u\n",
       "bZLqLiMCoC8ceQYkRLCBhAg2kBDBBhIi2EBCBBtIiGADCRFsICGCDSTE7K4pDA0NVa13OKj9f7h4\n",
       "8eKq9Q4FbLGBhAg2kBDBBhIi2EBCBBtIiGADCRFsICGCDSREsIGE2g4MmGt7i+2HSzcEoH9tt9jX\n",
       "StouKQr2AqAjbdYVP0nShZJ+JcnFOwLQtzZb7FskXSfpo8K9AOjIZ366y/ZFkvZGxBbbg592O2Z3\n",
       "AXV0NbvrbEkX275Q0nxJn7P964i4YuKNmN0F1NHJ7K6IuCEiBiLiZEmXSfrz5FADOPRM931sXhUH\n",
       "ZoHWK6hExFOSnirYC4COcOQZkBDBBhIi2EBCBBtIiGADCRFsICGCDSREsIGECDaQ0Kyb3VV7DtPm\n",
       "zZur1qut9hwtSRoZGalab/Xq1VXrHQrYYgMJEWwgIYINJESwgYQINpAQwQYSIthAQgQbSIhgAwm1\n",
       "OvLM9muS/i3pf5I+iIiVJZsC0J+2h5SGpMGIeLtkMwC6MZ1dceZ2AbNE22CHpD/aHrH9o5INAehf\n",
       "213xcyLiDdtfkPSE7Rcj4un9VzK7C6ijq9ldkqSIeKP5vs/2A5JWSjposAGU08nsLkmyvdD2Mc3p\n",
       "oySdL2lbJ10CKKLNFvuLkh6wvf/26yPi8aJdAejLlMGOiF2SzqzQC4COcOQZkBDBBhIi2EBCBBtI\n",
       "iGADCRFsICGCDSREsIGECDaQkCOivwewo9/HmI5XX321Wi1JWr58edV6t912W9V6GzZsqFpPkkZH\n",
       "R6vWyzx/zbYi4oC1EthiAwkRbCAhgg0kRLCBhAg2kBDBBhIi2EBCBBtIiGADCbVZpXSR7Y22d9je\n",
       "bvtrNRoD0Ls2q5T+XNLvI+J7tudJOqpwTwD69JnBtn2spHMj4geSFBEfSnqnRmMAejfVrvjJkvbZ\n",
       "vsv2323fbnthjcYA9G6qXfF5kpZJuioinrN9q6Q1km6ceCNmdwF1dDW7a4+kPRHxXHN+o8aD/QnM\n",
       "7gLq6GR2V0S8KWm37dOai86T9EI3LQIopc2r4ldLWm/7SEmjkq4s2xKAfrWZ3fUPSV+t0AuAjnDk\n",
       "GZAQwQYSIthAQgQbSIhgAwkRbCAhgg0kRLCBhAg2kNCsm91VW+1ZWuvWratab8WKFVXrSdLw8HD1\n",
       "mlkxuws4jBBsICGCDSREsIGECDaQEMEGEiLYQEIEG0iozYifL9veMuHrHdvX1GgOQG/arHm2U9JZ\n",
       "kmR7jqTXJT1QuC8AfZjurvh5kkYjYneJZgB0Y7rBvkzSb0s0AqA7rYPdrCv+HUkbyrUDoAttBgbs\n",
       "921JmyNi3+QrmN0F1NHV7K6JLpd078GuYHYXUEcns7v2s32Uxl84+10HvQEorNUWOyL+K+n4wr0A\n",
       "6AhHngEJEWwgIYINJESwgYQINpAQwQYSmrFgtzl6ZjbX27lzZ9V67733XtV6e/furVov+/Ol63oE\n",
       "u5CXXnqpar3333+/ar19+w44srio7M+XNMEGUA7BBhLqZHZXR70A6MHBZnf1HWwAhx52xYGECDaQ\n",
       "0IwE2/YFtl+0/bLt6wvXutP2W7a3lawzod6A7Sdtv2D7n6WXarY93/aztrfa3m775pL1mppzm6Wo\n",
       "Hy5dq6n3mu3nm5p/K1xrke2Ntnc0P8+vFaxVbmnviKj6JWmupFckLZV0hKStkk4vWO9cjS+fvK3S\n",
       "v+9Lks5sTh8taWfJf19TZ2HzfZ6kv0paVbjeTyWtl/RQpZ/pLknHVap1t6QfTvh5Hlup7hxJb0ga\n",
       "6OLxZmKLvVLSKxHxWkR8IOk+Sd8tVSwinpY0VurxD1LvzYjY2pz+j6QdkpYUrvluc/JIjf/ifLtU\n",
       "LdsnSbpQ0q8kHfBqbEHFa9k+VtK5EXGnJEXEhxHxTum6jU6X9p6JYJ8oaWLze5rL0rG9VON7C88W\n",
       "rjPH9lZJb0l6MiK2Fyx3i6TrJH1UsMZkIemPtkds/6hgnZMl7bN9l+2/277d9sKC9SbqdGnvmQj2\n",
       "YfH+mu2jJW2UdG2z5S4mIj6KiDMlnSTp67YHS9SxfZGkvRGxRXW31udExFkaXyn3x7bPLVRnnqRl\n",
       "kn4ZEcsk/VfSmkK1PlZiae+ZCPbrkgYmnB/Q+FY7DdtHSLpf0j0R8WCtus1u4yOSVhQqcbaki23v\n",
       "0viKtd+y/etCtT4WEW803/dpfLzUykKl9kjaExHPNec3ajzopX3q0t69molgj0g61fbS5jfVpZIe\n",
       "moE+irBtSXdI2h4Rt1aod7ztRc3pBZKGJG0pUSsiboiIgYg4WeO7jn+OiCtK1NrP9kLbxzSnj5J0\n",
       "vqQi73BExJuSdts+rbnoPEkvlKg1yacu7d2r6awr3omI+ND2VZIe0/gLPXdExI5S9WzfK+kbkj5v\n",
       "e7ekGyPirlL1JJ0j6fuSnre9P2A/i4g/FKp3gqS7m4GJcyT9JiL+VKjWZDX+rPqipAfGf19qnqT1\n",
       "EfF4wXpXS1rfbHRGJV1ZsNbEpb07fe2AQ0qBhDjyDEiIYAMJEWwgIYINJESwgYQINpAQwQYSIthA\n",
       "Qv8HNKm2kfINTK4AAAAASUVORK5CYII=\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x108294a90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# show it as an image; gray_r is reverse, so that min will be white, and max will be black.\n",
    "plt.imshow(digits.images[0], interpolation='nearest', cmap=plt.cm.gray_r)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
       "           metric_params=None, n_neighbors=3, p=2, weights='uniform')"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# kNN classifier; akin to http://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html\n",
    "from sklearn.neighbors import KNeighborsClassifier as knn\n",
    "knn3 = knn(n_neighbors=3)\n",
    "knn3.fit( digits.data[0:-1,] , digits.target[0:-1] ) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1796, 64)"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# to be clear, that indexing removes the last row from the 2d array\n",
    "digits.data[0:-1,].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[8]\n"
     ]
    }
   ],
   "source": [
    "# now let's predict that last one:\n",
    "print(knn3.predict(digits.data[-1,]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# and the truth\n",
    "digits.target[-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# great..."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}