notebooks/1_Linear_Image_Classifier.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "# A linear classifier for the MNIST handwritten digit dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Let's start by importing some packages we need."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "plt.rcParams['figure.figsize'] = (5.0, 4.0)\n",
    "plt.rcParams['image.cmap'] = 'Greys'\n",
    "\n",
    "import numpy as np\n",
    "np.set_printoptions(suppress=True)\n",
    "np.set_printoptions(precision=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Getting a feel for the data\n",
    "\n",
    "MNIST is a dataset that contains 70,000 labelled images of handwritten digits that look like the following.\n",
    "\n",
    "![MNIST Data Sample](images/mnist-sample.png \"MNIST Data Sample\")\n",
    "\n",
    "We're going to train a linear classifier on a part of this data set, and test it against another portion of the data set to see how well we did.\n",
    "\n",
    "The TensorFlow tutorial comes with a handy loader for this dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting MNIST_data/train-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/train-labels-idx1-ubyte.gz\n",
      "Extracting MNIST_data/t10k-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/t10k-labels-idx1-ubyte.gz\n"
     ]
    }
   ],
   "source": [
    "mnist = input_data.read_data_sets(\"MNIST_data/\", one_hot=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "The loader even handily splits the data set into three parts:\n",
    "\n",
    "* A training set (55000 examples) used to train the model\n",
    "* A validation set (5000 examples) used to optimise hyperparameters (not shown today)\n",
    "* A test set (10000 examples) used to gauge the accuracy of the trained model\n",
    "\n",
    "The images are greyscale and each 28 pixels wide by 28 pixels tall, and this is stored as an array of length 784.\n",
    "\n",
    "The labels are a *one hot* vector of length 10, meaning it is a vector of all zeros except at the location that corresponds to the label it's referring to. E.g. An image with a label `3` will be represented as `(0, 0, 0, 1, 0, 0, 0, 0, 0, 0)`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(55000, 784)\n",
      "(55000, 10)\n"
     ]
    }
   ],
   "source": [
    "print mnist.train.images.shape\n",
    "print mnist.train.labels.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(10000, 784)\n",
      "(10000, 10)\n"
     ]
    }
   ],
   "source": [
    "print mnist.test.images.shape\n",
    "print mnist.test.labels.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "We can get a better sense for one of these examples by visualising the image and looking at the label."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[ 0.  0.  0.  1.  0.  0.  0.  0.  0.  0.]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x100ad2a90>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADoJJREFUeJzt3W+IXfWdx/HPN7Z9MGkVNZNxsKMTS1gJoqlck4XKkLW2\n2LEY+0BtHoQsaiYPKjZYpKIPNogkItvGUaQwtUPHtSZdaMUY4m7d4B8KS/AqE43V3Yk6pQmTzARL\nasyDVPPtgzmWqc75nev9d27m+37BMPee7zlzv1z95Nx7fuecn7m7AMSzqOwGAJSD8ANBEX4gKMIP\nBEX4gaAIPxAU4QeCIvxAUIQfCOoL7XyxJUuWeH9/fztfEghlcnJSx44ds1rWbSj8ZnadpGFJZ0l6\n3N0fTK3f39+varXayEsCSKhUKjWvW/fHfjM7S9Jjkr4jaYWkdWa2ot6/B6C9GvnOv0rSQXd/191P\nSdopaW1z2gLQao2E/0JJf5rz/FC27B+Y2ZCZVc2sOjMz08DLAWimlh/td/cRd6+4e6W7u7vVLweg\nRo2E/7CkvjnPv5otA3AGaCT8r0habmbLzOxLkr4vaVdz2gLQanUP9bn7R2Z2h6T/1uxQ36i7v9m0\nzgC0VEPj/O6+R9KeJvUCoI04vRcIivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4g\nKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+\nICjCDwRF+IGgGpql18wmJX0g6WNJH7l7pRlNoX2OHz+erI+NjSXrmzdvTtbNLLfm7sltr7zyymT9\nscceS9ZXr16drEfXUPgz/+Lux5rwdwC0ER/7gaAaDb9L+p2ZvWpmQ81oCEB7NPqx/2p3P2xmSyU9\nb2Zvu/vLc1fI/lEYkqSLLrqowZcD0CwN7fnd/XD2e1rS05JWzbPOiLtX3L3S3d3dyMsBaKK6w29m\ni83sK588lvRtSQea1RiA1mrkY3+PpKezoZwvSHrK3f+rKV0BaLm6w+/u70q6oom9oE4nT57MrQ0P\nDye3ffTRR5P16enpZD01jl9LPWV8fDxZX79+fd3bd3V11dXTQsJQHxAU4QeCIvxAUIQfCIrwA0ER\nfiCoZlzVhxZ7/PHHk/WhofzLKoqG2oouqy3aftmyZcl6I6d0Hzp0KFmfmJhI1gcGBnJr1Wq1rp4W\nEvb8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4/xngKeeeipZT43FN3JJrVR8++yXXnopWW/k0tmi\ncfxLL700WS+6JDg69vxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBTj/B2g6PbYRdeep66pL7qevre3\nN1nfvn17sr5169Zk/e67786tnXPOOcltly9fnqyfPn06WV+0KH/ftmfPnuS2g4ODyfpCwJ4fCIrw\nA0ERfiAowg8ERfiBoAg/EBThB4IqHOc3s1FJ35U07e6XZcvOk/RrSf2SJiXd7O5/bl2bC9vSpUuT\n9XfeeSdZX7x4cW6t0amoi8bDt23blqxv2rQpt1Y0zr9v375kPTWOL6XvZbBmzZrkthHUsuf/paTr\nPrXsHkl73X25pL3ZcwBnkMLwu/vLkt7/1OK1ksayx2OSbmxyXwBarN7v/D3uPpU9PiKpp0n9AGiT\nhg/4+exkb7kTvpnZkJlVzaw6MzPT6MsBaJJ6w3/UzHolKfude2WKu4+4e8XdK93d3XW+HIBmqzf8\nuyRtyB5vkPRMc9oB0C6F4TezHZL+V9I/mdkhM7tN0oOSvmVmE5KuzZ4DOIMUjvO7+7qc0jeb3Aty\nlPl16fzzz0/Wr7jiimT97LPPzq3t3Lkzue1dd92VrM8ebsrX05N/HLrR8x8WAs7wA4Ii/EBQhB8I\nivADQRF+ICjCDwTFrbsXgNRU1kXTXBcN5aVuCy5J+/fvT9ZXrFiRWzty5Ehy26LpxS+44IJkveiS\n4OjY8wNBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIzzLwBjY2O5taJbaxddFls01l60fWosv5FLciXp\n/vvvT9b7+vqS9ejY8wNBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIzzL3BF4/Rlbn/DDTckt33kkUeS\ndcbxG8OeHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCKhznN7NRSd+VNO3ul2XLtkjaKGkmW+1ed9/T\nqiaRtmHDhtzae++9l9x2amoqWa9Wq8n6iRMnkvWUhx56KFlnHL+1atnz/1LSdfMs3+7uK7Mfgg+c\nYQrD7+4vS3q/Db0AaKNGvvPfYWavm9momZ3btI4AtEW94f+ZpK9JWilpStJP8lY0syEzq5pZdWZm\nJm81AG1WV/jd/ai7f+zupyX9XNKqxLoj7l5x90p3d3e9fQJosrrCb2a9c55+T9KB5rQDoF1qGerb\nIWmNpCVmdkjSv0laY2YrJbmkSUmbWtgjgBawonunN1OlUvGicWN0lqLjNPfdd1+yPjo6mlsbGBhI\nbrt79+5kvaurK1mPqFKpqFqt1nQTBs7wA4Ii/EBQhB8IivADQRF+ICjCDwTFrbtrdPLkydzaQh5y\nKjorc2RkJFn/8MMPc2s7duxIbvvss88m67fcckuyjjT2/EBQhB8IivADQRF+ICjCDwRF+IGgCD8Q\nFOP8mYmJiWR906b8WxZcfvnlyW0ffvjhunpaCLZs2ZJb27lzZ3LbAwfS94hhnL8x7PmBoAg/EBTh\nB4Ii/EBQhB8IivADQRF+IKgw4/yp6/Gl4jHjiy++OLcWeRz/1KlTyfq6detya+28bTw+iz0/EBTh\nB4Ii/EBQhB8IivADQRF+ICjCDwRVOM5vZn2SnpDUI8kljbj7sJmdJ+nXkvolTUq62d3/3LpWG/Pi\niy8m6/v370/Wr7/++iZ2c+aYnp5O1gcHB5P18fHx3JpZeibpovskoDG17Pk/kvQjd18h6Z8l/cDM\nVki6R9Jed18uaW/2HMAZojD87j7l7q9ljz+Q9JakCyWtlTSWrTYm6cZWNQmg+T7Xd34z65f0dUn7\nJPW4+1RWOqLZrwUAzhA1h9/MvizpN5I2u/tf5tZ89iTteU/UNrMhM6uaWXVmZqahZgE0T03hN7Mv\najb4v3L332aLj5pZb1bvlTTvkSF3H3H3irtXiiZ9BNA+heG32UOyv5D0lrv/dE5pl6QN2eMNkp5p\nfnsAWqWWS3q/IWm9pDfM7JNxm3slPSjpP83sNkl/lHRza1psjkqlkqyfPn06WX/uuedya9dee21y\n20suuSRZ7+vrS9aLHD9+PLeWGmqTpCeffDJZHx0dTdaLLstNDec98MADyW1vuummZB2NKQy/u/9e\nUt5/wW82tx0A7cIZfkBQhB8IivADQRF+ICjCDwRF+IGgwty6e+nSpcn6xo0bk/XUePc111yT3Lbo\n0tWBgYFkvcjbb7+dWyu6JLeRcfpaDA8P59ZuvfXWhv42GsOeHwiK8ANBEX4gKMIPBEX4gaAIPxAU\n4QeCCjPOX6Romu2DBw/m1l544YXktosWpf+NLbqteNFYe2qsvmjbrq6uZP2qq65K1rdt25asr169\nOllHedjzA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQjPNnisa7d+/enVsrGususnXr1mT99ttvT9aL\n7lWQcueddybrzLK0cLHnB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgrIb7tvdJekJSjySXNOLuw2a2\nRdJGSTPZqve6+57U36pUKl6tVhtuGsD8KpWKqtVqTZMt1HKSz0eSfuTur5nZVyS9ambPZ7Xt7v7v\n9TYKoDyF4Xf3KUlT2eMPzOwtSRe2ujEArfW5vvObWb+kr0valy26w8xeN7NRMzs3Z5shM6uaWXVm\nZma+VQCUoObwm9mXJf1G0mZ3/4ukn0n6mqSVmv1k8JP5tnP3EXevuHuF88SBzlFT+M3si5oN/q/c\n/beS5O5H3f1jdz8t6eeSVrWuTQDNVhh+m7396y8kveXuP52zvHfOat+TdKD57QFolVqO9n9D0npJ\nb5jZeLbsXknrzGylZof/JiVtakmHAFqilqP9v5c037hhckwfQGfjDD8gKMIPBEX4gaAIPxAU4QeC\nIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhbfubuqLmc1I+uOcRUskHWtbA59Pp/bW\nqX1J9FavZvZ2sbvXdL+8tob/My9uVnX3SmkNJHRqb53al0Rv9SqrNz72A0ERfiCossM/UvLrp3Rq\nb53al0Rv9Sqlt1K/8wMoT9l7fgAlKSX8Znadmf2fmR00s3vK6CGPmU2a2RtmNm5mpU4pnE2DNm1m\nB+YsO8/Mnjeziez3vNOkldTbFjM7nL1342Y2WFJvfWb2gpn9wczeNLMfZstLfe8SfZXyvrX9Y7+Z\nnSXp/yV9S9IhSa9IWufuf2hrIznMbFJSxd1LHxM2swFJJyQ94e6XZcsekvS+uz+Y/cN5rrv/uEN6\n2yLpRNkzN2cTyvTOnVla0o2S/lUlvneJvm5WCe9bGXv+VZIOuvu77n5K0k5Ja0voo+O5+8uS3v/U\n4rWSxrLHY5r9n6ftcnrrCO4+5e6vZY8/kPTJzNKlvneJvkpRRvgvlPSnOc8PqbOm/HZJvzOzV81s\nqOxm5tGTTZsuSUck9ZTZzDwKZ25up0/NLN0x7109M143Gwf8Putqd79S0nck/SD7eNuRfPY7WycN\n19Q0c3O7zDOz9N+V+d7VO+N1s5UR/sOS+uY8/2q2rCO4++Hs97Skp9V5sw8f/WSS1Oz3dMn9/F0n\nzdw838zS6oD3rpNmvC4j/K9IWm5my8zsS5K+L2lXCX18hpktzg7EyMwWS/q2Om/24V2SNmSPN0h6\npsRe/kGnzNycN7O0Sn7vOm7Ga3dv+4+kQc0e8X9H0n1l9JDT1yWS9mc/b5bdm6Qdmv0Y+FfNHhu5\nTdL5kvZKmpD0P5LO66De/kPSG5Je12zQekvq7WrNfqR/XdJ49jNY9nuX6KuU940z/ICgOOAHBEX4\ngaAIPxAU4QeCIvxAUIQfCIrwA0ERfiCovwGyCoCFRwOAggAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10233a190>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "example_image = mnist.train.images[1]\n",
    "example_image_reshaped = example_image.reshape((28, 28)) # Can't render a line of 784 numbers\n",
    "example_label = mnist.train.labels[1]\n",
    "\n",
    "print example_label\n",
    "plt.imshow(example_image_reshaped)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Setting up a score function, loss function and optimisation algorithm\n",
    "\n",
    "Now that we have a better sense of the dataset we're working with, let's move onto the machine learning bits.\n",
    "\n",
    "First, we setup some placeholders to hold batches of this training data for when we learn our model. The reason why we work in batches is that it's easier on memory than holding the entire set. And it's this notion of working with (random) batches of input rather than the entire set that moves us from the realm of *Gradient Descent* that we saw earlier, to *Stochastic Gradient Descent* that we have here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "x = tf.placeholder(tf.float32, [None, 784])\n",
    "y_ = tf.placeholder(tf.float32, [None, 10])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "We define our linear model for the score function after introducing two of parameters, **W** and **b**.\n",
    "\n",
    "![Linear model](images/linear.png \"Linear model\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "W = tf.Variable(tf.zeros([784, 10]))\n",
    "b = tf.Variable(tf.zeros([10]))\n",
    "y = tf.nn.softmax(tf.matmul(x, W) + b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "We define our loss function to measure how poorly this model performs on images with known labels. We use the a specific form called the [cross entropy loss](https://jamesmccaffrey.wordpress.com/2013/11/05/why-you-should-use-cross-entropy-error-instead-of-classification-error-or-mean-squared-error-for-neural-network-classifier-training/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_*tf.log(y), reduction_indices=[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Using the magic of blackbox optimisation algorithms provided by TensorFlow, we can define a single step of the stochastic gradient descent optimiser (to improve our parameters for our score function and reduce the loss) in one line of code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Training the model\n",
    "\n",
    "The way TensorFlow works, we haven't really executed any of the code above in the classic sense. All we've done is defined what's called the computational graph.\n",
    "\n",
    "Now we go ahead and initialise a session to actually train the model and evaluate its performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "init = tf.global_variables_initializer()\n",
    "sess = tf.Session()\n",
    "sess.run(init)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "We train the model iteratively for 1000 steps, loading a batch of example images each time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true,
    "deletable": true,
    "editable": true
   },
   "outputs": [],
   "source": [
    "for i in range(1000):\n",
    "  batch_xs, batch_ys = mnist.train.next_batch(100)\n",
    "  sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Verifying the results\n",
    "\n",
    "At this point, our model is trained. And we can deterime in the *accuracy* by passing in all the test images and labels, figuring out our own labels, and averaging out the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9205\n"
     ]
    }
   ],
   "source": [
    "correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))\n",
    "accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n",
    "print(sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels}))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "Around 92%, that's pretty good!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": true,
    "editable": true
   },
   "source": [
    "## Extension: Using the trained model\n",
    "\n",
    "Here we try to actually use the model we learn to classify a single test example. It is also easy to [save and restore trained parameters](https://www.tensorflow.org/programmers_guide/variables) for later use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x10d7ea410>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADo5JREFUeJzt3X+sVPWZx/HPsxSMCv5a7iKx/NpK1hATqZkQkyVas0sj\npFEbiakmGzS41z+qbg1/iBiyRCMS3LYSY1C6kFJhKU1aAxrcRXEVSZbG8TeiuyK5DVwRLlLpJSZW\n5Nk/7rG5wJ3vDDNn5pzr834lN3fmPOfMeTLwuWdmvnPO19xdAOL5q6IbAFAMwg8ERfiBoAg/EBTh\nB4Ii/EBQhB8IivADQRF+IKhvdXJnY8eO9cmTJ3dyl0AoPT09Onz4sDWybkvhN7PrJK2QNELSv7v7\nstT6kydPVrVabWWXABIqlUrD6zb9st/MRkh6QtJsSdMk3WJm05p9PACd1cp7/hmS9rj7Xnf/s6Rf\nS7ohn7YAtFsr4b9E0r5B9/dny05iZt1mVjWzal9fXwu7A5Cntn/a7+6r3L3i7pWurq527w5Ag1oJ\nf6+kCYPufztbBmAYaCX8r0maamZTzGyUpB9J2pxPWwDaremhPnc/bmZ3SfovDQz1rXH393LrDEBb\ntTTO7+5bJG3JqRcAHcTXe4GgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxA\nUIQfCKqjl+5Gc44fP56sz5o1q2Zt+/btyW3nzZuXrK9ZsyZZx/DFkR8IivADQRF+ICjCDwRF+IGg\nCD8QFOEHgmKcvwTqjeM/+OCDyfqrr75as2aWnq35qquuStbxzcWRHwiK8ANBEX4gKMIPBEX4gaAI\nPxAU4QeCammc38x6JPVL+krScXev5NFUNBs2bEjWly5dmqzPnTu3Zu2RRx5JbjtlypRkHd9ceXzJ\n51p3P5zD4wDoIF72A0G1Gn6XtNXMXjez7jwaAtAZrb7sn+nuvWb2N5JeMLMP3P2ki8ZlfxS6JWni\nxIkt7g5AXlo68rt7b/b7kKRnJM0YYp1V7l5x90pXV1cruwOQo6bDb2bnmtmYr29L+r6kXXk1BqC9\nWnnZP07SM9kpo9+S9B/u/p+5dAWg7ZoOv7vvlXRFjr2E1dvb29L2c+bMqVljHB+1MNQHBEX4gaAI\nPxAU4QeCIvxAUIQfCIpLd5fA0aNHk/VRo0Yl69dee22e7SAIjvxAUIQfCIrwA0ERfiAowg8ERfiB\noAg/EBTj/B3Q39+frC9fvjxZnz17drI+YcKEM+4J4MgPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0Ex\nzt8Bjz76aNEtDEt79+5N1g8ePNj0Y0+dOjVZHzt2bNOPPVxw5AeCIvxAUIQfCIrwA0ERfiAowg8E\nRfiBoOqO85vZGkk/kHTI3S/Pll0kaaOkyZJ6JN3s7n9sX5vD2/r161va/t57782pk85bvHhxzVq9\n5+Xw4cPJ+ueff95UT5J0wQUXJOtLly5N1ru7u5ved1k0cuT/paTrTlm2UNI2d58qaVt2H8AwUjf8\n7r5d0pFTFt8gaW12e62kG3PuC0CbNfuef5y7H8hufyJpXE79AOiQlj/wc3eX5LXqZtZtZlUzq/b1\n9bW6OwA5aTb8B81svCRlvw/VWtHdV7l7xd0rXV1dTe4OQN6aDf9mSfOy2/MkbcqnHQCdUjf8ZrZB\n0v9I+jsz229m8yUtkzTLzD6U9I/ZfQDDiA28Ze+MSqXi1Wq1Y/vrlC+//DJZv/TSS5P1ESNGJOv1\nzmtvxYkTJ5L1ffv2JetXX311sr5///6m933xxRcn63PmzEnWX3755Zq1es/ppEmTkvW33347WT//\n/POT9XapVCqqVqvWyLp8ww8IivADQRF+ICjCDwRF+IGgCD8QFJfuzsGmTenvOPX29ibrixYtyrOd\nk9SbHvzpp59O1u+5556W9j9x4sSmH3v+/PnJ+nnnnddUT5J02223Jevr1q1L1o8cOfVct5MVNdR3\nJjjyA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQjPPnYOfOnS1tP23atJw6OV296cEffvjhZN0sfXbo\n3Llzk/XHHnusZq3eKbvtdNlllxW277LgyA8ERfiBoAg/EBThB4Ii/EBQhB8IivADQTHOn4N65+u3\nW2oq65UrV7b02Pfff3+yvmTJkmS93mXJy+qaa65J1lPXKRguOPIDQRF+ICjCDwRF+IGgCD8QFOEH\ngiL8QFB1x/nNbI2kH0g65O6XZ8uWSPpnSX3ZaovcfUu7miy7zz77LFmvNw16q9OkP/HEEzVr9a4v\nf/fddyfrDz30UFM9ld3Ro0eT9bPOOitZH67fXxiskSP/LyVdN8Tyn7v79OwnbPCB4apu+N19u6T0\n4QPAsNPKe/67zOwdM1tjZhfm1hGAjmg2/CslfUfSdEkHJP201opm1m1mVTOr9vX11VoNQIc1FX53\nP+juX7n7CUm/kDQjse4qd6+4e6Wrq6vZPgHkrKnwm9n4QXd/KGlXPu0A6JRGhvo2SPqepLFmtl/S\nv0r6nplNl+SSeiTd2cYeAbRB3fC7+y1DLF7dhl6GrXrXtm+1Xk/qegL1Hvvjjz9uad9l1t/fX7O2\nfPny5LZ33HFH3u2UDt/wA4Ii/EBQhB8IivADQRF+ICjCDwTFpbu/AVasWFGztmVL+oTL5557Lllf\nvTo9qnvrrbcm62effXay3k433XRTzdo555yT3HbhwoV5t1M6HPmBoAg/EBThB4Ii/EBQhB8IivAD\nQRF+ICjG+RuUOj30o48+6mAnp0uNpX/wwQfJba+//vpk/c4705dqePbZZ5P1jRs31qzVuzz2m2++\nmazfd999yforr7xSs/b4448nt50yZUqy/k3AkR8IivADQRF+ICjCDwRF+IGgCD8QFOEHgmKcv0Fj\nxoypWbvyyiuT2+7ZsydZf/7555P11HnpkjRy5MiatdGjRye3femll5L1emPtkyZNSta/+OKLmrUl\nS5Ykt613ee165+SnxvK7u7uT20bAkR8IivADQRF+ICjCDwRF+IGgCD8QFOEHgqo7zm9mEyT9StI4\nSS5plbuvMLOLJG2UNFlSj6Sb3f2P7Wu1vJ566qlkvaenJ1lft25dsv7pp58m64sXL65Z6+rqSm5b\nz44dO5L1Bx54IFnfunVrzZq7J7e94oorkvUnn3wyWZ8xY0ayHl0jR/7jkha4+zRJV0n6sZlNk7RQ\n0jZ3nyppW3YfwDBRN/zufsDd38hu90t6X9Ilkm6QtDZbba2kG9vVJID8ndF7fjObLOm7kn4vaZy7\nH8hKn2jgbQGAYaLh8JvZaEm/lfQTd//T4JoPvHkb8g2cmXWbWdXMqn19fS01CyA/DYXfzEZqIPjr\n3f132eKDZjY+q4+XdGiobd19lbtX3L3S6odPAPJTN/xmZpJWS3rf3X82qLRZ0rzs9jxJm/JvD0C7\nWL3hFjObKelVSe9KOpEtXqSB9/2/kTRR0h80MNR3JPVYlUrFq9Vqqz0PO8eOHUvWZ86cmazv2rUr\nz3ZO0sC/f9v2vWDBgmS93im/RU7/XVaVSkXVarWhf7S64/zuvkNSrQf7hzNpDEB58A0/ICjCDwRF\n+IGgCD8QFOEHgiL8QFBcursD6l0+e+fOncn6iy++mKzv3r27Zm3ZsmXJbetNc93qOP/tt99es8Y3\nPovFkR8IivADQRF+ICjCDwRF+IGgCD8QFOEHgqp7Pn+eop7PD3TKmZzPz5EfCIrwA0ERfiAowg8E\nRfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgqobfjObYGb/bWa7zew9M/uX\nbPkSM+s1s7eynzntbxdAXhqZtOO4pAXu/oaZjZH0upm9kNV+7u7/1r72ALRL3fC7+wFJB7Lb/Wb2\nvqRL2t0YgPY6o/f8ZjZZ0ncl/T5bdJeZvWNma8zswhrbdJtZ1cyqfX19LTULID8Nh9/MRkv6raSf\nuPufJK2U9B1J0zXwyuCnQ23n7qvcveLuFeZmA8qjofCb2UgNBH+9u/9Oktz9oLt/5e4nJP1C0oz2\ntQkgb4182m+SVkt6391/Nmj5+EGr/VDSrvzbA9AujXza//eS/knSu2b2VrZskaRbzGy6JJfUI+nO\ntnQIoC0a+bR/h6ShrgO+Jf92AHQK3/ADgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxA\nUIQfCIrwA0ERfiAowg8EZe7euZ2Z9Un6w6BFYyUd7lgDZ6asvZW1L4nempVnb5PcvaHr5XU0/Kft\n3Kzq7pXCGkgoa29l7Uuit2YV1Rsv+4GgCD8QVNHhX1Xw/lPK2ltZ+5LorVmF9Fboe34AxSn6yA+g\nIIWE38yuM7P/NbM9ZrawiB5qMbMeM3s3m3m4WnAva8zskJntGrTsIjN7wcw+zH4POU1aQb2VYubm\nxMzShT53ZZvxuuMv+81shKT/kzRL0n5Jr0m6xd13d7SRGsysR1LF3QsfEzazqyUdk/Qrd788W7Zc\n0hF3X5b94bzQ3e8rSW9LJB0reubmbEKZ8YNnlpZ0o6TbVOBzl+jrZhXwvBVx5J8haY+773X3P0v6\ntaQbCuij9Nx9u6Qjpyy+QdLa7PZaDfzn6bgavZWCux9w9zey2/2Svp5ZutDnLtFXIYoI/yWS9g26\nv1/lmvLbJW01s9fNrLvoZoYwLps2XZI+kTSuyGaGUHfm5k46ZWbp0jx3zcx4nTc+8DvdTHe/UtJs\nST/OXt6Wkg+8ZyvTcE1DMzd3yhAzS/9Fkc9dszNe562I8PdKmjDo/rezZaXg7r3Z70OSnlH5Zh8+\n+PUkqdnvQwX38xdlmrl5qJmlVYLnrkwzXhcR/tckTTWzKWY2StKPJG0uoI/TmNm52QcxMrNzJX1f\n5Zt9eLOkednteZI2FdjLScoyc3OtmaVV8HNXuhmv3b3jP5LmaOAT/48kPVBEDzX6+ltJb2c/7xXd\nm6QNGngZ+KUGPhuZL+mvJW2T9KGkFyVdVKLenpb0rqR3NBC08QX1NlMDL+nfkfRW9jOn6Ocu0Vch\nzxvf8AOC4gM/ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANB/T94roqfr9Nd0AAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10d4ae8d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "example_image = mnist.test.images[11]\n",
    "example_image_reshaped = example_image.reshape((28, 28)) # Can't render a line of 784 numbers\n",
    "plt.imshow(example_image_reshaped)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "deletable": true,
    "editable": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 0.    0.    0.04  0.    0.    0.    0.84  0.    0.11  0.  ]]\n"
     ]
    }
   ],
   "source": [
    "normalised_scores = sess.run(y, feed_dict={x: [example_image]})\n",
    "print normalised_scores"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}