D03 神經網路101_MNIST手寫辨識.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. 神經網路\n",
    "\n",
    "我們終於要開始做生命中第一個神經網路..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. 讀入我們的 MNIST"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 由 TensorFlow 讀入 MNIST"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "這次是快速版本。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras.datasets import mnist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 來看看資料內容"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "n = 9487"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = x_train[n]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(28, 28)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "正確答案: 1\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAMVUlEQVR4nO3df+hddR3H8ddLt8CtlK191WHSKhQcSSsuI1DCiIUTcfZH0cBYKnz7Y4piSFJC/qEgkUVCS1aOptYiKHF/SDVGIEMYXsfa5kb5g5XTsX2Hui0YlO7dH99jfLd977l395xzz93ezwdczr3nfe89b+6+r517z+fc+3FECMD574K2GwAwGoQdSIKwA0kQdiAJwg4kMWeUG1u0aFEsWbJklJsEUtm/f7+OHDni2WqVwm77Rkk/k3ShpF9FxKNl91+yZIm63W6VTQIo0el0etaGfhtv+0JJP5e0UtJSSattLx32+QA0q8pn9uWSXouINyLiP5J+J2lVPW0BqFuVsF8h6c0Ztw8U605he9J213Z3amqqwuYAVFEl7LMdBDjj3NuIWB8RnYjoTExMVNgcgCqqhP2ApCtn3P6EpLertQOgKVXC/pKkq2x/yvZHJH1T0uZ62gJQt6GH3iLifdt3SfqzpofeNkTEK7V1BqBWlcbZI+J5Sc/X1AuABnG6LJAEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJjHTKZozewoULS+tHjx4tra9du7a0/vjjj591T2gHe3YgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIJx9vPAe++917MWEaWPtV1aX7duXWmdcfZzR6Ww294v6bikDyS9HxGdOpoCUL869uxfjogjNTwPgAbxmR1IomrYQ9JfbL9se3K2O9ietN213Z2amqq4OQDDqhr26yLiC5JWSlpr+0un3yEi1kdEJyI6ExMTFTcHYFiVwh4RbxfLw5KelbS8jqYA1G/osNueb/tjH16X9FVJe+pqDEC9qhyNv0zSs8U47RxJv42IP9XSFU7Rb6z8iSee6Fk7fvx43e2cYsuWLaX1FStWNLp9DG7osEfEG5I+V2MvABrE0BuQBGEHkiDsQBKEHUiCsANJ8BXXc8CxY8dK6w8++OCIOjnTtm3bSusMvY0P9uxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kATj7OeAuXPnltaXLl3as7Z3796628E5ij07kARhB5Ig7EAShB1IgrADSRB2IAnCDiTBOPs5YN68eaX122+/vWft/vvvr7udU2zatKm0ft999/WsXXLJJXW3gxLs2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcbZUcnrr79eWj9x4kTPGuPso9V3z257g+3DtvfMWLfQ9hbbrxbLBc22CaCqQd7G/1rSjaete0DS1oi4StLW4jaAMdY37BHxgqR3Tlu9StLG4vpGSbfW3BeAmg17gO6yiDgoScXy0l53tD1pu2u7OzU1NeTmAFTV+NH4iFgfEZ2I6ExMTDS9OQA9DBv2Q7YXS1KxPFxfSwCaMGzYN0taU1xfI+m5etoB0JS+4+y2N0m6QdIi2wck/VDSo5J+b/tOSf+S9PUmm0S5u+++u2ft4YcfLn3s0aNH624HY6pv2CNidY/SV2ruBUCDOF0WSIKwA0kQdiAJwg4kQdiBJPiK63lgzpze/4y2R9gJxhl7diAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiC77Of526++ebS+tNPP13p+U+ePFla3759e8/aqlWrKm0bZ4c9O5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kwTj7ee6OO+4orT/zzDOVnv+CC8r3F4899ljP2ooVK0ofO2/evKF6wuz67tltb7B92PaeGesesv2W7Z3F5aZm2wRQ1SBv438t6cZZ1v80IpYVl+frbQtA3fqGPSJekPTOCHoB0KAqB+jusr2reJu/oNedbE/a7truTk1NVdgcgCqGDfsvJH1G0jJJByX1PAoTEesjohMRnYmJiSE3B6CqocIeEYci4oOIOCnpl5KW19sWgLoNFXbbi2fc/JqkPb3uC2A89B1nt71J0g2SFtk+IOmHkm6wvUxSSNov6TsN9ohz2IsvvtiztmPHjtLHXn/99XW3k1rfsEfE6llWP9lALwAaxOmyQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kwU9Jn+cuv/zy0nq/Xw9q8qfEJicnS+t79+5tbNsZsWcHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQYZz/PXX311aX1lStXltafeuqpOts5BdOBjRZ7diAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgnH25B555JHSepPj7CdOnCit7969u7R+7bXX1tnOea/vnt32lbb/anuf7Vds31OsX2h7i+1Xi+WC5tsFMKxB3sa/L+m7EXGNpC9KWmt7qaQHJG2NiKskbS1uAxhTfcMeEQcjYkdx/bikfZKukLRK0sbibhsl3dpUkwCqO6sDdLaXSPq8pO2SLouIg9L0fwiSLu3xmEnbXdtdzoUG2jNw2G1/VNIfJN0bEccGfVxErI+ITkR0+v24IYDmDBR223M1HfTfRMQfi9WHbC8u6oslHW6mRQB16Dv0ZtuSnpS0LyJ+MqO0WdIaSY8Wy+ca6RCNmj9/fmm93/DWrl27ht72RRddVFq/5pprhn5unGmQcfbrJH1L0m7bO4t139d0yH9v+05J/5L09WZaBFCHvmGPiG2S3KP8lXrbAdAUTpcFkiDsQBKEHUiCsANJEHYgCb7imtzFF19cWr/llltK61XG2fuZM4c/zzqxZweSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJBjIRKnbbruttP7uu++W1tetW1dnO6iAPTuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJOGIGNnGOp1OdLvdkW0PyKbT6ajb7c76a9Ds2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgib5ht32l7b/a3mf7Fdv3FOsfsv2W7Z3F5abm2wUwrEF+vOJ9Sd+NiB22PybpZdtbitpPI+LHzbUHoC6DzM9+UNLB4vpx2/skXdF0YwDqdVaf2W0vkfR5SduLVXfZ3mV7g+0FPR4zabtruzs1NVWpWQDDGzjstj8q6Q+S7o2IY5J+IekzkpZpes//2GyPi4j1EdGJiM7ExEQNLQMYxkBhtz1X00H/TUT8UZIi4lBEfBARJyX9UtLy5toEUNUgR+Mt6UlJ+yLiJzPWL55xt69J2lN/ewDqMsjR+OskfUvSbts7i3Xfl7Ta9jJJIWm/pO800iGAWgxyNH6bpNm+H/t8/e0AaApn0AFJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5IY6ZTNtqck/XPGqkWSjoysgbMzrr2Na18SvQ2rzt4+GRGz/v7bSMN+xsbtbkR0WmugxLj2Nq59SfQ2rFH1xtt4IAnCDiTRdtjXt7z9MuPa27j2JdHbsEbSW6uf2QGMTtt7dgAjQtiBJFoJu+0bbf/d9mu2H2ijh15s77e9u5iGuttyLxtsH7a9Z8a6hba32H61WM46x15LvY3FNN4l04y3+tq1Pf35yD+z275Q0j8krZB0QNJLklZHxN6RNtKD7f2SOhHR+gkYtr8k6d+SnoqIzxbrfiTpnYh4tPiPckFEfG9MentI0r/bnsa7mK1o8cxpxiXdKunbavG1K+nrGxrB69bGnn25pNci4o2I+I+k30la1UIfYy8iXpD0zmmrV0naWFzfqOk/lpHr0dtYiIiDEbGjuH5c0ofTjLf62pX0NRJthP0KSW/OuH1A4zXfe0j6i+2XbU+23cwsLouIg9L0H4+kS1vu53R9p/EepdOmGR+b126Y6c+raiPss00lNU7jf9dFxBckrZS0tni7isEMNI33qMwyzfhYGHb686raCPsBSVfOuP0JSW+30MesIuLtYnlY0rMav6moD304g26xPNxyP/83TtN4zzbNuMbgtWtz+vM2wv6SpKtsf8r2RyR9U9LmFvo4g+35xYET2Z4v6asav6moN0taU1xfI+m5Fns5xbhM491rmnG1/Nq1Pv15RIz8IukmTR+Rf13SD9rooUdfn5b0t+LyStu9Sdqk6bd1/9X0O6I7JX1c0lZJrxbLhWPU29OSdkvapelgLW6pt+s1/dFwl6SdxeWmtl+7kr5G8rpxuiyQBGfQAUkQdiAJwg4kQdiBJAg7kARhB5Ig7EAS/wMNkrri4cnX+QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(X, cmap='Greys')\n",
    "print('正確答案:', y_train[n])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3 輸入資料整理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train = x_train.reshape(60000, 28*28)\n",
    "x_test = x_test.reshape(10000, 28*28)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3 輸出資料整理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train = tf.keras.utils.to_categorical(y_train,10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_test = tf.keras.utils.to_categorical(y_test,10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 打造第一個神經網路\n",
    "\n",
    "這裡我們用的是 Keras 2!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Dense\n",
    "from tensorflow.keras.optimizers import SGD"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = Sequential()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(Dense(250, input_dim=784, activation='relu'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(Dense(250, activation='relu'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.add(Dense(10, activation='softmax'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.4 Compile"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(loss='mse', optimizer=SGD(lr=0.1), metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.5 學習!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples\n",
      "Epoch 1/20\n",
      "60000/60000 [==============================] - 3s 52us/sample - loss: 0.1310 - accuracy: 0.3428\n",
      "Epoch 2/20\n",
      "60000/60000 [==============================] - 2s 25us/sample - loss: 0.1070 - accuracy: 0.4635\n",
      "Epoch 3/20\n",
      "60000/60000 [==============================] - 2s 25us/sample - loss: 0.0942 - accuracy: 0.5278\n",
      "Epoch 4/20\n",
      "60000/60000 [==============================] - 2s 25us/sample - loss: 0.0899 - accuracy: 0.5496\n",
      "Epoch 5/20\n",
      "60000/60000 [==============================] - 2s 28us/sample - loss: 0.0892 - accuracy: 0.5529\n",
      "Epoch 6/20\n",
      "60000/60000 [==============================] - 1s 23us/sample - loss: 0.0884 - accuracy: 0.5573\n",
      "Epoch 7/20\n",
      "60000/60000 [==============================] - 1s 21us/sample - loss: 0.0873 - accuracy: 0.5627\n",
      "Epoch 8/20\n",
      "60000/60000 [==============================] - 1s 21us/sample - loss: 0.0873 - accuracy: 0.5626\n",
      "Epoch 9/20\n",
      "60000/60000 [==============================] - 1s 21us/sample - loss: 0.0886 - accuracy: 0.5563\n",
      "Epoch 10/20\n",
      "60000/60000 [==============================] - 1s 21us/sample - loss: 0.0869 - accuracy: 0.5647\n",
      "Epoch 11/20\n",
      "60000/60000 [==============================] - 1s 21us/sample - loss: 0.0874 - accuracy: 0.5623\n",
      "Epoch 12/20\n",
      "60000/60000 [==============================] - 1s 21us/sample - loss: 0.0819 - accuracy: 0.5896\n",
      "Epoch 13/20\n",
      "60000/60000 [==============================] - 1s 21us/sample - loss: 0.0748 - accuracy: 0.6249\n",
      "Epoch 14/20\n",
      "60000/60000 [==============================] - 1s 21us/sample - loss: 0.0720 - accuracy: 0.6392\n",
      "Epoch 15/20\n",
      "60000/60000 [==============================] - 1s 21us/sample - loss: 0.0699 - accuracy: 0.6495\n",
      "Epoch 16/20\n",
      "60000/60000 [==============================] - 1s 22us/sample - loss: 0.0705 - accuracy: 0.6468\n",
      "Epoch 17/20\n",
      "60000/60000 [==============================] - 1s 25us/sample - loss: 0.0689 - accuracy: 0.6547\n",
      "Epoch 18/20\n",
      "60000/60000 [==============================] - 1s 24us/sample - loss: 0.0585 - accuracy: 0.7067\n",
      "Epoch 19/20\n",
      "60000/60000 [==============================] - 1s 21us/sample - loss: 0.0556 - accuracy: 0.7212\n",
      "Epoch 20/20\n",
      "60000/60000 [==============================] - 1s 21us/sample - loss: 0.0533 - accuracy: 0.7329\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x293f290d508>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.fit(x_train, y_train, batch_size=100, epochs=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. 試用我們的結果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "from ipywidgets import interact_manual"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = model.predict_classes(x_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def 辨識(測試編號):\n",
    "    plt.imshow(x_test[測試編號].reshape(28,28), cmap=\"Greys\")\n",
    "    print(\"神經網路判斷為:\", result[測試編號])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3b14cc1ed11c4dd0855e6f6f34b97e64",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "interactive(children=(IntSlider(value=4999, description='測試編號', max=9999), Button(description='Run Interact', …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "interact_manual(辨識, 測試編號 = (0, 9999));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 訓練好的神經網路存起來!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_json = model.to_json()\n",
    "open('handwriting_model_architecture.json', 'w').write(model_json)\n",
    "model.save_weights('handwriting_model_weights.h5')"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}