chapter02_mathematical-building-blocks.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TD2VFLNa1qIr"
      },
      "source": [
        "이 노트북은 [케라스 창시자에게 배우는 딥러닝 2판](https://tensorflow.blog/kerasdl2/)의 예제 코드를 담고 있습니다.\n",
        "\n",
        "<table align=\"left\"><tr><td>\n",
        "<a href=\"https://colab.research.google.com/github/rickiepark/deep-learning-with-python-2nd/blob/master/chapter02_mathematical-building-blocks.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
        "</td></tr></table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mokNYTez1qIt"
      },
      "source": [
        "# 신경망의 수학적 구성 요소"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TEnmFpmg1qIu"
      },
      "source": [
        "## 신경망과의 첫 만남"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nlRV5eGp1qIu"
      },
      "source": [
        "**케라스에서 MNIST 데이터셋 적재하기**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:05.419691Z",
          "iopub.status.busy": "2022-05-21T00:57:05.419221Z",
          "iopub.status.idle": "2022-05-21T00:57:07.931588Z",
          "shell.execute_reply": "2022-05-21T00:57:07.930532Z"
        },
        "id": "Kbsv3tf21qIu",
        "outputId": "f37b537c-b43a-4d2f-a63d-8a1897a5a08d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz\n",
            "11490434/11490434 [==============================] - 1s 0us/step\n"
          ]
        }
      ],
      "source": [
        "from tensorflow.keras.datasets import mnist\n",
        "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:07.936250Z",
          "iopub.status.busy": "2022-05-21T00:57:07.935565Z",
          "iopub.status.idle": "2022-05-21T00:57:07.945910Z",
          "shell.execute_reply": "2022-05-21T00:57:07.945184Z"
        },
        "id": "GjsNOiRZ1qIw",
        "outputId": "c4d9d74d-be5e-4c80-c817-8ddec7913cef",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(60000, 28, 28)"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ],
      "source": [
        "train_images.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:07.949595Z",
          "iopub.status.busy": "2022-05-21T00:57:07.948880Z",
          "iopub.status.idle": "2022-05-21T00:57:07.953413Z",
          "shell.execute_reply": "2022-05-21T00:57:07.952780Z"
        },
        "id": "BMvJUobh1qIw",
        "outputId": "7e1b2f82-a6c1-43e1-8215-7c40622d6aee",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "60000"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ],
      "source": [
        "len(train_labels)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:07.956620Z",
          "iopub.status.busy": "2022-05-21T00:57:07.956158Z",
          "iopub.status.idle": "2022-05-21T00:57:07.961433Z",
          "shell.execute_reply": "2022-05-21T00:57:07.960670Z"
        },
        "id": "VhW9RYKv1qIw",
        "outputId": "3f349af9-fd0e-4bb0-db71-269e79768d4c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([5, 0, 4, ..., 5, 6, 8], dtype=uint8)"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ],
      "source": [
        "train_labels"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:07.964700Z",
          "iopub.status.busy": "2022-05-21T00:57:07.964447Z",
          "iopub.status.idle": "2022-05-21T00:57:07.969443Z",
          "shell.execute_reply": "2022-05-21T00:57:07.968645Z"
        },
        "id": "s-H4DSiI1qIx",
        "outputId": "1881f79e-79d2-4474-b981-faf9c9322e7d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(10000, 28, 28)"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ],
      "source": [
        "test_images.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:07.973895Z",
          "iopub.status.busy": "2022-05-21T00:57:07.973604Z",
          "iopub.status.idle": "2022-05-21T00:57:07.978539Z",
          "shell.execute_reply": "2022-05-21T00:57:07.977669Z"
        },
        "id": "arJVF8HY1qIx",
        "outputId": "83c1503d-b06e-419e-9740-1b3f40ea625b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "10000"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ],
      "source": [
        "len(test_labels)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:07.982841Z",
          "iopub.status.busy": "2022-05-21T00:57:07.982277Z",
          "iopub.status.idle": "2022-05-21T00:57:07.987700Z",
          "shell.execute_reply": "2022-05-21T00:57:07.986719Z"
        },
        "id": "9mGmW7a81qIy",
        "outputId": "0f06adb9-43dd-4781-f805-c0054642c28b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([7, 2, 1, ..., 4, 5, 6], dtype=uint8)"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ],
      "source": [
        "test_labels"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lkR5Kdxs1qIy"
      },
      "source": [
        "**신경망 구조**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:07.991164Z",
          "iopub.status.busy": "2022-05-21T00:57:07.990885Z",
          "iopub.status.idle": "2022-05-21T00:57:08.025097Z",
          "shell.execute_reply": "2022-05-21T00:57:08.023960Z"
        },
        "id": "nGjggIFO1qIy"
      },
      "outputs": [],
      "source": [
        "from tensorflow import keras\n",
        "from tensorflow.keras import layers\n",
        "model = keras.Sequential([\n",
        "    layers.Dense(512, activation=\"relu\"),\n",
        "    layers.Dense(10, activation=\"softmax\")\n",
        "])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ztkKhME61qIy"
      },
      "source": [
        "**컴파일 단계**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:08.029264Z",
          "iopub.status.busy": "2022-05-21T00:57:08.028955Z",
          "iopub.status.idle": "2022-05-21T00:57:08.043873Z",
          "shell.execute_reply": "2022-05-21T00:57:08.042917Z"
        },
        "id": "XAll1_qf1qIz"
      },
      "outputs": [],
      "source": [
        "model.compile(optimizer=\"rmsprop\",\n",
        "              loss=\"sparse_categorical_crossentropy\",\n",
        "              metrics=[\"accuracy\"])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IfXepSHu1qIz"
      },
      "source": [
        "**이미지 데이터 준비하기**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:08.047878Z",
          "iopub.status.busy": "2022-05-21T00:57:08.047587Z",
          "iopub.status.idle": "2022-05-21T00:57:08.461821Z",
          "shell.execute_reply": "2022-05-21T00:57:08.460699Z"
        },
        "id": "0VedXTKL1qIz"
      },
      "outputs": [],
      "source": [
        "train_images = train_images.reshape((60000, 28 * 28))\n",
        "train_images = train_images.astype(\"float32\") / 255\n",
        "test_images = test_images.reshape((10000, 28 * 28))\n",
        "test_images = test_images.astype(\"float32\") / 255"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r9cuYTKi1qIz"
      },
      "source": [
        "**모델 훈련**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:08.466297Z",
          "iopub.status.busy": "2022-05-21T00:57:08.465542Z",
          "iopub.status.idle": "2022-05-21T00:57:19.300649Z",
          "shell.execute_reply": "2022-05-21T00:57:19.299825Z"
        },
        "id": "LstMGeYK1qIz",
        "outputId": "abc75889-c316-425f-b0c2-9b61730857f5",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/5\n",
            "469/469 [==============================] - 6s 3ms/step - loss: 0.2667 - accuracy: 0.9231\n",
            "Epoch 2/5\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.1069 - accuracy: 0.9689\n",
            "Epoch 3/5\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 0.0697 - accuracy: 0.9797\n",
            "Epoch 4/5\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.0510 - accuracy: 0.9847\n",
            "Epoch 5/5\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 0.0378 - accuracy: 0.9886\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x78578b2d2740>"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ],
      "source": [
        "model.fit(train_images, train_labels, epochs=5, batch_size=128)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iQgihs241qI0"
      },
      "source": [
        "**모델을 사용해 예측 만들기**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:19.304176Z",
          "iopub.status.busy": "2022-05-21T00:57:19.303848Z",
          "iopub.status.idle": "2022-05-21T00:57:19.406311Z",
          "shell.execute_reply": "2022-05-21T00:57:19.405251Z"
        },
        "id": "qV8_w05L1qI0",
        "outputId": "b22b939a-46f1-4708-9203-3175a1e827e2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "1/1 [==============================] - 0s 66ms/step\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([7.9244224e-08, 2.8882177e-08, 4.1038176e-05, 3.3128036e-03,\n",
              "       8.2901352e-11, 6.6504566e-07, 3.2575769e-12, 9.9654561e-01,\n",
              "       1.4622490e-06, 9.8360535e-05], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ],
      "source": [
        "test_digits = test_images[0:10]\n",
        "predictions = model.predict(test_digits)\n",
        "predictions[0]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:19.410311Z",
          "iopub.status.busy": "2022-05-21T00:57:19.409575Z",
          "iopub.status.idle": "2022-05-21T00:57:19.414980Z",
          "shell.execute_reply": "2022-05-21T00:57:19.414116Z"
        },
        "id": "zc_2oAmH1qI0",
        "outputId": "b9653158-7279-452f-9652-a3e04c2729a9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "7"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ],
      "source": [
        "predictions[0].argmax()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:19.418674Z",
          "iopub.status.busy": "2022-05-21T00:57:19.418395Z",
          "iopub.status.idle": "2022-05-21T00:57:19.424303Z",
          "shell.execute_reply": "2022-05-21T00:57:19.423160Z"
        },
        "id": "G96ao1pL1qI0",
        "outputId": "0f5974cb-66e5-43d7-ea6b-fa7c8e75c7e0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.9965456"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ],
      "source": [
        "predictions[0][7]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:19.428684Z",
          "iopub.status.busy": "2022-05-21T00:57:19.428277Z",
          "iopub.status.idle": "2022-05-21T00:57:19.433154Z",
          "shell.execute_reply": "2022-05-21T00:57:19.432273Z"
        },
        "id": "CmsTCGMe1qI0",
        "outputId": "9fc780aa-88f8-4069-91dc-cdfb6b5bcff5",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "7"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ],
      "source": [
        "test_labels[0]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xz0O7yfW1qI1"
      },
      "source": [
        "**새로운 데이터에서 모델 평가하기**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:19.437738Z",
          "iopub.status.busy": "2022-05-21T00:57:19.437092Z",
          "iopub.status.idle": "2022-05-21T00:57:20.109805Z",
          "shell.execute_reply": "2022-05-21T00:57:20.109057Z"
        },
        "id": "5n06x08R1qI1",
        "outputId": "19118217-b55c-4846-8196-6ba20aa9a860",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "313/313 [==============================] - 1s 2ms/step - loss: 0.0696 - accuracy: 0.9780\n",
            "테스트 정확도: 0.9779999852180481\n"
          ]
        }
      ],
      "source": [
        "test_loss, test_acc = model.evaluate(test_images, test_labels)\n",
        "print(f\"테스트 정확도: {test_acc}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_sf8F4Lt1qI1"
      },
      "source": [
        "## 신경망을 위한 데이터 표현"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0rIygsuR1qI1"
      },
      "source": [
        "### 스칼라 (랭크-0 텐서)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:20.113826Z",
          "iopub.status.busy": "2022-05-21T00:57:20.113141Z",
          "iopub.status.idle": "2022-05-21T00:57:20.119204Z",
          "shell.execute_reply": "2022-05-21T00:57:20.118308Z"
        },
        "id": "zaKeHFt51qI1",
        "outputId": "df343355-01d5-489a-fce3-1fe810134557",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array(12)"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ],
      "source": [
        "import numpy as np\n",
        "x = np.array(12)\n",
        "x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:20.122738Z",
          "iopub.status.busy": "2022-05-21T00:57:20.122472Z",
          "iopub.status.idle": "2022-05-21T00:57:20.127154Z",
          "shell.execute_reply": "2022-05-21T00:57:20.126388Z"
        },
        "id": "L7qdT7rT1qI1",
        "outputId": "9a7b42cd-33f9-4f6b-852f-3e0f1d21fb5c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ],
      "source": [
        "x.ndim"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UBwl26_c1qI2"
      },
      "source": [
        "### 벡터 (랭크-1 텐서)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:20.131021Z",
          "iopub.status.busy": "2022-05-21T00:57:20.130223Z",
          "iopub.status.idle": "2022-05-21T00:57:20.135705Z",
          "shell.execute_reply": "2022-05-21T00:57:20.135002Z"
        },
        "id": "_FZ_X24k1qI2",
        "outputId": "ff3d727a-7e69-437b-de7b-696ad4cd3cf8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([12,  3,  6, 14,  7])"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ],
      "source": [
        "x = np.array([12, 3, 6, 14, 7])\n",
        "x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:20.139180Z",
          "iopub.status.busy": "2022-05-21T00:57:20.138695Z",
          "iopub.status.idle": "2022-05-21T00:57:20.143498Z",
          "shell.execute_reply": "2022-05-21T00:57:20.142627Z"
        },
        "id": "bGycblhT1qI2",
        "outputId": "db924523-ef16-41b5-e764-f9da6a4bc2c4",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ],
      "source": [
        "x.ndim"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aEgHXF7o1qI2"
      },
      "source": [
        "### 행렬 (랭크-2 텐서)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:20.146816Z",
          "iopub.status.busy": "2022-05-21T00:57:20.146544Z",
          "iopub.status.idle": "2022-05-21T00:57:20.153035Z",
          "shell.execute_reply": "2022-05-21T00:57:20.152330Z"
        },
        "id": "w4Pfesli1qI2",
        "outputId": "f7b97926-dd23-4f4d-9bb8-5763d6ad2861",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ],
      "source": [
        "x = np.array([[5, 78, 2, 34, 0],\n",
        "              [6, 79, 3, 35, 1],\n",
        "              [7, 80, 4, 36, 2]])\n",
        "x.ndim"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WY7v3OYg1qI2"
      },
      "source": [
        "### 랭크-3 텐서와 더 높은 랭크의 텐서"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:20.156415Z",
          "iopub.status.busy": "2022-05-21T00:57:20.156159Z",
          "iopub.status.idle": "2022-05-21T00:57:20.163001Z",
          "shell.execute_reply": "2022-05-21T00:57:20.162138Z"
        },
        "id": "EJw4bMhQ1qI3",
        "outputId": "26e8b47c-44d5-4b85-a754-1959b3b7e82b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ],
      "source": [
        "x = np.array([[[5, 78, 2, 34, 0],\n",
        "               [6, 79, 3, 35, 1],\n",
        "               [7, 80, 4, 36, 2]],\n",
        "              [[5, 78, 2, 34, 0],\n",
        "               [6, 79, 3, 35, 1],\n",
        "               [7, 80, 4, 36, 2]],\n",
        "              [[5, 78, 2, 34, 0],\n",
        "               [6, 79, 3, 35, 1],\n",
        "               [7, 80, 4, 36, 2]]])\n",
        "x.ndim"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wKerwOYq1qI3"
      },
      "source": [
        "### 핵심 속성"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:20.166433Z",
          "iopub.status.busy": "2022-05-21T00:57:20.166160Z",
          "iopub.status.idle": "2022-05-21T00:57:20.596456Z",
          "shell.execute_reply": "2022-05-21T00:57:20.595567Z"
        },
        "id": "TTHc4oX71qI3"
      },
      "outputs": [],
      "source": [
        "from tensorflow.keras.datasets import mnist\n",
        "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:20.600816Z",
          "iopub.status.busy": "2022-05-21T00:57:20.600517Z",
          "iopub.status.idle": "2022-05-21T00:57:20.605652Z",
          "shell.execute_reply": "2022-05-21T00:57:20.604662Z"
        },
        "id": "o5fj8Tyc1qI3",
        "outputId": "30dc1b4c-7c21-4b06-cda9-5e28ba9aad9a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ],
      "source": [
        "train_images.ndim"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:20.608889Z",
          "iopub.status.busy": "2022-05-21T00:57:20.608612Z",
          "iopub.status.idle": "2022-05-21T00:57:20.614638Z",
          "shell.execute_reply": "2022-05-21T00:57:20.613826Z"
        },
        "id": "SWgMFCzV1qI3",
        "outputId": "24aa76c8-4925-4c5a-e654-e3c001796671",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(60000, 28, 28)"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ],
      "source": [
        "train_images.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:20.618889Z",
          "iopub.status.busy": "2022-05-21T00:57:20.618226Z",
          "iopub.status.idle": "2022-05-21T00:57:20.625094Z",
          "shell.execute_reply": "2022-05-21T00:57:20.624338Z"
        },
        "id": "FbDovWF61qI3",
        "outputId": "43c8f901-7d48-410d-df1c-44fad0802c95",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "dtype('uint8')"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ],
      "source": [
        "train_images.dtype"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JRoaVKPv1qI3"
      },
      "source": [
        "**다섯 번째 이미지 출력하기**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:20.628820Z",
          "iopub.status.busy": "2022-05-21T00:57:20.628306Z",
          "iopub.status.idle": "2022-05-21T00:57:21.431208Z",
          "shell.execute_reply": "2022-05-21T00:57:21.430286Z"
        },
        "id": "IIZkhp0i1qI4",
        "outputId": "29c98f45-f2f7-4ced-ad32-e170a403e420",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 430
        }
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaAAAAGdCAYAAABU0qcqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAbYklEQVR4nO3df2zU9R3H8deB9ERsryulvZ4ULKigAl2G0jUq4mgoXUZAyCbqFjAEIitG7JymTkSdWSdmzOgq/rPB3ESYiUD0DxxW286tsIESxn50tOkEAi1I0l4pUhj97I+G2w6K8D3u+u4dz0fyTejd99N78/XSp1/67bc+55wTAAD9bJD1AACAKxMBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJq6yHuBcPT09OnTokNLT0+Xz+azHAQB45JxTZ2enQqGQBg268HnOgAvQoUOHlJ+fbz0GAOAyHThwQCNHjrzg8wMuQOnp6ZJ6B8/IyDCeBgDgVTgcVn5+fuTr+YUkLEDV1dV66aWX1NraqsLCQr366quaMmXKRded/We3jIwMAgQASexi30ZJyEUIGzduVEVFhVauXKlPPvlEhYWFKi0t1ZEjRxLxcgCAJJSQAK1evVqLFy/WQw89pFtuuUWvv/66rrnmGv3qV79KxMsBAJJQ3AN06tQp7dq1SyUlJf97kUGDVFJSooaGhvP27+7uVjgcjtoAAKkv7gH6/PPPdebMGeXm5kY9npubq9bW1vP2r6qqUiAQiGxcAQcAVwbzH0StrKxUR0dHZDtw4ID1SACAfhD3q+Cys7M1ePBgtbW1RT3e1tamYDB43v5+v19+vz/eYwAABri4nwGlpaVp8uTJqqmpiTzW09OjmpoaFRcXx/vlAABJKiE/B1RRUaEFCxbotttu05QpU/Tyyy+rq6tLDz30UCJeDgCQhBISoPvuu09Hjx7VM888o9bWVn31q1/V1q1bz7swAQBw5fI555z1EP8vHA4rEAioo6ODOyEAQBK61K/j5lfBAQCuTAQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwETcA/Tss8/K5/NFbePHj4/3ywAAktxVifikt956qz744IP/vchVCXkZAEASS0gZrrrqKgWDwUR8agBAikjI94D27dunUCikMWPG6MEHH9T+/fsvuG93d7fC4XDUBgBIfXEPUFFRkdatW6etW7dqzZo1amlp0V133aXOzs4+96+qqlIgEIhs+fn58R4JADAA+ZxzLpEv0N7ertGjR2v16tVatGjRec93d3eru7s78nE4HFZ+fr46OjqUkZGRyNEAAAkQDocVCAQu+nU84VcHZGZm6qabblJTU1Ofz/v9fvn9/kSPAQAYYBL+c0DHjx9Xc3Oz8vLyEv1SAIAkEvcAPf7446qrq9O///1v/elPf9K9996rwYMH6/7774/3SwEAkljc/wnu4MGDuv/++3Xs2DGNGDFCd955p7Zv364RI0bE+6UAAEks7gHasGFDvD8lACAFcS84AIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMBEwn8hHZBMduzY4XnNb37zG89r6uvrPa/Zu3ev5zWx+tnPfuZ5TSgU8rzmD3/4g+c13/ve9zyvKSoq8rwGiccZEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAExwN2ykpI0bN8a07tFHH/W85ujRo57XOOc8r5k2bZrnNZ9//rnnNZL0+OOPx7TOq1iOQyx/pw0bNnheg8TjDAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMMHNSNGv/vOf/3he85e//MXzmsWLF3teI0ldXV2e19x9992e16xYscLzmjvvvNPzmu7ubs9rJOk73/mO5zXvv/9+TK/l1W233dYvr4PE4wwIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADDBzUjRr3772996XrNo0aIETNK3GTNmeF6zceNGz2syMjI8r4lFLLNJ/Xdj0fz8fM9rFixYkIBJYIEzIACACQIEADDhOUD19fWaNWuWQqGQfD6fNm/eHPW8c07PPPOM8vLyNHToUJWUlGjfvn3xmhcAkCI8B6irq0uFhYWqrq7u8/lVq1bplVde0euvv64dO3Zo2LBhKi0t1cmTJy97WABA6vB8EUJZWZnKysr6fM45p5dffllPP/20Zs+eLUl64403lJubq82bN2v+/PmXNy0AIGXE9XtALS0tam1tVUlJSeSxQCCgoqIiNTQ09Lmmu7tb4XA4agMApL64Bqi1tVWSlJubG/V4bm5u5LlzVVVVKRAIRLZYLssEACQf86vgKisr1dHREdkOHDhgPRIAoB/ENUDBYFCS1NbWFvV4W1tb5Llz+f1+ZWRkRG0AgNQX1wAVFBQoGAyqpqYm8lg4HNaOHTtUXFwcz5cCACQ5z1fBHT9+XE1NTZGPW1patHv3bmVlZWnUqFFavny5XnjhBd14440qKCjQihUrFAqFNGfOnHjODQBIcp4DtHPnTt1zzz2RjysqKiT13p9p3bp1euKJJ9TV1aUlS5aovb1dd955p7Zu3aqrr746flMDAJKezznnrIf4f+FwWIFAQB0dHXw/aIB7+umnPa/5yU9+4nmNz+fzvKa8vNzzGkl64YUXPK8ZyO/Tm2++OaZ1//rXv+I8Sd/eeecdz2vO/owhBq5L/TpufhUcAODKRIAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABOefx0DUs/zzz8f07pY7mzt9/s9ryktLfW85sUXX/S8RpKGDh0a0zqvTp486XnN73//e89rPvvsM89rJCmWm+SvWLHC8xrubH1l4wwIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADDBzUhTTHt7u+c1r732Wkyv5fP5PK+J5caimzdv9rymPzU1NXle8+CDD3pes3PnTs9rYvXtb3/b85onnngiAZMglXEGBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCY4GakKebUqVOe1xw9ejQBk/TtlVde8bzmyJEjntesXbvW8xpJ2rJli+c1f/vb3zyv6ezs9Lwmlpu/DhoU2/9jfve73/W8ZtiwYTG9Fq5cnAEBAEwQIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACa4GWmKSUtL87wmJycnpteK5Sah119/vec1sdyEsz9dd911ntdkZGR4XnPo0CHPa7Kzsz2vkaRZs2bFtA7wgjMgAIAJAgQAMOE5QPX19Zo1a5ZCoZB8Pp82b94c9fzChQvl8/mitpkzZ8ZrXgBAivAcoK6uLhUWFqq6uvqC+8ycOVOHDx+ObG+99dZlDQkASD2eL0IoKytTWVnZl+7j9/sVDAZjHgoAkPoS8j2g2tpa5eTkaNy4cVq6dKmOHTt2wX27u7sVDoejNgBA6ot7gGbOnKk33nhDNTU1evHFF1VXV6eysjKdOXOmz/2rqqoUCAQiW35+frxHAgAMQHH/OaD58+dH/jxx4kRNmjRJY8eOVW1traZPn37e/pWVlaqoqIh8HA6HiRAAXAESfhn2mDFjlJ2draampj6f9/v9ysjIiNoAAKkv4QE6ePCgjh07pry8vES/FAAgiXj+J7jjx49Hnc20tLRo9+7dysrKUlZWlp577jnNmzdPwWBQzc3NeuKJJ3TDDTeotLQ0roMDAJKb5wDt3LlT99xzT+Tjs9+/WbBggdasWaM9e/bo17/+tdrb2xUKhTRjxgz9+Mc/lt/vj9/UAICk5zlA06ZNk3Pugs+///77lzUQLk9mZqbnNefezeJSfetb3/K85ssuyb+QG264wfOa2bNne14j9d7Jw6usrCzPa/7/Yp1LFcvNSGN5HaC/cC84AIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmIj7r+RG8ikqKopp3dGjR+M8SXKqr6/3vKaurs7zGp/P53nNmDFjPK8B+gtnQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACW5GClymL774wvOaWG4sGsua+fPne14D9BfOgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAE9yMFLhMpaWl1iMASYkzIACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABDcjBS7T+++/bz0CkJQ4AwIAmCBAAAATngJUVVWl22+/Xenp6crJydGcOXPU2NgYtc/JkydVXl6u4cOH69prr9W8efPU1tYW16EBAMnPU4Dq6upUXl6u7du3a9u2bTp9+rRmzJihrq6uyD6PPfaY3n33Xb399tuqq6vToUOHNHfu3LgPDgBIbp4uQti6dWvUx+vWrVNOTo527dqlqVOnqqOjQ7/85S+1fv16feMb35AkrV27VjfffLO2b9+ur3/96/GbHACQ1C7re0AdHR2SpKysLEnSrl27dPr0aZWUlET2GT9+vEaNGqWGhoY+P0d3d7fC4XDUBgBIfTEHqKenR8uXL9cdd9yhCRMmSJJaW1uVlpamzMzMqH1zc3PV2tra5+epqqpSIBCIbPn5+bGOBABIIjEHqLy8XHv37tWGDRsua4DKykp1dHREtgMHDlzW5wMAJIeYfhB12bJleu+991RfX6+RI0dGHg8Ggzp16pTa29ujzoLa2toUDAb7/Fx+v19+vz+WMQAASczTGZBzTsuWLdOmTZv04YcfqqCgIOr5yZMna8iQIaqpqYk81tjYqP3796u4uDg+EwMAUoKnM6Dy8nKtX79eW7ZsUXp6euT7OoFAQEOHDlUgENCiRYtUUVGhrKwsZWRk6JFHHlFxcTFXwAEAongK0Jo1ayRJ06ZNi3p87dq1WrhwoSTp5z//uQYNGqR58+apu7tbpaWleu211+IyLAAgdXgKkHPuovtcffXVqq6uVnV1dcxDAcmkubnZegQgKXEvOACACQIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJiI6TeiAvifu+66y/OaS7mzPJDqOAMCAJggQAAAEwQIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAExwM1LgMk2cONHzmhtvvNHzmubm5n5ZI0kjRoyIaR3gBWdAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAIAJbkYKGHjqqac8r1m0aFG/vI4k/eIXv/C85pZbbonptXDl4gwIAGCCAAEATBAgAIAJAgQAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADDBzUgBA3PnzvW8ZsOGDZ7XbNu2zfMaSXr22Wc9r1m7dq3nNcOGDfO8BqmDMyAAgAkCBAAw4SlAVVVVuv3225Wenq6cnBzNmTNHjY2NUftMmzZNPp8vanv44YfjOjQAIPl5ClBdXZ3Ky8u1fft2bdu2TadPn9aMGTPU1dUVtd/ixYt1+PDhyLZq1aq4Dg0ASH6eLkLYunVr1Mfr1q1TTk6Odu3apalTp0Yev+aaaxQMBuMzIQAgJV3W94A6OjokSVlZWVGPv/nmm8rOztaECRNUWVmpEydOXPBzdHd3KxwOR20AgNQX82XYPT09Wr58ue644w5NmDAh8vgDDzyg0aNHKxQKac+ePXryySfV2Niod955p8/PU1VVpeeeey7WMQAASSrmAJWXl2vv3r36+OOPox5fsmRJ5M8TJ05UXl6epk+frubmZo0dO/a8z1NZWamKiorIx+FwWPn5+bGOBQBIEjEFaNmyZXrvvfdUX1+vkSNHfum+RUVFkqSmpqY+A+T3++X3+2MZAwCQxDwFyDmnRx55RJs2bVJtba0KCgouumb37t2SpLy8vJgGBACkJk8BKi8v1/r167Vlyxalp6ertbVVkhQIBDR06FA1Nzdr/fr1+uY3v6nhw4drz549euyxxzR16lRNmjQpIX8BAEBy8hSgNWvWSOr9YdP/t3btWi1cuFBpaWn64IMP9PLLL6urq0v5+fmaN2+enn766bgNDABIDZ7/Ce7L5Ofnq66u7rIGAgBcGXzuYlXpZ+FwWIFAQB0dHcrIyLAeBxgwYvkZuR/96EcxvdZrr73mec1f//pXz2tuueUWz2sw8F3q13FuRgoAMEGAAAAmCBAAwAQBAgCYIEAAABMECABgggABAEwQIACACQIEADBBgAAAJggQAMAEAQIAmOBmpACAuOJmpACAAY0AAQBMECAAgAkCBAAwQYAAACYIEADABAECAJggQAAAEwQIAGCCAAEATBAgAICJq6wHONfZW9OFw2HjSQAAsTj79ftitxodcAHq7OyUJOXn5xtPAgC4HJ2dnQoEAhd8fsDdDbunp0eHDh1Senq6fD5f1HPhcFj5+fk6cODAFX2nbI5DL45DL45DL45Dr4FwHJxz6uzsVCgU0qBBF/5Oz4A7Axo0aJBGjhz5pftkZGRc0W+wszgOvTgOvTgOvTgOvayPw5ed+ZzFRQgAABMECABgIqkC5Pf7tXLlSvn9futRTHEcenEcenEcenEceiXTcRhwFyEAAK4MSXUGBABIHQQIAGCCAAEATBAgAICJpAlQdXW1rr/+el199dUqKirSn//8Z+uR+t2zzz4rn88XtY0fP956rISrr6/XrFmzFAqF5PP5tHnz5qjnnXN65plnlJeXp6FDh6qkpET79u2zGTaBLnYcFi5ceN77Y+bMmTbDJkhVVZVuv/12paenKycnR3PmzFFjY2PUPidPnlR5ebmGDx+ua6+9VvPmzVNbW5vRxIlxKcdh2rRp570fHn74YaOJ+5YUAdq4caMqKiq0cuVKffLJJyosLFRpaamOHDliPVq/u/XWW3X48OHI9vHHH1uPlHBdXV0qLCxUdXV1n8+vWrVKr7zyil5//XXt2LFDw4YNU2lpqU6ePNnPkybWxY6DJM2cOTPq/fHWW2/144SJV1dXp/Lycm3fvl3btm3T6dOnNWPGDHV1dUX2eeyxx/Tuu+/q7bffVl1dnQ4dOqS5c+caTh1/l3IcJGnx4sVR74dVq1YZTXwBLglMmTLFlZeXRz4+c+aMC4VCrqqqynCq/rdy5UpXWFhoPYYpSW7Tpk2Rj3t6elwwGHQvvfRS5LH29nbn9/vdW2+9ZTBh/zj3ODjn3IIFC9zs2bNN5rFy5MgRJ8nV1dU553r/2w8ZMsS9/fbbkX3+8Y9/OEmuoaHBasyEO/c4OOfc3Xff7R599FG7oS7BgD8DOnXqlHbt2qWSkpLIY4MGDVJJSYkaGhoMJ7Oxb98+hUIhjRkzRg8++KD2799vPZKplpYWtba2Rr0/AoGAioqKrsj3R21trXJycjRu3DgtXbpUx44dsx4poTo6OiRJWVlZkqRdu3bp9OnTUe+H8ePHa9SoUSn9fjj3OJz15ptvKjs7WxMmTFBlZaVOnDhhMd4FDbibkZ7r888/15kzZ5Sbmxv1eG5urv75z38aTWWjqKhI69at07hx43T48GE999xzuuuuu7R3716lp6dbj2eitbVVkvp8f5x97koxc+ZMzZ07VwUFBWpubtZTTz2lsrIyNTQ0aPDgwdbjxV1PT4+WL1+uO+64QxMmTJDU+35IS0tTZmZm1L6p/H7o6zhI0gMPPKDRo0crFAppz549evLJJ9XY2Kh33nnHcNpoAz5A+J+ysrLInydNmqSioiKNHj1av/vd77Ro0SLDyTAQzJ8/P/LniRMnatKkSRo7dqxqa2s1ffp0w8kSo7y8XHv37r0ivg/6ZS50HJYsWRL588SJE5WXl6fp06erublZY8eO7e8x+zTg/wkuOztbgwcPPu8qlra2NgWDQaOpBobMzEzddNNNampqsh7FzNn3AO+P840ZM0bZ2dkp+f5YtmyZ3nvvPX300UdRv74lGAzq1KlTam9vj9o/Vd8PFzoOfSkqKpKkAfV+GPABSktL0+TJk1VTUxN5rKenRzU1NSouLjaczN7x48fV3NysvLw861HMFBQUKBgMRr0/wuGwduzYccW/Pw4ePKhjx46l1PvDOadly5Zp06ZN+vDDD1VQUBD1/OTJkzVkyJCo90NjY6P279+fUu+Hix2HvuzevVuSBtb7wfoqiEuxYcMG5/f73bp169zf//53t2TJEpeZmelaW1utR+tXP/jBD1xtba1raWlxf/zjH11JSYnLzs52R44csR4toTo7O92nn37qPv30UyfJrV692n366afus88+c84599Of/tRlZma6LVu2uD179rjZs2e7goIC98UXXxhPHl9fdhw6Ozvd448/7hoaGlxLS4v74IMP3Ne+9jV34403upMnT1qPHjdLly51gUDA1dbWusOHD0e2EydORPZ5+OGH3ahRo9yHH37odu7c6YqLi11xcbHh1PF3sePQ1NTknn/+ebdz507X0tLitmzZ4saMGeOmTp1qPHm0pAiQc869+uqrbtSoUS4tLc1NmTLFbd++3Xqkfnffffe5vLw8l5aW5q677jp33333uaamJuuxEu6jjz5yks7bFixY4JzrvRR7xYoVLjc31/n9fjd9+nTX2NhoO3QCfNlxOHHihJsxY4YbMWKEGzJkiBs9erRbvHhxyv1PWl9/f0lu7dq1kX2++OIL9/3vf9995Stfcddcc42799573eHDh+2GToCLHYf9+/e7qVOnuqysLOf3+90NN9zgfvjDH7qOjg7bwc/Br2MAAJgY8N8DAgCkJgIEADBBgAAAJggQAMAEAQIAmCBAAAATBAgAYIIAAQBMECAAgAkCBAAwQYAAACYIEADAxH8BB0q1GdOY6GMAAAAASUVORK5CYII=\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "digit = train_images[4]\n",
        "plt.imshow(digit, cmap=plt.cm.binary)\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:21.435121Z",
          "iopub.status.busy": "2022-05-21T00:57:21.434503Z",
          "iopub.status.idle": "2022-05-21T00:57:21.439445Z",
          "shell.execute_reply": "2022-05-21T00:57:21.438851Z"
        },
        "id": "SIQgEABY1qI4",
        "outputId": "3741db34-dd7c-421b-8eae-fffdbae49a07",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "9"
            ]
          },
          "metadata": {},
          "execution_count": 28
        }
      ],
      "source": [
        "train_labels[4]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wBkiHJlR1qI4"
      },
      "source": [
        "### 넘파이로 텐서 조작하기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:21.443500Z",
          "iopub.status.busy": "2022-05-21T00:57:21.443068Z",
          "iopub.status.idle": "2022-05-21T00:57:21.447465Z",
          "shell.execute_reply": "2022-05-21T00:57:21.446885Z"
        },
        "id": "c2A6S7-s1qI4",
        "outputId": "f955a915-0989-48da-cfab-4c52a563c1d9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(90, 28, 28)"
            ]
          },
          "metadata": {},
          "execution_count": 29
        }
      ],
      "source": [
        "my_slice = train_images[10:100]\n",
        "my_slice.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:21.450776Z",
          "iopub.status.busy": "2022-05-21T00:57:21.450356Z",
          "iopub.status.idle": "2022-05-21T00:57:21.454814Z",
          "shell.execute_reply": "2022-05-21T00:57:21.454187Z"
        },
        "id": "yTGVsW8W1qI4",
        "outputId": "9c90984d-1acb-42cb-f05e-c067180a17cc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(90, 28, 28)"
            ]
          },
          "metadata": {},
          "execution_count": 30
        }
      ],
      "source": [
        "my_slice = train_images[10:100, :, :]\n",
        "my_slice.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:21.458290Z",
          "iopub.status.busy": "2022-05-21T00:57:21.457804Z",
          "iopub.status.idle": "2022-05-21T00:57:21.462673Z",
          "shell.execute_reply": "2022-05-21T00:57:21.461775Z"
        },
        "id": "hQ-iwuuM1qI4",
        "outputId": "1f290800-df4f-4b54-999d-1eda1c18340b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(90, 28, 28)"
            ]
          },
          "metadata": {},
          "execution_count": 31
        }
      ],
      "source": [
        "my_slice = train_images[10:100, 0:28, 0:28]\n",
        "my_slice.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:21.465976Z",
          "iopub.status.busy": "2022-05-21T00:57:21.465505Z",
          "iopub.status.idle": "2022-05-21T00:57:21.469168Z",
          "shell.execute_reply": "2022-05-21T00:57:21.468362Z"
        },
        "id": "na1tzxx21qI4"
      },
      "outputs": [],
      "source": [
        "my_slice = train_images[:, 14:, 14:]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:21.472711Z",
          "iopub.status.busy": "2022-05-21T00:57:21.472282Z",
          "iopub.status.idle": "2022-05-21T00:57:21.476345Z",
          "shell.execute_reply": "2022-05-21T00:57:21.475454Z"
        },
        "id": "jTHEZSaa1qI5"
      },
      "outputs": [],
      "source": [
        "my_slice = train_images[:, 7:-7, 7:-7]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VHuR99nN1qI5"
      },
      "source": [
        "### 배치 데이터"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:21.479963Z",
          "iopub.status.busy": "2022-05-21T00:57:21.479396Z",
          "iopub.status.idle": "2022-05-21T00:57:21.482994Z",
          "shell.execute_reply": "2022-05-21T00:57:21.482245Z"
        },
        "id": "Ewj8YTjl1qI5"
      },
      "outputs": [],
      "source": [
        "batch = train_images[:128]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:21.486283Z",
          "iopub.status.busy": "2022-05-21T00:57:21.485817Z",
          "iopub.status.idle": "2022-05-21T00:57:21.489326Z",
          "shell.execute_reply": "2022-05-21T00:57:21.488653Z"
        },
        "id": "CKvQh_Dc1qI5"
      },
      "outputs": [],
      "source": [
        "batch = train_images[128:256]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:21.492485Z",
          "iopub.status.busy": "2022-05-21T00:57:21.491982Z",
          "iopub.status.idle": "2022-05-21T00:57:21.495891Z",
          "shell.execute_reply": "2022-05-21T00:57:21.495127Z"
        },
        "id": "FXxjaILB1qI5"
      },
      "outputs": [],
      "source": [
        "n = 3\n",
        "batch = train_images[128 * n:128 * (n + 1)]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zCoiKdln1qI5"
      },
      "source": [
        "### 텐서의 실제 사례"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IdfIlLai1qI5"
      },
      "source": [
        "### 벡터 데이터"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mraA72yU1qI5"
      },
      "source": [
        "### 시계열 데이터 또는 시퀀스 데이터"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sN_C-CyC1qI5"
      },
      "source": [
        "### 이미지 데이터"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V_1Nl_5a1qI6"
      },
      "source": [
        "### 비디오 데이터"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Up42nxHL1qI6"
      },
      "source": [
        "## 신경망의 톱니바퀴: 텐서 연산"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nkl--fF91qI6"
      },
      "source": [
        "### 원소별 연산"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 37,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:21.499755Z",
          "iopub.status.busy": "2022-05-21T00:57:21.499215Z",
          "iopub.status.idle": "2022-05-21T00:57:21.503803Z",
          "shell.execute_reply": "2022-05-21T00:57:21.502971Z"
        },
        "id": "t6Z3vq3l1qI6"
      },
      "outputs": [],
      "source": [
        "def naive_relu(x):\n",
        "    assert len(x.shape) == 2\n",
        "    x = x.copy()\n",
        "    for i in range(x.shape[0]):\n",
        "        for j in range(x.shape[1]):\n",
        "            x[i, j] = max(x[i, j], 0)\n",
        "    return x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:21.507340Z",
          "iopub.status.busy": "2022-05-21T00:57:21.506666Z",
          "iopub.status.idle": "2022-05-21T00:57:21.511473Z",
          "shell.execute_reply": "2022-05-21T00:57:21.510630Z"
        },
        "id": "uaoPSZmK1qI6"
      },
      "outputs": [],
      "source": [
        "def naive_add(x, y):\n",
        "    assert len(x.shape) == 2\n",
        "    assert x.shape == y.shape\n",
        "    x = x.copy()\n",
        "    for i in range(x.shape[0]):\n",
        "        for j in range(x.shape[1]):\n",
        "            x[i, j] += y[i, j]\n",
        "    return x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 39,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:21.514953Z",
          "iopub.status.busy": "2022-05-21T00:57:21.514566Z",
          "iopub.status.idle": "2022-05-21T00:57:21.524287Z",
          "shell.execute_reply": "2022-05-21T00:57:21.523672Z"
        },
        "id": "yahpjVBQ1qI6",
        "outputId": "b153fc39-8a8b-4593-db5a-96c4dc425ee1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "걸린 시간: 0.01 s\n"
          ]
        }
      ],
      "source": [
        "import time\n",
        "\n",
        "x = np.random.random((20, 100))\n",
        "y = np.random.random((20, 100))\n",
        "\n",
        "t0 = time.time()\n",
        "for _ in range(1000):\n",
        "    z = x + y\n",
        "    z = np.maximum(z, 0.)\n",
        "print(\"걸린 시간: {0:.2f} s\".format(time.time() - t0))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 40,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:21.527704Z",
          "iopub.status.busy": "2022-05-21T00:57:21.527269Z",
          "iopub.status.idle": "2022-05-21T00:57:23.525600Z",
          "shell.execute_reply": "2022-05-21T00:57:23.524629Z"
        },
        "id": "qA4UxC_g1qI6",
        "outputId": "caabda17-883c-4650-cd54-897b3dcd3573",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "걸린 시간: 1.74 s\n"
          ]
        }
      ],
      "source": [
        "t0 = time.time()\n",
        "for _ in range(1000):\n",
        "    z = naive_add(x, y)\n",
        "    z = naive_relu(z)\n",
        "print(\"걸린 시간: {0:.2f} s\".format(time.time() - t0))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9ZkDW9ie1qI7"
      },
      "source": [
        "### 브로드캐스팅"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 41,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:23.529492Z",
          "iopub.status.busy": "2022-05-21T00:57:23.529207Z",
          "iopub.status.idle": "2022-05-21T00:57:23.533586Z",
          "shell.execute_reply": "2022-05-21T00:57:23.532638Z"
        },
        "id": "Rpk8MkPx1qI7"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "X = np.random.random((32, 10))\n",
        "y = np.random.random((10,))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 42,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:23.536953Z",
          "iopub.status.busy": "2022-05-21T00:57:23.536681Z",
          "iopub.status.idle": "2022-05-21T00:57:23.541762Z",
          "shell.execute_reply": "2022-05-21T00:57:23.540974Z"
        },
        "id": "dBPKHsHi1qI7"
      },
      "outputs": [],
      "source": [
        "y = np.expand_dims(y, axis=0)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 43,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:23.545374Z",
          "iopub.status.busy": "2022-05-21T00:57:23.545120Z",
          "iopub.status.idle": "2022-05-21T00:57:23.549140Z",
          "shell.execute_reply": "2022-05-21T00:57:23.548341Z"
        },
        "id": "3fuvM3rd1qI7"
      },
      "outputs": [],
      "source": [
        "Y = np.concatenate([y] * 32, axis=0)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 44,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:23.552981Z",
          "iopub.status.busy": "2022-05-21T00:57:23.552734Z",
          "iopub.status.idle": "2022-05-21T00:57:23.557779Z",
          "shell.execute_reply": "2022-05-21T00:57:23.557013Z"
        },
        "id": "DM59177S1qI7"
      },
      "outputs": [],
      "source": [
        "def naive_add_matrix_and_vector(x, y):\n",
        "    assert len(x.shape) == 2\n",
        "    assert len(y.shape) == 1\n",
        "    assert x.shape[1] == y.shape[0]\n",
        "    x = x.copy()\n",
        "    for i in range(x.shape[0]):\n",
        "        for j in range(x.shape[1]):\n",
        "            x[i, j] += y[j]\n",
        "    return x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 45,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:23.561664Z",
          "iopub.status.busy": "2022-05-21T00:57:23.561412Z",
          "iopub.status.idle": "2022-05-21T00:57:23.566195Z",
          "shell.execute_reply": "2022-05-21T00:57:23.565424Z"
        },
        "id": "48yuFoqQ1qI7"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "x = np.random.random((64, 3, 32, 10))\n",
        "y = np.random.random((32, 10))\n",
        "z = np.maximum(x, y)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gtaTHBhr1qI7"
      },
      "source": [
        "### 텐서 곱셈"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 46,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:23.570018Z",
          "iopub.status.busy": "2022-05-21T00:57:23.569765Z",
          "iopub.status.idle": "2022-05-21T00:57:23.574104Z",
          "shell.execute_reply": "2022-05-21T00:57:23.573257Z"
        },
        "id": "Bfm1DPx91qI7"
      },
      "outputs": [],
      "source": [
        "x = np.random.random((32,))\n",
        "y = np.random.random((32,))\n",
        "z = np.dot(x, y)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 47,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:23.578149Z",
          "iopub.status.busy": "2022-05-21T00:57:23.577898Z",
          "iopub.status.idle": "2022-05-21T00:57:23.582594Z",
          "shell.execute_reply": "2022-05-21T00:57:23.581768Z"
        },
        "id": "0NHLG02B1qI7"
      },
      "outputs": [],
      "source": [
        "def naive_vector_dot(x, y):\n",
        "    assert len(x.shape) == 1\n",
        "    assert len(y.shape) == 1\n",
        "    assert x.shape[0] == y.shape[0]\n",
        "    z = 0.\n",
        "    for i in range(x.shape[0]):\n",
        "        z += x[i] * y[i]\n",
        "    return z"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 48,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:23.586856Z",
          "iopub.status.busy": "2022-05-21T00:57:23.586576Z",
          "iopub.status.idle": "2022-05-21T00:57:23.592373Z",
          "shell.execute_reply": "2022-05-21T00:57:23.591083Z"
        },
        "id": "vv6ks6x61qI8"
      },
      "outputs": [],
      "source": [
        "def naive_matrix_vector_dot(x, y):\n",
        "    assert len(x.shape) == 2\n",
        "    assert len(y.shape) == 1\n",
        "    assert x.shape[1] == y.shape[0]\n",
        "    z = np.zeros(x.shape[0])\n",
        "    for i in range(x.shape[0]):\n",
        "        for j in range(x.shape[1]):\n",
        "            z[i] += x[i, j] * y[j]\n",
        "    return z"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 49,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:23.595861Z",
          "iopub.status.busy": "2022-05-21T00:57:23.595594Z",
          "iopub.status.idle": "2022-05-21T00:57:23.599907Z",
          "shell.execute_reply": "2022-05-21T00:57:23.599230Z"
        },
        "id": "Y_AwR-rh1qI8"
      },
      "outputs": [],
      "source": [
        "def naive_matrix_vector_dot(x, y):\n",
        "    z = np.zeros(x.shape[0])\n",
        "    for i in range(x.shape[0]):\n",
        "        z[i] = naive_vector_dot(x[i, :], y)\n",
        "    return z"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 50,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:23.604217Z",
          "iopub.status.busy": "2022-05-21T00:57:23.603640Z",
          "iopub.status.idle": "2022-05-21T00:57:23.609035Z",
          "shell.execute_reply": "2022-05-21T00:57:23.608386Z"
        },
        "id": "_IJjhQB31qI8"
      },
      "outputs": [],
      "source": [
        "def naive_matrix_dot(x, y):\n",
        "    assert len(x.shape) == 2\n",
        "    assert len(y.shape) == 2\n",
        "    assert x.shape[1] == y.shape[0]\n",
        "    z = np.zeros((x.shape[0], y.shape[1]))\n",
        "    for i in range(x.shape[0]):\n",
        "        for j in range(y.shape[1]):\n",
        "            row_x = x[i, :]\n",
        "            column_y = y[:, j]\n",
        "            z[i, j] = naive_vector_dot(row_x, column_y)\n",
        "    return z"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kaVH1u8X1qI8"
      },
      "source": [
        "### 텐서 크기 변환"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 51,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:23.612816Z",
          "iopub.status.busy": "2022-05-21T00:57:23.612353Z",
          "iopub.status.idle": "2022-05-21T00:57:23.615796Z",
          "shell.execute_reply": "2022-05-21T00:57:23.615179Z"
        },
        "id": "Yc92w48C1qI8"
      },
      "outputs": [],
      "source": [
        "train_images = train_images.reshape((60000, 28 * 28))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 52,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:23.620077Z",
          "iopub.status.busy": "2022-05-21T00:57:23.619555Z",
          "iopub.status.idle": "2022-05-21T00:57:23.625554Z",
          "shell.execute_reply": "2022-05-21T00:57:23.624598Z"
        },
        "id": "0hKBrXod1qI8",
        "outputId": "ccbd2dcc-a6de-4231-d103-969d90b6011c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(3, 2)"
            ]
          },
          "metadata": {},
          "execution_count": 52
        }
      ],
      "source": [
        "x = np.array([[0., 1.],\n",
        "             [2., 3.],\n",
        "             [4., 5.]])\n",
        "x.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 53,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:23.629139Z",
          "iopub.status.busy": "2022-05-21T00:57:23.628859Z",
          "iopub.status.idle": "2022-05-21T00:57:23.634545Z",
          "shell.execute_reply": "2022-05-21T00:57:23.633514Z"
        },
        "id": "86-wf5je1qI8",
        "outputId": "34646e98-086c-4cd7-88bc-d63aaaec01e7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[0.],\n",
              "       [1.],\n",
              "       [2.],\n",
              "       [3.],\n",
              "       [4.],\n",
              "       [5.]])"
            ]
          },
          "metadata": {},
          "execution_count": 53
        }
      ],
      "source": [
        "x = x.reshape((6, 1))\n",
        "x"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "x = x.reshape((2, 3))\n",
        "x"
      ],
      "metadata": {
        "id": "gtw2H4682AOF",
        "outputId": "1e944fab-9d80-4602-dbe8-4133da32343d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 54,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[0., 1., 2.],\n",
              "       [3., 4., 5.]])"
            ]
          },
          "metadata": {},
          "execution_count": 54
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 55,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:23.639766Z",
          "iopub.status.busy": "2022-05-21T00:57:23.639078Z",
          "iopub.status.idle": "2022-05-21T00:57:23.644899Z",
          "shell.execute_reply": "2022-05-21T00:57:23.644035Z"
        },
        "id": "Dl9-o-GI1qI8",
        "outputId": "6bd7d643-c4d8-4887-ba41-01c0fb74e1fe",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(20, 300)"
            ]
          },
          "metadata": {},
          "execution_count": 55
        }
      ],
      "source": [
        "x = np.zeros((300, 20))\n",
        "x = np.transpose(x)\n",
        "x.shape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MniATBRC1qI9"
      },
      "source": [
        "### 텐서 연산의 기하학적 해석"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vkfc22a01qI9"
      },
      "source": [
        "### 딥러닝의 기하학적 해석"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sAFQT9yo1qI9"
      },
      "source": [
        "## 신경망의 엔진: 그레이디언트 기반 최적화"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Sa7y4mc-1qI9"
      },
      "source": [
        "### 도함수란?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DmdsFrvu1qI9"
      },
      "source": [
        "### 텐서 연산의 도함수: 그레이디언트"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HGJX4BYw1qI9"
      },
      "source": [
        "### 확률적 경사 하강법"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vuwyaSaB1qI9"
      },
      "source": [
        "### 도함수 연결: 역전파 알고리즘"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dLFHRaA_1qI9"
      },
      "source": [
        "#### 연쇄 법칙"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uFf4jwDb1qI9"
      },
      "source": [
        "#### 계산 그래프를 활용한 자동 미분"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1FH_nw__1qI9"
      },
      "source": [
        "#### 텐서플로의 그레이디언트 테이프"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 56,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:23.649769Z",
          "iopub.status.busy": "2022-05-21T00:57:23.649237Z",
          "iopub.status.idle": "2022-05-21T00:57:23.656183Z",
          "shell.execute_reply": "2022-05-21T00:57:23.655436Z"
        },
        "id": "gTu3wWTB1qI9"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "x = tf.Variable(0.)\n",
        "with tf.GradientTape() as tape:\n",
        "    y = 2 * x + 3\n",
        "grad_of_y_wrt_x = tape.gradient(y, x)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 57,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:23.659954Z",
          "iopub.status.busy": "2022-05-21T00:57:23.659320Z",
          "iopub.status.idle": "2022-05-21T00:57:23.665978Z",
          "shell.execute_reply": "2022-05-21T00:57:23.665192Z"
        },
        "id": "wQyEYbZo1qI-"
      },
      "outputs": [],
      "source": [
        "x = tf.Variable(tf.zeros((2, 2)))\n",
        "with tf.GradientTape() as tape:\n",
        "    y = 2 * x + 3\n",
        "grad_of_y_wrt_x = tape.gradient(y, x)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 58,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:23.669642Z",
          "iopub.status.busy": "2022-05-21T00:57:23.669139Z",
          "iopub.status.idle": "2022-05-21T00:57:23.677360Z",
          "shell.execute_reply": "2022-05-21T00:57:23.676599Z"
        },
        "id": "zbst9LJC1qI-"
      },
      "outputs": [],
      "source": [
        "W = tf.Variable(tf.random.uniform((2, 2)))\n",
        "b = tf.Variable(tf.zeros((2,)))\n",
        "x = tf.random.uniform((2, 2))\n",
        "with tf.GradientTape() as tape:\n",
        "    y = tf.matmul(x, W) + b\n",
        "grad_of_y_wrt_W_and_b = tape.gradient(y, [W, b])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ma33AlGI1qI-"
      },
      "source": [
        "## 첫 번째 예제 다시 살펴보기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 59,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:23.681093Z",
          "iopub.status.busy": "2022-05-21T00:57:23.680602Z",
          "iopub.status.idle": "2022-05-21T00:57:24.079600Z",
          "shell.execute_reply": "2022-05-21T00:57:24.078307Z"
        },
        "id": "QwI0nrkj1qI-"
      },
      "outputs": [],
      "source": [
        "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()\n",
        "train_images = train_images.reshape((60000, 28 * 28))\n",
        "train_images = train_images.astype(\"float32\") / 255\n",
        "test_images = test_images.reshape((10000, 28 * 28))\n",
        "test_images = test_images.astype(\"float32\") / 255"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 60,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:24.084071Z",
          "iopub.status.busy": "2022-05-21T00:57:24.083440Z",
          "iopub.status.idle": "2022-05-21T00:57:24.092511Z",
          "shell.execute_reply": "2022-05-21T00:57:24.091627Z"
        },
        "id": "lmY9b4qU1qI-"
      },
      "outputs": [],
      "source": [
        "model = keras.Sequential([\n",
        "    layers.Dense(512, activation=\"relu\"),\n",
        "    layers.Dense(10, activation=\"softmax\")\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 61,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:24.096195Z",
          "iopub.status.busy": "2022-05-21T00:57:24.095736Z",
          "iopub.status.idle": "2022-05-21T00:57:24.104256Z",
          "shell.execute_reply": "2022-05-21T00:57:24.103588Z"
        },
        "id": "hfeRt6LZ1qI-"
      },
      "outputs": [],
      "source": [
        "model.compile(optimizer=\"rmsprop\",\n",
        "              loss=\"sparse_categorical_crossentropy\",\n",
        "              metrics=[\"accuracy\"])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 62,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:24.107793Z",
          "iopub.status.busy": "2022-05-21T00:57:24.107143Z",
          "iopub.status.idle": "2022-05-21T00:57:34.534710Z",
          "shell.execute_reply": "2022-05-21T00:57:34.533527Z"
        },
        "id": "K903L3_y1qI-",
        "outputId": "c7e7dd91-3f1f-4665-b8c9-d76f3d23703c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/5\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 0.2645 - accuracy: 0.9236\n",
            "Epoch 2/5\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.1075 - accuracy: 0.9676\n",
            "Epoch 3/5\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.0708 - accuracy: 0.9779\n",
            "Epoch 4/5\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 0.0520 - accuracy: 0.9843\n",
            "Epoch 5/5\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0390 - accuracy: 0.9883\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x785796e57160>"
            ]
          },
          "metadata": {},
          "execution_count": 62
        }
      ],
      "source": [
        "model.fit(train_images, train_labels, epochs=5, batch_size=128)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XAmtOQyq1qI-"
      },
      "source": [
        "### 텐서플로를 사용해 첫 번째 예제를 밑바닥부터 다시 구현하기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RsBD9EST1qI_"
      },
      "source": [
        "#### 단순한 Dense 클래스"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 63,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:34.539073Z",
          "iopub.status.busy": "2022-05-21T00:57:34.538100Z",
          "iopub.status.idle": "2022-05-21T00:57:34.544603Z",
          "shell.execute_reply": "2022-05-21T00:57:34.543941Z"
        },
        "id": "2T0kW_M81qI_"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "\n",
        "class NaiveDense:\n",
        "    def __init__(self, input_size, output_size, activation):\n",
        "        self.activation = activation\n",
        "\n",
        "        w_shape = (input_size, output_size)\n",
        "        w_initial_value = tf.random.uniform(w_shape, minval=0, maxval=1e-1)\n",
        "        self.W = tf.Variable(w_initial_value)\n",
        "\n",
        "        b_shape = (output_size,)\n",
        "        b_initial_value = tf.zeros(b_shape)\n",
        "        self.b = tf.Variable(b_initial_value)\n",
        "\n",
        "    def __call__(self, inputs):\n",
        "        return self.activation(tf.matmul(inputs, self.W) + self.b)\n",
        "\n",
        "    @property\n",
        "    def weights(self):\n",
        "        return [self.W, self.b]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qVHHdEOF1qI_"
      },
      "source": [
        "#### 단순한 Sequential 클래스"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 64,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:34.549011Z",
          "iopub.status.busy": "2022-05-21T00:57:34.548061Z",
          "iopub.status.idle": "2022-05-21T00:57:34.553613Z",
          "shell.execute_reply": "2022-05-21T00:57:34.552900Z"
        },
        "id": "lQ8O3Mti1qI_"
      },
      "outputs": [],
      "source": [
        "class NaiveSequential:\n",
        "    def __init__(self, layers):\n",
        "        self.layers = layers\n",
        "\n",
        "    def __call__(self, inputs):\n",
        "        x = inputs\n",
        "        for layer in self.layers:\n",
        "           x = layer(x)\n",
        "        return x\n",
        "\n",
        "    @property\n",
        "    def weights(self):\n",
        "       weights = []\n",
        "       for layer in self.layers:\n",
        "           weights += layer.weights\n",
        "       return weights"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 65,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:34.557429Z",
          "iopub.status.busy": "2022-05-21T00:57:34.556712Z",
          "iopub.status.idle": "2022-05-21T00:57:34.565352Z",
          "shell.execute_reply": "2022-05-21T00:57:34.564638Z"
        },
        "id": "L40rhe-A1qI_"
      },
      "outputs": [],
      "source": [
        "model = NaiveSequential([\n",
        "    NaiveDense(input_size=28 * 28, output_size=512, activation=tf.nn.relu),\n",
        "    NaiveDense(input_size=512, output_size=10, activation=tf.nn.softmax)\n",
        "])\n",
        "assert len(model.weights) == 4"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gYioh4jn1qI_"
      },
      "source": [
        "#### 배치 제너레이터"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 66,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:34.569718Z",
          "iopub.status.busy": "2022-05-21T00:57:34.568911Z",
          "iopub.status.idle": "2022-05-21T00:57:34.575841Z",
          "shell.execute_reply": "2022-05-21T00:57:34.575023Z"
        },
        "id": "bRwX8xXS1qI_"
      },
      "outputs": [],
      "source": [
        "import math\n",
        "\n",
        "class BatchGenerator:\n",
        "    def __init__(self, images, labels, batch_size=128):\n",
        "        assert len(images) == len(labels)\n",
        "        self.index = 0\n",
        "        self.images = images\n",
        "        self.labels = labels\n",
        "        self.batch_size = batch_size\n",
        "        self.num_batches = math.ceil(len(images) / batch_size)\n",
        "\n",
        "    def next(self):\n",
        "        images = self.images[self.index : self.index + self.batch_size]\n",
        "        labels = self.labels[self.index : self.index + self.batch_size]\n",
        "        self.index += self.batch_size\n",
        "        return images, labels"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UZ89mMQQ1qI_"
      },
      "source": [
        "### 훈련 스텝 실행하기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 67,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:34.579548Z",
          "iopub.status.busy": "2022-05-21T00:57:34.579204Z",
          "iopub.status.idle": "2022-05-21T00:57:34.584549Z",
          "shell.execute_reply": "2022-05-21T00:57:34.583770Z"
        },
        "id": "ts_uBtwz1qJA"
      },
      "outputs": [],
      "source": [
        "def one_training_step(model, images_batch, labels_batch):\n",
        "    with tf.GradientTape() as tape:\n",
        "        predictions = model(images_batch)\n",
        "        per_sample_losses = tf.keras.losses.sparse_categorical_crossentropy(\n",
        "            labels_batch, predictions)\n",
        "        average_loss = tf.reduce_mean(per_sample_losses)\n",
        "    gradients = tape.gradient(average_loss, model.weights)\n",
        "    update_weights(gradients, model.weights)\n",
        "    return average_loss"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 68,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:34.587934Z",
          "iopub.status.busy": "2022-05-21T00:57:34.587645Z",
          "iopub.status.idle": "2022-05-21T00:57:34.591908Z",
          "shell.execute_reply": "2022-05-21T00:57:34.591071Z"
        },
        "id": "jaZr2QHV1qJA"
      },
      "outputs": [],
      "source": [
        "learning_rate = 1e-3\n",
        "\n",
        "def update_weights(gradients, weights):\n",
        "    for g, w in zip(gradients, weights):\n",
        "        w.assign_sub(g * learning_rate)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 69,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:34.595476Z",
          "iopub.status.busy": "2022-05-21T00:57:34.595223Z",
          "iopub.status.idle": "2022-05-21T00:57:34.600016Z",
          "shell.execute_reply": "2022-05-21T00:57:34.599071Z"
        },
        "id": "Nn_jpOBZ1qJA"
      },
      "outputs": [],
      "source": [
        "from tensorflow.keras import optimizers\n",
        "\n",
        "optimizer = optimizers.SGD(learning_rate=1e-3)\n",
        "\n",
        "def update_weights(gradients, weights):\n",
        "    optimizer.apply_gradients(zip(gradients, weights))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yQFTzrXG1qJA"
      },
      "source": [
        "### 전체 훈련 루프"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 70,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:34.603512Z",
          "iopub.status.busy": "2022-05-21T00:57:34.603225Z",
          "iopub.status.idle": "2022-05-21T00:57:34.608868Z",
          "shell.execute_reply": "2022-05-21T00:57:34.607981Z"
        },
        "id": "LeD5r6c51qJA"
      },
      "outputs": [],
      "source": [
        "def fit(model, images, labels, epochs, batch_size=128):\n",
        "    for epoch_counter in range(epochs):\n",
        "        print(f\"에포크 {epoch_counter}\")\n",
        "        batch_generator = BatchGenerator(images, labels)\n",
        "        for batch_counter in range(batch_generator.num_batches):\n",
        "            images_batch, labels_batch = batch_generator.next()\n",
        "            loss = one_training_step(model, images_batch, labels_batch)\n",
        "            if batch_counter % 100 == 0:\n",
        "                print(f\"{batch_counter}번째 배치 손실: {loss:.2f}\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 71,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:57:34.612683Z",
          "iopub.status.busy": "2022-05-21T00:57:34.612226Z",
          "iopub.status.idle": "2022-05-21T00:58:10.090355Z",
          "shell.execute_reply": "2022-05-21T00:58:10.089432Z"
        },
        "id": "fLGhrHuT1qJA",
        "outputId": "8b09f163-51fd-43bc-e1f6-d224a4195015",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "에포크 0\n",
            "0번째 배치 손실: 6.79\n",
            "100번째 배치 손실: 2.25\n",
            "200번째 배치 손실: 2.23\n",
            "300번째 배치 손실: 2.09\n",
            "400번째 배치 손실: 2.23\n",
            "에포크 1\n",
            "0번째 배치 손실: 1.92\n",
            "100번째 배치 손실: 1.89\n",
            "200번째 배치 손실: 1.85\n",
            "300번째 배치 손실: 1.70\n",
            "400번째 배치 손실: 1.83\n",
            "에포크 2\n",
            "0번째 배치 손실: 1.59\n",
            "100번째 배치 손실: 1.59\n",
            "200번째 배치 손실: 1.53\n",
            "300번째 배치 손실: 1.42\n",
            "400번째 배치 손실: 1.52\n",
            "에포크 3\n",
            "0번째 배치 손실: 1.33\n",
            "100번째 배치 손실: 1.35\n",
            "200번째 배치 손실: 1.26\n",
            "300번째 배치 손실: 1.21\n",
            "400번째 배치 손실: 1.28\n",
            "에포크 4\n",
            "0번째 배치 손실: 1.13\n",
            "100번째 배치 손실: 1.17\n",
            "200번째 배치 손실: 1.06\n",
            "300번째 배치 손실: 1.05\n",
            "400번째 배치 손실: 1.12\n",
            "에포크 5\n",
            "0번째 배치 손실: 0.99\n",
            "100번째 배치 손실: 1.03\n",
            "200번째 배치 손실: 0.92\n",
            "300번째 배치 손실: 0.93\n",
            "400번째 배치 손실: 1.00\n",
            "에포크 6\n",
            "0번째 배치 손실: 0.88\n",
            "100번째 배치 손실: 0.92\n",
            "200번째 배치 손실: 0.81\n",
            "300번째 배치 손실: 0.84\n",
            "400번째 배치 손실: 0.90\n",
            "에포크 7\n",
            "0번째 배치 손실: 0.79\n",
            "100번째 배치 손실: 0.84\n",
            "200번째 배치 손실: 0.73\n",
            "300번째 배치 손실: 0.77\n",
            "400번째 배치 손실: 0.83\n",
            "에포크 8\n",
            "0번째 배치 손실: 0.73\n",
            "100번째 배치 손실: 0.77\n",
            "200번째 배치 손실: 0.67\n",
            "300번째 배치 손실: 0.71\n",
            "400번째 배치 손실: 0.78\n",
            "에포크 9\n",
            "0번째 배치 손실: 0.68\n",
            "100번째 배치 손실: 0.71\n",
            "200번째 배치 손실: 0.62\n",
            "300번째 배치 손실: 0.67\n",
            "400번째 배치 손실: 0.74\n"
          ]
        }
      ],
      "source": [
        "from tensorflow.keras.datasets import mnist\n",
        "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()\n",
        "\n",
        "train_images = train_images.reshape((60000, 28 * 28))\n",
        "train_images = train_images.astype(\"float32\") / 255\n",
        "test_images = test_images.reshape((10000, 28 * 28))\n",
        "test_images = test_images.astype(\"float32\") / 255\n",
        "\n",
        "fit(model, train_images, train_labels, epochs=10, batch_size=128)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h5TNNZ_E1qJA"
      },
      "source": [
        "### 모델 평가하기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 72,
      "metadata": {
        "execution": {
          "iopub.execute_input": "2022-05-21T00:58:10.094649Z",
          "iopub.status.busy": "2022-05-21T00:58:10.094318Z",
          "iopub.status.idle": "2022-05-21T00:58:10.165292Z",
          "shell.execute_reply": "2022-05-21T00:58:10.164371Z"
        },
        "id": "S4XG6fSU1qJA",
        "outputId": "63d4b563-6b6b-40ba-cc37-beba1e4bcb6d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "정확도: 0.81\n"
          ]
        }
      ],
      "source": [
        "predictions = model(test_images)\n",
        "predictions = predictions.numpy()\n",
        "predicted_labels = np.argmax(predictions, axis=1)\n",
        "matches = predicted_labels == test_labels\n",
        "print(f\"정확도: {matches.mean():.2f}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DSlIQZ5f1qJB"
      },
      "source": [
        "## 요약"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "chapter02_mathematical-building-blocks.i",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "default:Python",
      "language": "python",
      "name": "conda-env-default-py"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.10"
    },
    "accelerator": "GPU",
    "gpuClass": "standard"
  },
  "nbformat": 4,
  "nbformat_minor": 0
}