chapter13_best-practices-for-the-real-world.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dXC-NAhWrH7H"
      },
      "source": [
        "이 노트북은 [케라스 창시자에게 배우는 딥러닝 2판](https://tensorflow.blog/kerasdl2/)의 예제 코드를 담고 있습니다.\n",
        "\n",
        "<table align=\"left\">\n",
        "    <tr>\n",
        "        <td>\n",
        "            <a href=\"https://colab.research.google.com/github/rickiepark/deep-learning-with-python-2nd/blob/main/chapter13_best-practices-for-the-real-world.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
        "        </td>\n",
        "    </tr>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "48rlVGf1rH7J"
      },
      "source": [
        "# 실전 문제 해결을 위한 모범 사례"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AVUS_pV3rH7K"
      },
      "source": [
        "## 모델의 최대 성능을 끌어 내기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WI5xmwxVrH7K"
      },
      "source": [
        "### 하이퍼파라미터 최적화"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bQo7Jfe3rH7K"
      },
      "source": [
        "#### KerasTuner 사용하기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "kdcoHqN8rH7K",
        "outputId": "29a6081d-2ba0-4db3-e2e8-5ddfb3ca4cf7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[?25l     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/129.5 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m129.5/129.5 kB\u001b[0m \u001b[31m3.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h\u001b[?25l     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/950.8 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K     \u001b[91m━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[91m╸\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m522.2/950.8 kB\u001b[0m \u001b[31m15.7 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K     \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[91m╸\u001b[0m \u001b[32m942.1/950.8 kB\u001b[0m \u001b[31m15.9 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m950.8/950.8 kB\u001b[0m \u001b[31m13.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h"
          ]
        }
      ],
      "source": [
        "!pip install keras-tuner -q"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0tMJEdJ8rH7L"
      },
      "source": [
        "**KerasTuner 모델 구축 함수**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "hQmZbvskrH7M"
      },
      "outputs": [],
      "source": [
        "from tensorflow import keras\n",
        "from tensorflow.keras import layers\n",
        "\n",
        "def build_model(hp):\n",
        "    units = hp.Int(name=\"units\", min_value=16, max_value=64, step=16)\n",
        "    model = keras.Sequential([\n",
        "        layers.Dense(units, activation=\"relu\"),\n",
        "        layers.Dense(10, activation=\"softmax\")\n",
        "    ])\n",
        "    optimizer = hp.Choice(name=\"optimizer\", values=[\"rmsprop\", \"adam\"])\n",
        "    model.compile(\n",
        "        optimizer=optimizer,\n",
        "        loss=\"sparse_categorical_crossentropy\",\n",
        "        metrics=[\"accuracy\"])\n",
        "    return model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qIVWG7jnrH7M"
      },
      "source": [
        "**KerasTuner의 `HyperModel`**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "pC1gmBOzrH7N",
        "outputId": "6d5fe647-d7f3-41f2-87d7-5ccf679a4412",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Using TensorFlow backend\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-3-643dcc05e516>:1: DeprecationWarning: `import kerastuner` is deprecated, please use `import keras_tuner`.\n",
            "  import kerastuner as kt\n"
          ]
        }
      ],
      "source": [
        "import kerastuner as kt\n",
        "\n",
        "class SimpleMLP(kt.HyperModel):\n",
        "    def __init__(self, num_classes):\n",
        "        self.num_classes = num_classes\n",
        "\n",
        "    def build(self, hp):\n",
        "        units = hp.Int(name=\"units\", min_value=16, max_value=64, step=16)\n",
        "        model = keras.Sequential([\n",
        "            layers.Dense(units, activation=\"relu\"),\n",
        "            layers.Dense(self.num_classes, activation=\"softmax\")\n",
        "        ])\n",
        "        optimizer = hp.Choice(name=\"optimizer\", values=[\"rmsprop\", \"adam\"])\n",
        "        model.compile(\n",
        "            optimizer=optimizer,\n",
        "            loss=\"sparse_categorical_crossentropy\",\n",
        "            metrics=[\"accuracy\"])\n",
        "        return model\n",
        "\n",
        "hypermodel = SimpleMLP(num_classes=10)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "79npb8nzrH7O"
      },
      "outputs": [],
      "source": [
        "tuner = kt.BayesianOptimization(\n",
        "    build_model,\n",
        "    objective=\"val_accuracy\",\n",
        "    # 코랩에서 정상 실행만 확인하기 위해 최대 탐색 횟수를 100에서 10으로 줄입니다\n",
        "    max_trials=10, # 100\n",
        "    executions_per_trial=2,\n",
        "    directory=\"mnist_kt_test\",\n",
        "    overwrite=True,\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "mmeQ4XirrH7O",
        "outputId": "642ce82e-c2f7-4b42-a06f-a5b893682129",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Search space summary\n",
            "Default search space size: 2\n",
            "units (Int)\n",
            "{'default': None, 'conditions': [], 'min_value': 16, 'max_value': 64, 'step': 16, 'sampling': 'linear'}\n",
            "optimizer (Choice)\n",
            "{'default': 'rmsprop', 'conditions': [], 'values': ['rmsprop', 'adam'], 'ordered': False}\n"
          ]
        }
      ],
      "source": [
        "tuner.search_space_summary()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "bO3eDmIrrH7O",
        "outputId": "e5ef58b3-d12b-4b76-cea0-bb5ca82d96c3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Trial 10 Complete [00h 01m 14s]\n",
            "val_accuracy: 0.9758500158786774\n",
            "\n",
            "Best val_accuracy So Far: 0.9758500158786774\n",
            "Total elapsed time: 00h 12m 40s\n"
          ]
        }
      ],
      "source": [
        "(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()\n",
        "x_train = x_train.reshape((-1, 28 * 28)).astype(\"float32\") / 255\n",
        "x_test = x_test.reshape((-1, 28 * 28)).astype(\"float32\") / 255\n",
        "x_train_full = x_train[:]\n",
        "y_train_full = y_train[:]\n",
        "num_val_samples = 10000\n",
        "x_train, x_val = x_train[:-num_val_samples], x_train[-num_val_samples:]\n",
        "y_train, y_val = y_train[:-num_val_samples], y_train[-num_val_samples:]\n",
        "callbacks = [\n",
        "    keras.callbacks.EarlyStopping(monitor=\"val_loss\", patience=5),\n",
        "]\n",
        "tuner.search(\n",
        "    x_train, y_train,\n",
        "    batch_size=128,\n",
        "    epochs=100,\n",
        "    validation_data=(x_val, y_val),\n",
        "    callbacks=callbacks,\n",
        "    verbose=2,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "svpbAxvfrH7P"
      },
      "source": [
        "**최상의 하이퍼파라미터 설정 확인하기**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "13or5ZBvrH7P"
      },
      "outputs": [],
      "source": [
        "top_n = 4\n",
        "best_hps = tuner.get_best_hyperparameters(top_n)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "id": "LW8KNOSDrH7P"
      },
      "outputs": [],
      "source": [
        "def get_best_epoch(hp):\n",
        "    model = build_model(hp)\n",
        "    callbacks=[\n",
        "        keras.callbacks.EarlyStopping(\n",
        "            monitor=\"val_loss\", mode=\"min\", patience=10)\n",
        "    ]\n",
        "    history = model.fit(\n",
        "        x_train, y_train,\n",
        "        validation_data=(x_val, y_val),\n",
        "        epochs=100,\n",
        "        batch_size=128,\n",
        "        callbacks=callbacks)\n",
        "    val_loss_per_epoch = history.history[\"val_loss\"]\n",
        "    best_epoch = val_loss_per_epoch.index(min(val_loss_per_epoch)) + 1\n",
        "    print(f\"Best epoch: {best_epoch}\")\n",
        "    return best_epoch"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "atgJLN2_rH7P",
        "outputId": "8ed1a94f-9d4d-4c56-ebcb-d3931036e7d0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<keras_tuner.src.engine.hyperparameters.hyperparameters.HyperParameters at 0x7a96f976c070>,\n",
              " <keras_tuner.src.engine.hyperparameters.hyperparameters.HyperParameters at 0x7a96f9a05720>,\n",
              " <keras_tuner.src.engine.hyperparameters.hyperparameters.HyperParameters at 0x7a96f994f460>,\n",
              " <keras_tuner.src.engine.hyperparameters.hyperparameters.HyperParameters at 0x7a9770703ca0>]"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ],
      "source": [
        "best_hps"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "9Wbd5MN1rH7Q",
        "outputId": "bdcc37ad-b715-41c9-a89e-407a0f97c45a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'rmsprop'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 10
        }
      ],
      "source": [
        "best_hps[0].Choice(name=\"optimizer\", values=[\"rmsprop\", \"adam\"])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "fMd-PWX2rH7Q"
      },
      "outputs": [],
      "source": [
        "model = build_model(best_hps[0])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "zTTXtVo6rH7Q"
      },
      "outputs": [],
      "source": [
        "from keras_tuner.engine.hyperparameters import Int"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "BD4-yhoorH7Q",
        "outputId": "23eb6c84-c319-4a44-e60c-1fc4dc3a3bd8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ],
      "source": [
        "best_hps[0]._conditions_are_active(Int(name=\"units\", min_value=0, max_value=1).conditions)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "WGHlc393rH7Q",
        "outputId": "40e3579a-c8df-41ef-92d3-539dce379e44",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "defaultdict(list,\n",
              "            {'units': [Int(name: 'units', min_value: 16, max_value: 64, step: 16, sampling: linear, default: 16)],\n",
              "             'optimizer': [Choice(name: 'optimizer', values: ['rmsprop', 'adam'], ordered: False, default: rmsprop)]})"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ],
      "source": [
        "best_hps[0]._hps"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "8ceR3te6rH7R",
        "outputId": "a612622f-7bdf-40eb-ffb3-5abd04cae168",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ],
      "source": [
        "best_hps[0]._exists('units', [])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "7nmuiOVJrH7R",
        "outputId": "24926bab-cc91-4264-cc90-bcdcdb361bcb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'units': 64, 'optimizer': 'rmsprop'}"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ],
      "source": [
        "best_hps[0].values"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "NoVQocz9rH7R",
        "outputId": "444a9ca9-cb94-4722-e0a8-cd7de99312f5",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "64"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ],
      "source": [
        "best_hps[0].Int(name=\"units\", min_value=0, max_value=1)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "id": "qYPQfGVrrH7R",
        "outputId": "21a4f43c-df3e-4f03-8c2b-a12c5ce58ff2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'name': 'sequential_1',\n",
              " 'layers': [{'module': 'keras.layers',\n",
              "   'class_name': 'Dense',\n",
              "   'config': {'name': 'dense_2',\n",
              "    'trainable': True,\n",
              "    'dtype': 'float32',\n",
              "    'units': 64,\n",
              "    'activation': 'relu',\n",
              "    'use_bias': True,\n",
              "    'kernel_initializer': {'module': 'keras.initializers',\n",
              "     'class_name': 'GlorotUniform',\n",
              "     'config': {'seed': None},\n",
              "     'registered_name': None},\n",
              "    'bias_initializer': {'module': 'keras.initializers',\n",
              "     'class_name': 'Zeros',\n",
              "     'config': {},\n",
              "     'registered_name': None},\n",
              "    'kernel_regularizer': None,\n",
              "    'bias_regularizer': None,\n",
              "    'activity_regularizer': None,\n",
              "    'kernel_constraint': None,\n",
              "    'bias_constraint': None},\n",
              "   'registered_name': None},\n",
              "  {'module': 'keras.layers',\n",
              "   'class_name': 'Dense',\n",
              "   'config': {'name': 'dense_3',\n",
              "    'trainable': True,\n",
              "    'dtype': 'float32',\n",
              "    'units': 10,\n",
              "    'activation': 'softmax',\n",
              "    'use_bias': True,\n",
              "    'kernel_initializer': {'module': 'keras.initializers',\n",
              "     'class_name': 'GlorotUniform',\n",
              "     'config': {'seed': None},\n",
              "     'registered_name': None},\n",
              "    'bias_initializer': {'module': 'keras.initializers',\n",
              "     'class_name': 'Zeros',\n",
              "     'config': {},\n",
              "     'registered_name': None},\n",
              "    'kernel_regularizer': None,\n",
              "    'bias_regularizer': None,\n",
              "    'activity_regularizer': None,\n",
              "    'kernel_constraint': None,\n",
              "    'bias_constraint': None},\n",
              "   'registered_name': None}]}"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ],
      "source": [
        "model.get_config()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "haIQjmkirH7R",
        "outputId": "f0982305-109c-4629-8db4-0039f3ff9a32",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/100\n",
            "391/391 [==============================] - 2s 4ms/step - loss: 0.4213 - accuracy: 0.8859 - val_loss: 0.2411 - val_accuracy: 0.9318\n",
            "Epoch 2/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.2200 - accuracy: 0.9371 - val_loss: 0.1795 - val_accuracy: 0.9511\n",
            "Epoch 3/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.1696 - accuracy: 0.9507 - val_loss: 0.1560 - val_accuracy: 0.9587\n",
            "Epoch 4/100\n",
            "391/391 [==============================] - 2s 4ms/step - loss: 0.1383 - accuracy: 0.9597 - val_loss: 0.1334 - val_accuracy: 0.9626\n",
            "Epoch 5/100\n",
            "391/391 [==============================] - 2s 4ms/step - loss: 0.1173 - accuracy: 0.9660 - val_loss: 0.1262 - val_accuracy: 0.9654\n",
            "Epoch 6/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.1017 - accuracy: 0.9710 - val_loss: 0.1200 - val_accuracy: 0.9661\n",
            "Epoch 7/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0892 - accuracy: 0.9745 - val_loss: 0.1160 - val_accuracy: 0.9676\n",
            "Epoch 8/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0800 - accuracy: 0.9766 - val_loss: 0.1050 - val_accuracy: 0.9703\n",
            "Epoch 9/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0714 - accuracy: 0.9792 - val_loss: 0.1008 - val_accuracy: 0.9720\n",
            "Epoch 10/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0653 - accuracy: 0.9811 - val_loss: 0.1062 - val_accuracy: 0.9710\n",
            "Epoch 11/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0591 - accuracy: 0.9824 - val_loss: 0.0989 - val_accuracy: 0.9713\n",
            "Epoch 12/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0541 - accuracy: 0.9841 - val_loss: 0.0977 - val_accuracy: 0.9725\n",
            "Epoch 13/100\n",
            "391/391 [==============================] - 2s 4ms/step - loss: 0.0495 - accuracy: 0.9859 - val_loss: 0.1010 - val_accuracy: 0.9715\n",
            "Epoch 14/100\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 0.0458 - accuracy: 0.9872 - val_loss: 0.0972 - val_accuracy: 0.9730\n",
            "Epoch 15/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0418 - accuracy: 0.9884 - val_loss: 0.0981 - val_accuracy: 0.9723\n",
            "Epoch 16/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0387 - accuracy: 0.9891 - val_loss: 0.0998 - val_accuracy: 0.9718\n",
            "Epoch 17/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0356 - accuracy: 0.9900 - val_loss: 0.0951 - val_accuracy: 0.9732\n",
            "Epoch 18/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0331 - accuracy: 0.9911 - val_loss: 0.0973 - val_accuracy: 0.9742\n",
            "Epoch 19/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0306 - accuracy: 0.9918 - val_loss: 0.1010 - val_accuracy: 0.9739\n",
            "Epoch 20/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0282 - accuracy: 0.9926 - val_loss: 0.0988 - val_accuracy: 0.9737\n",
            "Epoch 21/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0258 - accuracy: 0.9933 - val_loss: 0.0983 - val_accuracy: 0.9737\n",
            "Epoch 22/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0240 - accuracy: 0.9939 - val_loss: 0.1034 - val_accuracy: 0.9740\n",
            "Epoch 23/100\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 0.0224 - accuracy: 0.9940 - val_loss: 0.1095 - val_accuracy: 0.9720\n",
            "Epoch 24/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0204 - accuracy: 0.9948 - val_loss: 0.1026 - val_accuracy: 0.9750\n",
            "Epoch 25/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0187 - accuracy: 0.9955 - val_loss: 0.1045 - val_accuracy: 0.9744\n",
            "Epoch 26/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0176 - accuracy: 0.9956 - val_loss: 0.1080 - val_accuracy: 0.9724\n",
            "Epoch 27/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0163 - accuracy: 0.9960 - val_loss: 0.1115 - val_accuracy: 0.9724\n",
            "Best epoch: 17\n",
            "Epoch 1/20\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 0.3976 - accuracy: 0.8924\n",
            "Epoch 2/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.2027 - accuracy: 0.9425\n",
            "Epoch 3/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.1553 - accuracy: 0.9559\n",
            "Epoch 4/20\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.1267 - accuracy: 0.9632\n",
            "Epoch 5/20\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 0.1081 - accuracy: 0.9685\n",
            "Epoch 6/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0946 - accuracy: 0.9721\n",
            "Epoch 7/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0838 - accuracy: 0.9760\n",
            "Epoch 8/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0752 - accuracy: 0.9786\n",
            "Epoch 9/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0677 - accuracy: 0.9803\n",
            "Epoch 10/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0616 - accuracy: 0.9826\n",
            "Epoch 11/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0564 - accuracy: 0.9837\n",
            "Epoch 12/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0514 - accuracy: 0.9855\n",
            "Epoch 13/20\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 0.0478 - accuracy: 0.9867\n",
            "Epoch 14/20\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.0439 - accuracy: 0.9876\n",
            "Epoch 15/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0408 - accuracy: 0.9885\n",
            "Epoch 16/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0372 - accuracy: 0.9894\n",
            "Epoch 17/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0349 - accuracy: 0.9905\n",
            "Epoch 18/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0324 - accuracy: 0.9909\n",
            "Epoch 19/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0297 - accuracy: 0.9921\n",
            "Epoch 20/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0278 - accuracy: 0.9927\n",
            "313/313 [==============================] - 1s 2ms/step - loss: 0.0831 - accuracy: 0.9772\n",
            "Epoch 1/100\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 0.4217 - accuracy: 0.8879 - val_loss: 0.2536 - val_accuracy: 0.9257\n",
            "Epoch 2/100\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 0.2199 - accuracy: 0.9360 - val_loss: 0.1802 - val_accuracy: 0.9495\n",
            "Epoch 3/100\n",
            "391/391 [==============================] - 2s 4ms/step - loss: 0.1720 - accuracy: 0.9507 - val_loss: 0.1507 - val_accuracy: 0.9562\n",
            "Epoch 4/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.1426 - accuracy: 0.9591 - val_loss: 0.1331 - val_accuracy: 0.9619\n",
            "Epoch 5/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.1216 - accuracy: 0.9656 - val_loss: 0.1184 - val_accuracy: 0.9662\n",
            "Epoch 6/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.1060 - accuracy: 0.9691 - val_loss: 0.1182 - val_accuracy: 0.9665\n",
            "Epoch 7/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0935 - accuracy: 0.9730 - val_loss: 0.1101 - val_accuracy: 0.9687\n",
            "Epoch 8/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0834 - accuracy: 0.9759 - val_loss: 0.1029 - val_accuracy: 0.9713\n",
            "Epoch 9/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0744 - accuracy: 0.9782 - val_loss: 0.1013 - val_accuracy: 0.9707\n",
            "Epoch 10/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0674 - accuracy: 0.9811 - val_loss: 0.0972 - val_accuracy: 0.9714\n",
            "Epoch 11/100\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 0.0613 - accuracy: 0.9823 - val_loss: 0.0954 - val_accuracy: 0.9727\n",
            "Epoch 12/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0566 - accuracy: 0.9834 - val_loss: 0.0952 - val_accuracy: 0.9715\n",
            "Epoch 13/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0513 - accuracy: 0.9847 - val_loss: 0.0935 - val_accuracy: 0.9742\n",
            "Epoch 14/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0475 - accuracy: 0.9868 - val_loss: 0.0951 - val_accuracy: 0.9724\n",
            "Epoch 15/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0443 - accuracy: 0.9875 - val_loss: 0.0911 - val_accuracy: 0.9747\n",
            "Epoch 16/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0404 - accuracy: 0.9885 - val_loss: 0.0950 - val_accuracy: 0.9737\n",
            "Epoch 17/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0373 - accuracy: 0.9895 - val_loss: 0.0915 - val_accuracy: 0.9736\n",
            "Epoch 18/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0343 - accuracy: 0.9906 - val_loss: 0.0965 - val_accuracy: 0.9728\n",
            "Epoch 19/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0316 - accuracy: 0.9915 - val_loss: 0.0947 - val_accuracy: 0.9740\n",
            "Epoch 20/100\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 0.0292 - accuracy: 0.9921 - val_loss: 0.0956 - val_accuracy: 0.9748\n",
            "Epoch 21/100\n",
            "391/391 [==============================] - 2s 4ms/step - loss: 0.0272 - accuracy: 0.9928 - val_loss: 0.0958 - val_accuracy: 0.9742\n",
            "Epoch 22/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0249 - accuracy: 0.9934 - val_loss: 0.1003 - val_accuracy: 0.9729\n",
            "Epoch 23/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0232 - accuracy: 0.9941 - val_loss: 0.0998 - val_accuracy: 0.9739\n",
            "Epoch 24/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0213 - accuracy: 0.9946 - val_loss: 0.1040 - val_accuracy: 0.9737\n",
            "Epoch 25/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0199 - accuracy: 0.9954 - val_loss: 0.1063 - val_accuracy: 0.9735\n",
            "Best epoch: 15\n",
            "Epoch 1/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0260 - accuracy: 0.9930\n",
            "Epoch 2/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0237 - accuracy: 0.9937\n",
            "Epoch 3/18\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 0.0225 - accuracy: 0.9940\n",
            "Epoch 4/18\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.0204 - accuracy: 0.9949\n",
            "Epoch 5/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0191 - accuracy: 0.9956\n",
            "Epoch 6/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0177 - accuracy: 0.9955\n",
            "Epoch 7/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0163 - accuracy: 0.9960\n",
            "Epoch 8/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0152 - accuracy: 0.9963\n",
            "Epoch 9/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0139 - accuracy: 0.9970\n",
            "Epoch 10/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0130 - accuracy: 0.9972\n",
            "Epoch 11/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0122 - accuracy: 0.9970\n",
            "Epoch 12/18\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 0.0109 - accuracy: 0.9977\n",
            "Epoch 13/18\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.0102 - accuracy: 0.9978\n",
            "Epoch 14/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0091 - accuracy: 0.9983\n",
            "Epoch 15/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0086 - accuracy: 0.9984\n",
            "Epoch 16/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0077 - accuracy: 0.9987\n",
            "Epoch 17/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0074 - accuracy: 0.9987\n",
            "Epoch 18/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0066 - accuracy: 0.9988\n",
            "313/313 [==============================] - 1s 3ms/step - loss: 0.0996 - accuracy: 0.9763\n",
            "Epoch 1/100\n",
            "391/391 [==============================] - 2s 4ms/step - loss: 0.4269 - accuracy: 0.8856 - val_loss: 0.2321 - val_accuracy: 0.9352\n",
            "Epoch 2/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.2174 - accuracy: 0.9388 - val_loss: 0.1730 - val_accuracy: 0.9511\n",
            "Epoch 3/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.1671 - accuracy: 0.9514 - val_loss: 0.1489 - val_accuracy: 0.9577\n",
            "Epoch 4/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.1368 - accuracy: 0.9602 - val_loss: 0.1298 - val_accuracy: 0.9624\n",
            "Epoch 5/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.1151 - accuracy: 0.9668 - val_loss: 0.1156 - val_accuracy: 0.9652\n",
            "Epoch 6/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0983 - accuracy: 0.9717 - val_loss: 0.1035 - val_accuracy: 0.9707\n",
            "Epoch 7/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0864 - accuracy: 0.9749 - val_loss: 0.1069 - val_accuracy: 0.9683\n",
            "Epoch 8/100\n",
            "391/391 [==============================] - 2s 4ms/step - loss: 0.0766 - accuracy: 0.9785 - val_loss: 0.1013 - val_accuracy: 0.9707\n",
            "Epoch 9/100\n",
            "391/391 [==============================] - 2s 4ms/step - loss: 0.0681 - accuracy: 0.9805 - val_loss: 0.1033 - val_accuracy: 0.9693\n",
            "Epoch 10/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0621 - accuracy: 0.9823 - val_loss: 0.0935 - val_accuracy: 0.9723\n",
            "Epoch 11/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0558 - accuracy: 0.9841 - val_loss: 0.0903 - val_accuracy: 0.9740\n",
            "Epoch 12/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0512 - accuracy: 0.9854 - val_loss: 0.0913 - val_accuracy: 0.9734\n",
            "Epoch 13/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0469 - accuracy: 0.9870 - val_loss: 0.0913 - val_accuracy: 0.9735\n",
            "Epoch 14/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0427 - accuracy: 0.9885 - val_loss: 0.0965 - val_accuracy: 0.9722\n",
            "Epoch 15/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0391 - accuracy: 0.9895 - val_loss: 0.0891 - val_accuracy: 0.9730\n",
            "Epoch 16/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0360 - accuracy: 0.9900 - val_loss: 0.0980 - val_accuracy: 0.9713\n",
            "Epoch 17/100\n",
            "391/391 [==============================] - 2s 4ms/step - loss: 0.0329 - accuracy: 0.9913 - val_loss: 0.0885 - val_accuracy: 0.9748\n",
            "Epoch 18/100\n",
            "391/391 [==============================] - 2s 4ms/step - loss: 0.0298 - accuracy: 0.9923 - val_loss: 0.0946 - val_accuracy: 0.9737\n",
            "Epoch 19/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0280 - accuracy: 0.9926 - val_loss: 0.0960 - val_accuracy: 0.9745\n",
            "Epoch 20/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0260 - accuracy: 0.9934 - val_loss: 0.0911 - val_accuracy: 0.9746\n",
            "Epoch 21/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0238 - accuracy: 0.9939 - val_loss: 0.0919 - val_accuracy: 0.9748\n",
            "Epoch 22/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0214 - accuracy: 0.9947 - val_loss: 0.0971 - val_accuracy: 0.9741\n",
            "Epoch 23/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0201 - accuracy: 0.9950 - val_loss: 0.0969 - val_accuracy: 0.9742\n",
            "Epoch 24/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0182 - accuracy: 0.9958 - val_loss: 0.0991 - val_accuracy: 0.9741\n",
            "Epoch 25/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0168 - accuracy: 0.9959 - val_loss: 0.0995 - val_accuracy: 0.9745\n",
            "Epoch 26/100\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 0.0156 - accuracy: 0.9965 - val_loss: 0.1009 - val_accuracy: 0.9742\n",
            "Epoch 27/100\n",
            "391/391 [==============================] - 2s 4ms/step - loss: 0.0144 - accuracy: 0.9968 - val_loss: 0.1041 - val_accuracy: 0.9740\n",
            "Best epoch: 17\n",
            "Epoch 1/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0063 - accuracy: 0.9989\n",
            "Epoch 2/20\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 0.0057 - accuracy: 0.9991\n",
            "Epoch 3/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0051 - accuracy: 0.9993\n",
            "Epoch 4/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0046 - accuracy: 0.9993\n",
            "Epoch 5/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0044 - accuracy: 0.9993\n",
            "Epoch 6/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0038 - accuracy: 0.9994\n",
            "Epoch 7/20\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.0036 - accuracy: 0.9995\n",
            "Epoch 8/20\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 0.0032 - accuracy: 0.9996\n",
            "Epoch 9/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0028 - accuracy: 0.9997\n",
            "Epoch 10/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0027 - accuracy: 0.9996\n",
            "Epoch 11/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0023 - accuracy: 0.9997\n",
            "Epoch 12/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0021 - accuracy: 0.9998\n",
            "Epoch 13/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0020 - accuracy: 0.9998\n",
            "Epoch 14/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0017 - accuracy: 0.9998\n",
            "Epoch 15/20\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 0.0016 - accuracy: 0.9998\n",
            "Epoch 16/20\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 0.0014 - accuracy: 0.9999\n",
            "Epoch 17/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0012 - accuracy: 0.9999\n",
            "Epoch 18/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 0.0012 - accuracy: 0.9999\n",
            "Epoch 19/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 9.1361e-04 - accuracy: 0.9999\n",
            "Epoch 20/20\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 8.4428e-04 - accuracy: 0.9999\n",
            "313/313 [==============================] - 1s 2ms/step - loss: 0.1314 - accuracy: 0.9752\n",
            "Epoch 1/100\n",
            "391/391 [==============================] - 2s 4ms/step - loss: 0.4621 - accuracy: 0.8737 - val_loss: 0.2313 - val_accuracy: 0.9373\n",
            "Epoch 2/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.2151 - accuracy: 0.9390 - val_loss: 0.1744 - val_accuracy: 0.9530\n",
            "Epoch 3/100\n",
            "391/391 [==============================] - 2s 4ms/step - loss: 0.1654 - accuracy: 0.9531 - val_loss: 0.1444 - val_accuracy: 0.9603\n",
            "Epoch 4/100\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 0.1338 - accuracy: 0.9616 - val_loss: 0.1287 - val_accuracy: 0.9644\n",
            "Epoch 5/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.1129 - accuracy: 0.9675 - val_loss: 0.1170 - val_accuracy: 0.9655\n",
            "Epoch 6/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0967 - accuracy: 0.9723 - val_loss: 0.1082 - val_accuracy: 0.9683\n",
            "Epoch 7/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0841 - accuracy: 0.9758 - val_loss: 0.1018 - val_accuracy: 0.9699\n",
            "Epoch 8/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0744 - accuracy: 0.9787 - val_loss: 0.0996 - val_accuracy: 0.9700\n",
            "Epoch 9/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0664 - accuracy: 0.9812 - val_loss: 0.0931 - val_accuracy: 0.9719\n",
            "Epoch 10/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0596 - accuracy: 0.9826 - val_loss: 0.0939 - val_accuracy: 0.9717\n",
            "Epoch 11/100\n",
            "391/391 [==============================] - 2s 4ms/step - loss: 0.0534 - accuracy: 0.9847 - val_loss: 0.0886 - val_accuracy: 0.9737\n",
            "Epoch 12/100\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 0.0484 - accuracy: 0.9864 - val_loss: 0.0874 - val_accuracy: 0.9738\n",
            "Epoch 13/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0429 - accuracy: 0.9878 - val_loss: 0.0900 - val_accuracy: 0.9731\n",
            "Epoch 14/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0392 - accuracy: 0.9891 - val_loss: 0.0898 - val_accuracy: 0.9734\n",
            "Epoch 15/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0347 - accuracy: 0.9908 - val_loss: 0.0869 - val_accuracy: 0.9754\n",
            "Epoch 16/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0323 - accuracy: 0.9913 - val_loss: 0.0891 - val_accuracy: 0.9727\n",
            "Epoch 17/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0290 - accuracy: 0.9924 - val_loss: 0.0888 - val_accuracy: 0.9751\n",
            "Epoch 18/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0254 - accuracy: 0.9937 - val_loss: 0.0904 - val_accuracy: 0.9750\n",
            "Epoch 19/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0236 - accuracy: 0.9943 - val_loss: 0.0924 - val_accuracy: 0.9737\n",
            "Epoch 20/100\n",
            "391/391 [==============================] - 2s 4ms/step - loss: 0.0212 - accuracy: 0.9953 - val_loss: 0.0889 - val_accuracy: 0.9753\n",
            "Epoch 21/100\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 0.0198 - accuracy: 0.9954 - val_loss: 0.0910 - val_accuracy: 0.9750\n",
            "Epoch 22/100\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.0176 - accuracy: 0.9959 - val_loss: 0.0944 - val_accuracy: 0.9754\n",
            "Epoch 23/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0160 - accuracy: 0.9965 - val_loss: 0.0935 - val_accuracy: 0.9735\n",
            "Epoch 24/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0140 - accuracy: 0.9974 - val_loss: 0.0964 - val_accuracy: 0.9742\n",
            "Epoch 25/100\n",
            "391/391 [==============================] - 1s 4ms/step - loss: 0.0130 - accuracy: 0.9973 - val_loss: 0.1012 - val_accuracy: 0.9737\n",
            "Best epoch: 15\n",
            "Epoch 1/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 7.3676e-04 - accuracy: 1.0000\n",
            "Epoch 2/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 6.7955e-04 - accuracy: 1.0000\n",
            "Epoch 3/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 5.5185e-04 - accuracy: 1.0000\n",
            "Epoch 4/18\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 5.6912e-04 - accuracy: 1.0000\n",
            "Epoch 5/18\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 4.6914e-04 - accuracy: 1.0000\n",
            "Epoch 6/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 4.0728e-04 - accuracy: 1.0000\n",
            "Epoch 7/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 3.7213e-04 - accuracy: 1.0000\n",
            "Epoch 8/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 3.3281e-04 - accuracy: 1.0000\n",
            "Epoch 9/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 3.1870e-04 - accuracy: 1.0000\n",
            "Epoch 10/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 2.9429e-04 - accuracy: 1.0000\n",
            "Epoch 11/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 2.6402e-04 - accuracy: 1.0000\n",
            "Epoch 12/18\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 2.5543e-04 - accuracy: 1.0000\n",
            "Epoch 13/18\n",
            "469/469 [==============================] - 2s 4ms/step - loss: 2.3497e-04 - accuracy: 1.0000\n",
            "Epoch 14/18\n",
            "469/469 [==============================] - 2s 3ms/step - loss: 2.1791e-04 - accuracy: 1.0000\n",
            "Epoch 15/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 2.0572e-04 - accuracy: 1.0000\n",
            "Epoch 16/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 2.0051e-04 - accuracy: 1.0000\n",
            "Epoch 17/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 1.8811e-04 - accuracy: 1.0000\n",
            "Epoch 18/18\n",
            "469/469 [==============================] - 1s 3ms/step - loss: 1.7941e-04 - accuracy: 1.0000\n",
            "313/313 [==============================] - 1s 2ms/step - loss: 0.1388 - accuracy: 0.9764\n"
          ]
        }
      ],
      "source": [
        "def get_best_trained_model(hp):\n",
        "    best_epoch = get_best_epoch(hp)\n",
        "    model.fit(\n",
        "        x_train_full, y_train_full,\n",
        "        batch_size=128, epochs=int(best_epoch * 1.2))\n",
        "    return model\n",
        "\n",
        "best_models = []\n",
        "for hp in best_hps:\n",
        "    model = get_best_trained_model(hp)\n",
        "    model.evaluate(x_test, y_test)\n",
        "    best_models.append(model)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "8iXwwUA3rH7R"
      },
      "outputs": [],
      "source": [
        "best_models = tuner.get_best_models(top_n)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xcb-GhjHrH7S"
      },
      "source": [
        "#### 올바른 검색 공간을 만드는 기술"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q29-ZdXerH7S"
      },
      "source": [
        "#### 하이퍼파라미터 튜닝의 미래: 자동화된 머신 러닝"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KvZL8T66rH7S"
      },
      "source": [
        "### 모델 앙상블"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rW2elH73rH7S"
      },
      "source": [
        "## 대규모 모델 훈련하기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1zmKepQDrH7S"
      },
      "source": [
        "### 혼합 정밀도로 GPU에서 훈련 속도 높이기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FiTxOStwrH7S"
      },
      "source": [
        "#### 부동 소수점 정밀도 이해하기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "UVOLEGDprH7S",
        "outputId": "d8aa2f42-6ce9-4f48-a7db-47cdb5da6748",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tf.float64"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ],
      "source": [
        "import tensorflow as tf\n",
        "import numpy as np\n",
        "np_array = np.zeros((2, 2))\n",
        "tf_tensor = tf.convert_to_tensor(np_array)\n",
        "tf_tensor.dtype"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "fDY0inFkrH7S",
        "outputId": "7584d666-c664-4ef3-ba87-339c049e8efc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tf.float32"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ],
      "source": [
        "np_array = np.zeros((2, 2))\n",
        "tf_tensor = tf.convert_to_tensor(np_array, dtype=\"float32\")\n",
        "tf_tensor.dtype"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oxm1P30GrH7S"
      },
      "source": [
        "#### 혼합 정밀도로 훈련하기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "id": "L_6pGIwyrH7T"
      },
      "outputs": [],
      "source": [
        "from tensorflow import keras\n",
        "keras.mixed_precision.set_global_policy(\"mixed_float16\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "puO6Pg7MrH7T"
      },
      "source": [
        "### 다중 GPU 훈련"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "971AzBmmrH7T"
      },
      "source": [
        "#### 두 개 이상의 GPU 활용하기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bC_21YWarH7T"
      },
      "source": [
        "#### 단일 호스트, 다중 장치 동기 훈련"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "epTe-LuIrH7T"
      },
      "source": [
        "### TPU 훈련"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X401QfaSrH7T"
      },
      "source": [
        "#### 구글 코랩에서 TPU 사용하기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pEf-ix3IrH7T"
      },
      "source": [
        "#### 스텝 융합을 활용하여 TPU 활용도 높이기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wPo-xmzDrH7T"
      },
      "source": [
        "## 요약"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "chapter13_best-practices-for-the-real-world.i",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "default:Python",
      "language": "python",
      "name": "conda-env-default-py"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.10"
    },
    "accelerator": "GPU",
    "gpuClass": "standard"
  },
  "nbformat": 4,
  "nbformat_minor": 0
}