12_custom_models_and_training_with_tensorflow.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C7sjDEDiAxrK"
      },
      "source": [
        "**12장 – 텐서플로를 사용한 사용자 정의 모델과 훈련**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YUAFnQdfAxrM"
      },
      "source": [
        "_이 노트북에는 12장의 모든 샘플 코드와 연습 문제에 대한 해답, 부록 C의 코드 예제가 포함되어 있습니다_."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jX7ukQPOAxrM"
      },
      "source": [
        "<table align=\"left\">\n",
        "  <td>\n",
        "    <a href=\"https://colab.research.google.com/github/rickiepark/handson-ml3/blob/main/12_custom_models_and_training_with_tensorflow.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sFcMpxP2AxrN",
        "tags": []
      },
      "source": [
        "# 설정"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mwy7Yxo_AxrN"
      },
      "source": [
        "이 프로젝트에는 Python 3.7 이상이 필요합니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "xNKQVwanAxrN"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "assert sys.version_info >= (3, 7)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C1IOdJ7mAxrO"
      },
      "source": [
        "그리고 TensorFlow ≥ 2.8:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "0Ddo9OPsAxrO"
      },
      "outputs": [],
      "source": [
        "from packaging import version\n",
        "import tensorflow as tf\n",
        "\n",
        "assert version.parse(tf.__version__) >= version.parse(\"2.8.0\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JN8_1_vCAxrP",
        "outputId": "44c16aab-5f46-4e69-fd91-32b0b6707967"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "debconf: unable to initialize frontend: Dialog\n",
            "debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 78, <> line 1.)\n",
            "debconf: falling back to frontend: Readline\n",
            "debconf: unable to initialize frontend: Readline\n",
            "debconf: (This frontend requires a controlling tty.)\n",
            "debconf: falling back to frontend: Teletype\n",
            "dpkg-preconfigure: unable to re-open stdin: \n",
            "Selecting previously unselected package fonts-nanum.\n",
            "(Reading database ... 121752 files and directories currently installed.)\n",
            "Preparing to unpack .../fonts-nanum_20200506-1_all.deb ...\n",
            "Unpacking fonts-nanum (20200506-1) ...\n",
            "Setting up fonts-nanum (20200506-1) ...\n",
            "Processing triggers for fontconfig (2.13.1-4.2ubuntu5) ...\n"
          ]
        }
      ],
      "source": [
        "import sys\n",
        "# 코랩의 경우 나눔 폰트를 설치합니다.\n",
        "if 'google.colab' in sys.modules:\n",
        "    !sudo apt-get -qq -y install fonts-nanum\n",
        "    import matplotlib.font_manager as fm\n",
        "    font_files = fm.findSystemFonts(fontpaths=['/usr/share/fonts/truetype/nanum'])\n",
        "    for fpath in font_files:\n",
        "        fm.fontManager.addfont(fpath)\n",
        "\n",
        "# 나눔 폰트를 사용합니다.\n",
        "import matplotlib\n",
        "\n",
        "matplotlib.rc('font', family='NanumBarunGothic')\n",
        "matplotlib.rcParams['axes.unicode_minus'] = False"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y5lX_jAOAxrP"
      },
      "source": [
        "## 넘파이처럼 텐서플로 사용하기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PyPC0lBMAxrP"
      },
      "source": [
        "### 텐서 및 연산"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yiDDeZuHAxrQ"
      },
      "source": [
        "#### 텐서"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "V1OI3MGjAxrQ",
        "outputId": "a0f8f443-9e3a-472a-dc64-1c8806aaed57"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3), dtype=float32, numpy=\n",
              "array([[1., 2., 3.],\n",
              "       [4., 5., 6.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ],
      "source": [
        "t = tf.constant([[1., 2., 3.], [4., 5., 6.]]) # 행렬\n",
        "t"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "U5M77FruAxrQ",
        "outputId": "ba962d7e-1757-43e7-8048-009160a08086"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TensorShape([2, 3])"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ],
      "source": [
        "t.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CbvAXuawAxrR",
        "outputId": "67e36f35-556c-4e22-e5ea-02f25bc774bd"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tf.float32"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ],
      "source": [
        "t.dtype"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DeWM0-UNAxrR"
      },
      "source": [
        "#### 인덱싱"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_8b8zwp0AxrR",
        "outputId": "877bd8fb-050d-421a-a6a1-cd9d9fe20690"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=float32, numpy=\n",
              "array([[2., 3.],\n",
              "       [5., 6.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ],
      "source": [
        "t[:, 1:]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zr6WTVXWAxrR",
        "outputId": "f71dca92-75f0-4cde-bcd5-adcc274c4039"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 1), dtype=float32, numpy=\n",
              "array([[2.],\n",
              "       [5.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ],
      "source": [
        "t[..., 1, tf.newaxis]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r8gzp0IuAxrR"
      },
      "source": [
        "#### 연산"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "A2MP_j5ZAxrS",
        "outputId": "aa2a7036-de41-4d2c-88f5-8a46489d48b0"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3), dtype=float32, numpy=\n",
              "array([[11., 12., 13.],\n",
              "       [14., 15., 16.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ],
      "source": [
        "t + 10"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1pJxYGP_AxrS",
        "outputId": "8adf551b-7ad7-460f-eab3-13fc78d6e39e"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3), dtype=float32, numpy=\n",
              "array([[ 1.,  4.,  9.],\n",
              "       [16., 25., 36.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ],
      "source": [
        "tf.square(t)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mcLo21_iAxrS",
        "outputId": "3ccda071-c96d-46eb-9cf0-0db3d603fb02"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=float32, numpy=\n",
              "array([[14., 32.],\n",
              "       [32., 77.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ],
      "source": [
        "t @ tf.transpose(t)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WkBUbZ7hAxrS"
      },
      "source": [
        "#### 스칼라"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "omn-bldFAxrS",
        "outputId": "e827f26b-1285-4cc7-f02b-8bbdb18358ec"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=42>"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ],
      "source": [
        "tf.constant(42)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V46tEuLXAxrS"
      },
      "source": [
        "#### 케라스의 저수준 API"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Un53vnAOAxrT"
      },
      "source": [
        "여전히 케라스 저수준 API를 사용하는 코드를 실행할 수 있습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "L82N01k2AxrT",
        "outputId": "403454f8-65cd-4b64-bf89-93527bcfd7b3"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              "array([[11., 26.],\n",
              "       [14., 35.],\n",
              "       [19., 46.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ],
      "source": [
        "K = tf.keras.backend\n",
        "K.square(K.transpose(t)) + 10"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qkoesP6uAxrT"
      },
      "source": [
        "하지만 케라스는 더 이상 멀티 백엔드를 지원하지 않으므로 대신 TF의 저수준 API를 직접 사용해야 합니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gBDCe42PAxrT",
        "outputId": "a070702c-3ac3-411f-9d50-9caa509384ff"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              "array([[11., 26.],\n",
              "       [14., 35.],\n",
              "       [19., 46.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ],
      "source": [
        "tf.square(tf.transpose(t)) + 10"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lLBJR7srAxrT"
      },
      "source": [
        "### 텐서와 넘파이"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fi4jcBHHAxrT",
        "outputId": "ca79bef3-f8e1-481e-ec33-bfa8c3cbe826"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3,), dtype=float64, numpy=array([2., 4., 5.])>"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ],
      "source": [
        "import numpy as np\n",
        "\n",
        "a = np.array([2., 4., 5.])\n",
        "tf.constant(a)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DfgBFTIlAxrT",
        "outputId": "36dfae73-2006-46e4-8f7d-8fcdf97be55a"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[1., 2., 3.],\n",
              "       [4., 5., 6.]], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ],
      "source": [
        "t.numpy()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JBXYk1dXAxrT",
        "outputId": "b30df2f7-8f5b-4f99-9c8d-8e1070c5f276"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[1., 2., 3.],\n",
              "       [4., 5., 6.]], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ],
      "source": [
        "np.array(t)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "E555f4-eAxrU",
        "outputId": "f2ae468e-a1e4-4bd4-d1c8-a2ec0dbd4952"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3,), dtype=float64, numpy=array([ 4., 16., 25.])>"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ],
      "source": [
        "tf.square(a)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "u3zp3ymEAxrU",
        "outputId": "aafd529b-f2c6-45bf-8c3a-460458573ad9"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 1.,  4.,  9.],\n",
              "       [16., 25., 36.]], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ],
      "source": [
        "np.square(t)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bkrb8RMGAxrU"
      },
      "source": [
        "### 타입 변환"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1T2zObbgAxrU",
        "outputId": "2869cf23-8448-487b-8b56-d5d7dc65300e"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "cannot compute AddV2 as input #1(zero-based) was expected to be a float tensor but is a int32 tensor [Op:AddV2] name: \n"
          ]
        }
      ],
      "source": [
        "try:\n",
        "    tf.constant(2.0) + tf.constant(40)\n",
        "except tf.errors.InvalidArgumentError as ex:\n",
        "    print(ex)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "REJyTlwkAxrU",
        "outputId": "75bc3efa-b32e-4028-f582-ee93ea83f4cb"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "cannot compute AddV2 as input #1(zero-based) was expected to be a float tensor but is a double tensor [Op:AddV2] name: \n"
          ]
        }
      ],
      "source": [
        "try:\n",
        "    tf.constant(2.0) + tf.constant(40., dtype=tf.float64)\n",
        "except tf.errors.InvalidArgumentError as ex:\n",
        "    print(ex)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "voj491m-AxrU",
        "outputId": "97c2c8d7-b356-4993-cd7d-98507372c2cf"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=42.0>"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ],
      "source": [
        "t2 = tf.constant(40., dtype=tf.float64)\n",
        "tf.constant(2.0) + tf.cast(t2, tf.float32)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K6E_xZ7FAxrU"
      },
      "source": [
        "### 변수"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ojTQabFEAxrV",
        "outputId": "869b4cba-bc9f-49fa-a3bb-f002eecd502a"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Variable 'Variable:0' shape=(2, 3) dtype=float32, numpy=\n",
              "array([[1., 2., 3.],\n",
              "       [4., 5., 6.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ],
      "source": [
        "v = tf.Variable([[1., 2., 3.], [4., 5., 6.]])\n",
        "v"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zk6PmMzbAxrV",
        "outputId": "8d4941e0-e237-4a47-ff3c-37caa6c90e1f"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Variable 'UnreadVariable' shape=(2, 3) dtype=float32, numpy=\n",
              "array([[ 2.,  4.,  6.],\n",
              "       [ 8., 10., 12.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ],
      "source": [
        "v.assign(2 * v)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ydDSJtkDAxrV",
        "outputId": "676c9f23-4ced-4214-eabd-c5722ff8838a"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Variable 'UnreadVariable' shape=(2, 3) dtype=float32, numpy=\n",
              "array([[ 2., 42.,  6.],\n",
              "       [ 8., 10., 12.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ],
      "source": [
        "v[0, 1].assign(42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5GbX-IxqAxrV",
        "outputId": "861a63b1-7fcc-4499-fc29-c925c0fe6536",
        "tags": []
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Variable 'UnreadVariable' shape=(2, 3) dtype=float32, numpy=\n",
              "array([[ 2., 42.,  0.],\n",
              "       [ 8., 10.,  1.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ],
      "source": [
        "v[:, 2].assign([0., 1.])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4OX5vh4rAxrV",
        "outputId": "c32bf808-b867-499d-dffe-68591b914f98"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Variable 'UnreadVariable' shape=(2, 3) dtype=float32, numpy=\n",
              "array([[100.,  42.,   0.],\n",
              "       [  8.,  10., 200.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ],
      "source": [
        "v.scatter_nd_update(\n",
        "    indices=[[0, 0], [1, 2]], updates=[100., 200.])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EA-7BthwAxrV",
        "outputId": "5ca94099-31a3-4c4b-a48e-0033332ac77d"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Variable 'UnreadVariable' shape=(2, 3) dtype=float32, numpy=\n",
              "array([[4., 5., 6.],\n",
              "       [1., 2., 3.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 28
        }
      ],
      "source": [
        "# 추가 코드 - scatter_update() 사용 방법을 보여줍니다.\n",
        "sparse_delta = tf.IndexedSlices(values=[[1., 2., 3.], [4., 5., 6.]],\n",
        "                                indices=[1, 0])\n",
        "v.scatter_update(sparse_delta)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cXN3oYYjAxrV",
        "outputId": "59509d43-e918-4e70-f4ac-3e36e06bb96b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "'ResourceVariable' object does not support item assignment\n"
          ]
        }
      ],
      "source": [
        "try:\n",
        "    v[1] = [7., 8., 9.]\n",
        "except TypeError as ex:\n",
        "    print(ex)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d7G6WFDrAxrW"
      },
      "source": [
        "### 문자열"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zD1fDn0hAxrW"
      },
      "source": [
        "이 섹션과 이후 섹션의 코드는 부록 C에 해당합니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GsDN85PRAxrW",
        "outputId": "87fd515e-5f52-40ba-e796-a1d42b3dedfd"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=string, numpy=b'hello world'>"
            ]
          },
          "metadata": {},
          "execution_count": 30
        }
      ],
      "source": [
        "tf.constant(b\"hello world\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bSJynyadAxrW",
        "outputId": "7663e449-83ce-4e24-cb94-04fdf4c2ede7"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=string, numpy=b'caf\\xc3\\xa9'>"
            ]
          },
          "metadata": {},
          "execution_count": 31
        }
      ],
      "source": [
        "tf.constant(\"café\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-Xy-h4QkAxrW",
        "outputId": "ca670c20-a51e-4451-da95-37d94839aae1"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4,), dtype=int32, numpy=array([ 99,  97, 102, 233], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 32
        }
      ],
      "source": [
        "u = tf.constant([ord(c) for c in \"café\"])\n",
        "u"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rQL2muPVAxrW",
        "outputId": "79701799-ddb4-40a4-8ec3-a7259e743f61"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=4>"
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ],
      "source": [
        "b = tf.strings.unicode_encode(u, \"UTF-8\")\n",
        "tf.strings.length(b, unit=\"UTF8_CHAR\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2YvVp9bSAxrW",
        "outputId": "16ed86ea-abfb-4e3c-fdbf-efdc0c0e67ce"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4,), dtype=int32, numpy=array([ 99,  97, 102, 233], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 34
        }
      ],
      "source": [
        "tf.strings.unicode_decode(b, \"UTF-8\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6RpxZM16AxrW"
      },
      "source": [
        "### 다른 데이터 구조"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RI0_AWrFAxrW"
      },
      "source": [
        "이 섹션의 코드는 부록 C에 있습니다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gZNHPgBiAxrX"
      },
      "source": [
        "#### 문자열 배열"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4jMNXDJoAxrX",
        "outputId": "af8b6e1c-dc1e-422d-a4e1-abe269710f5a"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=string, numpy=b'hello world'>"
            ]
          },
          "metadata": {},
          "execution_count": 35
        }
      ],
      "source": [
        "tf.constant(b\"hello world\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "m2MTywzxAxrX",
        "outputId": "7be03bef-cb6b-43c7-9011-8d1be05680c1"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=string, numpy=b'caf\\xc3\\xa9'>"
            ]
          },
          "metadata": {},
          "execution_count": 36
        }
      ],
      "source": [
        "tf.constant(\"café\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 37,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Zi5cZWcHAxrX",
        "outputId": "d79f469d-188f-4f53-eec7-44d93c32cdf7"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4,), dtype=int32, numpy=array([ 99,  97, 102, 233], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 37
        }
      ],
      "source": [
        "u = tf.constant([ord(c) for c in \"café\"])\n",
        "u"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Mhgz8OoxAxrX",
        "outputId": "81793822-4387-47a2-bf5c-0eb19d884aab"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=string, numpy=b'caf\\xc3\\xa9'>"
            ]
          },
          "metadata": {},
          "execution_count": 38
        }
      ],
      "source": [
        "b = tf.strings.unicode_encode(u, \"UTF-8\")\n",
        "b"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 39,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "M_hZqjvnAxrX",
        "outputId": "ec54f994-81ed-4528-922a-b95961a389d0"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=4>"
            ]
          },
          "metadata": {},
          "execution_count": 39
        }
      ],
      "source": [
        "tf.strings.length(b, unit=\"UTF8_CHAR\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 40,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WB9KciAFAxrX",
        "outputId": "b00c4e22-32c3-49fe-a493-14278efeffcd"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4,), dtype=int32, numpy=array([ 99,  97, 102, 233], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 40
        }
      ],
      "source": [
        "tf.strings.unicode_decode(b, \"UTF-8\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 41,
      "metadata": {
        "id": "XYVKG6PCAxrX"
      },
      "outputs": [],
      "source": [
        "p = tf.constant([\"Café\", \"Coffee\", \"caffè\", \"咖啡\"])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 42,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "__ZYEY-IAxrX",
        "outputId": "c4640fb2-115c-4b17-f9c7-0bab2eed6ee4"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4,), dtype=int32, numpy=array([4, 6, 5, 2], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 42
        }
      ],
      "source": [
        "tf.strings.length(p, unit=\"UTF8_CHAR\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 43,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bw01QOHRAxrY",
        "outputId": "6675a9b9-53d8-4e67-9935-eeddca4e8b0b"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.RaggedTensor [[67, 97, 102, 233], [67, 111, 102, 102, 101, 101],\n",
              " [99, 97, 102, 102, 232], [21654, 21857]]>"
            ]
          },
          "metadata": {},
          "execution_count": 43
        }
      ],
      "source": [
        "r = tf.strings.unicode_decode(p, \"UTF8\")\n",
        "r"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5vRBco4hAxrY"
      },
      "source": [
        "#### 래그드 텐서"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 44,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-CM3R_m5AxrY",
        "outputId": "125a3873-17cf-459e-f84c-ab373e9da0f9"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(6,), dtype=int32, numpy=array([ 67, 111, 102, 102, 101, 101], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 44
        }
      ],
      "source": [
        "r[1]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 45,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3sgYgXB3AxrY",
        "outputId": "0df26412-7873-470f-aaa9-2bf13c818e57"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.RaggedTensor [[67, 111, 102, 102, 101, 101], [99, 97, 102, 102, 232]]>"
            ]
          },
          "metadata": {},
          "execution_count": 45
        }
      ],
      "source": [
        "r[1:3]  # 추가 코드 - 래그드 텐서의 슬라이스는 래그드 텐서입니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 46,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eWw-dmXJAxrY",
        "outputId": "ab04c6e8-ce99-4d76-c3fc-271a014dabd4"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.RaggedTensor [[67, 97, 102, 233], [67, 111, 102, 102, 101, 101],\n",
              " [99, 97, 102, 102, 232], [21654, 21857], [65, 66], [], [67]]>"
            ]
          },
          "metadata": {},
          "execution_count": 46
        }
      ],
      "source": [
        "r2 = tf.ragged.constant([[65, 66], [], [67]])\n",
        "tf.concat([r, r2], axis=0)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 47,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IgM0lcTlAxrY",
        "outputId": "2fb40420-3c97-425a-d7fc-079ca21e0a73"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<tf.RaggedTensor [[67, 97, 102, 233, 68, 69, 70], [67, 111, 102, 102, 101, 101, 71],\n",
            " [99, 97, 102, 102, 232], [21654, 21857, 72, 73]]>\n"
          ]
        }
      ],
      "source": [
        "r3 = tf.ragged.constant([[68, 69, 70], [71], [], [72, 73]])\n",
        "print(tf.concat([r, r3], axis=1))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 48,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7Q7rYycBAxrY",
        "outputId": "0ebeb3a0-d378-43cd-a484-0be247f4dade"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4, 6), dtype=int32, numpy=\n",
              "array([[   67,    97,   102,   233,     0,     0],\n",
              "       [   67,   111,   102,   102,   101,   101],\n",
              "       [   99,    97,   102,   102,   232,     0],\n",
              "       [21654, 21857,     0,     0,     0,     0]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 48
        }
      ],
      "source": [
        "r.to_tensor()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_-k1oueCAxrY"
      },
      "source": [
        "#### 희소 텐서"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 49,
      "metadata": {
        "id": "9czxHg_BAxrY"
      },
      "outputs": [],
      "source": [
        "s = tf.SparseTensor(indices=[[0, 1], [1, 0], [2, 3]],\n",
        "                    values=[1., 2., 3.],\n",
        "                    dense_shape=[3, 4])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 50,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LO9aP3-eAxrZ",
        "outputId": "41fced9c-5593-4d7b-dfdc-2723f0f7f257"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 4), dtype=float32, numpy=\n",
              "array([[0., 1., 0., 0.],\n",
              "       [2., 0., 0., 0.],\n",
              "       [0., 0., 0., 3.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 50
        }
      ],
      "source": [
        "tf.sparse.to_dense(s)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 51,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Hx-pnTOXAxrZ",
        "outputId": "4622761f-94dd-4179-b813-0837dc68e73f"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "SparseTensor(indices=tf.Tensor(\n",
              "[[0 1]\n",
              " [1 0]\n",
              " [2 3]], shape=(3, 2), dtype=int64), values=tf.Tensor([ 42.  84. 126.], shape=(3,), dtype=float32), dense_shape=tf.Tensor([3 4], shape=(2,), dtype=int64))"
            ]
          },
          "metadata": {},
          "execution_count": 51
        }
      ],
      "source": [
        "s * 42.0"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 52,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1AUkxI9zAxrZ",
        "outputId": "af5c5311-359f-46b6-c9cc-25e4d80f0dff"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "unsupported operand type(s) for +: 'SparseTensor' and 'float'\n"
          ]
        }
      ],
      "source": [
        "try:\n",
        "    s + 42.0\n",
        "except TypeError as ex:\n",
        "    print(ex)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 53,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pFgjJfkJAxrZ",
        "outputId": "6e2c9804-2f44-4db5-b34b-ee5e0aacd8ae"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              "array([[ 30.,  40.],\n",
              "       [ 20.,  40.],\n",
              "       [210., 240.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 53
        }
      ],
      "source": [
        "# 추가 코드 - 희소 텐서와 밀집 텐서를 곱하는 방법을 보여줍니다.\n",
        "s4 = tf.constant([[10., 20.], [30., 40.], [50., 60.], [70., 80.]])\n",
        "tf.sparse.sparse_dense_matmul(s, s4)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 54,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yHory7jKAxrZ",
        "outputId": "4245625c-74a9-452b-fcf6-adc2f1f9eade"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{{function_node __wrapped__SparseToDense_device_/job:localhost/replica:0/task:0/device:GPU:0}} indices[1] is out of order. Many sparse ops require sorted indices.\n",
            "  Use `tf.sparse.reorder` to create a correctly ordered copy.\n",
            "\n",
            "\n",
            "\t [[{{node SparseToDense}}]] [Op:SparseToDense] name: \n"
          ]
        }
      ],
      "source": [
        "# 추가 코드 - 희소 텐서를 생성할 때 값을 \"읽는 순서\"로 제공해야 하며, 그렇지 않으면 `to_dense()`가 실패합니다.\n",
        "s5 = tf.SparseTensor(indices=[[0, 2], [0, 1]],  # 잘못된 순서!\n",
        "                     values=[1., 2.],\n",
        "                     dense_shape=[3, 4])\n",
        "try:\n",
        "    tf.sparse.to_dense(s5)\n",
        "except tf.errors.InvalidArgumentError as ex:\n",
        "    print(ex)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 55,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OqVH-dpGAxrZ",
        "outputId": "8c88c415-3778-4634-c199-54bad595921a"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 4), dtype=float32, numpy=\n",
              "array([[0., 2., 1., 0.],\n",
              "       [0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 55
        }
      ],
      "source": [
        "# 추가 코드 - 희소 텐서 s5의 값을 재정렬하여 수정하는 방법을 보여줍니다.\n",
        "s6 = tf.sparse.reorder(s5)\n",
        "tf.sparse.to_dense(s6)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "To8AbsxmAxra"
      },
      "source": [
        "#### 텐서 배열"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 56,
      "metadata": {
        "id": "0BxExIGRAxra"
      },
      "outputs": [],
      "source": [
        "array = tf.TensorArray(dtype=tf.float32, size=3)\n",
        "array = array.write(0, tf.constant([1., 2.]))\n",
        "array = array.write(1, tf.constant([3., 10.]))\n",
        "array = array.write(2, tf.constant([5., 7.]))\n",
        "tensor1 = array.read(1)  # tf.constant([3., 10.])을 반환하고 (0으로 채웁니다!)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 57,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Pa6Wau4mAxra",
        "outputId": "53946ae6-cdd9-4bbd-81e2-7d0a8d41b031"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              "array([[1., 2.],\n",
              "       [0., 0.],\n",
              "       [5., 7.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 57
        }
      ],
      "source": [
        "array.stack()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 58,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BoO188ZgAxra",
        "outputId": "a6fe8f65-96f9-44e1-c0a6-1fd318175d88"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              "array([[ 1.,  2.],\n",
              "       [ 3., 10.],\n",
              "       [ 5.,  7.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 58
        }
      ],
      "source": [
        "# 추가 코드 - clear_after_read를 비활성화하는 방법을 보여줍니다.\n",
        "array2 = tf.TensorArray(dtype=tf.float32, size=3, clear_after_read=False)\n",
        "array2 = array2.write(0, tf.constant([1., 2.]))\n",
        "array2 = array2.write(1, tf.constant([3., 10.]))\n",
        "array2 = array2.write(2, tf.constant([5., 7.]))\n",
        "tensor2 = array2.read(1)  # tf.constant([3., 10.])를 반환합니다.\n",
        "array2.stack()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 59,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "a1bKgzxHAxra",
        "outputId": "d91004c7-4959-4f00-8f3b-43485b1195b8"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              "array([[1., 2.],\n",
              "       [0., 0.],\n",
              "       [5., 7.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 59
        }
      ],
      "source": [
        "# 추가 코드 - 동적 크기의 텐서 배열을 생성하고 사용하는 방법을 보여줍니다.\n",
        "array3 = tf.TensorArray(dtype=tf.float32, size=0, dynamic_size=True)\n",
        "array3 = array3.write(0, tf.constant([1., 2.]))\n",
        "array3 = array3.write(1, tf.constant([3., 10.]))\n",
        "array3 = array3.write(2, tf.constant([5., 7.]))\n",
        "tensor3 = array3.read(1)\n",
        "array3.stack()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z9DLr8QEAxra"
      },
      "source": [
        "#### 집합"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 60,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Lw6V_M4sAxra",
        "outputId": "69bd9941-38af-43ca-defd-4a7cea897ae9"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "SparseTensor(indices=tf.Tensor(\n",
              "[[0 0]\n",
              " [0 1]\n",
              " [0 2]\n",
              " [0 3]\n",
              " [0 4]], shape=(5, 2), dtype=int64), values=tf.Tensor([ 1  5  6  9 11], shape=(5,), dtype=int32), dense_shape=tf.Tensor([1 5], shape=(2,), dtype=int64))"
            ]
          },
          "metadata": {},
          "execution_count": 60
        }
      ],
      "source": [
        "a = tf.constant([[1, 5, 9]])\n",
        "b = tf.constant([[5, 6, 9, 11]])\n",
        "u = tf.sets.union(a, b)\n",
        "u"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 61,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JOCIBh5DAxra",
        "outputId": "5bc68e34-4338-4ddf-d6df-6dfc1fa5d6d4"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 5), dtype=int32, numpy=array([[ 1,  5,  6,  9, 11]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 61
        }
      ],
      "source": [
        "tf.sparse.to_dense(u)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 62,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ysq--SHTAxra",
        "outputId": "eab1207d-ed5d-471d-faee-8a9088921a3d"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 5), dtype=int32, numpy=\n",
              "array([[ 1,  5,  6,  9, 11],\n",
              "       [ 0, 10, 13,  0,  0]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 62
        }
      ],
      "source": [
        "a = tf.constant([[1, 5, 9], [10, 0, 0]])\n",
        "b = tf.constant([[5, 6, 9, 11], [13, 0, 0, 0]])\n",
        "u = tf.sets.union(a, b)\n",
        "tf.sparse.to_dense(u)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 63,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_v3UrL4iAxrb",
        "outputId": "11ea6e23-e36b-475c-8a87-51f369851e93"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 5), dtype=int32, numpy=\n",
              "array([[ 1,  5,  6,  9, 11],\n",
              "       [-1, 10, 13, -1, -1]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 63
        }
      ],
      "source": [
        "# 추가 코드 - 다른 기본값(이 경우 -1)을 사용하는 방법을 보여줍니다.\n",
        "a = tf.constant([[1, 5, 9], [10, -1, -1]])\n",
        "b = tf.constant([[5, 6, 9, 11], [13, -1, -1, -1]])\n",
        "u = tf.sets.union(a, b)\n",
        "tf.sparse.to_dense(u, default_value=-1)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 64,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KSgMPHJDAxrb",
        "outputId": "04f96a8a-df4b-4098-bedd-3fa470e17f3d"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              "array([[2, 3, 7],\n",
              "       [7, 0, 0]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 64
        }
      ],
      "source": [
        "# 추가 코드 - `tf.sets.difference()`를 사용하는 방법을 보여줍니다.\n",
        "set1 = tf.constant([[2, 3, 5, 7], [7, 9, 0, 0]])\n",
        "set2 = tf.constant([[4, 5, 6], [9, 10, 0]])\n",
        "tf.sparse.to_dense(tf.sets.difference(set1, set2))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 65,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0eooV96SAxrb",
        "outputId": "77dea6bd-8254-4c3e-be59-116db57ce199"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[5, 0],\n",
              "       [0, 9]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 65
        }
      ],
      "source": [
        "# 추가 코드 - `tf.sets.difference()`를 사용하는 방법을 보여줍니다.\n",
        "tf.sparse.to_dense(tf.sets.intersection(set1, set2))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 66,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6SwJyFhUAxrb",
        "outputId": "c938c589-296e-4af3-c3c9-7eb5d67402d3"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1,), dtype=bool, numpy=array([ True])>"
            ]
          },
          "metadata": {},
          "execution_count": 66
        }
      ],
      "source": [
        "# 추가 코드 - set1[0]에 5가 포함되어 있는지 확인합니다.\n",
        "tf.sets.size(tf.sets.intersection(set1[:1], tf.constant([[5, 0, 0, 0]]))) > 0"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bsHvqIjtAxrb"
      },
      "source": [
        "#### 큐"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 67,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1_1mwRJ3Axrb",
        "outputId": "bb5bb1c5-c993-4192-8516-9aef570a9c5e"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=2>"
            ]
          },
          "metadata": {},
          "execution_count": 67
        }
      ],
      "source": [
        "q = tf.queue.FIFOQueue(3, [tf.int32, tf.string], shapes=[(), ()])\n",
        "q.enqueue([10, b\"windy\"])\n",
        "q.enqueue([15, b\"sunny\"])\n",
        "q.size()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 68,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5WQ1dE_HAxrb",
        "outputId": "b281071a-1c42-47d9-ff0e-bbdb1ef6c313"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(), dtype=int32, numpy=10>,\n",
              " <tf.Tensor: shape=(), dtype=string, numpy=b'windy'>]"
            ]
          },
          "metadata": {},
          "execution_count": 68
        }
      ],
      "source": [
        "q.dequeue()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 69,
      "metadata": {
        "id": "udUYygHeAxrb"
      },
      "outputs": [],
      "source": [
        "q.enqueue_many([[13, 16], [b'cloudy', b'rainy']])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 70,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tJlE67eKAxrc",
        "outputId": "dfdc64e6-b1da-4b08-8ed0-4f0244dc93dc"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(3,), dtype=int32, numpy=array([15, 13, 16], dtype=int32)>,\n",
              " <tf.Tensor: shape=(3,), dtype=string, numpy=array([b'sunny', b'cloudy', b'rainy'], dtype=object)>]"
            ]
          },
          "metadata": {},
          "execution_count": 70
        }
      ],
      "source": [
        "q.dequeue_many(3)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dw_LFcqOAxrc"
      },
      "source": [
        "## 사용자 정의 손실"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 71,
      "metadata": {
        "id": "h8ITtrkcAxrc"
      },
      "outputs": [],
      "source": [
        "def huber_fn(y_true, y_pred):\n",
        "    error = y_true - y_pred\n",
        "    is_small_error = tf.abs(error) < 1\n",
        "    squared_loss = tf.square(error) / 2\n",
        "    linear_loss  = tf.abs(error) - 0.5\n",
        "    return tf.where(is_small_error, squared_loss, linear_loss)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 72,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 373
        },
        "id": "sMf7oMPXAxrc",
        "outputId": "92f06483-3dff-4b8a-9d70-82fd55cd5f20"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 800x350 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAqIAAAFkCAYAAAD7WFnHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB3fElEQVR4nO3deVhU1RvA8e+wCAiCigqI4pJLmpaaO+5LbuVapmZpm6Wm/krNJXdNM8stbdNcytTKXLPc9wVNXDIXLHMPUFBB1hmY+f1xAkRAGRi4M8P7eZ554t65c+f1dGHeOfec9+hMJpMJIYQQQggh8pmD1gEIIYQQQoiCSRJRIYQQQgihCUlEhRBCCCGEJiQRFUIIIYQQmpBEVAghhBBCaEISUSGEEEIIoQlJRIUQQgghhCYkERVCCCGEEJqQRFQIIfJAbGwsq1ev5uTJk1qHIoQQVksSUSFEgdO3b190Ol2mj7t371rkPW7dukXv3r1ZsWJFjs9RrVo1XF1dc/Te06ZNY8eOHVke06RJE8aPH5/j2IQQwhKctA5ACCHy26BBg2jfvn2mz7m7u2e6/86dO+zdu5fHH3+cxx9/PN1zW7ZswcPDgyZNmlgsxhs3bvDXX39Rv379HL12/PjxfPDBB7Rp0ybTYw4ePIivr29uwxRCiFyRRFQIUSAsWrSIyMjIRx736aefpv7cp08fAgICAAgJCaFbt25MnDiRSZMmpXtN//79efzxx9mzZ4/F4h03bhzJycno9XoSExNxcXHJ9mv/+ecfgNTYhRDCWkkiKoQoED799FNCQkLMek3Dhg01SeZmzJjBsmXLePrppzlx4gQdOnTg+++/x8/PL1uv/+233wAsNsxACCHyiowRFUIUCH/88Qfx8fFmPZo1a5avMYaHh9OnTx/Gjh1Lly5dOHz4MKtWreLIkSPUqFGDhQsXEhcX99BzXLp0ie+//x4HBwcmTpzIsWPH8il6IYQwnySiQogCoVChQri6uhITE8OHH35Is2bN8PPzw8PDg2LFivHEE0/w6quvcuTIEVxdXXF1dcXBIeOfyKCgIObOnZvu8ajk8GEMBgP79+/nnXfeoVKlSqxZs4bp06ezdu1anJ2d6dmzJydPnqRGjRq88847lC1bljfffJP169dnONe9e/d4/vnn0ev1bNmyhYCAANq3b8/Ro0dzHJ8QQuQluTUvhCgw7ty5Q506dbh9+zajRo2iefPmeHt7k5iYyNmzZ5k/fz4tWrRgyZIlvPrqq5me4/Dhw/z555/p9sXFxbF37150Op3ZMUVGRtK9e3cSExN58cUXGTt2LBUrVgTU2NOyZcsydepU9u7dy+7du/niiy/48ccfqVOnTrrz/PPPP3Tv3p1Tp06xYMEC2rZty7Zt22jZsiVNmzZlypQpDBs2LEez8IUQIs+YhBCigPj5559NgGn27NmZPh8XF2dydnY2NWvWLMNzhw8fNgGmqVOnZnjO39/fVKlSJdNnn32W+pg8ebIJMA0fPvyRcV27ds2k1+sz7Hd3dzc1aNAgw/77j01KSjLNnDnT5O7ubnJzczMtWbIk3bG3bt0yde7c2QSY2rVrl7ofMPXo0eORsQkhRF6SHlEhRIHRsGFDihYtyvz58wkICKBZs2Z4e3uTkJDAuXPnmDt3LgaDgY4dO5p9bn9/f955553U7cuXLzNx4sRsvbZMmTJmvZezs3Pqz46Ojuj1eho2bMj8+fOpXr16umNLlCjBhg0b2LNnD1WqVDHrfYQQIq9JIiqEKDBKly7NkSNHmDx5Mq+99hrR0dGpzzk4OFC7dm2WLVtGv3798iWe/v37s3z58ocec+TIkYfe8l+1ahXjxo1j3LhxAOj1evR6fYbj6tatC0BMTAygxpM6OclHgBBCW/JXSAhRoFSpUoXvv/8ek8lEeHg49+7do1ChQvj4+OT7+Ml+/frRsGHDXJ2jXr166banT5/O5MmTs/XaHj16sGbNmly9vxBC5IYkokIIu/b777+bXT/0QX379k39OTg4mMWLF6d7Pqez5lu2bEnLli0BNZFKp9NRtGjRHMcJ0L17dypVqvTI4/Kr11cIIR5GElEhhF1bvnw5CxcuzNU57k9E169fn2nppNyqV68erq6uGWbkm6tSpUqULl36kcflZIa/EEJYmtQRFULYtQULFmAymbJ8uLi40Lx584ceA2qi08OOseTynrnx8ccfU7JkyUc+kpOTtQ5VCCGkR1QIIezRggUL8Pf3f+gxj3peCCHymiSiQghhJW7fvs2CBQseeZyLiwtvvvnmQ4+JiIjA0dHxoceEhYXx1FNPUahQIbPiFEIIS5FEVAghrERoaChDhgx55HFeXl6PTEQnTZqUrffs1auXJKJCCM3oTCkDoIQQQgghhMhHOZ6sNHPmTHQ6XZYD9I1GI+PGjaNMmTKUKlWKDh06cPny5Zy+nRBCCCGEsDM5SkTPnDnDihUrHjrQfebMmaxdu5Zjx44RGhpK9erV6dSpE0lJSTkOVgghhBBC2A+zE9GkpCT69+/PZ599luXycCaTiXnz5jFu3Dh8fX1xdHRk2rRpXL16le3bt+c6aCGEEEIIYfvMTkQ//PBD6tevT4sWLbI85tKlS4SHhxMYGJi6z83NjTp16hAUFJSjQIUQQgghhH0xa9b88ePHWbFiBSdOnHjoceHh4QD4+Pik2+/j45P6XGYSExNJTExM3TYajdy+fRtvb29ZBUQIIYQQwgqZTCbu3btH6dKlcXAwr48z24moXq+nf//+fPnll3h4eDz0WKPRCGRcQs7BwSH1uczMmDGDyZMnZzckIYQQQghhJa5du0aZMmXMek22E9EpU6bQqFEjWrdu/chjvb29AVWc2c/PL3X/7du3qVixYpavGzNmDO+9917qdlRUFAEBAVy4cIHixYtnN9QCz2AwsHv3blq2bImzs3O2XqPbvRvdsWMY33sPHlEE2x7lpM2EtJu5YmNjKVeuHAAXL17Ey8tL44hsh1xr5ivQbRYTg1PNmhjHjME4YIBZLy3Q7ZZDt2/fpkqVKhQpUsTs12Y7ET169ChBQUH88MMPqfuio6N59tlnqV27Nvv370/dX6lSJby8vAgODubZZ58F1CSnEydOMOAhF4SLiwsuLi4Z9hcvXjw1uRWPZjAYKFy4MN7e3tn/JXr+efUooHLUZkLazUyurq6pPxcvXpyiRYtqF4yNkWvNfAW6zYoWhZUroVYtMDN/KNDtlks5GUaZ7Rv527ZtIzo6mrt376Y+AgIC+OWXX9ixYwfPPPMMc+bMAcDJyYm33nqLDz74gNDQUAwGA+PGjcPd3Z1OnTqZHaTIJ1FRMHcu3LundSRCCCFEzjk6wrPPgpm3iUX+y3FB+/slJSUREhLC1atXU/dNmzaNFi1a8NRTT1GqVCl+//13tm7dipubmyXeUuSF6GgYOxaksoEQQghbdeQIvPwy3L2rdSQiG3K11vz9KyVduXIl3XPOzs7MmzePefPm5eYtRH4qWxbCwyEHYzyEEEIIq3D7NoSFgaen1pGIbLBIj6iwI0WKQHKyfJMUQghhmzp0gO3bwcwyQkIb8n9JZNSuHQwdqnUUQgghhHmCglRvqLAZubo1L+zUe+9BiRJaRyGEEEKYZ8AANVP+22+1jkRkkySiIqOOHbWOQAghhDDf3r0QG6t1FMIMcmteZO74cejXT40XFUIIIayd0QjFiknJJhsjiajIXHIynDsnY22EEEJYv6tXoXx5OHZM60iEmSQRFZmrVw+OHgV/f60jEUIIkdeOHYNXXoFKlUCng3HjtI7IPDoddOkCjz+udSTCTJKIioc7fx5u3NA6CiGEEHnp4EE147xJE/Dysuy5f/wROnUCPz917mbN4MABy75H2bLw2Wfg4WHZ84o8J4moyJrBAE2bwoIFWkcihBAiLw0ZAhcuwLJlap12S5o7V1ViWbgQfvpJ3Wlr3RpOnbLM+XfuhB9+AJPJMucT+UpmzYusOTvDtm1QvbrWkQghhMhLeVn8fdMm8PZO227TBmrWVInp119b5vynT8OLL+b+XCLfSSIqHq52bfVfk0mNwRFCCCHMcX8SCirprVEDLl2yzPnnzoX4eMucS+Q7uTUvHu3rr6FFC7ntIYQQIveSk+H339XEqNy6fl39180t9+cSmpBEVDxa5cpqcLnBoHUkQgghbN2CBarc0qBBuTtPZKT6fFq2zCJhCW3IrXnxaC1bqocQQgiRG0eOwOjRqjxUzZq5O5eXF6xcqWb6C5slPaIie2JjVWmMmze1jkQIIYQtunxZ1fp87jmYODH353Nygm7doGTJ3J9LaEYSUZE9iYnqG6yla78JIYSwPS+8oCawPuxx/+fF3buqlmj58rB8ee4nv27eDK++qj6bhE2TW/Mie4oXV4PCixTROhIhhBCWdusW7N2rfo6LU4uZrFkD7u7QoUPG46tVg379Mu6/ehV271bl/558Uu3T66F7d3XeXbssM7EoNladz8Ul9+cSmpJEVGRfkSJgNEJEBJQqpXU0QgghLOXMGdXLmeLnn9WjXDl1S/1BU6Zk3Hf5sqqw4uysVlPy9FT7Bw1SSe6iRapkU0rZJheXtBKB5urZUz2EzZNEVJjntdfgr7/UcnBCCCHsQ25L9KUkof/+q5LQrl3TntuxQ3VivP56+tdkleQ+yrZtUKeOWq1J2DxJRIV5Bg6UMk5CCCHS3J+E/vSTmpD04POWotfDSy+pXtbJky13XqEZSUSFeRo00DoCIYQQ1uJRSailFSoE586Bo2Pevo/INzJrXpjvn3/UH5vwcK0jEUIIoZX8TkKNRkhKUrfkixXL2/cS+UYSUWG+YsXUhKV//9U6EiGEELn1qDJM9z9S3J+ErlmT90koqLGhFStCaGjev5fIN2Ynoj/++COBgYH4+Pjg7+9Px44dOXXqVKbHTpo0CQ8PD3x9fdM9EqXul20rVkxNVsrpbEchhBDWw2SCH36Ajh3B11fNdm/aFPbvV8/d/4C0JDQ0VCWhnTvnT5wVKkD//ipGYTfMTkQ3b97MJ598QlhYGFeuXKF27dq0b98eo9GY6fEjRowgLCws3cNF6n7Zh7//hmPHtI5CCCFEbs2dq255L1yobrP7+0Pr1vBgR9P9SehPP+VfEgpQtaoqG5XbYvjCqpg9WWn58uVpL3ZyomfPnkyfPp2IiAhKSW3JgmXQIHBwgC1btI5ECCFEbmzaBN7eadtt2qi14BcuhK+/Ttvfrx9cuQJVqsDaterxoD594JlnLBvf/PnqPdu3t+x5heZyNWs+NDSUmTNn8swzz+RpEpqUlGenFrmxaJEUthdCCHtwfxIKqpOhRo204vOgJgsFB6ufL1xQj8zcXxjfEkwm+PVXtUyoJKJW6Ztvcj7lKMeJaNOmTTlw4ACNGzdmw4YNWR43b948vvrqK5ydnalZsyYjR46kRYsWmR6bmJiYbvxodHQ0AH37OvDjjwY8PHIabcFi+K/OpyGv632WLq3+GxenVtKwYfnWZnZG2s0897eTwWCQdjODXGvmy1WbJSfj9PvvGNu2xXj/6+/cye6bm/+eD7NpEyQn50sda7nWss9ohDFjHJgzJ+fltHQmU86XUrh58yZjx47l0KFDHD9+HFdX13TP37lzh8KFC+Pi4sLt27dZvHgxEyZMYOfOnQQGBmY436RJk5icaYHaKCpWNDJuXBDFi8tEJ2vifeYM9T7+mD1z5pBQvLjW4Qhh1RISEujVqxcAq1evzvA3UwhrUXHTJp5YupQ9s2dzr3x5zeLQJSfjcf0698qV0ywGkbnERAfmzn2aw4dLA9GAF1FRUXimLO2aTblKRAH0ej1FihRh7dq1dOrU6ZHHt2rVimrVqrFw4cIMz2XWI1q2bFkgCvAkIMDEhg1JPPFEbiK2fwaDge3bt9O2bVuc87qn8u5dHD75BOOwYVCyZN6+Vx7K1zazI9Ju5omNjaXYf/UPb968SdGiRbUNyIbItWa+nLaZ7uhRHNu0wThiBMYJE/IwwmzEsm4dTi++iOHsWahUKV/eU661R7t1C7p3d+TIEXVL3sEhCqOxaI4SUbNuzScnJ+P4wGoGDg4OODo6ZtiflYSEBIpn0XPm4uKS6Yz6MmVMXL8OV6/qaN7cmbVr1WQ+8XDOzs55/0tUsiTMnIm9rHGRL21mh6Tdsuf+NpI2yxlpN/OZ1WaXL0OPHvDcczhOmYKj1jPUu3WDbdtwrlYt399arrXMXbigKn1dvKi23d3hm2+S+e9mj9nMGl16+vRpOnfuzOnTpwHVGzp8+HB8fHxo2rQpvXv3ZsSIEanHjx07litXrgCqt/Ojjz4iJCSEt99+26wgt25N4umn1c/R0Wqs8n2T94U1WLQI1q3TOgohhBA5dfcudOoE5curD1mtk1BQ8w/attU6CvGfgwehUaO0JLR0aVVutk2bnN9cNysRrVmzJm3btqV///74+PhQoUIFrly5wvbt23F3d+fChQv8888/qcd7eXnRoUMHSpUqhZ+fH4cOHWL//v34+/ubFaSPD+zdC889p7aTklRN20mT0urrCo399hsEBWkdhRBCiJzQ66F7dzX5dMMGcHPTOiJ4801VN1RYhR9+UHejb99W2zVqqI/93K5tY9ateUdHR4YMGcKQIUMyfT44pazDf0aNGsWoUaNyHt193N1Vh9uwYaqsGcDkyaqyxKJFUKiQRd5G5NRPP0E2h2cIIYSwMoMGqR6fRYvUB2tK2SYXF+1W0atUSfVECU2ZTPDxxzB6dNq+tm3Vx76XV+7Pn6s6ovnN0RE++wweewyGD1eN8+23cO2aqqkr4/415Oio/oecOAF16mgdjRBCCHPs2KFq8bz+evr95cqpcaNasFBHlsi5pCQYMgS+/DJt36uvwldfWa5qY84rkGpEp4N331WZeErlk927ITBQLfYgNLR+PTz9dNZFjoUQQliny5czritvMmmThMbHw5w5EBWV/+8tUt27p1ZwvT8JnToVvvnGsqXDbS4RTdGjh0pAS5RQ22fPQsOGsvS5pjp1gl27oHJlrSMRQghhq44cgbFjISJC60gKrH//hebN1fQPUInnd9/BuHGWn8Nms4koqMQzKCgt7wkLUw33yy/axlVgFSoELVuqq1RmkQkhhMiJFi3UB/pjj2kdSYF0+jQ0aKBG2oEa9rhtG/TtmzfvZ9OJKKjr9PBhaNJEbcfFQZcu8Pnn2sZVYJlM6mqdPl3rSIQQQtia8HC1lKclZsEIs+3YofKp69fVdvnycOiQ+m6QV2w+EQXw9obt20ktpmo0wuDBMGKE+lnkI51O1XSoWFHrSIQQQtiaPn3IcWV0kStLl0KHDqpeO0DduqqjL6/XErCpWfMP4+oK33+vsvePPlL7Pv1UjbP+7jvrKIlWYNxf40EIIYTIro8+UjVNRb4xmWDiRDURKUXnzrBypSqdmdfsokc0hYMDzJihygqklLT8+Wdo1Uqtiyry0Y0bqvCYjBUVQgiRXfXqqTI4Il/o9fDKK+mT0CFDVEnM/EhCwc4S0RQDBqgJSx4eajsoSE1skqpC+ejCBTVO9L6VtoQQQohMXbsG7drJZ0Y+unNHNfmKFWpbp1NVs+bNy9/1aewyEQW1Hv3+/WodVFDXdqNGcOCAtnEVGC1aqF5RmfUohBDiUSIiVPX0kiW1jqRAuHxZdTzv2aO2XV1hzRr43/8sX57pUew2EQWoVUuVI3vySbV9+7ZaJ/WHHzQNq2DQ6VS/flwc3LypdTRCCCGsWe3asHMnFCmidSR279gxdZf43Dm1XbKkqsvevbs28dh1IgpQpozqGW3bVm3r9WpC3syZMnwxz5lM6iuXLNMmhBAiKzt2wPnzWkdRIGzcqOqth4er7SpV1Mz4hg21i8nuE1EAT0/YvDn9ErqjR8PAgepOgMgjOh188gmMH691JEIIIazVuHHw4YdaR2H3FiyAbt3UjUpQ9UIPHdJ+BJ3dlG96FGdnWLRIlbf84AO176uv1Pr0P/4odwPyTOvWWkcghBDCmu3dm1a8Ulic0QgjR8Ls2Wn7evVSdUNdXbWLK0WB6BFNodOp5Wu//16tRgmwZQs0a6bm1Yg8cuwYNG0qf2iEEEKkMZnU54KLi0xSyiPx8fDCC+mT0DFjVB5kDUkoFLBENEWfPmrd1GLF1PbJk2p8xB9/aBqW/fLzg+LFVa0IIYQQAmDfPvD3h5AQrSOxSzdvqjrqa9eqbUdHdSd4+nRVd91aWFEo+at5czU2onx5tX39uhovsW2bpmHZJ39/2LABypXTOhIhhBDW4vHHYfJkNWNGWNSFC6pkZVCQ2vbwUPXVBwzQNq7MFNhEFNTvQFAQ1K+vtu/dg06dYMkSbeOyWwcOqBoRQgghhI8PvPde/heutHP796skNGVtgNKl1b727bWNKysFOhEF9Xuwezd07aq2k5LU7Ppx46S8k8XNmAGff651FEIIIbQ2blzakj7CYn74Adq0UXXTQdVRP3JE1VW3VgU+EQUoXFitKDBsWNq+Dz+Evn0hMVG7uOzOd9/JagJCCFHQmUxqPFxkpNaR2A2TSdVH79VL1UsHVT99/35VT92aFZjyTY/i6Ahz50KFCvDuu+p/6sqVajb9unVpE5tELhQvrv4bHq66ooUQQhQ8Oh0sW6Z1FHYjKQkGD4avv07b9/rr8MUXqnSltZMe0QcMG6YSTzc3tb13LzRuDJcuaRuX3QgKUpOXgoO1jkQIIUR+u31brTAjY98s4t49eO659EnotGmqbrotJKEgiWimunSBPXugVCm1ff68Ku909KimYdmHunXV17SqVbWORAghRH5buxZ69IBbt7SOxObduKHqoG/ZoradndWw2w8+sK35X2Ynoj/++COBgYH4+Pjg7+9Px44dOXXqVJbHz58/nwoVKlCqVCkCAwM5efJkbuLNN/Xrq867xx9X2zdvQosWsH69llHZAScnePNNVUtCCCFEwfL663DmTFpPj8iRP/5QHWQpKVXRorB9O7z0kpZR5YzZiejmzZv55JNPCAsL48qVK9SuXZv27dtjNBozHLtq1SqmT5/O1q1buXnzJj179qRdu3ZERUVZJPi8VqGCqjXavLnajo+H7t1h3jxt47ILs2er5R2EEEIUDDExqqtO68XNbdy2baru+fXrart8+fS5iq0xOxFdvnw5jRo1QqfT4eTkRM+ePQkLCyMiIiLDsXPmzGHo0KFU+a9Y7bBhw/D09GTlypW5jzyfFCsGW7emfcswmeB//1OP5GQtI7NxtnTfQAghRO4kJakaQrNmaR2JTVuyRNU7v3dPbderp+7eVqumbVy5kasxoqGhocycOZNnnnmGUg90s+v1ek6cOEFgYGC6/Y0bNyYopdS/jXBxUZWHxo1L2zdvHjz/PMTFaReXTXv3XVVXVAghRMEwfrz1VlW3ciaTykFef13l9KDms+zebftFaHJcvqlp06YcOHCAxo0bs2HDhgzPR0ZGkpSUhM8DLeTj48MfWSzqnpiYSOJ9hTujo6MBMBgMGAyGnIZqMRMmQECAjkGDHElK0rF+PTRvbmTdumSruhBS2soa2uyh9Hp0q1Zh6tJFDXDRkM20mZWRdjPP/e1kLX/XbIVca+azujbr00f911riyYK1tVtiIgwY4MiqVWl9h++8k8ysWUYcHa2jOXPTVjlORPfv38/NmzcZO3YszZo14/jx47i6uqY+nzJmVPfALVgHB4dMx5MCzJgxg8mTJ2fYv3v3bgoXLpzTUC2qVCn44IOSfPxxPeLjnTl2zIG6deMZPz6IMmVitA4vne3bt2sdwkO53LlDm8GDORkSwr9NmmgdDmD9bWatpN2yJyEhIfXnXbt2pfubKbJHrjXzad1mJU6dwu/oUc7064exUCFNYzGH1u0GEBPjzIwZ9TlzpgQAOp2J1177kzZt/mHrVo2Du09cLm4P60ym3BXz0uv1FClShLVr19KpU6fU/fHx8bi7u3Pw4EEaNWqUun/AgAHExMRkOk40sx7RsmXLEhoaire3d27CtLjTp6FLFyeuX1eJdtGiJtasSaZZM+1roxkMBrZv307btm1xtvZCYjdvWsXsSZtqMysi7Wae2NhYiv23OsbNmzcpqvGdAFsi15r5rKXNdN9/j8OqVSRv2mQT8wOspd0uXYLOnZ0ICVFt5uZmYvnyZLp21T7PeFBkZCR+fn5ERUXh6elp1mvN6hFNTk7G0dEx3T4HBwccHR0z7Hdzc6N69eoEBwenS0SPHj1K3759Mz2/i4sLLi4uGfY7Oztb3R+eOnXUAOFnn1XlE+7e1dGxoxNLl6bdfdCaNbZbBv7+avBLeDj4+modjW20mRWSdsue+9tI2ixnpN3Mp3mb9e8P/fvbXOFyLdvt999VfnHzptouWRI2bdLRoIF1LoiZm3Yy67o4ffo0nTt35vTp04DqDR0+fDg+Pj40bdqU3r17M2LEiNTjBw8ezMyZMwkJCcFoNLJw4UIuXbqUZSJqa/z9Yd8+6NBBbev1anb9hx/KohFmmTgRGjSwjoEuQgghLGfdOlW2SWTbhg2qFFNKElq1qur4atBA27jyilmpdc2aNWnbti39+/fn+vXrODk5Ua9ePbZv3467uzsXLlxId2t94MCBRERE0KpVK2JjY6latSpbt27F1wp6viylSBHYuBHeeQe++krtGzdOdanbyjqvmnvpJQgMVMXuhRBC2IfQUOjVS603+corWkdjEz77TC01ntKZ1ayZyuWLF9c2rrxk1ie/o6MjQ4YMYciQIZk+H5zJ+uHjx49n/PjxOYvORjg5qaSzYkUYNUrt++YbuHoV1qwBM4dLFDxVq8qSn0IIYW/8/OCvv9R/xUMlJ8OIETB3btq+3r1h6VJVQtKe2dqQDaul08H778Pq1WkXzfbtavWDa9e0jc0mJCSob86ZlAITQghhY+7dA6MRAgLk1uAjxMXBCy+kT0LHjlXrxtt7EgqSiFrciy/Cjh1p3einT6dfD1ZkwdVVPWRwrRBC2L7//Q/attU6Cqt38ya0aqVuvwM4OqqRDB9+CA4FJEOTQXl5oEkTOHwYOnaEixfh33+haVP48ce0iU0iE8uWaR2BEEIISxgwAMLCtI7CqoWEqJzg0iW17eGhhvO1a6dtXPmtgOTb+a9KFZWMNmyotmNi4Lnn4OuvtY3L6oWHw+efax2FEEKI3GjQQK1BKTK1fz80apSWhPr7w4EDBS8JBUlE81TJkrBrF/ToobaTk+Gtt2DMGDV0RmTi8GEYPVoG1gohhC0KC1O9LikZlshg1Spo0wbu3FHbTz6pyjM99ZS2cWlFEtE85uambsnfV16Vjz5SRe/vW+1PpOjcGa5cgbJltY5ECCGEucLC4O5d+G8FMZHGZIIZM9Tnv16v9j3zjOodLVNG29i0JIloPnBwgFmzYOHCtMHHP/ygxnFHRmobm9VxcFB/wBIS1OBaIYQQtqNWLZVZyfK16RgM6o7o2LFp+954A375RUo8SiKajwYNUtWJChdW2wcOqDEiFy9qG5dV6tRJ/dYKIYSwDb/8Ih9ombh3T41WWLQobd+HH6o5I1LZShLRfPfss2pZ0JTFpf76S01oCgrSNi6rM20azJ6tdRRCCCGyw2iEkSNh/nytI7Eq16+rqjlbt6rtQoXg++9Vz6hOp21s1kLKN2ng6adV4tmxI5w9CxER0LKlKl6bMrGpwGvUSP3XZJLfViGEsHYODhAcDPct813Q/fGH+py/cUNtFysG69erZTtFGukR1Ui5cnDwoCpkC2pI5AsvqE5Aqen+nytXVNZ++rTWkQghhMhKXJyaoFS4sExS+s+2baqmeEoSWqECHDokSWhmJBHVUNGi8Ntv8MorattkguHDYcgQVeqpwCtdGp54QjJzIYSwZgsWqOLZcXFaR2IVvvlG9YTeu6e269dXlQkff1zbuKyV3JrXWKFCakGhihVh0iS1b+FCuHpV1Rpzd9cyOo05O8N332kdhRBCiId5+WXV5ZcyE7eAMplg/Hg1ESlF165qTGgBb5qHkh5RK6DTwcSJKiF1+u+rwaZN0Lw5hIZqGpp1OH4c5s7VOgohhBCZ8fNTY8sKsMRE6Ns3fRI6bJhaslOS0IeTRNSK9OunZtZ5eant4GA1o/7MGW3j0tyhQ6ruhawAIIQQ1iM2Vk0sPXBA60g0dfu2Kky/cqXa1ulU38ncueDoqGVktkESUSvTqpWaxBQQoLavXoXAQLVUaIH11ltq+qGrq9aRCCGESBEdrVbBK8DLAl26pD6j9+1T225usHat6g0V2SOJqBV64glV3qlOHbUdFQXt28O332obl2acndXXymvXVFE2IYQQ2vPzU2tYly+vdSSaOHpU3bU8f15tlyoFe/aocaEi+yQRtVJ+frB3r1pgCNTyYP36wZQpBXQSudGoBs3ePwBHCCGENn76CXbu1DoKzaxfDy1awM2bartqVdWBVL++llHZJklErZiHh7rYBw1K2zdxIrz6Kuj1moWlDQcH9Yfv44+1jkQIIcR338Hq1VpHoYl586B7d4iPV9vNm6upDBUqaBuXrZLyTVbOyUmVaKtYEUaMUPuWL1d3qX/+WdUiLTCeflr9V69Xda+EEEJoY8OGtEysgEhOVrW+581L29enDyxZAi4u2sVl66RH1AbodOri/+mntPk6u3apVRuuXNE2tnx38qQaGH/unNaRCCFEwRMXp/7+6nQFqi5RXBw8/3z6JPSDD9TS3JKE5o4kojbk+edVAlqihNo+c0YNlD5+XNu48lX16vDaawWsK1gIIazE8uVQqxbcuqV1JPkmPBxatlRD5UDNnV20CKZNU/m4yB2zE9GjR4/SsWNHSpUqhZ+fH61ateLkyZOZHjtp0iQ8PDzw9fVN90hMTMxt3AVWo0ZqqbDKldV2WJhau/aXX7SNK98UKgQffaRmcwkhhMhfr78OW7ZAyZJaR5Ivzp9Xn7tHj6rtIkXg11/hjTe0jcuemJ2Ivv/++wwcOJDQ0FBu3LhBgwYN6NKlS5bHjxgxgrCwsHQPF+nHzpVKlVQyGhiotmNjoUsX+PxzbePKV5s2wciRWkchhBAFh8GgOgNattQ6knyxf7+Oxo1VrVAAf3/Yv18VrxeWY3YiumPHDp577jkcHR1xcHDg5Zdf5urVq4SHh+dFfCIL3t6wYwf07Km2jUYYPFjlZkajtrHli1u34MIF9YdRCCFE3oqKUrNmN27UOpJ8sXevPx06OHLnjtp+6ik4ckT9V1iW2Ymok1P6ifaHDx/Gx8eHEikDF0W+cXWFVatg1Ki0fZ98An36OJKYaOfDf199Vc3adHbWOhIhhLB/Op0qZl23rtaR5CmTCT76yIE5c+qi16sBoO3aqZWT/P01Ds5O5ap8099//82IESP47LPPcMxiQdV58+bx1Vdf4ezsTM2aNRk5ciQtWrTI9NjExMR040ejo6MBMBgMGKTnK0tTp0JAgANDhzqQnKxj7VoHzpwJpEEDA6VLax1d3tLt34/J21tNYsqFlOtLrjPzSLuZ5/52kr9r5pFrzXwWbTM3N1XIWp0w9+ezQgYDDBniyJIlafnM668bmT8/GWdn2/5nBwfrWLDAgaAgHRcv6hg9OpkpUyx3+zQ315jOZMrZOj137twhMDCQ9u3bM3v27CyPKVy4MC4uLty+fZvFixczYcIEdu7cSWDKAMf7TJo0icmTJ2fYv3LlSgoXoDIRORUcXIpZs+qRkKC+X/j6xjBhQhClS8dqHFkeSU6m9ZAhhNepw58yclzYgISEBHr16gXA6tWrcU2pxyaEFXts/XoM7u5cbdtW61DyTFycE7Nm1eXECZ/UfS+/fJbu3f+yi5nxmzZV5LffKlC16m2OHPGjU6d/eOml8xY7f1xcHH369CEqKgpPT0+zXpujRDQmJoY2bdpQuXJlvv32W3Rm/F9q1aoV1apVY+HChRmey6xHtGzZsoSGhuLt7W1umAXSiRPQtasjoaHq1ry3t4mff06mcWM7XRf0xg0oXTrXNTQMBgPbt2+nbdu2OMvt/myTdjNPbGwsxYoVA+DmzZsUlTJk2SbXmvks1WYOQ4aAtzfGSZMsF5wVuX4dunRx4vRp9TlSqJCJd94JZurU6nZzrRmNaoFCgMqVnejd22jRHtHIyEj8/PxylIiafWs+Pj6eZ599ltKlS7N06VKzklBQPQLFixfP9DkXF5dMZ9Q7OzvbzcWQ1+rXhwMHDLRsGcPVq55ERupo186Jb79Nm9hkV8qXV/+9ehUCAnJ9OrnWckbaLXvubyNps5yRdjNfrtvsyy/BZMLRHroGH3DqFHTqpPo0AIoVg59/TiY6+gbOzk/Z7bXm6OiIs3PmQypzIldfdMw5WK/X061bN1xcXFi9enWGiUu9e/dmRMo6lMDYsWO58t/SP4mJiXz00UeEhITw9ttv5zhg8Whly8KMGftp00Z920lMhBdfVMu052wghpU7dUrN5ty1S+tIhBDCfly/riaFmkx2Wbl961a1QmFKElqhgiqN2KSJPX5QWi+zEtHDhw+zdetWjh07RkBAQLoi9Tt27ODChQv8888/qcd7eXnRoUOH1OL3hw4dYv/+/fjL1LM85+6exIYNybz2Wtq+UaNg0CBIStIurjzx5JOwbBk0bqx1JEIIYT9++kkVsI+J0ToSi1u8WPWEpvzTGjSAoCCoWjXjscuWqTx82bKcv58lzmGvzLo137x5cx42pDQ4ODjd9qhRoxh1f20hka+cndUvW4UKMH682vfll2p9+h9+UCtE2AWdDvr2VT/b6Td3IYTId+++q8Z02c2HhRorOX48TJ+etq9bN7VmvMyJ1oadF5sUOh2MGwfffZdWcvO336B5c/j3X21js7jvv1e9osnJWkcihBC27b9hdfZUPDMxUfVZ3J+Evvuu6viVJFQ7kogWEH37wvbtkDJJ98QJdSvi9GlNw7KsKlXUgJ/7Ki8IIYQw0/nzatz9L79oHYnF3L4NbduqRWBAddLMnw+zZ0MWZdBFPpFEtABp3hwOHUqbaH79usrbduzQNCzLqVcPZs2Sr7ZCCJEblSurO0x2sqj6P/+om2X796ttNzdYtw6GDMn9uV94QSW1D3scOJD797FnuVpZSdieatXUgOznnoPff4foaOjQAb76inQTm2zakiXg4gIvvaR1JEIIYVuMRtVF+N/CC7buyBH1eXfrltouVUp19NarZ5nzV6umVj590NWrsHu3GhL35JPw99+Web+cunUL9u5VP8fFqU7vNWvA3V3lAFqSRLQA8vFRvyAvvaQqcyQlqYmRly7BlCl2MNdn/37w9JREVAghzGEyqV7QTp3U4Ekbt26d+hiIj1fbjz8Ov/6qJvBaypQpGfddvgwtWqgk9Mcf1ceR1s6cUb23KX7+WT3KlVPxakluzRdQ7u7qIhw6NG3ftGnwyit2MMRy8WKYN0/rKIQQwrYkJ0Pr1lCjhtaR5NrcudCjR1oSmjI0zZJJaGZSktB//1VJaNeueft+2dWihfqe8eBD6yQUpEe0QHN0VPlaxYrqy6/JpEpYXL8Oa9eqFSZskqOj+sf89hs0amTD/xAhhMhHTk4wZozWUeRKcjK8956aiJTipZfgm2/UiK28dH8S+tNP0KVLzs/VrRucO2fea779Vq2uaGskERUMG6a65/v0Ud8e9+xRA7stfQsjX0VGqvp3H3+sqvgLIYTI2po1cPEivP++zY7Pio1NG3KWYty4/BlyZskkFNRQuZAQ814TF5e799SKJKICULcPdu9OG9R9/jw0bGjZQd35qkQJOHkSHntM60iEEML6hYTAn3/abBIaHp42CRdU525+TcK1dBIK6uOroJAxoiLVg0uc3bypxtXc/+3SplSqpP6oXr2qdSRCCGHdPvgAVq7UOoocOXdOdZykJKFFisDmzfmfhK5ZY5kk1NIeVV7q/ocWJBEV6VSsqAZ0N2umtuPj1ViVzz7TNq4c27ZN/aP+/FPrSIQQwvpER6sENDnZJntD9+5VQ8lSJt2UKQMHD+ZPCdSUJDQ0VCWhnTvn/XvmhMmklvXu2BF8fdUs/qZNVYGZBycvaUESUZFB8eIqf+vTR22bTGp2/bvv2uDqmS1bwrJlqm6HEEKI9DZvVvX7wsK0jsRs33+vVku6e1dt16ql7urVrJn3731/EvrTT9abhKaYO1eNWFu4UMXr768KJJw6pXVkMkZUZMHFRc2gr1ABPvxQ7Zs7Vy0/vGKFDS1e5Oys1jcFVajZQb57CSFEqt69VfeYDa0pbzKp9eLHjUvb1769KpdUpEj+xNCvn/o8rFJFVZlZuzbjMX36WM/iVJs2gbd32nabNiphX7gQvv5au7hAElHxEDqdqi1avjy8/bbqDV23TnUybtqkVqiwGV99pWpb7N8vyagQQoCaoFS1qrqfbSMMBhg4UJVjSjFggEqonPIpozEaIThY/Xzhgnpk5v4C8lq7PwkF9TFYo4aana81+UQWj/TGG6qUU8o3zaNH1cBwc0tLaKpGDfXVNClJ60iEEEJ7f/0F1aurlU1sRHS0WvTp/iT0o4/gyy/zLwkFlcTFxGReIP7+R6dO+ReTuZKT1eSuSpW0jkQSUZFNzzwDBw6k3b25dEnVit+3T9u4si0wECZOhEKFtI5ECCG0V6mSup/83HNaR5It169DkyawfbvaLlQIVq+GUaNsco6V5hYsUAVlrKHMtiSiItuefBKOHIGnnlLbd+6ogeI2VfFj/nxYtEjrKIQQQjt6vcreunSxiS/nJ0+q8oKnT6vt4sVh50548UVNw7JZR47A6NFqjG1+TOx6FElEhVn8/dUwy/bt1bZer1aymD5du9IPZgkJUauHCCFEQZScrG5nffKJ1pFky5Ytai7Vv/+q7ZQSg02aaBuXrbp8WX3/eO45dZPQGkgiKsxWpAhs3Ahvvpm274MP1IBxg0G7uLJlwQI1qEgIIQoikwlefTWtWLQV+/prePZZNR4T1NyE+xddEea5e1eNWy1fHpYvt54hDZKIihxxdlYT0e/P6RYvVn80oqO1i+uRdLq06r7SMyqEKGicnOCdd6B+fa0jyZLRCGPGwFtvpdWu7t4ddu2CkiXzP55atVTvYa1a2p4jN/R61YZxcWq1RDc3beLIjJRvEjmm06mB4uXKqZpqer0qhN+0qaqRbLUVQRISYMQIGDIE3n9f62iEECJ/fPKJ+vt3fwFOK5OQoDpsV69O2/fee/Dxx+DoqE1MtWrlPoG0xDlyY9AgtQrVokVqsnFK2SYXF6hdW7u4QBJRYQG9eqmxo127wu3b8McfamD55s3a/uJlyc0NTpxQy0wIIURBkZioMj0rFRmpPkcOHFDbDg4wb57qwBW5s2OH6ml+/fX0+8uVS1seVStya15YRNOmagB5xYpq+99/1b6tW7WNK0spSWhQkPrtFEIIe/fBB2qVEit08aJaMz4lCS1cWC2gIkmoZVy+nHmtU62TUMhBInr06FE6duxIqVKl8PPzo1WrVpw8eTLL4+fPn0+FChUoVaoUgYGBDz1W2LaqVVVe17Ch2o6JUQOjrbZa0vnzavbohg1aRyKEEHnnzBn44gurXdAjKEj9KU5ZocjHR91Gtvb124VlmJ2Ivv/++wwcOJDQ0FBu3LhBgwYN6NKlS6bHrlq1iunTp7N161Zu3rxJz549adeuHVFRUbkOXFinkiXVgPLu3dV2crKaTT92rBV2PD7+uPprl8X1K4QQ9sBh1y6YOzdt5o8VWbtWLRt965barlZNJaZ162obl8g/ZieiO3bs4LnnnsPR0REHBwdefvllrl69Snh4eIZj58yZw9ChQ6lSpQoAw4YNw9PTk5U2VQFdmMvNDX76SQ0wTzFjhqo3mpioXVyZatZMDUS6c0frSIQQIk8YhwyB48fVzBQrYTLBnDnw/PNpw1ZbtICDB1V5IVFwmJ2IOj2woOvhw4fx8fGhxAMTP/R6PSdOnCAwMDDd/saNGxMUFJSDUIUtcXCATz+Fzz5TP4OaBdm2rRqQblU2bsSpUiVcU76SCyGEPUhMxOf331XW5+6udTSpkpNh2DDVWZGyEErfvqp4fbFi2sYm8l+uZs3//fffjBgxgs8++wzHB+oqREZGkpSUhI+PT7r9Pj4+/PHHH5meLzExkcT7usyi/ytIaTAYMFh9pXTrkdJW1tBmb70F/v46+vZ1JC5Ox/790KiRiU2bklInNmmuSRNM48ejL1rUKtrMlljTtWYL7m8n+btmHrnWzGf8+Wfqz5hBYp8+ULmy1uEAEBsLL7/syC+/pPWDjR2bzMSJRnQ661gURa418+WmrXQmU84WZrxz5w6BgYG0b9+e2bNnZ3j+xo0blClThvPnz1P1vmUQRo8ezfHjx9m2bVuG10yaNInJkydn2L9y5UoKFy6ckzCFlfj776JMm9aAu3ddAfDySmTs2CNUrWplt8RNJutZbkLYnYSEBHr16gXA6tWrcXV11TgiYe88rl0jpmxZrcMA4M4dFz78sAF//626PR0djQwadIrWra9qHJnIrbi4OPr06UNUVBSenp5mvTZHiWhMTAxt2rShcuXKfPvtt+gy+eCOj4/H3d2dgwcP0qhRo9T9AwYMICYmJtNxopn1iJYtW5bQ0FC8vb3NDbPAMhgMbN++nbZt2+Ls7Kx1OKkuX4bOnZ04f15dL66uJpYvT6ZbN+0XqTcYDJwfPZqn/vwT42+/pY0nEA9lrdeatYqNjaXYf/ceb968SdGiRbUNyIbItWamixcxBARYTZudOwddujhx+bL6++/paeKHH5Jp3Vr7v/8PkmvNfJGRkfj5+eUoETX71nx8fDzPPvsspUuXZunSpZkmoQBubm5Ur16d4ODgdIno0aNH6du3b6avcXFxwSWTwdTOzs5yMeSAtbVb5cpw+LCaUb97NyQk6OjVy4lPP4X//U/7jsg4X19wdMTZZFJrmIpss7ZrzVrd30bSZjkj7ZYNp05B7droNm8GtG+zPXugWze11jlA2bKwebOOmjWte00drdvNluSmnczq9tHr9XTr1g0XFxdWr16dYeJS7969GTFiROr24MGDmTlzJiEhIRiNRhYuXMilS5eyTESF/StaVA1If/lltW0yqQHrw4ZpX1kk8oknMH78McjtUiGELatZE1avxtSypdaRsGIFPPNMWhJaq5Yqz1SzppZRCWti1teRw4cPs3XrVooXL05AQEC651asWMGFCxfS3VofOHAgERERtGrVitjYWKpWrcrWrVvx9fW1TPTCJhUqBMuXq1WYUoYEf/aZunW/apUVTO78/nu1EK8Vr8cshBCZSkhQX6Z79tR05o/JpBZxmjAhbV/Hjqp6SpEimoUlrJBZiWjz5s152JDS4ODgDPvGjx/P+PHjzY9M2DWdDiZNUvXi3nxTLfixaZOqI7dpE2j6XeXaNfj7b5m4JISwLTExqqtxypS0204aMBjg7bdhyZK0fW+9BQsWgJN1340XGpAZGUJT/furW/UpY5uPHVNLhJ49q2FQo0bBsmWShAohbIuTk/pm36SJZiFERamez/uT0Jkz1QqjkoSKzEgiKjTXurVaTSNltMeVK9C4sZrQpImUBPTXX1WWLIQQtsDVVa2nXKGCJm9/7Ro0bQo7dqhtFxf44Qd4/335Xi+yJomosAo1aqgB7HXqqO2oKGjXDr77TsOgvvhCDWgSQghrZjLBCy+oOzkaOXECGjSA06fVdvHiKiHt2VOzkISNkERUWA0/P9i7Fzp1UtsGA7zyihrulLNlF3Jp1SpYulSDNxZCCDMYDGpgvUb1tn/7DZo1g9BQtf3YY6pUn4YjBIQNkURUWBUPD1i/HgYOTNs3cSK89hro9RoEo9OpgauXL+fzmwshRDYVKqRKjzz3XL6/9VdfqbeNiVHbDRuqJLRKlXwPRdgoSUSF1XFygoULYdastH3Llqme0qiofA7GYIAePWD+/Hx+YyGEyIZZs9RAzHxmNMLo0Wp2fEoN6B49YNcuKFky38MRNkzmsAmrpNPBiBFQrpyqQpKYqMYbBQaqOUQPlLHNO87OsH27KnoqhBDWxGRSqyjFxeXr2yYkqIon9+e/w4fDxx/L6sjCfJKICqv2wgvg7w+dO0NkJJw5o279/PJL2sSmPJdyj+niRTUOS/OK+0IIgfrGvmKF6p7MJ5GR0LUrHDigth0c1A2jwYPzLQRhZ+S7i7B6jRurGfWVKqnt0FA1MP6/ZZTzR0wM1K0Lc+fm45sKIUQWtmxJq5OUT92QFy9Co0ZpSWjhwmpMvyShIjckERU2oVIlNQC+cWO1HRurekm/+CKfAvDwgJ9/hnffzac3FEKIh1i+XC1VlE+CgtTdqL/+Utu+vrBvnybzo4SdkURU2IwSJWDnTnW7HtTdqEGDVLHkfLkz1aqV6gKIjMzXW2FCCJHBypX5Vmj555+hZUuIiFDb1aurxPTpp/Pl7YWdk0RU2BRXV1Vj/v330/bNmgW9ekF8fD4EcPMmVK6scaV9IUSBdeYMnDypxocWKZKnb2UywezZ6st/QoLa17KlWgmvXLk8fWtRgEgiKmyOg0Pa2sUpQ6N++gnatEn7xp5nSpVSb96xYx6/kRBCZOLjj9WU9Txe5SM5GYYOVbPhU97q5ZfV0NSiRfP0rUUBI4mosFlvvw2bNqVNYj90SA2kTxnDlGfefFMVyjMY8viNhBDiAYsWwYYNebp4e2wsdOuWfgjqxIlqWGqhQnn2tqKAkkRU2LSOHWH/frU8KMDff6tk9NChPH7jS5fULfqDB/P4jYQQAlUu5PJllQnm4X3xsDBo3lx9yQe1wMjSpTBpUp7mvqIAk0RU2LzatdXA+Ro11HZkpJpX9NNPefimAQFqGZHSpfPwTYQQ4j+TJqnxRynLGOWBs2fVzPjgYLXt6aluxffvn2dvKYQkosI+BASo2nZt2qjtxETo2VNNZMqToVSOjvDpp1ChQp6P1RJCCD75RC1l5OiYJ6ffvVuVx7tyRW2XLatu+LRunSdvJ0QqSUSF3fDyUst/3v/t/f33VYmnpKQ8etNLl6BBAwgJyaM3EEIUaPfuqds8RYrkWb2k776Ddu0gKkptP3iXSYi8JImosCvOzrBkCUyZkrbvyy+hSxe1OJLF+fqq8Vp5lukKIQq0qVPVqm56vcVPbTKpv5WvvJI297JjR1WoXkYdifwiiaiwOzodjB+vvuU7O6t9v/6qlgX9918Lv5mbmxqM+sQTFj6xEEIA772npq9beLq6Xg+vvaZmw6d4+201Id/Dw6JvJcRDSSIq7FbfvrBtW1rNuxMn1ED8P//Mgze7fl2tOXr9eh6cXAhR4BgMqo6Sry906mTRU0dFqVMuW5a27+OP4fPP1Sx5IfKTJKLCrrVokX4VkGvXIDAQduyw8Bt5eMDt2xAebuETCyEKpFmzoE6dtCWNLOTq1fR/A11c1ByokSOlPJPQhiSiwu6lrItct67ajo6GDh3S9wbkWtGiatq+LL4shLCEF16AUaPUusYWcvy4uit05oza9vaGnTtVhREhtJKjRPTatWvUr18fnU5H0kMmaUyaNAkPDw98fX3TPRITE3McsBA54esLe/aou+eg5ha9+qoaH2XR6ks3b8KAAap3VAghzJWcrP5AVa6sBnFaSMo4+dBQtf3YY3D4sOodFUJLZieiR44coXHjxtSqVStbx48YMYKwsLB0DxcXF3PfVohcc3eHtWthyJC0fVOmQL9+FpyQmpysMt7z5y10QiFEgTJnjiroacFZ8l9/7cBzz6khp6BWnzt8WOW6QmjN7ES0UqVKnDt3jj59+uRFPELkKUdHmD9f/a1PGQ/13XfQqZMjMTHOuX8DPz84d059kAghhLmaN4c+fSwyS95ohOXLq/POO44YjWrf88+r2/ElS+b69EJYhNmJqLe3Nx5S20HYuP/9D9asSRt+tXevA6NHN+XyZQuc3NFRFS0dOzatQrQQQjyM0ajGCdWrp/5A5VJCAvTt68i6dWndniNHqolJbm65Pr0QFpPnhRrmzZvHV199hbOzMzVr1mTkyJG0aNEi02MTExPTjR+Njo4GwGAwYEiptiseKaWtpM0e7rnnYMcOHd26OXLrlo7r14vQpImJDRuSePrpXA4cDQ/HaelSkps1w2THa+TJtWae+9tJ/q6Zx96vNYcFC9Bt3UryunW5rqEUEQHPP+/IoUOqr8nBwcTcuUbefttIcnKeLldvF+z9WssLuWkrncmUs6kae/bsoWXLlhgMBpyy+KW5c+cOhQsXxsXFhdu3b7N48WImTJjAzp07CcxkhPSkSZOYPHlyhv0rV66kcOHCOQlTiEcKCyvM1KkNuXGjCAAuLkkMH36M+vVzV4rJQa/HaOEi1MK2JSQk0KtXLwBWr16NqwVnRAvbVur4cYr+9RcXXnwxV+cJDXVn6tSG/PuvunPp4pLEiBHHqFdPSsuJvBMXF0efPn2IiorC09PTrNfmaSKamVatWlGtWjUWLlyY4bnMekTLli1LaGgo3t7eOQmzQDIYDGzfvp22bdvi7GyBcY8FQHi4gXbt4jh7tgSgehBmzzYyaJAxdyc2GHCYNw/ja69B8eIWiNS6yLVmntjYWIoVKwbAzZs3KZqy2oJ4JLu91kwmixXwDArS0b27IxER6nw+PiZGjtzLwIH17avN8pjdXmt5KDIyEj8/vxwlovm+hkJCQgLFs/hAdnFxyXRGvbOzs1wMOSDtln0+PjB58mHWrOnEDz84YDTq+N//HLlyxZFPPgGHnFbcvXULZs/GsUoV6NHDojFbE7nWsuf+NpI2yxm7a7eFC+HoUViyRI0vz6E1a9Rqcil9OU88AevXJ3HmTJT9tVk+kXbLvty0k0UL2vfu3ZsRI0akbo8dO5YrV64Aqrfzo48+IiQkhLffftuSbyuERTg7G1m+PJmxY9P2zZmj6krHxeXwpKVLw8WLdp2ECiFyoVgxVW0jh0moyQSffqqK0qckoa1aqfU1UlaUE8KaWTQRvXDhAv/880/qtpeXFx06dKBUqVL4+flx6NAh9u/fj7+/vyXfVgiLcXCADz+ERYvSPhfWrlV/2G/ezOFJPT3Vp8WSJWoWgRBCpOjTBz76KEcvTUqCd96BESPSFubo1w9++00t9iaELcjxrfkWLVrw4PDS4ODgdNujRo1i1KhROX0LITTzxhtQtqyquRcTA0eOqCLQv/4KVavm4ISRkap2iskEr79u8XiFEDZm+nS4dw9mzMjRy2NioHdv+OWXtH2TJsGECbJmvLAtsta8EFlo107d3krpwP/nH5WM7t+fg5OVKKFWW5IkVAgBqphnDqvBhIaquvcpSaiTEyxbppYsliRU2BpJRIV4iKeegqAgePJJtX3nDrRpA6tX5+BkKUuZrF8P169bKkQhhC16910YP97sl505Aw0bwvHjatvLC7ZsUbfkhbBFkogK8Qhlyqhe0Hbt1LZer26JffRR2risbIuLU4O6li2zdJhCCFswdqxaZzgHdu2CwEC4elVtBwTAwYNgx2tmiAJAElEhssHTEzZtUmNHU4wZA2+9BWYtKFG4sBpw+sEHFo9RCGHlTCb1ByMHSxt9+y20b5+2anCdOupuzRNPWDhGIfKZJKJCZJOzM3z9tZpjkGLRIrVU6L17ZpzI318N5DpwAE6ftnicQggrpdPBrFnqtnw2mUwwebK69Z7ypbdTJ9i7V1V9EsLWSSIqhBl0OtUTunIlpKzeuXUrNG1q5rBPoxGGDoW5c/MiTCGEtRk0CH780ayX6PXw2mtqNnyKgQPVMHMPD4tGJ4RmJBEVIgd694YdO1QtaoBTp9QEgj/+yOYJHBxg82b46qs8i1EIYSUMBnVP3YxxPFFR0LFj+uHks2aphZjMWFVbCKsniagQOdS0KRw+DBUqqO0bN6BJE9VDmi1+fuoT5ezZHNaEEkLYBGdn+P57eOmlbB1+9aqalLRzp9p2cVGdqSNGSHkmYX8kERUiF6pWVRMGGjRQ2/fuqfFbixebcZKxY9PfexNC2Ae9Hrp0gX37sv2S48fV35MzZ9S2t7eaLf/CC3kUoxAak0RUiFwqVUp9UHTrpraTk+HNN9XE+GyVd1q8WE3JF0LYl5gYdTs+m4Xrf/kFmjWDsDC1XamS+qLbuHEexiiExiQRFcICCheGn35KPxl2+nTo2xcSEx/x4hIl1Alu3ICff87TOIUQ+ah4cbUucN26jzz0iy9U52lsrNpu3FgN/alUKY9jFEJjkogKYSGOjjB7tqpV7fDfb9bKlfDMM3D7djZO8MUXaj36R2auQgirdvMmtGwJ58498lCjUf3aDxqkfgZ1G37nTvUdVQh7J4moEBY2ZAisW6eWkgY1PKxxY7VW/UNNmABHj6qZCUII2xUTo+q7pZTVyEJ8PLz4InzySdq+kSPVEsKurnkcoxBWQhJRIfJA586q4LSPj9oOCVHlnY4ceciLChVSXSBRUTBvXg7WDxVCaM5kgooVVfkMX98sD4uIgDZtYM0ate3gAJ9/Dh9/nHZHRYiCQC53IfJIvXpqokG1amr71i11t27duke8cO9emDgRLl3K8xiFEBYUFAQtWqhf9of46y9o1AgOHVLb7u5qvuLAgXkfohDWRhJRIfJQ+fJw8KD6bAJ1K65Hj0csqNS5s7qPX7Fi3gcohLAcR0fVC1q0aJaHHDyoktC//1bbfn5q+E7HjvkTohDWRhJRIfJYsWKwZYuaQQ/qzt2778KwYarUU6aKF1eTlqZOVePNhBDWy2hUv9j16sEPP6gC9pn46Sdo3RoiI9X2E0+oTtQ6dfIxViGsjCSiQuQDFxf49lsYPz5t3/z50L17WrmWDK5fV2NFDx/OlxiFEDk0Y4b6ZU6Z9v4Ak0ktz9mzZ1pRjNatVe9oQEA+ximEFZJEVIh8otPBlCmwZEnaWtEbN6pxo+HhmbzgscfUONG2bfM1TiGEmZ58Ut1vz2SWUVKSKs30/vtp+/r3V+VFvbzyL0QhrJUkokLks1dfhd9+A09Ptf3772pGfaYlB4sUUb0sM2ZIz6gQ1iYhQf33uefSZ5r/iYlRReq//DJtX8qX0UKF8ilGIaycJKJCaKBNGzhwAMqWVduXL6tao3v2ZHJwcrLqPgkKyscIhRAPlZysVquYMCHTp//9Vy3X+euvatvZOW14jk6Xj3EKYeUkERVCIzVrqtyydm21ffeu+lxbseKBA52d1WL2968fKoTQloODmoH4zDMZnvrzT3WX48QJte3lpSYsvvxyPscohA3IUSJ67do16tevj06nIykp6aHHzp8/nwoVKlCqVCkCAwM5efJkTt5SCLtUunT60i0Gg/qwmjr1gXr2KbNwV6yAcePyPU4hxH1u3VLdmgMGQJMm6Z7auRMCA+HaNbVdrpyqF9qqlQZxCmEDzE5Ejxw5QuPGjalVq9Yjj121ahXTp09n69at3Lx5k549e9KuXTuioqJyEqsQdsnDAzZsgLffTts3YQK88YZKTNOJiFD3/LKYnSuEyGN//aUmEm7alOGpZcugfXuIjlbbTz+t7npUr56/IQphS8xORCtVqsS5c+fo06fPI4+dM2cOQ4cOpUqVKgAMGzYMT09PVq5caX6kQtgxJ6e05f1SLFmiekrTfW8bNgy++UbdFpQlQIXIf489BtOnq/pL/zGZ1GJor76qZsmDmr+0d+9DV/kUQpCDRNTb2xsPD49HHqfX6zlx4gSBgYHp9jdu3JggmXQhRAY6HYwcqephu7iofTt2QNOmabf50OnUY/9+9YTcXRAifyQlqdIWDg7wzjtQuDAAej3066dmw6d45x21lK+7u0axCmFDnPLqxJGRkSQlJeHj45Nuv4+PD3/88Uemr0lMTCQxpdovEP3f/Q2DwYAhwz1KkZWUtpI2yz5rarNu3WDrVh09ejgSGanj9Glo0MDE+vVJqROb8PbG0ceHZL0+k/v3+cea2s0W3N9O8nfNPFpfaw7z5uEwZQpJFy6AtzegJhj27OnInj2qT0enM/Hxx0aGDjViNGo/gkbrNrNV0m7my01b5VkiavzvN1D3QJ0KBweH1OceNGPGDCZPnpxh/+7duyn837dPkX3bt2/XOgSbY01tNnWqO1OnNiQ01IPQUB3Nm+sYMeJ36ta9qQ545RU4dAjHxESSU7pQNWJN7WbNElLqTgK7du3C1dVVw2hsk1bXmkNAAN7Dh3PryBEAwsPdmDatIdeuqYLAhQol87//BVO5cii//aZJiFmS38+ckXbLvri4uBy/Vmcy5Wyg2Z49e2jZsiUGgwEnp4z5bHx8PO7u7hw8eJBGjRql7h8wYAAxMTGZjhPNrEe0bNmyhIaG4v3fN1DxaAaDge3bt9O2bVucs1jzWKRnrW0WEQHduzsSFKR6XBwdTcyfb+TNN//7MhcZiVPjxiSPH48pZTH7fGSt7WatYmNjKVasGAA3b96kaNGi2gZkQzS71k6dUvWXypdP3RUcrKNrV0fCw1VHS4kSJtatS6ZBA+saty2/nzkj7Wa+yMhI/Pz8iIqKwjNltZZsyrMeUTc3N6pXr05wcHC6RPTo0aP0zeID08XFBZdMenacnZ3lYsgBaTfzWVub+fmpEqL9+sFPP0Fyso7Bgx25csWRGTPAwccH3ngDp5Yt00o8acDa2s1a3d9G0mY5k+/t9v774OYGmzcDarJ8r16Q0gFUuTL89puOxx7Ls4/TXJNrLWek3bIvN+1k0YL2vXv3ZsSIEanbgwcPZubMmYSEhGA0Glm4cCGXLl3KMhEVQmTk5garV6uJTCk+/hh694aERB2MHat6a+LjVWknIYTl/PgjLFoEwMKF0LVrWhLapIlaefexx7QLTwhbZ9GvcBcuXEh3a33gwIFERETQqlUrYmNjqVq1Klu3bsVX6lkIYRYHB5V8li8PQ4aoSRA//gg3bsD69VCiBKp2zOXL6pNR1hAUIueMRlWiafBgKFECoxFGDofZs9MOefFFVTdUhvkKkTs5TkRbtGjBg8NLg4ODMxw3fvx4xo8fn9O3EULcZ9AgtVLLiy9CbCwcPKjWqP/1V6g0fjwkJEgSKkRu/fMPzJsH9eoR36wdL78MP/+c9vTo0fDhh+oLohAid+TXSAgb06mTWhY05cbCX39Bo0ZwOPoJtZSL0Qi//KJtkELYskqV4J9/uFWnHa1apSWhjo7w1Veo8dny6SmERcivkhA2qE4dOHIEnnhCbUdEQMuWsGYN8Ntv0KULnD6taYxC2JydO9Xauno9F0KL0LChWqIT1FK8mzap5eWFEJYjiagQNiogQN2aT1lpMDERXngBPjnXCdPJU1CzprYBCmFrIiMhIoIDhxxo1EjdoQcoXVotZtahg7bhCWGPJBEVwoZ5eanxof37p+0bORLe+bKGWvP6yy/TunSEEJlLmWTbsyc/9FpH63ZO3L6tdtWsqX6FatXSLDoh7JokokLYuEKFYMmS9Gtdf/45dO+cRPK336tMVQiRufh4aNIE09x5zJwJvXrr0OvVU23bqp7QsmW1DVEIeyaJqBB2QKeD8ePh22/T6tpv+s2JJvHbCR34X4aas0XUhLBvrq4kd+vBhweaM3p02u7XXlM17L28tAtNiIJAElEh7MjLL8OWLWkfnkEnXWnYEC5/vQ3atFE1n4QQqrrEqVPci9Hx7P7RjP+5VupTU6fC4sWaLlYmRIEhiagQdqZVKzh0SNUbBbh6Ffq9502YqZS2gQlhTRYtwtSgId0bhbJli9rl7AzffQfjxkk5XiHyiySiQtih6tXVBIunn1bb+2Kfpuz+VSxf4w6hoZCcrG2AQmjsdL3X6F3kF3ac8QOgaFHYtg1kBWoh8pckokLYKV9f2LsXnntObSclwdv947lbrSGmyVMe/mIh7NXKlRz65hyBLZz5IULVPitfXt1FaNFC08iEKJAkERXCjrm7w7p18M47ajsBN16LmsOwc2+nzgwWosAwGIgc/TEH31zGvXtqV7166u5BtWrahiZEQZXjteaFELbB0RHmz4eKFWH4cFhn6g5r4J/waH7ouwn3AS9pHaIQec5kNDFhijPzr+0lBg9ALUD2/ffqC5sQQhvSIypEAaDTwbvvqiVAXV3VvtL7V5M08B2uHQvXNjgh8pj+t51c8G3Ggml3iMYLI44MHarWkJckVAht2WWPqMFgILkAT8YwGAw4OTmRkJBQoNvBHPe3GYCzndZt6d4ddu9W40YXRbzJr8aOJD/nw6ZNULeu1tEJYXl37sDI8d60uFWOOAqj08GcOTBsmNaRCSHAzhLR6OhoIiIiSExZrq2AMplM+Pr6cu3aNXRSgyRbHmwzFxcXSpQogaenp9ahWVzDhmpMXMeOOi5cKIMuzMgfDd9G915rnv74Ra3DE8Jiru8Mod3gSpwNqcU3rMDNDVZ/D926aR2ZECKF3SSi0dHR3LhxAw8PD0qUKIGzs3OBTcKMRiMxMTF4eHjg4CCjL7Ijpc3c3d1JTk4mKiqKGzduANhlMvrYY2qWcNeucOiACZKTmPeJgQblYPBgraMTIveCd0dTvm0g3Uz/4yzjKFkSNm2CBg20jkwIcT+7SUQjIiLw8PCgTJkyBTYBTWE0GtHr9bi6ukoimk0pbebm5oaDgwNFihTh+vXrRERE2GUiCuDtDdu3w6uvOvL66iVggu/egejjfzNqUSXk0hG2asMG6N3bk0DTKo7QgKpV4ddf1YQ9IYR1sYuPGoPBQGJiIl5eXgU+CRWWodPp8PLyIjExEYPBoHU4ecbVVc0aHjNGbbdmByOXPM7otsHEx2sbmxA5sfr94+zqOp/4eNhBW2o38+TQIUlChbBWdpGIpkzIsdcJJkIbKdeTvU/4cnCA6dPhq69gr0Mr+rCSWbvq0KoV3LqldXRCZE9ysqoM8cesLfThe5zR07u3Wi2peHGtoxNCZMUuEtEU0hsqLKmgXU8DBsCmzQ785tET0OEatJsZT6zgwgWtIxPi4eLioF/XKObOhRmMoQV7GDm2ECtWgIuL1tEJIR7GrhJRIUTutG8PBw6Avz90Zy1tbq2kUUMTBw5oHZkQmbt5EwbWD+azX8rTgCAcHXV8tsiNDz9ExjkLYQPsZrKSEMIynnpKlXd6tuM8Qk4nknBHxwutIpn7bXFe7FWweomFdQsJgQ4d4PqlmpRhBH+712Lzz9CundaRCSGyS74vCiEyKFMG9h1woNkzbnhwjyBDHY70nsPMmWAyaR2dELBvH3z49FqcL4VgoBDL/T9g50FXSUKFsDFmJ6IJCQkMHDgQPz8/fHx86NWrF5GRkZkeO2nSJDw8PPD19U33KOgF54WwBZ6e8Msv0OuNIkxhAj/wIqNHw9tvQ1KS1tGJgmzVKmjfJonhsZMZwNc8+aTqxX/qKa0jE0KYy+xEdNiwYZw9e5aQkBCuXr0KQJ8+fbI8fsSIEYSFhaV7uMjocYubNGkSOp2Oy5cv29S5zTFgwACaNGmCyYwuucTERKpVq8bUqVPzMDL75ewMX38Nj334Ov/ijzsxlP16HN07JnDvntbRiYLGZIKZH+kY0Oce8QYnWrOTHc/MYv9+1YsvhLA9ZiWiUVFRLF26lBkzZuDp6YmLiwuffPIJ27Zt49y5c3kVoxAsXryY5cuXs2jRIrNms7u4uPD1118zefJkfv311zyM0H7pdDB2rKo3WtfpFG+yiIvbL9KsGfy3+JQQec5ggM8/fwq3CaPYxjM4kEy3N0qw8RcH7HTNCSEKBLMS0eDgYEwmE/Xr10/dV6ZMGQICAggKCrJ4cEIA3Lp1i/fee4+hQ4dSrVo1s1/ftGlTevXqxVtvvUW8VGnPsT59YMrOQJ4u+g9neYLrJ2/yz+NdiDwhbSryVvS/MQQ91o/Q7fdYSR/mM5Rp0x35+mvVay+EsF1mzZoPDw/H29sbJ6f0L/Px8SE8PDzT18ybN4+vvvoKZ2dnatasyciRI2nRokWmxyYmJqYbPxodHQ2olZMetrqNwWDAZDJhNBoxGo3m/JPsRsrtaqPRmPpzSptY8txatO/kyZNJSEhg6NChOX7/ESNG8P333zN79mzGpCwjdJ/M2iylLQ0GA46Ojjn/B9iRRo3gt32FeL5zIlsu16dC4hXCJwcT5O5Hw/811jo8q3f/37FH/V0TSujRa+hbd6ZV4hk2cYimzkEMW/wUvXsbZKzyI6RcX3KdmUfazXy5aSuzElGj0ZjpbVEHB4dME4Rhw4YxZswYXFxcuH37NosXL6Z9+/bs3LmTwMDADMfPmDGDyZMnZ9i/e/duChcunPU/wskJX19fYmJi0Ov15vyT7EZKAh8TE8O9/wbv3bPQIL77z53y5SC/REVFsWjRIrp06UKRIkUyvH+jRo04f/58lq8fPXo0o0aNonz58jRt2pTZs2fz1ltvZfgyleL+NtPr9cTHx7Nv3z6S5BMvnWlDokh6P5nyyQA3Of7+M3y/YxLFBlXXOjSrlpCQkPrzrl27cHV11TAa6xe9I5S2CydTwhRGecDEVca//ANeXuWRkTbZt337dq1DsEnSbtkXFxeX49ealYh6e3tz9+5dTCZTuoT09u3blChRIsPxxYoVS/25ePHivP/++2zZsoWVK1dmmoiOGTOG9957L3U7OjqasmXL0rJlS7y9vbOMKyEhgWvXruHh4VFg/7CnTABzd3dnxYoVLFy4kEuXLlG8eHGef/55Zs6cmXpMq1atuHz5Mv/880+G80yePJkpU6Zw8eJFypcvn+7chQsXZvHixXz99ddcvXqVkiVL0r17dyZOnEjRokVTz6HX61m4cCHffvstFy5cwMXFhYYNGzJx4kQaNGiQ4b127drF6tWr+fnnn4mMjOSbb76hf//+AKxduxa9Xk+3bt3wzGQg2EsvvZThm5her2fevHnEx8fTpk2b1Nd17dqV/fv3ExwcTNu2bdO9xmQyce/ePYoUKZJ6bSckJODm5kazZs0K7HX1MPHdnmFjvV48GbkX0NN/21h2xo6j6Y5x6BylMlxmYmNjU39u1apVut8bkd7BYWto98X/cEV9Ed7rVJEDo8fRbcyLspxzNhkMBrZv307btm2lzcwg7Wa+rKonZYdZiWjt2rXR6/WcOXOGGjVqACoJvXjxInXq1MnWORISEiiexcK/Li4umc6od3Z2fujFkJycjE6nw8HBAYcCupRGSvL0/vvvs337doYNG0b16tXZtWsXn332GUajkQULFqR7TWZtlXKe+9syZd+IESPYs2cPY8aM4bHHHmPLli3Mnz+fAwcOcPjwYQoVKoRer6dTp07s2rWLLl26MHbsWMLCwpg7dy4tWrRgy5YttGzZMt1533jjDYoUKcKcOXNSZ7mnvPfGjRvR6XQ888wzmcY7bty4dNuJiYl0796dhIQEvvjiC1q3bp36XLt27Rg+fDibNm2i3QPFBlN69FOuo5Q20Ol0j7z+CirnCr5U+WczW2u+RrurPwLQ+uA0gqpe4KnjS3HzzvouRkF1/3Uk11XmkpNM7Gw5jWcOTEjdd9KzGT4Hf8Dj4hFptxyQNssZabfsy007mZWI+vj48Pzzz/Puu++yZs0aChUqxNChQ6lbty5169ald+/e+Pv788knnwAwduxY3nrrLcqVK0diYiJz5swhJCSEn376KccB50TduhAWlq9vaRZfXzh2zDLn2rFjB0ePHsXX1xdPT0969+7NlStXWLZsGZ9++mmuSmft2rWLY8eOUaVKFQCef/55nJ2d+fLLL1m6dClvvfUWH330Ebt27WL48OGp1wGoEl+VK1dm8ODBnD17Nt15XV1dOXToUKbDL06cOIG3t3emPe4Pio+Pp2vXruzYsYNFixbx+uuvp3u+SpUq6HQ6goODc/LPF5lwdCtEwrze7F1aj6Yb38cBEw2v/sj5shcovGMTAY2lpo7Ivugb9zhdtz/PhK1N3Xewcn/qHf8KnYsOLmoYnBAiT5i9xOeiRYt45513qFixIkajkZYtW7J+/XoALly4kG6ykZeXFx06dCAiIoKkpCSaNGnC/v378ff3t9g/IDvCwgpOmZkPPviAKlWqpBtL2apVK7Zv387ff//NE088keNzDx8+PDUJTTFixAi+/PJLNm7cyJtvvsmCBQsoUaJEhrqdJUuWpHfv3nz55ZecPXuW6tXTxhKOHj06yzHAYWFhGd4zM3FxcXTu3Jndu3ezdOlSXnnllQzHODs74+3tTWhoaHb+ueIR4uPjadq0KVFRUQQHB3P048d54sPeFCGGx+NPcqtJHX7/6Afqvd9S61CFDbiyLQTjc50J1F8AwIiOw10+ovHakSQkJtC0kbrWWrZsKb1UQtgRsxNRT09Pvv3220yfe7CnadSoUYwaNSpnkVmQr6/WETycJePr0KFDhn0lS5YE1DCK3GjWrFmGfY899hhubm5cuHCBCxcucOvWLXr06EFycjIxMTHpji1btiwAISEh6RLRWrVqPfR9H1XAPjY2lk6dOnHgwAG+++67hy6wkNWEO2E+o9GY+jtvNBppOO1Z/qpziJgXO+GXdI2SplsUH9WGnbs+ouXmETg4SruLzB0Zu4FqM17BE/UFOgYP/p62isAPngUyXmtCCPthdiJqiyx129sWFClSJMO+lNJDycnJuTp3SkL7IE9PTxISElIHK//888/8/PPPWZ7nwdn8mcWcwtfXl7CHjKu4d+8eHTt2JCgoiFWrVvHCCy9keaxer+fOnTvZ6mEVOVO5e02iL5zgeIOXqHNrK44Yab31fQ6XDaL6kaV4lZXK4yKNPi6JvS0n0vbo9NR9f7k8gcvmddRqXVnDyIQQ+aVAJKIivUKFCmWZlD6sBENmJYySk5OJiIigZs2aeHl5AdCjRw/+97//ZXkecxLB2rVrs3HjRiIiIjKME42KiqJ9+/YEBwfz008/0bVr14ee68KFC5hMJp5++ulsv78wn2cFb2rd2My+Z6bQbM8UABqFruWfx/4k7NvVVO1VW+MIhTW4dvg6Ee360Pbe/tR9h8r0pMaRb/As7aFhZEKI/FQwp5gXcP7+/ty6dSvTW9579+7N8nUhISEZ9gUHB5OcnEydOnWoWrUqHh4eREdH06RJkywfpUqVynas3bp1w2QyZajndufOHdq0acOJEydYu3btI5NQgG3btgFk61iROw7OjjTbPZngSZu4qysKQEXDBSr0bsCebvMwJj98uIWwb0fG/4Jn4xrU/i8JTcKRg11n0ejKaklChShgJBEtgGrUqEFiYiK7d+9Ot/+rr77i6NGjWb7u008/zVCz8+OPPwbg5ZdfxtnZmTfeeIOdO3eyc+fOTM9x8OBBs2Lt2rUrLi4ubNy4Md3+vn37cuzYMRo2bMixY8eYNGlSusfnn3+e4VwbN26kRIkSWa7sJSzv6YnPEr3zGCFuTwFQCAMt1v+P332fI/zPWxpHJ/Jb4j09e+oOp8G05/AiCoB/Hcvy9+K9BK4bgc5BxhELUdDIrfkCqF+/fsyYMYM+ffowevRo/P392bx5M6tWraJhw4YEBQVl+rrLly/TsGFD3njjDYoVK8aPP/7IunXreP3111OTu2nTpnHo0CHat2/PK6+8QmBgIEWLFuWvv/5izZo1xMTEcO7cuWzH6uXlxYABA/jyyy+5fv06ZcqUwWg0sm/fPkD14GbWi9ulSxcGDRqUun3y5En27t3LjBkzslxVSeSNgJaPkRh+lL0tx9A8eDYADSI2E/HkEwRNWUnDcW00jlDkh/Nr/sSxb29aJP6Zuu93v+eocnAZpStkXltaCGH/pEe0ACpRogR79uyhdu3aTJgwgVdffZUbN26wd+/edCsfPei7776jQYMGTJw4kf79+3PhwgXmzp3L119/nXqMu7s7e/fuZerUqRw7dozBgwfTv39/vvnmGypXrsz3339vdrzjx4/H1dWVOXPmAKrQ/L179zCZTFk+UkqKpZg1axZlypRh2LBhZr+/yFqJEiUyXfHqQS5FCtH82Kccn/Yrt3VqlbQSpls0HN+WfTUHE38r5hFnELYqKTGZne1nUeGFp6n8XxKaSCH29phP3esb8MpmEprda00IYWNMViwqKsoEmCIiIh56XHx8vOns2bOm+Pj4fIrMuiUnJ5vu3LljSk5O1joUi1m0aJGpUKFCprNnz5r92n379pkcHR1NmzdvzvKYzNpMrqtH0+v1pvXr15v0en22XxNx+l/TiRJtTCZIfVx3CjAdn7cvDyO1DjExMSbABJju3LmjdTh57u9tF01/uDdM9//6kktVU8jq42afKyfXWkEnbZYz0m7mi4iIMAGmqKgos18rPaLCJrzxxhv069ePN99885F1Re+XmJjIgAEDmDhxIh07dszDCEV2edfw46mwrex/cQFxuAHgn3SVp4Y1Z/uTw4n6N/YRZxDWzmhIZk/3+fg88yQ1Y9VQHyM69jcYgV/4Saq8KJUThBCKDJYTNuP+IQDZ5eLiYtaYVJE/dI4ONF09mIuvtiPuhVeoee8wDphoe3o2oWV/4Ny4RTScnHFxBmH9zv9wCl57jRZxx1P3/esUwJ3PVtD07aYaRiaEsEbSIyqEyJH4+HjatGnDBx98QHx8fI7O8Vi7SjwRuZ8DXT8hARcA/Iw3aDilI4cDXuTWH7Icq62IDotjR93RVOr1NI/fl4QeqPk2Ra//yRO5SEItca0JIayTJKJCiBxJqV5w5syZXC276ODsSJN1w4nc9Qd/FGueur/RtR9xe6oyu7ovQB+XcTEFYR1MRhMHhq8lxr8KbYJn4oRaLOOyc2WOzz9Akz++oLBP1qunZYelrjUhhPWRRFQIYRX8W1ahZsRuDr61nMj/ZtZ7EEurdUO4XKwWR6Ztx4zhwSIfXNp4mpMl2tBkdg9KG28Aakb8vjZTKB15mjpDAjWOUAhh7SQRFUJYDZ2DjsAvX8F45jwHK/dP3V9Ff4YG45/hiE9nLv52QbsABQBhZyLZ88RgArrUovadXan7T3m35NaOP2i2fTyFirhoGKEQwlZIIiqEsDolq5Ug8MJSQpYe4qxH/dT9DW9tomzHGux98h3CT4VpGGHBFP1vDDuaT8WtRkVanP0cR9Rt8quOFTgyeh1P3dpJmdZVNY5SCGFL7CoRNaesjxCPIteT9qr2b0S1u4cJGvQtYY6lAbVMaPPTCylSqyK76o3i5rlIjaO0f4nRiezpPp+kMuVos28CXkQDEIcbu9t8iHf4WRrM6Ao6WaJTCGEeu0hEHR0dATKsgy5EbqRcTynXl9CGztGBhgtfxjP0AnubTyD+v9qjhYmn1bGPKVy9HNsDJxH59x2NI7U/cbcT2N3zC24Wr0qLdcMobroNQBKOHHriTeJP/UXL7WNx93bVOFIhhK2yi0TU2dkZFxcXoqKipBdLWITJZCIqKgoXFxecnZ21DsdqFS5cGBeX/BkLWLikO833TObe6cvsrfO/1HJPHsTS9tBkClUOYF/9EYQF38iXeOzZnav32NF+FvdKVKDlT4Mom3wl9bmgsi8QuuMsjf/8Gu8n/fMtpvy81oQQ+cduCtqXKFGCGzducP36dby8vHB2dkZXQG8TGY1G9Ho9CQkJODjYxXeNPJfSZvHx8SQnJxMVFUVMTAz+/vn3QWtr3N3duXv3Lr/++ivu7u759r6lapSiVPAcwo+9R3D/qTQ4swQnkilCDM1+/xRD3XnseewVSswYQY0XquVbXPYg7Nh1/nznCxocmU8bYtI993vJDrjPmUbDl+rke1xaXWtCiLxnN4mop6cnABEREdy4UbB7REwmE/Hx8bi5uRXYZNxcD7aZi4sL/v7+qdeVsD4+dcvi8+fXhB0azYW3PqXen0twIwFnkmhxcQn0XMIJz2YkvDmUelM74+QmPduZMSabOD5vP4bZn1Hvxjra/FcHFNSynL+X7UHxWWOo92L+J6BCCPtnN4koqGTU09MTg8FAcnLyo19gpwwGA/v27aNZs2ZyWzmbUtqsefPmcjvexvg2rojv6YWEnpzAkcHzqX14IV6mKABqR++DT/cRNrc0ZwMHUGFyfyq0KKdxxNbh1vlITo5eRZnfFlFX/0e655Jw4miVvpRdMIoGbR/XKEIhREFgV4loCmdn5wKdSDg6OpKUlISrq2uBbgdzpLSZJKHZl5CQQPfu3bl58yatWrXSvN38avngd/BD4sNHcXjYEsqs/Yyyhn8A8E3+F999k6DlJI57Nie688s8NfV5ipX30jTm/BYfbeDUjF9x+G45tW9soi3pV6y6pSvFH43fpsb8ATSuYz3DUqztWhNCWI5dJqJCiLyXnJzMb7/9lvqztXDz8aTR6v9hSh7K8U93kTh7IfXDN6bWvKwTvRdW7CVxxSAO+XfB1K071Ud0oFg5+0xK4+8mcmLWDhJXraX2pZ9pSFSGY856NSS63xDqTOtBayssRG+t15oQIvckERVC2CWdowN13m8D77ch7Pdr/D1+Of67v6OCXq3M5IKexjd+ggU/YVjgxPGizbjdsgflB3fisVblbLokZujJcP7+cgdOv26kxrXfaMy9DMfc1Plwru7LVJjUj+oda2gQpRBC5KB8U0JCAgMHDsTPzw8fHx969epFZGTWBaXnz59PhQoVKFWqFIGBgZw8eTI38QohhNl865WlyZZxlI8/z4XvjrC/1pDU9ewBnEmizt1dtFk3mEptyhNaKIAD1d5k/6BVXA4Ks/o17u9cusvBD35lZ633+NulOn61fWn6VV8aXfuRIvclofG4cbR8T36f/CtFY67T/OgsAiQJFUJoyOwe0WHDhnH+/HlCQkJwcXGhX79+9OnTh61bt2Y4dtWqVUyfPp19+/ZRpUoV5s2bR7t27bhw4QJeXvZ5G0wIYb10Djqq9K1Plb71SYr/lOOfHyDm+w1U+WMNvslp1TZKJ12j9PnFcH4xfAE3HMpyxbcBhlr18Gpdl9LP1qFk5aKa9JreuRbD35vOEbXjd1xOHqHc9QMEGP4hMIvjo/HkdKVuuL7UnZrvtqW+l1u+xiuEEA9jViIaFRXF0qVL2bNnT2pZm08++YSyZcty7tw5qlVLX7Nvzpw5DB06lCpVqgAqiV2wYAErV65k4MCBFvonCCGE+ZzcnKkzvCUMbwmmOVze9Ac3vtiIx+HtVIsKohBpK7X5G6/h/+81+HcN/AoMh3CdD6EelYnyq4axclXcqlegcCU/PKuWpnh1X4qUcMlRompMMnLrzE1uHr9O1JnrxP91HYfLFyn67zl875zFP/ka9R72enSEuNfhVq1n8OrRhupvNSWwsEzuEUJYJ7MS0eDgYEwmE/Xr10/dV6ZMGQICAggKCkqXiOr1ek6cOMGsWbPSnaNx48YEBQWZlYjGxsbi6ipLyGWXwWAgISGB2NhYmV2aTdJm5ouNjU33s623W8nWlSjZ+j3gPe7ciufiyiBif91L0T/3Uy7qNO7EpTvewxRO5XvhcO8AXAA2pz/fDbxIdHAnxrkY8c4eGBxciXZMa6MjlXrganLESR+Pu/42HklRFDLG40kUHiTjkUWcsQ9sG3DkousTRFZuROFnmlLp9WYEBBQn4L/n9SY9+lh9LlpGe/Z2reUX+buWM9Ju5rv/d9RcZiWi4eHheHt74+SU/mU+Pj6Eh4en2xcZGUlSUhI+Pj4Zjv3jj/Q161IkJiaSmJiYuh0VpWZ3lisndf+EsGZlypTROgQrFAXGKEj8FxIzPts+cpeF3icZEv6A03/A6a/gUwud1krJtSaE9crJMutmTVYyGo2ZrtTj4OCA0WjMcCyQ4fjMjk0xY8YMvLy8Uh8BAQGZHieEEEIIIazLwyavZ8WsHlFvb2/u3r2LyWRKl2Devn2bEiVKpDu2ePHi6HQ6bt++nW5/ZsemGDNmDO+9917q9t27dylXrhxXr16VyU1miI6OpmzZsly7dk2WqMwmabOckXYzn7RZzki7mU/aLGek3cwXFRVFQEAAxYsXN/u1ZiWitWvXRq/Xc+bMGWrUUCU/bt++zcWLF6lTJ/06xG5ublSvXp3g4GAaNWqUuv/o0aP07ds30/O7uLjg4pKxmLKXl5dcDDmQsuSpyD5ps5yRdjOftFnOSLuZT9osZ6TdzOfgYHZVUPNuzfv4+PD888/z7rvvEhUVRXx8PEOHDqVu3brUrVuX3r17M2LEiNTjBw8ezMyZMwkJCcFoNLJw4UIuXbqUZSIqhBBCCCEKDrPriC5atIh33nmHihUrYjQaadmyJevXrwfgwoUL6SYbDRw4kIiICFq1akVsbCxVq1Zl69at+Pr6WuwfIIQQQgghbJPZiainpyfffvttps8FBwdn2Dd+/HjGjx9vfmSoW/UTJ07M9Ha9yJq0m/mkzXJG2s180mY5I+1mPmmznJF2M19u2kxnyslceyGEEEIIIXLJ/FGlQgghhBBCWIAkokIIIYQQQhOSiAohhBBCCE3YVCL6ww8/oNPpWLZsmdahWL0dO3bQtm1bfHx8KF26NM2bN2fPnj1ah2XVQkJCePHFF/Hx8cHX15eGDRuya5ellmG0b3fu3KFjx47odDr+/vtvrcOxWgkJCQwcOBA/Pz98fHzo1atXjlYiKWiuXbtG/fr10el0JCUlaR2OTTh69CgdO3akVKlS+Pn50apVK06ePKl1WFbtxx9/JDAwEB8fH/z9/enYsSOnTp3SOiybMXPmTHQ6ndm5hs0kouHh4YwfP54nn3xS61BswsaNGxk5ciT//vsvN27coFevXnTs2FE+9B5izJgxdO7cmatXrxIWFka/fv3o2rUrUVFRWodm1a5cucKTTz5JpUqVtA7F6g0bNoyzZ88SEhLC1atXAejTp4/GUVm3I0eO0LhxY2rVqqV1KDbl/fffZ+DAgYSGhnLjxg0aNGhAly5dtA7Lqm3evJlPPvmEsLAwrly5Qu3atWnfvn2Wy5KLNGfOnGHFihX4+/ub/VqbmTXfpUsXunfvztKlS+nfvz/9+/fXOiSbEhUVRdGiRTly5Aj169fXOhyrlJSUhJNTWkWzuLg43N3dCQoKokGDBhpGZt1iYmJITEzk3r17VKhQgb/++kuS0kxERUVRsmRJ9uzZQ+PGjQG4fv06ZcuW5ezZs1SrVk3jCK1TZGQkLi4uHDt2jJYtW2IwGNL9norMPfj37OzZszzxxBOEhYXh4+OjYWS249SpU9SqVYvw8HBKlSqldThWKykpiUaNGjFr1iz69+/PsmXLaNGiRbZfbxM9osuXLycpKYl+/fppHYpNun37NlOmTKFGjRrSq/AQD364HT58GFdXVx577DGNIrINHh4eeHt7ax2G1QsODsZkMqX7IlimTBkCAgIICgrSMDLr5u3tjYeHh9Zh2JzM/p75+PhQokQJjSKyLaGhocycOZNnnnlGktBH+PDDD6lfv75Zyef9rP5r5fXr15k4cSIHDx7UOhSb9PLLL7NixQqqVavGpk2bKFSokNYh2YTIyEhef/11xo0bJ3+4hUWEh4fj7e2dIUHw8fEhPDxco6hEQfD3338zYsQIPvvsMxwdHbUOx+o1bdqUAwcO0LhxYzZs2KB1OFbt+PHjrFixghMnTuT4HFbRI1qvXj18fX0zPABef/11xo8fn6NxB/buYe2W4rvvvuPu3bv06NGDJk2aFPgPvOy0WWJiIj169KBu3bqMGTNGo0itS3baTTyc0WhEp9Nl2O/g4CBj0ESeuXPnDp07d+bVV1+lb9++WodjE/bv3094eDjVqlWjWbNmJCQkaB2SVdLr9fTv358vv/wyV3ctrKJH9Pfff890/+LFiwGVjIqMsmq3B3l5eTF16lRWrFjBmjVrGDx4cB5HZr0e1WZJSUn07NkTR0dHVqxYgYODVXxX01x2rzWRNW9vb+7evYvJZEqXkN6+fVt63UWeiImJoUOHDjz99NN8+umnWodjU0qVKsXnn39OkSJF2LlzJ506ddI6JKszZcoUGjVqROvWrXN1Hqv+lD1y5AiHDh2iaNGiqY8DBw4waNAgypUrp3V4VstkMmXaw+Lk5CS3ZR7CaDTy8ssvc+vWLTZs2ICrq6vWIQk7Urt2bfR6PWfOnEndd/v2bS5evEidOnU0jEzYo/j4eJ599llKly7N0qVLM+2NF2mSk5Mz7HNwcMDR0VE+N7Nw9OhRVq1alS5Hu3r1Ks8++yxNmzbN9nmsOhFdtGgR9+7d4+7du6mPJk2a8Pnnn3PlyhWtw7NaUVFRNGvWjMOHD6cmpbNmzeLWrVt07txZ6/Cskslk4s033+T8+fP8+uuvMjlCWJyPjw/PP/887777LlFRUcTHxzN06FDq1q1L3bp1tQ5P2BG9Xk+3bt1wcXFh9erVUmUgG06fPk3nzp05ffo0oNpw+PDh+Pj4mJVUFSTbtm0jOjo6XY4WEBDAL7/8wv79+7N9HqtOREXOFC1alIEDBzJy5Eh8fX0pU6YMW7ZsYefOnZQuXVrr8KzS1atXWbJkCZcuXeLxxx9PNw5y6dKlWocn7MSiRYvw8/OjYsWKlC5dmri4ONavX691WMLOHD58mK1bt3Ls2DECAgLS/T3bsWOH1uFZpZo1a9K2bVv69++Pj48PFSpU4MqVK2zfvh13d3etw7NrNlNHVAghhBBC2BfpERVCCCGEEJqQRFQIIYQQQmhCElEhhBBCCKEJSUSFEEIIIYQmJBEVQgghhBCakERUCCGEEEJoQhJRIYQQQgihCUlEhRBCCCGEJiQRFUIIIYQQmpBEVAghhBBCaEISUSGEyAfTp09Hp9NleMydO1fr0IQQQjOy1rwQQuSDe/fuERsbm7o9YcIEtm3bxoEDByhTpoyGkQkhhHactA5ACCEKgiJFilCkSBEAxo8fz7Zt29izZ48koUKIAk1uzQshRD6aMGEC3333HXv27KF8+fJahyOEEJqSRFQIIfLJxIkT+fbbbyUJFUKI/0giKoQQ+WDixIksX75cklAhhLiPjBEVQog8Nm3aNL744gs2btyIq6srYWFhABQrVgwXFxeNoxNCCO3IrHkhhMhDJpOJokWLEh0dneG5o0ePUq9ePQ2iEkII6yCJqBBCCCGE0ISMERVCCCGEEJqQRFQIIYQQQmhCElEhhBBCCKEJSUSFEEIIIYQmJBEVQgghhBCakERUCCGEEEJoQhJRIYQQQgihCUlEhRBCCCGEJiQRFUIIIYQQmpBEVAghhBBCaEISUSGEEEIIoQlJRIUQQgghhCYkERVCCCGEEJr4P6MHSnznfycIAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        }
      ],
      "source": [
        "# 추가 코드 - 후버 손실의 모습을 보여줍니다.\n",
        "\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "plt.figure(figsize=(8, 3.5))\n",
        "z = np.linspace(-4, 4, 200)\n",
        "z_center = np.linspace(-1, 1, 200)\n",
        "plt.plot(z, huber_fn(0, z), \"b-\", linewidth=2, label=\"huber($z$)\")\n",
        "plt.plot(z, z ** 2 / 2, \"r:\", linewidth=1)\n",
        "plt.plot(z_center, z_center ** 2 / 2, \"r\", linewidth=2)\n",
        "plt.plot([-1, -1], [0, huber_fn(0., -1.)], \"k--\")\n",
        "plt.plot([1, 1], [0, huber_fn(0., 1.)], \"k--\")\n",
        "plt.gca().axhline(y=0, color='k')\n",
        "plt.gca().axvline(x=0, color='k')\n",
        "plt.text(2.1, 3.5, r\"$\\frac{1}{2}z^2$\", color=\"r\", fontsize=15)\n",
        "plt.text(3.0, 2.2, r\"$|z| - \\frac{1}{2}$\", color=\"b\", fontsize=15)\n",
        "plt.axis([-4, 4, 0, 4])\n",
        "plt.grid(True)\n",
        "plt.xlabel(\"$z$\")\n",
        "plt.legend(fontsize=14)\n",
        "plt.title(\"후버 손실\", fontsize=14)\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L2CATH3GAxrc"
      },
      "source": [
        "사용자 정의 손실 함수를 테스트하기 위해 기본 케라스 모델을 만들고 캘리포니아 주택 데이터셋으로 훈련해 보겠습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 73,
      "metadata": {
        "id": "H1VF7XsaAxrc",
        "tags": []
      },
      "outputs": [],
      "source": [
        "# 추가 코드 - 캘리포니아 주택 데이터셋을 로드, 분할 및 스케일을 조정한 다음 간단한 케라스 모델을 생성합니다.\n",
        "\n",
        "from sklearn.datasets import fetch_california_housing\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.preprocessing import StandardScaler\n",
        "\n",
        "housing = fetch_california_housing()\n",
        "X_train_full, X_test, y_train_full, y_test = train_test_split(\n",
        "    housing.data, housing.target.reshape(-1, 1), random_state=42)\n",
        "X_train, X_valid, y_train, y_valid = train_test_split(\n",
        "    X_train_full, y_train_full, random_state=42)\n",
        "\n",
        "scaler = StandardScaler()\n",
        "X_train_scaled = scaler.fit_transform(X_train)\n",
        "X_valid_scaled = scaler.transform(X_valid)\n",
        "X_test_scaled = scaler.transform(X_test)\n",
        "\n",
        "input_shape = X_train.shape[1:]\n",
        "\n",
        "tf.keras.utils.set_random_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\", kernel_initializer=\"he_normal\",\n",
        "                          input_shape=input_shape),\n",
        "    tf.keras.layers.Dense(1),\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 74,
      "metadata": {
        "id": "48JLXr2TAxrc"
      },
      "outputs": [],
      "source": [
        "model.compile(loss=huber_fn, optimizer=\"nadam\", metrics=[\"mae\"])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 75,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tjwQXkydAxrc",
        "outputId": "2753c791-c153-424b-eaf6-a03233ab353a",
        "scrolled": true
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 7s 7ms/step - loss: 0.4854 - mae: 0.8352 - val_loss: 0.3478 - val_mae: 0.6526\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 2s 6ms/step - loss: 0.2414 - mae: 0.5418 - val_loss: 0.2630 - val_mae: 0.5472\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7cf0b11b51e0>"
            ]
          },
          "metadata": {},
          "execution_count": 75
        }
      ],
      "source": [
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4-07_kd7Axrd"
      },
      "source": [
        "## 사용자 정의 요소를 가진 모델을 저장하고 로드하기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 76,
      "metadata": {
        "id": "Q7tph7CeAxrd"
      },
      "outputs": [],
      "source": [
        "model.save(\"my_model_with_a_custom_loss\")  # 추가 코드 - 정상적으로 저장됩니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 77,
      "metadata": {
        "id": "AB2rQdS1Axrd"
      },
      "outputs": [],
      "source": [
        "model = tf.keras.models.load_model(\"my_model_with_a_custom_loss\",\n",
        "                                   custom_objects={\"huber_fn\": huber_fn})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 78,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NDkbf9O8Axrd",
        "outputId": "ac372378-99a4-46ae-cfc2-f183c0dbb4c6"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 4s 5ms/step - loss: 0.2052 - mae: 0.4910 - val_loss: 0.2210 - val_mae: 0.4946\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 3ms/step - loss: 0.1888 - mae: 0.4682 - val_loss: 0.2023 - val_mae: 0.4775\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7cf0a2705f90>"
            ]
          },
          "metadata": {},
          "execution_count": 78
        }
      ],
      "source": [
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 79,
      "metadata": {
        "id": "mJH4gpIDAxrd"
      },
      "outputs": [],
      "source": [
        "def create_huber(threshold=1.0):\n",
        "    def huber_fn(y_true, y_pred):\n",
        "        error = y_true - y_pred\n",
        "        is_small_error = tf.abs(error) < threshold\n",
        "        squared_loss = tf.square(error) / 2\n",
        "        linear_loss  = threshold * tf.abs(error) - threshold ** 2 / 2\n",
        "        return tf.where(is_small_error, squared_loss, linear_loss)\n",
        "    return huber_fn"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 80,
      "metadata": {
        "id": "T3d5H-VFAxrd"
      },
      "outputs": [],
      "source": [
        "model.compile(loss=create_huber(2.0), optimizer=\"nadam\", metrics=[\"mae\"])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 81,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BFdizsdgAxrd",
        "outputId": "4b50ecbf-9bb1-48d9-d434-f69a9897e9a1",
        "scrolled": true
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 3s 5ms/step - loss: 0.2051 - mae: 0.4598 - val_loss: 0.2253 - val_mae: 0.4583\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 3ms/step - loss: 0.1982 - mae: 0.4531 - val_loss: 0.2031 - val_mae: 0.4526\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7cf0b558d660>"
            ]
          },
          "metadata": {},
          "execution_count": 81
        }
      ],
      "source": [
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 82,
      "metadata": {
        "id": "59g4fFXUAxrd"
      },
      "outputs": [],
      "source": [
        "model.save(\"my_model_with_a_custom_loss_threshold_2\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 83,
      "metadata": {
        "id": "_Kq6XfV1Axrd"
      },
      "outputs": [],
      "source": [
        "model = tf.keras.models.load_model(\"my_model_with_a_custom_loss_threshold_2\",\n",
        "                                   custom_objects={\"huber_fn\": create_huber(2.0)})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 84,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VZM9Q3aSAxre",
        "outputId": "e3a68fe2-47fc-4cb9-b9cd-f814b7ceb550"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 2s 4ms/step - loss: 0.1935 - mae: 0.4465 - val_loss: 0.2019 - val_mae: 0.4410\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 4ms/step - loss: 0.1899 - mae: 0.4422 - val_loss: 0.1869 - val_mae: 0.4400\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7cf0b4ac6da0>"
            ]
          },
          "metadata": {},
          "execution_count": 84
        }
      ],
      "source": [
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 85,
      "metadata": {
        "id": "Geic7wZ9Axre"
      },
      "outputs": [],
      "source": [
        "class HuberLoss(tf.keras.losses.Loss):\n",
        "    def __init__(self, threshold=1.0, **kwargs):\n",
        "        self.threshold = threshold\n",
        "        super().__init__(**kwargs)\n",
        "\n",
        "    def call(self, y_true, y_pred):\n",
        "        error = y_true - y_pred\n",
        "        is_small_error = tf.abs(error) < self.threshold\n",
        "        squared_loss = tf.square(error) / 2\n",
        "        linear_loss  = self.threshold * tf.abs(error) - self.threshold**2 / 2\n",
        "        return tf.where(is_small_error, squared_loss, linear_loss)\n",
        "\n",
        "    def get_config(self):\n",
        "        base_config = super().get_config()\n",
        "        return {**base_config, \"threshold\": self.threshold}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 86,
      "metadata": {
        "id": "dq0XagVZAxre"
      },
      "outputs": [],
      "source": [
        "# 추가 코드 - 간단한 케라스 모델을 또 하나 생성합니다.\n",
        "tf.keras.utils.set_random_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\", kernel_initializer=\"he_normal\",\n",
        "                          input_shape=input_shape),\n",
        "    tf.keras.layers.Dense(1),\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 87,
      "metadata": {
        "id": "YU2e8QsUAxre"
      },
      "outputs": [],
      "source": [
        "model.compile(loss=HuberLoss(2.), optimizer=\"nadam\", metrics=[\"mae\"])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 88,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1sav9-HsAxre",
        "outputId": "327f49e0-0ee8-46d6-b13b-f3d0d8790251"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 2s 4ms/step - loss: 0.6486 - mae: 0.8463 - val_loss: 0.5092 - val_mae: 0.6722\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 2s 5ms/step - loss: 0.2911 - mae: 0.5551 - val_loss: 0.3714 - val_mae: 0.5681\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7cf0b486acb0>"
            ]
          },
          "metadata": {},
          "execution_count": 88
        }
      ],
      "source": [
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 89,
      "metadata": {
        "id": "hWAYNxvXAxre"
      },
      "outputs": [],
      "source": [
        "model.save(\"my_model_with_a_custom_loss_class\")  # 추가 코드 - 정상 저장"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 90,
      "metadata": {
        "id": "r7u-ejM6Axre"
      },
      "outputs": [],
      "source": [
        "model = tf.keras.models.load_model(\"my_model_with_a_custom_loss_class\",\n",
        "                                   custom_objects={\"HuberLoss\": HuberLoss})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 91,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Dj4Ng3SlAxre",
        "outputId": "ac8eec05-d41c-4f34-b5dc-18b76dc6bd66"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 2s 4ms/step - loss: 0.2415 - mae: 0.5034 - val_loss: 0.2923 - val_mae: 0.5057\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 4ms/step - loss: 0.2173 - mae: 0.4774 - val_loss: 0.2504 - val_mae: 0.4843\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7cf0b5270f40>"
            ]
          },
          "metadata": {},
          "execution_count": 91
        }
      ],
      "source": [
        "# 추가 코드 - 정상적으로 로드되고 모델을 정상적으로 사용할 수 있음을 보여줍니다.\n",
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 92,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YSN4u6loAxre",
        "outputId": "5a02fa61-a6f7-4222-9298-2f8abba1b928"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2.0"
            ]
          },
          "metadata": {},
          "execution_count": 92
        }
      ],
      "source": [
        "model.loss.threshold  # 추가 코드 - threshold가 올바르게 로드되었습니다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XJYIUz1bAxrf"
      },
      "source": [
        "## 기타 사용자 정의 기능"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 93,
      "metadata": {
        "id": "KgqccYnMAxrf"
      },
      "outputs": [],
      "source": [
        "def my_softplus(z):\n",
        "    return tf.math.log(1.0 + tf.exp(z))\n",
        "\n",
        "def my_glorot_initializer(shape, dtype=tf.float32):\n",
        "    stddev = tf.sqrt(2. / (shape[0] + shape[1]))\n",
        "    return tf.random.normal(shape, stddev=stddev, dtype=dtype)\n",
        "\n",
        "def my_l1_regularizer(weights):\n",
        "    return tf.reduce_sum(tf.abs(0.01 * weights))\n",
        "\n",
        "def my_positive_weights(weights):  # 반환 값은 tf.nn.relu(weights)입니다.\n",
        "    return tf.where(weights < 0., tf.zeros_like(weights), weights)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 94,
      "metadata": {
        "id": "AA-eG-A6Axrf"
      },
      "outputs": [],
      "source": [
        "layer = tf.keras.layers.Dense(1, activation=my_softplus,\n",
        "                              kernel_initializer=my_glorot_initializer,\n",
        "                              kernel_regularizer=my_l1_regularizer,\n",
        "                              kernel_constraint=my_positive_weights)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 95,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4tcdzl_bAxrf",
        "outputId": "16609724-8429-45bf-ed1b-0e9adadb7c9f"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 3s 4ms/step - loss: 1.4714 - mae: 0.8316 - val_loss: inf - val_mae: inf\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 2s 4ms/step - loss: 0.8095 - mae: 0.6172 - val_loss: 2.6156 - val_mae: 0.6058\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/keras/src/initializers/__init__.py:144: UserWarning: The `keras.initializers.serialize()` API should only be used for objects of type `keras.initializers.Initializer`. Found an instance of type <class 'function'>, which may lead to improper serialization.\n",
            "  warnings.warn(\n",
            "/usr/local/lib/python3.10/dist-packages/keras/src/regularizers.py:426: UserWarning: The `keras.regularizers.serialize()` API should only be used for objects of type `keras.regularizers.Regularizer`. Found an instance of type <class 'function'>, which may lead to improper serialization.\n",
            "  warnings.warn(\n",
            "/usr/local/lib/python3.10/dist-packages/keras/src/constraints.py:365: UserWarning: The `keras.constraints.serialize()` API should only be used for objects of type `keras.constraints.Constraint`. Found an instance of type <class 'function'>, which may lead to improper serialization.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 3s 4ms/step - loss: 0.6333 - mae: 0.5617 - val_loss: 1.1704 - val_mae: 0.5469\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 4ms/step - loss: 0.5570 - mae: 0.5303 - val_loss: 1.0443 - val_mae: 0.5250\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7cf0b072ca60>"
            ]
          },
          "metadata": {},
          "execution_count": 95
        }
      ],
      "source": [
        "# 추가 코드 - 사용자 정의 요소가 많이 포함된 모델을 빌드, 훈련, 저장, 로드, 훈련할 수 있습니다.\n",
        "\n",
        "tf.keras.utils.set_random_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\", kernel_initializer=\"he_normal\",\n",
        "                          input_shape=input_shape),\n",
        "    tf.keras.layers.Dense(1, activation=my_softplus,\n",
        "                          kernel_initializer=my_glorot_initializer,\n",
        "                          kernel_regularizer=my_l1_regularizer,\n",
        "                          kernel_constraint=my_positive_weights)\n",
        "])\n",
        "model.compile(loss=\"mse\", optimizer=\"nadam\", metrics=[\"mae\"])\n",
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))\n",
        "model.save(\"my_model_with_many_custom_parts\")\n",
        "model = tf.keras.models.load_model(\n",
        "    \"my_model_with_many_custom_parts\",\n",
        "    custom_objects={\n",
        "       \"my_l1_regularizer\": my_l1_regularizer,\n",
        "       \"my_positive_weights\": my_positive_weights,\n",
        "       \"my_glorot_initializer\": my_glorot_initializer,\n",
        "       \"my_softplus\": my_softplus,\n",
        "    }\n",
        ")\n",
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 96,
      "metadata": {
        "id": "Tm1vXkgqAxrf"
      },
      "outputs": [],
      "source": [
        "class MyL1Regularizer(tf.keras.regularizers.Regularizer):\n",
        "    def __init__(self, factor):\n",
        "        self.factor = factor\n",
        "\n",
        "    def __call__(self, weights):\n",
        "        return tf.reduce_sum(tf.abs(self.factor * weights))\n",
        "\n",
        "    def get_config(self):\n",
        "        return {\"factor\": self.factor}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 97,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ou_qdhJKAxrf",
        "outputId": "2364e517-ff77-465b-af78-3add234f0602"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 2s 4ms/step - loss: 1.4714 - mae: 0.8316 - val_loss: inf - val_mae: inf\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 4ms/step - loss: 0.8095 - mae: 0.6172 - val_loss: 2.6156 - val_mae: 0.6058\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/keras/src/initializers/__init__.py:144: UserWarning: The `keras.initializers.serialize()` API should only be used for objects of type `keras.initializers.Initializer`. Found an instance of type <class 'function'>, which may lead to improper serialization.\n",
            "  warnings.warn(\n",
            "/usr/local/lib/python3.10/dist-packages/keras/src/constraints.py:365: UserWarning: The `keras.constraints.serialize()` API should only be used for objects of type `keras.constraints.Constraint`. Found an instance of type <class 'function'>, which may lead to improper serialization.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 3s 4ms/step - loss: 0.6333 - mae: 0.5617 - val_loss: 1.1704 - val_mae: 0.5469\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 4ms/step - loss: 0.5570 - mae: 0.5303 - val_loss: 1.0443 - val_mae: 0.5250\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7cf0b4bf3ca0>"
            ]
          },
          "metadata": {},
          "execution_count": 97
        }
      ],
      "source": [
        "# 추가 코드 - 이번에는 사용자 정의 규제 클래스를 사용하는 경우 정상적으로 작동하는지 보여줍니다.\n",
        "\n",
        "tf.keras.utils.set_random_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\", kernel_initializer=\"he_normal\",\n",
        "                          input_shape=input_shape),\n",
        "    tf.keras.layers.Dense(1, activation=my_softplus,\n",
        "                          kernel_regularizer=MyL1Regularizer(0.01),\n",
        "                          kernel_constraint=my_positive_weights,\n",
        "                          kernel_initializer=my_glorot_initializer),\n",
        "])\n",
        "model.compile(loss=\"mse\", optimizer=\"nadam\", metrics=[\"mae\"])\n",
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))\n",
        "model.save(\"my_model_with_many_custom_parts\")\n",
        "model = tf.keras.models.load_model(\n",
        "    \"my_model_with_many_custom_parts\",\n",
        "    custom_objects={\n",
        "       \"MyL1Regularizer\": MyL1Regularizer,\n",
        "       \"my_positive_weights\": my_positive_weights,\n",
        "       \"my_glorot_initializer\": my_glorot_initializer,\n",
        "       \"my_softplus\": my_softplus,\n",
        "    }\n",
        ")\n",
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RXmm0A9rAxrf"
      },
      "source": [
        "## 사용자 정의 지표"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 98,
      "metadata": {
        "id": "xnKzilGhAxrf"
      },
      "outputs": [],
      "source": [
        "# 추가 코드 - 다시 한 번 기본 케라스 모델을 만들어 보겠습니다.\n",
        "tf.keras.utils.set_random_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\", kernel_initializer=\"he_normal\",\n",
        "                          input_shape=input_shape),\n",
        "    tf.keras.layers.Dense(1),\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 99,
      "metadata": {
        "id": "DJPKkv0FAxrg"
      },
      "outputs": [],
      "source": [
        "model.compile(loss=\"mse\", optimizer=\"nadam\", metrics=[create_huber(2.0)])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 100,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "s5sLLfxVAxrg",
        "outputId": "d2680000-5df2-4ada-8ccc-3e793c627912"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 2s 3ms/step - loss: 1.7459 - huber_fn: 0.6840\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 3ms/step - loss: 0.7842 - huber_fn: 0.3136\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7cf0b50020e0>"
            ]
          },
          "metadata": {},
          "execution_count": 100
        }
      ],
      "source": [
        "# 추가 코드 - 사용자 정의 지표로 모델 훈련하기\n",
        "model.fit(X_train_scaled, y_train, epochs=2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7Vh4j8hFAxrg"
      },
      "source": [
        "**노트**: 동일한 함수를 손실과 지표로 사용하면 약간 다른 결과에 놀랄 수 있습니다. 이는 부분적으로 연산이 정확히 같은 순서로 계산되지 않아 미세한 부동 소수점 오류가 발생할 수 있기 때문입니다. 더 중요한 것은 샘플 가중치 또는 클래스 가중치를 사용하는 경우 방정식이 약간 달라진다는 것입니다:\n",
        "* `fit()` 메서드는 에포크 시작 이후 지금까지 본 모든 배치 손실의 평균을 추적합니다. 각 배치 손실은 가중치가 적용된 샘플 손실의 합을 _배치 크기_로 나눈 값입니다. (가중치의 합으로 나눈 것이 아니므로 배치 손실은 손실의 가중 평균이 _아니다_.).\n",
        "* 에포크 시작 이후의 지표는 가중치가 적용된 샘플 손실의 합을 지금까지 본 모든 가중치의 합으로 나눈 값과 같습니다. 즉, 모든 샘플 손실의 가중 평균입니다. 위와 같지 않습니다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z8_GhFUHAxrg"
      },
      "source": [
        "### 스트리밍 지표"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 101,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fzk8uz9qAxrg",
        "outputId": "81c8167b-08db-4caf-f0c9-dd2c87b3ca12"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.8>"
            ]
          },
          "metadata": {},
          "execution_count": 101
        }
      ],
      "source": [
        "precision = tf.keras.metrics.Precision()\n",
        "precision([0, 1, 1, 1, 0, 1, 0, 1], [1, 1, 0, 1, 0, 1, 0, 1])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 102,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "uyltKAhEAxrg",
        "outputId": "23b76ae8-6175-40ea-a2ca-bc93301921f6"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.5>"
            ]
          },
          "metadata": {},
          "execution_count": 102
        }
      ],
      "source": [
        "precision([0, 1, 0, 0, 1, 0, 1, 1], [1, 0, 1, 1, 0, 0, 0, 0])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 103,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EzyuXv7wAxrg",
        "outputId": "6c86916c-eb99-4931-f4a9-b138f93a8624"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.5>"
            ]
          },
          "metadata": {},
          "execution_count": 103
        }
      ],
      "source": [
        "precision.result()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 104,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9CzGj48DAxrg",
        "outputId": "92d92759-f63e-4a57-ff50-4d14cf6b0c54"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Variable 'true_positives:0' shape=(1,) dtype=float32, numpy=array([4.], dtype=float32)>,\n",
              " <tf.Variable 'false_positives:0' shape=(1,) dtype=float32, numpy=array([4.], dtype=float32)>]"
            ]
          },
          "metadata": {},
          "execution_count": 104
        }
      ],
      "source": [
        "precision.variables"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 105,
      "metadata": {
        "id": "XGk8OUh5Axrg"
      },
      "outputs": [],
      "source": [
        "precision.reset_states()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HnU7bMCqAxrg"
      },
      "source": [
        "Creating a streaming metric:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 106,
      "metadata": {
        "id": "wfr8lxBUAxrh"
      },
      "outputs": [],
      "source": [
        "class HuberMetric(tf.keras.metrics.Metric):\n",
        "    def __init__(self, threshold=1.0, **kwargs):\n",
        "        super().__init__(**kwargs)  # 기본 매개변수 처리 (예를 들어, dtype)\n",
        "        self.threshold = threshold\n",
        "        self.huber_fn = create_huber(threshold)\n",
        "        self.total = self.add_weight(\"total\", initializer=\"zeros\")\n",
        "        self.count = self.add_weight(\"count\", initializer=\"zeros\")\n",
        "\n",
        "    def update_state(self, y_true, y_pred, sample_weight=None):\n",
        "        sample_metrics = self.huber_fn(y_true, y_pred)\n",
        "        self.total.assign_add(tf.reduce_sum(sample_metrics))\n",
        "        self.count.assign_add(tf.cast(tf.size(y_true), tf.float32))\n",
        "\n",
        "    def result(self):\n",
        "        return self.total / self.count\n",
        "\n",
        "    def get_config(self):\n",
        "        base_config = super().get_config()\n",
        "        return {**base_config, \"threshold\": self.threshold}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JJUFrZyaAxrh"
      },
      "source": [
        "**추가 자료** – 이 섹션의 나머지 부분에서는 `HuberMetric` 클래스를 테스트하고 `tf.keras.metrics.Mean`를 서브클래싱하는 또 다른 예를 보여줍니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 107,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1CXamD7gAxrh",
        "outputId": "9183c9fb-d27c-4aa3-d2e5-de64adc9ab1b"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=14.0>"
            ]
          },
          "metadata": {},
          "execution_count": 107
        }
      ],
      "source": [
        "m = HuberMetric(2.)\n",
        "\n",
        "# total = 2 * |10 - 2| - 2²/2 = 14\n",
        "# count = 1\n",
        "# result = 14 / 1 = 14\n",
        "m(tf.constant([[2.]]), tf.constant([[10.]]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 108,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "m2gmSzaEAxrh",
        "outputId": "35d5a038-0629-466f-9ecc-b74648a21fb6"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=7.0>"
            ]
          },
          "metadata": {},
          "execution_count": 108
        }
      ],
      "source": [
        "# total = total + (|1 - 0|² / 2) + (2 * |9.25 - 5| - 2² / 2) = 14 + 7 = 21\n",
        "# count = count + 2 = 3\n",
        "# result = total / count = 21 / 3 = 7\n",
        "m(tf.constant([[0.], [5.]]), tf.constant([[1.], [9.25]]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 109,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YtwdTpYDAxrh",
        "outputId": "7494c711-ecc8-4672-efc0-4667740aa06c"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=7.0>"
            ]
          },
          "metadata": {},
          "execution_count": 109
        }
      ],
      "source": [
        "m.result()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 110,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ae19HKP3Axrh",
        "outputId": "e87d37c8-28ed-4403-9051-42517b47d773"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Variable 'total:0' shape=() dtype=float32, numpy=21.0>,\n",
              " <tf.Variable 'count:0' shape=() dtype=float32, numpy=3.0>]"
            ]
          },
          "metadata": {},
          "execution_count": 110
        }
      ],
      "source": [
        "m.variables"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 111,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2yBflIIAAxrh",
        "outputId": "40480b1d-a680-4d9d-9c67-7dcf7be16e53"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Variable 'total:0' shape=() dtype=float32, numpy=0.0>,\n",
              " <tf.Variable 'count:0' shape=() dtype=float32, numpy=0.0>]"
            ]
          },
          "metadata": {},
          "execution_count": 111
        }
      ],
      "source": [
        "m.reset_states()\n",
        "m.variables"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IHirmthOAxrh"
      },
      "source": [
        "`HuberMetric` 클래스가 제대로 작동하는지 확인해 보겠습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 112,
      "metadata": {
        "id": "3AH5XvvXAxrh"
      },
      "outputs": [],
      "source": [
        "tf.keras.utils.set_random_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\", kernel_initializer=\"he_normal\",\n",
        "                          input_shape=input_shape),\n",
        "    tf.keras.layers.Dense(1),\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 113,
      "metadata": {
        "id": "m5ExIJ5oAxri"
      },
      "outputs": [],
      "source": [
        "model.compile(loss=create_huber(2.0), optimizer=\"nadam\",\n",
        "              metrics=[HuberMetric(2.0)])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 114,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wU0qMrn2Axri",
        "outputId": "a103665d-9587-427e-8952-6a24d8845dfc"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 3s 4ms/step - loss: 0.6486 - huber_metric_1: 0.6486\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 3ms/step - loss: 0.2911 - huber_metric_1: 0.2911\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7cf0b5550d60>"
            ]
          },
          "metadata": {},
          "execution_count": 114
        }
      ],
      "source": [
        "model.fit(X_train_scaled, y_train, epochs=2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 115,
      "metadata": {
        "id": "d38H2hPkAxri"
      },
      "outputs": [],
      "source": [
        "model.save(\"my_model_with_a_custom_metric\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 116,
      "metadata": {
        "id": "Fm1d2JgSAxri"
      },
      "outputs": [],
      "source": [
        "model = tf.keras.models.load_model(\n",
        "    \"my_model_with_a_custom_metric\",\n",
        "    custom_objects={\n",
        "        \"huber_fn\": create_huber(2.0),\n",
        "        \"HuberMetric\": HuberMetric\n",
        "    }\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 117,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4hsey5XSAxri",
        "outputId": "790e9af8-e685-4a18-c95f-5799c8bd733e"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 2s 3ms/step - loss: 0.2415 - huber_metric_1: 0.2415\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 3ms/step - loss: 0.2173 - huber_metric_1: 0.2173\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7cf0b0ceaf80>"
            ]
          },
          "metadata": {},
          "execution_count": 117
        }
      ],
      "source": [
        "model.fit(X_train_scaled, y_train, epochs=2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dAZg4651Axri"
      },
      "source": [
        "`model.metrics`은 모델 손실 다음에 모델의 지표를 담고 있으므로 `HuberMetric`은 `model.metrics[-1]`입니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 118,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7bIynYbPAxri",
        "outputId": "6e927786-4f9f-481d-bc7c-2cb7021b1bf2"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2.0"
            ]
          },
          "metadata": {},
          "execution_count": 118
        }
      ],
      "source": [
        "model.metrics[-1].threshold"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UZZVBn1lAxri"
      },
      "source": [
        "잘 작동하는 것 같습니다! 더 간단하게는 다음과 같이 클래스를 만들 수도 있습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 119,
      "metadata": {
        "id": "uB8h98UCAxri"
      },
      "outputs": [],
      "source": [
        "class HuberMetric(tf.keras.metrics.Mean):\n",
        "    def __init__(self, threshold=1.0, name='HuberMetric', dtype=None):\n",
        "        self.threshold = threshold\n",
        "        self.huber_fn = create_huber(threshold)\n",
        "        super().__init__(name=name, dtype=dtype)\n",
        "\n",
        "    def update_state(self, y_true, y_pred, sample_weight=None):\n",
        "        metric = self.huber_fn(y_true, y_pred)\n",
        "        super(HuberMetric, self).update_state(metric, sample_weight)\n",
        "\n",
        "    def get_config(self):\n",
        "        base_config = super().get_config()\n",
        "        return {**base_config, \"threshold\": self.threshold}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tbYo0q7gAxrj"
      },
      "source": [
        "이 클래스는 크기를 더 잘 처리하며 샘플 가중치도 지원합니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 120,
      "metadata": {
        "id": "KwwfYIUzAxrj"
      },
      "outputs": [],
      "source": [
        "tf.keras.utils.set_random_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\", kernel_initializer=\"he_normal\",\n",
        "                          input_shape=input_shape),\n",
        "    tf.keras.layers.Dense(1),\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 121,
      "metadata": {
        "id": "mQFwmPxvAxrj"
      },
      "outputs": [],
      "source": [
        "model.compile(loss=tf.keras.losses.Huber(2.0), optimizer=\"nadam\",\n",
        "              weighted_metrics=[HuberMetric(2.0)])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 122,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1Z760c4eAxrj",
        "outputId": "d64def5c-6e1e-4dca-8efd-82ef91cc3948",
        "scrolled": true
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 2s 3ms/step - loss: 0.3269 - HuberMetric: 0.6587\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 4ms/step - loss: 0.1448 - HuberMetric: 0.2918\n"
          ]
        }
      ],
      "source": [
        "np.random.seed(42)\n",
        "sample_weight = np.random.rand(len(y_train))\n",
        "history = model.fit(X_train_scaled, y_train, epochs=2,\n",
        "                    sample_weight=sample_weight)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 123,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tiohRlrmAxrj",
        "outputId": "8b9ac454-5508-4575-e35a-4d35899c4bcc"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(0.32688796520233154, 0.3268879972310725)"
            ]
          },
          "metadata": {},
          "execution_count": 123
        }
      ],
      "source": [
        "(history.history[\"loss\"][0],\n",
        " history.history[\"HuberMetric\"][0] * sample_weight.mean())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 124,
      "metadata": {
        "id": "ApXwk937Axrj"
      },
      "outputs": [],
      "source": [
        "model.save(\"my_model_with_a_custom_metric_v2\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 125,
      "metadata": {
        "id": "BIK-TcXEAxrj"
      },
      "outputs": [],
      "source": [
        "model = tf.keras.models.load_model(\"my_model_with_a_custom_metric_v2\",\n",
        "                                   custom_objects={\"HuberMetric\": HuberMetric})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 126,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DA9CXZgTAxrj",
        "outputId": "17605def-c6ad-49e3-cdb3-74c858c9a032"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 2s 3ms/step - loss: 0.2441 - HuberMetric: 0.2441\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 3ms/step - loss: 0.2184 - HuberMetric: 0.2184\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7cf0afc2e260>"
            ]
          },
          "metadata": {},
          "execution_count": 126
        }
      ],
      "source": [
        "model.fit(X_train_scaled, y_train, epochs=2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 127,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YGc3D2KbAxrk",
        "outputId": "f1c30cac-e663-45ad-b704-c27053ccf110",
        "scrolled": true
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2.0"
            ]
          },
          "metadata": {},
          "execution_count": 127
        }
      ],
      "source": [
        "model.metrics[-1].threshold"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LHEBZvKPAxrk"
      },
      "source": [
        "## 사용자 정의 층"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 128,
      "metadata": {
        "id": "z1_acb9zAxrk"
      },
      "outputs": [],
      "source": [
        "exponential_layer = tf.keras.layers.Lambda(lambda x: tf.exp(x))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 129,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "brtmfdE-Axrk",
        "outputId": "00c7c35f-f206-4662-f973-4bdcddeda930"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3,), dtype=float32, numpy=array([0.36787948, 1.        , 2.7182817 ], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 129
        }
      ],
      "source": [
        "# 추가 코드 - 다른 층과 마찬가지로 함수로 사용할 수 있습니다:\n",
        "exponential_layer([-1., 0., 1.])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xn6MwQRAAxrk"
      },
      "source": [
        "회귀 모델의 출력에 지수 층을 추가하면 예측할 값이 양수이고 스케일이 매우 다른 경우(예: 0.001, 10., 10000) 유용할 수 있습니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 130,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4YY1BOGbAxrk",
        "outputId": "ff78e445-e90b-4956-c6ed-82be086622e2"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/5\n",
            "363/363 [==============================] - 2s 3ms/step - loss: 0.7784 - val_loss: 0.4393\n",
            "Epoch 2/5\n",
            "363/363 [==============================] - 1s 3ms/step - loss: 0.5702 - val_loss: 0.4094\n",
            "Epoch 3/5\n",
            "363/363 [==============================] - 1s 3ms/step - loss: 0.4431 - val_loss: 0.3760\n",
            "Epoch 4/5\n",
            "363/363 [==============================] - 1s 3ms/step - loss: 0.4988 - val_loss: 0.3788\n",
            "Epoch 5/5\n",
            "363/363 [==============================] - 1s 3ms/step - loss: 0.3966 - val_loss: 0.3635\n",
            "162/162 [==============================] - 0s 2ms/step - loss: 0.3783\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.3782573938369751"
            ]
          },
          "metadata": {},
          "execution_count": 130
        }
      ],
      "source": [
        "tf.keras.utils.set_random_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\", input_shape=input_shape),\n",
        "    tf.keras.layers.Dense(1),\n",
        "    exponential_layer\n",
        "])\n",
        "model.compile(loss=\"mse\", optimizer=\"sgd\")\n",
        "model.fit(X_train_scaled, y_train, epochs=5,\n",
        "          validation_data=(X_valid_scaled, y_valid))\n",
        "model.evaluate(X_test_scaled, y_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gSgHQIJ6Axrk"
      },
      "source": [
        "또는 타깃을 타깃의 로그로 대체하고 출력 층에서 활성화 함수를 사용하지 않는 것이 더 나을 수 있습니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 131,
      "metadata": {
        "id": "vVR13Y31Axrk"
      },
      "outputs": [],
      "source": [
        "class MyDense(tf.keras.layers.Layer):\n",
        "    def __init__(self, units, activation=None, **kwargs):\n",
        "        super().__init__(**kwargs)\n",
        "        self.units = units\n",
        "        self.activation = tf.keras.activations.get(activation)\n",
        "\n",
        "    def build(self, batch_input_shape):\n",
        "        self.kernel = self.add_weight(\n",
        "            name=\"kernel\", shape=[batch_input_shape[-1], self.units],\n",
        "            initializer=\"he_normal\")\n",
        "        self.bias = self.add_weight(\n",
        "            name=\"bias\", shape=[self.units], initializer=\"zeros\")\n",
        "\n",
        "    def call(self, X):\n",
        "        return self.activation(X @ self.kernel + self.bias)\n",
        "\n",
        "    def get_config(self):\n",
        "        base_config = super().get_config()\n",
        "        return {**base_config, \"units\": self.units,\n",
        "                \"activation\": tf.keras.activations.serialize(self.activation)}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 132,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_Lh96KyuAxrk",
        "outputId": "563992cb-0f9a-4134-8ce1-86d50c4ab89b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 3s 4ms/step - loss: 3.1139 - val_loss: 6.9545\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 4ms/step - loss: 0.8698 - val_loss: 3.2633\n",
            "162/162 [==============================] - 0s 2ms/step - loss: 0.7037\n"
          ]
        }
      ],
      "source": [
        "# 추가 코드 - 사용자 정의 층을 보통처럼 사용할 수 있음을 보여줍니다.\n",
        "tf.keras.utils.set_random_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    MyDense(30, activation=\"relu\", input_shape=input_shape),\n",
        "    MyDense(1)\n",
        "])\n",
        "model.compile(loss=\"mse\", optimizer=\"nadam\")\n",
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))\n",
        "model.evaluate(X_test_scaled, y_test)\n",
        "model.save(\"my_model_with_a_custom_layer\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 133,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OMJn9UA7Axrk",
        "outputId": "e2e4b98b-54d6-48cc-fe1c-feeb5c08bcfb"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 2s 4ms/step - loss: 0.5943 - val_loss: 0.5311\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 4ms/step - loss: 0.4711 - val_loss: 0.5754\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7cf0b01ddb70>"
            ]
          },
          "metadata": {},
          "execution_count": 133
        }
      ],
      "source": [
        "# 추가 코드 - 사용자 정의 층을 가진 모델을 로드하는 방법을 보여줍니다.\n",
        "model = tf.keras.models.load_model(\"my_model_with_a_custom_layer\",\n",
        "                                   custom_objects={\"MyDense\": MyDense})\n",
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 134,
      "metadata": {
        "id": "8I7ct6m1Axrl"
      },
      "outputs": [],
      "source": [
        "class MyMultiLayer(tf.keras.layers.Layer):\n",
        "    def call(self, X):\n",
        "        X1, X2 = X\n",
        "        print(\"X1.shape: \", X1.shape ,\" X2.shape: \", X2.shape)  # extra code\n",
        "        return X1 + X2, X1 * X2, X1 / X2"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wuI7HeD9Axrl"
      },
      "source": [
        "사용자 정의 층은 다음과 같이 함수형 API를 사용하여 호출할 수 있습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 135,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "H6X91b3DAxrl",
        "outputId": "eef9d527-7694-4b80-c675-7ef09edaf877"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "X1.shape:  (None, 2)  X2.shape:  (None, 2)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<KerasTensor: shape=(None, 2) dtype=float32 (created by layer 'my_multi_layer')>,\n",
              " <KerasTensor: shape=(None, 2) dtype=float32 (created by layer 'my_multi_layer')>,\n",
              " <KerasTensor: shape=(None, 2) dtype=float32 (created by layer 'my_multi_layer')>)"
            ]
          },
          "metadata": {},
          "execution_count": 135
        }
      ],
      "source": [
        "# 추가 코드 - 심볼릭 입력으로 MyMultiLayer를 테스트합니다.\n",
        "inputs1 = tf.keras.layers.Input(shape=[2])\n",
        "inputs2 = tf.keras.layers.Input(shape=[2])\n",
        "MyMultiLayer()((inputs1, inputs2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e_tIIUkAAxrl"
      },
      "source": [
        "`call()` 메서드는 심볼릭 입력을 받고 심볼릭 출력을 반환한다는 점에 유의하세요. 이 단계에서는 부분적인 크기만 지정합니다. 배치 크기를 알 수 없기 때문에 첫 번째 차원이 `None`입니다.\n",
        "\n",
        "실제 데이터를 사용자 정의 층에 전달할 수도 있습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 136,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kvY8ufHVAxrl",
        "outputId": "a36e5d2e-0e58-4d96-ab83-74842c77d4d4"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "X1.shape:  (2, 2)  X2.shape:  (2, 2)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2, 2), dtype=float32, numpy=\n",
              " array([[ 9., 18.],\n",
              "        [ 6., 10.]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(2, 2), dtype=float32, numpy=\n",
              " array([[18., 72.],\n",
              "        [ 8., 21.]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(2, 2), dtype=float32, numpy=\n",
              " array([[0.5      , 0.5      ],\n",
              "        [0.5      , 2.3333333]], dtype=float32)>)"
            ]
          },
          "metadata": {},
          "execution_count": 136
        }
      ],
      "source": [
        "# 추가 코드 - 실제 데이터로 MyMultiLayer를 테스트합니다.\n",
        "X1, X2 = np.array([[3., 6.], [2., 7.]]), np.array([[6., 12.], [4., 3.]])\n",
        "MyMultiLayer()((X1, X2))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MofR-d5TAxrl"
      },
      "source": [
        "이제 훈련과 테스트 중에 다르게 동작하는 층을 만들어 보겠습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 137,
      "metadata": {
        "id": "YJkH97AEAxrl"
      },
      "outputs": [],
      "source": [
        "class MyGaussianNoise(tf.keras.layers.Layer):\n",
        "    def __init__(self, stddev, **kwargs):\n",
        "        super().__init__(**kwargs)\n",
        "        self.stddev = stddev\n",
        "\n",
        "    def call(self, X, training=None):\n",
        "        if training:\n",
        "            noise = tf.random.normal(tf.shape(X), stddev=self.stddev)\n",
        "            return X + noise\n",
        "        else:\n",
        "            return X\n",
        "\n",
        "    def compute_output_shape(self, batch_input_shape):\n",
        "        return batch_input_shape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6IPN9u64Axrl"
      },
      "source": [
        "다음은 이 사용자 정의 층을 사용하는 간단한 모델입니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 138,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "j96OkK2AAxrl",
        "outputId": "bcae4336-6d71-4f3c-c697-5f1764885f67"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 4s 4ms/step - loss: 2.2206 - val_loss: 25.1337\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 1s 3ms/step - loss: 1.4100 - val_loss: 17.0288\n",
            "162/162 [==============================] - 0s 2ms/step - loss: 1.1054\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.1053950786590576"
            ]
          },
          "metadata": {},
          "execution_count": 138
        }
      ],
      "source": [
        "# 추가 코드 - MyGaussianNoise 테스트\n",
        "tf.keras.utils.set_random_seed(42)\n",
        "model = tf.keras.Sequential([\n",
        "    MyGaussianNoise(stddev=1.0, input_shape=input_shape),\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\",\n",
        "                          kernel_initializer=\"he_normal\"),\n",
        "    tf.keras.layers.Dense(1)\n",
        "])\n",
        "model.compile(loss=\"mse\", optimizer=\"nadam\")\n",
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))\n",
        "model.evaluate(X_test_scaled, y_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZNNjOu3IAxrm"
      },
      "source": [
        "## 사용자 정의 모델"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 139,
      "metadata": {
        "id": "Tdb4LT1DAxrm"
      },
      "outputs": [],
      "source": [
        "class ResidualBlock(tf.keras.layers.Layer):\n",
        "    def __init__(self, n_layers, n_neurons, **kwargs):\n",
        "        super().__init__(**kwargs)\n",
        "        self.hidden = [tf.keras.layers.Dense(n_neurons, activation=\"relu\",\n",
        "                                             kernel_initializer=\"he_normal\")\n",
        "                       for _ in range(n_layers)]\n",
        "\n",
        "    def call(self, inputs):\n",
        "        Z = inputs\n",
        "        for layer in self.hidden:\n",
        "            Z = layer(Z)\n",
        "        return inputs + Z"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 140,
      "metadata": {
        "id": "9j7Am2wXAxrm"
      },
      "outputs": [],
      "source": [
        "class ResidualRegressor(tf.keras.Model):\n",
        "    def __init__(self, output_dim, **kwargs):\n",
        "        super().__init__(**kwargs)\n",
        "        self.hidden1 = tf.keras.layers.Dense(30, activation=\"relu\",\n",
        "                                             kernel_initializer=\"he_normal\")\n",
        "        self.block1 = ResidualBlock(2, 30)\n",
        "        self.block2 = ResidualBlock(2, 30)\n",
        "        self.out = tf.keras.layers.Dense(output_dim)\n",
        "\n",
        "    def call(self, inputs):\n",
        "        Z = self.hidden1(inputs)\n",
        "        for _ in range(1 + 3):\n",
        "            Z = self.block1(Z)\n",
        "        Z = self.block2(Z)\n",
        "        return self.out(Z)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 141,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eogB59RYAxrm",
        "outputId": "b48b5ad1-19d2-463e-b84a-2154ed39697f"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 5s 6ms/step - loss: 32.6311\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 3s 10ms/step - loss: 1.3847\n",
            "162/162 [==============================] - 1s 2ms/step - loss: 1.1566\n"
          ]
        }
      ],
      "source": [
        "# 추가 코드 - 모델을 정상적으로 사용할 수 있음을 보여줍니다.\n",
        "tf.keras.utils.set_random_seed(42)\n",
        "model = ResidualRegressor(1)\n",
        "model.compile(loss=\"mse\", optimizer=\"nadam\")\n",
        "history = model.fit(X_train_scaled, y_train, epochs=2)\n",
        "score = model.evaluate(X_test_scaled, y_test)\n",
        "model.save(\"my_custom_model\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 142,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vJWP5N0eAxrm",
        "outputId": "ba1f7729-4621-4c81-e150-26772ec382cd"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "363/363 [==============================] - 4s 5ms/step - loss: 1.3556\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 2s 5ms/step - loss: 0.7283\n",
            "1/1 [==============================] - 0s 91ms/step\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[1.2037897],\n",
              "       [1.5635332],\n",
              "       [4.2166495]], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 142
        }
      ],
      "source": [
        "# 추가 코드 - 모델을 로드하고 훈련을 계속하거나 이를 사용하여 예측을 수행할 수 있습니다.\n",
        "model = tf.keras.models.load_model(\"my_custom_model\")\n",
        "history = model.fit(X_train_scaled, y_train, epochs=2)\n",
        "model.predict(X_test_scaled[:3])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ftCLc4maAxrm"
      },
      "source": [
        "대신 시퀀셜 API를 사용하여 모델을 정의할 수도 있습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 143,
      "metadata": {
        "id": "g7DwZgzgAxrm"
      },
      "outputs": [],
      "source": [
        "tf.keras.utils.set_random_seed(42)\n",
        "block1 = ResidualBlock(2, 30)\n",
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\",\n",
        "                          kernel_initializer=\"he_normal\"),\n",
        "    block1, block1, block1, block1,\n",
        "    ResidualBlock(2, 30),\n",
        "    tf.keras.layers.Dense(1)\n",
        "])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dtgyQ107Axrm"
      },
      "source": [
        "## 모델 구성 요소에 기반한 손실과 지표"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e4xJVV_hAxrm"
      },
      "source": [
        "**경고**: TF 2.2의 문제([#46858](https://github.com/tensorflow/tensorflow/issues/46858))로 인해 `super().build()`가 실패합니다. 대신 `self.built = True`를 설정하여 이 문제를 해결할 수 있습니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 144,
      "metadata": {
        "id": "hw5LlKwGAxrm"
      },
      "outputs": [],
      "source": [
        "class ReconstructingRegressor(tf.keras.Model):\n",
        "    def __init__(self, output_dim, **kwargs):\n",
        "        super().__init__(**kwargs)\n",
        "        self.hidden = [tf.keras.layers.Dense(30, activation=\"relu\",\n",
        "                                             kernel_initializer=\"he_normal\")\n",
        "                       for _ in range(5)]\n",
        "        self.out = tf.keras.layers.Dense(output_dim)\n",
        "        self.reconstruction_mean = tf.keras.metrics.Mean(\n",
        "            name=\"reconstruction_error\")\n",
        "\n",
        "    def build(self, batch_input_shape):\n",
        "        n_inputs = batch_input_shape[-1]\n",
        "        self.reconstruct = tf.keras.layers.Dense(n_inputs)\n",
        "        self.built = True  # super().build(batch_input_shape)에 대한 해결 방법\n",
        "\n",
        "    def call(self, inputs, training=None):\n",
        "        Z = inputs\n",
        "        for layer in self.hidden:\n",
        "            Z = layer(Z)\n",
        "        reconstruction = self.reconstruct(Z)\n",
        "        recon_loss = tf.reduce_mean(tf.square(reconstruction - inputs))\n",
        "        self.add_loss(0.05 * recon_loss)\n",
        "        if training:\n",
        "            result = self.reconstruction_mean(recon_loss)\n",
        "            self.add_metric(result)\n",
        "        return self.out(Z)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 145,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oVJF57b8Axrn",
        "outputId": "5f87c7b8-5614-4f65-d3e5-1507d7246522"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/5\n",
            "363/363 [==============================] - 5s 5ms/step - loss: 0.8115 - reconstruction_error: 1.1001\n",
            "Epoch 2/5\n",
            "363/363 [==============================] - 2s 5ms/step - loss: 0.4983 - reconstruction_error: 0.5885\n",
            "Epoch 3/5\n",
            "363/363 [==============================] - 2s 5ms/step - loss: 0.4232 - reconstruction_error: 0.4467\n",
            "Epoch 4/5\n",
            "363/363 [==============================] - 2s 5ms/step - loss: 0.4303 - reconstruction_error: 0.3725\n",
            "Epoch 5/5\n",
            "363/363 [==============================] - 2s 5ms/step - loss: 0.3851 - reconstruction_error: 0.3284\n",
            "162/162 [==============================] - 0s 2ms/step\n"
          ]
        }
      ],
      "source": [
        "# 추가 코드\n",
        "tf.keras.utils.set_random_seed(42)\n",
        "model = ReconstructingRegressor(1)\n",
        "model.compile(loss=\"mse\", optimizer=\"nadam\")\n",
        "history = model.fit(X_train_scaled, y_train, epochs=5)\n",
        "y_pred = model.predict(X_test_scaled)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gP9ugK0OAxrn"
      },
      "source": [
        "## 자동 미분을 사용해 그레이디언트 계산하기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 146,
      "metadata": {
        "id": "E0es0JCqAxrn"
      },
      "outputs": [],
      "source": [
        "def f(w1, w2):\n",
        "    return 3 * w1 ** 2 + 2 * w1 * w2"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 147,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UWAjkQuUAxrn",
        "outputId": "8636e571-1b4b-47e9-b59c-0414f2beafbd"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "36.000003007075065"
            ]
          },
          "metadata": {},
          "execution_count": 147
        }
      ],
      "source": [
        "w1, w2 = 5, 3\n",
        "eps = 1e-6\n",
        "(f(w1 + eps, w2) - f(w1, w2)) / eps"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 148,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lj0wtEByAxrn",
        "outputId": "f0f33f14-5197-4506-ad4e-0e03c6d61ea3"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "10.000000003174137"
            ]
          },
          "metadata": {},
          "execution_count": 148
        }
      ],
      "source": [
        "(f(w1, w2 + eps) - f(w1, w2)) / eps"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 149,
      "metadata": {
        "id": "-ByM5o2uAxrn"
      },
      "outputs": [],
      "source": [
        "w1, w2 = tf.Variable(5.), tf.Variable(3.)\n",
        "with tf.GradientTape() as tape:\n",
        "    z = f(w1, w2)\n",
        "\n",
        "gradients = tape.gradient(z, [w1, w2])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 150,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CnXg-CRNAxrn",
        "outputId": "a4e21f7f-e58a-4166-c5a0-8ea56085204b"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(), dtype=float32, numpy=36.0>,\n",
              " <tf.Tensor: shape=(), dtype=float32, numpy=10.0>]"
            ]
          },
          "metadata": {},
          "execution_count": 150
        }
      ],
      "source": [
        "gradients"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 151,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3u9cxDqiAxrn",
        "outputId": "af2e5797-c31d-49b5-9c1e-135961d06210"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "A non-persistent GradientTape can only be used to compute one set of gradients (or jacobians)\n"
          ]
        }
      ],
      "source": [
        "with tf.GradientTape() as tape:\n",
        "    z = f(w1, w2)\n",
        "\n",
        "dz_dw1 = tape.gradient(z, w1)  # 텐서 36.0를 반환합니다.\n",
        "try:\n",
        "    dz_dw2 = tape.gradient(z, w2)  # RuntimeError가 발생합니다!\n",
        "except RuntimeError as ex:\n",
        "    print(ex)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 152,
      "metadata": {
        "id": "7ZnWfZUGAxrn"
      },
      "outputs": [],
      "source": [
        "with tf.GradientTape(persistent=True) as tape:\n",
        "    z = f(w1, w2)\n",
        "\n",
        "dz_dw1 = tape.gradient(z, w1)  # 텐서 36.0를 반환합니다.\n",
        "dz_dw2 = tape.gradient(z, w2)  # 텐서 10.0을 반환하고 이제 정상적으로 작동합니다!\n",
        "del tape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 153,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pGo4P-XXAxrn",
        "outputId": "3877ed5d-3984-4a9b-9e47-53842d74702c"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(), dtype=float32, numpy=36.0>,\n",
              " <tf.Tensor: shape=(), dtype=float32, numpy=10.0>)"
            ]
          },
          "metadata": {},
          "execution_count": 153
        }
      ],
      "source": [
        "dz_dw1, dz_dw2"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 154,
      "metadata": {
        "id": "6miw3KASAxrn"
      },
      "outputs": [],
      "source": [
        "c1, c2 = tf.constant(5.), tf.constant(3.)\n",
        "with tf.GradientTape() as tape:\n",
        "    z = f(c1, c2)\n",
        "\n",
        "gradients = tape.gradient(z, [c1, c2])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 155,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0ecoqYDZAxrn",
        "outputId": "334b3278-5527-4235-9ac7-124024cfa36e"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[None, None]"
            ]
          },
          "metadata": {},
          "execution_count": 155
        }
      ],
      "source": [
        "gradients"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 156,
      "metadata": {
        "id": "d1W-u8IvAxro"
      },
      "outputs": [],
      "source": [
        "with tf.GradientTape() as tape:\n",
        "    tape.watch(c1)\n",
        "    tape.watch(c2)\n",
        "    z = f(c1, c2)\n",
        "\n",
        "gradients = tape.gradient(z, [c1, c2])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 157,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "I3vK-4zvAxro",
        "outputId": "c041b23c-7f66-4415-cbcf-4c8620d2dcd4"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(), dtype=float32, numpy=36.0>,\n",
              " <tf.Tensor: shape=(), dtype=float32, numpy=10.0>]"
            ]
          },
          "metadata": {},
          "execution_count": 157
        }
      ],
      "source": [
        "gradients"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 158,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mzqVAdkIAxro",
        "outputId": "2a3a6ae0-0226-4f57-b7cb-9846e61481b7"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(), dtype=float32, numpy=136.0>,\n",
              " <tf.Tensor: shape=(), dtype=float32, numpy=30.0>]"
            ]
          },
          "metadata": {},
          "execution_count": 158
        }
      ],
      "source": [
        "# 추가 코드 - 벡터가 주어지면 tape.gradient()를 사용해 벡터의 합의 그레이디언트을 계산합니다.\n",
        "with tf.GradientTape() as tape:\n",
        "    z1 = f(w1, w2 + 2.)\n",
        "    z2 = f(w1, w2 + 5.)\n",
        "    z3 = f(w1, w2 + 7.)\n",
        "\n",
        "tape.gradient([z1, z2, z3], [w1, w2])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 159,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7MhsExJNAxro",
        "outputId": "195bf5db-f130-41fc-f6d3-08ecd6230cef"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(), dtype=float32, numpy=136.0>,\n",
              " <tf.Tensor: shape=(), dtype=float32, numpy=30.0>]"
            ]
          },
          "metadata": {},
          "execution_count": 159
        }
      ],
      "source": [
        "# 추가 코드 - 이전 셀과 동일한 결과를 얻었음을 보여줍니다.\n",
        "with tf.GradientTape() as tape:\n",
        "    z1 = f(w1, w2 + 2.)\n",
        "    z2 = f(w1, w2 + 5.)\n",
        "    z3 = f(w1, w2 + 7.)\n",
        "    z = z1 + z2 + z3\n",
        "\n",
        "tape.gradient(z, [w1, w2])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 160,
      "metadata": {
        "id": "tkA4n1OJAxro"
      },
      "outputs": [],
      "source": [
        "# 추가 코드 - 야코비안과 헤시안을 계산하는 방법을 보여줍니다.\n",
        "with tf.GradientTape(persistent=True) as hessian_tape:\n",
        "    with tf.GradientTape() as jacobian_tape:\n",
        "        z = f(w1, w2)\n",
        "    jacobians = jacobian_tape.gradient(z, [w1, w2])\n",
        "hessians = [hessian_tape.gradient(jacobian, [w1, w2])\n",
        "            for jacobian in jacobians]\n",
        "del hessian_tape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 161,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "N4BFsV00Axro",
        "outputId": "121f3a06-919d-45ca-afda-e92e047e2a0b"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(), dtype=float32, numpy=36.0>,\n",
              " <tf.Tensor: shape=(), dtype=float32, numpy=10.0>]"
            ]
          },
          "metadata": {},
          "execution_count": 161
        }
      ],
      "source": [
        "jacobians"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 162,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1YXKHV9kAxro",
        "outputId": "741ca1d2-460d-43d3-f69c-f863049d69b4"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[[<tf.Tensor: shape=(), dtype=float32, numpy=6.0>,\n",
              "  <tf.Tensor: shape=(), dtype=float32, numpy=2.0>],\n",
              " [<tf.Tensor: shape=(), dtype=float32, numpy=2.0>, None]]"
            ]
          },
          "metadata": {},
          "execution_count": 162
        }
      ],
      "source": [
        "hessians"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 163,
      "metadata": {
        "id": "nT-SMLBDAxro"
      },
      "outputs": [],
      "source": [
        "def f(w1, w2):\n",
        "    return 3 * w1 ** 2 + tf.stop_gradient(2 * w1 * w2)\n",
        "\n",
        "with tf.GradientTape() as tape:\n",
        "    z = f(w1, w2)  # stop_gradient()를 사용하지 않은 경우와 동일한 결과\n",
        "\n",
        "gradients = tape.gradient(z, [w1, w2])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 164,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Aq8qy9puAxro",
        "outputId": "9e98b33a-e488-4f9e-c80a-dcdb70bac2c0"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(), dtype=float32, numpy=30.0>, None]"
            ]
          },
          "metadata": {},
          "execution_count": 164
        }
      ],
      "source": [
        "gradients"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 165,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fI4KbdJBAxro",
        "outputId": "a19ed05e-e247-49ae-8c17-a35abbe1cd42"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(), dtype=float32, numpy=inf>]"
            ]
          },
          "metadata": {},
          "execution_count": 165
        }
      ],
      "source": [
        "x = tf.Variable(1e-50)\n",
        "with tf.GradientTape() as tape:\n",
        "    z = tf.sqrt(x)\n",
        "\n",
        "tape.gradient(z, [x])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 166,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ISzNiEoOAxro",
        "outputId": "10e79ff8-6f01-437b-fce4-945f074fb201"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=30.0>"
            ]
          },
          "metadata": {},
          "execution_count": 166
        }
      ],
      "source": [
        "tf.math.log(tf.exp(tf.constant(30., dtype=tf.float32)) + 1.)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 167,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qhjNdoqlAxrp",
        "outputId": "ba0e2469-9e10-43a7-bdf2-133626115bd5"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor: shape=(1,), dtype=float32, numpy=array([nan], dtype=float32)>]"
            ]
          },
          "metadata": {},
          "execution_count": 167
        }
      ],
      "source": [
        "x = tf.Variable([1.0e30])\n",
        "with tf.GradientTape() as tape:\n",
        "    z = my_softplus(x)\n",
        "\n",
        "tape.gradient(z, [x])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 168,
      "metadata": {
        "id": "EX9a8h0EAxrp"
      },
      "outputs": [],
      "source": [
        "def my_softplus(z):\n",
        "    return tf.math.log(1 + tf.exp(-tf.abs(z))) + tf.maximum(0., z)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rzYK3EMAAxrp"
      },
      "source": [
        "다음은 이 방정식이 log(1 + exp(_z_))와 같다는 것을 증명하는 예입니다:\n",
        "* softplus(_z_) = log(1 + exp(_z_))\n",
        "* softplus(_z_) = log(1 + exp(_z_)) - log(exp(_z_)) + log(exp(_z_)) ; **같은 값을 더하고 뺍니다**\n",
        "* softplus(_z_) = log\\[(1 + exp(_z_)) / exp(_z_)\\] + log(exp(_z_)) ; **since log(_a_) - log(_b_) = log(_a_ / _b_)**\n",
        "* softplus(_z_) = log\\[(1 + exp(_z_)) / exp(_z_)\\] + _z_ ; **since log(exp(_z_)) = _z_**\n",
        "* softplus(_z_) = log\\[1 / exp(_z_) + exp(_z_) / exp(_z_)\\] + _z_ ; **since (1 + _a_) / _b_ = 1 / _b_ + _a_ / _b_**\n",
        "* softplus(_z_) = log\\[exp(–_z_) + 1\\] + _z_ ; **1 / exp(_z_) = exp(–z)이고 exp(_z_) / exp(_z_) = 1 이므로**\n",
        "* softplus(_z_) = softplus(–_z_) + _z_ ; **맨 위의 정의를 사용합니다. 다만 –_z_ 입니다**\n",
        "* softplus(_z_) = softplus(–|_z_|) + max(0, _z_) ; **_z_ < 0 나 _z_ ≥ 0 인 경우를 고려하면**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 169,
      "metadata": {
        "id": "xDkt0wV5Axrp"
      },
      "outputs": [],
      "source": [
        "@tf.custom_gradient\n",
        "def my_softplus(z):\n",
        "    def my_softplus_gradients(grads):  # grads = 상위 층에서 역전파된 그레이디언트\n",
        "        return grads * (1 - 1 / (1 + tf.exp(z)))  # 안정적인 소프트플러스 그레이디언트\n",
        "\n",
        "    result = tf.math.log(1 + tf.exp(-tf.abs(z))) + tf.maximum(0., z)\n",
        "    return result, my_softplus_gradients"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 170,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HX9qyqsrAxrp",
        "outputId": "ea5fea73-3ea4-4014-a6fc-665eb5e799b6"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(1,), dtype=float32, numpy=array([1000.], dtype=float32)>,\n",
              " [<tf.Tensor: shape=(1,), dtype=float32, numpy=array([1.], dtype=float32)>])"
            ]
          },
          "metadata": {},
          "execution_count": 170
        }
      ],
      "source": [
        "# 추가 코드 - 함수와 이 함수의 그레이디언트가 안정적임을 보여줍니다.\n",
        "x = tf.Variable([1000.])\n",
        "with tf.GradientTape() as tape:\n",
        "    z = my_softplus(x)\n",
        "\n",
        "z, tape.gradient(z, [x])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lAS7G_AdAxrp"
      },
      "source": [
        "## 사용자 정의 훈련"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 171,
      "metadata": {
        "id": "ANc8FNOXAxrp"
      },
      "outputs": [],
      "source": [
        "tf.keras.utils.set_random_seed(42)  # 추가 코드 - 재현성 보장\n",
        "l2_reg = tf.keras.regularizers.l2(0.05)\n",
        "model = tf.keras.models.Sequential([\n",
        "    tf.keras.layers.Dense(30, activation=\"relu\", kernel_initializer=\"he_normal\",\n",
        "                          kernel_regularizer=l2_reg),\n",
        "    tf.keras.layers.Dense(1, kernel_regularizer=l2_reg)\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 172,
      "metadata": {
        "id": "_zmZ0EA9Axrp"
      },
      "outputs": [],
      "source": [
        "def random_batch(X, y, batch_size=32):\n",
        "    idx = np.random.randint(len(X), size=batch_size)\n",
        "    return X[idx], y[idx]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 173,
      "metadata": {
        "id": "tWl1XI8dAxrp"
      },
      "outputs": [],
      "source": [
        "def print_status_bar(step, total, loss, metrics=None):\n",
        "    metrics = \" - \".join([f\"{m.name}: {m.result():.4f}\"\n",
        "                          for m in [loss] + (metrics or [])])\n",
        "    end = \"\" if step < total else \"\\n\"\n",
        "    print(f\"\\r{step}/{total} - \" + metrics, end=end)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 174,
      "metadata": {
        "id": "ki9ew4DnAxrp"
      },
      "outputs": [],
      "source": [
        "tf.keras.utils.set_random_seed(42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 175,
      "metadata": {
        "id": "mR-EpV9IAxrp"
      },
      "outputs": [],
      "source": [
        "n_epochs = 5\n",
        "batch_size = 32\n",
        "n_steps = len(X_train) // batch_size\n",
        "optimizer = tf.keras.optimizers.SGD(learning_rate=0.01)\n",
        "loss_fn = tf.keras.losses.mean_squared_error\n",
        "mean_loss = tf.keras.metrics.Mean()\n",
        "metrics = [tf.keras.metrics.MeanAbsoluteError()]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 176,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gIAbJpB-Axrp",
        "outputId": "316ec157-a49f-4d21-d65c-5b27cf80739e",
        "scrolled": true
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/5\n",
            "362/362 - mean: 3.5419 - mean_absolute_error: 0.6640\n",
            "Epoch 2/5\n",
            "362/362 - mean: 1.8693 - mean_absolute_error: 0.5431\n",
            "Epoch 3/5\n",
            "362/362 - mean: 1.1428 - mean_absolute_error: 0.5030\n",
            "Epoch 4/5\n",
            "362/362 - mean: 0.8501 - mean_absolute_error: 0.4977\n",
            "Epoch 5/5\n",
            "362/362 - mean: 0.7280 - mean_absolute_error: 0.5014\n"
          ]
        }
      ],
      "source": [
        "for epoch in range(1, n_epochs + 1):\n",
        "    print(f\"Epoch {epoch}/{n_epochs}\")\n",
        "    for step in range(1, n_steps + 1):\n",
        "        X_batch, y_batch = random_batch(X_train_scaled, y_train)\n",
        "        with tf.GradientTape() as tape:\n",
        "            y_pred = model(X_batch, training=True)\n",
        "            main_loss = tf.reduce_mean(loss_fn(y_batch, y_pred))\n",
        "            loss = tf.add_n([main_loss] + model.losses)\n",
        "\n",
        "        gradients = tape.gradient(loss, model.trainable_variables)\n",
        "        optimizer.apply_gradients(zip(gradients, model.trainable_variables))\n",
        "\n",
        "        # 추가 코드 - 모델 변수에 제약이 있는 경우\n",
        "        for variable in model.variables:\n",
        "            if variable.constraint is not None:\n",
        "                variable.assign(variable.constraint(variable))\n",
        "\n",
        "        mean_loss(loss)\n",
        "        for metric in metrics:\n",
        "            metric(y_batch, y_pred)\n",
        "\n",
        "        print_status_bar(step, n_steps, mean_loss, metrics)\n",
        "\n",
        "    for metric in [mean_loss] + metrics:\n",
        "        metric.reset_states()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 177,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 209,
          "referenced_widgets": [
            "e593a5241b8d44ed99f448bd431da411",
            "8806ef0fc0c748379a3d56146690b5a2",
            "84a8e9a8661c4779af4dfe3a445a3576",
            "429658200e424f519d88f2e165f7effc",
            "eb71a955064a4c64adf96b629a0ae381",
            "ff54289fba384597848d41674bd53909",
            "e98258ef2daf4b5cb3defa802b95a8f2",
            "ea88e562b283474db0d66c5e8da78e7a",
            "cdc5517881d34975a4ddc499fbd087f7",
            "3a6ae70475ed437cbb643c1048deb00e",
            "b46c44164ffd442185efcea730834f91",
            "f7b2267996574d8986e80c295059bb42",
            "44bccc54092448f99db0566aeb69333d",
            "85dcf484389449929746aa13df982cd1",
            "d54bc5fd12174a7db88eba2c78a0c388",
            "140936ae0ded4165839188989accf0fd",
            "773171184d944860af9335f2ac80ad31",
            "0e2af8454c774354b5e01e4c28d427e2",
            "2bea8139a1644646b1c85e79009afd86",
            "6800c0c05ec54f3caf65888f07a5c6c9",
            "9e62b2b6b4a9482790b281e90147bea7",
            "2e3f1a346844471fb3fab643d6188e26",
            "eaf498e6f7a644d3bfa1d044256067ef",
            "6ae85a4465be4d3f8ad73d047071afd0",
            "02dbebc64fc143fdbaac11688c756529",
            "1bdc5c3357344011b4928ed49ec54c73",
            "17f7097a864a4ff5ab710968ca6f237a",
            "5f9ddae8700a4c26bc0a3c87f15bf590",
            "dea6bb2491e24ab19aef6a26df91e03c",
            "fbac2871c02a4736a4cde808ba031d82",
            "a281e30af9bc435fa6c5d7e7117717d7",
            "89ae1ad5288c4568b10843e09b1a4e19",
            "0765fd15d29f4cb199e4ba2052dc1380",
            "0940d99b57c04f0bab5140980362fcbc",
            "12355cd4f2ce420a8204f1517013a49b",
            "5f1907a04d6f416591d6dfbb8a5c2034",
            "93e0e6fe8a3d4dec9ba4454bfab155e2",
            "91dbeacf3278411e9bf3d0205fa0fd67",
            "ab8aa6b5bafc4789b42f68a7f6b79bc9",
            "ad492c643ab24f1cbbc25dd31f313b50",
            "d1ff579fe62a43a0a27202f53d705c1f",
            "5ae6523218c5402cb84d21cfe559b6e2",
            "8eb343a19b5441d9be04cb8813067880",
            "25e3c54988894854989b216354f1eaae",
            "3a78ba5865d447f68a274e4c73a3a8d4",
            "b05cd9e9b3b64a66b9c406cfd51953e7",
            "f0f2f5bc89cb4d1f98ba694fe1525a9d",
            "55034ab0f57649ee9c7f230a098ea174",
            "42648b1e10dd4ef5be1f09d9b8b1d3d3",
            "64cae0ba53de4ee8b12dfeb1a4bd13fe",
            "1c92ffd2d2044f2889e3d295ca4d2400",
            "5fb7a86014af45a994654bc5d44757be",
            "c4805b48675b4004be8f766176336399",
            "62792003e83240e585f81e5d068e6b6f",
            "68159a1057bf4bd19143b60a15c2ed64",
            "9dc7451db5274a9b822c612b531f2b92",
            "d6afb5651cf54d14bd0b3462470999b9",
            "d4ad7b7008eb4f4b829b76f6e92d8d88",
            "228bfa3e9a714d899fa087dd9f528b03",
            "c26625f17c094e1aac4587a441f10c17",
            "798bd66f76244413aa7fad4be1a5580f",
            "0a96edbe20144016be1776bbed4c0384",
            "c24a35d73f0e4cfd9a10cf5fe8ad7aad",
            "ff290b8f6aeb4d5dafa010459dadb0a2",
            "c171b8b4dd8943ddbb129ef53266bb3f",
            "3e1dbf3276b644888f5414c64dfb1539"
          ]
        },
        "id": "bsXR8eRhAxrq",
        "outputId": "7b1f64ce-1cfe-489f-de2f-7bef492edc74"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "All epochs:   0%|          | 0/5 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "e593a5241b8d44ed99f448bd431da411"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 1/5:   0%|          | 0/362 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "f7b2267996574d8986e80c295059bb42"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 2/5:   0%|          | 0/362 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "eaf498e6f7a644d3bfa1d044256067ef"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 3/5:   0%|          | 0/362 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "0940d99b57c04f0bab5140980362fcbc"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 4/5:   0%|          | 0/362 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "3a78ba5865d447f68a274e4c73a3a8d4"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 5/5:   0%|          | 0/362 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "9dc7451db5274a9b822c612b531f2b92"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "# 추가 코드 - tqdm 패키지를 사용하여 멋진 진행률 표시줄을 표시하는 방법을 보여줍니다.\n",
        "\n",
        "from tqdm.notebook import trange\n",
        "from collections import OrderedDict\n",
        "with trange(1, n_epochs + 1, desc=\"All epochs\") as epochs:\n",
        "    for epoch in epochs:\n",
        "        with trange(1, n_steps + 1, desc=f\"Epoch {epoch}/{n_epochs}\") as steps:\n",
        "            for step in steps:\n",
        "                X_batch, y_batch = random_batch(X_train_scaled, y_train)\n",
        "                with tf.GradientTape() as tape:\n",
        "                    y_pred = model(X_batch)\n",
        "                    main_loss = tf.reduce_mean(loss_fn(y_batch, y_pred))\n",
        "                    loss = tf.add_n([main_loss] + model.losses)\n",
        "\n",
        "                gradients = tape.gradient(loss, model.trainable_variables)\n",
        "                optimizer.apply_gradients(zip(gradients, model.trainable_variables))\n",
        "\n",
        "                for variable in model.variables:\n",
        "                    if variable.constraint is not None:\n",
        "                        variable.assign(variable.constraint(variable))\n",
        "\n",
        "                status = OrderedDict()\n",
        "                mean_loss(loss)\n",
        "                status[\"loss\"] = mean_loss.result().numpy()\n",
        "                for metric in metrics:\n",
        "                    metric(y_batch, y_pred)\n",
        "                    status[metric.name] = metric.result().numpy()\n",
        "\n",
        "                steps.set_postfix(status)\n",
        "\n",
        "        for metric in [mean_loss] + metrics:\n",
        "            metric.reset_states()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nufrN7ggAxrq"
      },
      "source": [
        "## 텐서플로 함수"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 178,
      "metadata": {
        "id": "00O0nYkAAxrq"
      },
      "outputs": [],
      "source": [
        "def cube(x):\n",
        "    return x ** 3"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 179,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "uEg23hqxAxrq",
        "outputId": "677a0c20-dbdb-4aea-e99d-6be98f6e67a6"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "8"
            ]
          },
          "metadata": {},
          "execution_count": 179
        }
      ],
      "source": [
        "cube(2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 180,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lrj1qhFyAxrq",
        "outputId": "49d5eca0-5f27-4f1d-f781-ab5b15d49e74"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=8.0>"
            ]
          },
          "metadata": {},
          "execution_count": 180
        }
      ],
      "source": [
        "cube(tf.constant(2.0))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 181,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "guBTABFmAxrq",
        "outputId": "b64f55ea-dde2-4bc7-de89-f46ba32f0b12"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.eager.polymorphic_function.polymorphic_function.Function at 0x7cf0b0c69b40>"
            ]
          },
          "metadata": {},
          "execution_count": 181
        }
      ],
      "source": [
        "tf_cube = tf.function(cube)\n",
        "tf_cube"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 182,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IfvIk4gFAxrq",
        "outputId": "37bb7264-b900-4c74-d365-1a12c63cbaae"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=8>"
            ]
          },
          "metadata": {},
          "execution_count": 182
        }
      ],
      "source": [
        "tf_cube(2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 183,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zFa3QVFYAxrq",
        "outputId": "148b38c8-9f91-457a-9406-cd441d2af5c5"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=8.0>"
            ]
          },
          "metadata": {},
          "execution_count": 183
        }
      ],
      "source": [
        "tf_cube(tf.constant(2.0))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 184,
      "metadata": {
        "id": "IWMZGSPEAxrq"
      },
      "outputs": [],
      "source": [
        "@tf.function\n",
        "def tf_cube(x):\n",
        "    return x ** 3"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GyTfJbwrAxrq"
      },
      "source": [
        "**노트:** 이 섹션의 나머지 코드는 부록 D의 내용입니다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-ksujIMzAxrr"
      },
      "source": [
        "### TF 함수와 콘크리트 함수"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 185,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ejIT2i2pAxrr",
        "outputId": "9579445a-749e-43fa-bd5f-37b3528331d5"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<ConcreteFunction (x: TensorSpec(shape=(), dtype=tf.float32, name=None)) -> TensorSpec(shape=(), dtype=tf.float32, name=None) at 0x7CF0AEEF2710>"
            ]
          },
          "metadata": {},
          "execution_count": 185
        }
      ],
      "source": [
        "concrete_function = tf_cube.get_concrete_function(tf.constant(2.0))\n",
        "concrete_function"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 186,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fTl77eYAAxrr",
        "outputId": "0314e191-b8d0-40b9-c4dd-39ed87f5e625"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=8.0>"
            ]
          },
          "metadata": {},
          "execution_count": 186
        }
      ],
      "source": [
        "concrete_function(tf.constant(2.0))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 187,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pb0a8LkdAxrr",
        "outputId": "a1f011d1-6742-4dbb-c4b1-a3eef04b63b1"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 187
        }
      ],
      "source": [
        "concrete_function is tf_cube.get_concrete_function(tf.constant(2.0))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iJH3KGaPAxrr"
      },
      "source": [
        "### 함수 정의와 그래프 탐험하기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 188,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EPPO52zAAxrr",
        "outputId": "cfa6c340-7285-4ce4-ccb7-10d02ade6b8d"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.framework.func_graph.FuncGraph at 0x7cf0aef0db40>"
            ]
          },
          "metadata": {},
          "execution_count": 188
        }
      ],
      "source": [
        "concrete_function.graph"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 189,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rRyMNm6ZAxrr",
        "outputId": "8d57dc29-892e-4951-a3de-fdd52b9a6c42"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Operation 'x' type=Placeholder>,\n",
              " <tf.Operation 'pow/y' type=Const>,\n",
              " <tf.Operation 'pow' type=Pow>,\n",
              " <tf.Operation 'Identity' type=Identity>]"
            ]
          },
          "metadata": {},
          "execution_count": 189
        }
      ],
      "source": [
        "ops = concrete_function.graph.get_operations()\n",
        "ops"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 190,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wghSPcJuAxrr",
        "outputId": "fed3107c-3d1e-46dd-88de-3c9e6119ea3b"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor 'x:0' shape=() dtype=float32>,\n",
              " <tf.Tensor 'pow/y:0' shape=() dtype=float32>]"
            ]
          },
          "metadata": {},
          "execution_count": 190
        }
      ],
      "source": [
        "pow_op = ops[2]\n",
        "list(pow_op.inputs)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 191,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8CAcuouzAxrr",
        "outputId": "2a0dc461-25e1-45be-d266-a05090f4fa77"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Tensor 'pow:0' shape=() dtype=float32>]"
            ]
          },
          "metadata": {},
          "execution_count": 191
        }
      ],
      "source": [
        "pow_op.outputs"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 192,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ki6l6QlIAxrr",
        "outputId": "264921ed-0eb5-4240-846b-18b1f311924e"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Operation 'x' type=Placeholder>"
            ]
          },
          "metadata": {},
          "execution_count": 192
        }
      ],
      "source": [
        "concrete_function.graph.get_operation_by_name('x')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 193,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LJe6V79wAxrr",
        "outputId": "798bb353-576f-4a1f-bdf1-455433fd3660"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor 'Identity:0' shape=() dtype=float32>"
            ]
          },
          "metadata": {},
          "execution_count": 193
        }
      ],
      "source": [
        "concrete_function.graph.get_tensor_by_name('Identity:0')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 194,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eMau0rWJAxrr",
        "outputId": "ec2b4ed5-40c8-4106-e62c-24054ac8c013"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "name: \"__inference_tf_cube_470259\"\n",
              "input_arg {\n",
              "  name: \"x\"\n",
              "  type: DT_FLOAT\n",
              "}\n",
              "output_arg {\n",
              "  name: \"identity\"\n",
              "  type: DT_FLOAT\n",
              "}"
            ]
          },
          "metadata": {},
          "execution_count": 194
        }
      ],
      "source": [
        "concrete_function.function_def.signature"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m3YK9ebhAxrs"
      },
      "source": [
        "### TF 함수가 파이썬 함수를 트레이싱하여 계산 그래프를 추출하는 방법"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 195,
      "metadata": {
        "id": "aQ2EBGBOAxrs"
      },
      "outputs": [],
      "source": [
        "@tf.function\n",
        "def tf_cube(x):\n",
        "    print(f\"x = {x}\")\n",
        "    return x ** 3"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 196,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yqAJdSL-Axrs",
        "outputId": "2e529b41-f4af-468e-9707-4a7c5a07a54a"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "x = Tensor(\"x:0\", shape=(), dtype=float32)\n"
          ]
        }
      ],
      "source": [
        "result = tf_cube(tf.constant(2.0))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 197,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rHNroERdAxrs",
        "outputId": "3b165a2c-4200-43a2-d3b0-44a90146b1ee"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=8.0>"
            ]
          },
          "metadata": {},
          "execution_count": 197
        }
      ],
      "source": [
        "result"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 198,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Du-arMTdAxrs",
        "outputId": "49fd0426-ee3e-40be-b2f9-74b890228c52"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "x = 2\n"
          ]
        }
      ],
      "source": [
        "result = tf_cube(2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 199,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-0FJXsF2Axrs",
        "outputId": "e4779542-bc1b-400f-869c-fed85343b2f9"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "x = 3\n"
          ]
        }
      ],
      "source": [
        "result = tf_cube(3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 200,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EQiq6YLJAxrs",
        "outputId": "7e0b1d72-82c4-4f9b-fa0b-50351fcdfd41"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "x = Tensor(\"x:0\", shape=(1, 2), dtype=float32)\n"
          ]
        }
      ],
      "source": [
        "result = tf_cube(tf.constant([[1., 2.]]))  # 새로운 크기: 트레이싱!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 201,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "J6Wec1VzAxrs",
        "outputId": "bd886584-6841-42c7-b7d6-98e0c37ace20"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "WARNING:tensorflow:5 out of the last 5 calls to <function tf_cube at 0x7cf0aef0f400> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "x = Tensor(\"x:0\", shape=(2, 2), dtype=float32)\n"
          ]
        }
      ],
      "source": [
        "result = tf_cube(tf.constant([[3., 4.], [5., 6.]]))  # 새로운 크기: 트레이싱!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 202,
      "metadata": {
        "id": "giQiBq6CAxrs"
      },
      "outputs": [],
      "source": [
        "result = tf_cube(tf.constant([[7., 8.], [9., 10.]]))  # 같은 크기: 트레이싱 없음"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5CIVAatSAxrs"
      },
      "source": [
        "It is also possible to specify a particular input signature:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 203,
      "metadata": {
        "id": "XK8U-vrNAxrs"
      },
      "outputs": [],
      "source": [
        "@tf.function(input_signature=[tf.TensorSpec([None, 28, 28], tf.float32)])\n",
        "def shrink(images):\n",
        "    print(\"Tracing\", images)  # 트레이싱을 출력하는 추가 코드\n",
        "    return images[:, ::2, ::2] # 행과 열의 절반을 삭제합니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 204,
      "metadata": {
        "id": "TXOCIeVhAxrt"
      },
      "outputs": [],
      "source": [
        "tf.keras.utils.set_random_seed(42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 205,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MBjUObyWAxrt",
        "outputId": "fb1f2d75-7a7d-480f-8331-e138cd2605bf"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Tracing Tensor(\"images:0\", shape=(None, 28, 28), dtype=float32)\n"
          ]
        }
      ],
      "source": [
        "img_batch_1 = tf.random.uniform(shape=[100, 28, 28])\n",
        "img_batch_2 = tf.random.uniform(shape=[50, 28, 28])\n",
        "preprocessed_images = shrink(img_batch_1)  # 정상 작동, 함수 트레이싱\n",
        "preprocessed_images = shrink(img_batch_2)  # 정상 작동, 동일한 콘크리트 함수"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 206,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 507
        },
        "id": "ootcVC_nAxrt",
        "outputId": "3c09613a-20a0-4b7b-d5d2-08093136cda8"
      },
      "outputs": [
        {
          "output_type": "error",
          "ename": "TypeError",
          "evalue": "Binding inputs to tf.function failed due to `Can not cast TensorSpec(shape=(2, 2, 2), dtype=tf.float32, name=None) to TensorSpec(shape=(None, 28, 28), dtype=tf.float32, name=None)`. Received args: (<tf.Tensor: shape=(2, 2, 2), dtype=float32, numpy=\narray([[[0.7413678 , 0.62854624],\n        [0.01738465, 0.3431449 ]],\n\n       [[0.51063764, 0.3777541 ],\n        [0.07321596, 0.02137029]]], dtype=float32)>,) and kwargs: {} for signature: (images: TensorSpec(shape=(None, 28, 28), dtype=tf.float32, name=None)).",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-206-493374f3af0d>\u001b[0m in \u001b[0;36m<cell line: 2>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0mimg_batch_3\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrandom\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0muniform\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m     \u001b[0mpreprocessed_images\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mshrink\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mimg_batch_3\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# ValueError! 호환되지 않는 입력\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      4\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0mValueError\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mex\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m     \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mex\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/tensorflow/python/util/traceback_utils.py\u001b[0m in \u001b[0;36merror_handler\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    151\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0mException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    152\u001b[0m       \u001b[0mfiltered_tb\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_process_traceback_frames\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__traceback__\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 153\u001b[0;31m       \u001b[0;32mraise\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwith_traceback\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfiltered_tb\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    154\u001b[0m     \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    155\u001b[0m       \u001b[0;32mdel\u001b[0m \u001b[0mfiltered_tb\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/tensorflow/python/eager/polymorphic_function/function_type_utils.py\u001b[0m in \u001b[0;36mbind_function_inputs\u001b[0;34m(args, kwargs, function_type, default_values)\u001b[0m\n\u001b[1;32m    444\u001b[0m     )\n\u001b[1;32m    445\u001b[0m   \u001b[0;32mexcept\u001b[0m \u001b[0mException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 446\u001b[0;31m     raise TypeError(\n\u001b[0m\u001b[1;32m    447\u001b[0m         \u001b[0;34mf\"Binding inputs to tf.function failed due to `{e}`. \"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    448\u001b[0m         \u001b[0;34mf\"Received args: {args} and kwargs: {sanitized_kwargs} for signature:\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mTypeError\u001b[0m: Binding inputs to tf.function failed due to `Can not cast TensorSpec(shape=(2, 2, 2), dtype=tf.float32, name=None) to TensorSpec(shape=(None, 28, 28), dtype=tf.float32, name=None)`. Received args: (<tf.Tensor: shape=(2, 2, 2), dtype=float32, numpy=\narray([[[0.7413678 , 0.62854624],\n        [0.01738465, 0.3431449 ]],\n\n       [[0.51063764, 0.3777541 ],\n        [0.07321596, 0.02137029]]], dtype=float32)>,) and kwargs: {} for signature: (images: TensorSpec(shape=(None, 28, 28), dtype=tf.float32, name=None))."
          ]
        }
      ],
      "source": [
        "img_batch_3 = tf.random.uniform(shape=[2, 2, 2])\n",
        "try:\n",
        "    preprocessed_images = shrink(img_batch_3)  # ValueError! 호환되지 않는 입력\n",
        "except ValueError as ex:\n",
        "    print(ex)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4OjBRI8BAxrt"
      },
      "source": [
        "### Autograph를 사용해 제어 흐름 표현하기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Xi63dzBGAxrt"
      },
      "source": [
        "`range()`를 사용하는 \"정적\" `for` 루프입니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 207,
      "metadata": {
        "id": "FgAkT4DFAxrt"
      },
      "outputs": [],
      "source": [
        "@tf.function\n",
        "def add_10(x):\n",
        "    for i in range(10):\n",
        "        x += 1\n",
        "    return x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 208,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0DaLQaE0Axrt",
        "outputId": "09902d26-cadd-4cae-e9b7-8ee187ba0100"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=15>"
            ]
          },
          "metadata": {},
          "execution_count": 208
        }
      ],
      "source": [
        "add_10(tf.constant(5))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 209,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "y4CNvOlwAxrt",
        "outputId": "3455d614-ba0c-4844-bd61-91f24f8d2f97"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Operation 'x' type=Placeholder>,\n",
              " <tf.Operation 'add/y' type=Const>,\n",
              " <tf.Operation 'add' type=AddV2>,\n",
              " <tf.Operation 'add_1/y' type=Const>,\n",
              " <tf.Operation 'add_1' type=AddV2>,\n",
              " <tf.Operation 'add_2/y' type=Const>,\n",
              " <tf.Operation 'add_2' type=AddV2>,\n",
              " <tf.Operation 'add_3/y' type=Const>,\n",
              " <tf.Operation 'add_3' type=AddV2>,\n",
              " <tf.Operation 'add_4/y' type=Const>,\n",
              " <tf.Operation 'add_4' type=AddV2>,\n",
              " <tf.Operation 'add_5/y' type=Const>,\n",
              " <tf.Operation 'add_5' type=AddV2>,\n",
              " <tf.Operation 'add_6/y' type=Const>,\n",
              " <tf.Operation 'add_6' type=AddV2>,\n",
              " <tf.Operation 'add_7/y' type=Const>,\n",
              " <tf.Operation 'add_7' type=AddV2>,\n",
              " <tf.Operation 'add_8/y' type=Const>,\n",
              " <tf.Operation 'add_8' type=AddV2>,\n",
              " <tf.Operation 'add_9/y' type=Const>,\n",
              " <tf.Operation 'add_9' type=AddV2>,\n",
              " <tf.Operation 'Identity' type=Identity>]"
            ]
          },
          "metadata": {},
          "execution_count": 209
        }
      ],
      "source": [
        "add_10.get_concrete_function(tf.constant(5)).graph.get_operations()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ih2sdaI1Axrt"
      },
      "source": [
        "`tf.while_loop()`를 사용하는 \"동적\" 루프입니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 210,
      "metadata": {
        "id": "ZSytfbE7Axrv"
      },
      "outputs": [],
      "source": [
        "# 추가 코드 - tf.while_loop를 사용하는 방법을 보여줍니다(일반적으로 @tf.function이 더 간단합니다).\n",
        "@tf.function\n",
        "def add_10(x):\n",
        "    condition = lambda i, x: tf.less(i, 10)\n",
        "    body = lambda i, x: (tf.add(i, 1), tf.add(x, 1))\n",
        "    final_i, final_x = tf.while_loop(condition, body, [tf.constant(0), x])\n",
        "    return final_x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 211,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rdfAbdJIAxrv",
        "outputId": "00cb2064-5a64-4248-ddd5-bd3fef8fdee7"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=15>"
            ]
          },
          "metadata": {},
          "execution_count": 211
        }
      ],
      "source": [
        "add_10(tf.constant(5))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 212,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ofZlMzAsAxrv",
        "outputId": "6b92cab1-a79f-4f55-ba03-28752411f51b"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Operation 'x' type=Placeholder>,\n",
              " <tf.Operation 'Const' type=Const>,\n",
              " <tf.Operation 'while/maximum_iterations' type=Const>,\n",
              " <tf.Operation 'while/loop_counter' type=Const>,\n",
              " <tf.Operation 'while' type=StatelessWhile>,\n",
              " <tf.Operation 'Identity' type=Identity>]"
            ]
          },
          "metadata": {},
          "execution_count": 212
        }
      ],
      "source": [
        "add_10.get_concrete_function(tf.constant(5)).graph.get_operations()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "To_1E-ULAxrv"
      },
      "source": [
        "(autograph가 감지한) `tf.range()`를 사용하는 \"동적\" `for` 루프:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 213,
      "metadata": {
        "id": "Ik3ff2wUAxrv"
      },
      "outputs": [],
      "source": [
        "@tf.function\n",
        "def add_10(x):\n",
        "    for i in tf.range(10):\n",
        "        x = x + 1\n",
        "    return x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 214,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LQGPCnrRAxrw",
        "outputId": "fb2aad69-5ef6-40f1-97ec-b7a87390ba6e"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<tf.Operation 'x' type=Placeholder>,\n",
              " <tf.Operation 'range/start' type=Const>,\n",
              " <tf.Operation 'range/limit' type=Const>,\n",
              " <tf.Operation 'range/delta' type=Const>,\n",
              " <tf.Operation 'range' type=Range>,\n",
              " <tf.Operation 'sub' type=Sub>,\n",
              " <tf.Operation 'floordiv' type=FloorDiv>,\n",
              " <tf.Operation 'mod' type=FloorMod>,\n",
              " <tf.Operation 'zeros_like' type=Const>,\n",
              " <tf.Operation 'NotEqual' type=NotEqual>,\n",
              " <tf.Operation 'Cast' type=Cast>,\n",
              " <tf.Operation 'add' type=AddV2>,\n",
              " <tf.Operation 'zeros_like_1' type=Const>,\n",
              " <tf.Operation 'Maximum' type=Maximum>,\n",
              " <tf.Operation 'while/maximum_iterations' type=Const>,\n",
              " <tf.Operation 'while/loop_counter' type=Const>,\n",
              " <tf.Operation 'while' type=StatelessWhile>,\n",
              " <tf.Operation 'Identity' type=Identity>]"
            ]
          },
          "metadata": {},
          "execution_count": 214
        }
      ],
      "source": [
        "add_10.get_concrete_function(tf.constant(0)).graph.get_operations()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h5cAtawIAxrw"
      },
      "source": [
        "### TF 함수에서 변수와 다른 리소스 다루기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 215,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PRVaBEmGAxrw",
        "outputId": "2ccc6008-d917-4c84-df63-f08b75cfab9a"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=2>"
            ]
          },
          "metadata": {},
          "execution_count": 215
        }
      ],
      "source": [
        "counter = tf.Variable(0)\n",
        "\n",
        "@tf.function\n",
        "def increment(counter, c=1):\n",
        "    return counter.assign_add(c)\n",
        "\n",
        "increment(counter)  # counter는 이제 1과 같습니다.\n",
        "increment(counter)  # counter는 이제 2와 같습니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 216,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pMIyf8CjAxrw",
        "outputId": "1841f2bb-7b52-4c4b-e378-757f42263f7c"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "name: \"counter\"\n",
              "type: DT_RESOURCE"
            ]
          },
          "metadata": {},
          "execution_count": 216
        }
      ],
      "source": [
        "function_def = increment.get_concrete_function(counter).function_def\n",
        "function_def.signature.input_arg[0]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 217,
      "metadata": {
        "id": "9HG88LCFAxrw"
      },
      "outputs": [],
      "source": [
        "counter = tf.Variable(0)\n",
        "\n",
        "@tf.function\n",
        "def increment(c=1):\n",
        "    return counter.assign_add(c)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 218,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XPosV6MJAxrw",
        "outputId": "fa15cac8-e992-437d-9dbd-dbb2d19cff3a"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=2>"
            ]
          },
          "metadata": {},
          "execution_count": 218
        }
      ],
      "source": [
        "increment()\n",
        "increment()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 219,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0i3CSugcAxrw",
        "outputId": "bee64272-827e-4360-aabc-20ddbe6c4674"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "name: \"assignaddvariableop_resource\"\n",
              "type: DT_RESOURCE"
            ]
          },
          "metadata": {},
          "execution_count": 219
        }
      ],
      "source": [
        "function_def = increment.get_concrete_function().function_def\n",
        "function_def.signature.input_arg[0]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 220,
      "metadata": {
        "id": "LlXsSHykAxrw"
      },
      "outputs": [],
      "source": [
        "class Counter:\n",
        "    def __init__(self):\n",
        "        self.counter = tf.Variable(0)\n",
        "\n",
        "    @tf.function\n",
        "    def increment(self, c=1):\n",
        "        return self.counter.assign_add(c)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 221,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PmsokL4UAxrw",
        "outputId": "3158d1cf-fe27-4c16-adb3-9b42fe289370"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=2>"
            ]
          },
          "metadata": {},
          "execution_count": 221
        }
      ],
      "source": [
        "c = Counter()\n",
        "c.increment()\n",
        "c.increment()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 222,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "uKTfW75IAxrw",
        "outputId": "efa0e008-103f-4059-f04c-c214f838b4f8",
        "scrolled": true
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "def tf__add(x):\n",
            "    with ag__.FunctionScope('add_10', 'fscope', ag__.ConversionOptions(recursive=True, user_requested=True, optional_features=(), internal_convert_user_code=True)) as fscope:\n",
            "        do_return = False\n",
            "        retval_ = ag__.UndefinedReturnValue()\n",
            "\n",
            "        def get_state():\n",
            "            return (x,)\n",
            "\n",
            "        def set_state(vars_):\n",
            "            nonlocal x\n",
            "            (x,) = vars_\n",
            "\n",
            "        def loop_body(itr):\n",
            "            nonlocal x\n",
            "            i = itr\n",
            "            x = ag__.ld(x)\n",
            "            x += 1\n",
            "        i = ag__.Undefined('i')\n",
            "        ag__.for_stmt(ag__.converted_call(ag__.ld(tf).range, (10,), None, fscope), None, loop_body, get_state, set_state, ('x',), {'iterate_names': 'i'})\n",
            "        try:\n",
            "            do_return = True\n",
            "            retval_ = ag__.ld(x)\n",
            "        except:\n",
            "            do_return = False\n",
            "            raise\n",
            "        return fscope.ret(retval_, do_return)\n",
            "\n"
          ]
        }
      ],
      "source": [
        "@tf.function\n",
        "def add_10(x):\n",
        "    for i in tf.range(10):\n",
        "        x += 1\n",
        "    return x\n",
        "\n",
        "print(tf.autograph.to_code(add_10.python_function))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 223,
      "metadata": {
        "id": "W_I7ORibAxrx"
      },
      "outputs": [],
      "source": [
        "# 추가 코드 - 구문 강조 표시로 autograph 코드를 표시하는 방법을 보여줍니다.\n",
        "def display_tf_code(func):\n",
        "    from IPython.display import display, Markdown\n",
        "    if hasattr(func, \"python_function\"):\n",
        "        func = func.python_function\n",
        "    code = tf.autograph.to_code(func)\n",
        "    display(Markdown(f'```python\\n{code}\\n```'))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 224,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 488
        },
        "id": "iFIBmaSWAxrx",
        "outputId": "adf2db57-47cb-4cbc-fb75-f6d0b637efd5"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ],
            "text/markdown": "```python\ndef tf__add(x):\n    with ag__.FunctionScope('add_10', 'fscope', ag__.ConversionOptions(recursive=True, user_requested=True, optional_features=(), internal_convert_user_code=True)) as fscope:\n        do_return = False\n        retval_ = ag__.UndefinedReturnValue()\n\n        def get_state():\n            return (x,)\n\n        def set_state(vars_):\n            nonlocal x\n            (x,) = vars_\n\n        def loop_body(itr):\n            nonlocal x\n            i = itr\n            x = ag__.ld(x)\n            x += 1\n        i = ag__.Undefined('i')\n        ag__.for_stmt(ag__.converted_call(ag__.ld(tf).range, (10,), None, fscope), None, loop_body, get_state, set_state, ('x',), {'iterate_names': 'i'})\n        try:\n            do_return = True\n            retval_ = ag__.ld(x)\n        except:\n            do_return = False\n            raise\n        return fscope.ret(retval_, do_return)\n\n```"
          },
          "metadata": {}
        }
      ],
      "source": [
        "display_tf_code(add_10)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PB0Sa4q9Axrx"
      },
      "source": [
        "## 케라스에 TF 함수 사용하기(또는 사용하지 않기)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SpjPf0nJAxrx"
      },
      "source": [
        "기본적으로 tf.keras는 사용자 지정 코드를 TF 함수로 자동 변환하므로 `tf.function()`을 사용할 필요가 없습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 225,
      "metadata": {
        "id": "7-1v1OwYAxrx"
      },
      "outputs": [],
      "source": [
        "# 사용자 정의 손실 함수\n",
        "def my_mse(y_true, y_pred):\n",
        "    print(\"손실 my_mse() 트레이싱\")\n",
        "    return tf.reduce_mean(tf.square(y_pred - y_true))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 226,
      "metadata": {
        "id": "p1skutQgAxrx"
      },
      "outputs": [],
      "source": [
        "# 사용자 정의 지표 함수\n",
        "def my_mae(y_true, y_pred):\n",
        "    print(\"지표 my_mae() 트레이싱\")\n",
        "    return tf.reduce_mean(tf.abs(y_pred - y_true))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 227,
      "metadata": {
        "id": "PpTUWOw7Axrx"
      },
      "outputs": [],
      "source": [
        "# 사용자 정의 층\n",
        "class MyDense(tf.keras.layers.Layer):\n",
        "    def __init__(self, units, activation=None, **kwargs):\n",
        "        super().__init__(**kwargs)\n",
        "        self.units = units\n",
        "        self.activation = tf.keras.activations.get(activation)\n",
        "\n",
        "    def build(self, input_shape):\n",
        "        self.kernel = self.add_weight(name='kernel',\n",
        "                                      shape=(input_shape[1], self.units),\n",
        "                                      initializer='uniform',\n",
        "                                      trainable=True)\n",
        "        self.biases = self.add_weight(name='bias',\n",
        "                                      shape=(self.units,),\n",
        "                                      initializer='zeros',\n",
        "                                      trainable=True)\n",
        "        super().build(input_shape)\n",
        "\n",
        "    def call(self, X):\n",
        "        print(\"MyDense.call() 트레이싱\")\n",
        "        return self.activation(X @ self.kernel + self.biases)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 228,
      "metadata": {
        "id": "YxycmeS9Axrx"
      },
      "outputs": [],
      "source": [
        "tf.keras.utils.set_random_seed(42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 229,
      "metadata": {
        "id": "PS8DpbmLAxrx"
      },
      "outputs": [],
      "source": [
        "# 사용자 정의 모델\n",
        "class MyModel(tf.keras.Model):\n",
        "    def __init__(self, **kwargs):\n",
        "        super().__init__(**kwargs)\n",
        "        self.hidden1 = MyDense(30, activation=\"relu\")\n",
        "        self.hidden2 = MyDense(30, activation=\"relu\")\n",
        "        self.output_ = MyDense(1)\n",
        "\n",
        "    def call(self, input):\n",
        "        print(\"MyModel.call() 트레이싱\")\n",
        "        hidden1 = self.hidden1(input)\n",
        "        hidden2 = self.hidden2(hidden1)\n",
        "        concat = tf.keras.layers.concatenate([input, hidden2])\n",
        "        output = self.output_(concat)\n",
        "        return output\n",
        "\n",
        "model = MyModel()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 230,
      "metadata": {
        "id": "93rXOAQQAxrx"
      },
      "outputs": [],
      "source": [
        "model.compile(loss=my_mse, optimizer=\"nadam\", metrics=[my_mae])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 231,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "P5NZCM16Axry",
        "outputId": "a72ab745-3ab9-4d83-9d2f-de29a2a4b624"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/2\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "359/363 [============================>.] - ETA: 0s - loss: 1.4387 - my_mae: 0.8244MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "363/363 [==============================] - 3s 4ms/step - loss: 1.4293 - my_mae: 0.8214 - val_loss: 0.4888 - val_my_mae: 0.4757\n",
            "Epoch 2/2\n",
            "363/363 [==============================] - 2s 5ms/step - loss: 0.4383 - my_mae: 0.4757 - val_loss: 1.1306 - val_my_mae: 0.4804\n",
            "162/162 [==============================] - 0s 3ms/step - loss: 0.4214 - my_mae: 0.4710\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[0.4213947057723999, 0.47099101543426514]"
            ]
          },
          "metadata": {},
          "execution_count": 231
        }
      ],
      "source": [
        "model.fit(X_train_scaled, y_train, epochs=2,\n",
        "          validation_data=(X_valid_scaled, y_valid))\n",
        "model.evaluate(X_test_scaled, y_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-d4A6OrzAxry"
      },
      "source": [
        "이 기능은 `dynamic=True`로 모델을 생성하거나 모델 생성자에서 `super().__init__(dynamic=True, **kwargs)`를 호출하여 해제할 수 있습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 232,
      "metadata": {
        "id": "31LkRTp5Axry"
      },
      "outputs": [],
      "source": [
        "tf.keras.utils.set_random_seed(42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 233,
      "metadata": {
        "id": "IAfSvsoIAxry"
      },
      "outputs": [],
      "source": [
        "model = MyModel(dynamic=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 234,
      "metadata": {
        "id": "OjqYAiKdAxry"
      },
      "outputs": [],
      "source": [
        "model.compile(loss=my_mse, optimizer=\"nadam\", metrics=[my_mae])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WC2KyPdXAxry"
      },
      "source": [
        "이제 사용자 정의 코드가 각 반복마다 호출됩니다. 너무 많은 출력을 피하기 위해 작은 데이터셋으로 검증과 평가를 수행해 보겠습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 235,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "T8s4Ap8cAxry",
        "outputId": "a470b3b8-5ba4-4c5e-c690-3eb340825893"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "WARNING:tensorflow:5 out of the last 14481 calls to <function _BaseOptimizer._update_step_xla at 0x7cf0a244f5b0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.\n",
            "WARNING:tensorflow:6 out of the last 14482 calls to <function _BaseOptimizer._update_step_xla at 0x7cf0a244f5b0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[5.549154281616211, 2.0611484050750732]"
            ]
          },
          "metadata": {},
          "execution_count": 235
        }
      ],
      "source": [
        "model.fit(X_train_scaled[:64], y_train[:64], epochs=1,\n",
        "          validation_data=(X_valid_scaled[:64], y_valid[:64]), verbose=0)\n",
        "model.evaluate(X_test_scaled[:64], y_test[:64], verbose=0)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fahebWVbAxry"
      },
      "source": [
        "또는 `run_eagerly=True`로 모델을 컴파일할 수 있습니다:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 236,
      "metadata": {
        "id": "GQeEapv4Axry"
      },
      "outputs": [],
      "source": [
        "tf.keras.utils.set_random_seed(42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 237,
      "metadata": {
        "id": "UCIW1vLWAxry"
      },
      "outputs": [],
      "source": [
        "model = MyModel()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 238,
      "metadata": {
        "id": "UUKjwuBtAxry"
      },
      "outputs": [],
      "source": [
        "model.compile(loss=my_mse, optimizer=\"nadam\", metrics=[my_mae], run_eagerly=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 239,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5JxmvAB_Axrz",
        "outputId": "a3df2e8b-f52d-4175-d623-b41a4220edca"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n",
            "MyModel.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "MyDense.call() 트레이싱\n",
            "손실 my_mse() 트레이싱\n",
            "지표 my_mae() 트레이싱\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[5.549154281616211, 2.0611484050750732]"
            ]
          },
          "metadata": {},
          "execution_count": 239
        }
      ],
      "source": [
        "model.fit(X_train_scaled[:64], y_train[:64], epochs=1,\n",
        "          validation_data=(X_valid_scaled[:64], y_valid[:64]), verbose=0)\n",
        "model.evaluate(X_test_scaled[:64], y_test[:64], verbose=0)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LpAV_M0WAxrz"
      },
      "source": [
        "## 추가 자료 - 사용자 정의 옵티마이저"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rDvHsV5UAxrz"
      },
      "source": [
        "사용자 정의 옵티마이저를 정의하는 것은 그리 일반적이지 않지만, 만약 여러분이 옵티마이저를 작성해야 한다면 다음 예시를 참고하세요:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 240,
      "metadata": {
        "id": "eg-Ls3HoAxrz"
      },
      "outputs": [],
      "source": [
        "class MyMomentumOptimizer(tf.keras.optimizers.Optimizer):\n",
        "    def __init__(self, learning_rate=0.001, momentum=0.0, name=\"MyMomentumOptimizer\", **kwargs):\n",
        "        super().__init__(name, **kwargs)\n",
        "        self._learning_rate = self._build_learning_rate(learning_rate)\n",
        "        self.momentum = momentum\n",
        "\n",
        "    def build(self, var_list):\n",
        "        \"\"\"옵티마이저 변수 초기화\n",
        "        매개변수:\n",
        "          var_list: 모델 변수 목록\n",
        "        \"\"\"\n",
        "        super().build(var_list)\n",
        "        if hasattr(self, \"_built\") and self._built:\n",
        "            return\n",
        "        self.momentums = []\n",
        "        for var in var_list:\n",
        "            self.momentums.append(\n",
        "                self.add_variable_from_reference(\n",
        "                    model_variable=var, variable_name=\"m\"\n",
        "                )\n",
        "            )\n",
        "        self._built = True\n",
        "\n",
        "    def update_step(self, gradient, variable):\n",
        "        \"\"\"주어진 그레이디언트와 관련된 모델 변수로 스텝을 업데이트합니다.\"\"\"\n",
        "        lr = tf.cast(self.learning_rate, variable.dtype)\n",
        "        m = None\n",
        "        var_key = self._var_key(variable)\n",
        "        momentum = tf.cast(self.momentum, variable.dtype)\n",
        "        m = self.momentums[self._index_dict[var_key]]\n",
        "\n",
        "        if isinstance(gradient, tf.IndexedSlices):\n",
        "            # 희소한 그레이디언트\n",
        "            add_value = tf.IndexedSlices(\n",
        "                -gradient.values * lr, gradient.indices\n",
        "            )\n",
        "            if m is not None:\n",
        "                m.assign(m * momentum)\n",
        "                m.scatter_add(add_value)\n",
        "                variable.assign_add(m)\n",
        "            else:\n",
        "                variable.scatter_add(add_value)\n",
        "        else:\n",
        "            # 밀집 그레이디언트\n",
        "            if m is not None:\n",
        "                m.assign(-gradient * lr + m * momentum)\n",
        "                variable.assign_add(m)\n",
        "            else:\n",
        "                variable.assign_add(-gradient * lr)\n",
        "\n",
        "    def get_config(self):\n",
        "        base_config = super().get_config()\n",
        "        return {\n",
        "            **base_config,\n",
        "            \"learning_rate\": self._serialize_hyperparameter(self._learning_rate),\n",
        "            \"momentum\": self._serialize_hyperparameter(self.momentum),\n",
        "        }"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 241,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "37SeM0eAAxrz",
        "outputId": "c8d73421-08cb-4e17-f5ba-52059087706b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/5\n",
            "363/363 [==============================] - 1s 2ms/step - loss: 4.0546\n",
            "Epoch 2/5\n",
            "363/363 [==============================] - 1s 2ms/step - loss: 1.2763\n",
            "Epoch 3/5\n",
            "363/363 [==============================] - 1s 2ms/step - loss: 0.7623\n",
            "Epoch 4/5\n",
            "363/363 [==============================] - 1s 2ms/step - loss: 0.6428\n",
            "Epoch 5/5\n",
            "363/363 [==============================] - 1s 2ms/step - loss: 0.6102\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7cf0a212b8b0>"
            ]
          },
          "metadata": {},
          "execution_count": 241
        }
      ],
      "source": [
        "optimizer = MyMomentumOptimizer()\n",
        "\n",
        "tf.keras.utils.set_random_seed(42)\n",
        "model = tf.keras.Sequential([tf.keras.layers.Dense(1, input_shape=[8])])\n",
        "model.compile(loss=\"mse\", optimizer=optimizer)\n",
        "model.fit(X_train_scaled, y_train, epochs=5)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YCQ5xZZG01bg"
      },
      "source": [
        "케라스에 내장된 모멘텀 옵티마이저와 비교해 보죠."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 242,
      "metadata": {
        "id": "af0sOTu501bg",
        "outputId": "af0ccee0-af0f-41c1-a573-dd813a1ca7f9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/5\n",
            "363/363 [==============================] - 1s 2ms/step - loss: 1.1844\n",
            "Epoch 2/5\n",
            "363/363 [==============================] - 1s 2ms/step - loss: 0.5635\n",
            "Epoch 3/5\n",
            "363/363 [==============================] - 1s 3ms/step - loss: 0.9703\n",
            "Epoch 4/5\n",
            "363/363 [==============================] - 1s 3ms/step - loss: 0.5678\n",
            "Epoch 5/5\n",
            "363/363 [==============================] - 1s 2ms/step - loss: 0.6350\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.callbacks.History at 0x7cf0a2163670>"
            ]
          },
          "metadata": {},
          "execution_count": 242
        }
      ],
      "source": [
        "optimizer = tf.keras.optimizers.SGD(learning_rate=0.001, momentum=0.9)\n",
        "\n",
        "tf.keras.utils.set_random_seed(42)\n",
        "model = tf.keras.Sequential([tf.keras.layers.Dense(1, input_shape=[8])])\n",
        "model.compile(loss=\"mse\", optimizer=optimizer)\n",
        "model.fit(X_train_scaled, y_train, epochs=5)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fVo-JwwwAxrz"
      },
      "source": [
        "# 연습문제 해답"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zcyVmMsAAxrz"
      },
      "source": [
        "## 1. to 11."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aviEodMhAxrz"
      },
      "source": [
        "부록 A 참조"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kAgNhFzmAxrz"
      },
      "source": [
        "## 12. _층 정규화_ 를 수행하는 사용자 정의 층을 구현하세요.\n",
        "\n",
        "_15장에서 순환 신경망을 사용할 때 이런 종류의 층을 사용합니다._"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8FwneyKMAxr0"
      },
      "source": [
        "### a.\n",
        "_문제: `build()` 메서드에서 두 개의 훈련 가능한 가중치 *α*와 *β*를 정의합니다. 두 가중치 모두 크기가 `input_shape[-1:]`이고 데이터 타입은 `tf.float32`입니다. *α*는 1로 초기화되고 *β*는 0으로 초기화되어야 합니다._"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sspawhAfAxr0"
      },
      "source": [
        "솔루션: 아래 참조."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gValdr8aAxr0"
      },
      "source": [
        "### b.\n",
        "_문제: `call()` 메서드는 샘플의 특성마다 평균 μ와 표준편차 σ를 계산해야 합니다. 이를 위해 전체 샘플의 평균 μ와 분산 σ<sup>2</sup>을 반환하는 `tf.nn.moments(inputs, axes=-1, keepdims=True)`을 사용할 수 있습니다(분산의 제곱근으로 표준편차를 계산합니다). 그다음 *α*⊗(*X* - μ)/(σ + ε) + *β*를 계산하여 반환합니다. 여기에서 ⊗는 원소별\n",
        "곱셈(`*`)을 나타냅니다. ε은 안전을 위한 항입니다(0으로 나누어지는 것을 막기 위한 작은 상수. 예를 들면 0.001)._"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 243,
      "metadata": {
        "id": "Zqsjv7PpAxr0"
      },
      "outputs": [],
      "source": [
        "class LayerNormalization(tf.keras.layers.Layer):\n",
        "    def __init__(self, eps=0.001, **kwargs):\n",
        "        super().__init__(**kwargs)\n",
        "        self.eps = eps\n",
        "\n",
        "    def build(self, batch_input_shape):\n",
        "        self.alpha = self.add_weight(\n",
        "            name=\"alpha\", shape=batch_input_shape[-1:],\n",
        "            initializer=\"ones\")\n",
        "        self.beta = self.add_weight(\n",
        "            name=\"beta\", shape=batch_input_shape[-1:],\n",
        "            initializer=\"zeros\")\n",
        "        super().build(batch_input_shape) # 반드시 끝에 와야 합니다\n",
        "\n",
        "    def call(self, X):\n",
        "        mean, variance = tf.nn.moments(X, axes=-1, keepdims=True)\n",
        "        return self.alpha * (X - mean) / (tf.sqrt(variance + self.eps)) + self.beta\n",
        "\n",
        "    def compute_output_shape(self, batch_input_shape):\n",
        "        return batch_input_shape\n",
        "\n",
        "    def get_config(self):\n",
        "        base_config = super().get_config()\n",
        "        return {**base_config, \"eps\": self.eps}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ozD6O4PdAxr0"
      },
      "source": [
        "_ε_ 하이퍼파라미터(`eps`)는 필수가 아닙니다. 또한 `tf.sqrt(variance) + self.eps` 보다 `tf.sqrt(variance + self.eps)`를 계산하는 것이 좋습니다. sqrt(z)의 도함수는 z=0에서 정의되지 않기 때문에 분산 벡터의 한 원소가 0에 가까우면 훈련이 이리저리 널뜁니다. 제곱근 안에 _ε_를 넣으면 이런 현상을 방지할 수 있습니다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7QH6eoWmAxr0"
      },
      "source": [
        "### c.\n",
        "_문제: 사용자 정의 층이 `tf.keras.layers.LayerNormalization` 층과 동일한(또는 거의 동일한) 출력을 만드는지 확인하세요._"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m0jDFJFKAxr0"
      },
      "source": [
        "각 클래스의 객체를 만들고 데이터(예를 들면, 훈련 세트)를 적용해 보죠. 차이는 무시할 수 있는 수준입니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 244,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "clVKIuKYAxr0",
        "outputId": "30ad35ac-4986-4d38-a7a5-6bf3ecb7d34e"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=3.894674e-08>"
            ]
          },
          "metadata": {},
          "execution_count": 244
        }
      ],
      "source": [
        "X = X_train.astype(np.float32)\n",
        "\n",
        "custom_layer_norm = LayerNormalization()\n",
        "keras_layer_norm = tf.keras.layers.LayerNormalization()\n",
        "\n",
        "tf.reduce_mean(tf.keras.losses.mean_absolute_error(\n",
        "    keras_layer_norm(X), custom_layer_norm(X)))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cLcPCLA1Axr0"
      },
      "source": [
        "네 충분히 가깝네요. 조금 더 확실하게 알파와 베타를 완전히 랜덤하게 지정하고 다시 비교해 보죠:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 245,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WSpbT-C4Axr0",
        "outputId": "8bda4ac5-e3ff-45fb-94bb-decf9c804b01"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=1.7224725e-08>"
            ]
          },
          "metadata": {},
          "execution_count": 245
        }
      ],
      "source": [
        "tf.keras.utils.set_random_seed(42)\n",
        "random_alpha = np.random.rand(X.shape[-1])\n",
        "random_beta = np.random.rand(X.shape[-1])\n",
        "\n",
        "custom_layer_norm.set_weights([random_alpha, random_beta])\n",
        "keras_layer_norm.set_weights([random_alpha, random_beta])\n",
        "\n",
        "tf.reduce_mean(tf.keras.losses.mean_absolute_error(\n",
        "    keras_layer_norm(X), custom_layer_norm(X)))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "etMAN21uAxr0"
      },
      "source": [
        "여전히 무시할 수 있는 수준입니다! 사용자 정의 층이 잘 동작합니다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tczap3MOAxr0"
      },
      "source": [
        "## 13. 사용자 정의 훈련 반복을 사용해 패션 MNIST 데이터셋으로 모델을 훈련해보세요.\n",
        "\n",
        "_패션 MNIST 데이터셋은 10장에서 소개했습니다._"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2tOj7JKKAxr1"
      },
      "source": [
        "### a.\n",
        "_문제: 에포크, 반복, 평균 훈련 손실, (반복마다 업데이트되는) 에포크의 평균 정확도는 물론 에포크 끝에서 검증 손실과 정확도를 출력하세요._"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 246,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HxmdPj7gAxr1",
        "outputId": "d9785be8-4ea4-40ba-d2be-3a701a7affa9"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz\n",
            "29515/29515 [==============================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz\n",
            "26421880/26421880 [==============================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz\n",
            "5148/5148 [==============================] - 0s 0us/step\n",
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz\n",
            "4422102/4422102 [==============================] - 0s 0us/step\n"
          ]
        }
      ],
      "source": [
        "(X_train_full, y_train_full), (X_test, y_test) = tf.keras.datasets.fashion_mnist.load_data()\n",
        "X_train_full = X_train_full.astype(np.float32) / 255.\n",
        "X_valid, X_train = X_train_full[:5000], X_train_full[5000:]\n",
        "y_valid, y_train = y_train_full[:5000], y_train_full[5000:]\n",
        "X_test = X_test.astype(np.float32) / 255."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 247,
      "metadata": {
        "id": "gf78zFM7Axr1"
      },
      "outputs": [],
      "source": [
        "tf.keras.utils.set_random_seed(42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 248,
      "metadata": {
        "id": "iKpfDKCsAxr1"
      },
      "outputs": [],
      "source": [
        "model = tf.keras.Sequential([\n",
        "    tf.keras.layers.Flatten(input_shape=[28, 28]),\n",
        "    tf.keras.layers.Dense(100, activation=\"relu\"),\n",
        "    tf.keras.layers.Dense(10, activation=\"softmax\"),\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 249,
      "metadata": {
        "id": "3qJ_XgmaAxr1"
      },
      "outputs": [],
      "source": [
        "n_epochs = 5\n",
        "batch_size = 32\n",
        "n_steps = len(X_train) // batch_size\n",
        "optimizer = tf.keras.optimizers.Nadam(learning_rate=0.01)\n",
        "loss_fn = tf.keras.losses.sparse_categorical_crossentropy\n",
        "mean_loss = tf.keras.metrics.Mean()\n",
        "metrics = [tf.keras.metrics.SparseCategoricalAccuracy()]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 250,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 209,
          "referenced_widgets": [
            "a5bf67e880ee4d489c535bbcb5c6f20c",
            "5275011748fd4bf58a82e1681eeb4922",
            "0c78a4d6df2c4b3c8d3cd2d0cabe31cd",
            "7aa8935aa937437c85d1e7f99929b91b",
            "5bbeda83519c40c68d983937cd5e1427",
            "c239fc78863a4cf59d94283e4ab0bff3",
            "3dc28e9e52f540c4ad4100cac468ac71",
            "50f7f33158f3430795f30cfbdb403fc1",
            "a5bbc61767c84346a957a9927545df15",
            "af583fffad9f4735848109acafec66cc",
            "54eb303dbbe94baeaaa27a32e4d2d1a3",
            "16e8a6a5694d4baf8c97636b27d42340",
            "64c2ec4df5d14ee098ea49eb623d6383",
            "0375f6d79e224a358c6f2dcd497100fd",
            "9550365c52e74ed88299f6abbfe546f3",
            "b33c062dd1aa4dc1acb328b14e030477",
            "53d902a9435e43b78becd5d0f3397662",
            "d75da4ead9d84260bc94a146da655432",
            "e776c18876b645c9b410012a8c2839ee",
            "947f75b505a84e9e8186291eddff37d5",
            "3940f30affcf44e4850f5cfc7d4ab2ef",
            "ac92a864ef3d4df9b452f7995f045d0f",
            "41a169bb652f4b159e88144d96b42630",
            "bac5345614024b0d9fce55ff48d6617b",
            "ce4e7fb875914a3a94428bc8ea396faa",
            "2c067f5a2c1d4a208cbef9143a5fc0bd",
            "8be807d184b640fa9dee8be49aa44c9a",
            "46988fa86c74460caf12e3c0f13056e0",
            "22172cdafbeb40f6a34d48ce2e959958",
            "b0bd186ad3b7474798553729787ed050",
            "963141319c234c809275f7ed9937ea81",
            "21895c30c0364b3d800125f451ccb0c7",
            "0bee1241aed145df8bde858b085b4472",
            "c7d4cadae597484488816817c7364cbb",
            "431f277a300842299dda90f4a14d0d3f",
            "0aa8a3273b284c649e2007335b6e6ab1",
            "396ac00cbe4049e7bd3df3803a370c55",
            "5977e0082b0043239abc185d5ca64cf0",
            "5c70dd398c8c42fa90d4ac4d8530c914",
            "563e717abb0a4ed5ba5aea925ec55e89",
            "a0410ba3e52d4f2e87af82b5995afb89",
            "428a2eb5eef04cc1a82c48b8ad67be21",
            "35f7be75d3724b6da1d28d955897a099",
            "57882a882d0e441f99caf39bee240fad",
            "210b9395240047b285140027e0104d2b",
            "5c8c1dad6c4249e082294af3a4c9e8e4",
            "ff21aed62a8e436c98923e766b700386",
            "577542a705ee4e2fa596bceb10b588fc",
            "68f0ed27908545b09fa2170c6158b509",
            "509cc203057e4d21b567a0c4b9c5d126",
            "c359589f1e6c4f78bb1e088d18486c7b",
            "7fc3aba71daa45cdb330407b71e7d0d6",
            "5bb93e7d16234a0a807f8e1ab631d50a",
            "2c76f8ee6a9848ee9b053c75e5ba4a95",
            "58a990b1be784743bd4fd438dc0dcfd9",
            "2fa6a92d7ea746d3af706461c4ce6f68",
            "fbe57b37a5d14410ac386e0484589dc9",
            "2d67a7559af1493e874a46b91eaeb3f0",
            "b807a014cabf4fef9d74f671f00de27c",
            "7ea5c6a990f04ceab01a92854ed173e7",
            "01a59b6835c345b59e9c2748f979f13f",
            "7a5878e8454c40718f68ae4c2dbe08d7",
            "6b5e6bd8111a4b608c41338f2655f00e",
            "8fea2d9d520a4656b04a3c3cf2d2be7e",
            "cfd33c56db5644bbbf8aa8f0450c76e8",
            "a938518cafd94fc39f1584d94ddc15df"
          ]
        },
        "id": "pSytRZyVAxr1",
        "outputId": "b0129a42-270d-4631-87f4-a10ce20f0cd1"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "All epochs:   0%|          | 0/5 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "a5bf67e880ee4d489c535bbcb5c6f20c"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 1/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "16e8a6a5694d4baf8c97636b27d42340"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 2/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "41a169bb652f4b159e88144d96b42630"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 3/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "c7d4cadae597484488816817c7364cbb"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 4/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "210b9395240047b285140027e0104d2b"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 5/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "2fa6a92d7ea746d3af706461c4ce6f68"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "with trange(1, n_epochs + 1, desc=\"All epochs\") as epochs:\n",
        "    for epoch in epochs:\n",
        "        with trange(1, n_steps + 1, desc=f\"Epoch {epoch}/{n_epochs}\") as steps:\n",
        "            for step in steps:\n",
        "                X_batch, y_batch = random_batch(X_train, y_train)\n",
        "                with tf.GradientTape() as tape:\n",
        "                    y_pred = model(X_batch)\n",
        "                    main_loss = tf.reduce_mean(loss_fn(y_batch, y_pred))\n",
        "                    loss = tf.add_n([main_loss] + model.losses)\n",
        "                gradients = tape.gradient(loss, model.trainable_variables)\n",
        "                optimizer.apply_gradients(zip(gradients, model.trainable_variables))\n",
        "                for variable in model.variables:\n",
        "                    if variable.constraint is not None:\n",
        "                        variable.assign(variable.constraint(variable))\n",
        "                status = OrderedDict()\n",
        "                mean_loss(loss)\n",
        "                status[\"loss\"] = mean_loss.result().numpy()\n",
        "                for metric in metrics:\n",
        "                    metric(y_batch, y_pred)\n",
        "                    status[metric.name] = metric.result().numpy()\n",
        "                steps.set_postfix(status)\n",
        "            y_pred = model(X_valid)\n",
        "            status[\"val_loss\"] = np.mean(loss_fn(y_valid, y_pred))\n",
        "            status[\"val_accuracy\"] = np.mean(tf.keras.metrics.sparse_categorical_accuracy(\n",
        "                tf.constant(y_valid, dtype=np.float32), y_pred))\n",
        "            steps.set_postfix(status)\n",
        "        for metric in [mean_loss] + metrics:\n",
        "            metric.reset_states()\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qt9pd6eJAxr1"
      },
      "source": [
        "### b.\n",
        "_문제: 상위 층과 하위 층에 학습률이 다른 옵티마이저를 따로 사용해보세요._"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 251,
      "metadata": {
        "id": "7W2-nkwWAxr1"
      },
      "outputs": [],
      "source": [
        "tf.keras.utils.set_random_seed(42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 252,
      "metadata": {
        "id": "ZESiHnZtAxr1"
      },
      "outputs": [],
      "source": [
        "lower_layers = tf.keras.Sequential([\n",
        "    tf.keras.layers.Flatten(input_shape=[28, 28]),\n",
        "    tf.keras.layers.Dense(100, activation=\"relu\"),\n",
        "])\n",
        "upper_layers = tf.keras.Sequential([\n",
        "    tf.keras.layers.Dense(10, activation=\"softmax\"),\n",
        "])\n",
        "model = tf.keras.Sequential([\n",
        "    lower_layers, upper_layers\n",
        "])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 253,
      "metadata": {
        "id": "lQZwhSFPAxr1"
      },
      "outputs": [],
      "source": [
        "lower_optimizer = tf.keras.optimizers.SGD(learning_rate=1e-4)\n",
        "upper_optimizer = tf.keras.optimizers.Nadam(learning_rate=1e-3)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 254,
      "metadata": {
        "id": "rRadcAKJAxr1"
      },
      "outputs": [],
      "source": [
        "n_epochs = 5\n",
        "batch_size = 32\n",
        "n_steps = len(X_train) // batch_size\n",
        "loss_fn = tf.keras.losses.sparse_categorical_crossentropy\n",
        "mean_loss = tf.keras.metrics.Mean()\n",
        "metrics = [tf.keras.metrics.SparseCategoricalAccuracy()]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 255,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 209,
          "referenced_widgets": [
            "5d4e8f146ea84d9399168f62a0b20a22",
            "7295024fb6e14912933f06f2e7fdd50a",
            "ff605a415aa3435f94d0c156cef6712f",
            "30814c93255b447baf16bb77ba004624",
            "af7953b6788f424694947f25f10d22b7",
            "2b05554510514e5b81e456e3051ecfc6",
            "3b45c0028c3647cbbc910744e7f1572a",
            "7ae7c237afcd401aba969bf38ee82f7d",
            "3659d1e6ae574720bfc5dcca4674f199",
            "dfc48b5ccf6944ea9244bc4d8e2ca61e",
            "e00cb0b2fb3d4a0da8d7997584797650",
            "ca930a32ebd849889cf12875dd0f67b6",
            "ada21ea4ca974932bacd823dc608d922",
            "43deb6bade0048b0a177bb1705e6f8c0",
            "05ebedfd4ec54fcf9e1e74834cab8aab",
            "03ed7016a45f4668937766a6ada6f341",
            "4d9e7c0435734007a80ba323ffcd6e8a",
            "6e1b71a829694e5ea98b8ad1974c0ffd",
            "2e9d4374618347769bfc17b696739864",
            "e185797b956345c6850e54d6863a8c88",
            "ce76b277e82f49ce9dfbb77432c6e17f",
            "2d9e4688242740e18e6504c563af7b06",
            "f0e31186bd2d48bfb2cb19bba6a72e7d",
            "45cb7611404344bb9b0c700b68f830ff",
            "8adcc12c64cc4c89903f307df647941b",
            "8fb9c0ddd14b4970be3f839a7f9b3f7c",
            "949de89c5b5341f7b9109debf207d9f7",
            "b089decce7a54a7c88d8fdafc41fa507",
            "a3e33c9748d14c9ba725fbcc1dc214b7",
            "7d39078a748a4f3cb6873ffad42e24a9",
            "ac0c420d7c0a4cba97b812536938f593",
            "d07e79cfcebd40b0a07a95c1acbfa4ef",
            "daeafd631cc94a8ba3185a65adfa9dd3",
            "baeae41f27eb44aa9ee10b0175a23963",
            "5e1c0d1633914a3e9c727f8d5e8d0a02",
            "144f057af6864174970cc00b84129319",
            "7b1a6f1880084faf81d9a6d90218209e",
            "1f61d7bbd81f420f99192e371fd7492f",
            "840429aa8c824433b0ba04b72ae9df19",
            "80c2a1dd166f4181b25d405094164a97",
            "b8e8750c344f4920a5c18a19807995a9",
            "5d126c9362ad483f96d21e36bfd2e18a",
            "a661a05063744fe5bb21718539f354da",
            "368897a043de49a6a63b37b100af0a9f",
            "c497ac7d70cb49ccb3444a9887570a8b",
            "ad194a229a3347e681d81290887db18b",
            "49c1e31accf24ddca07c3244fd7c2c28",
            "4c339586df73458f892ce3b4a35680b4",
            "4c63ae7ca560448f88139e9124c5e849",
            "46b590f489d34e15852c03f378d9edc6",
            "85bfd752cafd42ee8649e345f9fbe603",
            "6ec9aef1644d4729a38f2775e3a3364e",
            "2550a941d6e34f99a07cd67c0fad7385",
            "1dbdb1d667f049e199b1964260444ea8",
            "a7e86ff605f44c798eaf5ec67ca19a04",
            "c89d971ed2dc41dc8b3bc9ca1d23f885",
            "37cacb046e2b41198083898375175e06",
            "1a34e2c9150b4400ba819d480a46ae4f",
            "17fe2f0e26dd42d593d427eb42400677",
            "93f2b8bdb0144bc18c2cf3adde03a1c2",
            "91a39369e173414189c4ee832b73cdfa",
            "8068340cdc544fb3a2f2812c0eb7d812",
            "aef4e23b50e6425ab9e30d3f122ca19f",
            "37e7134b26154e39b0b824a848b1a98c",
            "197aa6c1f53946689081da33ac857a48",
            "885e07d27de14b5090594de705c72292"
          ]
        },
        "id": "jHSdXV7QAxr1",
        "outputId": "15a1b144-7143-4af7-ef1c-5e8ff0a866ab"
      },
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "All epochs:   0%|          | 0/5 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "5d4e8f146ea84d9399168f62a0b20a22"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 1/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "ca930a32ebd849889cf12875dd0f67b6"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 2/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "f0e31186bd2d48bfb2cb19bba6a72e7d"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 3/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "baeae41f27eb44aa9ee10b0175a23963"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 4/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "c497ac7d70cb49ccb3444a9887570a8b"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Epoch 5/5:   0%|          | 0/1718 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "c89d971ed2dc41dc8b3bc9ca1d23f885"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "with trange(1, n_epochs + 1, desc=\"All epochs\") as epochs:\n",
        "    for epoch in epochs:\n",
        "        with trange(1, n_steps + 1, desc=f\"Epoch {epoch}/{n_epochs}\") as steps:\n",
        "            for step in steps:\n",
        "                X_batch, y_batch = random_batch(X_train, y_train)\n",
        "                with tf.GradientTape(persistent=True) as tape:\n",
        "                    y_pred = model(X_batch)\n",
        "                    main_loss = tf.reduce_mean(loss_fn(y_batch, y_pred))\n",
        "                    loss = tf.add_n([main_loss] + model.losses)\n",
        "                for layers, optimizer in ((lower_layers, lower_optimizer),\n",
        "                                          (upper_layers, upper_optimizer)):\n",
        "                    gradients = tape.gradient(loss, layers.trainable_variables)\n",
        "                    optimizer.apply_gradients(zip(gradients, layers.trainable_variables))\n",
        "                del tape\n",
        "                for variable in model.variables:\n",
        "                    if variable.constraint is not None:\n",
        "                        variable.assign(variable.constraint(variable))\n",
        "                status = OrderedDict()\n",
        "                mean_loss(loss)\n",
        "                status[\"loss\"] = mean_loss.result().numpy()\n",
        "                for metric in metrics:\n",
        "                    metric(y_batch, y_pred)\n",
        "                    status[metric.name] = metric.result().numpy()\n",
        "                steps.set_postfix(status)\n",
        "            y_pred = model(X_valid)\n",
        "            status[\"val_loss\"] = np.mean(loss_fn(y_valid, y_pred))\n",
        "            status[\"val_accuracy\"] = np.mean(tf.keras.metrics.sparse_categorical_accuracy(\n",
        "                tf.constant(y_valid, dtype=np.float32), y_pred))\n",
        "            steps.set_postfix(status)\n",
        "        for metric in [mean_loss] + metrics:\n",
        "            metric.reset_states()"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "provenance": []
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.9"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "e593a5241b8d44ed99f448bd431da411": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_8806ef0fc0c748379a3d56146690b5a2",
              "IPY_MODEL_84a8e9a8661c4779af4dfe3a445a3576",
              "IPY_MODEL_429658200e424f519d88f2e165f7effc"
            ],
            "layout": "IPY_MODEL_eb71a955064a4c64adf96b629a0ae381"
          }
        },
        "8806ef0fc0c748379a3d56146690b5a2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ff54289fba384597848d41674bd53909",
            "placeholder": "​",
            "style": "IPY_MODEL_e98258ef2daf4b5cb3defa802b95a8f2",
            "value": "All epochs: 100%"
          }
        },
        "84a8e9a8661c4779af4dfe3a445a3576": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ea88e562b283474db0d66c5e8da78e7a",
            "max": 5,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_cdc5517881d34975a4ddc499fbd087f7",
            "value": 5
          }
        },
        "429658200e424f519d88f2e165f7effc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3a6ae70475ed437cbb643c1048deb00e",
            "placeholder": "​",
            "style": "IPY_MODEL_b46c44164ffd442185efcea730834f91",
            "value": " 5/5 [00:34&lt;00:00,  6.84s/it]"
          }
        },
        "eb71a955064a4c64adf96b629a0ae381": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ff54289fba384597848d41674bd53909": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e98258ef2daf4b5cb3defa802b95a8f2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ea88e562b283474db0d66c5e8da78e7a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cdc5517881d34975a4ddc499fbd087f7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "3a6ae70475ed437cbb643c1048deb00e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b46c44164ffd442185efcea730834f91": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f7b2267996574d8986e80c295059bb42": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_44bccc54092448f99db0566aeb69333d",
              "IPY_MODEL_85dcf484389449929746aa13df982cd1",
              "IPY_MODEL_d54bc5fd12174a7db88eba2c78a0c388"
            ],
            "layout": "IPY_MODEL_140936ae0ded4165839188989accf0fd"
          }
        },
        "44bccc54092448f99db0566aeb69333d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_773171184d944860af9335f2ac80ad31",
            "placeholder": "​",
            "style": "IPY_MODEL_0e2af8454c774354b5e01e4c28d427e2",
            "value": "Epoch 1/5: 100%"
          }
        },
        "85dcf484389449929746aa13df982cd1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2bea8139a1644646b1c85e79009afd86",
            "max": 362,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_6800c0c05ec54f3caf65888f07a5c6c9",
            "value": 362
          }
        },
        "d54bc5fd12174a7db88eba2c78a0c388": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9e62b2b6b4a9482790b281e90147bea7",
            "placeholder": "​",
            "style": "IPY_MODEL_2e3f1a346844471fb3fab643d6188e26",
            "value": " 362/362 [00:06&lt;00:00, 45.72it/s, loss=0.676, mean_absolute_error=0.504]"
          }
        },
        "140936ae0ded4165839188989accf0fd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "773171184d944860af9335f2ac80ad31": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0e2af8454c774354b5e01e4c28d427e2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2bea8139a1644646b1c85e79009afd86": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6800c0c05ec54f3caf65888f07a5c6c9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "9e62b2b6b4a9482790b281e90147bea7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2e3f1a346844471fb3fab643d6188e26": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "eaf498e6f7a644d3bfa1d044256067ef": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_6ae85a4465be4d3f8ad73d047071afd0",
              "IPY_MODEL_02dbebc64fc143fdbaac11688c756529",
              "IPY_MODEL_1bdc5c3357344011b4928ed49ec54c73"
            ],
            "layout": "IPY_MODEL_17f7097a864a4ff5ab710968ca6f237a"
          }
        },
        "6ae85a4465be4d3f8ad73d047071afd0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5f9ddae8700a4c26bc0a3c87f15bf590",
            "placeholder": "​",
            "style": "IPY_MODEL_dea6bb2491e24ab19aef6a26df91e03c",
            "value": "Epoch 2/5: 100%"
          }
        },
        "02dbebc64fc143fdbaac11688c756529": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fbac2871c02a4736a4cde808ba031d82",
            "max": 362,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a281e30af9bc435fa6c5d7e7117717d7",
            "value": 362
          }
        },
        "1bdc5c3357344011b4928ed49ec54c73": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_89ae1ad5288c4568b10843e09b1a4e19",
            "placeholder": "​",
            "style": "IPY_MODEL_0765fd15d29f4cb199e4ba2052dc1380",
            "value": " 362/362 [00:06&lt;00:00, 56.87it/s, loss=0.633, mean_absolute_error=0.497]"
          }
        },
        "17f7097a864a4ff5ab710968ca6f237a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5f9ddae8700a4c26bc0a3c87f15bf590": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dea6bb2491e24ab19aef6a26df91e03c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "fbac2871c02a4736a4cde808ba031d82": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a281e30af9bc435fa6c5d7e7117717d7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "89ae1ad5288c4568b10843e09b1a4e19": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0765fd15d29f4cb199e4ba2052dc1380": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "0940d99b57c04f0bab5140980362fcbc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_12355cd4f2ce420a8204f1517013a49b",
              "IPY_MODEL_5f1907a04d6f416591d6dfbb8a5c2034",
              "IPY_MODEL_93e0e6fe8a3d4dec9ba4454bfab155e2"
            ],
            "layout": "IPY_MODEL_91dbeacf3278411e9bf3d0205fa0fd67"
          }
        },
        "12355cd4f2ce420a8204f1517013a49b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ab8aa6b5bafc4789b42f68a7f6b79bc9",
            "placeholder": "​",
            "style": "IPY_MODEL_ad492c643ab24f1cbbc25dd31f313b50",
            "value": "Epoch 3/5: 100%"
          }
        },
        "5f1907a04d6f416591d6dfbb8a5c2034": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d1ff579fe62a43a0a27202f53d705c1f",
            "max": 362,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_5ae6523218c5402cb84d21cfe559b6e2",
            "value": 362
          }
        },
        "93e0e6fe8a3d4dec9ba4454bfab155e2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8eb343a19b5441d9be04cb8813067880",
            "placeholder": "​",
            "style": "IPY_MODEL_25e3c54988894854989b216354f1eaae",
            "value": " 362/362 [00:07&lt;00:00, 58.03it/s, loss=0.624, mean_absolute_error=0.498]"
          }
        },
        "91dbeacf3278411e9bf3d0205fa0fd67": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ab8aa6b5bafc4789b42f68a7f6b79bc9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ad492c643ab24f1cbbc25dd31f313b50": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d1ff579fe62a43a0a27202f53d705c1f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5ae6523218c5402cb84d21cfe559b6e2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "8eb343a19b5441d9be04cb8813067880": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "25e3c54988894854989b216354f1eaae": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3a78ba5865d447f68a274e4c73a3a8d4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_b05cd9e9b3b64a66b9c406cfd51953e7",
              "IPY_MODEL_f0f2f5bc89cb4d1f98ba694fe1525a9d",
              "IPY_MODEL_55034ab0f57649ee9c7f230a098ea174"
            ],
            "layout": "IPY_MODEL_42648b1e10dd4ef5be1f09d9b8b1d3d3"
          }
        },
        "b05cd9e9b3b64a66b9c406cfd51953e7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_64cae0ba53de4ee8b12dfeb1a4bd13fe",
            "placeholder": "​",
            "style": "IPY_MODEL_1c92ffd2d2044f2889e3d295ca4d2400",
            "value": "Epoch 4/5: 100%"
          }
        },
        "f0f2f5bc89cb4d1f98ba694fe1525a9d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5fb7a86014af45a994654bc5d44757be",
            "max": 362,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c4805b48675b4004be8f766176336399",
            "value": 362
          }
        },
        "55034ab0f57649ee9c7f230a098ea174": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_62792003e83240e585f81e5d068e6b6f",
            "placeholder": "​",
            "style": "IPY_MODEL_68159a1057bf4bd19143b60a15c2ed64",
            "value": " 362/362 [00:06&lt;00:00, 55.13it/s, loss=0.604, mean_absolute_error=0.494]"
          }
        },
        "42648b1e10dd4ef5be1f09d9b8b1d3d3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "64cae0ba53de4ee8b12dfeb1a4bd13fe": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1c92ffd2d2044f2889e3d295ca4d2400": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5fb7a86014af45a994654bc5d44757be": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c4805b48675b4004be8f766176336399": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "62792003e83240e585f81e5d068e6b6f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "68159a1057bf4bd19143b60a15c2ed64": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9dc7451db5274a9b822c612b531f2b92": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_d6afb5651cf54d14bd0b3462470999b9",
              "IPY_MODEL_d4ad7b7008eb4f4b829b76f6e92d8d88",
              "IPY_MODEL_228bfa3e9a714d899fa087dd9f528b03"
            ],
            "layout": "IPY_MODEL_c26625f17c094e1aac4587a441f10c17"
          }
        },
        "d6afb5651cf54d14bd0b3462470999b9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_798bd66f76244413aa7fad4be1a5580f",
            "placeholder": "​",
            "style": "IPY_MODEL_0a96edbe20144016be1776bbed4c0384",
            "value": "Epoch 5/5: 100%"
          }
        },
        "d4ad7b7008eb4f4b829b76f6e92d8d88": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c24a35d73f0e4cfd9a10cf5fe8ad7aad",
            "max": 362,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_ff290b8f6aeb4d5dafa010459dadb0a2",
            "value": 362
          }
        },
        "228bfa3e9a714d899fa087dd9f528b03": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c171b8b4dd8943ddbb129ef53266bb3f",
            "placeholder": "​",
            "style": "IPY_MODEL_3e1dbf3276b644888f5414c64dfb1539",
            "value": " 362/362 [00:06&lt;00:00, 59.65it/s, loss=0.626, mean_absolute_error=0.502]"
          }
        },
        "c26625f17c094e1aac4587a441f10c17": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "798bd66f76244413aa7fad4be1a5580f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0a96edbe20144016be1776bbed4c0384": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c24a35d73f0e4cfd9a10cf5fe8ad7aad": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ff290b8f6aeb4d5dafa010459dadb0a2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c171b8b4dd8943ddbb129ef53266bb3f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3e1dbf3276b644888f5414c64dfb1539": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a5bf67e880ee4d489c535bbcb5c6f20c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_5275011748fd4bf58a82e1681eeb4922",
              "IPY_MODEL_0c78a4d6df2c4b3c8d3cd2d0cabe31cd",
              "IPY_MODEL_7aa8935aa937437c85d1e7f99929b91b"
            ],
            "layout": "IPY_MODEL_5bbeda83519c40c68d983937cd5e1427"
          }
        },
        "5275011748fd4bf58a82e1681eeb4922": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c239fc78863a4cf59d94283e4ab0bff3",
            "placeholder": "​",
            "style": "IPY_MODEL_3dc28e9e52f540c4ad4100cac468ac71",
            "value": "All epochs: 100%"
          }
        },
        "0c78a4d6df2c4b3c8d3cd2d0cabe31cd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_50f7f33158f3430795f30cfbdb403fc1",
            "max": 5,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a5bbc61767c84346a957a9927545df15",
            "value": 5
          }
        },
        "7aa8935aa937437c85d1e7f99929b91b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_af583fffad9f4735848109acafec66cc",
            "placeholder": "​",
            "style": "IPY_MODEL_54eb303dbbe94baeaaa27a32e4d2d1a3",
            "value": " 5/5 [02:27&lt;00:00, 29.31s/it]"
          }
        },
        "5bbeda83519c40c68d983937cd5e1427": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c239fc78863a4cf59d94283e4ab0bff3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3dc28e9e52f540c4ad4100cac468ac71": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "50f7f33158f3430795f30cfbdb403fc1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a5bbc61767c84346a957a9927545df15": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "af583fffad9f4735848109acafec66cc": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "54eb303dbbe94baeaaa27a32e4d2d1a3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "16e8a6a5694d4baf8c97636b27d42340": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_64c2ec4df5d14ee098ea49eb623d6383",
              "IPY_MODEL_0375f6d79e224a358c6f2dcd497100fd",
              "IPY_MODEL_9550365c52e74ed88299f6abbfe546f3"
            ],
            "layout": "IPY_MODEL_b33c062dd1aa4dc1acb328b14e030477"
          }
        },
        "64c2ec4df5d14ee098ea49eb623d6383": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_53d902a9435e43b78becd5d0f3397662",
            "placeholder": "​",
            "style": "IPY_MODEL_d75da4ead9d84260bc94a146da655432",
            "value": "Epoch 1/5: 100%"
          }
        },
        "0375f6d79e224a358c6f2dcd497100fd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e776c18876b645c9b410012a8c2839ee",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_947f75b505a84e9e8186291eddff37d5",
            "value": 1718
          }
        },
        "9550365c52e74ed88299f6abbfe546f3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3940f30affcf44e4850f5cfc7d4ab2ef",
            "placeholder": "​",
            "style": "IPY_MODEL_ac92a864ef3d4df9b452f7995f045d0f",
            "value": " 1718/1718 [00:30&lt;00:00, 63.74it/s, loss=0.488, sparse_categorical_accuracy=0.823]"
          }
        },
        "b33c062dd1aa4dc1acb328b14e030477": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "53d902a9435e43b78becd5d0f3397662": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d75da4ead9d84260bc94a146da655432": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e776c18876b645c9b410012a8c2839ee": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "947f75b505a84e9e8186291eddff37d5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "3940f30affcf44e4850f5cfc7d4ab2ef": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ac92a864ef3d4df9b452f7995f045d0f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "41a169bb652f4b159e88144d96b42630": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_bac5345614024b0d9fce55ff48d6617b",
              "IPY_MODEL_ce4e7fb875914a3a94428bc8ea396faa",
              "IPY_MODEL_2c067f5a2c1d4a208cbef9143a5fc0bd"
            ],
            "layout": "IPY_MODEL_8be807d184b640fa9dee8be49aa44c9a"
          }
        },
        "bac5345614024b0d9fce55ff48d6617b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_46988fa86c74460caf12e3c0f13056e0",
            "placeholder": "​",
            "style": "IPY_MODEL_22172cdafbeb40f6a34d48ce2e959958",
            "value": "Epoch 2/5: 100%"
          }
        },
        "ce4e7fb875914a3a94428bc8ea396faa": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b0bd186ad3b7474798553729787ed050",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_963141319c234c809275f7ed9937ea81",
            "value": 1718
          }
        },
        "2c067f5a2c1d4a208cbef9143a5fc0bd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_21895c30c0364b3d800125f451ccb0c7",
            "placeholder": "​",
            "style": "IPY_MODEL_0bee1241aed145df8bde858b085b4472",
            "value": " 1718/1718 [00:29&lt;00:00, 62.91it/s, loss=0.403, sparse_categorical_accuracy=0.856]"
          }
        },
        "8be807d184b640fa9dee8be49aa44c9a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "46988fa86c74460caf12e3c0f13056e0": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "22172cdafbeb40f6a34d48ce2e959958": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b0bd186ad3b7474798553729787ed050": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "963141319c234c809275f7ed9937ea81": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "21895c30c0364b3d800125f451ccb0c7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0bee1241aed145df8bde858b085b4472": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c7d4cadae597484488816817c7364cbb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_431f277a300842299dda90f4a14d0d3f",
              "IPY_MODEL_0aa8a3273b284c649e2007335b6e6ab1",
              "IPY_MODEL_396ac00cbe4049e7bd3df3803a370c55"
            ],
            "layout": "IPY_MODEL_5977e0082b0043239abc185d5ca64cf0"
          }
        },
        "431f277a300842299dda90f4a14d0d3f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5c70dd398c8c42fa90d4ac4d8530c914",
            "placeholder": "​",
            "style": "IPY_MODEL_563e717abb0a4ed5ba5aea925ec55e89",
            "value": "Epoch 3/5: 100%"
          }
        },
        "0aa8a3273b284c649e2007335b6e6ab1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a0410ba3e52d4f2e87af82b5995afb89",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_428a2eb5eef04cc1a82c48b8ad67be21",
            "value": 1718
          }
        },
        "396ac00cbe4049e7bd3df3803a370c55": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_35f7be75d3724b6da1d28d955897a099",
            "placeholder": "​",
            "style": "IPY_MODEL_57882a882d0e441f99caf39bee240fad",
            "value": " 1718/1718 [00:28&lt;00:00, 65.13it/s, loss=0.377, sparse_categorical_accuracy=0.863]"
          }
        },
        "5977e0082b0043239abc185d5ca64cf0": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5c70dd398c8c42fa90d4ac4d8530c914": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "563e717abb0a4ed5ba5aea925ec55e89": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a0410ba3e52d4f2e87af82b5995afb89": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "428a2eb5eef04cc1a82c48b8ad67be21": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "35f7be75d3724b6da1d28d955897a099": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "57882a882d0e441f99caf39bee240fad": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "210b9395240047b285140027e0104d2b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_5c8c1dad6c4249e082294af3a4c9e8e4",
              "IPY_MODEL_ff21aed62a8e436c98923e766b700386",
              "IPY_MODEL_577542a705ee4e2fa596bceb10b588fc"
            ],
            "layout": "IPY_MODEL_68f0ed27908545b09fa2170c6158b509"
          }
        },
        "5c8c1dad6c4249e082294af3a4c9e8e4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_509cc203057e4d21b567a0c4b9c5d126",
            "placeholder": "​",
            "style": "IPY_MODEL_c359589f1e6c4f78bb1e088d18486c7b",
            "value": "Epoch 4/5: 100%"
          }
        },
        "ff21aed62a8e436c98923e766b700386": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7fc3aba71daa45cdb330407b71e7d0d6",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_5bb93e7d16234a0a807f8e1ab631d50a",
            "value": 1718
          }
        },
        "577542a705ee4e2fa596bceb10b588fc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2c76f8ee6a9848ee9b053c75e5ba4a95",
            "placeholder": "​",
            "style": "IPY_MODEL_58a990b1be784743bd4fd438dc0dcfd9",
            "value": " 1718/1718 [00:29&lt;00:00, 51.08it/s, loss=0.372, sparse_categorical_accuracy=0.865]"
          }
        },
        "68f0ed27908545b09fa2170c6158b509": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "509cc203057e4d21b567a0c4b9c5d126": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c359589f1e6c4f78bb1e088d18486c7b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "7fc3aba71daa45cdb330407b71e7d0d6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5bb93e7d16234a0a807f8e1ab631d50a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "2c76f8ee6a9848ee9b053c75e5ba4a95": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "58a990b1be784743bd4fd438dc0dcfd9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2fa6a92d7ea746d3af706461c4ce6f68": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_fbe57b37a5d14410ac386e0484589dc9",
              "IPY_MODEL_2d67a7559af1493e874a46b91eaeb3f0",
              "IPY_MODEL_b807a014cabf4fef9d74f671f00de27c"
            ],
            "layout": "IPY_MODEL_7ea5c6a990f04ceab01a92854ed173e7"
          }
        },
        "fbe57b37a5d14410ac386e0484589dc9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_01a59b6835c345b59e9c2748f979f13f",
            "placeholder": "​",
            "style": "IPY_MODEL_7a5878e8454c40718f68ae4c2dbe08d7",
            "value": "Epoch 5/5: 100%"
          }
        },
        "2d67a7559af1493e874a46b91eaeb3f0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6b5e6bd8111a4b608c41338f2655f00e",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_8fea2d9d520a4656b04a3c3cf2d2be7e",
            "value": 1718
          }
        },
        "b807a014cabf4fef9d74f671f00de27c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_cfd33c56db5644bbbf8aa8f0450c76e8",
            "placeholder": "​",
            "style": "IPY_MODEL_a938518cafd94fc39f1584d94ddc15df",
            "value": " 1718/1718 [00:29&lt;00:00, 62.99it/s, loss=0.366, sparse_categorical_accuracy=0.865]"
          }
        },
        "7ea5c6a990f04ceab01a92854ed173e7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "01a59b6835c345b59e9c2748f979f13f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7a5878e8454c40718f68ae4c2dbe08d7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6b5e6bd8111a4b608c41338f2655f00e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8fea2d9d520a4656b04a3c3cf2d2be7e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "cfd33c56db5644bbbf8aa8f0450c76e8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a938518cafd94fc39f1584d94ddc15df": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5d4e8f146ea84d9399168f62a0b20a22": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_7295024fb6e14912933f06f2e7fdd50a",
              "IPY_MODEL_ff605a415aa3435f94d0c156cef6712f",
              "IPY_MODEL_30814c93255b447baf16bb77ba004624"
            ],
            "layout": "IPY_MODEL_af7953b6788f424694947f25f10d22b7"
          }
        },
        "7295024fb6e14912933f06f2e7fdd50a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2b05554510514e5b81e456e3051ecfc6",
            "placeholder": "​",
            "style": "IPY_MODEL_3b45c0028c3647cbbc910744e7f1572a",
            "value": "All epochs: 100%"
          }
        },
        "ff605a415aa3435f94d0c156cef6712f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7ae7c237afcd401aba969bf38ee82f7d",
            "max": 5,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_3659d1e6ae574720bfc5dcca4674f199",
            "value": 5
          }
        },
        "30814c93255b447baf16bb77ba004624": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dfc48b5ccf6944ea9244bc4d8e2ca61e",
            "placeholder": "​",
            "style": "IPY_MODEL_e00cb0b2fb3d4a0da8d7997584797650",
            "value": " 5/5 [02:51&lt;00:00, 34.26s/it]"
          }
        },
        "af7953b6788f424694947f25f10d22b7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2b05554510514e5b81e456e3051ecfc6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3b45c0028c3647cbbc910744e7f1572a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "7ae7c237afcd401aba969bf38ee82f7d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3659d1e6ae574720bfc5dcca4674f199": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "dfc48b5ccf6944ea9244bc4d8e2ca61e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e00cb0b2fb3d4a0da8d7997584797650": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "ca930a32ebd849889cf12875dd0f67b6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_ada21ea4ca974932bacd823dc608d922",
              "IPY_MODEL_43deb6bade0048b0a177bb1705e6f8c0",
              "IPY_MODEL_05ebedfd4ec54fcf9e1e74834cab8aab"
            ],
            "layout": "IPY_MODEL_03ed7016a45f4668937766a6ada6f341"
          }
        },
        "ada21ea4ca974932bacd823dc608d922": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4d9e7c0435734007a80ba323ffcd6e8a",
            "placeholder": "​",
            "style": "IPY_MODEL_6e1b71a829694e5ea98b8ad1974c0ffd",
            "value": "Epoch 1/5: 100%"
          }
        },
        "43deb6bade0048b0a177bb1705e6f8c0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2e9d4374618347769bfc17b696739864",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_e185797b956345c6850e54d6863a8c88",
            "value": 1718
          }
        },
        "05ebedfd4ec54fcf9e1e74834cab8aab": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ce76b277e82f49ce9dfbb77432c6e17f",
            "placeholder": "​",
            "style": "IPY_MODEL_2d9e4688242740e18e6504c563af7b06",
            "value": " 1718/1718 [00:35&lt;00:00, 52.68it/s, loss=1.04, sparse_categorical_accuracy=0.679]"
          }
        },
        "03ed7016a45f4668937766a6ada6f341": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4d9e7c0435734007a80ba323ffcd6e8a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6e1b71a829694e5ea98b8ad1974c0ffd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2e9d4374618347769bfc17b696739864": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e185797b956345c6850e54d6863a8c88": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "ce76b277e82f49ce9dfbb77432c6e17f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2d9e4688242740e18e6504c563af7b06": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f0e31186bd2d48bfb2cb19bba6a72e7d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_45cb7611404344bb9b0c700b68f830ff",
              "IPY_MODEL_8adcc12c64cc4c89903f307df647941b",
              "IPY_MODEL_8fb9c0ddd14b4970be3f839a7f9b3f7c"
            ],
            "layout": "IPY_MODEL_949de89c5b5341f7b9109debf207d9f7"
          }
        },
        "45cb7611404344bb9b0c700b68f830ff": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b089decce7a54a7c88d8fdafc41fa507",
            "placeholder": "​",
            "style": "IPY_MODEL_a3e33c9748d14c9ba725fbcc1dc214b7",
            "value": "Epoch 2/5: 100%"
          }
        },
        "8adcc12c64cc4c89903f307df647941b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7d39078a748a4f3cb6873ffad42e24a9",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_ac0c420d7c0a4cba97b812536938f593",
            "value": 1718
          }
        },
        "8fb9c0ddd14b4970be3f839a7f9b3f7c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d07e79cfcebd40b0a07a95c1acbfa4ef",
            "placeholder": "​",
            "style": "IPY_MODEL_daeafd631cc94a8ba3185a65adfa9dd3",
            "value": " 1718/1718 [00:34&lt;00:00, 46.26it/s, loss=0.634, sparse_categorical_accuracy=0.788]"
          }
        },
        "949de89c5b5341f7b9109debf207d9f7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b089decce7a54a7c88d8fdafc41fa507": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a3e33c9748d14c9ba725fbcc1dc214b7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "7d39078a748a4f3cb6873ffad42e24a9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ac0c420d7c0a4cba97b812536938f593": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d07e79cfcebd40b0a07a95c1acbfa4ef": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "daeafd631cc94a8ba3185a65adfa9dd3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "baeae41f27eb44aa9ee10b0175a23963": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_5e1c0d1633914a3e9c727f8d5e8d0a02",
              "IPY_MODEL_144f057af6864174970cc00b84129319",
              "IPY_MODEL_7b1a6f1880084faf81d9a6d90218209e"
            ],
            "layout": "IPY_MODEL_1f61d7bbd81f420f99192e371fd7492f"
          }
        },
        "5e1c0d1633914a3e9c727f8d5e8d0a02": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_840429aa8c824433b0ba04b72ae9df19",
            "placeholder": "​",
            "style": "IPY_MODEL_80c2a1dd166f4181b25d405094164a97",
            "value": "Epoch 3/5: 100%"
          }
        },
        "144f057af6864174970cc00b84129319": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b8e8750c344f4920a5c18a19807995a9",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_5d126c9362ad483f96d21e36bfd2e18a",
            "value": 1718
          }
        },
        "7b1a6f1880084faf81d9a6d90218209e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a661a05063744fe5bb21718539f354da",
            "placeholder": "​",
            "style": "IPY_MODEL_368897a043de49a6a63b37b100af0a9f",
            "value": " 1718/1718 [00:33&lt;00:00, 55.39it/s, loss=0.564, sparse_categorical_accuracy=0.805]"
          }
        },
        "1f61d7bbd81f420f99192e371fd7492f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "840429aa8c824433b0ba04b72ae9df19": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "80c2a1dd166f4181b25d405094164a97": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b8e8750c344f4920a5c18a19807995a9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5d126c9362ad483f96d21e36bfd2e18a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "a661a05063744fe5bb21718539f354da": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "368897a043de49a6a63b37b100af0a9f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c497ac7d70cb49ccb3444a9887570a8b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_ad194a229a3347e681d81290887db18b",
              "IPY_MODEL_49c1e31accf24ddca07c3244fd7c2c28",
              "IPY_MODEL_4c339586df73458f892ce3b4a35680b4"
            ],
            "layout": "IPY_MODEL_4c63ae7ca560448f88139e9124c5e849"
          }
        },
        "ad194a229a3347e681d81290887db18b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_46b590f489d34e15852c03f378d9edc6",
            "placeholder": "​",
            "style": "IPY_MODEL_85bfd752cafd42ee8649e345f9fbe603",
            "value": "Epoch 4/5: 100%"
          }
        },
        "49c1e31accf24ddca07c3244fd7c2c28": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6ec9aef1644d4729a38f2775e3a3364e",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_2550a941d6e34f99a07cd67c0fad7385",
            "value": 1718
          }
        },
        "4c339586df73458f892ce3b4a35680b4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1dbdb1d667f049e199b1964260444ea8",
            "placeholder": "​",
            "style": "IPY_MODEL_a7e86ff605f44c798eaf5ec67ca19a04",
            "value": " 1718/1718 [00:34&lt;00:00, 54.23it/s, loss=0.531, sparse_categorical_accuracy=0.815]"
          }
        },
        "4c63ae7ca560448f88139e9124c5e849": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "46b590f489d34e15852c03f378d9edc6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "85bfd752cafd42ee8649e345f9fbe603": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6ec9aef1644d4729a38f2775e3a3364e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2550a941d6e34f99a07cd67c0fad7385": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1dbdb1d667f049e199b1964260444ea8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a7e86ff605f44c798eaf5ec67ca19a04": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c89d971ed2dc41dc8b3bc9ca1d23f885": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_37cacb046e2b41198083898375175e06",
              "IPY_MODEL_1a34e2c9150b4400ba819d480a46ae4f",
              "IPY_MODEL_17fe2f0e26dd42d593d427eb42400677"
            ],
            "layout": "IPY_MODEL_93f2b8bdb0144bc18c2cf3adde03a1c2"
          }
        },
        "37cacb046e2b41198083898375175e06": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_91a39369e173414189c4ee832b73cdfa",
            "placeholder": "​",
            "style": "IPY_MODEL_8068340cdc544fb3a2f2812c0eb7d812",
            "value": "Epoch 5/5: 100%"
          }
        },
        "1a34e2c9150b4400ba819d480a46ae4f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_aef4e23b50e6425ab9e30d3f122ca19f",
            "max": 1718,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_37e7134b26154e39b0b824a848b1a98c",
            "value": 1718
          }
        },
        "17fe2f0e26dd42d593d427eb42400677": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_197aa6c1f53946689081da33ac857a48",
            "placeholder": "​",
            "style": "IPY_MODEL_885e07d27de14b5090594de705c72292",
            "value": " 1718/1718 [00:34&lt;00:00, 53.22it/s, loss=0.516, sparse_categorical_accuracy=0.82]"
          }
        },
        "93f2b8bdb0144bc18c2cf3adde03a1c2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "91a39369e173414189c4ee832b73cdfa": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8068340cdc544fb3a2f2812c0eb7d812": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "aef4e23b50e6425ab9e30d3f122ca19f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "37e7134b26154e39b0b824a848b1a98c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "197aa6c1f53946689081da33ac857a48": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "885e07d27de14b5090594de705c72292": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}