redes-neuronales/Cuadernos/Pytorch_lightning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "116e67da-ad3d-4268-b485-65ef7172f1da",
   "metadata": {},
   "source": [
    "<figure>\n",
    "<img src=\"../Imagenes/logo-final-ap.png\"  width=\"80\" height=\"80\" align=\"left\"/> \n",
    "</figure>\n",
    "\n",
    "# <span style=\"color:blue\"><left>Aprendizaje Profundo</left></span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f36a866-5fee-4fc7-bce7-85c19e06d35a",
   "metadata": {
    "jupyter": {
     "source_hidden": true
    },
    "tags": []
   },
   "source": [
    "# <span style=\"color:red\"><center>Pytorch-lightning</center></span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8a2f8b6-3e8f-4bf0-8780-6e2ad9efee7e",
   "metadata": {},
   "source": [
    "<center>Introducción</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7436142-a95f-4973-9439-e100b0f0a2aa",
   "metadata": {},
   "source": [
    "<figure>\n",
    "<center>\n",
    "<img src=\"../Imagenes/trainer.png\" width=\"800\" height=\"800\" align=\"center\"/>\n",
    "</center>\n",
    "</figure>\n",
    "\n",
    "\n",
    "Fuente: Alvaro Montenegro"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9a4a353-7bea-4b7f-8713-69146e84066e",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## <span style=\"color:#4361EE\">Coordinador</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a9080bdf-eee3-41b7-bcc3-4763ab317d4d",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "- Campo Elías Pardo, PhD, cepardot@unal.edu.co"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8908ae7-7fbc-46da-bc45-2c7e1d463c2e",
   "metadata": {},
   "source": [
    "## <span style=\"color:#4361EE\">Profesores</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0743573c-77ca-42e3-ad2b-f5a8a007949b",
   "metadata": {},
   "source": [
    "1. Alvaro  Montenegro, PhD, ammontenegrod@unal.edu.co\n",
    "1. Camilo José Torres Jiménez, Msc, cjtorresj@unal.edu.co\n",
    "1. Daniel  Montenegro, Msc, dextronomo@gmail.com "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "11f24bc8-7e45-4817-832e-198a639c4a03",
   "metadata": {},
   "source": [
    "## <span style=\"color:#4361EE\">Asesora Medios y Marketing digital</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be1829f0-170c-4b38-803c-11a42de77000",
   "metadata": {},
   "source": [
    "1. Maria del Pilar Montenegro, pmontenegro88@gmail.com\n",
    "1. Jessica López Mejía, jelopezme@unal.edu.co\n",
    "1. Venus Celeste Puertas Gualtero, vpuertasg@unal.edu.co"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55f021c7-ef32-4240-b02a-9cea77400eaf",
   "metadata": {},
   "source": [
    "## <span style=\"color:#4361EE\">Jefe Jurídica</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b5b84ac-46aa-4cfe-8863-d6d89d9bb475",
   "metadata": {},
   "source": [
    "6. Paula Andrea Guzmán, guzmancruz.paula@gmail.com"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f3b914bf-2cd5-4971-9df7-7c5a910ce703",
   "metadata": {},
   "source": [
    "## <span style=\"color:#4361EE\">Coordinador Jurídico</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f98595c9-612d-493a-aa14-4313c3b7a2c3",
   "metadata": {},
   "source": [
    "7. David Fuentes, fuentesd065@gmail.com"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba2f9eff-0f61-49bb-83ef-321b568eca14",
   "metadata": {},
   "source": [
    "## <span style=\"color:#4361EE\">Desarrolladores Principales</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6dbcc173-5ef1-46d8-aca2-df611344decd",
   "metadata": {},
   "source": [
    "8. Dairo Moreno, damoralesj@unal.edu.co\n",
    "9. Joan Castro, jocastroc@unal.edu.co\n",
    "10. Bryan Riveros, briveros@unal.edu.co\n",
    "11. Rosmer Vargas, rovargasc@unal.edu.co\n",
    "12. Venus Puertas, vpuertasg@unal.edu.co"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76603d60-1f01-4f5b-8366-f9754fd6f491",
   "metadata": {},
   "source": [
    "## <span style=\"color:#4361EE\">Expertos en Bases de Datos</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9d2dd8d-5bd1-436d-8a81-ff5483ece369",
   "metadata": {
    "tags": []
   },
   "source": [
    "13. Giovvani Barrera, udgiovanni@gmail.com\n",
    "14. Camilo Chitivo, cchitivo@unal.edu.co"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d4899f1-168e-45d1-a190-26b377ee0ede",
   "metadata": {},
   "source": [
    "## <span style=\"color:blue\">Referencias</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c546c17-3fbe-4bd1-826e-0bcde6fe639f",
   "metadata": {},
   "source": [
    "1. [Alvaro Montenegro y Daniel Montenegro, Inteligencia Artificial y Aprendizaje Profundo, 2023](https://github.com/AprendizajeProfundo/Diplomado)\n",
    "1. [Alvaro Montenegro, Daniel Montenegro y Oleg Jarma,  Inteligencia Artificial y Aprendizaje Profundo Avanzado, 2023](https://github.com/AprendizajeProfundo/Diplomado-Avanzado)\n",
    "1. [Tutoriales de Pytorch](https://pytorch.org/tutorials/)\n",
    "1. [Pytorchlightning.ai](https://www.pytorchlightning.ai/)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59cded78-ef97-4957-9821-71f18d27073d",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## <span style=\"color:blue\">Contenido</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e473b6b-e560-42ed-b90e-deb76d9ee9bd",
   "metadata": {},
   "source": [
    "* [Introducción](#Introducción)\n",
    "* [Instalar Pytorch-lightning](#Instalar-Pytorch-lightning)\n",
    "* [Ejemplo de un módulo Lightning](#Ejemplo-de-un-módulo-Lightning)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab698e08-33bf-45c2-8b16-306084b6bf78",
   "metadata": {
    "tags": []
   },
   "source": [
    "## <span style=\"color:blue\">Introducción</span> "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "731753f8-1484-4c67-8b73-312c1969360c",
   "metadata": {},
   "source": [
    "Tomado de [Wikipedia](https://en.wikipedia.org/wiki/PyTorch_Lightning)\n",
    "\n",
    "PyTorch Lightning es una biblioteca Python de código abierto que proporciona una interfaz de alto nivel para PyTorch , un marco de aprendizaje profundo popular. Es un marco liviano y de alto rendimiento que organiza el código PyTorch para desvincular la investigación de la ingeniería, lo que hace que los experimentos de aprendizaje profundo sean más fáciles de leer y reproducir. Está diseñada para crear modelos de aprendizaje profundo escalables que pueden ejecutarse fácilmente en hardware distribuido y mantener los modelos independientes del hardware.\n",
    "\n",
    "En 2019, Lightning fue adoptado por NeurIPS Reproducibility Challenge como estándar para enviar el código PyTorch a la conferencia.\n",
    "\n",
    "En este ejemplo introductorio construiremos un autocodificador (autoencoder) a partir de los datos de MNIST. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a806629-c97b-4d2d-9609-89e04c5d9004",
   "metadata": {},
   "source": [
    "## <span style=\"color:blue\">Características principales de Pytorch Lightning</span> "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de12ad46-32b2-421f-9100-5ca4779d5e5c",
   "metadata": {},
   "source": [
    "Propuesto por ChatGPT y curado por los autores.\n",
    "\n",
    "PyTorch Lightning es un marco de trabajo de alto nivel para entrenar modelos de aprendizaje automático utilizando PyTorch. Proporciona una estructura organizada y simplificada para acelerar el proceso de desarrollo y entrenamiento de modelos, permitiendo a los investigadores y desarrolladores centrarse más en el diseño del modelo y la lógica del entrenamiento, en lugar de preocuparse por los detalles de implementación repetitivos.\n",
    "\n",
    "A continuación, te daré una introducción completa a PyTorch Lightning destacando sus principales características y beneficios:\n",
    "\n",
    "+ `Abstracción de alto nivel`: PyTorch Lightning proporciona una abstracción de alto nivel que simplifica la estructura y el flujo de trabajo de entrenamiento de modelos. Elimina la necesidad de escribir bucles de entrenamiento y validación manuales, y maneja muchas tareas repetitivas en segundo plano, lo que facilita la escritura y comprensión del código.\n",
    "\n",
    "+ `Estructura organizada`: PyTorch Lightning define una estructura clara y organizada para el entrenamiento de modelos, separando las partes esenciales del proceso en módulos individuales. Esto incluye la definición del modelo, la configuración del optimizador y la programación de los bucles de entrenamiento y validación. Esto promueve un código más limpio, modular y fácil de mantener.\n",
    "\n",
    "+ `Funcionalidad escalable`: PyTorch Lightning facilita la escalabilidad y distribución del entrenamiento de modelos en múltiples dispositivos y nodos. Permite el entrenamiento en paralelo y distribuido con solo unos pocos cambios en el código original, lo que permite aprovechar eficientemente el poder de cómputo de hardware como GPU y TPU.\n",
    "\n",
    "+ `Mayor reusabilidad`: PyTorch Lightning fomenta la reutilización del código y la portabilidad de modelos. Permite que los modelos entrenados se exporten y compartan fácilmente, lo que facilita su implementación en entornos de producción y su colaboración con otros investigadores y desarrolladores.\n",
    "\n",
    "+ `Funcionalidades integradas`: PyTorch Lightning ofrece una variedad de funcionalidades integradas para tareas comunes de entrenamiento, como la visualización de métricas de rendimiento, el registro de experimentos, la validación cruzada y el ajuste automático de hiperparámetros. Esto evita tener que implementar estas funcionalidades desde cero y agiliza el proceso de desarrollo y experimentación.\n",
    "\n",
    "+ `Compatibilidad con PyTorch`: PyTorch Lightning es compatible con todas las funcionalidades y características de PyTorch. Puedes utilizar todas las capas, funciones y utilidades de PyTorch en conjunto con PyTorch Lightning, lo que te brinda la flexibilidad y el poder de la biblioteca base de PyTorch.\n",
    "\n",
    "En general, PyTorch Lightning es una herramienta poderosa y versátil para entrenar modelos de aprendizaje automático con PyTorch. Facilita el desarrollo, entrenamiento y escalabilidad de modelos al proporcionar una estructura organizada, funcionalidades integradas y abstracciones de alto nivel. Si ya estás familiarizado con PyTorch, PyTorch Lightning es una excelente opción para mejorar tu flujo de trabajo de entrenamiento y aumentar tu productividad.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f35c718-80ee-445f-832a-5ba8c55b2d41",
   "metadata": {},
   "source": [
    "## <span style=\"color:blue\">Concepto de auto-codificador</span> "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d32e395d-a42a-448f-991e-b33e317f4180",
   "metadata": {},
   "source": [
    "Propuesto por ChatGPT y curado por los autores.\n",
    "\n",
    "Un auto-codificador (autoencoder en inglés) es un tipo de red neuronal que se utiliza para aprender representaciones eficientes de datos de entrada sin etiquetar. Consiste en una estructura de red neuronal que se entrena para reconstruir su propia entrada, lo que significa que el objetivo del auto-codificador es aprender a replicar la entrada en su capa de salida.\n",
    "\n",
    "La arquitectura básica de un auto-codificador consta de dos partes principales: el codificador y el decodificador. El codificador transforma los datos de entrada en una representación de menor dimensión, llamada codificación o espacio latente. El decodificador, por otro lado, toma la codificación y trata de reconstruir la entrada original a partir de ella.\n",
    "\n",
    "El proceso de entrenamiento del auto-codificador se basa en la comparación entre la entrada original y la salida generada por el decodificador. El objetivo es minimizar la diferencia entre la entrada y la reconstrucción, lo que se logra a través de la optimización de una función de pérdida, como el error cuadrático medio (MSE) o la entropía cruzada.\n",
    "\n",
    "La idea principal detrás de los auto-codificadores es que, al restringir la capacidad de la red para reconstruir la entrada, se obliga al modelo a aprender una representación compacta y significativa de los datos. En el proceso de entrenamiento, el codificador aprende a capturar las características más relevantes y útiles de los datos de entrada, mientras que el decodificador aprende a generar una salida que se asemeja a la entrada original.\n",
    "\n",
    "Los auto-codificadores pueden ser utilizados para una variedad de tareas y aplicaciones. Algunos usos comunes incluyen:\n",
    "\n",
    "Reducción de dimensionalidad: Al aprender una representación de menor dimensión, los auto-codificadores pueden ser utilizados para comprimir y reducir la dimensionalidad de los datos, lo que es útil en tareas de visualización y almacenamiento eficiente.\n",
    "\n",
    "Eliminación de ruido: Los auto-codificadores pueden ser entrenados para reconstruir la entrada original a pesar de la presencia de ruido o corrupción en los datos.\n",
    "\n",
    "Extracción de características: Al aprender representaciones significativas de los datos, los auto-codificadores pueden ser utilizados como extractores de características para tareas de clasificación u otras tareas de aprendizaje automático.\n",
    "\n",
    "Generación de datos: Al muestrear de manera aleatoria puntos en el espacio latente, los auto-codificadores pueden ser utilizados para generar nuevos datos similares a los datos de entrenamiento.\n",
    "\n",
    "En resumen, los auto-codificadores son redes neuronales utilizadas para aprender representaciones eficientes y compactas de los datos de entrada. Mediante la combinación de un codificador y un decodificador, los auto-codificadores pueden capturar las características más relevantes de los datos y reconstruirlos de manera efectiva.\n",
    "\n",
    "La siguiente imagen ilustra el concepto de auto-codificador."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6401f82-d004-49ee-a420-bc75d1121f8c",
   "metadata": {},
   "source": [
    "\n",
    "<figure>\n",
    "<center>\n",
    "<img src=https://upload.wikimedia.org/wikipedia/commons/2/28/Autoencoder_structure.png width=\"600\" height=\"600\" align=\"center\"/>\n",
    "</center>\n",
    "</figure>\n",
    "\n",
    "\n",
    "Fuente: <a href=\"https://commons.wikimedia.org/wiki/File:Autoencoder_structure.png\">Chervinskii</a>, <a href=\"https://creativecommons.org/licenses/by-sa/4.0\">CC BY-SA 4.0</a>, via Wikimedia Commons"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8fba9201-55b9-4d56-aac5-3cd4a2c73c5c",
   "metadata": {},
   "source": [
    "## <span style=\"color:blue\">Los datos de Mnist</span> "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60d057a7-eb92-41a4-a368-8a43a61d6903",
   "metadata": {},
   "source": [
    "Propuesto por ChatGPT y curado por los autores.\n",
    "\n",
    "El conjunto de datos MNIST es uno de los conjuntos de datos más populares y ampliamente utilizados en el campo del aprendizaje automático. MNIST representa el conjunto de dígitos escritos a mano en forma de imágenes en escala de grises. El nombre \"MNIST\" es una abreviatura de la base de datos modificada del Instituto Nacional de Estándares y Tecnología (MNIST en inglés: Modified National Institute of Standards and Technology).\n",
    "\n",
    "La historia de MNIST se remonta a finales de la década de 1990, cuando los dígitos escritos a mano se convirtieron en un desafío popular para la comunidad de aprendizaje automático. Los dígitos escritos a mano eran un problema interesante debido a su variabilidad y la capacidad de reconocerlos se consideraba un hito en el desarrollo de algoritmos de aprendizaje automático.\n",
    "\n",
    "El conjunto de datos MNIST se recopiló a partir de dos fuentes principales:\n",
    "\n",
    "NIST: La base de datos original fue creada por el Instituto Nacional de Estándares y Tecnología de los Estados Unidos (NIST). Contenía imágenes de dígitos escritos a mano recolectados de empleados del censo estadounidense y estudiantes de secundaria.\n",
    "\n",
    "Yann LeCun y su equipo: Yann LeCun, Corinna Cortes y Christopher J.C. Burges, recopilaron una versión modificada de la base de datos MNIST utilizando los datos del NIST. Ajustaron el formato y el tamaño de las imágenes, y crearon un conjunto de entrenamiento y un conjunto de prueba bien definidos.\n",
    "\n",
    "La versión modificada de MNIST se ha convertido en un conjunto de datos de referencia para probar y comparar algoritmos de aprendizaje automático, especialmente en tareas de reconocimiento de dígitos escritos a mano. El conjunto de datos consta de 60,000 imágenes de entrenamiento y 10,000 imágenes de prueba, todas de tamaño 28x28 píxeles en escala de grises. Cada imagen se etiqueta con el dígito que representa (del 0 al 9).\n",
    "\n",
    "MNIST ha sido utilizado ampliamente en la investigación y en la enseñanza de algoritmos de aprendizaje automático. Ha proporcionado una base sólida para el desarrollo de técnicas avanzadas de procesamiento de imágenes y reconocimiento de patrones. Además, ha permitido evaluar y comparar el rendimiento de diferentes modelos y algoritmos en una tarea común y bien definida.\n",
    "\n",
    "Aunque MNIST ha sido un conjunto de datos de referencia durante mucho tiempo, en los últimos años se ha considerado relativamente fácil para los modelos de aprendizaje profundo debido a su simplicidad y falta de variabilidad. Sin embargo, sigue siendo una buena opción para comenzar a explorar algoritmos y técnicas de aprendizaje automático antes de pasar a conjuntos de datos más complejos.\n",
    "\n",
    "Las imágenes en MNIST son imágenes en escala de grises de dígitos escritos a mano. Cada imagen tiene un tamaño de 28x28 píxeles y representa un solo dígito del 0 al 9. Cada píxel tiene un valor de intensidad entre 0 y 255, donde 0 representa el color negro y 255 representa el color blanco.\n",
    "\n",
    "El siguiente código de Pytorch ilustra mustra algunos ejemplos ampliados del conjunto de datos de MNIST:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "0259c4c8-4cd2-458d-a778-9efe6e501a54",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjkAAAJOCAYAAABVzBYUAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABZvElEQVR4nO3deZzO9f7/8ddlLGMZw6AGZZLdIUuyJYNsIVskpUwd0SadmihfajpqVCedVtliEhFtUqkczehU6iRRjiVkKDshDdnm/fvDz3Xm9Z6Za2auua6Zud7zuN9ubrfreX2294xX49Xnes/74zHGGAEAAHBMicIeAAAAQDDQ5AAAACfR5AAAACfR5AAAACfR5AAAACfR5AAAACfR5AAAACfR5AAAACfR5AAAACcFpclJSkoSj8eT7Z+UlBTvvomJifLee+9lOkdKSkqmfQvThg0bJCEhQVJTUwvkeue//qz+fP311wUyhsJGHQXGH3/8Iffdd5/UqFFDwsPDpXnz5rJw4cICu35ho44Cb9asWeLxeKRChQqFcv3CQB3l37Fjx2Ts2LHSvXt3qVatmng8HklISAjqNUsG8+Rz5syRhg0bZnq/cePG3teJiYkyaNAg6d+/v9qnZcuWsmrVKrVvYdqwYYM89thj0qlTJ7nkkksK7LqJiYnSuXNn9V6TJk0K7PpFAXWUPwMHDpRvv/1WnnzySalfv7688cYbMnToUElPT5cbb7yxQMZQFFBHgbFr1y6Jj4+XGjVqyNGjRwv02kUBdeS/Q4cOyYwZM6RZs2bSv39/mTVrVtCvGdQmp0mTJtKqVSu/jq1YsaK0bds2wCMKPfXq1Sv23wfqyH8fffSRLF++3NvYiIh07txZduzYIQ8++KAMGTJEwsLCCnmUBYM6Cow77rhDOnbsKFFRUfLWW28V9nAKHHXkv5iYGDl8+LB4PB45ePBggTQ5hTonx+PxSFpamrz22mveW36dOnUSkexv6yUlJUmDBg2kTJky0qhRI5k7d67ExcWpLjS7Y1NTU8Xj8UhSUpJ6f/Xq1dK3b1+JioqS8PBwadGihSxatEhdc/DgwSJy7h+I82M9f57ly5dLv3795KKLLpLw8HCpW7eujBo1Sg4ePBiIbxNyQB1l791335UKFSp4r3verbfeKrt375ZvvvnG73O7hjrK2bx582TlypUyderUfJ/LVdSR7++Nx+Px+3h/BPVOztmzZ+XMmTPqPY/H4/0/x1WrVkmXLl2kc+fOMnHiRBE51+lmJykpSW699Vbp16+fTJkyRY4ePSoJCQly8uRJKVHCv34tOTlZevbsKW3atJFp06ZJZGSkLFy4UIYMGSLHjx+XuLg46d27tyQmJsr48ePl5ZdflpYtW4qISJ06dUREZNu2bdKuXTsZMWKEREZGSmpqqjz77LPSoUMH+fHHH6VUqVLq64+Njc31Z7J333233HDDDVKuXDlp166dTJw4UTp06ODX1xqqqCP/62j9+vXSqFEjKVlS/6d+2WWXebe3b9/er6851FBH+ft5tH//frnvvvvkySeflIsuusivr88F1FH+/10rUCYI5syZY0Qkyz9hYWFq3/Lly5vhw4dnOkdycrIREZOcnGyMMebs2bOmRo0apmXLliY9Pd27X2pqqilVqpSJiYnJ9tjztm/fbkTEzJkzx/tew4YNTYsWLczp06fVvn369DHVq1c3Z8+eNcYYs3jx4izPaUtPTzenT582O3bsMCJilixZoraHhYWZLl26+DyHMcasWbPGjBkzxrz77rvm888/N7NnzzaNGjUyYWFh5uOPP87xeBdQR/mvo3r16pkePXpken/37t1GRExiYmKO5wh11FH+68gYY6677jrTvn1779c7fPhwU758+Vwd6wLqKDB1dN6BAweMiJhHH300T8flVVDv5MydO1caNWqk3vP3VtXmzZtl9+7dcv/996tzxMTESPv27f2aHb5161bZtGmTPPPMMyIiqjvv1auXfPDBB7J58+ZMX4Nt//798sgjj8iHH34ou3fvlvT0dO+2jRs3St++fb3Z/j+A7LRo0UJatGjhzVdddZUMGDBAmjZtKmPHjpUePXrk6jwuoI78ryMR39+rgr51XJioI//r6O2335alS5fK999/X6xqJivUUf5+HhW0oDY5jRo18nuClu3QoUMiIhIdHZ1pW3R0tF/FsG/fPhERiY+Pl/j4+Cz3yenzx/T0dOnevbvs3r1bJk6cKE2bNpXy5ctLenq6tG3bVk6cOJHncWWnUqVK0qdPH5k2bZqcOHFCypYtG7BzF2XUkf91VKVKFe/XnNFvv/0mIiJRUVF+nTcUUUf+1dEff/whd999t4wePVpq1KghR44cERGRU6dOiYjIkSNHpFSpUlK+fPk8nzsUUUeB/Xct2ILa5ARSlSpVRERk7969mbbZ74WHh4uIyMmTJ9X79l9s1apVRUTk4YcfloEDB2Z53QYNGvgc1/r162XdunWSlJQkw4cP976/detWn8f5yxgjIsXr/8ADqbjVUdOmTWXBggVy5swZNS/nxx9/FJHitxxBoBSnOjp48KDs27dPpkyZIlOmTMm0vXLlytKvX78s14WBb8WpjgpLoTc5ZcqUyVVX2KBBA6levbosWLBA3drbsWOHfPXVV1KjRg3vvudnpP/www/qY533338/0znr1asn69atk8TExBzHKSKZxnp+HOe3nzd9+vQcv6a8Onz4sHzwwQfSvHlzb8HjHOooawMGDJCZM2fK22+/LUOGDPG+/9prr0mNGjWkTZs2+Tq/a6ijzKKjoyU5OTnT+08++aSsXLlSli1b5v2HFedQR0VHUJuc9evXZ/lZXZ06daRatWoicu7/NFNSUmTp0qVSvXp1iYiIyLLLLFGihEyaNElGjBghAwYMkNtvv12OHDkiCQkJmW71RUdHS9euXWXy5MlSuXJliYmJkRUrVsg777yT6bzTp0+Xa665Rnr06CFxcXFSs2ZN+e2332Tjxo2yZs0aWbx4sYj87/94Z8yYIRERERIeHi61a9eWhg0bSp06deShhx4SY4xERUXJ0qVLZfny5Vl+T0qWLCmxsbGyYsUKn9+7G2+8UWrVqiWtWrWSqlWrypYtW2TKlCmyb9++TL8q6DrqKLPc1tE111wj3bp1kzvvvFN+//13qVu3rixYsEA+/vhjmTdvXrFZI0eEOspKbuooPDzc+yvQGSUlJUlYWFiW21xGHWWW259HIiLLli2TtLQ0OXbsmIicW5Dw/HpLvXr1knLlyuV4jjwJxmxmX7PQRcTMnDnTu+/atWvNlVdeacqVK2dExMTGxhpjsp9JPmvWLFOvXj1TunRpU79+fTN79mwzfPhwNQvdGGP27NljBg0aZKKiokxkZKQZNmyYWb16daZZ6MYYs27dOnP99debCy64wJQqVcpER0ebLl26mGnTpqn9nnvuOVO7dm0TFhamzrNhwwbTrVs3ExERYSpXrmwGDx5sdu7cmeXM8Yxfoy+TJ082zZs3N5GRkSYsLMxUq1bNDBgwwPznP//J8VhXUEf5ryNjjDl27Ji59957TXR0tCldurS57LLLzIIFC3J1rAuoo8DUkY3frqKOzstLHcXExGT7Pdy+fXuuzpEXnv8/wJAWFxcnKSkphfYcF7iBOkIgUEcIBOooMHgKOQAAcBJNDgAAcJITH1cBAADYuJMDAACcRJMDAACcRJMDAACcRJMDAACclOsVj3lWUugqSnPLqaPQRR0hEKgjBEJu64g7OQAAwEk0OQAAwEk0OQAAwEk0OQAAwEk0OQAAwEk0OQAAwEk0OQAAwEk0OQAAwEk0OQAAwEk0OQAAwEk0OQAAwEk0OQAAwEk0OQAAwEk0OQAAwEk0OQAAwEklC3sAgKsuv/xyle+55x6Vb7nlFpXnzp2r8osvvqjymjVrAjg6AHAfd3IAAICTaHIAAICTaHIAAICTPMYYk6sdPZ5gjyWowsLCVI6MjMz1sfZcinLlyqncoEEDle+++26Vn3nmGZWHDh2q8p9//qnyk08+qfJjjz2W67FmJZd/xQUi1OvIl+bNm6v82WefqVyxYsU8ne/o0aMqV6lSxa9xBQp15Iarr75a5fnz56scGxur8ubNmwN6feooNEyYMEFl+9+hEiX0PZJOnTqpvHLlyqCM67zc1hF3cgAAgJNocgAAgJNocgAAgJNCZp2cWrVqqVy6dGmV27dvr3KHDh1UrlSpksrXXXddwMb266+/qvzCCy+oPGDAAJWPHTum8rp161QO9meZCJzWrVt7X7/99ttqmz3vy/4M2a6DU6dOqWzPwWnbtq3K9ro59vHIXseOHVW2v9fvvvtuQQ6nQF1xxRUqf/vtt4U0EhQlcXFxKo8bN07l9PR0n8cXpblWGXEnBwAAOIkmBwAAOIkmBwAAOKnIzsnJac2RvKxzE2j2Z5P2egJ//PGHyvY6FHv27FH58OHDKgd6XQr4z14TqWXLlirPmzfP+7p69ep5OveWLVtUfvrpp1VeuHChyl9++aXKdt1Nnjw5T9cvzuw1PerVq6eyS3Ny7PVMateurXJMTIzKrB1TPNl1EB4eXkgjCSzu5AAAACfR5AAAACfR5AAAACcV2Tk5O3fuVPnQoUMqB3JOzjfffKPykSNHVO7cubPK9nokr7/+esDGgqJl+vTpKtvPHcsPe35PhQoVVLbXS7LnkVx22WUBG0txc8stt6i8atWqQhpJ8NlzxW6//XaVM84rExHZtGlT0MeEwte1a1eVR48e7XN/uy769Omj8r59+wIzsADjTg4AAHASTQ4AAHASTQ4AAHBSkZ2T89tvv6n84IMPqmx/Hvj999+rbD8/yrZ27Vrv627duqltaWlpKv/lL39RecyYMT7PjdB1+eWXq9y7d2+Vfa0hYs+hWbp0qcrPPPOMyrt371bZrmF7/aQuXbrkeizwzV47xmWzZs3yud1erwlusp/nOGfOHJVzmuf6j3/8Q+UdO3YEZmBBVnz+SwcAAMUKTQ4AAHASTQ4AAHBSkZ2TY3vvvfdUtp9ldezYMZWbNWum8l//+leVM86PsOfg2P773/+qPHLkSJ/7I3TYz0hbvny5yhUrVlTZGKPysmXLvK/tNXRiY2NVtp81Zc+VOHDggMrr1q1T2X5mmj1fyF53Z82aNYJz7DWFLrzwwkIaScHLaa6FXfNw0/Dhw1WuUaOGz/1TUlJUnjt3bqCHVCC4kwMAAJxEkwMAAJxEkwMAAJwUMnNybL///rvP7UePHvW5PePzW9588021zZ77AHfUr19fZXv9JXv+wsGDB1Xes2ePyq+99pr39R9//KG2ffjhhz5zfpUtW1blBx54QOWbbropoNcLZb169VLZ/t65xJ5vVLt2bZ/779q1K5jDQSGpWrWqyrfddpvK9r9z9jMbH3/88aCMq6BxJwcAADiJJgcAADiJJgcAADgpZOfk5CQhIUFl+5lEGdcw6dq1q9r26aefBm1cKFhlypRR2X5+lD1Xw15v6ZZbblF59erVKheluR21atUq7CEUWQ0aNPC53V4LK5TZNW7P0fnpp59UtmseoeuSSy7xvn777bfzdOyLL76ocnJyciCGVOi4kwMAAJxEkwMAAJxEkwMAAJzk7Jwc+3lUGdfFEdHP9Zk5c6baZn8Wac/DePnll1W2n2eEoqNFixYq23NwbP369VN55cqVAR8Tip5vv/22sIeQLfv5aT179lR52LBhKnfv3t3n+SZNmqSyvT4KQlfG2rCf12ZbsWKFys8//3xQxlTYuJMDAACcRJMDAACc5OzHVbZt27apHBcX5309Z84cte3mm2/2mcuXL6+y/Qh6e+l/FJ5nn31WZY/Ho7L9cVRR/niqRAn9/yQ8fiRwoqKi8nV8s2bNVLbrzF6m4qKLLlK5dOnS3tf24zjsv/cTJ06o/M0336h88uRJlUuW1D/mv/vuO4Eb+vfvr/KTTz6Z7b5ffPGFysOHD1c5p0chhSru5AAAACfR5AAAACfR5AAAACcVmzk5tnfffdf7esuWLWqbPY/j6quvVjkxMVHlmJgYlZ944gmVd+3a5fc4kTd9+vRRuXnz5irbv+7//vvvB3tIAWPPwbG/lrVr1xbgaEKLPY/F/t5NmzZN5fHjx+fp/Pav69pzcs6cOaPy8ePHVd6wYYP39ezZs9U2ewkLe97Yvn37VP71119Vth89smnTJkFoyvjYBpG8Pbrh559/VtmuG1dxJwcAADiJJgcAADiJJgcAADip2M7JyWj9+vUqX3/99Spfe+21Ktvr6owaNUrlevXqqdytW7f8DhG5ZM8/yLj+iIjI/v37VX7zzTeDPqbcKlOmjMoJCQk+9//ss89UfvjhhwM9JGfcddddKu/YsUPl9u3b5+v8O3fuVPm9995TeePGjSp//fXX+bpeRiNHjlS5WrVqKttzMRC6xo0bp3Je1srytYaOy7iTAwAAnESTAwAAnESTAwAAnMScnCwcOXJE5ddff13lWbNmqWw/G6Zjx44qd+rUSeWUlJR8jQ/+s5/rU5jPGbPn4EyYMEHlBx98UGV7/ZMpU6ao/McffwRwdG576qmnCnsIAWOv42XLy1oqKFrsdb66d++e62OXLFmi8ubNmwMxpJDDnRwAAOAkmhwAAOAkmhwAAOAk5uRI5ufODBo0SOUrrrhCZXsOji3jc2hERD7//PN8jA6BVJjPqrI/X7fn3AwZMkRl+zP16667LijjgtsyPqcPoeXTTz9VuXLlyj73z7j+UlxcXDCGFHK4kwMAAJxEkwMAAJxEkwMAAJxUbObkNGjQQOV77rnH+3rgwIFqW3R0dJ7OffbsWZXttVfy8nwR5I/H4/GZ+/fvr/KYMWOCNpa//e1vKk+cOFHlyMhIlefPn6/yLbfcEpyBAQgJVapUUTmnf0umTp3qfc26WedwJwcAADiJJgcAADiJJgcAADjJmTk59jyaoUOHqpxxDo6IyCWXXOL3tVavXq3yE088oXJhrsVS3BljfGa7Tl544QWVZ8+erfKhQ4dUbtu2rco333yz93WzZs3UtosuukjlnTt3qvzJJ5+onPHzdMBf9jy0+vXrq5xxLRUULXPmzFG5RIm83Yf46quvAjkcJ3AnBwAAOIkmBwAAOIkmBwAAOClk5uRceOGFKjdu3Fjll156SeWGDRv6fa1vvvlG5X/84x8q288UYh2c0BEWFqbyXXfdpbL9fKjff/9d5Xr16uX6Wvbn48nJySo/8sgjuT4XkFv2PLS8zutAwbGfZ9e1a1eV7X9bTp06pfLLL7+s8r59+wI3OEdQ/QAAwEk0OQAAwEk0OQAAwElFZk5OVFSUytOnT1fZ/uzy0ksvzdf1Ms6XmDJlitpmr19y4sSJfF0LBWfVqlUqf/vttypfccUVPo+319Gx54LZMq6js3DhQrUtmM/FAnKrXbt2KiclJRXOQJBJpUqVVM7puYm7du1SOT4+PtBDcg53cgAAgJNocgAAgJNocgAAgJMKbE5OmzZtVH7wwQdVbt26tco1a9bM1/WOHz+usv2MosTERO/rtLS0fF0LRcevv/6q8sCBA1UeNWqUyhMmTMjT+Z9//nmVX3nlFe/rrVu35ulcQDDYz64CijPu5AAAACfR5AAAACfR5AAAACcV2JycAQMG+Mw52bBhg8offPCBymfOnFHZXvvmyJEjeboe3LBnzx6VExISfGYg1CxbtkzlwYMHF9JIkFebNm1S2X7eXYcOHQpyOE7iTg4AAHASTQ4AAHASTQ4AAHCSxxhjcrUjay+ErFz+FRcI6ih0UUcIBOoIgZDbOuJODgAAcBJNDgAAcBJNDgAAcBJNDgAAcBJNDgAAcBJNDgAAcBJNDgAAcBJNDgAAcBJNDgAAcBJNDgAAcBJNDgAAcFKun10FAAAQSriTAwAAnESTAwAAnESTAwAAnESTAwAAnESTAwAAnESTAwAAnESTAwAAnESTAwAAnESTAwAAnESTAwAAnESTAwAAnESTAwAAnESTAwAAnESTAwAAnBSUJicpKUk8Hk+2f1JSUrz7JiYmynvvvZfpHCkpKZn2LUwbNmyQhIQESU1NLbBr/uc//5EePXpIRESEVKhQQTp37ixffvllgV2/sFFH+ffZZ5/JbbfdJg0bNpTy5ctLzZo1pV+/fvLdd98VyPWLAuoo/44dOyZjx46V7t27S7Vq1cTj8UhCQkKBXLuooI7yb+3atdK7d2+pVauWlC1bVqKioqRdu3Yyb968oF2zZNDOLCJz5syRhg0bZnq/cePG3teJiYkyaNAg6d+/v9qnZcuWsmrVKrVvYdqwYYM89thj0qlTJ7nkkkuCfr1vv/1WOnbsKK1bt5bXX39djDHy9NNPy9VXXy3JycnSrl27oI+hqKCO/PfKK6/IoUOHZMyYMdK4cWM5cOCATJkyRdq2bSuffPKJdOnSJehjKCqoI/8dOnRIZsyYIc2aNZP+/fvLrFmzgn7Nooo68t+RI0fk4osvlqFDh0rNmjUlLS1N5s+fLzfffLOkpqbKhAkTAn7NoDY5TZo0kVatWvl1bMWKFaVt27YBHlHomDhxolSqVEk+/vhjKVeunIiIdO3aVS699FKJj48vVnd0qCP/vfzyy3LBBReo93r27Cl169aVxMTEYtXkUEf+i4mJkcOHD4vH45GDBw8W6yaHOvJfp06dpFOnTuq9Pn36yPbt22XGjBlBaXIKdU6Ox+ORtLQ0ee2117y3/M5/A7K7rZeUlCQNGjSQMmXKSKNGjWTu3LkSFxenutDsjk1NTRWPxyNJSUnq/dWrV0vfvn0lKipKwsPDpUWLFrJo0SJ1zcGDB4uISOfOnb1jPX+e5cuXS79+/eSiiy6S8PBwqVu3rowaNUoOHjzo9/fmyy+/lE6dOnkbHBGRiIgI6dixo3z11VeyZ88ev8/tGuooe3aDIyJSoUIFady4sfzyyy9+n9dF1JHv743H4/H7+OKEOsq7qlWrSsmSwbnnEtQ7OWfPnpUzZ86o9zwej4SFhYmIyKpVq6RLly7SuXNnmThxooic63Szk5SUJLfeeqv069dPpkyZIkePHpWEhAQ5efKklCjhX7+WnJwsPXv2lDZt2si0adMkMjJSFi5cKEOGDJHjx49LXFyc9O7dWxITE2X8+PHy8ssvS8uWLUVEpE6dOiIism3bNmnXrp2MGDFCIiMjJTU1VZ599lnp0KGD/Pjjj1KqVCn19cfGxub4meypU6ekTJkymd4//96PP/4o1atX9+trDjXUkf91lJWjR4/KmjVritVdHBHqKNB1VFxRR/mvo/T0dElPT5fDhw/L4sWL5ZNPPpGXXnrJr681RyYI5syZY0Qkyz9hYWFq3/Lly5vhw4dnOkdycrIREZOcnGyMMebs2bOmRo0apmXLliY9Pd27X2pqqilVqpSJiYnJ9tjztm/fbkTEzJkzx/tew4YNTYsWLczp06fVvn369DHVq1c3Z8+eNcYYs3jx4izPaUtPTzenT582O3bsMCJilixZoraHhYWZLl26+DyHMcY0b97c1K9f33t9Y4w5ffq0ufTSS42ImDfeeCPHc4Q66ij/dZSVm266yZQsWdKsXr3ar+NDDXUU2Do6cOCAERHz6KOP5um4UEcdBa6ORo0a5f3elS5d2kydOjXXx+ZVUD+umjt3rnz77bfqzzfffOPXuTZv3iy7d++WG2+8Ud02jYmJkfbt2/t1zq1bt8qmTZvkpptuEhGRM2fOeP/06tVL9uzZI5s3b87xPPv375c77rhDLr74YilZsqSUKlVKYmJiRERk48aNat8zZ87IihUrcjzn6NGj5aeffpJ77rlHdu3aJb/88ovccccdsmPHDhERvzv8UEQd+V9HtokTJ8r8+fPln//8p1x++eV5Pj6UUUeBq6PijDrKfx2NHz9evv32W/nwww/ltttuk3vuuUeeeeaZPHyVuRfUj6saNWrk9wQt26FDh0REJDo6OtO26Ohov34Fbt++fSIiEh8fL/Hx8Vnuk9Pnj+np6dK9e3fZvXu3TJw4UZo2bSrly5eX9PR0adu2rZw4cSLP4xIRue222+TAgQPy+OOPyyuvvCIiIu3atZP4+Hh56qmnpGbNmn6dNxRRR/7XUUaPPfaYPP744/LEE0/IPffck+/zhRrqKDB1VNxRR/mvo1q1akmtWrVERKRXr14iIvLwww/L8OHDpVq1avk6ty2oTU4gValSRURE9u7dm2mb/V54eLiIiJw8eVK9b//FVq1aVUTOfXMHDhyY5XUbNGjgc1zr16+XdevWSVJSkgwfPtz7/tatW30elxvjxo2T++67T7Zs2SIRERESExMjo0aNkvLlyxe7/wsPlOJYRyLnGpyEhARJSEiQ8ePHB+ScxVlxrSMEFnV0TuvWrWXatGny888/B7zJKfTPPMqUKZOrrrBBgwZSvXp1WbBggRhjvO/v2LFDvvrqK7Xv+RnpP/zwg3r//fffz3TOevXqybp166RVq1ZZ/omIiPCOU0QyjfX8LUZ7kvD06dNz/Jpyo0yZMtKkSROJiYmRnTt3yptvvim33367lC1bNiDndwV1lL1JkyZJQkKCTJgwQR599NF8n89l1BECgTrKm+TkZClRooRceumlAT93UO/krF+/PtMsdJFzs7fPd2tNmzaVlJQUWbp0qVSvXl0iIiKy7DJLlCghkyZNkhEjRsiAAQPk9ttvlyNHjkhCQkKmW33R0dHStWtXmTx5slSuXFliYmJkxYoV8s4772Q67/Tp0+Waa66RHj16SFxcnNSsWVN+++032bhxo6xZs0YWL14sIufWRhARmTFjhkREREh4eLjUrl1bGjZsKHXq1JGHHnpIjDESFRUlS5culeXLl2f5PSlZsqTExsbm+Pnl+vXr5e2335ZWrVpJmTJlZN26dfLkk09KvXr1ZNKkST6PdQ11lFlu62jKlCnyyCOPSM+ePaV3797y9ddfq+3Fac0O6iiz3NaRiMiyZcskLS1Njh07JiLnFpJ76623ROTcRw4Zl7twGXWUWW7raOTIkVKxYkVp3bq1XHjhhXLw4EFZvHixvPnmm/Lggw8G/C6OiBT8b1eJiJk5c6Z337Vr15orr7zSlCtXzoiIiY2NNcZkP5N81qxZpl69eqZ06dKmfv36Zvbs2Wb48OFqFroxxuzZs8cMGjTIREVFmcjISDNs2DCzevXqTLPQjTFm3bp15vrrrzcXXHCBKVWqlImOjjZdunQx06ZNU/s999xzpnbt2iYsLEydZ8OGDaZbt24mIiLCVK5c2QwePNjs3Lkzy99AyPg1+rJ582bTsWNHExUVZUqXLm3q1q1rJkyYYP74448cj3UFdZT/OoqNjfX5PSwOqKP815ExxsTExGT7Pdy+fXuuzhHKqKP819Hs2bPNVVddZapWrWpKlixpKlWqZGJjY83rr7+e47H+8vz/AYa0uLg4SUlJKdDnSsE91BECgTpCIFBHgVHoc3IAAACCgSYHAAA4yYmPqwAAAGzcyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE7K9WMdMj4GHqGlKP0CHXUUuqgjBAJ1hEDIbR1xJwcAADiJJgcAADiJJgcAADiJJgcAADiJJgcAADiJJgcAADiJJgcAADiJJgcAADiJJgcAADiJJgcAADiJJgcAADiJJgcAADiJJgcAADiJJgcAADipZGEPAACKm+eff17le++91/t6/fr1alufPn1U3rFjR/AGBjiGOzkAAMBJNDkAAMBJNDkAAMBJzMkBgiQiIkLlChUqqNy7d2+Vq1WrpvKzzz6r8smTJwM4OhSkSy65ROVhw4apnJ6e7n3dqFEjta1hw4YqMyen+Kpfv77KpUqVUrljx47e11OnTlXbMtZYICxZskTlG264QeVTp04F9Hr+4k4OAABwEk0OAABwEk0OAABwEnNygHzIONdi3Lhxalu7du1UbtKkSZ7OXb16dZUzrqWC0HLgwAGVP//8c5X79u1bkMNBEfWXv/xF5bi4OJUHDx6scokS+j5FjRo1vK/tOTjGmACM8H/smp02bZrK9913n8q///57QK+fW9zJAQAATqLJAQAATqLJAQAATio2c3LatGmjcsZ1KmJjY9U2+3NRW3x8vMq7d+9WuUOHDirPmzdP5W+++cb3YFFk2GuU2J8z33TTTd7XZcuWVds8Ho/Kv/zyi8rHjh1T2V4f5frrr1fZXvdi06ZN2YwaRU1aWprKrHWDrEyePFnlXr16FdJI8u6WW25R+dVXX1X5yy+/LMjheHEnBwAAOIkmBwAAOIkmBwAAOMnZOTlDhgxR+fnnn1e5atWq3tf23ImUlBSV7WcK/eMf//B5bft89vH2Mz5QeCIjI1V+6qmnVLbryH4elS9btmxRuUePHirbz52x59hkrNGsMkJHpUqVVG7WrFnhDARF2vLly1XOaU7O/v37Vc44D8ZeQyenZ1e1b99eZXuuaqjiTg4AAHASTQ4AAHASTQ4AAHBSyM7JKVlSD71Vq1Yqz5w5U+Vy5cqpnPHZMZMmTVLbvvjiC5XLlCmj8qJFi1Tu3r27z7GuXr3a53YUngEDBqg8YsQIv8+1bds2lbt166ayvU5O3bp1/b4WQov986dWrVq5PvaKK65Q2Z67xZo77njllVdUfu+993zuf/r0aZX37t3r97UrVqyo8vr161XO+FysrNhjLSr/7nEnBwAAOIkmBwAAOIkmBwAAOClk5+RkfPaUiMisWbN87m+vP5Bx/ZPff//d57H2Wik5zcH59ddfVX7ttdd87o/CM3jw4Dztn5qaqvK3337rfT1u3Di1zZ6DY7OfVQV32c+3S0pKUjkhISHbY+1tR44cUfmll17Kx8hQlJw5c0blnH6GBJK9jlflypXzdLz9797JkyfzPaZA4E4OAABwEk0OAABwEk0OAABwUsjMybHXshk/frzKxhiVp06dqvKECRNUzmkeTkb/93//l+t9RUTuvfdelQ8cOJCn41Fwbr/9dpVHjhyp8qeffqry1q1bVbafHZMXF154od/HIrTZP898zckBgsF+hqL9s7Bs2bJ5Ot8jjzyS7zEFA3dyAACAk2hyAACAk2hyAACAk4rsnBz78z17Ds6pU6dU/uSTT1S21yw5ceJEttcKDw9X2V4Hx37OjMfjUfnxxx9XecmSJdleC0WLvX5JQc6NaNeuXYFdC0VbiRL/+//N9PT0QhwJXHHTTTep/NBDD6lsPzuvVKlSeTr/2rVrVbafo1VUcCcHAAA4iSYHAAA4iSYHAAA4qcjMyalUqZLKd911l8r2Ojj2HJz+/fvn6XoZP4+cP3++2nb55Zf7PPatt95S+emnn87TteGOjGsilS9fPk/HNm3a1Of2r776SuVVq1bl6fwIHRnn4dg/61B8XHLJJSrffPPNKnft2jXX5+rQoYPKea0rey05e07PRx99pLKvea+FiTs5AADASTQ5AADASUXm46rSpUurXLVqVZ/7249OuOCCC1S+9dZbVe7bt6/KTZo08b6uUKGC2mbf1rPzvHnzVE5LS/M5VoSOcuXKqdy4cWOVH330UZV79eqV7bky/lqwSM6/Gmz/Ortdw2fPnvV5PIDQkvHfIRGR999/X2V7+ZKC9O9//1vlGTNmFNJI8oc7OQAAwEk0OQAAwEk0OQAAwElFZk6O/ZiGAwcOqFytWjWVt2/frnJefz0u4/wH+1flqlevrvLBgwdVXrp0aZ6uhaLDXrq8RYsWKr/99tsq27Vg/5pkxjqyf8W7Z8+eKtvzfWwlS+r/HAcOHKjy888/r7L93wyA0GY/MsjOeZHXOYG2Pn36qHzNNdeovGzZMv8GVsC4kwMAAJxEkwMAAJxEkwMAAJxUZObkHDlyRGX7MQ0ffPCBylFRUSpv27ZN5SVLlqiclJSk8m+//eZ9vXDhQrXNnodhb0fosNdfsufJvPPOOz6Pf+yxx1T+7LPPVP7yyy+9r+2atPe118Sw2fPOJk+erPLOnTtVfu+991Q+efKkz/Oj6Mo4fyKnuRMdO3ZU+aWXXgrKmBB869evV7lTp04qDxs2TGX7cUZ//vmn39f+61//qvLo0aP9PldRxp0cAADgJJocAADgJJocAADgJI/J5QIz+fl9/aIo4+faK1euVNvsz8Tvu+8+lV988cWgjSsY8rqGUDAFu47sdXD+/ve/q/zggw/6PN5e++Hmm29W2Z47lnEezUcffaS2tWzZUmV7XZunn35aZXvOTr9+/XyO9V//+pfKTz31lMqHDx/2efzatWt9brcVpzoqaBmfS5bX7/Nll12m8oYNGwIypmChjoqGyMhIlQ8dOuRz/2uvvVblwl4nJ7d1xJ0cAADgJJocAADgJJocAADgpCKzTk5BK1u2rPe1PQfH/qyPdXKKrrCwMJUnTZqkcnx8vMppaWkqP/TQQyrbf9f2HJxWrVqpnHGNEvs5WFu2bFH5zjvvVDk5OVnlihUrqty+fXuVb7rpJpX79u2r8vLly8WXX375ReXatWv73B8FZ9q0ad7Xo0aNytOxI0eOVNmeQwhkpUePHoU9hALBnRwAAOAkmhwAAOAkmhwAAOCkYjsnx34GCEKTPR/BnoNz/Phxle35Dp9++qnKbdu2VfnWW29V+ZprrlE549wue02eOXPmqGzPibH9/vvvKn/88cc+89ChQ1W+8cYbfZ7/b3/7m8/tKDybNm0q7CEgCOx1u7p3766y/Xy7EydOBG0s9s+y559/PmjXKkq4kwMAAJxEkwMAAJxEkwMAAJxUbJ9dlXGNAPuZQ/a3pHr16iofOHAgeAMLApefFbNnzx6VMz5LSkTk5MmTKttzH8qXL69y3bp183T9hIQE7+vJkyerbRmfR+QCl+uoKPnpp59UrlOnjs/9S5TQ/69q1/C2bdsCM7AAcbmOOnTooPL//d//qdytWzeV7bWqcpq3l5OoqCjv6169eqlt9jMXIyIifJ7Lnh9kr8tlr/NV0Hh2FQAAKNZocgAAgJNocgAAgJOK7To5l156aWEPAQGwd+9ele05OWXKlFG5WbNmPs9nz8/6/PPPVX7vvfdUTk1N9b52bQ4OCsd///tflXP6WWU/ew+FJ+Oz7EREmjRp4nP/sWPHqnzs2LF8XT/jnJ+WLVuqbTnNYUlJSVH5lVdeUbmw5+D4izs5AADASTQ5AADASTQ5AADAScV2Ts6///1v72t7nQk+4w4dHTt2VLl///4q259L79+/X+XZs2erfPjwYZVPnTqVzxECeTNjxgyVr7322kIaCYLtzjvvLLBr2T/7li5dqvKYMWNU/vPPP4M+poLAnRwAAOAkmhwAAOAkmhwAAOCkYvvsqozsZ8XY61LYzyP5+uuvgz6mQHL5WTEoONRRwYiJiVH5gw8+ULlRo0Yq29+L+vXrq8yzq7IX6Dpq3ry5yqNHj1Z5+PDhAb2e/Xd7/Phx7+uM805FMs/1Wr9+fUDHUtB4dhUAACjWaHIAAICTaHIAAICTmJMjInFxcSrPmjVL5ZUrV6psf866YcOGoIwrUFz+DBwFhzpCIBSnOrKfnWf/W/P444+rXLlyZZXtZ+UtX75c5SVLlqhsP8vPZczJAQAAxRpNDgAAcBJNDgAAcBJzckSkYsWKKi9atEjlrl27qvzOO++ofOutt6qclpYWwNHlX3H6DBzBQx0hEKgjBAJzcgAAQLFGkwMAAJxEkwMAAJzEnJws2HN0nnjiCZXvvPNOlS+77DKVi9q6OXwGjkCgjhAI1BECgTk5AACgWKPJAQAATqLJAQAATmJOTjHAZ+AIBOoIgUAdIRCYkwMAAIo1mhwAAOAkmhwAAOCkXM/JAQAACCXcyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE4KSpOTlJQkHo8n2z8pKSnefRMTE+W9997LdI6UlJRM+xamDRs2SEJCgqSmphbI9Y4dOyZjx46V7t27S7Vq1cTj8UhCQkKBXLuooI4Cb9asWeLxeKRChQqFcv3CQB3l39q1a6V3795Sq1YtKVu2rERFRUm7du1k3rx5BXL9ooA6yr/zX39Wf77++uugXLNkUM76/82ZM0caNmyY6f3GjRt7XycmJsqgQYOkf//+ap+WLVvKqlWr1L6FacOGDfLYY49Jp06d5JJLLgn69Q4dOiQzZsyQZs2aSf/+/WXWrFlBv2ZRRR0Fxq5duyQ+Pl5q1KghR48eLdBrFwXUkf+OHDkiF198sQwdOlRq1qwpaWlpMn/+fLn55pslNTVVJkyYEPQxFBXUUf4lJiZK586d1XtNmjQJyrWC2uQ0adJEWrVq5dexFStWlLZt2wZ4RKEjJiZGDh8+LB6PRw4ePFismxzqKDDuuOMO6dixo0RFRclbb71V2MMpcNSR/zp16iSdOnVS7/Xp00e2b98uM2bMKFZNDnWUf/Xq1Suw70OhzsnxeDySlpYmr732mveW1fn/kLK7rZeUlCQNGjSQMmXKSKNGjWTu3LkSFxenutDsjk1NTRWPxyNJSUnq/dWrV0vfvn0lKipKwsPDpUWLFrJo0SJ1zcGDB4uISOfOnb1jPX+e5cuXS79+/eSiiy6S8PBwqVu3rowaNUoOHjyYr++Nx+Px+/jihDrK2bx582TlypUyderUfJ/LVdRR3lWtWlVKlgzq/yuHHOqoaAlqdZ49e1bOnDmj3vN4PBIWFiYiIqtWrZIuXbpI586dZeLEiSJyrtPNTlJSktx6663Sr18/mTJlihw9elQSEhLk5MmTUqKEf/1acnKy9OzZU9q0aSPTpk2TyMhIWbhwoQwZMkSOHz8ucXFx0rt3b0lMTJTx48fLyy+/LC1bthQRkTp16oiIyLZt26Rdu3YyYsQIiYyMlNTUVHn22WelQ4cO8uOPP0qpUqXU1x8bG1tkPpMNBdRR/upo//79ct9998mTTz4pF110kV9fnwuoo/z/PEpPT5f09HQ5fPiwLF68WD755BN56aWX/PpaQxV1lP86uvvuu+WGG26QcuXKSbt27WTixInSoUMHv77WHJkgmDNnjhGRLP+EhYWpfcuXL2+GDx+e6RzJyclGRExycrIxxpizZ8+aGjVqmJYtW5r09HTvfqmpqaZUqVImJiYm22PP2759uxERM2fOHO97DRs2NC1atDCnT59W+/bp08dUr17dnD171hhjzOLFi7M8py09Pd2cPn3a7Nixw4iIWbJkidoeFhZmunTp4vMctgMHDhgRMY8++miejgt11FFg6ui6664z7du39369w4cPN+XLl8/VsS6gjgL382jUqFHe713p0qXN1KlTc31sqKOO8l9Ha9asMWPGjDHvvvuu+fzzz83s2bNNo0aNTFhYmPn4449zPN4fQb2TM3fuXGnUqJF6z9+PYDZv3iy7d++W+++/X50jJiZG2rdv79fs8K1bt8qmTZvkmWeeERFR3XmvXr3kgw8+kM2bN2f6Gmz79++XRx55RD788EPZvXu3pKene7dt3LhR+vbt6832/wEgZ9SR/3X09ttvy9KlS+X7778v9h9/Ukf5/3k0fvx4GTFihOzfv1+WLl0q99xzj6SlpUl8fHyezhPKqCP/66hFixbSokULb77qqqtkwIAB0rRpUxk7dqz06NEjV+fJi6A2OY0aNfJ7gpbt0KFDIiISHR2daVt0dLRfxbBv3z4REYmPj8/2P9KcPn9MT0+X7t27y+7du2XixInStGlTKV++vKSnp0vbtm3lxIkTeR4XNOrIvzr6448/5O6775bRo0dLjRo15MiRIyIicurUKRE59xszpUqVkvLly+f53KGIOsr/z6NatWpJrVq1ROTcP5giIg8//LAMHz5cqlWrlq9zhwrqKLD/rlWqVEn69Okj06ZNkxMnTkjZsmUDdm6RIDc5gVSlShUREdm7d2+mbfZ74eHhIiJy8uRJ9b79F1u1alUROfcf6cCBA7O8boMGDXyOa/369bJu3TpJSkqS4cOHe9/funWrz+NQOIpTHR08eFD27dsnU6ZMkSlTpmTaXrlyZenXr1+W63nAt+JUR760bt1apk2bJj///HOxaXICiTo6xxgjIv7fEfOl0JucMmXK5KorbNCggVSvXl0WLFigbu3t2LFDvvrqK6lRo4Z33/Mz0n/44Qd1++v999/PdM569erJunXrJDExMcdxikimsZ4fx/nt502fPj3HrwmBQx1lFh0dLcnJyZnef/LJJ2XlypWybNky7w9EnEMd5U1ycrKUKFFCLr300qCcP1RRR7l3+PBh+eCDD6R58+beRi6QgtrkrF+/PsvP6urUqePt+ps2bSopKSmydOlSqV69ukRERGTZZZYoUUImTZokI0aMkAEDBsjtt98uR44ckYSEhEy3+qKjo6Vr164yefJkqVy5ssTExMiKFSvknXfeyXTe6dOnyzXXXCM9evSQuLg4qVmzpvz222+yceNGWbNmjSxevFhE/rdQ0YwZMyQiIkLCw8Oldu3a0rBhQ6lTp4489NBDYoyRqKgoWbp0qSxfvjzL70nJkiUlNjZWVqxYkeP3b9myZZKWlibHjh0TkXMLN51f36RXr15Srly5HM/hAuoos9zUUXh4eKa1TUTO/TZHWFhYlttcRh1lltufRyNHjpSKFStK69at5cILL5SDBw/K4sWL5c0335QHH3ywWN3FoY4yy20d3XjjjVKrVi1p1aqVVK1aVbZs2SJTpkyRffv2ZfoV+IAJxmxmX7PQRcTMnDnTu+/atWvNlVdeacqVK2dExMTGxhpjsp9JPmvWLFOvXj1TunRpU79+fTN79mwzfPhwNQvdGGP27NljBg0aZKKiokxkZKQZNmyYWb16daZZ6MYYs27dOnP99debCy64wJQqVcpER0ebLl26mGnTpqn9nnvuOVO7dm0TFhamzrNhwwbTrVs3ExERYSpXrmwGDx5sdu7cmeVvRGX8GnMSExOT7fdw+/btuTpHKKOOAlNHNn67ijo6L7d1NHv2bHPVVVeZqlWrmpIlS5pKlSqZ2NhY8/rrr+d4rCuoo/zX0eTJk03z5s1NZGSkCQsLM9WqVTMDBgww//nPf3I81l+e/z/AkBYXFycpKSmF9jwguIE6QiBQRwgE6igweAo5AABwEk0OAABwkhMfVwEAANi4kwMAAJxEkwMAAJxEkwMAAJxEkwMAAJyU6xWPi/sTjENZUZpbTh2FLuoIgUAdIRByW0fcyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE4qWdgDABAYK1asUNnj8ajcpUuXghwOMmjcuLHKffr0UXnkyJHe199++63a9v333/s893PPPafyqVOn/Bgh4Cbu5AAAACfR5AAAACfR5AAAACcVmzk5pUqVUrl9+/be14mJiWrblVdeWSBjAvLjn//8p8oZa1pEZO7cuQU5HGQwatQolZ955hmVK1SokO2xderUUfmGG27weS17Dk9ycnJuhggUC9zJAQAATqLJAQAATqLJAQAATvIYY0yudrTW3Ag1VatWVXn//v3e13v37lXbWrZsqbK9PdTk8q+4QIR6HRWmJ598UuUxY8aofPr0aZVHjBih8qJFi/J1feoo96KiolTeuHGjyhdccEHArnXkyBGVhwwZovKnn34asGsFAnWEQMhtHXEnBwAAOIkmBwAAOIkmBwAAOKnYrJPjS3R0tM8c6nNy4Ia2bduqbK/99MUXX6ic3zk48N9vv/2m8qOPPqrylClTVC5Xrpz39c6dO9W2WrVq+bxWpUqVVO7Zs6fKRW1ODtwQExOjctmyZVUeOnSoynfeeafP83344Ycq33rrrfkY3f9wJwcAADiJJgcAADiJJgcAADiJOTnCWgnIvY4dO6r8f//3f97X9mfQ9ryMvLLP16RJE5W3bdumcnx8fL6uh+CZNm2aynfccYfKzZo1877+/fff83Wtl156KV/HAyIiXbt2VXngwIEq2z+fIiMjVc7rekj2nMNA4U4OAABwEk0OAABwEk0OAABwEnNyJPNnh+Hh4YU0EhR1M2bMULlevXre140bN1bb7HVr8mr8+PEqV6lSReXbb79d5XXr1uXreig4jz/+uMoZ53Y1b948X+cuXbp0vo5H8TBr1iyVmzZtqvIVV1yRp/MdO3ZM5fnz56v87bffqrxgwQKV//zzzzxdL7e4kwMAAJxEkwMAAJxEkwMAAJzEnJwstGrVSuWvv/66kEaCoub48eMqZ5zPld+5XPZcDPvZMOnp6Sozdyx0vfXWWypnnL9lP2vKniuRE3u+z6BBg/I4OrjAnsM3efJklW+77TaV7XW9vvvuO5WffPJJldevX6/yiRMnVLafwVZYuJMDAACcRJMDAACcRJMDAACcVGzm5Jw5c0blo0ePel/bz9yoU6dOgYwJRd+kSZNUtudHbNy40fs6r+vUlC9fXuVx48apXK5cOZXtuWH2vA6EjptuuknljM+usp9Rllf5XZ8Jbpg4caLKf/3rX1V+8cUXVc64VpOIyB9//BGcgRUw7uQAAAAn0eQAAAAn0eQAAAAnFZs5OUeOHFH53//+t/d1nz59Cng0KKouvvhile3nQ9lzu+655x7v6wMHDuTpWs8++6zKgwcPVnn37t0qX3nllXk6PwpPw4YNVX733XdVrlu3rsolSwbuR/H7778fsHOhaLHn6dnz+G6++Wbv6/vuu09tS05OVvmTTz5ROVjPjips3MkBAABOoskBAABOoskBAABOKjZzcoCs2GuS2HMnqlatqrK9tsTKlStzfa34+HiV4+LifO7/xBNP5PrcKFoaNWqkcu3atVUO5Bwc29/+9jeVR48eHbRroWBNmDBBZXtOzqJFi7yv7WeguTrnJifcyQEAAE6iyQEAAE7i46os2I+oR+iyPxYYNmyYyq+++qrKJUrovj89PV3ldu3aqfzwww97X9u/Eh4VFaWy/SviHo9H5blz56o8ffp0QWiyP/YcO3asyk899ZTK4eHhAbt29erVA3YuFC0Zf96IiBhjVF6wYIH3dXH9eMrGnRwAAOAkmhwAAOAkmhwAAOAk5uRkoW/fvoU9BATIDTfcoPKsWbNUtj/TtufgbN26VeVWrVplm/v166e21axZU2V7roT9GIjbbrtN4KYXXnhB5S1btqhcqVKlbI+155W99NJLKlesWDF/g0PI+M9//qOy/fMoY22cOHFCbVu+fHnwBlaEcScHAAA4iSYHAAA4iSYHAAA4yWPsSQnZ7Wit6RHqMi59PmXKFLXt999/V9nX5+WhIJd/xQUi2HU0ZMgQlefNm6fymTNnVD5y5IjKN954o8qHDx9W2a6V2NjYbMdif63234Od9+7dq3KnTp1U3rZtW7bXKgjFqY6KEvtrTUhIUPmRRx5R2a6Tq6++WuUdO3YEbnB+oI7+p02bNip///33Kp86dUple+2te++9V+WJEyd6X//xxx8+r7Vp06a8DbaIyW0dcScHAAA4iSYHAAA4iSYHAAA4qdiuk7Nz585st5UqVUrlmJgYlQv7M21kb9SoUSrbf8+PP/64ynPmzMnT+UePHq1yxudL2c+1yok9HyA5OVnlwp6Dg6KhdOnSKttzcGynT59W+ezZswEfE3LHXhvrgw8+ULlWrVoqZ5wrKpJ5TuFvv/2msr1mUsY5ORUqVFDb7Pk8xQV3cgAAgJNocgAAgJNocgAAgJOK7Zwce72UjOy5EmXKlAn2cBAgS5YsUfmdd95R+ZdffsnX+atWrapykyZNst136NChKq9fv97nuX/99Vf/BwZn2fPIcvLqq6+qTF0VnjVr1qhsP2ds3LhxKttzcHIyZsyYbLf961//Ujmnnz+u4k4OAABwEk0OAABwEk0OAABwUrF9dlVGGzZsULlhw4YqT5s2TeW77ror6GMKJJ4V47/IyEiV7fkRGWvBXtemfv36wRtYIShOdVSlShWV7fWUFixY4DPnh722iv2MIXteh61OnToq//zzz4EZWIAUpzp6+OGHVZ4wYYLKZcuWzdP5tmzZonK9evVUzriG23XXXae22fODQh3PrgIAAMUaTQ4AAHASTQ4AAHBSsV0nJ6NPP/1U5Zo1a6p8//33F+RwUITY86/uvPNOlffv3+993aVLlwIZE4LvhRdeUPnaa69V2Z5vtXv3bpV37dql8tatW1W+/PLLsz3f2LFj1bac5uBMmTLF51hQeCZPnqyy/VyxFi1aqNy1a1ef56tcubLKH374ocrx8fHe13bNFVfcyQEAAE6iyQEAAE6iyQEAAE5iTk4W7N+/P3XqVCGNBAUtJiZG5REjRqhs18aMGTO8r3lGkDtefPFFlWvXrq1yu3btVE5JSVE5NTVVZXstrquuukrliIiIbMdi15y9bs6jjz6q8p9//pntuVC4nnnmmcIeQrHDnRwAAOAkmhwAAOAkmhwAAOAk5uRkwV6Xol+/fiq/++67BTkcFKDly5erbM/RmTdvnsr2fAi44euvv1Z51apVKr/++usqT506VeVLLrnEZ86Lw4cPq9y4cWO/zwUUN9zJAQAATqLJAQAATqLJAQAATmJOjohcf/31Kp88eVLljRs3FuRwUIjmzJmj8qRJk1ResmRJQQ4HRcQDDzygcpkyZVSuUKGCz+PtZxQNHTo0232PHj2qcrdu3XIzRABZ4E4OAABwEk0OAABwEk0OAABwksfYD0bJbkePJ9hjKTQLFy5UuVGjRir37dtX5R07dgR9TIGUy7/iAuFyHbmOOkIgUEcIhNzWEXdyAACAk2hyAACAk2hyAACAk5iTUwzwGTgCgTpCIFBHCATm5AAAgGKNJgcAADiJJgcAADiJJgcAADiJJgcAADiJJgcAADiJJgcAADiJJgcAADiJJgcAADiJJgcAADiJJgcAADgp18+uAgAACCXcyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE6iyQEAAE4KSpOTlJQkHo8n2z8pKSnefRMTE+W9997LdI6UlJRM+xamDRs2SEJCgqSmphbI9dauXSu9e/eWWrVqSdmyZSUqKkratWsn8+bNK5DrFwXUUeB88cUX0qtXL6lcubKULVtW6tWrJ5MmTSrQMRQW6ij/jh07JmPHjpXu3btLtWrVxOPxSEJCQoFcu6igjvIvLi7O5/fw66+/Dvg1Swb8jBnMmTNHGjZsmOn9xo0be18nJibKoEGDpH///mqfli1byqpVq9S+hWnDhg3y2GOPSadOneSSSy4J+vWOHDkiF198sQwdOlRq1qwpaWlpMn/+fLn55pslNTVVJkyYEPQxFBXUUf688cYbcvPNN8v1118vc+fOlQoVKsi2bdtk9+7dBXL9ooI68t+hQ4dkxowZ0qxZM+nfv7/MmjUr6Ncsqqgj/02cOFHuuOOOTO9fe+21UqZMGbniiisCfs2gNjlNmjSRVq1a+XVsxYoVpW3btgEeUejo1KmTdOrUSb3Xp08f2b59u8yYMaNYNTnUkf927dolI0eOlFGjRsnUqVO973fu3LkQR1U4qCP/xcTEyOHDh8Xj8cjBgweLdZNDHfmvTp06UqdOHfXeypUr5eDBgzJhwgQJCwsL+DULdU6Ox+ORtLQ0ee2117y3q87/w57dbb2kpCRp0KCBlClTRho1aiRz586VuLg41YVmd2xqaqp4PB5JSkpS769evVr69u0rUVFREh4eLi1atJBFixapaw4ePFhEzv3jcH6s58+zfPly6devn1x00UUSHh4udevWlVGjRsnBgwcD8W1SqlatKiVLBrU3DTnUUfZmzZolaWlpMm7cOL/PUVxQR76/Nx6Px+/jixPqKG9effVV8Xg8cttttwX0vOcF9V/Ls2fPypkzZ9R7Ho/H262tWrVKunTpIp07d5aJEyeKyLlONztJSUly6623Sr9+/WTKlCly9OhRSUhIkJMnT0qJEv71a8nJydKzZ09p06aNTJs2TSIjI2XhwoUyZMgQOX78uMTFxUnv3r0lMTFRxo8fLy+//LK0bNlSRMTbkW7btk3atWsnI0aMkMjISElNTZVnn31WOnToID/++KOUKlVKff2xsbG5/kw2PT1d0tPT5fDhw7J48WL55JNP5KWXXvLraw1V1JH/dfT5559LVFSUbNq0Sfr16yfr16+XqKgoGThwoDz99NM+v0+uoY7y//MI1FEg6+jo0aPy1ltvydVXXy21a9f262vNkQmCOXPmGBHJ8k9YWJjat3z58mb48OGZzpGcnGxExCQnJxtjjDl79qypUaOGadmypUlPT/ful5qaakqVKmViYmKyPfa87du3GxExc+bM8b7XsGFD06JFC3P69Gm1b58+fUz16tXN2bNnjTHGLF68OMtz2tLT083p06fNjh07jIiYJUuWqO1hYWGmS5cuPs+R0ahRo7zfu9KlS5upU6fm+thQRx3lv44aNGhgwsPDTUREhElMTDTJycnm6aefNmXLljVXXnml+h64ijoK3M8jY4w5cOCAERHz6KOP5um4UEcdBbaOjDHmlVdeMSJiFixYkOdjcyuod3Lmzp0rjRo1Uu/5e8tz8+bNsnv3brn//vvVOWJiYqR9+/Z+zQ7funWrbNq0SZ555hkREdWd9+rVSz744APZvHlzpq/Btn//fnnkkUfkww8/lN27d0t6erp328aNG6Vv377ebP8fQE7Gjx8vI0aMkP3798vSpUvlnnvukbS0NImPj8/TeUIZdeR/HaWnp8uff/4pjz76qDz00EMicm6+V+nSpeW+++6TFStWSNeuXXP9tYYy6ij/P49AHYkEro5effVVqVKligwYMMCv43MjqE1Oo0aN/J6gZTt06JCIiERHR2faFh0d7Vcx7Nu3T0RE4uPjs20acvr8MT09Xbp37y67d++WiRMnStOmTaV8+fKSnp4ubdu2lRMnTuR5XBnVqlVLatWqJSLnClRE5OGHH5bhw4dLtWrV8nXuUEEd+V9HVapUkS1btkiPHj3U+9dcc43cd999smbNmmLT5FBH+f95BOooUHX0ww8/yOrVq2XMmDFSpkyZfJ8vOyEzg7VKlSoiIrJ3795M2+z3wsPDRUTk5MmT6n37L7Zq1aoicq5pGDhwYJbXbdCggc9xrV+/XtatWydJSUkyfPhw7/tbt271eZy/WrduLdOmTZOff/652DQ5gVTc6uiyyy7Lcu0JY4yIiN+f+Rd3xa2OEBzFuY5effVVEREZMWJEwM6ZlUL/CVemTJlcdYUNGjSQ6tWry4IFC7w/oEVEduzYIV999ZXa9/yM9B9++EG9//7772c6Z7169WTdunXSqlWrLP9ERER4xykimcZ6/haj3YlOnz49x6/JH8nJyVKiRAm59NJLg3L+UEUdZe26664TEZFly5ap9z/66CMRkWL966xZoY4QCNSRbydPnpR58+ZJ69atpUmTJgE5Z3aCeidn/fr1WX5WV6dOHe9diKZNm0pKSoosXbpUqlevLhEREVl2mSVKlJBJkybJiBEjZMCAAXL77bfLkSNHJCEhIdOtvujoaOnatatMnjxZKleuLDExMbJixQp55513Mp13+vTpcs0110iPHj0kLi5OatasKb/99pts3LhR1qxZI4sXLxYR8f5FzJgxQyIiIiQ8PFxq164tDRs2lDp16shDDz0kxhiJioqSpUuXyvLly7P8npQsWVJiY2NlxYoVPr93I0eOlIoVK0rr1q3lwgsvlIMHD8rixYvlzTfflAcffLBY3cWhjjLLbR11795drr32Wvn73//uvdW8evVqeeyxx6RPnz7SoUMHn8e7hDrKLLd1JHKuUU5LS5Njx46JyLmF5N566y0ROfdRerly5XI8hwuoo8zyUkciIu+995789ttvQb+LIyIF/9tVImJmzpzp3Xft2rXmyiuvNOXKlTMiYmJjY40x2c8knzVrlqlXr54pXbq0qV+/vpk9e7YZPny4moVujDF79uwxgwYNMlFRUSYyMtIMGzbMrF69OtMsdGOMWbdunbn++uvNBRdcYEqVKmWio6NNly5dzLRp09R+zz33nKldu7YJCwtT59mwYYPp1q2biYiIMJUrVzaDBw82O3fuzPI3EDJ+jb7Mnj3bXHXVVaZq1aqmZMmSplKlSiY2Nta8/vrrOR7rCuoo/3VkjDHHjx8348aNMxdffLEpWbKkqVWrlnn44YfNn3/+mavjQx11FJg6iomJyfZ7uH379lydI5RRR4GpI2OM6datmylfvrz5/fffc32MvzzGZLhHFqLi4uIkJSWlwJ8HBLdQRwgE6giBQB0FRqHPyQEAAAgGmhwAAOAkJz6uAgAAsHEnBwAAOIkmBwAAOIkmBwAAOIkmBwAAOCnXKx77+5RVFL6iNLecOgpd1BECgTpCIOS2jriTAwAAnESTAwAAnESTAwAAnESTAwAAnESTAwAAnESTAwAAnESTAwAAnJTrdXIA5E/9+vVV/vjjj1UOCwtTOSYmJuhjAgCXcScHAAA4iSYHAAA4iSYHAAA4iTk5QJC8+OKLKg8ZMkTlqKgolT/44IOgjwkAihPu5AAAACfR5AAAACfR5AAAACd5jDEmVzt6PMEeC4Ikl3/FBcK1Orrwwgu9r9955x21rW3btirbfw/r169X+eqrr1b50KFDgRhiwFBHCATqCIGQ2zriTg4AAHASTQ4AAHASTQ4AAHBS0NbJqVChgsr2GiF//vmnypdffrnKERERKt90000qp6SkqLxr1y5/hikiInv37lV5yZIlKq9evdrvc8Mt9vOnnnnmGe/rNm3a+Dz24YcfVtmuq6I2BweBY8/9WLBggcq9evXyvm7cuLHa9uuvvwZvYIDjuJMDAACcRJMDAACcRJMDAACcFLR1cp5++mmV4+Pj83R8YUpPT1d5w4YNKtufp9s5NTU1KOPyF+tSBI699s0XX3yR7b721zps2DCV7bop6qgj/5UrV07lzZs3q1yzZk3v65EjR6pts2bNCt7ACgF1hEBgnRwAAFCs0eQAAAAn0eQAAAAnBW2dnIEDB+breHvNkB9++CFf57M/A2/QoIH3daVKldS2Fi1aqNykSROVn3jiCZ9jK2pzcuA/e12cN954Q2Vfn+nb/w3Y6y+h+Dh+/LjKW7ZsUTnjnJxq1aoVyJhQvDzwwAMqly5dWuVGjRqpbK9NZ9u0aZPKf/nLX/IxuuDhTg4AAHASTQ4AAHASTQ4AAHBS0Obk9OjRQ2V7bsNPP/3k83j7M+w9e/YEZmBZsJ+T9eOPP6pcq1Ytn8f37dtX5Q8//DAwA0Ohu/nmm1W2a+Gjjz7yvr7jjjvUtvw8Tw1ue/nll1Xu1KmT97U9NwLITmxsrMoZ54/a2wYMGKByTmsE5bQOTb169VS215Ozn8FWWLiTAwAAnESTAwAAnESTAwAAnBS0Z1eFkqFDh6o8f/58n/ufPHlS5auuukrl1atXB2ZgAcKzYnLvq6++Url58+Yq7969W+WePXt6X2/dujVo4yoKqKPAufjii1XesWOH9/WpU6fUttq1a6sczPmJBYE6+p/q1aurbD/P7tJLL/V5fGRkpMrly5f3vra/tu+++07lli1b5nqcuWHPQYyJiQno+W08uwoAABRrNDkAAMBJNDkAAMBJQVsnp6ixn9PxwgsveF/fcssteTpXu3btVF67dq3f40Lh6tevn8pt2rRR2f7cd/HixSr/+eefwRkYipWM8yfsn1X2OlzTp08vkDEh8Lp27aryzJkzVbbnauWHvU7NwYMHVa5atarKNWrUUHnOnDkqX3TRRT6vZ6+TU1RwJwcAADiJJgcAADiJJgcAADjJ2Tk5nTt3Vtl+BlFcXFy2x54+fVrle++9V+VNmzblb3AoNJUqVVLZXuMoJ4cPH1b5119/9XssY8aMUTmnz+Pj4+P9vhaKNl9rfthzdBC6xo4dq3Je5+DYa7SNGzdO5a+//tr7evPmzT7PdejQIZXtn0c5zcFJTU1V2f43tqjgTg4AAHASTQ4AAHASTQ4AAHCSM3NyWrdurfKnn36qclhYWK7PZX8+vnPnTpXPnj2bx9GhqLD/7i6//HKVS5TQfX96errKn3/+ea6v9be//c3n9tGjR6uc07NeHnjgAZXtz8ztZ8cAKFzdu3dXuW3btnk63v63x5738uWXX/o3sCzkNAfHtmTJEpXtdXiKCu7kAAAAJ9HkAAAAJ9HkAAAAJzkzJ+f6669XOS9zcGz2uhQffvihyqtXr1Z56dKlKr/77rsqr1+/3u+xILBiY2NVttfJsefg2J+J+/rcuXnz5j7PbT+DyJaWlqayvQZPgwYNVH7rrbdUvuGGG1TesWOHz+sBCC57Hl25cuV87v/VV1+p/Nhjj6mcnzk4lStXVrlnz54qd+zYMU9j++ijj/weS0HiTg4AAHASTQ4AAHCSMx9XvfPOOyo3atRI5SuuuEJl+zHzedGqVSuf+dFHH1X5ueeeU/npp59Wef/+/X6PBb5FRESoXLt2bZ/77969W+XXX39d5a1bt6pcv3597+sHH3xQbevXr5/K9kdd9jIHU6ZMUTkyMlLlzz77zOd2hC6Px+N97esRDwgtM2bMUNn+d+fo0aMq33jjjSrv3bs3YGO54447VJ40aZLP/f/73/+qbE8JCeTYgok7OQAAwEk0OQAAwEk0OQAAwEnOzMmxf72td+/eKteqVUvljJ+NXnjhhWrbwIEDVb7ttttUzvj5eVbsRwPcf//9KtuPErj66qtVtn+NGf7r0KGDyv/85z997j9z5kyV//73v6ts18ozzzzjfd2rVy+17dixYyovWrRI5fj4eJXr1aun8rRp03yeb8WKFSrzK+Ohi3k4bnr77bd95mC69tprVX7kkUd87n/mzBmV7Z8/oTIHx8adHAAA4CSaHAAA4CSaHAAA4CSPyeWHwTnNQ3HZTTfdpPLo0aNVbt26db7O/9BDD6lsr6OTX0Xp8/6CrqNx48ap/MQTT/jcv2RJ39PU7GXV27Rpk+2+9lyrlStXqty2bVuVv/jiC5/Xttdbsuf0BFtxrqNAu/jii1X2NZ+qc+fOKtt1FGqoo4Jx9uxZlXP6vt91110q22v8FDW5rSPu5AAAACfR5AAAACfR5AAAACc5s05OMM2fP1/lN998U+V//etfKuf0yHpb3bp1/RsYclSpUiWV7c/glyxZ4vP45s2bq3zJJZdke74HHnhAbbPnTmR8zpWIyBtvvOFzbPb57Dk5KB62bdtW2ENAiEhMTPS+ttdry2n9tVCf65Ud7uQAAAAn0eQAAAAn0eQAAAAnMSfHD/YzPr777juV8zon56effsr3mJA79toKeV2zw/5cO+Pxl112mdq2c+dOlcPDw1Xevn27yldddZXKR48ezdPYABQvpUuXVrlFixbe175+VomIjBkzRuUtW7YEeHRFA3dyAACAk2hyAACAk2hyAACAk0JmTk716tVVvv3221XetGmTyosWLQraWMLCwlRu1qxZno635/R8/fXX+R4Tsmavg/Pggw+q3K9fP5Xt50nZ6+RERERke61bbrlFZXvdm4MHD6qckJCg8q5du7I9N4qvMmXKFPYQUESUK1dO5WHDhqncrVu3bI9dsGCByvb6bzmtoxOquJMDAACcRJMDAACcRJMDAACcVGTn5ERHR6v88ccfq9y0aVOVK1euHLSxXHjhhSrff//9Knfp0iVP59u4caPKX3zxhX8DQ45Onz6t8vHjx1W2P+P+8ssvVc7rOjoZHTt2TGV7ntiyZcv8PjeKj169eqn84osvFtJIUNDsOYAzZ85UedCgQdke+7e//U3ll156SWVX5+DYuJMDAACcRJMDAACcRJMDAACcVGTn5Dz33HMq23NwbLVr11Z58+bNKp84cSLbY8uWLavy2LFjVbbn4PhaK0Uk8/oo9tyMe++91+fxCBz7uWJDhw5V2f677dSpU57O/9prr3lf//jjj2rb999/r/LKlSvzdG64a9++fSr/97//9b7+y1/+UtDDQRFVs2ZNlX3NwRER2bZtm/f1Cy+8EJQxhRru5AAAACfR5AAAACfR5AAAACcV2Tk5K1asUPn666/3uf+aNWtUtudDHD16NNtjIyMjVW7RokVuhpgtew7OgAEDVGZuRuH58MMPfWagIJw6dUrlP//8M9t97ecRsU6Ouxo2bKjyAw884HP/n376SeVrrrkm4GMKddzJAQAATqLJAQAATqLJAQAATiqyc3KWL1+u8sKFC1W+4YYbfB6f33k1vpw5c0Zle02ft99+W+VvvvkmaGMBEPrWrl3rfX355ZerbRUqVCjg0aCwTJw4UeUhQ4b43N+en7Vjx46AjynUcScHAAA4iSYHAAA4iSYHAAA4qcjOyUlNTVX51ltvVfn9999XuUuXLirb6wf07ds322tt2rTJ51g+++wzn/tn/DwdAPLqiSee8L5u0qSJ2rZo0aKCHg4KiP2csooVK/rcf8aMGSrb/zYhM+7kAAAAJ9HkAAAAJ9HkAAAAJ3mMMSZXO3o8wR4LgiSXf8UFgjoKXdQRAoE6+p+nnnpKZftZVfa6N7169VJ58+bNwRlYCMhtHXEnBwAAOIkmBwAAOIkmBwAAOIk5OcUAn4EjEKgjBAJ19D9XX321yp988onK1113ncpLliwJ+phCBXNyAABAsUaTAwAAnESTAwAAnMScnGKAz8ARCNQRAoE6QiAwJwcAABRrNDkAAMBJNDkAAMBJuZ6TAwAAEEq4kwMAAJxEkwMAAJxEkwMAAJxEkwMAAJxEkwMAAJxEkwMAAJxEkwMAAJxEkwMAAJxEkwMAAJz0/wDl6/25XOSb3AAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 600x600 with 16 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import torch\n",
    "import torchvision\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "# Descargar el conjunto de datos MNIST\n",
    "transform = torchvision.transforms.ToTensor()\n",
    "mnist_dataset = torchvision.datasets.MNIST(root='./data', train=True, download=True, transform=transform)\n",
    "\n",
    "# Crear una figura y un eje\n",
    "fig, ax = plt.subplots(4, 4, figsize=(6, 6))\n",
    "\n",
    "# Recorrer las primeras 16 imágenes del conjunto de datos\n",
    "for i in range(16):\n",
    "    image, label = mnist_dataset[i]\n",
    "\n",
    "    # Obtener las coordenadas del subeje actual\n",
    "    row = i // 4\n",
    "    col = i % 4\n",
    "\n",
    "    # Mostrar la imagen en el subeje correspondiente\n",
    "    ax[row, col].imshow(image.squeeze(), cmap='gray')# image.squeeze() elimina dimnsiones de tamaño 1.\n",
    "    ax[row, col].set_title(f'Etiqueta: {label}')\n",
    "    ax[row, col].axis('off')\n",
    "\n",
    "# Ajustar el espaciado entre subejes y mostrar la figura\n",
    "plt.tight_layout()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d074cefa-47c8-4499-acff-010c49b7c7c3",
   "metadata": {},
   "source": [
    "## <span style=\"color:blue\">Instalar Pytorch-lightning</span> "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88284dee-f27d-44d2-b1c4-8fecab39250d",
   "metadata": {},
   "source": [
    "Cree un ambiente nuevo. En consola ejecute los siguientews comando en orden. No olvide instalar Jupyter lab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "956bcf99-2651-4c25-ba8c-07023cbfddd3",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "#!conda install -c pytorch pytorch torchvision torchtext torchaudio \n",
    "#!conda install -c anaconda numpy pandas\n",
    "#!conda install -c conda-forge matplotlib \n",
    "\n",
    "#!conda install -c conda-forge pytorch-lightning\n",
    "#!conda install -c conda-forge jupyterlab"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ca3b9ad-e253-4a2f-97bf-c91b15a230fa",
   "metadata": {},
   "source": [
    "## <span style=\"color:blue\">Ejemplo de un módulo Lightning</span> "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "917ae178-4dcf-4dd4-897f-4ff332a380b8",
   "metadata": {},
   "source": [
    "Como ejemplo vamos a implementar el auto-encoder para los datos de MNIST, usando Lightning."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8efb1368-831a-470d-b5a9-727baf7e0252",
   "metadata": {},
   "source": [
    "### <span style=\"color:#4CC9F0\">Módulo Ligthning: Imports</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "a31ade0b-31e1-45bd-84b5-d43a4d12f27e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Versión instalada de Pytorch-lightning: 2.0.2\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "from torch import nn\n",
    "from torch.nn import functional as F\n",
    "from torch.utils.data import DataLoader\n",
    "from torch.utils.data import random_split\n",
    "from torchvision.datasets import MNIST\n",
    "from torchvision import transforms\n",
    "import pytorch_lightning as pl\n",
    "# Carga WandBLogger para hacer seguimiento (tracking) del entrenamiento\n",
    "from pytorch_lightning.loggers import WandbLogger\n",
    "print('Versión instalada de Pytorch-lightning:', pl.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12ef608b-d54d-4067-a813-af955f404d1d",
   "metadata": {},
   "source": [
    "### <span style=\"color:#4CC9F0\">Clase personalizada de ligthning</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "62cc989d-0049-4b8c-8d32-b53aadff7aee",
   "metadata": {},
   "outputs": [],
   "source": [
    "class LitAutoEncoder(pl.LightningModule):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        \n",
    "        self.encoder = nn.Sequential(\n",
    "            nn.Linear(28*28, 64),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(64, 3))\n",
    "        \n",
    "        self.decoder = nn.Sequential(\n",
    "            nn.Linear(3,64),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(64, 28*28))\n",
    "        \n",
    "    def forward(self, x):\n",
    "        embedding = self.encoder(x)\n",
    "        return embedding\n",
    "    \n",
    "    def configure_optimizers(self):\n",
    "        optimizer = torch.optim.Adam(self.parameters(), lr=1e-3)\n",
    "        return optimizer\n",
    "    \n",
    "    def training_step(self, train_batch, batch_idx):\n",
    "        x, y = train_batch\n",
    "        x = x.view(x.size(0), -1)\n",
    "        z = self.encoder(x)\n",
    "        x_hat = self.decoder(z)\n",
    "        loss = F.mse_loss(x_hat, x)\n",
    "        self.log('train_loss', loss)\n",
    "        return loss\n",
    "    \n",
    "    def validation_step(self, val_batch, batch_idx):\n",
    "        x, y = val_batch\n",
    "        x = x.view(x.size(0), -1)\n",
    "        z = self.encoder(x)\n",
    "        x_hat = self.decoder(z)\n",
    "        loss = F.mse_loss(x_hat,x)\n",
    "        self.log('val_loss', loss)\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1aa38864-8752-450a-96ca-d3d8d0461df0",
   "metadata": {},
   "source": [
    "### <span style=\"color:#4CC9F0\">Explicaciones</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf926910-3bdf-4f5c-9eec-e3ff92661df4",
   "metadata": {},
   "source": [
    "+ La clase `pl.LightningModule` es la clase base de Ligthning. Contiene toda la magia que vamos a usar externamente apara construir nuestros modelos. Es necesario implementar nuestras clases propias derivadas de *pl.LightningModule* si se quiere aprovechar el motor de Pytorch-Lightning.\n",
    "\n",
    "+ `configure_optimizers(self)`: este es un método abstracto que siempre debe definir el programador en su implementación propia. El plurar  se debe a que en algunas implementación, como por ejemplo, las redes generativas adversarias (GAN's) puede ser necesario definir más de un optimizador.\n",
    "\n",
    "+ En PyTorch, la función `view()` se utiliza para cambiar la forma de un tensor sin modificar sus datos subyacentes. *view()* toma como argumento una nueva forma deseada para el tensor y devuelve un nuevo tensor con la forma especificada. En el caso específico de x.view(x.size(0), -1), se realiza una operación de vista para cambiar la forma del tensor x. Aquí se explica qué hace cada parte:\n",
    "    * x.size(0): Devuelve el tamaño de la primera dimensión de x. En otras palabras, devuelve el tamaño del lote (batch size) si x es un tensor de imágenes en lotes, como es el caso en este ejemplo.\n",
    "    * -1: Esto se utiliza para que PyTorch infiera automáticamente el tamaño de la segunda dimensión, en función de la forma original y la primera dimensión especificada. Es decir, se le pide a PyTorch que calcule automáticamente el tamaño de la segunda dimensión para que el número total de elementos se mantenga igual.\n",
    "    * La línea de código x = x.view(x.size(0), -1) se utiliza comúnmente en el procesamiento de imágenes para aplanar un tensor de imágenes en lotes en una forma unidimensional, lo que facilita su uso en capas completamente conectadas o lineales en una red neuronal. \n",
    "    * El tamaño de la segunda dimensión se calcula automáticamente para asegurarse de que el número total de elementos se mantenga igual después de la operación de vista. En este caso, la operación view() transformará el tensor x en un tensor de forma (tamaño del lote, tamaño_plano).\n",
    "    * En consecuencia, en este ejmeplo `x.view(x.size(0), -1)` transforma el tensor de entrada que tiene forma $(32,28,28)$ en otro tensor de forma $(32, 784)$. En nuestro ejemplo 32 será el tamaño del lote, que de momento no hemos definido y 784 = 28*28.\n",
    "\n",
    "+ `training_step(self, train_batch, batch_idx)` este es el método núcleo del entrenamiento, en donde ocurre la acción. Es el método en donde se actualizan los pesos del modelo siguiendo el procedimiento determinado por el optimizador. Se hace un paso hacia adelante en el proceso.  Pero también es el sitio en donde se puede almacenar información de seguimiento del entrenamiento. El método recibe un lote de datos y un índice del lote el cual no se usará en este ejemplo, pero que debe ser recibido debido a la arquitectura del  modelo. La sentencia `self.log('train_loss', loss)` onta la información de la pérdida en el log para ser usada por ejemplo por `Tensorboard`.\n",
    "\n",
    "+ `validation_step(self, val_batch, batch_idx):` Tiene un código idéntico a *training_step()*. Sin embargo en este caso el método espera datos de prueba y únicamente calcula la función de pérdida. La actualización del gradiente es apagada en este método. \n",
    "\n",
    "+ En PyTorch, `forward(self)` es un método que se define en una clase que hereda de `torch.nn.Module`. Tenga en cuenta que `pl.LightningModule` se deriva de esa clase. El método forward() define la secuencia de operaciones que se aplican a los datos de entrada cuando se llama a la instancia de la clase. El propósito principal de forward() es definir la `propagación hacia adelante`(forward propagation) de los datos a través de la red neuronal. En otras palabras, especifica cómo se aplican las operaciones y transformaciones a los datos de entrada para obtener la salida deseada. Dentro de forward(), puedes definir las capas y operaciones que componen tu red neuronal. Esto puede incluir capas lineales, capas de convolución, capas de activación, operaciones de pooling, entre otras. También puedes realizar operaciones de preprocesamiento o postprocesamiento según sea necesario.  En nuestro ejemplo `forward()` calcula la codificación de las imágenes, transformando los tensores de tamaño 28*28 en tensores de tamaño 3, pasando por una capa intermedia de tamaño 64.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "020fa56a-a0f6-453c-8a13-6b2d1015520d",
   "metadata": {},
   "source": [
    "### <span style=\"color:#4CC9F0\">Datasets y DataLoaders</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "913fc9e8-6f09-4867-a666-8b21374fb031",
   "metadata": {},
   "source": [
    "En PyTorch, los conjuntos de datos (datasets) y los cargadores de datos (dataloaders) son componentes clave para el manejo y la carga eficiente de los datos utilizados en el entrenamiento y evaluación de modelos de aprendizaje automático.\n",
    "\n",
    "+ `Conjuntos de datos (datasets)`: Los conjuntos de datos en PyTorch son clases que representan una colección de datos utilizados para el entrenamiento, validación o prueba de un modelo. Un conjunto de datos típico contiene pares de datos de entrada y su correspondiente etiqueta o resultado objetivo. PyTorch proporciona la clase base torch.utils.data.Dataset que se puede heredar para crear conjuntos de datos personalizados.\n",
    "Los conjuntos de datos en PyTorch generalmente implementan los métodos __len__ y __getitem__. El método __len__ devuelve la longitud del conjunto de datos (es decir, el número total de muestras), mientras que el método __getitem__ permite acceder a una muestra individual a través de su índice. Es decir son clases iteradoras.\n",
    "\n",
    "+ `Cargadores de datos (dataloaders)`: Los cargadores de datos en PyTorch son objetos que facilitan la carga y el procesamiento eficiente de los datos en lotes durante el entrenamiento de un modelo. El cargador de datos toma un conjunto de datos y proporciona iteradores que generan lotes de datos de forma eficiente. PyTorch proporciona la clase `torch.utils.data.DataLoader` para crear cargadores de datos.\n",
    "\n",
    "Los cargadores de datos en PyTorch se configuran con varios parámetros, como el tamaño del lote (batch size), la mezcla de los datos, la paralelización y la asignación de los datos a dispositivos (por ejemplo, CPU o GPU). Estos parámetros permiten ajustar el rendimiento y la eficiencia del entrenamiento según los recursos disponibles.\n",
    "\n",
    "El uso de cargadores de datos en PyTorch ofrece varias ventajas. Primero, facilitan la carga eficiente de los datos en la memoria del dispositivo de entrenamiento en lotes, lo que acelera el entrenamiento. Además, los cargadores de datos proporcionan una forma sencilla de iterar sobre los datos durante el entrenamiento, ocultando la complejidad de la carga y el preprocesamiento de los datos.\n",
    "\n",
    "En resumen, los conjuntos de datos y los cargadores de datos en PyTorch son componentes esenciales para la manipulación y carga eficiente de los datos utilizados en el entrenamiento de modelos de aprendizaje automático. Los conjuntos de datos representan colecciones de datos, mientras que los cargadores de datos permiten la carga eficiente de los datos en lotes durante el entrenamiento, mejorando la velocidad y la eficiencia del proceso.\n",
    "\n",
    "En nuestro ejemplo usamos el objeto MNIST  disponible en `torchvision.datasets` para crear nuestro dataset.En otra lección aprendemos como crear los datasets desde cero (from scratch)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "b62ac500-b557-40a8-8314-8809bd7154b1",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = MNIST('', train=True, download=True, transform=transforms.ToTensor())\n",
    "mnist_train, mnist_val = random_split(dataset, [55000, 5000])\n",
    "\n",
    "train_dataloader = DataLoader(mnist_train, batch_size=32,num_workers=4)\n",
    "val_dataloader = DataLoader(mnist_val, batch_size=32, num_workers=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "566a0b1b-5592-40e5-a8a9-239affb78721",
   "metadata": {},
   "source": [
    "### <span style=\"color:#4CC9F0\">Explicaciones</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79eeaef9-7a43-4a7c-b3c1-f055a38a2849",
   "metadata": {},
   "source": [
    "+ El objeto MNIST cargara los datos desde una fuente. Si *download=True*  bajará los datos desde la fuente en Pytorch la priemra vez que se invoque le objeto. Si ya se han bajado buscará los datos localmente. \n",
    "+ `transforms.ToTensor()` transforma los datos crudos (raw data) en tensores de Pytorch.\n",
    "+ `random_split()`separa aleatoriamente los datos en subconjuntos de tamaños  definidos en la lista de tamaños. En ejemplo crea dos subconjuntos, el primero de tamaño 55.000 y el segundo de tamaño 5.000.\n",
    "+ `DataLoader` crea los lotes de datos en este caso de tamaño 32. `num_workers=4` indica cuantas cpus se usaran para procesar los datos en paralelo, independientemente del proceso optimización. El uso de numworkers permite el uso simultáneo de cpus y gpus en el proceso de entrenamiento. cpus se encargan de los datos y gpus del modelo."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80c2ee8c-5fc0-4838-812f-2ace9b58533e",
   "metadata": {},
   "source": [
    "### <span style=\"color:#4CC9F0\">Instancia el modelo</span> "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "8cb06735-76e6-4b48-9b3c-8ca7119597fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = LitAutoEncoder()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4866082d-e629-44d7-b35c-97e42e50af2e",
   "metadata": {},
   "source": [
    "### <span style=\"color:#4CC9F0\">Clase Trainer</span> "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d90e3a1-fd68-4aa7-a10f-4168c4a41dfb",
   "metadata": {},
   "source": [
    "La clase Trainer de PyTorch Lightning es uno de los componentes fundamentales que ofrece el marco de trabajo para el entrenamiento y evaluación de modelos de aprendizaje automático. \n",
    "\n",
    "Proporciona una interfaz de alto nivel para controlar y administrar el proceso de entrenamiento, incluyendo la configuración de hiperparámetros, el manejo de dispositivos, el seguimiento de métricas y la ejecución de bucles de entrenamiento y validación.\n",
    "\n",
    "A continuación, te brindaré una explicación detallada de los aspectos clave de la clase Trainer:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57f114cb-f8d6-4e53-840a-ed9d8cbca8a4",
   "metadata": {},
   "source": [
    "#### <span style=\"color:#4CC9F0\">Configuración de hiperparámetros</span> "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47a1c5ee-10c6-4358-b19e-ec9a718abd2c",
   "metadata": {},
   "source": [
    "+ **max_epochs**: Define el número máximo de épocas (iteraciones completas a través del conjunto de datos) para el entrenamiento.\n",
    "+ **gpus / tpu_cores**: Permite especificar qué GPU(s) o TPU(s) se utilizarán para el entrenamiento.\n",
    "+ **precision**: Define la precisión de los cálculos numéricos, como '16bit' (FP16) o '32bit' (FP32).\n",
    "+ **amp_level**: Define el nivel de precisión mixta automática para el entrenamiento, como 'O1', 'O2' o 'O3'.\n",
    "+ **auto_lr_find**: Realiza una búsqueda automática del learning rate óptimo utilizando el método lr_find().\n",
    "\n",
    "El nivel de precisión mixta automática se refiere a una técnica de optimización en el entrenamiento de modelos de aprendizaje automático utilizando aritmética de punto flotante de menor precisión. PyTorch proporciona la capacidad de realizar entrenamiento en precisión mixta automática utilizando los niveles 'O1', 'O2' y 'O3', que representan diferentes niveles de optimización de precisión mixta.\n",
    "\n",
    "La precisión mixta automática utiliza una combinación de números de punto flotante de precisión reducida (FP16) y de precisión completa (FP32) durante el entrenamiento. La idea es aprovechar los beneficios de la aritmética de menor precisión (FP16) para acelerar el cálculo y reducir los requisitos de memoria, mientras se minimiza la pérdida de precisión en el resultado final.\n",
    "\n",
    "Cada nivel de precisión mixta automática tiene diferentes características y compromisos:\n",
    "\n",
    "+ Nivel 'O1': En este nivel, la mayoría de las operaciones se realizan en FP32, pero se utilizan ciertas operaciones clave en FP16 para acelerar el cálculo y reducir la memoria. También se utiliza escalado de gradiente para evitar la pérdida de precisión durante la retropropagación.\n",
    "\n",
    "+ Nivel 'O2': En este nivel, más operaciones se realizan en FP16, incluyendo el cálculo de gradientes y la actualización de los pesos. También se utiliza escalado de gradiente y clipping para controlar la propagación de gradientes y evitar explosiones de gradiente.\n",
    "\n",
    "+ Nivel 'O3': En este nivel, se aplican técnicas más avanzadas, como el almacenamiento en matriz maestra (master weight) y el ajuste de precisión dinámica, para lograr un equilibrio adicional entre el rendimiento y la precisión.\n",
    "\n",
    "La elección del nivel de precisión mixta automática depende de varios factores, como el modelo específico, el hardware utilizado y los requisitos de precisión del problema. A medida que se avanza de 'O1' a 'O3', se obtiene un mayor rendimiento y ahorro de memoria, pero también puede haber una ligera pérdida de precisión.\n",
    "\n",
    "Es importante tener en cuenta que, si bien la precisión mixta automática puede ser beneficiosa en muchos casos, puede requerir una cuidadosa configuración y ajuste para obtener los mejores resultados."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c63fe7c5-3649-4b51-8686-3dea09662481",
   "metadata": {},
   "source": [
    "#### <span style=\"color:#4CC9F0\">Funcionalidades adicionales</span> "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ae371d0-7eb5-484a-a1f9-f2d67b1f7074",
   "metadata": {},
   "source": [
    "+ **callbacks**: Permite agregar callbacks personalizados para realizar acciones específicas durante el entrenamiento, como guardar modelos, ajustar hiperparámetros, etc.\n",
    "+ **logger**: Facilita el registro y la visualización de métricas y resultados del entrenamiento en herramientas como `TensorBoard`, `Comet.ml` y otras.\n",
    "+ **checkpoint_callback**: Controla el guardado de los checkpoints del modelo durante el entrenamiento.\n",
    "+ **resume_from_checkpoint**: Permite reanudar un entrenamiento a partir de un checkpoint guardado previamente.\n",
    "+ **limit_train_batches / limit_val_batches / limit_test_batches**: Establece límites en el número de lotes utilizados durante el entrenamiento, validación y pruebas, útil para acelerar la iteración durante la depuración."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a9dfa4b-1c64-4849-b148-ab4326638b67",
   "metadata": {},
   "source": [
    "#### <span style=\"color:#4CC9F0\">Métodos principales</span> "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8292d67a-fc46-4c23-a95a-2e6d10e1df2f",
   "metadata": {},
   "source": [
    "+ **fit(model, train_dataloader, val_dataloaders)**: Inicia el proceso de entrenamiento del modelo. Recibe el modelo, el dataloader de entrenamiento y los dataloaders de validación.\n",
    "+ **validate(model, val_dataloaders)**: Realiza la evaluación del modelo en los dataloaders de validación sin realizar el entrenamiento.\n",
    "+ **test(model, test_dataloaders)**: Ejecuta la evaluación del modelo en los dataloaders de prueba sin realizar el entrenamiento.\n",
    "+ **predict(model, dataloaders)**: Genera predicciones utilizando el modelo en los dataloaders proporcionados.\n",
    "+ **lr_find(model)**: Realiza una búsqueda del learning rate óptimo para el entrenamiento utilizando el método de learning rate finder."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1ff1af8-9770-4d7a-a730-f2c57b72d25f",
   "metadata": {},
   "source": [
    "#### <span style=\"color:#4CC9F0\">Métodos para el ajuste de hiperparámetros</span> "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2400f59f-508a-4e14-a29b-8ef4b07c9326",
   "metadata": {},
   "source": [
    "+ **tune(model, train_dataloader, val_dataloaders)**: Permite ajustar automáticamente los hiperparámetros del modelo utilizando herramientas como `Optuna` o `Ray Tune`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac57ef73-8415-4ccb-a238-fcbea0366475",
   "metadata": {},
   "source": [
    "### <span style=\"color:#4CC9F0\">Entrenamiento</span> "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "385dd0dd-2569-40bd-9359-65a733f556aa",
   "metadata": {},
   "source": [
    "Para poider usar Tensorboard como logger instalar así:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b55b4c03-3c3e-41b8-886d-36f1e62c440c",
   "metadata": {},
   "outputs": [],
   "source": [
    "#!conda install -c conda-forge tensorboard"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e54d96b4-abf7-4425-a4ca-1c9657471ce4",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[34m\u001b[1mwandb\u001b[0m: Currently logged in as: \u001b[33mammontenegrod\u001b[0m (\u001b[33maprendizaje-profundo\u001b[0m). Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "Tracking run with wandb version 0.15.2"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Run data is saved locally in <code>./wandb/run-20230514_110452-56proei5</code>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Syncing run <strong><a href='https://wandb.ai/aprendizaje-profundo/wandb-lightning-autoencoder/runs/56proei5' target=\"_blank\">comic-sunset-2</a></strong> to <a href='https://wandb.ai/aprendizaje-profundo/wandb-lightning-autoencoder' target=\"_blank\">Weights & Biases</a> (<a href='https://wandb.me/run' target=\"_blank\">docs</a>)<br/>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       " View project at <a href='https://wandb.ai/aprendizaje-profundo/wandb-lightning-autoencoder' target=\"_blank\">https://wandb.ai/aprendizaje-profundo/wandb-lightning-autoencoder</a>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       " View run at <a href='https://wandb.ai/aprendizaje-profundo/wandb-lightning-autoencoder/runs/56proei5' target=\"_blank\">https://wandb.ai/aprendizaje-profundo/wandb-lightning-autoencoder/runs/56proei5</a>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#from pytorch_lightning.loggers import TensorBoardXLogger\n",
    "#logger = TensorBoardXLogger(save_dir='../logs', name='experiment_minist_autoencoder')\n",
    "\n",
    "#from torch.utils.tensorboard import SummaryWriter\n",
    "#from pytorch_lightning.loggers import CometLogger\n",
    "# Crear el logger de TensorBoard\n",
    "#logger = pl.loggers.TensorBoardLogger(save_dir='../logs', name='experiment_name')\n",
    "#logger = CometLogger(api_key=\"428wVrhSY36ak1LoKfbJ285ix\", project_name=\"mnist_autoencoder\", \n",
    " #                    workspace=\"thttps://www.comet.com/ammontenegrod#projects\")\n",
    "\n",
    "# Inicializa wandb logger\n",
    "wandb_logger = WandbLogger(project='wandb-lightning-autoencoder', job_type='train')    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "525fb418-4c5a-446a-a574-a7e396ae11fd",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/alvaro/anaconda3/envs/lightning/lib/python3.11/site-packages/lightning_fabric/connector.py:562: UserWarning: 16 is supported for historical reasons but its usage is discouraged. Please set your precision to 16-mixed instead!\n",
      "  rank_zero_warn(\n",
      "/home/alvaro/anaconda3/envs/lightning/lib/python3.11/site-packages/pytorch_lightning/trainer/connectors/accelerator_connector.py:517: UserWarning: You passed `Trainer(accelerator='cpu', precision='16-mixed')` but AMP with fp16 is not supported on CPU. Using `precision='bf16-mixed'` instead.\n",
      "  rank_zero_warn(\n",
      "Using bfloat16 Automatic Mixed Precision (AMP)\n",
      "GPU available: False, used: False\n",
      "TPU available: False, using: 0 TPU cores\n",
      "IPU available: False, using: 0 IPUs\n",
      "HPU available: False, using: 0 HPUs\n"
     ]
    }
   ],
   "source": [
    "trainer = pl.Trainer(accelerator='cpu', \n",
    "                     max_epochs= 5, \n",
    "                     precision=16,\n",
    "                     logger=wandb_logger,\n",
    "                     limit_train_batches=0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "79a150a7-450c-43df-aab0-e8e785d598f4",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "  | Name    | Type       | Params\n",
      "---------------------------------------\n",
      "0 | encoder | Sequential | 50.4 K\n",
      "1 | decoder | Sequential | 51.2 K\n",
      "---------------------------------------\n",
      "101 K     Trainable params\n",
      "0         Non-trainable params\n",
      "101 K     Total params\n",
      "0.407     Total estimated model params size (MB)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0: 100%|████████████████████| 859/859 [00:43<00:00, 19.73it/s, v_num=oei5]\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                       | 0/157 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                          | 0/157 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   1%|                  | 1/157 [00:00<00:04, 35.10it/s]\u001b[A\n",
      "Validation DataLoader 0:   1%|▏                 | 2/157 [00:00<00:04, 37.01it/s]\u001b[A\n",
      "Validation DataLoader 0:   2%|▎                 | 3/157 [00:00<00:03, 39.75it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▍                 | 4/157 [00:00<00:03, 40.46it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                 | 5/157 [00:00<00:03, 42.03it/s]\u001b[A\n",
      "Validation DataLoader 0:   4%|▋                 | 6/157 [00:00<00:03, 43.57it/s]\u001b[A\n",
      "Validation DataLoader 0:   4%|▊                 | 7/157 [00:00<00:03, 43.60it/s]\u001b[A\n",
      "Validation DataLoader 0:   5%|▉                 | 8/157 [00:00<00:03, 44.89it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█                 | 9/157 [00:00<00:03, 45.49it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█                | 10/157 [00:00<00:03, 44.48it/s]\u001b[A\n",
      "Validation DataLoader 0:   7%|█▏               | 11/157 [00:00<00:03, 44.72it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▎               | 12/157 [00:00<00:03, 45.45it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▍               | 13/157 [00:00<00:03, 45.51it/s]\u001b[A\n",
      "Validation DataLoader 0:   9%|█▌               | 14/157 [00:00<00:03, 46.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▌               | 15/157 [00:00<00:03, 46.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▋               | 16/157 [00:00<00:03, 46.07it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|█▊               | 17/157 [00:00<00:03, 46.31it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|█▉               | 18/157 [00:00<00:02, 46.65it/s]\u001b[A\n",
      "Validation DataLoader 0:  12%|██               | 19/157 [00:00<00:02, 47.13it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▏              | 20/157 [00:00<00:02, 47.68it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▎              | 21/157 [00:00<00:02, 47.72it/s]\u001b[A\n",
      "Validation DataLoader 0:  14%|██▍              | 22/157 [00:00<00:02, 47.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  15%|██▍              | 23/157 [00:00<00:02, 47.81it/s]\u001b[A\n",
      "Validation DataLoader 0:  15%|██▌              | 24/157 [00:00<00:02, 48.11it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▋              | 25/157 [00:00<00:02, 47.89it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|██▊              | 26/157 [00:00<00:02, 47.10it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|██▉              | 27/157 [00:00<00:02, 46.93it/s]\u001b[A\n",
      "Validation DataLoader 0:  18%|███              | 28/157 [00:00<00:02, 47.22it/s]\u001b[A\n",
      "Validation DataLoader 0:  18%|███▏             | 29/157 [00:00<00:02, 47.22it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▏             | 30/157 [00:00<00:02, 47.28it/s]\u001b[A\n",
      "Validation DataLoader 0:  20%|███▎             | 31/157 [00:00<00:02, 47.42it/s]\u001b[A\n",
      "Validation DataLoader 0:  20%|███▍             | 32/157 [00:00<00:02, 47.11it/s]\u001b[A\n",
      "Validation DataLoader 0:  21%|███▌             | 33/157 [00:00<00:02, 47.13it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|███▋             | 34/157 [00:00<00:02, 47.35it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|███▊             | 35/157 [00:00<00:02, 47.60it/s]\u001b[A\n",
      "Validation DataLoader 0:  23%|███▉             | 36/157 [00:00<00:02, 47.42it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████             | 37/157 [00:00<00:02, 47.24it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████             | 38/157 [00:00<00:02, 46.64it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▏            | 39/157 [00:00<00:02, 46.78it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▎            | 40/157 [00:00<00:02, 46.94it/s]\u001b[A\n",
      "Validation DataLoader 0:  26%|████▍            | 41/157 [00:00<00:02, 47.15it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▌            | 42/157 [00:00<00:02, 47.32it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▋            | 43/157 [00:00<00:02, 47.18it/s]\u001b[A\n",
      "Validation DataLoader 0:  28%|████▊            | 44/157 [00:00<00:02, 47.15it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|████▊            | 45/157 [00:00<00:02, 47.24it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|████▉            | 46/157 [00:00<00:02, 47.33it/s]\u001b[A\n",
      "Validation DataLoader 0:  30%|█████            | 47/157 [00:00<00:02, 47.56it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▏           | 48/157 [00:01<00:02, 47.73it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▎           | 49/157 [00:01<00:02, 47.63it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▍           | 50/157 [00:01<00:02, 47.56it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▌           | 51/157 [00:01<00:02, 47.73it/s]\u001b[A\n",
      "Validation DataLoader 0:  33%|█████▋           | 52/157 [00:01<00:02, 47.85it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|█████▋           | 53/157 [00:01<00:02, 47.99it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|█████▊           | 54/157 [00:01<00:02, 48.08it/s]\u001b[A\n",
      "Validation DataLoader 0:  35%|█████▉           | 55/157 [00:01<00:02, 48.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  36%|██████           | 56/157 [00:01<00:02, 47.94it/s]\u001b[A\n",
      "Validation DataLoader 0:  36%|██████▏          | 57/157 [00:01<00:02, 47.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  37%|██████▎          | 58/157 [00:01<00:02, 48.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▍          | 59/157 [00:01<00:02, 48.15it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▍          | 60/157 [00:01<00:02, 48.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  39%|██████▌          | 61/157 [00:01<00:01, 48.15it/s]\u001b[A\n",
      "Validation DataLoader 0:  39%|██████▋          | 62/157 [00:01<00:01, 48.06it/s]\u001b[A\n",
      "Validation DataLoader 0:  40%|██████▊          | 63/157 [00:01<00:01, 48.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|██████▉          | 64/157 [00:01<00:01, 48.12it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████          | 65/157 [00:01<00:01, 48.03it/s]\u001b[A\n",
      "Validation DataLoader 0:  42%|███████▏         | 66/157 [00:01<00:01, 48.14it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▎         | 67/157 [00:01<00:01, 48.24it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▎         | 68/157 [00:01<00:01, 48.33it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|███████▍         | 69/157 [00:01<00:01, 48.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  45%|███████▌         | 70/157 [00:01<00:01, 48.56it/s]\u001b[A\n",
      "Validation DataLoader 0:  45%|███████▋         | 71/157 [00:01<00:01, 48.65it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|███████▊         | 72/157 [00:01<00:01, 48.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|███████▉         | 73/157 [00:01<00:01, 48.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  47%|████████         | 74/157 [00:01<00:01, 48.42it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████         | 75/157 [00:01<00:01, 48.49it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████▏        | 76/157 [00:01<00:01, 48.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  49%|████████▎        | 77/157 [00:01<00:01, 48.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|████████▍        | 78/157 [00:01<00:01, 48.57it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|████████▌        | 79/157 [00:01<00:01, 48.50it/s]\u001b[A\n",
      "Validation DataLoader 0:  51%|████████▋        | 80/157 [00:01<00:01, 48.41it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|████████▊        | 81/157 [00:01<00:01, 48.28it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|████████▉        | 82/157 [00:01<00:01, 48.22it/s]\u001b[A\n",
      "Validation DataLoader 0:  53%|████████▉        | 83/157 [00:01<00:01, 48.29it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████        | 84/157 [00:01<00:01, 48.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████▏       | 85/157 [00:01<00:01, 48.21it/s]\u001b[A\n",
      "Validation DataLoader 0:  55%|█████████▎       | 86/157 [00:01<00:01, 48.32it/s]\u001b[A\n",
      "Validation DataLoader 0:  55%|█████████▍       | 87/157 [00:01<00:01, 48.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|█████████▌       | 88/157 [00:01<00:01, 48.29it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|█████████▋       | 89/157 [00:01<00:01, 48.21it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|█████████▋       | 90/157 [00:01<00:01, 48.27it/s]\u001b[A\n",
      "Validation DataLoader 0:  58%|█████████▊       | 91/157 [00:01<00:01, 48.36it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|█████████▉       | 92/157 [00:01<00:01, 48.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████       | 93/157 [00:01<00:01, 48.54it/s]\u001b[A\n",
      "Validation DataLoader 0:  60%|██████████▏      | 94/157 [00:01<00:01, 48.57it/s]\u001b[A\n",
      "Validation DataLoader 0:  61%|██████████▎      | 95/157 [00:01<00:01, 48.40it/s]\u001b[A\n",
      "Validation DataLoader 0:  61%|██████████▍      | 96/157 [00:01<00:01, 48.33it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|██████████▌      | 97/157 [00:02<00:01, 48.35it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|██████████▌      | 98/157 [00:02<00:01, 48.38it/s]\u001b[A\n",
      "Validation DataLoader 0:  63%|██████████▋      | 99/157 [00:02<00:01, 48.36it/s]\u001b[A\n",
      "Validation DataLoader 0:  64%|██████████▏     | 100/157 [00:02<00:01, 48.44it/s]\u001b[A\n",
      "Validation DataLoader 0:  64%|██████████▎     | 101/157 [00:02<00:01, 48.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  65%|██████████▍     | 102/157 [00:02<00:01, 48.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|██████████▍     | 103/157 [00:02<00:01, 48.44it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|██████████▌     | 104/157 [00:02<00:01, 48.48it/s]\u001b[A\n",
      "Validation DataLoader 0:  67%|██████████▋     | 105/157 [00:02<00:01, 48.49it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|██████████▊     | 106/157 [00:02<00:01, 48.61it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|██████████▉     | 107/157 [00:02<00:01, 48.61it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|███████████     | 108/157 [00:02<00:01, 48.26it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|███████████     | 109/157 [00:02<00:00, 48.30it/s]\u001b[A\n",
      "Validation DataLoader 0:  70%|███████████▏    | 110/157 [00:02<00:00, 48.29it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|███████████▎    | 111/157 [00:02<00:00, 48.34it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|███████████▍    | 112/157 [00:02<00:00, 48.40it/s]\u001b[A\n",
      "Validation DataLoader 0:  72%|███████████▌    | 113/157 [00:02<00:00, 48.40it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|███████████▌    | 114/157 [00:02<00:00, 48.28it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|███████████▋    | 115/157 [00:02<00:00, 48.21it/s]\u001b[A\n",
      "Validation DataLoader 0:  74%|███████████▊    | 116/157 [00:02<00:00, 48.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|███████████▉    | 117/157 [00:02<00:00, 48.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|████████████    | 118/157 [00:02<00:00, 48.09it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|████████████▏   | 119/157 [00:02<00:00, 48.10it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|████████████▏   | 120/157 [00:02<00:00, 48.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  77%|████████████▎   | 121/157 [00:02<00:00, 48.13it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|████████████▍   | 122/157 [00:02<00:00, 48.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|████████████▌   | 123/157 [00:02<00:00, 48.22it/s]\u001b[A\n",
      "Validation DataLoader 0:  79%|████████████▋   | 124/157 [00:02<00:00, 48.27it/s]\u001b[A\n",
      "Validation DataLoader 0:  80%|████████████▋   | 125/157 [00:02<00:00, 48.29it/s]\u001b[A\n",
      "Validation DataLoader 0:  80%|████████████▊   | 126/157 [00:02<00:00, 48.22it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|████████████▉   | 127/157 [00:02<00:00, 48.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  82%|█████████████   | 128/157 [00:02<00:00, 48.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  82%|█████████████▏  | 129/157 [00:02<00:00, 48.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|█████████████▏  | 130/157 [00:02<00:00, 48.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|█████████████▎  | 131/157 [00:02<00:00, 48.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|█████████████▍  | 132/157 [00:02<00:00, 48.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  85%|█████████████▌  | 133/157 [00:02<00:00, 48.14it/s]\u001b[A\n",
      "Validation DataLoader 0:  85%|█████████████▋  | 134/157 [00:02<00:00, 48.21it/s]\u001b[A\n",
      "Validation DataLoader 0:  86%|█████████████▊  | 135/157 [00:02<00:00, 48.28it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|█████████████▊  | 136/157 [00:02<00:00, 48.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|█████████████▉  | 137/157 [00:02<00:00, 48.20it/s]\u001b[A\n",
      "Validation DataLoader 0:  88%|██████████████  | 138/157 [00:02<00:00, 48.15it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|██████████████▏ | 139/157 [00:02<00:00, 48.13it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|██████████████▎ | 140/157 [00:02<00:00, 48.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|██████████████▎ | 141/157 [00:02<00:00, 48.24it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|██████████████▍ | 142/157 [00:02<00:00, 48.21it/s]\u001b[A\n",
      "Validation DataLoader 0:  91%|██████████████▌ | 143/157 [00:02<00:00, 48.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|██████████████▋ | 144/157 [00:02<00:00, 48.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|██████████████▊ | 145/157 [00:03<00:00, 48.09it/s]\u001b[A\n",
      "Validation DataLoader 0:  93%|██████████████▉ | 146/157 [00:03<00:00, 48.16it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|██████████████▉ | 147/157 [00:03<00:00, 48.22it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|███████████████ | 148/157 [00:03<00:00, 48.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  95%|███████████████▏| 149/157 [00:03<00:00, 48.14it/s]\u001b[A\n",
      "Validation DataLoader 0:  96%|███████████████▎| 150/157 [00:03<00:00, 48.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  96%|███████████████▍| 151/157 [00:03<00:00, 48.13it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|███████████████▍| 152/157 [00:03<00:00, 48.14it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|███████████████▌| 153/157 [00:03<00:00, 48.15it/s]\u001b[A\n",
      "Validation DataLoader 0:  98%|███████████████▋| 154/157 [00:03<00:00, 48.12it/s]\u001b[A\n",
      "Validation DataLoader 0:  99%|███████████████▊| 155/157 [00:03<00:00, 48.09it/s]\u001b[A\n",
      "Validation DataLoader 0:  99%|███████████████▉| 156/157 [00:03<00:00, 48.14it/s]\u001b[A\n",
      "Epoch 0: 100%|████████████████████| 859/859 [00:47<00:00, 18.27it/s, v_num=oei5]\u001b[A\n",
      "Epoch 1: 100%|████████████████████| 859/859 [00:51<00:00, 16.58it/s, v_num=oei5]\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                       | 0/157 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                          | 0/157 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   1%|                  | 1/157 [00:00<00:05, 30.41it/s]\u001b[A\n",
      "Validation DataLoader 0:   1%|▏                 | 2/157 [00:00<00:04, 37.31it/s]\u001b[A\n",
      "Validation DataLoader 0:   2%|▎                 | 3/157 [00:00<00:03, 39.50it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▍                 | 4/157 [00:00<00:03, 40.90it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                 | 5/157 [00:00<00:03, 43.45it/s]\u001b[A\n",
      "Validation DataLoader 0:   4%|▋                 | 6/157 [00:00<00:03, 45.54it/s]\u001b[A\n",
      "Validation DataLoader 0:   4%|▊                 | 7/157 [00:00<00:03, 45.90it/s]\u001b[A\n",
      "Validation DataLoader 0:   5%|▉                 | 8/157 [00:00<00:03, 46.51it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█                 | 9/157 [00:00<00:03, 47.15it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█                | 10/157 [00:00<00:03, 47.82it/s]\u001b[A\n",
      "Validation DataLoader 0:   7%|█▏               | 11/157 [00:00<00:03, 48.42it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▎               | 12/157 [00:00<00:03, 48.20it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▍               | 13/157 [00:00<00:02, 48.09it/s]\u001b[A\n",
      "Validation DataLoader 0:   9%|█▌               | 14/157 [00:00<00:03, 47.49it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▌               | 15/157 [00:00<00:03, 45.66it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▋               | 16/157 [00:00<00:03, 44.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|█▊               | 17/157 [00:00<00:03, 45.57it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|█▉               | 18/157 [00:00<00:03, 45.90it/s]\u001b[A\n",
      "Validation DataLoader 0:  12%|██               | 19/157 [00:00<00:03, 45.80it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▏              | 20/157 [00:00<00:02, 45.78it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▎              | 21/157 [00:00<00:02, 46.16it/s]\u001b[A\n",
      "Validation DataLoader 0:  14%|██▍              | 22/157 [00:00<00:02, 46.51it/s]\u001b[A\n",
      "Validation DataLoader 0:  15%|██▍              | 23/157 [00:00<00:02, 46.97it/s]\u001b[A\n",
      "Validation DataLoader 0:  15%|██▌              | 24/157 [00:00<00:02, 47.26it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▋              | 25/157 [00:00<00:02, 47.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|██▊              | 26/157 [00:00<00:02, 47.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|██▉              | 27/157 [00:00<00:02, 47.61it/s]\u001b[A\n",
      "Validation DataLoader 0:  18%|███              | 28/157 [00:00<00:02, 48.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  18%|███▏             | 29/157 [00:00<00:02, 48.18it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▏             | 30/157 [00:00<00:02, 48.29it/s]\u001b[A\n",
      "Validation DataLoader 0:  20%|███▎             | 31/157 [00:00<00:02, 48.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  20%|███▍             | 32/157 [00:00<00:02, 48.91it/s]\u001b[A\n",
      "Validation DataLoader 0:  21%|███▌             | 33/157 [00:00<00:02, 49.18it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|███▋             | 34/157 [00:00<00:02, 49.08it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|███▊             | 35/157 [00:00<00:02, 49.26it/s]\u001b[A\n",
      "Validation DataLoader 0:  23%|███▉             | 36/157 [00:00<00:02, 49.32it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████             | 37/157 [00:00<00:02, 49.49it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████             | 38/157 [00:00<00:02, 49.44it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▏            | 39/157 [00:00<00:02, 49.68it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▎            | 40/157 [00:00<00:02, 49.41it/s]\u001b[A\n",
      "Validation DataLoader 0:  26%|████▍            | 41/157 [00:00<00:02, 49.09it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▌            | 42/157 [00:00<00:02, 49.24it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▋            | 43/157 [00:00<00:02, 49.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  28%|████▊            | 44/157 [00:00<00:02, 49.38it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|████▊            | 45/157 [00:00<00:02, 49.53it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|████▉            | 46/157 [00:00<00:02, 49.32it/s]\u001b[A\n",
      "Validation DataLoader 0:  30%|█████            | 47/157 [00:00<00:02, 49.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▏           | 48/157 [00:00<00:02, 49.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▎           | 49/157 [00:00<00:02, 49.35it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▍           | 50/157 [00:01<00:02, 49.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▌           | 51/157 [00:01<00:02, 49.09it/s]\u001b[A\n",
      "Validation DataLoader 0:  33%|█████▋           | 52/157 [00:01<00:02, 49.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|█████▋           | 53/157 [00:01<00:02, 49.24it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|█████▊           | 54/157 [00:01<00:02, 49.32it/s]\u001b[A\n",
      "Validation DataLoader 0:  35%|█████▉           | 55/157 [00:01<00:02, 49.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  36%|██████           | 56/157 [00:01<00:02, 49.60it/s]\u001b[A\n",
      "Validation DataLoader 0:  36%|██████▏          | 57/157 [00:01<00:02, 49.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  37%|██████▎          | 58/157 [00:01<00:01, 49.70it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▍          | 59/157 [00:01<00:01, 49.52it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▍          | 60/157 [00:01<00:01, 49.33it/s]\u001b[A\n",
      "Validation DataLoader 0:  39%|██████▌          | 61/157 [00:01<00:01, 49.29it/s]\u001b[A\n",
      "Validation DataLoader 0:  39%|██████▋          | 62/157 [00:01<00:01, 49.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  40%|██████▊          | 63/157 [00:01<00:01, 49.21it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|██████▉          | 64/157 [00:01<00:01, 49.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████          | 65/157 [00:01<00:01, 49.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  42%|███████▏         | 66/157 [00:01<00:01, 49.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▎         | 67/157 [00:01<00:01, 48.98it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▎         | 68/157 [00:01<00:01, 48.93it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|███████▍         | 69/157 [00:01<00:01, 48.87it/s]\u001b[A\n",
      "Validation DataLoader 0:  45%|███████▌         | 70/157 [00:01<00:01, 48.94it/s]\u001b[A\n",
      "Validation DataLoader 0:  45%|███████▋         | 71/157 [00:01<00:01, 49.02it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|███████▊         | 72/157 [00:01<00:01, 49.11it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|███████▉         | 73/157 [00:01<00:01, 49.11it/s]\u001b[A\n",
      "Validation DataLoader 0:  47%|████████         | 74/157 [00:01<00:01, 48.93it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████         | 75/157 [00:01<00:01, 48.99it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████▏        | 76/157 [00:01<00:01, 49.09it/s]\u001b[A\n",
      "Validation DataLoader 0:  49%|████████▎        | 77/157 [00:01<00:01, 49.18it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|████████▍        | 78/157 [00:01<00:01, 49.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|████████▌        | 79/157 [00:01<00:01, 49.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  51%|████████▋        | 80/157 [00:01<00:01, 49.12it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|████████▊        | 81/157 [00:01<00:01, 49.18it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|████████▉        | 82/157 [00:01<00:01, 49.11it/s]\u001b[A\n",
      "Validation DataLoader 0:  53%|████████▉        | 83/157 [00:01<00:01, 49.13it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████        | 84/157 [00:01<00:01, 49.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████▏       | 85/157 [00:01<00:01, 49.20it/s]\u001b[A\n",
      "Validation DataLoader 0:  55%|█████████▎       | 86/157 [00:01<00:01, 49.27it/s]\u001b[A\n",
      "Validation DataLoader 0:  55%|█████████▍       | 87/157 [00:01<00:01, 49.33it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|█████████▌       | 88/157 [00:01<00:01, 48.63it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|█████████▋       | 89/157 [00:01<00:01, 48.20it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|█████████▋       | 90/157 [00:01<00:01, 47.79it/s]\u001b[A\n",
      "Validation DataLoader 0:  58%|█████████▊       | 91/157 [00:01<00:01, 47.49it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|█████████▉       | 92/157 [00:01<00:01, 47.28it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████       | 93/157 [00:01<00:01, 47.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  60%|██████████▏      | 94/157 [00:02<00:01, 46.98it/s]\u001b[A\n",
      "Validation DataLoader 0:  61%|██████████▎      | 95/157 [00:02<00:01, 46.82it/s]\u001b[A\n",
      "Validation DataLoader 0:  61%|██████████▍      | 96/157 [00:02<00:01, 46.69it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|██████████▌      | 97/157 [00:02<00:01, 46.71it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|██████████▌      | 98/157 [00:02<00:01, 46.76it/s]\u001b[A\n",
      "Validation DataLoader 0:  63%|██████████▋      | 99/157 [00:02<00:01, 46.85it/s]\u001b[A\n",
      "Validation DataLoader 0:  64%|██████████▏     | 100/157 [00:02<00:01, 46.92it/s]\u001b[A\n",
      "Validation DataLoader 0:  64%|██████████▎     | 101/157 [00:02<00:01, 47.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  65%|██████████▍     | 102/157 [00:02<00:01, 47.07it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|██████████▍     | 103/157 [00:02<00:01, 47.12it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|██████████▌     | 104/157 [00:02<00:01, 47.14it/s]\u001b[A\n",
      "Validation DataLoader 0:  67%|██████████▋     | 105/157 [00:02<00:01, 47.08it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|██████████▊     | 106/157 [00:02<00:01, 47.06it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|██████████▉     | 107/157 [00:02<00:01, 46.64it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|███████████     | 108/157 [00:02<00:01, 46.63it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|███████████     | 109/157 [00:02<00:01, 46.67it/s]\u001b[A\n",
      "Validation DataLoader 0:  70%|███████████▏    | 110/157 [00:02<00:01, 46.62it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|███████████▎    | 111/157 [00:02<00:00, 46.30it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|███████████▍    | 112/157 [00:02<00:00, 46.28it/s]\u001b[A\n",
      "Validation DataLoader 0:  72%|███████████▌    | 113/157 [00:02<00:00, 46.32it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|███████████▌    | 114/157 [00:02<00:00, 46.37it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|███████████▋    | 115/157 [00:02<00:00, 46.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  74%|███████████▊    | 116/157 [00:02<00:00, 46.53it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|███████████▉    | 117/157 [00:02<00:00, 46.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|████████████    | 118/157 [00:02<00:00, 46.50it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|████████████▏   | 119/157 [00:02<00:00, 46.56it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|████████████▏   | 120/157 [00:02<00:00, 46.60it/s]\u001b[A\n",
      "Validation DataLoader 0:  77%|████████████▎   | 121/157 [00:02<00:00, 46.63it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|████████████▍   | 122/157 [00:02<00:00, 46.68it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|████████████▌   | 123/157 [00:02<00:00, 46.71it/s]\u001b[A\n",
      "Validation DataLoader 0:  79%|████████████▋   | 124/157 [00:02<00:00, 46.75it/s]\u001b[A\n",
      "Validation DataLoader 0:  80%|████████████▋   | 125/157 [00:02<00:00, 46.77it/s]\u001b[A\n",
      "Validation DataLoader 0:  80%|████████████▊   | 126/157 [00:02<00:00, 46.75it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|████████████▉   | 127/157 [00:02<00:00, 46.79it/s]\u001b[A\n",
      "Validation DataLoader 0:  82%|█████████████   | 128/157 [00:02<00:00, 46.81it/s]\u001b[A\n",
      "Validation DataLoader 0:  82%|█████████████▏  | 129/157 [00:02<00:00, 46.90it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|█████████████▏  | 130/157 [00:02<00:00, 46.94it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|█████████████▎  | 131/157 [00:02<00:00, 47.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|█████████████▍  | 132/157 [00:02<00:00, 46.95it/s]\u001b[A\n",
      "Validation DataLoader 0:  85%|█████████████▌  | 133/157 [00:02<00:00, 46.83it/s]\u001b[A\n",
      "Validation DataLoader 0:  85%|█████████████▋  | 134/157 [00:02<00:00, 46.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  86%|█████████████▊  | 135/157 [00:02<00:00, 46.89it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|█████████████▊  | 136/157 [00:02<00:00, 46.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|█████████████▉  | 137/157 [00:02<00:00, 46.91it/s]\u001b[A\n",
      "Validation DataLoader 0:  88%|██████████████  | 138/157 [00:02<00:00, 46.94it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|██████████████▏ | 139/157 [00:02<00:00, 46.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|██████████████▎ | 140/157 [00:02<00:00, 47.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|██████████████▎ | 141/157 [00:02<00:00, 47.06it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|██████████████▍ | 142/157 [00:03<00:00, 47.06it/s]\u001b[A\n",
      "Validation DataLoader 0:  91%|██████████████▌ | 143/157 [00:03<00:00, 46.99it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|██████████████▋ | 144/157 [00:03<00:00, 47.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|██████████████▊ | 145/157 [00:03<00:00, 47.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  93%|██████████████▉ | 146/157 [00:03<00:00, 47.10it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|██████████████▉ | 147/157 [00:03<00:00, 47.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|███████████████ | 148/157 [00:03<00:00, 47.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  95%|███████████████▏| 149/157 [00:03<00:00, 47.21it/s]\u001b[A\n",
      "Validation DataLoader 0:  96%|███████████████▎| 150/157 [00:03<00:00, 47.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  96%|███████████████▍| 151/157 [00:03<00:00, 47.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|███████████████▍| 152/157 [00:03<00:00, 47.26it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|███████████████▌| 153/157 [00:03<00:00, 47.32it/s]\u001b[A\n",
      "Validation DataLoader 0:  98%|███████████████▋| 154/157 [00:03<00:00, 47.35it/s]\u001b[A\n",
      "Validation DataLoader 0:  99%|███████████████▊| 155/157 [00:03<00:00, 47.37it/s]\u001b[A\n",
      "Validation DataLoader 0:  99%|███████████████▉| 156/157 [00:03<00:00, 47.39it/s]\u001b[A\n",
      "Epoch 1: 100%|████████████████████| 859/859 [00:55<00:00, 15.52it/s, v_num=oei5]\u001b[A\n",
      "Epoch 2: 100%|████████████████████| 859/859 [00:51<00:00, 16.68it/s, v_num=oei5]\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                       | 0/157 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                          | 0/157 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   1%|                  | 1/157 [00:00<00:05, 28.13it/s]\u001b[A\n",
      "Validation DataLoader 0:   1%|▏                 | 2/157 [00:00<00:05, 27.20it/s]\u001b[A\n",
      "Validation DataLoader 0:   2%|▎                 | 3/157 [00:00<00:05, 30.19it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▍                 | 4/157 [00:00<00:04, 30.81it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                 | 5/157 [00:00<00:04, 34.15it/s]\u001b[A\n",
      "Validation DataLoader 0:   4%|▋                 | 6/157 [00:00<00:04, 36.73it/s]\u001b[A\n",
      "Validation DataLoader 0:   4%|▊                 | 7/157 [00:00<00:04, 37.24it/s]\u001b[A\n",
      "Validation DataLoader 0:   5%|▉                 | 8/157 [00:00<00:04, 36.68it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█                 | 9/157 [00:00<00:04, 36.02it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█                | 10/157 [00:00<00:04, 36.51it/s]\u001b[A\n",
      "Validation DataLoader 0:   7%|█▏               | 11/157 [00:00<00:03, 37.12it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▎               | 12/157 [00:00<00:03, 37.87it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▍               | 13/157 [00:00<00:03, 37.58it/s]\u001b[A\n",
      "Validation DataLoader 0:   9%|█▌               | 14/157 [00:00<00:03, 38.08it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▌               | 15/157 [00:00<00:03, 38.13it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▋               | 16/157 [00:00<00:03, 37.40it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|█▊               | 17/157 [00:00<00:03, 37.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|█▉               | 18/157 [00:00<00:03, 37.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  12%|██               | 19/157 [00:00<00:03, 37.08it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▏              | 20/157 [00:00<00:03, 37.52it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▎              | 21/157 [00:00<00:03, 37.14it/s]\u001b[A\n",
      "Validation DataLoader 0:  14%|██▍              | 22/157 [00:00<00:03, 37.50it/s]\u001b[A\n",
      "Validation DataLoader 0:  15%|██▍              | 23/157 [00:00<00:03, 38.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  15%|██▌              | 24/157 [00:00<00:03, 38.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▋              | 25/157 [00:00<00:03, 37.29it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|██▊              | 26/157 [00:00<00:03, 37.56it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|██▉              | 27/157 [00:00<00:03, 36.57it/s]\u001b[A\n",
      "Validation DataLoader 0:  18%|███              | 28/157 [00:00<00:03, 36.32it/s]\u001b[A\n",
      "Validation DataLoader 0:  18%|███▏             | 29/157 [00:00<00:03, 36.74it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▏             | 30/157 [00:00<00:03, 36.79it/s]\u001b[A\n",
      "Validation DataLoader 0:  20%|███▎             | 31/157 [00:00<00:03, 36.79it/s]\u001b[A\n",
      "Validation DataLoader 0:  20%|███▍             | 32/157 [00:00<00:03, 36.50it/s]\u001b[A\n",
      "Validation DataLoader 0:  21%|███▌             | 33/157 [00:00<00:03, 36.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|███▋             | 34/157 [00:00<00:03, 36.78it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|███▊             | 35/157 [00:00<00:03, 37.12it/s]\u001b[A\n",
      "Validation DataLoader 0:  23%|███▉             | 36/157 [00:00<00:03, 37.53it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████             | 37/157 [00:00<00:03, 37.76it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████             | 38/157 [00:00<00:03, 38.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▏            | 39/157 [00:01<00:03, 38.43it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▎            | 40/157 [00:01<00:03, 38.52it/s]\u001b[A\n",
      "Validation DataLoader 0:  26%|████▍            | 41/157 [00:01<00:02, 38.84it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▌            | 42/157 [00:01<00:02, 38.98it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▋            | 43/157 [00:01<00:02, 39.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  28%|████▊            | 44/157 [00:01<00:02, 39.42it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|████▊            | 45/157 [00:01<00:02, 39.48it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|████▉            | 46/157 [00:01<00:02, 39.51it/s]\u001b[A\n",
      "Validation DataLoader 0:  30%|█████            | 47/157 [00:01<00:02, 39.37it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▏           | 48/157 [00:01<00:02, 39.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▎           | 49/157 [00:01<00:02, 39.34it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▍           | 50/157 [00:01<00:02, 39.54it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▌           | 51/157 [00:01<00:02, 39.68it/s]\u001b[A\n",
      "Validation DataLoader 0:  33%|█████▋           | 52/157 [00:01<00:02, 39.92it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|█████▋           | 53/157 [00:01<00:02, 39.84it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|█████▊           | 54/157 [00:01<00:02, 39.84it/s]\u001b[A\n",
      "Validation DataLoader 0:  35%|█████▉           | 55/157 [00:01<00:02, 40.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  36%|██████           | 56/157 [00:01<00:02, 40.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  36%|██████▏          | 57/157 [00:01<00:02, 40.34it/s]\u001b[A\n",
      "Validation DataLoader 0:  37%|██████▎          | 58/157 [00:01<00:02, 40.35it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▍          | 59/157 [00:01<00:02, 40.16it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▍          | 60/157 [00:01<00:02, 40.18it/s]\u001b[A\n",
      "Validation DataLoader 0:  39%|██████▌          | 61/157 [00:01<00:02, 40.07it/s]\u001b[A\n",
      "Validation DataLoader 0:  39%|██████▋          | 62/157 [00:01<00:02, 40.03it/s]\u001b[A\n",
      "Validation DataLoader 0:  40%|██████▊          | 63/157 [00:01<00:02, 40.08it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|██████▉          | 64/157 [00:01<00:02, 40.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████          | 65/157 [00:01<00:02, 40.10it/s]\u001b[A\n",
      "Validation DataLoader 0:  42%|███████▏         | 66/157 [00:01<00:02, 40.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▎         | 67/157 [00:01<00:02, 40.14it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▎         | 68/157 [00:01<00:02, 40.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|███████▍         | 69/157 [00:01<00:02, 40.08it/s]\u001b[A\n",
      "Validation DataLoader 0:  45%|███████▌         | 70/157 [00:01<00:02, 40.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  45%|███████▋         | 71/157 [00:01<00:02, 40.24it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|███████▊         | 72/157 [00:01<00:02, 40.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|███████▉         | 73/157 [00:01<00:02, 40.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  47%|████████         | 74/157 [00:01<00:02, 40.07it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████         | 75/157 [00:01<00:02, 40.03it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████▏        | 76/157 [00:01<00:02, 40.09it/s]\u001b[A\n",
      "Validation DataLoader 0:  49%|████████▎        | 77/157 [00:01<00:01, 40.08it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|████████▍        | 78/157 [00:01<00:01, 40.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|████████▌        | 79/157 [00:01<00:01, 40.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  51%|████████▋        | 80/157 [00:01<00:01, 40.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|████████▊        | 81/157 [00:02<00:01, 40.03it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|████████▉        | 82/157 [00:02<00:01, 39.85it/s]\u001b[A\n",
      "Validation DataLoader 0:  53%|████████▉        | 83/157 [00:02<00:01, 39.95it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████        | 84/157 [00:02<00:01, 40.08it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████▏       | 85/157 [00:02<00:01, 40.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  55%|█████████▎       | 86/157 [00:02<00:01, 40.32it/s]\u001b[A\n",
      "Validation DataLoader 0:  55%|█████████▍       | 87/157 [00:02<00:01, 40.38it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|█████████▌       | 88/157 [00:02<00:01, 40.44it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|█████████▋       | 89/157 [00:02<00:01, 40.61it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|█████████▋       | 90/157 [00:02<00:01, 40.37it/s]\u001b[A\n",
      "Validation DataLoader 0:  58%|█████████▊       | 91/157 [00:02<00:01, 40.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|█████████▉       | 92/157 [00:02<00:01, 40.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████       | 93/157 [00:02<00:01, 40.54it/s]\u001b[A\n",
      "Validation DataLoader 0:  60%|██████████▏      | 94/157 [00:02<00:01, 40.61it/s]\u001b[A\n",
      "Validation DataLoader 0:  61%|██████████▎      | 95/157 [00:02<00:01, 40.70it/s]\u001b[A\n",
      "Validation DataLoader 0:  61%|██████████▍      | 96/157 [00:02<00:01, 40.74it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|██████████▌      | 97/157 [00:02<00:01, 40.89it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|██████████▌      | 98/157 [00:02<00:01, 41.02it/s]\u001b[A\n",
      "Validation DataLoader 0:  63%|██████████▋      | 99/157 [00:02<00:01, 41.14it/s]\u001b[A\n",
      "Validation DataLoader 0:  64%|██████████▏     | 100/157 [00:02<00:01, 41.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  64%|██████████▎     | 101/157 [00:02<00:01, 41.21it/s]\u001b[A\n",
      "Validation DataLoader 0:  65%|██████████▍     | 102/157 [00:02<00:01, 41.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|██████████▍     | 103/157 [00:02<00:01, 41.30it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|██████████▌     | 104/157 [00:02<00:01, 41.26it/s]\u001b[A\n",
      "Validation DataLoader 0:  67%|██████████▋     | 105/157 [00:02<00:01, 41.27it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|██████████▊     | 106/157 [00:02<00:01, 41.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|██████████▉     | 107/157 [00:02<00:01, 41.51it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|███████████     | 108/157 [00:02<00:01, 41.65it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|███████████     | 109/157 [00:02<00:01, 41.77it/s]\u001b[A\n",
      "Validation DataLoader 0:  70%|███████████▏    | 110/157 [00:02<00:01, 41.90it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|███████████▎    | 111/157 [00:02<00:01, 42.02it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|███████████▍    | 112/157 [00:02<00:01, 42.02it/s]\u001b[A\n",
      "Validation DataLoader 0:  72%|███████████▌    | 113/157 [00:02<00:01, 41.99it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|███████████▌    | 114/157 [00:02<00:01, 41.95it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|███████████▋    | 115/157 [00:02<00:01, 41.83it/s]\u001b[A\n",
      "Validation DataLoader 0:  74%|███████████▊    | 116/157 [00:02<00:00, 41.85it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|███████████▉    | 117/157 [00:02<00:00, 41.76it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|████████████    | 118/157 [00:02<00:00, 41.58it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|████████████▏   | 119/157 [00:02<00:00, 41.47it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|████████████▏   | 120/157 [00:02<00:00, 41.52it/s]\u001b[A\n",
      "Validation DataLoader 0:  77%|████████████▎   | 121/157 [00:02<00:00, 41.57it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|████████████▍   | 122/157 [00:02<00:00, 41.49it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|████████████▌   | 123/157 [00:02<00:00, 41.56it/s]\u001b[A\n",
      "Validation DataLoader 0:  79%|████████████▋   | 124/157 [00:02<00:00, 41.38it/s]\u001b[A\n",
      "Validation DataLoader 0:  80%|████████████▋   | 125/157 [00:03<00:00, 41.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  80%|████████████▊   | 126/157 [00:03<00:00, 41.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|████████████▉   | 127/157 [00:03<00:00, 41.61it/s]\u001b[A\n",
      "Validation DataLoader 0:  82%|█████████████   | 128/157 [00:03<00:00, 41.69it/s]\u001b[A\n",
      "Validation DataLoader 0:  82%|█████████████▏  | 129/157 [00:03<00:00, 41.80it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|█████████████▏  | 130/157 [00:03<00:00, 41.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|█████████████▎  | 131/157 [00:03<00:00, 41.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|█████████████▍  | 132/157 [00:03<00:00, 42.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  85%|█████████████▌  | 133/157 [00:03<00:00, 42.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  85%|█████████████▋  | 134/157 [00:03<00:00, 42.07it/s]\u001b[A\n",
      "Validation DataLoader 0:  86%|█████████████▊  | 135/157 [00:03<00:00, 42.13it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|█████████████▊  | 136/157 [00:03<00:00, 42.20it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|█████████████▉  | 137/157 [00:03<00:00, 42.26it/s]\u001b[A\n",
      "Validation DataLoader 0:  88%|██████████████  | 138/157 [00:03<00:00, 42.29it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|██████████████▏ | 139/157 [00:03<00:00, 42.30it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|██████████████▎ | 140/157 [00:03<00:00, 42.38it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|██████████████▎ | 141/157 [00:03<00:00, 42.44it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|██████████████▍ | 142/157 [00:03<00:00, 42.51it/s]\u001b[A\n",
      "Validation DataLoader 0:  91%|██████████████▌ | 143/157 [00:03<00:00, 42.51it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|██████████████▋ | 144/157 [00:03<00:00, 42.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|██████████████▊ | 145/157 [00:03<00:00, 42.64it/s]\u001b[A\n",
      "Validation DataLoader 0:  93%|██████████████▉ | 146/157 [00:03<00:00, 42.71it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|██████████████▉ | 147/157 [00:03<00:00, 42.78it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|███████████████ | 148/157 [00:03<00:00, 42.80it/s]\u001b[A\n",
      "Validation DataLoader 0:  95%|███████████████▏| 149/157 [00:03<00:00, 42.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  96%|███████████████▎| 150/157 [00:03<00:00, 42.95it/s]\u001b[A\n",
      "Validation DataLoader 0:  96%|███████████████▍| 151/157 [00:03<00:00, 43.00it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|███████████████▍| 152/157 [00:03<00:00, 43.07it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|███████████████▌| 153/157 [00:03<00:00, 43.14it/s]\u001b[A\n",
      "Validation DataLoader 0:  98%|███████████████▋| 154/157 [00:03<00:00, 43.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  99%|███████████████▊| 155/157 [00:03<00:00, 43.32it/s]\u001b[A\n",
      "Validation DataLoader 0:  99%|███████████████▉| 156/157 [00:03<00:00, 43.42it/s]\u001b[A\n",
      "Epoch 2: 100%|████████████████████| 859/859 [00:55<00:00, 15.50it/s, v_num=oei5]\u001b[A\n",
      "Epoch 3: 100%|████████████████████| 859/859 [00:47<00:00, 18.09it/s, v_num=oei5]\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                       | 0/157 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                          | 0/157 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   1%|                  | 1/157 [00:00<00:10, 14.38it/s]\u001b[A\n",
      "Validation DataLoader 0:   1%|▏                 | 2/157 [00:00<00:07, 20.51it/s]\u001b[A\n",
      "Validation DataLoader 0:   2%|▎                 | 3/157 [00:00<00:05, 26.45it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▍                 | 4/157 [00:00<00:05, 28.66it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                 | 5/157 [00:00<00:04, 31.19it/s]\u001b[A\n",
      "Validation DataLoader 0:   4%|▋                 | 6/157 [00:00<00:04, 31.35it/s]\u001b[A\n",
      "Validation DataLoader 0:   4%|▊                 | 7/157 [00:00<00:04, 33.21it/s]\u001b[A\n",
      "Validation DataLoader 0:   5%|▉                 | 8/157 [00:00<00:04, 34.96it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█                 | 9/157 [00:00<00:04, 36.28it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█                | 10/157 [00:00<00:03, 36.83it/s]\u001b[A\n",
      "Validation DataLoader 0:   7%|█▏               | 11/157 [00:00<00:03, 37.60it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▎               | 12/157 [00:00<00:03, 37.55it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▍               | 13/157 [00:00<00:03, 38.42it/s]\u001b[A\n",
      "Validation DataLoader 0:   9%|█▌               | 14/157 [00:00<00:03, 38.98it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▌               | 15/157 [00:00<00:03, 39.83it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▋               | 16/157 [00:00<00:03, 40.54it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|█▊               | 17/157 [00:00<00:03, 41.22it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|█▉               | 18/157 [00:00<00:03, 41.68it/s]\u001b[A\n",
      "Validation DataLoader 0:  12%|██               | 19/157 [00:00<00:03, 41.87it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▏              | 20/157 [00:00<00:03, 42.36it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▎              | 21/157 [00:00<00:03, 42.95it/s]\u001b[A\n",
      "Validation DataLoader 0:  14%|██▍              | 22/157 [00:00<00:03, 43.38it/s]\u001b[A\n",
      "Validation DataLoader 0:  15%|██▍              | 23/157 [00:00<00:03, 43.35it/s]\u001b[A\n",
      "Validation DataLoader 0:  15%|██▌              | 24/157 [00:00<00:03, 43.68it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▋              | 25/157 [00:00<00:03, 43.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|██▊              | 26/157 [00:00<00:02, 43.80it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|██▉              | 27/157 [00:00<00:02, 44.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  18%|███              | 28/157 [00:00<00:02, 44.52it/s]\u001b[A\n",
      "Validation DataLoader 0:  18%|███▏             | 29/157 [00:00<00:02, 44.37it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▏             | 30/157 [00:00<00:02, 44.73it/s]\u001b[A\n",
      "Validation DataLoader 0:  20%|███▎             | 31/157 [00:00<00:02, 45.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  20%|███▍             | 32/157 [00:00<00:02, 44.94it/s]\u001b[A\n",
      "Validation DataLoader 0:  21%|███▌             | 33/157 [00:00<00:02, 45.15it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|███▋             | 34/157 [00:00<00:02, 45.32it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|███▊             | 35/157 [00:00<00:02, 45.67it/s]\u001b[A\n",
      "Validation DataLoader 0:  23%|███▉             | 36/157 [00:00<00:02, 45.98it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████             | 37/157 [00:00<00:02, 46.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████             | 38/157 [00:00<00:02, 46.43it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▏            | 39/157 [00:00<00:02, 46.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▎            | 40/157 [00:00<00:02, 46.35it/s]\u001b[A\n",
      "Validation DataLoader 0:  26%|████▍            | 41/157 [00:00<00:02, 46.12it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▌            | 42/157 [00:00<00:02, 46.29it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▋            | 43/157 [00:00<00:02, 46.52it/s]\u001b[A\n",
      "Validation DataLoader 0:  28%|████▊            | 44/157 [00:00<00:02, 46.78it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|████▊            | 45/157 [00:00<00:02, 46.91it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|████▉            | 46/157 [00:00<00:02, 46.89it/s]\u001b[A\n",
      "Validation DataLoader 0:  30%|█████            | 47/157 [00:01<00:02, 46.86it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▏           | 48/157 [00:01<00:02, 46.89it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▎           | 49/157 [00:01<00:02, 47.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▍           | 50/157 [00:01<00:02, 46.84it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▌           | 51/157 [00:01<00:02, 47.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  33%|█████▋           | 52/157 [00:01<00:02, 46.95it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|█████▋           | 53/157 [00:01<00:02, 46.80it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|█████▊           | 54/157 [00:01<00:02, 46.83it/s]\u001b[A\n",
      "Validation DataLoader 0:  35%|█████▉           | 55/157 [00:01<00:02, 46.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  36%|██████           | 56/157 [00:01<00:02, 46.93it/s]\u001b[A\n",
      "Validation DataLoader 0:  36%|██████▏          | 57/157 [00:01<00:02, 46.99it/s]\u001b[A\n",
      "Validation DataLoader 0:  37%|██████▎          | 58/157 [00:01<00:02, 47.14it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▍          | 59/157 [00:01<00:02, 47.27it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▍          | 60/157 [00:01<00:02, 47.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  39%|██████▌          | 61/157 [00:01<00:02, 47.37it/s]\u001b[A\n",
      "Validation DataLoader 0:  39%|██████▋          | 62/157 [00:01<00:02, 47.49it/s]\u001b[A\n",
      "Validation DataLoader 0:  40%|██████▊          | 63/157 [00:01<00:01, 47.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|██████▉          | 64/157 [00:01<00:01, 47.68it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████          | 65/157 [00:01<00:01, 47.79it/s]\u001b[A\n",
      "Validation DataLoader 0:  42%|███████▏         | 66/157 [00:01<00:01, 47.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▎         | 67/157 [00:01<00:01, 48.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▎         | 68/157 [00:01<00:01, 48.07it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|███████▍         | 69/157 [00:01<00:01, 47.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  45%|███████▌         | 70/157 [00:01<00:01, 48.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  45%|███████▋         | 71/157 [00:01<00:01, 48.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|███████▊         | 72/157 [00:01<00:01, 48.29it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|███████▉         | 73/157 [00:01<00:01, 48.36it/s]\u001b[A\n",
      "Validation DataLoader 0:  47%|████████         | 74/157 [00:01<00:01, 48.34it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████         | 75/157 [00:01<00:01, 48.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████▏        | 76/157 [00:01<00:01, 48.36it/s]\u001b[A\n",
      "Validation DataLoader 0:  49%|████████▎        | 77/157 [00:01<00:01, 48.36it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|████████▍        | 78/157 [00:01<00:01, 48.31it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|████████▌        | 79/157 [00:01<00:01, 48.42it/s]\u001b[A\n",
      "Validation DataLoader 0:  51%|████████▋        | 80/157 [00:01<00:01, 48.54it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|████████▊        | 81/157 [00:01<00:01, 48.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|████████▉        | 82/157 [00:01<00:01, 48.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  53%|████████▉        | 83/157 [00:01<00:01, 48.36it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████        | 84/157 [00:01<00:01, 48.38it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████▏       | 85/157 [00:01<00:01, 48.44it/s]\u001b[A\n",
      "Validation DataLoader 0:  55%|█████████▎       | 86/157 [00:01<00:01, 48.51it/s]\u001b[A\n",
      "Validation DataLoader 0:  55%|█████████▍       | 87/157 [00:01<00:01, 48.61it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|█████████▌       | 88/157 [00:01<00:01, 48.70it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|█████████▋       | 89/157 [00:01<00:01, 48.71it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|█████████▋       | 90/157 [00:01<00:01, 48.77it/s]\u001b[A\n",
      "Validation DataLoader 0:  58%|█████████▊       | 91/157 [00:01<00:01, 48.82it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|█████████▉       | 92/157 [00:01<00:01, 48.69it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████       | 93/157 [00:01<00:01, 48.73it/s]\u001b[A\n",
      "Validation DataLoader 0:  60%|██████████▏      | 94/157 [00:01<00:01, 48.78it/s]\u001b[A\n",
      "Validation DataLoader 0:  61%|██████████▎      | 95/157 [00:01<00:01, 48.82it/s]\u001b[A\n",
      "Validation DataLoader 0:  61%|██████████▍      | 96/157 [00:01<00:01, 48.87it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|██████████▌      | 97/157 [00:01<00:01, 48.82it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|██████████▌      | 98/157 [00:02<00:01, 48.85it/s]\u001b[A\n",
      "Validation DataLoader 0:  63%|██████████▋      | 99/157 [00:02<00:01, 48.87it/s]\u001b[A\n",
      "Validation DataLoader 0:  64%|██████████▏     | 100/157 [00:02<00:01, 48.95it/s]\u001b[A\n",
      "Validation DataLoader 0:  64%|██████████▎     | 101/157 [00:02<00:01, 49.03it/s]\u001b[A\n",
      "Validation DataLoader 0:  65%|██████████▍     | 102/157 [00:02<00:01, 49.11it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|██████████▍     | 103/157 [00:02<00:01, 49.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|██████████▌     | 104/157 [00:02<00:01, 49.12it/s]\u001b[A\n",
      "Validation DataLoader 0:  67%|██████████▋     | 105/157 [00:02<00:01, 49.11it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|██████████▊     | 106/157 [00:02<00:01, 49.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|██████████▉     | 107/157 [00:02<00:01, 49.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|███████████     | 108/157 [00:02<00:00, 49.16it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|███████████     | 109/157 [00:02<00:00, 49.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  70%|███████████▏    | 110/157 [00:02<00:00, 49.32it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|███████████▎    | 111/157 [00:02<00:00, 49.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|███████████▍    | 112/157 [00:02<00:00, 49.30it/s]\u001b[A\n",
      "Validation DataLoader 0:  72%|███████████▌    | 113/157 [00:02<00:00, 49.28it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|███████████▌    | 114/157 [00:02<00:00, 49.34it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|███████████▋    | 115/157 [00:02<00:00, 49.42it/s]\u001b[A\n",
      "Validation DataLoader 0:  74%|███████████▊    | 116/157 [00:02<00:00, 49.49it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|███████████▉    | 117/157 [00:02<00:00, 49.56it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|████████████    | 118/157 [00:02<00:00, 49.65it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|████████████▏   | 119/157 [00:02<00:00, 49.66it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|████████████▏   | 120/157 [00:02<00:00, 49.52it/s]\u001b[A\n",
      "Validation DataLoader 0:  77%|████████████▎   | 121/157 [00:02<00:00, 49.52it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|████████████▍   | 122/157 [00:02<00:00, 49.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|████████████▌   | 123/157 [00:02<00:00, 49.54it/s]\u001b[A\n",
      "Validation DataLoader 0:  79%|████████████▋   | 124/157 [00:02<00:00, 49.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  80%|████████████▋   | 125/157 [00:02<00:00, 49.48it/s]\u001b[A\n",
      "Validation DataLoader 0:  80%|████████████▊   | 126/157 [00:02<00:00, 49.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|████████████▉   | 127/157 [00:02<00:00, 49.09it/s]\u001b[A\n",
      "Validation DataLoader 0:  82%|█████████████   | 128/157 [00:02<00:00, 48.79it/s]\u001b[A\n",
      "Validation DataLoader 0:  82%|█████████████▏  | 129/157 [00:02<00:00, 48.49it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|█████████████▏  | 130/157 [00:02<00:00, 48.36it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|█████████████▎  | 131/157 [00:02<00:00, 48.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|█████████████▍  | 132/157 [00:02<00:00, 47.91it/s]\u001b[A\n",
      "Validation DataLoader 0:  85%|█████████████▌  | 133/157 [00:02<00:00, 47.84it/s]\u001b[A\n",
      "Validation DataLoader 0:  85%|█████████████▋  | 134/157 [00:02<00:00, 47.83it/s]\u001b[A\n",
      "Validation DataLoader 0:  86%|█████████████▊  | 135/157 [00:02<00:00, 47.85it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|█████████████▊  | 136/157 [00:02<00:00, 47.77it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|█████████████▉  | 137/157 [00:02<00:00, 47.82it/s]\u001b[A\n",
      "Validation DataLoader 0:  88%|██████████████  | 138/157 [00:02<00:00, 47.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|██████████████▏ | 139/157 [00:02<00:00, 47.93it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|██████████████▎ | 140/157 [00:02<00:00, 47.83it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|██████████████▎ | 141/157 [00:02<00:00, 47.84it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|██████████████▍ | 142/157 [00:02<00:00, 47.89it/s]\u001b[A\n",
      "Validation DataLoader 0:  91%|██████████████▌ | 143/157 [00:02<00:00, 47.92it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|██████████████▋ | 144/157 [00:03<00:00, 47.98it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|██████████████▊ | 145/157 [00:03<00:00, 48.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  93%|██████████████▉ | 146/157 [00:03<00:00, 48.07it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|██████████████▉ | 147/157 [00:03<00:00, 48.02it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|███████████████ | 148/157 [00:03<00:00, 47.97it/s]\u001b[A\n",
      "Validation DataLoader 0:  95%|███████████████▏| 149/157 [00:03<00:00, 48.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  96%|███████████████▎| 150/157 [00:03<00:00, 48.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  96%|███████████████▍| 151/157 [00:03<00:00, 48.12it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|███████████████▍| 152/157 [00:03<00:00, 48.17it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|███████████████▌| 153/157 [00:03<00:00, 48.15it/s]\u001b[A\n",
      "Validation DataLoader 0:  98%|███████████████▋| 154/157 [00:03<00:00, 48.14it/s]\u001b[A\n",
      "Validation DataLoader 0:  99%|███████████████▊| 155/157 [00:03<00:00, 48.12it/s]\u001b[A\n",
      "Validation DataLoader 0:  99%|███████████████▉| 156/157 [00:03<00:00, 48.18it/s]\u001b[A\n",
      "Epoch 3: 100%|████████████████████| 859/859 [00:50<00:00, 16.84it/s, v_num=oei5]\u001b[A\n",
      "Epoch 4: 100%|████████████████████| 859/859 [00:45<00:00, 18.81it/s, v_num=oei5]\u001b[A\n",
      "Validation: 0it [00:00, ?it/s]\u001b[A\n",
      "Validation:   0%|                                       | 0/157 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   0%|                          | 0/157 [00:00<?, ?it/s]\u001b[A\n",
      "Validation DataLoader 0:   1%|                  | 1/157 [00:00<00:04, 33.77it/s]\u001b[A\n",
      "Validation DataLoader 0:   1%|▏                 | 2/157 [00:00<00:03, 41.97it/s]\u001b[A\n",
      "Validation DataLoader 0:   2%|▎                 | 3/157 [00:00<00:03, 46.14it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▍                 | 4/157 [00:00<00:03, 47.33it/s]\u001b[A\n",
      "Validation DataLoader 0:   3%|▌                 | 5/157 [00:00<00:03, 48.61it/s]\u001b[A\n",
      "Validation DataLoader 0:   4%|▋                 | 6/157 [00:00<00:03, 48.48it/s]\u001b[A\n",
      "Validation DataLoader 0:   4%|▊                 | 7/157 [00:00<00:03, 49.95it/s]\u001b[A\n",
      "Validation DataLoader 0:   5%|▉                 | 8/157 [00:00<00:03, 49.56it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█                 | 9/157 [00:00<00:02, 50.35it/s]\u001b[A\n",
      "Validation DataLoader 0:   6%|█                | 10/157 [00:00<00:02, 50.87it/s]\u001b[A\n",
      "Validation DataLoader 0:   7%|█▏               | 11/157 [00:00<00:02, 50.99it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▎               | 12/157 [00:00<00:02, 51.49it/s]\u001b[A\n",
      "Validation DataLoader 0:   8%|█▍               | 13/157 [00:00<00:02, 51.59it/s]\u001b[A\n",
      "Validation DataLoader 0:   9%|█▌               | 14/157 [00:00<00:02, 50.28it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▌               | 15/157 [00:00<00:02, 50.72it/s]\u001b[A\n",
      "Validation DataLoader 0:  10%|█▋               | 16/157 [00:00<00:02, 50.99it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|█▊               | 17/157 [00:00<00:02, 51.11it/s]\u001b[A\n",
      "Validation DataLoader 0:  11%|█▉               | 18/157 [00:00<00:02, 51.36it/s]\u001b[A\n",
      "Validation DataLoader 0:  12%|██               | 19/157 [00:00<00:02, 51.58it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▏              | 20/157 [00:00<00:02, 51.05it/s]\u001b[A\n",
      "Validation DataLoader 0:  13%|██▎              | 21/157 [00:00<00:02, 49.31it/s]\u001b[A\n",
      "Validation DataLoader 0:  14%|██▍              | 22/157 [00:00<00:02, 48.61it/s]\u001b[A\n",
      "Validation DataLoader 0:  15%|██▍              | 23/157 [00:00<00:02, 48.10it/s]\u001b[A\n",
      "Validation DataLoader 0:  15%|██▌              | 24/157 [00:00<00:02, 46.78it/s]\u001b[A\n",
      "Validation DataLoader 0:  16%|██▋              | 25/157 [00:00<00:02, 45.65it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|██▊              | 26/157 [00:00<00:02, 44.82it/s]\u001b[A\n",
      "Validation DataLoader 0:  17%|██▉              | 27/157 [00:00<00:02, 45.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  18%|███              | 28/157 [00:00<00:02, 44.41it/s]\u001b[A\n",
      "Validation DataLoader 0:  18%|███▏             | 29/157 [00:00<00:02, 44.83it/s]\u001b[A\n",
      "Validation DataLoader 0:  19%|███▏             | 30/157 [00:00<00:02, 44.49it/s]\u001b[A\n",
      "Validation DataLoader 0:  20%|███▎             | 31/157 [00:00<00:02, 44.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  20%|███▍             | 32/157 [00:00<00:02, 43.68it/s]\u001b[A\n",
      "Validation DataLoader 0:  21%|███▌             | 33/157 [00:00<00:02, 43.72it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|███▋             | 34/157 [00:00<00:02, 44.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  22%|███▊             | 35/157 [00:00<00:02, 43.92it/s]\u001b[A\n",
      "Validation DataLoader 0:  23%|███▉             | 36/157 [00:00<00:02, 44.22it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████             | 37/157 [00:00<00:02, 44.31it/s]\u001b[A\n",
      "Validation DataLoader 0:  24%|████             | 38/157 [00:00<00:02, 44.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▏            | 39/157 [00:00<00:02, 44.62it/s]\u001b[A\n",
      "Validation DataLoader 0:  25%|████▎            | 40/157 [00:00<00:02, 44.87it/s]\u001b[A\n",
      "Validation DataLoader 0:  26%|████▍            | 41/157 [00:00<00:02, 45.03it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▌            | 42/157 [00:00<00:02, 44.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  27%|████▋            | 43/157 [00:00<00:02, 45.07it/s]\u001b[A\n",
      "Validation DataLoader 0:  28%|████▊            | 44/157 [00:00<00:02, 45.27it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|████▊            | 45/157 [00:00<00:02, 45.49it/s]\u001b[A\n",
      "Validation DataLoader 0:  29%|████▉            | 46/157 [00:01<00:02, 45.70it/s]\u001b[A\n",
      "Validation DataLoader 0:  30%|█████            | 47/157 [00:01<00:02, 45.77it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▏           | 48/157 [00:01<00:02, 45.95it/s]\u001b[A\n",
      "Validation DataLoader 0:  31%|█████▎           | 49/157 [00:01<00:02, 46.09it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▍           | 50/157 [00:01<00:02, 46.12it/s]\u001b[A\n",
      "Validation DataLoader 0:  32%|█████▌           | 51/157 [00:01<00:02, 46.28it/s]\u001b[A\n",
      "Validation DataLoader 0:  33%|█████▋           | 52/157 [00:01<00:02, 46.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|█████▋           | 53/157 [00:01<00:02, 46.26it/s]\u001b[A\n",
      "Validation DataLoader 0:  34%|█████▊           | 54/157 [00:01<00:02, 46.38it/s]\u001b[A\n",
      "Validation DataLoader 0:  35%|█████▉           | 55/157 [00:01<00:02, 46.59it/s]\u001b[A\n",
      "Validation DataLoader 0:  36%|██████           | 56/157 [00:01<00:02, 46.79it/s]\u001b[A\n",
      "Validation DataLoader 0:  36%|██████▏          | 57/157 [00:01<00:02, 46.68it/s]\u001b[A\n",
      "Validation DataLoader 0:  37%|██████▎          | 58/157 [00:01<00:02, 46.68it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▍          | 59/157 [00:01<00:02, 46.80it/s]\u001b[A\n",
      "Validation DataLoader 0:  38%|██████▍          | 60/157 [00:01<00:02, 46.99it/s]\u001b[A\n",
      "Validation DataLoader 0:  39%|██████▌          | 61/157 [00:01<00:02, 47.16it/s]\u001b[A\n",
      "Validation DataLoader 0:  39%|██████▋          | 62/157 [00:01<00:02, 47.28it/s]\u001b[A\n",
      "Validation DataLoader 0:  40%|██████▊          | 63/157 [00:01<00:01, 47.45it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|██████▉          | 64/157 [00:01<00:01, 47.57it/s]\u001b[A\n",
      "Validation DataLoader 0:  41%|███████          | 65/157 [00:01<00:01, 47.66it/s]\u001b[A\n",
      "Validation DataLoader 0:  42%|███████▏         | 66/157 [00:01<00:01, 47.69it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▎         | 67/157 [00:01<00:01, 47.79it/s]\u001b[A\n",
      "Validation DataLoader 0:  43%|███████▎         | 68/157 [00:01<00:01, 47.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  44%|███████▍         | 69/157 [00:01<00:01, 47.95it/s]\u001b[A\n",
      "Validation DataLoader 0:  45%|███████▌         | 70/157 [00:01<00:01, 47.94it/s]\u001b[A\n",
      "Validation DataLoader 0:  45%|███████▋         | 71/157 [00:01<00:01, 48.03it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|███████▊         | 72/157 [00:01<00:01, 48.15it/s]\u001b[A\n",
      "Validation DataLoader 0:  46%|███████▉         | 73/157 [00:01<00:01, 48.24it/s]\u001b[A\n",
      "Validation DataLoader 0:  47%|████████         | 74/157 [00:01<00:01, 48.20it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████         | 75/157 [00:01<00:01, 48.16it/s]\u001b[A\n",
      "Validation DataLoader 0:  48%|████████▏        | 76/157 [00:01<00:01, 48.27it/s]\u001b[A\n",
      "Validation DataLoader 0:  49%|████████▎        | 77/157 [00:01<00:01, 48.34it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|████████▍        | 78/157 [00:01<00:01, 48.34it/s]\u001b[A\n",
      "Validation DataLoader 0:  50%|████████▌        | 79/157 [00:01<00:01, 48.46it/s]\u001b[A\n",
      "Validation DataLoader 0:  51%|████████▋        | 80/157 [00:01<00:01, 48.48it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|████████▊        | 81/157 [00:01<00:01, 48.21it/s]\u001b[A\n",
      "Validation DataLoader 0:  52%|████████▉        | 82/157 [00:01<00:01, 47.89it/s]\u001b[A\n",
      "Validation DataLoader 0:  53%|████████▉        | 83/157 [00:01<00:01, 47.87it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████        | 84/157 [00:01<00:01, 47.97it/s]\u001b[A\n",
      "Validation DataLoader 0:  54%|█████████▏       | 85/157 [00:01<00:01, 47.89it/s]\u001b[A\n",
      "Validation DataLoader 0:  55%|█████████▎       | 86/157 [00:01<00:01, 47.84it/s]\u001b[A\n",
      "Validation DataLoader 0:  55%|█████████▍       | 87/157 [00:01<00:01, 47.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  56%|█████████▌       | 88/157 [00:01<00:01, 47.81it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|█████████▋       | 89/157 [00:01<00:01, 47.80it/s]\u001b[A\n",
      "Validation DataLoader 0:  57%|█████████▋       | 90/157 [00:01<00:01, 47.79it/s]\u001b[A\n",
      "Validation DataLoader 0:  58%|█████████▊       | 91/157 [00:01<00:01, 47.88it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|█████████▉       | 92/157 [00:01<00:01, 47.95it/s]\u001b[A\n",
      "Validation DataLoader 0:  59%|██████████       | 93/157 [00:01<00:01, 48.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  60%|██████████▏      | 94/157 [00:01<00:01, 47.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  61%|██████████▎      | 95/157 [00:01<00:01, 47.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  61%|██████████▍      | 96/157 [00:02<00:01, 47.69it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|██████████▌      | 97/157 [00:02<00:01, 47.78it/s]\u001b[A\n",
      "Validation DataLoader 0:  62%|██████████▌      | 98/157 [00:02<00:01, 47.83it/s]\u001b[A\n",
      "Validation DataLoader 0:  63%|██████████▋      | 99/157 [00:02<00:01, 47.92it/s]\u001b[A\n",
      "Validation DataLoader 0:  64%|██████████▏     | 100/157 [00:02<00:01, 47.93it/s]\u001b[A\n",
      "Validation DataLoader 0:  64%|██████████▎     | 101/157 [00:02<00:01, 47.96it/s]\u001b[A\n",
      "Validation DataLoader 0:  65%|██████████▍     | 102/157 [00:02<00:01, 47.97it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|██████████▍     | 103/157 [00:02<00:01, 47.98it/s]\u001b[A\n",
      "Validation DataLoader 0:  66%|██████████▌     | 104/157 [00:02<00:01, 48.02it/s]\u001b[A\n",
      "Validation DataLoader 0:  67%|██████████▋     | 105/157 [00:02<00:01, 48.04it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|██████████▊     | 106/157 [00:02<00:01, 48.13it/s]\u001b[A\n",
      "Validation DataLoader 0:  68%|██████████▉     | 107/157 [00:02<00:01, 48.24it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|███████████     | 108/157 [00:02<00:01, 48.32it/s]\u001b[A\n",
      "Validation DataLoader 0:  69%|███████████     | 109/157 [00:02<00:00, 48.33it/s]\u001b[A\n",
      "Validation DataLoader 0:  70%|███████████▏    | 110/157 [00:02<00:00, 48.33it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|███████████▎    | 111/157 [00:02<00:00, 48.40it/s]\u001b[A\n",
      "Validation DataLoader 0:  71%|███████████▍    | 112/157 [00:02<00:00, 48.48it/s]\u001b[A\n",
      "Validation DataLoader 0:  72%|███████████▌    | 113/157 [00:02<00:00, 48.47it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|███████████▌    | 114/157 [00:02<00:00, 48.55it/s]\u001b[A\n",
      "Validation DataLoader 0:  73%|███████████▋    | 115/157 [00:02<00:00, 48.67it/s]\u001b[A\n",
      "Validation DataLoader 0:  74%|███████████▊    | 116/157 [00:02<00:00, 48.75it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|███████████▉    | 117/157 [00:02<00:00, 48.82it/s]\u001b[A\n",
      "Validation DataLoader 0:  75%|████████████    | 118/157 [00:02<00:00, 48.82it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|████████████▏   | 119/157 [00:02<00:00, 48.91it/s]\u001b[A\n",
      "Validation DataLoader 0:  76%|████████████▏   | 120/157 [00:02<00:00, 48.93it/s]\u001b[A\n",
      "Validation DataLoader 0:  77%|████████████▎   | 121/157 [00:02<00:00, 49.01it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|████████████▍   | 122/157 [00:02<00:00, 49.09it/s]\u001b[A\n",
      "Validation DataLoader 0:  78%|████████████▌   | 123/157 [00:02<00:00, 49.18it/s]\u001b[A\n",
      "Validation DataLoader 0:  79%|████████████▋   | 124/157 [00:02<00:00, 49.26it/s]\u001b[A\n",
      "Validation DataLoader 0:  80%|████████████▋   | 125/157 [00:02<00:00, 49.30it/s]\u001b[A\n",
      "Validation DataLoader 0:  80%|████████████▊   | 126/157 [00:02<00:00, 49.21it/s]\u001b[A\n",
      "Validation DataLoader 0:  81%|████████████▉   | 127/157 [00:02<00:00, 49.24it/s]\u001b[A\n",
      "Validation DataLoader 0:  82%|█████████████   | 128/157 [00:02<00:00, 49.29it/s]\u001b[A\n",
      "Validation DataLoader 0:  82%|█████████████▏  | 129/157 [00:02<00:00, 49.34it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|█████████████▏  | 130/157 [00:02<00:00, 49.34it/s]\u001b[A\n",
      "Validation DataLoader 0:  83%|█████████████▎  | 131/157 [00:02<00:00, 49.34it/s]\u001b[A\n",
      "Validation DataLoader 0:  84%|█████████████▍  | 132/157 [00:02<00:00, 49.37it/s]\u001b[A\n",
      "Validation DataLoader 0:  85%|█████████████▌  | 133/157 [00:02<00:00, 49.13it/s]\u001b[A\n",
      "Validation DataLoader 0:  85%|█████████████▋  | 134/157 [00:02<00:00, 49.16it/s]\u001b[A\n",
      "Validation DataLoader 0:  86%|█████████████▊  | 135/157 [00:02<00:00, 49.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|█████████████▊  | 136/157 [00:02<00:00, 49.23it/s]\u001b[A\n",
      "Validation DataLoader 0:  87%|█████████████▉  | 137/157 [00:02<00:00, 49.25it/s]\u001b[A\n",
      "Validation DataLoader 0:  88%|██████████████  | 138/157 [00:02<00:00, 49.31it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|██████████████▏ | 139/157 [00:02<00:00, 49.34it/s]\u001b[A\n",
      "Validation DataLoader 0:  89%|██████████████▎ | 140/157 [00:02<00:00, 49.19it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|██████████████▎ | 141/157 [00:02<00:00, 49.21it/s]\u001b[A\n",
      "Validation DataLoader 0:  90%|██████████████▍ | 142/157 [00:02<00:00, 49.27it/s]\u001b[A\n",
      "Validation DataLoader 0:  91%|██████████████▌ | 143/157 [00:02<00:00, 49.34it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|██████████████▋ | 144/157 [00:02<00:00, 49.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  92%|██████████████▊ | 145/157 [00:02<00:00, 49.34it/s]\u001b[A\n",
      "Validation DataLoader 0:  93%|██████████████▉ | 146/157 [00:02<00:00, 49.38it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|██████████████▉ | 147/157 [00:02<00:00, 49.42it/s]\u001b[A\n",
      "Validation DataLoader 0:  94%|███████████████ | 148/157 [00:02<00:00, 49.37it/s]\u001b[A\n",
      "Validation DataLoader 0:  95%|███████████████▏| 149/157 [00:03<00:00, 49.39it/s]\u001b[A\n",
      "Validation DataLoader 0:  96%|███████████████▎| 150/157 [00:03<00:00, 49.42it/s]\u001b[A\n",
      "Validation DataLoader 0:  96%|███████████████▍| 151/157 [00:03<00:00, 49.50it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|███████████████▍| 152/157 [00:03<00:00, 49.57it/s]\u001b[A\n",
      "Validation DataLoader 0:  97%|███████████████▌| 153/157 [00:03<00:00, 49.60it/s]\u001b[A\n",
      "Validation DataLoader 0:  98%|███████████████▋| 154/157 [00:03<00:00, 49.67it/s]\u001b[A\n",
      "Validation DataLoader 0:  99%|███████████████▊| 155/157 [00:03<00:00, 49.73it/s]\u001b[A\n",
      "Validation DataLoader 0:  99%|███████████████▉| 156/157 [00:03<00:00, 49.76it/s]\u001b[A\n",
      "Epoch 4: 100%|████████████████████| 859/859 [00:49<00:00, 17.52it/s, v_num=oei5]\u001b[A\n",
      "Epoch 4: 100%|████████████████████| 859/859 [00:49<00:00, 17.52it/s, v_num=oei5]\u001b[A"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "`Trainer.fit` stopped: `max_epochs=5` reached.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 4: 100%|████████████████████| 859/859 [00:49<00:00, 17.51it/s, v_num=oei5]\n"
     ]
    }
   ],
   "source": [
    "trainer.fit(model, train_dataloader, val_dataloader)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "233f78e0-9cf9-4e60-9550-461796f5b5cc",
   "metadata": {},
   "outputs": [],
   "source": [
    "#trainer.test()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "39a81509-1305-445c-9b9c-68e66cd0c8d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "wandb.finish()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}