ML4KG-tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# you need to install pykeen beforehand, see https://pykeen.readthedocs.io/en/stable/installation.html \n",
    "import os\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import pykeen\n",
    "import seaborn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Dataset exploration\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PyKeen comes with its own datasets that can be used directly in a pipeline.\n",
    "Below we import it so that we can explore it later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ritten/.pyenv/versions/3.10.13/envs/KRW_tutorials-3.10.13/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    }
   ],
   "source": [
    "from pykeen.datasets import Nations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, we want to be able tfo work with our own datasets as well, so we etch the online GoT dataset as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Reconstructing all label-based triples. This is expensive and rarely needed.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([['Abelar Hightower', 'ALLIED_WITH',\n",
       "        'House Hightower of the Hightower'],\n",
       "       ['Acorn Hall', 'SEAT_OF', 'House Smallwood of Acorn Hall'],\n",
       "       ['Addam Frey', 'ALLIED_WITH', 'House Frey of the Crossing'],\n",
       "       ['Addam Marbrand', 'ALLIED_WITH', 'House Marbrand of Ashemark'],\n",
       "       ['Addam Osgrey', 'ALLIED_WITH', 'House Osgrey of Standfast']],\n",
       "      dtype='<U44')"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import requests\n",
    "from pykeen import triples\n",
    "\n",
    "url = 'https://ampligraph.s3-eu-west-1.amazonaws.com/datasets/GoT.csv'\n",
    "open('GoT.csv', 'wb').write(requests.get(url).content)\n",
    "\n",
    "# Format that can be read by a pd.from_csv should also be able to be read here, but the delimiter needs to be adjusted\n",
    "# PyKEEN uses tabs as defaults\n",
    "got = triples.TriplesFactory.from_path('GoT.csv',load_triples_kwargs=dict(delimiter=','))\n",
    "got_triples = got.triples\n",
    "got_triples[:5,]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 1\n",
    "\n",
    "List the unique subject and object entities found in the dataset. Then list all of the relationships that link the entities (note that some entities are not linked). Create an RDF version of the dataset, using your own namespaces, and save is as a ttl file. \n",
    "\n",
    "Using SPARQL, answer the following questions : \n",
    "1. How many instances per class? Use ORDER BY to show the most popular class\n",
    "2. What is the most common relation per each class?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "### your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Defining train and test datasets\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As is typical in machine learning, we need to split our dataset into training and test (and sometimes validation) datasets.\n",
    "\n",
    "What differs from the standard method of randomly sampling N points to make up our test set, is that our data points are two entities linked by some relationship, and we need to take care to ensure that all entities are represented in train and test sets by at least one triple.\n",
    "\n",
    "To accomplish this, PyKEEN provides the <b>pykeen.triples.TriplesFactory.split()</b> function, which defaults to an 80/20 split. It is also by default stratified, to ensure that the distribution of the test set corresponds to that of the training set. If you want to use early stopping, you will also need a validation set. The function takes a list of percentages as argument: if you want a 95/5 split you give it <b>[0.95,0.05]</b> as argument, if you want 90/5/5 (which would include a validation set as well) you give it <b>[0.9,0.05,0.05]</b> as argument and it will return 3 datasets.\n",
    "\n",
    "For sake of example, we will create a small test size that includes only 5% of triples. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "using automatically assigned random_state=2389664012\n",
      "Reconstructing all label-based triples. This is expensive and rarely needed.\n",
      "Reconstructing all label-based triples. This is expensive and rarely needed.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train set size:  (3016, 3)\n",
      "Test set size:  (159, 3)\n"
     ]
    }
   ],
   "source": [
    "# got_training, got_testing = got.split()\n",
    "got_training, got_testing = got.split([0.95,0.05])\n",
    "\n",
    "print('Train set size: ', got_training.triples.shape)\n",
    "print('Test set size: ', got_testing.triples.shape)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 2\n",
    "\n",
    "Create three train-test sets of different sizes from the GoT data. Give them different names. Make sure the test set is not too big when compared to the training set (test set should be max 15% of the total dataset)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "### your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Training and testing the model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PyKEEN has implemented several Knoweldge Graph Embedding models (TransE, ComplEx, DistMult, HolE, etc.). We will use the ComplEx model with default values for this tutorial.\n",
    "\n",
    "You can find the list of all implemented models in the documentation: https://pykeen.readthedocs.io/en/stable/reference/models.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Importing a model and instantiate it:\n",
    "There are two ways to import and use a model, both are shown below and don't give different results but not importing the model before hand might cause the automatic importing to be slower, especially if you plan to use the same model multiple times."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training epochs on cuda:0:   0%|                                                               | 0/5 [00:00<?, ?epoch/s]\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  20%|█████▊                       | 1/5 [00:00<00:00,  7.01epoch/s, loss=16.1, prev_loss=nan]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  40%|███████████▏                | 2/5 [00:00<00:00,  7.32epoch/s, loss=16.1, prev_loss=16.1]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  60%|████████████████▊           | 3/5 [00:00<00:00,  7.27epoch/s, loss=15.7, prev_loss=16.1]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  80%|██████████████████████▍     | 4/5 [00:00<00:00,  6.77epoch/s, loss=14.9, prev_loss=15.7]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0: 100%|████████████████████████████| 5/5 [00:00<00:00,  6.94epoch/s, loss=14.7, prev_loss=14.9]\u001b[A\n",
      "INFO:pykeen.evaluation.evaluator:Starting batch_size search for evaluation now...\n",
      "INFO:pykeen.evaluation.evaluator:Concluded batch_size search with batch_size=159.\n",
      "Evaluating on cuda:0: 100%|███████████████████████████████████████████████████████| 159/159 [00:00<00:00, 2.96ktriple/s]\n",
      "INFO:pykeen.evaluation.evaluator:Evaluation took 0.06s seconds\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<Axes: title={'center': 'Losses Plot'}, xlabel='Epoch', ylabel='marginranking Loss'>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAHHCAYAAABXx+fLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABcCUlEQVR4nO3deXhM9/4H8Pdkksm+ySKyiCwytiCxRyKWKKVq62Zvq1RpL7qht4v1hhZVS0tRSlB+qFqqaqnaYilCQmQPiSyyIatEcn5/qGkjEZlkJmeW9+t55rmZM2e+8/nck0nfzvmecySCIAggIiIi0iMGYhdARERE1NAYgIiIiEjvMAARERGR3mEAIiIiIr3DAERERER6hwGIiIiI9A4DEBEREekdBiAiIiLSOwxAREREpHcYgIiI1GT37t2Qy+VITU0VuxQiegIDEBHVy+P/yEdGRopdSoNZsWIF5HK54tGuXTsMGDAAX3/9NQoKClTyGfv27cPGjRtVMhYRVWUodgFERNpq9uzZMDMzQ1FREU6fPo3Vq1fj3Llz2LZtGyQSSb3G3r9/P+Li4vD666+rplgiqoQBiIiojvr164dGjRoBAEaMGIH33nsPv//+OyIiIuDn5ydydURUEx4CI6IGcf36dbz11lvw9/eHn58fxo0bh4iIiErrlJWVYeXKlXjuuefg6+uLLl26YMSIETh9+rRinaysLMyaNQs9evRAmzZtEBgYiHfeeafKPJs///wTI0eORPv27eHn54eJEyciLi6u0jq1Hau2unbtCgDPfP+WLVswcOBAxWfOmTMH9+/fV7w+ZswYHD9+HLdv31YcZuvdu3edaiKi6nEPEBGpXVxcHEaNGgVzc3O89dZbMDQ0xPbt2zFmzBiEhYWhXbt2AICVK1dizZo1ePnll9G2bVsUFBQgKioK165dQ/fu3QEA7733HuLj4zF69Gi4uLggNzcXp0+fRnp6OlxdXQEAe/bswcyZMxEYGIgPP/wQxcXF2LZtG0aOHImff/5ZsV5txlLGrVu3AAA2NjZPXWfFihVYuXIlAgICMGLECCQlJWHbtm2IjIzEtm3bYGRkhEmTJiE/Px8ZGRmYNWsWAMDc3FzpeoioBgIRUT3s2rVL8PHxEa5evfrUdSZPniy0bt1auHXrlmJZZmam4OfnJ4waNUqx7MUXXxQmTpz41HHu3bsn+Pj4COvWrXvqOgUFBULHjh2FTz/9tNLyrKwsoUOHDorltRnraZYvXy74+PgIiYmJQk5OjpCSkiL89NNPQps2bYSAgAChqKhIEIR//r9JSUkRBEEQcnJyhNatWwtvvvmmUF5erhgvLCxM8PHxEXbu3KlYNnHiRKFXr15K10ZEtcNDYESkVuXl5Th9+jRCQkLg5uamWO7o6IgXXngBFy9eVJw5ZWVlhbi4OCQnJ1c7lomJCYyMjHD+/Hncu3ev2nXOnDmD+/fvY+DAgcjNzVU8DAwM0K5dO5w7d67WYz1L//790a1bN/Tp0weff/453N3dsWbNGpiamj61trKyMowdOxYGBv/8+X355ZdhYWGBP//8s051EJHyeAiMiNQqNzcXxcXF8PDwqPKal5cXKioqkJ6ejubNm+M///kPJk+ejH79+sHHxweBgYEYPHgwWrRoAQCQyWT48MMPsWjRInTv3h3t2rVDz549MWTIEDg4OACAIjyNGzeu2nosLCxqPdazrFixAhYWFjA0NISTkxOaNm1a4/ppaWkAAE9Pz0rLZTIZ3NzccPv27Vp9LhHVHwMQEWmMTp064fDhwzh69ChOnz6NnTt34scff8ScOXPw8ssvAwBef/119O7dG0eOHMGpU6fwzTff4Pvvv8ePP/6IVq1aQRAEAMCXX35ZbZCRSqWKn5811rN07NhRcRYYEWkXHgIjIrVq1KgRTE1NkZSUVOW1xMREGBgYoEmTJoplNjY2GD58OJYuXYrjx49DLpdjxYoVld7XtGlTvPnmm/jhhx+wf/9+lJWV4YcffgAAxWE2Ozs7BAQEVHl06dKl1mOpmrOzs6LvfystLUVqaipcXFwUy+p7HSEiqhkDEBGplVQqRffu3XH06NFKp4dnZ2dj//796NChg+KwVF5eXqX3mpubo2nTpigtLQUAFBcX48GDB5XWadq0KczNzRXrBAUFwcLCAmvWrEFZWVmVenJzc2s9lqoFBATAyMgImzdvVuypAoCdO3ciPz8fwcHBimWmpqbIz89XSx1ExENgRKQiu3btwsmTJ6ssHzt2LKZNm4YzZ85g5MiRGDlyJKRSKbZv347S0lJ89NFHinUHDhyIzp07o3Xr1rCxsUFkZCQOHTqE0aNHA3g0v+f1119H//794e3tDalUiiNHjiA7OxsDBw4E8GiOz+zZs/Hxxx9j2LBhGDBgABo1aoS0tDT8+eef8Pf3x+eff16rsVStUaNGePvtt7Fy5Uq89dZb6N27N5KSkrB161b4+vrixRdfVKzbunVr/PrrrwgNDYWvry/MzMx4LSAiFWIAIiKV2LZtW7XLhw0bhubNm2PLli1YsmQJ1qxZA0EQ0LZtW3z11VeKawABjy4AeOzYMZw+fRqlpaVwdnbGtGnTMH78eACAk5MTBg4ciPDwcOzduxdSqRSenp5YtmwZ+vXrpxhn0KBBcHR0xPfff4/169ejtLQUjRs3RseOHTFs2DClxlK19957D40aNUJYWBhCQ0NhbW2NV155Be+//z6MjIwU640cORLR0dHYvXs3Nm7cCBcXFwYgIhWSCP/eD0tERESkBzgHiIiIiPQOAxARERHpHQYgIiIi0jsMQERERKR3GICIiIhI7zAAERERkd5hACIiIiK9wwBEREREeodXgq5BTk4+VH2ZSIkEsLOzVMvYmoD9aT9d71HX+wN0v0f2p/3U1ePjcWuDAagGggC1/fKpc2xNwP60n673qOv9AbrfI/vTfmL2yENgREREpHcYgIiIiEjvMAARERGR3mEAIiIiIr3DAERERER6hwGIiIiI9A4DEBEREekdBiAiIiLSOwxAREREpHcYgIiIiEjvMAARERGR3mEAIiIiIr3DANTAikvLUVJWLnYZREREeo13g29AiTmFGLXpEh5WCHC0kMHF2gTONqZwsTKBi40JXKwfPezMZZBIJGKXS0REpLMYgBqQqZEUDhYypN9/gDsFpbhTUIrLt+9XWc/Y0ADO1v8Eokc/mypCkqmRVITqiYiIdAcDUANqYmWCvRM6w9DMBFcTs5F6txi375UoHml3i5GZ/wAPHlYgKacISTlF1Y7TyMzon2BkY6oISi7WJnCwMIbUgHuPiIiIasIA1MAkEglszWVo3cQSrZwsq7z+sLwCGfkP/glGd0uQdu9RUEq7V4J7JQ+RW1SG3KIyRKbnV3m/kVSCJlYmlfYg/TskWRhzkxMREfG/hhrGUGoAVxtTuNqYVvt6fslDpN0rwe17T+w9+vtRVi7gVl4xbuUVV/t+axPDKofUHoUkEzS2NIEh9x4REZEeYADSMpYmhpCbWEDe2KLKa+UVArIKHij2HN3+156j2/dKkFtUhnslD3GvpADRmQVV3i+VAI2tTCodUnOxMVXsTbI2MeTkbCIi0gkMQDpEaiCBk5UJnKxM0MGt6utFpeWV9h6l/fsw2/0SPHhYodiTdKGa8c1l0iqH1B6HoyZWJpAZ8qoKRESkHRiA9IiZTApvB3N4O5hXea1CEJBTWPr3nqOSKofZsgpKUVhajtisQsRmFVZ5vwSAo6UxXKxN4NXYEnYm0n8OtVmboJGZEfceERGRxmAAIgCAgUQCBwtjOFgYo72rdZXXS8rKkX7/AW7fK6605+j230GpuKwCmfkPkJn/AJdS71V5v4mhwd9zjkyfmKBtAmcrE5jw1H4iImpADEBUKyZGUnjYmcHDzqzKa4IgIK+4THEo7W5ZBWLT7ilCUmb+A5Q8rEBCdhESsqs/td/eXFbpkNrjsORibQJ7CxkMuPeIiIhUiAGI6k0ikaCRmQyNzGRo62IFe3tLZGfnQxAevV5WXoGMv/ceVd5z9GjvUcGDcmQXliK7sBRX0qpeGFImlcD53xeE/PfeI2sTmMv4a0xERMrhfzlI7YykBnCzNYWbbfWn9t8vKatySO3xzxn5D1BaLiA5txjJucUA8qq839bU6Ik9R/+c5u/IC0MSEVE1GIBIdFYmRrAyMULLxtVcGLJCwJ38B5VC0b+vfXS3uAx5fz+uZVS9MKTUQIImVsb/hKJ/hSRnaxNYmRg1RItERKRhGIBIoxka/HP4q1PTqq8XPHj4z6TseyW4ffefU/zT7j+6MGTq3RKk3i0BcLfK+y2NDZ/Yc/TPRG0nK2MYSXlqPxGRLmIAIq1mYWwIH0cL+DhWvTBkhSAgq6C0yt6jx4Epp7AU+Q8e4sadAty4U/XCkAYSoLGlcaVDas7WJvDzqoCjkQSPTv4nIiJtxABEOstAIkFjS2M0tjSGv2vV14vLyqvsPfr38wcPK5B+/wHS7z/AXymVT+3v42OPT5/z4b3ViIi0FP96k94yNZLCy94cXvZVLwwpCAJyisoqHVJ7/L9X0+7jaGw2Yu4UYOELraq9LQkREWk2BiCiakgkEtiby2BvLkM7F+t/LQdSih7inc0XkXq3BG9uu4z3e3lhWNsmvNI1EZEW4QxPIiX5NbXFlrH+CPRshNJyAQuPxOOzX2+gsPSh2KUREVEtMQAR1YG1qRGWDGmN//TwgFQCHLqRhbFhlxFfzX3SiIhI8zAAEdWRgUSCMZ3csObVdnC0kOFWXjFe33oZeyMzIDy+DDYREWkkBiCiemrnYo2wMf7o2swWDx5WYN7vsZjzWwyKy8rFLo2IiJ6CAYhIBWzNZPhmWBtMDmwGAwlw4PodjNtyGYk5PCRGRKSJGICIVMRAIsEbXZri25fbws5chqScIowLu4xfr2eKXRoRET2BAYhIxTq42WDLGH90amqDkocV+OJgDOYfikUJD4kREWkMBiAiNbAzl2HFcF9M7OYOCYBfojLw5rYI3MwtErs0IiKCyAHowoULmDRpEgIDAyGXy3HkyJEq6yQkJGDSpEno0KED2rdvj+HDhyMtLa3GcQ8ePIj+/fvD19cXgwYNwp9//qmuFoieSmogwYQAd6x4yReNzIwQl1WIsWGX8fuNO2KXRkSk90QNQEVFRZDL5fjiiy+qff3WrVsYOXIkPD09sXnzZuzduxeTJ0+GsbHxU8e8dOkSPvjgA7z00kvYs2cP+vTpgylTpiA2NlZdbRDVqIu7LcLG+MPf1RpFZeX474EbWHgkDg8eVohdGhGR3hL1VhjBwcEIDg5+6utff/01evTogY8//lixrGnTpjWOuWnTJgQFBeGtt94CAEybNg1nzpxBWFgY5s6dq5rCiZTkYGGMVS+3xfdnkrHhXAp2XUlHVHo+Fg5qCVcbU7HLIyLSOxo7B6iiogLHjx9Hs2bNMH78eHTr1g0vv/xytYfJ/i0iIgLdunWrtCwwMBARERFqrJbo2QwNJJgc6IFvhrWBtYkhYu4UYPTmSzgWmyV2aUREekdjb4aak5ODoqIirF27FtOmTcOHH36IkydP4t1338WmTZvQuXPnat+XnZ0Ne3v7Ssvs7OyQnZ2tdA3quLfl4zF19b6Z7O/Zuns2wpax/vhk/w1cTbuPGfui8Zr/PUwN9oSRVPx/k3Abaj9d75H9aT919ajMeBobgCoqHs2P6NOnD15//XUAQMuWLXHp0iX89NNPTw1AqmRnZ6mVY2sC9lcze3tL7Jpih8WHYrDmRCJ+upSG6DuFWDnSH26NzFRUZf1wG2o/Xe+R/Wk/MXvU2ABka2sLQ0NDeHl5VVru5eWFixcvPvV99vb2Vfb25OTkVNkrVBs5OflQ9S2dJJJHG1wdY2sC9qecCZ1dIbczxeyDMbiSeg8DvjmJ2c/LEextV//B64jbUPvpeo/sT/upq8fH49aGxgYgmUwGX19fJCUlVVqenJwMFxeXp76vffv2OHv2rGKvEQCcOXMG7du3V7oGQYDafvnUObYmYH+1F+Rph7Ax/pi1LxrXMvLxwZ5rGNXBFe8GNYOhiIfEuA21n673yP60n5g9ijrhoLCwENHR0YiOjgYApKamIjo6WnGdn/Hjx+PgwYPYsWMHbt68ibCwMPzxxx8YMWKEYoyPP/4YS5YsUTwfO3YsTp48iR9++AEJCQlYsWIFoqKiMHr06IZtjkgJTaxMsPa1dhjh/yjcb7mYionbryLjfonIlRER6SZRA1BUVBSGDBmCIUOGAABCQ0MxZMgQLF++HADQt29fzJ49G+vWrcOgQYPwf//3f1i+fDk6duyoGCM9PR1ZWf+cRePv74/Fixdj+/btGDx4MA4dOoRVq1bBx8enQXsjUpaR1ADv9/LCohdbwcJYisj0+xi9+RJOJ+WKXRoRkc6RCIKu72Cru+xs9cwBsre3VMvYmoD9qUbq3WLM2heNG3cKAADjOrthUvdmMDRQ/2kh3IbaT9d7ZH/aT109Ph63NsQ/55aIqnC1McW6Ee3xUrsmAIAfz6dg8v9dRVbBA5ErIyLSDQxARBrK2NAAM0KaY8HAFjCXSXE59R5GbbqEc8l5YpdGRKT1GICINNxzLRzx4yg/NHcwR15xGd7bFYk1p5NRXqGj+8aJiBoAAxCRFnBvZIYfRrTHEF8nCADWnb2Fd3dFIruwVOzSiIi0EgMQkZYwMZLiv8/5YM7zcpgYGuCvW3cxevMl/HXrrtilERFpHQYgIi0zoFVjbBrtD087M+QUlmLKzqtYf/YmKnT1dBEiIjVgACLSQh52Ztg4yg8vtG6MCgFYffompu6KQl4RD4kREdUGAxCRljI1kuKL/nJ81s8HxoYGOHszD6M3X0JE6j2xSyMi0ngMQERa7sU2Ttg4yg/utqa4U1CKSTuu4MfzKTwkRkRUAwYgIh3gbW+OTaP90a+FA8oFYOXJJHyw5xruFpeJXRoRkUZiACLSEWYyKeYNaIFZfZtDJpXgVGIuRm++hMi0+2KXRkSkcRiAiHSIRCLBsLZN8MNIP7jZmCAz/wEmbL+CrRdTwdv+ERH9gwGISAfJHS2wabQ/QnzsUV4h4Ovjifh473XcL+EhMSIigAGISGdZGBvify+0xEe9vWEkleB4fA7GbL6E6xn5YpdGRCQ6BiAiHSaRSPCKnzPWvdYeztYmSLv/AG/9FIEdl2/zkBgR6TUGICI90MrJEmGj/dHT2w5l5QK+OpaAT/ZHo+DBQ7FLIyISBQMQkZ6wNDHEly+2wvSenpAaSHAkNhtjwy4h5k6B2KURETU4BiAiPSKRSDCygyvWvtoOTpbGSLlbgje3Xsbuq+k8JEZEeoUBiEgP+TpbIWyMPwI9G6G0XEDo4Th89usNFJWWi10aEVGDYAAi0lPWpkZYMqQ13gvygFQCHLqRhXFbLiE+q1Ds0oiI1I4BiEiPGUgkGNvZDatfaQdHCxmSc4sxbstl7PgrRezSiIjUigGIiNDe1RphY/zRtZktHjyswMc7r2LOwRiUlPGQGBHpJgYgIgIA2JrJ8M2wNngnsBkMJMC+a5kYt+UyknKKxC6NiEjlGICISMFAIsH4rk2x5a2usDOXITGnCOO2XMKv1zPFLo2ISKUYgIioim5edtg61h8dm9qguKwCXxyMwYLfY3lIjIh0BgMQEVXLzlyGlcN9MaFbU0gA7InMwJvbInAzl4fEiEj7MQAR0VNJDSSYGNAMK4b7wtbUCHFZhRi35TIOx2SJXRoRUb0wABHRM3VpZostY/3h52qNwtJyfLI/GouOxKH0YYXYpRER1QkDEBHVioOFMb59uS1e7+wGANh5JR3jt0Ug9W6xyJURESmPAYiIas3QQIIpQR5YNqwNrE0MceNOAcaEXcIfcdlil0ZEpBQGICJSWnePRggb44+2zlYoeFCOj/dex5I/ElBWzkNiRKQdGICIqE6crEyw5pW2GN3RFQDw06XbmPDTFaTfLxG5MiKiZ2MAIqI6M5QaYGqwJ5YMaQ0rE0Ncy8jH6M2XcCIhR+zSiIhqxABERPXWw8sOm0f7o7WTJe6XPMQHe65h+Z+JeMhDYkSkoRiAiEglnK1NsPa1dnjN3wUAsPmvVLy94yoy8x+IXBkRUVUMQESkMkZSA3zQywuLBrWEuUyKq2n3MWrTRZxJyhW7NCKiShiAiEjlevs4IGyMP1o4WuBeyUNM3R2Fb08l4WGFIHZpREQAGICISE1cbUyxbkR7vNSuCQBgw7kUTPm/q8gq4CExIhKfqAHowoULmDRpEgIDAyGXy3HkyJFKr8+cORNyubzSY/z48TWOWV5ejmXLlqF3795o27YtQkJCsGrVKggC/+VJ1NCMDQ0wI6Q5FgxsATMjKS6l3sPozZdw/mae2KURkZ4zFPPDi4qKIJfLMXz4cLz77rvVrhMUFITQ0FDFc5lMVuOYa9euxbZt27Bo0SJ4e3sjKioKs2bNgqWlJcaOHavS+omodp5r4Qi5owVm7Y9GXFYh3t0Zibe6NcX4ru6QGkjELo+I9JCoASg4OBjBwcE1riOTyeDg4FDrMS9fvow+ffqgZ8+eAABXV1ccOHAAV69erU+pRFRP7o3M8MOI9lj8RwJ+iczA2vBbiLh9H/MGtICdec3/sCEiUjVRA1BtnD9/Ht26dYOVlRW6du2KadOmwdbW9qnr+/n5YceOHUhKSoKHhwdu3LiBixcvYubMmUp/tkQN/zB9PKY6xtYE7E/7qbNHU5kUn/XzQQc3a/zv9zhcuHUXozZfwv9eaIEObjaq/8BqcBtqP/an/dTVozLjSQQNmRwjl8uxatUqhISEKJYdOHAAJiYmcHV1RUpKCpYuXQozMzNs374dUqm02nEqKiqwdOlSrFu3DlKpFOXl5Zg+fTrefvvthmqFiGohLjMfk7dcQtydAhhIgA+ek+OdYC8Y8JAYETUAjd4DNHDgQMXPjydBh4SEKPYKVefgwYPYt28flixZAm9vb0RHRyM0NBSOjo4YOnSoUp+fk5MPVcdDiQSws7NUy9iagP1pv4bq0VYK/PBaOyw6Go/91zLx1aEYnI65g7kDWsDGzEhtn8ttqP3Yn/ZTV4+Px60NjQ5AT3Jzc4OtrS1u3rz51AD05ZdfYuLEiYrwJJfLkZaWhjVr1igdgAQBavvlU+fYmoD9ab+G6NHESIov+svh52qNL4/G40xyHkZuuoj/vdAS7Vys1frZ3Ibaj/1pPzF71KrrAGVkZODu3bs1ToouKSmB5ImDgFKplKfBE2mwF9s4YeNIP7jbmuJOQSne3n4Fmy+koILfWyJSE1EDUGFhIaKjoxEdHQ0ASE1NRXR0NNLS0lBYWIhFixYhIiICqampCA8Px+TJk+Hu7o6goCDFGOPGjUNYWJjiea9evbB69WocP34cqampOHz4MDZs2FBpbhERaR5vB3P8ONoP/Vo4oFwAlp9Iwgd7ruFecZnYpRGRDhL1EFhUVFSla/M8vt7P0KFDMXv2bMTGxmLPnj3Iz8+Ho6MjunfvjqlTp1a6FlBKSgry8v65qNqnn36Kb775BnPmzEFOTg4cHR3x6quvYsqUKQ3XGBHVibnMEPMGtIC/qzWW/JGAU4m5GL35EkIHtUSbJlZil0dEOkRjzgLTRNnZ6pkEbW9vqZaxNQH7036a0mNMZgFm7b+OlLslMDSQ4L0eHhjh71LlELeyNKU/ddL1Htmf9lNXj4/HrQ2tmgNERPpD3tgCm0b7o4+PPR5WCPj6eCI+3nsd+SUPxS6NiHQAAxARaSwLY0OEvtASH/X2gqGBBMfjczA67BKiM/PFLo2ItBwDEBFpNIlEglf8XLB+RHs4Wxkj7V4Jxm+LwI7LaTy7k4jqjAGIiLRCKydLbB7jj2AvO5SVC/jqWDw+2X8DBQ94SIyIlMcARERaw8rECF8NboXpPT0hNZDgSGwWxm25jNg7BWKXRkRahgGIiLSKRCLByA6uWPtqOzS2NMatvGK8sfUyfr6azkNiRFRrDEBEpJV8na0QNsYfgZ6NUFou4H+H4/D5wRgUlZaLXRoRaQEGICLSWjamRlgypDXeC/KAVAL8Fn0H47ZcQnx2odilEZGGYwAiIq1mIJFgbGc3rH6lHRwsZEjOLcbrWy5jX1SG2KURkQZjACIindDe1Rpbxvijq7stHjyswNxDsZjzWwxKynhIjIiqYgAiIp1haybDN8PbYFJ3dxhIgP3XMvH61stIzikSuzQi0jAMQESkUwwkEozv6o5VL7WFnbkMCdlFGLvlEg5GZ4pdGhFpEAYgItJJHZvaIGyMPzq6WaO4rAKf/xqD/x2O5SExIgLAAEREOszeXIaVL7XFW12bQgLg56sZeHNrBJJ4lhiR3mMAIiKdJjWQ4O3uzbBiuC9sTY0Qm1WIF1eeQurdYrFLIyIRMQARkV7o0swWW8b6o2VjC+SXPMTiYwm8cjSRHlM6AJ04cQJ//fWX4vmWLVswePBgfPDBB7h3755KiyMiUiUHC2PMG9ACRlIJTiXm4kRCjtglEZFIlA5AX331FQoLHx0/j4mJwcKFCxEcHIzU1FQsXLhQ5QUSEalSMzszTAjyBAAs+SOBk6KJ9JTSASg1NRVeXl4AgN9//x29evXC+++/j88//xwnTpxQeYFERKr2bm9vOFkaI/3+A/xw7pbY5RCRCJQOQEZGRigpKQEAnDlzBt27dwcAWFtbo6CgQLXVERGpgZnMEB/2fvQPuc0XUnmhRCI9pHQA8vf3R2hoKFatWoXIyEj07NkTAJCcnAwnJydV10dEpBbB3nbo7tEIDysEfHksnhOiifSM0gHo888/h6GhIQ4dOoQvvvgCjRs3BvBocnRQUJDKCyQiUgeJRIIPe3vB2NAAF27dxeGYLLFLIqIGZKjsG5ydnbFmzZoqyz/55BOVFERE1FBcbUwxrrMbvj9zE18fT0SARyNYGCv9Z5GItJDSe4CuXbuGmJgYxfMjR45g8uTJWLp0KUpLS1VaHBGRuo3t5AZXGxNkF5ZibfhNscshogZSp0NgycnJAICUlBS8//77MDU1xW+//YavvvpK1fUREamVsaEBPurtDQDYfuk24rN4mwwifaB0AEpOTkbLli0BAAcPHkSnTp2wZMkShIaG4vfff1d5gURE6hbg0Qi9m9ujXAAWHolDBSdEE+k8pQOQIAioqKgAAISHh6NHjx4AgCZNmiAvL0+11RERNZDpPT1hamSAK2n3ceBaptjlEJGaKR2A2rRpg++++w579uzBhQsXFKfBp6amwt7eXtX1ERE1CCcrE7zV1R0AsOJEEu6XlIlcERGpk9IB6JNPPsH169cxb948TJo0Ce7uj/5gHDp0CH5+fiovkIiooYzo4AIPOzPkFZfh21PJYpdDRGqk9PmeLVq0wL59+6os//jjj2FgwJvLE5H2MpIaYEYfb0zacRW7r6TjxTZOaOVkKXZZRKQGdU4sUVFR+OWXX/DLL7/g2rVrMDY2hpGRkSprIyJqcB3cbNC/pSMEPJoQXV7BCdFEukjpPUA5OTmYNm0aLly4ACsrKwDA/fv30aVLF3z99ddo1KiRyoskImpIU4M9cTIhB9GZBdgTmY7h7ZzFLomIVEzpPUDz5s1DUVERDhw4gPPnz+P8+fPYv38/CgoKMH/+fHXUSETUoOzNZXinezMAwLenkpFXxIu8EukapQPQyZMn8cUXX8DLy0uxzNvbG1988QVOnDih0uKIiMQyvL0zfBzMcb/kIVacSBK7HCJSMaUDUEVFRbVzfQwNDRXXByIi0naGBhLMCGkOANh3LRNXbt8TuSIiUiWlA1DXrl2xYMECZGb+c6GwzMxMhIaGolu3biotjohITG2drTDY1wkAsOhoPB5yQjSRzqjTvcAKCgrQp08fhISEICQkBH369EFBQQE+/fRTddRIRCSadwM9YG1iiLisQuy4fFvscohIRZQ+C6xJkyb4+eefcebMGSQmJgIAvLy8EBAQoPLiiIjEZmNmhClBHvjf4Th8f+Ym+sod4GBhLHZZRFRPdboOkEQiQffu3TFmzBiMGTMGAQEBSEhIQL9+/ZQa58KFC5g0aRICAwMhl8tx5MiRSq/PnDkTcrm80mP8+PHPHDczMxMffvghunTpgrZt22LQoEGIjIxUqjYioscG+zqhTRNLFJaWY9nxRLHLISIVUHoP0NOUlZXh1q1bSr2nqKgIcrkcw4cPx7vvvlvtOkFBQQgNDVU8l8lkNY557949jBgxAl26dMHatWtha2uLmzdvwtraWqnaiIgeM5BIMKOPN8ZtuYzfY7Iw2NcJnd1txS6LiOpBZQGoLoKDgxEcHFzjOjKZDA4ODrUec+3atXBycqoUmtzc3OpcIxERALRobImX2jljR0Qavjwaj23jOsBIytv/EGkrUQNQbZw/fx7dunWDlZUVunbtimnTpsHW9un/8jp27BgCAwPxn//8BxcuXEDjxo0xcuRIvPLKK0p/tkRSn8prHlMdY2sC9qf9dL3H+vT3TmAzHI3Nws28Ymy5mIo3ujRVbXEqwm2o3XS9P0B9PSoznkYHoKCgIPTt2xeurq5ISUnB0qVLMWHCBGzfvh1SqbTa96SkpGDbtm144403MGnSJERGRmL+/PkwMjLC0KFDlfp8Ozv13QRRnWNrAvan/XS9x7r0Zw/g00GtMH37Faw/ewsjAjzgamum+uJUhNtQu+l6f4C4PdY6AHXq1AmSGqLVw4cPVVLQvw0cOFDx8+NJ0CEhIYq9QtURBAFt2rTB+++/DwBo1aoV4uLi8NNPPykdgHJy8iGo+LIfEsmjDa6OsTUB+9N+ut5jffsLdLWCv6s1LqXew6e7rmLxkNaqL7KeuA21m673B6ivx8fj1katA9Ann3xS54JUxc3NTTGp+WkByMHBodJtOgDA09MThw4dUvrzBAFq++VT59iagP1pP13vse79SfBxH2+M2nwJx+NzcDIhB4GedqouTyW4DbWbrvcHiNtjrQOQsntP1CEjIwN3796tcVK0v78/kpIq37cnOTkZLi4u6i6PiPSEl705Rvi7IOyvVHx1LAEd3WxgYlT9YXki0kyinsJQWFiI6OhoREdHAwBSU1MRHR2NtLQ0FBYWYtGiRYiIiEBqairCw8MxefJkuLu7IygoSDHGuHHjEBYWVun5lStXsHr1aty8eRP79u3Djh07MHLkyAbvj4h014Ru7nC0kCHtXgl+PJ8idjlEpCRRJ0FHRUVh7NixiuePT10fOnQoZs+ejdjYWOzZswf5+flwdHRE9+7dMXXq1ErXAkpJSUFeXp7iedu2bbFy5UosXboUq1atgqurKz755BO8+OKLDdcYEek8M5kU7/fywsx90dh0IQXPt2qMpramYpdFRLUkEQRdP8JYd9nZ6pkEbW9vqZaxNQH703663qMq+xMEAf/ZFYWzN/PQtZktlg9rU+PJIg2F21C76Xp/gPp6fDxubfAqXkREdSSRSPBRH28YSSU4m5yHP+KyxS6JiGqJAYiIqB6a2ppibKdHV5tf8kcCikrLRa6IiGpD6TlA/77FxL9JJBIYGxujadOm6NOnD2xsbOpbGxGRVni9sxsORt9B2r0SrD97E+/18BS7JCJ6BqUD0PXr13H9+nVUVFTAw8MDAJCUlASpVApPT09s3boVixYtwtatW+Ht7a3ygomINI2JkRQf9vLC+3uuYcvF2xjQqjG87M3FLouIaqD0IbA+ffogICAAJ0+exO7du7F7926cOHECAQEBGDhwIE6cOIGOHTs+dU8REZEuCvKyQ7CXHcorBHx5NB48v4RIsykdgNavX4+pU6fCwsJCsczS0hLvvfce1q1bB1NTU0yZMgVRUVEqLZSISNN90NsLxoYGuJR6D7/duCN2OURUA6UDUEFBAXJycqosz83NRUFBAQDAysoKZWVl9a+OiEiLNLEywfiuj+4Qv+x4IvJLVH+PRCJSDaUDUO/evfHJJ5/g8OHDyMjIQEZGBg4fPoz//ve/CAkJAQBcvXoVzZo1U3WtREQab1QHVzS1NUVuURnWnEkWuxwiegqlJ0HPnTsXoaGhmD59OsrLH53uKZVKMXToUMyaNQvAo5uPLliwQLWVEhFpAZmhAT7u4413d0bi/yLSMKi1E+SNLZ79RiJqUEoHIHNzc8yfPx+zZs1CSsqj+9+4ubnB3PyfMx5atmypugqJiLRMF3db9JU74HBMFhYdjcO6Ee1hoAFXiCaif9T5Qojm5uZo0aIFWrRoUSn8EBERMC3YE2ZGUkSm52NvZIbY5RDRE5TeA1RUVITvv/8eZ8+eRU5ODioqKiq9fvToUZUVR0SkrRwtjTExwB3L/kzEypNJ6NncHjamRmKXRUR/UzoAffrppzh//jwGDx4MBwcHjbjxHxGRJnrVzxn7r2UiPrsQq04m4b/P+YhdEhH9TekAdOLECaxZswYdOnRQRz1ERDrDUGqAGX28MWH7FeyJzMCLbZzg62wldllEhDrMAbKysuJ9voiIaqm9qzUGtm4MAFh0NB7lFbxCNJEmUDoATZ06Fd988w2Ki4vVUQ8Rkc75Tw8PWBobIuZOAXZdSRO7HCJCHQ6BbdiwAbdu3UJAQABcXV1haFh5iJ9//lllxRER6YJGZjJMDmyGRUfj8d3pZPT2cYC9uUzssoj0mtIB6PHVnomIqPaGtm2CvVEZiM4swPI/EzF3QAuxSyLSa0oHoHfffVcddRAR6TSpgQQzQprjjS2XcTD6Dgb7OqGDm43YZRHprTpfCJGIiJTT2skSw9o1AQB8eTQeD8srnvEOIlKXWu0B6ty5M3777Tc0atQInTp1qvHaP+fPn1dZcUREuuad7s1wNDYbiTlF2HbpNsZ0chO7JCK9VKsANGvWLFhYWCh+5sUPiYjqxtrUCO/18MC8Q7FYG34Tz7VwRGNLY7HLItI7tQpAQ4cOVfw8bNiwp65XUlJS/4qIiHTcC60bY29kBq6k3cfXxxOwcFArsUsi0jtKzwGaP39+tcuLioowYcKEehdERKTrDCQSzAjxhlQCHI3NRnhyrtglEekdpQPQ8ePHsXz58krLioqK8NZbb6G8vFxlhRER6bLmDhZ4xc8FAPDV0Xg8eMgJ0UQNSekA9MMPP2DHjh3YuHEjAKCgoABvvvkmJBIJ1q1bp+r6iIh01sQAd9iby5BytwSbL6SIXQ6RXlE6ADVt2hTr1q3Dd999h02bNmH8+PEwMjLC2rVrYWZmpo4aiYh0koWxIab39AQAbDyfgtv3eIshooZSp+sAtWjRAqtXr8bXX38NExMThh8iojrqK3dAx6Y2ePCwAouPJUAQeLNUooZQq7PAhgwZUu2p7zKZDHfu3MGIESMUy3gvMCKi2pNIJJjR2xsjNl3EqcRcnEjIQbC3vdhlEem8WgUg3v+LiEh9mtmZYVRHV/x4PgVL/khAF3dbmBhJxS6LSKfVKgDx/l9EROo1vmtTHIq+g/T7D/DDuVuYHOghdklEOq3O9wIrLS1FRkYG0tLSKj2IiEh5pkZSfNDLCwCw+UIqknOKRK6ISLcpfTf4pKQk/Pe//8Xly5crLRcEARKJBNHR0SorjohInwR726G7RyOcTsrFl8fiseolX956iEhNlA5As2bNgqGhIVavXg1HR0d+OYmIVEQikeDD3l7468e7uHDrLg7HZOG5Fo5il0Wkk5QOQDdu3MCuXbvg5eWljnqIiPSaq40pxnV2w/dnbuLr44kI8GgEC2Ol/1QT0TMoPQfIy8sLeXl56qiFiIgAjO3kBlcbE2QXlmJt+E2xyyHSSUoHoA8//BCLFy/GuXPnkJeXh4KCgkoPIiKqH2NDA3zU2xsAsP3SbcRnFYpcEZHuUToAvfHGG4iIiMDrr7+OgIAAdOrUCZ06dULHjh3RqVMnpca6cOECJk2ahMDAQMjlchw5cqTS6zNnzoRcLq/0GD9+fK3H//777yGXy7FgwQKl6iIiEluARyP0bm6PcgFYeCQOFbxCNJFKKX1gedOmTSr78KKiIsjlcgwfPvyp1xoKCgpCaGio4rlMJqvV2FevXsVPP/0EuVyuklqJiBra9J6eCE/OxZW0+zhwLROD2jiJXRKRzlA6AHXu3FllHx4cHIzg4OAa15HJZHBwcFBq3MLCQnz00UeYP38+vvvuu/qUSEQkGicrE7zV1R0rTiZhxYkkBHvbwcrESOyyiHRCnS+EWFxcjISEBNy4caPSQ9XOnz+Pbt26oV+/fvjiiy9qNQF77ty5CA4ORkBAgMrrISJqSCM6uMCjkRnyisvw7alkscsh0hlK7wHKzc3FrFmzcOLEiWpfV+WFEIOCgtC3b1+4uroiJSUFS5cuxYQJE7B9+3ZIpdXfJ+fAgQO4fv06du7cWe/PV8cljh6PqauXT2J/2k/Xe9S2/mSGBpgR4o1JO65i95V0DPZ1Qisnyxrfo209Kov9aT919ajMeEoHoAULFuD+/fvYsWMHxo4di5UrVyI7OxvfffcdZs6cqexwNRo4cKDi58eToENCQhR7hZ6Unp6OBQsW4IcffoCxsXG9P9/OruY/Mpo6tiZgf9pP13vUpv7621tiSGw29kSkYfHxRPw8uTukBs/+S69NPdYF+9N+YvaodAA6d+4cvv32W/j6PrpEu7OzM7p37w4LCwusWbMGPXv2VEOZj7i5ucHW1hY3b96sNgBdu3YNOTk5GDZsmGJZeXk5Lly4gC1btiAyMvKpe46qk5OTD1WfeCGRPNrg6hhbE7A/7afrPWprf5O6NcXh65m4mnoPa4/F4qX2zk9dV1t7rC32p/3U1ePjcWtD6QBUVFSERo0aAQCsra2Rm5sLDw8P+Pj44Pr168oOp5SMjAzcvXv3qZOiu3btin379lVaNmvWLHh6emLChAlKhR8AEASo7ZdPnWNrAvan/XS9R23rz85Mhne6N8PiPxLw7alk9G5uD1uzms+K1bYelcX+tJ+YPSo9CdrDwwNJSUkAHh2W2r59OzIzM/HTTz/V6Wyt6Ohoxbyh1NRUREdHIy0tDYWFhVi0aBEiIiKQmpqK8PBwTJ48Ge7u7ggKClKMMW7cOISFhQEALCws4OPjU+lhZmYGGxsb+Pj4KNsqEZFGGd7eGT4O5rhf8hArTiSJXQ6RVlN6D9DYsWORlZUFAHj33Xfx1ltvYd++fTAyMsLChQuVGisqKgpjx45VPH98vZ+hQ4di9uzZiI2NxZ49e5Cfnw9HR0d0794dU6dOrXQtoJSUFN6ag4j0gqGBBDNCmmP8tgjsu5aJwb5OaOdiLXZZRFpJIgj12/lUXFyMxMRENGnSRHFoTFdkZ6tnDpC9vaVaxtYE7E/76XqPutDf/EOx+CUqA80dzLFptD8Mn5gQrQs91oT9aT919fh43NpQ6hBYWVkZQkJCkJCQoFhmamqK1q1b61z4ISLSVO8GecDaxBBxWYXYcfm22OUQaSWlApCRkREePHigrlqIiKgWbMyMMCXIAwDw/ZmbyCrg32UiZSk9CXrUqFFYu3YtHj58qI56iIioFgb7OqFNE0sUlpZj2fFEscsh0jpKT4KOjIxEeHg4Tp06BblcDlNT00qvr1y5UmXFERFR9QwkEszo441xWy7j95gsDPZ1Qmd3W7HLItIaSgcgKysr9OvXTx21EBGRElo0tsRL7ZyxIyINXx6Nx7ZxHWAkrfMtHon0itIB6PGp6kREJL5J3ZvhSGwWbuYVI+yvVLzRpanYJRFpBf5TgYhIi1maGGJqsCcAYP3ZW0i/XyJyRUTaQek9QNnZ2Vi0aBHCw8ORm5uLJy8jpMq7wRMR0bM939IReyIzcDn1Hpb+kYDFQ1qLXRKRxlM6AM2cORPp6emYPHkyHB0d1VETEREpQfL3hOhRmy/heHwOTiXkYEgtLwZHpK+UDkAXL17E1q1b0bJlS3XUQ0REdeBlb44R/i4I+ysVXx1LQH9/N7FLItJoSs8BatKkSZXDXkREJL4J3dzhaCHD7Xsl+PZ4wrPfQKTHlA5An3zyCZYsWYLU1FR11ENERHVkJpPi/V5eAIDVfybgVl6xyBURaS6lD4FNnz4dxcXF6Nu3L0xMTGBkZFTp9fPnz6usOCIiUk7v5vbo2swWZ5Pz8NXReHwzrA0kEsmz30ikZ5QOQJ988ok66iAiIhWQSCT4uI83Xtt4EeHJefgjLhu9fRzELotI4ygdgIYOHaqOOoiISEWa2ppiUrAnlh+Lx5I/EtC1WSOYyaRil0WkUWo1B6igoKDSzzU9iIhIfJN7ecPF2gR3Ckqx/uxNscsh0ji12gPUqVMnnDp1CnZ2dujYsWO1x5MFQYBEIuGFEImINICJkRQf9vbC9J+vYcvF2xjQqjG87M3FLotIY9QqAP3444+wtrYGAGzatEmtBRERkWoEedmhh5cdTiTk4Muj8Vj9SltOiCb6W60CUOfOnav9mYiINNsHvbxw7mYeLqXew2837uD5lo3FLolIIyg9CfrGjRvVLpdIJDA2NoazszNkMlm9CyMiovpztjbB+K5N8e2pZCw7nohADztYmij9p59I5yj9LRgyZEiNu1ANDQ0xYMAAzJ07F8bGxvUqjoiI6m9UB1fsv5aJW3nFWHMmGR/29ha7JCLRKX0l6JUrV8Ld3R1z587Fnj17sGfPHsydOxceHh5YsmQJFixYgLNnz2LZsmVqKJeIiJQlMzTAx30ehZ7/i0hDTCbP2CVSeg/Q6tWr8d///hdBQUGKZXK5HE5OTvjmm2+wc+dOmJmZYeHChZgxY4ZKiyUiorrp4m6LvnIHHI7JwqKjcVg3oj0MOCGa9JjSe4BiY2Ph7OxcZbmzszNiY2MBAC1atEBWVlb9qyMiIpWZFuwJMyMpItPzsTcyQ+xyiESldADy9PTE2rVrUVpaqlhWVlaGtWvXwtPTEwCQmZkJOzs71VVJRET15mhpjIkB7gCAlSeTcLe4TOSKiMSj9CGwzz//HO+88w6Cg4Mhl8sBPNorVF5ejjVr1gAAUlJSMHLkSNVWSkRE9faqnzP2X8tEfHYhVp1Mwn+f8xG7JCJRKB2A/P39cfToUezbtw/JyckAgP79++OFF16AhYUFgEdnihERkeYxlBpgRh9vTNh+BXsiM/BiGyf4OluJXRZRg1MqAJWVleH555/HmjVrMGLECHXVREREatTe1RoDWzfGgWuZWHQ0Hj+O8oPUgBOiSb8oNQfIyMgIDx48UFctRETUQP7TwwOWxoaIuVOAXVfSxC6HqMEpPQl61KhRWLt2LR4+fKiOeoiIqAE0MpNhcmAzAMB3p5ORXVha8xuIdIzSc4AiIyMRHh6OU6dOQS6Xw9TUtNLrK1euVFlxRESkPkPbNsHeqAxEZxZg+Z+JmDughdglETUYpQOQlZUV+vXrp45aiIioAUkNJJgR0hxvbLmMg9F3MNjXCR3cbMQui6hBKB2AQkND1VEHERGJoLWTJYa1a4JdV9Lx5dF4bBnjD0Op0rMjiLQOf8uJiPTcO92bwcbUCIk5Rdh26bbY5RA1CKX3AAHAb7/9hoMHDyI9PR1lZZWvJPrzzz+rpDAiImoY1qZGeK+HB+YdisXa8Jt4roUjGlsai10WkVopvQdo06ZNmDVrFuzt7XH9+nX4+vrCxsYGKSkp6NGjhzpqJCIiNXuhdWO0dbZCcVkFvj6eIHY5RGqndADaunUr5s2bh88++wxGRkaYMGECNmzYgDFjxiA/P18dNRIRkZoZSCSYGeINqQQ4GpuN8ORcsUsiUiulA1B6ejr8/PwAACYmJigsLAQADB48GAcOHFBtdURE1GCaO1jgFT8XAMBXR+Px4GGFyBURqY/SAcje3h737t0DADRp0gQREREAgNTUVAiCoNRYFy5cwKRJkxAYGAi5XI4jR45Uen3mzJmQy+WVHuPHj69xzDVr1mD48OHw8/NDt27dMHnyZCQmJipVFxGRvpoY4A57cxlS7pZg84UUscshUhulA1DXrl1x7NgxAMDw4cMRGhqKN954A9OnT0dISIhSYxUVFUEul+OLL7546jpBQUE4deqU4rF06dIaxzx//jxGjRqFHTt2YMOGDXj48CHGjx+PoqIipWojItJHFsaGmBbsCQDYeD4Ft+8Vi1wRkXoofRbYvHnzUFHxaLfoqFGjYGNjg8uXL6N379549dVXlRorODgYwcHBNa4jk8ng4OBQ6zHXr19f6fnChQvRrVs3XLt2DZ06dVKqPiIiffRcCwfsicrAX7fuYvGxBCwd0hoSCW+WSrpF6QBkYGAAA4N/dhwNHDgQAwcOVGlR/3b+/Hl069YNVlZW6Nq1K6ZNmwZbW9tav//xxGxra2ulP1sd3/fHY+rq3xL2p/10vUdd7w+of48SiQQz+3jjtR8v4lRiLk4k5qCnt73qCqwnXd+Gut4foL4elRlPIig7cQfAgwcPEBMTg5ycHMXeoMf69Omj7HAAALlcjlWrVlU6jHbgwAGYmJjA1dUVKSkpWLp0KczMzLB9+3ZIpdJnjllRUYF33nkH9+/fx7Zt2+pUFxGRvlr02w18dzwBLjamOPJ+MExlz/67S6QtlN4DdOLECcyYMQN5eXlVXpNIJIiOjlZJYQAq7Vl6PAk6JCREsVfoWebMmYO4uDhs3bq1Tp+fk5MP5eNhzSQSwM7OUi1jawL2p/10vUdd7w9QXY8j2zrh54upuH23GF8euIYpQR6qK7IedH0b6np/gPp6fDxubSgdgObPn4/+/ftjypQpsLdv2F2ibm5usLW1xc2bN58ZgObOnYvjx48jLCwMTk5Odfo8QYDafvnUObYmYH/aT9d71PX+gPr3aGIkxQe9vPDR3uvYfCEVA1o2RjM7M9UVWE+6vg11vT9A3B6VPgssOzsbb7zxRoOHHwDIyMjA3bt3a5wULQgC5s6di8OHD+PHH3+Em5tbA1ZIRKRbgr3t0N2jER5WCPjyWLzSlzsh0lRKB6B+/frh3LlzKvnwwsJCREdHKw6bpaamIjo6GmlpaSgsLMSiRYsQERGB1NRUhIeHY/LkyXB3d0dQUJBijHHjxiEsLEzxfM6cOdi7dy+WLFkCc3NzZGVlISsrCyUlJSqpmYhIn0gkEnzY2wsyqQQXbt3F4ZgssUsiUgmlD4F9/vnnmDp1Ki5evAgfHx8YGlYeYuzYsbUeKyoqqtL6oaGhAIChQ4di9uzZiI2NxZ49e5Cfnw9HR0d0794dU6dOhUwmU7wnJSWl0nykx5Odx4wZU+mzQkNDMWzYsNo3SkREAABXG1O83qUpvj9zE18fT0SARyNYGNfpXtpEGkPp3+D9+/fj9OnTkMlkOH/+fKXXJBKJUgGoS5cuiImJeerrT17TpzqPL8r4WE3jERFR3Yzt5IZfr2ci9W4J1obfxPSeXmKXRFQvSgegZcuW4b333sPEiRMrXQ+IiIh0l7GhAT7q7Y2pu6Ow/dJtDGrtBG8Hc7HLIqozpRNMWVkZBgwYwPBDRKRnAjwaoVdze5QLwKKjcZwQTVpN6RQzZMgQ/Prrr+qohYiINNz7PT1hamSAiNv3ceB6ptjlENWZ0ofAKioqsG7dOpw6dQpyubzKJOhZs2aprDgiItIsTlYmeKurO1acTMLyP5PQw8sOViZGYpdFpDSlA1BMTAxatmwJAIiNja30Gm+WR0Sk+0Z0cMH+a5lIyi3Ct6eSMTOkudglESlN6QC0efNmddRBRERawkhqgBkh3pi04yp2X0nHi22c0MqpdrcfINIUnMlMRERK6+Bmg/4tHSEAWHgkDuUVnBBN2oUBiIiI6mRqsCfMZVJEZxZgT2S62OUQKYUBiIiI6sTeXIZJ3ZsBAL49lYy8olJxCyJSAgMQERHV2UvtneHjYI77JQ+x4kSS2OUQ1RoDEBER1ZmhgQQz/j4LbN+1TFy5fU/kiohqhwGIiIjqpa2zFQa3cQIALDoaj4ecEE1agAGIiIjq7d0gD1ibGCIuqxA7Lt8WuxyiZ2IAIiKierMxM8KUIA8AwPdnbiKr4IHIFRHVjAGIiIhUYrCvE9o0sURhaTmWHU8UuxyiGjEAERGRShhIJJjRxxsGEuD3mCycv5kndklET8UAREREKtOisSVeaucMAPjyaDzKyitEroioegxARESkUpO6N0MjMyPczCtG2F+pYpdDVC0GICIiUilLE0NMDfYEAKw/ewvp90tEroioKgYgIiJSuedbOsLP1RoPHlZg6R8JYpdDVAUDEBERqZxEIsHHfbwhNZDgeHwOTiXmiF0SUSUMQEREpBbe9uYY4e8CAPjqWAJKyspFrojoHwxARESkNhO6ucPRQoa0eyX48XyK2OUQKTAAERGR2pjJpHi/lxcAYNOFFNzKKxa5IqJHGICIiEiteje3R1d3W5SWC/jqWDwEgTdLJfExABERkVpJJBJ81McbRlIJzibn4Y+4bLFLImIAIiIi9Wtqa4oxndwAAEv+SEBRKSdEk7gYgIiIqEG80dkNztYmuFNQivVnb4pdDuk5BiAiImoQJkZSfPj3hOgtF28jIbtQ5IpInzEAERFRgwnyskMPLzuUVwj48ignRJN4GICIiKhBfdDLC8aGBriUeg+/3bgjdjmkpxiAiIioQTlbm2B816YAgGXHE5Ff8lDkikgfMQAREVGDG9XBFU1tTZFbVIY1Z5LFLof0EAMQERE1OJmhAT7u4w0A+L+INMRkFohcEekbBiAiIhJFF3db9JU7oEIAFh2NQwUnRFMDYgAiIiLRTAv2hJmRFJHp+dgbmSF2OaRHGICIiEg0jpbGmBjgDgBYeTIJd4vLRK6I9AUDEBERiepVP2d42ZvhXslDrDqZJHY5pCdEDUAXLlzApEmTEBgYCLlcjiNHjlR6febMmZDL5ZUe48ePf+a4W7ZsQe/eveHr64uXX34ZV69eVVcLRERUT4ZSA8zs0xwA8EtkBiLT7otcEekDUQNQUVER5HI5vvjii6euExQUhFOnTikeS5curXHMX3/9FaGhoZgyZQp+/vlntGjRAuPHj0dOTo6qyyciIhVp72qNga0bQwCw6Gg8yis4IZrUS9QAFBwcjOnTp6Nv375PXUcmk8HBwUHxsLa2rnHMDRs24JVXXsHw4cPh7e2NOXPmwMTEBLt27VJ1+UREpEL/6eEBS2NDxNwpwK4raWKXQzrOUOwCnuX8+fPo1q0brKys0LVrV0ybNg22trbVrltaWopr167h7bffViwzMDBAQEAALl++rPRnSyR1LvuZY6pjbE3A/rSfrveo6/0B2tujnbkMU4KaYeGReHx3Ohl95A6wN5dVWU9b+6stXe8PUF+Pyoyn0QEoKCgIffv2haurK1JSUrB06VJMmDAB27dvh1QqrbJ+Xl4eysvLYWdnV2m5nZ0dEhMTlf58OzvLOtcu5tiagP1pP13vUdf7A7Szxwm9ffDrjSxcTb2HNWdT8PWr7Z+6rjb2pwxd7w8Qt0eNDkADBw5U/Px4EnRISIhir5C65eTkQ9XX5ZJIHm1wdYytCdif9tP1HnW9P0D7e/wg2BOvb7mMny/fRn8fO3Rws6n0urb39yy63h+gvh4fj1sbGh2AnuTm5gZbW1vcvHmz2gBka2sLqVRaZcJzTk4O7O3tlf48QYDafvnUObYmYH/aT9d71PX+AO3tsZWTJYa2bYLdV9Ox6Eg8tozxh6G06pRVbe2vtnS9P0DcHrXqOkAZGRm4e/cuHBwcqn1dJpOhdevWCA8PVyyrqKhAeHg4/Pz8GqpMIiKqp8mBzWBjaoTEnCJsu3Rb7HJIB4kagAoLCxEdHY3o6GgAQGpqKqKjo5GWlobCwkIsWrQIERERSE1NRXh4OCZPngx3d3cEBQUpxhg3bhzCwsIUz9944w3s2LEDP//8MxISEjB79mwUFxdj2LBhDd4fERHVjbWpEd7r4QEAWBt+E5n5D0SuiHSNqIfAoqKiMHbsWMXz0NBQAMDQoUMxe/ZsxMbGYs+ePcjPz4ejoyO6d++OqVOnQib756yAlJQU5OXlKZ4PGDAAubm5WL58ObKystCyZUusW7euTofAiIhIPC+0boxfIjNwNe0+vj6egIWDWoldEukQiSDo+hHGusvOVs8kaHt7S7WMrQnYn/bT9R51vT9At3qMvVOAMWGXUCEAy4e3QbdmjXSqv+roen+A+np8PG5taNUcICIi0i8+jhZ41c8FAPDV0Xg8eFghckWkKxiAiIhIo00McIe9uQwpd0uw+UKK2OWQjmAAIiIijWZhbIhpwZ4AgI3nU3D7brHIFZEuYAAiIiKN91wLB3RsaoMHDyvw1bEEcPoq1RcDEBERaTyJRIIZvb1haCDBqcRcHL6eKXZJpOUYgIiISCs0szPDqI6uAIB3t17GnN9iEJ9VKHJVpK0YgIiISGuM79oUnd1tUFpegX1RmRix6SLe3XkVp5NyUcHDYqQErboXGBER6TdTIym+fbktbhU9xHdH43A0Ngvnbt7FuZt34dHIDCM6uOD5lo4wMZKKXSppOO4BIiIirePf1Bahg1ri5/GdMbKDC8xlUiTlFuF/h+MwaO15rDmdjJzCUrHLJA3GAERERFrL2doE03t6Yf/ELpje0xNNrIxxt7gM687ewqC15zCX84ToKXgIjIiItJ6FsSFGdnDFK34uOB6Xja0XbyMy/T72XcvEvmuZ6OJug5EdXNGtmS0kEonY5ZIGYAAiIiKdYWggQYjcASFyB1xNu49tF1NxLC77n3lCdmYY4c95QsQAREREOqqtsxXaOrdC2r0SbL98G79EZiAp59E8oW9PJeOldk3wUntn2JnLxC6VRMA5QEREpNOeNU9o3qEYxGdznpC+4R4gIiLSC1XnCaUiMj0fe6MysTeK84T0DQMQERHpFc4TIoABiIiI9BjnCekvzgEiIiK9x3lC+od7gIiIiP725DyhLRdTEfWveUJd3W0xooML5wnpAAYgIiKiJzw5T2jrxVT8EZeNszfzcPZmHjzszDDS3wX9OU9IazEAERER1eDxPKHb94qx43KaYp7Qgr/nCQ3nPCGtxDlAREREteBibaqYJzQt2BNOlsbI4zwhrcU9QEREREqwMDbEqI6ueNX/6fOERnZ0QVd3zhPSZAxAREREdVDbeULPt2oMY0MecNE0DEBERET19O95QtsvVZ0n9FL7JhjejvOENAkjKRERkYq4WJvi/V5eOPB25XlCa8Nv4UXOE9Io3ANERESkYv+eJ/TH3/cd4zwhzcIAREREpCaGBhL0lTugbw3zhEZ1cEH/lpwn1NAYgIiIiBrA0+YJzf89DqtOPponxOsJNRzGTSIiogb073lCU5+YJzTo+3OYdygWsZn5Ypep87gHiIiISAQWxoYY3dEVr/09T2jLX6m4lpGPXyIz8EtkBro2s8XIDpwnpC4MQERERCJ6PE8oxMf+73lCt3E8Phtnk/NwNjkPnnZmGMl5QirHAERERKQBJBIJ2rlYo72rNYokUnx3NBa/RGYg8V/zhF5u74zh7ZugkRnnCdUXoyQREZGGaWpnhg96/zNPqPHf84S+D7+JQd+fw/xDsUjg9YTqhXuAiIiINNS/5wkdi83C1ou3H80TisrAL1GP5gmN6uCCLpwnpDQGICIiIg1naCDBcy0c/3U9Ic4Tqi8GICIiIi3xeJ5QOxdrpN4txvbLadjLeUJ1ImpMvHDhAiZNmoTAwEDI5XIcOXLkqet+/vnnkMvl2LhxY41jlpeXY9myZejduzfatm2LkJAQrFq1CoIgqLh6IiIi8bjamOKDXl7YP7EL/tPDg/OElCTqHqCioiLI5XIMHz4c77777lPXO3z4MK5cuQJHR8dnjrl27Vps27YNixYtgre3N6KiojBr1ixYWlpi7NixqiyfiIhIdJYmhhjTyQ0j/F1wLC4bWy7exnXOE3omUQNQcHAwgoODa1wnMzMT8+bNw/r16/H2228/c8zLly+jT58+6NmzJwDA1dUVBw4cwNWrV1VRMhERkUYylBpUmie05eJtHI+rPE9oVAdX9GvpyHlC0PDT4CsqKvDRRx9h/PjxaN68ea3e4+fnh7NnzyIpKQkAcOPGDVy8eBE9evRQZ6lEREQa4fE8oS9fbIXd4zvhVT9nmBlJkZhThHm/x+LFteew9sxN5BaVil2qqDR6EvTatWthaGio1KGriRMnoqCgAM8//zykUinKy8sxffp0vPjii0p/vjr2FD4eU1f3QrI/7afrPep6f4Du98j+as/N1hQf9fHGpO7NsCcyHT9dSkNm/gN8H34TG8/fwoBWjTGigwu87M3r/2FKUNc2VGY8jQ1AUVFR2LRpE3bv3q3UMcuDBw9i3759WLJkCby9vREdHY3Q0FA4Ojpi6NChStVgZ2epbNkaMbYmYH/aT9d71PX+AN3vkf3Vnj2A6a62ePe5FjgYlYH1JxNxJfUe9kRmYE9kBnr4OOCtQA8ENbdv0HlCYm5DiaAhp0fJ5XKsWrUKISEhAICNGzdi4cKFMDD45yhdeXk5DAwM0KRJExw7dqzacYKDgzFx4kSMGjVKsezbb7/F3r178dtvvylVU05OPlT9/45E8miDq2NsTcD+tJ+u96jr/QG63yP7qz9BEHAl7T62/PVontDjj/G0M8Oojq7or+Z5Qurq8fG4taGxe4AGDx6MgICASsvGjx+PwYMHY9iwYU99X0lJSZX0KpVK63QavCBAjb986htbE7A/7afrPep6f4Du98j+6kOCds7WaPfio+sJ/XTpNvZGPbqe0LxDsVh1MgkvtXfGS+2awFaN1xMScxuKGoAKCwtx69YtxfPU1FRER0fD2toazs7OsLW1rbS+kZER7O3t4enpqVg2btw49O3bF6NHjwYA9OrVC6tXr4azs7PiENiGDRswfPjwhmmKiIhIi7jamOLD3t54O+DxPKHbuFNQiu/P3MTGc7fwfKvGGNnBBZ52DTtPSN1EDUBRUVGVJjiHhoYCAIYOHYqFCxfWaoyUlBTk5eUpnn/66af45ptvMGfOHOTk5MDR0RGvvvoqpkyZotriiYiIdMiT1xMK+ysV0ZkF+CUyA79EZqBbM1uM6uCKzu42OnE9IY2ZA6SJsrPVMwfI3t5SLWNrAvan/XS9R13vD9D9HtlfwxAEAVdu38fWS5XnCXnZm2Gkf/2uJ6SuHh+PWxsaOweIiIiIxCORSNDe1RrtXSvPE0rIfnQ9oVWnGmaekLpo9IUQiYiISHyP5wkdmNgV/+nhAUcLGXKLyvD9mZt44ftzmP97LBJztOu+Y9wDRERERLXy73lCR2OzseWi9s4TYgAiIiIipRhKDdCvpSOea+GAK7fvY8vFVPwZn4Pw5DyEJ+c9mifUwRX9WmjufccYgIiIiKhOapwn1IDXE6oLzYxlREREpFUezxPaP7FLtfOEFmjYPCHuASIiIiKVsTIxqnae0OP7jnVrZotRHV0x0M5C1DoZgIiIiEjl/j1PKOL2fWx9Yp7QnZJyvNjCXrz6RPtkIiIi0nkSiQR+rtbw+9c8oT/jc2BvaSxqXQxARERE1CAezxP6qI+34krQYuEkaCIiItI7DEBERESkdxiAiIiISO8wABEREZHeYQAiIiIivcMARERERHqHAYiIiIj0DgMQERER6R0GICIiItI7DEBERESkdxiAiIiISO8wABEREZHeYQAiIiIivcMARERERHrHUOwCNJlEor4x1TG2JmB/2k/Xe9T1/gDd75H9aT919ajMeBJBEATVfjwRERGRZuMhMCIiItI7DEBERESkdxiAiIiISO8wABEREZHeYQAiIiIivcMARERERHqHAYiIiIj0DgMQERER6R0GICIiItI7DEBERESkdxiA1GDLli3o3bs3fH198fLLL+Pq1as1rn/w4EH0798fvr6+GDRoEP78888GqrRulOlv9+7dkMvllR6+vr4NWK1yLly4gEmTJiEwMBByuRxHjhx55nvOnTuHoUOHok2bNujbty92797dAJXWjbL9nTt3rsr2k8vlyMrKaqCKlbNmzRoMHz4cfn5+6NatGyZPnozExMRnvk+bvoN16VGbvodbt27FoEGD4O/vD39/f7z66qvP3B7atP2U7U+btl11vv/+e8jlcixYsKDG9UTZhgKp1IEDB4TWrVsLO3fuFOLi4oRPP/1U6Nixo5CdnV3t+hcvXhRatmwprF27VoiPjxe+/vproXXr1kJMTEwDV147yva3a9cuwd/fX7hz547ikZWV1cBV197x48eFpUuXCr///rvg4+MjHD58uMb1b926JbRr104IDQ0V4uPjhc2bNwstW7YUTpw40UAVK0fZ/s6ePSv4+PgIiYmJlbZheXl5A1WsnDfffFPYtWuXEBsbK0RHRwsTJkwQevbsKRQWFj71Pdr2HaxLj9r0PTx69Khw/PhxISkpSUhMTBSWLl0qtG7dWoiNja12fW3bfsr2p03b7klXrlwRevXqJQwaNEiYP3/+U9cTaxsyAKnYSy+9JMyZM0fxvLy8XAgMDBTWrFlT7fpTp04VJk6cWGnZyy+/LHz22WdqrbOulO1v165dQocOHRqqPJWqTUD48ssvhYEDB1ZaNm3aNOHNN99UZ2kqoUwAunfvXgNVpVo5OTmCj4+PcP78+aeuo23fwSfVpkdt/h4KgiB06tRJ2LFjR7Wvafv2E4Sa+9PWbVdQUCA899xzwunTp4XRo0fXGIDE2oY8BKZCpaWluHbtGgICAhTLDAwMEBAQgMuXL1f7noiICHTr1q3SssDAQERERKiz1DqpS38AUFRUhF69eiE4OBjvvPMO4uLiGqLcBqFN268+hgwZgsDAQLzxxhu4ePGi2OXUWn5+PgDA2tr6qeto+zasTY+Adn4Py8vLceDAARQVFcHPz6/adbR5+9WmP0A7t93cuXMRHBxc6b8XTyPWNjRU6+h6Ji8vD+Xl5bCzs6u03M7O7qnH6LOzs2Fvb19l/ezsbLXVWVd16c/DwwP/+9//IJfLkZ+fjx9++AGvvfYaDhw4ACcnp4YoW62q23729vYoKChASUkJTExMRKpMNRwcHDBnzhy0adMGpaWl+L//+z+MHTsWO3bsQOvWrcUur0YVFRX43//+B39/f/j4+Dx1PW36Dj6ptj1q2/cwJiYGr732Gh48eAAzMzOsWrUK3t7e1a6rjdtPmf60bdsBwIEDB3D9+nXs3LmzVuuLtQ0ZgEit/Pz8Kv3Lxs/PDwMGDMBPP/2EadOmiVcY1Yqnpyc8PT0Vz/39/ZGSkoKNGzfiq6++ErGyZ5szZw7i4uKwdetWsUtRm9r2qG3fQw8PD+zZswf5+fk4dOgQZsyYgbCwsKeGBG2jTH/atu3S09OxYMEC/PDDDzA2Nha7nBoxAKmQra0tpFIpcnJyKi3Pycmpkm4fs7e3r5Jya1pfTHXp70lGRkZo2bIlbt26pY4SG1x12y87OxsWFhZav/fnaXx9fXHp0iWxy6jR3Llzcfz4cYSFhT3zX8na9B38N2V6fJKmfw9lMhnc3d0BAG3atEFkZCQ2bdqEuXPnVllXG7efMv09SdO33bVr15CTk4Nhw4YplpWXl+PChQvYsmULIiMjIZVKK71HrG3IOUAqJJPJ0Lp1a4SHhyuWVVRUIDw8/KnHd9u3b4+zZ89WWnbmzBm0b99enaXWSV36e1J5eTliY2Ph4OCgrjIblDZtP1W5ceOGxm4/QRAwd+5cHD58GD/++CPc3Nye+R5t24Z16fFJ2vY9rKioQGlpabWvadv2q05N/T1J07dd165dsW/fPuzZs0fxaNOmDQYNGoQ9e/ZUCT+AiNtQrVOs9dCBAweENm3aCLt37xbi4+OFzz77TOjYsaPitMWPPvpIWLx4sWL9ixcvCq1atRLWr18vxMfHC8uXL9foUziV7W/FihXCyZMnhVu3bglRUVHC9OnTBV9fXyEuLk6sFmpUUFAgXL9+Xbh+/brg4+MjbNiwQbh+/bpw+/ZtQRAEYfHixcJHH32kWP/xafCLFi0S4uPjhbCwMI0+DV7Z/jZs2CAcPnxYSE5OFmJiYoT58+cLLVq0EM6cOSNWCzX64osvhA4dOgjnzp2rdNpwcXGxYh1t/w7WpUdt+h4uXrxYOH/+vJCSkiLcuHFDWLx4sSCXy4VTp04JgqD920/Z/rRp2z3Nk2eBaco25CEwFRswYAByc3OxfPlyZGVloWXLlli3bp1iV156ejoMDP7Z8ebv74/Fixdj2bJlWLp0KZo1a4ZVq1bVOKFRTMr2d//+fXz22WfIysqCtbU1WrdujZ9++kljj+VHRUVh7NixiuehoaEAgKFDh2LhwoXIyspCenq64nU3NzesWbMGoaGh2LRpE5ycnDB//nwEBQU1eO21oWx/ZWVlWLRoETIzM2FqagofHx9s2LABXbt2bfDaa2Pbtm0AgDFjxlRaHhoaqtglr+3fwbr0qE3fw5ycHMyYMQN37tyBpaUl5HI51q9fj+7duwPQ/u2nbH/atO1qS1O2oUQQBEGtn0BERESkYTgHiIiIiPQOAxARERHpHQYgIiIi0jsMQERERKR3GICIiIhI7zAAERERkd5hACIiIiK9wwBERFQLcrkcR44cEbsMIlIRXgmaiDTezJkz8fPPP1dZHhgYiPXr14tQERFpOwYgItIKQUFBilt3PCaTyUSqhoi0HQ+BEZFWkMlkcHBwqPSwtrYG8Ojw1NatW/HWW2+hbdu26NOnD3777bdK74+JicHYsWPRtm1bdOnSBZ999hkKCwsrrbNz504MHDgQbdq0QWBgIObOnVvp9by8PEyZMgXt2rXDc889h6NHj6q3aSJSGwYgItIJ33zzDfr164dffvkFgwYNwvvvv4+EhAQAQFFREcaPHw9ra2vs3LkTy5Ytw5kzZzBv3jzF+7du3Yq5c+filVdewb59+/Dtt9+iadOmlT5j5cqVeP7557F371706NEDH374Ie7evduQbRKRijAAEZFWOH78OPz8/Co9Vq9erXi9f//+ePnll+Hh4YFp06ahTZs22Lx5MwBg//79KC0txaJFi+Dj44Nu3brh888/xy+//ILs7GwAwHfffYc33ngD48aNg4eHB9q2bYvXX3+9Ug1Dhw7FCy+8AHd3d7z//vsoKirC1atXG+z/AyJSHc4BIiKt0KVLF8yePbvSsseHwADAz8+v0mvt27dHdHQ0ACAhIQFyuRxmZmaK1/39/VFRUYGkpCRIJBLcuXMH3bp1q7EGuVyu+NnMzAwWFhbIzc2ta0tEJCIGICLSCqampnB3d1fL2MbGxrVaz8jIqNJziUSCiooKdZRERGrGQ2BEpBMiIiIqPb9y5Qq8vLwAAF5eXoiJiUFRUZHi9UuXLsHAwAAeHh6wsLCAi4sLwsPDG7JkIhIRAxARaYXS0lJkZWVVevz78NNvv/2GnTt3IikpCcuXL8fVq1cxevRoAMCgQYMgk8kwc+ZMxMbG4uzZs5g3bx4GDx4Me3t7AMB7772HDRs2YNOmTUhOTsa1a9cUc4iISPfwEBgRaYWTJ08iMDCw0jIPDw/F6e7vvfcefv31V8yZMwcODg5YsmQJvL29ATw6fLZ+/XosWLAAL730EkxNTfHcc89h5syZirGGDh2KBw8eYOPGjfjyyy9hY2OD/v37N1yDRNSgJIIgCGIXQURUH3K5HKtWrUJISIjYpRCRluAhMCIiItI7DEBERESkd3gIjIiIiPQO9wARERGR3mEAIiIiIr3DAERERER6hwGIiIiI9A4DEBEREekdBiAiIiLSOwxAREREpHcYgIiIiEjvMAARERGR3vl/An+UOnOkfPYAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# wee need the pipeline to run a model, so it is simpler to import it directly.\n",
    "# Pykeen lets you train a model with the minimal amount of custom parameters\n",
    "\n",
    "from pykeen.pipeline import pipeline\n",
    "\n",
    "# here we don't import the model, but let PyKEEN do the importing.\n",
    "pipeline_result_simple = pipeline(\n",
    "    random_seed=0,\n",
    "    model='ComplEx',\n",
    "    training=got_training,\n",
    "    testing=got_testing,\n",
    ")\n",
    "pipeline_result_simple.plot_losses()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pykeen.pipeline.api:Using device: None\n",
      "Training epochs on cuda:0:   0%|                                                               | 0/5 [00:00<?, ?epoch/s]\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  20%|█████▊                       | 1/5 [00:00<00:00,  7.42epoch/s, loss=16.1, prev_loss=nan]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  40%|███████████▏                | 2/5 [00:00<00:00,  7.70epoch/s, loss=16.1, prev_loss=16.1]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  60%|████████████████▊           | 3/5 [00:00<00:00,  7.75epoch/s, loss=15.7, prev_loss=16.1]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  80%|██████████████████████▍     | 4/5 [00:00<00:00,  7.24epoch/s, loss=14.9, prev_loss=15.7]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0: 100%|████████████████████████████| 5/5 [00:00<00:00,  7.31epoch/s, loss=14.7, prev_loss=14.9]\u001b[A\n",
      "INFO:pykeen.evaluation.evaluator:Starting batch_size search for evaluation now...\n",
      "INFO:pykeen.evaluation.evaluator:Concluded batch_size search with batch_size=159.\n",
      "Evaluating on cuda:0: 100%|███████████████████████████████████████████████████████| 159/159 [00:00<00:00, 2.76ktriple/s]\n",
      "INFO:pykeen.evaluation.evaluator:Evaluation took 0.07s seconds\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<Axes: title={'center': 'Losses Plot'}, xlabel='Epoch', ylabel='marginranking Loss'>"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAHHCAYAAABXx+fLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABcCUlEQVR4nO3deXhM9/4H8Pdkksm+ySKyiCwytiCxRyKWKKVq62Zvq1RpL7qht4v1hhZVS0tRSlB+qFqqaqnaYilCQmQPiSyyIatEcn5/qGkjEZlkJmeW9+t55rmZM2e+8/nck0nfzvmecySCIAggIiIi0iMGYhdARERE1NAYgIiIiEjvMAARERGR3mEAIiIiIr3DAERERER6hwGIiIiI9A4DEBEREekdBiAiIiLSOwxAREREpHcYgIiI1GT37t2Qy+VITU0VuxQiegIDEBHVy+P/yEdGRopdSoNZsWIF5HK54tGuXTsMGDAAX3/9NQoKClTyGfv27cPGjRtVMhYRVWUodgFERNpq9uzZMDMzQ1FREU6fPo3Vq1fj3Llz2LZtGyQSSb3G3r9/P+Li4vD666+rplgiqoQBiIiojvr164dGjRoBAEaMGIH33nsPv//+OyIiIuDn5ydydURUEx4CI6IGcf36dbz11lvw9/eHn58fxo0bh4iIiErrlJWVYeXKlXjuuefg6+uLLl26YMSIETh9+rRinaysLMyaNQs9evRAmzZtEBgYiHfeeafKPJs///wTI0eORPv27eHn54eJEyciLi6u0jq1Hau2unbtCgDPfP+WLVswcOBAxWfOmTMH9+/fV7w+ZswYHD9+HLdv31YcZuvdu3edaiKi6nEPEBGpXVxcHEaNGgVzc3O89dZbMDQ0xPbt2zFmzBiEhYWhXbt2AICVK1dizZo1ePnll9G2bVsUFBQgKioK165dQ/fu3QEA7733HuLj4zF69Gi4uLggNzcXp0+fRnp6OlxdXQEAe/bswcyZMxEYGIgPP/wQxcXF2LZtG0aOHImff/5ZsV5txlLGrVu3AAA2NjZPXWfFihVYuXIlAgICMGLECCQlJWHbtm2IjIzEtm3bYGRkhEmTJiE/Px8ZGRmYNWsWAMDc3FzpeoioBgIRUT3s2rVL8PHxEa5evfrUdSZPniy0bt1auHXrlmJZZmam4OfnJ4waNUqx7MUXXxQmTpz41HHu3bsn+Pj4COvWrXvqOgUFBULHjh2FTz/9tNLyrKwsoUOHDorltRnraZYvXy74+PgIiYmJQk5OjpCSkiL89NNPQps2bYSAgAChqKhIEIR//r9JSUkRBEEQcnJyhNatWwtvvvmmUF5erhgvLCxM8PHxEXbu3KlYNnHiRKFXr15K10ZEtcNDYESkVuXl5Th9+jRCQkLg5uamWO7o6IgXXngBFy9eVJw5ZWVlhbi4OCQnJ1c7lomJCYyMjHD+/Hncu3ev2nXOnDmD+/fvY+DAgcjNzVU8DAwM0K5dO5w7d67WYz1L//790a1bN/Tp0weff/453N3dsWbNGpiamj61trKyMowdOxYGBv/8+X355ZdhYWGBP//8s051EJHyeAiMiNQqNzcXxcXF8PDwqPKal5cXKioqkJ6ejubNm+M///kPJk+ejH79+sHHxweBgYEYPHgwWrRoAQCQyWT48MMPsWjRInTv3h3t2rVDz549MWTIEDg4OACAIjyNGzeu2nosLCxqPdazrFixAhYWFjA0NISTkxOaNm1a4/ppaWkAAE9Pz0rLZTIZ3NzccPv27Vp9LhHVHwMQEWmMTp064fDhwzh69ChOnz6NnTt34scff8ScOXPw8ssvAwBef/119O7dG0eOHMGpU6fwzTff4Pvvv8ePP/6IVq1aQRAEAMCXX35ZbZCRSqWKn5811rN07NhRcRYYEWkXHgIjIrVq1KgRTE1NkZSUVOW1xMREGBgYoEmTJoplNjY2GD58OJYuXYrjx49DLpdjxYoVld7XtGlTvPnmm/jhhx+wf/9+lJWV4YcffgAAxWE2Ozs7BAQEVHl06dKl1mOpmrOzs6LvfystLUVqaipcXFwUy+p7HSEiqhkDEBGplVQqRffu3XH06NFKp4dnZ2dj//796NChg+KwVF5eXqX3mpubo2nTpigtLQUAFBcX48GDB5XWadq0KczNzRXrBAUFwcLCAmvWrEFZWVmVenJzc2s9lqoFBATAyMgImzdvVuypAoCdO3ciPz8fwcHBimWmpqbIz89XSx1ExENgRKQiu3btwsmTJ6ssHzt2LKZNm4YzZ85g5MiRGDlyJKRSKbZv347S0lJ89NFHinUHDhyIzp07o3Xr1rCxsUFkZCQOHTqE0aNHA3g0v+f1119H//794e3tDalUiiNHjiA7OxsDBw4E8GiOz+zZs/Hxxx9j2LBhGDBgABo1aoS0tDT8+eef8Pf3x+eff16rsVStUaNGePvtt7Fy5Uq89dZb6N27N5KSkrB161b4+vrixRdfVKzbunVr/PrrrwgNDYWvry/MzMx4LSAiFWIAIiKV2LZtW7XLhw0bhubNm2PLli1YsmQJ1qxZA0EQ0LZtW3z11VeKawABjy4AeOzYMZw+fRqlpaVwdnbGtGnTMH78eACAk5MTBg4ciPDwcOzduxdSqRSenp5YtmwZ+vXrpxhn0KBBcHR0xPfff4/169ejtLQUjRs3RseOHTFs2DClxlK19957D40aNUJYWBhCQ0NhbW2NV155Be+//z6MjIwU640cORLR0dHYvXs3Nm7cCBcXFwYgIhWSCP/eD0tERESkBzgHiIiIiPQOAxARERHpHQYgIiIi0jsMQERERKR3GICIiIhI7zAAERERkd5hACIiIiK9wwBEREREeodXgq5BTk4+VH2ZSIkEsLOzVMvYmoD9aT9d71HX+wN0v0f2p/3U1ePjcWuDAagGggC1/fKpc2xNwP60n673qOv9AbrfI/vTfmL2yENgREREpHcYgIiIiEjvMAARERGR3mEAIiIiIr3DAERERER6hwGIiIiI9A4DEBEREekdBiAiIiLSOwxAREREpHcYgIiIiEjvMAARERGR3mEAIiIiIr3DANTAikvLUVJWLnYZREREeo13g29AiTmFGLXpEh5WCHC0kMHF2gTONqZwsTKBi40JXKwfPezMZZBIJGKXS0REpLMYgBqQqZEUDhYypN9/gDsFpbhTUIrLt+9XWc/Y0ADO1v8Eokc/mypCkqmRVITqiYiIdAcDUANqYmWCvRM6w9DMBFcTs5F6txi375UoHml3i5GZ/wAPHlYgKacISTlF1Y7TyMzon2BkY6oISi7WJnCwMIbUgHuPiIiIasIA1MAkEglszWVo3cQSrZwsq7z+sLwCGfkP/glGd0uQdu9RUEq7V4J7JQ+RW1SG3KIyRKbnV3m/kVSCJlYmlfYg/TskWRhzkxMREfG/hhrGUGoAVxtTuNqYVvt6fslDpN0rwe17T+w9+vtRVi7gVl4xbuUVV/t+axPDKofUHoUkEzS2NIEh9x4REZEeYADSMpYmhpCbWEDe2KLKa+UVArIKHij2HN3+156j2/dKkFtUhnslD3GvpADRmQVV3i+VAI2tTCodUnOxMVXsTbI2MeTkbCIi0gkMQDpEaiCBk5UJnKxM0MGt6utFpeWV9h6l/fsw2/0SPHhYodiTdKGa8c1l0iqH1B6HoyZWJpAZ8qoKRESkHRiA9IiZTApvB3N4O5hXea1CEJBTWPr3nqOSKofZsgpKUVhajtisQsRmFVZ5vwSAo6UxXKxN4NXYEnYm0n8OtVmboJGZEfceERGRxmAAIgCAgUQCBwtjOFgYo72rdZXXS8rKkX7/AW7fK6605+j230GpuKwCmfkPkJn/AJdS71V5v4mhwd9zjkyfmKBtAmcrE5jw1H4iImpADEBUKyZGUnjYmcHDzqzKa4IgIK+4THEo7W5ZBWLT7ilCUmb+A5Q8rEBCdhESsqs/td/eXFbpkNrjsORibQJ7CxkMuPeIiIhUiAGI6k0ikaCRmQyNzGRo62IFe3tLZGfnQxAevV5WXoGMv/ceVd5z9GjvUcGDcmQXliK7sBRX0qpeGFImlcD53xeE/PfeI2sTmMv4a0xERMrhfzlI7YykBnCzNYWbbfWn9t8vKatySO3xzxn5D1BaLiA5txjJucUA8qq839bU6Ik9R/+c5u/IC0MSEVE1GIBIdFYmRrAyMULLxtVcGLJCwJ38B5VC0b+vfXS3uAx5fz+uZVS9MKTUQIImVsb/hKJ/hSRnaxNYmRg1RItERKRhGIBIoxka/HP4q1PTqq8XPHj4z6TseyW4ffefU/zT7j+6MGTq3RKk3i0BcLfK+y2NDZ/Yc/TPRG0nK2MYSXlqPxGRLmIAIq1mYWwIH0cL+DhWvTBkhSAgq6C0yt6jx4Epp7AU+Q8e4sadAty4U/XCkAYSoLGlcaVDas7WJvDzqoCjkQSPTv4nIiJtxABEOstAIkFjS2M0tjSGv2vV14vLyqvsPfr38wcPK5B+/wHS7z/AXymVT+3v42OPT5/z4b3ViIi0FP96k94yNZLCy94cXvZVLwwpCAJyisoqHVJ7/L9X0+7jaGw2Yu4UYOELraq9LQkREWk2BiCiakgkEtiby2BvLkM7F+t/LQdSih7inc0XkXq3BG9uu4z3e3lhWNsmvNI1EZEW4QxPIiX5NbXFlrH+CPRshNJyAQuPxOOzX2+gsPSh2KUREVEtMQAR1YG1qRGWDGmN//TwgFQCHLqRhbFhlxFfzX3SiIhI8zAAEdWRgUSCMZ3csObVdnC0kOFWXjFe33oZeyMzIDy+DDYREWkkBiCiemrnYo2wMf7o2swWDx5WYN7vsZjzWwyKy8rFLo2IiJ6CAYhIBWzNZPhmWBtMDmwGAwlw4PodjNtyGYk5PCRGRKSJGICIVMRAIsEbXZri25fbws5chqScIowLu4xfr2eKXRoRET2BAYhIxTq42WDLGH90amqDkocV+OJgDOYfikUJD4kREWkMBiAiNbAzl2HFcF9M7OYOCYBfojLw5rYI3MwtErs0IiKCyAHowoULmDRpEgIDAyGXy3HkyJEq6yQkJGDSpEno0KED2rdvj+HDhyMtLa3GcQ8ePIj+/fvD19cXgwYNwp9//qmuFoieSmogwYQAd6x4yReNzIwQl1WIsWGX8fuNO2KXRkSk90QNQEVFRZDL5fjiiy+qff3WrVsYOXIkPD09sXnzZuzduxeTJ0+GsbHxU8e8dOkSPvjgA7z00kvYs2cP+vTpgylTpiA2NlZdbRDVqIu7LcLG+MPf1RpFZeX474EbWHgkDg8eVohdGhGR3hL1VhjBwcEIDg5+6utff/01evTogY8//lixrGnTpjWOuWnTJgQFBeGtt94CAEybNg1nzpxBWFgY5s6dq5rCiZTkYGGMVS+3xfdnkrHhXAp2XUlHVHo+Fg5qCVcbU7HLIyLSOxo7B6iiogLHjx9Hs2bNMH78eHTr1g0vv/xytYfJ/i0iIgLdunWrtCwwMBARERFqrJbo2QwNJJgc6IFvhrWBtYkhYu4UYPTmSzgWmyV2aUREekdjb4aak5ODoqIirF27FtOmTcOHH36IkydP4t1338WmTZvQuXPnat+XnZ0Ne3v7Ssvs7OyQnZ2tdA3quLfl4zF19b6Z7O/Zuns2wpax/vhk/w1cTbuPGfui8Zr/PUwN9oSRVPx/k3Abaj9d75H9aT919ajMeBobgCoqHs2P6NOnD15//XUAQMuWLXHp0iX89NNPTw1AqmRnZ6mVY2sC9lcze3tL7Jpih8WHYrDmRCJ+upSG6DuFWDnSH26NzFRUZf1wG2o/Xe+R/Wk/MXvU2ABka2sLQ0NDeHl5VVru5eWFixcvPvV99vb2Vfb25OTkVNkrVBs5OflQ9S2dJJJHG1wdY2sC9qecCZ1dIbczxeyDMbiSeg8DvjmJ2c/LEextV//B64jbUPvpeo/sT/upq8fH49aGxgYgmUwGX19fJCUlVVqenJwMFxeXp76vffv2OHv2rGKvEQCcOXMG7du3V7oGQYDafvnUObYmYH+1F+Rph7Ax/pi1LxrXMvLxwZ5rGNXBFe8GNYOhiIfEuA21n673yP60n5g9ijrhoLCwENHR0YiOjgYApKamIjo6WnGdn/Hjx+PgwYPYsWMHbt68ibCwMPzxxx8YMWKEYoyPP/4YS5YsUTwfO3YsTp48iR9++AEJCQlYsWIFoqKiMHr06IZtjkgJTaxMsPa1dhjh/yjcb7mYionbryLjfonIlRER6SZRA1BUVBSGDBmCIUOGAABCQ0MxZMgQLF++HADQt29fzJ49G+vWrcOgQYPwf//3f1i+fDk6duyoGCM9PR1ZWf+cRePv74/Fixdj+/btGDx4MA4dOoRVq1bBx8enQXsjUpaR1ADv9/LCohdbwcJYisj0+xi9+RJOJ+WKXRoRkc6RCIKu72Cru+xs9cwBsre3VMvYmoD9qUbq3WLM2heNG3cKAADjOrthUvdmMDRQ/2kh3IbaT9d7ZH/aT109Ph63NsQ/55aIqnC1McW6Ee3xUrsmAIAfz6dg8v9dRVbBA5ErIyLSDQxARBrK2NAAM0KaY8HAFjCXSXE59R5GbbqEc8l5YpdGRKT1GICINNxzLRzx4yg/NHcwR15xGd7bFYk1p5NRXqGj+8aJiBoAAxCRFnBvZIYfRrTHEF8nCADWnb2Fd3dFIruwVOzSiIi0EgMQkZYwMZLiv8/5YM7zcpgYGuCvW3cxevMl/HXrrtilERFpHQYgIi0zoFVjbBrtD087M+QUlmLKzqtYf/YmKnT1dBEiIjVgACLSQh52Ztg4yg8vtG6MCgFYffompu6KQl4RD4kREdUGAxCRljI1kuKL/nJ81s8HxoYGOHszD6M3X0JE6j2xSyMi0ngMQERa7sU2Ttg4yg/utqa4U1CKSTuu4MfzKTwkRkRUAwYgIh3gbW+OTaP90a+FA8oFYOXJJHyw5xruFpeJXRoRkUZiACLSEWYyKeYNaIFZfZtDJpXgVGIuRm++hMi0+2KXRkSkcRiAiHSIRCLBsLZN8MNIP7jZmCAz/wEmbL+CrRdTwdv+ERH9gwGISAfJHS2wabQ/QnzsUV4h4Ovjifh473XcL+EhMSIigAGISGdZGBvify+0xEe9vWEkleB4fA7GbL6E6xn5YpdGRCQ6BiAiHSaRSPCKnzPWvdYeztYmSLv/AG/9FIEdl2/zkBgR6TUGICI90MrJEmGj/dHT2w5l5QK+OpaAT/ZHo+DBQ7FLIyISBQMQkZ6wNDHEly+2wvSenpAaSHAkNhtjwy4h5k6B2KURETU4BiAiPSKRSDCygyvWvtoOTpbGSLlbgje3Xsbuq+k8JEZEeoUBiEgP+TpbIWyMPwI9G6G0XEDo4Th89usNFJWWi10aEVGDYAAi0lPWpkZYMqQ13gvygFQCHLqRhXFbLiE+q1Ds0oiI1I4BiEiPGUgkGNvZDatfaQdHCxmSc4sxbstl7PgrRezSiIjUigGIiNDe1RphY/zRtZktHjyswMc7r2LOwRiUlPGQGBHpJgYgIgIA2JrJ8M2wNngnsBkMJMC+a5kYt+UyknKKxC6NiEjlGICISMFAIsH4rk2x5a2usDOXITGnCOO2XMKv1zPFLo2ISKUYgIioim5edtg61h8dm9qguKwCXxyMwYLfY3lIjIh0BgMQEVXLzlyGlcN9MaFbU0gA7InMwJvbInAzl4fEiEj7MQAR0VNJDSSYGNAMK4b7wtbUCHFZhRi35TIOx2SJXRoRUb0wABHRM3VpZostY/3h52qNwtJyfLI/GouOxKH0YYXYpRER1QkDEBHVioOFMb59uS1e7+wGANh5JR3jt0Ug9W6xyJURESmPAYiIas3QQIIpQR5YNqwNrE0MceNOAcaEXcIfcdlil0ZEpBQGICJSWnePRggb44+2zlYoeFCOj/dex5I/ElBWzkNiRKQdGICIqE6crEyw5pW2GN3RFQDw06XbmPDTFaTfLxG5MiKiZ2MAIqI6M5QaYGqwJ5YMaQ0rE0Ncy8jH6M2XcCIhR+zSiIhqxABERPXWw8sOm0f7o7WTJe6XPMQHe65h+Z+JeMhDYkSkoRiAiEglnK1NsPa1dnjN3wUAsPmvVLy94yoy8x+IXBkRUVUMQESkMkZSA3zQywuLBrWEuUyKq2n3MWrTRZxJyhW7NCKiShiAiEjlevs4IGyMP1o4WuBeyUNM3R2Fb08l4WGFIHZpREQAGICISE1cbUyxbkR7vNSuCQBgw7kUTPm/q8gq4CExIhKfqAHowoULmDRpEgIDAyGXy3HkyJFKr8+cORNyubzSY/z48TWOWV5ejmXLlqF3795o27YtQkJCsGrVKggC/+VJ1NCMDQ0wI6Q5FgxsATMjKS6l3sPozZdw/mae2KURkZ4zFPPDi4qKIJfLMXz4cLz77rvVrhMUFITQ0FDFc5lMVuOYa9euxbZt27Bo0SJ4e3sjKioKs2bNgqWlJcaOHavS+omodp5r4Qi5owVm7Y9GXFYh3t0Zibe6NcX4ru6QGkjELo+I9JCoASg4OBjBwcE1riOTyeDg4FDrMS9fvow+ffqgZ8+eAABXV1ccOHAAV69erU+pRFRP7o3M8MOI9lj8RwJ+iczA2vBbiLh9H/MGtICdec3/sCEiUjVRA1BtnD9/Ht26dYOVlRW6du2KadOmwdbW9qnr+/n5YceOHUhKSoKHhwdu3LiBixcvYubMmUp/tkQN/zB9PKY6xtYE7E/7qbNHU5kUn/XzQQc3a/zv9zhcuHUXozZfwv9eaIEObjaq/8BqcBtqP/an/dTVozLjSQQNmRwjl8uxatUqhISEKJYdOHAAJiYmcHV1RUpKCpYuXQozMzNs374dUqm02nEqKiqwdOlSrFu3DlKpFOXl5Zg+fTrefvvthmqFiGohLjMfk7dcQtydAhhIgA+ek+OdYC8Y8JAYETUAjd4DNHDgQMXPjydBh4SEKPYKVefgwYPYt28flixZAm9vb0RHRyM0NBSOjo4YOnSoUp+fk5MPVcdDiQSws7NUy9iagP1pv4bq0VYK/PBaOyw6Go/91zLx1aEYnI65g7kDWsDGzEhtn8ttqP3Yn/ZTV4+Px60NjQ5AT3Jzc4OtrS1u3rz51AD05ZdfYuLEiYrwJJfLkZaWhjVr1igdgAQBavvlU+fYmoD9ab+G6NHESIov+svh52qNL4/G40xyHkZuuoj/vdAS7Vys1frZ3Ibaj/1pPzF71KrrAGVkZODu3bs1ToouKSmB5ImDgFKplKfBE2mwF9s4YeNIP7jbmuJOQSne3n4Fmy+koILfWyJSE1EDUGFhIaKjoxEdHQ0ASE1NRXR0NNLS0lBYWIhFixYhIiICqampCA8Px+TJk+Hu7o6goCDFGOPGjUNYWJjiea9evbB69WocP34cqampOHz4MDZs2FBpbhERaR5vB3P8ONoP/Vo4oFwAlp9Iwgd7ruFecZnYpRGRDhL1EFhUVFSla/M8vt7P0KFDMXv2bMTGxmLPnj3Iz8+Ho6MjunfvjqlTp1a6FlBKSgry8v65qNqnn36Kb775BnPmzEFOTg4cHR3x6quvYsqUKQ3XGBHVibnMEPMGtIC/qzWW/JGAU4m5GL35EkIHtUSbJlZil0dEOkRjzgLTRNnZ6pkEbW9vqZaxNQH7036a0mNMZgFm7b+OlLslMDSQ4L0eHhjh71LlELeyNKU/ddL1Htmf9lNXj4/HrQ2tmgNERPpD3tgCm0b7o4+PPR5WCPj6eCI+3nsd+SUPxS6NiHQAAxARaSwLY0OEvtASH/X2gqGBBMfjczA67BKiM/PFLo2ItBwDEBFpNIlEglf8XLB+RHs4Wxkj7V4Jxm+LwI7LaTy7k4jqjAGIiLRCKydLbB7jj2AvO5SVC/jqWDw+2X8DBQ94SIyIlMcARERaw8rECF8NboXpPT0hNZDgSGwWxm25jNg7BWKXRkRahgGIiLSKRCLByA6uWPtqOzS2NMatvGK8sfUyfr6azkNiRFRrDEBEpJV8na0QNsYfgZ6NUFou4H+H4/D5wRgUlZaLXRoRaQEGICLSWjamRlgypDXeC/KAVAL8Fn0H47ZcQnx2odilEZGGYwAiIq1mIJFgbGc3rH6lHRwsZEjOLcbrWy5jX1SG2KURkQZjACIindDe1Rpbxvijq7stHjyswNxDsZjzWwxKynhIjIiqYgAiIp1haybDN8PbYFJ3dxhIgP3XMvH61stIzikSuzQi0jAMQESkUwwkEozv6o5VL7WFnbkMCdlFGLvlEg5GZ4pdGhFpEAYgItJJHZvaIGyMPzq6WaO4rAKf/xqD/x2O5SExIgLAAEREOszeXIaVL7XFW12bQgLg56sZeHNrBJJ4lhiR3mMAIiKdJjWQ4O3uzbBiuC9sTY0Qm1WIF1eeQurdYrFLIyIRMQARkV7o0swWW8b6o2VjC+SXPMTiYwm8cjSRHlM6AJ04cQJ//fWX4vmWLVswePBgfPDBB7h3755KiyMiUiUHC2PMG9ACRlIJTiXm4kRCjtglEZFIlA5AX331FQoLHx0/j4mJwcKFCxEcHIzU1FQsXLhQ5QUSEalSMzszTAjyBAAs+SOBk6KJ9JTSASg1NRVeXl4AgN9//x29evXC+++/j88//xwnTpxQeYFERKr2bm9vOFkaI/3+A/xw7pbY5RCRCJQOQEZGRigpKQEAnDlzBt27dwcAWFtbo6CgQLXVERGpgZnMEB/2fvQPuc0XUnmhRCI9pHQA8vf3R2hoKFatWoXIyEj07NkTAJCcnAwnJydV10dEpBbB3nbo7tEIDysEfHksnhOiifSM0gHo888/h6GhIQ4dOoQvvvgCjRs3BvBocnRQUJDKCyQiUgeJRIIPe3vB2NAAF27dxeGYLLFLIqIGZKjsG5ydnbFmzZoqyz/55BOVFERE1FBcbUwxrrMbvj9zE18fT0SARyNYGCv9Z5GItJDSe4CuXbuGmJgYxfMjR45g8uTJWLp0KUpLS1VaHBGRuo3t5AZXGxNkF5ZibfhNscshogZSp0NgycnJAICUlBS8//77MDU1xW+//YavvvpK1fUREamVsaEBPurtDQDYfuk24rN4mwwifaB0AEpOTkbLli0BAAcPHkSnTp2wZMkShIaG4vfff1d5gURE6hbg0Qi9m9ujXAAWHolDBSdEE+k8pQOQIAioqKgAAISHh6NHjx4AgCZNmiAvL0+11RERNZDpPT1hamSAK2n3ceBaptjlEJGaKR2A2rRpg++++w579uzBhQsXFKfBp6amwt7eXtX1ERE1CCcrE7zV1R0AsOJEEu6XlIlcERGpk9IB6JNPPsH169cxb948TJo0Ce7uj/5gHDp0CH5+fiovkIiooYzo4AIPOzPkFZfh21PJYpdDRGqk9PmeLVq0wL59+6os//jjj2FgwJvLE5H2MpIaYEYfb0zacRW7r6TjxTZOaOVkKXZZRKQGdU4sUVFR+OWXX/DLL7/g2rVrMDY2hpGRkSprIyJqcB3cbNC/pSMEPJoQXV7BCdFEukjpPUA5OTmYNm0aLly4ACsrKwDA/fv30aVLF3z99ddo1KiRyoskImpIU4M9cTIhB9GZBdgTmY7h7ZzFLomIVEzpPUDz5s1DUVERDhw4gPPnz+P8+fPYv38/CgoKMH/+fHXUSETUoOzNZXinezMAwLenkpFXxIu8EukapQPQyZMn8cUXX8DLy0uxzNvbG1988QVOnDih0uKIiMQyvL0zfBzMcb/kIVacSBK7HCJSMaUDUEVFRbVzfQwNDRXXByIi0naGBhLMCGkOANh3LRNXbt8TuSIiUiWlA1DXrl2xYMECZGb+c6GwzMxMhIaGolu3biotjohITG2drTDY1wkAsOhoPB5yQjSRzqjTvcAKCgrQp08fhISEICQkBH369EFBQQE+/fRTddRIRCSadwM9YG1iiLisQuy4fFvscohIRZQ+C6xJkyb4+eefcebMGSQmJgIAvLy8EBAQoPLiiIjEZmNmhClBHvjf4Th8f+Ym+sod4GBhLHZZRFRPdboOkEQiQffu3TFmzBiMGTMGAQEBSEhIQL9+/ZQa58KFC5g0aRICAwMhl8tx5MiRSq/PnDkTcrm80mP8+PHPHDczMxMffvghunTpgrZt22LQoEGIjIxUqjYioscG+zqhTRNLFJaWY9nxRLHLISIVUHoP0NOUlZXh1q1bSr2nqKgIcrkcw4cPx7vvvlvtOkFBQQgNDVU8l8lkNY557949jBgxAl26dMHatWtha2uLmzdvwtraWqnaiIgeM5BIMKOPN8ZtuYzfY7Iw2NcJnd1txS6LiOpBZQGoLoKDgxEcHFzjOjKZDA4ODrUec+3atXBycqoUmtzc3OpcIxERALRobImX2jljR0Qavjwaj23jOsBIytv/EGkrUQNQbZw/fx7dunWDlZUVunbtimnTpsHW9un/8jp27BgCAwPxn//8BxcuXEDjxo0xcuRIvPLKK0p/tkRSn8prHlMdY2sC9qf9dL3H+vT3TmAzHI3Nws28Ymy5mIo3ujRVbXEqwm2o3XS9P0B9PSoznkYHoKCgIPTt2xeurq5ISUnB0qVLMWHCBGzfvh1SqbTa96SkpGDbtm144403MGnSJERGRmL+/PkwMjLC0KFDlfp8Ozv13QRRnWNrAvan/XS9x7r0Zw/g00GtMH37Faw/ewsjAjzgamum+uJUhNtQu+l6f4C4PdY6AHXq1AmSGqLVw4cPVVLQvw0cOFDx8+NJ0CEhIYq9QtURBAFt2rTB+++/DwBo1aoV4uLi8NNPPykdgHJy8iGo+LIfEsmjDa6OsTUB+9N+ut5jffsLdLWCv6s1LqXew6e7rmLxkNaqL7KeuA21m673B6ivx8fj1katA9Ann3xS54JUxc3NTTGp+WkByMHBodJtOgDA09MThw4dUvrzBAFq++VT59iagP1pP13vse79SfBxH2+M2nwJx+NzcDIhB4GedqouTyW4DbWbrvcHiNtjrQOQsntP1CEjIwN3796tcVK0v78/kpIq37cnOTkZLi4u6i6PiPSEl705Rvi7IOyvVHx1LAEd3WxgYlT9YXki0kyinsJQWFiI6OhoREdHAwBSU1MRHR2NtLQ0FBYWYtGiRYiIiEBqairCw8MxefJkuLu7IygoSDHGuHHjEBYWVun5lStXsHr1aty8eRP79u3Djh07MHLkyAbvj4h014Ru7nC0kCHtXgl+PJ8idjlEpCRRJ0FHRUVh7NixiuePT10fOnQoZs+ejdjYWOzZswf5+flwdHRE9+7dMXXq1ErXAkpJSUFeXp7iedu2bbFy5UosXboUq1atgqurKz755BO8+OKLDdcYEek8M5kU7/fywsx90dh0IQXPt2qMpramYpdFRLUkEQRdP8JYd9nZ6pkEbW9vqZaxNQH703663qMq+xMEAf/ZFYWzN/PQtZktlg9rU+PJIg2F21C76Xp/gPp6fDxubfAqXkREdSSRSPBRH28YSSU4m5yHP+KyxS6JiGqJAYiIqB6a2ppibKdHV5tf8kcCikrLRa6IiGpD6TlA/77FxL9JJBIYGxujadOm6NOnD2xsbOpbGxGRVni9sxsORt9B2r0SrD97E+/18BS7JCJ6BqUD0PXr13H9+nVUVFTAw8MDAJCUlASpVApPT09s3boVixYtwtatW+Ht7a3ygomINI2JkRQf9vLC+3uuYcvF2xjQqjG87M3FLouIaqD0IbA+ffogICAAJ0+exO7du7F7926cOHECAQEBGDhwIE6cOIGOHTs+dU8REZEuCvKyQ7CXHcorBHx5NB48v4RIsykdgNavX4+pU6fCwsJCsczS0hLvvfce1q1bB1NTU0yZMgVRUVEqLZSISNN90NsLxoYGuJR6D7/duCN2OURUA6UDUEFBAXJycqosz83NRUFBAQDAysoKZWVl9a+OiEiLNLEywfiuj+4Qv+x4IvJLVH+PRCJSDaUDUO/evfHJJ5/g8OHDyMjIQEZGBg4fPoz//ve/CAkJAQBcvXoVzZo1U3WtREQab1QHVzS1NUVuURnWnEkWuxwiegqlJ0HPnTsXoaGhmD59OsrLH53uKZVKMXToUMyaNQvAo5uPLliwQLWVEhFpAZmhAT7u4413d0bi/yLSMKi1E+SNLZ79RiJqUEoHIHNzc8yfPx+zZs1CSsqj+9+4ubnB3PyfMx5atmypugqJiLRMF3db9JU74HBMFhYdjcO6Ee1hoAFXiCaif9T5Qojm5uZo0aIFWrRoUSn8EBERMC3YE2ZGUkSm52NvZIbY5RDRE5TeA1RUVITvv/8eZ8+eRU5ODioqKiq9fvToUZUVR0SkrRwtjTExwB3L/kzEypNJ6NncHjamRmKXRUR/UzoAffrppzh//jwGDx4MBwcHjbjxHxGRJnrVzxn7r2UiPrsQq04m4b/P+YhdEhH9TekAdOLECaxZswYdOnRQRz1ERDrDUGqAGX28MWH7FeyJzMCLbZzg62wldllEhDrMAbKysuJ9voiIaqm9qzUGtm4MAFh0NB7lFbxCNJEmUDoATZ06Fd988w2Ki4vVUQ8Rkc75Tw8PWBobIuZOAXZdSRO7HCJCHQ6BbdiwAbdu3UJAQABcXV1haFh5iJ9//lllxRER6YJGZjJMDmyGRUfj8d3pZPT2cYC9uUzssoj0mtIB6PHVnomIqPaGtm2CvVEZiM4swPI/EzF3QAuxSyLSa0oHoHfffVcddRAR6TSpgQQzQprjjS2XcTD6Dgb7OqGDm43YZRHprTpfCJGIiJTT2skSw9o1AQB8eTQeD8srnvEOIlKXWu0B6ty5M3777Tc0atQInTp1qvHaP+fPn1dZcUREuuad7s1wNDYbiTlF2HbpNsZ0chO7JCK9VKsANGvWLFhYWCh+5sUPiYjqxtrUCO/18MC8Q7FYG34Tz7VwRGNLY7HLItI7tQpAQ4cOVfw8bNiwp65XUlJS/4qIiHTcC60bY29kBq6k3cfXxxOwcFArsUsi0jtKzwGaP39+tcuLioowYcKEehdERKTrDCQSzAjxhlQCHI3NRnhyrtglEekdpQPQ8ePHsXz58krLioqK8NZbb6G8vFxlhRER6bLmDhZ4xc8FAPDV0Xg8eMgJ0UQNSekA9MMPP2DHjh3YuHEjAKCgoABvvvkmJBIJ1q1bp+r6iIh01sQAd9iby5BytwSbL6SIXQ6RXlE6ADVt2hTr1q3Dd999h02bNmH8+PEwMjLC2rVrYWZmpo4aiYh0koWxIab39AQAbDyfgtv3eIshooZSp+sAtWjRAqtXr8bXX38NExMThh8iojrqK3dAx6Y2ePCwAouPJUAQeLNUooZQq7PAhgwZUu2p7zKZDHfu3MGIESMUy3gvMCKi2pNIJJjR2xsjNl3EqcRcnEjIQbC3vdhlEem8WgUg3v+LiEh9mtmZYVRHV/x4PgVL/khAF3dbmBhJxS6LSKfVKgDx/l9EROo1vmtTHIq+g/T7D/DDuVuYHOghdklEOq3O9wIrLS1FRkYG0tLSKj2IiEh5pkZSfNDLCwCw+UIqknOKRK6ISLcpfTf4pKQk/Pe//8Xly5crLRcEARKJBNHR0SorjohInwR726G7RyOcTsrFl8fiseolX956iEhNlA5As2bNgqGhIVavXg1HR0d+OYmIVEQikeDD3l7468e7uHDrLg7HZOG5Fo5il0Wkk5QOQDdu3MCuXbvg5eWljnqIiPSaq40pxnV2w/dnbuLr44kI8GgEC2Ol/1QT0TMoPQfIy8sLeXl56qiFiIgAjO3kBlcbE2QXlmJt+E2xyyHSSUoHoA8//BCLFy/GuXPnkJeXh4KCgkoPIiKqH2NDA3zU2xsAsP3SbcRnFYpcEZHuUToAvfHGG4iIiMDrr7+OgIAAdOrUCZ06dULHjh3RqVMnpca6cOECJk2ahMDAQMjlchw5cqTS6zNnzoRcLq/0GD9+fK3H//777yGXy7FgwQKl6iIiEluARyP0bm6PcgFYeCQOFbxCNJFKKX1gedOmTSr78KKiIsjlcgwfPvyp1xoKCgpCaGio4rlMJqvV2FevXsVPP/0EuVyuklqJiBra9J6eCE/OxZW0+zhwLROD2jiJXRKRzlA6AHXu3FllHx4cHIzg4OAa15HJZHBwcFBq3MLCQnz00UeYP38+vvvuu/qUSEQkGicrE7zV1R0rTiZhxYkkBHvbwcrESOyyiHRCnS+EWFxcjISEBNy4caPSQ9XOnz+Pbt26oV+/fvjiiy9qNQF77ty5CA4ORkBAgMrrISJqSCM6uMCjkRnyisvw7alkscsh0hlK7wHKzc3FrFmzcOLEiWpfV+WFEIOCgtC3b1+4uroiJSUFS5cuxYQJE7B9+3ZIpdXfJ+fAgQO4fv06du7cWe/PV8cljh6PqauXT2J/2k/Xe9S2/mSGBpgR4o1JO65i95V0DPZ1Qisnyxrfo209Kov9aT919ajMeEoHoAULFuD+/fvYsWMHxo4di5UrVyI7OxvfffcdZs6cqexwNRo4cKDi58eToENCQhR7hZ6Unp6OBQsW4IcffoCxsXG9P9/OruY/Mpo6tiZgf9pP13vUpv7621tiSGw29kSkYfHxRPw8uTukBs/+S69NPdYF+9N+YvaodAA6d+4cvv32W/j6PrpEu7OzM7p37w4LCwusWbMGPXv2VEOZj7i5ucHW1hY3b96sNgBdu3YNOTk5GDZsmGJZeXk5Lly4gC1btiAyMvKpe46qk5OTD1WfeCGRPNrg6hhbE7A/7afrPWprf5O6NcXh65m4mnoPa4/F4qX2zk9dV1t7rC32p/3U1ePjcWtD6QBUVFSERo0aAQCsra2Rm5sLDw8P+Pj44Pr168oOp5SMjAzcvXv3qZOiu3btin379lVaNmvWLHh6emLChAlKhR8AEASo7ZdPnWNrAvan/XS9R23rz85Mhne6N8PiPxLw7alk9G5uD1uzms+K1bYelcX+tJ+YPSo9CdrDwwNJSUkAHh2W2r59OzIzM/HTTz/V6Wyt6Ohoxbyh1NRUREdHIy0tDYWFhVi0aBEiIiKQmpqK8PBwTJ48Ge7u7ggKClKMMW7cOISFhQEALCws4OPjU+lhZmYGGxsb+Pj4KNsqEZFGGd7eGT4O5rhf8hArTiSJXQ6RVlN6D9DYsWORlZUFAHj33Xfx1ltvYd++fTAyMsLChQuVGisqKgpjx45VPH98vZ+hQ4di9uzZiI2NxZ49e5Cfnw9HR0d0794dU6dOrXQtoJSUFN6ag4j0gqGBBDNCmmP8tgjsu5aJwb5OaOdiLXZZRFpJIgj12/lUXFyMxMRENGnSRHFoTFdkZ6tnDpC9vaVaxtYE7E/76XqPutDf/EOx+CUqA80dzLFptD8Mn5gQrQs91oT9aT919fh43NpQ6hBYWVkZQkJCkJCQoFhmamqK1q1b61z4ISLSVO8GecDaxBBxWYXYcfm22OUQaSWlApCRkREePHigrlqIiKgWbMyMMCXIAwDw/ZmbyCrg32UiZSk9CXrUqFFYu3YtHj58qI56iIioFgb7OqFNE0sUlpZj2fFEscsh0jpKT4KOjIxEeHg4Tp06BblcDlNT00qvr1y5UmXFERFR9QwkEszo441xWy7j95gsDPZ1Qmd3W7HLItIaSgcgKysr9OvXTx21EBGRElo0tsRL7ZyxIyINXx6Nx7ZxHWAkrfMtHon0itIB6PGp6kREJL5J3ZvhSGwWbuYVI+yvVLzRpanYJRFpBf5TgYhIi1maGGJqsCcAYP3ZW0i/XyJyRUTaQek9QNnZ2Vi0aBHCw8ORm5uLJy8jpMq7wRMR0bM939IReyIzcDn1Hpb+kYDFQ1qLXRKRxlM6AM2cORPp6emYPHkyHB0d1VETEREpQfL3hOhRmy/heHwOTiXkYEgtLwZHpK+UDkAXL17E1q1b0bJlS3XUQ0REdeBlb44R/i4I+ysVXx1LQH9/N7FLItJoSs8BatKkSZXDXkREJL4J3dzhaCHD7Xsl+PZ4wrPfQKTHlA5An3zyCZYsWYLU1FR11ENERHVkJpPi/V5eAIDVfybgVl6xyBURaS6lD4FNnz4dxcXF6Nu3L0xMTGBkZFTp9fPnz6usOCIiUk7v5vbo2swWZ5Pz8NXReHwzrA0kEsmz30ikZ5QOQJ988ok66iAiIhWQSCT4uI83Xtt4EeHJefgjLhu9fRzELotI4ygdgIYOHaqOOoiISEWa2ppiUrAnlh+Lx5I/EtC1WSOYyaRil0WkUWo1B6igoKDSzzU9iIhIfJN7ecPF2gR3Ckqx/uxNscsh0ji12gPUqVMnnDp1CnZ2dujYsWO1x5MFQYBEIuGFEImINICJkRQf9vbC9J+vYcvF2xjQqjG87M3FLotIY9QqAP3444+wtrYGAGzatEmtBRERkWoEedmhh5cdTiTk4Muj8Vj9SltOiCb6W60CUOfOnav9mYiINNsHvbxw7mYeLqXew2837uD5lo3FLolIIyg9CfrGjRvVLpdIJDA2NoazszNkMlm9CyMiovpztjbB+K5N8e2pZCw7nohADztYmij9p59I5yj9LRgyZEiNu1ANDQ0xYMAAzJ07F8bGxvUqjoiI6m9UB1fsv5aJW3nFWHMmGR/29ha7JCLRKX0l6JUrV8Ld3R1z587Fnj17sGfPHsydOxceHh5YsmQJFixYgLNnz2LZsmVqKJeIiJQlMzTAx30ehZ7/i0hDTCbP2CVSeg/Q6tWr8d///hdBQUGKZXK5HE5OTvjmm2+wc+dOmJmZYeHChZgxY4ZKiyUiorrp4m6LvnIHHI7JwqKjcVg3oj0MOCGa9JjSe4BiY2Ph7OxcZbmzszNiY2MBAC1atEBWVlb9qyMiIpWZFuwJMyMpItPzsTcyQ+xyiESldADy9PTE2rVrUVpaqlhWVlaGtWvXwtPTEwCQmZkJOzs71VVJRET15mhpjIkB7gCAlSeTcLe4TOSKiMSj9CGwzz//HO+88w6Cg4Mhl8sBPNorVF5ejjVr1gAAUlJSMHLkSNVWSkRE9faqnzP2X8tEfHYhVp1Mwn+f8xG7JCJRKB2A/P39cfToUezbtw/JyckAgP79++OFF16AhYUFgEdnihERkeYxlBpgRh9vTNh+BXsiM/BiGyf4OluJXRZRg1MqAJWVleH555/HmjVrMGLECHXVREREatTe1RoDWzfGgWuZWHQ0Hj+O8oPUgBOiSb8oNQfIyMgIDx48UFctRETUQP7TwwOWxoaIuVOAXVfSxC6HqMEpPQl61KhRWLt2LR4+fKiOeoiIqAE0MpNhcmAzAMB3p5ORXVha8xuIdIzSc4AiIyMRHh6OU6dOQS6Xw9TUtNLrK1euVFlxRESkPkPbNsHeqAxEZxZg+Z+JmDughdglETUYpQOQlZUV+vXrp45aiIioAUkNJJgR0hxvbLmMg9F3MNjXCR3cbMQui6hBKB2AQkND1VEHERGJoLWTJYa1a4JdV9Lx5dF4bBnjD0Op0rMjiLQOf8uJiPTcO92bwcbUCIk5Rdh26bbY5RA1CKX3AAHAb7/9hoMHDyI9PR1lZZWvJPrzzz+rpDAiImoY1qZGeK+HB+YdisXa8Jt4roUjGlsai10WkVopvQdo06ZNmDVrFuzt7XH9+nX4+vrCxsYGKSkp6NGjhzpqJCIiNXuhdWO0dbZCcVkFvj6eIHY5RGqndADaunUr5s2bh88++wxGRkaYMGECNmzYgDFjxiA/P18dNRIRkZoZSCSYGeINqQQ4GpuN8ORcsUsiUiulA1B6ejr8/PwAACYmJigsLAQADB48GAcOHFBtdURE1GCaO1jgFT8XAMBXR+Px4GGFyBURqY/SAcje3h737t0DADRp0gQREREAgNTUVAiCoNRYFy5cwKRJkxAYGAi5XI4jR45Uen3mzJmQy+WVHuPHj69xzDVr1mD48OHw8/NDt27dMHnyZCQmJipVFxGRvpoY4A57cxlS7pZg84UUscshUhulA1DXrl1x7NgxAMDw4cMRGhqKN954A9OnT0dISIhSYxUVFUEul+OLL7546jpBQUE4deqU4rF06dIaxzx//jxGjRqFHTt2YMOGDXj48CHGjx+PoqIipWojItJHFsaGmBbsCQDYeD4Ft+8Vi1wRkXoofRbYvHnzUFHxaLfoqFGjYGNjg8uXL6N379549dVXlRorODgYwcHBNa4jk8ng4OBQ6zHXr19f6fnChQvRrVs3XLt2DZ06dVKqPiIiffRcCwfsicrAX7fuYvGxBCwd0hoSCW+WSrpF6QBkYGAAA4N/dhwNHDgQAwcOVGlR/3b+/Hl069YNVlZW6Nq1K6ZNmwZbW9tav//xxGxra2ulP1sd3/fHY+rq3xL2p/10vUdd7w+of48SiQQz+3jjtR8v4lRiLk4k5qCnt73qCqwnXd+Gut4foL4elRlPIig7cQfAgwcPEBMTg5ycHMXeoMf69Omj7HAAALlcjlWrVlU6jHbgwAGYmJjA1dUVKSkpWLp0KczMzLB9+3ZIpdJnjllRUYF33nkH9+/fx7Zt2+pUFxGRvlr02w18dzwBLjamOPJ+MExlz/67S6QtlN4DdOLECcyYMQN5eXlVXpNIJIiOjlZJYQAq7Vl6PAk6JCREsVfoWebMmYO4uDhs3bq1Tp+fk5MP5eNhzSQSwM7OUi1jawL2p/10vUdd7w9QXY8j2zrh54upuH23GF8euIYpQR6qK7IedH0b6np/gPp6fDxubSgdgObPn4/+/ftjypQpsLdv2F2ibm5usLW1xc2bN58ZgObOnYvjx48jLCwMTk5Odfo8QYDafvnUObYmYH/aT9d71PX+gPr3aGIkxQe9vPDR3uvYfCEVA1o2RjM7M9UVWE+6vg11vT9A3B6VPgssOzsbb7zxRoOHHwDIyMjA3bt3a5wULQgC5s6di8OHD+PHH3+Em5tbA1ZIRKRbgr3t0N2jER5WCPjyWLzSlzsh0lRKB6B+/frh3LlzKvnwwsJCREdHKw6bpaamIjo6GmlpaSgsLMSiRYsQERGB1NRUhIeHY/LkyXB3d0dQUJBijHHjxiEsLEzxfM6cOdi7dy+WLFkCc3NzZGVlISsrCyUlJSqpmYhIn0gkEnzY2wsyqQQXbt3F4ZgssUsiUgmlD4F9/vnnmDp1Ki5evAgfHx8YGlYeYuzYsbUeKyoqqtL6oaGhAIChQ4di9uzZiI2NxZ49e5Cfnw9HR0d0794dU6dOhUwmU7wnJSWl0nykx5Odx4wZU+mzQkNDMWzYsNo3SkREAABXG1O83qUpvj9zE18fT0SARyNYGNfpXtpEGkPp3+D9+/fj9OnTkMlkOH/+fKXXJBKJUgGoS5cuiImJeerrT17TpzqPL8r4WE3jERFR3Yzt5IZfr2ci9W4J1obfxPSeXmKXRFQvSgegZcuW4b333sPEiRMrXQ+IiIh0l7GhAT7q7Y2pu6Ow/dJtDGrtBG8Hc7HLIqozpRNMWVkZBgwYwPBDRKRnAjwaoVdze5QLwKKjcZwQTVpN6RQzZMgQ/Prrr+qohYiINNz7PT1hamSAiNv3ceB6ptjlENWZ0ofAKioqsG7dOpw6dQpyubzKJOhZs2aprDgiItIsTlYmeKurO1acTMLyP5PQw8sOViZGYpdFpDSlA1BMTAxatmwJAIiNja30Gm+WR0Sk+0Z0cMH+a5lIyi3Ct6eSMTOkudglESlN6QC0efNmddRBRERawkhqgBkh3pi04yp2X0nHi22c0MqpdrcfINIUnMlMRERK6+Bmg/4tHSEAWHgkDuUVnBBN2oUBiIiI6mRqsCfMZVJEZxZgT2S62OUQKYUBiIiI6sTeXIZJ3ZsBAL49lYy8olJxCyJSAgMQERHV2UvtneHjYI77JQ+x4kSS2OUQ1RoDEBER1ZmhgQQz/j4LbN+1TFy5fU/kiohqhwGIiIjqpa2zFQa3cQIALDoaj4ecEE1agAGIiIjq7d0gD1ibGCIuqxA7Lt8WuxyiZ2IAIiKierMxM8KUIA8AwPdnbiKr4IHIFRHVjAGIiIhUYrCvE9o0sURhaTmWHU8UuxyiGjEAERGRShhIJJjRxxsGEuD3mCycv5kndklET8UAREREKtOisSVeaucMAPjyaDzKyitEroioegxARESkUpO6N0MjMyPczCtG2F+pYpdDVC0GICIiUilLE0NMDfYEAKw/ewvp90tEroioKgYgIiJSuedbOsLP1RoPHlZg6R8JYpdDVAUDEBERqZxEIsHHfbwhNZDgeHwOTiXmiF0SUSUMQEREpBbe9uYY4e8CAPjqWAJKyspFrojoHwxARESkNhO6ucPRQoa0eyX48XyK2OUQKTAAERGR2pjJpHi/lxcAYNOFFNzKKxa5IqJHGICIiEiteje3R1d3W5SWC/jqWDwEgTdLJfExABERkVpJJBJ81McbRlIJzibn4Y+4bLFLImIAIiIi9Wtqa4oxndwAAEv+SEBRKSdEk7gYgIiIqEG80dkNztYmuFNQivVnb4pdDuk5BiAiImoQJkZSfPj3hOgtF28jIbtQ5IpInzEAERFRgwnyskMPLzuUVwj48ignRJN4GICIiKhBfdDLC8aGBriUeg+/3bgjdjmkpxiAiIioQTlbm2B816YAgGXHE5Ff8lDkikgfMQAREVGDG9XBFU1tTZFbVIY1Z5LFLof0EAMQERE1OJmhAT7u4w0A+L+INMRkFohcEekbBiAiIhJFF3db9JU7oEIAFh2NQwUnRFMDYgAiIiLRTAv2hJmRFJHp+dgbmSF2OaRHGICIiEg0jpbGmBjgDgBYeTIJd4vLRK6I9AUDEBERiepVP2d42ZvhXslDrDqZJHY5pCdEDUAXLlzApEmTEBgYCLlcjiNHjlR6febMmZDL5ZUe48ePf+a4W7ZsQe/eveHr64uXX34ZV69eVVcLRERUT4ZSA8zs0xwA8EtkBiLT7otcEekDUQNQUVER5HI5vvjii6euExQUhFOnTikeS5curXHMX3/9FaGhoZgyZQp+/vlntGjRAuPHj0dOTo6qyyciIhVp72qNga0bQwCw6Gg8yis4IZrUS9QAFBwcjOnTp6Nv375PXUcmk8HBwUHxsLa2rnHMDRs24JVXXsHw4cPh7e2NOXPmwMTEBLt27VJ1+UREpEL/6eEBS2NDxNwpwK4raWKXQzrOUOwCnuX8+fPo1q0brKys0LVrV0ybNg22trbVrltaWopr167h7bffViwzMDBAQEAALl++rPRnSyR1LvuZY6pjbE3A/rSfrveo6/0B2tujnbkMU4KaYeGReHx3Ohl95A6wN5dVWU9b+6stXe8PUF+Pyoyn0QEoKCgIffv2haurK1JSUrB06VJMmDAB27dvh1QqrbJ+Xl4eysvLYWdnV2m5nZ0dEhMTlf58OzvLOtcu5tiagP1pP13vUdf7A7Szxwm9ffDrjSxcTb2HNWdT8PWr7Z+6rjb2pwxd7w8Qt0eNDkADBw5U/Px4EnRISIhir5C65eTkQ9XX5ZJIHm1wdYytCdif9tP1HnW9P0D7e/wg2BOvb7mMny/fRn8fO3Rws6n0urb39yy63h+gvh4fj1sbGh2AnuTm5gZbW1vcvHmz2gBka2sLqVRaZcJzTk4O7O3tlf48QYDafvnUObYmYH/aT9d71PX+AO3tsZWTJYa2bYLdV9Ox6Eg8tozxh6G06pRVbe2vtnS9P0DcHrXqOkAZGRm4e/cuHBwcqn1dJpOhdevWCA8PVyyrqKhAeHg4/Pz8GqpMIiKqp8mBzWBjaoTEnCJsu3Rb7HJIB4kagAoLCxEdHY3o6GgAQGpqKqKjo5GWlobCwkIsWrQIERERSE1NRXh4OCZPngx3d3cEBQUpxhg3bhzCwsIUz9944w3s2LEDP//8MxISEjB79mwUFxdj2LBhDd4fERHVjbWpEd7r4QEAWBt+E5n5D0SuiHSNqIfAoqKiMHbsWMXz0NBQAMDQoUMxe/ZsxMbGYs+ePcjPz4ejoyO6d++OqVOnQib756yAlJQU5OXlKZ4PGDAAubm5WL58ObKystCyZUusW7euTofAiIhIPC+0boxfIjNwNe0+vj6egIWDWoldEukQiSDo+hHGusvOVs8kaHt7S7WMrQnYn/bT9R51vT9At3qMvVOAMWGXUCEAy4e3QbdmjXSqv+roen+A+np8PG5taNUcICIi0i8+jhZ41c8FAPDV0Xg8eFghckWkKxiAiIhIo00McIe9uQwpd0uw+UKK2OWQjmAAIiIijWZhbIhpwZ4AgI3nU3D7brHIFZEuYAAiIiKN91wLB3RsaoMHDyvw1bEEcPoq1RcDEBERaTyJRIIZvb1haCDBqcRcHL6eKXZJpOUYgIiISCs0szPDqI6uAIB3t17GnN9iEJ9VKHJVpK0YgIiISGuM79oUnd1tUFpegX1RmRix6SLe3XkVp5NyUcHDYqQErboXGBER6TdTIym+fbktbhU9xHdH43A0Ngvnbt7FuZt34dHIDCM6uOD5lo4wMZKKXSppOO4BIiIirePf1Bahg1ri5/GdMbKDC8xlUiTlFuF/h+MwaO15rDmdjJzCUrHLJA3GAERERFrL2doE03t6Yf/ELpje0xNNrIxxt7gM687ewqC15zCX84ToKXgIjIiItJ6FsSFGdnDFK34uOB6Xja0XbyMy/T72XcvEvmuZ6OJug5EdXNGtmS0kEonY5ZIGYAAiIiKdYWggQYjcASFyB1xNu49tF1NxLC77n3lCdmYY4c95QsQAREREOqqtsxXaOrdC2r0SbL98G79EZiAp59E8oW9PJeOldk3wUntn2JnLxC6VRMA5QEREpNOeNU9o3qEYxGdznpC+4R4gIiLSC1XnCaUiMj0fe6MysTeK84T0DQMQERHpFc4TIoABiIiI9BjnCekvzgEiIiK9x3lC+od7gIiIiP725DyhLRdTEfWveUJd3W0xooML5wnpAAYgIiKiJzw5T2jrxVT8EZeNszfzcPZmHjzszDDS3wX9OU9IazEAERER1eDxPKHb94qx43KaYp7Qgr/nCQ3nPCGtxDlAREREteBibaqYJzQt2BNOlsbI4zwhrcU9QEREREqwMDbEqI6ueNX/6fOERnZ0QVd3zhPSZAxAREREdVDbeULPt2oMY0MecNE0DEBERET19O95QtsvVZ0n9FL7JhjejvOENAkjKRERkYq4WJvi/V5eOPB25XlCa8Nv4UXOE9Io3ANERESkYv+eJ/TH3/cd4zwhzcIAREREpCaGBhL0lTugbw3zhEZ1cEH/lpwn1NAYgIiIiBrA0+YJzf89DqtOPponxOsJNRzGTSIiogb073lCU5+YJzTo+3OYdygWsZn5Ypep87gHiIiISAQWxoYY3dEVr/09T2jLX6m4lpGPXyIz8EtkBro2s8XIDpwnpC4MQERERCJ6PE8oxMf+73lCt3E8Phtnk/NwNjkPnnZmGMl5QirHAERERKQBJBIJ2rlYo72rNYokUnx3NBa/RGYg8V/zhF5u74zh7ZugkRnnCdUXoyQREZGGaWpnhg96/zNPqPHf84S+D7+JQd+fw/xDsUjg9YTqhXuAiIiINNS/5wkdi83C1ou3H80TisrAL1GP5gmN6uCCLpwnpDQGICIiIg1naCDBcy0c/3U9Ic4Tqi8GICIiIi3xeJ5QOxdrpN4txvbLadjLeUJ1ImpMvHDhAiZNmoTAwEDI5XIcOXLkqet+/vnnkMvl2LhxY41jlpeXY9myZejduzfatm2LkJAQrFq1CoIgqLh6IiIi8bjamOKDXl7YP7EL/tPDg/OElCTqHqCioiLI5XIMHz4c77777lPXO3z4MK5cuQJHR8dnjrl27Vps27YNixYtgre3N6KiojBr1ixYWlpi7NixqiyfiIhIdJYmhhjTyQ0j/F1wLC4bWy7exnXOE3omUQNQcHAwgoODa1wnMzMT8+bNw/r16/H2228/c8zLly+jT58+6NmzJwDA1dUVBw4cwNWrV1VRMhERkUYylBpUmie05eJtHI+rPE9oVAdX9GvpyHlC0PDT4CsqKvDRRx9h/PjxaN68ea3e4+fnh7NnzyIpKQkAcOPGDVy8eBE9evRQZ6lEREQa4fE8oS9fbIXd4zvhVT9nmBlJkZhThHm/x+LFteew9sxN5BaVil2qqDR6EvTatWthaGio1KGriRMnoqCgAM8//zykUinKy8sxffp0vPjii0p/vjr2FD4eU1f3QrI/7afrPep6f4Du98j+as/N1hQf9fHGpO7NsCcyHT9dSkNm/gN8H34TG8/fwoBWjTGigwu87M3r/2FKUNc2VGY8jQ1AUVFR2LRpE3bv3q3UMcuDBw9i3759WLJkCby9vREdHY3Q0FA4Ojpi6NChStVgZ2epbNkaMbYmYH/aT9d71PX+AN3vkf3Vnj2A6a62ePe5FjgYlYH1JxNxJfUe9kRmYE9kBnr4OOCtQA8ENbdv0HlCYm5DiaAhp0fJ5XKsWrUKISEhAICNGzdi4cKFMDD45yhdeXk5DAwM0KRJExw7dqzacYKDgzFx4kSMGjVKsezbb7/F3r178dtvvylVU05OPlT9/45E8miDq2NsTcD+tJ+u96jr/QG63yP7qz9BEHAl7T62/PVontDjj/G0M8Oojq7or+Z5Qurq8fG4taGxe4AGDx6MgICASsvGjx+PwYMHY9iwYU99X0lJSZX0KpVK63QavCBAjb986htbE7A/7afrPep6f4Du98j+6kOCds7WaPfio+sJ/XTpNvZGPbqe0LxDsVh1MgkvtXfGS+2awFaN1xMScxuKGoAKCwtx69YtxfPU1FRER0fD2toazs7OsLW1rbS+kZER7O3t4enpqVg2btw49O3bF6NHjwYA9OrVC6tXr4azs7PiENiGDRswfPjwhmmKiIhIi7jamOLD3t54O+DxPKHbuFNQiu/P3MTGc7fwfKvGGNnBBZ52DTtPSN1EDUBRUVGVJjiHhoYCAIYOHYqFCxfWaoyUlBTk5eUpnn/66af45ptvMGfOHOTk5MDR0RGvvvoqpkyZotriiYiIdMiT1xMK+ysV0ZkF+CUyA79EZqBbM1uM6uCKzu42OnE9IY2ZA6SJsrPVMwfI3t5SLWNrAvan/XS9R13vD9D9HtlfwxAEAVdu38fWS5XnCXnZm2Gkf/2uJ6SuHh+PWxsaOweIiIiIxCORSNDe1RrtXSvPE0rIfnQ9oVWnGmaekLpo9IUQiYiISHyP5wkdmNgV/+nhAUcLGXKLyvD9mZt44ftzmP97LBJztOu+Y9wDRERERLXy73lCR2OzseWi9s4TYgAiIiIipRhKDdCvpSOea+GAK7fvY8vFVPwZn4Pw5DyEJ+c9mifUwRX9WmjufccYgIiIiKhOapwn1IDXE6oLzYxlREREpFUezxPaP7FLtfOEFmjYPCHuASIiIiKVsTIxqnae0OP7jnVrZotRHV0x0M5C1DoZgIiIiEjl/j1PKOL2fWx9Yp7QnZJyvNjCXrz6RPtkIiIi0nkSiQR+rtbw+9c8oT/jc2BvaSxqXQxARERE1CAezxP6qI+34krQYuEkaCIiItI7DEBERESkdxiAiIiISO8wABEREZHeYQAiIiIivcMARERERHqHAYiIiIj0DgMQERER6R0GICIiItI7DEBERESkdxiAiIiISO8wABEREZHeYQAiIiIivcMARERERHrHUOwCNJlEor4x1TG2JmB/2k/Xe9T1/gDd75H9aT919ajMeBJBEATVfjwRERGRZuMhMCIiItI7DEBERESkdxiAiIiISO8wABEREZHeYQAiIiIivcMARERERHqHAYiIiIj0DgMQERER6R0GICIiItI7DEBERESkdxiA1GDLli3o3bs3fH198fLLL+Pq1as1rn/w4EH0798fvr6+GDRoEP78888GqrRulOlv9+7dkMvllR6+vr4NWK1yLly4gEmTJiEwMBByuRxHjhx55nvOnTuHoUOHok2bNujbty92797dAJXWjbL9nTt3rsr2k8vlyMrKaqCKlbNmzRoMHz4cfn5+6NatGyZPnozExMRnvk+bvoN16VGbvodbt27FoEGD4O/vD39/f7z66qvP3B7atP2U7U+btl11vv/+e8jlcixYsKDG9UTZhgKp1IEDB4TWrVsLO3fuFOLi4oRPP/1U6Nixo5CdnV3t+hcvXhRatmwprF27VoiPjxe+/vproXXr1kJMTEwDV147yva3a9cuwd/fX7hz547ikZWV1cBV197x48eFpUuXCr///rvg4+MjHD58uMb1b926JbRr104IDQ0V4uPjhc2bNwstW7YUTpw40UAVK0fZ/s6ePSv4+PgIiYmJlbZheXl5A1WsnDfffFPYtWuXEBsbK0RHRwsTJkwQevbsKRQWFj71Pdr2HaxLj9r0PTx69Khw/PhxISkpSUhMTBSWLl0qtG7dWoiNja12fW3bfsr2p03b7klXrlwRevXqJQwaNEiYP3/+U9cTaxsyAKnYSy+9JMyZM0fxvLy8XAgMDBTWrFlT7fpTp04VJk6cWGnZyy+/LHz22WdqrbOulO1v165dQocOHRqqPJWqTUD48ssvhYEDB1ZaNm3aNOHNN99UZ2kqoUwAunfvXgNVpVo5OTmCj4+PcP78+aeuo23fwSfVpkdt/h4KgiB06tRJ2LFjR7Wvafv2E4Sa+9PWbVdQUCA899xzwunTp4XRo0fXGIDE2oY8BKZCpaWluHbtGgICAhTLDAwMEBAQgMuXL1f7noiICHTr1q3SssDAQERERKiz1DqpS38AUFRUhF69eiE4OBjvvPMO4uLiGqLcBqFN268+hgwZgsDAQLzxxhu4ePGi2OXUWn5+PgDA2tr6qeto+zasTY+Adn4Py8vLceDAARQVFcHPz6/adbR5+9WmP0A7t93cuXMRHBxc6b8XTyPWNjRU6+h6Ji8vD+Xl5bCzs6u03M7O7qnH6LOzs2Fvb19l/ezsbLXVWVd16c/DwwP/+9//IJfLkZ+fjx9++AGvvfYaDhw4ACcnp4YoW62q23729vYoKChASUkJTExMRKpMNRwcHDBnzhy0adMGpaWl+L//+z+MHTsWO3bsQOvWrcUur0YVFRX43//+B39/f/j4+Dx1PW36Dj6ptj1q2/cwJiYGr732Gh48eAAzMzOsWrUK3t7e1a6rjdtPmf60bdsBwIEDB3D9+nXs3LmzVuuLtQ0ZgEit/Pz8Kv3Lxs/PDwMGDMBPP/2EadOmiVcY1Yqnpyc8PT0Vz/39/ZGSkoKNGzfiq6++ErGyZ5szZw7i4uKwdetWsUtRm9r2qG3fQw8PD+zZswf5+fk4dOgQZsyYgbCwsKeGBG2jTH/atu3S09OxYMEC/PDDDzA2Nha7nBoxAKmQra0tpFIpcnJyKi3Pycmpkm4fs7e3r5Jya1pfTHXp70lGRkZo2bIlbt26pY4SG1x12y87OxsWFhZav/fnaXx9fXHp0iWxy6jR3Llzcfz4cYSFhT3zX8na9B38N2V6fJKmfw9lMhnc3d0BAG3atEFkZCQ2bdqEuXPnVllXG7efMv09SdO33bVr15CTk4Nhw4YplpWXl+PChQvYsmULIiMjIZVKK71HrG3IOUAqJJPJ0Lp1a4SHhyuWVVRUIDw8/KnHd9u3b4+zZ89WWnbmzBm0b99enaXWSV36e1J5eTliY2Ph4OCgrjIblDZtP1W5ceOGxm4/QRAwd+5cHD58GD/++CPc3Nye+R5t24Z16fFJ2vY9rKioQGlpabWvadv2q05N/T1J07dd165dsW/fPuzZs0fxaNOmDQYNGoQ9e/ZUCT+AiNtQrVOs9dCBAweENm3aCLt37xbi4+OFzz77TOjYsaPitMWPPvpIWLx4sWL9ixcvCq1atRLWr18vxMfHC8uXL9foUziV7W/FihXCyZMnhVu3bglRUVHC9OnTBV9fXyEuLk6sFmpUUFAgXL9+Xbh+/brg4+MjbNiwQbh+/bpw+/ZtQRAEYfHixcJHH32kWP/xafCLFi0S4uPjhbCwMI0+DV7Z/jZs2CAcPnxYSE5OFmJiYoT58+cLLVq0EM6cOSNWCzX64osvhA4dOgjnzp2rdNpwcXGxYh1t/w7WpUdt+h4uXrxYOH/+vJCSkiLcuHFDWLx4sSCXy4VTp04JgqD920/Z/rRp2z3Nk2eBaco25CEwFRswYAByc3OxfPlyZGVloWXLlli3bp1iV156ejoMDP7Z8ebv74/Fixdj2bJlWLp0KZo1a4ZVq1bVOKFRTMr2d//+fXz22WfIysqCtbU1WrdujZ9++kljj+VHRUVh7NixiuehoaEAgKFDh2LhwoXIyspCenq64nU3NzesWbMGoaGh2LRpE5ycnDB//nwEBQU1eO21oWx/ZWVlWLRoETIzM2FqagofHx9s2LABXbt2bfDaa2Pbtm0AgDFjxlRaHhoaqtglr+3fwbr0qE3fw5ycHMyYMQN37tyBpaUl5HI51q9fj+7duwPQ/u2nbH/atO1qS1O2oUQQBEGtn0BERESkYTgHiIiIiPQOAxARERHpHQYgIiIi0jsMQERERKR3GICIiIhI7zAAERERkd5hACIiIiK9wwBERFQLcrkcR44cEbsMIlIRXgmaiDTezJkz8fPPP1dZHhgYiPXr14tQERFpOwYgItIKQUFBilt3PCaTyUSqhoi0HQ+BEZFWkMlkcHBwqPSwtrYG8Ojw1NatW/HWW2+hbdu26NOnD3777bdK74+JicHYsWPRtm1bdOnSBZ999hkKCwsrrbNz504MHDgQbdq0QWBgIObOnVvp9by8PEyZMgXt2rXDc889h6NHj6q3aSJSGwYgItIJ33zzDfr164dffvkFgwYNwvvvv4+EhAQAQFFREcaPHw9ra2vs3LkTy5Ytw5kzZzBv3jzF+7du3Yq5c+filVdewb59+/Dtt9+iadOmlT5j5cqVeP7557F371706NEDH374Ie7evduQbRKRijAAEZFWOH78OPz8/Co9Vq9erXi9f//+ePnll+Hh4YFp06ahTZs22Lx5MwBg//79KC0txaJFi+Dj44Nu3brh888/xy+//ILs7GwAwHfffYc33ngD48aNg4eHB9q2bYvXX3+9Ug1Dhw7FCy+8AHd3d7z//vsoKirC1atXG+z/AyJSHc4BIiKt0KVLF8yePbvSsseHwADAz8+v0mvt27dHdHQ0ACAhIQFyuRxmZmaK1/39/VFRUYGkpCRIJBLcuXMH3bp1q7EGuVyu+NnMzAwWFhbIzc2ta0tEJCIGICLSCqampnB3d1fL2MbGxrVaz8jIqNJziUSCiooKdZRERGrGQ2BEpBMiIiIqPb9y5Qq8vLwAAF5eXoiJiUFRUZHi9UuXLsHAwAAeHh6wsLCAi4sLwsPDG7JkIhIRAxARaYXS0lJkZWVVevz78NNvv/2GnTt3IikpCcuXL8fVq1cxevRoAMCgQYMgk8kwc+ZMxMbG4uzZs5g3bx4GDx4Me3t7AMB7772HDRs2YNOmTUhOTsa1a9cUc4iISPfwEBgRaYWTJ08iMDCw0jIPDw/F6e7vvfcefv31V8yZMwcODg5YsmQJvL29ATw6fLZ+/XosWLAAL730EkxNTfHcc89h5syZirGGDh2KBw8eYOPGjfjyyy9hY2OD/v37N1yDRNSgJIIgCGIXQURUH3K5HKtWrUJISIjYpRCRluAhMCIiItI7DEBERESkd3gIjIiIiPQO9wARERGR3mEAIiIiIr3DAERERER6hwGIiIiI9A4DEBEREekdBiAiIiLSOwxAREREpHcYgIiIiEjvMAARERGR3vl/An+UOnOkfPYAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# here we import the model and use it directly.\n",
    "from pykeen.models import ComplEx\n",
    "\n",
    "pipeline_result_imported = pipeline(\n",
    "    random_seed=0,\n",
    "    model=ComplEx,\n",
    "    training=got_training,\n",
    "    testing=got_testing,\n",
    ")\n",
    "pipeline_result_imported.plot_losses()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can retrieve different metrics from the results. Here we retrieve the mean reciprocal rank (MRR). The result is the same for both the simple and imported model, because we used the same random seed (0)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0038159494288265705\n",
      "0.0038159494288265705\n"
     ]
    }
   ],
   "source": [
    "print(pipeline_result_imported.get_metric('mrr'))\n",
    "print(pipeline_result_simple.get_metric('mrr'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pykeen.pipeline.api:Using device: None\n",
      "Training epochs on cuda:0:   0%|                                                             | 0/200 [00:00<?, ?epoch/s]\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   0%|▏                            | 1/200 [00:00<00:26,  7.40epoch/s, loss=16, prev_loss=nan]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   1%|▎                             | 2/200 [00:00<00:26,  7.53epoch/s, loss=16, prev_loss=16]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   2%|▍                           | 3/200 [00:00<00:26,  7.54epoch/s, loss=15.6, prev_loss=16]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   2%|▌                         | 4/200 [00:00<00:27,  7.12epoch/s, loss=14.9, prev_loss=15.6]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   2%|▋                         | 5/200 [00:00<00:28,  6.95epoch/s, loss=14.7, prev_loss=14.9]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   3%|▊                         | 6/200 [00:00<00:27,  7.02epoch/s, loss=13.4, prev_loss=14.7]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   4%|▉                         | 7/200 [00:00<00:27,  7.05epoch/s, loss=13.5, prev_loss=13.4]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   4%|█                         | 8/200 [00:01<00:27,  7.00epoch/s, loss=13.4, prev_loss=13.5]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   4%|█▏                        | 9/200 [00:01<00:26,  7.16epoch/s, loss=12.5, prev_loss=13.4]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   5%|█▎                       | 10/200 [00:01<00:26,  7.27epoch/s, loss=12.4, prev_loss=12.5]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   6%|█▍                       | 11/200 [00:01<00:25,  7.40epoch/s, loss=11.6, prev_loss=12.4]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   6%|█▌                       | 12/200 [00:01<00:25,  7.34epoch/s, loss=11.5, prev_loss=11.6]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   6%|█▋                       | 13/200 [00:01<00:25,  7.30epoch/s, loss=10.9, prev_loss=11.5]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   7%|█▊                       | 14/200 [00:01<00:25,  7.26epoch/s, loss=10.6, prev_loss=10.9]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   8%|█▉                       | 15/200 [00:02<00:25,  7.26epoch/s, loss=10.4, prev_loss=10.6]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   8%|██                       | 16/200 [00:02<00:25,  7.29epoch/s, loss=10.1, prev_loss=10.4]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   8%|██▏                       | 17/200 [00:02<00:25,  7.14epoch/s, loss=9.7, prev_loss=10.1]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   9%|██▎                       | 18/200 [00:02<00:25,  7.15epoch/s, loss=8.99, prev_loss=9.7]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  10%|██▋                         | 19/200 [00:02<00:26,  6.82epoch/s, loss=9, prev_loss=8.99]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  10%|██▊                         | 20/200 [00:02<00:26,  6.70epoch/s, loss=9.03, prev_loss=9]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  10%|██▋                      | 21/200 [00:02<00:26,  6.85epoch/s, loss=8.48, prev_loss=9.03]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  11%|██▊                      | 22/200 [00:03<00:25,  6.94epoch/s, loss=8.31, prev_loss=8.48]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  12%|██▉                      | 23/200 [00:03<00:25,  6.87epoch/s, loss=8.01, prev_loss=8.31]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  12%|███                      | 24/200 [00:03<00:26,  6.56epoch/s, loss=7.72, prev_loss=8.01]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  12%|███▏                     | 25/200 [00:03<00:26,  6.49epoch/s, loss=7.16, prev_loss=7.72]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  13%|███▎                     | 26/200 [00:03<00:26,  6.65epoch/s, loss=7.44, prev_loss=7.16]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  14%|███▍                     | 27/200 [00:03<00:25,  6.72epoch/s, loss=6.75, prev_loss=7.44]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  14%|███▋                      | 28/200 [00:04<00:25,  6.73epoch/s, loss=6.5, prev_loss=6.75]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  14%|███▉                       | 29/200 [00:04<00:25,  6.74epoch/s, loss=6.3, prev_loss=6.5]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  15%|███▉                      | 30/200 [00:04<00:25,  6.59epoch/s, loss=5.93, prev_loss=6.3]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  16%|███▉                     | 31/200 [00:04<00:26,  6.31epoch/s, loss=5.85, prev_loss=5.93]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  16%|████                     | 32/200 [00:04<00:27,  6.16epoch/s, loss=5.87, prev_loss=5.85]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  16%|████▎                     | 33/200 [00:04<00:27,  6.07epoch/s, loss=5.8, prev_loss=5.87]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  17%|████▍                     | 34/200 [00:04<00:26,  6.21epoch/s, loss=5.98, prev_loss=5.8]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  18%|████▍                    | 35/200 [00:05<00:25,  6.39epoch/s, loss=5.24, prev_loss=5.98]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  18%|████▌                    | 36/200 [00:05<00:25,  6.51epoch/s, loss=5.19, prev_loss=5.24]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  18%|████▊                     | 37/200 [00:05<00:24,  6.61epoch/s, loss=4.8, prev_loss=5.19]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  19%|████▉                     | 38/200 [00:05<00:24,  6.52epoch/s, loss=4.69, prev_loss=4.8]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  20%|████▉                    | 39/200 [00:05<00:24,  6.48epoch/s, loss=4.52, prev_loss=4.69]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  20%|█████                    | 40/200 [00:05<00:25,  6.29epoch/s, loss=4.59, prev_loss=4.52]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  20%|█████▏                   | 41/200 [00:06<00:25,  6.23epoch/s, loss=4.44, prev_loss=4.59]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  21%|█████▎                   | 42/200 [00:06<00:24,  6.34epoch/s, loss=4.21, prev_loss=4.44]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  22%|█████▍                   | 43/200 [00:06<00:24,  6.40epoch/s, loss=4.37, prev_loss=4.21]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  22%|█████▌                   | 44/200 [00:06<00:24,  6.31epoch/s, loss=4.21, prev_loss=4.37]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  22%|█████▋                   | 45/200 [00:06<00:24,  6.39epoch/s, loss=3.96, prev_loss=4.21]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  23%|█████▊                   | 46/200 [00:06<00:24,  6.41epoch/s, loss=4.24, prev_loss=3.96]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  24%|█████▉                   | 47/200 [00:06<00:23,  6.46epoch/s, loss=3.85, prev_loss=4.24]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  24%|██████                   | 48/200 [00:07<00:23,  6.49epoch/s, loss=3.82, prev_loss=3.85]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  24%|██████▏                  | 49/200 [00:07<00:23,  6.48epoch/s, loss=3.38, prev_loss=3.82]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  25%|██████▌                   | 50/200 [00:07<00:23,  6.42epoch/s, loss=3.4, prev_loss=3.38]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  26%|██████▋                   | 51/200 [00:07<00:23,  6.41epoch/s, loss=3.45, prev_loss=3.4]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  26%|██████▊                   | 52/200 [00:07<00:23,  6.38epoch/s, loss=3.2, prev_loss=3.45]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  26%|██████▉                   | 53/200 [00:07<00:23,  6.36epoch/s, loss=3.46, prev_loss=3.2]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  27%|██████▊                  | 54/200 [00:08<00:23,  6.34epoch/s, loss=2.99, prev_loss=3.46]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  28%|██████▉                  | 55/200 [00:08<00:22,  6.33epoch/s, loss=2.76, prev_loss=2.99]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  28%|███████                  | 56/200 [00:08<00:22,  6.36epoch/s, loss=3.11, prev_loss=2.76]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  28%|███████                  | 57/200 [00:08<00:22,  6.34epoch/s, loss=2.65, prev_loss=3.11]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  29%|███████▏                 | 58/200 [00:08<00:22,  6.30epoch/s, loss=2.74, prev_loss=2.65]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  30%|███████▍                 | 59/200 [00:08<00:22,  6.31epoch/s, loss=2.57, prev_loss=2.74]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  30%|███████▌                 | 60/200 [00:09<00:22,  6.25epoch/s, loss=2.68, prev_loss=2.57]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  30%|███████▋                 | 61/200 [00:09<00:22,  6.12epoch/s, loss=2.81, prev_loss=2.68]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  31%|███████▊                 | 62/200 [00:09<00:23,  5.95epoch/s, loss=2.43, prev_loss=2.81]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  32%|███████▉                 | 63/200 [00:09<00:23,  5.74epoch/s, loss=2.41, prev_loss=2.43]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  32%|████████                 | 64/200 [00:09<00:23,  5.82epoch/s, loss=2.54, prev_loss=2.41]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  32%|████████▏                | 65/200 [00:09<00:22,  5.95epoch/s, loss=2.29, prev_loss=2.54]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  33%|████████▎                | 66/200 [00:10<00:22,  6.08epoch/s, loss=2.08, prev_loss=2.29]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  34%|████████▍                | 67/200 [00:10<00:21,  6.06epoch/s, loss=2.09, prev_loss=2.08]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  34%|████████▌                | 68/200 [00:10<00:22,  5.99epoch/s, loss=2.24, prev_loss=2.09]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  34%|████████▋                | 69/200 [00:10<00:21,  6.05epoch/s, loss=2.16, prev_loss=2.24]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  35%|█████████                 | 70/200 [00:10<00:21,  6.10epoch/s, loss=2.4, prev_loss=2.16]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  36%|█████████▏                | 71/200 [00:10<00:20,  6.24epoch/s, loss=2.03, prev_loss=2.4]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  36%|█████████                | 72/200 [00:11<00:20,  6.30epoch/s, loss=2.21, prev_loss=2.03]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  36%|█████████▏               | 73/200 [00:11<00:19,  6.35epoch/s, loss=2.04, prev_loss=2.21]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  37%|█████████▎               | 74/200 [00:11<00:19,  6.40epoch/s, loss=1.81, prev_loss=2.04]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  38%|█████████▍               | 75/200 [00:11<00:19,  6.30epoch/s, loss=1.97, prev_loss=1.81]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  38%|█████████▌               | 76/200 [00:11<00:19,  6.36epoch/s, loss=1.85, prev_loss=1.97]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  38%|██████████                | 77/200 [00:11<00:19,  6.27epoch/s, loss=1.9, prev_loss=1.85]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  39%|██████████▏               | 78/200 [00:11<00:19,  6.24epoch/s, loss=1.78, prev_loss=1.9]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  40%|█████████▉               | 79/200 [00:12<00:19,  6.33epoch/s, loss=1.62, prev_loss=1.78]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  40%|██████████               | 80/200 [00:12<00:18,  6.37epoch/s, loss=1.86, prev_loss=1.62]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  40%|██████████▏              | 81/200 [00:12<00:18,  6.41epoch/s, loss=1.59, prev_loss=1.86]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  41%|██████████▎              | 82/200 [00:12<00:18,  6.41epoch/s, loss=1.55, prev_loss=1.59]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  42%|██████████▍              | 83/200 [00:12<00:18,  6.42epoch/s, loss=1.57, prev_loss=1.55]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  42%|██████████▌              | 84/200 [00:12<00:17,  6.48epoch/s, loss=1.73, prev_loss=1.57]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  42%|██████████▋              | 85/200 [00:13<00:17,  6.49epoch/s, loss=1.55, prev_loss=1.73]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  43%|██████████▊              | 86/200 [00:13<00:17,  6.46epoch/s, loss=1.55, prev_loss=1.55]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  44%|██████████▉              | 87/200 [00:13<00:17,  6.48epoch/s, loss=1.54, prev_loss=1.55]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  44%|███████████              | 88/200 [00:13<00:17,  6.45epoch/s, loss=1.39, prev_loss=1.54]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  44%|███████████▏             | 89/200 [00:13<00:17,  6.35epoch/s, loss=1.52, prev_loss=1.39]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  45%|███████████▎             | 90/200 [00:13<00:18,  6.09epoch/s, loss=1.37, prev_loss=1.52]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  46%|███████████▍             | 91/200 [00:14<00:18,  5.92epoch/s, loss=1.37, prev_loss=1.37]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  46%|███████████▉              | 92/200 [00:14<00:18,  5.96epoch/s, loss=1.3, prev_loss=1.37]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  46%|████████████              | 93/200 [00:14<00:17,  6.10epoch/s, loss=1.37, prev_loss=1.3]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  47%|███████████▊             | 94/200 [00:14<00:17,  6.13epoch/s, loss=1.13, prev_loss=1.37]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  48%|███████████▉             | 95/200 [00:14<00:16,  6.19epoch/s, loss=1.28, prev_loss=1.13]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  48%|████████████             | 96/200 [00:14<00:16,  6.24epoch/s, loss=1.25, prev_loss=1.28]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  48%|████████████▏            | 97/200 [00:15<00:16,  6.28epoch/s, loss=1.28, prev_loss=1.25]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  49%|████████████▎            | 98/200 [00:15<00:16,  6.34epoch/s, loss=1.16, prev_loss=1.28]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  50%|████████████▍            | 99/200 [00:15<00:15,  6.35epoch/s, loss=1.22, prev_loss=1.16]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  50%|████████████            | 100/200 [00:15<00:16,  6.21epoch/s, loss=1.04, prev_loss=1.22]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  50%|████████████            | 101/200 [00:15<00:16,  6.05epoch/s, loss=1.11, prev_loss=1.04]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 116.87batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  51%|████████████▏           | 102/200 [00:15<00:16,  5.77epoch/s, loss=1.07, prev_loss=1.11]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  52%|████████████▎           | 103/200 [00:16<00:17,  5.69epoch/s, loss=1.09, prev_loss=1.07]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  52%|████████████▍           | 104/200 [00:16<00:16,  5.79epoch/s, loss=1.07, prev_loss=1.09]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  52%|████████████▌           | 105/200 [00:16<00:16,  5.90epoch/s, loss=1.06, prev_loss=1.07]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  53%|████████████▋           | 106/200 [00:16<00:15,  6.01epoch/s, loss=1.04, prev_loss=1.06]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  54%|████████████▎          | 107/200 [00:16<00:15,  6.07epoch/s, loss=0.944, prev_loss=1.04]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  54%|████████████▍          | 108/200 [00:16<00:15,  6.11epoch/s, loss=1.15, prev_loss=0.944]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  55%|████████████▌          | 109/200 [00:17<00:14,  6.15epoch/s, loss=0.968, prev_loss=1.15]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  55%|████████████          | 110/200 [00:17<00:14,  6.09epoch/s, loss=0.956, prev_loss=0.968]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  56%|████████████▊          | 111/200 [00:17<00:15,  5.87epoch/s, loss=1.04, prev_loss=0.956]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 119.57batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  56%|████████████▉          | 112/200 [00:17<00:15,  5.62epoch/s, loss=0.911, prev_loss=1.04]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  56%|████████████▍         | 113/200 [00:17<00:15,  5.72epoch/s, loss=0.953, prev_loss=0.911]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  57%|████████████▌         | 114/200 [00:17<00:14,  5.78epoch/s, loss=0.908, prev_loss=0.953]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  57%|████████████▋         | 115/200 [00:18<00:14,  5.95epoch/s, loss=0.882, prev_loss=0.908]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  58%|████████████▊         | 116/200 [00:18<00:13,  6.11epoch/s, loss=0.865, prev_loss=0.882]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  58%|████████████▊         | 117/200 [00:18<00:13,  6.19epoch/s, loss=0.768, prev_loss=0.865]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  59%|████████████▉         | 118/200 [00:18<00:13,  6.23epoch/s, loss=0.768, prev_loss=0.768]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  60%|█████████████         | 119/200 [00:18<00:12,  6.29epoch/s, loss=0.791, prev_loss=0.768]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  60%|█████████████▏        | 120/200 [00:18<00:12,  6.33epoch/s, loss=0.733, prev_loss=0.791]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  60%|█████████████▎        | 121/200 [00:18<00:12,  6.38epoch/s, loss=0.806, prev_loss=0.733]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  61%|█████████████▍        | 122/200 [00:19<00:12,  6.40epoch/s, loss=0.798, prev_loss=0.806]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  62%|█████████████▌        | 123/200 [00:19<00:12,  6.26epoch/s, loss=0.698, prev_loss=0.798]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  62%|█████████████▋        | 124/200 [00:19<00:12,  6.26epoch/s, loss=0.877, prev_loss=0.698]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  62%|█████████████▊        | 125/200 [00:19<00:11,  6.26epoch/s, loss=0.714, prev_loss=0.877]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  63%|█████████████▊        | 126/200 [00:19<00:11,  6.25epoch/s, loss=0.804, prev_loss=0.714]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  64%|██████████████▌        | 127/200 [00:19<00:11,  6.30epoch/s, loss=0.69, prev_loss=0.804]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  64%|███████████████▎        | 128/200 [00:20<00:11,  6.35epoch/s, loss=0.69, prev_loss=0.69]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  64%|██████████████▊        | 129/200 [00:20<00:11,  6.36epoch/s, loss=0.783, prev_loss=0.69]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  65%|██████████████▎       | 130/200 [00:20<00:10,  6.42epoch/s, loss=0.582, prev_loss=0.783]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  66%|██████████████▍       | 131/200 [00:20<00:10,  6.37epoch/s, loss=0.611, prev_loss=0.582]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  66%|██████████████▌       | 132/200 [00:20<00:10,  6.31epoch/s, loss=0.622, prev_loss=0.611]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  66%|██████████████▋       | 133/200 [00:20<00:10,  6.30epoch/s, loss=0.543, prev_loss=0.622]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  67%|██████████████▋       | 134/200 [00:21<00:10,  6.26epoch/s, loss=0.633, prev_loss=0.543]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  68%|██████████████▊       | 135/200 [00:21<00:10,  6.28epoch/s, loss=0.628, prev_loss=0.633]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  68%|██████████████▉       | 136/200 [00:21<00:10,  6.03epoch/s, loss=0.689, prev_loss=0.628]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  68%|███████████████       | 137/200 [00:21<00:10,  6.11epoch/s, loss=0.567, prev_loss=0.689]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  69%|███████████████▏      | 138/200 [00:21<00:10,  6.20epoch/s, loss=0.679, prev_loss=0.567]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  70%|███████████████▎      | 139/200 [00:21<00:09,  6.11epoch/s, loss=0.623, prev_loss=0.679]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  70%|████████████████       | 140/200 [00:22<00:09,  6.16epoch/s, loss=0.68, prev_loss=0.623]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  70%|████████████████▏      | 141/200 [00:22<00:09,  6.25epoch/s, loss=0.616, prev_loss=0.68]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  71%|███████████████▌      | 142/200 [00:22<00:09,  6.09epoch/s, loss=0.628, prev_loss=0.616]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  72%|███████████████▋      | 143/200 [00:22<00:09,  5.95epoch/s, loss=0.502, prev_loss=0.628]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  72%|███████████████▊      | 144/200 [00:22<00:09,  6.10epoch/s, loss=0.567, prev_loss=0.502]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  72%|████████████████▋      | 145/200 [00:22<00:08,  6.19epoch/s, loss=0.52, prev_loss=0.567]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  73%|████████████████▊      | 146/200 [00:23<00:08,  6.22epoch/s, loss=0.545, prev_loss=0.52]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  74%|████████████████▏     | 147/200 [00:23<00:08,  6.07epoch/s, loss=0.703, prev_loss=0.545]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  74%|████████████████▎     | 148/200 [00:23<00:08,  6.07epoch/s, loss=0.473, prev_loss=0.703]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  74%|████████████████▍     | 149/200 [00:23<00:08,  5.98epoch/s, loss=0.561, prev_loss=0.473]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  75%|████████████████▌     | 150/200 [00:23<00:08,  6.07epoch/s, loss=0.518, prev_loss=0.561]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  76%|████████████████▌     | 151/200 [00:23<00:08,  6.10epoch/s, loss=0.435, prev_loss=0.518]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  76%|████████████████▋     | 152/200 [00:23<00:07,  6.17epoch/s, loss=0.602, prev_loss=0.435]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  76%|████████████████▊     | 153/200 [00:24<00:07,  6.24epoch/s, loss=0.577, prev_loss=0.602]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  77%|████████████████▉     | 154/200 [00:24<00:07,  6.09epoch/s, loss=0.517, prev_loss=0.577]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  78%|█████████████████     | 155/200 [00:24<00:07,  6.02epoch/s, loss=0.457, prev_loss=0.517]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  78%|█████████████████▏    | 156/200 [00:24<00:07,  6.06epoch/s, loss=0.446, prev_loss=0.457]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  78%|█████████████████▎    | 157/200 [00:24<00:07,  6.00epoch/s, loss=0.443, prev_loss=0.446]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  79%|█████████████████▍    | 158/200 [00:24<00:06,  6.13epoch/s, loss=0.467, prev_loss=0.443]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  80%|█████████████████▍    | 159/200 [00:25<00:06,  6.13epoch/s, loss=0.476, prev_loss=0.467]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  80%|█████████████████▌    | 160/200 [00:25<00:06,  6.12epoch/s, loss=0.458, prev_loss=0.476]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  80%|█████████████████▋    | 161/200 [00:25<00:06,  5.97epoch/s, loss=0.454, prev_loss=0.458]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  81%|█████████████████▊    | 162/200 [00:25<00:06,  5.88epoch/s, loss=0.522, prev_loss=0.454]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  82%|█████████████████▉    | 163/200 [00:25<00:06,  5.90epoch/s, loss=0.487, prev_loss=0.522]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  82%|██████████████████    | 164/200 [00:25<00:06,  5.92epoch/s, loss=0.334, prev_loss=0.487]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  82%|██████████████████▏   | 165/200 [00:26<00:05,  5.96epoch/s, loss=0.475, prev_loss=0.334]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  83%|██████████████████▎   | 166/200 [00:26<00:05,  6.08epoch/s, loss=0.413, prev_loss=0.475]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  84%|██████████████████▎   | 167/200 [00:26<00:05,  6.08epoch/s, loss=0.364, prev_loss=0.413]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  84%|██████████████████▍   | 168/200 [00:26<00:05,  5.98epoch/s, loss=0.417, prev_loss=0.364]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  84%|███████████████████▍   | 169/200 [00:26<00:05,  5.78epoch/s, loss=0.44, prev_loss=0.417]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  85%|███████████████████▌   | 170/200 [00:27<00:05,  5.73epoch/s, loss=0.434, prev_loss=0.44]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  86%|███████████████████▋   | 171/200 [00:27<00:05,  5.72epoch/s, loss=0.38, prev_loss=0.434]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  86%|███████████████████▊   | 172/200 [00:27<00:04,  5.75epoch/s, loss=0.414, prev_loss=0.38]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  86%|███████████████████   | 173/200 [00:27<00:04,  5.78epoch/s, loss=0.404, prev_loss=0.414]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  87%|███████████████████▏  | 174/200 [00:27<00:04,  5.80epoch/s, loss=0.414, prev_loss=0.404]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 102.77batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  88%|████████████████████▏  | 175/200 [00:27<00:04,  5.58epoch/s, loss=0.36, prev_loss=0.414]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  88%|████████████████████▏  | 176/200 [00:28<00:04,  5.69epoch/s, loss=0.325, prev_loss=0.36]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 117.93batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  88%|███████████████████▍  | 177/200 [00:28<00:04,  5.59epoch/s, loss=0.376, prev_loss=0.325]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  89%|███████████████████▌  | 178/200 [00:28<00:03,  5.58epoch/s, loss=0.385, prev_loss=0.376]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 116.58batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  90%|███████████████████▋  | 179/200 [00:28<00:03,  5.52epoch/s, loss=0.351, prev_loss=0.385]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 119.83batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  90%|███████████████████▊  | 180/200 [00:28<00:03,  5.49epoch/s, loss=0.395, prev_loss=0.351]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  90%|███████████████████▉  | 181/200 [00:28<00:03,  5.54epoch/s, loss=0.403, prev_loss=0.395]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  91%|████████████████████  | 182/200 [00:29<00:03,  5.52epoch/s, loss=0.387, prev_loss=0.403]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  92%|████████████████████▏ | 183/200 [00:29<00:03,  5.48epoch/s, loss=0.299, prev_loss=0.387]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 117.84batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  92%|████████████████████▏ | 184/200 [00:29<00:02,  5.42epoch/s, loss=0.395, prev_loss=0.299]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  92%|████████████████████▎ | 185/200 [00:29<00:02,  5.62epoch/s, loss=0.354, prev_loss=0.395]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  93%|████████████████████▍ | 186/200 [00:29<00:02,  5.48epoch/s, loss=0.363, prev_loss=0.354]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  94%|████████████████████▌ | 187/200 [00:30<00:02,  5.65epoch/s, loss=0.394, prev_loss=0.363]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  94%|████████████████████▋ | 188/200 [00:30<00:02,  5.57epoch/s, loss=0.302, prev_loss=0.394]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  94%|█████████████████████▋ | 189/200 [00:30<00:01,  5.57epoch/s, loss=0.38, prev_loss=0.302]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 115.46batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  95%|█████████████████████▊ | 190/200 [00:30<00:01,  5.52epoch/s, loss=0.327, prev_loss=0.38]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  96%|█████████████████████▉ | 191/200 [00:30<00:01,  5.59epoch/s, loss=0.35, prev_loss=0.327]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  96%|██████████████████████ | 192/200 [00:30<00:01,  5.67epoch/s, loss=0.281, prev_loss=0.35]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  96%|█████████████████████▏| 193/200 [00:31<00:01,  5.72epoch/s, loss=0.281, prev_loss=0.281]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  97%|█████████████████████▎| 194/200 [00:31<00:01,  5.81epoch/s, loss=0.292, prev_loss=0.281]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  98%|█████████████████████▍| 195/200 [00:31<00:00,  5.73epoch/s, loss=0.382, prev_loss=0.292]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  98%|█████████████████████▌| 196/200 [00:31<00:00,  5.77epoch/s, loss=0.281, prev_loss=0.382]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  98%|█████████████████████▋| 197/200 [00:31<00:00,  5.80epoch/s, loss=0.276, prev_loss=0.281]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  99%|█████████████████████▊| 198/200 [00:32<00:00,  5.67epoch/s, loss=0.378, prev_loss=0.276]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0: 100%|█████████████████████▉| 199/200 [00:32<00:00,  5.72epoch/s, loss=0.283, prev_loss=0.378]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0: 100%|██████████████████████| 200/200 [00:32<00:00,  6.18epoch/s, loss=0.361, prev_loss=0.283]\u001b[A\n",
      "INFO:pykeen.evaluation.evaluator:Starting batch_size search for evaluation now...\n",
      "INFO:pykeen.evaluation.evaluator:Concluded batch_size search with batch_size=159.\n",
      "Evaluating on cuda:0: 100%|███████████████████████████████████████████████████████| 159/159 [00:00<00:00, 2.28ktriple/s]\n",
      "INFO:pykeen.evaluation.evaluator:Evaluation took 0.08s seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.004466169513761997\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training epochs on cpu:  78%|██████████████    | 156/200 [00:45<00:13,  3.20epoch/s, loss=0.549, prev_loss=0.428]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 53.50batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 53.58batch/s]\u001b[A\n",
      "Training epochs on cpu:  78%|██████████████▏   | 157/200 [00:45<00:13,  3.18epoch/s, loss=0.478, prev_loss=0.549]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 52.02batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 51.73batch/s]\u001b[A\n",
      "Training epochs on cpu:  79%|██████████████▏   | 158/200 [00:46<00:13,  3.14epoch/s, loss=0.498, prev_loss=0.478]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 51.92batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 52.25batch/s]\u001b[A\n",
      "Training epochs on cpu:  80%|███████████████    | 159/200 [00:46<00:13,  3.13epoch/s, loss=0.38, prev_loss=0.498]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 54.68batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 53.39batch/s]\u001b[A\n",
      "Training epochs on cpu:  80%|███████████████▏   | 160/200 [00:46<00:12,  3.14epoch/s, loss=0.445, prev_loss=0.38]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 52.33batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 53.36batch/s]\u001b[A\n",
      "Training epochs on cpu:  80%|██████████████▍   | 161/200 [00:47<00:12,  3.14epoch/s, loss=0.481, prev_loss=0.445]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 54.54batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 55.06batch/s]\u001b[A\n",
      "Training epochs on cpu:  81%|██████████████▌   | 162/200 [00:47<00:12,  3.14epoch/s, loss=0.439, prev_loss=0.481]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 52.57batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 54.08batch/s]\u001b[A\n",
      "Training epochs on cpu:  82%|██████████████▋   | 163/200 [00:47<00:11,  3.14epoch/s, loss=0.421, prev_loss=0.439]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 53.99batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 55.48batch/s]\u001b[A\n",
      "Training epochs on cpu:  82%|██████████████▊   | 164/200 [00:48<00:11,  3.20epoch/s, loss=0.437, prev_loss=0.421]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 63.70batch/s]\u001b[A\n",
      "Training epochs on cpu:  82%|██████████████▊   | 165/200 [00:48<00:10,  3.33epoch/s, loss=0.463, prev_loss=0.437]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 60.66batch/s]\u001b[A\n",
      "Training epochs on cpu:  83%|██████████████▉   | 166/200 [00:48<00:10,  3.35epoch/s, loss=0.472, prev_loss=0.463]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 58.86batch/s]\u001b[A\n",
      "Training epochs on cpu:  84%|███████████████   | 167/200 [00:48<00:09,  3.38epoch/s, loss=0.409, prev_loss=0.472]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 64.23batch/s]\u001b[A\n",
      "Training epochs on cpu:  84%|███████████████   | 168/200 [00:49<00:09,  3.50epoch/s, loss=0.399, prev_loss=0.409]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 62.20batch/s]\u001b[A\n",
      "Training epochs on cpu:  84%|████████████████   | 169/200 [00:49<00:08,  3.54epoch/s, loss=0.52, prev_loss=0.399]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 59.82batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 59.41batch/s]\u001b[A\n",
      "Training epochs on cpu:  85%|████████████████▏  | 170/200 [00:49<00:08,  3.51epoch/s, loss=0.374, prev_loss=0.52]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 59.55batch/s]\u001b[A\n",
      "Training epochs on cpu:  86%|███████████████▍  | 171/200 [00:50<00:08,  3.52epoch/s, loss=0.496, prev_loss=0.374]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 56.90batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 57.87batch/s]\u001b[A\n",
      "Training epochs on cpu:  86%|███████████████▍  | 172/200 [00:50<00:08,  3.47epoch/s, loss=0.378, prev_loss=0.496]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 63.90batch/s]\u001b[A\n",
      "Training epochs on cpu:  86%|███████████████▌  | 173/200 [00:50<00:07,  3.48epoch/s, loss=0.402, prev_loss=0.378]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 65.57batch/s]\u001b[A\n",
      "Training epochs on cpu:  87%|████████████████▌  | 174/200 [00:50<00:07,  3.58epoch/s, loss=0.42, prev_loss=0.402]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 61.16batch/s]\u001b[A\n",
      "Training epochs on cpu:  88%|████████████████▋  | 175/200 [00:51<00:07,  3.55epoch/s, loss=0.411, prev_loss=0.42]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 64.07batch/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training epochs on cpu:  88%|███████████████▊  | 176/200 [00:51<00:06,  3.62epoch/s, loss=0.323, prev_loss=0.411]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 62.52batch/s]\u001b[A\n",
      "Training epochs on cpu:  88%|███████████████▉  | 177/200 [00:51<00:06,  3.54epoch/s, loss=0.416, prev_loss=0.323]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 56.89batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 55.68batch/s]\u001b[A\n",
      "Training epochs on cpu:  89%|████████████████  | 178/200 [00:52<00:06,  3.44epoch/s, loss=0.387, prev_loss=0.416]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 51.22batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 52.68batch/s]\u001b[A\n",
      "Training epochs on cpu:  90%|████████████████  | 179/200 [00:52<00:06,  3.32epoch/s, loss=0.418, prev_loss=0.387]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 57.73batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 56.39batch/s]\u001b[A\n",
      "Training epochs on cpu:  90%|████████████████▏ | 180/200 [00:52<00:06,  3.30epoch/s, loss=0.363, prev_loss=0.418]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 52.02batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 52.60batch/s]\u001b[A\n",
      "Training epochs on cpu:  90%|████████████████▎ | 181/200 [00:52<00:05,  3.23epoch/s, loss=0.368, prev_loss=0.363]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 52.97batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 52.46batch/s]\u001b[A\n",
      "Training epochs on cpu:  91%|████████████████▍ | 182/200 [00:53<00:05,  3.19epoch/s, loss=0.286, prev_loss=0.368]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 54.77batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 54.28batch/s]\u001b[A\n",
      "Training epochs on cpu:  92%|████████████████▍ | 183/200 [00:53<00:05,  3.15epoch/s, loss=0.349, prev_loss=0.286]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 57.48batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 54.58batch/s]\u001b[A\n",
      "Training epochs on cpu:  92%|████████████████▌ | 184/200 [00:53<00:05,  3.15epoch/s, loss=0.347, prev_loss=0.349]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 52.32batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 52.31batch/s]\u001b[A\n",
      "Training epochs on cpu:  92%|████████████████▋ | 185/200 [00:54<00:04,  3.13epoch/s, loss=0.476, prev_loss=0.347]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 58.07batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 54.27batch/s]\u001b[A\n",
      "Training epochs on cpu:  93%|█████████████████▋ | 186/200 [00:54<00:04,  3.14epoch/s, loss=0.34, prev_loss=0.476]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 51.99batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 53.92batch/s]\u001b[A\n",
      "Training epochs on cpu:  94%|█████████████████▊ | 187/200 [00:54<00:04,  3.12epoch/s, loss=0.355, prev_loss=0.34]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 53.58batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 53.35batch/s]\u001b[A\n",
      "Training epochs on cpu:  94%|████████████████▉ | 188/200 [00:55<00:03,  3.09epoch/s, loss=0.285, prev_loss=0.355]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 53.57batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 53.91batch/s]\u001b[A\n",
      "Training epochs on cpu:  94%|█████████████████▉ | 189/200 [00:55<00:03,  3.11epoch/s, loss=0.29, prev_loss=0.285]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 52.27batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 53.28batch/s]\u001b[A\n",
      "Training epochs on cpu:  95%|██████████████████ | 190/200 [00:55<00:03,  3.10epoch/s, loss=0.224, prev_loss=0.29]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 52.84batch/s]\u001b[A\n",
      "Training epochs on cpu:  96%|██████████████████▏| 191/200 [00:56<00:02,  3.18epoch/s, loss=0.24, prev_loss=0.224]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 65.48batch/s]\u001b[A\n",
      "Training epochs on cpu:  96%|██████████████████▏| 192/200 [00:56<00:02,  3.28epoch/s, loss=0.353, prev_loss=0.24]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 60.40batch/s]\u001b[A\n",
      "Training epochs on cpu:  96%|█████████████████▎| 193/200 [00:56<00:02,  3.34epoch/s, loss=0.347, prev_loss=0.353]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 67.91batch/s]\u001b[A\n",
      "Training epochs on cpu:  97%|█████████████████▍| 194/200 [00:57<00:01,  3.41epoch/s, loss=0.262, prev_loss=0.347]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 67.54batch/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training epochs on cpu:  98%|█████████████████▌| 195/200 [00:57<00:01,  3.53epoch/s, loss=0.299, prev_loss=0.262]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 65.36batch/s]\u001b[A\n",
      "Training epochs on cpu:  98%|█████████████████▋| 196/200 [00:57<00:01,  3.54epoch/s, loss=0.359, prev_loss=0.299]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 64.20batch/s]\u001b[A\n",
      "Training epochs on cpu:  98%|█████████████████▋| 197/200 [00:57<00:00,  3.57epoch/s, loss=0.275, prev_loss=0.359]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 63.67batch/s]\u001b[A\n",
      "Training epochs on cpu:  99%|█████████████████▊| 198/200 [00:58<00:00,  3.59epoch/s, loss=0.322, prev_loss=0.275]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 61.97batch/s]\u001b[A\n",
      "Training epochs on cpu: 100%|█████████████████▉| 199/200 [00:58<00:00,  3.66epoch/s, loss=0.281, prev_loss=0.322]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 63.70batch/s]\u001b[A\n",
      "Training epochs on cpu: 100%|██████████████████| 200/200 [00:58<00:00,  3.41epoch/s, loss=0.254, prev_loss=0.281]\u001b[A\n",
      "INFO:pykeen.evaluation.evaluator:Currently automatic memory optimization only supports GPUs, but you're using a CPU. Therefore, the batch_size will be set to the default value.\n",
      "INFO:pykeen.evaluation.evaluator:No evaluation batch_size provided. Setting batch_size to '32'.\n",
      "Evaluating on cpu: 100%|███████████████████████████████████████████████████| 159/159 [00:00<00:00, 1.27ktriple/s]\n",
      "INFO:pykeen.evaluation.evaluator:Evaluation took 0.13s seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.00306299460009209\n"
     ]
    }
   ],
   "source": [
    "# but to get a better performing model, you want to set different things\n",
    "pipeline_result = pipeline(\n",
    "    random_seed=0,\n",
    "    model='ComplEx',\n",
    "    training=got_training,\n",
    "    testing=got_testing,\n",
    "    epochs=200,\n",
    "    dimensions=150,\n",
    "    optimizer='adam',\n",
    "    optimizer_kwargs={'lr':1e-3},\n",
    "    loss='pairwisehinge', \n",
    "    regularizer='LP', \n",
    "    regularizer_kwargs={'p':3, 'weight':1e-5}, \n",
    ")\n",
    "print(pipeline_result.get_metric('mrr'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Understanding the parameters:\n",
    "\n",
    "- dimensions : the dimensionality of the embedding space\n",
    "- negative_sampler : the negative samplic strategy, here set to default (not used in arguments).\n",
    "- batch_size : the number of batches in which the training set is split during the training loop. If you are having into low memory issues than settings this to a higher number may help.\n",
    "- epochs : the number of epochs to train the model for.\n",
    "- optimizer : the Adam optimizer, with a learning rate of $1e-3$ set via the <i>optimizer_kwarg</i>.\n",
    "- loss : pairwise loss, with a margin of $0.5$ set via the <i>loss_kwarg</i>.\n",
    "- regularizer :  regularization with $p=2$, i.e. $l_2$ regularization. $\\lambda$ = $1e-5$, set via the <i>regularizer_kwarg</i>."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Filtering Negatives\n",
    "\n",
    "To ensure our model can be trained and evaluated correctly, we need to define a filter to ensure that no negative statements generated by the corruption procedure are actually positives. This is simply done by concatenating train and test sets. When negative triples are generated by the corruption strategy, we can check that they aren't actually true statements.\n",
    "\n",
    "With PyKEEN this is made very easy, and can simply be passed as an argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:pykeen.pipeline.api:No random seed is specified. Setting to 522401844.\n",
      "INFO:pykeen.pipeline.api:Using device: None\n",
      "Training epochs on cuda:0:   0%|                                                             | 0/200 [00:00<?, ?epoch/s]\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 109.45batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   0%|▏                            | 1/200 [00:00<00:39,  5.03epoch/s, loss=17, prev_loss=nan]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 112.40batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   1%|▎                             | 2/200 [00:00<00:38,  5.21epoch/s, loss=16, prev_loss=17]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 116.86batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   2%|▍                           | 3/200 [00:00<00:37,  5.23epoch/s, loss=16.1, prev_loss=16]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 119.78batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   2%|▌                         | 4/200 [00:00<00:37,  5.27epoch/s, loss=15.2, prev_loss=16.1]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 115.21batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   2%|▋                         | 5/200 [00:00<00:36,  5.30epoch/s, loss=14.7, prev_loss=15.2]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 101.84batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   3%|▊                         | 6/200 [00:01<00:37,  5.22epoch/s, loss=13.9, prev_loss=14.7]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 105.68batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   4%|▉                         | 7/200 [00:01<00:36,  5.23epoch/s, loss=13.8, prev_loss=13.9]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 106.70batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   4%|█                         | 8/200 [00:01<00:36,  5.25epoch/s, loss=13.7, prev_loss=13.8]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 105.21batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   4%|█▏                        | 9/200 [00:01<00:36,  5.24epoch/s, loss=12.6, prev_loss=13.7]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 108.57batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   5%|█▎                         | 10/200 [00:01<00:36,  5.28epoch/s, loss=12, prev_loss=12.6]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 107.59batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   6%|█▍                         | 11/200 [00:02<00:35,  5.29epoch/s, loss=12.2, prev_loss=12]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  83%|███████████████████████████████████████████▎        | 10/12 [00:00<00:00, 97.22batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   6%|█▌                       | 12/200 [00:02<00:35,  5.23epoch/s, loss=11.6, prev_loss=12.2]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  83%|███████████████████████████████████████████▎        | 10/12 [00:00<00:00, 97.59batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   6%|█▋                       | 13/200 [00:02<00:36,  5.13epoch/s, loss=11.3, prev_loss=11.6]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 101.05batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   7%|█▊                       | 14/200 [00:02<00:36,  5.10epoch/s, loss=11.4, prev_loss=11.3]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  58%|██████████████████████████████▉                      | 7/12 [00:00<00:00, 65.62batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   8%|█▉                       | 15/200 [00:02<00:40,  4.55epoch/s, loss=10.8, prev_loss=11.4]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  58%|██████████████████████████████▉                      | 7/12 [00:00<00:00, 64.62batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   8%|██                       | 16/200 [00:03<00:43,  4.27epoch/s, loss=10.1, prev_loss=10.8]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  58%|██████████████████████████████▉                      | 7/12 [00:00<00:00, 65.56batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   8%|██▏                      | 17/200 [00:03<00:44,  4.12epoch/s, loss=9.62, prev_loss=10.1]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  58%|██████████████████████████████▉                      | 7/12 [00:00<00:00, 65.55batch/s]\u001b[A\n",
      "Training epochs on cuda:0:   9%|██▎                      | 18/200 [00:03<00:44,  4.06epoch/s, loss=9.35, prev_loss=9.62]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  10%|██▍                      | 19/200 [00:03<00:40,  4.52epoch/s, loss=9.33, prev_loss=9.35]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  10%|██▌                      | 20/200 [00:04<00:37,  4.79epoch/s, loss=8.85, prev_loss=9.33]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  10%|██▋                      | 21/200 [00:04<00:36,  4.91epoch/s, loss=8.62, prev_loss=8.85]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  11%|██▊                      | 22/200 [00:04<00:33,  5.28epoch/s, loss=8.37, prev_loss=8.62]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  12%|██▉                      | 23/200 [00:04<00:31,  5.57epoch/s, loss=8.05, prev_loss=8.37]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  12%|███                      | 24/200 [00:04<00:31,  5.58epoch/s, loss=7.73, prev_loss=8.05]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  12%|███▎                      | 25/200 [00:04<00:29,  5.91epoch/s, loss=7.6, prev_loss=7.73]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  13%|███▍                      | 26/200 [00:05<00:28,  6.03epoch/s, loss=7.31, prev_loss=7.6]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  14%|███▍                     | 27/200 [00:05<00:28,  6.12epoch/s, loss=6.88, prev_loss=7.31]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  14%|███▋                      | 28/200 [00:05<00:28,  6.12epoch/s, loss=6.9, prev_loss=6.88]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  14%|███▊                      | 29/200 [00:05<00:27,  6.23epoch/s, loss=7.06, prev_loss=6.9]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  15%|███▊                     | 30/200 [00:05<00:27,  6.12epoch/s, loss=7.05, prev_loss=7.06]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  16%|███▉                     | 31/200 [00:05<00:27,  6.19epoch/s, loss=6.28, prev_loss=7.05]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  16%|████                     | 32/200 [00:06<00:27,  6.08epoch/s, loss=5.83, prev_loss=6.28]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  16%|████▏                    | 33/200 [00:06<00:27,  6.13epoch/s, loss=5.98, prev_loss=5.83]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  17%|████▎                    | 34/200 [00:06<00:26,  6.20epoch/s, loss=5.46, prev_loss=5.98]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  18%|████▍                    | 35/200 [00:06<00:26,  6.25epoch/s, loss=5.57, prev_loss=5.46]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  18%|████▋                     | 36/200 [00:06<00:26,  6.20epoch/s, loss=5.5, prev_loss=5.57]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  18%|████▊                     | 37/200 [00:06<00:26,  6.11epoch/s, loss=5.42, prev_loss=5.5]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  19%|████▊                    | 38/200 [00:07<00:26,  6.10epoch/s, loss=5.13, prev_loss=5.42]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  20%|████▉                    | 39/200 [00:07<00:26,  6.10epoch/s, loss=5.19, prev_loss=5.13]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  20%|█████                    | 40/200 [00:07<00:26,  6.07epoch/s, loss=4.72, prev_loss=5.19]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  20%|█████▏                   | 41/200 [00:07<00:26,  6.05epoch/s, loss=4.72, prev_loss=4.72]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  21%|█████▎                   | 42/200 [00:07<00:26,  6.02epoch/s, loss=4.67, prev_loss=4.72]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  22%|█████▍                   | 43/200 [00:07<00:26,  6.02epoch/s, loss=4.03, prev_loss=4.67]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  22%|█████▌                   | 44/200 [00:08<00:26,  5.93epoch/s, loss=4.33, prev_loss=4.03]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  22%|█████▋                   | 45/200 [00:08<00:26,  5.93epoch/s, loss=4.16, prev_loss=4.33]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  23%|█████▊                   | 46/200 [00:08<00:25,  5.95epoch/s, loss=3.94, prev_loss=4.16]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  24%|█████▉                   | 47/200 [00:08<00:25,  5.96epoch/s, loss=4.13, prev_loss=3.94]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  24%|██████▏                   | 48/200 [00:08<00:25,  5.93epoch/s, loss=3.8, prev_loss=4.13]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  24%|██████▎                   | 49/200 [00:08<00:25,  5.85epoch/s, loss=3.72, prev_loss=3.8]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  25%|██████▎                  | 50/200 [00:09<00:25,  5.87epoch/s, loss=3.54, prev_loss=3.72]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 107.79batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  26%|██████▍                  | 51/200 [00:09<00:26,  5.69epoch/s, loss=3.36, prev_loss=3.54]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  26%|██████▌                  | 52/200 [00:09<00:25,  5.70epoch/s, loss=3.49, prev_loss=3.36]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  26%|██████▋                  | 53/200 [00:09<00:25,  5.74epoch/s, loss=3.13, prev_loss=3.49]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  27%|██████▊                  | 54/200 [00:09<00:25,  5.79epoch/s, loss=3.18, prev_loss=3.13]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 113.11batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  28%|██████▉                  | 55/200 [00:09<00:25,  5.65epoch/s, loss=3.26, prev_loss=3.18]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 117.15batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  28%|███████                  | 56/200 [00:10<00:26,  5.52epoch/s, loss=2.81, prev_loss=3.26]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 112.22batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  28%|███████                  | 57/200 [00:10<00:26,  5.42epoch/s, loss=2.98, prev_loss=2.81]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 109.43batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  29%|███████▏                 | 58/200 [00:10<00:26,  5.37epoch/s, loss=3.05, prev_loss=2.98]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 115.79batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  30%|███████▋                  | 59/200 [00:10<00:25,  5.43epoch/s, loss=2.8, prev_loss=3.05]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 118.50batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  30%|███████▊                  | 60/200 [00:10<00:25,  5.39epoch/s, loss=2.74, prev_loss=2.8]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 106.96batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  30%|███████▋                 | 61/200 [00:11<00:25,  5.37epoch/s, loss=2.73, prev_loss=2.74]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 111.59batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  31%|███████▊                 | 62/200 [00:11<00:25,  5.36epoch/s, loss=2.55, prev_loss=2.73]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 109.77batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  32%|███████▉                 | 63/200 [00:11<00:25,  5.35epoch/s, loss=2.41, prev_loss=2.55]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 115.54batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  32%|████████                 | 64/200 [00:11<00:25,  5.41epoch/s, loss=2.49, prev_loss=2.41]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 105.52batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  32%|████████▏                | 65/200 [00:11<00:25,  5.39epoch/s, loss=2.29, prev_loss=2.49]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 109.78batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  33%|████████▎                | 66/200 [00:12<00:25,  5.35epoch/s, loss=2.31, prev_loss=2.29]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 114.73batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  34%|████████▍                | 67/200 [00:12<00:24,  5.38epoch/s, loss=2.05, prev_loss=2.31]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 115.96batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  34%|████████▌                | 68/200 [00:12<00:24,  5.40epoch/s, loss=2.31, prev_loss=2.05]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 108.53batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  34%|████████▋                | 69/200 [00:12<00:24,  5.36epoch/s, loss=2.14, prev_loss=2.31]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 108.11batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  35%|████████▊                | 70/200 [00:12<00:24,  5.30epoch/s, loss=2.08, prev_loss=2.14]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 109.32batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  36%|████████▉                | 71/200 [00:12<00:24,  5.29epoch/s, loss=1.98, prev_loss=2.08]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 112.54batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  36%|█████████                | 72/200 [00:13<00:24,  5.33epoch/s, loss=2.09, prev_loss=1.98]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  83%|███████████████████████████████████████████▎        | 10/12 [00:00<00:00, 99.66batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  36%|█████████▏               | 73/200 [00:13<00:24,  5.27epoch/s, loss=2.02, prev_loss=2.09]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 105.58batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  37%|█████████▎               | 74/200 [00:13<00:24,  5.23epoch/s, loss=1.89, prev_loss=2.02]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  50%|██████████████████████████▌                          | 6/12 [00:00<00:00, 52.36batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|████████████████████████████████████████████████████| 12/12 [00:00<00:00, 56.43batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  38%|█████████▍               | 75/200 [00:13<00:27,  4.49epoch/s, loss=1.93, prev_loss=1.89]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  58%|██████████████████████████████▉                      | 7/12 [00:00<00:00, 63.47batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  38%|█████████▌               | 76/200 [00:14<00:29,  4.26epoch/s, loss=1.85, prev_loss=1.93]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  58%|██████████████████████████████▉                      | 7/12 [00:00<00:00, 64.11batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  38%|█████████▋               | 77/200 [00:14<00:29,  4.10epoch/s, loss=1.84, prev_loss=1.85]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  39%|█████████▊               | 78/200 [00:14<00:26,  4.61epoch/s, loss=1.65, prev_loss=1.84]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  40%|█████████▉               | 79/200 [00:14<00:24,  5.02epoch/s, loss=1.73, prev_loss=1.65]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  40%|██████████▍               | 80/200 [00:14<00:22,  5.30epoch/s, loss=1.7, prev_loss=1.73]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  40%|██████████▌               | 81/200 [00:14<00:21,  5.46epoch/s, loss=1.68, prev_loss=1.7]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  41%|██████████▎              | 82/200 [00:15<00:20,  5.67epoch/s, loss=1.76, prev_loss=1.68]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  42%|██████████▍              | 83/200 [00:15<00:19,  5.90epoch/s, loss=1.65, prev_loss=1.76]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  42%|██████████▉               | 84/200 [00:15<00:19,  6.00epoch/s, loss=1.7, prev_loss=1.65]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  42%|███████████               | 85/200 [00:15<00:19,  5.97epoch/s, loss=1.45, prev_loss=1.7]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  43%|██████████▊              | 86/200 [00:15<00:19,  5.84epoch/s, loss=1.55, prev_loss=1.45]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  44%|███████████▎              | 87/200 [00:15<00:19,  5.90epoch/s, loss=1.5, prev_loss=1.55]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  44%|███████████▍              | 88/200 [00:16<00:18,  6.04epoch/s, loss=1.56, prev_loss=1.5]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  44%|███████████▌              | 89/200 [00:16<00:18,  6.00epoch/s, loss=1.4, prev_loss=1.56]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  45%|███████████▋              | 90/200 [00:16<00:18,  6.00epoch/s, loss=1.35, prev_loss=1.4]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  46%|███████████▍             | 91/200 [00:16<00:18,  6.02epoch/s, loss=1.38, prev_loss=1.35]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  46%|███████████▌             | 92/200 [00:16<00:18,  5.99epoch/s, loss=1.37, prev_loss=1.38]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  46%|███████████▋             | 93/200 [00:16<00:18,  5.94epoch/s, loss=1.29, prev_loss=1.37]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  47%|███████████▊             | 94/200 [00:17<00:17,  6.02epoch/s, loss=1.18, prev_loss=1.29]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  48%|███████████▉             | 95/200 [00:17<00:17,  6.09epoch/s, loss=1.18, prev_loss=1.18]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  48%|████████████             | 96/200 [00:17<00:16,  6.12epoch/s, loss=1.15, prev_loss=1.18]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  48%|████████████▏            | 97/200 [00:17<00:16,  6.06epoch/s, loss=1.18, prev_loss=1.15]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 113.40batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  49%|████████████▎            | 98/200 [00:17<00:17,  5.76epoch/s, loss=1.34, prev_loss=1.18]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 119.93batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  50%|████████████▊             | 99/200 [00:18<00:17,  5.66epoch/s, loss=1.1, prev_loss=1.34]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  50%|████████████▌            | 100/200 [00:18<00:17,  5.73epoch/s, loss=1.11, prev_loss=1.1]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 114.94batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  50%|████████████            | 101/200 [00:18<00:17,  5.65epoch/s, loss=1.17, prev_loss=1.11]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  51%|███████████▋           | 102/200 [00:18<00:17,  5.53epoch/s, loss=0.979, prev_loss=1.17]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 105.84batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  52%|███████████▊           | 103/200 [00:18<00:18,  5.35epoch/s, loss=1.18, prev_loss=0.979]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 119.31batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  52%|████████████▍           | 104/200 [00:18<00:17,  5.41epoch/s, loss=1.04, prev_loss=1.18]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 106.97batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  52%|████████████▌           | 105/200 [00:19<00:17,  5.34epoch/s, loss=1.06, prev_loss=1.04]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 116.80batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  53%|████████████▏          | 106/200 [00:19<00:17,  5.37epoch/s, loss=0.961, prev_loss=1.06]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 117.93batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  54%|████████████▎          | 107/200 [00:19<00:17,  5.38epoch/s, loss=1.08, prev_loss=0.961]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  54%|████████████▍          | 108/200 [00:19<00:17,  5.37epoch/s, loss=0.993, prev_loss=1.08]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 118.33batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  55%|███████████▉          | 109/200 [00:19<00:16,  5.40epoch/s, loss=0.881, prev_loss=0.993]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 104.83batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  55%|████████████          | 110/200 [00:20<00:17,  5.20epoch/s, loss=0.963, prev_loss=0.881]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  83%|███████████████████████████████████████████▎        | 10/12 [00:00<00:00, 96.94batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  56%|████████████▊          | 111/200 [00:20<00:18,  4.92epoch/s, loss=1.01, prev_loss=0.963]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 116.60batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  56%|████████████▉          | 112/200 [00:20<00:17,  4.94epoch/s, loss=0.995, prev_loss=1.01]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 112.78batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  56%|████████████▉          | 113/200 [00:20<00:17,  5.03epoch/s, loss=1.07, prev_loss=0.995]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 109.83batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  57%|█████████████▋          | 114/200 [00:20<00:17,  4.99epoch/s, loss=0.93, prev_loss=1.07]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 107.38batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  57%|█████████████▏         | 115/200 [00:21<00:17,  5.00epoch/s, loss=0.875, prev_loss=0.93]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 115.59batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  58%|████████████▊         | 116/200 [00:21<00:16,  5.11epoch/s, loss=0.884, prev_loss=0.875]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  67%|███████████████████████████████████▎                 | 8/12 [00:00<00:00, 77.44batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  58%|████████████▊         | 117/200 [00:21<00:17,  4.69epoch/s, loss=0.744, prev_loss=0.884]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  50%|██████████████████████████▌                          | 6/12 [00:00<00:00, 59.04batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  59%|████████████▉         | 118/200 [00:21<00:19,  4.23epoch/s, loss=0.811, prev_loss=0.744]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  58%|██████████████████████████████▉                      | 7/12 [00:00<00:00, 63.60batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  60%|█████████████         | 119/200 [00:22<00:20,  4.02epoch/s, loss=0.767, prev_loss=0.811]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  58%|██████████████████████████████▉                      | 7/12 [00:00<00:00, 65.26batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  60%|█████████████▊         | 120/200 [00:22<00:20,  3.91epoch/s, loss=0.74, prev_loss=0.767]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  60%|██████████████▌         | 121/200 [00:22<00:17,  4.39epoch/s, loss=0.77, prev_loss=0.74]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  61%|██████████████         | 122/200 [00:22<00:16,  4.61epoch/s, loss=0.864, prev_loss=0.77]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  62%|█████████████▌        | 123/200 [00:22<00:15,  4.99epoch/s, loss=0.822, prev_loss=0.864]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  62%|█████████████▋        | 124/200 [00:23<00:14,  5.15epoch/s, loss=0.745, prev_loss=0.822]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  62%|█████████████▊        | 125/200 [00:23<00:13,  5.43epoch/s, loss=0.734, prev_loss=0.745]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  63%|█████████████▊        | 126/200 [00:23<00:13,  5.68epoch/s, loss=0.749, prev_loss=0.734]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  64%|███████████████▏        | 127/200 [00:23<00:12,  5.90epoch/s, loss=0.7, prev_loss=0.749]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  64%|███████████████▎        | 128/200 [00:23<00:11,  6.05epoch/s, loss=0.695, prev_loss=0.7]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  64%|██████████████▏       | 129/200 [00:23<00:11,  6.10epoch/s, loss=0.627, prev_loss=0.695]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  65%|██████████████▎       | 130/200 [00:24<00:11,  6.08epoch/s, loss=0.723, prev_loss=0.627]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  66%|██████████████▍       | 131/200 [00:24<00:11,  6.02epoch/s, loss=0.607, prev_loss=0.723]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  66%|██████████████▌       | 132/200 [00:24<00:11,  5.98epoch/s, loss=0.797, prev_loss=0.607]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  66%|██████████████▋       | 133/200 [00:24<00:11,  6.04epoch/s, loss=0.726, prev_loss=0.797]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  67%|██████████████▋       | 134/200 [00:24<00:11,  5.83epoch/s, loss=0.647, prev_loss=0.726]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  68%|██████████████▊       | 135/200 [00:24<00:10,  5.91epoch/s, loss=0.614, prev_loss=0.647]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  68%|██████████████▉       | 136/200 [00:25<00:10,  5.89epoch/s, loss=0.637, prev_loss=0.614]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  68%|███████████████       | 137/200 [00:25<00:10,  5.87epoch/s, loss=0.755, prev_loss=0.637]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  69%|███████████████▏      | 138/200 [00:25<00:10,  5.87epoch/s, loss=0.586, prev_loss=0.755]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  70%|███████████████▎      | 139/200 [00:25<00:10,  5.79epoch/s, loss=0.611, prev_loss=0.586]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  70%|███████████████▍      | 140/200 [00:25<00:10,  5.81epoch/s, loss=0.575, prev_loss=0.611]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  70%|███████████████▌      | 141/200 [00:25<00:10,  5.74epoch/s, loss=0.658, prev_loss=0.575]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 113.86batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  71%|███████████████▌      | 142/200 [00:26<00:10,  5.63epoch/s, loss=0.601, prev_loss=0.658]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 115.81batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  72%|███████████████▋      | 143/200 [00:26<00:10,  5.56epoch/s, loss=0.509, prev_loss=0.601]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  72%|███████████████▊      | 144/200 [00:26<00:10,  5.54epoch/s, loss=0.552, prev_loss=0.509]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  72%|███████████████▉      | 145/200 [00:26<00:09,  5.60epoch/s, loss=0.564, prev_loss=0.552]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 113.06batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  73%|████████████████      | 146/200 [00:26<00:09,  5.46epoch/s, loss=0.573, prev_loss=0.564]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 113.28batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  74%|████████████████▏     | 147/200 [00:27<00:09,  5.41epoch/s, loss=0.481, prev_loss=0.573]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  74%|████████████████▎     | 148/200 [00:27<00:09,  5.50epoch/s, loss=0.528, prev_loss=0.481]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  74%|████████████████▍     | 149/200 [00:27<00:09,  5.59epoch/s, loss=0.505, prev_loss=0.528]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 117.24batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  75%|████████████████▌     | 150/200 [00:27<00:09,  5.44epoch/s, loss=0.522, prev_loss=0.505]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  76%|████████████████▌     | 151/200 [00:27<00:08,  5.51epoch/s, loss=0.516, prev_loss=0.522]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 108.14batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  76%|████████████████▋     | 152/200 [00:27<00:08,  5.38epoch/s, loss=0.584, prev_loss=0.516]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 108.51batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  76%|████████████████▊     | 153/200 [00:28<00:08,  5.28epoch/s, loss=0.607, prev_loss=0.584]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 116.15batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  77%|████████████████▉     | 154/200 [00:28<00:08,  5.31epoch/s, loss=0.456, prev_loss=0.607]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 113.81batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  78%|█████████████████     | 155/200 [00:28<00:08,  5.35epoch/s, loss=0.499, prev_loss=0.456]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 113.65batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  78%|█████████████████▏    | 156/200 [00:28<00:08,  5.39epoch/s, loss=0.504, prev_loss=0.499]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 106.98batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  78%|█████████████████▎    | 157/200 [00:28<00:08,  5.36epoch/s, loss=0.495, prev_loss=0.504]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 111.90batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  79%|█████████████████▍    | 158/200 [00:29<00:07,  5.31epoch/s, loss=0.419, prev_loss=0.495]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 114.40batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  80%|██████████████████▎    | 159/200 [00:29<00:07,  5.30epoch/s, loss=0.44, prev_loss=0.419]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 115.87batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  80%|██████████████████▍    | 160/200 [00:29<00:07,  5.34epoch/s, loss=0.373, prev_loss=0.44]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 108.84batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  80%|█████████████████▋    | 161/200 [00:29<00:07,  5.19epoch/s, loss=0.534, prev_loss=0.373]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 113.29batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  81%|█████████████████▊    | 162/200 [00:29<00:07,  5.14epoch/s, loss=0.471, prev_loss=0.534]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 109.91batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  82%|██████████████████▋    | 163/200 [00:30<00:07,  5.17epoch/s, loss=0.45, prev_loss=0.471]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 115.81batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  82%|██████████████████▊    | 164/200 [00:30<00:06,  5.21epoch/s, loss=0.397, prev_loss=0.45]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 106.45batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  82%|██████████████████▏   | 165/200 [00:30<00:06,  5.18epoch/s, loss=0.311, prev_loss=0.397]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 111.73batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  83%|██████████████████▎   | 166/200 [00:30<00:06,  5.22epoch/s, loss=0.397, prev_loss=0.311]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 106.78batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  84%|██████████████████▎   | 167/200 [00:30<00:06,  5.21epoch/s, loss=0.453, prev_loss=0.397]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  92%|██████████████████████████████████████████████▊    | 11/12 [00:00<00:00, 106.77batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  84%|██████████████████▍   | 168/200 [00:30<00:06,  5.17epoch/s, loss=0.419, prev_loss=0.453]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  50%|██████████████████████████▌                          | 6/12 [00:00<00:00, 55.81batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|████████████████████████████████████████████████████| 12/12 [00:00<00:00, 57.31batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  84%|██████████████████▌   | 169/200 [00:31<00:06,  4.48epoch/s, loss=0.424, prev_loss=0.419]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  50%|██████████████████████████▌                          | 6/12 [00:00<00:00, 59.89batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  85%|██████████████████▋   | 170/200 [00:31<00:07,  4.17epoch/s, loss=0.459, prev_loss=0.424]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  58%|██████████████████████████████▉                      | 7/12 [00:00<00:00, 64.40batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  86%|██████████████████▊   | 171/200 [00:31<00:07,  3.99epoch/s, loss=0.405, prev_loss=0.459]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0:  58%|██████████████████████████████▉                      | 7/12 [00:00<00:00, 65.87batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  86%|██████████████████▉   | 172/200 [00:32<00:07,  3.91epoch/s, loss=0.342, prev_loss=0.405]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  86%|███████████████████▉   | 173/200 [00:32<00:06,  4.48epoch/s, loss=0.36, prev_loss=0.342]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  87%|████████████████████▉   | 174/200 [00:32<00:05,  4.87epoch/s, loss=0.35, prev_loss=0.36]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  88%|████████████████████▏  | 175/200 [00:32<00:04,  5.31epoch/s, loss=0.339, prev_loss=0.35]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  88%|███████████████████▎  | 176/200 [00:32<00:04,  5.63epoch/s, loss=0.301, prev_loss=0.339]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  88%|███████████████████▍  | 177/200 [00:32<00:03,  5.85epoch/s, loss=0.388, prev_loss=0.301]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  89%|███████████████████▌  | 178/200 [00:33<00:03,  5.91epoch/s, loss=0.353, prev_loss=0.388]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  90%|███████████████████▋  | 179/200 [00:33<00:03,  6.08epoch/s, loss=0.324, prev_loss=0.353]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  90%|███████████████████▊  | 180/200 [00:33<00:03,  6.16epoch/s, loss=0.384, prev_loss=0.324]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  90%|███████████████████▉  | 181/200 [00:33<00:03,  6.26epoch/s, loss=0.368, prev_loss=0.384]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  91%|████████████████████  | 182/200 [00:33<00:02,  6.17epoch/s, loss=0.296, prev_loss=0.368]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  92%|████████████████████▏ | 183/200 [00:33<00:02,  6.04epoch/s, loss=0.327, prev_loss=0.296]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  92%|████████████████████▏ | 184/200 [00:34<00:02,  5.85epoch/s, loss=0.318, prev_loss=0.327]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  92%|█████████████████████▎ | 185/200 [00:34<00:02,  5.68epoch/s, loss=0.34, prev_loss=0.318]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  93%|██████████████████████▎ | 186/200 [00:34<00:02,  5.76epoch/s, loss=0.43, prev_loss=0.34]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  94%|█████████████████████▌ | 187/200 [00:34<00:02,  5.82epoch/s, loss=0.279, prev_loss=0.43]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  94%|████████████████████▋ | 188/200 [00:34<00:02,  5.76epoch/s, loss=0.394, prev_loss=0.279]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  94%|████████████████████▊ | 189/200 [00:34<00:01,  5.82epoch/s, loss=0.298, prev_loss=0.394]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  95%|████████████████████▉ | 190/200 [00:35<00:01,  5.82epoch/s, loss=0.324, prev_loss=0.298]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  96%|█████████████████████ | 191/200 [00:35<00:01,  5.86epoch/s, loss=0.294, prev_loss=0.324]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  96%|█████████████████████ | 192/200 [00:35<00:01,  5.83epoch/s, loss=0.242, prev_loss=0.294]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  96%|█████████████████████▏| 193/200 [00:35<00:01,  5.89epoch/s, loss=0.352, prev_loss=0.242]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  97%|█████████████████████▎| 194/200 [00:35<00:01,  5.91epoch/s, loss=0.278, prev_loss=0.352]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 114.64batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  98%|█████████████████████▍| 195/200 [00:35<00:00,  5.75epoch/s, loss=0.314, prev_loss=0.278]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cuda:0: 100%|███████████████████████████████████████████████████| 12/12 [00:00<00:00, 119.75batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  98%|█████████████████████▌| 196/200 [00:36<00:00,  5.70epoch/s, loss=0.304, prev_loss=0.314]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  98%|█████████████████████▋| 197/200 [00:36<00:00,  5.74epoch/s, loss=0.298, prev_loss=0.304]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0:  99%|█████████████████████▊| 198/200 [00:36<00:00,  5.70epoch/s, loss=0.251, prev_loss=0.298]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0: 100%|█████████████████████▉| 199/200 [00:36<00:00,  5.70epoch/s, loss=0.342, prev_loss=0.251]\u001b[A\n",
      "Training batches on cuda:0:   0%|                                                             | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cuda:0: 100%|██████████████████████| 200/200 [00:36<00:00,  5.43epoch/s, loss=0.282, prev_loss=0.342]\u001b[A\n",
      "INFO:pykeen.evaluation.evaluator:Starting batch_size search for evaluation now...\n",
      "INFO:pykeen.evaluation.evaluator:Concluded batch_size search with batch_size=159.\n",
      "Evaluating on cuda:0: 100%|███████████████████████████████████████████████████████| 159/159 [00:00<00:00, 2.39ktriple/s]\n",
      "INFO:pykeen.evaluation.evaluator:Evaluation took 0.07s seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.004788666032254696\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 50.09batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 50.09batch/s]\u001b[A\n",
      "Training epochs on cpu:  74%|█████████████▍    | 149/200 [00:45<00:16,  3.05epoch/s, loss=0.494, prev_loss=0.434]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  42%|████████████████████▍                            | 5/12 [00:00<00:00, 47.20batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|████████████████████████████████████████        | 10/12 [00:00<00:00, 47.01batch/s]\u001b[A\n",
      "Training epochs on cpu:  75%|█████████████▌    | 150/200 [00:45<00:16,  2.99epoch/s, loss=0.473, prev_loss=0.494]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 54.34batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 54.19batch/s]\u001b[A\n",
      "Training epochs on cpu:  76%|█████████████▌    | 151/200 [00:46<00:16,  3.03epoch/s, loss=0.512, prev_loss=0.473]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 56.16batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 55.92batch/s]\u001b[A\n",
      "Training epochs on cpu:  76%|█████████████▋    | 152/200 [00:46<00:15,  3.11epoch/s, loss=0.478, prev_loss=0.512]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 64.45batch/s]\u001b[A\n",
      "Training epochs on cpu:  76%|█████████████▊    | 153/200 [00:46<00:14,  3.15epoch/s, loss=0.507, prev_loss=0.478]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  42%|████████████████████▍                            | 5/12 [00:00<00:00, 49.94batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|████████████████████████████████████████        | 10/12 [00:00<00:00, 48.87batch/s]\u001b[A\n",
      "Training epochs on cpu:  77%|█████████████▊    | 154/200 [00:47<00:14,  3.07epoch/s, loss=0.519, prev_loss=0.507]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 61.02batch/s]\u001b[A\n",
      "Training epochs on cpu:  78%|█████████████▉    | 155/200 [00:47<00:14,  3.15epoch/s, loss=0.452, prev_loss=0.519]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 56.25batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 52.85batch/s]\u001b[A\n",
      "Training epochs on cpu:  78%|██████████████    | 156/200 [00:47<00:13,  3.17epoch/s, loss=0.458, prev_loss=0.452]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  42%|████████████████████▍                            | 5/12 [00:00<00:00, 42.76batch/s]\u001b[A\n",
      "Training batches on cpu:  92%|████████████████████████████████████████████    | 11/12 [00:00<00:00, 48.44batch/s]\u001b[A\n",
      "Training epochs on cpu:  78%|██████████████▏   | 157/200 [00:48<00:14,  3.07epoch/s, loss=0.541, prev_loss=0.458]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 51.40batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 48.48batch/s]\u001b[A\n",
      "Training epochs on cpu:  79%|██████████████▏   | 158/200 [00:48<00:13,  3.01epoch/s, loss=0.458, prev_loss=0.541]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 58.77batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 56.84batch/s]\u001b[A\n",
      "Training epochs on cpu:  80%|██████████████▎   | 159/200 [00:48<00:13,  3.11epoch/s, loss=0.477, prev_loss=0.458]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  42%|████████████████████▍                            | 5/12 [00:00<00:00, 48.78batch/s]\u001b[A\n",
      "Training batches on cpu:  92%|████████████████████████████████████████████    | 11/12 [00:00<00:00, 51.17batch/s]\u001b[A\n",
      "Training epochs on cpu:  80%|██████████████▍   | 160/200 [00:48<00:12,  3.09epoch/s, loss=0.453, prev_loss=0.477]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 55.48batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 56.43batch/s]\u001b[A\n",
      "Training epochs on cpu:  80%|██████████████▍   | 161/200 [00:49<00:12,  3.17epoch/s, loss=0.399, prev_loss=0.453]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 53.80batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 54.59batch/s]\u001b[A\n",
      "Training epochs on cpu:  81%|██████████████▌   | 162/200 [00:49<00:12,  3.14epoch/s, loss=0.385, prev_loss=0.399]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 52.42batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 51.20batch/s]\u001b[A\n",
      "Training epochs on cpu:  82%|██████████████▋   | 163/200 [00:49<00:11,  3.13epoch/s, loss=0.463, prev_loss=0.385]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  42%|████████████████████▍                            | 5/12 [00:00<00:00, 44.95batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|████████████████████████████████████████        | 10/12 [00:00<00:00, 41.95batch/s]\u001b[A\n",
      "Training epochs on cpu:  82%|███████████████▌   | 164/200 [00:50<00:12,  2.93epoch/s, loss=0.42, prev_loss=0.463]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 52.99batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 51.50batch/s]\u001b[A\n",
      "Training epochs on cpu:  82%|███████████████▋   | 165/200 [00:50<00:11,  2.97epoch/s, loss=0.313, prev_loss=0.42]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 50.58batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 51.23batch/s]\u001b[A\n",
      "Training epochs on cpu:  83%|██████████████▉   | 166/200 [00:50<00:11,  3.00epoch/s, loss=0.383, prev_loss=0.313]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 50.36batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 50.57batch/s]\u001b[A\n",
      "Training epochs on cpu:  84%|███████████████   | 167/200 [00:51<00:11,  2.98epoch/s, loss=0.353, prev_loss=0.383]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  42%|████████████████████▍                            | 5/12 [00:00<00:00, 49.46batch/s]\u001b[A\n",
      "Training batches on cpu:  92%|████████████████████████████████████████████    | 11/12 [00:00<00:00, 53.78batch/s]\u001b[A\n",
      "Training epochs on cpu:  84%|███████████████   | 168/200 [00:51<00:10,  3.02epoch/s, loss=0.401, prev_loss=0.353]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 52.31batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 54.87batch/s]\u001b[A\n",
      "Training epochs on cpu:  84%|███████████████▏  | 169/200 [00:51<00:10,  3.10epoch/s, loss=0.326, prev_loss=0.401]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 53.49batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 52.92batch/s]\u001b[A\n",
      "Training epochs on cpu:  85%|███████████████▎  | 170/200 [00:52<00:09,  3.11epoch/s, loss=0.337, prev_loss=0.326]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 50.38batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 51.10batch/s]\u001b[A\n",
      "Training epochs on cpu:  86%|████████████████▏  | 171/200 [00:52<00:09,  3.09epoch/s, loss=0.45, prev_loss=0.337]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  42%|████████████████████▍                            | 5/12 [00:00<00:00, 49.72batch/s]\u001b[A\n",
      "Training batches on cpu:  92%|████████████████████████████████████████████    | 11/12 [00:00<00:00, 50.80batch/s]\u001b[A\n",
      "Training epochs on cpu:  86%|████████████████▎  | 172/200 [00:52<00:09,  3.06epoch/s, loss=0.417, prev_loss=0.45]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 52.55batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 53.27batch/s]\u001b[A\n",
      "Training epochs on cpu:  86%|████████████████▍  | 173/200 [00:53<00:08,  3.09epoch/s, loss=0.41, prev_loss=0.417]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 52.59batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 52.52batch/s]\u001b[A\n",
      "Training epochs on cpu:  87%|████████████████▌  | 174/200 [00:53<00:08,  3.09epoch/s, loss=0.373, prev_loss=0.41]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 52.48batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 52.92batch/s]\u001b[A\n",
      "Training epochs on cpu:  88%|███████████████▊  | 175/200 [00:53<00:08,  3.09epoch/s, loss=0.328, prev_loss=0.373]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 57.26batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 55.43batch/s]\u001b[A\n",
      "Training epochs on cpu:  88%|███████████████▊  | 176/200 [00:54<00:07,  3.13epoch/s, loss=0.273, prev_loss=0.328]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 52.09batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 56.36batch/s]\u001b[A\n",
      "Training epochs on cpu:  88%|███████████████▉  | 177/200 [00:54<00:07,  3.19epoch/s, loss=0.329, prev_loss=0.273]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 53.58batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 53.44batch/s]\u001b[A\n",
      "Training epochs on cpu:  89%|████████████████  | 178/200 [00:54<00:06,  3.18epoch/s, loss=0.374, prev_loss=0.329]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 56.52batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 55.54batch/s]\u001b[A\n",
      "Training epochs on cpu:  90%|████████████████  | 179/200 [00:55<00:06,  3.20epoch/s, loss=0.262, prev_loss=0.374]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 55.81batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 56.01batch/s]\u001b[A\n",
      "Training epochs on cpu:  90%|████████████████▏ | 180/200 [00:55<00:06,  3.18epoch/s, loss=0.339, prev_loss=0.262]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 53.66batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 54.21batch/s]\u001b[A\n",
      "Training epochs on cpu:  90%|████████████████▎ | 181/200 [00:55<00:05,  3.18epoch/s, loss=0.353, prev_loss=0.339]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 54.67batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 52.80batch/s]\u001b[A\n",
      "Training epochs on cpu:  91%|████████████████▍ | 182/200 [00:56<00:05,  3.17epoch/s, loss=0.352, prev_loss=0.353]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 54.49batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 54.71batch/s]\u001b[A\n",
      "Training epochs on cpu:  92%|████████████████▍ | 183/200 [00:56<00:05,  3.19epoch/s, loss=0.332, prev_loss=0.352]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 51.37batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 48.22batch/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training epochs on cpu:  92%|████████████████▌ | 184/200 [00:56<00:05,  3.14epoch/s, loss=0.363, prev_loss=0.332]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 61.26batch/s]\u001b[A\n",
      "Training epochs on cpu:  92%|████████████████▋ | 185/200 [00:56<00:04,  3.23epoch/s, loss=0.273, prev_loss=0.363]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 55.38batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 56.28batch/s]\u001b[A\n",
      "Training epochs on cpu:  93%|████████████████▋ | 186/200 [00:57<00:04,  3.23epoch/s, loss=0.264, prev_loss=0.273]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 62.41batch/s]\u001b[A\n",
      "Training epochs on cpu:  94%|████████████████▊ | 187/200 [00:57<00:03,  3.37epoch/s, loss=0.277, prev_loss=0.264]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 55.88batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 55.80batch/s]\u001b[A\n",
      "Training epochs on cpu:  94%|████████████████▉ | 188/200 [00:57<00:03,  3.34epoch/s, loss=0.306, prev_loss=0.277]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 63.08batch/s]\u001b[A\n",
      "Training epochs on cpu:  94%|█████████████████ | 189/200 [00:58<00:03,  3.42epoch/s, loss=0.338, prev_loss=0.306]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 58.94batch/s]\u001b[A\n",
      "Training epochs on cpu:  95%|█████████████████ | 190/200 [00:58<00:02,  3.49epoch/s, loss=0.332, prev_loss=0.338]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 53.70batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 53.94batch/s]\u001b[A\n",
      "Training epochs on cpu:  96%|██████████████████▏| 191/200 [00:58<00:02,  3.37epoch/s, loss=0.29, prev_loss=0.332]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 52.09batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 52.40batch/s]\u001b[A\n",
      "Training epochs on cpu:  96%|██████████████████▏| 192/200 [00:59<00:02,  3.29epoch/s, loss=0.292, prev_loss=0.29]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 50.62batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 51.22batch/s]\u001b[A\n",
      "Training epochs on cpu:  96%|█████████████████▎| 193/200 [00:59<00:02,  3.23epoch/s, loss=0.313, prev_loss=0.292]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 56.15batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 53.16batch/s]\u001b[A\n",
      "Training epochs on cpu:  97%|█████████████████▍| 194/200 [00:59<00:01,  3.21epoch/s, loss=0.328, prev_loss=0.313]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 53.93batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 52.28batch/s]\u001b[A\n",
      "Training epochs on cpu:  98%|█████████████████▌| 195/200 [01:00<00:01,  3.21epoch/s, loss=0.306, prev_loss=0.328]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 56.19batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 55.70batch/s]\u001b[A\n",
      "Training epochs on cpu:  98%|█████████████████▋| 196/200 [01:00<00:01,  3.21epoch/s, loss=0.254, prev_loss=0.306]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 51.24batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 51.92batch/s]\u001b[A\n",
      "Training epochs on cpu:  98%|█████████████████▋| 197/200 [01:00<00:00,  3.15epoch/s, loss=0.264, prev_loss=0.254]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 51.87batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 52.45batch/s]\u001b[A\n",
      "Training epochs on cpu:  99%|█████████████████▊| 198/200 [01:00<00:00,  3.13epoch/s, loss=0.376, prev_loss=0.264]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 52.66batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 52.20batch/s]\u001b[A\n",
      "Training epochs on cpu: 100%|█████████████████▉| 199/200 [01:01<00:00,  3.09epoch/s, loss=0.317, prev_loss=0.376]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 53.06batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 52.56batch/s]\u001b[A\n",
      "Training epochs on cpu: 100%|██████████████████| 200/200 [01:01<00:00,  3.24epoch/s, loss=0.266, prev_loss=0.317]\u001b[A\n",
      "INFO:pykeen.evaluation.evaluator:Currently automatic memory optimization only supports GPUs, but you're using a CPU. Therefore, the batch_size will be set to the default value.\n",
      "INFO:pykeen.evaluation.evaluator:No evaluation batch_size provided. Setting batch_size to '32'.\n",
      "Evaluating on cpu: 100%|███████████████████████████████████████████████████| 159/159 [00:00<00:00, 1.05ktriple/s]\n",
      "INFO:pykeen.evaluation.evaluator:Evaluation took 0.16s seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0033772025468214507\n"
     ]
    }
   ],
   "source": [
    "pipeline_result = pipeline(\n",
    "    model='ComplEx',\n",
    "    training=got_training,\n",
    "    testing=got_testing,\n",
    "    epochs=200,\n",
    "    dimensions=150,\n",
    "    optimizer='adam',\n",
    "    optimizer_kwargs={'lr':1e-3},\n",
    "    loss='pairwisehinge', \n",
    "    regularizer='LP', \n",
    "    regularizer_kwargs={'p':3, 'weight':1e-5}, \n",
    "    \n",
    "    negative_sampler='basic',\n",
    "    negative_sampler_kwargs=dict(\n",
    "        filtered=True,\n",
    "    )\n",
    ")\n",
    "print(pipeline_result.get_metric('mrr'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To save your learned model and also the results, we need to add checkpoints to the pipeline.\n",
    "By adding training kwargs to the pipeline, the model will be automatically saved. By default, it saves the model after every epoch (checkpoint_frequency=0). You can also set the directory to which the models are saved, but by default they will end up in ~/.data/pykeen/checkpoints."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:pykeen.pipeline.api:loaded random seed 242643974 from checkpoint.\n",
      "INFO:pykeen.pipeline.api:Using device: None\n",
      "INFO:pykeen.training.training_loop:=> loading checkpoint 'checkpoint_dir/got_complex_checkpoint.pt'\n",
      "INFO:pykeen.training.training_loop:=> loaded checkpoint 'checkpoint_dir/got_complex_checkpoint.pt' stopped after having finished epoch 200\n",
      "INFO:pykeen.stoppers.stopper:=> loading stopper summary dict from training loop checkpoint in 'checkpoint_dir/got_complex_checkpoint.pt'\n",
      "INFO:pykeen.stoppers.stopper:=> loaded stopper summary dictionary from checkpoint in 'checkpoint_dir/got_complex_checkpoint.pt'\n",
      "WARNING:pykeen.training.training_loop:the training loop was configured with a stopper but no stopper configuration was saved in the checkpoint\n",
      "Training epochs on cuda:0: 100%|███████████████████████████████████████████████████████████| 200/200 [00:00<?, ?epoch/s]\n",
      "INFO:pykeen.evaluation.evaluator:Starting batch_size search for evaluation now...\n",
      "INFO:pykeen.evaluation.evaluator:Concluded batch_size search with batch_size=159.\n",
      "Evaluating on cuda:0: 100%|███████████████████████████████████████████████████████| 159/159 [00:00<00:00, 2.34ktriple/s]\n",
      "INFO:pykeen.evaluation.evaluator:Evaluation took 0.08s seconds\n"
     ]
    }
   ],
   "source": [
    "pipeline_result = pipeline(\n",
    "    model='ComplEx',\n",
    "    training=got_training,\n",
    "    testing=got_testing,\n",
    "    training_kwargs=dict(\n",
    "        num_epochs=200,\n",
    "        checkpoint_name='got_complex_checkpoint.pt',\n",
    "        checkpoint_directory='checkpoint_dir/',\n",
    "        checkpoint_frequency=20,\n",
    "    ),\n",
    "    dimensions=150,\n",
    "    optimizer='adam',\n",
    "    optimizer_kwargs={'lr':1e-3},\n",
    "    loss='pairwisehinge', \n",
    "    regularizer='LP', \n",
    "    regularizer_kwargs={'p':3, 'weight':1e-5}, \n",
    "    negative_sampler='basic',\n",
    "    negative_sampler_kwargs=dict(\n",
    "        filtered=True,\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is another way to save models, but for that we need to do the training and evaluating outside of the pipeline model. Below is an example of the above model training outside of the pipeline module."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:pykeen.models.base:No random seed is specified. This may lead to non-reproducible results.\n",
      "INFO:pykeen.training.training_loop:Currently automatic memory optimization only supports GPUs, but you're using a CPU. Therefore, the batch_size will be set to the default value '{batch_size}'\n",
      "Training epochs on cpu:   0%|                                                                | 0/200 [00:00<?, ?epoch/s]\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|█████████████████████████████████████████████▊         | 10/12 [00:00<00:00, 92.13batch/s]\u001b[A\n",
      "Training epochs on cpu:   0%|▏                             | 1/200 [00:00<00:41,  4.81epoch/s, loss=11.4, prev_loss=nan]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:   1%|▎                              | 2/200 [00:00<00:33,  5.86epoch/s, loss=11, prev_loss=11.4]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:   2%|▍                              | 3/200 [00:00<00:33,  5.90epoch/s, loss=10.6, prev_loss=11]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:   2%|▌                            | 4/200 [00:00<00:33,  5.79epoch/s, loss=10.1, prev_loss=10.6]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|█████████████████████████████████████████████▊         | 10/12 [00:00<00:00, 93.22batch/s]\u001b[A\n",
      "Training epochs on cpu:   2%|▋                            | 5/200 [00:00<00:36,  5.39epoch/s, loss=9.93, prev_loss=10.1]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:   3%|▊                            | 6/200 [00:01<00:34,  5.68epoch/s, loss=9.32, prev_loss=9.93]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:   4%|█                            | 7/200 [00:01<00:33,  5.80epoch/s, loss=8.93, prev_loss=9.32]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:   4%|█▏                           | 8/200 [00:01<00:32,  5.99epoch/s, loss=8.94, prev_loss=8.93]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:   4%|█▎                           | 9/200 [00:01<00:30,  6.21epoch/s, loss=8.79, prev_loss=8.94]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:   5%|█▍                          | 10/200 [00:01<00:30,  6.30epoch/s, loss=8.29, prev_loss=8.79]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:   6%|█▌                          | 11/200 [00:01<00:30,  6.29epoch/s, loss=8.19, prev_loss=8.29]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:   6%|█▋                          | 12/200 [00:01<00:29,  6.36epoch/s, loss=8.08, prev_loss=8.19]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:   6%|█▊                          | 13/200 [00:02<00:29,  6.34epoch/s, loss=7.63, prev_loss=8.08]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:   7%|█▉                          | 14/200 [00:02<00:29,  6.30epoch/s, loss=7.67, prev_loss=7.63]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:   8%|██▏                          | 15/200 [00:02<00:29,  6.30epoch/s, loss=7.5, prev_loss=7.67]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|█████████████████████████████████████████████▊         | 10/12 [00:00<00:00, 93.60batch/s]\u001b[A\n",
      "Training epochs on cpu:   8%|██▎                          | 16/200 [00:02<00:31,  5.81epoch/s, loss=7.21, prev_loss=7.5]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:   8%|██▍                          | 17/200 [00:02<00:30,  6.03epoch/s, loss=7.4, prev_loss=7.21]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:   9%|██▌                          | 18/200 [00:02<00:29,  6.11epoch/s, loss=7.02, prev_loss=7.4]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  10%|██▋                         | 19/200 [00:03<00:29,  6.17epoch/s, loss=6.82, prev_loss=7.02]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 117.11batch/s]\u001b[A\n",
      "Training epochs on cpu:  10%|██▊                         | 20/200 [00:03<00:30,  5.98epoch/s, loss=6.32, prev_loss=6.82]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 118.14batch/s]\u001b[A\n",
      "Training epochs on cpu:  10%|██▉                         | 21/200 [00:03<00:30,  5.84epoch/s, loss=6.52, prev_loss=6.32]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|█████████████████████████████████████████████▊         | 10/12 [00:00<00:00, 96.34batch/s]\u001b[A\n",
      "Training epochs on cpu:  11%|███▏                         | 22/200 [00:03<00:32,  5.50epoch/s, loss=6.6, prev_loss=6.52]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  12%|███▎                         | 23/200 [00:03<00:31,  5.67epoch/s, loss=6.25, prev_loss=6.6]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|█████████████████████████████████████████████▊         | 10/12 [00:00<00:00, 99.15batch/s]\u001b[A\n",
      "Training epochs on cpu:  12%|███▎                        | 24/200 [00:04<00:32,  5.42epoch/s, loss=6.45, prev_loss=6.25]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  75%|██████████████████████████████████████████              | 9/12 [00:00<00:00, 87.84batch/s]\u001b[A\n",
      "Training epochs on cpu:  12%|███▌                        | 25/200 [00:04<00:34,  5.12epoch/s, loss=6.15, prev_loss=6.45]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  13%|███▋                        | 26/200 [00:04<00:32,  5.31epoch/s, loss=5.98, prev_loss=6.15]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  92%|█████████████████████████████████████████████████▌    | 11/12 [00:00<00:00, 105.28batch/s]\u001b[A\n",
      "Training epochs on cpu:  14%|███▊                        | 27/200 [00:04<00:32,  5.27epoch/s, loss=6.08, prev_loss=5.98]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 112.54batch/s]\u001b[A\n",
      "Training epochs on cpu:  14%|███▉                        | 28/200 [00:04<00:32,  5.32epoch/s, loss=5.92, prev_loss=6.08]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  14%|████                        | 29/200 [00:05<00:30,  5.52epoch/s, loss=5.64, prev_loss=5.92]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  15%|████▏                       | 30/200 [00:05<00:29,  5.83epoch/s, loss=5.75, prev_loss=5.64]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  16%|████▎                       | 31/200 [00:05<00:27,  6.05epoch/s, loss=5.67, prev_loss=5.75]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  16%|████▍                       | 32/200 [00:05<00:27,  6.13epoch/s, loss=5.41, prev_loss=5.67]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|█████████████████████████████████████████████▊         | 10/12 [00:00<00:00, 98.84batch/s]\u001b[A\n",
      "Training epochs on cpu:  16%|████▌                       | 33/200 [00:05<00:28,  5.77epoch/s, loss=5.52, prev_loss=5.41]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  17%|████▊                       | 34/200 [00:05<00:27,  5.95epoch/s, loss=5.53, prev_loss=5.52]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  18%|████▉                       | 35/200 [00:05<00:27,  6.09epoch/s, loss=5.33, prev_loss=5.53]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 112.02batch/s]\u001b[A\n",
      "Training epochs on cpu:  18%|█████                       | 36/200 [00:06<00:27,  5.88epoch/s, loss=5.29, prev_loss=5.33]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  18%|█████▏                      | 37/200 [00:06<00:26,  6.05epoch/s, loss=5.22, prev_loss=5.29]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  92%|█████████████████████████████████████████████████▌    | 11/12 [00:00<00:00, 106.36batch/s]\u001b[A\n",
      "Training epochs on cpu:  19%|█████▎                      | 38/200 [00:06<00:28,  5.67epoch/s, loss=5.34, prev_loss=5.22]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  20%|█████▍                      | 39/200 [00:06<00:28,  5.70epoch/s, loss=5.27, prev_loss=5.34]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|█████████████████████████████████████████████▊         | 10/12 [00:00<00:00, 96.19batch/s]\u001b[A\n",
      "Training epochs on cpu:  20%|█████▌                      | 40/200 [00:06<00:29,  5.43epoch/s, loss=5.07, prev_loss=5.27]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|█████████████████████████████████████████████▊         | 10/12 [00:00<00:00, 95.85batch/s]\u001b[A\n",
      "Training epochs on cpu:  20%|█████▋                      | 41/200 [00:07<00:30,  5.25epoch/s, loss=5.13, prev_loss=5.07]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  21%|██████                       | 42/200 [00:07<00:29,  5.44epoch/s, loss=5.2, prev_loss=5.13]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  22%|██████▏                      | 43/200 [00:07<00:27,  5.69epoch/s, loss=5.14, prev_loss=5.2]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|█████████████████████████████████████████████▊         | 10/12 [00:00<00:00, 93.44batch/s]\u001b[A\n",
      "Training epochs on cpu:  22%|██████▏                     | 44/200 [00:07<00:28,  5.46epoch/s, loss=4.98, prev_loss=5.14]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  22%|██████▎                     | 45/200 [00:07<00:27,  5.66epoch/s, loss=5.16, prev_loss=4.98]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  23%|██████▍                     | 46/200 [00:07<00:26,  5.85epoch/s, loss=5.07, prev_loss=5.16]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  24%|██████▌                     | 47/200 [00:08<00:25,  5.99epoch/s, loss=4.93, prev_loss=5.07]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  24%|██████▋                     | 48/200 [00:08<00:25,  5.89epoch/s, loss=4.98, prev_loss=4.93]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  24%|██████▊                     | 49/200 [00:08<00:25,  5.87epoch/s, loss=5.06, prev_loss=4.98]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  25%|███████                     | 50/200 [00:08<00:25,  5.97epoch/s, loss=4.88, prev_loss=5.06]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|█████████████████████████████████████████████▊         | 10/12 [00:00<00:00, 98.32batch/s]\u001b[A\n",
      "Training epochs on cpu:  26%|███████▏                    | 51/200 [00:08<00:26,  5.60epoch/s, loss=5.16, prev_loss=4.88]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  26%|███████▎                    | 52/200 [00:08<00:25,  5.70epoch/s, loss=4.81, prev_loss=5.16]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  26%|███████▍                    | 53/200 [00:09<00:25,  5.84epoch/s, loss=4.85, prev_loss=4.81]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  27%|███████▌                    | 54/200 [00:09<00:24,  5.98epoch/s, loss=4.73, prev_loss=4.85]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  28%|███████▋                    | 55/200 [00:09<00:24,  5.91epoch/s, loss=5.09, prev_loss=4.73]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  28%|███████▊                    | 56/200 [00:09<00:24,  5.84epoch/s, loss=4.84, prev_loss=5.09]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  28%|███████▉                    | 57/200 [00:09<00:24,  5.87epoch/s, loss=4.85, prev_loss=4.84]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 114.77batch/s]\u001b[A\n",
      "Training epochs on cpu:  29%|████████                    | 58/200 [00:10<00:24,  5.69epoch/s, loss=4.85, prev_loss=4.85]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  30%|████████▎                   | 59/200 [00:10<00:24,  5.71epoch/s, loss=4.65, prev_loss=4.85]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  30%|████████▍                   | 60/200 [00:10<00:23,  5.84epoch/s, loss=4.56, prev_loss=4.65]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  30%|████████▌                   | 61/200 [00:10<00:23,  5.86epoch/s, loss=4.86, prev_loss=4.56]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  92%|█████████████████████████████████████████████████▌    | 11/12 [00:00<00:00, 108.03batch/s]\u001b[A\n",
      "Training epochs on cpu:  31%|████████▋                   | 62/200 [00:10<00:24,  5.66epoch/s, loss=4.52, prev_loss=4.86]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  32%|████████▊                   | 63/200 [00:10<00:23,  5.83epoch/s, loss=4.72, prev_loss=4.52]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  32%|█████████▎                   | 64/200 [00:11<00:23,  5.83epoch/s, loss=4.6, prev_loss=4.72]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  32%|█████████▍                   | 65/200 [00:11<00:22,  5.99epoch/s, loss=4.66, prev_loss=4.6]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  33%|█████████▏                  | 66/200 [00:11<00:22,  6.07epoch/s, loss=4.61, prev_loss=4.66]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  34%|█████████▋                   | 67/200 [00:11<00:22,  6.03epoch/s, loss=4.7, prev_loss=4.61]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  34%|██████████▏                   | 68/200 [00:11<00:21,  6.22epoch/s, loss=4.7, prev_loss=4.7]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  34%|██████████                   | 69/200 [00:11<00:21,  6.22epoch/s, loss=4.53, prev_loss=4.7]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  35%|█████████▊                  | 70/200 [00:11<00:20,  6.32epoch/s, loss=4.65, prev_loss=4.53]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  36%|█████████▉                  | 71/200 [00:12<00:21,  6.13epoch/s, loss=4.76, prev_loss=4.65]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  36%|██████████                  | 72/200 [00:12<00:20,  6.18epoch/s, loss=4.64, prev_loss=4.76]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  36%|██████████▏                 | 73/200 [00:12<00:20,  6.07epoch/s, loss=4.38, prev_loss=4.64]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 119.82batch/s]\u001b[A\n",
      "Training epochs on cpu:  37%|██████████▎                 | 74/200 [00:12<00:21,  5.93epoch/s, loss=4.53, prev_loss=4.38]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  38%|██████████▌                 | 75/200 [00:12<00:20,  6.08epoch/s, loss=4.63, prev_loss=4.53]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  38%|██████████▋                 | 76/200 [00:12<00:20,  6.13epoch/s, loss=4.33, prev_loss=4.63]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 119.56batch/s]\u001b[A\n",
      "Training epochs on cpu:  38%|██████████▊                 | 77/200 [00:13<00:20,  5.97epoch/s, loss=4.58, prev_loss=4.33]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  92%|█████████████████████████████████████████████████▌    | 11/12 [00:00<00:00, 100.13batch/s]\u001b[A\n",
      "Training epochs on cpu:  39%|██████████▉                 | 78/200 [00:13<00:21,  5.66epoch/s, loss=4.78, prev_loss=4.58]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  92%|█████████████████████████████████████████████████▌    | 11/12 [00:00<00:00, 108.26batch/s]\u001b[A\n",
      "Training epochs on cpu:  40%|███████████                 | 79/200 [00:13<00:21,  5.53epoch/s, loss=4.45, prev_loss=4.78]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  40%|███████████▏                | 80/200 [00:13<00:20,  5.78epoch/s, loss=4.45, prev_loss=4.45]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  92%|█████████████████████████████████████████████████▌    | 11/12 [00:00<00:00, 101.33batch/s]\u001b[A\n",
      "Training epochs on cpu:  40%|███████████▋                 | 81/200 [00:13<00:21,  5.57epoch/s, loss=4.5, prev_loss=4.45]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  41%|███████████▉                 | 82/200 [00:14<00:20,  5.84epoch/s, loss=4.48, prev_loss=4.5]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  42%|███████████▌                | 83/200 [00:14<00:19,  6.00epoch/s, loss=4.58, prev_loss=4.48]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  42%|███████████▊                | 84/200 [00:14<00:19,  6.01epoch/s, loss=4.36, prev_loss=4.58]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  42%|███████████▉                | 85/200 [00:14<00:18,  6.06epoch/s, loss=4.24, prev_loss=4.36]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  43%|████████████                | 86/200 [00:14<00:19,  5.99epoch/s, loss=4.33, prev_loss=4.24]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 112.24batch/s]\u001b[A\n",
      "Training epochs on cpu:  44%|████████████▏               | 87/200 [00:14<00:19,  5.82epoch/s, loss=4.52, prev_loss=4.33]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  44%|████████████▎               | 88/200 [00:15<00:18,  6.03epoch/s, loss=4.42, prev_loss=4.52]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  44%|████████████▍               | 89/200 [00:15<00:17,  6.17epoch/s, loss=4.21, prev_loss=4.42]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  45%|█████████████                | 90/200 [00:15<00:17,  6.16epoch/s, loss=4.5, prev_loss=4.21]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  46%|█████████████▏               | 91/200 [00:15<00:17,  6.29epoch/s, loss=4.22, prev_loss=4.5]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 111.44batch/s]\u001b[A\n",
      "Training epochs on cpu:  46%|████████████▉               | 92/200 [00:15<00:18,  5.95epoch/s, loss=4.34, prev_loss=4.22]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  46%|█████████████               | 93/200 [00:15<00:18,  5.90epoch/s, loss=4.27, prev_loss=4.34]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 112.14batch/s]\u001b[A\n",
      "Training epochs on cpu:  47%|█████████████▏              | 94/200 [00:16<00:18,  5.76epoch/s, loss=4.29, prev_loss=4.27]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|█████████████████████████████████████████████▊         | 10/12 [00:00<00:00, 98.22batch/s]\u001b[A\n",
      "Training epochs on cpu:  48%|█████████████▊               | 95/200 [00:16<00:19,  5.45epoch/s, loss=4.2, prev_loss=4.29]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  48%|█████████████▉               | 96/200 [00:16<00:18,  5.54epoch/s, loss=4.54, prev_loss=4.2]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  48%|█████████████▌              | 97/200 [00:16<00:18,  5.71epoch/s, loss=4.12, prev_loss=4.54]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  49%|█████████████▋              | 98/200 [00:16<00:17,  5.76epoch/s, loss=4.32, prev_loss=4.12]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  50%|█████████████▊              | 99/200 [00:16<00:17,  5.92epoch/s, loss=4.18, prev_loss=4.32]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  50%|█████████████▌             | 100/200 [00:17<00:16,  5.93epoch/s, loss=3.97, prev_loss=4.18]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  50%|█████████████▋             | 101/200 [00:17<00:16,  6.04epoch/s, loss=4.23, prev_loss=3.97]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  51%|█████████████▊             | 102/200 [00:17<00:15,  6.22epoch/s, loss=3.97, prev_loss=4.23]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  52%|█████████████▉             | 103/200 [00:17<00:15,  6.30epoch/s, loss=4.29, prev_loss=3.97]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  52%|██████████████             | 104/200 [00:17<00:15,  6.16epoch/s, loss=4.14, prev_loss=4.29]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  52%|██████████████▏            | 105/200 [00:17<00:15,  6.19epoch/s, loss=4.09, prev_loss=4.14]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  53%|██████████████▎            | 106/200 [00:18<00:14,  6.29epoch/s, loss=4.15, prev_loss=4.09]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  54%|██████████████▍            | 107/200 [00:18<00:14,  6.26epoch/s, loss=4.11, prev_loss=4.15]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  54%|██████████████▌            | 108/200 [00:18<00:15,  6.12epoch/s, loss=4.23, prev_loss=4.11]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  92%|█████████████████████████████████████████████████▌    | 11/12 [00:00<00:00, 109.74batch/s]\u001b[A\n",
      "Training epochs on cpu:  55%|██████████████▋            | 109/200 [00:18<00:15,  5.82epoch/s, loss=4.04, prev_loss=4.23]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  55%|██████████████▊            | 110/200 [00:18<00:15,  5.81epoch/s, loss=3.94, prev_loss=4.04]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  56%|██████████████▉            | 111/200 [00:18<00:14,  6.01epoch/s, loss=4.05, prev_loss=3.94]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  56%|███████████████            | 112/200 [00:19<00:14,  5.93epoch/s, loss=4.04, prev_loss=4.05]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  56%|███████████████▎           | 113/200 [00:19<00:14,  6.00epoch/s, loss=4.14, prev_loss=4.04]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 112.85batch/s]\u001b[A\n",
      "Training epochs on cpu:  57%|███████████████▍           | 114/200 [00:19<00:14,  5.82epoch/s, loss=3.86, prev_loss=4.14]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|█████████████████████████████████████████████▊         | 10/12 [00:00<00:00, 99.12batch/s]\u001b[A\n",
      "Training epochs on cpu:  57%|███████████████▌           | 115/200 [00:19<00:15,  5.58epoch/s, loss=3.81, prev_loss=3.86]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  58%|███████████████▋           | 116/200 [00:19<00:14,  5.74epoch/s, loss=3.95, prev_loss=3.81]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  58%|███████████████▊           | 117/200 [00:19<00:14,  5.72epoch/s, loss=4.13, prev_loss=3.95]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 117.64batch/s]\u001b[A\n",
      "Training epochs on cpu:  59%|███████████████▉           | 118/200 [00:20<00:14,  5.68epoch/s, loss=4.15, prev_loss=4.13]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  60%|████████████████           | 119/200 [00:20<00:13,  5.82epoch/s, loss=3.94, prev_loss=4.15]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  60%|████████████████▏          | 120/200 [00:20<00:13,  5.97epoch/s, loss=3.98, prev_loss=3.94]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  92%|█████████████████████████████████████████████████▌    | 11/12 [00:00<00:00, 108.19batch/s]\u001b[A\n",
      "Training epochs on cpu:  60%|████████████████▎          | 121/200 [00:20<00:13,  5.68epoch/s, loss=3.94, prev_loss=3.98]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  61%|████████████████▍          | 122/200 [00:20<00:13,  5.83epoch/s, loss=3.74, prev_loss=3.94]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|█████████████████████████████████████████████▊         | 10/12 [00:00<00:00, 98.39batch/s]\u001b[A\n",
      "Training epochs on cpu:  62%|████████████████▌          | 123/200 [00:21<00:13,  5.54epoch/s, loss=3.87, prev_loss=3.74]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  62%|████████████████▋          | 124/200 [00:21<00:13,  5.69epoch/s, loss=3.58, prev_loss=3.87]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  62%|████████████████▉          | 125/200 [00:21<00:13,  5.73epoch/s, loss=3.87, prev_loss=3.58]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  63%|█████████████████          | 126/200 [00:21<00:12,  5.81epoch/s, loss=3.81, prev_loss=3.87]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  64%|█████████████████▏         | 127/200 [00:21<00:12,  5.85epoch/s, loss=3.78, prev_loss=3.81]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  64%|█████████████████▎         | 128/200 [00:21<00:11,  6.00epoch/s, loss=3.85, prev_loss=3.78]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 110.69batch/s]\u001b[A\n",
      "Training epochs on cpu:  64%|█████████████████▍         | 129/200 [00:22<00:12,  5.81epoch/s, loss=4.04, prev_loss=3.85]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  65%|█████████████████▌         | 130/200 [00:22<00:11,  6.01epoch/s, loss=3.66, prev_loss=4.04]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  66%|█████████████████▋         | 131/200 [00:22<00:11,  5.95epoch/s, loss=3.68, prev_loss=3.66]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  66%|█████████████████▊         | 132/200 [00:22<00:11,  6.09epoch/s, loss=3.59, prev_loss=3.68]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  66%|█████████████████▉         | 133/200 [00:22<00:11,  6.08epoch/s, loss=3.64, prev_loss=3.59]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  67%|██████████████████         | 134/200 [00:22<00:10,  6.01epoch/s, loss=3.76, prev_loss=3.64]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  68%|██████████████████▏        | 135/200 [00:22<00:10,  6.17epoch/s, loss=3.59, prev_loss=3.76]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  68%|██████████████████▎        | 136/200 [00:23<00:10,  6.20epoch/s, loss=3.71, prev_loss=3.59]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  68%|██████████████████▍        | 137/200 [00:23<00:10,  6.07epoch/s, loss=3.63, prev_loss=3.71]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  69%|██████████████████▋        | 138/200 [00:23<00:10,  6.19epoch/s, loss=3.56, prev_loss=3.63]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  70%|██████████████████▊        | 139/200 [00:23<00:09,  6.28epoch/s, loss=3.73, prev_loss=3.56]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  70%|██████████████████▉        | 140/200 [00:23<00:09,  6.16epoch/s, loss=3.58, prev_loss=3.73]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  70%|███████████████████        | 141/200 [00:23<00:09,  6.35epoch/s, loss=3.57, prev_loss=3.58]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  71%|███████████████████▏       | 142/200 [00:24<00:09,  6.34epoch/s, loss=3.57, prev_loss=3.57]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  72%|███████████████████▎       | 143/200 [00:24<00:09,  6.18epoch/s, loss=3.66, prev_loss=3.57]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 114.45batch/s]\u001b[A\n",
      "Training epochs on cpu:  72%|███████████████████▍       | 144/200 [00:24<00:09,  5.96epoch/s, loss=3.55, prev_loss=3.66]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  72%|███████████████████▌       | 145/200 [00:24<00:09,  6.04epoch/s, loss=3.59, prev_loss=3.55]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  73%|███████████████████▋       | 146/200 [00:24<00:08,  6.23epoch/s, loss=3.61, prev_loss=3.59]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 115.96batch/s]\u001b[A\n",
      "Training epochs on cpu:  74%|███████████████████▊       | 147/200 [00:24<00:08,  5.98epoch/s, loss=3.48, prev_loss=3.61]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  74%|███████████████████▉       | 148/200 [00:25<00:08,  6.07epoch/s, loss=3.49, prev_loss=3.48]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  74%|████████████████████       | 149/200 [00:25<00:08,  6.12epoch/s, loss=3.61, prev_loss=3.49]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  75%|█████████████████████       | 150/200 [00:25<00:08,  6.14epoch/s, loss=3.6, prev_loss=3.61]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  76%|█████████████████████▏      | 151/200 [00:25<00:07,  6.26epoch/s, loss=3.55, prev_loss=3.6]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  76%|████████████████████▌      | 152/200 [00:25<00:07,  6.35epoch/s, loss=3.41, prev_loss=3.55]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|█████████████████████████████████████████████▊         | 10/12 [00:00<00:00, 94.94batch/s]\u001b[A\n",
      "Training epochs on cpu:  76%|████████████████████▋      | 153/200 [00:25<00:08,  5.84epoch/s, loss=3.48, prev_loss=3.41]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 118.11batch/s]\u001b[A\n",
      "Training epochs on cpu:  77%|████████████████████▊      | 154/200 [00:26<00:08,  5.73epoch/s, loss=3.42, prev_loss=3.48]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  78%|████████████████████▉      | 155/200 [00:26<00:07,  5.85epoch/s, loss=3.43, prev_loss=3.42]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  78%|█████████████████████      | 156/200 [00:26<00:07,  5.97epoch/s, loss=3.23, prev_loss=3.43]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  78%|█████████████████████▏     | 157/200 [00:26<00:07,  6.03epoch/s, loss=3.43, prev_loss=3.23]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  92%|█████████████████████████████████████████████████▌    | 11/12 [00:00<00:00, 103.36batch/s]\u001b[A\n",
      "Training epochs on cpu:  79%|█████████████████████▎     | 158/200 [00:26<00:07,  5.66epoch/s, loss=3.57, prev_loss=3.43]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  80%|██████████████████████▎     | 159/200 [00:26<00:07,  5.84epoch/s, loss=3.5, prev_loss=3.57]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 118.16batch/s]\u001b[A\n",
      "Training epochs on cpu:  80%|██████████████████████▍     | 160/200 [00:27<00:06,  5.75epoch/s, loss=3.43, prev_loss=3.5]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  80%|█████████████████████▋     | 161/200 [00:27<00:06,  5.82epoch/s, loss=3.29, prev_loss=3.43]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  81%|█████████████████████▊     | 162/200 [00:27<00:06,  6.08epoch/s, loss=3.31, prev_loss=3.29]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  82%|██████████████████████▊     | 163/200 [00:27<00:06,  5.95epoch/s, loss=3.6, prev_loss=3.31]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  82%|██████████████████████▉     | 164/200 [00:27<00:05,  6.08epoch/s, loss=3.25, prev_loss=3.6]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  82%|███████████████████████     | 165/200 [00:27<00:05,  6.16epoch/s, loss=3.3, prev_loss=3.25]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 115.67batch/s]\u001b[A\n",
      "Training epochs on cpu:  83%|███████████████████████▏    | 166/200 [00:28<00:05,  5.94epoch/s, loss=3.45, prev_loss=3.3]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 118.71batch/s]\u001b[A\n",
      "Training epochs on cpu:  84%|██████████████████████▌    | 167/200 [00:28<00:05,  5.80epoch/s, loss=3.22, prev_loss=3.45]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  84%|███████████████████████▌    | 168/200 [00:28<00:05,  5.98epoch/s, loss=3.3, prev_loss=3.22]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  84%|███████████████████████▋    | 169/200 [00:28<00:05,  5.89epoch/s, loss=3.36, prev_loss=3.3]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  85%|██████████████████████▉    | 170/200 [00:28<00:05,  5.97epoch/s, loss=3.25, prev_loss=3.36]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 114.35batch/s]\u001b[A\n",
      "Training epochs on cpu:  86%|███████████████████████    | 171/200 [00:28<00:05,  5.75epoch/s, loss=3.15, prev_loss=3.25]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  92%|█████████████████████████████████████████████████▌    | 11/12 [00:00<00:00, 106.91batch/s]\u001b[A\n",
      "Training epochs on cpu:  86%|███████████████████████▏   | 172/200 [00:29<00:05,  5.59epoch/s, loss=3.16, prev_loss=3.15]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  86%|████████████████████████▏   | 173/200 [00:29<00:04,  5.80epoch/s, loss=3.1, prev_loss=3.16]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  87%|████████████████████████▎   | 174/200 [00:29<00:04,  5.96epoch/s, loss=3.12, prev_loss=3.1]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  88%|███████████████████████▋   | 175/200 [00:29<00:04,  6.07epoch/s, loss=3.34, prev_loss=3.12]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  88%|███████████████████████▊   | 176/200 [00:29<00:03,  6.17epoch/s, loss=3.27, prev_loss=3.34]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 118.50batch/s]\u001b[A\n",
      "Training epochs on cpu:  88%|███████████████████████▉   | 177/200 [00:29<00:03,  5.96epoch/s, loss=3.23, prev_loss=3.27]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  89%|████████████████████████   | 178/200 [00:30<00:03,  6.05epoch/s, loss=3.19, prev_loss=3.23]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  90%|████████████████████████▏  | 179/200 [00:30<00:03,  6.20epoch/s, loss=3.08, prev_loss=3.19]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  90%|█████████████████████████▏  | 180/200 [00:30<00:03,  6.09epoch/s, loss=3.2, prev_loss=3.08]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  90%|█████████████████████████▎  | 181/200 [00:30<00:03,  6.08epoch/s, loss=3.09, prev_loss=3.2]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  91%|████████████████████████▌  | 182/200 [00:30<00:02,  6.23epoch/s, loss=3.05, prev_loss=3.09]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  92%|█████████████████████████████████████████████████▌    | 11/12 [00:00<00:00, 102.92batch/s]\u001b[A\n",
      "Training epochs on cpu:  92%|████████████████████████▋  | 183/200 [00:30<00:02,  5.84epoch/s, loss=3.04, prev_loss=3.05]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|█████████████████████████████████████████████▊         | 10/12 [00:00<00:00, 99.11batch/s]\u001b[A\n",
      "Training epochs on cpu:  92%|████████████████████████▊  | 184/200 [00:31<00:02,  5.57epoch/s, loss=3.01, prev_loss=3.04]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  92%|████████████████████████▉  | 185/200 [00:31<00:02,  5.75epoch/s, loss=2.84, prev_loss=3.01]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 119.22batch/s]\u001b[A\n",
      "Training epochs on cpu:  93%|█████████████████████████  | 186/200 [00:31<00:02,  5.60epoch/s, loss=3.02, prev_loss=2.84]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  94%|█████████████████████████▏ | 187/200 [00:31<00:02,  5.80epoch/s, loss=3.07, prev_loss=3.02]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  94%|█████████████████████████▍ | 188/200 [00:31<00:02,  5.76epoch/s, loss=3.12, prev_loss=3.07]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  94%|█████████████████████████▌ | 189/200 [00:32<00:01,  5.95epoch/s, loss=3.03, prev_loss=3.12]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  95%|█████████████████████████▋ | 190/200 [00:32<00:01,  5.89epoch/s, loss=3.06, prev_loss=3.03]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  96%|█████████████████████████▊ | 191/200 [00:32<00:01,  6.08epoch/s, loss=2.87, prev_loss=3.06]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  96%|█████████████████████████▉ | 192/200 [00:32<00:01,  6.15epoch/s, loss=2.95, prev_loss=2.87]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  96%|██████████████████████████ | 193/200 [00:32<00:01,  6.16epoch/s, loss=3.11, prev_loss=2.95]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 115.48batch/s]\u001b[A\n",
      "Training epochs on cpu:  97%|██████████████████████████▏| 194/200 [00:32<00:01,  5.94epoch/s, loss=2.84, prev_loss=3.11]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  98%|██████████████████████████▎| 195/200 [00:33<00:00,  5.96epoch/s, loss=3.02, prev_loss=2.84]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  92%|█████████████████████████████████████████████████▌    | 11/12 [00:00<00:00, 105.93batch/s]\u001b[A\n",
      "Training epochs on cpu:  98%|██████████████████████████▍| 196/200 [00:33<00:00,  5.69epoch/s, loss=2.82, prev_loss=3.02]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  98%|███████████████████████████▌| 197/200 [00:33<00:00,  5.80epoch/s, loss=2.8, prev_loss=2.82]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu:  99%|███████████████████████████▋| 198/200 [00:33<00:00,  5.89epoch/s, loss=2.95, prev_loss=2.8]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training epochs on cpu: 100%|██████████████████████████▊| 199/200 [00:33<00:00,  6.09epoch/s, loss=2.83, prev_loss=2.95]\u001b[A\n",
      "Training batches on cpu:   0%|                                                                | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|██████████████████████████████████████████████████████| 12/12 [00:00<00:00, 117.35batch/s]\u001b[A\n",
      "Training epochs on cpu: 100%|███████████████████████████| 200/200 [00:33<00:00,  5.90epoch/s, loss=2.96, prev_loss=2.83]\u001b[A\n",
      "INFO:pykeen.evaluation.evaluator:Currently automatic memory optimization only supports GPUs, but you're using a CPU. Therefore, the batch_size will be set to the default value.\n",
      "WARNING:pykeen.utils:The filtered setting was enabled, but there were no `additional_filter_triples`\n",
      "given. This means you probably forgot to pass (at least) the training triples. Try:\n",
      "\n",
      "    additional_filter_triples=[dataset.training.mapped_triples]\n",
      "\n",
      "Or if you want to use the Bordes et al. (2013) approach to filtering, do:\n",
      "\n",
      "    additional_filter_triples=[\n",
      "        dataset.training.mapped_triples,\n",
      "        dataset.validation.mapped_triples,\n",
      "    ]\n",
      "\n",
      "INFO:pykeen.evaluation.evaluator:No evaluation batch_size provided. Setting batch_size to '32'.\n",
      "Evaluating on cpu: 100%|██████████████████████████████████████████████████████████| 159/159 [00:00<00:00, 2.03ktriple/s]\n",
      "INFO:pykeen.evaluation.evaluator:Evaluation took 0.08s seconds\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.002304213121533394\n"
     ]
    }
   ],
   "source": [
    "from pykeen.models import ComplEx\n",
    "model = ComplEx(triples_factory=got_training)\n",
    "\n",
    "from pykeen.optimizers import Adam\n",
    "optimizer = Adam(params=model.get_grad_params())\n",
    "\n",
    "# from pykeen.regularizers import LP\n",
    "# regularizer = LP(p=3,weight=1e-5)\n",
    "\n",
    "from pykeen.training import SLCWATrainingLoop\n",
    "training_loop = SLCWATrainingLoop(model=model,\n",
    "                                  triples_factory=got_training,\n",
    "                                  optimizer=optimizer)\n",
    "\n",
    "#training\n",
    "_ = training_loop.train(triples_factory=got_training,\n",
    "                    num_epochs=200)\n",
    "\n",
    "#evaluating\n",
    "from pykeen.evaluation import RankBasedEvaluator\n",
    "evaluator = RankBasedEvaluator()\n",
    "mapped_triples = got_testing.mapped_triples\n",
    "\n",
    "results = evaluator.evaluate(\n",
    "            model=model,\n",
    "            mapped_triples=mapped_triples,\n",
    "            )\n",
    "\n",
    "print(results.get_metric('mrr'))\n",
    "\n",
    "#save results, this works also with the pipeline results, as the results object \n",
    "#returned by the evaluator is the same as the one returned from the pipeline\n",
    "save_dir = 'got_complex'\n",
    "if not os.path.isdir(save_dir):\n",
    "    os.mkdir(save_dir)\n",
    "results.to_df().to_csv(save_dir+os.path.sep+'results.csv')\n",
    "\n",
    "import torch\n",
    "torch.save(model,'trained_model.pkl')\n",
    "\n",
    "#to load the model use the following command\n",
    "# my_pykeen_model = torch.load('trained_model.pkl')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 3\n",
    "\n",
    "Try changing the parameters of your training process. See if you obtain a better model in terms of average loss. Save it as ./data/best_model.pkl. Which parameters work best for the dataset? \n",
    "\n",
    "Now use the training and test set you created in Exercise 2. Which loss you obtain, and for which parameters? \n",
    "\n",
    "Remember to save each model locally with a different name, so you can find them back."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "### your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. Evaluating the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now get some evaluation metrics for our model, they were already computed during evaluation time as part of the pipeline, and print them out.\n",
    "\n",
    "We are going to use the following evaluation metrics:\n",
    "- <i>mrr</i> (mean reciprocal rank) : this function computes the mean of the reciprocal of elements of a vector of rankings ranks\n",
    "- <i>hits_at_n</i> : this function computes how many elements of a vector of rankings ranks make it to the $top_n$ positions.\n",
    "\n",
    "NB : The choice of which _N_ makes more sense depends on the application and the size of the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.012578616352201259"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pipeline_result.get_metric('hits_at_10')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MRR: 0.0243\n",
      "\n",
      "Hits@10: 0.012579\n",
      "Interpretation: on average, the model guessed the correct subject or object 1.3% of the time when considering the top-10 better ranked triples.\n",
      "\n",
      "Hits@3: 0.000000\n",
      "Interpretation: on average, the model guessed the correct subject or object 0.0% of the time when considering the top-3 better ranked triples.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# from ampligraph.evaluation import mr_score, mrr_score, hits_at_n_score\n",
    "\n",
    "mrr = pipeline_result.get_metric('mrr')\n",
    "print(\"MRR: %.4f\" % (mrr))\n",
    "print()\n",
    "\n",
    "hits_10 = pipeline_result.get_metric('hits_at_10')\n",
    "print(\"Hits@10: %.6f\" % (hits_10))\n",
    "print(\"Interpretation: on average, the model guessed the correct subject or object %.1f%% of the time when considering the top-10 better ranked triples.\\n\" % (hits_10*100))\n",
    "\n",
    "hits_3 = pipeline_result.get_metric('hits_at_3')\n",
    "print(\"Hits@3: %.6f\" % (hits_3))\n",
    "print(\"Interpretation: on average, the model guessed the correct subject or object %.1f%% of the time when considering the top-3 better ranked triples.\\n\" % (hits_3*100))\n",
    "\n",
    "# hits_1 = hits_at_n_score(ranks, n=1)\n",
    "# print(\"Hits@1: %.2f\" % (hits_1))\n",
    "# print(\"Interpretation: on average, the model guessed the correct subject or object %.1f%% of the time when considering the top-1 better ranked triples.\\n\" % (hits_1*100))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 4\n",
    "\n",
    "Evaluate the models you created before (different set sizes, different parameters). Summarise your results in a table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "### your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. Link Prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Link prediction allows to infer missing links in a graph. This has many real-world use cases, such as predicting connections between people in a social network, interactions between proteins in a biological network, and music recommendation based on prior user taste.\n",
    "\n",
    "In our case, we are going to see which of the following candidate statements is more likely to be true. Note that the candidate statements below are made up, i.e. they are not in the original dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_unseen = np.array([\n",
    "    ['Jorah Mormont', 'SPOUSE', 'Daenerys Targaryen'],\n",
    "    ['Tyrion Lannister', 'SPOUSE', 'Missandei'],\n",
    "    [\"King's Landing\", 'SEAT_OF', 'House Lannister of Casterly Rock'],\n",
    "    ['Sansa Stark', 'SPOUSE', 'Petyr Baelish'],\n",
    "    ['Daenerys Targaryen', 'SPOUSE', 'Jon Snow'],\n",
    "    ['Daenerys Targaryen', 'SPOUSE', 'Craster'],\n",
    "    ['House Stark of Winterfell', 'IN_REGION', 'The North'],\n",
    "    ['House Stark of Winterfell', 'IN_REGION', 'Dorne'],\n",
    "    ['House Tyrell of Highgarden', 'IN_REGION', 'Beyond the Wall'],\n",
    "    ['Brandon Stark', 'ALLIED_WITH', 'House Stark of Winterfell'],\n",
    "    ['Brandon Stark', 'ALLIED_WITH', 'House Lannister of Casterly Rock'],    \n",
    "    ['Rhaegar Targaryen', 'PARENT_OF', 'Jon Snow'],\n",
    "    ['House Hutcheson', 'SWORN_TO', 'House Tyrell of Highgarden'],\n",
    "    ['Daenerys Targaryen', 'ALLIED_WITH', 'House Stark of Winterfell'],\n",
    "    ['Daenerys Targaryen', 'ALLIED_WITH', 'House Lannister of Casterly Rock'],\n",
    "    ['Jaime Lannister', 'PARENT_OF', 'Myrcella Baratheon'],\n",
    "    ['Robert I Baratheon', 'PARENT_OF', 'Myrcella Baratheon'],\n",
    "    ['Cersei Lannister', 'PARENT_OF', 'Myrcella Baratheon'],\n",
    "    ['Cersei Lannister', 'PARENT_OF', 'Brandon Stark'],\n",
    "    [\"Tywin Lannister\", 'PARENT_OF', 'Jaime Lannister'],\n",
    "    [\"Missandei\", 'SPOUSE', 'Grey Worm'],\n",
    "    [\"Brienne of Tarth\", 'SPOUSE', 'Jaime Lannister']\n",
    "])\n",
    "\n",
    "## we need to map the above triples to the id's which we used in our training/testing.\n",
    "## This information is stored in the triple factory \"got\", which we created at the beginning\n",
    "\n",
    "# unseen_filter = np.array(list({tuple(i) for i in np.vstack((positives_filter, X_unseen))}))\n",
    "#     filter_triples=unseen_filter,   # Corruption strategy filter defined above \n",
    "#     corrupt_side = 's+o',\n",
    "#     use_default_protocol=False, # corrupt subj and obj separately while evaluating\n",
    "#     verbose=True\n",
    "# )\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:torch_max_mem.api:Encountered tensors on device_types={'cpu'} while only ['cuda'] are considered safe for automatic memory utilization maximization. This may lead to undocumented crashes (but can be safe, too).\n"
     ]
    }
   ],
   "source": [
    "from pykeen import predict\n",
    "# from pykeen.predict import predict_triples\n",
    "\n",
    "# got_unseen = triples.get_mapped_tripples(X_unseen,factory=got)\n",
    "pack = predict.predict_triples(model=pipeline_result.model, triples=X_unseen, triples_factory=got)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>statement</th>\n",
       "      <th>score</th>\n",
       "      <th>prob</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Robert I Baratheon PARENT_OF Myrcella Baratheon</td>\n",
       "      <td>-64.575478</td>\n",
       "      <td>9.020413e-29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Daenerys Targaryen SPOUSE Jon Snow</td>\n",
       "      <td>-46.604897</td>\n",
       "      <td>5.751096e-21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Daenerys Targaryen ALLIED_WITH House Lannister...</td>\n",
       "      <td>-45.276077</td>\n",
       "      <td>2.171948e-20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Daenerys Targaryen ALLIED_WITH House Stark of ...</td>\n",
       "      <td>-31.533962</td>\n",
       "      <td>2.018246e-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Cersei Lannister PARENT_OF Brandon Stark</td>\n",
       "      <td>-30.185539</td>\n",
       "      <td>7.772968e-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>House Tyrell of Highgarden IN_REGION Beyond th...</td>\n",
       "      <td>-29.823235</td>\n",
       "      <td>1.116693e-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Rhaegar Targaryen PARENT_OF Jon Snow</td>\n",
       "      <td>-27.808620</td>\n",
       "      <td>8.372778e-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Brandon Stark ALLIED_WITH House Lannister of C...</td>\n",
       "      <td>-11.176892</td>\n",
       "      <td>1.399366e-05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>House Stark of Winterfell IN_REGION Dorne</td>\n",
       "      <td>-9.995385</td>\n",
       "      <td>4.560785e-05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>House Stark of Winterfell IN_REGION The North</td>\n",
       "      <td>-5.470758</td>\n",
       "      <td>4.190406e-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Daenerys Targaryen SPOUSE Craster</td>\n",
       "      <td>-5.079366</td>\n",
       "      <td>6.185358e-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>King's Landing SEAT_OF House Lannister of Cast...</td>\n",
       "      <td>-2.306306</td>\n",
       "      <td>9.060206e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Tywin Lannister PARENT_OF Jaime Lannister</td>\n",
       "      <td>1.026075</td>\n",
       "      <td>7.361543e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Jaime Lannister PARENT_OF Myrcella Baratheon</td>\n",
       "      <td>2.340347</td>\n",
       "      <td>9.121639e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Cersei Lannister PARENT_OF Myrcella Baratheon</td>\n",
       "      <td>10.521611</td>\n",
       "      <td>9.999731e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Jorah Mormont SPOUSE Daenerys Targaryen</td>\n",
       "      <td>12.006670</td>\n",
       "      <td>9.999939e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Brienne of Tarth SPOUSE Jaime Lannister</td>\n",
       "      <td>19.622749</td>\n",
       "      <td>1.000000e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Missandei SPOUSE Grey Worm</td>\n",
       "      <td>20.836506</td>\n",
       "      <td>1.000000e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Sansa Stark SPOUSE Petyr Baelish</td>\n",
       "      <td>27.986877</td>\n",
       "      <td>1.000000e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>House Hutcheson SWORN_TO House Tyrell of Highg...</td>\n",
       "      <td>30.699150</td>\n",
       "      <td>1.000000e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Tyrion Lannister SPOUSE Missandei</td>\n",
       "      <td>34.414974</td>\n",
       "      <td>1.000000e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Brandon Stark ALLIED_WITH House Stark of Winte...</td>\n",
       "      <td>57.917721</td>\n",
       "      <td>1.000000e+00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            statement      score          prob\n",
       "16    Robert I Baratheon PARENT_OF Myrcella Baratheon -64.575478  9.020413e-29\n",
       "4                  Daenerys Targaryen SPOUSE Jon Snow -46.604897  5.751096e-21\n",
       "14  Daenerys Targaryen ALLIED_WITH House Lannister... -45.276077  2.171948e-20\n",
       "13  Daenerys Targaryen ALLIED_WITH House Stark of ... -31.533962  2.018246e-14\n",
       "18           Cersei Lannister PARENT_OF Brandon Stark -30.185539  7.772968e-14\n",
       "8   House Tyrell of Highgarden IN_REGION Beyond th... -29.823235  1.116693e-13\n",
       "11               Rhaegar Targaryen PARENT_OF Jon Snow -27.808620  8.372778e-13\n",
       "10  Brandon Stark ALLIED_WITH House Lannister of C... -11.176892  1.399366e-05\n",
       "7           House Stark of Winterfell IN_REGION Dorne  -9.995385  4.560785e-05\n",
       "6       House Stark of Winterfell IN_REGION The North  -5.470758  4.190406e-03\n",
       "5                   Daenerys Targaryen SPOUSE Craster  -5.079366  6.185358e-03\n",
       "2   King's Landing SEAT_OF House Lannister of Cast...  -2.306306  9.060206e-02\n",
       "19          Tywin Lannister PARENT_OF Jaime Lannister   1.026075  7.361543e-01\n",
       "15       Jaime Lannister PARENT_OF Myrcella Baratheon   2.340347  9.121639e-01\n",
       "17      Cersei Lannister PARENT_OF Myrcella Baratheon  10.521611  9.999731e-01\n",
       "0             Jorah Mormont SPOUSE Daenerys Targaryen  12.006670  9.999939e-01\n",
       "21            Brienne of Tarth SPOUSE Jaime Lannister  19.622749  1.000000e+00\n",
       "20                         Missandei SPOUSE Grey Worm  20.836506  1.000000e+00\n",
       "3                    Sansa Stark SPOUSE Petyr Baelish  27.986877  1.000000e+00\n",
       "12  House Hutcheson SWORN_TO House Tyrell of Highg...  30.699150  1.000000e+00\n",
       "1                   Tyrion Lannister SPOUSE Missandei  34.414974  1.000000e+00\n",
       "9   Brandon Stark ALLIED_WITH House Stark of Winte...  57.917721  1.000000e+00"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# scores are real numbers that need to be translated into probabilities [0,1] \n",
    "# for this, we use the expit transform.\n",
    "\n",
    "from scipy.special import expit\n",
    "processed_results = pack.process().df\n",
    "# print(processed_results)\n",
    "\n",
    "probs = expit(processed_results['score'])\n",
    "# print(probs)\n",
    "\n",
    "processed_results['prob'] = probs\n",
    "processed_results['triple'] = list(zip([' '.join(x) for x in X_unseen]))\n",
    "\n",
    "# processed_results\n",
    "pd.DataFrame(list(zip([' '.join(x) for x in X_unseen],  \n",
    "                      np.squeeze(processed_results['score']),\n",
    "                      np.squeeze(probs))), \n",
    "             columns=['statement', 'score', 'prob']).sort_values(\"score\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NB : the probabilities are not calibrated in any sense. To calibrate them, one may use a procedure such as [Platt scaling](https://en.wikipedia.org/wiki/Platt_scaling) or [Isotonic regression](https://en.wikipedia.org/wiki/Isotonic_regression). The challenge is to define what is a true triple and what is a false one, as the calibration of the probability of a triple being true depends on the base rate of positives and negatives."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 5\n",
    "\n",
    "Analyse the results in the tables. Some predicted links are very likely to be true, others  capture things that never really happened. Can you spot which ones?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 6 Visualisation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Tensorboard](https://www.tensorflow.org/tensorboard) allows to dig into the workings of our model, plot how it is learning, and visualize [high-dimensional embeddings](https://projector.tensorflow.org/). See [this tutorial](https://www.tensorflow.org/tensorboard/get_started) to get started with Tensorflow and see [here](https://pykeen.readthedocs.io/en/stable/tutorial/trackers/using_tensorboard.html) for Tensorboard with PyKEEN.\n",
    "\n",
    "First ytou neeed to start the tensorboard web application from the command line with \n",
    "\n",
    "$ tensorboard --logdir=~/.data/pykeen/logs/tensorboard/\n",
    "\n",
    "and then we can add tensorboard as the result_tracker in our pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  67%|████████████████████████████████▋                | 8/12 [00:00<00:00, 74.01batch/s]\u001b[A\n",
      "Training epochs on cpu:  45%|█████████▍           | 90/200 [00:30<00:35,  3.07epoch/s, loss=1.47, prev_loss=1.47]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 64.25batch/s]\u001b[A\n",
      "Training epochs on cpu:  46%|█████████▌           | 91/200 [00:30<00:35,  3.06epoch/s, loss=1.67, prev_loss=1.47]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 63.01batch/s]\u001b[A\n",
      "Training epochs on cpu:  46%|█████████▋           | 92/200 [00:31<00:36,  2.96epoch/s, loss=1.33, prev_loss=1.67]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 62.24batch/s]\u001b[A\n",
      "Training epochs on cpu:  46%|█████████▊           | 93/200 [00:31<00:37,  2.89epoch/s, loss=1.34, prev_loss=1.33]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 61.56batch/s]\u001b[A\n",
      "Training epochs on cpu:  47%|█████████▊           | 94/200 [00:31<00:37,  2.82epoch/s, loss=1.18, prev_loss=1.34]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 64.46batch/s]\u001b[A\n",
      "Training epochs on cpu:  48%|█████████▉           | 95/200 [00:32<00:37,  2.80epoch/s, loss=1.09, prev_loss=1.18]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 61.86batch/s]\u001b[A\n",
      "Training epochs on cpu:  48%|██████████           | 96/200 [00:32<00:37,  2.77epoch/s, loss=1.14, prev_loss=1.09]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 68.47batch/s]\u001b[A\n",
      "Training epochs on cpu:  48%|██████████▋           | 97/200 [00:33<00:35,  2.86epoch/s, loss=1.3, prev_loss=1.14]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 62.27batch/s]\u001b[A\n",
      "Training epochs on cpu:  49%|██████████▎          | 98/200 [00:33<00:36,  2.80epoch/s, loss=0.958, prev_loss=1.3]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 59.10batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 58.59batch/s]\u001b[A\n",
      "Training epochs on cpu:  50%|█████████▉          | 99/200 [00:33<00:36,  2.78epoch/s, loss=1.17, prev_loss=0.958]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 62.70batch/s]\u001b[A\n",
      "Training epochs on cpu:  50%|██████████          | 100/200 [00:34<00:36,  2.73epoch/s, loss=1.13, prev_loss=1.17]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 59.45batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 55.80batch/s]\u001b[A\n",
      "Training epochs on cpu:  50%|██████████          | 101/200 [00:34<00:37,  2.67epoch/s, loss=1.18, prev_loss=1.13]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 61.85batch/s]\u001b[A\n",
      "Training epochs on cpu:  51%|██████████▏         | 102/200 [00:34<00:36,  2.67epoch/s, loss=1.09, prev_loss=1.18]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 61.62batch/s]\u001b[A\n",
      "Training epochs on cpu:  52%|██████████▎         | 103/200 [00:35<00:36,  2.68epoch/s, loss=1.04, prev_loss=1.09]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 57.76batch/s]\u001b[A\n",
      "Training epochs on cpu:  52%|█████████▉         | 104/200 [00:35<00:36,  2.65epoch/s, loss=0.987, prev_loss=1.04]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 68.73batch/s]\u001b[A\n",
      "Training epochs on cpu:  52%|█████████▉         | 105/200 [00:36<00:35,  2.70epoch/s, loss=1.09, prev_loss=0.987]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 60.13batch/s]\u001b[A\n",
      "Training epochs on cpu:  53%|██████████         | 106/200 [00:36<00:34,  2.70epoch/s, loss=0.955, prev_loss=1.09]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 60.69batch/s]\u001b[A\n",
      "Training epochs on cpu:  54%|██████████▏        | 107/200 [00:36<00:34,  2.68epoch/s, loss=1.01, prev_loss=0.955]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 53.86batch/s]\u001b[A\n",
      "Training epochs on cpu:  54%|██████████▊         | 108/200 [00:37<00:34,  2.65epoch/s, loss=1.12, prev_loss=1.01]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 54.01batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 54.52batch/s]\u001b[A\n",
      "Training epochs on cpu:  55%|██████████▉         | 109/200 [00:37<00:35,  2.60epoch/s, loss=1.14, prev_loss=1.12]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 60.06batch/s]\u001b[A\n",
      "Training epochs on cpu:  55%|███████████▌         | 110/200 [00:37<00:34,  2.58epoch/s, loss=0.9, prev_loss=1.14]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 58.75batch/s]\u001b[A\n",
      "Training epochs on cpu:  56%|███████████▋         | 111/200 [00:38<00:33,  2.67epoch/s, loss=1.11, prev_loss=0.9]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 65.35batch/s]\u001b[A\n",
      "Training epochs on cpu:  56%|██████████▋        | 112/200 [00:38<00:32,  2.73epoch/s, loss=0.971, prev_loss=1.11]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 65.23batch/s]\u001b[A\n",
      "Training epochs on cpu:  56%|██████████▏       | 113/200 [00:38<00:31,  2.77epoch/s, loss=0.922, prev_loss=0.971]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 65.31batch/s]\u001b[A\n",
      "Training epochs on cpu:  57%|██████████▎       | 114/200 [00:39<00:30,  2.86epoch/s, loss=0.926, prev_loss=0.922]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 64.44batch/s]\u001b[A\n",
      "Training epochs on cpu:  57%|██████████▎       | 115/200 [00:39<00:29,  2.84epoch/s, loss=0.965, prev_loss=0.926]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 62.68batch/s]\u001b[A\n",
      "Training epochs on cpu:  58%|██████████▍       | 116/200 [00:40<00:29,  2.83epoch/s, loss=0.999, prev_loss=0.965]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 59.91batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 58.63batch/s]\u001b[A\n",
      "Training epochs on cpu:  58%|██████████▌       | 117/200 [00:40<00:29,  2.77epoch/s, loss=0.824, prev_loss=0.999]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 60.99batch/s]\u001b[A\n",
      "Training epochs on cpu:  59%|███████████▏       | 118/200 [00:40<00:29,  2.78epoch/s, loss=0.71, prev_loss=0.824]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 59.93batch/s]\u001b[A\n",
      "Training epochs on cpu:  60%|███████████▎       | 119/200 [00:41<00:29,  2.75epoch/s, loss=0.921, prev_loss=0.71]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 59.81batch/s]\u001b[A\n",
      "Training epochs on cpu:  60%|██████████▊       | 120/200 [00:41<00:29,  2.74epoch/s, loss=0.733, prev_loss=0.921]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 60.82batch/s]\u001b[A\n",
      "Training epochs on cpu:  60%|██████████▉       | 121/200 [00:41<00:28,  2.79epoch/s, loss=0.815, prev_loss=0.733]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 57.26batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 57.89batch/s]\u001b[A\n",
      "Training epochs on cpu:  61%|██████████▉       | 122/200 [00:42<00:28,  2.73epoch/s, loss=0.793, prev_loss=0.815]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 56.93batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 56.74batch/s]\u001b[A\n",
      "Training epochs on cpu:  62%|███████████▋       | 123/200 [00:42<00:28,  2.69epoch/s, loss=0.65, prev_loss=0.793]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 56.06batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 57.69batch/s]\u001b[A\n",
      "Training epochs on cpu:  62%|███████████▊       | 124/200 [00:42<00:28,  2.65epoch/s, loss=0.761, prev_loss=0.65]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 61.39batch/s]\u001b[A\n",
      "Training epochs on cpu:  62%|███████████▎      | 125/200 [00:43<00:28,  2.65epoch/s, loss=0.805, prev_loss=0.761]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 60.63batch/s]\u001b[A\n",
      "Training epochs on cpu:  63%|███████████▎      | 126/200 [00:43<00:27,  2.66epoch/s, loss=0.707, prev_loss=0.805]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 59.41batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 59.75batch/s]\u001b[A\n",
      "Training epochs on cpu:  64%|███████████▍      | 127/200 [00:44<00:27,  2.65epoch/s, loss=0.718, prev_loss=0.707]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 57.51batch/s]\u001b[A\n",
      "Training epochs on cpu:  64%|███████████▌      | 128/200 [00:44<00:26,  2.73epoch/s, loss=0.721, prev_loss=0.718]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 61.26batch/s]\u001b[A\n",
      "Training epochs on cpu:  64%|███████████▌      | 129/200 [00:44<00:25,  2.74epoch/s, loss=0.684, prev_loss=0.721]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  67%|████████████████████████████████▋                | 8/12 [00:00<00:00, 76.17batch/s]\u001b[A\n",
      "Training epochs on cpu:  65%|███████████▋      | 130/200 [00:45<00:24,  2.82epoch/s, loss=0.599, prev_loss=0.684]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 65.30batch/s]\u001b[A\n",
      "Training epochs on cpu:  66%|███████████▊      | 131/200 [00:45<00:24,  2.83epoch/s, loss=0.665, prev_loss=0.599]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 61.65batch/s]\u001b[A\n",
      "Training epochs on cpu:  66%|███████████▉      | 132/200 [00:45<00:24,  2.82epoch/s, loss=0.842, prev_loss=0.665]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 63.66batch/s]\u001b[A\n",
      "Training epochs on cpu:  66%|███████████▉      | 133/200 [00:46<00:23,  2.81epoch/s, loss=0.518, prev_loss=0.842]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 65.53batch/s]\u001b[A\n",
      "Training epochs on cpu:  67%|████████████      | 134/200 [00:46<00:22,  2.87epoch/s, loss=0.575, prev_loss=0.518]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 66.82batch/s]\u001b[A\n",
      "Training epochs on cpu:  68%|████████████▏     | 135/200 [00:46<00:22,  2.89epoch/s, loss=0.517, prev_loss=0.575]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 66.80batch/s]\u001b[A\n",
      "Training epochs on cpu:  68%|████████████▏     | 136/200 [00:47<00:22,  2.88epoch/s, loss=0.658, prev_loss=0.517]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 63.59batch/s]\u001b[A\n",
      "Training epochs on cpu:  68%|████████████▎     | 137/200 [00:47<00:21,  2.87epoch/s, loss=0.725, prev_loss=0.658]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 65.23batch/s]\u001b[A\n",
      "Training epochs on cpu:  69%|████████████▍     | 138/200 [00:47<00:21,  2.87epoch/s, loss=0.499, prev_loss=0.725]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 62.85batch/s]\u001b[A\n",
      "Training epochs on cpu:  70%|█████████████▏     | 139/200 [00:48<00:21,  2.84epoch/s, loss=0.67, prev_loss=0.499]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 62.20batch/s]\u001b[A\n",
      "Training epochs on cpu:  70%|█████████████▎     | 140/200 [00:48<00:21,  2.84epoch/s, loss=0.591, prev_loss=0.67]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 57.64batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 57.18batch/s]\u001b[A\n",
      "Training epochs on cpu:  70%|████████████▋     | 141/200 [00:49<00:21,  2.77epoch/s, loss=0.552, prev_loss=0.591]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 59.09batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 57.55batch/s]\u001b[A\n",
      "Training epochs on cpu:  71%|████████████▊     | 142/200 [00:49<00:20,  2.77epoch/s, loss=0.564, prev_loss=0.552]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 58.35batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 57.02batch/s]\u001b[A\n",
      "Training epochs on cpu:  72%|████████████▊     | 143/200 [00:49<00:20,  2.74epoch/s, loss=0.539, prev_loss=0.564]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 57.38batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 57.93batch/s]\u001b[A\n",
      "Training epochs on cpu:  72%|████████████▉     | 144/200 [00:50<00:20,  2.70epoch/s, loss=0.513, prev_loss=0.539]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 60.61batch/s]\u001b[A\n",
      "Training epochs on cpu:  72%|█████████████     | 145/200 [00:50<00:20,  2.69epoch/s, loss=0.558, prev_loss=0.513]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 54.71batch/s]\u001b[A\n",
      "Training epochs on cpu:  73%|█████████████▏    | 146/200 [00:50<00:20,  2.66epoch/s, loss=0.546, prev_loss=0.558]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 62.15batch/s]\u001b[A\n",
      "Training epochs on cpu:  74%|█████████████▏    | 147/200 [00:51<00:19,  2.66epoch/s, loss=0.598, prev_loss=0.546]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 57.72batch/s]\u001b[A\n",
      "Training epochs on cpu:  74%|█████████████▎    | 148/200 [00:51<00:19,  2.66epoch/s, loss=0.501, prev_loss=0.598]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 60.93batch/s]\u001b[A\n",
      "Training epochs on cpu:  74%|█████████████▍    | 149/200 [00:52<00:18,  2.73epoch/s, loss=0.456, prev_loss=0.501]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 67.13batch/s]\u001b[A\n",
      "Training epochs on cpu:  75%|█████████████▌    | 150/200 [00:52<00:17,  2.80epoch/s, loss=0.478, prev_loss=0.456]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 64.55batch/s]\u001b[A\n",
      "Training epochs on cpu:  76%|█████████████▌    | 151/200 [00:52<00:17,  2.85epoch/s, loss=0.454, prev_loss=0.478]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 57.78batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 54.81batch/s]\u001b[A\n",
      "Training epochs on cpu:  76%|██████████████▍    | 152/200 [00:53<00:17,  2.74epoch/s, loss=0.46, prev_loss=0.454]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 65.84batch/s]\u001b[A\n",
      "Training epochs on cpu:  76%|██████████████▌    | 153/200 [00:53<00:16,  2.85epoch/s, loss=0.553, prev_loss=0.46]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 69.21batch/s]\u001b[A\n",
      "Training epochs on cpu:  77%|█████████████▊    | 154/200 [00:53<00:15,  2.94epoch/s, loss=0.483, prev_loss=0.553]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  67%|████████████████████████████████▋                | 8/12 [00:00<00:00, 70.46batch/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training epochs on cpu:  78%|█████████████▉    | 155/200 [00:54<00:15,  2.94epoch/s, loss=0.462, prev_loss=0.483]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 62.83batch/s]\u001b[A\n",
      "Training epochs on cpu:  78%|██████████████    | 156/200 [00:54<00:15,  2.90epoch/s, loss=0.561, prev_loss=0.462]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 67.28batch/s]\u001b[A\n",
      "Training epochs on cpu:  78%|██████████████▏   | 157/200 [00:54<00:14,  2.90epoch/s, loss=0.507, prev_loss=0.561]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  67%|████████████████████████████████▋                | 8/12 [00:00<00:00, 74.73batch/s]\u001b[A\n",
      "Training epochs on cpu:  79%|██████████████▏   | 158/200 [00:55<00:14,  2.94epoch/s, loss=0.484, prev_loss=0.507]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 63.55batch/s]\u001b[A\n",
      "Training epochs on cpu:  80%|██████████████▎   | 159/200 [00:55<00:13,  2.95epoch/s, loss=0.401, prev_loss=0.484]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  67%|████████████████████████████████▋                | 8/12 [00:00<00:00, 73.10batch/s]\u001b[A\n",
      "Training epochs on cpu:  80%|██████████████▍   | 160/200 [00:55<00:13,  3.03epoch/s, loss=0.457, prev_loss=0.401]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  67%|████████████████████████████████▋                | 8/12 [00:00<00:00, 71.25batch/s]\u001b[A\n",
      "Training epochs on cpu:  80%|██████████████▍   | 161/200 [00:56<00:12,  3.07epoch/s, loss=0.356, prev_loss=0.457]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 62.75batch/s]\u001b[A\n",
      "Training epochs on cpu:  81%|██████████████▌   | 162/200 [00:56<00:12,  2.98epoch/s, loss=0.344, prev_loss=0.356]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 69.06batch/s]\u001b[A\n",
      "Training epochs on cpu:  82%|██████████████▋   | 163/200 [00:56<00:12,  2.97epoch/s, loss=0.448, prev_loss=0.344]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  67%|████████████████████████████████▋                | 8/12 [00:00<00:00, 72.50batch/s]\u001b[A\n",
      "Training epochs on cpu:  82%|██████████████▊   | 164/200 [00:57<00:12,  2.98epoch/s, loss=0.454, prev_loss=0.448]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 61.86batch/s]\u001b[A\n",
      "Training epochs on cpu:  82%|██████████████▊   | 165/200 [00:57<00:11,  2.98epoch/s, loss=0.283, prev_loss=0.454]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 60.30batch/s]\u001b[A\n",
      "Training epochs on cpu:  83%|██████████████▉   | 166/200 [00:57<00:11,  2.88epoch/s, loss=0.422, prev_loss=0.283]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 59.85batch/s]\u001b[A\n",
      "Training epochs on cpu:  84%|███████████████   | 167/200 [00:58<00:11,  2.86epoch/s, loss=0.377, prev_loss=0.422]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 57.29batch/s]\u001b[A\n",
      "Training epochs on cpu:  84%|███████████████   | 168/200 [00:58<00:11,  2.86epoch/s, loss=0.395, prev_loss=0.377]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 60.23batch/s]\u001b[A\n",
      "Training epochs on cpu:  84%|███████████████▏  | 169/200 [00:58<00:11,  2.80epoch/s, loss=0.448, prev_loss=0.395]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 62.37batch/s]\u001b[A\n",
      "Training epochs on cpu:  85%|████████████████▏  | 170/200 [00:59<00:10,  2.76epoch/s, loss=0.44, prev_loss=0.448]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 60.01batch/s]\u001b[A\n",
      "Training epochs on cpu:  86%|█████████████████   | 171/200 [00:59<00:10,  2.72epoch/s, loss=0.38, prev_loss=0.44]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 61.22batch/s]\u001b[A\n",
      "Training epochs on cpu:  86%|████████████████▎  | 172/200 [00:59<00:10,  2.75epoch/s, loss=0.449, prev_loss=0.38]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 59.27batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 57.19batch/s]\u001b[A\n",
      "Training epochs on cpu:  86%|███████████████▌  | 173/200 [01:00<00:09,  2.76epoch/s, loss=0.333, prev_loss=0.449]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 65.26batch/s]\u001b[A\n",
      "Training epochs on cpu:  87%|███████████████▋  | 174/200 [01:00<00:09,  2.75epoch/s, loss=0.419, prev_loss=0.333]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 55.74batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 48.32batch/s]\u001b[A\n",
      "Training epochs on cpu:  88%|███████████████▊  | 175/200 [01:01<00:09,  2.59epoch/s, loss=0.358, prev_loss=0.419]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 58.49batch/s]\u001b[A\n",
      "Training epochs on cpu:  88%|███████████████▊  | 176/200 [01:01<00:09,  2.63epoch/s, loss=0.395, prev_loss=0.358]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 61.22batch/s]\u001b[A\n",
      "Training epochs on cpu:  88%|███████████████▉  | 177/200 [01:01<00:08,  2.64epoch/s, loss=0.423, prev_loss=0.395]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 59.20batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 57.56batch/s]\u001b[A\n",
      "Training epochs on cpu:  89%|████████████████▉  | 178/200 [01:02<00:08,  2.63epoch/s, loss=0.41, prev_loss=0.423]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  42%|████████████████████▍                            | 5/12 [00:00<00:00, 48.27batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|████████████████████████████████████████        | 10/12 [00:00<00:00, 49.14batch/s]\u001b[A\n",
      "Training epochs on cpu:  90%|█████████████████  | 179/200 [01:02<00:08,  2.50epoch/s, loss=0.321, prev_loss=0.41]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 51.94batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 49.10batch/s]\u001b[A\n",
      "Training epochs on cpu:  90%|████████████████▏ | 180/200 [01:03<00:08,  2.40epoch/s, loss=0.276, prev_loss=0.321]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  42%|████████████████████▍                            | 5/12 [00:00<00:00, 49.90batch/s]\u001b[A\n",
      "Training batches on cpu:  92%|████████████████████████████████████████████    | 11/12 [00:00<00:00, 52.62batch/s]\u001b[A\n",
      "Training epochs on cpu:  90%|████████████████▎ | 181/200 [01:03<00:07,  2.42epoch/s, loss=0.391, prev_loss=0.276]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 57.37batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 56.04batch/s]\u001b[A\n",
      "Training epochs on cpu:  91%|████████████████▍ | 182/200 [01:03<00:07,  2.50epoch/s, loss=0.301, prev_loss=0.391]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 63.78batch/s]\u001b[A\n",
      "Training epochs on cpu:  92%|████████████████▍ | 183/200 [01:04<00:06,  2.57epoch/s, loss=0.304, prev_loss=0.301]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 52.40batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 50.94batch/s]\u001b[A\n",
      "Training epochs on cpu:  92%|████████████████▌ | 184/200 [01:04<00:06,  2.50epoch/s, loss=0.345, prev_loss=0.304]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 50.73batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 51.61batch/s]\u001b[A\n",
      "Training epochs on cpu:  92%|████████████████▋ | 185/200 [01:05<00:06,  2.45epoch/s, loss=0.373, prev_loss=0.345]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 56.57batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 55.78batch/s]\u001b[A\n",
      "Training epochs on cpu:  93%|████████████████▋ | 186/200 [01:05<00:05,  2.49epoch/s, loss=0.338, prev_loss=0.373]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 54.94batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 49.97batch/s]\u001b[A\n",
      "Training epochs on cpu:  94%|████████████████▊ | 187/200 [01:05<00:05,  2.42epoch/s, loss=0.288, prev_loss=0.338]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  42%|████████████████████▍                            | 5/12 [00:00<00:00, 46.70batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|████████████████████████████████████████        | 10/12 [00:00<00:00, 43.69batch/s]\u001b[A\n",
      "Training epochs on cpu:  94%|████████████████▉ | 188/200 [01:06<00:05,  2.32epoch/s, loss=0.358, prev_loss=0.288]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  25%|████████████▎                                    | 3/12 [00:00<00:00, 25.71batch/s]\u001b[A\n",
      "Training batches on cpu:  67%|████████████████████████████████▋                | 8/12 [00:00<00:00, 34.52batch/s]\u001b[A\n",
      "Training epochs on cpu:  94%|█████████████████▉ | 189/200 [01:06<00:04,  2.21epoch/s, loss=0.32, prev_loss=0.358]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 53.26batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 51.51batch/s]\u001b[A\n",
      "Training epochs on cpu:  95%|██████████████████ | 190/200 [01:07<00:04,  2.23epoch/s, loss=0.324, prev_loss=0.32]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  42%|████████████████████▍                            | 5/12 [00:00<00:00, 40.06batch/s]\u001b[A\n",
      "Training batches on cpu:  92%|████████████████████████████████████████████    | 11/12 [00:00<00:00, 46.45batch/s]\u001b[A\n",
      "Training epochs on cpu:  96%|█████████████████▏| 191/200 [01:07<00:04,  2.18epoch/s, loss=0.265, prev_loss=0.324]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  42%|████████████████████▍                            | 5/12 [00:00<00:00, 47.03batch/s]\u001b[A\n",
      "Training batches on cpu:  92%|████████████████████████████████████████████    | 11/12 [00:00<00:00, 49.32batch/s]\u001b[A\n",
      "Training epochs on cpu:  96%|█████████████████▎| 192/200 [01:08<00:03,  2.20epoch/s, loss=0.286, prev_loss=0.265]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 51.57batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 51.25batch/s]\u001b[A\n",
      "Training epochs on cpu:  96%|█████████████████▎| 193/200 [01:08<00:03,  2.23epoch/s, loss=0.245, prev_loss=0.286]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  33%|████████████████▎                                | 4/12 [00:00<00:00, 38.51batch/s]\u001b[A\n",
      "Training batches on cpu:  75%|████████████████████████████████████▊            | 9/12 [00:00<00:00, 43.02batch/s]\u001b[A\n",
      "Training epochs on cpu:  97%|█████████████████▍| 194/200 [01:09<00:02,  2.21epoch/s, loss=0.223, prev_loss=0.245]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 52.38batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 51.12batch/s]\u001b[A\n",
      "Training epochs on cpu:  98%|█████████████████▌| 195/200 [01:09<00:02,  2.26epoch/s, loss=0.342, prev_loss=0.223]\u001b[A\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 54.70batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 53.33batch/s]\u001b[A\n",
      "Training epochs on cpu:  98%|█████████████████▋| 196/200 [01:10<00:01,  2.31epoch/s, loss=0.199, prev_loss=0.342]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  42%|████████████████████▍                            | 5/12 [00:00<00:00, 48.84batch/s]\u001b[A\n",
      "Training batches on cpu:  83%|████████████████████████████████████████        | 10/12 [00:00<00:00, 49.46batch/s]\u001b[A\n",
      "Training epochs on cpu:  98%|█████████████████▋| 197/200 [01:10<00:01,  2.28epoch/s, loss=0.286, prev_loss=0.199]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 58.47batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 59.27batch/s]\u001b[A\n",
      "Training epochs on cpu:  99%|█████████████████▊| 198/200 [01:10<00:00,  2.38epoch/s, loss=0.235, prev_loss=0.286]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  58%|████████████████████████████▌                    | 7/12 [00:00<00:00, 60.20batch/s]\u001b[A\n",
      "Training epochs on cpu: 100%|█████████████████▉| 199/200 [01:11<00:00,  2.42epoch/s, loss=0.237, prev_loss=0.235]\u001b[A\n",
      "Training batches on cpu:   0%|                                                         | 0/12 [00:00<?, ?batch/s]\u001b[A\n",
      "Training batches on cpu:  50%|████████████████████████▌                        | 6/12 [00:00<00:00, 54.33batch/s]\u001b[A\n",
      "Training batches on cpu: 100%|████████████████████████████████████████████████| 12/12 [00:00<00:00, 55.96batch/s]\u001b[A\n",
      "Training epochs on cpu: 100%|██████████████████| 200/200 [01:11<00:00,  2.79epoch/s, loss=0.275, prev_loss=0.237]\u001b[A\n",
      "INFO:pykeen.evaluation.evaluator:Currently automatic memory optimization only supports GPUs, but you're using a CPU. Therefore, the batch_size will be set to the default value.\n",
      "INFO:pykeen.evaluation.evaluator:No evaluation batch_size provided. Setting batch_size to '32'.\n",
      "Evaluating on cpu: 100%|█████████████████████████████████████████████████████| 159/159 [00:00<00:00, 919triple/s]\n",
      "INFO:pykeen.evaluation.evaluator:Evaluation took 0.19s seconds\n"
     ]
    }
   ],
   "source": [
    "pipeline_result = pipeline(\n",
    "    model='ComplEx',\n",
    "    training=got_training,\n",
    "    testing=got_testing,\n",
    "    training_kwargs=dict(\n",
    "        num_epochs=200\n",
    "    ),\n",
    "    dimensions=150,\n",
    "    optimizer='adam',\n",
    "    optimizer_kwargs={'lr':1e-3},\n",
    "    loss='pairwisehinge', \n",
    "    regularizer='LP', \n",
    "    regularizer_kwargs={'p':3, 'weight':1e-5}, \n",
    "    negative_sampler='basic',\n",
    "    negative_sampler_kwargs=dict(\n",
    "        filtered=True,\n",
    "    ),\n",
    "    result_tracker='tensorboard'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 7 Your Own Data now\n",
    "\n",
    "Choose a dataset of your own. Best if it is the data you are using in your group project. \n",
    "\n",
    "- Create a training and testset. \n",
    "- Train your model to compute Knowledge Graph Embeddings, and save the best parameters model. - Predict new links over your dataset\n",
    "- Visualise the embeddings you computed \n",
    "- Optional : cluster your embeddings, [see this tutorial](https://docs.ampligraph.org/en/1.4.0/tutorials/ClusteringAndClassificationWithEmbeddings.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "c30f2af5f468e7f5b45bcc30fca5f4886c90d54777aed916ed5f6294dfb24bf2"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}