course2/week2/C2_W2_Assignment.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Train a review classifier with BERT and Amazon SageMaker"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Introduction\n",
    "\n",
    "In the previous lab you performed Feature Engineering on the raw dataset, preparing it for training the model. Now you will train a text classifier using a variant of BERT called [RoBERTa](https://arxiv.org/abs/1907.11692) - a Robustly Optimized BERT Pretraining Approach - within a PyTorch model ran as a SageMaker Training Job.\n",
    "\n",
    "### Table of Contents\n",
    "\n",
    "- [1. Configure dataset, hyper-parameters and evaluation metrics](#c2w2-1.)\n",
    "  - [1.1. Configure dataset](#c2w2-1.1.)\n",
    "    - [Exercise 1](#c2w2-ex-1)\n",
    "    - [Exercise 2](#c2w2-ex-2)\n",
    "    - [Exercise 3](#c2w2-ex-3)\n",
    "  - [1.2. Configure model hyper-parameters](#c2w2-1.2.)\n",
    "  - [1.3. Setup evaluation metrics](#c2w2-1.3.)\n",
    "  - [1.4. Setup Debugger and Profiler](#c2w2-1.4.)\n",
    "- [2. Train model](#c2w2-2.)\n",
    "  - [2.1. Setup the RoBERTa and PyTorch script to run on SageMaker](#c2w2-2.1.)\n",
    "    - [Exercise 4](#c2w2-ex-4)\n",
    "    - [Exercise 5](#c2w2-ex-5)\n",
    "    - [Exercise 6](#c2w2-ex-6)\n",
    "  - [2.2. Analyze Debugger results](#c2w2-2.2.)\n",
    "  - [2.3. Download SageMaker debugger profiling report](#c2w2-2.3.)\n",
    "- [3. Deploy the model](#c2w2-3.)\n",
    "- [4. Test model](#c2w2-4.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's review Amazon SageMaker \"Bring Your Own Script\" scheme:\n",
    "\n",
    "![](images/sagemaker_scriptmode.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this lab you will cover each part of the scheme. First, install and import the required modules:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.7/site-packages/secretstorage/dhcrypto.py:16: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead\n",
      "  from cryptography.utils import int_from_bytes\n",
      "/opt/conda/lib/python3.7/site-packages/secretstorage/util.py:25: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead\n",
      "  from cryptography.utils import int_from_bytes\n",
      "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\n",
      "Collecting package metadata (current_repodata.json): ...working... done\n",
      "Solving environment: ...working... done\n",
      "\n",
      "## Package Plan ##\n",
      "\n",
      "  environment location: /opt/conda\n",
      "\n",
      "  added / updated specs:\n",
      "    - pytorch==1.6.0\n",
      "\n",
      "\n",
      "The following packages will be downloaded:\n",
      "\n",
      "    package                    |            build\n",
      "    ---------------------------|-----------------\n",
      "    ca-certificates-2021.7.5   |       h06a4308_1         113 KB\n",
      "    certifi-2021.5.30          |   py37h06a4308_0         139 KB\n",
      "    cudatoolkit-10.2.89        |       hfd86e86_1       365.1 MB\n",
      "    ninja-1.10.2               |       hff7bd54_1         1.4 MB\n",
      "    openssl-1.1.1l             |       h7f8727e_0         2.5 MB\n",
      "    pytorch-1.6.0              |py3.7_cuda10.2.89_cudnn7.6.5_0       537.7 MB  pytorch\n",
      "    ------------------------------------------------------------\n",
      "                                           Total:       907.0 MB\n",
      "\n",
      "The following NEW packages will be INSTALLED:\n",
      "\n",
      "  cudatoolkit        pkgs/main/linux-64::cudatoolkit-10.2.89-hfd86e86_1\n",
      "  ninja              pkgs/main/linux-64::ninja-1.10.2-hff7bd54_1\n",
      "  pytorch            pytorch/linux-64::pytorch-1.6.0-py3.7_cuda10.2.89_cudnn7.6.5_0\n",
      "\n",
      "The following packages will be UPDATED:\n",
      "\n",
      "  ca-certificates    conda-forge::ca-certificates-2021.5.3~ --> pkgs/main::ca-certificates-2021.7.5-h06a4308_1\n",
      "  openssl                                 1.1.1k-h27cfd23_0 --> 1.1.1l-h7f8727e_0\n",
      "\n",
      "The following packages will be SUPERSEDED by a higher-priority channel:\n",
      "\n",
      "  certifi            conda-forge::certifi-2021.5.30-py37h8~ --> pkgs/main::certifi-2021.5.30-py37h06a4308_0\n",
      "\n",
      "\n",
      "Preparing transaction: ...working... done\n",
      "Verifying transaction: ...working... done\n",
      "Executing transaction: ...working... done\n",
      "/opt/conda/lib/python3.7/site-packages/secretstorage/dhcrypto.py:16: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead\n",
      "  from cryptography.utils import int_from_bytes\n",
      "/opt/conda/lib/python3.7/site-packages/secretstorage/util.py:25: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead\n",
      "  from cryptography.utils import int_from_bytes\n",
      "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "# please ignore warning messages during the installation\n",
    "!pip install --disable-pip-version-check -q sagemaker==2.35.0\n",
    "!conda install -q -y pytorch==1.6.0 -c pytorch\n",
    "!pip install --disable-pip-version-check -q transformers==3.5.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import boto3\n",
    "import sagemaker\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import botocore\n",
    "\n",
    "config = botocore.config.Config(user_agent_extra='dlai-pds/c2/w2')\n",
    "\n",
    "# low-level service client of the boto3 session\n",
    "sm = boto3.client(service_name='sagemaker', \n",
    "                  config=config)\n",
    "\n",
    "sm_runtime = boto3.client('sagemaker-runtime',\n",
    "                          config=config)\n",
    "\n",
    "sess = sagemaker.Session(sagemaker_client=sm,\n",
    "                         sagemaker_runtime_client=sm_runtime)\n",
    "\n",
    "bucket = sess.default_bucket()\n",
    "role = sagemaker.get_execution_role()\n",
    "region = sess.boto_region_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "%config InlineBackend.figure_format='retina'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='c2w2-1.'></a>\n",
    "# 1. Configure dataset, hyper-parameters and evaluation metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='c2w2-1.1.'></a>\n",
    "### 1.1. Configure dataset\n",
    "\n",
    "You have already transformed and balanced the data into a format that the model expects. Let's copy this data to S3. You will be using training and validation datasets to train the model. Test dataset will be used for tuning later. Setup the paths:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "processed_train_data_s3_uri = 's3://{}/data/sentiment-train/'.format(bucket)\n",
    "processed_validation_data_s3_uri = 's3://{}/data/sentiment-validation/'.format(bucket)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Upload the data to S3 bucket:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "upload: data/sentiment-train/part-algo-1-womens_clothing_ecommerce_reviews.tsv to s3://sagemaker-us-east-1-645012351640/data/sentiment-train/part-algo-1-womens_clothing_ecommerce_reviews.tsv\n",
      "upload: data/sentiment-validation/part-algo-1-womens_clothing_ecommerce_reviews.tsv to s3://sagemaker-us-east-1-645012351640/data/sentiment-validation/part-algo-1-womens_clothing_ecommerce_reviews.tsv\n"
     ]
    }
   ],
   "source": [
    "!aws s3 cp --recursive ./data/sentiment-train $processed_train_data_s3_uri\n",
    "!aws s3 cp --recursive ./data/sentiment-validation $processed_validation_data_s3_uri"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check the existence of those files in the S3 bucket:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2021-09-14 18:12:35    4894416 data/sentiment-train/part-algo-1-womens_clothing_ecommerce_reviews.tsv\n"
     ]
    }
   ],
   "source": [
    "!aws s3 ls --recursive $processed_train_data_s3_uri"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2021-09-14 18:12:36     276522 data/sentiment-validation/part-algo-1-womens_clothing_ecommerce_reviews.tsv\n"
     ]
    }
   ],
   "source": [
    "!aws s3 ls --recursive $processed_validation_data_s3_uri"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will need to setup the input data channels, wrapping the S3 locations in a `TrainingInput` object to use with the SageMaker Training Job. This can be organized as a dictionary\n",
    "\n",
    "```python\n",
    "data_channels = {\n",
    "    'train': ..., # training data\n",
    "    'validation': ... # validation data\n",
    "}\n",
    "```\n",
    "\n",
    "where training and validation data are the Amazon SageMaker channels for S3 input data sources."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='c2w2-ex-1'></a>\n",
    "### Exercise 1\n",
    "\n",
    "Create a train data channel.\n",
    "\n",
    "**Instructions**: Pass the S3 input path for training data into the `sagemaker.inputs.TrainingInput` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "s3_input_train_data = sagemaker.inputs.TrainingInput(\n",
    "    ### BEGIN SOLUTION - DO NOT delete this comment for grading purposes\n",
    "    s3_data=processed_train_data_s3_uri # Replace None\n",
    "    ### END SOLUTION - DO NOT delete this comment for grading purposes\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='c2w2-ex-2'></a>\n",
    "### Exercise 2\n",
    "\n",
    "Create a validation data channel.\n",
    "\n",
    "**Instructions**: Pass the S3 input path for validation data into the `sagemaker.inputs.TrainingInput` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "s3_input_validation_data = sagemaker.inputs.TrainingInput(\n",
    "    ### BEGIN SOLUTION - DO NOT delete this comment for grading purposes\n",
    "    s3_data=processed_validation_data_s3_uri # Replace None\n",
    "    ### END SOLUTION - DO NOT delete this comment for grading purposes\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='c2w2-ex-3'></a>\n",
    "### Exercise 3\n",
    "\n",
    "Organize data channels defined above as a dictionary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_channels = {\n",
    "    ### BEGIN SOLUTION - DO NOT delete this comment for grading purposes\n",
    "    'train': s3_input_train_data, # Replace None\n",
    "    'validation': s3_input_validation_data # Replace None\n",
    "    ### END SOLUTION - DO NOT delete this comment for grading purposes\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='c2w2-1.2.'></a>\n",
    "### 1.2. Configure model hyper-parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set the Training Job parameters including the instance type, instance count, learning rate, batch size etc. For the purposes of this lab, you will use a relatively small instance type. Please refer to [this link](https://aws.amazon.com/sagemaker/pricing/) for additional instance types that may work for your use cases outside of this lab."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "max_seq_length=128 # maximum number of input tokens passed to BERT model\n",
    "freeze_bert_layer=False # specifies the depth of training within the network\n",
    "epochs=3\n",
    "learning_rate=2e-5\n",
    "train_batch_size=256\n",
    "train_steps_per_epoch=50\n",
    "validation_batch_size=256\n",
    "validation_steps_per_epoch=50\n",
    "seed=42\n",
    "run_validation=True\n",
    "\n",
    "train_instance_count=1\n",
    "train_instance_type='ml.c5.9xlarge'\n",
    "train_volume_size=256\n",
    "input_mode='File'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some of them will be passed into the PyTorch estimator in the hyperparameters argument. Let's setup the dictionary for that:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "hyperparameters={\n",
    "    'max_seq_length': max_seq_length,\n",
    "    'freeze_bert_layer': freeze_bert_layer,\n",
    "    'epochs': epochs,\n",
    "    'learning_rate': learning_rate,\n",
    "    'train_batch_size': train_batch_size,\n",
    "    'train_steps_per_epoch': train_steps_per_epoch,\n",
    "    'validation_batch_size': validation_batch_size,\n",
    "    'validation_steps_per_epoch': validation_steps_per_epoch,    \n",
    "    'seed': seed,\n",
    "    'run_validation': run_validation\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='c2w2-1.3.'></a>\n",
    "### 1.3. Setup evaluation metrics\n",
    "\n",
    "Choose loss and accuracy as the evaluation metrics. The regular expressions `Regex` will capture the values of metrics that the algorithm will emit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "metric_definitions = [\n",
    "     {'Name': 'validation:loss', 'Regex': 'val_loss: ([0-9.]+)'},\n",
    "     {'Name': 'validation:accuracy', 'Regex': 'val_acc: ([0-9.]+)'},\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For example, these sample log lines...\n",
    "```\n",
    "[step: 100] val_loss: 0.76 - val_acc: 70.92%\n",
    "```\n",
    "\n",
    "...will produce the following metrics in CloudWatch:\n",
    "\n",
    "`validation:loss` =  0.76\n",
    "\n",
    "`validation:accuracy` = 70.92"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/cloudwatch_validation_metrics.png\" align=\"left\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='c2w2-1.4.'></a>\n",
    "### 1.4. Setup Debugger and Profiler\n",
    "\n",
    "Amazon SageMaker Debugger can be used to profile machine learning models, helping to identify and fix training issues caused by hardware resource usage. Setting some parameters in the SageMaker estimator, without any change to the training code, you can enable the collection of infrastructure and model metrics such as: CPU and GPU, RAM and GPU RAM, data loading time, time spent in ML operators running on CPU and GPU, distributed training metrics and many more. In addition, you can visualize how much time is spent in different phases, such as preprocessing, training loop, and postprocessing. If needed, you can drill down on each training epoch, and even on each function in your training script.\n",
    "    \n",
    "Define Debugger Rules as described here:  https://docs.aws.amazon.com/sagemaker/latest/dg/debugger-built-in-rules.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.debugger import Rule, ProfilerRule, rule_configs\n",
    "from sagemaker.debugger import DebuggerHookConfig\n",
    "from sagemaker.debugger import ProfilerConfig, FrameworkProfile"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`DebuggerHookConfig` provides options to customize how debugging information is emitted and saved. `s3_output_path` argument value defines the location in Amazon S3 to store the output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "debugger_hook_config = DebuggerHookConfig(\n",
    "    s3_output_path='s3://{}'.format(bucket),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`ProfilerConfig` sets the configuration for collecting system and framework metrics of SageMaker Training Jobs. Parameter `system_monitor_interval_millis` sets the time interval to collect system metrics (in milliseconds). Parameter `framework_profile_params` is the object for framework metrics profiling. Here you will set its local path, the step at which to start profiling, `start_step`, and the number of steps to profile, `num_steps`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.debugger import ProfilerConfig, FrameworkProfile\n",
    "\n",
    "profiler_config = ProfilerConfig(\n",
    "    system_monitor_interval_millis=500,\n",
    "    framework_profile_params=FrameworkProfile(local_path=\"/opt/ml/output/profiler/\", start_step=5, num_steps=10)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For monitoring and profiling the built-in rules you can use the `ProfilerReport`. It creates a profiling report and updates when the individual rules are triggered. If you trigger this `ProfilerReport` rule without any customized parameter as in the cell below, then the `ProfilerReport` rule triggers all of the built-in rules for monitoring and profiling with their default parameter values.\n",
    "\n",
    "The profiling report can be downloaded while the Training Job is running or after the job has finished."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "rules=[ProfilerRule.sagemaker(rule_configs.ProfilerReport())]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='c2w2-2.'></a>\n",
    "# 2. Train model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='c2w2-2.1.'></a>\n",
    "### 2.1. Setup the RoBERTa and PyTorch script to run on SageMaker\n",
    "You will prepare the PyTorch model to run as a SageMaker Training Job in a separate Python file, which will be called during the training. \n",
    "\n",
    "Here you will be using the pre-trained model `roberta-base`. The information about the available models can be found in the [Hugging Face website](https://huggingface.co/models)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='c2w2-ex-4'></a>\n",
    "### Exercise 4\n",
    "\n",
    "1. Open the file [src/train.py](src/train.py). Go through the comments to understand its content.\n",
    "2. Find and review the `configure_model()` function, which contains the RoBERTa model configuration. \n",
    "3. In the following function investigate given mapping `label2id` of a 0-indexed list of classes used by BERT \\[0, 1, 2\\] to the list of the sentiment values \\[-1, 0, 1\\]:\n",
    "\n",
    "```python\n",
    "    config = RobertaConfig.from_pretrained(\n",
    "        PRE_TRAINED_MODEL_NAME, \n",
    "        num_labels=len(classes),\n",
    "        id2label={\n",
    "            ...: ...,\n",
    "            ...: ...,\n",
    "            ...: ...,\n",
    "        },\n",
    "        label2id={\n",
    "            -1: 0,\n",
    "            0: 1,\n",
    "            1: 2,\n",
    "        }\n",
    "    )\n",
    "\n",
    "``` \n",
    "\n",
    "4. Update the function setting up the opposite mapping `id2label`: sentiment values \\[-1, 0, 1\\] to a 0-indexed list of classes used by BERT.\n",
    "\n",
    "5. Save the file [src/train.py](src/train.py) (with the menu command File -> Save Python File)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "252276544fd14a75990dad9683fe51c2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=481.0, style=ProgressStyle(description_…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "##################\n",
      "Updated correctly!\n",
      "##################\n"
     ]
    }
   ],
   "source": [
    "import sys, importlib\n",
    "sys.path.append('src/')\n",
    "\n",
    "import train\n",
    "\n",
    "# reload the module if it has been previously loaded\n",
    "if 'train' in sys.modules:\n",
    "    importlib.reload(train)\n",
    "\n",
    "# Ignore warnings below\n",
    "config = train.configure_model()\n",
    "\n",
    "label_0 = config.id2label[0]\n",
    "label_1 = config.id2label[1]\n",
    "label_2 = config.id2label[2]\n",
    "\n",
    "updated_correctly = False\n",
    "\n",
    "if label_0 != -1 or label_1 != 0 or label_2 != 1:\n",
    "    print('#######################################################################################')\n",
    "    print('Please check that the function \\'configure_model\\' in the file src/train.py is complete.')\n",
    "    print('########################################################################################')\n",
    "    raise Exception('Please check that the function \\'configure_model\\' in the file src/train.py is complete.')\n",
    "else:\n",
    "    print('##################')    \n",
    "    print('Updated correctly!')        \n",
    "    print('##################')        \n",
    "\n",
    "    updated_correctly = True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Setup the PyTorch estimator to train our model. For more information on the PyTorch estimator, see the documentation [here](https://sagemaker.readthedocs.io/en/stable/frameworks/pytorch/sagemaker.pytorch.html). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.pytorch import PyTorch as PyTorchEstimator\n",
    "\n",
    "if updated_correctly:\n",
    "    estimator = PyTorchEstimator(\n",
    "        entry_point='train.py',\n",
    "        source_dir='src',\n",
    "        role=role,\n",
    "        instance_count=train_instance_count,\n",
    "        instance_type=train_instance_type,\n",
    "        volume_size=train_volume_size,\n",
    "        py_version='py3', # dynamically retrieves the correct training image (Python 3)\n",
    "        framework_version='1.6.0', # dynamically retrieves the correct training image (PyTorch)\n",
    "        hyperparameters=hyperparameters,\n",
    "        metric_definitions=metric_definitions,\n",
    "        input_mode=input_mode,\n",
    "        debugger_hook_config=debugger_hook_config,\n",
    "        profiler_config=profiler_config,\n",
    "        rules=rules\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='c2w2-ex-5'></a>\n",
    "### Exercise 5\n",
    "\n",
    "Launch the SageMaker Training Job which will be fitting the model to the dataset.\n",
    "\n",
    "**Instructions**: Use the `estimator.fit` function, passing the configured train and validation inputs (data channels).\n",
    "\n",
    "```python\n",
    "estimator.fit(\n",
    "    inputs=..., # train and validation input\n",
    "    wait=False # do not wait for the job to complete before continuing\n",
    ")\n",
    "``` "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "estimator.fit(\n",
    "    ### BEGIN SOLUTION - DO NOT delete this comment for grading purposes\n",
    "    inputs=data_channels, # Replace None\n",
    "    ### END SOLUTION - DO NOT delete this comment for grading purposes\n",
    "    wait=False\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can refer to the last Training Job using the estimator function `latest_training_job`. Then the Training Job name can be found with the `name` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Job name: pytorch-training-2021-09-14-18-16-16-595\n"
     ]
    }
   ],
   "source": [
    "training_job_name = estimator.latest_training_job.name\n",
    "\n",
    "print('Training Job name: {}'.format(training_job_name))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also load the information about the Training Job using the function `describe()`. The result is in dictionary format. Let's check that it has the same Training Job name:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Job name: pytorch-training-2021-09-14-18-16-16-595\n"
     ]
    }
   ],
   "source": [
    "training_job_name = estimator.latest_training_job.describe()['TrainingJobName']\n",
    "\n",
    "print('Training Job name: {}'.format(training_job_name))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='c2w2-ex-6'></a>\n",
    "### Exercise 6\n",
    "\n",
    "Pull the Training Job status from the Training Job description.\n",
    "\n",
    "**Instructions**: Print the keys of the Training Job description dictionary, choose the one related to the primary status of the Training Job and print the value of it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dict_keys(['TrainingJobName', 'TrainingJobArn', 'TrainingJobStatus', 'SecondaryStatus', 'HyperParameters', 'AlgorithmSpecification', 'RoleArn', 'InputDataConfig', 'OutputDataConfig', 'ResourceConfig', 'StoppingCondition', 'CreationTime', 'LastModifiedTime', 'SecondaryStatusTransitions', 'EnableNetworkIsolation', 'EnableInterContainerTrafficEncryption', 'EnableManagedSpotTraining', 'DebugHookConfig', 'ProfilerConfig', 'ProfilerRuleConfigurations', 'ProfilerRuleEvaluationStatuses', 'ProfilingStatus', 'ResponseMetadata'])\n"
     ]
    }
   ],
   "source": [
    "print(estimator.latest_training_job.describe().keys())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training Job status: InProgress\n"
     ]
    }
   ],
   "source": [
    "### BEGIN SOLUTION - DO NOT delete this comment for grading purposes\n",
    "training_job_status_primary = estimator.latest_training_job.describe()['TrainingJobStatus'] # Replace None\n",
    "### END SOLUTION - DO NOT delete this comment for grading purposes\n",
    "print('Training Job status: {}'.format(training_job_status_primary))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Review the Training Job in the console.\n",
    "\n",
    "**Instructions**: \n",
    "- open the link\n",
    "- notice that you are in the section `Amazon SageMaker` -> `Training jobs`\n",
    "- check the name of the Training Job, its status and other available information\n",
    "- review metrics in the `Monitor` section"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<b>Review <a target=\"blank\" href=\"https://console.aws.amazon.com/sagemaker/home?region=us-east-1#/jobs/pytorch-training-2021-09-14-18-16-16-595\">Training Job</a></b>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "\n",
    "display(HTML('<b>Review <a target=\"blank\" href=\"https://console.aws.amazon.com/sagemaker/home?region={}#/jobs/{}\">Training Job</a></b>'.format(region, training_job_name)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Review the Cloud Watch logs (after about 5 minutes).\n",
    "\n",
    "**Instructions**: \n",
    "- open the link\n",
    "- open the log stream with the name, which starts from the training job name\n",
    "- have a quick look at the log messages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<b>Review <a target=\"blank\" href=\"https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#logStream:group=/aws/sagemaker/TrainingJobs;prefix=pytorch-training-2021-09-14-18-16-16-595;streamFilter=typeLogStreamPrefix\">CloudWatch logs</a> after about 5 minutes</b>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "\n",
    "display(HTML('<b>Review <a target=\"blank\" href=\"https://console.aws.amazon.com/cloudwatch/home?region={}#logStream:group=/aws/sagemaker/TrainingJobs;prefix={};streamFilter=typeLogStreamPrefix\">CloudWatch logs</a> after about 5 minutes</b>'.format(region, training_job_name)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<b>Review <a target=\"blank\" href=\"https://s3.console.aws.amazon.com/s3/buckets/sagemaker-us-east-1-645012351640/pytorch-training-2021-09-14-18-16-16-595/?region=us-east-1&tab=overview\">S3 output data</a> after the Training Job has completed</b>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "\n",
    "display(HTML('<b>Review <a target=\"blank\" href=\"https://s3.console.aws.amazon.com/s3/buckets/{}/{}/?region={}&tab=overview\">S3 output data</a> after the Training Job has completed</b>'.format(bucket, training_job_name, region)))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wait for the Training Job to complete.\n",
    "\n",
    "### _This cell will take approximately 30-40 minutes to run._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "2021-09-14 18:16:48 Starting - Launching requested ML instances......\n",
      "2021-09-14 18:17:25 Starting - Preparing the instances for training................\n",
      "2021-09-14 18:18:49 Downloading - Downloading input data....\n",
      "2021-09-14 18:19:13 Training - Downloading the training image..\n",
      "2021-09-14 18:19:29 Training - Training image download completed. Training in progress.............................................................................................................................................................................................................................................................................................................................................................................................................................................................\n",
      "2021-09-14 18:56:49 Uploading - Uploading generated training model....................................\n",
      "2021-09-14 18:59:53 Completed - Training job completed\n",
      "CPU times: user 2 s, sys: 299 ms, total: 2.3 s\n",
      "Wall time: 43min 5s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "estimator.latest_training_job.wait(logs=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_Wait until the ^^ Training Job ^^ completes above_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Review the training metrics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>timestamp</th>\n",
       "      <th>metric_name</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>validation:loss</td>\n",
       "      <td>1.10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1260.0</td>\n",
       "      <td>validation:loss</td>\n",
       "      <td>1.02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1740.0</td>\n",
       "      <td>validation:loss</td>\n",
       "      <td>0.66</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.0</td>\n",
       "      <td>validation:accuracy</td>\n",
       "      <td>34.77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1260.0</td>\n",
       "      <td>validation:accuracy</td>\n",
       "      <td>50.39</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1740.0</td>\n",
       "      <td>validation:accuracy</td>\n",
       "      <td>69.14</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   timestamp          metric_name  value\n",
       "0        0.0      validation:loss   1.10\n",
       "1     1260.0      validation:loss   1.02\n",
       "2     1740.0      validation:loss   0.66\n",
       "3        0.0  validation:accuracy  34.77\n",
       "4     1260.0  validation:accuracy  50.39\n",
       "5     1740.0  validation:accuracy  69.14"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_metrics = estimator.training_job_analytics.dataframe()\n",
    "df_metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can query and plot the training metrics:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x7fc34fb07a90>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuIAAAILCAYAAACtqPl1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAWJQAAFiUBSVIk8AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdeXxU9d328esXAgFCEgg7ZEFE9jWBJG6ooHXFFRBJfOzdp73rWtdaH6uWVqultlqXqr2rt1UTQEGrRUXrgqLVBEjYdwSysENYspD99/wxk4XMJAQyYbbP+/XidThzfjPztXG5ejjnXMZaKwAAAACnV4i3BwAAAACCEUEcAAAA8AKCOAAAAOAFBHEAAADACwjiAAAAgBcQxAEAAAAvIIgDAAAAXkAQBwAAALyAIA4AAAB4AUEcAAAA8AKCOAAAAOAFBHEAAADAC0K9PUBbMMZslxQpaYeXRwEAAEBgGyDpqLX2jJN9Y0AGcUmRnTp1ih42bFi0twcBAABA4NqwYYOOHTt2Su8N1CC+Y9iwYdHZ2dnengMAAAABLDExUTk5OTtO5b1cIw4AAAB4AUEcAAAA8IJWB3FjzI+NMfYEv6rdvO8cY8zHxphCY0ypMWa1MeYeY0y71s4EAAAA+DpPXCO+UtJvmzh2vqRJkhY1fNEYc42kdyWVSXpbUqGkKZKelXSupGkemAsAAADwWa0O4tbalXKEcRfGmO+dv/2fBq9FSvq7pGpJF1prlztff1TSl5KmGmNmWGvntXY2AAAAwFe12TXixpiRklIk7ZT0UYNDUyX1lDSvNoRLkrW2TNIjzt3b2mouAAAAwBe05c2aP3duX7PWNrxGfJJz+4mb9yyRVCrpHGNMWBvOBgAAAHhVmzxH3BjTSVKapBpJrzY6PMS53dz4fdbaKmcr5ghJAyVtOMH3NPWg8KEtnbWmpkaFhYUqKipSeXm5rLUtfSv8jDFGYWFhioiIUHR0tEJCeGgQAADwnrYq9Jkuqaukj6y1+Y2ORTm3R5p4b+3rXdtisIZqamqUn5+v0tLStv4q+ABrrcrKylRWVqaSkhLFxsYSxgEAgNe0VRD/b+f2b6fwXuPcnvDUtLU20e0HOM6UJ5zo/YWFhSotLVVoaKj69Omj8PBwglkAq6mpUUlJifbs2aPS0lIVFhaqR48e3h4LAAAEKY+nTmPMcEnnSCqQ9LGbJbVnvKPcHJOkyEbr2kxRUZEkqU+fPoqIiCCEB7iQkBBFRESoT58+kup//gAAAN7QFsmzqZs0a21ybgc3PmCMCZV0hqQqSdvaYLbjlJeXS5LCw8Pb+qvgQ2p/3rU/fwAAAG/waBA3xnSUdLMcN2m+1sSyL53by9wcmyips6TvrLVtnpJqb8zkTHhwMcZx9RM35gIAAG/ydAKdJqmbpI/d3KRZa4GkA5JmGGPG177oDPFPOHdf9vBcQJ3aIA4AAOBNnr5Zs/Ymzf9paoG19qgx5mdyBPKvjDHz5Ki4v1qORxsukKP2HgAAAAhYHjsjbowZJuk8NX2TZh1r7fuSLpCjwOcGSXdJqpR0n6QZlmsGAAAA0EJrdx7R6oLD3h7jpHnsjLi1doPqHz3YkvX/kXSFp74fvm3AgAGSpB07dnh1DgAAEFgOlVTo529la39RuX53zQjNSIrz9kgtxl2KAAAA8EvVNVZ3v71SOw8fU0V1jX7/8QbtL/Kfp6IRxAEAAOCXnvtii5Zs3l+3/8z0seoZEebFiU4OQRwAAAB+58uNe/X8F1vq9m+/8ExdMry3Fyc6eQRx6Pvvv5cxRtdff32Ta4YNG6awsDAVFhaqoqJCL774oq644grFx8crLCxM0dHRuvjii7Vo0aIWf++sWbNkjNFXX33lcmzHjh0yxujHP/6xy7HS0lI99dRTGjt2rMLDw9WlSxedffbZmjt3bou/GwAA+K+8g6W6Z97Kuv3zBvXQ/T8a4sWJTg1BHDr77LM1ZMgQffjhhzp48KDL8aVLl2rjxo2aMmWKoqOjVVhYqLvvvltFRUW65JJLdN999+nqq6/WihUrdMUVV+jVV19ts1kPHz6s8847Tw8//LDatWunn/zkJ7rlllu0f/9+zZw5U4888kibfTcAAPC+sspq3ZqeraNlVZKkflEd9dyMsWoX4n89IZ5+jjj81C233KKHH35Yc+fO1Z133nncsTfeeKNujSR169ZNubm5iomJOW7dkSNHdO655+rBBx9UamqqOnXq5PE577nnHq1YsUKzZ8/Wgw8+WPd6WVmZrr32Wj355JOaOnWqxo4d6/HvBgAA3mWt1SPvr9X63UclSe3bGf01NUHdu/jPdeENEcSbMeChj7w9Qovt+MOVrXr/zTffrEceeURvvPHGcUG8oqJC8+bNU69evXT55ZdLksLCwlxCuCRFRUXpJz/5ie6//34tW7ZMEydObNVMjR08eFDp6ekaP378cSFckjp27KjZs2fr008/1Zw5cwjiAAAEoLlL87Ugu6Bu/zdTRmhcXDcvTtQ6BHFIkmJiYjR58mR99tlnWr9+vYYPHy5JWrhwoQoLC3XvvfcqNLT+b5d169bp6aef1pIlS7R7926VlZUd93k7d+70+IzLli1TdXW1jDGaNWuWy/HKykpJ0oYNGzz+3QAAwLtW5R/WrH+tq9u/PqG/UpP955nh7hDEUefHP/6xPvvsM73xxhuaPXu2JNfLUiQpMzNTkyZNUlVVlSZPnqyrr75akZGRCgkJ0cqVK/XBBx+ovNzzz/CsvX592bJlWrZsWZPriouLPf7dAADAewpLKnR7Ro4qqmskSUP7ROj3146SMf53XXhDBPFmtPZyD39z3XXXKTIyUunp6XryySdVWFioRYsWacyYMRozZkzduieeeELHjh3T4sWLdeGFFx73GU899ZQ++OCDFn1fSIjjXuGqqiqXY4cPu9bURkVFSZLuvfdePfPMMy39ywIAAH6susbq7nkrtPPwMUlSRMdQ/e3mRHXq0M7Lk7UeT01BnU6dOmn69OnatWuXPv/8c2VkZKiqquq4s+GStHXrVkVHR7uEcEn6+uuvW/x93bo5runKz893ObZ8+XKX15KSkhQSEqJvvvmmxd8BAAD8218+36xvthyo379xrOK7h3txIs8hiOM4tc/tfvPNN/Xmm28qNDRUqampx60ZMGCACgsLtXr16uNef+211/Tpp5+2+LuSkpIkSa+//vpxZ8Xz8/P1u9/9zmV9r169lJqaquXLl+vxxx93eyb9hx9+0Pbt21s8AwAA8F1fbNirF77cWrd/50WDNHmYf5X2NIdLU3Ccc889V4MGDdL8+fNVWVmpKVOmqFevXsetueeee/Tpp5/qvPPO0/Tp0xUVFaXly5fr22+/1dSpU7VgwYIWfVdycrImTpyoJUuWKCkpSZMmTdLevXu1cOFCXXrppW7PlL/44ovasmWLHnvsMb311ls677zz1Lt3b+3atUsbNmzQsmXLNHfuXJ1xxhke+d8DAAB4R+7BEt3zdn1pz/ln9dC9lwz24kSexxlxuLjlllvqnkDS+LIUSbrsssu0cOFCDR8+XG+//bZee+01hYWFafHixbryypO7rv6DDz7QT3/6UxUUFOiFF17QihUr9Mc//rHuZtHGIiMj9fXXX+uFF15Qjx499O677+qZZ57R4sWLFRERoWeffVaXXHLJyf9FAwAAn3Gsolq3pueoyFna079rJz03Y5xflvY0x1hrvT2DxxljshMSEhKys7ObXVf7mLthw4adjrHgQ/jZAwDgm6y1un/+Kr2X43gUcod2IZp/69kaE9vVy5O5l5iYqJycnBxrbeLJvpcz4gAAAPAZGVl5dSFckn5z9XCfDeGtRRAHAACAT1iZf1i/W7i+bn9qYoxmJvl3aU9zCOIAAADwuoPF5bo9PbuutGd430g9ce1Ivy/taQ5BHAAAAF7lKO1ZqV1HyiRJkR1D9Upaojq29//SnuYQxAEAAOBVz3y2Sd9ubVDaM2Os4rp39uJEpwdBHAAAAF7z2fq9+uviH+r2fzFpkCYNDZzSnuYQxBF0AvGRnQAA+KMdB0p0X6PSnrsvDqzSnuYEdRCvvfi/pqbGy5PgdKoN4oF88wcAAL7OUdqTraLy+tKe5wOwtKc5QR3Ew8LCJEklJSVengSnU+3Pu/bnDwAATi9rrX79zzXauKdIkqO05+W0BHUL7+DlyU6voA7iERERkqQ9e/aoqKhINTU1XLYQoKy1qqmpUVFRkfbs2SOp/ucPAABOr/SsPL23or6057fXjNDomMAs7WlOqLcH8Kbo6GiVlJSotLRUBQUF3h4Hp1Hnzp0VHR3t7TEAAAg6OXmH9LuF6+r2p4+P0YwJsV6cyHuCOoiHhIQoNjZWhYWFKioqUnl5OWfEA5gxRmFhYYqIiFB0dLRCQoL6D4QAADjtDhaX646MHFVWO/LWiH6R+t01gV3a05ygDuKSI4z36NFDPXr08PYoAAAAAauqukZ3zV2h3c7SnqhO7YOitKc5nBIEAABAm/vzZ5v13Q8HJUnGOEp7YqMDv7SnOQRxAAAAtKl/r9ujl79qWNpzli4a0suLE/kGgjgAAADazPYDJbr/nVV1+xcM7qm7J5/lxYl8B0EcAAAAbaK0okq3vlVf2hPTrZOemzFWIUFU2tMcgjgAAAA8zlqrh99bo017naU9oSF6JS1RXTsHV2lPcwjiAAAA8Li3MnP1/spddfuPXzNCI/tHeXEi30MQBwAAgEdl5x7S4x+ur9ufMSFWN06I8+JEvokgDgAAAI850Ki0Z1T/KM26eoSXp/JNBHEAAAB4RFV1je6as0J7jjpKe7p2bq+XUhOCurSnOQRxAAAAeMTT/96k77c1KO25kdKe5hDEAQAA0GqfrN2jv329rW7/nsmDdSGlPc0iiAMAAKBVtu0v1gPz60t7LhrSU3dNGuTFifwDQRwAAACnrLSiSremZ6vYWdoTG91Jz95IaU9LEMQBAABwSqy1eujdNdq8t1iSFBYaopdTKe1pKY8GcWPM+caYd40xu40x5c7tv40xVzRYM8AYY5v5Nc+TMwEAAKBtvPHdDv1rVYPSnmtHUtpzEkI99UHGmEckPS7pgKQPJe2W1EPSOEkXSvq40VtWSXrfzUet9dRMAAAAaBvLdxTqiY821O3flBSr6eNjvTiR//FIEDfGTJMjhH8u6XprbVGj4+3dvG2ltXaWJ74fAAAAp8/+onLdMSdHVTWO0p7RMVH6zRRKe05Wqy9NMcaESJotqVTSzMYhXJKstZWt/R4AAAB4X1V1je6am6O9R8slUdrTGp44I36OpDMkLZB0yBhzpaSRksokLbXWft/E+/oZY34uqbukg5K+t9au9sA8AAAAaCNPf7pJmdsKJTlKe56fMU4x3SjtORWeCOITnNu9knIkjWp40BizRNJUa+3+Ru+7xPmr4dqvJN1irc1ryRcbY7KbODS0Je8HAABAyy1as1t/W1Jf2nPfxYM1cXBPL07k3zzx1JTayqRbJXWSdLGkCDnOin8qaaKk+Q3Wl8pxPXmipG7OXxdIWizHTZ1fGGPCPTAXAAAAPOSH/cX65YL6ixcmD+2lOy6itKc1PHFGvPaCICPHme/aWqV1xpjrJG2WdIEx5mxr7ffW2n2SHmv0GUuMMT+S9K2kZEk/lfTcib7YWpvo7nXnmfKEk/9LAQAAQGMl5VW69a360p646M56ZjqlPa3liTPih5zbbQ1CuCTJWntMjrPikpTU3IdYa6skvercneiBuQAAANBK1lr96t3V2rKvQWlPWoKiOrt7KB5OhieC+Cbn9nATx2uDeqcWfFbtdeRcmgIAAOADXv/PDn24enfd/u+vG6UR/Sjt8QRPBPElkqoknWWMcddnOtK53dGCz0pxbrc1uwoAAABtbtmOQj35cX1pz8zkOE1NjPHiRIGl1UHcWntA0tuSotTo2m9jzCWSLpV0RNInzteS3QV2Y8wkSfc6d9NbOxcAAABO3b6iMt2RUV/aMyYmSr+ZMtzLUwUWT1Xc3yfHTZa/NsZMlLRUUryk6yRVS/qZtbb20pXZkkY4H1VY4HxttKRJzt8/aq39zkNzAQAA4CRVVtfozjkrtK/IUdrTrXN7vZSWqLBQSns8ySNB3Fq7zxiTLOkROcJ3iqQiSR9Jespam9lg+VvONRMkXS6pvRzPIH9H0ovW2m88MRMAAABOzR8/2ail2xuU9tw0Tv27tuR2P5wMT50Rl7W2UI4z4/edYN1rkl7z1PcCAADAcz5avVt//2Z73f4DPxqi88+itKcteOJmTQAAAASArfuK9OCC+qdRXzysl2674EwvThTYCOIAAABQcXmVbk3PUUlFtSQpvntn/ZnSnjZFEAcAAAhy1lr9asFqbXWW9nRsH6KXUxMV1YnSnrZEEAcAAAhyr327XR+tqS/tefK6URreL9KLEwUHgjgAAEAQW7q9UE8t2li3n5YSp+sTKO05HQjiAAAAQWrf0TLdMSdH1c7SnrGxXfXoVZT2nC4EcQAAgCBUWV2jO+bkaL+ztCc6vINeSk2gtOc0IogDAAAEoT8s2qhlOw5JkkKM9MJN49SP0p7TiiAOAAAQZD5cvUuvfdugtOfSITp3UA8vThScCOIAAABBZMveIj24YHXd/iXDe1Pa4yUEcQAAgCDhKO3JVqmztGdA98768/QxMobSHm8giAMAAAQBa60eXLBKP+wvkeQo7Xnl5kRFdqS0x1sI4gAAAEHg1W+26+M1e+r2/3D9aA3tQ2mPNxHEAQAAAlzWtoP6wyf1pT3/5+x4XTuuvxcngkQQBwAACGh7j5bpjjkr6kp7xsV11SNXUtrjCwjiAAAAAaqyukZ3ZOToQLGjtKe7s7SnQygR0BfwUwAAAAhQT368Qctzjy/t6RtFaY+vIIgDAAAEoH+t2qXX/7Ojbv+Xlw7VOZT2+BSCOAAAQIDZvLdIv2pQ2nPpiN669YKBXpwI7hDEAQAAAkhRWaVufStbxyodpT1n9AjX09Mo7fFFBHEAAIAAYa3VL+ev1rYDjtKeTu3b6ZU0Snt8FUEcAAAgQPzPkm36ZF2D0p4bRmlInwgvToTmEMQBAAACwPc/HNTsBqU9Pz5ngK4ZS2mPLyOIAwAA+Lk9R8p019wcOTt7lBDXVQ9fMcy7Q+GECOIAAAB+rKKqRrdnZOtAcYUkqUeXDnopNZHSHj/ATwgAAMCPPfnxBuXkHZYktQsxeuGmBPWJ6ujlqdASBHEAAAA/9cHKnfrHdzvq9h+8dIjOPrO79wbCSSGIAwAA+KFNe4r00Ltr6vYvG9FH/z2R0h5/QhAHAADwM0fLKnVren1pz8Ce4Xp62mhKe/wMQRwAAMCPWGv1wDurtL1RaU8EpT1+hyAOAADgR175epv+vX5v3f7sqaM1uDelPf6IIA4AAOAnvtt6QE9/Wl/a81/nDtDVY/p5cSK0BkEcAADAD+w+ckx3zV1RV9ozPr4bpT1+jiAOAADg4xylPTk6WFJb2hOmv6YmqH07opw/46cHAADg4574aL1WNCjt+evMceodSWmPvyOIAwAA+LD3V+zUm9/n1u0/dNlQJQ+ktCcQEMQBAAB81IbdR/XQe6vr9q8Y1Uc/Pf8ML04ETyKIAwAA+KAjxyp1W3q2yiprJEln9gzXH6eOobQngBDEAQAAfExNjdUD81dpx8FSSVLnDu30t5sT1SUs1MuTwZMI4gAAAD7mlSU/6LMGpT1/nDpag3pR2hNoCOIAAAA+5D9bD+hPn26q2/+/552hq0ZT2hOICOIAAAA+Ytfh40t7JgzopocuH+rdodBmCOIAAAA+oLyqWrdn5KjQWdrTMyJMf51JaU8g8+hP1hhzvjHmXWPMbmNMuXP7b2PMFW7WnmOM+dgYU2iMKTXGrDbG3GOMaefJmQAAAPzB4x+u18r8hqU9CepFaU9A89itt8aYRyQ9LumApA8l7ZbUQ9I4SRdK+rjB2mskvSupTNLbkgolTZH0rKRzJU3z1FwAAAC+7r2cAqVn5tXt/7/LhyrpjGgvToTTwSNB3BgzTY4Q/rmk6621RY2Ot2/w+0hJf5dULelCa+1y5+uPSvpS0lRjzAxr7TxPzAYAAODL1u86qof/uaZu/8rRffV/z6O0Jxi0+tIUY0yIpNmSSiXNbBzCJclaW9lgd6qknpLm1YZw55oySY84d29r7VwAAAC+7sixSt2WUV/aM6hXF82+YTSlPUHCE2fEz5F0hqQFkg4ZY66UNFKOy06WWmu/b7R+knP7iZvPWiJHoD/HGBNmrS33wHwAAAA+p6bG6v53VirXWdoT3qGdXkmjtCeYeOInPcG53SspR9KohgeNMUskTbXW7ne+NMS53dz4g6y1VcaY7ZJGSBooaUNzX2yMyW7iEM/5AQAAPu3lr3/Q5xv21e0/PW2MBvXq4sWJcLp54qkpvZzbWyV1knSxpAg5zop/KmmipPkN1kc5t0ea+Lza17t6YDYAAACf882W/frTv+tLe352/hm6YlRfL04Eb/DEGfHaxw0aOc58r3LurzPGXCfHme8LjDFnu7lMxZ3ai6LsiRZaaxPdfoDjTHlCC74LAADgtNp5+Jh+MXeFrDPpJJ0RrV9dxh/mByNPnBE/5NxuaxDCJUnW2mNynBWXpCTntvaMd5Tci2y0DgAAICCUV1Xr9vRsHSp1PMeiV0SYXpw5TqGU9gQlT/zUa/9c5XATx2uDeqdG6wc3XmiMCZXjxs8qSds8MBsAAIDP+O3C9VpV4DjXGBpi9NfUBPWKoLQnWHkiiC+RIzifZYzp4Ob4SOd2h3P7pXN7mZu1EyV1lvQdT0wBAACBZEF2geZk1Zf2PHzFME0YQGlPMGt1ELfWHpCjHTNK0mMNjxljLpF0qRyXmdQ+rnCBHO2bM4wx4xus7SjpCefuy62dCwAAwFes23VEv25Q2nPV6L76r3MHeG8g+ARPPajyPknJkn5tjJkoaamkeEnXydGg+TNr7WFJstYeNcb8TI5A/pUxZp4cFfdXy/FowwVyBHsAAAC/d6S0Urel56i8ylHacxalPXDyyJ0B1tp9cgTxZyXFSvqFHMU9H0k631o7v9H69yVdIMdlLTdIuktSpRyBfoa19oRPTAEAAPB1NTVW976zUnmFjtKeLmGheuXmRIVT2gN57oy4rLWFcgTp+1q4/j+SrvDU9wMAAPiavy7eqi83NijtmTpaZ/aktAcOPCsHAACgDSzZvF/PfF5fJP7ziQN1OaU9aIAgDgAA4GEFh0r1i3n1pT0pA6P1y0uHeHco+ByCOAAAgAeVVVbr9owcHXaW9vSODNMLNyVQ2gMX/B0BAADgQb9duF6rG5T2vJSaoJ4RYV6eCr6IIA4AAOAh7yzP19yl9aU9j1w5TInxlPbAPYI4AACAB6zdeUSPvr+2bv/qMf10yzkDvDcQfB5BHAAAoJUOl1botozsutKeIb0j9IcbRlHag2YRxAEAAFqhpsbq3rdXKr/wmCQpIixUL6clqHMHSnvQPII4AABAK7zw5VYt3rS/bv/paWM0kNIetABBHAAA4BR9tWmf/vJFfWnPrRecqctG9vHiRPAnBHEAAIBTkF9Yqrvnrawr7Tl7YHc98KPB3h0KfoUgDgAAcJJqS3uOHHOU9vSJ7KgXZo6jtAcnhb9bAAAATtKsf63Tmp2O0p727Yz+mpqgHl0o7cHJIYgDAACchLeX5Wnesvy6/UevGq7E+G5enAj+iiAOAADQQmsKjujRD9bV7V87tp9uTon34kTwZwRxAACAFjhU4ijtqWhQ2vPk9ZT24NQRxAEAAE6gusbqnrdXquBQfWnPKzcnUtqDViGIAwAAnMDzX2zR15vrS3v+PH2MzugR7sWJEAgI4gAAAM1YvHGfnv9yS93+bReeqR+NoLQHrUcQBwAAaEJ+Yanuebu+tOfcQd11/yWU9sAzCOIAAABulFVW69b07LrSnr5RHfX8DEp74Dn8nQQAAODGYx+s1bpdRyU5SnteSk1Qd0p74EEEcQAAgEbmLc3TO8sL6vYfmzJC4+Io7YFnEcQBAAAaWF1wWI81KO25flx/pSXHeXEiBCqCOAAAgNOhkgrdlp6jimpHac/QPhH6/XWU9qBtEMQBAADkKO25++2V2nnYWdrTMVSvpCWqU4d2Xp4MgYogDgAAIOm5zzdrSYPSnmenj9UASnvQhgjiAAAg6H25ca+e/3Jr3f4dF52pi4f39uJECAYEcQAAENTyDpbqnnkr6/bPG9RD910yxIsTIVgQxAEAQNCqLe05WlYlSeoX1VHP3zRO7UK4ORNtjyAOAACCkrVWj7y/Vut3O0p7OrQL0UtpiYoO7+DlyRAsCOIAACAozV2arwXZ9aU9v7l6uMbGdvXiRAg2BHEAABB0VuUf1qx/1Zf23JAQo5lJlPbg9CKIAwCAoFJYUqHb0rPrSnuG9Y3UE9eOpLQHpx1BHAAABI3qGqu7563QriNlkqTIjqF6JS2B0h54BUEcAAAEjWc/26xvthyo379xrOK7U9oD7yCIAwCAoPD5+r16cXF9ac9dkwZp8jBKe+A9BHEAABDwdhwo0b3v1Jf2nH9WD91z8WAvTgQQxAEAQIA7VuEo7Slylvb079pJz8+gtAfeRxAHAAABy1qrX7+/Rhv3FElylvakJqgbpT3wAQRxAAAQsDKy8vRezs66/d9eM0JjKO2BjyCIAwCAgLQi75B+u7C+tGdaYoxmTIj14kTA8TwSxI0xO4wxtolfexqtHdDMWmuMmeeJmQAAQPA6WFyu2zNyVFltJUkj+kXqcUp74GNCPfhZRyT9xc3rxU2sXyXpfTevr/XYRAAAIOg4SntWaneD0p6XUxPVsT2lPfAtngzih621s05i/cqTXA8AAHBCz3y2Sd9urS/teW7GOMV17+zFiQD3uEYcAAAEjH+v26O/Lv6hbv8Xk8/SRUN7eXEioGmePCMeZoxJkxQnqUTSaklLrLXVTazvZ4z5uaTukg5K+t5au9qD8wAAgCCy40CJ7n9nVd3+xME9dffks7w4EdA8TwbxPpLeavTadmPMf1lrv3az/hLnrzrGmK8k3WKtzWvJFxpjsps4NLQl7wcAAIGhrrSnvL6057kbx1LaA5/mqUtTXpc0WY4wHi5plKS/SRogaZExZkyDtUCdZ/QAACAASURBVKWSHpeUKKmb89cFkhZLulDSF8aYcA/NBQAAApy1Vg//s0FpT2iIXklLpLQHPs8jZ8Sttb9t9NJaSbcaY4ol3S9plqTrnGv3SXqs0folxpgfSfpWUrKkn0p6rgXfm+judeeZ8oST+EsAAAB+Kj0zV/9cUV/a87urR2hUTJQXJwJapq1v1nzFuZ14ooXW2ipJr7Z0PQAAQE7eIf3uw/V1+zeOj9WMpDgvTgS0XFsH8X3ObUsvNdl/kusBAECQOlBcrtvT60t7RvaP1G+vGeHlqYCWa+sgfrZzu62F61NOcj0AAAhCVdU1+sXcFdpz1FHaE9WpPaU98DutDuLGmBHGmGg3r8dLetG5m97g9WRjjMvdE8aYSZLubbweAACgsT9/tlnf/XBQkmSM9JcZYxUbTWkP/IsnbtacJukhY8xiSdslFUk6U9KVkjpK+ljSnxqsny1phPNRhQXO10ZLmuT8/aPW2u88MBcAAAhAn67bo5e/qi/tuXvyWbpoCKU98D+eCOKLJQ2RNE6OS1HCJR2W4wkob0l6y1prG6x/S44nqEyQdLmk9pL2SnpH0ovW2m88MBMAAAhA2w+U6IEGpT0XDumpX0yitAf+qdVB3FnW466wp6n1r0l6rbXfCwAAgktpRZVufau+tCemWyf95caxCqG0B36qrW/WBAAAaDVrrf7fe2u0ae/xpT1dO1PaA/9FEAcAAD7vze9z9cHKXXX7T1wzUiP7U9oD/0YQBwAAPi07t1CPNyjtuSkpVtMnxHpxIsAzCOIAAMBn7S8q1+0ZOaqqcTz3YVT/KP1mCqU9CAwEcQAA4JOqqmt019wc7T1aLknq2rm9Xk5LoLQHAYMgDgAAfNLT/96kzG2FkhylPc/NGKeYbpT2IHAQxAEAgM/5ZO1u/e3rbXX79148WBcM7unFiQDPI4gDAACf8sP+Yj0wf3Xd/qShvXTnRYO8OBHQNgjiAADAZ5RWVOm29GwVO0t74qI769nplPYgMBHEAQCAT7DW6qF312jz3mJJUlhoiF5OS1BU5/ZengxoGwRxAADgE/7x3Q79a1WD0p5rR2pEP0p7ELgI4gAAwOuW7yjU7z/aULc/MzlO08ZT2oPARhAHAABeta+oTHfMqS/tGRMTpd9MGe7lqYC2RxAHAABeU1Vdo7vmrKgr7enWub1eSktUWCilPQh8BHEAAOA1f/x0k7K2H1/a079rJy9PBZweBHEAAOAVi9bs1v8sqS/tuf+SwZpIaQ+CCEEcAACcdlv3FeuB+avq9i8e1ku3X0hpD4ILQRwAAJxWJeVVujU9WyUV1ZIcpT1/prQHQYggDgAAThtrrX717mpt3Vdf2vNKWqKiOlHag+BDEAcAAKfN//5nhz5cvbtu/8nrRml4v0gvTgR4D0EcAACcFst2FOqpj+tLe1KT43RDYowXJwK8iyAOAADa3L6jZbo9o0FpT2xXPUZpD4IcQRwAALSpyuoa3TlnhfYXOUp7osM76OXUBEp7EPQI4gAAoE3NXrRRS3c4SntCjPT8jHHqR2kPQBAHAABt56PVu/Xqt9vr9u//0RCdd1YPL04E+A6COAAAaBNb9xXplwsalvb01m0XnOnFiQDfQhAHAAAeV1xepZ+/la1SZ2lPfPfO+vP0MZT2AA0QxAEAgEdZa/WrBav1w/4SSVLH9pT2AO4QxAEAgEe99u12fbSmvrTnqetHaVhfSnuAxgjiAADAY7K2HdRTizbW7d+cEq/rxlHaA7hDEAcAAB6x72iZ7py7QtXO0p5xcV316FWU9gBNIYgDAIBWq6yu0R1zcupKe7qHd9BLqQnqEErUAJrCPx0AAKDVnvp4o5btOCTJUdrzwk3j1DeK0h6gOQRxAADQKh+u3qX//U99ac8vLx2qcwZR2gOcCEEcAACcsi17i/TggtV1+z8a3lu3XjDQixMB/oMgDgAATklRWaV+nl5f2nNGj3D9afoYGUNpD9ASBHEAAHDSrLV6cMFqbXOW9nRq304vpyUosiOlPUBLEcQBAMBJe/Wb7Vq0dk/d/h9uGKWhfSjtAU4GQRwAAJyUzG0H9YdP6kt7bjk7XteM7e/FiQD/RBAHAAAttvdome6cU1/akxDXVb++ktIe4FQQxAEAQItUVNXo9owcHSh2lPb06NJBL6UmUtoDnCL+yQEAAC3y5McblJ1bX9rz/E3j1Ceqo5enAvwXQRwAAJzQByt36h/f7ajb/9VlQ3XOmZT2AK0R6okPMcbskBTfxOG91to+bt5zjqRHJKVI6ihpq6T/lfSCtbbaE3MBAIBTV1FVo8/W71VGVq6+++Fg3euXjeij/55IaQ/QWh4J4k5HJP3FzevFjV8wxlwj6V1JZZLellQoaYqkZyWdK2maB+cCAAAnoeBQqeYtzde8Zfl114PXGtgjXE9PG01pD+ABngzih621s060yBgTKenvkqolXWitXe58/VFJX0qaaoyZYa2d58HZAABAM6prrL7evE/pmXlavGmfrD3+eLsQo4uH9dJjU0YogtIewCM8GcRbaqqknpLerA3hkmStLTPGPCLpC0m3SSKIAwDQxvYVlemdZfmauzRfOw8fczneJ7KjZiTF6sYJseob1ckLEwKBy5NBPMwYkyYpTlKJpNWSlri53nuSc/uJm89YIqlU0jnGmDBrbbmbNQAAoBWstfp+20FlZObp03V7VFVjXdZMHNxTqclxmjy0l0Lb8WwHoC14Moj3kfRWo9e2G2P+y1r7dYPXhji3mxt/gLW2yhizXdIISQMlbWjuC40x2U0cGtqykQEACB6HSyu0ILtAc7LytO1Aicvx6PAOmjY+RjOT4hTfPdwLEwLBxVNB/HVJ30haJ6lIjhB9p6T/lrTIGHO2tXaVc22Uc3ukic+qfb2rh2YDACBoWWu1Iv+wMjLz9OHqXSqvqnFZkzQgWqkpcbpsZB+FhbbzwpRAcPJIELfW/rbRS2sl3WqMKZZ0v6RZkq5r4cfV3obt+udkrt+b6PYDHGfKE1r4fQAABJzi8ip9sHKn0jPztGH3UZfjEWGhuj6hv1JT4jW4d4QXJgTQ1jdrviJHEJ/Y4LXaM95RrsslSZGN1gEAgBbasPuo0jNz9f6KnSqpcK3lGNk/UmnJ8bp6bD917uCNZzYAqNXW/wTuc24bXmi2SdJ4SYMlHXeNtzEmVNIZkqokbWvj2QAACAhlldX6aPVuZWTlKifvsMvxju1DdPWYfkpLidfoGK78BHxFWwfxs53bhqH6S0mpki6TNLfR+omSOsvxtBWemAIAQDO27S/WnKw8Lcgp0OHSSpfjg3p1UVpynK5LiFFUJ579DfiaVgdxY8wISbuttYWNXo+X9KJzN73BoQWSZkuaYYx5oUGhT0dJTzjXvNzauQAACESV1fW18//ZetDlePt2RpeP7KvU5DglnRFNAybgwzxxRnyapIeMMYslbZfjqSlnSrpSUkdJH0v6U+1ia+1RY8zP5AjkXxlj5slRcX+1HI82XCBH7T0AAHDaefiY5i3N07xl+dpf5PqHxrHRnTQzKV7TxseoR5cwL0wI4GR5IogvliNAj5PjUpRwSYclfSvHc8Xfsvb4olxr7fvGmAsk/VrSDXIE9q2S7pP0fOP1AAAEo+oaqyWb9ysjK1dfbtynxr07IUaaPKy3UpPjNPGsngoJ4ew34E9aHcSdZT1fn3Ch6/v+I+mK1n4/AACBZn9Rud5Znq+5S/NUcMi1dr5XRJhmJMVpxoRY9etK7Tzgr3huEQAAPsBaq8xthUrPytW/1+1RZbXrHw6ff1YPR+38sN5qT+084PcI4gAAeNGR0kotyClQRlautu13rZ3v1rm9po+P1U1JcRrQg9p5IJAQxAEAOM2stVpVcETpmblauMp97fyEAd2Umhyvy0b2Ucf21M4DgYggDgDAaVJSXqUPVu5SRlau1u1yrZ3v4qydn5kcp6F9It18AoBAQhAHAKCNbdxzVBmZefrnip0qLq9yOT6iX6TSUuJ19Zh+Cg/jP81AsOCfdgAA2kBZZbUWrd2tjMw8Lc895HK8Y/sQTRldWzsfRfEOEIQI4gAAeND2AyWak5Wr+dnua+fP7Bmu1OR43ZAQo6jO1M4DwYwgDgBAK1VW1+jz9XuVkZWnb7cecDnevp3RpSP6KC0lXsnUzgNwIogDAHCKdjWond/npnY+plsnzUyO07TEWPWMoHYewPEI4gAAnISaGquvt+xXRmaevty4123t/KShvZWa4qidb0ftPIAmEMQBAGiBA8WO2vk5Wc3Uzk+I1Y1JcepP7TyAFiCIAwDQBGutsrYXKiMrT5+s3e22dv68QY7a+YuHUzsP4OQQxAEAaOTIsUq9l1OgjKw8bd1X7HK8a+f2mpYYo5uS4jSwZxcvTAggEBDEAQCQ4+z36tra+dW7VFbpWjufGN9NqclxumJUX2rnAbQaQRwAENRKK+pr59fudK2dD+/QTtcl9FdqcryG9aV2HoDnEMQBAEFp054iZWTl6p85O1XkpnZ+eF9n7fzYfupC7TyANsC/WQAAQaO8qlqL1uxRRlaulu1wrZ0PCw3RlDH9lJocp7GxXSneAdCmCOIAgIC340CJ5i7N0/zsAhWWVLgcH1hXO99fXTt38MKEAIIRQRwAEJCqqmv0+YZ9ysjK1TdbXGvnQ0OMLh3ZR2nJ8UoZSO08gNOPIA4ACCi7jxzTvKX5mrcsT3uPutbO9+/qrJ0fH6NeER29MCEAOBDEAQB+r6bG6putB5SemasvNrjWzhsjTRrSS2kp8Zo4mNp5AL6BIA4A8FsHiss1f3mB5izNVX6ha+18z9ra+QmxiunW2QsTAkDTCOIAAL9irdVSZ+38oiZq588d1F2pyfG6hNp5AD6MIA4A8AtHjlXqn87a+S1uauejOjlr55PjdCa18wD8AEEcAODTVhccVkZmnv61apeOVVa7HE+I66rU5HhdOZraeQD+hSAOAPA5pRVVWrhql9Iz87Rm5xGX4+Ed2unacY7a+eH9qJ0H4J8I4gAAn7F5b5EyMnP1XhO180P7RCgtJV7XjutP7TwAv8e/xQAAXlVeVa1P1u5RRmaelu4odDneITREV43uq7SUeI2jdh5AACGIAwC8Iu9gqTKW5mr+8iZq53uEa2ZynKYmxlA7DyAgEcQBAKdNVXWNvti4TxlZeVqyeb/L8dAQo0tH9FFqcpzOPrM7Z78BBDSCOACgze05UqZ5y/I0b2m+9hwtcznev2sn3ZQUq+njY9Urktp5AMGBIA4AaBM1NVbfbj2gjKxcfb5hn6ob9c4bI100pJdSk+N04ZBe1M4DCDoEcQCARx0sLteC7ALNWZqn3IOlLsd7dOmgGyfEasaEOMVGUzsPIHgRxAEArWat1fLcQ0rPzNWiNXtUUV3jsubsgd2VluKone8QSu08ABDEAQCn7GhZpd5fsVMZmXnatLfI5XhUp/aamhijm5LiNKgXtfMA0BBBHABw0tYUHFFGVq4+WOm+dn6cs3b+KmrnAaBJBHEAQIscq6jWwlW7lJGVq1UFrrXznZ218zOT4jSyf5QXJgQA/0IQBwA0a8veImVk5endnAIVlbmvnU9Nide1Y/spomN7L0wIAP6JIA4AcFFeVa1P1+1Vemaulm5vonZ+VF+lpsQpIa4bxTsAcAoI4gCAOvmFpcrIytP85fk66KZ2fkD3zkpNjtfUxBh1C6d2HgBagyAOAEGuqrpGX9bWzm/ZL3t8747ahRj9aHhvpaXE6+yB3RVC8Q4AeARBHACC1N6jZZq3NF/zluVp9xHX2vl+UR11U1Kcpk+IVW9q5wHA49osiBtjbpb0pnP3Z9baVxscu1DS4mbePtta+1BbzQYAwaqmxuo/PxxQRmaePtuw123t/IWDeyo1OV4XDump0HYU7wBAW2mTIG6MiZX0gqRiSc01OHwt6Ss3r3/bBmMBQNAqLKnQgux8zcnK044mauenj4/VTUnUzgPA6eLxIG4ct86/LumgpPckPdDM8q+stbM8PQMAwFE7n517SBlZefpozW5VVLnWzqcMjFZqcrwuHdGH2nkAOM3a4oz4LyRNknShcwsAOI2Kamvns/K0cY9r7Xxkx1DdkBij1OQ4DeoV4YUJAQCSh4O4MWaYpD9Ies5au8QYc6IgPsgYc6ekSEl7JH1jrd3iyZkAIFis3VlfO19a4Vo7Pya2q1KT4zRldD916kDtPAB4m8eCuDEmVNJbkvIkPdzCt6U6fzX8nHfluLnzUAu+M7uJQ0Nb+P0A4NeOVVTrw9W7lJ6Vp1X5h12Od2rfTteO66fU5Hhq5wHAx3jyjPhjksZJOs9ae+wEa/dLekjSR5J2SOooabykJyXdIKmPMWaitdb1gkYAgLbuK1ZGVq7ezS7QUTe180N6RygtJU7XjOuvSGrnAcAneSSIG2OS5DgL/mdr7fcnWm+tXSdpXYOXiiV9Yoz5TtJKSedKmiLpgxN8TmIT82RLSmjZ9ADgHyqqavTpuj3KyMpV5jY3tfPtQnTl6L5KTY5TYjy18wDg61odxBtckrJZ0qOt+Sxr7VFjzBxJv5Y0UScI4gAQDPILSzV3aZ7eWZ6vA8WutfPx3TsrNTlOUxNjFU3tPAD4DU+cEe8iabDz92VNnIH5uzHm73LcxHnPCT5vv3Mb7oHZAMAvVddYLd64T+lZufp6s/va+UuG9VZqSpzOPbMHtfMA4Ic8EcTLJb3WxLEEOa4b/1bSJkknvGxFUopzu631owGAf9l3tEzzluVr3tI87XJTO983qqNmTIjTjCRq5wHA37U6iDtvzPypu2PGmFlyBPE3GlXcnyvp+8Y3Yxpj0iTdKKlC0jutnQ0A/EFNjdV3PxxURlauPlu/V1VuaucnntVTaSnxuojaeQAIGG1Scd8CGZJCnDdnFsjx1JQJkpIkVUn6ubV2h5dmA4DT4lBJhRZkF2jO0jxtP1Dicrx7eAdNnxCrmybEKa47tfMAEGi8FcRflnSxHE9H6SHJSNop6R+S/mKtXeWluQCgTVlrlZN3SBmZefqwidr55DOilZoSr0tH9FZYKMU7ABCo2jSIW2tnSZrl5vXZkma35XcDgC8pKqvU+yt3KSMz123tfETHUN2Q4KidP6s3tfMAEAy8dUYcAILCul1HlJGVpw9W7FSJu9r5mCilJsfrqjF91bkD/0oGgGDCv/UBwMPKKqv14erdSs/M1comauevGeuonR8VQ+08AAQrgjgAeMgP+4uVkZmnd3MKdORYpcvxwb27KDU5XtclUDsPACCIA0CrVFTV6LP1e5Wemavvtx10Od6hXYguH9VHaSnxGk/tPACgAYI4AJyCgkOO2vm3lxXoQHG5y/G46Nra+Rh17xLmhQkBAL6OIA4ALVRdY/XVpn3KyMrT4k373NbOTx7aS2kp8TpvELXzAIDmEcQB4AT2FZXpnWX5mrs0XzsPH3M53ieyo2YkxerGCbHqG9XJCxMCAPwRQRwA3LDW6vsfDiojK0+frtvjUjsvSRMH91RqcpwmD+1F7TwA4KQRxAGggcOlztr5rDxtc1M7Hx3eQdPGx2hmUpziu4d7YUIAQKAgiAMIeo7a+cPKyMrVh6vd184nDYhWakqcLhvZh9p5AIBHEMQBBK3i8iq9v2KnMrLytGH3UZfjEWGhuiExRjOT4zSY2nkAgIcRxAEEnfW7jiojK1fvN1E7P6p/lNJS4jRlTD9q5wEAbYb/wgAICmWV1fpo9W5lZOUqJ8+1dr5j+xBdM6a/UlPiNDqmqxcmBAAEG4I4gIC2bX+x5mTlaUFOgQ6XutbOn9Wri1KT43RdQoyiOlE7DwA4fQjiAAJOZXV97fx3P7jWzrdvZ3T5yL5KTY5T0hnR1M4DALyCIA4gYOw8fExzs/L09vJ87S9yrZ2Pje6kmUnxmjY+Rj2onQcAeBlBHIBfq66xWrJ5v9Izc7V40z417t0JMdLkYb2VlhKv86mdBwD4EII4AL+0v6hc7yzP15ysPLe1870jwzRjQpxmJFE7DwDwTQRxAH7DWqvvtzlr59e6r50//6weSk2O1+RhvdSe2nkAgA8jiAPweUdKK7Ugp0AZWbnatt+1dr5b5/aaPj5WNyXFaUAPaucBAP6BIA7AJ1lrtTL/sDKy8rRw1S6Vu6mdnzCgm1KT43XZyD7q2J7aeQCAfyGIA/ApJeVV+mDlLmVk5WrdLtfa+S5hobo+ob9mJsdpaJ9IL0wIAIBnEMQB+ISNe44qIzNP/1yxU8XlVS7HR/SLVFpKvK4e00/hYfyrCwDg//ivGQCvKaus1sdrdisjK0/ZuYdcjndsH6Ipo/spLSVeo2OiKN4BAAQUgjiA0277gRLNycrV/Gz3tfODnLXz14+LUVRnaucBAIGJIA7gtKisrtHn6/cqIytP32494HK8fTujy5y188nUzgMAggBBHECb2nX4mOYtzdO8Zfna56Z2PqZbJ81MjtO0xFj1jKB2HgAQPAjiADyuusZqyZb9ysjM1Zcb3dfOTxraW6kpcbrgrJ7UzgMAghJBHIDH1NbOz12ap4JDrrXzvSLCNGNCrGYkxalfV2rnAQDBjSAOoFWstcraXqj0zFx9um6PKqtda+fPG9RDaSlxmjysN7XzAAA4EcQBnJIjpZV611k7/4Ob2vmundtrWmKMZibH6wxq5wEAcEEQB9Bi1lqtKjiijMxcLVy9S2WVrrXzifHdlJYSp8tH9qV2HgCAZhDEAZxQSXmV/rXKUTu/dqf72vnrxjlq54f1pXYeAICWIIgDaNKmPUXKyMrVP3N2qshN7fzwvs7a+bH91IXaeQAATgr/5QRwnLLKan2ydo/SM3O13E3tfFhoiKaM6afU5DiNje1K8Q4AAKeIIA5AkrTjQInmLs3TO8vzdchN7fzAnuFKTY7X1ARq5wEA8ASCOBDEqqpr9PmGfcrIytU3W1xr50NDjC4d2UdpyfFKGUjtPAAAnkQQB4LQ7iPHNHdpvt5elqe9R11r5/t3ddbOj49Rr4iOXpgQAIDARxAHgkRNbe18Vp6+2LDXpXbeGGnSkF5KS4nXxME91Y7aeQAA2hRBHAhwB4rLNX95geYszVV+oWvtfE9n7fyNE2IV062zFyYEACA4EcSBAGSt1dLthUrPytMna3e7rZ0/d1B3pSbH65Lh1M4DAOANBHEggBw5Vqn3cgqUkZWnrfuKXY5HdaqtnY/TwJ5dvDAhAACoRRAHAsDqgsNKz8zVv1a5r51PiOuqtJR4XTGK2nkAAHxFmwRxY8zNkt507v7MWvuqmzVXSXpA0jhJ7SStk/SStfaNtpgJCDSlFVX618pdysjK05qdR1yOh3dop2vH9VdqcryG96N2HgAAX+PxIG6MiZX0gqRiSW7/7NsYc6dzzUFJ6ZIqJE2V9A9jzChr7QOengsIFJv3FikjM1fvNVE7P6xvpNJS4nTN2P7UzgMA4MM8+l9p42j7eF2OgP2eHGe8G68ZIOlPkgoljbfW7nC+/jtJyyTdb4x511r7vSdnA/xZeZWjdj4jM09LdxS6HA8LDdFVo/spNSVO46idBwDAL3j6dNkvJE2SdKFz685PJIVJml0bwiXJWnvIGPOkpNck3SqJII6gl3uwRHOW5mn+8gIVllS4HB/YI1wzk+M0NTFGXTt38MKEAADgVHksiBtjhkn6g6TnrLVLjDFNBfHa1z9xc2xRozVA0KmqrtEXG/cpPbOZ2vkRfZSaHKezz+zO2W8AAPyUR4K4MSZU0luS8iQ9fILlQ5zbzY0PWGt3G2NKJMUYYzpba0tP8L3ZTRwaeoIZAJ+z50iZ5i3L07yl+dpztMzleP+unXRTUqymT4ildh4AgADgqTPij8nx9JPzrLWu1X3Hi3JuXR/zUP96uHNds0Ec8Hc1NVbfbj2g9MxcfbFxn6ob9c4bI100pJdSk+N04ZBe1M4DABBAWh3EjTFJcpwF/7OHbrCsTRquVYCNWGsTm5gpW1KCB2YB2sTB4nLNzy7QnKw85RW6/v/NHl3CdOOEGN2UFEftPAAAAapVQbzBJSmbJT3awrcdkdRDjjPeB90cr33g8dHWzAb4Gmutlu04pIysXC1as0cV1a7FO2cP7K60FEftfIdQaucBAAhkrT0j3kXSYOfvy5q4aezvxpi/y3ET5z2SNskRxAer0ZNRjDF95bgspeBE14cD/uJoWaX+mbNTGVm52rzXfe381ETH2e9BvaidBwAgWLQ2iJfL8bhBdxLkuG78WznCd23o/lLSuZIuk+sjCi9vsAbwa2sKjigjK1cfrNylY5XVLsfHxXVVanK8rhpN7TwAAMGoVUHceWPmT90dM8bMkiOIv9Go4v51SQ9KutMY83qDQp9uqn/iyiutmQvwltKKKi1c5aidX13gej9yZ2ft/MykOI3sH+XmEwAAQLA47f3X1trtxphfSnpe0nJjzNuqr7iPkedu+gROmy17i5SRlad3cwpUVOZaOz+0T4RSU+J17dh+iujY3gsTAgAAX3Pag7gkWWtfMMbskPSApP8jKUTSekmPWGvf8MZMwMmqq53PytPS7a618x1CQ3TVqL5KTYlXQhy18wAA4HhtFsSttbMkzWrm+EJJC9vq+/9/e3ceZlV15nv8+zIjAjLITBVqRIkoKkKVqFFMYrQjDnFCC6+m23SSm2s0N+nbSW5M45PufpKnk2ti7NuZTLgtpTi1GhPHqKhRKREBJ5AoUFWMhUwyD8V7/1ir9OTUrrlO7VOnfp/n2c+Gtfc6e+911j71nn3W3q9IrlRt3h3TzlezOSHt/FFD+1FWUsRlp45hUD+lnRcREZFkqVwRF+lsDtYe4tnlNZRXVPHCXzbhWU+579HNOO+E4ZSVFHP60UPopsQ7IiIi0gQF4iKN2LB9L/curGbewirWb6+fdn7UwD5cPbWIq6aMZdgApZ0XERGR5lMgLpLl0CHnpfdD2vk/LUtOO3/O+CMpKylm+vFKOy8iIiKto0BcJNqyaz8PLKrm9FfZnQAAHDRJREFU7ooqVm9OSjvfiytPG8vVU4sYO1hp50VERKRtFIhLl+buLKrcytwFlTzWQNr50qMHU1ZSzOdOGKG08yIiItJuFIhLl7Rj7wEeWryW8gVVvLtxR73lA/r04LLJYygrKVbaeREREckJBeLSpby19uO087v31087P2nsEcwqKeLCk0bRt5fSzouIiEjuKBCXgrdnfy2PvhHSzi+t3lZv+WG9unPxyaMpK1HaeREREek4CsSlYL1XE9POL1rDhwlp548b3p9ZpUVcfMpoBijtvIiIiHQwBeJSUPYfPMSTb2+gvKKSBSsT0s5378bnTxpJWUkRk4sHKe28iIiIpEaBuBSE6i27uefVKu57rZoPdtZPO1885DDKSoq4fPJYBivtvIiIiOQBBeLSadUe8ph2vpLnV9RPO9+9m/HZCcMpKy3ijGOGKu28iIiI5BUF4tLpbPwwpp1/tYp1CWnnRw7sw8wpRcycOpbhSjsvIiIieUqBuHQKhw45L7+/mfKKSp56Z2Ni2vmz69LOH3ckPbor8Y6IiIjkNwXikte27trPA4vWcPerVaz6YFe95UP69eLKKWO5ekoRRUOUdl5EREQ6DwXiknfcndertjJ3QRV/fHM9+w/WTztfctRgykqL+dwJw+ndQ4l3REREpPNRIC55Y8feAzy8ZB3lCypZvqF+2vn+fXpw2aljmFVaxCeG9U9hD0VERETajwJxSd3b67Yzd0EVjyxZm5x2fsxAykqKmTFJaedFRESkcCgQl1TsPVDLo0tD2vklCWnn+/bszsUnj6KspJgTxyjtvIiIiBQeBeLSod6r2cndFVU8sKg6Me38+OGHM6u0mEuUdl5EREQKnAJxybn9Bw/x1DsbKF9QxSsrN9db3qt7N/7mxBGUlRZzmtLOi4iISBehQFxypnrLbuYtrOLehWv4YOe+esuLhxzGNVOLuHzyGIYc3juFPRQRERFJjwJxaVe1h5z579ZQXlHFc+/WJKad/8yEYZSVFHPmJ5R2XkRERLouBeLSLmp27OW+hdXc82o1a7ftqbd8xIA+zJw6lplTihgxUGnnRURERBSIS6u5O6+8v5m5FZU89fZGDmalnQf41PgjKSsp4tPHD1PaeREREZEMCsSlxbbu2s+Dr6/h7ooqViaknR/crxdXnDaGa6YWUTykXwp7KCIiIpL/FIhLs4S089sor6jkD28kp52fOm4wZaVFnD9xhNLOi4iIiDRBgbg0aue+gzy8eC3lFVUsW/9hveX9e/fgssljuKakiPHDlXZeREREpLkUiEuid9Z9yNyKSh5ZvJZdCWnnTxw9kFmlRcyYNIrDeqkbiYiIiLSUIij5yN4DtfzxjfXMrahkcVX9tPN9enbj4kmjKSst4qQxR6SwhyIiIiKFQ4G4sHLTTsorqnhg0Rq27zlQb/mxww6nrKSIS08dw8C+SjsvIiIi0h4UiHdRB2oP8dTbGymvqOTl9+unne/Z3bhg4khmlRYzZZzSzouIiIi0NwXiXcyarbuZ92o1975WzaYd9dPOFw0+jGtKQtr5oUo7LyIiIpIzCsS7gNpDzvMraihfENLOZ+fd6WbwmQnDKSst5iylnRcRERHpEArEC1jNjr3c/1pIvJOUdn74gN7MnFLEzKljGTmwbwp7KCIiItJ1KRAvMO7OKys3U15RxZNvbUhMO3/WsUMpKynmMxOUdl5EREQkLQrEC8S23ft58PW1lFdUsnJT/bTzgw7ryZWnjeXqqUWMG6q08yIiIiJpUyDeibk7S6q3MXdBFX94Yx37EtLOTxk3iLKSYs6fOII+PZV2XkRERCRfKBDvhHbtO8jDS9ZSvqCKdxLSzh/euwdfOHU0ZSXFHDdCaedFRERE8pEC8U5k2foPKa+o5OHF69i572C95RNHD2BWSTEzJo2iX2+9tSIiIiL5TNFantt7oJbH3lxPeUUViyq31lvep2c3Lpo0irKSYk4aM1CJd0REREQ6CQXieWrVB7u4u6KS+xetYdvu+mnnPxHTzn/hlDEMPExp50VEREQ6m3YJxM3sR8BpwHhgKLAHqAQeBu5w980Z644DVjXycve6+8z22K/O5kDtIf70zkbmVlTy0nvJaefPnziSspIiSo4arKvfIiIiIp1Ye10R/wbwOvA0UAP0A0qB2cDfm1mpu1dn1VlKCNSzvdVO+9RprN22h3mvVjFvYXLa+TGD+nJNSRFXTB7Lkf2Vdl5ERESkELRXID7A3fdmF5rZvwDfBb4D/PesxUvcfXY7bb/TqT3kvLBiE+UVlTy7PDnt/LnHD6estIizjz1SaedFRERECky7BOJJQXh0HyEQP7Y9tlMINu3Yx32vVXPPq1Ws2Vo/7fyw/r2ZObWImVPGMuoIpZ0XERERKVS5vllzRpy/kbBslJl9GRgCbAZecfek9To9d2fByi2UV1Ty5NsbOFDbUNr5Ij49YTg9lXZeREREpOC1ayBuZt8CDgcGEm7ePJMQhP8wYfXPximz/nzgOnevaub2FjWw6Phm7nJObd99gAdfX0N5RSXvN5B2/oqYdv4opZ0XERER6VLa+4r4t4DhGf9/Arje3TdllO0GfkC4UXNlLDuJcGPndOAZMzvZ3etHrp2Au7N0zXbKF1Ty6Bvr2Hugftr504oHUVZaxAUTRyrtvIiIiEgX1a6BuLuPADCz4cA0wpXwxWZ2obu/HtepAb6fVfUFMzsP+DNQAtwA/KwZ25ucVB6vlJ/a2uNojV37DvL7peuYu6CSt9clp52/9JTRXFNSxISRAzpy10REREQkD+VkjLi7bwQeMrPXgRXAfwITm6hz0Mx+QwjEP0UzAvF8smDlZr7zX2/WK//kyAHMKi3m4pOVdl5EREREPpbTyNDdK83sHeBkMxvq7h80UaVuCEunGzB9znHDGDWwD+u276V3j27MmDSKWaXFTFLaeRERERFJ0BGXaEfFeW0z1i2N85WNrpWHunczvv7pY9m1v5bLT1XaeRERERFpXJsDcTM7Htjm7huyyrsRbsocBrzs7ltjeQmw2N33Z61/LiFDJ8Dctu5XGmZOLUp7F0RERESkk2iPK+LnA/9mZi8A7xOeCT4cOBs4GtgAfClj/R8BJ8RHFa6JZScB58Z/3+LuL7fDfomIiIiI5K32CMT/BPwKOAOYBBwB7CLcpHkXcLu7b8lY/y7gUmAKcAHQE9hIyMJ5h7u/2A77JCIiIiKS19ociLv7W8DXWrD+ncCdbd2uiIiIiEhnplzqIiIiIiIpUCAuIiIiIpICBeIiIiIiIilQIC4iIiIikgIF4iIiIiIiKVAgLiIiIiKSAgXiIiIiIiIpUCAuIiIiIpICBeIiIiIiIilQIC4iIiIikgIF4iIiIiIiKVAgLiIiIiKSAgXiIiIiIiIpMHdPex/anZlt7tu37+AJEyakvSsiIiIiUsCWLVvGnj17trj7kJbWLdRAfBUwAFjdwZs+Ps6Xd/B2uwq1b+6obXNL7Zs7atvcUvvmjto2tzqyfccBH7r7US2tWJCBeFrMbBGAu09Oe18Kkdo3d9S2uaX2zR21bW6pfXNHbZtbnaV9NUZcRERERCQFCsRFRERERFKgQFxEREREJAUKxEVEREREUqBAXEREREQkBXpqioiIiIhICnRFXEREREQkBQrERURERERSoEBcRERERCQFCsRFRERERFKgQFxEREREJAUKxEVEREREUqBAXEREREQkBQrE24GZjTGz35rZOjPbZ2arzeynZjYo7X3LF2Y2xMxuMLOHzOw9M9tjZtvN7M9m9ndm1i1r/XFm5o1M8xrZ1nVm9qqZ7YzbmG9mF+b+KNMV+11D7bWhgTrTzOwxM9tiZrvN7A0zu9nMujeynQtjm26PbVxhZtfl7sjSZWbXN9EX3cxqM9ZX301gZpeb2c/N7EUz+zC2xdwm6nRI/+zs7d6StjWzY83sH83sWTOrNrP9ZrbRzB4xs+kN1GnqHPhKA/X6mtmtZvaume01sxozu8/MJrTn8edaC9u3w85/M+sez4c3LPxN3RLPl2ntcdwdoYVtO6cZn8XPZNXJ+77bo71fsKsxs2OAl4FhwCPAcmAqcBNwvpmd4e6bU9zFfHEF8B/AeuA5oAoYDnwB+A1wgZld4fUzTC0FHk54vbeSNmJmPwa+CawBfg30AmYCj5rZje5+RzscSz7bDvw0oXxndoGZXQw8COwF7gW2ADOA24AzCO9Zdp3/Afwc2AzMBfYDlwNzzOxEd/9W+xxGXlkC3NrAsrOAc4HHE5ap7/617wGTCH1xDXB8Yyt3VP8skHZvSdv+ALgKeAd4jNCuxwEXAReZ2U3ufnsDdR8hnA/ZXssuMLPewNOE9+o14GfAWML79nkzO9fdK5o+tLzQor4b5fT8NzMD5hH697vAHcBgwnv7gpld5u6PNGM/09aStn0YWN3AsmuBo0n+LIZ87rvurqkNE/Ak4MCNWeX/J5b/Iu19zIeJEKzMALpllY8gBOUOXJZRPi6WzWnBNqbFOu8Bg7JeazPhD/q4tNsih228GljdzHUHADXAPuC0jPI+hC+WDszMqjMutuHmzHYEBsU2d+D0tNuhg9v8lXjcF2W1k/pu/WOcDhwLGHBOPN65DazbIf2zUNq9hW17PXBKQvnZhC8u+4CRCXUcuL4F+/SdWOd+Mj73gYtj+dtk/T3I16mF7dsh5z9wdazzEtAno3xKfA9rgP5pt117tm0jr3EEsDse99CsZXnfdzU0pQ3M7GjgPEIA9O9Zi/8J2AVca2b9OnjX8o67P+vuj7r7oazyDcAv4n/PaeNm6n5i+hd335qxjdWE96c38MU2bqNQXA4cCcxz94+uCLj7XsIVCoCvZtX5W0Ib3hHbtK7OVuBf438Tf+YrRGY2ESgF1gJ/bOPLFXzfdffn3P0vHv+iNaGj+mdBtHtL2tbd57j74oTy54H5hCuxbRraEK/W1rXt/8r83PdwlfZF4JOE4D/vtbDvtkZr+mFd//9ePC/q6iwk/IJ0JOE8ymvt1LbXAn2B/3L3D9qyP2n0XQXibXNunD+VEGDuIHxTPYzwx1oadiDODyYsG2VmXzaz78b5SY28Tt378UTCssez1ilUvc1sVmyvm8xsuiWPp22srV4gXF2YFn+ia06drtK+mb4c53e6e23CcvXd1uuo/ql2/2uNfRYDnBzHJH/bzK41szENrHcMUASscPdVCcu7Qtvm7PyP/X4a4Tx4sTl1CtyX4vxXjayTt31XY8Tb5rg4X9HA8r8QrpiPB55pYJ0uzcx6AP8t/jfpQ+izccqsMx+4zt2rMsr6AaOBne6+PuF1/hLn49u6z3luBHBXVtkqM/tivOJVp8G+6+4HzWwVcAJhzN2yZtRZb2a7gDFmdpi7727LQeQ7M+sLzAIOEe5xSKK+23o5759q979mZsXApwnB3QsNrHZT1v9rzew3wM2ZV2Vp3t9GKOy2zeX5/wmgO7DS3ZO+NHWF9gXAzE4HTiQEzs81smre9l1dEW+bgXG+vYHldeVHdMC+dFY/BCYCj7n7kxnluwk3FU0mjPEcRPgp6DnCEJZnsob86L2A3xH+kI4A+hE+nH5JGGf4uJlNyli3Ne3V3DoDG1heSK4ktM3j7l6dtUx9t+06on+q3aN4hbWcMARidubwiGgVcCMhSOkHjCKcA6sJvwz9Nmv9rty2HXH+d+X2zfb3cf7rBpbnfd9VIJ5bFue5GlfWqZnZ1wl3iS8njPH6iLvXuPv33f11d98WpxcIvzBUEK4I3NCKzRbse+Hut8ax+Bvdfbe7v+XuXyHcONwXmN2Cl2tN3+1K/b3uw/+X2QvUdztER/bPgm73OHTtLsITIu4Ffpy9jrs/7+53uPuK+Nmy3t3vJ9xotxW4OuuLfpObrXvpNu5+3smT879g2zeTmQ0kBNX7gTlJ63SGvqtAvG2augI4IGs9iczsa4RHAr0DTHf3Lc2pF3+GqxsK8KmMRU29F019yy1kdTfDtqS9kvpuc+t82KK962TM7JOE8ZlrCI9/axb13RbpiP7Z5ds9BuFzCY9luw+Y1ZKb5uKvQXXnQFs/XwpaO5//at9gFuE+vBbfpJlPfVeBeNu8G+cNjRU6Ns4bGmvUJZnZzYRnnr5FCMITk800YlOcf/TznrvvIjy94nAzG5lQpyu/FzVxnvlzaIN9N47bP4pww9bKZtYZGV9/TaGPD6fpmzQbo77bPDnvn1293WM73kN4VvXdwDUNjDduSr0+jf42NqS9zv/3gFrg6Pg+NqdOIaq7SbPeL5PNlBd9V4F429TdGHCe1c8M2Z/wU98eYEFH71i+MrN/JCTkWEIIwmuaqJKk7ik0K7PKn43z8xPqXJC1TldyepxntldjbfUpwlWGl919XzPrdIn2NbM+hGFUh4A7W/ES6rvN01H9s0u2u5n1Ah4gXAn/T+DaVnyprFMS55l9+n1CfojxZnZUQp2CbdsmtMv5H/v9y4Tz4Kzm1Ck0ZlZCSAS0wt3nt/Jl8qPveh480L0zTyihT0va6pbYJq8Bg5tYtwTolVB+LiG5gQPTspYVRHKOVrbtCUltChQT7vJ24LsZ5QMIVwNakjDlKLp4Qh9CEO7Ao42so77bdDueQ9MJfXLePwux3ZvRtr0Jz713wlCJJhOTAGcllBkfJz7ZBAzIWl4wCX1a2L4dcv7TvIQ+A1pzjPnatlnr3hnX/WYT6+V937X44tJKCSnulxFOxOmEny6muVLcY2bXEW6mqCWkoE4aX7Xa3efE9ecTgsv5hLG4ACfx8bM7b3H3f07Yzk+A/xnrPEBITnEVMITwZakzpKtuMTObDXyb8CvNKmAH4XmonycEL48Bl7r7/ow6lxDaaC8hVfIWQprr42L5lZ71AWFmNwK3E/443MvHKcTHAD/xwkxx/xEzexE4k5BJ89EG1pmP+m49sb9dEv87Avgc4UpU3XOQP8jsPx3VPwuh3VvStmb2O0K2wQ+A/0vyTWfzPeMqo5k54e/ZQsIwioGEX3wnEp4Scqm7P5W1T70JVw2nES6+PEN4PvMVhPel06S4b2H7zqcDzv+YeOY+Qv9eDjwa172K8JnfKVLct/RzIdYZAKwDegKjvZHx4Z2i76b9DagQJmAs4dFx6+ObVEm4EbHRq75daSI8scObmOZnrP93wB8IjxjaSfiGX0X441rvG27Wtq6LJ90uQkD6PHBh2m2Q4/Y9mzDeczmwjZCYYxPwNOE57dZAvTMIQfpWwjCqN4FvAN0b2daM2KY7YhsvJDwbN/V2yHEbT4j9tLqJ9lHfTT62pj4DVifU6ZD+2dnbvSVtSwgQm/osnp31+v8W22Qd4YvR7vhZcwdwdCP71Re4lfCr3L74mXQ/8Mm02yyH7dth5z8hF8w34nmxJ54nj5F1xT2fp1Z+Lnw1LrunGa+f931XV8RFRERERFKgmzVFRERERFKgQFxEREREJAUKxEVEREREUqBAXEREREQkBQrERURERERSoEBcRERERCQFCsRFRERERFKgQFxEREREJAUKxEVEREREUqBAXEREREQkBQrERURERERSoEBcRCQHzOwcM3Mzm532voiISH5SIC4i0kpmNi4G23PS3pf2ZmbXx2O7Pu19EREpVD3S3gERkQL1KjAB+CDtHRERkfykQFxEJAfcfTewPO39EBGR/KWhKSIirRDHfq+K/70uDuOom65vaIy4mc2P5T3N7Ptm9r6Z7TWz5Wb2pYz1vmJmb5rZHjNbY2a3mlniZ7aZlZjZA2a2wcz2m1m1mf3SzEYlrHu0mf3KzN6Lr70lbucXZjakbh+B38Uqv8s6tnFxnVFx/1/K2O46M7vbzCYkbPejYTxmdkzc381mtsPMnjKziXG9I+P+rY/tstDMpie1f3y9c8zsOjNbHI+nxsx+a2YjmngLRURSpyviIiKtMx84ArgJWAo8nLFsSVzWmHlACfAYcAC4HPiVmR0ATgKuA/4APANcBHwf2A38KPNFzOyLwK+BfcDvgWrgWOAGYIaZlbp7VVx3JLAQGBC3+yDQBzgKuBa4A9gMzAG2ARcDj8TjqbMtzj8FfBt4Lr7Ozrjdy4GLzOwMd1+acNzjgApgWdzOOOBSYL6ZnQ48AXwI3AsMBmYCj5vZ+LrjyPIN4Ly4/hPAmcAXgXPMrMTdNyXUERHJD+6uSZMmTZpaMRGCSAfmJCw7Jy6bnVU+P5YvBI7IKD8a2A9sJVxpH52x7AjCWPNNQI+M8vGxznuZ68dl5wK1wEMZZTfGbd+UsL/9gL4Z/78+rnt9A8c+DOifUD6JEJQ/3kBbOfC/s5bdEsu3AL8AumUsuzYuuy2rzuxYvh84JWvZbXHZnWn3EU2aNGlqbNLQFBGRdHzb3euuLuPuK4E/E4LuH7j72oxl24BHgaHA6IzX+CrQkxBYr80ox92fJVwhn2Fm/bO2vSd7Z9x9l7vXK2+Iu9e4+46E8qXAs8B0M+uZUHU18MOssv8X572Bf3D3QxnL7gYOAic3sCt3ufvirLLZwHbgGjPr3dhxiIikSUNTRETS8VpC2bo4X5SwrC7QHgNUxn+fHudnm9mUhDrDgO6EK+eLCIH5vwL/bmafA54EXgLecXdv6QGY2eeBrwCnEb4kZP9NGQqszypb4u61WWV1x70iO7h391oz20g47iTPZxe4+3YzWwKcTXhyzZJ6tURE8oACcRGRFLj79oTig3He2LLMq8xD4vwfmtjc4XGblWY2lXDF+HzgC3F5tZn92N1vb2q/65jZ14GfEYbSPA1UEcawO3AJYYhK0tXoesfm7gfNLHFZdJC/Pu5MGxso3xDnAxtYLiKSOgXiIiKdV13gOtDdP2xOBXdfBlxlZj0IwfJnCGPHf2Zmu9z9zqZeI9a9lRDsnuru67OWn55YMTeGN1Be99SUhoJ7EZHUaYy4iEjr1Q2x6J7S9hfE+VktrejuB919kbv/CLg6Fl+SsUpjxzaUMJb95YQg/HDg1JbuTxucnV1gZgMJY8r3Ep7OIiKSlxSIi4i03lbCUIyilLZ/B+HRh7eZ2fjshWbWy8zOyvj/VDNLuoJcV7Y7o2xznCcdW01cd3IMvOtevydhuMrQFh1F21xrZqdklc0mDEm5x933deC+iIi0iIamiIi0krvvNLMK4CwzKwdWEK4k/76Dtr/czP4W+C3wtpk9EfehJyGAPovwyMPjY5VrgK+Z2fOERx5uBY4BZhCeQ/7TjJd/hRBs32xmg/l4LPbP482QtxOeI/6mmT0C9AKmE579/Vz8d0d4HHjJzO4j3Bh6ZpxWx/0TEclbCsRFRNrmWsJzq88nDPEwYA0hEMw5d59rZkuBbxKC3/OAXYQnkTxASHRT5x7CDZTTCMNH+hKexjIP+Im7v5XxulvN7DLgnwgJcvrFRXMJ465vIQT5NwBfjmVPA98jjB/vKLcBDwE3A1cRnmE+B/iuu9d04H6IiLSYteKJVSIiIqkys9mELwnT3X1+unsjItI6GiMuIiIiIpICBeIiIiIiIilQIC4iIiIikgKNERcRERERSYGuiIuIiIiIpECBuIiIiIhIChSIi4iIiIikQIG4iIiIiEgKFIiLiIiIiKRAgbiIiIiISAoUiIuIiIiIpECBuIiIiIhIChSIi4iIiIikQIG4iIiIiEgKFIiLiIiIiKRAgbiIiIiISAoUiIuIiIiIpOD/A9YOh1EPpewVAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 261,
       "width": 369
      },
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_metrics.query(\"metric_name=='validation:accuracy'\").plot(x='timestamp', y='value')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='c2w2-2.2.'></a>\n",
    "### 2.2. Analyze Debugger results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can now explore the debugger output data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<b>Review <a target=\"blank\" href=\"https://s3.console.aws.amazon.com/s3/buckets/sagemaker-us-east-1-645012351640?prefix=pytorch-training-2021-09-14-18-16-16-595/\">S3 debugger output data</a></b>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "\n",
    "display(\n",
    "    HTML(\n",
    "        '<b>Review <a target=\"blank\" href=\"https://s3.console.aws.amazon.com/s3/buckets/{}?prefix={}/\">S3 debugger output data</a></b>'.format(\n",
    "            bucket, training_job_name\n",
    "        )\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='c2w2-2.3.'></a>\n",
    "### 2.3. Download SageMaker debugger profiling report"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can download and review the debugger profiling report. Here is the path in the S3 bucket:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "profiler_report_s3_uri = \"s3://{}/{}/rule-output/ProfilerReport/profiler-output\".format(bucket, training_job_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can list the report files:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                           PRE profiler-reports/\n",
      "2021-09-14 18:57:04     364692 profiler-report.html\n",
      "2021-09-14 18:57:04     211748 profiler-report.ipynb\n"
     ]
    }
   ],
   "source": [
    "!aws s3 ls $profiler_report_s3_uri/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The folder `profiler-reports` contains the built-in rule analysis components, stored in JSON and a Jupyter notebook. They are aggregated into the report."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "download: s3://sagemaker-us-east-1-645012351640/pytorch-training-2021-09-14-18-16-16-595/rule-output/ProfilerReport/profiler-output/profiler-reports/BatchSize.json to profiler_report/profiler-reports/BatchSize.json\n",
      "download: s3://sagemaker-us-east-1-645012351640/pytorch-training-2021-09-14-18-16-16-595/rule-output/ProfilerReport/profiler-output/profiler-reports/CPUBottleneck.json to profiler_report/profiler-reports/CPUBottleneck.json\n",
      "download: s3://sagemaker-us-east-1-645012351640/pytorch-training-2021-09-14-18-16-16-595/rule-output/ProfilerReport/profiler-output/profiler-reports/GPUMemoryIncrease.json to profiler_report/profiler-reports/GPUMemoryIncrease.json\n",
      "download: s3://sagemaker-us-east-1-645012351640/pytorch-training-2021-09-14-18-16-16-595/rule-output/ProfilerReport/profiler-output/profiler-reports/IOBottleneck.json to profiler_report/profiler-reports/IOBottleneck.json\n",
      "download: s3://sagemaker-us-east-1-645012351640/pytorch-training-2021-09-14-18-16-16-595/rule-output/ProfilerReport/profiler-output/profiler-reports/LoadBalancing.json to profiler_report/profiler-reports/LoadBalancing.json\n",
      "download: s3://sagemaker-us-east-1-645012351640/pytorch-training-2021-09-14-18-16-16-595/rule-output/ProfilerReport/profiler-output/profiler-reports/MaxInitializationTime.json to profiler_report/profiler-reports/MaxInitializationTime.json\n",
      "download: s3://sagemaker-us-east-1-645012351640/pytorch-training-2021-09-14-18-16-16-595/rule-output/ProfilerReport/profiler-output/profiler-report.ipynb to profiler_report/profiler-report.ipynb\n",
      "download: s3://sagemaker-us-east-1-645012351640/pytorch-training-2021-09-14-18-16-16-595/rule-output/ProfilerReport/profiler-output/profiler-reports/LowGPUUtilization.json to profiler_report/profiler-reports/LowGPUUtilization.json\n",
      "download: s3://sagemaker-us-east-1-645012351640/pytorch-training-2021-09-14-18-16-16-595/rule-output/ProfilerReport/profiler-output/profiler-reports/Dataloader.json to profiler_report/profiler-reports/Dataloader.json\n",
      "download: s3://sagemaker-us-east-1-645012351640/pytorch-training-2021-09-14-18-16-16-595/rule-output/ProfilerReport/profiler-output/profiler-reports/OverallSystemUsage.json to profiler_report/profiler-reports/OverallSystemUsage.json\n",
      "download: s3://sagemaker-us-east-1-645012351640/pytorch-training-2021-09-14-18-16-16-595/rule-output/ProfilerReport/profiler-output/profiler-reports/StepOutlier.json to profiler_report/profiler-reports/StepOutlier.json\n",
      "download: s3://sagemaker-us-east-1-645012351640/pytorch-training-2021-09-14-18-16-16-595/rule-output/ProfilerReport/profiler-output/profiler-reports/OverallFrameworkMetrics.json to profiler_report/profiler-reports/OverallFrameworkMetrics.json\n",
      "download: s3://sagemaker-us-east-1-645012351640/pytorch-training-2021-09-14-18-16-16-595/rule-output/ProfilerReport/profiler-output/profiler-report.html to profiler_report/profiler-report.html\n"
     ]
    }
   ],
   "source": [
    "!aws s3 cp --recursive $profiler_report_s3_uri ./profiler_report/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can review the profiler report in the console.\n",
    "\n",
    "**Note**: Click `Trust HTML` in the profiler-report.html tab that opens (on top of the document)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<b>Review <a target=\"blank\" href=\"./profiler_report/profiler-report.html\">profiler report</a></b>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "\n",
    "display(HTML('<b>Review <a target=\"blank\" href=\"./profiler_report/profiler-report.html\">profiler report</a></b>'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='c2w2-3.'></a>\n",
    "# 3. Deploy the model\n",
    "Create a custom `SentimentPredictor` that encapsulates a JSONLines serializer and deserializer. To be passed into the `PyTorchModel` it needs to be wrapped as a class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.predictor import Predictor\n",
    "from sagemaker.serializers import JSONLinesSerializer\n",
    "from sagemaker.deserializers import JSONLinesDeserializer\n",
    "\n",
    "class SentimentPredictor(Predictor):\n",
    "    def __init__(self, endpoint_name, sagemaker_session):\n",
    "        super().__init__(endpoint_name, \n",
    "                         sagemaker_session=sagemaker_session, \n",
    "                         serializer=JSONLinesSerializer(),\n",
    "                         deserializer=JSONLinesDeserializer())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "from sagemaker.pytorch.model import PyTorchModel\n",
    "\n",
    "timestamp = int(time.time())\n",
    "\n",
    "pytorch_model_name = '{}-{}-{}'.format(training_job_name, 'pt', timestamp)\n",
    "\n",
    "model = PyTorchModel(name=pytorch_model_name,\n",
    "                     model_data=estimator.model_data,\n",
    "                     predictor_cls=SentimentPredictor,\n",
    "                     entry_point='inference.py',\n",
    "                     source_dir='src',\n",
    "                     framework_version='1.6.0',\n",
    "                     py_version='py3',\n",
    "                     role=role)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pytorch-training-2021-09-14-18-16-16-595-pt-1631645999\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "\n",
    "pytorch_endpoint_name = '{}-{}-{}'.format(training_job_name, 'pt', timestamp)\n",
    "\n",
    "print(pytorch_endpoint_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### _This cell will take approximately 5-10 minutes to run._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-----------------!CPU times: user 2min 12s, sys: 10.1 s, total: 2min 22s\n",
      "Wall time: 10min 50s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "predictor = model.deploy(initial_instance_count=1, \n",
    "                         instance_type='ml.m5.large', \n",
    "                         endpoint_name=pytorch_endpoint_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Review the Endpoint in the AWS console.\n",
    "\n",
    "**Instructions**: \n",
    "- open the link\n",
    "- notice that you are in the section `Amazon SageMaker` -> `Endpoints`\n",
    "- check the name of the Endpoint, its status and other available information"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<b>Review <a target=\"blank\" href=\"https://console.aws.amazon.com/sagemaker/home?region=us-east-1#/endpoints/pytorch-training-2021-09-14-18-16-16-595-pt-1631645999\">SageMaker REST Endpoint</a></b>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "\n",
    "display(HTML('<b>Review <a target=\"blank\" href=\"https://console.aws.amazon.com/sagemaker/home?region={}#/endpoints/{}\">SageMaker REST Endpoint</a></b>'.format(region, pytorch_endpoint_name)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='c2w2-4.'></a>\n",
    "# 4. Test model\n",
    "Here, we will pass sample strings of text to the endpoint in order to see the sentiment. We give you one example of each, however, feel free to play around and change the strings yourself!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicted class 1 with probability 0.9605445861816406\n",
      "Predicted class 0 with probability 0.5798221230506897\n",
      "Predicted class -1 with probability 0.7667604684829712\n"
     ]
    }
   ],
   "source": [
    "inputs = [\n",
    "    {\"features\": [\"I love this product!\"]},\n",
    "    {\"features\": [\"OK, but not great.\"]},\n",
    "    {\"features\": [\"This is not the right product.\"]},\n",
    "]\n",
    "\n",
    "predictor = SentimentPredictor(endpoint_name=pytorch_endpoint_name,\n",
    "                               sagemaker_session=sess)\n",
    "\n",
    "predicted_classes = predictor.predict(inputs)\n",
    "\n",
    "for predicted_class in predicted_classes:\n",
    "    print(\"Predicted class {} with probability {}\".format(predicted_class['predicted_label'], predicted_class['probability']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Upload the notebook and train.py file into S3 bucket for grading purposes.\n",
    "\n",
    "**Note**: you may need to save the file before the upload."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "upload: ./C2_W2_Assignment.ipynb to s3://sagemaker-us-east-1-645012351640/C2_W2_Assignment_Learner.ipynb\n",
      "upload: src/train.py to s3://sagemaker-us-east-1-645012351640/src/C2_W2_train_Learner.py\n"
     ]
    }
   ],
   "source": [
    "!aws s3 cp ./C2_W2_Assignment.ipynb s3://$bucket/C2_W2_Assignment_Learner.ipynb\n",
    "!aws s3 cp ./src/train.py s3://$bucket/src/C2_W2_train_Learner.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Please go to the main lab window and click on `Submit` button (see the `Finish the lab` section of the instructions)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (Data Science)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/datascience-1.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}