tutorial_pytorch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tutorial: PyTorch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "__author__ = \"Ignacio Cases\"\n",
    "__version__ = \"CS224u, Stanford, Spring 2022\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contents\n",
    "\n",
    "1. [Motivation](#Motivation)\n",
    "1. [Importing PyTorch](#Importing-PyTorch)\n",
    "1. [Tensors](#Tensors)\n",
    "    1. [Tensor creation](#Tensor-creation)\n",
    "    1. [Operations on tensors](#Operations-on-tensors)\n",
    "1. [GPU computation](#GPU-computation)\n",
    "1. [Neural network foundations](#Neural-network-foundations)\n",
    "    1. [Automatic differentiation](#Automatic-differentiation)\n",
    "    1. [Modules](#Modules)\n",
    "    1. [Sequential](#Sequential)\n",
    "    1. [Criteria and loss functions](#Criteria-and-loss-functions)\n",
    "    1. [Optimization](#Optimization)\n",
    "    1. [Training a simple model](#Training-a-simple-model)\n",
    "1. [Reproducibility](#Reproducibility)\n",
    "1. [References](#References)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Motivation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PyTorch is a Python package designed to carry out scientific computation. We use PyTorch in a range of different environments: local model development, large-scale deployments on big clusters, and even _inference_ in embedded, low-power systems. While similar in many aspects to NumPy, PyTorch enables us to perform fast and efficient training of deep learning and reinforcement learning models not only on the CPU but also on a GPU or other ASICs (Application Specific Integrated Circuits) for AI, such as Tensor Processing Units (TPU)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importing PyTorch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tutorial assumes a working installation of PyTorch using your `nlu` environment, but the content applies to any regular installation of PyTorch. If you don't have a working installation of PyTorch, please follow the instructions in [the setup notebook](setup.ipynb).\n",
    "\n",
    "To get started working with PyTorch we simply begin by importing the torch module:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Side note**: why not `import pytorch`? The name of the package is `torch` for historical reasons: `torch` is the orginal name of the ancestor of the PyTorch library that got started back in 2002 as a C library with Lua scripting. It was only much later that the original `torch` was ported to Python. The PyTorch project decided to prefix the Py to make clear that this library refers to the Python version, as it was confusing back then to know which `torch` one was referring to. All the internal references to the library use just `torch`. It's possible that PyTorch will be renamed at some point, as the original `torch` is no longer maintained and there is no longer confusion."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see the version installed and determine whether or not we have a GPU-enabled PyTorch install by issuing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PyTorch version 2.1.2\n",
      "GPU-enabled installation? True\n"
     ]
    }
   ],
   "source": [
    "print(\"PyTorch version {}\".format(torch.__version__))\n",
    "print(\"GPU-enabled installation? {}\".format(torch.cuda.is_available() or (torch.backends.mps.is_available() and torch.backends.mps.is_built())))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PyTorch has good [documentation](https://pytorch.org/docs/stable/index.html) but it can take some time to familiarize oneself with the structure of the package; it's worth the effort to do so!\n",
    "\n",
    "We will also make use of other imports:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tensors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tensors collections of numbers represented as an array, and are the basic building blocks in PyTorch.\n",
    "\n",
    "You are probably already familiar with several types of tensors:\n",
    "    \n",
    "- A scalar, a single number, is a zero-th order tensor.\n",
    "    \n",
    "- A column vector $v$ of dimensionality $d_c \\times 1$ is a tensor of order 1.\n",
    "    \n",
    "- A row vector $x$ of dimensionality $1 \\times d_r$ is a tensor of order 1.\n",
    "    \n",
    "- A matrix $A$ of dimensionality $d_r \\times d_c$ is a tensor of order 2.\n",
    "    \n",
    "- A cube $T$ of dimensionality $d_r \\times d_c \\times d_d$ is a tensor of order 3. \n",
    "\n",
    "Tensors are the fundamental blocks that carry information in our mathematical models, and they are composed using several operations to create mathematical graphs in which information can flow (propagate) forward (functional application) and backwards (using the chain rule). \n",
    "\n",
    "We have seen multidimensional arrays in NumPy. These NumPy objects are also a representation of tensors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Side note**: what is a tensor __really__? Tensors are important mathematical objects with applications in multiple domains in mathematics and physics. The term \"tensor\" comes from the usage of these mathematical objects to describe the stretching of a volume of matter under *tension*. They are central objects of study in a subfield of mathematics known as differential geometry, which deals with the geometry of continuous vector spaces. As a very high-level summary (and as first approximation), tensors are defined as multi-linear \"machines\" that have a number of slots (their order, a.k.a. rank), taking a number of \"column\" vectors and \"row\" vectors *to produce a scalar*. For example, a tensor $\\mathbf{A}$ (represented by a matrix with rows and columns that you could write on a sheet of paper) can be thought of having two slots. So when $\\mathbf{A}$ acts upon a column vector $\\mathbf{v}$ and a row vector $\\mathbf{x}$, it returns a scalar:\n",
    "    \n",
    "$$\\mathbf{A}(\\mathbf{x}, \\mathbf{v}) = s$$\n",
    "    \n",
    "If $\\mathbf{A}$ only acts on the column vector, for example, the result will be another column tensor $\\mathbf{u}$ of one order less than the order of $\\mathbf{A}$. Thus, when $\\mathbf{v}$ acts is similar to \"removing\" its slot: \n",
    "\n",
    "$$\\mathbf{u} = \\mathbf{A}(\\mathbf{v})$$\n",
    "\n",
    "The resulting $\\mathbf{u}$ can later interact with another row vector to produce a scalar or be used in any other way. \n",
    "\n",
    "This can be a very powerful way of thinking about tensors, as their slots can guide you when writing code, especially given that PyTorch has a _functional_ approach to modules in which this view is very much highlighted. As we will see below, these simple equations above have a completely straightforward representation in the code. In the end, most of what our models will do is to process the input using this type of functional application so that we end up having a tensor output and a scalar value that measures how good our output is with respect to the real output value in the dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tensor creation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's get started with tensors in PyTorch. The framework supports eight different types ([Lapan 2018](#References)):\n",
    "\n",
    "- 3 float types (16-bit, 32-bit, 64-bit): `torch.FloatTensor` is the class name for the commonly used 32-bit tensor.\n",
    "- 5 integer types (signed 8-bit, unsigned 8-bit, 16-bit, 32-bit, 64-bit): common tensors of these types are the 8-bit unsigned tensor `torch.ByteTensor` and the 64-bit `torch.LongTensor`.\n",
    "\n",
    "There are three fundamental ways to create tensors in PyTorch ([Lapan 2018](#References)):\n",
    "\n",
    "- Call a tensor constructor of a given type, which will create a non-initialized tensor. So we then need to fill this tensor later to be able to use it.\n",
    "- Call a built-in method in the `torch` module that returns a tensor that is already initialized.\n",
    "- Use the PyTorch–NumPy bridge."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Calling the constructor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's first create a 2 x 3 dimensional tensor of the type `float`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0., 0., 0.],\n",
      "        [0., 0., 0.]])\n",
      "torch.Size([2, 3])\n"
     ]
    }
   ],
   "source": [
    "t = torch.FloatTensor(2, 3)\n",
    "print(t)\n",
    "print(t.size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that we specified the dimensions as the arguments to the constructor by passing the numbers directly – and not a list or a tuple, which would have very different outcomes as we will see below! We can always inspect the size of the tensor using the `size()` method.\n",
    "\n",
    "The constructor method allocates space in memory for this tensor. However, the tensor is *non-initialized*. In order to initialize it, we need to call any of the tensor initialization methods of the basic tensor types. For example, the tensor we just created has a built-in method `zero_()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0., 0., 0.],\n",
       "        [0., 0., 0.]])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.zero_()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The underscore after the method name is important: it means that the operation happens _in place_: the returned object is the same object but now with different content. A very handy way to construct a tensor using the constructor happens when we have available the content we want to put in the tensor in the form of a Python iterable. In this case, we just pass it as the argument to the constructor:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[1., 2., 3.],\n",
       "        [4., 5., 6.]])"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "torch.FloatTensor([[1, 2, 3], [4, 5, 6]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Calling a method in the torch module"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A very convenient way to create tensors, in addition to using the constructor method, is to use one of the multiple methods provided in the `torch` module. In particular, the `tensor` method allows us to pass a number or iterable as the argument to get the appropriately typed tensor:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A 64-bit integer tensor: tensor([1, 2, 3]), torch.LongTensor\n",
      "A 32-bit float tensor: tensor([1., 2., 3.]), torch.FloatTensor\n"
     ]
    }
   ],
   "source": [
    "tl = torch.tensor([1, 2, 3])\n",
    "t = torch.tensor([1., 2., 3.])\n",
    "print(\"A 64-bit integer tensor: {}, {}\".format(tl, tl.type()))\n",
    "print(\"A 32-bit float tensor: {}, {}\".format(t, t.type()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can create a similar 2x3 tensor to the one above by using the `torch.zeros()` method, passing a sequence of dimensions to it: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0., 0., 0.],\n",
      "        [0., 0., 0.]])\n"
     ]
    }
   ],
   "source": [
    "t = torch.zeros(2, 3)\n",
    "print(t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are many methods for creating tensors. We list some useful ones:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0., 0., 0.],\n",
      "        [0., 0., 0.]])\n",
      "tensor([[1., 1., 1.],\n",
      "        [1., 1., 1.]])\n",
      "tensor([[5., 5., 5.],\n",
      "        [5., 5., 5.]])\n",
      "tensor([[0.1968, 0.2382, 0.1056],\n",
      "        [0.2136, 0.3114, 0.9221]])\n",
      "tensor([[-0.8992,  0.7760,  0.6889],\n",
      "        [ 0.7333, -1.5166, -0.5639]])\n"
     ]
    }
   ],
   "source": [
    "t_zeros = torch.zeros_like(t)        # zeros_like returns a new tensor\n",
    "t_ones = torch.ones(2, 3)            # creates a tensor with 1s\n",
    "t_fives = torch.empty(2, 3).fill_(5) # creates a non-initialized tensor and fills it with 5\n",
    "t_random = torch.rand(2, 3)          # creates a uniform random tensor\n",
    "t_normal = torch.randn(2, 3)         # creates a normal random tensor\n",
    "\n",
    "print(t_zeros)\n",
    "print(t_ones)\n",
    "print(t_fives)\n",
    "print(t_random)\n",
    "print(t_normal)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now see emerging two important paradigms in PyTorch. The _imperative_ approach to performing operations, using _inplace_ methods, is in marked contrast with an additional paradigm also used in PyTorch, the _functional_ approach, where the returned object is a copy of the original object. Both paradigms have their specific use cases as we will be seeing below. The rule of thumb is that _inplace_ methods are faster and don't require extra memory allocation in general, but they can be tricky to understand (keep this in mind regarding the computational graph that we will see below). _Functional_ methods make the code referentially transparent, which is a highly desired property that makes it easier to understand the underlying math, but we rely on the efficiency of the implementation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# creates a new copy of the tensor that is still linked to\n",
    "# the computational graph (see below)\n",
    "t1 = torch.clone(t)\n",
    "assert id(t) != id(t1), 'Functional methods create a new copy of the tensor'\n",
    "\n",
    "# To create a new _independent_ copy, we do need to detach\n",
    "# from the graph\n",
    "t1 = torch.clone(t).detach()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Using the PyTorch–NumPy bridge"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A quite useful feature of PyTorch is its almost seamless integration with NumPy, which allows us to perform operations on NumPy and interact from PyTorch with the large number of NumPy libraries as well. Converting a NumPy multi-dimensional array into a PyTorch tensor is very simple: we only need to call the `tensor` method with NumPy objects as the argument:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NumPy array: [1. 2. 3.], type: float64\n",
      "Torch tensor: tensor([1., 2., 3.], dtype=torch.float64), type: torch.float64\n"
     ]
    }
   ],
   "source": [
    "# Create a new multi-dimensional array in NumPy with the np datatype (np.float32)\n",
    "a = np.array([1., 2., 3.])\n",
    "\n",
    "# Convert the array to a torch tensor\n",
    "t = torch.tensor(a)\n",
    "\n",
    "print(\"NumPy array: {}, type: {}\".format(a, a.dtype))\n",
    "print(\"Torch tensor: {}, type: {}\".format(t, t.dtype))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also seamlessly convert a PyTorch tensor into a NumPy array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1., 2., 3.])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t.numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Side note**: why not `torch.from_numpy(a)`? The `from_numpy()` method is depecrated in favor of `tensor()`, which is a more capable method in the torch package. `from_numpy()` is only there for backwards compatibility. It can be a little bit quirky, so I recommend using the newer method in PyTorch >= 0.4."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Indexing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "Indexing works as expected with NumPy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([-1.4784, -0.4138])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "t = torch.randn(2, 3)\n",
    "t[ : , 0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PyTorch also supports indexing using long tensors, for example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-8.2419e-01, -5.6042e-01, -1.9861e+00,  2.0703e+00,  1.9030e-01,\n",
      "          2.8481e-01],\n",
      "        [-9.6856e-01, -8.7125e-01,  1.6743e-01, -6.9585e-01,  5.1997e-01,\n",
      "          2.4957e+00],\n",
      "        [-6.6483e-01, -7.5324e-02, -8.6668e-01, -5.5361e-01, -6.9560e-01,\n",
      "          1.0776e+00],\n",
      "        [-4.4967e-01,  5.3673e-01, -7.6465e-04, -3.2480e-01, -3.8235e-01,\n",
      "          1.4507e+00],\n",
      "        [-6.8185e-01,  3.1295e-02, -6.3419e-01,  5.8635e-01, -1.5615e+00,\n",
      "          8.8179e-01]])\n",
      "tensor([[-9.6856e-01, -8.7125e-01,  1.6743e-01, -6.9585e-01,  5.1997e-01,\n",
      "          2.4957e+00],\n",
      "        [-4.4967e-01,  5.3673e-01, -7.6465e-04, -3.2480e-01, -3.8235e-01,\n",
      "          1.4507e+00]])\n",
      "tensor([0.5200, 1.4507])\n"
     ]
    }
   ],
   "source": [
    "t = torch.randn(5, 6)\n",
    "print(t)\n",
    "i = torch.tensor([1, 3])\n",
    "j = torch.tensor([4, 5])\n",
    "print(t[i])                          # selects rows 1 and 3\n",
    "print(t[i, j])                       # selects (1, 4) and (3, 5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Type conversion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each tensor has a set of convenient methods to convert types. For example, if we want to convert the tensor above to a 32-bit float tensor, we use the method `.float()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-8.2419e-01, -5.6042e-01, -1.9861e+00,  2.0703e+00,  1.9030e-01,\n",
      "          2.8481e-01],\n",
      "        [-9.6856e-01, -8.7125e-01,  1.6743e-01, -6.9585e-01,  5.1997e-01,\n",
      "          2.4957e+00],\n",
      "        [-6.6483e-01, -7.5324e-02, -8.6668e-01, -5.5361e-01, -6.9560e-01,\n",
      "          1.0776e+00],\n",
      "        [-4.4967e-01,  5.3673e-01, -7.6465e-04, -3.2480e-01, -3.8235e-01,\n",
      "          1.4507e+00],\n",
      "        [-6.8185e-01,  3.1295e-02, -6.3419e-01,  5.8635e-01, -1.5615e+00,\n",
      "          8.8179e-01]])\n",
      "tensor([[-8.2419e-01, -5.6042e-01, -1.9861e+00,  2.0703e+00,  1.9030e-01,\n",
      "          2.8481e-01],\n",
      "        [-9.6856e-01, -8.7125e-01,  1.6743e-01, -6.9585e-01,  5.1997e-01,\n",
      "          2.4957e+00],\n",
      "        [-6.6483e-01, -7.5324e-02, -8.6668e-01, -5.5361e-01, -6.9560e-01,\n",
      "          1.0776e+00],\n",
      "        [-4.4967e-01,  5.3673e-01, -7.6465e-04, -3.2480e-01, -3.8235e-01,\n",
      "          1.4507e+00],\n",
      "        [-6.8185e-01,  3.1295e-02, -6.3419e-01,  5.8635e-01, -1.5615e+00,\n",
      "          8.8179e-01]], dtype=torch.float64)\n",
      "tensor([[  0,   0, 255,   2,   0,   0],\n",
      "        [  0,   0,   0,   0,   0,   2],\n",
      "        [  0,   0,   0,   0,   0,   1],\n",
      "        [  0,   0,   0,   0,   0,   1],\n",
      "        [  0,   0,   0,   0, 255,   0]], dtype=torch.uint8)\n"
     ]
    }
   ],
   "source": [
    "t = t.float()   # converts to 32-bit float\n",
    "print(t)\n",
    "t = t.double()  # converts to 64-bit float\n",
    "print(t)\n",
    "t = t.byte()    # converts to unsigned 8-bit integer\n",
    "print(t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Operations on tensors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we know how to create tensors, let's create some of the fundamental tensors and see some common operations on them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(42)\n"
     ]
    }
   ],
   "source": [
    "# Scalars =: creates a tensor with a scalar\n",
    "# (zero-th order tensor,  i.e. just a number)\n",
    "s = torch.tensor(42)\n",
    "print(s)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Tip**: a very convenient to access scalars is with `.item()`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "42"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.item()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see higher-order tensors – remember we can always inspect the dimensionality of a tensor using the `.size()` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Row vector\n",
      "tensor([[ 0.7549, -0.6580,  0.4060]])\n",
      "with size torch.Size([1, 3])\n",
      "Column vector\n",
      "tensor([[ 0.4038],\n",
      "        [-0.8217],\n",
      "        [-0.0148]])\n",
      "with size torch.Size([3, 1])\n",
      "Matrix\n",
      "tensor([[ 0.2288, -0.9596,  0.2591],\n",
      "        [ 0.8632, -0.4465, -0.2665],\n",
      "        [-0.0490,  0.9633, -0.0357]])\n",
      "with size torch.Size([3, 3])\n"
     ]
    }
   ],
   "source": [
    "# Row vector\n",
    "x = torch.randn(1,3)\n",
    "print(\"Row vector\\n{}\\nwith size {}\".format(x, x.size()))\n",
    "\n",
    "# Column vector\n",
    "v = torch.randn(3,1)\n",
    "print(\"Column vector\\n{}\\nwith size {}\".format(v, v.size()))\n",
    "\n",
    "# Matrix\n",
    "A = torch.randn(3, 3)\n",
    "print(\"Matrix\\n{}\\nwith size {}\".format(A, A.size()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A common operation is matrix-vector multiplication (and in general tensor-tensor multiplication). For example, the product $\\mathbf{A}\\mathbf{v} + \\mathbf{b}$ is as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 0.8770],\n",
      "        [ 0.7193],\n",
      "        [-0.8108]])\n",
      "tensor([[ 0.0072],\n",
      "        [-0.1506],\n",
      "        [-1.1664]])\n"
     ]
    }
   ],
   "source": [
    "u = torch.matmul(A, v)\n",
    "print(u)\n",
    "b = torch.randn(3,1)\n",
    "y = u + b              # we can also do torch.add(u, b)\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "where we retrieve the expected result (a column vector of dimensions 3x1). We can of course compose operations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-0.14041739702224731\n"
     ]
    }
   ],
   "source": [
    "s = torch.matmul(x, torch.matmul(A, v))\n",
    "print(s.item())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are many functions implemented for every tensor, and we encourage you to study the documentation. Some of the most common ones:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0, 1, 2],\n",
       "        [0, 1, 2]])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# common tensor methods (they also have the counterpart in\n",
    "# the torch package, e.g. as torch.sum(t))\n",
    "t = torch.randn(2,3)\n",
    "t.sum(dim=0)\n",
    "t.t()                   # transpose\n",
    "t.numel()               # number of elements in tensor\n",
    "t.nonzero()             # indices of non-zero elements\n",
    "t.view(-1, 2)           # reorganizes the tensor to these dimensions\n",
    "t.squeeze()             # removes size 1 dimensions\n",
    "t.unsqueeze(0)          # inserts a dimension\n",
    "\n",
    "# operations in the package\n",
    "torch.arange(0, 10)     # tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])\n",
    "torch.eye(3, 3)         # creates a 3x3 matrix with 1s in the diagonal (identity in this case)\n",
    "t = torch.arange(0, 3)\n",
    "torch.cat((t, t))       # tensor([0, 1, 2, 0, 1, 2])\n",
    "torch.stack((t, t))     # tensor([[0, 1, 2],\n",
    "                        #         [0, 1, 2]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## GPU computation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Deep Learning frameworks take advantage of the powerful computational capabilities of modern graphic processing units (GPUs). GPUs were originally designed to perform frequent operations for graphics very efficiently and fast, such as linear algebra operations, which makes them ideal for our interests. PyTorch makes it very easy to use the GPU: the common scenario is to tell the framework that we want to instantiate a tensor with a type that makes it a GPU tensor, or move a given CPU tensor to the GPU. All the tensors that we have seen above are CPU tensors, and PyTorch has the counterparts for GPU tensors in the `torch.cuda` module. Let's see how this works.\n",
    "\n",
    "One way to explicitly declare the tensor type as a CUDA based GPU tensor is through the use of the constructor method for tensor creation inside the `torch.cuda` module:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "type torch.cuda.FloatTensor not available. Torch not compiled with CUDA enabled.\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    t_gpu = torch.cuda.FloatTensor(3, 3)   # creation of a GPU tensor\n",
    "    t_gpu.zero_()                          # initialization to zero\n",
    "except TypeError as err:\n",
    "    print(err)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, a more common approach that gives us flexibility is through the use of devices. A device in PyTorch refers to either the CPU (indicated by the string \"cpu\") or one of the possible CUDA based GPU cards in the machine (indicated by the string \"cuda:$n$\", where $n$ is the index of the card) or the Metal GPU (indicated by the string \"mps\") for Mac computers with Apple silicon or AMD GPUs. Let's create a random gaussian matrix using a method from the `torch` package, and set the computational device to be the GPU by specifying the `device` to be `cuda:0` or `mps`, the first GPU card in our machine (this code will fail if you don't have a GPU, but we will work around that below): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 2.4210e+00, -7.1112e-02,  3.6360e-02],\n",
       "        [ 6.4747e-01, -1.6809e-01, -4.6107e-04],\n",
       "        [ 9.1403e-01,  2.7094e-01, -1.5029e+00]], device='mps:0')"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "try:\n",
    "    t_gpu = torch.randn(3, 3, device=\"mps\" if torch.backends.mps.is_available() and torch.backends.mps.is_built() else \"cuda:0\")\n",
    "except AssertionError as err:\n",
    "    print(err)\n",
    "    t_gpu = None\n",
    "\n",
    "t_gpu"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can notice, the tensor now has the explicit device set to be a CUDA device, not a CPU device. Let's now create a tensor in the CPU and move it to the GPU:\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 1.4454, -0.3414,  0.1322],\n",
       "        [ 0.6452,  1.4311, -2.6089],\n",
       "        [ 2.3189,  0.1944,  0.0579]])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# we could also state explicitly the device to be the\n",
    "# CPU with torch.randn(3,3,device=\"cpu\")\n",
    "t = torch.randn(3, 3)\n",
    "t"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, the device is the CPU, but PyTorch does not explicitly say that given that this is the default behavior. To copy the tensor to the GPU we use the `.to()` method that every tensor implements, passing the device as an argument. This method creates a copy in the specified device or, if the tensor already resides in that device, it returns the original tensor ([Lapan 2018](#References)): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 1.4454, -0.3414,  0.1322],\n",
      "        [ 0.6452,  1.4311, -2.6089],\n",
      "        [ 2.3189,  0.1944,  0.0579]], device='mps:0')\n",
      "mps:0\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    t_gpu = t.to(\"mps\" if torch.backends.mps.is_available() and torch.backends.mps.is_built() else \"cuda:0\")  # copies the tensor from CPU to GPU\n",
    "    # note that if we do now t_to_gpu.to(\"cuda:0\") it will\n",
    "    # return the same tensor without doing anything else\n",
    "    # as this tensor already resides on the GPU\n",
    "    print(t_gpu)\n",
    "    print(t_gpu.device)\n",
    "except AssertionError as err:\n",
    "    print(err)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Tip**: When we program PyTorch models, we will have to specify the device in several places (not so many, but definitely more than once). A good practice that is consistent accross the implementation and makes the code more portable is to declare early in the code a device variable by querying the framework if there is a GPU available that we can use. We can do this by writing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mps\n"
     ]
    }
   ],
   "source": [
    "device = torch.device(\"cpu\")\n",
    "\n",
    "if torch.cuda.is_available():\n",
    "    device = torch.device(\"cuda:0\")\n",
    "elif torch.backends.mps.is_available() and torch.backends.mps.is_built():\n",
    "    device = torch.device(\"mps\")\n",
    "\n",
    "print(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can then use `device` as an argument of the `.to()` method in the rest of our code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[ 1.4454, -0.3414,  0.1322],\n",
       "        [ 0.6452,  1.4311, -2.6089],\n",
       "        [ 2.3189,  0.1944,  0.0579]], device='mps:0')"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# moves t to the device (this code will **not** fail if the\n",
    "# local machine has not access to a GPU)\n",
    "t.to(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Side note**: having good GPU backend support is a critical aspect of a deep learning framework. Some models depend crucially on performing computations on a GPU. Most frameworks, including PyTorch, only provide good support for GPUs manufactured by Nvidia. This is mostly due to the heavy investment this company made on CUDA (Compute Unified Device Architecture), the underlying parallel computing platform that enables this type of scientific computing (and the reason for the device label), with specific implementations targeted to Deep Neural Networks as cuDNN. Other GPU manufacturers, most notably AMD, are making efforts to towards enabling ML computing in their cards, but their support is still partial."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Neural network foundations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Computing gradients is a crucial feature in deep learning, given that the training procedure of neural networks relies on optimization techniques that update the parameters of the model by using the gradient information of a scalar magnitude – the loss function. How is it possible to compute the derivatives? There are different methods, namely\n",
    "\n",
    "- **Symbolic Differentiation**: given a symbolic expression, the software provides the derivative by performing symbolic transformations (e.g. Wolfram Alpha). The benefits are clear, but it is not always possible to compute an analytical expression.\n",
    "\n",
    "- **Numerical Differentiation**: computes the derivatives using expressions that are suitable to be evaluated numerically, using the finite differences method to several orders of approximation. A big drawback is that these methods are slow.\n",
    "\n",
    "- **Automatic Differentiation**: a library adds to the set of functional primitives an implementation of the derivative for each of these functions. Thus, if the library contains the function $sin(x)$, it also implements the derivative of this function, $\\frac{d}{dx}sin(x) = cos(x)$. Then, given a composition of functions, the library can compute the derivative with respect a variable by successive application of the chain rule, a method that is known in deep learning as backpropagation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Automatic differentiation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Modern deep learning libraries are capable of performing automatic differentiation. The two main approaches to computing the graph are _static_ and _dynamic_ processing ([Lapan 2018](#References)):\n",
    "\n",
    "- **Static graphs**: the deep learning framework converts the computational graph into a static representation that cannot be modified. This allows the library developers to do very aggressive optimizations on this static graph ahead of computation time, pruning some areas and transforming others so that the final product is highly optimized and fast. The drawback is that some models can be really hard to implement with this approach. For example, TensorFlow uses static graphs. Having static graphs is part of the reason why TensorFlow has excellent support for sequence processing, which makes it very popular in NLP.\n",
    "\n",
    "- **Dynamic graphs**: the framework does not create a graph ahead of computation, but records the operations that are performed, which can be quite different for different inputs. When it is time to compute the gradients, it unrolls the graph and perform the computations. A major benefit of this approach is that implementing complex models can be easier in this paradigm. This flexibility comes at the expense of the major drawback of this approach: speed. Dynamic graphs cannot leverage the same level of ahead-of-time optimization as static graphs, which makes them slower. PyTorch uses dynamic graphs as the underlying paradigm for gradient computation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is simple graph to compute $y = wx + b$ (from [Rao and MacMahan 2019](#References-and-Further-Reading)):"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"fig/simple_computation_graph.png\" width=500 />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PyTorch computes the graph using the Autograd system. Autograd records a graph when performing the forward pass (function application), keeping track of all the tensors defined as inputs. These are the leaves of the graph. The output tensors are the roots of the graph. By navigating this graph from root to leaves, the gradients are automatically computed using the chain rule. In summary,\n",
    "\n",
    "- Forward pass (the successive function application) goes from leaves to root. We use the `apply` method in PyTorch.\n",
    "- Once the forward pass is completed, Autograd has recorded the graph and the backward pass (chain rule) can be done. We use the method `backwards` on the root of the graph."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Modules"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The base implementation for all neural network models in PyTorch is the class `Module` in the package `torch.nn`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.nn as nn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All our models subclass this base `nn.Module` class, which provides an interface to important methods used for constructing and working with our models, and which contains sensible initializations for our models. Modules can contain other modules (and usually do).\n",
    "\n",
    "Let's see a simple, custom implementation of a multi-layer feed forward network. In the example below, our simple mathematical model is\n",
    "\n",
    "$$\\mathbf{y} = \\mathbf{U}(f(\\mathbf{W}(\\mathbf{x})))$$\n",
    "\n",
    "where $f$ is a non-linear function (a `ReLU`), is directly translated into a similar expression in PyTorch. To do that, we simply subclass `nn.Module`, register the two affine transformations and the non-linearity, and implement their composition within the `forward` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MyCustomModule(nn.Module):\n",
    "    def __init__(self, n_inputs, n_hidden, n_output_classes):\n",
    "        # call super to initialize the class above in the hierarchy\n",
    "        super(MyCustomModule, self).__init__()\n",
    "        # first affine transformation\n",
    "        self.W = nn.Linear(n_inputs, n_hidden)\n",
    "        # non-linearity (here it is also a layer!)\n",
    "        self.f = nn.ReLU()\n",
    "        # final affine transformation\n",
    "        self.U = nn.Linear(n_hidden, n_output_classes)\n",
    "\n",
    "    def forward(self, x):\n",
    "        y = self.U(self.f(self.W(x)))\n",
    "        return y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, we can use our new module as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 0.2161, -0.1701]], grad_fn=<AddmmBackward0>)\n"
     ]
    }
   ],
   "source": [
    "# set the network's architectural parameters\n",
    "n_inputs = 3\n",
    "n_hidden= 4\n",
    "n_output_classes = 2\n",
    "\n",
    "# instantiate the model\n",
    "model = MyCustomModule(n_inputs, n_hidden, n_output_classes)\n",
    "\n",
    "# create a simple input tensor\n",
    "# size is [1,3]: a mini-batch of one example,\n",
    "# this example having dimension 3\n",
    "x = torch.FloatTensor([[0.3, 0.8, -0.4]])\n",
    "\n",
    "# compute the model output by **applying** the input to the module\n",
    "y = model(x)\n",
    "\n",
    "# inspect the output\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we see, the output is a tensor with its gradient function attached – Autograd tracks it for us."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Tip**: modules overrides the `__call__()` method, where the framework does some work. Thus, instead of directly calling the `forward()` method, we apply the input to the model instead."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sequential"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A powerful class in the `nn` package is `Sequential`, which allows us to express the code above more succinctly:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MyCustomModule(nn.Module):\n",
    "    def __init__(self, n_inputs, n_hidden, n_output_classes):\n",
    "        super(MyCustomModule, self).__init__()\n",
    "        self.network = nn.Sequential(\n",
    "            nn.Linear(n_inputs, n_hidden),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(n_hidden, n_output_classes))\n",
    "\n",
    "    def forward(self, x):\n",
    "        y = self.network(x)\n",
    "        return y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can imagine, this can be handy when we have a large number of layers for which the actual names are not that meaningful. It also improves readability:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MyCustomModule(nn.Module):\n",
    "    def __init__(self, n_inputs, n_hidden, n_output_classes):\n",
    "        super(MyCustomModule, self).__init__()\n",
    "        self.p_keep = 0.7\n",
    "        self.network = nn.Sequential(\n",
    "            nn.Linear(n_inputs, n_hidden),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(n_hidden, 2*n_hidden),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(2*n_hidden, n_output_classes),\n",
    "            # dropout argument is probability of dropping\n",
    "            nn.Dropout(1 - self.p_keep),\n",
    "            # applies softmax in the data dimension\n",
    "            nn.Softmax(dim=1)\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        y = self.network(x)\n",
    "        return y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Side note**: Another important package in `torch.nn` is `Functional`, typically imported as `F`. Functional contains many useful functions, from non-linear activations to convolutional, dropout, and even distance functions. Many of these functions have counterpart implementations as layers in the `nn` package so that they can be easily used in pipelines like the one above implemented using `nn.Sequential`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0., 0., 0., 5.]])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch.nn.functional as F\n",
    "\n",
    "y = F.relu(torch.FloatTensor([[-5, -1, 0, 5]]))\n",
    "\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Criteria and loss functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PyTorch has implementations for the most common criteria in the `torch.nn` package. You may notice that, as with many of the other functions, there are two implementations of loss functions: the reference functions in `torch.nn.functional` and practical class in `torch.nn`, which are the ones we typically use. Probably the two most common ones are ([Lapan 2018](#References)):\n",
    "\n",
    "- `nn.MSELoss` (mean squared error): squared $L_2$ norm used for regression.\n",
    "- `nn.CrossEntropyLoss`: criterion used for classification as the result of combining `nn.LogSoftmax()` and `nn.NLLLoss()` (negative log likelihood), operating on the input scores directly. When possible, we recommend using this class instead of using a softmax layer plus a log conversion and `nn.NLLLoss`, given that the `LossSoftmax` implementation guards against common numerical errors, resulting in less instabilities.\n",
    "\n",
    "Once our model produces a prediction, we pass it to the criteria to obtain a measure of the loss:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.9048, grad_fn=<NllLossBackward0>)\n"
     ]
    }
   ],
   "source": [
    "# the true label (in this case, 2) from our dataset wrapped\n",
    "# as a tensor of minibatch size of 1\n",
    "y_gold = torch.tensor([1])\n",
    "\n",
    "# our simple classification criterion for this simple example\n",
    "criterion = nn.CrossEntropyLoss()\n",
    "\n",
    "# forward pass of our model (remember, using apply instead of forward)\n",
    "y = model(x)\n",
    "\n",
    "# apply the criterion to get the loss corresponding to the pair (x, y)\n",
    "# with respect to the real y (y_gold)\n",
    "loss = criterion(y, y_gold)\n",
    "\n",
    "\n",
    "# the loss contains a gradient function that we can use to compute\n",
    "# the gradient dL/dw (gradient with respect to the parameters\n",
    "# for a given fixed input)\n",
    "print(loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Optimization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once we have computed the loss for a training example or minibatch of examples, we update the parameters of the model guided by the information contained in the gradient. The role of updating the parameters belongs to the optimizer, and PyTorch has a number of implementations available right away – and if you don't find your preferred optimizer as part of the library, chances are that you will find an existing implementation. Also, coding your own optimizer is indeed quite easy in PyTorch.\n",
    "\n",
    "**Side Note** The following is a summary of the most common optimizers. It is intended to serve as a reference (I use this table myself quite a lot). In practice, most people pick an optimizer that has been proven to behave well on a given domain, but optimizers are also a very active area of research on numerical analysis, so it is a good idea to pay some attention to this subfield. We recommend using second-order dynamics with an adaptive time step:\n",
    "\n",
    "- First-order dynamics\n",
    "    - Search direction only: `optim.SGD`\n",
    "    - Adaptive: `optim.RMSprop`, `optim.Adagrad`, `optim.Adadelta`\n",
    "    \n",
    "- Second-order dynamics\n",
    "    - Search direction only: Momentum `optim.SGD(momentum=0.9)`, Nesterov, `optim.SGD(nesterov=True)`\n",
    "    - Adaptive: `optim.Adam`, `optim.Adamax` (Adam with $L_\\infty$)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training a simple model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to illustrate the different concepts and techniques above, let's put them together in a very simple example: our objective will be to fit a very simple non-linear function, a sine wave:\n",
    "\n",
    "$$y = a \\sin(x + \\phi)$$\n",
    "\n",
    "where $a, \\phi$ are the given amplitude and phase of the sine function. Our objective is to learn to adjust this function using a feed forward network, this is:\n",
    "\n",
    "$$ \\hat{y} = f(x)$$\n",
    "\n",
    "such that the error between $y$ and $\\hat{y}$ is minimal according to our criterion. A natural criterion is to minimize the squared distance between the actual value of the sine wave and the value predicted by our function approximator, measured using the $L_2$ norm.\n",
    "\n",
    "**Side Note**: Although this example is easy, simple variations of this setting can pose a big challenge, and are used currently to illustrate difficult problems in learning, especially in a very active subfield known as meta-learning."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's import all the modules that we are going to need:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import torch.utils.data as data\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import math"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Early on the code, we define the device that we want to use:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "device = torch.device(\"cpu\")\n",
    "\n",
    "if torch.cuda.is_available():\n",
    "    device = torch.device(\"cuda:0\")\n",
    "elif torch.backends.mps.is_available() and torch.backends.mps.is_built():\n",
    "    device = torch.device(\"mps\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's fix $a=1$, $\\phi=1$ and generate traning data in the interval $x \\in [0,2\\pi)$ using NumPy:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjgAAAGdCAYAAAAfTAk2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAABW6ElEQVR4nO3deViUZdsG8HNmWF0AUWRRElyREHBJxC0TEkQty1JLc3kNy9ypTFq0NzXKysqlLJfU0rTNSjSKcFcUExFxRzEXGFAJEFQGZub7o0/eSMD7gXmY7fwdxxzf53Dd4zW8xpw8z70o9Hq9HkREREQWRGnsBoiIiIgMjQGHiIiILA4DDhEREVkcBhwiIiKyOAw4REREZHEYcIiIiMjiMOAQERGRxWHAISIiIotjY+wGjEGn0yE7OxuNGzeGQqEwdjtEREQkQK/X48aNG/Dy8oJSWfM1GqsMONnZ2fD29jZ2G0RERFQLly5dQsuWLWusscqA07hxYwB/f4OcnJyM3A0RERGJKCoqgre3d8XneE2sMuDcuS3l5OTEgENERGRmRKaXcJIxERERWRwGHCIiIrI4DDhERERkcRhwiIiIyOIw4BAREZHFYcAhIiIii8OAQ0RERBaHAYeIiIgsjlVu9Ed0h1anR0pWPvJu3Ebzxg7o2qoJDv/5V8Wfu/u6QqXkeWVEROaGAYeshlanx4Fz15F8/hrKdXpkXC7EwQv5KNPqqx3TpIEtnunRCuU6PbILbqGFiyN6tm2GHq2bMvgQEZkwhV6vr/6nu4UqKiqCs7MzCgsLeVSDFdCU6/DKd0fxY1o2DPWP3cFGgfeeDMaQIC8DvSIREd2LlM9vWefg7N69G0OGDIGXlxcUCgV+/PHHe47ZuXMnunTpAnt7e7Rt2xZr1qy5q2bZsmXw8fGBg4MDQkJCkJKSYvjmyaxpdXokn7uOsasOov3rv2CzAcMNANwu12Pq10cQsiARCxNOYV/mNWh1Vve7AhGRyZI14JSUlCAoKAjLli0Tqs/KysKgQYPw0EMPIS0tDTNmzMCzzz6LX3/9taJm06ZNiImJwdy5c5GamoqgoCBEREQgLy9PrrdBZmbL0WwEv/UbnlpxALvOXpP178q9ocEnO89h1MqD6PDaNiz67TSDDhGRCai3W1QKhQKbN2/G0KFDq6155ZVXsHXrVmRkZFQ8N3LkSBQUFCAhIQEAEBISggceeABLly4FAOh0Onh7e2Pq1KmYPXu2UC+8RWV5tDo99p65ihnfpOGvm2XGbge92jTByrEhcLRTGbsVIiKLYTK3qKRKTk5GeHh4peciIiKQnJwMANBoNDh8+HClGqVSifDw8IqaqpSWlqKoqKjSgyxHfFo22r+2DWPXHDKJcAMA+879hY5zEjBhDW+fEhEZg0kFHLVaDXd390rPubu7o6ioCLdu3cK1a9eg1WqrrFGr1dW+blxcHJydnSse3t7esvRP9UtTrsPDi3ZiysYjqGEhlFElnbqKvguTjN0GEZHVMamAI5fY2FgUFhZWPC5dumTslqiWNOU6rNh9Hn3fTUL713/B2bwSY7d0Txfzb+OxpXs5N4eIqB6Z1D44Hh4eyM3NrfRcbm4unJyc4OjoCJVKBZVKVWWNh4dHta9rb28Pe3t7WXqm+hO37QQ+251l7DZq5cjlQgS++SveeyIIUYGexm6HiMjimdQVnNDQUCQlVb6cn5iYiNDQUACAnZ0dunbtWqlGp9MhKSmpooYskzmHmztKNFq8sCEVjy3dg31nuayciEhOsgac4uJipKWlIS0tDcDfy8DT0tJw8eJFAH/fOhozZkxF/fPPP4/z589j1qxZOHXqFD755BN88803mDlzZkVNTEwMVqxYgbVr1+LkyZOYNGkSSkpKMH78eDnfChnRLY3W7MPNPx25XIRRqw6i4xvbEJ92xdjtEBFZJFlvUf3xxx946KGHKv4cExMDABg7dizWrFmDnJycirADAL6+vti6dStmzpyJjz/+GC1btsTKlSsRERFRUTNixAhcvXoVc+bMgVqtRnBwMBISEu6aeEyWYcvRbEzfeETWv8OtkS1aNW2Eh/2bY3yv1gCAtfuzcDArHzc15dDp9Lh2oxSX/rqFUgPOZtZogSkb07B6/wV8+3xPHv1ARGRAPKqB++CYrAlrUpB06qpBXsu1oS2cHGzh09QRNiolPJwc4dusIZ4J9YGdjfiFzFsaLZ778hD2Zl6HIe8w2dkosHhkZ0QGcH4OEVF1pHx+M+Aw4JikqI924IT6Zp1fx6WBDVJefVhSiBFx5+DOfZnX8MmucwZ73eWjuzDkEBFVgwHnHhhwTFvfhdtxMf9WnV9nfK9WmDskwAAd3Vufd5JwqeB2nV/HwUaB428N5O0qIqIqmO1OxmTdNOU6PPzBzjqHm7ZuDXFm/sB6CzcAsGd2GD4cHgzbOgaT2+V6RH64iyusiIjqiFdweAXHJMRtO4HPd2fV+cTv8I5uWDm2u0F6qg2tTo8D56/j/V9P4sil2h8JogSweGQwBge3MFxzRERmjldwyKzc2eOmLuGmkb0KS0d2Nmq4AQCVUoFebZth8+Q+iO7jW+vX0eHvFVbR6w4ZrjkiIivCgENGpSnX1WmPm56tm+Dr6B44OjcCg4O9DNhZ3b02yB+fPN0Fqjr8V5Z4Ig8Ltp4wXFNERFaCAYeMRqvTo9/C7bUe38rVERsm9kRom6YmOyk3KtATZ+ZHoct9zrV+jRV7sqAp1xmwKyIiy8eAQ0aRkJGDTm/+iuyi0lqND/B0wq5Z/Q3clTxUSgV+eKE3Tr4ViV5tXGv1GgM/3m3groiILBsDDtW7hIwcPP9VKm5qtLUa/1B7V8RP72PgruTnaKfC+uhQhPg0kTz23NUSDPpop+GbIiKyUAw4VK+0Oj1eWJ9a6/HhHZvji/+Y98GqXz7bo1bjjqtL0Hdh0r0LiYiIAYfq17u/nKjVEQeN7FRY8lRnrBz7gOGbqmd2Nko817d2K6wu5t/GuFUHDNwREZHlYcChejMv/gQ+33NB8rgWzg44+mYEhgSZ1iqpuoiN8sf4Xq1qNXbn2esYu/qggTsiIrIsDDhUL6LXHcKqvbVbDr7j5YdMdpVUXcwdEoD+fm61GrvrzDX0rcMKNCIiS8eAQ7KLT7uCxBN5tRob3cfX4AdlmpLV47ojsGXtdtO+mH8LgxfvMnBHRESWwXI/OcgkaHV6TNmYVqux4R3d8Nogf8M2ZIJ+ntIHi4cH1WpsRnYx5sVnGLgjIiLzx4BDsur/Xu1uo0zo6WP0Yxfq0yNdWuLM/IGozY24VXv/5EaARET/woBDshm8eDf+/Ou25HHRfXzxxiP3y9CRabOzUeLT0V1qNXYQNwIkIqqEAYdkMWFNCjKyb0get3RkZ6u4LVWdyABPfPK09JBz9moJ5sUfl6EjIiLzxIBDBheflo2kU1clj/vk6c4md2CmMUQFemJ6WDvJ41btvYCfUy/L0BERkflhwCGD0ur0mLHpiORxHw4PQlQgw80d08LawclBJX3cN0cRt42njxMRMeCQQT25fB/KJe5UHODVGI91aSlPQ2ZKpVRg4RO1W1n12e4sbEvPMXBHRETmhQGHDGbB1hNIvVgoaYy9Coif1lemjsxbZIAnlo/uAptabHI45etUaGtzJgYRkYVgwCGD0JTrsGKP9J2K0+ZGytCN5YgM8MSJt6R/j3R64IlP98nQERGReWDAIYN46Vvp827C/NzgaCd9nom1sbNRYlr/NpLHHblUiPi0KzJ0RERk+hhwqM62pefg56NqSWPuc3XAqnHWs5FfXU0P7wD7WhxZMfObo7xVRURWiQGH6iQhIwcvbEiVNKZfBzfsnhUmU0eWSaVU4OORwZLHlen0mL5R+tU1IiJzx4BDtabV6fHK9+mSxvi5N8Ka8bxyUxt3Jh1L/Y82Pj2HRzkQkdVhwKFaW7r9LApvlUsas3lyb5m6sQ6RAZ44NX+g5HE8yoGIrA0DDtWKVqfHZ7vPSxoT3rE5JxUbgJ2NEp883VnSmLNXSzjhmIisCgMO1crS7WdxU6MVrvd0tsfKsQ/I2JF1iQr0wnN9fSWN4YRjIrImDDgkmVanxxf7LgjXKwDserm/bP1Yq9gofzwW5ClcX6bT48nl+2XsiIjIdNRLwFm2bBl8fHzg4OCAkJAQpKSkVFvbr18/KBSKux6DBg2qqBk3btxdX4+M5IZx9WX/2WsouFUmXD+xry/sarHEme7t3SeDJdWnXizAvPgMeZohIjIhsn/qbNq0CTExMZg7dy5SU1MRFBSEiIgI5OXlVVn/ww8/ICcnp+KRkZEBlUqFJ598slJdZGRkpbqvv/5a7rdC+Ps4hjFfVB9Q/y26jy9io/xl7Mi62dkoERXgLmnMqr1/8qwqIrJ4sgecRYsWITo6GuPHj4e/vz+WL1+OBg0aYPXq1VXWu7q6wsPDo+KRmJiIBg0a3BVw7O3tK9U1adJE7rdi9aLXHcKKPVkQncUxrX9bvDaI4UZuS57uCqkXyKZvPML5OERk0WQNOBqNBocPH0Z4ePj//kKlEuHh4UhOThZ6jVWrVmHkyJFo2LBhped37tyJ5s2bo0OHDpg0aRKuX79e7WuUlpaiqKio0oOkiU/LRuKJqq+6VcWlgS2mh7eXsSO6Q6VUYOnTXSSNKdPpMe1rbgBIRJZL1oBz7do1aLVauLtXvoTu7u4OtfreW/unpKQgIyMDzz77bKXnIyMjsW7dOiQlJeHdd9/Frl27MHDgQGi1Va/qiYuLg7Ozc8XD29u79m/KCml1erz0/VFJY955vBNUtTgFm2onMsATjweLTzgGgK3HuAEgEVkuk575uWrVKnTq1Andu1fe+XbkyJF45JFH0KlTJwwdOhTx8fE4dOgQdu7cWeXrxMbGorCwsOJx6dKleujecixJOovbZWIfhPYqBZaP7oLIAGkftlR37zwRLHnM4CV7DN8IEZEJkDXgNGvWDCqVCrm5uZWez83NhYeHR41jS0pKsHHjRkyYMOGef0/r1q3RrFkzZGZmVvl1e3t7ODk5VXqQmISMHHyUdFa4fmAnT4YbI7GzUUreG+dMbjHi07Jl6oiIyHhkDTh2dnbo2rUrkpKSKp7T6XRISkpCaGhojWO//fZblJaWYvTo0ff8ey5fvozr16/D05MfrIak1ekx+/tjksYM69JSpm5IRGyUP6L7+EgaM20TJxwTkeWR/RZVTEwMVqxYgbVr1+LkyZOYNGkSSkpKMH78eADAmDFjEBsbe9e4VatWYejQoWjatGml54uLi/Hyyy/jwIEDuHDhApKSkvDoo4+ibdu2iIiIkPvtWJWl2zMl7Xdjb6NEz7bNZOyIRLw26H58ODxIuF6nBz5MPCVjR0RE9c9G7r9gxIgRuHr1KubMmQO1Wo3g4GAkJCRUTDy+ePEilMrKOev06dPYu3cvfvvtt7teT6VSIT09HWvXrkVBQQG8vLwwYMAAzJs3D/b29nK/Hauh1enx4e9nJI35cHgwJxabiMe6tMSnuzJxJrdEqH7pjvOY+bAf//cjIouh0Ov1VndtuqioCM7OzigsLOR8nGo8v+4QEiQsCx8c6Cl5qTLJS1OuQ/vXfxGu79rKBd9P6iVjR0REdSPl89ukV1GRcWjKdZLCjQLAxyOlnW5N8rOzUcLFUfwi7eE/C3BLwgGqRESmjAGH7vLqD+mS6ju1cOKtDRP13INtJNUHvpkgUydERPWLAYcq0er0+FHisuEN0TWviCPjmdC7taT6Mh0w5ydpK+eIiEwRAw5VEr5oB8olLBl2b2yHRg6yz1WnWrKzUWJwoLTtE9YlX+QOx0Rk9hhwqMKza1OQde2WpDF7XgmTqRsylI9HdoajrbRbiIMXc4djIjJvDDgEALil0eL3k1cljXmury/spB5jTfVOpVTgwxHSJoGfySvGgq0nZOqIiEh+/HQiAMDDi3ZKqu/c0gmxUf7yNEMGFxngiQ+eCJQ0ZsWeLN6qIiKzxYBDuKXR4nLBbUljXoroKFM3JJdh3bzh3cRB0pjZEk+RJyIyFQw4hIc/3CmpvpG9DXq0aXrvQjI5e14Jg51KfD7OlvQcnlNFRGaJAcfKLdh6HJf/knb1ZuGwQO57Y8a+GNdduLZMq0dKVr6M3RARyYMBx4ppynVYseeCpDGhvq6IkrjsmExLjzZNYa8Sr/98d6Z8zRARyYQBx4qt2ntO8pi1E0Jk6ITqk0qpwAdPBgvX7zh9jSuqiMjsMOBYsc92Sgs4QwI9uCzcQgwOboHwjm7C9Sv2ZGFbeo6MHRERGRY/razUtvRsFNwWP1jRRgl8NJKnhVuSlWO7w6dpA+H66ZuOcMIxEZkNBhwrpNXpMeXrI5LGfDS8MycWW6DBgV7CtWVaPfaelbYZJBGRsTDgWKHdJ/Mg5Rfxzt5OGBws/kFI5iNU4nL/5RJvaxIRGQsDjhV66fs04VqlAvhuUm/5miGj6tG6KewlzKtKv1IgXzNERAbEgGNlotcdwvWb5cL1S5/qwltTFkylVGDSg22E60s0Ok42JiKzwIBjReLTspF4Ik+4vpGdinveWIGpYe0k/SB4YUMqEjIYcojItDHgWAmtTo+XJJ4r9EI/8d/syXyplAp8NDxI0piYb45yRRURmTQGHCtx4Px13C6TdjL0s30ZcKzFI11aoo2b+JLxmxotFiedkbEjIqK6YcCxEsnnrkuqf66vLzf1szK/zewHKdOtVu7J4lUcIjJZ/ASzEpsOXRSundDbB7FR/jJ2Q6ZIpVTg8c4thetLNFoexElEJosBxwr8d8sxXC3WCNUOvN8dbwy+X+aOyFS9/XgnSfUvfSNtw0giovrCgGPhtqVn44t9YldvlApg6aiuMndEpszORonoPj7C9VcKS/FT2hX5GiIiqiUGHAum1enx8vfpwvUd3BtzzxvCa4Puh2sDW+H66RvTOBeHiEwOA44FO3DuOkpKxQ/U7ObTRMZuyJwMkrj/0YeJp2XqhIiodhhwLNj7v52SVP8qJxbT/5P6b2HZjnO8ikNEJoUBx0JpynU4cqlQuD68Y3M42qlk7IjMiaOdCi2cHYTr9QCmfX1YvoaIiCRiwLFQr/4gPvemdbMGWDn2ARm7IXP0+4v9JNVvPZYLTbm0zSSJiORSLwFn2bJl8PHxgYODA0JCQpCSklJt7Zo1a6BQKCo9HBwq/yap1+sxZ84ceHp6wtHREeHh4Th79qzcb8NsJGTk4LtUsZUtCgCJMf1k7YfMk6OdCg/5uUkas3Z/lkzdEBFJI3vA2bRpE2JiYjB37lykpqYiKCgIERERyMur/tBHJycn5OTkVDz+/PPPSl9fuHAhFi9ejOXLl+PgwYNo2LAhIiIicPv2bbnfjsnT6vSYskF8b5JhXVpy5RRV64tx3eHkYCNc/9WBP+9dRERUD2QPOIsWLUJ0dDTGjx8Pf39/LF++HA0aNMDq1aurHaNQKODh4VHxcHd3r/iaXq/HRx99hNdffx2PPvooAgMDsW7dOmRnZ+PHH3+U++2YvA9/O41ywcmeCkjf2I2sz5E5A4Rr/8y/hW3p2TJ2Q0QkRtaAo9FocPjwYYSHh//vL1QqER4ejuTk5GrHFRcXo1WrVvD29sajjz6K48ePV3wtKysLarW60ms6OzsjJCSkxte0BlqdHp/sPCdc39/PjedN0T2plApE+Lvfu/D/zfo+nSuqiMjoZP10u3btGrRabaUrMADg7u4OtVpd5ZgOHTpg9erV+Omnn/DVV19Bp9OhZ8+euHz5MgBUjJPymqWlpSgqKqr0sETTNx6BlCmez/bhaeEkZkyoj3BtcakWS7dzThwRGZfJ/foeGhqKMWPGIDg4GA8++CB++OEHuLm54bPPPqv1a8bFxcHZ2bni4e3tbcCOTYOmXIf49Bzhelsl0N3XVcaOyJL0aNMUDe3FtxFYsj2TV3GIyKhkDTjNmjWDSqVCbm5upedzc3Ph4eEh9Bq2trbo3LkzMjMzAaBinJTXjI2NRWFhYcXj0qVLUt+Kyftin7TVK8892IaTi0mYSqnAe8MChevLdXpM+5oHcRKR8cgacOzs7NC1a1ckJSVVPKfT6ZCUlITQ0FCh19BqtTh27Bg8Pf/eOt7X1xceHh6VXrOoqAgHDx6s9jXt7e3h5ORU6WFpPtmRKVxrowRmPtxBxm7IEkUFeiG6j69w/dZjOdwXh4iMRvZbVDExMVixYgXWrl2LkydPYtKkSSgpKcH48eMBAGPGjEFsbGxF/VtvvYXffvsN58+fR2pqKkaPHo0///wTzz77LIC/V1jNmDED8+fPx88//4xjx45hzJgx8PLywtChQ+V+OyZpy9FsFN4uF65f+nQXXr2hWnltkD+8JOxwvEbilUUiIkMR3+CilkaMGIGrV69izpw5UKvVCA4ORkJCQsUk4YsXL0Kp/F/O+uuvvxAdHQ21Wo0mTZqga9eu2L9/P/z9/3c2zqxZs1BSUoKJEyeioKAAvXv3RkJCwl0bAloDrU6PmG/ShOsf6+yFyABpBykS/dP9LZyQXSi251RCRg4mPsjJ7ERU/xR6vd7qZgIWFRXB2dkZhYWFZn+7as/pq3jmi+p3hv63M/MHcmk41cmK3eexYNtJoVpbJXD27UEyd0RE1kLK5zc/6czc90cuC9f6ezZmuKE6G9vTR7i2TAc8u1Y8gBMRGQo/7cxc2sW/hGsn9BKfIEpUHTsbJTp6NBKu//3kVWw5yt2Niah+MeCYsW3pObiQf0u43qtJAxm7IWvywwu9JdXHbDrCfXGIqF4x4JgprU6P13/KEK53b2zHjf3IYBztVOjWykW4vkwH7M+8Jl9DRET/woBjplKy8pFfohGu/++jAVwaTga16bmekPIv6vtU8fliRER1xYBjplbuET9Uc2Z4Oy4NJ4NTKRV4rLOXcP3lv27K2A0RUWUMOGZIU65D0qmrQrWuDW0xpX87mTsia/XOsCBjt0BEVCUGHDM0eMlu4dr5vDVFMrKzUSIqwF2o9o8/CxC37YTMHRER/Y0Bx8zc0mhxJrdEqNbfqzGiAsVvIRDVxpKnu8JGMER/tjsL29K5ZJyI5MeAY2Ye/2SvcO2wzi1l7ITobyqlAkuf7ixcP+v7dC4ZJyLZMeCYkW3pOTipLhaufybUR75miP4hMsATAwM8hGqLS7U4cP66zB0RkbVjwDETUve9aeHiwGMZqF61cWsoXPtewikZOyEiYsAxG1L3vXn7sU4ydkN0t9DWzYRr0y4XYlt6jozdEJG1Y8AxE1L2vbFVKdC7nZuM3RDdrUebprBXia/Ym76RxzcQkXwYcMyAlH1vAODjEZ25NJzqnUqpwKR+bYTry3R67D0j/u+aiEgKBhwz8MyqA8K10X18EBXIXYvJOKaGtYeDrfiPldjNx2TshoisGQOOiduWno2DWX8J1bZydcRrg+6XuSOi6qmUCix6Mli4PrvwNjTlOvkaIiKrxYBjwqSunOpyXxMZuyESExXoiY4ejYTrY39Il7EbIrJWDDgm7O+VU2XC9Y934cZ+ZBp+eKG3cO33qVc42ZiIDI4Bx4SpC28J1zrYKtGzrfgyXSI5Odqp4OlkL1w/7JM9MnZDRNaIAceEbTp0Ubh20ZPBXDlFJmXhE+InjaddvoFbGq2M3RCRtWHAMVGach0OCE4uDvZ25sopMjlSryiGL9opTyNEZJUYcEzUqxImXvZuy039yPSolAp0cBc/vuFKwW1exSEig2HAMUEJGTn4LvWKcH1om6YydkNUe69G+kuqX7D1uEydEJG1YcAxMVqdHjGb0oTrHWyV6NGaAYdMU+8ObpJ+yGw6dFm2XojIujDgmJj9mddws0x847P3hwVxcjGZLJVSgWlhbYXry3R6/PjHJRk7IiJrwYBjYqZ+nSpcG9DCCYODvWTshqjupoa1h4ON+I+amO/TuS8OEdUZA44Jmb/lBApulQvXvxYlbX4DkTGolAosGh4sXK/TA/vPXpOvISKyCgw4JkJTrsOqfVnC9XYqBbr7usrYEZHhRAV6olVTR+H6KRvFr2QSEVWFAcdEfJl8AVIuyj/k15xzb8isbJ3aV7i28FY5fjoivpKQiOjfGHBMRNb1Ekn1Y0J95GmESCaNHGzgIeH4humb0jgXh4hqrV4CzrJly+Dj4wMHBweEhIQgJSWl2toVK1agT58+aNKkCZo0aYLw8PC76seNGweFQlHpERkZKffbkNXhC/nCtS6ONlwaTmZp96z+kuo//v2MTJ0QkaWTPeBs2rQJMTExmDt3LlJTUxEUFISIiAjk5eVVWb9z50489dRT2LFjB5KTk+Ht7Y0BAwbgypXKl6sjIyORk5NT8fj666/lfiuy0ZTrcFJdLFz/zrBA3p4is2Rno4S9jfi/3eW7zvEqDhHViuwBZ9GiRYiOjsb48ePh7++P5cuXo0GDBli9enWV9evXr8cLL7yA4OBg+Pn5YeXKldDpdEhKSqpUZ29vDw8Pj4pHkyZN5H4rshm8ZLdw7cAAd0QG8NwpMl9d7nMRrtVo9Thw/rp8zRCRxZI14Gg0Ghw+fBjh4eH/+wuVSoSHhyM5OVnoNW7evImysjK4ulZeMbRz5040b94cHTp0wKRJk3D9evU/BEtLS1FUVFTpYSq2HM3GmVzx+TejQ3zka4aoHjzfR3zjPwDYc/aqTJ0QkSWTNeBcu3YNWq0W7u7ulZ53d3eHWq0Weo1XXnkFXl5elUJSZGQk1q1bh6SkJLz77rvYtWsXBg4cCK226oP64uLi4OzsXPHw9vau/ZsyIK1Oj1c3HxOut1Mp0IPnTpGZ693BDXYq8dtUxy4XytgNEVkqk15F9c4772Djxo3YvHkzHBwcKp4fOXIkHnnkEXTq1AlDhw5FfHw8Dh06hJ07d1b5OrGxsSgsLKx4XLpkGlvBp2Tl48Zt8Y39nu5+H+fekNlTKRVY/FRn4frLf92UsRsislSyBpxmzZpBpVIhNze30vO5ubnw8PCocez777+Pd955B7/99hsCAwNrrG3dujWaNWuGzMzMKr9ub28PJyenSg9T8NsJsatYd0Rw7g1ZiMgATwzrInbMyJ/5txC37YTMHRGRpZE14NjZ2aFr166VJgjfmTAcGhpa7biFCxdi3rx5SEhIQLdu3e7591y+fBnXr1+Hp6f5BACtTo/1B/4Urvd0duDOxWRR4h4PgkLwguRnu7OgKRc/hJaISPZbVDExMVixYgXWrl2LkydPYtKkSSgpKcH48eMBAGPGjEFsbGxF/bvvvos33ngDq1evho+PD9RqNdRqNYqL/15GXVxcjJdffhkHDhzAhQsXkJSUhEcffRRt27ZFRESE3G/HYJYknYVGK778de4Qf96eIotiZ6PExD6+wvVjVh2QsRsisjQ2cv8FI0aMwNWrVzFnzhyo1WoEBwcjISGhYuLxxYsXoVT+L2d9+umn0Gg0eOKJJyq9zty5c/Hmm29CpVIhPT0da9euRUFBAby8vDBgwADMmzcP9vbiu6Qak1anx8q954XrZ4a359JwskixUf5IOJ6LP6/fe57Ngay/oCnXwU7CyeREZL0Uer3e6nbRKioqgrOzMwoLC40yHyf53HU8tULst9EGdkocezOSV2/IYs3YeAQ/pmUL1b4W5Yfovm1k7oiITJWUz2/+KmQEeTduC9dG92nDcEMWbVjnlsK1Px8VC0JERAw4RtC8scO9iwDY2ygxLaydzN0QGVfPds0gGuGPXSni0Q1EJIQBxwi2nxJbHv7h8CBevSGLp1Iq0MlL/Fbxk8v3y9gNEVkKBpx6pinXYdXeC/ese7aXD6ICxfYJITJ3Q4LF/62nXixAvOCcHSKyXgw49ezL5AsQucLu6eIofzNEJmJsT/Hl4gDw0vdHeauKiGrEgFPPzl8TO1jzz3xuT0/Ww85GicGdxLdCuF2m4ynjRFQjBpx6tGDrCaw/eFGotpVrA5m7ITItHz/VGVKmnCWfY8Ahouox4NST6HWHsGJPllCtUgE8E+ojb0NEJkalVGDKQ22F67emcx4OEVWPAacexKddQeKJPOH66D6+3K2VrNL08PYQ/aefdf0mbmm08jZERGaLn6Iy0+r0mL35mFCtAsBzfX0RG+Uvb1NEJkqlVGDxyM7C9W/zlHEiqgYDjsxSsvJRXCr2W2aYnxvDDVm9qEAvtHAR2wxz+ynxK6NEZF0YcGSmLrwlXNvIwVbGTojMR3+/5kJ1VwpuI45XcYioCgw4MluYcFq4VsqZPESW7FUJVzI/35MFTblOxm6IyBwx4Mho/pYTyCkSO1jTzkaJnu2aydwRkXlwtFPhYX+xqzh6PfDqD2Lz3IjIejDgyERTrsPKfWLLwgFgcj+eGk70TyvGPADvJmJzcX44cpk7GxNRJQw4MpHyG6W9SoEp/XlqONG/jRM8wkGnBz7+Xfx2MBFZPgYcGWh1evx89Ipw/ZCgFrx6Q1QFKRteLtl+jldxiKgCA44MDpy/Do1W/Aft2493krEbIvNlZ6NEE0ex1YV6AHvPXJW3ISIyGww4Mnj1h3Th2qCWTty1mKgGrd0aCtdy4z8iuoOfrAZ2S6PFn/nie9/0aSe2UoTIWkX4ewjXns4r4W0qIgLAgGNwUn+DDG3TVKZOiCzDuN5iE43vWJJ0RqZOiMicMOAY2KEL+cK1NkoFerRmwCGqiZ2NEgPvdxeuX7Ijk1dxiIgBx5C0Oj3OqIuF6yf1bc3VU0QClo7qCtH/UrQ6TjYmIgYcgxrx2X6IbhivVAAzBnSQtR8iS6FSKjA9THyvqLhfONmYyNox4BjILY0Wf/xZIFw/Pawdr94QSTBVQsA5lcvJxkTWjgHHQJ778g/h2kb2Ku5cTCSRSqmAn3sj4fppX6fK2A0RmToGHANIyMjB7rPXhOsXDgvk1RuiWogd2FG4dtsxNU8ZJ7JiDDh1pNXp8d8t4vf7vZwdEBXoJWNHRJard3s3qAR/N9AD+DL5gpztEJEJY8Cpo5SsfOQU3haujxvKYxmIakullHYw7Rf7LsjXDBGZNAacOsq7IR5u7GyU6N3BTcZuiCzftLB2UAn+5LpccAu3NFp5GyIik8SAU0fNGtkL1y4eGcy5N0R1pFIqMDrkPuH6xz/dJ2M3RPRvWp0eyeeu46e0K0g+d91oKxrrJeAsW7YMPj4+cHBwQEhICFJSUmqs//bbb+Hn5wcHBwd06tQJ27Ztq/R1vV6POXPmwNPTE46OjggPD8fZs2flfAvV0gmeGj61f1tEBnjK3A2RdYgMEJ/HdjLnBicbE9WT+LQrCH7rVzy14gCmb0zDUysOoPe725GQkVPvvcgecDZt2oSYmBjMnTsXqampCAoKQkREBPLy8qqs379/P5566ilMmDABR44cwdChQzF06FBkZGRU1CxcuBCLFy/G8uXLcfDgQTRs2BARERG4fVv8dpGhHLxwXahOr+eeHESG0t3XFY624j++Zn9/VMZuiAgAotcdwpSNabhxu/Jt4ZzC25j0VWq9hxzZA86iRYsQHR2N8ePHw9/fH8uXL0eDBg2wevXqKus//vhjREZG4uWXX0bHjh0xb948dOnSBUuXLgXwd1D46KOP8Prrr+PRRx9FYGAg1q1bh+zsbPz4449yv50qiN5y4q0pIkNRKRV497FA4fqf0rK58R+RjBZsPYHEE1VfuAD+XtX43y0n6vW/Q1kDjkajweHDhxEeHv6/v1CpRHh4OJKTk6sck5ycXKkeACIiIirqs7KyoFarK9U4OzsjJCSk2tcsLS1FUVFRpYehiJ4GzlPDiQzrkS4t0EDwKo5WDyzdbpzb2ESWTlOuw8o9Wfesyym8jZQs8QOp60rWgHPt2jVotVq4u1c+Cdjd3R1qtbrKMWq1usb6O/9XymvGxcXB2dm54uHt7V2r91OVHq2bwqWBbY01TRrY8tRwIhlMD28vXLtkO08ZJ5JD7A/pEP0vS8rK47qyilVUsbGxKCwsrHhcunTJYK+tUirwzuM1720T93gnrp4iksH4Xr7CteU6PYZ/tl/GboisT0JGDr5PvSJc37yxg4zdVCZrwGnWrBlUKhVyc3MrPZ+bmwsPD48qx3h4eNRYf+f/SnlNe3t7ODk5VXoYUmSAJ5aP7gIPp8r/w3k6O2D56C5cPUUkEzsbJaL7iIecw38WcF8cIgORupO/g60S3X1dZeyoMlkDjp2dHbp27YqkpKSK53Q6HZKSkhAaGlrlmNDQ0Er1AJCYmFhR7+vrCw8Pj0o1RUVFOHjwYLWvWR8iAzyxb3Z/fB3dAx+PDMbX0T2w95X+DDdEMnttkD+6tnIRrl+wVfwHMhFVT+pO/tF9Wtfr3QzZb1HFxMRgxYoVWLt2LU6ePIlJkyahpKQE48ePBwCMGTMGsbGxFfXTp09HQkICPvjgA5w6dQpvvvkm/vjjD0yZMgUAoFAoMGPGDMyfPx8///wzjh07hjFjxsDLywtDhw6V++3USKVUILRNUzwa3AKhbZrythRRPfnmuZ7C6xR/ShO/nE5E1ft89znhWlulAjMkzJkzBBu5/4IRI0bg6tWrmDNnDtRqNYKDg5GQkFAxSfjixYtQKv+Xs3r27IkNGzbg9ddfx6uvvop27drhxx9/REBAQEXNrFmzUFJSgokTJ6KgoAC9e/dGQkICHBzq794eEZkOlVKBDu6NcCq3+J61N0q1uKXRwtFOVQ+dEVkmTbkOO05fFa4fE9qq3n/pV+itcAe6oqIiODs7o7Cw0ODzcYjIOHadzMPYtYeEakeHeGO+hH10iKiy4cv3IeVCgXD919E9DLJdipTPb6tYRUVElk/KQbY/pWXL2AmRZdOU6ySFG9eGtvU6ufgOBhwisggqpQJ+7o2Eau/cpiIi6dbuvyCpfv6jAUaZk8qAQ0QWI3ZgR+HaYTxlnKhW1h+4IFxrp1IgKlD8cFxDYsAhIovRu72b8GqqEzk3sC2dt6qIpNCU63Ah/5ZwfRcJWzgYGgMOEVkMlVKBvu2bCdfHfHuUxzcQSbB2/73PnPqnSX3bytTJvTHgEJFFWT66m3Dt7TId9mdek7EbIsvy+e7zwrV2Ngr0bi8++d/QGHCIyKI42qkk7Wz85s8Z8jVDZEEWbD2Bq8Ua4frFIzsbdcNbBhwisjhSdjY+d+0mNOU6WfshMneach1W7BG/PTWtf1ujH1XEgENEFkelVMC3WQPheqnzCoiszezv04VrbZTA9Ho+lqEqDDhEZJFGPnCfcK2UeQVE1kar0+Ono+IrDh/29zCJsxgZcIjIIo3r5Stce7VYgwVbj8vYDZH5OnD+uqTVhqN7tJKxG3EMOERkkexslIgK8BCuX7HnAufiEFVh31nxlYaOtkr0aF33M6cMgQGHiCzWkqe7CE82BoAvky/I1QqR2frx6BXh2oVPBJnE7SmAAYeILJhKqUCkhKs4n+w8J2M3ROZnW3o2sgtuC9U2a2iLIUHGOZahKgw4RGTRpMwHuF6iQfHtchm7ITIfWp0eM785Klz/3INtZOxGOgYcIrJoPVo3RSN7lXD9jE1HZOyGyHwsTjqDUgnz0sb2FJ/YXx8YcIjIoqmUCgzv5i1cf1p9Q8ZuiMyDVqfH8l3i2ycMDvSEnY1pRQrT6oaISAYP+4vPw7ml4S0qogPnrgtfvbFVKvDxyM4ydyQdAw4RWbzuvq5wshf7cXetpAzb0nNk7ojItK1LFt/d+5FgL5NZOfVPDDhEZPFUSgXeGRYkXB/zbZqkjc2ILIlWp8dvJ/KE63u3M96J4TVhwCEiqxAV6IUwP7EfxLfLdJj2darMHRGZpv1nr0FKvPdwcpCtl7pgwCEiq/FsH/FlrFuPqXmriqzSx0mnhWsb2SvR3ddVxm5qjwGHiKxGd19XNLQTXzI+67ujvFVFVkWr0+PwxULh+nceCzTJ+TcAAw4RWRGVUoFn+7QWri/WaLF0e6aMHRGZlhkbU4VvTzk72mBwcAtZ+6kLBhwisirTwtrBRsJvnF/sz+JVHLIKmnId4tPVwvWT+7WVsZu6Y8AhIquiUiow5SHxH8wFN8uQkpUvY0dEpuHL5AuSJheP62VaOxf/GwMOEVmdqWHtJM3FURfekrEbItPwfar4qeHRfXxNbufifzPt7oiIZKBSKvDBcPF9cfZmXpOxGyLj25aejRM5RUK17Zs3wmuD/GXuqO4YcIjIKkUGeCLyfrEjHLYczeY8HLJYWp0er/+UIVwfP62PjN0YDgMOEVmtts0bCtVptHpM38hTxskypWTlI7+kTKg2zM/N5G9N3WEeXRIRySC0dTPh2vj0HG78Rxbpt+Pi/66lbJZpbLIGnPz8fIwaNQpOTk5wcXHBhAkTUFxcXGP91KlT0aFDBzg6OuK+++7DtGnTUFhYedMhhUJx12Pjxo1yvhUiskA92jRFI3vxycZv/JTBW1VkUbQ6Pb4+dEmo1snBxmR3La6KrAFn1KhROH78OBITExEfH4/du3dj4sSJ1dZnZ2cjOzsb77//PjIyMrBmzRokJCRgwoQJd9V+8cUXyMnJqXgMHTpUxndCRJZIpVRg4bBA4frrJRouGSeLsnR7Jm6X6YRqh3VpYbK7FlfFRq4XPnnyJBISEnDo0CF069YNALBkyRJERUXh/fffh5eX111jAgIC8P3331f8uU2bNliwYAFGjx6N8vJy2Nj8r10XFxd4eIhNECQiqk5UoBcGZ6gRL3j7iUvGyVJodXos2ym+U/eA+z1l7MbwZLuCk5ycDBcXl4pwAwDh4eFQKpU4ePCg8OsUFhbCycmpUrgBgMmTJ6NZs2bo3r07Vq9eDb2++svGpaWlKCoqqvQgIrrj45Gd4WAj9pvppj/ELucTmbrFSWegKRe7etO0oZ1Z3Z4CZAw4arUazZs3r/ScjY0NXF1doVaLbQV97do1zJs3767bWm+99Ra++eYbJCYmYtiwYXjhhRewZMmSal8nLi4Ozs7OFQ9vb2/pb4iILJZKqcBT3e8Tqj1wPp+TjcnsaXV6LN91Xrh+3qMBZnV7CqhFwJk9e3aVk3z/+Th16lSdGysqKsKgQYPg7++PN998s9LX3njjDfTq1QudO3fGK6+8glmzZuG9996r9rViY2NRWFhY8bh0ib+BEVFlUi6/x3ybxsnGZNYOnLuOUsGrN529XRAVaF63p4BazMF58cUXMW7cuBprWrduDQ8PD+Tl5VV6vry8HPn5+fecO3Pjxg1ERkaicePG2Lx5M2xtbWusDwkJwbx581BaWgp7e/u7vm5vb1/l80REd3T3dYWLoy0Kbt17P5DbZTosSTqDGQ93qIfOiAzvq4MXhGtfijDPf+eSA46bmxvc3NzuWRcaGoqCggIcPnwYXbt2BQBs374dOp0OISEh1Y4rKipCREQE7O3t8fPPP8PBweGef1daWhqaNGnCEENEtaZSKjC+ly8+/P2MUP2ynecwNay92V22J9Lq9Eg8kXfvQgAOtkr0aN1U5o7kIdscnI4dOyIyMhLR0dFISUnBvn37MGXKFIwcObJiBdWVK1fg5+eHlJQUAH+HmwEDBqCkpASrVq1CUVER1Go11Go1tFotAGDLli1YuXIlMjIykJmZiU8//RRvv/02pk6dKtdbISIrMaV/W9iqxAJLmVaP/TyjiszQ/rPXUC54i/X5vm3MNsTLtkwcANavX48pU6YgLCwMSqUSw4YNw+LFiyu+XlZWhtOnT+PmzZsAgNTU1IoVVm3btq30WllZWfDx8YGtrS2WLVuGmTNnQq/Xo23btli0aBGio6PlfCtEZAVUSgXC/NyQcFzst9v/bjmO31/sJ29TRAb2XeploTqVApga1k7mbuSj0Ne0vtpCFRUVwdnZuWIJOhHRHfvOXsOoVeJbWXzydBeznIBJ1ikhIweT16dCK/DJ36ZZQyS91E/2nqSQ8vnNs6iIiP6hR5umkHKWYOwPR7miisxCQkYOnv9KLNwAQKcWzvI2JDMGHCKif1ApFXihX9t7F/6/wttaHt9AJk+r02PmpjRJY4Z1aSlPM/WEAYeI6F+mh7eXVJ9347ZMnRAZxpKks7gleOYUADSwU6Fnu2YydiQ/Bhwion9RKRUIbCE+P29d8gX5miGqI61Oj2U7xM+cAoBFw4PMdvXUHQw4RERV2BAdKlx7+M8C3NJoZeyGqPYWJ51BmYR5YjPD2yEywPwnzjPgEBFVoZGDDQJbil/FCV+0U75miGpJq9Nj5Z4s4frGDjaY0t98l4b/EwMOEVE1fp7SBw6CS6quFNzmVRwyOSlZ+SiR8O/yiS4tzf7W1B0MOERENejaqolw7fNf/SFjJ0TSSZ0AP+D+ms+KNCcMOERENZjYu7Vw7b7M69wTh0xK4olc4VpPZwd093WVsZv6xYBDRFSD3h3cIHrFvlynx4Hz1+VtiEiQplyHbcdyhOvnDvG3mNtTAAMOEVGNVEoFPnoySLh+0nrepiLT8GXyBYhcULRTKbB8dBeLWDn1Tww4RET38EiXlmje2E6otuiWFvN+zpC5I6J7W7P/glDd8Ae8LS7cAAw4RERCPhzeWbh21f4/oSkX3zWWyNCi1x3Cpb9uCdX6Nm0oczfGwYBDRCRA6iGcX3J3YzKSWxotEk/kCdUqFcAzoT7yNmQkDDhERAJUSgV6thE/m+eTndK2xicyFCnbFUR18oSdlORuRizzXRERyeCzZ7oJ114vKcP8Lcdl7IboblqdHvsyxVfyPezvLmM3xsWAQ0QkyNFOhTA/N+H6lfsucC4O1aul28+iXMJeTM0bO8jYjXEx4BARSbBqXHfYqcT3Con9IV3Gboj+R6vTY5mEW6Puje0samO/f2PAISKSKLCls3Dtj0eucHdjqhdLks5CUy7+b+2/jwZY1MZ+/8aAQ0Qk0YCO4uf1aPXAkqQzMnZD9PfVm093nROuH9+zlUXuffNPDDhERBKN6+0rqX7x9kxexSFZzdiYilIJ870G3G/Z4QZgwCEikszORokeEuYu6PTA3jNXZeyIrNm29GxsSVcL17s2tLXouTd3MOAQEdXCugkhkupjNx+TqROyZlqdHi99d1TSmPkWPvfmDgYcIqJasLNRIrqPj3B9duFtLhkngztw7jpuasT/XUX38UFUoJeMHZkOBhwiolp6bdD9aNLAVrh+4Ee7ZOyGrNH+89eEa1u5OuK1QffL2I1pYcAhIqqDj4YHC9eeu3YTW45my9cMWZ1fM8Tn3ozu4SNfIyaIAYeIqA56txff2RgAXvwmjSuqyCC2pecg82qJcP3Ynj7yNWOCGHCIiOpApVSgk1dj4XqNVo8lSWdl7IisgVanx6zvxXfJDm7pbLGHalbHut4tEZEMvp7YU1L9yr1ZvIpDdXLg3HUUl5YL178c6SdjN6aJAYeIqI4aOdjA08leuL64tBwpWfkydkSW7v3fTgrXNrK3QY/WTWXsxjQx4BARGcCuWf0l1f+akSNTJ2TpNOU6HLlUJFy/cFigVex782+yBpz8/HyMGjUKTk5OcHFxwYQJE1BcXFzjmH79+kGhUFR6PP/885VqLl68iEGDBqFBgwZo3rw5Xn75ZZSXi1+qIyIyNKn74mz64xJvU1GtPLhwh3BtsLcTogIt/1iGqsgacEaNGoXjx48jMTER8fHx2L17NyZOnHjPcdHR0cjJyal4LFy4sOJrWq0WgwYNgkajwf79+7F27VqsWbMGc+bMkfOtEBHd02uD7kdLFweh2ltlOt6mIsnmxR9HTtFt4fqXB3SUsRvTJlvAOXnyJBISErBy5UqEhISgd+/eWLJkCTZu3Ijs7Jr3gWjQoAE8PDwqHk5OThVf++2333DixAl89dVXCA4OxsCBAzFv3jwsW7YMGo1GrrdDRCRkfC/xgzgXbD0uYydkaTTlOqzae0G43tFWiR5trG/uzR2yBZzk5GS4uLigW7duFc+Fh4dDqVTi4MGDNY5dv349mjVrhoCAAMTGxuLmzZuVXrdTp05wd3eveC4iIgJFRUU4frzqHxalpaUoKiqq9CAiksMzoT5QCE53yMi+gXnxJ+RtiCzG7O+lnTm18Ikgq5x7c4dsAUetVqN58+aVnrOxsYGrqyvU6up3Xnz66afx1VdfYceOHYiNjcWXX36J0aNHV3rdf4YbABV/ru514+Li4OzsXPHw9vau7dsiIqqRnY0Sz/YWv4qzam8WtqVzd2OqmVanx5Z08YnpHk72GBJkHWdOVUdywJk9e/Zdk4D//Th16lStG5o4cSIiIiLQqVMnjBo1CuvWrcPmzZtx7ty5Wr9mbGwsCgsLKx6XLl2q9WsREd3La4P80bKJ2FwcAHj9pwxOOKYapWTlo0wr/m/kvSeCZOzGPNhIHfDiiy9i3LhxNda0bt0aHh4eyMvLq/R8eXk58vPz4eHhIfz3hYSEAAAyMzPRpk0beHh4ICUlpVJNbm4uAFT7uvb29rC3F9+jgoiorsb39MW8rWJ7leSXlCElKx+hVjxfgmqmLrwlXGujBHq2bSZjN+ZBcsBxc3ODm9u9z14JDQ1FQUEBDh8+jK5duwIAtm/fDp1OVxFaRKSlpQEAPD09K153wYIFyMvLq7gFlpiYCCcnJ/j7+0t8N0RE8ngm1Afzt56E6O/cv59QM+BQtTYduihcOyqklVXPvblDtjk4HTt2RGRkJKKjo5GSkoJ9+/ZhypQpGDlyJLy8/r4veOXKFfj5+VVckTl37hzmzZuHw4cP48KFC/j5558xZswY9O3bF4GBgQCAAQMGwN/fH8888wyOHj2KX3/9Fa+//jomT57MqzREZDLsbJR4VsK+OKv2XUACN/+jKkSvO4QDWX8J10cGWOe+N/8m6z4469evh5+fH8LCwhAVFYXevXvj888/r/h6WVkZTp8+XbFKys7ODr///jsGDBgAPz8/vPjiixg2bBi2bNlSMUalUiE+Ph4qlQqhoaEYPXo0xowZg7feekvOt0JEJNlrg+5HeEfx08Znf3+Mc3Gokvi0bCSeyLt34f9zbWiL7r6uMnZkPhR6vd7q/msqKiqCs7MzCgsLK+2xQ0Qkh8gPd+FUbs27uN+xfkIIerXj/An6e+VU+9e2QcLcYnzydGdEBVru6ikpn988i4qISGa2NuI/apPPX5OxEzInTy7fJyncPNfX16LDjVQMOEREMnN2sBWu3ZvJgENAfNoVpF4sFK5/wKcJYqO40OafGHCIiGQW3ae1cG3apUJsk7ChG1kerU6Pl75PlzRmWv92MnVjvhhwiIhk1ru9G1QSVu1O/TqVk42t2IFz13G7TCdcrwT3vakKAw4RkcxUSgWmhbUXrtfqgakbUmXsiEzZvnNXJdU/1rkF972pAgMOEVE9mNK/LRrZq4Trt2WooSkX/y2eLMehrHxJ9XHDAmXqxLwx4BAR1QOVUoGFEj+Ixq4+IFM3ZKoSMnJw6M8C4frn+vrCTsIqPWvC7woRUT2JCvRCD98mwvXJ5//iSeNWRKvTI+abo8L1nb1duHKqBgw4RET1aN2EHpLqZ35zlBOOrcT0jUdwU6MVrn8pooOM3Zg/BhwionpkZ6PEhN4+wvWl5TosSTorX0NkEjTlOsRL2B6gSQNb9GjNw1lrwoBDRFTP3hh8PzydxQ8HXrYjk1dxLNygxbsl1cc93okrp+6BAYeIyAhmRXYUri3T6TF1wx8ydkPGFJ+WjbN5JcL1nzzdmSeGC2DAISIyAg8nB0n12zLyELfthEzdkLFodXrM/DZNuP4BnyY8b0oQAw4RkRF093WVHHI+253FvXEszJKksyiTcKLmtId4JIMoBhwiIiNQKRV48xHpS3ylztUg06XV6fGxhAnkNkqgZzseySCKAYeIyEgiAzwxrX9bSWPO5pXgloSlxGS6pm44DClTx0eFtOLEYgkYcIiIjGh6eHtI/ciaH39cll6o/mjKddiWkStpDCcWS8OAQ0RkRCqlAouGB0ka88ORKzJ1Q/Vl7KqDkuo9nR3Q3ddVpm4sEwMOEZGRPdalJVwcbITrb5XpMI9XcczWtvQcJEs8UHPuEH/enpKIAYeIyASkvP6wpPpVey/wnCozpNXpMX3TEUljpj3UlrenaoEBh4jIBNjZKBEV4CFpzMxv0rjDsZn5OPGMpGXhjrYKTH+4vYwdWS4GHCIiE7Hk6S6SfiiXluvx0e9nZOuHDEur02PJjkxJYz4c0Zm3pmqJAYeIyESolAosHhksacyS7ZlIyBA/pJGMZ5rEZeHTw9rx1lQdMOAQEZmQwcEtEN7RTdKYGN6qMnmach22SlgWbqMApoVx1+K6YMAhIjIxK8d2h5OEVVU3NTrM2JgqY0dUV2v3X5BU/8JDbXlrqo4YcIiITNDkh6TtcLwlXc1zqkzUtvRsvJtwUrheqfh7A0iqGwYcIiITNL6Xr+QxsT+ky9AJ1UXcthN4YcMRSMmei0cE8+qNATDgEBGZoL+XjbtLGvPjkSuci2NCtqXn4LPdWZLGhPm5YXBwC5k6si4MOEREJmrJ012hkvBTWqsHnly+T76GSJhWp8fUjdI29HNrZIdV47rL1JH1YcAhIjJRKqUCS0Z2kTQm9WIhFmw9IVNHJOqj305Lvpo24oH7ZOrGOjHgEBGZsKhAT0T38ZE0ZsWeLE44NiKtTo+lO89JHhfapqkM3VgvWQNOfn4+Ro0aBScnJ7i4uGDChAkoLi6utv7ChQtQKBRVPr799tuKuqq+vnHjRjnfChGR0bw26H7J83HGSDytmgxnxsZUSRv6AUAjexV6tGbAMSRZA86oUaNw/PhxJCYmIj4+Hrt378bEiROrrff29kZOTk6lx3//+180atQIAwcOrFT7xRdfVKobOnSonG+FiMioljzdFQ424j+yD2TlY1s6dziub5pyHbakqyWPWzgskCunDEx8JymJTp48iYSEBBw6dAjdunUDACxZsgRRUVF4//334eXlddcYlUoFD4/Kh81t3rwZw4cPR6NGjSo97+LiclctEZGlUikVWDQ8GC9sEN/Q7/WfMhAR4MEPzno0+/ujksc828sHUYF3fyZS3ch2BSc5ORkuLi4V4QYAwsPDoVQqcfCg2KXTw4cPIy0tDRMmTLjra5MnT0azZs3QvXt3rF69Gnp99RcES0tLUVRUVOlBRGRuogI9MUjCieP5JRos3X5Wxo7onxIycvDDkWxJY8I7NsfrQ+6XqSPrJlvAUavVaN68eaXnbGxs4OrqCrVa7PLdqlWr0LFjR/Ts2bPS82+99Ra++eYbJCYmYtiwYXjhhRewZMmSal8nLi4Ozs7OFQ9vb2/pb4iIyAQslnji+Ie/n+VhnPVAq9NjsoSrawAQ3ccHK8c+IFNHJDngzJ49u9qJwHcep06dqnNjt27dwoYNG6q8evPGG2+gV69e6Ny5M1555RXMmjUL7733XrWvFRsbi8LCworHpUuX6twfEZExqJQKTA2TdozDlA2p3ABQZj3f+R1aCQvXZg1oj9cG8cqNnCTPwXnxxRcxbty4Gmtat24NDw8P5OXlVXq+vLwc+fn5QnNnvvvuO9y8eRNjxoy5Z21ISAjmzZuH0tJS2Nvb3/V1e3v7Kp8nIjJH08La4/Pd53GrTOwTtVwHhH+wEztefkjmzqzTj39cQm6RRtKYZ/u2kakbukNywHFzc4Obm9s960JDQ1FQUIDDhw+ja9euAIDt27dDp9MhJCTknuNXrVqFRx55ROjvSktLQ5MmTRhiiMgqqJQKfDgiGM9/JX5LJOv6TczfcgKvD/GXsTPro9XpMeM7aWeADQn0gJ2EFXFUO7J9hzt27IjIyEhER0cjJSUF+/btw5QpUzBy5MiKFVRXrlyBn58fUlJSKo3NzMzE7t278eyzz971ulu2bMHKlSuRkZGBzMxMfPrpp3j77bcxdepUud4KEZHJiQzwxMzwdpLGrNzHDQANrcfbv0uqb2ivwkcSd6em2pE1Qq5fvx5+fn4ICwtDVFQUevfujc8//7zi62VlZTh9+jRu3rxZadzq1avRsmVLDBgw4K7XtLW1xbJlyxAaGorg4GB89tlnWLRoEebOnSvnWyEiMjlT+reDvY20JeAhCxJl6sb6DPpoJ64WS7s19cGTQVy2X08U+prWV1uooqIiODs7o7CwEE5OTsZuh4io1hb9ehqLd2RKGvPh8CA81qWlTB1ZhwlrUpB06qqkMZP7tcbLkR1l6sg6SPn85k1AIiIzNv3h9pB6QSDmm6NcVVUHPx25IjncKAHEDPCTpyGqEgMOEZEZUykVWDyis6QxegD939suT0MWblt6NqZvSpM87uOnOvPWVD1jwCEiMnODg70Q3rH5vQv/4c+/bmPw4j0ydWSZEjJy8MKGI5LHtWnmgCFBPIqhvjHgEBFZgJVjH8CD7V0ljcnILsIjSxhyRGh1esz+Xtpy8Dt+mcH9h4yBAYeIyEKs/U8o3BrZShqTfqUIP6dekakjy3Hg3HUU3CqXPO4/PX24542R8LtORGRBDrz6sOQx075J4/449zBp/R+Sx9zn6oA5j/A4BmNhwCEisiAqpQKLhwdJHuc/5xceylmNwYt3o+i2VtIYn6aO2D0rTKaOSAQDDhGRhXmkS0v4NmsgaUy5Dnj+q1SGnH/5708ZyMi+IWmMAkDSi5x3Y2wMOEREFuj3mH61+gH/ynfcI+eO+VtO4IvkPyWP+3h4MJeEmwAGHCIiC6RSKvDJaOlnHhXe1uKjxNMydGReFmw9jpX7siSP69TCCY90aSFDRyQVAw4RkYWKDPDEkqekbQIIAEt2nEP0ukMydGQetqXnYMWeC5LHeTdxwJapfQzfENUKAw4RkQUbEuSF8b1aSR6XeCIP8+KPy9CRadOU6zB5Q6rkcc0cVdjzCicVmxIGHCIiCzd3SAACW0g/WHjV3guIT7OePXK2pefg/jm/oDYzkPa/NsDg/VDdMOAQEVmBn6f2QVs3aSurAGDKxjSrWFkVt+0EXtiQirJabAc0oXcrbuZngvi/CBGRldg2/cFajZv0VSr2nLlqsaurtqVn47Pd0icUA0BgCye8MTjAwB2RITDgEBFZCTsbJSb0lj4fRw/gmdUp6Do/0eKu5mjKdYj59mitxvZr74afOanYZDHgEBFZkTcGB6CTV+NajS24WWZRmwFuOZqNoP/+itu1uC/VytUBa/7TXYauyFAYcIiIrMyWaX3RqRaTju+Y/cMxs79dNWFNCqZ+fQS3ahFuVApg+0v9ZeiKDIkBh4jICm2Z2gfje0q/XQX8fSWn33vbcUsj7XwmUzFk8W4knbpa6/HLRnXhTsVmgAGHiMhKzX0kAGN73lersZf+uo2OcxLwny9SDNyVfDTlOoR/sAPHJJ4tdUcDOyWWj+6CyABPA3dGcmDAISKyYv99pBMCvBrVevz201fR9a1fDdiRPBZsPY72r/+CzKs3azX+sc5eOPZmJMONGWHAISKycvHTHqzTnJzrN8vR4bVtBuzIsJ5de6hWRy/c0dBOifef5AGa5oYBh4iIsGVqH/ynl0+tx5dq9QiY8wv2ZV4zmQnIWp0ek778A7+fzKvT67z3RBDDjRlS6PV60/iXWI+Kiorg7OyMwsJCODnV/rcWIiJLE592BVM2ptXpNRrZq7BwWBCiAo1zO0er0+Pj389gyfbMWh278E/RfXzx2iB/g/RFdSfl89umnnoiIiIzMDi4BQDUKeQUl2rxwoZUPLC/CdY/26NejzHYlp6NaRuPoLwWRy7824TeDDfmjLeoiIioksHBLep0u+qOQxf+QvvXf8GCrSfq3pSA/27JwAsbDBNuovv44I3BDDfmjLeoeIuKiKhKgxfvRkYtl1T/m3sjW/ynT2s0d3KEh5MDuvu61npei1anR0pWPtSFt3CtWIOrxbexbv8F3C6v+8eZAsCSkcEVV7LItEj5/GbAYcAhIqpWn3eTcOmv2wZ/XXuVAoODvBD3eKDQLaw7oWbrsSvYmHLJIFdpqvLJ012MNneI7o0B5x4YcIiIxE1Yk1KnnX/vxcEGaOHSAPa2KjjYKOHt2gCPB7cElMCPR67gXN4NnFTfQC1OVRDWpIEt4h7vxH1uTBwDzj0w4BARSXNLo0XY+9uRXaQxdisGpVIA08LaYUr/dlwKbgakfH7LNsl4wYIF6NmzJxo0aAAXFxehMXq9HnPmzIGnpyccHR0RHh6Os2fPVqrJz8/HqFGj4OTkBBcXF0yYMAHFxcUyvAMiIrrD0U6F/a8+jFaujsZuxWB8mzbAmQVRmB7enuHGAskWcDQaDZ588klMmjRJeMzChQuxePFiLF++HAcPHkTDhg0RERGB27f/d/931KhROH78OBITExEfH4/du3dj4sSJcrwFIiL6l12z+iPMr7mx26izML9m2PHyQww2Fkz2W1Rr1qzBjBkzUFBQUGOdXq+Hl5cXXnzxRbz00ksAgMLCQri7u2PNmjUYOXIkTp48CX9/fxw6dAjdunUDACQkJCAqKgqXL1+Gl5eXUE+8RUVEVDe3NFo8tmwPTuWWGLsVSWyUCnw4IhhDgsQ+L8i0mMQtKqmysrKgVqsRHh5e8ZyzszNCQkKQnJwMAEhOToaLi0tFuAGA8PBwKJVKHDx4sN57JiKyVo52KiTM7IdPnu4CGzO4CqIEMLV/G5yeP5DhxkqYzE7GarUaAODu7l7peXd394qvqdVqNG9e+dKojY0NXF1dK2qqUlpaitLS0oo/FxUVGaptIiKrFhXoiYgAD3z0+xks25EJEzmGqoKLgw2WPNUFPds14+0oKyPpCs7s2bOhUChqfJw6dUquXmstLi4Ozs7OFQ9vb29jt0REZDFUSgVeHNABZxdEYbCJ7CGjALB4eDDS3oxAnw5uDDdWSNIVnBdffBHjxo2rsaZ169a1asTDwwMAkJubC0/P//0Hkpubi+Dg4IqavLzKp8KWl5cjPz+/YnxVYmNjERMTU/HnoqIihhwiIgNTKRVY+nQXLBquw6zv0vBjWk6996AAMLlfa8wc4MdQY+UkBRw3Nze4ubnJ0oivry88PDyQlJRUEWiKiopw8ODBipVYoaGhKCgowOHDh9G1a1cAwPbt26HT6RASElLta9vb28Pe3l6WvomIqDI7GyU+GtkFHwzX48C569h+So34Y2oU3y5DiUae3foa2CowsW9bTA3jfjb0N9nm4Fy8eBH5+fm4ePEitFot0tLSAABt27ZFo0aNAAB+fn6Ii4vDY489BoVCgRkzZmD+/Plo164dfH198cYbb8DLywtDhw4FAHTs2BGRkZGIjo7G8uXLUVZWhilTpmDkyJHCK6iIiKh+qJQK9GrXDL3aNcMbQwIA/H3kwuKks/h0ZyY02rpN2LG3UWJ4t5aI6uRVp7OtyDLJFnDmzJmDtWvXVvy5c+fOAIAdO3agX79+AIDTp0+jsLCwombWrFkoKSnBxIkTUVBQgN69eyMhIQEODg4VNevXr8eUKVMQFhYGpVKJYcOGYfHixXK9DSIiMiCVUoGZD7fHtLB2OHDuOpLPX4MeQGM7G5xUF+GmRovmje3h7GgHvV6P/FulOJtbgpLScnRwb4wAL2e4OTvU+cBOsnw8qoH74BAREZkFs9wHh4iIiMhQGHCIiIjI4jDgEBERkcVhwCEiIiKLw4BDREREFocBh4iIiCwOAw4RERFZHAYcIiIisjgMOERERGRxZDuqwZTd2by5qKjIyJ0QERGRqDuf2yKHMFhlwLlx4wYAwNvb28idEBERkVQ3btyAs7NzjTVWeRaVTqdDdnY2GjduDIXCcAe1FRUVwdvbG5cuXeIZV1Xg96d6/N7UjN+fmvH7UzN+f2pmTt8fvV6PGzduwMvLC0plzbNsrPIKjlKpRMuWLWV7fScnJ5P/R2JM/P5Uj9+bmvH7UzN+f2rG70/NzOX7c68rN3dwkjERERFZHAYcIiIisjgMOAZkb2+PuXPnwt7e3titmCR+f6rH703N+P2pGb8/NeP3p2aW+v2xyknGREREZNl4BYeIiIgsDgMOERERWRwGHCIiIrI4DDhERERkcRhwDGTZsmXw8fGBg4MDQkJCkJKSYuyWTMbu3bsxZMgQeHl5QaFQ4McffzR2SyYjLi4ODzzwABo3bozmzZtj6NChOH36tLHbMhmffvopAgMDKzYgCw0NxS+//GLstkzWO++8A4VCgRkzZhi7FZPw5ptvQqFQVHr4+fkZuy2TcuXKFYwePRpNmzaFo6MjOnXqhD/++MPYbRkEA44BbNq0CTExMZg7dy5SU1MRFBSEiIgI5OXlGbs1k1BSUoKgoCAsW7bM2K2YnF27dmHy5Mk4cOAAEhMTUVZWhgEDBqCkpMTYrZmEli1b4p133sHhw4fxxx9/oH///nj00Udx/PhxY7dmcg4dOoTPPvsMgYGBxm7FpNx///3IycmpeOzdu9fYLZmMv/76C7169YKtrS1++eUXnDhxAh988AGaNGli7NYMQ0911r17d/3kyZMr/qzVavVeXl76uLg4I3ZlmgDoN2/ebOw2TFZeXp4egH7Xrl3GbsVkNWnSRL9y5Upjt2FSbty4oW/Xrp0+MTFR/+CDD+qnT59u7JZMwty5c/VBQUHGbsNkvfLKK/revXsbuw3Z8ApOHWk0Ghw+fBjh4eEVzymVSoSHhyM5OdmInZE5KiwsBAC4uroauRPTo9VqsXHjRpSUlCA0NNTY7ZiUyZMnY9CgQZV+DtHfzp49Cy8vL7Ru3RqjRo3CxYsXjd2Syfj555/RrVs3PPnkk2jevDk6d+6MFStWGLstg2HAqaNr165Bq9XC3d290vPu7u5Qq9VG6orMkU6nw4wZM9CrVy8EBAQYux2TcezYMTRq1Aj29vZ4/vnnsXnzZvj7+xu7LZOxceNGpKamIi4uztitmJyQkBCsWbMGCQkJ+PTTT5GVlYU+ffrgxo0bxm7NJJw/fx6ffvop2rVrh19//RWTJk3CtGnTsHbtWmO3ZhBWeZo4kSmaPHkyMjIyOEfgXzp06IC0tDQUFhbiu+++w9ixY7Fr1y6GHACXLl3C9OnTkZiYCAcHB2O3Y3IGDhxY8f8HBgYiJCQErVq1wjfffIMJEyYYsTPToNPp0K1bN7z99tsAgM6dOyMjIwPLly/H2LFjjdxd3fEKTh01a9YMKpUKubm5lZ7Pzc2Fh4eHkboiczNlyhTEx8djx44daNmypbHbMSl2dnZo27Ytunbtiri4OAQFBeHjjz82dlsm4fDhw8jLy0OXLl1gY2MDGxsb7Nq1C4sXL4aNjQ20Wq2xWzQpLi4uaN++PTIzM43diknw9PS86xeFjh07WsxtPAacOrKzs0PXrl2RlJRU8ZxOp0NSUhLnCdA96fV6TJkyBZs3b8b27dvh6+tr7JZMnk6nQ2lpqbHbMAlhYWE4duwY0tLSKh7dunXDqFGjkJaWBpVKZewWTUpxcTHOnTsHT09PY7diEnr16nXXthRnzpxBq1atjNSRYfEWlQHExMRg7Nix6NatG7p3746PPvoIJSUlGD9+vLFbMwnFxcWVfmPKyspCWloaXF1dcd999xmxM+ObPHkyNmzYgJ9++gmNGzeumLfl7OwMR0dHI3dnfLGxsRg4cCDuu+8+3LhxAxs2bMDOnTvx66+/Grs1k9C4ceO75ms1bNgQTZs25TwuAC+99BKGDBmCVq1aITs7G3PnzoVKpcJTTz1l7NZMwsyZM9GzZ0+8/fbbGD58OFJSUvD555/j888/N3ZrhmHsZVyWYsmSJfr77rtPb2dnp+/evbv+wIEDxm7JZOzYsUMP4K7H2LFjjd2a0VX1fQGg/+KLL4zdmkn4z3/+o2/VqpXezs5O7+bmpg8LC9P/9ttvxm7LpHGZ+P+MGDFC7+npqbezs9O3aNFCP2LECH1mZqax2zIpW7Zs0QcEBOjt7e31fn5++s8//9zYLRmMQq/X642UrYiIiIhkwTk4REREZHEYcIiIiMjiMOAQERGRxWHAISIiIovDgENEREQWhwGHiIiILA4DDhEREVkcBhwiIiKyOAw4REREZHEYcIiIiMjiMOAQERGRxWHAISIiIovzf5yr+NyUuQbIAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "M = 1200\n",
    "\n",
    "# sample from the x axis M points\n",
    "x = np.random.rand(M) * 2*math.pi\n",
    "\n",
    "# add noise\n",
    "eta = np.random.rand(M) * 0.01\n",
    "\n",
    "# compute the function\n",
    "y = np.sin(x) + eta\n",
    "\n",
    "# plot\n",
    "_ = plt.scatter(x,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# use the NumPy-PyTorch bridge\n",
    "x_train = torch.tensor(x[0:1000]).float().view(-1, 1).to(device)\n",
    "y_train = torch.tensor(y[0:1000]).float().view(-1, 1).to(device)\n",
    "\n",
    "x_test = torch.tensor(x[1000:]).float().view(-1, 1).to(device)\n",
    "y_test = torch.tensor(y[1000:]).float().view(-1, 1).to(device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "class SineDataset(data.Dataset):\n",
    "    def __init__(self, x, y):\n",
    "        super(SineDataset, self).__init__()\n",
    "        assert x.shape[0] == y.shape[0]\n",
    "        self.x = x\n",
    "        self.y = y\n",
    "\n",
    "    def __len__(self):\n",
    "        return self.y.shape[0]\n",
    "\n",
    "    def __getitem__(self, index):\n",
    "        return self.x[index], self.y[index]\n",
    "\n",
    "sine_dataset = SineDataset(x_train, y_train)\n",
    "\n",
    "sine_dataset_test = SineDataset(x_test, y_test)\n",
    "\n",
    "sine_loader = torch.utils.data.DataLoader(\n",
    "    sine_dataset, batch_size=32, shuffle=True)\n",
    "\n",
    "sine_loader_test = torch.utils.data.DataLoader(\n",
    "    sine_dataset_test, batch_size=32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "class SineModel(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(SineModel, self).__init__()\n",
    "        self.network = nn.Sequential(\n",
    "            nn.Linear(1, 5),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(5, 5),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(5, 5),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(5, 1))\n",
    "\n",
    "    def forward(self, x):\n",
    "        return self.network(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi8AAAGdCAYAAADaPpOnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAADENklEQVR4nOzdd3gc5bX48e/M9pW0u+rNkix3y703bDAY22BMCEkoCaEFCLnpzs0NkEJI8rtACgm5JCGQkAYEDIRiigEbTHHvTS6yLUtW79v7zO+PlWXLsowAS3I5n+fxg3fmndEZLK3OvuW8iq7rOkIIIYQQZwm1vwMQQgghhPg4JHkRQgghxFlFkhchhBBCnFUkeRFCCCHEWUWSFyGEEEKcVSR5EUIIIcRZRZIXIYQQQpxVJHkRQgghxFnF2N8BnG6aplFTU0NKSgqKovR3OEIIIYToAV3X8Xq95OXloaqn7ls555KXmpoaCgoK+jsMIYQQQnwCR44cYcCAAadsc84lLykpKUDi4R0ORz9HI4QQQoie8Hg8FBQUdPweP5VzLnk5OlTkcDgkeRFCCCHOMj2Z8iETdoUQQghxVpHkRQghhBBnFUlehBBCCHFWkeRFCCGEEGcVSV6EEEIIcVaR5EUIIYQQZxVJXoQQQghxVpHkRQghhBBnlXOuSJ04v+iaTrjcjeaNoKaYsRQ7UVTZ00oIIc5lkryIs1ZwVxNtyw4Sd0c6jhmcZlyLB2MrSYWKNeCrh+RsKJoJqqEfoxVCCHG6SPIizkrBXU00P7kHXddpjumEdLAqGrmeHUSf+QMW+/uokdZjFzjyYOGDUHJl/wUthBDitJDkRZwV4lqcLQ1baAw0kmHNIG9ZnJqIxs5gnJB+rF1SMJPZjnocxycuAJ5aWHojXPNPSWCEEOIsJ8mLOOOtqFjBA+sfQKlqwxY2ELTEyTJMYEbgOkDh+BkuPi2d5W3/w0LXLxlsXXfcGR1QYPldMGKRDCEJIcRZTJIXcUZbUbGC3z3zQ+aUppIUyuk4rqse4rYDGM1DO7VXUNCBt/1fY6B5E1F9JBqpqLRiUXejeKoTc2GKZ/fxkwghhDhdJHkRZ6y4FudvL/6Si7ZkdDmnaH5i/mXo8RmohlRQklCN+SiKigLEow52+v9Ghim14xoDjbhMj2Hz1ffhUwghhDjdJHkRZ6zNdZsYvjVRiqjz4NAx8dBa4kdfKMmY7HMxtPfGlNpdzIke15Z0mqP3kN6gYOvFuIUQQvQuKVInzlgVpTtIChm7TVy60H1E/cuIR8oA8FlP/PZOvG7bYEbXdIQQQpydJHkRZyx72IAOxOwpRB1pxOwp9CTliAZW4bbqZBpOlvQoxN0RwuXu0xytEEKIviLDRuKMENd0NpS30OANkZViZWpxGsnGHPxDxqKbzB3tlGgES30lJm9b9zfTvezNa+RLLfndNtG8kW7PCSGEOLNJ8iL63fJdtdy3rJRad6jjWIbdwJhoGUVGU6e2utFEKH8wVB88ZQJzQZuPUy2GVlPMpzgrhBDiTCbJi+g3mqax9N0t/HllKTomFFLQ2+e3NAVivMtg5pqgwNBKrTODgNmCPRImt62RcHYBRm9borWik5wTwGiPEQsY8dXZmey30d2sXIPTgqXYiabp1Ja14feESXJYyB3qQpV9kYQQ4ownyYvoF6Wlpby07DUiQT8XtneC+HUT66OFVGppgALorNYGEp41jbDF2nFtUijIrIM7GFVdjtNZj2uwB2tahOScAIoKUb8Z6/56aCw86dd2LR7Eoe2NfPBsGf62MBoatY6DxF1BZswuYfGsSzBIETshhDhjKbqun1PLLjweD06nE7fbjcPh6O9wxEmUlpaydOlSdB2U4zo6jn4nvhsdTKWWRjzLSnSkE6wn5NjtDa9a+TRDD+zpOGxKipI/sx7XIC/okLvpVhytczrOa+E2kibZaBs1luV/3gXAobTtrB74H/yWto526aZMfjTrHuYVzTu9Dy6EEKJbH+f3t/S8iD6laRpvLF+eKNZ/wgiNoiTykqmmIxx25hIdn3bym7Q3XDltEYMP7kVtT2aifiOH385n4KXVOAb6eC/zfZR1TWToKqOaDkJzGb7lsH7h7wGFQ2nbeWvYE11u3xxpZMmqJTx00UOSwAghxBlIkhfRpyoqKvB6PB2VWxQ0iqgmGT8+kqhUcinhCFdkGqhry2S9czQjfOU4YgGsXivmgI19hnSOJKn4zRbq07LJba4DINOST2GSj7rSZJ49Mpx3wlPQhieqAWQEJ3LnjpcoCUYIBBU0NFYX/YeMYAZWzUrIEKLJ2pQYrVJAR+fBDQ8yt2CuDCEJIcQZRpIX0ad8Pl/H30dSxkJW4eTYMQ0FFZ1bDsIzhXPZmVLM7pThiZOpkBRqY9LB55m6v44ao5m30odxi9bK1SkbyE36EIMOG7QRLNbcXGVayyuxmbylT6XZ6uAXU2/i6/s+xA7Upe5iTsMM7HF7x9cOGAJsT99OTVINqq4zoPkwh1/7FoOTcqHogsR+SJLICCFEv5PkRfSp5ORkIJG4XMOrXc4ruk5Dhpn7iufzgv2/O50zBzZiaX2SPSkt7BmTOOYMbifdGCLQZmGpfwKPGD5PHekd12TFW7gksIl3kyeiKzpLB4/hTsOHFJuPEI0bCDncmC0hon4TM/ZWMLIijSrzeHyWQaQoQXaWbyDf8iwbnW+zLzwYQ1oJI4ZNYMqcORhPWMYthBCib/TqhN3333+fX/3qV2zevJna2lpefPFFrrrqqlNes2rVKpYsWcLu3bspKCjgRz/6ETfffHOPv6ZM2D2zaZrGbx/6DfO9S4lEcknSdFLUJvLMpSgKNKSb+UdhGn+w/IWQObljYow5sBFH0+8BOm8W0P7dO2L3KDaqNyRedJpMk2hw9YF38Gak4cpWSFKObXikECY5uJGgI8xW3/VMbhuGQz9WeNqraFRavRTSijEWIUY9UYsbzWJlxqSJTM/Jp+yDD/HH4limTmXS3DkYjfKZQAghPq4zZsKu3+9n3Lhx3HrrrVx99dUf2b68vJxFixZx55138tRTT7Fy5Upuu+02cnNzWbBgQW+GKvrI3r170VttvNvyEwg3ocWqGGCr5PO5pcR0hb96v8lTe2cQmpqCqutMaImREdIIe1dTalXQlRNybQUUDVozahlauZ+AMZkaay66oh5rgM47oyYxwnGAaKuDJKMbTYdSWwuHMjYQM/mYHRzId9iB3bIdt7uEvcESdAwk6wqjgk7A2X6/YejhIJHwu2wIvo+yo5QBVdXYAe3Jf/J2QQGea65j6iUXUVRUhKrKDhxCCHG69dlSaUVRPrLn5Qc/+AGvvfYau3bt6jh23XXX0dbWxvLly3v0daTn5cxVWlrKU/98j6wGlVjgbdATFXVHOBow5Dj4S/RbbNJV4rk2Lshx8N97wmSHj317NhpbeTT7OdY4tnUcK6yzMa00jaTQsTzca0jig/QLOJg0qNPX/+9JvydWl8/2hnGst3pR859lrC3Gd+NBxhx2Y41oAOgaeJpSOdByKXv1SbS5hoByLAnRO1Y3LSPqDLDonQ+pyc9jy8SJBO3H5tCkpKRw2WWXUVJScvr+JwohxDnq4/z+PqM+Fq5du5Z58zovTV2wYAFr167t9ppwOIzH4+n0R5x5YrEo/3j2RVLaDMT8yzoSFx3Ylzyc/4p+h4OJ3IELvfDgthCZ4c55dXrMxY+qb2emZzyQSFzmbsnEHuo8iTY57ueyhjcZ7D/U6Xh9ZSF+Z4B3o8Uo2csZa4vxA7xM3t+KpT1x8RyxcmBZNjXv2LBv+5CJ2x9m5rofo4d3sKvQTEWGistdRnbDJjIjhaS0HqZ1WjqrZ80iaOtc0tfj9bB06bOUlpaepv+LQggh4AybsFtXV0d2dnanY9nZ2Xg8HoLBIDZb13rv999/P/fdd19fhSg+gbL1a3jzr39k65SbWLDvrx3HoykuQpkDWKMMZazmYZvqRAW+6098W56YWasoaOh8tf7zrE/ezrTSRB0YpfMsmPaBIpjdvJpD9oEdQ0jT2iq4u/UWDPbDGExtfM4VYfhmf8c1niNWqlendonfHG7j4rV/piG5mks2riGrraXjXNBsZceUUQAYAl6UWJS4wURAN2NXImhJybz48r8ZMeI+GUISQojT5IxKXj6Ju+++myVLlnS89ng8FBQU9GNE4nhl69fwykP/S1tqHk5PC+iJZdHRZCdx5yAKjbWMNhzmQCiNHVzGOExkn6JDUEUhK5bGBfUjSQr5u22nAClxH3mhWmpsueTQwmXqBp6MXspGQ4TBFo2B/kinoaL6Lc7jrj7+a4IGXPd219VR1kiIqas3YxhUR0PKsS0M7AYjcZMZvTZOPHsAb2x6FSXDSKY9k4lZE6V2jBBCfApnVPKSk5NDfX19p2P19fU4HI6T9roAWCwWLBZLX4QnPiZNi/PO3x8DIGhNQtePJRvDkhqZn7yCZKV9J2krfIVlPBO+G/QxH3nvrLAL6D55OSop7gcU7jX9C4Oik0UbeqQIh0HHEjk2LBVoNBMLdp9QdJdOHe3lGVVZS6wwG0ssTthkpMVuQY1FaUrLwdnawOqX/8myYfsAyDClcffMHzJ/4PyPjF8IIURXZ1Q/9owZM1i5cmWnY2+//TYzZszop4jEp1G9Zze+liYAbCE/ldku0sy5zM48xGdT1x9LXNpl4OF249M9unfDcXsRnYrVEONPpt+x0LAxcR0utHAubeFkwuZjPSyx0CfvCVEAW0xj+qFaJlQ2MP1gDXP3VpLt9pPfUI1NN2NWhnJl8zhC9unUKFksWfXf/GvFI5/4awohxPmsV5MXn8/Htm3b2LZtG5BYCr1t2zYqKyuBxJDPjTfe2NH+zjvv5NChQ/zP//wPe/fu5Y9//CNLly7lu9/9bm+GKXqJt6W54++O1homBFPItZUxJb36xJEZIHHIrO5GVZrQOfkiOB2diBLmYq0OgzHaTatES7MxzgvO/8dCw0Y0HWr0dDZoIwCF0oqrqbSZCZlVdMBojX+6hz2BNRpnYkU9Q+MZZJBOZlUNhS2TuG/bYIaYf0Bb3m/5Zd1W9q778LR+XSGEOB/0avKyadMmJkyYwIQJEwBYsmQJEyZM4Cc/+QkAtbW1HYkMQHFxMa+99hpvv/0248aN4ze/+Q1/+ctfpMbLWahs/Rre/cfjHa8NKHx5VzkXZe9BUU6auwCgKhqpxj8DoHVJTXQUIMfwEHOD+xlobesYtjmxHcDC7H2YVB2tvcF90S9jIpGkRL1jeLp8HvsGJwFgy4xgtMVPerdPon2LJIoqDzEn+zNcWHAbhfapzInP4s8bg7y82sJow538vzVvoGmnN3ESQohzXZ/VeekrUuel/x2dpHuUgsLsLAep5hqGpGzp1FYH2pwmwmYFS0TH5Y6iAPtyLkY/fAfJJHe0DSteGiL72ZxWy3OZJYzdu5kp7t0EQir+2LF5TynGEHOzDzHUkej5qdHTuS/6Zd7UprDAuJeIYmJ9tIAAFi53LueO/NcYUdVK5KD5uNVGx9Ir7bhX3SVdp2Kb9T2MmcPRgUqbQlFQp31VOD8eGecnE1MZPGbcJ7izEEKcO86YCrvi/HP8JF2AfPsw5mSlkGn+P9aFv8wQjiUvDelm9g9JImw5Nt/EEo4z7IAfX9YmXmzOo75pOpcM2U2h8wCP1MH9kR3cNOpPjNrzMqvGrKL2yBA0Qzbz6quxRSOUWwrZZBvEu4pCZsRDAy42aCPICLfxiOlhLjduZG3kUsb4J3I40kqowcCzpTMpTq9mWIGGY3oU01YvxnCsI6am1DRWTp7JdW+/isbH766Mh9sote+n2eAmLe4gl6GYUdGAb5Yb+a+aHXxXyWLh6NxP+H9dCCHOL5K8iNPq+Em6+fZhzMpazFbT7/l59A8MNFYxu71dQ7qZnSUpXa4Pm9WO4/ts2Uye8h8ysnfw/t7B3BLfwYrWqXzpxV+zYWQzqDr7isoYUr2Pfw9TcCcrZDa2Ymh2ENAz2eYYiXdAIcUttYwIVbO7NYPmI5PIUCsY3VbO1CM5tCQngRbAVeoGdNoG2Cm8MEhO1E08ZGBT9ii+P/v71Nhz2FM8lG8s/UenOi898eCg51hbFOh4nR518bX6LzDLO4GckE6B2c5/PbmZP94wSRIYIYToAUlexGl1YNN6IDFUNDH9EtYoh7kr9hV0oEEbQY2eRg4t7B+SmGvSeRPF9te6jo7KtUVvkGRq4dC7Axhj30WosgDLoHqmz28iO6LyitvEkYgBLWUQ6TGFEYeCmIMpOPUgaWYXpY46/Oo+MtK9FDeFmJu9m+rgNbxmKMCUUc6UvA8Yub4Se9uxjRpba+G50eMZ5KqlzTuQZj2L2es24UgNE1M1Vl0xjayDTZg1Bc3XyvhDlSQFwycdTtKB5hRYNzCxXPuoZmMbv8h/nB9V384s7wTGECDVspc/veTn0pJrMKifZHBKCCHOH5K8iNOmbP0atrz+MgDp1gF49STux9U+BVZBR+MB5QpudH5I2HKKbRwUBQWdI0eS0NalMiPpMJFxKcQ+34JdBcWYzgetXo5EDaBARI0xrXlaYjynPSfyUUthE9BkAFz4gFLfF/BEZjIKIFiAW7mAV68tI2Q4hD+YhM9pZejgreTHt9Ic1/ngQA4epRTNoJIcTmZo21B0PYXGLANKJISSlsJql8L81fvROfl8mL/PU9ENJyZogA5/zn6e6d5x6F4TdnOU8dE9LHtvA1fNnfYp/hWEEOLcd0bVeRFnL02Ls/Jvf6YuNYg5N5+gfRFLw1HaMAMKxUnb+ULSGtJIZ715bI/uqcZjFE+vZqTJz7gv/Y3Ro3/LxAlPMWfWh9Qr+R3bAoQMoY+4U8KgBVeQ5DquoKGuYi3PoqHVTDw8hWgsm/Jds8hZl8uV+z0sjEa5aLWR0buiDKnTcbnryK5rxdJSj7mtCWvNYfZphfxiyo00WZ2dvlbMnsJvrlbZMKKbHzEFGk2tbLKX8lwkmWCsCIDdG95H07STXyOEEAKQnhdxmqz/z7MYXOVMG5RL3ZZriAHJDesheRyDkrYyOx4FEolDJHLyasknGjnmCMPWBEm97tuQPrPTubum3sWSVUtQUGiyNhEwBLDFbV32OTrK4XAw45JxzLhEobasDb8nzN7gLn586Gfoig76MkY1TSTXV8RqLmJd62xGt5bSZFTIblFQWkJAiAANnX5oRofrqcmewbcuvYtxbXu4OlDDWOtYPhjYxobCf37kMz6WtQnv6GG8WJ7P9W2HiTfV8uHLL1A8bDj5I0ehyjYCQgjRhfS8iE+tbN1qjmx/noHzaqjd8yVAIUPVGFqzhvnqeqaSGCI6mli43VmEw3a6XaSvQziaxNi9HgbaLDDnv7s0mVc0j4cueogsexYosD19e/ulJ7/pwoULUVUVVVXIH57KsCk5XDlnHr+Z+xuy7Fnois6uzM2sy1pHxBhBU43sSB9LMHvg0ZC6BolOzsX7WTTrKR4d9xzZQ/1sH5RK0F9DTz8XROJlXPXqX8hxNrBnwAXoZitrXnuZpT+7h8e//hXK1q/p0X2EEOJ8InVexKfi39FA9b82Ujv3Xtrc2cTX/DdjbAYOHXyNke6l7JrrZEP0qi7XpadXMrLkPeCEObvt343J+ycyrf5tuOafUHJlt18/rsXZ0rCFxkAjsboYZevK8Hq8HecdDgcLFy6kpKSkR/fItGcyPmM8VUeq8Hg9/GzLzwjXVTF1TypJoWMJScQWpWBmPblDvJ3utXPfYGybCijyGvjJgm0EDcFOk2EUXSHXMxh7xIFuCDA1Xo6rUiVYn8m7F13PTI9CJLAfa/VBTN42AK5ccg9Dp3XueRJCiHON1HkRfSK4q4mWp/eip1cTs7YSO3A9U+0GttVsZMDeZeQtbmOVkn/Sa5ubC9lTeiGDh2zEYjm2jNgYSiNr3xfJC73zkYkLgEE1MCVnSuLFINCma1RUVODz+UhOTqaoqAhVPXUHY6d7tCsuLgbga86vsWTVEo7kBMlqMWMLGwha4tSnhZmQFOXGE+41wNDAoYvy2PBBCuOax7Euax1HZ/MWN49l1uGrSY6kdrRPUpuY7fgrqUO3kLTbgHHg90jSl7M/uwCjN1FB+N1/PMbgKdNkCEkIIdpJ8iI+EV3TaX3lIOgQt7rJaIiSXzuO7TUbGV/7f+QubsNo1cnSfRzq5h7NzYU0Nw/A6WxgoMXEKO8YLK3D8ZiCJP34v8D88XcLV1W1I/E4HY4OTz2w4QHqlM47nnviXZMioz2Oa1c5D1/1E67asIppDdPYkb6DHPdQ5u+/tUt7v5bG8rb/YYHzl9yR/h7fah6N2TiW6aatlNtTMAa8eJubWPfiBxSNHkfuUBeqLKUWQpznJHkRn8j+VTtI8kRQFIVWXx17dn2HeZqRQU1/ZcCsVnQN/PVmpoTqWV0YQFVOPplWR6He5+Kq+oUYMKCg47/YjPETJC69ZV7RPOYWzO0YWkqzuPjhm3dwKKTQGlNwGfSOoa/knACV7+Rx5YEVtBozyA+o5PnzSGuaCnCS/wcqoLHaeysDLRu4x/RvLog8TL3Nz82uFewMONFR2PRaKVvf1khyWZh97VAGT8jq0/8HQghxJpEJu+JjKy0tZf27iYmkB9Va3o5YydfDHPQ+y7DxNXirrBx8NZvKdzNoWOtizuotKDqcOL3q6OTawWRgwkAjGqvGRpg+9+I+f6aPcnRo6fJBlzM9EufupiZ0FF5sNQF0TD5WVMifWU/m6p24U4I0WVrQ43ZMcXu3K6FAxadlUh8dSZ7awlR1L1u843hz8HjuHLqOISlN5JkPA+BvC7P8z7s4uLWh9x9aCCHOUJK8iI9F0zSWL3+DIGHeT95EpeUpPlP8by4wPcIlWU8TaLBQvTqVWPDYt9aAqmpmrl6NPRjsdK+wIYgWrabJnco38fMFfAyZNqGvH+nj89UzLxDkoYYm6n3wt2YzbfFjiYlrkJcBc1pJ9YTIPBIkpMROcbNj/FpiLkwWboK6mU36dMojk7lywB4WZP6DPPMOlPZdsT9cWoamnVNz7YUQosdk2Eh8LBUVFXg8Xlan72S2z02xzcH4w4ndm7U41G9xcrJaswVV1eRX19AwII0jF0EmPnLXh3l4xu1st9pp9cfJcVqZWpzWtw/0SSRnAzAvEGRuIMgWq4VGgwG7HYqIUdU6gr3V9xLJ9bByUpAr122DHsy1TVJbAUhLbWC4VkZWjZMH7UO5x3OAfJefxa57CerpfOj5CodaZ1Bb1kb+8NSPuKsQQpx7JHkRH4vP56PKXkVpZjJXVGRxcfKboCaWO7cethELdv9bWtV1co40M67CTVWdg1CdmTenXoRxZysGf4h7F5ecHfv6FM0ERx54ajGgMyUUThz3J/7jCThAV3FE0ykb5GSZDtdtDKIp1m6GjjSS1WZyzHsIWlSyBlZwYaQNtVin3pvPa2+N5EpKiVhTKTHVsND1S5a3/Q9+T/fLv4UQ4lwmyYv4WHZE4qwuhO+tcJNqLyHF0NxxLuw29+ge9ZuHYvQ38Ks7voGmqqSZjDxww8SzZ0dl1QALH4SlN9KxUVE7XYdkQ6IHpagpToo/zP5BJTxnDPH5dYFEg06FbTRAYZbjCVRF40eh23lh89yOs6mWVsaPt5C91wtDMxkVq0HX4QLHE3iSv9onjyuEEGcamfMiTimu66xu9fJifSub1lWy7/2lfHdHBjnJF5JhtKLrx76F1JQBPbpnzDmAe+/4Lh+Mn0KGwcCGO2efPYnLUSVXJurQODrHrVvSYX8QS6gVVdNZsDUCwJ5CKx8OaiBm7FzULlltZqHrl2Qlb+XOyHd4IXxRp/OtYRfvuufwoWEsk5q3AoncJ8XQRK6ltPeeTwghzmDS8yK69VpjGz8qq6Y2HGVufZT7twX4kjKTJN0J1kSbuvBfcZkew2ZYS9LANFpLQQ+1dntPxZbKwzf+Fx8U2lEUhQdHFmA2nKU5dMmVMGIRVKwBXz0kZ6MWzSRfh0kPPsWaChcjqyJ8frWPNyfaeXfKSFKsP8QXzEVVLyMrWEmWcRetw/dwzY5f06ylkVhMrZOterERJYiJei2ZD7JnM65yL8MdTRwdWVP9suJICHF+kuRFnNRrjW3ctuswOom5Kj/cFcKAgl1zdJqPGyed5ug9JGsvoSqNWMZeSmjDo93e1zLmWmpTLORZzfx8aD6LMl29/iy9SjVA8exOhxRgwj030vTIPzm4zcXIaoXhNW4qM4wMSvshNcrPebloGQ3p32LOHh8p+kiaw+kAFKotTDNVkqREO+7n102sjxay31hMVWAfhUnuxIn2icNCCHG+OUs/8oreFNd1flRWnahqr8e50rODJC0IOijKyYqsKfjin8UTvwNT3jisU76KYu28CkaxpWKd8lWCBSP4VpKHjTNKzv7E5SNc/F9f4oV5v+dw8mtouo+BjTGyD4W5o/luZrZGGVD6Lm5/Cu5wYg+PQrWFuaaD2Il2uo+dKHNNB9GSHLzuGw9AyGBho8VEXIv39WMJIUS/k54X0cW6Nh+14SiT9XXcyBMMiGZi0u462Qrok1Ax5U/EmDeOeNNB9LAbxeJETR+CoqrsqHmJWQU3Y+iSBJ17DKqBH8y4iyXhJSj6CnI8g7BHHZR6MqFBw+fcSnPrtcwt+AAFnWmmSuCE+bztr3UdMh1m/OUp7LVlYLGk8tRLX+LuzGLumnoX84rm9cMTCiFE/5CeF9FJXNNZc7CZqS3v8R39V6TpzdhbRnac19CpUVs5qNZRo7aicWKhNAVdB00xYMwcjmnAVIyZw1EUhTX1L1Ed2I+vrfs5Meeao3sjZSZlUuM8wIGMLaxJPkijkoNZCaOjsr52EoMtNSQp0S6Jy1GKAjGrmUm1R3j34FjuKb4Zp/dKco4Y+e6732NFxYq+fTAhhOhH0vMiOizfVct9y0qpcwf4zZy/A4lfmg3lChlAudrAOtN+/Eq445ok3cL06DCKtWN77SiKggH4PUEW6yaKMbCm4SWqAmUAJLvOr8JqJ+6NVLrfT+Wb7xGw2/Ea29jaOJapSfugByNAIauNSeWH2b3LzD/s84B5KC1t/DjyPHNvm4tBdp4WQpwHpOdFAInE5WtPbqHWHWJ46gGcVg+KAi0HUth25BBl1LDStBM/4U7X+Qmz0rSTcrXrypccVAZoHg56/9mRuKSkZ5A/clSfPNOZpNPeSFPm4/S0snf4LQyw7wKgMtizpeLmrAC2nCCXtO0hV/WgoKPHnNQeuoI/rVnTm48ghBBnDEleBHFN575lpR0DQIXWegB0DfRNi8gfNIGd2e+RmVmO01VHorBau/ZhjnWm/V2GkHLUw+xofYBNzXUdx+bedAfqed47MCPNwfYLr+TSD16nbPo1DE2uRddi+HUTenfbFek6DjxcNGQnY2bWcknBU3w1+iKfM2+nsH1bgSfe9hCX/Y6EEOcBGTYSbChvodYd6ngdidgB0F++hvCkejIK15NjCXScD4ftHDwwhebmwsQBJdEDU6e2kacdGxIKBvdQ600sAU5Jz2DuTXcwdNrMPniiM5tBUfjqZQt4MBpj/nsvsWbyxfzPzt+TFkhnlyGvaxXe9oxmIe+htieIKcYwNxnf5Bc+J4NsyQBUhtP48J1yLpw3qM+fSQgh+pIkL4IGb6jTa7VV4Z+bF2Kb8Rafz27r0t5sDjCy5D32lF54LIEBgh1DSjoh1c8lN32JoN9LsiuV/JGjzvsel+MtynTBlYv48b6xuA6XsTZ/FMXxMi4pK2KNto+gzdrR1oGXhcp7lHCg49jRFUjfsr7CxOgippqOcERL5fDqKuZcXIxyNuwRJYQQn5AkL4KslGO/KBVdZ6PVjJL5Hj/JSCQ13S3dHTR4I83NAzg6+mjDwtF9furGRblg/MS+CP+stSjTxcIMJy+lWfnPu80kG/awAS+fr82m6vCL+AdYGTmsmiKluqPH5XiKAk5TiMtCa1htHcdFaeuIVQwjXD4W62BX3z+QEEL0EZnzcp6KazprDzbz8rZqNF0n1arwufgeHgxvwJi7jFmRgaQa9VMu3bVaAzidDaAnVh3laC5UmnAPfpsLrr2ybx/oLGVQFD47YjCTwl706hxa8vbwyNgduAYvpDBSRzFVJ01cjpenN9MadjGpYCOa7a8c2LS2j6IXQoj+IcnLeWj5rlouePAdrn98Hd9+Zhtf+st6TKEWplPOTksaBd4iZjTO6tG9zKYgABMjOsnWnxCc+Dqjbv95b4Z/zlFVlYULF7KDwdzZ9CaHjCncOOffvDnW1aPrbe4QYxoP4A6mMGBmLW+8/Xc0qbwrhDiHSfJynjl+SfTxXJFUvNZRxLL3MOvw1cRCzh7dLxayYIxU8WTh3/hJm4bJJkNFn0RJSQmXzZ3NL6JfZrxiIt89hVcKm3HHzN2uQIrpKiuSJhM2FrEwWseMf8RJ2a+TO/Qwv1z+7759ACGE6EMy5+U8cuKS6KMmZW7nK8P/g8XeSrhqBNr+VIJNTqKBVIy21pMOHek6KCEnbTureWFuJbpi5fqmOKmr/g8WfTWxYaH4WL5w0UR+t66Nlrqt7My8jAl5Y3l1/wt80bCjywKkV9Nn8+Mh36LWmgWTE8dSAnGuWN/IJeoThCq24NkyD7PTiqXYKRN4hRDnFOl5OY+cuCQaYFLWNr42/q+YbYlaIVnxxCaB6Cr1W69L/PWEbEfXE+Vd3OuHsqGkER2ddLfOhaU+tJZWqJBiaZ+EQVX46ZUlbNRHYmgIcWR3MY9PvIWf+q7HE7N0tHstYza3j/o5tZbMTtd7bSr/viibJ01fZ1TZYjxLy2h6fCd1D24guKuprx9HCCF6jSQv55F6T+fEpUht4taRzwDHPtWbbG0d533VE6le8zViwc7l/GPBVBpWX8xfk8qpzE7Uf7l5hYYrGMJojYOvvvce4hy3cHQuj94wEZfdTGu9kcbGbP6RdCUTY4/xDe+d/KnhUn5Q8N1E79nJloEBqyYkcbshwCo9AkDcHaH5yT2SwAghzhkybHQeMbijHX8vVFv4TOZarO3F53RNIdA0lFgwFYPFQyycgoKCr3oivprx2DPKMFjbiIWcNDUN5Mkpd6MbNNLdcNOKOJMadUKXxrC7IpCc3V+PeE5YODqXS0tyeOSdA/xpbw2DbAc4FM7mVeNsmuy5NDnSu79YUcBmxJ1h5kctQf4fChdiQgfalh3CWpIuQ0hCiLOeJC/nkSF2C8ka+BWdaaZKLOZE4uI+MpHaLV+CsKOjrYJOomaLArpKoHE4evtsmaj3Lb75apRUH4w4oqHq0Hp7DIctghLKhyKpovtpGVSFb88bysSJOXxhu5M5721it9+Jx5XSo+tVRxPG2EF+E0liRngCZgzE3WHC5W6pASOEOOtJ8nIe2RULc0HIxLakBhyBZsI1sK1lPuaqq1EUleM/j7enLZ2EzEFSyt/n6rJlHcdiqdD6+RihCToFByKw8AGZrHsazUxNJt1g4P05k/jcDh/lfl+PrrOm/gezfS9R4MvhZL7VeD2zvBOIeSK9G7AQQvQBSV7OE3Fd56+rN1GsbuGKijJs8WB7P0oLEeWvmOxzMZiHdrRXEvsVYzT5GGdMoW7Yc6w+2MSMOVtoWgQGt0LcqRMZooOig64wYOafoESK051OBkXhgeEDuH33YV4Ym8yQrSmk+H147Uld57wA6BpqvBVTeF/HIY/Zxy/yH+dH1bczMFbAtD6MXwgheoNM2D1PrGpoZlz5OgbX7sAaD3Y+qfuI+pcRj5R1OqygEI8mY7L6eC59E4UX7CA0UScyTCc4RSMyTE98BykKhYVfQR19dd890HlkcfMHfH3fCxCKc3BcMan723fpPtkyMBSSW59sH/Zr157jPJzzFB/85pu89+QTfRK3EEL0Ful5OQ+UlpbyzvMvkulJJCfdTdeMBlahmgajKJ1z2nUpu6gJTeaibCON8TcA7bizKoUFX2Ho0Lt6JfbzXukrsPRGltSbyHyujP+34CvETTHytlXgH5qBOzm5o6kabyG59UkswU1d76OA1xhg2+AWWPYf6vx1fP72H2CQIT4hxFlIkpdzXGlpKUuXLsXib0OJfkTJeN2LFqvGYCrodHhDRKFZ/4Dw8CeYm/VrqqqeJBisxGYrZMCAG1BVcy8+wXlMi8PyHwA6sZCBC2t2oC7/Cz9ffAdZajNzl62gZvhAthoKMBj3k5zyp849Liexf5ib2XGVynfWsMC1gLum38W8onl98zxCCHGayLDROUzTNF569SV0dCyqp2cX6f5jf0UnogTJ0nYB0BhoRFXNFBbeyvDhP6Ww8FZJXHpTxRrw1AAk6ucAs2t38uNXH8OtOVl1xSVMa61gXtNO7C2BkyYuqq4wxj+UC92TGeMfSlhXMeY2kFrsIb00yJJVS1hRsaJPH0sIIT4t6Xk5h5UfLicSiFAZTyPD4CGHho++SEkC6FgW3RTaTG1mYoVLpj2z28s+qbims6G8hQZviKwUK1OL0zBIHZKE44r92TMjGG1xYkGV2TU7KXr9N6yYegMjrblE4zUMiGm8HrUTMQY6xgVnesZzZ/0XyIwdKzLYaGxlj7aRZMN+slqi7Bnk5cENDzK3YK4MIQkhzhqSvJzDXt79MhXxVN6NDuYSu48s42GUWPSkc150QFFSUI35AHgVnUhgE/P2vsJjA4NkudKYmHV6N11cvquW+5aVdtqyINdp5d7FJSwcnXtav9ZZ6bhif4oK2RPdVK9OBXQKq6qYYnueoaO+Q4aviXWuIBNbxrIuax3oMNM7nh9V397llukxF7O5lA1twxmevJf9Pj811LClYQtTcqb04cMJIcQnJ8NG56i4FufwujdYH03MX9kTKCQ1azyFSSPJtBagHJfCHB1sWJ82i1eTYryd1EBe7B/M2fsCOW4/Tr/OzSviqKeeTvGxdLe7dZ07xNee3MLyXbWn74udrYpmgiOPo10pjoIQ+bNaMdoSE6bTG+tY4d5AQTCTeS/+hzt92xlTn4otqnJn/RcAOv07A6jti+DHpzjRW7KYWTOVPH8ejYHGPn00IYT4NKTn5Ry1pXYTI6vMvJJr4UIM/LeeR6plIGQlzgdiHrY0r6Q6sB/daKIucxgpkVq+aHiJK2yb0E0WDnhSAYVLt+iUVLUS2LSZpGlTP3Vs3e1uDceK4923rJRLS3LO7yEk1QALH4SlN5L4v6LjKAiRkh8i0GgBX5xtLat5bdBsLjY5eN1TzM3J7/D3yIxOQ0UnuTEWgwunVoHuuZg7Wo1kXJXRRw8lhBCfnvS8nKO8b/yLaDSZzykhfoEdl2LqdN5mTGFW1lVkFl5EoGg4mb5qSpq3EzXUYFR1TNYQ9swIUYuTkVWJa2KNp+fT+cl2tz6eDtS6Q2wobzktX++sVnIlXPNPcBwbRlNUSBqawfAf/BJrSirhg++zdUImC9/cRoN7Ft9u6dmPtdVgYF28Bl80j/o/7u6tJxBCiNNOkpdzUekrWN5aQ4rRwO16+35FOjRFNaoiGk1RDXTQFYVJhnEkH9yNydsGwPPpFo4uqDZa41QPv4Y2V6LyrjHz9EzYbfB2n7h8knbnvJIr4Tu74KZX4XN/Tfz3OztRR1/FpV/5KqBgoA3fBWlkb4kxONqzpG/p3EWMGrCPnY59DD9QSygoWwcIIc4Okryca7Q42ms/IJybAoPGkoyV2ojOW54Yq/1xNgfirPbHecsToy6i4cRKuPgCdHR81hh7MmNssVoAiBddxaG08YTNTow5OdgnTzotIWalWE9ru/OCaoDi2TDm84n/tq8MylfNTKyoo7nNxvCcSgom3EF1YzoqjXQuJniMjo7bFEa3HeQP067jooF72WjbwvIf/5vAtgZCB9vQtdM4wUkIIU4zSV7ONRVrWGsy0TIniiHupCaisTEQJ3TC76KQDhsDcWoiGhVDp6MpChtKWtAVaFANxLQMNlkXA2CJeMi+524Uw+lZSju1OI1cp7XbSr8KiVVHU4vTTsvXO5fFGhvJcfuZu6eSA3uLKFT/D0dkAYdjT5H4P3liAqOhAIP4NS/s/C6b1l/HqrSpzCnYitev0PLMPpoe30ndgxsI7mrq+wcSQogekOSlr2hxKP8Adj6f+K/2EdVuP+H9Y7v/g7ckRjRiw6+H2BaMwimqru4Kxqly2nl1jovKnMSeR5nxOE3R26mJKFiibkru+zqO+fNPW6gGVeHexSVA160Kjr6+d3HJ+T1Zt4eODuUpQPLOJspLQ5jTl5O6ooY14X8ROK7oIIBKM+mm/8VsWMtGq4WtJj9fOPIInuQkPkytJN7+vRJ3R2h+co8kMEKIM5KsNuoLpa8kyry3V0sFEktgFz54enZhPu7+1Vk29KEuAsEUWuLg0k+dnwZ1aI7pNGfasPl1smI6aW3fJqbMIsMUZfIdM3BNyvn0MZ5g4ehc/nTDxC51XnKkzsvHYp88CWNODrH6etB1kpsq2Tm+AGVQNiXLV3P/jcNoHHohf979MGmxCqzqblYmWXggPY9647Ef/6xYjKzYAbZ7ZjCRY1WT25YdwlqSjiKJpBDiDCLJS29r31ivS++HpzZx/Jp/froE5oT777GPwNMKRyrHYdFMp762XVJIp9XSho7C1EO3YCFRrGzqJQUM6oXE5aiFo3O5tCRHKux+CorBQPY9d1P97e+AomC0xrkk/jpPm3+IMijCiMPluEdkkK+/CgZYYbfxvYx0RhyBIX6N1mTYU6DQYFBpyNvMy0lWxtVej6G9UzbuDhMud2Md7OrX5xRCiONJ8tKbjttYr6v2iibL74IRizomYH7a+++JDiF4IFHjQ1OjPbqN3+wjzV/FZTXTWRiajGJOJA95Y0//dgAnMqgKMwan9/rXOZc55s+Hh39H/f/eTyxUgUmJ0TI5TM7ygex0HibflwdAHHijOZVHntPI8B67vikF/n6pyobhChtcq7k5aTd31n+BWd4JAGheWYUkhDizyJyX3nTcxnonp4OnOtHuNNy/LDSdaCiZSCQJUIia3cTVcMc+RSf56rhtClNbd/G5qul8zjuWPLOKjo7BacFS7PxkcYk+55g/nyErV5Dxy6fQTGl80/dLXps9niu2H6YlsBeA7fUpfOVlSPd2vjbNC9/7j8bUfYnJvU3GNn6R/zirU7YCoKbI5ptCiDOLJC+96biN9U5Lu1Nct1sfytacURRk7Dx2XgGf4wBAlwTm6Otsl4HMuiYWaltxGsagtfcIuRYPknkOZxnFYCBp+gzUzz6MEZ0Fqa+Q7XFRpu2mTnNi2pDYdPPEf1WVRCJ789saiqZ3NHg0+3l0uypJrBDijCPJS286bmO909Kum+tKGcLz+mVkDnsPk7lzYbeItRmPqxRN7dz177dCapaRMbFNXK2+Tk7sMsBAq8lD6heHYxst5eLPWu1VeRfFDlIzNZevvRHmb74BGP1qt8vTVSDDCyOPtCe5CjSZWtlp3tdXUQshRI9J8tKbTthYrysFHPmJdp/w/jFjOsv1ixiVV4NmbaPGlY7PbO3UzxKxNtOSuZ621O24nXuoyd7Lw1ek4k3fhOeiP1JpnsxPs/fxg8Lf0XirleSxWZ8sHnHmaK/Ke923vs36tOsYtbW1R5el+jq/3unfRrjc3QsBCiHEJyfJS286urEe0G1Fk4UPfLLJuu33P5S+EFNGKzmZiQ2I3EoqawaPxRR2Yg5mYgo7O+YGRyxuwrZG3hk1EM2gohrfQ1F0GhlIlSnOLYu+xrzieZ8sFnHmUQ0YBs/hqv++nXWOnv27tiZ3fp3lrSLc5OmF4IQQ4pOT5KW3nWRjPSDRI3MalkkH6zficQZ4+fBcAFxVaVyxzkZq6zic7pG4WseR1jgNcygdn8XGWyVTKc/MwxYJMsSQmMgZD7l4YNIDzCuSxOVctHB0LrPv+z4Bi7nbcoUaiVVHewrak2odHDqUFB3g4KpX+ypUIYToEVkq3RdKrkwsh65Yk5hkm5ydGCr6pD0u0LFMeoN5NI+X3sBl6nqChyagbroCBxrH9/SomhlHWwlvlSRTnmEGXSdotLBryHCGBWoINA0lxSX7CJ3LLhtfSMUPv4T/J3872hHX4eh3y98vVdFVBUVPTOdetMdCeZGJ5MMbmKh99dN9vwohxGkkPS99pZuN9T6p+OHVxN21PO6/kgXqRv7P+AgN264BQDlhiCrxWmH+1gCKppNfXwuqSpuSSv2WazFbVHKHuj5VPOLMV3TN/7Dra+PRDZ0/s7hT4DdXq2wYnng7yI7Heai+ibutB0jZHaciH2IHP+yPkIUQ4qSk5+UstHxXLWteXMll2gga4y4Wm9byQujzBGPdrxBSAGdQ54KdFXwwvhiA2K7ZeKuLmX97Caosiz7nrahYwc9du5i+WOMHdV+lWdnEINM7DMn2Y0qy8LkGA5nxOBNDYQwkpkrNdlVQERzF4++V8bWhF/b3IwghBCA9L2ed5btq+dqTW2htDfG2NgkTcb4R/TZ/i32uR9e3pOaBruMIxMnalc7A0TGGTe69LQDEmSGuxXlgwwMArBupsm5YDaYcM5l5fkwGmBIKc7k/wJT2xAVAUcBhCjMg6Cd0eD33v17afw8ghBDHkeTlLBLXdO5bVooO7IsO4Yn4ZUTaO8/8Svc7Rx/PZ030sHzlpRcoHBnjim8u6K1wxRlkS8MW6gPtRQ11eD13D21asEfXWmNWstJ8/PP9Ml7fVt2LUQohRM/IsNFZZEN5S8cOzIdsid4SFYVxGEg3KtiMOoFY1zkvkKio67GpuK0BfvrYX8lLSeHI/nQObm1g8ASp63Kuaww0HnuhwE69nOrkcYkNjz7C36fMoiYyiIkpe2l6Jhmfaia5D/a9EkKI7kjychZp8CYSl1RbKa3BEuZg5DtYyUJNTGpJhqCmsyMYo67TnoyJ9SXvjTbx1K/uIHh1BN9wF7x2NR8uLaN4XKbMeTnHZdo7JxsXVAxga4bCZxrspCgBlJP882tArSWLVRmT0RQDXjcYg4cpfqaNyeqFUoVZCNFv+mTY6A9/+AMDBw7EarUybdo0NmzY0G3bv//97yiK0umP1SrLeAGyUqzkONYSTN3OHIz8P2xknNDLYlFgqt1IrunY8YglyvMzk7jU8ifa7gsTnqBjsrdizyjD1xqmtqytj59E9LWJWRPJtmd39MqZUrKJR5w8UbgYAP2EUUet/b8/HvxNNDXxGafVlsJ7w0p427SDLS+tQdd6NlQphBCnW68nL88++yxLlizh3nvvZcuWLYwbN44FCxbQ0NDQ7TUOh4Pa2tqOPxUVFb0d5lnhNxsfQM99HYPq4zsktgA4cbcaVVHQgUF2FUfmK+ybUM5jC6xcaXiaCXnrOv2LG6xtAPg94T57BtE/DKqBu6beBSSGFVPcHsa3qjww+L+4reSnNJpdndrXWrK4reTnvJ45J3FAUQhYbCRFw9S6MngvvIngwZ5tOSCEEKdbrw8bPfTQQ9x+++3ccsstADz66KO89tprPPHEE9x1110nvUZRFHJyZAXM8f7r1V9R515DwBplnGJIDBV1Q1UUMlBozswgw7eT31sfwOLXiZ7QLh5yAZDksPRe4OKMMa9oHg9d9BAPbHgAl6eUadoQckMNvJ55IW9kzmG6ewdZkWYazOmsc45FU7rWIjLE44TjcSIGlfJDBxg1dGo/PIkQ4nzXqz0vkUiEzZs3M2/esbLzqqoyb9481q5d2+11Pp+PoqIiCgoK+MxnPsPu3bt7M8wzXjAS5r26F8kyJ2ZXOn0lPbpuvDvErAnLUNHQLce6+HUdooFUAk1DSU61SIG688i8onm8+bk3uWr0ZRh3HuGH+x4FFDQU1rgm8FLWPNa4Jpw0cQFoSklF9/sxelrZJ4XrhBD9pFeTl6amJuLxONnZ2Z2OZ2dnU1dXd9Jrhg8fzhNPPMHLL7/Mk08+iaZpzJw5k6qqqpO2D4fDeDyeTn/ONb9e8xSqyYvdECHqGUVda892odZsVtBBbYNYfuLY0bkN9VuvA13lgmuGymTd84xBNTB4zmdQ2raQttnD47t/TE646eSNdR21JYxaG0BtCaMBb8++nEqrnZYjUvdFCNE/zrg6LzNmzODGG29k/PjxXHjhhfznP/8hMzOTP//5zydtf//99+N0Ojv+FBQU9HHEvW9jRQ0AORUZaNXXsZ04DbqGduIsy3Y6GhG1ldai5QAYWuj4l44FU6le8zUIzGDhV0fLMunzVLDFSsYoD1kbq9lVYebDdTdwc/mLiey2/ftKrQ9iea8O88YmzDtaMW9swvJ+PWp9kBWzrsRf3oqm9WCttRBCnGa9OuclIyMDg8FAfX19p+P19fU9ntNiMpmYMGECBw4cOOn5u+++myVLlnS89ng851QCE9d09lfamWqegScynAgmBvsPsSNQwyUZl6HrOspx61w1dBQUyniPVOpwPm7A4Fdw/eBOIo7R6KESxlxjJ3eoS3pczmOxphb8dVayR3tobvHz7VGTuWHVh2SUNfPIRV8i0gqmbS0AKOhkq15sRAlETdRvjxMan87ro2fzmdJdDBw9rp+fRghxvunV5MVsNjNp0iRWrlzJVVddBYCmaaxcuZJvfOMbPbpHPB5n586dXH755Sc9b7FYsFjO3Qmn6w41cskhK2lFxWzXUhjsP8RlDW/SBKzWwkxMvwS70dHRvlWL8l7aRoxb4aIXjSioGLOzKZrxLRSD7AosEoyZmXirbOg6zAgd5ieKmenGsSx6cSWGqjC/G3Q1AEVqC9NMlSQpx6Z7+3UT6/cWczhlGL/e9hSPSPIihOhjvb7aaMmSJdx0001MnjyZqVOn8rvf/Q6/39+x+ujGG28kPz+f+++/H4Cf/exnTJ8+nSFDhtDW1savfvUrKioquO2223o71DPSm2s/xFyUhYEG0HVmNycmSSpAdWA/NYEyMqwDsBmSCcZ9lEfb2GUdwnwCKO1jRdn33C2Ji+jEPnkSxpwcfDX1DEcjzauzNbuWC4CgOR8lrFGotjDXdLDrtUSZq+9nX2Aw9bv2EYmGMZvO3Q8QQogzT6/Pebn22mv59a9/zU9+8hPGjx/Ptm3bWL58ecck3srKSmprazvat7a2cvvttzNy5Eguv/xyPB4Pa9asoaSkZytsziU7du1C37iJnYPMABSHjpAS93eq7KKj0xg6QqV/D42hIyTHvaheA7nGZozZ2eQ//Dsc8+f3zwOIM5ZiMJB9z90AqKjc+rbGh0MacCcZSfXGUNCZZqpMtD1hdPHo6+GxcqbtSOaFNX/qy9CFEAJF17uZ9XmW8ng8OJ1O3G43Dofjoy84Q/3y/Wd5ruw3pETnM9ySSnaFF4O7GXtN+UdeuyZnCE9cMhzHojukx0Wckuett6j/3/uJ1dWxbpjCziIL0w9N46FxF7LQvO8jr9eMcTJb3+ebv3mvD6IVQpzLPs7vb9nb6Az0qw+e41+HfgEGyI5k8MOaP7CUK/EbTT26/rOGgziv+A2okriIzuKazobyFhq8IbJSrEyddylDLrmEwKbN5DU2knfYT3X1k2R3KWl4cu8MmUpxGejxuCTKQog+I8nLGSYSi/Gvst+DCnmBPC70VzAg2sBCVrHUvgjNaEKJRU+yb3Ri+0XdpDJ0yiJJXEQXy3fVct+y0o6dyQFynVbuXVzCwmmJSrnzYho/qXqNEQR6dE9n0MvGCRez9I3fc+0V3+2VuIUQ4kRnXJ2X893T21ehG9pQdZ3PNmVygX8TACUc4BrlNdTsxO7AXcf6dBR0StKsOAZf2acxizPf8l21fO3JLZ0SF4A6d4ivPbmF5bsS884MRpXh46fQaNsFutJlw8bj6cDkiv2kBoL8vayCuNR8EUL0EUleziCaFmfP4d1c4g/wTkULd2rLmMGOjvMlHOC7jqVMz6/Hauz8iyLFGGZx/h4ucG3AkdKz4SVxfohrOvctKz1JwnssCb5vWSnx9l2iL7lwANuH1GNLMaEo+kmvg8SKN0WHi/fvwKIP5IGtS3sheiGE6EqGjc4QZevX8M7fH8OVH+WuWFPit8pxY0NHX6roXODYz8yU/VQHnPhiZpKNEfLtbo7WnEuxlAJz+v4hxBlpQ3lLlx6X4+lArTvEhvIWZgxOx27N4fqUHWzI/ioLIv/mSOQURR8VMMVCXNCWxf/VeCh2ruKLQy467c8ghBDHk+TlDFC2fg2vPPS/aIrCzQNKQT/J8lQ65zOqAgOS3Ced+6L6G3o3YHFWafB2n7icrJ3LNYXpqRlU7XiBFKVnbxFjo8l8rvwFXm4xs9g7ApPDiqXYiSJVnIUQvUCSl36maXHe+ftjRFNcmIsHkBt9n5NmJHQ93O2vheTs7s6I81BWivVjtVMUA8OG/oRRL/yOmHNAj651KVauPHIdQ2oycO8pA8DgNONaPBjb6IxPFrgQQnRD5rz0s+o9u2mNxgjlD8ac1LNPyN1NotR10B35UNSzXafF+WFqcRq5Tmu3ya5CYtXR1OK0jmNZWQvIHjaf4u1rsSj6yWaIJ+iQpFvI0Vy4jZ2/f+PuCM1P7iG4q5sdq4UQ4hOS5KWfeVpbCGcXoqBh9SlUh0exP3gB1eFRaHr3/zzaCa91HVAUlIUPyDJp0YlBVbh3caJCdXe9d/cuLsFwwhDPReNn8n7GJIYktSUOnJjAtL+eHh2GisLK1DWsTtna5eu3LTuErp1TtTCFEP1Mkpd+5o/G0U1mcrxZBGou46XWX/C2+3u81PoL/tn4Zw6Gpne01QE3ybwQupM6c2an+8R0M8pFd0OJLJMWXS0cncufbphIjrPzEFKO08qfbpjIwtG5Xa5JSaqlKKkKZ+BDZkdGkaR33r8oCQuXRMcwMJ5JUIvzYfoH/Dn7eeInpNZxd5hwufv0P5QQ4rwlc176mTU9A3MwnVighKP9JRo6VUYNv+Jkj+97fIWHGGpdC8ByLuLV3Il8a8wX+Mb2pYz2HmCCdyMFyR5IH9x/DyLOeAtH53JpSU7nCrvFaV16XAAofQXlhZu5wGpn3wU2+OAw1wRmUq+2EVQi2EgMFakooIBNMfDX8p/xaPZz7LYfYGxgWKfbad5IHz2lEOJ8IMlLP0tOSiHFM5Q4OtVGnQPGOKXmOMHj+sTe0L7OT2Im4kYHexhKWW4y80o3o/lN+NpiFDg97TeTibri1AyqwozB6adupMVh+Q8AnXyLn0N+C5ELHuWD9yxMCY0lr5thyfSYix9V385uW9edqNUU82mIXgghEmTYqJ+Zok4OGA085gjzbHKEzdbOiQuAR1H579hXWR6bQtSoMm/3egY119CSo3JFeD2ggEzUFadLxRrw1ACJOTEjDvtQgLKpS7l+xHfwqAFOVrpORUEHhoWKOo7p6BicFizFzr6JXQhxXpCel/6gxRO/IHz1/HldGi/bP6JLvb3Iy/pYId8Mv02zz0hrUTIlH1aSUhROtJGJuuJ08dV3epnVHGFMqYfYMJXayGgcmr3bS1UUzHqiwrOu64m8etFAqfcihDitJHnpa6WvJLrkPTUsS5/D0w3/lTj+Ue/tikIAC231HkxeD0NrXFyTvALFlZdIXGSirjhdTjL8mNUcIaMiwqWWpB7fRg+28rphDVV7D/OLsTeezgiFEOc5GTbqS6WvwNIbwVNDHJW7M76OFtU+OnE5zuFYBraIh+suGYzjR8/Dd3ZK4iJOr6KZ4MjjxG/MnKow+SmtPbrFAXMl/q1PUKa+xs5ttby3r65j7yQhhPi0JHnpK8dNggRY5xxLi+742Lcx4ad1xCxaL70JimfLUJE4/VQDLHyw/cWxBCaXIEHbPqKWFvQulYYSNHQajC18e9CD3PalakpzdHZZbdz0t81M+8WbHbtXCyHEpyHJS185bhIkQIUvBywfI/HQdZJjXjJSD6ObzPh8vl4IUoh2JVfCNf8Ex7H6L6oKA9cnUZf1NqB0SWA0dBTgz9nPoyk6HmuYqkyVPGU9AE2BGHc+uUUSGCHEpybJS185bhJkLK6wP5iOlmpGt6jdVl4/JtHiIucqSEmsKEpOTu6VMIXoUHIlfGcXfPllsKUCkKsX4q8x4lM1dCXeqXmTsZVf5D/OGse2xIH2TptI5m4G+Q9wdOb5fctKZQhJCPGpyITdvtI+CXJ5fAo/DN1MbmszjkAc3wgnpu2tnXaMPpGdCJ+LHmFgXgNHjuTgcDgoKirqprUQp5FqSPwJJua6qLRi0mfxjrmemd5sVhsOs3fAO7QY3ey2H0BTTkhKFAjY4kzW3qZcH4SuqNS6Q2wob/noejNCCNENSV76SLxgGs8znR9EvgkqXBlzM3BrkOdnJRMdB6a9bggf64bXTQoDLG5Gh6vJibcwaVwTR3wpACxcuBBVlU4z0UeO6zW0qLtJjgep1w+xLRDmQFo97zk3feQtYg6dvEAt1bZ8ABq8PduEVAghTkZ+A/aRjQ3b+Hn0ywAsUDaQbi5icGMjX/zgAMV4iczJJjIlg8jYVIyjU7ncYuHSyD5yVS/p7jDbC8IYTRlcc801lJSU9PPTiPPKcUunFUVjhOFpzArUUoUtmtKjW8RdOaiO0o7Xmcmm0x6mEOL8IT0vfWTdwQZ88VQWGDawOLqfLdlWNEMEhw6X7gNtrwWPaTimcAaFTTFUXcHtzCAcr+XNi6/ncvX33PaVBzAY5E1f9LGjS6c9tYBOkmEtF1qTeMc+BFUzo2oqmtLNkn8dbHEb+cEBHMx7H2NLDtbWgeQFa4GsPn4QIcS5Qnpe+khzZQMqGjeaVrHJPgJN7VxVVyWMK7qDXE8davu0gRR3ETUDsqlwtXH1yC9J4iL6x0mWTl8QWsGAoiaUwBpSw4nJvF1mnre/Htc8DgUFa9yKI/NFrm96A19bc5+ELoQ4N0ny0gdKS0sJVB1kqrqH95mTOHjip9T21z7HwY59Y7yOD1GjRu4pMpKbvaDvAhbiRCcsnVaAK5RXYICKgsL0hunY4rZOl9jiNqY3TCc/kJjnEjKEiFoCKI569r/2Zl8/gRDiHCLDRr1M0zSWL19Oof0IRneUsNL9vjAooBnCRExtmCIWDhfsY0jhRL445KI+i1eIbpVcCSMWdezLlZ2czcCqTby+7xlckSwWHrmMZmsTIUMIa9zMpGCEAVoKcYxUksTUxilsy9iOtaWJog+WE//5LzCYpDdRCPHxSfLSyyoqKvB4PGhFLjaVTGf2gR0feY2uRtia/SrxUCrfufyzfRClED2kGhKVndul2kZh2dfE5hwPs6vSyQhlUMIBLok3E43eSJxMAMYAkWiQ92oKicdf5vAYhdp//IYrbrurnx5ECHE2k2GjXnZwVw37CzJ4rOgWWpN6tjLDb9rBxsGbGFyYhNlk7uUIhfjkciwmUDOxNph4q2Qyg5U9LI43E4ouIU5Gp7ZmbMyLjSW/6AY8g8wcfm8dcS3ezZ2FEKJ7krz0Ik3T+UtjhHeKZwFQ68zAZ7Z2X1FX11GiYeqTSnGg8l+XfrHPYhXik5juSiaYY2Py/r3UONKZr66jLXpH+9muy48UYDTFxP2XoGaY2Fy3uU/jFUKcGyR56SWaFuf/3lrPqoEuUBRQFHRFYfWQscDJF2boioIeD1CWUs11OVbSU6f1ddhCfCwGReHWgUPYPLKJq8tewBUpRCOT7utFKygoXKiMojwlnwMHDvRluEKIc4QkL72gbP0a/vyN2/itFkskLscpz8zjrZKp+M3WTsd9FhtvlUzmuZlXcukQL9dO+BmKIjtGizPfzZ4kmp0xUgOvoJHabbs4Gjvs+1nl2EhZ8hEG1ak0v7+/DyMVQpwrZMLuaVa2fg2vPPS/VOYVE7KdfPPE8sw8Dmfkkutuwh4JEzBbqHVmoLcnOnWHv0VWliyNFmcHvamZO18P8d5lOqqvtct5DZ3XHKt5OutV2kyejuMp2VYGVQ9j4fo1DJ02sy9DFkKc5SR5OY00Lc47f38MAL/91JNzdUWhxpV50nOmpIGnOzQheo0xM5Np+3XW5I6iLX81SrQNHRcA5WoDzzpX8l72B12u81pCbB+0g1+98DCPTpmGqkpPoxCiZ2TY6DSq3rMbX0sTAEkB7ye+z7ihdacrJCF6nX3yJIw5OYw7mMT91tG4jH8EdMrVelaYdrAho33jxm4KM24feoi/v/x+X4YshDjLSfJyGvnajnWZF9RWkBmMg97t2qKudJ2M1mZGcPIeGSHORIrBQPY9d5MRcvNyyxL+as9EU19nnamMJmsTQWPwVPN3CZhDrFz7PHHtY/ysCCHOa5K8nEbJrmOTFbMsA/jG/vb9i06WwJx4rP31N5f+A+NeX2+FKESvcMyfz/wff5OMsI/nXWa+qAzCr4QJGUM9ul61tbGhvKWXoxRCnCskeTmN8keOIjktUZgr01bIoroYv9wWIivcOVFJimo4o52PZQXj3PfY75izbSO2ULjPYhbidEldsIDPXp5CvTedpNSNAFjj1o+4KiHP4KKpSfY7EkL0jEzYPY1U1UDejBHsfnc7TZkFEIWLG2Jc2BBja6qBJotCRlhnQmuiqujWVAN/HGrmkrooi5//A0pNomBXzsiR/fkYQnxi88dk86+95URMbeDLJyOUgS1mI2joZuhIBy3mxBwajtH3W3T9WikRIIT4SNLzchrt2rWTzYeCaNkzqTLmdxw3AJNb4yysizG5NY7huGPOcBx7+TMoNZsTherS00maMqWfnkCIT2di1kTys5tpNtUSMAQAGNc8LnHyZJUZgXD9Ig5EqgluTGbbtuf7LlghxFlLkpfTpLS0lOeffx5zKANHWwk+zxECMQ96NxN2dV3HH/PgaXgNZ807He/rA+79CYpBPnmKs5NBNXD3tLvRjdBiKgcgL5DH9Ibp2OK2Tm0disotGWHun/A8Iwa0saVmDOvXLUfTtP4IXQhxFpFho9NA0zTeWPYi6JDsGQJARqubLfpWZmVdha7rKMdV2j2a0GxtXokjeT0Tt0Yx5eSSfc/dOObP75dnEOJ0mVc0j99e9Ft+uuanZG5zELMVkB/IJy+QR5O1iai1jSGZlVzQfBGW2hwi9npcJc+w1zCLygPpbN32HJMmXtvfjyGEOIMpenddA2cpj8eD0+nE7XbjcDj65Gs++84G9rz/OuZwCiP9JvyObey2bydzew559uFMTL8Eu/FYLP6Yh63NK6kO7EctaOD2zz1M0tSp0uMizilxLc6fv34nfx7ix9a8iNtGPoPL3kzhwS+Q0zwJ9biEXtN1al3beLthP8Mcu/jCT1fK3BchzjMf5/e39Lx8Sq/vqGHFOyuZZyzjMtP7ONLay59HwD20klV1zbx6pIwM6wBshmSCcR9NoSo0dBRLjFGjLiZ5xoz+fQghesHbpQ1EAzFm+HSqBz9Fcf4+HJu/QU7zpC5tFSCvbTwXKGY27atgx5N3Me7Lv+r7oIUQZwWZ8/IpbN+5m+//ewPzoke4hldJUTydzqcYI1yZv4fBKQ00ho5Q6d9DY+hIInFBJ39GC4aief0UvRC9J67p3LeslP2ONIr9BYwPhtFjBrKbEomLcsKGpUdfDzKVYNBVVrxaStkLD/d53EKIs4MkL5/Q02+u5pGlywnHDcy3LQe6bCCNCugKXJRzCOW4pRaKJUbRpdU0xkYzsKi4D6MWom9sKG+h1h1iuX0OY7dtI5kskvZeh6ooXRKXoxRFQVUUhrouAuCdF19Di0X6MGohxNlCkpdP4I2d1by8bw+b04u4XttHKs3dVj9XAacxQvWkct4b38jyaXWolx6mNjiRQGA4xcWSvIhzT4M3UVk3rhhptpoo2NsGvvxTX9Qu05oMKPiiZiqf/UnvBSmEOGtJ8vIxxTWdv7z4EiO2vcPIqn+Tln6gR9cZkkKU5wWoSw9zoOoimpsLWbx4Maoq/wTi3JOVcqyy7k9H3EH+unpa4vEeXZtpOdjRU+nb/jpoPbtOCHH+kN+cH9Nry95kxu43aEmuY3ORiQqG9ui6JtWIous4I1YKQpO45pprKCkp6eVohegfU4vTyHVaUYCYauS9nKlUt2xG0/VT1j7SdB1N2UG+3Q2AU29DO7y6DyMXQpwNJHn5GDQtzr4X/46GzuribArKb6KoqgRvPL3bzaN1HbzxDGrDJegoTCv/Itdfd7UkLuKcZlAV7l2c+B5XgCdGL8ZW6aMmtBOgSwJz9PXBcJw33d9DMQ7FbI+QZ3fzn3//ldLS0j6NXwhxZpPk5WOo2r0TQn58I6Jk1HyezwTMmONmPvR8Beh2o2g+9NxKWiSNS/fdwtD4ZAqGp/Vx5EL0vYWjc/nTDRPJcSaGkNblDqDC829irD3pTgEHwhqlIR3QaVYXkzYgSpvTjDdqZOnSpZLACCE6SJ2Xj+GVPYd4/Ivf5YIDy5nb4ERDp0nV0cLTeDryZS43v0Qq3o72Pi2DDz23cig8gzsqQ1Q12pl+13BUtbvpvUKcWxaOzuXSkhw2lLdweKeF6n/vJaa/zuueKQyyqCQp4NfhUPj4LQFUNMXF4dhi/uqbjDcT1EZYvnw5I0aMkHliQghJXnrq1fpWfpFRzMyWrVQynRSjzkpbhJBjF+lZywhY3PxSdzExZGNAyEJu48XQvBgdBbOlldyDzeQuHMngCVn9/ShC9CmDqjBjcDrTiufy+GuPs6k1HwwnJixdWWMGMpvrCcaKiKsaHo+HiooKWaEnhJBho57YuXs33922D4BRDSpDqgK8Yg9ROGA5M4f9nbyUFhR0NEVhk83KSy6FPw19l4NpOwGF8c5NWDxvkT9lSP8+iBD9SFUNXHzbt6kLdP+2o+sa8egR4pG9RKLNHHIWM2JPA0VVNQD4fL6+ClcIcQaTnpePUFpayu/fehfv+Au4pW0186zreKRwJL8c/gxpFn9HO18YlrpN7AibEjMUdVgz8AW+qi2nSPfSkpOEfXLXsuhCnE+GTptJwaLN1K3ygJLSqWBdPFJGNPAu6IkExewHY6OJA/mDmbWpCk2vwlMXgzH9Fb0Q4kwhPS+noGkay5cvJ2C2MFVfy7yU33LosJGvjvkrqWZ/p7bJZrg1M8rVLdHEAQV8Fg+tzl1YDKU4vnCzbLwoBKDOvQ7HrJcTFanbZ7XHI2VE/cs6EpejlFgUS+1etg8bw9it29j7VgOadk7tJSuE+AQkeTmFiooKPB4PQbWSW7W/kv7mZHInrgW6bgVwtMTuRQVRZu47VlTrCClo5nSSPvetPopaiDNbttXK0oJp5M34E4ohiK5riR6Xkzj6YxaM7CBmzqU+UEFNWUvfBSuEOCNJ8nIKPp+Pans1mus/pOhtRA+XYXJEuyYuRymgpcEXD8ZRtMRkxFhrIepVvwNVel2EAJjuSqbaOpuXUydjSHsNr+39Lj0ux1MAU9TP6pJhNDsOU1sjS6aFON9J8nIKtiQb21O34jRoVO6cjWrxfvRFQLJJZ2a1jjNiY2zW5VByZS9HKsTZw6Ao/GJoPp5YFvsBI409uu6DsUMIRw+ycd/7vRugEOKMJ8nLKTRZGgmaw4w/lM6H/lEY3D2rz2JwK+QGdBYfnkLBrJm9HKUQZ59FmS5+nG1ghLqFqK2hR9fUZWSzdUYh/k1lstu0EOc5SV5OoezgdgbW2jhcP56qrDxWpdiIRVS6lAc9Sge1BcwHFCa1FnBlQxPJUyf3acxCnC0m5A3iZmUvrxRVErbYTvVjhSfJSVXuQCLWdIJpFXz4na/2ZahCiDOMJC+nYA2qzN6Vidk1jLEpm9hxeRSDqZvCWu3vvI7njURtKRTGs/HNnyUrjIToTtFMjI48Lqk3s3FKKtD1c8HR1+/MuhxdVVFjDQRc2WSsWMd7jz/Tp+EKIc4ckrycQkE4nULTcGw5E4gerOXyrDAAmqJSyijWcAGljEJr/9+o+MC6XcEx+gZysmupuuCL/Rm+EGc21YBy2YNMtxyizZfEK5deizfJ0amJN8nJy/Ovp6y4BEWP429bgckHrXYrhj8+TCwa66fghRD9SYrUnUJ5cyu+UYtoyHyHgcEqUo06G5nGP7mVFiWjo12a3sSNyhNMSVmPsvgKrGoBq9LN5Fst/Ri9EGeBkisxfPAXRtXGaUgN8NgXv8eAugqSAl789hSqcgeity/vKwi8hsUQJ1ttZsfQ0Vy4fRObl73LtKsv7eeHEEL0Nel5OQWjz8L/jUolLdzKSFM9G5nG7/g+LaR3atdCGr/j+2xkGuRl4TI9xtsDZjDdldxPkQtx9pg39Tr8Fh8zKrYxqXkDVXlF7B06jiP5g9BVFQWNwsAygs1L+awryrBptbw9eRzvT5zBvpdf6+/whRD9QHpeTuFgWhoNVpWMqJ8p4Tbu5tbEiRMLvSgq6Br/5FZuiP2ckHE/i2f+HUO3BWGEEEeZXQVM8lbSFJ7Jl4wvMyfpYV7VLyOoZqPGGvC3rcBiiHNdepRx9jgQ5EvNO3hi2ky+8dTTxKMRDCZzfz+GEOe8uBZnS8MWGgONZNozmZg1EUM/1TCT5OUUdsWtANQ2Ovl97WdpmZDRfWNFpYUMDhugzTeZS45UQl5OH0UqxFmsaCZ3ptXyc49C2KQzyR5lgv4KB8MqHlXBkakz2BzH2RZnV/NY6pRMHNnDmbblA1bNHM/Q555lyBe/3N9PIcQ5bUXFCh7Y8AD1gfqOY9n2bO6aehfziub1eTwybHQKB2PlDD20m+wNe9k5oGe7we1XbmDwsoPEGntWeEuI855qwHj5/cxUttAaHJg4pMBQq8akpDgzfUG8+8fw1aTHuafw5/y+4Fv8YvwC/v2F22jMy+XgyjX9G78Q57gVFStYsmpJp8QFoD5Qz3dXfZcVFSv6PCZJXk4hy3eQS95fhqYolA4b36NrvEfcgI4xM7NXYxPinFJyJQuuvZ3k1nTiwaSONdKZTWEqG8bz9RE/pdbSueezze7k9ckL+XBkmPiul/o+ZiHOA3EtzgMbHkBv/6HUT1LP4L419xHX4l0v7kV9krz84Q9/YODAgVitVqZNm8aGDRtO2f65555jxIgRWK1WxowZw+uvv94XYXZxcdhESshHde5AgraPnnxriuvQsBdjTg72yZP6IEIhziElVzLtc9fQcGBO4rWmM/hAkB8P/lbijVPp/HalKyoKOs8Ov5H4c7dA6Sv9ELQQ57YtDVs6ely6JC4kPme0htt4fOfjfRpXrycvzz77LEuWLOHee+9ly5YtjBs3jgULFtDQcPKS4GvWrOH666/nK1/5Clu3buWqq67iqquuYteuXb0dahfJluEA+O0pPWofNShktbWQfc/dUpxOiE8gxenkYEsGadu/SnqTlW220dRas7okLkfpioon2cUyywyCr3wH+vjTnxDnugZfPTnNFjKbrBijMLlyIJ9vGMDV/jSGGHSU9h6Zv+18sk97X3o9eXnooYe4/fbbueWWWygpKeHRRx/FbrfzxBNPnLT9ww8/zMKFC/n+97/PyJEj+fnPf87EiRN55JFHejvULrTkRNXPpEDPNmR0+tws/MYdOObP782whDhnFRUVkeJ0sL01iQE7rqPBnP7RFwGVsRxsoWbWrvh9L0coxPmjbP0ayn/1FBfUGhjeUsTXyzO5dvx+Lpi0nzkjqvhGXpD/zYox2uAiEHezsW5zn8XWq8lLJBJh8+bNzJt3bCayqqrMmzePtWvXnvSatWvXdmoPsGDBgm7bh8NhPB5Ppz+nS8kFk/EaksivPUyyz33yPjNIHNfC/GiYRuqChaft6wtxvlFVlYULF3LY0MgmQytZkeYeXbcvJ5+GdDOvNLYRj0vVXSE+rbL1ayh9fAlfSl9BcnQsE+MNFF9SgSmp88+XzRLltrwaSvRc1leU91l8vZq8NDU1EY/Hyc7O7nQ8Ozuburq6k15TV1f3sdrff//9OJ3Ojj8FBQWnJ3hg+pBMNudcgKrrXLy6vRjWiQlM++tFSUf48vC5p+1rC3G+Kikp4ZprrmG7xcmYljJyQw0oejd7igHoOi8NuZo/5FxBYfww6/adek6dEOLUNC1O+b/vZa5eTu2WS9CbmhgwMzHvpUuZs/bX12dWEm7ou+orZ/1qo7vvvhu3293x58iRI6ft3gZV4Ws3X80bWfPJq6zgM2/9m2R/554dJeqlwPsPHpv62dP2dYU435WUlHDb4IsoCy3m5wd+Dyjd93y2v3s+bbmJ8KoGDrzxh74LVIhzUHXpDsa3lVK9Oo1dgy8gJaMNc3KsS+JylKJAii1Kevj0jXx8lF5NXjIyMjAYDNTXn7A2vL6enJyTF3DLycn5WO0tFgsOh6PTn9Pp8rH5zL96OP8svoJdwSEMfHMreR/uwbyzkuTyp8mo+yb3j7+i36oMCnHOUrLJNtzAiG02vnPg710/8nVqq+BNcVHuHEzTBjd/+eWvWHOgibjWTcIjhOhW/OAHuLcn0TDoMuoCCkZ7D4diQ31X76VXkxez2cykSZNYuXJlxzFN01i5ciUzZsw46TUzZszo1B7g7bff7rZ9X/jB3AU8elsO+rg1VA3bRDD9GRyOH1GcspXfXvSbfqkuKMS5zuBKVLi2Zn4LZ/XUTudUTWPc/lIu3riacftLUbXEsJLfnoIOVG3bzJceX8v0n7/N8l21fR26EGc1S2Ul1UmTKM2/FJQkYoGeDQdF46FejuyYXh+gWrJkCTfddBOTJ09m6tSp/O53v8Pv93PLLbcAcOONN5Kfn8/9998PwLe//W0uvPBCfvOb37Bo0SKeeeYZNm3axGOPPdbboZ7S/OJLuaTo4jNmXwchznXWIS58q6oAyLEXdxyfvXUD31j6D7LaWjqOuZ0pPHTNrYy4YD+uuAelHPJCtVSTz51PbuHRGyaycHRunz+DEGej+tBwdpcsANWAasgDxUospGKwaCftANV1aIsrpKklfRZjrycv1157LY2NjfzkJz+hrq6O8ePHs3z58o5JuZWVlajqsQ6gmTNn8vTTT/OjH/2Ie+65h6FDh/LSSy8xevTo3g71IxlUA1NypvR3GEKcFyyDXKh2I/FAlAmtcTLDYUbs3sZ9j/+uS1un28N9j/+O6LUhGi41cPhtsPsC0P5Ge9+yUi4tycGgymapQpzKq/WtHDxchCNQjzq8juwJz2Cyt3ac1/XOI7hHp6K9XWvmW645fRanouvdzYI7O3k8HpxOJ263+7TPfxFC9K3griaan9yDjs47GQoDn7+LzLYWTp6C6BjtcQLfD1KTlMTfXr+WDa5pHWf/fft0ZgzuWd0YIc5HrzW28Yu39vKNV7aRnrES49VbgK7JyvGvQwEjT/tV0rct4DtjpjPoi9d94q//cX5/n/WrjYQQ5y7b6AzSbxiJ1+zhwj37yOo2cQFQiAWMFKyLYk6KMitpQ6cl1g3evhuPF+JsE9d1Xnz6GX770BIm7Pgd6rwdwMmXRus6xEI2at78Ci9/YCTp8CLips9SaUjqs3gleRFCnNFsozM4dOVKnk3+d4/aH8hKxh6IkWzxkx+q6TielWLtrRCFOOu9/9RvWfKnh3H4W9kybSAG56mXRhutQfIaPyRpwiJWjbqefb4ATc5hfRavJC9CiDPetOLP4cnpWc9JOA0CdiPGTJVcrQXQyXVamVqc1rtBCnGW0mIRnH/4Z+LvKLxbMq5H11mH7GdGxluklrZxwKyRP6TvhmUleRFCnPEy0mZgHzqYYLINOPk0PR2dWKpOZCiAQsbwFuzpiRVJ9y4ukcm6QnSj4b2nsbWGUYDdGYOIKD0b/onl6aSrXlz6DnLsZqYOkuRFCCE6KIqByYUX8dzir7Qf6ZzA6O2vPZ+PJd7VFLBaA4xIbcI0QiWWKUNGQnTH/+6HHX/fMH4Cg3wxwiF7t0WtNR2ibjNKIPHaZfHy06tH9+kHBElehBBnhXmTbuStqcNQZ1yMktx5r6NImsIH3x3OOxNmUcootPa3NodRo6h1O3e8u5fXGtv6IWohzmyet94i/PxqAGKqiiHbBqgcPJgoC3Ky7fwUIPh6JkZv4ufs1jmT+ryOUt/toiSEEJ+CwWDkvmydn035IreODmJO+gCDW2FN/lSeyLuFFiWjo22a3sSNPIHNo5NtrSOy4wA/MBpY+LkpGE61zYAQ5xEtGqP2vl90vN4wbiL2eAQUaG4uZE/phQweshGLJdDRJhy2U76hiAs/OETrRJ2IlsbCCQv7PHZJXoQQZ40rJs5ny+ZHeKNyJHNGbWC3aRwPK//dpV0LafyO7/MZz9OMKt3LiEx4e5eBNXNHMDtd6j8JEdzVROPjr6I1N3aUH2hzujq1aW4upLl5AE5nA2ZzkEjEhtudhbX6MC051USHhInULURR+r7SvAwbCSHOKvfcfCcrDTHKtw3mX8qtiYNdilEk3trembEIY1THVn2Q+e61rD3U1MfRCnHmOVr8cWPocKfjLnfbSVqruN05NDYW43bnACpKLIp3qs6arbMZ/P/+g+ett/og6hOjEkKIs4jRZOTL2WZesU9IDBWdohiFN8VFdW4xiq5jr60gZ9crfRusEGcYXdNpW3aQlZkGyi1tnc5N3bYZSzDYdaJLx8U6SjSMIeDFqiWT/4YPAzr1/3s/ejze+8EfR5IXIcRZ58YbZmG2pfSord+eAoqCrmvkr3oBtL59kxXiTBIudxNxR/hLgQ+/y0FbcjJbho5k5eSZ7BwygglbElsCnHSmLmCpP4JiUWltTGNm7W7QdWJ1dQQ2be7T55A5L0KIs0566lRGBN9haw/a5sesKCiJOjCHQ8QPvo9h6Nxej1GIM9H28i1Up1rR42EOZuVz009/iycpueN8Zmsz169chiHJTNBu7zhuDQahuQaTrxVlWAZZ75V3um+ssbHPngGk50UIcRZSFAOjrV5sAf8pu7izgxq3KnO5ouBO8u3DSPLF+PDeuyhbv6ZvAxbiDLCiYgW/3/8HNqcZGOU+zFslU/HYOxeka3Sl8vvP3YTzSD1z33mH6WvWMvedd1i87FVy2o6Qf1Ej0VAmGY2d548ZMzP78lEkeRFCnJ2mjRnPJZtWJl7oneu+HE1ovrc3jAGwGVKYlXUVWY6RtOLjlYf+VxIYcV6Ja3Ee2PAAtiP7iEe8rC8akThx0snuOn/4/JdJb2yiqLKSrIZGVF0nMjxEY2wUE7duQz36oUFRMObkYJ88qU+fR5IXIcRZaeTUqymuLWN+6QbsUW+nc9khnV9uC3FxQwwApf0N2jrmWoxZRSi6xrv/eAxN5r+I88SWhi0Uba1lyYtxUlvfpc6aeYrJ7iqNaRnsHDKi0+GDkVxGvHyEAVXV7e0S12ffczeKoW+XS8ucFyHEWclgMDPo8lnE1ldwsf4qh7MHMqDmi2SEdSa0xjn6VqqjEUzdR8zixhh24oyPg9ABvM1NVO/ZTcGosf36HEL0hUZvPTe/neih1HvYbdHsTAVAA1pSwFXhZUDVsZ3ajdnZZN9zN4758093uB9JkhchxFnrmsVf5eYjj5ITKufyaDN5dbFO571Zm2gY8RQxa2vHMTWcRNg2nsodOfjaWk+8pRDnpJwDLdi8cGRAAd54zzZQTHe3opHYDuBvc5OZa7sMm8OFHnajWJxk/+Cz2Ib13WaMx5NhIyHEWcugKFx3zfW8mzsdXWnrdM6btYmacY8Qs3ROUDSznyljV+MZ6cTmcPVdsEL0o6KoE01RqBp7FaN2hchsbe46V+woXSOzpYkxB/bSkgK//qxK3Hktc8PTMGYOxzRgKsbM4eiB/ht2leRFCHFWW5TpYsHwK6iNtRC1NKOjoaPRMOKpRIMTh/WVxJ7Unxn1Bu/8e2lfhytEv1jefISmzAzKU0LopPGNpf8g8cNwssnuCsWH/8XPr1f4r9tdVGTM5v95p3W5p5pi7pPYT0aSFyHEWe+mCYPY2zqJmrQVgEIgdW9iqKib+YiqAum2NpS6bawoW96nsQrR15bvquW+qlY+HG5kV8p22lxDmLqnjJ8+9lsyTxg6zWxt5oo3/8GAsmZ2ZuZhMkT4S+N1Xe5pcFqwFDv76hG6kDkvQoiznqoqpOVeS0XFUpLyVmDQe/aJ0JtuZd0f72Xuby7FoPb95nJC9La4pnPfslK0SDpvTGgmKeoERaVsyBeYs+1xZm3fxK4hI2h2ppLubmX0gb0YdJ2/LcjHYKvnmooFmOj6s+FaPAhF7b8d2iV5EUKcE/ImevnnhilQs4r0MZt79Oa2I20kE7YF2NKwhSk5U3o9RiH62obyFmrdIYZEKqhP0vDoB/GZW9Ezx7Fr1O0MPfAc48v2dLQPWVxsGXEZH45bxuIDuVwVvBBMx+5ncFpwLR6EbXRGPzzNMZK8CCHOCdnJWewq+CcXH4lyha+WHaEkQhbDSUtZ6DrEgi4GtqVxcMQoirwNkNP3MQvR2xq8IQCs+YnJtbqis3rgf5i//1YaMsfRmDEWV9sBLBEPIXMKba4hGKY8zg07HTQ1w+vhx8iwDsBmSGbaF68n/+IJ/drjcpTMeRFCnBMmZk0kL68ZY0YrDlME10Er0O3+ctRvvZZU9zhMrlZ2v72rj6MVom9kpViZkRNk+8iJ6ChELCPYU2Dn9VFv4je3gaLSljqM+uzJVGVnoU55nML8A8RrTSSHzejoNIaOUOnfg9/mPSMSF5CeFyHEOcKgGrhr6l28XX8HNIHe7GBP6WiGFm/HZG/raBcLplK/9Tp81RMBjUGVl7F18oNEIt/FbO6/1RNC9Ab7Oy+xedAQpuw7QKvtTppsA2lMy2FLhsqWwREGb/GS0hLFr+jUWkPcZo/iXz0QS8BL1NY5RUh2pfbTU3QlyYsQ4pwxr2geqZO/BcvvJRk/npoxHCz9IvaMMgzWNuIhF4GmoceVGFWJBdOYQyZ//dUSvvbDR/o1fiFOJ/ebb7LxtWXc4LJSnVPIjlmL8CUft0LIauLgzDRM21owNIQg7uTRHV/hu1WbiFk3drpXSnoG+SNH9fETdE+GjYQQ55RJU7+Jz5ZOoV5NkqaBrhJoHI73yDQCjcNPWhtdq8omuW4FbW+83g8RC3H66fE4mx74GW2WGFU5hbw8/3p8SY6Tto2WuEiMpiqAwt+yR6EpnX9O5t50B+oZtCJPkhchxLlFNVA24+sois4UdV2PLmncYUaN2Kn40V3oO1/q3fiE6ANH3vwDu7Oz0BWFd2YtShzssoN0+2uLgXjqsSHTNpOdGmsukOhxuXLJPQydNrMvwu4xGTYSQpxzxlzwbb5bF+LHux+n1HgdkZiTk31W03UddC9arJoGdy4m/yH8f/wqyd9UoeTKvg9ciNNA1+NUvvcXonohVXnF+O0OihpjJAc1fDaVygwj+tGJt+0JjJ5mgdZIxz0GjpnO1ZdNJn/kqDOqx+UoSV6EEOcco2pk2oQZjE2bzw/f/BC8Y9F1HeW4T556+7KjaGAVAJqu0JRkY7XpMm5ffheMWARn4Ju2EB/Fu+7XOEKJDRONhkF889U2nMFjy+7cNoW3Jiaxd0D3E9SnfvYzFAzun00Xe0KGjYQQ56QvDr2YhZFN/HvyUPwpq0D3dW6ge4n6l6FFD3QcerdwLO84JvAn8yyoWNO3AQtxGrQufx3jGw9Ql2skbbiNUTYLjlDnDRQdQZ3Pr/YxoupYT4vaEu74e67TytTitD6L+ZOQnhchxDnr81nF/K83TrlmYIj7L6jGfFCSQPejxaqBzkVgShbtw1zp4i9ZV3Gbp+74wqJCnPFeefnXlDb9ickzXRjMhygEYAvRwPHlARLTcnVg/hY/+3KNoIVRW0OJyewK3Lu4BMMZUs+lO9LzIoQ4Z106/hIKmg7hd6ThNdiJx6rQovvQYlV0Tlx0TElRXEUNTLlwGd/Y/iceWba3v8IW4mN7u/wtng39lckDdTB1/tVutLWSP/NPJOdv6TimAM6gTmFTDHv9yxjt5WTazfzpixO5yGYjsK2B0ME2dO2EKo9nCElehBDnLJPRxJQByWTbraxPnwac2Ndy7Ej+zPqOKS45M2rJ2v9P9vzfw30WqxCfVFyL8+Can/PZ1AigdLuoKHvCM6Bonc6l723CFn+db8zL4t3Pjmf8a5U0Pb6Tlmf20fT4Tuoe3EBwV1PfPMjHIMmLEOKc9u3FN7NXbyUnXeeNrPlETZZO501JMQZeWo1rkBdIvNGb7W3snzSdoqr7iO/8T3+ELUSPbWnYQoqhmVRj19XQRykKmOyt2DPKOh1P832QKCugZND29F7i7kin83F3hOYn95xxCYwkL0KIc5pBVfj65TdCOB9sB6kfN57BiysouqSawYsrKLn+ILmufFJqp2FrGQF64t2/MDiYnU4T6gu3QOkr/fwUQnSvMdDI4EBBj9oarG0A6OiElQj1WStIN2YycFXSKa9rW3bojBpCkgm7Qohz3sLRuThmTuDND56kIlZESl4AgOT6SWR9+CVM4WMrK6KWFhpGPIXqz6bKNIqp+maQpdPiDOar8zOoai4MPPCRbeMhF3r7UGnIWUY8JZnbXXfAztipr3OHCZe7sQ52nY6QPzXpeRFCnBcmlmRzY/OVGAgQCbhIqptE3vZvYAwnNpuLo7HDvp8PLQdoKptPSttQjE1piW54T7UsnRZnJE3TqFrRhDWSTNRv6LKL+lG6DtFAKoGmoWhqGI+rlIi1mZn1l/OZ3Lk9+lqNew6gafGPbtgHpOdFCHFesBQ7sQ0aw2zL/ayt+AK3NIwEQEFhdcpWHs1+jiZTW0f79DwXX/jAhO4CRQXNWyef9sQZJa7F+fCx3zB+W4BDw0NUrc5h4KXV6HrnuS9HE5rD+y6gzbWTqNmdWG4EXDryAoxOS9ebn8S7z/2V4GtBLr75jn7fLkB+FoUQ5wVFVUj9zFAqnTYu8DdhCqd1JC6/yH+cJmNbp/bNxjYendvIa7EUAJ5vkrdLceZYUbGC//nZHDJ/93dc+ZOJBoy4yx0cfiufuK/z8GY4bGdP6YXUeV1ELe6OQi9q3ELRcCumomQMzu6r7eq6jj/moSlUha+liVce+l/K1vdvT6T0vAghzhu20Rl4Ng8nQ3UDiaGiR7OfS5w8cZVG+xv8rwamcfGBAC9sbSRpTBuLMl19GbIQXayoWMH33vkuj7wWw5AxDH9yCLffyIA2HyWvt2B9RSUyRCHu1ImGrBxyTaZ5QOGxG7R3zSR7BvPir/6M3RFi3qW3Y9nQ9Wsd3UZja/PKjrkyAO/+4zEGT5nWb/seyUcJIcR55bY2P+/mZwKw234gMVTUXTFRBVosOv/2Dufexx/mxSefJd7dpAIh+kBci/PAhgcYcUQjwwtV+bmsNO8iu7WZiRX1WKNxFF3BUqZi32TAsSvKrA/Xkn+kquMeyUoAR+sILOEMUJLwtTTx0rP3E56qdumBCcS9rG54ierA/k7Hvc1NVO/Z3SfPfDKSvAghzispWQNIfmU/IZObZoO7R9esKxyMpijcsPRfvLB6bS9HKET3tjRsoT5QT6oPNEVh/bAsFC3GyOB7BCfHiQzV0JVjCfbRvHzixg0kNTqYW7+NklYnlnBW4oQe6Gi7csXjZH1/EoEZGmsbXuGd2qd57cijXRKXo3xtrb31mB9Jho2EEOcV56DpzNn6GLuLC0iLFffomiGKl8bMDHIaGln93LNoM/uvu1yc3xoDjQC0JsPukhKSs6oYVrwO/9xjGyuqreB8zohtW6J/QgHskSiFdQo5Ljcr9MRkW5sCyUZoal8l7W1uomZfKbYhqVT693xkLMmu1NP7cB+D9LwIIc4rSZOnEk13krliBW6fQnrEebI9AwBQdJ2cWIyvhdZyaIAVgAN6Nk+/ubEPIxbimEx7YsizTlnEkQtTKSl5D6Mt3KmN5oLW22MEx3feCsAU9VBqu6Lj9WibAbshuVMbX1sr+SNHkZyWcco4UtIzyB856lM8yacjyYsQ4ryiGAwU/uRnOLxeWr2VXFwzGgUd5YS5LEdf/6C5FQdh8ot9ieNxnR+918zyXbV9HrsQE7MmYvZNJBYYy6jBiSHMLlsCtL92fz7WaQgpaMumJjYWmwJT7AbyzCrBuK/TpcmuVFTVwMU333HKOObedEe/9j5K8iKEOO+4Fsyn4Wf/j4LKjcz3+niooYmseOfiW9nxOA81NDEvEATgQu8+yi/MI8UYBeC+ZaXEz6By6eL8sPLwCkZXf4afWvdisgW73csIBbQ0iAxJrBEKWpy0uYYwzGLkUoeRHLOCJ+6nKXRsIu/xvSlDp83kyiX3dOmBSUnP4Mol9/R7nReZ8yKEOC9d/IXP8hePSub2XzMvEGRuIMgWq4VGg4HMeJyJoTDHf65s2ZvMkIYjtA3YjH18K/92z+eRd8r49rxh/fYM4vwS1+K89sK7/Cw2n/eVNeT15BpnIsEuHbkAFJVMo4oOKDpsb3qr0/LnE3tThk6byeAp06jesxtfWyvJrlTyR446I+Z7SfIihDhvTU01sderMSMNDMCUUPik7eLA+xMsHIlaST9YSfZ2F7PHbuO3K2B4TgoLR+f2adzi/LSpagO37UylquynBKb1LIGIB0ysnjUDX9I4nFFINyo0oPOBey3x9lVEKekZzL3p5FVzVdVAwaixp/U5TgdJXoQQ5y3XsAFUBVwEY0asxthJy728abPzi4xM2ow6oMM4neTIdlIrk1Bjo7lvWSmXluRgULvrvxfi9NC+/3uM27cRciXjq8sm4rNgSgqfdOhI1yHmN/PGqM+ikERao5Phdp3/szVx0VVT+ZwpFV/bjDOqN+XjkORFCHHeGlAyCiU5lbfqhnJl/p4uxep+aR3Dv7Ld6OidTvnMOr7Bqyk8aOawewEbyluYMTi9T2MX55d4MIhzVykAweSRWByfo2F7Gfkz/9TtXkYH941DV4042wqZYjewJ2k3X73xcgYPGgAM6PuHOI1kwq4Q4rylqgaGfOkWyrwZvFw9kvhxb4mvxyfzj4zEKuruJkUG8j9A0WM0eEN9E7A4b9X98lco8QgNGeOoGHwTipqMr3oi1Wu+RizYud5K1G+kcuUAmtxj8Csxxtst7EnaTUtqmOKBRf30BKeX9LwIIc5rl8+9mMePNBF490XqI/nc6FtFTVsyPxxxMarpme4vVCBoi5Ft2sjhppK+C1icl/bt2UwOCvtLvtDeQ5jIqH3VE/HVjMeeUYbB2ko8bKJl/yrm5RwhGJpCW877rDLvAgWuWXgNqnpu9FlI8iKEOK8ZFIWvXr6A2/MGc8Xq14g6jDhDbryArQfXJzl28dsV+xmekywTd0WvePrRz5OUMx+LSyOinqSqra4SaBze8dKet41xjg851LKXFclHKLA6ufPiOykpOXeS7HMjBRNCiE9hUaaLB4xBajLzCR/wUZvhQo+l9OjadK+BrJE7pO6L6BWvLH+E1513sj83g9ZBPZtXlZ6RWDWXpLYSMHsozS5l+IjhH3HV2UWSFyGEAG5Iq+fzuypJzgmQNLGVeKAYLeqku02kdR0sITOfjVmYoFYzxB1iQ3lL3wYtzmn/+c9jfNU8i/E1R7CVZ9BU2LO9hJyWxIaJTaqRWsdB6gP1bGnY0puh9jlJXoQQAlAd2Qwzb8RojzHMdYAM3U24LrEPzIkJzNHX03c7sZXs5Gdtz3JV3iuUfVDZx1GLc1Xrm2/wc+tA1HiU5EOJ/bdixiEYbS2A1s1VGkZbCxPYgTeeztL83R3bAxzd0PFcIcmLEEIAFM1keuFh4iE7aZ4IPzP9nZh3NKGqL6HHnJ2a6jEnI3ePYoTHSFJOgLLBSczyvM2Yfa3EYt39YhGiZ/T/3959x0dR548ff81szW6S3fRGCEkIJCGU0KuKIGAB29l7PT39nZ5X9O5rOe9U9Kqn5+lZ7tRTTz3PAnrSEZEWIPQaQgiQ3nY3u5tsm/n9sUkgpBAgIQl+no9HxJmdmX3vkAfz3k95fwIBVi34F2Uhsfxs5WcEAlYkScJTM5SIYZ8THKx74u+ZAkhEDf+Q6Hovfw0dRGH0jpZXmxd0PFeI5EUQBAFA1iBf/AJTlYNoHBKXaDfypP8tjLZUXAcexV18Dw0l1yMduJHr/5fCldvKkFQLWyqH4zFqMJgdaP272b7u6MnfSxA6ceDDZdToZLT+Rqz+w4Qlb8AUsw+AhrJRJE56FW2IrdU52hAbiZNeJadhC48nzeGD+GBLi4REvCme0bGjz/bH6FFitpEgCEKz7Hlk3wvrP/kbsI47w1Yw2bGddyvPowYL2VWHmXZoPyG+pkUcD1dStc3CoXmpBAbsoyjaRsrRfcDA3vwUQj9WuKWSbZ9sIzrnKH8L3IN5jhtYj6LC0SOjUEonELCnkTHjeZzOGAKNVjRGG37LIfLqxvBA1scMrZsPzmDiAvDo+EfR9LMKuicjqWpHw9H6J4fDgcViwW63Ex4e3tvhCILQH3kbcD+bSojUEGygV6G4MJLGTYamA45VrVOatpZfHsPq1BjmekZzw4OPotV07bthQFHJK6qlsr6R2DAj41MjxVID31OKovLur9ZSE7uIkSM/AIIFEre5NXxeY6YOf8ux0T4rdzgnMUaNRuux8p8IJ3/NmI454MBU+iAA8aZ4Hh3/KDNTZvbK5zlVp/L8Fi0vgiAIJ9KHUFI3kcERK1tW4PXv0jW92DqxkAkmMOOXOnnj/9ViNlv5+tXXufWCK09a92XRzjKeXribMvuxCr0JFiNPzc0WNWO+h8oKbGwy+7lw+P+AY4nLP2v0oPpb/epVa2383vo1j5fcw+T6TK50e/nbYJUHYkrITH+BGFMMo2NHn3MtLs3EmBdBEIR2hOdcy+qDOdSrobir9PgbNLRZ/KiJDIS7G8g6orKF7TgUN/e/l8+inWUdXn/RzjLufy+/VeICUG5vPOm5wrnJYW+kaMwhLBobkhRs8fvMpgOktr96Tdt/j/sEBRWL18DrWjuPjLqRS9IuYVz8uHM2cQGRvAiCILTLPCKdyE11vC5fwp+dV3fpnAsOjyWgNFKQFEaq62CHhesCisrTC3fTXp998z5R9O77Z78R4gMlLduFHhlboJPHtARVujp2mQ4AcKFlWE+H2GeI5EUQBKEd5nFj0UdGM9xeSYF+UJfOCfVmoVFkovTFpOuPUG5zt1u4Lq+otk2Ly/FUoEwUvfve8ccaiLeZW7Ydga6NfarV2gGQw/Q9EldfJJIXQRCEdkgaDfFPPU7ougA/SFhBldHSbksJBJONxlA9NcOPMrFyGoOLajEk+0hsLGt3xemurkItVqv+fqn4dimTbDUoisSemsGU2AZ36bwIfzh+ow39oNAejrDvEAN2BUEQOhA+axZZfi9Hlr7LnuGpRG/8NjiA97hj1Kb/um9wMTz3a4Y0moh918vnw2ZgMnuorvcQUNRWM4hiw4xdev+uHif0f/vWf0di/kp2Zdv5cPWvqfNEAArmwS8ga+3tD7dSIdofwTD3YCpG/g2LI4aIiIlnO/Re0aMtL7W1tdx0002Eh4djtVq56667cDqdnZ5zwQUXIElSq5/77ruvJ8MUBEHoUGSWhZIolcDACNZMmUJDSOu1pgMRKnX3+GnMDaYxeoMb+z1+ZhesITrTxjNf7WLqCytaDcAdnxpJgsXYwfDf4HMqwRKcNi2c+3x+L4vfeImFKZN4dfdN1HmsTa/IeCrmotJ2iYrmZsC7qy6mYuTfcMZtxuOp7LaYAorKusIavthawrrCmj43/qpHW15uuukmysrKWLp0KT6fjzvuuIN7772XDz74oNPz7rnnHn7zm9+0bJtMpp4MUxAEoWPOCiaZV/JFpIkSXSqlSYlEV1WSM2A1UqQHX4ba6mugJBFch2aek9QtRzAmGDBUuLj/vUZevXk0c3IS0MgST83N5v738mk6/Nj5TX8+NTdb1Hv5HlhWvIzXv/wdl1uSWVvW3O1z7O/dX59DY8nNGOIWIunsLfvD/KH8sNpK5pBXsUUEp/EbDLHdElN/mMLfYy0ve/bsYdGiRbz55ptMmDCBqVOn8vLLL/Phhx9SWlra6bkmk4n4+PiWH1FsThCEXhMah9XuIzF5A7IqoUoy3iHgn9SIb6ja/r+iEiiRkOE9zIqyYQyw1jJbzms1g2hOTgKv3jyaeEvrrqF4i7ElyRHObcuKl/HIN4/QWOtgYdQYLH4vqXIt8bID6biU1l+f07JExaTyHOaUjeH1ox6uU9djs2oBCYMhAat13BnH1F+m8PdYy8u6deuwWq2MHTu2Zd/MmTORZZkNGzZw5ZVXdnju+++/z3vvvUd8fDxz587liSee6LD1xePx4PF4WrYdDkf3fQhBEISUyUjhiTiPDmBI9kr27r4Ava6hS6f6QhqQPCr/qpvBYt3PudA+lryiWialRwHBBOai7PhurbArKvb2DwElwPN5z3NRQTLTtsDm3Eoy9ce6fVzARimUQw1DCWbIMgF3Ohf7FjFPswYZ2JEeBlIwex6S8QSSdGZ1XU42hV8iOIX/ouz4Xv+d6rHkpby8nNjY1k1YWq2WyMhIysvLOzzvxhtvJCUlhcTERLZv386jjz7Kvn37+PTTT9s9fv78+Tz99NPdGrsgCEILWQNzXiBu0Z+YH3M91wz9grqKAV06tdjgRPIE8Kg6ltlnM968l8r61gvkaWSpJZk5U/2huV8Iyq/MZ9JWPdM2eFg7eTI6twM54EPV6giYwjBJcL5ajz7pHxx0TMRfnwNAPLU0yEbyQgagRtswGOIZkvEEsbGzzzimU5nC312/s6frlJOXxx57jBdeeKHTY/bs2XPaAd17770t/z98+HASEhKYMWMGhYWFpKentzn+l7/8JY888kjLtsPhIDk5+bTfXxAEoY3seUxU4eghM4/F/5bbjy4ly/kdGrM3OMblBKoKPpeWyB3x3MtbLIu5CFugjlFRdmLD7uyREJub+0/81tzc3C+6ovqGgKqy3uZk2ZEDXLSslrXjx2Au3IHs97Uco2h1eOIG4gu3MsqWQemA92gsvZGY+kTGy3v51DibjAkXkDI8F6t13Bm3uDTrT1P4Tzl5+elPf8rtt9/e6TFpaWnEx8dTWdl65LPf76e2tpb4+Pguv9+ECRMAOHDgQLvJi8FgwGAwtNkvCILQnTTD5vFsdC137yomL/ZCJqw+SticRZw4d7p5VkjJ2jhQJfQoXFy5GP2IAONM+0kN3QjM6dbY+lNz//fZV1U2Hi8ooczj46rNW6mxmNFWFrc5TvL7MJYUAulI4RHENEZTGfcVTzQa0Egquqo6Rky9F7mby//3pyn8p5y8xMTEEBMTc9LjJk2ahM1mY/PmzYwZMwaAFStWoChKS0LSFVu3bgUgIUF8YxAEoXddGhfJm7LMH0t2UFJ/NUmLIHziEtQIpeUYn0tLydo47EXHJhpIgG+HhYLkyWgOPENc7EXd9m0Z+ldz//fVV1U27t55CDkQYM6BUi6sNHEkIgzwt7tskQoYKg7jD7NiVIzIumqizBU4vKGkjLy/2xMXODaFv9ze2G4iLBEcUN4XpvD32GyjrKws5syZwz333ENeXh5r1qzhwQcf5PrrrycxMRGAkpISMjMzycvLA6CwsJDf/va3bN68mUOHDrFgwQJuvfVWzjvvPEaMGNFToQqCIHTZpTFWFt84BZ+2kaP1V7Fn6SuUL5jDoWWJHFgwkN3/TsfXKFOfU01pdi1lUQ0oqKA6cdfO4NDaVGy2jd0aU39q7v8+CqgqjxeUcMP2gyxdXMGPDqsYfJH42klcmkmA7PehcdfTqAn+vVVrNGx1pZE1aXqPxNk8hb/5/U+MB/rOFP4erfPy/vvv8+CDDzJjxgxkWebqq6/mpZdeannd5/Oxb98+3G43AHq9nmXLlvHiiy/icrlITk7m6quv5vHHH+/JMAVBEE6JTiNz8e2jWPrmHlQ01DiG4SsrxpLqIHD1QT5o1BxbUG9QPaF+DaP3JjKkxkl94ZW4XeVERHRfPP2puf/7aL3NyfSN+7iyXOGrkG24ZA/ahhpCTn4qPsVFtbEagEOBFK6w/B5DqqXHYm2ewn/iwG9LqJ7bZw7momFdH/bRkyRVbVO3r19zOBxYLBbsdruoDyMIQo968T9v4VltJcTlxBTzFo6p5fyzpnlxPAkJlXSDQrhGweGXiT00laHlM0nJKuDq+/6v2+IIKCpTX1hx0ub+7x69sE98a/6++bSkmrCXF7PGUBDcIYHG5cB0eP9Jz9040s+uxBLwh/G3bbcy9oc/ICQnuocjDv5OvbT1MG8eKKdOA0qEHiSJBIOOZzKSuDTG2u3veSrPb7G2kSAIwml6+Jq7uMxxK5aKUG4cVsnbNl3TKxIjQvxcafURoT2WTtiiVlKyz4ezQk9FxdfExV3cLXGIir19W+Tm/Ww0HA5uNP0VBExhKFodkt/XfteRqhLQquxOKAFghmP0WUtcABbV2PmdvQ41pvWEmHKPj7t3HuLNnEE9ksB0lVhVWhAE4QxMiL6QhgHrOKILYA/INCcud0R5sWpat4NYNArZ2d/iDPdQ8N4TqGqg2+IQFXv7LrmqApfkaT2QRJLwxA0EaNta1tQhMrSkEr1Py/DaSTw386lWiUtPrj3UPEano9lrAE8UlBDoxY4b0fIiCIJwBuJCRhBit+AIuACQULnSGqzZcWINGEkKPpcGDd3MM8sf5JY33+Gme7qv7ktPVOwVzly9r/3K7/7wCBpJx1BxGOm4Oi9Gn5/s0hri7S4u3zyIa/251FbuRZIlQnKie7wY4XqbkzKPr8PXVaDU42O9zcmUiLAzfr/TIZIXQRCEM5CTlszn+0YSHh1s3k83KK26ik4kSWA0uomPL+f/DozB88Vu7rw8u9vi6c6KvUL3MCeEQweF5f3hEfjDrGjc9Qzbto3YmloiXY0tjTSxVfWYMoK/H7aFB1mleLn/gy09Woyw0uvv1uN6gug2EgRBOAMT0qIprZ5FkmTAqgkOzu0KvT64PtJfVhfw4cqDPRmi0MtS09PQB6R2+oeaSRg0IQw9UkbUcYkLgNWtRWpavyhg9/CfL/Z22p1z/OKfpytW37V2ja4e1xNE8iIIgnAGNLLEVWMHsWr3jVxp9eEIdNxFo6oSNlsclZWDUB3RyIBdq+W7hQf5sqLu7AUtnBWKEuDIru24aqqY4E0L7jwxr1CD/8ndtAH5uDEkCuDWQ13qqFaHS67Ou3OaixGeiYnWUBIMuk5r0CQadEy0hp7R+5wJ0W0kCIJwhi7KjuemNSN5xzoT4pZiD0C43HrMS3V1MoUHxuH1mgFIwsmt+u3s8aSAL4a/rTnExVda0bS3WJLQ7xRsWMuKt1/HWVtNtiWS4ZH34GoIYZthPwGNt+W4ELeb3C35DDha0rJP4ViV3dEhl7a6bk3HzTctzrQYoUaSeCYjibt2HgoO0jr+d1JVUSWJ32Yk9ervqkheBEEQztD41EjiLEbeKJjHVQEd5vQvgWP/7ldXJ7Nn9/ltT9R4yQopQPVp2OOw9OoASOEMKQEoXgvOCgoO2Vjw3ucAJJmGMNwQTqlX4Yg7kggm4NPbUWQvsqInoeQgMVWuVpeqDYOaUNg8eDKXom/Z7zdr2eY6+Qy17ihGmFpVyqxdG1iTPgKX8Vg5PbOngSmFO0iN00MvTpUWyYsgCMIZOr7OSlpJFhMHfIfWYENWdCiyn8ID45qObOebqqSiMR/EZUjlq30VTJkokpd+Z/cCWPQoOEpRVFhxYDygJ8k0lAn+gfiKlrEj+TwgWLpQ77W2nFoTM5o10SMxuLayOfEdArLKqIMShZG56LTXYzzuVyb68sHEfbW9x9ceUhSFRYsWkeZwMKi6jDJLNG69AZPXQ4K9GhlYtGgRmZmZyHLvjD4RY14EQRC6QXOdldUhBt7YezWoEinfPYu9OqWpq6jjEQSK1stNq5bxD3e9GPvS3+xeAB/fCo5SAErcFpx+AxIyuZHTadzxCflRM3B5jxDw7iXgO4Kqth7ULUkavKFjiOd8kt3DOTTkGlaEX09GQCZKK6GxGIi6OYvQETFnZe2h4uJiHI7g9G4ZSLJXk1FVQlJT4gLBarjFxW1XxD5bRMuLIAhCN5mTk8DKUgMfrsllyaaHudtUg7eLEz982kQMngYe33uEi2PF2Jd+QQkEW1yOawdx+oPdPNHGAdS5/GwZfTNu9zfQ6Dx2nhSKzjQdjT6j1eUipFHURpSwwZvG+Q06Rk5RiJk0AkOqBakpIelo7aH4bqzz4nQ6T37QKRzXE0TyIgiC0I2GpA7GVxugdmsG/vjtLVOiTyaQsI/fvfstD937kBj70l8Ur21pcWkWqg0Oxg0PGUOetwKf68u256lOvK6FODSXEqMZgtzUbhLq9bFITmG0J5SG6EL8k67FmG5tc3pPFyMMDe3aLKKuHtcTRLeRIAhCN7olbTRyVCMVaWa0HgsWSyV6vaul5HsbKsgBA+e7DhB9xMaP/vM25Sv/jRrovqUDhB7irGizK8lkx6z1UU4yPvc3nZ4uu1bxepib/boABgnMmm8ICdeyQKdlccPATmcNNRcjvHxUEpPSo7q1inJKSspJF0YMDw8nJSWl297zVInkRRAEoRvpNVoeSNKwOd1Eoz0NtcFAXNxBoJ0iZU3boY5U9tiuIZBs58K8b4h79lUKZszAsWTJ2Q5fOBWhcW12yRIMSs3C6ysHteNuFQkICzgJ95TxhcnLQZ0Hp66BzfZIlKaidN0xa+h0yLLMnDlzOj1mzpw5vTZYF0TyIgiC0O1+mX0e8+qLmJ8ZRtymOQTsCYTbspEVfavjZMVAuC0bgycWpxJDaeQwTB4/G9ITWRphZPMTvxIJTF+WMhnCEzl++Owu0rGHxIDq6vi845gUNwALDS72+cYiNV0t4QxmDamKSmOhDffWShoLbainUXE3Ozuba6+9tk0LTHh4ONdeey3Z2d23pMXpEGNeBEEQesBfL72c3Ec/5e3oMUTUFhPpiUZfFdWqxofOa0E67sF3xDSIsX6FSFcjtWYj+SlxyH/8HefNmIGk0fTipxHaJWtgzgvB2UZIKMB34Tno60wgmbt0CbfGBBLUa0KorY1EaspXTnfWUMPOamwLCwnYjxXC01j0WOemt1qVuiuys7PJzMykuLgYp9NJaGgoKSkpvdri0qz3IxAEQTgHaXVankouY53fTbk7OPi2ucaHsTEWvdfaKnEB+DpjPDc8+zL7UrNbqpruNGlw5m086/F/3yiKQlFRETt27KCoqAhF6doaVWTPg2vfhfAEikkipsBBua0RVR8NUscDWlWgXhNKqfHY7KBKjY54i/G0F1ds2FlNzXt7WiUuAAG7l5r39tCws/qUrynLMqmpqQwfPpzU1NQ+kbiAaHkRBEHoMT+4YjRf/XEpDaES3kYPekVH+98ZFbyyl6I0K6ps5u2r7uTyJf9mSNFuGvU6ju7aTtakiWc7/O+N3bt3s2jRopbaJgrgSBxI+phxjE4bxERraOdT17PnQeal1P/xGSpVB4ctMom+Q4R7LsDf3myjJqujpqBKx34fMnJG8vn9E0+rxUVVVGwLCzs9xrbwIMbsqJZp1/2ZSF4EQRB6SspkXh/0Q2Ye+REVqVouLpSaOheOT2AUVCQWZIWjapr2qyorJl/C4EN7kFWV0opSsnoj/u+B3bt38/HHH7dsF0YnsDpjFI16AzgCsLWQBIOO3wyOJ145SJW7ihhTDKNjR6ORj3XlqSoc+W4bZaOGE0sD1ZKKJsaMSZpDwPVdq8G7DZpQVkZNodCc1nKyIeDj/10w5LRnDXmK7G1aXE4UsHvwFNnbnX7d34jkRRAEoafIGnSXPM9lry+houQA/82+mtl7PYQd1yPhkCWWZBnYk3NcF4MkUR9m5WjCIAaWFnHw0BamKwFk+fs77iWgBMivzO8weTgdzWXwm61LzWZbckbrhQiBmrq1PLrkPTSBY6s1x5nieGz8Y8xMmQlA/cYN5A8fhsVSgV7fQLg3hMXV4xiZVI7VOx6ny0+R38BRnZGSkMRjLS5NU+hHB0wMHBrR5dgDitqqzstwR9em1iv1nSc4/YVIXgRBEHpS9jx+ejfc8k4BYwv+wxuXXkdMqUSoK4DTrKE41XisxeUE9aYw6kO07E2ZxZzd20nOyT3LwfcNy4qX8Xze81S4j9VVOTF5OB3Hl8EvjE4IJi4n0Ls3El79Upv9le5KHvnmEf50wZ8YERJg09bfMmaKHYPB3XJMRuM63tl7DSXV8YSE+alTQ7CpIRw/O8mgeBnhM/MT2Yiv2IGmC60ii3aWtamwO8McwtPoTnquHKY/6TH9Qd8YeSMIgnAOk3Pm8dhVWdQahnLrkv8xwF/CzpFhHBps6jBxAYgcpOKKCrAyS6Vg/eKzGHH3URSVkn117N9YTsm+OpSTTNs9ceDs0qKlPPLNI60SFziWPCwrXnbasTWXt1eA1Rmjgi0ux7e6qAqhde8BbdcSUpuK9Hy29UkWbn4QfUYZer271TEmg5P7R/6T1OgCipQobKoJkJBlL7qI1cyUv+D/uUP4uWwkUS93qVVk0c4y7n8vv1XiAvCNq4FKlHYXbGymsRgwpFpO+h79gWh5EQRBOAtyJs5m6JEn+XPkVdyzZhXbBqZRHyK36aIAQFWIpJZZaZ+wTs0mrWIJi2rv5vyAH42m//yzXbilktUfFeCyeVr2GcxaRk5PZswlg5BPGN9x4sBZAI/WQ0JkAqXm1mX4VVQkJF7Ie4HpydNPqwupubx9mSU6OMblBDrPvlZdRSeS1QBD3OVYQ4IJ6Il/lZIU7BW6OfMTLHUm3BgIwUdp1P8YX++hJHcyoaUOEsuCcZysVSSgqDy9cHe7CUoA+AuNPEPrlp3jWeemtRqse2LXU3cuMdDTRMuLIAjCWSBJGuaOCsWiujiUPpFBpUuDL5ywwnBwW+JW/oGMwrBBuxjgDGdLVB6frn6XgNI/lg0o3FLJor/vbJW4AHhcfvK+LOIfP1tN4ZbKlv3NA2ePT1wA9H49EysnkuhKbPMeKirl7nLyK/NPK8bmMvjudhIXADlg6/DcGS43i4+W0uDKxqpV281BIZjAGI1uhkYWkqqpwSLXc/lGmclTytgXNQJrYxLQtVaRvKLaNi0ux1uFn/+jAb+5dYLbvCr18XVeFu0sY+oLK7jhjfU89OFWbnhjPeOeXcZvFu5iXWENgdMobHc2ieRFEAThLFDVADVVX3J34BVKdIWU69/njj0LiKT1N/tIanmY3zOODUgSRGhVXEk78Wi28JviPzP7v7PPqKvkbFAUldUfFXR6jMftZ9Hfd1K4pbLNwNnjNdfCGVkzsu3yCk2q3FWnFWdzGXyT19Pu64rG2u7+GS43f6qspkKZylzntC69l07XAKqEt6qeyNkV1BCFrTGV3LrghzqxVaQ9na111Oxb/Gy7NJnoe4YTef1Qou8ZTvyj49okLu11PdW6vPxjzSFueGM9U19YwaKdZV36bL2h/7Q/CoIg9GM220Y8nnJydWUcddazXAsTfWu4kPfYq2ZhIwIrdWSyBxkFFNAfkNDYJYZIjdRqwkEPFe6KloGi7Q1WVRUVT5Edpd6LHKbHkGo563U9ygpsbVpcOvLdxwVIFlebFpfjSUiYAiaiG6OpDmlbaC3GFNN6hxIIrvjsrAiuP5QyOVgNtx3Z2dn8WFX55mg99TpDS9+PrKqMcA0mpf4CbHIJu0wHUFWF7MMKj/ttbFCzMUZNoibzgy59TtWtw1HlIMG5n7AEF3/hfq4qKkdvScY6N61L1W+7utZRrCWkw+nQnXU9Ha/M3sj97+WfdsG8niaSF0EQhLPA42nqIpEkJhj3sNwfSpHJy3AUstnV6ljjFgnLf7RobMEH6Y3AnNBtrJyQyuphPkpNpTyf93ybsR7dWRr+TLgcXUtcAJx1Ho4UtF2duT3GQOuHt4REnCmO0bGjj+3cvQAWPQqO48bIhCcGy/hnz2v3usOHDePP0XXcvbsYgOkVPn62x0OcRwWuBaCmYjWe7R9SkHUecwdMZHJsKXNGvMVe2kk8j6OqIDk0TH97DabUBJTJZl5R72F/dR73jLmU+PHjupxcjk+NJMFipNze2G7yIQHxJ1kT6WRdTyd6euFuLsqO73NjYUS3kSAIwllgMMS2/H90vEy4V+JtbTU2f+vVpo1bJCLe0CLbWp9vdSpctbyQ6TtNxPrGUe6uZME377a83hOl4U+XObz9MSQdkdWTT/EFaNQce+g2dyc9Ov7RYwnc7gXBdYYcrQf34igL7t+9oPV+JQBFq2HHJ1zm3slb2QO5qkblha2NxHiO/aX4SvPRr/sXO1Mu5v8GXEI5VsKHHeQh6VWelX/DK9JPeFb6DQ/xKhuZcOz6ajChiPhIQqPAoYFO/L5wDpR9gKFhE4mDUk6pVUwjSzw1N7vp87fWvH2yNZG60vV0XPiU2RvJK+p40HJvEcmLIAjCWWC1jsNgiAckZAmuTmgkIEtsKh8YPEAFFLD8J9ggfuK6R3LTIVM37mNcaRp1CX/i3Y1bWP/ZEgJ+pUul4U9ndeHTkZBhxWztegIzKGVQm9WLT9SobaTaeCwBiwuJ4zfJPyHpqI4ju7aj+L3BFpd22ySa9i16LJiwAMrOzznym7HseekOjrz7CMrbl3HJe1N5fLcDiWMPR1VV8Gz/iAAyf82YAkBWSj2v6u6nlqhW71JLJC/y85YERuMNI+rDZEK2Bq8WZU7ntUE7kFCJMES0bjHqojk5Cbx682jiLa1bobq6JlJXu56OdyoJz9kiuo0EQRDOAknSMCTjSXbsfACQGGkKMFfR4XCkkFg9l8rM95EP21q6itojA1H1KuG1u4l3jGXduDv47Ns17PnOwDAVEvUdfx89m6XhZVli2nUZLPr7zpMeGxphIGloJHPUOa3K9LemcO2lI5lnisful+CohcOfr6Sw9r80p2wZcSrzIks7OB9ABUcJFK+lID+fFZ9+idOfdCwOrYeZ8XoMOrlV2hioLkBtrGPV6EuolTSgwv709OCLbeZGy6AqvMudjGEjcdvnoln9XxTAFmbCrzuPlFqZoqjtPD7x8dOuEDwnJ4GLsuNPa5rzybqe2nM6CU9PEy0vgiAIZ0ls7GyG57yCwRAHwHXGSpBqCascS9q3fyRmS/tjMk4U0uDhfs+rjGU9X+eOolrdx0Z3gFJv5yshn83S8Om5scz5YQ4Gc+ffkadem4EsS2RnZ3Pttde2aYFJTq7i/AsWUVf3OA0lL6OveAk889FYilodp2no2oyjgo3rWfDxEpz+1jVVnH49+XXBFaQCwKYIDYvitRRqHSiSxOoBenLRcL7FjFNnbL8+D4AkUytFs5cs1D1bmlaugk+mZ6OvG8zkQ1dxR9YdzBo0q0vxdkQjS0xKj+LyUUlMSo/q8piUzrqe2nwUIOEkY2h6i2h5EQRBOItiY2cTEzMTm20j6sFVXLHsL1Rp64jyWzH6s2ng65NeQ6+Jo7wonJt1H/FetES8rKGWADsbIEEnIXXwYD3bpeHTc2NJHRnDpq+L2L78KB63v+W10AgDU6/NID332Fig7OxsMjMzKS4uxul0IklbqKh8r01VXp3Zz6CLSji0FOxFwWTnxGSkPYoKKxZvaNpqO2qkIeBiRayWP2QZqDQGv9uPDInnDukS/k86n8KIML6K6dpj0+GOQbNhDXVh4bx21e0kUoTGoSHMG8EVlgu7dI2e0tz1dOISA8fr6hia3iKSF0EQhLNMkjREREyE3HHMXPU2641vITl/ihQ9GMkYgdpY1+55KuAx6gg3laOzOShcN4zLU79iZ9RIzKqKxplGjV8lWtf2YdNbpeFlWWL8pWmMvTg1OIXa4cEcbiAhw9qmwm7weJnU1FRUNcCatffR3hiW5sq1SZMrsB8KA1WixG2h3qcnVOvtoFFEokRKw+lspKM2h7UJ4Xw+6lgXiaQGuNqzj+r0K7k2O4Q6U9c7K+RNHjakJfHqdfeTK+1i4NrZLa+dymysnnJ819Oy3eV8trWEWpev5fV4i5Gn5mb3yWnSIJIXQRCE3iNrYM4LTPz4Vpw6DzX+H2IYcR2Nea+1ObT5Eb55QAz2o17AixEbjlJI1O2hKlbF5VvPUfdEoi1Zbc7vShG0niTLEkmnsGpyc12cjkgS6EP9hMa7cZaZUZFYUZHOvKQ9qJyYngS3nFk3w54V7V5PkSSWTbkkGCtwffV2Ju+r4jt1Jh9OMbU9QVU7XNrB0mhHs9dOcWwcI/wlTD4aglM9lvic6mysntLc9TQpPYpfXZrdr5YKEMmLIAhCb8qeB9e+i+l/P8FcfyeelGHsMgwgkF9OmOtYN0ujTkN+Shx2c0ibS/h9HiJKttGQlM7mkEP4Gz2M1Y0Cgi0uXS2C1pd4PJUoqsT+unTsnnAsBgdDIgqRpdYtMVrTsXt0oD6aBSVZXJpRjfb4MTDhiTDneULVQfBp+8nL0YRBOEMtTK/wcVe+nQLHAMqkAXw1t6klpqOFi1ot5BhMm87/9mtkVWW1eRLj9h/FaTvW6hIaEWx16muaE5n+QiQvgiAIvS17HtLQOXz92WgOHynmPo0DfwIsr82kujAUr6xSaw7peJBoE0PFYVzpw9kaWs3AcRqys7J7pcJud/jukIVnv/01dZ5jrTURhjpuyPwvY+K2t+zzu1s/xir0mcg/fR2ObGhTYTdJCRAaGY2ztm3NG5cpjOkVPmblOdjrlgCVwkQ39SGdDFY94e8jzGnnwrX/I+HwYb6OnU2dHIVSn9H0ajCxaR6g3F8FlAD5lflUuauIMcUwOnb0ac+aOhMieREEQegDJI2e3Gm/Z+PHPwJAK0OoIYadnZc/aUX2+9A0OAmYwli+azWjLpnYLxOXRTvL+PnnTlSsrfbXeaz8bdtd/GjkW4yO3Y7PpcVZ3rpLZ/pt9yJr9ZDads0hWdZw4e33suBPz7V5LdRdz+jtDsobVMKauplqLRVA8knjHbu/kYvKaiiqPshGzWCWDJiGKkkMrzlMsjY4syw0wthmgHJ/s6x4Gc/nPU+F+1hF5DhTHI+Nf6zdpSp6kpgqLQiC0EckxF3MtIiJLdvmYz0ixBk7XvvneLqG4DReV0MDxcXF3R1ij2u99k77dWQ/3Hs1iipRsjYO1OC+sKho5j3yKzImTO70+hkTJjPvkV8RGtm6G228V8/XHh/hqowqSVSE7sEQ6FpxtqyjXs6vD6U4NIOSkCQiFC9Da4u5fsYQZt+RzRU/yeWWZyd3b+JyXHVgila3FN/rKcuKl/HIN4+0SlwAKt2VPPLNI2d9sVDR8iIIgtCHTL/uHVxPJxCi9ROmdQKhDA6rJtdayn+OjDjp+UZPKo1qCUjgdDp7PuBudvK1dyRqPRH4rX9h9i3JOG11hFojSMoahtzF7ouMCZNJHzeBkj27cNrq2KYeYMV3FSRUSuxJ0rE/6hDTt+SREJqGubEBl6GDui6qSrjbw8CaBuQwA+GecsY01nFYG8UPz0/nqht6qDWivfWbTFEw4joYekmnC1GejoAS4Pm851HbmfmloiIh8ULeC23W2upJouVFEAShD5F0eqRhDyOpkGLahlkTwoVxhSSZ7IRoTvLtWgpDK6ei9QanRBcU7EdROi9c19d0tRS9TzuK5GEjyJpyPsnDRnQ5cWkmyxqSh43gSJKXp7cs4YgGlOH7+O8kE5M3fY3s8yIDUwqbxteoJzy4m7bnrduFbDnMHyQPCUVbuaFkC0sui+Oqm+ecUjxd1tH6Te4aWP83eOcyeDGn7TpOZyC/Mr9Ni8vxVFTK3eXkV+Z323uejEheBEEQ+hjTDU/QkPkzFJ/M7MQSwnReNDIMi+z8n2yd6QIkSUaj6AGV7dt28Ic/zGf37t1nJ/Bu0NVS9N1Rsn5/fhnvrPgz8zN288iYf7I2JZsB5cWEuRxoGpxIPi9pVaXM2p2H2dM6qTJ7GrgsfzWhdevZF1VEvq+em26YzNWf/ZOI2bM7eMczpAQ6Wb/pOB0tRHmaqtxdq17c1eO6g0heBEEQ+iDTDU/w9Q/y2KAcm+0yOKwOnXkuSKGtD5bC0JnnotEHZ7YosheQQILQ+hA+/vjjfpPANK+909Ew4+4qWV+4pZKFK37HHSn7CNW52UsWtVI0Znd9y/sYKg4DkFZVyk0bFjN363fM2L2RuVtXc9P6xVy+/EuWj9tIuUbPLCmcYddcg6TpwW6T4rVtW1za1XYhyjMRY4rp1uO6g0heBEEQ+qjYCCufG89v2U7Q7yHcGIHBcie60GvQmS9BF3oNBstdaPQZqKgE5EZ8OhsalwOtvYb1ThW3J5kvF3zRL7qQOlt7p7tK1iuKyuqP9jF02KLgNjK7GA5AjTUGpWl8i67ehrGkEMnvQwaS7NVkVJUwoLqUUZs2snnAUa6McZNd7iPdqaWswHbaMXWJs+Oum7aOLUR5pkbHjibOFNdmpfNmEhLxpvjTWiX7dInkRRAEoY8anxrJYfMIqghHBWRJYVr4W0iSBo0uCY0+E40uGUmSWwZTujVrMRdux3R4PyGlRYwsXYmpeAkl5fV8+uHz1NSsQ1V7dmbKmWpeeyfe0rprKN5i5NWbR59xyfqyAhsNKe9w2DWATdIEHpZe5XPpGgCqYhJ57eafs39QsEqxrt6G+cB2Qor3YSw5SMihvUTtzieptIjwuX5GIBHjmA6chbL/oXGnfs4pJTzt08gaHhv/GECbBKZ5+9Hxj57Vei+Sqp44Cql/czgcWCwW7HZ7m9VJBUEQ+ptFO8v4/IO/8ar+JSDY+lDYOJHVjrtwKcem+wbkRtyatWgrt7Yc16z5H3k1LQZvaByZWfuYNvU+YmN7aGxGNwkoao+UrH/no99RVbOO/4VczO6UccGdbSrlwuWLP2DIoT1t9ucWV6C7xoY3V6Fk7T3Ul0wA4Iqf5J7SEginTAkEB+M6yjjpuJdmt33Zbs2b09FenZd4UzyPjn+0W+q8nMrzWyQvgiAIfdz7q7+gfN2/ucu3GKsv2PWjqDK7/SPY7D+PI3Ikfp0Nc+F2JL+v3cZ9lWAhvPrBw0CWSRqwm8vn3d3nE5ju5vV5+faPY/mp7ZfUXpCKT2/ocBp0qMvBD9//A3LTY9Lo9TG4yo35inI8uQHknbHs3vMcoBIaYeSWZyf3fPXc5tlGJ01epOCyCA/v6PZp0z1VYfdUnt+izosgCEIfd0lOHG/WbeWZ0gtwSjsYU6dBpz+fP0XvJVSfw+QDB7HY65H9vg6vIQEEvESWJuEOdVCiZrF69WtcddVMJOnsl3fvDaqi8tz7LxK5eQzRF9mpMHQyY0mScIZa+NdV9zFx+xayjjRiIZGGu5djjDuKtCyEHZX3I+uCdU7OWtn/prWw2tR5aR188I85z3dr4gLBLqRx8eO69ZqnQ4x5EQRB6OOs1nFMiohk+DoDVQ3DeTlehzFkC7OMORyMG8q/M6ezvbFrMz1C6raSVBaK+VAD67dlU/bdehoLbajKOdUI34ZzRwULf/9Pxn2zDdc0E+enr+vSeTkNHobXTEaxjqJ+xGb05aVsfjuTLUUjkbVJNBgczLon++yW/c+eBw/vDHYJTfwRmE5YdDM8MZjgZM87ezGdZaLlRRAEoY+TJA2p7hsJFH3M8NQLCXWFsip1ExqdG4BApIEyQ9eSF4f2KF7vIUYcdVFVruXuhAyu3yhzkUfGOje9360+fTKqovLtgi9ZufG/6GstTBi8h5EZEoUkden88sbN7LLuwR6TyY8PrmdtVSoqsDdd5Ujc3/jZ3B+RkRrfsx+iPbImOJYldRrMeiY4q+iEhSjPZSJ5EQRB6AdCfalEV1UT4nbjTzmPWfsT+DTZBVZAkrAOGUpIaRgNgfqTXqtRr2FnUji5xRXc8MXbvHH1XQRIY8pXn2JsNBOWno7VOq7fdyc17Kym/LPdmHd8x2xdHuWzLNQbIwgzHCITB5FqNbVEgtROJ4SqIAfq8PkWMsXzI0ybNrE2kIqCyqrcKhrTPTzWTQNVz1hzIvM9IrqNBEEQ+gFtTAyyqjI6Px+Tt5EfTbuBEQVuZH8NqApRoUZGR83o4tWCBey2p0WRXVnJZUv/zT/DD/LJuFUUOJ4kf8tNrFl7HpWVi3v0M/Wkhp3VVL63i6LiLykP38SK7MvYUTSdIxXBWiQyCrfyD0AC9YT6N6oCSITWvsct3+gwezMoRoMKLBk4hNGDL2Px1Yv6RuLyPSWSF0EQhH7ANHYM2vh4BpSU8oP/LcDscbMubhZRFR8AEtV6GGAeypTYKwjRhHXhihIBRcfOiXpCUvYz98DfeSXwEBsJTvttaKzkP9/+mX99+zXrCmsI9KMxMaqi8uXSj7kj7f+orcjnmxF383XICD5WLZQaolqOG8cGHub3RFLb6vxIarmoaiX3f76VVPkmyhsdOHCwOmkKF40+xEPTZ57VmiZCW2KqtCAIQj/hWLKEkoceBmDVqHE8fdePuXzRE6wYF4c74ma+WGMgxqOCqrCz7jv22E8+KDXm/BKyF/kpGTGAKCWWgsAgCvUSCwJDcCiWluMSLEaempt9xgXizob/bfiCR/c8TvYhSLY+xmpzDU7pY2SdnWzvhfwoaQ1+Q13LpBwFmb1kYSMCq1pHTmM5pZ+fh64mjqqY0bjilmMavpkhEYXoNRGcf8H6ft+l1heJOi8ieREE4RzlWLKEiufm4y8v59tR4/h0ynlEF3/NviGHGanM4NeHLgEVqhqPsLL83ye9XvrcYkq3+XB6LuNj0wVUH1clJszvxanRokpyy97uqHDbE9RAAPemzXgqK7iy6hkmaUeTY5tCfUMtf4t7I1jnRgJJlfl7/W24s18JnthONT/zrgc5eMhBjXs8jTG7yL3gNSQp+OLwnL9972rjnC0ieRHJiyAI57DmB7W/qgopOobfF9SydvdRpnqWkKwMItcyj5jGAJ+W/YmAx0/bVYIAVIwhPiZctoe/+k1s9uhRfOF4Kubhr89pPgRQGVp3mH2RKUhAfLief18Wi7fIRojGTHRmGsa0CKSzUeOkA7bFSyj57TPI1VXsu0hBnmuivuYSpmybxU2DH8eutbWqQze+8Wp+6rJQlfk+fmNdy35tQyQx+27kK38MnupS/CEusoatIjr6CCCTM+wl4uIuPvsf8HtCFKkTBEE4h0kaDeYJ41u2L4mpYemeOqTwbLZpQnhyipncugDDigcRt+EATfV1j7tC8DvrrMgDZOx08Aetk1/HDWexZSiyeROagxCoz2k6RaLcEsmk0h1UWszcUlWK992pmLThQAM13+1CMULMD7K6fZq1qqh4iuwo9V7kMD2GVAuSLKGqAWy2jTR4Kln73+1kv/w+iiRRdF0MoeeVoKou0rZNZ1dIAQ6drU3qlmf8L3/kau7f8AfCTfvxG+xoPRYc7qEsDC3B6l5EbUQcGekbmxIXyMn5C3GxInHpK0TyIgiC0M+NT43EI4fhUnU0hppQJInNkVrc1lzuDF1F3fo4fC5dy/FhWg/T4w6SEV4DQKRf4aWSbfxCexHrTZfjGezDe9CJw2ZClWTsmjDSDAYudxYxJfaKNu8vNahUv7eH6JvPLIE5vkVJ8UbSsFdGcXhbXpctekrH11Cn+y2yvxIUyPyXlgOT01kzdTRTkxYAUPHNZWRhYKvO0eF75Rn/y8bUz8j0TSfMMYUEWxlm/TJia3djGVNGesRaJEnFYEhgSMYToquojxHJiyAIQj+nkSWeyA3nxU0DOd9WiMHrwaM3YJXq8Sa7yEo9gKsshLQdHqx4GGCyc3wvj0ywLeaJ4td4kbtQkXHqjewfYMCxX6bYlEa0K4/RiTcDIJ2wFpAkSaiqSt3CQozZUS1dSIoSoGTPLpy2OkKtESRlDUPuYJbO8WN5tAm5GMff1+a9AnYPcUtD8Y9MxhVTyYBvvCTNcJKjW8M0ex75gywoAYnszS4YCZF+S7vv1UyVFPbol6M6EnjaKXOoMp/B0RcTmWAkJj0WgyH2nKh3cy4SyYsgCMI54MorprDu8+WsGjCY7LKDbEkJzp4ZZFA45JHJMNkZYe64JUICLDhJoYRDJGP2NpJb46bgAhND3V8RpZMpVZyYCBCvWJFpm8Aodi+eIjvGdCsFG9ay4u3XcdZWtxwTGhnNhbffS8aEya3ObZlFpQa7t7S5N7Zcs3WMEioKkTtuZ59XT5p5C0a1BFeFHketEX2IxBFbIpnaPYTIqxjdaCXWa6FSZ2932I+qguq3cKsPZNcRJsdcTuyFw0ifMuWU7r1w9onkRRAE4RwgaTQ8ccsUPnz+Lf7aeCW6JC97NZnUEMVAQw0Rts7nZgSQWW8ZwV79QGq8UeRot5AxOI/zDG6IAJLA7dnOjgPjaChPYYI9kjTzCKQTqtMq9V4KNqxlwZ+ea/MeztpqFvzpOeY98quWBEYNBKh4bn5T4gK1WZNI0XfcYiIhY1LCMFqjWVQwmvHb9ZgaGgFI3d1A7ph9+FOy8Ndtwaip4YvDh3k8Xsdys6lVAqOqwc059kmcX6lhUFIcYddcgXnEWVyjSDhtInkRBEE4R4TPmsX1wHnPPc+7rrH84/obeJc7eJg/4DF0PBvoq+hpPJ7+Y8qMxx7ca9RMblEbGM+Gln16vZus7FXs4TxWahU8eX8iI3o6uqQxLcdIoVpW/P31Nu+hSBJHEwbhNIVRsmAhL4wdj06jDY5xKS9vOcY3tGvrBAWsZkoSqljriCentIqMsGpCMkZQ67kLOTTy2PvW1/JE9VuEZx/is/BjCZzZY+L8Q0P4QTEMSthFzNP/6NUZU8KpEVOlBUEQzjHNA1//V1vPb/UGBko7uU19i7kbDmDwKq16UL6Knsbd2b9FRW29xk9TifyH+T3jjktgVBU8HhMbN1xBiLuRS7/8Cm/uvcQkj8YmqywN3Yi6fRVWyxDKjUnUKH6OhPrInzwFV5i15TpWReEPw1IY9uXLOFYsRHWqLJdHYY1NYnbkrJN+xv+Hk+2mAkKkKkbUHOQPbjeBiJ8Arbub1KbPEeJ4hbz4mVQrZcgHdzBw9y6WTYrjgYFziPvFo6d5p4XuJKZKC4IgfI81T6W+BrhKVVlfO4ylW2NRjCu41vuvlonTAWQeT/9x28QFgtuqwrvcyRg2IhNc/0eSwGh0Y7FWYZfiqY6JxrLrI0rjRvDKyFLWW2OYKz3Ih44AVU1TsiVVIW3FPgZoK/BEh7F6zDRsGg337yjgdxUHiUlLpl4TICaigedDpzJ5r4tQjanNmBcABZUqycuBtOcw6W0ANEb78O57AQ3tDSaWUVUFt/4Ghm38E64D1ey1DuCjGSH8vztfJi4zp3tvvnBWiORFEAThHKaRJKZERTBx+pVMzbPwrTeK3+newizVs94yolVXURuSTC3R7FWzyGZXq5f0+gYkRUFFwuixUVX2FtuHbcNgl1gUG45HnQf1OaS7DjKt5jvCAq7giUdg8salSMnhXGtZQ5zkQNFK7DKnsUmbxX1r3mKbmsSUuCtRVfWEVhQVSYLXE99GPa5+S2ZZFlpjJB2RJBkpJAq3JoLt0Sa2DK8gEDOWIUOyT/e2Cr1MJC+CIAjfAxpZ4qm52dz/XiOLG3J5I/E5tqckdelcGxFt9oUfdTJx0ZeYGhoASN6azz8KNCy6wIMtrYyqyDcptc/josp9rc5TAXe4hSGBSqyqGySQURnuKiTC6+Bz12BKcLG28lMmxyYRIntQiMCjDMMdcLGlfhkXnr8BZ72W7Q3BR1hCY3iXlhmuM0WyYOwhUrXTefDiO5FlsTZxfyWSF0EQhO+JOTkJvHrzaJ78PJ9/7r+PC1gI1pOfZ+VYCX1VhYDbwOjPdyGprbtojC4/V34lkzjFhSW5Ebv+XVanJ1OBGb9bS50rEU9sCpm6Yq5kU5v3Cdi8eCQtg8OquTAujzDdsQJ1Lr+J/OpBlNRHsXT5fUwf/R1O/T4OejWUhTjAc/LP0RiVzeyGHEJT48nJEd1F/ZlIOwVBEL5H5uQksO5Xl3D9HXHotplJaKhEUpX2D1YVItVqMtkT3Gya3hH+Hw2SKrUpnRKswyJxMC+GbcYodkw2EzLTzqCZpQyed5gRV24nKr6UOXzTdHxrhxqyGRxWzbykPYRqva1eM2nczE3azeCwagKNMq9uv4soX7Dl6H8pB/A31tLR/BNVVVAaatnk3kvq8Cyuvuemrt4uoY/qseTl2WefZfLkyZhMJqxWa5fOUVWVJ598koSEBEJCQpg5cyYFBQU9FaIgCML3kkaWuGTcpcy74WZ+vOofgNQ2gWkqGHcr/2gZrOvxmChZNwprnr/dpR4hmJAYfQHsFhmPofUjRm9wk5m9Gk+Ut835hY0T2d5wLRfGFQavc8IBzdvT4w7SqAkBIL9kOgABDXwS8nFT2K0TGLXpc1QZq3jo9b9z3g2XdHJnhP6ix5IXr9fLNddcw/3339/lc373u9/x0ksv8dprr7FhwwbMZjOzZ8+msbGxp8IUBEH43rLMmsPVM+7kd1+/TLynutVroQ0qP1hbT9zKaZSsu5vC7+5j75JfYtjWte4WjUNqk4E0b+5PN3N8iqGoMqsddzHAZCdM522TuBx/frjOQ5LJDkjY7MPRqMHS/e9kbucj+e8EPHWtzgl46miMrWXMC3ej1YqREueKHvubfPrppwF4++23u3S8qqq8+OKLPP7441x++eUAvPvuu8TFxfH5559z/fXX91SogiAI31vhs2Zx84wZJH/5Ght3/plkUwhJ3lrmW35N5lEFF0NbWkmMgF/ueImB4ymW9rtwJAk8Rg02i44Iuw+AMm8WLiWaRP2eLl07VnI0LYwtE+JNwmk4jKoGE5j3Atu5pHgwCQ3hlBkdDM2czG23/7xL1xX6jz4z5qWoqIjy8nJmzpzZss9isTBhwgTWrVvX4XkejweHw9HqRxAEQeg6SaPhgssfIPfan/KPGDv/CCkhRf03n0w2Ux/SuhnkaHwGrhArHVU3VQF/hIp3cOf1Txv1x67rUiJa/XkylceNMrb7mxdNDF4voIGFaQf4Z/oeGv3TmDTsli5dU+hf+kwbWnlTeei4uLhW++Pi4lpea8/8+fNbWnkEQRCE0zczZSbTk6eTX5nPysMrMW75hI8vugiDzcK0vW4MHok4u8LBtGvI2fVGS7G7Zs3piuMH/pN+NTZ41ZbzzXKwq6fMm0V9IIpQuabdriNFhXKiyFMyARWLauOCQhs2fzb6+ji8YRWYZQcGn5UAt5LuN2AON5zxfRH6nlNqeXnssceQJKnTn7179/ZUrO365S9/id1ub/k5cuTIWX1/QRCEc4lG1jAufhy/GP8LLp0zmxsLn+OZ0ifINX5Fgj2AjEJVzCh2ZN9Fo07X6txGnYb8QXE4MjR0tPCMqkJjo4ml9lk4CAUgQb8Hs1yNisR3jrtajjvxPICnfbegIIEKTzreR0oroXRgIbZBO8mtn0565RD0nmkM9RsIjTCQkGHtztsj9BGn1PLy05/+lNtvv73TY9LS0k4rkPj44GJcFRUVJCQktOyvqKhg1KhRHZ5nMBgwGERmLQiC0N1mpsxk+gPT2Vyah/nfr5NS8xp7Im7EI1uojh1NhcVMWOUnGHx+PDottWYjSBKNa+MYdFEJJzbNNCcgBwvHUcNA9jKYQYESRtr2ES6txclcCj0TWWT7BVPD3yJMU9NybpUaxRP+W1isjCemoY4f+//N6uH7WW4ygdnDsLIMbI2raDBPY6AmE4Cp12Ygi8UWz0mnlLzExMQQExPTI4GkpqYSHx/P8uXLW5IVh8PBhg0bTmnGkiAIgtB9NLKG8QMmwc8n8ftVH2F4800i6wdQkjWBEPtgvJljsFXvQmk8liR4yo0EjujRJns4PnupVcPI33MznoMpGCUv0T4Dsq0EdtaR1bCYAUMPszvpeg56JlFUNZ4E/R4iNcWk6ddQ4NIyTN1GVOg6yoYc4g9mPYpkwuyxEFs8jZ3eVIZFpjDEr8McYWDatRmk53ay9IHQr/XYmJfDhw9TW1vL4cOHCQQCbN26FYDBgwcTGhpsKszMzGT+/PlceeWVSJLEww8/zDPPPENGRgapqak88cQTJCYmcsUVV/RUmIIgCEIX/fz86/g9kPGT+fjionkvLpkrK+egDZmJIToPpFqOGhpYm72ZPD3UVxlQgVBdKHW6WQzaNouskgAAeqmeioRlNMTns3+CDovNR2xBJTMOPoWSHYVHZ8Es15Gg30ODoiMlxEqMz4VedbO3KpRcu4XtunDW6MNwJO5mgH8/h/1hzBh3M5dNnSxaXM5xPZa8PPnkk7zzzjst27m5uQCsXLmSCy64AIB9+/Zht9tbjvnFL36By+Xi3nvvxWazMXXqVBYtWoTRaOypMAVBEIRT8PPzr6P2WQumnzzCX+fm8oXJzIUNOmTXZADinBC9+ipKLAepHuCmQW9EdRsYWlyINvQ7fNE6CiL38V1EHooc7FeSUZiWECB+WBUxTg1D1iQSU1NKVaTMItNoQg6EcdmvHiU83g/OCpyuo7xyeCFlDQdbHmKqKZ4Hxt/PzJSZHcYunDsktaN6yv2Uw+HAYrFgt9sJDw/v7XAEQRDOSY4lS/jPKx/w28wfIKkwIKDBrEpY9RpG28CMpuVYv76WDQMXsiMmv2WfkRCuDmvAamzEEZAI16gMUI1489PIen//sTeSJLRxcQxevgxJc+yaASVAfmU+Ve4qYkwxjI4djUY+9rrQ/5zK81skL4IgCMJpUQMBPvj3O/xpt5EaxdKyP1rr4pbIPQyM2IteVkitnojblsYzg/ewJcqCIocTaojhReVBHJXpaPxx6HZVMeh/RWj9xz2SmuZLJ/3lRcJnzTrbH084y07l+d1n6rwIgiAI/Yuk0XDTzXdyXcDPyp3rKKuzEW8NJ3eAlY01Pr7cYcJbn0BloolNowagyCmgKoDEbfwesymW3IseJjZ2No4lS6jIm4//uLpe2rg44n71S5G4CG2IlhdBEAShR3xVZePxghLKPL6WffE6lZ/HVnNJTBhW6zgk6VhXjxoI4N60GX9VFdqYGExjx7TqKhLObaLlRRAEQeh1l8ZYmRNtYb3NSaXXT6xey0RrKJoOVl6UNBrME8af5SiF/kgkL4IgCEKP0UgSUyLCejsM4RzTZxZmFARBEARB6AqRvAiCIAiC0K+I5EUQBEEQhH5FJC+CIAiCIPQrInkRBEEQBKFfEcmLIAiCIAj9ikheBEEQBEHoV0TyIgiCIAhCvyKSF0EQBEEQ+pVzrsJu81JNDoejlyMRBEEQBKGrmp/bXVly8ZxLXurr6wFITk7u5UgEQRAEQThV9fX1WCyWTo8551aVVhSF0tJSwsLCkDpY/Ot0OBwOkpOTOXLkiFituh3i/nRO3J/OifvTOXF/OifuT+f6y/1RVZX6+noSExOR5c5HtZxzLS+yLDNgwIAeu354eHif/svvbeL+dE7cn86J+9M5cX86J+5P5/rD/TlZi0szMWBXEARBEIR+RSQvgiAIgiD0KyJ56SKDwcBTTz2FwWDo7VD6JHF/OifuT+fE/emcuD+dE/enc+fi/TnnBuwKgiAIgnBuEy0vgiAIgiD0KyJ5EQRBEAShXxHJiyAIgiAI/YpIXgRBEARB6FdE8tIFr7zyCoMGDcJoNDJhwgTy8vJ6O6Q+49tvv2Xu3LkkJiYiSRKff/55b4fUZ8yfP59x48YRFhZGbGwsV1xxBfv27evtsPqMV199lREjRrQUzpo0aRJff/11b4fVZz3//PNIksTDDz/c26H0Cb/+9a+RJKnVT2ZmZm+H1aeUlJRw8803ExUVRUhICMOHD2fTpk29HVa3EMnLSXz00Uc88sgjPPXUU+Tn5zNy5Ehmz55NZWVlb4fWJ7hcLkaOHMkrr7zS26H0OatWreKBBx5g/fr1LF26FJ/Px6xZs3C5XL0dWp8wYMAAnn/+eTZv3symTZu48MILufzyy9m1a1dvh9bnbNy4kb///e+MGDGit0PpU4YNG0ZZWVnLz3fffdfbIfUZdXV1TJkyBZ1Ox9dff83u3bv54x//SERERG+H1j1UoVPjx49XH3jggZbtQCCgJiYmqvPnz+/FqPomQP3ss896O4w+q7KyUgXUVatW9XYofVZERIT65ptv9nYYfUp9fb2akZGhLl26VD3//PPVhx56qLdD6hOeeuopdeTIkb0dRp/16KOPqlOnTu3tMHqMaHnphNfrZfPmzcycObNlnyzLzJw5k3Xr1vViZEJ/ZLfbAYiMjOzlSPqeQCDAhx9+iMvlYtKkSb0dTp/ywAMPcOmll7b6d0gIKigoIDExkbS0NG666SYOHz7c2yH1GQsWLGDs2LFcc801xMbGkpubyxtvvNHbYXUbkbx0orq6mkAgQFxcXKv9cXFxlJeX91JUQn+kKAoPP/wwU6ZMIScnp7fD6TN27NhBaGgoBoOB++67j88++4zs7OzeDqvP+PDDD8nPz2f+/Pm9HUqfM2HCBN5++20WLVrEq6++SlFREdOmTaO+vr63Q+sTDh48yKuvvkpGRgaLFy/m/vvv58c//jHvvPNOb4fWLc65VaUFoS964IEH2Llzp+iTP8HQoUPZunUrdrudTz75hNtuu41Vq1aJBAY4cuQIDz30EEuXLsVoNPZ2OH3OxRdf3PL/I0aMYMKECaSkpPDxxx9z11139WJkfYOiKIwdO5bnnnsOgNzcXHbu3Mlrr73Gbbfd1svRnTnR8tKJ6OhoNBoNFRUVrfZXVFQQHx/fS1EJ/c2DDz7Il19+ycqVKxkwYEBvh9On6PV6Bg8ezJgxY5g/fz4jR47kL3/5S2+H1Sds3ryZyspKRo8ejVarRavVsmrVKl566SW0Wi2BQKC3Q+xTrFYrQ4YM4cCBA70dSp+QkJDQ5ktAVlbWOdO1JpKXTuj1esaMGcPy5ctb9imKwvLly0W/vHBSqqry4IMP8tlnn7FixQpSU1N7O6Q+T1EUPB5Pb4fRJ8yYMYMdO3awdevWlp+xY8dy0003sXXrVjQaTW+H2Kc4nU4KCwtJSEjo7VD6hClTprQpzbB//35SUlJ6KaLuJbqNTuKRRx7htttuY+zYsYwfP54XX3wRl8vFHXfc0duh9QlOp7PVN52ioiK2bt1KZGQkAwcO7MXIet8DDzzABx98wBdffEFYWFjLOCmLxUJISEgvR9f7fvnLX3LxxRczcOBA6uvr+eCDD/jmm29YvHhxb4fWJ4SFhbUZH2U2m4mKihLjpoCf/exnzJ07l5SUFEpLS3nqqafQaDTccMMNvR1an/CTn/yEyZMn89xzz3HttdeSl5fH66+/zuuvv97boXWP3p7u1B+8/PLL6sCBA1W9Xq+OHz9eXb9+fW+H1GesXLlSBdr83Hbbbb0dWq9r774A6j//+c/eDq1PuPPOO9WUlBRVr9erMTEx6owZM9QlS5b0dlh9mpgqfcx1112nJiQkqHq9Xk1KSlKvu+469cCBA70dVp+ycOFCNScnRzUYDGpmZqb6+uuv93ZI3UZSVVXtpbxJEARBEAThlIkxL4IgCIIg9CsieREEQRAEoV8RyYsgCIIgCP2KSF4EQRAEQehXRPIiCIIgCEK/IpIXQRAEQRD6FZG8CIIgCILQr4jkRRAEQRCEfkUkL4IgCIIg9CsieREEQRAEoV8RyYsgCIIgCP2KSF4EQRAEQehX/j/cAQ+VB3HUvAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# declare the model\n",
    "model = SineModel().to(device)\n",
    "\n",
    "# define the criterion\n",
    "criterion = nn.MSELoss()\n",
    "\n",
    "# select the optimizer and pass to it the parameters of the model it will optimize\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr = 0.01)\n",
    "\n",
    "epochs = 1000\n",
    "\n",
    "# training loop\n",
    "for epoch in range(epochs):\n",
    "    for i, (x_i, y_i) in enumerate(sine_loader):\n",
    "\n",
    "        y_hat_i = model(x_i)            # forward pass\n",
    "\n",
    "        loss = criterion(y_hat_i, y_i)  # compute the loss and perform the backward pass\n",
    "\n",
    "        optimizer.zero_grad()           # cleans the gradients\n",
    "        loss.backward()                 # computes the gradients\n",
    "        optimizer.step()                # update the parameters\n",
    "\n",
    "    if epoch % 20:\n",
    "        plt.scatter(x_i.data.cpu().numpy(), y_hat_i.data.cpu().numpy())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.0027208030078327283\n"
     ]
    }
   ],
   "source": [
    "# testing\n",
    "with torch.no_grad():\n",
    "    model.eval()\n",
    "    total_loss = 0.\n",
    "    for k, (x_k, y_k) in enumerate(sine_loader_test):\n",
    "        y_hat_k = model(x_k)\n",
    "        loss_test = criterion(y_hat_k, y_k)\n",
    "        total_loss += float(loss_test)\n",
    "\n",
    "print(total_loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reproducibility"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "def enforce_reproducibility(seed=42):\n",
    "    # Sets seed manually for both CPU and CUDA\n",
    "    torch.manual_seed(seed)\n",
    "    # For atomic operations there is currently\n",
    "    # no simple way to enforce determinism, as\n",
    "    # the order of parallel operations is not known.\n",
    "    #\n",
    "    # CUDNN\n",
    "    torch.backends.cudnn.deterministic = True\n",
    "    torch.backends.cudnn.benchmark = False\n",
    "    # System based\n",
    "    np.random.seed(seed)\n",
    "\n",
    "enforce_reproducibility()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function `utils.fix_random_seeds()` extends the above to the random seeds for NumPy and the Python `random` library."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lapan, Maxim (2018) *Deep Reinforcement Learning Hands-On*. Birmingham: Packt Publishing\n",
    "\n",
    "Rao, Delip and Brian McMahan (2019) *Natural Language Processing with PyTorch*. Sebastopol, CA: O'Reilly Media"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}