notebooks/Aula_18_b_Redes_Neurais_com_PyTorch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/storopoli/ciencia-de-dados/main?filepath=notebooks%2FAula_18_b__Redes_Neurais_com_PyTorch.ipynb)\n",
    "<br>\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/storopoli/ciencia-de-dados/blob/main/notebooks/Aula_18_b_Redes_Neurais_com_PyTorch.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Redes Neurais com PyTorch\n",
    "\n",
    "**Objetivos**: Aprender Redes Neurais Artificiais (RNA) usando a biblioteca `PyTorch`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## PyTorch\n",
    "\n",
    "[**PyTorch**](https://www.pytorch.org/) é uma biblioteca de código aberto para aprendizado de máquina aplicável a uma ampla variedade de tarefas. Foi criada pelo **Facebook** em 2016 é a principal biblioteca para criação e treinamento de redes neurais artificiais. A API toda é escrita em Python mas é executada em C++ na CPU ou em CUDA/ROCM na GPU.\n",
    "\n",
    "No momento que eu escrevo esse tutorial (Abril de 2021), PyTorch está superando o TensorFlow (Google) em desempenho e adoção de uso. Isso acontece tanto na [academia](http://horace.io/pytorch-vs-tensorflow/) (mensurado pela adoção de artigos científicos nos principais encontros científicos de Aprendizagem Profunda e Aprendizagem de Máquina) quanto na [indústria](https://www.infoworld.com/article/3597904/why-enterprises-are-turning-from-tensorflow-to-pytorch.html) (mensurado pela adoção de grandes e renomadas empresas de tecnologia).\n",
    "\n",
    "### Atualização (Junho 2022):\n",
    "\n",
    "TensorFlow está morto, Vida Longa ao PyTorch: [Google lost the battle for machine learning to Meta, insiders say. Now it's betting the future of its own products on a new internal AI project.](https://www.businessinsider.com/facebook-pytorch-beat-google-tensorflow-jax-meta-ai-2022-6).\n",
    "\n",
    "> Now, under the shadow of PyTorch, Google has been quietly building out a machine learning framework, called JAX (at one point an acronym for \"Just After eXecution,\" but officially no longer stands for anything), that many see as the successor to TensorFlow.\n",
    "\n",
    "Meus comentários: JAX é mais um backend de _autodiff_ do que uma biblioteca de redes neurais. \n",
    "Ou seja tem um uso muito mais amplo que PyTorch.\n",
    "Por exemplo, como um amostrador monte de correntes Markov"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/jpeg": "/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAUDBAUFBQUFBQgFBQUGBwUIBwcHBw8FBQUFBQUGCAUFBQUIChwXBwgOCQUIDiEODhEdEx8fCAsiJBYeJBweHxIBBQUFBwYHDwgIDxUVEhUeFRkdHRcWFxUVFxcWFRUVFRUVFR4VHhUVFRUVFRUVHh4VHh4eHh4VFR4VHhUeHh4VHv/AABEIAWgB4AMBIgACEQEDEQH/xAAcAAEBAQEAAwEBAAAAAAAAAAAABgcFAgMEAQj/xABJEAABBAECAgcFBAYFCgcAAAAAAQIDBAUGERJSBxMUFiGS0yIxMkFRFUJhcSMzYnKBkQgkgqKxNDdDRFNzobK18BclY3bC0eH/xAAZAQEBAAMBAAAAAAAAAAAAAAAABQIDBAb/xAAqEQEAAgIBAwIFBAMAAAAAAAAAAQIDBBEFITFBUQYSEyLwFGHR4TKBwf/aAAwDAQACEQMRAD8A/jIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFF3Qvc9Tzu9Md0L3PU87vTAnQUXdC9z1PO70x3Qvc9Tzu9MCdBRd0L3PU87vTHdC9z1PO70wJ0FF3Qvc9Tzu9Md0L3PU87vTAnQUXdC9z1PO70x3Qvc9Tzu9MCdBRd0L3PU87vTHdC9z1PO70wJ0FF3Qvc9Tzu9Md0L3PU87vTAnQUXdC9z1PO70x3Qvc9Tzu9MCdBRd0L3PU87vTHdC9z1PO70wJ0FF3Qvc9Tzu9Md0L3PU87vTAnQUXdC9z1PO70x3Qvc9Tzu9MCdBRd0L3PU87vTHdC9z1PO70wJ0FF3Qvc9Tzu9Md0L3PU87vTAnQUXdC9z1PO70x3Qvc9Tzu9MCdBRd0L3PU87vTHdC9z1PO70wJ0FF3Qvc9Tzu9Md0L3PU87vTAnQUXdC9z1PO70x3Qvc9Tzu9MCdBRd0L3PU87vTHdC9z1PO70wJ0FF3Qvc9Tzu9Md0L3PU87vTAnQUXdC9z1PO70x3Qvc9Tzu9MCdBRd0L3PU87vTHdC9z1PO70wJ0FF3Qvc9Tzu9Md0L3PU87vTAnQUXdC9z1PO70x3Qvc9Tzu9MCdBRd0L3PU87vTHdC9z1PO70wJ0FF3Qvc9Tzu9Md0L3PU87vTAnQUXdC9z1PO70x3Qvc9Tzu9MCdBRd0L3PU87vTHdC9z1PO70wJ0FF3Qvc9Tzu9Md0L3PU87vTAnQUXdC9z1PO70x3Qvc9Tzu9MCdBRd0L3PU87vTHdC9z1PO70wLgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAPOGJ8r2RRNc+V6ta1rU3c9zvha1v1EMT5Xsiia58r1a1rWpu57nfC1rTU9FaXZjGNsWNn3np4r8Ta7XfFFF+19Xf9ry7W3TBXmfKh0/p+TcycR49Z/PVK5zTLcVhEsWOF96SaFHbLxMhY5siuiZ9V9lN1/Aky96Tc5XlY3GQ/pXska+V6L7ETmNVGwt+rvHx+n+EvpzA2ctK9lfgYyJGq+SRfYZxb8DfBF3VeFf5GrUzW+lOTL2/hv6hrY/1MYdeOeI47e7lA6GexFnFWOy2uDiVrXMexeJkrHOVOJiqifNqpsv0OedtLxePmr4S8mO2O00vHEwAAyYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHlDG+V7Iomq971a1rWpxK5zvBrWtPE0DotwzeF+VmTicqvjr7/AHWt8Jpm/iq7t/g76mjZzxgxzeXXo6ltrNGOPyHX0XpdmMY2xYRH3np4r8SV0d/oovx+rv8AtefrrVnZ+PH4939Y9ps0zV/U80UTv9p+Py/P3e3pB1KtJv2fUXhtSN3lkT4q8TvhYz6SKnz+Sfmm2ZkzU1rbNvrZf9R+ei91Depp4/0ut248z+eoqnd0jqN+FfN+jbYhmRnGzi4Htczfgex+y86+BwgV8mOuSvyWjs83iz3xXjJSeJh1tU5uXMW22HMbExjWsjjReNzW8Sr7TtvFyq7/AAPjyOOt0nRNtRvhdKxr2cX3mO/wVPmnvQ+zSV2tSyVea2xssKLtuv8Aq73bcNhrfnw/j9d/eiGoaqxLMtRfD7PWonHA/wCkrW7t9rlcngv5/gcGbZjVvXHFftV9bSnqGO+abc39mMg/XNc1XNcjkVFciovxNc3wVrj8KSHMcdgr+iHRLdZ512Hdb+yWNp3bTrK1u3bMqOi4mdnSVu+6S777/d9y7kga9/RJZx61mZujOPC5tvE5dmM4uypxPX5NQD01OiPD5pk0WjNT4rUmVihfM3HSUJMJbtxRN3elR08q9Y/bbw2RPFN1RPEydfic1fZe1dnNXwex33mvRfcv4G49HeE0r0eZOvqnJ6l0/nrGNhtLTxuCl+0rF63YqSQMbLYav6GPhnf8SIm6oquREVF5NqdusOjTIWXMi+29IZSW3J1bU62bT2obEj5W8SInE2OdzvybVb9QM+0VjMTk8j2TM5NmnKXUyvW46o7JtbKxzOqr9kiei+1xO9rfw4fxNNh6INHy4exqBmsq64evaZTls93peCK9KyN7K7oltbqqtmYu6Jt7XvOZ01tZp/DaP0PE2JluhTbksu5Gp1rszmOORlaWX/0o3yJt9JY/oh54P/MpqH/3ZR/6bjgIjXeHwmKu16+DyzNTVpIWvfZZRfiupsOlkb2Ts8z1V68LGO4kX7+3yJ/mb80XZU+813K76KbJp/KS6G6PMPqPCR1Wak1JkMnCuSlgbbmxWOxb5IuzUUlaqMe90SOVVTx4nbouybfmqsq/XXRvldS5mOs7Umncjj665GGFtSXK47J9VG2vdZE1Ec9j7CuTZE24G7Im67hjXE36/Jy/2W/E78kP3f4W/eX3Jzfu/U/pDpm6TsxpG9p6jp2PHUZpdPYGe5cfVZas3oVSwytjnrI39HXZ1Ujtm+KrO7xT5zs+ds9Hmi9JZDT7KkWo9WNyd69l5KzLdqCtXmhSHH0mzNVI27W27psqbscu267oEH0I6ex+pdY4TCZNJX0Lcl1szYpOpl/q+Ntzsa2VPFvt127/AD239xLZWNkN27Cz2YobNiJu68XC2KZ7WMc7m2abH0Oayy2q+lHR9zNLUluwMyUKzQVm1H2mpicm9JbaRIiPk9vbdERNmp4e/ei6IekLK6j1jLorJwYZdLX/ALdruxkFBletCyvDZkZKx6IrnSKtfxc9y7q5V8F2A/nIsujjo7yeqW27bJaeJwVD/Lstff1OOqeyi9U1y/rptnIvCmyJxN3VN03jXpw7t+LhVyePxO4eb8T+iNfZDR+nNK6B09mKeYzML8PTya0qlz7Hxk9vIpx2Mnesx+1Yn61ZeFieyiOdv702DD9Z47GY3K2KmHvsz2PiSHgvNrupMsSuhYthqV5FXZqScaIqKqKiIu6nNjq2HwvsMisPrs4uOZsTnV2cPxccyJs3+Kmyf+GunMhqHo7t4Z11mltXy2HOp238dyjLh/ayOOdYRd3sdwLGjt1VFR68S7pt6dXdOOrsbqC9Xw0tTFYTF2rVSriGU4vs/sWPsSQtistWPiVXpEu/C5NuLw22AxsoejbTLtUajxOn2z9hdkZJmdo6rtPU9TSsTq/s/G3j8K6ptxJ7/wADSOlTQGPyWsK/2TY0/pWtmMFjs2sWUtfZmPrWLsqx2KdV7Y12eqtR/BsifrNtkREPb0L6Pdp/pO0jCuR09netTLTdZh7v2lDX6rEXWcFl6xpwPXj3RPpuBnHSjo6bR+oLeCnlS6yJlWWC22PqWXqlqFHxWGQ8a8KcXGz4l8Y3eJQ4LopsXtA5HXMtxKkVVbTq9Fa3WvyFapNFDNZbb65Opakr5W7cC/qvf4+HZ1dTsa10NpLMUm9ozeIvd2raJ8UsVqZnd6WVdt+FOujZv9bD/oXubuQJR6SdJY93HjNJaRxmOiX7s16u+eXLW/33ScLF/GBQMDy7NLd3sM/HS5F+qFmtplopW7Y+GvxydkdWfw+Llb1fucvvfvt4ITiOb9W+7f8Ast+9+X4msa1a3/wq6NvBPayGqN/D4v8AzKRPH6+CJ/I0Dph6Uc7pnpGmxmGZialJZMGltEosfYyzLFSrxsvW3pvs1j+BEYqbI1Pf4gfzOrm/Vv8AMK5qfEvDv4eJ/SuGwmPo9MOvKVaGGKpHg8tLHAjEWGKW3j8TPM6KLb2U6yd6oie7iVEM8/oyxMlyGq2ytY9rdHajciOaj+F6S47Z7d/c72l8fxAhYNOXpcDb1K1IfsmpdhozPWVEmbdmijkYxIfm3hlb4/icdXN4eLdNvrv7Jv2k+kPO4nohbdqLjmzU9QVMZW4qMb4uwswsMq9dErdpp1dxKsrvaX5qOi2F7NMZ7X3atL4zVOUzctSrkcynUY3Eo+JLNx+PhZC5GW3rLIiIrdkRifJFRwYFzN+8nvT7zf3m/IoujPS/erUeK0/1/wBn/aD7DO0dT2vqez0rFji7Pxt49+z8PxJ8W/y2NL6X8tWyujqzs7l9Kaj1nTyjW17eGejrFjAzVV62DJbV2fDLuvg3b2Yvmq7zH9GX/ORpX/f5D/omQA7Ffon0pevOw+J1jjbOadLLBFVs4abGRTXI3qzsyXXzqiPVzFamyKq/JFOPojQcMzukSrno5mZDTGFzFiKOOXgYzJ4/rGtc9W/rot2IqfJUVC30y3okq6x7T2rVb8xHlpn1kycMbsFBnftF61JZuxMR6wss8CpxORNmoqr71PzTtXMVct04w550EuY7uZ19p8G6VZX2IlmifVa74YVilZsi+KJsi+KKBkt1ulW6Voy15ck7V3bZu2wvbw4xmK4Z+zyxP4fj9mH7yrusu6bIh8mrNOXtPy4+HJpCyXIUKeQrpHKk3FRvdZ2d79vhevVO3b8tjRbN+XGdEmkcjVSHtNXVmRlj62JtiF0sNK8rOuhkRUlb7KeDk+RSf0lukLOxWKunWrR7DkdOYd9vemztPX3m2O1dTZ23hbsxmyN228dgP5861nMzzIeSLxe032vyP6Q6MekfUb8db1Vqp+Kr6Sxn6FjIsXFFe1BleDavicY9W+Oypu6RPBNlTw2crMG1jnZs9mMlmbEdetLenWVYYE2r128KNihi+qNYxqb/ADVFXw3A5IAAAAAAAAAAAAAAAAAAAAD8U3DDQMpY2rF8LYIGcX7zWcUrv4rxKYepuk6ddSe1n+krvRv9uH2f+Yj9XntSP3em+HIjnJb14Ynkbb7dqxaf8c0j3r+zu72W/kibJ/A9AQFWkRFYiHnctptebT5AAZtYa30eXXWcPDxrxPrufCq/sxbLF/cexP4GSGndFUbm4qZ7vhfZlVv7rYomu/4tX+RM6rWJw8z7rvw9e0bXEeJiUZrmq2vmrrGpwte9siJ/vmI9395zjiFJ0lPa7NzcP3Y67V/e6ri/+aE2derMzhrM+yd1GsV2bxHvIWPQ/rOLSObly01eW8yShkKnVRyJC5rrvVcMvG5F8E6r3fiRx+Kv/A6HG/GJs1G/RGp5TXf6Kc8sGq700vVOwcOHyz851ycddmJZEj2caL4cfXsj2Rfl1u3zM41Jp/JYSWpBk4uzy26le3E3jR/FVtOkazjcxV4JEdDI1WL7SK1UVEKTM9KuqMhgU01LNUr4zqasEqVajalq3UpMRtWrbsRp7caIxqbIibp4LuiqihPa0z9jUObymcs7tmyNmafhVeJYYnu2r1/yZC1jP7B2KGsYodDZHR7oJXTXMxXyKWkkRsUTIataLs7oNt1cvZVXffb2vwPnXQ+SZwxW58FjLrkYraF7KRUcn7bUWJk1d7/6q9UcmzZ3Nd4+5Dg5Shbx9qaleilqW67uCWGVvBLE/hRW8Tfmio5FRU8FRUVN0VFAudEa7xMOBl0nq2hYzOBS065TkqTpUyuHvPRUmdUe5UR8b+Nyq1VTxe/fffZPXrnXGKsYKPSWlKE2G0+ll1y063P2vK5jIcHBFNdlb4MjYjW7Maqp7DPdtssCe+9UfVexj3V3ufFXlRYZm2mNZahZKxj3xqvDKiPRFYviioqKiKgFR0r6xi1XkcbehglpNp4jGY9WSSJK6V+PdYV1hrmomzXdoTw/ZO5pbpAwM2nKmlta427mMfjpppsZcoWEq5XHdpcrrFb9IqJJCrnqvi7b4U4V2RUgtPY1+TyeNxjHJE+/co1GSOTiZE+9ZjhZK9qe9qLKirt9D5LEfVSyxb8XVve1V5uB6t4v7oGjYPW+l9P6xwWoNPYrJVMbi47TZa9i72i9krFqrbi7S+V26QKiW2pwtVU2j+W5yejHWkWmdYV9US15bcUUmTetdkiRSu+0K9mNretcn3e0Ivu8eEntOYS1mLEteo6rE6CvYsyyWbDaVaGpV4OumlsSqiNRONvv+p9GY0vksfV7cvYruP6xsS26F2LLVIrD0VzIbEtWRezvVGO2SRE34V232A4zl4nOd9Vcvmcadh9f6ayWCxWC1xjMhk3YRj4sbkMbZSpkGUX7cOPtskciPjajGIjt18Gt8EVFV2YAC/1p0lTXshpx+n67dP4rS6NTDVUkW3LDL10ck1u7M79dJIsDN0Xfw4t1VVVVochr7o3y1xc9m9NZJ+dlc2W1XqZBI8DkLqeL57ETnIrGvc3dWoxd9134t1VcfPomqPZXr2nOrqyw6w1rGzNfYY6r1XG6xVau8LV65Nlcib8Ltt9lA7PSLq69q3NWszkEiifKkUcMEX+T0aVdOGvUh397Wpuqr81c5dk32T7OiDV0OkdTUdQTQS3mVG3U6iN6QvetmlLCio9yLtt12/8AAkgBonQr0oS6GsZV7qjMtVyEdVezyTdnZDkKMyyUbvEsbt1ar3+CIi/Cu6bHwaL119lVNbRXopshZ1TjrVZ87XpF1N20+w+W5MxU9tFfaVdkIoAV+d1jFe0jpbTLYJYpcHZy0z7CyI6K03I23ytYyJE9hWo/bxVfce3pR1rFqjV02poq8tSKR2Md2d8iSyt+z4YY3N61qJ8XUqvu+ZFgDVo+l1kXSNe1syk99HIxdmtY+SZHSvovx9avM1lhG7cfFUY9N02Xbbw33T7MD0i6B047MN05gsxX+0sTkqPard9LN2F13qkirxV1kVsdVOBXK/dXqrI0295joA0TQOtdPVtM29JaroZPI45+RiyVeXG2G1rcN1lRld8UrZVRFjVjF8d1+N3h7lTy0fr/AA+Miz2n8hjJsronLW3WY6LrPBlcVKx21S1UuJ8UyRsiau6pv1aLxe9HZyALDXWS0TNUr19KYzMY+w2frJ7mSupamlrthkb2KKpG5WtbxPa7j3Rf0e3jv4fL0Wani0rqbE6glhfdioPtPdAx6RPl66jZga1srkXbZbCL/ZJkAa23XvRxVvLmKelLk+VSw61GtzNySUkuumWVsz6+yo5EkdxcPDt4IcfGdJ1jtevcllYXW72rcXeoq6FyQw0X2ourhe1jt1dFGxrGo3ffZieKrupngArcnq2C1ofFaR6iVktHK3b7rSyI6KVlmvYj6lsO3g5O0Iu+/wB0o9c660jqjHQ2sljMyzV0OKqUGWYbjW4XraTXpXuvrq7dfjcvBw/e23XZFMvAG16o6T+j/O1sVSvaczKUcRB1FGrXzK0qlVjtuN7a8O3HK7hTeR27l29/iu+Sajnx02QtzYevNj8Y97Vr1ppltzV4uBiOY+y79YvE1y7rzInyOeAAAAAAAAAAAAAAAAAAAAAAAa/oPINu4mr48UsCdTIn3uKFqIxzvzbwr/EyA7mjM67E2+J/E6pNwtmantcKN+CZjeZvEv5oqnD1DXnNi4jzCr0jdjVz/d4ntP8AL91xiH43Iy8Kf1ewrpIXfd4Xu3fF+bXO930VpwjbMlRpZqk1kvDNXkRr45WL7THcPsSxP+S//qKZtnNHZOkrnRMddr/J8Sbva39uH3ov5boadLerav08k8Wh19U6TkpecuGOaz37en9JwHm+KVjuF7Xsd9HNVrv5Kh0MZgMnec1teCXhX7729TCn7XWu23/hupQtlpWOZlEpgy3n5a1mZfBWglsSxQwtc+WRzWsanxOc74TacLSixWPhruVrWQRuWR/wtc/xfM/8t3OU5mkdKw4pvXSq2xdcmyv29iFrviZC1f8AmXxX8DjdI+o28D8VUdxOd4WXtX2Wtb/qqO+q/P8Al81IuzmndyRjx+HqdLWjpeC2fN/lPiP+IrN3e23rVv8A20j3NRfkz3RN/g1rT4wC5WsVrFYeUyXnJebT5kKTo0wbcznqNWVnW1Iett22K5GNfRxrFnsV3PkVEb1vVNhRVVE3naTZ0sflXVMflaLI2Ofk0pRyTq724qVSx2iWoxm3ukmirOVd/dXRNvFTJguNU4fUF7TWTyGbiYy9jspLfRzbUVvjx+pZUZlYYoq8rlbHFdiqvTdEREuSk90Sxo/VWH9lkr4325YGOTdsuQqY23PimOavv3tQwJt81VEORpnK/ZVqWbqmWIbFXIVLMCu6htqpkKskMrFlRF4Var2SIu3gsTF+R8NOxNVmhtV3vhsQSRSxSsXglhmhej4polT3ORzUVPyA8HSvmV00rnzSzKskkj143zSyu4pZZXr8TnOcqqq+9VU0mlVhyFLRsuQYyxYdgdfM/SJxOfSwNHLuwMzvr1UjHsav0qsT3IhxGah01duxXc3h5nzSScdxcZkvsqpdc9281j7MfXd1MjvFVbFKxqqq7I3fw+jNZ9tXN4/PVLGMzFJIZq0NBsD8Y2jieySVZsFbxquXsrVgvTNR8cr0VXPdxqu4HG0zUrz4zVU0rGSy1cZSlgc5OJ1eZ+o8PC+WL6OWOxI38nqUMmKxtWa7k5q0NiritOaNttpLvFVu5bNYzCRMlu9W5FdD1+QlmciKiuVqJuiKpx36kxkOPy2NxWM7EzJwV4ZbFm+uTvRNr5OnbY2KVIWNSLelw8KMRy8SKr12RDwh1bKy92p9avYqS4rGYq5Sle7s+Qx+OxtKs1XSs2WGVXYyKZHt8Wua3bdEXcOjo/OLktUaRY+phaj489gnJLRotxkr2OytZOzysr7NlYngqKreLw+I9WuKNTAusYd0TLeWsrFYtXnt46tWrYd19apgXL4TNc17VdZTdF8Ws8EVX/HXz2Jo5DD5DE42arLjshSuuWzk1yD7XYrEc0VJrmVmJDFvF8XArvH3+Gy/LY1A+1jX427Eyy2OZ8uPmV6stYl1iZZLdWJ+y9dUk4lVYnbIjvaRUVXI8Or0WVu02NRV+sr1+t07nW9bYk7PVi4nVPbmmVPYZ+J4TLSwOC1FU7dj8rkMvBRrshx7n2qlSGpkq91923dkia1029JsbWR8W3XSKqp4IvGwGXXG/aPDG2Xt+Ou0V3dwdUy71W8ybIvEqdV7vD3nKe3ia5v1RyfzA1TUtTG29ZJozGUsVjMfayGMqS2uyslyrH2H1nW7VW69F7O1qPexsbNmqieKKqqp6r0eFlr5aGwuhKuMSlkXY1MdKs2eq3q9eR+Hb9odXxX3ySMZG/rXK1escqI3ZNo3N6ktW89LqKunYbq2obUPVu63stir1awvY9zfa2dC1fFD6MxmsFbbamZiOyZKyj1V8WSd9lVbEruKWxSxPU7xJu5ypG6ZzE32RNkRDIdhcBRyOY07eYxlLBZSk29eSFOCHHswMUjdW14m7+x4498iJ9LsCfND6cPg8bnn6P4oa+Miy+X1i+31G0PVYzHsxlpKUUuy8LY4XTMaqou3Hv4/ObxmqbFTA5PAsjiey9K17bKqvaKUUrqy5OvX/YsfZlJF/Ctt47qeNHVFulDp9lRGQ2MHcyVuCZf0vWy5B1BXRTQqnjGn2eiKm67pK5FAp8q3GWMbmG3XaErNZUdJi24V6vytfIxWIOz1O1dWjr8L4XStc6dVXfhcioqbHtq47CV827Ud6pE/S7MRjcq+jGn6F82VbHj34yLZfDgyPbHIifKmpJ5fJ4GxDN2LEvx9ubgVHJlHWsfSdxo6VuPx6wIrWqjXNRJZX7I75qiKi1qe1NpylpxzIm16tyax16L+mliVsq16T27fqo5LtyRPH32V92yGI6Wb0/FgsZnorbYrFtczFjaMzk9vsmMiktXshX/CSO1jdl+lhfqR53tV6otZuvha9hkUSYqkysjmLxLdlY2OLt1jdPCVa9SrEvv8KzV+ZwQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA7Wm9SXcS7hiVJq6ru6B6+xxfeWJ33Hfl4fVFL7F62xNlqda91KX5tmT2OL9mZu6bfnsZODi2NDFmnmY4n9lTT6tsasfLE8x7S29uXxr28TbNJyfXr2/wD2fHf1Th6reJ9mKVyfdhXtD3fs+xvt/FUMcByx0inPe0qFviTLMfbSIlY6j1zYstdDj2uqQu8FkVf609P2dv1Sfluv4oRwBRw6+PBX5aQibO5m2rfNknkABucwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH733uSp5HeoO997kqeR3qAXAIfvfe5Knkd6g733uSp5HeoBcAh+997kqeR3qDvfe5Knkd6gFwCH733uSp5HeoO997kqeR3qAXAIfvfe5Knkd6g733uSp5HeoBcAh+997kqeR3qDvfe5Knkd6gFwCH733uSp5HeoO997kqeR3qAXAIfvfe5Knkd6g733uSp5HeoBcAh+997kqeR3qDvfe5Knkd6gFwCH733uSp5HeoO997kqeR3qAXAIfvfe5Knkd6g733uSp5HeoBcAh+997kqeR3qDvfe5Knkd6gFwCH733uSp5HeoO997kqeR3qAXAIfvfe5Knkd6g733uSp5HeoBcAh+997kqeR3qDvfe5Knkd6gFwCH733uSp5HeoO997kqeR3qAXAIfvfe5Knkd6g733uSp5HeoBcAh+997kqeR3qDvfe5Knkd6gFwCH733uSp5HeoO997kqeR3qAXAIfvfe5Knkd6g733uSp5HeoBcAh+997kqeR3qDvfe5Knkd6gFwCH733uSp5HeoO997kqeR3qAXAIfvfe5Knkd6g733uSp5HeoBcAh+997kqeR3qDvfe5Knkd6gFwCH733uSp5HeoO997kqeR3qAXAIfvfe5Knkd6g733uSp5HeoBcAh+997kqeR3qDvfe5Knkd6gFwCH733uSp5HeoO997kqeR3qAXAIfvfe5Knkd6g733uSp5HeoBcAh+997kqeR3qDvfe5Knkd6gE6AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/9k=\n",
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"600\"\n",
       "            height=\"400\"\n",
       "            src=\"https://www.youtube.com/embed/u7x8RXwLKcA\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "            \n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.YouTubeVideo at 0x7f71b6bea290>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.display import YouTubeVideo\n",
    "\n",
    "YouTubeVideo('u7x8RXwLKcA', width=600, height=400)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    },
    "tags": []
   },
   "source": [
    "## O que é uma Rede Neural Artificial?\n",
    "\n",
    "Redes neurais artificiais (RNAs) são modelos computacionais inspirados pelo sistema nervoso central (em particular o cérebro) que são capazes de realizar o aprendizado de máquina bem como o reconhecimento de padrões. Redes neurais artificiais geralmente são apresentadas como sistemas de \"neurônios interconectados, que podem computar valores de entradas\", simulando o comportamento de redes neurais biológicas."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img src=\"images/redes_neurais.jpeg\" alt=\"redes neurais\" style=\"width: 500px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Como a rede neural aprende?\n",
    "\n",
    "Em cada neurônio há uma função de ativação (*activation function*) que processa uma combinação linear entre inputs e pesos sinápticos, gerando assim um sinal de saída.\n",
    "\n",
    "A informação flui da *input layer* para as *hidden layers* e por fim para a *output layer*. Nesse fluxo os inputs de dados da *input layer* são alimentados para os neurônios das *hidden layers* que por fim alimentam o neurônio final da *output layer*.\n",
    "\n",
    "A primeira passada de informação (propagação) pela rede é geralmente feita com parâmetros aleatórios para as funções de ativação dos neurônios.\n",
    "\n",
    "Ao realizar a propagação, chamada de *feed forward*, temos sinais de saídas nos neurônios da output layer. \n",
    "\n",
    "No fim da propagação, a função custo (uma métrica de erro) é calculada e o modelo então ajusta os parâmetros dos neurônios na direção de um menor custo (por meio do gradiente - derivada multivariada).\n",
    "\n",
    "Assim uma nova propagação é gerada e a numa nova função custo e calculada. Assim como é realizado a atualização dos parâmetros dos neurônios.\n",
    "\n",
    "O nome desse algoritmo é **Retro-propagação** (*Backpropagation*). E cada vez que ele é executado denomina-se como época (*epoch*). E quandos as épocas estabelecidas se encerram, a rede neural encerra o seu treinamento/aprendizagem."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img src=\"images/backpropagation.gif\" alt=\"backpropagation\" style=\"width: 500px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Funções de Ativação\n",
    "\n",
    "| **Sigmoid**                                                  | **Tanh**                                                     | **ReLU**                                                     | **Leaky ReLU**                                               |\n",
    "| ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |\n",
    "| $g(z)=\\frac{1}{1+e^{-z}}$                                    | $g(z)=\\frac{e^{z}-e^{-z}}{e^{z}+e^{-z}}$                     | $g(z)=\\max (0, z)$                                           | $\\begin{array}{c}{g(z)=\\max (\\epsilon z, z)} \\\\ {\\text { com } \\epsilon \\ll 1}\\end{array}$ |\n",
    "| ![Illustration](images/sigmoid.png) | ![Illustration](images/tanh.png) | ![Illustration](images/relu.png) | ![Illustration](images/leaky-relu.png) |\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Estruturação dos módulos de PyTorch\n",
    "\n",
    "```python\n",
    "import torch\n",
    "```\n",
    "* [`torch.Tensor`](https://pytorch.org/docs/stable/tensors.html) - Tensores (arrays N-D)\n",
    "* [`torch.nn`](https://pytorch.org/docs/stable/nn.html) - Redes Neurais (_**N**eural **N**etworks_)\n",
    "* [`torch.optim`](https://pytorch.org/docs/stable/optim.html) - Otimização (_**Optim**ization_)\n",
    "* [`torch.data`](https://pytorch.org/docs/stable/data.html) - *Datasets* e Ferramentas de Streaming de Dados\n",
    "* [`torch.autograd`](https://pytorch.org/docs/stable/autograd.html) - Diferenciação Automática (_**Auto**matic Differentiation_)\n",
    "* [`torch.vision`](https://pytorch.org/docs/stable/torchvision/index.html) - Ferramentas de Manipulação de Imagens e Visão Computacional\n",
    "* [`torch.audio`](https://pytorch.org/audio/stable/index.html) - Ferramentas de Manipulação de Áudio\n",
    "* [`torch.jit`](https://pytorch.org/docs/stable/jit.html) - Compilação _**j**ust-**i**n-**t**ime_ de modelos PyTorch em binários\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### `torch.Tensor`\n",
    "\n",
    "* `NumPy` - `np.ndarray`\n",
    "* `pandas` - `pd.Series` e `pd.DataFrame`\n",
    "* `PyTorch` - `torch.Tensor`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Algoritmos de Otimização\n",
    "\n",
    "PyTorch possui diversos:\n",
    "* **SGD**: _**S**tochastic **G**radient **D**escent_\n",
    "* **SGD com Momento**: SGD com Momento usando a derivada (ou gradiente) do ponto atual\n",
    "* **SGD com Momento Nesterov**: SGD com Momento mas  usa a derivada (ou o gradiente) parcial do ponto seguinte (Nesterov, 1983)\n",
    "* **RMSprop**: SGD com taxa de aprendizagem adaptativa  (Hinton, Srivastava & Swersky, 2012) - `RMSProp`\n",
    "* **AdaGrad**: SGD com taxa de aprendizagem adaptativa (Duchi, Hazan, & Yoram, 2011) -  `AdaGrad`\n",
    "* **Adam**: SGD com taxa de aprendizagem adaptativa e momento (Kingma, Diederick & Jimmy, 2014) - `ADAM`\n",
    "\n",
    "Os mais importantes são o SGD e o Adam"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### SGD - Stochastic Gradient Descent\n",
    "\n",
    "[`torch.optim.sgd()`](https://pytorch.org/docs/stable/_modules/torch/optim/sgd.html):\n",
    "\n",
    "* `lr` - Taxa de Aprendizagem $\\eta > 0$\n",
    "* `momentum=0.0` - hyperparâmetro $\\geq 0$ que acelera o *gradient descent* na direção relevante e mitiga oscilações. \n",
    "* `nesterov=False` - `bool` para se aplica *Nesterov Momentum* ou não. *Nesterov Momentum* usa posições intermediárias do gradiente no cálculo do *momentum*. Proposto por Yuri Nesterov em 1983.\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Momentum\n",
    "\n",
    "<img src=\"images/momentum.gif\" alt=\"momentum\" style=\"width: 300px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Adam\n",
    "\n",
    "Adam é um método de SGD que usa uma estimativa adaptativa dos momentos de primeira ordem e momentos de segunda ordem. Proposto por Kingma & Ba (2014).\n",
    "\n",
    "[`torch.optim.Adam()`](https://pytorch.org/docs/stable/_modules/torch/optim/adam.html):\n",
    "\n",
    "* `lr=0.001` - Taxa de Aprendizagem $\\eta > 0$\n",
    "* `betas=(0.9, 0.999)` - Uma tupla de valores:\n",
    "    1. `betas[0]` - decréscimo exponencial da estimativa dos momentos de primeira ordem\n",
    "    2. `betas[1]` - decréscimo exponencial da estimativa dos momentos de segunda ordem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Momentum\n",
    "\n",
    "<img src=\"images/comparacao_otimizadores.gif\" alt=\"comparacao_otimizadores\" style=\"width: 500px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Momentum\n",
    "\n",
    "<img src=\"images/opt1.gif\" alt=\"comparacao_otimizadores_2\" style=\"width: 500px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Momentum\n",
    "\n",
    "<img src=\"images/opt2.gif\" alt=\"comparacao_otimizadores_3\" style=\"width: 500px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Funções Custo\n",
    "\n",
    "As funções custos se dividem em dois tipos:\n",
    "\n",
    "1. Funções Custo de **Classificação**\n",
    "2. Funções Custo de **Regressão**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Funcões Custo de Classificação\n",
    "\n",
    "Mais utilizadas\n",
    "\n",
    "\n",
    "* *Binary Cross-entropy* (Entropia cruzada binária): [`torch.nn.BCELoss()`](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html)\n",
    "* *Categorical Cross-entropy - Negative Log-Likelihood Loss* (Entropia cruzada categórica): [`torch.nn.NLLLoss()`](https://pytorch.org/docs/stable/generated/torch.nn.NLLLoss.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Funcões Custo de Regressão\n",
    "\n",
    "Mais utilizadas\n",
    "* MSE - *Mean Squared Error* (Erro quadrado médio): [`torch.nn.MSELoss()`](https://pytorch.org/docs/stable/generated/torch.nn.MSELoss.html)\n",
    "* MAE - *Mean Absolute Error - $\\| . \\|_1$* (Erro absoluto médio): [`torch.nn.L1Loss()`](https://pytorch.org/docs/stable/generated/torch.nn.L1Loss.html#torch.nn.L1Loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Como construir sua rede neural no PyTorch\n",
    "\n",
    "Construir redes neurais com o **PyTorch** é tão fácil quanto com **Keras**.\n",
    "\n",
    "Temos que criar uma Rede Neural a partir de uma classe [`nn.Module()`](https://pytorch.org/docs/stable/generated/torch.nn.Module.html?highlight=module#torch.nn.Module) e criar um construtor com o método `__init__()` e implementar todas as layers e propagações desejadas com o método `forward()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model(\n",
      "  (fc1): Linear(in_features=4, out_features=4, bias=True)\n",
      "  (fc2): Linear(in_features=4, out_features=1, bias=True)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "class Model(nn.Module):\n",
    "    # Construtor\n",
    "    def __init__(self):  # Chamado de dunder methods __init__\n",
    "        super(Model, self).__init__()\n",
    "        self.fc1 = nn.Linear(4, 4) # primeira hidden layer\n",
    "        self.fc2 = nn.Linear(4, 1) # segunda hidden layer\n",
    "\n",
    "    # Propagação (Feed Forward)\n",
    "    def forward(self, x):\n",
    "        x = F.relu(self.fc1(x)) # poderia colocar F.leaky_relu\n",
    "        x = F.relu(self.fc2(x))\n",
    "        return x\n",
    "\n",
    "\n",
    "# Instancia o Model()\n",
    "model = Model()\n",
    "\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Exemplo de Classificação Binária"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ClassBin(\n",
      "  (fc1): Linear(in_features=20, out_features=20, bias=True)\n",
      "  (fc2): Linear(in_features=20, out_features=1, bias=True)\n",
      "  (sigmoid): Sigmoid()\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "class ClassBin(nn.Module):\n",
    "    # Construtor\n",
    "    def __init__(self):\n",
    "        super(ClassBin, self).__init__()\n",
    "        self.fc1 = nn.Linear(20, 20) # primeira hidden layer\n",
    "        self.fc2 = nn.Linear(20, 1) # segunda hidden layer\n",
    "        self.sigmoid = nn.Sigmoid() # output layer com ativação Sigmoid\n",
    "\n",
    "    # Propagação (Feed Forward)\n",
    "    def forward(self, x):\n",
    "        x = F.relu(self.fc1(x))\n",
    "        x = F.relu(self.fc2(x))\n",
    "        x = self.sigmoid(x)\n",
    "        return x\n",
    "\n",
    "model = ClassBin()\n",
    "\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Exemplo de Regressão"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reg(\n",
      "  (fc1): Linear(in_features=20, out_features=20, bias=True)\n",
      "  (fc2): Linear(in_features=20, out_features=1, bias=True)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "class Reg(nn.Module):\n",
    "    # Construtor\n",
    "    def __init__(self):\n",
    "        super(Reg, self).__init__()\n",
    "        self.fc1 = nn.Linear(20, 20) # primeira hidden layer\n",
    "        self.fc2 = nn.Linear(20, 1) # segunda hidden layer output 1 único neurônio\n",
    "\n",
    "    # Propagação (Feed Forward)\n",
    "    def forward(self, x):\n",
    "        x = F.relu(self.fc1(x))\n",
    "        x = F.relu(self.fc2(x))\n",
    "        return x\n",
    "\n",
    "model = Reg()\n",
    "\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Exemplo de Multiclassificação (não-binária - acima de duas classes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MultiClass(\n",
      "  (fc1): Linear(in_features=20, out_features=20, bias=True)\n",
      "  (fc2): Linear(in_features=20, out_features=10, bias=True)\n",
      "  (softmax): Softmax(dim=10)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "class MultiClass(nn.Module):\n",
    "    # Construtor\n",
    "    def __init__(self):\n",
    "        super(MultiClass, self).__init__()\n",
    "        self.fc1 = nn.Linear(20, 20)  # primeira hidden layer\n",
    "        self.fc2 = nn.Linear(20, 10)  # segunda hidden layer\n",
    "        self.softmax = nn.Softmax(10) # output layer com ativação ativação softmax com 10 classes\n",
    "\n",
    "    # Propagação (Feed Forward)\n",
    "    def forward(self, x):\n",
    "        x = F.relu(self.fc1(x))\n",
    "        x = F.relu(self.fc2(x))\n",
    "        x = self.softmax(x)\n",
    "        return x\n",
    "\n",
    "model = MultiClass()\n",
    "\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Como treinar sua rede neural no PyTorch\n",
    "\n",
    "Uma vez especificado e instanciado o modelo, podemos manipulá-lo de maneira dinâmica. Não é preciso \"compilar\" que nem o TensorFlow/Keras. Escolhemos a função custo (`loss_fn`) como `nn.NLLLoss()` e também a taxa de aprendizagem $\\eta$ em `1e-6` e a quantidade de épocas a serem treinadas (`epochs`):\n",
    "\n",
    "```python\n",
    "model = Sua_rede_neural()\n",
    "loss_fn = nn.NLLLoss()\n",
    "learning_rate = 1e-6\n",
    "epochs = 100\n",
    "\n",
    "# Instânciar o Otimizador SGD\n",
    "optimizer = torch.optim.sgd(model.parameters(), lr=learning_rate)\n",
    "\n",
    "for t in range(epochs):\n",
    "    model.train() # Colocar o modelo em modo de treinamento (calcula os gradientes)\n",
    "    \n",
    "    # Propagação (Feed Forward)\n",
    "    y_pred = model(X_train)\n",
    "    \n",
    "    # Calcular erro usando a função-custo\n",
    "    loss = loss_fn(y_pred, y_train)\n",
    "    print(t, loss.item())\n",
    "\n",
    "    # Zera os gradientes antes da Retro-propagação (Backpropagation)\n",
    "    model.zero_grad()\n",
    "\n",
    "    # Retro-propagação (Backpropagation)\n",
    "    loss.backward()\n",
    "\n",
    "    # Atualização dos parâmetros\n",
    "    optimizer.step()\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## Como ajustar o treinamento sua rede neural no PyTorch\n",
    "\n",
    "* Batch Size\n",
    "* Dropout"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Batch Size\n",
    "\n",
    "Tamanho do Batch de dados que passa por vez pela rede neural antes da atualização dos parâmetros pelo *backpropagation*. Tamanhos grandes resultam em instabilidade no treinamento. Geralmente usam-se potências de $2$ $(2,4,8,16,\\dots, 2^n)$.\n",
    "\n",
    "Em Abril de 2018, Yann LeCun, um dos principais pesquisadores sobre redes neurais e ganhador do \"nobel\" da computação (Prêmio Turing) [twittou](https://twitter.com/ylecun/status/989610208497360896) em resposta à Masters & Luschi (2018) que mostrava diversos contextos de *batch size*:\n",
    "\n",
    ">\"Friends don't let friends use mini-batches larger than 32\"\n",
    "\n",
    "Então 32 é um valor empiricamente verificado que dá estabilidade ao treinamento.\n",
    "\n",
    "> Dominic Masters and Carlo Luschi. \"Revisiting small batch training for deep neural networks.\" _arXiv preprint arXiv:1804.07612_ (2018)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Para controlar como que os dados são inseridos no modelo e, logo, o Batch Size é preciso implementar um [`torch.utils.data.DataLoader()`](https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader):\n",
    "\n",
    "```python\n",
    "import torch\n",
    "from torch.utils.data import TorchDataset, DataLoader\n",
    "\n",
    "# Converter X e y para torch.Tensor\n",
    "X = torch.Tensor(X)\n",
    "y = torch.Tensor(y)\n",
    "\n",
    "# Um Dataset de Tensores - Array [X, y]\n",
    "dataset = TensorDataset(X, y)\n",
    "loader = DataLoader(dataset, batch_size=32, shuffle=True)\n",
    "```\n",
    "\n",
    "Argumentos do `DataLoader()`: \n",
    "\n",
    "* **`dataset`**: um `Dataset` PyTorch\n",
    "    * tem [vários tipos](https://pytorch.org/docs/stable/data.html)\n",
    "    * no nosso exemplo vou usar um simples [`TensorDataset`](https://pytorch.org/docs/stable/_modules/torch/utils/data/dataset.html#TensorDataset) que é um wrapper de `np.ndarray` e `pd.Series` para `torch.Tensor`)\n",
    "* **batch_size**: `int` - tamanho do Batch Size, padrão é 1\n",
    "* **shuffle**: `bool` - se vai embaralhar os dados antes de enviar em batches ao modelo, padrão é `False`. Recomendo usar `shuffle=True`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Dropout\n",
    "\n",
    "Uma medida de regularização na qual evita-se overfitting proposta por Hinton em 2012. *Dropout* é um algoritmo que especifica que a cada iteração de época do treino os neurônios possuem uma probabilidade de serem removidos (não utilizados) para a aprendizagem. Geralmente a probabilidade ideal fica em torno de 20% ($0.2$).\n",
    "\n",
    "Coloca-se como se fosse uma camada após a camada que deseja aplicar o dropout:\n",
    "\n",
    "```python\n",
    "model = torch.nn.Sequential(\n",
    "    nn.Linear(4, 4) ,  # hidden layer\n",
    "    nn.Dropout(0.2)    # dropout layer\n",
    ")\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img src=\"images/dropout.gif\" alt=\"dropout\" style=\"width: 500px;\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Exemplo com o dataset Titanic\n",
    "\n",
    "Contém 891 passageiros reais do Titanic que afundou em 15/04/1912 matando 1502 de 2224 passageiros e tripulação que estavam a bordo.\n",
    "\n",
    "* `survived`: *dummy* `0` ou `1` \n",
    "* `pclass`: Classe do Passageiro\n",
    "    - `1`: Primeira Classe\n",
    "    - `2`: Segunda Classe\n",
    "    - `3`: Terceira Classe\n",
    "* `sex`: Sexo `male` ou `female`\n",
    "* `age`: Idade\n",
    "* `sibsp`: Número de Irmãos (*Siblings*) e Esposas (*spouse*) a bordo\n",
    "* `parch`: Número de pais/filhos a bordo\n",
    "* `fare`: Valor pago pela passagem em libras\n",
    "* `embarked`: Porto que embarcou\n",
    "    - `C`: Cherbourg\n",
    "    - `Q`: Queenstown\n",
    "    - `S`: Southampton)\n",
    "* `class`: Mesmo que `pclass` só que em texto\n",
    "* `adult_male`: *dummy* para `age > 16` e `sex == 'male'`\n",
    "* `deck`: Qual deck a cabine do passageiro se situava\n",
    "* `alive`: Mesmo que survived só que com `yes` ou `no`\n",
    "* `alone`: *dummy* para se viajava sozinho\n",
    "\n",
    ">Obs: usar `random_state = 123`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "<img src=\"images/titanic.png\" alt=\"titanic\" style=\"width: 500px;\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "titanic = sns.load_dataset('titanic')\n",
    "\n",
    "feature_names = ['pclass', 'female', 'age', 'fare']\n",
    "titanic['female'] = titanic['sex'].map({'male': 0, 'female': 1})\n",
    "titanic.dropna(subset=feature_names, inplace=True)  #891 para 714\n",
    "\n",
    "X = titanic[feature_names].to_numpy()\n",
    "y = titanic['survived'].to_numpy()\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y,\n",
    "                                                    test_size=0.25,\n",
    "                                                    random_state=123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tamanho de X_train:  (535, 4)\n",
      "Tamanho de X_test:  (179, 4)\n",
      "Tamanho de y_train:  (535,)\n",
      "Tamanho de y_test:  (179,)\n"
     ]
    }
   ],
   "source": [
    "print('Tamanho de X_train: ', X_train.shape)\n",
    "print('Tamanho de X_test: ', X_test.shape)\n",
    "print('Tamanho de y_train: ', y_train.shape)\n",
    "print('Tamanho de y_test: ', y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true,
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ClassBin(\n",
      "  (linear1): Linear(in_features=4, out_features=4, bias=True)\n",
      "  (dropout1): Dropout(p=0.2, inplace=False)\n",
      "  (linear2): Linear(in_features=4, out_features=4, bias=True)\n",
      "  (dropout2): Dropout(p=0.2, inplace=False)\n",
      "  (linear3): Linear(in_features=4, out_features=1, bias=True)\n",
      "  (dropout3): Dropout(p=0.2, inplace=False)\n",
      "  (sigmoid): Sigmoid()\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "class ClassBin(nn.Module):\n",
    "    # Construtor\n",
    "    def __init__(self):\n",
    "        super(ClassBin, self).__init__()\n",
    "        self.linear1 = nn.Linear(4, 4)    # primeira hidden layer\n",
    "        self.dropout1 = nn.Dropout(0.2)   # dropout layer\n",
    "        self.linear2 = nn.Linear(4, 4)    # segunda hidden layer\n",
    "        self.dropout2 = nn.Dropout(0.2)   # dropout layer\n",
    "        self.linear3 = nn.Linear(4, 1)    # terceira hidden layer\n",
    "        self.dropout3 = nn.Dropout(0.2)   # dropout layer\n",
    "        self.sigmoid = nn.Sigmoid()\n",
    "\n",
    "    # Propagação (Feed Forward)\n",
    "    def forward(self, x):\n",
    "        x = F.relu(self.linear1(x))\n",
    "        x = self.dropout1(x)\n",
    "        x = F.relu(self.linear2(x))\n",
    "        x = self.dropout2(x)\n",
    "        x = F.relu(self.linear3(x))\n",
    "        x = self.dropout3(x)\n",
    "        x = self.sigmoid(x)\n",
    "        return x\n",
    "\n",
    "model = ClassBin()\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "loss_fn = nn.BCELoss()\n",
    "epochs = 100\n",
    "batch_size = 32  # X_train 535 / 32 = 16.71 (então são 17 batches de 32)\n",
    "learning_rate = 0.1\n",
    "\n",
    "# Instânciar o Otimizador Adam\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "from torch.utils.data import TensorDataset, DataLoader\n",
    "\n",
    "# Converter X e y para torch.Tensor\n",
    "X_train = torch.Tensor(X_train)\n",
    "y_train = torch.Tensor(y_train)\n",
    "X_test = torch.Tensor(X_test)\n",
    "y_test = torch.Tensor(y_test)\n",
    "\n",
    "# Um Dataset de Tensores - Array [X, y]\n",
    "train = TensorDataset(X_train, y_train)\n",
    "train_loader = DataLoader(train, batch_size=batch_size, shuffle=True)\n",
    "\n",
    "test = TensorDataset(X_test, y_test)\n",
    "test_loader = DataLoader(test, batch_size=batch_size, shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Época 1,\n",
      "          Custo Treino: 0.693\n",
      "Época 2,\n",
      "          Custo Treino: 0.693\n",
      "Época 3,\n",
      "          Custo Treino: 0.693\n",
      "Época 4,\n",
      "          Custo Treino: 0.693\n",
      "Época 5,\n",
      "          Custo Treino: 0.693\n",
      "Época 6,\n",
      "          Custo Treino: 0.693\n",
      "Época 7,\n",
      "          Custo Treino: 0.693\n",
      "Época 8,\n",
      "          Custo Treino: 0.693\n",
      "Época 9,\n",
      "          Custo Treino: 0.693\n",
      "Época 10,\n",
      "          Custo Treino: 0.693\n",
      "Época 11,\n",
      "          Custo Treino: 0.693\n",
      "Época 12,\n",
      "          Custo Treino: 0.693\n",
      "Época 13,\n",
      "          Custo Treino: 0.693\n",
      "Época 14,\n",
      "          Custo Treino: 0.693\n",
      "Época 15,\n",
      "          Custo Treino: 0.693\n",
      "Época 16,\n",
      "          Custo Treino: 0.693\n",
      "Época 17,\n",
      "          Custo Treino: 0.693\n",
      "Época 18,\n",
      "          Custo Treino: 0.693\n",
      "Época 19,\n",
      "          Custo Treino: 0.693\n",
      "Época 20,\n",
      "          Custo Treino: 0.693\n",
      "Época 21,\n",
      "          Custo Treino: 0.693\n",
      "Época 22,\n",
      "          Custo Treino: 0.693\n",
      "Época 23,\n",
      "          Custo Treino: 0.693\n",
      "Época 24,\n",
      "          Custo Treino: 0.693\n",
      "Época 25,\n",
      "          Custo Treino: 0.693\n",
      "Época 26,\n",
      "          Custo Treino: 0.693\n",
      "Época 27,\n",
      "          Custo Treino: 0.693\n",
      "Época 28,\n",
      "          Custo Treino: 0.693\n",
      "Época 29,\n",
      "          Custo Treino: 0.693\n",
      "Época 30,\n",
      "          Custo Treino: 0.693\n",
      "Época 31,\n",
      "          Custo Treino: 0.693\n",
      "Época 32,\n",
      "          Custo Treino: 0.693\n",
      "Época 33,\n",
      "          Custo Treino: 0.693\n",
      "Época 34,\n",
      "          Custo Treino: 0.693\n",
      "Época 35,\n",
      "          Custo Treino: 0.693\n",
      "Época 36,\n",
      "          Custo Treino: 0.693\n",
      "Época 37,\n",
      "          Custo Treino: 0.693\n",
      "Época 38,\n",
      "          Custo Treino: 0.693\n",
      "Época 39,\n",
      "          Custo Treino: 0.693\n",
      "Época 40,\n",
      "          Custo Treino: 0.693\n",
      "Época 41,\n",
      "          Custo Treino: 0.693\n",
      "Época 42,\n",
      "          Custo Treino: 0.693\n",
      "Época 43,\n",
      "          Custo Treino: 0.693\n",
      "Época 44,\n",
      "          Custo Treino: 0.693\n",
      "Época 45,\n",
      "          Custo Treino: 0.693\n",
      "Época 46,\n",
      "          Custo Treino: 0.693\n",
      "Época 47,\n",
      "          Custo Treino: 0.693\n",
      "Época 48,\n",
      "          Custo Treino: 0.693\n",
      "Época 49,\n",
      "          Custo Treino: 0.693\n",
      "Época 50,\n",
      "          Custo Treino: 0.693\n",
      "Época 51,\n",
      "          Custo Treino: 0.693\n",
      "Época 52,\n",
      "          Custo Treino: 0.693\n",
      "Época 53,\n",
      "          Custo Treino: 0.693\n",
      "Época 54,\n",
      "          Custo Treino: 0.693\n",
      "Época 55,\n",
      "          Custo Treino: 0.693\n",
      "Época 56,\n",
      "          Custo Treino: 0.693\n",
      "Época 57,\n",
      "          Custo Treino: 0.693\n",
      "Época 58,\n",
      "          Custo Treino: 0.693\n",
      "Época 59,\n",
      "          Custo Treino: 0.693\n",
      "Época 60,\n",
      "          Custo Treino: 0.693\n",
      "Época 61,\n",
      "          Custo Treino: 0.693\n",
      "Época 62,\n",
      "          Custo Treino: 0.693\n",
      "Época 63,\n",
      "          Custo Treino: 0.693\n",
      "Época 64,\n",
      "          Custo Treino: 0.693\n",
      "Época 65,\n",
      "          Custo Treino: 0.693\n",
      "Época 66,\n",
      "          Custo Treino: 0.693\n",
      "Época 67,\n",
      "          Custo Treino: 0.693\n",
      "Época 68,\n",
      "          Custo Treino: 0.693\n",
      "Época 69,\n",
      "          Custo Treino: 0.693\n",
      "Época 70,\n",
      "          Custo Treino: 0.693\n",
      "Época 71,\n",
      "          Custo Treino: 0.693\n",
      "Época 72,\n",
      "          Custo Treino: 0.693\n",
      "Época 73,\n",
      "          Custo Treino: 0.693\n",
      "Época 74,\n",
      "          Custo Treino: 0.693\n",
      "Época 75,\n",
      "          Custo Treino: 0.693\n",
      "Época 76,\n",
      "          Custo Treino: 0.693\n",
      "Época 77,\n",
      "          Custo Treino: 0.693\n",
      "Época 78,\n",
      "          Custo Treino: 0.693\n",
      "Época 79,\n",
      "          Custo Treino: 0.693\n",
      "Época 80,\n",
      "          Custo Treino: 0.693\n",
      "Época 81,\n",
      "          Custo Treino: 0.693\n",
      "Época 82,\n",
      "          Custo Treino: 0.693\n",
      "Época 83,\n",
      "          Custo Treino: 0.693\n",
      "Época 84,\n",
      "          Custo Treino: 0.693\n",
      "Época 85,\n",
      "          Custo Treino: 0.693\n",
      "Época 86,\n",
      "          Custo Treino: 0.693\n",
      "Época 87,\n",
      "          Custo Treino: 0.693\n",
      "Época 88,\n",
      "          Custo Treino: 0.693\n",
      "Época 89,\n",
      "          Custo Treino: 0.693\n",
      "Época 90,\n",
      "          Custo Treino: 0.693\n",
      "Época 91,\n",
      "          Custo Treino: 0.693\n",
      "Época 92,\n",
      "          Custo Treino: 0.693\n",
      "Época 93,\n",
      "          Custo Treino: 0.693\n",
      "Época 94,\n",
      "          Custo Treino: 0.693\n",
      "Época 95,\n",
      "          Custo Treino: 0.693\n",
      "Época 96,\n",
      "          Custo Treino: 0.693\n",
      "Época 97,\n",
      "          Custo Treino: 0.693\n",
      "Época 98,\n",
      "          Custo Treino: 0.693\n",
      "Época 99,\n",
      "          Custo Treino: 0.693\n",
      "Época 100,\n",
      "          Custo Treino: 0.693\n"
     ]
    }
   ],
   "source": [
    "for t in range(epochs):\n",
    "    model.train() # Colocar o modelo em modo de treinamento (calcula os gradientes)\n",
    "    \n",
    "    # Batch Size\n",
    "    for data in train_loader:\n",
    "        # dar nome aos bois\n",
    "        X = data[0]\n",
    "        y = data[1]\n",
    "        \n",
    "        # Propagação (Feed Forward)\n",
    "        y_pred = model(X)\n",
    "    \n",
    "        # Calcular erro usando a função-custo\n",
    "        # y precisa virar um Tensor com tamanho (batch_size, 1)\n",
    "        loss = loss_fn(y_pred, y.unsqueeze_(1))\n",
    "        \n",
    "        # Zera os gradientes antes da Retro-propagação (Backpropagation)\n",
    "        model.zero_grad()\n",
    "\n",
    "        # Retro-propagação (Backpropagation)\n",
    "        loss.backward()\n",
    "\n",
    "        # Atualização dos parâmetros\n",
    "        optimizer.step()\n",
    "\n",
    "    # Fim da Época\n",
    "    print(f\"\"\"Época {t + 1},\n",
    "          Custo Treino: {round(loss.item(), 3)}\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Acurácia do Modelo\n",
    "\n",
    "Usar o comando `model.evaluate()`\n",
    "\n",
    "Para a métrica acurácia, retorna um score de acurácia `float` entre $0$ e $1$\n",
    "    \n",
    "> Obs: Regressão Logística acurácias: 0.69 Treino e 0.7 Teste\n",
    "\n",
    "> Obs: *Support Vector Machines* acurácias: 0.79 Treino e 0.75 Teste\n",
    "\n",
    "> Obs: Árvores de Decisão acurácias: 0.79 Treino e 0.79 Teste\n",
    "\n",
    "> Obs: Florestas Aleatórias acurácias: 0.84 Treino e 0.82 Teste"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Acurácia de Treino: 0.590654194355011\n",
      "\n",
      " ---------------------------\n",
      "\n",
      "Acurácia de Teste: 0.6033519506454468\n"
     ]
    }
   ],
   "source": [
    "model.eval() # coloca o modelo em modo de avaliação (sem calcular gradientes)\n",
    "\n",
    "train_pred = model(X_train)\n",
    "train_pred = train_pred.detach().apply_(lambda x : 1 if x > 0.5 else 0)\n",
    "train_acc = torch.sum(train_pred.flatten() == y_train) / train_pred.size(0)\n",
    "\n",
    "test_pred = model(X_test)\n",
    "test_pred = test_pred.detach().apply_(lambda x : 1 if x > 0.5 else 0)\n",
    "test_acc = torch.sum(test_pred.flatten() == y_test) / test_pred.size(0)\n",
    "\n",
    "print(f\"Acurácia de Treino: {train_acc}\")\n",
    "print('\\n ---------------------------\\n')\n",
    "print(f\"Acurácia de Teste: {test_acc}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Referências\n",
    "\n",
    "* Hinton, Geoffrey, Nitish Srivastava, and Kevin Swersky. “Neural Networks for Machine Learning Lecture 6a Overview of Mini--Batch Gradient Descent,” 2012.\n",
    "* Kingma, Diederik P., and Jimmy Ba. “Adam: A Method for Stochastic Optimization,” December 22, 2014. https://arxiv.org/abs/1412.6980.\n",
    "* Nesterov, Y. A method of solving a convex programming problem with convergence rate O(1/sqr(k)). Soviet Mathematics Doklady, 27:372–376, 1983.\n",
    "* Srivastava, Nitish, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. “Dropout: A Simple Way to Prevent Neural Networks from Overfitting.” Journal of Machine Learning Research 15, no. 56 (2014): 1929–58."
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4"
  },
  "rise": {
   "autolaunch": true,
   "enable_chalkboard": true,
   "progress": true,
   "scroll": true,
   "slideNumber": true,
   "theme": "black"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}