Navigation_SNH_run_saved.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Navigation: Run Pre-Trained Agents\n",
    "\n",
    "---\n",
    "\n",
    "This noetbook is adapted from the first project of the [Deep Reinforcement Learning Nanodegree](https://www.udacity.com/course/deep-reinforcement-learning-nanodegree--nd893).\n",
    "\n",
    "There is a separate notebook, Navigation_SNH.ipynb, for the training of the learning agent. This additional notebook is just for running a previously trained agent.\n",
    "\n",
    "### 1. Start the Environment\n",
    "\n",
    "We begin by importing some necessary packages.  If the code cell below returns an error, please revisit the project instructions to double-check that you have installed [Unity ML-Agents](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Installation.md) and [NumPy](http://www.numpy.org/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from unityagents import UnityEnvironment\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import torch\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we will start the environment!  **_Before running the code cell below_**, change the `file_name` parameter to match the location of the Unity environment that you downloaded.\n",
    "\n",
    "- **Mac**: `\"path/to/Banana.app\"`\n",
    "- **Windows** (x86): `\"path/to/Banana_Windows_x86/Banana.exe\"`\n",
    "- **Windows** (x86_64): `\"path/to/Banana_Windows_x86_64/Banana.exe\"`\n",
    "- **Linux** (x86): `\"path/to/Banana_Linux/Banana.x86\"`\n",
    "- **Linux** (x86_64): `\"path/to/Banana_Linux/Banana.x86_64\"`\n",
    "- **Linux** (x86, headless): `\"path/to/Banana_Linux_NoVis/Banana.x86\"`\n",
    "- **Linux** (x86_64, headless): `\"path/to/Banana_Linux_NoVis/Banana.x86_64\"`\n",
    "\n",
    "For instance, if you are using a Mac, then you downloaded `Banana.app`.  If this file is in the same folder as the notebook, then the line below should appear as follows:\n",
    "```\n",
    "env = UnityEnvironment(file_name=\"Banana.app\")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:unityagents:\n",
      "'Academy' started successfully!\n",
      "Unity Academy name: Academy\n",
      "        Number of Brains: 1\n",
      "        Number of External Brains : 1\n",
      "        Lesson number : 0\n",
      "        Reset Parameters :\n",
      "\t\t\n",
      "Unity brain name: BananaBrain\n",
      "        Number of Visual Observations (per agent): 0\n",
      "        Vector Observation space type: continuous\n",
      "        Vector Observation space size (per agent): 37\n",
      "        Number of stacked Vector Observation: 1\n",
      "        Vector Action space type: discrete\n",
      "        Vector Action space size (per agent): 4\n",
      "        Vector Action descriptions: , , , \n"
     ]
    }
   ],
   "source": [
    "env = UnityEnvironment(file_name=\"Banana_Windows_x86_64/Banana.exe\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Environments contain **_brains_** which are responsible for deciding the actions of their associated agents. Here we check for the first brain available, and set it as the default brain we will be controlling from Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the default brain\n",
    "brain_name = env.brain_names[0]\n",
    "brain = env.brains[brain_name]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of agents: 1\n",
      "Number of actions: 4\n",
      "States look like: [1.         0.         0.         0.         0.84408134 0.\n",
      " 0.         1.         0.         0.0748472  0.         1.\n",
      " 0.         0.         0.25755    1.         0.         0.\n",
      " 0.         0.74177343 0.         1.         0.         0.\n",
      " 0.25854847 0.         0.         1.         0.         0.09355672\n",
      " 0.         1.         0.         0.         0.31969345 0.\n",
      " 0.        ]\n",
      "States have length: 37\n",
      "<unityagents.brain.BrainInfo object at 0x000002B5EF3DBA20>\n"
     ]
    }
   ],
   "source": [
    "# reset the environment\n",
    "env_info = env.reset(train_mode=True)[brain_name]\n",
    "\n",
    "# number of agents in the environment\n",
    "print('Number of agents:', len(env_info.agents))\n",
    "\n",
    "# number of actions\n",
    "action_size = brain.vector_action_space_size\n",
    "print('Number of actions:', action_size)\n",
    "\n",
    "# examine the state space \n",
    "state = env_info.vector_observations[0]\n",
    "print('States look like:', state)\n",
    "state_size = len(state)\n",
    "print('States have length:', state_size)\n",
    "\n",
    "print(env_info)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Declare the Agent\n",
    "Most of the parameters for Agent, described in more detail in the training notebook Navigation_SNH.ipynb, are irrelevant unless training is performed.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda:0\n"
     ]
    }
   ],
   "source": [
    "from collections import deque\n",
    "from dqn_agent import Agent, load_and_run_agent\n",
    "\n",
    "agent = Agent(\n",
    "    state_size=state_size,\n",
    "    action_size=action_size\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Load and Run the Pre-Trained Agent\n",
    "The agent is run for a specificed number of episdoes and the score is printed each episode. To load in a different checkpoint files, change the file name sent to load_and_run_agent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Score: 16.0\n",
      "Score: 23.0\n",
      "Score: 15.0\n",
      "Score: 23.0\n",
      "Score: 19.0\n",
      "Score: 4.0\n",
      "Score: 21.0\n",
      "Score: 14.0\n",
      "Score: 20.0\n",
      "Score: 15.0\n",
      "Score: 19.0\n",
      "Score: 17.0\n",
      "Score: 18.0\n",
      "Score: 17.0\n",
      "Score: 19.0\n",
      "Score: 14.0\n",
      "Score: 18.0\n",
      "Score: 20.0\n",
      "Score: 23.0\n",
      "Score: 12.0\n",
      "Score: 17.0\n",
      "Score: 18.0\n",
      "Score: 19.0\n",
      "Score: 17.0\n",
      "Score: 19.0\n",
      "Score: 20.0\n",
      "Score: 16.0\n",
      "Score: 23.0\n",
      "Score: 16.0\n",
      "Score: 21.0\n",
      "Score: 13.0\n",
      "Score: 18.0\n",
      "Score: 12.0\n",
      "Score: 22.0\n",
      "Score: 11.0\n",
      "Score: 24.0\n",
      "Score: 14.0\n",
      "Score: 17.0\n",
      "Score: 20.0\n",
      "Score: 17.0\n",
      "Score: 18.0\n",
      "Score: 10.0\n",
      "Score: 19.0\n",
      "Score: 23.0\n",
      "Score: 17.0\n",
      "Score: 18.0\n",
      "Score: 19.0\n",
      "Score: 23.0\n",
      "Score: 20.0\n",
      "Score: 25.0\n",
      "Score: 14.0\n",
      "Score: 21.0\n",
      "Score: 24.0\n",
      "Score: 24.0\n",
      "Score: 18.0\n",
      "Score: 24.0\n",
      "Score: 21.0\n",
      "Score: 23.0\n",
      "Score: 25.0\n",
      "Score: 20.0\n",
      "Score: 21.0\n",
      "Score: 23.0\n",
      "Score: 18.0\n",
      "Score: 16.0\n",
      "Score: 17.0\n",
      "Score: 21.0\n",
      "Score: 18.0\n",
      "Score: 15.0\n",
      "Score: 18.0\n",
      "Score: 11.0\n",
      "Score: 19.0\n",
      "Score: 21.0\n",
      "Score: 18.0\n",
      "Score: 17.0\n",
      "Score: 16.0\n",
      "Score: 20.0\n",
      "Score: 21.0\n",
      "Score: 15.0\n",
      "Score: 18.0\n",
      "Score: 23.0\n",
      "Score: 23.0\n",
      "Score: 26.0\n",
      "Score: 16.0\n",
      "Score: 25.0\n",
      "Score: 20.0\n",
      "Score: 13.0\n",
      "Score: 16.0\n",
      "Score: 18.0\n",
      "Score: 19.0\n",
      "Score: 19.0\n",
      "Score: 15.0\n",
      "Score: 18.0\n",
      "Score: 24.0\n",
      "Score: 15.0\n",
      "Score: 21.0\n",
      "Score: 18.0\n",
      "Score: 22.0\n",
      "Score: 22.0\n",
      "Score: 20.0\n",
      "Score: 19.0\n",
      "Average Score: 18.62\n"
     ]
    }
   ],
   "source": [
    "load_and_run_agent(agent, env, 'checkpoint_final_buffer1e6_every4_tau.pth', 100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Score: 15.0\n",
      "Score: 7.0\n",
      "Score: 18.0\n",
      "Score: 13.0\n",
      "Score: 19.0\n",
      "Score: 15.0\n",
      "Score: 13.0\n",
      "Score: 23.0\n",
      "Score: 17.0\n",
      "Score: 17.0\n",
      "Score: 14.0\n",
      "Score: 18.0\n",
      "Score: 19.0\n",
      "Score: 20.0\n",
      "Score: 16.0\n",
      "Score: 17.0\n",
      "Score: 10.0\n",
      "Score: 19.0\n",
      "Score: 11.0\n",
      "Score: 22.0\n",
      "Score: 17.0\n",
      "Score: 24.0\n",
      "Score: 21.0\n",
      "Score: 7.0\n",
      "Score: 21.0\n",
      "Score: 15.0\n",
      "Score: 14.0\n",
      "Score: 0.0\n",
      "Score: 23.0\n",
      "Score: 13.0\n",
      "Score: 18.0\n",
      "Score: 17.0\n",
      "Score: 18.0\n",
      "Score: 16.0\n",
      "Score: 21.0\n",
      "Score: 20.0\n",
      "Score: 17.0\n",
      "Score: 20.0\n",
      "Score: 19.0\n",
      "Score: 15.0\n",
      "Score: 29.0\n",
      "Score: 13.0\n",
      "Score: 12.0\n",
      "Score: 21.0\n",
      "Score: 21.0\n",
      "Score: 21.0\n",
      "Score: 18.0\n",
      "Score: 16.0\n",
      "Score: 21.0\n",
      "Score: 23.0\n",
      "Score: 19.0\n",
      "Score: 16.0\n",
      "Score: 16.0\n",
      "Score: 17.0\n",
      "Score: 20.0\n",
      "Score: 14.0\n",
      "Score: 17.0\n",
      "Score: 17.0\n",
      "Score: 18.0\n",
      "Score: 20.0\n",
      "Score: 25.0\n",
      "Score: 17.0\n",
      "Score: 18.0\n",
      "Score: 15.0\n",
      "Score: 20.0\n",
      "Score: 18.0\n",
      "Score: 19.0\n",
      "Score: 15.0\n",
      "Score: 17.0\n",
      "Score: 17.0\n",
      "Score: 21.0\n",
      "Score: 17.0\n",
      "Score: 17.0\n",
      "Score: 20.0\n",
      "Score: 16.0\n",
      "Score: 8.0\n",
      "Score: 20.0\n",
      "Score: 16.0\n",
      "Score: 17.0\n",
      "Score: 14.0\n",
      "Score: 18.0\n",
      "Score: 23.0\n",
      "Score: 16.0\n",
      "Score: 19.0\n",
      "Score: 20.0\n",
      "Score: 3.0\n",
      "Score: 12.0\n",
      "Score: 9.0\n",
      "Score: 21.0\n",
      "Score: 13.0\n",
      "Score: 16.0\n",
      "Score: 22.0\n",
      "Score: 21.0\n",
      "Score: 5.0\n",
      "Score: 14.0\n",
      "Score: 18.0\n",
      "Score: 17.0\n",
      "Score: 17.0\n",
      "Score: 25.0\n",
      "Score: 15.0\n",
      "Average Score: 16.99\n"
     ]
    }
   ],
   "source": [
    "load_and_run_agent(agent, env, 'checkpoint_final_buffer1e6_every4_tau_prioritized.pth', 100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When finished, you can close the environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "ERROR:root:Exception calling application: \n",
      "Traceback (most recent call last):\n",
      "  File \"C:\\Users\\Paperspace\\AppData\\Local\\conda\\conda\\envs\\drlnd\\lib\\multiprocessing\\connection.py\", line 312, in _recv_bytes\n",
      "    nread, err = ov.GetOverlappedResult(True)\n",
      "BrokenPipeError: [WinError 109] The pipe has been ended\n",
      "\n",
      "During handling of the above exception, another exception occurred:\n",
      "\n",
      "Traceback (most recent call last):\n",
      "  File \"C:\\Users\\Paperspace\\AppData\\Local\\conda\\conda\\envs\\drlnd\\lib\\site-packages\\grpc\\_server.py\", line 385, in _call_behavior\n",
      "    return behavior(argument, context), True\n",
      "  File \"C:\\Users\\Paperspace\\AppData\\Local\\conda\\conda\\envs\\drlnd\\lib\\site-packages\\unityagents\\rpc_communicator.py\", line 27, in Exchange\n",
      "    return self.child_conn.recv()\n",
      "  File \"C:\\Users\\Paperspace\\AppData\\Local\\conda\\conda\\envs\\drlnd\\lib\\multiprocessing\\connection.py\", line 250, in recv\n",
      "    buf = self._recv_bytes()\n",
      "  File \"C:\\Users\\Paperspace\\AppData\\Local\\conda\\conda\\envs\\drlnd\\lib\\multiprocessing\\connection.py\", line 321, in _recv_bytes\n",
      "    raise EOFError\n",
      "EOFError\n"
     ]
    }
   ],
   "source": [
    "env.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "drlnd",
   "language": "python",
   "name": "drlnd"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}