getting_started/01_Grid2opFramework.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# This notebook present the most basic use of Grid2Op\n",
    "Try me out interactively with: [![Binder](./img/badge_logo.svg)](https://mybinder.org/v2/gh/rte-france/Grid2Op/master)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Objectives**\n",
    "\n",
    "This notebook will cover some basic raw functionalities at first. It will then show how these raw functionalities are encapsulated in easy-to-use functions.\n",
    "\n",
    "As we will see later, the recommended way to use these functionalities is to go through the Runner, and not through the instanciation of the different classes one after the other."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://colab.research.google.com/assets/colab-badge.svg\" width=\"200\">\n",
    "Execute the cell below by removing the # character if you use google colab !\n",
    "\n",
    "Cell will look like:\n",
    "```python\n",
    "!pip install grid2op[optional]  # for use with google colab (grid2Op is not installed by default)\n",
    "```\n",
    "<img src=\"https://colab.research.google.com/assets/colab-badge.svg\" width=\"200\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install grid2op[optional]  # for use with google colab (grid2Op is not installed by default)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import sys\n",
    "import grid2op"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "res = None\n",
    "try:\n",
    "    from jyquickhelper import add_notebook_menu\n",
    "    res = add_notebook_menu()\n",
    "except ModuleNotFoundError:\n",
    "    print(\"Impossible to automatically add a menu / table of content to this notebook.\\nYou can download \\\"jyquickhelper\\\" package with: \\n\\\"pip install jyquickhelper\\\"\")\n",
    "res"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## I) Creating an Environment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### I.A) Default settings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We provide a function that will handle the creation of the Environment with default values in a single call of the function.\n",
    "\n",
    "In this example we will use the `rte_case14_redisp` environment, in test mode.\n",
    "\n",
    "To define/create it, we can call:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "env = grid2op.make(\"rte_case14_redisp\", test=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NB** By setting \"test=True\" in the above call, we only use the data for 2 different months for our environment. If you remove it, grid2op.make will attempt to download more data. By default, the data corresponding to this environment will be downloaded to your \"home\" directory, which corresponds to the location returned by this script:\n",
    "\n",
    "```python\n",
    "print(f\"grid2op dataset will be downloaded in \\\"{grid2op.get_current_local_dir()}\\\"\")\n",
    "```\n",
    "\n",
    "Other environments can be used and are available through the \"make\" command. To get a list of the available environments, you can do:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grid2op.list_available_remote_env()  # this only works if you have an internet connection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also possible to list the environments that you have already downloaded (if any). \n",
    "\n",
    "**NB** : Downloading is automatic and is done the first time you call `make` with an environment that has not been already locally downloaded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grid2op.list_available_local_env()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For more customization on where the environment are located (*e.g.* changing the `grid2op.get_current_local_dir()` the documentation provides extra information at https://grid2op.readthedocs.io/en/latest/makeenv.html#cache-manipulation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### I.B) Custom settings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the `make` function, you can pass additional arguments to customize the environment (this is useful for training):\n",
    " - `param`: The parameters used for the Environment. See `grid2op.Parameters.Parameters`.\n",
    " - `backend` : The backend to use for the computation. If provided, it must be an instance of the class `grid2op.Backend.Backend`.\n",
    " - `action_class`: The type of BaseAction that the BaseAgent will be able to perform. If provided, it must be a subclass of `grid2op.BaseAction.BaseAction`.\n",
    " - `observation_class`: The type of BaseObservation that the BaseAgent will receive. If provided, It must be a subclass of `grid2op.BaseAction.BaseObservation`.\n",
    " - `reward_class`: The type of reward signal that the BaseAgent will receive. If provided, It must be a subclass of `grid2op.BaseReward.BaseReward`.\n",
    " - `gamerules_class`: The type of \"Rules\" that the BaseAgent will need to comply with. Rules are here to model some operational constraints. If provided, it must be a subclass of `grid2op.RulesChecker.BaseRules`.\n",
    " - `data_feeding_kwargs`: A dictionnary that is used to build the `data_feeding` (chronics) objects.\n",
    " - `chronics_class`: The type of chronics that represents the dynamics of the created Environment. Usually they come from different folders.\n",
    " - `data_feeding`: The type of chronics handler you want to use.\n",
    " - `volagecontroler_class`: The type of `grid2op.VoltageControler.VoltageControler` to use.\n",
    " - `chronics_path`: The path where to look for the chronics dataset (optional).\n",
    " - `grid_path`: The path where the powergrid is located. If provided, it must be a string and point to a valid file on the hard drive.\n",
    " \n",
    "For example, to set the number of maximum allowed substation changes per step:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from grid2op.Parameters import Parameters\n",
    "\n",
    "custom_params = Parameters()\n",
    "custom_params.MAX_SUB_CHANGED = 1\n",
    "env = grid2op.make(\"l2rpn_case14_sandbox\", param=custom_params, test=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NB** The `make` function is highly customizable. For example, you can change the reward that you are using:\n",
    "\n",
    "```python\n",
    "from grid2op.Reward import L2RPNReward\n",
    "env = grid2op.make(reward_class=L2RPNReward)\n",
    "```\n",
    "\n",
    "We also give the possibility to assess different rewards. This can be done with the following code:\n",
    "\n",
    "```python\n",
    "\n",
    "from grid2op.Reward import L2RPNReward, FlatReward\n",
    "env = grid2op.make(reward_class=L2RPNReward,\n",
    "                   other_rewards={\"other_reward\" : FlatReward })\n",
    "```\n",
    "The results for these rewards can be accessed with the `info` object returned by `env.step` (`info` is the 4th object returned by `env.step`, as you can see below). See the official reward documentation [here](https://grid2op.readthedocs.io/en/latest/reward.html) for more information."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## II) Creating an Agent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An *Agent* is the name given to the \"operator\" / \"bot\" / \"algorithm\" that will perform some modifications of the powergrid when it faces some \"observation\".\n",
    "\n",
    "Examples of Agents are provided in the grid2Op/Agent directory of the grid2Op code repository.\n",
    "\n",
    "A deeper look at the different provided Agents can be found in the [05_StudyYourAgent](05_StudyYourAgent.ipynb) notebook. We suppose here that we use the most simple Agent, the one that does nothing (`DoNothingAgent`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from grid2op.Agent import DoNothingAgent\n",
    "my_agent = DoNothingAgent(env.action_space)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## III) Assess how the Agent is performing (manually)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The performance of an Agent is assessed with the cumulated reward it receives over time. In this example, the cumulated reward is a *FlatReward* that simply computes how many time steps the *Agent* has sucessfully managed before breaking any rules. For more control over this reward, it is recommended to use look at the documentation of the Environment class.\n",
    "\n",
    "More examples of rewards are also available on the official documentation or [here](https://grid2op.readthedocs.io/en/latest/reward.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "done = False\n",
    "time_step = int(0)\n",
    "cum_reward = 0.\n",
    "obs = env.reset()\n",
    "reward = env.reward_range[0]\n",
    "max_iter = 10\n",
    "while not done:\n",
    "    act = my_agent.act(obs, reward, done) # chose an action to do, in this case \"do nothing\"\n",
    "    obs, reward, done, info = env.step(act) # implement this action on the powergrid\n",
    "    cum_reward += reward\n",
    "    time_step += 1\n",
    "    if time_step >= max_iter:\n",
    "        break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now evaluate how well this *agent* is performing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"This agent managed to survive {} timesteps\".format(time_step))\n",
    "print(\"It's final cumulated reward is {}\".format(cum_reward))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## IV) More convenient ways to assess the performance of an agent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All the steps above have been detailed as a \"quick start\", to give an example of the main classes of the Grid2Op package. Having to code all of the above can be quite tedious, but offers a lot of flexibility.\n",
    "\n",
    "Implementing all this before starting to evaluate an agent can be tiring. What we show here is a much shorter way to perfom all this. In this section we will exhibit 2 ways:\n",
    "* The quickest way, using the grid2op.main API, most suited when basic computations need to be carried out.\n",
    "* The recommended way using a *Runner*. This gives more flexibility than the grid2op.main API but can be harder to configure.\n",
    "\n",
    "In this section, we assume the same as before:\n",
    "* The Agent is the \"Do Nothing\" agent\n",
    "* The Environment is the default Environment\n",
    "* PandaPower is used as the backend\n",
    "* The chronics comes from the files included in this package\n",
    "* etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### IV.A) Using the grid2op.runner API"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When only simple assessments need to be performed, the grid2op.main API is perfectly suited. This API can also be accessed with the command line:\n",
    "```bash\n",
    "python3 -m grid2op.main\n",
    "```\n",
    "\n",
    "We detail here its usage as an API, to assess the performance of a given Agent.\n",
    "\n",
    "As opposed to building en environment from scratch (see the previous section), this requires much less effort: we don't need to initialize (instanciate) anything. Everything is carried out inside the Runner called by the `main` function.\n",
    "\n",
    "We simulate 1 episode here (eg. we play one scenario until: either the agent does a game over, or the scenario ends), but this method would work too if we wanted to simulate more episodes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from grid2op.Runner import Runner\n",
    "runner = Runner(**env.get_params_for_runner(), agentClass=DoNothingAgent)\n",
    "res = runner.run(nb_episode=1, max_iter=max_iter)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A call of the single 2 lines above will:\n",
    "* Create a valid environment\n",
    "* Create a valid agent\n",
    "* Assess how well an agent performs on one episode."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"The results are:\")\n",
    "for chron_name, _, cum_reward, nb_time_step, max_ts in res:\n",
    "    msg_tmp = \"\\tFor chronics located at {}\\n\".format(chron_name)\n",
    "    msg_tmp += \"\\t\\t - cumulative reward: {:.2f}\\n\".format(cum_reward)\n",
    "    msg_tmp += \"\\t\\t - number of time steps completed: {:.0f} / {:.0f}\".format(nb_time_step, max_ts)\n",
    "    print(msg_tmp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is particularly suited for evaluating different agents. For example, we can quickly evaluate a second agent. In the example below, we can import an agent class *PowerLineSwitch* whose job is to connect and disconnect the power lines in the power network. This *PowerLineSwitch* Agent will **simulate** the effect of disconnecting a powerline for each powerline in the powergrid, and take the best action found ie the one whose simulated effect is the best (its execution can take a long time, depending on the scenario and the amount of powerlines in the grid). **The execution of the code below can take a little time**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from grid2op.Agent import PowerLineSwitch\n",
    "runner = Runner(**env.get_params_for_runner(), agentClass=PowerLineSwitch)\n",
    "res = runner.run(nb_episode=1, max_iter=max_iter)\n",
    "print(\"The results are:\")\n",
    "for chron_name, _, cum_reward, nb_time_step, max_ts in res:\n",
    "    msg_tmp = \"\\tFor chronics located at {}\\n\".format(chron_name)\n",
    "    msg_tmp += \"\\t\\t - cumulative reward: {:.2f}\\n\".format(cum_reward)\n",
    "    msg_tmp += \"\\t\\t - number of time steps completed: {:.0f} / {:.0f}\".format(nb_time_step, max_ts)\n",
    "    print(msg_tmp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also possible using this API to store the results for a detailed examination of the actions taken by the Agent. Note that writing on the hard drive has an overhead on the computation time.\n",
    "\n",
    "To do this, only a simple argument needs to be added to the *main* function call (`path_save`, which indicates where the outcome of the experiment will be stored). An example can be found below :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "runner = Runner(**env.get_params_for_runner(),\n",
    "                agentClass=PowerLineSwitch\n",
    "               )\n",
    "path_save_expe = os.path.abspath(\"saved_experiment_donothing\")\n",
    "res = runner.run(nb_episode=1, max_iter=max_iter, path_save=path_save_expe)\n",
    "print(\"The results are:\")\n",
    "for chron_name, _, cum_reward, nb_time_step, max_ts in res:\n",
    "    msg_tmp = \"\\tFor chronics located at {}\\n\".format(chron_name)\n",
    "    msg_tmp += \"\\t\\t - cumulative reward: {:.2f}\\n\".format(cum_reward)\n",
    "    msg_tmp += \"\\t\\t - number of time steps completed: {:.0f} / {:.0f}\".format(nb_time_step, max_ts)\n",
    "    print(msg_tmp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "os.listdir(os.path.join(path_save_expe, \"0\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All the outcomes of the experiment are shown above. For more information, please don't hesitate to read the documentation of [Runner](https://grid2op.readthedocs.io/en/latest/runner.html).\n",
    "\n",
    "**NB**: A lot more of information about *Actions* is provided in the [(03_Action](03_Action.ipynb) notebook. In the [04_TrainingAnAgent](04_TrainingAnAgent.ipynb) (last section), there is an quick example of how to read / write an action from a saved repository.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using `make` and `Runner` makes it easy to assess the performance of a trained agent. Besides, the `Runner` has been particularly integrated with other tools and makes it easy to replay and analyse an episode after it is finished. It is the recommended method to use in grid2op for the evaluation."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}