getting_started/09_EnvironmentModifications.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Some powerline can be disconnected, and there is ~~nothing~~ something you can do about it\n",
    "Try this notebook out interactively with: [![Binder](./img/badge_logo.svg)](https://mybinder.org/v2/gh/rte-france/Grid2Op/master)\n",
    "\n",
    "<img src=\"https://colab.research.google.com/assets/colab-badge.svg\" width=\"200\">\n",
    "Execute the cell below by removing the # character if you use google colab !\n",
    "\n",
    "Cell will look like:\n",
    "```python\n",
    "!pip install grid2op[optional]  # for use with google colab (grid2Op is not installed by default)\n",
    "```\n",
    "<img src=\"https://colab.research.google.com/assets/colab-badge.svg\" width=\"200\">\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# !pip install grid2op[optional]  # for use with google colab (grid2Op is not installed by default)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To model the reality of powersystem operation and planning grid2op offers the possibility, when setting up an environment, to include powerline disconnection.\n",
    "\n",
    "This is a really important part of the power system decision process. Indeed, when a powerline is being built it is often for more than a few decades (in some TSOs, some equipment was built and is still in service since almost 100 years ago!). Therefore, the **maintenance** of such equipment is important to model properly. In the case of grid2op, the **maitnenance** is modeled as a disconnection of a powerline and to block to reconnect this line for a given timespan. As this operation is planned you are given the relevant information in the observation:\n",
    "- [`obs.time_next_maintenance`](https://grid2op.readthedocs.io/en/master/observation.html#grid2op.Observation.BaseObservation.time_next_maintenance) will give you, for each powerline, in how many time steps this powerline will be disconnected from the grid\n",
    "- [`obs.duration_next_maintenance`](https://grid2op.readthedocs.io/en/master/observation.html#grid2op.Observation.BaseObservation.duration_next_maintenance) will give you, for each powerline, for how long (in number of time steps) this powerline will stay disconnected\n",
    "\n",
    "The other important property of the powergrid is that they cover large areas, usually, they are the side of a whole country/state, for example, counting hundreds of thousands of \"equipment\" (varying in size from the tiny screws to fix two things together to wide transformers of the size of a building...). The probability, in these conditions, that any of this equipment fails is far from neglectable. For the record, the \"failure\" of equipment can have dozens of possible causes; for instance, it can be in a bad shape (not enough maintenance), it suffers from natural external aggression (wind storm, struck by lightning bolts, a tree falls on it, the external temperature is too hot causing some thermal issues, etc.), be the victim of a malicious attack (a person hack a piece of software or physically attack the equipment with a bulldozer or a bombing for example), or it is not operated outside of its standard operation range (human error, bad data are sent to the control center, etc.), or any other cause you can imagine. However, most of these possible causes have a common consequence which is that one or more powerlines will be disconnected from the grid. This is what we call **hazards** in the grid2op framework. Compared to maintenance, **hazards** are not predictable (nothing, before it happens, indicates that there will be such hazards in the next few time steps).\n",
    "\n",
    "Actually, in our framework, we also added the capability to explicitly model attacks on the grid. We call this feature **Opponent**. For now, we model the opponent as an entity that can take, at each time step, the decision to disconnect a powerline. It has the same effect as the **hazards** (you know what it will do before it is happening) but is different in the following sense:\n",
    "\n",
    "- **hazards** will refer to random unpredictable events (let's call it *natural failures*), they are, by essence, random\n",
    "- **attack** (the action performed by an Opponent) will refer to possibly adversarial attacks, targeting equipment (not necessarily at random).\n",
    "\n",
    "\n",
    "In both cases, you can know when a **maintenance**, a **hazard**, or an **attack** is happening if you look at the [`obs.time_before_cooldown_line`](https://grid2op.readthedocs.io/en/master/observation.html#grid2op.Observation.BaseObservation.time_before_cooldown_line) vector. It gives you, for each powerline, in how many time steps at **minimum** you need to wait before being able to reconnect a powerline. If this is 0 you can change its status. If this > 0 you need to wait for the appropriate number of time steps.\n",
    "\n",
    "**NB** whether it's an attack, a hazard, or maintenance, the powerline will be automatic *disconnected*. They will stay *disconnected* until you reconnect them. This entails that if you don't reconnect them, they will remain disconnected."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Handling maintenance\n",
    "\n",
    "The following shows the effect of maintenances and how you can deal with them.\n",
    "\n",
    "We start by loading an environment (here for test purpose) and reset it. We will study the powerline with id `56` because... why not. It is the line that goes from substations 26 to substation 30."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import grid2op\n",
    "from grid2op.PlotGrid import PlotMatplot\n",
    "import numpy as np\n",
    "\n",
    "try:\n",
    "    from lightsim2grid import LightSimBackend\n",
    "    bk_cls = LightSimBackend\n",
    "except ImportError as exc:\n",
    "    print(f\"Error: {exc} when importing faster LightSimBackend\")\n",
    "    from grid2op.Backend import PandaPowerBackend\n",
    "    bk_cls = PandaPowerBackend\n",
    "\n",
    "\n",
    "env = grid2op.make(\"l2rpn_wcci_2020\", difficulty=\"0\", test=True, backend=bk_cls())\n",
    "plot_helper = PlotMatplot(env.observation_space)\n",
    "env.seed(0)  # to get reproducible results\n",
    "env.set_id(0)  # indicate to always read the same chronics, the first one\n",
    "obs = env.reset()\n",
    "do_nothing_action = env.action_space()\n",
    "line_id = 56"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"The powerline id {} will be in maintenance in {} time steps. This maintenance will last for {} \"\\\n",
    "      \"consecutive timesteps\".format(line_id, obs.time_next_maintenance[line_id], obs.duration_next_maintenance[line_id]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now skip a few time steps and define some action usefull for the next cells."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# first maintenance at 1260 (number of time step)\n",
    "start_maintenance = 1259  \n",
    "\n",
    "# define the action to reconnect it\n",
    "reconnect_action = env.action_space({\"set_line_status\": [(line_id, +1)]})\n",
    "\n",
    "# we skip a few time step, until the maintenance occur\n",
    "env.fast_forward_chronics(start_maintenance)\n",
    "# we get the observation at this very time steps\n",
    "obs = env.get_obs()\n",
    "print(\"Can I act on the powerline {} ? {}\".format(line_id, obs.time_before_cooldown_line[line_id] == 0))\n",
    "print(\"Is the powerline connected: {}\".format(obs.line_status[line_id]))\n",
    "print(\"The powerline id {} will be in maintenance in {} time steps. This maintenance will last for {} \"\\\n",
    "      \"consecutive timesteps\".format(line_id,\n",
    "                                     obs.time_next_maintenance[line_id],\n",
    "                                     obs.duration_next_maintenance[line_id]))\n",
    "_ = plot_helper.plot_obs(obs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you see, just before the maintenance is planned, the powerline is still connected (it is located in the top right corner of the graph, connect substation 26 to 30)\n",
    "\n",
    "Now, let's see what happens in the first timestep where the powerline is in maintenance:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# one step pass\n",
    "obs, reward, done, info = env.step(do_nothing_action)\n",
    "print(\"Can I act on the powerline? {}\".format(obs.time_before_cooldown_line[line_id] == 0))\n",
    "print(\"In how many time steps will I be able to reconnect it? {}\".format(obs.time_before_cooldown_line[line_id]))\n",
    "print(\"Is the powerline connected? {}\".format(obs.line_status[line_id]))\n",
    "_ = plot_helper.plot_obs(obs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now Tadaaa! You can see that the powerline is disconnected automatically (the powerline is dashed and black). By the way, this disconnection of the powerline has a very strong impact on the grid as it causes the overload of 2 other powerlines shown here in red (the from connecting substations 26 to 28 and the one connected substation 28 to 31).\n",
    "\n",
    "For academic purposes, let's see what is happening if I try to reconnect it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's try to reconnect the powerline\n",
    "obs, reward, done, info = env.step(reconnect_action)\n",
    "print(\"Can I act on the powerline? {}\".format(obs.time_before_cooldown_line[line_id] == 0))\n",
    "print(\"In how many time steps will I be able to reconnect it? {}\".format(obs.time_before_cooldown_line[line_id]))\n",
    "print(\"Is the powerline connected? {}\".format(obs.line_status[line_id]))\n",
    "_ = plot_helper.plot_obs(obs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unfortunately for us, this has absolutely no effect. The maintenance is still happening, the powerline status is locked until the maintenance is over (in 95 time steps)\n",
    "\n",
    "Let's then do nothing for 95 remaining time step (I have to wait for 95 timesteps before i am able to reconnect it.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for _ in range(95):\n",
    "    obs, reward, done, info = env.step(do_nothing_action)\n",
    "print(\"Can I act on the powerline? {}\".format(obs.time_before_cooldown_line[line_id] == 0))\n",
    "print(\"In how many time steps will I be able to reconnect it? {}\".format(obs.time_before_cooldown_line[line_id]))\n",
    "print(\"Is the powerline connected? {}\".format(obs.line_status[line_id]))\n",
    "print(\"The flow on it is {:.1f}A (of course because it is switched off)\".format(obs.a_or[line_id]))\n",
    "_ = plot_helper.plot_obs(obs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, after the 95 time steps, the powerline can be reconnected (information printed), but that does not mean it is reconnected. Actually, it is not reconnected until you do it. \n",
    "\n",
    "For example, we can do a few more steps, and you will see the powerline still has not to be reconnected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "obs, reward, done, info = env.step(do_nothing_action)\n",
    "_ = plot_helper.plot_obs(obs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But now (we could have done that the preceeding time steps though) the powerline can be reconnected. Let us do it and see the resulting grid."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# and now reconnect it\n",
    "obs, reward, done, info = env.step(reconnect_action)\n",
    "print(\"Can I act on the powerline? {}\".format(obs.time_before_cooldown_line[line_id] == 0))\n",
    "print(\"In how many time steps will I be able to reconnect it? {}\".format(obs.time_before_cooldown_line[line_id]))\n",
    "print(\"Is the powerline connected? {}\".format(obs.line_status[line_id]))\n",
    "print(\"The flow on it is {:.1f}A\".format(obs.a_or[line_id]))\n",
    "_ = plot_helper.plot_obs(obs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**SUCCESS**: the powerline has been properly reconnected, and everything on this part of the grid is now in order.\n",
    "\n",
    "# Hazards / Opponent\n",
    "\n",
    "In some settings you also have a \"purely stochastic\" component directly embeded into the environment. It is called an \"Opponent\" and will disconect (at random most of the time) some powerlines."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import grid2op\n",
    "from grid2op.PlotGrid import PlotMatplot\n",
    "from grid2op.Parameters import Parameters\n",
    "# i will disable the powerline disconnection in case of overflow\n",
    "param = Parameters()\n",
    "param.NO_OVERFLOW_DISCONNECTION = True\n",
    "env_name2 = \"rte_case14_opponent\"\n",
    "env_opp = grid2op.make(env_name2, test=True, param=param, backend=bk_cls())\n",
    "plot_helper_opp = PlotMatplot(env_opp.observation_space)\n",
    "line_id_opp = 3\n",
    "reconnect_action_opp = env_opp.action_space({\"set_line_status\": [(line_id_opp, +1)]})\n",
    "do_nothing_opp = env_opp.action_space()\n",
    "env_opp.seed(0)  # make sure i have reproducible experiments\n",
    "obs = env_opp.reset()\n",
    "_ = plot_helper_opp.plot_obs(obs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We know (but that is because we set the seed and because we coded this environment) that an attack will happen at time step 96 so we do_nothing until that time. And, to be transparent, we show the state of the powergrid."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(95):\n",
    "    obs, reward, done, info = env_opp.step(do_nothing_opp)\n",
    "print(\"The next maintenance is schedule in {} time steps (-1 = never)\"\\\n",
    "      \"\".format(obs.time_next_maintenance[line_id_opp]))\n",
    "_ = plot_helper_opp.plot_obs(obs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's do another step. Note that not maintenance are planned for the forseeable future, yet a powerline will be disconnected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "obs, reward, done, info = env_opp.step(do_nothing_opp)\n",
    "_ = plot_helper_opp.plot_obs(obs)\n",
    "print(\"The next maintenance is schedule in {} time steps (-1 = never)\"\\\n",
    "      \"\".format(obs.time_next_maintenance[line_id_opp]))\n",
    "print(\"The flow on this powerline is {:.1f}%\"\\\n",
    "      \"\".format(100*obs.rho[line_id_opp]))\n",
    "print(\"This powerline is unavailable for {} time steps\".format(obs.time_before_cooldown_line[line_id_opp]))\n",
    "print(\"I can also spot an attack by looking at the \\\"info\\\" dictionnary, that tells me that an attack is taking \" \\\n",
    "      \"place on powerline: {}, and this attack will last {} time steps (in total, it started this time step \"\\\n",
    "      \"so it will be over in 47 = 48 - 1 time steps).\" \\\n",
    "      \"\".format(np.where(info[\"opponent_attack_line\"])[0], info[\"opponent_attack_duration\"]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have been victim of an \"attack\" on the grid. The powerline 3, connecting substation 1 to substation 3 is now out of service for 47 time steps. As it was the case for maintenance, any attempt to reconnect it will fail, as shown in the image bellow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "obs, reward, done, info = env_opp.step(do_nothing_opp)\n",
    "_ = plot_helper_opp.plot_obs(obs)\n",
    "print(\"The powerline will be unavailble for again {} time steps.\"\\\n",
    "      \"\".format(obs.time_before_cooldown_line[line_id_opp]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is nothing really interesting in here, so we will do nothing for 46 time steps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(46):\n",
    "    obs, reward, done, info = env_opp.step(do_nothing_opp)\n",
    "_ = plot_helper_opp.plot_obs(obs)\n",
    "print(\"The next maintenance is schedule in {} time steps (-1 = never)\"\\\n",
    "      \"\".format(obs.time_next_maintenance[line_id_opp]))\n",
    "print(\"The powerline will be unavailble for again {} time steps.\"\\\n",
    "      \"\".format(obs.time_before_cooldown_line[line_id_opp]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As for the maintenance, we see here that the powerline can be reconnected (`obs.time_before_cooldown_line[line_id_opp] = 0`), but it has not been reconnected automatically (it is still disconnected). But hopefully, now we know how to reconnect them, what we'll do in the next, concluding this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "obs, reward, done, info = env_opp.step(reconnect_action_opp)\n",
    "_ = plot_helper_opp.plot_obs(obs)\n",
    "print(\"The powerline will be unavailble for again {} time steps.\"\\\n",
    "      \"\".format(obs.time_before_cooldown_line[line_id_opp]))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}