lisflood-notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "exclusive-history",
   "metadata": {},
   "source": [
    "<p><center> <img src=\"doc/header.png\" width=\"1000\"/> </p></center>\n",
    "\n",
    "# LISFLOOD - a distributed hydrological rainfall-runoff model\n",
    "\n",
    "LISFLOOD is a spatially distributed, semi-physical hydrological rainfall-runoff model that has been developed by the Joint Research Centre (JRC) of the European Commission in late 90ies. Since then LISFLOOD has been applied to a wide range of applications such as all kind of water resourses assessments looking at e.g. the effects of climate and land-use change as well as river regulation measures. Its most prominent application is probably within the [European Flood Awareness System](https://www.efas.eu/) (EFAS) operated under [Copernicus Emergency Management Service](https://emergency.copernicus.eu/) (EMS).\n",
    "\n",
    "It’s wide applicability is due to its modular structure as well as its temporal and spatial flexibility. The model can be extended with additional modules when need arises, to satisfy the new target objective. In that sense it can be extended to include anything from a better representation of a particular hydrological flow to the implementation of anthropogenic-influenced processes. At the same time the model has been designed to be applied across a wide range of spatial and temporal scales. LISFLOOD is grid-based, and applications so far have employed grid cells of as little as 100 metres for medium-sized catchments, to 5000 metres for modelling the whole of Europe and up to 0.1° (around 10 km) for modelling globally. Long-term water balance can be simulated (using a daily time step), as well as individual flood events (using hourly time intervals, or even smaller).\n",
    "\n",
    "Although LISFLOOD’s primary output product is channel discharge, all internal rate and state variables (soil moisture, for example) can be written as output as well. All output can be written as grids, or time series at user-defined points or areas. The user has complete control over how output is written, thus minimising any waste of disk space or CPU time.\n",
    "\n",
    "Go to [Lisflood OS page](https://ec-jrc.github.io/lisflood/) for more information. Other useful resources:\n",
    "\n",
    "| **Project** | **Documentation** | **Source code** |\n",
    "| :--- | :--- | :--- |\n",
    "| Lisflood | [Model docs](https://ec-jrc.github.io/lisflood-model/) | https://github.com/ec-jrc/lisflood-code (this repository) |\n",
    "| | [User guide](https://ec-jrc.github.io/lisflood-code/) | |\n",
    "| Lisvap | [Docs](https://ec-jrc.github.io/lisflood-lisvap/) | https://github.com/ec-jrc/lisflood-lisvap |\n",
    "| Calibration tool | [Docs](https://ec-jrc.github.io/lisflood-calibration/) | https://github.com/ec-jrc/lisflood-calibration |\n",
    "| Lisflood Utilities | | https://github.com/ec-jrc/lisflood-utilities |\n",
    "| Lisflood Usecases | | https://github.com/ec-jrc/lisflood-usecases |\n",
    "\n",
    "\n",
    "## Initialization\n",
    "\n",
    "This section loads relevant Python modules for the following analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "headed-prayer",
   "metadata": {
    "code_folding": [
     0
    ]
   },
   "outputs": [],
   "source": [
    "# Click to EXPAND\n",
    "\n",
    "#####################################################\n",
    "###################### IMPORTS ######################\n",
    "#####################################################\n",
    "\n",
    "%matplotlib ipympl\n",
    "\n",
    "import datetime\n",
    "import xml.etree.ElementTree as ET\n",
    "\n",
    "import altair as alt\n",
    "import cartopy.crs as ccrs\n",
    "import ipyleaflet\n",
    "import ipywidgets\n",
    "import pandas as pd\n",
    "from ipyfilechooser import FileChooser\n",
    "from matplotlib import rc\n",
    "\n",
    "rc('animation', html='jshtml')\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import glob\n",
    "import os\n",
    "import netCDF4\n",
    "import cartopy.io.img_tiles as cimgt\n",
    "import functools\n",
    "import warnings"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "exclusive-faith",
   "metadata": {},
   "source": [
    "## Processing\n",
    "\n",
    "After running the following code cell, please choose the respective settings files. Subsequently, a variety of input parameters such as the simulation period, calibration parameters, and output variables can be determined. If the discharge time series output is requested, an interactive map allows selecting the desired location. When all inputs are set, please click *'Start Processing'* to start the simulation. All settings are then updated in the settings files. The outputs are saved in a folder called *'out'*, which is automatically created at the level of the settings files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "available-hamburg",
   "metadata": {
    "code_folding": [
     0,
     7,
     223,
     291,
     299
    ],
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Click to EXPAND\n",
    "\n",
    "####################################################\n",
    "############### FUNCTION DEFINITIONS ###############\n",
    "####################################################\n",
    "\n",
    "# callback function to show settings after XML files are chosen\n",
    "def show_settings(chooser):\n",
    "    # checks whether PRE-RUN and RUN settings files are given\n",
    "    if fc[0].selected != None and fc[1].selected != None:\n",
    "\n",
    "        global tree\n",
    "        global CalendarDayStart\n",
    "        global DtSec_xml\n",
    "        global DtSec_box\n",
    "        global StepStart\n",
    "        global StepStart_picker\n",
    "        global StepEnd\n",
    "        global StepEnd_picker\n",
    "        global calibration_parameters_xml\n",
    "        global calibration_sliders\n",
    "        global optional_modules_xml\n",
    "        global module_checkboxes\n",
    "        global m\n",
    "        global marker\n",
    "        global coordinates\n",
    "        \n",
    "        # create output folder if it does not exist\n",
    "        out_dir = os.path.join(fc[1].selected_path, \"out\")\n",
    "        if not os.path.exists(out_dir): os.makedirs(out_dir, exist_ok=True)\n",
    "\n",
    "        # opens settings file of PRE-RUN ([0]) and RUN ([1]) in list\n",
    "        tree = [ET.parse(fc[0].selected), ET.parse(fc[1].selected)]\n",
    "        # gets root of XML\n",
    "        root = [tree[0].getroot(), tree[1].getroot()]\n",
    "\n",
    "        # gets timestep and calendar day start from XML files\n",
    "        DtSec_xml = [root[0].find(\"./lfuser/group/textvar[@name='DtSec']\"),\n",
    "                     root[1].find(\"./lfuser/group/textvar[@name='DtSec']\")]\n",
    "        # creates text box for timestep\n",
    "        DtSec_box = ipywidgets.BoundedIntText(value=DtSec_xml[0].attrib['value'],\n",
    "                                                 min=1,\n",
    "                                                 max=31536000,\n",
    "                                                 step=60,\n",
    "                                                 description='Timestep [s]:',\n",
    "                                                 layout=ipywidgets.Layout(width='40%'),\n",
    "                                                 style={'description_width': '25ex'})\n",
    "        date_str = root[1].find(\"./lfuser/group/textvar[@name='CalendarDayStart']\").attrib['value'].split(' ')[0]\n",
    "        time_str = root[1].find(\"./lfuser/group/textvar[@name='CalendarDayStart']\").attrib['value'].split(' ')[1]\n",
    "        CalendarDayStart = datetime.datetime(year=int(date_str.split('/')[2]),\n",
    "                                             month=int(date_str.split('/')[1]),\n",
    "                                             day=int(date_str.split('/')[0]),\n",
    "                                             hour=int(time_str.split(':')[0]),\n",
    "                                             minute=int(time_str.split(':')[1]))\n",
    "        # finds simulation start date in XML files\n",
    "        StepStart = [root[0].find(\"./lfuser/group/textvar[@name='StepStart']\"),\n",
    "                     root[1].find(\"./lfuser/group/textvar[@name='StepStart']\")]\n",
    "        # converts date format of StepStart\n",
    "        StepStart_iso = [\n",
    "            datetime.datetime.strptime(StepStart[0].attrib['value'].split()[0], \"%d/%m/%Y\").strftime(\"%Y-%m-%d\"),\n",
    "            datetime.datetime.strptime(StepStart[1].attrib['value'].split()[0], \"%d/%m/%Y\").strftime(\"%Y-%m-%d\")]\n",
    "        # creates date picker widgets with date set to XML input\n",
    "        StepStart_picker = [ipywidgets.DatePicker(description='Pre Start Date',\n",
    "                                                  value=datetime.date.fromisoformat(StepStart_iso[0]),\n",
    "                                                  layout=ipywidgets.Layout(width='40%'),\n",
    "                                                  style={'description_width': '25ex'}),\n",
    "                            ipywidgets.DatePicker(description='Start Date',\n",
    "                                                  value=datetime.date.fromisoformat(StepStart_iso[1]),\n",
    "                                                  layout=ipywidgets.Layout(width='40%'),\n",
    "                                                  style={'description_width': '25ex'})]\n",
    "        # finds simulation end date in XML files\n",
    "        StepEnd = [root[0].find(\"./lfuser/group/textvar[@name='StepEnd']\"),\n",
    "                   root[1].find(\"./lfuser/group/textvar[@name='StepEnd']\")]\n",
    "        # converts date format of StepEnd\n",
    "        StepEnd_iso = [\n",
    "            datetime.datetime.strptime(StepEnd[0].attrib['value'].split()[0], \"%d/%m/%Y\").strftime(\"%Y-%m-%d\"),\n",
    "            datetime.datetime.strptime(StepEnd[1].attrib['value'].split()[0], \"%d/%m/%Y\").strftime(\"%Y-%m-%d\")]\n",
    "        # creates date picker widgets with date set to XML input\n",
    "        StepEnd_picker = [ipywidgets.DatePicker(description='Pre Stop Date',\n",
    "                                                value=datetime.date.fromisoformat(StepEnd_iso[0]),\n",
    "                                                layout=ipywidgets.Layout(width='40%'),\n",
    "                                                style={'description_width': '25ex'}),\n",
    "                          ipywidgets.DatePicker(description='Stop Date',\n",
    "                                                value=datetime.date.fromisoformat(StepEnd_iso[1]),\n",
    "                                                layout=ipywidgets.Layout(width='40%'),\n",
    "                                                style={'description_width': '25ex'})]\n",
    "\n",
    "\n",
    "        # creates empty list for slider widgets\n",
    "        calibration_sliders = {}\n",
    "        # finds calibration parameter group in XML files\n",
    "        calibration_parameters_xml = [root[0].find(\"./lfuser\")[2], root[1].find(\"./lfuser\")[2]]\n",
    "        # loops through calibration parameters listed in RUN settings file\n",
    "        for element in calibration_parameters_xml[1]:\n",
    "            # checks element type in XML file and allowed data range in parameter_specs dictionary\n",
    "            if (element.tag == 'textvar' and element.attrib['name'] in parameter_specs):\n",
    "                # creates slider widgets and appends to slider list\n",
    "                calibration_sliders[element.attrib['name']] = ipywidgets.HBox([\n",
    "                    ipywidgets.FloatSlider(value=element.attrib['value'],\n",
    "                                                 min=parameter_specs[element.attrib['name']][0],\n",
    "                                                 max=parameter_specs[element.attrib['name']][1],\n",
    "                                                 step=parameter_specs[element.attrib['name']][2],\n",
    "                                                 description=element.attrib['name'],\n",
    "                                                 disabled=False,\n",
    "                                                 continuous_update=False,\n",
    "                                                 orientation='horizontal',\n",
    "                                                 readout=True,\n",
    "                                                 readout_format=parameter_specs[element.attrib['name']][3],\n",
    "                                                 layout=ipywidgets.Layout(width='80%'),\n",
    "                                                 style={'description_width': '25ex'}),\n",
    "                    ipywidgets.Label(value=parameter_specs[element.attrib['name']][4])])\n",
    "\n",
    "        # creates empty list for checkbox widgets\n",
    "        module_checkboxes = {}\n",
    "        # finds optional modules group in XML files\n",
    "        optional_modules_xml = [root[0].find(\"./lfoptions\"), root[1].find(\"./lfoptions\")]\n",
    "        # loops through optional modules listed in RUN settings file\n",
    "        for element in optional_modules_xml[1]:\n",
    "            # creates checkbox widgets and appends to checkbox list\n",
    "            module_checkboxes[element.attrib['name']] = ipywidgets.Checkbox(\n",
    "                value=bool(int(element.attrib['choice'])),\n",
    "                description=element.attrib['name'],\n",
    "                disabled=False,\n",
    "                style={'description_width': '0ex'})\n",
    "\n",
    "        # reads coordinates from XML RUN file\n",
    "        coordinates = [root[0].findall(\"./lfuser/group/textvar/[@name='Gauges']\")[0],\n",
    "                       root[1].findall(\"./lfuser/group/textvar/[@name='Gauges']\")[0]]\n",
    "        lon, lat = coordinates[1].attrib['value'].split()\n",
    "        center = (lat, lon)\n",
    "        # creates interactive map with draggable marker\n",
    "        m = ipyleaflet.Map(zoom=10, center=center, scroll_wheel_zoom=True)\n",
    "        marker = ipyleaflet.Marker(location=center, draggable=True)\n",
    "        m.add_layer(marker)\n",
    "        # initializes visibility of map\n",
    "        if module_checkboxes['repDischargeTs'].value == True:\n",
    "            m.layout.display = \"block\"\n",
    "        else:\n",
    "            m.layout.display = \"none\"\n",
    "\n",
    "        # creates tab widget\n",
    "        tabs = ipywidgets.Tab()\n",
    "        # sets tab titles\n",
    "        tabs.set_title(0, 'Outputs')\n",
    "        tabs.set_title(1, 'Date Selection')\n",
    "        tabs.set_title(2, 'Calibration')\n",
    "        tabs.set_title(3, 'Optional Modules')\n",
    "\n",
    "        output_grid = ipywidgets.GridspecLayout(13, 3, height='auto')\n",
    "        output_grid[0:8, 0] = ipywidgets.VBox([ipywidgets.HTML(value='<b>Surface Water Outputs:</b>'),\n",
    "                                               module_checkboxes['repDischargeMaps'],\n",
    "                                               module_checkboxes['repDischargeTs'],\n",
    "                                               module_checkboxes['repsimulateReservoirs'],\n",
    "                                               module_checkboxes['repsimulateLakes'],\n",
    "                                               module_checkboxes['repSnowCoverMaps'],\n",
    "                                               module_checkboxes['repSnowMeltMaps'],\n",
    "                                               module_checkboxes['repSurfaceRunoffMaps']])\n",
    "        output_grid[0:9, 1] = ipywidgets.VBox([ipywidgets.HTML(value='<b>Soil Outputs:</b>'),\n",
    "                                               module_checkboxes['repThetaMaps'],\n",
    "                                               module_checkboxes['repThetaForestMaps'],\n",
    "                                               module_checkboxes['repThetaIrrigationMaps']])\n",
    "        output_grid[0:9, 2] = ipywidgets.VBox([ipywidgets.HTML(value='<b>Groundwater Outputs:</b>'),\n",
    "                                               module_checkboxes['repPFMaps'],\n",
    "                                               module_checkboxes['repPFForestMaps'],\n",
    "                                               module_checkboxes['repUZMaps'],\n",
    "                                               module_checkboxes['repLZMaps']])\n",
    "        output_grid[9:13, 0] = ipywidgets.VBox([ipywidgets.HTML(value='<b>Water Use Outputs:</b>'),\n",
    "                                                module_checkboxes['repTotalAbs'],\n",
    "                                                module_checkboxes['repTotalWUse']])\n",
    "        output_grid[9:13, 1] = ipywidgets.VBox([ipywidgets.HTML(value='<b>State and End Maps:</b>'),\n",
    "                                                module_checkboxes['repStateMaps'],\n",
    "                                                module_checkboxes['repEndMaps']])\n",
    "\n",
    "        # adds content to tabs\n",
    "        tabs.children = [ipywidgets.VBox([output_grid, m]),\n",
    "                         ipywidgets.VBox(\n",
    "                             [StepStart_picker[0], StepEnd_picker[0], StepStart_picker[1], StepEnd_picker[1],\n",
    "                              DtSec_box]),\n",
    "                         ipywidgets.VBox(list(calibration_sliders.values())),\n",
    "                         ipywidgets.VBox([ipywidgets.GridBox([module_checkboxes['InitLisflood'],\n",
    "                                                              module_checkboxes['InitLisfloodwithoutSplit'],\n",
    "                                                              module_checkboxes['gridSizeUserDefined'],\n",
    "                                                              module_checkboxes['SplitRouting'],\n",
    "                                                              module_checkboxes['inflow'],\n",
    "                                                              module_checkboxes['simulateReservoirs'],\n",
    "                                                              module_checkboxes['simulateLakes'],\n",
    "                                                              module_checkboxes['openwaterevapo'],\n",
    "                                                              module_checkboxes['drainedIrrigation'],\n",
    "                                                              module_checkboxes['riceIrrigation'],\n",
    "                                                              module_checkboxes['wateruse'],\n",
    "                                                              module_checkboxes['useWaterDemandAveYear'],\n",
    "                                                              module_checkboxes['TransientWaterDemandChange'],\n",
    "                                                              module_checkboxes['wateruseRegion'],\n",
    "                                                              module_checkboxes['groundwaterSmooth'],\n",
    "                                                              module_checkboxes['indicator'],\n",
    "                                                              module_checkboxes['readNetcdfStack'],\n",
    "                                                              module_checkboxes['writeNetcdf'],\n",
    "                                                              module_checkboxes['writeNetcdfStack']],\n",
    "                                                              layout=ipywidgets.Layout(grid_template_columns=\"33% 33% 33%\"))])]\n",
    "\n",
    "        if module_checkboxes['SplitRouting'].value == True:\n",
    "            module_checkboxes['InitLisflood'].value = True\n",
    "            module_checkboxes['InitLisfloodwithoutSplit'].value = False\n",
    "            module_checkboxes['InitLisfloodwithoutSplit'].disabled = True\n",
    "        else:\n",
    "            module_checkboxes['InitLisflood'].value = False\n",
    "            module_checkboxes['InitLisflood'].disabled = True\n",
    "            module_checkboxes['InitLisfloodwithoutSplit'].value = True\n",
    "\n",
    "        module_checkboxes['SplitRouting'].observe(on_SplitRouting_clicked, names='value')\n",
    "        module_checkboxes['repDischargeTs'].observe(on_repDischargeTs_clicked, names='value')\n",
    "\n",
    "        # display options\n",
    "        display(tabs)\n",
    "\n",
    "        # button to start processing method\n",
    "        processingButton = ipywidgets.Button(description=\"Start processing\")\n",
    "        processingButton.on_click(on_processingButton_clicked)\n",
    "        display(processingButton)\n",
    "\n",
    "\n",
    "# callback function to write input data to XML files and start processing of PRE-RUN and RUN\n",
    "def on_processingButton_clicked(b):    \n",
    "    # checks whether variable 'datasets' with reference to NetCDF files exists\n",
    "    # if yes, closes all NetCDF files to re-write them\n",
    "    # if no, creates empty list 'datasets'\n",
    "    if 'datasets' in locals():\n",
    "        for item in datasets:\n",
    "            item[1].close()\n",
    "        datasets.clear()\n",
    "    else:\n",
    "        datasets = []\n",
    "\n",
    "    # loops through references to PRE-RUN and RUN settings files\n",
    "    for calibration_list in calibration_parameters_xml:\n",
    "        # loops through calibration parameters listed in RUN settings file\n",
    "        for element in calibration_list:\n",
    "            # checks element type in XML file and allowed data range in parameter_specs dictionary\n",
    "            if (element.tag == 'textvar' and element.attrib['name'] in parameter_specs):\n",
    "                # updates value based on user input section\n",
    "                element.attrib['value'] = str(calibration_sliders[element.attrib['name']].children[0].value)\n",
    "\n",
    "    # loops through references to PRE-RUN and RUN settings files\n",
    "    for module_list in optional_modules_xml:\n",
    "        # loops through calibration parameters listed in RUN settings file\n",
    "        for element in module_list:\n",
    "            # updates value based on user input section\n",
    "            #print('\\nVorher ', element.attrib['choice'], '\\nNachher', str(int(module_checkboxes[element.attrib['name']].value)), element.attrib)\n",
    "            element.attrib['choice'] = str(int(module_checkboxes[element.attrib['name']].value))\n",
    "\n",
    "    # sets allowed combinations for SplitRouting\n",
    "    if module_checkboxes['SplitRouting'].value:\n",
    "        optional_modules_xml[0].findall(\"setoption[@name='SplitRouting']\")[0].attrib['choice'] = '1'\n",
    "        optional_modules_xml[0].findall(\"setoption[@name='InitLisflood']\")[0].attrib['choice'] = '1'\n",
    "        optional_modules_xml[0].findall(\"setoption[@name='InitLisfloodwithoutSplit']\")[0].attrib['choice'] = '0'\n",
    "        optional_modules_xml[1].findall(\"setoption[@name='SplitRouting']\")[0].attrib['choice'] = '1'\n",
    "        optional_modules_xml[1].findall(\"setoption[@name='InitLisflood']\")[0].attrib['choice'] = '0'\n",
    "        optional_modules_xml[1].findall(\"setoption[@name='InitLisfloodwithoutSplit']\")[0].attrib['choice'] = '0'\n",
    "    else:\n",
    "        optional_modules_xml[0].findall(\"setoption[@name='SplitRouting']\")[0].attrib['choice'] = '0'\n",
    "        optional_modules_xml[0].findall(\"setoption[@name='InitLisflood']\")[0].attrib['choice'] = '0'\n",
    "        optional_modules_xml[0].findall(\"setoption[@name='InitLisfloodwithoutSplit']\")[0].attrib['choice'] = '1'\n",
    "        optional_modules_xml[1].findall(\"setoption[@name='SplitRouting']\")[0].attrib['choice'] = '0'\n",
    "        optional_modules_xml[1].findall(\"setoption[@name='InitLisflood']\")[0].attrib['choice'] = '0'\n",
    "        optional_modules_xml[1].findall(\"setoption[@name='InitLisfloodwithoutSplit']\")[0].attrib['choice'] = '0'\n",
    "\n",
    "    # writes simulation start and stop date, timestep and coordinates from user input section to both settings files\n",
    "    for i in range(len(tree)):\n",
    "        StepStart[i].attrib['value'] = '%s %s' % (\n",
    "        str(datetime.datetime.strptime(str(StepStart_picker[i].value), \"%Y-%m-%d\").strftime(\"%d/%m/%Y\")),\n",
    "        StepStart[i].attrib['value'].split()[1])\n",
    "        StepEnd[i].attrib['value'] = '%s %s' % (\n",
    "        str(datetime.datetime.strptime(str(StepEnd_picker[i].value), \"%Y-%m-%d\").strftime(\"%d/%m/%Y\")),\n",
    "        StepEnd[i].attrib['value'].split()[1])\n",
    "        DtSec_xml[i].attrib['value'] = str(DtSec_box.value)\n",
    "        if module_checkboxes['repDischargeTs'].value == True: \n",
    "            coordinates[i].attrib['value'] = '{} {}'.format(marker.location[1], marker.location[0])\n",
    "        tree[i].write(fc[i].selected)\n",
    "\n",
    "    # starts LISFLOOD PRE-RUN and RUN according to (new) input\n",
    "    path1 = fc[0].selected\n",
    "    path2 = fc[1].selected\n",
    "    print('PRE-RUN...\\n')\n",
    "    !lisflood $path1\n",
    "    print('\\n\\n\\n\\nRUN...\\n')\n",
    "    !lisflood $path2\n",
    "    print(\"\\nDone\")\n",
    "\n",
    "\n",
    "# callback function to change map visibility\n",
    "def on_repDischargeTs_clicked(button):\n",
    "    if button['new']:\n",
    "        m.layout.display = \"block\"\n",
    "    else:\n",
    "        m.layout.display = \"none\"\n",
    "\n",
    "\n",
    "# callback function to prevent false input regarding SplitRouting\n",
    "def on_SplitRouting_clicked(button):\n",
    "    if button['new']:\n",
    "        module_checkboxes['InitLisflood'].disabled = False\n",
    "        module_checkboxes['InitLisfloodwithoutSplit'].value = False\n",
    "        module_checkboxes['InitLisfloodwithoutSplit'].disabled = True\n",
    "    else:\n",
    "        module_checkboxes['InitLisflood'].value = False\n",
    "        module_checkboxes['InitLisflood'].disabled = True\n",
    "        module_checkboxes['InitLisfloodwithoutSplit'].disabled = False\n",
    "\n",
    "\n",
    "\n",
    "####################################################\n",
    "####################### CODE #######################\n",
    "####################################################\n",
    "\n",
    "# dictionary which contains editable calibration parameters including allowed data range\n",
    "parameter_specs = {\n",
    "    # parameter name           min   max    step  format  unit\n",
    "    \"UpperZoneTimeConstant\": [0.01, 40, 0.01, '.2f', r'\\(\\left[\\text{days}\\right]\\)'],\n",
    "    \"LowerZoneTimeConstant\": [1000, 10500, 5, '.0f', r'\\(\\left[\\text{days}\\right]\\)'],\n",
    "    \"GwPercValue\": [0.01, 2, 0.01, '.2f', r'\\(\\left[-\\right]\\)'],\n",
    "    \"GwLoss\": [0, 0.5, 0.01, '.2f', r'\\(\\frac{\\text{mm}}{\\text{day}}\\)'],\n",
    "    \"b_Xinanjiang\": [0.5, 5, 0.01, '.2f', r'\\(\\left[-\\right]\\)'],\n",
    "    \"PowerPrefFlow\": [0.5, 8, 0.01, '.2f', r'\\(\\left[-\\right]\\)'],\n",
    "    \"CalChanMan\": [0.1, 20, 0.01, '.2f', r'\\(\\left[-\\right]\\)'],\n",
    "    \"SnowMeltCoef\": [2.5, 6.5, 0.01, '.2f', r'\\(\\frac{\\text{mm}}{\\text{C day}}\\)'],\n",
    "    \"CalChanMan2\": [0.1, 20, 0.01, '.2f', r'\\(\\left[-\\right]\\)'],\n",
    "}\n",
    "\n",
    "# creates list of file choosers for PRE-RUN and RUN settings file\n",
    "fc = [FileChooser(os.getcwd()), FileChooser(os.getcwd())]\n",
    "# sets title of file chooser\n",
    "fc[0].title = '<b>Please select PRE-RUN Settings File:</b>'\n",
    "fc[1].title = '<b>Please select RUN Settings File:</b>'\n",
    "# initializes file chooser\n",
    "for chooser in fc:\n",
    "    # only show XML files and folders\n",
    "    chooser.filter_pattern = '*.xml'\n",
    "    # uses icon for folder to easier distinguish between folders and files\n",
    "    chooser.use_dir_icons = True\n",
    "    # callback function for RUN XML file\n",
    "    chooser.register_callback(show_settings)\n",
    "# display file chooser\n",
    "ipywidgets.VBox([fc[0], fc[1]])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wrapped-newman",
   "metadata": {},
   "source": [
    "## Outputs\n",
    "\n",
    "Please run the code cell below to visualize the outputs. The notebook can also be used to visualize data of previous LISFLOOD runs. When running the [Outputs](#Outputs) code cell, the notebook checks whether the [Processing](#Processing) code cell has been executed before. If yes, it opens the output files coming from this LISFLOOD run. If not, it opens another file chooser where you can choose the output folder that you would like to visualize. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ranging-wagon",
   "metadata": {
    "code_folding": [
     0,
     3,
     11,
     18,
     27,
     38
    ],
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Click to EXPAND\n",
    "\n",
    "# returns element in NetCDF array\n",
    "def subfinder(mylist, parameter):\n",
    "    for i in range(len(mylist)):\n",
    "        if mylist[i][0] == parameter:\n",
    "            match = mylist[i][1]\n",
    "    return match\n",
    "\n",
    "\n",
    "# updates date of spatial plot from time slider\n",
    "def update_time(date):\n",
    "    mm.set_array(subfinder(datasets, parameter_dropdown.value)[parameter_dropdown.value][:][date].ravel())\n",
    "    title = '{}: {}'.format(parameter_dropdown.value, datevar[0][date].strftime('%d %b %Y'))\n",
    "    plt.title(title, size='xx-large')\n",
    "    plt.draw()\n",
    "\n",
    "\n",
    "#  updates parameter of spatial plot from dropdown menu\n",
    "def update_parameter(parameter):\n",
    "    mm.set_array(subfinder(datasets, parameter)[parameter][:][date_slider.value].ravel())\n",
    "    mm.autoscale()\n",
    "    cbar.update_normal(mm)\n",
    "    title = '{}: {}'.format(parameter, datevar[0][date_slider.value].strftime('%d %b %Y'))\n",
    "    plt.title(title, size='xx-large')\n",
    "    plt.draw()\n",
    "\n",
    "\n",
    "# adds CSV data to time series data frame\n",
    "def addData(df, path, setting):\n",
    "    df_temp = pd.read_csv(path, delim_whitespace=True, header=None)\n",
    "    df_temp = df_temp.drop([0, 1, 2, 3], axis=0)\n",
    "    df_temp = df_temp.drop([2, 3, 4, 5, 6, 7, 8, 9], axis=1)\n",
    "    df_temp.columns = ['date', 'value']\n",
    "    df_temp.insert(0, 'setting', [setting] * len(df_temp.index), True)\n",
    "\n",
    "    return pd.concat([df, df_temp], axis=0, ignore_index=True)\n",
    "\n",
    "\n",
    "# plots spatial and time series output data\n",
    "def plot(chooser, output_dir):\n",
    "    # sets path to output directory depending on function parameters\n",
    "    if output_dir:\n",
    "        path = chooser.selected_path\n",
    "    else:\n",
    "        path = os.path.join(chooser.selected_path, 'out')\n",
    "\n",
    "    # checks whether output data is present\n",
    "    if len(glob.glob('{}/*.nc'.format(path))) == 0 and len(glob.glob('{}/*.tss'.format(path))) == 0:\n",
    "        print('No output files in {}.'.format(path))\n",
    "\n",
    "    global datevar\n",
    "    \n",
    "    # checks for 'dis_run.tss' file\n",
    "    if os.path.isfile('{}/dis_run.tss'.format(path)):\n",
    "        # creates data frame and reads 'dis_run.tss'\n",
    "        df = pd.DataFrame()\n",
    "        df = addData(df, '{}/dis_run.tss'.format(path), 'Discharge')\n",
    "        # if output was generated using the processing section of this notebook,\n",
    "        # the calendar start day can be read from the XML file\n",
    "        if 'CalendarDayStart' in globals():\n",
    "            dates = []\n",
    "            DtSec = datetime.timedelta(seconds=int(DtSec_xml[1].attrib['value']))\n",
    "            for factor in df['date']:\n",
    "                dates.append(CalendarDayStart + (int(factor) - 1) * DtSec)\n",
    "            df['date'] = dates\n",
    "        elif 'datevar' in globals():\n",
    "            df['date'] = datevar[0].tolist()\n",
    "        # if not, x-axis can not be reformated\n",
    "        else:\n",
    "            print('No reference data on CalendarDayStart to format x-axis of time series plot.')\n",
    "        # plots data\n",
    "        selection = alt.selection_multi(fields=['setting'], bind='legend')\n",
    "        chart = alt.Chart(df\n",
    "                ).mark_line(point=True\n",
    "                ).encode(x='date:T',\n",
    "                         y='value:Q',\n",
    "                         color=alt.Color('setting', legend=alt.Legend(title=\"Setting\")),\n",
    "                         opacity=alt.condition(selection, alt.value(1), alt.value(0.2))\n",
    "                ).add_selection(selection\n",
    "                ).interactive(bind_y=False\n",
    "                ).properties(width=800, height=300)\n",
    "        # display outputs\n",
    "        display(ipywidgets.HTML(value = f\"<center><b><font size=5>{'Discharge Time Series'}</b></center>\"))\n",
    "        display(chart)\n",
    "\n",
    "    # check for spatial output files in output folder\n",
    "    if len(glob.glob('{}/*.nc'.format(path))) != 0:\n",
    "\n",
    "        global datasets\n",
    "        global date_slider\n",
    "        global parameter_dropdown\n",
    "        global mm\n",
    "        global cbar\n",
    "\n",
    "        # list of paths to  all spatial output files except avgdis and lzavin\n",
    "        paths_datasets = list(filter(lambda number: (not number.endswith('avgdis.nc') and not number.endswith('lzavin.nc')),\n",
    "                                     glob.glob('{}/*.nc'.format(path))))\n",
    "        \n",
    "        if 'datasets' in locals():\n",
    "            datasets.clear()\n",
    "        else:\n",
    "            datasets = []\n",
    "\n",
    "        i = 0\n",
    "        # open netCDF datasets\n",
    "        for dataset in paths_datasets:\n",
    "            datasets.append([dataset.split('/')[-1].split('.')[0], netCDF4.Dataset(paths_datasets[i])])\n",
    "            i = i + 1\n",
    "\n",
    "        # get dates from NetCDF file\n",
    "        try:\n",
    "            # get calendar\n",
    "            t_cal = datasets[0][1].variables['time'].calendar\n",
    "        except AttributeError:\n",
    "            t_cal = u\"gregorian\"\n",
    "        datevar = []\n",
    "        with warnings.catch_warnings():\n",
    "            warnings.filterwarnings(\"ignore\")\n",
    "            datevar.append(netCDF4.num2date(datasets[0][1].variables['time'][:],  # get values\n",
    "                                        units=datasets[0][1].variables['time'].units,  # get unit\n",
    "                                        calendar=t_cal))\n",
    "\n",
    "        # create drowdown menu with all available outputs\n",
    "        parameter_dropdown = ipywidgets.Dropdown(options=[item[0] for item in datasets], description='Parameter:')\n",
    "        # create date slider for given time period\n",
    "        date_slider = ipywidgets.IntSlider(min=0, max=len(datasets[0][1].dimensions['time']) - 1, step=1, value=0,\n",
    "                                           description='Date:')\n",
    "        # create simulation controls\n",
    "        play = ipywidgets.Play(\n",
    "            min=0,\n",
    "            max=len(datasets[0][1].dimensions['time']) - 1,\n",
    "            step=1,\n",
    "            description=\"Press play\",\n",
    "            disabled=False\n",
    "        )\n",
    "\n",
    "        # get chosen parameter\n",
    "        parameter = parameter_dropdown.value\n",
    "        # create figure\n",
    "        out_spatial = ipywidgets.Output()\n",
    "        with out_spatial:\n",
    "            # create figure with open street map\n",
    "            request = cimgt.OSM()\n",
    "            fig, ax = plt.subplots(figsize=(8,4), subplot_kw=dict(projection=request.crs))\n",
    "            ax.add_image(request, 8)\n",
    "            cmap = plt.cm.cool\n",
    "            vmin = np.min(subfinder(datasets, parameter)[parameter][:])\n",
    "            vmax = np.max(subfinder(datasets, parameter)[parameter][:])\n",
    "            #vmax = np.percentile(subfinder(datasets, parameter)[parameter][:], 97.5)\n",
    "            mm = ax.pcolormesh(subfinder(datasets, parameter)['lon'][:],\n",
    "                               subfinder(datasets, parameter)['lat'][:],\n",
    "                               subfinder(datasets, parameter)[parameter][:][0],\n",
    "                               vmin=vmin,\n",
    "                               vmax=vmax,\n",
    "                               transform=ccrs.PlateCarree(),\n",
    "                               cmap=cmap,\n",
    "                               alpha=0.5)\n",
    "            cbar = plt.colorbar(mm, shrink=0.7)\n",
    "            cbar.ax.tick_params(labelsize=17)\n",
    "            update_time(0)\n",
    "            \n",
    "        # update plot when user changes parameter in dropdown menu\n",
    "        ipywidgets.interactive(update_parameter, parameter=parameter_dropdown)\n",
    "        # update plot when user changes date slider\n",
    "        ipywidgets.interactive(update_time, date=date_slider)\n",
    "        # update plot when user starts simulation\n",
    "        ipywidgets.jslink((play, 'value'), (date_slider, 'value'))\n",
    "\n",
    "        # display outputs\n",
    "        display(ipywidgets.HTML(value = f\"<center><b><font size=5>{'Spatial Outputs'}</b></center>\"))\n",
    "        display(out_spatial)\n",
    "        display(ipywidgets.HBox([play, date_slider, parameter_dropdown]))\n",
    "        print('\\n\\n\\n\\n\\n\\n\\n\\n')\n",
    "        \n",
    "\n",
    "####################################################\n",
    "####################### CODE #######################\n",
    "####################################################\n",
    "\n",
    "# check if filechooser exist from processing\n",
    "if 'fc' in locals():\n",
    "    plot(fc[1], output_dir=False)\n",
    "# if not show file chooser\n",
    "else:\n",
    "    # create file chooser for output folder\n",
    "    chooser = FileChooser(os.getcwd())\n",
    "    # sets title of file chooser\n",
    "    chooser.title = '<b>Please select output folder:</b>'\n",
    "    # only show folders\n",
    "    chooser.show_only_dirs = True\n",
    "    # show directory icons\n",
    "    chooser.use_dir_icons = True\n",
    "    # callback function\n",
    "    chooser.register_callback(functools.partial(plot, output_dir=True))\n",
    "    # display file chooser\n",
    "    display(chooser)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}