Issue 371 cli command for creating a schedule (#372)

Add CLI command for creating a schedule. Also introduce marshmallow/click validation for quantities. * Rename variable Signed-off-by: F.N. Claessen <felix@seita.nl> * Allow passing an explicit sensor id to find prices Signed-off-by: F.N. Claessen <felix@seita.nl> * Add CLI command to create a schedule Signed-off-by: F.N. Claessen <felix@seita.nl> * Add docstring Signed-off-by: F.N. Claessen <felix@seita.nl> * Expose soc_min, soc_max and roundtrip_efficiency parameters as CLI options, too Signed-off-by: F.N. Claessen <felix@seita.nl> * Allow setting SoC targets Signed-off-by: F.N. Claessen <felix@seita.nl> * Fix attribute Signed-off-by: F.N. Claessen <felix@seita.nl> * Allow use of make_schedule outside of job context Signed-off-by: F.N. Claessen <felix@seita.nl> * Fix SoC target input and corresponding type annotation Signed-off-by: F.N. Claessen <felix@seita.nl> * Add note about current limitations Signed-off-by: F.N. Claessen <felix@seita.nl> * Add notes about units Signed-off-by: F.N. Claessen <felix@seita.nl> * Remove print statement Signed-off-by: F.N. Claessen <felix@seita.nl> * Switch the schedule to the resolution of the power sensor Signed-off-by: F.N. Claessen <felix@seita.nl> * Rename variable in tests, too Signed-off-by: F.N. Claessen <felix@seita.nl> * check parameters: do sensors exist, do we have soc-min and soc-max information available? Signed-off-by: Nicolas Höning <nicolas@seita.nl> * Actually the code expects these soc attributes to be on the sensor Signed-off-by: Nicolas Höning <nicolas@seita.nl> * use utility function to check attribute Signed-off-by: Nicolas Höning <nicolas@seita.nl> * identify sensor by instance, as two may have the same name across assets Signed-off-by: Nicolas Höning <nicolas@seita.nl> * Consistent capitalization Signed-off-by: F.N. Claessen <felix@seita.nl> * Allow unit conversion for individual int/float values Signed-off-by: F.N. Claessen <felix@seita.nl> * Allow unit conversion to and from a percentage of some capacity Signed-off-by: F.N. Claessen <felix@seita.nl> * Test unit conversion to and from a percentage of some capacity Signed-off-by: F.N. Claessen <felix@seita.nl> * CLI scheduling command uses % SoC units by default Signed-off-by: F.N. Claessen <felix@seita.nl> * Round charging schedules Signed-off-by: F.N. Claessen <felix@seita.nl> * Fix calculation of belief time (by handing the problem over to timely beliefs) Signed-off-by: F.N. Claessen <felix@seita.nl> * flake8 Signed-off-by: F.N. Claessen <felix@seita.nl> * fix (thanks mypy) Signed-off-by: F.N. Claessen <felix@seita.nl> * Fix timezone issue for trimming planning window Signed-off-by: F.N. Claessen <felix@seita.nl> * Fix timezone issue for setting SoC targets for schedules crossing DST transitions Signed-off-by: F.N. Claessen <felix@seita.nl> * turn --end parameter into --duration Signed-off-by: Nicolas Höning <nicolas@seita.nl> * Switch efficiency input to % values and add input validation using marshmallow Signed-off-by: F.N. Claessen <felix@seita.nl> * Force users to be specific upon providing input of percentage values or ratios for the round-trip efficiency Signed-off-by: F.N. Claessen <felix@seita.nl> * Fix percentage range and add range example for power quantity Signed-off-by: F.N. Claessen <felix@seita.nl> * Switch validation to return Quantity objects instead of magnitudes Signed-off-by: F.N. Claessen <felix@seita.nl> * Switch SOC input to percentages only Signed-off-by: F.N. Claessen <felix@seita.nl> * Rename factor_id to optimization_context_id Signed-off-by: F.N. Claessen <felix@seita.nl> * Delete obsolete class Signed-off-by: F.N. Claessen <felix@seita.nl> * Rename validator class Signed-off-by: F.N. Claessen <felix@seita.nl> * flake8 and mypy Signed-off-by: F.N. Claessen <felix@seita.nl> Co-authored-by: Nicolas Höning <nicolas@seita.nl>
FlexMeasures · Mar 4, 2022 · 8ad4802 · 8ad4802
1 parent 68ee6cd
commit 8ad4802
Show file tree

Hide file tree

Showing 13 changed files with 327 additions and 41 deletions.
diff --git a/documentation/changelog.rst b/documentation/changelog.rst
@@ -12,13 +12,14 @@ New features
 * Three new CLI commands for cleaning up your database: delete 1) unchanged beliefs, 2) NaN values or 3) a sensor and all of its time series data [see `PR #328 <http://www.github.com/FlexMeasures/flexmeasures/pull/328>`_]
 * Add CLI option to pass a data unit when reading in time series data from CSV, so data can automatically be converted to the sensor unit [see `PR #341 <http://www.github.com/FlexMeasures/flexmeasures/pull/341>`_]
 * Add CLI option to specify custom strings that should be interpreted as NaN values when reading in time series data from CSV [see `PR #357 <http://www.github.com/FlexMeasures/flexmeasures/pull/357>`_]
-* Add CLI commands ``flexmeasures add sensor``, ``flexmeasures add asset-type``, ``flexmeasures add beliefs`` (which were experimental features before). [see `PR #337 <http://www.github.com/FlexMeasures/flexmeasures/pull/337>`_]
+* Add CLI commands ``flexmeasures add sensor``, ``flexmeasures add asset-type``, ``flexmeasures add beliefs`` (which were experimental features before) [see `PR #337 <http://www.github.com/FlexMeasures/flexmeasures/pull/337>`_]
 * Add CLI commands for showing data [see `PR #339 <http://www.github.com/FlexMeasures/flexmeasures/pull/339>`_]
 
 * Add CLI command for attaching annotations to assets: ``flexmeasures add holidays`` adds public holidays [see `PR #343 <http://www.github.com/FlexMeasures/flexmeasures/pull/343>`_]
 * Add CLI command for resampling existing sensor data to new resolution [see `PR #360 <http://www.github.com/FlexMeasures/flexmeasures/pull/360>`_]
 * Add CLI command to edit/add an attribute on an asset or sensor. [see `PR #380 <http://www.github.com/FlexMeasures/flexmeasures/pull/380>`_]
-* Add CLI command to add a toy account for tutorials and trying things [see `PR #368 <http://www.github.com/FlexMeasures/flexmeasures/pull/368>`_].
+* Add CLI command to add a toy account for tutorials and trying things [see `PR #368 <http://www.github.com/FlexMeasures/flexmeasures/pull/368>`_]
+* Add CLI command to create a charging schedule [see `PR #372 <http://www.github.com/FlexMeasures/flexmeasures/pull/372>`_]
 * Support for percent (%) and permille (‰) sensor units [see `PR #359 <http://www.github.com/FlexMeasures/flexmeasures/pull/359>`_]
 
 Bugfixes

diff --git a/documentation/cli/change_log.rst b/documentation/cli/change_log.rst
@@ -11,6 +11,7 @@ since v0.9.0 | January 26, 2022
 * Add ``flexmeasures edit resample-data`` CLI command to resample sensor data to a different resolution.
 * Add ``flexmeasures edit attribute`` CLI command to edit/add an attribute on an asset or sensor.
 * Add ``flexmeasures add toy-account`` for tutorials and trying things.
+* Add ``flexmeasures add schedule`` to create a new schedule for a given power sensor.
 * Rename ``flexmeasures add structure`` to ``flexmeasures add initial-structure``. 
 
 

diff --git a/documentation/cli/commands.rst b/documentation/cli/commands.rst
@@ -34,6 +34,7 @@ of which some are referred to in this documentation.
 ``flexmeasures add external-weather-forecasts``   Collect weather forecasts from the DarkSky API.
 ``flexmeasures add beliefs``                      Load beliefs from file.
 ``flexmeasures add forecasts``                    Create forecasts.
+``flexmeasures add schedule``                     Create a charging schedule.
 ``flexmeasures add toy-account``                  Create a toy account, for tutorials and trying things.
 ================================================= =======================================
 

diff --git a/flexmeasures/api/v1_3/tests/test_api_v1_3.py b/flexmeasures/api/v1_3/tests/test_api_v1_3.py
@@ -74,7 +74,7 @@ def test_post_udi_event_and_get_device_message(
         len(app.queues["scheduling"]) == 1
     )  # only 1 schedule should be made for 1 asset
     job = app.queues["scheduling"].jobs[0]
-    assert job.kwargs["asset_id"] == sensor.id
+    assert job.kwargs["sensor_id"] == sensor.id
     assert job.kwargs["start"] == parse_datetime(message["datetime"])
     assert job.id == message["event"]
 

diff --git a/flexmeasures/api/v1_3/tests/test_api_v1_3_fresh_db.py b/flexmeasures/api/v1_3/tests/test_api_v1_3_fresh_db.py
@@ -38,7 +38,7 @@ def test_post_udi_event_and_get_device_message_with_unknown_prices(
             len(app.queues["scheduling"]) == 1
         )  # only 1 schedule should be made for 1 asset
         job = app.queues["scheduling"].jobs[0]
-        assert job.kwargs["asset_id"] == sensor.id
+        assert job.kwargs["sensor_id"] == sensor.id
         assert job.kwargs["start"] == parse_datetime(message["datetime"])
         assert job.id == message["event"]
         assert (

diff --git a/flexmeasures/cli/data_add.py b/flexmeasures/cli/data_add.py
@@ -1,9 +1,11 @@
 """CLI Tasks for populating the database - most useful in development"""
 
 from datetime import timedelta
-from typing import Dict, List, Optional
+from typing import Dict, List, Optional, Tuple
 import json
 
+from marshmallow import validate
+import numpy as np
 import pandas as pd
 import pytz
 from flask import current_app as app
@@ -13,6 +15,7 @@
 from sqlalchemy.exc import IntegrityError
 import timely_beliefs as tb
 from workalendar.registry import registry as workalendar_registry
+import isodate
 
 from flexmeasures.data import db
 from flexmeasures.data.scripts.data_gen import (
@@ -21,14 +24,20 @@
     add_default_asset_types,
 )
 from flexmeasures.data.services.forecasting import create_forecasting_jobs
+from flexmeasures.data.services.scheduling import make_schedule
 from flexmeasures.data.services.users import create_user
 from flexmeasures.data.models.user import Account, AccountRole, RolesAccounts
 from flexmeasures.data.models.time_series import (
     Sensor,
     TimedBelief,
 )
+from flexmeasures.data.models.validation_utils import (
+    check_required_attributes,
+    MissingAttributeException,
+)
 from flexmeasures.data.models.annotations import Annotation, get_or_create_annotation
 from flexmeasures.data.schemas.sensors import SensorSchema
+from flexmeasures.data.schemas.units import QuantityField
 from flexmeasures.data.schemas.generic_assets import (
     GenericAssetSchema,
     GenericAssetTypeSchema,
@@ -41,7 +50,7 @@
 )
 from flexmeasures.utils import flexmeasures_inflection
 from flexmeasures.utils.time_utils import server_now
-from flexmeasures.utils.unit_utils import convert_units
+from flexmeasures.utils.unit_utils import convert_units, ur
 
 
 @click.group("add")
@@ -132,7 +141,7 @@ def new_user(
         raise click.Abort
     account = db.session.query(Account).get(account_id)
     if account is None:
-        print(f"No account with id {account_id} found!")
+        print(f"No account with ID {account_id} found!")
         raise click.Abort
     pwd1 = getpass.getpass(prompt="Please enter the password:")
     pwd2 = getpass.getpass(prompt="Please repeat the password:")
@@ -410,7 +419,7 @@ def add_beliefs(
     """
     sensor = Sensor.query.filter(Sensor.id == sensor_id).one_or_none()
     if sensor is None:
-        print(f"Failed to create beliefs: no sensor found with id {sensor_id}.")
+        print(f"Failed to create beliefs: no sensor found with ID {sensor_id}.")
         return
     if source.isdigit():
         _source = get_source_or_none(int(source), source_type="CLI script")
@@ -756,6 +765,167 @@ def create_forecasts(
         )
 
 
+@fm_add_data.command("schedule")
+@with_appcontext
+@click.option(
+    "--sensor-id",
+    "power_sensor_id",
+    required=True,
+    help="Create schedule for this sensor. Follow up with the sensor's ID.",
+)
+@click.option(
+    "--optimization-context-id",
+    "optimization_context_sensor_id",
+    required=True,
+    help="Optimize against this sensor, which measures a price factor or CO₂ intensity factor. Follow up with the sensor's ID.",
+)
+@click.option(
+    "--from",
+    "start_str",
+    required=True,
+    help="Schedule starts at this datetime. Follow up with a timezone-aware datetime in ISO 6801 format.",
+)
+@click.option(
+    "--duration",
+    "duration_str",
+    required=True,
+    help="Duration of schedule, after --from. Follow up with a duration in ISO 6801 format, e.g. PT1H (1 hour) or PT45M (45 minutes).",
+)
+@click.option(
+    "--soc-at-start",
+    "soc_at_start",
+    type=QuantityField("%", validate=validate.Range(min=0, max=1)),
+    required=True,
+    help="State of charge (e.g 32.8%, or 0.328) at the start of the schedule. Use --soc-unit to set a different unit.",
+)
+@click.option(
+    "--soc-target",
+    "soc_target_strings",
+    type=click.Tuple(
+        types=[QuantityField("%", validate=validate.Range(min=0, max=1)), str]
+    ),
+    multiple=True,
+    required=False,
+    help="Target state of charge (e.g 100%, or 1) at some datetime. Follow up with a float value and a timezone-aware datetime in ISO 6081 format."
+    " Use --soc-unit to set a different unit."
+    " This argument can be given multiple times."
+    " For example: --soc-target 100% 2022-02-23T13:40:52+00:00",
+)
+@click.option(
+    "--soc-min",
+    "soc_min",
+    type=QuantityField("%", validate=validate.Range(min=0, max=1)),
+    required=False,
+    help="Minimum state of charge (e.g 20%, or 0.2) for the schedule. Use --soc-unit to set a different unit.",
+)
+@click.option(
+    "--soc-max",
+    "soc_max",
+    type=QuantityField("%", validate=validate.Range(min=0, max=100)),
+    required=False,
+    help="Maximum state of charge (e.g 80%, or 0.8) for the schedule. Use --soc-unit to set a different unit.",
+)
+@click.option(
+    "--roundtrip-efficiency",
+    "roundtrip_efficiency",
+    type=QuantityField("%", validate=validate.Range(min=0, max=1)),
+    required=False,
+    default=1,
+    help="Round-trip efficiency (e.g. 85% or 0.85) to use for the schedule. Defaults to 100% (no losses).",
+)
+def create_schedule(
+    power_sensor_id: int,
+    optimization_context_sensor_id: int,
+    start_str: str,
+    duration_str: str,
+    soc_at_start: ur.Quantity,
+    soc_target_strings: List[Tuple[ur.Quantity, str]],
+    soc_min: Optional[ur.Quantity] = None,
+    soc_max: Optional[ur.Quantity] = None,
+    roundtrip_efficiency: Optional[ur.Quantity] = None,
+):
+    """Create a new schedule for a given power sensor.
+
+    Current limitations:
+    - only supports battery assets and Charge Points
+    - only supports datetimes on the hour or a multiple of the sensor resolution thereafter
+    """
+
+    # Parse input
+    power_sensor: Sensor = Sensor.query.filter(
+        Sensor.id == power_sensor_id
+    ).one_or_none()
+    if power_sensor is None:
+        click.echo(f"No sensor found with ID {power_sensor_id}.")
+        raise click.Abort()
+    if not power_sensor.measures_power:
+        click.echo(f"Sensor with ID {power_sensor_id} is not a power sensor.")
+        raise click.Abort()
+    optimization_context_sensor: Sensor = Sensor.query.filter(
+        Sensor.id == optimization_context_sensor_id
+    ).one_or_none()
+    if optimization_context_sensor is None:
+        click.echo(f"No sensor found with ID {optimization_context_sensor_id}.")
+        raise click.Abort()
+    start = pd.Timestamp(start_str)
+    end = start + isodate.parse_duration(duration_str)
+    for attribute in ("min_soc_in_mwh", "max_soc_in_mwh"):
+        try:
+            check_required_attributes(power_sensor, [(attribute, float)])
+        except MissingAttributeException:
+            click.echo(f"{power_sensor} has no {attribute} attribute.")
+            raise click.Abort()
+    soc_targets = pd.Series(
+        np.nan,
+        index=pd.date_range(
+            pd.Timestamp(start).tz_convert(power_sensor.timezone),
+            pd.Timestamp(end).tz_convert(power_sensor.timezone),
+            freq=power_sensor.event_resolution,
+            closed="right",
+        ),  # note that target values are indexed by their due date (i.e. closed="right")
+    )
+
+    # Convert round-trip efficiency to dimensionless
+    if roundtrip_efficiency is not None:
+        roundtrip_efficiency = roundtrip_efficiency.to(
+            ur.Quantity("dimensionless")
+        ).magnitude
+
+    # Convert SoC units to MWh, given the storage capacity
+    capacity_str = f"{power_sensor.get_attribute('max_soc_in_mwh')} MWh"
+    soc_at_start = convert_units(soc_at_start.magnitude, soc_at_start.units, "MWh", capacity=capacity_str)  # type: ignore
+    for soc_target_tuple in soc_target_strings:
+        soc_target_value_str, soc_target_dt_str = soc_target_tuple
+        soc_target_value = convert_units(
+            soc_target_value_str.magnitude,
+            str(soc_target_value_str.units),
+            "MWh",
+            capacity=capacity_str,
+        )
+        soc_target_datetime = pd.Timestamp(soc_target_dt_str)
+        soc_targets.loc[soc_target_datetime] = soc_target_value
+    if soc_min is not None:
+        soc_min = convert_units(soc_min.magnitude, str(soc_min.units), "MWh", capacity=capacity_str)  # type: ignore
+    if soc_max is not None:
+        soc_max = convert_units(soc_max.magnitude, str(soc_max.units), "MWh", capacity=capacity_str)  # type: ignore
+
+    success = make_schedule(
+        sensor_id=power_sensor_id,
+        start=start,
+        end=end,
+        belief_time=server_now(),
+        resolution=power_sensor.event_resolution,
+        soc_at_start=soc_at_start,
+        soc_targets=soc_targets,
+        soc_min=soc_min,
+        soc_max=soc_max,
+        roundtrip_efficiency=roundtrip_efficiency,
+        price_sensor=optimization_context_sensor,
+    )
+    if success:
+        print("New schedule is stored.")
+
+
 @fm_add_data.command("toy-account")
 @with_appcontext
 @click.option(

diff --git a/flexmeasures/data/models/planning/battery.py b/flexmeasures/data/models/planning/battery.py
@@ -25,6 +25,8 @@ def schedule_battery(
     soc_max: Optional[float] = None,
     roundtrip_efficiency: Optional[float] = None,
     prefer_charging_sooner: bool = True,
+    price_sensor: Optional[Sensor] = None,
+    round_to_decimals: Optional[int] = 6,
 ) -> Union[pd.Series, None]:
     """Schedule a battery asset based directly on the latest beliefs regarding market prices within the specified time
     window.
@@ -55,10 +57,17 @@ def schedule_battery(
 
     # Check for known prices or price forecasts, trimming planning window accordingly
     prices, (start, end) = get_prices(
-        sensor, (start, end), resolution, allow_trimmed_query_window=True
+        (start, end),
+        resolution,
+        price_sensor=price_sensor,
+        sensor=sensor,
+        allow_trimmed_query_window=True,
     )
+    start = pd.Timestamp(start).tz_convert("UTC")
+    end = pd.Timestamp(end).tz_convert("UTC")
     if soc_targets is not None:
         # soc targets are at the end of each time slot, while prices are indexed by the start of each time slot
+        soc_targets = soc_targets.tz_convert("UTC")
         soc_targets = soc_targets[start + resolution : end]
 
     # Add tiny price slope to prefer charging now rather than later, and discharging later rather than now.
@@ -131,4 +140,8 @@ def schedule_battery(
     else:
         battery_schedule = ems_schedule[0]
 
+    # Round schedule
+    if round_to_decimals:
+        battery_schedule = battery_schedule.round(round_to_decimals)
+
     return battery_schedule
diff --git a/flexmeasures/data/models/planning/charging_station.py b/flexmeasures/data/models/planning/charging_station.py
@@ -1,7 +1,7 @@
 from typing import Optional, Union
 from datetime import datetime, timedelta
 
-from pandas import Series, Timestamp
+import pandas as pd
 
 from flexmeasures.data.models.time_series import Sensor
 from flexmeasures.data.models.planning.solver import device_scheduler
@@ -20,12 +20,14 @@ def schedule_charging_station(
     end: datetime,
     resolution: timedelta,
     soc_at_start: float,
-    soc_targets: Series,
+    soc_targets: pd.Series,
     soc_min: Optional[float] = None,
     soc_max: Optional[float] = None,
     roundtrip_efficiency: Optional[float] = None,
     prefer_charging_sooner: bool = True,
-) -> Union[Series, None]:
+    price_sensor: Optional[Sensor] = None,
+    round_to_decimals: Optional[int] = 6,
+) -> Union[pd.Series, None]:
     """Schedule a charging station asset based directly on the latest beliefs regarding market prices within the specified time
     window.
     For the resulting consumption schedule, consumption is defined as positive values.
@@ -52,12 +54,16 @@ def schedule_charging_station(
 
     # Check for known prices or price forecasts, trimming planning window accordingly
     prices, (start, end) = get_prices(
-        sensor, (start, end), resolution, allow_trimmed_query_window=True
+        (start, end),
+        resolution,
+        price_sensor=price_sensor,
+        sensor=sensor,
+        allow_trimmed_query_window=True,
     )
     # soc targets are at the end of each time slot, while prices are indexed by the start of each time slot
     soc_targets = soc_targets.tz_convert("UTC")
-    start = Timestamp(start).tz_convert("UTC")
-    end = Timestamp(end).tz_convert("UTC")
+    start = pd.Timestamp(start).tz_convert("UTC")
+    end = pd.Timestamp(end).tz_convert("UTC")
     soc_targets = soc_targets[start + resolution : end]
 
     # Add tiny price slope to prefer charging now rather than later, and discharging later rather than now.
@@ -134,4 +140,8 @@ def schedule_charging_station(
     else:
         charging_station_schedule = ems_schedule[0]
 
+    # Round schedule
+    if round_to_decimals:
+        charging_station_schedule = charging_station_schedule.round(round_to_decimals)
+
     return charging_station_schedule