Issue 273 add roundtrip efficiency to scheduler (#291)

Rewrite our generic device scheduler to: - Deal with asymmetric efficiency losses of individual devices. - Deal with asymmetric up and down prices for deviating from previous commitments. Also allow round-trip efficiency to be communicated as a new optional field when POSTing UDI Events, with efficiency losses being assigned equally to charging and discharging. * Query TimedBelief rather than Power in api v1.3 tests Signed-off-by: F.N. Claessen <felix@seita.nl> * Query TimedBelief rather than Power in api v1.3 implementations Signed-off-by: F.N. Claessen <felix@seita.nl> * Query TimedBelief rather than Power in user services tests Signed-off-by: F.N. Claessen <felix@seita.nl> * Query TimedBelief rather than Power in query tests Signed-off-by: F.N. Claessen <felix@seita.nl> * Query TimedBelief rather than Power in forecasting tests Signed-off-by: F.N. Claessen <felix@seita.nl> * Query TimedBelief rather than Power in scheduling tests Signed-off-by: F.N. Claessen <felix@seita.nl> * Query TimedBelief rather than Power in api v1 tests Signed-off-by: F.N. Claessen <felix@seita.nl> * Simplify data deletion, like, by a lot Signed-off-by: F.N. Claessen <felix@seita.nl> * Count ex-ante TimedBeliefs after populating time series forecasts Signed-off-by: F.N. Claessen <felix@seita.nl> * Query TimedBelief rather than Price in api v1_1 tests Signed-off-by: F.N. Claessen <felix@seita.nl> * Query TimedBelief rather than Power/Price/Weather in Resource.load_sensor_data Signed-off-by: F.N. Claessen <felix@seita.nl> * Query TimedBelief rather than Power/Price/Weather in api v2.0 tests Signed-off-by: F.N. Claessen <felix@seita.nl> * Refactor: simplify duplicate query construction Signed-off-by: F.N. Claessen <felix@seita.nl> * Add custom join target to get rid of SA warning Signed-off-by: F.N. Claessen <felix@seita.nl> * Filter criteria should work for both TimedBeliefs and TimedValues Signed-off-by: F.N. Claessen <felix@seita.nl> * Clarify docstring Signed-off-by: F.N. Claessen <felix@seita.nl> * Query TimedBelief rather than Power in api v1 implementations Signed-off-by: F.N. Claessen <felix@seita.nl> * Schedules should contain one deterministic belief per event Signed-off-by: F.N. Claessen <felix@seita.nl> * Fix type annotation Signed-off-by: F.N. Claessen <felix@seita.nl> * flake8 Signed-off-by: F.N. Claessen <felix@seita.nl> * Query TimedBelief rather than Price/Weather for analytics Signed-off-by: F.N. Claessen <felix@seita.nl> * Query deterministic TimedBelief rather than Price for planning queries Signed-off-by: F.N. Claessen <felix@seita.nl> * Forecast TimedBelief rather than Power/Price/Weather Signed-off-by: F.N. Claessen <felix@seita.nl> * Schedule TimedBelief rather than Power Signed-off-by: F.N. Claessen <felix@seita.nl> * Apparently, to initialize a TimedBelief is to save a TimedBelief, too Signed-off-by: F.N. Claessen <felix@seita.nl> * Create TimedBelief rather than Power/Price/Weather in data generation script Signed-off-by: F.N. Claessen <felix@seita.nl> * Bump timely-beliefs dependency Signed-off-by: F.N. Claessen <felix@seita.nl> * Fix latest state query Signed-off-by: F.N. Claessen <felix@seita.nl> * Revert "Apparently, to initialize a TimedBelief is to save a TimedBelief, too" This reverts commit fb58ec7. * Prevent saving TimedBelief to session upon updating Sensor or Source Signed-off-by: F.N. Claessen <felix@seita.nl> * Create only TimedBeliefs in conftests Signed-off-by: F.N. Claessen <felix@seita.nl> * Use session.add_all calls instead of session.bulk_save_objects or individual session.add calls Signed-off-by: F.N. Claessen <felix@seita.nl> * API directly creates TimedBeliefs Signed-off-by: F.N. Claessen <felix@seita.nl> * CLI uses TimedBeliefs only Signed-off-by: F.N. Claessen <felix@seita.nl> * Data scripts use TimedBeliefs only Signed-off-by: F.N. Claessen <felix@seita.nl> * One more conftest switched to creating TimedBeliefs instead of Weather objects Signed-off-by: F.N. Claessen <felix@seita.nl> * Expand docstring note on forbidden replacements Signed-off-by: F.N. Claessen <felix@seita.nl> * Clarify docstring note on saving changed beliefs only Signed-off-by: F.N. Claessen <felix@seita.nl> * Remove redundant flush Signed-off-by: F.N. Claessen <felix@seita.nl> * Catch forbidden belief replacements with more specific exception Signed-off-by: F.N. Claessen <felix@seita.nl> * Rename variable Signed-off-by: F.N. Claessen <felix@seita.nl> * One transaction per request Signed-off-by: F.N. Claessen <felix@seita.nl> * Only enqueue forecasting jobs upon successfully saving new data Signed-off-by: F.N. Claessen <felix@seita.nl> * Flush instead of commit Signed-off-by: F.N. Claessen <felix@seita.nl> * Expand test for forbidden data replacement Signed-off-by: F.N. Claessen <felix@seita.nl> * Simplify play mode excemption for replacing beliefs Signed-off-by: F.N. Claessen <felix@seita.nl> * Add note about potential session rollback Signed-off-by: F.N. Claessen <felix@seita.nl> * Typo Signed-off-by: F.N. Claessen <felix@seita.nl> * Move UniqueViolation catching logic to error handler Signed-off-by: F.N. Claessen <felix@seita.nl> * flake8 Signed-off-by: F.N. Claessen <felix@seita.nl> * Rewrite solver to deal with asymmetry in up and down commitment prices Signed-off-by: F.N. Claessen <felix@seita.nl> * Add optional roundtrip_efficiency field to UDI events, and use it to scale prices Signed-off-by: F.N. Claessen <felix@seita.nl> * Add test cases for various round-trip efficiencies Signed-off-by: F.N. Claessen <felix@seita.nl> * Add changelog entries Signed-off-by: F.N. Claessen <felix@seita.nl> * Add documentation for the new API field Signed-off-by: F.N. Claessen <felix@seita.nl> * Grammar corrections Signed-off-by: F.N. Claessen <felix@seita.nl> * Fix return value for empty EMS Signed-off-by: F.N. Claessen <felix@seita.nl> * Allow efficiencies per device for multi-device EMS, by stopping the application of round-trip efficiency as price scalars and modeling device flows in more detail Signed-off-by: F.N. Claessen <felix@seita.nl> * Relax tests using some tolerance Signed-off-by: F.N. Claessen <felix@seita.nl> * Fix mistake Signed-off-by: F.N. Claessen <felix@seita.nl> * Add test docstring Signed-off-by: F.N. Claessen <felix@seita.nl> * Check round-trip efficiency for acceptable range Signed-off-by: F.N. Claessen <felix@seita.nl> * Expand docstring Signed-off-by: F.N. Claessen <felix@seita.nl>
FlexMeasures · Jan 3, 2022 · e0af47a · e0af47a
1 parent d788d9d
commit e0af47a
Show file tree

Hide file tree

Showing 9 changed files with 216 additions and 34 deletions.
diff --git a/documentation/api/change_log.rst b/documentation/api/change_log.rst
@@ -39,6 +39,14 @@ v2.0-0 | 2020-11-14
 - REST endpoints for managing assets: `/assets/` (GET, POST) and `/asset/<id>` (GET, PATCH, DELETE).
 
 
+v1.3-11 | 2022-01-01
+""""""""""""""""""""
+
+*Affects all versions since v1.3*.
+
+- Extended the *postUdiEvent* endpoint with an optional "roundtrip_efficiency" field, for use in scheduling.
+
+
 v1.3-10 | 2021-11-08
 """"""""""""""""""""
 

diff --git a/documentation/changelog.rst b/documentation/changelog.rst
@@ -11,6 +11,7 @@ v0.8.0 | November XX, 2021
 New features
 -----------
 * Charts with sensor data can be requested in one of the supported  [`vega-lite themes <https://github.com/vega/vega-themes#included-themes>`_] (incl. a dark theme) [see `PR #221 <http://www.github.com/FlexMeasures/flexmeasures/pull/221>`_]
+* Schedulers take into account round-trip efficiency if set [see `PR #291 <http://www.github.com/FlexMeasures/flexmeasures/pull/291>`_]
 
 Bugfixes
 -----------

diff --git a/flexmeasures/api/v1_3/implementations.py b/flexmeasures/api/v1_3/implementations.py
@@ -280,6 +280,9 @@ def post_udi_event_response(unit):
     if unit == "kWh":
         value = value / 1000.0
 
+    # get optional efficiency
+    roundtrip_efficiency = form.get("roundtrip_efficiency", None)
+
     # set soc targets
     start_of_schedule = datetime
     end_of_schedule = datetime + current_app.config.get("FLEXMEASURES_PLANNING_HORIZON")
@@ -349,6 +352,7 @@ def post_udi_event_response(unit):
         belief_time=datetime,
         soc_at_start=value,
         soc_targets=soc_targets,
+        roundtrip_efficiency=roundtrip_efficiency,
         udi_event_ea=form.get("event"),
         enqueue=True,
     )

diff --git a/flexmeasures/api/v1_3/routes.py b/flexmeasures/api/v1_3/routes.py
@@ -104,6 +104,7 @@ def post_udi_event():
     This "PostUdiEventRequest" message posts a state of charge (soc) of 12.1 kWh at 10.00am,
     and a target state of charge of 25 kWh at 4.00pm,
     as UDI event 204 of device 10 of owner 7.
+    Roundtrip efficiency for use in scheduling is set to 98%.
 
     .. code-block:: json
 
@@ -118,7 +119,8 @@ def post_udi_event():
                     "value": 25,
                     "datetime": "2015-06-02T16:00:00+00:00"
                 }
-            ]
+            ],
+            "roundtrip_efficiency": 0.98
         }
 
     **Example response**

diff --git a/flexmeasures/data/models/planning/battery.py b/flexmeasures/data/models/planning/battery.py
@@ -21,6 +21,7 @@ def schedule_battery(
     resolution: timedelta,
     soc_at_start: float,
     soc_targets: Optional[pd.Series] = None,
+    roundtrip_efficiency: Optional[float] = None,
     prefer_charging_sooner: bool = True,
 ) -> Union[pd.Series, None]:
     """Schedule a battery asset based directly on the latest beliefs regarding market prices within the specified time
@@ -37,6 +38,13 @@ def schedule_battery(
         ],
     )
 
+    # Check for round-trip efficiency
+    if roundtrip_efficiency is None:
+        # Get default from sensor, or use 100% otherwise
+        roundtrip_efficiency = sensor.get_attribute("roundtrip_efficiency", 1)
+    if roundtrip_efficiency <= 0 or roundtrip_efficiency > 1:
+        raise ValueError("roundtrip_efficiency expected within the interval (0, 1]")
+
     # Check for known prices or price forecasts, trimming planning window accordingly
     prices, (start, end) = get_prices(
         sensor, (start, end), resolution, allow_trimmed_query_window=True
@@ -69,6 +77,8 @@ def schedule_battery(
         "derivative equals",
         "derivative max",
         "derivative min",
+        "derivative down efficiency",
+        "derivative up efficiency",
     ]
     device_constraints = [initialize_df(columns, start, end, resolution)]
     if soc_targets is not None:
@@ -90,6 +100,10 @@ def schedule_battery(
     )
     device_constraints[0]["derivative max"] = sensor.get_attribute("capacity_in_mw")
 
+    # Apply round-trip efficiency evenly to charging and discharging
+    device_constraints[0]["derivative down efficiency"] = roundtrip_efficiency ** 0.5
+    device_constraints[0]["derivative up efficiency"] = roundtrip_efficiency ** 0.5
+
     # Set up EMS constraints (no additional constraints)
     columns = ["derivative max", "derivative min"]
     ems_constraints = initialize_df(columns, start, end, resolution)

diff --git a/flexmeasures/data/models/planning/charging_station.py b/flexmeasures/data/models/planning/charging_station.py
@@ -1,4 +1,4 @@
-from typing import Union
+from typing import Optional, Union
 from datetime import datetime, timedelta
 
 from pandas import Series, Timestamp
@@ -21,6 +21,7 @@ def schedule_charging_station(
     resolution: timedelta,
     soc_at_start: float,
     soc_targets: Series,
+    roundtrip_efficiency: Optional[float] = None,
     prefer_charging_sooner: bool = True,
 ) -> Union[Series, None]:
     """Schedule a charging station asset based directly on the latest beliefs regarding market prices within the specified time
@@ -32,6 +33,13 @@ def schedule_charging_station(
     # Check for required Sensor attributes
     sensor.check_required_attributes([("capacity_in_mw", (float, int))])
 
+    # Check for round-trip efficiency
+    if roundtrip_efficiency is None:
+        # Get default from sensor, or use 100% otherwise
+        roundtrip_efficiency = sensor.get_attribute("roundtrip_efficiency", 1)
+    if roundtrip_efficiency <= 0 or roundtrip_efficiency > 1:
+        raise ValueError("roundtrip_efficiency expected within the interval (0, 1]")
+
     # Check for known prices or price forecasts, trimming planning window accordingly
     prices, (start, end) = get_prices(
         sensor, (start, end), resolution, allow_trimmed_query_window=True
@@ -95,6 +103,10 @@ def schedule_charging_station(
     else:
         device_constraints[0]["derivative max"] = sensor.get_attribute("capacity_in_mw")
 
+    # Apply round-trip efficiency evenly to charging and discharging
+    device_constraints[0]["derivative down efficiency"] = roundtrip_efficiency ** 0.5
+    device_constraints[0]["derivative up efficiency"] = roundtrip_efficiency ** 0.5
+
     # Set up EMS constraints (no additional constraints)
     columns = ["derivative max", "derivative min"]
     ems_constraints = initialize_df(columns, start, end, resolution)

diff --git a/flexmeasures/data/models/planning/solver.py b/flexmeasures/data/models/planning/solver.py
@@ -10,6 +10,8 @@
     RangeSet,
     Param,
     Reals,
+    NonNegativeReals,
+    NonPositiveReals,
     Constraint,
     Objective,
     minimize,
@@ -30,8 +32,11 @@ def device_scheduler(  # noqa C901
     commitment_downwards_deviation_price: Union[List[pd.Series], List[float]],
     commitment_upwards_deviation_price: Union[List[pd.Series], List[float]],
 ) -> Tuple[List[pd.Series], float, SolverResults]:
-    """Schedule devices given constraints on a device and EMS level, and given a list of commitments by the EMS.
-    The commitments are assumed to be with regards to the flow of energy to the device (positive for consumption,
+    """This generic device scheduler is able to handle an EMS with multiple devices,
+    with various types of constraints on the EMS level and on the device level,
+    and with multiple market commitments on the EMS level.
+    A typical example is a house with many devices.
+    The commitments are assumed to be with regard to the flow of energy to the device (positive for consumption,
     negative for production). The solver minimises the costs of deviating from the commitments.
 
     Device constraints are on a device level. Handled constraints (listed by column name):
@@ -41,6 +46,8 @@ def device_scheduler(  # noqa C901
         derivative max: maximum flow (e.g. in MW or boxes/h)
         derivative min: minimum flow
         derivative equals: exact amount of flow (we do this by clamping derivative min and derivative max)
+        derivative down efficiency: ratio of downwards flows (flow into EMS : flow out of device)
+        derivative up efficiency: ratio of upwards flows (flow into device : flow out of EMS)
     EMS constraints are on an EMS level. Handled constraints (listed by column name):
         derivative max: maximum flow
         derivative min: minimum flow
@@ -54,13 +61,13 @@ def device_scheduler(  # noqa C901
 
     All Series and DataFrames should have the same resolution.
 
-    For now we pass in the various constraints and prices as separate variables, from which we make a MultiIndex
+    For now, we pass in the various constraints and prices as separate variables, from which we make a MultiIndex
     DataFrame. Later we could pass in a MultiIndex DataFrame directly.
     """
 
     # If the EMS has no devices, don't bother
     if len(device_constraints) == 0:
-        return [], 0
+        return [], 0, SolverResults()
 
     # Check if commitments have the same time window and resolution as the constraints
     start = device_constraints[0].index.to_pydatetime()[0]
@@ -164,6 +171,18 @@ def ems_derivative_min_select(m, j):
         else:
             return v
 
+    def device_derivative_down_efficiency(m, d, j):
+        try:
+            return device_constraints[d]["derivative down efficiency"].iloc[j]
+        except KeyError:
+            return 1
+
+    def device_derivative_up_efficiency(m, d, j):
+        try:
+            return device_constraints[d]["derivative up efficiency"].iloc[j]
+        except KeyError:
+            return 1
+
     model.up_price = Param(model.c, model.j, initialize=price_up_select)
     model.down_price = Param(model.c, model.j, initialize=price_down_select)
     model.commitment_quantity = Param(
@@ -179,45 +198,107 @@ def ems_derivative_min_select(m, j):
     )
     model.ems_derivative_max = Param(model.j, initialize=ems_derivative_max_select)
     model.ems_derivative_min = Param(model.j, initialize=ems_derivative_min_select)
+    model.device_derivative_down_efficiency = Param(
+        model.d, model.j, initialize=device_derivative_down_efficiency
+    )
+    model.device_derivative_up_efficiency = Param(
+        model.d, model.j, initialize=device_derivative_up_efficiency
+    )
 
     # Add variables
-    model.power = Var(model.d, model.j, domain=Reals, initialize=0)
+    model.ems_power = Var(model.d, model.j, domain=Reals, initialize=0)
+    model.device_power_down = Var(
+        model.d, model.j, domain=NonPositiveReals, initialize=0
+    )
+    model.device_power_up = Var(model.d, model.j, domain=NonNegativeReals, initialize=0)
+    model.commitment_downwards_deviation = Var(
+        model.c, model.j, domain=NonPositiveReals, initialize=0
+    )
+    model.commitment_upwards_deviation = Var(
+        model.c, model.j, domain=NonNegativeReals, initialize=0
+    )
 
     # Add constraints as a tuple of (lower bound, value, upper bound)
     def device_bounds(m, d, j):
         return (
             m.device_min[d, j],
-            sum(m.power[d, k] for k in range(0, j + 1)),
+            sum(
+                m.device_power_down[d, k] + m.device_power_up[d, k]
+                for k in range(0, j + 1)
+            ),
             m.device_max[d, j],
         )
 
     def device_derivative_bounds(m, d, j):
         return (
             m.device_derivative_min[d, j],
-            m.power[d, j],
+            m.device_power_down[d, j] + m.device_power_up[d, j],
+            m.device_derivative_max[d, j],
+        )
+
+    def device_down_derivative_bounds(m, d, j):
+        return (
+            m.device_derivative_min[d, j],
+            m.device_power_down[d, j],
+            0,
+        )
+
+    def device_up_derivative_bounds(m, d, j):
+        return (
+            0,
+            m.device_power_up[d, j],
             m.device_derivative_max[d, j],
         )
 
     def ems_derivative_bounds(m, j):
-        return m.ems_derivative_min[j], sum(m.power[:, j]), m.ems_derivative_max[j]
+        return m.ems_derivative_min[j], sum(m.ems_power[:, j]), m.ems_derivative_max[j]
+
+    def ems_flow_commitment_equalities(m, j):
+        """Couple EMS flows (sum over devices) to commitments."""
+        return (
+            0,
+            sum(m.commitment_quantity[:, j])
+            + sum(m.commitment_downwards_deviation[:, j])
+            + sum(m.commitment_upwards_deviation[:, j])
+            - sum(m.ems_power[:, j]),
+            0,
+        )
+
+    def device_derivative_equalities(m, d, j):
+        """Couple device flows to EMS flows per device, applying efficiencies."""
+        return (
+            0,
+            m.device_power_up[d, j] / m.device_derivative_up_efficiency[d, j]
+            + m.device_power_down[d, j] * m.device_derivative_down_efficiency[d, j]
+            - m.ems_power[d, j],
+            0,
+        )
 
     model.device_energy_bounds = Constraint(model.d, model.j, rule=device_bounds)
     model.device_power_bounds = Constraint(
         model.d, model.j, rule=device_derivative_bounds
     )
+    model.device_power_down_bounds = Constraint(
+        model.d, model.j, rule=device_down_derivative_bounds
+    )
+    model.device_power_up_bounds = Constraint(
+        model.d, model.j, rule=device_up_derivative_bounds
+    )
     model.ems_power_bounds = Constraint(model.j, rule=ems_derivative_bounds)
+    model.ems_power_commitment_equalities = Constraint(
+        model.j, rule=ems_flow_commitment_equalities
+    )
+    model.device_power_equalities = Constraint(
+        model.d, model.j, rule=device_derivative_equalities
+    )
 
     # Add objective
     def cost_function(m):
         costs = 0
         for c in m.c:
             for j in m.j:
-                ems_power_in_j = sum(m.power[d, j] for d in m.d)
-                ems_power_deviation = ems_power_in_j - m.commitment_quantity[c, j]
-                if value(ems_power_deviation) >= 0:
-                    costs += ems_power_deviation * m.up_price[c, j]
-                else:
-                    costs += ems_power_deviation * m.down_price[c, j]
+                costs += m.commitment_downwards_deviation[c, j] * m.down_price[c, j]
+                costs += m.commitment_upwards_deviation[c, j] * m.up_price[c, j]
         return costs
 
     model.costs = Objective(rule=cost_function, sense=minimize)
@@ -230,7 +311,10 @@ def cost_function(m):
     planned_costs = value(model.costs)
     planned_power_per_device = []
     for d in model.d:
-        planned_device_power = [model.power[d, j].value for j in model.j]
+        planned_device_power = [
+            model.device_power_down[d, j].value + model.device_power_up[d, j].value
+            for j in model.j
+        ]
         planned_power_per_device.append(
             pd.Series(
                 index=pd.date_range(
@@ -243,5 +327,5 @@ def cost_function(m):
     # model.pprint()
     # print(results.solver.termination_condition)
     # print(planned_costs)
-    # input()
+    # model.display()
     return planned_power_per_device, planned_costs, results