Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: add command flexmeasures add schedule for-process #768

Merged
merged 65 commits into from Aug 1, 2023
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
f64709a
test: add shiftable_load fixture
victorgarcia98 Jun 14, 2023
35ca7e7
test: move fixture setup_dummy_sensors from test_reporting.py to conf…
victorgarcia98 Jun 14, 2023
99af08c
feat: add ShiftableLoadFlexModelSchema
victorgarcia98 Jun 14, 2023
5216fa3
test: add ShiftableLoadFlexModelSchema tests
victorgarcia98 Jun 14, 2023
8faf7e5
feat: add ShiftableLoadScheduler
victorgarcia98 Jun 14, 2023
b1472a5
tests: add ShiftableLoadScheduler tests
victorgarcia98 Jun 14, 2023
e32767a
test: add required parameter
victorgarcia98 Jun 14, 2023
c1e9db5
Merge branch 'main' into feature/shiftable-load-scheduler
victorgarcia98 Jun 14, 2023
d26cd53
docs: improve docstrings
victorgarcia98 Jun 15, 2023
3016bdd
Merge branch 'main' into feature/shiftable-load-scheduler
victorgarcia98 Jul 20, 2023
d9151df
fix: pandas 2.0 deprecated argument
victorgarcia98 Jul 20, 2023
48eed71
fix: pandas 2.0 deprecated argument
victorgarcia98 Jul 20, 2023
b54af10
feat: add minimum valuable version of the command flexmeasures add re…
victorgarcia98 Jul 20, 2023
84c23be
Merge branch 'main' into feature/shiftable-load-scheduler
victorgarcia98 Jul 21, 2023
850dd51
docs: add changelog
victorgarcia98 Jul 21, 2023
b099e25
refactor: move TimeIntervalSchema to data.schemas.time
victorgarcia98 Jul 21, 2023
bf8c20d
refactor: rename cost_sensor to consumption_price_sensor
victorgarcia98 Jul 21, 2023
c9496a8
Merge branch 'feature/shiftable-load-scheduler' into feature/cli/trig…
victorgarcia98 Jul 21, 2023
e820ffc
feat: add forbid option
victorgarcia98 Jul 21, 2023
405ca40
docs: add attribute description
victorgarcia98 Jul 21, 2023
c20669c
address change requests
victorgarcia98 Jul 25, 2023
fbffd2e
Merge branch 'main' into feature/shiftable-load-scheduler
victorgarcia98 Jul 25, 2023
5b15830
Merge branch 'feature/shiftable-load-scheduler' into feature/cli/trig…
victorgarcia98 Jul 25, 2023
98d4a16
use consumption_price_sensor from flex_context
victorgarcia98 Jul 25, 2023
a8b9ffa
Merge branch 'feature/shiftable-load-scheduler' into feature/cli/trig…
victorgarcia98 Jul 25, 2023
c4e87f6
communicate consumption_price_sensor through the flex_context
victorgarcia98 Jul 25, 2023
de5e3cc
add clarifying comments
victorgarcia98 Jul 25, 2023
b991863
making block_invalid_starting_times_for_whole_process_scheduling work…
victorgarcia98 Jul 25, 2023
6822958
remove consumption_price_sensor from flex_model
victorgarcia98 Jul 25, 2023
7dc3187
add changelog entry
victorgarcia98 Jul 26, 2023
51de43e
CLI changelog entry
victorgarcia98 Jul 26, 2023
f35fdf7
add flexmeasures add schedule for-shiftable-load to commands list
victorgarcia98 Jul 26, 2023
449ad3f
Merge branch 'feature/shiftable-load-scheduler' into feature/cli/trig…
victorgarcia98 Jul 26, 2023
05c91e3
add docstring for fixture shiftable_load_power_sensor
victorgarcia98 Jul 26, 2023
5938f50
simplify schedule sensor attributes
victorgarcia98 Jul 26, 2023
a673c48
fix flexmeasures add schedule for-shiftable command name
victorgarcia98 Jul 26, 2023
b548ee5
fix: wrong resolution
victorgarcia98 Jul 26, 2023
1529a91
refactor: rename shiftable_load to process
victorgarcia98 Jul 26, 2023
98feddd
Merge branch 'feature/shiftable-load-scheduler' into feature/cli/trig…
victorgarcia98 Jul 26, 2023
6aeb7ef
rename shiftable_load to process
victorgarcia98 Jul 26, 2023
827df19
rename optimization_sense to optimization_direction
victorgarcia98 Jul 27, 2023
9d7e6e8
consistent capitalization of INFLEXIBLE, SHIFTABLE AND BREAKABLE
victorgarcia98 Jul 27, 2023
8f7c276
fix typo
victorgarcia98 Jul 27, 2023
d642b2a
Merge branch 'main' into feature/shiftable-load-scheduler
victorgarcia98 Jul 27, 2023
1e3d164
Merge branch 'feature/shiftable-load-scheduler' into feature/cli/trig…
victorgarcia98 Jul 27, 2023
2123ac5
harmonize capitalization
victorgarcia98 Jul 27, 2023
e5cad35
fix capitalizationof `inflexible-device-sensors`
victorgarcia98 Jul 27, 2023
ab81e41
Merge remote-tracking branch 'origin/feature/shiftable-load-scheduler…
victorgarcia98 Jul 27, 2023
4513904
Merge branch 'feature/shiftable-load-scheduler' into feature/cli/trig…
victorgarcia98 Jul 27, 2023
2cf72a9
update changelog
victorgarcia98 Jul 27, 2023
3769d3c
fix potental bug
victorgarcia98 Jul 27, 2023
ba189c4
add underscores
victorgarcia98 Jul 27, 2023
db85ce4
fix test
victorgarcia98 Jul 28, 2023
92b5d97
rename OptimizationSense to OptimizationDirection
victorgarcia98 Jul 31, 2023
d11a191
fix missing optimization direction renaming
victorgarcia98 Jul 31, 2023
d3d6b9c
Merge branch 'main' into feature/shiftable-load-scheduler
victorgarcia98 Jul 31, 2023
cb8c7c5
Merge branch 'feature/shiftable-load-scheduler' into feature/cli/trig…
victorgarcia98 Jul 31, 2023
159d33b
simplify test
victorgarcia98 Jul 31, 2023
5acdc12
Merge branch 'main' into feature/cli/trigger_shiftable_load
victorgarcia98 Jul 31, 2023
3092389
test run pytest ci
victorgarcia98 Aug 1, 2023
dd1c186
add skip
victorgarcia98 Aug 1, 2023
edd9770
Merge branch 'main' into feature/cli/trigger_shiftable_load
victorgarcia98 Aug 1, 2023
9c99196
fix fixture
victorgarcia98 Aug 1, 2023
b02f01c
add process fixture back
victorgarcia98 Aug 1, 2023
d0b7eb7
add skip
victorgarcia98 Aug 1, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 2 additions & 2 deletions documentation/api/notation.rst
Expand Up @@ -202,13 +202,13 @@ Here are the three types of flexibility models you can expect to be built-in:

For some examples, see the `[POST] /sensors/(id)/schedules/trigger <../api/v3_0.html#post--api-v3_0-sensors-(id)-schedules-trigger>`_ endpoint docs.

2) For **shiftable processes**
2) For **processes**

- ``power``: nominal power of the load.
- ``duration``: time that the load last.
- ``optimization_sense``: objective of the scheduler, to maximize or minimize.
- ``time_restrictions``: time periods in which the load cannot be schedule to.
- ``load_type``: Inflexible, Breakable or Shiftable.
- ``process_type``: INFLEXIBLE, BREAKABLE or SHIFTABLE.


3) For **buffer devices** (e.g. thermal energy storage systems connected to heat pumps), use the same flexibility parameters described above for storage devices. Here are some tips to model a buffer with these parameters:
Expand Down
2 changes: 1 addition & 1 deletion documentation/changelog.rst
Expand Up @@ -17,7 +17,7 @@ New features
* DataSource table now allows storing arbitrary attributes as a JSON (without content validation), similar to the Sensor and GenericAsset tables [see `PR #750 <https://www.github.com/FlexMeasures/flexmeasures/pull/750>`_]
* Added API endpoint `/sensor/<id>` for fetching a single sensor. [see `PR #759 <https://www.github.com/FlexMeasures/flexmeasures/pull/759>`_]
* The CLI now allows to set lists and dicts as asset & sensor attributes (formerly only single values) [see `PR #762 <https://www.github.com/FlexMeasures/flexmeasures/pull/762>`_]
* Add `ProcessScheduler` class and CLI command `flexmeasures add schedule for-process` to optimize the starting time of processes one of the following policies: inflexible, shiftable and breakable [see `PR #729 <https://www.github.com/FlexMeasures/flexmeasures/pull/729>`_ and `PR #768 <https://www.github.com/FlexMeasures/flexmeasures/pull/768>`_]
* Add `ProcessScheduler` class to optimize the starting time of processes one of the policies developed (INFLEXIBLE, SHIFTABLE and BREAKABLE), accessible via the CLI command `flexmeasures add schedule for-process` [see `PR #729 <https://www.github.com/FlexMeasures/flexmeasures/pull/729>`_ and `PR #768 <https://www.github.com/FlexMeasures/flexmeasures/pull/768>`_]

Bugfixes
-----------
Expand Down
8 changes: 5 additions & 3 deletions flexmeasures/cli/data_add.py
Expand Up @@ -1271,11 +1271,13 @@ def add_schedule_process(
"power": process_power,
"time-restrictions": [TimeIntervalSchema().dump(f) for f in forbid],
},
flex_context={
"consumption-price-sensor": consumption_price_sensor.id,
},
)

if consumption_price_sensor is not None:
scheduling_kwargs["flex_context"] = {
"consumption-price-sensor": consumption_price_sensor.id,
}

if as_job:
job = create_scheduling_job(sensor=power_sensor, **scheduling_kwargs)
if job:
Expand Down
35 changes: 28 additions & 7 deletions flexmeasures/cli/tests/conftest.py
Expand Up @@ -2,11 +2,14 @@

from datetime import datetime, timedelta
from pytz import utc
import pandas as pd

from flexmeasures.data.models.data_sources import DataSource
from flexmeasures.data.models.generic_assets import GenericAsset, GenericAssetType
from flexmeasures.data.models.time_series import Sensor, TimedBelief

from flexmeasures.data.models.planning.utils import initialize_index


@pytest.fixture(scope="module")
@pytest.mark.skip_github
Expand Down Expand Up @@ -100,32 +103,50 @@ def reporter_config_raw(app, db, setup_dummy_data):

@pytest.mark.skip_github
@pytest.fixture(scope="module")
def process_power_sensor(db, app):
def process_power_sensor(db, app, setup_markets, setup_sources):
"""
Create an asset of type "ProcessType" and a power sensor to hold the result of
the scheduler.
Create an asset of type "process", power sensor to hold the result of
the scheduler and price data consisting of 8 expensive hours, 8 cheap hours, and again 8 expensive hours-

"""

process_asset_type = GenericAssetType(name="process")

db.session.add(process_asset_type)

processasset = GenericAsset(
name="Test Asset", generic_asset_type=process_asset_type
process_asset = GenericAsset(
name="Test Process Asset", generic_asset_type=process_asset_type
)

db.session.add(processasset)
db.session.add(process_asset)

power_sensor = Sensor(
"power",
generic_asset=processasset,
generic_asset=process_asset,
event_resolution=timedelta(hours=1),
unit="MW",
)

db.session.add(power_sensor)

time_slots = initialize_index(
start=pd.Timestamp("2015-01-02").tz_localize("Europe/Amsterdam"),
end=pd.Timestamp("2015-01-03").tz_localize("Europe/Amsterdam"),
resolution="1H",
)
values = [100] * 8 + [90] * 8 + [100] * 8
beliefs = [
TimedBelief(
event_start=dt,
belief_horizon=timedelta(hours=0),
event_value=val,
source=setup_sources["Seita"],
sensor=setup_markets["epex_da"].corresponding_sensor,
victorgarcia98 marked this conversation as resolved.
Show resolved Hide resolved
)
for dt, val in zip(time_slots, values)
]
db.session.add_all(beliefs)

db.session.commit()

yield power_sensor.id
4 changes: 2 additions & 2 deletions flexmeasures/cli/tests/test_data_add.py
Expand Up @@ -215,10 +215,10 @@ def test_add_reporter(app, db, setup_dummy_data, reporter_config_raw):

@pytest.mark.skip_github
@pytest.mark.parametrize("process_type", [("INFLEXIBLE"), ("SHIFTABLE"), ("BREAKABLE")])
def test_add_process(app, db, process_power_sensor, add_market_prices, process_type):
def test_add_process(app, db, process_power_sensor, process_type):
"""
Schedule a 4h of consumption block at a constant power of 400kW in a day using
the three process policies: inflexible, shiftable and breakable.
the three process policies: INFLEXIBLE, SHIFTABLE and BREAKABLE.
"""

from flexmeasures.cli.data_add import add_schedule_process
Expand Down
30 changes: 15 additions & 15 deletions flexmeasures/data/models/planning/process.py
Expand Up @@ -24,16 +24,16 @@ class ProcessScheduler(Scheduler):
__author__ = "Seita"

def compute(self) -> pd.Series | None:
"""Schedule a prrocess, defined as a `power` and a `duration`, within the specified time window.
"""Schedule a process, defined as a `power` and a `duration`, within the specified time window.
To schedule a battery, please, refer to the StorageScheduler.

For example, this scheduler can plan the start of a process of type `Shiftable` that lasts 5h and requires a power of 10kW.
For example, this scheduler can plan the start of a process of type `SHIFTABLE` that lasts 5h and requires a power of 10kW.
In that case, the scheduler will find the best (as to minimize/maximize the cost) hour to start the process.

This scheduler supports three types of `process_types`:
- Inflexible: this process needs to be scheduled as soon as possible.
- Breakable: this process can be divisible in smaller consumption periods.
- Shiftable: this process can start at any time within the specified time window.
- INFLEXIBLE: this process needs to be scheduled as soon as possible.
- BREAKABLE: this process can be divisible in smaller consumption periods.
- SHIFTABLE: this process can start at any time within the specified time window.

The resulting schedule provides the power flow at each time period.

Expand All @@ -45,9 +45,9 @@ def compute(self) -> pd.Series | None:
power: nominal power of the process.
duration: time that the process last.

optimization_sense: objective of the scheduler, to maximize or minimize.
optimization_direction: objective of the scheduler, to maximize or minimize.
time_restrictions: time periods in which the process cannot be schedule to.
process_type: Inflexible, Breakable or Shiftable.
process_type: INFLEXIBLE, BREAKABLE or SHIFTABLE.

:returns: The computed schedule.
"""
Expand All @@ -66,7 +66,7 @@ def compute(self) -> pd.Series | None:
)
duration: timedelta = self.flex_model.get("duration")
power = self.flex_model.get("power")
optimization_sense = self.flex_model.get("optimization_sense")
optimization_direction = self.flex_model.get("optimization_direction")
process_type: ProcessType = self.flex_model.get("process_type")
time_restrictions = self.flex_model.get("time_restrictions")

Expand Down Expand Up @@ -125,7 +125,7 @@ def compute(self) -> pd.Series | None:
elif process_type == ProcessType.BREAKABLE:
self.compute_breakable(
schedule,
optimization_sense,
optimization_direction,
time_restrictions,
cost,
rows_to_fill,
Expand All @@ -134,7 +134,7 @@ def compute(self) -> pd.Series | None:
elif process_type == ProcessType.SHIFTABLE:
self.compute_shiftable(
schedule,
optimization_sense,
optimization_direction,
start_time_restrictions,
cost,
rows_to_fill,
Expand Down Expand Up @@ -207,16 +207,16 @@ def compute_inflexible(
def compute_breakable(
self,
schedule: pd.Series,
optimization_sense: OptimizationSense,
optimization_direction: OptimizationSense,
time_restrictions: pd.Series,
cost: pd.DataFrame,
rows_to_fill: int,
energy: float,
) -> None:
"""Break up schedule and divide it over the time slots with the largest utility (max/min cost depending on optimization_sense)."""
"""Break up schedule and divide it over the time slots with the largest utility (max/min cost depending on optimization_direction)."""
cost = cost[~time_restrictions].reset_index()

if optimization_sense == OptimizationSense.MIN:
if optimization_direction == OptimizationSense.MIN:
cost_ranking = cost.sort_values(
by=["event_value", "event_start"], ascending=[True, True]
)
Expand All @@ -230,7 +230,7 @@ def compute_breakable(
def compute_shiftable(
self,
schedule: pd.Series,
optimization_sense: OptimizationSense,
optimization_direction: OptimizationSense,
time_restrictions: pd.Series,
cost: pd.DataFrame,
rows_to_fill: int,
Expand All @@ -241,7 +241,7 @@ def compute_shiftable(
cost.rolling(rows_to_fill).sum().shift(-rows_to_fill + 1)
)

if optimization_sense == OptimizationSense.MIN:
if optimization_direction == OptimizationSense.MIN:
start = block_cost[~time_restrictions].idxmin()
else:
start = block_cost[~time_restrictions].idxmax()
Expand Down
6 changes: 3 additions & 3 deletions flexmeasures/data/models/planning/tests/test_process.py
Expand Up @@ -16,7 +16,7 @@
"process_type, optimal_start",
[("INFLEXIBLE", datetime(2015, 1, 2, 0)), ("SHIFTABLE", datetime(2015, 1, 2, 8))],
)
def test_processscheduler(add_battery_assets, process, process_type, optimal_start):
def test_process_scheduler(add_battery_assets, process, process_type, optimal_start):
"""
Test scheduling a process of 4kW of power that last 4h using the ProcessScheduler
without time restrictions.
Expand Down Expand Up @@ -94,7 +94,7 @@ def test_duration_exceeds_planning_window(
assert (schedule == 4).all()


def test_processscheduler_time_restrictions(add_battery_assets, process):
def test_process_scheduler_time_restrictions(add_battery_assets, process):
"""
Test ProcessScheduler with a time restrictions consisting of a block of 2h starting
at 8am. The resulting schedules avoid the 8am-10am period and schedules for a valid period.
Expand Down Expand Up @@ -145,7 +145,7 @@ def test_processscheduler_time_restrictions(add_battery_assets, process):

def test_breakable_scheduler_time_restrictions(add_battery_assets, process):
"""
Test breakable process_type of ProcessScheduler by introducing four 1-hour restrictions
Test BREAKABLE process_type of ProcessScheduler by introducing four 1-hour restrictions
interspaced by 1 hour. The equivalent mask would be the following: [0,...,0,1,0,1,0,1,0,1,0, ...,0].
Trying to get the best prices (between 9am and 4pm), his makes the schedule choose time periods between
the time restrictions.
Expand Down
2 changes: 1 addition & 1 deletion flexmeasures/data/schemas/scheduling/process.py
Expand Up @@ -48,7 +48,7 @@ class ProcessSchedulerFlexModelSchema(Schema):
load_default=[],
)
# objective of the scheduler, to maximize or minimize.
optimization_sense = fields.Enum(
optimization_direction = fields.Enum(
OptimizationSense,
load_default=OptimizationSense.MIN,
data_key="optimization-sense",
Expand Down