Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue 371 cli command for creating a schedule #372

Merged
merged 43 commits into from Mar 4, 2022

Conversation

Flix6x
Copy link
Contributor

@Flix6x Flix6x commented Feb 23, 2022

@nhoening to test out, I suggest you adjust this to fit your tutorial:

flexmeasures add schedule --sensor-id 5 --factor-id 13 --from 2021-10-09T12:00+02:00 --until 2021-10-09T22:00+02:00 --soc-at-start 0.03 --soc-target 0.03 2021-10-09T14:00:00+02:00 --soc-target 0.03 2021-10-09T19:00+02:00 --soc-min 0.01 --soc-max 0.06

I hope it fits what you had in mind.

Are you planning to add an automated test that follows your tutorial?

closes #371

Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
…tions, too

Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x added the CLI label Feb 23, 2022
@Flix6x Flix6x self-assigned this Feb 23, 2022
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x marked this pull request as ready for review February 23, 2022 15:21
@Flix6x Flix6x requested a review from nhoening February 23, 2022 15:23
@coveralls
Copy link
Collaborator

coveralls commented Feb 23, 2022

Pull Request Test Coverage Report for Build 1935835862

  • 62 of 107 (57.94%) changed or added relevant lines in 7 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-0.2%) to 68.721%

Changes Missing Coverage Covered Lines Changed/Added Lines %
flexmeasures/data/models/planning/utils.py 3 4 75.0%
flexmeasures/utils/unit_utils.py 8 9 88.89%
flexmeasures/data/schemas/units.py 16 21 76.19%
flexmeasures/cli/data_add.py 20 58 34.48%
Totals Coverage Status
Change from base Build 1935252524: -0.2%
Covered Lines: 6753
Relevant Lines: 9329

💛 - Coveralls

@nhoening
Copy link
Contributor

Does soc translate well to assets other than batteries?

I was planning on an automated test for the toy-account CLI command. I'd expect we make a test only for this if it's not too much trouble.

@Flix6x
Copy link
Contributor Author

Flix6x commented Feb 23, 2022

Does soc translate well to assets other than batteries?

No, not to assets with shifting flexibility, for example, but it does translate well to assets with storage or buffer flexibility.

I was planning on an automated test for the toy-account CLI command. I'd expect we make a test only for this if it's not too much trouble.

Okay. And do the current CLI options work well for the tutorial, or would you like to see adjustments / smarter defaults? For example, starting the schedule now and ending the schedule some default duration from now.

@nhoening
Copy link
Contributor

Two ideas to simplify:

  • --until could maybe become a --duration parameter. Duration (e.g. 4h or PT4H) is simpler than a complete datetime
  • For the soc specs, can either --soc-at-start, --soc-min and --soc-max be left out - i.e. can they in principle be inferred from asset/sensor/beliefs data? For instance, maybe the last two could also be asset attributes?

@nhoening
Copy link
Contributor

Could --from be optional, and then FlexMeasures selects the next sensible interval (given the sensor resolution)? Can also be a ticket.

@nhoening
Copy link
Contributor

For flexibility, but probably also for ease of use in many use cases, I believe it might be helpful to use soc values as percentages.

  • a lot, but not many use cases are electricity
  • percentages are often what we have in mind when we sketch a scheduling problem (certainly with min and max, but possibly also with the starting state and targets).

…formation available?

Signed-off-by: Nicolas Höning <nicolas@seita.nl>
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
@nhoening
Copy link
Contributor

This review is not in-line, but rather comments, sorry. Most of them are about making it slightly easier to use.

I made a few light additions to the CLI command, and one small fix.

Also, here is a successful run.

I believe it seems to work, I only have to small observations on the generated data, which get me a bit confused. For the tutorial, we could show the schedule as it is visualised in the UI, so I'm happy to see as issues handled outside of this PR if that works better:

Here are my asset and charging sensor:

a1@127:a1> select * from sensor where id = 242;
+------+----------+--------+------------------+--------------------+-------------------------+----------------------------+--------------------+--------------+
| id   | name     | unit   | timezone         | event_resolution   | knowledge_horizon_fnc   | knowledge_horizon_par      | generic_asset_id   | attributes   |
|------+----------+--------+------------------+--------------------+-------------------------+----------------------------+--------------------+--------------|
| 242  | charging | kW     | Europe/Amsterdam | 0:15:00            | ex_post                 | {"ex_post_horizon": "P0D"} | 171                | {}           |
+------+----------+--------+------------------+--------------------+-------------------------+----------------------------+--------------------+--------------+
SELECT 1
Time: 0.012s
a1@127:a1> select * from generic_asset where id = 171;
+------+-------------+------------+-------------+-------------------------+--------------+-------------------------------------------------------------------------------+
| id   | name        | latitude   | longitude   | generic_asset_type_id   | account_id   | attributes                                                                    |
|------+-------------+------------+-------------+-------------------------+--------------+-------------------------------------------------------------------------------|
| 171  | toy-battery | 52.374     | 4.88969     | 3                       | 35           | {"capacity_in_mw": 0.005, "min_soc_in_mwh": 0.0005, "max_soc_in_mwh": 0.0045} |
+------+-------------+------------+-------------+-------------------------+--------------+-------------------------------------------------------------------------------+

I used these prices:

+------------------------+-------------------------+--------------------------+---------------+-------------+-------------+
| event_start            | belief_horizon          | cumulative_probability   | event_value   | sensor_id   | source_id   |
|------------------------+-------------------------+--------------------------+---------------+-------------+-------------|
| 2022-02-24 00:00:00+00 | -1 day, 3:07:39.306486  | 0.5                      | 10.0          | 239         | 148         |
| 2022-02-24 01:00:00+00 | -1 day, 4:07:39.306486  | 0.5                      | 11.0          | 239         | 148         |
| 2022-02-24 02:00:00+00 | -1 day, 5:07:39.306486  | 0.5                      | 12.0          | 239         | 148         |
| 2022-02-24 03:00:00+00 | -1 day, 6:07:39.306486  | 0.5                      | 15.0          | 239         | 148         |
| 2022-02-24 04:00:00+00 | -1 day, 7:07:39.306486  | 0.5                      | 18.0          | 239         | 148         |
| 2022-02-24 05:00:00+00 | -1 day, 8:07:39.306486  | 0.5                      | 17.0          | 239         | 148         |
| 2022-02-24 06:00:00+00 | -1 day, 9:07:39.306486  | 0.5                      | 10.5          | 239         | 148         |
| 2022-02-24 07:00:00+00 | -1 day, 10:07:39.306486 | 0.5                      | 9.0           | 239         | 148         |
| 2022-02-24 08:00:00+00 | -1 day, 11:07:39.306486 | 0.5                      | 9.5           | 239         | 148         |
| 2022-02-24 09:00:00+00 | -1 day, 12:07:39.306486 | 0.5                      | 9.0           | 239         | 148         |
| 2022-02-24 10:00:00+00 | -1 day, 13:07:39.306486 | 0.5                      | 8.5           | 239         | 148         |
| 2022-02-24 11:00:00+00 | -1 day, 14:07:39.306486 | 0.5                      | 10.0          | 239         | 148         |
| 2022-02-24 12:00:00+00 | -1 day, 15:07:39.306486 | 0.5                      | 8.0           | 239         | 148         |
| 2022-02-24 13:00:00+00 | -1 day, 16:07:39.306486 | 0.5                      | 5.0           | 239         | 148         |
| 2022-02-24 14:00:00+00 | -1 day, 17:07:39.306486 | 0.5                      | 4.0           | 239         | 148         |
| 2022-02-24 15:00:00+00 | -1 day, 18:07:39.306486 | 0.5                      | 4.0           | 239         | 148         |
| 2022-02-24 16:00:00+00 | -1 day, 19:07:39.306486 | 0.5                      | 5.5           | 239         | 148         |
| 2022-02-24 17:00:00+00 | -1 day, 20:07:39.306486 | 0.5                      | 8.0           | 239         | 148         |
| 2022-02-24 18:00:00+00 | -1 day, 21:07:39.306486 | 0.5                      | 12.0          | 239         | 148         |
| 2022-02-24 19:00:00+00 | -1 day, 22:07:39.306486 | 0.5                      | 13.0          | 239         | 148         |
| 2022-02-24 20:00:00+00 | -1 day, 23:07:39.306486 | 0.5                      | 14.0          | 239         | 148         |
| 2022-02-24 21:00:00+00 | 0:07:39.306486          | 0.5                      | 12.5          | 239         | 148         |
| 2022-02-24 22:00:00+00 | 1:07:39.306486          | 0.5                      | 10.0          | 239         | 148         |
| 2022-02-24 23:00:00+00 | 2:07:39.306486          | 0.5                      | 7.0           | 239         | 148         |
+------------------------+-------------------------+--------------------------+---------------+-------------+-------------+

With this command:

flexmeasures add schedule --sensor-id 242 --factor-id 239 --from 2022-02-24T03:00+02:00 --until 2022-02-24T16:00:00+02:00 --soc-at-start 0.002 --roundtrip-efficiency .9

lead to this schedule:

+------------------------+--------------------------+--------------------------+-----------------------+-------------+-------------+
| event_start            | belief_horizon           | cumulative_probability   | event_value           | sensor_id   | source_id   |
|------------------------+--------------------------+--------------------------+-----------------------+-------------+-------------|
| 2022-02-24 01:00:00+00 | -3 days, 7:51:48.568804  | 0.5                      | -0.005                | 242         | 27          |
| 2022-02-24 01:15:00+00 | -3 days, 8:06:48.568804  | 0.5                      | -0.005                | 242         | 27          |
| 2022-02-24 01:30:00+00 | -3 days, 8:21:48.568804  | 0.5                      | -5.948686e-12         | 242         | 27          |
| 2022-02-24 01:45:00+00 | -3 days, 8:36:48.568804  | 0.5                      | 0.0                   | 242         | 27          |
| 2022-02-24 02:00:00+00 | -3 days, 8:51:48.568804  | 0.5                      | 0.0                   | 242         | 27          |
| 2022-02-24 02:15:00+00 | -3 days, 9:06:48.568804  | 0.5                      | 0.0                   | 242         | 27          |
| 2022-02-24 02:30:00+00 | -3 days, 9:21:48.568804  | 0.5                      | 0.0                   | 242         | 27          |
| 2022-02-24 02:45:00+00 | -3 days, 9:36:48.568804  | 0.5                      | 0.0                   | 242         | 27          |
| 2022-02-24 03:00:00+00 | -3 days, 9:51:48.568804  | 0.5                      | 1.0000004e-12         | 242         | 27          |
| 2022-02-24 03:15:00+00 | -3 days, 10:06:48.568804 | 0.5                      | 1.0000004e-12         | 242         | 27          |
| 2022-02-24 03:30:00+00 | -3 days, 10:21:48.568804 | 0.5                      | 1.0000004e-12         | 242         | 27          |
| 2022-02-24 03:45:00+00 | -3 days, 10:36:48.568804 | 0.5                      | 1.0000004e-12         | 242         | 27          |
| 2022-02-24 04:00:00+00 | -3 days, 10:51:48.568804 | 0.5                      | 0.001                 | 242         | 27          |
| 2022-02-24 04:15:00+00 | -3 days, 11:06:48.568804 | 0.5                      | 0.005                 | 242         | 27          |
| 2022-02-24 04:30:00+00 | -3 days, 11:21:48.568804 | 0.5                      | 0.005                 | 242         | 27          |
| 2022-02-24 04:45:00+00 | -3 days, 11:36:48.568804 | 0.5                      | 0.005                 | 242         | 27          |
| 2022-02-24 05:00:00+00 | -3 days, 11:51:48.568804 | 0.5                      | 0.0                   | 242         | 27          |
| 2022-02-24 05:15:00+00 | -3 days, 12:06:48.568804 | 0.5                      | 0.0                   | 242         | 27          |
| 2022-02-24 05:30:00+00 | -3 days, 12:21:48.568804 | 0.5                      | 0.0                   | 242         | 27          |
| 2022-02-24 05:45:00+00 | -3 days, 12:36:48.568804 | 0.5                      | 0.0                   | 242         | 27          |
| 2022-02-24 06:00:00+00 | -3 days, 12:51:48.568804 | 0.5                      | -1.9486834e-12        | 242         | 27          |
| 2022-02-24 06:15:00+00 | -3 days, 13:06:48.568804 | 0.5                      | 0.0                   | 242         | 27          |
| 2022-02-24 06:30:00+00 | -3 days, 13:21:48.568804 | 0.5                      | 0.0                   | 242         | 27          |
| 2022-02-24 06:45:00+00 | -3 days, 13:36:48.568804 | 0.5                      | 0.0                   | 242         | 27          |
| 2022-02-24 07:00:00+00 | -3 days, 13:51:48.568804 | 0.5                      | -1.1384196e-11        | 242         | 27          |
| 2022-02-24 07:15:00+00 | -3 days, 14:06:48.568804 | 0.5                      | 9.486833e-13          | 242         | 27          |
| 2022-02-24 07:30:00+00 | -3 days, 14:21:48.568804 | 0.5                      | 9.486833e-13          | 242         | 27          |
| 2022-02-24 07:45:00+00 | -3 days, 14:36:48.568804 | 0.5                      | 9.486833e-13          | 242         | 27          |
| 2022-02-24 08:00:00+00 | -3 days, 14:51:48.568804 | 0.5                      | 9.486833e-13          | 242         | 27          |
| 2022-02-24 08:15:00+00 | -3 days, 15:06:48.568804 | 0.5                      | 9.486833e-13          | 242         | 27          |
| 2022-02-24 08:30:00+00 | -3 days, 15:21:48.568804 | 0.5                      | 9.486833e-13          | 242         | 27          |
| 2022-02-24 08:45:00+00 | -3 days, 15:36:48.568804 | 0.5                      | 9.486833e-13          | 242         | 27          |
| 2022-02-24 09:00:00+00 | -3 days, 15:51:48.568804 | 0.5                      | 9.486833e-13          | 242         | 27          |
| 2022-02-24 09:15:00+00 | -3 days, 16:06:48.568804 | 0.5                      | 9.486833e-13          | 242         | 27          |
| 2022-02-24 09:30:00+00 | -3 days, 16:21:48.568804 | 0.5                      | 9.486833e-13          | 242         | 27          |
| 2022-02-24 09:45:00+00 | -3 days, 16:36:48.568804 | 0.5                      | 9.486833e-13          | 242         | 27          |
| 2022-02-24 10:00:00+00 | -3 days, 16:51:48.568804 | 0.5                      | -0.005                | 242         | 27          |
| 2022-02-24 10:15:00+00 | -3 days, 17:06:48.568804 | 0.5                      | -0.005                | 242         | 27          |
| 2022-02-24 10:30:00+00 | -3 days, 17:21:48.568804 | 0.5                      | -0.005                | 242         | 27          |
| 2022-02-24 10:45:00+00 | -3 days, 17:36:48.568804 | 0.5                      | -0.001                | 242         | 27          |
| 2022-02-24 11:00:00+00 | -3 days, 17:51:48.568804 | 0.5                      | 0.0010000000009486834 | 242         | 27          |

In general, I see it works: charge in the first hour (cheapest), discharge at 4 to 5pm (highest price).

  • I computed it on 26th of February around 6pm. Is the horizon not somewhat off? I'd expected -2days plus a few hours. I the price horizon subtracted?
  • I gave a roundtrip-efficiency, and I see several zeroes which is good, but it still seems that there are very small values in there (e.g. 9.486833e-13). Can we get rid of them (also makes it easier to understand for first-time users).

Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
… beliefs)

Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Copy link
Contributor Author

Flix6x commented Feb 28, 2022

I computed it on 26th of February around 6pm. Is the horizon not somewhat off? I'd expected -2days plus a few hours. I the price horizon subtracted?

I checked the code to see where the horizon was being determined. The horizon was indeed off, but only by the knowledge horizon (in your case, that should have been just 15 minutes). I fixed this just now in b6603de. I believe an explanation for the remaining discrepancy (you expected something like 2 days instead of 3 days) should be found in your interpretation of printed negative timedeltas (which, I'll admit, is definitely confusing):

>>> import pandas as pd
>>> pd.Timedelta("-60H")  # 2.5 days
-3 days +12:00:00

Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
… transitions

Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x requested a review from nhoening February 28, 2022 16:36
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
Copy link
Contributor

@nhoening nhoening left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the improvements! The only things I'd do (if possible) would be

  • add the command to changelog and cli/changelog
  • change factor-id to optimization-context-id
  • write a quick test, but I believe there is no quick test for this. We probably are testing the scheduling code already somewhere else in API tests. The only thing to be tested really would be if the command translates CLI input to function parameters, which could be done with a monkeypatch of make_schedule. But I'm okay with not having that, so we can move on snd use this.

@nhoening
Copy link
Contributor

nhoening commented Mar 1, 2022

One more thing: I now realise that soc is in percent, while roundtrip efficiency is given in a [0,1] range.

What would you recommend ― are both of these the most natural units?

…rshmallow

Signed-off-by: F.N. Claessen <felix@seita.nl>
…or ratios for the round-trip efficiency

Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x requested a review from nhoening March 4, 2022 16:29
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x merged commit 8ad4802 into main Mar 4, 2022
@Flix6x Flix6x deleted the Issue-371_CLI_command_for_creating_a_schedule branch March 4, 2022 21:02
@Flix6x Flix6x added this to the 0.9.0 milestone Mar 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CLI command for creating a schedule
3 participants