Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

481 extend documentation and toy tutorial with scheduling within the context of a building with PV #534

Conversation

Flix6x
Copy link
Contributor

@Flix6x Flix6x commented Nov 15, 2022

Most of this PR ended up being about making it technically feasible to create the envisioned chart design in #481, and also making more sensible use of the color encoding and data source attribute. Here's roughly what I did:

Expand tutorial:

  • Expand toy tutorial with creation of solar sensor and solar forecasts from CSV
  • Expand toy tutorial with new schedule that takes into account solar
  • Check whether scheduler now takes into account the limited headroom (yay, the new scheduling feature worked without having to touch the scheduler logic)
  • Add a chart showing the new schedule [Moved to Expand toy tutorial with a sensor that reports available headroom #584]

Try to create the envisioned chart design:

  • Allow nesting sensors in sensors_to_show, to denote that multiple sensors should be shown together, and deal with all of its implications (axis labels, use of colors, etc.)
  • Visually distinguish forecasts, schedules and measurements by source type rather than by belief horizon (incl. a simple db migration)
  • Add a CLI command to create a new DataSource (so that forecasts are visually presented as originating from a forecaster)
  • Add a 'capacity' or 'headroom' sensor whose data is derived from an Asset attribute (capacity) in combination with sensor data (in this case that would only be the solar sensor). I feel that showing the headroom (like in the envisioned chart design) is what makes the example click. I don't have this (yet), but I feel we should first have a work session on derived sensor data (and/or KPIs) before I continue. [Moved to Expand toy tutorial with a sensor that reports available headroom #584]

Current status of the chart:

  • For the toy tutorial:

image

  • For V2G use case:

image

I also encountered a number of fixes that we should definitely cherry-pick for the 0.12 release (see #545). Specifically:

Needs SeitaBV/timely-beliefs#122.
Closes #481.

Signed-off-by: F.N. Claessen <felix@seita.nl>
…ames)

Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
…nger assume utc, but raise instead (see PR #521)

Signed-off-by: F.N. Claessen <felix@seita.nl>
…s to be updated by simply running `flexmeasures add toy-account` again

Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
…able headroom for the battery

Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x added documentation Improvements or additions to documentation CLI labels Nov 15, 2022
@Flix6x Flix6x added this to the 0.13.0 milestone Nov 15, 2022
@Flix6x Flix6x self-assigned this Nov 15, 2022
Signed-off-by: F.N. Claessen <felix@seita.nl>
@coveralls
Copy link
Collaborator

coveralls commented Nov 15, 2022

Pull Request Test Coverage Report for Build 4677953700

  • 59 of 164 (35.98%) changed or added relevant lines in 12 files are covered.
  • 6 unchanged lines in 3 files lost coverage.
  • Overall coverage decreased (-0.3%) to 64.5%

Changes Missing Coverage Covered Lines Changed/Added Lines %
flexmeasures/api/v3_0/assets.py 1 2 50.0%
flexmeasures/data/schemas/generic_assets.py 18 19 94.74%
flexmeasures/utils/flexmeasures_inflection.py 1 2 50.0%
flexmeasures/data/services/data_sources.py 2 4 50.0%
flexmeasures/data/models/data_sources.py 0 3 0.0%
flexmeasures/cli/data_show.py 0 4 0.0%
flexmeasures/data/models/generic_assets.py 3 12 25.0%
flexmeasures/cli/data_add.py 9 24 37.5%
flexmeasures/data/services/utils.py 8 30 26.67%
flexmeasures/data/models/charts/belief_charts.py 8 55 14.55%
Files with Coverage Reduction New Missed Lines %
flexmeasures/cli/data_add.py 1 33.27%
flexmeasures/data/models/generic_assets.py 1 45.52%
flexmeasures/data/models/charts/belief_charts.py 4 11.21%
Totals Coverage Status
Change from base Build 4677372721: -0.3%
Covered Lines: 6940
Relevant Lines: 10012

💛 - Coveralls

@Flix6x Flix6x mentioned this pull request Nov 15, 2022
…ion-and-toy-tutorial-with-scheduling-within-the-context-of-a-building-with-pv

# Conflicts:
#	documentation/tut/toy-example-from-scratch.rst
#	flexmeasures/cli/data_add.py
…th-scheduling-within-the-context-of-a-building-with-pv
… a single row within the multi-row asset chart

Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
…. no job)

Signed-off-by: F.N. Claessen <felix@seita.nl>
…hare a name don't share a color (note that sensors belonging to the same asset aren't allowed to have the same name)

Signed-off-by: F.N. Claessen <felix@seita.nl>
documentation/tut/toy-example-from-scratch.rst Outdated Show resolved Hide resolved
documentation/tut/toy-example-from-scratch.rst Outdated Show resolved Hide resolved
documentation/tut/toy-example-from-scratch.rst Outdated Show resolved Hide resolved
--start ${TOMORROW}T07:00+01:00 --duration PT12H \
--soc-at-start 50% --roundtrip-efficiency 90%
New schedule is stored.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running the tutorial with a fresh DB creates different assets to those shown in the tutorial.

Creating Toy Account

Input:

flexmeasures add toy-account --kind battery

Output:

  Created <GenericAsset None: 'toy-battery' (battery)>
  Created discharging
  Adding transmission zone type ...
  Adding NL transmission zone ...
  Created day-ahead prices
  Created <GenericAsset None: 'toy-solar' (solar)>
  Created production
  Toy account Toy Account with user toy-user@flexmeasures.io created successfully. You might want to run `flexmeasures show account --id 1`
  The sensor recording battery discharging is discharging (ID: 2).
  The sensor recording day-ahead prices is day-ahead prices (ID: 3).
  The sensor recording solar forecasts is production (ID: 4).

Showing account

Input:

flexmeasures show account --id 1

Output

===========================
Account Toy Account (ID: 1)
===========================

Account has no roles.

All users:
 
  ID  Name      Email                     Last Login    Roles
----  --------  ------------------------  ------------  -------------
   1  toy-user  toy-user@flexmeasures.io  None          account-admin

All assets:
 
  ID  Name         Type     Location
----  -----------  -------  -----------------
   1  toy-battery  battery  (52.374, 4.88969)
   3  toy-solar    solar    (52.374, 4.88969)

In the tutorial, here appears toy-building, as well.

Flix6x added 14 commits April 5, 2023 22:28
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
…e choice of y-axis ticks)

Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>

# Conflicts:
#	flexmeasures/data/queries/data_sources.py
This reverts commit b393cad.
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x linked an issue Apr 9, 2023 that may be closed by this pull request
@Flix6x Flix6x mentioned this pull request Apr 9, 2023
4 tasks
@nhoening
Copy link
Contributor

I am still getting the following error when I add a schedule (on different laptop, using a fresh database):

sqlalchemy.exc.InvalidRequestError: Can't attach instance <Sensor at 0x7f6257982550>; another instance with key (<class 'flexmeasures.data.models.time_series.Sensor'>, (1,), None) is already present in this session.

I wonder if you guys have also tried to compute both types of schedules in one run through the tutorial? With --inflexible-device-sensor and without? Maybe you're only testing the former option right now?

It could also be that this behavior is not caused by this PR (and we can merge it), but by some other artifact we need to address some day in another PR. But this is a good opportunity to describe it well.

I can now explain a bit more what causes this for me:

  • If I do either way first, the other way will fail, like described above.
  • If I re-do the same way, it does not fail.
  • If I delete the computed belief data (delete from timed_belief where sensor_id = 1;), then computing the schedule in whichever of the two ways will work again. So it has to do with existing data in the database.
  • The complaint is always about sensor 1, in both cases.

Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
@nhoening
Copy link
Contributor

As my previous research showed, an error occurs if there is previous data. It does not seem to matter whether the extra sensor is used in computation or not. What might be the culprit here, is that the pre-existing data is not the same as the new data that would overwrite it (I'm computing a schedule for a time interval which already has a schedule).

So I propose to open a new issue to investigate this. Overwriting of pre-existing data might be broken.

@victorgarcia98
Copy link
Contributor

My requests were addressed.

Thanks for updating the tutorial @Flix6x :D

Copy link
Contributor

@nhoening nhoening left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work!

…ion-and-toy-tutorial-with-scheduling-within-the-context-of-a-building-with-pv
@Flix6x Flix6x merged commit 81ea6c3 into main Apr 12, 2023
5 checks passed
@Flix6x Flix6x deleted the 481-extend-documentation-and-toy-tutorial-with-scheduling-within-the-context-of-a-building-with-pv branch April 12, 2023 12:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLI documentation Improvements or additions to documentation Scheduling
Projects
None yet
4 participants