Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FLEXMEASURES_MAX_PLANNING_HORIZON can be int #583

Conversation

Flix6x
Copy link
Contributor

@Flix6x Flix6x commented Jan 12, 2023

Closes #570.

Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x added this to the 0.13.0 milestone Jan 12, 2023
@Flix6x Flix6x self-assigned this Jan 12, 2023
@Flix6x Flix6x linked an issue Jan 12, 2023 that may be closed by this pull request
@coveralls
Copy link
Collaborator

coveralls commented Jan 12, 2023

Pull Request Test Coverage Report for Build 3943213675

  • 18 of 19 (94.74%) changed or added relevant lines in 4 files are covered.
  • 1 unchanged line in 1 file lost coverage.
  • Overall coverage increased (+0.02%) to 65.362%

Changes Missing Coverage Covered Lines Changed/Added Lines %
flexmeasures/data/models/planning/storage.py 3 4 75.0%
Files with Coverage Reduction New Missed Lines %
flexmeasures/data/models/planning/storage.py 1 88.4%
Totals Coverage Status
Change from base Build 3943014511: 0.02%
Covered Lines: 6847
Relevant Lines: 9835

💛 - Coveralls

…divisible by 1-10, which yields pleasant-looking durations for common sensor resolutions

Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x marked this pull request as ready for review January 13, 2023 12:32
@Flix6x Flix6x requested a review from nhoening January 13, 2023 12:32

Default: ``timedelta(days=7, hours=1)``
Default: ``2520`` (e.g. 7 days for a 4-minute resolution sensor, 105 days for a 1-hour resolution sensor)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was just thinking about this, and it seems to me that this is a rather large default horizon. This will eat a lot of CPU budget.
To me, ~10% of this is reasonable (a bit over 2 weeks for a 1-hour sensor, a day for a 5-minute sensor)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also a thought for the future: this might have to change per account (at first via an attribute, I guess), as this is possibly relevant for billing flex scheduling services.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was just thinking about this, and it seems to me that this is a rather large default horizon. This will eat a lot of CPU budget.

This is not the default horizon (which is currently set to 2 days), but the default max horizon. I agree that this is a rather large value, but it is what the FlexMeasures scheduler can currently handle without the scheduling jobs timing out after 180 seconds.

I also thought of coupling this PR with the introduction of a config setting to allow FlexMeasures hosts to overwrite python-rq's default job timeout of 180 seconds for the scheduling queue, but I decided to save that for a separate PR. That would be a simple change in app.py, something like:

Queue(name="scheduling", default_timeout=app.config["FLEXMEASURES_PLANNING_TIMEOUT"])

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yes, sorry for the oversight. Actually, I would advise we give FLEXMEASURES_PLANNING_HORIZON the same treatment (that it can be an int).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about that, too, but I didn't find the use case particularly convincing. Is it a worthwhile extra feature, or just resolves an itchy asymmetry?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The latter. But these things matter for user experience. I would expect these two to work the same.
And I believe we are not breaking existing settings here. Now is the cheapest time to do this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now is the cheapest time to do this.

I started implementing your request, but it turns out this is probably not true. The FLEXMEASURES_PLANNING_HORIZON is used in API versions 1.2 and 1.3, which allow retrieving schedules for multiple sensors in a single API call. That code assumes that the planning horizon is the same for all of these sensors, which with an integer FLEXMEASURES_PLANNING_HORIZON would no longer be the case. So this would currently require refactoring of already deprecated code.

I'd say it is cheaper to implement your request after these API versions are sunset.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds reasonable, can we make a ticket for version 0.14?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flexmeasures/data/models/planning/storage.py Outdated Show resolved Hide resolved
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x requested a review from nhoening January 13, 2023 13:25
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x merged commit d742749 into main Jan 17, 2023
@Flix6x Flix6x deleted the 570-take-resolution-into-account-for-maximum-size-of-planning-window branch January 17, 2023 21:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Take resolution into account for maximum size of planning window
3 participants