feat(ENTSOE.py): Parse and combine both kinds of exchange forecasts #6577
Conversation
VIKTORVAV99
changed the title
| @@ -926,13 +936,45 @@ def parse_exchange( | |||
| datetime_start = datetime.fromisoformat( | |||
| zulu_to_utc(timeseries.find_all("start")[0].contents[0]) | |||
| ) | |||
| # Only use contract_marketagreement.type == A01 (Total to avoid double counting some columns) | |||
There was a problem hiding this comment.
What about this? Are we not risking the double counting if we start ingesting the other forecasts?
There was a problem hiding this comment.
Both kinds are returned in the same API call so in order to parse it we need to make this dynamic.
Right now it calls the parser function twice, first for the day-ahead values and a second time for the total values to then combine them.
Functionally it should do the same but just use a dynamic config instead of a static value.
|
|
||
| return ExchangeList(logger).merge_exchanges(raw_exchange_lists, logger) | ||
| if raw_exchange is None: |
There was a problem hiding this comment.
Can't we separate more clearly the forecast and non forecasts cases?
There was a problem hiding this comment.
We can but it would likely result in more code duplication, if that's okay I can do right ahead and split them apart further.
| marketAgreementType = timeseries.find("contract_marketagreement.type").contents[ | ||
| 0 | ||
| ] | ||
| if marketAgreementType and marketAgreementType != market_type.value: |
There was a problem hiding this comment.
@pierresegonne here is the functional equivalent of the previous check.
Issue
Currently we are only parsing the Total forecasts which are the most accurate but this means we are missing out on the day ahead forecasts which are available further into the future than the total forecasts are.
Description
This PR duplicates some functionality for the sake of simple logic in order to parse the day_ahead forecasts and then update them with the total forecasts so we get benefits of the longer day_ahead forecasts while retaining the shorter term accuracy of the total forecasts.
TODO:
Preview
Latest value on master
{'datetime': datetime.datetime(2024, 3, 23, 22, 0, tzinfo=datetime.timezone.utc), 'netFlow': 0.0, 'sortedZoneKeys': 'EE->FI', 'source': 'entsoe.eu', 'sourceType': <EventSourceType.forecasted: 'forecasted'>}Latest value on this branch
{'datetime': datetime.datetime(2024, 3, 24, 22, 0, tzinfo=datetime.timezone.utc), 'netFlow': -358.0, 'sortedZoneKeys': 'EE->FI', 'source': 'entsoe.eu', 'sourceType': <EventSourceType.forecasted: 'forecasted'>}Double check
poetry run test_parser "zone_key"pnpx prettier@2 --write .andpoetry run formatin the top level directory to format my changes.