Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incompatible search filters with most_recent_beliefs_only and most_recent_events_only #97

Closed
2 tasks done
Flix6x opened this issue Jan 17, 2022 · 0 comments · Fixed by #147
Closed
2 tasks done

Comments

@Flix6x
Copy link
Collaborator

Flix6x commented Jan 17, 2022

It turns out that the most_recent_beliefs_only and most_recent_events_only subqueries in timely-beliefs are missing the optional filters that are also applied to the main query.

Unfortunately, I also found an incompatibility. Two of those filters only roughly filter within the query, and only properly filter on the query results: namely beliefs_after and beliefs_before. They do so because they require taking into account the sensor's knowledge horizon function, which are python functions and not suitable for injection into the SQL query. The SQL filters essentially introduce some slack around the restricted range of belief times, with the size of this slack equal to the bounds of the specific knowledge horizon function. For example, the knowledge horizon for events of a day-ahead market with a gate closure at noon is normally bounded between 12 and 36 hours (to account for possibly double daylight savings time transitions this is even widened to between 10 and 38 hours). In any case, this messes up selecting the minimum horizon in the subquery, especially for economic sensors with non-trivial knowledge horizons (physical sensors are more straightforward and typically lead to no slack being introduced).

Originally posted by @Flix6x in FlexMeasures/flexmeasures#312 (comment)

My suggested approach to resolve this is:

  • Add the optional filter from the main query to both subqueries.
  • Avoid using the most_recent_beliefs_only subquery in the special case of querying with the most_recent_beliefs_only and beliefs_after (and/or beliefs_before) filters on sensors with unsuitable knowledge horizon functions, and use belief_utils.select_most_recent_belief on the query results instead. Resolved in Fix incompatible query filters #117.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant