New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[reporting] Droplevels pandas reporter #1043
base: main
Are you sure you want to change the base?
Conversation
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Handy. However, this only works if one row corresponds to one event. Simply dropping the index levels other than event_start
doesn't adequately deal with multi-sourced and/or multi-horizon and/or probabilistic beliefs about a single event. We should at least verify that that one row corresponds to one event. We could raise otherwise, but preferably, we would handle those other cases, too. We could pick the most recent belief (not grouped by source, which is what the corresponding timely-beliefs filter is doing) and make it deterministic. That way, in case of multiple sources having a belief about an event, we get the most recent one. For example, a measurement rather than a schedule/forecast.
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
True, here the use actually is for when we are operating with different inputs that don't share belief_time, source and event_start. For instance, operations like sum fail if they don't have the exact same index. Said that, I agree with you. I'm adding an assert to make sure that we are not accidentally getting more than one row per event. Regarding the support more ways to reduce the multi index, I think we should tackle that issue in a separate ticket/corresponding PR. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks. I'd only recommend still adding a changelog entry. Given the potential for existing reporters to start raising exceptions, it's nice if the changelog already gives a hint for potential debugging.
Description
Reporters can have input sensors with different sources, for example, one power sensor with a source of type
Scheduler
and another one of typeUser
. This makes simple operations like summing their values fail. For this reason, we prepended most of our reporters with an instruction to drop all indexes butevent_start
. This PR introduces a little feature to automate this process with a new flag to control its behavior.Related Items
#869