Reporting backend infrastructure

[nhoening]

We want to be able to automatically produce data derived from existing data, for reporting.
An example are KPIs, but often it's data which is used to better explain to users what happened in the past.
These reports should be triggerable by a CLI command, so we can periodically create and persist reporting data.

Work on this issue will add the CLI command and four generic report functions will be added, which can build many straightforward formulas.

Furthermore, we can lay the foundation for a function store for FlexMeausures and plugins to use.

Outcomes:

• A new CLI command to run an existing report function: flexmeasures add report --name <str> [--input-sensors <Union[List[int], Dict[str, int]]] --output-sensor <int> [--preferred-unit <str>] [--last-hour <bool>] [--yesterday <bool>] [last-week <bool>] [last-month <bool>] Note: Many arguments are optional, because (a) only one time range identifier is needed and (b) custom reports will often decide sensor and units themselves.
• A function store for reporting functions, which we'll keep in app.config["func_store"]["reports"]. Functions are loaded with the util logic we've just developed for Using custom scheduling algorithms #9. Also provide a utility function register_function_in_store(app: Flask, name: str, func: Callable, store: str).
• Four simple implementations, inspired by earlier work we did in our E-mission plugin. These are registered when FlexMeasures starts up. One is add_andor_multiply (from E-mission), the others add_andor_divide, subtract_andor_multiply and subtract_andor_divide.
• Documentation for plugins how they can add their own reporting functions (by writing a function adhering to the signature we impose and adding it to __functions__.py) under REPORTS, a list (we could also enable to use SCHEDULERS, actually, a dict).

Potentially we can keep the function store out of this, but it might be necessary in #9 anyways.

[Flix6x]

After refactoring scheduling functionality into Scheduler classes (rather than functions), and having a thorough discussion offline about reporting backend infrastructure, we arrived at major updates to this ticket. Mainly:

1. Functionality around Forecasters, Schedulers and Reporters should look and feel similar. We will work towards this goal iteratively.
2. Reporting functionality would benefit from being able to pipe multiple Pandas methods on FlexMeasures sensor data, rather than doing a single operation with each CLI call (and having to store intermediate results). Even relatively straightforwards reports (such as aggregate power and daily costs) require several operations and careful consideration of the relevant sensor resolutions, signs and units.
3. Sensor attributes may contain a report_config, which specifies how reports (new sensor data) should be computed (i.e. which data to use from other sensors, and what operations to perform on the data).
4. We want to include a few Reporter subclasses with predefined operations (for computing aggregates and costs), that need to be instantiated with sensor ids.
5. We want to allow plugins to register custom Reporter subclasses. Creation of Reporter class #641 and [follow-up ticket] see note
6. CLI commands are needed to:
   1. trigger a new report 642 add report cli command #659
   2. crud on reporters [follow-up ticket]
7. API commands for CRUD on reporters [follow-up ticket]
8. UI widget for CRUD on reporters [follow-up ticket]

[nhoening]

Should this become a project with sub-tickets? (then we can also link it on the roadmap)

[Flix6x]

Yes. I opened https://github.com/FlexMeasures/flexmeasures/projects/19.

[Flix6x]

Closing in favour of project 19.