Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

642 add report cli command #659

Merged
merged 100 commits into from May 29, 2023
Merged

642 add report cli command #659

merged 100 commits into from May 29, 2023

Conversation

victorgarcia98
Copy link
Contributor

This PR introduces the command flexmeasures add report which allows to create reports (custom calculations defined in classes of type Reporter) and save them to the database or export them to Excel or CSV. It also includes some flags to define the time scope of the report in some predefined ways: --last-hour, --last-day, ...

Closes #642.

victorgarcia98 and others added 30 commits April 14, 2023 10:50
@victorgarcia98
Creating Reporter and PandasReporter classes with their corresponding…
32e300c 
… report_config schemas.

Signed-off-by: victor <victor@seita.nl>
@victorgarcia98
Added Tibber Reporter.
ce17568 
Signed-off-by: victor <victor@seita.nl>
@victorgarcia98
- Fixing wong DA Price value.
848e39a 
- Renaming BWV and EB to english words.
- Simplifying calculation (pandas pipeline).
- Adding units to sensors.
- Changing units from EUR/kWh to EUR/MWh
- Adding assert to check maximum error
- deserialize_report_config -> deserialize_reporter_config
- Warning when a string starting with  `@` is used in the method query or eval.
- Making process_pandas_args, process_pandas_kwargs and apply_transformation private methods.

Signed-off-by: victor <victor@seita.nl>
@victorgarcia98
Updating VAT units.
d9867fc 
Signed-off-by: victor <victor@seita.nl>
@victorgarcia98
- Attatching report to sensor
1c9de43 
- Output type of compute is BeliefDataFrame
- Added a global input resolution to schem
- ISO datetime and timedeltas
- start, end and input_resolution are considered serialized when passed to the method compute
- assert to check that result resolution = sensor resolution

Signed-off-by: victor <victor@seita.nl>
@victorgarcia98
Fixing wrong arguments to search_beliefs method.
f61825b 
Signed-off-by: victor <victor@seita.nl>
@victorgarcia98
FIxing wrong type conversion logic.
b01f5c3 
Signed-off-by: victor <victor@seita.nl>
@Flix6x
Small reporter fixes (#647)
487233c 
* No return value

Signed-off-by: F.N. Claessen <felix@seita.nl>

* typo

Signed-off-by: F.N. Claessen <felix@seita.nl>

* plural

Signed-off-by: F.N. Claessen <felix@seita.nl>

* indentation

Signed-off-by: F.N. Claessen <felix@seita.nl>

* fix return type annotations

Signed-off-by: F.N. Claessen <felix@seita.nl>

* format docstring example

Signed-off-by: F.N. Claessen <felix@seita.nl>

* add type annotations

Signed-off-by: F.N. Claessen <felix@seita.nl>

* predefine instance attributes

Signed-off-by: F.N. Claessen <felix@seita.nl>

* remove redundant variable

Signed-off-by: F.N. Claessen <felix@seita.nl>

* grammar

Signed-off-by: F.N. Claessen <felix@seita.nl>

* support aliases

Signed-off-by: F.N. Claessen <felix@seita.nl>

* grammar/typos

Signed-off-by: F.N. Claessen <felix@seita.nl>

* test exact match

Signed-off-by: F.N. Claessen <felix@seita.nl>

---------

Signed-off-by: F.N. Claessen <felix@seita.nl>
@victorgarcia98
Add superclass to Reporter that will be common to all three data gene…
f34f5e1 
…rator classes: Reporter, Scheduler, Forecaster.

Signed-off-by: victor <victor@seita.nl>
@victorgarcia98
Merge remote-tracking branch 'origin/626-add-reporter-class' into 626…
1f8457e 
…-add-reporter-class
@victorgarcia98
Add start, end, resolution, beliefs_after and beliefs_before to the `…
dd21c99 
…compute` method signature.

Signed-off-by: victor <victor@seita.nl>
@victorgarcia98
Add FLEXMEASURES_DEFAULT_DATASOURCE config to be the feault datasourc…
58d405f 
…e for data generators.

Signed-off-by: victor <victor@seita.nl>
@victorgarcia98
Fixing wrong input type.
bebc320 
Signed-off-by: victor <victor@seita.nl>
@victorgarcia98
Rename DataGenerator class to DataGeneratorMixin
0dfe5f4 
Signed-off-by: victor <victor@seita.nl>
@victorgarcia98
Reduce logging level from warning to debug.
7830e33 
Signed-off-by: victor <victor@seita.nl>
@victorgarcia98
Merge branch 'main' into 626-add-reporter-class
e378ac6 
Signed-off-by: victor <victor@seita.nl>
@victorgarcia98
Merge branch 'main' into 626-add-reporter-class
5166264
@victorgarcia98
Register Reporter to the app context.
6ab84df 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
Allowing to use BeliefsDataFrame specific method in the schema.
79fe12f 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
Merge remote-tracking branch 'origin/626-add-reporter-class' into 626…
a64550b 
…-add-reporter-class
@victorgarcia98
Fixed wrong method. TODO: test with a plugin.
4c86878 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
Using module name instead of the module object.
5ca52eb 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@Flix6x
use belief_time instead of beliefs_before and beliefs_after (#652)
12267d3 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@victorgarcia98
Merge remote-tracking branch 'origin/626-add-reporter-class' into 626…
723ed02 
…-add-reporter-class
@victorgarcia98
Merge branch 'main' into 626-add-reporter-class
5a6590e
@victorgarcia98
Fixing example.
cc47742 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
Fixing grammar.
cc11809 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
Require at least 1 input sensor for the tb_query_config.
c50df17 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
Merge branch 'main' into 626-add-reporter-class
cc8aa22
@victorgarcia98
Bug fix: compute function was overriding the variables to the default…
c36b904 
… value (None)

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
fix: potential bug -> multiple data source creation
d8bdd1e 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
fix: issubclass instead of isinstance
650ddd3 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
docs: clarify test_add_reporter docstring
9883b49 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
feat: add function to apply an offset chain
129a549 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
feat: add start_offset and end_offset to add_report
a90105d 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
feat: add start and end offsets
86bb329 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
Merge branch 'main' into 642-add-report-cli-command-2
1a73c1a
@victorgarcia98
docs: add entry for flexmeasures add report
11778a0 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
feat: create the CLI command flexmeasures show reporters (#686)
c412802 
* feat: create show reporter command

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

* style: add hint to list reporters

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

* style: add module name
style: print empty string when the author is missing

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

* style: change column order

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

* docs: add entry for PR #686

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

* docs: add command flexmeasures show reporters

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

---------

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
Merge remote-tracking branch 'origin/642-add-report-cli-command-2' in…
f2bd1b7 
…to 642-add-report-cli-command-2
Flix6x
Flix6x requested changes May 19, 2023
View reviewed changes
flexmeasures/cli/data_add.py Outdated Show resolved Hide resolved
flexmeasures/cli/utils.py Outdated Show resolved Hide resolved
flexmeasures/cli/data_add.py Outdated Show resolved Hide resolved
flexmeasures/cli/data_add.py Outdated Show resolved Hide resolved
flexmeasures/cli/data_add.py Outdated Show resolved Hide resolved
flexmeasures/cli/data_add.py Outdated Show resolved Hide resolved
flexmeasures/cli/data_add.py Outdated Show resolved Hide resolved
flexmeasures/cli/data_add.py Outdated Show resolved Hide resolved
victorgarcia98 and others added 2 commits May 19, 2023 17:12
@victorgarcia98
docs: add entry to changelog
b0813c4 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@Flix6x
Small report cli fixes (#691)
1d5d7be 
* style: capitalize CSV

Signed-off-by: F.N. Claessen <felix@seita.nl>

* fix: reporter is no longer a required parameter

Signed-off-by: F.N. Claessen <felix@seita.nl>

* style: required parameters on top

Signed-off-by: F.N. Claessen <felix@seita.nl>

* style: fix parameter name in docstring

Signed-off-by: F.N. Claessen <felix@seita.nl>

* fix: avoid converting dt type

Signed-off-by: F.N. Claessen <felix@seita.nl>

* feature: add test case with timezone

Signed-off-by: F.N. Claessen <felix@seita.nl>

* feature: add test case for Pandas structure

Signed-off-by: F.N. Claessen <felix@seita.nl>

* fix: type annotations

Signed-off-by: F.N. Claessen <felix@seita.nl>

* fix: use parameter 'end'

Signed-off-by: F.N. Claessen <felix@seita.nl>

* style: fix type annotation

Signed-off-by: F.N. Claessen <felix@seita.nl>

* style: justified indentation

Signed-off-by: F.N. Claessen <felix@seita.nl>

---------

Signed-off-by: F.N. Claessen <felix@seita.nl>
Flix6x
Flix6x reviewed May 19, 2023
View reviewed changes
documentation/changelog.rst Outdated Show resolved Hide resolved
@victorgarcia98
docs: update PR number in the changelog
db0a94c 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
fix: changing types
50e2dd3 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
test: move fixture to conftest
880232d 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
fix: handle empty result
954f8f0 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
style: fix inaccurate message to the user
459ad9c 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98
refactor: remove last-X parameters
6d0f3e0 
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
Flix6x
Flix6x approved these changes May 29, 2023
View reviewed changes
Copy link
Contributor

@Flix6x Flix6x left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nicely done!

@victorgarcia98
Copy link
Contributor Author

Thanks for the review!

I'm trying to merge but it's blocked by the condition to not merge with unresolved comments. Checking all the conversations in Files Changed, there's no unresolved comments so I'm selecting the option Dismiss Review to see if that allows the merge.

@victorgarcia98
Copy link
Contributor Author

Thanks for the review!

I'm trying to merge but it's blocked by the condition to not merge with unresolved comments. Checking all the conversations in Files Changed, there's no unresolved comments so I'm selecting the option Dismiss Review to see if that allows the merge.

Instead, I think it would better if any of @nhoening or @Flix6x merges this to bypass the blockage 😄

@Flix6x Flix6x merged commit cc40d37 into main May 29, 2023
6 of 7 checks passed
Reporting backend infrastructure automation moved this from In progress to Done May 29, 2023
@Flix6x Flix6x deleted the 642-add-report-cli-command-2 branch May 29, 2023 14:12
@Flix6x Flix6x mentioned this pull request Jun 9, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Flix6x Flix6x Flix6x approved these changes

@nhoening nhoening Awaiting requested review from nhoening

Assignees

@victorgarcia98 victorgarcia98

Labels
enhancement New feature or request Reporting Still Needs Changelog Entry
Projects
Reporting backend infrastructure
  
Done
Milestone
0.14.0
Development

Successfully merging this pull request may close these issues.

Add CLI command flexmeasures add report to compute reports using a Reporter
3 participants
@victorgarcia98 @nhoening @Flix6x