Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add attributes column to data_source table #750

Merged

Conversation

victorgarcia98
Copy link
Contributor

@victorgarcia98 victorgarcia98 commented Jun 28, 2023

  • Add attributes column to the table data_source to store DataGenerator configuration. Type: JSON.
  • Add attributes_hash column to the table data_source to store a SHA256 hash of the attributes column. Type LargeBinary(length=256), in PostgreSQL this is represented as a BYTEA type.
  • Add sensorsbackref to the DataSource to allow to get the sensors that use a DataSource (in a Timed Belief).
  • Implementation of the methods get_attributes, has_attributes and set_attribute.
  • Update the function get_or_create_source to take the attributes column into account.

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@victorgarcia98 victorgarcia98 marked this pull request as ready for review June 28, 2023 13:04
@victorgarcia98 victorgarcia98 self-assigned this Jun 28, 2023
Copy link
Contributor

@Flix6x Flix6x left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice to see you are writing your own db migration. And thanks for dividing your work in these smaller PRs.

I'm looking forward to seeing how this will tie in with persisting reporter configurations. I have some small comments for now, and the changelog should also get a db upgrade notification for 0.15.

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
…-column-data-source' into feature/reporting/add-attributes-column-data-source
Copy link
Contributor

@Flix6x Flix6x left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a changelog entry for this could note that the DataSource table now allows storing arbitrary attributes as a JSON (without content validation), similar to the Sensor and GenericAsset tables.

flexmeasures/data/models/data_sources.py Outdated Show resolved Hide resolved
flexmeasures/data/models/data_sources.py Outdated Show resolved Hide resolved
flexmeasures/data/models/data_sources.py Outdated Show resolved Hide resolved
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
@nhoening nhoening added this to the 0.15.0 milestone Jul 4, 2023
Copy link
Contributor

@Flix6x Flix6x left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉 Just still add the db upgrade warning in the changelog (you can copy the warning from earlier versions in the changelog).

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>
…-column-data-source' into feature/reporting/add-attributes-column-data-source
@victorgarcia98 victorgarcia98 merged commit 3772b08 into main Jul 6, 2023
7 checks passed
@victorgarcia98 victorgarcia98 deleted the feature/reporting/add-attributes-column-data-source branch July 6, 2023 21:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants