Issue 247 generic asset crud #290

nhoening · 2021-12-29T16:51:06Z

Add a new API for our new asset data model: generic assets. For now reachable at /api/dev, as the old API will stay for a bit.
Move the asset CRUD in the UI to use this new API.

Also:

Auth: Change "create" permission to "create-children", as you get the permission from the the ACL of the instance one level up in the hierarchy (and the highest objects, like accounts, are created by site-admins on CLI level)
Auth2: add another method for the permission-for-context endpoint decorator to define a context (with a Callable, useful for defaults, like current_user.account)
Auth3: Add ACLs for GenericAsset, Sensor, Account & User
Auth4: Fix a bug in parsing ACLs for the case that there are two possible ways to authorize for the same permission
Data model: add unique contraints: on GenericAssetType.name & on GenericAssets for name+account_id (the latter is also checked on the schema level, for nicer responses)

For testing: try if the UI Asset Crud is doing its job. It is using the generic asset API internally.

TODO:

make crud tests work
move API tests to new API & make work
check if UI can consistently show asset type icon
changelog entry (incl deprecation notice)

… steps forward for use cases not covered yet by auth policy

…eady defines what happens)

…g the need for issue#200); fix modelling problem if several distinct principal formulations exist for one permission

…ets when we don't already have them (distinguish by name); re-add old asset tests;

…r name+account_id

Flix6x

I request small changes mostly.

The bigger issue is that we need some sensor CRUD (at least the R) before we can update our documentation. In particular, we are missing a sensor listing where users can look up entity addresses, so they can actually use the data API.

flexmeasures/auth/tests/test_principal_matching.py

flexmeasures/conftest.py

flexmeasures/data/models/time_series.py

flexmeasures/data/models/user.py

flexmeasures/data/schemas/generic_assets.py

Flix6x · 2022-01-03T11:27:43Z

flexmeasures/api/v2_0/implementations/assets.py

@@ -17,6 +17,11 @@
 from flexmeasures.api.common.responses import required_info_missing


+"""
+Deprecated. Use /api/dev/generic_assets.
+TODO: Can/should we add a deprecation warning to responses?


Yes. At the very least in the documentation of the relevant endpoints.

We should also discuss updating documentation before releasing.

documentation/changelog.rst

flexmeasures/api/dev/tests/test_assets_api.py

flexmeasures/ui/templates/crud/assets.html

nhoening · 2022-01-03T12:23:02Z

Maybe the easiest path towards releasing this is to a listing of sensors on each asset page.

Later, we can link from there to a sensor page where you can see a plot and find a link to edit its meta data or delete it. Those are less crucial for now.

Flix6x · 2022-01-03T13:19:52Z

Maybe the easiest path towards releasing this is to a listing of sensors on each asset page.

I agree.

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

-e Signed-off-by: Nicolas Höning <nicolas@seita.nl>

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

Remove the entity addresses from the generic asset listing, and list the number of sensors instead. * grammar Signed-off-by: F.N. Claessen <felix@seita.nl> * Update generic asset listing, replacing entity addresses with number of sensors Signed-off-by: F.N. Claessen <felix@seita.nl> * Avoid crashing on formatting None values Signed-off-by: F.N. Claessen <felix@seita.nl> * Fix wrong variable name Signed-off-by: F.N. Claessen <felix@seita.nl> * Fix incorrect type annotation Signed-off-by: F.N. Claessen <felix@seita.nl> * Partly revert cefe320 Signed-off-by: F.N. Claessen <felix@seita.nl>

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

Signed-off-by: F.N. Claessen <felix@seita.nl>

Flix6x

Thanks for all the help moving our asset crud over! It seems project 9 is about finished.

documentation/dev/note-on-datamodel-transition.rst

Signed-off-by: F.N. Claessen <felix@seita.nl>

… to fm1 scheme. Signed-off-by: F.N. Claessen <felix@seita.nl>

nhoening added 3 commits December 29, 2021 17:41

add a new method to specify an auth context (by callable) and discuss…

a36eeaa

… steps forward for use cases not covered yet by auth policy

add ACL for GenericAsset

00d5651

added /dev/generic_assets API, CRUD uses it (passing manual testing)

63ee1fd

nhoening changed the base branch from main to project-9 December 29, 2021 16:55

nhoening added 9 commits December 29, 2021 18:02

fix asset icon in asset view

02edb47

remove verbs from /api/dev/generic_assets API routes (HTTP method alr…

c6a0b50

…eady defines what happens)

make CRUD UI tests (mocking the asset API) work

6018004

auth policy: change 'create' to 'create-children' permission (removin…

e922e39

…g the need for issue#200); fix modelling problem if several distinct principal formulations exist for one permission

separate fresh tests from others; only generate copies of generic ass…

4c6e213

…ets when we don't already have them (distinguish by name); re-add old asset tests;

add missing mock in another ui test

f304a98

add unique contraints: on GenericAssetType.name & on GenericAssets fo…

b0fbff6

…r name+account_id

improve explanation of our authorization policy implementation

3982e0f

update changelog

2ae8aad

nhoening requested a review from Flix6x January 1, 2022 22:24

improve changelog, also mention temporary API location

fdfca9c

Flix6x requested changes Jan 3, 2022

View reviewed changes

nhoening added 11 commits January 3, 2022 21:33

comments next to test cases

4c95ff9

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

create_user function hashes the password

5518c17

-e Signed-off-by: Nicolas Höning <nicolas@seita.nl>

more informative validation errror

0c492a0

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

allow creation of public Assets in UI

2ab9f5b

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

Fix display of owning account

f75982c

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

better changelog docs

912b81e

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

list sensors on asset page

7d7e5a1

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

do not fail getting latest state if capacity_in_mw is not in attributes

b2479db

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

better default for capacity_in_mw

850b92d

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

Merge branch 'project-9' into issue-247-GenericAsset-CRUD

c5ce5a9

use id in the dict, for sensors lookup

2d463bb

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

nhoening mentioned this pull request Jan 4, 2022

Suggested fixes to pr 290 #299

Merged

Flix6x and others added 4 commits January 4, 2022 12:37

small improvement on locating ea addresses

50248bb

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

Describe current state of transition

8359502

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

deprecation notice in old-style asset API documentation

d50c4f4

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

nhoening requested a review from Flix6x January 4, 2022 12:30

Flix6x added 4 commits January 4, 2022 14:46

Fix type annotation

133e04b

Signed-off-by: F.N. Claessen <felix@seita.nl>

Update documentation

93c9a69

Signed-off-by: F.N. Claessen <felix@seita.nl>

Update v2.0 connection entity addresses in documentation

c4262ce

Signed-off-by: F.N. Claessen <felix@seita.nl>

Correct quickrefs in v2.0 documentation

f9633e5

Signed-off-by: F.N. Claessen <felix@seita.nl>

Flix6x approved these changes Jan 4, 2022

View reviewed changes

documentation/dev/note-on-datamodel-transition.rst Show resolved Hide resolved

Flix6x added 3 commits January 5, 2022 10:00

Update other v2.0 sensor entity addresses in documentation

81d6f6a

Signed-off-by: F.N. Claessen <felix@seita.nl>

Fix entity address dates corresponding to older previous domain name

70f180c

Signed-off-by: F.N. Claessen <felix@seita.nl>

Adapt more occurrences of older dates in entity addresses, and switch…

9165e79

… to fm1 scheme. Signed-off-by: F.N. Claessen <felix@seita.nl>

Flix6x mentioned this pull request Jan 5, 2022

Update entity address scheme in documentation #142

Closed

nhoening merged commit 1bb2819 into project-9 Jan 5, 2022

nhoening deleted the issue-247-GenericAsset-CRUD branch January 5, 2022 15:59

Flix6x linked an issue Jan 6, 2022 that may be closed by this pull request

Implement GenericAsset crud #247

Closed

Flix6x added this to the 0.8.0 milestone Jan 20, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue 247 generic asset crud #290

Issue 247 generic asset crud #290

nhoening commented Dec 29, 2021 •

edited

Flix6x left a comment

Flix6x Jan 3, 2022

Flix6x Jan 3, 2022

nhoening commented Jan 3, 2022

Flix6x commented Jan 3, 2022

Flix6x left a comment

Issue 247 generic asset crud #290

Issue 247 generic asset crud #290

Conversation

nhoening commented Dec 29, 2021 • edited

Flix6x left a comment

Choose a reason for hiding this comment

Flix6x Jan 3, 2022

Choose a reason for hiding this comment

Flix6x Jan 3, 2022

Choose a reason for hiding this comment

nhoening commented Jan 3, 2022

Flix6x commented Jan 3, 2022

Flix6x left a comment

Choose a reason for hiding this comment

nhoening commented Dec 29, 2021 •

edited