Allow editing asset attributes in the UI (#474)

* Allow asset attributes in the UI (more specifically, the JSON attributes column on the generic_asset table) Signed-off-by: F.N. Claessen <felix@seita.nl> * Avoid redundant serialization step Signed-off-by: F.N. Claessen <felix@seita.nl> * Move attributes field down Signed-off-by: F.N. Claessen <felix@seita.nl> * Rename field label Signed-off-by: F.N. Claessen <felix@seita.nl> * changelog entry Signed-off-by: F.N. Claessen <felix@seita.nl> * Add field to new asset page Signed-off-by: F.N. Claessen <felix@seita.nl> * Add empty default attributes Signed-off-by: F.N. Claessen <felix@seita.nl> * Clarify attributes label Signed-off-by: F.N. Claessen <felix@seita.nl> * Raise ValidationError if not able to deserialize JSON string Signed-off-by: F.N. Claessen <felix@seita.nl> * Update documentation for asset page Signed-off-by: F.N. Claessen <felix@seita.nl> * flake8 Signed-off-by: F.N. Claessen <felix@seita.nl> * Add tests Signed-off-by: F.N. Claessen <felix@seita.nl> * improve error visualisation in asset form Signed-off-by: Nicolas Höning <nicolas@seita.nl> Signed-off-by: F.N. Claessen <felix@seita.nl> Signed-off-by: Nicolas Höning <nicolas@seita.nl> Co-authored-by: Nicolas Höning <nicolas@seita.nl>
FlexMeasures · Aug 25, 2022 · 643e4d5 · 643e4d5
1 parent 7ff1480
commit 643e4d5
Show file tree

Hide file tree

Showing 7 changed files with 103 additions and 11 deletions.
diff --git a/documentation/changelog.rst b/documentation/changelog.rst
@@ -31,6 +31,7 @@ Infrastructure / Support
 * Allow access tokens to be passed as env vars as well [see `PR #443 <http://www.github.com/FlexMeasures/flexmeasures/pull/443>`_]
 * Queue workers can get initialised without a custom name and name collisions are handled [see `PR #455 <http://www.github.com/FlexMeasures/flexmeasures/pull/455>`_]
 * New API endpoint to get public assets [see `PR #461 <http://www.github.com/FlexMeasures/flexmeasures/pull/461>`_]
+* Allow editing an asset's JSON attributes through the UI [see `PR #474 <http://www.github.com/FlexMeasures/flexmeasures/pull/474>`_]
 
 
 v0.10.1 | August 12, 2022

diff --git a/documentation/views/asset-data.rst b/documentation/views/asset-data.rst
@@ -4,7 +4,10 @@
 Assets & data
 **************
 
-The asset page allows to see data from the asset's sensors, and also to edit meta information, like its location:
+The asset page allows to see data from the asset's sensors, and also to edit attributes of the asset, like its location.
+Other attributes are stored as a JSON string, which can be edited here as well.
+This is meant for meta information that may be used to customize views or functionality, e.g. by plugins.
+This includes the possibility to specify which sensors the asset page should show. For instance, here we include a price sensor from a public asset, by setting ``{"sensor_to_show": [3, 2]}`` (sensor 3 on top, followed by sensor 2 below).
 
 
 .. image:: https://github.com/FlexMeasures/screenshots/raw/main/screenshot_asset.png
@@ -14,6 +17,6 @@ The asset page allows to see data from the asset's sensors, and also to edit met
 |
 |
 
-.. note:: It is possible to determine which sensors this page should show. For instance, here we include a price sensor from a public asset.
-          For this, set the `sensors_to_show` attribute with the CLI command ``flexmeasures edit attribute``.
+.. note:: While it is possible to show an arbitrary number of sensors this way, we recommend showing only the most crucial ones for faster loading, less page scrolling, and generally, a quick grasp of what the asset is up to.
+.. note:: Asset attributes can be edited through the CLI as well, with the CLI command ``flexmeasures edit attribute``.
 
diff --git a/flexmeasures/api/v3_0/tests/test_assets_api.py b/flexmeasures/api/v3_0/tests/test_assets_api.py
@@ -1,3 +1,5 @@
+import json
+
 from flask import url_for
 import pytest
 
@@ -144,6 +146,50 @@ def test_alter_an_asset(client, setup_api_test_data, setup_accounts):
     assert asset_edit_response.status_code == 200
 
 
+@pytest.mark.parametrize(
+    "bad_json_str",
+    [
+        None,
+        "{",
+        '{"hallo": world}',
+    ],
+)
+def test_alter_an_asset_with_bad_json_attributes(
+    client, setup_api_test_data, setup_accounts, bad_json_str
+):
+    """Check whether updating an asset's attributes with a badly structured JSON fails."""
+    with UserContext("test_prosumer_user@seita.nl") as prosumer1:
+        auth_token = prosumer1.get_auth_token()
+    with AccountContext("Test Prosumer Account") as prosumer:
+        prosumer_asset = prosumer.generic_assets[0]
+    asset_edit_response = client.patch(
+        url_for("AssetAPI:patch", id=prosumer_asset.id),
+        headers={"content-type": "application/json", "Authorization": auth_token},
+        json={"attributes": bad_json_str},
+    )
+    print(f"Editing Response: {asset_edit_response.json}")
+    assert asset_edit_response.status_code == 422
+
+
+def test_alter_an_asset_with_json_attributes(
+    client, setup_api_test_data, setup_accounts
+):
+    """Check whether updating an asset's attributes with a properly structured JSON succeeds."""
+    with UserContext("test_prosumer_user@seita.nl") as prosumer1:
+        auth_token = prosumer1.get_auth_token()
+    with AccountContext("Test Prosumer Account") as prosumer:
+        prosumer_asset = prosumer.generic_assets[0]
+    asset_edit_response = client.patch(
+        url_for("AssetAPI:patch", id=prosumer_asset.id),
+        headers={"content-type": "application/json", "Authorization": auth_token},
+        json={
+            "attributes": json.dumps(prosumer_asset.attributes)
+        },  # we're not changing values to keep other tests clean here
+    )
+    print(f"Editing Response: {asset_edit_response.json}")
+    assert asset_edit_response.status_code == 200
+
+
 def test_post_an_asset_with_existing_name(client, setup_api_test_data):
     """Catch DB error (Unique key violated) correctly"""
     with UserContext("test_admin_user@seita.nl") as admin_user:

diff --git a/flexmeasures/data/schemas/generic_assets.py b/flexmeasures/data/schemas/generic_assets.py
@@ -1,4 +1,5 @@
 from typing import Optional
+import json
 
 from marshmallow import validates, validates_schema, ValidationError, fields
 
@@ -12,6 +13,17 @@
 )
 
 
+class JSON(fields.Field):
+    def _deserialize(self, value, attr, data, **kwargs):
+        try:
+            return json.loads(value)
+        except ValueError:
+            raise ValidationError("Not a valid JSON string.")
+
+    def _serialize(self, value, attr, data, **kwargs):
+        return json.dumps(value)
+
+
 class GenericAssetSchema(ma.SQLAlchemySchema):
     """
     GenericAsset schema, with validations.
@@ -23,6 +35,7 @@ class GenericAssetSchema(ma.SQLAlchemySchema):
     latitude = ma.auto_field()
     longitude = ma.auto_field()
     generic_asset_type_id = fields.Integer(required=True)
+    attributes = JSON(required=False)
 
     class Meta:
         model = GenericAsset

diff --git a/flexmeasures/ui/crud/assets.py b/flexmeasures/ui/crud/assets.py
@@ -1,5 +1,6 @@
 from typing import Union, Optional, List, Tuple
 import copy
+import json
 
 from flask import url_for, current_app
 from flask_classful import FlaskView
@@ -47,6 +48,7 @@ class AssetForm(FlaskForm):
         places=4,
         render_kw={"placeholder": "--Click the map or enter a longitude--"},
     )
+    attributes = StringField("Other attributes (JSON)")
 
     def validate_on_submit(self):
         if (
@@ -125,7 +127,12 @@ def expunge_asset():
     if asset_id:
         asset_data["id"] = asset_id
     if make_obj:
-        asset = GenericAsset(**asset_data)  # TODO: use schema?
+        asset = GenericAsset(
+            **{
+                **asset_data,
+                **{"attributes": json.loads(asset_data.get("attributes", "{}"))},
+            }
+        )  # TODO: use schema?
         asset.generic_asset_type = GenericAssetType.query.get(
             asset.generic_asset_type_id
         )
@@ -285,11 +292,14 @@ def post(self, id: str):
                         f"Internal asset API call unsuccessful [{post_asset_response.status_code}]: {post_asset_response.text}"
                     )
                     asset_form.process_api_validation_errors(post_asset_response.json())
-                    if (
-                        "message" in post_asset_response.json()
-                        and "json" in post_asset_response.json()["message"]
-                    ):
-                        error_msg = str(post_asset_response.json()["message"]["json"])
+                    if "message" in post_asset_response.json():
+                        asset_form.process_api_validation_errors(
+                            post_asset_response.json()["message"]
+                        )
+                        if "json" in post_asset_response.json()["message"]:
+                            error_msg = str(
+                                post_asset_response.json()["message"]["json"]
+                            )
             if asset is None:
                 msg = "Cannot create asset. " + error_msg
                 return render_flexmeasures_template(
@@ -339,7 +349,9 @@ def post(self, id: str):
                     f"Internal asset API call unsuccessful [{patch_asset_response.status_code}]: {patch_asset_response.text}"
                 )
                 msg = "Cannot edit asset."
-                asset_form.process_api_validation_errors(patch_asset_response.json())
+                asset_form.process_api_validation_errors(
+                    patch_asset_response.json().get("message")
+                )
                 asset = GenericAsset.query.get(id)
 
         latest_measurement_time_str, asset_plot_html = _get_latest_power_plot(asset)

diff --git a/flexmeasures/ui/templates/crud/asset.html b/flexmeasures/ui/templates/crud/asset.html
@@ -86,14 +86,22 @@ <h3>Edit {{ asset.name }}</h3>
                                             value="{{ asset.generic_asset_type.name }}" disabled></input>
                                      </div>
                                 </div>
-
                                 <div class="form-group">
                                     <label for="asset-id" class="col-sm-6 control-label">Asset id</label>
                                     <div class="col-sm-6">
                                         <input class="form-control" id="asset-id" name="asset-id" type="text" value="{{ asset.id }}"
                                             disabled></input>
                                     </div>
                                 </div>
+                                <div class="form-group">
+                                    {{ asset_form.attributes.label(class="col-sm-3 control-label") }}
+                                    <div class="col-sm-3">
+                                        {{ asset_form.attributes(class_="form-control") }}
+                                        {% for error in asset_form.errors.attributes %}
+                                        <span style="color: red;">[{{error}}]</span>
+                                        {% endfor %}
+                                    </div>
+                                </div>
                                 <label class="control-label">Location</label>
                                 <small>(Click map to edit latitude and longitude in form)</small>
                                 <div id="mapid"></div>

diff --git a/flexmeasures/ui/templates/crud/asset_new.html b/flexmeasures/ui/templates/crud/asset_new.html
@@ -70,6 +70,15 @@ <h2> Creating a new asset </h2>
                             {% endfor %}
                         </div>
                     </div>
+                    <div class="form-group">
+                        {{ asset_form.attributes.label(class="col-sm-6 control-label") }}
+                        <div class="col-sm-6">
+                            {{ asset_form.attributes(class_="form-control") }}
+                            {% for error in asset_form.errors.attributes %}
+                            <span style="color: red;">[{{error}}]</span>
+                            {% endfor %}
+                        </div>
+                    </div>
                     <div class="col-sm-6"></div>
                     <div class="col-sm-6">
                         <input type="submit" value="Create">