Issue 247 generic asset crud

documentation/api/introduction.rst

@@ -204,12 +204,18 @@ Here is a full example for a FlexMeasures connection address:
 
 where FlexMeasures runs at `company.flexmeasures.io` (which the current domain owner started using in February 2021), and the locally unique string is of scheme `fm0` (see below) and the asset ID is 73. The asset's owner ID is 30, but this part is optional.
 
-Both the owner ID and the asset ID, as well as the full entity address can be obtained on the asset's listing:
+Assets are listed at:
 
 .. code-block:: html
 
     https://company.flexmeasures.io/assets
 
+Both the owner ID and the full entity address can be obtained on the asset's page, e.g.:
+
+.. code-block:: html
+
+    https://company.flexmeasures.io/assets/73
+
 
 Entity address structure
 """"""""""""""""""""""""""

documentation/changelog.rst

@@ -6,7 +6,7 @@ v0.8.0 | November XX, 2021
 ===========================
 
 .. warning:: Upgrading to this version requires running ``flexmeasures db upgrade`` (you can create a backup first with ``flexmeasures db-ops dump``).
-.. warning:: Changes to asset attributes made via the UI or API are not reflected in the new data model until `issue #247 <https://github.com/FlexMeasures/flexmeasures/issues/247>`_ is resolved.
+.. note:: v0.8.0 is doing much of the work we need to do to move to the new data model (see :ref:`note_on_datamodel_transition`). We hope to keep the migration steps for users very limited. One thing you'll notice is that we are copying over existing data to the new model (which will be kept in sync) with the `db upgrade` command (see warning above), which can take a few minutes.
 
 New features
 -----------
@@ -25,6 +25,7 @@ Infrastructure / Support
 * Add sensor method to obtain just its latest state (excl. forecasts) [see `PR #235 <http://www.github.com/FlexMeasures/flexmeasures/pull/235>`_]
 * Migrate attributes of assets, markets and weather sensors to our new sensor model [see `PR #254 <http://www.github.com/FlexMeasures/flexmeasures/pull/254>`_ and `project 9 <http://www.github.com/FlexMeasures/flexmeasures/projects/9>`_]
 * Migrate all time series data to our new sensor data model based on the `timely beliefs <https://github.com/SeitaBV/timely-beliefs>`_ lib [see `PR #286 <http://www.github.com/FlexMeasures/flexmeasures/pull/286>`_ and `project 9 <http://www.github.com/FlexMeasures/flexmeasures/projects/9>`_]
+* Support the new asset model (which describes the organisational structure, rather than sensors and data) in UI and API. Until the transition to our new data model is completed, the new API for assets is at `/api/dev/generic_assets`. [see `PR #251 <http://www.github.com/FlexMeasures/flexmeasures/pull/251>`_ and `PR #290 <http://www.github.com/FlexMeasures/flexmeasures/pulls/290>`_]
 
 
 v0.7.1 | November 08, 2021

documentation/dev/note-on-datamodel-transition.rst

@@ -58,3 +58,13 @@ Here is a brief list:
 - `Sensor relations and GeneralizedAssets with metadata <https://github.com/FlexMeasures/flexmeasures/projects/9>`_: We are generalizing our database structure for organising energy data, to support all sorts of sensors and relationships between them. We do this so we can better support the diverse set of use cases for energy flexibility.
 - `UI views for GeneralizedAssets <https://github.com/FlexMeasures/flexmeasures/projects/10>`_: We are updating our UI views (dashboard maps and analytics charts) according to our new database structure for organising energy data. We do this so users can customize what they want to see.
 - `Deprecate old database models <https://github.com/FlexMeasures/flexmeasures/projects/11>`_: We are deprecating the Power, Price and Weather tables in favour of the TimedBelief table, and deprecating the Asset, Market and WeatherSensor tables in favour of the Sensor and GeneralizedAsset tables. We are doing this so users can move their data to the new database model.
+
+
+The state of the transition (January 2022, v0.8.0)
+---------------------------------------------------
+
+Project 9 was implemented, which moved a lot of structure over, as well as actual data and some UI (dashboard, assets). We believe that was the hardest part.
+
+We are now close to being able to deprecate the old database models and route the API to the new model (see project 11). The API for assets is still in place, but the new one is already working (at /api/dev/generic_assets) and is powering what is shown in the UI.
+
+We take care to support people on the old data mode so the transition will be as smooth as possible, as we said above. One part of this is that the ``flexmeasures db upgrade`` command copies your data to the new model. Also, creating new data (e.g. old-style assets) creates new-style data (e.g. assets/sensors) automatically. However, some edge cases are not supported in this way. For instance, edited asset meta data might have to be re-entered later. Feel free to contact us to discuss the transition if needed.

flexmeasures/api/common/schemas/users.py

@@ -10,21 +10,23 @@ class AccountIdField(fields.Integer):
     Field that represents an account ID. It de-serializes from the account id to an account instance.
     """
 
-    def __init__(self, *args, **kwargs):
-        kwargs["load_default"] = (
-            lambda: current_user.account if not current_user.is_anonymous else None
-        )
-        super().__init__(*args, **kwargs)
-
-    def _deserialize(self, account_id: int, attr, obj, **kwargs) -> Account:
+    def _deserialize(self, account_id: str, attr, obj, **kwargs) -> Account:
         account: Account = Account.query.filter_by(id=int(account_id)).one_or_none()
         if account is None:
-            raise abort(404, f"Account {id} not found")
+            raise abort(404, f"Account {account_id} not found")
         return account
 
     def _serialize(self, account: Account, attr, data, **kwargs) -> int:
         return account.id
 
+    @classmethod
+    def load_current(cls):
+        """
+        Use this with the load_default arg to __init__ if you want the current user's account
+        by default.
+        """
+        return current_user.account if not current_user.is_anonymous else None
+
 
 class UserIdField(fields.Integer):
     """

flexmeasures/api/dev/__init__.py

@@ -8,9 +8,13 @@ def register_at(app: Flask):
     """This can be used to register FlaskViews."""
 
     from flexmeasures.api.dev.sensors import SensorAPI
+    from flexmeasures.api.dev.assets import AssetAPI
     from flexmeasures.api.dev.sensor_data import post_data as post_sensor_data_impl
 
-    SensorAPI.register(app, route_prefix="/api/dev")
+    dev_api_prefix = "/api/dev"
+
+    SensorAPI.register(app, route_prefix=dev_api_prefix)
+    AssetAPI.register(app, route_prefix=dev_api_prefix)
 
     @app.route("/sensorData", methods=["POST"])
     @auth_token_required

flexmeasures/api/dev/assets.py

@@ -0,0 +1,86 @@
+from flask import current_app
+from flask_classful import FlaskView, route
+from flask_json import as_json
+from webargs.flaskparser import use_kwargs, use_args
+
+from flexmeasures.auth.decorators import permission_required_for_context
+from flexmeasures.data.models.user import Account
+from flexmeasures.data.models.generic_assets import GenericAsset as AssetModel
+from flexmeasures.data.schemas.generic_assets import GenericAssetSchema as AssetSchema
+from flexmeasures.api.common.schemas.generic_assets import AssetIdField
+from flexmeasures.api.common.schemas.users import AccountIdField
+from flexmeasures.data.config import db
+
+
+asset_schema = AssetSchema()
+assets_schema = AssetSchema(many=True)
+
+
+class AssetAPI(FlaskView):
+    """
+    This API view exposes generic assets.
+    Under development until it replaces the original Asset API.
+    """
+
+    route_base = "/generic_assets"
+
+    @route("/", methods=["GET"])
+    @use_kwargs(
+        {
+            "account": AccountIdField(
+                data_key="account_id", load_default=AccountIdField.load_current
+            ),
+        },
+        location="query",
+    )
+    @permission_required_for_context("read", arg_name="account")
+    @as_json
+    def index(self, account: Account):
+        """List all assets owned by a certain account."""
+        return assets_schema.dump(account.generic_assets), 200
+
+    @route("/", methods=["POST"])
+    @permission_required_for_context(
+        "create-children", arg_loader=AccountIdField.load_current
+    )
+    @use_args(AssetSchema())
+    def post(self, asset_data):
+        """Create new asset"""
+        asset = AssetModel(**asset_data)
+        db.session.add(asset)
+        db.session.commit()
+        return asset_schema.dump(asset), 201
+
+    @route("/<id>", methods=["GET"])
+    @use_kwargs({"asset": AssetIdField(data_key="id")}, location="path")
+    @permission_required_for_context("read", arg_name="asset")
+    @as_json
+    def fetch_one(self, id, asset):
+        """Fetch a given asset"""
+        return asset_schema.dump(asset), 200
+
+    @route("/<id>", methods=["PATCH"])
+    @use_args(AssetSchema(partial=True))
+    @use_kwargs({"db_asset": AssetIdField(data_key="id")}, location="path")
+    @permission_required_for_context("update", arg_name="db_asset")
+    @as_json
+    def patch(self, asset_data, id, db_asset):
+        """Update an asset given its identifier"""
+        ignored_fields = ["id", "account_id"]
+        for k, v in [(k, v) for k, v in asset_data.items() if k not in ignored_fields]:
+            setattr(db_asset, k, v)
+        db.session.add(db_asset)
+        db.session.commit()
+        return asset_schema.dump(db_asset), 200
+
+    @route("/<id>", methods=["DELETE"])
+    @use_kwargs({"asset": AssetIdField(data_key="id")}, location="path")
+    @permission_required_for_context("delete", arg_name="asset")
+    @as_json
+    def delete(self, id, asset):
+        """Delete an asset given its identifier"""
+        asset_name = asset.name
+        db.session.delete(asset)
+        db.session.commit()
+        current_app.logger.info("Deleted asset '%s'." % asset_name)
+        return {}, 204

flexmeasures/api/dev/tests/conftest.py

@@ -6,17 +6,19 @@
 from flexmeasures.data.models.time_series import Sensor
 
 
-@pytest.fixture(scope="module", autouse=True)
-def setup_api_test_data(db, setup_roles_users):
+@pytest.fixture(scope="module")
+def setup_api_test_data(db, setup_roles_users, setup_generic_assets):
     """
     Set up data for API dev tests.
     """
-    print("Setting up data for API v2.0 tests on %s" % db.engine)
+    print("Setting up data for API dev tests on %s" % db.engine)
     add_gas_sensor(db, setup_roles_users["Test Prosumer User 2"])
 
 
 @pytest.fixture(scope="function")
-def setup_api_fresh_test_data(fresh_db, setup_roles_users_fresh_db):
+def setup_api_fresh_test_data(
+    fresh_db, setup_roles_users_fresh_db, setup_generic_assets_fresh_db
+):
     """
     Set up fresh data for API dev tests.
     """