Multi-tenancy (#159)

* add account table and according data migration script * fix the preservation of generic_asset.owner_id information between up & downgrading the database migration * mention account name on user page and listings * change name of /account page to /logged-in-user to avoid confusion * adapt data.services.users:create_user to deal with the account_id; make sure all test users have an account * display account name on logged-in-user page, not account ID * give automatically generated generic assets the correct account ID (derived from asset.owner_id) * Support adding & deleting accounts in CLI, also add --account-id to the add user command. Make sure deleting GenericAssets actually cascades to deleting the connected sensors. * API: protect change in asset.owner_id from switching accounts * a few user/asset API docs improvements found on the way, also adding account_id to /user docs * Update change_log.rst * Show which users and generic assets would be deleted when CLI task is used to delete an account * add account_id to GenericAsset schema and also to the CLI task for adding them * minor improvements from review * ffix test * add changelog entry * document make show-data-model better & make --dev work for the --schema option, as well. * add info about make show-data-model --uml --dev to changelog * more help with db upgrading in version 0.6.0 Co-authored-by: Nicolas Höning <nicolas@seita.nl> Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
FlexMeasures · Aug 27, 2021 · 247c747 · 247c747
1 parent 0ad268d
commit 247c747
Show file tree

Hide file tree

Showing 39 changed files with 627 additions and 84 deletions.
diff --git a/Makefile b/Makefile
@@ -74,4 +74,8 @@ upgrade-db:
 	flask db current
 
 show-data-model:
-	./flexmeasures/data/scripts/visualize_data_model.py --uml --store  # also try with --schema for database model
+	# This generates the data model, as currently written in code, as a PNG picture.
+	# Also try with --schema for the database model. 
+	# With --dev, you'll see the currently experimental parts, as well.
+	# Use --help to learn more. 
+	./flexmeasures/data/scripts/visualize_data_model.py --uml --store
diff --git a/documentation/api/introduction.rst b/documentation/api/introduction.rst
@@ -102,7 +102,7 @@ A fresh "<token>" can be generated on the user's profile after logging in:
 
 .. code-block:: html
 
-    https://company.flexmeasures.io/account
+    https://company.flexmeasures.io/logged-in-user
 
 or through a POST request to the following endpoint:
 

diff --git a/documentation/changelog.rst b/documentation/changelog.rst
@@ -2,16 +2,17 @@
 FlexMeasures Changelog
 **********************
 
-v0.6.0 | July XX, 2021
+v0.6.0 | August XX, 2021
 ===========================
 
 .. warning:: Upgrading to this version requires running ``flexmeasures db upgrade`` (you can create a backup first with ``flexmeasures db-ops dump``).
-             In case you are using experimental developer features and have previously set up sensors, be sure to check out the upgrade instructions in `PR #157 <https://github.com/SeitaBV/flexmeasures/pull/157>`_.
+             In case you are using experimental developer features and have previously set up sensors, be sure to check out the upgrade instructions in `PR #157 <https://github.com/SeitaBV/flexmeasures/pull/157>`_. Furthermore, if you want to create custom user/account relationships while upgrading (otherwise the upgrade script creates accounts based on email domains), check out the upgrade instructions in `PR #159 <https://github.com/SeitaBV/flexmeasures/pull/159>`_. If you want to use both of these custom upgrade features, do the upgrade in two steps. First, as described in PR 157 and upgrading up to revision b6d49ed7cceb, then as described in PR 159 for the rest.
 
 New features
 -----------
 * Analytics view offers grouping of all assets by location [see `PR #148 <http://www.github.com/SeitaBV/flexmeasures/pull/148>`_]
 * Add (experimental) endpoint to post sensor data for any sensor. Also supports our ongoing integration with data internally represented using the `timely beliefs <https://github.com/SeitaBV/timely-beliefs>`_ lib [see `PR #147 <http://www.github.com/SeitaBV/flexmeasures/pull/147>`_]
+* Multi-tenancy: Supporting multiple customers per FlexMeasures server, by introducing the `Account` concept. Accounts have users and assets associated. [see `PR #159 <http://www.github.com/SeitaBV/flexmeasures/pull/159>`_] 
 
 Bugfixes
 -----------
@@ -23,6 +24,8 @@ Infrastructure / Support
 * Add CLI task to monitor if tasks ran successfully and recently enough [see `PR #146 <http://www.github.com/SeitaBV/flexmeasures/pull/146>`_]
 * Document how to use a custom favicon in plugins [see `PR #152 <http://www.github.com/SeitaBV/flexmeasures/pull/152>`_]
 * Continue experimental integration with `timely beliefs <https://github.com/SeitaBV/timely-beliefs>`_ lib: link multiple sensors to a single asset [see `PR #157 <https://github.com/SeitaBV/flexmeasures/pull/157>`_]
+* The experimental parts of the data model can now be visualised, as well, via `make show-data-model --uml --dev` [also in `PR #157 <https://github.com/SeitaBV/flexmeasures/pull/157>`_]
+
 
 
 v0.5.0 | June 7, 2021

diff --git a/documentation/cli/change_log.rst b/documentation/cli/change_log.rst
@@ -4,6 +4,13 @@
 FlexMeasures CLI Changelog
 **********************
 
+
+since v0.6.0 | April 2, 2021
+=====================
+
+* add ``flexmeasures add account``, ``flexmeasures delete account``, and the ``--account-id`` param to ``flexmeasures add user``.
+
+
 since v0.4.0 | April 2, 2021
 =====================
 
@@ -15,4 +22,4 @@ since v0.3.0 | April 2, 2021
 
 * Refactor CLI into the main groups ``add``, ``delete``, ``jobs`` and ``db-ops``
 * Add ``flexmeasures add asset``,  ``flexmeasures add user`` and ``flexmeasures add weather-sensor``
-* split the ``populate-db`` command into ``flexmeasures add structure`` and ``flexmeasures add forecasts``
+* split the ``populate-db`` command into ``flexmeasures add structure`` and ``flexmeasures add forecasts``
diff --git a/documentation/cli/commands.rst b/documentation/cli/commands.rst
@@ -25,6 +25,7 @@ of which some are referred to in this documentation.
 ================================================= =======================================
 ``flexmeasures add structure``                    Initialize structural data like asset types, 
                                                   market types and weather sensor types.
+``flexmeasures add account``                      Create a FlexMeasures tenant account.
 ``flexmeasures add user``                         Create a FlexMeasures user.
 ``flexmeasures add asset``                        Create a new asset.
 ``flexmeasures add weather-sensor``               Add a weather sensor.
@@ -39,6 +40,7 @@ of which some are referred to in this documentation.
 ================================================= =======================================
 ``flexmeasures delete structure``                 Delete all structural (non time-series) data like assets (types), 
                                                   markets (types) and weather sensors (types) and users.
+``flexmeasures delete account``                   Delete a tenant account & also their users (with assets and power measurements).
 ``flexmeasures delete user``                      Delete a user & also their assets and power measurements.
 ``flexmeasures delete measurements``              Delete measurements (with horizon <= 0).
 ``flexmeasures delete prognoses``                 Delete forecasts and schedules (forecasts > 0).

diff --git a/documentation/getting-started.rst b/documentation/getting-started.rst
@@ -96,14 +96,22 @@ This suffices for a quick start.
 
 
 
-Add a user
-^^^^^^^^^^
+Add an account & user
+^^^^^^^^^^^^^^^^^^^^^
+
+FlexMeasures is a tenant-based platform ― multiple clients can enjoy its services on one server. Let's create a tenant account first: 
+
+.. code-block::
+
+   flexmeasures add account --name  "Some company"
+
+This command will tell us the ID of this account. Let's assume it was ``2``.
 
-FlexMeasures is a web-based platform, so we need a user account:
+FlexMeasures is also a web-based platform, so we need to create a user to authenticate:
 
 .. code-block::
 
-   flexmeasures add user --username <your-username> --email <your-email-address> --roles=admin
+   flexmeasures add user --username <your-username> --email <your-email-address> --account-id 2 --roles=admin
 
 
 * This will ask you to set a password for the user.
@@ -215,4 +223,4 @@ More information (e.g. for installing on Windows) on `the Cbc website <https://p
 Install and configure Redis
 ^^^^^^^^^^^^^^^^^^^^^^^
 
-To let FlexMeasures queue forecasting and scheduling jobs, install a `Redis <https://redis.io/>`_ server (or rent one) and configure access to it within FlexMeasures' config file (see above). You can find the necessary settings in :ref:`redis-config`.
+To let FlexMeasures queue forecasting and scheduling jobs, install a `Redis <https://redis.io/>`_ server (or rent one) and configure access to it within FlexMeasures' config file (see above). You can find the necessary settings in :ref:`redis-config`.
diff --git a/documentation/views/admin.rst b/documentation/views/admin.rst
@@ -4,7 +4,7 @@
 Administration
 **************
 
-The administrator can edit assets and user accounts here.
+The administrator can edit assets and users here.
 
 Assets
 ------
@@ -36,4 +36,4 @@ Viewing one user:
 
 .. image:: https://github.com/SeitaBV/screenshots/raw/main/screenshot_user.png
     :align: center
-..    :scale: 40%
+..    :scale: 40%
diff --git a/flexmeasures/api/tests/conftest.py b/flexmeasures/api/tests/conftest.py
@@ -8,7 +8,7 @@
 
 
 @pytest.fixture(scope="module", autouse=True)
-def setup_api_test_data(db, setup_roles_users):
+def setup_api_test_data(db, setup_account, setup_roles_users):
     """
     Adding the task-runner
     """
@@ -31,6 +31,7 @@ def setup_api_test_data(db, setup_roles_users):
         username="test user",
         email="task_runner@seita.nl",
         password=hash_password("testtest"),
+        account_id=setup_account.id,
     )
     user_datastore.add_role_to_user(test_task_runner, test_task_runner_role)
 

diff --git a/flexmeasures/api/v1/tests/conftest.py b/flexmeasures/api/v1/tests/conftest.py
@@ -10,7 +10,7 @@
 
 
 @pytest.fixture(scope="module", autouse=True)
-def setup_api_test_data(db, setup_roles_users, add_market_prices):
+def setup_api_test_data(db, setup_account, setup_roles_users, add_market_prices):
     """
     Set up data for API v1 tests.
     """
@@ -24,6 +24,7 @@ def setup_api_test_data(db, setup_roles_users, add_market_prices):
         username="anonymous user with Prosumer role",
         email="demo@seita.nl",
         password=hash_password("testtest"),
+        account_name=setup_account.name,
         user_roles=[
             "Prosumer",
             dict(name="anonymous", description="Anonymous test user"),
@@ -54,6 +55,7 @@ def setup_api_test_data(db, setup_roles_users, add_market_prices):
         username="test user without roles",
         email="test_user@seita.nl",
         password=hash_password("testtest"),
+        account_name=setup_account.name,
     )
 
     # Create 5 test assets for the test_prosumer user

diff --git a/flexmeasures/api/v1_1/tests/conftest.py b/flexmeasures/api/v1_1/tests/conftest.py
@@ -12,7 +12,7 @@
 
 
 @pytest.fixture(scope="module")
-def setup_api_test_data(db, setup_roles_users, add_market_prices):
+def setup_api_test_data(db, setup_account, setup_roles_users, add_market_prices):
     """
     Set up data for API v1.1 tests.
     """
@@ -28,6 +28,7 @@ def setup_api_test_data(db, setup_roles_users, add_market_prices):
         username="test user with improper registration",
         email="test_improper_user@seita.nl",
         password=hash_password("testtest"),
+        account_id=setup_account.id,
     )
     role = user_datastore.find_role("Prosumer")
     user_datastore.add_role_to_user(user, role)
@@ -37,6 +38,7 @@ def setup_api_test_data(db, setup_roles_users, add_market_prices):
         username="test user without roles",
         email="test_user@seita.nl",
         password=hash_password("testtest"),
+        account_name=setup_account.name,
     )
 
     # Create 3 test assets for the test_prosumer user

diff --git a/flexmeasures/api/v2_0/implementations/assets.py b/flexmeasures/api/v2_0/implementations/assets.py
@@ -9,6 +9,7 @@
 from marshmallow import fields
 
 from flexmeasures.data.services.resources import get_assets
+from flexmeasures.data.models.user import User
 from flexmeasures.data.models.assets import Asset as AssetModel
 from flexmeasures.data.schemas.assets import AssetSchema
 from flexmeasures.data.auth_setup import unauthorized_handler
@@ -115,6 +116,19 @@ def decorated_endpoint(*args, **kwargs):
     return wrapper
 
 
+def ensure_asset_remains_in_account(db_asset: AssetModel, new_owner_id: int):
+    """
+    Temporary protection of information kept in two places
+    (Asset.owner_id, GenericAsset.account_id) until we use GenericAssets throughout.
+    """
+    new_owner = User.query.get(new_owner_id)
+    if new_owner and new_owner.account != db_asset.owner.account:
+        raise abort(
+            400,
+            f"New owner {new_owner_id} not allowed, belongs to different account than current owner.",
+        )
+
+
 @load_asset()
 @as_json
 def fetch_one(asset):
@@ -129,6 +143,8 @@ def patch(db_asset, asset_data):
     """Update an asset given its identifier"""
     ignored_fields = ["id"]
     for k, v in [(k, v) for k, v in asset_data.items() if k not in ignored_fields]:
+        if k == "owner_id":
+            ensure_asset_remains_in_account(db_asset, v)
         setattr(db_asset, k, v)
     db.session.add(db_asset)
     try:

diff --git a/flexmeasures/api/v2_0/routes.py b/flexmeasures/api/v2_0/routes.py
@@ -47,14 +47,14 @@
 )
 v2_0_service_listing["services"].append(
     {
-        "name": "PATCH /assets/<id>",
+        "name": "PATCH /asset/<id>",
         "access": [],
         "description": "Edit an asset.",
     },
 )
 v2_0_service_listing["services"].append(
     {
-        "name": "DELETE /assets/<id>",
+        "name": "DELETE /asset/<id>",
         "access": [],
         "description": "Delete an asset and its data.",
     },
@@ -117,11 +117,11 @@ def get_assets():
                 "id": 1,
                 "latitude": 10,
                 "longitude": 100,
-                "market": 1,
+                "market_id": 1,
                 "max_soc_in_mwh": 5,
                 "min_soc_in_mwh": 0,
                 "name": "Test battery",
-                "owner": 2,
+                "owner_id": 2,
                 "soc_datetime": "2015-01-01T00:00:00+00:00",
                 "soc_in_mwh": 2.5,
                 "soc_udi_event_id": 203,
@@ -166,8 +166,8 @@ def post_assets():
             "name": "Test battery",
             "asset_type": "battery",
             "unit": "kW",
-            "owner": 2,
-            "market": 1,
+            "owner_id": 2,
+            "market_id": 1,
             "event_resolution": 5,
             "capacity_in_mw": 4.2,
             "latitude": 40,
@@ -197,8 +197,8 @@ def post_assets():
             "max_soc_in_mwh": 5,
             "min_soc_in_mwh": 0,
             "name": "Test battery",
-            "owner": 2,
-            "market": 1,
+            "owner_id": 2,
+            "market_id": 1,
             "soc_datetime": null,
             "soc_in_mwh": null,
             "soc_udi_event_id": null
@@ -238,11 +238,11 @@ def get_asset(id: int):
             "id": 1,
             "latitude": 10,
             "longitude": 100,
-            "market": 1,
+            "market_id": 1,
             "max_soc_in_mwh": 5,
             "min_soc_in_mwh": 0,
             "name": "Test battery",
-            "owner": 2,
+            "owner_id": 2,
             "soc_datetime": "2015-01-01T00:00:00+00:00",
             "soc_in_mwh": 2.5,
             "soc_udi_event_id": 203,
@@ -300,11 +300,11 @@ def patch_asset(id: int):
             "id": 1,
             "latitude": 11.1,
             "longitude": 99.9,
-            "market": 1,
+            "market_id": 1,
             "max_soc_in_mwh": 5,
             "min_soc_in_mwh": 0,
             "name": "Test battery",
-            "owner": 2,
+            "owner_id": 2,
             "soc_datetime": "2015-01-01T00:00:00+00:00",
             "soc_in_mwh": 2.5,
             "soc_udi_event_id": 203,
@@ -403,6 +403,7 @@ def get_user(id: int):
     .. sourcecode:: json
 
         {
+            'account_id': 1,
             'active': True,
             'email': 'test_prosumer@seita.nl',
             'flexmeasures_roles': [1, 3],
@@ -435,7 +436,7 @@ def patch_user(id: int):
     Only the user themselves or admins are allowed to update its data,
     while a non-admin can only edit a few of their own fields.
 
-    Several fields are not allowed to be updated, e.g. id. They are ignored.
+    Several fields are not allowed to be updated, e.g. id and account_id. They are ignored.
 
     **Example request**
 
@@ -452,6 +453,7 @@ def patch_user(id: int):
     .. sourcecode:: json
 
         {
+            'account_id': 1,
             'active': True,
             'email': 'test_prosumer@seita.nl',
             'flexmeasures_roles': [1, 3],

diff --git a/flexmeasures/api/v2_0/tests/conftest.py b/flexmeasures/api/v2_0/tests/conftest.py
@@ -23,7 +23,7 @@ def setup_api_test_data(db, setup_roles_users, add_market_prices, add_battery_as
 
 
 @pytest.fixture(scope="module")
-def setup_inactive_user(db, setup_roles_users):
+def setup_inactive_user(db, setup_account, setup_roles_users):
     """
     Set up one inactive user.
     """
@@ -34,5 +34,6 @@ def setup_inactive_user(db, setup_roles_users):
         username="inactive test user",
         email="inactive@seita.nl",
         password=hash_password("testtest"),
+        account_id=setup_account.id,
         active=False,
     )