Auth policy

documentation/changelog.rst

@@ -20,6 +20,7 @@ Bugfixes
 
 Infrastructure / Support
 ----------------------
+* Account-based authorization, incl. new decorators for endpoints [see `PR #210 <http://www.github.com/SeitaBV/flexmeasures/pull/210>`_]
 * FlexMeasures plugins can be Python packages now. We provide `a cookie-cutter template <https://github.com/SeitaBV/flexmeasures-plugin-template>`_ for this approach. [see `PR #182 <http://www.github.com/SeitaBV/flexmeasures/pull/182>`_]
 * Set default timezone for new users using the FLEXMEASURES_TIMEZONE config setting [see `PR #190 <http://www.github.com/SeitaBV/flexmeasures/pull/190>`_]
 * To avoid databases from filling up with irrelevant information, only beliefs data representing *changed beliefs are saved*, and *unchanged beliefs are dropped* [see `PR #194 <http://www.github.com/SeitaBV/flexmeasures/pull/194>`_]

documentation/concepts/services.rst → documentation/concepts/inbuilt-smart-functionality.rst

@@ -3,9 +3,16 @@
 .. _services:
 
 **************************************************
-Services 
+In-built smart functionality 
 **************************************************
 
+The main purpose of the FlexMeasures platform is to serve as a basis to rapidly built energy flexibility services.
+Much software architecture and wiring groundwork is already included for this purpose, like an API, support for plotting and multi-tenancy and extensibility.
+
+That said, several smart features come with FlexMeasures. Once the sensor structure and data is in place, they should be usable without much coding.
+
+.. todo:: We'll write more tutorials on this. 
+
 Monitoring
 -------------
 

documentation/concepts/security_auth.rst

@@ -18,16 +18,32 @@ data and time series for energy consumption/generation or weather).
 
 .. _auth:
 
-Authentication and Authorisation
----------------------------------
+Authentication 
+----------------
 
 *Authentication* is the system by which users tell the FlexMeasures platform that they are who they claim they are.
 This involves a username/password combination ("credentials") or an access token.
 
 * No user passwords are stored in clear text on any server - the FlexMeasures platform only stores the hashed passwords (encrypted with the `bcrypt hashing algorithm <https://passlib.readthedocs.io/en/stable/lib/passlib.hash.bcrypt.html>`_). If an attacker steals these password hashes, they cannot compute the passwords from them in a practical amount of time.
 * Access tokens are used so that the sending of usernames and passwords is limited (even if they are encrypted via https, see above) when dealing with the part of the FlexMeasures platform which sees the most traffic: the API functionality. Tokens thus have use cases for some scenarios, where developers want to treat authentication information with a little less care than credentials should be treated with, e.g. sharing among computers. However, they also expire fast, which is a common industry practice (by making them short-lived and requiring refresh, FlexMeasures limits the time an attacker can abuse a stolen token). At the moment, the access tokens on FlexMeasures platform expire after six hours. Access tokens are encrypted and validated with the `sha256_crypt algorithm <https://passlib.readthedocs.io/en/stable/lib/passlib.hash.sha256_crypt.html>`_, and `the functionality to expire tokens is realised by storing the seconds since January 1, 2011 in the token <https://pythonhosted.org/itsdangerous/#itsdangerous.TimestampSigner>`_. The maximum age of access tokens in FlexMeasures can be altered by setting the env variable `SECURITY_TOKEN_MAX_AGE` to the number of seconds after which tokens should expire.
 
-*Authorisation* is the system by which the FlexMeasures platform decides whether an authenticated user can access a feature. For instance, many features are reserved for administrators, others for Prosumers (the owner of assets).
 
-* This is achieved via *roles*. Each user has at least one role, but could have several, as well.
-* Roles cannot be edited via the UI at the moment. They are decided when a user is created.
+.. note:: Authentication (and authorization, see below) affects the FlexMeasures API and UI. The CLI (command line interface) can only be used if the user is already on the server and can execute ``flexmeasures`` commands, thus we can safely assume they are admins.
+
+
+Authorisation
+--------------
+
+*Authorisation* is the system by which the FlexMeasures platform decides whether an authenticated user can access a feature. For instance, many features are reserved for administrators, others for users belonging to certain accounts. An example for the latter is that a user might need to belong to an account with the "Prosumer" account role (usually the owner of assets).
+
+.. note:: Each user belongs to exactly one account.
+
+.. todo:: Data which belongs to a specific account should only be viewable by users within that account (and platform admins). We want to anchor this crucial security measure on a deep level, `see this ticket <https://github.com/SeitaBV/flexmeasures/issues/201>`_.
+
+All other authorization is achieved via *roles*. 
+
+* Account roles are most commonly used for deciding who can access a resource (usually guarded by an API endpoint). We support several roles which are mentioned in the USEF framework but more roles are possible (e.g. defined by customer services, see below). 
+* User roles give a user personal authorizations. For instance, we have a few `admin`s who can perform all actions, and `admin-reader`s who can read everything. Other roles have only an effect within the user's account.
+* Roles cannot be edited via the UI at the moment. They are decided when a user or account is created in the CLI (for adding roles later, we use the database for now). Editing roles in UI and CLI is future work.
+
+.. note:: Custom energy flexibility services developed on top of FlexMeasures can use account roles to achieve their custom authorization. E.g. if several services run on one FlexMeasures server, each service could define a "MyService-subscriber" account role, to make sure that only users of such accounts can use the endpoints. More on this in :ref:`auth-dev`.

documentation/concepts/users.rst

@@ -3,7 +3,8 @@
 Who can use the platform?
 =========================
 
-Potential users of the platform are listed below by their role in the Universal Smart Energy Framework (USEF).
+Various parties might be beneficiaries of using the FlexMeasures platform, and services built on top of it.
+Listed below are a few types, listed by their role in the Universal Smart Energy Framework (USEF), which are supported as account roles (see :ref:`auth`).
 
 
 Prosumers

documentation/dev/auth.rst

@@ -0,0 +1,29 @@
+.. _auth-dev:
+
+Custom authorization
+======================
+
+Section :ref:`auth` describes general authentication and authorization handling in FlexMeasures. However, custom energy flexibility services developed on top of FlexMeasures probably also want to their custom authorization. 
+
+One means for this is to define custom account roles. E.g. if several services run on one FlexMeasures server, each service could define a "MyService-subscriber" account role. To make sure that only users of such accounts can use the endpoints:
+
+.. code-block:: python
+
+    @flexmeasures_ui.route("/bananas")
+    @account_roles_required("MyService-subscriber")
+    def bananas_view:
+        pass
+
+.. note:: This endpoint decorator lists required roles, so the authenticated user's account needs to have each role. You can also use the ``account_roles_accepted`` decorator. Then the user's account only needs to have at least one of the roles.
+
+There is also decorators to check user roles:
+
+.. code-block:: python 
+
+    @flexmeasures_ui.route("/bananas")
+    @roles_required("account-admin")
+    def bananas_view:
+        pass
+
+.. note:: You can also use the ``roles_accepted`` decorator.
+

documentation/index.rst

@@ -39,6 +39,7 @@ Building customer-facing services is where developers make impact. We make their
 - Well-documented API
 - Plugin support
 - Plotting support
+- Multi-tenancy
 
 
 For more on FlexMeasures services, read :ref:`services`. Or head right over to :ref:`getting_started`.
@@ -66,9 +67,9 @@ The platform operator of FlexMeasures can be an Aggregator.
     :caption: Concepts
     :maxdepth: 1
 
-    concepts/services
     concepts/benefits
     concepts/benefits_of_flex
+    concepts/inbuilt-smart-functionality
     concepts/algorithms
     concepts/security_auth
 
@@ -120,6 +121,7 @@ The platform operator of FlexMeasures can be an Aggregator.
     dev/api
     dev/ci
     dev/plugins
+    dev/auth
     dev/error-monitoring
     dev/modes
 

flexmeasures/api/__init__.py

@@ -8,6 +8,7 @@
 from flexmeasures.api.common.utils.args_parsing import (
     validation_error_handler,
 )
+from flexmeasures.api.common.responses import invalid_sender
 from flexmeasures.data.schemas.utils import FMValidationError
 
 # The api blueprint. It is registered with the Flask app (see app.py)
@@ -83,6 +84,7 @@ def register_at(app: Flask):
 
     # handle API specific errors
     app.register_error_handler(FMValidationError, validation_error_handler)
+    app.unauthorized_handler_api = invalid_sender
 
     app.register_blueprint(
         flexmeasures_api, url_prefix="/api"

flexmeasures/api/common/implementations.py

@@ -8,7 +8,7 @@
 
 from flexmeasures.data.config import db
 from flexmeasures.data.models.task_runs import LatestTaskRun
-from flexmeasures.data.auth_setup import UNAUTH_STATUS_CODE, FORBIDDEN_STATUS_CODE
+from flexmeasures.auth.error_handling import UNAUTH_STATUS_CODE, FORBIDDEN_STATUS_CODE
 
 
 @as_json

flexmeasures/api/common/responses.py

@@ -1,7 +1,9 @@
-from typing import Optional, Tuple, Union, Sequence
+from typing import Callable, List, Optional, Tuple, Union, Sequence
 import inflect
 from functools import wraps
 
+from flexmeasures.auth.error_handling import FORBIDDEN_MSG, FORBIDDEN_STATUS_CODE
+
 p = inflect.engine()
 
 
@@ -140,28 +142,23 @@ def invalid_role(requested_access_role: str) -> ResponseTuple:
 
 
 def invalid_sender(
-    user_role_names: Union[str, Sequence[str]], *allowed_role_names: str
+    calling_function: Optional[Callable] = None,
+    allowed_role_names: Optional[List[str]] = None,
 ) -> ResponseTuple:
-    if isinstance(user_role_names, str):
-        user_role_names = [user_role_names]
-    if not user_role_names:
-        user_roles_str = "have no role"
-    else:
-        user_role_names = [p.a(role_name) for role_name in user_role_names]
-        user_roles_str = "are %s" % p.join(user_role_names)
-    allowed_role_names = tuple(
-        [pluralize(role_name) for role_name in allowed_role_names]
-    )
-    allowed_role_names = p.join(allowed_role_names)
+    """
+    Signify that a sender is invalid.
+    We use this as a stand-in for flask-security auth handlers,
+    thus the arguments fit it.
+    - calling_function is usually an auth decorator like roles_required.
+    - allowed_role_names can be used if the security check involved roles.
+    """
+    message = FORBIDDEN_MSG
+    if allowed_role_names:
+        allowed_role_names = [pluralize(role_name) for role_name in allowed_role_names]
+        message += f" It is reserved for {p.join(allowed_role_names)}."
     return (
-        dict(
-            result="Rejected",
-            status="INVALID_SENDER",
-            message="You don't have the right role to access this service. "
-            "You %s while this service is reserved for %s."
-            % (user_roles_str, allowed_role_names),
-        ),
-        403,
+        dict(result="Rejected", status="INVALID_SENDER", message=message),
+        FORBIDDEN_STATUS_CODE,
     )
 
 

flexmeasures/api/common/schemas/sensor_data.py

@@ -16,7 +16,7 @@
 
 
 class SingleValueField(fields.Float):
-    """Field that both deserializes and serializes a single value to a list of floats (length 1)."""
+    """Field that both de-serializes and serializes a single value to a list of floats (length 1)."""
 
     def _deserialize(self, value, attr, obj, **kwargs) -> List[float]:
         return [self._validated(value)]
@@ -42,7 +42,7 @@ def select_schema_to_ensure_list_of_floats(
             "values": 3.7
         }
 
-    Either will be deserialized to [3.7].
+    Either will be de-serialized to [3.7].
 
     Note that serialization always results in a list of floats.
     This ensures that we are not requiring the same flexibility from users who are retrieving data.

flexmeasures/api/common/utils/validators.py

@@ -10,7 +10,6 @@
 from pandas.tseries.frequencies import to_offset
 from flask import request, current_app
 from flask_json import as_json
-from flask_principal import Permission, RoleNeed
 from flask_security import current_user
 import marshmallow
 
@@ -27,7 +26,6 @@
     unapplicable_resolution,
     invalid_resolution_str,
     conflicting_resolutions,
-    invalid_sender,
     invalid_source,
     invalid_timezone,
     invalid_unit,
@@ -76,8 +74,8 @@ def validate_user_sources(sources: Union[int, str, List[Union[int, str]]]) -> Li
             except TypeError:
                 current_app.logger.warning("Could not retrieve data source %s" % source)
                 pass
-        else:  # Parse as role name
-            user_ids = [user.id for user in get_users(source)]
+        else:  # Parse as account role name
+            user_ids = [user.id for user in get_users(account_role_name=source)]
             user_source_ids.extend(
                 [
                     params[0]
@@ -912,46 +910,3 @@ def decorated_service(*args, **kwargs):
         return decorated_service
 
     return wrapper
-
-
-def usef_roles_accepted(*usef_roles):
-    """Decorator which specifies that a user must have at least one of the
-    specified USEF roles (or must be an admin). Example:
-
-        @app.route('/postMeterData')
-        @roles_accepted('Prosumer', 'MDC')
-        def post_meter_data():
-            return 'Meter data posted'
-
-    The current user must have either the `Prosumer` role or `MDC` role in
-    order to use the service.
-    And finally, users with the anonymous user role are never accepted.
-
-    :param usef_roles: The possible roles.
-    """
-
-    def wrapper(fn):
-        @wraps(fn)
-        @as_json
-        def decorated_service(*args, **kwargs):
-            perm = Permission(*[RoleNeed(role) for role in usef_roles])
-            if current_user.has_role(
-                "anonymous"
-            ):  # TODO: this role needs to go, we should not mix permissive and restrictive roles
-                current_app.logger.warning(
-                    "Anonymous user is not accepted for this service"
-                )
-                return invalid_sender("anonymous user", "non-anonymous user")
-            elif perm.can() or current_user.has_role("admin"):
-                return fn(*args, **kwargs)
-            else:
-                current_app.logger.warning(
-                    "User does not have necessary authorization for this service"
-                )
-                return invalid_sender(
-                    [role.name for role in current_user.roles], *usef_roles
-                )
-
-        return decorated_service
-
-    return wrapper

flexmeasures/api/dev/__init__.py

@@ -1,5 +1,7 @@
 from flask import Flask
-from flask_security import auth_token_required, roles_accepted
+from flask_security import auth_token_required
+
+from flexmeasures.auth.decorators import account_roles_accepted
 
 
 def register_at(app: Flask):
@@ -12,7 +14,7 @@ def register_at(app: Flask):
 
     @app.route("/sensorData", methods=["POST"])
     @auth_token_required
-    @roles_accepted("admin", "MDC", "Prosumer")
+    @account_roles_accepted("MDC", "Prosumer")
     def post_sensor_data():
         """
         Post sensor data to FlexMeasures.

flexmeasures/api/dev/sensors.py

@@ -7,6 +7,7 @@
 from webargs.flaskparser import use_kwargs
 from werkzeug.exceptions import abort
 
+from flexmeasures.auth.policy import ADMIN_ROLE
 from flexmeasures.data.schemas.times import AwareDateTimeField
 from flexmeasures.data.models.time_series import Sensor
 
@@ -67,7 +68,7 @@ def get(self, id: int):
 def get_sensor_or_abort(id: int) -> Sensor:
     sensor = Sensor.query.filter(Sensor.id == id).one_or_none()
     if not (
-        current_user.has_role("admin")
+        current_user.has_role(ADMIN_ROLE)
         or sensor.generic_asset.owner is None  # public
         or sensor.generic_asset.owner == current_user.account  # private but authorized
     ):

flexmeasures/api/dev/tests/conftest.py

@@ -1,8 +1,8 @@
 from datetime import timedelta
 
-from flask_security import SQLAlchemySessionUserDatastore
 import pytest
 
+from flexmeasures.data.models.user import Account, User
 from flexmeasures.data.models.generic_assets import GenericAssetType, GenericAsset
 from flexmeasures.data.models.time_series import Sensor
 
@@ -13,8 +13,8 @@ def setup_api_test_data(db, setup_roles_users):
     Set up data for API dev tests.
     """
     print("Setting up data for API v2.0 tests on %s" % db.engine)
-    add_gas_sensor(db, setup_roles_users["Test Supplier"])
-    give_prosumer_the_MDC_role(db)
+    add_gas_sensor(db, setup_roles_users["Test User 2"])
+    move_user2_to_supplier()
 
 
 @pytest.fixture(scope="function")
@@ -25,8 +25,8 @@ def setup_api_fresh_test_data(fresh_db, setup_roles_users_fresh_db):
     print("Setting up fresh data for API dev tests on %s" % fresh_db.engine)
     for sensor in Sensor.query.all():
         fresh_db.delete(sensor)
-    add_gas_sensor(fresh_db, setup_roles_users_fresh_db["Test Supplier"])
-    give_prosumer_the_MDC_role(fresh_db)
+    add_gas_sensor(fresh_db, setup_roles_users_fresh_db["Test User 2"])
+    move_user2_to_supplier()
 
 
 def add_gas_sensor(db, test_supplier):
@@ -38,6 +38,7 @@ def add_gas_sensor(db, test_supplier):
     incineration_asset = GenericAsset(
         name="incineration line",
         generic_asset_type=incineration_type,
+        account_id=test_supplier.account_id,
     )
     db.session.add(incineration_asset)
     db.session.flush()
@@ -51,11 +52,12 @@ def add_gas_sensor(db, test_supplier):
     gas_sensor.owner = test_supplier
 
 
-def give_prosumer_the_MDC_role(db):
-
-    from flexmeasures.data.models.user import User, Role
-
-    user_datastore = SQLAlchemySessionUserDatastore(db.session, User, Role)
-    test_prosumer = user_datastore.find_user(email="test_prosumer@seita.nl")
-    mdc_role = user_datastore.create_role(name="MDC", description="Meter Data Company")
-    user_datastore.add_role_to_user(test_prosumer, mdc_role)
+def move_user2_to_supplier():
+    """
+    move the user 2 to the supplier account
+    """
+    supplier_account = Account.query.filter(
+        Account.name == "Test Supplier Account"
+    ).one_or_none()
+    user2 = User.query.filter(User.email == "test_user_2@seita.nl").one_or_none()
+    user2.account = supplier_account