feat: in play mode, allow showing any sensor on the asset page (#740)

This PR uses the FlexMeasures play mode to allow showing sensors across organisations. When showing simulation results, we want to be able to show results from different scenarios (stored under different accounts) on the same asset page. For example, when comparing a scenario against some other benchmark scenario.


* feat: in play mode, allow showing any sensor on the asset page

Signed-off-by: F.N. Claessen <felix@seita.nl>

* fix checking for config value

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

* more consistent naming of account variable

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

* chore: rename variable of accessible sensors for readability

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

* docs: mention this new possibility in function docstring

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

* fix: do not fail if sensors are not accessible, log a warning for them as well

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

* docs: changelog entry

Signed-off-by: F.N. Claessen <felix@seita.nl>

* docs: changelog warning

Signed-off-by: F.N. Claessen <felix@seita.nl>

---------

Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
Co-authored-by: Nicolas Höning <nicolas@seita.nl>

documentation/changelog.rst

@@ -9,11 +9,14 @@ v0.15.0 | July XX, 2023
 .. warning:: Upgrading to this version requires running ``flexmeasures db upgrade`` (you can create a backup first with ``flexmeasures db-ops dump``).
 .. warning:: Upgrading to this version requires installing the LP/MILP solver HiGHS using ``pip install highspy``.
 
+.. warning:: If your server is running in play mode (``FLEXMEASURES_MODE = "play"``), users will be able to see sensor data from any account [see `PR #740 <https://www.github.com/FlexMeasures/flexmeasures/pull/740>`_].
+
 New features
 -------------
 
 * Allow deleting multiple sensors with a single call to ``flexmeasures delete sensor`` by passing the ``--id`` option multiple times [see `PR #734 <https://www.github.com/FlexMeasures/flexmeasures/pull/734>`_]
 * Make it a lot easier to read off the color legend on the asset page, especially when showing many sensors, as they will now be ordered from top to bottom in the same order as they appear in the chart (as defined in the ``sensors_to_show`` attribute), rather than alphabetically [see `PR #742 <https://www.github.com/FlexMeasures/flexmeasures/pull/742>`_]
+* Users on FlexMeasures servers in play mode (``FLEXMEASURES_MODE = "play"``) can use the ``sensors_to_show`` attribute to show any sensor on their asset pages, rather than only sensors registered to assets in their own account or to public assets [see `PR #740 <https://www.github.com/FlexMeasures/flexmeasures/pull/740>`_]
 * Having percentages within the [0, 100] domain is such a common use case that we now always include it in sensor charts with % units, making it easier to read off individual charts and also to compare across charts [see `PR #739 <https://www.github.com/FlexMeasures/flexmeasures/pull/739>`_]
 * DataSource table now allows storing arbitrary attributes as a JSON (without content validation), similar to the Sensor and GenericAsset tables [see `PR #750 <https://www.github.com/FlexMeasures/flexmeasures/pull/750>`_]
 * Added API endpoints `/sensors/<id>` for fetching a single sensor and `/sensors` (POST) for adding a sensor. [see `PR #759 <https://www.github.com/FlexMeasures/flexmeasures/pull/759>`_] and [see `PR #767 <https://www.github.com/FlexMeasures/flexmeasures/pull/767>`_]

documentation/host/modes.rst

@@ -39,3 +39,4 @@ Small features
 - [API] Posted UDI events are not enforced to be consecutive.
 - [API] Names in ``GetConnectionResponse`` are the connections' unique database names rather than their display names (this feature is planned to be deprecated).
 - [UI] The dashboard plot showing the latest power value is not enforced to lie in the past (in case of simulating future values).
+- [UI] On the asset page, the ``sensors_to_show`` attribute can be used to show any sensor from any account, rather than only sensors from assets owned by the user's organization.

flexmeasures/data/models/generic_assets.py

@@ -4,6 +4,7 @@
 from typing import Any, Dict, Optional, Tuple, List, Union
 import json
 
+from flask import current_app
 from flask_security import current_user
 import pandas as pd
 from sqlalchemy.engine import Row
@@ -434,7 +435,7 @@ def sensors_to_show(self) -> list["Sensor" | list["Sensor"]]:  # noqa F821
         Sensors to show are defined as a list of sensor ids, which
         is set by the "sensors_to_show" field of the asset's "attributes" column.
         Valid sensors either belong to the asset itself, to other assets in the same account,
-        or to public assets.
+        or to public assets. In play mode, sensors from different accounts can be added.
         In case the field is missing, defaults to two of the asset's sensors.
 
         Sensor ids can be nested to denote that sensors should be 'shown together',
@@ -453,25 +454,52 @@ def sensors_to_show(self) -> list["Sensor" | list["Sensor"]]:  # noqa F821
         if not self.has_attribute("sensors_to_show"):
             return self.sensors[:2]
 
+        # Only allow showing sensors from assets owned by the user's organization,
+        # except in play mode, where any sensor may be shown
+        accounts = [self.owner]
+        if current_app.config.get("FLEXMEASURES_MODE") == "play":
+            from flexmeasures.data.models.user import Account
+
+            accounts = Account.query.all()
+
         from flexmeasures.data.services.sensors import get_sensors
 
         sensor_ids_to_show = self.get_attribute("sensors_to_show")
-        sensor_map = {
+        accessible_sensor_map = {
             sensor.id: sensor
             for sensor in get_sensors(
-                account=self.owner,
+                account=accounts,
                 include_public_assets=True,
                 sensor_id_allowlist=flatten_unique(sensor_ids_to_show),
             )
         }
 
-        # Return sensors in the order given by the sensors_to_show attribute, and with the same nesting
+        # Build list of sensor objects that are accessible
         sensors_to_show = []
+        missed_sensor_ids = []
+
+        # we make sure to build in the order given by the sensors_to_show attribute, and with the same nesting
         for s in sensor_ids_to_show:
             if isinstance(s, list):
-                sensors_to_show.append([sensor_map[sensor_id] for sensor_id in s])
+                inaccessible = [sid for sid in s if sid not in accessible_sensor_map]
+                missed_sensor_ids.extend(inaccessible)
+                if len(inaccessible) < len(s):
+                    sensors_to_show.append(
+                        [
+                            accessible_sensor_map[sensor_id]
+                            for sensor_id in s
+                            if sensor_id in accessible_sensor_map
+                        ]
+                    )
             else:
-                sensors_to_show.append(sensor_map[s])
+                if s not in accessible_sensor_map:
+                    missed_sensor_ids.append(s)
+                else:
+                    sensors_to_show.append(accessible_sensor_map[s])
+        if missed_sensor_ids:
+            current_app.logger.warning(
+                f"Cannot include sensor(s) {missed_sensor_ids} in sensors_to_show on asset {self}, as it is not accessible to user {current_user}."
+            )
         return sensors_to_show
 
     @property