FLEXMEASURES_PLUGINS configurable from env (#660)

* remove deprecated (since v0.7) setting FLEXMEASURES_PLUGIN_PATHS

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

* Allow FLEXMEASURES_PLUGINS to be read as env var and to be a comma-separated string; document wich settings are also read as env vars

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

* document how a plugin can add its own Docker image with the FLEXMEASURES_PLUGINS setting

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

* Not sunsetting the setting just yet, improve documentation

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

* protect better against trailing commas

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

* improve documentation of plugin setting

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

* add missing bracket

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

---------

Signed-off-by: Nicolas Höning <nicolas@seita.nl>

.vscode/spellright.dict

@@ -246,3 +246,7 @@ deserializing
 kwargs
 fsdomain
 filepath
+Changelog
+Bugfixes
+Dockerfile
+―

documentation/changelog.rst

@@ -15,6 +15,10 @@ Bugfixes
 Infrastructure / Support
 ----------------------
 
+* The setting FLEXMEASURES_PLUGINS can be set as environment variable now (as a comma-separated list) [see `PR #660 <https://www.github.com/FlexMeasures/flexmeasures/pull/660>`_]
+
+.. warning:: The setting `FLEXMEASURES_PLUGIN_PATHS` has been deprecated since v0.7. It has now been sunset. Please replace it with :ref:`plugin-config`.
+
 
 v0.13.0 | May 1, 2023
 ============================

documentation/configuration.rst

@@ -6,7 +6,7 @@ Configuration
 The following configurations are used by FlexMeasures.
 
 Required settings (e.g. postgres db) are marked with a double star (**).
-To enable easier quickstart tutorials, these required settings can be set by environment variables.
+To enable easier quickstart tutorials, continuous integration use cases and basic usage of FlexMeasures within other projects, these required settings, as well as a few others, can be set by environment variables ― this is also noted per setting.
 Recommended settings (e.g. mail, redis) are marked by one star (*).
 
 .. note:: FlexMeasures is best configured via a config file. The config file for FlexMeasures can be placed in one of two locations: 
@@ -26,6 +26,8 @@ Level above which log messages are added to the log file. See the ``logging`` pa
 
 Default: ``logging.WARNING``
 
+.. note:: This setting is also recognized as environment variable.
+
 
 .. _modes-config:
 
@@ -72,6 +74,7 @@ FLEXMEASURES_PLUGINS
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 
 A list of plugins you want FlexMeasures to load (e.g. for custom views or CLI functions). 
+This can be a Python list (e.g. ``["plugin1", "plugin2"]``) or a comma-separated string (e.g. ``"plugin1, plugin2"``).
 
 Two types of entries are possible here:
 
@@ -80,9 +83,10 @@ Two types of entries are possible here:
 
 Added functionality in plugins needs to be based on Flask Blueprints. See :ref:`plugins` for more information and examples.
 
-
 Default: ``[]``
 
+.. note:: This setting is also recognized as environment variable (since v0.14, which is also the version required to pass this setting as a string).
+
 
 FLEXMEASURES_DB_BACKUP_PATH
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -299,6 +303,8 @@ Token for accessing the MapBox API (for displaying maps on the dashboard and ass
 
 Default: ``None``
 
+.. note:: This setting is also recognized as environment variable.
+
 .. _sentry_access_token:
 
 SENTRY_SDN
@@ -309,6 +315,8 @@ E.g.: ``https://<examplePublicKey>@o<something>.ingest.sentry.io/<project-Id>``
 
 Default: ``None``
 
+.. note:: This setting is also recognized as environment variable.
+
 
 SQLAlchemy
 ----------
@@ -323,6 +331,9 @@ Connection string to the postgres database, format: ``postgresql://<user>:<passw
 
 Default: ``None``
 
+.. note:: This setting is also recognized as environment variable.
+
+
 SQLALCHEMY_ENGINE_OPTIONS
 ^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -432,6 +443,8 @@ For FlexMeasures to be able to send email to users (e.g. for resetting passwords
 This is only a selection of the most important settings.
 See `the Flask-Mail Docs <https://flask-mail.readthedocs.io/en/latest/#configuring-flask-mail>`_ for others.
 
+.. note:: The mail settings are also recognized as environment variables.
+
 MAIL_SERVER (*)
 ^^^^^^^^^^^^^^^
 
@@ -543,6 +556,9 @@ Redis
 
 FlexMeasures uses the Redis database to support our forecasting and scheduling job queues.
 
+.. note:: The redis settings are also recognized as environment variables.
+
+
 FLEXMEASURES_REDIS_URL (*)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
 

documentation/plugin/customisation.rst

@@ -16,7 +16,7 @@ but in the background your custom scheduling algorithm is being used.
 Let's walk through an example!
 
 First, we need to write a a class (inhering from the Base Scheduler) with a `schedule` function which accepts arguments just like the in-built schedulers (their code is `here <https://github.com/FlexMeasures/flexmeasures/tree/main/flexmeasures/data/models/planning>`_).
-The following minimal example gives you an idea of some meta information you can add for labelling your data, as well as the inputs and outputs of such a scheduling function:
+The following minimal example gives you an idea of some meta information you can add for labeling your data, as well as the inputs and outputs of such a scheduling function:
 
 .. code-block:: python
 
@@ -81,6 +81,34 @@ get computed by your custom function! For later lookup, the data will be linked
 .. todo:: We're planning to use a similar approach to allow for custom forecasting algorithms, as well.
 
 
+Deploying your plugin via Docker
+----------------------------------
+
+You can extend the FlexMeasures Docker image with your plugin's logic.
+
+Imagine your plugin package (with an ``__init__.py`` file, one of the setups we discussed in :ref:`plugin_showcase`) is called ``flexmeasures_testplugin``.
+Then, this is a minimal possible Dockerfile ― containers based on this will serve FlexMeasures (see the original Dockerfile in the FlexMeasures repository) with the plugin logic, like endpoints:
+
+.. code-block:: docker
+
+    FROM lfenergy/flexmeasures
+
+    COPY flexmeasures_testplugin/ /app/flexmeasures_testplugin
+    ENV FLEXMEASURES_PLUGINS="/app/flexmeasures_testplugin"
+
+You can of course also add multiple plugins this way.
+
+If you also want to install your requirements, you could for instance add these layers:
+
+.. code-block:: docker
+
+    COPY requirements/app.in /app/requirements/flexmeasures_testplugin.txt
+    RUN pip3 install --no-cache-dir -r requirements/flexmeasures_testplugin.txt
+
+.. note:: No need to install flexmeasures here, as the Docker image we are based on already installed FlexMeasures from code. If you pip3-install your plugin here (assuming it's on Pypi), check if it recognizes that FlexMeasures installation as it should.
+
+
+
 Adding your own style sheets
 ----------------------------
 

flexmeasures/utils/config_defaults.py

@@ -95,7 +95,7 @@ class Config(object):
     # This setting contains the domain on which FlexMeasures runs
     # and the first month when the domain was under the current owner's administration
     FLEXMEASURES_HOSTS_AND_AUTH_START: dict = {"flexmeasures.io": "2021-01"}
-    FLEXMEASURES_PLUGINS: List[str] = []
+    FLEXMEASURES_PLUGINS: List[str] | str = []  # str will be checked for commas
     FLEXMEASURES_PROFILE_REQUESTS: bool = False
     FLEXMEASURES_DB_BACKUP_PATH: str = "migrations/dumps"
     FLEXMEASURES_MENU_LOGO_PATH: str = ""

flexmeasures/utils/config_utils.py

@@ -163,7 +163,7 @@ def read_env_vars(app: Flask):
     for var in (
         required
         + list(warnable.keys())
-        + ["LOGGING_LEVEL", "MAPBOX_ACCESS_TOKEN", "SENTRY_SDN"]
+        + ["LOGGING_LEVEL", "MAPBOX_ACCESS_TOKEN", "SENTRY_SDN", "FLEXMEASURES_PLUGINS"]
     ):
         app.config[var] = os.getenv(var, app.config.get(var, None))
     # DEBUG in env can come in as a string ("True") so make sure we don't trip here

flexmeasures/utils/plugin_utils.py

@@ -24,12 +24,15 @@ def register_plugins(app: Flask):
     (last part of the path).
     """
     plugins = app.config.get("FLEXMEASURES_PLUGINS", [])
-    if not plugins:
-        # this is deprecated behaviour which we should remove in version 1.0
-        app.logger.debug(
-            "No plugins configured. Attempting deprecated setting FLEXMEASURES_PLUGIN_PATHS ..."
+    if not plugins and "FLEXMEASURES_PLUGIN_PATHS" in app.config:
+        app.logger.warning(
+            "Plugins found via FLEXMEASURES_PLUGIN_PATHS. This setting will be sunset in v0.14. Please switch to FLEXMEASURES_PLUGINS."
         )
         plugins = app.config.get("FLEXMEASURES_PLUGIN_PATHS", [])
+    if isinstance(plugins, str):
+        plugins = [
+            plugin.strip() for plugin in plugins.split(",") if len(plugin.strip()) > 0
+        ]
     if not isinstance(plugins, list):
         app.logger.error(
             f"The value of FLEXMEASURES_PLUGINS is not a list: {plugins}. Cannot install plugins ..."