Sunset API versions 1 and 2 (#650)

This PR sunsets API versions 1, 1.1, 1.2, 1.3 and 2.0. - Move worthwhile tests to API v3 - Remove all other tests for sunset API versions - Add sunset notice to changelog - Remove obsolete documentation (keeping a reference to snapshot on readthedocs.org) - Remove obsolete script for manual simulations against sunset API versions - Add service listing to v3 - Document public endpoints in v3 (authentication, version listing and service listing) * remove tests already present in v3 Signed-off-by: F.N. Claessen <felix@seita.nl> * move asset deletion test to v3 Signed-off-by: F.N. Claessen <felix@seita.nl> * move asset creation test to v3 Signed-off-by: F.N. Claessen <felix@seita.nl> * remove last remaining asset test from v2 Signed-off-by: F.N. Claessen <felix@seita.nl> * stop testing post_price_data and post_prognosis on v2 Signed-off-by: F.N. Claessen <felix@seita.nl> * move test scheduling with unknown prices from v1.3 to v3 Signed-off-by: F.N. Claessen <felix@seita.nl> * remove v1.3 test already present in v3 Signed-off-by: F.N. Claessen <felix@seita.nl> * move test for wrong job id from v1.3 to v3 Signed-off-by: F.N. Claessen <felix@seita.nl> * no tests worth moving in v1.2 Signed-off-by: F.N. Claessen <felix@seita.nl> * remove test for posting weather data from v1.1 Signed-off-by: F.N. Claessen <felix@seita.nl> * move test for posting data with incompatible resolution from v1.1 to v3 Signed-off-by: F.N. Claessen <felix@seita.nl> * move test for posting data with a user that is not properly set up as a data source, from v1.1 to v3; also test posting is not allowed for inactive admins Signed-off-by: F.N. Claessen <felix@seita.nl> * move test for getting data for an empty period from v1.1 to v3 Signed-off-by: F.N. Claessen <felix@seita.nl> * move v3 tests that just GET data to test module that uses db fixture rather than fresh_db fixture Signed-off-by: F.N. Claessen <felix@seita.nl> * remove v1.1 test utils including old simulation script that uses one of them Signed-off-by: F.N. Claessen <felix@seita.nl> * salvage only 1 test from v1, which logs out a user Signed-off-by: F.N. Claessen <felix@seita.nl> * add missing fixtures to let tests succeed on their own, too Signed-off-by: F.N. Claessen <felix@seita.nl> * remove v1.0 test utils Signed-off-by: F.N. Claessen <felix@seita.nl> * add changelog warning Signed-off-by: F.N. Claessen <felix@seita.nl> * update API documentation Signed-off-by: F.N. Claessen <felix@seita.nl> * Document public endpoints in v3 Signed-off-by: F.N. Claessen <felix@seita.nl> * black Signed-off-by: F.N. Claessen <felix@seita.nl> * Implement getService for API v3 Signed-off-by: F.N. Claessen <felix@seita.nl> * Document service listing for v3 Signed-off-by: F.N. Claessen <felix@seita.nl> * Show quickref contents rather than summary line of docstring Signed-off-by: F.N. Claessen <felix@seita.nl> * Fix removal of suffix and prefix, by not using rstrip and lstrip, respectively Signed-off-by: F.N. Claessen <felix@seita.nl> * API changelog entry Signed-off-by: F.N. Claessen <felix@seita.nl> * Make getService more RESTful Signed-off-by: F.N. Claessen <felix@seita.nl> * Add warning to API developer docs Signed-off-by: F.N. Claessen <felix@seita.nl> * More robust against spaces Signed-off-by: F.N. Claessen <felix@seita.nl> * Update main changelog entry Signed-off-by: F.N. Claessen <felix@seita.nl> --------- Signed-off-by: F.N. Claessen <felix@seita.nl>
FlexMeasures · May 1, 2023 · 4c66fbf · 4c66fbf
1 parent 47f71f4
commit 4c66fbf
Show file tree

Hide file tree

Showing 45 changed files with 481 additions and 2,846 deletions.
diff --git a/documentation/api/change_log.rst b/documentation/api/change_log.rst
@@ -5,6 +5,12 @@ API change log
 
 .. note:: The FlexMeasures API follows its own versioning scheme. This is also reflected in the URL, allowing developers to upgrade at their own pace.
 
+v3.0-9 | 2023-04-26
+"""""""""""""""""""
+
+- Added missing documentation for the public endpoints for authentication and listing active API versions.
+- Added REST endpoint for listing available services for a specific API version: `/api/v3_0` (GET). This functionality is similar to the *getService* endpoint for older API versions, but now also returns the full URL for each available service.
+
 v3.0-8 | 2023-03-23
 """""""""""""""""""
 
@@ -82,7 +88,7 @@ v3.0-0 | 2022-03-25
     - *getDeviceMessage* -> use `/sensors/<id>/schedules/<uuid>` (GET) instead, where <id> is the sensor id from the "event" field and <uuid> is the value of the "schedule" field returned by `/sensors/<id>/schedules/trigger` (POST)
     - *getMeterData* -> use `/sensors/data` (GET) instead, replacing the "connection" field with "sensor"
     - *getPrognosis* -> use `/sensors/data` (GET) instead, replacing the "connection" field with "sensor"
-    - *getService* -> consult the public API documentation instead, at https://flexmeasures.readthedocs.io
+    - *getService* -> use `/api/v3_0` (GET) instead (since v3.0-9), or consult the public API documentation instead, at https://flexmeasures.readthedocs.io
     - *postMeterData* -> use `/sensors/data` (POST) instead, replacing the "connection" field with "sensor"
     - *postPriceData* -> use `/sensors/data` (POST) instead, replacing the "market" field with "sensor"
     - *postPrognosis* -> use `/sensors/data` (POST) instead, replacing the "connection" field with "sensor"
@@ -106,8 +112,13 @@ v3.0-0 | 2022-03-25
     - Rewrote the sections on roles and sources into a combined section that refers to account roles rather than USEF roles.
     - Deprecated the section on group notation.
 
+v2.0-6 | 2022-04-26
+"""""""""""""""""""
+
+*API v2.0 is sunset.*
+
 v2.0-5 | 2022-02-13
-""""""""""""""""""""
+"""""""""""""""""""
 
 *API v2.0 is deprecated.*
 
@@ -152,6 +163,11 @@ v2.0-0 | 2020-11-14
 
 - Added REST endpoints for managing assets: `/assets/` (GET, POST) and `/asset/<id>` (GET, PATCH, DELETE).
 
+v1.3-13 | 2022-04-26
+""""""""""""""""""""
+
+*API v1.3 is sunset.*
+
 v1.3-12 | 2022-02-13
 """"""""""""""""""""
 
@@ -243,8 +259,13 @@ v1.3-0 | 2020-01-28
 - The *postUdiEvent* endpoint now triggers scheduling jobs to be set up (rather than scheduling directly triggered by the *getDeviceMessage* endpoint)
 - The *getDeviceMessage* now queries the job queue and database for an available schedule
 
+v1.2-5 | 2022-04-26
+"""""""""""""""""""
+
+*API v1.2 is sunset.*
+
 v1.2-4 | 2022-02-13
-""""""""""""""""""""
+"""""""""""""""""""
 
 *API v1.2 is deprecated.*
 
@@ -294,8 +315,13 @@ v1.2-0 | 2018-09-08
 - Added a description of the *postUdiEvent* endpoint in the Prosumer and Simulation sections
 - Added a description of the *getDeviceMessage* endpoint in the Prosumer and Simulation sections
 
+v1.1-7 | 2022-04-26
+"""""""""""""""""""
+
+*API v1.1 is sunset.*
+
 v1.1-6 | 2022-02-13
-""""""""""""""""""""
+"""""""""""""""""""
 
 *API v1.1 is deprecated.*
 
@@ -352,8 +378,13 @@ v1.1-0 | 2018-07-15
 
 - Added a description of the *getPrognosis* endpoint in the Supplier section
 
+v1.0-3 | 2022-04-26
+"""""""""""""""""""
+
+*API v1.0 is sunset.*
+
 v1.0-2 | 2022-02-13
-""""""""""""""""""""
+"""""""""""""""""""
 
 *API v1.0 is deprecated.*
 

diff --git a/documentation/api/v1.rst b/documentation/api/v1.rst
@@ -3,20 +3,5 @@
 Version 1.0
 ===========
 
-.. warning:: This API version is deprecated since December 14, 2022, and will likely be sunset in February 2023. Please update to :ref:`v3_0`. For more information about how FlexMeasures handles deprecation and sunsetting, see :ref:`api_deprecation`.
 
-Summary
--------
-
-.. qrefflask:: flexmeasures.app:create(env="documentation")
-    :blueprints: flexmeasures_api, flexmeasures_api_play, flexmeasures_api_v1
-    :order: path
-    :include-empty-docstring:
-
-API Details
------------
-
-.. autoflask:: flexmeasures.app:create(env="documentation")
-    :blueprints: flexmeasures_api, flexmeasures_api_play, flexmeasures_api_v1
-    :order: path
-    :include-empty-docstring:
+.. note:: This API version has been sunset. Please update to :ref:`v3_0`. For more information about how FlexMeasures handles deprecation and sunsetting, see :ref:`api_deprecation`. In case your FlexMeasures host still runs ``flexmeasures<0.13``, a snapshot of documentation for API version 1.0 will stay available `here on readthedocs.org <https://flexmeasures.readthedocs.io/en/v0.12.3/api/v1.html>`_.
diff --git a/documentation/api/v1_1.rst b/documentation/api/v1_1.rst
@@ -3,20 +3,5 @@
 Version 1.1
 ===========
 
-.. warning:: This API version is deprecated since December 14, 2022, and will likely be sunset in February 2023. Please update to :ref:`v3_0`. For more information about how FlexMeasures handles deprecation and sunsetting, see :ref:`api_deprecation`.
 
-Summary
--------
-
-.. qrefflask:: flexmeasures.app:create(env="documentation")
-    :blueprints: flexmeasures_api, flexmeasures_api_play, flexmeasures_api_v1_1
-    :order: path
-    :include-empty-docstring:
-
-API Details
------------
-
-.. autoflask:: flexmeasures.app:create(env="documentation")
-    :blueprints: flexmeasures_api, flexmeasures_api_play, flexmeasures_api_v1_1
-    :order: path
-    :include-empty-docstring:
+.. note:: This API version has been sunset. Please update to :ref:`v3_0`. For more information about how FlexMeasures handles deprecation and sunsetting, see :ref:`api_deprecation`. In case your FlexMeasures host still runs ``flexmeasures<0.13``, a snapshot of documentation for API version 1.1 will stay available `here on readthedocs.org <https://flexmeasures.readthedocs.io/en/v0.12.3/api/v1_1.html>`_.
diff --git a/documentation/api/v1_2.rst b/documentation/api/v1_2.rst
@@ -3,20 +3,5 @@
 Version 1.2
 ===========
 
-.. warning:: This API version is deprecated since December 14, 2022, and will likely be sunset in February 2023. Please update to :ref:`v3_0`. For more information about how FlexMeasures handles deprecation and sunsetting, see :ref:`api_deprecation`.
 
-Summary
--------
-
-.. qrefflask:: flexmeasures.app:create(env="documentation")
-    :blueprints: flexmeasures_api, flexmeasures_api_play, flexmeasures_api_v1_2
-    :order: path
-    :include-empty-docstring:
-
-API Details
------------
-
-.. autoflask:: flexmeasures.app:create(env="documentation")
-    :blueprints: flexmeasures_api, flexmeasures_api_play, flexmeasures_api_v1_2
-    :order: path
-    :include-empty-docstring:
+.. note:: This API version has been sunset. Please update to :ref:`v3_0`. For more information about how FlexMeasures handles deprecation and sunsetting, see :ref:`api_deprecation`. In case your FlexMeasures host still runs ``flexmeasures<0.13``, a snapshot of documentation for API version 1.2 will stay available `here on readthedocs.org <https://flexmeasures.readthedocs.io/en/v0.12.3/api/v1_2.html>`_.
diff --git a/documentation/api/v1_3.rst b/documentation/api/v1_3.rst
@@ -3,20 +3,5 @@
 Version 1.3
 ===========
 
-.. warning:: This API version is deprecated since December 14, 2022, and will likely be sunset in February 2023. Please update to :ref:`v3_0`. For more information about how FlexMeasures handles deprecation and sunsetting, see :ref:`api_deprecation`.
 
-Summary
--------
-
-.. qrefflask:: flexmeasures.app:create(env="documentation")
-    :blueprints: flexmeasures_api, flexmeasures_api_play, flexmeasures_api_v1_3
-    :order: path
-    :include-empty-docstring:
-
-API Details
------------
-
-.. autoflask:: flexmeasures.app:create(env="documentation")
-    :blueprints: flexmeasures_api, flexmeasures_api_play, flexmeasures_api_v1_3
-    :order: path
-    :include-empty-docstring:
+.. note:: This API version has been sunset. Please update to :ref:`v3_0`. For more information about how FlexMeasures handles deprecation and sunsetting, see :ref:`api_deprecation`. In case your FlexMeasures host still runs ``flexmeasures<0.13``, a snapshot of documentation for API version 1.3 will stay available `here on readthedocs.org <https://flexmeasures.readthedocs.io/en/v0.12.3/api/v1_3.html>`_.
diff --git a/documentation/api/v2_0.rst b/documentation/api/v2_0.rst
@@ -3,20 +3,4 @@
 Version 2.0
 ===========
 
-.. warning:: This API version is deprecated since December 14, 2022, and will likely be sunset in February 2023. Please update to :ref:`v3_0`. For more information about how FlexMeasures handles deprecation and sunsetting, see :ref:`api_deprecation`.
-
-Summary
--------
-
-.. qrefflask:: flexmeasures.app:create(env="documentation")
-    :blueprints: flexmeasures_api, flexmeasures_api_play, flexmeasures_api_v2_0
-    :order: path
-    :include-empty-docstring:
-
-API Details
------------
-
-.. autoflask:: flexmeasures.app:create(env="documentation")
-    :blueprints: flexmeasures_api, flexmeasures_api_play, flexmeasures_api_v2_0
-    :order: path
-    :include-empty-docstring:
+.. note:: This API version has been sunset. Please update to :ref:`v3_0`. For more information about how FlexMeasures handles deprecation and sunsetting, see :ref:`api_deprecation`. In case your FlexMeasures host still runs ``flexmeasures<0.13``, a snapshot of documentation for API version 2.0 will stay available `here on readthedocs.org <https://flexmeasures.readthedocs.io/en/v0.12.3/api/v2_0.html>`_.
diff --git a/documentation/api/v3_0.rst b/documentation/api/v3_0.rst
@@ -7,14 +7,14 @@ Summary
 -------
 
 .. qrefflask:: flexmeasures.app:create(env="documentation")
-    :modules: flexmeasures.api.v3_0.assets, flexmeasures.api.v3_0.sensors, flexmeasures.api.v3_0.users, flexmeasures.api.v3_0.health
+    :modules: flexmeasures.api, flexmeasures.api.v3_0.assets, flexmeasures.api.v3_0.sensors, flexmeasures.api.v3_0.users, flexmeasures.api.v3_0.health, flexmeasures.api.v3_0.public
     :order: path
     :include-empty-docstring:
 
 API Details
 -----------
 
 .. autoflask:: flexmeasures.app:create(env="documentation")
-    :modules: flexmeasures.api.v3_0.assets, flexmeasures.api.v3_0.sensors, flexmeasures.api.v3_0.users, flexmeasures.api.v3_0.health
+    :modules: flexmeasures.api, flexmeasures.api.v3_0.assets, flexmeasures.api.v3_0.sensors, flexmeasures.api.v3_0.users, flexmeasures.api.v3_0.health, flexmeasures.api.v3_0.public
     :order: path
     :include-empty-docstring:
diff --git a/documentation/changelog.rst b/documentation/changelog.rst
@@ -6,6 +6,9 @@ FlexMeasures Changelog
 v0.13.0 | April XX, 2023
 ============================
 
+.. warning:: Sunset notice for API versions 1.0, 1.1, 1.2, 1.3 and 2.0: after upgrading to ``flexmeasures==0.13``, users of these API versions may receive ``HTTP status 410 (Gone)`` responses.
+             The relevant endpoints have been deprecated since ``flexmeasures==0.12``.
+
 .. warning:: The API endpoint (`[POST] /sensors/(id)/schedules/trigger <api/v3_0.html#post--api-v3_0-sensors-(id)-schedules-trigger>`_) to make new schedules sunsets the deprecated (since v0.12) storage flexibility parameters (they move to the ``flex-model`` parameter group), as well as the parameters describing other sensors (they move to ``flex-context``).
 
 .. warning:: Upgrading to this version requires running ``flexmeasures db upgrade`` (you can create a backup first with ``flexmeasures db-ops dump``).
@@ -27,6 +30,7 @@ Bugfixes
 
 Infrastructure / Support
 ----------------------
+* Sunset API versions 1.0, 1.1, 1.2, 1.3 and 2.0 [see `PR #650 <https://www.github.com/FlexMeasures/flexmeasures/pull/650>`_]
 * Sunset several API fields for `/sensors/<id>/schedules/trigger` (POST) that have moved into the ``flex-model`` or ``flex-context`` fields [see `PR #580 <https://www.github.com/FlexMeasures/flexmeasures/pull/580>`_]
 * Fix broken `make show-data-model` command [see `PR #638 <https://www.github.com/FlexMeasures/flexmeasures/pull/638>`_]
 * Bash script for a clean database to run toy-tutorial by using `make clean-db db_name=database_name` command [see `PR #640 <https://github.com/FlexMeasures/flexmeasures/pull/640>`_]
@@ -533,7 +537,7 @@ Infrastructure / Support
 * Integration with `timely beliefs <https://github.com/SeitaBV/timely-beliefs>`__ lib: Sensors [see `PR #13 <https://www.github.com/FlexMeasures/flexmeasures/pull/13>`_]
 * Apache 2.0 license [see `PR #16 <https://www.github.com/FlexMeasures/flexmeasures/pull/16>`_]
 * Load js & css from CDN [see `PR #21 <https://www.github.com/FlexMeasures/flexmeasures/pull/21>`_]
-* Start using marshmallow for input validation, also introducing ``HTTP status 422`` in the API [see `PR #25 <https://www.github.com/FlexMeasures/flexmeasures/pull/25>`_]
+* Start using marshmallow for input validation, also introducing ``HTTP status 422 (Unprocessable Entity)`` in the API [see `PR #25 <https://www.github.com/FlexMeasures/flexmeasures/pull/25>`_]
 * Replace ``solarpy`` with ``pvlib`` (due to license conflict) [see `PR #16 <https://www.github.com/FlexMeasures/flexmeasures/pull/16>`_]
 * Stop supporting the creation of new users on asset creation (to reduce complexity) [see `PR #36 <https://www.github.com/FlexMeasures/flexmeasures/pull/36>`_]
 
diff --git a/documentation/dev/api.rst b/documentation/dev/api.rst
@@ -7,6 +7,8 @@ The FlexMeasures API is the main way that third-parties can automate their inter
 
 This is a small guide for creating new versions of the API and its docs.
 
+.. warning:: This guide was written for API versions below v3.0 and is currently out of date.
+
 .. todo:: A guide for endpoint design, e.g. using Marshmallow schemas and common validators.
 
 .. contents:: Table of contents

diff --git a/flexmeasures/__init__.py b/flexmeasures/__init__.py
@@ -1,7 +1,12 @@
 from importlib_metadata import version, PackageNotFoundError
 
 from flexmeasures.data.models.annotations import Annotation  # noqa F401
-from flexmeasures.data.models.user import Account, AccountRole, User  # noqa F401
+from flexmeasures.data.models.user import (  # noqa F401
+    Account,
+    AccountRole,
+    User,
+    Role as UserRole,
+)
 from flexmeasures.data.models.data_sources import DataSource as Source  # noqa F401
 from flexmeasures.data.models.generic_assets import (  # noqa F401
     GenericAsset as Asset,

diff --git a/flexmeasures/api/__init__.py b/flexmeasures/api/__init__.py
@@ -73,9 +73,9 @@ def get_versions() -> dict:
     """
     response = {
         "message": "For these API versions a public endpoint is available, listing its service. For example: "
-        "/api/v2_0/getService and /api/v3_0/getService. An authentication token can be requested at: "
+        "/api/v3_0. An authentication token can be requested at: "
         "/api/requestAuthToken",
-        "versions": ["v1", "v1_1", "v1_2", "v1_3", "v2_0", "v3_0"],
+        "versions": ["v3_0"],
         "flexmeasures_version": flexmeasures_version,
     }
     return response

diff --git a/flexmeasures/api/common/schemas/sensor_data.py b/flexmeasures/api/common/schemas/sensor_data.py
@@ -10,12 +10,12 @@
 import pandas as pd
 
 from flexmeasures.data import ma
-from flexmeasures.data.models.data_sources import DataSource
 from flexmeasures.data.models.time_series import Sensor
 from flexmeasures.api.common.schemas.sensors import SensorField
 from flexmeasures.api.common.utils.api_utils import upsample_values
 from flexmeasures.data.models.planning.utils import initialize_index
 from flexmeasures.data.schemas import AwareDateTimeField, DurationField, SourceIdField
+from flexmeasures.data.services.data_sources import get_or_create_source
 from flexmeasures.data.services.time_series import simplify_index
 from flexmeasures.utils.time_utils import (
     decide_resolution,
@@ -350,14 +350,7 @@ def load_bdf(sensor_data: dict) -> BeliefsDataFrame:
         """
         Turn the de-serialized and validated data into a BeliefsDataFrame.
         """
-        source = DataSource.query.filter(
-            DataSource.user_id == current_user.id
-        ).one_or_none()
-        if not source:
-            raise ValidationError(
-                f"User {current_user.id} is not an accepted data source."
-            )
-
+        source = get_or_create_source(current_user)
         num_values = len(sensor_data["values"])
         event_resolution = sensor_data["duration"] / num_values
         dt_index = pd.date_range(

diff --git a/flexmeasures/api/v1/tests/__init__.py b/flexmeasures/api/v1/tests/__init__.py