FlexMeasures · nhoening · May 21, 2021 · May 7, 2021 · May 7, 2021 · May 7, 2021
diff --git a/documentation/api/introduction.rst b/documentation/api/introduction.rst
@@ -4,28 +4,67 @@ Introduction
 ============
 
 This document details the Application Programming Interface (API) of the FlexMeasures web service. The API supports user automation for flexibility valorisation in the energy sector, both in a live setting and for the purpose of simulating scenarios. The web service adheres to the concepts and terminology used in the Universal Smart Energy Framework (USEF).
-We assume in this document that the FlexMeasures instance you want to connect to is hosted at https://company.flexmeasures.io.
 
 
-New versions of the API are released on:
+.. _api_versions:
+
+Main endpoint and API versions
+------------------------------
+
+All versions of the API are released on:
+
+.. code-block:: html
+
+    https://<flexmeasures-root-url>/api
+
+So if you are running FlexMeasures on your computer, it would be:
+
+.. code-block:: html
+
+    https://localhost:5000/api
+
+At Seita, we run servers for our clients at:
 
 .. code-block:: html
 
     https://company.flexmeasures.io/api
 
-A list of services offered by (a version of) the FlexMeasures web service can be obtained by sending a *getService* request. An optional field "access" can be used to specify a user role for which to obtain only the relevant services.
+where `company` is a hosting customer of ours. All their accounts' data lives on that server.
 
-**Example request**
+We assume in this document that the FlexMeasures instance you want to connect to is hosted at https://company.flexmeasures.io.
 
-.. code-block:: json
+Let's see what the ``/api`` endpoint returns:
 
-    {
-        "type": "GetServiceRequest",
-        "version": "1.0"
+.. code-block:: python
+
+    >>> import requests
+    >>> res = requests.get("https://company.flexmeasures.io/api")
+    >>> res.json()
+    {'flexmeasures_version': '0.4.0',
+     'message': 'For these API versions a public endpoint is available, listing its service. For example: /api/v1/getService and /api/v1_1/getService. An authentication token can be requested at: /api/requestAuthToken',
+     'status': 200,
+     'versions': ['v1', 'v1_1', 'v1_2', 'v1_3', 'v2_0']
     }
 
-**Example response**
+So this tells us which API versions exist. For instance, we know that the latest API version is available at
+
+.. code-block:: html
 
+    https://company.flexmeasures.io/api/v2_0
+
+
+Also , we see that a list of services offered by (a version of) the FlexMeasures web service can be obtained by sending a ``getService`` request. An optional field "access" can be used to specify a user role for which to obtain only the relevant services.
+
+**Example request**
+
+Let's ask which endpoints are available for meter data companies (MDC):
+
+.. code-block:: html
+
+    https://company.flexmeasures.io/api/v2_0/getService?access=MDC
+
+
+**Example response**
 
 .. code-block:: json
 
@@ -46,6 +85,8 @@ A list of services offered by (a version of) the FlexMeasures web service can be
         ]
     }
 
+.. _api_auth:
+
 Authentication
 --------------
 
@@ -78,6 +119,15 @@ using the following JSON message for the POST request data:
         "password": "<user password>"
     }
 
+which gives a response like this if the credentials are correct:
+
+.. code-block:: json
+
+    {
+        "auth_token": "<authentication token>",
+        "user_id": "<ID of the user>"
+    }
+
 .. note:: Each access token has a limited lifetime, see :ref:`auth`.
 
 
@@ -339,8 +389,8 @@ These fields denote that the data should have been recorded at least 6 hours bef
 
 .. _prognoses:
 
-Prognoses
-^^^^^^^^^
+Knowledge time of Prognoses
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Some POST endpoints have two optional fields to allow setting the time at which beliefs are recorded explicitly.
 This is useful to keep an accurate history of what was known at what time, especially for prognoses.
@@ -425,8 +475,8 @@ Valid units for timeseries data in version 1 of the API are "MW" only.
 
 .. _signs:
 
-Signs
-^^^^^
+Signs of values
+^^^^^^^^^^^^^^^
 
 USEF recommends to use positive power values to indicate consumption and negative values to indicate production, i.e.
 to take the perspective of the Prosumer.

diff --git a/documentation/changelog.rst b/documentation/changelog.rst
@@ -20,9 +20,10 @@ Bugfixes
 
 Infrastructure / Support
 ----------------------
-* Make assets use MW as their default unit and enforce that in CLI, as well (API already did) [see `PR #108 <http://www.github.com/SeitaBV/flexmeasures/pull/108>`_]
+* Add tutorials on how to add and read data from FlexMeasures via its API [see `PR #130 <http://www.github.com/SeitaBV/flexmeasures/pull/130>`_]
 * For weather forecasts, switch from Dark Sky (closed from Aug 1, 2021) to OpenWeatherMap API [see `PR #113 <http://www.github.com/SeitaBV/flexmeasures/pull/113>`_]
 * Re-use the database between automated tests, if possible. This shaves 2/3rd off of the time it takes for the FlexMeasures test suite to run [see `PR #115 <http://www.github.com/SeitaBV/flexmeasures/pull/115>`_]
+* Make assets use MW as their default unit and enforce that in CLI, as well (API already did) [see `PR #108 <http://www.github.com/SeitaBV/flexmeasures/pull/108>`_]
 * Let CLI package and plugins use Marshmallow Field definitions [see `PR #125 <http://www.github.com/SeitaBV/flexmeasures/pull/125>`_]
 * add time_utils.get_recent_clock_time_window() function [see `PR #135 <http://www.github.com/SeitaBV/flexmeasures/pull/135>`_]
 

diff --git a/documentation/concepts/algorithms.rst b/documentation/concepts/algorithms.rst
@@ -9,6 +9,8 @@ Algorithms
     :depth: 2
 
 
+.. _algorithms_forecasting:
+
 Forecasting
 -----------
 
@@ -73,6 +75,8 @@ Improvements:
 - Most assets have yearly seasonality (e.g. wind, solar) and therefore forecasts would benefit from >= 2 years of history.
 
 
+.. _algorithms_scheduling:
+
 Scheduling 
 ------------
 

diff --git a/documentation/getting-started.rst b/documentation/getting-started.rst
@@ -8,6 +8,9 @@ Quickstart
 
 This section walks you through getting FlexMeasures to run with the least effort. We'll cover making a secret key, connecting a database and creating one user & one asset.
 
+.. note:: Are you not hosting FlexMeasures, but wanting to learn how to use it? Head over to our tutorials, starting with :ref:`tut_posting_data`.
+
+
 Install FlexMeasures
 ^^^^^^^^^^^^^^^^^^^^
 
@@ -134,7 +137,7 @@ Add your first asset
 
 There are three ways to add assets:
 
-Head over to ``http://localhost:5000/assets`` and add a new asset there.
+Head over to ``http://localhost:5000/assets`` (after you started FlexMeasures, see next step) and add a new asset there in a web form.
 
 Or, use the ``flexmeasures`` :ref:`cli`:
 
@@ -209,39 +212,7 @@ We provide a script for installing from source (without requiring ``sudo`` right
 
 More information (e.g. for installing on Windows) on `the Cbc website <https://projects.coin-or.org/Cbc>`_.
 
-Start collecting weather data
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-To collect weather measurements and forecasts from the DarkSky API, there is a task you could run periodically, probably once per hour. Here is an example:
-
-.. code-block::
-
-   flexmeasures add external-weather-forecasts --location 33.4366,126.5269 --store-in-db 
-
-.. note::  DarkSky is not handing out tokens any more, as they have been bought by Apple (see `issue 3 <https://github.com/SeitaBV/flexmeasures/issues/56>`_).
-
-
-Preparing the job queue database and start workers
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-To let FlexMeasures queue forecasting and scheduling jobs, install a `Redis <https://redis.io/>`_ server and configure access to it within FlexMeasures' config file (see above). You can find the necessary settings in :ref:`redis-config`.
-
-Then run one worker for each kind of job (in a separate terminal):
-
-.. code-block::
-
-   flexmeasures run-worker --queue forecasting
-   flexmeasures run-worker --queue scheduling
-
-
-You can also clear the job queues:
-
-.. code-block::
-
-   flexmeasures clear-queue --queue forecasting
-   flexmeasures clear-queue --queue scheduling
-
-
-When the main FlexMeasures process runs (e.g. by ``flexmeasures run``\ ), the queues of forecasting and scheduling jobs can be visited at ``http://localhost:5000/tasks/forecasting`` and ``http://localhost:5000/tasks/schedules``\ , respectively (by admins).
+Install and configure Redis
+^^^^^^^^^^^^^^^^^^^^^^^
 
-When forecasts and schedules have been generated, they should be visible at ``http://localhost:5000/analytics``. You can also access forecasts via the FlexMeasures API at `GET  /api/v2_0/getPrognosis <api/v2_0.html#get--api-v2_0-getPrognosis>`_\ , and schedules via `GET  /api/v2_0/getDeviceMessage <api/v2_0.html#get--api-v2_0-getDeviceMessage>`_. 
+To let FlexMeasures queue forecasting and scheduling jobs, install a `Redis <https://redis.io/>`_ server (or rent one) and configure access to it within FlexMeasures' config file (see above). You can find the necessary settings in :ref:`redis-config`.
diff --git a/documentation/index.rst b/documentation/index.rst
@@ -45,6 +45,14 @@ The platform operator of FlexMeasures can be an Aggregator.
     concepts/security_auth
 
 
+.. toctree::
+    :caption: Tutorials
+    :maxdepth: 1
+
+    tut/posting_data
+    tut/forecasting_scheduling
+    tut/building_uis
+
 .. toctree::
     :caption: The in-built UI
     :maxdepth: 1
@@ -60,7 +68,6 @@ The platform operator of FlexMeasures can be an Aggregator.
     :maxdepth: 1
 
     api/introduction
-    api/simulation
     api/v2_0
     api/v1_3
     api/v1_2

diff --git a/documentation/int/introduction.rst b/documentation/int/introduction.rst
diff --git a/documentation/scripts/init_docs.py b/documentation/scripts/init_docs.py