Pip-installable

.gitignore

@@ -1,3 +1,6 @@
+build
+dist
+
 raw_data
 instance
 venv

Configuration.md

@@ -0,0 +1,277 @@
+# Configuration
+
+The following configurations are used by FlexMeasures.
+
+Required settings (e.g. postgres db) are marked with a double star (\*\*).
+To enable easier quickstart tutorials, these settings can be set by env vars.
+Recommended settings (e.g. mail, redis) are marked by one star (\*).
+
+Note: FlexMeasures is best configured via a config file. The config file for FlexMeasures can be placed in one of two locations: 
+
+* in the user's home directory (e.g. `~/.flexmeasures.cfg` on Unix). In this case, note the dot at the beginning of the filename!
+* in the apps's instance directory (e.g. `/path/to/your/flexmeasures/code/instance/flexmeasures.cfg`). The path to that instance directory is shown to you by running flexmeasures (e.g. `flexmeasures run`) with required settings missing or otherwise by running `flexmeasures shell`.
+
+
+## Basic functionality
+
+### LOGGING_LEVEL
+Level above which log messages are added to the log file. See the `logging` package in the Python standard library.
+
+Default: `logging.WARNING`
+
+### FLEXMEASURES_MODE
+The mode in which FlexMeasures is being run, e.g. "demo" or "play".
+This is used to turn on certain extra behaviours.
+
+Default: `""`
+
+### FLEXMEASURES_LP_SOLVER
+The command to run the scheduling solver.
+
+Default: `"cbc"`
+
+### FLEXMEASURES_HOSTS_AND_AUTH_START
+Configuration used for entity addressing. This contains the domain on which FlexMeasures runs
+and the first month when the domain was under the current owner's administration.
+
+Default: `{"flexmeasures.io": "2021-01"}`
+
+### FLEXMEASURES_DB_BACKUP_PATH
+Relative path to the folder where database backups are stored if that feature is being used.
+
+Default: `"migrations/dumps"`
+
+### FLEXMEASURES_PROFILE_REQUESTS
+Whether to turn on a feature which times requests made through FlexMeasures. Interesting for developers.
+
+Default: `False`
+
+
+## UI
+
+### FLEXMEASURES_PLATFORM_NAME
+Name being used in headings
+
+Default: `"FlexMeasures"`
+
+### FLEXMEASURES_HIDE_NAN_IN_UI
+Whether to hide the word "nan" if any value in metrics tables is `NaN`.
+
+Default: `False`
+
+### RQ_DASHBOARD_POLL_INTERVAL
+
+Interval in which viewing the queues dashboard refreshes itself, in miliseconds.
+
+Default: `3000` (3 seconds) 
+
+
+## Timing
+
+### FLEXMEASURES_TIMEZONE
+Timezone in which the platform operates. This is useful when datetimes are being localized.
+
+Default: `"Asia/Seoul"`
+
+### FLEXMEASURES_PLANNING_TTL
+Time to live for UDI event ids of successful scheduling jobs. Set a negative timedelta to persist forever.
+
+Default: `timedelta(days=7)`
+
+### FLEXMEASURES_PLANNING_HORIZON
+The horizon to use when making schedules.
+
+Default: `timedelta(hours=2 * 24)`
+
+## Tokens
+
+### DARK_SKY_API_KEY
+Token for accessing the DarkSky weather forecasting service.
+
+Note: DarkSky will be soon (Aug 1, 2021) become non-public, so thay are not giving out new tokens. We'll use another service soon, [see this issue](https://github.com/SeitaBV/flexmeasures/issues/3).
+
+This is unfortunate. In the meantime, if you can't find anybody lending their token, you can add weather forecasts to the FlexMeasures db yourself. 
+
+Default: `None`
+
+### MAPBOX_ACCESS_TOKEN
+Token for accessing the mapbox API (for displaying maps on the dashboard and asset pages). You can learn how to obtain one [here](https://docs.mapbox.com/help/glossary/access-token/)
+
+Default: `None`
+
+
+### FLEXMEASURES_TASK_CHECK_AUTH_TOKEN
+Token which external services can use to check on the status of recurring tasks within FlexMeasures.
+
+Default: `None`
+
+
+## SQLAlchemy
+
+This is only a selection of the most important settings.
+See [the Flask-SQLAlchemy Docs](https://flask-sqlalchemy.palletsprojects.com/en/master/config) for all possibilities.
+
+### SQLALCHEMY_DATABASE_URI (**)
+Connection string to the postgres database, format: `postgresql://<user>:<password>@<host-address>[:<port>]/<db>`
+
+Default: `None`
+
+### SQLALCHEMY_ENGINE_OPTIONS
+Configuration of the SQLAlchemy engine.
+
+Default: 
+```
+    {
+        "pool_recycle": 299,
+        "pool_pre_ping": True,
+        "connect_args": {"options": "-c timezone=utc"},
+    }
+```
+
+## Security
+
+This is only a selection of the most important settings.
+See [the Flask-Security Docs](https://flask-security-too.readthedocs.io/en/stable/configuration.html) as well as the [Flask-CORS docs](https://flask-cors.readthedocs.io/en/latest/configuration.html) for all possibilities.
+
+### SECRET_KEY (**)
+Used to sign user sessions and also as extra salt (a.k.a. pepper) for password salting if `SECURITY_PASSWORD_SALT` is not set.
+This is actually part of Flask - but is also used by Flask-Security to sign all tokens.
+
+It is critical this is set to a strong value. For python3 consider using: `secrets.token_urlsafe()`
+
+You can also set this in a file (which some Flask tutorials advised). Leave this setting set to `None` to get more instructions.
+
+Default: `None`
+
+### SECURITY_PASSWORD_SALT
+
+Extra password salt (a.k.a. pepper)
+
+Default: `None` (falls back to `SECRET_KEY`)
+
+
+### SECURITY_TOKEN_AUTHENTICATION_HEADER
+Name of the header which carries the auth bearer token in API requests.
+
+Default: `Authorization`
+
+### SECURITY_TOKEN_MAX_AGE
+Maximal age of security tokens in seconds.
+
+Default: `60 * 60 * 6`  (six hours)
+
+### SECURITY_TRACKABLE
+Wether to track user statistics. Turning this on requires certain user fields.
+We do not use this feature, but we do track number of logins.
+
+Default: `False`
+
+### CORS_ORIGINS
+Allowed cross-origins. Set to "*" to allow all. For development (e.g. javascript on localhost) you might use "null" in this list.
+
+Default: `[]`
+
+### CORS_RESOURCES:
+FlexMeasures resources which get cors protection. This can be a regex, a list of them or dict with all possible options.
+
+Default: `[r"/api/*"]`
+
+### CORS_SUPPORTS_CREDENTIALS
+Allows users to make authenticated requests. If true, injects the Access-Control-Allow-Credentials header in responses. This allows cookies and credentials to be submitted across domains.
+
+Note:  This option cannot be used in conjunction with a “*” origin
+
+Default: `True`
+
+
+## Mail
+For FlexMeasures to be able to send email to users (e.g. for resetting passwords), you need an email account which can do that (e.g. GMail).
+
+This is only a selection of the most important settings.
+See [the Flask-Mail Docs](https://flask-mail.readthedocs.io/en/latest/#configuring-flask-mail) for others.
+
+### MAIL_SERVER (*)
+
+Email name server domain.
+
+Default: `"localhost"`
+
+### MAIL_PORT (*)
+SMTP port of the mail server.
+
+Default: `25`
+
+### MAIL_USE_TLS
+Whether to use TLS.
+
+Default: `False`
+
+### MAIL_USE_SSL
+Whether to use SSL.
+
+Default: `False`
+
+### MAIL_USERNAME (*)
+
+Login name of the mail system user.
+
+Default: `None`
+
+### MAIL_DEFAULT_SENDER (*)
+Tuple of shown name of sender and their email address.
+
+Default:
+```
+(
+    "FlexMeasures",
+    "no-reply@example.com",
+)
+```
+
+## MAIL_PASSWORD:
+Password of mail system user.
+
+Default: `None`
+
+
+## Redis
+FlexMeasures uses the Redis database to support our forecasting and scheduling job queues.
+
+### FLEXMEASURES_REDIS_URL (*)
+URL of redis server.
+
+Default: `"localhost"`
+
+### FLEXMEASURES_REDIS_PORT (*)
+Port of redis server.
+
+Default: `6379`
+
+### FLEXMEASURES_REDIS_DB_NR (*)
+Number of the redis database to use (Redis per default has 16 databases, nubered 0-15)
+
+Default: `0`
+
+### FLEXMEASURES_REDIS_PASSWORD (*)
+Password of the redis server.
+
+Default: `None`
+
+
+## Demonstrations
+
+### FLEXMEASURES_PUBLIC_DEMO_CREDENTIALS = False
+When `FLEXMEASURES_MODE=demo`, this can hold login credentials (demo user email and password, e.g. `("demo at seita.nl", "flexdemo")`), so anyone can log in and try out the platform.
+
+Default: `None`
+
+### FLEXMEASURES_DEMO_YEAR = 2015
+When `FLEXMEASURES_MODE=demo`, this setting can be used to make the FlexMeasures platform select data from a specific year (e.g. 2015),
+so that old imported data can be demoed as if it were current
+
+Default: `None`
+
+### FLEXMEASURES_SHOW_CONTROL_UI
+The control page is still mocked, so this setting controls if it is to be shown.
+
+Default: `False`

Developing.md

@@ -21,11 +21,28 @@ Install all dependencies including the ones needed for development:
 
     make install-for-dev
 
+## Configuration
+
+Follow the confguration Quickstart advice in `Installation.md`.
+
+
+## Loading data
+
+If you have a SQL Dump file, you can load that:
+
+    psql -U {user_name} -h {host_name} -d {database_name} -f {file_path}
+
+
+
 ## Run locally
 
 Now, to start the web application, you can run:
 
-    python flexmeasures/run-local.py
+    flexmeasures run
+
+Or:
+
+    python run-local.py
 
 And access the server at http://localhost:5000
 
@@ -47,3 +64,16 @@ You can add --cov-report=html after which a htmlcov/index.html is generated.
 It's also possible to use:
 
     python setup.py test
+
+
+## Versioning
+
+We use [setuptool_scm](https://github.com/pypa/setuptools_scm/) for versioning, which bases the FlexMeasures version on the latest git tag and the commits since then.
+
+So as a developer, it's crucial to use git tags for versions only.
+
+We use semantic versioning, and we always include the patch version, not only max and min, so that setuptools_scm makes the correct guess about the next minor version. Thus, we should use `2.0.0` instead of `2.0`.
+
+See `to_pypi.sh` for more commentary on the development versions.
+
+Our API has its own version, which moves much slower. This is important to explicitly support outside apps who were coded against older versions.