Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge project 9 #254

Merged
merged 51 commits into from Jan 19, 2022
Merged

Merge project 9 #254

merged 51 commits into from Jan 19, 2022

Commits on Nov 25, 2021

  1. Move meta data from asset/market/weather classes and types to generic…

    … asset (types) (#240)
    
    Refactoring:
    
    - Renamed generic_asset variables to old_sensor wherever the variable was expecting a type Union[Asset, Market, WeatherSensor].
    - Forecasting sub-package gets its attributes from GenericAsset or Sensor instead of from old sensor models.
    - Planning/scheduling sub-package gets its attributes from GenericAsset or Sensor instead of from old sensor models.
    - API packages gets its attributes from GenericAsset or Sensor instead of from old sensor models.
    - API package gets and sets metadata on GenericAssets #243
    
    Moving over old data:
    
    - Sensor units, event resolutions and knowledge horizons (fnc and par) copied from old sensors (these already had dedicated columns in both the old and new models) in a database migration. One-way transition.
    - Copy over all old sensor attributes that have no dedicated column in the GenericAsset model (just the display name for Market and WeatherSensor, but a lot of other attributes for Assets). These will be put in the new GenericAsset.attributes column as JSON, or in the new Sensor.attributes column as JSON.
    - Copy over all old sensor type attributes that have no dedicated column in the GenericAsset model (just seasonalities for Market and WeatherSensor, plus some is_consumer- and can_curtail-like attributes for Assets). Note that I'm moving these to the GenericAsset or Sensor, rather than to the GenericAssetType model.
    - Copy display name to both GenericAsset and Sensor.
    
    Move over crud functionality / make tests work:
    
    - Make sure to copy over units, event resolutions, knowledge horizons and other attributes (incl. weather correlations), whenever an Asset gets created, to the already simultaneously created new sensor and generic asset.
    - Do this for Markets and WeatherSensors, too.
    - API package gets and sets metadata on GenericAssets #243
    - Allow JSON fields to be updated (see here). API package gets and sets metadata on GenericAssets #243
    
    
    
    * Create draft PR for #239
    
    * Db migration that copies over attributes from old data models
    
    * - In Asset.__init__, copy over attributes to GenericAsset.
    - Start having our model_spec_factory get attributes it needs from GenericAsset.
    - Rename variables, specifically, variables that were annotated as a union of our old sensor models were named generic_asset, which was too easily confused with instances of our GenericAsset class.
    
    * model_spec_factory now gets its attributes from GenericAsset instead of old sensor model types
    
    * More renaming to avoid confusion
    
    * Have db migration copy over sensor attributes: unit, event_resolution and knowledge horizons
    
    * In Asset.__init__, copy over sensor attributes: unit, event_resolution and knowledge horizons
    
    * model_spec_factory now gets event_resolution and name from Sensor
    
    * Fix tests
    
    * Factor out use of corresponding_generic_asset attribute
    
    * Factor out use of corresponding_generic_asset attribute
    
    * More renaming
    
    * Pass time series class to model configurator explicitly
    
    * Finally, model_spec_factory doesn't need the old sensor model anymore
    
    * Allow setting the collect function name for TBSeriesSpecs to something custom
    
    * In Asset.__init__, copy over additional asset attributes to GenericAsset
    
    * Planning subpackage uses sensors instead of assets
    
    * Move some simple attributes in the UI package
    
    * Refactor to stop explicitly passing the market to the scheduler, and instead have the scheduler check for an applicable market
    
    * Revert "Move some simple attributes in the UI package", because this needs to be done jointly with moving over asset crud (which we test for)
    
    This reverts commit 56ff279.
    
    * Add notes about how each attribute is to be copied from an old class to a new class
    
    * Intend to copy display_name to both GenericAsset and Sensor
    
    * Introduce Sensor attributes and copy most old model attributes there instead of to GenericAsset attributes
    
    * Adjust attribute copying in Asset.__init__
    
    * Implement Sensor method to get an attribute
    
    * Give old sensor classes a generic_asset property
    
    * Give old sensor classes a get_attribute property
    
    * Derive Sensor class lat/lng location from GenericAsset
    
    * Get attributes from Sensor rather than from GenericAsset
    
    * Set default attributes on generic assets, too
    
    * Add clarity to method docstring
    
    Co-authored-by: Flix6x <Flix6x@users.noreply.github.com>
    Co-authored-by: F.N. Claessen <felix@seita.nl>
    3 people committed Nov 25, 2021
    Copy the full SHA
    58698ed View commit details
    Browse the repository at this point in the history

Commits on Dec 1, 2021

  1. Copy the full SHA
    9729401 View commit details
    Browse the repository at this point in the history

Commits on Dec 3, 2021

  1. API package gets and sets metadata on GenericAssets and Sensors (#243)

    Make JSON attributes mutable and have the API package get and set metadata on GenericAssets and Sensors. Applies to all version of the API, but excludes CRUD functionality on assets in API v2.0.
    
    
    * Create draft PR for #239
    
    * Db migration that copies over attributes from old data models
    
    * - In Asset.__init__, copy over attributes to GenericAsset.
    - Start having our model_spec_factory get attributes it needs from GenericAsset.
    - Rename variables, specifically, variables that were annotated as a union of our old sensor models were named generic_asset, which was too easily confused with instances of our GenericAsset class.
    
    * model_spec_factory now gets its attributes from GenericAsset instead of old sensor model types
    
    * More renaming to avoid confusion
    
    * Have db migration copy over sensor attributes: unit, event_resolution and knowledge horizons
    
    * In Asset.__init__, copy over sensor attributes: unit, event_resolution and knowledge horizons
    
    * model_spec_factory now gets event_resolution and name from Sensor
    
    * Fix tests
    
    * Factor out use of corresponding_generic_asset attribute
    
    * Factor out use of corresponding_generic_asset attribute
    
    * More renaming
    
    * Pass time series class to model configurator explicitly
    
    * Finally, model_spec_factory doesn't need the old sensor model anymore
    
    * Allow setting the collect function name for TBSeriesSpecs to something custom
    
    * In Asset.__init__, copy over additional asset attributes to GenericAsset
    
    * Planning subpackage uses sensors instead of assets
    
    * Move some simple attributes in the UI package
    
    * Refactor to stop explicitly passing the market to the scheduler, and instead have the scheduler check for an applicable market
    
    * Revert "Move some simple attributes in the UI package", because this needs to be done jointly with moving over asset crud (which we test for)
    
    This reverts commit 56ff279.
    
    * Create draft PR for #242
    
    * Allow config setting specs as module variables, too.
    
    
    Support reading config setting specs from module (#237)
    
    * Support reading config setting specs from module
    
    * Add additional documentation (review suggestion)
    
    * Amend changelog entry
    
    * Create draft PR for #242
    
    * Make JSON attributes mutable
    
    * Set Asset owner at initialization, so it is copied to GenericAsset
    
    * API v1_2 gets and sets asset attributes in the new data model
    
    * Deprecate use of Asset class in api v1_2
    
    * Deprecate use of Asset class in api v1_3
    
    * Revert "Allow config setting specs as module variables, too."
    
    This reverts commit 327b8b6.
    
    * Work around black issue by updating pre-commit-config
    
    * Deprecate use of Asset class in api v1_0
    
    * Deprecate use of Asset class in part of api v2_0
    
    * Fix docstring
    
    * Revert upgrade of black in pre-commit-config
    
    * Still battling black
    
    * Correct docstring
    
    * Deprecate use of Asset class in SensorField deserialization
    
    * Set Asset owner at initialization, so it is copied to GenericAsset
    
    * Simplify conftest
    
    * Add notes about how each attribute is to be copied from an old class to a new class
    
    * Rename variables
    
    * Refactor attribute copying to util function
    
    * In Market.__init__, copy over attributes from old models to new models
    
    * In Weather.__init__, copy over attributes from old models to new models
    
    * Deprecate use of Market class in SensorField deserialization
    
    * Deprecate use of WeatherSensor class in SensorField deserialization for fm1
    
    * Simplify fm1 entity type schema for deserialization
    
    * Deprecate use of WeatherSensor class in SensorField deserialization for fm0
    
    * Refactor query
    
    * Rename variable
    
    * Refactor query to arrive at a single combined query
    
    * Update todos
    
    * Intend to copy display_name to both GenericAsset and Sensor
    
    * Introduce Sensor attributes and copy most old model attributes there instead of to GenericAsset attributes
    
    * Adjust attribute copying in Asset.__init__
    
    * Implement Sensor method to get an attribute
    
    * Give old sensor classes a generic_asset property
    
    * Give old sensor classes a get_attribute property
    
    * Derive Sensor class lat/lng location from GenericAsset
    
    * Get attributes from Sensor rather than from GenericAsset
    
    * Resolve merge conflict
    
    * Refactor after resolving merge conflict
    
    * Adjust attribute copying in Market.__init__
    
    * Adjust attribute copying in WeatherSensor.__init__
    
    * Post-merge cherry-pick: Set default attributes on generic assets, too
    
    * Post-merge cherry-pick: Add clarity to method docstring
    
    * Introduce has_attribute and set_attribute on the Sensor class, too
    
    * Remove redundant import
    
    * Get attributes from Sensor
    
    * Simplify (as requested in PR review)
    
    * Add docstring to migration util functions, explaining their parameters
    
    * Add module docstring
    
    * Add todos
    
    * Make Sensor attributes mutable, too
    
    * Avoid assumptions on db type (specifically, postgres)
    
    * Update upgrade migration docstring
    
    * Deprecate use of Market class in api v1_1
    
    * Separate setup of market types and markets for tests, otherwise we run into flush issues
    
    * Remove redundant copy, now that we initialize super() first
    
    * Fix bugs: work on kwargs before copying from it and move up initialization of super()
    
    * Increase the chance of identifying a unique sensor by just its name, if you also know the name of its generic asset type
    
    * Simplify API tests by removing the owner id from event-type entity addresses, as the server ignores this optional field anyways
    
    * Simplify API tests by removing the owner id from event-type entity addresses, as the server ignores this optional field anyways
    
    * Deprecate use of Asset class in v1_3 tests
    
    * Deprecate use of Asset class in v1_2 tests
    
    * Deprecate use of Asset class in v1 tests
    
    * Deprecate use of Market class in v1_1 tests
    
    * Fix merge errors
    
    * Update string status code
    
    * Rename legacy migration module
    
    Co-authored-by: Flix6x <Flix6x@users.noreply.github.com>
    Co-authored-by: F.N. Claessen <felix@seita.nl>
    Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
    4 people committed Dec 3, 2021
    Copy the full SHA
    0792962 View commit details
    Browse the repository at this point in the history
  2. Issue 245 planners should check for required and optional sensor attr…

    …ibutes more explicitly (#256)
    
    Code can now check for required Sensor attributes of an expected type, by calling `sensor.check_required_attributes`.
    
    
    * Allow passing a default attribute
    
    * Set default for optional attributes
    
    * Check for required Sensor attributes
    
    * Fix tests
    
    * Allow checking for attribute type, too
    
    * Allow checking for one of multiple allowed types
    
    * Check required attribute types
    
    * Move util function to become Sensor attribute
    
    * Move util function to separate module to resolve circular import
    Flix6x committed Dec 3, 2021
    Copy the full SHA
    9cb39b9 View commit details
    Browse the repository at this point in the history
  3. Issues 252 foreign keys to sensor table on old sensor data tables (#255)

    Connect old sensor data tables to the Sensor table, with a database migration that replaces Power.asset_id with Power.sensor_id, Price.market_id with Price.sensor_id, and Weather.(weather_)sensor_id with Weather.sensor_id. It also simplifies the table joins in verify_power_in_db and verify_prices_in_db, moves over API conftests that create Power, Price and Weather instances, and refactors the use of services.forecasting to get rid of some legacy util functions.
    
    
    * Add new foreign key for Power model and start db migration
    
    * Simplify verify_power_in_db and verify_prices_in_db
    
    * Add new foreign key for Price model
    
    * Rename Weather attributes
    
    * Add new foreign key for Weather model
    
    * Bidirectional id syncing
    
    * Switch over Power, Price and Weather initialization
    
    * Reverse direction of id syncing
    
    * Unlink Power, Price and Weather from Asset, Market and WeatherSensor
    
    * Fix tests after unlinking Power, Price and Weather from Asset, market and WeatherSensor
    
    * Replace another mention of Market with Sensor in api v2_0
    
    * Deprecate use of Market class in v2_0 implementation
    
    * Get Sensor instead of Market, for planning
    
    * Check for Sensor instead of Market, for market_id validation in AssetSchema
    
    * Analytics view gets price data via Sensor name rather than via Market name
    
    * Update db migration and revise to rename columns without touching rows
    
    * Collect time series as bdf with new sensor rather than with old sensor
    
    * Refactor to pass forecasting jobs the value type explicitly instead of a string, so we can get rid of two util functions and a couple more references to the old sensor models
    
    * Replace references to Market in test_solver.py
    
    * Fix type annotation
    
    * Fix CLI test input
    
    * Refactor query to util function
    
    * Adjust type annotation
    
    * Make sensor name optional, too, in query util
    
    * Rename variable to plural
    
    * Simplify sensor queries in CLI test, and attempt to store TimedBelief instead
    
    * Rename test CLI options
    
    * Fix test CLI option
    
    * Add todo
    
    * Temporarily revert to creating data with the old sensor data model
    
    * Hopefully clarify in-line comment
    
    * Remove unused auto generated Alembic commands
    Flix6x committed Dec 3, 2021
    Copy the full SHA
    2c62988 View commit details
    Browse the repository at this point in the history
  4. Clean up sensor schema (#258)

    Clean up the deserialization of entity addresses into Sensors and the serialization vice versa.
    
    
    * Refactor to remove noqa
    
    * Stop supporting serialization of old sensor models into entity addresses
    Flix6x committed Dec 3, 2021
    Copy the full SHA
    33baa90 View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2021

  1. Issue 259 synchronize how we collect data from data models (#260)

    This PR synchronizes the function signatures of our old and new data collection methods, thereby supporting us in moving over data to our new data model. It introduces a new TimedBelief.collect method whose function signature is compatible with calls to TimedValue.collect. To ensure equivalency, some query filter utils have been refactored to become filter criteria (i.e. the SQLAlchemy term for that which you pass to a query filter), and timely-beliefs 1.8.0 supports passing additional custom criteria for its search query filters.
    
    
    * Revise collect function of old sensor data models: break up query_window into event_starts_after and event_ends_before
    
    * Revise collect function of old sensor data models: break up belief_time_window into beliefs_after and beliefs_before
    
    * Revise collect function of old sensor data models: break up belief_horizon_window into horizons_at_least and horizons_at_most
    
    * Copy options to filter by source type
    
    * Fix type annotations
    
    * Refactor query utils: switch over, from applying filters to a query, to building up filter criteria and then applying those criteria at once
    
    * Treat empty lists differently from None values
    
    * Use exactly the same source filter criteria in TimedBelief.search
    
    * Refactor
    
    * Refactor more
    
    * Start TimedBelief.collect
    
    * Allow passing sensor ids
    
    * Allow passing single sensor (id)
    
    * Get sensor instance from bdf (because the sensor variable might be an int)
    
    * Upgrade tb dependency
    
    * Also allow sensor names to be passed to the new collect function
    
    * TimedBelief.collect is a class method
    
    * Don't resample if resolution is None
    
    * Add the remaining optional search filters to the collect method
    
    * Add tests for the two new ways of querying beliefs
    
    * Speed up resampling in case of requiring most recent beliefs only
    
    * By default, return most recent beliefs only
    
    * Remove reference to resolved pandas issue
    
    * Add docstring to new collect method
    
    * Explain difference between passing a sensor id or a sensor name
    
    * Change type annotation for filter criteria
    
    * notin_ deprecated in favour of not_in
    
    * pip-compile
    Flix6x committed Dec 6, 2021
    Copy the full SHA
    db2ea57 View commit details
    Browse the repository at this point in the history
  2. Merge search and collect (#262)

    Merging the new TimedBelief.collect method with the existing TimedBelief.search method. This also entails renaming all occurences of TimedValue.collect to become TimedValue.search, in order to safeguard the goal of PR #260.
    
    
    * Rename old collect method to search
    
    * Merge new collect method into TimedBelief.search
    
    * Deprecate the sensors argument in favor of sensor
    
    * Small annotation revert in scope of project 9
    
    * Fix deprecation of required argument
    Flix6x committed Dec 6, 2021
    Copy the full SHA
    b42ef1a View commit details
    Browse the repository at this point in the history

Commits on Dec 7, 2021

  1. Issue 261 add changelog documentation for project 9 (#264)

    * Changelog entry for project 9
    
    * Add upgrade warning
    
    * Add asset CRUD warning
    Flix6x committed Dec 7, 2021
    Copy the full SHA
    cb58920 View commit details
    Browse the repository at this point in the history
  2. Copy the full SHA
    03b675a View commit details
    Browse the repository at this point in the history

Commits on Dec 9, 2021

  1. Fall back scheduler heuristics (#267)

    Introduce a fallback policy for charging schedules of batteries and Charge Points, in cases where the solver is presented with an infeasible problem.
    
    
    * Implement and test fallback policy for infeasible scheduler results
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Reform old Asset properties into Sensor properties
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Simplify
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Dec 9, 2021
    Copy the full SHA
    84c959f View commit details
    Browse the repository at this point in the history
  2. Issue 265 copy over weather correlations from the asset model to the …

    …generic asset model (#266)
    
    Revise a previous db migration (6cf5b241b85f), but still under the scope of project 9. GenericAssets that correspond to an old Asset should now have a new attribute "correlations", listing names of generic asset types.
    
    
    * Migrate weather correlations to GenericAsset attributes
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Clean up todos
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Dec 9, 2021
    Copy the full SHA
    98172df View commit details
    Browse the repository at this point in the history

Commits on Dec 10, 2021

  1. Ensure backwards compatibility of Power init and Market init (#269)

    In case old plugins are still initializing Power and Price objects, ensure backwards compatibility and warn for deprecation.
    
    NB Weather objects were already backwards compatible, because we already used sensor_id to mean weather_sensor_id for the old model.
    
    
    * Ensure backwards compatibility of Power init and Market init
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Use tb utils to also warn about deprecation
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Pop old argument from kwargs
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Dec 10, 2021
    Copy the full SHA
    38c3726 View commit details
    Browse the repository at this point in the history
  2. Switch order of charging fallback policy to prioritize approaching SO…

    …C targets above staying within SOC constraints (#270)
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Dec 10, 2021
    Copy the full SHA
    50e1c2d View commit details
    Browse the repository at this point in the history
  3. Fix PR #269 and test the fix (#271)

    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Dec 10, 2021
    Copy the full SHA
    ae0a61e View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2021

  1. Issue 272 move over functionality to find closest sensor (#274)

    Also refactor and expand documentation.
    
    
    * Copy great circle distance methods from WeatherSensor to GenericAsset
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Test finding the closest sensor
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Return Sensor instead of WeatherSensor
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Look for Sensor instead of WeatherSensor
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Refactor old query
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Return list of Sensors instead of list of WeatherSensors
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Refactor: rename function and argument
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Move tested function
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix type annotation
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Refactor: move query function to queries subpackage
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add docstring
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add farther temperature sensors to test
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Move geo math to geo_utils
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add note regarding dependency on Postgres extensions
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Typo
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Dec 20, 2021
    Copy the full SHA
    41cfef2 View commit details
    Browse the repository at this point in the history
  2. Copy Asset and WeatherSensor locations to GenericAsset (#276)

    Copy Asset and WeatherSensor locations to GenericAsset (Markets didn't have a location). Sensor attributes get a copy, too, because multiple Sensors under one GenericAsset may have slightly different locations, while the GenericAsset has only one location.
    
    
    * Copy Asset and WeatherSensor locations to GenericAsset (Sensor attributes get a copy, too, because multiple Sensors under one GenericAsset may have slightly different locations, while the GenericAsset has only one location)
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Rename
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Dec 20, 2021
    Copy the full SHA
    f9dca07 View commit details
    Browse the repository at this point in the history
  3. Overwrite Sensor.name with the name of the old sensor. (#278)

    * Copy Asset and WeatherSensor locations to GenericAsset (Sensor attributes get a copy, too, because multiple Sensors under one GenericAsset may have slightly different locations, while the GenericAsset has only one location)
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Overwrite Sensor.name with the name of the old sensor.
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Dec 20, 2021
    Copy the full SHA
    e0e440d View commit details
    Browse the repository at this point in the history

Commits on Dec 21, 2021

  1. Let a Sensor define a custom location in its attributes column. If no…

    …t specified, the Sensor location defaults to that of the GenericAsset it belongs to.
    
    Allow distinct sensor locations per generic asset (#281)
    
    * Add missing way to get an attribute from GenericAsset
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Derive sensor location using get_attributes
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Dec 21, 2021
    Copy the full SHA
    f126f88 View commit details
    Browse the repository at this point in the history
  2. Stop planned (dis)charging after target (or constraint) is reached. (#…

    …280)
    
    * Stop planned (dis)charging after target is reached.
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Stop planned (dis)charging after constraint is met.
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Refactor
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Be more explicit about initial state
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Dec 21, 2021
    Copy the full SHA
    0d26f7c View commit details
    Browse the repository at this point in the history
  3. UI: Dashboard using GenericAssets (#251)

    * Create draft PR for #249
    
    * working state of new dashboard; created new file asset_grouping with class AssetGroup, which is aimed to replace resources.Resource; created new view /sensor/<id>/state
    
    * smaller items from review and add a default ACL for generic assets
    
    * move querying assets by type to a data.queries module
    
    * make asset type grouping configurable & simplify initialising an AssetGroup
    
    * query asset for its state (via one of its power sensors) and also identify power sensors by unit
    
    * Dashboard only shows asset groups where at least one asset has a location and power sensors
    
    * add missing module
    
    * restore grouping by location behaviour for later
    
    * move intro text to modal dialogue
    
    * make is_power_unit only care about power
    
    * include energy assets on dashboard
    
    * fix docstring
    
    Co-authored-by: nhoening <nhoening@users.noreply.github.com>
    Co-authored-by: Nicolas Höning <nicolas@seita.nl>
    3 people committed Dec 21, 2021
    Copy the full SHA
    754b46b View commit details
    Browse the repository at this point in the history

Commits on Dec 23, 2021

  1. Sub issue 284a initialize timed belief when initializing power/price/…

    …weather (#285)
    
    Ensure that new data is saved in both the old (Power/Price/Weather) and the new model (TimedBelief). Resolve interference between some tests using day-ahead price data by adjusting the data source of the price data for tests using the new model.
    
    
    * Initialize TimedBelief when initializing Weather
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add deprecation notice for Weather class
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Initialize TimedBelief when initializing Price, and add deprecation notice for Price class
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix test data periodicity
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Resolve test interference by using a different data source in conftest
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Initialize TimedBelief when initializing Power, and add deprecation notice for Power class
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Initialize TimedBelief when initializing forecasts
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Update incorrect test: a shorter scheduling horizon may lead to a different result
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Dec 23, 2021
    Copy the full SHA
    8f78b20 View commit details
    Browse the repository at this point in the history

Commits on Dec 27, 2021

  1. Issue 282 use pint to check and convert units (#283)

    Prepare for future user functionality to auto-convert data to desired units when POSTing and GETting data, and hook up unit conversion to our dev endpoint for posting sensor data.
    
    
    * Use pint for unit_utils, using h to denote hour, adding the world's currencies, and simplifying units according to preference; also add tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add new dependencies to requirements
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix docstring: cubic unit with unicode
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix docstring examples
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Convert units and data in SensorDataSchema
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix case of simple multiplier
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Test more unit conversions
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Refactor unit utils
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix converting offset units (such as degrees Celsius)
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Extra testing of unit util functions, incl. conversion of offset units
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add missing docstring
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Black
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add inline note
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Clarify what test util function does
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add missing type annotation
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Dec 27, 2021
    Copy the full SHA
    dc9ecfa View commit details
    Browse the repository at this point in the history

Commits on Dec 29, 2021

  1. Sub issue 284b query timed belief rather than power/price/weather (#287)

    Move time series queries to the new data model, and also have forecasting and scheduling jobs directly create TimedBeliefs.
    
    
    * Query TimedBelief rather than Power in api v1.3 tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in api v1.3 implementations
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in user services tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in query tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in forecasting tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in scheduling tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in api v1 tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Simplify data deletion, like, by a lot
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Count ex-ante TimedBeliefs after populating time series forecasts
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Price in api v1_1 tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power/Price/Weather in Resource.load_sensor_data
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power/Price/Weather in api v2.0 tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Refactor: simplify duplicate query construction
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add custom join target to get rid of SA warning
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Filter criteria should work for both TimedBeliefs and TimedValues
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Clarify docstring
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in api v1 implementations
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Schedules should contain one deterministic belief per event
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix type annotation
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * flake8
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Price/Weather for analytics
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query deterministic TimedBelief rather than Price for planning queries
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Forecast TimedBelief rather than Power/Price/Weather
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Schedule TimedBelief rather than Power
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Apparently, to initialize a TimedBelief is to save a TimedBelief, too
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Create TimedBelief rather than Power/Price/Weather in data generation script
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Bump timely-beliefs dependency
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix latest state query
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Revert "Apparently, to initialize a TimedBelief is to save a TimedBelief, too"
    
    This reverts commit fb58ec7.
    
    * Prevent saving TimedBelief to session upon updating Sensor or Source
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Expand inline note
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Refactor: simplify block in data_gen.py
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add back test and fix it by selecting only the most recent beliefs when resampling, and by solving a bug in a deprecated optional argument. Also, remove redundant index conversions in test, and move one conversion closer to where it is needed
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Dec 29, 2021
    Copy the full SHA
    f651eea View commit details
    Browse the repository at this point in the history
  2. Sub issue 284c migrate power/price/weather data to timed belief (#288)

    Copy over all time series data, in batches.
    
    
    * Query TimedBelief rather than Power in api v1.3 tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in api v1.3 implementations
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in user services tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in query tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in forecasting tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in scheduling tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in api v1 tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Simplify data deletion, like, by a lot
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Count ex-ante TimedBeliefs after populating time series forecasts
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Price in api v1_1 tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power/Price/Weather in Resource.load_sensor_data
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power/Price/Weather in api v2.0 tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Refactor: simplify duplicate query construction
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add custom join target to get rid of SA warning
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Filter criteria should work for both TimedBeliefs and TimedValues
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Clarify docstring
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in api v1 implementations
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Schedules should contain one deterministic belief per event
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix type annotation
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * flake8
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Price/Weather for analytics
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query deterministic TimedBelief rather than Price for planning queries
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Forecast TimedBelief rather than Power/Price/Weather
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Schedule TimedBelief rather than Power
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Apparently, to initialize a TimedBelief is to save a TimedBelief, too
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Create TimedBelief rather than Power/Price/Weather in data generation script
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Bump timely-beliefs dependency
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Start database migration file and copy time series data
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Remove raise statement used for debugging
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add changelog entry
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Dec 29, 2021
    Copy the full SHA
    4fb564b View commit details
    Browse the repository at this point in the history

Commits on Jan 3, 2022

  1. Unit tooling improvements (#293)

    * check for validity of unit strings to avoid crashing; use variable name which is usable with pdb debugging; remove deprecation warning about .getitem methid in UnitRegistry
    
    * fix tests
    nhoening committed Jan 3, 2022
    Copy the full SHA
    2414292 View commit details
    Browse the repository at this point in the history
  2. Sub issue 284d stop saving to power/price/weather tables (#289)

    Switch all occurrences of initializing a Power, Price or Weather object within FlexMeasures to initializing a TimedBelief instead. Also refactor the save_to_db method, moving it from the api package to the data package. Because some FlexMeasures plugins are known to depend on direct initialization of Power, Price and Weather objects, and some plugins are known to use the save_to_db method from the api package, the old functionality is left intact, with any new data automatically copied to the new model, with the appropriate deprecation warnings.
    
    
    * Query TimedBelief rather than Power in api v1.3 tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in api v1.3 implementations
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in user services tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in query tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in forecasting tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in scheduling tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in api v1 tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Simplify data deletion, like, by a lot
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Count ex-ante TimedBeliefs after populating time series forecasts
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Price in api v1_1 tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power/Price/Weather in Resource.load_sensor_data
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power/Price/Weather in api v2.0 tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Refactor: simplify duplicate query construction
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add custom join target to get rid of SA warning
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Filter criteria should work for both TimedBeliefs and TimedValues
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Clarify docstring
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in api v1 implementations
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Schedules should contain one deterministic belief per event
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix type annotation
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * flake8
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Price/Weather for analytics
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query deterministic TimedBelief rather than Price for planning queries
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Forecast TimedBelief rather than Power/Price/Weather
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Schedule TimedBelief rather than Power
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Apparently, to initialize a TimedBelief is to save a TimedBelief, too
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Create TimedBelief rather than Power/Price/Weather in data generation script
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Bump timely-beliefs dependency
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix latest state query
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Revert "Apparently, to initialize a TimedBelief is to save a TimedBelief, too"
    
    This reverts commit fb58ec7.
    
    * Prevent saving TimedBelief to session upon updating Sensor or Source
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Create only TimedBeliefs in conftests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Use session.add_all calls instead of session.bulk_save_objects or individual session.add calls
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * API directly creates TimedBeliefs
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * CLI uses TimedBeliefs only
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Data scripts use TimedBeliefs only
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * One more conftest switched to creating TimedBeliefs instead of Weather objects
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Expand docstring note on forbidden replacements
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Clarify docstring note on saving changed beliefs only
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Remove redundant flush
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Catch forbidden belief replacements with more specific exception
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Rename variable
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * One transaction per request
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Only enqueue forecasting jobs upon successfully saving new data
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Flush instead of commit
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Expand test for forbidden data replacement
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Simplify play mode excemption for replacing beliefs
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add note about potential session rollback
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Typo
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Move UniqueViolation catching logic to error handler
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * flake8
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Clean up
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Refactor: move error handler to api_utils.py
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * flake8
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Jan 3, 2022
    Copy the full SHA
    5ba1dc7 View commit details
    Browse the repository at this point in the history
  3. Merge branch 'project-9' into Issue-284_Move_sensor_data_from_Power/P…

    …rice/Weather_to_TimedBelief
    
    # Conflicts:
    #	flexmeasures/api/dev/tests/test_sensor_data.py
    Flix6x committed Jan 3, 2022
    Copy the full SHA
    dd54ebc View commit details
    Browse the repository at this point in the history
  4. Merge pull request #286 from FlexMeasures/Issue-284_Move_sensor_data_…

    …from_Power/Price/Weather_to_TimedBelief
    
    Issue 284 Move sensor data from Power/Price/Weather to TimedBelief
    Flix6x committed Jan 3, 2022
    Copy the full SHA
    d788d9d View commit details
    Browse the repository at this point in the history
  5. Issue 273 add roundtrip efficiency to scheduler (#291)

    Rewrite our generic device scheduler to:
    - Deal with asymmetric efficiency losses of individual devices.
    - Deal with asymmetric up and down prices for deviating from previous commitments.
    Also allow round-trip efficiency to be communicated as a new optional field when POSTing UDI Events, with efficiency losses being assigned equally to charging and discharging.
    
    
    * Query TimedBelief rather than Power in api v1.3 tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in api v1.3 implementations
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in user services tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in query tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in forecasting tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in scheduling tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in api v1 tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Simplify data deletion, like, by a lot
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Count ex-ante TimedBeliefs after populating time series forecasts
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Price in api v1_1 tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power/Price/Weather in Resource.load_sensor_data
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power/Price/Weather in api v2.0 tests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Refactor: simplify duplicate query construction
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add custom join target to get rid of SA warning
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Filter criteria should work for both TimedBeliefs and TimedValues
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Clarify docstring
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Power in api v1 implementations
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Schedules should contain one deterministic belief per event
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix type annotation
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * flake8
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query TimedBelief rather than Price/Weather for analytics
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Query deterministic TimedBelief rather than Price for planning queries
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Forecast TimedBelief rather than Power/Price/Weather
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Schedule TimedBelief rather than Power
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Apparently, to initialize a TimedBelief is to save a TimedBelief, too
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Create TimedBelief rather than Power/Price/Weather in data generation script
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Bump timely-beliefs dependency
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix latest state query
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Revert "Apparently, to initialize a TimedBelief is to save a TimedBelief, too"
    
    This reverts commit fb58ec7.
    
    * Prevent saving TimedBelief to session upon updating Sensor or Source
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Create only TimedBeliefs in conftests
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Use session.add_all calls instead of session.bulk_save_objects or individual session.add calls
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * API directly creates TimedBeliefs
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * CLI uses TimedBeliefs only
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Data scripts use TimedBeliefs only
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * One more conftest switched to creating TimedBeliefs instead of Weather objects
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Expand docstring note on forbidden replacements
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Clarify docstring note on saving changed beliefs only
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Remove redundant flush
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Catch forbidden belief replacements with more specific exception
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Rename variable
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * One transaction per request
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Only enqueue forecasting jobs upon successfully saving new data
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Flush instead of commit
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Expand test for forbidden data replacement
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Simplify play mode excemption for replacing beliefs
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add note about potential session rollback
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Typo
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Move UniqueViolation catching logic to error handler
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * flake8
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Rewrite solver to deal with asymmetry in up and down commitment prices
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add optional roundtrip_efficiency field to UDI events, and use it to scale prices
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add test cases for various round-trip efficiencies
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add changelog entries
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add documentation for the new API field
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Grammar corrections
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix return value for empty EMS
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Allow efficiencies per device for multi-device EMS, by stopping the application of round-trip efficiency as price scalars and modeling device flows in more detail
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Relax tests using some tolerance
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix mistake
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add test docstring
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Check round-trip efficiency for acceptable range
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Expand docstring
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Jan 3, 2022
    Copy the full SHA
    e0af47a View commit details
    Browse the repository at this point in the history

Commits on Jan 5, 2022

  1. Issue 247 generic asset crud (#290)

    * add a new method to specify an auth context (by callable) and discuss steps forward for use cases not covered yet by auth policy
    
    * add ACL for GenericAsset
    
    * added /dev/generic_assets API, CRUD uses it (passing manual testing)
    
    * fix asset icon in asset view
    
    * remove verbs from /api/dev/generic_assets API routes (HTTP method already defines what happens)
    
    * make CRUD UI tests (mocking the asset API) work
    
    * auth policy: change 'create' to 'create-children' permission (removing the need for issue#200); fix modelling problem if several distinct principal formulations exist for one permission
    
    * separate fresh tests from others; only generate copies of generic assets when we don't already have them (distinguish by name); re-add old asset tests;
    
    * add missing mock in another ui test
    
    * add unique contraints: on GenericAssetType.name & on GenericAssets for name+account_id
    
    * improve explanation of our authorization policy implementation
    
    * update changelog
    
    * improve changelog, also mention temporary API location
    
    * comments next to test cases
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * create_user function hashes the password
    -e
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * more informative validation errror
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * allow creation of public Assets in UI
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * Fix display of owning account
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * better changelog docs
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * list sensors on asset page
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * do not fail getting latest state if capacity_in_mw is not in attributes
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * better default for capacity_in_mw
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * use id in the dict, for sensors lookup
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * Suggested fixes to pr 290 (#299)
    
    Remove the entity addresses from the generic asset listing, and list the number of sensors instead.
    
    
    * grammar
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Update generic asset listing, replacing entity addresses with number of sensors
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Avoid crashing on formatting None values
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix wrong variable name
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix incorrect type annotation
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Partly revert cefe320
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * small improvement on locating ea addresses
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * Describe current state of transition
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * deprecation notice in old-style asset API documentation
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * Fix type annotation
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Update documentation
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Update v2.0 connection entity addresses in documentation
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Correct quickrefs in v2.0 documentation
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Update other v2.0 sensor entity addresses in documentation
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix entity address dates corresponding to older previous domain name
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Adapt more occurrences of older dates in entity addresses, and switch to fm1 scheme.
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>
    Co-authored-by: F.N. Claessen <felix@seita.nl>
    3 people committed Jan 5, 2022
    Copy the full SHA
    1bb2819 View commit details
    Browse the repository at this point in the history
  2. Small addendum to 290 (#301)

    * fix typo
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * stop using click.echo in util code
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    nhoening committed Jan 5, 2022
    Copy the full SHA
    4f0956a View commit details
    Browse the repository at this point in the history

Commits on Jan 6, 2022

  1. Merge branch 'main' into project-9

    # Conflicts:
    #	documentation/api/change_log.rst
    Flix6x committed Jan 6, 2022
    Copy the full SHA
    cfda19c View commit details
    Browse the repository at this point in the history
  2. Merge database migrations

    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Jan 6, 2022
    Copy the full SHA
    dcbdaf3 View commit details
    Browse the repository at this point in the history
  3. Bump timely-beliefs dependency to get rid of false deprecation warnings

    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Jan 6, 2022
    Copy the full SHA
    57d31e3 View commit details
    Browse the repository at this point in the history
  4. Bump timely-beliefs dependency given previously yanked release

    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Jan 6, 2022
    Copy the full SHA
    cc0ffa1 View commit details
    Browse the repository at this point in the history
  5. Skip autoscheduling if none of the posted values represent a state ch…

    …ange (#303)
    
    Stop autoscheduling when API calls save nothing new to the database, thereby saving redundant computation.
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Jan 6, 2022
    Copy the full SHA
    868f483 View commit details
    Browse the repository at this point in the history

Commits on Jan 7, 2022

  1. fix open weather map import: adapt weather sensor location access to …

    …new model (#304)
    
    * adapt weather sensor location access to new model
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * small review improvement
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    nhoening committed Jan 7, 2022
    Copy the full SHA
    7a5cee3 View commit details
    Browse the repository at this point in the history
  2. black some docstrings

    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    nhoening committed Jan 7, 2022
    Copy the full SHA
    b5b65be View commit details
    Browse the repository at this point in the history
  3. Stop saving unchanged weather forecasts (#305)

    * stop saving unchanged weather forecasts
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * Catch trivial aggregation case
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * print save_to_db status; rename variable for more clarity of current data model assumptions; refactor complex function
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    Co-authored-by: F.N. Claessen <felix@seita.nl>
    nhoening and Flix6x committed Jan 7, 2022
    Copy the full SHA
    569dbd5 View commit details
    Browse the repository at this point in the history

Commits on Jan 11, 2022

  1. Commit session after calling new save_to_db function (#308)

    * commit when using modern save_to_db and improve docstring
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * mention Seita's Github profile
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    nhoening committed Jan 11, 2022
    Copy the full SHA
    82387cc View commit details
    Browse the repository at this point in the history

Commits on Jan 12, 2022

  1. Sensor charts load most recent beliefs by default (#307)

    This represents a significant speed-up for loading charts of sensors with many beliefs, such as charts for power actuators whose planned schedules are updated many times before actuation.
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Jan 12, 2022
    Copy the full SHA
    f7c6ab0 View commit details
    Browse the repository at this point in the history
  2. Mobile friendly (responsive) charts of sensor data, and such charts c…

    …an be requested with a custom width and height.
    
    Responsive sensor chart (#313)
    
    * Append to transforms possibly already coming out of the chart specs
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Let chart specs derive title, quantity and unit from Sensor object
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Derive bar width from sensor's event resolution
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix attribute call
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Responsive sensor charts: scales width to container, which is better for mobile. Also exposes width and height as overridable chart properties for getting charts by API.
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Changelog entry
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Jan 12, 2022
    Copy the full SHA
    932034f View commit details
    Browse the repository at this point in the history

Commits on Jan 13, 2022

  1. Fix resolution of bar charts (#310)

    Better bar widths, based on sensor resolution.
    
    
    * Append to transforms possibly already coming out of the chart specs
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Let chart specs derive title, quantity and unit from Sensor object
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Derive bar width from sensor's event resolution
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Fix attribute call
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Changelog entry
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Jan 13, 2022
    Copy the full SHA
    14ffcde View commit details
    Browse the repository at this point in the history

Commits on Jan 14, 2022

  1. add legacy comments (and reasoning) to a list of modules and functions (

    #309)
    
    * add legacy comments (and reasoning) to a list of modules and functions
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * add legacy notes to models and schemas
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * add some legacy comments to CLI add/delete
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    nhoening committed Jan 14, 2022
    Copy the full SHA
    874b060 View commit details
    Browse the repository at this point in the history
  2. Access to public assets (#316)

    * better name for the EVERYONE constant - we only mean all logged-in users.
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * allow every logged-in user to read public assets and sensors
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * better docstring in __acl__
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * fix __acl__ docstrings
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    
    * even better __acl__ docstring
    
    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    nhoening committed Jan 14, 2022
    Copy the full SHA
    6cad7e8 View commit details
    Browse the repository at this point in the history

Commits on Jan 17, 2022

  1. Issue 311 sensible defaults for searching for beliefs (#312)

    The default for search methods on beliefs data is now to search most recent beliefs only.
    
    
    * Sensor.search_beliefs loads most recent beliefs by default
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * TimedBelief.search loads most recent beliefs by default
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add docstring comments
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Changelog entry
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add todo for open timely-beliefs issue
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Jan 17, 2022
    Copy the full SHA
    f310793 View commit details
    Browse the repository at this point in the history

Commits on Jan 18, 2022

  1. move /sensorData into /api/dev prefix (#317)

    Signed-off-by: Nicolas Höning <nicolas@seita.nl>
    nhoening committed Jan 18, 2022
    Copy the full SHA
    2054f10 View commit details
    Browse the repository at this point in the history
  2. Issue 257 attribution requirements (#292)

    * update attribution for Map (adding Improve this map) and add MapBox logo
    
    * add changelog entry
    
    * use https
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    Co-authored-by: F.N. Claessen <felix@seita.nl>
    nhoening and Flix6x committed Jan 18, 2022
    Copy the full SHA
    a30b7aa View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2022

  1. Wrap up project 9 (#320)

    Update main changelog and documentation notes on data model transition.
    
    
    * Add missing database upgrade warning for v0.7.0
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add changelog entry (infrastructure) for unit support
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add a changelog entry (infrastructure) for improving the scheduler for asymmetric device efficiencies asymmetric EMS commitment prices (such as a different feed-in tariff)
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add changelog entry (new features) for fallback policy for Charge Points
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Notes on data model transition: change order and add check marks
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Notes on data model transition: add new projects and update project descriptions
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add changelog entry for Altair sensor charts, which have now become part of the UI for the first time (linked to from the assets page)
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add changelog entry for skipping auto-forecasting in case posted data does not represent a state change
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Add changelog entry for fixing multi-sourced belief charts
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Jan 19, 2022
    Copy the full SHA
    279a3be View commit details
    Browse the repository at this point in the history
  2. Deprecate portfolio and analytics views (#321)

    Unlink that which wasn't made compatible at this point with the new data model, and is therefore essentially broken.
    
    
    * Remove portfolio and analytics from default menu views
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Unlink portfolio and analytics from UI documentation
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Unlink control page from UI documentation
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Comment out links to analytics page on dashboard
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Comment out links to analytics page on asset pages
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    
    * Changelog entry
    
    Signed-off-by: F.N. Claessen <felix@seita.nl>
    Flix6x committed Jan 19, 2022
    Copy the full SHA
    93e481a View commit details
    Browse the repository at this point in the history