Skip to content

Latest commit

 

History

History
461 lines (343 loc) · 25 KB

RELEASE_NOTES.rst

File metadata and controls

461 lines (343 loc) · 25 KB
Raw

Next release

All changes

  • Update tutorial Westeros multinode to include code-based hints for in-depth questions (798).
  • .make_df can now create partly-filled DataFrames <pandas.DataFrame> for indexed sets; not only parameters (784).
  • New function .util.copy_model that exposes the behaviour of the message-ix copy-model CLI command to other Python code (784).
  • New test fixture .tmp_model_dir (784).
  • Bug fix: .Scenario.rename would not rename keys where the index set and index name differed (601, 791).
  • Increase minimum requirement for genno dependency to 1.20 (783).

v3.8.0 (2024-01-12)

Migration notes

Update code that imports from the following modules:

  • message_ix.reporting → use message_ix.report.
  • message_ix.reporting.computations → use message_ix.report.operator.

Code that imports from the old locations will continue to work, but will raise DeprecationWarning.

All changes

  • message_ix is tested and compatible with Python 3.12 (767). Support for Python 3.7, which reached end-of-life on 2023-06-27, is dropped (738). message_ix now requires Python version 3.8 or greater.
  • Rename message_ix.report (761).
  • Add the LPdiag tool <tools/lp_diag> to diagnose and analyze numerical issues in linear programming (LP) problems stored in MPS-format files (704).
  • GDX files generated by .GAMSModel (thus .MESSAGE, .MACRO, and .MESSAGE_MACRO) will contain an ixmp_version set with information on the versions of ixmp and message_ix that generated the file (747, 767).
  • New reporting operator .model_periods and automatic keys y::model and y0 (738).
  • Improve readability of LaTeX equations in docs (721).
  • Replace MESSAGE_ITEMS and MACRO_ITEMS with .MESSAGE.items and .MACRO.items, respectively (761).
  • Bugfix: .Scenario.add_macro would not correctly handle configuration that mapped a MESSAGE (commodity, level) to MACRO sector when the commodity and sector names were different (719).
  • Expand macro documentation, particularly code documentation (315, 719).
  • Bugfix: .operator.as_message_df would error if a particular dimension was supplied via the common argument but not present in qty (719).

v3.7.0 (2023-05-16)

Migration notes

  • The default lpmethod has been changed from "Dual Simplex" (lpmethod=2) to "Barrier" (lpmethod=4). NOTE: this may result in changes to the solution. In order to use the previous default lpmethod, the user-specific default setting can be set through the user's ixmp configuration file. Alternatively, the lpmethod can be specified directly as an argument when solving a scenario. Both of these configuration methods are further explained here <message_ix.models.GAMSModel>.

  • The dimensionality of one set and two parameters (map_tec_storage, storage_initial, and storage_self_discharge) are extended to allow repesentation of the mode of operation of storage technologies and the temporal level of storage containers. If these items are already populated with data in a Scenario, this data will be incompatible with the MESSAGE GAMS implementation in this release; a UserWarning will be emitted when the .Scenario is instantiated, and ~.message_ix.Scenario.solve will raise a ValueError. (If these items are empty, their dimensions will be updated automatically. New Scenarios are unaffected.)

    Users must update data for these items, specifically:

    Existing parameter or set Dimension(s) to add
    map_tec_storage mode, storage_mode, lvl_temporal
    storage_initial mode
    storage_self_discharge mode

    For the set map_tec_storage, values for the new dimensions represent, respectively, the mode of operation for charge/discharge technologies, and the storage_mode and lvl_temporal for the corresponding storage device. For the two parameters, .expand_dims is provided to help:

    from message_ix import Scenario
from message_ix.util import expand_dims

scen, platform = Scenario.from_url("…")

# Re-use the existing data in `scen`, adding the `mode` dimension
expand_dims(scen, "storage_initial", mode="an existing mode")

All changes

  • Add a tutorial for Westeros multi-node and different trade possibilities (683).
  • Add additional oscillation detection mechanism for macro iterations (645, 676)
  • Adjust default lpmethod from "Dual Simplex" (2) to "Barrier" (4); do NOT remove cplex.opt file(s) after solving workflow completes (657).
  • Adjust .Scenario.add_macro calculations for pandas 1.5.0 (656).
  • Ensure levelized_cost are also calculated for technologies with only variable costs (653).
  • Correct calculation of COST_NODAL_NET for standalone MESSAGE (648)
  • Account for difference in period-length in equations NEW_CAPACITY_CONSTRAINT_LO and NEW_CAPACITY_CONSTRAINT_UP (654)
  • Extend functionality of storage solutions to include "mode" and temporal level (633)
  • Introduce a citation file CITATION.cff with citation information (695).
  • Correct GAMS for the assignment of "capacity_factor" at "year" (705).

v3.6.0 (2022-08-17)

Migration notes

  • The in_horizon argument to .vintage_and_active_years is deprecated, and will be removed in message_ix 4.0 or later. At the same time, the behaviour will change to be the equivalent of providing in_horizon = False, i.e. the method will no longer filter to the scenario time horizon by default. To prepare for this change, user code that expects values confined to the time horizon can be altered to use .pandas.DataFrame.query:

    df = scen.vintage_and_active_years().query(f"{scen.y0} <= year_vtg")

  • The default reports <default-reports> (tables in IAMC format) available in a .Reporter have changed keys to e.g. message::default with two colons. Code using e.g. message:default (one colon) should be updated to use the current keys.

    This matches fixed behaviour upstream in genno version 1.12 to avoid unintended confusion with keys like A:i: i (after the first colon) is the name for the sole dimension of a 1-dimensional quantity, whereas default in message::default is a tag.

All changes

  • Adjust keys for IAMC-format reporting nodes (628, 641)
  • New reporting computation .as_message_df (628).
  • Extend functionality of .vintage_and_active_years; add aliases .yv_ya, .ya, and .y0 (572, 623).
  • Add scripts and HOWTO for documentation videos (396).

v3.5.0 (2022-05-06)

Migration notes

The format of input data files for MACRO calibration has been changed in 327. Files compatible with v3.4.0 and earlier will not work with this version and should be updated; see details of the current data file format in the documentation <macro>.

561 corrected the model internal logic for handling zero values in the capacity_factor <params-tech> parameter. Before this change, the GAMS code inserted a capacity_factor value of 1.0 where such zero values appeared; now, zeros are preserved, so the technologies may be created (CAP) but none of their capacity will be usable at the (n, t, yV, y, h) where zero values appear. This is consistent with the general concept of a “capacity factor”: for instance, a solar photovoltaic technology for electricity generation may have a non-zero capacity with a capacity factor of 0 at h = night. This may cause changes in model output for scenarios where such zero values appear; see 591 for discussion, including methods to check for and adjust/remove such values.

All changes

  • Extend documentation on historical capacity and activity values (496)
  • Extend documentation on decision variables "CAP_NEW" and "CAP" (595)
  • Extend documentation to guide users through the Westeros tutorials (594).
  • Add new logo and diagram to the documentation (597).
  • Correct typo in GAMS formulation, equation_renewables_equivalence (581).
  • Handle zero values in capacity_factor in models with sub-annual time resolution; expand tests (515, 561).
  • Extend explanations, update .make_df signature in tutorials (524).
  • Improve configurability of .macro; see the documentation <macro> (327).
  • Split .Reporter.add_tasks for use without an underlying :class:.Scenario (567).
  • Allow setting the “model_dir” and “solve_options” options for .GAMSModel (and subclasses .MESSAGE, .MACRO, and .MESSAGE_MACRO) through the user's ixmp configuration file; expand documentation (557).

v3.4.0 (2022-01-27)

All changes

  • Expand the documentation with an outlook of the MESSAGEix usage (520).
  • Adjust test suite for pyam v1.1.0 compatibility (499).
  • Add Westeros tutorial <tutorials> on historical parameters (478).
  • Update reference for activity and capacity soft constraints (474).
  • Update .years_active to use sorted results (491).
  • Adjust the Westeros reporting tutorial to pyam 1.0 deprecations (492).
  • Change precision of GAMS check for parameter "duration_time" (513).
  • Update light and historic demand in Westeros baseline tutorial (523).
  • Enhance mathematical formulation to represent sub-annual time slices consistently (509).

v3.3.0 (2021-05-28)

Migration notes

rmessageix (and rixmp) are deprecated and removed, as newer versions of the R reticulate package allow direct import and use of the Python modules with full functionality. See the updated page for rmessageix, and the updated instructions on how to install-r.

All changes

  • Update the Westeros tutorial <tutorials> on flexible generation (369).
  • Add a Westeros tutorial <tutorials> on modeling renewable resource supply curves (370).
  • Update the Westeros tutorial <tutorials> on firm capacity (368).
  • Remove rmessageix (473).
  • Expand documentation of commodity storage <gams-storage> sets, parameters, and equations (473).
  • Add two new Westeros tutorial <tutorials> on creating scenarios from Excel files (450).
  • Fix bug in .years_active to use the lifetime corresponding to the vintage year for which the active years are being retrieved (456).
  • Add a PowerPoint document usable to generate the RES diagrams for the Westeros tutorials (408).
  • Expand documentation install for installing GAMS under macOS (460).
  • Add new Westeros tutorial <tutorials> on add-on technologies (365).
  • Expand documentation of dynamic constraint parameters <section_parameter_dynamic_constraints> (454).
  • Adjust message_ix.report to use genno / ixmp.report changes in ixmp PR #397 (441).

v3.2.0 (2021-01-24)

Migration notes

  • Code that uses .make_df can be adjusted in one of two ways. See the function documentation for details. The function should be imported from the top level:

    from message_ix import make_df

All changes

  • 407: Use .report in tutorials; add .util.tutorial for shorthand code used to streamline tutorials.
  • 407: Make .Reporter a top-level class.
  • 415: Improve .make_df to generate empty, partially-, or fully-filled data frames with the correct columns for any MESSAGE or MACRO parameter.
  • 415: Make complete lists of .MESSAGE_ITEMS <.MESSAGE.items>, .MACRO_ITEMS <.MACRO.items> and their dimensions accessible through the Python API.
  • 421: Fix discounting from forward-looking to backward-looking and provide an explanation of the period structure and discounting in documentation of time.

v3.1.0 (2020-08-28)

message_ix v3.1.0 coincides with ixmp v3.1.0.

For citing message_ix, distinct digital object identifiers (DOIs) are available for every release from v3.1.0 onwards; see the user guidelines and notice <notice-cite> for more information and how to cite.

All changes

  • 367: Add new westeros tutorial on share constraints.
  • 366: Add new Westeros tutorial on modeling fossil resource supply curves.
  • 391, 392: Add a documentation page on pre-requisite knowledge & skills <prereqs>; expand guidelines on contributing.
  • 389: Fix a bug in .pyam.concat using non-pyam objects.
  • 286, 381, 389: Improve .add_horizon to also set duration_period; add documentation of time.
  • 377: Improve the rmessageix <rmessageix> R package, tutorials, and expand documentation and installation instructions.
  • 382: Update discount factor from df_year to df_period in documentation of the objective function to match the GAMS formulation.

v3.0.0 (2020-06-07)

message_ix v3.0.0 coincides with ixmp v3.0.0.

Migration notes

The generic storage formulation <gams-storage> introduces new ixmp items (sets, parameters, variables, and equations) to the MESSAGE model scheme. When loading a Scenario created with a version of message_ix older than 3.0.0, .MESSAGE.initialize will initialized these items (and leave them empty), using at most one call to ~message_ix.Scenario.commit.

See also the migration notes for ixmp 3.0.0.

All changes

  • 190: Add generic mathematical formulation of technologies that store commodities <gams-storage>, such as water and energy.
  • 343, 345: Accept .MESSAGE_MACRO iteration control parameters through .solve; document how to tune these to avoid numerical issues.
  • 340: Allow cplex.opt to be used by message_ix from multiple processes.
  • 328: Expand automatic reporting of emissions prices and mapping sets; improve robustness of .Reporter.convert_pyam <genno.compat.pyam.operator.add_as_pyam>.
  • 321: Move .Scenario.to_excel, .read_excel to ixmp.Scenario; they continue to work with message_ix.Scenario.
  • 323: Add units, replace_vars arguments to .Reporter.convert_pyam.
  • 308: Expand automatic reporting of add-on technologies.
  • 313: Include all tests in the message_ix package.
  • 307: Adjust to deprecations in ixmp 2.0.
  • 223: Add methods for parametrization and calibration of MACRO based on an existing MESSAGE Scenario.

v2.0.0 (2020-01-14)

message_ix v2.0.0 coincides with ixmp v2.0.0.

Migration notes

Support for Python 2.7 is dropped as it has reached end-of-life, meaning no further releases will be made even to fix bugs. See PEP-0373 and https://python3statement.org. message_ix users must upgrade to Python 3.

Command-line interface (CLI). Use message-ix as the program for all command-line operations:

  • message-ix copy-model replaces messageix-config.
  • message-ix dl replaces messageix-dl.
  • message-ix also provides all the features of the ixmp CLI.

Configuration. ixmp adds a streamlined system for storing information about different platforms, backends, and databases that store Scenario data. See the ixmp release notes <ixmp:whatsnew> for migration notes.

All changes

  • 285: Drop support for Python 2.
  • 284: Add a suggested sequence/structure to how to run the Westeros tutorials.
  • 281: Test and improve logic of .years_active and .vintage_and_active_years.
  • 269: Enforce year-indexed columns as integers.
  • 256: Update to use ixmp.config and improve CLI.
  • 255: Add message_ix.testing.nightly and message-ix nightly CLI command group for slow-running tests.
  • 249, 259: Build MESSAGE and MESSAGE_MACRO classes on ixmp model API; adjust Scenario.
  • 235: Add a reporting tutorial.
  • 236, 242, 263: Enhance reporting.
  • 232: Add Westeros tutorial for modelling seasonality, update existing tutorials.
  • 276: Improve add_year for bounds and code cleanup

v1.2.0 (2019-06-25)

MESSAGEix 1.2.0 adds an option to set the commodity balance to strict equality, rather than a supply > demand inequality. It also improves the support for models with non-equidistant years.

Other improvements include an experimental reporting module, support for CPLEX solver options via ~.Scenario.solve, and a reusable message_ix.testing module.

Release 1.2.0 coincides with ixmp release 0.2.0, which provides full support for ~.Scenario.clone across platforms (database instances), e.g. from a remote database to a local HSQL database; as well as other improvements. See the ixmp release notes for further details.

All changes

  • 161: A feature for adding new periods to a scenario.
  • 205: Implement required changes related to timeseries-support and cloning across platforms (see ixmp#142).
  • 196: Improve testing by re-using ixmp apparatus.
  • 187: Test for cumulative bound on emissions.
  • 182: Fix cross-platform cloning.
  • 178: Bugfix of the PRICE_EMISSION variable in models with non-equidistant period durations.
  • 176: Add message_ix.report module.
  • 173: The meth:~.Scenario.solve command now takes additional arguments when solving with CPLEX. The cplex.opt file is now generated on the fly during the solve command and removed after successfully solving.
  • 172: Add option to set COMMODITY_BALANCE to equality.
  • 154: Enable documentation build on ReadTheDocs.
  • 138: Update documentation and tutorials.
  • 131: Update clone function argument scen to scenario with planned deprecation of the former.

v1.1.0 (2018-11-21)

Migration notes

This patch introduces a few backwards-incompatible changes to database management.

Database Migration

If you see an error message like:

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
usr/local/lib/python2.7/site-packages/ixmp/core.py:81: in __init__
    self._jobj = java.ixmp.Platform("Python", dbprops)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <jpype._jclass.at.ac.iiasa.ixmp.Platform object at 0x7ff1a8e98410>
args = ('Python', '/tmp/kH07wz/test.properties')

    def _javaInit(self, *args):
        object.__init__(self)

        if len(args) == 1 and isinstance(args[0], tuple) \
           and args[0][0] is _SPECIAL_CONSTRUCTOR_KEY:
            self.__javaobject__ = args[0][1]
        else:
            self.__javaobject__ = self.__class__.__javaclass__.newClassInstance(
>               *args)
E           org.flywaydb.core.api.FlywayExceptionPyRaisable: org.flywaydb.core.api.FlywayException: Validate failed: Migration checksum mismatch for migration 1
E           -> Applied to database : 1588531206
E           -> Resolved locally    : 822227094

Then you need to update your local database. There are two methods to do so:

  1. Delete it (you will lose all data and need to regenerate it). The default location is ~/.local/ixmp/localdb/.
  2. Manually apply the underlying migrations. This is not particularly easy, but allows you to save all your data. If you want help, feel free to get in contact on the listserv.

New Property File Layout

If you see an error message like:

usr/local/lib/python2.7/site-packages/jpype/_jclass.py:111: at.ac.iiasa.ixmp.exceptions.IxExceptionPyRaisable
---------------------------- Captured stdout setup -----------------------------
2018-11-13 08:15:17,410 ERROR at.ac.iiasa.ixmp.database.DbConfig:357 - missing property 'config.server.config' in /tmp/hhvE1o/test.properties
2018-11-13 08:15:17,412 ERROR at.ac.iiasa.ixmp.database.DbConfig:357 - missing property 'config.server.password' in /tmp/hhvE1o/test.properties
2018-11-13 08:15:17,412 ERROR at.ac.iiasa.ixmp.database.DbConfig:357 - missing property 'config.server.username' in /tmp/hhvE1o/test.properties
2018-11-13 08:15:17,413 ERROR at.ac.iiasa.ixmp.database.DbConfig:357 - missing property 'config.server.url' in /tmp/hhvE1o/test.properties
------------------------------ Captured log setup ------------------------------
core.py                     80 INFO     launching ixmp.Platform using config file at '/tmp/hhvE1o/test.properties'
_________________ ERROR at setup of test_add_spatial_multiple __________________

    @pytest.fixture(scope="session")
    def test_mp():
        test_props = create_local_testdb()

        # start jvm
        ixmp.start_jvm()

        # launch Platform and connect to testdb (reconnect if closed)
>       mp = ixmp.Platform(test_props)

Then you need to update your property configuration file. The old file looks like:

config.name = message_ix_test_db@local
jdbc.driver.1 = org.hsqldb.jdbcDriver
jdbc.url.1 = jdbc:hsqldb:file:/path/to/database
jdbc.user.1 = ixmp
jdbc.pwd.1 = ixmp
jdbc.driver.2 = org.hsqldb.jdbcDriver
jdbc.url.2 = jdbc:hsqldb:file:/path/to/database
jdbc.user.2 = ixmp
jdbc.pwd.2 = ixmp

The new file should look like:

config.name = message_ix_test_db@local
jdbc.driver = org.hsqldb.jdbcDriver
jdbc.url = jdbc:hsqldb:file:/path/to/database
jdbc.user = ixmp
jdbc.pwd = ixmp

All changes

  • 202: Added the "Development rule of thumb" section from the wiki and the Tutorial style guide to the Contributor guidelines. Tweaked some formatting to improve readibility.
  • 113: Upgrading to MESSAGEix 1.1: improved representation of renewables, share constraints, etc.
  • 109: MACRO module added for initializing models to be solved with MACRO. Added scenario-based CI on circleci.
  • 99: Fixing an error in the compuation of the auxiliary GAMS reporting variable PRICE_EMISSION.
  • 89: Fully implementing system reliability and flexibity considerations (cf. Sullivan).
  • 88: Reformulated capacity maintainance constraint to ensure that newly installed capacity cannot be decommissioned within the same model period as it is built in.
  • 84: message_ix.Scenario.vintage_active_years() now limits active years to those after the first model year or the years of a certain technology vintage.
  • 82: Introducing "add-on technologies" for mitigation options, etc.
  • 81: Share constraints by mode added.
  • 80: Share constraints by commodity/level added.
  • 78: Bugfix: message_ix.Scenario.solve() uses 'MESSAGE' by default, but can be provided other model names.
  • 77: rename() function can optionally keep old values in the model (i.e., copy vs. copy-with-replace).
  • 74: Activity upper and lower bounds can now be applied to all modes of a technology.
  • 67: Use of advanced basis in cplex.opt turned off by default to avoid conflicts with barrier method.
  • 65: Bugfix for downloading tutorials. Now downloads current installed version by default.
  • 60: Add basic ability to write and read model input to/from Excel.
  • 59: Added MacOSX CI support.