Skip to content

Latest commit

 

History

History
557 lines (343 loc) · 16.4 KB

CHANGELOG.md

File metadata and controls

557 lines (343 loc) · 16.4 KB
Raw

Release Notes

v1.0.0 Update to pytorch 2.0 (10/04/2023)

Breaking Changes

  • Upgraded to pytorch 2.0 and lightning 2.0. This brings a couple of changes, such as configuration of trainers. See the lightning upgrade guide. For PyTorch Forecasting, this particularly means if you are developing own models, the class method epoch_end has been renamed to on_epoch_end and replacing model.summarize() with ModelSummary(model, max_depth=-1) and Tuner(trainer) is its own class, so trainer.tuner needs replacing. (#1280)
  • Changed the predict() interface returning named tuple - see tutorials.

Changes

  • The predict method is now using the lightning predict functionality and allows writing results to disk (#1280).

Fixed

  • Fixed robust scaler when quantiles are 0.0, and 1.0, i.e. minimum and maximum (#1142)

v0.10.3 Poetry update (07/09/2022)

Fixed

  • Removed pandoc from dependencies as issue with poetry install (#1126)
  • Added metric attributes for torchmetric resulting in better multi-GPU performance (#1126)

Added

  • "robust" encoder method can be customized by setting "center", "lower" and "upper" quantiles (#1126)

v0.10.2 Multivariate networks (23/05/2022)

Added

  • DeepVar network (#923)
  • Enable quantile loss for N-HiTS (#926)
  • MQF2 loss (multivariate quantile loss) (#949)
  • Non-causal attention for TFT (#949)
  • Tweedie loss (#949)
  • ImplicitQuantileNetworkDistributionLoss (#995)

Fixed

  • Fix learning scale schedule (#912)
  • Fix TFT list/tuple issue at interpretation (#924)
  • Allowed encoder length down to zero for EncoderNormalizer if transformation is not needed (#949)
  • Fix Aggregation and CompositeMetric resets (#949)

Changed

  • Dropping Python 3.6 suppport, adding 3.10 support (#479)
  • Refactored dataloader sampling - moved samplers to pytorch_forecasting.data.samplers module (#479)
  • Changed transformation format for Encoders to dict from tuple (#949)

Contributors

  • jdb78

v0.10.1 Bugfixes (24/03/2022)

Fixed

  • Fix with creating tensors on correct devices (#908)
  • Fix with MultiLoss when calculating gradient (#908)

Contributors

  • jdb78

v0.10.0 Adding N-HiTS network (N-BEATS successor) (23/03/2022)

Added

  • Added new N-HiTS network that has consistently beaten N-BEATS (#890)
  • Allow using torchmetrics as loss metrics (#776)
  • Enable fitting EncoderNormalizer() with limited data history using max_length argument (#782)
  • More flexible MultiEmbedding() with convenience output_size and input_size properties (#829)
  • Fix concatentation of attention (#902)

Fixed

  • Fix pip install via github (#798)

Contributors

  • jdb78
  • christy
  • lukemerrick
  • Seon82

v0.9.2 Maintenance Release (30/11/2021)

Added

  • Added support for running lightning.trainer.test (#759)

Fixed

  • Fix inattention mutation to x_cont (#732).
  • Compatability with pytorch-lightning 1.5 (#758)

Contributors

  • eavae
  • danielgafni
  • jdb78

v0.9.1 Maintenance Release (26/09/2021)

Added

  • Use target name instead of target number for logging metrics (#588)
  • Optimizer can be initialized by passing string, class or function (#602)
  • Add support for multiple outputs in Baseline model (#603)
  • Added Optuna pruner as optional parameter in TemporalFusionTransformer.optimize_hyperparameters (#619)
  • Dropping support for Python 3.6 and starting support for Python 3.9 (#639)

Fixed

  • Initialization of TemporalFusionTransformer with multiple targets but loss for only one target (#550)
  • Added missing transformation of prediction for MLP (#602)
  • Fixed logging hyperparameters (#688)
  • Ensure MultiNormalizer fit state is detected (#681)
  • Fix infinite loop in TimeDistributedEmbeddingBag (#672)

Contributors

  • jdb78
  • TKlerx
  • chefPony
  • eavae
  • L0Z1K

v0.9.0 Simplified API (04/06/2021)

Breaking changes

  • Removed dropout_categoricals parameter from TimeSeriesDataSet. Use categorical_encoders=dict(<variable_name>=NaNLabelEncoder(add_nan=True)) instead (#518)

  • Rename parameter allow_missings for TimeSeriesDataSet to allow_missing_timesteps (#518)

  • Transparent handling of transformations. Forward methods should now call two new methods (#518):

    • transform_output to explicitly rescale the network outputs into the de-normalized space
    • to_network_output to create a dict-like named tuple. This allows tracing the modules with PyTorch's JIT. Only prediction is still required which is the main network output.

    Example:

    def forward(self, x):
    normalized_prediction = self.module(x)
    prediction = self.transform_output(prediction=normalized_prediction, target_scale=x["target_scale"])
    return self.to_network_output(prediction=prediction)

Fixed

  • Fix quantile prediction for tensors on GPUs for distribution losses (#491)
  • Fix hyperparameter update for RecurrentNetwork.from_dataset method (#497)

Added

  • Improved validation of input parameters of TimeSeriesDataSet (#518)

v0.8.5 Generic distribution loss(es) (27/04/2021)

Added

  • Allow lists for multiple losses and normalizers (#405)
  • Warn if normalization is with scale < 1e-7 (#429)
  • Allow usage of distribution losses in all settings (#434)

Fixed

  • Fix issue when predicting and data is on different devices (#402)
  • Fix non-iterable output (#404)
  • Fix problem with moving data to CPU for multiple targets (#434)

Contributors

  • jdb78
  • domplexity

v0.8.4 Simple models (07/03/2021)

Added

  • Adding a filter functionality to the timeseries datasset (#329)
  • Add simple models such as LSTM, GRU and a MLP on the decoder (#380)
  • Allow usage of any torch optimizer such as SGD (#380)

Fixed

  • Moving predictions to CPU to avoid running out of memory (#329)
  • Correct determination of output_size for multi-target forecasting with the TemporalFusionTransformer (#328)
  • Tqdm autonotebook fix to work outside of Jupyter (#338)
  • Fix issue with yaml serialization for TensorboardLogger (#379)

Contributors

  • jdb78
  • JakeForsey
  • vakker

v0.8.3 Bugfix release (31/01/2021)

Added

  • Make tuning trainer kwargs overwritable (#300)
  • Allow adding categories to NaNEncoder (#303)

Fixed

  • Underlying data is copied if modified. Original data is not modified inplace (#263)
  • Allow plotting of interpretation on passed figure for NBEATS (#280)
  • Fix memory leak for plotting and logging interpretation (#311)
  • Correct shape of predict() method output for multi-targets (#268)
  • Remove cloudpickle to allow GPU trained models to be loaded on CPU devices from checkpoints (#314)

Contributors

  • jdb78
  • kigawas
  • snumumrik

v0.8.2 Fix for output transformer (12/01/2021)

  • Added missing output transformation which was switched off by default (#260)

v0.8.1 Adding support for lag variables (10/01/2021)

Added

  • Add "Release Notes" section to docs (#237)
  • Enable usage of lag variables for any model (#252)

Changed

  • Require PyTorch>=1.7 (#245)

Fixed

  • Fix issue for multi-target forecasting when decoder length varies in single batch (#249)
  • Enable longer subsequences for min_prediction_idx that were previously wrongfully excluded (#250)

Contributors

  • jdb78

v0.8.0 Adding multi-target support (03/01/2021)

Added

  • Adding support for multiple targets in the TimeSeriesDataSet (#199) and amended tutorials.
  • Temporal fusion transformer and DeepAR with support for multiple tagets (#199)
  • Check for non-finite values in TimeSeriesDataSet and better validate scaler argument (#220)
  • LSTM and GRU implementations that can handle zero-length sequences (#235)
  • Helpers for implementing auto-regressive models (#236)

Changed

  • TimeSeriesDataSet's y of the dataloader is a tuple of (target(s), weight) - potentially breaking for model or metrics implementation Most implementations will not be affected as hooks in BaseModel and MultiHorizonMetric were modified. (#199)

Fixed

  • Fixed autocorrelation for pytorch 1.7 (#220)
  • Ensure reproducibility by replacing python set() with dict.fromkeys() (mostly TimeSeriesDataSet) (#221)
  • Ensures BetaDistributionLoss does not lead to infinite loss if actuals are 0 or 1 (#233)
  • Fix for GroupNormalizer if scaling by group (#223)
  • Fix for TimeSeriesDataSet when using min_prediction_idx (#226)

Contributors

  • jdb78
  • JustinNeumann
  • reumar
  • rustyconover

v0.7.1 Tutorial on how to implement a new architecture (07/12/2020)

Added

  • Tutorial on how to implement a new architecture covering basic and advanced use cases (#188)
  • Additional and improved documentation - particularly of implementation details (#188)

Changed (breaking for new model implementations)

  • Moved multiple private methods to public methods (particularly logging) (#188)
  • Moved get_mask method from BaseModel into utils module (#188)
  • Instead of using label to communicate if model is training or validating, using self.training attribute (#188)
  • Using sample((n,)) of pytorch distributions instead of deprecated sample_n(n) method (#188)

v0.7.0 New API for transforming inputs and outputs with encoders (03/12/2020)

Added

  • Beta distribution loss for probabilistic models such as DeepAR (#160)

Changed

  • BREAKING: Simplifying how to apply transforms (such as logit or log) before and after applying encoder. Some transformations are included by default but a tuple of a forward and reverse transform function can be passed for arbitrary transformations. This requires to use a transformation keyword in target normalizers instead of, e.g. log_scale (#185)

Fixed

  • Incorrect target position if len(static_reals) > 0 leading to leakage (#184)
  • Fixing predicting completely unseen series (#172)

Contributors

  • jdb78
  • JakeForsey

v0.6.1 Bugfixes and DeepAR improvements (24/11/2020)

Added

  • Using GRU cells with DeepAR (#153)

Fixed

  • GPU fix for variable sequence length (#169)
  • Fix incorrect syntax for warning when removing series (#167)
  • Fix issue when using unknown group ids in validation or test dataset (#172)
  • Run non-failing CI on PRs from forks (#166, #156)

Docs

  • Improved model selection guidance and explanations on how TimeSeriesDataSet works (#148)
  • Clarify how to use with conda (#168)

Contributors

  • jdb78
  • JakeForsey

v0.6.0 Adding DeepAR (10/11/2020)

Added

  • DeepAR by Amazon (#115)
    • First autoregressive model in PyTorch Forecasting
    • Distribution loss: normal, negative binomial and log-normal distributions
    • Currently missing: handling lag variables and tutorial (planned for 0.6.1)
  • Improved documentation on TimeSeriesDataSet and how to implement a new network (#145)

Changed

  • Internals of encoders and how they store center and scale (#115)

Fixed

  • Update to PyTorch 1.7 and PyTorch Lightning 1.0.5 which came with breaking changes for CUDA handling and with optimizers (PyTorch Forecasting Ranger version) (#143, #137, #115)

Contributors

  • jdb78
  • JakeForesey

v0.5.3 Bug fixes (31/10/2020)

Fixes

  • Fix issue where hyperparameter verbosity controlled only part of output (#118)
  • Fix occasional error when .get_parameters() from TimeSeriesDataSet failed (#117)
  • Remove redundant double pass through LSTM for temporal fusion transformer (#125)
  • Prevent installation of pytorch-lightning 1.0.4 as it breaks the code (#127)
  • Prevent modification of model defaults in-place (#112)

v0.5.2 Fixes to interpretation and more control over hyperparameter verbosity (18/10/2020)

Added

  • Hyperparameter tuning with optuna to tutorial
  • Control over verbosity of hyper parameter tuning

Fixes

  • Interpretation error when different batches had different maximum decoder lengths
  • Fix some typos (no changes to user API)

v0.5.1 PyTorch Lightning 1.0 compatibility (14/10/2020)

This release has only one purpose: Allow usage of PyTorch Lightning 1.0 - all tests have passed.

v0.5.0 PyTorch Lightning 0.10 compatibility and classification (12/10/2020)

Added

  • Additional checks for TimeSeriesDataSet inputs - now flagging if series are lost due to high min_encoder_length and ensure parameters are integers
  • Enable classification - simply change the target in the TimeSeriesDataSet to a non-float variable, use the CrossEntropy metric to optimize and output as many classes as you want to predict

Changed

  • Ensured PyTorch Lightning 0.10 compatibility
    • Using LearningRateMonitor instead of LearningRateLogger
    • Use EarlyStopping callback in trainer callbacks instead of early_stopping argument
    • Update metric system update() and compute() methods
    • Use Tuner(trainer).lr_find() instead of trainer.lr_find() in tutorials and examples
  • Update poetry to 1.1.0

v0.4.1 Various fixes models and data (01/10/2020)

Fixes

Model

  • Removed attention to current datapoint in TFT decoder to generalise better over various sequence lengths
  • Allow resuming optuna hyperparamter tuning study

Data

  • Fixed inconsistent naming and calculation of encoder_lengthin TimeSeriesDataSet when added as feature

Contributors

  • jdb78

v0.4.0 Metrics, performance, and subsequence detection (28/09/2020)

Added

Models

  • Backcast loss for N-BEATS network for better regularisation
  • logging_metrics as explicit arguments to models

Metrics

  • MASE (Mean absolute scaled error) metric for training and reporting
  • Metrics can be composed, e.g. 0.3* metric1 + 0.7 * metric2
  • Aggregation metric that is computed on mean prediction over all samples to reduce mean-bias

Data

  • Increased speed of parsing data with missing datapoints. About 2s for 1M data points. If numba is installed, 0.2s for 1M data points
  • Time-synchronize samples in batches: ensure that all samples in each batch have with same time index in decoder

Breaking changes

  • Improved subsequence detection in TimeSeriesDataSet ensures that there exists a subsequence starting and ending on each point in time.
  • Fix min_encoder_length = 0 being ignored and processed as min_encoder_length = max_encoder_length

Contributors

  • jdb78
  • dehoyosb

v0.3.1 More tests and better docs (13/09/2020)

  • More tests driving coverage to ~90%
  • Performance tweaks for temporal fusion transformer
  • Reformatting with sort
  • Improve documentation - particularly expand on hyper parameter tuning

Fixed

  • Fix PoissonLoss quantiles calculation
  • Fix N-Beats visualisations

v0.3.0 More testing and interpretation features (02/09/2020)

Added

  • Calculating partial dependency for a variable
  • Improved documentation - in particular added FAQ section and improved tutorial
  • Data for examples and tutorials can now be downloaded. Cloning the repo is not a requirement anymore
  • Added Ranger Optimizer from pytorch_ranger package and fixed its warnings (part of preparations for conda package release)
  • Use GPU for tests if available as part of preparation for GPU tests in CI

Changes

  • BREAKING: Fix typo "add_decoder_length" to "add_encoder_length" in TimeSeriesDataSet

Bugfixes

  • Fixing plotting predictions vs actuals by slicing variables

v0.2.4 Fix edge case in prediction logging (26/08/2020)

Fixed

Fix bug where predictions were not correctly logged in case of decoder_length == 1.

Added

  • Add favicon to docs page

v0.2.3 Make pip installable from master branch (23/08/2020)

Update build system requirements to be parsed correctly when installing with pip install git+https://github.com/jdb78/pytorch-forecasting

v0.2.2 Improving tests (23/08/2020)

  • Add tests for MacOS
  • Automatic releases
  • Coverage reporting

v0.2.1 Patch release (23/08/2020)

This release improves robustness of the code.

  • Fixing bug across code, in particularly

    • Ensuring that code works on GPUs
    • Adding tests for models, dataset and normalisers
    • Test using GitHub Actions (tests on GPU are still missing)

  • Extend documentation by improving docstrings and adding two tutorials.

  • Improving default arguments for TimeSeriesDataSet to avoid surprises

v0.2.0 Minor release (16/08/2020)

Added

  • Basic tests for data and model (mostly integration tests)
  • Automatic target normalization
  • Improved visualization and logging of temporal fusion transformer
  • Model bugfixes and performance improvements for temporal fusion transformer

Modified

  • Metrics are reduced to calculating loss. Target transformations are done by new target transformer