Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Add support for Min-T reconciliation ("optimal reconciliation" using trace minimization and historical errors) #56

Open
kelliemac opened this issue Apr 12, 2021 · 6 comments
Labels
feature-request New feature needed

Comments

@kelliemac
Copy link

Is your feature request related to a problem? Please describe.
Not a problem exactly, but a missing piece of functionality that would make the package more useful (IMO) and more on par with the advanced hierarchical reconciliation capabilities available in R.

Describe the solution you'd like
I want to be able to do "optimal reconciliation" via trace minimization as in the R package hts (see https://www.rdocumentation.org/packages/hts/versions/6.0.1/topics/MinT, https://robjhyndman.com/papers/mint.pdf). This type of reconciliation allows the user to account for covariances of historical errors in reconciliation. Ideally, the solution would also allow for a nonnegative version (where reconciled forecasts are constrained to be nonnegative), as in https://robjhyndman.com/publications/nnmint.

Implementing this feature would entail extending the RevisionMethod class to include one more option for the reconciliation methods, as well as adding auxiliary code to support the Min-T approach in functions.py. Min-T reconciliation requires access to residuals (not just mse), so those will need to be passed to methods _revise() in HTSRegressor and revise() in RevisionMethod. (There would be a similar change for the convenience function revise_forecasts() as well.)

Describe alternatives you've considered
WLSV reconciliation is the closest to Min-T, but it doesn't account for covariances between nodes, only for variances of individual nodes in the hierarchy.

Additional context
I'm happy to work on developing this (I've already started, actually), but I thought I would post this first to generate more discussion and visibility! Please chime in if you have any thoughts related to Min-T reconciliation or extending the package in this way more generally.

@kelliemac kelliemac added the feature-request New feature needed label Apr 12, 2021
@carlomazzaferro
Copy link
Owner

We're always looking to improve the package and keep it up to date with current research, though my time is limited. I'd be extremely happy to get that into the package. If you have ongoing work on this, please feel free to open a PR, as it seems like the perfect extension (and a minimally invasive one) of the methods hereby offered.

@kelliemac
Copy link
Author

Awesome! Hopefully in the next month or so I'll be able to submit a PR. This is related to work I've done at Microsoft and I need to get company approval before publishing anything to open source, but I don't anticipate any problems getting it approved (it just might take some time). And I need to clean up my code a bit and make sure the changes are minimally invasive and include appropriate tests. Looking forward to making a contribution to this great package!

@carlomazzaferro
Copy link
Owner

Sounds excellent. Looking forward as well! Thanks for taking the time.

@wilfreddesert
Copy link
Contributor

@kelliemac , hi there! Are there any updates regarding this PR?

@kelliemac
Copy link
Author

@wilfreddesert hi, thanks for asking! Unfortunately a family tragedy happened on my end soon after my original post, and I had to take a step back from a lot of things including this. It is still on my to-do list, but it won’t be ready for probably another month.

Were you thinking of working on this feature, or hoping to use it, or interested for another reason altogether? I’m happy to try to coordinate if that makes sense.

@rafal-wasowski
Copy link

@kelliemac sorry to hear about your family situation, I realise it must have been really hard if you're taking time off everything. It feels wrong to ask you for an update in this context, but if and when you're back I for one would be very interested in the development of this feature.
I have a quite basic understanding of both Python and linear algebra so I don't think I would be able to contribute much as far as development goes but I would be more than happy to help with testing if that's of any value.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature-request New feature needed
Projects
None yet
Development

No branches or pull requests

4 participants