You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Current implementation of BottomUpSparse reconciliation method causes an undesired conversion to a dense matrix, potentially leading to an OOM on large datasets. The lack of speedup was already observed before as noted in makoren's comment in the class definition.
@nulam Thanks for using this! Why don't you open a PR, I haven't worked on this recently but I'll have a look (I now got my hands on 200k dataset as well), and then hopefully someone from nixtla finds the time to check if they have any problem with idx_bottom or anything else.
What happened + What you expected to happen
Current implementation of
BottomUpSparse
reconciliation method causes an undesired conversion to a dense matrix, potentially leading to an OOM on large datasets. The lack of speedup was already observed before as noted in makoren's comment in the class definition.The culprit is the
idx_bottom
indexed assignment:hierarchicalforecast/hierarchicalforecast/methods.py
Line 249 in 2296c25
Locally I managed to fix it by replacing this line with the following
However, the performance is quite suboptimal, but it's sufficient for my use case, hence I would rather raise this issue than create a PR :)
Versions / Dependencies
hierarchicalforecast==0.4.1
Reproduction script
using an S matrix with n>200k leads to OOM with BottomUpSparse reconciliation
Issue Severity
Medium: It is a significant difficulty but I can work around it.
The text was updated successfully, but these errors were encountered: