How is Neural Prophet being trained? #247
-
|
I'm not very familiar with PyTorch, so it hard for me to piece this together from the code: How is Neural Prophet being trained? Are all the model parameters being trained together, or are the GAM components and the FNN components being trained separately? More specifically: One of AR-Net's main contributions, based on my reading of the paper, is the highly specialized regularization being used to optimize the autoregressive order p. Is that still being used in Neural Prophet? I can't tell from the code. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
Hi @SkanderHn , thank you for your question. We can achieve this as all components are defined as model parameters, recognized by PyTorch as trainable weights. Regarding the AR-regularization: Instead of the original AR-Net regularization, we use a tuned-back, optional, regularization for AR weights in order to ensure stable training when combined with other model components. However, it should still work similarly - if you encounter any difficulties, please report back to us! |
Beta Was this translation helpful? Give feedback.
Hi @SkanderHn , thank you for your question.
All components are jointly trained by an AdamW optimizer with a OneCycle learning rate schedule.
See code reference here.
We can achieve this as all components are defined as model parameters, recognized by PyTorch as trainable weights.
Regarding the AR-regularization: Instead of the original AR-Net regularization, we use a tuned-back, optional, regularization for AR weights in order to ensure stable training when combined with other model components. However, it should still work similarly - if you encounter any difficulties, please report back to us!