Can make_reduction Be Used with an Expanding Window Strategy?

[fspinna]

Hello sktime Community!

I'm exploring the make_reduction functionality and have a question regarding its application with an expanding window approach, as opposed to a sliding window.

Suppose I have a univariate time series, [1, 2, 3, 4, 5, 6], and I aim to create a forecaster that employs expanding windows to predict the subsequent value. The sequence of expanding windows and predictions would be:

• [1] to forecast 2.
• [1, 2] to forecast 3.
• [1, 2, 3] to forecast 4.
• [1, 2, 3, 4] to forecast 5.
• [1, 2, 3, 4, 5] to forecast 6.

However, this format is not directly compatible with a scikit-learn regressor, as the input dimensions vary and cannot be represented as a standard square matrix. I would like to apply a transformer that can convert this series into a suitable tabular format. For instance, a simple transformer might compute the average of each expanding window, which would then serve as the input for the sklearn model to make predictions. To illustrate:

• The average of [1] is 1, which would be used to predict 2.
• The average of [1, 2] is 1.5, which would be used to predict 3.
• The average of [1, 2, 3] is 2, which would be used to predict 4, and so on.

(Of course, the idea is to use more complex transformers.)

My Question:
Is the expanding window approach currently feasible with make_reduction? I appreciate any suggestions or pointers you may provide!

[fkiraly]

Hm, not with make_reduction, as that hard-codes the window strategy.

You might be able to build what you want, though, by using:

• the YtoX compositor to make your y data exogenous
• SplitterSummarizer with ExpandingWindowSplitter, applied to that exogenous data
• the YfromX reducer to predict y from exogenous data
• in a forecasting pipeline (via ** dunder, or directly calling ForecastingPipeline)

I would be curious to hear whether that approach works - if yes code would be appreciated, so we can add this as an integration test case. If not, code would be appreciated in a bug report...

[fkiraly]

maybe you also have to add a Lag somewhere.

sth like (YtoX() * SplitterSummarizer(stuff) * Lag(1)) ** YfromX(sklearn_reg)

Perhaps this is getting too complicated to be pleasant...

[fkiraly]

should there perhaps be a way to customize the index that SplitterSummarizer returns?

[fspinna]

Thank you a lot for the answer! I'll need to take some time to give it a look (I'm not super expert on complex pipelines).

[fkiraly]

I would advise, first try the steps individually and then plug them together

[fspinna]

Ok, I made a first attempt:

1. I initialized a DataFrame y with a single series of numbers from 1 to 10.

y = pd.DataFrame([[pd.Series(np.arange(1, 11))]])

2. I constructed the pipeline that should compute the means:

pipe = (
    YtoX()
    * SplitterSummarizer(
        splitter=ExpandingWindowSplitter(initial_window=1, fh=1),
        transformer=SummaryTransformer(summary_function=["mean"], quantiles=None),
    )
    * Lag(1)
)

3. After applying fit_transform to the pipeline, I obtain the means which I then concatenate with the original y for inspection. This is basically the reduced dataset right?

means = pipe.fit_transform(y, y).iloc[0,0]
print(pd.concat([means, y.iloc[0,0]], axis=1))

lag_1__mean   0
0          NaN   1
1          1.0   2
2          1.5   3
3          2.0   4
4          2.5   5
5          3.0   6
6          3.5   7
7          4.0   8
8          4.5   9
9          5.0  10

4. I extended the pipeline by adding an imputation step to handle NaN values, followed by a Ridge regressor.

pipe = (
    YtoX()
    * SplitterSummarizer(
        splitter=ExpandingWindowSplitter(initial_window=1, fh=1),
        transformer=SummaryTransformer(summary_function=["mean"], quantiles=None),
    )
    * Lag(1)
    * Imputer(method="nearest")
) ** YfromX(Ridge())

5. The pipeline fits without issues:

pipe.fit(y, y)

However, when I attempt to predict the future values using pipe.predict(fh=1), I encounter the following error:

ValueError: The `window_length` and the forecasting horizon are incompatible with the length of `y`. Found `window_length`=1, `max(fh)`=1, but len(y)=1. It is required that the window length plus maximum forecast horizon is smaller than the length of the time series `y` itself.

What am I missing?

Full code below:

from sktime.transformations.compose import YtoX
from sktime.forecasting.compose import YfromX
from sktime.transformations.series.summarize import (
    SplitterSummarizer,
    SummaryTransformer,
)
from sktime.forecasting.model_selection import ExpandingWindowSplitter
from sktime.transformations.series.lag import Lag
from sklearn.linear_model import Ridge
import numpy as np
import pandas as pd
from sktime.transformations.series.impute import Imputer

y = pd.DataFrame([[pd.Series(np.arange(1, 11))]])
pipe = (
    YtoX()
    * SplitterSummarizer(
        splitter=ExpandingWindowSplitter(initial_window=1, fh=1),
        transformer=SummaryTransformer(summary_function=["mean"], quantiles=None),
    )
    * Lag(1)
)
means = pipe.fit_transform(y, y).iloc[0, 0]
print(pd.concat([means, y.iloc[0, 0]], axis=1))
pipe = (
    YtoX()
    * SplitterSummarizer(
        splitter=ExpandingWindowSplitter(initial_window=1, fh=1),
        transformer=SummaryTransformer(summary_function=["mean"], quantiles=None),
    )
    * Lag(1)
    * Imputer(method="nearest")
) ** YfromX(Ridge())
pipe.fit(y, y)
pipe.predict(fh=1)