When performing model selection with ForecastingGridSearchCV in sktime, why do you need to specify a forecaster to instantiate the gridsearch?

[fkiraly]

Originally from stackoverflow:
https://stackoverflow.com/questions/75903578/when-performing-model-selection-with-forecastinggridsearchcv-in-sktime-why-do-y
and requested by Dan0815 to answer.

This is the original question:

---

In the Pydata 2022 Global sktime tutorial on AutoML there is an example of using sktime.forecasting.model_selection.ForecastingGridSearchCV to select a forecaster:

from sktime.forecasting.theta import ThetaForecaster
from sktime.forecasting.trend import STLForecaster
from sktime.forecasting.naive import NaiveForecaster

fcster = TransformedTargetForecaster(
    steps=[
        ("detrender", Detrender()),
        ("deseasonalizer", Deseasonalizer()),
        ("scaler", TabularToSeriesAdaptor(RobustScaler())),
        ("minmax2", TabularToSeriesAdaptor(MinMaxScaler((1, 10)))),
        ("forecaster", NaiveForecaster()),
    ]
)

gscv = ForecastingGridSearchCV(
    forecaster=fcster,
    param_grid=[
        {
            "scaler__transformer__with_scaling": [True, False],
            "forecaster": [NaiveForecaster()],
            "forecaster__strategy": ["drift", "last", "mean"],
            "forecaster__sp": [4, 6, 12],
        },
        {
            "scaler__transformer__with_scaling": [True, False],
            "forecaster": [STLForecaster(), ThetaForecaster()],
            "forecaster__sp": [4, 6, 12],
        },
    ],
    cv=cv,
    n_jobs=-1,
)
gscv.fit(y)
gscv.best_params_

My question is why do we have ("forecaster", NaiveForecaster()) in the steps of fcster

If you change NaiveForecaster() to None, you get an AttributeError: 'NoneType' object has no attribute 'clone' so it's clearly required.

However, in gcsv, forecaster is one of the hyperparameters to tune. So presumably anything specified for forecaster in fcster prior to tuning will be swapped out during tuning?

[fkiraly]

That's an interesting question.

I think the reason is simply, the ForecastingGridSearchCV internally clones the template and then uses set_params, rather than constructing with the full params:

the relevant lines are 160-164:

# Clone forecaster.
forecaster = self.forecaster.clone()

# Set parameters.
forecaster.set_params(**params)

The clone tries to clone the None, which it expects to be a forecatser, but it doesn't have a clone method.

We could, alternatively, first use get_params, update with the paramset, and then construct from scratch which would not call the nested clone, although I'm not sure whether that's better.

I'm interested to understand why one would put None there - just for brevity?