How to concatenate features extracted from y with other exogenous features?

[shabnamnazmids]

Hi,
I am trying to make a forecast based on features extracted from y, as well as other exogenous features as X, and I haven't been able to build a pipeline that does this. Here is the shop version of what I'd like to do:

regressor = CNNRegressor()
forecaster = make_reduction(regressor, window_length=12, strategy="recursive")

pipeline = (Id() + (Lag([1], index_out="original")['col_name'] * Imputer("nearest")) + 
YtoX() * (Lag([1], index_out="original") * Imputer("nearest")))) ** forecaster

pipeline.fit(y=y, X=X_train)
y_pred = pipeline.predict(fh=[1, 2, 3], X=X_test)

Applying the transformer part of the pipeline, I can see the columns correctly transformed. However, the .predict() operator gives as error as follows:

"TypeError: _transform output of <class 'sktime.transformations.compose._ytox.YtoX'> does not comply with sktime mtype specifications."

I'd appreciate any hints on how to build such a pipeline.

[fkiraly]

hm, on first glance, it seems like this should work.

Have you tried to remove parts until it works? Would be good to have the "minimal failing example", to investigate.

[shabnamnazmids]

Thanks @fkiraly for the suggestion. I tested the pipeline with y-only features with

pipeline = (YtoX() * (Lag([1], index_out="original") * Imputer("nearest")))) ** forecaster
pipeline.fit(y=y)
y_pred = pipeline.predict(fh=[1, 2, 3])

and with X-only features with

pipeline = (Id()) ** forecaster

pipeline.fit(y=y, X=X_train)
y_pred = pipeline.predict(fh=[1, 2, 3], X=X_test)

Both of these pipelines worked with no errors. But once I add the two to get a FeatureUnion as follows, the predict function doesn't work anymore and throws the above error.

pipeline = (Id() + 
                  YtoX() * (Lag([1], index_out="original") * Imputer("nearest"))) ** forecaster

pipeline.fit(y=y, X=X_train)
y_pred = pipeline.predict(fh=[1, 2, 3], X=X_test)

P.S. I dropped the (Lag([1], index_out="original")['col_name'] * Imputer("nearest") from the example to simplify.

[fkiraly]

ok, seems like a bug - let's investigate this in a bug report.

What would help if you also produce the code you refer to as "once I add the two into the FeatureUnion", and/or open a bug report in the issue tracker.

[shabnamnazmids]

Thanks, @fkiraly for the suggestion. Submitted a bug report here.

[sbuse]

I tested the pipeline part with the y-only features and encountered an unexpected behavior. It seems like pipelines that are constructed with a make_reduction don't work as expected. Defining something like the following does not seem to respect what is defined in the first pair of brackets. I get exactly the same final model regardless if it says (YtoX()*Lag([1,2,3])*Differencer([1])) or some other combination of lags and differences e.g. Differencer([5,6,7]).

y = load_airline()
y_train, y_test = temporal_train_test_split(y)
fh = ForecastingHorizon(y_test.index, is_relative=False)

forecaster = make_reduction(Ridge(random_state=42), window_length=12, strategy="recursive")
pipeline = (YtoX()*Lag([1,2,3])*Differencer([1])) ** forecaster
pipeline.fit(y=y_train)

y_pred = pipeline.predict(fh=fh)

Checking pipeline.get_fitted_params() shows that the model does not get extended by the lagged or differenced features and always hast just n=len(window_length) parameters. Maybe i'm doing something fundamentally wrong but this seems rather unexpected to me.

[fkiraly]

this is likely an instance of this bug: #5127