Huge model growth with tree-based feature selection

[jondo]

When introducing a tree-based feature selection, i.e. switching from

make_pipeline(
    MinMaxScaler(),
    MLPRegressor(solver='lbfgs', random_state=1))

to

make_pipeline(
    MinMaxScaler(),
    SelectFromModel(ExtraTreesRegressor(n_estimators=50)),
    MLPRegressor(solver='lbfgs', random_state=1))

, pickle.dumped files of fitted models become much larger: by a factor of about 200!

Does this growth come from an embedded fitted ExtraTreesRegressor? But why is it embedded?
I thought that the ExtraTreesRegressor could be cleared after fit, leaving behind just the list of selected features for saving?

(I am currently using scikit-learn version 1.2.2.)

[adrinjalali]

selectfrommodel.estimator_ keeps the fitted estimator which can be used for inspection as well. If you don't want to store it, you can delete that attribute before dumping your model.

[jondo]

Thank you for the hint. But deleting the estimator_ also removes its feature_importances_. These (in combination with SelectFromModel's threshold) represent the selected features, which should be saved.

[jondo]

I will try replacing the SelectFromModel in the fitted pipeline with a ColumnTransformer that contains the selected features.

[jondo]

Replacing with a ColumnTransformer worked!
This is how I kept the selected features:

selected_features = pipeline[1].get_feature_names_out(X_fit.columns)
column_transformer = ColumnTransformer(transformers=[('keep', 'passthrough', selected_features)])
column_transformer.fit(X_fit, y_fit)
pipeline.steps[1] =  ('dimension_reduction', column_transformer)

For this, the first step needed to be configured as MinMaxScaler().set_output(transform="pandas").