Implement BayesSearchCV in scikit-learn

[earlev4]

Hi! First off, thanks so much to the excellent work done by all the scikit-learn contributors! The project is truly a gift and your work is greatly appreciated!

I still consider myself a novice when it comes to scikit-learn. In my usage, I typically will attempt to use GridSearchCV when searching for the best parameters. However, depending on the search space, GridSearchCV might not always be the best option and can be computationally expensive and time-consuming in some scenarios. RandomizedSearchCV can be an alternative in these situations, but does not always seem to provide the best parameters compared to scikit-optimize BayesSearchCV. In my humble opinion, scikit-optimize BayesSearchCV seems to be a nice compromise between GridSearchCV and RandomizedSearchCV, providing good parameters in a reasonable time.

Unfortunately, scikit-optimize BayesSearchCV seems to be no longer supported. The last commit was in 2021. As of NumPy 1.24, NumPy now results in an error, unless a workaround of np.int = int is used. This is just one example. There are numerous issues that have not been touched since 2021. It would be a shame to lose a project such as scikit-optimize BayesSearchCV and humbly ask the contributors of scikit-learn if a similar version could be implemented in scikit-learn.

Thanks so much for your consideration! Looking forward to the discussion.

[adrinjalali]

Thanks for the investigation, I opened #26170 to discuss this.

[earlev4]

Hi @adrinjalali! Thanks so much for the consideration and the discussion! I am very grateful for the help!

[earlev4]

Hi @adrinjalali! Are you aware of any updates on this discussion? I see @thomasjpfan added Needs Decision - Include Feature on April 14th to #26170, but there has been no further discussion after date. Thank so much!