This repository provide a realization of VF-iDCA on hyperparameter selection problems in Python with codes for reproduce the numerical result of experiments and a general function.
The algorithm and the models are presented in the paper Value Function Based Difference-of-Convex Algorithm for Bilevel Hyperparameter Selection Problems
Based on anaconda (numpy for calculation, pandas for saving data and matplotlib for plotting) and cvxpy for defining and solving optimization problems.
To run the experiments for comparison, you should also install hyperopt (for Bayes method).
The code for synthetic data generation and experiments on IGJO algoritm are based on this repo.
IFDM algorithm is based on sparse_ho with little modification to make sure the data was shared.
- Clone this repository
- Run the following command under the experiments folder
You could change the parameters in the python file according to your demand.
python ElasticNet_Experiments.py
VF_iDCA.py provides a framework of our alogirthm, however, you need to write the model on your own as wlasso.py does, programming details are recorded in tutorial_wLasso.ipynb.
wlasso.py provides an example on using VF-iDCA solve hyperparameter selection problem on wlasso model.
If you use this code, please cite:
@article{gao2022difference,
title={Value Function Based Difference-of-Convex Algorithm for Bilevel Hyperparameter Selection Problems},
author={Gao, Lucy and Ye, Jane J. and Yin, Haian and Zeng, Shangzhi and Zhang, Jin},
journal={arXiv preprint arXiv:2206.05976},
year={2022}
}