Authors: Max Zimmer, Christoph Spiegel, Sebastian Pokutta
This repository contains the code to reproduce the experiments from the ICLR24 paper "Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging". The code is based on PyTorch 1.9 and the experiment-tracking platform Weights & Biases.
Experiments are started from the following file:
main.py
: Starts experiments using the dictionary format of Weights & Biases.
The rest of the project is structured as follows:
strategies
: Contains the strategies used for training, pruning and model averaging.runners
: Contains classes to control the training and collection of metrics.metrics
: Contains all metrics as well as FLOP computation methods.models
: Contains all model architectures used.utilities
: Contains useful auxiliary functions and classes.
In case you find the paper or the implementation useful for your own research, please consider citing:
@inproceedings{zimmer2024sparse,
title={Sparse Model Soups: A Recipe for Improved Pruning via Model Averaging},
author={Max Zimmer and Christoph Spiegel and Sebastian Pokutta},
booktitle={The Twelfth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=xx0ITyHp3u}
}