PyTorch implementation of our matrix-multiplication-only KFAC based on Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning (ICML 2023)

• Download the dataset from https://www.kaggle.com/datasets/ambityga/imagenet100

• The dataset should be saved in the data folder (i.e., data/imagenet100/)

• You can reproduce the results on imagenet-100 by using the provided bash scripts

• Todo:

  • Add code for optimization on SPD manifolds