Model selection

[ivan-chai]

Hi!

It's not clear from the paper whether the best or last training checkpoint is used for testing. What is the default choice for both small datasets and ImageNet?

[zjysteven]

For most training it was selected according to the validation ID accuracy.

OpenOOD/openood/recorders/base_recorder.py

Line 40 in 989c41d

if val_metrics['acc'] >= self.best_acc:

For certain methods the selection criterion could be different (e.g., OpenGAN explicitly needs to look at val OOD AUROC). You can refer to the code under https://github.com/Jingkang50/OpenOOD/tree/main/openood/recorders for details.