Skip to content

Algorithms for abstention, calibration and domain adaptation to label shift.

Notifications You must be signed in to change notification settings

kundajelab/abstention

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Abstention, Calibration & Label Shift

Algorithms for abstention, calibration and domain adaptation to label shift.

Associated papers:

Shrikumar A*†, Alexandari A*, Kundaje A†, A Flexible and Adaptive Framework for Abstention Under Class Imbalance

Alexandari A*, Kundaje A†, Shrikumar A*†, Maximum Likelihood with Bias-Corrected Calibration is Hard-To-Beat at Label Shift Adaptation

*co-first authors † co-corresponding authors

Examples

See https://github.com/blindauth/abstention_experiments and https://github.com/blindauth/labelshiftexperiments for colab notebooks reproducing the experiments in the papers.

Installation

pip install abstention

Algorithms implemented

For calibration:

  • Platt Scaling
  • Isotonic Regression
  • Temperature Scaling
  • Vector Scaling
  • Bias-Corrected Temperature Scaling
  • No-Bias Vector Scaling

For domain adaptation to label shift:

  • Expectation Maximization (Saerens et al., 2002)
  • Black-Box Shift Learning (BBSL) (Lipton et al., 2018)
  • Regularized Learning under Label Shifts (RLLS) (Azizzadenesheli et al., 2019)

For abstention:

Contact

If you have any questions, please contact:

Avanti Shrikumar: avanti [dot] shrikumar [at] gmail.com

Amr Alexandari: amr [dot] alexandari [at] gmail.com

Anshul Kundaje: akundaje [at] stanford [dot] edu