Recalibrating binary probabilistic classifiers
This work addresses the need for reliable recalibration methods in domains like credit risk management, though it appears incremental as it builds on existing distribution shift perspectives.
The paper tackles the problem of recalibrating binary probabilistic classifiers to a target prior probability, particularly in credit risk management, by proposing two new methods (CSPD and QMM) that leverage distribution shift assumptions and AUC, with QMM showing appropriately conservative results in evaluations with concave functionals like risk weight functions.
Recalibration of binary probabilistic classifiers to a target prior probability is an important task in areas like credit risk management. We analyse methods for recalibration from a distribution shift perspective. Distribution shift assumptions linked to the area under the curve (AUC) of a probabilistic classifier are found to be useful for the design of meaningful recalibration methods. Two new methods called parametric covariate shift with posterior drift (CSPD) and ROC-based quasi moment matching (QMM) are proposed and tested together with some other methods in an example setting. The outcomes of the test suggest that the QMM methods discussed in the paper can provide appropriately conservative results in evaluations with concave functionals like for instance risk weights functions for credit risk.