Venn-Abers predictors
This work addresses the need for reliable uncertainty quantification in machine learning predictions, particularly for binary classification tasks, though it appears incremental as it builds on existing Venn prediction methods.
The paper tackles the problem of producing well-calibrated probability predictions in binary classification by introducing Venn-Abers predictors, which are based on isotonic regression and guarantee calibration under standard assumptions, with promising empirical results reported.
This paper continues study, both theoretical and empirical, of the method of Venn prediction, concentrating on binary prediction problems. Venn predictors produce probability-type predictions for the labels of test objects which are guaranteed to be well calibrated under the standard assumption that the observations are generated independently from the same distribution. We give a simple formalization and proof of this property. We also introduce Venn-Abers predictors, a new class of Venn predictors based on the idea of isotonic regression, and report promising empirical results both for Venn-Abers predictors and for their more computationally efficient simplified version.