Large-scale probabilistic predictors with and without guarantees of validity
This work addresses the need for reliable probabilistic predictions in machine learning, offering a method that balances calibration and accuracy, though it is incremental in nature.
The paper tackles the problem of creating probabilistic predictors that are perfectly calibrated, achieving this through a method that yields imprecise probabilities which become almost precise for large datasets. When these are merged into precise probabilities, the resulting predictors show improved accuracy over existing methods in empirical studies.
This paper studies theoretically and empirically a method of turning machine-learning algorithms into probabilistic predictors that automatically enjoys a property of validity (perfect calibration) and is computationally efficient. The price to pay for perfect calibration is that these probabilistic predictors produce imprecise (in practice, almost precise for large data sets) probabilities. When these imprecise probabilities are merged into precise probabilities, the resulting predictors, while losing the theoretical property of perfect calibration, are consistently more accurate than the existing methods in empirical studies.