Measuring Classification Decision Certainty and Doubt
This addresses uncertainty assessment in classification for machine learning practitioners, but appears incremental as it builds on existing frameworks without clear breakthroughs.
The authors tackled the problem of quantifying uncertainty in classification decisions by proposing intuitive certainty and doubt scores that work in both Bayesian and frequentist frameworks, but no concrete results or numbers are provided.
Quantitative characterizations and estimations of uncertainty are of fundamental importance in optimization and decision-making processes. Herein, we propose intuitive scores, which we call certainty and doubt, that can be used in both a Bayesian and frequentist framework to assess and compare the quality and uncertainty of predictions in (multi-)classification decision machine learning problems.