A Convex Loss Function for Set Prediction with Optimal Trade-offs Between Size and Conditional Coverage
This work addresses uncertainty quantification in supervised learning for researchers and practitioners, offering a novel approach to set prediction with conditional coverage, though it is incremental in advancing existing methods for uncertainty estimation.
The paper tackles the problem of set prediction with uncertainty estimates by proposing a convex loss function based on Choquet integrals, which enables optimal trade-offs between conditional coverage and set size. The method demonstrates improvements over marginal coverage approaches in experiments on synthetic classification and regression datasets.
We consider supervised learning problems in which set predictions provide explicit uncertainty estimates. Using Choquet integrals (a.k.a. Lov{á}sz extensions), we propose a convex loss function for nondecreasing subset-valued functions obtained as level sets of a real-valued function. This loss function allows optimal trade-offs between conditional probabilistic coverage and the ''size'' of the set, measured by a non-decreasing submodular function. We also propose several extensions that mimic loss functions and criteria for binary classification with asymmetric losses, and show how to naturally obtain sets with optimized conditional coverage. We derive efficient optimization algorithms, either based on stochastic gradient descent or reweighted least-squares formulations, and illustrate our findings with a series of experiments on synthetic datasets for classification and regression tasks, showing improvements over approaches that aim for marginal coverage.