Ranking with Abstention
This addresses ranking tasks where abstention can improve reliability, though it is incremental as it builds on existing consistency theory.
The paper tackles the problem of ranking with abstention, where a learner can abstain from predictions at a limited cost, by providing state-of-the-art H-consistency bounds for linear functions and neural networks, with experimental results demonstrating its effectiveness.
We introduce a novel framework of ranking with abstention, where the learner can abstain from making prediction at some limited cost $c$. We present a extensive theoretical analysis of this framework including a series of $H$-consistency bounds for both the family of linear functions and that of neural networks with one hidden-layer. These theoretical guarantees are the state-of-the-art consistency guarantees in the literature, which are upper bounds on the target loss estimation error of a predictor in a hypothesis set $H$, expressed in terms of the surrogate loss estimation error of that predictor. We further argue that our proposed abstention methods are important when using common equicontinuous hypothesis sets in practice. We report the results of experiments illustrating the effectiveness of ranking with abstention.