Learning to Abstain From Uninformative Data
This addresses the challenge of handling high noise-to-signal ratios in critical domains like Finance and Healthcare, offering a method to improve reliability in noisy data scenarios.
The paper tackles the problem of learning and decision-making in noisy environments where data contains both informative and uninformative samples, proposing a novel approach that guarantees near-optimal decisions by distinguishing between them, with empirical evaluation in various settings.
Learning and decision-making in domains with naturally high noise-to-signal ratio, such as Finance or Healthcare, is often challenging, while the stakes are very high. In this paper, we study the problem of learning and acting under a general noisy generative process. In this problem, the data distribution has a significant proportion of uninformative samples with high noise in the label, while part of the data contains useful information represented by low label noise. This dichotomy is present during both training and inference, which requires the proper handling of uninformative data during both training and testing. We propose a novel approach to learning under these conditions via a loss inspired by the selective learning theory. By minimizing this loss, the model is guaranteed to make a near-optimal decision by distinguishing informative data from uninformative data and making predictions. We build upon the strength of our theoretical guarantees by describing an iterative algorithm, which jointly optimizes both a predictor and a selector, and evaluates its empirical performance in a variety of settings.