Auditing: Active Learning with Outcome-Dependent Query Costs
This addresses cost-sensitive learning for fraud detection, offering a novel framework but is incremental in its application to simple hypothesis classes.
The paper tackles the problem of binary classification where labeling costs depend on the unknown label value, focusing on the extreme case where only negative labels incur cost, motivated by applications like fraud detection. They propose auditing algorithms for simple hypothesis classes and show that auditing complexity can be significantly lower than active label complexity.
We propose a learning setting in which unlabeled data is free, and the cost of a label depends on its value, which is not known in advance. We study binary classification in an extreme case, where the algorithm only pays for negative labels. Our motivation are applications such as fraud detection, in which investigating an honest transaction should be avoided if possible. We term the setting auditing, and consider the auditing complexity of an algorithm: the number of negative labels the algorithm requires in order to learn a hypothesis with low relative error. We design auditing algorithms for simple hypothesis classes (thresholds and rectangles), and show that with these algorithms, the auditing complexity can be significantly lower than the active label complexity. We also discuss a general competitive approach for auditing and possible modifications to the framework.