Improve Cost Efficiency of Active Learning over Noisy Dataset
This work addresses cost efficiency in active learning for domains like finance where labeling costs are high, but it is incremental as it builds on existing uncertainty sampling methods.
The paper tackles the problem of high labeling costs in active learning for binary classification, particularly when positive instances are expensive, by proposing a shifted normal distribution sampling function that reduces noisy and positive label selection, achieving 20% to 32% improved cost efficiency in simulations.
Active learning is a learning strategy whereby the machine learning algorithm actively identifies and labels data points to optimize its learning. This strategy is particularly effective in domains where an abundance of unlabeled data exists, but the cost of labeling these data points is prohibitively expensive. In this paper, we consider cases of binary classification, where acquiring a positive instance incurs a significantly higher cost compared to that of negative instances. For example, in the financial industry, such as in money-lending businesses, a defaulted loan constitutes a positive event leading to substantial financial loss. To address this issue, we propose a shifted normal distribution sampling function that samples from a wider range than typical uncertainty sampling. Our simulation underscores that our proposed sampling function limits both noisy and positive label selection, delivering between 20% and 32% improved cost efficiency over different test datasets.