Minority Class Oriented Active Learning for Imbalanced Datasets
This addresses the limitation of existing active learning methods for imbalanced datasets, which are common in real-life applications, though it is incremental as it adapts known concepts to a specific bottleneck.
The paper tackles the problem of active learning for imbalanced datasets by introducing a method that favors minority class samples to reduce imbalance and improve representation, showing it outperforms baselines on three imbalanced datasets. It also finds that a transfer learning training scheme outperforms fine-tuning when features are transferable, a surprising result that encourages further exploration in deep active learning.
Active learning aims to optimize the dataset annotation process when resources are constrained. Most existing methods are designed for balanced datasets. Their practical applicability is limited by the fact that a majority of real-life datasets are actually imbalanced. Here, we introduce a new active learning method which is designed for imbalanced datasets. It favors samples likely to be in minority classes so as to reduce the imbalance of the labeled subset and create a better representation for these classes. We also compare two training schemes for active learning: (1) the one commonly deployed in deep active learning using model fine tuning for each iteration and (2) a scheme which is inspired by transfer learning and exploits generic pre-trained models and train shallow classifiers for each iteration. Evaluation is run with three imbalanced datasets. Results show that the proposed active learning method outperforms competitive baselines. Equally interesting, they also indicate that the transfer learning training scheme outperforms model fine tuning if features are transferable from the generic dataset to the unlabeled one. This last result is surprising and should encourage the community to explore the design of deep active learning methods.