Deep Active Learning for Biased Datasets via Fisher Kernel Self-Supervision
This work addresses the problem of biased data in active learning for deep neural networks, offering a practical solution for reducing labeling costs in real-world applications, though it is incremental in improving existing methods.
The paper tackles active learning for biased datasets by deriving an optimal acquisition function and implementing it with a self-supervised Fisher kernel method, achieving a 40% reduction in labeling efforts and outperforming state-of-the-art methods on benchmarks like MNIST, SVHN, and ImageNet with only 1/10th of the processing.
Active learning (AL) aims to minimize labeling efforts for data-demanding deep neural networks (DNNs) by selecting the most representative data points for annotation. However, currently used methods are ill-equipped to deal with biased data. The main motivation of this paper is to consider a realistic setting for pool-based semi-supervised AL, where the unlabeled collection of train data is biased. We theoretically derive an optimal acquisition function for AL in this setting. It can be formulated as distribution shift minimization between unlabeled train data and weakly-labeled validation dataset. To implement such acquisition function, we propose a low-complexity method for feature density matching using self-supervised Fisher kernel (FK) as well as several novel pseudo-label estimators. Our FK-based method outperforms state-of-the-art methods on MNIST, SVHN, and ImageNet classification while requiring only 1/10th of processing. The conducted experiments show at least 40% drop in labeling efforts for the biased class-imbalanced data compared to existing methods.