An Empirical Study on the Efficacy of Deep Active Learning for Image Classification
This study provides practical guidance for practitioners to reduce labeling costs in image classification, though it is incremental as it synthesizes existing methods rather than introducing new ones.
This paper comprehensively evaluated 19 deep active learning methods in a uniform setting to address inconsistent results, finding that most traditional methods do not outperform random selection, while semi-supervised techniques significantly improve performance, especially with abundant unlabeled data.
Deep Active Learning (DAL) has been advocated as a promising method to reduce labeling costs in supervised learning. However, existing evaluations of DAL methods are based on different settings, and their results are controversial. To tackle this issue, this paper comprehensively evaluates 19 existing DAL methods in a uniform setting, including traditional fully-\underline{s}upervised \underline{a}ctive \underline{l}earning (SAL) strategies and emerging \underline{s}emi-\underline{s}upervised \underline{a}ctive \underline{l}earning (SSAL) techniques. We have several non-trivial findings. First, most SAL methods cannot achieve higher accuracy than random selection. Second, semi-supervised training brings significant performance improvement compared to pure SAL methods. Third, performing data selection in the SSAL setting can achieve a significant and consistent performance improvement, especially with abundant unlabeled data. Our findings produce the following guidance for practitioners: one should (i) apply SSAL early and (ii) collect more unlabeled data whenever possible, for better model performance.