A Comparative Survey of Deep Active Learning
This work addresses the need for standardized evaluation in DAL, which is incremental as it synthesizes existing methods rather than introducing new ones.
The authors tackled the lack of fair performance comparisons in Deep Active Learning (DAL) by constructing DeepAL+, a toolkit that re-implements 19 highly-cited DAL methods and conducts comparative experiments across datasets and algorithms, providing better references for researchers.
While deep learning (DL) is data-hungry and usually relies on extensive labeled data to deliver good performance, Active Learning (AL) reduces labeling costs by selecting a small proportion of samples from unlabeled data for labeling and training. Therefore, Deep Active Learning (DAL) has risen as a feasible solution for maximizing model performance under a limited labeling cost/budget in recent years. Although abundant methods of DAL have been developed and various literature reviews conducted, the performance evaluation of DAL methods under fair comparison settings is not yet available. Our work intends to fill this gap. In this work, We construct a DAL toolkit, DeepAL+, by re-implementing 19 highly-cited DAL methods. We survey and categorize DAL-related works and construct comparative experiments across frequently used datasets and DAL algorithms. Additionally, we explore some factors (e.g., batch size, number of epochs in the training process) that influence the efficacy of DAL, which provides better references for researchers to design their DAL experiments or carry out DAL-related applications.