Deep Active Learning: A Reality Check
This work provides a reality check for researchers applying active learning to deep learning tasks with limited annotation budgets, revealing limitations of current methods and offering concrete recommendations.
The paper conducted a comprehensive evaluation of state-of-the-art deep active learning methods, finding that under general settings, no single-model method decisively outperforms entropy-based active learning, with some even falling short of random sampling.
We conduct a comprehensive evaluation of state-of-the-art deep active learning methods. Surprisingly, under general settings, no single-model method decisively outperforms entropy-based active learning, and some even fall short of random sampling. We delve into overlooked aspects like starting budget, budget step, and pretraining's impact, revealing their significance in achieving superior results. Additionally, we extend our evaluation to other tasks, exploring the active learning effectiveness in combination with semi-supervised learning, and object detection. Our experiments provide valuable insights and concrete recommendations for future active learning studies. By uncovering the limitations of current methods and understanding the impact of different experimental settings, we aim to inspire more efficient training of deep learning models in real-world scenarios with limited annotation budgets. This work contributes to advancing active learning's efficacy in deep learning and empowers researchers to make informed decisions when applying active learning to their tasks.