Model-Centric and Data-Centric Aspects of Active Learning for Deep Neural Networks
This work addresses the problem of improving active learning efficiency for deep neural networks, but it is incremental as it builds on existing methods without introducing a new paradigm.
The paper investigates various aspects of active learning for deep neural networks, including training modes, model configurations, query strategies, statistical analyses, and pseudo-label usage, resulting in more efficient querying procedures and insights into active learning behavior.
We study different aspects of active learning with deep neural networks in a consistent and unified way. i) We investigate incremental and cumulative training modes which specify how the newly labeled data are used for training. ii) We study active learning w.r.t. the model configurations such as the number of epochs and neurons as well as the choice of batch size. iii) We consider in detail the behavior of query strategies and their corresponding informativeness measures and accordingly propose more efficient querying procedures. iv) We perform statistical analyses, e.g., on actively learned classes and test error estimation, that reveal several insights about active learning. v) We investigate how active learning with neural networks can benefit from pseudo-labels as proxies for actual labels.