Deep Active Learning with a Neural Architecture Search
This addresses the limitation of assuming a known architecture in active learning, offering a novel approach for improving efficiency in machine learning tasks.
The paper tackles the problem of active learning for deep neural networks by proposing a strategy that simultaneously searches for effective architectures while actively learning, showing that it overwhelmingly outperforms active learning with fixed architectures.
We consider active learning of deep neural networks. Most active learning works in this context have focused on studying effective querying mechanisms and assumed that an appropriate network architecture is a priori known for the problem at hand. We challenge this assumption and propose a novel active strategy whereby the learning algorithm searches for effective architectures on the fly, while actively learning. We apply our strategy using three known querying techniques (softmax response, MC-dropout, and coresets) and show that the proposed approach overwhelmingly outperforms active learning using fixed architectures.