Prediction stability as a criterion in active learning
This work addresses the annotation bottleneck in deep learning for researchers and practitioners, but it is incremental as it builds on existing active learning frameworks.
The paper tackled the problem of reducing annotation needs in deep learning by proposing a new active learning criterion called prediction stability, which leverages information during training rather than after, and achieved competitive accuracy on CIFAR-10 and outperformed traditional methods on CIFAR-100.
Recent breakthroughs made by deep learning rely heavily on large number of annotated samples. To overcome this shortcoming, active learning is a possible solution. Beside the previous active learning algorithms that only adopted information after training, we propose a new class of method based on the information during training, named sequential-based method. An specific criterion of active learning called prediction stability is proposed to prove the feasibility of sequential-based methods. Experiments are made on CIFAR-10 and CIFAR-100, and the results indicates that prediction stability is effective and works well on fewer-labeled datasets. Prediction stability reaches the accuracy of traditional acquisition functions like entropy on CIFAR-10, and notably outperforms them on CIFAR-100.