Zero-shot Active Learning Using Self Supervised Learning
This addresses the tedious and expensive task of data annotation for deep learning models, though it appears incremental as it builds on existing active learning and self-supervised learning concepts.
The paper tackles the problem of selecting the best subset of data for annotation under a fixed budget in deep learning by proposing a new active learning approach that is model-agnostic and non-iterative, leveraging self-supervised learning to obtain useful feature representations without annotations.
Deep learning algorithms are often said to be data hungry. The performance of such algorithms generally improve as more and more annotated data is fed into the model. While collecting unlabelled data is easier (as they can be scraped easily from the internet), annotating them is a tedious and expensive task. Given a fixed budget available for data annotation, Active Learning helps selecting the best subset of data for annotation, such that the deep learning model when trained over that subset will have maximum generalization performance under this budget. In this work, we aim to propose a new Active Learning approach which is model agnostic as well as one doesn't require an iterative process. We aim to leverage self-supervised learnt features for the task of Active Learning. The benefit of self-supervised learning, is that one can get useful feature representation of the input data, without having any annotation.