On weighted uncertainty sampling in active learning
This is an incremental improvement for active learning systems, particularly in real-world scenarios with biased starting data.
The paper tackles the problem of active learning by exploring probabilistic sampling weighted by uncertainty, showing that this method is beneficial and strikes a good compromise between exploration and representation, especially when the initial labeled data is biased, as demonstrated on publicly available datasets.
This note explores probabilistic sampling weighted by uncertainty in active learning. This method has been previously used and authors have tangentially remarked on its efficacy. The scheme has several benefits: (1) it is computationally cheap, (2) it can be implemented in a single-pass streaming fashion which is a benefit when deployed in real-world systems where different subsystems perform the suggestion scoring and extraction of user feedback, and (3) it is easily parameterizable. In this paper, we show on publicly available datasets that using probabilistic weighting is often beneficial and strikes a good compromise between exploration and representation especially when the starting set of labelled points is biased.