The Use of Unlabeled Data versus Labeled Data for Stopping Active Learning for Text Classification
This addresses the annotation bottleneck in text classification systems, but the work is incremental as it compares existing stopping methods without introducing new techniques.
The paper tackled the problem of determining when to stop labeling data in active learning for text classification, finding that methods using unlabeled data are more effective than those using labeled data.
Annotation of training data is the major bottleneck in the creation of text classification systems. Active learning is a commonly used technique to reduce the amount of training data one needs to label. A crucial aspect of active learning is determining when to stop labeling data. Three potential sources for informing when to stop active learning are an additional labeled set of data, an unlabeled set of data, and the training data that is labeled during the process of active learning. To date, no one has compared and contrasted the advantages and disadvantages of stopping methods based on these three information sources. We find that stopping methods that use unlabeled data are more effective than methods that use labeled data.