Seed Word Selection for Weakly-Supervised Text Classification with Unsupervised Error Estimation
This addresses the challenge of seed word selection in weakly-supervised text classification, making it more accessible by reducing reliance on expert input, though it is incremental as it builds on existing weakly-supervised methods.
The paper tackles the problem of selecting effective seed words for weakly-supervised text classification without expert annotation by mining candidates, training interim models, and estimating error rates unsupervisedly. It shows that this method outperforms baseline category name seed words and matches performance with expert-annotated seed words across six binary classification tasks on four datasets.
Weakly-supervised text classification aims to induce text classifiers from only a few user-provided seed words. The vast majority of previous work assumes high-quality seed words are given. However, the expert-annotated seed words are sometimes non-trivial to come up with. Furthermore, in the weakly-supervised learning setting, we do not have any labeled document to measure the seed words' efficacy, making the seed word selection process "a walk in the dark". In this work, we remove the need for expert-curated seed words by first mining (noisy) candidate seed words associated with the category names. We then train interim models with individual candidate seed words. Lastly, we estimate the interim models' error rate in an unsupervised manner. The seed words that yield the lowest estimated error rates are added to the final seed word set. A comprehensive evaluation of six binary classification tasks on four popular datasets demonstrates that the proposed method outperforms a baseline using only category name seed words and obtained comparable performance as a counterpart using expert-annotated seed words.