Weakly-Supervised Scientific Document Classification via Retrieval-Augmented Multi-Stage Training
This addresses the high cost of labeling for scientific document classification, though it is incremental as it builds on existing weakly-supervised techniques.
The paper tackles the problem of scientific document classification with limited labeled data by proposing a weakly-supervised method using only label names, achieving an average performance improvement of 11.9% over baselines on three datasets.
Scientific document classification is a critical task for a wide range of applications, but the cost of obtaining massive amounts of human-labeled data can be prohibitive. To address this challenge, we propose a weakly-supervised approach for scientific document classification using label names only. In scientific domains, label names often include domain-specific concepts that may not appear in the document corpus, making it difficult to match labels and documents precisely. To tackle this issue, we propose WANDER, which leverages dense retrieval to perform matching in the embedding space to capture the semantics of label names. We further design the label name expansion module to enrich the label name representations. Lastly, a self-training step is used to refine the predictions. The experiments on three datasets show that WANDER outperforms the best baseline by 11.9% on average. Our code will be published at https://github.com/ritaranx/wander.