Lbl2Vec: An Embedding-Based Approach for Unsupervised Document Retrieval on Predefined Topics
This addresses the problem of retrieving documents without labeled data for researchers and practitioners in NLP, offering a practical tool with incremental improvements over existing methods.
The paper tackles unsupervised document retrieval for predefined topics using only keywords, achieving average AUC values of 0.95 and 0.92 on datasets and improving F1 scores from 76.6 to 82.7 and 61.0 to 75.1 compared to baselines.
In this paper, we consider the task of retrieving documents with predefined topics from an unlabeled document dataset using an unsupervised approach. The proposed unsupervised approach requires only a small number of keywords describing the respective topics and no labeled document. Existing approaches either heavily relied on a large amount of additionally encoded world knowledge or on term-document frequencies. Contrariwise, we introduce a method that learns jointly embedded document and word vectors solely from the unlabeled document dataset in order to find documents that are semantically similar to the topics described by the keywords. The proposed method requires almost no text preprocessing but is simultaneously effective at retrieving relevant documents with high probability. When successively retrieving documents on different predefined topics from publicly available and commonly used datasets, we achieved an average area under the receiver operating characteristic curve value of 0.95 on one dataset and 0.92 on another. Further, our method can be used for multiclass document classification, without the need to assign labels to the dataset in advance. Compared with an unsupervised classification baseline, we increased F1 scores from 76.6 to 82.7 and from 61.0 to 75.1 on the respective datasets. For easy replication of our approach, we make the developed Lbl2Vec code publicly available as a ready-to-use tool under the 3-Clause BSD license.