DRAFT: Dense Retrieval Augmented Few-shot Topic classifier Framework
This addresses the need for efficient topic classification in diverse information environments, though it appears incremental as it builds on existing retrieval and fine-tuning methods.
The paper tackles the problem of few-shot topic classification by introducing DRAFT, a framework that uses a dense retriever to construct a customized dataset from few examples and fine-tunes a classifier, achieving competitive or superior performance to large language models like GPT-3 175B with 177 times fewer parameters.
With the growing volume of diverse information, the demand for classifying arbitrary topics has become increasingly critical. To address this challenge, we introduce DRAFT, a simple framework designed to train a classifier for few-shot topic classification. DRAFT uses a few examples of a specific topic as queries to construct Customized dataset with a dense retriever model. Multi-query retrieval (MQR) algorithm, which effectively handles multiple queries related to a specific topic, is applied to construct the Customized dataset. Subsequently, we fine-tune a classifier using the Customized dataset to identify the topic. To demonstrate the efficacy of our proposed approach, we conduct evaluations on both widely used classification benchmark datasets and manually constructed datasets with 291 diverse topics, which simulate diverse contents encountered in real-world applications. DRAFT shows competitive or superior performance compared to baselines that use in-context learning, such as GPT-3 175B and InstructGPT 175B, on few-shot topic classification tasks despite having 177 times fewer parameters, demonstrating its effectiveness.