CONVERSER: Few-Shot Conversational Dense Retrieval with Synthetic Data Generation
This addresses the challenge of expensive data collection for conversational search, enabling few-shot learning in information retrieval, though it is incremental as it builds on existing dense retrieval and synthetic data generation methods.
The paper tackles the problem of training conversational dense retrievers without large in-domain datasets by proposing CONVERSER, a framework that uses large language models to generate synthetic conversational queries from passages, achieving comparable performance to fully-supervised models on benchmarks like OR-QuAC and TREC CAsT 19 with at most 6 in-domain examples.
Conversational search provides a natural interface for information retrieval (IR). Recent approaches have demonstrated promising results in applying dense retrieval to conversational IR. However, training dense retrievers requires large amounts of in-domain paired data. This hinders the development of conversational dense retrievers, as abundant in-domain conversations are expensive to collect. In this paper, we propose CONVERSER, a framework for training conversational dense retrievers with at most 6 examples of in-domain dialogues. Specifically, we utilize the in-context learning capability of large language models to generate conversational queries given a passage in the retrieval corpus. Experimental results on conversational retrieval benchmarks OR-QuAC and TREC CAsT 19 show that the proposed CONVERSER achieves comparable performance to fully-supervised models, demonstrating the effectiveness of our proposed framework in few-shot conversational dense retrieval. All source code and generated datasets are available at https://github.com/MiuLab/CONVERSER