CONQRR: Conversational Query Rewriting for Retrieval with Reinforcement Learning
This addresses the problem of expensive retraining for conversational retrieval in question answering systems, offering an incremental improvement by enabling off-the-shelf retrievers to handle dialogue contexts.
The paper tackles the challenge of adapting non-conversational retrievers for conversational question answering by developing CONQRR, a query rewriting model trained with reinforcement learning to optimize retrieval, achieving state-of-the-art results on an open-domain CQA dataset with three sources and effectiveness across two retrievers.
Compared to standard retrieval tasks, passage retrieval for conversational question answering (CQA) poses new challenges in understanding the current user question, as each question needs to be interpreted within the dialogue context. Moreover, it can be expensive to re-train well-established retrievers such as search engines that are originally developed for non-conversational queries. To facilitate their use, we develop a query rewriting model CONQRR that rewrites a conversational question in the context into a standalone question. It is trained with a novel reward function to directly optimize towards retrieval using reinforcement learning and can be adapted to any off-the-shelf retriever. CONQRR achieves state-of-the-art results on a recent open-domain CQA dataset containing conversations from three different sources, and is effective for two different off-the-shelf retrievers. Our extensive analysis also shows the robustness of CONQRR to out-of-domain dialogues as well as to zero query rewriting supervision.