UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversations
This work addresses inefficiencies in conversational search systems for users by integrating retrieval and generation, though it is incremental as it builds on existing unified model approaches.
The paper tackles the problem of separating retrieval and generation in conversational search systems by proposing a unified model that jointly fine-tunes dense retrieval and response generation, resulting in mutual improvement and outperforming baselines on five datasets.
The rapid advancement of conversational search systems revolutionizes how information is accessed by enabling the multi-turn interaction between the user and the system. Existing conversational search systems are usually built with two different models. This separation restricts the system from leveraging the intrinsic knowledge of the models simultaneously, which cannot ensure the effectiveness of retrieval benefiting the generation. The existing studies for developing unified models cannot fully address the aspects of understanding conversational context, managing retrieval independently, and generating responses. In this paper, we explore how to unify dense retrieval and response generation for large language models in conversation. We conduct joint fine-tuning with different objectives and design two mechanisms to reduce the inconsistency risks while mitigating data discrepancy. The evaluations on five conversational search datasets demonstrate that our unified model can mutually improve both tasks and outperform the existing baselines.