Improving Ad-hoc Search Effectiveness for Conversational Information Retrieval via Model Merging
This work addresses the problem of catastrophic forgetting and high cost in conversational retrieval for researchers and practitioners, offering a practical solution that enhances ad-hoc performance while maintaining generalizability.
The paper introduces model merging as a training-free strategy to improve ad-hoc search effectiveness for conversational information retrieval, achieving up to 15% higher NDCG@3 under zero-shot conditions without additional fine-tuning.
Conversational information retrieval is challenging since it requires the consideration of the conversation history which potentially gives rise to topic shifts and coreference resolution across previous turns. To address these challenges, previous work mainly rely on traditional fine-tuning of ad-hoc retrievers on conversational datasets or extrapolates their generalizability through multi-tasking. However, this mainstream approach is costly - since it requires model re-training - and exhibits catastrophic forgetting, where the model loses its foundational ad-hoc retrieval performance. In this paper, we fill this gap by introducing model merging as a training-free strategy enabling the design of a single retrieval model that operates across both ad-hoc and conversational settings with no additional fine-tuning. We conduct experiments using linear and non-linear parameter-wise merging strategies - namely Model Soup and Slerp - on standard ad-hoc search and conversational retrieval datasets. Our results demonstrate that model merging significantly enhances the ad-hoc search capabilities of conversational retrievers while improving generalizability across task-specific datasets, achieving up to 15% higher NDCG@3 under zero-shot conditions.