Contrastive Fine-tuning Improves Robustness for Neural Rankers
This addresses robustness issues in neural rankers for information retrieval, offering a method that outperforms data augmentation, but it is incremental as it builds on existing fine-tuning techniques.
The paper tackles the problem of neural rankers' performance deteriorating with noisy inputs or new domains by introducing a contrastive fine-tuning method that improves robustness to out-of-domain data and query perturbations, achieving improvements across four passage ranking datasets for BERT and BART based rankers.
The performance of state-of-the-art neural rankers can deteriorate substantially when exposed to noisy inputs or applied to a new domain. In this paper, we present a novel method for fine-tuning neural rankers that can significantly improve their robustness to out-of-domain data and query perturbations. Specifically, a contrastive loss that compares data points in the representation space is combined with the standard ranking loss during fine-tuning. We use relevance labels to denote similar/dissimilar pairs, which allows the model to learn the underlying matching semantics across different query-document pairs and leads to improved robustness. In experiments with four passage ranking datasets, the proposed contrastive fine-tuning method obtains improvements on robustness to query reformulations, noise perturbations, and zero-shot transfer for both BERT and BART based rankers. Additionally, our experiments show that contrastive fine-tuning outperforms data augmentation for robustifying neural rankers.