Saving Dense Retriever from Shortcut Dependency in Conversational Search
This addresses a robustness issue in conversational search systems, which is incremental as it builds on existing dense retrieval methods to mitigate a specific dependency problem.
The paper tackled the problem of dense retrievers in conversational search exploiting a retrieval shortcut that relies on partial history instead of the latest question, and showed that using model-based hard negatives during training significantly improves performance, achieving an 11.0 point gain in Recall@10 on the QReCC benchmark.
Conversational search (CS) needs a holistic understanding of conversational inputs to retrieve relevant passages. In this paper, we demonstrate the existence of a retrieval shortcut in CS, which causes models to retrieve passages solely relying on partial history while disregarding the latest question. With in-depth analysis, we first show that naively trained dense retrievers heavily exploit the shortcut and hence perform poorly when asked to answer history-independent questions. To build more robust models against shortcut dependency, we explore various hard negative mining strategies. Experimental results show that training with the model-based hard negatives effectively mitigates the dependency on the shortcut, significantly improving dense retrievers on recent CS benchmarks. In particular, our retriever outperforms the previous state-of-the-art model by 11.0 in Recall@10 on QReCC.