Deep Learning Based Dense Retrieval: A Comparative Study
This work addresses robustness issues in information retrieval for critical applications, but it is incremental as it focuses on evaluating existing models rather than proposing new defenses.
The study assessed the vulnerability of dense retrieval systems to tokenizer poisoning, finding that supervised models like BERT and DPR suffer significant performance degradation, while unsupervised models like ANCE are more resilient, with small perturbations severely impacting retrieval accuracy.
Dense retrievers have achieved state-of-the-art performance in various information retrieval tasks, but their robustness against tokenizer poisoning remains underexplored. In this work, we assess the vulnerability of dense retrieval systems to poisoned tokenizers by evaluating models such as BERT, Dense Passage Retrieval (DPR), Contriever, SimCSE, and ANCE. We find that supervised models like BERT and DPR experience significant performance degradation when tokenizers are compromised, while unsupervised models like ANCE show greater resilience. Our experiments reveal that even small perturbations can severely impact retrieval accuracy, highlighting the need for robust defenses in critical applications.