One word at a time: adversarial attacks on retrieval models
This addresses the vulnerability of retrieval systems to adversarial manipulation, which is an incremental but important step in security for information retrieval applications.
The paper tackles the problem of evaluating the robustness of ranking models to adversarial attacks, finding that with just 1-3 token changes, attackers can generate semantically similar documents that fool rankers into lowering a document's rank by several positions.
Adversarial examples, generated by applying small perturbations to input features, are widely used to fool classifiers and measure their robustness to noisy inputs. However, little work has been done to evaluate the robustness of ranking models through adversarial examples. In this work, we present a systematic approach of leveraging adversarial examples to measure the robustness of popular ranking models. We explore a simple method to generate adversarial examples that forces a ranker to incorrectly rank the documents. Using this approach, we analyze the robustness of various ranking models and the quality of perturbations generated by the adversarial attacker across two datasets. Our findings suggest that with very few token changes (1-3), the attacker can yield semantically similar perturbed documents that can fool different rankers into changing a document's score, lowering its rank by several positions.