Exploiting Instruction-Following Retrievers for Malicious Information Retrieval
This work highlights a critical safety problem for AI systems using retrieval-augmented generation, showing incremental risks as retriever capabilities increase.
The study investigated the safety risks of instruction-following retrievers by testing their ability to satisfy malicious queries, finding that most retrievers, such as LLM2Vec, could select relevant harmful passages for over 50% of queries, with LLM2Vec achieving 61.35% accuracy, and that even safety-aligned LLMs like Llama3 could misuse retrieved harmful information.
Instruction-following retrievers have been widely adopted alongside LLMs in real-world applications, but little work has investigated the safety risks surrounding their increasing search capabilities. We empirically study the ability of retrievers to satisfy malicious queries, both when used directly and when used in a retrieval augmented generation-based setup. Concretely, we investigate six leading retrievers, including NV-Embed and LLM2Vec, and find that given malicious requests, most retrievers can (for >50% of queries) select relevant harmful passages. For example, LLM2Vec correctly selects passages for 61.35% of our malicious queries. We further uncover an emerging risk with instruction-following retrievers, where highly relevant harmful information can be surfaced by exploiting their instruction-following capabilities. Finally, we show that even safety-aligned LLMs, such as Llama3, can satisfy malicious requests when provided with harmful retrieved passages in-context. In summary, our findings underscore the malicious misuse risks associated with increasing retriever capability.