RobustRAG

“one cannot expect that LLMs always generate correct judgments, and thus the manipulated final input might lose crucial information or include wrong information before conducting answer generation”

“Although existing defenses can mitigate the impact of poisoning attacks on RAG systems to some extent, they remain vulnerable to advanced attacks, where the attacker craft sophisticated strategies to bypass current safeguards”

“These approaches share an implicit assumption: if the system can identify poisoned evidence, it will naturally avoid acting on it. We show this assumption is incorrect. The deeper problem is the monitoring-control gap: models may detect contradictions and untrustworthy evidence, yet this awareness does not reliably govern their final output.”

“they suffer from additional computational overhead, such as multiple LLM inferences or substantial memory consumption.”

“For instance, RobustRAG~xiang2024certifiably fails when an attacker poisons more than half of the retrieved texts for a target question.”

“RobustRAG~xiang2024certifiably follows an "isolate-then-aggregate" pipeline, where answers are independently generated for each retrieved document and then aggregated. This approach not only incurs high inference costs but also becomes ineffective when the proportion of negative documents is high.”

“RobustRAG~xiang2024RobustRAG, as the major existing RAG framework for adversarial robustness, suffers from limited performance in benign (no-attack) scenarios and struggles in complex generation tasks.”

“For numerical manipulation: the poisoned passage is semantically identical to the original and will appear in top-k with high rank. The isolated response from the poisoned passage will say `$15,500` while responses from other passages (if any discuss the same topic) will say `$15,000.` However, in a mixed corpus with 1,000+ passages, the probability that multiple top-k passages discuss the exact same numerical claim is low. The poisoned passage often stands alone on its topic, making majority vote ineffective because there is no majority to outvote it.”

“the aforementioned defense methods necessitate the integration of additional large models, incurring considerable overheads. Meanwhile, it is difficult to promptly assess whether the current response of RAG is trustworthy or not. Moreover, they're all ``best-effort'' schemes, offering no guarantee on the defense effectiveness.”

“However, CAR~weller-etal-2024-defending and RobustRAG~xiang2024certifiably require multiple rounds of model inference, leading to inefficiency.”

“RobustRAG extracts document keywords as context, which also results in information loss.”

“Heuristic aggregation or filtering~xiang2024certifiably often causes utility loss, while optimization-based consistency selection~shenreliabilityrag typically relies on approximations without strong guarantees.”