TrustRAG

“Existing defenses treat poisoning as a content-quality problem: filtering bad documents ragdefender, detecting anomalous signals revprag,avfilter, scoring trustworthiness trustrag, or isolating passages robustrag. These approaches share an implicit assumption: if the system can identify poisoned evidence, it will naturally avoid acting on it. We show this assumption is incorrect.”

“it makes the unrealistic assumption that malicious documents form a separate cluster in the embedding space.”

“For numerical manipulation: the poisoned passage clusters with the legitimate passage (cosine similarity 0.9997 means they are in the same cluster). Stage 1 clustering cannot separate them. Stage 2 LLM self-assessment would need to notice that `$15,500` ≠ `$15,000` across two passages in the context window, but research on LLM numerical reasoning shows this is unreliable, especially when the numbers are embedded in otherwise identical text.”

“Although promising, these approaches have two major limitations: Majority-voting often fails under high poisoning, while heuristic and aggressive filtering may lose relevant content under low poisoning.”