Who Checks the Citations? Benchmarking Legal Hallucination Detection
For legal professionals and AI developers, this work provides a benchmark and dataset for detecting citation hallucinations in legal documents, revealing that current AI systems are insufficient for reliable detection.
The paper evaluates AI-based systems for detecting hallucinated legal citations, finding that even the best model (GPT-5) achieves only 82.8% recall and 60.5% F1 score in an agentic framework, with all models struggling on subtle errors. The authors introduce a taxonomy and dataset of 1,300 brief excerpts with injected errors, and highlight policy concerns due to restricted information access.
Attorneys, judges, and pro se filers increasingly use AI to draft legal documents, yet these tools frequently fabricate citations. Despite predictions that newer models would hallucinate less or that court sanctions would deter negligent filers, we found over 1,000 filings containing fabricated citations -- with this number growing year-over-year. This study evaluates whether AI-based systems can mitigate these errors by automatically detecting hallucinations. We propose a taxonomy of legal citation hallucinations grounded in actual court filings and introduce a dataset of 1,300 brief excerpts containing injected errors. Benchmarking five models in agentic and non-agentic settings reveals that while the latest iterations perform better -- GPT-5 achieves 82.8% recall and a 60.5% F1 score in an agentic framework -- all models struggle with subtle error categories. Agentic verification remains resource-intensive, with GPT-5 averaging 16.9 steps per excerpt. Furthermore, restricted information access limits the efficacy of even the best agents. This gap creates policy concerns, as it disadvantages both AI systems and litigants who lack subscriptions to commercial legal databases. Together, our dataset, tools, and policy recommendations provide a foundation for building and auditing reliable legal citation checking tools.