Legal-DC: Benchmarking Retrieval-Augmented Generation for Legal Documents
It addresses the problem of improving legal document consultation for Chinese legal scenarios, though it is incremental as it builds on existing RAG technology with domain-specific adaptations.
This study tackled the lack of specialized benchmarks and systems for Retrieval-Augmented Generation (RAG) in Chinese legal documents by creating the Legal-DC benchmark dataset and proposing the LegRAG framework, which outperformed existing state-of-the-art methods by 1.3% to 5.6% on key metrics.
Retrieval-Augmented Generation (RAG) has emerged as a promising technology for legal document consultation, yet its application in Chinese legal scenarios faces two key limitations: existing benchmarks lack specialized support for joint retriever-generator evaluation, and mainstream RAG systems often fail to accommodate the structured nature of legal provisions. To address these gaps, this study advances two core contributions: First, we constructed the Legal-DC benchmark dataset, comprising 480 legal documents (covering areas such as market regulation and contract management) and 2,475 refined question-answer pairs, each annotated with clause-level references, filling the gap for specialized evaluation resources in Chinese legal RAG. Second, we propose the LegRAG framework, which integrates legal adaptive indexing (clause-boundary segmentation) with a dual-path self-reflection mechanism to ensure clause integrity while enhancing answer accuracy. Third, we introduce automated evaluation methods for large language models to meet the high-reliability demands of legal retrieval scenarios. LegRAG outperforms existing state-of-the-art methods by 1.3% to 5.6% across key evaluation metrics. This research provides a specialized benchmark, practical framework, and empirical insights to advance the development of Chinese legal RAG systems. Our code and data are available at https://github.com/legal-dc/Legal-DC.