MedFact: Benchmarking the Fact-Checking Capabilities of Large Language Models on Chinese Medical Texts
This addresses the need for reliable fact-checking in medical AI to ensure patient safety, though it is incremental as it focuses on benchmarking rather than proposing new methods.
The authors tackled the problem of evaluating fact-checking capabilities of large language models (LLMs) in Chinese medical contexts, introducing MedFact, a benchmark with 2,116 expert-annotated instances, and found that models often detect errors but struggle with precise localization, with top performers not matching human performance.
Deploying Large Language Models (LLMs) in medical applications requires fact-checking capabilities to ensure patient safety and regulatory compliance. We introduce MedFact, a challenging Chinese medical fact-checking benchmark with 2,116 expert-annotated instances from diverse real-world texts, spanning 13 specialties, 8 error types, 4 writing styles, and 5 difficulty levels. Construction uses a hybrid AI-human framework where iterative expert feedback refines AI-driven, multi-criteria filtering to ensure high quality and difficulty. We evaluate 20 leading LLMs on veracity classification and error localization, and results show models often determine if text contains errors but struggle to localize them precisely, with top performers falling short of human performance. Our analysis reveals the "over-criticism" phenomenon, a tendency for models to misidentify correct information as erroneous, which can be exacerbated by advanced reasoning techniques such as multi-agent collaboration and inference-time scaling. MedFact highlights the challenges of deploying medical LLMs and provides resources to develop factually reliable medical AI systems.