Evaluating LLMs in Medicine: A Call for Rigor, Transparency
This work addresses the problem of unreliable LLM evaluations in medicine for researchers and practitioners, but it is incremental as it reviews existing limitations without proposing a new solution.
The paper identified that most existing benchmark datasets for evaluating large language models (LLMs) in medical question answering lack clinical realism, transparency, and robust validation, highlighting the need for more rigorous and representative datasets. It concluded that a standardized framework and collaborative efforts are essential to improve evaluation methodologies in this domain.
Objectives: To evaluate the current limitations of large language models (LLMs) in medical question answering, focusing on the quality of datasets used for their evaluation. Materials and Methods: Widely-used benchmark datasets, including MedQA, MedMCQA, PubMedQA, and MMLU, were reviewed for their rigor, transparency, and relevance to clinical scenarios. Alternatives, such as challenge questions in medical journals, were also analyzed to identify their potential as unbiased evaluation tools. Results: Most existing datasets lack clinical realism, transparency, and robust validation processes. Publicly available challenge questions offer some benefits but are limited by their small size, narrow scope, and exposure to LLM training. These gaps highlight the need for secure, comprehensive, and representative datasets. Conclusion: A standardized framework is critical for evaluating LLMs in medicine. Collaborative efforts among institutions and policymakers are needed to ensure datasets and methodologies are rigorous, unbiased, and reflective of clinical complexities.