CRG Score: A Distribution-Aware Clinical Metric for Radiology Report Generation
This addresses the problem of clinical accuracy assessment in radiology report generation for medical AI applications, though it is incremental as it builds on existing metrics.
The paper tackles the challenge of evaluating radiology report generation by proposing the CRG Score, a distribution-aware metric that focuses on clinically relevant abnormalities and balances penalties based on label distribution, resulting in fairer and more robust evaluation.
Evaluating long-context radiology report generation is challenging. NLG metrics fail to capture clinical correctness, while LLM-based metrics often lack generalizability. Clinical accuracy metrics are more relevant but are sensitive to class imbalance, frequently favoring trivial predictions. We propose the CRG Score, a distribution-aware and adaptable metric that evaluates only clinically relevant abnormalities explicitly described in reference reports. CRG supports both binary and structured labels (e.g., type, location) and can be paired with any LLM for feature extraction. By balancing penalties based on label distribution, it enables fairer, more robust evaluation and serves as a clinically aligned reward function.