Scaling medical imaging report generation with multimodal reinforcement learning
This addresses the challenge of generating accurate and generalizable medical reports from imaging data, which is crucial for healthcare applications, though it appears incremental as it builds on existing reinforcement learning methods.
The paper tackles the problem of medical imaging report generation by introducing UniRG, a framework that uses reinforcement learning to optimize for evaluation metrics, achieving new state-of-the-art results on the ReXrank benchmark with significant performance improvements.
Frontier models have demonstrated remarkable capabilities in understanding and reasoning with natural-language text, but they still exhibit major competency gaps in multimodal understanding and reasoning especially in high-value verticals such as biomedicine. Medical imaging report generation is a prominent example. Supervised fine-tuning can substantially improve performance, but they are prone to overfitting to superficial boilerplate patterns. In this paper, we introduce Universal Report Generation (UniRG) as a general framework for medical imaging report generation. By leveraging reinforcement learning as a unifying mechanism to directly optimize for evaluation metrics designed for end applications, UniRG can significantly improve upon supervised fine-tuning and attain durable generalization across diverse institutions and clinical practices. We trained UniRG-CXR on publicly available chest X-ray (CXR) data and conducted a thorough evaluation in CXR report generation with rigorous evaluation scenarios. On the authoritative ReXrank benchmark, UniRG-CXR sets new overall SOTA, outperforming prior state of the art by a wide margin.