Medical AI Consensus: A Multi-Agent Framework for Radiology Report Generation and Evaluation
This work addresses the need for clinically reliable systems and rigorous evaluation in radiology AI, though it is incremental as it builds on existing multi-agent and LLM methods.
The paper tackles the challenge of automating radiology report generation by introducing a multi-agent reinforcement learning framework that integrates LLMs and LVMs for both generation and evaluation, demonstrating its implementation on public datasets with chatGPT-4o and radiologist feedback.
Automating radiology report generation poses a dual challenge: building clinically reliable systems and designing rigorous evaluation protocols. We introduce a multi-agent reinforcement learning framework that serves as both a benchmark and evaluation environment for multimodal clinical reasoning in the radiology ecosystem. The proposed framework integrates large language models (LLMs) and large vision models (LVMs) within a modular architecture composed of ten specialized agents responsible for image analysis, feature extraction, report generation, review, and evaluation. This design enables fine-grained assessment at both the agent level (e.g., detection and segmentation accuracy) and the consensus level (e.g., report quality and clinical relevance). We demonstrate an implementation using chatGPT-4o on public radiology datasets, where LLMs act as evaluators alongside medical radiologist feedback. By aligning evaluation protocols with the LLM development lifecycle, including pretraining, finetuning, alignment, and deployment, the proposed benchmark establishes a path toward trustworthy deviance-based radiology report generation.