JustLLMGRPO: Radiographic Control for Chest X-Ray Generation
This work addresses the problem of generating more realistic and accurate chest X-rays from text prompts for medical imaging researchers and practitioners, offering a substantial improvement over existing methods.
The authors propose a method to improve text-conditioned chest X-ray generation by optimizing the textual prompts rather than the image generator. Their JustLLMGRPO method, which applies Group Relative Policy Optimization to an LLM prompt policy, reduces RadDINO-FID from 54.225 to 26.780 on CheXGenBench, a 50.6% improvement, while maintaining source-prompt alignment.
Text-conditioned chest X-ray generation aims to synthesize realistic radiographs that faithfully depict specified findings. Existing work has primarily improved quality by updating image generators, implicitly treating prompts as fixed after CXR-domain adaptation. We show that this generator-centric view leaves a substantial optimization dimension underexplored. With a CXR-adapted Sana generator frozen, one-pass reformulation by an unmodified LLM reduces RadDINO-FID from 54.225 to 27.572. Prompt analysis shows that the LLM suppresses temporal comparisons, uncertainty, and other non-renderable report content while emphasizing visible radiographic findings. However, unconstrained reformulation reduces BioViL-T alignment with source prompts from 0.695 to 0.609. We therefore introduce JustLLMGRPO, which applies standard Group Relative Policy Optimization (GRPO) only to the LLM prompt policy while keeping Sana frozen. Group-relative radiology-aware image feedback retains visual focus while preserving source-prompt alignment. On CheXGenBench, JustLLMGRPO reduces RadDINO-FID to 26.780, a 50.6% improvement over direct prompting, while maintaining alignment (0.696 versus 0.695). It also achieves state-of-the-art distribution coverage and downstream classification utility. These results show that substantial performance can remain latent in how radiographic information is expressed to an adapted generator. Code is publicly available at https://github.com/pxcai/JustLLMGRPO.