MedQ-Engine: A Closed-Loop Data Engine for Evolving MLLMs in Medical Image Quality Assessment
This addresses the high cost and adaptability issues in Med-IQA for clinical AI deployment, representing a strong domain-specific advancement.
The paper tackles the problem of improving multimodal large language models (MLLMs) for medical image quality assessment (Med-IQA) by proposing MedQ-Engine, a closed-loop data engine that iteratively identifies model weaknesses, retrieves relevant images, and fine-tunes with human annotations, resulting in an 8B-parameter model surpassing GPT-4o by over 13% and reducing the gap to human experts to 4.34% using only 10K annotations.
Medical image quality assessment (Med-IQA) is a prerequisite for clinical AI deployment, yet multimodal large language models (MLLMs) still fall substantially short of human experts, particularly when required to provide descriptive assessments with clinical reasoning beyond simple quality scores. However, improving them is hindered by the high cost of acquiring descriptive annotations and by the inability of one-time data collection to adapt to the model's evolving weaknesses. To address these challenges, we propose MedQ-Engine, a closed-loop data engine that iteratively evaluates the model to discover failure prototypes via data-driven clustering, explores a million-scale image pool using these prototypes as retrieval anchors with progressive human-in-the-loop annotation, and evolves through quality-assured fine-tuning, forming a self-improving cycle. Models are evaluated on complementary perception and description tasks. An entropy-guided routing mechanism triages annotations to minimize labeling cost. Experiments across five medical imaging modalities show that MedQ-Engine elevates an 8B-parameter model to surpass GPT-4o by over 13% and narrow the gap with human experts to only 4.34%, using only 10K annotations with more than 4x sample efficiency over random sampling.