RadIR: A Scalable Framework for Multi-Grained Medical Image Retrieval via Radiology Report Mining
This addresses the lack of scalable datasets and benchmarks for medical imaging retrieval, benefiting healthcare professionals, but it is incremental as it builds on existing retrieval methods.
The paper tackled the challenge of defining similarity for medical image retrieval by leveraging radiology reports to create datasets and systems, achieving state-of-the-art results on 77 out of 78 metrics.
Developing advanced medical imaging retrieval systems is challenging due to the varying definitions of `similar images' across different medical contexts. This challenge is compounded by the lack of large-scale, high-quality medical imaging retrieval datasets and benchmarks. In this paper, we propose a novel methodology that leverages dense radiology reports to define image-wise similarity ordering at multiple granularities in a scalable and fully automatic manner. Using this approach, we construct two comprehensive medical imaging retrieval datasets: MIMIC-IR for Chest X-rays and CTRATE-IR for CT scans, providing detailed image-image ranking annotations conditioned on diverse anatomical structures. Furthermore, we develop two retrieval systems, RadIR-CXR and model-ChestCT, which demonstrate superior performance in traditional image-image and image-report retrieval tasks. These systems also enable flexible, effective image retrieval conditioned on specific anatomical structures described in text, achieving state-of-the-art results on 77 out of 78 metrics.