Open-PMC-18M: A High-Fidelity Large Scale Medical Dataset for Multimodal Representation Learning
This addresses the need for high-quality biomedical vision-language data for researchers, though it is incremental as it builds on existing subfigure extraction methods.
The authors tackled the problem of large-scale subfigure extraction from compound figures in biomedical literature by introducing a scalable pipeline and releasing OPEN-PMC-18M, a dataset of 18 million subfigure-caption pairs, which improved vision-language model performance on retrieval, zero-shot classification, and robustness benchmarks.
Compound figures, which are multi-panel composites containing diverse subfigures, are ubiquitous in biomedical literature, yet large-scale subfigure extraction remains largely unaddressed. Prior work on subfigure extraction has been limited in both dataset size and generalizability, leaving a critical open question: How does high-fidelity image-text alignment via large-scale subfigure extraction impact representation learning in vision-language models? We address this gap by introducing a scalable subfigure extraction pipeline based on transformer-based object detection, trained on a synthetic corpus of 500,000 compound figures, and achieving state-of-the-art performance on both ImageCLEF 2016 and synthetic benchmarks. Using this pipeline, we release OPEN-PMC-18M, a large-scale high quality biomedical vision-language dataset comprising 18 million clinically relevant subfigure-caption pairs spanning radiology, microscopy, and visible light photography. We train and evaluate vision-language models on our curated datasets and show improved performance across retrieval, zero-shot classification, and robustness benchmarks, outperforming existing baselines. We release our dataset, models, and code to support reproducible benchmarks and further study into biomedical vision-language modeling and representation learning.