Robust Polyp Detection and Diagnosis through Compositional Prompt-Guided Diffusion Models
This work addresses a critical problem in colorectal cancer screening by enhancing model robustness for medical professionals, though it is incremental as it builds on existing diffusion models with a novel prompt integration approach.
The paper tackles the challenge of generalizing deep learning models for polyp detection, classification, and segmentation across diverse clinical environments, particularly with out-of-distribution data, by proposing a Progressive Spectrum Diffusion Model (PSDM) that uses compositional prompts from clinical annotations to generate synthetic images, resulting in improvements such as a 2.12% increase in F1 score and 3.09% increase in mean average precision on the PolypGen dataset.
Colorectal cancer (CRC) is a significant global health concern, and early detection through screening plays a critical role in reducing mortality. While deep learning models have shown promise in improving polyp detection, classification, and segmentation, their generalization across diverse clinical environments, particularly with out-of-distribution (OOD) data, remains a challenge. Multi-center datasets like PolypGen have been developed to address these issues, but their collection is costly and time-consuming. Traditional data augmentation techniques provide limited variability, failing to capture the complexity of medical images. Diffusion models have emerged as a promising solution for generating synthetic polyp images, but the image generation process in current models mainly relies on segmentation masks as the condition, limiting their ability to capture the full clinical context. To overcome these limitations, we propose a Progressive Spectrum Diffusion Model (PSDM) that integrates diverse clinical annotations-such as segmentation masks, bounding boxes, and colonoscopy reports-by transforming them into compositional prompts. These prompts are organized into coarse and fine components, allowing the model to capture both broad spatial structures and fine details, generating clinically accurate synthetic images. By augmenting training data with PSDM-generated samples, our model significantly improves polyp detection, classification, and segmentation. For instance, on the PolypGen dataset, PSDM increases the F1 score by 2.12% and the mean average precision by 3.09%, demonstrating superior performance in OOD scenarios and enhanced generalization.