Boosting Dermatoscopic Lesion Segmentation via Diffusion Models with Visual and Textual Prompts
This work addresses data scarcity in medical image analysis for dermatology, though it is incremental as it applies existing diffusion models to a specific domain.
The authors tackled the problem of generating controlled dermatoscopic images for data augmentation by adapting diffusion models with visual and textual prompts, resulting in a 9% increase in SSIM and over 5% improvement in Dice coefficients for segmentation.
Image synthesis approaches, e.g., generative adversarial networks, have been popular as a form of data augmentation in medical image analysis tasks. It is primarily beneficial to overcome the shortage of publicly accessible data and associated quality annotations. However, the current techniques often lack control over the detailed contents in generated images, e.g., the type of disease patterns, the location of lesions, and attributes of the diagnosis. In this work, we adapt the latest advance in the generative model, i.e., the diffusion model, with the added control flow using lesion-specific visual and textual prompts for generating dermatoscopic images. We further demonstrate the advantage of our diffusion model-based framework over the classical generation models in both the image quality and boosting the segmentation performance on skin lesions. It can achieve a 9% increase in the SSIM image quality measure and an over 5% increase in Dice coefficients over the prior arts.