PathDiff: Histopathology Image Synthesis with Unpaired Text and Mask Conditions
This work addresses data scarcity in medical imaging for researchers and clinicians, but it is incremental as it builds on existing diffusion models with a novel conditioning approach.
The paper tackles the problem of synthesizing histopathology images by addressing the lack of paired text and mask data, proposing PathDiff, a diffusion framework that learns from unpaired data to generate high-quality images with improved fidelity, text-image alignment, and faithfulness for tasks like nuclei segmentation and classification.
Diffusion-based generative models have shown promise in synthesizing histopathology images to address data scarcity caused by privacy constraints. Diagnostic text reports provide high-level semantic descriptions, and masks offer fine-grained spatial structures essential for representing distinct morphological regions. However, public datasets lack paired text and mask data for the same histopathological images, limiting their joint use in image generation. This constraint restricts the ability to fully exploit the benefits of combining both modalities for enhanced control over semantics and spatial details. To overcome this, we propose PathDiff, a diffusion framework that effectively learns from unpaired mask-text data by integrating both modalities into a unified conditioning space. PathDiff allows precise control over structural and contextual features, generating high-quality, semantically accurate images. PathDiff also improves image fidelity, text-image alignment, and faithfulness, enhancing data augmentation for downstream tasks like nuclei segmentation and classification. Extensive experiments demonstrate its superiority over existing methods.