Diffusion Prism: Enhancing Diversity and Morphology Consistency in Mask-to-Image Diffusion
This addresses a constraint in data augmentation for generative AI, particularly in biological and material science domains, though it appears incremental as it builds on existing controllable diffusion models.
The paper tackles the problem of limited diversity in mask-to-image diffusion models when input images have low entropy, proposing Diffusion Prism to generate realistic and diverse samples while preserving morphological features, with demonstrations on nano-dendritic patterns showing improved results compared to existing methods.
The emergence of generative AI and controllable diffusion has made image-to-image synthesis increasingly practical and efficient. However, when input images exhibit low entropy and sparse, the inherent characteristics of diffusion models often result in limited diversity. This constraint significantly interferes with data augmentation. To address this, we propose Diffusion Prism, a training-free framework that efficiently transforms binary masks into realistic and diverse samples while preserving morphological features. We explored that a small amount of artificial noise will significantly assist the image-denoising process. To prove this novel mask-to-image concept, we use nano-dendritic patterns as an example to demonstrate the merit of our method compared to existing controllable diffusion models. Furthermore, we extend the proposed framework to other biological patterns, highlighting its potential applications across various fields.