Graph Conditioned Diffusion for Controllable Histopathology Image Generation
This work addresses the problem of generating controlled medical images for sensitive areas like histopathology, which is incremental as it builds on existing diffusion models by incorporating graph-based conditioning.
The paper tackled the challenge of controlled generation in medical imaging by proposing a graph-conditioned diffusion model that uses object-level graph representations to ensure meaningful control over generated histopathology images, demonstrating that the generated data can reliably substitute for annotated patient data in downstream segmentation tasks.
Recent advances in Diffusion Probabilistic Models (DPMs) have set new standards in high-quality image synthesis. Yet, controlled generation remains challenging, particularly in sensitive areas such as medical imaging. Medical images feature inherent structure such as consistent spatial arrangement, shape or texture, all of which are critical for diagnosis. However, existing DPMs operate in noisy latent spaces that lack semantic structure and strong priors, making it difficult to ensure meaningful control over generated content. To address this, we propose graph-based object-level representations for Graph-Conditioned-Diffusion. Our approach generates graph nodes corresponding to each major structure in the image, encapsulating their individual features and relationships. These graph representations are processed by a transformer module and integrated into a diffusion model via the text-conditioning mechanism, enabling fine-grained control over generation. We evaluate this approach using a real-world histopathology use case, demonstrating that our generated data can reliably substitute for annotated patient data in downstream segmentation tasks. The code is available here.