Guiding Diffusion with Deep Geometric Moments: Balancing Fidelity and Variation
This work addresses the need for more flexible control mechanisms in diffusion-based image generation, offering a solution to the trade-off between fidelity and variation.
The paper tackled the problem of fine-grained control in text-to-image generation by introducing Deep Geometric Moments (DGM) as a novel guidance method, which effectively balances control and diversity in diffusion models.
Text-to-image generation models have achieved remarkable capabilities in synthesizing images, but often struggle to provide fine-grained control over the output. Existing guidance approaches, such as segmentation maps and depth maps, introduce spatial rigidity that restricts the inherent diversity of diffusion models. In this work, we introduce Deep Geometric Moments (DGM) as a novel form of guidance that encapsulates the subject's visual features and nuances through a learned geometric prior. DGMs focus specifically on the subject itself compared to DINO or CLIP features, which suffer from overemphasis on global image features or semantics. Unlike ResNets, which are sensitive to pixel-wise perturbations, DGMs rely on robust geometric moments. Our experiments demonstrate that DGM effectively balance control and diversity in diffusion-based image generation, allowing a flexible control mechanism for steering the diffusion process.