Hierarchical Diffusion Autoencoders and Disentangled Image Manipulation
This work addresses the need for more interpretable and controllable image generation in computer vision, though it is incremental as it builds on prior diffusion autoencoders.
The paper tackles the problem of interpreting and manipulating the latent space of diffusion models for image synthesis by proposing Hierarchical Diffusion Autoencoders (HDAE) to encode fine-grained and abstract semantic hierarchies, enabling applications like detail-preserving image manipulation and multi-modal synthesis.
Diffusion models have attained impressive visual quality for image synthesis. However, how to interpret and manipulate the latent space of diffusion models has not been extensively explored. Prior work diffusion autoencoders encode the semantic representations into a semantic latent code, which fails to reflect the rich information of details and the intrinsic feature hierarchy. To mitigate those limitations, we propose Hierarchical Diffusion Autoencoders (HDAE) that exploit the fine-grained-to-abstract and lowlevel-to-high-level feature hierarchy for the latent space of diffusion models. The hierarchical latent space of HDAE inherently encodes different abstract levels of semantics and provides more comprehensive semantic representations. In addition, we propose a truncated-feature-based approach for disentangled image manipulation. We demonstrate the effectiveness of our proposed approach with extensive experiments and applications on image reconstruction, style mixing, controllable interpolation, detail-preserving and disentangled image manipulation, and multi-modal semantic image synthesis.