Automated Learning of Semantic Embedding Representations for Diffusion Models
This work addresses the problem of learning semantic embeddings for diffusion models, which could benefit general-purpose deep learning applications, though it appears incremental as it builds on existing DDM frameworks.
The paper tackles the lack of efficient representation learning for denoising diffusion models (DDMs) by proposing a multi-level denoising autoencoder framework with Diffusion Transformers and a timestep-dependent encoder, resulting in embeddings that surpass state-of-the-art self-supervised methods in most cases.
Generative models capture the true distribution of data, yielding semantically rich representations. Denoising diffusion models (DDMs) exhibit superior generative capabilities, though efficient representation learning for them are lacking. In this work, we employ a multi-level denoising autoencoder framework to expand the representation capacity of DDMs, which introduces sequentially consistent Diffusion Transformers and an additional timestep-dependent encoder to acquire embedding representations on the denoising Markov chain through self-conditional diffusion learning. Intuitively, the encoder, conditioned on the entire diffusion process, compresses high-dimensional data into directional vectors in latent under different noise levels, facilitating the learning of image embeddings across all timesteps. To verify the semantic adequacy of embeddings generated through this approach, extensive experiments are conducted on various datasets, demonstrating that optimally learned embeddings by DDMs surpass state-of-the-art self-supervised representation learning methods in most cases, achieving remarkable discriminative semantic representation quality. Our work justifies that DDMs are not only suitable for generative tasks, but also potentially advantageous for general-purpose deep learning applications.