ClusterDDPM: An EM clustering framework with Denoising Diffusion Probabilistic Models
This work addresses clustering challenges for machine learning practitioners by offering a novel method that leverages DDPMs, though it appears incremental as it adapts an existing EM framework with a new generative model.
The authors tackled clustering by proposing an expectation-maximization framework using denoising diffusion probabilistic models, achieving superior performance in clustering, unsupervised conditional generation, and latent representation learning as validated by comprehensive experiments.
Variational autoencoder (VAE) and generative adversarial networks (GAN) have found widespread applications in clustering and have achieved significant success. However, the potential of these approaches may be limited due to VAE's mediocre generation capability or GAN's well-known instability during adversarial training. In contrast, denoising diffusion probabilistic models (DDPMs) represent a new and promising class of generative models that may unlock fresh dimensions in clustering. In this study, we introduce an innovative expectation-maximization (EM) framework for clustering using DDPMs. In the E-step, we aim to derive a mixture of Gaussian priors for the subsequent M-step. In the M-step, our focus lies in learning clustering-friendly latent representations for the data by employing the conditional DDPM and matching the distribution of latent representations to the mixture of Gaussian priors. We present a rigorous theoretical analysis of the optimization process in the M-step, proving that the optimizations are equivalent to maximizing the lower bound of the Q function within the vanilla EM framework under certain constraints. Comprehensive experiments validate the advantages of the proposed framework, showcasing superior performance in clustering, unsupervised conditional generation and latent representation learning.