Unsupervised Decomposition and Recombination with Discriminator-Driven Diffusion Models
This work addresses the problem of learning reusable components from data without supervision for applications in image synthesis and robotic video generation, representing an incremental improvement over prior methods.
The paper tackles unsupervised decomposition and recombination of complex data into factorized representations using diffusion models, achieving lower FID scores and better disentanglement metrics like MIG and MCC on datasets such as CelebA-HQ and Virtual KITTI, and demonstrates increased state-space coverage for robotic exploration on the LIBERO benchmark.
Decomposing complex data into factorized representations can reveal reusable components and enable synthesizing new samples via component recombination. We investigate this in the context of diffusion-based models that learn factorized latent spaces without factor-level supervision. In images, factors can capture background, illumination, and object attributes; in robotic videos, they can capture reusable motion components. To improve both latent factor discovery and quality of compositional generation, we introduce an adversarial training signal via a discriminator trained to distinguish between single-source samples and those generated by recombining factors across sources. By optimizing the generator to fool this discriminator, we encourage physical and semantic consistency in the resulting recombinations. Our method outperforms implementations of prior baselines on CelebA-HQ, Virtual KITTI, CLEVR, and Falcor3D, achieving lower FID scores and better disentanglement as measured by MIG and MCC. Furthermore, we demonstrate a novel application to robotic video trajectories: by recombining learned action components, we generate diverse sequences that significantly increase state-space coverage for exploration on the LIBERO benchmark.