SlotDiffusion: Object-Centric Generative Modeling with Diffusion Models
This work addresses the challenge of high-quality visual generation for object-centric learning, enabling better controllable image generation and object manipulation, though it is incremental as it builds on existing diffusion models.
The paper tackles the problem of blurry images and distorted objects in slot-based generative modeling by introducing SlotDiffusion, an object-centric Latent Diffusion Model that improves slot-to-image decoding, resulting in superior performance in unsupervised object segmentation and visual generation across six datasets.
Object-centric learning aims to represent visual data with a set of object entities (a.k.a. slots), providing structured representations that enable systematic generalization. Leveraging advanced architectures like Transformers, recent approaches have made significant progress in unsupervised object discovery. In addition, slot-based representations hold great potential for generative modeling, such as controllable image generation and object manipulation in image editing. However, current slot-based methods often produce blurry images and distorted objects, exhibiting poor generative modeling capabilities. In this paper, we focus on improving slot-to-image decoding, a crucial aspect for high-quality visual generation. We introduce SlotDiffusion -- an object-centric Latent Diffusion Model (LDM) designed for both image and video data. Thanks to the powerful modeling capacity of LDMs, SlotDiffusion surpasses previous slot models in unsupervised object segmentation and visual generation across six datasets. Furthermore, our learned object features can be utilized by existing object-centric dynamics models, improving video prediction quality and downstream temporal reasoning tasks. Finally, we demonstrate the scalability of SlotDiffusion to unconstrained real-world datasets such as PASCAL VOC and COCO, when integrated with self-supervised pre-trained image encoders.