Slot-Guided Adaptation of Pre-trained Diffusion Models for Object-Centric Learning and Compositional Generation
This addresses the need for better object-centric generative models in AI, particularly for compositional image generation, though it is incremental as it builds on existing slot attention and diffusion methods.
The paper tackles the problem of object-centric learning and compositional generation by adapting pretrained diffusion models with slot attention, avoiding text-centric bias and enhancing object alignment without external supervision. It outperforms state-of-the-art methods in object discovery and image generation across multiple datasets, including real images, and shows strong performance on complex real-world images for compositional tasks.
We present SlotAdapt, an object-centric learning method that combines slot attention with pretrained diffusion models by introducing adapters for slot-based conditioning. Our method preserves the generative power of pretrained diffusion models, while avoiding their text-centric conditioning bias. We also incorporate an additional guidance loss into our architecture to align cross-attention from adapter layers with slot attention. This enhances the alignment of our model with the objects in the input image without using external supervision. Experimental results show that our method outperforms state-of-the-art techniques in object discovery and image generation tasks across multiple datasets, including those with real images. Furthermore, we demonstrate through experiments that our method performs remarkably well on complex real-world images for compositional generation, in contrast to other slot-based generative methods in the literature. The project page can be found at https://kaanakan.github.io/SlotAdapt/.