Towards Improving the Generation Quality of Autoregressive Slot VAEs
This work addresses the challenge of generating coherent multi-object scenes without conditioning, which is important for applications in computer vision and robotics, though it is incremental as it builds on existing slot-based models.
The paper tackled the problem of poor unconditional scene generation in slot-based models by strengthening object correlation learning through a global scene-level variable and a learned consistent order for autoregressive generation, resulting in clear gains in generation quality across three multi-object environments.
Unconditional scene inference and generation are challenging to learn jointly with a single compositional model. Despite encouraging progress on models that extract object-centric representations (''slots'') from images, unconditional generation of scenes from slots has received less attention. This is primarily because learning the multi-object relations necessary to imagine coherent scenes is difficult. We hypothesize that most existing slot-based models have a limited ability to learn object correlations. We propose two improvements that strengthen object correlation learning. The first is to condition the slots on a global, scene-level variable that captures higher-order correlations between slots. Second, we address the fundamental lack of a canonical order for objects in images by proposing to learn a consistent order to use for the autoregressive generation of scene objects. Specifically, we train an autoregressive slot prior to sequentially generate scene objects following a learned order. Ordered slot inference entails first estimating a randomly ordered set of slots using existing approaches for extracting slots from images, then aligning those slots to ordered slots generated autoregressively with the slot prior. Our experiments across three multi-object environments demonstrate clear gains in unconditional scene generation quality. Detailed ablation studies are also provided that validate the two proposed improvements.