CObL: Toward Zero-Shot Ordinal Layering without User Prompting
This addresses the challenge of unsupervised object-centric representation learning for vision tasks, enabling automatic scene decomposition without manual input, though it is incremental as it builds on diffusion models and focuses on tabletop domains.
The paper tackles the problem of inferring occlusion-ordered object layers from images without user prompting or prior knowledge of object count, introducing CObL, a diffusion-based architecture that zero-shot generalizes to real-world tabletop scenes with novel objects, achieving this with only a few thousand synthetic training images.
Vision benefits from grouping pixels into objects and understanding their spatial relationships, both laterally and in depth. We capture this with a scene representation comprising an occlusion-ordered stack of "object layers," each containing an isolated and amodally-completed object. To infer this representation from an image, we introduce a diffusion-based architecture named Concurrent Object Layers (CObL). CObL generates a stack of object layers in parallel, using Stable Diffusion as a prior for natural objects and inference-time guidance to ensure the inferred layers composite back to the input image. We train CObL using a few thousand synthetically-generated images of multi-object tabletop scenes, and we find that it zero-shot generalizes to photographs of real-world tabletops with varying numbers of novel objects. In contrast to recent models for amodal object completion, CObL reconstructs multiple occluded objects without user prompting and without knowing the number of objects beforehand. Unlike previous models for unsupervised object-centric representation learning, CObL is not limited to the world it was trained in.