Loci-Segmented: Improving Scene Segmentation Learning
This addresses scene segmentation for computer vision applications, offering a novel method that could serve as a foundation model for downstream tasks.
The paper tackles the problem of compositional scene segmentation from images and videos without requiring background information or slot assignments, and it achieves largely superior video decomposition performance on datasets like MOVi.
Current slot-oriented approaches for compositional scene segmentation from images and videos rely on provided background information or slot assignments. We present a segmented location and identity tracking system, Loci-Segmented (Loci-s), which does not require either of this information. It learns to dynamically segment scenes into interpretable background and slot-based object encodings, separating rgb, mask, location, and depth information for each. The results reveal largely superior video decomposition performance in the MOVi datasets and in another established dataset collection targeting scene segmentation. The system's well-interpretable, compositional latent encodings may serve as a foundation model for downstream tasks.