Learning Object Permanence from Videos via Latent Imaginations
This addresses the challenge of enabling AI systems to understand physical concepts like object permanence without supervision, which is incremental as it builds on existing slot-based methods but introduces a novel loop for learning from videos.
The paper tackles the problem of deep learning models failing to recognize object permanence by introducing Loci-Looped, a slot-based autoregressive system that learns to track objects through occlusions and anticipate reappearances from video data alone, outperforming state-of-the-art baselines in handling occlusions and sensory interruptions.
While human infants exhibit knowledge about object permanence from two months of age onwards, deep-learning approaches still largely fail to recognize objects' continued existence. We introduce a slot-based autoregressive deep learning system, the looped location and identity tracking model Loci-Looped, which learns to adaptively fuse latent imaginations with pixel-space observations into consistent latent object-specific what and where encodings over time. The novel loop empowers Loci-Looped to learn the physical concepts of object permanence, directional inertia, and object solidity through observation alone. As a result, Loci-Looped tracks objects through occlusions, anticipates their reappearance, and shows signs of surprise and internal revisions when observing implausible object behavior. Notably, Loci-Looped outperforms state-of-the-art baseline models in handling object occlusions and temporary sensory interruptions while exhibiting more compositional, interpretable internal activity patterns. Our work thus introduces the first self-supervised interpretable learning model that learns about object permanence directly from video data without supervision.