Learning 3D object-centric representation through prediction
This addresses the challenge of unsupervised 3D object perception for AI systems, aiming to mimic human cognitive development, though it appears incremental in combining existing ideas like prediction-based learning with 3D inference.
The paper tackles the problem of learning 3D object-centric representations without supervision, similar to human infants, by developing a novel network architecture that segments objects, infers 3D locations, and perceives depth using only image sequences and self-motion as training data, resulting in object representations emerging as a byproduct of prediction learning.
As part of human core knowledge, the representation of objects is the building block of mental representation that supports high-level concepts and symbolic reasoning. While humans develop the ability of perceiving objects situated in 3D environments without supervision, models that learn the same set of abilities with similar constraints faced by human infants are lacking. Towards this end, we developed a novel network architecture that simultaneously learns to 1) segment objects from discrete images, 2) infer their 3D locations, and 3) perceive depth, all while using only information directly available to the brain as training data, namely: sequences of images and self-motion. The core idea is treating objects as latent causes of visual input which the brain uses to make efficient predictions of future scenes. This results in object representations being learned as an essential byproduct of learning to predict.