Successes and Limitations of Object-centric Models at Compositional Generalisation
This addresses the challenge of robust compositional learning in AI for visual tasks, though it is incremental in extending prior work.
The paper tackles the problem of compositional generalization in visual models, showing that object-centric architectures can generalize to novel combinations of object properties, not just scenes, and identifies sources of these skills and limitations.
In recent years, it has been shown empirically that standard disentangled latent variable models do not support robust compositional learning in the visual domain. Indeed, in spite of being designed with the goal of factorising datasets into their constituent factors of variations, disentangled models show extremely limited compositional generalisation capabilities. On the other hand, object-centric architectures have shown promising compositional skills, albeit these have 1) not been extensively tested and 2) experiments have been limited to scene composition -- where models must generalise to novel combinations of objects in a visual scene instead of novel combinations of object properties. In this work, we show that these compositional generalisation skills extend to this later setting. Furthermore, we present evidence pointing to the source of these skills and how they can be improved through careful training. Finally, we point to one important limitation that still exists which suggests new directions of research.