OC-NMN: Object-centric Compositional Neural Module Network for Generative Visual Analogical Reasoning
This addresses the challenge of compositional generalization in visual reasoning for AI systems, though it is incremental as it builds on existing neural module networks.
The paper tackles the problem of enabling machine learning systems to perform generative visual analogical reasoning by composing learned concepts in novel ways, and shows that their modular architecture improves out-of-distribution generalization on a benchmark involving arithmetic operations on MNIST digits.
A key aspect of human intelligence is the ability to imagine -- composing learned concepts in novel ways -- to make sense of new scenarios. Such capacity is not yet attained for machine learning systems. In this work, in the context of visual reasoning, we show how modularity can be leveraged to derive a compositional data augmentation framework inspired by imagination. Our method, denoted Object-centric Compositional Neural Module Network (OC-NMN), decomposes visual generative reasoning tasks into a series of primitives applied to objects without using a domain-specific language. We show that our modular architectural choices can be used to generate new training tasks that lead to better out-of-distribution generalization. We compare our model to existing and new baselines in proposed visual reasoning benchmark that consists of applying arithmetic operations to MNIST digits.