How Modular Should Neural Module Networks Be for Systematic Generalization?
This work addresses systematic generalization for VQA systems, offering incremental improvements in NMN architectures.
The paper investigates how the degree of modularity in Neural Module Networks (NMNs) affects systematic generalization in Visual Question Answering (VQA), finding that tuning modularity, particularly at the image encoder stage, leads to substantially higher generalization across three VQA datasets.
Neural Module Networks (NMNs) aim at Visual Question Answering (VQA) via composition of modules that tackle a sub-task. NMNs are a promising strategy to achieve systematic generalization, i.e., overcoming biasing factors in the training distribution. However, the aspects of NMNs that facilitate systematic generalization are not fully understood. In this paper, we demonstrate that the degree of modularity of the NMN have large influence on systematic generalization. In a series of experiments on three VQA datasets (VQA-MNIST, SQOOP, and CLEVR-CoGenT), our results reveal that tuning the degree of modularity, especially at the image encoder stage, reaches substantially higher systematic generalization. These findings lead to new NMN architectures that outperform previous ones in terms of systematic generalization.