Structure Learning for Neural Module Networks
This work addresses the challenge of automating module design in neural networks for tasks like visual question answering, but it is incremental as it builds on existing frameworks.
The paper tackled the problem of learning the internal structure of neural modules in Neural Module Networks, which are typically human-specified, by simultaneously learning both the module structure and sequencing without extra supervision, achieving performance comparable to hand-designed modules.
Neural Module Networks, originally proposed for the task of visual question answering, are a class of neural network architectures that involve human-specified neural modules, each designed for a specific form of reasoning. In current formulations of such networks only the parameters of the neural modules and/or the order of their execution is learned. In this work, we further expand this approach and also learn the underlying internal structure of modules in terms of the ordering and combination of simple and elementary arithmetic operators. Our results show that one is indeed able to simultaneously learn both internal module structure and module sequencing without extra supervisory signals for module execution sequencing. With this approach, we report performance comparable to models using hand-designed modules.