Improving Systematic Generalization Through Modularity and Augmentation
This work addresses the problem of systematic generalization for AI systems aiming to learn more like humans, though it is incremental as it builds on existing principles like modularity and augmentation in a controlled synthetic setting.
The paper tackled systematic generalization in neural networks for grounded language learning, finding that modular networks with extensive data augmentation achieved up to 70% and 40% exact match improvements over state-of-the-art on specific gSCAN tests, while non-modular baselines failed to generalize even with large augmented datasets.
Systematic generalization is the ability to combine known parts into novel meaning; an important aspect of efficient human learning, but a weakness of neural network learning. In this work, we investigate how two well-known modeling principles -- modularity and data augmentation -- affect systematic generalization of neural networks in grounded language learning. We analyze how large the vocabulary needs to be to achieve systematic generalization and how similar the augmented data needs to be to the problem at hand. Our findings show that even in the controlled setting of a synthetic benchmark, achieving systematic generalization remains very difficult. After training on an augmented dataset with almost forty times more adverbs than the original problem, a non-modular baseline is not able to systematically generalize to a novel combination of a known verb and adverb. When separating the task into cognitive processes like perception and navigation, a modular neural network is able to utilize the augmented data and generalize more systematically, achieving 70% and 40% exact match increase over state-of-the-art on two gSCAN tests that have not previously been improved. We hope that this work gives insight into the drivers of systematic generalization, and what we still need to improve for neural networks to learn more like humans do.