Fodor and Pylyshyn's Legacy -- Still No Human-like Systematic Compositionality in Neural Networks
This is an incremental position paper that challenges optimistic claims about neural networks' ability to model human-like compositionality, relevant for researchers in cognitive science and AI.
The paper critically revisits claims that meta-learning enables neural networks to achieve human-like systematic compositionality, concluding that such capabilities are only possible under narrow definitions and that neural networks still lack this capacity.
Strong meta-learning capabilities for systematic compositionality are emerging as an important skill for navigating the complex and changing tasks of today's world. However, in presenting models for robust adaptation to novel environments, it is important to refrain from making unsupported claims about the performance of meta-learning systems that ultimately do not stand up to scrutiny. While Fodor and Pylyshyn famously posited that neural networks inherently lack this capacity as they are unable to model compositional representations or structure-sensitive operations, and thus are not a viable model of the human mind, Lake and Baroni recently presented meta-learning as a pathway to compositionality. In this position paper, we critically revisit this claim and highlight limitations in the proposed meta-learning framework for compositionality. Our analysis shows that modern neural meta-learning systems can only perform such tasks, if at all, under a very narrow and restricted definition of a meta-learning setup. We therefore claim that `Fodor and Pylyshyn's legacy' persists, and to date, there is no human-like systematic compositionality learned in neural networks.