Relational reasoning and generalization using non-symbolic neural networks
This addresses the problem of understanding the capabilities of neural networks for abstract reasoning, challenging previous claims that they lack symbolic abilities, though it is incremental in revisiting and extending prior work.
The study tackled the problem of whether neural networks can represent and generalize the concept of equality, a basic form of relational reasoning, and found that they can learn mathematical identity, handle sequential equality patterns with only positive examples, and achieve zero-shot generalization to complex hierarchical tasks.
The notion of equality (identity) is simple and ubiquitous, making it a key case study for broader questions about the representations supporting abstract relational reasoning. Previous work suggested that neural networks were not suitable models of human relational reasoning because they could not represent mathematically identity, the most basic form of equality. We revisit this question. In our experiments, we assess out-of-sample generalization of equality using both arbitrary representations and representations that have been pretrained on separate tasks to imbue them with structure. We find neural networks are able to learn (1) basic equality (mathematical identity), (2) sequential equality problems (learning ABA-patterned sequences) with only positive training instances, and (3) a complex, hierarchical equality problem with only basic equality training instances ("zero-shot'" generalization). In the two latter cases, our models perform tasks proposed in previous work to demarcate human-unique symbolic abilities. These results suggest that essential aspects of symbolic reasoning can emerge from data-driven, non-symbolic learning processes.