Abrupt and spontaneous strategy switches emerge in simple regularised neural networks
This addresses the problem of understanding the origins of insight-like behavior in AI for cognitive science, showing it can arise from basic learning mechanisms rather than complex processes, though it is incremental in linking neural networks to human cognition.
The study investigated whether insight-like sudden strategy switches, similar to human 'aha-moments', can emerge in simple regularized neural networks trained with gradual gradient descent, and found that these networks closely mimicked human behavioral characteristics such as delay, suddenness, and selective occurrence, with key dependencies on regularized gating and noise.
Humans sometimes have an insight that leads to a sudden and drastic performance improvement on the task they are working on. Sudden strategy adaptations are often linked to insights, considered to be a unique aspect of human cognition tied to complex processes such as creativity or meta-cognitive reasoning. Here, we take a learning perspective and ask whether insight-like behaviour can occur in simple artificial neural networks, even when the models only learn to form input-output associations through gradual gradient descent. We compared learning dynamics in humans and regularised neural networks in a perceptual decision task that included a hidden regularity to solve the task more efficiently. Our results show that only some humans discover this regularity, whose behaviour was marked by a sudden and abrupt strategy switch that reflects an aha-moment. Notably, we find that simple neural networks with a gradual learning rule and a constant learning rate closely mimicked behavioural characteristics of human insight-like switches, exhibiting delay of insight, suddenness and selective occurrence in only some networks. Analyses of network architectures and learning dynamics revealed that insight-like behaviour crucially depended on a regularised gating mechanism and noise added to gradient updates, which allowed the networks to accumulate "silent knowledge" that is initially suppressed by regularised (attentional) gating. This suggests that insight-like behaviour can arise naturally from gradual learning in simple neural networks, where it reflects the combined influences of noise, gating and regularisation.