On Numerosity of Deep Neural Networks
This work addresses a critical issue in machine learning and cognitive science by correcting a misleading claim about emergent abilities in neural networks, highlighting challenges in achieving human-like abstract reasoning.
The paper disproves a prior claim that number sense emerges spontaneously in deep neural networks trained for object recognition, showing the statistical analysis was flawed and could produce false positives in untrained networks. It finds that even when explicitly trained on numerosity, deep networks struggle to acquire abstract number concepts but show some robustness to distribution shifts for small numbers.
Recently, a provocative claim was published that number sense spontaneously emerges in a deep neural network trained merely for visual object recognition. This has, if true, far reaching significance to the fields of machine learning and cognitive science alike. In this paper, we prove the above claim to be unfortunately incorrect. The statistical analysis to support the claim is flawed in that the sample set used to identify number-aware neurons is too small, compared to the huge number of neurons in the object recognition network. By this flawed analysis one could mistakenly identify number-sensing neurons in any randomly initialized deep neural networks that are not trained at all. With the above critique we ask the question what if a deep convolutional neural network is carefully trained for numerosity? Our findings are mixed. Even after being trained with number-depicting images, the deep learning approach still has difficulties to acquire the abstract concept of numbers, a cognitive task that preschoolers perform with ease. But on the other hand, we do find some encouraging evidences suggesting that deep neural networks are more robust to distribution shift for small numbers than for large numbers.