Single neuron-based neural networks are as efficient as dense deep neural networks in binary and multi-class recognition problems
This work addresses the challenge of designing more efficient and easier-to-train neural networks for AI recognition problems, potentially reducing computational costs, but it appears incremental as it builds on prior neuroscience findings and existing dense network comparisons.
The paper tackled the problem of modeling high-dimensional recognition tasks using sparse networks of single or few neurons, finding that such networks can be as efficient as dense deep neural networks in binary and multi-class recognition, with single neuron networks showing superior performance in binary classification and competitive results in multi-class tasks.
Recent advances in neuroscience have revealed many principles about neural processing. In particular, many biological systems were found to reconfigure/recruit single neurons to generate multiple kinds of decisions. Such findings have the potential to advance our understanding of the design and optimization process of artificial neural networks. Previous work demonstrated that dense neural networks are needed to shape complex decision surfaces required for AI-level recognition tasks. We investigate the ability to model high dimensional recognition problems using single or several neurons networks that are relatively easier to train. By employing three datasets, we test the use of a population of single neuron networks in performing multi-class recognition tasks. Surprisingly, we find that sparse networks can be as efficient as dense networks in both binary and multi-class tasks. Moreover, single neuron networks demonstrate superior performance in binary classification scheme and competing results when combined for multi-class recognition.