Max and Coincidence Neurons in Neural Networks
This work addresses the need for more compact and efficient neural networks for machine learning practitioners, though it appears incremental by building on existing neural architecture search methods.
The paper tackled the problem of neural network design by incorporating max and coincidence neurons with specific processing functions, achieving an average 2% accuracy improvement and 25% reduction in network size across various datasets.
Network design has been a central topic in machine learning. Large amounts of effort have been devoted towards creating efficient architectures through manual exploration as well as automated neural architecture search. However, todays architectures have yet to consider the diversity of neurons and the existence of neurons with specific processing functions. In this work, we optimize networks containing models of the max and coincidence neurons using neural architecture search, and analyze the structure, operations, and neurons of optimized networks to develop a signal-processing ResNet. The developed network achieves an average of 2% improvement in accuracy and a 25% improvement in network size across a variety of datasets, demonstrating the importance of neuronal functions in creating compact, efficient networks.