Dendritic Integration Based Quadratic Neural Networks Outperform Traditional Aritificial Ones
This work addresses the problem of improving computational capabilities in machine learning for researchers and practitioners by introducing a brain-inspired model with theoretical generalization analysis, though it appears incremental as it builds on known biological insights.
The authors tackled the challenge of enhancing artificial neural networks by incorporating biological dendritic quadratic integration, proposing a novel model (DIQNN) that outperforms traditional ANNs in classification tasks, with a low-rank variant reducing computational cost while maintaining performance.
Incorporating biological neuronal properties into Artificial Neural Networks (ANNs) to enhance computational capabilities poses a formidable challenge in the field of machine learning. Inspired by recent findings indicating that dendrites adhere to quadratic integration rules for synaptic inputs, we propose a novel ANN model, Dendritic Integration-Based Quadratic Neural Network (DIQNN). This model shows superior performance over traditional ANNs in a variety of classification tasks. To reduce the computational cost of DIQNN, we introduce the Low-Rank DIQNN, while we find it can retain the performance of the original DIQNN. We further propose a margin to characterize the generalization error and theoretically prove this margin will increase monotonically during training. And we show the consistency between generalization and our margin using numerical experiments. Finally, by integrating this margin into the loss function, the change of test accuracy is indeed accelerated. Our work contributes a novel, brain-inspired ANN model that surpasses traditional ANNs and provides a theoretical framework to analyze the generalization error in classification tasks.