Parameter efficient dendritic-tree neurons outperform perceptrons
This work addresses the need for more powerful and parameter-efficient neural network components for machine learning practitioners, though it is incremental as it builds on known biological insights.
The paper tackled the problem of enhancing artificial perceptrons by incorporating biologically inspired dendritic branching and input dropout, resulting in improved accuracy and generalization on MNIST classification compared to existing perceptron architectures.
Biological neurons are more powerful than artificial perceptrons, in part due to complex dendritic input computations. Inspired to empower the perceptron with biologically inspired features, we explore the effect of adding and tuning input branching factors along with input dropout. This allows for parameter efficient non-linear input architectures to be discovered and benchmarked. Furthermore, we present a PyTorch module to replace multi-layer perceptron layers in existing architectures. Our initial experiments on MNIST classification demonstrate the accuracy and generalization improvement of dendritic neurons compared to existing perceptron architectures.