Beyond Weighted Summation: Learnable Nonlinear Aggregation Functions for Robust Artificial Neurons
This work addresses robustness to noise in neural networks for AI applications, but it is incremental as it builds on existing neuron designs with novel aggregation mechanisms.
The paper tackled the problem of artificial neurons being sensitive to noisy inputs due to their default weighted summation aggregation, by introducing learnable nonlinear aggregation functions like F-Mean and Gaussian Support neurons. The result was improved robustness under noise, with a three-way hybrid achieving scores up to 0.991 compared to 0.890 for the baseline, and modest gains on clean data.
Weighted summation has remained the default input aggregation mechanism in artificial neurons since the earliest neural network models. While computationally efficient, this design implicitly behaves like a mean-based estimator and is therefore sensitive to noisy or extreme inputs. This paper investigates whether replacing fixed linear aggregation with learnable nonlinear alternatives can improve neural network robustness without sacrificing trainability. Two differentiable aggregation mechanisms are introduced: an F-Mean neuron based on a learnable power-weighted aggregation rule, and a Gaussian Support neuron based on distance-aware affinity weighting. To preserve the optimisation stability of standard neurons, hybrid neurons are proposed that interpolate between linear and nonlinear aggregation through a learnable blending parameter. Evaluated in multilayer perceptrons and convolutional neural networks on CIFAR-10 and a noisy CIFAR-10 variant with additive Gaussian corruption, hybrid neurons consistently improve robustness under noise while F-Mean hybrids also yield modest gains on clean data. The three-way hybrid achieves robustness scores of up to 0.991 compared to 0.890 for the standard baseline, and learned parameters converge consistently to sub-linear aggregation (p $\approx$ 0.43--0.50) and high novelty utilisation ($α$ $\approx$ 0.69--0.79). These findings suggest that neuron-level aggregation is a meaningful and underexplored design dimension for building more noise-tolerant neural networks.