Improving Neural Networks by Adopting Amplifying and Attenuating Neurons
This work addresses performance enhancement for neural network practitioners, but it appears incremental as it builds on existing architectures with simple modifications.
The authors tackled the problem of improving neural network performance by proposing amplifying and attenuating neurons, which are easy to implement with minimal computational overhead. The result showed that networks with these neurons yield more accurate results compared to those without them, as verified through numerical experiments.
In the present study, an amplifying neuron and attenuating neuron, which can be easily implemented into neural networks without any significant additional computational effort, are proposed. The activated output value is squared for the amplifying neuron, while the value becomes its reciprocal for the attenuating one. Theoretically, the order of neural networks increases when the amplifying neuron is placed in the hidden layer. The performance assessments of neural networks were conducted to verify that the amplifying and attenuating neurons enhance the performance of neural networks. From the numerical experiments, it was revealed that the neural networks that contain the amplifying and attenuating neurons yield more accurate results, compared to those without them.