Training Artificial Neural Networks by Generalized Likelihood Ratio Method: Exploring Brain-like Learning to Improve Robustness
This work addresses the need for more robust neural networks in AI applications by introducing a novel training approach, though it appears incremental as it builds on existing brain-inspired concepts.
The authors tackled the problem of training artificial neural networks with brain-like learning mechanisms, such as discontinuous activation functions, which traditional backpropagation cannot handle, and found that their generalized likelihood ratio method significantly improved robustness against natural noises and adversarial attacks.
In this work, we propose a generalized likelihood ratio method capable of training the artificial neural networks with some biological brain-like mechanisms,.e.g., (a) learning by the loss value, (b) learning via neurons with discontinuous activation and loss functions. The traditional back propagation method cannot train the artificial neural networks with aforementioned brain-like learning mechanisms. Numerical results show that the robustness of various artificial neural networks trained by the new method is significantly improved when the input data is affected by both the natural noises and adversarial attacks. Code is available: \url{https://github.com/LX-doctorAI/GLR_ADV} .