A Novel Noise Injection-based Training Scheme for Better Model Robustness
This work addresses robustness issues in neural networks, particularly for spiking neural networks, but appears incremental as it builds on existing noise injection methods with efficiency improvements.
The authors tackled the problem of improving model robustness in neural networks by proposing a novel noise injection-based training scheme that estimates gradients for both weights and noise levels, with an approximation to reduce computational costs. Their method achieved significantly better adversarial robustness and slightly better original accuracy on MNIST and Fashion-MNIST datasets compared to conventional gradient-based training.
Noise injection-based method has been shown to be able to improve the robustness of artificial neural networks in previous work. In this work, we propose a novel noise injection-based training scheme for better model robustness. Specifically, we first develop a likelihood ratio method to estimate the gradient with respect to both synaptic weights and noise levels for stochastic gradient descent training. Then, we design an approximation for the vanilla noise injection-based training method to reduce memory and improve computational efficiency. Next, we apply our proposed scheme to spiking neural networks and evaluate the performance of classification accuracy and robustness on MNIST and Fashion-MNIST datasets. Experiment results show that our proposed method achieves a much better performance on adversarial robustness and slightly better performance on original accuracy, compared with the conventional gradient-based training method.