Improved robustness of reinforcement learning policies upon conversion to spiking neuronal network platforms applied to ATARI games
This addresses robustness issues in reinforcement learning for applications like gaming, though it is incremental as it extends existing conversion methods to a new domain.
The paper tackles the sensitivity of deep reinforcement learning to noisy or incomplete input data by converting standard neural networks to spiking neural networks, showing improved robustness to occlusion in ATARI games with a proof-of-principle demonstration.
Deep Reinforcement Learning (RL) demonstrates excellent performance on tasks that can be solved by trained policy. It plays a dominant role among cutting-edge machine learning approaches using multi-layer Neural networks (NNs). At the same time, Deep RL suffers from high sensitivity to noisy, incomplete, and misleading input data. Following biological intuition, we involve Spiking Neural Networks (SNNs) to address some deficiencies of deep RL solutions. Previous studies in image classification domain demonstrated that standard NNs (with ReLU nonlinearity) trained using supervised learning can be converted to SNNs with negligible deterioration in performance. In this paper, we extend those conversion results to the domain of Q-Learning NNs trained using RL. We provide a proof of principle of the conversion of standard NN to SNN. In addition, we show that the SNN has improved robustness to occlusion in the input image. Finally, we introduce results with converting full-scale Deep Q-network to SNN, paving the way for future research to robust Deep RL applications.