Prime and Modulate Learning: Generation of forward models with signed back-propagation and environmental cues
This work addresses gradient issues in closed-loop learning for robotics, offering an incremental method that avoids normalization and leverages environmental relevance.
The paper tackles the exploding and vanishing gradient problem in deep neural networks by introducing a Prime and Modulate paradigm that uses the sign of the error signal for priming and environmental cues for modulation, resulting in a significant improvement in convergence speed compared to conventional back-propagation.
Deep neural networks employing error back-propagation for learning can suffer from exploding and vanishing gradient problems. Numerous solutions have been proposed such as normalisation techniques or limiting activation functions to linear rectifying units. In this work we follow a different approach which is particularly applicable to closed-loop learning of forward models where back-propagation makes exclusive use of the sign of the error signal to prime the learning, whilst a global relevance signal modulates the rate of learning. This is inspired by the interaction between local plasticity and a global neuromodulation. For example, whilst driving on an empty road, one can allow for slow step-wise optimisation of actions, whereas, at a busy junction, an error must be corrected at once. Hence, the error is the priming signal and the intensity of the experience is a modulating factor in the weight change. The advantages of this Prime and Modulate paradigm is twofold: it is free from normalisation and it makes use of relevant cues from the environment to enrich the learning. We present a mathematical derivation of the learning rule in z-space and demonstrate the real-time performance with a robotic platform. The results show a significant improvement in the speed of convergence compared to that of the conventional back-propagation.