Generalising E-prop to Deep Networks
This work addresses the challenge of biologically plausible online learning in deep recurrent networks, which is incremental as it builds on existing E-prop methods.
The authors tackled the problem of training deep recurrent networks without backpropagation through time by extending the E-prop framework to handle arbitrarily deep networks, deriving a novel recursion relationship across depth that allows online learning with eligibility traces. Their result demonstrates an algorithm that can perform accurate credit assignment across both time and depth simultaneously, enabling training of deep recurrent networks without backpropagation through time.
Recurrent networks are typically trained with backpropagation through time (BPTT). However, BPTT requires storing the history of all states in the network and then replaying them sequentially backwards in time. This computation appears extremely implausible for the brain to implement. Real Time Recurrent Learning (RTRL) proposes an mathematically equivalent alternative where gradient information is propagated forwards in time locally alongside the regular forward pass, however it has significantly greater computational complexity than BPTT which renders it impractical for large networks. E-prop proposes an approximation of RTRL which reduces its complexity to the level of BPTT while maintaining a purely online forward update which can be implemented by an eligibility trace at each synapse. However, works on RTRL and E-prop ubiquitously investigate learning in a single layer with recurrent dynamics. However, learning in the brain spans multiple layers and consists of both hierarchal dynamics in depth as well as time. In this mathematical note, we extend the E-prop framework to handle arbitrarily deep networks, deriving a novel recursion relationship across depth which extends the eligibility traces of E-prop to deeper layers. Our results thus demonstrate an online learning algorithm can perform accurate credit assignment across both time and depth simultaneously, allowing the training of deep recurrent networks without backpropagation through time.