Temporal Credit Is Free
This work addresses the computational and memory bottlenecks in training recurrent neural networks for real-time applications, offering a more efficient alternative to traditional methods.
The paper tackles the problem of online adaptation in recurrent networks by showing that immediate derivatives with gradient normalization can match or exceed full RTRL performance, scaling to n = 1024 with 1000x less memory across various architectures and benchmarks.
Recurrent networks do not need Jacobian propagation to adapt online. The hidden state already carries temporal credit through the forward pass; immediate derivatives suffice if you stop corrupting them with stale trace memory and normalize gradient scales across parameter groups. An architectural rule predicts when normalization is needed: \b{eta}2 is required when gradients must pass through a nonlinear state update with no output bypass, and unnecessary otherwise. Across ten architectures, real primate neural data, and streaming ML benchmarks, immediate derivatives with RMSprop match or exceed full RTRL, scaling to n = 1024 at 1000x less memory.