Efficient Time-Series Approximation with Linear Recurrent Neural Networks: Architecture Learning and Predictive Power
This provides an efficient method for time-series approximation with reduced computational cost, though it appears incremental as it builds on linear recurrent networks.
The paper tackles the problem of approximating time-dependent functions using autoregressive linear recurrent neural networks (LRNNs), showing they can be learned by solving linear equations without backpropagation and reduced in size by analyzing eigenvalues, and demonstrates they outperform state-of-the-art on the MSO task with minimal units.
Recurrent neural networks are a powerful means to cope with time series. We show how autoregressive linear, i.e., linearly activated recurrent neural networks (LRNNs) can approximate any time-dependent function f(t). The approximation can effectively be learned by simply solving a linear equation system; no backpropagation or similar methods are needed. Furthermore, and this is the main contribution of this paper, the size of an LRNN can be reduced significantly in one step after inspecting the spectrum of the network transition matrix, i.e., its eigenvalues, by taking only the most relevant components. Therefore, in contrast to other approaches, we do not only learn network weights but also the network architecture. LRNNs have interesting properties: They end up in ellipse trajectories in the long run and allow the prediction of further values and compact representations of functions. We demonstrate this by several case studies, among them multiple superimposed oscillators (MSO), robotic soccer (RoboCup), and stock price prediction. LRNNs outperform the previous state-of-the-art for the MSO task with a minimal number of units.