Learning Topology and Dynamics of Large Recurrent Neural Networks
This work addresses the challenge of learning complex, nonlinear dynamics in large-scale networks, which is incremental as it builds on existing sparse regression methods by incorporating stability constraints.
The paper tackles the problem of identifying connections and estimating parameters in large recurrent neural networks from noisy, limited observations, and proposes algorithms with rigorous convergence guarantees that show excellent performance in topology identification and forecasting.
Large-scale recurrent networks have drawn increasing attention recently because of their capabilities in modeling a large variety of real-world phenomena and physical mechanisms. This paper studies how to identify all authentic connections and estimate system parameters of a recurrent network, given a sequence of node observations. This task becomes extremely challenging in modern network applications, because the available observations are usually very noisy and limited, and the associated dynamical system is strongly nonlinear. By formulating the problem as multivariate sparse sigmoidal regression, we develop simple-to-implement network learning algorithms, with rigorous convergence guarantee in theory, for a variety of sparsity-promoting penalty forms. A quantile variant of progressive recurrent network screening is proposed for efficient computation and allows for direct cardinality control of network topology in estimation. Moreover, we investigate recurrent network stability conditions in Lyapunov's sense, and integrate such stability constraints into sparse network learning. Experiments show excellent performance of the proposed algorithms in network topology identification and forecasting.