Learning Stochastic Dynamical Systems as an Implicit Regularization with Graph Neural Networks
This work addresses the problem of modeling complex, spatially correlated stochastic systems for researchers in machine learning and dynamical systems, offering an incremental improvement with a novel framework.
The paper tackled learning high-dimensional time series with spatial correlations by proposing Stochastic Gumbel Graph Networks (S-GGNs) to model drift and diffusion terms in stochastic differential equations, and it demonstrated superior convergence, robustness, and generalization compared to state-of-the-art methods in experiments.
Stochastic Gumbel graph networks are proposed to learn high-dimensional time series, where the observed dimensions are often spatially correlated. To that end, the observed randomness and spatial-correlations are captured by learning the drift and diffusion terms of the stochastic differential equation with a Gumble matrix embedding, respectively. In particular, this novel framework enables us to investigate the implicit regularization effect of the noise terms in S-GGNs. We provide a theoretical guarantee for the proposed S-GGNs by deriving the difference between the two corresponding loss functions in a small neighborhood of weight. Then, we employ Kuramoto's model to generate data for comparing the spectral density from the Hessian Matrix of the two loss functions. Experimental results on real-world data, demonstrate that S-GGNs exhibit superior convergence, robustness, and generalization, compared with state-of-the-arts.