Memory-Gated Recurrent Networks
This work provides an improved recurrent network architecture for researchers and practitioners working with complex multivariate time series data, offering better performance in capturing intricate dependencies.
This paper addresses the challenge of extracting dependencies in multivariate sequential data, which often exhibit both serial dependencies within components (marginal memory) and cross-sectional dependencies (joint memory). The authors propose Memory-Gated Recurrent Networks (mGRN) to explicitly regulate these two types of memories, demonstrating consistent outperformance against state-of-the-art architectures on various public datasets.
The essence of multivariate sequential learning is all about how to extract dependencies in data. These data sets, such as hourly medical records in intensive care units and multi-frequency phonetic time series, often time exhibit not only strong serial dependencies in the individual components (the "marginal" memory) but also non-negligible memories in the cross-sectional dependencies (the "joint" memory). Because of the multivariate complexity in the evolution of the joint distribution that underlies the data generating process, we take a data-driven approach and construct a novel recurrent network architecture, termed Memory-Gated Recurrent Networks (mGRN), with gates explicitly regulating two distinct types of memories: the marginal memory and the joint memory. Through a combination of comprehensive simulation studies and empirical experiments on a range of public datasets, we show that our proposed mGRN architecture consistently outperforms state-of-the-art architectures targeting multivariate time series.