Recurrent Graph Tensor Networks: A Low-Complexity Framework for Modelling High-Dimensional Multi-Way Sequence
This addresses the curse of dimensionality in sequence modelling for applications dealing with large multidimensional data, though it appears incremental as it builds on existing RNN and tensor network concepts.
The authors tackled the problem of exponential parameter growth in Recurrent Neural Networks (RNNs) when handling high-dimensional multi-way sequences, and developed a Recurrent Graph Tensor Network (RGTN) that outperforms standard RNNs while reducing complexity.
Recurrent Neural Networks (RNNs) are among the most successful machine learning models for sequence modelling, but tend to suffer from an exponential increase in the number of parameters when dealing with large multidimensional data. To this end, we develop a multi-linear graph filter framework for approximating the modelling of hidden states in RNNs, which is embedded in a tensor network architecture to improve modelling power and reduce parameter complexity, resulting in a novel Recurrent Graph Tensor Network (RGTN). The proposed framework is validated through several multi-way sequence modelling tasks and benchmarked against traditional RNNs. By virtue of the domain aware information processing of graph filters and the expressive power of tensor networks, we show that the proposed RGTN is capable of not only out-performing standard RNNs, but also mitigating the Curse of Dimensionality associated with traditional RNNs, demonstrating superior properties in terms of performance and complexity.