Complex Gated Recurrent Neural Networks
This work addresses the challenge of improving RNN stability and convergence for sequence modeling, though it appears incremental as it builds on existing gated RNN architectures with complex-valued adaptations.
The paper tackles the problem of integrating complex numbers into deep learning for time series processing by introducing a complex gated recurrent cell, which shows competitive performance on synthetic memory tasks and human motion prediction.
Complex numbers have long been favoured for digital signal processing, yet complex representations rarely appear in deep learning architectures. RNNs, widely used to process time series and sequence information, could greatly benefit from complex representations. We present a novel complex gated recurrent cell, which is a hybrid cell combining complex-valued and norm-preserving state transitions with a gating mechanism. The resulting RNN exhibits excellent stability and convergence properties and performs competitively on the synthetic memory and adding task, as well as on the real-world tasks of human motion prediction.