GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs
This work addresses graph learning challenges for applications like traffic forecasting, though it appears incremental as it builds upon existing attention mechanisms.
The authors tackled the problem of learning on large and spatiotemporal graphs by proposing Gated Attention Networks (GaAN), which uses a convolutional sub-network to gate attention heads, achieving state-of-the-art results on inductive node classification and traffic speed forecasting across three real-world datasets.
We propose a new network architecture, Gated Attention Networks (GaAN), for learning on graphs. Unlike the traditional multi-head attention mechanism, which equally consumes all attention heads, GaAN uses a convolutional sub-network to control each attention head's importance. We demonstrate the effectiveness of GaAN on the inductive node classification problem. Moreover, with GaAN as a building block, we construct the Graph Gated Recurrent Unit (GGRU) to address the traffic speed forecasting problem. Extensive experiments on three real-world datasets show that our GaAN framework achieves state-of-the-art results on both tasks.