T-Graphormer: Using Transformers for Spatiotemporal Forecasting
This addresses forecasting challenges in domains like traffic prediction, but it is incremental as it builds on existing Graphormer and Transformer architectures.
The paper tackles spatiotemporal forecasting by introducing T-Graphormer, a Transformer-based method that models spatiotemporal correlations simultaneously, achieving up to 20% reduction in RMSE and 10% reduction in MAPE on traffic prediction benchmarks.
Spatiotemporal data is ubiquitous, and forecasting it has important applications in many domains. However, its complex cross-component dependencies and non-linear temporal dynamics can be challenging for traditional techniques. Existing methods address this by learning the two dimensions separately. Here, we introduce Temporal Graphormer (T-Graphormer), a Transformer-based approach capable of modelling spatiotemporal correlations simultaneously. By adding temporal encodings in the Graphormer architecture, each node attends to all other tokens within the graph sequence, enabling the model to learn rich spacetime patterns with minimal predefined inductive biases. We show the effectiveness of T-Graphormer on real-world traffic prediction benchmark datasets. Compared to state-of-the-art methods, T-Graphormer reduces root mean squared error (RMSE) and mean absolute percentage error (MAPE) by up to 20% and 10%.