Revealing the Power of Masked Autoencoders in Traffic Forecasting
This work addresses data scarcity and model stability issues in traffic forecasting for urban planning, but it is incremental as it builds upon existing spatial-temporal models.
The authors tackled the problem of traffic forecasting by proposing a plug-and-play framework called Spatial-Temporal Masked AutoEncoders (STMAE) to enhance existing models, resulting in significant improvements in forecasting accuracy on traffic benchmarks.
Traffic forecasting, crucial for urban planning, requires accurate predictions of spatial-temporal traffic patterns across urban areas. Existing research mainly focuses on designing complex models that capture spatial-temporal dependencies among variables explicitly. However, this field faces challenges related to data scarcity and model stability, which results in limited performance improvement. To address these issues, we propose Spatial-Temporal Masked AutoEncoders (STMAE), a plug-and-play framework designed to enhance existing spatial-temporal models on traffic prediction. STMAE consists of two learning stages. In the pretraining stage, an encoder processes partially visible traffic data produced by a dual-masking strategy, including biased random walk-based spatial masking and patch-based temporal masking. Subsequently, two decoders aim to reconstruct the masked counterparts from both spatial and temporal perspectives. The fine-tuning stage retains the pretrained encoder and integrates it with decoders from existing backbones to improve forecasting accuracy. Our results on traffic benchmarks show that STMAE can largely enhance the forecasting capabilities of various spatial-temporal models.