ST-ReP: Learning Predictive Representations Efficiently for Spatial-Temporal Forecasting
This work addresses efficiency and scalability issues in spatial-temporal forecasting for domains like traffic and energy, but it is incremental as it builds on existing self-supervised methods.
The paper tackled challenges in self-supervised learning for spatial-temporal forecasting, such as difficulty in selecting negative pairs and overlooking spatial correlations, by proposing ST-ReP, a lightweight model that integrates reconstruction and prediction into pre-training; experimental results showed it surpasses baselines in learning compact and semantically enriched representations with superior scalability.
Spatial-temporal forecasting is crucial and widely applicable in various domains such as traffic, energy, and climate. Benefiting from the abundance of unlabeled spatial-temporal data, self-supervised methods are increasingly adapted to learn spatial-temporal representations. However, it encounters three key challenges: 1) the difficulty in selecting reliable negative pairs due to the homogeneity of variables, hindering contrastive learning methods; 2) overlooking spatial correlations across variables over time; 3) limitations of efficiency and scalability in existing self-supervised learning methods. To tackle these, we propose a lightweight representation-learning model ST-ReP, integrating current value reconstruction and future value prediction into the pre-training framework for spatial-temporal forecasting. And we design a new spatial-temporal encoder to model fine-grained relationships. Moreover, multi-time scale analysis is incorporated into the self-supervised loss to enhance predictive capability. Experimental results across diverse domains demonstrate that the proposed model surpasses pre-training-based baselines, showcasing its ability to learn compact and semantically enriched representations while exhibiting superior scalability.