Cross-RAG: Zero-Shot Retrieval-Augmented Time Series Forecasting via Cross-Attention
This work addresses a domain-specific bottleneck in time series forecasting for researchers and practitioners, offering an incremental enhancement over existing retrieval-augmented approaches.
The paper tackles the problem of limited generalization in zero-shot time series forecasting with foundation models by proposing Cross-RAG, a framework that selectively attends to relevant retrieved samples, resulting in consistent performance improvements across various models and retrieval methods.
Recent advances in time series foundation models (TSFMs) demonstrate strong expressive capacity through large-scale pretraining across diverse time series domains. Zero-shot time series forecasting with TSFMs, however, exhibits limited generalization to unseen datasets, which retrieval-augmented forecasting addresses by leveraging an external knowledge base. Existing approaches rely on a fixed number of retrieved samples that may introduce irrelevant information. To this end, we propose Cross-RAG, a zero-shot retrieval-augmented forecasting framework that selectively attends to query-relevant retrieved samples. Cross-RAG models input-level relevance between the query and retrieved samples via query-retrieval cross-attention, while jointly incorporating information from the query and retrieved samples. Extensive experiments demonstrate that Cross-RAG consistently improves zero-shot forecasting performance across various TSFMs and RAG methods, and additional analyses confirm its effectiveness across diverse retrieval scenarios. Code is available at https://github.com/seunghan96/cross-rag/.