Rethinking Time Series Forecasting with LLMs via Nearest Neighbor Contrastive Learning
This addresses the problem of effectively leveraging LLMs for time series forecasting, which is incremental as it builds on existing adaptation approaches but introduces a novel prompt formulation technique.
The paper tackles the challenge of adapting Large Language Models (LLMs) for time series forecasting by proposing NNCL-TLLM, which uses nearest neighbor contrastive learning to create time series-compatible text prototypes and fine-tunes only specific LLM components. The method outperforms in few-shot forecasting and achieves competitive or superior performance in long-term and short-term forecasting tasks compared to state-of-the-art methods.
Adapting Large Language Models (LLMs) that are extensively trained on abundant text data, and customizing the input prompt to enable time series forecasting has received considerable attention. While recent work has shown great potential for adapting the learned prior of LLMs, the formulation of the prompt to finetune LLMs remains challenging as prompt should be aligned with time series data. Additionally, current approaches do not effectively leverage word token embeddings which embody the rich representation space learned by LLMs. This emphasizes the need for a robust approach to formulate the prompt which utilizes the word token embeddings while effectively representing the characteristics of the time series. To address these challenges, we propose NNCL-TLLM: Nearest Neighbor Contrastive Learning for Time series forecasting via LLMs. First, we generate time series compatible text prototypes such that each text prototype represents both word token embeddings in its neighborhood and time series characteristics via end-to-end finetuning. Next, we draw inspiration from Nearest Neighbor Contrastive Learning to formulate the prompt while obtaining the top-$k$ nearest neighbor time series compatible text prototypes. We then fine-tune the layer normalization and positional embeddings of the LLM, keeping the other layers intact, reducing the trainable parameters and decreasing the computational cost. Our comprehensive experiments demonstrate that NNCL-TLLM outperforms in few-shot forecasting while achieving competitive or superior performance over the state-of-the-art methods in long-term and short-term forecasting tasks.