Using Pre-trained LLMs for Multivariate Time Series Forecasting
This addresses forecasting problems for domains like supply chain or retail, but it is incremental as it adapts existing LLMs to a new task.
The paper tackled multivariate demand time series forecasting by mapping time series into LLM token embeddings using a novel patching strategy, achieving results competitive with state-of-the-art models.
Pre-trained Large Language Models (LLMs) encapsulate large amounts of knowledge and take enormous amounts of compute to train. We make use of this resource, together with the observation that LLMs are able to transfer knowledge and performance from one domain or even modality to another seemingly-unrelated area, to help with multivariate demand time series forecasting. Attention in transformer-based methods requires something worth attending to -- more than just samples of a time-series. We explore different methods to map multivariate input time series into the LLM token embedding space. In particular, our novel multivariate patching strategy to embed time series features into decoder-only pre-trained Transformers produces results competitive with state-of-the-art time series forecasting models. We also use recently-developed weight-based diagnostics to validate our findings.