VE: Modeling Multivariate Time Series Correlation with Variate Embedding
This work addresses a key limitation in multivariate time series forecasting models for applications requiring accurate correlation modeling, representing an incremental improvement by integrating VE into existing frameworks.
The paper tackles the problem of capturing correlations among variates in multivariate time series forecasting by introducing a variate embedding (VE) pipeline that learns unique embeddings for each variate, combined with Mixture of Experts and Low-Rank Adaptation techniques to enhance performance while controlling parameters, and demonstrates its effectiveness on four widely-used datasets.
Multivariate time series forecasting relies on accurately capturing the correlations among variates. Current channel-independent (CI) models and models with a CI final projection layer are unable to capture these dependencies. In this paper, we present the variate embedding (VE) pipeline, which learns a unique and consistent embedding for each variate and combines it with Mixture of Experts (MoE) and Low-Rank Adaptation (LoRA) techniques to enhance forecasting performance while controlling parameter size. The VE pipeline can be integrated into any model with a CI final projection layer to improve multivariate forecasting. The learned VE effectively groups variates with similar temporal patterns and separates those with low correlations. The effectiveness of the VE pipeline is demonstrated through experiments on four widely-used datasets. The code is available at: https://github.com/swang-song/VE.