Wavelet Mixture of Experts for Time Series Forecasting
This work addresses the problem of efficient and accurate multi-channel time series forecasting for domains like finance or IoT, though it appears incremental as it builds on existing wavelet and MoE techniques.
The authors tackled the limitations of large-scale Transformers and lightweight MLP models in time series forecasting by proposing WaveTS-B and WaveTS-M, which combine wavelet transforms with MLP and a Mixture of Experts framework, achieving state-of-the-art performance with significantly fewer parameters across eight real-world datasets.
The field of time series forecasting is rapidly advancing, with recent large-scale Transformers and lightweight Multilayer Perceptron (MLP) models showing strong predictive performance. However, conventional Transformer models are often hindered by their large number of parameters and their limited ability to capture non-stationary features in data through smoothing. Similarly, MLP models struggle to manage multi-channel dependencies effectively. To address these limitations, we propose a novel, lightweight time series prediction model, WaveTS-B. This model combines wavelet transforms with MLP to capture both periodic and non-stationary characteristics of data in the wavelet domain. Building on this foundation, we propose a channel clustering strategy that incorporates a Mixture of Experts (MoE) framework, utilizing a gating mechanism and expert network to handle multi-channel dependencies efficiently. We propose WaveTS-M, an advanced model tailored for multi-channel time series prediction. Empirical evaluation across eight real-world time series datasets demonstrates that our WaveTS series models achieve state-of-the-art (SOTA) performance with significantly fewer parameters. Notably, WaveTS-M shows substantial improvements on multi-channel datasets, highlighting its effectiveness.