Multi-Task Time Series Forecasting With Shared Attention
This addresses the problem of limited data in time series forecasting for industrial and business applications, but it is incremental as it builds on existing attention-based models.
The paper tackles the problem of insufficient training instances in single-task time series forecasting by proposing self-attention based sharing schemes for multi-task forecasting, resulting in architectures that outperform state-of-the-art single-task and RNN-based multi-task baselines on real-world tasks.
Time series forecasting is a key component in many industrial and business decision processes and recurrent neural network (RNN) based models have achieved impressive progress on various time series forecasting tasks. However, most of the existing methods focus on single-task forecasting problems by learning separately based on limited supervised objectives, which often suffer from insufficient training instances. As the Transformer architecture and other attention-based models have demonstrated its great capability of capturing long term dependency, we propose two self-attention based sharing schemes for multi-task time series forecasting which can train jointly across multiple tasks. We augment a sequence of paralleled Transformer encoders with an external public multi-head attention function, which is updated by all data of all tasks. Experiments on a number of real-world multi-task time series forecasting tasks show that our proposed architectures can not only outperform the state-of-the-art single-task forecasting baselines but also outperform the RNN-based multi-task forecasting method.