Causal Time-Series Synchronization for Multi-Dimensional Forecasting
This addresses the problem of transfer learning for Digital Twins in the process industry, which is an incremental advancement in a domain-specific context.
The paper tackles the challenge of developing foundational models for Digital Twins in the process industry by addressing multi-dimensional time-series data with lagged causal dependencies, proposing a channel-dependent pre-training strategy that synchronizes cause-effect pairs. The results show significant improvements in forecasting accuracy and generalization capability compared to traditional methods.
The process industry's high expectations for Digital Twins require modeling approaches that can generalize across tasks and diverse domains with potentially different data dimensions and distributional shifts i.e., Foundational Models. Despite success in natural language processing and computer vision, transfer learning with (self-) supervised signals for pre-training general-purpose models is largely unexplored in the context of Digital Twins in the process industry due to challenges posed by multi-dimensional time-series data, lagged cause-effect dependencies, complex causal structures, and varying number of (exogenous) variables. We propose a novel channel-dependent pre-training strategy that leverages synchronized cause-effect pairs to overcome these challenges by breaking down the multi-dimensional time-series data into pairs of cause-effect variables. Our approach focuses on: (i) identifying highly lagged causal relationships using data-driven methods, (ii) synchronizing cause-effect pairs to generate training samples for channel-dependent pre-training, and (iii) evaluating the effectiveness of this approach in channel-dependent forecasting. Our experimental results demonstrate significant improvements in forecasting accuracy and generalization capability compared to traditional training methods.