AI Foundation Model for Time Series with Innovations Representation
This work addresses the need for causal, real-time monitoring and control in engineering domains, representing an incremental adaptation of foundation models to time series data.
The paper tackles the problem of developing an AI foundation model for time series in engineering applications, where existing large-language-model-based approaches may be ineffective due to physical laws, and demonstrates its effectiveness by forecasting real-time locational marginal prices using historical data from U.S. independent system operators.
This paper introduces an Artificial Intelligence (AI) foundation model for time series in engineering applications, where causal operations are required for real-time monitoring and control. Since engineering time series are governed by physical, rather than linguistic, laws, large-language-model-based AI foundation models may be ineffective or inefficient. Building on the classical innovations representation theory of Wiener, Kallianpur, and Rosenblatt, we propose Time Series GPT (TS-GPT) -- an innovations-representation-based Generative Pre-trained Transformer for engineering monitoring and control. As an example of foundation model adaptation, we consider Probabilistic Generative Forecasting, which produces future time series samples from conditional probability distributions given past realizations. We demonstrate the effectiveness of TS-GPT in forecasting real-time locational marginal prices using historical data from U.S. independent system operators.