time2time: Causal Intervention in Hidden States to Simulate Rare Events in Time Series Foundation Models
This enables semantic 'what-if' analysis for strategic stress-testing in time series forecasting, shifting interpretability from post-hoc methods to direct causal intervention.
The paper tackled the problem of whether transformer-based foundation models internalize semantic concepts like market regimes and can simulate rare events such as market crashes, by introducing activation transplantation to manipulate hidden states, which steered forecasts to induce downturn predictions or restore stability, with latent vector norms correlating with event severity across two models.
While transformer-based foundation models excel at forecasting routine patterns, two questions remain: do they internalize semantic concepts such as market regimes, or merely fit curves? And can their internal representations be leveraged to simulate rare, high-stakes events such as market crashes? To investigate this, we introduce activation transplantation, a causal intervention that manipulates hidden states by imposing the statistical moments of one event (e.g., a historical crash) onto another (e.g., a calm period) during the forward pass. This procedure deterministically steers forecasts: injecting crash semantics induces downturn predictions, while injecting calm semantics suppresses crashes and restores stability. Beyond binary control, we find that models encode a graded notion of event severity, with the latent vector norm directly correlating with the magnitude of systemic shocks. Validated across two architecturally distinct TSFMs, Toto (decoder only) and Chronos (encoder-decoder), our results demonstrate that steerable, semantically grounded representations are a robust property of large time series transformers. Our findings provide evidence for a latent concept space that governs model predictions, shifting interpretability from post-hoc attribution to direct causal intervention, and enabling semantic "what-if" analysis for strategic stress-testing.