TRACE: A Temporal Conditional Estimation for Multimodal Time Series Foundation Models
This work addresses the problem of robust multimodal time series modeling under missingness and irregular sampling, which is critical for real-world applications in healthcare and affective computing.
TRACE proposes a conditional estimation paradigm for multimodal time series foundation models that handles temporal misalignment and missing modalities by inferring incomplete target modalities from available auxiliary ones, outperforming prior fusion approaches on healthcare and affective computing benchmarks.
Time series foundation models (TS-FMs) aim to learn generalizable temporal representations that can be adapted to a wide range of downstream tasks. In real-world multimodal settings, time series are frequently affected by temporal misalignment and partial modality missingness, where different modalities are observed at heterogeneous time scales or are partially absent. Existing approaches typically rely on naive imputation or masking strategies, which fail to account for cross-modal dependencies and often lead to misaligned or degraded representations. We propose TRACE, a conditional estimation paradigm for multimodal time series foundation model pipelines under missingness and irregular sampling, allowing incomplete target modalities to be systematically inferred from available auxiliary modalities. We evaluate TRACE on diverse multimodal benchmarks spanning healthcare and affective computing, including the MIMIC-IV clinical dataset and the CMU-MOSI and CMU-MOSEI benchmarks for multimodal sentiment analysis. Across a range of downstream prediction tasks and missing-modality settings, TRACE consistently outperforms prior multimodal fusion approaches, demonstrating improved robustness to severe modality missingness and more reliable cross-modal representations.