Causal Temporal Representation Learning with Nonstationary Sparse Transition
This work addresses a limitation in causal temporal representation learning for real-world scenarios where prior knowledge of domain variables is unavailable, representing an incremental advancement.
The paper tackles the problem of identifying temporal causal dynamics in nonstationary sequences without requiring prior knowledge of domain variables, by introducing a sparse transition assumption and a novel framework called CtrlNS, which shows significant improvements over existing baselines in experiments.
Causal Temporal Representation Learning (Ctrl) methods aim to identify the temporal causal dynamics of complex nonstationary temporal sequences. Despite the success of existing Ctrl methods, they require either directly observing the domain variables or assuming a Markov prior on them. Such requirements limit the application of these methods in real-world scenarios when we do not have such prior knowledge of the domain variables. To address this problem, this work adopts a sparse transition assumption, aligned with intuitive human understanding, and presents identifiability results from a theoretical perspective. In particular, we explore under what conditions on the significance of the variability of the transitions we can build a model to identify the distribution shifts. Based on the theoretical result, we introduce a novel framework, Causal Temporal Representation Learning with Nonstationary Sparse Transition (CtrlNS), designed to leverage the constraints on transition sparsity and conditional independence to reliably identify both distribution shifts and latent factors. Our experimental evaluations on synthetic and real-world datasets demonstrate significant improvements over existing baselines, highlighting the effectiveness of our approach.