TCD-Arena: Assessing Robustness of Time Series Causal Discovery Methods Against Assumption Violations
For researchers and practitioners in causal discovery, this work provides a systematic framework to assess and compare method robustness under realistic assumption violations, addressing a key barrier to practical adoption.
The paper introduces TCD-Arena, a modular testing kit for evaluating robustness of time series causal discovery methods against assumption violations, and demonstrates its use in a large-scale study of 33 violations across ~30 million CD attempts, revealing nuanced robustness profiles and showing that ensembles can improve general robustness.
Causal Discovery (CD) is a powerful framework for scientific inquiry. Yet, its practical adoption is hindered by a reliance on strong, often unverifiable assumptions and a lack of robust performance assessment. To address these limitations and advance empirical CD evaluation, we present TCD-Arena, a modularized, highly customizable, and extendable testing kit to assess the robustness of time series CD algorithms against stepwise more severe assumption violations. For demonstration, we conduct an extensive empirical study comprising around 30 million individual CD attempts and reveal nuanced robustness profiles for 33 distinct assumption violations. Further, we investigate CD ensembles and find that they have the potential to improve general robustness, which has implications for real-world applications. With this, we strive to ultimately facilitate the development of CD methods that are reliable for a diverse range of synthetic and potentially real-world data conditions.