CausalMan: A physics-based simulator for large-scale causality
This provides a new benchmark for researchers in causality, though it is incremental as it focuses on simulation rather than novel causal inference methods.
The paper tackled the problem of lacking realistic causal models for fair benchmarking by introducing the CausalMan simulator, a physics-based tool modeled after a production line with diverse mechanisms and challenging behaviors, and demonstrated the inadequacy of many state-of-the-art approaches while analyzing performance and tractability differences.
A comprehensive understanding of causality is critical for navigating and operating within today's complex real-world systems. The absence of realistic causal models with known data generating processes complicates fair benchmarking. In this paper, we present the CausalMan simulator, modeled after a real-world production line. The simulator features a diverse range of linear and non-linear mechanisms and challenging-to-predict behaviors, such as discrete mode changes. We demonstrate the inadequacy of many state-of-the-art approaches and analyze the significant differences in their performance and tractability, both in terms of runtime and memory complexity. As a contribution, we will release the CausalMan large-scale simulator. We present two derived datasets, and perform an extensive evaluation of both.