CAUSAL3D: A Comprehensive Benchmark for Causal Learning from Visual Data
This addresses the need for standardized evaluation in causal learning for computer vision researchers, though it is incremental as it builds on existing causal reasoning concepts.
The authors tackled the lack of benchmarks for assessing causal reasoning from visual data by introducing Causal3D, a comprehensive benchmark with 19 3D-scene datasets integrating structured and visual data, and found that performance of state-of-the-art methods declines significantly as causal complexity increases without prior knowledge.
True intelligence hinges on the ability to uncover and leverage hidden causal relations. Despite significant progress in AI and computer vision (CV), there remains a lack of benchmarks for assessing models' abilities to infer latent causality from complex visual data. In this paper, we introduce \textsc{\textbf{Causal3D}}, a novel and comprehensive benchmark that integrates structured data (tables) with corresponding visual representations (images) to evaluate causal reasoning. Designed within a systematic framework, Causal3D comprises 19 3D-scene datasets capturing diverse causal relations, views, and backgrounds, enabling evaluations across scenes of varying complexity. We assess multiple state-of-the-art methods, including classical causal discovery, causal representation learning, and large/vision-language models (LLMs/VLMs). Our experiments show that as causal structures grow more complex without prior knowledge, performance declines significantly, highlighting the challenges even advanced methods face in complex causal scenarios. Causal3D serves as a vital resource for advancing causal reasoning in CV and fostering trustworthy AI in critical domains.