Algorithmic causal structure emerging through compression
This work addresses a foundational challenge in causality for machine learning, offering a novel perspective that could impact models like large language models, though it appears incremental by building on known compression-learning connections.
The paper tackles the problem of causal structure identification when traditional assumptions fail, proposing algorithmic causality as an alternative definition that emerges from compressing data across environments, with results demonstrated through minimizing Kolmogorov complexity bounds without intervention knowledge.
We explore the relationship between causality, symmetry, and compression. We build on and generalize the known connection between learning and compression to a setting where causal models are not identifiable. We propose a framework where causality emerges as a consequence of compressing data across multiple environments. We define algorithmic causality as an alternative definition of causality when traditional assumptions for causal identifiability do not hold. We demonstrate how algorithmic causal and symmetric structures can emerge from minimizing upper bounds on Kolmogorov complexity, without knowledge of intervention targets. We hypothesize that these insights may also provide a novel perspective on the emergence of causality in machine learning models, such as large language models, where causal relationships may not be explicitly identifiable.