Inducing Causal World Models in LLMs for Zero-Shot Physical Reasoning
This addresses the limitation of LLMs in real-world scenarios requiring causal reasoning, representing a novel method for a known bottleneck rather than incremental progress.
The paper tackles the problem of LLMs lacking intuitive understanding of physical dynamics by introducing Causal World Model Induction (CWMI), a framework that embeds an explicit causal physics model, resulting in significant outperformance over state-of-the-art LLMs on zero-shot physical reasoning tasks like PIQA and PhysiCa-Bench.
Large Language Models (LLMs), despite their advanced linguistic capabilities, fundamentally lack an intuitive understanding of physical dynamics, which limits their effectiveness in real-world scenarios that require causal reasoning. In this paper, we introduce Causal World Model Induction (CWMI), a novel framework designed to embed an explicit model of causal physics within an LLM. Our approach incorporates a dedicated Causal Physics Module (CPM) and a new training objective called Causal Intervention Loss, encouraging the model to learn cause-and-effect relationships from multimodal data. By training the model to predict the outcomes of hypothetical interventions instead of merely capturing statistical correlations, CWMI develops a robust internal representation of physical laws. Experimental results show that CWMI significantly outperforms state-of-the-art LLMs on zero-shot physical reasoning tasks, including the PIQA benchmark and our newly proposed PhysiCa-Bench dataset. These findings demonstrate that inducing a causal world model is a critical step toward more reliable and generalizable AI systems.