On the Eligibility of LLMs for Counterfactual Reasoning: A Decompositional Study
This work addresses the challenge of improving the reliability and adaptability of LLM-based reasoning systems for AI researchers, though it is incremental as it builds on prior studies of counterfactual reasoning.
The paper tackled the problem of understanding why large language models (LLMs) struggle with counterfactual reasoning by proposing a decompositional strategy to analyze factors like modality and intermediate reasoning across 11 diverse datasets, finding that these elements significantly influence performance.
Counterfactual reasoning has emerged as a crucial technique for generalizing the reasoning capabilities of large language models (LLMs). By generating and analyzing counterfactual scenarios, researchers can assess the adaptability and reliability of model decision-making. Although prior work has shown that LLMs often struggle with counterfactual reasoning, it remains unclear which factors most significantly impede their performance across different tasks and modalities. In this paper, we propose a decompositional strategy that breaks down the counterfactual generation from causality construction to the reasoning over counterfactual interventions. To support decompositional analysis, we investigate 11 datasets spanning diverse tasks, including natural language understanding, mathematics, programming, and vision-language tasks. Through extensive evaluations, we characterize LLM behavior across each decompositional stage and identify how modality type and intermediate reasoning influence performance. By establishing a structured framework for analyzing counterfactual reasoning, this work contributes to the development of more reliable LLM-based reasoning systems and informs future elicitation strategies.