Bridging Physical Reasoning and Task Generalization via Visual Action Outcome Reasoning Alignment
For vision-language models in interactive physical reasoning, VAORA addresses the critical problems of hallucinated reasoning and misalignment between reasoning and actions, enabling generalization to unseen tasks and environments.
VAORA introduces a reward design with visual alignment and visual-action alignment to suppress hallucinated chain-of-thought and reduce reasoning-action misalignment in VLMs for interactive physical reasoning, achieving strong performance on PHYRE and Virtual Tool across novel-task and unseen-environment settings.
Vision-language models (VLMs) struggle to generalize in interactive physical reasoning, particularly under unseen tasks and environments. Two key failure modes are prominent: hallucinated chain-of-thought (CoT) reasoning that contradicts physical reality, and misalignment between the model's reasoning and actions. We present VAORA (Visual Action Outcome Reasoning Alignment), a novel reward design that directly addresses both issues. VAORA introduces two complementary rewards: Visual Alignment Reward, which anchors VLM reasoning to the visual context independent of the agent action itself, and Visual-Action Alignment Reward, which grounds reasoning in the visual outcome induced by the model's action. Together, these rewards suppress hallucinated CoT and reduce the gap between reasoning and behavior. To improve training stability, we further employ smooth, dense rewards by estimating success probabilities using a pre-trained in-domain expert agent. Experiments on PHYRE and Virtual Tool support our performances across novel-task and unseen-environment settings, confirming that grounded and generalizable physical intelligence can be induced through VAORA.