Align While Search: Belief-Guided Exploratory Inference for World-Grounded Embodied Agents
This addresses the challenge of aligning AI agents with latent world states in partially observable environments, representing an incremental improvement over existing inference-time methods.
The paper tackles the problem of world-grounded embodied agents operating under partial observability by proposing a test-time adaptive agent that performs exploratory inference through posterior-guided belief refinement, resulting in outperforming inference-time scaling baselines with significantly lower integration overhead.
In this paper, we propose a test-time adaptive agent that performs exploratory inference through posterior-guided belief refinement without relying on gradient-based updates or additional training for LLM agent operating under partial observability. Our agent maintains an external structured belief over the environment state, iteratively updates it via action-conditioned observations, and selects actions by maximizing predicted information gain over the belief space. We estimate information gain using a lightweight LLM-based surrogate and assess world alignment through a novel reward that quantifies the consistency between posterior belief and ground-truth environment configuration. Experiments show that our method outperforms inference-time scaling baselines such as prompt-augmented or retrieval-enhanced LLMs, in aligning with latent world states with significantly lower integration overhead.