Prune-Then-Plan: Step-Level Calibration for Stable Frontier Exploration in Embodied Question Answering
This addresses a specific issue in embodied AI for more stable and efficient navigation, representing an incremental improvement through calibration and planning.
The paper tackled the problem of frontier oscillations in embodied question answering agents using large vision-language models, which cause inefficient navigation and degraded answer quality, by proposing Prune-Then-Plan, a framework that stabilizes exploration through step-level calibration, achieving relative improvements of up to 49% in SPL and 33% in LLM-Match metrics over baselines.
Large vision-language models (VLMs) have improved embodied question answering (EQA) agents by providing strong semantic priors for open-vocabulary reasoning. However, when used directly for step-level exploration, VLMs often exhibit frontier oscillations, unstable back-and-forth movements caused by overconfidence and miscalibration, leading to inefficient navigation and degraded answer quality. We propose Prune-Then-Plan, a simple and effective framework that stabilizes exploration through step-level calibration. Instead of trusting raw VLM scores, our method prunes implausible frontier choices using a Holm-Bonferroni inspired pruning procedure and then delegates final decisions to a coverage-based planner. This separation converts overconfident predictions into conservative, interpretable actions by relying on human-level judgments to calibrate the step-level behavior of VLMs. Integrated into the 3D-Mem EQA framework, our approach achieves relative improvements of up to 49% and 33% in visually grounded SPL and LLM-Match metrics respectively over baselines. Overall, our method achieves better scene coverage under equal exploration budgets on both OpenEQA and EXPRESS-Bench datasets.