PIGEON: VLM-Driven Object Navigation via Points of Interest Selection
This addresses the challenge of balancing decision frequency and intelligence in embodied AI for object navigation, offering incremental improvements over existing methods.
The paper tackles the problem of object navigation in unknown environments by proposing PIGEON, a method that uses a Visual-Language Model to select Points of Interest for decision-making, achieving state-of-the-art performance on benchmarks with zero-shot transfer and further improvements via Reinforcement Learning with Verifiable Reward.
Navigating to a specified object in an unknown environment is a fundamental yet challenging capability of embodied intelligence. However, current methods struggle to balance decision frequency with intelligence, resulting in decisions lacking foresight or discontinuous actions. In this work, we propose PIGEON: Point of Interest Guided Exploration for Object Navigation with VLM, maintaining a lightweight and semantically aligned snapshot memory during exploration as semantic input for the exploration strategy. We use a large Visual-Language Model (VLM), named PIGEON-VL, to select Points of Interest (PoI) formed during exploration and then employ a lower-level planner for action output, increasing the decision frequency. Additionally, this PoI-based decision-making enables the generation of Reinforcement Learning with Verifiable Reward (RLVR) data suitable for simulators. Experiments on classic object navigation benchmarks demonstrate that our zero-shot transfer method achieves state-of-the-art performance, while RLVR further enhances the model's semantic guidance capabilities, enabling deep reasoning during real-time navigation.