AntiGrounding: Lifting Robotic Actions into VLM Representation Space for Decision Making
This addresses robotic manipulation tasks, offering a novel approach for zero-shot decision making.
The paper tackles the problem of robotic manipulation by proposing AntiGrounding, a framework that lifts candidate actions into VLM representation space for decision making, enabling zero-shot synthesis of optimal closed-loop trajectories and outperforming baselines in experiments.
Vision-Language Models (VLMs) encode knowledge and reasoning capabilities for robotic manipulation within high-dimensional representation spaces. However, current approaches often project them into compressed intermediate representations, discarding important task-specific information such as fine-grained spatial or semantic details. To address this, we propose AntiGrounding, a new framework that reverses the instruction grounding process. It lifts candidate actions directly into the VLM representation space, renders trajectories from multiple views, and uses structured visual question answering for instruction-based decision making. This enables zero-shot synthesis of optimal closed-loop robot trajectories for new tasks. We also propose an offline policy refinement module that leverages past experience to enhance long-term performance. Experiments in both simulation and real-world environments show that our method outperforms baselines across diverse robotic manipulation tasks.