LINGO-Space: Language-Conditioned Incremental Grounding for Space
This addresses the challenge of interpreting ambiguous spatial instructions for robots, though it appears incremental as it builds on existing grounding methods with a novel probabilistic approach.
The paper tackles the problem of space grounding—spatially localizing composite instructions referring to space—by proposing LINGO-Space, a probabilistic method that identifies and incrementally updates a distribution of referred-to space using configurable polar distributions. Results show successful grounding in 20 table-top manipulation tests, with incremental updates narrowing the space accurately, and robustness in simulated and real robot tasks.
We aim to solve the problem of spatially localizing composite instructions referring to space: space grounding. Compared to current instance grounding, space grounding is challenging due to the ill-posedness of identifying locations referred to by discrete expressions and the compositional ambiguity of referring expressions. Therefore, we propose a novel probabilistic space-grounding methodology (LINGO-Space) that accurately identifies a probabilistic distribution of space being referred to and incrementally updates it, given subsequent referring expressions leveraging configurable polar distributions. Our evaluations show that the estimation using polar distributions enables a robot to ground locations successfully through $20$ table-top manipulation benchmark tests. We also show that updating the distribution helps the grounding method accurately narrow the referring space. We finally demonstrate the robustness of the space grounding with simulated manipulation and real quadruped robot navigation tasks. Code and videos are available at https://lingo-space.github.io.