LIEREx: Language-Image Embeddings for Robotic Exploration
This addresses the challenge of handling out-of-distribution knowledge in robotic mapping, which is incremental over existing open-set mapping approaches.
The paper tackles the problem of robotic exploration in partially unknown environments by integrating Vision-Language Foundation Models with 3D Semantic Scene Graphs, enabling target-directed exploration without relying on pre-defined object classes.
Semantic maps allow a robot to reason about its surroundings to fulfill tasks such as navigating known environments, finding specific objects, and exploring unmapped areas. Traditional mapping approaches provide accurate geometric representations but are often constrained by pre-designed symbolic vocabularies. The reliance on fixed object classes makes it impractical to handle out-of-distribution knowledge not defined at design time. Recent advances in Vision-Language Foundation Models, such as CLIP, enable open-set mapping, where objects are encoded as high-dimensional embeddings rather than fixed labels. In LIEREx, we integrate these VLFMs with established 3D Semantic Scene Graphs to enable target-directed exploration by an autonomous agent in partially unknown environments.