A Modular Vision-Language-Action Robotics Framework for Indoor Environments
This work provides a practical system for the CMU VLA Challenge, but the approach is incremental, combining existing techniques (OwlViT, VLMs) without major novelty.
The paper presents a modular VLA framework for indoor robotics that integrates semantic mapping and language processing to execute natural language commands, achieving task completion within a 500-second exploration limit.
This paper presents an integrated system for the CMU Vision-Language-Action (VLA) Challenge, designed to enable an autonomous agent to perform complex tasks based on natural language instructions. Our framework employs a modular architecture that orchestrates environment mapping, question processing, and navigation. The system operates in two parallel streams: a perception pipeline that constructs a semantic voxel map from real-time camera feeds using OwlViT embeddings, and a language pipeline that classifies user commands with a Vision-Language Model. The mapping is time-constrained; the system proceeds with a partial map if a 500-second exploration limit is reached. The classified query is then grounded in the geometric and semantic context of the map to generate a detailed prompt for the VLM. This yields an actionable output, demonstrating a capable solution for bridging the gap between human language and robotic action.