Situated Multimodal Control of a Mobile Robot: Navigation through a Virtual Environment
This addresses the challenge of intuitive human-robot interaction for navigation in robotics, though it appears incremental as it builds on existing multimodal control methods.
The paper tackles the problem of controlling a mobile robot in novel environments by developing an interface that uses coordinated gesture and language for navigation, enabling a human partner to guide the robot through tasks with spoken English and gestures relative to a simulated environment.
We present a new interface for controlling a navigation robot in novel environments using coordinated gesture and language. We use a TurtleBot3 robot with a LIDAR and a camera, an embodied simulation of what the robot has encountered while exploring, and a cross-platform bridge facilitating generic communication. A human partner can deliver instructions to the robot using spoken English and gestures relative to the simulated environment, to guide the robot through navigation tasks.