From Woofs to Words: Towards Intelligent Robotic Guide Dogs with Verbal Communication
This addresses the problem of enhancing spatial awareness and safety for visually impaired individuals through intelligent assistive robotics, representing a domain-specific incremental advance.
The paper tackled enabling verbal communication for robotic guide dogs to assist visually impaired people, developing a dialog system using LLMs to verbalize navigational plans and scenes, with evaluation through human and simulation studies showing improved collaborative decision-making.
Assistive robotics is an important subarea of robotics that focuses on the well-being of people with disabilities. A robotic guide dog is an assistive quadruped robot that helps visually impaired people in obstacle avoidance and navigation. Enabling language capabilities for robotic guide dogs goes beyond naively adding an existing dialog system onto a mobile robot. The novel challenges include grounding language in the dynamically changing environment and improving spatial awareness for the human handler. To address those challenges, we develop a novel dialog system for robotic guide dogs that uses LLMs to verbalize both navigational plans and scenes. The goal is to enable verbal communication for collaborative decision-making within the handler-robot team. In experiments, we conducted a human study to evaluate different verbalization strategies and a simulation study to assess the efficiency and accuracy in navigation tasks.