The Helping Hand: An Assistive Manipulation Framework Using Augmented Reality and a Tongue-Drive Interfaces
This work addresses manipulation challenges for individuals with physical disabilities, representing an incremental improvement by integrating existing technologies into a novel framework.
The paper tackles enabling collaborative manipulation for people with physical disabilities by proposing a human-in-the-loop system that uses augmented reality and a tongue-drive interface to interpret user intent and control a robotic arm, achieving competitive results with faster completion times compared to existing methods.
A human-in-the-loop system is proposed to enable collaborative manipulation tasks for person with physical disabilities. Studies show that the cognitive burden of subject reduces with increased autonomy of assistive system. Our framework obtains high-level intent from the user to specify manipulation tasks. The system processes sensor input to interpret the user's environment. Augmented reality glasses provide ego-centric visual feedback of the interpretation and summarize robot affordances on a menu. A tongue drive system serves as the input modality for triggering a robotic arm to execute the tasks. Assistance experiments compare the system to Cartesian control and to state-of-the-art approaches. Our system achieves competitive results with faster completion time by simplifying manipulation tasks.