TASC: Task-Aware Shared Control for Relational Telemanipulation
This work addresses the challenge of inferring high-level intent from low-level motion commands in teleoperation, enabling more intuitive and efficient control for robotic manipulation tasks.
TASC infers task-level user intent from motion-only input and provides shared control assistance for relational telemanipulation, improving task efficiency and reducing user input effort compared to prior methods, with zero-shot generalization across diverse tasks.
We present TASC, a Task-Aware Shared Control framework for relational telemanipulation that infers task-level user intent and provides assistance from motion-only input. To support prehensile relational tasks without predefined templates, TASC constructs an open-vocabulary interaction graph from visual input to represent functional object relationships, and infers user intent accordingly. A shared control policy then provides assistance during both grasping and object interaction, guided by spatial constraints predicted by a vision-language model. Our method addresses two key challenges in relational telemanipulation under shared control: (1) task-level intent inference from low-level motion commands, and (2) generalizable assistance across diverse objects and tasks. Experiments in both simulation and the real world demonstrate that TASC improves task efficiency and reduces user input effort compared to prior methods, while enabling zero-shot generalization across diverse relational telemanipulation tasks. The code that supports our experiments is publicly available at https://github.com/fitz0401/tasc.