Enhancing Reusability of Learned Skills for Robot Manipulation via Gaze Information and Motion Bottlenecks
This addresses the problem of skill reusability in robot manipulation for autonomous agents, representing an incremental improvement with specific gains in generalization.
The paper tackles the challenge of generalizing learned robot manipulation skills to unseen scenarios by proposing GazeBot, a novel algorithm that leverages gaze information and motion bottlenecks, achieving high success rates compared to state-of-the-art imitation learning methods when object positions and end-effector poses differ from demonstrations.
Autonomous agents capable of diverse object manipulations should be able to acquire a wide range of manipulation skills with high reusability. Although advances in deep learning have made it increasingly feasible to replicate the dexterity of human teleoperation in robots, generalizing these acquired skills to previously unseen scenarios remains a significant challenge. In this study, we propose a novel algorithm, Gaze-based Bottleneck-aware Robot Manipulation (GazeBot), which enables high reusability of learned motions without sacrificing dexterity or reactivity. By leveraging gaze information and motion bottlenecks, both crucial features for object manipulation, GazeBot achieves high success rates compared with state-of-the-art imitation learning methods, particularly when the object positions and end-effector poses differ from those in the provided demonstrations. Furthermore, the training process of GazeBot is entirely data-driven once a demonstration dataset with gaze data is provided. Videos and code are available at https://crumbyrobotics.github.io/gazebot.