TAIHRI: Task-Aware 3D Human Keypoints Localization for Close-Range Human-Robot Interaction
This addresses the need for precise metric-scale localization of body parts in human-robot interaction, enabling safer and more natural interactions, though it appears incremental as it builds on existing keypoints estimation methods with a task-aware focus.
The paper tackles the problem of 3D human keypoints localization for close-range human-robot interaction by proposing TAIHRI, a vision-language model that directs robot attention to task-relevant body parts, achieving superior estimation accuracy on egocentric benchmarks.
Accurate 3D human keypoints localization is a critical technology enabling robots to achieve natural and safe physical interaction with users. Conventional 3D human keypoints estimation methods primarily focus on the whole-body reconstruction quality relative to the root joint. However, in practical human-robot interaction (HRI) scenarios, robots are more concerned with the precise metric-scale spatial localization of task-relevant body parts under the egocentric camera 3D coordinate. We propose TAIHRI, the first Vision-Language Model (VLM) tailored for close-range HRI perception, capable of understanding users' motion commands and directing the robot's attention to the most task-relevant keypoints. By quantizing 3D keypoints into a finite interaction space, TAIHRI precisely localize the 3D spatial coordinates of critical body parts by 2D keypoint reasoning via next token prediction, and seamlessly adapt to downstream tasks such as natural language control or global space human mesh recovery. Experiments on egocentric interaction benchmarks demonstrate that TAIHRI achieves superior estimation accuracy for task-critical body parts. We believe TAIHRI opens new research avenues in the field of embodied human-robot interaction. Code is available at: https://github.com/Tencent/TAIHRI.