Geometric Retargeting: A Principled, Ultrafast Neural Hand Retargeting Algorithm
This provides a scalable and practical solution for real-time hand retargeting in teleoperation systems, though it is incremental as it builds on existing retargeting methods with novel geometric objectives.
The paper tackles the problem of real-time hand retargeting for teleoperation by introducing Geometric Retargeting (GeoRT), which converts human finger keypoints to robot hand keypoints at 1KHz, achieving state-of-the-art speed and accuracy with fewer hyperparameters.
We introduce Geometric Retargeting (GeoRT), an ultrafast, and principled neural hand retargeting algorithm for teleoperation, developed as part of our recent Dexterity Gen (DexGen) system. GeoRT converts human finger keypoints to robot hand keypoints at 1KHz, achieving state-of-the-art speed and accuracy with significantly fewer hyperparameters. This high-speed capability enables flexible postprocessing, such as leveraging a foundational controller for action correction like DexGen. GeoRT is trained in an unsupervised manner, eliminating the need for manual annotation of hand pairs. The core of GeoRT lies in novel geometric objective functions that capture the essence of retargeting: preserving motion fidelity, ensuring configuration space (C-space) coverage, maintaining uniform response through high flatness, pinch correspondence and preventing self-collisions. This approach is free from intensive test-time optimization, offering a more scalable and practical solution for real-time hand retargeting.