TopoRetarget: Interaction-Preserving Retargeting for Dexterous Manipulation
For researchers in dexterous manipulation RL, this method solves the bottleneck of preserving hand-object interaction during retargeting, which is critical for effective policy learning from human demonstrations.
TopoRetarget introduces an interaction-preserving retargeting framework that adapts human hand-object demonstrations to dexterous robot hands while maintaining task-relevant contact structure. It achieves the best contact precision on ContactPose, improves Pen-Spin training success by 40.6 percentage points over baselines, and enables zero-shot hardware transfer.
Human hand-object demonstrations provide dense reference motions for training dexterous manipulation reinforcement learning (RL) policies through reference tracking. However, to use such demonstrations for RL policy learning, retargeting must preserve hand pose and task-relevant hand-object contact structure. Otherwise, contact and feasibility artifacts can degrade downstream RL policy performance. We introduce TopoRetarget, an interaction-preserving retargeting framework that uses a single set of parameters across diverse retargeting conditions while maintaining task-relevant hand-object interaction and adapting human demonstrations to dexterous robot hands. The method constructs a sparse interaction graph over hand and object keypoints and optimizes distance-weighted Laplacian deformation with directional consistency, kinematic constraints, and penetration handling. Evaluations show that the generated references improve both interaction fidelity and policy learning: TopoRetarget achieves the best contact precision and alignment over all baselines on the ContactPose Dataset, improves Pen-Spin training success by 40.6 percentage points over the existing baseline methods, and enables zero-shot transfer to Wuji Hand hardware on cube reorientation and pen spinning.