TReF-6: Inferring Task-Relevant Frames from a Single Demonstration for One-Shot Skill Generalization
This addresses the challenge of interpretable spatial representation for one-shot imitation learning in robotics, though it appears incremental as it builds on existing DMP and vision-language models.
The paper tackles the problem of robots struggling to generalize from a single demonstration by introducing TReF-6, a method that infers a 6DoF Task-Relevant Frame from a single trajectory to enable one-shot skill generalization, showing robustness to trajectory noise and successful deployment on real-world manipulation tasks.
Robots often struggle to generalize from a single demonstration due to the lack of a transferable and interpretable spatial representation. In this work, we introduce TReF-6, a method that infers a simplified, abstracted 6DoF Task-Relevant Frame from a single trajectory. Our approach identifies an influence point purely from the trajectory geometry to define the origin for a local frame, which serves as a reference for parameterizing a Dynamic Movement Primitive (DMP). This influence point captures the task's spatial structure, extending the standard DMP formulation beyond start-goal imitation. The inferred frame is semantically grounded via a vision-language model and localized in novel scenes by Grounded-SAM, enabling functionally consistent skill generalization. We validate TReF-6 in simulation and demonstrate robustness to trajectory noise. We further deploy an end-to-end pipeline on real-world manipulation tasks, showing that TReF-6 supports one-shot imitation learning that preserves task intent across diverse object configurations.