DexKnot: Generalizable Visuomotor Policy Learning for Dexterous Bag-Knotting Manipulation
This work tackles the problem of dexterous manipulation of deformable objects, specifically bag knotting, for robots, offering improved generalization compared to existing methods.
This paper addresses the challenge of robotic knotting of plastic bags, which is difficult due to their high degrees of freedom and complex dynamics. The authors propose DexKnot, a framework that uses keypoint affordance and diffusion policy to learn a generalizable bag-knotting policy, demonstrating reliable and consistent performance across various unseen bag instances and deformations.
Knotting plastic bags is a common task in daily life, yet it is challenging for robots due to the bags' infinite degrees of freedom and complex physical dynamics. Existing methods often struggle in generalization to unseen bag instances or deformations. To address this, we present DexKnot, a framework that combines keypoint affordance with diffusion policy to learn a generalizable bag-knotting policy. Our approach learns a shape-agnostic representation of bags from keypoint correspondence data collected through real-world manual deformation. For an unseen bag configuration, the keypoints can be identified by matching the representation to a reference. These keypoints are then provided to a diffusion transformer, which generates robot action based on a small number of human demonstrations. DexKnot enables effective policy generalization by reducing the dimensionality of observation space into a sparse set of keypoints. Experiments show that DexKnot achieves reliable and consistent knotting performance across a variety of previously unseen instances and deformations.