Diffusion Dynamics Models with Generative State Estimation for Cloth Manipulation
This addresses the challenge of deformable object manipulation for robotics, with incremental improvements in accuracy and control.
The paper tackles the problem of cloth manipulation by proposing a diffusion-based generative approach for state estimation and dynamics modeling, achieving an order of magnitude reduction in long-horizon prediction errors and enabling effective cloth folding on real robots.
Cloth manipulation is challenging due to its highly complex dynamics, near-infinite degrees of freedom, and frequent self-occlusions, which complicate both state estimation and dynamics modeling. Inspired by recent advances in generative models, we hypothesize that these expressive models can effectively capture intricate cloth configurations and deformation patterns from data. Therefore, we propose a diffusion-based generative approach for both perception and dynamics modeling. Specifically, we formulate state estimation as reconstructing full cloth states from partial observations and dynamics modeling as predicting future states given the current state and robot actions. Leveraging a transformer-based diffusion model, our method achieves accurate state reconstruction and reduces long-horizon dynamics prediction errors by an order of magnitude compared to prior approaches. We integrate our dynamics models with model predictive control and show that our framework enables effective cloth folding on real robotic systems, demonstrating the potential of generative models for deformable object manipulation under partial observability and complex dynamics.