DyWA: Dynamics-adaptive World Action Model for Generalizable Non-prehensile Manipulation
This addresses the challenge of generalizable robot manipulation for objects that are difficult to grasp, though it appears incremental as it builds on learning-based methods.
The paper tackled the problem of nonprehensile manipulation in unstructured environments by proposing the Dynamics-Adaptive World Action Model (DyWA), which improved success rates by 31.5% in simulation using single-view point clouds and achieved 68% in real-world experiments.
Nonprehensile manipulation is crucial for handling objects that are too thin, large, or otherwise ungraspable in unstructured environments. While conventional planning-based approaches struggle with complex contact modeling, learning-based methods have recently emerged as a promising alternative. However, existing learning-based approaches face two major limitations: they heavily rely on multi-view cameras and precise pose tracking, and they fail to generalize across varying physical conditions, such as changes in object mass and table friction. To address these challenges, we propose the Dynamics-Adaptive World Action Model (DyWA), a novel framework that enhances action learning by jointly predicting future states while adapting to dynamics variations based on historical trajectories. By unifying the modeling of geometry, state, physics, and robot actions, DyWA enables more robust policy learning under partial observability. Compared to baselines, our method improves the success rate by 31.5% using only single-view point cloud observations in the simulation. Furthermore, DyWA achieves an average success rate of 68% in real-world experiments, demonstrating its ability to generalize across diverse object geometries, adapt to varying table friction, and robustness in challenging scenarios such as half-filled water bottles and slippery surfaces.