DoughNet: A Visual Predictive Model for Topological Manipulation of Deformable Objects
This addresses a critical challenge in robotic manipulation of elastoplastic objects, enabling better planning for tasks like tool selection, but it is incremental as it builds on prior work with a novel hybrid approach.
The paper tackles the problem of predicting topological changes like splitting and merging in deformable objects such as dough, and presents DoughNet, a Transformer-based model that outperforms existing methods by handling both geometrical deformation and topological changes in latent space.
Manipulation of elastoplastic objects like dough often involves topological changes such as splitting and merging. The ability to accurately predict these topological changes that a specific action might incur is critical for planning interactions with elastoplastic objects. We present DoughNet, a Transformer-based architecture for handling these challenges, consisting of two components. First, a denoising autoencoder represents deformable objects of varying topology as sets of latent codes. Second, a visual predictive model performs autoregressive set prediction to determine long-horizon geometrical deformation and topological changes purely in latent space. Given a partial initial state and desired manipulation trajectories, it infers all resulting object geometries and topologies at each step. DoughNet thereby allows to plan robotic manipulation; selecting a suited tool, its pose and opening width to recreate robot- or human-made goals. Our experiments in simulated and real environments show that DoughNet is able to significantly outperform related approaches that consider deformation only as geometrical change.