Graph-based Task-specific Prediction Models for Interactions between Deformable and Rigid Objects
This work addresses a specific challenge in robotic manipulation for scenes with mixed rigid and deformable objects, presenting an incremental improvement with a novel dataset and method.
The paper tackles the problem of predicting future scene states for robotic manipulation involving interactions between rigid and deformable objects, by proposing an object-centric graph representation and two graph neural network modules, achieving benefits in single time step and long-term prediction tasks as shown in an ablation study.
Capturing scene dynamics and predicting the future scene state is challenging but essential for robotic manipulation tasks, especially when the scene contains both rigid and deformable objects. In this work, we contribute a simulation environment and generate a novel dataset for task-specific manipulation, involving interactions between rigid objects and a deformable bag. The dataset incorporates a rich variety of scenarios including different object sizes, object numbers and manipulation actions. We approach dynamics learning by proposing an object-centric graph representation and two modules which are Active Prediction Module (APM) and Position Prediction Module (PPM) based on graph neural networks with an encode-process-decode architecture. At the inference stage, we build a two-stage model based on the learned modules for single time step prediction. We combine modules with different prediction horizons into a mixed-horizon model which addresses long-term prediction. In an ablation study, we show the benefits of the two-stage model for single time step prediction and the effectiveness of the mixed-horizon model for long-term prediction tasks. Supplementary material is available at https://github.com/wengzehang/deformable_rigid_interaction_prediction