DIPN: Deep Interaction Prediction Network with Application to Clutter Removal
This addresses the challenge of robotic manipulation in cluttered environments, offering an incremental improvement over existing methods.
The paper tackles the problem of predicting complex interactions for robot pushing of multiple objects with unknown properties, resulting in a system that significantly outperforms previous state-of-the-art methods in clutter removal tasks and achieves better performance on real hardware than in simulation.
We propose a Deep Interaction Prediction Network (DIPN) for learning to predict complex interactions that ensue as a robot end-effector pushes multiple objects, whose physical properties, including size, shape, mass, and friction coefficients may be unknown a priori. DIPN "imagines" the effect of a push action and generates an accurate synthetic image of the predicted outcome. DIPN is shown to be sample efficient when trained in simulation or with a real robotic system. The high accuracy of DIPN allows direct integration with a grasp network, yielding a robotic manipulation system capable of executing challenging clutter removal tasks while being trained in a fully self-supervised manner. The overall network demonstrates intelligent behavior in selecting proper actions between push and grasp for completing clutter removal tasks and significantly outperforms the previous state-of-the-art. Remarkably, DIPN achieves even better performance on the real robotic hardware system than in simulation. Videos, code, and experiments log are available at https://github.com/rutgers-arc-lab/dipn.