Transferring Experience from Simulation to the Real World for Precise Pick-And-Place Tasks in Highly Cluttered Scenes
This work addresses the challenge of robotic pick-and-place tasks in cluttered environments, offering an incremental improvement by combining simulation training with domain randomization for real-world transfer.
The paper tackles the problem of grasping and precisely placing known rigid objects in cluttered scenes by introducing a learning-based approach that transfers experience from simulation to the real world using domain randomization. The result is a Placement Quality Network (PQ-Net) that estimates object pose and grasp quality at 92 fps, outperforming other model-free methods in grasping success rate and scaling to new objects without human intervention.
In this paper, we introduce a novel learning-based approach for grasping known rigid objects in highly cluttered scenes and precisely placing them based on depth images. Our Placement Quality Network (PQ-Net) estimates the object pose and the quality for each automatically generated grasp pose for multiple objects simultaneously at 92 fps in a single forward pass of a neural network. All grasping and placement trials are executed in a physics simulation and the gained experience is transferred to the real world using domain randomization. We demonstrate that our policy successfully transfers to the real world. PQ-Net outperforms other model-free approaches in terms of grasping success rate and automatically scales to new objects of arbitrary symmetry without any human intervention.