Building 3D Object Models during Manipulation by Reconstruction-Aware Trajectory Optimization
This work addresses the tedious process of acquiring 3D models for multiple objects in robotics, offering an incremental improvement by integrating reconstruction into manipulation tasks.
The paper tackles the problem of obtaining accurate 3D object models for robotic manipulation by proposing a reconstruction-aware trajectory optimization that uses Gaussian process implicit surfaces (GPIS) to plan motions that improve geometry estimation during pick-and-place tasks, validated on eleven YCB objects with higher-quality reconstruction compared to baselines.
Object shape provides important information for robotic manipulation; for instance, selecting an effective grasp depends on both the global and local shape of the object of interest, while reaching into clutter requires accurate surface geometry to avoid unintended contact with the environment. Model-based 3D object manipulation is a widely studied problem; however, obtaining the accurate 3D object models for multiple objects often requires tedious work. In this letter, we exploit Gaussian process implicit surfaces (GPIS) extracted from RGB-D sensor data to grasp an unknown object. We propose a reconstruction-aware trajectory optimization that makes use of the extracted GPIS model plan a motion to improve the ability to estimate the object's 3D geometry, while performing a pick-and-place action. We present a probabilistic approach for a robot to autonomously learn and track the object, while achieve the manipulation task. We use a sampling-based trajectory generation method to explore the unseen parts of the object using the estimated conditional entropy of the GPIS model. We validate our method with physical robot experiments across eleven different objects of varying shape from the YCB object dataset. Our experiments show that our reconstruction-aware trajectory optimization provides higher-quality 3D object reconstruction when compared with directly solving the manipulation task or using a heuristic to view unseen portions of the object.