Predicting and Attending to Damaging Collisions for Placing Everyday Objects in Photo-Realistic Simulations
This addresses a fundamental task for domestic service robots, but appears incremental as it builds on existing multimodal and attention mechanisms for a specific domain.
The paper tackles the problem of predicting damaging collisions when placing objects in cluttered home environments for domestic service robots, and shows that their method, PonNet, improves accuracy over baseline approaches using a dataset of 12,000 photo-realistic images.
Placing objects is a fundamental task for domestic service robots (DSRs). Thus, inferring the collision-risk before a placing motion is crucial for achieving the requested task. This problem is particularly challenging because it is necessary to predict what happens if an object is placed in a cluttered designated area. We show that a rule-based approach that uses plane detection, to detect free areas, performs poorly. To address this, we develop PonNet, which has multimodal attention branches and a self-attention mechanism to predict damaging collisions, based on RGBD images. Our method can visualize the risk of damaging collisions, which is convenient because it enables the user to understand the risk. For this purpose, we build and publish an original dataset that contains 12,000 photo-realistic images of specific placing areas, with daily life objects, in home environments. The experimental results show that our approach improves accuracy compared with the baseline methods.