ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging
This provides a large-scale dataset for analyzing and predicting grasp contact, addressing a bottleneck in robotics and human-computer interaction, though it is incremental as it builds on existing sensing and modeling techniques.
The paper tackles the challenge of observing hand-object contact during grasping by introducing ContactDB, a novel dataset of contact maps for household objects captured via thermal imaging, which includes 3750 3D meshes and 375K frames of synchronized images, and shows that state-of-the-art algorithms can predict diverse contact patterns from object shape.
Grasping and manipulating objects is an important human skill. Since hand-object contact is fundamental to grasping, capturing it can lead to important insights. However, observing contact through external sensors is challenging because of occlusion and the complexity of the human hand. We present ContactDB, a novel dataset of contact maps for household objects that captures the rich hand-object contact that occurs during grasping, enabled by use of a thermal camera. Participants in our study grasped 3D printed objects with a post-grasp functional intent. ContactDB includes 3750 3D meshes of 50 household objects textured with contact maps and 375K frames of synchronized RGB-D+thermal images. To the best of our knowledge, this is the first large-scale dataset that records detailed contact maps for human grasps. Analysis of this data shows the influence of functional intent and object size on grasping, the tendency to touch/avoid 'active areas', and the high frequency of palm and proximal finger contact. Finally, we train state-of-the-art image translation and 3D convolution algorithms to predict diverse contact patterns from object shape. Data, code and models are available at https://contactdb.cc.gatech.edu.