EgoPressure: A Dataset for Hand Pressure and Pose Estimation in Egocentric Vision
This addresses the lack of comprehensive datasets for touch contact and pressure estimation in egocentric vision, benefiting mixed reality and robotics applications, but it is incremental as it primarily provides a new dataset.
The paper tackles the challenge of estimating hand pressure and pose from egocentric vision by introducing EgoPressure, a novel dataset with 5 hours of interactions from 21 participants, providing high-resolution pressure annotations and accurate hand pose meshes, and baseline models show that pressure and hand pose are complementary for understanding interactions.
Touch contact and pressure are essential for understanding how humans interact with and manipulate objects, insights which can significantly benefit applications in mixed reality and robotics. However, estimating these interactions from an egocentric camera perspective is challenging, largely due to the lack of comprehensive datasets that provide both accurate hand poses on contacting surfaces and detailed annotations of pressure information. In this paper, we introduce EgoPressure, a novel egocentric dataset that captures detailed touch contact and pressure interactions. EgoPressure provides high-resolution pressure intensity annotations for each contact point and includes accurate hand pose meshes obtained through our proposed multi-view, sequence-based optimization method processing data from an 8-camera capture rig. Our dataset comprises 5 hours of recorded interactions from 21 participants captured simultaneously by one head-mounted and seven stationary Kinect cameras, which acquire RGB images and depth maps at 30 Hz. To support future research and benchmarking, we present several baseline models for estimating applied pressure on external surfaces from RGB images, with and without hand pose information. We further explore the joint estimation of the hand mesh and applied pressure. Our experiments demonstrate that pressure and hand pose are complementary for understanding hand-object interactions. ng of hand-object interactions in AR/VR and robotics research. Project page: \url{https://yiming-zhao.github.io/EgoPressure/}.