CADDY Underwater Stereo-Vision Dataset for Human-Robot Interaction (HRI) in the Context of Diver Activities
This dataset enables research in underwater human-robot interaction by providing a large-scale resource for testing algorithms against distortions like color attenuation and light backscatter, though it is incremental as it builds on existing dataset efforts in robotics.
The authors introduced the CADDY underwater stereo-vision dataset, collected from field trials with an AUV interacting with divers, to address tasks like object classification, segmentation, and human pose estimation in challenging underwater conditions. The dataset includes over 22,000 stereo images with gestures and free-swimming footage, along with synchronized IMU data for ground-truth pose estimation.
In this article we present a novel underwater dataset collected from several field trials within the EU FP7 project "Cognitive autonomous diving buddy (CADDY)", where an Autonomous Underwater Vehicle (AUV) was used to interact with divers and monitor their activities. To our knowledge, this is one of the first efforts to collect a large dataset in underwater environments targeting object classification, segmentation and human pose estimation tasks. The first part of the dataset contains stereo camera recordings (~10K) of divers performing hand gestures to communicate and interact with an AUV in different environmental conditions. These gestures samples serve to test the robustness of object detection and classification algorithms against underwater image distortions i.e., color attenuation and light backscatter. The second part includes stereo footage (~12.7K) of divers free-swimming in front of the AUV, along with synchronized IMUs measurements located throughout the diver's suit (DiverNet) which serve as ground-truth for human pose and tracking methods. In both cases, these rectified images allow investigation of 3D representation and reasoning pipelines from low-texture targets commonly present in underwater scenarios. In this paper we describe our recording platform, sensor calibration procedure plus the data format and the utilities provided to use the dataset.