Robust Gesture-Based Communication for Underwater Human-Robot Interaction in the context of Search and Rescue Diver Missions
This addresses the challenge of reliable human-robot interaction for divers in high-risk underwater tasks, such as search and rescue, though it appears incremental by building on standardized gestures and existing recognition methods.
The paper tackles the problem of enabling divers to communicate with Autonomous Underwater Vehicles (AUVs) using gestures in search and rescue missions, proposing a robust pipeline that includes a gesture language (CADDIAN), hierarchical classification for recognition, and syntax checks, with experimental results showing good performance in field tests under various conditions.
We propose a robust gesture-based communication pipeline for divers to instruct an Autonomous Underwater Vehicle (AUV) to assist them in performing high-risk tasks and helping in case of emergency. A gesture communication language (CADDIAN) is developed, based on consolidated and standardized diver gestures, including an alphabet, syntax and semantics, ensuring a logical consistency. A hierarchical classification approach is introduced for hand gesture recognition based on stereo imagery and multi-descriptor aggregation to specifically cope with underwater image artifacts, e.g. light backscatter or color attenuation. Once the classification task is finished, a syntax check is performed to filter out invalid command sequences sent by the diver or generated by errors in the classifier. Throughout this process, the diver receives constant feedback from an underwater tablet to acknowledge or abort the mission at any time. The objective is to prevent the AUV from executing unnecessary, infeasible or potentially harmful motions. Experimental results under different environmental conditions in archaeological exploration and bridge inspection applications show that the system performs well in the field.