FR-GESTURE: An RGBD Dataset For Gesture-based Human-Robot Interaction In First Responder Operations
This dataset addresses the need for gesture-based human-robot interaction in first responder operations, though it is incremental as it builds on existing gesture concepts.
The authors tackled the problem of facilitating first responder operations by creating FR-GESTURE, an RGBD dataset with 3312 pairs for gesture-based unmanned ground vehicle control, introducing 12 commands and baseline experiments for future improvement.
The ever increasing intensity and number of disasters make even more difficult the work of First Responders (FRs). Artificial intelligence and robotics solutions could facilitate their operations, compensating these difficulties. To this end, we propose a dataset for gesture-based UGV control by FRs, introducing a set of 12 commands, drawing inspiration from existing gestures used by FRs and tactical hand signals and refined after incorporating feedback from experienced FRs. Then we proceed with the data collection itself, resulting in 3312 RGBD pairs captured from 2 viewpoints and 7 distances. To the best of our knowledge, this is the first dataset especially intended for gesture-based UGV guidance by FRs. Finally we define evaluation protocols for our RGBD dataset, termed FR-GESTURE, and we perform baseline experiments, which are put forward for improvement. We have made data publicly available to promote future research on the domain: https://doi.org/10.5281/zenodo.18131333.