MODS -- A USV-oriented object detection and obstacle segmentation benchmark
This work addresses the problem of inconsistent evaluation and limited data for USV obstacle detection, which hinders cross-paper comparisons and practical navigation improvements, though it is incremental as it builds on existing maritime and ground vehicle datasets.
The authors tackled the lack of standardized datasets and evaluation protocols for obstacle detection in unmanned surface vehicles (USVs) by introducing MODS, a new benchmark with a diverse dataset of approximately 81k stereo images and over 60k annotated objects, and evaluating 19 state-of-the-art methods to facilitate progress in the field.
Small-sized unmanned surface vehicles (USV) are coastal water devices with a broad range of applications such as environmental control and surveillance. A crucial capability for autonomous operation is obstacle detection for timely reaction and collision avoidance, which has been recently explored in the context of camera-based visual scene interpretation. Owing to curated datasets, substantial advances in scene interpretation have been made in a related field of unmanned ground vehicles. However, the current maritime datasets do not adequately capture the complexity of real-world USV scenes and the evaluation protocols are not standardised, which makes cross-paper comparison of different methods difficult and hinders the progress. To address these issues, we introduce a new obstacle detection benchmark MODS, which considers two major perception tasks: maritime object detection and the more general maritime obstacle segmentation. We present a new diverse maritime evaluation dataset containing approximately 81k stereo images synchronized with an on-board IMU, with over 60k objects annotated. We propose a new obstacle segmentation performance evaluation protocol that reflects the detection accuracy in a way meaningful for practical USV navigation. Nineteen recent state-of-the-art object detection and obstacle segmentation methods are evaluated using the proposed protocol, creating a benchmark to facilitate development of the field. The proposed dataset, as well as evaluation routines, are made publicly available at vicos.si/resources.