ORCA: Object Recognition and Comprehension for Archiving Marine Species
This provides a comprehensive benchmark for advancing marine domain research, addressing a domain-specific problem for marine ecosystem monitoring and protection.
The authors tackled the lack of training data and systematic task formulation for marine visual understanding by introducing ORCA, a multi-modal benchmark with 14,647 images, 42,217 bounding box annotations, and 22,321 instance captions, and they evaluated 18 state-of-the-art models on three tasks, highlighting challenges like species diversity and morphological overlap.
Marine visual understanding is essential for monitoring and protecting marine ecosystems, enabling automatic and scalable biological surveys. However, progress is hindered by limited training data and the lack of a systematic task formulation that aligns domain-specific marine challenges with well-defined computer vision tasks, thereby limiting effective model application. To address this gap, we present ORCA, a multi-modal benchmark for marine research comprising 14,647 images from 478 species, with 42,217 bounding box annotations and 22,321 expert-verified instance captions. The dataset provides fine-grained visual and textual annotations that capture morphology-oriented attributes across diverse marine species. To catalyze methodological advances, we evaluate 18 state-of-the-art models on three tasks: object detection (closed-set and open-vocabulary), instance captioning, and visual grounding. Results highlight key challenges, including species diversity, morphological overlap, and specialized domain demands, underscoring the difficulty of marine understanding. ORCA thus establishes a comprehensive benchmark to advance research in marine domain. Project Page: http://orca.hkustvgd.com/.