DeepSea MOT: A benchmark dataset for multi-object tracking on deep-sea video
This provides a benchmark for researchers in marine science and computer vision to evaluate multi-object tracking models on deep-sea footage, though it is incremental as it applies existing methods to new data.
The authors tackled the lack of benchmark datasets for multi-object tracking in deep-sea video by creating a novel dataset and evaluating object detection models and trackers, achieving performance assessed using the Higher Order Tracking Accuracy metric.
Benchmarking multi-object tracking and object detection model performance is an essential step in machine learning model development, as it allows researchers to evaluate model detection and tracker performance on human-generated 'test' data, facilitating consistent comparisons between models and trackers and aiding performance optimization. In this study, a novel benchmark video dataset was developed and used to assess the performance of several Monterey Bay Aquarium Research Institute object detection models and a FathomNet single-class object detection model together with several trackers. The dataset consists of four video sequences representing midwater and benthic deep-sea habitats. Performance was evaluated using Higher Order Tracking Accuracy, a metric that balances detection, localization, and association accuracy. To the best of our knowledge, this is the first publicly available benchmark for multi-object tracking in deep-sea video footage. We provide the benchmark data, a clearly documented workflow for generating additional benchmark videos, as well as example Python notebooks for computing metrics.