MOTCOM: The Multi-Object Tracking Dataset Complexity Metric
This addresses a problem for MOT researchers by providing a tool to improve explainability and dataset comparisons, though it is incremental as it builds on existing sub-problems in the field.
The paper tackles the lack of a comprehensive metric for describing the complexity of Multi-Object Tracking (MOT) sequences by introducing MOTCOM, a novel metric combining occlusion, erratic motion, and visual similarity, and shows it outperforms conventional metrics like density and number of tracks on datasets such as MOT17, MOT20, and MOTSynth.
There exists no comprehensive metric for describing the complexity of Multi-Object Tracking (MOT) sequences. This lack of metrics decreases explainability, complicates comparison of datasets, and reduces the conversation on tracker performance to a matter of leader board position. As a remedy, we present the novel MOT dataset complexity metric (MOTCOM), which is a combination of three sub-metrics inspired by key problems in MOT: occlusion, erratic motion, and visual similarity. The insights of MOTCOM can open nuanced discussions on tracker performance and may lead to a wider acknowledgement of novel contributions developed for either less known datasets or those aimed at solving sub-problems. We evaluate MOTCOM on the comprehensive MOT17, MOT20, and MOTSynth datasets and show that MOTCOM is far better at describing the complexity of MOT sequences compared to the conventional density and number of tracks. Project page at https://vap.aau.dk/motcom