Beyond standard benchmarks: Parameterizing performance evaluation in visual object tracking
This work provides a more robust evaluation framework for visual object tracking researchers, though it is incremental as it builds on existing benchmarking methods.
The paper tackles the problem of evaluating visual object trackers by addressing the limitations of standard benchmarks, which have poorly defined motion attributes, and introduces a new evaluation system using omnidirectional videos to generate parameterized motion patterns, resulting in a complementary analysis of tracking paradigms.
Object-to-camera motion produces a variety of apparent motion patterns that significantly affect performance of short-term visual trackers. Despite being crucial for designing robust trackers, their influence is poorly explored in standard benchmarks due to weakly defined, biased and overlapping attribute annotations. In this paper we propose to go beyond pre-recorded benchmarks with post-hoc annotations by presenting an approach that utilizes omnidirectional videos to generate realistic, consistently annotated, short-term tracking scenarios with exactly parameterized motion patterns. We have created an evaluation system, constructed a fully annotated dataset of omnidirectional videos and the generators for typical motion patterns. We provide an in-depth analysis of major tracking paradigms which is complementary to the standard benchmarks and confirms the expressiveness of our evaluation approach.