SQE: a Self Quality Evaluation Metric for Parameters Optimization in Multi-Object Tracking
This work addresses a practical limitation in multi-object tracking by allowing optimization in test or real-world scenarios where ground truth is unavailable, though it is incremental as it builds on existing tracking frameworks.
The paper tackles the problem of evaluating multi-object tracking performance without ground truth annotations by introducing a self quality evaluation metric (SQE) that uses a two-class Gaussian mixture model based on feature distance distributions. Experiments on MOT16 datasets show SQE correlates with existing metrics and enables parameter self-optimization to improve tracking performance.
We present a novel self quality evaluation metric SQE for parameters optimization in the challenging yet critical multi-object tracking task. Current evaluation metrics all require annotated ground truth, thus will fail in the test environment and realistic circumstances prohibiting further optimization after training. By contrast, our metric reflects the internal characteristics of trajectory hypotheses and measures tracking performance without ground truth. We demonstrate that trajectories with different qualities exhibit different single or multiple peaks over feature distance distribution, inspiring us to design a simple yet effective method to assess the quality of trajectories using a two-class Gaussian mixture model. Experiments mainly on MOT16 Challenge data sets verify the effectiveness of our method in both correlating with existing metrics and enabling parameters self-optimization to achieve better performance. We believe that our conclusions and method are inspiring for future multi-object tracking in practice.