Generalization Limits in Vehicle Re-Identification
For researchers in vehicle re-identification, this work highlights the overestimation of generalization in current benchmarks and provides a more realistic evaluation framework.
The paper identifies that current vehicle re-identification datasets contain similar vehicles in both training and test sets, leading to methods that memorize rather than generalize. The authors propose a new evaluation approach that measures generalization to unseen vehicle types and a view-based split, finding that state-of-the-art methods struggle with unseen types and limited viewpoint robustness.
Vehicle re-identification focuses on retrieving images of the same vehicle from a gallery given a query image. Upon closer inspection of commonly used datasets, we observe that vehicles with few visual differences-e.g., the same make, model, and color-appear in both the training and test sets. As a result, methods that effectively memorize the training data tend to perform well on these test sets but struggle to generalize to other datasets. In this paper, we address this issue by proposing a novel evaluation approach that more effectively measures generalization capability to unseen vehicle types. To further study generalization performance, we also propose splitting the evaluation based on view, allowing us to differentiate the effect of viewpoint robustness from that of same-view re-identification. Our findings reveal that most state-of-the-art methods struggle with unseen vehicle types, and that their robustness to viewpoint changes and attention to detail are limited to vehicle types seen during training.