Benchmarking person re-identification datasets and approaches for practical real-world implementations
This addresses the challenge of practical deployment for security and surveillance systems, but it is incremental as it focuses on benchmarking rather than proposing new methods.
The paper tackles the problem of domain shift in person re-identification models when deployed in new environments, finding that performance decreases significantly due to differences in geographic and detection factors, with benchmarks showing up to 30% drops in accuracy.
Recently, Person Re-Identification (Re-ID) has received a lot of attention. Large datasets containing labeled images of various individuals have been released, allowing researchers to develop and test many successful approaches. However, when such Re-ID models are deployed in new cities or environments, the task of searching for people within a network of security cameras is likely to face an important domain shift, thus resulting in decreased performance. Indeed, while most public datasets were collected in a limited geographic area, images from a new city present different features (e.g., people's ethnicity and clothing style, weather, architecture, etc.). In addition, the whole frames of the video streams must be converted into cropped images of people using pedestrian detection models, which behave differently from the human annotators who created the dataset used for training. To better understand the extent of this issue, this paper introduces a complete methodology to evaluate Re-ID approaches and training datasets with respect to their suitability for unsupervised deployment for live operations. This method is used to benchmark four Re-ID approaches on three datasets, providing insight and guidelines that can help to design better Re-ID pipelines in the future.