Modeling Thousands of Human Annotators for Generalizable Text-to-Image Person Re-identification
This addresses the high annotation cost and limited generalization in text-to-image person re-identification, though it is incremental by enhancing existing MLLM-based methods.
The paper tackles the lack of diversity in captions generated by Multi-modal Large Language Models for text-to-image person re-identification by proposing a Human Annotator Modeling approach to mimic thousands of human description styles, which significantly improves the generalization ability of ReID models as shown in extensive experiments.
Text-to-image person re-identification (ReID) aims to retrieve the images of an interested person based on textual descriptions. One main challenge for this task is the high cost in manually annotating large-scale databases, which affects the generalization ability of ReID models. Recent works handle this problem by leveraging Multi-modal Large Language Models (MLLMs) to describe pedestrian images automatically. However, the captions produced by MLLMs lack diversity in description styles. To address this issue, we propose a Human Annotator Modeling (HAM) approach to enable MLLMs to mimic the description styles of thousands of human annotators. Specifically, we first extract style features from human textual descriptions and perform clustering on them. This allows us to group textual descriptions with similar styles into the same cluster. Then, we employ a prompt to represent each of these clusters and apply prompt learning to mimic the description styles of different human annotators. Furthermore, we define a style feature space and perform uniform sampling in this space to obtain more diverse clustering prototypes, which further enriches the diversity of the MLLM-generated captions. Finally, we adopt HAM to automatically annotate a massive-scale database for text-to-image ReID. Extensive experiments on this database demonstrate that it significantly improves the generalization ability of ReID models.