Person Re-Identification with Vision and Language
This addresses the problem of identifying individuals across different camera views for security and surveillance applications, representing an incremental advance by combining existing modalities.
The paper tackles person re-identification by integrating images and natural language descriptions, proposing a joint vision and language model that improves state-of-the-art performance on standard benchmarks like CUHK03 and VIPeR.
In this paper we propose a new approach to person re-identification using images and natural language descriptions. We propose a joint vision and language model based on CCA and CNN architectures to match across the two modalities as well as to enrich visual examples for which there are no language descriptions. We also introduce new annotations in the form of natural language descriptions for two standard Re-ID benchmarks, namely CUHK03 and VIPeR. We perform experiments on these two datasets with techniques based on CNN, hand-crafted features as well as LSTM for analysing visual and natural description data. We investigate and demonstrate the advantages of using natural language descriptions compared to attributes as well as CNN compared to LSTM in the context of Re-ID. We show that the joint use of language and vision can significantly improve the state-of-the-art performance on standard Re-ID benchmarks.