Text-based Person Search in Full Images via Semantic-Driven Proposal Generation
This addresses a practical gap in intelligent video surveillance by enabling person search directly from full images without pre-cropped bounding boxes, though it is incremental as it builds on existing person search datasets.
The paper tackles the problem of text-based person search in full scene images, where existing methods rely on cropped pedestrian images, by proposing an end-to-end framework that jointly optimizes detection, identification, and feature embedding, achieving state-of-the-art performance on two new benchmark datasets.
Finding target persons in full scene images with a query of text description has important practical applications in intelligent video surveillance.However, different from the real-world scenarios where the bounding boxes are not available, existing text-based person retrieval methods mainly focus on the cross modal matching between the query text descriptions and the gallery of cropped pedestrian images. To close the gap, we study the problem of text-based person search in full images by proposing a new end-to-end learning framework which jointly optimize the pedestrian detection, identification and visual-semantic feature embedding tasks. To take full advantage of the query text, the semantic features are leveraged to instruct the Region Proposal Network to pay more attention to the text-described proposals. Besides, a cross-scale visual-semantic embedding mechanism is utilized to improve the performance. To validate the proposed method, we collect and annotate two large-scale benchmark datasets based on the widely adopted image-based person search datasets CUHK-SYSU and PRW. Comprehensive experiments are conducted on the two datasets and compared with the baseline methods, our method achieves the state-of-the-art performance.