Symbiotic Adversarial Learning for Attribute-based Person Search
This addresses the challenge of identifying persons from attribute descriptions in applications like criminal identification, though it is incremental as it builds on existing GAN-based methods.
The paper tackles the problem of attribute-based person search, where there is a modality gap between images and attribute descriptions and many unseen attribute combinations, by proposing a symbiotic adversarial learning framework that uses two GANs to synthesize unseen features and optimize cross-modal embeddings, achieving state-of-the-art results on benchmarks like PETA and Market-1501.
Attribute-based person search is in significant demand for applications where no detected query images are available, such as identifying a criminal from witness. However, the task itself is quite challenging because there is a huge modality gap between images and physical descriptions of attributes. Often, there may also be a large number of unseen categories (attribute combinations). The current state-of-the-art methods either focus on learning better cross-modal embeddings by mining only seen data, or they explicitly use generative adversarial networks (GANs) to synthesize unseen features. The former tends to produce poor embeddings due to insufficient data, while the latter does not preserve intra-class compactness during generation. In this paper, we present a symbiotic adversarial learning framework, called SAL.Two GANs sit at the base of the framework in a symbiotic learning scheme: one synthesizes features of unseen classes/categories, while the other optimizes the embedding and performs the cross-modal alignment on the common embedding space .Specifically, two different types of generative adversarial networks learn collaboratively throughout the training process and the interactions between the two mutually benefit each other. Extensive evaluations show SAL's superiority over nine state-of-the-art methods with two challenging pedestrian benchmarks, PETA and Market-1501. The code is publicly available at: https://github.com/ycao5602/SAL .