Generative Prompt Model for Weakly Supervised Object Localization
This addresses the challenge of localizing full object extent from image category labels for computer vision applications, representing a novel generative approach in WSOL.
The paper tackles the problem of weakly supervised object localization (WSOL) by proposing a generative prompt model (GenPromp) that localizes less discriminative object parts, achieving improvements of 5.2% on CUB-200-2011 and 5.6% on ILSVRC in Top-1 Loc compared to best discriminative models.
Weakly supervised object localization (WSOL) remains challenging when learning object localization models from image category labels. Conventional methods that discriminatively train activation models ignore representative yet less discriminative object parts. In this study, we propose a generative prompt model (GenPromp), defining the first generative pipeline to localize less discriminative object parts by formulating WSOL as a conditional image denoising procedure. During training, GenPromp converts image category labels to learnable prompt embeddings which are fed to a generative model to conditionally recover the input image with noise and learn representative embeddings. During inference, enPromp combines the representative embeddings with discriminative embeddings (queried from an off-the-shelf vision-language model) for both representative and discriminative capacity. The combined embeddings are finally used to generate multi-scale high-quality attention maps, which facilitate localizing full object extent. Experiments on CUB-200-2011 and ILSVRC show that GenPromp respectively outperforms the best discriminative models by 5.2% and 5.6% (Top-1 Loc), setting a solid baseline for WSOL with the generative model. Code is available at https://github.com/callsys/GenPromp.