VITAL: A Visual Interpretation on Text with Adversarial Learning for Image Labeling
This addresses the challenge of enhancing image labeling accuracy by integrating visual interpretations from text, though it appears incremental as it builds on existing GAN methods.
The paper tackles the problem of interpreting text information for image labeling by using visual features from synthetic images generated by a modified Text-to-Image GAN, resulting in improved performance on two benchmark datasets.
In this paper, we propose a novel way to interpret text information by extracting visual feature presentation from multiple high-resolution and photo-realistic synthetic images generated by Text-to-image Generative Adversarial Network (GAN) to improve the performance of image labeling. Firstly, we design a stacked Generative Multi-Adversarial Network (GMAN), StackGMAN++, a modified version of the current state-of-the-art Text-to-image GAN, StackGAN++, to generate multiple synthetic images with various prior noises conditioned on a text. And then we extract deep visual features from the generated synthetic images to explore the underlying visual concepts for text. Finally, we combine image-level visual feature, text-level feature and visual features based on synthetic images together to predict labels for images. We conduct experiments on two benchmark datasets and the experimental results clearly demonstrate the efficacy of our proposed approach.