Synthesizing Novel Pairs of Image and Text
This work addresses the challenge of synthesizing paired data for multimodal AI applications, but it appears incremental as it builds on existing methods without claiming major breakthroughs.
The paper tackles the problem of generating novel image-text pairs by leveraging existing captioning datasets, using GANs and sequence-to-sequence models, and explores cycles between image and text generation along with connections to autoencoders.
Generating novel pairs of image and text is a problem that combines computer vision and natural language processing. In this paper, we present strategies for generating novel image and caption pairs based on existing captioning datasets. The model takes advantage of recent advances in generative adversarial networks and sequence-to-sequence modeling. We make generalizations to generate paired samples from multiple domains. Furthermore, we study cycles -- generating from image to text then back to image and vise versa, as well as its connection with autoencoders.