SuperCaptioning: Image Captioning Using Two-dimensional Word Embedding
This is an incremental improvement for image captioning, applying an existing text classification technique to a vision-language task.
The paper tackles image captioning by integrating language and vision into a single CNN model using two-dimensional word embeddings, achieving high-quality captions on the Flickr30k dataset.
Language and vision are processed as two different modal in current work for image captioning. However, recent work on Super Characters method shows the effectiveness of two-dimensional word embedding, which converts text classification problem into image classification problem. In this paper, we propose the SuperCaptioning method, which borrows the idea of two-dimensional word embedding from Super Characters method, and processes the information of language and vision together in one single CNN model. The experimental results on Flickr30k data shows the proposed method gives high quality image captions. An interactive demo is ready to show at the workshop.