Simple Image Description Generator via a Linear Phrase-Based Approach
This work addresses the problem of generating textual descriptions from images for applications in computer vision and NLP, but it is incremental as it simplifies existing methods without major performance gains.
The paper tackles image caption generation by proposing a simple bilinear model that learns a metric between image features and phrases, combined with a language model based on syntax statistics, achieving results comparable to state-of-the-art models on the Microsoft COCO dataset.
Generating a novel textual description of an image is an interesting problem that connects computer vision and natural language processing. In this paper, we present a simple model that is able to generate descriptive sentences given a sample image. This model has a strong focus on the syntax of the descriptions. We train a purely bilinear model that learns a metric between an image representation (generated from a previously trained Convolutional Neural Network) and phrases that are used to described them. The system is then able to infer phrases from a given image sample. Based on caption syntax statistics, we propose a simple language model that can produce relevant descriptions for a given test image using the phrases inferred. Our approach, which is considerably simpler than state-of-the-art models, achieves comparable results on the recently release Microsoft COCO dataset.