Generating captions without looking beyond objects
This work provides incremental insights into evaluation and word category contributions in image captioning, primarily benefiting researchers in computer vision and natural language processing.
The paper introduces a noun translation task for image captioning, showing that translating from nouns to captions achieves comparable performance to full captioning, indicating that non-noun words can be generated by a language model without losing n-gram precision. It also analyzes the contribution of different word categories to BLEU scores, identifying potential improvements for nouns, verbs, and prepositions.
This paper explores new evaluation perspectives for image captioning and introduces a noun translation task that achieves comparative image caption generation performance by translating from a set of nouns to captions. This implies that in image captioning, all word categories other than nouns can be evoked by a powerful language model without sacrificing performance on n-gram precision. The paper also investigates lower and upper bounds of how much individual word categories in the captions contribute to the final BLEU score. A large possible improvement exists for nouns, verbs, and prepositions.