RefineCap: Concept-Aware Refinement for Image Captioning
This work addresses image captioning for applications like accessibility and content generation, but it is incremental as it builds on existing visual-concept methods.
The paper tackles the problem of generating semantically descriptive captions from images by refining the language decoder's vocabulary using visual semantics, achieving superior performance on the MS-COCO dataset compared to previous visual-concept based models.
Automatically translating images to texts involves image scene understanding and language modeling. In this paper, we propose a novel model, termed RefineCap, that refines the output vocabulary of the language decoder using decoder-guided visual semantics, and implicitly learns the mapping between visual tag words and images. The proposed Visual-Concept Refinement method can allow the generator to attend to semantic details in the image, thereby generating more semantically descriptive captions. Our model achieves superior performance on the MS-COCO dataset in comparison with previous visual-concept based models.