Intrinsic Image Captioning Evaluation
This work is significant for researchers and developers in image captioning, offering a complementary evaluation metric that better captures intrinsic information between captions.
The paper addresses the lack of comprehensive metrics for evaluating image captioning models by proposing a learning-based metric called Intrinsic Image Captioning Evaluation (I2CE). Experiments on the MS COCO dataset show that I2CE provides robust and flexible scores, particularly for semantically similar or less aligned captions.
The image captioning task is about to generate suitable descriptions from images. For this task there can be several challenges such as accuracy, fluency and diversity. However there are few metrics that can cover all these properties while evaluating results of captioning models.In this paper we first conduct a comprehensive investigation on contemporary metrics. Motivated by the auto-encoder mechanism and the research advances of word embeddings we propose a learning based metrics for image captioning, which we call Intrinsic Image Captioning Evaluation(I2CE). We select several state-of-the-art image captioning models and test their performances on MS COCO dataset with respects to both contemporary metrics and the proposed I2CE. Experiment results show that our proposed method can keep robust performance and give more flexible scores to candidate captions when encountered with semantic similar expression or less aligned semantics. On this concern the proposed metric could serve as a novel indicator on the intrinsic information between captions, which may be complementary to the existing ones.