Technical Report: Image Captioning with Semantically Similar Images
This work addresses image captioning for vision-language tasks, but it is incremental as it builds on existing methods without major innovations.
The authors tackled image captioning by selecting captions from semantically similar images using CNN embeddings and unigram frequencies, achieving competitive performance in Turing test pass rates and human preference despite low automated scores.
This report presents our submission to the MS COCO Captioning Challenge 2015. The method uses Convolutional Neural Network activations as an embedding to find semantically similar images. From these images, the most typical caption is selected based on unigram frequencies. Although the method received low scores with automated evaluation metrics and in human assessed average correctness, it is competitive in the ratio of captions which pass the Turing test and which are assessed as better or equal to human captions.