SubICap: Towards Subword-informed Image Captioning
This work provides an incremental improvement for image captioning systems struggling with rare and out-of-vocabulary words, particularly benefiting models that need to operate with smaller vocabularies.
This paper addresses the limitation of image captioning systems in handling rare and out-of-vocabulary words by decomposing words into subwords. This approach allows for the representation of all words in a corpus with a significantly smaller subword vocabulary, leading to improved metric scores and a training vocabulary size approximately 90% less than baseline models.
Existing Image Captioning (IC) systems model words as atomic units in captions and are unable to exploit the structural information in the words. This makes representation of rare words very difficult and out-of-vocabulary words impossible. Moreover, to avoid computational complexity, existing IC models operate over a modest sized vocabulary of frequent words, such that the identity of rare words is lost. In this work we address this common limitation of IC systems in dealing with rare words in the corpora. We decompose words into smaller constituent units 'subwords' and represent captions as a sequence of subwords instead of words. This helps represent all words in the corpora using a significantly lower subword vocabulary, leading to better parameter learning. Using subword language modeling, our captioning system improves various metric scores, with a training vocabulary size approximately 90% less than the baseline and various state-of-the-art word-level models. Our quantitative and qualitative results and analysis signify the efficacy of our proposed approach.