Chittron: An Automatic Bangla Image Captioning System
It addresses the problem of limited resources for Bangla image captioning, which is incremental as it applies existing methods to a new language and dataset.
The paper tackles the lack of Bangla image captioning systems by developing 'Chittron', which uses a VGG16 and LSTM model trained on a new dataset of 16,000 manually annotated images, generating captions accurately in many cases with reported BLEU scores.
Automatic image caption generation aims to produce an accurate description of an image in natural language automatically. However, Bangla, the fifth most widely spoken language in the world, is lagging considerably in the research and development of such domain. Besides, while there are many established data sets to related to image annotation in English, no such resource exists for Bangla yet. Hence, this paper outlines the development of "Chittron", an automatic image captioning system in Bangla. Moreover, to address the data set availability issue, a collection of 16,000 Bangladeshi contextual images has been accumulated and manually annotated in Bangla. This data set is then used to train a model which integrates a pre-trained VGG16 image embedding model with stacked LSTM layers. The model is trained to predict the caption when the input is an image, one word at a time. The results show that the model has successfully been able to learn a working language model and to generate captions of images quite accurately in many cases. The results are evaluated mainly qualitatively. However, BLEU scores are also reported. It is expected that a better result can be obtained with a bigger and more varied data set.