Alleviating Noisy Data in Image Captioning with Cooperative Distillation
This work addresses the problem of generating more expressive image captions for users by mitigating the impact of noisy web-scale datasets, which is an incremental improvement in data utilization.
This paper addresses the problem of noisy data in image captioning by proposing cooperative distillation, a technique that combines clean, curated datasets with the large-scale, automatically extracted, and potentially noisy Google Conceptual Captions dataset. This approach aims to leverage the rich vocabulary of the larger dataset to produce more expressive captions while maintaining accuracy.
Image captioning systems have made substantial progress, largely due to the availability of curated datasets like Microsoft COCO or Vizwiz that have accurate descriptions of their corresponding images. Unfortunately, scarce availability of such cleanly labeled data results in trained algorithms producing captions that can be terse and idiosyncratically specific to details in the image. We propose a new technique, cooperative distillation that combines clean curated datasets with the web-scale automatically extracted captions of the Google Conceptual Captions dataset (GCC), which can have poor descriptions of images, but is abundant in size and therefore provides a rich vocabulary resulting in more expressive captions.