Assistive Image Annotation Systems with Deep Learning and Natural Language Capabilities: A Review
It addresses the bottleneck of high-quality annotated data in computer vision for researchers and practitioners, but is incremental as it reviews existing works without introducing new methods.
This paper reviews AI-assistive deep learning systems for image annotation that use natural language to provide textual suggestions, aiming to improve annotation efficiency and quality for tasks like classification and segmentation, but notes limited publicly available work in this area.
While supervised learning has achieved significant success in computer vision tasks, acquiring high-quality annotated data remains a bottleneck. This paper explores both scholarly and non-scholarly works in AI-assistive deep learning image annotation systems that provide textual suggestions, captions, or descriptions of the input image to the annotator. This potentially results in higher annotation efficiency and quality. Our exploration covers annotation for a range of computer vision tasks including image classification, object detection, regression, instance, semantic segmentation, and pose estimation. We review various datasets and how they contribute to the training and evaluation of AI-assistive annotation systems. We also examine methods leveraging neuro-symbolic learning, deep active learning, and self-supervised learning algorithms that enable semantic image understanding and generate free-text output. These include image captioning, visual question answering, and multi-modal reasoning. Despite the promising potential, there is limited publicly available work on AI-assistive image annotation with textual output capabilities. We conclude by suggesting future research directions to advance this field, emphasizing the need for more publicly accessible datasets and collaborative efforts between academia and industry.