SemAug: Semantically Meaningful Image Augmentations for Object Detection Through Language Grounding
This work addresses the need for more effective data augmentation in object detection, offering a novel approach that introduces new object instances and categories without requiring additional training overhead, though it is incremental relative to existing augmentation techniques.
The paper tackled the problem of improving object detection generalization by proposing SemAug, a semantically meaningful image augmentation method that injects contextually appropriate objects into scenes using language grounding, achieving ~2-4% mAP improvement on Pascal VOC and ~1-2% on COCO datasets.
Data augmentation is an essential technique in improving the generalization of deep neural networks. The majority of existing image-domain augmentations either rely on geometric and structural transformations, or apply different kinds of photometric distortions. In this paper, we propose an effective technique for image augmentation by injecting contextually meaningful knowledge into the scenes. Our method of semantically meaningful image augmentation for object detection via language grounding, SemAug, starts by calculating semantically appropriate new objects that can be placed into relevant locations in the image (the what and where problems). Then it embeds these objects into their relevant target locations, thereby promoting diversity of object instance distribution. Our method allows for introducing new object instances and categories that may not even exist in the training set. Furthermore, it does not require the additional overhead of training a context network, so it can be easily added to existing architectures. Our comprehensive set of evaluations showed that the proposed method is very effective in improving the generalization, while the overhead is negligible. In particular, for a wide range of model architectures, our method achieved ~2-4% and ~1-2% mAP improvements for the task of object detection on the Pascal VOC and COCO datasets, respectively.