Robust pedestrian detection in thermal imagery using synthesized images
This work provides a method for improving pedestrian detection in thermal imagery, which is crucial for applications like autonomous driving and surveillance in low-light conditions, by overcoming the limitation of scarce labeled thermal data.
This paper addresses the problem of pedestrian detection in thermal imagery by synthesizing thermal images from RGB inputs using a generative adversarial network. This approach allows a YOLOv3 detector to achieve state-of-the-art results on the KAIST Multispectral Pedestrian Detection Benchmark, even when using less than 50% of the available real thermal training data.
In this paper we propose a method for improving pedestrian detection in the thermal domain using two stages: first, a generative data augmentation approach is used, then a domain adaptation method using generated data adapts an RGB pedestrian detector. Our model, based on the Least-Squares Generative Adversarial Network, is trained to synthesize realistic thermal versions of input RGB images which are then used to augment the limited amount of labeled thermal pedestrian images available for training. We apply our generative data augmentation strategy in order to adapt a pretrained YOLOv3 pedestrian detector to detection in the thermal-only domain. Experimental results demonstrate the effectiveness of our approach: using less than 50\% of available real thermal training data, and relying on synthesized data generated by our model in the domain adaptation phase, our detector achieves state-of-the-art results on the KAIST Multispectral Pedestrian Detection Benchmark; even if more real thermal data is available adding GAN generated images to the training data results in improved performance, thus showing that these images act as an effective form of data augmentation. To the best of our knowledge, our detector achieves the best single-modality detection results on KAIST with respect to the state-of-the-art.