Borrow from Anywhere: Pseudo Multi-modal Object Detection in Thermal Imagery
This work addresses the challenge of limited data in thermal imaging for object detection, offering a practical solution for applications like surveillance or autonomous systems, though it is incremental as it builds on existing image-to-image translation and multi-modal methods.
The paper tackles the problem of improving object detection in thermal imagery by leveraging features from the visual RGB domain, achieving superior performance over existing benchmarks without requiring paired training examples and demonstrating the ability to learn effectively with less thermal data.
Can we improve detection in the thermal domain by borrowing features from rich domains like visual RGB? In this paper, we propose a pseudo-multimodal object detector trained on natural image domain data to help improve the performance of object detection in thermal images. We assume access to a large-scale dataset in the visual RGB domain and relatively smaller dataset (in terms of instances) in the thermal domain, as is common today. We propose the use of well-known image-to-image translation frameworks to generate pseudo-RGB equivalents of a given thermal image and then use a multi-modal architecture for object detection in the thermal image. We show that our framework outperforms existing benchmarks without the explicit need for paired training examples from the two domains. We also show that our framework has the ability to learn with less data from thermal domain when using our approach. Our code and pre-trained models are made available at https://github.com/tdchaitanya/MMTOD