Drone Object Detection Using RGB/IR Fusion
This work addresses the problem of object detection in low-light or occluded conditions for drone applications, but it is incremental as it builds on existing fusion and simulation methods.
The paper tackled the lack of training data for drone infrared (IR) imagery by developing strategies to create synthetic IR images using AIRSim and CycleGAN, and used an illumination-aware fusion framework for object detection, achieving processing times of about 28 milliseconds per image pair on an NVIDIA Jetson Xavier.
Object detection using aerial drone imagery has received a great deal of attention in recent years. While visible light images are adequate for detecting objects in most scenarios, thermal cameras can extend the capabilities of object detection to night-time or occluded objects. As such, RGB and Infrared (IR) fusion methods for object detection are useful and important. One of the biggest challenges in applying deep learning methods to RGB/IR object detection is the lack of available training data for drone IR imagery, especially at night. In this paper, we develop several strategies for creating synthetic IR images using the AIRSim simulation engine and CycleGAN. Furthermore, we utilize an illumination-aware fusion framework to fuse RGB and IR images for object detection on the ground. We characterize and test our methods for both simulated and actual data. Our solution is implemented on an NVIDIA Jetson Xavier running on an actual drone, requiring about 28 milliseconds of processing per RGB/IR image pair.