Deep Learning based Multi-Modal Sensing for Tracking and State Extraction of Small Quadcopters
This work addresses the problem of robust tracking and localization of small quadcopters for surveillance and monitoring applications, offering an incremental improvement over existing methods.
This paper proposes a multi-sensor system combining RGB, thermal, and lidar data to detect, track, and localize small quadcopters. The system successfully tracks and localizes UAVs in sample experiments, comparing favorably to existing methods.
This paper proposes a multi-sensor based approach to detect, track, and localize a quadcopter unmanned aerial vehicle (UAV). Specifically, a pipeline is developed to process monocular RGB and thermal video (captured from a fixed platform) to detect and track the UAV in our FoV. Subsequently, a 2D planar lidar is used to allow conversion of pixel data to actual distance measurements, and thereby enable localization of the UAV in global coordinates. The monocular data is processed through a deep learning-based object detection method that computes an initial bounding box for the UAV. The thermal data is processed through a thresholding and Kalman filter approach to detect and track the bounding box. Training and testing data are prepared by combining a set of original experiments conducted in a motion capture environment and publicly available UAV image data. The new pipeline compares favorably to existing methods and demonstrates promising tracking and localization capacity of sample experiments.