Robust Vision-Based Runway Detection through Conformal Prediction and Conformal mAP
This work addresses safety certification for ML systems in aerospace by providing statistically sound uncertainty quantification for runway detection.
The paper tackles runway detection for vision-based landing systems by applying conformal prediction to YOLO models, introducing a new Conformal mAP metric to align detection performance with statistical uncertainty guarantees, resulting in improved reliability for safety-critical aerospace applications.
We explore the use of conformal prediction to provide statistical uncertainty guarantees for runway detection in vision-based landing systems (VLS). Using fine-tuned YOLOv5 and YOLOv6 models on aerial imagery, we apply conformal prediction to quantify localization reliability under user-defined risk levels. We also introduce Conformal mean Average Precision (C-mAP), a novel metric aligning object detection performance with conformal guarantees. Our results show that conformal prediction can improve the reliability of runway detection by quantifying uncertainty in a statistically sound way, increasing safety on-board and paving the way for certification of ML system in the aerospace domain.