Enhancing Evaluation Methods for Infrared Small-Target Detection in Real-world Scenarios
This work addresses the problem of inadequate evaluation methods for IRSTD algorithms, which is crucial for researchers and practitioners in computer vision, though it is incremental as it focuses on improving metrics rather than detection methods.
The paper tackled the lack of extensive investigation into evaluation metrics for infrared small-target detection (IRSTD) by analyzing existing metrics' shortcomings and proposing new ones aligned with real-world requirements, demonstrating that these new metrics provide consistent and meaningful quantitative assessments when evaluating five widely recognized algorithms.
Infrared small target detection (IRSTD) poses a significant challenge in the field of computer vision. While substantial efforts have been made over the past two decades to improve the detection capabilities of IRSTD algorithms, there has been a lack of extensive investigation into the evaluation metrics used for assessing their performance. In this paper, we employ a systematic approach to address this issue by first evaluating the effectiveness of existing metrics and then proposing new metrics to overcome the limitations of conventional ones. To achieve this, we carefully analyze the necessary conditions for successful detection and identify the shortcomings of current evaluation metrics, including both pre-thresholding and post-thresholding metrics. We then introduce new metrics that are designed to align with the requirements of real-world systems. Furthermore, we utilize these newly proposed metrics to compare and evaluate the performance of five widely recognized small infrared target detection algorithms. The results demonstrate that the new metrics provide consistent and meaningful quantitative assessments, aligning with qualitative observations.