Beyond Academic Benchmarks: Critical Analysis and Best Practices for Visual Industrial Anomaly Detection
This work addresses the problem of unreliable visual anomaly detection in manufacturing for industry practitioners, highlighting incremental improvements through better benchmarking.
The paper tackles the disconnect between academic anomaly detection methods and industrial viability by establishing benchmarks using real production data and providing a fair comparison of state-of-the-art methods, showing significant performance degradation in practical settings.
Anomaly detection (AD) is essential for automating visual inspection in manufacturing. This field of computer vision is rapidly evolving, with increasing attention towards real-world applications. Meanwhile, popular datasets are typically produced in controlled lab environments with artificially created defects, unable to capture the diversity of real production conditions. New methods often fail in production settings, showing significant performance degradation or requiring impractical computational resources. This disconnect between academic results and industrial viability threatens to misdirect visual anomaly detection research. This paper makes three key contributions: (1) we demonstrate the importance of real-world datasets and establish benchmarks using actual production data, (2) we provide a fair comparison of existing SOTA methods across diverse tasks by utilizing metrics that are valuable for practical applications, and (3) we present a comprehensive analysis of recent advancements in this field by discussing important challenges and new perspectives for bridging the academia-industry gap. The code is publicly available at https://github.com/abc-125/viad-benchmark