A Review of Benchmarks for Visual Defect Detection in the Manufacturing Industry
This review addresses the problem of benchmark selection for researchers and practitioners in industrial visual inspection, but it is incremental as it synthesizes existing information without introducing new methods or data.
The paper tackles the challenge of selecting appropriate benchmarks for visual defect detection in manufacturing by reviewing existing datasets, proposing guidelines based on characteristics like real-world conditions and precise labeling.
The field of industrial defect detection using machine learning and deep learning is a subject of active research. Datasets, also called benchmarks, are used to compare and assess research results. There is a number of datasets in industrial visual inspection, of varying quality. Thus, it is a difficult task to determine which dataset to use. Generally speaking, datasets which include a testing set, with precise labeling and made in real-world conditions should be preferred. We propose a study of existing benchmarks to compare and expose their characteristics and their use-cases. A study of industrial metrics requirements, as well as testing procedures, will be presented and applied to the studied benchmarks. We discuss our findings by examining the current state of benchmarks for industrial visual inspection, and by exposing guidelines on the usage of benchmarks.