Are Foundation Models Ready for Industrial Defect Recognition? A Reality Check on Real-World Data
This reveals a critical limitation for industrial quality inspection, where FMs could save labeling efforts but currently underperform, indicating an incremental step in assessing their real-world readiness.
The study tested recent Foundation Models (FMs) on real-world industrial defect recognition data and found that they all failed, despite performing well on public benchmark datasets, highlighting a gap in their applicability to practical manufacturing settings.
Foundation Models (FMs) have shown impressive performance on various text and image processing tasks. They can generalize across domains and datasets in a zero-shot setting. This could make them suitable for automated quality inspection during series manufacturing, where various types of images are being evaluated for many different products. Replacing tedious labeling tasks with a simple text prompt to describe anomalies and utilizing the same models across many products would save significant efforts during model setup and implementation. This is a strong advantage over supervised Artificial Intelligence (AI) models, which are trained for individual applications and require labeled training data. We test multiple recent FMs on both custom real-world industrial image data and public image data. We show that all of those models fail on our real-world data, while the very same models perform well on public benchmark datasets.