Automatic Prompt Generation and Grounding Object Detection for Zero-Shot Image Anomaly Detection
This addresses the problem of slow, subjective manual inspection for quality control in industrial manufacturing, offering an efficient and scalable solution, though it is incremental as it combines existing foundation models.
The paper tackles automated industrial image anomaly detection by proposing a zero-shot, training-free method that uses GPT-3 to generate prompts, Grounding DINO for object detection, and CLIP for image-text matching, achieving high accuracy on MVTec-AD and VisA datasets.
Identifying defects and anomalies in industrial products is a critical quality control task. Traditional manual inspection methods are slow, subjective, and error-prone. In this work, we propose a novel zero-shot training-free approach for automated industrial image anomaly detection using a multimodal machine learning pipeline, consisting of three foundation models. Our method first uses a large language model, i.e., GPT-3. generate text prompts describing the expected appearances of normal and abnormal products. We then use a grounding object detection model, called Grounding DINO, to locate the product in the image. Finally, we compare the cropped product image patches to the generated prompts using a zero-shot image-text matching model, called CLIP, to identify any anomalies. Our experiments on two datasets of industrial product images, namely MVTec-AD and VisA, demonstrate the effectiveness of this method, achieving high accuracy in detecting various types of defects and anomalies without the need for model training. Our proposed model enables efficient, scalable, and objective quality control in industrial manufacturing settings.