CLIP-AD: A Language-Guided Staged Dual-Path Model for Zero-shot Anomaly Detection
This addresses the problem of detecting anomalies in industrial inspection without labeled data, offering a significant but incremental advance over existing methods.
The paper tackles zero-shot anomaly detection without reference images by proposing CLIP-AD, a framework that leverages CLIP's vision-language capabilities through improved text features and a staged dual-path model; it achieves state-of-the-art performance with improvements of +4.2/+10.7 in F1-max/PRO metrics on MVTec-AD.
This paper considers zero-shot Anomaly Detection (AD), performing AD without reference images of the test objects. We propose a framework called CLIP-AD to leverage the zero-shot capabilities of the large vision-language model CLIP. Firstly, we reinterpret the text prompts design from a distributional perspective and propose a Representative Vector Selection (RVS) paradigm to obtain improved text features. Secondly, we note opposite predictions and irrelevant highlights in the direct computation of the anomaly maps. To address these issues, we introduce a Staged Dual-Path model (SDP) that leverages features from various levels and applies architecture and feature surgery. Lastly, delving deeply into the two phenomena, we point out that the image and text features are not aligned in the joint embedding space. Thus, we introduce a fine-tuning strategy by adding linear layers and construct an extended model SDP+, further enhancing the performance. Abundant experiments demonstrate the effectiveness of our approach, e.g., on MVTec-AD, SDP outperforms the SOTA WinCLIP by +4.2/+10.7 in segmentation metrics F1-max/PRO, while SDP+ achieves +8.3/+20.5 improvements.