Looking 3D: Anomaly Detection with 2D-3D Alignment
This addresses anomaly detection for manufacturing and quality assessment, but it is incremental as it builds on existing conditional detection methods with a new dataset and model.
The paper tackles the problem of detecting anomalies in images by comparing them to reference 3D shapes, using a transformer-based approach with feature alignment and a customized attention mechanism, achieving results on a new large dataset of 180K images and 8,143 3D shapes.
Automatic anomaly detection based on visual cues holds practical significance in various domains, such as manufacturing and product quality assessment. This paper introduces a new conditional anomaly detection problem, which involves identifying anomalies in a query image by comparing it to a reference shape. To address this challenge, we have created a large dataset, BrokenChairs-180K, consisting of around 180K images, with diverse anomalies, geometries, and textures paired with 8,143 reference 3D shapes. To tackle this task, we have proposed a novel transformer-based approach that explicitly learns the correspondence between the query image and reference 3D shape via feature alignment and leverages a customized attention mechanism for anomaly detection. Our approach has been rigorously evaluated through comprehensive experiments, serving as a benchmark for future research in this domain.