SimLabel: Consistency-Guided OOD Detection with Pretrained Vision-Language Models
This addresses OOD detection for safety-critical machine learning applications, offering an incremental improvement by leveraging semantic connections across ID classes.
The paper tackles the problem of out-of-distribution (OOD) detection in vision-language models by proposing SimLabel, a post-hoc strategy that improves separability between ID and OOD samples using consistency over similar class labels, achieving superior performance on zero-shot OOD detection benchmarks.
Detecting out-of-distribution (OOD) data is crucial in real-world machine learning applications, particularly in safety-critical domains. Existing methods often leverage language information from vision-language models (VLMs) to enhance OOD detection by improving confidence estimation through rich class-wise text information. However, when building OOD detection score upon on in-distribution (ID) text-image affinity, existing works either focus on each ID class or whole ID label sets, overlooking inherent ID classes' connection. We find that the semantic information across different ID classes is beneficial for effective OOD detection. We thus investigate the ability of image-text comprehension among different semantic-related ID labels in VLMs and propose a novel post-hoc strategy called SimLabel. SimLabel enhances the separability between ID and OOD samples by establishing a more robust image-class similarity metric that considers consistency over a set of similar class labels. Extensive experiments demonstrate the superior performance of SimLabel on various zero-shot OOD detection benchmarks. The proposed model is also extended to various VLM-backbones, demonstrating its good generalization ability. Our demonstration and implementation codes are available at: https://github.com/ShuZou-1/SimLabel.