GHOST: Gaussian Hypothesis Open-Set Technique
This addresses fairness issues in evaluating large-scale recognition methods for AI/ML practitioners, but it is incremental as it builds on existing OSR techniques.
The paper tackles the problem of fairness and misrepresentation in Open-Set Recognition by showing that per-class performance varies dramatically, and introduces GHOST, a hyperparameter-free algorithm that models deep features with class-wise Gaussian distributions and Z-score normalization, achieving statistically significant improvements in metrics like AUOSCR, AUROC, and FPR95 across multiple ImageNet-1K pre-trained networks and unknown datasets.
Evaluations of large-scale recognition methods typically focus on overall performance. While this approach is common, it often fails to provide insights into performance across individual classes, which can lead to fairness issues and misrepresentation. Addressing these gaps is crucial for accurately assessing how well methods handle novel or unseen classes and ensuring a fair evaluation. To address fairness in Open-Set Recognition (OSR), we demonstrate that per-class performance can vary dramatically. We introduce Gaussian Hypothesis Open Set Technique (GHOST), a novel hyperparameter-free algorithm that models deep features using class-wise multivariate Gaussian distributions with diagonal covariance matrices. We apply Z-score normalization to logits to mitigate the impact of feature magnitudes that deviate from the model's expectations, thereby reducing the likelihood of the network assigning a high score to an unknown sample. We evaluate GHOST across multiple ImageNet-1K pre-trained deep networks and test it with four different unknown datasets. Using standard metrics such as AUOSCR, AUROC and FPR95, we achieve statistically significant improvements, advancing the state-of-the-art in large-scale OSR. Source code is provided online.