Familiarity-Based Open-Set Recognition Under Adversarial Attacks
This work addresses security concerns for open-set recognition systems in real-world applications, but it is incremental as it builds on existing familiarity-based methods.
The paper investigates the vulnerability of familiarity-based open-set recognition methods to adversarial attacks, specifically false familiarity and false novelty attacks on TinyImageNet, and proposes an adversarial reaction score as an alternative scoring rule that correlates highly with the maximum logit score.
Open-set recognition (OSR), the identification of novel categories, can be a critical component when deploying classification models in real-world applications. Recent work has shown that familiarity-based scoring rules such as the Maximum Softmax Probability (MSP) or the Maximum Logit Score (MLS) are strong baselines when the closed-set accuracy is high. However, one of the potential weaknesses of familiarity-based OSR are adversarial attacks. Here, we study gradient-based adversarial attacks on familiarity scores for both types of attacks, False Familiarity and False Novelty attacks, and evaluate their effectiveness in informed and uninformed settings on TinyImageNet. Furthermore, we explore how novel and familiar samples react to adversarial attacks and formulate the adversarial reaction score as an alternative OSR scoring rule, which shows a high correlation with the MLS familiarity score.