SCISSOR: Mitigating Semantic Bias through Cluster-Aware Siamese Networks for Robust Classification
This addresses model generalization issues for AI systems by mitigating semantic biases, though it appears incremental as it builds on existing debiasing approaches with a new method.
The paper tackles shortcut learning in AI models by showing that imbalances in semantic embeddings cause spurious correlations, and proposes SCISSOR, a Siamese network-based debiasing method that improves robustness without data augmentation. Results include absolute F1 score gains of +5.3 to +7.7 points across four benchmarks and ~9.5-11.9% improvements for lightweight models.
Shortcut learning undermines model generalization to out-of-distribution data. While the literature attributes shortcuts to biases in superficial features, we show that imbalances in the semantic distribution of sample embeddings induce spurious semantic correlations, compromising model robustness. To address this issue, we propose SCISSOR (Semantic Cluster Intervention for Suppressing ShORtcut), a Siamese network-based debiasing approach that remaps the semantic space by discouraging latent clusters exploited as shortcuts. Unlike prior data-debiasing approaches, SCISSOR eliminates the need for data augmentation and rewriting. We evaluate SCISSOR on 6 models across 4 benchmarks: Chest-XRay and Not-MNIST in computer vision, and GYAFC and Yelp in NLP tasks. Compared to several baselines, SCISSOR reports +5.3 absolute points in F1 score on GYAFC, +7.3 on Yelp, +7.7 on Chest-XRay, and +1 on Not-MNIST. SCISSOR is also highly advantageous for lightweight models with ~9.5% improvement on F1 for ViT on computer vision datasets and ~11.9% for BERT on NLP. Our study redefines the landscape of model generalization by addressing overlooked semantic biases, establishing SCISSOR as a foundational framework for mitigating shortcut learning and fostering more robust, bias-resistant AI systems.