STING-BEE: Towards Vision-Language Model for Real-World X-ray Baggage Security Inspection
This work addresses the need for more effective computer-aided screening systems in airport security by providing a novel dataset and model for detecting sophisticated threats, though it is incremental in applying vision-language models to a specific domain.
The authors tackled the problem of limited datasets and closed-set approaches in X-ray baggage security inspection by introducing STCray, a multimodal dataset with 46,642 image-caption pairs across 21 threat categories, and trained STING-BEE, a domain-aware visual AI assistant that achieves state-of-the-art generalization in cross-domain settings.
Advancements in Computer-Aided Screening (CAS) systems are essential for improving the detection of security threats in X-ray baggage scans. However, current datasets are limited in representing real-world, sophisticated threats and concealment tactics, and existing approaches are constrained by a closed-set paradigm with predefined labels. To address these challenges, we introduce STCray, the first multimodal X-ray baggage security dataset, comprising 46,642 image-caption paired scans across 21 threat categories, generated using an X-ray scanner for airport security. STCray is meticulously developed with our specialized protocol that ensures domain-aware, coherent captions, that lead to the multi-modal instruction following data in X-ray baggage security. This allows us to train a domain-aware visual AI assistant named STING-BEE that supports a range of vision-language tasks, including scene comprehension, referring threat localization, visual grounding, and visual question answering (VQA), establishing novel baselines for multi-modal learning in X-ray baggage security. Further, STING-BEE shows state-of-the-art generalization in cross-domain settings. Code, data, and models are available at https://divs1159.github.io/STING-BEE/.