SAViL-Det: Semantic-Aware Vision-Language Model for Multi-Script Text Detection
This addresses the challenge of multi-script text detection for applications like document analysis and autonomous systems, representing a strong specific gain rather than a foundational breakthrough.
The paper tackled the problem of detecting text in natural scenes, especially for diverse scripts and arbitrarily shaped instances, by introducing SAViL-Det, a semantic-aware vision-language model that integrates textual prompts with visual features, achieving state-of-the-art F-scores of 84.8% on MLT-2019 and 90.2% on CTW1500.
Detecting text in natural scenes remains challenging, particularly for diverse scripts and arbitrarily shaped instances where visual cues alone are often insufficient. Existing methods do not fully leverage semantic context. This paper introduces SAViL-Det, a novel semantic-aware vision-language model that enhances multi-script text detection by effectively integrating textual prompts with visual features. SAViL-Det utilizes a pre-trained CLIP model combined with an Asymptotic Feature Pyramid Network (AFPN) for multi-scale visual feature fusion. The core of the proposed framework is a novel language-vision decoder that adaptively propagates fine-grained semantic information from text prompts to visual features via cross-modal attention. Furthermore, a text-to-pixel contrastive learning mechanism explicitly aligns textual and corresponding visual pixel features. Extensive experiments on challenging benchmarks demonstrate the effectiveness of the proposed approach, achieving state-of-the-art performance with F-scores of 84.8% on the benchmark multi-lingual MLT-2019 dataset and 90.2% on the curved-text CTW1500 dataset.