Deformable Attentive Visual Enhancement for Referring Segmentation Using Vision-Language Model
This work addresses the problem of accurately segmenting objects based on natural language expressions for computer vision applications, representing an incremental improvement with specific architectural enhancements.
The paper tackles referring image segmentation by proposing SegVLM, a vision-language model that integrates architectural improvements like SE blocks, deformable convolutions, and a novel RAF loss, achieving consistent performance gains and strong generalization across diverse datasets.
Image segmentation is a fundamental task in computer vision, aimed at partitioning an image into semantically meaningful regions. Referring image segmentation extends this task by using natural language expressions to localize specific objects, requiring effective integration of visual and linguistic information. In this work, we propose SegVLM, a vision-language model that incorporates architectural improvements to enhance segmentation accuracy and cross-modal alignment. The model integrates squeeze-and-excitation (SE) blocks for dynamic feature recalibration, deformable convolutions for geometric adaptability, and residual connections for deep feature learning. We also introduce a novel referring-aware fusion (RAF) loss that balances region-level alignment, boundary precision, and class imbalance. Extensive experiments and ablation studies demonstrate that each component contributes to consistent performance improvements. SegVLM also shows strong generalization across diverse datasets and referring expression scenarios.