Empowering Semantic-Sensitive Underwater Image Enhancement with VLM
This work addresses the challenge of semantic-sensitive enhancement for underwater images, which is incremental as it builds on existing UIE baselines with a novel guidance mechanism.
The paper tackles the problem of distribution shifts in underwater image enhancement (UIE) that hinder semantic cue extraction for downstream tasks by proposing a new learning mechanism that uses Vision-Language Models (VLMs) to generate semantic guidance, resulting in significantly boosted performance on perceptual quality metrics and enhanced detection and segmentation tasks.
In recent years, learning-based underwater image enhancement (UIE) techniques have rapidly evolved. However, distribution shifts between high-quality enhanced outputs and natural images can hinder semantic cue extraction for downstream vision tasks, thereby limiting the adaptability of existing enhancement models. To address this challenge, this work proposes a new learning mechanism that leverages Vision-Language Models (VLMs) to empower UIE models with semantic-sensitive capabilities. To be concrete, our strategy first generates textual descriptions of key objects from a degraded image via VLMs. Subsequently, a text-image alignment model remaps these relevant descriptions back onto the image to produce a spatial semantic guidance map. This map then steers the UIE network through a dual-guidance mechanism, which combines cross-attention and an explicit alignment loss. This forces the network to focus its restorative power on semantic-sensitive regions during image reconstruction, rather than pursuing a globally uniform improvement, thereby ensuring the faithful restoration of key object features. Experiments confirm that when our strategy is applied to different UIE baselines, significantly boosts their performance on perceptual quality metrics as well as enhances their performance on detection and segmentation tasks, validating its effectiveness and adaptability.