AgentPolyp: Accurate Polyp Segmentation via Image Enhancement Agent
This work addresses polyp segmentation challenges in medical imaging for endoscopic diagnostics, representing an incremental improvement with a modular approach.
The paper tackles the problem of polyp segmentation in noisy endoscopic images by proposing AgentPolyp, a framework that integrates CLIP-based semantic guidance and dynamic image enhancement, achieving robust preprocessing for segmentation tasks.
Since human and environmental factors interfere, captured polyp images usually suffer from issues such as dim lighting, blur, and overexposure, which pose challenges for downstream polyp segmentation tasks. To address the challenges of noise-induced degradation in polyp images, we present AgentPolyp, a novel framework integrating CLIP-based semantic guidance and dynamic image enhancement with a lightweight neural network for segmentation. The agent first evaluates image quality using CLIP-driven semantic analysis (e.g., identifying ``low-contrast polyps with vascular textures") and adapts reinforcement learning strategies to dynamically apply multi-modal enhancement operations (e.g., denoising, contrast adjustment). A quality assessment feedback loop optimizes pixel-level enhancement and segmentation focus in a collaborative manner, ensuring robust preprocessing before neural network segmentation. This modular architecture supports plug-and-play extensions for various enhancement algorithms and segmentation networks, meeting deployment requirements for endoscopic devices.