Crafting Query-Aware Selective Attention for Single Image Super-Resolution
This work addresses computational bottlenecks in SISR for image processing applications, offering an incremental improvement over prior attention mechanisms.
The paper tackles the problem of inefficient selective attention in Vision Transformer-based single image super-resolution (SISR) by proposing SSCAN, a method that dynamically selects key-value windows based on query similarity, resulting in up to 0.14 dB PSNR improvement on urban datasets while maintaining computational efficiency.
Single Image Super-Resolution (SISR) reconstructs high-resolution images from low-resolution inputs, enhancing image details. While Vision Transformer (ViT)-based models improve SISR by capturing long-range dependencies, they suffer from quadratic computational costs or employ selective attention mechanisms that do not explicitly focus on query-relevant regions. Despite these advancements, prior work has overlooked how selective attention mechanisms should be effectively designed for SISR. We propose SSCAN, which dynamically selects the most relevant key-value windows based on query similarity, ensuring focused feature extraction while maintaining efficiency. In contrast to prior approaches that apply attention globally or heuristically, our method introduces a query-aware window selection strategy that better aligns attention computation with important image regions. By incorporating fixed-sized windows, SSCAN reduces memory usage and enforces linear token-to-token complexity, making it scalable for large images. Our experiments demonstrate that SSCAN outperforms existing attention-based SISR methods, achieving up to 0.14 dB PSNR improvement on urban datasets, guaranteeing both computational efficiency and reconstruction quality in SISR.