Global-Local Stepwise Generative Network for Ultra High-Resolution Image Restoration
This work addresses the problem of ultra high-resolution image restoration for applications like photography and vision systems, introducing a novel method and dataset, but it is incremental as it builds on existing generative network approaches.
The paper tackles the challenge of restoring ultra high-resolution (e.g., 4K) images by proposing the Global-Local Stepwise Generative Network (GLSGN), which uses a stepwise strategy with local and global pathways to address computational and memory issues, and it consistently outperforms state-of-the-art methods in tasks like reflection removal, rain streak removal, and dehazing.
While the research on image background restoration from regular size of degraded images has achieved remarkable progress, restoring ultra high-resolution (e.g., 4K) images remains an extremely challenging task due to the explosion of computational complexity and memory usage, as well as the deficiency of annotated data. In this paper we present a novel model for ultra high-resolution image restoration, referred to as the Global-Local Stepwise Generative Network (GLSGN), which employs a stepwise restoring strategy involving four restoring pathways: three local pathways and one global pathway. The local pathways focus on conducting image restoration in a fine-grained manner over local but high-resolution image patches, while the global pathway performs image restoration coarsely on the scale-down but intact image to provide cues for the local pathways in a global view including semantics and noise patterns. To smooth the mutual collaboration between these four pathways, our GLSGN is designed to ensure the inter-pathway consistency in four aspects in terms of low-level content, perceptual attention, restoring intensity and high-level semantics, respectively. As another major contribution of this work, we also introduce the first ultra high-resolution dataset to date for both reflection removal and rain streak removal, comprising 4,670 real-world and synthetic images. Extensive experiments across three typical tasks for image background restoration, including image reflection removal, image rain streak removal and image dehazing, show that our GLSGN consistently outperforms state-of-the-art methods.