Masked Autoencoders as Image Processors
This work addresses the problem of improving image processing tasks for computer vision applications, representing an incremental advancement by extending MAE pre-training to low-level vision.
The paper tackled the insufficient exploration of masked autoencoder (MAE) pre-training for low-level vision tasks by developing MAEIP, an MAE architecture for image processing, and CSformer, an efficient Transformer model; the result was state-of-the-art performance on tasks like Gaussian denoising, real image denoising, motion deblurring, defocus deblurring, and image deraining.
Transformers have shown significant effectiveness for various vision tasks including both high-level vision and low-level vision. Recently, masked autoencoders (MAE) for feature pre-training have further unleashed the potential of Transformers, leading to state-of-the-art performances on various high-level vision tasks. However, the significance of MAE pre-training on low-level vision tasks has not been sufficiently explored. In this paper, we show that masked autoencoders are also scalable self-supervised learners for image processing tasks. We first present an efficient Transformer model considering both channel attention and shifted-window-based self-attention termed CSformer. Then we develop an effective MAE architecture for image processing (MAEIP) tasks. Extensive experimental results show that with the help of MAEIP pre-training, our proposed CSformer achieves state-of-the-art performance on various image processing tasks, including Gaussian denoising, real image denoising, single-image motion deblurring, defocus deblurring, and image deraining.