Restore Anything Pipeline: Segment Anything Meets Image Restoration
This work provides a user-friendly, interactive approach for image restoration that allows per-object control, addressing preferences in applications like photography and digital media, though it is incremental as it builds on existing segmentation and restoration models.
The paper tackles the problem of image restoration by addressing the limitation of existing methods that treat the entire image uniformly, introducing the Restore Anything Pipeline (RAP) which integrates the Segment Anything Model for per-object restoration and user control, resulting in superior visual outcomes compared to state-of-the-art methods in tasks like deblurring, denoising, and JPEG artifact removal.
Recent image restoration methods have produced significant advancements using deep learning. However, existing methods tend to treat the whole image as a single entity, failing to account for the distinct objects in the image that exhibit individual texture properties. Existing methods also typically generate a single result, which may not suit the preferences of different users. In this paper, we introduce the Restore Anything Pipeline (RAP), a novel interactive and per-object level image restoration approach that incorporates a controllable model to generate different results that users may choose from. RAP incorporates image segmentation through the recent Segment Anything Model (SAM) into a controllable image restoration model to create a user-friendly pipeline for several image restoration tasks. We demonstrate the versatility of RAP by applying it to three common image restoration tasks: image deblurring, image denoising, and JPEG artifact removal. Our experiments show that RAP produces superior visual results compared to state-of-the-art methods. RAP represents a promising direction for image restoration, providing users with greater control, and enabling image restoration at an object level.