1 Sensing, Interaction & Perception Lab, Department of Computer Science, ETH Zürich, Switzerland
Recent image restoration methods have produced significant advancements using deep learning. However, existing methods tend to treat the whole image as a single entity, failing to account for the distinct objects in the image that exhibit individual texture properties. Existing methods also typically generate a single result, which may not suit the preferences of different users. In this paper, we introduce the Restore Anything Pipeline (RAP), a novel interactive and per-object level image restoration approach that incorporates a controllable model to generate different results that users may choose from. RAP incorporates image segmentation through the recent Segment Anything Model (SAM) into a controllable image restoration model to create a user-friendly pipeline for several image restoration tasks. We demonstrate the versatility of RAP by applying it to three common image restoration tasks: image deblurring, image denoising, and JPEG artifact removal. Our experiments show that RAP produces superior visual results compared to state-of-the-art methods. RAP represents a promising direction for image restoration, providing users with greater control, and enabling image restoration at an object level.
Image Segmentation + Controllable Object-Level Image Restoration and Enhancement
We will release our codes very soon.
@misc{jiang2023restore,
title={Restore Anything Pipeline: Segment Anything Meets Image Restoration},
author={Jiaxi Jiang and Christian Holz},
year={2023},
eprint={2305.13093},
archivePrefix={arXiv},
primaryClass={cs.CV}
}