SUPIR - High-Fidelity AI Image Restoration and Quality Enhancement Model

What is SUPIR

SUPIR (Scaling-UP Image Restoration) is a breakthrough image restoration and quality enhancement method that leverages the large-scale generative model StableDiffusion-XL (SDXL) and model scaling techniques to achieve high-quality restoration of low-quality images through deep learning and multimodal methods. This method supports fine-grained control of image restoration via text prompts, capable of adjusting various aspects of restoration based on user input, such as object textures and the semantic content of the scene. SUPIR was jointly introduced by researchers from the Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shanghai AI Laboratory, University of Sydney, Hong Kong Polytechnic University, Tencent ARC Lab, and The Chinese University of Hong Kong.

SUPIR Official Website Entry

• Official Project Homepage:https://supir.xpixel.group/
• GitHub Source Code Repository:https://github.com/Fanghua-Yu/SUPIR
• arXiv Research Paper:https://arxiv.org/abs/2401.13627

SUPIR's Main Features

• High-Quality Image Restoration: SUPIR can restore low-quality images to high-quality versions close to their original state, and can handle image degradation caused by various reasons (such as compression, noise, blur, etc.).
• Restore Multiple Types of Images: SUPIR can effectively restore multiple types of degraded images, including landscapes, faces, animals, game graphics, old movies, and old photos, enhancing image details, restoring clarity and realism, and giving images a new lease on life.
• Text Prompt-Guided Restoration: SUPIR allows users to guide the image restoration process through text prompts, specifying particular aspects of restoration, such as repairing a blurry object in the image, changing an object's material texture, or adjusting the restoration effect based on high-level semantics.
• Negative Quality Prompts: SUPIR uses negative quality prompts to improve the perceptual quality of images. By telling the model which image features are undesirable (e.g., 'oil painting effect, cartoonization, blurriness, messiness, low quality', etc.), the model can avoid these features during restoration, thereby improving the overall image quality.

How SUPIR Works

1. Generative Prior: SUPIR uses StableDiffusion-XL (SDXL) as its Generative Prior, a large pre-trained image generation model containing 2.6 billion parameters. The generative prior is the knowledge of image data distribution learned by the model, used to guide the image generation and restoration process.
2. Dataset and Text Annotations: To train SUPIR, researchers collected a dataset containing 20 million high-resolution, high-quality images, each with detailed descriptive text annotations. These annotations provide additional information about the image content, enabling the model to better understand and restore images.
3. Adapter: To effectively apply the SDXL model, researchers designed and trained an adapter with over 600 million parameters. The adapter's role is to identify the content in low-quality images and finely control the generation process at the pixel level.
   
4. Text Prompt Control: SUPIR can guide image restoration based on text prompts provided by the user. Prompts can be specific descriptions of image content or high-level semantic descriptions of desired image quality.
5. Negative Quality Prompts and Training Samples: To improve the perceptual quality of images, SUPIR introduces negative quality prompts to help the model understand undesirable image features. The researchers used SDXL to generate images corresponding to the negative quality prompts and incorporated these images into the training data to ensure that the model can learn these negative quality concepts.
6. Restoration-Guided Sampling: To prevent image distortion during the generation process, SUPIR adopts a novel restoration-guided sampling method, which selectively guides the prediction results toward the low-quality image during the diffusion process to maintain restoration consistency.
7. Model Training and Sampling Settings: During training, SUPIR used a synthetic degradation model and was trained for 10 days on 64 Nvidia A6000 GPUs. At test time, the model is able to process images with a size of 1024×1024 pixels.

Application Scenarios of SUPIR

• Old Photo Restoration：SUPIR can be used to restore aged, damaged, or faded old photos, recovering their original colors and details so that precious memories can be preserved and passed on.
• Blurry Image Enhancement：For photos caused by hand shake, focusing errors, or motion blur, SUPIR can improve image clarity through advanced algorithms, making originally blurry images sharper.
• Noise Removal：Photos taken under high ISO or low-light conditions often contain noise. SUPIR can effectively identify and remove this noise while preserving important details in the image.
• Color Correction and Enhancement：SUPIR can identify and correct color distortion issues, such as oversaturation or color deviation, making the colors of the image more realistic and vivid.