InstructIR ✏️🖼️ 🤗

High-Quality Image Restoration Following Human Instructions

Marcos V. Conde, Gregor Geigle, Radu Timofte

Computer Vision Lab, University of Wuerzburg | Sony PlayStation, FTG

TL;DR: quickstart

InstructIR takes as input an image and a human-written instruction for how to improve that image. The (single) neural model performs all-in-one image restoration. InstructIR achieves state-of-the-art results on several restoration tasks including image denoising, deraining, deblurring, dehazing, and (low-light) image enhancement. 🚀 You can start with the demo tutorial. Check our github for more information

Abstract (click me to read)

Image restoration is a fundamental problem that involves recovering a high-quality clean image from its degraded observation. All-In-One image restoration models can effectively restore images from various types and levels of degradation using degradation-specific information as prompts to guide the restoration model. In this work, we present the first approach that uses human-written instructions to guide the image restoration model. Given natural language prompts, our model can recover high-quality images from their degraded counterparts, considering multiple degradation types. Our method, InstructIR, achieves state-of-the-art results on several restoration tasks including image denoising, deraining, deblurring, dehazing, and (low-light) image enhancement. InstructIR improves +1dB over previous all-in-one restoration methods. Moreover, our dataset and results represent a novel benchmark for new research on text-guided image restoration and enhancement.

Disclaimer: please remember this is not a product, thus, you will notice some limitations. Due to the GPU memory limitations, the app might crash if you feed a high-resolution image (2K, 4K).
The model was trained using mostly synthetic data, thus it might not work great on real-world complex images. You can also try general image enhancement prompts (e.g., "retouch this image", "enhance the colors") and see how it improves the colors. As you can see, the model is quite efficient.

Datasets: We use these datasets BSD100, BSD68, Urban100, WED, Rain100, Aobe MIT5K, LOL, GoPro, SOTS (haze). This demo expects an image with some degradations (blur, noise, rain, low-light, haze).


@article{conde2024high,
  title={High-Quality Image Restoration Following Human Instructions},
  author={Conde, Marcos V and Geigle, Gregor and Timofte, Radu},
  journal={arXiv preprint arXiv:2401.16468},
  year={2024}
}