Quantization Guided JPEG Artifact Correction

Max Ehrlich1,2 Larry Davis1 Ser-Nam Lim2 Abhinav Shrivastava1

1 University of Maryland Institute for Advanced Computer Studies 2 Facebook AI

Paper Code Talk Slides Cite

We develop a novel method for JPEG artifact correction that solves three major problems left open in prior works:
  1. Prior works train an ensemble of models, one for each JPEG quality. We use a single network parameterized by the JPEG quantization matrix.
  2. Prior works deal with grayscale images only, with the assumption that their models can be applied channel-wise. We show that single-channel networks have trouble generalizing and design a network for color correction.
  3. Prior works focus on CNN regression which causes blurry and textureless results. We introduce a novel GAN loss that includes an explicit texture restoring term, this yields a more realistic result.
Our method achieves state-of-the-art results on color artifact correction. The paper was published in the proceedings of the European Conference on Computer Vision.

Abstract

The JPEG image compression algorithm is the most popular method of image compression because of its ability for large compression ratios. However, to achieve such high compression, information is lost. For aggressive quantization settings, this leads to a noticeable reduction in image quality. Artifact correction has been studied in the context of deep neural networks for some time, but the current state-of-the-art methods require a different model to be trained for each quality setting, greatly limiting their practical application. We solve this problem by creating a novel architecture which is parameterized by the JPEG files quantization matrix. This allows our single model to achieve state-of-the-art performance over models trained for specific quality settings.

Talk


Download Slides

Code

Our code is freely available and includes pre-trained models for grayscale, color, and GAN correction. By using our code, JPEG files can be stored at low qualities for a significant space savings and restored only for display.

Get the Code

Example Results

JPEG

Corrected

Acknowledgement

This project was partially supported by Facebook AI and Defense Advanced Research Projects Agency (DARPA) MediFor program (FA87501620191). The views, opinions, and findings expressed are those of the authors and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government. There is no collaboration between Facebook and DARPA.