Scribd
Upload
44 views

Single Image Super-Resolution Using Deep Learning

This document summarizes a deep learning approach for single image super-resolution. The goal is to obtain a high-resolution image from a low-resolution input using a deep neural network trained in an end-to-end fashion. The network architecture uses downscaling layers to increase the receptive field, skip connections to preserve details, and residual blocks for convergence. A GAN loss is also proposed to boost photorealistic image features. Evaluation on standard datasets shows the approach outperforms other methods in terms of PSNR, SSIM, and a high-frequency error norm metric for measuring detail.

Uploaded by

Sighman Says
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
44 views

Single Image Super-Resolution Using Deep Learning

This document summarizes a deep learning approach for single image super-resolution. The goal is to obtain a high-resolution image from a low-resolution input using a deep neural network trained in an end-to-end fashion. The network architecture uses downscaling layers to increase the receptive field, skip connections to preserve details, and residual blocks for convergence. A GAN loss is also proposed to boost photorealistic image features. Evaluation on standard datasets shows the approach outperforms other methods in terms of PSNR, SSIM, and a high-frequency error norm metric for measuring detail.

Uploaded by

Sighman Says
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 1

SINGLE IMAGE SUPER-RESOLUTION USING DEEP LEARNING

Dmitry Korobchenko, Marco Foco


NVIDIA
June 2017

GOAL ARCHITECTURE GAN


Obtain high-resolution image by a given low- • G(x) = Bicubic(x) + DNN(x) Photorealistic image features could be boosted
resolution image. • Downscaling layers (D): to increase receptive by means of Generative Adversarial Networks.
field and capture more semantic features • Generator: our pretrained super-res model
• Skipped connections (S): to propagate low- • Discriminator: binary classifier to distinguish
level features and avoid loss of details after upscaled from real high-resolution images
Upscale downscaling
Generator 𝐺(𝑥)
• Residual blocks (R): to improve convergence 𝑥
and increase number of layers 𝐺(𝑥)
LOW-RES INPUT
R Discriminator

J-Res-Net* 4x R U 𝑦 𝐷(𝑦)
Real high-res images
DEEP LEARNING 𝑥 R

D
S

U
R U
Real high-res or
generated?
R S R
APPROACH D R U GAN loss = −𝑙𝑛𝐷(𝐺 𝑥 )
Our super-resolution model is based on deep Bicubic Total loss = Content loss + GAN loss
neural network. It is trained in end-to-end
fashion to produce high-resolution output from
a given low-resolution input by minimizing a LOSS FUNCTION RESULTS
distance from the output to the ground-truth. Mean values for Set5+Set14+BSDS100 datasets*** 4X UPSCALING
• MSE loss: corresponds to PSNR metric, which
Ground-truth poorly represents perceptual image quality 4x PSNR 4x SSIM 4x HFENI**
• HFENN loss:
27.63 NVIDIA 0.7432 NVIDIA 2.2634 NVIDIA
𝑦ො
27.60 LapSRN 0.7393 LapSRN 2.2271 LapSRN
• High-Frequency Error Norm (Normalized)
27.56 VDSR 0.7385 VDSR 2.1719 VDSR
2Τ
Deep Neural Net Loss • 𝐻𝐹𝐸𝑁𝑁 = 𝐿𝑜𝐺(𝑦ො − 𝑦) 𝑐𝑜𝑛𝑠𝑡 27.53 DRCN 0.7372 DRCN 2.1477 DRCN

𝑦
𝑥 8x PSNR 8x SSIM 8x HFENI**
𝐺(𝑥) 24.75 NVIDIA 0.6087 NVIDIA 0.8954 NVIDIA
LoG (Laplacian of Gaussian)
Deep learning approach exploits prior • Boosts reconstruction of high-frequency 24.62 LapSRN 0.5983 LapSRN 0.8173 LapSRN

knowledge and statistics, extracted from details 24.36 SCN 0.5844 SCN 0.7628 SCN

training images. • Composite loss: 𝑀𝑆𝐸 + 𝛼 ∗ 𝐻𝐹𝐸𝑁𝑁


24.29 VDSR 0.5823 VDSR 0.7519 VDSR 4X UPSCALING (GAN)

Super-resolution technology is released within * J-Net: following U-Net notation idea (Ronneberger et al.)
NVIDIA GameWorks Materials & Textures service ** Inversed HFENN, suitable for evaluation of high-frequency details
gwmt.nvidia.com *** Result images for other algorithms were taken from LapSRN work (Lai et al.)

You might also like