Very Deep Net for Super-Resolution

Increase the resolution of an image

Released in 2016, this net uses an architecture inspired by VGG in order to create super-resolution images. It takes an interpolated low-resolution image and refines the details to create a sharp upsampling.

Number of layers: 40 | Parameter count: 665,921 | Trained size: 3 MB |

Training Set Information

A dataset of 291 images, expanded with data augmentation.

Performance

This model achieves a peak signal-to-noise ratio of 30.76 on the Urban 100 dataset with a scale factor of 2.

Examples

Download Example Notebook

Open in Wolfram Cloud

Resource retrieval

Get the pre-trained net:

In[1]:=

Out[1]=

Evaluation function

Write an evaluation function to handle net resizing and color conversion:

In[2]:=

$netevaluate[img_, imgScale_] := Block[{net, interpolated, ycbcr, channels, resizedNet, diff, rgb}, net = NetModel["Very Deep Net for Super-Resolution"]; interpolated = ImageResize[img, Scaled[imgScale], Resampling -> "Cubic"];(* upscale to the final size *) ycbcr = ImageApply[{{0.257, 0.504, 0.098}, {-0.148, -0.291, 0.439}, {0.439, -0.368, -0.071}}.# + {0.063, 0.502, 0.502} &, interpolated]; channels = ColorSeparate[ycbcr]; resizedNet = NetReplacePart[net, "Input" -> NetEncoder[{"Image", ImageDimensions@interpolated, ColorSpace -> "Grayscale"}]]; diff = Image@resizedNet[channels[[1]]]; ycbcr = ColorCombine[{channels[[1]] + diff, channels[[2]], channels[[3]]}]; rgb = ImageApply[{{1.164, 0., 1.596}, {1.164, -0.392, -0.813}, {1.164, 2.017, 0.}}.# + {-0.874, 0.532, -1.086} &, ycbcr]; rgb ]$

Basic usage

Get an image:

In[3]:=

(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/428e8456-38ea-4841-977d-e0422f89dde5"]

Downscale the image by a factor of 3:

In[4]:=

In[5]:=

Out[5]=

Upscale the downscaled image using the net:

In[6]:=

Out[6]=

Compare the details with a naively upscaled version:

In[7]:=

Out[7]=

Evaluate the peak signal-to-noise ratio:

In[8]:=

Out[8]=

Net information

Inspect the number of parameters of all arrays in the net:

In[9]:=

$NetInformation[ NetModel["Very Deep Net for Super-Resolution"], \ "ArraysElementCounts"]$

Out[9]=

Obtain the total number of parameters:

In[10]:=

$NetInformation[ NetModel["Very Deep Net for Super-Resolution"], \ "ArraysTotalElementCount"]$

Out[10]=

Obtain the layer type counts:

In[11]:=

Out[11]=

Display the summary graphic:

In[12]:=

Out[12]=

Export to MXNet

Export the net into a format that can be opened in MXNet:

In[13]:=

Out[13]=

Export also creates a net.params file containing parameters:

In[14]:=

Out[14]=

Get the size of the parameter file:

In[15]:=

Out[15]=

The size is similar to the byte count of the resource object:

In[16]:=

Out[16]=

Represent the MXNet net as a graph:

In[17]:=

Out[17]=

Construction Notebook

Download Construction Notebook

Open in Wolfram Cloud

Requirements

Wolfram Language 11.3 (March 2018) or above

Resource History

Date Created: 26 July 2018

Reference

J. Kim, J. Kwon Lee and K. Mu Lee, "Accurate Image Super-Resolution Using Very Deep Convolutional Networks," Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Available from: https://github.com/huangzehao/caffe-vdsr
Rights: MIT License