Basic usage

Get a noisy video:

In[3]:=

Define a noise level for a video:

In[4]:=

Denoise a video:

In[5]:=

Out[6]=

Visualize the five frames of a noisy and denoised video:

In[7]:=

Out[7]=

Adapt to any size

Automatic frame resizing can be avoided by replacing the NetEncoder. First get the net:

In[8]:=

Get a noisy video:

In[9]:=

Create a new NetEncoder with the desired dimensions (to get a resizable net, the spatial dimensions should be divisible by 4):

In[10]:=

Out[10]=

Attach the NetEncoder, define a noise level and run the net:

In[11]:=

Visualize the three frames of a noisy and denoised video:

In[12]:=

Out[12]=

FastDVDNet architecture

The video encoder takes the first five frames of the input video and encodes it into a vector of size (5,3,256,256):

In[13]:=

Out[14]=

Note that the first three blocks are shared. (Frame_i, Frame_i+1, Frame_i+2, Noise) are given as inputs to block_i to obtain an intermediate denoised frame to obtain an intermediate denoised output frame . Finally, (, , , Noise) are given as inputs to the last block to get the final denoised output frame:

In[15]:=

Out[16]=

Extract the first and the last blocks of the net:

In[17]:=

All the blocks share a U-Net type architecture. Explore the input block:

In[18]:=

Out[18]=

Get a noisy video and define a noise level for a video:

In[19]:=

Simply mapping the net to a list of frames can be inefficient due some blocks processing the same frame multiple times. To avoid double counting, split the inference into two steps:

In[20]:=

Get the final denoised video:

In[21]:=

Visualize the three frames of a noisy video and the intermediate and final result:

In[22]:=

Out[22]=

Memory-efficient evaluation

Define a custom videoDenoise function for memory-efficient processing of video with Gaussian or Poisson noise:

In[23]:=

Get a noisy video:

In[24]:=

Get a final denoised video assuming a Gaussian distribution on the input noise:

In[25]:=

Out[26]=

Get a final denoised video assuming a Poisson distribution on the input noise:

In[27]:=

Out[28]=

Visualize the four frames of a noisy video and the denoised results assuming a Gaussian and Poisson distribution on the input noise:

In[29]:=

Out[29]=

Net information

Inspect the number of parameters of all arrays in the net:

In[30]:=

Out[31]=

Obtain the total number of parameters:

In[32]:=

Out[33]=

Obtain the layer type counts:

In[34]:=

Out[35]=

Display the summary graphic:

In[36]:=

Out[37]=

Export to ONNX

Export the net to the ONNX format:

In[38]:=

Out[39]=

Get the size of the ONNX file:

In[40]:=

Out[40]=

The byte count of the resource object is smaller because shared arrays are currently being duplicated when exporting to ONNX:

In[41]:=

Out[42]=

Check some metadata of the ONNX model:

In[43]:=

Out[43]=

Import the model back into the Wolfram Language. However, the NetEncoder and NetDecoder will be absent because they are not supported by ONNX:

In[44]:=

Out[44]=