AdaIN-Style Trained on MS-COCO and Painter by Numbers Data

Transfer the style of one image to another image

This model is also available through the built-in function ImageRestyle

Released in 2017, this is the first real-time feedforward image stylization model to accept arbitrary styles. Building on the interpretation of neural style transfer as a statistical domain adaptation task, the model leverages a novel technique called Adaptive Instance Normalization (AdaIN). The AdaIN layer inside the net performs the style transfer by aligning the mean and variance of the content and style feature maps.

Number of layers: 109 | Parameter count: 10,516,675 | Trained size: 42 MB |

Training Set Information

MS-COCO, a dataset for image recognition, segmentation and captioning consisting of more than 300,000 images overall. Painter by Numbers, a dataset consisting of 23,000 images of paintings mostly sourced from wikiart.org.

Examples

Download Example Notebook

Open in Wolfram Cloud

Resource retrieval

Get the pre-trained net:

In[1]:=

Out[1]=

Basic usage

Restyle an image:

In[2]:=

(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/85bff3d8-9c15-4ac8-a24a-8902f04921b1"]

Out[2]=

Control the stylization weighting

More flexibility can be obtained by manipulating the different subnetworks separately. Separate the NetGraph into its components:

In[3]:=

Out[3]=

In[4]:=

Out[4]=

In[5]:=

Out[5]=

In[6]:=

Out[6]=

In[7]:=

Out[7]=

In[8]:=

Out[8]=

Obtain the encoded content and style features:

In[9]:=

(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/291e1401-a21d-44a0-917b-7248bc39ac1c"]

In[10]:=

(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/42315d18-3804-496c-b09d-8af634321639"]

Run the net on the features:

In[11]:=

To control the stylization weight, blend the adapted features with the content features:

In[12]:=

Stylize the content with different style weights:

In[13]:=

Out[13]=

In[14]:=

Out[14]=

Adapt to any size

Automatic image resizing can be avoided by replacing the net encoders. First get the net:

In[15]:=

Out[15]=

Extract the original mean channel values:

In[16]:=

Out[16]=

Obtain the content and style features:

In[17]:=

In[18]:=

(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/5f99f29d-68eb-47d4-bf3d-51ebab5de059"]

Create the new net encoders with the desired dimensions:

In[19]:=

In[20]:=

Attach the new net encoders:

In[21]:=

resizedNet = NetReplacePart[
net, {"Content" -> netEncC, "Style" -> netEncS, "Output" -> NetDecoder["Image"]}]

Out[21]=

Restyle an image using the new resized network:

In[22]:=

Out[22]=

Net information

Inspect the number of parameters of all arrays in the net:

In[23]:=

Out[23]=

Obtain the total number of parameters:

In[24]:=

Out[24]=

Obtain the layer type counts:

In[25]:=

Out[25]=

Display the summary graphic:

In[26]:=

Out[26]=

Export to MXNet

Export the net into a format that can be opened in MXNet:

In[27]:=

Out[27]=

Export also creates a net.params file containing parameters:

In[28]:=

Out[28]=

Get the size of the parameter file:

In[29]:=

Out[29]=

The size is similar to the byte count of the resource object:

In[30]:=

Out[30]=

Represent the MXNet net as a graph:

In[31]:=

Out[31]=

Construction Notebook

Download Construction Notebook

Open in Wolfram Cloud

Requirements

Wolfram Language 11.2 (September 2017) or above

Resource History

Date Created: 19 July 2017
Latest Update: 30 July 2021

Reference

X. Huang, S. Belongie, "Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization," arXiv:1703.06868 (2017)
Available from: https://github.com/xunhuang1995/AdaIN-style
Rights: MIT License