#
Wolfram Neural Net Repository

Immediate Computable Access to Neural Net Models

Generate hand-drawn sketches

Released in 2017, this collection features the models for unconditional generation of Sketch-RNN, which produce simple hand-drawn sketches (or complete a partial input sketch) represented as a sequence of pen strokes. The nets predict the next pen movement given an input sequence (possibly empty) of movements and are trained with teacher forcing. Each pen movement is sampled from a mixture of normal distributions, while a categorical distribution regulates whether the pen is drawing a line, it's lifted from the paper or the drawing has ended. The nets produce the parameters of such distributions.

- QuickDraw, a dataset of vector drawings obtained by the Quick, Draw! website. The dataset consists of hundreds of classes of objects, each having 70,000 sketches for training, 2,500 for validation and 2,500 for testing.

Get the pre-trained net:

In[1]:= |

Out[1]= |

This model consists of a family of individual nets, each identified by a specific parameter combination. Inspect the available parameters and their default values:

In[2]:= |

Out[2]= |

Pick a non-default net by specifying the parameters:

In[3]:= |

Out[3]= |

Pick a non-default uninitialized net:

In[4]:= |

Out[4]= |

Check the default parameter combination:

In[5]:= |

Out[5]= |

Define an evaluation function to generate a sketch from a fixed initial condition using temperature sampling:

In[6]:= |

In[7]:= |

Generate four sketches of a cat:

In[8]:= |

Out[8]= |

The third optional argument is a “temperature” parameter that regulates sampling. A higher temperature increases the variability in the output, increasing the probability of sampling less likely strokes:

In[9]:= |

Out[9]= |

Very high temperature settings are equivalent to sampling from a flat distribution:

In[10]:= |

Out[10]= |

Very low temperature settings further increase the probability of extracting more likely strokes. Sampling at zero temperature is equivalent to always picking the stroke with maximum probability, and the function produces the same sketch every time:

In[11]:= |

Out[11]= |

Create a function which displays an animation showing the object being drawn from the sequence of pen strokes chosen by the network:

In[12]:= |

In[13]:= |

Out[13]= |

In[14]:= |

Out[14]= |

This model represents sketches as a sequence of pen strokes, each identified as a vector of five elements: {Δx, Δy, p_{1}, p_{2}, p_{3}}. The elements {Δx, Δy} represent the pen movement in the sketch plane, while {p_{1}, p_{2}, p_{3}} is a one-hot vector representing the state of the pen. The state {1, 0, 0} indicates that the pen is touching the paper and a segment will be drawn, {0, 1, 0} means the pen is not touching the paper and {0, 0, 1} indicates that the sketch is finished:

In[15]:= |

The model works like a language model, reading a sequence of pen strokes and predicting the next one:

In[16]:= |

Out[16]= |

Extract the “Function” net decoder:

In[17]:= |

Out[17]= |

Inspect the associated Function:

In[18]:= |

Out[18]= |

This decoder supports properties. The default one, “Decision”, returns the pen stroke with highest associated probability:

In[19]:= |

In[20]:= |

Out[20]= |

It is possible to perform temperature sampling from the output distribution with the property “RandomSample”. In this case, the evaluation will return different results every time:

In[21]:= |

Out[21]= |

In[22]:= |

Out[22]= |

The property “Distribution” returns the full probability distribution for the stroke direction and the pen state. The stroke direction distribution is a mixture of two-dimensional normal distributions, while the distribution of the pen state is a categorical distribution with three probability values for 1, 2 and 3:

In[23]:= |

Out[23]= |

Inspect the number of parameters of all arrays in the net:

In[24]:= |

Out[24]= |

Obtain the total number of parameters:

In[25]:= |

Out[25]= |

Obtain the layer type counts:

In[26]:= |

Out[26]= |

Display the summary graphic:

In[27]:= |

Out[27]= |

Wolfram Language 12.0 (April 2019) or above

- D. Ha, D. Eck, "A Neural Representation of Sketch Drawings," arXiv:1704.03477 (2017)
- (available from https://github.com/tensorflow/magenta-demos/tree/master/sketch-rnn-js)
- Rights: Apache 2.0 License