Sketch-RNN Trained on QuickDraw Data

Generate hand-drawn sketches

Released in 2017, this collection features the models for unconditional generation of Sketch-RNN, which produce simple hand-drawn sketches (or complete a partial input sketch) represented as a sequence of pen strokes. The nets predict the next pen movement given an input sequence (possibly empty) of movements and are trained with teacher forcing. Each pen movement is sampled from a mixture of normal distributions, while a categorical distribution regulates whether the pen is drawing a line, it's lifted from the paper or the drawing has ended. The nets produce the parameters of such distributions.

Number of models: 114

Training Set Information

Examples

Resource retrieval

Get the pre-trained net:

In[1]:=
NetModel["Sketch-RNN Trained on QuickDraw Data"]
Out[1]=

Evaluation function

Define an evaluation function to generate a sketch from a fixed initial condition using temperature sampling:

In[6]:=
drawSketch[obj_, temp_ : 0.01, maxLen_ : 300] := Block[{stateObject, lastPos, pos, stroke, segments, time, lastAction, action, offset}, stateObject = NetStateObject@
     NetModel[{"Sketch-RNN Trained on QuickDraw Data", "Object" -> obj}];
   lastPos = pos = {0, 0};
   stroke = {0, 0, 0, 0, 0};
   segments = Table[{}, maxLen];
   time = 0; action = 1;
   While[time++ < maxLen,
    lastAction = action;
    stroke = stateObject[{stroke}, {"RandomSample", "Temperature" -> N@temp}];
    offset = stroke[[;; 2]];
    action = First@Ordering[stroke[[3 ;;]], -1];
    lastPos = pos;
    pos += offset*{1, -1};
    Switch[lastAction,
     1, segments[[time]] = {lastPos, pos},
     3, Break[]]
    ];
   segments
   ];
In[7]:=
netevaluate[obj_, temp_ : 0.01, maxLen_ : 300] := Graphics@Line@drawSketch[obj, temp, maxLen]

Basic usage

Generate four sketches of a cat:

In[8]:=
Table[netevaluate["Cat"], 4]
Out[8]=

The third optional argument is a “temperature” parameter that regulates sampling. A higher temperature increases the variability in the output, increasing the probability of sampling less likely strokes:

In[9]:=
Table[netevaluate["Cat", 0.4], 4]
Out[9]=

Very high temperature settings are equivalent to sampling from a flat distribution:

In[10]:=
Table[netevaluate["Elephant", 3], 4]
Out[10]=

Very low temperature settings further increase the probability of extracting more likely strokes. Sampling at zero temperature is equivalent to always picking the stroke with maximum probability, and the function produces the same sketch every time:

In[11]:=
Table[netevaluate["Chair", 0], 4]
Out[11]=

Requirements

Wolfram Language 12.0 (April 2019) or above

Resource History

Reference