Resource retrieval

Get the pre-trained net:

In[1]:=

Out[1]=

NetModel parameters

This model consists of a family of individual nets, each identified by a specific parameter combination. Inspect the available parameters:

In[2]:=

Out[2]=

Pick a non-default model by specifying the parameters:

In[3]:=

Out[3]=

Pick a non-default untrained net:

In[4]:=

Out[4]=

Basic usage

For each token, the net produces a vector of features:

In[5]:=

Out[5]=

Obtain the dimensions of the vectors:

In[6]:=

Out[6]=

Visualize the embeddings:

In[7]:=

Out[7]=

Use the embedding layer inside a NetChain:

In[8]:=

Out[8]=

BPE tokenization

The BPE tokenization can be extracted from the model as a NetEncoder:

In[9]:=

Out[9]=

The encoder segments the input string into words and subwords using BPE tokenization and outputs integer codes for each token:

In[10]:=

Out[10]=

Obtain the tokens. Rare words are usually split up into subwords:

In[11]:=

Out[11]=

Write a function to tokenize a string in any language:

In[12]:=

Compare tokenizations in various languages:

In[13]:=

Out[14]=

Feature visualization

Create two lists of related words:

In[15]:=

In[16]:=

Visualize relationships between the words using the net as a feature extractor:

In[17]:=

Out[17]=

Net information

Inspect the number of parameters of all arrays in the net:

In[18]:=

Out[18]=

Obtain the layer type counts:

In[19]:=

Out[19]=

Export to MXNet

Export the net into a format that can be opened in MXNet:

In[20]:=

Out[20]=

Export also creates a net.params file containing parameters:

In[21]:=

Out[21]=

Get the size of the parameter file:

In[22]:=

Out[22]=