ConceptNet Numberbatch Word Vectors V17.06

Represent words as vectors

Number of layers: 1 | Parameter count: 125,158,500 | Trained size: 503 MB |

Training Set Information

ConceptNet, a knowledge graph that connects words and phrases of natural language with labeled edges. GloVe, word vectors pre-trained on various datasets. word2vec, word vectors pre-trained on the Google News dataset. Open Subtitles 2016, a collection of movie subtitles.

Examples

Download Example Notebook

Open in Wolfram Cloud

Resource retrieval

Get the pre-trained net:

In[1]:=

Out[1]=

Basic usage

Use the net to obtain a list of word vectors:

In[2]:=

Out[2]=

Obtain the dimensions of the vectors:

In[3]:=

Out[3]=

Use the embedding layer inside a NetChain:

In[4]:=

Out[4]=

Feature visualization

Create two lists of related words:

In[5]:=

animals = {"alligator", "ant", "bear", "bee", "bird", "camel", "cat", "cheetah", "chicken", "chimpanzee", "cow", "crocodile", "deer", "dog", "dolphin", "duck", "eagle", "elephant", "fish", "fly"};

In[6]:=

fruits = {"apple", "apricot", "avocado", "banana", "blackberry", "blueberry", "cherry", "coconut", "cranberry", "grape", "turnip", "mango", "melon", "papaya", "peach", "pineapple", "raspberry", "strawberry", "ribes", "fig"};

Visualize relationships between the words using the net as a feature extractor:

In[7]:=

Out[7]=

Word analogies

Get the pre-trained net:

In[8]:=

Out[8]=

Get a list of tokens:

In[9]:=

Out[9]=

Obtain the vectors:

In[10]:=

Create an association whose keys are tokens and whose values are vectors:

In[11]:=

Find the eight nearest tokens to "king":

In[12]:=

Out[12]=

France is to Paris as Germany is to:

In[13]:=

Out[13]=

Net information

Inspect the number of parameters of all arrays in the net:

In[14]:=

$NetInformation[ NetModel["ConceptNet Numberbatch Word Vectors V17.06"], \ "ArraysElementCounts"]$

Out[14]=

Obtain the total number of parameters:

In[15]:=

$NetInformation[ NetModel["ConceptNet Numberbatch Word Vectors V17.06"], \ "ArraysTotalElementCount"]$

Out[15]=

Obtain the layer type counts:

In[16]:=

$NetInformation[ NetModel["ConceptNet Numberbatch Word Vectors V17.06"], \ "LayerTypeCounts"]$

Out[16]=

Export to MXNet

Export the net into a format that can be opened in MXNet:

In[17]:=

Out[17]=

Export also creates a net.params file containing parameters:

In[18]:=

Out[18]=

Get the size of the parameter file:

In[19]:=

Out[19]=

The size is similar to the byte count of the resource object:

In[20]:=

Out[20]=

Represent the MXNet net as a graph:

In[21]:=

Out[21]=

Construction Notebook

Download Construction Notebook

Open in Wolfram Cloud

Requirements

Wolfram Language 11.3 (March 2018) or above

Resource History

Date Created: 23 January 2018
Latest Update: 16 May 2022

Reference

R. Speer, J. Chin, C. Havasi, "ConceptNet 5.5: An Open Multilingual Graph of General Knowledge," arXiv:1612.03975 (2016)
Available from: https://github.com/commonsense/conceptnet-numberbatch
Rights: Creative Commons Attribution 4.0 International (CC BY-SA 4.0)