Wolfram Computation Meets Knowledge

Wolfram ImageIdentify Net V1

Identify the main object in an image

Released in 2017 by Wolfram Research, this net was trained on over 4,000 classes of objects. It is part of the back end for the ImageIdentify function in Wolfram Language 11.1. It was designed to achieve a good balance among classification accuracy, size and evaluation speed.

Number of layers: 232 | Parameter count: 14,713,147 | Trained size: 65 MB

Training Set Information

Examples

Resource retrieval

Retrieve the resource object:

In[1]:=
ResourceObject["Wolfram ImageIdentify Net V1"]
Out[1]=

Get the pre-trained net:

In[2]:=
NetModel["Wolfram ImageIdentify Net V1"]
Out[2]=

Basic usage

Classify an image:

In[3]:=
Out[3]=

The prediction is an Entity object, which can be queried:

In[4]:=
pred["Definition"]
Out[4]=

Get a list of available properties of the predicted Entity:

In[5]:=
pred["Properties"]
Out[5]=

Obtain the probabilities of the ten most likely entities predicted by the net:

In[6]:=
Out[6]=

An object outside the list of the ImageIdentify dataset will be misidentified:

In[7]:=
Out[7]=

Obtain the list of names of all available classes:

In[8]:=
EntityValue[
 NetExtract[NetModel["Wolfram ImageIdentify Net V1"], "Output"][[
  "Labels"]], "Name"]
Out[8]=

Feature extraction

Remove the last two layers of the trained net so that the net produces a vector representation of an image:

In[9]:=
extractor = Take[NetModel["Wolfram ImageIdentify Net V1"], {1, -3}]
Out[9]=

Get a set of images:

In[10]:=

Visualize the features of a set of images:

In[11]:=
FeatureSpacePlot[imgs, FeatureExtractor -> extractor]
Out[11]=

Visualize convolutional weights

Extract the weights of the first convolutional layer in the trained net:

In[12]:=
weights = 
  NetExtract[
   NetModel["Wolfram ImageIdentify Net V1"], {"conv_1", "Weights"}];

Show the dimensions of the weights:

In[13]:=
Dimensions[weights]
Out[13]=

Visualize the weights as a list of 64 images of size 7x7:

In[14]:=
ImageAdjust[Image[#, Interleaving -> False]] & /@ weights
Out[14]=

Transfer learning

Use the pre-trained model to build a classifier for telling apart images of dogs and cats. Create a test set and a training set:

In[15]:=
In[16]:=

Remove the linear layer from the pre-trained net:

In[17]:=
tempNet = Take[NetModel["Wolfram ImageIdentify Net V1"], {1, -3}]
Out[17]=

Create a new net composed of the pre-trained net followed by a linear layer and a softmax layer:

In[18]:=
newNet = NetChain[<|"pretrainedNet" -> tempNet, 
   "linearNew" -> LinearLayer[], "softmax" -> SoftmaxLayer[]|>, 
  "Output" -> NetDecoder[{"Class", {"cat", "dog"}}]]
Out[18]=

Train on the dataset, freezing all the weights except for those in the "linearNew" layer (use TargetDevice -> "GPU" for training on a GPU):

In[19]:=
trainedNet = 
 NetTrain[newNet, trainSet, 
  LearningRateMultipliers -> {"linearNew" -> 1, _ -> 0}]
Out[19]=

Perfect accuracy is obtained on the test set:

In[20]:=
ClassifierMeasurements[trainedNet, testSet, "Accuracy"]
Out[20]=

Export to MXNet

Export the net into a format that can be opened in MXNet :

In[21]:=
jsonPath = 
 Export[FileNameJoin[{$TemporaryDirectory, "net.json"}], 
  NetModel["Wolfram ImageIdentify Net V1"], "MXNet"]
Out[21]=

Export also creates a net.params file containing parameters:

In[22]:=
paramPath = FileNameJoin[{DirectoryName[jsonPath], "net.params"}]
Out[22]=

Get the size of the parameter file:

In[23]:=
FileByteCount[paramPath]
Out[23]=

The size is similar to the byte count of the resource object:

In[24]:=
ResourceObject["Wolfram ImageIdentify Net V1"]["ByteCount"]
Out[24]=

Requirements

Wolfram Language 11.1 (March 2017) or above

Reference