Wolfram ImageIdentify Net V2

Identify the main object in an image

This model is also available through the built-in function ImageIdentify

Released by Wolfram Research, this model is a part of the back end for the ImageIdentify function. It recognizes images across over four thousand categories and is available in two optimization variants. The "Quality" version uses the EfficientNet-V1 architecture for higher accuracy, while the "Speed" version uses the ShuffleNet-V2 architecture for faster inference.

Training Set Information

Model Information

Examples

Resource retrieval

Get the pre-trained net:

In[1]:=
NetModel["Wolfram ImageIdentify Net V2"]
Out[1]=

NetModel parameters

This model consists of a family of individual nets, each identified by a specific parameter combination. Inspect the available parameters:

In[2]:=
NetModel["Wolfram ImageIdentify Net V2", "ParametersInformation"]
Out[2]=

Pick a non-default net by specifying the parameters:

In[3]:=
NetModel["Wolfram ImageIdentify Net V2", "PerformanceGoal" -> "Speed"]
Out[3]=

Pick a default uninitialized net:

In[4]:=
NetModel["Wolfram ImageIdentify Net V2", "UninitializedEvaluationNet"]
Out[4]=

Basic usage

Classify an image:

In[5]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/315d9c51-6cf4-4811-814a-c1db3aaacab7"]
Out[5]=

The prediction is an Entity object, which can be queried:

In[6]:=
pred["Definition"]
Out[6]=

Get a list of available properties for the predicted Entity:

In[7]:=
pred["Properties"]
Out[7]=

Obtain the probabilities of the 10 most likely entities predicted by the net:

In[8]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/cdab1fc8-82c3-4921-8eca-ede5b1ebea1c"]
Out[8]=

An object outside the list of the ImageIdentify Net classes will be misidentified:

In[9]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/9a321a08-abb3-450f-b576-376b8c00da5b"]
Out[9]=

Obtain the list of names of all available classes:

In[10]:=
EntityValue[
  NetExtract[NetModel["Wolfram ImageIdentify Net V2"], "Output"][[
   "Labels"]], "Name"] // Short
Out[10]=

Feature extraction

Remove the last three layers of the trained net so that the net produces a vector representation of an image:

In[11]:=
extractor = Take[NetModel["Wolfram ImageIdentify Net V2"], {1, -4}]
Out[11]=

Get a set of images:

In[12]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/39378fac-b708-4bf8-8620-9c24f5003a71"]

Visualize the features of the set of images:

In[13]:=
FeatureSpacePlot[imgs, FeatureExtractor -> extractor, LabelingSize -> 60, ImageSize -> Large]
Out[13]=

Transfer learning

Use the pre-trained model to build a classifier for telling apart images of motorcycles and bicycles. Create a test set and a training set:

In[14]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/d0541ad4-c743-41d0-a1fe-baaf9fe18d0e"]
In[15]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/df4f93f4-e0b6-479f-ba2a-56cd69ba5fcc"]

Remove the linear layer from the pre-trained net:

In[16]:=
tempNet = Take[NetModel["Wolfram ImageIdentify Net V2"], {1, -4}]
Out[16]=

Create a new net composed of the pre-trained net followed by a linear layer and a softmax layer:

In[17]:=
newNet = NetChain[<|"pretrainedNet" -> tempNet, "linearNew" -> LinearLayer[], "softmax" -> SoftmaxLayer[]|>, "Output" -> NetDecoder[{"Class", {"bicycle", "motorcycle", "dog", "cat"}}]]
Out[17]=

Train on the dataset, freezing all the weights except for those in the linearNew layer (use TargetDevice -> "GPU" for training on a GPU):

In[18]:=
trainedNet = NetTrain[newNet, trainSet, LearningRateMultipliers -> {"linearNew" -> 1, _ -> 0}]
Out[18]=

Perfect accuracy is obtained on the test set:

In[19]:=
ClassifierMeasurements[trainedNet, testSet, "Accuracy"]
Out[19]=

Net information

Inspect the number of parameters of all arrays in the net:

In[20]:=
Information[NetModel["Wolfram ImageIdentify Net V2"], "ArraysElementCounts"] // Short
Out[20]=

Obtain the total number of parameters:

In[21]:=
Information[
 NetModel["Wolfram ImageIdentify Net V2"], "ArraysTotalElementCount"]
Out[21]=

Obtain the layer type counts:

In[22]:=
Information[
 NetModel["Wolfram ImageIdentify Net V2"], "LayerTypeCounts"]
Out[22]=

Export to ONNX

Export the net to the ONNX format:

In[23]:=
onnxFile = Export[FileNameJoin[{$TemporaryDirectory, "net.onnx"}], NetModel["Wolfram ImageIdentify Net V2"]]
Out[23]=

Get the size of the ONNX file:

In[24]:=
FileByteCount[onnxFile]
Out[24]=

Check some metadata of the ONNX model:

In[25]:=
{OpsetVersion, IRVersion} = {Import[onnxFile, "OperatorSetVersion"], Import[onnxFile, "IRVersion"]}
Out[25]=

Import the model back into Wolfram Language. However, the NetEncoder and NetDecoder will be absent because they are not supported by ONNX:

In[26]:=
Import[onnxFile]
Out[26]=
In[27]:=
NetModel["Wolfram ImageIdentify Net V2"]
Out[27]=

Requirements

Wolfram Language 13.2 (December 2022) or above

Resource History

Reference

  • Wolfram Research