Ademxapp Model A1 Trained on PASCAL VOC2012 and MS-COCO Data

Segment an image into various semantic component classes

Released in 2016 by the University of Adelaide, this model exploits recent progress in the understanding of residual architectures. With only 17 residual units, it is able to outperform previous, much deeper architectures.

Number of layers: 141 | Parameter count: 124,090,325 | Trained size: 497 MB |

Training Set Information

PASCAL VOC2012, a dataset for various computer vision tasks using 20 object classes and 10 action classes. MS-COCO 2014, a dataset for various computer vision tasks using 91 object classes.

Performance

This model achieves 82.86% mean IoU accuracy on the PASCAL VOC2012 dataset.

Examples

Download Example Notebook

Open in Wolfram Cloud

Resource retrieval

Get the pre-trained net:

In[1]:=

$NetModel["Ademxapp Model A1 Trained on PASCAL VOC2012 and MS-COCO \ Data"]$

Out[1]=

Evaluation function

Write an evaluation function to handle net reshaping and resampling of input and output:

In[2]:=

$netevaluate[img_, device_ : "CPU"] := Block[ {net, resized, encData, dec, mean, var, prob}, net = NetModel[ "Ademxapp Model A1 Trained on PASCAL VOC2012 and MS-COCO Data"]; resized = ImageResize[img, {504}]; encData = Normal@NetExtract[net, "Input"]; dec = NetExtract[net, "Output"]; {mean, var} = Lookup[encData, {"MeanImage", "VarianceImage"}]; prob = NetReplacePart[net, {"Input" -> NetEncoder[{"Image", ImageDimensions@resized, "MeanImage" -> mean, "VarianceImage" -> var}], "Output" -> Automatic} ][resized, TargetDevice -> device]; prob = ArrayResample[prob, Append[Reverse@ImageDimensions@img, 21]]; dec[prob] ]$

Label list

Define the label list for this model. Integers in the model’s output correspond to elements in the label list:

In[3]:=

labels = {"background", "airplane", "bicycle", "bird", "boat", "bottle", "bus", "car", "cat", "chair", "cow", "table", "dog", "horse", "motorcycle", "person", "plant", "sheep", "sofa", "train",
"television"};

Basic usage

Obtain a segmentation mask for a given image:

In[4]:=

(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/ad92120e-e6a8-4425-9dca-5ae1cac63db1"]

Inspect which classes are detected:

In[5]:=

Out[5]=

In[6]:=

Out[6]=

Visualize the mask:

In[7]:=

Out[7]=

Advanced visualization

Associate classes to colors:

In[8]:=

Write a function to overlap the image and the mask with a legend:

In[9]:=

$result[img_, device_ : "CPU"] := Block[ {mask, classes, maskPlot, composition}, mask = netevaluate[img, device]; classes = DeleteDuplicates[Flatten@mask]; maskPlot = Colorize[mask, ColorRules -> indexToColor]; composition = ImageCompose[img, {maskPlot, 0.5}]; Legended[ Row[Image[#, ImageSize -> Large] & /@ {maskPlot, composition}], SwatchLegend[indexToColor[[classes, 2]], labels[[classes]]]] ]$

Inspect the results:

In[10]:=

Out[10]=

In[11]:=

(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/d0f8546e-bf43-4f7a-9c0f-341756b4496c"]

Out[11]=

In[12]:=

(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/fb70cd13-75a3-4c5a-8dd7-4a224178811a"]

Out[12]=

Net information

Inspect the number of parameters of all arrays in the net:

In[13]:=

$NetInformation[ NetModel["Ademxapp Model A1 Trained on PASCAL VOC2012 and MS-COCO \ Data"], "ArraysElementCounts"]$

Out[13]=

Obtain the total number of parameters:

In[14]:=

$NetInformation[ NetModel["Ademxapp Model A1 Trained on PASCAL VOC2012 and MS-COCO \ Data"], "ArraysTotalElementCount"]$

Out[14]=

Obtain the layer type counts:

In[15]:=

$NetInformation[ NetModel["Ademxapp Model A1 Trained on PASCAL VOC2012 and MS-COCO \ Data"], "LayerTypeCounts"]$

Out[15]=

Display the summary graphic:

In[16]:=

$NetInformation[ NetModel["Ademxapp Model A1 Trained on PASCAL VOC2012 and MS-COCO \ Data"], "SummaryGraphic"]$

Out[16]=

Export to MXNet

Export the net into a format that can be opened in MXNet:

In[17]:=

$jsonPath = Export[FileNameJoin[{$TemporaryDirectory, "net.json"}], NetModel["Ademxapp Model A1 Trained on PASCAL VOC2012 and MS-COCO \ Data"], "MXNet"]$

Out[17]=

Export also creates a net.params file containing parameters:

In[18]:=

Out[18]=

Get the size of the parameter file:

In[19]:=

Out[19]=

The size is similar to the byte count of the resource object:

In[20]:=

$ResourceObject[ "Ademxapp Model A1 Trained on PASCAL VOC2012 and MS-COCO \ Data"]["ByteCount"]$

Out[20]=

Construction Notebook

Download Construction Notebook

Open in Wolfram Cloud

Requirements

Wolfram Language 11.3 (March 2018) or above

Resource History

Date Created: 22 March 2018
Latest Update: 21 June 2018

Reference

Z. Wu, C. Shen, A. van den Hengel, "Wider or Deeper: Revisiting the ResNet Model for Visual Recognition," arXiv:1611.10080 (2016)
Available from: https://github.com/itijyou/ademxapp
Rights: Apache 2.0 License