Identify and Label Parts of an Image

Open in Cloud
Download Notebook

Use a pre-trained neural net to segment an image into semantic concepts

Get a neural net model that segments an image into semantic concepts:

In[1]:=

net=NetModel["Ademxapp Model A1 Trained on Cityscapes Data"]

Out[1]=

NetChain



Input port:	image
Output port:	class



Define the image to segment:

In[2]:=

image=

;

Write an evaluation function to handle net reshaping and resampling of input and output:

In[3]:=

netevaluate[img_,device_:"CPU"]:=Block[{net,resized,encData,dec,mean,var,prob},net=NetModel["Ademxapp Model A1 Trained on Cityscapes Data"];resized=ImageResize[img,{1024}];encData=Normal@NetExtract[net,"Input"];dec=NetExtract[net,"Output"];{mean,var}=Lookup[encData,{"MeanImage","VarianceImage"}];prob=NetReplacePart[net,{"Input"NetEncoder[{"Image",ImageDimensions@resized,"MeanImage"mean,"VarianceImage"var}],"Output"Automatic}][resized,TargetDevicedevice];prob=ArrayResample[Normal@prob,Append[Reverse@ImageDimensions@img,19]];dec[prob]]

Run the evaluation function:

In[4]:=

mask=netevaluate[image];

Define the labels associated with each class number in this

NetModel

In[5]:=

labels={"road","sidewalk","building","wall","fence","pole","traffic light","traffic sign","vegetation","terrain","sky","person","rider","car","truck","bus","train","motorcycle","bicycle"};

Identify the detected labels and create an image legend with them:

In[6]:=

detected=DeleteDuplicates[Flatten[mask]];legends=AssociationThread[detected,labels〚detected〛]

Out[6]=

3building,11sky,7traffic light,8traffic sign,6pole,15truck,12person,14car,18motorcycle,1road,2sidewalk,13rider,16bus

Highlight the resulting segmentation over the original image:

In[7]:=

HighlightImage[image,Legended[mask,legends]]

Out[7]=

Related Symbols

FindFaces
ImageCases
NetModel
NetChain
NetEncoder
NetExtract
HighlightImage

Publisher Information

Contributed by: Wolfram Staff

Wolfram Language Example Repository

Identify and Label Parts of an Image

See Also

Publisher Information