Classify Pollen Images Using Neural Nets

Use image feature extraction and neural networks to classify pollen samples of different flowers

Get the dataset containing pollen name, sample image of the flower, size of the pollen and a few sample pollen images taken under the microscope:

In[1]:=

data=ResourceData["Example""Classify Pollen Using Neural Nets","PollenData"];

Take a look into the dataset:

In[2]:=

Dataset[data]

Out[2]=

Display the first record of the dataset:

In[3]:=

First[data]//Normal//TableForm

Out[3]//TableForm=

Obtain the training set:

In[4]:=

trainingSet=Flatten@Map[Thread[#["Pollen"]#["Name"]]&,data];

Notice that this dataset has only 43 sample files:

In[5]:=

Length[trainingSet]

Out[5]=

Training a neural net on a small number of data samples is a challenge. Transfer learning and data augmentation can be used to cope with this situation. Create a function using

ImageRotate

and

ImageReflect

to augment images by rotation and reflection:

In[6]:=

SetAttributes[imageAugmentation,Listable];imageAugmentation[img_Image]:=With[{imgs=NestList[ImageRotate,img,3]},Join[imgs,Map[ImageReflect,imgs]]]

Generate a randomized training set with 344 samples through augmentation:

In[7]:=

augmentedSet=RandomSample@Flatten@Map[Thread[imageAugmentation[First[#]]Last[#]]&,trainingSet];

In[8]:=

Length[augmentedSet]

Out[8]=

344

Utilize transfer learning by building a feature extractor based on a neural network that has been trained on image features in general:

In[9]:=

fef=FeatureExtraction[augmentedSet〚All,1〛,"ImageFeatures"]

Out[9]=

FeatureExtractorFunction



Input type: Image

Output type: NumericalVector

(157)

Data not saved. Save now



Convert all pollen images to 157-dimensional semantic feature vectors:

In[10]:=

featureSet=MapAt[fef,augmentedSet,{All,1}];

Here is one of these feature vectors:

In[11]:=

featureSet//First

Out[11]=

{2.6024,-11.8079,-6.47409,5.71261,2.40785,0.838556,4.45632,-12.1499,-6.80065,1.34055,-1.7219,-2.72864,-6.02721,3.45356,-6.29699,2.01598,0.673213,4.46025,0.618075,-1.53008,-0.0526222,7.38471,-1.5456,-4.11362,0.478968,0.461671,0.445116,-0.788801,-3.9558,-2.90067,-0.766725,0.435891,-1.78143,1.33622,-1.65682,-0.284767,1.77236,-0.328646,-0.38451,-0.0437036,-1.27227,1.81501,-0.221405,0.00856021,-3.03945,-2.38487,1.65127,0.0320591,1.23521,-0.976385,-0.396959,1.4833,-1.56013,-2.76185,-1.26223,3.07055,-1.32523,-0.554053,-0.333728,0.176417,-0.538688,-1.11603,-3.66852,-0.951365,1.34953,1.20856,-0.332294,-1.49809,-0.727843,0.277333,-0.248638,-1.13719,-0.627032,1.42019,-0.575863,2.03381,-0.782347,-1.55485,-0.632207,-1.60282,-1.82471,0.452884,-1.0883,-0.424244,0.85956,1.45536,0.724384,-1.34133,0.610167,2.35685,0.740861,0.674889,1.57106,1.07816,-0.486559,1.10303,0.316914,3.82732,3.21979,1.78603,-0.213873,-2.13581,0.280816,0.141642,-2.59566,-0.572406,-1.7619,0.37532,-1.36935,0.320929,0.993498,-0.243302,-0.511394,-0.145177,-2.04656,2.15532,1.41785,0.367625,2.46577,-1.12193,0.968279,-0.661242,-0.964397,1.14479,-2.27708,0.683897,-0.0885762,-0.350247,-0.568102,-0.491821,-0.924311,0.439325,0.365899,-0.290101,0.721689,-2.45672,-2.14613,1.76566,-0.266531,-0.681453,0.0683797,0.576281,0.469676,-0.105426,-1.33819,-0.629104,0.204208,0.635878,-0.722099,-0.842307,-0.341967,1.59128,0.256555,1.43807,0.565141,-1.23458,0.0411607}lavandula angustifolia

Display the neural network features via a three-dimensional

-distributed stochastic neighbor embedding (t-SNE):

In[12]:=

fsp=FeatureSpacePlot3DtrainingSetAll,1,FeatureExtractorfef,

...

/.img_ImageRemoveBackground[img]

Out[12]=

Build a small, two-layer neural network to act as a pollen classifier:

In[13]:=

dim=Length[featureSet〚1,1〛];lables=Union[featureSet〚All,2〛];nn=NetChain[{LinearLayer[64],BatchNormalizationLayer[],ElementwiseLayer[Ramp],LinearLayer[6],SoftmaxLayer[]},"Input"dim,"Output"NetDecoder[{"Class",lables}]]

Out[13]=

NetChain

