Learn a Parameterization of a Manifold

Use an autoencoder to reconstruct original input data

Learn a parameterization of the manifold along which the input data lies by using an autoencoder, a network with a "bottleneck" layer that learns to reconstruct the original input.

Sample training data from a part of a synthetic two-dimensional manifold:

In[1]:=

manifold=Table[AngleVector[{x,0.9Pix}]+x/20*RandomVariate[NormalDistribution[],2],{x,0,1,0.001}];

Plot the training data along with the original manifold:

In[2]:=

Show[plot=ListPlot[manifold,PlotStyleOrange],original=ParametricPlot[AngleVector[{x,0.9Pix}],{x,0,1},PlotStyleStandardRed]]

Out[2]=

Create a net with a "bottleneck" layer in order to learn a parameterization of the manifold:

In[3]:=

net=NetChain[{25,Ramp,1,25,Ramp,2},"Input"2]

Out[3]=

Create a loss network that computes a loss based on the "reconstruction error"—a measure of the degree to which the network can produce an output that is identical to its input:

In[4]:=

lossNet=NetGraph["bottleneck"net,"loss"MeanSquaredLossLayer[],{"bottleneck""loss",NetPort["Input"]NetPort[2,"Target"]}]

Out[4]=

Train the loss network on the manifold and extract the original net from the loss network:

In[5]:=

lossNet=NetTrain[lossNet,"Input"manifold,BatchSize4096];trained=NetExtract[lossNet,"bottleneck"]

Out[5]=

Visualize how the net projects arbitrary points onto the manifold:

In[6]:=

{{xmin,xmax},{ymin,ymax}}=CoordinateBounds[manifold,.2];Show[plot,StreamPlot[trained[{x,y}]-{x,y},{x,xmin,xmax},{y,ymin,ymax}]]

Out[6]=

Split the net into an "encoder" and "decoder" network (the encoder parameterizes points using a single scalar value, whereas the decoder reconstructs the point from this parameterization):

In[7]:=

decoder=Drop[trained,3]encoder=Take[trained,3]

Out[7]=

Out[8]=

Color each point in the original manifold by its parameterization under the encoder:

In[9]:=

ListPlot[Style[#,Hue[First[0.3+encoder[#]]/3]]&/@manifold]

Out[9]=

Obtain the range of the parameterization by applying the encoder to the manifold:

In[10]:=

{min,max}=MinMax[encoder[manifold]]

Out[10]=

{-0.0316851,2.56169}

Show the reconstruction over this range along with the original manifold and sampled data:

In[11]:=

LegendedShow[plot,original,ListLinePlot[Table[decoder[x],{x,min,max,.01}]]],

LineLegend[

]



Out[11]=

	data
	manifold
	reconstruction

Publisher Information

Contributed by: Wolfram Staff

Learn a Parameterization of a Manifold

Related Symbols

Publisher Information