Wolfram Computation Meets Knowledge

Single-Image Depth Perception Net Trained on Depth in the Wild Data

Estimate the depth map of an image

Released in 2016, this neural net was trained to predict the relative depth map from a single image using a novel technique based on sparse ordinal annotations. Each training example only needs to be annotated with a pair of points and its relative distance to the camera. After training, the net is able to reconstruct the full depth map. Its architecture is based on the "hourglass" design.

Number of layers: 501 | Parameter count: 5,385,185 | Trained size: 23 MB

Training Set Information

Performance

Examples

Resource retrieval

Retrieve the resource object:

In[1]:=
ResourceObject["Single-Image Depth Perception Net Trained on Depth in \ the Wild Data"]
Out[1]=

Get the pre-trained net:

In[2]:=
NetModel["Single-Image Depth Perception Net Trained on Depth in the \ Wild Data"]
Out[2]=

Basic usage

Obtain the depth map of an image:

In[3]:=
CloudGet["https://www.wolframcloud.com/objects/5658e194-16f2-4bf4-9295-d260e859b3fc"] (* Evaluate this cell to copy the example input from a cloud object *)
Out[3]=

Obtain the dimensions of the depth map:

In[4]:=
Dimensions[depthMap]
Out[4]=

Show the depth map:

In[5]:=
ImageAdjust[Image[depthMap]]
Out[5]=

Visualize a 3D model

Get an image:

In[6]:=
CloudGet["https://www.wolframcloud.com/objects/1672fe0a-c7a5-4bfa-8399-e8480195a31b"] (* Evaluate this cell to copy the example input from a cloud object *)

Obtain the depth map:

In[8]:=
depthMap = NetModel["Single-Image Depth Perception Net Trained on Depth in the \ Wild Data"][img]
Out[8]=

Visualize a 3D model using the depth map:

In[9]:=
ListPlot3D[-depthMap, PlotStyle -> Texture[ImageReflect[img]], PlotTheme -> {"Minimal", "NoAxes"}, ViewPoint -> {0, 0.76, 1.84}]
Out[9]=

Adapt to any size

The recommended way to deal with image sizes and aspect ratios is to resample the depth map after the net evaluation. Get an image:

In[10]:=
CloudGet["https://www.wolframcloud.com/objects/d8f9d1e4-ec6d-49ef-b2b2-8e702974b798"] (* Evaluate this cell to copy the example input from a cloud object *)

Obtain the dimensions of the image:

In[11]:=
ImageDimensions[img]
Out[11]=

Obtain the depth map

In[12]:=
depthMap = NetModel["Single-Image Depth Perception Net Trained on Depth in the \ Wild Data"][img];

Obtain the dimensions of the depth map:

In[13]:=
Dimensions[depthMap]
Out[13]=

Resample the depth map and visualize it:

In[14]:=
ImageAdjust[ Image[ArrayResample[depthMap, Reverse[ImageDimensions[img]]]]]
Out[14]=

Net information

Inspect the number of parameters of all arrays in the net:

In[15]:=
NetInformation[ NetModel["Single-Image Depth Perception Net Trained on Depth in the \ Wild Data"], "ArraysElementCounts"]
Out[15]=

Obtain the total number of parameters:

In[16]:=
NetInformation[ NetModel["Single-Image Depth Perception Net Trained on Depth in the \ Wild Data"], "ArraysTotalElementCount"]
Out[16]=

Obtain the layer type counts:

In[17]:=
NetInformation[ NetModel["Single-Image Depth Perception Net Trained on Depth in the \ Wild Data"], "LayerTypeCounts"]
Out[17]=

Display the summary graphic:

In[18]:=
NetInformation[ NetModel["Single-Image Depth Perception Net Trained on Depth in the \ Wild Data"], "SummaryGraphic"]
Out[18]=

Export to MXNet

Export the net into a format that can be opened in MXNet:

In[19]:=
jsonPath = Export[FileNameJoin[{$TemporaryDirectory, "net.json"}], NetModel["Single-Image Depth Perception Net Trained on Depth in the \ Wild Data"], "MXNet"]
Out[19]=

Export also creates a net.params file containing parameters:

In[20]:=
paramPath = FileNameJoin[{DirectoryName[jsonPath], "net.params"}]
Out[20]=

Get the size of the parameter file:

In[21]:=
FileByteCount[paramPath]
Out[21]=

The size is similar to the byte count of the resource object:

In[22]:=
ResourceObject[ "Single-Image Depth Perception Net Trained on Depth in the Wild \ Data"]["ByteCount"]
Out[22]=

Requirements

Wolfram Language 11.2 (September 2017) or above

Reference