Wolfram Neural Net Repository

Immediate Computable Access to Neural Net Models

Unguided Volumetric Regression Net for 3D Face Reconstruction

Reconstruct a 3D facial image from a 2D facial image

Released in 2017, this net tackles the problem of regressing a 3D facial model in an end-to-end fashion, bypassing many of the difficulties found in complex pipelines involving multiple stages. Starting from a single input image, the facial model is directly reconstructed as a three-dimensional array of pixel intensities. Its architecture is based on the "hourglass" design.

Number of layers: 1,029 | Parameter count: 19,277,384 | Trained size: 81 MB |

Training Set Information

Performance

Examples

Resource retrieval

Get the pre-trained net:

In[1]:=
NetModel["Unguided Volumetric Regression Net for 3D Face \ Reconstruction"]
Out[1]=

Basic usage

Get the volumetric model of a facial image:

In[2]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/1417bdb8-5ffa-4108-8fd8-a6dd0015ca4c"]

Visualize the volumetric representation:

In[3]:=
img3D = Image3D[255*face3D, "Byte", BoxRatios -> {1, 1, 0.5}, ViewPoint -> Below]
Out[3]=

Extract a 3D mesh:

In[4]:=
ImageMesh[img3D, Method -> "DualMarchingCubes", BoxRatios -> {1, 1, 0.5}, ViewPoint -> Below]
Out[4]=

Robustness to facial crop size

Get an image:

In[5]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/1fa90aed-0118-4b51-b1bd-e68d901e8e84"]

Crop the image at various sizes:

In[6]:=
crops = Table[ImageCrop[img, s, Bottom], {s, 290, 500, 70}]
Out[6]=

Inspect the Net performance across the crops:

In[7]:=
Image3D[255* NetModel[ "Unguided Volumetric Regression Net for 3D Face \ Reconstruction"][#], "Byte", BoxRatios -> {1, 1, 0.5}, ViewPoint -> Below, ImageSize -> Small] & /@ crops
Out[7]=

Net information

Inspect the number of parameters of all arrays in the net:

In[8]:=
NetInformation[ NetModel["Unguided Volumetric Regression Net for 3D Face \ Reconstruction"], "ArraysElementCounts"]
Out[8]=

Obtain the total number of parameters:

In[9]:=
NetInformation[ NetModel["Unguided Volumetric Regression Net for 3D Face \ Reconstruction"], "ArraysTotalElementCount"]
Out[9]=

Obtain the layer type counts:

In[10]:=
NetInformation[ NetModel["Unguided Volumetric Regression Net for 3D Face \ Reconstruction"], "LayerTypeCounts"]
Out[10]=

Display the summary graphic:

In[11]:=
NetInformation[ NetModel["Unguided Volumetric Regression Net for 3D Face \ Reconstruction"], "SummaryGraphic"]
Out[11]=

Export to MXNet

Export the net into a format that can be opened in MXNet:

In[12]:=
jsonPath = Export[FileNameJoin[{$TemporaryDirectory, "net.json"}], NetModel["Unguided Volumetric Regression Net for 3D Face \ Reconstruction"], "MXNet"]
Out[12]=

Export also creates a net.params file containing parameters:

In[13]:=
paramPath = FileNameJoin[{DirectoryName[jsonPath], "net.params"}]
Out[13]=

Get the size of the parameter file:

In[14]:=
FileByteCount[paramPath]
Out[14]=

The size is similar to the byte count of the resource object:

In[15]:=
ResourceObject[ "Unguided Volumetric Regression Net for 3D Face \ Reconstruction"]["ByteCount"]
Out[15]=

Requirements

Wolfram Language 11.2 (September 2017) or above

Resource History

  • Date Created: 17 October 2017
  • Latest Update: 21 June 2018

Reference

  • A. S. Jackson, A. Bulat, V. Argyriou, G. Tzimiropoulos, "Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression," arXiv:1703.07834 (2017)
  • Available from: https://github.com/AaronJackson/vrn
  • Rights: MIT License