Wolfram Computation Meets Knowledge

Pix2pix Street-Map-to-Photo Translation

Generate a satellite photo from a street map

Released in 2016, this model is an application of a powerful method for general-purpose image-to-image translation using conditional adversarial networks. The automatic learning of the loss function with the adversarial networks technique allows the same paradigm to generalize across a wide range of image translation tasks. The architecture enables an efficient aggregation of features of multiple scales through skip connections with concatenations. This particular model was trained to generate a street map from a satellite photo.

Number of layers: 56 | Parameter count: 54,419,459 | Trained size: 218 MB

Training Set Information

Examples

Resource retrieval

Get the pre-trained net:

In[1]:=
NetModel["Pix2pix Street-Map-to-Photo Translation"]
Out[2]=

Basic usage

Start with a street map:

In[3]:=
CloudGet["https://www.wolframcloud.com/objects/902a9970-3cda-41dc-9999-27b73e7f132e"] (* Evaluate this cell to copy the example input from a cloud object *)

Draw a satellite photo from a street map:

In[4]:=
NetModel["Pix2pix Street-Map-to-Photo Translation"][map]
Out[5]=

Evaluate accuracy

Overlap map and prediction:

In[6]:=
ImageCompose[NetModel["Pix2pix Street-Map-to-Photo Translation"][map],
  SetAlphaChannel[map, 0.6]]
Out[7]=

Compare the generated satellite photo with the actual satellite photo:

In[8]:=
CloudGet["https://www.wolframcloud.com/objects/32450ded-d6c8-48ea-bebc-6a6ad6ff00ef"] (* Evaluate this cell to copy the example input from a cloud object *)
Out[9]=

Net information

Inspect the number of parameters of all arrays in the net:

In[10]:=
NetInformation[
 NetModel["Pix2pix Street-Map-to-Photo Translation"], \
"ArraysElementCounts"]
Out[110]=

Obtain the total number of parameters:

In[111]:=
NetInformation[
 NetModel["Pix2pix Street-Map-to-Photo Translation"], \
"ArraysTotalElementCount"]
Out[112]=

Obtain the layer type counts:

In[113]:=
NetInformation[
 NetModel["Pix2pix Street-Map-to-Photo Translation"], \
"LayerTypeCounts"]
Out[114]=

Display the summary graphic:

In[115]:=
NetInformation[
 NetModel["Pix2pix Street-Map-to-Photo Translation"], \
"SummaryGraphic"]
Out[116]=

Export to MXNet

Export the net into a format that can be opened in MXNet:

In[117]:=
jsonPath = 
 Export[FileNameJoin[{$TemporaryDirectory, "net.json"}], 
  NetModel["Pix2pix Street-Map-to-Photo Translation"], "MXNet"]
Out[118]=

Export also creates a net.params file containing parameters:

In[119]:=
paramPath = FileNameJoin[{DirectoryName[jsonPath], "net.params"}]
Out[119]=

Get the size of the parameter file:

In[120]:=
FileByteCount[paramPath]
Out[120]=

The size is similar to the byte count of the resource object:

In[121]:=
ResourceObject["Pix2pix Street-Map-to-Photo Translation"]["ByteCount"]
Out[122]=

Represent the MXNet net as a graph:

In[123]:=
Import[jsonPath, {"MXNet", "NodeGraphPlot"}]
Out[123]=

Requirements

Wolfram Language 11.2 (September 2017) or above

Resource History

Reference