Resource retrieval

Get the pre-trained net:

In[1]:=

Out[1]=

Basic usage

Identify an Audio object:

In[2]:=

Out[2]=

The prediction is an Entity object, which can be queried:

In[3]:=

Out[3]=

Get a list of available properties of the predicted Entity:

In[4]:=

Out[4]=

Obtain the probabilities of the ten most likely entities predicted by the net:

In[5]:=

Out[5]=

The probabilities do not sum to 1 since the net was trained as a collection of independent binary classifiers, one per each class. This reflects the possibility of having multiple sound classes in a single recording.

The network was trained on the AudioSet dataset, where each audio signal is annotated with the sound classes/sources that are present in the recording. The labels are organized in an ontology of about 632 classes that span a very wide domain of sound types or sources, from musical instruments and music types to animal, mechanical and human sounds. Obtain the list of names of all available classes:

In[6]:=

Out[6]=

NetModel parameters

Inspect the available parameters:

In[7]:=

Out[7]=

Pick a non-default model by specifying the parameters:

In[8]:=

Out[8]=

Pick a non-default untrained net:

In[9]:=

Out[9]=

Feature extraction

The core of the network takes a fixed-size chunk of the mel-spectrogram of the input signal and is mapped over overlapping chunks using NetMapOperator. Extract the core net:

In[10]:=

Out[10]=

Chop off the last few layers in charge of the classification:

In[11]:=

Out[11]=

This net takes a single chunk of the input signal and outputs a tensor of semantically meaningful features. Reconstruct the whole variable-length net using NetMapOperator to compute the features on each chunk and AggregationLayer to aggregate them over the time dimension:

In[12]:=

Out[12]=

Get a set of Audio objects:

In[13]:=

Visualize the features of a set of recordings:

In[14]:=

Out[14]=

Transfer learning

Use the pre-trained model to build a classifier for telling apart recordings of cows and birds. Create a test set and a training set:

In[15]:=

Remove the classification layers from the pre-trained net:

In[16]:=

Out[16]=

Create a classifier net using a simple LinearLayer:

In[17]:=

Out[17]=

Precompute the result of the feature net to avoid redundant evaluations. This is equivalent to freezing all the weights except for those in the new classifier net:

In[18]:=

Train on the dataset (use TargetDevice -> "GPU" for training on a GPU):

In[19]:=

Out[19]=

Perfect accuracy is obtained on the test set:

In[20]:=

Out[20]=

Net information

Inspect the number of parameters of all arrays in the net:

In[21]:=

Out[21]=

Obtain the total number of parameters:

In[22]:=

Out[22]=

Obtain the layer type counts:

In[23]:=

Out[23]=

Display the summary graphic:

In[24]:=

Out[24]=

Export to MXNet

Export the net into a format that can be opened in MXNet:

In[25]:=

Out[25]=

Export also creates a net.params file containing parameters:

In[26]:=

Out[26]=

Get the size of the parameter file:

In[27]:=

Out[27]=