Wolfram Research

Wolfram AudioIdentify V1 Trained on AudioSet Data

Identify sounds in an audio signal

Released in 2019 by Wolfram Research, this net is part of the back end for the AudioIdentify function in Wolfram Language 12.0. It was designed to achieve a good balance between classification accuracy, size and evaluation speed.

Number of layers: 156 | Parameter count: 4,664,911 | Trained size: 20 MB

Training Set Information

Examples

Resource retrieval

Get the pre-trained net:

In[1]:=
NetModel["Wolfram AudioIdentify V1 Trained on AudioSet Data"]
Out[1]=

Basic usage

Identify an Audio object:

In[2]:=
\!\(\*
GraphicsBox[
TagBox[RasterBox[CompressedData["
1:eJztXc+L60h+N8klx/wLOeTa5+ToQxaWHAy7BHLwIbtktOSimNnEoQ+6GMHG
UQIRObgDAoMWR4dGkDaD++DxYDqG2EOUYE/GDp7GdAxtxj3TnpafXlszVj+n
6vstyZJbtuV+/V5nZuvzZLVcKtWPb1V9P/X9Vsnv937+lz8RfiuVSv3yd8jp
Jz/7m/SHH/7s+Ke/S778ifjLv/iF+MGf/1j8qw9+8cGHf/Dz3yaBv08+f0ji
0+sVBwcHBwcHBwcHBwfHDw6DweDfnxVfffXVS9eJg4ODg+MHBUIuqzc+Vvih
xwr+4j+4Yt/9O2/W0fwHCa6vry3LetEKcXBwcHD80ECo6s2bhzf0Q894CvDw
EA158I9teHhDuY+Dg4ODg+P5QKnqgYCy0gNcUS7CAP8TDn4IzozhHsI3CThV
cXBwcHA8LwizPDwHPE5VHBwcHBzPhNn19dUVOa5mC/oVqIrwjOd59OMB5fh/
1hzksTgeC2Ghnk9R68eQqhYzmgv9YDYcHBwcHBxJMVePUohCe7YCqgKSQt4B
7vE8RkuMmbwf/W0nFOI9ROIEz7GHkaraRZbLUbH90lXm4ODg4Ph+YX6STqWO
z25mN3PfqvIZJzhv4kfFT//47z+NveUtI48sPUZVi/kNyaJRODoq8g2BHBwc
HBwHgVJVWl3TB6Wq5XJJPwG80Hd6TaiKHH/6T//pBVE3L9Z3wmtVPTXNqYqD
g4OD40DEUNV3y+8QS3qxjB40BKmKHB9oXQxfkjPEXn/D6+8iVGW9e6q6uigf
FytXy3eayTvC4qJcLJQb2xfzaIRiUVXJp9JOUMXlVa990bZulqv5Ve/ion15
84SFQteq6rpuGORT63pxEbp1Q1VkWVE1szmNi/H/BBOrWtJrk1AJvWlHEgRR
qdpvnbg7sXRNq3Ymb53SJiZNTRCEUn28EX47bGko95LRnbrPnu/7x/eiOZ4X
b98nPdd1nO9X6z9NL8VQ1bffffct+UfO9IoCr/GSfAmoihx//S+f09h4IMMF
z8OX90xVJItUKt2eH/7kopdNpVKZkxl8OcuRZE72JLO8qhTVi6uwnOcVSAXW
/VZtlX45sRKWBpyxR+pse4SK4K8sxkRbDi7OARc3jMYWlQyNqw7mA5p0igrf
G5dEQZBMZ/2gU5UFIW9uGRpOTSEPAETjcZxxTYFbeVnOC6J+m6yqbwlvUocy
KcNDxmjfyJN6dsM17xtQMWlPOm5fEqLIb4oCk8ob/QMKFM5hahl6PZboodhE
tN1woDdtYkFkhTSP2JwcNkXo6lQdBk85fZNKszaiKTu3I6upKbLWHO1NZ0i7
ziaM/hOpJmlzJE7qyc3xBLzvPklg9zUpGJty1drHy96kphvWJEH53GHQrmRk
Vzv7e8IBicfqJcDyZnB+Biqsd/PoqTiqcoGU8AwXwSUiTFXk+IePvghiPn7q
PVPVoEzYIZOUHMJYWCC/1ElvzlhjL1XN24Q51F6EqsqQSqFBRH1zjAla28kn
WgLagnszxSxiogU7ZNZM3StnsUZXpwLwMBG+Y9LBEaIqb6ySDqntbhenKkWf
YnBpuKAk6MrPiaHJRpJuHcCNI6pXpX60DtPRcHy7b3y5Y1PTdA05Wyrpml7r
b3CDO6rSe0/VjSOQbz+2IN7tsD/cKONtRwNSeNQgyYCPK76l1tUpGwJz2fqa
jffXZdIxS5oeSIZelrTW+OlMk6g5EuAtm+MJeN99cuXWIENFM2smtJ9c2/OM
0yXNZCShUrdP0xYVw9CQC2VznySTJx6vlyjmlrp9910MVS1c1124C/rnWzfA
Av/QkA2qIsev/+1/wxEg+rcYsouqllfFTPr4pFwUgFvTuYtrtAcW54VM5vj0
cnAupKn+PRIqqH0vG+UsKuS0cLpm3tlZUYDADNx8IlVlUUzC6XK1CNPB4rpd
yKZRhMflNmWmxaCQzWSwbOlMhl5mKsBxZUzl+GJx00iFqWp+eXKM945y6lnA
XsubdgFKnc6CyZTZnqnfZNuoijTlRhNbKo16MSMW1wmdxtCSuE2V+pOmREEZ
ilodrbyh5Osle9hUcZ4mymYn7HGKoapRXZMkFluSpbyYrw3pfXdcz4ul4XRc
VdEWYfNMe2zpLESQFHMMcZslWS6BpsuX6tUS6LvWvu5+q5P4siqTs9ryWGFK
Yr4E+RM1VZdEsVRD/rSbOth9eSwqUwsQnxSfGCWkAPV1vZyxWUKdI6pGM2Ie
eDDVlKrh4k0sE9IVZUUK68ZxpyrjIM8r9SHqLrelyXm1arUMvKMYHRdKK+eJ
9LCMFPm8SivijjQJAmgZJb86EF9i8WkV8qJaG67sriKKatXXJ96YzLd3SdLp
SmvlBvIUtSkKdzqZDJv5g/R8jGRIZSWpVB+PWgqUVVRqpE6ePWmapTybrxPJ
MAHHN4c71vJ5vWnVNWwRqdaf+unfNg0VxSip5shvv23NsRVxzU07cL5k9TtY
UFHWR/tnBO+9T7pDmoSos6/Ore1L3512SUfD+KVq14XsNb/TQBbkUsLRGg9w
I0gmlNbuQt4yHcWeM2xVsUExcW9n4tuaO04vrbHFoomjqhi494vFPVyQzwZP
/dGv2vf3eJec3chz94tdVDVnhkwqkyvkMsimN1gqdoOEZHJCJqtSDXzdKAAf
5cqVE1DrRw2I3VZBpR9ljzGRt7CqMgJN4ezqem3gsEIeFQil5mhGudNLQlVF
QcgixWSEXC4nZIXTwTywqggdlU8Ie6b9hrguIokeFwvoxMudUVZeXh0z4s0J
rOzbM/WltoOqwk1JJXZ+TI2sxWp5WYGS0IeGJu2GQ3tCJ9aiYbsj8l3v2u6k
znqgrkub88MYqpp2TDKm8qiOVVWRlToMaWdo+hNz6p7KS9oYOnSV9uV8STcY
YSlNj5h4ElIdo7A8TS7iD3kM9H2VOqMOnfEpI9ALWCl8EAuAaqprMHVTYizJ
1MK4qZOyqSqMwrXzc6oHhhMaCmpzbT3hVFNae0odmLrTQVhScTxiplOwWQRC
SzUTkhE7VJCssrTqfvz6xPOmHVVRmM4iJaKC1EdULYwNlRZRCqVMq3/b1RQ2
gc+TBxRZqxOVYlNj2ZeG3dWFPdN7nJOD/rnt0JmCEXIwuuvZSyI8kky4sqR1
VUWSIf0x5qrqpoEaTMZpTHxzOIHfVWZiYC5ot65gMxlVk/qzCUs625tjO+Kb
O+jAIpEuaOC9oniJPunWGX0p1VbI5cqEJmpmVQcxqMR2dkc6ne+IrM+ERuu2
Bg13gL5Bk6wS9pm2sJsapoFzWq0z3ZH4tuaO1UsBtuy+i6EqyjsUeL6Pfl3c
v74P89SP/67j34097ndRFbDD0TEo7dXyLHfkO6+Yws8Uw3sMkL/S51ezxWJ5
3aZ0fHx+zdaYUgL64YCvH1MVOPRCiDEwoTBZtVKku/dPzwn9AR30TrLAFD3K
vLNeLrWmieWgDGbsYiOjTPHkGOioUKErV4Sq5j1a2nThAuOA5UULieFHSFur
mep7HXdkehBVLa7OcscVOmVZXhaF4gBKCs4fpTts4giod+gAN/oOeoGqONe6
7QBpBNy0zQHoQXjE0GAjXSxtztpcx3YcezoZWjDI6GD0k/WALvs29O1Nf8gG
RnR1TOyQhy10gtFhimqhv6EWmMudmXUw4jYSd8P1wgfzzBcKS3jh+L5CDgJQ
YkbXhpuBLkItnW9NbNf1pl0aXmpNmQwFsQ7+sWmrJISWdcABKA3jVpy8OF8W
OI4EM+RvsS3dz8gDTa6Od65fTcC6ro5cG4i1Gm6tqKbaj0eS8SsrSHonatl5
Nu0Et+NRFzhmw4cZaQ62RCibwLjguBZkEt2bQNeVzKlLBOy0YPGVMMKW5tiK
bc2N4QqxVf0y7LXOXqZPOiNd9teqyCDsTle+H1KtD6lLyx6qoVGMHclM5gAM
dwBMEyvlkPZznelk1DKVMIlvSTy+uWP1UoDkVtXr1/d4kM/GBT3uXwc89dN/
/I/X7NZrvBV8CbCXqgJH5cB3YPpUlbEiVbguHKU2QBnH3w6Bqrt3EktVyyvr
onHho9FoDx6tH2FhyoPriwK13IjtE2KNaK5sSwMSDRQ4Is9M2bqgrJNtX1th
qgrsXKxpuTf3wzGFNQftyPQgqooFdn7YBUGMGDIFVWCku7A/QrTYqLnV80mo
ioXbj9LXuxtr696wrgXjSmAjCB+vIlWZI2cc57qPwmbTdZG5IfJaZ7VbLfgq
NBwnKKz5SC2E6EOO6PBN2yEisdC0eRqtJxSWbopA7S3hgpQ7rIbziitbRJ4b
2jKsPXxMqZYk9gXz3uwjGjCm5Gq3Q1s6ymvPRlXSxuqbO+4oUeFEqxBpDt8N
NcRvXV+ro+iiyG904FihbWBbc2O4yTx3SUTxcn2S5D3p++5BarMES2ahzse2
AEXqtRuxVhUxlLxpzffhC6IYlnBs4vuaOx7JqcqheB06HP8MF68d5Kk/++f/
Il9fYxz8B0cQQi+d/VTl545WVSZkVW0wDhgdqXTjer6YU8xmM/raMvrK0nuo
qndeKVd8lMtn7etNSbDC9FYr5pRD+gMzLZU7GywX85mfLZOddQK2z9WmPImh
dX2eKzaWy14GqQqWC7MnvVBNqf3IwsuD4FmsyI5M/Wjqk6mKrVPQeaPVLIlB
J+9oRGWJLVwNwAntep8bbsYwtlFVONwfXNG4Tj8PPRs2nbn+YDycqkDBCnlF
0zVNK0FFqJr1RxPVjHbXYIMInSH5XWohMoOFzVcy041eky6GhLyR/oqMr9Q9
mM/nwblHNFCgG21w8OQ7UzL9pLDpRHS11t4h3RWlqnjPZ2KqWk3qdINMiSou
0o579wRCOUG7iqVO5A4sXwYcsR87qGpDCUMnaME6JUp7g6oeW1Whub1PVaOa
AA1x64KhDhL2tjbHVmxr7kjTJKGqF+qTk6419h+06OClpiVmqjZH3lo2jl9f
Wi+1nmADf3iSwNaqqFcTLXEV9xRF7c24xPc2dzySU9Urx3n1yqFnevEKrx0M
hIPw1Ie//u9XPnv5MdlVcIHXu6mK7UE4Oa2ouC8CbQf09WU29pzjvpGjbPG8
bV2cnWQzhUtqduH+8FS2WDmvFLasVfm7HVJHcBDWeCSKEG9eVrAwlDWIrcp4
6+zCujgvZLPlYBFw1sA9H6eNi7PyyflgvinPuZVGY2rRE9AKrJyfYk1xV/zs
Io3VPj0vH6cDftyVKWU6ei+nlsvqSfsmeL8qMVX5O1EJNUyh7+G4sHGXrKjU
Wk10LCi1YGefhxFVo1o1zO5toAZ3UFW0Tzpd6LZay+qYuJIgPYWqxlQbr3kQ
Rjqdy6H7S5T1ek1nq+3gi8NdUrJea9U0DI5VC5GlamID1Vp1A5dD4JY7pu+W
aWwdXyXXJvVroRNPyJearTrew5GLW7hJYVrdvtU0ZUkDnx86BiOLF+vZMpgE
edW0up2qUQ3vodtCVVKMkFn5E2wGw0T8reZ6l61quRPLMEzToEpflEpm1TRq
1v6kUP1GqSpc2XUgdAKz2WpWtbhp9kZzxFMV4QZc7VGMercPb/UZtNtva46t
2NLch1LVi/RJdwx8TQw4s1YzcMZJ2cSd4FsMRMhW32ppslwNJGx3cINLvWM1
q2Zr31qVIJdMU8NVP7lKaQupSjGbXYstlK0lHJP43uaOR3Kqms9fzeeUpOZA
VORMQ+AKz7/61/+Zz4PbPp3N4XqOdPaK3Z+/SkBVR8yxd5TzFS+yT26w+WbY
onGSW7vEskVrBvFv2rkgieOnb1bPBt7IxUAIORVvrEpmnWumvLbIlu0TIbiT
LfcoJ4blCfSHfr/FZWP9UlS2GHgNrxpsf2YqLRxnjxJkupr1ToOVt+PzwKZL
TFX+jJeqEezwCtNsYR8d7k8LYA/ref9WqTWJJCXFUJW52SfdjuG7JmRNJ6ok
sKqkEFXtWaty1psBWF6wjmOSyerQ32kklAzq0sLBtbrtMoIRRLX0OHG/AEEp
Qy4LomSYPyPmvSqsslMv+RtC6PtNglIdscqaahCXpNO3Pb/wbJFiU0reWF/n
ITZD76egUtpw6KF6fCRktl6jJdsv7b8KJAcycUePfGtyfT9VoTqNUhVUVh1F
H7b7VV+6kq4/rkK0OWBOFVQcDMlgmWYYWqYRCMPD5Glbc2wveFxzR5oGDMad
3tQX6pNk8lfTgiFJbjR96rnt10JdScI1LIDXNZV1w1a3W81oJmMCedlo+ouo
7ijIUjHo3r7QZCAm8X3NHY/k2yrm87mNH/sVOehfcgFnG28FZxuOuR9uA6nR
Yx5c7F+rohp+OZvNk/7CxHIxp7sNNqIvSV6bYc+JJfXCzWPeqqY+ulnSrKHo
j6IuF1uqvzVTeGQWTSlms/pT4Lm2bbux3qMdt5Km/U7frPd8V9tmOLH9Dyq2
69AF+6S5OvG5ktrSZA7JGByGbyFgdwJ8t2dDxQsDe9HzpASij1Z2a3Nsx0HN
fSDedZ/0nPgMPGeLHGKFdhCoZ3HL8zGJH97cVjEpVdl3NnCQfUcu7hgf0TAk
Jv/i7o6d6a0724/gg4bTCLs3q6f5b60/G2JeAeb4jQF7IVTA/cMcHN9DHPoK
8B3wDj3HHjtuxUXe+QrwdaNSObce7XDgeAqWl226y7HRCK1fcfymwO3WTcOs
WaP389NWHBzPj+XsstFoxO/QjqOqb765o8cdnL+x6QW9tu8g/O4udKYHpSQW
cgcgj/hPkVvv+YeVODg4ODh+cIihqmdHkDinKg4ODg6OwxFDVTfPCk5VHBwc
HBxvB/bKavD9nVIV/68VOTg4ODgOB/z0gVDuDQbXsF16N1V98sknhULh448/
DkIKgL1UNb++JFmc5o6SvfjDwcHBwcERYP07rgX47wCTUBXBRx99FPBUEqqy
iv5Gak5VHBwcHBwHYoE/qOe/P7vXAdhsNgtR7I6PVLXcyIaDg4ODg+OpSLJW
9emnnybkqZvoWhUHBwcHB8fbI+G2is8///z09DRJTE5VHBwcHBzPi3e6A5CD
g4ODg+Pt8bxU9eWXX7bb/Cf+ODg4ODieE4Sqvvjii2fkqa+//vql68TBwcHB
8UPDZ5999lw/qcR5ioODg4Pje43/Ay4T47g=
"], {{0, 37}, {569, 0}}, {0, 255},
ColorFunction->RGBColor],
BoxForm`ImageTag["Byte", ColorSpace -> "RGB", Interleaving -> True],
Selectable->False],
DefaultBaseStyle->"ImageGraphics",
ImageSize->Automatic,
ImageSizeRaw->{569, 37},
PlotRange->{{0, 569}, {0, 37}}]\)
Out[2]=

The prediction is an Entity object, which can be queried:

In[3]:=
pred["Description"]
Out[3]=

Get a list of available properties of the predicted Entity:

In[4]:=
pred["Properties"]
Out[4]=

Obtain the probabilities of the ten most likely entities predicted by the net:

In[5]:=
\!\(\*
GraphicsBox[
TagBox[RasterBox[CompressedData["
1:eJztnU2L61h6x02yCVnlK+QL1DpbEzIwZFEwQyCLWmSGtIZslKInUaiFNoVg
4iiBiCzsgMCgoaJFIUiJxrVw12AcQ+wmSrAnbQd3YSoGm3HN2HOlq3utbuu2
55znHMmSLb9U3Zq+fW89v+vy1cvR0XnT8z/nOUdVf/jDv/6e8Du5XO7Hv0e+
vveDv8t//PEPzr7/B2Tnz8Qf/9WPxI/+8rvi33z0o48+/qMf/i45+Mfk5/dJ
eLq9RBAEQRAEQRDkvaXX6/3nk/LLX/7yXecJQRAEeVuIPV9+HbFkP/SzhP/Z
P9ji+9GZr1fBogsJo9HIcZx3miEEQRDkCSDq8PXXb76mP/SbfcW8eZM+8ib6
bOPN11RuEARBkPccqg5vCFQI3sAWNf/sQPSTPPwm/uai8iZ5koDqgCAI8gFA
jPmbpyBEdUAQBHmfmY1Gd3fkczeb011QB2LawzCkPyFY+ei/ldkPeZiQH+FH
w0gVVpehOiAIgryHeNpRjnHenC1BHUAXmKkHcx+GXAm4GITf+ftW4kj4JhUm
vo5fjOqAIAjyHuKV8rnc2dX97N6Lxg6RkY+/1/lO4bM//cfPMk+Fi9QlixDV
AUEQ5H2EqkNeWy06peqwWCzoT0yY2KfbRB3I58//5b/DOOj6xuoMqgOCIMh7
SIY6fLX4irGgG4v0hx5h6kA+H+ltdnxBviH0ao9tf/UtV4d5vVw4L9/MdwYo
FDSN/Fw0F/sjXNx1mvWmc79Yeneder15e7897q0Ejm0YhmmSn0o7zArQrpqa
qiiqplu1SVaIbwljxy4alXEiheGkJQuCqNruW0cejB1D1+3W+K1jWmdc0wVB
KFaHa8en/YbOyr1otifBk9/3m+e9qI6n5e3bZBgEvv9+1f7j7FKGOnz51Vdf
kn/km25R2DbbJDuxOpDP3/7b5zQ0+zBRia+HnV3qMO+c5HK549IMdq5Oc7l8
yduXy4uCVr9LZs27gFhg2mTZ1OhOydkTTTL7uSNttj3AhRBNzGQEW/Tq10D9
nivH/OKYhtV6Xo9GnTsqOMtwWBQFQbb81YW+rQiCZG1pjX5FJRcAorkZZlhR
4ZSkKJIgGtPDsvqWhOMqpEntP+Sx6JoSyWc7mfOuCRmT98QTdGUhjbReFCwq
yew+IEHJO0wc06hmaiskmxRtO3kwnNRYQhSVVI9YGz9MldsGtUDxVX7XoqVZ
GdCY/enAqemqotcGe+Pp06azjtl9pHU/tDoOjurR1fEIvuk2SXC7uhw/m4rt
7JPCcFwxTGd8QPqCflyv5Mm2W/tbwgMiz7RLwOK+d30FJqxzv3FVljoEoAPs
GzbiTUZSHcjnnz75Ig65edVOdXAgyblSx+OGeq86eE1irLVOSh3KEMv5Dcnd
/RmL0Nlu7zcLbe9N2S0ygsVz+vlmdK5TPmE5ursUQPpI2foWbY8JdQiHGmkD
+u63yH1bTl/FCehxQT2g9TwlfYs3XsN5gBwNqCmTu+k8TAb94XRfkw6Glq4b
OpNJuWjoRqW7Zo6DgU3PPdYcDaB8u5kJCaf9bn8tjdOWDnZ4o0IOg12uRuOR
tkEFCMTCNVYCuD8v45ZV1I24ZOhmUW8MH2/cD6qOA3jL6ngE33SbXAYVuKGq
WxUL6k+p7LnGb5NqMg9Rr6BL4xZV09SZ/CjWvpI8PPJsu0TxHI13fgvNjYsy
1GEeBME8mNP/vgxi5uw/emRNHcjnp//x/8kAEPxLdmS3OpywlAmXi+U8aYHn
o+b5SZ6l+qzcpGIw752fHB/nqTk+yh8f083jC5CVMovlrD6/v8kl1cG7LZ2x
c0en2lUsGIv75vkxjSd/AgOD4+03jUppmzqQ0lsrVUejQeszMq4oUbGmKQlq
GnVUTIhNMFXNHizDvhyZArdf01hvRFSsVtKVkaEOg6ouyzy0rMiSKFX69Hww
rEpisT8Z2hrrcfPelDt0DH5EkFVrCGFrRUUpgnGRilW7CCamsa+FTQ0SXtEU
8q01Qp6YoigV4f7EMlRlUSxWmGS5NQNGNxJLKn8SITxJPul6kwRUV/nyh1aR
PeaiZtZSneAQOlSynUze2LEgXlFR5aQ5GrZshT1XklrtM3MRNHRF0mynYbIz
qtkKILWKREqPpZEiSRrNSDDQZThA0yhH2YHwMg9PsyCJWqW/dNuqKGp29AiH
Q9Kr3FWSflte2RMoT1GfsMKdjMf9mvQg05pRMiSzslysDgcNFdIqqhWSp9Ad
16yixHulpGR4AWdXRzDUJcmoOVWd1Yhc6U6i+Kc1U2PFKGvWIKq/bdWxlazq
pg1YKjrdFkuoqBiD/SL8jbfJoE+jEA2+60/dqPSDSZs0NBa+aLcDuL0eNRq4
BdmU2dOaDQyWZQtS67bh3gp9ikO/37BZhbLIw52Rb6vuLLu0wtHy8WgiQZY6
ZBC8ns9fwwb5WZOGP/lJ8/VrdpZ8B6nrXs/3jh2OBdr1v7obrbrxHhtTHJ2X
yoVTaq5PL2+JOhQE4YRZ9WPh9PRUOBEue148diAKUC4RWcxHeR8VoGN/fFY4
Z96h0yvq/lncsfFFXjgV2IXH228aldIOdUiWHmF0fUaHEvPl4vYCUkIv6lu0
5vvumHYfRdMNBmTfaLvBuMor3TDk9V5QhjpMWhZpxhKzgJqmKmoVniK/b0Xd
T+r3kGR9CG3Ips1HKhom1wi1FpKBjMzUhauGRKNLDbQ3YU6VYmvQov0adQCP
IssUu5AlgFmGtsmf8CIXJv4kDmsGSZumQcNfedUmRjw8YN1hrbYaI7AOlbxy
wfnQQaXtvqixR4DddAI9c4EoQcWCaMQWLUieWZr1KHx1HIaTlqaq3EyQFNGC
NAb0SRyaGk2inIiZZn/a1lXeTZXIBaqiV8lT7NIhYVQabtsQ9nRiWc8THvlp
i4qzmfBcBasOw0FslEwys6R2NVVWIP4hu6tmWCYzGgrrOWRXhx879BReDNy3
GVRVVk2mbVFHKREmf3t1bCe7uuMGLJLSBaO3tyjeRZsMqlwxVLuR8OXxQhN1
yzagGDQyQgwGBu1iiLzNJJ7WbRWabABdk0ZpE4M/abBmalom60bqrcmOyLdV
d6ZdiukcrA7U1FPY9+v07vz1q9dJafjuP7Sis5mf13vV4US7KNBVtZfXWp5Z
4E7pBIxzh+rLrHOaW1nmRa8M46OUZ4lk4bhQOgMFOL+gsxBEHbwOlcj8eZ2F
gfHFMSkSdvyIKcVypkXurB03fZA6zO+uTs8uqDAvbgtCoQcpBa+C2u7XWKOr
tugzZXZ95l6wWY9i2gI7HcvBNs9SCMdT3Wn+cInF9b5J4Lu+707GfQfaNW3/
UbQhKFTXhea0PtBeY0BnOsQWudhh3hX6ZLAnsbv2JHL3KR+8QCNfizxI5otd
KHEnG0zHJMNHNjA+wErMbLtwMn78mWGUGmM3CMJJmx4vNia8DAWxCo6XSaMo
JFz04FmS+1mzB2GWkwQ8EoKVGMi7jhHdKATjqQ13zkWMYQxpDwIXtMxO1lba
OOxno2SizAqy0UqPX0KXNoLpcNAGs77mHEtVB5/uUSwQOfCICgoJHo6h6crW
JCAF7DdgIo0Y4S3VsZVt1c2Oq2REFqVh7xjk3bRJf2Ao0bwDeQjbk2Xk4NKq
feorcfta4ilmDck6zLOUbAAsTpYpn9Rf4E/Gg4alJnVzS+TZ1Z1pl2IOHzu8
evWafcjP2gb9vH4VS8P3//m/XvFTr9ipeCdm/9ih3BvVz4nxPSU9/IShThHN
CDPbDvMUqSwcl506NfQnzZGTVId4ANUDeSh3vOg4i2Fl9nfc9EHqkAlrbzCJ
TLrqpKOlwsMVwPSy6PCGOjWkQ9SBH3c34jfaa1OTYb+qx01Z4I2WXW4zdbAG
/jDLDZvG5Z1SkY9vJb213P0kRlYrGSZOrLXxJCYstpIym+s95FSJJTqHk3Q+
IbF0TpkZTJlNLgR9O3mvrLSlynPNQCUf2IgJNUykF83dAvtsOwwZFLvdojWd
lpInUwd5bSYlGLbUdOGks5Cqjsi/0Wd77ciQsqJLI6014MxCW2NbdbPjFncJ
HVIU765NknuPu5HfifbM4+mPROPjKyhS+dpN5tiBDAfCSSVyDguimCzhzMj3
VXc2h6uDT3mV+PjRN2y88pk0/MW//g/ZfcXCsH/wiY/QTX+/OuS1znLJvT3M
yQMustzpVW8x92YeYTabccPsOSXo4d+tZ4EMJ0bXp4WbxaJzzNQBZltOSh0I
s7g6PWJzx/x4uRdfy8z+jptGwbRHqwP3OdPekVMrinG7aunESogN5tll3bbV
yhw2l21uU4fk8ag9p8P6XQkaEyyTCaL2/3B1AJsmSKpu6LpehIxQyxY1YGqM
3LbJ2y0bZUu7nsRUPw2WiyjcHIU16thOuLki73pkR0PotUrgNSIPfWyOXPAc
SK0J6WRRXNrdWq4MZsJcpNUh26V2sDosx1W6vqBIbQWpx72rmCCdYNDEYit1
BqaiYrO8nx3qsGb3oBE0YM6JlfaaOmyOHRI92EgdBhUBKmIawHAUSjjcWh1b
2Vbdqao5RB3eUZsct51hdKFDH146gGI31WqDcFU2fpRfmi+tesAq36Qu83kH
6i5j402NLclIj6qyIt9b3dkcrg4vff/lS59+042XbNtnB+FDpOHjn/7vy0gw
opB8K95g2yt18HqacHJyWrqNjSxXB3r32wshFzlzyCCIS8VV3alfn5+clOM5
lNkNTEsLlzf1q3LpuuetZ8Fz8mzIMO8IbCr+4vpSE1h8NJZZHSae89rldfks
H0vSrptScYGJC61c1krN+/i9h4PVIVquRqzxBKqbNUWXLaUT1UqjxkasaiVe
ixSygJpp26bVnsaWZ4c6pJuB34aWojeclsW8wvJj1GFIDeBKeuDhoj0W5lcR
FaNaMfhkJTh52LoOxag0Kjo7nPkkpmb6SE+/0qiazLUNp4IhfedD59OgGtm2
qMOEeYcEqVhrVNk59rCwdZ4kMY1216lZiqyDM4l5nFKO6FWfEDq+kmY57ZZt
2slVP1vUQc4oZJ7+A5avsEii9ahGm89QBGPHNC3LpHZWlIuWbZkVZ39UzOKl
1SGZ2dVBaARWrVGz9azO5Fp1ZKsDMcfMc6+a1XYX3rYxabPfVh1b2VLdD1WH
d9ImgyFIJBmmWJWKyTp51IAHY7bUmRSy03UauqLYcQm7LbY+oNpyarbV2Dfv
IChFy9LZDI5iU6Vg6qBatbbDJz1WJZwR+d7qzuZwdfC8l55HdcEDbSDf9Ahs
se+f/Pv/eV58OlIQD7Y9piAv+XnvZawOXrPAxgfn9cjqwpolvrZq3hMSy4fu
HbY+l3l4jsvNUZTARbMkxGdOyh3q9klmARSHOZTmtzerlxVOCrE76u6GL+LK
5YWzk6MDbrqcdS7z0Zmz63jkcrA6RP06+uSyNqZyY5J0/rAVNTFuvypFp4qN
cSoqOUMdrPVmELTMaMyr6AZ5euOxg5xQhz3zDv5qLpXfC3zyFumS9aO1EULR
pL4S1p6X0za36YKoFTcjjxIQpzIxFibPNR8oZ7zvwLLsV4vRfDp970BQ7QHP
rKXFYUk8XTeMEs8dzuulFA6N1T3EWmLdOLMDa54iZpE2Cpn73vXDFlVGS/SV
uEyCwYbTRqnuVwdmwdLqAJnVBumL3a4dla5sGJtZSFcHdGPijMNwKXa59xMu
d4GIKvRXtlXH9oRnVXeqamBYtNNN947aJOlvVfT4kSQnapG1n3YriaYks/kI
IGxb6qpi7e1jQzYYZBFIilmLJsSCQXxL1aSrkRL6mxH5vurO5vBZac/zXPbj
viQf+j/ZgG+XnYq/Xfh40XEXdIR+vHgjVofFffMUbLVw0dubWnYFde94Ge/0
UecPOXHAq8uEuTefbwZdzEkMWRFsvSlcMkvHlLGi9TGEARmLBpluiR2nDo37
t/peZxj5cNaPk0Hlg5Id+HS+89C7+tl3Jbml0TzkxuCJeosCDsYgMXvmo98x
rBU9TUxQ9OnMbq2O7Tyouh/Ib7tNhn72DUJ/SzlkFtqDoC6rLddnRP7w6nYK
h6qD+8IFs+++IBsvuATQY0wLoo0XL/g3PfXCjQJE0OM0QHLegS+m6mz4799X
Mt6GQ54N/N0ogS0yRJD3kIe+DfcCTD39zvzsOJUVOFaH0Q14lvKFu8P6/O8D
i9tm/aZev7lJzEUgz4WgXbVMq+IMvpnfZYIgT89idntzc0NNWG/z90tkqMOv
f/2Cfl7A969dukG33Rdw/MWLxDf9UBXgR14A5JLoKnJqpQ710mnh+sDfcIEg
CIK8UzLU4cl5h9lDEARBHkWGOtw/KagOCIIg7yH8VbJ4H9UBQRAE4W8BC+VO
rzeCBZ671eFnP/vZ+fn5p59+Gh85B1AdEARBPixAHdhLavDncw5RB8Inn3wS
SwOqA4IgyIfHfO4x2Hteez1LtVrtPM3u8KgOCIIgHwCHzDt89tlnB0oDqgOC
IMiHwYGz0p9//vnl5eUhIVEdEARBPgBwzRKCIAiyydOqwy9+8Ytm861/SR2C
IAjyriHq8MUXXzyhNPzqV79613lCEARBnoCf//znT/U7NFAaEARBEARBEARB
EARBEARBEARBEARBEARBEARBkA+M+Wx0dzeiP7P5u04LgiAI8m2hWcht/8vp
CIIgyDNl7t3fz+5vzo+OCs7+0AiCIMhzoqPlUR0QBEGQNRxUBwR5vizuOs16
07lfLL27Tr3evL3/1s1CBmNHL5rtaXhgcMc2dMMeBtmnxzVdEIRidRjHbhuG
YZqmYdit8UZwt6KKgiA3xgfdPZy0ZEEQVds9LK0bfLuqA9UBQZ4Vo0796qo+
W7C9+cUxnXzUel4P/uIpswZ9WxE2MLsPtnl9W40vFyVZt50tNnsXftcklxvt
A+/uWxIJLrX97NNdk54WjTbfD/o0iSJNoWR210MHfVYQRjLv4bhimM44Iyss
qURN+o/IJyW7Oh7AYj66dcqFsxOh0EsIy2LWKQjHR0dH+eOzm1tvdcK7vbm6
dm5nmZGhOiDI88EpncBSlHwzshCdMj1S6nh3lwLZOC5RazBuWUXdMHQVrKZM
N4t6Y1t3fDt9ixpXxbBsy1AgLjXutB+M37fIhdahBte3ZXrP/rbOfjjtd/vT
tciCvpypDjT4sDtIjxz8NsmKuSU9k0F/uB77A8isjsPxnGixUe44ruLl4vYU
Dp0IAvzN83z9PgrfKfG/gX4z2owN1QFBng0z7Sh3dH6zSBxytGNqLmbLRY8a
Cs1JdCND6DnLdtLYuf2aJoOhFxWrxUy9Xy3KkmpULA16zpLR4BLQt2TakWa2
1W9LkQUOhlVJLPYnQ1uT4RKV21p/aBVZb13UzBrrrzN1kDWuVYpeZcdDd1yz
ihIfmajVPg8O6iAomsoHBCwxwUCXJZmgqKoiFyuDVMEE3TV1ICmURRpeoeh9
NhIhkSiyLIlsKCTTTbkC5wbVIgtOYleK1cTAZVozNZGNKDRrEJ0IJm1d4WmX
VcMZTPdXxyHQFxWcAtWAYydSh9H1Gdh/kITbSyoT5U7imln5OHvlKqoDgjwb
vFI+l9dSzzuYjnxzTjqYF2COEm6HoAvqYMVelWBcZba4aBjMrBvOlDpz2I4g
FXVuCWvQ3U6qQzCsiVQ5qEuHGXwGsaeSrA9pmInBxirFaNSi1cJEYJlYXjCo
stUnoYcVGJhohmUyjVBAYrg6rCcmHJqaqmpcjeS1YcKmOowbmkKCa3BHuR2p
g0G1hcYqyfSsqqhVsPjDmqHQ6EHapLjEgqrKcmTaVlEEqYWYWCIlw65WLJ3c
QdSd/dVxKPPycUodnBLZP2mObq9KF53RrZanPYTkBdvWJqE6IMizYVY4Wu8l
zu+uTs8uaA91cVtIO6tjdYh7wm2D2kqb9aSnLXDwk7Pc1rWg/+s6RmzAuWeJ
WE1VTk5ecINPhg+JTjY7KHE76cPUh9z1+XG9NYHDbXllfkPX9X13Ohy0wfKS
yP3Y8DruemIY4cA+RB1iBjQdNBlrMWzxdAX07hIvsXBcA0WwJkEQhH6DppIJ
DZ8csRrdaZDyWu2qjkPx1tShQ9VBuL6mTqd84epCyOWOtOSYhKhA7ry+GRGq
A4I8C+a983xu5WE4hHV1CCowg+vw/SnVipU6cBPKHUFgaZk6iJIkyaSrrde6
E3YlC7M20cwOxnPfYJapEvF5B26gXZPflAxGWmyEEQPXbk1M8i6HqwMb/nQ3
VMzqZ057wzAqUoegbwvr8OnyYOxoUnxMawweu8opg3V14GOHu075XKvf3dKz
p1dJ72KPCkbu6PRybaCC6oAgz4LFXemUWo1C/dHqsGyBF6Qx4Wehk266sUGG
3rTbpt11xWZjBzk+niQSgpSBZQt+FN7PD2vULURtaRQ4WLspdL/lxtCPr02p
w0ZikrfOVIfkECMmQx26NAaturn8dcnvHqvDoAJ+JXsaBD4Z5pCBjuvyoQL8
F7qTdg0WUQnqYO+aWa+nCScnp6XbPd6mBVOHTiQAt5d0Urrcg32vfkQnu5Pz
DsseTIWfFK7XRiqoDgjybPC0Dc/SLnwwxQl1cNmKTVGtNGpsEZJKp3e5q19S
zVoVbDZVkNW8w+bi0rVhAoctLhVEo9KomsxfTz1IkRtKsWtVdlMFbsrUwao1
araeHDtYm4lJvCuxQx1YviyrkUzWpjos3RZMS6vVllOzrcYgeY6rQxTDlJt+
s9rutqumpppgbF2HDoA0y+kOBt0azFVow33q4EW//Oi8vm2qet68KJVK2skR
HQqcaWT76n5Br6RycSSULy/OYc3a5W1KB8CzhLPSCPKcyZiV3gUz1wl1IPSr
euzOUc0W9NDjiWCGaESvlYFnKW1agchZtO5OSTqLRMXoJ7xDajRzIes17mPq
2lFY2TC0KEIfJqslWVpPTHSLCuRpzYkUtq341QwlOdjJykIycGpgwouCTz3z
rPK1vAARBCoCwTheyMTSX+lOt1cDZ3HfPKVmPydc9LYE8eAliSR86fLMuYjP
FK5v1y7DWWkEefbQFa25s6vZYn/QXYSB67qJ2dToFQNiVEPCWyZyGfh0Ejfj
tr7r+unjLCXbUklieUhiIH7/wCvCgIbOSubhgUP/IXdcLqOFTJ1HLGSiL8rN
ZjNvc5p7flfI44pWBHnudMoC+8XMzUcZmC3seT0ZeRJGN+BZyhfu3lLcE3gd
jQ0oSk7GbBSqA4I8K2ajnuP0nlIclstJt1WpsKkG5LfFqF46LVw/8O24fcxH
TtO52/InflAdEARBkE1QHRAEQZBNUB0QBEGQTfCv/yAIgiBJvNFtp9e7PD16
wPpnBEEQ5EPHKRzx9yVQHRAEQZCIxXzuMeZPt3wWQRAEQRAE+TbxG+bTLX0=
"], {{0, 67}, {522, 0}}, {0, 255},
ColorFunction->RGBColor],
BoxForm`ImageTag["Byte", ColorSpace -> "RGB", Interleaving -> True],
Selectable->False],
DefaultBaseStyle->"ImageGraphics",
ImageSize->Automatic,
ImageSizeRaw->{522, 67},
PlotRange->{{0, 522}, {0, 67}}]\)
Out[5]=

The probabilities do not sum to 1 since the net was trained as a collection of independent binary classifiers, one per each class. This reflects the possibility of having multiple sound classes in a single recording.

The network was trained on the AudioSet dataset, where each audio signal is annotated with the sound classes/sources that are present in the recording. The labels are organized in an ontology of about 632 classes that span a very wide domain of sound types or sources, from musical instruments and music types to animal, mechanical and human sounds. Obtain the list of names of all available classes:

In[6]:=
EntityValue[
 NetExtract[
   NetModel["Wolfram AudioIdentify V1 Trained on AudioSet Data"], 
   "Output"][["Labels"]], "Name"]
Out[6]=

NetModel parameters

Inspect the available parameters:

In[7]:=
NetModel["Wolfram AudioIdentify V1 Trained on AudioSet Data", \
"ParametersInformation"]
Out[7]=

Pick a non-default model by specifying the parameters:

In[8]:=
NetModel[{"Wolfram AudioIdentify V1 Trained on AudioSet Data", 
  "Size" -> "Small"}]
Out[8]=

Pick a non-default untrained net:

In[9]:=
NetModel[{"Wolfram AudioIdentify V1 Trained on AudioSet Data", 
  "Size" -> "Large"}, "UninitializedEvaluationNet"]
Out[9]=

Feature extraction

The core of the network takes a fixed-size chunk of the mel-spectrogram of the input signal and is mapped over overlapping chunks using NetMapOperator. Extract the core net:

In[10]:=
coreNet = 
 NetExtract[
  NetModel["Wolfram AudioIdentify V1 Trained on AudioSet Data"], {1, 
   "Net"}]
Out[10]=

Chop off the last few layers in charge of the classification:

In[11]:=
singleFrameFeatureExtractor = NetDrop[coreNet, -3]
Out[11]=

This net takes a single chunk of the input signal and outputs a tensor of semantically meaningful features. Reconstruct the whole variable-length net using NetMapOperator to compute the features on each chunk and AggregationLayer to aggregate them over the time dimension:

In[12]:=
extractor = 
 NetChain[{NetMapOperator[singleFrameFeatureExtractor], 
   AggregationLayer[Max, 1], FlattenLayer[]}, 
  "Input" -> 
   NetModel["Wolfram AudioIdentify V1 Trained on AudioSet Data"][[
    "Input"]]]
Out[12]=

Get a set of Audio objects:

In[13]:=
audios = Flatten[
   Thread[WebAudioSearch[#, "Samples", #Duration < 5 &, 
        MaxItems -> 20] -> #] & /@ {"cow", "bird", "cat"}];

Visualize the features of a set of recordings:

In[14]:=
FeatureSpacePlot[audios, FeatureExtractor -> extractor]
Out[14]=

Transfer learning

Use the pre-trained model to build a classifier for telling apart recordings of cows and birds. Create a test set and a training set:

In[15]:=
SeedRandom[42]; {trainSet, testSet} = 
 TakeDrop[RandomSample[
   Select[audios, MatchQ[#[[2]], "cow" | "bird"] &]], 30];

Remove the classification layers from the pre-trained net:

In[16]:=
featuresNet = 
 NetChain[{NetMapOperator[
    NetDrop[NetExtract[
      NetModel[
       "Wolfram AudioIdentify V1 Trained on AudioSet Data"], {1, 
       "Net"}], -3]], AggregationLayer[Max, 1], FlattenLayer[]}, 
  "Input" -> 
   NetModel["Wolfram AudioIdentify V1 Trained on AudioSet Data"][[
    "Input"]]]
Out[16]=

Create a classifier net using a simple LinearLayer:

In[17]:=
classifier = 
 NetChain[{LinearLayer[2], SoftmaxLayer[]}, 
  "Output" -> NetDecoder[{"Class", {"bird", "cow"}}]]
Out[17]=

Precompute the result of the feature net to avoid redundant evaluations. This is equivalent to freezing all the weights except for those in the new classifier net:

In[18]:=
trainSet[[All, 1]] = featuresNet[trainSet[[All, 1]]];

Train on the dataset (use TargetDevice -> "GPU" for training on a GPU):

In[19]:=
trainedNet = NetTrain[classifier, trainSet]
Out[19]=

Perfect accuracy is obtained on the test set:

In[20]:=
ClassifierMeasurements[
 NetJoin[featuresNet, trainedNet], testSet, "Report"]
Out[20]=

Net information

Inspect the number of parameters of all arrays in the net:

In[21]:=
NetInformation[
 NetModel["Wolfram AudioIdentify V1 Trained on AudioSet Data"], \
"ArraysElementCounts"]
Out[21]=

Obtain the total number of parameters:

In[22]:=
NetInformation[
 NetModel["Wolfram AudioIdentify V1 Trained on AudioSet Data"], \
"ArraysTotalElementCount"]
Out[22]=

Obtain the layer type counts:

In[23]:=
NetInformation[
 NetModel["Wolfram AudioIdentify V1 Trained on AudioSet Data"], \
"LayerTypeCounts"]
Out[23]=

Display the summary graphic:

In[24]:=
NetInformation[
 NetModel["Wolfram AudioIdentify V1 Trained on AudioSet Data"], \
"SummaryGraphic"]
Out[24]=

Export to MXNet

Export the net into a format that can be opened in MXNet:

In[25]:=
jsonPath = 
 Export[FileNameJoin[{$TemporaryDirectory, "net.json"}], 
  NetModel["Wolfram AudioIdentify V1 Trained on AudioSet Data"], 
  "MXNet"]
Out[25]=

Export also creates a net.params file containing parameters:

In[26]:=
paramPath = FileNameJoin[{DirectoryName[jsonPath], "net.params"}]
Out[26]=

Get the size of the parameter file:

In[27]:=
FileByteCount[paramPath]
Out[27]=

Requirements

Wolfram Language 12.0 (April 2019) or above

Resource History

Reference