ShuffleNet-3D V1 Trained on Video Datasets

Identify the main action in a video

Released in 2019, this family of nets consists of three-dimensional (3D) versions of the original ShuffleNet V1 architecture for video classification. The ShuffleNet V1 architecture utilizes pointwise group convolutions and channel shuffling, two new operations that greatly reduce computational cost while maintaining accuracy. With the availability of large-scale video datasets such as Jester and Kinetics-600, these models achieve much better accuracies compared to their two-dimensional counterparts for video classification task.

Number of models: 8

Training Set Information

Performance

Examples

Resource retrieval

Get the pre-trained net:

In[1]:=
NetModel["ShuffleNet-3D V1 Trained on Video Datasets"]
Out[1]=

NetModel parameters

This model consists of a family of individual nets, each identified by a specific parameter combination. Inspect the available parameters:

In[2]:=
NetModel["ShuffleNet-3D V1 Trained on Video Datasets", "ParametersInformation"]
Out[2]=

Pick a non-default net by specifying the parameters:

In[3]:=
NetModel[{"ShuffleNet-3D V1 Trained on Video Datasets", "Width" -> 1.0, "Dataset" -> "Kinetics"}]
Out[3]=

Pick a non-default uninitialized net:

In[4]:=
NetModel[{"ShuffleNet-3D V1 Trained on Video Datasets", "Width" -> 2.0, "Dataset" -> "Jester"}, "UninitializedEvaluationNet"]
Out[4]=

Basic usage

Identify the main action in a video:

In[5]:=
bbq = ResourceData["Sample Video: Barbecuing"];
In[6]:=
NetModel["ShuffleNet-3D V1 Trained on Video Datasets"][bbq]
Out[6]=

Obtain the probabilities of the 10 most likely entities predicted by the net:

In[7]:=
NetModel["ShuffleNet-3D V1 Trained on Video Datasets"][bbq, {"TopProbabilities", 10}]
Out[7]=

Obtain the list of names of all available classes:

In[8]:=
NetExtract[NetModel["ShuffleNet-3D V1 Trained on Video Datasets"], "Output"][["Labels"]]
Out[8]=

NetModel architecture

ShuffleNet-3D V1 features an efficient and elegant implementation of a channel shuffle operation: a feature map with gn channels is reshaped to expand the channel dimension to two dimensions of sizes (g, n), then transposed and further flattened back as the input of the next layer:

In[9]:=
NetExtract[
 NetModel["ShuffleNet-3D V1 Trained on Video Datasets"], {"block1a", "channel_shuffle"}]
Out[9]=

In[10]:=
\!\(\*
GraphicsBox[
TagBox[RasterBox[CompressedData["
1:eJztfQt0VcXVf8oz1FSsS0SL2kLbr7VUDQWLZVlcSqt1Ue0Cl0pl1frASHgE
RKgPQAiiCBElYOQptJJaELA8wqMSHhKKREUDBnkFSCoQHkahBQQsnO/3v/vv
Xps5j3tmzr3JhW9+a8G6mTNnzp6ZPXv2b54tH+rXNateWlra4+n4r+uDT9w8
YMCDT911Ef64O+fx7EdzHulxe87ARx59ZMAND9VHYCv8K2+Qlvb/fjsWFhYW
FhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYW
FhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhbnJo4e
PXr8+PGQkf/73/8eOXIE/ydVJAuLVMCePXuKi4vxf10LYhEftrIsLCwsLCws
tDBmzJh7Y5g4caJZCh988MG9Xjh9+nRCJPz3v/9d8TX+85//UOCuXbsoZO/e
vVESX7Zs2fMx/PWvf02EsHGwefPmvLy8AQMGZMXQq1cvfHr58uWekUFOZ82a
9cwzz2RnZ1P8QYMGTZs2rbq6Gk83bdr0vBeOHTtWCxmxsEgGVq1alfU13nnn
nboW5/9j9+7d3L72799f1+KkCowr6/XXX6fCnDt3bvLEszjnkNS+vs6hlbuj
R4+yzamqqoqb+Kuvvup2BuDeJCUn8VBQUEACrF+/vk4EUJBSegUBFi5cuCCG
L774Qj7CnxQO01pr8sD/XBCIzz//vNaE+T+ClNKB9957j7747rvvynAQENaB
ZAwv33HHHWkxPPjgg2YpQLA0L3z11VcJkXDmzJmcJlgbBTZt2pRCGjVq5PnW
mTNnPvroo7feegucevz48fgBjnzkyBElGign+U5PP/10QqQNwPz587O8kJ+f
745cU1Pzpz/9yTM+bCYirFu3zvMpm9YogK1esWJFcQwHDx6Uj1CGFF5SUhL9
Q8nG6dOn0bN88sknIOwnT570jIMS48wCK1eu/Pjjjw8dOpSQwRPUI6fsVj8n
1gNyBD8JEwh8AhksDoG1a9cmWxg3RowYwZo8atSo2hfAE9Aflupf//pXXYuT
GETXTOPKGjNmDL0Fh9k8A8lEUm2ChR/M+vpzBVq5QxfMjUvxCT3x1FNPuZ2B
N954Iyk5iYcnn3ySBPjb3/5WJwIoSCm9+uc//8nCoNY85fzpT38a/UNTpkxB
ZgcOHBjsV3zrW9/ydOAZcDVrTZgUhIHkS5YsUUaBdu/eLSOklA7cdNNN9MVL
L71UdnB///vfWchkzM1F555Lly415p5x6wiYPXs2pzlo0CAKbNasGYU0adLE
nSz8w+eee85tivv27bt8+XJZvLXGPT/99NNHH32UJRk2bFheXh71F/Bt3PEn
TZrEkfv16zd69GjkqHfv3o899hjJDyqdPO65adMmThDaKx/BDaNwUOPoH5o7
dy78z4kTJ4ZfgRwS7733HvxhlJgsnIKCAmLuEpJZSODdOXPmnDhxIooYspo8
54ZeeeUVegoToXi2KBOYfdR7rxgWLlwYRRJCeXm5Z2bdgP8Q/XO6eO2111gA
45UYCcd5yT2jaCbBuLJSn3sm1SZIJM8AJhuVlZWvxuC3bscABn39vHnzFBeC
1gXVCeDgwUqPHz8+NzfX/VQrd7rcc/jw4W51Pbe455kzZ2Bd4ZL179+furxE
7TMy0KuysjJU4tSpU2Enw3wivB5K3gH+e/jwYX6UWN6RmZlJqR06dCggWu1w
z5DCpCAMJP/jH/+olCGYjoyQUjrA3BOAYBz+0EMPcXhqck8nNrtEkDw0DPeM
W0fO2ex78ODBFNiiRQuuOCU+6tFvxpAAn4oj1xr3hMfCApSWlnL4/v373atk
v/zyS44M6sFuCbo2OffNxQ6exfHRZ0WXVnJPUOYDBw7wo8RyT66pBC5lRCsb
O3asX+1nZ2dv3bpVxvfzMwnoRrdt22YsDKoSfSgl5fa00bf27duXnv75z3/m
8O3bt48YMUIOVgBoCMZiMFKce+7bty8/Px8lhv9TZ4ndeck9zTRTwriyzl3u
mRCbIJEMA1g7KCkpIcmnTZuWqDR1+3pHODCe7lNt4s0332zevDnJ0KBBA3cE
rdzpck8QN/YHnnnmGXqxdnYSuWHAPWFkQDmVhpYo7mmgV9OnT+dXOnToAFsX
/Inweih5BwAHjx8llndccskllFpI7nn77bcP9EJlZWWtCZOCMJDczWuU1e8p
pQOSe/LIDMzIpZdeyuEpyz0Z//jHP1haM+7p3qEg0xw5ciQF/vCHP6SQyy67
TInPdBLIzc2dN2/eihUrUKGPP/44BX700UfuyMnmnjyJADHiLtyqqqriLIDr
xU1ccs+Ez3sCr7/+Oj9KIPdEOTC9SqDrBXbA050DBgyYMGFCQUFBv379ODuo
Ahlf+pnIKRQGlcWOd1Zsmy0cdWN5IAClk5OTo3SmYMH8FamWcDayXEgI96yu
rp5zNpiAjBo1SoYvXrw4+ufOD5yX3NMx0syE4NzinsmwCYQkGcDawcKFC0ny
BHJP3b4e+N3vfqe4EHXCPdFjNm7cmGX45je/6Y6jlTtd7inB3PMcmvcEJXR3
eYningZ6haJLT0/nt2688cbgT4TXQ4V3NGvWjCcX4vKOkydPwhrDa4Ub9tln
n/kJA98bXIM/sWPHjs8F5CybI7jnX/7yl+A8IllO5MyZM4pg/Ejxb7WEAShc
Wtfy8vK33noL4hUXF586dcpTNvjM8Figbx988EGY+ZcjR46sX7++sLBw9erV
bhnMJGcQr2natCkPBykRUkoHJPdEo6BA2BwpoSf3RPNERzl79my0Lzk/5QfI
U1RUNH/+fJpB69KlCyXuxz1D5pRgxj0D6gjYsmULp8kkqFOnThTy85//XIk/
bNgwMlzKImdU7ptvvvn222/LyMw9hwwZ4sQGD1EsyCkoMHoTxbsArav6GkrT
Q1VSeE1NDQeiBBB+8ODBvXv3slTjxo2rEpA7rY4ePYrXwQ6Y3wEbNmyQ8T2L
UZd7ws9BC4UavPPOOzt37nQvBVe4Z3Z2Nu83j8s94yaOikBqiCYdPOhY1dmI
m4sAQABQqkWLFnHnBdXKy8vjz8n1MFKMTz/9lAJRES+99BKHQ3OMheEJAkCZ
LoFFpfA+ffrIgkK0gq/B3DAh3NONxx57jNKHMH5xUIyKepNZQHuHC4qK9jyO
AF0AGu+KFSvQKUBFPberQxmqfODZg6Ap4RHPsuH1jRs3QtnWrl0bvB0ebXn7
9u3Q3iVLlkDy4DFMZId0uLS0FO0X1cE1mCjuiZIsKytDyUAk9AueyzhhEKgo
eGkEMu5nnQygq5m6lUWAGYQdWL58OWwvKC2chxdffJES9+OecW2IRJiS1IWx
TUBLQR1B2yH8qlWrkI57Ma2xAQyTOAM9VEVFBVyIpUuXQmE+/vjjuGeGxG0j
sKLosBAOMSZOnEiS5+fnK5K7tw+HFEa3r3fEfN+sWbOCff6k4oUXXqCvd+/e
Hbnz9JC1cie5J6wQEkQBUiNCUw1Wci3uqdt8IAmqGH47ulcYkMrKSndm/bgn
9yOAMtgCFkD9ndxFHsw9wzcHA73iF3l6VLp2boTXQ4V3AOPHj6dHAbwDfQF6
6oYNG8oX27Vrh45ARoO3oMRxQ+Ha4bknWq5bZsKdd95J4VdddRXXgoEwYJcU
Ttty33///TZt2sj4Xbt2lfFR+/fccw9v3SU0atRo8ODBfv0jlLZ9+/aKGNdc
c40828dAcgniNRdddJFfhJTSAeKe3/72t+kR2pET2z+O3zxtqnBPNL0xY8Zk
ZGTINDMzM+WSTgk4UZ07d5aRb7vttl//+tf02809Q+ZUwox7BtSRE+upv/GN
b1CafP5Jjx49KKRbt25KfF65MXbs2LgCMPd8/vnnkS/Q1SwBlL90NdesWcOP
lNzhdQqXU4SgjVnxgL6Y4+fm5gZHhlJ55iI894QOgPwqyaKZQ1QZTeGewOzZ
s+lRAPcMmTgczrjF4pfT8HD7q7KUNm/ezOGefialwPrQs2dP4wOE4Ynx7IbC
H0eOHEnh6HP9XmcZ6pB78t7AGTNmOLGSVNZHwfXiyLBL69at81RmVL3ic86b
N89PB0BaFTHYH0OZoGeZNm2asiwZdMA9fgWvYPXq1XLimzB58mR3Y4HwTLsY
cgl3dO6Jj06fPl35BMrTva2GrRMoWHl5+aBBg+QrinUygK5malUWARIOGTJE
xoRXwEtQ3NwzpA0hhC9JXRjYBPjScBuUDeYkvOKFGhjA8IkTQH94tFMCgWi8
xm0ERRFX8qyzOzUtYXT7egm0fYpWJ9yT9kalp6cHzIZo5U5yT1BI5SghmAJQ
Ub8PheSeus0HNQVDxI2X0bdvX3gFsh49uSdMK49XuJWEgU9wHD/uqdscoujV
K6+8QtFCntkbVw+Zd3Ts2JEWaYOvEX/34x27du367ne/60kiwLPkCiW5R88P
xtwTeOCBByhykyZNeO8S+gUKRCHL00sMhHn77bcpvHv37jAOoAZKfJ62BsBZ
AlJWWCrhkUceCXiFd5cYSC4Rnnumgg4Q9wT7/s53voMf4DIIbN26NX7zbL7k
nuj1rr/+es9k69Wr99JLLymZheSXX355gDAK9wyfU4lkcE+AJef5jueee45C
lEOiHLGgC4BpDR6tkgt0PQEXmqc4A7gnD9alMvfcvn17nz59/BKXR0Yw94Sj
mJOTkxWb/qCU/bhn+MRrh3u6AcPC6Uut8PMzZWYB9ylY4QFqRolIywnPnxMP
OFQ2pbhnfn5+aWmpwviAOXPmcGTOrCfQUqSXYsY9AfavFChHQ6Dx8nJ3N554
4gllqJyXofohIt0D9XY7bwxlZyVbJ/hy7jLPOts6mUFLM3W5544dO3jW3hMK
9wxvQ3RLUhe6NgH0yrOCGJI76xpArcQBsJLgYlfc/vBtxIB76gqj1ddL1C33
TLgnI22dJ7Kzs/1sURjuqdt84BgHG/YFCxZwZDf3hM2fOnUqR4Yl8RMsLvfU
bQ6EWtOr8NwzMzMTBpx+06CuJ+9A0d14440Unp6eDn4NTwCebdu2bSkQjjq3
0Llz51IhyGmm+++/X5YPPAopD3NPkLUCF5RxKqhlq1atKH779u3x9OjRozw1
DJ2UkQ2EYe4JLsbEs2nTppCtW7du+LQsVZBlinDttdf27t0b0qIXk7QFfEQm
js6dH6EkQXny8vKgq9dddx1CWrZsydedGEguEZ57poIOEPe88sorH374Yfy4
4YYb0K/RWygf+iG5Z9++fTlZcGdwVaTGm0MbNmyobNLhOfG02Nwl8gtOcfHF
F3Og5J5aOZVIEvfs0KEDScKOFowqfUU5hdU5myFmxY7KQcdK88huKNyzf//+
cAx4zJ/Awzu63LOqqmrm1+A7PdEvzBSAIeX4ixYtokB4+PyhadOmcWS/riQM
9/zyyy957BQmAknBSy8pKeERaWgUNz3mnnA8Zs+eTb+pc/HknlqJQzMpO3IR
LOpRFktCNqpAWw4fPowO+t1335WL5ZSONcDPrBIbb/2WE4TBsmXLOB1ekLZ+
/XpW0YARg5TiniQt1yla1uTJk2G95VqI4uJiVvXCwkKwkoULF8o5O3mq6ubN
m2eeDY4WzD0JaKqTJk2SG/GgVPIVJMKPoI34NDyTWbNmcS54Qt+JrfDhyFBj
6AnkkWNZWZG5p5zUGzVqVFFREQSQ803ScdKyTmbQ0kytyoLfKAfT8Hv+/PnI
kawsyT21bIhuSepC1yacPHmSNBw8Cy4QFB4FImd5YEjZi9M1gFqJO7FjUvgR
Im/btm3jxo2geHQU6tChQ5V1leHbCAgLS8ijBIMHD1a0QnZqusJo9fUS5wT3
DJ87xdZBseFaoAuQbcpvtUwY7qnbfKQtQvN87bXXUInwculb6KTkhK/CPaGc
8Kv59eBNoHG5p25z0C15BcnjnmBSBw8ebNKkCX7/6Ec/gswskuQdyBo71XKd
P8qB10miOpSvwCfkt6Kcc+tePwb569evT0/Hjx8/ZMgQzg54qOcnwgvD3JNw
xRVXgFP4UQmUGCinMiZZXV3dqFEjeh2+DYfv2rWLw3/84x/v2LFDpgMd+PDD
D6NILhGee6aCDhD3BMFHU0qLzV3CIJAAzO6Ze6Lz4iUEPXr04IaGbpGmTdNi
gxKcOJozy9CtWzeuSlTT1VdfTeGSe5rl1Eka9+zevTtVDYfwTlj3Ae+wLfLw
f9mrwqdSjJK0qKAn3HbY+wJWr15NgbrcU8JzUa4f0Hfzh8Jc7BuGe8qVhNL5
P378OI9L83Q2c090RlDanj17ZsVcX/iHfKG85J5aiTPgyvJbyThqA7Wp6ACy
gx5TqbsAP1OmEOWKE+SO0+GDI9grA7sJeDfVuGdWbGlTWVmZXwNHOFgbVEhO
yckSCD6chKMFc090OlB7Cof7yg1QHiR17NgxXpcFXiMFZo2FfrJN4PMu4HVL
Trdo0SL+bhTuCa+b0wFnZ88KhcNrmPmgOUfTOpkhimY6gZUlhYSLwpvCampq
2DWV3FPLhuiWpC4MbAJ8dbRQZcml7IncpzGEN4BaifPoBHReeu9oj/CvlKsf
dNsIg2fBgpuzljCOZl8vcU5wz/C5k7YOZciuI2zds88+y4+Ui+kJcbmnbvOp
qKjg+Gi88lBrvEsbRWX6knuiouHz8OvwZoOXaoRZc2vQ1mpNr8Jzz0suucQR
s0hz5swh51/hHXfddRcFXn/99UpSvOhUkixC8rgnwHwTpImO2AIfCeiJzLgn
ykert4UxgYs+Y8aMli1bUgqdO3fmpwinQHCr999/P2SayeaeqaADxD1BzJnf
NWjQAP/DzsA40OvMPdGVU8gFF1ygnP+Tn5/PJcy7UXgjPFRF6eM8z7k1y6mT
NO4JhRw+fLicroLZGR7DwYMHPV8pKSlhAyghXThHeHewtNLQyRMI2ec/p7nn
6NGjKcLTTz8NHftKgNcZ8ngCc08UgiOsOuqXj2CV3FMrcUbtc0+oBBRDiRbg
Z8Jh5kdyWakBePqGy4GnApXzrxSkGveEnQxfWdBGuCVr166dP38+Mwh5orgb
/KFg7qn0dHzqZpZom7JmIYNUy927d/MjsiHgOBwSMDMehXuiBimR7OxsZS+A
lJ8faVknYxhrphNYWbxeAqIqVtrznFstG6JbkrqIYhNQU/v27SsrK4O1lMst
3OckGBjAMInL+ejc3Fy0lIBORKuNSITknlrCOEZ9PaFuuSed2ZhATybgnNt1
69bxI8+7fuJyT93ms3TpUg7csmVLcB4dwT3fiIHfharEPeQ/DPdkhG9rxnqF
Nk565b6CwRPhuScdhlxVVUVnqrRr1w6tw807vve971Eg+F2Ls8GHsUiSRTDg
nqDn41zwrAIYB2XHX05OTsAnzLhnmEOSIQk6mnvuuYcnxSTatm3LMXv37k2B
P/nJT+ImayC5RHjumQo6QNwTKeM3LbslbN++nTc7M/dkAug++RndGb/Lm6P/
8Ic/UIicDCV4ck+znDpJ455mQKspLS1l0seYNGkSxwm4Y4V3Q7CDce5yTxj8
gI1UjFGjRlF8edYQcrp37176DaeUaQhzT93EGeFdL/QvGwPhOTqHQJQ2/FsU
vtwhggYle8AAP/PAgQP8KOIEE08i0HgycsQpB/d9qcY9w5QDlHDRokXyxEKJ
4PuMOFow91T8MUjFj/iAO+kyBYCWCO7cuZNDlPXVieKezKFA95RHYOj8CbhS
FKhlnYxhrJlOYGXxoj46QlzCzT11bYhuSerCzCbAoE2ePNl9Yg/BfVOGFvcM
nzi6DOV8p6wY71uwYIF7PkirjUiE5J5awkRBXXFP9D4FBQW0CjEzMzNRyQbY
OpggfuSZ2bjcU7f58IrWkGsJmHsqGzPhzcZ9NyT31G1rxkD9kl794he/KCoq
iktAdLmnI64a5A2JzDtgGGkSKhi/+c1vlK8YcM8wZw0x0AFx+qAJwVdgmHHP
uFc2fPHFFx07dpTlgLJq2bIl7z/92c9+xpF586O7rBIiuYQu93TqVAck91y+
fDnFb9OmjSMO2mLuyYfT/va3v1XSgQ7w59jy/OpXv/KL7+aexjl1Uox7Mnbv
3g0zJU0TV0SAd8eD/+cB94Q3Hrw9n+DHPRGC7pX+hIGiH8w9dRNnhHe95DFB
nti1a1dwER0+fFiu/JGnsQX4meXl5fzIb8twSEBCToru8aHfw4cPD34x1bhn
XIWsrq5WlhxAPRDCW56TwT2lx8LckzfdB4P8alnX8Ddk4oninp4H+xBQaPwJ
vtJXyzoZw1gzncDKQnv3y6ybe+raEN2S1IWuTThz5oz7eGQ4xvJ8V2PuaZD4
sWPHYC7cnnnPnj0LCwulS6/VRiRCck8tYaKgrrjnlVdeSd+tV69ewJFxugiw
dVLDPUcC43JP3ebDG5Pd8T3hueQsK7buOuBWIEJc7mnW1owBf5hvY0mLLUQM
jm/APbds2UJ76GhTatrZc168ghRP3fOSBPehwZJ3BDtXBtyztLRU4Qi0QM4P
4YXR4p433HADxczIyBg6dOiGDRtoTgHeOIVL7pmdnU2BfIVlGISXXMKAe9ah
DkjuiRZHpwbRzQVu7skTo5BESQd9BH+O90N169aNQjp27KjE95z3NMupk6rc
k7BgwQI2TWFmFoK5p3J5UIpzT0dcPQPOWOwDPp/KzT3ZQeXpCbnmVitxhnS9
5BYSN6JzTwJXk7x/J8DPlMc/Gq/fI6DHlEsZOeW4jPLc4p4w/ixwr1695s2b
h6ohF4IXYdYa9+RTj4CioiI/zaQzdiQFUw7LTRT35OkD91Tg1q1b+RN8AVDt
cE9jzXQCK4sPTHPPrXiuudWyIbolqQtdm8C74LNiBHndunWkVHIhdzD3DDCA
Zok7sQVp6EpmzpyJIs0SkH2QVhuRYP4yefLkMEUaRpgoMOCeIMVxqVBcMPds
2LBhwPVzugiwdVI5PVfAxuWeus1nypQpFAKKF0Z4hXtCHh5ZijtSEZd7GjcH
Y/Ts2ZPdWrgQwZENuCfQtWtXSeUk7+BHt9xyS3iZ0Z1xap6n6DB0uSdaDegb
Ewr+SsAd6OGFCc899+zZwzGV/Zue3JO3LoJkwZMPk1MtySUMuKdTdzoguSew
ZMmSCRMm0E58N/fkkPT0dGW3/ujRo+lR/fr12a7y1uALL7xQuQTck3ua5dSJ
rXzjzIahA8ngnigQFJS7r5QW24x7lpaWcgr79u3jmPgWL4GrE+4Jl4zjK7v+
GXxHITqCuMMCbu4pUyBI7qmVOEOeGOkeV5eoqKh4JxDK5Zvbtm3zvOyet4RA
Tg708zPlBMfIkSMjXmbhCGVDcfFpn3Gvbjm3uKfcI6Z4/rXPPaEGHBj3zkf0
dBxZ2e8pOUIU7gnfntOB8stH8lghtl21wz0dU810AiuL93nB7VQW/DBtkdxT
y4bolqQudG0CL2IErZYOc7A/HNIAmiUucfr06Y0bNz7xxBMUGVXM6Wi1EQnm
4O4FLcEIECYKtLgnFOPuu+9u0qRJgwYNfv/73ytn7WoBPi2Sok9rLecLRoCt
k1toPa8T5SFW9117BN3mIxdmh5nbldwzPz//1KlT8nqmYF2Nyz2jNwctwPeg
ym3evHn//v2VYUk3zLgn2JMf75Anf7qPSkDf5DlshTKvV68evRVwG4ijzz2Z
if/gBz9Aw7nvvvvoz6ZNm/r5/OGFCc89V65cyTHlWu5Vq1Z9//vfp3DJPbds
2cJzta1atZLDRLBIKGRkJIrkEmbcs650QOGeEm7uiX6K07z33nv5CEH0mM2a
NaNwuRUUXiVLPmDAAA6fO3cu8k7hfufchs+pE+tMA150I+HcExYbflpW7FBW
EGGyzFAt+FHydioz7w4azinMmDGDmsbBgwflfpY64Z7y2jWZPiuGc3b3MXXq
VDlvi/L58MMPpVPtyT2lP5Z1NvfUSpwhvX1lD2YUbwTf6tWrF/QcbZnTRC7Q
afLoK12l5M4XShKvHDhwAE1GLgL0Y/RakBSGyzAupT23uGdZWRnH5BWJKFL0
FExqao174gfnDl9XzuWoqamR56k6wmXKzs5m4aurq2UDj8I95V4qGAQeHkFz
wxcpXM4S1hr3NNNMJ7Cy0Nz4KdxF2pGNNOX6E8k9tWyIbknqQtcm8Gmuubm5
HIiuQZ5/4vaHQxpA3cRnzZoF/o42q9QgbzxBLniDvG4bYcgzZBSfU2ZES5go
0OKeQ4cOlW4eHKGIX6c76eB9RUyHIW3duHHj5PVnrOFyBFWCL5BF2bK5ls6A
bvORI4r9+vWT/OvYsWPQBESW6bMhHT16NIWj/2VGDDUL2Egel3uatTVjQDdI
SUJO0JtxT0fspEtz3e14880386O2bdvClYUwcLDplbvvvtvzQ7feeiu/1aVL
l7wYevToAUYmGyNzz4yMjP/xgsx4UVERp0lLH6GZPPXfvn17qQYGwoTnnugO
ZHE99dRT0IfbbrtNtmvJPR0xH5oWW6UAMdCxgoMwV/WclA8puYQZ93TqSAe0
uKcT80g5zczMTLgiDz/88IUXXkghjRs3Li8vl5Jfc801HP+Xv/wluptOnTrx
RS1prvs9zXKKF5s3b84y3HHHHeCAHTp0UO6cZSSce7oXZ8KJUnaawBHi+Fre
HdqCvCYbmeJLGRh1wj3Rd+fk5PAr6Fjz8/NBvuSwJ8w4qo/j4CkkgR+IEqNr
9fCUtdGTezqx65hlwZolLiHfGj58+MyZMwsLC9E6evfuraxqDg95xA26ubFj
x6IclJNM5AJg6Wei0vnGAYbndUIGQEkqqihbNAM9+5MC7O7iXQ5ULk2OgsRy
zz179kg1QFubMmUKDQcxao17AuAs8tNQBvgSaNFoIFSw8jiakpISjtmzZ0/o
4ZgxY9gZI0S831NuOkb5FBQU4Ctcxfixc+dOjlxr3DOkZroRUFlIk+/rzIpd
ugdlUBbjSe6pa0O0SlIXujZBXi8Ifxt/wubIziLLxx8OYwC1EpeDP0OGDAGL
X7RoETw6XjaZ5Zqp1GojDNlHoI2AS4Jm4nMoLrYkBsIYQ4t78kwlAd1ExK8n
YxRdVgokfPnll2GO5OiH31z54sWLOQ4UCdQVfA0vytES3ebDq1ZYV1HjcAu5
aUj+rtzvSUDXwEoLd0jySmg1d2289j4rdi49AY4WD1AYtzUzJO+OFYV3yIk8
yTuc2ORC69at03xQr169yspK94fWr1/PF3EqkCswg+9YAXiT76FDh9i9f+CB
BziFNWvW8If8TqMKKYzWfk86XNqNa6+9ln4o3BMq5/dKWoyNetqikJJLGHPP
OtEBXe6Jjkk54kmWIdqjkgh8A57ilOCjiSX3NM6pExtzc8e/+OKLPZdRJdxi
Hz9+/I033gg4tgI9oOcdKyG9O2mxGXDbxo8fT7/rhHs6sUXabsHgP8vM7t+/
302WJXj42o97Si9Fck+txCXcEy4M470z8Bt5QZcnlJs6lflcCfRoy5Yti3ss
fHhIpyvLZz8a7FKA8IQwx4+HRML3eyprsxm8z6s2uacTu1AgwCDAweaY6Jv4
pg8JST8jck8YKEk3JCCkcox/rXFPJ5xmuhFQWU5sL4DC3BVI7ulo2hCtktSF
rk3YsmWLp4716dOHSb2nPxzGAGoljreUoR4FOTk5yjJLR6eNSPC+XQW85MlM
GDNocQRQYOmfXHfddRG/nlTu6Vk1AY4EmgYf7CYhbYVu8zl16hTIr1894hXZ
fXhyTyfGLPgVz/gBYE/GuK2ZIeHcE3SAIii8wxGH5yi8w4kN5YFiX3755VJv
L7jgAnBAOc3k/tbVV1+teONIXC5mgNL6+fkE5p73338/hbRo0UK5WDY3N5ce
gcIEZDyuMOhH+FFc7nn48GHQFl4Cih+ZmZko/xMnTlCgwj0JixcvBjnlt9Ji
V0927tw54PKgMJJLxLUGKaUDt9xyS1psrtD9LrwCiq+MRZ85cwYckwciqPBB
SP1GwxCOquHI4IPPPvssBKA/3QdVmeUUnTLeSk9P51egje3atfNcppuks4Yq
KipABnkPJqF///7QOmV5T8AeNE/vDlo9bdo0ThOe1Ysvvrhv374NGzZQiN98
AXNPfDGu/Mw9USmSPAYAbnNRUZHi6Y0YMULZDoweBIWglEzv3r2hY9IHkK6X
5J5QOT7HQOGe4RNXsHXrVjk/QsBXlLNGtYCOFb0e7KFSIKgsd+uQO54IkGfC
hAlz5851X30eEfJUKPhdngMytcw9Wc8DuKfcUBx3MARek0Jkhg0bRidC0J/B
3JOX5rrpDNoCex3huacT2wuAqleUYciQIStXrlQMAtoRWqic50LPi9d5oVfc
a0fiAhYSPhhTfgLsg3t/pa51ioIwmulGQGURkCm2GFmxYTqoGQ9huVcUaNmQ
8CWpCwObgALkSqGcoqeoqanhAQQ/fziMAdRKHDoMG/LCCy9w7RCg/1OnTlWO
TmKEbyMM9IaFhYUKEQAFUKazDYQxgBZHgOZMnjyZV7i1bt064tcT7smwrYOT
Bn9GNiIovPuWagV79uxRrrhCo1YsvEHzgZ8jJcmKDcXA31NG5HiTgtJP4Yt8
ZC5yx5fsaHFPJ0JbM0DCuWdEIJtr1qxZvnw5yjzkGRRQhmXLluEt9OMRj0yM
joQL89lnn6E5QCXiclUJuIhlZWVwG2B+Q+7wCi95ss9QTR0dAK2DnpeWlion
rngC3y0uLkaxhz87xSCncP/wCbAhSBVws0yy6+jw4cObN2+GI+G5Jd8YUPId
O3agNhOyUSWBgDxwIVDyO3fuDD5nAwQBxYLCQe1EP0UneuLQMbBC9HqQ3/OY
IDPAhUaC8OKgusaLeC0MAPWDDsC3jH6SZKKALgamEspQWVkZrAyICYWBQmp1
Z7qAzkOSXbt2nfeaiRYNa4DMhl9CoGVDUqQkScFgxPbt26drVOMaQLPEkSzK
8JNPPoGfH2Y8IXwbYaDTQcmDH6G+9u/fH+DI6QqjBQOfv7q6ml7p0qVLxK+T
J5ORkZHARTIKUHqbNm0Kf9GD83WB0wEdAQWu23xOnDgBLUV8qGKirsjRRZS2
poXCwsKU4p4WqY/avL/Dwgy2jiwsLCwsLCyiwMDn79u3L73y8ssvR/x6Xl4e
JeW3PtniXMSpU6fatGlDNRtycbjlnhbEa+rXr89T8xFvh7dIOGwdWVhYWFhY
WBhg0qRJ5Dnw6YhhfH5wisGDB1P8Fi1aRJ8u37ZtG28yatas2a233hoxQYu6
xerVq8E6GzduTHV61VVXBcc300OL8xLEaySWLFlS10JZnAVbRxYWFhYWFhYG
4AvKGXF9/t27d7dq1Yoig1wUFxcnRJLc3Fw+5qJBgwYJSdOirjB9+nTWqIyM
jPXr1wfHN9BDi/MVAwcOvOJs+J1KZFFXsHVkYWFhYWFhYYAePXooLoS8P8sT
J0+evOyyy8AO7rzzzpDnOYdERUXFgAEDunbt2qlTpwQma1H7WLJkyU033XTf
ffeNHTtWOdPVEwZ6aGFhYWFhYWFhYWFx3mP27Nlxj4q1sLCwsLCwsLCwsLCw
sLCwsLCwsLCwsLCwsLCwsLCwUPC/q+OIZA==
"], {{0, 17.}, {618., 0}}, {0, 255},
ColorFunction->RGBColor,
ImageResolution->{144., 144.}],
BoxForm`ImageTag["Byte", ColorSpace -> "RGB", Interleaving -> True],
Selectable->False],
DefaultBaseStyle->"ImageGraphics",
ImageSize->Automatic,
ImageSizeRaw->{618., 17.},
PlotRange->{{0, 618.}, {0, 17.}}]\)
Out[10]=

Feature extraction

Remove the last two layers of the trained net so that the net produces a vector representation of an image:

In[11]:=
brushing = ResourceData["Tooth Brushing Video Samples"];
In[12]:=
cheerleading = ResourceData["Cheerleading Video Samples"];
In[13]:=
extractor = NetTake[NetModel[
   "ShuffleNet-3D V1 Trained on Video Datasets"], {1, -4}]
Out[13]=

Get a set of videos:

In[14]:=
videos = Join[brushing, cheerleading];

Visualize the features of a set of videos:

In[15]:=
FeatureSpacePlot[videos, FeatureExtractor -> extractor, LabelingFunction -> (Callout[
     Thumbnail@VideoExtractFrames[#1, Quantity[1, "Frames"]]] &), LabelingSize -> 50, ImageSize -> 600]
Out[15]=

Transfer learning

Use the pre-trained model to build a classifier for telling apart images from two action classes not present in the dataset. Create a test set and a training set:

In[16]:=
videos = <|
   ResourceData["Sample Video: Reading a Book"] -> "reading book", ResourceData["Sample Video: Blowing Glitter"] -> "blowing glitter"|>;
In[17]:=
dataset = Join @@ KeyValueMap[
    Thread[
      VideoSplit[#1, Most@Table[
          Quantity[i, "Frames"], {i, 16, Information[#1, "FrameCount"][[1]], 16}]] -> #2] &,
    videos
    ];
In[18]:=
{train, test} = ResourceFunction["TrainTestSplit"][dataset, "TrainingSetSize" -> 0.7];

Remove the linear layer from the pre-trained net:

In[19]:=
tempNet = NetTake[NetModel[
   "ShuffleNet-3D V1 Trained on Video Datasets"], {1, -3}]
Out[19]=

Create a new net composed of the pre-trained net followed by a linear layer and a softmax layer:

In[20]:=
newNet = NetJoin[tempNet, NetChain[{"Linear" -> LinearLayer[], "Softmax" -> SoftmaxLayer[]}], "Output" -> NetDecoder[{"Class", {"blowing glitter", "reading book"}}]]
Out[20]=

Train on the dataset, freezing all the weights except for those in the "Linear" new layer (use TargetDevice -> "GPU" for training on a GPU):

In[21]:=
trainedNet = NetTrain[newNet, train, LearningRateMultipliers -> {"Linear" -> 1, _ -> 0}, ValidationSet -> Scaled[0.1]]
Out[21]=

Perfect accuracy is obtained on the test set:

In[22]:=
ClassifierMeasurements[trainedNet, test, "Accuracy"]
Out[22]=

Net information

Inspect the number of parameters of all arrays in the net:

In[23]:=
Information[
 NetModel["ShuffleNet-3D V1 Trained on Video Datasets"], "ArraysElementCounts"]
Out[23]=

Obtain the total number of parameters:

In[24]:=
Information[
 NetModel["ShuffleNet-3D V1 Trained on Video Datasets"], "ArraysTotalElementCount"]
Out[24]=

Obtain the layer type counts:

In[25]:=
Information[
 NetModel["ShuffleNet-3D V1 Trained on Video Datasets"], "LayerTypeCounts"]
Out[25]=

Display the summary graphic:

In[26]:=
Information[
 NetModel["ShuffleNet-3D V1 Trained on Video Datasets"], "SummaryGraphic"]
Out[26]=

Export to ONNX

Export the net to the ONNX format:

In[27]:=
onnxFile = Export[FileNameJoin[{$TemporaryDirectory, "net.onnx"}], NetModel["ShuffleNet-3D V1 Trained on Video Datasets"]]
Out[27]=

Get the size of the ONNX file:

In[28]:=
FileByteCount[onnxFile]
Out[28]=

Check some metadata of the ONNX model:

In[29]:=
{opsetVersion, irVersion} = {Import[onnxFile, "OperatorSetVersion"], Import[onnxFile, "IRVersion"]}
Out[29]=

Import the model back into the Wolfram Language. However, the NetEncoder and NetDecoder will be absent because they are not supported by ONNX:

In[30]:=
Import[onnxFile]
Out[30]=

Resource History

Reference