Keypoint-Based Video Stabilization

Use keypoints and estimate geometric transformations between video frames to stabilize a video

Start with a video that is fairly shaky:

In[1]:=

v=ResourceData["Example: Keypoint-Based Video Stabilization","Content"];

Extract the video frames:

In[2]:=

framelist=VideoExtractFrames[v,All];

Compare the timings for different keypoint methods. Note that "Oriented FAST and rotated BRIEF" (ORB) features are computationally less expensive than the others:

In[3]:=

BarChartMap1000First@RepeatedTiming@ImageCorrespondingPointsframelist2,framelist1,Method#,

...

&,{"BRISK","KAZE","AKAZE","SURF","ORB"},



Out[3]=

Confine the search for keypoints to image regions that are known to be stable. In this particular case, use the bubble support in the lower half of the image by masking:

In[4]:=

mask=

;HighlightImage[framelist〚1〛,{Orange,mask}]

Out[4]=

Use

BlockMap

to step through the list of all frames and compute corresponding keypoints between each frame and the previous frame. Compute the keypoints for all frames confined to the the region of interest:

In[5]:=

features=BlockMapImageCorrespondingPointsSequence@@#,Method"ORB",Maskingmask,

...

&,framelist,2,1;

Display the key points on all frames:

In[6]:=

FrameListVideoTableHighlightImageframelistk,

...

,featuresk-1,1,

...

,mask,

...

,{k,2,80,1}

Out[6]=

Assuming a constant distance between camera and object, the image transformation between frames can be restricted to rigid transformations:

In[7]:=

mutualTrafos=Apply[Last@FindGeometricTransform[##,TransformationClass"Rigid"]&,features,{1}];

Accumulate the transformations from one frame to the next to obtain transformations in relation to the first frame:

In[8]:=

trafos=FoldList[Composition,TranslationTransform[{0,0}],mutualTrafos];

Transform all frames to undo the shaky camera motion:

In[9]:=

stableFrames=MapThreadImagePerspectiveTransformation#1,#2,

&,{framelist,trafos};

See the initial and stabilized video side by side:

In[10]:=

FrameListVideoMapThreadImageAssemble{##},

...

&,{framelist,stableFrames}

Out[10]=

Publisher Information

Contributed by: Wolfram Staff

Keypoint-Based Video Stabilization

Related Symbols

Publisher Information