Wolfram Research

Function Repository Resource:

ContentAwareImageResize

Source Notebook

Perform content-aware image resizing by seam carving

Contributed by: Sander Huisman

ResourceFunction["ContentAwareImageResize"][image,width]

performs content-aware image resizing on image to have width width.

ResourceFunction["ContentAwareImageResize"][image,{height}]

performs content-aware image resizing on image to have height height.

ResourceFunction["ContentAwareImageResize"][image,{width,height}]

performs content-aware image resizing on image to have dimensions {width,height}.

ResourceFunction["ContentAwareImageResize"][image,"Width"]

returns a ContentAwareImageResizeObject that allows the width to be dynamically changed.

ResourceFunction["ContentAwareImageResize"][image,"Height"]

returns a ContentAwareImageResizeObject that allows the height to be dynamically changed.

ResourceFunction["ContentAwareImageResize"][ContentAwareImageResizeObject[],s]

returns an image of size s.

Details and Options

Content-aware image resizing uses seam carving or seam insertion to crop or to extend images. Important content in an image is found through "EnergyFunction". Low values are deleted before high values. Seams are found by a simple dynamic programming algorithm with a unit neighborhood.
The following options can be given:
"ProtectionMask" None mask that protects the content given in the mask
"RemovalMask" None mask that tries to remove content of the mask first
"DirectionOrder" "WidthHeight" order of seam carving, either "WidthHeight" or "HeightWidth"
"EnergyFunction" GradientFilter[#,1,Method“ShenCastan”]& function to apply to the image that gives an importance value to each pixel
Method Automatic valid options are "ForwardEnergy", "BackwardEnergy" and Automatic
"MaskMultiplier" 10.04 multiplier used for the masks
The option "ProtectionMask" should be a binary image with the same dimensions as the image.
The option "RemovalMask" should be a binary image with the same dimensions as the image.
The option "EnergyFunction" should be a function that returns a grayscale image in which the most important parts are white and the least important parts black.
Using the "EnergyFunction", an importance is specified for each pixel:
Continuous seams are found from top to bottom or left to right using a neighborhood of 1 that minimizes the energy. The first continuous seam from top to bottom is highlighted in red:
By removing seams iteratively, the size of the image can be resized.
A removal mask is ignored when extending an image.
Extending an image is done in the order as if one removes seams. Except for that seams are added as the mean values of the neighbors.
Changing the width and height is done by first seam carving along either the width or the height and then by doing the height or the width.
The Method "BackwardEnergy" calculates the energy of the current image, then carves. "ForwardEnergy" looks at the energy after carving.

Examples

Basic Examples

Make the width of the image 200 pixels:

In[1]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/375d0d4b-97e7-422b-b9cf-66e3f1eb3c2e"]
Out[1]=

Make the height of the image 120 pixels:

In[2]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/8402cdba-1af3-4b25-a80e-bc261b8b4456"]
Out[2]=

Scope

Change the width and the height:

In[3]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/882b6d0a-ae6b-4963-83f8-97a98b087a2e"]
Out[3]=

Extend the image by 100 pixels in the width:

In[4]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/7cb37058-1a60-4b6e-bb2f-1a99c98ec79e"]
Out[4]=

Extend the image by 50 pixels in the height:

In[5]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/c83a567d-9d6a-439f-8989-9d7b11f14a8b"]
Out[5]=

Make a ContentAwareImageResizeObject for the width:

In[6]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/7302cbbb-132c-4e0a-adf9-d05bde9709a1"]
Out[6]=

Dynamically change the width in real time:

In[7]:=
Manipulate[
 ResourceFunction["ContentAwareImageResize"][cairo, j], {{j, cairo["MaxSize"]/2}, Round[0.25 cairo["MaxSize"]], Round[0.75 cairo["MaxSize"]], 1}]
Out[7]=

Options

ProtectionMask

Change the width while protecting the booby:

In[8]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/e017088d-3186-4a16-b690-277a95665e72"]
Out[8]=

Without a mask, the main subject can be deformed because the background also has many details:

In[9]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/d6cfdde9-ef06-405e-8d02-dcc96d003af5"]
Out[9]=

Change the height while protecting the booby:

In[10]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/a635d927-4a00-41d9-adcb-a484f7cb98ae"]
Out[10]=

RemovalMask

Remove a feature of a photo:

In[11]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/0122df30-e44e-47ff-bb10-00a1d01de28b"]
Out[12]=

Having a removal mask while having a protection mask is possible:

In[13]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/39982ce0-05b1-4d9e-82f6-05a05d222d63"]
Out[13]=

DirectionOrder

Change the order of removal:

In[14]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/9a119485-e324-4fcd-bcde-3bccc63e44c5"]
Out[15]=

Method

Change the method and compare them:

In[16]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/8579c736-f628-4036-a6f0-566d9f4a1b20"]
Out[17]=

EnergyFunction

Change the function that provides the importance of the pixels:

In[18]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/4fde293e-3630-425e-b773-4dcbc5eaca51"]
Out[19]=

MaskMultiplier

Give an importance to the mask and see its impact:

In[20]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/555de1de-fd9f-4c92-9a9c-d3865b17b0d2"]
Out[20]=

Applications

Make a series of photos with a variety of widths suitable for websites for different devices:

In[21]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/26c51457-2de7-4b92-8139-910defa50839"]
Out[21]=

Properties and Relations

Compare to other methods of image resizing:

In[22]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/c3c0ae9b-900c-43fa-9677-07f321bdd4a1"]
Out[23]=

Possible Issues

Without a mask, a face might be considered smooth and without much energy:

In[24]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/0ff5e55c-4c38-4940-aac2-ea49baddd194"]
Out[24]=

Use a mask to protect the body:

In[25]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/04128d0a-a70f-43d8-9676-3b4f2970237b"]
Out[25]=

The width has to be a positive integer:

In[26]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/94f22ec7-5d20-4df6-912c-c942dce33f8f"]
Out[26]=

One cannot extend beyond twice the original width:

In[27]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/413d796d-b8ec-4228-91d9-a3447d8b5dbc"]
Out[27]=

Neat Examples

Trim or extend an image:

In[28]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/4a9c4881-5d1e-4e6f-90e8-cdae82acabc3"]
Out[29]=

Change the height dynamically:

In[30]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/6ffbf02a-1e70-4cd5-b47d-461c3bf34237"]
Out[31]=

Resource History

Source Metadata

Related Resources

Author Notes

Does not support 3D images. Does not support images with alpha channels.

License Information