Wolfram Function Repository

Instant-use add-on functions for the Wolfram Language

Function Repository Resource:

 ImageROIConvert

Source Notebook

Simple tool for working with regions of interest on images

Contributed by: Michael Sollami

ResourceFunction["ImageROIConvert"][roi]

represents a rectangular region of interest roi, given by<|"x"→xmin,"y"→ymin,"w"→width,"h"→height|>.

ResourceFunction["ImageROIConvert"][roi,image,transform]

represents a rectangular region of interest roi in an image after applying a coordinate system transform.

ResourceFunction["ImageROIConvert"][roi, {width,height}, transform]

represents a rectangular region of interest roi in an image of given dimensions after applying a coordinate system transform.

Details and Options

ResourceFunction["ImageROIConvert"] is a simple helper function when working with rectangles in systems other than the Wolfram Language. Many imaging and machine learning libraries assume a coordinate system whose origin is placed at the top left corner of an image with the positive y axis going down vertically (i.e. "TopLeft"), as opposed to the Wolfram Language convention of a bottom left corner origin with a positive y axis going up (i.e. "BottomLeft").
The first argument roi may be a Rectangle or an Association with keys "x","y","w","h", and both may be given with scaled or unscaled values depending on the option "ScaledCoordinates".
If the optional third argument is not provided, the input roi is assumed to be given in the normal "BottomRight" coordinate system (the default for both Graphics and Image), and no transformation is performed.
ResourceFunction["ImageROIConvert"] takes a boolean-valued option "ScaledCoordinates" which indicates the roi was given in scaled coordinates (defaults to False).
ResourceFunction["ImageROIConvert"] always returns a Rectangle object in unscaled coordinates.
The examples were tested with Python 3.8.5 and OpenCV version 4.4.0 (both installed with anaconda). Make sure to have libraries cv2 and zmq installed and configured for the external evaluator.

Examples

Basic Examples (4) 

Sometimes ROIs (regions of interest) are provided in terms of width and height:

In[1]:=
img = RandomImage[1, {10, 10}, ImageSize -> 200]; HighlightImage[img, ResourceFunction[ "ImageROIConvert"][<|"x" -> 1, "y" -> 5, "w" -> 8, "h" -> 3|>], Frame -> True]
Out[2]=

Having to switch between coordinate systems with a top or bottom origin is a very common. When converting to a "TopLeft" or "BottomRight" origin, you need to supply the original image (or its dimensions):

In[3]:=
HighlightImage[img, ResourceFunction[ "ImageROIConvert"][<|"x" -> 1, "y" -> 5, "w" -> 8, "h" -> 3|>, img, "TopLeft"], Frame -> True]
Out[3]=

As a convenience, ImageROIConvert handles regions given in XYWH form. They can be given as real pixel coordinate values in the range [0,w]⨯[0,h] or as scaled coordinates in the range [0,1]⨯[0,1]:

In[4]:=
img = RandomImage[1, {20, 10}, ImageSize -> 200, ColorSpace -> "CMYK"]; rect = <|"x" -> .25, "y" -> 0, "w" -> .5, "h" -> .5|>; HighlightImage[img, ResourceFunction["ImageROIConvert"][rect, img, "BottomLeft", "ScaledCoordinates" -> True], "Darken", Frame -> True]
Out[4]=

To run this example, you first need to connect to a python executable in an environment (e.g. here my environment is named dl) where both opencv and pyzmq are installed (for more details see this workflow):

In[5]:=
$sess = StartExternalSession[{"Python", "Executable" -> "/Users/msollami/anaconda3/envs/dl/bin/python"}]
Out[5]=

Now let's draw a rectangle on an image with the python computer vision library OpenCV (which is named cv2 in code):

In[6]:=
img = RandomImage[10, {50, 50}, ColorSpace -> "RGB"]; ExternalEvaluate[$sess, "imgD=" <> ExportString[ImageData[img, "Byte"], "PythonExpression"]] Image[ExternalEvaluate[$sess, "import numpy as np; import cv2 as cv; img = np.asarray(imgD, np.uint8); cv.rectangle(img, (10, 10), (20, 20), (255, 0, 0), 1); img"], ImageSize -> 150]
Out[7]=

Convert the coordinate system's origin to "BottomLeft" to plot it:

In[8]:=
HighlightImage[img, {EdgeForm[{Thick, Red}], FaceForm[None], ResourceFunction["ImageROIConvert"][Rectangle[{10, 10}, {20, 20}], img, "BottomLeft"]}, ImageSize -> 150]
Out[8]=

Create a Rectangle region of interest and highlight it:

In[9]:=
img = ConstantImage[0, {20, 20}, ImageSize -> 150]; rect = Rectangle[{3, 5}, {10, 15}]; HighlightImage[img, {White, EdgeForm[Thickness[.05]], FaceForm[None], rect}]
Out[10]=

To use this in OpenCV (whose python library is named cv2), we ask for a "TopLeft" origin coordinate transform:

In[11]:=
converted = ResourceFunction["ImageROIConvert"][rect, img, "TopLeft"]
Out[11]=

Now we can send it to a session for drawing (requires a python installation):

In[12]:=
ExternalEvaluate[$sess, "[[x1,y1], [x2,y2]]=" <> ExportString[List @@ converted, "PythonExpression"]]; Image[ExternalEvaluate[$sess, "import numpy as np; import cv2 as cv; img = np.zeros((20, 20, 3), np.uint8); cv.rectangle(img, (x1, y1), (x2, y2), (255, 255, 255), 1); img"], ImageSize -> 150]
Out[13]=

Options (2) 

Create a Rectangle with scaled coordinates for use in OpenCV (this example requires a python installation):

In[14]:=
img = ConstantImage[0, {100, 100}, ImageSize -> 150]; rect = Rectangle[RandomReal[1, 2], RandomReal[1, 2]]; (* scaled coordinates *) pyrect = List @@ ResourceFunction["ImageROIConvert"][rect, img, "TopLeft", "ScaledCoordinates" -> True]; ExternalEvaluate[$sess, "[[x1,y1], [x2,y2]]=" <> ExportString[pyrect, "PythonExpression"]]; GraphicsRow[{Labeled[ HighlightImage[ img, {Red, EdgeForm[Thickness[.05]], FaceForm[None], ResourceFunction["ImageROIConvert"][rect, img, "ScaledCoordinates" -> True]}], "MMA"], Labeled[Image[ ExternalEvaluate[$sess, "import numpy as np; import cv2 as cv; img = np.zeros((100, 100, 3), np.uint8); cv.rectangle(img, (x1, y1), (x2, y2), (255, 0, 0), 3); img"], ImageSize -> 150], "OpenCV"]}, ImageSize -> 400]
Out[18]=

If setting the option "ScaledCoordinates"→True, the coordinates of the input are assumed to lie between zero and one:

In[19]:=
rect = Rectangle[{.5, .5}, {1, 1}]; HighlightImage[img = RandomImage[10, {60, 25}, ImageSize -> 250], ResourceFunction["ImageROIConvert"][rect, img, "ScaledCoordinates" -> True], Frame -> True]
Out[20]=
In[21]:=
rect = <|"x" -> .5, "y" -> .5, "w" -> .5, "h" -> .5|>; HighlightImage[img = RandomImage[20, {40, 20}, ImageSize -> 250], ResourceFunction["ImageROIConvert"][rect, img, "BottomRight", "ScaledCoordinates" -> True], Frame -> True]
Out[22]=

Applications (5) 

Convert object detections from the Wolfram Language to Python (this example requires a python installation):

In[23]:=
(* Evaluate this cell to get the example input *) CloudGet["https://www.wolframcloud.com/obj/0ae6ad10-8f60-4c16-877b-fccdab51f058"]
In[24]:=
boxes = ImageBoundingBoxes[i, Entity["Species", "Class:Aves"]]
Out[24]=

Highlight the identified bounding boxes:

In[25]:=
HighlightImage[i, %]
Out[25]=

Convert the bird's bounding boxes to have top left origin:

In[26]:=
birds = (ResourceFunction["ImageROIConvert"][#, i, "TopLeft"] & /@ boxes)
Out[26]=

Export the variables to the python session:

In[27]:=
ExternalEvaluate[$sess, "imgData=" <> ExportString[ImageData[i, "Byte"], "PythonExpression"]]; ExternalEvaluate[$sess, "birds=" <> ExportString[List @@@ birds, "PythonExpression"]];

Draw the boxes:

In[28]:=
Image[ExternalEvaluate[$sess, "import numpy as np; import cv2 as cv; img = np.asarray(imgData, np.uint8); birds = np.asarray(birds, np.uint8); for b in birds: cv.rectangle(img, tuple(b[0]), tuple(b[1]), (255, 0, 0), 1); img"], ImageSize -> 300]
Out[28]=

Publisher

Michael Sollami

Version History

  • 1.0.0 – 23 December 2020

Related Resources

License Information