In computer vision tasks, we need to crop a rotated rectangle from the original image sometimes, for example, to crop a rotated text box. In this post, I would like to introduce how to do this in OpenCV.

If you search on the internet about cropping rotated rectangle, there are several answers in the Stack Overflow which suggest using minAreaRect() to find the minimum bounding rectangle, rotating the original image and finally cropping the rectangle from the image. You can find these questions here and here. While some of the answers work, they only work in certain conditions. But if the rotated rectangle is near the edge of the original image, some part of the cropped rectangle is cut out in the output.

# The imperfect way

Take the following image with rotated text as an example, The corner points1 are (in top left, top right, bottom right, bottom left order):

[64, 49], [122, 11], [391, 326], [308, 373]


If you crop the rectangle using the following script (based on this answer):

Click to see the code.
import cv2
import numpy as np

def main():
cnt = np.array([
[[64, 49]],
[[122, 11]],
[[391, 326]],
[[308, 373]]
])
# find the exact rectangle enclosing the text area
# rect is a tuple consisting of 3 elements: the first element is the center
# of the rectangle, the second element is the width, height, and the
# third element is the detected rotation angle.
# Example output: ((227.5, 187.50003051757812),
# (94.57575225830078, 417.98736572265625), -36.982906341552734)
rect = cv2.minAreaRect(cnt)
print("rect: {}".format(rect))

box = cv2.boxPoints(rect)
box = np.int0(box)

# print("bounding box: {}".format(box))
cv2.drawContours(img, [box], 0, (0, 0, 255), 2)

# img_crop will the cropped rectangle, img_rot is the rotated image
img_crop, img_rot = crop_rect(img, rect)
cv2.imwrite("cropped_img.jpg", img_crop)
cv2.waitKey(0)

def crop_rect(img, rect):
# get the parameter of the small rectangle
center, size, angle = rect, rect, rect
center, size = tuple(map(int, center)), tuple(map(int, size))

# get row and col num in img
height, width = img.shape, img.shape

# calculate the rotation matrix
M = cv2.getRotationMatrix2D(center, angle, 1)
# rotate the original image
img_rot = cv2.warpAffine(img, M, (width, height))

# now rotated rectangle becomes vertical, and we crop it
img_crop = cv2.getRectSubPix(img_rot, size, center)

return img_crop, img_rot

if __name__ == "__main__":
main()


In the above code, we first find the rectangle enclosing the text area based on the four points we provide using the cv2.minAreaRect() method. Then in function crop_rect(), we calculate a rotation matrix and rotate the original image around the rectangle center to straighten the rotated rectangle. Finally, the rectangle text area is cropped from the rotated image using cv2.getRectSubPix method.

The original, rotated and cropped image are shown below: We can see clearly that some parts of the text are cut out in the final result. Of course, you can pad the image beforehand, and crop the rectangle2 from the padded image, which will prevent the cutting-out effect.

# The better way

Is there a better way? Yes!

In the above code, when we want to draw the rectangle area in the image, we use cv2.boxPoints() method to get the four corner points of the real rectangle. We also know the width and height of rectangle from rect. Then we can directly warp the rectangle from the image using cv2.warpPerspective() function. The following script shows an example:

Click to see the code.
import cv2
import numpy as np

def main():
# points for test.jpg
cnt = np.array([
[[64, 49]],
[[122, 11]],
[[391, 326]],
[[308, 373]]
])
print("shape of cnt: {}".format(cnt.shape))
rect = cv2.minAreaRect(cnt)
print("rect: {}".format(rect))

# the order of the box points: bottom left, top left, top right,
# bottom right
box = cv2.boxPoints(rect)
box = np.int0(box)

print("bounding box: {}".format(box))
cv2.drawContours(img, [box], 0, (0, 0, 255), 2)

# get width and height of the detected rectangle
width = int(rect)
height = int(rect)

src_pts = box.astype("float32")
# coordinate of the points in box points after the rectangle has been
# straightened
dst_pts = np.array([[0, height-1],
[0, 0],
[width-1, 0],
[width-1, height-1]], dtype="float32")

# the perspective transformation matrix
M = cv2.getPerspectiveTransform(src_pts, dst_pts)

# directly warp the rotated rectangle to get the straightened rectangle
warped = cv2.warpPerspective(img, M, (width, height))

# cv2.imwrite("crop_img.jpg", warped)
cv2.waitKey(0)

if __name__ == "__main__":
main()


Now the cropped rectangle becomes If you check carefully, you will notice that there are some black area in the cropped image. That is because a small part of the detected rectangle is out of the bound of the image. To remedy this, you may pad the image a bit and do the crop after that. An example is given here (By me :)).

# About the angle of the return rectangle

The last element of the returned rect is the detected angle of the rectangle. But it has confused a lot of people, for example, see here and here.

This angle is in the range $[-90, 0)$. After much experiment, I have found that the relationship between the rectangle orientation and output angle of minAreaRect(). It can be summarized in the following image The following description assume that we have a rectangle with unequal height and width length, i.e., it is not square.

If the rectangle lies vertically (width < height), then the detected angle is -90. If the rectangle lies horizontally, then the detected angle is also -90 degree.

If the top part of the rectangle is in first quadrant, then the detected angle decreases as the rectangle rotate from horizontal to vertical position, until the detected angle becomes -90 degrees. In first quadrant, the width of detected rectangle is longer than its height.

If the top part of the detected rectangle is in second quadrant, then the angle decreases as the rectangle rotate from vertical to horizontal position. But there is a difference between second and first quadrant. If the rectangle approaches vertical position but has not been in vertical position, its angle approaches 0. If the rectangle approaches horizontal position but has not been in horizontal position, its angle approaches -90 degrees.

This post here is also good at explaining this.

# Conclusion

In this post, I compared the two methods to crop the rotated rectangle from the image and also explained the meaning of angle returned by cv2.minAreaRect() method. Overall, I like the second method since it does not require rotating the image and can deal with this problem more elegantly with less code.

1. We only need to provide roughly the 4 corner points of the rectangle. Then we can use OpenCV to find the exact the rectangle enclosing the rectangle text area. ↩︎

2. You need to re-calculate the corner points of the rectangle after padding the original image. ↩︎