How to detect text using OpenCV

Question

I'm working with a project where I should detect and extract the text from a image to later make it available to a search software .

I'm studying OpenCV but I don't find much content in Java just in Python. I'd like to implement this in Java. But if you know how to do this in Python, C++ it's ok. I just need the algorithm to get any idea.

My plan would be rotate the image 90°, make it a binary image (thresholding), detect the ROI (Region of Interest), in this case the text or maybe the shape rectangle, crop the white rectangle that holds the text and finally use OCR with Tesseract to obtain the text (PISF - 28 - 1469 - 3).

But to extract the text using tesseract is OK, I know how to to this. I just need to get the white rectangle that holds the text or the minimum region that holds the text and save it in the better form to later use it with Tesseract (OCR).

I would use the script in batch because I don't have just this image. And the other images can have different sizes.

Could anyone help me?

I'd apreciate any help.

My image is this:

Answer 1

Here is one way to do that in Python/OpenCV

• Read the input
• Convert to grayscale
• Threshold
• Use morphology to remove small white or black regions and to close over the text with white
• Get the largest vertically oriented rectangle's contour
• Extract the text from the bounding box of that contour
• Save results

Input:

import cv2
import numpy as np

# load image
img = cv2.imread("rock.jpg")

# convert to gray
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# threshold image
thresh = cv2.threshold(gray, 150, 255, cv2.THRESH_BINARY)[1]

# apply morphology to clean up small white or black regions
kernel = np.ones((5,5), np.uint8)
morph = cv2.morphologyEx(thresh, cv2.MORPH_CLOSE, kernel)
morph = cv2.morphologyEx(morph, cv2.MORPH_OPEN, kernel)

# thin region to remove excess black border
kernel = np.ones((3,3), np.uint8)
morph = cv2.morphologyEx(morph, cv2.MORPH_ERODE, kernel)

# find contours
cntrs = cv2.findContours(morph, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cntrs = cntrs[0] if len(cntrs) == 2 else cntrs[1]

# Contour filtering -- keep largest, vertically oriented object (h/w > 1)
area_thresh = 0
for c in cntrs:
    area = cv2.contourArea(c)
    x,y,w,h = cv2.boundingRect(c)
    aspect = h / w
    if area > area_thresh and aspect > 1:
        big_contour = c
        area_thresh = area

# extract region of text contour from image
x,y,w,h = cv2.boundingRect(big_contour)
text = img[y:y+h, x:x+w]

# extract region from thresholded image
binary_text = thresh[y:y+h, x:x+w]  

# write result to disk
cv2.imwrite("rock_thresh.jpg", thresh)
cv2.imwrite("rock_morph.jpg", morph)
cv2.imwrite("rock_text.jpg", text)
cv2.imwrite("rock_binary_text.jpg", binary_text)

cv2.imshow("THRESH", thresh)
cv2.imshow("MORPH", morph)
cv2.imshow("TEXT", text)
cv2.imshow("BINARY TEXT", binary_text)
cv2.waitKey(0)
cv2.destroyAllWindows()

Thresholded image:

Morphology cleaned image:

Extracted text region image:

Extracted binary text region image: