Finding coordinated of rectangular structure using OpenCV

Question

I'm trying to draw a rectangular contour around the prominent rectangle shown in the picture below and find its coordinates. The original image is : [

I have arrived at the below image using:

th1 = cv2.threshold(img, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]

[

PS: To make it more clear- I want to draw a rectangle using opencv as shown below: [

Please help me out with how to proceed.

Answer 1

Ask yourself: Do you need a "qualitative" answer, i.e. to draw a box whose intent is to highlight the "interesting" portion of the image, with no guarantees that it represents anything statistically meaningful?

If the answer is yes, given that the problem is clearly 1D, I'd average up all the columns to get a single curve, then pick a threshold that makes sense (say, 5% drop from the average value of the curve), find the outermost points where the threshold is crossed, and those define the horizontal location of the box. Rinse and repeat with different choices of the threshold until you get something that visually makes sense for your data. Again, the point here is "visually".

If the answer is no, and you need the box to represent some actual conclusions on the statistics of your data, then you need to dive deeper in your data. Start by asking how the data are generated, express that in a mathematical model of the probability that you observe a black dot at a certain horizontal coordinate "x". The model will depend on some parameters (e.g. if your model is a simple box, the parameters will be its extent and location). Define a likelihood function that expresses "how well" a particular instance of the model (a particular choice of its parameters), fits the data. Then find the parameter values that optimize that likelihood on a given dataset. If I were to take a stab at this problem, just looking at this one picture and without knowing anything about the process that generates this data, my first guess would be to use a Gaussian mixture, since I see both a large-ish "box" and some separate tight "lines", and the density of black dots inside the large box appears higher on the right than on the left side.

If you are unfamiliar with data analysis and modeling techniques, there is a plethora of literature to choose from. Sivia's "Bayesian Data Analysis" is a delightfully short and well-readable introductory textbook.

Answer 2

Solution using smoothing + thresholding:

1. Convolve your image with a large 2D kernel (average filtering)

2. Apply thresholding:

3. Find bounding rectangle that includes all non-zero pixels (cv2.boundingRect):

Code:

import cv2
import numpy as np

image = cv2.imread('block.jpg')
mask = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

kernel = np.ones((100, 100), np.float32)
kernel /= kernel.size
mask = cv2.filter2D(mask, -1, kernel=kernel)

mask = cv2.threshold(mask, 100, 255, cv2.THRESH_BINARY_INV)[1]

x, y, w, h = cv2.boundingRect(mask)
cv2.rectangle(image, (x, y), (x + w, y + h), (0, 0, 255), 2)