Separating grid cells from an image and cropping them

Question

I've been trying to take a weekly menu pdf and separate it into grid boxes for cropping and later OCR each with TesseractOCR.

I've seen lineJunctions which might be helpful here, but wasn't able to find them in imagemagick php documentation. I've also seen Hough Lines in a similar stackoverflow question, but again wasn't able to find them in the php documentation.

//read the image
$im = new Imagick();
$im->readimage('menu.png');

//resize and contrast
$im->resizeImage($im->getImageWidth()/6, $im->getImageHeight()/6 , 9, 1);
$im->thresholdImage( 0.65 * Imagick::getQuantum() );;

//remove "noise"
//this is done by creating two new images where only horizontal lines, then vertical are preserved using morphology and then combined into one
$horizontalLines = clone $im;
$verticalLines = clone $im;

$horizontalLineKernel = \ImagickKernel::fromBuiltIn(\Imagick::KERNEL_RECTANGLE, "19x1");
$horizontalLines->morphology(\Imagick::MORPHOLOGY_CLOSE, 1, $horizontalLineKernel);

$verticalLineKernel = \ImagickKernel::fromBuiltIn(\Imagick::KERNEL_RECTANGLE, "1x15");
$verticalLines->morphology(\Imagick::MORPHOLOGY_CLOSE, 1, $verticalLineKernel);

$horizontalLines->compositeimage($verticalLines, 5, 0, 0);

$im = clone $horizontal;

$horizontalLines->clear(); 
$horizontalLines->destroy();
$verticalLines->clear(); 
$verticalLines->destroy();

// Create boxes at corners
// These are at points from which I intent to create the individual grid boxes
$plusKernel = \ImagickKernel::fromBuiltIn(\Imagick::KERNEL_PLUS, "4");
$im->morphology(\Imagick::MORPHOLOGY_OPEN, 1, $plusKernel);

$squareKernel = \ImagickKernel::fromBuiltIn(\Imagick::KERNEL_SQUARE, "2");
$im->morphology(\Imagick::MORPHOLOGY_CLOSE, 1, $squareKernel);

By doing this I end up with a image with boxes which if I can get a x,y,width and height, I should be able to get the coordinates, however it misses the bottom right corner and is very messy. I'm sure there has to be a better approach.

The image is downscaled and then I'm planning to upscale the coordinates by 6 as seen at $im->resizeImage(). Is there a better way I should go about this?

Answer 1

One way to do this (assuming the lines are horizontal and vertical) in ImageMagick is to scale to one row and to one column, threshold, and filter txt: output for black pixels.

xlist=`convert cells.png -scale x1! -auto-level -threshold 27% -negate -morphology Thinning:-1 Skeleton -negate txt:- | grep "black" | cut -d, -f1`
echo "$xlist"
38
109
180
251
322
394
465
536


ylist=`convert cells.png -scale 1x! -auto-level -threshold 27% -negate -morphology Thinning:-1 Skeleton -negate txt:- | grep "black" | cut -d: -f1 | cut -d, -f2`
echo "$ylist"
45
141
256
381

The combination of all x values and all y values gives you the array of intersections.

xArr=($xlist)
yArr=($ylist)
numx=${#xArr[*]}
numy=${#yArr[*]}
pointlist=""
for ((j=0; j<numy; j++)); do
for ((i=0; i<numx; i++)); do
pointlist="$pointlist ${xArr[$i]},${yArr[$j]}"
done
done
echo "pointlist=$pointlist"
pointlist= 38,45 109,45 180,45 251,45 322,45 394,45 465,45 536,45 38,141 109,141 180,141 251,141 322,141 394,141 465,141 536,141 38,256 109,256 180,256 251,256 322,256 394,256 465,256 536,256 38,381 109,381 180,381 251,381 322,381 394,381 465,381 536,381

You can visualize by:

convert cells.png -scale x1! -scale 550x50! -auto-level -threshold 27% tmp1.png

convert cells.png -scale 1x! -scale 50x425! -auto-level -threshold 27% tmp2.png

Without the thinning, the top horizontal line is thicker than one pixel.