Detecting the locations of circles and crosses in an image

Question

I'm new to OpenCV and was wondering if anybody could direct me to the most suitable algorithm(s) to tackle the challenge of identifying the locations of circles and crosses in images that look like the following . .

[

Sometimes there are lines connecting . .

[

They might even be hand drawn like this one . .

So far I have looked at the template matching example, but it is probably not the correct approach, and it doesn't scale the sizes of the templates to the images.

So given the following observations . . .

• The crosses and circles may overlap.
• If the diagram is in colour, the colours will be identical for crosses and identical for circles.
• Sometimes they will be joined by lines, sometimes not.
• There may be other shape symbols in the plots
• They symbols will be of similar size and shape, but might not be computer generated, so will not necessarily be identical.

Where should I begin my adventure?

Answer 1

Not an easy task.

For the colored case, you should start by separating the color planes. There is some chance that you can get the markers apart.

But for the b&w case, there is no escape, you must go deeper.

I would try to detect the grid lines first, for example using a Hough line detector, as accurately as possible. Then erase those lines.

Then try to find the crosses, which are short oblique line segments (most of the time broken by the previous operations).

The circles might be detected by a Hough circle detector, using a small range of radii.

Alternatively, a rige or edge detector can be used to get short segments and short curved arcs. You may have to add some filtering criteria to avoid the joining lines.

Answer 2

Like said, it's not a easy task. One of possible way could be machine learning. I think of cascade classifier (aka Viola Jones method) that pretty nice for detection of object. It's pretty easy to implement with openCV but it need to understand how it works and a large amount of sample.

Answer 3

You can try to use couple of ideas:

1) FFT can help to remove grid. Something like this.

2) Markers (crosses and circles) are the objects with angle of gradient, that differs from right angles. It may help to localize them.