What device/instrument/technology should I use for detecting object’s lying on a given surface?

Question

First of: Thanks for taking the time to help me with my problem. It is much appreciated :)

I am building a natural user interface. I’d like the interface to detect several (up to 40) objects lying on it. The interface should detect if the objects are moved on it’s the canvas. It is not important what the actual object on surface is

e.x. “bottle”

or what color it has – only the shape and the placement of the object is of interest

e.x. “circle” .

So far I’m using a webcam connected to my computer and Processing’s blob functionality to detect the objects on the surface of the interface (see picture 1). This has some major disadvantages to what I am trying to accomplish:

1. I do not want the user to see the camera or any alternative device because this is detracting the user’s attention. Actually the surface should be completely dark.

2. Whenever I am reaching with my hand to rearrange the objects on the interface, the blob detection gets very busy and is recognizing objects (my hand) which are not touching the canvas directly. This problem can hardly be tackled using a Kinect, because the depth functionality is not working through glass/acrylic glass – correct me if I am wrong.

3. It would be nice to install a few LEDs on the canvas controlled by an Arduino. Unfortunately, the light of the LEDs would disturb the blob detection.
4. Because of the camera’s focal length, the table needs to be unnecessarily high (60 cm / 23 inch).

Do you have any idea on an alternative device/technology to detect the objects? Would be nice if the device would work well with Processing and Arduino.

Thanks in advance! :)

Answer 1

Possibilities:

• Use Reflective tinted glass so that the surface would dark or reflective
• Illuminate the area, where you place the webcam with array of IR LED's.
• I would suggest colour based detection and contouring of the objects.
• If you are using colour based detection convert frames to HSV and CrCb colour space. These are much better for segmentation of required area while using colour based detection.
• I do recommend you to check out https://github.com/atduskgreg/opencv-processing. This interfaces Open-CV with processing, you will be getting lot functionalities of Open-CV in processing .

Answer 2

One possibility:

• Use a webcam with infrared capability (such as a security camera with built-in IR illumination). Apparently some normal webcams can be converted to IR use by removing a filter, I have no idea how common that is.
• Make the tabletop out of some material that is IR-transparent, but opaque or nearly so to visible light. (Look at the lens on most any IR remote control for an example.)

This doesn't help much with #2, unfortunately. Perhaps you can be a bit pickier about the size/shape of the blobs you recognize as being your objects?

If you only need a few distinct points of illumination for #3, you could put laser diodes under the table, out of the path of the camera - that should make a visible spot on top, if the tabletop material isn't completely opaque. If you need arbitrary positioning of the lights - perhaps a projector on the ceiling, pointing down?

Answer 3

Look into OpenCV. It's an open source computer vision project.

Answer 4

In addition to existing ideas (which are great), I'd like to suggest trying TUIO Processing.

Once you have the camera setup (with the right field of view/lens/etc. based on your physical constraints) you could probably get away with sticking TUIO markers to the bottom of your objects.

The software will pickup detect the markers and you'll differentiate the objects by ID, but also be able to get position/rotation/etc. and your hands will not be part of that.