FREENECT_DEPTH_REGISTERED has no effect with libfreenect

Question

I'm playing around with a Kinect (the original Xbox version) on the libfreenect driver (I'm on Ubuntu 12.04 by the way). I have cloned the most recent version from git and installed it manually, as per the instructions here: http://openkinect.org/wiki/Getting_Started#Ubuntu_Manual_Install

I would like to access the registered depth values. As far as I understand, the Kinect is factory calibrated, and there is a lookup-table matching depth pixels to the proper RGB pixels.

I can open the Kinect just fine and retrieve the raw 11 bit depth data. That gives me values ranging non-linearly from around 730 to 1045 for distances from 1 to 7.5 meter.

Running device->setDepthFormat(FREENECT_DEPTH_MM); makes the Kinect output distances in mm, so setDepthFormat seems to work.

Running device->setDepthFormat(FREENECT_DEPTH_REGISTERED); seems to have no effect, as only the raw depth values are output. What am I missing?

Answer 1

Both FREENECT_DEPTH_MM and FREENECT_DEPTH_REGISTERED should return depth in mm. The difference is that the latter is aligned to match the RGB video image.

The freenect_depth_format enum in libfreenect.h gives these options:

FREENECT_DEPTH_11BIT = 0, /**< 11 bit depth information in one uint16_t/pixel */
FREENECT_DEPTH_10BIT = 1, /**< 10 bit depth information in one uint16_t/pixel */
FREENECT_DEPTH_11BIT_PACKED = 2, /**< 11 bit packed depth information */
FREENECT_DEPTH_10BIT_PACKED = 3, /**< 10 bit packed depth information */
FREENECT_DEPTH_REGISTERED = 4, /**< processed depth data in mm, aligned to 640x480 RGB */
FREENECT_DEPTH_MM = 5, /**< depth to each pixel in mm, but left unaligned to RGB image */
FREENECT_DEPTH_DUMMY = 2147483647, /**< Dummy value to force enum to be 32 bits wide */

Source: the libfrenect git issues page

The registration transformation produces a depth image as if the RGB and IR cameras were physically situated in the same place, and not offset by 2.5cm. Have a peek at the source code if you're curious about the details.

Source: libfreenect source code