Retrieving Channels from Kinect device using python OpenCV

Question

I put together some simple code in python to grab different channels from OpenNI devices. I built OpenCV myself with all the PrimeSense and OpenNI support enabled. The OpenNI samples work perfectly for both the Kinect sensor and PrimeSense sensor, as well as the OpenCV samples for testing OpenNI support (./cpp-example-openni_capture).

Here is the code I put together.

import cv2
import cv2.cv as cv

capture = cv2.VideoCapture(cv.CV_CAP_OPENNI)
capture.set(cv.CV_CAP_OPENNI_IMAGE_GENERATOR_OUTPUT_MODE, cv.CV_CAP_OPENNI_VGA_30HZ)

print capture.get(cv.CV_CAP_PROP_OPENNI_REGISTRATION)

while True:
    if not capture.grab():
        print "Unable to Grab Frames from camera"
        break
    okay1, depth_map = capture.retrieve(cv.CV_CAP_OPENNI_DEPTH_MAP)
    if not okay1:
        print "Unable to Retrieve Disparity Map from camera"
        break
    okay2, gray_image = capture.retrieve(cv.CV_CAP_OPENNI_GRAY_IMAGE)
    if not okay2:
        print "Unable to retrieve Gray Image from device"
        break
    cv2.imshow("depth camera", depth_map)
    cv2.imshow("rgb camera", gray_image)
    if cv2.waitKey(10) == 27:
        break
cv2.destroyAllWindows()
capture.release()

So everything runs fine, but the results that are being displayed are not the correct channels... For example if I wanted to access the gray image channel and the depth map channel, both images being displayed are depth_maps.

Yes I've tried accessing other channels and changing the OPENNI_IMAGE_GENERATOR_MODE. Unfortunately the results have stayed consistent. No matter what I try I always get the same depth channel back. depth_map-gray_image yields an all black image.

Like I said the C++ OpenCV OpenNI examples all work perfectly for both the Kinect sensor and primesense sensor. It seems like a problem with the Python modules, or I am doing something really stupid.

EDIT: Running on Ubuntu 12.04 LTS

Thanks for helping. Drew

Answer 1

Retrieve looks like this (http://docs.opencv.org/modules/highgui/doc/reading_and_writing_images_and_video.html#videocapture-retrieve ):

Python: cv2.VideoCapture.retrieve([image[, channel]]) → retval, image

So you need two values to pass to retrieve.

I'm not sure what image is supposed to be, but sending in a place holder seems to do the trick.


okay1, depth_map = capture.retrieve(0, cv.CV_CAP_OPENNI_DEPTH_MAP)
okay2, gray_image = capture.retrieve(0, cv.CV_CAP_OPENNI_GRAY_IMAGE)