I am currently working with an Orbbec Astra Mini depth sensor. I downloaded and installed the Astra Driver and OpenNI2 package. The depth sensor is shown in the device manager on my Windows 10 as soon as I attach it to the USB port. So far everything is fine.
My goal is to read RGB images
and the depth map
with OpenCV and show them with imshow
.
The following code compiles fine in Visual Studio 2015, but I get this error:
OpenCV Error: Assertion failed (size.width>0 && size.height>0) in cv::imshow, file C:\build\master_winpack-build-win64-vc14\opencv\modules\highgui\src\window.cpp
... so I guess that OpenCV couldn't open the device and therefore my cv::Mat keeps being empty.
I somewhere read that I need to compile OpenCV with the OpenNI flag - is that true? If so, any useful links that might help me to make some progress? Any ideas what I might have missed?
#include<iostream>
#include<algorithm>
#include<fstream>
#include<System.h>
#include<time.h>
#include<opencv2/core/core.hpp>
using namespace std;
int main(int argc, char **argv) {
cv::VideoCapture cap;
cap.open(CV_CAP_OPENNI);
cap.set(CV_CAP_OPENNI_IMAGE_GENERATOR_OUTPUT_MODE, CV_CAP_OPENNI_VGA_30HZ);
cap.set(CV_CAP_OPENNI_DEPTH_GENERATOR_REGISTRATION, 1);
cv::Mat im, dm; // rgb image, depth map
while (true)
{
if (cap.grab()) {
cap.retrieve(im, CV_CAP_OPENNI_BGR_IMAGE);
cap.retrieve(dm, CV_CAP_OPENNI_DEPTH_MAP);
} else {
cout << "ERROR: Could not grab image data." << endl;
}
if (!im.data) {
cout << "ERROR: RGB not retrieved." << endl;
}
if (!dm.data) {
cout << "ERROR: Depth map not retrieved." << endl;
}
cv::imshow("Image", im);
cv::imshow("Depth", dm);
if (cv::waitKey(1) >= 0)
break;
}
cap.release();
return 0;
}
UPDATE 19.02.18:
Issue solved by compiling the OpenCV libraries with the OpenNI flag as described here. Thank you Dmitrii!
However, the depth image is still quite weak. Any ideas where/how to adjust the parameters? Although, when running the sample applications of the Orbbec Astra SDK, the depth image is quite solid (which are working with OpenGL). So I guess the problem is somewhere within OpenCV?
Here's a screenshot of the depth map (inverted to white, quite weak resonance):
I wrote an comprehensive guide how to to compile ORB SLAM 2 (which was my preliminary goal) with OpenCV including OpenNI2 in order to use my Orbbec Astra camera as depth sensor. Hopefully someone else stumbling over this thread can use it.