Convert pixel value to distance

Question

I got the Orbbec camera.

I use C#.

I have the C# wrapper for their SDK.

I have the depth stream and can get images from that stream. But, how do I convert the pixel value to distance (in mms)?

I have looked around ie Googling and I have emailed the Orbbec people.

So far my code:

    private void HandleDepthFrame(
        Astra.ReaderFrame frame )
    {
        var depthFrame = frame.GetFrame<Astra.DepthFrame>();
        if ( depthBuffer.Update( depthFrame ) )
        {
            BitmapSource image = depthBuffer.ImageSource;
            dispatcher.BeginInvoke(DispatcherPriority.DataBind, new Action(() => getBitmap(image)));

            if ( frameRateCalculator.RegisterFrame() )
                RaisePropertyChanged( nameof(FramesPerSecond) );
        }
    }

    private void getBitmap(BitmapSource source)
    {

        Bitmap bmp = new Bitmap(source.PixelWidth, source.PixelHeight, System.Drawing.Imaging.PixelFormat.Format24bppRgb);
        BitmapData data = bmp.LockBits(new System.Drawing.Rectangle(System.Drawing.Point.Empty, bmp.Size), ImageLockMode.WriteOnly, System.Drawing.Imaging.PixelFormat.Format24bppRgb);
        source.CopyPixels(Int32Rect.Empty, data.Scan0, data.Height * data.Stride, data.Stride);
        bmp.UnlockBits(data);
        Image<Gray, int16> imageCV = new Image<Gray, int16>(bmp);

??how to read and trsanslate the mat data to distance? }

Thanks

Answer 1

I use the C++ setup for astra and I was able to achieve this by using some code in the demos they provide in the Guide. This code here:

int main(int argc, char** argv)
{

    while (window.isOpen())
    {
        astra_update();

        sf::Event event;
        while (window.pollEvent(event))
        {
            switch (event.type)
            {
                case sf::Event::MouseMoved:
                {
                    auto coordinateMapper = depthStream.coordinateMapper();
                    listener.update_mouse_position(window, coordinateMapper);
                }break;
            }
        }
    }
}

void update_mouse_position(sf::RenderWindow& window,
                           const astra::CoordinateMapper& coordinateMapper)
{
    const sf::Vector2i position = sf::Mouse::getPosition(window);
    const sf::Vector2u windowSize = window.getSize();

    float mouseNormX = position.x / float(windowSize.x);
    float mouseNormY = position.y / float(windowSize.y);

    mouseX_ = depthWidth_ * mouseNormX;
    mouseY_ = depthHeight_ * mouseNormY;

    if (mouseX_ >= depthWidth_ ||
        mouseY_ >= depthHeight_ ||
        mouseX_ < 0 ||
        mouseY_ < 0) { return; }

    const size_t index = (depthWidth_ * mouseY_ + mouseX_);
    const short z = depthData_[index];

    coordinateMapper.convert_depth_to_world(float(mouseX_),
                                            float(mouseY_),
                                            float(z),
                                            mouseWorldX_,
                                            mouseWorldY_,
                                            mouseWorldZ_);
}

So in this demo you can actually see the values in depth being showned and using this sample code its possible to manipulate them to our favor. Hope this helps.