Converting In32Rect coordinates to Span<BGRA> using Azure Kinect DK

Question

I've been working through the production of a series of tutorials using the new Azure Kinect DK and I've stumbled on something that's got me stumped.

It's a WPF app with an MvvM pattern that takes the output from the Kinect and has a combobox to allow the user to select the output type from various options.

The most recent option I'm working through is using Brand Recognition using azure custom vision AI. I've trained a simple model with multiple branded soft drinks and it correctly detects the brand and gives me a bounding box which references % of the original image for the position.

I've got the colour camera output as pixels in a Span using the following code:

<BGRA> colourBuffer = capture.Color.GetPixels<BGRA>().Span;

Span<BGRA> outputBuffer = outputImage.GetPixels<BGRA>().Span;

My goal is to then shade the pixels of the output from the colour camera which I'm already doing successfully for body tracking: Successfully shading pictures

I've got my brand predictions from the custom vision AI which come as bounding boxes represented as percentages of the original image. I'm converting these to Int32Rects for easier use with the aspect ratio of the colour camera output (1920x1080).

My problem is that when I shade the pixels, the span doesn't correspond to the pixel I'm shading. The whole code is here: https://github.com/craiggilchrist/mancavecoding-kinectdk/blob/feature/tutorial-3/src/Part%201%20-%20Connecting/KinectViewModel.cs but the particularly important part is this:

foreach (var prediction in _predictions)
{
    // Pixels to colour will start at the top left pixel and finish after the width plus height has been iterated.
    var bbX = (int)Math.Round(prediction.BoundingBox.Left * _colourWidth);
    var bbX2 = bbX + ((int)Math.Round(prediction.BoundingBox.Width * _colourWidth));

    var bbY = (int)Math.Round(prediction.BoundingBox.Top * _colourHeight);
    var bbY2 = bbY + ((int)Math.Round(prediction.BoundingBox.Height * _colourHeight));

    var region = new Int32Rect(
        (int)(capture.Color.WidthPixels * prediction.BoundingBox.Left),
        (int)(capture.Color.HeightPixels * prediction.BoundingBox.Top),
        (int)(capture.Color.WidthPixels * prediction.BoundingBox.Width),
        (int)(capture.Color.HeightPixels * prediction.BoundingBox.Height));

        for (int x = region.X; x < region.X + region.Width; x++)
        {
            for (int y = region.Y; y < region.Y + region.Height; y++)
            {
                outputBuffer[(x * y)].R = 255;
            }
        }
    }

This results in the following pixels being shaded red: Badly shaded pixels

I can't work out how to properly stride through the contiguous memory and tie it back to the rectangles that I need shading.

Can anyone help?

Answer 1

Turns out I was just being stupid with my for loop. The correct for loop should have been:

for (int y = region.Y; y < region.Y + region.Height; y++)
{
    for (int x = region.X; x < region.X + region.Width; x++)
    {
        var index = (y * _colourWidth) + x;
        outputBuffer[index].R = 255;
    }
}