How to find the white pixel coordinates on Vivado HLS?

Question

I have a B&W image filled with white pixels (255). How can I obtain all the white pixel coordinates present in the image on Vivado HLS?

I'm using hls::Mat to store images.

Here's my top-level function on Vivado HLS:

 #include "top.h"
 #include <iostream>
 
void dust_detect(AXI_STREAM& input_data, AXI_STREAM& output_data, int m, int n)

    {

auto int pixel;

#pragma HLS DATAFLOW
//Create AXI streaming interfaces for the core
#pragma HLS INTERFACE axis port=input_data
#pragma HLS INTERFACE axis port=output_data
#pragma HLS INTERFACE ap_ctrl_none port=return


/************* Arrays used ***************/

     gray_IMAGE img_0;
#pragma HLS STREAM variable=img_0
    gray_IMAGE img_1;
#pragma HLS STREAM variable=img_1
    gray_IMAGE img_2;
#pragma HLS STREAM variable=img_2
    gray_IMAGE img_2a;
#pragma HLS STREAM variable=img_2a
    gray_IMAGE img_2b;
#pragma HLS STREAM variable=img_2b
    gray_IMAGE img_3;
#pragma HLS STREAM variable=img_3
    gray_IMAGE img_4;
#pragma HLS STREAM variable=img_4
    gray_IMAGE img_5;
#pragma HLS STREAM variable=img_5
    gray_IMAGE img_6;
#pragma HLS STREAM variable=img_6
    gray_IMAGE img_7;
#pragma HLS STREAM variable=img_7
    gray_IMAGE img_7a;
#pragma HLS STREAM variable=img_7a
    gray_IMAGE img_7b;
#pragma HLS STREAM variable=img_7b


    const char coefficients1[7][10] = { { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
                                        { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
                                        { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
                                        { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
                                        { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
                                        { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
                                        { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1} };

     hls::Window<7,10,char> erodewindow;
        for (int i=0;i<7;i++){
            for (int j=0;j<10;j++){
               erodewindow.val[i][j]=coefficients1[i][j];
          }
       }

 const char coefficients2[9][12] = { { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
                                     { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
                                     { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
                                     { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
                                     { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
                                     { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
                                     { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
                                     { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1},
                                     { 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}  };

    hls::Window<9,12,char> dilatewindow;
        for (int i=0;i<9;i++){
            for (int j=0;j<12;j++){
                   dilatewindow.val[i][j]=coefficients2[i][j];
                      }
                   }


hls::AXIvideo2Mat(input_data, img_0);

hls::Threshold(img_0,img_1,80,255,HLS_THRESH_BINARY);//OTSU THRESHOLDING

hls::Threshold(img_1,img_2,80,255,HLS_THRESH_BINARY_INV);//Invert the Thresholded output

hls::Duplicate(img_2,img_2a,img_2b);

hls::Erode<4,4>(img_2a,img_3,erodewindow);

hls::Dilate<6,6>(img_3,img_4,dilatewindow);

hls::Threshold(img_4,img_5,100,255,HLS_THRESH_BINARY_INV);//Invert the Dilated output

hls::Threshold(img_5,img_6,100,1,HLS_THRESH_BINARY);

hls::Mul(img_2b,img_6,img_7);

hls::Duplicate(img_7,img_7a,img_7b);

    for(m=0; m<MAX_HEIGHT; m++) {

        for(n=0; n<MAX_WIDTH; n++) {

#pragma HLS PIPELINE IT=1

            auto pixel = img_7a.read();

            if(pixel != 0)
            {
                printf("White pixel found at x: " + m + "\ty: " + n) ; // White pixel found at (x,y)
            }

                                 }
                              }

    hls::Mat2AXIvideo(img_7b,output_data);

}

I need help with the part of the code after hls::Mul that's used to find the white pixel coordinates in image img_7a.

Answer 1

The question is fairly broad, as there can be many ways to achieve you're goal depending on your setup and requirements.

However, if you're using an hls::Mat type for storing your image, the actual image is stored into a FIFO. So for identifying all the white pixels, you would be forced to scan all the pixels of the image in sequence, something like:

for (i = 0; i < img.rows; ++i) {
  for (j = 0; j < img.cols; ++j) {
#pragma HLS PIPELINE II=1
    auto pixel = img.read();
    if (pixel != 0) {
      // White pixel found: use i and j (the coordinates) or store them somewhere.
    }
    // Eventually write back the pixel into the image: img.write(pixel);
  }
}

If the image is instead stored in a buffer like registers or BRAM, then the above loop over the rows and columns of the image can be parallelized. For instance:

const int kUnrollFactor = 4;
pixel_t img[H][W];
#pragma HLS ARRAY_PARTITION variable=img block factor=kUnrollFactor dim=2 

for (i = 0; i < H; ++i) {
  for (j = 0; j < W; ++j) {
#pragma HLS PIPELINE II=1
#pragma HLS UNROLL factor=kUnrollFactor
    if (img[i][j] != 0) {
      // White pixel found: use i and j (the coordinates) or store them somewhere.
    }
  }
}

As for eventually storing your result: since there can be at most W * H white pixels, you might need a buffer of W * H * log2(max(W, H)) bits.

Edit

Based on the updated question, the last part of the code might give some problems.

Since a definitions of AXI_STREAM and gray_IMAGE are not given, the data type of the stream (and in turn of the internal streams img_*) might not be comparable with 255 (the white pixel value).

Suppose the pixel is defined as a struct, then doing pixel == 255 would simply not compile. On the other end, if it is a "flatten" RGB array like ap_uint<24> (three 8bit pixels), comparing with 255 might be meaningless (you would instead need to compare it with 0xFFFFFF).

Otherwise, if the pixel type is integer or unsigned char, then comparing it with 255 or 0 would give no problems.