How to draw bounding box on an image returned from server?

Question

I have a python server returning a list of bounding boxes after running OCR (like Tesseract or something)

"bbox": [{
          "x1": 223,
          "y1": 426,
          "x2": 1550,
          "y2": 1736
        }..]

Now to draw this bounding box on the client, I'm using the Q-img tag and created a div tag within it like this

 <q-img :src="imageData.src">
        <div
          style="
            position: absolute;
            background-color: rgba(0, 100, 0, 0.1);
            left: 223px;
            top: 426px;
            height: 1310px;
            width: 1327px;
            border: 0.5px solid blue;
          "
        />
      </q-img>

The problem is that this bounding box drawn on the client is way off from the actual location of the object on the image which the server returned.

Answer 1

It sounds like there's a misalignment between the coordinate systems used by your Python server for the bounding boxes and the coordinate system used by the Q-img tag in your frontend.

Check image dimension to make sure that the image dimensions (imageData.width and imageData.height) provided by the server match the actual dimensions of the image displayed by the Q-img tag. Any resizing or scaling on the client-side could offset the coordinates.

Also check if the server returns bounding box coordinates relative to the image dimensions (e.g., percentages) or in absolute pixels. Your CSS styles currently use absolute pixel values. If the server provides absolute pixel coordinates and your div tag needs relative positioning, you'll need to calculate the percentages:

  left: ((box.x1 / imageData.width) * 100) + '%', 
  top: ((box.y1 / imageData.height) * 100) + '%', 
  width: ((box.x2 - box.x1) / imageData.width) * 100) + '%', 
  height: ((box.y2 - box.y1) / imageData.height) * 100) + '%'

Log the bounding box coordinates received from the server and the computed styles of the div element in your browser's console (using console.log). Compare them to see if the mismatch becomes obvious. Use your browser's developer tools to inspect the image and the overlaying div. Check if the div tag is rendered at the expected location and if it has the correct dimensions. You're gunna need to do some investigation here.

Assuming your server returns coordinates as percentages (0.0 to 1.0), here's how you could modify the template:

<q-img :src="imageData.src">
  <div v-for="box in bboxes" :key="box.x1"
       style="
         position: absolute;
         background-color: rgba(0, 100, 0, 0.1);
         left: {{ box.x1 * 100 }}%; 
         top: {{ box.y1 * 100 }}%;
         width: {{ (box.x2 - box.x1) * 100 }}%;
         height: {{ (box.y2 - box.y1) * 100 }}%;
         border: 0.5px solid blue;
       "
  >
  </div>
</q-img>