Why does the "geometry" parameter not appear in the response from the aws rekognition detect-custom-labels command?

Question

Hello I am trying to use the DetectCustomLabels service from AWS rekognition to get the prediction of a label detecting model but I can not get a response with the geometry parameter.

I try using the aws rekognition detect-custom-labels command following the detect-custom-labels documentation

aws rekognition detect-custom-labels \
 --project-version-arn "arn:aws:rekognition:us-east-2:257838013610:project/dog_cat_test/version/dog_cat_test.2023-12-10T17.13.12/1702246396893" \
 --image '{"S3Object": {"Bucket": "dog-cat-bucket", "Name": "cats.jpeg"}}' \
 --region us-east-2 \
 --min-confidence 0

and from the reading on the documentation I understand that the response from this request is composed by an object with "Name", "Confidence" and "Geometry", but I only get the Name and Confidence parameters when I execute the command:

{
    "CustomLabels": [
        {
            "Name": "cat",
            "Confidence": 96.12000274658203
        },
        {
            "Name": "dog",
            "Confidence": 3.880000114440918
        }
    ]
}

Do you have any idea of how to get this geometry parameter on the response from this service? Or what service could help me to achieve this?

Answer 1

According to the documentation on DetectCustomLabels, you will only get that location information if it exists.

For each object that the model version detects on an image, the API returns a (CustomLabel) object in an array (CustomLabels). Each CustomLabel object provides the label name (Name), the level of confidence that the image contains the object (Confidence), and object location information, if it exists, for the label on the image (Geometry). Note that for the DetectCustomLabelsLabels operation, Polygons are not returned in the Geometry section of the response.

It also says that you will only get the bounding box if the model you are using is trained to return bounding boxes.

Custom labels are returned in an array of Custom Label objects. Each custom label represents a single object, scene, or concept found in the image. A custom label includes:

A label for the object, scene, or concept found in the image.

A bounding box for objects found in the image. The bounding box coordinates show where the object is located on the source image. The coordinate values are a ratio of the overall image size. For more information, see BoundingBox. DetectCustomLabels returns bounding boxes only if the model is trained to detect object locations.

The confidence that Amazon Rekognition Custom Labels has in the accuracy of the label and bounding box.

Can you confirm that your model supports the bounding box?