I have in a scene several ArUco markers and a green cardboard. Taking the ArUco markers as reference I want to calculate the area per-pixel of the scene. The green cardboard is an example to probe that the system works. Sample of the scene:
As observed, the markers are not always parallel to the camera, they can present arbitrary orientations. Therefore, I have to analyze these orientations to get the per-pixel area. To simplify this process, I now consider that the green patch is always parallel to the camera and all the elements are at the same distance.
With Python and OpenCV I have already detected the green patch per-pixel mask, and the ArUco marker's location, rotation and translation vectors, and angle between its normal (Z-axes) w.r.t. the Z-axes of the camera. This is calculated as in this post.
First, I thought that the per-pixel area only depends on this angle between the Z-axes of the ArUco markers and the camera Z-axes, but it didn't work.
# gt_marker_area_mm is the real size in mm of the ArUco marker
# area is the number of pixel that the ArUco marker presents
# To calculate the per-pixel area with no orientation information
area_mm_per_pixel = gt_marker_area_mm / area
# Calculate per-pixel area with the orientation information.
# The more parallel the ArUco marker normal is to the camera X-axis the bigger the per-pixel area is. It grows quadratically
# Angle radians is the angle between the ArUco marker normal and the camera Z-axes
area_mm_per_pixel_norm = area_mm_per_pixel / area_mm_per_pixel * (np.cos(angle_radians)**2)
Final area of the green patch is calculated as:
area_mm_per_pixel_norm * number_of_green_pixels
This systems works well when the ArUco marker is parallel to the camera (marker #0), but not when its orientation changes.
Any clue on how I can use the ArUco marker information to calculate the per-pixel-area?