So I have been able to triangulate keypoints from pose estimation of two camera and able to get 3D position of joints of a person, now I want to place a shelf in the space and seen by 2 camera views, the position of the shelf is fixed forever in the two views - It always stay fixed in 3D space. I want to be able to model the shelf (or at least shelf surface)'s position in 3D space so that when the hand joint is close I can detect. I have found two potential ways but I do not yet know which way to go and how:
I will create a point cloud of the shelf in 3D space ? Probably I will manually determine position of a bounding box around the object in one view, use ORB to find correspondences of every pixel within the bounding box in the other view and triangulate everything => A point cloud in 3D space - Only need to be done once after the cameras are calibrated because the object never move
I will find like four points corresponding to the corners of the object shelf, find its correspondences in the other view, triangulate and get the position of those corners in 3D space, then => I will fit a plane to those points, and then => I will create 4 lines using the points to create bounding lines of the object in 3D space => Then I will create 4 planes perpendicular to the object plane, and each of the new plane will have to pass through exactly one of the 4 bounding lines => with this I am able to "clip" the object plane => The way I am going to determine if a hand joint is near the shelf is going to be: Determine using the object plane normal vector if the hand joint is near the plane -> Determine if the hand joint is in the "box" created by the "4 clipping planes", this way I am going to be able to determine exactly when a hand is near the shelf in 3D or not Demonstrating image - For demonstration purpose, lets say my object plane is the plane containing "point B'" in the image, and the 4 lines on that planes or the bounding lines created by 4 corner points of the shelf object, and then each line has a plane passing through it and perpendicular to the object plane, then you can see i have created a sort of "box/tunnel" in 3D like what i said above I would love to hear some advices, thank you
