How to get world coordinates?

Question

I have a camera with width and height resolution and camera parameters:

[[position_x, position_y, position_z], [focal_point_x, focal_point_y, focal_point_z], [view_up_x, view_up_y, view_up_z]]

Say I was given a point of the camera image plane with coordinates x,y ( 0,0 top left corner width-1, height-1 bottom right corner).

How to get world coordinates of my point x,y from such Camera, Image data?

Answer 1

I'll try to bring some highlights that I found in a old book (reference at end of this answer).

First of all, you can't map directly z-axis of the world coordinate from your (x,y) image coordinate, because it's a one to many problem: one pixel point can be at any z-axis line point. However there's a lot of studies and algorithm about estimating this z-axis with just one image perspective and others using two image perspectives (stereo vision).

But if you already have the word coordinate Z axis, you can compute the the X and Y using the above formula.

where:

1. (x,y) = image coordinates of some point
2. (X0,Y0,Z0) = camera word coordinates
3. (X,Y,Z) = word coordinates of some point
4. α = angle between x and X axis
5. θ = angle between z and Z axis
6. λ = focal lenght

I'm not an expert in robotics, but maybe this can clarify your path. I use the images from below book (page 313).

Reference book: FU, K. S.; GONZALEZ, R. C.; LEE, C. S. G.. Robotics: control, sensing, vision and intelligence. [s. L.]: Mcgraw - Hill, 1987.