Extracting View Frustum Planes (Gribb & Hartmann method)

Question

I have been grappling with the Gribb/Hartmann method of extracting the Frustum planes for some time now, with little success. I want to build a camera view-frustum to cull my scene.

I am working with column-major matrices in a right-handed coordinate system. (OpenGL style - I'm using C# and Playstation Mobile, but the math should be the same)

I want to get my planes in World-Space, so I build my frustum from the View-Projection Matrix (that's projectionMatrix * viewMatrix). The view Matrix is the inverse of the camera's World-Transform.

The problem is; regardless of what I tweak, I can't seem to get a correct frustum. I think that I may be missing something obvious.

If I "strafe" my camera left or right while still looking down the z-axis, the normals of my planes change so that they are always pointing at the origin of the scene - which makes me think that they are not in world-space...

Answer 1

The planes from a projection matrix can be extracted using the Gribb/Hartmann method as follows, (column major):

void extract_planes_from_projmat(
        const float mat[4][4],
        float left[4], float right[4],
        float bottom[4], float top[4],
        float near[4], float far[4])
{
    for (int i = 4; i--; ) { left[i]   = mat[i][3] + mat[i][0]; }
    for (int i = 4; i--; ) { right[i]  = mat[i][3] - mat[i][0]; }
    for (int i = 4; i--; ) { bottom[i] = mat[i][3] + mat[i][1]; }
    for (int i = 4; i--; ) { top[i]    = mat[i][3] - mat[i][1]; }
    for (int i = 4; i--; ) { near[i]   = mat[i][3] + mat[i][2]; }
    for (int i = 4; i--; ) { far[i]    = mat[i][3] - mat[i][2]; }
}

Where mat4 is the product of the projection matrix and the model-view matrix.

See:

• https://fgiesen.wordpress.com/2012/08/31/frustum-planes-from-the-projection-matrix/
• http://www8.cs.umu.se/kurser/5DV051/HT12/lab/plane_extraction.pdf

Note: if the matrix components aren't normalized and you require a Hessian Normal Form plane, then you will need to normalize the resulting planes.

Answer 2

The missing part:

comboMatrix = projection_matrix * Matrix4_Transpose(modelview_matrix)

Then the world-space frustum plane extraction for OpenGL is exactly as mentioned in the Gribb/Hartmann method:

p_planes[0].a = comboMatrix._41 + comboMatrix._11;
p_planes[0].b = comboMatrix._42 + comboMatrix._12;
p_planes[0].c = comboMatrix._43 + comboMatrix._13;
p_planes[0].d = comboMatrix._44 + comboMatrix._14;
// Right clipping plane
p_planes[1].a = comboMatrix._41 - comboMatrix._11;
p_planes[1].b = comboMatrix._42 - comboMatrix._12;
p_planes[1].c = comboMatrix._43 - comboMatrix._13;
p_planes[1].d = comboMatrix._44 - comboMatrix._14;
// Top clipping plane
p_planes[2].a = comboMatrix._41 - comboMatrix._21;
p_planes[2].b = comboMatrix._42 - comboMatrix._22;
p_planes[2].c = comboMatrix._43 - comboMatrix._23;
p_planes[2].d = comboMatrix._44 - comboMatrix._24;
// Bottom clipping plane
p_planes[3].a = comboMatrix._41 + comboMatrix._21;
p_planes[3].b = comboMatrix._42 + comboMatrix._22;
p_planes[3].c = comboMatrix._43 + comboMatrix._23;
p_planes[3].d = comboMatrix._44 + comboMatrix._24;
// Near clipping plane
p_planes[4].a = comboMatrix._41 + comboMatrix._31;
p_planes[4].b = comboMatrix._42 + comboMatrix._32;
p_planes[4].c = comboMatrix._43 + comboMatrix._33;
p_planes[4].d = comboMatrix._44 + comboMatrix._34;
// Far clipping plane
p_planes[5].a = comboMatrix._41 - comboMatrix._31;
p_planes[5].b = comboMatrix._42 - comboMatrix._32;
p_planes[5].c = comboMatrix._43 - comboMatrix._33;
p_planes[5].d = comboMatrix._44 - comboMatrix._34;

These planes now are in world-space and can be used to frustum cull world-space objects.

for(int i = 0; i < 6; i++)
{
    var dist = dot3(world_space_point.xyz, p_planes[i].xyz) + p_planes[i].d + sphere_radius;
    if(dist < 0) return false; // sphere culled
}