How do you calculate the angle between two normals in glsl?

Question

How do you calculate the angle between two normals in glsl? I am trying to add the fresnel effect to the outer edges of an object (combining that effect with phong shading), and I think that the angle is the only thing I am missing.

Fragment Shader:

varying vec3 N;
varying vec3 v;

void main(void) {
  v = vec3(gl_ModelViewMatrix * gl_Vertex);
  N = normalize(gl_NormalMatrix * gl_Normal);
  gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;
}

Vertex Shader:

varying vec3 N;
varying vec3 v;

void main(void) {
  vec3 L = normalize(gl_LightSource[0].position.xyz - v);
  vec3 E = normalize(-v);
  vec3 R = normalize(-reflect(L,N));

  vec4 Iamb = gl_FrontLightProduct[0].ambient
  vec4 Idiff = gl_FrontLightProduct[0].diffuse * max(dot(N,L), 0.0);
  vec4 Ispec = gl_FrontLightProduct[0].specular * pow(max(dot(R,E),0.0), gl_FrontMaterial.shininess);
  vec4 Itot = gl_FrontLightModelProduct.sceneColor + Iamb + Idiff + Ispec;

  vec3 A = //calculate the angle between the lighting direction and the normal//
  float F = 0.33 + 0.67*(1-cos(A))*(1-cos(A))*(1-cos(A))*(1-cos(A))*(1-cos(A));
  vec4 white = {1.0, 1.0, 1.0, 1.0};

  gl_FragColor = F*white + (1.0-F)*Itot;
}

varying vec3

Answer 1

From the dot product of two vectors you can get the cosine of the angle between them

cos A = DotProduct(v1, v2) / (Length(v1) * Length(v2))

Using this, you don't need to calculate the cosine when calculating F. Since your vectors are unit vectors, e.g., have length one, you can even avoid the division.

Answer 2

dot product between two vectors will return the cosine of the angle (in GLSL it's dot(a,b)). Taking arc-cosine of that will return angle in radians (in GLSL it's acos(x)).

Dot product is very cheap, arc-cosine is quite expensive.

However, Fresnel effect does not really need the angle. Just having dot result between the vectors is enough. There are many approximations for the Fresnel effect, one of the cheapest is just using the dot directly. Or squaring it (x*x), or raising to some other power.

In your shader above, it looks like you just want to raise dot to 5th power. Something like:

float oneMinusDot = 1.0 - dot(L, N);
float F = pow(oneMinusDot, 5.0);