Shadows via shadowmaps and other textures - how to combine? OpenGL

Question

Good day. I draw a scene with shadows using shadow maps method (when we're rendering scene from light point of view to retrieve depth buffer, making shadow texture and projecting it on the scene, rendered from a camera point of view) As I use shadowmap texture, all other textured objects, of course, lose their texturing. But I really DO want textured scene with shadows:) I read about multitexturing, I actually tried to apply it, but failed. What exactly should I do? (I took code from OpenGl superbible) Here is the main setup procedure's code. I marked new strings (those for multitexturing) with //<====

    void SetupRC()
{

        ambientShadowAvailable = GL_TRUE;


        npotTexturesAvailable = GL_TRUE;



    glGetIntegerv(GL_MAX_TEXTURE_SIZE, &maxTexSize);

    fprintf(stdout, "Controls:\n");
    fprintf(stdout, "\tRight-click for menu\n\n");
    fprintf(stdout, "\tx/X\t\tMove +/- in x direction\n");
    fprintf(stdout, "\ty/Y\t\tMove +/- in y direction\n");
    fprintf(stdout, "\tz/Z\t\tMove +/- in z direction\n\n");
    fprintf(stdout, "\tf/F\t\tChange polygon offset factor +/-\n\n");
    fprintf(stdout, "\tq\t\tExit demo\n\n");

    // Black background
    glClearColor(0.32f, 0.44f, 0.85f, 0.5f );

    // Hidden surface removal
    glEnable(GL_DEPTH_TEST);
    glDepthFunc(GL_LEQUAL);
    glPolygonOffset(factor, 0.0f);

    // Set up some lighting state that never changes
    glShadeModel(GL_SMOOTH);
    glEnable(GL_LIGHTING);
    glEnable(GL_COLOR_MATERIAL);
    glEnable(GL_NORMALIZE);
    glEnable(GL_LIGHT0);

    // Set up some texture state that never changes
    glActiveTexture(GL_TEXTURE1); //<=====

    glGenTextures(1, &shadowTextureID);
    glBindTexture(GL_TEXTURE_2D, shadowTextureID);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_2D, GL_DEPTH_TEXTURE_MODE, GL_INTENSITY);
   // if (ambientShadowAvailable)
        glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_FAIL_VALUE_ARB, 
                        0.5f);
    glTexGeni(GL_S, GL_TEXTURE_GEN_MODE, GL_EYE_LINEAR);
    glTexGeni(GL_T, GL_TEXTURE_GEN_MODE, GL_EYE_LINEAR);
    glTexGeni(GL_R, GL_TEXTURE_GEN_MODE, GL_EYE_LINEAR);
    glTexGeni(GL_Q, GL_TEXTURE_GEN_MODE, GL_EYE_LINEAR);
    ::scene->fog->init();
    RegenerateShadowMap();
}

Here is shadowmap generation procedure:

        void RegenerateShadowMap(void)
    {
        GLfloat lightToSceneDistance, nearPlane, fieldOfView;
        GLfloat lightModelview[16], lightProjection[16];
        GLfloat sceneBoundingRadius = 200.0f; // based on objects in scene

        // Save the depth precision for where it's useful
        lightToSceneDistance = sqrt(lightPos[0] * lightPos[0] + 
                                    lightPos[1] * lightPos[1] + 
                                    lightPos[2] * lightPos[2]);
        nearPlane = lightToSceneDistance - sceneBoundingRadius;
        // Keep the scene filling the depth texture
        fieldOfView = (GLfloat)m3dRadToDeg(2.0f * atan(sceneBoundingRadius / lightToSceneDistance));

        glMatrixMode(GL_PROJECTION);
        glLoadIdentity();
        gluPerspective(fieldOfView, 1.0f, nearPlane, nearPlane + (2.0f * sceneBoundingRadius));
        glGetFloatv(GL_PROJECTION_MATRIX, lightProjection);
        // Switch to light's point of view
        glMatrixMode(GL_MODELVIEW);
        glLoadIdentity();
        gluLookAt(lightPos[0], lightPos[1], lightPos[2], 
                  0.0f, 0.0f, 0.0f, 0.0f, 1.0f, 0.0f);
        glGetFloatv(GL_MODELVIEW_MATRIX, lightModelview);
        glViewport(0, 0, shadowWidth, shadowHeight);

        // Clear the depth buffer only
        glClear(GL_DEPTH_BUFFER_BIT);

        // All we care about here is resulting depth values
        glShadeModel(GL_FLAT);
        glDisable(GL_LIGHTING);
        glDisable(GL_COLOR_MATERIAL);
        glDisable(GL_NORMALIZE);
        glActiveTexture(GL_TEXTURE0); //<=====
        glDisable(GL_TEXTURE_2D);
        glActiveTexture(GL_TEXTURE1); //<=====
        glColorMask(0, 0, 0, 0);

        // Overcome imprecision
        glEnable(GL_POLYGON_OFFSET_FILL);

        // Draw objects in the scene except base plane
        // which never shadows anything
        DrawModels(GL_FALSE);

        // Copy depth values into depth texture
        glCopyTexImage2D(GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT, 
                         0, 0, shadowWidth, shadowHeight, 0);

        // Restore normal drawing state
        glShadeModel(GL_SMOOTH);
        glEnable(GL_LIGHTING);
        glEnable(GL_COLOR_MATERIAL);
        glEnable(GL_NORMALIZE);
        glActiveTexture(GL_TEXTURE0); //<=====
        glEnable(GL_TEXTURE_2D);
glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE);
        glColorMask(1, 1, 1, 1);
        glDisable(GL_POLYGON_OFFSET_FILL);

        // Set up texture matrix for shadow map projection,
        // which will be rolled into the eye linear
        // texture coordinate generation plane equations
        M3DMatrix44f tempMatrix;
        m3dLoadIdentity44(tempMatrix);
        m3dTranslateMatrix44(tempMatrix, 0.5f, 0.5f, 0.5f);
        m3dScaleMatrix44(tempMatrix, 0.5f, 0.5f, 0.5f);
        m3dMatrixMultiply44(textureMatrix, tempMatrix, lightProjection);
        m3dMatrixMultiply44(tempMatrix, textureMatrix, lightModelview);
        // transpose to get the s, t, r, and q rows for plane equations
        m3dTransposeMatrix44(textureMatrix, tempMatrix);

    }

Scene render proc:

    void RenderScene(void)
{
    // Track camera angle
    glMatrixMode(GL_PROJECTION);
    glLoadIdentity();
    if (windowWidth > windowHeight)
    {
        GLdouble ar = (GLdouble)windowWidth / (GLdouble)windowHeight;
        glFrustum(-ar * cameraZoom, ar * cameraZoom, -cameraZoom, cameraZoom, 1.0, 1000.0);
    }
    else
    {
        GLdouble ar = (GLdouble)windowHeight / (GLdouble)windowWidth;
        glFrustum(-cameraZoom, cameraZoom, -ar * cameraZoom, ar * cameraZoom, 1.0, 1000.0);
    }

    glMatrixMode(GL_MODELVIEW);
    glLoadIdentity();
    gluLookAt(cameraPos[0], cameraPos[1], cameraPos[2], 
              0.0f, 0.0f, 0.0f, 0.0f, 1.0f, 0.0f);

    glViewport(0, 0, windowWidth, windowHeight);

    // Track light position
    glLightfv(GL_LIGHT0, GL_POSITION, lightPos);

    // Clear the window with current clearing color
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

    if (showShadowMap)
    {
        // Display shadow map for educational purposes
        glActiveTexture(GL_TEXTURE1); //<=====
        glMatrixMode(GL_PROJECTION);
        glLoadIdentity();
        glMatrixMode(GL_MODELVIEW);
        glLoadIdentity();
        glMatrixMode(GL_TEXTURE);
        glPushMatrix();
        glLoadIdentity();
        glEnable(GL_TEXTURE_2D);
        glDisable(GL_LIGHTING);
        glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_REPLACE);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_MODE, GL_NONE);
        // Show the shadowMap at its actual size relative to window
        glBegin(GL_QUADS);
            glTexCoord2f(0.0f, 0.0f);
            glVertex2f(-1.0f, -1.0f);
            glTexCoord2f(1.0f, 0.0f);
            glVertex2f(((GLfloat)shadowWidth/(GLfloat)windowWidth)*2.0f-1.0f, 
                       -1.0f);
            glTexCoord2f(1.0f, 1.0f);
            glVertex2f(((GLfloat)shadowWidth/(GLfloat)windowWidth)*2.0f-1.0f, 
                       ((GLfloat)shadowHeight/(GLfloat)windowHeight)*2.0f-1.0f);
            glTexCoord2f(0.0f, 1.0f);
            glVertex2f(-1.0f, 
                       ((GLfloat)shadowHeight/(GLfloat)windowHeight)*2.0f-1.0f);
        glEnd();
        glDisable(GL_TEXTURE_2D);
        glEnable(GL_LIGHTING);
        glPopMatrix();
        glMatrixMode(GL_PROJECTION);
        gluPerspective(45.0f, 1.0f, 1.0f, 1000.0f);
        glMatrixMode(GL_MODELVIEW);
    }
    else if (noShadows)
    {
        // Set up some simple lighting
        glLightfv(GL_LIGHT0, GL_AMBIENT, ambientLight);
        glLightfv(GL_LIGHT0, GL_DIFFUSE, diffuseLight);

        // Draw objects in the scene including base plane
        DrawModels(GL_TRUE);
    }
    else
    {
        if (!ambientShadowAvailable)
        {
            GLfloat lowAmbient[4] = {0.1f, 0.1f, 0.1f, 1.0f};
            GLfloat lowDiffuse[4] = {0.35f, 0.35f, 0.35f, 1.0f};

            // Because there is no support for an "ambient"
            // shadow compare fail value, we'll have to
            // draw an ambient pass first...
            glLightfv(GL_LIGHT0, GL_AMBIENT, lowAmbient);
            glLightfv(GL_LIGHT0, GL_DIFFUSE, lowDiffuse);

            // Draw objects in the scene, including base plane
            DrawModels(GL_TRUE);

            // Enable alpha test so that shadowed fragments are discarded
            glAlphaFunc(GL_GREATER, 0.9f);
            glEnable(GL_ALPHA_TEST);
        }
        glLightfv(GL_LIGHT0, GL_AMBIENT, ambientLight);
        glLightfv(GL_LIGHT0, GL_DIFFUSE, diffuseLight);

        // Set up shadow comparison
        glActiveTexture(GL_TEXTURE1); //<=====
        glEnable(GL_TEXTURE_2D);
        glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE);
        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_MODE, 
                        GL_COMPARE_R_TO_TEXTURE);

        // Set up the eye plane for projecting the shadow map on the scene
        glEnable(GL_TEXTURE_GEN_S);
        glEnable(GL_TEXTURE_GEN_T);
        glEnable(GL_TEXTURE_GEN_R);
        glEnable(GL_TEXTURE_GEN_Q);
        glTexGenfv(GL_S, GL_EYE_PLANE, &textureMatrix[0]);
        glTexGenfv(GL_T, GL_EYE_PLANE, &textureMatrix[4]);
        glTexGenfv(GL_R, GL_EYE_PLANE, &textureMatrix[8]);
        glTexGenfv(GL_Q, GL_EYE_PLANE, &textureMatrix[12]);

        // Draw objects in the scene, including base plane
        DrawModels(GL_TRUE);
        //glPushMatrix();
        //glScalef(1, -1, 1);
        //DrawModels(GL_TRUE);
        //glPopMatrix();
        glDisable(GL_ALPHA_TEST);
        glDisable(GL_TEXTURE_2D);
        glDisable(GL_TEXTURE_GEN_S);
        glDisable(GL_TEXTURE_GEN_T);
        glDisable(GL_TEXTURE_GEN_R);
        glDisable(GL_TEXTURE_GEN_Q);
    }

    if (glGetError() != GL_NO_ERROR)
        fprintf(stderr, "GL Error!\n");
    //glBindTexture
    // Flush drawing commands
    glutSwapBuffers();
    //RegenerateShadowMap();

}

And an example of textured object draw:

   CTeapot::CTeapot(std::string fn, float s, float iX, float iY, float iZ)
{
    this->setCoords(iX, iY, iZ);
    this->size = s;
    glActiveTexture(GL_TEXTURE0); //<=====
    try
    {
    this->texture = new C2DTexture(fn);
    }
    catch(ERR::CError err)
    {
        throw err;
    }   
    glActiveTexture(GL_TEXTURE1); //<=====
}

void CTeapot::draw()
{
    glPushMatrix();
    glTranslatef(this->coords[0], this->coords[1], this->coords[2]);
    if(this->angle[0] != 0.0f)
        glRotatef(this->angle[0], 1.0f, 0.0f, 0.0f);
    if(this->angle[1] != 0.0f)
        glRotatef(this->angle[1], 0.0f, 1.0f, 0.0f);
    if(this->angle[2] != 0.0f)
        glRotatef(this->angle[2], 0.0f, 0.0f, 1.0f);
    glScalef(this->size, this->size, this->size);
    glActiveTexture(GL_TEXTURE0); //<=====
    //glEnable(GL_TEXTURE_2D);
    glBindTexture(GL_TEXTURE_2D, this->texture->getGLTexture());
    glutSolidTeapot(this->size);
    glPopMatrix();
    glActiveTexture(GL_TEXTURE1); //<=====
    //glEnable(GL_TEXTURE_2D);
}

C2DTexture texture generaton proc:

C2DTexture::C2DTexture(std::string fn)
{
    this->filename = fn;
    this->imgTexture = auxDIBImageLoad(this->filename.c_str());
    if(this->imgTexture == NULL)
        throw ERR::CError(ERR::ERR_NOSUCHFILE, ERR::ERR_NOSUCHFILETEXT + this->filename);
    // Creating a texture
    glGenTextures(1, &this->glTexture);
    glBindTexture(GL_TEXTURE_2D, this->glTexture);
    // Setting filters
    glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_NEAREST);
    glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
    glTexImage2D(GL_TEXTURE_2D, 0, 3, this->imgTexture->sizeX, this->imgTexture->sizeY, 0, GL_RGB, GL_UNSIGNED_BYTE, this->imgTexture->data);
}

Answer 1

You tried to apply multi-texturing ? It does not show in your code. You do need to use it. One texture unit for the shadow texture, one for your diffuse map. If you tried, you should show the code with multi-texturing.

Multi-texturing is handled through glActiveTexture (and for fixed function that you seem to be using, glClientActiveTexture to handle the texture coordinates specifications).

Some pieces of advice:

• it's easier to understand exactly what you're doing if you use shaders.
• you want to map the depth texture to the texture unit 1: the setup of the texture unit for shadow mapping needs to be preceded by a glActiveTexture(GL_TEXTURE1) -- the BindTexture, the TexGen, and the texturing related Enable/Disable. Of course, you need to switch back to the texture unit 0 for the rest.
• you don't want any texturing when you draw to the depth map.
• It is faster to draw directly to the texture with the framebuffer_object extension, than to copy to it

Hope this helps.

Edit: Since you've changed quite a bit your question, let me add some pieces of advice and answers to your comments:

A single texture unit will always fetch from a single texture object. You use glActiveTexture followed by glBindTexture to specify which texture will be fetched from on that texture unit. Note that to get any texturing on that unit, you still need to call glEnable(GL_TEXTURE_2D) for that unit.

What to apply first... Well, this is where using shaders simplifies quite a lot the discussion. In general, the order of application completely depends on what fragment math you want to end up with. Let's put the following nomenclature:

• T_0 the result of the first texture fetch,
• T_1 the result of the second texture fetch.
• C_f The input color that OpenGL computed and rasterized for this fragment (You're using the fixed function lighting, that's what I'm talking about)
• C_o The final color of the fragment
• T_s the result of the shadow texture fetch,
• T_d the result of the diffuse texture fetch.

The result you'll get, with 2 texture units enabled, is something like

C_o = TexEnv1(TexEnv0(C_f,T_0), T_1)

The result you want is likely

C_o = C_f * T_s * T_d

What does that tell us ?

• to implement the multiplications, you want modulate as your TexEnv for both texture unit 0 and texture unit 1
• the order does not matter in this case (this is because the multiplication -aka modulation- is commutative)
• what I showed is pretty much shader code. A lot easier to read than TexEnv settings.

Now to get back to your problem... At this point, I hope you understand what OpenGL state you should have come draw time. However, trying to know exactly what state you actually have from reading your code is a perilous exercise at best. If you are serious about using OpenGL, I recommend either of the following:

• use an OpenGL debugger. There are a number of tools out there that will show the exact state at a specific draw call.
• build your own debugging state tracking
• dump the OpenGL state of interest at the time of the draw. OpenGL provides getter methods for every bit of its state (or almost, I won't go into the dirtiest details here), You want to do that only for debugging purposes, Getters are not guaranteed to be efficient at all).