Only draw part of a mesh in WebGL

Question

I have a mesh that I rotate and move around. I would like to only draw part of it.

Eg:

In my WebGL game, I have a track map mesh, and I only want to show the part that is in the red circle (on the top left), the rest I don't want drawn.

Should I be looking into using a fragment shader? Or can I somehow tell the track map mesh to only draw the part that is in the circle?

Thanks!

PS: It would be great if I could fade the track map out around the circle, instead of just cutting it off. :-)

Answer 1

There are several ways to achieve this, for a simple rectangular cutout you could use scissor testing like this:

gl.enable(gl.SCISSOR_TEST);
gl.scissor(0,gl.canvas.clientHeight-256,256,256);
// draw track here
gl.disable(gl.SCISSOR_TEST);

For a circular cutout you could use a stencil buffer, draw a disk to the stencil buffer, enable stenciling and draw your track, this however requires you to setup your rendering pipeline using a framebuffer with an attached stencil buffer which may be a bit overwhelming and only gives a sharp cutout.

Using a custom fragment shader and blending in combination with the aforementioned scissor rect seems like the most sensible solution here, setup your scissor as described before, then in the fragment shader:

void main(void) {
    // do what you need to do to get the final color
    gl_FragColor.rgb = finalColor;
    gl_FragColor.a = smoothstep(128.,100.,length(min(vec2(gl_FragCoord.x,resolution.y-gl_FragCoord.y),256.)-128.));
}

What we're doing here is calculating the distance of our current pixel to the center of the scissor region(assumed to be 256x256 in the top left corner of the screen) then doing a linear fade out between 100 and 128 The result is a mask looking like this:

Note that technically we don't need the scissor test anymore, as the mask in the fragment shader masks everything outside the circular region. However keeping the scissor test makes sure we only do the fragment shader work where we need it.

Answer 2

Since you would also like to fade the trackmap in the circle area, a quick and dirty solution would be to use a fragment shader where fragments outside the circle area are discarded and the ones inside have their alpha value interpolated linearly along the distance from the center.

The fragment shader would more or less look like this

precision mediump float;

varying vec3  Color;

uniform vec2  MapPos;     //map's center position in screen pixels

//constant value to decide how big the red area is
const float circle_radius = 90.0;
const float falloff       = 1.0;

void main() 
{
    float distance = length(FragCoord.xy - MapPos.xy);
    if(distance > circle_radius) discard;  //discarding fragments outside circle area

    //alpha fades from 1 to 0, falloff decides how strong the fade is on the edge of the circle
    float alpha  = pow(1.0 - distance / 90.0, falloff);
    gl_FragColor = vec4(Color, alpha);
}

Keep in mind this is just one of the many solution you could use for this particular problem. You could do it directly inside the vertex shader by discarding vertices outside the circle area or set up a stencil buffer to achieve the same exact thing