How to add alpha value in blur shader

Question

In the following blur shader currently the output image has alpha value of 1.0 , but i want the alpha to be dependent on the blur of the image.

The shader code has been taken from the following question.

OpenGL es 2.0 Gaussian blur on triangle

The portion of image which does not show the blur should have alpha value of 0.

I have tried many options but i really dont understand how can we add the alpha values to the final image.

#version 330 core
out vec4 FragColor;
in vec2 pos;

// texture sampler
uniform sampler2D u_texture;
uniform vec2      u_textureSize;
uniform float     u_sigma;
uniform float     u_radius;
uniform vec2      u_dir;

float CalcGauss( float x, float sigma)
{
   if( sigma <= 0.0 )
    return 0.0;
return exp( -(x * x) / ( 2.0 * sigma)) / ( 2.0 * 3.14157 * sigma);

}

void main()
{
    vec2 texC     = pos.st * 0.5 + 0.5;
    vec4 texCol   = texture2D( u_texture, texC );
    vec4 gaussCol = vec4( texCol.rgb, 1.0 );
    vec2 step     = u_dir / u_textureSize;
    float weight;
    for ( int i = 1; i <= 16; ++ i )
    {
        weight = CalcGauss( float(i) / 16.0, u_sigma * 0.5 );
        if ( weight < 1.0/255.0 )
            break;
        texCol    = texture2D( u_texture, texC + u_radius * step * float(i) );
        gaussCol += vec4( texCol.rgb * weight, weight );
        texCol    = texture2D( u_texture, texC - u_radius * step * float(i) );
        gaussCol += vec4( texCol.rgb * weight, weight );
    }
    
   gaussCol.rgb = clamp( gaussCol.rgb / gaussCol.w, 0.0, 1.0 );
   FragColor = vec4( gaussCol.rgb , 1.0 );

}

i have changed the shader this is the new code.

void main()
{
    vec2 texC     = pos.st * 0.5 + 0.5;
    vec4 texCol   = texture2D( u_texture, texC );
    vec4 gaussCol = vec4(texCol);
    vec2 step     = u_dir / u_textureSize;
    float weight;
    for ( int i = 1; i <= 64; ++ i )
    {
        weight = CalcGauss( float(i) / 64, u_sigma * 0.5 );
        if ( weight < 1.0 / 255.0 )
            break;
        texCol    = texture2D( u_texture, texC + u_radius * step * float(i) );
        gaussCol += vec4( texCol.rgb * weight ,  weight);
        texCol    = texture2D( u_texture, texC - u_radius * step * float(i) );
        gaussCol += vec4( texCol.rgb * weight ,  weight);
    }
    
    gaussCol.rgba = clamp(gaussCol.rgba, 0.0, 1.0);  
    FragColor =   vec4(gaussCol.rgb , gaussCol.a);

}

The following image is the new result.

Answer 1

You have copied the shader form a WebGL. WebGL by default uses premultiplied alpha, but your desktop application does not. See Straight versus premultiplied. In compare to the WebGL shader you have to skip the division of the weighted sum of the color channels by the weighted sum of the alpha channel:

gaussCol.rgb = clamp( gaussCol.rgb / gaussCol.w, 0.0, 1.0 );

gaussCol.rgb = clamp(gaussCol.rgb, 0.0, 1.0);

---

If the back ground is cleared with an alpha channel of 0.0 (e.g. glClearColor(0.0, 0.0, 0.0, 0.0), which is default, then you can "blur" the alpha channel, too.

Initialize gaussCol with all the 4 channels of the texture:

vec4 gaussCol = texCol.rgba;

Take into account the final alpha channel:

gaussCol.rgba = clamp(gaussCol.rgba, 0.0, 1.0);
FragColor = gaussCol;

respectively

FragColor = clamp(gaussCol, 0.0, 1.0);

---

If you want that the alpha channel is either 0 or 1, then you have to set the channel dependent on the alpha value:

gaussCol = clamp(gaussCol, 0.0, 1.0);
FragColor = vec4(gaussCol.rgb, gaussCol > 0.001 ? 1.0 : 0.0);