Cache on Varnish 4+ based on Cache Control header and not cookies

Question

I want to cache on varnish solely based on cache control header. If public or max-age is mentioned, then page should be cached irrespective of cookie, In case of no-store or no-cache cache should bypass and proper session connection should be made with cookie. Not sure what I am doing wrong here.

sub vcl_recv {
 if ( req.http.X-Force-Cache )
{
unset req.http.Cookie;
unset req.http.Cache-Control;
unset req.http.pragma;
unset req.http.ttl;
}
      if (req.http.Authorization ) {
           /* Not cacheable by default */
           return (pass);
      }

if (req.method == "POST" ) {
       #set req.hash_always_miss = true;
           return (pass);
    }

}

sub vcl_hash {

if ( req.http.X-Force-Cache )
{
hash_data(req.http.X-Force-Cache);
}
}

sub vcl_backend_response {
if ( (beresp.http.Cache-control ~ "(no-cache|no-store)"))
{
    #unset bereq.http.Cookie;
    #unset beresp.http.Set-cookie;
    unset bereq.http.X-Force-Cache;
}
elseif ( ( bereq.http.X-Force-Cache  ) )
{
 unset beresp.http.Set-cookie;
    unset bereq.http.cookie;
    unset beresp.http.cookie;
#    unset beresp.http.Cache-Control;
    unset bereq.http.X-Force-Cache;
     set beresp.ttl = 7d;
    unset beresp.http.Cache-Control;
}
else
{
    unset beresp.http.Set-cookie;
    unset bereq.http.cookie;
    unset beresp.http.cookie;
#    unset beresp.http.Cache-Control;
    set bereq.http.X-Force-Cache = "1";
     set beresp.ttl = 7d;
    unset beresp.http.Cache-Control;
 }

}


sub vcl_deliver {
    if ( req.http.X-Force-Cache )
    {
    unset req.http.Cookie;
    unset resp.http.Cookie;
    unset resp.http.Set-cookie;
    return(restart);
    }
     if (obj.hits > 0) { # Add debug header to see if it's a HIT/MISS and the number of hits, disable when not needed
        set resp.http.X-Cache = "HIT";
    } else {
        set resp.http.X-Cache = "MISS";
    }
 
    set resp.http.X-Cache-Hits = obj.hits;
    set resp.http.Mobile = req.http.CloudFront-Is-Mobile-Viewer;
    set resp.http.X-Device = req.http.X-Device;
    set resp.http.X-Force-Cache = req.http.X-Force-Cache;

   
}

I also would like to hide cache control header from user in case its cached.

Answer 1

Storing objects in the cache

This should do the trick when it comes to respecting the Cache-Control header and disregarding the Set-Cookie header.

sub vcl_backend response {
   if(beresp.http.Cache-Control ~ "private|no-cache|no-store" || beresp.ttl <= 0s) {
      set beresp.uncacheable = true;
      set beresp.ttl = 120s
   }
   return (deliver);
}

This code will store objects in cache that have a Set-Cookie header unless a no-cache, no-store, max-age=0 or s-maxage=0 appears in the Cache-Control header.

It only differs slightly from the Varnish built-in VCL behavior for vcl_backend_response.

However, the Set-Cookie header will also be stored in the cache when that happens, which seems dangerous.

Maybe you'd want to strip off the Set-Cookie header then. Here's the code to conditionally remove Set-Cookie headers:

sub vcl_backend response {
   if(beresp.http.Cache-Control ~ "private|no-cache|no-store" || beresp.ttl <= 0s) {
      set beresp.uncacheable = true;
      set beresp.ttl = 120s
   } else {
      unset beresp.http.Set-Cookie
   }
   return (deliver);
}

Serving objects from the cache

We also need to define the caching behavior of incoming requests that have a Cookie request header.

Here's the code I'd use for that:

sub vcl_recv {
    if (req.method == "PRI") {
        /* This will never happen in properly formed traffic (see: RFC7540) */
        return (synth(405));
    }
    if (!req.http.host &&
      req.esi_level == 0 &&
      req.proto ~ "^(?i)HTTP/1.1") {
        /* In HTTP/1.1, Host is required. */
        return (synth(400));
    }
    if (req.method != "GET" &&
      req.method != "HEAD" &&
      req.method != "PUT" &&
      req.method != "POST" &&
      req.method != "TRACE" &&
      req.method != "OPTIONS" &&
      req.method != "DELETE" &&
      req.method != "PATCH") {
        /* Non-RFC2616 or CONNECT which is weird. */
        return (pipe);
    }

    if (req.method != "GET" && req.method != "HEAD") {
        /* We only deal with GET and HEAD by default */
        return (pass);
    }

    if (req.http.Authorization) {
        return (pass);
    }
    return (hash);
}

The only way this differs from the Varnish built-in VCL for vcl_recv is the fact that the req.http.Cookie check is removed.

There's a lot of extra code there to prevent caching from taking place for uncacheable request methods or in case an Authorization header is sent.

However, if you don't care about any of that, you could just as well use the following code:

sub vcl_recv { 
    if (req.method == "GET" || req.method == "HEAD") {   
        return(hash);
    }
}

This will serve everything from cache for cacheable request methods and uses built-in VCL behavior for all other cases. With the measures in place in vcl_backend_response the Cache-Control: private, no-cache, no-store header will ensure uncacheable responses never end up in the cache.

Varnish's built-in VCL behavior

It's important to understand that Varnish has a built-in VCL behavior. Here's a tutorial about the various aspects of this behavior: https://www.varnish-software.com/developers/tutorials/varnish-builtin-vcl.

VCL should only extend the behavior in situations where it has a detrimental effect on the hit rate or the flexibility of your application.

Please have a look at the tutorial to understand the impact of the VCL code I'm suggesting.

All-in-one VCL code:

Here's the all-in-one VCL code you can use to get the job done. You might want to tweak it a bit:

vcl 4.1;

backend default {
    .host = "localhost";
    .port = "8080";
}

sub vcl_recv { 
    if (req.method == "GET" || req.method == "HEAD") {   
        return(hash);
    }
}
sub vcl_backend response {
   if(beresp.http.Cache-Control ~ "private|no-cache|no-store" || beresp.ttl <= 0s) {
      set beresp.uncacheable = true;
      set beresp.ttl = 120s
   } else {
      unset beresp.http.Set-Cookie
   }
   return (deliver);
}