How can I include esi tags into main esi tag?

Question

I use esi and Varnish as caching proxy server in my project. The main block has ttl with 1 hour and several blocks into with ttl 10 minutes inside. Is it possible to include esi tags into main esi tag like described below?

main.html.twig

    <esi:include src="/?action=menu/topBar"/>
    <esi:remove>
        {% include('partials/header/topBar.html.twig') %}
    </esi:remove>

topBar.html.twig

    <esi:include src="/?action=index/extraLink" />
    <esi:remove>
        {% include('partials/header/extraLink.html.twig') %}
    </esi:remove>

Answer 1

It's definitely possible to process ESI recursively. It's all just a matter of how your VCL is configured.

I see you're using Twig to render the ESI tags. I'm going to assume you're using the Symfony framework to process those Twig templates. If so, the solution is quite simple.

ESI , Twig & Symfony

Please have a look at https://symfony.com/doc/current/http_cache/esi.html first.

Symfony has a render function for Twig that processes controller routes as an internal subrequest and returns the HTML directly. That's an alternative to your {% include %} call.

But the cool thing is that Symfony also has a render_esi function for Twig. This will take a controller route and automatically output it as an ESI tag with the necessary src attribute.

Symfony validates whether or not it sits behind a Varnish server that supports ESI. It does this using a handshaking process. If it turns out that the ESI validation didn't succeed, it will fallback to the typical render behavior and just return the HTML as an internal subrequest.

This behavior is identical to your Twig scripts where you use both <exi:include /> and <esi:remove /> as a fallback. The render_esi function handles this in a more efficient way.

If your Symfony installation doesn't support the render_esi function, please install the following Composer package: https://packagist.org/packages/symfony/twig-bridge

ESI validation handshake

To make sure Symfony can do the correct validation, the following happens behind the scenes:

• Varnish sends a Surrogate-Capability header to announce it offers ESI support
• Symfony looks for that header and if it finds it, it sends back a Surrogate-Control header to indicate to Varnish that it too can handle ESI
• Varnish looks for that header, and if it finds it, it starts processing ESI

Here's an example of these headers:

• Varnish sends this one: Surrogate-Capability: abc=ESI/1.0
• Symfony sends this one: Surrogate-Control: content="ESI/1.0"

The VCL you need to make this happen

As described on https://symfony.com/doc/current/http_cache/varnish.html, you can use the following VCL snippet to process the VCL:

sub vcl_recv {
    // Add a Surrogate-Capability header to announce ESI support.
    set req.http.Surrogate-Capability = "abc=ESI/1.0";
}

sub vcl_backend_response {
    // Check for ESI acknowledgement and remove Surrogate-Control header
    if (beresp.http.Surrogate-Control ~ "ESI/1.0") {
        unset beresp.http.Surrogate-Control;
        set beresp.do_esi = true;
    }
}

Recursive ESI

The Twig syntax you need is the following:

main.html.twig:

{% render_esi("/?action=menu/topBar") %}

topBar.html.twig:

{% render_esi("/?action=index/extraLink") %}

See https://symfony.com/doc/current/reference/twig_reference.html#render-esi for more information about render_esi.