Varnish not processing ESI for Wordpress back end

Question

Problem: It seems that Varnish (ver 3.0) is unable to process esi when the URL is called through Wordpress.

Background : Idea is to cache article pages or for that matter any static page (of Wordpres site) for a longer duration, but for certain dynamic widgets. To achieve this, we set wrote a php page that will generate the sidebar and menu page through single.php and header.php. These will also include a header called 'Set-Esi'.

Varnish will process ESI if it detects presence of the header.

Tests: We added a php page in root of the site. Lets say 'esitest.php'. Inside this page we have the header, call to the widget menus and echo for date time for testing purpose.

PHP Test page Code

Varnish VCL (under vcl_fetch, ver 3.0):

  if (req.http.host ~ "(dev.|beta.|beta1.)domain.com$") {
    ## Perform ESI TTL setting:
    if (beresp.http.Set-Esi){
      set beresp.do_esi = true;
      set beresp.ttl = 15m;
    }
    else if (req.url ~ "(esisidebar.php|esimenu.php|esimorefromcategory.php)") {
      set beresp.ttl = 3s;
    }
    else if (req.url ~ "^/wp-admin/admin-ajax.php") {
            set beresp.ttl = 3m;
    }
    else {
      set beresp.ttl = 5m; unset beresp.http.expires; unset req.http.Cookie;
    }
  }

Outcome:

1. Test php urls worked as expected. ie. http://domain.com/esitest.php
2. Wordpress urls did not process esi include. It displayed content under esi:remove tag. But ttl show 15 mins which is set if the header is present. This means that it goes through the loop in vcl that does esi processing.

Note: We use batcache with memcache storage, Nginx web server (gzip disabled).