varnish not bundling simultaneous requests?

Rich Rauenzahn rrauenza at
Fri Apr 29 03:34:11 CEST 2011

We're thinking of switching from squid primarily because squid 3.x
doesn't bundle simultaneous requests into a single backend request.
We use squid as a cache for accelerating the distribution of large
files via http off of NFS.   We have a central one at corp, and then
leaf ones at remote sites.  The lack of bundling is killing our
performance as when these files are made available, many automated
processes start simultaneously and begin consuming the new files.
Some sites also try to use axel to accelerate the download, which
means the file doesn't get cached in squid.

I just ran a test where I requested (via curl) the same resource in
parallel from varnish, and I got multiple backend requests (I checked
my apache log).

Could it be that there is a small race condition that is possible to
hit such that two backend requests get initiated if they are close
enough in time?  (my test was a curl ... & curl ... & -- if I do it by
hand, which is slower, they seem to get bundled.)

I'm hoping it is just something subtle I need to adjust in my vcl?

backend download1 {
  .host = "download1";
  .port = "80";

sub vcl_fetch {
    set beresp.ttl   = 1d;
    set beresp.grace = 1h;
    set beresp.keep  = 365d;

sub vcl_recv {
    set req.ttl   = 1d;
    set req.grace = 1h;
    set req.keep  = 365d;

sub vcl_hit {
    set obj.ttl   = 1d;
    set obj.grace = 1h;
    set obj.keep  = 365d;

More information about the varnish-misc mailing list