[Varnish] #1788: ObjIter has terrible performance profile when busyobj != NULL

Varnish varnish-bugs at varnish-cache.org
Thu Sep 10 16:28:03 CEST 2015

#1788: ObjIter has terrible performance profile when busyobj != NULL
 Reporter:  tnt    |       Type:  defect
   Status:  new    |   Priority:  normal
Milestone:         |  Component:  build
  Version:  4.0.3  |   Severity:  normal
 Keywords:         |
 The loop of ObjIter when there is a busyobj (i.e. dl from the backend
 isn't over), will do a linear scan of all the 'struct storage' for every
 chunk served to the client. This mean a N^2 complexity which is _really_

 This causes issues when the file is largish (i.e hundreds of Mbytes), the
 download time is several order of magnitude larger through varnish than it
 is hitting the backend directly.

 Especially when the client connection is about the same speed as the
 backend connection (and so busyobj is != NULL for a while)

 I tested this on 4.1-rp1 and 4.0.3.

 To reproduce, use this .vtc

 vcl 4.0;

 backend be {
         .host = "";
         .port = "5000";

 sub vcl_recv {
         return (pass);

 sub vcl_backend_response {
         set beresp.do_stream = true;

 and download like a 1G file through varnish. In my case it takes 2m30s to
 download (from and my cpu is 100% the whole time. While hitting
 the backend directly it takes 3s.

Ticket URL: <https://www.varnish-cache.org/trac/ticket/1788>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

More information about the varnish-bugs mailing list