Varnish 2.0.3 consuming excessive memory

[Darryl Dixon - Winterhouse Consulting]

> Hi All,
>
> I have an odd problem that I have only noticed happening since moving from
> 1.1.2 to 2.0.3 - excessive memory consumption of the varnish child
> process. For example, I have a varnish instance with a 256MB cache
> allocated, that is currently consuming 4.9GB of resident memory (6.5GB
> virtual). The instance has only been running for 4 days and has only got
> 25MB of objects in the cache.
>
> This is clearly excessive and is causing us some serious problems in terms
> of memory pressure on the machine. Our VCL is largely unchanged from our
> 1.1.2 setup to the 2.0.3 except for the obvious vcl.syntax changes, and
> the introduction of request restarts under certain limited scenarios. Can
> anyone shed some light?
>

One further footnote to this. I have a second varnish instance running on
the same machine which talks to different backend servers (still primarily
the same sort of content though), with the VCL only fractionally different
- it does not seem to suffer from the runaway memory consumption of the
first instance. The only difference in the VCL between the two is that in
the one with runaway memory this is present in vcl_recv():

+    if (req.http.Pragma ~ ".*no-cache.*" || req.http.Cache-Control ~
".*no-cache.*") {
+        purge_url(regsub(req.url, "[?].*$", ".*$"));
+    }
+

Is there possibly something in the regsub engine being triggered that is
very expensive and would cause it to consume and hold on to large amounts
of memory?

regards,
Darryl Dixon
Winterhouse Consulting Ltd
http://www.winterhouseconsulting.com