Varnish 2.0.3 consuming excessive memory

[Michael S. Fischer]

Not that I have an answer, but I'd be curious to see the differences  
in 'pmap -x <pid>' output for the different children.

--Michael

On Apr 7, 2009, at 6:27 PM, Darryl Dixon - Winterhouse Consulting wrote:

>> Hi All,
>>
>> I have an odd problem that I have only noticed happening since  
>> moving from
>> 1.1.2 to 2.0.3 - excessive memory consumption of the varnish child
>> process. For example, I have a varnish instance with a 256MB cache
>> allocated, that is currently consuming 4.9GB of resident memory  
>> (6.5GB
>> virtual). The instance has only been running for 4 days and has  
>> only got
>> 25MB of objects in the cache.
>>
>> This is clearly excessive and is causing us some serious problems  
>> in terms
>> of memory pressure on the machine. Our VCL is largely unchanged  
>> from our
>> 1.1.2 setup to the 2.0.3 except for the obvious vcl.syntax changes,  
>> and
>> the introduction of request restarts under certain limited  
>> scenarios. Can
>> anyone shed some light?
>>
>
> One further footnote to this. I have a second varnish instance  
> running on
> the same machine which talks to different backend servers (still  
> primarily
> the same sort of content though), with the VCL only fractionally  
> different
> - it does not seem to suffer from the runaway memory consumption of  
> the
> first instance. The only difference in the VCL between the two is  
> that in
> the one with runaway memory this is present in vcl_recv():
>
> +    if (req.http.Pragma ~ ".*no-cache.*" || req.http.Cache-Control ~
> ".*no-cache.*") {
> +        purge_url(regsub(req.url, "[?].*$", ".*$"));
> +    }
> +
>
> Is there possibly something in the regsub engine being triggered  
> that is
> very expensive and would cause it to consume and hold on to large  
> amounts
> of memory?
>
> regards,
> Darryl Dixon
> Winterhouse Consulting Ltd
> http://www.winterhouseconsulting.com
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc