Varnish 2.0.3 consuming excessive memory

[Darryl Dixon - Winterhouse Consulting]

Hi DES,

> "Darryl Dixon - Winterhouse Consulting"
> <darryl.dixon at winterhouseconsulting.com> writes:
>> +    if (req.http.Pragma ~ ".*no-cache.*" || req.http.Cache-Control ~
>> ".*no-cache.*") {
>> +        purge_url(regsub(req.url, "[?].*$", ".*$"));
>> +    }
>> +
>
> It would be interesting to see how often this condition is true...
>

Shouldn't be too often in this environment - I have watched with
varnishlog and the rate is very low...

>> Is there possibly something in the regsub engine being triggered that is
>> very expensive and would cause it to consume and hold on to large
>> amounts
>> of memory?
>
> I don't think so.  However, your code will add an entry to the ban list
> every time a request that matches the above condition is received, and
> this entry will remain on the list until every object that was in the
> cache before the entry was added has been requested at least once.
>

That sounds like a pretty good candidate then, as I can envisage that
there are possibly many items that may fit that criteria in my scenario.
However it definitely seems out of proportion to the actual cache
utilisation (only 25MB of objects in the cache, but memory growing to
multiple GB)

> If you really want no-cache to purge the requested object, you should do
> it in vcl_hit (set obj.ttl to 0 and pass).  Otherwise, you should simply
> pass directly from vcl_recv.
>

My understanding of using 'pass' in vcl_hit is that it causes the object
to be marked as 'hit for pass' and therefore is uncacheable from that
point onwards. In fact, pass'ing out of vcl_hit was what I used to do a
while ago, but it caused that exact behaviour (items would no longer be
cached by varnish) so I swapped it for purge_url...

regards,
Darryl Dixon
Winterhouse Consulting Ltd
http://www.winterhouseconsulting.com