Varnish 2.0.3 consuming excessive memory

[Darryl Dixon - Winterhouse Consulting]

>> | My understanding of using 'pass' in vcl_hit is that it causes the
>> object
>> | to be marked as 'hit for pass' and therefore is uncacheable from that
>> | point onwards. In fact, pass'ing out of vcl_hit was what I used to do
>> a
>> | while ago, but it caused that exact behaviour (items would no longer
>> be
>> | cached by varnish) so I swapped it for purge_url...
>>
>> You can try setting obj.ttl = 0s and then restart in vcl_hit.  I believe
>> that should work.
>>
>
> Thanks Tollef - I will try this today and report back.

This seems to correct the problem - thanks Tollef!

Further to root cause of the original issue: using purge_url seems to
result inside EXP_Rearm with the objexp (object expire?) being added
unconditionally to the exp_heap binheap, and it's not clear to me where
(if ever) this heap is pruned. For example, I see in varnishlog multiple
calls to the same URL from the same client with the same headers resulting
in multiple unique entries being added to exp_heap, eg:
ExpBan       c 981217322 was banned
[...]
ExpBan       c 981217323 was banned
[...etc...]

Where, if anywhere, is exp_heap pruned for entries like this? I couldn't
quite make sense of exp_timer() to be certain of under what conditions it
actually removes things from this heap - it looks like that function is
mainly concerned with items expiring from the Expires http header or
similar...

regards,
Darryl Dixon
Winterhouse Consulting Ltd
http://www.winterhouseconsulting.com