expired objects are not removed quickly enough

[Lei Wang]

Varnish is used in one of our highest traffic API cache internally. It all
worked fine for quite a long time. But recently I noticed one strange
behavior that the cached objects number continue up (to about 50M objects)
until Varnish ran out memory (configured to use 128GB per server). The only
way to recover is restart Varnish. But few days late the same thing
happened again.

I noticed that maximum cache_miss+n_expired is quite stable at about 875
HPS no matter what is the request traffic. So I guess that if average
cache_miss is more than half the 875 HPS, then the total objects number
will keep added up since the expired objects cannot be removed quickly
enough to free the space.

Questions:
1. Is there any Varnish internal limit for cache_miss+n_expired. Can it be
adjusted with run time value?
2. If the hard limit is there and cannot be adjusted, how do I change other
configuration to avoid this?


Varnish version below:
-logbash-3.2$ varnishd -V
varnishd (varnish-3.0.4 revision 9f83e8f)
Copyright (c) 2006 Verdens Gang AS
Copyright (c) 2006-2011 Varnish Software AS


Thanks,

Lei
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20151023/d7270f14/attachment.html>