Varnish memory usage

[Andreas Plesner Jacobsen]

On Sun, Jan 22, 2012 at 04:58:58PM +0100, Andreas Plesner Jacobsen wrote:
> > >
> > > Same rules apply for transient as for any other malloc store: TTL, LRU or
> > > ban/purge.
> > 
> > Will Varnish keep multiple generations of objects that expire, and
> > potentially wait with LRU eviction until it gets closer to the known
> > sensible maximum memory allocation (the host has 128GB RAM)?
> 
> Sorry, forgot this part: No, Transient is unbounded unless you configure it
> otherwise. It should really only be used for short-lived objects.

After looking over the code with help from DocWilco, it looks like Transient is
used in three circumstances:

1. Short-lived objects
2. vcl_error
3. When allocations in all other stores fail

It looks like you're hitting 3:
SMA.s0.c_fail       412116470473   1589097.21 Allocator failures

My guess is that you're stressing LRU and are hitting some race conditions
where one request LRUs and makes space for another object, while not actually
making space for itself. I think your only option right now is to be more
aggressive with your TTL and ensure that your main storage has a little space
free.

-- 
Andreas