Varnish caching issues, cache size, expires, cache resets, whatsgoing on?

[Demitrious Kelly]

Seems like you might be right.

We added 20Gb of swap space, and ran varnish with -s malloc,16G (we have 
8Gb ram.)  What started happening, though, is that we would hit between 
9Gb and 11Gb of memory usage, and varnish would spike the load on the 
box to 20 (at which point monit restarts it.) This would generally 
happen in 6-8 hours. So last night I recompiled varnish against googles 
tcmalloc.  Interestingly the memory usage has stayed under 5Gb for 12 
hours now, load have been more stable than ever, and it shows no real 
signs of quitting. This is much more the behavior we want.

Also, interestingly, the response is higher than our other server (we 
have two handling a split load, what I detailed was just one of them, 
the other is left alone while we tweak this one.)  A 1x1 pixel image 
request coming from the original (which crashes all the time) gets sent 
in 0.08x seconds, and this one with tcmalloc in 0.11x seconds. I 
definitely consider 0.03 seconds a price worth stability, but its an 
interesting observation nonetheless

We're going to run it over the weekend and see what happens. If it stays 
good we'll probably roll it out globally on all of our varnish servers.

Thanks for your advice!

Cheers!
DK

Brian Smith wrote:
> Demitrious Kelly wrote:
>> Does anybody have any insight into this behavior?  We would like to not
>> have to resort to multiple varnish servers with mickey mouse hacks like
>> staggering restarts to reduce the effect on the back end servers.
>
> Maybe a fragmentation issue?  When heaps get fragmented then it is 
> common to see the behavior you are experiencing. Try to use the 
> malloc-based storage instead of the mmap-based storage, and experiment 
> with different malloc implementations that are known to be good w.r.t. 
> fragmentation.
>
> Regards,
> Brian