Varnish 3.0.5 RAM/SWAP usage

[geodni at free.fr]

Hi,

sorry to reply so late, here are the results of my tests :
0 --
- system has 32GB of RAM, 8GB of SWAP and 3TB of available storage mounted on /var/cache/varnish
- allocation : "file,/var/cache/varnish/limited100GB.bin,100GB"
- 4.2 million of objects max in cache
- 30GB of RAM used and 1.4GB of swap
- varnishd child process not killed any more
- 200k requests per minute before starting eating SWAP, 95k after eating some SWAP
- oldest objects are replaced by newer ones, IO average is 53% on HDD

Today, I ran some other tests after a memory upgrade from 32GB to 64GB, here are the results :
1 --
- system has 64GB of RAM, 8GB of SWAP and 3TB of available storage mounted on /var/cache/varnish
- allocation : "file,/var/cache/varnish/limited200GB.bin,200GB"
- 8.8 million of objects max in cache
- 56GB of RAM used and 1.9GB of swap
- varnishd child process not killed any more
- starting at 250k requests per minute for 10 minutes then 200k before starting eating SWAP, 145k after eating some SWAP
- oldest objects are replaced by newer ones, IO average is 53% on HDD

2 --
- system has 64GB of RAM, 50MB of SWAP and 3TB of available storage mounted on /var/cache/varnish
- allocation : "file,/var/cache/varnish/limited300GB.bin,300GB"
- 6.8 million of objects max in cache before varnishd killed
- 60GB of RAM used and 500MB of swap
- varnishd child process was killed because running out of swap space
- starting at 250k requests per minute for 10 minutes then 200k before starting eating SWAP, 95k after eating some SWAP
- oldest objects are replaced by newer ones, IO average is 53% on HDD

Does anybody have a large platform running Varnish ? I would like to raise 200 millions of objects but it seems it needs a very large amount of RAM or I have a problem with memory leaks using Varnish 3.0.5 under FreeBSD 10...

Regards,
Denis

> De: "Paul Armstrong" <psa at otoh.org>
> Envoyé: Mercredi 6 Août 2014 17:20:38
> 
> Look at varnishstat and see if the temporary storage is large as this
> can also be restricted (but isn't by default). For my workloads I
> typically have malloc set to only 25-50% of RAM in order to avoid
> running out.
> 
> Paul
> 
> On 6 Aug 2014, at 5:19, geodni at free.fr wrote:
> 
> >> I run a similar configuration except instead of file I use
> >> 
> >> malloc,26G
> >> 
> >> Have you tried that? does it react the same?
> > 
> > Hi,
> > I tried that and the system became unstable after 3 minutes. I had
> > to reboot because of <defunct> processes appears after that. No
> > hardware failure nor specific system log. After the reboot, I made
> > another injection test with "file" storage.
> > 
> > My conclusion is that Varnish really eats 1GB per 1 million objects
> > but it also consume 22GB RAM for the rest of its activity !!!
> > What the hell is it doing with 22GB of RAM ? 80MB for SHM logs,
> > 100MB for threads ok but what about the rest ?
> > 
> > The platform I use runs at 2k requests per second for one injection
> > job. One thing I can do is limiting "file"  storage to 100GB to be
> > sure the system won't use SWAP. I am currently running this test.
> > 
> > Regards,
> > Denis