Vanrish 2.1.5 eating memory, hit % decrease

[Ken Brownfield]

This is expected behavior with "-sfile" -- Varnish will use all available
RAM to cache its much larger on-disk cache component.  If you can get away
with memory-only caching, you can use "-smalloc" to utilize a specific
amount of RAM.

If you do want to use "-sfile" but are having issue with swap or memory
allocation, increasing /proc/sys/vm/min_free_kbytes (or vm.min_free_kbytes
via sysctl) to something like 131072 will stabilize performance /in my
experience/.

Hope it helps,
-- 
kb



On Thu, Apr 7, 2011 at 03:38, Jean-Francois Laurens <
jean-francois.laurens at rts.ch> wrote:

>  Hi there !
>
> I recently updated varnish from version 2.0.6 to 2.1.5 on a centos 5.4
> server.
>
> After pretty much a day running 2.1.5, the system went out of memory. The
> server has 8G of memory and we specified a file of 50Gb as varnish cache
> file.
>
> I changed some options yesterday following Kristian Lyngstol
> recommendations ( *
> http://kristianlyng.wordpress.com/2009/10/19/high-end-varnish-tuning/)*and did a restart so the system is now running.
>
> I decreased the size of the cache file to 40Gb to.
>
> Here is a top output where you can see that varnish is consuming pretty
> much all the memory available:
> top -cbn 1
> top - 11:54:24 up 625 days, 19:51,  1 user,  load average: 0.52, 0.41, 0.36
> Tasks: 110 total,   1 running, 109 sleeping,   0 stopped,   0 zombie
> Cpu(s):  0.1%us,  0.2%sy,  0.0%ni, 99.1%id,  0.5%wa,  0.0%hi,  0.1%si,
>  0.0%st
> Mem:   8177764k total,  8132712k used,    45052k free,    13100k buffers
> Swap:  4096564k total,   263324k used,  3833240k free,  7337568k cached
>
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>
>  3733 varnish   15   0 44.6g 7.1g 6.7g S  2.0 91.4   9:56.07
> /usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -f
> /etc/varnish/default.vcl -T 0.0.0.0:6082 -t 120 -w 100,3000,120 -u varnish
> -g varn
> ....
> 3732 root      15   0  106m  720  492 S  0.0  0.0   0:00.72
> /usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -f
> /etc/varnish/default.vcl -T 0.0.0.0:6082 -t 120 -w 100,3000,120 -u varnish
> -g varn
>
> Regularly it’s consuming more and more swap aswell:
> From 15mn ago:
> free -m
>              total       used       free     shared    buffers     cached
> Mem:          7986       7941         44          0         12       7163
> -/+ buffers/cache:        765       7220
> Swap:         4000        258       3742
>
> From 5 mn ago:
> free -m
>              total       used       free     shared    buffers     cached
> Mem:          7986       7940         45          0         15       7163
> -/+ buffers/cache:        761       7224
> Swap:         4000        260       3739
>
> From 1mn ago:
> free -m
>              total       used       free     shared    buffers     cached
> Mem:          7986       7940         45          0         17       7160
> -/+ buffers/cache:        763       7222
> Swap:         4000        262       3737
>
> I understood that varnish relies on the system itself for memory
> allocation.
> Could we say that the bigger you file cache size is the more varnish need
> memory ?
> Or Am I facing an issue with my vcl that could have this side effect or ??
>
> In addition to this I've seen the percentage hit decreasing from 65%
> average with 2.0.6 to 40% with 2.1.5.
> Apart from changing obj to beresp in the vcl_fetch, I had to specify
> return(deliver) or return(pass) instead of only deliver or pass.
> Would it be a reason for this ?
> Would you be able to give me pointers for me to dig a bit more ?
>
> I Attached a plot from cacti wich would show the difference between 2.0.6
> and 2.1.5 (the gap is due to the time to update the server, the trafic was
> sent to an other server ...)
>
>
> Thanks in advance for your advices !
>
> Jean-Francois Laurens
> Ingénieur Système Unix
> Resources et Développement
> Secteur Backend
> *RTS - Radio Télévision Suisse
> *Quai Ernest-Ansermet 20
> Case postale 234
> CH - 1211 Genève 8
> T +41 (0)58 236 81 63
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110407/8584f174/attachment-0003.html>