Vanrish 2.1.5 eating memory, hit % decrease

[Jean-Francois Laurens]

Hi there !

I recently updated varnish from version 2.0.6 to 2.1.5 on a centos 5.4
server.

After pretty much a day running 2.1.5, the system went out of memory. The
server has 8G of memory and we specified a file of 50Gb as varnish cache
file.

I changed some options yesterday following Kristian Lyngstol recommendations
( http://kristianlyng.wordpress.com/2009/10/19/high-end-varnish-tuning/) and
did a restart so the system is now running.

I decreased the size of the cache file to 40Gb to.

Here is a top output where you can see that varnish is consuming pretty much
all the memory available:
top -cbn 1
top - 11:54:24 up 625 days, 19:51,  1 user,  load average: 0.52, 0.41, 0.36
Tasks: 110 total,   1 running, 109 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.1%us,  0.2%sy,  0.0%ni, 99.1%id,  0.5%wa,  0.0%hi,  0.1%si,
0.0%st
Mem:   8177764k total,  8132712k used,    45052k free,    13100k buffers
Swap:  4096564k total,   263324k used,  3833240k free,  7337568k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 3733 varnish   15   0 44.6g 7.1g 6.7g S  2.0 91.4   9:56.07
/usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -f
/etc/varnish/default.vcl -T 0.0.0.0:6082 -t 120 -w 100,3000,120 -u varnish
-g varn
....
3732 root      15   0  106m  720  492 S  0.0  0.0   0:00.72
/usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -f
/etc/varnish/default.vcl -T 0.0.0.0:6082 -t 120 -w 100,3000,120 -u varnish
-g varn

Regularly it¹s consuming more and more swap aswell:
>From 15mn ago:
free -m
             total       used       free     shared    buffers     cached
Mem:          7986       7941         44          0         12       7163
-/+ buffers/cache:        765       7220
Swap:         4000        258       3742

>From 5 mn ago:
free -m
             total       used       free     shared    buffers     cached
Mem:          7986       7940         45          0         15       7163
-/+ buffers/cache:        761       7224
Swap:         4000        260       3739

>From 1mn ago:
free -m
             total       used       free     shared    buffers     cached
Mem:          7986       7940         45          0         17       7160
-/+ buffers/cache:        763       7222
Swap:         4000        262       3737

I understood that varnish relies on the system itself for memory allocation.
Could we say that the bigger you file cache size is the more varnish need
memory ?
Or Am I facing an issue with my vcl that could have this side effect or ??

In addition to this I've seen the percentage hit decreasing from 65% average
with 2.0.6 to 40% with 2.1.5.
Apart from changing obj to beresp in the vcl_fetch, I had to specify
return(deliver) or return(pass) instead of only deliver or pass.
Would it be a reason for this ?
Would you be able to give me pointers for me to dig a bit more ?

I Attached a plot from cacti wich would show the difference between 2.0.6
and 2.1.5 (the gap is due to the time to update the server, the trafic was
sent to an other server ...)


Thanks in advance for your advices !

Jean-Francois Laurens
Ingénieur Système Unix
Resources et Développement
Secteur Backend
RTS - Radio Télévision Suisse
Quai Ernest-Ansermet 20
Case postale 234   
CH - 1211 Genève 8
T +41 (0)58 236 81 63


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110407/f8521454/attachment-0003.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graph_vanrish01.png
Type: application/octet-stream
Size: 27804 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110407/f8521454/attachment-0003.obj>