Varnish Memory Consumption

[Chaim Keren Tzion]

Just wanted to mention in order to avoid confusion, that the settings below are actually from another installation that I have which is running release 3175 (2.0B1+). It is also having the same problems described below.



----- Original Message ----
From: Chaim Keren Tzion <chakatz at yahoo.com>
To: varnish-misc at projects.linpro.no
Sent: Wednesday, September 10, 2008 8:31:45 PM
Subject: Varnish Memory Consumption

Hi,

I'm running Varnish -r2626 on CentOS 5.2

The problem I'm having is that Varnish eats up all 16GB of RAM that I have and then fills up the 2GB Swap space. Also, one of the three Varnish machines I have is a Virtual Machine (8GB RAM +2GB Swap). The VM panics some time after the Swap fills up and becomes unresponsive. The hardware machines keep on running.

My questions are:

Regarding the memory consumption:
Is this normal?
Do I need to enlarge my Swap?
Do I have to define a storage file that is less than the RAM+Swap that I can give to Varnish, in order to ensure that they don't get filled up?
Should I switch to malloc?
Will an upgrade to a newer revision help?

Regarding the Virtual Machine:
Is there a way to protect the Virtual Machine?
It it a bad idea to use a Virtual Machine?

Thanks,
Chaim

My settings are below:

NFILES=131072
MEMLOCK=82000
VARNISH_STORAGE_SIZE=75% ( This is 91GB on the Hardware machines)
-h classic,500009
-s file,/var/lib/varnish/varnish_storage.bin,75%

user                       varnish (101)
group                      varnish (102)
default_ttl                864000 [seconds]
thread_pools               4 [pools]
thread_pool_max            4000 [threads]
thread_pool_min            1 [threads]
thread_pool_timeout        120 [seconds]
thread_pool_purge_delay    1000 [milliseconds]
thread_pool_add_threshold  2 [requests]
thread_pool_add_delay      20 [milliseconds]
thread_pool_fail_delay     200 [milliseconds]
overflow_max               100 [%]
rush_exponent              3 [requests per request]
sess_workspace             8192 [bytes]
obj_workspace              2048 [bytes]
default_grace              10
sess_timeout               5 [seconds]
pipe_timeout               864000 [seconds]
send_timeout               600 [seconds]
auto_restart               on [bool]
fetch_chunksize            128 [kilobytes]
vcl_trace                  off [bool]
listen_address             :80
listen_depth               4096 [connections]
srcaddr_hash               1049 [buckets]
srcaddr_ttl                30 [seconds]
backend_http11             off [bool]
client_http11              off [bool]
cli_timeout                5 [seconds]
ping_interval              864000 [seconds]
lru_interval               3600 [seconds]
cc_command                 "exec cc -fpic -shared -Wl,-x -o %o %s"
max_restarts               4 [restarts]
esi_syntax                 0 [restarts]
max_esi_includes           5 [restarts]
cache_vbe_conns            off [bool]
connect_timeout            400 [ms]
clock_skew                 10 [s]
prefer_ipv6                off [bool]
session_linger             0 [ms]
cli_buffer                 8192 [bytes]
log_hashstring             off [bool]
log_local_address          off [bool]
acceptor                   default (epoll, poll)
diag_bitmap                0x0 [bitmap]
err_ttl                    0 [seconds]


      
_______________________________________________
varnish-misc mailing list
varnish-misc at projects.linpro.no
http://projects.linpro.no/mailman/listinfo/varnish-misc