Varnish shutting down under heavy load

[Anup Shukla]

Hi All,

This is in continuation of the previous thread "Best Practices - 
Suggestion Request"

The problem of varnish shutting down every 7-10 days still continues.
This resulted in significant amount of downtime, and hence i shifted all 
but one server to "pound".
pound has been working fine.. but the one server that is running varnish 
still kept crashing.

This is what i found in the logs
......
......
......
  536 TxHeader     c X-Varnish: 1278080177
  536 TxHeader     c Age: 4
  536 TxHeader     c Via: 1.1 varnish
  536 ReqEnd       c 1278080177 1183663532.742941000 
1183663536.720536000 0.867369000 3.489705000 0.487890000
    0 StatAddr       66.249.67.59 0 2220 340 4388 0 0 4373 1729613 349338293
    0 WorkThread     0x2ccb851c60 start
    0 CLI            Rd vcl.load boot /tmp/vcl.XXBL9f1R
    0 Error          CLI write failed (errno=32)
[Log Ends Here]

Not that i never saw the logs, but i never make varnish save the log to 
a file.
Yesterday night, i finally decided to let varnish log everything to a 
file, and this is what i saw this morning.

Do i need to change something?

I am using the rpms provided for 1.0.4 on Centos 4.

/etc/sysconfig/varnish
---------------------------------------------------------------------
NFILES=131072
VARNISH_VCL_CONF=/etc/varnish/default.vcl
VARNISH_LISTEN_ADDRESS=XXX..XXX.XXX.XXX
VARNISH_LISTEN_PORT=80
# VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
# VARNISH_ADMIN_LISTEN_PORT=6082
VARNISH_MIN_THREADS=1
VARNISH_MAX_THREADS=1000
VARNISH_THREAD_TIMEOUT=120
VARNISH_STORAGE_FILE=/cache/varnish/varnish_storage.bin
VARNISH_STORAGE_SIZE=8G
VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"
VARNISH_TTL=120

DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
             -f ${VARNISH_VCL_CONF} \
             -t ${VARNISH_TTL} \
             -w 
${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},${VARNISH_THREAD_TIMEOUT} \
             -s ${VARNISH_STORAGE}"


/etc/varnish/default.vcl
---------------------------------------------------------------------
backend default {
        set backend.host = "127.0.0.1";
        set backend.port = "8000";
}

sub vcl_recv {
        if (req.request == "GET" && req.http.cookie) {
                lookup;
        }
}

sub vcl_fetch {
        if (!obj.valid) {
                error;
        }
        if (!obj.cacheable) {
                pass;
        }
        if (resp.http.Content-Type ~ "text/html") {
                pass;
        }
        if (resp.http.Set-Cookie) {
                insert;
        }
        insert;
}


Regards
A.S