Varnish 2.0.3 hanging after a period of activity

[Lars Jørgensen]

Hi,

We're currently employing Varnish to cache a largish web site but are  
experiencing difficulties.

The site went live today and the increased load seemed to kill off  
Varnish every so often. Version 2.0.2 would hang with timeouts after  
about 5-7 minutes of run time. The backends are responding fine if we  
hit them directly. Restarting Varnish immediately makes it work fine  
again for another 5 minutes or so.

I upgraded to version 2.0.3 which was running fine for about an hour  
but then the same thing happened. I don't see any particular errors in  
the log, except:

     0 StatAddr     - 91.194.202.5 0 552 1 163 0 0 0 28688 0
     0 ExpPick      - 843611540 ttl
     0 VCL_call     - timeout
     0 VCL_return   - discard
     0 ExpKill      - 843611540 -30
     0 ExpPick      - 843611541 ttl
     0 VCL_call     - timeout
     0 VCL_return   - discard
     0 ExpKill      - 843611541 -30
     0 ExpPick      - 843611555 ttl
     0 VCL_call     - timeout
     0 VCL_return   - discard
     0 ExpKill      - 843611555 -30
     0 ExpPick      - 843611556 ttl
     0 VCL_call     - timeout
     0 VCL_return   - discard
     0 ExpKill      - 843611556 -30
     0 ExpPick      - 843611557 ttl
     0 VCL_call     - timeout
     0 VCL_return   - discard
     0 ExpKill      - 843611557 -30
     0 ExpPick      - 843611561 ttl
     0 VCL_call     - timeout
     0 VCL_return   - discard
     0 ExpKill      - 843611561 -30
     0 ExpPick      - 843611562 ttl
     0 VCL_call     - timeout
     0 VCL_return   - discard
     0 ExpKill      - 843611562 -30

Right now I have a cron job restarting Varnish every 30 minutes but  
that seems less than optimal.

I'm very new to Varnish and I have Googled as much as I could. Is  
there anything I should be aware of? Any kind of tuning of Varnish or  
the OS I should do?

I've disabled KeepAlives on the backend Apache-servers. That seemed to  
help a bit with 2.0.2 but I don't know if that is still an issue on  
2.0.3.

Varnish is running on Debian Etch on a virtual machine with 4GB of  
memory. The cache file is 512MB. I start it with:

/usr/local/sbin/varnishd -a 123.123.123.123:80 -T localhost:5000 -P / 
var/run/varnishd.pid -f /etc/varnish/website.vcl -s file,/var/varnish/ 
varnish_storage.bin,512M

I could post the VCL if you need it, but I don't want to spam your  
mailboxes unnecessarily.


-- 
Lars