Failure cenarios?

[James Quacinella]

Gaute Amundsen wrote:
> On Tuesday 03 July 2007 11:27, Dag-Erling Smørgrav wrote:
>   
>>
>> I would recommend retrieving a page (or a set of pages).  Simply
>> checking the pid won't help you if Varnish has gone off into la-la land,
>> or been SIGSTOPped or something.
>>
>> DES
>>     
> Not what I _wanted_ to hear, but what I expected i guess :)
>   

I use monit for monitoring programs. Here is a snippet I had used when 
monitoring a varnish install (too bad it never went into production; 
change values to you liking / environment):

##
## Check Varnishd
check process varnishd with pidfile /var/run/varnishd.pid
   start program = "/etc/init.d/varnishd start"
   stop program  = "/etc/init.d/varnishd stop"
   if cpu > 60% for 2 cycles then alert
   if cpu > 80% for 5 cycles then alert
   if children > 50 then alert
   if loadavg(5min) greater than 5 for 2 cycles then alert
   if 3 restarts within 3 cycles then timeout
   if failed host ipaddy port 80 type tcp then restart
   if failed host ipaddy port 8080 type tcp send "ping\r\n" then restart

Monit also allows you to check the response using a regex, though I 
never got it to work. Check the manual at 
http://www.tildeslash.com/monit/doc/manual.php#connection_testing

Also, maybe you can use swatch to monitor its log file for nasty things 
and automatically restart it / email you when it happens?

-- james