503 service unavailable error

Thu Jul 9 03:31:30 CEST 2015

that interval and window on your web server is scary..... what you're
saying is 'check each web server every 10 minutes, and only fail it
after 3 failures'

next time you see the issue, look at:

varnishadm -n <varnish_name> debug.health

I'd be willing to bet that varnish is just failing the backends.  Try
running the healthcheck manually from the varnish boxes:

curl -H "Host:kiki.example.com" -v "http://10.10.10.26/healthcheck.php"

And see if you're actually getting good healthchecks.  If you're not,
then you need to look at your backends (specifically healthcheck.php)

On Wed, Jul 8, 2015 at 12:14 PM, Tim Dunphy <bluethundr at gmail.com> wrote:
> Hi guys,
>
>
>  I'm having an issue where my varnish server will stop working after a while
> of browsing around the site I'm using it with and throw a 503 server
> unavailable error.
>
> In my varnish logs I'm getting a 'no backend connection error':
>
>    10 FetchError   c no backend connection
>    10 VCL_call     c error deliver
>    10 VCL_call     c deliver deliver
>    10 TxProtocol   c HTTP/1.1
>    10 TxStatus     c 503
>    10 TxResponse   c Service Unavailable
>    10 TxHeader     c Server: Varnish
>
>
> And if I do a GET on the healthcheck from the command line on the varnish
> server, I get a 503 response from varnish:
>
> #GET http://wiki.example.com/healthcheck.php
>
> <?xml version="1.0" encoding="utf-8"?>
> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
>  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
> <html>
>   <head>
>     <title>503 Service Unavailable</title>
>   </head>
>   <body>
>     <h1>Error 503 Service Unavailable</h1>
>     <p>Service Unavailable</p>
>     <h3>Guru Meditation:</h3>
>     <p>XID: 2107225059</p>
>     <hr>
>     <p>Varnish cache server</p>
>   </body>
> </html>
>
> But if I do another GET on the healthcheck file from the varnish server to
> another apache VHOST on the same server as the wiki site that responds to
> the IP of the web server instead of the IP for the varnish server, the GET
> works:
>
> #GET http://ops1.example.com/healthcheck.php
> good
>
>
> So I'm not sure why varnish is having trouble reaching the HC file. The web
> server is a little far from the varnish server. The varnish machines are in
> NYC and the web servers are in northern Virginia.
>
> So I tried setting the timeouts in the varnish config to a really high
> number. And that was working for a while. But today I noticed that it
> stopped working. I'll have to restart the varnish service and browse the
> site for a while. Then it'll stop working again and produce the 503 error.
> It's pretty annoying!
>
> I was wondering if there might be something in my VCL I could tweak to make
> this work? Or if the fact is that the web servers are simply too far from
> varnish for this to be practical.
>
> Here's my VCL file. It's pretty basic:
>
> backend web1 {
>   .host = "10.10.10.25";
>   .port = "80";
>   .connect_timeout = 1200s;
>   .first_byte_timeout = 1200s;
>   .between_bytes_timeout = 1200s;
>   .max_connections = 70;
>   .probe = {
>   .request =
>    "GET /healthcheck.php HTTP/1.1"
>    "Host: wiki.example.com"
>    "Connection: close";
>    .interval = 10m;
>    .timeout = 60s;
>    .window = 3;
>    .threshold = 2;
>    }
> }
>
> backend web2 {
>   .host = "10.10.10.26";
>   .port = "80";
>   .connect_timeout = 1200s;
>   .first_byte_timeout = 1200s;
>   .between_bytes_timeout = 1200s;
>   .max_connections = 70;
>   .probe = {
>   .request =
>    "GET /healthcheck.php HTTP/1.1"
>    "Host: wiki.example.com"
>    "Connection: close";
>    .interval = 10m;
>    .timeout = 60s;
>    .window = 3;
>    .threshold = 2;
>    }
> }
>
> director www round-robin {
>   { .backend = web1;   }
>   { .backend = web2;  }
>  }
>
> sub vcl_recv {
>
>     if (req.url ~ "&action=submit($|/)") {
>         return (pass);
>     }
>
>     set req.backend = www;
>     return (lookup);
> }
>
> sub vcl_fetch {
>       set beresp.ttl = 3600s;
>       set beresp.grace = 4h;
>       return (deliver);
> }
>
>
> sub vcl_deliver {
>      if (obj.hits> 0) {
>       set resp.http.X-Cache = "HIT";
>      } else {
>         set resp.http.X-Cache = "MISS";
>      }
>  }
>
> Thanks,
> Tim
>
>
>
> --
> GPG me!!
>
> gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc