intermittent 503's

[Jennings, Raymond]

I can't say with any definitive response but I have seen cases where
Varnish times out before Apache finishes the response.  I see this when
there is a request that causes a heavy database transaction on the backend.
 Does looking at your backend servers provide any more details?


On Wed, Apr 16, 2014 at 8:37 AM, L Cruzero <lcruzero at gmail.com> wrote:

> Hi,  in our varnish setup(varnish-3.0.3-1.el5) using a loadbalanced really
> Healthy VIP as the only backend, we seem to be getting intermittent 503's
>
>
> stdout captured with varnishlog -c -m TxStatus:503
>
>    35 ReqStart     c 10.33.13.254 24582 1166630723
>    35 RxRequest    c POST
>    35 RxURL        c
> /index.php?option=com_jce&task=plugin&plugin=imgmanager&file=imgmanager&method=form&cid=20&6bc427c8a7981f4fe1f5ac65c1246b5f=cf6dd3cf1923c950586d0dd595c8e20b
>    35 RxProtocol   c HTTP/1.1
>    35 RxHeader     c User-Agent: BOT/0.1 (BOT for JCE)
>    35 RxHeader     c Content-Type: multipart/form-data;
> boundary=---------------------------41184676334
>    35 RxHeader     c Content-Length: 5000
>    35 RxHeader     c True-Client-IP: 202.80.119.178
>    35 RxHeader     c X-Akamai-CONFIG-LOG-DETAIL: true
>    35 RxHeader     c TE:  chunked;q=1.0
>    35 RxHeader     c Connection: TE
>    35 RxHeader     c Akamai-Origin-Hop: 2
>    35 RxHeader     c Via: 1.1 v1-akamaitech.net(ghost) (AkamaiGHost), 1.1
> akamai.net(ghost) (AkamaiGHost)
>    35 RxHeader     c X-Forwarded-For: 202.80.119.178, 114.4.39.206
>    35 RxHeader     c Host: www.somedomain.com <http://www.oregonlive.com>
>    35 RxHeader     c Cache-Control: max-age=120
>    35 RxHeader     c Connection: keep-alive
>    35 RxHeader     c X-Forwarded-For: 23.73.180.223
>    35 RxHeader     c Accept-Encoding: identity
>    35 VCL_call     c recv pass
>    35 VCL_call     c hash
>    35 Hash         c
> /index.php?option=com_jce&task=plugin&plugin=imgmanager&file=imgmanager&method=form&cid=20&6bc427c8a7981f4fe1f5ac65c1246b5f=cf6dd3cf1923c950586d0dd595c8e20b
>    35 Hash         c www.somedomain.com <http://www.oregonlive.com>
>    35 VCL_return   c hash
>    35 VCL_call     c pass pass
>    35 Backend      c 29 www_prod www_prod
>    35 FetchError   c backend write error: 104 (Connection reset by peer)
>    35 Backend      c 42 www_prod www_prod
>    35 FetchError   c backend write error: 104 (Connection reset by peer)
>    35 VCL_call     c error deliver
>    35 VCL_call     c deliver deliver
>    35 TxProtocol   c HTTP/1.1
>    35 TxStatus     c 503
>    35 TxResponse   c Service Unavailable
>    35 TxHeader     c Server: Varnish
>    35 TxHeader     c Content-Type: text/html; charset=utf-8
>    35 TxHeader     c Retry-After: 5
>    35 TxHeader     c Content-Length: 419
>    35 TxHeader     c Accept-Ranges: bytes
>    35 TxHeader     c Date: Tue, 15 Apr 2014 19:31:22 GMT
>    35 TxHeader     c X-Varnish: 1166630723
>    35 TxHeader     c Age: 0
>    35 TxHeader     c Via: 1.1 varnish
>    35 TxHeader     c Connection: close
>    35 Length       c 419
>    35 ReqEnd       c 1166630723 1397590282.467632055 1397590282.899372101
> 0.033779144 0.431685925 0.0000541
>
>
> here is super simple pass thru NO cache config :
>
>
> backend www_prod {
>   .host = "cs-****.*****.*****.net";
>   .port = "80";
>   .probe = {
>         .url = "/";
>         .timeout = 5s;
>         .interval = 1s;
>         .window = 5;
>         .threshold = 2;
>   }
> }
>
>  sub vcl_recv {
>       if (req.http.X-ADI-MISS) {
>         # Force a cache miss
>                  set req.hash_always_miss = true;
>         }
>
>         if (req.url == "/varnish-health/" || req.url ~ "^/stack-check*") {
>                 error 200 "Varnish is responding";
>                 set req.http.Connection = "close";
>   }
>
>
>     if (req.http.host ~ "^(origin|www)") {
>         set req.http.host = regsub(req.http.host, "^origin\.", "www.");
>         set req.backend = www_prod;
>         return(pass);
>
> }
>
>
> ## we tried with and w/o these conditions intermittent 503's persisted
>
> #       if (req.backend.healthy) {
> #       set req.grace = 30s;
> #   } else {
> #       set req.grace = 24h;
> #   }
>
>  }
>
>
> sub vcl_fetch {
> #            set beresp.do_esi = true;
>             set beresp.ttl = 1m;
>
> #           set beresp.grace = 24h;
> }
>
>
>
> Any helpful thoughts and or possible leads on this will be much appreciated.
>
>
> kind regards,
>
>
> -LC
>
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20140416/b05ca3c9/attachment.html>