intermittent 503's

[Boris Guéry]

Le 16/04/2014 22:12, Jennings, Raymond a écrit :
> I can't say with any definitive response but I have seen cases
> where Varnish times out before Apache finishes the response.  I see
> this when there is a request that causes a heavy database
> transaction on the backend.  Does looking at your backend servers
> provide any more details?
> 
Well I don't think it is some sort of timeout, the `backend write
error: 104 (Connection reset by peer)` error is straight forward.

Still, that's probably on the backend side. Try to check your apache
log and processes through `mod_status` to make sure you're not running
out of processes/threads, whatever you use.
> 
> On Wed, Apr 16, 2014 at 8:37 AM, L Cruzero <lcruzero at gmail.com 
> <mailto:lcruzero at gmail.com>> wrote:
> 
> Hi,  in our varnish setup(varnish-3.0.3-1.el5) using a
> loadbalanced really Healthy VIP as the only backend, we seem to be
> getting intermittent 503's
> 
> 
> stdout captured with varnishlog -c -m TxStatus:503
> 
> 35 ReqStart     c 10.33.13.254 24582 1166630723 35 RxRequest    c
> POST 35 RxURL        c 
> /index.php?option=com_jce&task=plugin&plugin=imgmanager&file=imgmanager&method=form&cid=20&6bc427c8a7981f4fe1f5ac65c1246b5f=cf6dd3cf1923c950586d0dd595c8e20b
>
> 
35 RxProtocol   c HTTP/1.1
> 35 RxHeader     c User-Agent: BOT/0.1 (BOT for JCE) 35 RxHeader
> c Content-Type: multipart/form-data; 
> boundary=---------------------------41184676334 35 RxHeader     c
> Content-Length: 5000 35 RxHeader     c True-Client-IP:
> 202.80.119.178 35 RxHeader     c X-Akamai-CONFIG-LOG-DETAIL: true 
> 35 RxHeader     c TE:  chunked;q=1.0 35 RxHeader     c Connection:
> TE 35 RxHeader     c Akamai-Origin-Hop: 2 35 RxHeader     c Via:
> 1.1 v1-akamaitech.net <http://v1-akamaitech.net>(ghost)
> (AkamaiGHost), 1.1 akamai.net <http://akamai.net>(ghost)
> (AkamaiGHost) 35 RxHeader     c X-Forwarded-For: 202.80.119.178,
> 114.4.39.206 35 RxHeader     c Host: www.somedomain.com 
> <http://www.oregonlive.com> 35 RxHeader     c Cache-Control:
> max-age=120 35 RxHeader     c Connection: keep-alive 35 RxHeader
> c X-Forwarded-For: 23.73.180.223 35 RxHeader     c Accept-Encoding:
> identity 35 VCL_call     c recv pass 35 VCL_call     c hash 35 Hash
> c 
> /index.php?option=com_jce&task=plugin&plugin=imgmanager&file=imgmanager&method=form&cid=20&6bc427c8a7981f4fe1f5ac65c1246b5f=cf6dd3cf1923c950586d0dd595c8e20b
>
> 
35 Hash         c www.somedomain.com <http://www.oregonlive.com>
> 35 VCL_return   c hash 35 VCL_call     c pass pass 35 Backend
> c 29 www_prod www_prod 35 FetchError   c backend write error: 104
> (Connection reset by peer) 35 Backend      c 42 www_prod www_prod 
> 35 FetchError   c backend write error: 104 (Connection reset by
> peer) 35 VCL_call     c error deliver 35 VCL_call     c deliver
> deliver 35 TxProtocol   c HTTP/1.1 35 TxStatus     c 503 35
> TxResponse   c Service Unavailable 35 TxHeader     c Server:
> Varnish 35 TxHeader     c Content-Type: text/html; charset=utf-8 35
> TxHeader     c Retry-After: 5 35 TxHeader     c Content-Length:
> 419 35 TxHeader     c Accept-Ranges: bytes 35 TxHeader     c Date:
> Tue, 15 Apr 2014 19:31:22 GMT 35 TxHeader     c X-Varnish:
> 1166630723 35 TxHeader     c Age: 0 35 TxHeader     c Via: 1.1
> varnish 35 TxHeader     c Connection: close 35 Length       c 419 
> 35 ReqEnd       c 1166630723 1397590282.467632055 
> 1397590282.899372101 0.033779144 0.431685925 0 
> <tel:431685925%200>.0000541
> 
> 
> here is super simple pass thru NO cache config :
> 
> 
> backend www_prod { .host = "cs-****.*****.*****.net"; .port =
> "80"; .probe = { .url = "/"; .timeout = 5s; .interval = 1s; .window
> = 5; .threshold = 2; } }
> 
> sub vcl_recv { if (req.http.X-ADI-MISS) { # Force a cache miss set
> req.hash_always_miss = true; }
> 
> if (req.url == "/varnish-health/" || req.url ~ "^/stack-check*") { 
> error 200 "Varnish is responding"; set req.http.Connection =
> "close"; }
> 
> 
> if (req.http.host ~ "^(origin|www)") { set req.http.host =
> regsub(req.http.host, "^origin\.", "www."); set req.backend =
> www_prod; return(pass);
> 
> }
> 
> 
> ## we tried with and w/o these conditions intermittent 503's
> persisted
> 
> #       if (req.backend.healthy) { #       set req.grace = 30s; #
> } else { #       set req.grace = 24h; #   }
> 
> }
> 
> 
> sub vcl_fetch { #            set beresp.do_esi = true; set
> beresp.ttl = 1m;
> 
> #           set beresp.grace = 24h; }
> 
> 
> 
> Any helpful thoughts and or possible leads on this will be much
> appreciated.
> 
> 
> kind regards,
> 
> 
> -LC
> 
> 
> _______________________________________________ varnish-misc
> mailing list varnish-misc at varnish-cache.org
> <mailto:varnish-misc at varnish-cache.org> 
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
> 
> 
> 
> 
> _______________________________________________ varnish-misc
> mailing list varnish-misc at varnish-cache.org 
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
> 


-- 
Boris Guéry

mobile: +33 6 86 83 03 12
skype:  borisguery
pgp:    0x034C6265