503s and FetchError - V3.0.7

[Vlad Rusu]

Good day everyone, 

I am trying to get to the bottom of the 503s sporadically logged by our Varnish servers. During monitoring, it all comes down to monitoring the state of my backends ++ varnishlog.

Most of the 503s come with a FetchError “backend write error: 0 (Success)” — and I can’t seem to find anything on the internet to describe what this might mean.

—> FetchError   c backend write error: 0 (Success) <—

Also, none of the relevant counters in varnishstat are showing any increase (i.e. backend_unhealthy, backend_busy, backend_fail, fetch_failed..)

For that same request, looking at ReqEnd, it doesn’t look to me like any timeout was reached (like connect_timeout, first_byte_timeout, between_bytes_timeout, ..)

—> ReqEnd       c 221990096 1444847550.110866785 1444847550.116131306 0.000048876 0.000905752 0.004358768 <—


Any chance we could get a list of possible FetchError messages and what they might mean — at least something to put us on a right track? Would surely help knowing where to start looking, especially when everything seems fine.

Also, in this FetcError scenario, is it expected to see retries from Varnish? If so, why would Varnish only retry some of the backends in a director and not all of them (or the .retries value)?

   44 VCL_return   c hash
   44 VCL_call     c pass pass
   44 Backend      c 20 drpau_ssl_director drpau34ssl
   44 FetchError   c backend write error: 0 (Success)
   44 Backend      c 51 drpau_ssl_director drpau31ssl
   44 FetchError   c backend write error: 0 (Success)
   44 VCL_call     c error deliver
   44 VCL_call     c deliver deliver
   44 TxProtocol   c HTTP/1.1
   44 TxStatus     c 503
   44 TxResponse   c Service Unavailable

Should these be reflected in varnishstat’s “backend_retry”?

Thank you guys!
Vlad Rusu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20151014/6a49d4b2/attachment.html>