[Varnish] #1242: Varnish throws 503 without touching the healthy backend

[Varnish]

#1242: Varnish throws 503 without touching the healthy backend
-------------------+----------------------
 Reporter:  xjia   |       Type:  defect
   Status:  new    |   Priority:  normal
Milestone:         |  Component:  varnishd
  Version:  3.0.0  |   Severity:  critical
 Keywords:         |
-------------------+----------------------
 I'm running Varnish 3.0.0 (revision cbf1284) under Ubuntu 11.10. Requests
 are sent to Varnish first, then should be dispatched to another server
 running nginx 1.0.5 and MediaWiki 1.20.2.

 varnishlog shows the backend is healthy. When I curl a wiki page, which is
 accessible directly from nginx, Varnish gives 503 Service Unavailable.
 However, I don't see any request in nginx log files.

 varnishlog:
 {{{
     0 CLI          - Rd ping
     0 CLI          - Wr 200 19 PONG 1357050460 1.0
     0 Backend_health - mediawiki Still healthy 4--X-RH 5 3 5 0.202436
 0.173041 HTTP/1.1 200 OK
     4 SessionOpen  c 172.16.6.106 37505 :81
     4 ReqStart     c 172.16.6.106 37505 1372273948
     4 RxRequest    c GET
     4 RxURL        c /wiki/What_The_Fuck
     4 RxProtocol   c HTTP/1.1
     4 RxHeader     c User-Agent: curl/7.21.6 (x86_64-pc-linux-gnu)
 libcurl/7.21.6 OpenSSL/1.0.0e zlib/1.2.3.4 libidn/1.22 librtmp/2.3
     4 RxHeader     c Host: 172.16.6.105:81
     4 RxHeader     c Accept: */*
     4 VCL_call     c recv lookup
     4 VCL_call     c hash
     4 Hash         c /wiki/What_The_Fuck
     4 Hash         c 172.16.6.105:81
     4 VCL_return   c hash
     4 VCL_call     c miss fetch
     4 Backend      c 5 mediawiki mediawiki
     4 TTL          c 1372273948 RFC 120 1357050462 0 0 0 0
     4 VCL_call     c fetch deliver
     4 ObjProtocol  c HTTP/1.1
     4 ObjResponse  c Service Unavailable
     4 ObjHeader    c Server: nginx/1.0.5
     4 ObjHeader    c Date: Tue, 01 Jan 2013 14:35:12 GMT
     4 ObjHeader    c Content-Type: text/html; charset=utf-8
     4 ObjHeader    c Retry-After: 5
     4 ObjHeader    c X-Varnish: 158017276
     4 ObjHeader    c Via: 1.1 varnish
     4 VCL_call     c deliver deliver
     4 TxProtocol   c HTTP/1.1
     4 TxStatus     c 503
     4 TxResponse   c Service Unavailable
     4 TxHeader     c Server: nginx/1.0.5
     4 TxHeader     c Content-Type: text/html; charset=utf-8
     4 TxHeader     c Retry-After: 5
     4 TxHeader     c X-Varnish: 158017276
     4 TxHeader     c Via: 1.1 varnish
     4 TxHeader     c Content-Length: 418
     4 TxHeader     c Accept-Ranges: bytes
     4 TxHeader     c Date: Tue, 01 Jan 2013 14:27:41 GMT
     4 TxHeader     c X-Varnish: 1372273948
     4 TxHeader     c Age: 0
     4 TxHeader     c Via: 1.1 varnish
     4 TxHeader     c Connection: keep-alive
     4 Length       c 418
     4 ReqEnd       c 1372273948 1357050461.651694775 1357050461.656989098
 0.000372171 0.005029202 0.000265121
     4 SessionClose c EOF
     4 StatSess     c 172.16.6.106 37505 0 1 1 0 0 1 306 418
     0 CLI          - Rd ping
     0 CLI          - Wr 200 19 PONG 1357050463 1.0
     0 Backend_health - mediawiki Still healthy 4--X-RH 5 3 5 0.225957
 0.186270 HTTP/1.1 200 OK
 }}}

 I tried to increase timeouts etc. but still doesn't work:
 {{{
 backend mediawiki {
     .host = "172.16.6.106";
     .port = "80";
     .connect_timeout = 300s;
     .first_byte_timeout = 60s;
     .between_bytes_timeout = 30s;
     .probe = {
         .url = "/wiki/Main_Page";
         .interval = 5s;
         .timeout = 3s;
         .window = 5;
         .threshold = 3;
     }
 }
 }}}

-- 
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1242>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator