[Varnish] #1612: Bug with Transfer-Encoding and zero-byte chunks (and keep-alive)

[Varnish]

#1612: Bug with Transfer-Encoding and zero-byte chunks (and keep-alive)
----------------------+----------------------
 Reporter:  zviratko  |       Type:  defect
   Status:  new       |   Priority:  normal
Milestone:            |  Component:  varnishd
  Version:  4.0.1     |   Severity:  normal
 Keywords:            |
----------------------+----------------------
 Varnish should not enable keep-alive unless the response is either chunked
 or has a Content-Length - if neither condition is met, there is no way for
 the client to tell if it got the data.

 What I see

 1) client makes a request:

 -   ReqMethod      GET
 -   ReqURL         /aden_prace/ad/86e3d6a0-239f-4cef-a52a-
 f50978c3fd95/orders/item
 -   ReqProtocol    HTTP/1.1
 -   ReqHeader      Host: Aden

 2) varnish makes a request:

 -   BereqMethod    GET
 -   BereqURL       /aden_prace/ad/86e3d6a0-239f-4cef-a52a-
 f50978c3fd95/orders/item
 -   BereqProtocol  HTTP/1.1
 -   BereqHeader    Host: Aden

 (so far so good)

 3) varnish gets a reply:

  BerespProtocol HTTP/1.1
 -   BerespStatus   200
 -   BerespReason   OK
 -   BerespHeader   Server: Apache-Coyote/1.1
 -   BerespHeader   Cache-Control: no-cache, no-transform
 -   BerespHeader   Content-Type: text/plain
 -   BerespHeader   Transfer-Encoding: chunked
 -   BerespHeader   Date: Mon, 20 Oct 2014 10:55:44 GMT
 -   TTL            RFC 30 -1 -1 1413802545 1413802545 1413802544 0 0
 -   VCL_call       BACKEND_RESPONSE
 -   BerespHeader   X-BE: aden01
 -   TTL            VCL 120 10 0 1413802545
 -   VCL_return     deliver
 -   Storage        malloc Transient
 -   ObjProtocol    HTTP/1.1
 -   ObjStatus      200
 -   ObjReason      OK
 -   ObjHeader      Server: Apache-Coyote/1.1
 -   ObjHeader      Cache-Control: no-cache, no-transform
 -   ObjHeader      Content-Type: text/plain
 -   ObjHeader      Date: Mon, 20 Oct 2014 10:55:44 GMT
 -   ObjHeader      X-BE: aden01
 -   Fetch_Body     2 chunked stream
 -   BackendReuse   47 aden01(10.0.36.31,,8080)
 -   Timestamp      BerespBody: 1413802544.791182 0.007085 0.000257
 -   Length         0

 emulated with telnet to see the body:

 GET /aden_prace/ad/86e3d6a0-239f-4cef-a52a-f50978c3fd95/orders/item
 HTTP/1.1
 Host: aden

 HTTP/1.1 200 OK
 Server: Apache-Coyote/1.1
 Cache-Control: no-cache, no-transform
 Content-Type: text/plain
 Transfer-Encoding: chunked
 Date: Mon, 20 Oct 2014 13:32:24 GMT

 0

 4) varnish sends a response, note that there is no Content-Length or T-E:
 Chunked:

  RespProtocol   HTTP/1.1
 -   RespStatus     200
 -   RespReason     OK
 -   RespHeader     Server: Apache-Coyote/1.1
 -   RespHeader     Cache-Control: no-cache, no-transform
 -   RespHeader     Content-Type: text/plain
 -   RespHeader     Date: Mon, 20 Oct 2014 10:55:44 GMT
 -   RespHeader     X-BE: aden01
 -   RespHeader     X-Varnish: 1442041
 -   RespHeader     Age: 0
 -   RespHeader     Via: 1.1 varnish-v4
 -   VCL_call       DELIVER
 -   RespHeader     X-Vokurka: not found!
 -   RespHeader     X-Server: Skrutener Zviratko 31.5
 -   VCL_return     deliver
 -   Timestamp      Process: 1413802544.791202 1.451451 0.000036
 -   Debug          "RES_MODE 2"
 -   RespHeader     Connection: keep-alive
 -   RespHeader     Accept-Ranges: bytes
 -   Timestamp      Resp: 1413802544.791260 1.451510 0.000058
 -   Debug          "XXX REF 1"
 -   ReqAcct        92 0 92 315 0 315
 -   End

 How this looks with telnet:

 GET /aden_prace/ad/86e3d6a0-239f-4cef-a52a-f50978c3fd95/orders/item
 HTTP/1.1
 Host: aden

 HTTP/1.1 200 OK
 Server: Apache-Coyote/1.1
 Cache-Control: no-cache, no-transform
 Content-Type: text/plain
 Date: Mon, 20 Oct 2014 13:37:52 GMT
 X-BE: aden01
 X-Varnish: 377168
 Age: 0
 Via: 1.1 varnish-v4
 X-Vokurka: not found!
 X-Server: Skrutener Zviratko 31.5
 Connection: keep-alive
 Accept-Ranges: bytes

 <hangs in there until timeout!!!>

 -------------------

 Varnish doesn't even send the 0-byte chunk it got from the backend, so
 client is kept waiting for the body to arrive (or in effect the connection
 to close).

 I believe this is a serious bug. I didn't notice it until I raised all the
 timeouts to something like 300s and encountered timeout in applications
 waiting for the data.

-- 
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1612>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator