[Varnish] #788: v2.1.3 w/ http_range_support on fails to support sets in byte range request

[Varnish]

#788: v2.1.3 w/ http_range_support on fails to support sets in byte range request
-----------------------------------------------+----------------------------
 Reporter:  jim.robinson                       |        Type:  defect
   Status:  new                                |    Priority:  normal
Milestone:                                     |   Component:  build 
  Version:  2.1.3                              |    Severity:  normal
 Keywords:  http_range_support byte range set  |  
-----------------------------------------------+----------------------------
 Varnish 2.1.3 byte range support fails to work when handed a set of
 ranges.

 The HTTP 1.1 specification, section "14.35.1 Byte Ranges" specifies
 that

 {{{
   "A byte range operation MAY specify a single range of bytes,
    or a set of ranges within a single entity."
 }}}

 Given a simple VCL of:

 {{{
 backend default {
   .host = "127.0.0.1";
   .port = "80";
 }
 }}}

 where the backend is an Apache 2.2.14 server, we can start up varnishd
 on port 4040 and activate range support:

 {{{
 $ sudo /usr/local/varnish/2.1.3/sbin/varnishd -d -P /var/run/varnish.pid
 -a localhost:4040 -f /usr/local/varnish/2.1.3/etc/varnish/test.vcl -u
 nobody -g nobody -s malloc,10M

 storage_malloc: max size 10 MB.
 Using old SHMFILE
 Varnish on Darwin,9.8.0,i386,-smalloc,-hcritbit
 200 193
 -----------------------------
 Varnish HTTP accelerator CLI.
 -----------------------------
 Type 'help' for command list.
 Type 'quit' to close CLI session.
 Type 'start' to launch worker process.

 start
 child (69026) Started
 200 0

 param.set http_range_support on
 200 0
 }}}

 In another window we first issue a curl request to our apache server and
 ask for the first four bytes as a set of 1-byte ranges:

 {{{
 $  curl -s -i -HRange:bytes=0-0,1-1,2-2,3-3  http://127.0.0.1/test.pdf
 HTTP/1.1 206 Partial Content
 Date: Mon, 04 Oct 2010 19:46:25 GMT
 Server: Apache/2.2.14 (Unix) DAV/2 mod_jk/1.2.28 mod_ssl/2.2.14
 OpenSSL/0.9.7a
 Last-Modified: Fri, 09 Jul 2010 16:52:27 GMT
 ETag: "11fc13b-11893e-48af73a5d3ba1"
 Accept-Ranges: bytes
 Content-Length: 389
 Content-Type: multipart/byteranges; boundary=491cfccba1f8e35b9


 --491cfccba1f8e35b9
 Content-type: application/pdf
 Content-range: bytes 0-0/1149246

 %
 --491cfccba1f8e35b9
 Content-type: application/pdf
 Content-range: bytes 1-1/1149246

 P
 --491cfccba1f8e35b9
 Content-type: application/pdf
 Content-range: bytes 2-2/1149246

 D
 --491cfccba1f8e35b9
 Content-type: application/pdf
 Content-range: bytes 3-3/1149246

 F
 --491cfccba1f8e35b9--
 }}}


 Now while we can issue a single range request to varnish and get back the
 right thing:

 {{{
 $ curl -s -i -HRange:bytes=1-1  http://localhost:4040/test.pdf ; echo
 HTTP/1.1 206 Partial Content
 Server: Apache/2.2.14 (Unix) DAV/2 mod_jk/1.2.28 mod_ssl/2.2.14
 OpenSSL/0.9.7a
 Last-Modified: Fri, 09 Jul 2010 16:52:27 GMT
 ETag: "11fc13b-11893e-48af73a5d3ba1"
 Content-Type: application/pdf
 Accept-Ranges: bytes
 Date: Mon, 04 Oct 2010 19:46:33 GMT
 X-Varnish: 716983327 716983316
 Age: 189
 Via: 1.1 varnish
 Connection: keep-alive
 Content-Range: bytes 1-1/1149246
 Content-Length: 1

 P
 }}}

 But we cannot issue a set of ranges:

 {{{
 $ curl -s -i -HRange:bytes=0-0,1-1,2-2,3-3  http://localhost:4040/test.pdf
 | strings | head -51param.set http_range_support on
 HTTP/1.1 200 OK
 Server: Apache/2.2.14 (Unix) DAV/2 mod_jk/1.2.28 mod_ssl/2.2.14
 OpenSSL/0.9.7a
 Last-Modified: Fri, 09 Jul 2010 16:52:27 GMT
 ETag: "11fc13b-11893e-48af73a5d3ba1"
 Content-Type: application/pdf
 Content-Length: 1149246
 Accept-Ranges: bytes
 Date: Mon, 04 Oct 2010 19:46:46 GMT
 X-Varnish: 716983328 716983316
 Age: 203
 Via: 1.1 varnish
 Connection: keep-alive
 %PDF-1.4
 45 0 obj
 ... rest of the entire PDF elided...
 }}}

 The problem appears to be that cache_response.c has an assumption built
 into the logic that byte range requests aren't multi-sequence:

 {{{
 278     if (sp->disable_esi || sp->esis == 0) {
 279             /* For none ESI and non ESI-included objects, try Range */
 280             if (params->http_range_support &&
 281                 (sp->disable_esi || sp->esis == 0) &&
 282                 sp->obj->response == 200 &&
 283                 sp->wantbody &&
 284                 http_GetHdr(sp->http, H_Range, &r))
 285                     res_dorange(sp, r, &low, &high);
 286
 287             sp->acct_tmp.hdrbytes += http_Write(sp->wrk,
 sp->wrk->resp, 1);
 }}}

 The res_dorange request that calculates the range and the subsequent call
 to http_Write assume only a single range, not a set of ranges.

 Jim

-- 
Ticket URL: <http://www.varnish-cache.org/trac/ticket/788>
Varnish <http://varnish-cache.org/>
The Varnish HTTP Accelerator