Please help break Varnish GZIP/ESI support before 3.0

[Kristian Grønfeldt Sørensen]

On tir, 2011-01-25 at 11:04 +0000, Poul-Henning Kamp wrote:
> One of the major features of Varnish 3.0 is now feature complete, and
> I need people to start beating it up and help me find the bugs before
> we go into the 3.0 release cycle.
> 
> 
> GZIP support
> ------------
> 
> Varnish will ask the backend for gzip'ed objects by default and for
> the minority of clients that do not grok that, ungzip during delivery.
> 
> If the backend can not or will not gzip the objects, varnish can
> be told in VCL to gzip during fetch from the backend.  (It can also
> gunzip, but I don't know why would you do that ?)
> 
> In addition to bandwidth, this should save varnish storage (one
> gzip copy, rather than two copies, one gzip'ed one not).
> 
> GZIP support is on by default, but can be disabled with a parameter.
> 
> 
> 
> ESI support
> -----------
> 
> Well, we have ESI support already, the difference is that it also
> understands GZIP'ing.  This required a total rewrite of the ESI
> parser, much improving the readability of it, I might add.
> 
> So now you can use ESI with compression, something that has hitherto
> been a faustian bargain, often requiring an afterburner of some kind
> to do the compression.
> 
> There are a lot of weird cornercases in this code, (such as including
> a gzip'ed object in an uncomressed object) so this code really
> needs beaten up.
> 
> 
> How you can help
> ----------------
> 
> The code is newly written, and bugs are to be expected, so I do not
> expect you to put it in production right away, but rather to run
> some stand alone tests, to see that it works for your site and
> content.
> 
> The code is feature complete, but still lacks sensible stats counters,
> debug handles and so on, these will be added in coming days.
> 
> The reports I am looking for are, in order of priority:
> 
>    1. How to crash varnish by sending legit traffic through it.
> 
>    2. How to crash varnish with worst-case traffic.
> 
>    3. How to make varnish send wrong content
> 
>    4. How to make varnish use a lot of resources
> 
>    5. Any other pertinent observations of trouble.
> 

I believe I've found a case of Varnish returning wrong content.
We use ESI to populate a json-array. The new ESI-code seems to handle
this just as well as it did in 2.1.x, as long as we keep gzip out of the
equation. When I use "set beresp.do_gzip=true" in vcl_fetch() some parts
of the document disappears from the response. My backend is not
configured to use gzip, so I let varnish do the gzip'ing. We use ESI in
mode 0x00000001 for ESI to work with json objects.

It doesn't seem to matter whether or not the response is served from
cache or not. It only seem to depend on the setting of beresp.do_gzip
when the objects are put in to the cache.  

I've attached 3 files: 
json.nogzip.txt showing the correct ESI-parsed response when gzip is
disabled.
json.gzip.txt showing the ESI-parsed response when beresp.do_gzip=true
is actived in the VCL.
json.esi.txt showing the response from the backend that Varnish tries to
parse. 

All files are requested using wget with no "Accept-Encoding"-headers.
Note that the segment just before the ESI-tags is missing from the
gzipped response.

Regards
Kristian Sørensen
-------------- next part --------------
{"10":[{"program":true,"id":859105,"vendorId":"15210857","starttime":1296168300000,"endtime":1296171900000,"title":"DR K Jazz: Minh og Alex Riel","oTitle":null,"genre":"0x30x0","timeshiftEnabled":true},{"program":true,"id":859309,"vendorId":"15210858","starttime":1296171900000,"endtime":1296173400000,"title":"Great Artists","oTitle":"true","genre":"0x20x3","timeshiftEnabled":true},{"program":true,"id":859258,"vendorId":"15210859","starttime":1296173400000,"endtime":1296173700000,"title":"Dagens sang","oTitle":null,"genre":"0x30x0","timeshiftEnabled":true},{"program":false,"id":-1,"vendorId":"-1","starttime":1296173700000,"endtime":1296184499000,"timeshiftEnabled":false}],"16":[{"program":false,"id":-1,"vendorId":"-1","starttime":1296171899000,"endtime":1296184499000,"timeshiftEnabled":false}],"start":1296171899000,"end":1296184499000,"epgSetId":-1,"size":50}
-------------- next part --------------
[{"program":true,"id":859105,"vendorId":"15210857","starttime":1296168300000,"endtime":1296171900000,"title":"DR K Jazz: Minh og Alex Riel","oTitle":null,"genre":"0x30x0","timeshiftEnabled":true},{"program":true,"id":859309,"vendorId":"15210858","starttime":1296171900000,"endtime":1296173400000,"title":"Great Artists","oTitle":"true","genre":"0x20x3","timeshiftEnabled":true},{"program":true,"id":859258,"vendorId":"15210859","starttime":1296173400000,"endtime":1296173700000,"title":"Dagens sang","oTitle":null,"genre":"0x30x0","timeshiftEnabled":true},{"program":false,"id":-1,"vendorId":"-1","starttime":1296173700000,"endtime":1296184499000,"timeshiftEnabled":false}]
-------------- next part --------------
{"10":<esi:include src="http://localhost:6081/client-portal/json/load?json=%7B%22starttime%22%3A1296171899000%2C%22endtime%22%3A1296184499000%2C%22channelType%22%3A%22TV%22%2C%22language%22%3A%22da-DK%22%2C%22channel%22%3A10%7D" />,"16":<esi:include src="http://localhost:6081/client-portal/json/load?json=%7B%22starttime%22%3A1296171899000%2C%22endtime%22%3A1296184499000%2C%22channelType%22%3A%22TV%22%2C%22language%22%3A%22da-DK%22%2C%22channel%22%3A16%7D" />,"start":1296171899000,"end":1296184499000,"epgSetId":-1,"size":50}