Please help break Varnish GZIP/ESI support before 3.0

[Poul-Henning Kamp]

One of the major features of Varnish 3.0 is now feature complete, and
I need people to start beating it up and help me find the bugs before
we go into the 3.0 release cycle.


GZIP support
------------

Varnish will ask the backend for gzip'ed objects by default and for
the minority of clients that do not grok that, ungzip during delivery.

If the backend can not or will not gzip the objects, varnish can
be told in VCL to gzip during fetch from the backend.  (It can also
gunzip, but I don't know why would you do that ?)

In addition to bandwidth, this should save varnish storage (one
gzip copy, rather than two copies, one gzip'ed one not).

GZIP support is on by default, but can be disabled with a parameter.



ESI support
-----------

Well, we have ESI support already, the difference is that it also
understands GZIP'ing.  This required a total rewrite of the ESI
parser, much improving the readability of it, I might add.

So now you can use ESI with compression, something that has hitherto
been a faustian bargain, often requiring an afterburner of some kind
to do the compression.

There are a lot of weird cornercases in this code, (such as including
a gzip'ed object in an uncomressed object) so this code really
needs beaten up.


How you can help
----------------

The code is newly written, and bugs are to be expected, so I do not
expect you to put it in production right away, but rather to run
some stand alone tests, to see that it works for your site and
content.

The code is feature complete, but still lacks sensible stats counters,
debug handles and so on, these will be added in coming days.

The reports I am looking for are, in order of priority:

   1. How to crash varnish by sending legit traffic through it.

   2. How to crash varnish with worst-case traffic.

   3. How to make varnish send wrong content

   4. How to make varnish use a lot of resources

   5. Any other pertinent observations of trouble.

I have written the beginning of the documentation in our sphinx docs:

	http://www.varnish-cache.org/docs/trunk/phk/gzip.html

To test this, you need to be aware that we switched from SVN to GIT
last week, so to pull a copy of -trunk the magic command now is:

	git clone git://git.varnish-cache.org/varnish-cache/

Thank you for using Varnish, and thank you for helping make 3.0
our best release ever.

Poul-Henning

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.