Testing Varnish

[Anders Berg]

> In message <0F08FB20-F4EE-4345-88C3-A271B340E5B3 at yahoo-inc.com>, Mark
> Nottingham writes:

>>> All that however, does not change the fact that in the separate
>>> roles of client and origin server, Varnish should comply with
>>> RFC2616 to the extent possible, reasonable and productive.
>>
>>That's good to hear. There seem to be a number of HTTP violations
>>(that aren't specific to proxies) in Varnish; again, I'd encourage
>>you to talk to the Measurement Factory folks and make their tests (or
>>something like them) part of your QA suite.
>
> Yes, we have a lot of work to do in this area, and we need to build
> some kind of test-rig so we can test the VCL primitives comprehensively
> as well.

I would like to add that when it came to RFC 2616 compliance, the team
discussed it a fair bit, without going into all the gory details possible.
"What??!!" You may ask, "You have to do just exactly that!". Well, here
comes one of the cool parts about Varnish: We discussed implementing RFC
2616 in VCL, as a default VCL config file. While, as you notice, we ended
up with a not so strict interpetation as default, I (with the fear of
saying something wrong, since I did not write the code) am pretty sure
that VCL is so flexible as to be able to do exactly that.

In other words, one can say that while Varnish is a "cache" that be
anything you want it to be :) Think of it as a cache that is _quick_ at
doing a lookup/serv and scale good on hardware, and at the same time has
the flexibility of a "programming language" (VCL). We are fully aware that
VCL itself puts a threshold on our users. Thats regretful, but we worked
hard at getting it as easy as possible (Insane to say right now since the
doc on VCL is hidden somewhere in /dev/null land), but at the same time
very powerful.
When the day is over, we/I feel that the primary users that Varnish tries
to serv, are users that may need to do complex things with HTTP 1.0-1. And
we give them the tool for this. Every website that will benefit alot from
Varnish has the resources to learn and appreciate VCL quickly. At the same
time we give the hobby website hackers a okay default run-and-forget
config, we think...

Thanks for pointing us towards Measurement Factory. Their tool(s) may be
great for finding/making/testing vcl files for their compliance or such.
I'll dig into it.

I will have to stand corrected if what I say here is completly off the
hook, but this is at least part of the dialog and conclusion the team has
ended up with in regards to RFC 2616 and VCL.

Anders Berg