Architectural heads-up/call for comments

[Rob S]

Poul-Henning Kamp wrote:
> 1.  Kill the magic default VCL.
>   
This will make life a lot simpler for people starting out with Varnish, 
and help ensure that those people with advanced configurations don't 
overlook something.  I'd like to mutter something here about adding some 
form of "return pass", rather than just writing "pass".  This will 
further clear up the fact that execution will, at the time your write 
"pass" cease.  (Please let me know if I've got this wrong).

> 2.  Client identity
>   
Access to cookies is great, but I don't necessarily think that a 
client.ident idea is sensible.  Is the "client" object going to be 
persistent between requests?  If so, what other properties will it 
have?  If it isn't persistent, then with existing VCL you can create a 
.ident property of a req, and use that to switch between pools.  Is this 
point really that you want to establish a mechanism for passing 
information into more advanced directors? 

> 3. Synth replies (and vcl_error{} ?)
>
> I want to make it possible to produce a synthetic reply anywhere
> in VCL...
Great.  Definitely useful.  A use case we had recently, but which we 
couldn't implement, was to synthetically generate some information about 
the backend pool.  So, we wanted a request for a particular javascript 
file to be generated by Varnish and say 
"randomhealthbackendhostname='"+be.name+"';".  We couldn't do this, but 
it might help guide thinking as to how this might get used.

> I also want to make it possible to suck in the synth body from a
> file
>   
Make sure we can specify the mime type of the synthetic reply!

> (The file will be read at compile-timer!)
>   
Nervous of this.

> 4. VCL conversions
>   
This would definitely be worth doing.  We've got a few bits of config 
which could be enormously improved if we could parse header responses 
and put them as TTL values etc.  We use varnish in front of some apps 
which use session state / cookies / other things that make caching 
hard.  So, for apps we've been through and sanitised for varnish, we 
specify X-Varnish-Cache-Control which we parse in VCL and use to control 
Varnish's own data store, and X-External-Cache-Control, which is what we 
present out to the end requestor.  If the response only has 
X-Cache-Control, we assume that the app hasn't been 
prepared-for-varnish, and thus mustn't be cached by varnish, regardless 
of what's specified.  This allows us to fail-safe, but has the impact 
that we currently can't set the ttl to the precise value we want.  Your 
proposed conversions would enable us to improve our config.
> Thanks for listening, now it's your turn to tell me if this is
> stupid...
>   
Definitely good stuff.  Let us know how we can continue to help.

There's one other thing that I'd like people to comment / think about.  
We operate multiple varnish front ends, so that we can cope if one 
fails.  How do people distribute purges between them?


Rob