VCL language

[Poul-Henning Kamp]

I spent most of yesterday crunching paper into small balls and throwing
them out, but I think I got closer to understanding things.

The problem is this:  We want to give the user full control using
VCL but we also want him to be able to start without having to write
2000 lines of VCL just to emulate the default behaviour.

How we tackle this is deeply intertwined with the language and data
model we choose for VCL.

One approach we can take is to give the user a number of places
where he can hook VCL code into the default processing, and that
was more or less how I envisioned it originally.  Unfortunately,
that seems to end up rather complex and confusing once you try to
actually hash out all the details.

So then I took another route which said there will be only one VCL
hook, and it will implement the entire thing and sufficient levels
of DWIE (Do What I Expect) behind the scenes should make it simple
for the user to replace this one VCL function.

The learning curve for the user in this case is directly proportional
with the length of the "default VCL code", so I started to look at
how small we could make that while still giving the user the full
flexibility to do whatever he damn well pleases.

The default VCL is the one used for the case where only a backend
IP# is specified:

	varnishd -b 10.0.0.1

If I boil it all the way down, the smallest default VCL I can come
up with looks something like this:

    1     backend default {
    2         set backend.ip = 10.0.0.1;
    3     }
    4
    5     proc varnish {
    6	      if (req.request != "GET" && req.request != "HEAD") {
    7	           pass
    8	      }
    9	      lookup;
   10	      if (!obj.valid) {
   11	          fetch;
   12	          if (obj.cacheable) {
   13	              insert;
   14	          }
   15	      }
   16	  }

Now, please look at this carefully, before you read my explanation
below, I want to hear your first hand impression:  Is it intuitively
obvious to the casual observer what goes on here ?


Explanations:

  1:	All VCL programs must specify a backend named "default", which
	is the backend assigned to all requests unless the VCL explicitly
	assigns another one.  This is more or less what the "-b IP"
	command line option translates to.
  5:	The "varnish" function could also be called "main" if we prefer.
  6:	Normally only GET/HEAD can be cached.
  7:	"pass" just pipes the request to the backend and sends the result
	back without inspecting anything either way and no object is created
	for the transaction.
  9:	"lookup" will hash the url and lookup an object.  If no existing
	object is found a new (empty) one will be created.
 10:	If the object is not valid, we must fill it from the backend.
 11:	"fetch" does that.
 12:	If the object is cacheable, we insert it so future lookup's will
	find it.  Obviously the variable "obj.cacheable" is actually a
	meta variable which implements the RFC2616 logic by inspecting
	the headers returned from the backend etc.
 16:	After "varnish" returns, an implict "finish" is executed if the
	VCL code didn't already do so.

Does this make sense ?

Could you try to embellish this "default VCL" to implement a couple of
likely policies ?

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.