VCL language

[Anders Berg]

> The problem is this:  We want to give the user full control using
> VCL but we also want him to be able to start without having to write
> 2000 lines of VCL just to emulate the default behaviour.
> [...]
> The learning curve for the user in this case is directly proportional
> with the length of the "default VCL code", so I started to look at
> how small we could make that while still giving the user the full
> flexibility to do whatever he damn well pleases.

Yes, I can see why you have been crumbling paper. Hard to see this one
coming, and even harder to solve out when you start thinking about it.
I would have said we _have_ to have the flexibility, but you are right.
This quickly becomes tricky.

Seeing the default .cfg later on, we might have to consider the
flexibility. Not because I don't think it is important (it's one of the
main pilars!), but we need an "easy/ier" start. Darn...

Is one approach to "hide" the default .cfg and say: Hey, this is probably
what you wanna do. Don't look at it. ?

Or is it better to let users have 2 "levels"?

1. Easy .cfg approach. Low flexibility.
2. Hard .cfg. High flex.

We can provide sample .cfg files for the general needs at web-site?

Just ideas, I don't like them. I have a hard time coming up with
alternative approaches though. I _know_ you have wrestled the default .cfg
file, so we have to come up with a solution another way.

> The default VCL is the one used for the case where only a backend
> IP# is specified:
>
> 	varnishd -b 10.0.0.1
>
> If I boil it all the way down, the smallest default VCL I can come
> up with looks something like this:
>
>     1     backend default {
>     2         set backend.ip = 10.0.0.1;
>     3     }
>     4
>     5     proc varnish {
>     6	      if (req.request != "GET" && req.request != "HEAD") {
>     7	           pass
>     8	      }
>     9	      lookup;
>    10	      if (!obj.valid) {
>    11	          fetch;
>    12	          if (obj.cacheable) {
>    13	              insert;
>    14	          }
>    15	      }
>    16	  }
>
> Now, please look at this carefully, before you read my explanation
> below, I want to hear your first hand impression:  Is it intuitively
> obvious to the casual observer what goes on here ?

I did, and you are not gonna like my answer. As you mention later on, it's
also a choise of semantics, but this is my head when I read it:

1-3: Fine
5: proc varnish?
6: Strange to have here, but okay, I understand it.
7: Pass. Fine. Wait, I wanna pass GET's and HEAD's or don't I? Pass? Hmm.
I must read on.
9: lookup? DNS? Must read on.
10: !obj.valid, sure. When my document is not valid in cache.
11: fetch. Must mean fetch from backend. Fine.
12: Hold on, why are we still in loop. Read 10-11, and see where if ends.
Okay, I now know that this must be something that must be in the loop. Not
sure why. obj.cacheable, sure.
13: insert. Okay. It's cacheable, insert in cache.
XX: Why is the obj.cacheable inside the loop? Could it not be outside the
loop? Head crash. Heck, it probably has to be that way.

lol, I am not claiming to be the sharpest tool, but I guess many may feel
like how I just did. Sorry for the details of my head, but it may help us
getting closer to a good solution.
I could live with a .cfg like that, but I _certainly_ would not deploy it
before I read some documentation.

> Does this make sense ?

Yes, it all does, but not the default .cfg :)

> Could you try to embellish this "default VCL" to implement a couple of
> likely policies ?

At VG Nett we config a cache mainly to try to do 3 thing.

1. docs with ending .XXX must be cached for XX min. (.hbs 5 min, .gif 1
weekish, .jpg 1 weekish. etc...)
2. Please cache everything in this damn poorly written app for 10 sec (if
it's a soccer event etc.) or 5 min in general.
3. My app is written good for once. Please confim to its expiry.

So I guess I would try something like:

1     backend default {
2         set backend.ip = 10.0.0.1;
3     }
4
5     proc varnish {
6	      if (req.request != "GET" && req.request != "HEAD") {
7	           pass
8	      }
9	      lookup;
10	      if (!obj.valid) {
11	          fetch;
12                 if (obj.url ~ .hbs){ # It feels strange to use 
req.url.path here!
13                   obj.ttl_in_cache = 5 min # No variable to use. Not
ttl. for sure.
14                   insert # ?
15                 }
16-28  # just like .hbs with .jpg and .gif
29                  if (obj.path ~ /poor_written){
30                     obj.ttl_in_cache = 5 min
31                   }
32                   if (obj.path ~ /nicly_written){
33                     obey_header # No variable to use.
34                   }
35	          if (obj.cacheable) {
36	              insert;
37	          }
38	      }
39	  }

To put it this way. I understand why you have been crumpeling paper all
day. I feel bad not to come up with a solution here and now, but I really
can't...

I will think about this, and I am sure you and Dag will also.
Beeing born in Bodø, far up in Norway, leaves me beeing overly optimistic
though :)

Anders Berg