RFC: new vcl_lookup{} proposal

[Martin Blix Grydeland]

* Sorry about the early incomplete version of this email. I sent it
unfinished by mistake *

As we are moving more logic from varnishd into VCL, I believe some
rethinking with how we deal with the default_vcl logic might be in order.
This to make it easier to make the easy VCL changes and keep the default
logic around still.

> A common vcl error I have observed is where you want to match on something
to perform an action, and then stop further processing on this request so
further rules won't match on the url. E.g.:

> sub vcl_fetch {
    if (req.url ~ "\.(gif|jpg|css") {
        # Cache our static resources "forever"
        set beresp.ttl = 180d;
        return (deliver);
    }
}

> This will ofc bypass any set-cookie checks performed by the default
vcl_fetch logic. Also the explicit return(deliver) prevents any further
application level rules that may be applied later, which is convenient. But
the problem comes if for some reason one of these static resources should
actually return a set-cookie header.

> If we had restructured the default vcl logic bits into vcl subroutines of
their own, this would be much easier to do safe. Given that we have a
default_vcl_fetch routine that contains exactly what vcl_fetch contains
today, the above example could become:
>
> sub vcl_fetch {
    if (req.url ~ "\.(gif|jpg|css") {
        # Cache our static resources "forever"
        set beresp.ttl = 180d;
        call default_vcl_fetch; # Will not return
    }
}

The default vcl_fetch routine would simply become:

sub vcl_fetch {
    call default_vcl_fetch;
}


To take it a little further, I also think that some of the default logic
could do with a way to modify it, without having to copy all and redoing
it. If e.g. the default_vcl_recv looked like this:

sub default_vcl_recv {
    call default_vcl_recv_xff; # Handle X-Forwarded-For header creation
    call default_vcl_recv_check_method; # Pass on non-recognized http
methods
    call default_vcl_recv_check_pass; # Pass on anything but GET and HEAD
    call default_vcl_recv_check_auth; # Pass on auth and cookie headers
present
    return (lookup);
}

We could then change part of the logic only by overriding only one of the
functions. So if I had a need to do the X-Forwarded-For headers differently
for my site, I could redefine only that function. E.g.:

sub default_vcl_recv_xff {
    if (client.ip ~ acl_fw) {
        set req.http.X-Forwarded-For = req.http.X-Orig-IP;
        exit; # Exit from current subroutine (only allowed from
non-callback subs, could be implemented by a goto in libvcl)
    }
    # Default vcl would take over here doing the normal XFF handling
}

Martin

-- 
Martin Blix Grydeland
Varnish Software AS
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20121001/92b0a4d2/attachment.html>