RFC: new vcl_lookup{} proposal

[Rangel, Raul]

This reply is in response to https://www.varnish-cache.org/trac/ticket/1107.

I like the explicit vcl. The less magic the easier it is to understand what is going on. I would also like to add that I love the idea of not having the first client block once the ttl expires. 

For my specific scenario having vcl_lookup is good...

I would like the ability to return a graced object instead of the current response from the backend. Sometimes my app server fails to get data from one of its external data sources. If this happens the app server can tag the request as being "not-ideal". If varnish gets a non-ideal response I want it to return the grace object if it exists otherwise return the non-ideal response.

An example of a non-ideal response would be an ESI that renders a gallery. If the app server fails to connect to the gallery data source it will render an empty gallery instead of returning a 500. Now if varnish has a stale copy then I would rather the client get a stale gallery. If varnish does not have a stale copy I would rather the user get an empty gallery. This way the user still gets the correct HTML and I can try and render the gallery on the client side. This is almost like saint mode but I don't want to mark the backend as sick.

Here is some sample vcl I dreamed up:

sub vcl_lookup {
	if (obj && req.http.X-Serve-Stale) {
		obj.ttl = 2s;
		return (deliver);
	}
}

sub vcl_fetch {
	if (beresp.status == 208) { // 208: The content might not be complete and a stale copy should be used
		set beresp.status = 200;
		if (!req.http.X-Serve-Stale) {
			set req.http.X-Serve-Stale = "yes";
			return (restart);
		}
	}
}

This is not currently possible with vcl_miss because we don't have the grace object. Though having a stale_obj and return(stale) in vcl_fetch might be the ideal way to handle this.

Raul