Elaborating on proposed vcl_stale() and vcl_refresh() // Proposal/specs for backend conditional requests / aka "GET If-Modified-Since" (GET IMS))

[Nils Goroll]

Hi,

I've had a first discussion with phk on IRC. IIUC, his main concern was that
introducing the new states & VCL procedures would increase complexity and raise
the entry barrier, in particular with regard to users wanting to (easily)
customize the default vcl.

I will elaborate on the reasons for the proposed solution.

a) vcl_stale()
--------------

The alternative we've discussed on IRC is to fall into vcl_miss(): If an obj
eligible for refresh was found, default backend conditional request headers were
added to bereq, and vcl_miss() would see obj. It wouldn't for the current case
(a real miss). To avoid a backend conditional request, admins could remove the
conditional headers (If-...) from bereq. return(refresh) wouldn't exist.

Here's my subjective perspective on why vcl_stale() would make sense:

- Without vcl_stale(), the semantics of vcl_miss() would mean "I have not found
  an object eligible for delivery, but might have found one for refresh", which
  I believe will confuse users more than help.

  Mostly, this does open up a whole lot of oddities and questions:

  - vcl_miss would see obj only if it was a "stale miss"

    Odd: What would obj.status return for a "real miss"?

  - cnt_miss could prepare bereq.http.If-... headers only if it was a "stale
    miss"

  - We would be able to return (deliver) for a "stale miss", but not for a real
    miss: What do we do here? We can't decide on the correctness of the VCL
    at compile time. Would return (deliver) fall back to a return (fetch)
    is it's a "real miss"?

- Users would need to put more conditionals into vcl_miss for special handling
  of "stale" cases, à la:

  if (obj) {
	# do stale case handling
  } else {
	# existing code
  }

Depending on whether we want to default to the new or current behavior, a
default vcl_stale() would look like:

vcl_stale() {
	return (refresh);
}

or

vcl_stale() {
	call vcl_miss;
}



If we put the mechanics into vcl_miss, retaining current behavior would need
something like this with a full backend conditional:

vcl_miss() {
	if (obj) {
		unset bereq.http.If-Match;
		unset bereq.http.If-Modified-Since;
		unset bereq.http.If-None-Match;
		unset bereq.http.If-Range;
		unset bereq.http.If-Unmodified-Since;
	}
}


My personal impression is that vcl_stale would help retain clarity rather than
complicate things, in particular because it would help to limit additional
processing to the cases when it's actually needed.

b) vcl_refresh()
----------------

The alternative to vcl_refresh() would be to put additional logic into vcl_fetch().

Similar to the stale case, we would want to have access to the original object
in the cache and the beresp from the refresh, so obj would be available only if
vcl_refresh was entered with (beresp.status == 304).



We really want to retain clarity and not introduce unneeded additional
complexity. I believe that introducing more VCL states and procedures would help
to achieve this, in particular because the preconditions for each case were
clear and visibility of a cache object was well defined.

Thanks, Nils