Cross-verb cache invalidation (POST/PUT invalidates GET)

[Jonathan Matthews]

On 13 May 2011 20:11, Jonathan Matthews <contact at jpluscplusm.com> wrote:
> How can I (and *should* I?) instruct Varnish to obey the TTL that the
> backed sets for content received by GETting a specific URI (say
> http://example.com/api/object-1/) but to invalidate that content when
> a PUT/POST to the same URI is observed?
>
> I've not started poking at it through Varnish yet and, given the
> definition of the HTTP verbs in the RFCs, I kind of hope it might Just
> Work

OK, having read the RFCs a bit more I note that this behaviour is
mandated: http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13.10.
Indeed, it's item #325 in the table at
http://www.varnish-cache.org/trac/wiki/HTTPFeatures.

The default VCL doesn't implement this, but a naive and possibly
incomplete implementation is obviously trivial:

sub vcl_recv {
  if (req.request == "POST") {
  purge("req.url == " req.url " && req.http.host == " req.http.host);
  }
}

How can this go wrong, however? In what circumstances might we wish
not to take this rather draconian action?

Well, if we go by the RFC, the response code to the POST request is of
no consequence in determining if the content should be invalidated. So
my reading of this suggests we're technically safe in taking action in
the vcl_recv stage, and not after a lookup/fetch/deliver. Which is
lucky, since the default VCL, I believe, (someone please do correct me
on this!) doesn't put us in a situation where we *could* take any
action in those places, due to us being in pass mode by this point.

So what other gotchas are there? One that comes to mind is
authentication.  We don't want to allow an unauthenticated or
wrongly-authenticated POST request (malicious or otherwise) to drop
authenticated content from the cache, but that's about it.  Both POST
requests and the presence of Authentication headers cause the default
VCL to enter pass mode, so I *think* that the only situation that
needs thought is when content exists in cache (hence resulted from an
unauthenticated request) and an authenticated POST request arrives. As
before, the RFC seems unforgiving, not mentioning any mitigating
circumstances, so I suppose we might as well just invalidate in that
circumstance.

So, coming back to the initial naive implementation, it looks like it
could be correct. But what have I missed? There's got to be some
complication to explain why this (or something like it) isn't in the
default VCL - was a decision taken not to adhere to section 13.10 of
RFC2616 at some point in the past?

Jonathan
-- 
Jonathan Matthews
London, UK
http://www.jpluscplusm.com/contact.html