Conditional Requests (revisited)

[Brian Smith]

Poul-Henning Kamp wrote:

> Brian Smith writes:
>
>>(1) I have a back-end that is very fast for validating ETags and 
>>generating
>>304 Not Modified, but very slow when it has to return a new response body.
>>My understanding is that Varnish will never send conditional requests to 
>>the
>>backend, and it cannot use a 304 response to update the freshness
>>information for an already-cached response.

>>Would you be willing to accept patches to implement that behavior in the 
>>core?
>
> Provided they are well thought out: yes, then all patches are welcome.
>
> Think hard about how to enable this sensibly in VCL.

I am not sure I will go ahead and implement this but I did think about it a 
little bit.  In VCL, we can run different instructions for a cache hit or a 
cache miss. As far as I understand, a cache hit occurs whenever Varnish is 
able to return a response from the cache without contacting the back-end, 
and a miss occurs otherwise. However, I think a third case is needed: 
vcl_stale, where there is a cached entry but it is out of date. To me, it 
seems like Varnish could manage stale hits itself (without running any VCL 
code), by automatically revalidating with the back-end, processing the 
response, and then calling the hit VCL or the miss VCL, based on whether the 
response was a 304.

In other words, it seems reasonable to implement this in such a way that 
there would be no changes to VCL. Do you think that is reasonable?

One benefit to doing everthing automatically is that, when we have 
"varnishd -t 0" and no VCL file, Varnish would become a transparent cache 
(in the RFC 2616 sense). Even though the developers seem to not find the 
idea of a transparent cache very compelling, I think such functionality 
would make Varnish compelling for many more users.

Thoughts?

Regards,
Brian