Using varnish to increase the cache time of slow pages

[Jonathan Matthews]

On 9 March 2012 22:33, Thiago Figueiro <TFigueiro at au.westfield.com> wrote:
> I wrote a blog post describing quick-win modification I made to our Varnish configuration last week.  It lets us keep the default page cache-time low so content is fresh but specifically increases the TTL for pages which are slow to generate.
>
> The code is for Varnish 2.1.5 but I'm sure it can be easily adapted to 3.x so I thought I'd share it with the list.
>
> http://microrants.blogspot.com.au/2012/03/using-varnish-to-increase-cache-time-of.html

Thanks for posting this. I like the idea of tuning a TTL coarsely
based on the time it took you to generate that resource.

However, I have a couple of quibbles with this *implementation*:

1) Every bit of logic you embed into VCL is something you have to work
at if you move to another HTTP cache. The wonderful thing about HTTP
is it's a *standard* - and it includes its own method(s) for
communicating the TTL of a particular resource. By embedding this
logic in Varnish, you make it so that any other cache you try to use
later either has to support the same capabilities as Varnish, or makes
you do what I'm about to suggest: make the *back-end* decide how long
the resource can be cached.

By doing *that* (i.e. making the origin server output
standards-compliant caching headers, possibly based on how long it
took to produce the content) you not only enable Varnish to do The
Right Thing, but *every* HTTP-speaking cache/proxy/etc you might use
down the line.

Basically (and I totally accept this is a bigger challenge than just
munging the TTL in VCL!), fixing your back-end's code so it outputs
the right caching headers, possibly using the same methodology, is a
more significant win. IMHO :-)

2) From a more concrete PoV, I'm not sure this code copes well with
content the back-end explicitly marks as non-cachable (i.e. ttl=0).
There may be good reasons for a back-end to do this (possibly not in
/your/ app, but you are presenting it as having wider applicability
than just your situation, hence maybe caveats should be made more
visible) which are irrespective of the time it took to generate the
content.

Anyway, thanks for introducing the concept! Sorry if this looks a bit
negative - it's not really meant to come over as such :-)

Jonathan
-- 
Jonathan Matthews
London, Oxford, UK
http://www.jpluscplusm.com/contact.html