[Varnish] #690: Allow for a negative grace period for internal crawler

[Varnish]

#690: Allow for a negative grace period for internal crawler
------------------------+---------------------------------------------------
 Reporter:  mikeytown2  |        Type:  enhancement
   Status:  new         |    Priority:  normal     
Milestone:              |   Component:  build      
  Version:  2.0         |    Severity:  normal     
 Keywords:  grace       |  
------------------------+---------------------------------------------------

Comment(by LaurentChavet):

 Let me explain what the idea is here as I wrote the code.
 We want our users to always get cache hit; to do that we can cache pages
 for a very long time (could be forever if we have enough memory).
 That works fine until pages are changed (we know it at the backend level);
 at that point we want to put a new version of the page in the cache
 without impacting our users (they should still get their pages from the
 cache).
 With the changes above, here is what happens:
  Page FOO is cached with a large TTL (say 1 week); that part can be
 triggered by an internal crawler.
  It is served from cache to our users.
  1 hour later a change needs to be pushed to that page: the crawler visit
 the page with user-agent="crawler" which (through the code change) sets
 the current cached page to expired with a grace period so it is still
 served. When the crawler request completes it is added to the cache and
 any future requests from our users get the new page without anyone (except
 for the crawler) getting a cache miss.
 I couldn't find any way to implement this feature without the code above.
 Let me know if there is a different solution.

 Laurent Chavet

-- 
Ticket URL: <http://www.varnish-cache.org/ticket/690#comment:2>
Varnish <http://varnish-cache.org/>
The Varnish HTTP Accelerator