Proposal/specs for backend conditional requests / aka "GET If-Modified-Since" (GET IMS))

[Nils Goroll]

Hi All,

we propose a new feature for varnish: Backend conditional requests
(sometimes simplified as "GET If-Modified-Since" (GET IMS)). A working
implementation by Rackspace, Inc. exists.

While both Rackspace and UPLEX do need this feature and would continue
to improve the implementation anyway, we would very much favor a
design and implementation coordinated with the core developers and all
of the Varnish developer community. Our focus is to achieve an
implementation which is fully supported by the core team and will be
integrated into mainline Varnish one day.

We have therefore prepared a "design specification" and would highly
appreciate any constructive feedback, corrections and suggestions for
improvements.

While we are prepared to do all of the required implementation,
documentation and testing (including final tests on highly loaded
production systems), we're open for joint development with other
community members.


Thank you,

Adrian Otto, Rackspace Inc.
Nils Goroll, UPLEX
Geoffrey Simmons, UPLEX


Design Spec Suggestion for Varnish Backend Cache Validation using
Conditional GET
======================================================================

Intro
-----

We need to fulfill the following customer requirement: When a cached
object in Varnish expires in accordance with its TTL, and the object
has not changed on the backend server, the expired object can be
"refreshed" in the Varnish cache to allow its continued use without
re-copying the object over the network from the backend server.

Detail
------

In cases where a backend server can efficiently determine when an
object was last modified, or uses entity tags to provide versioning
capabilities, bandwidth utilization and request latency between the
Varnish server and the backend server can be significantly
improved. This is very important for cached content that is
dynamically generated, or if the object is very large.

RFC 2616 section 13.3 provides a suitable solution for this problem,
but Varnish must be extended to support conditional backend requests
in order to employ the solution.

Currently varnish makes no attempt to generate conditional requests to
the backend servers when content expires in the cache. It always sends
a normal GET request (no If-Modified-Since, If-None-Match, etc. header
is added), which results in a 200 OK response. The body of the
response must be copied over the network repeatedly when TTLs expire.

We intend to extend Varnish to use conditional GET requests when
refreshing expired content in accordance with section 13.3 of RFC
2616. This enables the backend the option to respond with a "304 Not
Modified" response to instruct Varnish to continue to use the object
it already has in cache.

The proposed solution will be based upon an existing implementation by
Adrian Otto, and Dale Chase (c) 2010 Rackspace, US Inc. This work
demonstrates that an RFC-2616 compliant implementation for conditional
back-end requests produces a dramatic improvement in system
efficiency, particularly in cases where cached objects are large, and
expensive to generate on the backend server.

This work will be extended to include relevant VCL support so that
backend requests can be manipulated with separate logic for a refresh
event.

Scope
-----

In a first iteration, we'll be implementing a GET with
If-Modified-Since conditional set to the Last-Modified time and
If-None-match for an ETag, if present, as stored for the cache object;
implementing the full generality of conditional requests would be an
obvious road for future development.

Cache Validation Behavior
-------------------------

If the backend responds to the conditional request with 200 OK, then
the object is updated as usual. If the response is 304 Not Modified,
we update the cache object based on the response headers (primarily
this means updating its TTL, as currently implemented in rfc2616.c).

Additional states, VCL support
------------------------------

We propose to implement cache validation using new states "stale" and
"refresh" into the Varnish state machine.

* cnt_stale() / vcl_stale()

  "stale" parallels "hit" and "miss", to be implemented by adding
  cnt_stale() to cache_center.c and vcl_stale() to VCL.

  An object is stale if it is found in the cache, but its TTL is
  expired. Note that obj.grace can be used to ensure objects will stay
  in the cache for longer than their TTL dictates.

  vcl_stale() is called just after the backend request is prepared
  from req, including all necessary conditionals with respect to the
  existing (stale) cache object. This way, users can modify the
  properly prepared conditional bereq at their wish.

  vcl_stale() provides access to req, bereq and obj. The possible
  return states from vcl_stale() are fetch, refresh, deliver, pass,
  error and restart:

  - fetch: default, same behavior as a miss: Fetch the obj from the
    backend with an unconditional request. All conditional request
    headers will be removed.

    Besides preserving the default behavior, this return value is
    intended to trigger a full backend fetch even if a refresh /
    conditional GET would be available, for instance when

    - backends don't implement conditionals correctly, or

    - the administrator knows that for certain objects there is no
      advantage in sending a conditional request

  - refresh: Continue with the (conditional) GET

    Note: If all conditional request headers are removed in
    vcl_stale(), return(refresh) shall be semantically equivalent to
    return(fetch).

  - deliver / pass / error / restart : obvious

  Besides controlling backend request behavior, the new VCL function
  is intended to allow easy manipulation of request headers only for
  the refresh-case.

* cnt_refresh() / vcl_refresh()

  The state "refresh" parallels "fetch", and does most of what is done
  in fetch, except that the cache object is only updated based on the
  response headers if so indicated.

  In vcl_refresh(), it is possible to access req, bereq, beresp and
  obj (which in this case refers to the cached object, as in
  vcl_hit(), vcl_miss() or vcl_stale()). The possible return states
  for vcl_refresh() are pass, restart, deliver and error (just as for
  vcl_fetch()).

  Since cnt_refresh() will likely duplicate much of what cnt_fetch() does,
  we intend to delegate the parts they have in common into
  subroutines.

  There is also a risk of code duplication for VCL users, if
  vcl_refresh() and vcl_fetch() need to perform common tasks, but this
  can also be alleviated by calling subroutines (vcl_refresh() could
  even call vcl_fetch(), if so desired, and would by default).

Reasons and examples for proposing new VCL procedures
-----------------------------------------------------

* What is vcl_stale() good for?

  Modifying request headers only for the case that a refresh is
  possible:

  One would change backend request headers as in vcl_recv, but only
  in case a refresh is possible. We imagine one would like to
  actively remove conditional statements which are irrelevant, add
  more conditional statements, or tweak the If-Modified-Since.

  Intelligent decisions about when to do a fetch or deliver stale
  content:

  The other major reason for vcl_stale is that the administrator might
  have to handle a buggy backend, which wouldn't deliver correct
  content with conditionals, so a fall back to fetch is needed.

  On the other end, administrators might decide that stale content is
  "fresh enough" and deliver anyway. We imagine a kind of "intelligent
  grace mode" where stale content is delivered without a backend
  refresh, for instance when all backends are down, certain request
  headers are present etc.

* Why vcl_refresh() in addition to vcl_fetch?

  The result of the built in refresh action (header update) might need
  to be amended or corrected, for instance the administrator might
  want to modify the Expires: or Cache-Control: headers.

Updating the in-cache object
----------------------------

The in-cache object must be updated with headers from the 304
response, in particular the cache control headers (Cache-Control,
Expires, Etag ...) must be changed.

The existing Rackspace Inc. implementation copies, for objects smaller
than a configurable threshold, the body data from the existing
object. For larger objects it modifies the in-cache object headers,
which we now believe will lead to inconsistencies when the object is
being read concurrently.

We plan to implement the update of the data in cache by not actually
touching the existing object, but

- by copying any headers not in the 304 response from the existing
  object,
- changing obj.status of the 304 object to the status of the existing
  object and
- letting the new object point to the existing in-cache body data.

To allow multiple cache objects to share body data, we want to add
reference counters to struct storage following the example of the
existing implementation for objects (HSH_Ref(), HSH_Unref() etc). We
will have a new STV_Ref(), STV_Free() will become STV_Unref() and will
be modified to only actually free storage when the reference counter
is zero (after decrementing). STV_alloc() will set the reference
counter to 1 to not break existing code.

We prefer this approach over a read/writer lock on the object because
it should yield better concurrency. Allowing to keep multiple
references to the same struct storage might also turn out to be useful
for other future improvements (ideas like "cache dedupe").

Notes
-----

RFC 2616 generally calls this process "validation", not "refresh". We prefer
the term "refresh" for use within Varnish c and VCL code.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: varnish_state_conditional.pdf
Type: application/pdf
Size: 24980 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20100923/cadbe6b8/attachment-0003.pdf>