[ESI] Best practice for caching user specific content

[Nikhil Lanjewar]

Hi,

I have been playing with Varnish for some time now. I will try to explain
the scenario in a lengthy email that follows. However, here's the summary
too. Please bear with me and take a look at the detailed scenario in case
the summary doesn't make enough sense.

*Summary*:

I would like to pass some query parameters from a source URL to an ESI URL
that was returned while fetching the source URL object itself. Is there a
way to do this?

*Detailed Explanation:*

My web application is a typical consumer web application with the following
features:

   1. Consumer pages do not need Login/Signup. User identification takes
   place via a unique value passed as a query parameter.
   2. Consumer pages show user's email in the header in case user identity
   was present in the URL's query parameters. These are absent in case the
   pages are being accessed without a user's identity.
   3. Some links on the page are also appended with the user's identity in
   case it was present with the parent page.
   4. Every other piece of information on these pages is the same for every
   user, except for the links (3) and email shown in the header (2).
   5. Listing pages contain pagination controls. Paged listings contain
   page number as one of the query parameters in the URL.
   6. The whole system is RESTful and does not rely on sessions or even
   Cookies as of now.

Given the above, I have come up with the following Varnish setup:

   1. vcl_recv - Extracts user identity and sets it on a custom request
   header such as X-UserIdentity. Cleans req.url by removing user specific
   query parameter and other front-end related query parameters such as Google
   Analytics query parameters used by GA's Javascript.
   2. Backend always receives a request without the user's identity so that
   single copy of the parent page can be cached.
   3. Internal links on the parent page that require user's identity are
   rendered via <esi:include> tags. Similarly, email shown in the header is
   also replaced by an <esi:include> which calls an HTTP end-point which is
   capable of returning a user's email if an identity was provided.

Problem:

   1. ESI URLs can't contain any user specific parameters. If they do, the
   first user hit will be cached and every subsequent request will result in
   reflecting the first user's identity. This is not desired at all.
   2. Since ESI URLs do not contain any user specific parameters, the onus
   of adding user context lies with Varnish (since Varnish vcl_recv is where
   user's identity is removed).
   3. I tried setting user identity in a request header for the parent URL
   (as explained above in 1. vcl_recv). This header is lost while fetching ESI
   URLs.
   4. I tried setting a cookie in vcl_deliver. However, the cookie is
   inaccessible to the ESI URL fetch initiated during the same cycle. This is
   probably because the cookie is actually set *after* the first parent
   request completes and a response is sent back to the browser. I would want
   to make the cookie accessible *during* the parent request's ESI processing
   activity.

--

Nikhil Lanjewar

http://twitter.com/rhetonik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20131004/2607dc1d/attachment-0001.html>