ESI substitutions, rewrite VMOD & friends

[Carlos Abalde]

Hi all,

I want to do something with Varnish Cache that probably is not possible. Anyway, I'm wondering if maybe I'm missing some possible workaround, or maybe there is something planned for future releases that could be useful. At the moment this is more a theoretical than a real thing, but I'm curious about it.

I think that the real use case could look a little bit weird, so let me describe something similar, still weird, but sightly simpler and easier to explain :)


For example, suppose you have some kind of key-value storage integrated with Varnish Cache (using the Memcached VMOD or anything similar). During vcl_deliver, you check the incoming URL, and if it matches some pattern, you hash a canonical representation of the URL and use that value to fetch some data from the key-value storage (it could be a small JSON string containing meta-information about the URL: authors, date... whatever). Let's assume the JSON strings are inserted in the storage by some other backend component not relevant to the discussion.

So, at some point during vcl_deliver you have the HTML response ready to be delivered to the client side, and you also have some JSON data fetched from Memached you need to deliver to the client side, together with the HTML. In other words, the JSON contents are designed to be consumed in the client side using some JavaScript, and you need to attach them to the HTML.


I'm wondering what are the options to do something like that in Varnish:

  - Someone could think about using some ad hoc HTTP header to transport the JSON value to the client side. However you cannot access HTTP headers using JavaScript. That's only possible during AJAX requests.

  - Someone could think about using a second AJAX request to fetch the JSON contents (i.e. execute the Memached GET request later), but doing extra requests is not an option.

  - Someone could think about using a temporary cookie to transport the JSON value to the client side (basically, an HTTP header that is readable using JavaScript). Some JavaScript could be used to extract the JSON data from the cookie and immediately remove the cookie. Sadly this is not going to work properly in the presence of concurrency in the client side.

  - Someone could suggest using the Rewrite VMOD and embed in the HTML some JavaScript + JSON, but I'm not sure about how stable / reasonable is that implementation, or even if it is possible to migrate that logic to Varnish 4.

  - Similar to the previous idea, someone could suggest using an ESI substitution, but I'm afraid that's not implemented in Varnish (I'm assuming here that is could be possible to add the ESI placeholder in all pages generated by the backend).


Any other idea? :)

Cheers,

—
Carlos Abalde.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20140630/034ebca1/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 203 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20140630/034ebca1/attachment.pgp>