Using Varnish as a backend for another Varnish server, with ESIs

[AD]

I would personally solve this another way as I dont think its worth all this
pain just to save 1 extra hit.  What i would suggest is you use a "hash"
algorithm on your load balancer that sends the same URL to the same varnish
server, might be easier.

On Wed, Aug 17, 2011 at 3:51 PM, Kolychev, Aleksey <
akolychev at marthastewart.com> wrote:

>  Are you load-balancing between the two varnish instances? If you are, you
> should check if you’re hashing the cache keys properly, e.g., not sending
> the same response to the other varnish instance that you’d send to all other
> clients, so that your outer varnish instance doesn’t serve cached pages with
> unparsed ESI tags to clients.****
>
> ** **
>
> You could try doing something along the lines of –****
>
> ** **
>
> sub vcl_recv {****
>
>   if(client.ip ~ other-varnish) {****
>
>     set req.http.X-Via-Varnish = 1;****
>
>   }****
>
> }****
>
> ** **
>
> sub vcl_hash {****
>
>   set req.hash += req.http.X-Via-Varnish;****
>
> }****
>
> ** **
>
> sub vcl_fetch {
>   if (beresp.http.X-Meta-Has-Esi) {
>       unset beresp.http.X-Meta-Has-Esi;
>   } else {****
>
>       esi;****
>
>   }
> }****
>
> ** **
>
> (Disclaimer – I didn’t test this, and cannot be held responsible for syntax
> errors)****
>
> ** **
>
> --****
>
> Lex Kolychev****
>
> Application and Systems Architect****
>
> Internet Technology Dept.****
>
> Martha Stewart Living Omnimedia
> Ph:   212-827-8948****
>
>
>
> ****
>
> *From: *Pax Dickinson <pax at businessinsider.com>****
>
> *Date: *August 17, 2011 2:38:16 PM EDT****
>
> *To: *"varnish-misc at varnish-cache.org" <varnish-misc at varnish-cache.org>***
> *
>
> *Subject: Using Varnish as a backend for another Varnish server, with ESIs
> *****
>
> ** **
>
> Hi,
>
> I'm attempting to use two varnish servers as the preferred caching
> backend for each other, falling back to my apache backends if neither
> varnish server has the content in cache. I'm doing it to reduce my
> backend traffic and keep my caches hot more efficiently than having
> each varnish server totally separate, which is what I do now. Also
> this will scale better if the day comes that I need a third server.
>
> I match against an ACL and if a request comes from anywhere other than
> my other varnish server, I set the varnish backend. This is working
> fine, except my edge side includes need to be processed only by the
> server that received the original request. If the secondary server
> were to process and include my ESIs at that point they would be
> returned to the primary server and become part of its cached copy, and
> that defeats the purpose of having ESIs.
>
> It seems that it should be as simple as:
>
> sub vcl_recv {
>   if (!client.ip ~ othervarnish) {
>       set req.backend = varnish;
>   }
> }
>
> sub vcl_fetch {
>   if (beresp.http.X-Meta-Has-Esi && !client.ip ~ othervarnish) {
>       esi;
>   }
> }
>
> BUT.. If I call it as above, ESIs work and are processed correctly by
> whichever server received the request first. However, when hitting the
> secondary server and getting a cached result the ESIs are not
> processed, vcl_fetch is never called, and the esi tags remain in the
> result. Also, it seems that the esi function can only be called in
> vcl_fetch. In vcl_deliver it does nothing.
>
> So for example:
>
> The initial request works:
>
> Request ---> VarnishA MISS --> VarnishB MISS --> Backend (success with
> esi parsed)
>
> However subsequent requests look like this:
>
> New Request ---> VarnishA HIT (success with esi parsed)
> New Request ---> VarnishB HIT (fail, esi will not be parsed)
>
> Is there a way to make this work? Or is using varnish as a backend for
> another varnish incompatible with edge side includes? I'm using
> varnish 2.1.5, is this possible in 3.0?
>
> Thanks..
>
> --
> Pax Dickinson
> Chief Architect
> Business Insider
> 257 Park Avenue South, 13th floor
> New York, NY 10010
> pax at businessinsider.com
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc****
>
> ** **
>
> ------------------------------
> Don't miss a brand new season of the Emmy award-winning "Martha Stewart
> Show", followed by the all-new series "Emeril's Table"...Beginning September
> 26, only on Hallmark Channel.
>
> Find Martha Stewart on Facebook: http://facebook.com/MarthaStewartLiving
>
> Follow Martha Stewart on Twitter: http://twitter.com/MarthaStewart
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110817/a0309b0a/attachment-0003.html>