<div dir="ltr">So somebody more skilled than myself might be able to offer a better example, but I believe a multi-varnish reverse-proxy setup requires two layers, which I'll call a "frontend" and "backend". These are two separate varnish processes running on different ports. <div>
<br></div><div style>Your frontend.vcl has knowledge of every one of your varnish caching server's backend processes, and is set to use the hash director. It doesn't do any caching and has no knowledge of a purge. Your backend.vcl contains information on the web server you are acting as a reverse proxy against and does all of the caching.</div>
<div style><br></div><div style>Request go from your LB to your frontend on <i>any</i> varnish server, it doesn't matter which one. The request is then hashed to a <i>single</i> one of your varnish servers's backends. The backend process can be on the server itself, or on another server in the cluster. If it's a hit, that server serves it up. If it's a miss it goes to your backend.</div>
<div style><br></div><div style>Does this make sense? I'm certainly not an expert on this. The idea is that the object is only stored once on your cluster, and any request to purge it will only get sent to the server that might have it in its cache.</div>
<div style><br></div><div style>Good luck.</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, May 29, 2013 at 11:00 AM, Puneet <span dir="ltr"><<a href="mailto:puneet.arora@insticator.com" target="_blank">puneet.arora@insticator.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple"><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">The VCL subroutines are following:<u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></b></p><p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">sub vcl_recv {<u></u><u></u></span></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> if (req.request == "PURGE") {<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> if (!client.ip ~ purge) {<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> error 405 "Not allowed.";<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> }<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> <u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> return(lookup);<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> }<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">//// rest of the code<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">}<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">sub vcl_miss {<u></u><u></u></span></b></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">##purge content from cache only via localhost<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> if (req.request == "PURGE") {<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> purge;<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> error 404 "Not in cache.";<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> }<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">return (fetch);<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">}<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">sub vcl_hit {<u></u><u></u></span></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> if (req.request == "PURGE") {<u></u><u></u></span></p><p class="MsoNormal">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> purge;<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> error 200 "Purged.";<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> } <u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> return (deliver);<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> }<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">sub vcl_hash {<u></u><u></u></span></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">hash_data(req.url);<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> if (req.http.host) {<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> hash_data(req.http.host);<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> } else {<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> hash_data(server.ip);<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> }<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> return (hash);<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">}<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Thanks<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Puneet<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> Stephen Wood [mailto:<a href="mailto:smwood4@gmail.com" target="_blank">smwood4@gmail.com</a>] <br>
<b>Sent:</b> Wednesday, May 29, 2013 1:32 PM</span></p><div><div class="h5"><br><b>To:</b> Puneet<br><b>Cc:</b> <a href="mailto:varnish-misc@varnish-cache.org" target="_blank">varnish-misc@varnish-cache.org</a><br><b>Subject:</b> Re: PURGING objects on multiple instances of varnish<u></u><u></u></div>
</div><p></p><div><div class="h5"><p class="MsoNormal"><u></u> <u></u></p><div><p class="MsoNormal">What does your VCL look like?<u></u><u></u></p></div><div><p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
<div><p class="MsoNormal">On Wed, May 29, 2013 at 9:50 AM, Puneet <<a href="mailto:puneet.arora@insticator.com" target="_blank">puneet.arora@insticator.com</a>> wrote:<u></u><u></u></p><blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi Stephen,</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Yes the Varnish will use hash_director.</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">The Load Banalcer is executing in Round Robin manner. The LB is connected to 4 instances of Varnish which will be given requests in Round Robin fashion.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Now, in order to send the request to Varnish directly, I need the host to be the IP & Port of the varnish instances.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">i.e. let I have ONE varnish instance at 192.168.0.23 at port 3002</span><u></u><u></u></p><p class="MsoNormal">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">so the request will be for “ –XPURGE <a href="http://192.168.0.23:3002/%3cpath-to-object" target="_blank">http://192.168.0.23:3002/<path-to-object</a>> “</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">By, doing that Varnish gives a response stating “405 Object Not in Cache”</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Thanks</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Puneet</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> Stephen Wood [mailto:<a href="mailto:smwood4@gmail.com" target="_blank">smwood4@gmail.com</a>] <br>
<b>Sent:</b> Tuesday, May 28, 2013 5:42 PM<br><b>To:</b> Puneet<br><b>Cc:</b> <a href="mailto:varnish-misc@varnish-cache.org" target="_blank">varnish-misc@varnish-cache.org</a><br><b>Subject:</b> Re: PURGING objects on multiple instances of varnish</span><u></u><u></u></p>
<div><div><p class="MsoNormal"> <u></u><u></u></p><div><div><p class="MsoNormal">Generally speaking, the varnish setup you are describing would use a hash director and not round-robin, so that the object would only be cached on one of your varnish nodes and therefore only require a single purge request.<u></u><u></u></p>
</div><div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal">If your object lives on multiple nodes, then you could do something simple like send a HTTP request to <i>every</i> one of your hosts. This would require you to keep a list of your hosts and also create the appropriate permissions on all of your varnish instances: <u></u><u></u></p>
</div><div><p class="MsoNormal"> <u></u><u></u></p></div><p class="MsoNormal"><span style="font-family:"Courier New"">$ for i in `cat varnish_host.list`; do curl -XPURGE </span><a href="http://$i/object;done" target="_blank"><span style="font-family:"Courier New"">http://$i/object;done</span></a><u></u><u></u></p>
<div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal" style="margin-bottom:12.0pt">I hope that helps. Maybe somebody can offer a better solution than myself.<u></u><u></u></p><div><p class="MsoNormal">
On Tue, May 28, 2013 at 2:17 PM, Puneet <<a href="mailto:puneet.arora@insticator.com" target="_blank">puneet.arora@insticator.com</a>> wrote:<u></u><u></u></p><blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div><div><p class="MsoNormal">Hi All,<u></u><u></u></p><p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal">I am using a Load Balancer (LB) in front of Varnish instances. Just say the LB is listening on Port: 80 and there are multiple instances of Varnish on different port nos.<u></u><u></u></p>
<p class="MsoNormal">i.e. I have a single LB which is connected to 4 instances of varnish, and LB is executing in a round-robin fashion. <u></u><u></u></p><p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal">Now, how can I purge objects on Varnish and maintain consistency among different instances of Varnish.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p><p class="MsoNormal">Thanks<u></u><u></u></p><p class="MsoNormal"><span style="color:#888888">Puneet</span><u></u><u></u></p><p class="MsoNormal"><span style="color:#888888"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="color:#888888"> </span><u></u><u></u></p></div></div><p class="MsoNormal"><br>_______________________________________________<br>varnish-misc mailing list<br><a href="mailto:varnish-misc@varnish-cache.org" target="_blank">varnish-misc@varnish-cache.org</a><br>
<a href="https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc" target="_blank">https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc</a><u></u><u></u></p></blockquote></div><p class="MsoNormal"> <u></u><u></u></p>
</div></div></div></div></div></div></blockquote></div><p class="MsoNormal"><u></u> <u></u></p></div></div></div></div></div></blockquote></div><br></div>