Varnish and Load Balancing

[Juergen Schreck]

Aside from the application differences, we have a setup very similar to the one illustrated here (minus the Varnish servers currently):

http://www.lullabot.com/articles/varnish-multiple-web-servers-drupal

The websites are deployed sync'd on at least two nodes in the server farm and the load-balancer does it's thing for each site's nodes. We have two balancers setup with Linux-HA failover.

My understanding is that Varnish's built-in load balancer would do the same work, plus it would also provide the cache benefit.

So wouldn't Varnish actually replace the load-balancers then? Why would I want to load balance varnish servers?

Thanks,
Juergen




On Apr 3, 2012, at 9:35 PM, Travis Crowder wrote:

> 
> 
> On 4/3/2012 8:31 PM, Juergen Schreck wrote:
>> Hello -
>> 
>> I'm currently researching my options to implement varnish into our hosting recipe.
>> 
>> We provide load-balanced hosting services for Magento. We run a software load-balancer that handles all of our dedicated and VPS nodes. We've optimized each Magento instance such that we're using Apache/mod_php for the dynamic requests and nginx on a separate port and cookieless domain for all static assets. So we're not employing an reverse proxy methods at this point. Apache does all the php stuff and we're using memcached to help it out on the backend.
>> 
>> I could a Varnish server picking up a lot of slack of what we do there and improve upon it. I don't know that I'd want it to cache assets, because that'll be moving to a CDN. But with the Page Cache Powered By Varnish Magento module, I think it could work very nicely.
>> 
>> What I'm pondering is whether I should still run load-balancers in front of a Varnish server or let the Varnish built-in balancer do that work.
>> 
>> I'm also not sure how much a single Varnish server could provide with multiple dual-node vps backends and how I would scale it as the number of sites/vps grows.
>> 
>> Any suggestions to that layout? Right now it seems to me like I'd have some redundant puzzle pieces… Would appreciate your thoughts to help me sort it out.
>> 
>> Thanks,
>> Juergen
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
> 
> We currently use Varnish 3 to load balance several backend Drupal multi-sites (20 so far, about 70 by the end of the year) using the round robin director and it handles upwards of 2000+ connections per second using memory based storage with a 0.01 load on the server.  We use an F5 in front of Varnish to fail-over to a backup instance in case there is ever a problem and after several months of running, Varnish hasn't needed to go down or crashed once.
> 
> Scaling is pretty easy horizontally.  If we spin up another node, we just add it to the VCL and reload.
> 
> HTH,
> Travis