Varnish w/hardware loadbalancing woes

[Copper, Candace L]

Thanks again Michael - I was able to gather some more information regarding our VIP today - here is what was given to me:

VIP-
server virtual perf-drupal.domain.com 192.168.X.X  sym-priority 4  predictor round-robin  port http sticky  port ssl sticky  bind http server1 http server2 http server3 http  bind ssl server1 ssl server2 ssl server3 ssl

Real Server(s) - 

server real server1 192.168.X.X
 port http                                                        
 port http keepalive
 port http url "HEAD /"
 port ssl
 port ssl keepalive
!
server real server2 192.168.X.X
 port http
 port http keepalive
 port http url "HEAD /"
 port ssl
 port ssl keepalive
!
server real server3 192.168.X.X
 port http
 port http keepalive
 port http url "HEAD /"
 port ssl
 port ssl keepalive


So the way we have it configured now it goes: VIP (hardware based) -> Server1, 2 or 3 -> Varnish on Server1, 2 or 3 (listening on port 80) -> Apache (listening on port 8080). Varnish is running on Server1, 2 and 3 as 80 and it points to apache on 8080. 

Hope this helps some.

Candace Copper 


-----Original Message-----
From: Michael Loftis [mailto:mloftis at wgops.com] 
Sent: Thursday, September 23, 2010 4:45 PM
To: Copper, Candace L; 'varnish-misc at varnish-cache.org'
Subject: RE: Varnish w/hardware loadbalancing woes

Well if you could diagram or walk-through your specific setup and configs I 
can certainly help you (I've deployed probably a couple hundred or so 
various load balanced applications using both hardware and software LB 
approaches, and with approaches with proxies like varnish too)

The *only* machine directly receiving traffic for a VIP should be the 
hardware load balancer itself.  It will then decide which (real) machine 
(in your case apparently running varnish on top of ?) to send the traffic 
on to.  If the real machines are answering for the VIP because they're all 
sharing a VLAN/LAN then you're going to get failures randomly.

Generally the VIP is configured on a loopback device as a /32 (or /128 in 
IPv6) -- return traffic can either be DR or via the LB itself.  in the 
latter you might be doing NAT in which case the real machines won't have 
the VIP configured at all.  In a DR situation the real machines have to 
have the VIP because they'll send all traffic back to their (default) 
router, not the load balancer.

--On Thursday, September 23, 2010 4:23 PM -0700 "Copper, Candace L" 
<Candace.Copper at goAAA.com> wrote:

> Michael, had the ARP disabled on the VIP and then the site wouldn't load
> at all with the VIP and still saw the Unable to connect errors. We
> re-enabled ARP and then our site still wouldn't load, and we couldn't
> kill any httpd processes and had to reboot.
>
> Just weird stuff happened with that.
>
> Candace Copper
>
>
> -----Original Message-----
> From: varnish-misc-bounces at varnish-cache.org
> [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Michael
> Loftis Sent: Thursday, September 23, 2010 10:57 AM
> To: 'varnish-misc at varnish-cache.org'
> Subject: Re: Varnish w/hardware loadbalancing woes
>
> Sounds more like a load balancer setup problem than a varnish issue.
> Under  Linux atleast with DR setups you must disable ARP for the VIP,
> unless  you're using a tunnel setup or a setup where your VIPs are
> completely  separate VLAN/LAN.
>
> It sounds like this is what's happening, that the real servers are
> receiving the traffic sometimes and the load balancer at others because
> of  the ARP issue.
>
> --On Thursday, September 23, 2010 9:21 AM -0700 "Copper, Candace L"
> <Candace.Copper at goAAA.com> wrote:
>
>>
>>
>> I've looked all over for information on how to configure varnish to use
>> a VIP and have not been able to find much – so I'm hoping someone
>> here can assist.
>>
>>
>>
>> Our setup goes like this: Hardware Loadbalancer (sticky sessions enabled)
>> with one VIP(perf-drupal.domain.com) directs to three Apache identical
>> webservers (each running Varnish – (perf-drupal1.domain.com,
>> perf-drupal2.domain.com & perf-drupal3.domain.com)). With the VCL that we
>> currently have, it works like a charm on each individual server, but when
>> we try to use the VIP we get errors (50% of the time) when trying to
>> access the site, stating that it is not available.
>>
>>
>>
>> I've tried:
>>
>>
>>
>> backend newsite {
>>
>>     .host = "localhost";
>>
>>     .port = "8080";
>>
>> }
>>
>> set req.http.host = "perf-drupal.domain.com";
>>
>> set req.backend = newsite;
>>
>>
>>
>> and using the default backend:
>>
>>
>>
>> backend default {
>>
>>   .host = "127.0.0.1";
>>
>>   .port = "8080";
>>
>> }
>>
>>
>>
>> I've read about the DNS Director, but that's not available in the
>> version I have installed - 2.1.3. But since we are only running one site,
>> I don't know if it will help. I don't have access to any additional
>> hardware, so I cannot split out Varnish from the Apache web server.
>>
>>
>>
>> Any assistance would be greatly appreciated!  J
>>
>>
>>
>> Candace Copper
>>
>>
>>
>
>
>
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc