Varnish w/hardware loadbalancing woes

[Michael Loftis]

--On Monday, September 27, 2010 11:20 AM -0700 "Copper, Candace L" 
<Candace.Copper at goAAA.com> wrote:

> Thanks again Michael - I was able to gather some more information
> regarding our VIP today - here is what was given to me:
>

As has been pointed out, varnish cant/wont do SSL.  Its not clear what your 
network topology looks like from this.  Have you verified sessions are all 
setting up on the load balancer?  And that the real servers are indeed NOT 
responding to ARP requests for the shared VIP?

> VIP-
> server virtual perf-drupal.domain.com 192.168.X.X  sym-priority 4
> predictor round-robin  port http sticky  port ssl sticky  bind http
> server1 http server2 http server3 http  bind ssl server1 ssl server2 ssl
> server3 ssl
>
> Real Server(s) -
>
> server real server1 192.168.X.X
>  port http
>  port http keepalive
>  port http url "HEAD /"
>  port ssl
>  port ssl keepalive
> !
> server real server2 192.168.X.X
>  port http
>  port http keepalive
>  port http url "HEAD /"
>  port ssl
>  port ssl keepalive
> !
> server real server3 192.168.X.X
>  port http
>  port http keepalive
>  port http url "HEAD /"
>  port ssl
>  port ssl keepalive
>
>
> So the way we have it configured now it goes: VIP (hardware based) ->
> Server1, 2 or 3 -> Varnish on Server1, 2 or 3 (listening on port 80) ->
> Apache (listening on port 8080). Varnish is running on Server1, 2 and 3
> as 80 and it points to apache on 8080.
>
> Hope this helps some.
>
> Candace Copper
>
>
> -----Original Message-----
> From: Michael Loftis [mailto:mloftis at wgops.com]
> Sent: Thursday, September 23, 2010 4:45 PM
> To: Copper, Candace L; 'varnish-misc at varnish-cache.org'
> Subject: RE: Varnish w/hardware loadbalancing woes
>
> Well if you could diagram or walk-through your specific setup and configs
> I  can certainly help you (I've deployed probably a couple hundred or so
> various load balanced applications using both hardware and software LB
> approaches, and with approaches with proxies like varnish too)
>
> The *only* machine directly receiving traffic for a VIP should be the
> hardware load balancer itself.  It will then decide which (real) machine
> (in your case apparently running varnish on top of ?) to send the traffic
> on to.  If the real machines are answering for the VIP because they're
> all  sharing a VLAN/LAN then you're going to get failures randomly.
>
> Generally the VIP is configured on a loopback device as a /32 (or /128 in
> IPv6) -- return traffic can either be DR or via the LB itself.  in the
> latter you might be doing NAT in which case the real machines won't have
> the VIP configured at all.  In a DR situation the real machines have to
> have the VIP because they'll send all traffic back to their (default)
> router, not the load balancer.
>
> --On Thursday, September 23, 2010 4:23 PM -0700 "Copper, Candace L"
> <Candace.Copper at goAAA.com> wrote:
>
>> Michael, had the ARP disabled on the VIP and then the site wouldn't load
>> at all with the VIP and still saw the Unable to connect errors. We
>> re-enabled ARP and then our site still wouldn't load, and we couldn't
>> kill any httpd processes and had to reboot.
>>
>> Just weird stuff happened with that.
>>
>> Candace Copper
>>
>>
>> -----Original Message-----
>> From: varnish-misc-bounces at varnish-cache.org
>> [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Michael
>> Loftis Sent: Thursday, September 23, 2010 10:57 AM
>> To: 'varnish-misc at varnish-cache.org'
>> Subject: Re: Varnish w/hardware loadbalancing woes
>>
>> Sounds more like a load balancer setup problem than a varnish issue.
>> Under  Linux atleast with DR setups you must disable ARP for the VIP,
>> unless  you're using a tunnel setup or a setup where your VIPs are
>> completely  separate VLAN/LAN.
>>
>> It sounds like this is what's happening, that the real servers are
>> receiving the traffic sometimes and the load balancer at others because
>> of  the ARP issue.
>>
>> --On Thursday, September 23, 2010 9:21 AM -0700 "Copper, Candace L"
>> <Candace.Copper at goAAA.com> wrote:
>>
>>>
>>>
>>> I've looked all over for information on how to configure varnish to use
>>> a VIP and have not been able to find much – so I'm hoping someone
>>> here can assist.
>>>
>>>
>>>
>>> Our setup goes like this: Hardware Loadbalancer (sticky sessions
>>> enabled) with one VIP(perf-drupal.domain.com) directs to three Apache
>>> identical webservers (each running Varnish – (perf-drupal1.domain.com,
>>> perf-drupal2.domain.com & perf-drupal3.domain.com)). With the VCL that
>>> we currently have, it works like a charm on each individual server, but
>>> when we try to use the VIP we get errors (50% of the time) when trying
>>> to access the site, stating that it is not available.
>>>
>>>
>>>
>>> I've tried:
>>>
>>>
>>>
>>> backend newsite {
>>>
>>>     .host = "localhost";
>>>
>>>     .port = "8080";
>>>
>>> }
>>>
>>> set req.http.host = "perf-drupal.domain.com";
>>>
>>> set req.backend = newsite;
>>>
>>>
>>>
>>> and using the default backend:
>>>
>>>
>>>
>>> backend default {
>>>
>>>   .host = "127.0.0.1";
>>>
>>>   .port = "8080";
>>>
>>> }
>>>
>>>
>>>
>>> I've read about the DNS Director, but that's not available in the
>>> version I have installed - 2.1.3. But since we are only running one
>>> site, I don't know if it will help. I don't have access to any
>>> additional hardware, so I cannot split out Varnish from the Apache web
>>> server.
>>>
>>>
>>>
>>> Any assistance would be greatly appreciated!  J
>>>
>>>
>>>
>>> Candace Copper
>>>
>>>
>>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>
>
>
>
>