Varnish round-robin loadbalancing with Virtualhost (Namebased)

Mon Sep 2 18:03:16 CEST 2013

It's maybe possible to do it in VMOD, if you're not scared by developing in C.

You need to recreate the round-robin behavior by implementing yourself this feature.. and then let the VMOD selecting the right backend for you.

Jonathan Huot
Phone: +33(0)1.47.62.78.65

From: varnish-misc-bounces+jonathan.huot=thomsonreuters.com at varnish-cache.org [mailto:varnish-misc-bounces+jonathan.huot=thomsonreuters.com at varnish-cache.org] On Behalf Of Tharanga Abeyseela
Sent: Friday, 30 August 2013 02:56 PM
To: MAGNIEN, Thierry
Cc: Varnish misc
Subject: Re: Varnish round-robin loadbalancing with Virtualhost (Namebased)

That is my requirement !!!.  i'm using the same IP address for both hosts, and need to do the load-balancing between two servers. thats what i wanted to know,  Actually i'm playing with different varnish configs :) on my virtual box.,  thought to give this config a try....but that didnt work. it is working perfectly with different IP's, or IP based Vhosts.
Thanks Thierry for the response.
cheers,
Tharanga

On Fri, Aug 30, 2013 at 10:46 PM, MAGNIEN, Thierry <thierry.magnien at sfr.com<mailto:thierry.magnien at sfr.com>> wrote:
Hi,

I think I understood the point :
- you have 2 backends hosted under the same IP address : www.backend1.com<http://www.backend1.com> and www.backend2.com<http://www.backend2.com>
- your varnish server gets requests (for example) for www.frontend.com<http://www.frontend.com>
- you want requests to be load-balanced between your 2 backends BUT this would need the bereq.host to be rewritten to either www.backend1.com<http://www.backend1.com> or www.backend2.com<http://www.backend2.com>
- BUT you can't do this in your VCL because you don't know which backend will be selected.

In fact, this would need the director to rewrite the host header once the real backend has been selected, which I don't think is possible.

Anyway, I still wonder why the hell you need varnish for this, where a HAProxy would certainly do the trick ;-)

Regards,
Thierry

De : varnish-misc-bounces+thierry.magnien=sfr.com at varnish-cache.org<mailto:sfr.com at varnish-cache.org> [mailto:varnish-misc-bounces+thierry.magnien<mailto:varnish-misc-bounces%2Bthierry.magnien>=sfr.com at varnish-cache.org<mailto:sfr.com at varnish-cache.org>] De la part de Per Buer
Envoyé : vendredi 30 août 2013 13:36
À : Tharanga Abeyseela
Cc : Varnish misc
Objet : Re: Varnish round-robin loadbalancing with Virtualhost (Namebased)

Hi Tharanga,

You seem not to understand. Varnish will just pass the host header in the request to the backend you specify. Looking at the VCL you provided you put both the backends into one director. This is probably not what you want to do.

Try removing the director. Name the directors xxx and yyy and have the VCL look like.

sub vcl_recv {
    if (req.http.host ~ "xxx.com<http://xxx.com>") {
        req.backend = xxx;
    } else if (..

}
Also the next bit doesn't make sense:
if (req.request)
    {
    return(pass);
    }

}

On Fri, Aug 30, 2013 at 10:42 AM, Tharanga Abeyseela <tharanga.abeyseela at gmail.com<mailto:tharanga.abeyseela at gmail.com>> wrote:
Hi,

Actually i tried that link, still it is not working. it should work for a ip based virtual host, but namebased virtual host is not working.
becuase main request coming to varnish/80 and varnish should redirect the traffic to the backend server based on my round-robin director. when you resolve dns for those two hosts (xxx.com<http://xxx.com> and yyy.com<http://yyy.com>) it reuturns the same IP. that is 192.168.0.100. But there should be a way to send the host-header to apache. Not sure how it works here. But when i use different server (192,168.0.200/ zzz.com<http://zzz.com>) it works without any issue, it roundrobin the traffic. This is an issue with namebased virtualhost.
varnish recevie the traffic (cluster.com<http://cluster.com> is my  varnish server), then it use it vcl to redirect based on my round-robin setting. but how it can send the traffic to the correct virtualhost. because it gets only the same IP address from the DNS. I managed to change the default behaviour by forcefully setting the host-header as follows to my other virtualhost(yyy..com)
set.req.host="yyy.com<http://yyy.com>" ;
then all request went to yyy.com<http://yyy.com> instead of xxx.com<http://xxx.com>. but what i want is  a round-robin fashio...to send traffic. first req to xxx.com/<http://xxx.com/> 2nd req to yyy.com<http://yyy.com> and so on..

I was trying to find some resourses on the net, stackoverflow etc..but no luck yet.

Thanks again for your help
Cheers,
Tharanga

On Fri, Aug 30, 2013 at 5:05 PM, Per Buer <perbu at varnish-software.com<mailto:perbu at varnish-software.com>> wrote:
Hi Tharanga,

You seem to be a bit confused about how the directors and backends work in Varnish. Please read this:

https://www.varnish-cache.org/docs/trunk/users-guide/vcl-backends.html#backends-and-virtual-hosts-in-varnish

Note that the .host property of the backend has NOTHING to do with the virtual host.

On Fri, Aug 30, 2013 at 2:02 AM, Tharanga Abeyseela <tharanga.abeyseela at gmail.com<mailto:tharanga.abeyseela at gmail.com>> wrote:
Hi Per,
Thanks for the reply. Actually my set up looks like this.

cluster.com<http://cluster.com> - 192.168.0.200 (varnish/port 80)
xxx.com<http://xxx.com> - 192.168.0.100  (apache,namebased vhost/8080 - backendname - website)
yyy.com<http://yyy.com> - 192.168.0.100  (apache,namebased vhost/8080 -backendname - api)
cluster.com<http://cluster.com> is the varnish server and front-end connections coming to this and rewrite to other defined back-ends (round-robin based balancing)

backend website {
    .host = "xxx.com<http://xxx.com>";
    .port = "8080";

}
backend api {
    .host = "yyy.com<http://yyy.com>";
    .port = "8080";

}

director clust round-robin {
{ .backend = api;  }
{ .backend = website;  }

}

sub vcl_recv {
set req.backend = clust;
if (req.request)
    {
    return(pass);
    }

}

when i hit the cluster.com<http://cluster.com> , it is always going to xxx.com<http://xxx.com>, but what i need to do is first request go to xxx.com<http://xxx.com> second request yyy.com<http://yyy.com> and so on...when i add another server (different host/different IP say 192.168.0.111/zzz.com<http://192.168.0.111/zzz.com>, and a different backend) , it goes like this
first request - xxx.com<http://xxx.com>
second request - xxxx.com<http://xxxx.com>
third request - zzz.com<http://zzz.com>

but i can change the default behavior by setting up set req.host = yyy.com<http://yyy.com> and then it will goes to
first request - yyy.com<http://yyy.com>
second request - yyy.com<http://yyy.com>
third request - zzz.com<http://zzz.com>
this is something to do with the host-header forwarding to the correct back-end. how should i add that functionality to the vcl_recv ?
appreciate your help on this, this is working perfectly with other servers (different servers, not with namebased vhosts)
cheers,
Tharanga

On Thu, Aug 29, 2013 at 11:58 PM, Per Buer <perbu at varnish-software.com<mailto:perbu at varnish-software.com>> wrote:
Hi mate,

On Thu, Aug 29, 2013 at 3:17 PM, Tharanga Abeyseela <tharanga.abeyseela at gmail.com<mailto:tharanga.abeyseela at gmail.com>> wrote:
(..)

But how should i send host-headers to varnish to redirect to the correct server, This is always going to xxx.com<http://xxx.com>.

You don't. :-)

You need to dispatch the request to the right backend in vcl_recv and have the backend handle that host. You could start rewriting the host in vcl_recv, but I would advise against it.

So, if you have two vhosts handled by "website" you just set that as the backend in vcl_recv.

--
Per Buer
CTO | Varnish Software AS
Phone: +47 958 39 117<tel:%2B47%20958%2039%20117> | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013

--

Per Buer
CTO | Varnish Software AS
Phone: +47 958 39 117<tel:%2B47%20958%2039%20117> | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013

--

Per Buer
CTO | Varnish Software AS
Phone: +47 958 39 117<tel:%2B47%20958%2039%20117> | Skype: per.buer
We Make Websites Fly!

Winner of the Red Herring Top 100 Europe Award 2013

This email was sent to you by Thomson Reuters, the global news and information company. Any views expressed in this message are those of the individual sender, except where the sender specifically states them to be the views of Thomson Reuters.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20130902/05d60623/attachment.html>