dns director efficiency on large number of hosts?

[Fred Clift]

So I'm setting up a test varnish server, and have configured configured a
director block that looks something like:

director www dns {
    .list = {
            .port = "80";
            "10.1.55.165/32";
            "10.1.54.126/32";
            "10.1.222.1/32";
        }
    .ttl = 5m;
    .suffix = "";
}

(IP addressed changed to protect the innocent)


There is a valid DNS entry for each of 3 domains, each pointing at one of
those addresses.

When testing, this appears to do what I would like - which is cache for 3
unrelated sites.

For now, I change my local resolver to, for my test client, point the
domains at the varnish server, while having a .suffix of "" means they'll
look up the real public IPs of the semi-live sites and it will cache those
sites for just me.

Eventually I will point public DNS for those sites at the varnish server
and maintain a domain-to-IP mapping in my DNS server (hm /etc/hosts on
varnish server?  nsswitch works right?)

Ideally I'd like to just load a config file with public.dns.name./
backend.dns.name. mappings  or public.dns.name./IP mappings but I've found
several archived discussions of this (e.g. dns resolved at vcl compilation
time...) that doesn't show promise for now.  I guess I'll just have to
update the config (scripted) if a domain changes back-end IP address.

My real question is:  How efficient is this for a large number of hosts?
 say 1K?  10K?  Is the hostname to IP lookup efficient?

I know that having many sites all cached separately will not be great for
cache hit rate.... I believe that if it becomes a problem I can throw more
ram and SSD swap at that problem, to some extent anyway.


I guess an alternative for me would be to write my own director code that
included some file with hostname/backend mappings and compiled it into the
VCL along with a simple hash.  or maybe just some inline C that would allow:

sub vcl_recv {
    req.backend = myDictionaryLookup(req.http.host)
...
}

rather than having to do 1000 if/elseif/elseif blocks....

Is there some other easier way to do this?  would the DNS director perform
adequately?

Thanks.

Fred Clift
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20120503/cbe3729c/attachment.html>