Large number of (dynamic?) backends

[Jason Price]

I've written a custom program to generate backends based off DNS records
(It's deeply custom to our use case, so sharing it probably won't help you
too much).

In short it is called by cron every minute.  It wakes up, gathers all the
IPs of all the various backends, and builds the backend definition for
them, and wraps each backend up in a director [1].

If the resulting file is different from the current one, swap the file out,
and replace it with the new one, then call 'service varnish reload'.  Every
hour another cron job wakes up and runs 'varnishadm vcl.discard <inactive
configs>'.

This process works fairly well at high load, and doesn't flush the cache.
It also relies on probe definitions requiring that new IPs pass one health
check (at least) before traffic is sent to them.

-Jason

[1] Alternatively, in AWS it could wake up, scan the instances for various
tag attributes and rely on those as backends.  I've used that formula with
success as well.

On Mon, Mar 30, 2015 at 1:57 PM, Kevin Day <toasty at dragondata.com> wrote:

> I have a use case where I need to be able to take an incoming request like
> “/XXX/filename” and translate that to fetching “
> http://nodeXXX.example.com/filename” where there’d literally be hundreds
> of backend servers. i.e. XXX would be a varying 3 digit number.
>
> I saw this post here:
>
>
> https://www.varnish-software.com/blog/truly-dynamic-backend-definition-varnish
>
> From 2012, but I know a lot has changed since then. Is there an easy way
> this can be accomplished now?
>
> — Kevin
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20150330/416f8b9e/attachment.html>