[Varnish] #1481: directors vmod needs a simpler way to declare a massive set of backends

[Varnish]

#1481: directors vmod needs a simpler way to declare a massive set of backends
--------------------------+-------------------------
 Reporter:  mattrobenolt  |       Type:  enhancement
   Status:  new           |   Priority:  normal
Milestone:                |  Component:  vmod
  Version:  4.0.0         |   Severity:  normal
 Keywords:  directors     |
--------------------------+-------------------------
 In 3.0, declaring a long list of backend servers for a director was
 trivial. An except from one of our VCLs looks like the following:

 {{{
 director apps random {
     { .backend = { .host = "app-1.i"; .probe = isup; } .weight = 50; }
     { .backend = { .host = "app-2.i"; .probe = isup; } .weight = 100; }
     { .backend = { .host = "app-3.i"; .probe = isup; } .weight = 100; }
     { .backend = { .host = "app-4.i"; .probe = isup; } .weight = 100; }
     { .backend = { .host = "app-5.i"; .probe = isup; } .weight = 100; }
     { .backend = { .host = "app-6.i"; .probe = isup; } .weight = 100; }
     { .backend = { .host = "app-7.i"; .probe = isup; } .weight = 100; }
     { .backend = { .host = "app-8.i"; .probe = isup; } .weight = 100; }
     { .backend = { .host = "app-9.i"; .probe = isup; } .weight = 100; }
     # ... 100 more
 }
 }}}

 In 4.0, the amount of  lines has effective doubled and introduces the fact
 that each backend needs to have a proper name making using any automated
 tooling trickier and much more difficult to work with.

 For example:

 {{{
 backend app1 { .host = "app-1.i" .probe = isup; }
 backend app2 { .host = "app-2.i" .probe = isup; }
 backend app3 { .host = "app-3.i" .probe = isup; }
 backend app4 { .host = "app-4.i" .probe = isup; }
 backend app5 { .host = "app-5.i" .probe = isup; }
 backend app6 { .host = "app-6.i" .probe = isup; }
 backend app7 { .host = "app-7.i" .probe = isup; }
 backend app8 { .host = "app-8.i" .probe = isup; }
 backend app9 { .host = "app-9.i" .probe = isup; }
 # ... 100 more

 sub vcl_init {
     new apps = backend.random();
     apps.add_backend(app1, 100);
     apps.add_backend(app2, 100);
     apps.add_backend(app3, 100);
     apps.add_backend(app4, 100);
     apps.add_backend(app5, 100);
     apps.add_backend(app6, 100);
     apps.add_backend(app7, 100);
     apps.add_backend(app8, 100);
     apps.add_backend(app9, 100);
     # ... 100 more
 }
 }}}

 This list of servers is typically managed by some configuration management
 system, such as puppet or chef. In the previous model, it's trivial to
 iterate over a list of servers and repeat the same line and just swap out
 the hostname. In the new model, a unique name for the backend needs to be
 generated that needs to then pair up with what's inside `vcl_init`.

 Now, I understand the advantages of moving directors to a vmod, so I'm not
 arguing against that.

 I'd suggest more functions on the vmod that take parameters as input to
 create the backends automatically.

 I don't think it's possible in current vmod syntax to support something
 like: `director.add_backend({ .host = "foo" }, 1)`

 Something like `director.add_anonymous_backend` could probably fill that
 gap. `add_anonymous_backend` would look something like:
 `director.add_anonymous_backend("app-1.i", ...)`

 As it stands today, this just made switching to 4.0 an almost deal breaker
 in my opinion.

-- 
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1481>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator