Lots of configs

[Robert Shilston]

On 9 Mar 2011, at 08:18, Poul-Henning Kamp wrote:

> In message <AANLkTi=cCgmWvH77r_CVyv01+hp0uic2YAQQH5Jqby+4 at mail.gmail.com>, AD w
> rites:
> 
>> As the OP, i would like to get the discussion on this thread back to
>> something useful.  That being said...
> 
> Arthur and I brainstormed this issue on our way to cake after VUG3
> and a couple of ideas came up which may be worth looking at.
> 
> At the top-heavy end, is having VCL files tell which domains they
> apply to, possibly something like:
> 
> 	domains {
> 		"*.example.com";
> 		! "notthisone.example.com";
> 		"*.example.biz";
> 	}

I was chatting to a Varnish administrator at a PHP conference in London a couple of weeks ago.  They run Varnish for a very high profile site which has lots of sub-sites that have delegated web teams.  So, for example, all traffic to www.example.com hits Varnish, and www.example.com/alpha is managed by a completely separate team to www.example.com/beta.  Thanks to Varnish, each base path can be routed to different backends.  However, the varnish behaviour itself is different for different paths.  My understanding is that each team submits their VCL to the central administrator who sticks it together, and that each path/site has a separate set of vcl_* functions.

Whilst I obviously don't know exactly how they're doing this, I think that this different level of behaviour splitting would be worth considering as part of these discussions.  So, perhaps it might make sense to have individual VCL files that declare what they're interested in, such as:

==alpha.vcl==
appliesto {
   "alpha";
   "alpha.example.com";
   "alpha.example.co.uk";
}
sub vcl_recv {
  set req.backend = alphapool;
}
==

and then in the main VCL, do something pseudo-code like:

==default.vcl==
include "alpha.vcl"
sub vcl_recv {

  if (req.http.host == "www.example.com") {

     /* Do some regex to find the first part of the path, and see if there's a valid config for it */
     callconfig(reg_sub(req.url, "/(.*)(/.*)?", "\1"));

  } else {

     /* Try to see if there's a match for this hostname */
     callconfig(req.http.host);
  }

  /* By this point, nothing has matched, so call some default behaviour */
  callconfig("default");

}
==

So, callconfig effectively works a bit like the current 'return' statement, but only if a config that 'appliesto' the defined string is found in a config - once the config is called, no further code in the calling function is executed.  By detaching this behaviour from the concept of a "domain" in PHK's example, then this pattern could be used for a wider range of scenarios - perhaps switching based on the requestor's IP / ACL matches or whatever else Varnish users might need.



Rob