a single varnish, multiple sites, and a probe for each

[Michael Alger]

On Thu, Oct 13, 2011 at 11:23:41AM +0100, Dan wrote:
> I am trying to setup varnish to allow for a healthcheck on each site
> even though multiple sites are on the same server. This would mean
> that if one sites probe.php failed, the other sites wouldn't go down
> (unless their probes also failed). I have tried to get started on
> this, but am finding it hard to get a reliable solution, I am
> thinking that if each site has it's own directer, with the director
> pointing to the same backend, then this should get the director to
> only fail if it's site is down

The probe is associated with the backend, not the director. Looking at
the configuration you posted on the forum, how do you imagine Varnish
would be able to determine that you have X number of sites all of
which have a probe.php and need to be monitored independently?

> This assumption is of course not working, so I am trying to work out
> why, and whether there are any docs on this.  I have posted this
> onto the forum, but was advised that this may be a better place to
> post.  I have attached a link to the forum where there is also some
> code snippets
> 
> Link to forum post: https://www.varnish-cache.org/forum/topic/166

Directors are used to group multiple backends for the same site
together for failover and load balancing purposes. If you only have
the one backend server for a particular site, you don't need to use
directors at all.

You want to use a separate backend definition for every site you host.
Each backend has its own health status which is updated using the
probe configured for that backend.

For your purposes, you probably want to use the alternate method of
specifying the probe request, which lets you control all the headers
sent in the probe. It'll look something like this:

  .probe = {
    .request =
      "GET /probe.php HTTP/1.0"
      "Host: www.virtualhost.example.com"
      "Connection: close";
    .interval = 5s;
    .timeout = 1 s;
  }

In this case, I'm assuming your backend does "name-based" virtual
hosting so you need to be able to send the appropriate Host: header in
order to check the correct site.

Just in case it's not clear: you can have multiple Varnish backends
configured for a single real server. They don't even need to be
different, although if they're identical they'll both get sick/healthy
at the same time so it won't serve much purpose. But, you might probe
a different path in order to test different parts of the site's
functionality.

As an example, if you have a complex site hosting a store and a web
forum on the same server, you might set up separate backends so you
can probe the store and forum separately, rather than e.g. having a
special probe page that checks both, or not checking the forum at all.
That way if the forum falls over, your store can still be available,
because your VCL directs the request to a different Varnish backend.

So, you're on the right track, but you want to be using separate
backends for each site, not separate directors. Directors should only
be used if the backends they reference are functionally identical.