Q: multiple backends

[Stig Sandbeck Mathisen]

"Poul-Henning Kamp" <phk at phk.freebsd.dk> writes:

> Yes, this is indeed on our plans, I keep trying to get somebody to
> tell me how it should work, but with very little luck :-)
>
> Just saying "fault-tolerance" or "load-balancing" isn't saying much
> you see, we need to formulate strategies and algorithms...

Common load-balancing strategies are:

* Round robin.  (easy to implement)

* Weighted, i.e. this host can handle twice as much as that host.
  (twice as much what?  Bytes? Requests?)  Some black boxes can do a
  TCP connect or a HTTP GET to the backend to fetch a number
  representing load from the web server, used to weigh backends.

* Weighted based on measured response time from backend.  (Send
  requests to the fastest responding node, one should perhaps base
  this on 2xx responses only)

* Hash on client IP, Host: header, substring of session cookie, url,
  or phase of the moon to select backend.  Some application servers
  generate session cookies where the first n bytes represent the ID of
  a backend server.

What to do with requests that are waiting for a failing backend?

* Return a 5xx message

* Try again on another backend?  (Could have undesirable results, and
  not be possible for all request types.  For instance, it's hard to
  replay a piped request)

How often should one retry a failed backend?  
> Never attribute to malice what can adequately be explained by incompetence.

Any sufficiently advanced incompetence is indistinguishable from malice.

-- 
Stig Sandbeck Mathisen, Linpro