Dynamic backends revisited, again

[Dridi Boukelmoune]

On Mon, Aug 17, 2015 at 5:20 PM, Dridi Boukelmoune <dridi at varni.sh> wrote:
> 0002: should be self-explanatory

Patch number 2 was rejected, so I updated my DNS director draft to
track VCL events and act accordingly. When a director's VCL emits a
COLD event, there's still a race condition if the event is emitted
while the director is adding backends. There's no locking workaround,
and you can't batch backends additions.

The only solution I see so far, would be to check between every
insertions whether the VCL emitted a COLD event. I don't like that too
much, for it leaves a half updated backends list in a cold VCL that
could become WARM again.

Currently, I have a background loop that does this:

while (not discarded) {
  if neither WARM nor USE
    cond_wait

  if neither WARM nor USE
    continue

  i = lookup (blocking call)

  if i != 0 {
    log error
    continue
  }

  if neither WARM nor USE
    continue

  update_backends
  cond_timedwait
}

The cond_wait is signaled whenever a new event occurs, the
cond_timedwait is signaled when the VCL is discarded.

So basically I need to move the last WARM/USE check inside the
update_backends loop, which is OK for a DNS director, once the VCL
becomes WARM again it does a lookup straight away. I'm not sure that's
enough to prevent the race, I need to take a closer look at the locks
involved in both process (adding backends and cooling the VCL).

I'm not asking to reconsider the patch, I'm sharing my observations
regarding the current constraints.

Also, can it be checked with the witness facility? I think that the
witness code only works for the Lck_ functions and macros, but I
didn't get into the specifics when it was introduced.

Best Regards,
Dridi