[Varnish] #1285: All worker threads can block on the vca_pipe under high load

[Varnish]

#1285: All worker threads can block on the vca_pipe under high load
-------------------+--------------------
 Reporter:  mark   |       Type:  defect
   Status:  new    |   Priority:  high
Milestone:         |  Component:  build
  Version:  3.0.3  |   Severity:  major
 Keywords:         |
-------------------+--------------------
 Varnish uses a pipe in non-blocking mode to communicate between the worker
 threads and the (epoll/kqueue) waiter thread. Under high load, the pipe
 can fill up, quickly blocking all worker threads. At this point,
 everything goes to hell, and Varnish does not recover from it until a
 restart.

 It appears that seemingly several high traffic sites have experienced this
 problem regularly over the years, but somehow a fix never made it into
 Varnish. Let's change that. :)

 I've attached a patch that fixes the problem, in two ways. It changes the
 waiter thread loop to read from the vca_pipe first, before doing anything
 else. This reduces the chance of blocking any worker threads.

 The patch also changes the worker thread side of the vca_pipe to non-
 blocking, as Varnish doesn't actually depend on blocking mode at all. This
 change alone probably suffices as well. (based on a patch originally for
 Varnish 2.x by Artur Bergman)

-- 
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1285>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator