[Varnish] #1229: Varnish Banlurker patch

[Varnish]

#1229: Varnish Banlurker patch
------------------------------------------------+-------------------------
 Reporter:  celly                               |       Type:  enhancement
   Status:  new                                 |   Priority:  high
Milestone:  Later                               |  Component:  varnishd
  Version:  3.0.3                               |   Severity:  critical
 Keywords:  banlist, huge, performance, lurker  |
------------------------------------------------+-------------------------
 Hi guys,

 we are using your proxies on our projects, that are speciffic by lots of
 req/s and bans/s. Last few months, we were faced a performance issues
 (slow response to req). We found out, that this is because huge banlist.
 We were using banlurker, ofcourse, but it cannot handle so much new bans
 in combination with a lot of cached objects. Setting up ban_lurker_sleep
 to a very small number didn't help, so we realised, that we should look
 into a code.

 Month ago, we found, sorry the word I used, problem and make a patch
 (attached). Since that time, our Varnishes works great again and have very
 fast responses. I made some graphs screenshots (attached) for comparision.
 Each graph is made in three versions:
 * M1 - month 1 (from 2 months ago to 1 month)
 * M2 - month 2 (from 1 month ago to now)
 * O - overview (from 2 months ago to now)
 Files names:
 * G1 - banlist size
 * G2 add/gone ban
  a. light blue - bans added per second
  a. green - bans marked as gone per second but remains in list
 * G3 - TCP connections
 * G4 - CPU detailed stats:
  a. blue - user
  a. yellow - sys
  a. green - soft
 * G5 - each color is a different CPU

 Simply said, patch clears banlist in ban_lurker_work, because
 ban_lurker_work can take a lots of time to finnish in our case. As I
 wrote, our service has quite big hitrate, so many of bans in banlist is
 gone by hit before ban_lurker touches it.

 We like your software and we will be glad, if you make this patch as a
 part of future releases. We can fork you on GitHub and make pull request,
 if you prefer this way.

-- 
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1229>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator