Varnish Lurker is getting slower / Ban lists keeps increasing

[Olivier Hanesse]

Hello,

We are running Varnish 5.1.3 revision 05c5ac6b9.

During business hours (peak traffic is around 300 req/s ), our ban list
keeps increasing to reach 100K objects (and sometimes more).
Ban traffic is around 4-5 ban/s during this period.  We are using only
"smart bans".

We are pushing varnishstat output to Graphite, and  we have noticed than
the lurker either stop working for a long period of time or is getting
really slow.

Here is a screenshot where you can see that the number of deleted BANs by
the lurker after 9am is decreasing and the BAN list keeps increasing (scale
is log base 10).
The blue bar shows that the lurker didn't delete any ban during 12:50 and
16:30

https://pasteboard.co/GHUPY6t.png

After business hours, everything returns to normal.

I've already tried to change (increase/decrease) both "ban_lurker_batch"
and "ban_lurker_sleep" parameters. Same things.

I don't know how I can get more lurker logs to debug this.
Is there a way to make it more aggressive ?

Any ideas ?

Regards

Olivier
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20170829/df108b2c/attachment.html>