Soliciting input on the ban lurker

[Poul-Henning Kamp]

I have been looking at ticket 1045 and the ban-lurker and I think
I need some thoughtfull input before I proceed.

The ban-lurker we have now, will try test the objects which sit
at the oldest bans and check them against all bans.

This can give three results:

  1) Som ban matches, object is killed.

  2) We cannot tell test certain bans, because they require req.*

  3) All bans tested, none match, object moved to top of ban-list.

Eventually, this may empty the last ban on the list so it can be
removed.

The advantage to this strategy is that the ban lurker works one
object through the ban-list at a time, and cleans up the ban-list
from the tail end, where presumbly, we find the objects which are
not going to "shake loose" due to traffic.

The disadvantage of this strategy is that we get stuck on bans
requiring req.*, those seem to be very common, and the ban-lurker
does surprisingly much duplicate work when that happens.


Another strategy could go like this:

Find the last ban on the list that does not require req.*, test
it against all objects further down the ban-list, then mark the
ban "GONE" because no object will need to be tested against it again.

The advantage of this strategy is that the "non-req" bans will
all be dealt with by the ban-lurker and that no duplicate work
is done at all:  All tests are productive.

The disadvantage is that it works the other way with respect to
memory pressure:  It takes one ban against many objects, objects
that we have not used for a long time and which therefore introduces
memory pressure.  Worst case is where you add a non-req ban, and
the lurker will test it against every single object in your cache.

It is not obvious to me that either of these strategies are optimal
and I am getting to the point where I wonder if maybe we need to
offer a choice of strategy ?

Some input and operational experience would be most welcome...

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.