ban_lurker_age, ban_lurker_sleep

[Nils Goroll]

On 18/04/16 10:39, Poul-Henning Kamp wrote:
> --------
> In message <571112C2.5070607 at schokola.de>, Nils Goroll writes:
> 
>> I've just pushed two minor improvements on the documentation, but I'd suggest
>> two changes
>>
>> * ban_lurker_age:
> 
>> but we also increase the likelihood of bans requiring request-time evaluation,
>> which is bad for latency.
> 
> The important thing is for the lurker to not get in the way of
> request-time evaluation, because it will be definition waste a lot
> of time on objects clients don't care for, thus needlessly delaying
> the ones the clients ask for.

This probably is the scenratio which is most relevant for cases of good use of
the ban lurker. The case I'm working on is 40k expensive bans on 400k objects
when the ban luker cannot keep up for long periods of time (yes, I know this is
bad use of bans, but still an intersting case).

With such numbers of bans, avoiding ban tests at lookup time should be relevant,
so kicking in the lurker early should help.

> The right thing to do with respect to both ban_lurker_{age|sleep}
> is probably to look at the rate of req-time evaluations and modulate
> the lurker activity to minimize interference.
> 
> Likewise the mutex contest delay should increase in response to
> rate of req-time evaluations.

Yeah, maybe, I'm not sure. The drawback is that such "magic" auto-tuning will
make the whole system less predictable and can lead to oscillation effects. For
the time being, I'd prefer to just lower the ban_lurker_age default.

> For instance we could make a param which says how many ban-checks
> per second we allow.  If the req-time processing leaves any
> unused, the lurker is free to "fill up the quota".  Unfortunately
> such "quota" parameters need to scale and I have no idea on what (ncpu ?)

As long as the ban lurker is single threaded, it's pretty hard rate-limited
already on any real-life system of relevance.


Nils