high iowait

[Lasse Karstensen]

On Tue, Mar 19, 2013 at 09:46:37PM -0400, Sean Allen wrote:
> One of our varrnish servers is spending about 40-50% of its time in iowait.
> Is this just from the varnishlog getting written? Our IO performance is not
> great and I'm looking to be able to get the amount of time we are spending
> doing IO down. This occurs even when everything is running nicely in memory
> and we aren't overflowing into swap ( which was detailed in a different
> email as I think they might be different issues ).
> varnishd (varnish-3.0.3 revision 9e6a70f)
[..]
> # Without "ban_lurker_sleep," nothing banned from the cache ever gets
> evicted.

Ban lurker default sleep is 10ms in 3.0.3. You are increasing it a lot, why?

Are you using smart bans?

	https://www.varnish-software.com/static/book/Cache_invalidation.html#smart-bans

> DAEMON_OPTS="-a :6081 \
>              -T :6082 \
>              -f /etc/varnish/default.vcl \
>              -u varnish -g varnish \
>              -p thread_pool_min=500 \
>              -p thread_pool_max=5000 \
>              -s malloc,6G \
>              -p thread_pools=2 \
>              -p thread_pool_add_delay=1 \
>              -p ban_lurker_sleep=60s"

Your thread count is a bit on the high side (10k); ~1 thread / concurrent connection is good.

But it doesn't explain why you are in iowait.
Is this running on a virtualised server? Do you have proper network drivers?


-- 
With regards,
Lasse Karstensen
Varnish Software AS