Connections dropped under load

[Kristian Lyngstol]

thanks.
posting,
top
stop
Please

On Tue, Jan 11, 2011 at 12:59:58AM +0100, George Georgovassilis wrote:
> Thank you for the hint. Here are the values:
> 
> thread_pools = 2
> thread_pool_min = 2
> thread_pool_max = 200 (was 2 at the time of my initial tests)
> thread_pool_add_delay = 2

As have already been pointed out, this is a low value. This also explains
why session_linger is an issue to you. Unless you are on 32-bit (which you
shouldn't ever ever ever be), there's no reason to not always have a
thousand threads laying around. Your settings also means that you have FOUR
threads available when you start your tests. Not exactly a lot of room for
bursts of traffic.

Your other mail actually had a thread_pool_max of 16. That will give you a
maximum of 16 concurrent requests that can be handled, with an other 32
that can be queued. With session_linger, these threads will remain
allocated to the connection for a longer duration, thus it's obvious that
in this case, your thread starvation was the real issue and you just
triggered it faster with a higher session_linger. It's a perfectly obvious
and mystery-free explanation. Session lingering is a mechanism to avoid
trashing your system during high load by constantly moving data around
between threads, but it depends on reasonable thread-settings - or rather:
an abundance of threads.

http://kristianlyng.wordpress.com/2010/01/26/varnish-best-practices/ sounds
like a good place to start reading. Specially about threads.

- Kristian