thread pool issues

[Kristian Lyngstol]

Greetings,

On Fri, Jun 10, 2011 at 08:32:11AM +0200, Dennis Hendriksen wrote:
> We're running Varnish 2.0.6 on a dual quad core server which is doing
> about 500 req/s with a 97% hit ratio serving mostly images with. When we
> increase the load to about 800 req/s than we encounter two problems that
> seem to be related with the thread pool increase.

You really should see if you can't move to at least Varnish 2.1.5.

> When we double the varnish load then the "N worker threads limited"
> increases rapidly (100k+) while the "N worker threads created" does not
> increase (8 pools, min pool size 25, max pool size 1000). Varnish is
> unresponsive and client connections hang.

That'll give you 200 threads at startup.

I would typically recommend something closer to minimum 500, pools 2 and
max 5000.

You also want to reduce the thread_pool_add_delay from the (2.0.6)
default 20ms to 2ms for instance. That will limit the rate that threads
are started at, and 20ms is often way too slow.

How many connections (not requests) are you doing during these tests?

> At other times we see the number of worker threads increasing but again
> connections 'hang' while Varnish doesn't show any dropped connections
> (only overflows).

Do you use keep-alive and long-lasting connections? You may want to see
if reducing session_linger helps.

Are you testing with real traffic or synthetic tests?

If possible, varnishstat -1 output would be useful.

- Kristian