thread_pool_max - is it total of all thread pools?

[Ken Brownfield]

The min is per pool, the max is aggregate.  That was a little confusing to me too, at first.

Memory (stacks) is the primary bottleneck; you'll run out RAM long before modern kernels have trouble with thread counts.

Note that increasing the thread count is only useful (vs being a crutch for a bad backend) for long-lived or unreliable backend requests and long-lived client requests (mainly HTTP keepalives).  Past around double your CPU core count is all you'd need if Varnish were purely CPU-bound.  And that will happen only with many gigabits of traffic. :-)

So, assuming you're not obscuring some sort of failure up- or down-stream from Varnish, I'd tune your max to something like 2-3x your steady state worker count, and increase as necessary.

Hope it helps,
-- 
Ken

On Jul 15, 2010, at 10:19 AM, Caunter, Stefan wrote:

> Hi, running varnish on dual processor, dual core system with 4GB ram, we hit thread pool limit (500) this morning, and so are defining 4 thread pools, 200 min, and a 2500 maximum limit.
>  
> I now see 800 threads created on start. I cannot see in the docs anything that says max limit is for all thread pools. We’ve set kern.threads.max_threads_per_proc to 4096 and increased the other tuning settings.
>  
> Can we assume thread_pool_max is the summary of all thread pools?
>  
> Can we get the thread pool maximum higher with better hardware and a custom kernel set for higher tuning settings?
>  
> Stefan Caunter :: Senior Systems Administrator :: TOPS
> e: scaunter at topscms.com  ::  m: (416) 561-4871
> www.thestar.com www.topscms.com
>  
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20100715/326800e6/attachment-0003.html>