Bug? Barage of hits leads to failure creating worker threads / stats tracking

[Rafael Umann]

Take a look at you FDs:
(linux)
# cat /proc/sys/fs/file-nr
11730	0	5049800

Varnish works with a limit of 65k file descriptors. Anything above  
that will be a problem.

http://varnish.projects.linpro.no/changeset/3631

If you are getting 65k FD`S we hit the same problem.

Another tip: if you are running on a 32bits system, thats your problem!

[]s,


On Apr 13, 2009, at 2:57 AM, Ray Barnes wrote:

> On Sat, Apr 11, 2009 at 5:48 PM, Artur Bergman <sky at crucially.net>  
> wrote:
> I've never seen it do worker threads not created.
>
> Are there any limits on number of threads?
>
> Apparently there are; thanks for pointing me in the right  
> direction.  I found a C program that attempts to spawn threads and  
> lets you know at what point it hits an error - http://people.redhat.com/alikins/tuning_utils/thread-limit.c 
>  - it reports that I can't open more than 383 threads.  The question  
> is why.  Here's what I've done thus far:
>
> 1) Recompiled glibc per http://people.redhat.com/alikins/system_tuning.html#threads 
>  - the definition of  PTHREAD_THREADS_MAX is tied to the value in / 
> usr/include/linux/limits.h so I adjusted that value, installed the  
> source RPM, rebuilt all glibc RPMs and installed using 'rpm -Uvh -- 
> force' to overcome pre/post installation errors within the RPM  
> (hopefully that did what it was supposed to).
>
> 2) Set /proc/sys/kernel/threads-max to 65535 (was 3000ish before),  
> no change
>
> 3) Set /etc/security/limits.conf to "* soft nofile 1024" and "* hard  
> nofile 10240" and added "session required /lib/security/ 
> pam_limits.so" to /etc/pam.d/login with no change, per the advice at http://www.mail-archive.com/java-linux@java.blackdown.org/msg15247.html 
>  where the poster indicates he did not have to recompile glibc to do  
> this
>
> I've tried the same C program above on a few other Linux boxes and  
> they all seem to be somewhere between 200 and 383 allowed threads.   
> The first obvious solution would be to dump Linux and use FBSD - a  
> direction i'll look into in the future.  But for now we're stuck on  
> Linux.  Any ideas?
>
> -Ray
>
>
>
>
>
> E-mail verificado pelo Terra Anti-Spam.
> Para classificar esta mensagem como spam ou não spam, clique aqui.
> Verifique periodicamente a pasta Spam para garantir que apenas  
> mensagens
> indesejadas sejam classificadas como Spam.
>
> Esta mensagem foi verificada pelo E-mail Protegido Terra.
> Atualizado em 12/04/2009
>
> _______________________________________________
> varnish-dev mailing list
> varnish-dev at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20090413/cf14a924/attachment-0001.html>