Child dying with "Too many open files"
Andrew Knapp
andrew at imeem.com
Mon Mar 3 22:03:01 CET 2008
I'm actually getting this a lot more frequently while running trunk
(r2544). Every time the child dies it's cleaning out the cache and
starting over. Right now it's happening about every 15 seconds, which
seems crazy.
Any ideas?
-Andy
> -----Original Message-----
> From: varnish-misc-bounces at projects.linpro.no [mailto:varnish-misc-
> bounces at projects.linpro.no] On Behalf Of Andrew Knapp
> Sent: Friday, February 29, 2008 12:54 PM
> To: Michael S. Fischer
> Cc: varnish-misc at projects.linpro.no
> Subject: RE: Child dying with "Too many open files"
>
> I'm still getting the "Too many open files" error on the child.
>
> $ sudo sysctl -a | grep file
> fs.file-max = 131072
>
> NFILES is also set to 131072. Any ideas?
>
> -Andy
>
> > -----Original Message-----
> > From: michaelonlaw at gmail.com [mailto:michaelonlaw at gmail.com] On
> Behalf
> > Of Michael S. Fischer
> > Sent: Thursday, February 28, 2008 3:51 PM
> > To: Andrew Knapp
> > Cc: varnish-misc at projects.linpro.no
> > Subject: Re: Child dying with "Too many open files"
> >
> > I can't help but wonder if you'd set it too high. What happens when
> > you set NFILES and fs.file-max both to 131072? I've tested that as
a
> > known good value.
> >
> > --Michael
> >
> > On Thu, Feb 28, 2008 at 2:58 PM, Andrew Knapp <andrew at imeem.com>
> wrote:
> > > Yup, it is. Here's some output:
> > >
> > > $ ps auxwww | grep varnish
> > > root 12036 0.0 0.0 27704 648 ? Ss 14:54 0:00
> > > /usr/sbin/varnishd -a :80 -f /etc/varnish/photo.vcl -T
> > <internalip>:6082
> > >
> > > -t 120 -w 10,700,30 -s file,/c01/varnish/varnish_storage.bin,12G
-u
> > > varnish -g varnish -P /var/run/varnish.pid
> > > varnish 12037 1.2 0.4 13119108 39936 ? Sl 14:54 0:00
> > > /usr/sbin/varnishd -a :80 -f /etc/varnish/photo.vcl -T
> > <internalip>:6082
> > >
> > > -t 120 -w 10,700,30 -s file,/c01/varnish/varnish_storage.bin,12G
-u
> > > varnish -g varnish -P /var/run/varnish.pid
> > >
> > > -Andy
> > >
> > >
> > > > -----Original Message-----
> > > > From: michaelonlaw at gmail.com [mailto:michaelonlaw at gmail.com] On
> > Behalf
> > > > Of Michael S. Fischer
> > >
> > >
> > > > Sent: Thursday, February 28, 2008 1:57 PM
> > > > To: Andrew Knapp
> > > > Cc: varnish-misc at projects.linpro.no
> > > > Subject: Re: Child dying with "Too many open files"
> > > >
> > > > Is varnishd being started as root? (even if it drops
privileges
> > > > later) Only root can have > 1024 file descriptors open, to my
> > > > knowledge.
> > > >
> > > > --Michael
> > > >
> > > > On Thu, Feb 28, 2008 at 11:48 AM, Andrew Knapp
> <andrew at imeem.com>
> > > > wrote:
> > > > > Didn't really get a answer to this, so I'm trying again.
> > > > >
> > > > > I've done some testing with the NFILES variable, and I keep
> > getting
> > > > the
> > > > > same error as before ("Too many open files"). I've also
> > verified
> > > > that
> > > > > the limit is actually being applied by putting a ulimit -a
in
> > the
> > > > > /etc/init.d/varnish script.
> > > > >
> > > > > Anyone have any ideas? I'm running the 1.1.2-5 rpms from
> sf.net
> > on
> > > > > Centos 5.1.
> > > > >
> > > > > Thanks,
> > > > > Andy
> > > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: varnish-misc-bounces at projects.linpro.no
> > [mailto:varnish-
> > > > misc-
> > > > > > bounces at projects.linpro.no] On Behalf Of Andrew Knapp
> > > > >
> > > > > > Sent: Wednesday, February 20, 2008 5:52 PM
> > > > > > To: Michael S. Fischer
> > > > > > Cc: varnish-misc at projects.linpro.no
> > > > >
> > > > >
> > > > > > Subject: RE: Child dying with "Too many open files"
> > > > > >
> > > > > > Here's the output:
> > > > > >
> > > > > > $ sysctl fs.file-max
> > > > > > fs.file-max = 767606
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: michaelonlaw at gmail.com
> > [mailto:michaelonlaw at gmail.com] On
> > > > > > Behalf
> > > > > > > Of Michael S. Fischer
> > > > > > > Sent: Wednesday, February 20, 2008 5:48 PM
> > > > > > > To: Andrew Knapp
> > > > > > > Cc: varnish-misc at projects.linpro.no
> > > > > > > Subject: Re: Child dying with "Too many open files"
> > > > > > >
> > > > > > > Does 'sysctl fs.file-max' say? It should be >= the
> ulimit.
> > > > > > >
> > > > > > > --Michael
> > > > > > >
> > > > > > > On Wed, Feb 20, 2008 at 4:04 PM, Andrew Knapp
> > > <andrew at imeem.com>
> > > > > > wrote:
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Hello,
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > I'm getting this error when running varnishd:
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > >>
> > > > > > > >
> > > > > > > > Child said (2, 15369): <<Assert error in wrk_thread(),
> > > > > cache_pool.c
> > > > > > > line
> > > > > > > > 217:
> > > > > > > >
> > > > > > > > Condition((pipe(w->pipe)) == 0) not true.
> > > > > > > >
> > > > > > > > errno = 24 (Too many open files)
> > > > > > > >
> > > > > > > > >>
> > > > > > > >
> > > > > > > > Cache child died pid=15369 status=0x6
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > uname -a:
> > > > > > > >
> > > > > > > > Linux <hostname> 2.6.18-53.1.4.el5 #1 SMP Fri Nov 30
> > 00:45:55
> > > > EST
> > > > > > > 2007
> > > > > > > > x86_64 x86_64 x86_64 GNU/Linux
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > command used to start varnish:
> > > > > > > >
> > > > > > > > /usr/sbin/varnishd -d -d -a :80 -f
> /etc/varnish/photo.vcl
> > -T
> > > > > > > > <internalIP>:6082 -t 120 -w 10,700,30 -s
> > > > > > > > file,/c01/varnish/varnish_storage.bin,12G -u varnish
-g
> > > > varnish -P
> > > > > > > > /var/run/varnish.pid
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > I have NFILES=270000 set in /etc/sysconfig/varnish. Do
> I
> > just
> > > > need
> > > > > > to
> > > > > > > up
> > > > > > > > that value?
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Andy
> > > > > > > > _______________________________________________
> > > > > > > > varnish-misc mailing list
> > > > > > > > varnish-misc at projects.linpro.no
> > > > > > > > http://projects.linpro.no/mailman/listinfo/varnish-
> misc
> > > > > > > >
> > > > > > > >
> > > > > > _______________________________________________
> > > > > > varnish-misc mailing list
> > > > > > varnish-misc at projects.linpro.no
> > > > > > http://projects.linpro.no/mailman/listinfo/varnish-misc
> > > > > _______________________________________________
> > > > > varnish-misc mailing list
> > > > > varnish-misc at projects.linpro.no
> > > > > http://projects.linpro.no/mailman/listinfo/varnish-misc
> > > > >
> > > > >
> > >
> > >
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc
More information about the varnish-misc
mailing list