[Varnish] #573: varnish 2.0.4 crash very

Varnish varnish-bugs at projects.linpro.no
Tue Nov 3 14:15:36 CET 2009


#573: varnish 2.0.4 crash very
-------------------------+--------------------------------------------------
 Reporter:  adungaos     |        Type:  defect 
   Status:  new          |    Priority:  highest
Milestone:               |   Component:  build  
  Version:  trunk        |    Severity:  normal 
 Keywords:  crash panic  |  
-------------------------+--------------------------------------------------
 I find my varnishd crashd about 4-5 times today, please help me.

 my os:
 {{{
 $ cat /etc/redhat-release
 Red Hat Enterprise Linux Server release 5.2 (Tikanga)
 $ uname -a
 Linux s14.cache 2.6.18-92.el5 #1 SMP Tue Apr 29 13:16:15 EDT 2008 x86_64
 x86_64 x86_64 GNU/Linux
 $ varnishd  -V
 varnishd (varnish-2.0.4)
 Copyright (c) 2006-2009 Linpro AS / Verdens Gang AS
 }}}
 varnishd:
 {{{
 /usr/sbin/varnishd -P /var/run/varnish.pid -u varnish -g varnish -a
 0.0.0.0:80 -T 127.0.0.1:8888 -w 16,65535,120 -l 240m -p thread_pools 8 -p
 listen_depth 4096 -p lru_interval 3600 -h classic,50000 -s malloc,80G -f
 /etc/varnish/default.vcl
 }}}
 crash message:
 {{{
 Nov  3 12:34:11 s14 varnishd[7894]: Child (7895) died signal=6 (core
 dumped)
 Nov  3 12:34:11 s14 varnishd[7894]: Child (7895) Panic message: Assert
 error in vsl_hdr(), shmlog.c line 85:   Condition(id < 0x10000) not true.
 errno = 11 (Resource temporarily unavailable)  thread = (cache-worker)sp =
 0x2aad96f7f008 {   fd = 65537, id = 65537, xid = 0,   client =
 122.4.141.17:21793,   step = STP_FIRST,   handling = error,   ws =
 0x2aad96f7f078 {      id = "sess",     {s,f,r,e} =
 {0x2aad96f7f808,,+19,(nil),+16384},   },     worker = 0x73838bd0 {     },
 },
 Nov  3 12:34:11 s14 varnishd[7894]: child (12679) Started
 Nov  3 12:34:11 s14 varnishd[7894]: Child (12679) said Closed fds: 3 4 5 8
 9 11 12
 Nov  3 12:34:11 s14 varnishd[7894]: Child (12679) said Child starts
 Nov  3 12:34:11 s14 varnishd[7894]: Child (12679) said Ready

 Nov  2 12:51:28 s14 varnishd[7817]: Child (5197) not responding to ping,
 killing it.
 Nov  2 12:51:29 s14 varnishd[7817]: Child (5197) died signal=6
 Nov  2 12:51:29 s14 varnishd[7817]: Child (5197) Panic message: Assert
 error in vsl_hdr(), shmlog.c line 85:   Condition(id < 0x10000) not true.
 errno = 110 (Connection timed out)  thread = (cache-worker)sp =
 0x2aace8693008 {   fd = 65536, id = 65536, xid = 0,   client =
 222.38.164.93:4339,   step = STP_FIRST,   handling = error,   ws =
 0x2aace8693078 {      id = "sess",     {s,f,r,e} =
 {0x2aace8693808,,+19,(nil),+16384},   },     worker = 0x2aab0b800bd0 {
 }, },
 Nov  2 12:51:29 s14 varnishd[7817]: child (7409) Started
 Nov  2 12:51:29 s14 varnishd[7817]: Child (7409) said Closed fds: 3 4 5 8
 9 11 12
 Nov  2 12:51:29 s14 varnishd[7817]: Child (7409) said Child starts
 Nov  2 12:51:29 s14 varnishd[7817]: Child (7409) said Ready
 Nov  2 20:52:25 s14 kernel: possible SYN flooding on port 80. Sending
 cookies.
 Nov  2 12:53:23 s14 varnishd[7817]: Child (7409) died signal=6
 Nov  2 12:53:23 s14 varnishd[7817]: Child (7409) Panic message: Assert
 error in vsl_hdr(), shmlog.c line 85:   Condition(id < 0x10000) not true.
 errno = 104 (Connection reset by peer)  thread = (cache-worker)sp =
 0x2ab25869d008 {   fd = 65542, id = 65542, xid = 0,   client =
 121.70.179.89:2513,   step = STP_FIRST,   handling = error,   ws =
 0x2ab25869d078 {      id = "sess",     {s,f,r,e} =
 {0x2ab25869d808,,+19,(nil),+16384},   },     worker = 0x2ab21bf00bd0 {
 }, },
 Nov  2 12:53:23 s14 varnishd[7817]: child (10480) Started
 Nov  2 12:53:24 s14 varnishd[7817]: Child (10480) said Closed fds: 3 4 5 8
 9 11 12
 Nov  2 12:53:24 s14 varnishd[7817]: Child (10480) said Child starts
 Nov  2 12:53:24 s14 varnishd[7817]: Child (10480) said Ready

 }}}
 gdb backtrace:
 {{{
 # gdb /usr/sbin/varnishd core.7895
 GNU gdb Red Hat Linux (6.5-37.el5rh)
 Copyright (C) 2006 Free Software Foundation, Inc.
 GDB is free software, covered by the GNU General Public License, and you
 are
 welcome to change it and/or distribute copies of it under certain
 conditions.
 Type "show copying" to see the conditions.
 There is absolutely no warranty for GDB.  Type "show warranty" for
 details.
 This GDB was configured as "x86_64-redhat-linux-gnu"...Using host
 libthread_db library "/lib64/libthread_db.so.1".

 Error while mapping shared library sections:
 ./vcl.1P9zoqAU.so: No such file or directory.
 Error while mapping shared library sections:
 ./vcl.FANefPfn.so: No such file or directory.
 Error while mapping shared library sections:
 ./vcl.OaGbHZXp.so: No such file or directory.
 Reading symbols from /usr/lib64/libvarnish.so.1...done.
 Loaded symbols for /usr/lib64/libvarnish.so.1
 Reading symbols from /lib64/librt.so.1...done.
 Loaded symbols for /lib64/librt.so.1
 Reading symbols from /usr/lib64/libvarnishcompat.so.1...done.
 Loaded symbols for /usr/lib64/libvarnishcompat.so.1
 Reading symbols from /usr/lib64/libvcl.so.1...done.
 Loaded symbols for /usr/lib64/libvcl.so.1
 Reading symbols from /lib64/libdl.so.2...done.
 Loaded symbols for /lib64/libdl.so.2
 Reading symbols from /lib64/libpthread.so.0...done.
 Loaded symbols for /lib64/libpthread.so.0
 Reading symbols from /lib64/libnsl.so.1...done.
 Loaded symbols for /lib64/libnsl.so.1
 Reading symbols from /lib64/libm.so.6...done.
 Loaded symbols for /lib64/libm.so.6
 Reading symbols from /lib64/libc.so.6...done.
 Loaded symbols for /lib64/libc.so.6
 Reading symbols from /lib64/ld-linux-x86-64.so.2...done.
 Loaded symbols for /lib64/ld-linux-x86-64.so.2
 Reading symbols from /lib64/libnss_files.so.2...done.
 Loaded symbols for /lib64/libnss_files.so.2
 Error while reading shared library symbols:
 ./vcl.1P9zoqAU.so: No such file or directory.
 Error while reading shared library symbols:
 ./vcl.FANefPfn.so: No such file or directory.
 Error while reading shared library symbols:
 ./vcl.OaGbHZXp.so: No such file or directory.
 Error while reading shared library symbols:
 ./vcl.1P9zoqAU.so: No such file or directory.
 Error while reading shared library symbols:
 ./vcl.FANefPfn.so: No such file or directory.
 Error while reading shared library symbols:
 ./vcl.OaGbHZXp.so: No such file or directory.
 Core was generated by `/usr/sbin/varnishd -P /var/run/varnish.pid -u
 varnish -g varnish -a 0.0.0.0:80'.
 Program terminated with signal 6, Aborted.
 #0  0x0000003b6fe30155 in raise () from /lib64/libc.so.6
 (gdb) bt
 #0  0x0000003b6fe30155 in raise () from /lib64/libc.so.6
 #1  0x0000003b6fe31bf0 in abort () from /lib64/libc.so.6
 #2  0x000000000041af57 in pan_ic (func=<value optimized out>, file=<value
 optimized out>, line=<value optimized out>,
     cond=<value optimized out>, err=0, xxx=<value optimized out>) at
 cache_panic.c:317
 #3  0x000000000042df39 in vsl_hdr (tag=SLT_SessionOpen, p=0x2b1f4b21b945
 <Address 0x2b1f4b21b945 out of bounds>, len=29, id=65537)
     at shmlog.c:85
 #4  0x000000000042eca3 in VSL (tag=SLT_SessionOpen, id=65537, fmt=0x43a7c5
 "%s %s %s") at shmlog.c:168
 #5  0x0000000000409503 in VCA_Prep (sp=0x2aad96f7f008) at
 cache_acceptor.c:127
 #6  0x00000000004106f0 in CNT_Session (sp=0x2aad96f7f008) at
 cache_center.c:466
 #7  0x000000000041c4f2 in wrk_do_cnt_sess (w=0x73838bd0, priv=<value
 optimized out>) at cache_pool.c:398
 #8  0x000000000041bbbf in wrk_thread (priv=0x2b1f4703c1f0) at
 cache_pool.c:310
 #9  0x0000003b706062f7 in start_thread () from /lib64/libpthread.so.0
 #10 0x0000003b6fed1b6d in clone () from /lib64/libc.so.6
 (gdb)

 }}}

 thanks.

-- 
Ticket URL: <http://varnish.projects.linpro.no/ticket/573>
Varnish <http://varnish.projects.linpro.no/>
The Varnish HTTP Accelerator


More information about the varnish-bugs mailing list