[Varnish] #534: Threads stuck in trunk

Varnish varnish-bugs at projects.linpro.no
Thu Aug 20 13:43:06 CEST 2009


#534: Threads stuck in trunk
---------------------------+------------------------------------------------
 Reporter:  anders         |        Owner:  phk  
     Type:  defect         |       Status:  new  
 Priority:  high           |    Milestone:       
Component:  varnishd       |      Version:  trunk
 Severity:  critical       |   Resolution:       
 Keywords:  threads stuck  |  
---------------------------+------------------------------------------------
Comment (by anders):

 PHK wanted to know more about the threads which are not in __error, so
 here we go.

 Running trunk/4144, I got the thread problem with all threads used up.
 Summarizing the threads, I get this list (count in left column):

 {{{
    1  in kevent ()
    1  in nanosleep ()
    1  in writev ()
    2  in poll ()
 3995  in __error ()
 }}}

 The threads of interest:

 {{{
   4000 Thread 0x8010020b0 (LWP 100434)  0x0000000800d7b07c in poll ()
   3999 Thread 0x8010023d0 (LWP 100059)  0x0000000800db0e0c in nanosleep ()
   3994 Thread 0x801002a10 (LWP 100104)  0x0000000800db1f8c in kevent ()
   3993 Thread 0x801002ba0 (LWP 100110)  0x0000000800d7b07c in poll ()
   1916 Thread 0x187aa80ba0 (LWP 103321)  0x0000000800db67ec in writev ()
 }}}

 Backtrace from these:

 {{{
 (gdb) thread 4000
 [Switching to thread 4000 (Thread 0x8010020b0 (LWP 100434))]#0
 0x0000000800d7b07c in poll () from /lib/libc.so.7
 (gdb) bt
 #0  0x0000000800d7b07c in poll () from /lib/libc.so.7
 #1  0x0000000800ac180e in poll () from /lib/libthr.so.3
 #2  0x000000000041308f in CLI_Run () at cache_cli.c:157
 #3  0x000000000041ed81 in child_main () at cache_main.c:141
 #4  0x000000000042d57d in start_child (cli=Variable "cli" is not
 available.
 ) at mgt_child.c:318
 #5  0x000000000042d8f4 in MGT_Run () at mgt_child.c:545
 #6  0x000000000043b5f6 in main (argc=35, argv=0x7fffffffea20) at
 varnishd.c:737
 (gdb) thread 3999
 [Switching to thread 3999 (Thread 0x8010023d0 (LWP 100059))]#0
 0x0000000800db0e0c in nanosleep () from /lib/libc.so.7
 (gdb) bt
 #0  0x0000000800db0e0c in nanosleep () from /lib/libc.so.7
 #1  0x0000000800ac1915 in nanosleep () from /lib/libthr.so.3
 #2  0x0000000800682a81 in TIM_sleep (t=Variable "t" is not available.
 ) at time.c:162
 #3  0x0000000000421c03 in wrk_herdtimer_thread (priv=Variable "priv" is
 not available.
 ) at cache_pool.c:429
 #4  0x0000000800abf4d1 in pthread_getprio () from /lib/libthr.so.3
 #5  0x0000000000000000 in ?? ()
 Error accessing memory address 0x7fffffbff000: Bad address.
 (gdb) thread 3994
 [Switching to thread 3994 (Thread 0x801002a10 (LWP 100104))]#0
 0x0000000800db1f8c in kevent () from /lib/libc.so.7
 (gdb) bt
 #0  0x0000000800db1f8c in kevent () from /lib/libc.so.7
 #1  0x000000000040a9d2 in vca_kqueue_main (arg=Variable "arg" is not
 available.
 ) at cache_waiter_kqueue.c:172
 #2  0x0000000800abf4d1 in pthread_getprio () from /lib/libthr.so.3
 #3  0x0000000000000000 in ?? ()
 Error accessing memory address 0x7fffff1fa000: Bad address.
 (gdb) thread 3993
 [Switching to thread 3993 (Thread 0x801002ba0 (LWP 100110))]#0
 0x0000000800d7b07c in poll () from /lib/libc.so.7
 (gdb) bt
 #0  0x0000000800d7b07c in poll () from /lib/libc.so.7
 #1  0x0000000800ac180e in poll () from /lib/libthr.so.3
 #2  0x00000000004098f4 in vca_acct (arg=Variable "arg" is not available.
 ) at cache_acceptor.c:215
 #3  0x0000000800abf4d1 in pthread_getprio () from /lib/libthr.so.3
 #4  0x0000000000000000 in ?? ()
 Error accessing memory address 0x7ffffeff9000: Bad address.
 (gdb) thread 1916
 [Switching to thread 1916 (Thread 0x187aa80ba0 (LWP 103321))]#0
 0x0000000800db67ec in writev () from /lib/libc.so.7
 (gdb) bt
 #0  0x0000000800db67ec in writev () from /lib/libc.so.7
 #1  0x0000000800ac103e in writev () from /lib/libthr.so.3
 #2  0x000000000042a06f in WRW_Flush (w=0x7ffefabda2c0) at cache_wrw.c:104
 #3  0x000000000042a4ae in WRW_FlushRelease (w=0x7ffefabda2c0)
     at cache_wrw.c:124
 #4  0x000000000042246a in RES_WriteObj (sp=0x187de2f008)
     at cache_response.c:197
 #5  0x0000000000411e4a in cnt_deliver (sp=0x187de2f008) at
 cache_center.c:201
 #6  0x0000000000412a43 in CNT_Session (sp=0x187de2f008) at steps.h:42
 #7  0x0000000000422121 in wrk_do_cnt_sess (w=0x7ffefabda2c0, priv=Variable
 "priv" is not available.
 )
     at cache_pool.c:281
 #8  0x0000000000421417 in wrk_thread_real (qp=0x80100f7e0,
 shm_workspace=Variable "shm_workspace" is not available.
 )
     at cache_pool.c:176
 #9  0x0000000800abf4d1 in pthread_getprio () from /lib/libthr.so.3
 #10 0x0000000000000000 in ?? ()
 Error accessing memory address 0x7ffefabdb000: Bad address.
 }}}

 Anything else I can check?

-- 
Ticket URL: <http://varnish.projects.linpro.no/ticket/534#comment:4>
Varnish <http://varnish.projects.linpro.no/>
The Varnish HTTP Accelerator


More information about the varnish-bugs mailing list