[Varnish] #437: Varnish 2.0.2 hanging with large amounts of traffic

[Varnish]

#437: Varnish 2.0.2 hanging with large amounts of traffic
-------------------+--------------------------------------------------------
 Reporter:  ajt    |        Type:  defect  
   Status:  new    |    Priority:  normal  
Milestone:         |   Component:  build   
  Version:  trunk  |    Severity:  critical
 Keywords:         |  
-------------------+--------------------------------------------------------
 Varnish 2.0.2 stops accepting connections after a couple of hours of
 accepting 55-60Mbits of traffic. This is on an ubuntu 8.04 LTS server
 build from the 2.0.2 source. Ulimit is set to 200000.

 Uname:

 {{{
 Linux ded203 2.6.24-16-server #1 SMP Thu Apr 10 13:15:38 UTC 2008 x86_64
 GNU/Linux
 }}}



 Here is a varnishstat:

 {{{
 0+09:18:17
 ded203
 Hitrate ratio:        1        1        1
 Hitrate avg:     0.6119   0.6119   0.6119

       526888         0.00        15.73 Client connections accepted
       964084         0.00        28.78 Client requests received
       115238         0.00         3.44 Cache hits
            1         0.00         0.00 Cache hits for pass
        73091         0.00         2.18 Cache misses
       848889         0.00        25.34 Backend connections success
            0         0.00         0.00 Backend connections not attempted
            0         0.00         0.00 Backend connections too many
            5         0.00         0.00 Backend connections failures
         8246         0.00         0.25 Backend connections reuses
        11101         0.00         0.33 Backend connections recycles
            0         0.00         0.00 Backend connections unused
         1324          .            .   N struct srcaddr
 18446744073709551613          .            .   N active struct srcaddr
          671          .            .   N struct sess_mem
            1          .            .   N struct sess
        33041          .            .   N struct object
        28096          .            .   N struct objecthead
        72658          .            .   N struct smf
 }}}

 Here is an iostat:
 {{{
 iostat -x -k 5
 Linux 2.6.24-16-server (ded203)         02/04/2009

 avg-cpu:  %user   %nice %system %iowait  %steal   %idle
            0.10    0.00    0.18    0.09    0.00   99.63

 Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz
 avgqu-sz   await  svctm  %util
 sda               0.45     7.24    0.26    1.08     7.28    33.26    60.90
 0.10   74.04   3.12   0.42

 avg-cpu:  %user   %nice %system %iowait  %steal   %idle
            0.19    0.00    0.19    0.05    0.00   99.58

 Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz
 avgqu-sz   await  svctm  %util
 sda               0.00     0.00    0.40    0.00     2.40     0.00    12.00
 0.00    5.00   5.00   0.20

 avg-cpu:  %user   %nice %system %iowait  %steal   %idle
            0.09    0.00    0.24    0.14    0.00   99.53

 Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz
 avgqu-sz   await  svctm  %util
 sda               2.40     0.00    0.40    0.00    11.20     0.00    56.00
 0.00   10.00  10.00   0.40
 }}}

-- 
Ticket URL: <http://varnish.projects.linpro.no/ticket/437>
Varnish <http://varnish.projects.linpro.no/>
The Varnish HTTP Accelerator