[Varnish] #273: running out of virtual memory on 32 bit server

[Varnish]

#273: running out of virtual memory on 32 bit server
------------------------+---------------------------------------------------
 Reporter:  chrisrixon  |       Owner:  des        
     Type:  defect      |      Status:  new        
 Priority:  normal      |   Milestone:  Varnish 1.2
Component:  build       |     Version:  1.1.2      
 Severity:  major       |    Keywords:             
------------------------+---------------------------------------------------
 Well we are still seeing varnish run out of virtual memory. Here is a
 varnishdump of when it reaches about 3.5G.

 We wonder if we are falling foul of the policy of:
 Allocate enough memory for 99% of the cases up front
 Unused & untouched memory is free – as in ”peanuts”

 All we can think of is that our server is so busy requests are coming in
 faster than they can be garbage collected.

 I guess its time to go 64bit, seems silly when the current servers are 90%
 idle!

 If anyone has any ideas it would be great to hear from you.

 Centos 5, 32 bit, Basic settings with 128M file

 0+00:03:11
 Hitrate ratio:       10      100      558
 Hitrate avg:     0.8030   0.7862   0.7786

        15732        85.01        82.37 Client connections accepted
        29255       156.01       153.17 Client requests received
        22413       126.01       117.35 Cache hits
          248         0.00         1.30 Cache hits for pass
         6508        29.00        34.07 Cache misses
         6839        29.00        35.81 Backend connections success
            0         0.00         0.00 Backend connections failures
         6084        27.00        31.85 Backend connections reuses
         6759        28.00        35.39 Backend connections recycles
            1        -1.00         0.01 Backend connections unused
         3954          .            .   N struct srcaddr
          723          .            .   N active struct srcaddr
          816          .            .   N struct sess_mem
          782          .            .   N struct sess
         2259          .            .   N struct object
         2244          .            .   N struct objecthead
         2024          .            .   N struct smf
           19          .            .   N small free smf
            1          .            .   N large free smf
           27          .            .   N struct vbe_conn
          270          .            .   N worker threads
          270         0.00         1.41 N worker threads created
            0         0.00         0.00 N worker threads not created
            0         0.00         0.00 N worker threads limited
            0         0.00         0.00 N queued work requests
          270         0.00         1.41 N overflowed work requests
            0         0.00         0.00 N dropped work requests
         4501          .            .   N expired objects
         1041          .            .   N objects on deathrow
            0         0.00         0.00 HTTP header overflows
            0         0.00         0.00 Objects sent with sendfile
        25350       125.01       132.72 Objects sent with write
        15574        80.01        81.54 Total Sessions
        29001       146.01       151.84 Total Requests
           31         0.00         0.16 Total pipe
            0         0.00         0.00 Total pass
         6759        30.00        35.39 Total fetch
      9348459     45370.50     48944.81 Total header bytes
    125553523    560967.24    657348.29 Total body bytes
          526         6.00         2.75 Session Closed
           11         0.00         0.06 Session Pipeline
            0         0.00         0.00 Session Read Ahead
        28709       142.01       150.31 Session herd
      1448734      7050.54      7584.99 SHM records

-- 
Ticket URL: <http://varnish.projects.linpro.no/ticket/273>
Varnish <http://varnish.projects.linpro.no/>
The Varnish HTTP Accelerator