[Varnish] #235: Varnish Linux performance

Varnish varnish-bugs at projects.linpro.no
Wed Apr 30 23:27:53 CEST 2008

#235: Varnish Linux performance
 Reporter:  rafaelumann  |       Owner:  phk        
     Type:  defect       |      Status:  new        
 Priority:  high         |   Milestone:             
Component:  varnishd     |     Version:  trunk      
 Severity:  normal       |    Keywords:  performance

 ### Situation:

 We are using Quad Xeon 3Ghz HT servers in our production environment, and
 one varnish thread makes one CPU usage goes to 0% free.

 Process strace repeats:

 clock_gettime(CLOCK_MONOTONIC, {431276, 305144924}) = 0
 epoll_wait(12, {{EPOLLIN, {u32=1371080556, u64=1371080556}}}, 1, 100) = 1
 read(139, "GET /ads//i/bv1024/bv_ml07_v2.pn"..., 4095) = 403
 epoll_ctl(12, EPOLL_CTL_DEL, 139, {0, {u32=0, u64=0}}) = 0
 write(4002, "\317", 1)                  = 1

 This process seems to be the responsible for distributing connections
 between threads and clean the timeout sockets.

 The problem possibly happens because this process tries to cleanup the
 timeout sockets each new connection.

 ### Kernel:

 Kernel: 2.6.18-53.el5PAE

 ### Solution:

 First we tried to use the same code used in varnish for freebsd (kqueue),
 that just searches for sockets with "dotimer=1" to clean timeout sockets,
 but it didn't work out because there are too many connections that would
 never be cleaned, and so too many sockets stayed opened.

 So, as a work around, I annexed a firsthand patch for test, that just add
 a "timeout counter" and cleans the timeout sockets every 60 seconds
 instead of cleaning it in every new connection received. We are using it
 here, and the performance is much better now.

 In the trunk or 1.1.2 version varnish doesn't go further than 4000 "Client
 requests received" with real traffic, but it goes much longer with Apache
 Benchmark traffic. While in our patched version we easily get to "8197.18"
 "Client requests received", without AB.

 Anyway, we don't think that this should be the final solution. Maybe a new
 thread to cleanup the timeout sockets will be a better solution.

 Rafael Umann

Ticket URL: <http://varnish.projects.linpro.no/ticket/235>
Varnish <http://varnish.projects.linpro.no/>
The Varnish HTTP Accelerator

More information about the varnish-bugs mailing list