[Varnish] #188: thread pileup

[Varnish]

#188: thread pileup
----------------------+-----------------------------------------------------
 Reporter:  steinove  |        Owner:  phk     
     Type:  defect    |       Status:  assigned
 Priority:  high      |    Milestone:          
Component:  varnishd  |      Version:  1.1.1   
 Severity:  normal    |   Resolution:          
 Keywords:            |  
----------------------+-----------------------------------------------------
Comment (by frekri):

 Replying to [comment:5 frekri]:
 > Replying to [comment:1 phk]:
 > > My best diagnosis, is that the backend is very slow for some
 particular popular object.
 > >
 > > One thread picks it up from the backend and a LOT of requests gets
 parked on the busy object until it comes back from the backend.
 > >
 > > Once it comes back, all the parked requests gets reinjected and causes
 the thread explosion.
 > >
 > > Some kind of pacing algorithm is obviously called for, the trick is to
 find one that doesn't need too many carefully tweaked numbers to work
 generally and reliably.
 > >
 > > Thinking...
 > >
 > > /phk
 >
 > On Nettby.no varnish crash almost every day. We run 4 varnish servers.
 After I read your post I moved all objects with high traffic to lighttpd
 and objects with less traffic to varnish. In total these objects had a lot
 of traffic.
 >
 > On each server we had 1100 hits/s before the last crash today. Before
 the crash we had increased iowait, but hits and everything else were
 normal.

 Last described crash was because of to slow Swap Space(Disk). I have
 reconfigured varnish to use less memory than available ram.

 I was able to recreate the thread problem this weekend and I'm able to
 crash varnish like Steinove described on purpose when I gave it enough
 non-cached requests from our users.

 Let us know when you have a patch and we will test it for you.

-- 
Ticket URL: <http://varnish.projects.linpro.no/ticket/188#comment:6>
Varnish <http://varnish.projects.linpro.no/>
The Varnish HTTP Accelerator