varnish 2.0.4 backend errors

[Ken Brownfield]

On Jul 14, 2009, at 3:05 AM, Kristian Lyngstol wrote:
> On Tue, Jul 14, 2009 at 11:46:58AM +0200, Lazy wrote:
>> the site is usually not so busy, but it has sometimes spikes of  
>> static
>> traffic (about 50Mbps) that's why i upped the thread limit, 3000 was
>> to low
>
> I seriously doubt 3k was too low. More likely, the min threads was  
> hurting
> you. Three thousand threads is quite a bit. Remember that these are  
> actual
> requests being handled, not keep-alive connections and the like.

I just wanted to humbly second this good advice; if you're familiar  
with Apache, this is akin to making sure your MinSpareServers is set  
to a high enough level to handle any transient spikes by avoiding the  
cost of spawning new processes.  Varnish will handle 10x the traffic  
you're seeing in <64 threads.  Anything you're seeing is more likely a  
concurrency spike causing a temporary slowdown while threads are  
spawned, or Varnish is simply passing on fail or slowness from your  
back-end.

I'm wondering if this could also be a large object that's taking a  
while to cache and blocking other children for a while? (rush_exponent)
-- 
Ken.