abnormally high load?

[Ken Brownfield]

I never found a way to see how much stack is /used/ vs. /allocated/ in  
a process or thread, so it would be great if someone had ideas?

I could only experiment in production, first moving us to 1MB, then  
256KB.  I've yet to see any issues at 256KB, but we can reach the  
upper limits of thread-count sanity on our boxes with that setting, so  
I haven't dropped it further in production.

The minimums I reached in minimal testing were 128KB with the ulimit  
method, and 64KB with the (IMHO cleaner) backend/worker-thread-only  
approach.  I'm not sure what in Varnish would use more than that much  
stack, but 256KB seems like the sweet spot.

We're 64-bit Ubuntu, and I would assume that a somewhat smaller stack  
would work on 32-bit, possibly making 128KB safe.

Unless you're doing recursion or using large declared structures in  
inline C, I wouldn't think you'd see large stack allocations or huge  
shifts in allocation during operation.  I don't /believe/ objects are  
ever allocated on the stack, or that there's a lot of recursion in the  
code.

FWIW I personally don't see any red flags.
-- 
Ken

On Aug 12, 2009, at 5:38 PM, Mark Moseley wrote:
> Not looking to hijack the thread, but that got my attention. Is there
> any rule of thumb that would determine how small of a stack size a
> varnish could away with and still perform ok? I dropped our stack size
> to 1 meg, which seemed drastic at the time. But if you're doing 256k,
> that's even better. We're using varnish in a web hosting environment
> on 32-bit Debian Etch boxes (without hope of going 64-bit anytime
> soon), so I can cram about 800 threads total without hitting the 3gb
> limit. Traffic patterns are pretty much random, i.e. size of requests
> are all over the board, though the average is about 40k. Hit rate is
> ~25% (which sounds awful but for web hosting, we're overjoyed). Any
> red flags for 256kb threads there?