how small can I make the thread stack size?

[Ken Brownfield]

+1.  256k was as low as I could safely go on x86_64, but it's been stable.

I used the ulimit hack for a bit, and it worked fine.

The official patch only controls workers, but the patch below will extend the control to the backend threads, which also scale with load.  If you aren't already building varnish from source, the ulimit trick is probably just fine.
-- 
Ken

--- bin/varnishd/cache_backend_poll.c.orig	2009-04-01 06:35:15.000000000 -0700
+++ bin/varnishd/cache_backend_poll.c	2009-07-27 13:23:55.000000000 -0700
@@ -406,6 +414,7 @@
 {
 	struct vbp_target *vt;
 	struct vsb *vsb;
+	pthread_attr_t be_attr;
 
 	ASSERT_CLI();
 
@@ -444,7 +453,15 @@
 
 	VTAILQ_INSERT_TAIL(&vbp_list, vt, list);
 
-	AZ(pthread_create(&vt->thread, NULL, vbp_wrk_poll_backend, vt));
+	/* Set the stacksize for backend threads */
+	pthread_attr_init(&be_attr);
+	if (params->wthread_stacksize) {
+		int pt_res = pthread_attr_setstacksize(&be_attr, params->wthread_stacksize);
+		VSL(SLT_Debug, 0, "Attempt to set backend thread stacksize to %u %s",
+		    params->wthread_stacksize, pt_res ? "FAILED" : "succeeded");
+	}
+
+	AZ(pthread_create(&vt->thread, &be_attr, vbp_wrk_poll_backend, vt));
 }
 
 void


On Jul 28, 2010, at 2:24 AM, Chris Hecker wrote:

> 
> Alright, I'll try 256kb and see how it goes.  I'm doing it with a config param to varnishd, which unfortunately makes it still use 10mb stacks for the other 6 threads it's running.  I wonder if I should use ulimit in the init script to avoid this and save 60mb...
> 
> Thanks,
> Chris
> 
> 
> On 2010/07/28 01:29, Cosimo Streppone wrote:
>> On Wed, 28 Jul 2010 08:02:50 +0200, Chris Hecker <checker at d6.com> wrote:
>> 
>>> [stack size] I've seen people talking about 256kb, but then the docs
>>> are pretty sketchy on whether it's a good idea to change it at all,
>>> Any advice?
>> 
>> We're running several varnish instances.
>> All of them x86-64 though, so YMMV.
>> 
>> For the busiest machines, stack size has been set to 256kb, by putting
>> 'ulimit -s 256' in the init script.
>> 
>> IIRC, we started doing this because we saw increases in the
>> 'thread_limited'
>> counter.
>> We saw that varnishd couldn't create new threads,
>> even if the hard limit (thread_pool_max) hadn't been reached yet.
>> Check:
>> 
>> http://lists.varnish-cache.org/pipermail/varnish-misc/2009-August/002977.html
>> 
>> 
>> These particular instances are constantly running 7k threads,
>> occasionally going to 8-9k. With 24 Gb of RAM.
>>