Reducing overflowed and dropped work requests

Simon Males sime at
Thu Aug 18 01:31:09 CEST 2011

On Wed, Aug 3, 2011 at 5:05 PM, Tollef Fog Heen
<tfheen at> wrote:
> ]] Simon Males
> | Since this post, I increased the thread_pools to 8, though today I've
> | had another spurt of dropped work requests. It's definitely related to
> | a sudden rise of traffic, as the dropped requests happen moment after
> | sending an E-Mail to our users.
> |
> | The servers have 16 cores, so I've increased the thread_pools to 16
> | now. With 10 minimum threads per pool, I now have 160 threads waiting
> | to serve.
> If you're on 2.1 you might want to set thread_pool_add_delay to 2 (ms)
> or similar.  It's set a bit too high there, which means it takes a
> little while for Varnish to respond to rapid spikes.

Just wanted to say thanks to the contributors to this thread Václav
Bílek and Tollef Fog Heen.

I actually held back on the advice and wanted to see how far 160
minimum threads would get me (16 pools x 10 minimum). The next spike
produced 320k limited worker threads (n_wrk_max) per server.

I set the following parameters on the fly hoping to manage the spike.
But the real test will be the next spike.
thread_pool_add_delay 2
thread_pool_max 250
thread_pool_min 100

Though now my committed memory usage (according to Munin) is 16G. The
server has a total of 4G and 2G malloc cache. Is there an equation I
can use to determine how much memory I need with my current

Additionally can I deduce from varnishstats how many threads are being
used? e.g. How many of the minimum 1600 are being used.

Simon Males

More information about the varnish-misc mailing list