Reducing overflowed and dropped work requests
sime at sime.net.au
Thu Aug 18 01:31:09 CEST 2011
On Wed, Aug 3, 2011 at 5:05 PM, Tollef Fog Heen
<tfheen at varnish-software.com> wrote:
> ]] Simon Males
> | Since this post, I increased the thread_pools to 8, though today I've
> | had another spurt of dropped work requests. It's definitely related to
> | a sudden rise of traffic, as the dropped requests happen moment after
> | sending an E-Mail to our users.
> | The servers have 16 cores, so I've increased the thread_pools to 16
> | now. With 10 minimum threads per pool, I now have 160 threads waiting
> | to serve.
> If you're on 2.1 you might want to set thread_pool_add_delay to 2 (ms)
> or similar. It's set a bit too high there, which means it takes a
> little while for Varnish to respond to rapid spikes.
Just wanted to say thanks to the contributors to this thread Václav
Bílek and Tollef Fog Heen.
I actually held back on the advice and wanted to see how far 160
minimum threads would get me (16 pools x 10 minimum). The next spike
produced 320k limited worker threads (n_wrk_max) per server.
I set the following parameters on the fly hoping to manage the spike.
But the real test will be the next spike.
Though now my committed memory usage (according to Munin) is 16G. The
server has a total of 4G and 2G malloc cache. Is there an equation I
can use to determine how much memory I need with my current
Additionally can I deduce from varnishstats how many threads are being
used? e.g. How many of the minimum 1600 are being used.
More information about the varnish-misc