Reducing overflowed and dropped work requests
sime at sime.net.au
Wed Aug 3 08:40:30 CEST 2011
On Fri, Jul 15, 2011 at 11:15 AM, Simon Males <sime at sime.net.au> wrote:
>> The Varnish munin plugins have alerted me a number of times (say less
>> then 5) that N dropped work requests (n_wrk_drop) is greater then 1.
>> According to Kristian Lyngstol's post  this is represents when the
>> request queue it full. Additionally I believe the overflowed work
>> requests (n_wrk_overflow) will always be greater then n_wrk_drop.
>> I think I should be trying to tackle two issues: Limit requests
>> overflowing, followed by increasing the size of the request queue.
>> Of the below startup parameters I believe thread_pools should match
>> the number of cores the server has and increase the maximum number of
>> threads to possibly double (from 25 to 50?).
>> DAEMON_OPTS="-a :80 \
>> -T localhost:6082 \
>> -f /etc/varnish/default.vcl \
>> -S /etc/varnish/secret \
>> -w 10,25,120 \
>> -p listen_depth=8192 \
>> -p log_hashstring=off \
>> -p lru_interval=60 \
>> -p sess_timeout=10 \
>> -p shm_workspace=32768 \
>> -p ping_interval=1 \
>> -p thread_pools=4 \
>> -s malloc,2G"
> As overflow_max is default set to 100%, I believe I essentially need
> to increase the number of worker threads to increase the ratio. To
> increase the threads, I'll increase the thread_pools.
Since this post, I increased the thread_pools to 8, though today I've
had another spurt of dropped work requests. It's definitely related to
a sudden rise of traffic, as the dropped requests happen moment after
sending an E-Mail to our users.
The servers have 16 cores, so I've increased the thread_pools to 16
now. With 10 minimum threads per pool, I now have 160 threads waiting
Hopefully this will minimise the chance of dropping work requests.
More information about the varnish-misc