Reducing overflowed and dropped work requests

[Simon Males]

On Fri, Jul 15, 2011 at 11:15 AM, Simon Males <sime at sime.net.au> wrote:
>> The Varnish munin plugins have alerted me a number of times (say less
>> then 5) that N dropped work requests (n_wrk_drop) is greater then 1.
>>
>> According to Kristian Lyngstol's post [1] this is represents when the
>> request queue it full. Additionally I believe the overflowed work
>> requests (n_wrk_overflow) will always be greater then n_wrk_drop.
>>
>> I think I should be trying to tackle two issues: Limit requests
>> overflowing, followed by increasing the size of the request queue.
>>
>> Of the below startup parameters I believe thread_pools should match
>> the number of cores the server has and increase the maximum number of
>> threads to possibly double (from 25 to 50?).
>>
>> DAEMON_OPTS="-a :80 \
>>             -T localhost:6082 \
>>             -f /etc/varnish/default.vcl \
>>             -S /etc/varnish/secret \
>>             -w 10,25,120 \
>>             -p listen_depth=8192 \
>>             -p log_hashstring=off \
>>             -p lru_interval=60 \
>>             -p sess_timeout=10 \
>>             -p shm_workspace=32768 \
>>             -p ping_interval=1 \
>>             -p thread_pools=4 \
>>             -s malloc,2G"
>
> As overflow_max is default set to 100%, I believe I essentially need
> to increase the number of worker threads to increase the ratio. To
> increase the threads, I'll increase the thread_pools.

Since this post, I increased the thread_pools to 8, though today I've
had another spurt of dropped work requests. It's definitely related to
a sudden rise of traffic, as the dropped requests happen moment after
sending an E-Mail to our users.

The servers have 16 cores, so I've increased the thread_pools to 16
now. With 10 minimum threads per pool, I now have 160 threads waiting
to serve.

Hopefully this will minimise the chance of dropping work requests.

-- 
Simon Males