100% CPU IOwait

[Per Buer]

Hi,


On Tue, Nov 3, 2015 at 1:42 PM, Caires Vinicius <cairesvs at gmail.com> wrote:

> We've started to use Varnish 4 with Amazon Linux with EBS SSD of 40GB,
> memory of 7.5GB. We use the file storage with 20G allocated with ttl of 11
> minutes and grace of 5 hours, all the other configs are standard.
>
I would, on a general basis, recommend against using the file backend. It
will start to struggle with fragmentation relatively quickly and the
performance isn't all that great (lots of unnecessary synchronous reads).

> Sometimes when we have a lot of request that result into cache miss we
> started to notice that our request latency grows and the iowait stays at
> 100%, something similar to this
> https://www.varnish-cache.org/lists/pipermail/varnish-misc/2008-April/01...
> <https://www.varnish-cache.org/lists/pipermail/varnish-misc/2008-April/016139.html>.
> And our threads reaches the maximum (1000).
>
> Do you guys have any idea why is that?
>
Yeah. New objects get assign a piece of memory, starts writing, triggers
pagefault, kernel takes over and reads/merges the underlying page, varnish
then overwrites that page which then gets written back to disk. This
naturally slows down delivery so Varnish spawns new threads.

Try malloc. You should start with -s malloc,30G or there about - if you
have lots of small objects you might need to go a bit down to avoid
swapping.

Not related: You should also move /var/lib/varnish onto tempfs. Linux will
do a lot of writing if the shared memory segment is visible on a filesystem
that is backed by a disk.
-- 
*Per Buer*
CTO | Varnish Software AS
Cell: +47 95839117
We Make Websites Fly!
www.varnish-software.com
<http://info.varnish-software.com/signature>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20151103/390807fb/attachment-0001.html>