100% CPU IOwait

[Caires Vinicius]

We had some problems with malloc with the same kind of aws instance and the -s
malloc,5.8G(80% of the memory total). The only trace of the error was a cannot
fork cannot allocate memory into syslog. We're probably missing some point,
maybe the instance size ins't the right fit for us.

On Tue, Nov 3, 2015 at 10:56 AM Per Buer <perbu at varnish-software.com> wrote:

> Hi,
>
>
> On Tue, Nov 3, 2015 at 1:42 PM, Caires Vinicius <cairesvs at gmail.com>
> wrote:
>
>> We've started to use Varnish 4 with Amazon Linux with EBS SSD of 40GB,
>> memory of 7.5GB. We use the file storage with 20G allocated with ttl of 11
>> minutes and grace of 5 hours, all the other configs are standard.
>>
> I would, on a general basis, recommend against using the file backend. It
> will start to struggle with fragmentation relatively quickly and the
> performance isn't all that great (lots of unnecessary synchronous reads).
>
>> Sometimes when we have a lot of request that result into cache miss we
>> started to notice that our request latency grows and the iowait stays at
>> 100%, something similar to this
>> https://www.varnish-cache.org/lists/pipermail/varnish-misc/2008-April/01...
>> <https://www.varnish-cache.org/lists/pipermail/varnish-misc/2008-April/016139.html>.
>> And our threads reaches the maximum (1000).
>>
>> Do you guys have any idea why is that?
>>
> Yeah. New objects get assign a piece of memory, starts writing, triggers
> pagefault, kernel takes over and reads/merges the underlying page, varnish
> then overwrites that page which then gets written back to disk. This
> naturally slows down delivery so Varnish spawns new threads.
>
> Try malloc. You should start with -s malloc,30G or there about - if you
> have lots of small objects you might need to go a bit down to avoid
> swapping.
>
> Not related: You should also move /var/lib/varnish onto tempfs. Linux will
> do a lot of writing if the shared memory segment is visible on a filesystem
> that is backed by a disk.
> --
> *Per Buer*
> CTO | Varnish Software AS
> Cell: +47 95839117
> We Make Websites Fly!
> www.varnish-software.com
> <http://info.varnish-software.com/signature>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20151103/37fd3ff2/attachment.html>