Varnish using malloc, but still writing to disk?

[Jorge]

Yes, if you want to avoid writes to disk, mount tmpfs over this directory,
that's what I did.

Cheers.

On Mon, Apr 27, 2015 at 7:18 PM, Jason Price <japrice at gmail.com> wrote:

> Self reply:  Does varnish rewrite the
> /var/lib/varnish/<VARNISH_NAME>/_.vsm file periodically?
>
> <googling>
>
> http://www.gossamer-threads.com/lists/varnish/misc/24678
>
> <d'oh!>
>
> Looks like the big control-knob for this is vm.dirty_expire_centisecs
> for modern linux kernels.  Every interval, an async copy of the shared
> memory log goes to disk.  I'll be evaluating just tweaking this, or
> moving it to a tmpfs volume.  yay.
>
> -Jason
>
> On Mon, Apr 27, 2015 at 11:50 AM, Jason Price <japrice at gmail.com> wrote:
> > Varnish invocation:
> >
> > /usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -n vorigin -f
> > /etc/varnish/vorigin.vcl -T 127.0.0.1:6082 -t 120 -w 2000,8000,300 -p
> > thread_pools=2 -p thread_pool_workspace=100000 -p
> > sess_workspace=1000000 -u varnish -g varnish -S /etc/varnish/secret -s
> > malloc,90G
> >
> > We've had a production issue and I've been asserting that varnish
> > doesn't write anything to disk, so the root volume IO spike we're
> > seeing CAN'T be varnish.
> >
> > Of course, I look into iotop (specifically 'iotop -obPk') I see
> > periodic times where the output is as follows:
> >
> >   PID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
> >     6 be/4 root        0.00 K/s    0.00 K/s  0.00 % 33.26 %
> [kworker/u128:0]
> >  4185 be/4 varnish     0.00 K/s 6673.66 K/s  0.00 %  0.00 % varnishd
> > -P /var/run/varnish.pid -a :80 -n vorigin -f /etc/varnish/vorigin.vcl
> > -T 127.0.0.1:6082 -t 120 -w 2000,8000,300 -p thread_pools=2 -p
> > thread_pool_workspace=100000 -p sess_workspace=1000000 -u varnish -g
> > varnish -S /etc/varnish/secret -s malloc,90G
> > Total DISK READ: 0.00 K/s | Total DISK WRITE: 0.00 K/s
> >   PID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN      IO    COMMAND
> >  4185 be/4 varnish     0.00 K/s 12659.41 K/s  0.00 %  0.00 % varnishd
> > -P /var/run/varnish.pid -a :80 -n vorigin -f /etc/varnish/vorigin.vcl
> > -T 127.0.0.1:6082 -t 120 -w 2000,8000,300 -p thread_pools=2 -p
> > thread_pool_workspace=100000 -p sess_workspace=1000000 -u varnish -g
> > varnish -S /etc/varnish/secret -s malloc,90G
> >
> > I've dug into it with LSOF and sundry, and I can't figure out
> > what/where varnish is doing the writing.  It's not constant... 10-20
> > seconds will pass before I see it again for several seconds (2-8ish).
> >
> > NOTE: normal load for these boxes is 3-10k req/sec.
> >
> > Any thoughts?  Why is varnish writing to disk (more to the point:
> > writing tens megabytes at a time)?
> >
> > varnish version 3.0.5
> > Amazon Linux
> > Kernel 3.14.35-28
> >
> > Thank you for any help;
> > -Jason
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>



-- 
Jorge Nerín
<jnerin at gmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20150427/5b40a555/attachment.html>