file vs malloc
Wido den Hollander
wido at widodh.nl
Fri Nov 26 22:47:45 CET 2010
> - If you have SSD drives you can mostly do what you want. I don't know
> of anyone running huge varnish process that swap alot - but it _might_
> work. If you are, let me know.
Just wanted to add, there is someone running with a lot of storage with
On Fri, 2010-11-26 at 20:39 +0100, Per Buer wrote:
> On Fri, Nov 26, 2010 at 4:41 PM, Angelo Höngens
> <A.Hongens at netmatch.nl> wrote:
> Per wrote in his survey result: "62% use -s file, which I find
> somewhat surprising. -s file doesn't really perform that well
> under pressure on rotating hard drives due to some changes in
> recent Linux and FreeBSD kernels."
> But does anyone have a link to (or write up) some explanation
> for dummies on cache file types? I know there's 'file' and
> 'malloc', but I don't know the difference or the pros and
> In Varnish 1.0 we had -s file and everything was fine. People where
> using Varnish on Linux 2.6.9 (and thereabout) and FreeBSD6. Then came
> 2.6.18 with RHEL5 and some other Linux distros and Varnish started
> behaving a bit strange. The IO pressure had increased and we saw load
> skyrocket with Varnish being stuck IO sleep a lot of the time, waiting
> for writes to finish. Then FreeBSD7 came out and showed the
> same behavior. Obviously the kernel developers of the world where
> ganging up on us.
> Alterations in vm behavior are difficult things to track down and the
> chances of changing the behavior back might be slim - Varnish being
> something of an oddball doing crazy stuff like mmaping several
> gigabytes and accessing it in a random fasion (and most of them saying
> things like "Wow, that actually works?" when hearing what we do).
> Anyway, Poul came up with -s malloc, which just allocates slabs of
> ordinary memory and stores everything there. Since ordinary memory
> isn't backed by disk a write to memory doesn't trigger a write to disk
> so there are no sync disk writes to slow us down. On the flip side -
> at least on Linux, if your data set doesn't dit in memory - the moment
> you start to actively use your swapping space things grind to a halt.
> FreeBSD seems to cope better with programs being larger then the
> amount of memory.
> Several people are running high traffic sites with -s file backed by
> SSD. That seem to perform very well on Linux. Our main FreeBSD user,
> Anders N, uses -s malloc so I don't know much about how -s file on
> FreeBSD anymore.
> So, to sum up:
> - If you have SSD drives you can mostly do what you want. I don't
> know of anyone running huge varnish process that swap alot - but it
> _might_ work. If you are, let me know.
> - If your varnish process doesn't write that much you're probably in
> the clear no matter what you'll do.
> - If you have a small data set that fits neatly in memory, use -s
> - If you you have a large data set that doesn't fit in memory you
> have to get SSD drives.
> Oh, and at least on Linux there are a bunch of tunables that might
> alter or even rectify some of the misbehavior we've seen.
> I should clean this up, get some tests done and write it up
> Per Buer, Varnish Software
> Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
More information about the varnish-misc