file vs malloc

[Wido den Hollander]

Hi,

> - If you have SSD drives you can mostly do what you want. I don't know
> of anyone running huge varnish process that swap alot - but it _might_
> work. If you are, let me know.

Just wanted to add, there is someone running with a lot of storage with
malloc/swap:

* http://varnish-cache.org/trac/ticket/784

Wido

On Fri, 2010-11-26 at 20:39 +0100, Per Buer wrote:
> 
> 
> On Fri, Nov 26, 2010 at 4:41 PM, Angelo Höngens
> <A.Hongens at netmatch.nl> wrote:
>         Per wrote in his survey result: "62% use -s file, which I find
>         somewhat surprising. -s file doesn't really perform that well
>         under pressure on rotating hard drives due to some changes in
>         recent Linux and FreeBSD kernels."
>         
>         But does anyone have a link to (or write up) some explanation
>         for dummies on cache file types? I know there's 'file' and
>         'malloc', but I don't know the difference or the pros and
>         cons.
> 
> 
> In Varnish 1.0 we had -s file and everything was fine. People where
> using Varnish on  Linux 2.6.9 (and thereabout) and FreeBSD6. Then came
> 2.6.18 with RHEL5 and some other Linux distros and Varnish started
> behaving a bit strange. The IO pressure had increased and we saw load
> skyrocket with Varnish being stuck IO sleep a lot of the time, waiting
> for writes to finish. Then FreeBSD7 came out and showed the
> same behavior. Obviously the kernel developers of the world where
> ganging up on us.
> 
> 
> Alterations in vm behavior are difficult things to track down and the
> chances of changing the behavior back might be slim - Varnish being
> something of an oddball doing crazy stuff like mmaping several
> gigabytes and accessing it in a random fasion (and most of them saying
> things like "Wow, that actually works?" when hearing what we do). 
> 
> 
> Anyway, Poul came up with -s malloc, which just allocates slabs of
> ordinary memory and stores everything there. Since ordinary memory
> isn't backed by disk a write to memory doesn't trigger a write to disk
> so there are no sync disk writes to slow us down. On the flip side -
> at least on Linux, if your data set doesn't dit in memory - the moment
> you start to actively use your swapping space things grind to a halt.
> FreeBSD seems to cope better with programs being larger then the
> amount of memory.
> 
> 
> Several people are running high traffic sites with -s file backed by
> SSD. That seem to perform very well on Linux. Our main FreeBSD user,
> Anders N, uses -s malloc so I don't know much about how -s file on
> FreeBSD anymore.
> 
> 
> So, to sum up:
>  - If you have SSD drives you can mostly do what you want. I don't
> know of anyone running huge varnish process that swap alot - but it
> _might_ work. If you are, let me know.
>  - If your varnish process doesn't write that much you're probably in
> the clear no matter what you'll do.
>  - If you have a small data set that fits neatly in memory, use -s
> malloc.
>  - If you you have a large data set that doesn't fit in memory you
> have to get SSD drives.
> 
> 
> Oh, and at least on Linux there are a bunch of tunables that might
> alter or even rectify some of the misbehavior we've seen. 
> 
> 
> 
> 
> I should clean this up, get some tests done and write it up
> somewhere. 
> -- 
> Per Buer,  Varnish Software
> Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / skype: per.buer
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc