Proper cache allocation (nuked objects but sm_bfree)

[Kristian Lyngstol]

Ok, I'll reply to my own mail, since I've gotten two, distinct, replies,
both which point out a few mistakes I made.

On Thu, Sep 08, 2011 at 09:40:13AM +0200, Kristian Lyngstol wrote:
> On Wed, Sep 07, 2011 at 11:46:34AM -0700, Damon Snyder wrote:
> > varnishstat -1 -f n_lru_nuked,sm_bfree,sm_balloc
> > n_lru_nuked         135193763          .   N LRU nuked objects
> > sm_balloc          5468946432          .   bytes allocated
> > sm_bfree           2047246336          .   bytes free
> 
> sm_bfree is a counter of how much memory has been freed, not how much is
> available. Every time an object is removed, expired, etc, this will
> increase, and it is never reduced. _balloc is the counter part of that,
> and is increased every time something is allocated, and never reduced.
> From these numbers you can calculate how much memory is currently used:
> 
>  5468946432 - 2047246336 = 3421700096 (a little over 3GB)

Ok, for Varnish 3 and -s malloc in Varnish 2, this is true. For -s file,
which you are using, it is NOT true. Terribly sorry for the confusion.

In Varnish 2, sma_balloc and sma_bfree are counters that never are
reduced, and sma_nbytes is the value of sma_balloc-sma_bfree.

In Varnish 2, sm_balloc is the size of allocated data, while sm_bfree is
the size of data you can allocate. They both go up and down, and there
is no sm_nbytes.

So, to correct myself:

You are using roughly 5GB of space and have roughly 2GB of space
available. That does not necessarily map directly to real memory.

If you are concerned with disk I/O, I would use -s malloc on 2.1. And
regardless, I would but /var/lib/varnish/ on a tmpfs (but make sure you
don't also use -s file,/var/lib/varnish/..., or you'll have a bit of a
problem). The tmpfs for /var/lib/varnish/ is meant for the 80MB shmlog,
to ensure it's never flushed to disk.

As for why you have lru nukes, I really can't say. If you have a few
very large files, that could explain it, or if your size usage goes up
and down.

I haven't done the math for 8GB systems on 2.1 recently, but given that
you're using -s file,7GB and the number of objects you have indicate a
little less than 200MB of overhead, + whatever else (let's say 100MB for
threads, sessions and just misc), you're cutting it a bit close. But
then again, this could just be the other reasons stated above.

The rest of my mail (about the 1kB+/object oberhead) holds true.

- Kristian