Proper cache allocation (nuked objects but sm_bfree)

[Matt Schurenko]

I've been trying to understand these stats myself. A couple of questions regarding your post Kristian:

1) Where can I find this _nbytes statistic you speak of? This is all I get from running varnishstat on varnish 2.1.5:

[root at varnish-test ~]# varnishstat -1 | grep _nbytes
sma_nbytes                  0          .   SMA outstanding bytes
sms_nbytes                  0          .   SMS outstanding bytes

2) If sm_bfree and sm_balloc are indeed counters wouldn't they go up indefinitely? I find that if I add those two values together the sum equals the size of the cache I specified (in my case 40GB):

[root at varnish-test ~]# varnishstat -1 | grep ^sm_bfree
sm_bfree          32433844224          .   bytes free
[root at varnish-test ~]# varnishstat -1 | grep ^sm_balloc
sm_balloc         10515988480          .   bytes allocated
[root at varnish-test ~]# echo  '(32433844224 + 10515988480) / 1024 / 1024 / 1024' | bc
40
[root at varnish-test ~]# ps -ef | grep varnish
root      3574     1  0 Aug24 ?        00:00:00 /usr/local/sbin/varnishd -s file,/tmp/varnish-cache,40G -T 127.0.0.1:2000 -a 0.0.0.0:80 -t 604800 -f /usr/local/etc/varnish/default.vcl 


Thanks, 

Matt Schurenko
Systems Administrator
 
airG share your world
Suite 710, 1133 Melville Street
Vancouver, BC  V6E 4E5
P: +1.604.408.2228
F: +1.866.874.8136
E: MSchurenko at airg.com
W: www.airg.com



-----Original Message-----
From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Kristian Lyngstol
Sent: September-08-11 12:40 AM
To: Damon Snyder
Cc: varnish-misc at varnish-cache.org
Subject: Re: Proper cache allocation (nuked objects but sm_bfree)

On Wed, Sep 07, 2011 at 11:46:34AM -0700, Damon Snyder wrote:
> According to the docs, the key statistic to look at is n_lru_nuked. This
> value is constantly increasing. Every time you run 'varnishstat -1 -f
> n_lru_nuked' the value changes. However, the value of sm_bfree seems to
> always show some space available:
> 
> varnishstat -1 -f n_lru_nuked,sm_bfree,sm_balloc
> n_lru_nuked         135193763          .   N LRU nuked objects
> sm_balloc          5468946432          .   bytes allocated
> sm_bfree           2047246336          .   bytes free

sm_bfree is a counter of how much memory has been freed, not how much is
available. Every time an object is removed, expired, etc, this will
increase, and it is never reduced. _balloc is the counter part of that,
and is increased every time something is allocated, and never reduced.
>From these numbers you can calculate how much memory is currently used:

 5468946432 - 2047246336 = 3421700096 (a little over 3GB)

However, you don't have to actually do that yourself, as _nbytes is
doing exactly that.

An important detail is that the size specified in the -s arguments is
/not/ the total memory footprint varnish will have. It is only the total
cache size for actual data, not counting overhead. For Varnish 2.1, we
know that there's an additional overhead of slightly over 1kB for each
object stored, assuming 64-bit systems (your millage may vary, but this
gives you an idea).

On top of that is a bit of data for things like threads and sessions,
but I rarely take that into consideration myself as it will be a
practically constant size measured in MB. Assuming the memory-footprint
for each thread is 10kB (it's unlikely that it's that large, specially
considering copy-on-write and whatnot), 1000 threads will give a
constant overhead of 10MB, so not something to consider.

However, 10 million objects will give 10GB of overhead, so that should
be accounted for when you decide how much memory the cache will use with
-s. Examples I've run with is -s malloc,24GB and -s malloc,28GB on two
different sites, both running on a 32GB-system but with different
average object-size.

Hopefully this helps you figure out what's going on.

- Kristian


_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc