Hi Kristian,<div>Thanks again for the explanation. It forced me to look at the code to try and get a better sense for what is going on. </div><div><br></div><div>Can you explain the meaning of n_smf_frag and n_smf_large? They are missing explanations here <a href="https://www.varnish-cache.org/trac/wiki/StatsExplained">https://www.varnish-cache.org/trac/wiki/StatsExplained</a>. From the code, it appears that they are just counters. On one of our boxes n_smf_large is > 10k and there is no nuking. On the other n_smf_large is close to 0 and there is a lot of nuking. I was trying to understand the relationship.</div>
<div><br></div><div>Maybe we should re-evaluate our storage type. From a quick look at the malloc storage type, it looks like objects of arbitrary size are being allocated as opposed to rounding up to the next nearest page size (more space efficient) and the large objects aren't confined to the last bucket.</div>
<div><br></div><div>Thanks,</div><div>Damon<br><br><div class="gmail_quote">On Fri, Sep 9, 2011 at 6:50 AM, Kristian Lyngstol <span dir="ltr"><<a href="mailto:kristian@varnish-software.com">kristian@varnish-software.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Ok, I'll reply to my own mail, since I've gotten two, distinct, replies,<br>
both which point out a few mistakes I made.<br>
<div class="im"><br>
On Thu, Sep 08, 2011 at 09:40:13AM +0200, Kristian Lyngstol wrote:<br>
> On Wed, Sep 07, 2011 at 11:46:34AM -0700, Damon Snyder wrote:<br>
</div><div class="im">> > varnishstat -1 -f n_lru_nuked,sm_bfree,sm_balloc<br>
> > n_lru_nuked 135193763 . N LRU nuked objects<br>
> > sm_balloc 5468946432 . bytes allocated<br>
> > sm_bfree <a href="tel:2047246336" value="+12047246336">2047246336</a> . bytes free<br>
><br>
> sm_bfree is a counter of how much memory has been freed, not how much is<br>
> available. Every time an object is removed, expired, etc, this will<br>
> increase, and it is never reduced. _balloc is the counter part of that,<br>
> and is increased every time something is allocated, and never reduced.<br>
> From these numbers you can calculate how much memory is currently used:<br>
><br>
> 5468946432 - <a href="tel:2047246336" value="+12047246336">2047246336</a> = 3421700096 (a little over 3GB)<br>
<br>
</div>Ok, for Varnish 3 and -s malloc in Varnish 2, this is true. For -s file,<br>
which you are using, it is NOT true. Terribly sorry for the confusion.<br>
<br>
In Varnish 2, sma_balloc and sma_bfree are counters that never are<br>
reduced, and sma_nbytes is the value of sma_balloc-sma_bfree.<br>
<br>
In Varnish 2, sm_balloc is the size of allocated data, while sm_bfree is<br>
the size of data you can allocate. They both go up and down, and there<br>
is no sm_nbytes.<br>
<br>
So, to correct myself:<br>
<br>
You are using roughly 5GB of space and have roughly 2GB of space<br>
available. That does not necessarily map directly to real memory.<br>
<br>
If you are concerned with disk I/O, I would use -s malloc on 2.1. And<br>
regardless, I would but /var/lib/varnish/ on a tmpfs (but make sure you<br>
don't also use -s file,/var/lib/varnish/..., or you'll have a bit of a<br>
problem). The tmpfs for /var/lib/varnish/ is meant for the 80MB shmlog,<br>
to ensure it's never flushed to disk.<br>
<br>
As for why you have lru nukes, I really can't say. If you have a few<br>
very large files, that could explain it, or if your size usage goes up<br>
and down.<br>
<br>
I haven't done the math for 8GB systems on 2.1 recently, but given that<br>
you're using -s file,7GB and the number of objects you have indicate a<br>
little less than 200MB of overhead, + whatever else (let's say 100MB for<br>
threads, sessions and just misc), you're cutting it a bit close. But<br>
then again, this could just be the other reasons stated above.<br>
<br>
The rest of my mail (about the 1kB+/object oberhead) holds true.<br>
<font color="#888888"><br>
- Kristian<br>
</font></blockquote></div><br></div>