Question about varnish memory usage

[Per Buer]

Hi

On Thu, Sep 18, 2014 at 10:30 PM, Adam Schumacher <
adam.schumacher at flightaware.com> wrote:

>
> There are 24 thread pools because there are 24 cores in the server and I’m
> sure someone read somewhere that a pool per was the way to go.  Probably
> that was a long time ago in a galaxy far, far away….
>


There was an assumption that the thread pools locking structures could stay
local on each core and as such we did at some point recommend having one
pool per core. However, we've been unable to see any performance increase
in going past two pools. I think there might even be a small performance
degradation with using many pools, but I'm not certain. Search the archive
if you want to find out.

Anyways, two pool is enough.


> What are the significant differences between classic and cribit hashing?
>

During the 2.1 dev cycle critbit replaces the classic hash. Critbit is
autoscaling and is more or less lockless so it became the default. Why we
haven't retired the class hashing is a relevant question. We probably
should.


>
> The system is currently fully using available physical memory and has
> burned through about 15% of the swap space.   The varnishstat was taken
> after the physical memory was exhausted and with just a few Gigs of swap in
> use.  Here is a current snapshot from top and varnishstat –1 if that helps.
>
>

Could it be that your memory allocator is leaking memory? The varnishstat
looks decent, there are no counters that are off the charts. Last time we
saw behaviour like that was when we were seeing seeing leaks with malloc on
Linux, before we switched to jemalloc.

I think Varnish will try to link with jemalloc if it is available on your
system. Is it? As you can see there are quite a few allocations happening
and if that memory is not reclaimed and reused you'll be leaking quite a
bit.


-- 
*Per Buer*
CTO | Varnish Software AS
Cell: +47 95839117
We Make Websites Fly!
www.varnish-software.com
[image: Register now]
<http://info.varnish-software.com/varnish-summits-autumn-2014-registration>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20140919/78ac5441/attachment.html>