high memory usage with malloc and file backends configured

[Dridi Boukelmoune]

On Mon, Jul 10, 2017 at 1:55 PM, Guillaume Quintard
<guillaume at varnish-software.com> wrote:
> Hi Charles,
>
> So, if I'm reading this right, there's no discrepancy. Varnish will malloc
> the full storage (malloc), and will mmap a file the size of the full storage
> (file). So even though the storage is not used, it's allocated.
>
> Does it make sense, or did I miss something?

Technically malloc storage makes individual allocations, so only file
storage will pre-allocate the full storage.

Allocations also come with an overhead (house-keeping, alignment
requirements...) not reported by Varnish since we can't technically
tell what's happening under our feet (for instance, jemalloc). Also
storage size is only for... storage. So while we have a rule of thumb
of 1kB of overhead per object, that's not the sole non-storage memory
footprint.

Dridi