varnish eats all RAM

[Geoff Simmons]

On 4/29/21 17:21, Marco Dickert - evolver group wrote:
> 
> we have a weird problem with one of our varnish instances running on debian
> buster. The problem is, that varnish consumes all RAM over time and does not
> free it again.

Just a hunch, but this is reminiscent of problems that I've seen caused
by transparent huge pages, which are enabled by default in recent Linux
versions.

Why THP can be a problem with Varnish & jemalloc takes some explaining,
but maybe first check if that could be your problem.

On Linux you can look for the value of AnonHugePages in /proc/meminfo,
value in KB. You can also look for rss_huge in
/sys/fs/cgroup/memory/memory.stat, where the value is the number of huge
pages. So you'd have to multiply the latter value by the size of a
hugepage, which appears as Hugepagesize in /proc/meminfo (usually it's
2MB on Linux).

If those values are very large and seem to account for most of your high
RAM consumption, considerably higher than you would have expected for
Varnish, then THP may be your problem.

If you have jemalloc since version 4.5, you can turn off THP
specifically for Varnish with the configuration thp:never for jemalloc,
either in /etc/jemalloc.conf, or by passing it in the environment
variable MALLOC_CONF to the varnishd invocation -- set
MALLOC_CONF=thp:never and make sure Varnish inherits that in its
environment.

thp:never is not supported by jemalloc 3.6.0, which is still very widely
deployed on many systems (unfortunately, since jemalloc is up to version
5.2.1 now).

You can also turn off THP system-wide (for all processes):

$ echo never > /sys/kernel/mm/transparent_hugepage/enabled

That might be your only option if the jemalloc version is too old.

Either way, then try it and see if your RAM issues improve.


HTH,
Geoff
-- 
** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

Tel +49 40 2880 5731
Mob +49 176 636 90917
Fax +49 40 42949753

http://uplex.de

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20210429/55dea7cd/attachment.bin>