Transparent hugepages on RHEL8

[info+varnish at shee.org]

Am Montag, den 25.05.2020, 10:33 +0000 schrieb Dridi Boukelmoune:
> On Sun, May 24, 2020 at 8:12 AM Geoff Simmons <geoff at uplex.de> wrote:
> > On 5/24/20 01:29, info+varnish at shee.org wrote:
> > > This notes
> > > 
> > > https://varnish-cache.org/docs/trunk/installation/platformnotes.html
> > > 
> > > has a comment about "Transparent hugepages".
> > > 
> > > Does this still apply to EL8?
> > 
> > That's a good heads-up that those docs need to be updated -- they
> > refer
> > to RHEL6 and Linux kernel 3.2. If I'm not mistaken, enabling THP by
> > default was fairly new at the time, but it's still the default and
> > that's old news now, as your settings confirmed (just checked that
> > it's
> > also the default on my Debian stretch laptop).
> > 
> > The issue is not really the distro or kernel version, but the use
> > of the
> > THP feature, and it's still a problem, probably always will be.
> > AFAICT
> > THP does nothing good for Varnish. It's harmless if you're lucky,
> > but it
> > can be very disruptive.
> > 
> > I haven't tried it with RHEL8. The doc says that it "is known to
> > cause
> > sporadic crashes of Varnish", but while I haven't seen crashes, on
> > RHEL7
> > I've seen that the memory usage of the cache process bloats up
> > enormously, orders of magnitude larger than the actual size of the
> > cache
> > and anything else in Varnish that occupies memory. After disabling
> > THP
> > for Varnish (as detailed below), I saw memory usage become much
> > smaller,
> > more like what you'd expect from the cache size and other overhead.
> > 
> > There's an explanation for why THP causes that, but suffice it to
> > say
> > that THP creates trouble for a variety of apps that manage a lot of
> > memory. MongoDB, Oracle, redis and many other projects advise you
> > to
> > turn it off. THP is inevitably a problem for the jemalloc memory
> > allocator, which is invariably used with Varnish.
> 
> I just wanted to react to the "invariably" word here. This is not
> accurate, it should read "by default" instead.
> 
> See ./configure --help:
> 
> > --with-jemalloc use jemalloc memory allocator. Default is yes on
> > Linux, no elsewhere
> 
> And considering that jemalloc is not available in el8, but epel8
> instead. I suspect Red Hat ships a varnish package that uses glibc
> with no custom allocator.
> 

Seems so:

# strings /usr/sbin/varnishd | grep -i jemalloc
<empty>

and comparing with redis that is also in the distribution base repo:

# strings /usr/bin/redis-server|grep -i jemalloc|head -4
je_jemalloc_prefork
je_jemalloc_postfork_child
je_jemalloc_postfork_parent
jemalloc-5.1.0


are the implications that varnish build that way in this context
is more resilient with hugepages enabled?

--
Leon