[master] bf84027da jail_linux: Add THP control

Walid Boudebouda walid.boudebouda at varnish-software.com
Fri Feb 14 11:14:46 UTC 2025


I find this a bit confusing, is there an actual difference between setting
thp_setting to 'enable' vs 'ignore' ?

On Fri, Feb 14, 2025 at 11:27 AM Nils Goroll <nils.goroll at uplex.de> wrote:

>
> commit bf84027da5f54e5b85bae47b38752722ad21cb48
> Author: Thibaut Artis <thibaut.artis at varnish-software.com>
> Date:   Wed Sep 11 18:02:14 2024 +0200
>
>     jail_linux: Add THP control
>
>     Disabling Transparent Hugepage has often been the solution to solve
>     hard-to-diagnose instability issues and despite improvements in this
> area
>     compared to the RHEL6 era, our recommandation is still to avoid THP to
> this day.
>
>     In addition to refreshing the documentation on this topic, we add thp
> control to
>     the linux jail
>
>     Committer edit:
>
>     - Updated to master
>     - Added "try-disable" option as default
>     - Made "enable" and "disable" options fail
>     - Ensured default gets called for -jlinux
>     - Edited documentation
>     - polished
>
> diff --git a/bin/varnishd/mgt/mgt_jail_linux.c
> b/bin/varnishd/mgt/mgt_jail_linux.c
> index b8f136020..2dd7f89a2 100644
> --- a/bin/varnishd/mgt/mgt_jail_linux.c
> +++ b/bin/varnishd/mgt/mgt_jail_linux.c
> @@ -44,12 +44,84 @@
>  #include <sys/vfs.h>
>
>  #include "mgt/mgt.h"
> +#include "common/heritage.h"
> +
> +static int
> +vjl_set_thp(const char *arg, struct vsb *vsb)
> +{
> +       int r, val, must;
> +
> +       if (!strcmp(arg, "ignore"))
> +               return (0);
> +       must = 1;
> +       if (!strcmp(arg, "enable"))
> +               val = 0;
> +       else if (!strcmp(arg, "disable"))
> +               val = 1;
> +       else if (!strcmp(arg, "try-disable")) {
> +               arg = "disable";
> +               val = 1;
> +               must = 0;
> +       }
> +       else {
> +               VSB_printf(vsb, "linux jail: unknown value '%s' for
> argument"
> +                   " transparent_hugepage.", arg);
> +               return (1);
> +       }
> +       r = prctl(PR_SET_THP_DISABLE, val, 0, 0, 0);
> +       if (r) {
> +               VSB_printf(vsb, "linux jail: Could not %s "
> +                   "Transparent Hugepage: %s (%d)",
> +                   arg, VAS_errtxt(errno), errno);
> +       }
> +       return (r && must);
> +}
>
>  static int
>  vjl_init(char **args)
>  {
> +       struct vsb *vsb;
> +       char **unix_args;
> +       const char *val;
> +       int seen = 0, ret = 0;
> +       size_t i;
> +
> +       vsb = VSB_new_auto();
> +       AN(vsb);
> +
> +       if (args == NULL) {
> +               /* Autoconfig */
> +               AZ(vjl_set_thp("try-disable", vsb));
> +               MGT_ComplainVSB(C_INFO, vsb);
> +               VSB_destroy(&vsb);
> +               return (jail_tech_unix.init(NULL));
> +       }
> +
> +       for (i = 0; args[i] != NULL; i++);
> +       unix_args = calloc(i + 1, sizeof *unix_args);
> +       AN(unix_args);
> +
> +       for (i = 0; *args != NULL && ret == 0; args++) {
> +               val = keyval(*args, "transparent_hugepage=");
> +               if (val == NULL) {
> +                       unix_args[i++] = *args;
> +                       continue;
> +               }
> +
> +               ret |= vjl_set_thp(val, vsb);
> +               seen++;
> +       }
> +
> +       if (seen == 0)
> +               AZ(vjl_set_thp("try-disable", vsb));
> +
> +       MGT_ComplainVSB(ret ? C_ERR : C_INFO, vsb);
> +       VSB_destroy(&vsb);
>
> -       return jail_tech_unix.init(args);
> +       if (ret == 0)
> +               ret = jail_tech_unix.init(unix_args);
> +       free(unix_args);
> +       return (ret);
>  }
>
>  static void
> diff --git a/doc/sphinx/installation/platformnotes.rst
> b/doc/sphinx/installation/platformnotes.rst
> index 371105ffb..3c645befc 100644
> --- a/doc/sphinx/installation/platformnotes.rst
> +++ b/doc/sphinx/installation/platformnotes.rst
> @@ -32,6 +32,10 @@ column, no additional action is necessary.
>  Otherwise, consider creating a ``tmpfs`` mountpoint at *workdir*, or
> configure
>  *workdir* on an existing ``tmpfs``.
>
> +The ``tmpfs`` for *workdir* should be mounted with Transparent Hugepage
> +disabled. Consider mounting the working directory with the ``huge=never``
> mount
> +option if that is not the default.
> +
>  Note: Very valid reasons exist for *not* following this recommendation,
> if you
>  know what you are doing.
>
> @@ -55,21 +59,24 @@ See :ref:`ref-vsm` for details.
>
>  .. _platform-thp:
>
> -Transparent hugepages on Redhat Enterprise Linux 6
> -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +Transparent Hugepage on Linux
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +On certain Linux distributions Transparent Hugepage (THP) kernel support
> is
> +enabled by default. This is known to cause instabilities of Varnish.
>
> -On RHEL6 Transparent Hugepage kernel support is enabled by default.
> -This is known to cause sporadic crashes of Varnish.
> +By default, Varnish tries to disable the THP feature, but does not fail
> if it
> +can't. The ``linux`` :ref:`ref-varnishd-opt_j` offers to optionally
> enable,
> +disable or ignore THP.
>
> -It is recommended to disable transparent hugepages on affected
> -systems. This can be done with
> -``echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled``
> -(runtime) or by adding "transparent_hugepage=never" to the kernel boot
> -line in the "/etc/grub.conf" file (persistent).
> +Alternatively, THP can be disabled system-wide. If Varnish is the only
> +significant service running on this system, this can be done during
> runtime
> +with::
>
> -On Debian/Ubuntu systems running 3.2 kernels the default value is
> "madvise" and
> -does not need to be changed.
> +  echo never > /sys/kernel/mm/transparent_hugepage/enabled
>
> +The setting can be also be persisted in the bootloader configuration by
> adding
> +``transparent_hugepage=never`` to the kernel command line.
>
>  OpenVZ
>  ~~~~~~
> diff --git a/doc/sphinx/reference/varnishd.rst
> b/doc/sphinx/reference/varnishd.rst
> index a3c445d14..d93721cae 100644
> --- a/doc/sphinx/reference/varnishd.rst
> +++ b/doc/sphinx/reference/varnishd.rst
> @@ -454,13 +454,26 @@ specific options. Available jails are:
>
>      -j solaris,worker=basic
>
> --j <linux[,user=`user`][,ccgroup=`group`][,workuser=`user`]>
> +-j <linux[,transparent_hugepage=`thp_setting`][,`unix jail option`...]>
>
>    Default on Linux platforms, it extends the UNIX jail with
>    Linux-specific mechanisms:
>
>    - It warns when *workdir* is not on a ``tmpfs``.
>    - It tries to keep the process dumpable after dropping privileges.
> +  - It adds control over the transparent hugepage (THP) setting.
> +
> +  `thp_setting` can take these values:
> +
> +  - ``ignore``: Do nothing
> +  - ``enable``: Enable THP (see Note below)
> +  - ``disable``: Disable THP
> +  - ``try-disable`` (default): Try to disable, ignore failure (but emit a
> +    warning)
> +
> +  Note: Technically, ``enable`` is "disable the disable", so it does not
> +  necessarily enable THP. The setting names have been chosen to avoid a
> +  confusing double negation.
>
>  -j <unix[,user=`user`][,ccgroup=`group`][,workuser=`user`]>
>
> _______________________________________________
> varnish-commit mailing list
> varnish-commit at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-commit
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-commit/attachments/20250214/f9f464a8/attachment-0001.html>


More information about the varnish-commit mailing list