Changes in Varnish 6.2

For information about updating your current Varnish deployment to the new version, see Upgrading to Varnish 6.2.

A more detailed and technical account of changes in Varnish, with links to issues that have been fixed and pull requests that have been merged, may be found in the change log.

varnishd

Cache lookups have undergone a number of optimizations, among them reduction in lock contention, and to shorten and simplify the critical section of lookup code. We expect that this will improve performance and scalability.

We have added a "watchdog" for thread pools that will panic the worker process, causing it to restart, if scheduling tasks onto worker threads appears to be deadlocking. The length of time until the panic is set by the thread_pool_watchdog parameter. If this happens, it probably means that thread pools are too small, and you should consider increasing the parameters thread_pool_min, thread_pool_max and/or thread_pools.

Parameters

The default value of thread_pool_stack has been increased from 48k to 56k on 64-bit platforms and to 52k on 32-bit platforms.

Recently we had occasional reports of stack overflow, apparently related to changes in external libraries that are not under control of the Varnish project (such as glibc). This may also have been related to stack overflow issues on some platforms when recent versions of jemalloc, the recommended memory allocator for Varnish, have been used together with pcre with JIT compilation enabled. Compiler hardening flags may also increase stack usage and on some systems such stack protector flags may be enabled by default. With the addition of new mitigations to new compiler releases, stack consumption may also increase on that front.

Tests have shown that Varnish runs stably with the new default stack size on a number of platforms, under conditions that previously may have led to stack overflow -- such as ESI includes up to the default limit of max_esi_depth, relatively deep VCL subroutine call depth, and recent jemalloc together with pcre-jit.

Different sites have different requirements regarding the stack size. For example, if your site uses a high depth of ESI includes, you are probably already using an increased value of thread_pool_stack. If you don't have such requirements, and you want to reduce memory footprint, you can consider lowering thread_pool_stack, but make sure to test the result.

Some parameters that have been long deprecated are now retired. See Runtime parameters in Upgrading to Varnish 6.2.

Added thread_pool_watchdog, see above.

The debug parameter now has a flag vcl_keep. When the flag is turned on, C sources and shared object libraries that were generated from VCL sources are retained in the Varnish working directory (see the notes about varnishtest below).

For 32-bit platforms, we have increased the default workspace_backend from 16k to 20k accommodate larger response headers which have become common.

Other changes in varnishd

The VCL syntax version is now displayed in a panic message, as 41 for VCL 4.1 and 40 for VCL 4.0.

Changes to VCL

VCL variables

Added req.is_hitmiss and req.is_hitpass, see VCL.

Other changes to VCL

Runtime restrictions concerning the accessibility of Unix domain sockets have been relaxed, see VCL in Upgrading to Varnish 6.2.

return(miss) from vcl_hit{} did never work as intended for the common case (it actually turned into a pass), so we now removed it and changed the builtin.vcl. See VCL.

VMODs

The type-conversion functions in VMOD std - Varnish Standard Module have been reworked to make them more flexible and easier to use. The std.x2y conversion functions are now deprecated. See Type conversion functions in VMOD std.

The function BACKEND lookup(STRING) has been added to VMOD directors - Varnish Directors Module, only for use in vcl_init or vcl_fini.

varnishlog(1), varnishncsa(1) and vsl(7)

The performance of bundled log readers, including varnishlog and varnishncsa (and any tool using the internal VUT interface for Varnish utilities) has been improved. They continue reading log contents in bulk as long as more contents are known to be available, not stopping as frequently (and unnecessarily) to check the status of the shared memory mapping.

varnishlog and varnishncsa now have the -R command-line option for rate-limiting, to limit the number of log transactions read per unit time. This can make it less likely for log reads to fall behind and fail with overrun errors under heavy loads. See varnishlog and varnishncsa for details.

Timing information is now uniformly reported in the log with microsecond precision. This affects the tags ExpKill and ExpRearm (previously with nanosecond precision).

varnishadm(1) and varnish-cli(7)

The output formats of the vcl.list and backend.list commands have changed, see Listing backends and Listing VCLs in Upgrading to Varnish 6.2, as well as varnish-cli for details.

JSON output

JSON responses, requested with the -j option, are now possible for the following commands (see varnish-cli):

  • status -j
  • vcl.list -j
  • param.show -j
  • ban.list -j
  • storage.list -j
  • panic.show -j

The -j option was already available for backend.list, ping and help in previous versions.

For automated parsing of CLI responses (varnishadm output), we recommend the use of JSON format.

param.reset <param>

Added the command param.reset to reset a parameter's value to its default, see varnish-cli.

Banning by expiration parameters

Bans may now be defined with respect to obj.ttl, obj.age, obj.grace and obj.keep, referring to the expiration and age properties of the cached object. A ban expression may also be defined with one of the comparison operators <, <=, > and >=; these may only be used with one of the new duration variables for bans. Duration constants (such as 5m for five minutes of 3h for three hours) may be used in the <arg> position against which these objects are compared in a ban expression.

obj.ttl and obj.age are evaluated with respect to the time at which the ban was defined, while obj.grace and obj.keep are evaluated as the grace or keep time assigned to the object. So to issue a ban for objects whose TTL expires more than 5 hours from now and whose keep parameter is greater than 3 hours, use this expression:

obj.ttl > 5h && obj.keep > 3h

See VCL and Purging and banning for details.

varnishstat(1) and varnish-counters(7)

Added the ws_*_overflow and client_resp_500 counters to better diagnose workspace overflow issues, see varnish-counters.

In curses mode, varnishstat now allows use of the + and - keys to increase or decrease the refresh rate of the curses window.

varnishtest

When varnishtest is invoked with either of the -L or -l options to retain the temporary directory after tests, the vcl_keep flag for the debug parameter is switched on (see Parameters above). This means that C sources and shared objects generated from VCL can also be inspected after a test. By default, the temporary directory is deleted after each test.

Since around the time of the last release, we have begun the project VTest, which is adapted from varnishtest, but is made available as a stand-alone program useful for testing various HTTP clients, servers and proxies (not just Varnish). But for the time being, we still use varnishtest for our own testing.

Changes for developers and VMOD authors

Python tools that generate code now require Python 3.

Directors

The director API has been changed slightly: The most relevant design change is that the healthy callback now is the only means to determine a director's health state dynamically, the sick member of struct director has been removed. Consequently, VRT_SetHealth() has been removed and VRT_SetChanged() added to update the last health state change time.

Presence of the healthy callback now also signifies if the director is considered to have a probe with respect to the CLI.

The signature of the list callback has been changed to reflect the retirement of the undocumented backend.list -v parameter and to add a VRT_CTX.

eof