Changes in Varnish 6.2¶
For information about updating your current Varnish deployment to the new version, see Upgrading to Varnish 6.2.
A more detailed and technical account of changes in Varnish, with links to issues that have been fixed and pull requests that have been merged, may be found in the change log.
varnishd¶
Cache lookups have undergone a number of optimizations, among them reduction in lock contention, and to shorten and simplify the critical section of lookup code. We expect that this will improve performance and scalability.
We have added a “watchdog” for thread pools that will panic the worker process, causing it to restart, if scheduling tasks onto worker threads appears to be deadlocking. The length of time until the panic is set by the thread_pool_watchdog parameter. If this happens, it probably means that thread pools are too small, and you should consider increasing the parameters thread_pool_min, thread_pool_max and/or thread_pools.
Parameters¶
The default value of thread_pool_stack has been increased from 48k to 56k on 64-bit platforms and to 52k on 32-bit platforms.
Recently we had occasional reports of stack overflow, apparently related to changes in external libraries that are not under control of the Varnish project (such as glibc). This may also have been related to stack overflow issues on some platforms when recent versions of jemalloc, the recommended memory allocator for Varnish, have been used together with pcre with JIT compilation enabled. Compiler hardening flags may also increase stack usage and on some systems such stack protector flags may be enabled by default. With the addition of new mitigations to new compiler releases, stack consumption may also increase on that front.
Tests have shown that Varnish runs stably with the new default stack size on a number of platforms, under conditions that previously may have led to stack overflow – such as ESI includes up to the default limit of max_esi_depth, relatively deep VCL subroutine call depth, and recent jemalloc together with pcre-jit.
Different sites have different requirements regarding the stack size. For example, if your site uses a high depth of ESI includes, you are probably already using an increased value of thread_pool_stack. If you don’t have such requirements, and you want to reduce memory footprint, you can consider lowering thread_pool_stack, but make sure to test the result.
Some parameters that have been long deprecated are now retired. See Runtime parameters in Upgrading to Varnish 6.2.
Added thread_pool_watchdog, see above.
The debug parameter now has a flag vcl_keep. When
the flag is turned on, C sources and shared object libraries that were
generated from VCL sources are retained in the Varnish working
directory (see the notes about varnishtest below).
For 32-bit platforms, we have increased the default workspace_backend from 16k to 20k accommodate larger response headers which have become common.
Other changes in varnishd¶
The VCL syntax version is now displayed in a panic message, as 41 for VCL 4.1 and 40 for VCL 4.0.
Changes to VCL¶
VCL variables¶
Added req.is_hitmiss and req.is_hitpass, see VCL.
Other changes to VCL¶
Runtime restrictions concerning the accessibility of Unix domain sockets have been relaxed, see VCL in Upgrading to Varnish 6.2.
return(miss) from vcl_hit{} did never work as intended for the
common case (it actually turned into a pass), so we now removed it and
changed the builtin.vcl. See
VCL.
VMODs¶
The type-conversion functions in VMOD std - Varnish Standard Module have been reworked
to make them more flexible and easier to use. The std.x2y
conversion functions are now deprecated. See
Type conversion functions in VMOD std.
The function BACKEND lookup(STRING) has been added to
VMOD directors - Varnish Directors Module, only for use in vcl_init or
vcl_fini.
varnishlog(1), varnishncsa(1) and vsl(7)¶
The performance of bundled log readers, including varnishlog and
varnishncsa (and any tool using the internal VUT interface for
Varnish utilities) has been improved. They continue reading log
contents in bulk as long as more contents are known to be available,
not stopping as frequently (and unnecessarily) to check the status of
the shared memory mapping.
varnishlog and varnishncsa now have the -R command-line
option for rate-limiting, to limit the number of log transactions read
per unit time. This can make it less likely for log reads to fall
behind and fail with overrun errors under heavy loads. See
varnishlog and varnishncsa for details.
Timing information is now uniformly reported in the log with
microsecond precision. This affects the tags ExpKill and
ExpRearm (previously with nanosecond precision).
varnishadm(1) and varnish-cli(7)¶
The output formats of the vcl.list and backend.list commands
have changed, see Listing backends and
Listing VCLs in
Upgrading to Varnish 6.2, as well as varnish-cli
for details.
JSON output¶
JSON responses, requested with the -j option, are now possible for
the following commands (see varnish-cli):
status -jvcl.list -jparam.show -jban.list -jstorage.list -jpanic.show -j
The -j option was already available for backend.list, ping
and help in previous versions.
For automated parsing of CLI responses (varnishadm output), we recommend the use of JSON format.
param.reset <param>¶
Added the command param.reset to reset a parameter’s value to its
default, see varnish-cli.
Banning by expiration parameters¶
Bans may now be defined with respect to obj.ttl, obj.age,
obj.grace and obj.keep, referring to the expiration and age
properties of the cached object. A ban expression may also be defined
with one of the comparison operators <, <=, > and >=;
these may only be used with one of the new duration variables for
bans. Duration constants (such as 5m for five minutes of 3h
for three hours) may be used in the <arg> position against which
these objects are compared in a ban expression.
obj.ttl and obj.age are evaluated with respect to the time at
which the ban was defined, while obj.grace and obj.keep are
evaluated as the grace or keep time assigned to the object. So to issue
a ban for objects whose TTL expires more than 5 hours from now and
whose keep parameter is greater than 3 hours, use this expression:
obj.ttl > 5h && obj.keep > 3h
See VCL and Purging and banning for details.
varnishstat(1) and varnish-counters(7)¶
Added the ws_*_overflow and client_resp_500 counters to better
diagnose workspace overflow issues, see varnish-counters.
In curses mode, varnishstat now allows use of the + and
- keys to increase or decrease the refresh rate of the curses
window.
varnishtest¶
When varnishtest is invoked with either of the -L or
-l options to retain the temporary directory after tests, the
vcl_keep flag for the debug parameter is switched
on (see Parameters above). This means that C sources and shared
objects generated from VCL can also be inspected after a test. By
default, the temporary directory is deleted after each test.
Since around the time of the last release, we have begun the project VTest, which is adapted from varnishtest, but is made available as a stand-alone program useful for testing various HTTP clients, servers and proxies (not just Varnish). But for the time being, we still use varnishtest for our own testing.