Upgrading to Varnish 5.2¶
Varnish statistics and logging¶
There are extensive changes under the hood with respect to statistics counters, but these should all be transparent at the user-level.
varnishd parameters¶
The vsm_space and cli_buffer
parameters are now deprecated and ignored. They will be removed
in a future major release.
The updated shared memory implementation manages space automatically, so it no longer needs vsm_space. Memory for the CLI command buffer is now dynamically allocated.
We have updated the documentation for send_timeout, idle_send_timeout, timeout_idle and ban_cutoff.
Added the debug bit vmod_so_keep
, see debug and
the notes about changes for developers below.
Changes to VCL¶
We have added a few new variables and clarified some matters. VCL written for Varnish 5.1 should run without changes on 5.2.
Consistent symbol names¶
VCL symbols originate from various parts of Varnish: there are built-in
variables, subroutines, functions, and the free-form headers. Symbols
may live in a namespace denoted by the '.'
(dot) character as in
req.http.Cache-Control
. When you create a VCL label, a new symbol
becomes available, named after the label. Storage backends always have
a name, even if you don’t specify one, and they can also be accessed in
VCL: for example storage.Transient
.
Because headers and VCL names could contain dashes, while subroutines or
VMOD objects couldn’t, this created an inconsistency. All symbols follow
the same rules now and must follow the same (case-insensitive) pattern:
[a-z][a-z0-9_-]*
.
You can now write code like:
sub my-sub {
new my-obj = my_vmod.my_constuctor(storage.my-store);
}
sub vcl_init {
call my-sub;
}
As you may notice in the example above, it is not possible yet to have dashes in a vmod symbol.
Long storage backend names used to be truncated due to a limitation in the VSC subsystem, this is no longer the case.
VCL variables¶
req.hash
and bereq.hash
¶
Added req.hash
and bereq.hash
, which contain the hash value
computed by Varnish for cache lookup in the current transaction, to
be used in client or backend context, respectively. Their data type
is BLOB, and they contain the raw binary hash.
You can use VMOD blob - Utilities for the VCL blob type, encoding and decoding to work with the hashes:
import blob;
sub vcl_backend_fetch {
# Send the transaction hash to the backend as a hex string
set bereq.http.Hash = blob.encode(HEX, blob=bereq.hash);
}
sub vcl_deliver {
# Send the hash in a response header as a base64 string
set resp.http.Hash = blob.encode(BASE64, blob=req.hash);
}
server.identity
¶
If the -i
option is not set in the invocation of varnishd
,
then server.identity
is set to the host name (as returned by
gethostname(3)
). Previously, server.identity
defaulted to the
value of the -n
option (or the default instance name if -n
was
not set). See varnishd.
bereq.is_bgfetch
¶
Added bereq.is_bgfetch
, which is readable in backend contexts, and
is true if the fetch takes place in the background. That is, it is
true if Varnish found a response in the cache whose TTL was expired,
but was still in grace time. Varnish returns the stale cached response
to the client, and initiates the background fetch to refresh the cache
object.
req.backend_hint
¶
We have clarified what happens to req.backend_hint
on a client
restart – it gets reset to the default backend. So you might want to
make sure that the backend hint gets set the way you want in that
situation.
vmod_std¶
New VMODs in the standard distribution¶
See VMOD blob - Utilities for the VCL blob type, encoding and decoding, VMOD purge - Varnish Purge Module and VMOD vtc - Utility module for varnishtest. Read about them in New VMODs in the standard distribution.
Bans¶
We have clarified the interpretation of a ban when a comparison in the ban expression is attempted against an unset field, see ban(STRING) in VCL.
Other changes¶
varnishd(1)
:The total size of the shared memory space for logs and counters no longer needs to be configured explicitly and therefore the second subargument to
-l
is now ignored.The default value of
server.identity
when the-i
option is not set has been changed as noted above.Also,
-i
no longer determines theident
field used bysyslog(3)
; now Varnish is always identified by the stringvarnishd
in the syslog.On a system that supports
setproctitle(3)
, the Varnish management process will appear in the output ofps(1)
asVarnish-Mgt
, and the child process asVarnish-Child
. If the-i
option has been set, then these strings in the ps output are followed by-i
and the identity string set by the option.The
-f
option for a VCL source file now honors thevcl_path
parameter if a relative file name is used, see varnishd and vcl_path.The
-a
option can now take a name, for example-a admin=127.0.0.1:88
to identify an address used for administrative requests but not regular client traffic. Otherwise, a default name is selected for the listen address (a0
,a1
and so forth). Endpoint names appear in the log output, as noted below, and may become accessible in VCL in the future.
varnishstat(1)
:In curses mode, the top two lines showing uptimes for the management and child processes show the text
Not Running
if one or both of the processes are down.The interpretation of multiple
-f
options in the command line has changed slightly, see varnishstat.The
type
andident
fields have been removed from the XML and JSON output formats, see varnishstat.The
MAIN.s_req
statistic has been removed, as it was identical toMAIN.client_req
.Added the counter
req_dropped
. Similar tosess_dropped
, this is the number of times an HTTP/2 stream was refused because the internal queue is full. See varnish-counters and thread_queue_limit.
varnishlog(1)
:The
Hit
,HitMiss
andHitPass
log records grew an additional field with the remaining TTL of the object at the time of the lookup. While this should greatly help troubleshooting, it might break tools relying on those records to get the VXID of the object hit during lookup.Instead of using
Hit
, such tools should now useHit[1]
, and the same applies toHitMiss
andHitPass
.The
Hit
record also grew two more fields for the grace and keep periods. This should again be useful for troubleshooting.See VSL.
The
SessOpen
log record displays the name of the listen address instead of the endpoint in its 3rd field.See VSL.
The output format of
VCL_trace
log records, which appear if you have switched on theVCL_trace
flag in the VSL mask, has changed to include the VCL configuration name. See VSL and vsl_mask.
varnishtest(1)
andvtc(7)
:When varnishtest is invoked with
-L
or-l
, Varnish instances started by a test do not clean up their copies of VMOD shared objects when they stop. See the note aboutvmod_so_keep
below.Added the feature switch
ignore_unknown_macro
for test cases, see VTC.
varnishncsa(1)
Field specifiers (such as the 1 in
Hit[1]
) are now limited to to 255, see varnishncsa.
The
-N
command-line option, which was previously available forvarnishlog(1)
,varnishstat(1)
,varnishncsa(1)
andvarnishhist(1)
, is not compatible with the changed internal logging API, and has been retired.Changes for developers:
The VSM and VSC APIs for shared memory and statistics have changed, and may necessitate changes in client applications, see VSM/VSC API changes.
Added the
$ABI
directive for VMOD vcc declarations, see $ABI [strict|vrt].There have been some minor changes in the VRT API, which may be used for VMODs and client apps, see VRT API changes.
The VUT API (for Varnish UTilities), which facilitates the development of client apps, is now publicly available, see Added VUT API.
The debug bit
vmod_so_keep
instructs Varnish not to clean up its copies of VMOD shared objects when it stops. This makes it possible for VMOD authors to load their code into a debugger after a varnishd crash. See debug.
eof