.. Copyright (c) 2018-2019 Varnish Software AS SPDX-License-Identifier: BSD-2-Clause See LICENSE file for full text of license .. _whatsnew_upgrading_6.0: %%%%%%%%%%%%%%%%%%%%%%%% Upgrading to Varnish 6.0 %%%%%%%%%%%%%%%%%%%%%%%% .. _upd_6_0_uds_acceptor: Unix domain sockets as listen addresses ======================================= The ``varnishd -a`` command-line argument now has this form, where the ``address`` may be a Unix domain socket, identified as such when it begins with ``/`` (see varnishd :ref:`ref-varnishd-options`):: -a [name=][address][:port][,PROTO][,user=][,group=][,mode=] For example:: varnishd -a /path/to/listen.sock,PROXY,user=vcache,group=varnish,mode=660 That means that an absolute path must always be specified for the socket file. The socket file is created when Varnish starts, and any file that may exist at that path is unlinked first. You can use the optional ``user``, ``group`` and ``mode`` sub-arguments to set permissions of the new socket file; use names for ``user`` and ``group`` (not numeric IDs), and a 3-digit octal number for ``mode``. This is done by the management process, so creating the socket file and setting permissions are done with the privileges of the management process owner. There are some platform-specific restrictions on the use of UDSen to which you will have to conform. Here are some things we know of, but this list is by no means authoritative or exhaustive; always consult your platform documentation (usually in ``man unix``): * There is a maximum permitted length of the path for a socket file, considerably shorter than the maximum for the file system; usually a bit over 100 bytes. * On FreeBSD and other BSD-derived systems, the permissions of the socket file do not restrict which processes can connect to the socket. * On Linux, a process connecting to the socket must have write permissions on the socket file. On any system, a process connecting to the socket must be able to access the socket file. So you can reliably restrict access by restricting permissions on the directory containing the socket (but that must be done outside of the Varnish configuration). When UDS listeners are in use, VCL >= 4.1 will be required for all VCL programs loaded by Varnish. If you attempt to load a VCL source with ``vcl 4.0;``, the load will fail with a message that the version is not supported. If you continue using only IP addresses in your ``-a`` arguments, you won't have to change them, and you can continue using VCL 4.0. .. _upd_6_0_uds_backend: Unix domain sockets as backend addresses ======================================== A backend declaration may now have the ``.path`` field to specify a Unix domain socket to which Varnish connects:: backend my_uds_backend { .path = "/path/to/backend.sock"; } One of the fields ``.host`` or ``.path`` must be specified for a backend (but not both). The value of ``.path`` must be an absolute path (beginning with ``/``), and the file at that path must exist and be accessible to Varnish at VCL load time; and it must be a socket. The platform-specific restrictions on UDSen mentioned above apply of course to backends as well; but in this case your deployment of the peer component listening at the socket file must fulfill those conditions, otherwise Varnish may not be able to connect to the backend. The path of a socket file may also be specified in the ``varnishd -b`` command-line option (see varnishd :ref:`ref-varnishd-options`):: $ varnishd -b /path/to/backend.sock The value of ``-b`` must fulfill the same conditions as the ``.path`` field in a backend declaration. Backends with the ``.path`` specification require VCL 4.1, as do paths with the ``-b`` argument. If you don't use UDS backends, you can continue using VCL 4.0. varnishd parameters =================== The ``cli_buffer`` parameter, which was deprecated as of Varnish 5.2, is now retired. :ref:`ref_param_max_restarts` now works more correctly -- it is the number of ``return(restart)`` calls permitted per request. (It had been one less than the number of permitted restarts.) The parameters :ref:`ref_param_tcp_keepalive_intvl`, :ref:`ref_param_tcp_keepalive_probes` and :ref:`ref_param_tcp_keepalive_time` are silently ignored for listen addresses that are Unix domain sockets. The parameters :ref:`ref_param_accept_filter` and :ref:`ref_param_tcp_fastopen` (which your platform may or may not support in the first place) almost certainly have no effect on a UDS. It is not an error to use any of these parameters with a UDS; you may get error messages in the log for ``accept_filter`` or ``tcp_fastopen`` (with the VSL tag ``Error`` in raw grouping), but they are harmless. :ref:`ref_param_workspace_thread` is now used for IO buffers during the delivery of the client response. This space had previously been taken from :ref:`ref_param_workspace_client`. If you need to reduce memory footprint, consider reducing ``workspace_client`` by the amount in ``workspace_thread``. Added `ref_param_esi_iovs`. tl;dr: Don't touch it, unless advised to do so by someone familiar with the innards of Varnish. Changes to VCL ============== VCL 4.0 and 4.1 ~~~~~~~~~~~~~~~ The first line of code in a VCL program may now be either ``vcl 4.0;`` or ``vcl 4.1;``, establishing the version of the language for that instance of VCL. Varnish 6.0 supports both versions. The VCL version mainly affects which variables may be used in your VCL program, or in some cases, whether the variable is writable or read-only. Only VCL 4.1 is permitted when Unix domain sockets are in use. For details, see :ref:`vcl_variables`, and the notes in the present document. VCL variables ~~~~~~~~~~~~~ ``local.socket`` and ``local.endpoint`` --------------------------------------- These read-only variables are available as of VCL 4.1, and provide information about the listener address over which the current client request was received. ``local.socket`` is the name provided in the ``-a`` command-line argument for the current listener, which defaults to ``a0``, ``a1`` and so on (see varnishd :ref:`ref-varnishd-options`). ``local.endpoint`` is the value of the ``address[:port]`` or ``path`` field provided as the ``-a`` value for the current listener, exactly as given on the command line. For example:: # When varnishd is invoked with these -a arguments ... $ varnishd -a foo=12.34.56.78:4711 -a bar=/path/to/listen.sock # ... then in VCL, for requests received over the first listener: local.socket == "foo" local.endpoint == "12.34.56.78:4711" # ... and for requests received over the second listener: local.socket == "bar" local.endpoint == "/path/to/listen.sock" # With this invocation ... $ varnishd -a :80 -a 87.65.43.21 # ... then for requests received over the first listener: local.socket == "a0" local.endpoint == ":80" # ... and for the second listener local.socket == "a1" local.endpoint == "87.65.43.21" So if you have more than one listener and need to tell them apart in VCL, for example a listener for "regular" client traffic and another one for "admin" requests that you must restrict to internal systems, these two variables can help you do so. ``local.socket`` and ``local.endpoint`` are available on both the client and backend sides. But the values on the backend side are not necessarily the same as they were on the side of the client request that initiated the backend request. This is because of the separation of client and backend threads -- a backend thread may be re-used that was initiated by a client request over another listener, and ``local.socket`` and ``local.endpoint`` on that thread retain the values for the original listener. So if, in your backend VCL code, you need to be sure about the listener that was used on the client side of the same transaction, assign ``local.socket`` and/or ``local.endpoint`` to a client request header, and retrieve the value from a backend request header:: sub vcl_miss { set req.http.X-Listener = local.socket; } sub vcl_backend_fetch { if (bereq.http.X-Listener == "a0") { # ... } } ``sess.xid`` ------------ This is the unique ID assigned by Varnish to the current session, which stands for the "conversation" with a single client connection that comprises one or more request/response transactions. It is the same XID shown in the log for session transactions (with ``-g session`` grouping). ``sess.xid`` is read-only and is available as of VCL 4.1. Variable changes in VCL 4.0 and 4.1 ----------------------------------- The ``*.proto`` variables (``req.proto``, ``resp.proto``, ``bereq.proto`` and ``beresp.proto``) are read-only as of VCL 4.1, but are still writable in VCL 4.0. ``req.esi`` is available in VCL 4.0, but no longer in 4.1. In its place, ``resp.do_esi`` has been introduced in VCL 4.1. Set ``resp.do_esi`` to false in ``vcl_deliver`` if you want to selectively disable ESI processing for a client response (even though ``beresp.do_esi`` was true during fetch). ``beresp.backend.ip`` and ``beresp.storage_hint`` are discontinued as of VCL 4.1, but are still available in 4.0. Note that ``beresp.storage_hint`` has been deprecated since Varnish 5.1; you should use ``beresp.storage`` instead. Client-side variable access --------------------------- ``req.storage``, ``req.hash_ignore_busy`` and ``req.hash_always_miss`` are now accessible from all of the client side subroutines (previously only in ``vcl_recv{}``). Unix domain sockets and VCL ~~~~~~~~~~~~~~~~~~~~~~~~~~~ We have made an effort to adapt the support of Unix domain sockets in VCL so that you may not have to change anything in your VCL deployment at all, other than changing the version to 4.1. The short story is that where VCL requires an IP value, the value is ``0.0.0.0:0`` for a connection that was addressed as a UDS -- the "any IPv4" address with port 0. So your use of IP-valued elements in VCL will continue to work and may not have to change, but there are some consequences that you should consider, covered in the following. As we shall see, for a variety of reasons you get the best results if the component forwarding to Varnish via UDS uses the PROXY protocol, which sets ``client.ip`` and ``server.ip`` to the addresses sent in the PROXY header. If you don't use UDSen, then nothing about VCL changes with respect to network addressing. UDS support requires version 4.1, so if you are keeping your VCL level at 4.0 (and hence are staying with IP addresses), then none of the following is of concern. ``client.ip``, ``server.ip``, ``local.ip`` and ``remote.ip`` ------------------------------------------------------------ These variables have the value ``0.0.0.0`` for a connection that was addressed as a UDS. If you are using the PROXY protocol, then ``client.ip`` and ``server.ip`` have the "real" IP address values sent via PROXY, but ``local.ip`` and ``remote.ip`` are always ``0.0.0.0`` for a UDS listener. If you have more than one UDS listener (more than one ``-a`` command-line argument specifying a socket path), then you may not be able to use the ``*.ip`` variables to tell them apart, especially since ``local.ip`` will be ``0.0.0.0`` for all of them. If you need to distinguish such addresses in VCL, you can use ``local.socket``, which is the name given for the ``-a`` argument (``a0``, ``a1`` etc. by default), or ``local.endpoint``, which in the case of UDS is the path given in the ``-a`` argument. You can, for example, use string operations such as regex matching on ``local.endpoint`` to determine properties of the path address:: # admin requests allowed only on the listener whose path ends in # "admin.sock" if (req.url ~ "^/admin") { if (local.endpoint !~ "admin.sock$") { # wrong listener, respond with "403 Forbidden" return( synth(403) ); } else { # process the admin request ... } } # superadmin requests only allowed on the "superadmin.sock" listener if (req.url ~ "^/superadmin") { if (local.endpoint !~ "superadmin.sock$") { return( synth(403) ); } else { # superadmin request ... } } ACLs ---- As before, ACLs can only specify ranges of IP addresses, and matches against ACLs can only be run against IP-valued elements. This means that if a ``*.ip`` variable whose value is ``0.0.0.0`` due to the use of UDS is matched against an ACL, the match can only succeed if the ACL includes ``0.0.0.0``. If you currently have a security requirement that depends on IP addresses *not* matching an ACL unless they belong to a specified range, then that will continue to work with a UDS listener (since you almost certainly have not included ``0.0.0.0`` in that range). Recall again that ``client.ip`` and ``server.ip`` are set by the PROXY protocol. So if you have a UDS listener configured to use PROXY and are using an ACL to match against one of those two variables, the matches will continue working against the "real" IPs sent via PROXY. You can of course define an ACL to match in the UDS case, by including ``0.0.0.0``:: # matches local.ip and remote.ip when the listener is UDS acl uds { "0.0.0.0"; } But such an ACL cannot distinguish different UDS listeners, if you have more than one. For that, you can achieve a similar effect by inspecting ``local.socket`` and/or ``local.endpoint``, as discussed above. ``client.identity`` and the hash and shard directors ---------------------------------------------------- As before, ``client.identity`` defaults to ``client.ip``; that is, if its value has not been explicitly set in VCL, then it returns the same value as ``client.ip`` when it is read. A common use of ``client.identity`` is to configure the hash and shard directors (see :ref:`vmod_directors(3)`). This is a way to achieve "client-sticky" distribution of requests to backends -- requests from the same clients are always sent to the same backends. Such a configuration will almost certainly not do what you want if: * The listener is set to a UDS address. * PROXY is not used to set ``client.ip``. * ``client.identity`` is not set to a distinct value before it is used to configure the director. Since ``client.identity`` defaults to ``client.ip``, which is always ``0.0.0.0`` under these conditions, the result will be that the director sends all requests to just one backend, and no requests to any other backend. To avoid that result, change one of the conditions listed above -- use PROXY to set distinct values for ``client.ip``, or set ``client.identity`` to distinct values before it is used. ``server.ip`` and default hashing for the cache ----------------------------------------------- The default algorithm for computing a hash value for the cache (the implementation of ``vcl_hash`` in ``builtin.vcl``) mixes ``req.url`` and the Host header (``req.http.Host``) into the hash data. If there is no Host header, then ``server.ip`` is used instead. Considering the Host header or ``server.ip`` is a way of achieving a kind of "virtual hosting" -- if your site receives requests with different Host headers or at distinct server addresses, then requests for the same URL will not hit the same cached response, if the requests are different in those other respects. If you have UDS listeners and are not using PROXY to set distinct values of ``server.ip``, then requests without a Host header will have the same value of ``server.ip == 0.0.0.0`` mixed into the hash. In that case, requests with the same URL will result in the same hash value, and hit the same cached responses. That doesn't matter, of course, if you don't need the "virtual hosting" effect -- you only have one listener, you never receive different host headers, or you never receive the same URL for what should lead to distinct responses. But if you need to avoid that result, then you can make one or more of these changes: * Use the PROXY protocol to set distinct ``server.ip`` values. * Write your own implementation of ``vcl_hash``, for example to mix ``local.socket`` or ``local.endpoint`` into the hash. * Set ``req.http.Host`` to a distinct value if it is absent before ``vcl_hash`` is entered. X-Forwarded-For --------------- Varnish automatically appends the value of ``client.ip`` to the ``X-Forwarded-For`` request header that is passed on to backends, or it creates the header with that value if it is not already present in the client request. If the client request is received over a UDS listener and the PROXY protocol is not used, then ``0.0.0.0`` will be added to ``X-Forwarded-For``. If you prefer, you can change that in VCL:: sub vcl_backend_fetch { # Assuming that server.identity has been set to an IP # address with the -i command-line argument. set bereq.http.X-Forwarded-For = regsub(bereq.http-X-Forwarded-For, "0.0.0.0$", server.identity); # ... } Again, this is probably not a concern if ``client.ip`` is set via the PROXY protocol. UDS backends and the Host header -------------------------------- By default, Varnish forwards the Host header from a client request to the backend. If there is no Host header in the client request, and the ``.host_header`` field was set in the backend declaration, then that value is used for the backend Host header. For backends declared with the ``.host`` field (with a domain name or IP address), then if there is neither a client Host header nor a ``.host_header`` declaration, the value of ``.host`` is set as the Host header of the backend request. If the backend was declared with ``.path`` for a socket path, then the backend Host header is set to ``0.0.0.0`` under those conditions. To re-state that: * If the backend was declared with ``.path`` to connect to a Unix domain socket, ... * and ``.host_header`` was not set in the backend declaration, ... * and there is no Host header in the client request, ... * then the Host header in the backend request is set to ``0.0.0.0``. If you want to avoid that, set a ``.host_header`` value for the backend, or set a value for the Host header in VCL. VMOD std -------- :ref:`std.port()` always returns 0 when applied to a ``*.ip`` variable whose value is set to ``0.0.0.0`` because the listener is UDS. :ref:`std.set_ip_tos()` is silently ignored when the listener is UDS. The ``shard`` director ---------------------- The ``alg`` argument of the shard director's ``.reconfigure()`` and ``.key()`` methods has been removed. The choice of hash algorithms was experimental, and we have settled on SHA256 as providing the best dispersal. If you have been using other choices of ``alg`` for ``.reconfigure()``, then after upgrading and removing ``alg``, the sharding of requests to backends will change once and only once. If you have been using other values of ``alg`` for ``.key()`` and need to preserve the previous behavior, see the `change log `_ for advice on how to do so. With the ``resolve=LAZY`` argument of the ``.backend()`` method, the shard director will now defer the selection of a backend to when a backend connection is actually made, which is how all other bundled directors work as well. In ``vcl_init``, ``resolve=LAZY`` is default and enables layering the shard director below other directors -- you can now use something like ``mydirector.add_backend(myshard.backend())`` to set the shard director as a backend for another director. Use of ``resolve=LAZY`` on the client side is limited to using the default or associated parameters. The shard director now provides a ``shard_param`` object that serves as a store for a set of parameters for the director's ``.backend()`` method. This makes it possible to re-use a set of parameter values without having to restate them in every ``.backend()`` call. The ``.backend()`` method has an argument ``param`` whose value, if it is used, must be returned from the ``shard_param.use()`` method. Because of these changes, support for positional arguments of the shard director ``.backend()`` method had to be removed. In other words, all parameters to the shard director ``.backend()`` method now need to be named. See :ref:`vmod_directors(3)` for details. Restarts ~~~~~~~~ Restarts now leave all of the properties of the client request unchanged (all of the ``req.*`` variables, including the headers), except for ``req.restarts`` and ``req.xid``, which change by design. If you need to reset the client request headers to their original state (before changes in VCL), call :ref:`std.rollback()`. ``return(restart)`` can now be called from ``vcl_recv{}``. New VMODs ~~~~~~~~~ VMOD unix --------- :ref:`vmod_unix(3)` provides functions to determine the credentials of the peer process (user and group of the process owner) that connected to Varnish over a listener at a Unix domain socket. You can use this, for example, to impose tighter restrictions on who can access certain resources:: import unix; sub vcl_recv { # Return "403 Forbidden" if the connected peer is # not running as the user "trusteduser". if (unix.user() != "trusteduser") { return( synth(403) ); } This is not available on every platform. As always, check the documentation and test the code before you attempt something like this in production. VMOD proxy ---------- :ref:`vmod_proxy(3)` provides functions to extract TLV attributes that may be optionally sent over a PROXYv2 connection to a Varnish listener. Most of these are properties of the peer component's TLS connection:: import proxy; # Get the authority attribute -- corresponds to the SNI of a TLS # connection. set req.http.Authority = proxy.authority(); Not all implementations send TLV attributes, and those that do don't necessarily support all of them; test your code to see what works in your configuration. See the `PROXY v2 specification `_ for more information about TLV attributes. Packaging changes ================= Supported platforms ~~~~~~~~~~~~~~~~~~~ Official Varnish packages went through major changes for this release, and target Debian 9, Ubuntu 16.04 LTS and (Red Hat) Enterprise Linux 7. Ubuntu 14.04 LTS will likely reach its end of life before Varnish 6 and the venerable Enterprise Linux 6 is getting too old and forced time-consuming workarounds so for these reasons we dropped community support for those platforms. Services ~~~~~~~~ As a result we ended up with systemd-only platforms for the official packages. The old services are still available as we archived them in the ``pkg-varnish-cache`` source tree. This was the occasion to remove differences between Red Hat and Debian derivatives since there's no more reasons to have them diverge: we initially inherited packaging support from downstream package maintainers, and they deserve many thanks for that. The biggest change that resulted in unifying our systemd setup across all official packages is that the ``varnish.params`` file available on Red Hat derivatives is gone. We noticed that using an environment file to hide the fact that ``varnishd`` is configured via command line arguments misled some people into thinking that what was proposed in the file was your only set of configuration options. For example, you can specify multiple listen addresses using multiple ``-a`` options but you only get variables for one address and a catch-all variable ``DAEMON_OPTS`` for anything not fitting in the template. In addition using an environment file pollutes the process's environment. Another big difference between Red Hat and Debian derivatives was the way we handled VCL reloads via the service manager. We introduced a new ``varnishreload`` script that operates on top of ``varnishadm`` to perform hot reloads of one VCL configuration or label at a time. All you need is enough privileges to contact ``varnishd``'s command line interface, which should not be a problem for package managers. Once the ``varnish`` package is installed, you can learn more:: varnishreload -h Again, many thanks to downstream maintainers and some early adopters for their help in testing the new script. To stay on the topic of the command line interface, packages no longer create a secret file for the CLI, and services omit ``-S`` and ``-T`` options on the ``varnishd`` command. This means that out of the box, you can only connect to the CLI locally with enough privileges to read a secret generated randomly. This means less noise in our packages, and you need to change the service configuration to enable remote access to the CLI. With previous packages, you also needed to change your configuration because the CLI would only listen to the loopback interface anyway. To change how ``varnishd`` is started, please refer to the systemd documentation. Virtual provides ~~~~~~~~~~~~~~~~ Last but not least in the packaging space, we took a first step towards improving dependency management between official ``varnish`` packages and VMODs built on top of them. RPMs and Deb packages now provide the strict and VRT ABIs from ``varnishd`` and the goal is to ultimately prevent a package installation or upgrade that would prevent a VMOD from being loaded. For Deb packages:: Provides: varnishd-abi-SHA1, varnishd-vrt (= x.y) And for RPMs:: Provides: varnishd(abi)(x86-64) = SHA1 Provides: varnishd(vrt)(x86-64) = x.y For VMOD authors or downstream distributors, this means that depending on the ``$ABI`` stanza in the VMOD descriptor, they can either tie their backend manually to the git hash Varnish was built from or to the VRT version used at the time. For example, a VMOD RPM built against Varnish 6.0.0 could have:: Requires: varnishd(vrt)%{?_isa} >= 7.0 Requires: varnishd(vrt)%{?_isa} < 8 Future plans include the ability to automate this for out-of-tree VMODs and remove manual steps. To learn more about the history behind this change, it was formalized via the Varnish Improvement Process: https://github.com/varnishcache/varnish-cache/wiki/VIP20%3A-Varnish-ABI-and-packaging Another thing available only to RPM packages as of 6.0.0 is virtual provides for VMODs. Instead of showing shared objects that aren't even in the dynamic linker's default path:: Provides: libvmod_std.so(64bit) Provides: libvmod_directors.so(64bit) [...] You get virtual VMOD provides with a version:: Provides: vmod(std)(x86-64) = 6.0.0-1 Provides: vmod(directors)(x86-64) = 6.0.0-1 [...] With the same mechanism it becomes possible to require a VMOD directly and let it bring along its dependencies, like ``varnish``. As this is currently not automated for out-of-tree VMODs, consider this a preview of what you will be able to do once VIP 20 is completed. Other changes ============= * ``varnishd(1)``: * The ``umem`` storage allocator, which was removed as of Varnish 5.1, has been restored and is now the default on a system where ``libumem`` is available (SunOS and descendants). * ``varnishlog(1)``: * Added a third field to the ``ReqStart`` log record that contains the name of the listener address over which the request was received, see :ref:`vsl(7)`. * ``0.0.0.0`` and port ``0`` appear in the logs where an IP and port otherwise appear, when the connection in question was addressed as a Unix domain socket. This affects ``ReqStart``, ``SessOpen``, ``BackendStart`` and ``BackendOpen``. If you have more than one UDS listener, they can be distinguished with the "listener name" field -- the third field for both ``ReqStart`` and ``SessOpen``. If you have more than one UDS backend, they can be distinguished with the backend name field -- the second field in ``BackendOpen``. * The byte counters logged with ``ReqAcct`` now report the numbers returned from the operating system telling us how many bytes were actually sent in a request and response, rather than what Varnish thought it was going to send. This gives a more accurate account when there are errors, for example when a client hung up early without receiving the entire response. The figures also include any overhead in a request or response body, for example due to chunked encoding. * Debugging logs for the PROXY protocol are turned off by default. They can be turned on with the ``protocol`` flag of the varnishd :ref:`ref_param_debug` parameter (``-p debug=+protocol``). * ``varnishstat(1)`` * Added the counter ``cache_hit_grace`` -- how often objects in the cache were hit when their TTL had expired, but they were still in grace. * ``varnishncsa(1)`` * The ``%h`` formatter (remote host) gets its value from ``ReqStart`` for client requests and ``BackendStart`` for backend requests. The value will be ``0.0.0.0`` for client requests when the listener is UDS, and for backend requests when the backend is UDS. * The ``%r`` formatter (first line of the request) is reconstructed in part from the Host request header. For UDS backends, Host may be ``0.0.0.0`` for the reasons explained above (no client Host header and no ``.host_header`` setting for the backend), so that may appear in the output for ``%r``. You can avoid that with the measures discussed above. * If you have more than one UDS listener and/or more than one UDS backend, and you want to tell them apart in the ``varnishncsa`` output (rather than just see ``0.0.0.0``), use the ``%{VSL}x`` formatter to capture the listener name and the backend name. For the listener name, use ``%{VSL:ReqStart[3]}x`` for client logs (the third field of ``ReqStart`` logs). For the backend name, use ``%{VSL:BackendOpen[2]}x`` for backend logs. * varnishncsa does not accept output format strings (from the ``-F`` command-line argument or a configuration file) if they specify tags for log entries whose payloads may contain control or binary characters. * ``varnishtest(1)`` and ``vtc(7)``: * The ``client -connect`` and ``server -listen`` commands in vtc scripts now allow Unix domain sockets as addresses, recognized when the argument begins with a ``/``. A client attempts the connection immediately, so the socket file must exist at the given path when the client is started, and the client must be able to access it. The ``server -listen`` command must be able to create the socket file when it executes ``bind(2)``. To make it easier for other processes to connect to the socket, the server's umask is temporarily set to 0 before the listen is attempted, to minimize issues with permissions. No further attempt is made to set the socket's permissions. To test a Varnish instance listening at a UDS, just use the ``varnish -arg`` command with the appropriate settings for the ``-a`` command line argument, see :ref:`varnishd(1)`. The ``varnish -vcl+backend`` command now works to include backend definitions for server objects that are listening at UDS. Backend declarations are implicitly included for such servers with the appropriate ``.path`` setting. A convenient location for socket files to be used in a test is the temporary directory created by ``varnishtest`` for each test, whose path is held in the macro ``${tmpdir}``. So this is a common idiom for tests that involve UDSen:: server s1 -listen "${tmpdir}/s1.sock" { ... } -start varnish v1 -arg "-a ${tmpdir}/v1.sock" -vcl+backend { ... } -start client c1 -connect "${tmpdir}/v1.sock" { ... } -run When a Varnish instance in a vtc test is listening at a UDS, then its ``vN_*`` macros are set like this: * ``v1_addr``: ``/path/to/socket`` * ``v1_port``: ``-`` (hyphen) * ``v1_sock``: ``/path/to/socket -`` When a server ``s1`` is listening at a UDS: * ``s1_addr``: ``0.0.0.0`` * ``s1_port``: ``0`` * ``s1_sock``: ``/path/to/socket`` The vtc variables ``remote.ip`` and ``remote.port``, which can be used in ``expect`` expressions for both server and client scripts, are set to ``0.0.0.0`` and ``0``, respectively, when the peer address is a UDS. We have added the variable ``remote.path`` as a counterpart to the other two. Its value is the path when the peer address is a UDS, and NULL otherwise (matching ```` in the latter case). * Changes for developers: * The VRT API version has been bumped to 7.0, and comprises a variety of new additions and changes. See ``vrt.h`` and the `change log `_ for details. * There are new rules about including API headers -- some may only be included once, others must included in a specific order. Only ``cache.h`` *or* ``vrt.h`` may be included (``cache.h`` includes ``vrt.h``). See the ``#error`` directives in the headers. * VMOD authors can use the ``VRT_VSC_*()`` series of functions and the new ``vsctool`` to create statistics for a VMOD that will be displayed by varnishstat. Varnish uses the same technique to create its counters, so you can look to the core code to see how it's done. * The ``VCL_INT`` and ``VCL_BYTES`` types are now defined to be strictly 64 bit (rather than leave it to whatever your platform defines as ``long``). But you may not get that full precision, for reasons discussed in the `change log `_. * As part of VRT version 7.0, the ``path`` field has been added to to ``struct vrt_backend``, which a VMOD can use with ``VRT_new_backend()`` to create a dynamic backend with a UDS address (see ``vrt.h``). If ``path`` is non-NULL, then both of the IPv4 and IPv6 addresses must be NULL. If ``path`` is NULL, then (as before) one or both of the IP addresses must be non-NULL. The ``dyn_uds`` object in VMOD debug (available in the source tree) illustrates how this can be done. * VMOD vcc sources may now include a directive ``$Prefix``, whose value is the string prepended to the names of C objects and functions in the generated C interface (in ``vcc_if.h``). So you may choose another prefix besides ``vmod_``, if so desired. * vcc sources may also include a directive ``$Synopsis`` whose value may be ``auto`` or ``manual``, default ``auto``. When ``$Synopsis`` is ``auto``, the vmodtool generates a more comprehensive ``SYNOPSIS`` section in the documentation than in previous versions -- an overview of the objects, methods and functions in your VMOD, with their type signatures. When ``$Synopsis`` is ``manual``, the ``SYNOPSIS`` is left out of the generated docs altogether; so you can write the ``SYNOPSIS`` section yourself, if you prefer. * Support for a new declaration of optional arguments in vcc files has been added: ``[ argname ]`` can be used to mark *argname* as optional. If this declaration is used for any argument, _all_ user arguments and ``PRIV_*`` pointers (no object pointers) to the respective function/method will be passed in a ``struct`` *funcname*\ ``_arg`` specific to this function which contains the arguments by their name (or the name ``arg``\ *n* for unnamed arguments, *n* being the argument position starting with 1) plus ``valid_``\ *argname* members for optional arguments which are being set to non-zero iff the respective *argname* was provided. Argument presence is determined at VCC time, so it is not possible to pass an unset argument from another function call. *eof*