[master] aacf14f71 whats-new: First pass on the changes since 6.6.0

Dridi Boukelmoune dridi.boukelmoune at gmail.com
Thu Sep 2 11:57:05 UTC 2021

commit aacf14f71056e1aaf27ea0fefd9f4c73c137a072
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Thu Sep 2 13:53:56 2021 +0200

    whats-new: First pass on the changes since 6.6.0
    I didn't bother adding references to other parts of the documentation.

diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-trunk.rst
index 0a9e8457f..dda4957ab 100644
--- a/doc/sphinx/whats-new/changes-trunk.rst
+++ b/doc/sphinx/whats-new/changes-trunk.rst
@@ -22,56 +22,245 @@ merged, may be found in the `change log`_.
 .. _change log: https://github.com/varnishcache/varnish-cache/blob/master/doc/changes.rst
+One major change for this release is the migration of the regular expression
+engine from PCRE to PCRE2. This change should be mostly transparent anywhere
+regular expressions are used, like VCL, ban expressions, VSL queries etc.
+There were some parameters changes, see the upgrade notes for more details.
+Structured Fields
+TODO (probably deserves its own section too)
-**XXX changes in -p parameters**
+There were changes to the parameters:
+- new ``pcre2_jit_compilation`` boolean defaulting to on
+- the default value increased to 16kB for ``vsl_buffer``
+- the default value increased to 96kB for ``workspace_client``
+- the default value increased to 96kB for ``workspace_backend``
+- the minimum value increased to 384B for ``workspace_session``
+- the minimum value increased to 65535B for ``h2_initial_window_size``
+- the minimum value increased to 10ms for ``thread_pool_fail_delay``
+- the default value increased to 80kB for ``thread_pool_stack``
+- the default value increased to 64kB for ``thread_pool_stack`` on 32bit
+  systems
+- ``vcc_acl_pedantic`` was removed, see upgrade notes for more details.
+- ``pcre_match_limit`` was renamed to ``pcre2_match_limit``
+- ``pcre_match_limit_recursion`` was renamed to ``pcre2_depth_limit``
+- new ``h2_rxbuf_storage`` defaulting to ``Transient``, see upgrade notes for
+  more details.
 Other changes in varnishd
+For pass transactions, ``varnishd`` no longer strips ``Range`` headers from
+client requests or ``Accept-Range`` and ``Content-Range`` headers from backend
+responses to allow partial delivery directly from the backend.
+When ``http_range_support`` is on (the default) a consistency check is
+performed on the backend response and malformed or inconsistent responses
+are treated as fetch errors.
+There is a new buffer for HTTP/2 request bodies allocated from storage.
+See upgrade notes for more details on both topics.
 Changes to VCL
 VCL variables
-**XXX new, deprecated or removed variables, or changed semantics**
+A new ``req.hash_ignore_vary`` flag allows to skip vary header checks during a
+lookup. This can be useful when only the freshness of a resource is relevant,
+and not a slight difference in representation.
+For interoperability purposes, it is now possible to quote header names that
+aren't valid VCL symbols, but valid HTTP header names, for example::
+    req.http."dotted.name"
+This is rarely observed and should only be needed to better integrate with the
+specific needs of certain clients or servers.
+Some global VCL symbols can be referenced before their declaration, this was
+extended to all global VCL symbols for the following keywords:
+- ``acl``
+- ``backend``
+- ``probe``
+- ``sub``
+Consider the following example::
+    sub vcl_recv {
+        set req.backend_hint = b;
+    }
+    backend b {
+        .host = "example.org";
+    }
+It used to fail the VCL compilation with "Symbol not found: 'b'" in
+``vcl_recv``, and is now supported.
+Bit flags
+There is a new bit flag syntax for certain VCL keywords::
+    keyword +flag -other ...
+Similarly to bit flag ``varnishd`` parameters ``debug``, ``feature`` and
+``vsl_mask``, a ``+`` prefix means that a flag is raised and a ``-`` prefix
+that a flag is cleared.
+The ``acl`` keyword supports the following flags:
+- ``log``
+- ``pedantic`` (enabled by default)
+- ``table``
+For example::
+    acl +log -pedantic { ... }
+The ``include`` keyword supports a ``glob`` flag.
+For example::
+    include +glob "example.org/*.vcl";
-Other changes to VCL
+See upgrade notes for more details.
-**XXX changes in the bundled VMODs**
+New ``BASE64CF`` encoding scheme in ``vmod_blob``. It is similar to
+``BASE64URL``, with the following changes to ``BASE64``:
+- ``+`` replaced with ``-``
+- ``/`` replaced with ``~``
+- ``_`` as the padding character
+It is used by a certain CDN provider who also inspired the name.
-**XXX changes concerning varnishlog(1) and/or vsl(7)**
+If a cache hit occurs on a streaming object, an object that is still being
+fetched, ``Hit`` records contain progress of the fetch task. This should help
+troubleshooting when cache hits appear to be slow, whether or not the backend
+is still serving the response.
+By default ``VCL_acl`` records are no longer emitted. They can be brought back
+by adding a ``+log`` flag to the ACL  declaration.
+New ``%{...}t`` time formats:
+- ``sec``
+- ``msec``
+- ``usec``
+- ``msec_frac``
+- ``usec_frac``
+See the varnishncsa manual for more information.
-**XXX changes concerning varnishadm(1) and/or varnish-cli(7)**
+The ``-t`` option sets up the timeout for both attaching to a running
+``varnishd`` instance and individual commands sent to that instance.
-**XXX changes concerning varnishstat(1) and/or varnish-counters(7)**
+Command completion should be more accurate in interactive mode.
-**XXX changes concerning varnishtest(1) and/or vtc(7)**
+Test cases should be generally more reactive, whether it is detecting
+a ``varnishd`` startup failure, waiting for ``varnishd`` to stop, or
+when fail tests and there are barriers waiting for a synchronization.
+Clients and servers can have up to 64 headers in requests and responses.
+The ``feature`` command allows to skip gracefully test cases that are
+missing specific requirements. It is now possible to skip a test based on
+the presence of a feature.
+For example, for test cases targeting 32bit environment with a working DNS
+    feature dns !64bit
+There are new feature checks:
+- ``coverage``
+- ``asan``
+- ``msan``
+- ``tsan``
+- ``ubsan``
+- ``sanitizer``
+- ``workspace_emulator``
+The undocumented ``pcre_jit`` feature check is gone.
+There is a new ``tunnel`` command that acts as a proxy between two peers. A
+tunnel can pause and control how much data goes in each direction, and can
+be used to trigger socket timeouts, possibly in the middle of protocol frames,
+without having to change how the peers are implemented.
+There is a new dynamic macro ``${string,repeat,<uint>,<string>}`` to avoid
+very long lines or potential mistakes when maintained by hand. For example,
+the two following strings are equivalent::
+    "AAA"
+    "${string,repeat,3,A}"
+There were also various improvements to HTTP/2 testing, and more should be
 Changes for developers and VMOD authors
+Varnish now comes with a second workspace implementation called the workspace
+emulator. It needs to be enabled during the build with the configure flag
+The workspace emulator performs sparse allocations and can help VMOD authors
+interested in fuzzing, especially when the Address Sanitizer is enabled as
+In order to make the emulator possible, some adjustments were needed for the
+workspace API. Deprecated functions ``WS_Front()`` and ``WS_Inside()`` were
+removed independently of the emulator.
+The ``STRING_LIST`` type is gone in favor of ``STRANDS``. All the VRT symbols
+related to ``STRING_LIST`` are either gone or changed.
+Convenience constants ``vrt_null_strands`` and ``vrt_null_blob`` were added.
+The migration to PCRE2 also brought many changes to the VRE API. The VRT
+functions handling ``REGEX`` arguments didn't change.
+The VNUM API also changed substantially for structured field number parsing.
+The deprecated functions ``VSB_new()`` and ``VSB_delete()`` were removed.
+See upgrade notes for more information.
 **XXX changes concerning VRT, the public APIs, source code organization,
 builds etc.**

More information about the varnish-commit mailing list