From phk at FreeBSD.org  Sat Mar  4 08:55:10 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Sat,  4 Mar 2023 08:55:10 +0000 (UTC)
Subject: [master] 40f5a532c Make struct waited::priv2 a const void * instead
 of uintptr_t
Message-ID: <20230304085510.B0C81A5030@lists.varnish-cache.org>


commit 40f5a532c30e1bb026a1eba939be71cbf1037f71
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Sat Mar 4 08:53:34 2023 +0000

    Make struct waited::priv2 a const void * instead of uintptr_t

diff --git a/bin/varnishd/cache/cache_session.c b/bin/varnishd/cache/cache_session.c
index d0834fc6c..dfa2a0bf7 100644
--- a/bin/varnishd/cache/cache_session.c
+++ b/bin/varnishd/cache/cache_session.c
@@ -454,8 +454,7 @@ ses_handle(struct waited *wp, enum wait_event ev, vtim_real now)
 
 	CHECK_OBJ_NOTNULL(wp, WAITED_MAGIC);
 	CAST_OBJ_NOTNULL(sp, wp->priv1, SESS_MAGIC);
-	CAST_OBJ_NOTNULL(xp, (const void*)wp->priv2, TRANSPORT_MAGIC);
-	AN(wp->priv2);
+	CAST_OBJ_NOTNULL(xp, wp->priv2, TRANSPORT_MAGIC);
 	assert(WS_Reservation(sp->ws) == wp);
 	FINI_OBJ(wp);
 
@@ -526,7 +525,7 @@ SES_Wait(struct sess *sp, const struct transport *xp)
 	INIT_OBJ(wp, WAITED_MAGIC);
 	wp->fd = sp->fd;
 	wp->priv1 = sp;
-	wp->priv2 = (uintptr_t)xp;
+	wp->priv2 = xp;
 	wp->idle = sp->t_idle;
 	wp->func = ses_handle;
 	wp->tmo = SESS_TMO(sp, timeout_idle);
diff --git a/bin/varnishd/waiter/waiter.h b/bin/varnishd/waiter/waiter.h
index 912f76669..ac00650dd 100644
--- a/bin/varnishd/waiter/waiter.h
+++ b/bin/varnishd/waiter/waiter.h
@@ -62,7 +62,7 @@ struct waited {
 	int			fd;
 	unsigned		idx;
 	void			*priv1;
-	uintptr_t		priv2;
+	const void		*priv2;
 	waiter_handle_f		*func;
 	vtim_dur		tmo;
 	vtim_real		idle;

From dridi.boukelmoune at gmail.com  Mon Mar  6 06:39:06 2023
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon,  6 Mar 2023 06:39:06 +0000 (UTC)
Subject: [master] fe3049ae3 Start skeleton release notes for the next version.
Message-ID: <20230306063906.A04C410F761@lists.varnish-cache.org>


commit fe3049ae374d522c7aca3a48adf7b43cf6b73379
Author: Geoff Simmons <geoff at uplex.de>
Date:   Tue Sep 25 16:31:17 2018 +0200

    Start skeleton release notes for the next version.
    
    Restructured so that:
    
    * 'Upgrading' is limited to work that has to be done to upgrade from
      a current deployment to the new version.
    
    * 'Changes' is a comprehensive, user-level description of changes and
      new features.
    
    Conflicts:
            doc/sphinx/whats-new/index.rst

diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-trunk.rst
new file mode 100644
index 000000000..fbda9d6b9
--- /dev/null
+++ b/doc/sphinx/whats-new/changes-trunk.rst
@@ -0,0 +1,73 @@
+**Note: This is a working document for a future release, with running
+updates for changes in the development branch. For changes in the
+released versions of Varnish, see:** :ref:`whats-new-index`
+
+.. _whatsnew_changes_CURRENT:
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+Changes in Varnish **$NEXT_RELEASE**
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+For information about updating your current Varnish deployment to the
+new version, see :ref:`whatsnew_upgrading_CURRENT`.
+
+A more detailed and technical account of changes in Varnish, with
+links to issues that have been fixed and pull requests that have been
+merged, may be found in the `change log`_.
+
+.. _change log: https://github.com/varnishcache/varnish-cache/blob/master/doc/changes.rst
+
+varnishd
+========
+
+Parameters
+~~~~~~~~~~
+
+**XXX changes in -p parameters**
+
+Other changes in varnishd
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Changes to VCL
+==============
+
+VCL variables
+~~~~~~~~~~~~~
+
+**XXX new, deprecated or removed variables, or changed semantics**
+
+Other changes to VCL
+~~~~~~~~~~~~~~~~~~~~
+
+VMODs
+=====
+
+**XXX changes in the bundled VMODs**
+
+varnishlog
+==========
+
+**XXX changes concerning varnishlog(1) and/or vsl(7)**
+
+varnishadm
+==========
+
+**XXX changes concerning varnishadm(1) and/or varnish-cli(7)**
+
+varnishstat
+===========
+
+**XXX changes concerning varnishstat(1) and/or varnish-counters(7)**
+
+varnishtest
+===========
+
+**XXX changes concerning varnishtest(1) and/or vtc(7)**
+
+Changes for developers and VMOD authors
+=======================================
+
+**XXX changes concerning VRT, the public APIs, source code organization,
+builds etc.**
+
+*eof*
diff --git a/doc/sphinx/whats-new/index.rst b/doc/sphinx/whats-new/index.rst
index 9d2561e96..380480c8f 100644
--- a/doc/sphinx/whats-new/index.rst
+++ b/doc/sphinx/whats-new/index.rst
@@ -13,6 +13,19 @@ This section describes the changes and improvements between different
 versions of Varnish, and what upgrading between the different versions
 entail.
 
+Varnish **$NEXT_RELEASE**
+-------------------------
+
+**Note: These are working documents for a future release, with running
+updates for changes in the development branch. For changes in the
+released versions of Varnish, see the chapters listed below.**
+
+.. toctree::
+   :maxdepth: 2
+
+   changes-trunk
+   upgrading-trunk
+
 Varnish **7.2**
 ---------------
 
diff --git a/doc/sphinx/whats-new/upgrading-trunk.rst b/doc/sphinx/whats-new/upgrading-trunk.rst
new file mode 100644
index 000000000..6143fde99
--- /dev/null
+++ b/doc/sphinx/whats-new/upgrading-trunk.rst
@@ -0,0 +1,33 @@
+**Note: This is a working document for a future release, with running
+updates for changes in the development branch. For changes in the
+released versions of Varnish, see:** :ref:`whats-new-index`
+
+.. _whatsnew_upgrading_CURRENT:
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+Upgrading to Varnish **$NEXT_RELEASE**
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+**XXX: how to upgrade from previous deployments to this
+version. Limited to work that has to be done for an upgrade, new
+features are listed in "Changes". Explicitly mention what does *not*
+have to be changed, especially in VCL. May include, but is not limited
+to:**
+
+* Elements of VCL that have been removed or are deprecated, or whose
+  semantics have changed.
+
+* -p parameters that have been removed or are deprecated, or whose
+  semantics have changed.
+
+* Changes in the CLI.
+
+* Changes in the output or interpretation of stats or the log, including
+  changes affecting varnishncsa/-hist/-top.
+
+* Changes that may be necessary in VTCs or in the use of varnishtest.
+
+* Changes in public APIs that may require changes in VMODs or VAPI/VUT
+  clients.
+
+*eof*

From dridi.boukelmoune at gmail.com  Mon Mar  6 14:27:06 2023
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon,  6 Mar 2023 14:27:06 +0000 (UTC)
Subject: [master] 8a413ed6d vsl: Fix cute typo in a comment
Message-ID: <20230306142706.8121911D8F7@lists.varnish-cache.org>


commit 8a413ed6d4c928cf1882f38be0de8f307e8be266
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Mar 6 10:10:47 2023 +0100

    vsl: Fix cute typo in a comment

diff --git a/include/vapi/vsl_int.h b/include/vapi/vsl_int.h
index b3378d103..1ac59e0f0 100644
--- a/include/vapi/vsl_int.h
+++ b/include/vapi/vsl_int.h
@@ -104,7 +104,7 @@
 #define VSL_WRAPMARKER	(((uint32_t)SLT__Reserved << 24) | 0x575757) /* "WWW" */
 
 /*
- * The identifiers in shmlogtag are "SLT_" + XML tag.  A script may be run
+ * The identifiers in shmlogtag are "SLT_" + VSL tag.  A script may be run
  * on this file to extract the table rather than handcode it
  */
 #define SLT__MAX 256

From dridi.boukelmoune at gmail.com  Mon Mar  6 14:27:06 2023
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon,  6 Mar 2023 14:27:06 +0000 (UTC)
Subject: [master] 9c9b9ff76 vsl: Bump SHMLOG marker for Log segments
Message-ID: <20230306142706.8D5DB11D8F9@lists.varnish-cache.org>


commit 9c9b9ff76e46fe84601c1c06a24b671fda6ff243
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Mar 6 10:11:59 2023 +0100

    vsl: Bump SHMLOG marker for Log segments
    
    This means an older varnishlog can no longer read logs from a live
    current varnishd server, and vice versa. It used to be interesting
    to use a more modern VUT to process logs for example to get better
    performance or new features like generalized -E.

diff --git a/include/vsl_priv.h b/include/vsl_priv.h
index 7fd71b1ce..500aac584 100644
--- a/include/vsl_priv.h
+++ b/include/vsl_priv.h
@@ -58,7 +58,7 @@
  */
 
 struct VSL_head {
-#define VSL_HEAD_MARKER		"VSLHEAD1"	/* Incr. as version# */
+#define VSL_HEAD_MARKER		"VSLHEAD2"	/* Incr. as version# */
 	char			marker[8];
 	ssize_t			segsize;
 	unsigned		segment_n;

From dridi.boukelmoune at gmail.com  Mon Mar  6 14:27:06 2023
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon,  6 Mar 2023 14:27:06 +0000 (UTC)
Subject: [master] 69693970c vsl: Don't make copies of VSL_FILE_ID
Message-ID: <20230306142706.A94B711D8FC@lists.varnish-cache.org>


commit 69693970cef5e19e60e0220fc4e8fb35a7a0297e
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Mar 6 10:14:41 2023 +0100

    vsl: Don't make copies of VSL_FILE_ID
    
    It will no longer be valid for assignments, and in one case the
    assignment was superfluous.

diff --git a/lib/libvarnishapi/vsl.c b/lib/libvarnishapi/vsl.c
index 8b0f095fa..5179d4fdf 100644
--- a/lib/libvarnishapi/vsl.c
+++ b/lib/libvarnishapi/vsl.c
@@ -413,8 +413,8 @@ VSL_PrintTransactions(struct VSL_data *vsl, struct VSL_transaction * const pt[],
 FILE*
 VSL_WriteOpen(struct VSL_data *vsl, const char *name, int append, int unbuf)
 {
-	const char head[] = VSL_FILE_ID;
 	FILE* f;
+
 	if (!strcmp(name, "-"))
 		f = stdout;
 	else
@@ -426,7 +426,8 @@ VSL_WriteOpen(struct VSL_data *vsl, const char *name, int append, int unbuf)
 	if (unbuf)
 		setbuf(f, NULL);
 	if (ftell(f) == 0 || f == stdout) {
-		if (fwrite(head, 1, sizeof head, f) != sizeof head) {
+		if (fwrite(VSL_FILE_ID, 1, sizeof VSL_FILE_ID, f) !=
+		    sizeof VSL_FILE_ID) {
 			vsl_diag(vsl, "%s", strerror(errno));
 			(void)fclose(f);
 			return (NULL);
diff --git a/lib/libvarnishapi/vsl_cursor.c b/lib/libvarnishapi/vsl_cursor.c
index 8cf259394..1dedb3cc9 100644
--- a/lib/libvarnishapi/vsl_cursor.c
+++ b/lib/libvarnishapi/vsl_cursor.c
@@ -527,7 +527,7 @@ VSL_CursorFile(struct VSL_data *vsl, const char *name, unsigned options)
 	struct vslc_file *c;
 	int fd;
 	int close_fd = 0;
-	char buf[] = VSL_FILE_ID;
+	char buf[sizeof VSL_FILE_ID];
 	ssize_t i;
 
 	CHECK_OBJ_NOTNULL(vsl, VSL_MAGIC);

From dridi.boukelmoune at gmail.com  Mon Mar  6 14:27:06 2023
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon,  6 Mar 2023 14:27:06 +0000 (UTC)
Subject: [master] 12a27e8d2 vsl: Bump on-disk header of VSL dumps
Message-ID: <20230306142706.C659111D902@lists.varnish-cache.org>


commit 12a27e8d21fa7eacf7b6a754eed72f92a94f31e3
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Mar 6 10:15:45 2023 +0100

    vsl: Bump on-disk header of VSL dumps
    
    This prevents VUTs from attempting to read incompatible formats, while
    preserving the current header size, aligning it with the SHMLOG header
    at the format number 2.

diff --git a/lib/libvarnishapi/vsl_api.h b/lib/libvarnishapi/vsl_api.h
index b374be38c..848a02b25 100644
--- a/lib/libvarnishapi/vsl_api.h
+++ b/lib/libvarnishapi/vsl_api.h
@@ -31,7 +31,9 @@
  *
  */
 
-#define VSL_FILE_ID			"VSL"
+static const char			vsl_file_id[] = {'V', 'S', 'L', '2'};
+
+#define VSL_FILE_ID			(vsl_file_id)
 
 /*lint -esym(534, vsl_diag) */
 int vsl_diag(struct VSL_data *vsl, const char *fmt, ...) v_printflike_(2, 3);

From dridi.boukelmoune at gmail.com  Mon Mar  6 14:27:06 2023
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon,  6 Mar 2023 14:27:06 +0000 (UTC)
Subject: [master] acea9eb33 whats-new: First pass on changes
Message-ID: <20230306142706.E4A5B11D906@lists.varnish-cache.org>


commit acea9eb33675441f5c3c7d49fa2e5c01bd4a54c6
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Mar 6 12:56:25 2023 +0100

    whats-new: First pass on changes

diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-trunk.rst
index fbda9d6b9..9fddc2441 100644
--- a/doc/sphinx/whats-new/changes-trunk.rst
+++ b/doc/sphinx/whats-new/changes-trunk.rst
@@ -25,9 +25,33 @@ Parameters
 
 **XXX changes in -p parameters**
 
+There is a new parameter ``transit_buffer`` disabled by default to limit the
+amount of storage used for uncacheable responses. This is useful in situations
+where slow clients may consume large but uncacheable objects, to prevent them
+from filling up storage too fast at the expense of cacheable resources. When
+transit buffer is enabled, a client request will effectively hold its backend
+connection open until the client response delivery completes.
+
 Other changes in varnishd
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 
+In addition to classic Unix-domain sockets, abstract sockets can now be used
+on Linux. Instead of an absolute path, the syntax ``-a @name`` can be used to
+bind the abstract socket called ``name``.
+
+Weak ``Last-Modified`` headers are no longer candidates for revalidation. This
+means that a subsequent fetch will not, when such a stale object is available,
+include an ``If-Modified-Since`` header. A weak ``Last-Modified`` header does
+not prevent ``Etag`` revalidation.
+
+A cache hit on an object being streamed no longer prevents delivery of partial
+responses (status code 206) to range requests.
+
+Response status codes other than 200 and 204 are now considered errors for ESI
+fragments. The default behavior was changed, errors are no longer delivered by
+default. The feature flag ``esi_include_onerror`` can be raised to allow a
+backend to specify whether to continue.
+
 Changes to VCL
 ==============
 
@@ -36,9 +60,44 @@ VCL variables
 
 **XXX new, deprecated or removed variables, or changed semantics**
 
+The variables ``req.xid``, ``bereq.xid`` and ``sess.xid`` are now integers
+instead of strings, but should remain usable without a VCL change in a string
+context.
+
+Transit buffer can be controlled per fetch with the ``beresp.transit_buffer``
+variable.
+
 Other changes to VCL
 ~~~~~~~~~~~~~~~~~~~~
 
+Backends have a new ``.via`` attribute referencing another backend::
+
+    backend detour {
+        .host = "...";
+    }
+
+    backend destination {
+        .host = "...";
+        .via = detour;
+    }
+
+Attempting a connection for ``destination`` connects to ``detour`` with a
+PROXYv2 protocol header targeting ``destination``'s address. Optionally, the
+``destination`` backend could use the other new ``.authority`` attribute to
+define an authority TLV in the PROXYv2 header.
+
+Backends can connect to abstract sockets on linux::
+
+    backend abstract {
+        .path = "@name";
+    }
+
+This is the same syntax as the ``varnishd -a`` command line option.
+
+Probes have a new ``.expect_close`` attribute defaulting to ``true``, matching
+the current behavior. Setting it to ``false`` will defer final checks until
+after the probe times out.
+
 VMODs
 =====
 
@@ -49,11 +108,27 @@ varnishlog
 
 **XXX changes concerning varnishlog(1) and/or vsl(7)**
 
+The in-memory and on-disk format of VSL records changed to allow 64bit VXID
+numbers. The new binary format is not compatible with previous versions, and
+log dumps performed with a previous Varnish release are no longer readable
+from now on.
+
+The VXID range is limited to ``VRT_INTEGER`` to fit in VCL the variables
+``req.xid``, ``bereq.xid`` and ``sess.xid``.
+
+A ``ReqStart`` record is emitted for bad requests, allowing ``varnishncsa`` to
+find the client IP address.
+
 varnishadm
 ==========
 
 **XXX changes concerning varnishadm(1) and/or varnish-cli(7)**
 
+The ``debug.xid`` command generally used by ``varnishtest`` used to set up the
+current VXID. As the intent usually is to set up the next VXID, this forced to
+set an off-by-one value. To simplify its usage it now sets up the next VXID
+directly.
+
 varnishstat
 ===========
 
@@ -64,10 +139,29 @@ varnishtest
 
 **XXX changes concerning varnishtest(1) and/or vtc(7)**
 
+It is now possible to send special keys NPAGE, PPAGE, HOME and END to a
+process.
+
+The ``-nolen`` option is implied for ``txreq`` and ``txresp`` when either
+``Content-Length`` or ``Transfer-Encoding`` headers are present.
+
+A new ``stream.peer_window`` variable similar to ``stream.window`` is
+available for HTTP/2 checks.
+
 Changes for developers and VMOD authors
 =======================================
 
 **XXX changes concerning VRT, the public APIs, source code organization,
 builds etc.**
 
+There is a new convenience macro ``WS_TASK_ALLOC_OBJ()`` to allocate objects
+from the current tasks' workspace.
+
+The ``NO_VXID`` macro can be used to explicitly log records outside of a
+transaction.
+
+Custom backend implementations are now in charge of printing headers, which
+avoids duplicates when a custom implementation relied on ``http_*()`` that
+would also log the headers being set up.
+
 *eof*

From dridi.boukelmoune at gmail.com  Mon Mar  6 14:27:07 2023
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon,  6 Mar 2023 14:27:07 +0000 (UTC)
Subject: [master] 424f2c6b3 whats-new: First pass on upgrade notes
Message-ID: <20230306142707.104B611D90C@lists.varnish-cache.org>


commit 424f2c6b3b79f4065d98ff92c4d967a024a02d34
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Mar 6 14:42:04 2023 +0100

    whats-new: First pass on upgrade notes

diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-trunk.rst
index 9fddc2441..4df5b8236 100644
--- a/doc/sphinx/whats-new/changes-trunk.rst
+++ b/doc/sphinx/whats-new/changes-trunk.rst
@@ -164,4 +164,12 @@ Custom backend implementations are now in charge of printing headers, which
 avoids duplicates when a custom implementation relied on ``http_*()`` that
 would also log the headers being set up.
 
+The ``VRT_new_backend*()`` functions take an additional backend argument, the
+optional via backend. It can not be a custom backend implementation, but it
+can be a director resolving a native backend.
+
+There is a new ``authority`` field for via backends in ``struct vrt_backend``.
+
+There is a new ``exp_close`` field in ``struct vrt_backend_probe``.
+
 *eof*
diff --git a/doc/sphinx/whats-new/upgrading-trunk.rst b/doc/sphinx/whats-new/upgrading-trunk.rst
index 6143fde99..6a2f56cff 100644
--- a/doc/sphinx/whats-new/upgrading-trunk.rst
+++ b/doc/sphinx/whats-new/upgrading-trunk.rst
@@ -30,4 +30,37 @@ to:**
 * Changes in public APIs that may require changes in VMODs or VAPI/VUT
   clients.
 
+New VSL format
+==============
+
+The binary format of Varnish logs changed to increase the space for VXIDs from
+32 bits to 64. This is not a change that older versions of the Varnish logging
+utilities can understand, and the new utilities can also not process old logs.
+
+There is no conversion tool from the old format to the new one, so this should
+become a problem only when raw logs are stored for future processing. If old
+binary logs need to remain usable, the only solution is to use a compatible
+Varnish version and at the time of this release, the 6.0 branch is the only
+one without an EOL date.
+
+Via backends
+============
+
+The new backend argument to the ``VRT_new_backend*()`` functions is optional
+and ``NULL`` can be passed to match the previous behavior.
+
+suckaddr
+========
+
+The following functions return or accept ``const`` pointers from now on:
+
+- ``VSA_Clone()``
+- ``VSA_getsockname()``
+- ``VSA_getpeername()``
+- ``VSA_free()``
+- ``VSA_Malloc()``
+- ``VSA_Build*()``
+- ``VSS_ResolveOne()``
+- ``VSS_ResolveFirst()``
+
 *eof*

From nils.goroll at uplex.de  Mon Mar  6 14:41:05 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon,  6 Mar 2023 14:41:05 +0000 (UTC)
Subject: [master] 81381076f gc unused SLTs (VSL tags)
Message-ID: <20230306144105.9E5C611E6DC@lists.varnish-cache.org>


commit 81381076fa9d5bdae543e3fd788fea0586f2fbf1
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon Mar 6 15:33:57 2023 +0100

    gc unused SLTs (VSL tags)
    
    Now that we broke the VSL format, this is the time to act.

diff --git a/doc/changes.rst b/doc/changes.rst
index 7d63ec4b1..fbda69fe8 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -72,6 +72,12 @@ Varnish Cache NEXT (2023-03-15)
 
     ``debug.xid 20000000000``
 
+* Consequently, VSL clients (log processing tools) are now
+  incompatible with logs and in-memory data written by previous
+  versions, and vice versa.
+
+* Unused log tags (SLTs) have been removed.
+
 * The ``debug.xid`` CLI command now sets the next XID to be used,
   rather than "one less than the next XID to be used"
 
diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-trunk.rst
index 4df5b8236..8333e2bde 100644
--- a/doc/sphinx/whats-new/changes-trunk.rst
+++ b/doc/sphinx/whats-new/changes-trunk.rst
@@ -111,7 +111,7 @@ varnishlog
 The in-memory and on-disk format of VSL records changed to allow 64bit VXID
 numbers. The new binary format is not compatible with previous versions, and
 log dumps performed with a previous Varnish release are no longer readable
-from now on.
+from now on. Consequently, unused log tags have been removed.
 
 The VXID range is limited to ``VRT_INTEGER`` to fit in VCL the variables
 ``req.xid``, ``bereq.xid`` and ``sess.xid``.
diff --git a/include/tbl/vsl_tags.h b/include/tbl/vsl_tags.h
index 9e28d92e0..20a563dae 100644
--- a/include/tbl/vsl_tags.h
+++ b/include/tbl/vsl_tags.h
@@ -119,18 +119,6 @@ SLTM(BackendOpen, 0, "Backend connection opened",
 	"\n"
 )
 
-SLTM(BackendReuse, 0, "Backend connection put up for reuse",
-	"Logged when a backend connection is put up for reuse by a later"
-	" connection.\n\n"
-	"The format is::\n\n"
-	"\t%d %s\n"
-	"\t|  |\n"
-	"\t|  +- Backend display name\n"
-	"\t+---- Connection file descriptor\n"
-	"\n"
-	NOSUP_NOTICE
-)
-
 SLTM(BackendClose, 0, "Backend connection closed",
 	"Logged when a backend connection is closed.\n\n"
 	"The format is::\n\n"
@@ -166,19 +154,6 @@ SLTM(ProxyGarbage, 0, "Unparseable PROXY request",
 	"A PROXY protocol header was unparseable.\n\n"
 )
 
-SLTM(Backend, 0, "Backend selected",
-	"Logged when a connection is selected for handling a backend"
-	" request.\n\n"
-	"The format is::\n\n"
-	"\t%d %s %s\n"
-	"\t|  |  |\n"
-	"\t|  |  +- Backend display name\n"
-	"\t|  +---- VCL name\n"
-	"\t+------- Connection file descriptor\n"
-	"\n"
-	NOSUP_NOTICE
-)
-
 SLTM(Length, 0, "Size of object body",
 	"Logs the size of a fetch object body.\n\n"
 )
@@ -623,18 +598,6 @@ SLTM(Witness, 0, "Lock order witness records",
 	"Diagnostic recording of locking order.\n"
 )
 
-SLTM(BackendStart, 0, "Backend request start",
-	"Start of backend processing. Logs the backend IP address and port"
-	" number.\n\n"
-	"The format is::\n\n"
-	"\t%s %s\n"
-	"\t|  |\n"
-	"\t|  +- Backend Port number\n"
-	"\t+---- Backend IP4/6 address\n"
-	"\n"
-	NOSUP_NOTICE
-)
-
 SLTM(H2RxHdr, SLT_F_BINARY, "Received HTTP2 frame header",
 	"Binary data"
 )

From nils.goroll at uplex.de  Mon Mar  6 14:54:06 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon,  6 Mar 2023 14:54:06 +0000 (UTC)
Subject: [master] 939585cf2 Inform about the default working directory in
 usage information
Message-ID: <20230306145406.46D0211EE2D@lists.varnish-cache.org>


commit 939585cf2079fc6e59551c257c8ed2572b7c707c
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu Mar 2 12:17:14 2023 +0100

    Inform about the default working directory in usage information
    
    I noticed that users, apparently, have no way of finding out what
    their distribution configured?

diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index 2d77b7d63..698a6bbe6 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -82,6 +82,8 @@ static const char opt_spec[] = "?a:b:CdE:f:Fh:i:I:j:l:M:n:P:p:r:S:s:T:t:VW:x:";
 static void
 usage(void)
 {
+	char buf[1024];
+
 #define FMT "  %-28s # %s\n"
 
 	printf( "Usage: varnishd [options]\n");
@@ -102,7 +104,8 @@ usage(void)
 	printf(FMT, "", "  default: \":80\"");
 	printf(FMT, "-f vclfile", "VCL program");
 	printf(FMT, "", "Can be specified multiple times.");
-	printf(FMT, "-n dir", "Working directory");
+	bprintf(buf, "Working directory (default: %s)", VIN_n_Arg(NULL));
+	printf(FMT, "-n dir", buf);
 
 	printf("\n-b can be used only once, and not together with -f\n");
 

From nils.goroll at uplex.de  Mon Mar  6 15:19:06 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon,  6 Mar 2023 15:19:06 +0000 (UTC)
Subject: [master] 919a9016b Noop-Polish VRT_Assign_Backend()
Message-ID: <20230306151906.D5ECD11FA6D@lists.varnish-cache.org>


commit 919a9016b4c0f71cc92ad1bdf3f21062f7285891
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon Feb 27 21:17:36 2023 +0100

    Noop-Polish VRT_Assign_Backend()
    
    Use a local vdir variable for clarity like elsewhere in the code.
    Use the lock in vdir, not the pointer to it in VCL_BACKEND for
    consistency with VRT_DelDirector() a few lines above.

diff --git a/bin/varnishd/cache/cache_vrt_vcl.c b/bin/varnishd/cache/cache_vrt_vcl.c
index 2ba29b0a0..93b3064c2 100644
--- a/bin/varnishd/cache/cache_vrt_vcl.c
+++ b/bin/varnishd/cache/cache_vrt_vcl.c
@@ -284,29 +284,32 @@ VRT_DelDirector(VCL_BACKEND *bp)
 void
 VRT_Assign_Backend(VCL_BACKEND *dst, VCL_BACKEND src)
 {
+	struct vcldir *vdir;
 	int busy;
 
 	AN(dst);
 	CHECK_OBJ_ORNULL((*dst), DIRECTOR_MAGIC);
 	CHECK_OBJ_ORNULL(src, DIRECTOR_MAGIC);
 	if (*dst != NULL) {
-		CHECK_OBJ_NOTNULL((*dst)->vdir, VCLDIR_MAGIC);
-		if (!((*dst)->vdir->flags & VDIR_FLG_NOREFCNT)) {
-			Lck_Lock((*dst)->mtx);
-			assert((*dst)->vdir->refcnt > 0);
-			busy = --(*dst)->vdir->refcnt;
-			Lck_Unlock((*dst)->mtx);
+		vdir = (*dst)->vdir;
+		CHECK_OBJ_NOTNULL(vdir, VCLDIR_MAGIC);
+		if (!(vdir->flags & VDIR_FLG_NOREFCNT)) {
+			Lck_Lock(&vdir->dlck);
+			assert(vdir->refcnt > 0);
+			busy = --vdir->refcnt;
+			Lck_Unlock(&vdir->dlck);
 			if (!busy)
 				retire_backend(dst);
 		}
 	}
 	if (src != NULL) {
-		CHECK_OBJ_NOTNULL(src->vdir, VCLDIR_MAGIC);
-		if (!(src->vdir->flags & VDIR_FLG_NOREFCNT)) {
-			Lck_Lock(src->mtx);
-			assert(src->vdir->refcnt > 0);
-			src->vdir->refcnt++;
-			Lck_Unlock(src->mtx);
+		vdir = src->vdir;
+		CHECK_OBJ_NOTNULL(vdir, VCLDIR_MAGIC);
+		if (!(vdir->flags & VDIR_FLG_NOREFCNT)) {
+			Lck_Lock(&vdir->dlck);
+			assert(vdir->refcnt > 0);
+			vdir->refcnt++;
+			Lck_Unlock(&vdir->dlck);
 		}
 	}
 	*dst = src;

From nils.goroll at uplex.de  Mon Mar  6 15:19:06 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon,  6 Mar 2023 15:19:06 +0000 (UTC)
Subject: [master] 94aad8c3c Refactor retire_backend(VCL_BACKEND *) to
 vcldir_retire(struct vcldir *)
Message-ID: <20230306151906.EC8BF11FA70@lists.varnish-cache.org>


commit 94aad8c3cc497849b59ff294ef2198897e0e0ec6
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon Feb 27 21:50:12 2023 +0100

    Refactor retire_backend(VCL_BACKEND *) to vcldir_retire(struct vcldir *)
    
    This is in preparation of follow-up commits. Reasoning:
    
    - in both call sites, we already use the struct vcldir *
    - once call site actually used TAKE_OBJ semantics, but those
      can easily be moved

diff --git a/bin/varnishd/cache/cache_vrt_vcl.c b/bin/varnishd/cache/cache_vrt_vcl.c
index 93b3064c2..ec7cb9cde 100644
--- a/bin/varnishd/cache/cache_vrt_vcl.c
+++ b/bin/varnishd/cache/cache_vrt_vcl.c
@@ -240,17 +240,12 @@ VRT_StaticDirector(VCL_BACKEND b)
 }
 
 static void
-retire_backend(VCL_BACKEND *bp)
+vcldir_retire(struct vcldir *vdir)
 {
-	struct vcldir *vdir;
 	const struct vcltemp *temp;
-	VCL_BACKEND d;
 
-	TAKE_OBJ_NOTNULL(d, bp, DIRECTOR_MAGIC);
-	vdir = d->vdir;
 	CHECK_OBJ_NOTNULL(vdir, VCLDIR_MAGIC);
 	assert(vdir->refcnt == 0);
-	assert (d == vdir->dir);
 	CHECK_OBJ_NOTNULL(vdir->vcl, VCL_MAGIC);
 
 	Lck_Lock(&vcl_mtx);
@@ -259,26 +254,27 @@ retire_backend(VCL_BACKEND *bp)
 	Lck_Unlock(&vcl_mtx);
 
 	if (temp->is_warm)
-		VDI_Event(d, VCL_EVENT_COLD);
+		VDI_Event(vdir->dir, VCL_EVENT_COLD);
 	if (vdir->methods->destroy != NULL)
-		vdir->methods->destroy(d);
-	assert (d == vdir->dir);
+		vdir->methods->destroy(vdir->dir);
 	vcldir_free(vdir);
 }
 
 void
-VRT_DelDirector(VCL_BACKEND *bp)
+VRT_DelDirector(VCL_BACKEND *dirp)
 {
+	VCL_BACKEND dir;
 	struct vcldir *vdir;
 
-	AN(bp);
-	vdir = (*bp)->vdir;
+	TAKE_OBJ_NOTNULL(dir, dirp, DIRECTOR_MAGIC);
+
+	vdir = dir->vdir;
 	CHECK_OBJ_NOTNULL(vdir, VCLDIR_MAGIC);
 	Lck_Lock(&vdir->dlck);
 	assert(vdir->refcnt == 1);
 	vdir->refcnt = 0;
 	Lck_Unlock(&vdir->dlck);
-	retire_backend(bp);
+	vcldir_retire(vdir);
 }
 
 void
@@ -299,7 +295,7 @@ VRT_Assign_Backend(VCL_BACKEND *dst, VCL_BACKEND src)
 			busy = --vdir->refcnt;
 			Lck_Unlock(&vdir->dlck);
 			if (!busy)
-				retire_backend(dst);
+				vcldir_retire(vdir);
 		}
 	}
 	if (src != NULL) {

From nils.goroll at uplex.de  Mon Mar  6 15:19:07 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon,  6 Mar 2023 15:19:07 +0000 (UTC)
Subject: [master] 7f0e41c92 Add vcldir_deref() and call it from
 VRT_DelDirector() also
Message-ID: <20230306151907.1A9B811FA74@lists.varnish-cache.org>


commit 7f0e41c92981257687e26f443d7674559f313cec
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon Feb 27 22:05:13 2023 +0100

    Add vcldir_deref() and call it from VRT_DelDirector() also
    
    The last reference to a director might go away with VRT_DelDirector
    _or_ VRT_Asssign_Backend, which the former needs to account for.
    
    We assert for the VDIR_FLG_NOREFCNT case that there was only one
    reference such that a single deref yields no reference left.
    
    Part one of the fix for #3895

diff --git a/bin/varnishd/cache/cache_vrt_vcl.c b/bin/varnishd/cache/cache_vrt_vcl.c
index ec7cb9cde..01c660220 100644
--- a/bin/varnishd/cache/cache_vrt_vcl.c
+++ b/bin/varnishd/cache/cache_vrt_vcl.c
@@ -260,6 +260,24 @@ vcldir_retire(struct vcldir *vdir)
 	vcldir_free(vdir);
 }
 
+static int
+vcldir_deref(struct vcldir *vdir)
+{
+	int busy;
+
+	CHECK_OBJ_NOTNULL(vdir, VCLDIR_MAGIC);
+	AZ(vdir->flags & VDIR_FLG_NOREFCNT);
+
+	Lck_Lock(&vdir->dlck);
+	assert(vdir->refcnt > 0);
+	busy = --vdir->refcnt;
+	Lck_Unlock(&vdir->dlck);
+
+	if (!busy)
+		vcldir_retire(vdir);
+	return (busy);
+}
+
 void
 VRT_DelDirector(VCL_BACKEND *dirp)
 {
@@ -270,18 +288,19 @@ VRT_DelDirector(VCL_BACKEND *dirp)
 
 	vdir = dir->vdir;
 	CHECK_OBJ_NOTNULL(vdir, VCLDIR_MAGIC);
-	Lck_Lock(&vdir->dlck);
-	assert(vdir->refcnt == 1);
-	vdir->refcnt = 0;
-	Lck_Unlock(&vdir->dlck);
-	vcldir_retire(vdir);
+
+	if (vdir->flags & VDIR_FLG_NOREFCNT) {
+		vdir->flags &= ~VDIR_FLG_NOREFCNT;
+		AZ(vcldir_deref(vdir));
+	} else {
+		(void) vcldir_deref(vdir);
+	}
 }
 
 void
 VRT_Assign_Backend(VCL_BACKEND *dst, VCL_BACKEND src)
 {
 	struct vcldir *vdir;
-	int busy;
 
 	AN(dst);
 	CHECK_OBJ_ORNULL((*dst), DIRECTOR_MAGIC);
@@ -289,14 +308,8 @@ VRT_Assign_Backend(VCL_BACKEND *dst, VCL_BACKEND src)
 	if (*dst != NULL) {
 		vdir = (*dst)->vdir;
 		CHECK_OBJ_NOTNULL(vdir, VCLDIR_MAGIC);
-		if (!(vdir->flags & VDIR_FLG_NOREFCNT)) {
-			Lck_Lock(&vdir->dlck);
-			assert(vdir->refcnt > 0);
-			busy = --vdir->refcnt;
-			Lck_Unlock(&vdir->dlck);
-			if (!busy)
-				vcldir_retire(vdir);
-		}
+		if (!(vdir->flags & VDIR_FLG_NOREFCNT))
+			vcldir_deref(vdir);
 	}
 	if (src != NULL) {
 		vdir = src->vdir;

From nils.goroll at uplex.de  Mon Mar  6 15:19:07 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon,  6 Mar 2023 15:19:07 +0000 (UTC)
Subject: [master] e8c77d893 Add a director release callback to let go of all
 backends
Message-ID: <20230306151907.6018711FA81@lists.varnish-cache.org>


commit e8c77d893eab70debd1018c0b59aed89dab3db79
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon Feb 27 22:14:53 2023 +0100

    Add a director release callback to let go of all backends
    
    Before this patch, layered directors needed to be destroyed top to
    bottom, and whenever that order was missed, we would panic, because a
    to-be-destroyed director still had references to it.
    
    One special case where this issue would always trigger are looped
    directors. Those should not be used and will cause havoc, which is a
    separate issue #3899. But we should still be able to unconfigure such
    a configuration.
    
    We solve the destruction order issue by making it a two step process:
    
    When a director is destroyed through VRT_DelDirector, a new release
    function is called, which has to disassociate any backends. The
    director then loses a reference, and when all references are gone, the
    destroy function is called.
    
    The new callback would not be necessary for the cases in varnish-cache
    today, directors could simply disassociate any backends before calling
    VRT_DelDirector. But this would complicate or even make impossible
    transfer of director ownership, where the code responsible for
    creating a director is not the same as the one calling
    VRT_DelDirector(). As a side effect, it also helps clarity.
    
    Fixes #3895

diff --git a/bin/varnishd/cache/cache_vrt_vcl.c b/bin/varnishd/cache/cache_vrt_vcl.c
index 01c660220..3ab464c07 100644
--- a/bin/varnishd/cache/cache_vrt_vcl.c
+++ b/bin/varnishd/cache/cache_vrt_vcl.c
@@ -289,6 +289,9 @@ VRT_DelDirector(VCL_BACKEND *dirp)
 	vdir = dir->vdir;
 	CHECK_OBJ_NOTNULL(vdir, VCLDIR_MAGIC);
 
+	if (vdir->methods->release != NULL)
+		vdir->methods->release(vdir->dir);
+
 	if (vdir->flags & VDIR_FLG_NOREFCNT) {
 		vdir->flags &= ~VDIR_FLG_NOREFCNT;
 		AZ(vcldir_deref(vdir));
diff --git a/bin/varnishtest/tests/r03895.vtc b/bin/varnishtest/tests/r03895.vtc
new file mode 100644
index 000000000..372b94513
--- /dev/null
+++ b/bin/varnishtest/tests/r03895.vtc
@@ -0,0 +1,48 @@
+
+
+varnishtest "looped backends"
+
+server s1 {
+} -start
+
+server s2 {
+} -start
+
+server s3 {
+} -start
+
+server s4 {
+} -start
+
+varnish v1 -vcl+backend {
+	import directors;
+	import std;
+
+	sub vcl_init {
+		new rr = directors.round_robin();
+		rr.add_backend(s1);
+		rr.add_backend(s2);
+		rr.add_backend(s3);
+		rr.add_backend(s4);
+		new rr2 = directors.round_robin();
+		rr2.add_backend(rr.backend());
+
+		rr.add_backend(rr2.backend());
+	}
+} -start
+
+varnish v1 -vcl+backend {
+	import directors;
+	import std;
+
+	sub vcl_init {
+		new rr2 = directors.round_robin();
+		rr2.add_backend(s1);
+		rr2.add_backend(s2);
+		rr2.add_backend(s3);
+		rr2.add_backend(s4);
+	}
+}
+
+varnish v1 -cliok "vcl.discard vcl1"
+varnish v1 -cliok "vcl.list"
diff --git a/doc/changes.rst b/doc/changes.rst
index fbda69fe8..d53f67986 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -91,6 +91,18 @@ Varnish Cache NEXT (2023-03-15)
   * The ``VRT_new_backend_clustered()`` and ``VRT_new_backend()``
     signatures have been changed
 
+* Directors which take and hold references to other directors via
+  ``VRT_Assign_Backend()`` (typically any director which has other
+  directors as backends) are now expected to implement the new
+  ``.release`` callback of type ``void
+  vdi_release_f(VCL_BACKEND)``. This function is called by
+  ``VRT_DelDirector()``. The implementation is expected drop any
+  backend references which the director holds (again using
+  ``VRT_Assign_Backend()`` with ``NULL`` as the second argument).
+
+  Failure to implement this callback can result in deadlocks, in
+  particular during VCL discard.
+
 ================================
 Varnish Cache 7.2.0 (2022-09-15)
 ================================
diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-trunk.rst
index 8333e2bde..fc1e56cdb 100644
--- a/doc/sphinx/whats-new/changes-trunk.rst
+++ b/doc/sphinx/whats-new/changes-trunk.rst
@@ -172,4 +172,13 @@ There is a new ``authority`` field for via backends in ``struct vrt_backend``.
 
 There is a new ``exp_close`` field in ``struct vrt_backend_probe``.
 
+Directors which take and hold references to other directors via
+``VRT_Assign_Backend()`` (typically any director which has other
+directors as backends) are now expected to implement the new
+``.release`` callback of type ``void
+vdi_release_f(VCL_BACKEND)``. This function is called by
+``VRT_DelDirector()``. The implementation is expected drop any backend
+references which the director holds (again using
+``VRT_Assign_Backend()`` with ``NULL`` as the second argument).
+
 *eof*
diff --git a/include/vrt.h b/include/vrt.h
index 5d01f542d..adb7713d5 100644
--- a/include/vrt.h
+++ b/include/vrt.h
@@ -64,6 +64,7 @@
  *	VRT_new_backend() signature changed
  *	VRT_new_backend_clustered() signature changed
  *	authority field added to struct vrt_backend
+ *	release field added to struct vdi_methods
  * 16.0 (2022-09-15)
  *	VMOD C-prototypes moved into JSON
  *	VRT_AddVDP() deprecated
@@ -692,6 +693,7 @@ typedef VCL_IP vdi_getip_f(VRT_CTX, VCL_BACKEND);
 typedef void vdi_finish_f(VRT_CTX, VCL_BACKEND);
 typedef stream_close_t vdi_http1pipe_f(VRT_CTX, VCL_BACKEND);
 typedef void vdi_event_f(VCL_BACKEND, enum vcl_event_e);
+typedef void vdi_release_f(VCL_BACKEND);
 typedef void vdi_destroy_f(VCL_BACKEND);
 typedef void vdi_panic_f(VCL_BACKEND, struct vsb *);
 typedef void vdi_list_f(VRT_CTX, VCL_BACKEND, struct vsb *, int, int);
@@ -707,6 +709,9 @@ struct vdi_methods {
 	vdi_getip_f			*getip;
 	vdi_finish_f			*finish;
 	vdi_event_f			*event;
+	// called by VRT_DelDirector: deref all backends
+	vdi_release_f			*release;
+	// when refcount goes 0
 	vdi_destroy_f			*destroy;
 	vdi_panic_f			*panic;
 	vdi_list_f			*list;
diff --git a/vmod/vmod_directors.c b/vmod/vmod_directors.c
index 2130c4b55..7f429c9cc 100644
--- a/vmod/vmod_directors.c
+++ b/vmod/vmod_directors.c
@@ -92,17 +92,27 @@ vdir_new(VRT_CTX, struct vdir **vdp, const char *vcl_name,
 	AN(vd->healthy);
 }
 
+void
+vdir_release(struct vdir *vd)
+{
+	unsigned u;
+
+	CHECK_OBJ_NOTNULL(vd, VDIR_MAGIC);
+
+	for (u = 0; u < vd->n_backend; u++)
+		VRT_Assign_Backend(&vd->backend[u], NULL);
+	vd->n_backend = 0;
+}
+
 void
 vdir_delete(struct vdir **vdp)
 {
 	struct vdir *vd;
-	unsigned u;
 
 	TAKE_OBJ_NOTNULL(vd, vdp, VDIR_MAGIC);
 
 	AZ(vd->dir);
-	for (u = 0; u < vd->n_backend; u++)
-		VRT_Assign_Backend(&vd->backend[u], NULL);
+	AZ(vd->n_backend);
 	free(vd->backend);
 	free(vd->weight);
 	AZ(pthread_rwlock_destroy(&vd->mtx));
diff --git a/vmod/vmod_directors.h b/vmod/vmod_directors.h
index 00b47b131..5cb13f67d 100644
--- a/vmod/vmod_directors.h
+++ b/vmod/vmod_directors.h
@@ -46,6 +46,7 @@ struct vdir {
 
 void vdir_new(VRT_CTX, struct vdir **vdp, const char *vcl_name,
     const struct vdi_methods *, void *priv);
+void vdir_release(struct vdir *vd);
 void vdir_delete(struct vdir **vdp);
 void vdir_rdlock(struct vdir *vd);
 void vdir_wrlock(struct vdir *vd);
diff --git a/vmod/vmod_directors_fall_back.c b/vmod/vmod_directors_fall_back.c
index 0e5d159cb..b24e25e8e 100644
--- a/vmod/vmod_directors_fall_back.c
+++ b/vmod/vmod_directors_fall_back.c
@@ -175,6 +175,16 @@ vmod_fallback_resolve(VRT_CTX, VCL_BACKEND dir)
 	return (be);
 }
 
+static void v_matchproto_(vdi_release_f)
+vmod_fallback_release(VCL_BACKEND dir)
+{
+	struct vmod_directors_fallback *fallback;
+
+	CHECK_OBJ_NOTNULL(dir, DIRECTOR_MAGIC);
+	CAST_OBJ_NOTNULL(fallback, dir->priv, VMOD_DIRECTORS_FALLBACK_MAGIC);
+	vdir_release(fallback->vd);
+}
+
 static void v_matchproto_(vdi_destroy_f)
 vmod_fallback_destroy(VCL_BACKEND dir)
 {
@@ -191,6 +201,7 @@ static const struct vdi_methods vmod_fallback_methods[1] = {{
 	.type =			"fallback",
 	.healthy =		vmod_fallback_healthy,
 	.resolve =		vmod_fallback_resolve,
+	.release =		vmod_fallback_release,
 	.destroy =		vmod_fallback_destroy,
 	.list =			vmod_fallback_list
 }};
diff --git a/vmod/vmod_directors_hash.c b/vmod/vmod_directors_hash.c
index cfa212b1c..fd8d42ab2 100644
--- a/vmod/vmod_directors_hash.c
+++ b/vmod/vmod_directors_hash.c
@@ -45,6 +45,16 @@ struct vmod_directors_hash {
 	struct vdir				*vd;
 };
 
+static void v_matchproto_(vdi_release_f)
+vmod_hash_release(VCL_BACKEND dir)
+{
+	struct vmod_directors_hash *hash;
+
+	CHECK_OBJ_NOTNULL(dir, DIRECTOR_MAGIC);
+	CAST_OBJ_NOTNULL(hash, dir->priv, VMOD_DIRECTORS_HASH_MAGIC);
+	vdir_release(hash->vd);
+}
+
 static void v_matchproto_(vdi_destroy_f)
 vmod_hash_destroy(VCL_BACKEND dir)
 {
@@ -59,6 +69,7 @@ vmod_hash_destroy(VCL_BACKEND dir)
 static const struct vdi_methods vmod_hash_methods[1] = {{
 	.magic =		VDI_METHODS_MAGIC,
 	.type =			"hash",
+	.release =		vmod_hash_release,
 	.destroy =		vmod_hash_destroy
 }};
 
diff --git a/vmod/vmod_directors_random.c b/vmod/vmod_directors_random.c
index f58acd702..3d430609e 100644
--- a/vmod/vmod_directors_random.c
+++ b/vmod/vmod_directors_random.c
@@ -86,6 +86,16 @@ vmod_random_resolve(VRT_CTX, VCL_BACKEND dir)
 	return (be);
 }
 
+static void v_matchproto_(vdi_release_f)
+vmod_random_release(VCL_BACKEND dir)
+{
+	struct vmod_directors_random *random;
+
+	CHECK_OBJ_NOTNULL(dir, DIRECTOR_MAGIC);
+	CAST_OBJ_NOTNULL(random, dir->priv, VMOD_DIRECTORS_RANDOM_MAGIC);
+	vdir_release(random->vd);
+}
+
 static void v_matchproto_(vdi_destroy_f)
 vmod_random_destroy(VCL_BACKEND dir)
 {
@@ -102,6 +112,7 @@ static const struct vdi_methods vmod_random_methods[1] = {{
 	.type =			"random",
 	.healthy =		vmod_random_healthy,
 	.resolve =		vmod_random_resolve,
+	.release =		vmod_random_release,
 	.destroy =		vmod_random_destroy,
 	.list =			vmod_random_list
 }};
diff --git a/vmod/vmod_directors_round_robin.c b/vmod/vmod_directors_round_robin.c
index 1cc6ac82d..015e060da 100644
--- a/vmod/vmod_directors_round_robin.c
+++ b/vmod/vmod_directors_round_robin.c
@@ -96,6 +96,16 @@ vmod_rr_resolve(VRT_CTX, VCL_BACKEND dir)
 	return (be);
 }
 
+static void v_matchproto_(vdi_release_f)
+vmod_rr_release(VCL_BACKEND dir)
+{
+	struct vmod_directors_round_robin *rr;
+
+	CHECK_OBJ_NOTNULL(dir, DIRECTOR_MAGIC);
+	CAST_OBJ_NOTNULL(rr, dir->priv, VMOD_DIRECTORS_ROUND_ROBIN_MAGIC);
+	vdir_release(rr->vd);
+}
+
 static void v_matchproto_(vdi_destroy_f)
 vmod_rr_destroy(VCL_BACKEND dir)
 {
@@ -112,6 +122,7 @@ static const struct vdi_methods vmod_rr_methods[1] = {{
 	.type =			"round-robin",
 	.healthy =		vmod_rr_healthy,
 	.resolve =		vmod_rr_resolve,
+	.release =		vmod_rr_release,
 	.destroy =		vmod_rr_destroy,
 	.list =			vmod_rr_list
 }};
diff --git a/vmod/vmod_directors_shard.c b/vmod/vmod_directors_shard.c
index bdd3e0134..8ef2a4f91 100644
--- a/vmod/vmod_directors_shard.c
+++ b/vmod/vmod_directors_shard.c
@@ -205,6 +205,15 @@ shard__assert(void)
 	assert(t2a == t2b);
 }
 
+static void v_matchproto_(vdi_release_f)
+vmod_shard_release(VCL_BACKEND dir)
+{
+	struct sharddir *shardd;
+
+	CAST_OBJ_NOTNULL(shardd, dir->priv, SHARDDIR_MAGIC);
+	sharddir_release(shardd);
+}
+
 static void v_matchproto_(vdi_destroy_f)
 vmod_shard_destroy(VCL_BACKEND dir)
 {
@@ -219,6 +228,7 @@ static const struct vdi_methods vmod_shard_methods[1] = {{
 	.type =		"shard",
 	.resolve =	vmod_shard_resolve,
 	.healthy =	vmod_shard_healthy,
+	.release =	vmod_shard_release,
 	.destroy =	vmod_shard_destroy,
 	.list =		vmod_shard_list
 }};
diff --git a/vmod/vmod_directors_shard_cfg.c b/vmod/vmod_directors_shard_cfg.c
index 35ecfa9c1..aec4a0901 100644
--- a/vmod/vmod_directors_shard_cfg.c
+++ b/vmod/vmod_directors_shard_cfg.c
@@ -449,7 +449,7 @@ shardcfg_backend_add(struct backend_reconfig *re,
 	bb[i].replicas = replicas;
 }
 
-static void
+void
 shardcfg_backend_clear(struct sharddir *shardd)
 {
 	unsigned i;
@@ -675,10 +675,8 @@ shardcfg_reconfigure(VRT_CTX, struct sharddir *shardd, VCL_INT replicas)
 void
 shardcfg_delete(const struct sharddir *shardd)
 {
-	unsigned i;
 
-	for (i = 0; i < shardd->n_backend; i++)
-		shardcfg_backend_free(&shardd->backend[i]);
+	AZ(shardd->n_backend);
 	if (shardd->backend)
 		free(shardd->backend);
 	if (shardd->hashcircle)
diff --git a/vmod/vmod_directors_shard_dir.c b/vmod/vmod_directors_shard_dir.c
index f587cd19c..cedfcdcd7 100644
--- a/vmod/vmod_directors_shard_dir.c
+++ b/vmod/vmod_directors_shard_dir.c
@@ -204,6 +204,13 @@ sharddir_set_param(struct sharddir *shardd,
 	shardd->param = param;
 }
 
+void
+sharddir_release(struct sharddir *shardd)
+{
+	CHECK_OBJ_NOTNULL(shardd, SHARDDIR_MAGIC);
+	shardcfg_backend_clear(shardd);
+}
+
 void
 sharddir_delete(struct sharddir **sharddp)
 {
diff --git a/vmod/vmod_directors_shard_dir.h b/vmod/vmod_directors_shard_dir.h
index d3cc52998..fdf475144 100644
--- a/vmod/vmod_directors_shard_dir.h
+++ b/vmod/vmod_directors_shard_dir.h
@@ -112,6 +112,7 @@ void sharddir_new(struct sharddir **sharddp, const char *vcl_name,
     const struct vmod_directors_shard_param *param);
 void sharddir_set_param(struct sharddir *shardd,
     const struct vmod_directors_shard_param *param);
+void sharddir_release(struct sharddir *shardd);
 void sharddir_delete(struct sharddir **sharddp);
 void sharddir_rdlock(struct sharddir *shardd);
 void sharddir_wrlock(struct sharddir *shardd);
@@ -121,5 +122,6 @@ VCL_BACKEND sharddir_pick_be(VRT_CTX, struct sharddir *, uint32_t, VCL_INT,
    VCL_REAL, VCL_BOOL, VCL_ENUM healthy);
 
 /* in shard_cfg.c */
+void shardcfg_backend_clear(struct sharddir *shardd);
 void shardcfg_delete(const struct sharddir *shardd);
 VCL_DURATION shardcfg_get_rampup(const struct sharddir *shardd, unsigned host);

From nils.goroll at uplex.de  Mon Mar  6 15:19:07 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon,  6 Mar 2023 15:19:07 +0000 (UTC)
Subject: [master] 6160fb216 Update director developer reference
Message-ID: <20230306151907.7CDF111FA84@lists.varnish-cache.org>


commit 6160fb216290d4f0ea7056789b6ce02d6addfeba
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon Mar 6 16:12:38 2023 +0100

    Update director developer reference

diff --git a/doc/sphinx/reference/directors.rst b/doc/sphinx/reference/directors.rst
index 450515df7..774babf5e 100644
--- a/doc/sphinx/reference/directors.rst
+++ b/doc/sphinx/reference/directors.rst
@@ -49,6 +49,7 @@ code instead::
         vdi_getip_f                     *getip;
         vdi_finish_f                    *finish;
         vdi_event_f                     *event;
+        vdi_release_f                   *release;
         vdi_destroy_f                   *destroy;
         vdi_panic_f                     *panic;
         vdi_list_f                      *list;
@@ -60,6 +61,7 @@ code instead::
         void                            *priv;
         char                            *vcl_name;
         struct vcldir                   *vdir;
+        struct lock                     *mtx;
     };
 
 A director can be summed up as:
@@ -93,10 +95,10 @@ The fundamental steps towards a director implementation are:
 
 - in your destructor or other finalizer, call ``VRT_DelDirector()``
 
-For forwards compatibility, it is strongly recommended for the last
-step not to destroy the actual director private state, but rather
-implement and declare in ``struct vdi_methods`` a ``destroy``
-callback.
+- implement a ``destroy`` callback to destroy the actual director
+  private state. It will be called when all references to the director
+  are gone, until then the private state must remain intact and
+  ``vdi_methods`` functions callable (but they may return errors).
 
 While vmods can implement functions returning directors,
 :ref:`ref-vmod-vcl-c-objects` are usually a more natural
@@ -116,6 +118,10 @@ director. Directors are walked until a leaf director is found. A leaf director
 doesn't have a ``resolve`` function and is used to actually make the backend
 request, just like the backends you declare in VCL.
 
+*load balancing* directors use ``VRT_Assign_Backend()`` to take
+references to other directors. They *must* implement a ``release``
+callback which has to release all references to other directors and
+ensure that none are gained after it returns.
 
 Dynamic Backends
 ================

From nils.goroll at uplex.de  Mon Mar  6 15:30:05 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon,  6 Mar 2023 15:30:05 +0000 (UTC)
Subject: [master] 90227fb09 Update director developer reference on refcounting
Message-ID: <20230306153005.ED0641205AF@lists.varnish-cache.org>


commit 90227fb09e8a6583b97173528e433a9b4b30c2b2
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon Mar 6 16:28:20 2023 +0100

    Update director developer reference on refcounting

diff --git a/doc/sphinx/reference/directors.rst b/doc/sphinx/reference/directors.rst
index 774babf5e..1f5cd5520 100644
--- a/doc/sphinx/reference/directors.rst
+++ b/doc/sphinx/reference/directors.rst
@@ -123,6 +123,14 @@ references to other directors. They *must* implement a ``release``
 callback which has to release all references to other directors and
 ensure that none are gained after it returns.
 
+Static Directors
+================
+
+As opposed to dynamic backends covered below, directors which are
+guaranteed to have VCL lifetime (that is, they do not get destroyed
+before the VCL goes cold) can call ``VRT_StaticDirector()`` to avoid
+reference counting overhead.
+
 Dynamic Backends
 ================
 
@@ -153,9 +161,8 @@ its VCL, it can be deleted any time with ``VRT_delete_backend``. The
 VCL will delete the remaining backends once discarded, you don't need
 to take care of it.
 
-.. XXX this does not quite work yet because the deleted backend could
-   be referenced, but at least that's where we want to get to. See
-   also https://github.com/varnishcache/varnish-cache/pull/2725
+Reference counting is used to ensure that backends which are no longer
+referenced are destroyed.
 
 Finally, Varnish will take care of event propagation for *all* native backends,
 but dynamic backends can only be created when the VCL is warm. If your backends

From dridi.boukelmoune at gmail.com  Mon Mar  6 16:50:07 2023
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon,  6 Mar 2023 16:50:07 +0000 (UTC)
Subject: [master] cf71ad8e9 man: There is no optional argument in vcl.state
Message-ID: <20230306165007.5F9BA7047@lists.varnish-cache.org>


commit cf71ad8e9b985ab0d708ec0e4bfc3ac701da2ec1
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Mar 6 17:48:28 2023 +0100

    man: There is no optional argument in vcl.state
    
    The square brackets were probably the result of copy-pasta from vcl.load
    where the initial state is indeed optional.

diff --git a/include/tbl/cli_cmds.h b/include/tbl/cli_cmds.h
index d62774e3d..b4b9c3194 100644
--- a/include/tbl/cli_cmds.h
+++ b/include/tbl/cli_cmds.h
@@ -91,7 +91,7 @@ CLI_CMD(VCL_INLINE,
 
 CLI_CMD(VCL_STATE,
 	"vcl.state",
-	"vcl.state <configname> [auto|cold|warm]",
+	"vcl.state <configname> auto|cold|warm",
 	"  Force the state of the named configuration.",
 	"",
 	2, 2

From phk at FreeBSD.org  Mon Mar  6 20:40:12 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon,  6 Mar 2023 20:40:12 +0000 (UTC)
Subject: [master] d0efd253d Silence 30 (!) flexelint warnings.
Message-ID: <20230306204012.6D851103344@lists.varnish-cache.org>


commit d0efd253dd559e254c5404e30533dc423ca15917
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Mar 6 20:39:23 2023 +0000

    Silence 30 (!) flexelint warnings.

diff --git a/lib/libvarnishapi/vsl.c b/lib/libvarnishapi/vsl.c
index 5179d4fdf..8c3d33753 100644
--- a/lib/libvarnishapi/vsl.c
+++ b/lib/libvarnishapi/vsl.c
@@ -51,6 +51,8 @@
 
 /*--------------------------------------------------------------------*/
 
+const char			vsl_file_id[] = {'V', 'S', 'L', '2'};
+
 const char * const VSL_tags[SLT__MAX] = {
 #  define SLTM(foo,flags,sdesc,ldesc)       [SLT_##foo] = #foo,
 #  include "tbl/vsl_tags.h"
diff --git a/lib/libvarnishapi/vsl_api.h b/lib/libvarnishapi/vsl_api.h
index 848a02b25..cb5b7ac82 100644
--- a/lib/libvarnishapi/vsl_api.h
+++ b/lib/libvarnishapi/vsl_api.h
@@ -31,7 +31,7 @@
  *
  */
 
-static const char			vsl_file_id[] = {'V', 'S', 'L', '2'};
+extern const char			vsl_file_id[4];
 
 #define VSL_FILE_ID			(vsl_file_id)
 

From phk at FreeBSD.org  Mon Mar  6 21:06:07 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon,  6 Mar 2023 21:06:07 +0000 (UTC)
Subject: [master] 94c4b10d9 Flexelinting: Use 'rr' instead of 'random' which
 overloads random(3)
Message-ID: <20230306210607.EB217103FC1@lists.varnish-cache.org>


commit 94c4b10d9c3823bfd8b197cba4ad57638aa1c80a
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Mar 6 21:04:15 2023 +0000

    Flexelinting:  Use 'rr' instead of 'random' which overloads random(3)

diff --git a/vmod/vmod_directors_random.c b/vmod/vmod_directors_random.c
index 3d430609e..41868b9bb 100644
--- a/vmod/vmod_directors_random.c
+++ b/vmod/vmod_directors_random.c
@@ -89,11 +89,11 @@ vmod_random_resolve(VRT_CTX, VCL_BACKEND dir)
 static void v_matchproto_(vdi_release_f)
 vmod_random_release(VCL_BACKEND dir)
 {
-	struct vmod_directors_random *random;
+	struct vmod_directors_random *rr;
 
 	CHECK_OBJ_NOTNULL(dir, DIRECTOR_MAGIC);
-	CAST_OBJ_NOTNULL(random, dir->priv, VMOD_DIRECTORS_RANDOM_MAGIC);
-	vdir_release(random->vd);
+	CAST_OBJ_NOTNULL(rr, dir->priv, VMOD_DIRECTORS_RANDOM_MAGIC);
+	vdir_release(rr->vd);
 }
 
 static void v_matchproto_(vdi_destroy_f)

From phk at FreeBSD.org  Tue Mar  7 11:35:08 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue,  7 Mar 2023 11:35:08 +0000 (UTC)
Subject: [master] 79718684a Explicitly show that we throw away returnvalue
 (Flexelinting)
Message-ID: <20230307113508.EB84111EAF1@lists.varnish-cache.org>


commit 79718684aa312e6ad5ae8e2523fa3e9395b06916
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Mar 7 10:49:43 2023 +0000

    Explicitly show that we throw away returnvalue (Flexelinting)

diff --git a/bin/varnishd/cache/cache_vrt_vcl.c b/bin/varnishd/cache/cache_vrt_vcl.c
index 3ab464c07..c98274e9d 100644
--- a/bin/varnishd/cache/cache_vrt_vcl.c
+++ b/bin/varnishd/cache/cache_vrt_vcl.c
@@ -312,7 +312,7 @@ VRT_Assign_Backend(VCL_BACKEND *dst, VCL_BACKEND src)
 		vdir = (*dst)->vdir;
 		CHECK_OBJ_NOTNULL(vdir, VCLDIR_MAGIC);
 		if (!(vdir->flags & VDIR_FLG_NOREFCNT))
-			vcldir_deref(vdir);
+			(void)vcldir_deref(vdir);
 	}
 	if (src != NULL) {
 		vdir = src->vdir;

From phk at FreeBSD.org  Tue Mar  7 11:35:08 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue,  7 Mar 2023 11:35:08 +0000 (UTC)
Subject: [master] a02743ccb Make sure complaints from the starting child
 process reach stderr
Message-ID: <20230307113509.03E6211EAF3@lists.varnish-cache.org>


commit a02743ccb051b81da8c71d5a68c9afcc68829b28
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Mar 7 11:33:55 2023 +0000

    Make sure complaints from the starting child process reach stderr

diff --git a/bin/varnishd/mgt/mgt_child.c b/bin/varnishd/mgt/mgt_child.c
index 2776f9de6..0bb82c51c 100644
--- a/bin/varnishd/mgt/mgt_child.c
+++ b/bin/varnishd/mgt/mgt_child.c
@@ -233,12 +233,14 @@ MCH_Fd_Inherit(int fd, const char *what)
  * Listen to stdout+stderr from the child
  */
 
+static const char *whining_child = C_ERR;
+
 static int v_matchproto_(vlu_f)
 child_line(void *priv, const char *p)
 {
 	(void)priv;
 
-	MGT_Complain(C_INFO, "Child (%jd) said %s", (intmax_t)child_pid, p);
+	MGT_Complain(whining_child, "Child (%jd) said %s", (intmax_t)child_pid, p);
 	return (0);
 }
 
@@ -435,6 +437,7 @@ mgt_launch_child(struct cli *cli)
 		assert(u == CLIS_OK);
 		fprintf(stderr, "Child launched OK\n");
 	}
+	whining_child = C_INFO;
 
 	AZ(ev_listen);
 	e = VEV_Alloc();
diff --git a/bin/varnishd/mgt/mgt_util.c b/bin/varnishd/mgt/mgt_util.c
index d9ec5a0f1..f43b43b0d 100644
--- a/bin/varnishd/mgt/mgt_util.c
+++ b/bin/varnishd/mgt/mgt_util.c
@@ -172,7 +172,7 @@ MGT_Complain(const char *loud, const char *fmt, ...)
 	else
 		WRONG("Wrong complaint loudness");
 
-	if (complain_to_stderr && loud != C_CLI)
+	if (loud != C_CLI && (complain_to_stderr || loud != C_DEBUG))
 		fprintf(stderr, "%s %s\n", loud, VSB_data(vsb));
 
 	if (!MGT_DO_DEBUG(DBG_VTC_MODE))

From phk at FreeBSD.org  Tue Mar  7 11:35:09 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue,  7 Mar 2023 11:35:09 +0000 (UTC)
Subject: [master] 191cbe478 More coverage of -sfile argument processing
Message-ID: <20230307113509.2494911EAF9@lists.varnish-cache.org>


commit 191cbe478113165429ed56a98cdc4f606cd63f52
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Mar 7 11:34:12 2023 +0000

    More coverage of -sfile argument processing

diff --git a/bin/varnishtest/tests/s00003.vtc b/bin/varnishtest/tests/s00003.vtc
index 35cc25206..d1077dd93 100644
--- a/bin/varnishtest/tests/s00003.vtc
+++ b/bin/varnishtest/tests/s00003.vtc
@@ -18,6 +18,7 @@ server s1 {
 
 varnish v1 \
 	-arg "-sTransient=file,${tmpdir}/_.file,10m" \
+	-arg "-sdir=file,${tmpdir}/,10m" \
 	-vcl+backend {
 		sub vcl_backend_response {
 			set beresp.do_stream = false;
@@ -39,3 +40,20 @@ client c1 {
 	rxresp
 	expect resp.bodylen == 262
 } -run
+
+varnish v1 -vsl_catchup
+
+varnish v1 -cliok "ban obj.http.date ~ ."
+
+process p1 {
+	varnishd -sTransient=file,${tmpdir}/foo,xxx -blocalhost -a:0 -n ${tmpdir} 2>&1
+} -expect-exit 255 -dump -start -expect-text 0 0 "Invalid number" -wait -screen_dump
+
+process p1 {
+	varnishd -sTransient=file,${tmpdir}/foo,10M,xxx -blocalhost -a:0 -n ${tmpdir} 2>&1
+} -expect-exit 255 -dump -start -expect-text 0 0 "granularity" -wait -screen_dump
+
+process p1 {
+	varnishd -sTransient=file,${tmpdir}/foo,10m,,foo -blocalhost -a:0 -n ${tmpdir} 2>&1
+} -expect-exit 255 -dump -start -expect-text 0 0 "invalid advice" -wait
+

From phk at FreeBSD.org  Tue Mar  7 12:23:05 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue,  7 Mar 2023 12:23:05 +0000 (UTC)
Subject: [master] 4d180a8bc Remove old no longer working test-code.
Message-ID: <20230307122305.3C4311203E7@lists.varnish-cache.org>


commit 4d180a8bc57736290c260155192b2f1f6c5e9a0d
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Mar 7 12:21:57 2023 +0000

    Remove old no longer working test-code.

diff --git a/bin/varnishd/storage/storage_file.c b/bin/varnishd/storage/storage_file.c
index 8c692d106..b5148ac36 100644
--- a/bin/varnishd/storage/storage_file.c
+++ b/bin/varnishd/storage/storage_file.c
@@ -504,65 +504,3 @@ const struct stevedore smf_stevedore = {
 	.allocbuf	=	SML_AllocBuf,
 	.freebuf	=	SML_FreeBuf,
 };
-
-#ifdef INCLUDE_TEST_DRIVER
-
-void vca_flush(struct sess *sp) {}
-
-#define N	100
-#define M	(128*1024)
-
-struct storage *s[N];
-
-static void
-dumpit(void)
-{
-	struct smf_sc *sc = smf_stevedore.priv;
-	struct smf *s;
-
-	return (0);
-	printf("----------------\n");
-	printf("Order:\n");
-	VTAILQ_FOREACH(s, &sc->order, order) {
-		printf("%10p %12ju %12ju %12ju\n",
-		    s, s->offset, s->size, s->offset + s->size);
-	}
-	printf("Used:\n");
-	VTAILQ_FOREACH(s, &sc->used, status) {
-		printf("%10p %12ju %12ju %12ju\n",
-		    s, s->offset, s->size, s->offset + s->size);
-	}
-	printf("Free:\n");
-	VTAILQ_FOREACH(s, &sc->free, status) {
-		printf("%10p %12ju %12ju %12ju\n",
-		    s, s->offset, s->size, s->offset + s->size);
-	}
-	printf("================\n");
-}
-
-int
-main(int argc, char **argv)
-{
-	int i, j;
-
-	setbuf(stdout, NULL);
-	smf_init(&smf_stevedore, "");
-	smf_open(&smf_stevedore);
-	while (1) {
-		dumpit();
-		i = random() % N;
-		do
-			j = random() % M;
-		while (j == 0);
-		if (s[i] == NULL) {
-			s[i] = smf_alloc(&smf_stevedore, j);
-			printf("A %10p %12d\n", s[i], j);
-		} else {
-			smf_free(s[i]);
-			printf("D %10p\n", s[i]);
-			s[i] = NULL;
-		}
-	}
-}
-
-#endif /* INCLUDE_TEST_DRIVER */

From daghf at varnish-software.com  Fri Mar 10 19:06:07 2023
From: daghf at varnish-software.com (Dag Haavi Finstad)
Date: Fri, 10 Mar 2023 19:06:07 +0000 (UTC)
Subject: [master] 22f666f4f h2: Improve pseudo-header handling
Message-ID: <20230310190607.1D523116491@lists.varnish-cache.org>


commit 22f666f4feef841bf3ebd3c0300e80ff09ec9a8b
Author: Dag Haavi Finstad <daghf at varnish-software.com>
Date:   Fri Mar 10 19:59:43 2023 +0100

    h2: Improve pseudo-header handling

diff --git a/bin/varnishd/http2/cache_http2_hpack.c b/bin/varnishd/http2/cache_http2_hpack.c
index 51968317a..0efe5fa66 100644
--- a/bin/varnishd/http2/cache_http2_hpack.c
+++ b/bin/varnishd/http2/cache_http2_hpack.c
@@ -135,6 +135,15 @@ h2h_addhdr(struct http *hp, char *b, size_t namelen, size_t len)
 			n = HTTP_HDR_URL;
 			disallow_empty = 1;
 
+			// rfc7540,l,3060,3071
+			if ((len > 0 && *b != '/') ||
+			    (len > 1 && *(b+1) == '/')) {
+				VSLb(hp->vsl, SLT_BogoHeader,
+				    "Illegal :path pseudo-header %.*s",
+				    (int)len, b);
+				return (H2SE_PROTOCOL_ERROR);
+			}
+
 			/* Second field cannot contain LWS or CTL */
 			for (p = b, u = 0; u < len; p++, u++) {
 				if (vct_islws(*p) || vct_isctl(*p))
diff --git a/bin/varnishtest/tests/a02027.vtc b/bin/varnishtest/tests/a02027.vtc
new file mode 100644
index 000000000..731f72aca
--- /dev/null
+++ b/bin/varnishtest/tests/a02027.vtc
@@ -0,0 +1,29 @@
+varnishtest "Malformed :path handling"
+
+server s1 {
+} -start
+
+varnish v1 -vcl+backend {
+	sub vcl_recv {
+		return (synth(200));
+	}
+} -start
+varnish v1 -cliok "param.set feature +http2"
+
+client c1 {
+	stream 1 {
+		txreq -noadd -hdr ":authority" "foo.com" -hdr ":path" "foobar" -hdr ":scheme" "http" -hdr ":method" "GET"
+		rxrst
+		expect rst.err == PROTOCOL_ERROR
+	} -run
+
+} -run
+
+client c1 {
+	stream 1 {
+		txreq -noadd -hdr ":authority" "foo.com" -hdr ":path" "//foo" -hdr ":scheme" "http" -hdr ":method" "GET"
+		rxrst
+		expect rst.err == PROTOCOL_ERROR
+	} -run
+
+} -run

From daghf at varnish-software.com  Fri Mar 10 19:06:07 2023
From: daghf at varnish-software.com (Dag Haavi Finstad)
Date: Fri, 10 Mar 2023 19:06:07 +0000 (UTC)
Subject: [master] 8d97b6719 h2: Allow :path * for OPTIONS
Message-ID: <20230310190607.31A66116493@lists.varnish-cache.org>


commit 8d97b67195e44d6238c9b2b91f82cbbb1e8fc701
Author: Dag Haavi Finstad <daghf at varnish-software.com>
Date:   Fri Mar 10 19:59:50 2023 +0100

    h2: Allow :path * for OPTIONS

diff --git a/bin/varnishd/http2/cache_http2_hpack.c b/bin/varnishd/http2/cache_http2_hpack.c
index 0efe5fa66..36570a751 100644
--- a/bin/varnishd/http2/cache_http2_hpack.c
+++ b/bin/varnishd/http2/cache_http2_hpack.c
@@ -136,8 +136,9 @@ h2h_addhdr(struct http *hp, char *b, size_t namelen, size_t len)
 			disallow_empty = 1;
 
 			// rfc7540,l,3060,3071
-			if ((len > 0 && *b != '/') ||
-			    (len > 1 && *(b+1) == '/')) {
+			if (((len > 0 && *b != '/') ||
+			    (len > 1 && *(b+1) == '/')) &&
+			    (strncmp(b, "*", len) != 0)) {
 				VSLb(hp->vsl, SLT_BogoHeader,
 				    "Illegal :path pseudo-header %.*s",
 				    (int)len, b);
diff --git a/bin/varnishd/http2/cache_http2_proto.c b/bin/varnishd/http2/cache_http2_proto.c
index fa50c6ec7..c1ced04b6 100644
--- a/bin/varnishd/http2/cache_http2_proto.c
+++ b/bin/varnishd/http2/cache_http2_proto.c
@@ -620,6 +620,13 @@ h2_end_headers(struct worker *wrk, struct h2_sess *h2,
 	}
 	AN(req->http->hd[HTTP_HDR_PROTO].b);
 
+	if (*req->http->hd[HTTP_HDR_URL].b == '*' &&
+	    (Tlen(req->http->hd[HTTP_HDR_METHOD]) != 7 ||
+	    strncmp(req->http->hd[HTTP_HDR_METHOD].b, "OPTIONS", 7))) {
+		VSLb(h2->vsl, SLT_BogoHeader, "Illegal :path pseudo-header");
+		return (H2SE_PROTOCOL_ERROR); //rfc7540,l,3068,3071
+	}
+
 	assert(req->req_step == R_STP_TRANSPORT);
 	VCL_TaskEnter(req->privs);
 	VCL_TaskEnter(req->top->privs);
diff --git a/bin/varnishtest/tests/a02027.vtc b/bin/varnishtest/tests/a02027.vtc
index 731f72aca..ff34b0071 100644
--- a/bin/varnishtest/tests/a02027.vtc
+++ b/bin/varnishtest/tests/a02027.vtc
@@ -27,3 +27,43 @@ client c1 {
 	} -run
 
 } -run
+
+client c1 {
+	stream 3 {
+		txreq -noadd -hdr ":authority" "foo.com" -hdr ":path" "*a" -hdr ":scheme" "http" -hdr ":method" "GET"
+		rxrst
+		expect rst.err == PROTOCOL_ERROR
+	} -run
+} -run
+
+client c1 {
+	stream 1 {
+		txreq -noadd -hdr ":authority" "foo.com" -hdr ":path" "*" -hdr ":scheme" "http" -hdr ":method" "GET"
+		rxrst
+		expect rst.err == PROTOCOL_ERROR
+	} -run
+} -run
+
+client c1 {
+	stream 1 {
+		txreq -noadd -hdr ":authority" "foo.com" -hdr ":path" "*" -hdr ":scheme" "http" -hdr ":method" "OPTIONS"
+		rxresp
+		expect resp.status == 200
+	} -run
+} -run
+
+client c1 {
+	stream 1 {
+		txreq -noadd -hdr ":authority" "foo.com" -hdr ":path" "*" -hdr ":scheme" "http" -hdr ":method" "OPTIONs"
+		rxrst
+		expect rst.err == PROTOCOL_ERROR
+	} -run
+} -run
+
+client c1 {
+	stream 1 {
+		txreq -noadd -hdr ":authority" "foo.com" -hdr ":path" "*" -hdr ":scheme" "http" -hdr ":method" "OPTIONSx"
+		rxrst
+		expect rst.err == PROTOCOL_ERROR
+	} -run
+} -run

From phk at FreeBSD.org  Mon Mar 13 08:42:09 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 13 Mar 2023 08:42:09 +0000 (UTC)
Subject: [master] ad0279a49 Fix insignificant memory leak spotted by Coverity
Message-ID: <20230313084209.F40291137D0@lists.varnish-cache.org>


commit ad0279a494f0efb4c47e697053a374bb662c79dd
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Mar 13 08:41:34 2023 +0000

    Fix insignificant memory leak spotted by Coverity

diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index 698a6bbe6..c1bdaf812 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -82,9 +82,10 @@ static const char opt_spec[] = "?a:b:CdE:f:Fh:i:I:j:l:M:n:P:p:r:S:s:T:t:VW:x:";
 static void
 usage(void)
 {
-	char buf[1024];
+	char *p;
 
-#define FMT "  %-28s # %s\n"
+#define FMT_NONL "  %-28s # %s"
+#define FMT FMT_NONL "\n"
 
 	printf( "Usage: varnishd [options]\n");
 
@@ -104,8 +105,12 @@ usage(void)
 	printf(FMT, "", "  default: \":80\"");
 	printf(FMT, "-f vclfile", "VCL program");
 	printf(FMT, "", "Can be specified multiple times.");
-	bprintf(buf, "Working directory (default: %s)", VIN_n_Arg(NULL));
-	printf(FMT, "-n dir", buf);
+	printf(FMT, "-n dir", "Working directory");
+
+	p = VIN_n_Arg(NULL);
+	AN(p);
+	printf(FMT_NONL "  default: %s\n", "", "", p);
+	free(p);
 
 	printf("\n-b can be used only once, and not together with -f\n");
 

From nils.goroll at uplex.de  Tue Mar 14 12:47:12 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 14 Mar 2023 12:47:12 +0000 (UTC)
Subject: [master] 365dd9fcd Changelog additions
Message-ID: <20230314124712.5AA4812048A@lists.varnish-cache.org>


commit 365dd9fcdf20c8f0f1785eb761c9af8f8734f59f
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue Mar 14 12:46:26 2023 +0100

    Changelog additions

diff --git a/doc/changes.rst b/doc/changes.rst
index d53f67986..7dd962007 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -54,6 +54,37 @@ Varnish Cache NEXT (2023-03-15)
   their status might be, and not even a partial delivery of them
   will fail the parent ESI request.  To be used with great caution.
 
+* The macro ``WS_TASK_ALLOC_OBJ`` as been added to handle the common
+  case of allocating mini objects on a workspace.
+
+* ``xid`` variables in VCL are now of type ``INT``.
+
+* The new ``beresp.transit_buffer`` variable has been added to VCL,
+  which defaults to the newly added parameter ``transit_buffer``. This
+  variable limits the number of bytes varnish pre-fetches for
+  uncacheable streaming fetches.
+
+* Varnish now supports abstract unix domain sockets. If the operating
+  system supports them, abstract sockets can be specified using the
+  commonplace ``@`` notation for accept sockets, e.g.::
+
+    varnishd -a @kandinsky
+
+  and backend paths, e.g.::
+
+    backend miro {
+      .path = "@miro";
+    }
+
+* For backend requests, the timestamp from the ``Last-Modified``
+  response header is now only used to create an ``If-Modified-Since``
+  conditional ``GET`` request if it is at least one second older than
+  the timestamp from the ``Date`` header.
+
+* Various interfaces of varnish's own socket address abstraction, VSA,
+  have been changed to return or take pointers to
+  ``const``. ``VSA_free()`` has been added.
+
 * VXIDs are 64 bit now and the binary format of SHM and raw saved
   VSL files has changed as a consequence.
 
@@ -78,11 +109,28 @@ Varnish Cache NEXT (2023-03-15)
 
 * Unused log tags (SLTs) have been removed.
 
+.. `RFC9110`_ : https://httpwg.org/specs/rfc9110.html#field.content-range
+
+* Processing of Range requests has been improved: Previously, varnish
+  would send a 200 response with the full body when it could not
+  reliably determine (yet) the object size during streaming.
+
+  Now a 206 response is sent even in this case (for HTTP/1.1 as
+  chunked encoding) with ``*`` in place of the ``complete-length`` as
+  per `RFC9110`_.
+
 * The ``debug.xid`` CLI command now sets the next XID to be used,
   rather than "one less than the next XID to be used"
 
 * Backend implementations are in charge of logging their headers.
 
+* VCL backend ``probe``\ s gained an ``.expect_close`` boolean
+  attribute. By setting to to ``false``, backends which fail to honor
+  ``Connection: close`` can be probed.
+
+  Notice that the probe ``.timeout`` needs to be reached for a probe
+  with ``.expect_close = false`` to return.
+
 * Support for backend connections through a proxy with a PROXY2
   preamble has been added:
 
@@ -103,6 +151,9 @@ Varnish Cache NEXT (2023-03-15)
   Failure to implement this callback can result in deadlocks, in
   particular during VCL discard.
 
+* Handling of the HTTP/2 :path pseudo header has been improved.
+
+
 ================================
 Varnish Cache 7.2.0 (2022-09-15)
 ================================

From nils.goroll at uplex.de  Tue Mar 14 12:47:12 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 14 Mar 2023 12:47:12 +0000 (UTC)
Subject: [master] b36a9a37a Improve transit_buffer documentation
Message-ID: <20230314124712.43FD9120488@lists.varnish-cache.org>


commit b36a9a37aaafef2c421000ecbe5cf5aaa5e9a239
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue Mar 14 13:16:30 2023 +0100

    Improve transit_buffer documentation
    
    - cross-reference parameter and vcl variable
    
    - clarify that a zero value means no limit (I found 'no buffering' confusing when
      in fact the full object is buffered, potentially)
    
    - improve wording, hopefully

diff --git a/doc/sphinx/reference/vcl_var.rst b/doc/sphinx/reference/vcl_var.rst
index 41bbe98db..49ec33f66 100644
--- a/doc/sphinx/reference/vcl_var.rst
+++ b/doc/sphinx/reference/vcl_var.rst
@@ -1067,8 +1067,12 @@ beresp.transit_buffer
 
 	Writable from: vcl_backend_response
 
+	Default: ``transit_buffer`` parameter, see :ref:`varnishd(1)`.
+
 	The maximum number of bytes the client can be ahead of the
-	backend during a streaming pass if ``beresp`` is uncacheable.
+	backend during a streaming pass if ``beresp`` is
+	uncacheable. See also ``transit_buffer`` parameter
+	documentation in :ref:`varnishd(1)`.
 
 
 beresp.ttl
diff --git a/include/tbl/params.h b/include/tbl/params.h
index 5ed5d7f86..cf46b1658 100644
--- a/include/tbl/params.h
+++ b/include/tbl/params.h
@@ -960,13 +960,21 @@ PARAM_SIMPLE(
 	/* def */	"0k",
 	/* units */	"bytes",
 	/* descr */
-	"The amount of buffer allowed through Varnish for uncacheable "
-	"backend streaming fetches. Zero means that no buffering is "
-	"done and the object is fetched as fast as possible.\n\n"
-	"When a client is slow, this can prevent large uncacheable objects "
-	"from being stored indefinitely when the intent is to simply stream "
-	"them to the client. As a result, a slow client transaction holds "
-	"onto a backend connection until the end of the delivery."
+
+	"The number of bytes which Varnish buffers for uncacheable "
+	"backend streaming fetches - in other words, how many bytes "
+	"Varnish reads from the backend ahead of what has been sent to "
+	"the client.\n"
+	"A zero value means no limit, the object is fetched as fast as "
+	"possible.\n\n"
+	"When dealing with slow clients, setting this parameter to "
+	"non-zero can prevent large uncacheable objects from being "
+	"stored in full when the intent is to simply stream them to the "
+	"client. As a result, a slow client transaction holds onto a "
+	"backend connection until the end of the delivery.\n\n"
+	"This parameter is the default to the VCL variable "
+	"``beresp.transit_buffer``, which can be used to control the "
+	"transit buffer per backend request."
 )
 
 PARAM_SIMPLE(

From nils.goroll at uplex.de  Tue Mar 14 12:47:12 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 14 Mar 2023 12:47:12 +0000 (UTC)
Subject: [master] f5cdf0993 Reorder changes.rst to commit order (approximately)
Message-ID: <20230314124712.7A48B12048D@lists.varnish-cache.org>


commit f5cdf0993a51af74f4cffa523d74d9fa5b22fd46
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue Mar 14 13:45:46 2023 +0100

    Reorder changes.rst to commit order (approximately)
    
    At least for me, this makes it easier to check if we missed anything.

diff --git a/doc/changes.rst b/doc/changes.rst
index 7dd962007..2d053ec66 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -35,25 +35,6 @@ release process.
 Varnish Cache NEXT (2023-03-15)
 ===============================
 
-* Do not ESI:include failed objects unless instructed to.
-
-  Previously, any ESI:include object would be included, no matter
-  what the status of it were, 200, 503, didn't matter.
-
-  From now on, by default, only objects with 200 and 204 status
-  will be included and any other status code will fail the parent
-  ESI request.
-
-  If objects with other status should be delivered, they should
-  have their status changed to 200 in VCL, for instance in
-  ``sub vcl_backend_error{}``, ``vcl_synth{}`` or ``vcl_deliver{}``.
-
-  If ``param.set feature +esi_include_onerror`` is used, and the
-  ``<esi:include ?>`` tag has a ``onerror="continue"`` attribute,
-  any and all ESI:include objects will be delivered, no matter what
-  their status might be, and not even a partial delivery of them
-  will fail the parent ESI request.  To be used with great caution.
-
 * The macro ``WS_TASK_ALLOC_OBJ`` as been added to handle the common
   case of allocating mini objects on a workspace.
 
@@ -85,6 +66,19 @@ Varnish Cache NEXT (2023-03-15)
   have been changed to return or take pointers to
   ``const``. ``VSA_free()`` has been added.
 
+* Processing of Range requests has been improved: Previously, varnish
+  would send a 200 response with the full body when it could not
+  reliably determine (yet) the object size during streaming.
+
+.. `RFC9110`_ : https://httpwg.org/specs/rfc9110.html#field.content-range
+
+  Now a 206 response is sent even in this case (for HTTP/1.1 as
+  chunked encoding) with ``*`` in place of the ``complete-length`` as
+  per `RFC9110`_.
+
+* The ``debug.xid`` CLI command now sets the next XID to be used,
+  rather than "one less than the next XID to be used"
+
 * VXIDs are 64 bit now and the binary format of SHM and raw saved
   VSL files has changed as a consequence.
 
@@ -107,20 +101,24 @@ Varnish Cache NEXT (2023-03-15)
   incompatible with logs and in-memory data written by previous
   versions, and vice versa.
 
-* Unused log tags (SLTs) have been removed.
+* Do not ESI:include failed objects unless instructed to.
 
-.. `RFC9110`_ : https://httpwg.org/specs/rfc9110.html#field.content-range
+  Previously, any ESI:include object would be included, no matter
+  what the status of it were, 200, 503, didn't matter.
 
-* Processing of Range requests has been improved: Previously, varnish
-  would send a 200 response with the full body when it could not
-  reliably determine (yet) the object size during streaming.
+  From now on, by default, only objects with 200 and 204 status
+  will be included and any other status code will fail the parent
+  ESI request.
 
-  Now a 206 response is sent even in this case (for HTTP/1.1 as
-  chunked encoding) with ``*`` in place of the ``complete-length`` as
-  per `RFC9110`_.
+  If objects with other status should be delivered, they should
+  have their status changed to 200 in VCL, for instance in
+  ``sub vcl_backend_error{}``, ``vcl_synth{}`` or ``vcl_deliver{}``.
 
-* The ``debug.xid`` CLI command now sets the next XID to be used,
-  rather than "one less than the next XID to be used"
+  If ``param.set feature +esi_include_onerror`` is used, and the
+  ``<esi:include ?>`` tag has a ``onerror="continue"`` attribute,
+  any and all ESI:include objects will be delivered, no matter what
+  their status might be, and not even a partial delivery of them
+  will fail the parent ESI request.  To be used with great caution.
 
 * Backend implementations are in charge of logging their headers.
 
@@ -139,6 +137,8 @@ Varnish Cache NEXT (2023-03-15)
   * The ``VRT_new_backend_clustered()`` and ``VRT_new_backend()``
     signatures have been changed
 
+* Unused log tags (SLTs) have been removed.
+
 * Directors which take and hold references to other directors via
   ``VRT_Assign_Backend()`` (typically any director which has other
   directors as backends) are now expected to implement the new
@@ -153,7 +153,6 @@ Varnish Cache NEXT (2023-03-15)
 
 * Handling of the HTTP/2 :path pseudo header has been improved.
 
-
 ================================
 Varnish Cache 7.2.0 (2022-09-15)
 ================================

From nils.goroll at uplex.de  Tue Mar 14 13:18:07 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 14 Mar 2023 13:18:07 +0000 (UTC)
Subject: [master] f247a9791 Polish release docs
Message-ID: <20230314131807.B97B26801@lists.varnish-cache.org>


commit f247a97918f38d1e38e675709edcb672d64518dd
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue Mar 14 14:14:01 2023 +0100

    Polish release docs
    
    - I think the excellent ESI processing changes documentation should
      be promoted from changes.rst. I think it is more helpful to
      users planning their upgrade, because it gives better advice.
    
    - I thought we could be a little more verbose on abstract UDSes
      and the weak Last-Modified changes.
    
    - The director .release callback was not yet mentioned
    
    All other changes are not worth mentioning I guess.

diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-trunk.rst
index fc1e56cdb..8b1887150 100644
--- a/doc/sphinx/whats-new/changes-trunk.rst
+++ b/doc/sphinx/whats-new/changes-trunk.rst
@@ -32,26 +32,48 @@ from filling up storage too fast at the expense of cacheable resources. When
 transit buffer is enabled, a client request will effectively hold its backend
 connection open until the client response delivery completes.
 
+ESI processing changes
+----------------------
+
+Response status codes other than 200 and 204 are now considered errors for ESI
+fragments.
+
+Previously, any ``ESI:include`` object would be included, no matter what
+the status of it were, 200, 503, didn't matter.
+
+From now on, by default, only objects with 200 and 204 status will be
+included and any other status code will fail the parent ESI request.
+
+If objects with other status should be delivered, they should have
+their status changed to 200 in VCL, for instance in ``sub
+vcl_backend_error{}``, ``vcl_synth{}`` or ``vcl_deliver{}``.
+
+If ``param.set feature +esi_include_onerror`` is used, and the
+``<esi:include ?>`` tag has a ``onerror="continue"`` attribute, any
+and all ESI:include objects will be delivered, no matter what their
+status might be, and not even a partial delivery of them will fail the
+parent ESI request.  To be used with great caution.
+
+
 Other changes in varnishd
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 
-In addition to classic Unix-domain sockets, abstract sockets can now be used
-on Linux. Instead of an absolute path, the syntax ``-a @name`` can be used to
-bind the abstract socket called ``name``.
+In addition to classic Unix-domain sockets, Varnish now supports
+abstract sockets. If the operating system supports them, as does any
+fairly recent Linux kernel, abstract sockets can be specified using
+the commonplace ``@`` notation for accept sockets, e.g.::
+
+    varnishd -a @kandinsky
 
-Weak ``Last-Modified`` headers are no longer candidates for revalidation. This
-means that a subsequent fetch will not, when such a stale object is available,
-include an ``If-Modified-Since`` header. A weak ``Last-Modified`` header does
-not prevent ``Etag`` revalidation.
+Weak ``Last-Modified`` headers whose timestamp lies within one second
+of the corresponding ``Date`` header are no longer candidates for
+revalidation. This means that a subsequent fetch will not, when a
+stale object is available, include an ``If-Modified-Since`` header. A
+weak ``Last-Modified`` header does not prevent ``Etag`` revalidation.
 
 A cache hit on an object being streamed no longer prevents delivery of partial
 responses (status code 206) to range requests.
 
-Response status codes other than 200 and 204 are now considered errors for ESI
-fragments. The default behavior was changed, errors are no longer delivered by
-default. The feature flag ``esi_include_onerror`` can be raised to allow a
-backend to specify whether to continue.
-
 Changes to VCL
 ==============
 
@@ -70,7 +92,7 @@ variable.
 Other changes to VCL
 ~~~~~~~~~~~~~~~~~~~~
 
-Backends have a new ``.via`` attribute referencing another backend::
+Backends have a new ``.via`` attribute optionally referencing another backend::
 
     backend detour {
         .host = "...";
@@ -88,8 +110,8 @@ define an authority TLV in the PROXYv2 header.
 
 Backends can connect to abstract sockets on linux::
 
-    backend abstract {
-        .path = "@name";
+    backend miro {
+      .path = "@miro";
     }
 
 This is the same syntax as the ``varnishd -a`` command line option.
@@ -108,10 +130,11 @@ varnishlog
 
 **XXX changes concerning varnishlog(1) and/or vsl(7)**
 
-The in-memory and on-disk format of VSL records changed to allow 64bit VXID
-numbers. The new binary format is not compatible with previous versions, and
-log dumps performed with a previous Varnish release are no longer readable
-from now on. Consequently, unused log tags have been removed.
+The in-memory and on-disk format of VSL records changed to allow 64bit
+VXID numbers. The new binary format is **not compatible** with
+previous versions, and log dumps performed with a previous Varnish
+release are no longer readable from now on. Consequently, unused log
+tags have been removed.
 
 The VXID range is limited to ``VRT_INTEGER`` to fit in VCL the variables
 ``req.xid``, ``bereq.xid`` and ``sess.xid``.
@@ -124,10 +147,8 @@ varnishadm
 
 **XXX changes concerning varnishadm(1) and/or varnish-cli(7)**
 
-The ``debug.xid`` command generally used by ``varnishtest`` used to set up the
-current VXID. As the intent usually is to set up the next VXID, this forced to
-set an off-by-one value. To simplify its usage it now sets up the next VXID
-directly.
+The ``debug.xid`` command generally used by ``varnishtest`` now sets
+up the next VXID directly.
 
 varnishstat
 ===========
diff --git a/doc/sphinx/whats-new/upgrading-trunk.rst b/doc/sphinx/whats-new/upgrading-trunk.rst
index 6a2f56cff..98b5b92d2 100644
--- a/doc/sphinx/whats-new/upgrading-trunk.rst
+++ b/doc/sphinx/whats-new/upgrading-trunk.rst
@@ -43,24 +43,45 @@ binary logs need to remain usable, the only solution is to use a compatible
 Varnish version and at the time of this release, the 6.0 branch is the only
 one without an EOL date.
 
+For developers and VMOD authors: C interface changes requiring adjustments
+==========================================================================
+
 Via backends
-============
+------------
 
 The new backend argument to the ``VRT_new_backend*()`` functions is optional
 and ``NULL`` can be passed to match the previous behavior.
 
 suckaddr
-========
+--------
 
 The following functions return or accept ``const`` pointers from now on:
 
 - ``VSA_Clone()``
 - ``VSA_getsockname()``
 - ``VSA_getpeername()``
-- ``VSA_free()``
 - ``VSA_Malloc()``
 - ``VSA_Build*()``
 - ``VSS_ResolveOne()``
 - ``VSS_ResolveFirst()``
 
+``VSA_free()`` has been added to free heap memory allocated by
+``VSA_Malloc()`` or one of the ``VSA_Build*()`` functions with a
+``NULL`` first argument.
+
+directors
+---------
+
+Directors which take and hold references to other directors via
+``VRT_Assign_Backend()`` (typically any director which has other
+directors as backends) are now expected to implement the new
+``.release`` callback of type ``void
+vdi_release_f(VCL_BACKEND)``. This function is called by
+``VRT_DelDirector()``. The implementation is expected drop any backend
+references which the director holds (again using
+``VRT_Assign_Backend()`` with ``NULL`` as the second argument).
+
+Failure to implement this callback can result in deadlocks, in
+particular during VCL discard.
+
 *eof*

From simon.stridsberg at varnish-software.com  Wed Mar 15 14:58:06 2023
From: simon.stridsberg at varnish-software.com (Simon Stridsberg)
Date: Wed, 15 Mar 2023 14:58:06 +0000 (UTC)
Subject: [master] d8b1eaa2d Merge tag 'varnish-7.3.0'
Message-ID: <20230315145806.A964B103AB9@lists.varnish-cache.org>


commit d8b1eaa2df5aa97ff48aac12728c7ba21b3193cf
Merge: f247a9791 84d79120b
Author: Simon Stridsberg <simon.stridsberg at varnish-software.com>
Date:   Wed Mar 15 15:56:51 2023 +0100

    Merge tag 'varnish-7.3.0'
    
    Releasing 7.3.0

From simon.stridsberg at varnish-software.com  Thu Mar 16 05:57:08 2023
From: simon.stridsberg at varnish-software.com (Simon Stridsberg)
Date: Thu, 16 Mar 2023 05:57:08 +0000 (UTC)
Subject: [7.3] a817c266f Switch to the 7.3 branch of pkg-varnish-cache
Message-ID: <20230316055708.CD74D634B9@lists.varnish-cache.org>


commit a817c266f074147ba75e29c84051acb631d058cd
Author: Simon Stridsberg <simon.stridsberg at varnish-software.com>
Date:   Thu Mar 16 06:55:30 2023 +0100

    Switch to the 7.3 branch of pkg-varnish-cache

diff --git a/.circleci/config.yml b/.circleci/config.yml
index accac691a..1dde7243e 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -6,7 +6,7 @@ parameters:
     default: "HEAD"
   pkg-commit:
     type: string
-    default: "master"
+    default: "7.3"
   dist-url:
     type: string
     default: ""

From nils.goroll at uplex.de  Thu Mar 16 08:44:06 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu, 16 Mar 2023 08:44:06 +0000 (UTC)
Subject: [master] 4c525af65 Start NEXT sections in vrt.h and changes.rst
Message-ID: <20230316084406.8993D102772@lists.varnish-cache.org>


commit 4c525af652015339eb846165b044e9b6d737b444
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu Mar 16 09:41:13 2023 +0100

    Start NEXT sections in vrt.h and changes.rst
    
    to anchor patches

diff --git a/doc/changes.rst b/doc/changes.rst
index 08e53512e..e043ac7a3 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -31,6 +31,10 @@ http://varnish-cache.org/docs/trunk/whats-new/index.html and via
 individual releases. These documents are updated as part of the
 release process.
 
+===============================
+Varnish Cache NEXT (2023-09-15)
+===============================
+
 ================================
 Varnish Cache 7.3.0 (2023-03-15)
 ================================
diff --git a/include/vrt.h b/include/vrt.h
index f250ea9fc..4f922d265 100644
--- a/include/vrt.h
+++ b/include/vrt.h
@@ -57,6 +57,7 @@
  * Whenever something is deleted or changed in a way which is not
  * binary/load-time compatible, increment MAJOR version
  *
+ * NEXT (2023-09-15)
  * 17.0 (2023-03-15)
  *	VXID is 64 bit
  *	[cache.h] http_GetRange() changed

From nils.goroll at uplex.de  Thu Mar 16 16:37:09 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu, 16 Mar 2023 16:37:09 +0000 (UTC)
Subject: [master] 9f101cff6 Fix reference to debug.srandom
Message-ID: <20230316163709.B95D211346F@lists.varnish-cache.org>


commit 9f101cff6be08ccbd370bf4ae299ec3df1763d3b
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu Mar 16 17:33:50 2023 +0100

    Fix reference to debug.srandom
    
    m00002.vtc has long been renamed to vmod/tests/std_b00001.vtc,
    so we could also reference that test case, but I guess a debug
    CLI command should be warning enough...

diff --git a/vmod/vmod_std.vcc b/vmod/vmod_std.vcc
index cef9590c4..04130136d 100644
--- a/vmod/vmod_std.vcc
+++ b/vmod/vmod_std.vcc
@@ -48,8 +48,9 @@ $Function REAL random(REAL lo, REAL hi)
 Returns a random real number between *lo* and *hi*.
 
 This function uses the "testable" random generator in varnishd which
-enables deterministic tests to be run (See ``m00002.vtc``).  This
-function should not be used for cryptographic applications.
+enables deterministic tests to be run (See ``debug.srandom`` CLI
+command).  This function should not be used for cryptographic
+applications.
 
 Example::
 

From nils.goroll at uplex.de  Mon Mar 20 11:01:06 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 20 Mar 2023 11:01:06 +0000 (UTC)
Subject: [master] 50e03b07f fix bug in abstract sockets
Message-ID: <20230320110106.A623A11E696@lists.varnish-cache.org>


commit 50e03b07fd569ee331566863cce9316df8b526bc
Author: Walid Boudebouda <walid.boudebouda at gmail.com>
Date:   Thu Mar 16 18:32:35 2023 +0100

    fix bug in abstract sockets
    
    with uds abstract sockets, sun_path should start with a NULL character followed
    by the socket's name. The name is not considered to be NULL terminated and can
    contain NULL bytes which have no special meaning. socklen is used to determine
    the length of name and must be set to the length of the struct sockaddr_un up to
    the last character of name, otherwise the 108 characters of sun_path will be
    treated as the name of the socket, including NULL bytes.

diff --git a/bin/varnishtest/vtc_client.c b/bin/varnishtest/vtc_client.c
index fb7277d5c..8fb3bdd3e 100644
--- a/bin/varnishtest/vtc_client.c
+++ b/bin/varnishtest/vtc_client.c
@@ -127,7 +127,9 @@ uds_open(void *priv, const struct sockaddr_un *uds)
 	double *p;
 	int s, i, tmo;
 	struct pollfd fds[1];
-	socklen_t sl = sizeof(*uds);
+	socklen_t sl;
+
+	sl = VUS_socklen(uds);
 
 	AN(priv);
 	AN(uds);
diff --git a/include/vus.h b/include/vus.h
index 20d56fd15..c818f3939 100644
--- a/include/vus.h
+++ b/include/vus.h
@@ -36,6 +36,7 @@ int VUS_resolver(const char *path, vus_resolved_f *func, void *priv,
 		 const char **err);
 int VUS_bind(const struct sockaddr_un *uds, const char **errp);
 int VUS_connect(const char *path, int msec);
+unsigned int VUS_socklen(const struct sockaddr_un *uds);
 
 static inline int
 VUS_is(const char *path)
diff --git a/lib/libvarnish/vus.c b/lib/libvarnish/vus.c
index c9920ef28..a110a1ec1 100644
--- a/lib/libvarnish/vus.c
+++ b/lib/libvarnish/vus.c
@@ -86,6 +86,8 @@ VUS_resolver(const char *path, vus_resolved_f *func, void *priv,
 	if (ret)
 		return (ret);
 
+	assert(uds.sun_path[1] != '\0');
+
 	if (func != NULL)
 		ret = func(priv, &uds);
 	return (ret);
@@ -95,7 +97,9 @@ int
 VUS_bind(const struct sockaddr_un *uds, const char **errp)
 {
 	int sd, e;
-	socklen_t sl = sizeof(*uds);
+	socklen_t sl;
+
+	sl = VUS_socklen(uds);
 
 	if (errp != NULL)
 		*errp = NULL;
@@ -133,13 +137,18 @@ VUS_connect(const char *path, int msec)
 	int s, i;
 	struct pollfd fds[1];
 	struct sockaddr_un uds;
-	socklen_t sl = (socklen_t) sizeof(uds);
+	socklen_t sl;
 
 	if (path == NULL)
 		return (-1);
 	i = sun_init(&uds, path, NULL);
 	if (i)
 		return (i);
+
+	assert(uds.sun_path[1] != '\0');
+
+	sl = VUS_socklen(&uds);
+
 	AN(sl);
 
 	s = socket(PF_UNIX, SOCK_STREAM, 0);
@@ -182,3 +191,19 @@ VUS_connect(const char *path, int msec)
 
 	return (VTCP_connected(s));
 }
+
+socklen_t
+VUS_socklen(const struct sockaddr_un *uds)
+{
+	socklen_t sl;
+	char *p;
+	if (*uds->sun_path)
+		sl = sizeof(*uds);
+	else {
+		p = strchr(uds->sun_path + 1, '\0');
+		assert(p != NULL);
+		sl = p - (const char*)uds;
+	}
+	assert(sl <= sizeof(*uds));
+	return sl;
+}

From nils.goroll at uplex.de  Mon Mar 20 11:02:06 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 20 Mar 2023 11:02:06 +0000 (UTC)
Subject: [master] 305d47d18 Constify (from flexelint)
Message-ID: <20230320110206.DF9BA11E86D@lists.varnish-cache.org>


commit 305d47d1803759682b51510652b3705720079ddb
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon Mar 20 11:58:51 2023 +0100

    Constify (from flexelint)

diff --git a/lib/libvarnish/vus.c b/lib/libvarnish/vus.c
index a110a1ec1..01187442f 100644
--- a/lib/libvarnish/vus.c
+++ b/lib/libvarnish/vus.c
@@ -196,7 +196,7 @@ socklen_t
 VUS_socklen(const struct sockaddr_un *uds)
 {
 	socklen_t sl;
-	char *p;
+	const char *p;
 	if (*uds->sun_path)
 		sl = sizeof(*uds);
 	else {

From nils.goroll at uplex.de  Mon Mar 20 11:07:05 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 20 Mar 2023 11:07:05 +0000 (UTC)
Subject: [master] 2475ad6c6 Changelog
Message-ID: <20230320110705.7628E11EC81@lists.varnish-cache.org>


commit 2475ad6c6afd2a23ad4a79a98f005478cceec2bb
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon Mar 20 12:05:05 2023 +0100

    Changelog

diff --git a/doc/changes.rst b/doc/changes.rst
index e043ac7a3..aef12cd4f 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -35,6 +35,11 @@ release process.
 Varnish Cache NEXT (2023-09-15)
 ===============================
 
+.. _3908: https://github.com/varnishcache/varnish-cache/pull/3908
+
+* The newly introduced abstract socket support was incompatible with
+  other implementations, this has been fixed (3908_).
+
 ================================
 Varnish Cache 7.3.0 (2023-03-15)
 ================================

From phk at FreeBSD.org  Wed Mar 22 12:39:10 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 22 Mar 2023 12:39:10 +0000 (UTC)
Subject: [master] 36f755211 Skip if HAproxy is version 1
Message-ID: <20230322123910.A45F7105CA1@lists.varnish-cache.org>


commit 36f755211599a8b1d8a68342979336065a2f4fd1
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Mar 22 12:38:11 2023 +0000

    Skip if HAproxy is version 1

diff --git a/bin/varnishtest/tests/c00114.vtc b/bin/varnishtest/tests/c00114.vtc
index 28d691642..85f73a369 100644
--- a/bin/varnishtest/tests/c00114.vtc
+++ b/bin/varnishtest/tests/c00114.vtc
@@ -14,6 +14,9 @@ feature ignore_unknown_macro
 
 feature cmd {haproxy --version 2>&1 | grep -q 'HA-*Proxy version'}
 
+# not sure which haproxy versions work, but 1.0 certainly do not.
+feature cmd "haproxy --version 2>&1 | grep 'HAProxy version [^1][.]'"
+
 server s1 {
 	rxreq
 	txresp -hdr "Foo: bar"

From nils.goroll at uplex.de  Sun Mar 26 11:09:10 2023
From: nils.goroll at uplex.de (Nils Goroll)
Date: Sun, 26 Mar 2023 11:09:10 +0000 (UTC)
Subject: [master] e62c75b3c Support macro values up to 2KB
Message-ID: <20230326110910.6221111A69B@lists.varnish-cache.org>


commit e62c75b3c30b6e209338f58f74c8b10a33a91731
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Sat Mar 25 23:09:42 2023 +0100

    Support macro values up to 2KB
    
    URLs are getting longer and longer...
    
    ref: da4554344757f4b080c7be092b56963b42048522

diff --git a/bin/varnishtest/vtc.c b/bin/varnishtest/vtc.c
index 06982c5bf..f4ffeedcc 100644
--- a/bin/varnishtest/vtc.c
+++ b/bin/varnishtest/vtc.c
@@ -102,7 +102,7 @@ static struct macro *
 macro_def_int(const char *name, macro_f *func, const char *fmt, va_list ap)
 {
 	struct macro *m;
-	char buf[512];
+	char buf[2048];
 
 	VTAILQ_FOREACH(m, &macro_list, list)
 		if (!strcmp(name, m->name))

From phk at FreeBSD.org  Wed Mar 29 08:59:08 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 29 Mar 2023 08:59:08 +0000 (UTC)
Subject: [master] 3f0e12add Add missing dependencies on vsctool.py
Message-ID: <20230329085908.9DF0F103511@lists.varnish-cache.org>


commit 3f0e12addf952702b6937f82c931c557ea73078a
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Mar 29 07:28:23 2023 +0000

    Add missing dependencies on vsctool.py

diff --git a/vmod/Makefile.am b/vmod/Makefile.am
index 29ffcf6bd..197508b28 100644
--- a/vmod/Makefile.am
+++ b/vmod/Makefile.am
@@ -33,10 +33,10 @@ include $(srcdir)/automake_boilerplate_vtc.am
 # Post-boilerplate tweaks
 #
 
-.vsc.c:
+.vsc.c: $(top_srcdir)/lib/libvsc/vsctool.py
 	$(PYTHON) $(top_srcdir)/lib/libvsc/vsctool.py -c $<
 
-.vsc.h:
+.vsc.h: $(top_srcdir)/lib/libvsc/vsctool.py
 	$(PYTHON) $(top_srcdir)/lib/libvsc/vsctool.py -h $<
 
 EXTRA_DIST += VSC_debug.vsc

From phk at FreeBSD.org  Wed Mar 29 08:59:08 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 29 Mar 2023 08:59:08 +0000 (UTC)
Subject: [master] 99400dfea Make vmodtool leave output files alone if
 unchanged. This prevents unecessary docs builds. However,
 we do need to touch the *if.c mtime,
 otherwise a doc-only change will cause vmodtool to run on every make
 afterwards.
Message-ID: <20230329085908.E1668103514@lists.varnish-cache.org>


commit 99400dfeaccb5d6a39f548102febcefeb9c29ce7
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Mar 29 08:14:28 2023 +0000

    Make vmodtool leave output files alone if unchanged.
    This prevents unecessary docs builds.
    However, we do need to touch the *if.c mtime, otherwise a
    doc-only change will cause vmodtool to run on every make afterwards.
    
    Relevant to: #3901

diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py
index da7157766..8f102d2cd 100755
--- a/lib/libvcc/vmodtool.py
+++ b/lib/libvcc/vmodtool.py
@@ -76,8 +76,11 @@ $(libvmod_XXX_la_OBJECTS): PFX.h
 
 PFX.h vmod_XXX.rst vmod_XXX.man.rst: PFX.c
 
+# A doc-change will not update mtime on the .h and .c files, so a
+# touch(1) is necessary to signal that vmodtool was in fact run.
 PFX.c: $(vmodtool) $(srcdir)/VCC
 \t at PYTHON@ $(vmodtool) $(vmodtoolargs_XXX) $(srcdir)/VCC
+\ttouch PFX.c
 
 clean-local: clean-vmod-XXX
 
@@ -924,7 +927,15 @@ class vcc():
 
     def commit(self):
         for i in self.commit_files:
-            os.rename(i + ".tmp", i)
+            try:
+                before = open(i, "rb").read()
+            except:
+                before = None
+            after = open(i + ".tmp", "rb").read()
+            if before != after:
+                os.rename(i + ".tmp", i)
+            else:
+                os.remove(i + ".tmp")
 
     def parse(self):
         global inputline
diff --git a/vmod/automake_boilerplate_blob.am b/vmod/automake_boilerplate_blob.am
index c1934c9b6..c49d42a6f 100644
--- a/vmod/automake_boilerplate_blob.am
+++ b/vmod/automake_boilerplate_blob.am
@@ -32,8 +32,11 @@ $(libvmod_blob_la_OBJECTS): vcc_blob_if.h
 
 vcc_blob_if.h vmod_blob.rst vmod_blob.man.rst: vcc_blob_if.c
 
+# A doc-change will not update mtime on the .h and .c files, so a
+# touch(1) is necessary to signal that vmodtool was in fact run.
 vcc_blob_if.c: $(vmodtool) $(srcdir)/vmod_blob.vcc
 	@PYTHON@ $(vmodtool) $(vmodtoolargs_blob) $(srcdir)/vmod_blob.vcc
+	touch vcc_blob_if.c
 
 clean-local: clean-vmod-blob
 
diff --git a/vmod/automake_boilerplate_cookie.am b/vmod/automake_boilerplate_cookie.am
index 2ab718f63..fa3ccfb7f 100644
--- a/vmod/automake_boilerplate_cookie.am
+++ b/vmod/automake_boilerplate_cookie.am
@@ -25,8 +25,11 @@ $(libvmod_cookie_la_OBJECTS): vcc_cookie_if.h
 
 vcc_cookie_if.h vmod_cookie.rst vmod_cookie.man.rst: vcc_cookie_if.c
 
+# A doc-change will not update mtime on the .h and .c files, so a
+# touch(1) is necessary to signal that vmodtool was in fact run.
 vcc_cookie_if.c: $(vmodtool) $(srcdir)/vmod_cookie.vcc
 	@PYTHON@ $(vmodtool) $(vmodtoolargs_cookie) $(srcdir)/vmod_cookie.vcc
+	touch vcc_cookie_if.c
 
 clean-local: clean-vmod-cookie
 
diff --git a/vmod/automake_boilerplate_debug.am b/vmod/automake_boilerplate_debug.am
index d546d4587..e97ac77da 100644
--- a/vmod/automake_boilerplate_debug.am
+++ b/vmod/automake_boilerplate_debug.am
@@ -28,8 +28,11 @@ $(libvmod_debug_la_OBJECTS): vcc_debug_if.h
 
 vcc_debug_if.h vmod_debug.rst vmod_debug.man.rst: vcc_debug_if.c
 
+# A doc-change will not update mtime on the .h and .c files, so a
+# touch(1) is necessary to signal that vmodtool was in fact run.
 vcc_debug_if.c: $(vmodtool) $(srcdir)/vmod_debug.vcc
 	@PYTHON@ $(vmodtool) $(vmodtoolargs_debug) $(srcdir)/vmod_debug.vcc
+	touch vcc_debug_if.c
 
 clean-local: clean-vmod-debug
 
diff --git a/vmod/automake_boilerplate_directors.am b/vmod/automake_boilerplate_directors.am
index 157d06138..6cec72ead 100644
--- a/vmod/automake_boilerplate_directors.am
+++ b/vmod/automake_boilerplate_directors.am
@@ -35,8 +35,11 @@ $(libvmod_directors_la_OBJECTS): vcc_directors_if.h
 
 vcc_directors_if.h vmod_directors.rst vmod_directors.man.rst: vcc_directors_if.c
 
+# A doc-change will not update mtime on the .h and .c files, so a
+# touch(1) is necessary to signal that vmodtool was in fact run.
 vcc_directors_if.c: $(vmodtool) $(srcdir)/vmod_directors.vcc
 	@PYTHON@ $(vmodtool) $(vmodtoolargs_directors) $(srcdir)/vmod_directors.vcc
+	touch vcc_directors_if.c
 
 clean-local: clean-vmod-directors
 
diff --git a/vmod/automake_boilerplate_proxy.am b/vmod/automake_boilerplate_proxy.am
index 2fa58068d..24be960ea 100644
--- a/vmod/automake_boilerplate_proxy.am
+++ b/vmod/automake_boilerplate_proxy.am
@@ -25,8 +25,11 @@ $(libvmod_proxy_la_OBJECTS): vcc_proxy_if.h
 
 vcc_proxy_if.h vmod_proxy.rst vmod_proxy.man.rst: vcc_proxy_if.c
 
+# A doc-change will not update mtime on the .h and .c files, so a
+# touch(1) is necessary to signal that vmodtool was in fact run.
 vcc_proxy_if.c: $(vmodtool) $(srcdir)/vmod_proxy.vcc
 	@PYTHON@ $(vmodtool) $(vmodtoolargs_proxy) $(srcdir)/vmod_proxy.vcc
+	touch vcc_proxy_if.c
 
 clean-local: clean-vmod-proxy
 
diff --git a/vmod/automake_boilerplate_purge.am b/vmod/automake_boilerplate_purge.am
index 35b893abb..7d2186d11 100644
--- a/vmod/automake_boilerplate_purge.am
+++ b/vmod/automake_boilerplate_purge.am
@@ -25,8 +25,11 @@ $(libvmod_purge_la_OBJECTS): vcc_purge_if.h
 
 vcc_purge_if.h vmod_purge.rst vmod_purge.man.rst: vcc_purge_if.c
 
+# A doc-change will not update mtime on the .h and .c files, so a
+# touch(1) is necessary to signal that vmodtool was in fact run.
 vcc_purge_if.c: $(vmodtool) $(srcdir)/vmod_purge.vcc
 	@PYTHON@ $(vmodtool) $(vmodtoolargs_purge) $(srcdir)/vmod_purge.vcc
+	touch vcc_purge_if.c
 
 clean-local: clean-vmod-purge
 
diff --git a/vmod/automake_boilerplate_std.am b/vmod/automake_boilerplate_std.am
index 57d87f773..4900e4471 100644
--- a/vmod/automake_boilerplate_std.am
+++ b/vmod/automake_boilerplate_std.am
@@ -28,8 +28,11 @@ $(libvmod_std_la_OBJECTS): vcc_std_if.h
 
 vcc_std_if.h vmod_std.rst vmod_std.man.rst: vcc_std_if.c
 
+# A doc-change will not update mtime on the .h and .c files, so a
+# touch(1) is necessary to signal that vmodtool was in fact run.
 vcc_std_if.c: $(vmodtool) $(srcdir)/vmod_std.vcc
 	@PYTHON@ $(vmodtool) $(vmodtoolargs_std) $(srcdir)/vmod_std.vcc
+	touch vcc_std_if.c
 
 clean-local: clean-vmod-std
 
diff --git a/vmod/automake_boilerplate_unix.am b/vmod/automake_boilerplate_unix.am
index be3afe8cf..ab9ca4bca 100644
--- a/vmod/automake_boilerplate_unix.am
+++ b/vmod/automake_boilerplate_unix.am
@@ -26,8 +26,11 @@ $(libvmod_unix_la_OBJECTS): vcc_unix_if.h
 
 vcc_unix_if.h vmod_unix.rst vmod_unix.man.rst: vcc_unix_if.c
 
+# A doc-change will not update mtime on the .h and .c files, so a
+# touch(1) is necessary to signal that vmodtool was in fact run.
 vcc_unix_if.c: $(vmodtool) $(srcdir)/vmod_unix.vcc
 	@PYTHON@ $(vmodtool) $(vmodtoolargs_unix) $(srcdir)/vmod_unix.vcc
+	touch vcc_unix_if.c
 
 clean-local: clean-vmod-unix
 
diff --git a/vmod/automake_boilerplate_vtc.am b/vmod/automake_boilerplate_vtc.am
index 2d8d9a666..cae60718d 100644
--- a/vmod/automake_boilerplate_vtc.am
+++ b/vmod/automake_boilerplate_vtc.am
@@ -25,8 +25,11 @@ $(libvmod_vtc_la_OBJECTS): vcc_vtc_if.h
 
 vcc_vtc_if.h vmod_vtc.rst vmod_vtc.man.rst: vcc_vtc_if.c
 
+# A doc-change will not update mtime on the .h and .c files, so a
+# touch(1) is necessary to signal that vmodtool was in fact run.
 vcc_vtc_if.c: $(vmodtool) $(srcdir)/vmod_vtc.vcc
 	@PYTHON@ $(vmodtool) $(vmodtoolargs_vtc) $(srcdir)/vmod_vtc.vcc
+	touch vcc_vtc_if.c
 
 clean-local: clean-vmod-vtc
 

From phk at FreeBSD.org  Wed Mar 29 08:59:09 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 29 Mar 2023 08:59:09 +0000 (UTC)
Subject: [master] 9fec14d8f We are trunk again
Message-ID: <20230329085909.0A348103517@lists.varnish-cache.org>


commit 9fec14d8f2c5a1e7abbf2b1cf9c7e5942da6ec0c
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Mar 29 08:19:08 2023 +0000

    We are trunk again

diff --git a/configure.ac b/configure.ac
index 1dc434839..70aa89818 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2,7 +2,7 @@ AC_PREREQ(2.69)
 AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS
 Copyright (c) 2006-2023 Varnish Software])
 AC_REVISION([$Id$])
-AC_INIT([Varnish], [7.3.0], [varnish-dev at varnish-cache.org])
+AC_INIT([Varnish], [trunk], [varnish-dev at varnish-cache.org])
 AC_CONFIG_SRCDIR(include/miniobj.h)
 AC_CONFIG_HEADERS([config.h])
 AC_CONFIG_MACRO_DIR([m4])

From phk at FreeBSD.org  Wed Mar 29 08:59:09 2023
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Wed, 29 Mar 2023 08:59:09 +0000 (UTC)
Subject: [master] 0d6038723 Put doc/ back on -trunk too
Message-ID: <20230329085909.297F610351C@lists.varnish-cache.org>


commit 0d603872379bc506fc86d40e341d716c99ec1fd3
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Wed Mar 29 08:30:02 2023 +0000

    Put doc/ back on -trunk too

diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-trunk.rst
new file mode 100644
index 000000000..244eca45e
--- /dev/null
+++ b/doc/sphinx/whats-new/changes-trunk.rst
@@ -0,0 +1,16 @@
+.. _whatsnew_changes_CURRENT:
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+Changes in Varnish **${NEXT_RELEASE}**
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+For information about updating your current Varnish deployment to the
+new version, see :ref:`whatsnew_upgrading_CURRENT`.
+
+A more detailed and technical account of changes in Varnish, with
+links to issues that have been fixed and pull requests that have been
+merged, may be found in the `change log`_.
+
+.. _change log: https://github.com/varnishcache/varnish-cache/blob/master/doc/changes.rst
+
+*eof*
diff --git a/doc/sphinx/whats-new/index.rst b/doc/sphinx/whats-new/index.rst
index 3173ec42a..7b4442bd9 100644
--- a/doc/sphinx/whats-new/index.rst
+++ b/doc/sphinx/whats-new/index.rst
@@ -13,6 +13,19 @@ This section describes the changes and improvements between different
 versions of Varnish, and what upgrading between the different versions
 entail.
 
+Varnish **$NEXT_RELEASE**
+-------------------------
+
+**Note: These are working documents for a future release, with running
+updates for changes in the development branch. For changes in the
+released versions of Varnish, see the chapters listed below.**
+
+.. toctree::
+   :maxdepth: 2
+ 
+   changes-trunk
+   upgrading-trunk
+
 Varnish 7.3
 -----------
 
diff --git a/doc/sphinx/whats-new/upgrading-trunk.rst b/doc/sphinx/whats-new/upgrading-trunk.rst
new file mode 100644
index 000000000..c0a2a0a3f
--- /dev/null
+++ b/doc/sphinx/whats-new/upgrading-trunk.rst
@@ -0,0 +1,17 @@
+**Note: This is a working document for a future release, with running
+updates for changes in the development branch. For changes in the
+released versions of Varnish, see:** :ref:`whats-new-index`
+ 
+.. _whatsnew_upgrading_CURRENT:
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+Upgrading to Varnish **$NEXT_RELEASE**
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+**XXX: how to upgrade from previous deployments to this
+version. Limited to work that has to be done for an upgrade, new
+features are listed in "Changes". Explicitly mention what does *not*
+have to be changed, especially in VCL. May include, but is not limited
+to:**
+
+*eof*