From dridi.boukelmoune at gmail.com  Fri Nov  1 08:39:09 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Fri,  1 Nov 2019 08:39:09 +0000 (UTC)
Subject: [master] 952c9576d Generate a proper manual name for VMODs
Message-ID: <20191101083909.69D9D1107F2@lists.varnish-cache.org>


commit 952c9576d2f346fc63442c6ffbc710f398382f2e
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Nov 1 09:34:36 2019 +0100

    Generate a proper manual name for VMODs
    
    Fixes #3110

diff --git a/lib/libvcc/vmodtool.py b/lib/libvcc/vmodtool.py
index 6181e8ffa..a556d9035 100755
--- a/lib/libvcc/vmodtool.py
+++ b/lib/libvcc/vmodtool.py
@@ -610,7 +610,7 @@ class ModuleStanza(Stanza):
     def rsthead(self, fo, man):
 
         if man:
-            write_rst_hdr(fo, "VMOD " + self.vcc.modname, "=", "=")
+            write_rst_hdr(fo, "vmod_" + self.vcc.modname, "=", "=")
             write_rst_hdr(fo, self.vcc.moddesc, "-", "-")
             fo.write("\n")
             fo.write(":Manual section: " + self.vcc.mansection + "\n")

From nils.goroll at uplex.de  Fri Nov  1 21:27:08 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Fri,  1 Nov 2019 21:27:08 +0000 (UTC)
Subject: [master] 6c3861c38 fix vsl_mask doc
Message-ID: <20191101212709.44CE69C7D@lists.varnish-cache.org>


commit 6c3861c3858429d29f6ea10836f0bdfc3a9f77f6
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Fri Nov 1 22:24:27 2019 +0100

    fix vsl_mask doc
    
    unmask/mask was inverted
    
    remove superfluous comma

diff --git a/bin/varnishd/mgt/mgt_param_bits.c b/bin/varnishd/mgt/mgt_param_bits.c
index 8fc6f61b9..df2ca2320 100644
--- a/bin/varnishd/mgt/mgt_param_bits.c
+++ b/bin/varnishd/mgt/mgt_param_bits.c
@@ -253,7 +253,7 @@ struct parspec VSL_parspec[] = {
 	{ "vsl_mask", tweak_vsl_mask, NULL, NULL, NULL,
 		"Mask individual VSL messages from being logged.\n"
 		"\tdefault\tSet default value\n"
-		"\nUse +/- prefix in front of VSL tag name, to mask/unmask "
+		"\nUse +/- prefix in front of VSL tag name to unmask/mask "
 		"individual VSL messages.",
 		0, "default", "" },
 	{ "debug", tweak_debug, NULL, NULL, NULL,
diff --git a/include/tbl/params.h b/include/tbl/params.h
index fe1589371..f8c9e6de9 100644
--- a/include/tbl/params.h
+++ b/include/tbl/params.h
@@ -1616,7 +1616,7 @@ PARAM(
 	"Mask individual VSL messages from being logged.\n"
 	"	default	Set default value\n"
 	"\n"
-	"Use +/- prefix in front of VSL tag name, to mask/unmask "
+	"Use +/- prefix in front of VSL tag name to unmask/mask "
 	"individual VSL messages.",
 	/* l-text */	"",
 	/* func */	NULL

From fgsch at lodoss.net  Sat Nov  2 00:11:10 2019
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Sat,  2 Nov 2019 00:11:10 +0000 (UTC)
Subject: [master] 27f778050 Also test without unwind enabled
Message-ID: <20191102001111.22AA56EC08@lists.varnish-cache.org>


commit 27f778050b6ef807d8217829e2331e4b63e4a920
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Sat Nov 2 09:09:54 2019 +0900

    Also test without unwind enabled

diff --git a/.travis.yml b/.travis.yml
index c5d157204..0d4c431de 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -31,6 +31,9 @@ jobs:
           fi
     - <<: *test-linux
       compiler: gcc
+      before_script:
+        - ./autogen.sh
+        - ./configure
     - <<: *test-linux
       stage: sanitizers
       addons:

From fgsch at lodoss.net  Mon Nov  4 09:21:06 2019
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Mon,  4 Nov 2019 09:21:06 +0000 (UTC)
Subject: [master] af4687513 Fold packaging into README and convert to markdown
Message-ID: <20191104092106.D8639107972@lists.varnish-cache.org>


commit af46875135b9efd09bcfb0a25aabb3616c2b8c4c
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon Nov 4 09:20:15 2019 +0000

    Fold packaging into README and convert to markdown

diff --git a/README.Packaging b/README.Packaging
deleted file mode 100644
index 0b4275893..000000000
--- a/README.Packaging
+++ /dev/null
@@ -1,27 +0,0 @@
-Packaging
-=========
-
-Varnish Cache packaging files are kept outside of the main distribution.
-
-The main reason for this is to decouple the development work from the packaging work.
-
-We want to be able to tag a release and do a tarball release without having
-to wait for the packagers to finish their work/changes.
-
-
-Official packages
------------------
-
-The official Debian and Redhat packages are built by the Varnish Cache team
-and made available on https://packagecloud.io/varnishcache/ .
-
-Packaging files and scripts for Debian and Redhat:
-
-    https://github.com/varnishcache/pkg-varnish-cache
-
-
-Third-party packages
---------------------
-
-Varnish Cache is built and packaged in many different operating systems and
-distributions. Please see https://varnish-cache.org/ for more information.
diff --git a/README.md b/README.md
new file mode 100644
index 000000000..39710ecd7
--- /dev/null
+++ b/README.md
@@ -0,0 +1,44 @@
+Varnish Cache
+=============
+
+[![CircleCI Build Status](https://circleci.com/gh/varnishcache/varnish-cache/tree/master.svg?style=svg)](https://circleci.com/gh/varnishcache/varnish-cache/tree/master)
+[![TravisCI Build Status](https://travis-ci.org/varnishcache/varnish-cache.svg?branch=master)](https://travis-ci.org/varnishcache/varnish-cache)
+
+This is Varnish Cache, the high-performance HTTP accelerator.
+
+Documentation and additional information about Varnish is available
+on https://www.varnish-cache.org/
+
+Technical questions about Varnish and this release should be addressed
+to <varnish-misc at varnish-cache.org>.
+
+Please see CONTRIBUTING for how to contribute patches and report bugs.
+
+Questions about commercial support and services related to Varnish
+should be addressed to <sales at varnish-software.com>.
+
+More platforms are tested via [vtest](https://varnish-cache.org/vtest/).
+
+## Packaging
+
+Varnish Cache packaging files are kept outside of the main distribution.
+
+The main reason for this is to decouple the development work from the packaging work.
+
+We want to be able to tag a release and do a tarball release without having
+to wait for the packagers to finish their work/changes.
+
+### Official packages
+
+The official Debian and Redhat packages are built by the Varnish
+Cache team and made available on https://packagecloud.io/varnishcache/.
+
+Packaging files and scripts for Debian and Redhat:
+
+    https://github.com/varnishcache/pkg-varnish-cache
+
+### Third-party packages
+
+Varnish Cache is built and packaged in many different operating
+systems and distributions. Please see https://varnish-cache.org/
+for more information.

From dridi at varni.sh  Mon Nov  4 09:23:37 2019
From: dridi at varni.sh (Dridi Boukelmoune)
Date: Mon, 4 Nov 2019 09:23:37 +0000
Subject: [master] af4687513 Fold packaging into README and convert to
 markdown
In-Reply-To: <20191104092106.D8639107972@lists.varnish-cache.org>
References: <20191104092106.D8639107972@lists.varnish-cache.org>
Message-ID: <CABoVN9Aw3NtNMerGYW_j86DpkeL1FCeNX5+DjK-DbmV9jDUVrg@mail.gmail.com>

On Mon, Nov 4, 2019 at 9:21 AM Federico G. Schwindt <fgsch at lodoss.net> wrote:
>
>
> commit af46875135b9efd09bcfb0a25aabb3616c2b8c4c
> Author: Federico G. Schwindt <fgsch at lodoss.net>
> Date:   Mon Nov 4 09:20:15 2019 +0000
>
>     Fold packaging into README and convert to markdown
>
> diff --git a/README.Packaging b/README.Packaging
> deleted file mode 100644
> index 0b4275893..000000000
> --- a/README.Packaging
> +++ /dev/null
> @@ -1,27 +0,0 @@
> -Packaging
> -=========
> -
> -Varnish Cache packaging files are kept outside of the main distribution.
> -
> -The main reason for this is to decouple the development work from the packaging work.
> -
> -We want to be able to tag a release and do a tarball release without having
> -to wait for the packagers to finish their work/changes.
> -
> -
> -Official packages
> ------------------
> -
> -The official Debian and Redhat packages are built by the Varnish Cache team
> -and made available on https://packagecloud.io/varnishcache/ .
> -
> -Packaging files and scripts for Debian and Redhat:
> -
> -    https://github.com/varnishcache/pkg-varnish-cache
> -
> -
> -Third-party packages
> ---------------------
> -
> -Varnish Cache is built and packaged in many different operating systems and
> -distributions. Please see https://varnish-cache.org/ for more information.
> diff --git a/README.md b/README.md
> new file mode 100644
> index 000000000..39710ecd7
> --- /dev/null
> +++ b/README.md

Why create README.md? We already had README.rst in use.

> @@ -0,0 +1,44 @@
> +Varnish Cache
> +=============
> +
> +[![CircleCI Build Status](https://circleci.com/gh/varnishcache/varnish-cache/tree/master.svg?style=svg)](https://circleci.com/gh/varnishcache/varnish-cache/tree/master)
> +[![TravisCI Build Status](https://travis-ci.org/varnishcache/varnish-cache.svg?branch=master)](https://travis-ci.org/varnishcache/varnish-cache)
> +
> +This is Varnish Cache, the high-performance HTTP accelerator.
> +
> +Documentation and additional information about Varnish is available
> +on https://www.varnish-cache.org/
> +
> +Technical questions about Varnish and this release should be addressed
> +to <varnish-misc at varnish-cache.org>.
> +
> +Please see CONTRIBUTING for how to contribute patches and report bugs.
> +
> +Questions about commercial support and services related to Varnish
> +should be addressed to <sales at varnish-software.com>.
> +
> +More platforms are tested via [vtest](https://varnish-cache.org/vtest/).
> +
> +## Packaging
> +
> +Varnish Cache packaging files are kept outside of the main distribution.
> +
> +The main reason for this is to decouple the development work from the packaging work.
> +
> +We want to be able to tag a release and do a tarball release without having
> +to wait for the packagers to finish their work/changes.
> +
> +### Official packages
> +
> +The official Debian and Redhat packages are built by the Varnish
> +Cache team and made available on https://packagecloud.io/varnishcache/.
> +
> +Packaging files and scripts for Debian and Redhat:
> +
> +    https://github.com/varnishcache/pkg-varnish-cache
> +
> +### Third-party packages
> +
> +Varnish Cache is built and packaged in many different operating
> +systems and distributions. Please see https://varnish-cache.org/
> +for more information.
> _______________________________________________
> varnish-commit mailing list
> varnish-commit at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-commit

From fgsch at lodoss.net  Mon Nov  4 09:26:06 2019
From: fgsch at lodoss.net (Federico G. Schwindt)
Date: Mon,  4 Nov 2019 09:26:06 +0000 (UTC)
Subject: [master] 90f084c61 Revert last commit
Message-ID: <20191104092606.9E0EB107D77@lists.varnish-cache.org>


commit 90f084c613c3a67feef75cbccf2a4fe3d78c2957
Author: Federico G. Schwindt <fgsch at lodoss.net>
Date:   Mon Nov 4 09:23:39 2019 +0000

    Revert last commit
    
    This was meant to go to a different remote before proposing it.
    Sorry!

diff --git a/README.Packaging b/README.Packaging
new file mode 100644
index 000000000..0b4275893
--- /dev/null
+++ b/README.Packaging
@@ -0,0 +1,27 @@
+Packaging
+=========
+
+Varnish Cache packaging files are kept outside of the main distribution.
+
+The main reason for this is to decouple the development work from the packaging work.
+
+We want to be able to tag a release and do a tarball release without having
+to wait for the packagers to finish their work/changes.
+
+
+Official packages
+-----------------
+
+The official Debian and Redhat packages are built by the Varnish Cache team
+and made available on https://packagecloud.io/varnishcache/ .
+
+Packaging files and scripts for Debian and Redhat:
+
+    https://github.com/varnishcache/pkg-varnish-cache
+
+
+Third-party packages
+--------------------
+
+Varnish Cache is built and packaged in many different operating systems and
+distributions. Please see https://varnish-cache.org/ for more information.
diff --git a/README.md b/README.md
deleted file mode 100644
index 39710ecd7..000000000
--- a/README.md
+++ /dev/null
@@ -1,44 +0,0 @@
-Varnish Cache
-=============
-
-[![CircleCI Build Status](https://circleci.com/gh/varnishcache/varnish-cache/tree/master.svg?style=svg)](https://circleci.com/gh/varnishcache/varnish-cache/tree/master)
-[![TravisCI Build Status](https://travis-ci.org/varnishcache/varnish-cache.svg?branch=master)](https://travis-ci.org/varnishcache/varnish-cache)
-
-This is Varnish Cache, the high-performance HTTP accelerator.
-
-Documentation and additional information about Varnish is available
-on https://www.varnish-cache.org/
-
-Technical questions about Varnish and this release should be addressed
-to <varnish-misc at varnish-cache.org>.
-
-Please see CONTRIBUTING for how to contribute patches and report bugs.
-
-Questions about commercial support and services related to Varnish
-should be addressed to <sales at varnish-software.com>.
-
-More platforms are tested via [vtest](https://varnish-cache.org/vtest/).
-
-## Packaging
-
-Varnish Cache packaging files are kept outside of the main distribution.
-
-The main reason for this is to decouple the development work from the packaging work.
-
-We want to be able to tag a release and do a tarball release without having
-to wait for the packagers to finish their work/changes.
-
-### Official packages
-
-The official Debian and Redhat packages are built by the Varnish
-Cache team and made available on https://packagecloud.io/varnishcache/.
-
-Packaging files and scripts for Debian and Redhat:
-
-    https://github.com/varnishcache/pkg-varnish-cache
-
-### Third-party packages
-
-Varnish Cache is built and packaged in many different operating
-systems and distributions. Please see https://varnish-cache.org/
-for more information.

From phk at FreeBSD.org  Mon Nov  4 12:46:06 2019
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon,  4 Nov 2019 12:46:06 +0000 (UTC)
Subject: [master] e50acd546 Simplify and appeace FlexeLint
Message-ID: <20191104124606.4B73110CD6C@lists.varnish-cache.org>


commit e50acd5460fbddbd75bf3570c1de041b629120bb
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Nov 4 12:34:53 2019 +0000

    Simplify and appeace FlexeLint

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index e13de2250..631a8a09d 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -104,8 +104,8 @@ cnt_transport(struct worker *wrk, struct req *req)
  * Deliver an object to client
  */
 
-static int
-resp_setup_deliver(struct req *req)
+int
+Resp_Setup_Deliver(struct req *req)
 {
 	struct http *h;
 	struct objcore *oc;
@@ -145,12 +145,11 @@ resp_setup_deliver(struct req *req)
 	    ObjCheckFlag(req->wrk, oc, OF_GZIPED) &&
 	    !RFC2616_Req_Gzip(req->http))
 		RFC2616_Weaken_Etag(h);
-
-	return (0);
+	return(0);
 }
 
-static int
-resp_setup_synth(struct req *req)
+void
+Resp_Setup_Synth(struct req *req)
 {
 	struct http *h;
 
@@ -175,23 +174,6 @@ resp_setup_synth(struct req *req)
 	 */
 	if (req->want100cont)
 		http_SetHeader(h, "Connection: close");
-
-	return (0);
-}
-
-int
-Resp_Setup(struct req *req, unsigned method)
-{
-	switch (method) {
-	case VCL_MET_DELIVER:
-		return (resp_setup_deliver(req));
-	case VCL_MET_SYNTH:
-		return (resp_setup_synth(req));
-	default:
-		WRONG("vcl method");
-	}
-
-	return (0);
 }
 
 static enum req_fsm_nxt
@@ -209,7 +191,7 @@ cnt_deliver(struct worker *wrk, struct req *req)
 
 	ObjTouch(req->wrk, req->objcore, req->t_prev);
 
-	if (resp_setup_deliver(req)) {
+	if (Resp_Setup_Deliver(req)) {
 		(void)HSH_DerefObjCore(wrk, &req->objcore, HSH_RUSH_POLICY);
 		req->err_code = 500;
 		req->req_step = R_STP_SYNTH;
@@ -297,7 +279,7 @@ cnt_synth(struct worker *wrk, struct req *req)
 	if (req->err_code < 100)
 		req->err_code = 501;
 
-	(void) resp_setup_synth(req);
+	Resp_Setup_Synth(req);
 
 	synth_body = VSB_new_auto();
 	AN(synth_body);
diff --git a/bin/varnishd/cache/cache_varnishd.h b/bin/varnishd/cache/cache_varnishd.h
index f242b229c..b8ccb4704 100644
--- a/bin/varnishd/cache/cache_varnishd.h
+++ b/bin/varnishd/cache/cache_varnishd.h
@@ -361,7 +361,8 @@ void VRB_Free(struct req *);
 
 /* cache_req_fsm.c [CNT] */
 
-int Resp_Setup(struct req *, unsigned);
+int Resp_Setup_Deliver(struct req *);
+void Resp_Setup_Synth(struct req *);
 
 enum req_fsm_nxt {
 	REQ_FSM_MORE,
diff --git a/bin/varnishd/cache/cache_vrt.c b/bin/varnishd/cache/cache_vrt.c
index 0fa600f2a..959b4be16 100644
--- a/bin/varnishd/cache/cache_vrt.c
+++ b/bin/varnishd/cache/cache_vrt.c
@@ -747,8 +747,10 @@ VRT_Rollback(VRT_CTX, VCL_HTTP hp)
 	if (hp == ctx->http_req) {
 		CHECK_OBJ_NOTNULL(ctx->req, REQ_MAGIC);
 		Req_Rollback(ctx->req);
-		if ((ctx->method & (VCL_MET_DELIVER | VCL_MET_SYNTH)) != 0)
-			Resp_Setup(ctx->req, ctx->method);
+		if (ctx->method & VCL_MET_DELIVER)
+			XXXAZ(Resp_Setup_Deliver(ctx->req));
+		if (ctx->method & VCL_MET_SYNTH)
+			Resp_Setup_Synth(ctx->req);
 	} else if (hp == ctx->http_bereq) {
 		CHECK_OBJ_NOTNULL(ctx->bo, BUSYOBJ_MAGIC);
 		Bereq_Rollback(ctx->bo);
diff --git a/bin/varnishd/cache/cache_wrk.c b/bin/varnishd/cache/cache_wrk.c
index 01a379ddd..9f1386c5f 100644
--- a/bin/varnishd/cache/cache_wrk.c
+++ b/bin/varnishd/cache/cache_wrk.c
@@ -482,7 +482,7 @@ pool_breed(struct pool *qp)
 		if (cache_param->wthread_add_delay > 0.0)
 			VTIM_sleep(cache_param->wthread_add_delay);
 		else
-			sched_yield();
+			(void)sched_yield();
 	}
 
 	AZ(pthread_attr_destroy(&tp_attr));

From nils.goroll at uplex.de  Tue Nov  5 13:21:05 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue,  5 Nov 2019 13:21:05 +0000 (UTC)
Subject: [master] 6dbfff12d avoid re-setting Accept-Encoding for ESI
Message-ID: <20191105132105.C4A7A9C2C5@lists.varnish-cache.org>


commit 6dbfff12dca895534b090ef9283e75f85659fcfa
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue Nov 5 14:18:30 2019 +0100

    avoid re-setting Accept-Encoding for ESI
    
    we should not emit ReqUnset / ReqHeader if A-E is already correct

diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c
index 498aca6f2..f9fe82241 100644
--- a/bin/varnishd/cache/cache_esi_deliver.c
+++ b/bin/varnishd/cache/cache_esi_deliver.c
@@ -161,9 +161,10 @@ ved_include(struct req *preq, const char *src, const char *host,
 	http_Unset(req->http, H_Range);
 
 	/* Set Accept-Encoding according to what we want */
-	http_Unset(req->http, H_Accept_Encoding);
 	if (ecx->isgzip)
 		http_ForceHeader(req->http, H_Accept_Encoding, "gzip");
+	else
+		http_Unset(req->http, H_Accept_Encoding);
 
 	/* Client content already taken care of */
 	http_Unset(req->http, H_Content_Length);

From nils.goroll at uplex.de  Tue Nov  5 18:01:07 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue,  5 Nov 2019 18:01:07 +0000 (UTC)
Subject: [master] bea1da6e0 get an idea about the margin before r02219.vtc
 fails
Message-ID: <20191105180107.9BC37A5AEB@lists.varnish-cache.org>


commit bea1da6e0d2557143689acd2842b82b5b4268dec
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue Nov 5 18:50:36 2019 +0100

    get an idea about the margin before r02219.vtc fails
    
    When making other changes which require additional workspace, this test
    may fail for too big a request in the first place.
    
    This additional logging facilitates before/after analysis

diff --git a/bin/varnishtest/tests/r02219.vtc b/bin/varnishtest/tests/r02219.vtc
index ff1a924f7..8b4d1c188 100644
--- a/bin/varnishtest/tests/r02219.vtc
+++ b/bin/varnishtest/tests/r02219.vtc
@@ -10,7 +10,10 @@ server s1 {
 } -start
 
 varnish v1 -arg "-p workspace_client=9k" -proto PROXY -vcl+backend {
+	import std;
+	import vtc;
 	sub vcl_recv {
+		std.log(vtc.workspace_free(client));
 		return (pass);
 	}
 } -start

From lasse at isokron.no  Tue Nov  5 09:07:09 2019
From: lasse at isokron.no (Lasse Karstensen)
Date: Tue,  5 Nov 2019 09:07:09 +0000 (UTC)
Subject: [master] e6e706104 Be clear that you should never try to
 std.fileread() a jpeg.
Message-ID: <20191105090709.3C7AD935B9@lists.varnish-cache.org>


commit e6e706104f2d1b06e66e5ec23d38386b1e037513
Author: Lasse Karstensen <lasse at isokron.no>
Date:   Tue Nov 5 10:05:46 2019 +0100

    Be clear that you should never try to std.fileread() a jpeg.

diff --git a/lib/libvmod_std/vmod.vcc b/lib/libvmod_std/vmod.vcc
index 1a12d9fc8..d1af75f3c 100644
--- a/lib/libvmod_std/vmod.vcc
+++ b/lib/libvmod_std/vmod.vcc
@@ -188,6 +188,8 @@ $Function STRING fileread(PRIV_CALL, STRING)
 Reads a file and returns a string with the content. The result is
 cached indefinitely per filename.
 
+This function should not be used for reading binary files.
+
 Example::
 
 	synthetic("Response was served by " + std.fileread("/etc/hostname"));

From nils.goroll at uplex.de  Wed Nov  6 14:47:07 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed,  6 Nov 2019 14:47:07 +0000 (UTC)
Subject: [master] ac979b67f always assign ctx->handling = &wrk->handling
Message-ID: <20191106144707.85A51102529@lists.varnish-cache.org>


commit ac979b67f10b217912d4c596a5803af11c49cbb8
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed May 15 11:33:30 2019 +0200

    always assign ctx->handling = &wrk->handling
    
    This is option 'always add ctx->handling' from #2997:
    
    we add the (struct vrt_ctx).handling pointer even where there is no VCL
    method for consistency to enable use of VRT_fail() consistently.

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 9adb3f431..2a26c34a3 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -77,6 +77,7 @@ VCL_Bo2Ctx(struct vrt_ctx *ctx, struct busyobj *bo)
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
+	CHECK_OBJ_NOTNULL(bo->wrk, WORKER_MAGIC);
 	ctx->vcl = bo->vcl;
 	ctx->vsl = bo->vsl;
 	ctx->http_bereq = bo->bereq;
@@ -85,6 +86,8 @@ VCL_Bo2Ctx(struct vrt_ctx *ctx, struct busyobj *bo)
 	ctx->sp = bo->sp;
 	ctx->now = bo->t_prev;
 	ctx->ws = bo->ws;
+	ctx->handling = &bo->wrk->handling;
+	*ctx->handling = 0;
 }
 
 void
@@ -94,6 +97,7 @@ VCL_Req2Ctx(struct vrt_ctx *ctx, struct req *req)
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 	CHECK_OBJ_ORNULL(req->topreq, REQ_MAGIC);
+	CHECK_OBJ_NOTNULL(req->wrk, WORKER_MAGIC);
 
 	ctx->vcl = req->vcl;
 	ctx->vsl = req->vsl;
@@ -104,6 +108,8 @@ VCL_Req2Ctx(struct vrt_ctx *ctx, struct req *req)
 	ctx->sp = req->sp;
 	ctx->now = req->t_prev;
 	ctx->ws = req->ws;
+	ctx->handling = &req->wrk->handling;
+	*ctx->handling = 0;
 }
 
 /*--------------------------------------------------------------------*/
diff --git a/bin/varnishd/cache/cache_vrt_vcl.c b/bin/varnishd/cache/cache_vrt_vcl.c
index 451d7aea3..2617db9e1 100644
--- a/bin/varnishd/cache/cache_vrt_vcl.c
+++ b/bin/varnishd/cache/cache_vrt_vcl.c
@@ -441,8 +441,6 @@ vcl_call_method(struct worker *wrk, struct req *req, struct busyobj *bo,
 	ctx.syntax = ctx.vcl->conf->syntax;
 	ctx.specific = specific;
 	ctx.method = method;
-	wrk->handling = 0;
-	ctx.handling = &wrk->handling;
 	aws = WS_Snapshot(wrk->aws);
 	wrk->cur_method = method;
 	wrk->seen_methods |= method;
diff --git a/include/vrt.h b/include/vrt.h
index 6eaa34690..c78f2c402 100644
--- a/include/vrt.h
+++ b/include/vrt.h
@@ -258,7 +258,7 @@ struct vrt_ctx {
 
 	unsigned			syntax;
 	unsigned			method;
-	unsigned			*handling;	// not in director context
+	unsigned			*handling;
 	unsigned			vclver;
 
 	struct vsb			*msg;	// Only in ...init()

From nils.goroll at uplex.de  Wed Nov  6 14:47:07 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed,  6 Nov 2019 14:47:07 +0000 (UTC)
Subject: [master] cb456b828 Add worker to the busyobj for pipe mode
Message-ID: <20191106144707.9EDA210252C@lists.varnish-cache.org>


commit cb456b82809409102f7389e85ba7b4fc22e96f12
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed May 15 11:43:10 2019 +0200

    Add worker to the busyobj for pipe mode

diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 631a8a09d..650ce3a6c 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -725,6 +725,7 @@ cnt_pipe(struct worker *wrk, struct req *req)
 		req->want100cont = 0;
 	}
 
+	bo->wrk = wrk;
 	VCL_pipe_method(req->vcl, wrk, req, bo, NULL);
 
 	switch (wrk->handling) {

From nils.goroll at uplex.de  Wed Nov  6 14:47:07 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed,  6 Nov 2019 14:47:07 +0000 (UTC)
Subject: [master] 8e7b0f01d add a vsl to the cli ctx if there is no msg buffer
Message-ID: <20191106144707.BD89B102530@lists.varnish-cache.org>


commit 8e7b0f01d7c2bb0ae57c016f8bfb01d1fbd00dcb
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed May 15 13:30:39 2019 +0200

    add a vsl to the cli ctx if there is no msg buffer
    
    This is another requirement for use of VRT_fail() in director context.
    
    For code checking for (struct vrt_ctx *)->vsl the logging order now may
    change, and we might want to review our VSL() calls if the ctx vsl
    buffer may now be used instead.
    
    Ref #2997

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 2a26c34a3..8131260c9 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -68,6 +68,7 @@ static struct vrt_ctx ctx_cli;
 static unsigned handling_cli;
 static struct ws ws_cli;
 static uintptr_t ws_snapshot_cli;
+static struct vsl_log vsl_cli;
 
 /*--------------------------------------------------------------------*/
 
@@ -127,6 +128,8 @@ VCL_Get_CliCtx(int msg)
 	if (msg) {
 		ctx_cli.msg = VSB_new_auto();
 		AN(ctx_cli.msg);
+	} else {
+		ctx_cli.vsl = &vsl_cli;
 	}
 	ctx_cli.ws = &ws_cli;
 	WS_Assert(ctx_cli.ws);
@@ -142,6 +145,8 @@ VCL_Rel_CliCtx(struct vrt_ctx **ctx)
 	AN((*ctx)->handling);
 	if (ctx_cli.msg)
 		VSB_destroy(&ctx_cli.msg);
+	if (ctx_cli.vsl)
+		VSL_Flush(ctx_cli.vsl, 0);
 	WS_Assert(ctx_cli.ws);
 	WS_Reset(&ws_cli, ws_snapshot_cli);
 	INIT_OBJ(*ctx, VRT_CTX_MAGIC);
@@ -915,6 +920,8 @@ static struct cli_proto vcl_cmds[] = {
 void
 VCL_Init(void)
 {
+	unsigned sz;
+	char *vslbuf;
 
 	assert(cache_param->workspace_client > 0);
 	WS_Init(&ws_cli, "cli", malloc(cache_param->workspace_client),
@@ -922,4 +929,8 @@ VCL_Init(void)
 	ws_snapshot_cli = WS_Snapshot(&ws_cli);
 	CLI_AddFuncs(vcl_cmds);
 	Lck_New(&vcl_mtx, lck_vcl);
+	sz = cache_param->vsl_buffer;
+	vslbuf = malloc(sz);
+	AN(vslbuf);
+	VSL_Setup(&vsl_cli, vslbuf, sz);
 }
diff --git a/bin/varnishtest/tests/v00041.vtc b/bin/varnishtest/tests/v00041.vtc
index 7d14b388f..ce162378e 100644
--- a/bin/varnishtest/tests/v00041.vtc
+++ b/bin/varnishtest/tests/v00041.vtc
@@ -106,15 +106,18 @@ logexpect l1 -v v1 -g raw -d 1 {
 	expect 0 =    BereqHeader	{^bx0: b /snafu}
 	expect 0 =    VCL_Log		^bar
 
+	# this is not the order of events because std.log logs
+	# into the CLI ctx->vsl, while vmod_debug VSL() goes straight
+	# to the log.
 	expect * 0    Debug		{^vcl1: VCL_EVENT_COLD}
 	expect * 0    CLI		{^Rd vcl.discard}
 	expect 0 =    Debug		{^test_priv_task.*new.$}
 	expect 0 =    Debug		{^test_priv_task.*update.$}
 	expect 0 =    Debug		{^test_priv_task.*exists.$}
-	expect * =    VCL_Log		{^func cleaning up}
 	expect 0 =    Debug		{^test_priv_task.*exists.$}
-	expect 0 =    VCL_Log		{^obj cleaning up}
 	expect 0 =    Debug		{^priv_task_free}
+	expect * =    VCL_Log		{^func cleaning up}
+	expect 0 =    VCL_Log		{^obj cleaning up}
 } -start
 
 client c1 {

From nils.goroll at uplex.de  Wed Nov  6 14:47:07 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed,  6 Nov 2019 14:47:07 +0000 (UTC)
Subject: [master] 0109411b9 test VRT_fail from director contexts
Message-ID: <20191106144707.DB01A102534@lists.varnish-cache.org>


commit 0109411b99e0560b99d2daf2327a9f8af03f54d8
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed May 15 13:07:40 2019 +0200

    test VRT_fail from director contexts
    
    Tests #2997

diff --git a/bin/varnishtest/tests/d00040.vtc b/bin/varnishtest/tests/d00040.vtc
new file mode 100644
index 000000000..2f16e9ed6
--- /dev/null
+++ b/bin/varnishtest/tests/d00040.vtc
@@ -0,0 +1,33 @@
+varnishtest "Test failing in director callbacks"
+
+varnish v1 -vcl {
+	import debug;
+	import std;
+
+	backend dummy { .host = "${bad_ip}"; }
+
+	sub vcl_init {
+		new d = debug.director();
+	}
+
+	sub vcl_recv {
+		if (req.url == "/healthy") {
+			if (std.healthy(d.fail())) {
+				return (synth(200));
+			} else {
+				return (synth(404));
+			}
+		}
+		set req.backend_hint = d.fail();
+	}
+} -start
+
+client c1 {
+	txreq -url "/"
+	rxresp
+	expect resp.status == 503
+
+	txreq -url "/healthy"
+	rxresp
+	expect resp.status == 503
+} -run
diff --git a/lib/libvmod_debug/vmod.vcc b/lib/libvmod_debug/vmod.vcc
index 4a40ecc9b..72b24b3ca 100644
--- a/lib/libvmod_debug/vmod.vcc
+++ b/lib/libvmod_debug/vmod.vcc
@@ -268,3 +268,9 @@ $Function IP get_ip(PRIV_TASK)
 
 Get the IP address previously stored by ``debug.store_ip()`` in the same
 transaction.
+
+$Object director()
+
+$Method BACKEND .fail()
+
+Return a backend which fails in director context
diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c
index 83b102176..d6171d677 100644
--- a/lib/libvmod_debug/vmod_debug.c
+++ b/lib/libvmod_debug/vmod_debug.c
@@ -922,3 +922,80 @@ const struct vmod_data Vmod_wrong3_Data = {
 };
 
 //lint -restore
+
+/*---------------------------------------------------------------------*/
+
+struct VPFX(debug_director) {
+	unsigned			magic;
+#define VMOD_DEBUG_DIRECTOR_MAGIC	0x66b9ff3d
+	VCL_BACKEND			dir;
+};
+
+/* XXX more callbacks ? */
+static vdi_healthy_f vmod_debug_director_healthy;
+static vdi_resolve_f vmod_debug_director_resolve;
+
+static const struct vdi_methods vmod_debug_director_methods[1] = {{
+	.magic =	VDI_METHODS_MAGIC,
+	.type =		"debug.director",
+	.resolve =	vmod_debug_director_resolve,
+	.healthy =	vmod_debug_director_healthy
+}};
+
+VCL_VOID v_matchproto_(td_xyzzy_debug_director__init)
+xyzzy_director__init(VRT_CTX, struct VPFX(debug_director) **dp,
+    const char *vcl_name)
+{
+	struct VPFX(debug_director) *d;
+
+	AN(dp);
+	AZ(*dp);
+	ALLOC_OBJ(d, VMOD_DEBUG_DIRECTOR_MAGIC);
+	AN(d);
+
+	*dp = d;
+	d->dir = VRT_AddDirector(ctx, vmod_debug_director_methods, d,
+	    "%s", vcl_name);
+}
+
+VCL_VOID v_matchproto_(td_xyzzy_debug_director__fini)
+xyzzy_director__fini(struct VPFX(debug_director) **dp)
+{
+	struct VPFX(debug_director) *d;
+
+	TAKE_OBJ_NOTNULL(d, dp, VMOD_DEBUG_DIRECTOR_MAGIC);
+	VRT_DelDirector(&d->dir);
+	FREE_OBJ(d);
+}
+
+VCL_BACKEND v_matchproto_(td_xyzzy_debug_director_fail)
+xyzzy_director_fail(VRT_CTX, struct VPFX(debug_director) *d)
+{
+	CHECK_OBJ_NOTNULL(d, VMOD_DEBUG_DIRECTOR_MAGIC);
+	(void) ctx;
+
+	return (d->dir);
+}
+
+static VCL_BOOL v_matchproto_(vdi_healthy_f)
+vmod_debug_director_healthy(VRT_CTX, VCL_BACKEND dir, VCL_TIME *changed)
+{
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+
+	(void) dir;
+	(void) changed;
+
+	VRT_fail(ctx, "fail");
+	return (1);
+}
+
+static VCL_BACKEND v_matchproto_(vdi_resolve_f)
+vmod_debug_director_resolve(VRT_CTX, VCL_BACKEND dir)
+{
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+
+	(void) dir;
+
+	VRT_fail(ctx, "fail");
+	return (NULL);
+}

From nils.goroll at uplex.de  Wed Nov  6 14:47:07 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed,  6 Nov 2019 14:47:07 +0000 (UTC)
Subject: [master] e1853a3d1 do not count deliberately ending the backend
 connection as fetch failure
Message-ID: <20191106144707.F3B9F102538@lists.varnish-cache.org>


commit e1853a3d1caf420bb3b9a151d0ebd01dbb4b6bdf
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu Aug 8 16:05:29 2019 +0200

    do not count deliberately ending the backend connection as fetch failure
    
    now that we pass the bo's handling to the director method, in the case
    of return(error) from v_b_f, we do no not want the deliberate
    VDI_Finish() to change the handling from vcl.

diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c
index 38a25d6dd..8e59d5321 100644
--- a/bin/varnishd/cache/cache_fetch.c
+++ b/bin/varnishd/cache/cache_fetch.c
@@ -286,6 +286,7 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 {
 	int i;
 	vtim_real now;
+	unsigned handling;
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC);
@@ -416,9 +417,15 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo)
 
 	if (wrk->handling == VCL_RET_ABANDON || wrk->handling == VCL_RET_FAIL ||
 	    wrk->handling == VCL_RET_ERROR) {
+		/* do not count deliberately ending the backend connection as
+		 * fetch failure
+		 */
+		handling = wrk->handling;
 		if (bo->htc)
 			bo->htc->doclose = SC_RESP_CLOSE;
 		vbf_cleanup(bo);
+		wrk->handling = handling;
+
 		if (wrk->handling == VCL_RET_ERROR)
 			return (F_STP_ERROR);
 		else

From nils.goroll at uplex.de  Thu Nov  7 11:50:06 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu,  7 Nov 2019 11:50:06 +0000 (UTC)
Subject: [master] f2947248a gc references to panicstr and fix gdb help
Message-ID: <20191107115006.8AEDC765F@lists.varnish-cache.org>


commit f2947248a5d64f3e7cc70a0774e424d81fe836a2
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu Nov 7 12:49:10 2019 +0100

    gc references to panicstr and fix gdb help

diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index 26be29cf7..8fae5c619 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -52,12 +52,13 @@
 #include "vcs.h"
 
 /*
- * The panic string is constructed in memory, then copied to the
+ * The panic string is constructed in a VSB, then copied to the
  * shared memory.
  *
  * It can be extracted post-mortem from a core dump using gdb:
  *
- * (gdb) printf "%s", panicstr
+ * (gdb) p *(char **)((char *)pan_vsb+8)
+ *
  */
 
 static struct vsb pan_vsb_storage, *pan_vsb;
diff --git a/bin/varnishd/flint.lnt b/bin/varnishd/flint.lnt
index 20166d91b..74fb46180 100644
--- a/bin/varnishd/flint.lnt
+++ b/bin/varnishd/flint.lnt
@@ -131,8 +131,6 @@
 -esym(759, vcc_ProcAction) // could be moved to module
 -esym(714, vcc_ProcAction) // not ref.
 
--esym(765, panicstr)	// Could be static
-
 -emacro(506, isnan, isfinite)	// constant value boolean
 -emacro(736, isfinite)	// loss of precision
 -emacro(774, HTTPH)	// always false

From dridi.boukelmoune at gmail.com  Thu Nov  7 21:46:07 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Thu,  7 Nov 2019 21:46:07 +0000 (UTC)
Subject: [master] 0592e3965 Stabilize c35
Message-ID: <20191107214607.30AF1107C41@lists.varnish-cache.org>


commit 0592e3965df20703a4f858ee0346cadd53a12f0e
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Thu Nov 7 22:44:42 2019 +0100

    Stabilize c35
    
    Spotted via vtest.

diff --git a/bin/varnishtest/tests/c00035.vtc b/bin/varnishtest/tests/c00035.vtc
index 4bffe363f..f063096e7 100644
--- a/bin/varnishtest/tests/c00035.vtc
+++ b/bin/varnishtest/tests/c00035.vtc
@@ -1,6 +1,7 @@
 varnishtest "Dropping polling of a backend"
 
 server s1 -repeat 40 {
+	non_fatal
 	rxreq
 	txresp
 } -start

From phk at FreeBSD.org  Fri Nov  8 08:27:08 2019
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Fri,  8 Nov 2019 08:27:08 +0000 (UTC)
Subject: [master] d9195348b Don't replicate existing default code.
Message-ID: <20191108082708.8D172114AF2@lists.varnish-cache.org>


commit d9195348b627a8d5541ae264c18997a251a5d1dc
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Nov 8 08:22:25 2019 +0000

    Don't replicate existing default code.

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 8131260c9..b0ee19899 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -920,8 +920,6 @@ static struct cli_proto vcl_cmds[] = {
 void
 VCL_Init(void)
 {
-	unsigned sz;
-	char *vslbuf;
 
 	assert(cache_param->workspace_client > 0);
 	WS_Init(&ws_cli, "cli", malloc(cache_param->workspace_client),
@@ -929,8 +927,5 @@ VCL_Init(void)
 	ws_snapshot_cli = WS_Snapshot(&ws_cli);
 	CLI_AddFuncs(vcl_cmds);
 	Lck_New(&vcl_mtx, lck_vcl);
-	sz = cache_param->vsl_buffer;
-	vslbuf = malloc(sz);
-	AN(vslbuf);
-	VSL_Setup(&vsl_cli, vslbuf, sz);
+	VSL_Setup(&vsl_cli, NULL, 0);
 }

From phk at FreeBSD.org  Fri Nov  8 08:53:06 2019
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Fri,  8 Nov 2019 08:53:06 +0000 (UTC)
Subject: [master] 7532c3a7e Eliminate a level of structure on vrt_privs,
 the extra 'magic' does not buy us anything because the compiler can
 see at all times what goes on.
Message-ID: <20191108085306.87DAE116669@lists.varnish-cache.org>


commit 7532c3a7ed6c12635af1a2278e024fe61a351568
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Nov 8 08:52:00 2019 +0000

    Eliminate a level of structure on vrt_privs, the extra 'magic' does
    not buy us anything because the compiler can see at all times what
    goes on.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 40f1f9185..3c3f1ba94 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -199,13 +199,7 @@ struct vxid_pool {
 
 /*--------------------------------------------------------------------*/
 
-VRBT_HEAD(vrt_priv_tree,vrt_priv);
-
-struct vrt_privs {
-	unsigned		magic;
-#define VRT_PRIVS_MAGIC		0x03ba7502
-	struct vrt_priv_tree	privs;
-};
+VRBT_HEAD(vrt_privs, vrt_priv);
 
 /* Worker pool stuff -------------------------------------------------*/
 
diff --git a/bin/varnishd/cache/cache_busyobj.c b/bin/varnishd/cache/cache_busyobj.c
index 6eac91434..98b7f9a74 100644
--- a/bin/varnishd/cache/cache_busyobj.c
+++ b/bin/varnishd/cache/cache_busyobj.c
@@ -153,8 +153,6 @@ VBO_ReleaseBusyObj(struct worker *wrk, struct busyobj **pbo)
 	AZ(bo->htc);
 	AZ(bo->stale_oc);
 
-	AZ(bo->privs->magic);
-
 	VSLb(bo->vsl, SLT_BereqAcct, "%ju %ju %ju %ju %ju %ju",
 	    (uintmax_t)bo->acct.bereq_hdrbytes,
 	    (uintmax_t)bo->acct.bereq_bodybytes,
diff --git a/bin/varnishd/cache/cache_req.c b/bin/varnishd/cache/cache_req.c
index 8fb6ec7f2..767903682 100644
--- a/bin/varnishd/cache/cache_req.c
+++ b/bin/varnishd/cache/cache_req.c
@@ -212,8 +212,6 @@ Req_Cleanup(struct sess *sp, struct worker *wrk, struct req *req)
 	req->director_hint = NULL;
 	req->restarts = 0;
 
-	AZ(req->privs->magic);
-
 	if (req->vcl != NULL)
 		VCL_Recache(wrk, &req->vcl);
 
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 650ce3a6c..4fb23dd92 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -1082,7 +1082,6 @@ CNT_Request(struct req *req)
 
 	AN(req->vsl->wid & VSL_CLIENTMARKER);
 	AN(req->vcl);
-	CHECK_OBJ(req->privs, VRT_PRIVS_MAGIC);
 
 	for (nxt = REQ_FSM_MORE; nxt == REQ_FSM_MORE; ) {
 		/*
diff --git a/bin/varnishd/cache/cache_vrt_priv.c b/bin/varnishd/cache/cache_vrt_priv.c
index 76152cbf3..c2c46d4a9 100644
--- a/bin/varnishd/cache/cache_vrt_priv.c
+++ b/bin/varnishd/cache/cache_vrt_priv.c
@@ -50,7 +50,8 @@ struct vrt_privs cli_task_privs[1];
 
 static inline int vrt_priv_dyncmp(const struct vrt_priv *,
     const struct vrt_priv *);
-VRBT_PROTOTYPE_STATIC(vrt_priv_tree, vrt_priv, entry, vrt_priv_dyncmp)
+
+VRBT_PROTOTYPE_STATIC(vrt_privs, vrt_priv, entry, vrt_priv_dyncmp)
 
 /*--------------------------------------------------------------------
  */
@@ -64,9 +65,8 @@ pan_privs(struct vsb *vsb, const struct vrt_privs *privs)
 	if (PAN_already(vsb, privs))
 		return;
 	VSB_indent(vsb, 2);
-	PAN_CheckMagic(vsb, privs, VRT_PRIVS_MAGIC);
-	if (privs->magic == VRT_PRIVS_MAGIC) {
-		VRBT_FOREACH(vp, vrt_priv_tree, &privs->privs) {
+	if (privs != NULL) {
+		VRBT_FOREACH(vp, vrt_privs, privs) {
 			PAN_CheckMagic(vsb, vp, VRT_PRIV_MAGIC);
 			VSB_printf(vsb,
 			    "priv {p %p l %ld f %p} vmod %jx\n",
@@ -89,8 +89,7 @@ static void
 VRTPRIV_init(struct vrt_privs *privs)
 {
 
-	INIT_OBJ(privs, VRT_PRIVS_MAGIC);
-	VRBT_INIT(&privs->privs);
+	VRBT_INIT(privs);
 }
 
 static inline int
@@ -103,10 +102,10 @@ vrt_priv_dyncmp(const struct vrt_priv *vp1, const struct vrt_priv *vp2)
 	return (0);
 }
 
-VRBT_GENERATE_STATIC(vrt_priv_tree, vrt_priv, entry, vrt_priv_dyncmp)
+VRBT_GENERATE_STATIC(vrt_privs, vrt_priv, entry, vrt_priv_dyncmp)
 
 static struct vmod_priv *
-vrt_priv_dynamic(struct ws *ws, struct vrt_privs *vps, uintptr_t id,
+vrt_priv_dynamic(struct ws *ws, struct vrt_privs *privs, uintptr_t id,
     uintptr_t vmod_id)
 {
 	struct vrt_priv *vp;
@@ -115,11 +114,10 @@ vrt_priv_dynamic(struct ws *ws, struct vrt_privs *vps, uintptr_t id,
 		.vmod_id = vmod_id,
 	};
 
-	CHECK_OBJ_NOTNULL(vps, VRT_PRIVS_MAGIC);
 	AN(id);
 	AN(vmod_id);
 
-	vp = VRBT_FIND(vrt_priv_tree, &vps->privs, &needle);
+	vp = VRBT_FIND(vrt_privs, privs, &needle);
 	if (vp) {
 		CHECK_OBJ(vp, VRT_PRIV_MAGIC);
 		assert(vp->id == id);
@@ -133,16 +131,13 @@ vrt_priv_dynamic(struct ws *ws, struct vrt_privs *vps, uintptr_t id,
 	INIT_OBJ(vp, VRT_PRIV_MAGIC);
 	vp->id = id;
 	vp->vmod_id = vmod_id;
-	VRBT_INSERT(vrt_priv_tree, &vps->privs, vp);
+	VRBT_INSERT(vrt_privs, privs, vp);
 	return (vp->priv);
 }
 
 struct vmod_priv *
 VRT_priv_task(VRT_CTX, const void *vmod_id)
 {
-	uintptr_t id;
-	struct vrt_privs *vps;
-	struct vmod_priv *vp;
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	assert(ctx->req == NULL || ctx->bo == NULL ||
@@ -150,27 +145,35 @@ VRT_priv_task(VRT_CTX, const void *vmod_id)
 
 	if (ctx->req) {
 		CHECK_OBJ_NOTNULL(ctx->req, REQ_MAGIC);
-		id = (uintptr_t)ctx->req;
-		CAST_OBJ_NOTNULL(vps, ctx->req->privs, VRT_PRIVS_MAGIC);
-	} else if (ctx->bo) {
+		return (vrt_priv_dynamic(
+		    ctx->ws,
+		    ctx->req->privs,
+		    (uintptr_t)ctx->req,
+		    (uintptr_t)vmod_id
+		));
+	}
+	if (ctx->bo) {
 		CHECK_OBJ_NOTNULL(ctx->bo, BUSYOBJ_MAGIC);
-		id = (uintptr_t)ctx->bo;
-		CAST_OBJ_NOTNULL(vps, ctx->bo->privs, VRT_PRIVS_MAGIC);
-	} else {
-		ASSERT_CLI();
-		id = (uintptr_t)cli_task_privs;
-		CAST_OBJ_NOTNULL(vps, cli_task_privs, VRT_PRIVS_MAGIC);
+		return (vrt_priv_dynamic(
+		    ctx->ws,
+		    ctx->bo->privs,
+		    (uintptr_t)ctx->bo,
+		    (uintptr_t)vmod_id
+		));
 	}
-
-	vp = vrt_priv_dynamic(ctx->ws, vps, id, (uintptr_t)vmod_id);
-	return (vp);
+	ASSERT_CLI();
+
+	return (vrt_priv_dynamic(
+	    ctx->ws,
+	    cli_task_privs,
+	    (uintptr_t)cli_task_privs,
+	    (uintptr_t)vmod_id
+	));
 }
 
 struct vmod_priv *
 VRT_priv_top(VRT_CTX, const void *vmod_id)
 {
-	uintptr_t id;
-	struct vrt_privs *vps;
 	struct req *req;
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
@@ -181,9 +184,12 @@ VRT_priv_top(VRT_CTX, const void *vmod_id)
 	}
 	CHECK_OBJ_NOTNULL(ctx->req, REQ_MAGIC);
 	req = ctx->req->topreq;
-	id = (uintptr_t)&req->topreq;
-	CAST_OBJ_NOTNULL(vps, req->privs, VRT_PRIVS_MAGIC);
-	return (vrt_priv_dynamic(req->ws, vps, id, (uintptr_t)vmod_id));
+	return (vrt_priv_dynamic(
+	    req->ws,
+	    req->privs,
+	    (uintptr_t)&req->topreq,
+	    (uintptr_t)vmod_id
+	));
 }
 
 /*--------------------------------------------------------------------
@@ -204,7 +210,6 @@ VCL_TaskEnter(const struct vcl *vcl, struct vrt_privs *privs)
 {
 
 	AN(vcl);
-	AZ(privs->magic);
 	VRTPRIV_init(privs);
 }
 
@@ -214,12 +219,12 @@ VCL_TaskLeave(const struct vcl *vcl, struct vrt_privs *privs)
 	struct vrt_priv *vp, *vp1;
 
 	AN(vcl);
-	CHECK_OBJ_NOTNULL(privs, VRT_PRIVS_MAGIC);
-	/* NB: We don't bother removing entries as we finish them because it's
+	/*
+	 * NB: We don't bother removing entries as we finish them because it's
 	 * a costly operation. Instead we safely walk the whole tree and clear
 	 * the head at the very end.
 	 */
-	VRBT_FOREACH_SAFE(vp, vrt_priv_tree, &privs->privs, vp1)
+	VRBT_FOREACH_SAFE(vp, vrt_privs, privs, vp1)
 		VRT_priv_fini(vp->priv);
 	ZERO_OBJ(privs, sizeof *privs);
 }

From dridi.boukelmoune at gmail.com  Fri Nov  8 09:20:07 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Fri,  8 Nov 2019 09:20:07 +0000 (UTC)
Subject: [master] bd3b87ba2 Avoid VSB_printf for plain text
Message-ID: <20191108092007.5FC0211710E@lists.varnish-cache.org>


commit bd3b87ba204d8313e8ad58de0a23f21206d549df
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Nov 8 10:05:03 2019 +0100

    Avoid VSB_printf for plain text
    
    We have a new printf_nofmt.cocci semantic patch to catch future
    offenders.

diff --git a/bin/varnishd/cache/cache_backend_probe.c b/bin/varnishd/cache/cache_backend_probe.c
index d51da01a2..97538cade 100644
--- a/bin/varnishd/cache/cache_backend_probe.c
+++ b/bin/varnishd/cache/cache_backend_probe.c
@@ -505,7 +505,7 @@ vbp_bitmap(struct vsb *vsb, char c, uint64_t map, const char *lbl)
 	int i;
 	uint64_t u = (1ULL << 63);
 
-	VSB_printf(vsb, "  ");
+	VSB_cat(vsb, "  ");
 	for (i = 0; i < 64; i++) {
 		if (map & u)
 			VSB_putc(vsb, c);
@@ -539,7 +539,7 @@ VBP_Status(struct vsb *vsb, const struct backend *be, int details, int json)
 	}
 
 	if (json) {
-		VSB_printf(vsb, "{\n");
+		VSB_cat(vsb, "{\n");
 		VSB_indent(vsb, 2);
 #define BITMAP(nn, cc, tt, bb)					\
 		VSB_printf(vsb, "\"bits_%c\": %ju,\n", cc, vt->nn);
@@ -548,8 +548,8 @@ VBP_Status(struct vsb *vsb, const struct backend *be, int details, int json)
 		VSB_printf(vsb, "\"threshold\": %u,\n", vt->threshold);
 		VSB_printf(vsb, "\"window\": %u", vt->window);
 		VSB_indent(vsb, -2);
-		VSB_printf(vsb, "\n");
-		VSB_printf(vsb, "},\n");
+		VSB_cat(vsb, "\n");
+		VSB_cat(vsb, "},\n");
 		return;
 	}
 
@@ -558,7 +558,7 @@ VBP_Status(struct vsb *vsb, const struct backend *be, int details, int json)
 	    vt->good, vt->threshold, vt->window);
 	VSB_printf(vsb,
 	    "  Average response time of good probes: %.6f\n", vt->avg);
-	VSB_printf(vsb,
+	VSB_cat(vsb,
 	    "  Oldest ======================"
 	    "============================ Newest\n");
 
@@ -588,8 +588,8 @@ vbp_build_req(struct vbp_target *vt, const struct vrt_backend_probe *vbp,
 		    vbp->url != NULL ?  vbp->url : "/");
 		if (be->hosthdr != NULL)
 			VSB_printf(vsb, "Host: %s\r\n", be->hosthdr);
-		VSB_printf(vsb, "Connection: close\r\n");
-		VSB_printf(vsb, "\r\n");
+		VSB_cat(vsb, "Connection: close\r\n");
+		VSB_cat(vsb, "\r\n");
 	}
 	AZ(VSB_finish(vsb));
 	vt->req = strdup(VSB_data(vsb));
diff --git a/bin/varnishd/cache/cache_director.c b/bin/varnishd/cache/cache_director.c
index 0c1098da9..192347f3d 100644
--- a/bin/varnishd/cache/cache_director.c
+++ b/bin/varnishd/cache/cache_director.c
@@ -285,9 +285,9 @@ VDI_Panic(const struct director *d, struct vsb *vsb, const char *nm)
 	if (d->vdir->methods->panic != NULL)
 		d->vdir->methods->panic(d, vsb);
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 }
 
 
diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index 8fae5c619..c5ad6eb8e 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -133,13 +133,13 @@ PAN_already(struct vsb *vsb, const void *ptr)
 	int i;
 
 	if (ptr == NULL) {
-		VSB_printf(vsb, "},\n");
+		VSB_cat(vsb, "},\n");
 		return (1);
 	}
 	for (i = 0; i < already_idx; i++) {
 		if (already_list[i] == ptr) {
-			VSB_printf(vsb, "  [Already dumped, see above]\n");
-			VSB_printf(vsb, "},\n");
+			VSB_cat(vsb, "  [Already dumped, see above]\n");
+			VSB_cat(vsb, "},\n");
 			return (1);
 		}
 	}
@@ -160,7 +160,7 @@ pan_ws(struct vsb *vsb, const struct ws *ws)
 	VSB_indent(vsb, 2);
 	PAN_CheckMagic(vsb, ws, WS_MAGIC);
 	if (ws->id[0] != '\0' && (!(ws->id[0] & 0x20)))
-		VSB_printf(vsb, "OVERFLOWED ");
+		VSB_cat(vsb, "OVERFLOWED ");
 	VSB_printf(vsb, "id = \"%s\",\n", ws->id);
 	VSB_printf(vsb, "{s, f, r, e} = {%p", ws->s);
 	if (ws->f >= ws->s)
@@ -175,9 +175,9 @@ pan_ws(struct vsb *vsb, const struct ws *ws)
 		VSB_printf(vsb, ", +%ld", (long) (ws->e - ws->s));
 	else
 		VSB_printf(vsb, ", %p", ws->e);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 }
 
 /*--------------------------------------------------------------------*/
@@ -208,7 +208,7 @@ pan_htc(struct vsb *vsb, const struct http_conn *htc)
 	VSB_printf(vsb, "between_bytes_timeout = %f,\n",
 	    htc->between_bytes_timeout);
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 }
 
 /*--------------------------------------------------------------------*/
@@ -224,7 +224,7 @@ pan_http(struct vsb *vsb, const char *id, const struct http *h)
 	VSB_indent(vsb, 2);
 	PAN_CheckMagic(vsb, h, HTTP_MAGIC);
 	pan_ws(vsb, h->ws);
-	VSB_printf(vsb, "hdrs {\n");
+	VSB_cat(vsb, "hdrs {\n");
 	VSB_indent(vsb, 2);
 	for (i = 0; i < h->nhd; ++i) {
 		if (h->hd[i].b == NULL && h->hd[i].e == NULL)
@@ -233,9 +233,9 @@ pan_http(struct vsb *vsb, const char *id, const struct http *h)
 		    (int)(h->hd[i].e - h->hd[i].b), h->hd[i].b);
 	}
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 }
 
 /*--------------------------------------------------------------------*/
@@ -252,7 +252,7 @@ pan_boc(struct vsb *vsb, const struct boc *boc)
 	VSB_printf(vsb, "vary = %p,\n", boc->vary);
 	VSB_printf(vsb, "stevedore_priv = %p,\n", boc->stevedore_priv);
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 }
 
 /*--------------------------------------------------------------------*/
@@ -268,21 +268,21 @@ pan_objcore(struct vsb *vsb, const char *typ, const struct objcore *oc)
 	VSB_indent(vsb, 2);
 	PAN_CheckMagic(vsb, oc, OBJCORE_MAGIC);
 	VSB_printf(vsb, "refcnt = %d,\n", oc->refcnt);
-	VSB_printf(vsb, "flags = {");
+	VSB_cat(vsb, "flags = {");
 
 /*lint -save -esym(438,p) -esym(838,p) -e539 */
 	p = "";
 #define OC_FLAG(U, l, v) \
 	if (oc->flags & v) { VSB_printf(vsb, "%s" #l, p); p = ", "; }
 #include "tbl/oc_flags.h"
-	VSB_printf(vsb, "},\n");
-	VSB_printf(vsb, "exp_flags = {");
+	VSB_cat(vsb, "},\n");
+	VSB_cat(vsb, "exp_flags = {");
 	p = "";
 #define OC_EXP_FLAG(U, l, v) \
 	if (oc->exp_flags & v) { VSB_printf(vsb, "%s" #l, p); p = ", "; }
 #include "tbl/oc_exp_flags.h"
 /*lint -restore */
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 
 	if (oc->boc != NULL)
 		pan_boc(vsb, oc->boc);
@@ -294,18 +294,18 @@ pan_objcore(struct vsb *vsb, const char *typ, const struct objcore *oc)
 		VSB_printf(vsb, " (%s", oc->stobj->stevedore->name);
 		if (strlen(oc->stobj->stevedore->ident))
 			VSB_printf(vsb, " %s", oc->stobj->stevedore->ident);
-		VSB_printf(vsb, ")");
+		VSB_cat(vsb, ")");
 		if (oc->stobj->stevedore->panic) {
-			VSB_printf(vsb, " {\n");
+			VSB_cat(vsb, " {\n");
 			VSB_indent(vsb, 2);
 			oc->stobj->stevedore->panic(vsb, oc);
 			VSB_indent(vsb, -2);
-			VSB_printf(vsb, "}");
+			VSB_cat(vsb, "}");
 		}
 	}
-	VSB_printf(vsb, ",\n");
+	VSB_cat(vsb, ",\n");
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 }
 
 /*--------------------------------------------------------------------*/
@@ -325,13 +325,13 @@ pan_wrk(struct vsb *vsb, const struct worker *wrk)
 	pan_ws(vsb, wrk->aws);
 
 	m = wrk->cur_method;
-	VSB_printf(vsb, "VCL::method = ");
+	VSB_cat(vsb, "VCL::method = ");
 	if (m == 0) {
-		VSB_printf(vsb, "none,\n");
+		VSB_cat(vsb, "none,\n");
 		return;
 	}
 	if (!(m & 1))
-		VSB_printf(vsb, "inside ");
+		VSB_cat(vsb, "inside ");
 	m &= ~1;
 	hand = VCL_Method_Name(m);
 	if (hand != NULL)
@@ -344,7 +344,7 @@ pan_wrk(struct vsb *vsb, const struct worker *wrk)
 		VSB_printf(vsb, "VCL::return = %s,\n", hand);
 	else
 		VSB_printf(vsb, "VCL::return = 0x%x,\n", wrk->handling);
-	VSB_printf(vsb, "VCL::methods = {");
+	VSB_cat(vsb, "VCL::methods = {");
 	m = wrk->seen_methods;
 	p = "";
 	for (u = 1; m ; u <<= 1) {
@@ -354,9 +354,9 @@ pan_wrk(struct vsb *vsb, const struct worker *wrk)
 			p = ", ";
 		}
 	}
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 }
 
 static void
@@ -374,7 +374,7 @@ pan_vfp(struct vsb *vsb, const struct vfp_ctx *vfc)
 	VSB_printf(vsb, "oc = %p,\n", vfc->oc);
 
 	if (!VTAILQ_EMPTY(&vfc->vfp)) {
-		VSB_printf(vsb, "filters = {\n");
+		VSB_cat(vsb, "filters = {\n");
 		VSB_indent(vsb, 2);
 		VTAILQ_FOREACH(vfe, &vfc->vfp, list) {
 			VSB_printf(vsb, "%s = %p {\n", vfe->vfp->name, vfe);
@@ -383,15 +383,15 @@ pan_vfp(struct vsb *vsb, const struct vfp_ctx *vfc)
 			VSB_printf(vsb, "priv2 = %zd,\n", vfe->priv2);
 			VSB_printf(vsb, "closed = %d\n", vfe->closed);
 			VSB_indent(vsb, -2);
-			VSB_printf(vsb, "},\n");
+			VSB_cat(vsb, "},\n");
 		}
 		VSB_indent(vsb, -2);
-		VSB_printf(vsb, "},\n");
+		VSB_cat(vsb, "},\n");
 	}
 
 	VSB_printf(vsb, "obj_flags = 0x%x,\n", vfc->obj_flags);
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 }
 
 static void
@@ -406,17 +406,17 @@ pan_vdp(struct vsb *vsb, const struct vdp_ctx *vdc)
 	VSB_printf(vsb, "retval = %d,\n", vdc->retval);
 
 	if (!VTAILQ_EMPTY(&vdc->vdp)) {
-		VSB_printf(vsb, "filters = {\n");
+		VSB_cat(vsb, "filters = {\n");
 		VSB_indent(vsb, 2);
 		VTAILQ_FOREACH(vde, &vdc->vdp, list)
 			VSB_printf(vsb, "%s = %p { priv = %p }\n",
 			    vde->vdp->name, vde, vde->priv);
 		VSB_indent(vsb, -2);
-		VSB_printf(vsb, "},\n");
+		VSB_cat(vsb, "},\n");
 	}
 
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 }
 
 static void
@@ -462,25 +462,25 @@ pan_busyobj(struct vsb *vsb, const struct busyobj *bo)
 
 	// fetch_task left out
 
-	VSB_printf(vsb, "flags = {");
+	VSB_cat(vsb, "flags = {");
 	p = "";
 /*lint -save -esym(438,p) -e539 */
 #define BO_FLAG(l, r, w, d) \
 	if (bo->l) { VSB_printf(vsb, "%s" #l, p); p = ", "; }
 #include "tbl/bo_flags.h"
 /*lint -restore */
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 
 	// timeouts/timers/acct/storage left out
 
 	VDI_Panic(bo->director_req, vsb, "director_req");
 	if (bo->director_resp == bo->director_req)
-		VSB_printf(vsb, "director_resp = director_req,\n");
+		VSB_cat(vsb, "director_resp = director_req,\n");
 	else
 		VDI_Panic(bo->director_resp, vsb, "director_resp");
 	VCL_Panic(vsb, "vcl", bo->vcl);
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 }
 
 /*--------------------------------------------------------------------*/
@@ -501,13 +501,13 @@ pan_req(struct vsb *vsb, const struct req *req)
 	    xp == NULL ? "NULL" : xp->name);
 
 	if (xp != NULL && xp->req_panic != NULL) {
-		VSB_printf(vsb, " {\n");
+		VSB_cat(vsb, " {\n");
 		VSB_indent(vsb, 2);
 		xp->req_panic(vsb, req);
 		VSB_indent(vsb, -2);
-		VSB_printf(vsb, "}");
+		VSB_cat(vsb, "}");
 	}
-	VSB_printf(vsb, "\n");
+	VSB_cat(vsb, "\n");
 	switch (req->req_step) {
 #define REQ_STEP(l, u, arg) case R_STP_##u: stp = "R_STP_" #u; break;
 #include "tbl/steps.h"
@@ -552,23 +552,23 @@ pan_req(struct vsb *vsb, const struct req *req)
 	if (req->objcore != NULL)
 		pan_objcore(vsb, "REQ", req->objcore);
 
-	VSB_printf(vsb, "flags = {\n");
+	VSB_cat(vsb, "flags = {\n");
 	VSB_indent(vsb, 2);
 #define REQ_FLAG(l, r, w, d) if (req->l) VSB_printf(vsb, #l ",\n");
 #include "tbl/req_flags.h"
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 
 	pan_privs(vsb, req->privs);
 
-	VSB_printf(vsb, "topreq = {\n");
+	VSB_cat(vsb, "topreq = {\n");
 	VSB_indent(vsb, 2);
 	pan_req(vsb, req->topreq);
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 }
 
 /*--------------------------------------------------------------------*/
@@ -594,13 +594,13 @@ pan_sess(struct vsb *vsb, const struct sess *sp)
 	VSB_printf(vsb, "transport = %s",
 	    xp == NULL ? "<none>" : xp->name);
 	if (xp != NULL && xp->sess_panic != NULL) {
-		VSB_printf(vsb, " {\n");
+		VSB_cat(vsb, " {\n");
 		VSB_indent(vsb, 2);
 		xp->sess_panic(vsb, sp);
 		VSB_indent(vsb, -2);
-		VSB_printf(vsb, "}");
+		VSB_cat(vsb, "}");
 	}
-	VSB_printf(vsb, "\n");
+	VSB_cat(vsb, "\n");
 	ci = SES_Get_String_Attr(sp, SA_CLIENT_IP);
 	cp = SES_Get_String_Attr(sp, SA_CLIENT_PORT);
 	if (VALID_OBJ(sp->listen_sock, LISTEN_SOCK_MAGIC))
@@ -610,7 +610,7 @@ pan_sess(struct vsb *vsb, const struct sess *sp)
 		VSB_printf(vsb, "client = %s %s <unknown>\n", ci, cp);
 
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 }
 
 /*--------------------------------------------------------------------*/
@@ -626,7 +626,7 @@ pan_backtrace(struct vsb *vsb)
 	char fname[1024];
 	int ret;
 
-	VSB_printf(vsb, "Backtrace:\n");
+	VSB_cat(vsb, "Backtrace:\n");
 	VSB_indent(vsb, 2);
 
 	ret = unw_getcontext(&uc);
@@ -673,14 +673,14 @@ pan_backtrace(struct vsb *vsb)
 		VSB_printf(vsb, "Backtrace not available (ret=%zu)\n", size);
 		return;
 	}
-	VSB_printf(vsb, "Backtrace:\n");
+	VSB_cat(vsb, "Backtrace:\n");
 	VSB_indent(vsb, 2);
 	for (i = 0; i < size; i++) {
 		bprintf(buf, "%p", array[i]);
 		VSB_printf(vsb, "%s: ", buf);
 		strings = backtrace_symbols(&array[i], 1);
 		if (strings == NULL || strings[0] == NULL) {
-			VSB_printf(vsb, "(?)");
+			VSB_cat(vsb, "(?)");
 		} else {
 			p = strings[0];
 			if (!memcmp(buf, p, strlen(buf))) {
@@ -692,7 +692,7 @@ pan_backtrace(struct vsb *vsb)
 			}
 			VSB_printf(vsb, "%s", p);
 		}
-		VSB_printf (vsb, "\n");
+		VSB_cat(vsb, "\n");
 		free(strings);
 	}
 	VSB_indent(vsb, -2);
@@ -816,10 +816,9 @@ pan_ic(const char *func, const char *file, int line, const char *cond,
 			VSL_Flush(bo->vsl, 0);
 		VMOD_Panic(pan_vsb);
 	} else {
-		VSB_printf(pan_vsb,
-		    "Feature short panic supressed details.\n");
+		VSB_cat(pan_vsb, "Feature short panic supressed details.\n");
 	}
-	VSB_printf(pan_vsb, "\n");
+	VSB_cat(pan_vsb, "\n");
 	VSB_putc(pan_vsb, '\0');	/* NUL termination */
 
 	if (FEATURE(FEATURE_NO_COREDUMP))
@@ -860,7 +859,7 @@ PAN_Init(void)
 	AN(heritage.panic_str_len);
 	AN(VSB_new(pan_vsb, heritage.panic_str, heritage.panic_str_len,
 	    VSB_FIXEDLEN));
-	VSB_printf(pan_vsb, "This is a test\n");
+	VSB_cat(pan_vsb, "This is a test\n");
 	AZ(VSB_finish(pan_vsb));
 	VSB_clear(pan_vsb);
 	heritage.panic_str[0] = '\0';
diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index b0ee19899..f9072c068 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -222,24 +222,24 @@ VCL_Panic(struct vsb *vsb, const char *nm, const struct vcl *vcl)
 	VSB_printf(vsb, "discard = %u,\n", vcl->discard);
 	VSB_printf(vsb, "state = %s,\n", vcl->state);
 	VSB_printf(vsb, "temp = %s,\n", (const volatile char *)vcl->temp);
-	VSB_printf(vsb, "conf = {\n");
+	VSB_cat(vsb, "conf = {\n");
 	VSB_indent(vsb, 2);
 	if (vcl->conf == NULL) {
-		VSB_printf(vsb, "conf = NULL\n");
+		VSB_cat(vsb, "conf = NULL\n");
 	} else {
 		PAN_CheckMagic(vsb, vcl->conf, VCL_CONF_MAGIC);
 		VSB_printf(vsb, "syntax = \"%u\",\n", vcl->conf->syntax);
-		VSB_printf(vsb, "srcname = {\n");
+		VSB_cat(vsb, "srcname = {\n");
 		VSB_indent(vsb, 2);
 		for (i = 0; i < vcl->conf->nsrc; ++i)
 			VSB_printf(vsb, "\"%s\",\n", vcl->conf->srcname[i]);
 		VSB_indent(vsb, -2);
-		VSB_printf(vsb, "},\n");
+		VSB_cat(vsb, "},\n");
 	}
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 }
 
 /*--------------------------------------------------------------------*/
@@ -388,18 +388,18 @@ VCL_Open(const char *fn, struct vsb *msg)
 #endif
 	dlh = dlopen(fn, RTLD_NOW | RTLD_LOCAL);
 	if (dlh == NULL) {
-		VSB_printf(msg, "Could not load compiled VCL.\n");
+		VSB_cat(msg, "Could not load compiled VCL.\n");
 		VSB_printf(msg, "\tdlopen() = %s\n", dlerror());
 		return (NULL);
 	}
 	cnf = dlsym(dlh, "VCL_conf");
 	if (cnf == NULL) {
-		VSB_printf(msg, "Compiled VCL lacks metadata.\n");
+		VSB_cat(msg, "Compiled VCL lacks metadata.\n");
 		(void)dlclose(dlh);
 		return (NULL);
 	}
 	if (cnf->magic != VCL_CONF_MAGIC) {
-		VSB_printf(msg, "Compiled VCL has mangled metadata.\n");
+		VSB_cat(msg, "Compiled VCL has mangled metadata.\n");
 		(void)dlclose(dlh);
 		return (NULL);
 	}
diff --git a/bin/varnishd/cache/cache_vrt_priv.c b/bin/varnishd/cache/cache_vrt_priv.c
index c2c46d4a9..d8694ccdd 100644
--- a/bin/varnishd/cache/cache_vrt_priv.c
+++ b/bin/varnishd/cache/cache_vrt_priv.c
@@ -78,7 +78,7 @@ pan_privs(struct vsb *vsb, const struct vrt_privs *privs)
 		}
 	}
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 }
 
 
diff --git a/bin/varnishd/cache/cache_vrt_vmod.c b/bin/varnishd/cache/cache_vrt_vmod.c
index bfc9496c5..c760606bc 100644
--- a/bin/varnishd/cache/cache_vrt_vmod.c
+++ b/bin/varnishd/cache/cache_vrt_vmod.c
@@ -118,7 +118,7 @@ VPI_Vmod_Init(VRT_CTX, struct vmod **hdl, unsigned nbr, void *ptr, int len,
 		    strcmp(d->file_id, file_id)) {
 			VSB_printf(ctx->msg, "Loading vmod %s from %s (%s):\n",
 			    nm, backup, path);
-			VSB_printf(ctx->msg,
+			VSB_cat(ctx->msg,
 			    "This is no longer the same file seen by"
 			    " the VCL-compiler.\n");
 			(void)dlclose(v->hdl);
@@ -134,7 +134,7 @@ VPI_Vmod_Init(VRT_CTX, struct vmod **hdl, unsigned nbr, void *ptr, int len,
 		    d->json == NULL) {
 			VSB_printf(ctx->msg, "Loading vmod %s from %s (%s):\n",
 			    nm, backup, path);
-			VSB_printf(ctx->msg, "VMOD data is mangled.\n");
+			VSB_cat(ctx->msg, "VMOD data is mangled.\n");
 			(void)dlclose(v->hdl);
 			FREE_OBJ(v);
 			return (1);
@@ -197,13 +197,13 @@ VMOD_Panic(struct vsb *vsb)
 {
 	struct vmod *v;
 
-	VSB_printf(vsb, "vmods = {\n");
+	VSB_cat(vsb, "vmods = {\n");
 	VSB_indent(vsb, 2);
 	VTAILQ_FOREACH(v, &vmods, list)
 		VSB_printf(vsb, "%s = {%p, %s, %u.%u},\n",
 		    v->nm, v, v->abi, v->vrt_major, v->vrt_minor);
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "},\n");
+	VSB_cat(vsb, "},\n");
 }
 
 /*---------------------------------------------------------------------*/
diff --git a/bin/varnishd/http2/cache_http2_panic.c b/bin/varnishd/http2/cache_http2_panic.c
index 9e80eb67b..122e43d83 100644
--- a/bin/varnishd/http2/cache_http2_panic.c
+++ b/bin/varnishd/http2/cache_http2_panic.c
@@ -45,7 +45,7 @@ h2_sess_panic(struct vsb *vsb, const struct sess *sp)
 
 	h2 = (void*)*up;
 	CHECK_OBJ_NOTNULL(h2, H2_SESS_MAGIC);
-	VSB_printf(vsb, "streams {\n");
+	VSB_cat(vsb, "streams {\n");
 	VSB_indent(vsb, 2);
 	VTAILQ_FOREACH(r2, &h2->streams, list) {
 		PAN_CheckMagic(vsb, r2, H2_REQ_MAGIC);
@@ -57,8 +57,8 @@ h2_sess_panic(struct vsb *vsb, const struct sess *sp)
 			VSB_printf(vsb, " State %d", r2->state);
 			break;
 		}
-		VSB_printf(vsb, "\n");
+		VSB_cat(vsb, "\n");
 	}
 	VSB_indent(vsb, -2);
-	VSB_printf(vsb, "}\n");
+	VSB_cat(vsb, "}\n");
 }
diff --git a/bin/varnishd/mgt/mgt_child.c b/bin/varnishd/mgt/mgt_child.c
index b28a26953..ca6037706 100644
--- a/bin/varnishd/mgt/mgt_child.c
+++ b/bin/varnishd/mgt/mgt_child.c
@@ -541,7 +541,7 @@ mgt_reap_child(void)
 	}
 #ifdef WCOREDUMP
 	if (WCOREDUMP(status)) {
-		VSB_printf(vsb, " (core dumped)");
+		VSB_cat(vsb, " (core dumped)");
 		exit_status |= 0x80;
 		VSC_C_mgt->child_dump++;
 	}
diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c
index 02b9c7ed0..b4407db14 100644
--- a/bin/varnishd/mgt/mgt_main.c
+++ b/bin/varnishd/mgt/mgt_main.c
@@ -590,15 +590,15 @@ main(int argc, char * const *argv)
 			REPLACE(fa->farg, "<-b argument>");
 			vsb = VSB_new_auto();
 			AN(vsb);
-			VSB_printf(vsb, "vcl 4.1;\n");
-			VSB_printf(vsb, "backend default {\n");
+			VSB_cat(vsb, "vcl 4.1;\n");
+			VSB_cat(vsb, "backend default {\n");
 			if (*optarg != '/')
 				VSB_printf(vsb, "    .host = \"%s\";\n",
 					   optarg);
 			else
 				VSB_printf(vsb, "    .path = \"%s\";\n",
 					   optarg);
-			VSB_printf(vsb, "}\n");
+			VSB_cat(vsb, "}\n");
 			AZ(VSB_finish(vsb));
 			fa->src = strdup(VSB_data(vsb));
 			AN(fa->src);
@@ -710,8 +710,8 @@ main(int argc, char * const *argv)
 			if (getenv("TMPDIR") != NULL)
 				VSB_printf(vsb, "%s", getenv("TMPDIR"));
 			else
-				VSB_printf(vsb, "/tmp");
-			VSB_printf(vsb, "/varnishd_C_XXXXXXX");
+				VSB_cat(vsb, "/tmp");
+			VSB_cat(vsb, "/varnishd_C_XXXXXXX");
 			AZ(VSB_finish(vsb));
 			Cn_arg = strdup(VSB_data(vsb));
 			AN(Cn_arg);
diff --git a/bin/varnishd/mgt/mgt_param_bits.c b/bin/varnishd/mgt/mgt_param_bits.c
index df2ca2320..279994a73 100644
--- a/bin/varnishd/mgt/mgt_param_bits.c
+++ b/bin/varnishd/mgt/mgt_param_bits.c
@@ -151,7 +151,7 @@ tweak_vsl_mask(struct vsb *vsb, const struct parspec *par, const char *arg)
 			}
 		}
 		if (*s == '\0')
-			VSB_printf(vsb, "(all enabled)");
+			VSB_cat(vsb, "(all enabled)");
 		if (arg == JSON_FMT)
 			VSB_putc(vsb, '"');
 	}
@@ -194,7 +194,7 @@ tweak_debug(struct vsb *vsb, const struct parspec *par, const char *arg)
 			}
 		}
 		if (*s == '\0')
-			VSB_printf(vsb, "none");
+			VSB_cat(vsb, "none");
 		if (arg == JSON_FMT)
 			VSB_putc(vsb, '"');
 	}
@@ -238,7 +238,7 @@ tweak_feature(struct vsb *vsb, const struct parspec *par, const char *arg)
 			}
 		}
 		if (*s == '\0')
-			VSB_printf(vsb, "none");
+			VSB_cat(vsb, "none");
 		if (arg == JSON_FMT)
 			VSB_putc(vsb, '"');
 	}
diff --git a/bin/varnishd/mgt/mgt_param_tweak.c b/bin/varnishd/mgt/mgt_param_tweak.c
index 30e58e2fc..78f95d3ff 100644
--- a/bin/varnishd/mgt/mgt_param_tweak.c
+++ b/bin/varnishd/mgt/mgt_param_tweak.c
@@ -143,7 +143,7 @@ tweak_bool(struct vsb *vsb, const struct parspec *par, const char *arg)
 		else if (!strcasecmp(arg, "true"))
 			*dest = 1;
 		else {
-			VSB_printf(vsb, "use \"on\" or \"off\"\n");
+			VSB_cat(vsb, "use \"on\" or \"off\"\n");
 			return (-1);
 		}
 	} else if (arg == JSON_FMT) {
@@ -207,7 +207,7 @@ tweak_generic_uint(struct vsb *vsb, volatile unsigned *dest, const char *arg,
 		}
 		*dest = u;
 	} else if (*dest == UINT_MAX && arg != JSON_FMT) {
-		VSB_printf(vsb, "unlimited");
+		VSB_cat(vsb, "unlimited");
 	} else {
 		VSB_printf(vsb, "%u", *dest);
 	}
@@ -248,7 +248,7 @@ fmt_bytes(struct vsb *vsb, uintmax_t t)
 			return;
 		}
 	}
-	VSB_printf(vsb, "(bogus number)");
+	VSB_cat(vsb, "(bogus number)");
 }
 
 static int
@@ -275,21 +275,19 @@ tweak_generic_bytes(struct vsb *vsb, volatile ssize_t *dest, const char *arg,
 		}
 		p = VNUM_2bytes(arg, &r, 0);
 		if (p != NULL) {
-			VSB_printf(vsb, "Could not convert to bytes.\n");
+			VSB_cat(vsb, "Could not convert to bytes.\n");
 			VSB_printf(vsb, "%s\n", p);
-			VSB_printf(vsb,
-			    "  Try something like '80k' or '120M'\n");
+			VSB_cat(vsb, "  Try something like '80k' or '120M'\n");
 			return (-1);
 		}
 		if ((uintmax_t)((ssize_t)r) != r) {
 			fmt_bytes(vsb, r);
-			VSB_printf(vsb,
-			    " is too large for this architecture.\n");
+			VSB_cat(vsb, " is too large for this architecture.\n");
 			return (-1);
 		}
 		if (max != NULL && r > rmax) {
 			VSB_printf(vsb, "Must be no more than %s\n", max);
-			VSB_printf(vsb, "\n");
+			VSB_cat(vsb, "\n");
 			return (-1);
 		}
 		if (min != NULL && r < rmin) {
@@ -396,13 +394,13 @@ tweak_poolparam(struct vsb *vsb, const struct parspec *par, const char *arg)
 
 	pp = par->priv;
 	if (arg == JSON_FMT) {
-		VSB_printf(vsb, "{\n");
+		VSB_cat(vsb, "{\n");
 		VSB_indent(vsb, 8);
 		VSB_printf(vsb, "\"min_pool\": %u,\n", pp->min_pool);
 		VSB_printf(vsb, "\"max_pool\": %u,\n", pp->max_pool);
 		VSB_printf(vsb, "\"max_age\": %g\n", pp->max_age);
 		VSB_indent(vsb, -4);
-		VSB_printf(vsb, "}");
+		VSB_cat(vsb, "}");
 	} else if (arg == NULL) {
 		VSB_printf(vsb, "%u,%u,%g",
 		    pp->min_pool, pp->max_pool, pp->max_age);
@@ -415,7 +413,7 @@ tweak_poolparam(struct vsb *vsb, const struct parspec *par, const char *arg)
 				break;
 			}
 			if (av[1] == NULL || av[2] == NULL || av[3] == NULL) {
-				VSB_printf(vsb,
+				VSB_cat(vsb,
 				    "Three fields required:"
 				    " min_pool, max_pool and max_age\n");
 				retval = -1;
@@ -437,7 +435,7 @@ tweak_poolparam(struct vsb *vsb, const struct parspec *par, const char *arg)
 			if (retval)
 				break;
 			if (px.min_pool > px.max_pool) {
-				VSB_printf(vsb,
+				VSB_cat(vsb,
 				    "min_pool cannot be larger"
 				    " than max_pool\n");
 				retval = -1;
diff --git a/bin/varnishd/mgt/mgt_vcc.c b/bin/varnishd/mgt/mgt_vcc.c
index 58b4ec152..a8997b32e 100644
--- a/bin/varnishd/mgt/mgt_vcc.c
+++ b/bin/varnishd/mgt/mgt_vcc.c
@@ -237,11 +237,11 @@ mgt_vcc_compile(struct vcc_priv *vp, struct vsb *sb, int C_flag)
 		VSB_cat(sb, csrc);
 		free(csrc);
 
-		VSB_printf(sb, "/* EXTERNAL SYMBOL TABLE\n");
+		VSB_cat(sb, "/* EXTERNAL SYMBOL TABLE\n");
 		csrc = VFIL_readfile(NULL, VSB_data(vp->symfile), NULL);
 		AN(csrc);
 		VSB_cat(sb, csrc);
-		VSB_printf(sb, "*/\n");
+		VSB_cat(sb, "*/\n");
 		free(csrc);
 	}
 
diff --git a/bin/varnishd/proxy/cache_proxy_proto.c b/bin/varnishd/proxy/cache_proxy_proto.c
index ee9c19601..bb142f568 100644
--- a/bin/varnishd/proxy/cache_proxy_proto.c
+++ b/bin/varnishd/proxy/cache_proxy_proto.c
@@ -618,9 +618,9 @@ VPX_Send_Proxy(int fd, int version, const struct sess *sp)
 		AN(p2);
 		VTCP_name(sas, ha, sizeof ha, pa, sizeof pa);
 		if (proto == PF_INET6)
-			VSB_printf(vsb, " TCP6 ");
+			VSB_cat(vsb, " TCP6 ");
 		else if (proto == PF_INET)
-			VSB_printf(vsb, " TCP4 ");
+			VSB_cat(vsb, " TCP4 ");
 		VSB_printf(vsb, "%s %s %s %s\r\n", p1, ha, p2, pa);
 	} else if (version == 2) {
 		AZ(SES_Get_client_addr(sp, &sac));
diff --git a/lib/libvarnish/vsb.c b/lib/libvarnish/vsb.c
index f0e26e2f4..2adf50d02 100644
--- a/lib/libvarnish/vsb.c
+++ b/lib/libvarnish/vsb.c
@@ -525,9 +525,9 @@ VSB_quote_pfx(struct vsb *s, const char *pfx, const void *v, int len, int how)
 		for (w = u; w < u + len; w++)
 			if (*w != 0x00)
 				break;
-		VSB_printf(s, "0x");
+		VSB_cat(s, "0x");
 		if (w == u + len && len > 4) {
-			VSB_printf(s, "0...0");
+			VSB_cat(s, "0...0");
 		} else {
 			for (w = u; w < u + len; w++)
 				VSB_printf(s, "%02x", *w);
diff --git a/lib/libvarnish/vsub.c b/lib/libvarnish/vsub.c
index 3a497c9c4..515169d82 100644
--- a/lib/libvarnish/vsub.c
+++ b/lib/libvarnish/vsub.c
@@ -178,8 +178,8 @@ VSUB_run(struct vsb *sb, vsub_func_f *func, void *priv, const char *name,
 			VSB_printf(sb, ", signal %d", WTERMSIG(status));
 		}
 		if (WCOREDUMP(status))
-			VSB_printf(sb, ", core dumped");
-		VSB_printf(sb, "\n");
+			VSB_cat(sb, ", core dumped");
+		VSB_cat(sb, "\n");
 		assert(rv != -1);
 		return (rv);
 	}
diff --git a/lib/libvarnishapi/vxp.c b/lib/libvarnishapi/vxp.c
index 5c5049827..4b23a8882 100644
--- a/lib/libvarnishapi/vxp.c
+++ b/lib/libvarnishapi/vxp.c
@@ -50,7 +50,7 @@ vxp_ErrToken(const struct vxp *vxp, const struct token *t)
 {
 
 	if (t->tok == EOI)
-		VSB_printf(vxp->sb, "end of input");
+		VSB_cat(vxp->sb, "end of input");
 	else
 		VSB_printf(vxp->sb, "'%.*s'", PF(t));
 }
@@ -120,7 +120,7 @@ vxp_NextToken(struct vxp *vxp)
 	AN(vxp->t);
 	vxp->t = VTAILQ_NEXT(vxp->t, list);
 	if (vxp->t == NULL) {
-		VSB_printf(vxp->sb,
+		VSB_cat(vxp->sb,
 		    "Ran out of input, something is missing or"
 		    " maybe unbalanced parenthesis\n");
 		vxp->err = 1;
diff --git a/lib/libvarnishapi/vxp_lexer.c b/lib/libvarnishapi/vxp_lexer.c
index b1e952a12..49a8dfed4 100644
--- a/lib/libvarnishapi/vxp_lexer.c
+++ b/lib/libvarnishapi/vxp_lexer.c
@@ -145,7 +145,7 @@ vxp_Lexer(struct vxp *vxp)
 			}
 			vxp_append_token(vxp, VAL, p, q);
 			if (quote != '\0') {
-				VSB_printf(vxp->sb, "Unterminated string ");
+				VSB_cat(vxp->sb, "Unterminated string ");
 				vxp_ErrWhere(vxp, vxp->t, q - p);
 				return;
 			}
@@ -177,7 +177,7 @@ vxp_Lexer(struct vxp *vxp)
 
 		/* Error */
 		vxp_append_token(vxp, EOI, p, p + 1);
-		VSB_printf(vxp->sb, "Syntax error ");
+		VSB_cat(vxp->sb, "Syntax error ");
 		vxp_ErrWhere(vxp, vxp->t, q - p);
 		return;
 	}
diff --git a/lib/libvarnishapi/vxp_parse.c b/lib/libvarnishapi/vxp_parse.c
index 16f65f42f..b11e8dd50 100644
--- a/lib/libvarnishapi/vxp_parse.c
+++ b/lib/libvarnishapi/vxp_parse.c
@@ -86,7 +86,7 @@ vxp_expr_lhs(struct vxp *vxp, struct vex_lhs **plhs)
 		}
 		(*plhs)->level = (int)strtol(vxp->t->dec, &p, 0);
 		if ((*plhs)->level < 0) {
-			VSB_printf(vxp->sb, "Expected positive integer ");
+			VSB_cat(vxp->sb, "Expected positive integer ");
 			vxp_ErrWhere(vxp, vxp->t, -1);
 			return;
 		}
@@ -98,7 +98,7 @@ vxp_expr_lhs(struct vxp *vxp, struct vex_lhs **plhs)
 			p++;
 		}
 		if (*p) {
-			VSB_printf(vxp->sb, "Syntax error in level limit ");
+			VSB_cat(vxp->sb, "Syntax error in level limit ");
 			vxp_ErrWhere(vxp, vxp->t, -1);
 			return;
 		}
@@ -123,17 +123,17 @@ vxp_expr_lhs(struct vxp *vxp, struct vex_lhs **plhs)
 			    (*plhs)->tags);
 		}
 		if (i == -1) {
-			VSB_printf(vxp->sb, "Tag name matches zero tags ");
+			VSB_cat(vxp->sb, "Tag name matches zero tags ");
 			vxp_ErrWhere(vxp, vxp->t, -1);
 			return;
 		}
 		if (i == -2) {
-			VSB_printf(vxp->sb, "Tag name is ambiguous ");
+			VSB_cat(vxp->sb, "Tag name is ambiguous ");
 			vxp_ErrWhere(vxp, vxp->t, -1);
 			return;
 		}
 		if (i == -3) {
-			VSB_printf(vxp->sb, "Syntax error in tag name ");
+			VSB_cat(vxp->sb, "Syntax error in tag name ");
 			vxp_ErrWhere(vxp, vxp->t, -1);
 			return;
 		}
@@ -171,7 +171,7 @@ vxp_expr_lhs(struct vxp *vxp, struct vex_lhs **plhs)
 		}
 		(*plhs)->field = (int)strtol(vxp->t->dec, &p, 0);
 		if (*p || (*plhs)->field <= 0) {
-			VSB_printf(vxp->sb, "Expected positive integer ");
+			VSB_cat(vxp->sb, "Expected positive integer ");
 			vxp_ErrWhere(vxp, vxp->t, -1);
 			return;
 		}
@@ -186,7 +186,7 @@ vxp_expr_lhs(struct vxp *vxp, struct vex_lhs **plhs)
 	if ((*plhs)->vxid > 1 || (*plhs)->level >= 0 ||
 	    (*plhs)->field > 0 || (*plhs)->prefixlen > 0 ||
 	    (*plhs)->taglist > 0) {
-		VSB_printf(vxp->sb, "Unexpected taglist selection for vxid ");
+		VSB_cat(vxp->sb, "Unexpected taglist selection for vxid ");
 		vxp_ErrWhere(vxp, vxp->t, -1);
 	}
 }
@@ -210,7 +210,7 @@ vxp_expr_num(struct vxp *vxp, struct vex_rhs **prhs, unsigned vxid)
 		(*prhs)->type = VEX_FLOAT;
 		(*prhs)->val_float = VNUM(vxp->t->dec);
 		if (isnan((*prhs)->val_float)) {
-			VSB_printf(vxp->sb, "Floating point parse error ");
+			VSB_cat(vxp->sb, "Floating point parse error ");
 			vxp_ErrWhere(vxp, vxp->t, -1);
 			return;
 		}
@@ -220,7 +220,7 @@ vxp_expr_num(struct vxp *vxp, struct vex_rhs **prhs, unsigned vxid)
 		while (isspace(*endptr))
 			endptr++;
 		if (*endptr != '\0') {
-			VSB_printf(vxp->sb, "Integer parse error ");
+			VSB_cat(vxp->sb, "Integer parse error ");
 			vxp_ErrWhere(vxp, vxp->t, -1);
 			return;
 		}
diff --git a/lib/libvcc/vcc_acl.c b/lib/libvcc/vcc_acl.c
index 023a73d52..da3f282d1 100644
--- a/lib/libvcc/vcc_acl.c
+++ b/lib/libvcc/vcc_acl.c
@@ -135,9 +135,9 @@ vcc_acl_add_entry(struct vcc *tl, const struct acl_e *ae, int l,
 			 */
 			if (aen->not == ae2->not)
 				return;
-			VSB_printf(tl->sb, "Conflicting ACL entries:\n");
+			VSB_cat(tl->sb, "Conflicting ACL entries:\n");
 			vcc_ErrWhere(tl, ae2->t_addr);
-			VSB_printf(tl->sb, "vs:\n");
+			VSB_cat(tl->sb, "vs:\n");
 			vcc_ErrWhere(tl, aen->t_addr);
 			return;
 		}
@@ -311,13 +311,13 @@ vcc_acl_entry(struct vcc *tl)
 		e = NULL;
 		ae->mask = strtoul(sl, &e, 10);
 		if (*e != '\0') {
-			VSB_printf(tl->sb, ".../mask is not numeric.\n");
+			VSB_cat(tl->sb, ".../mask is not numeric.\n");
 			vcc_ErrWhere(tl, ae->t_addr);
 			return;
 		}
 		ae->t_mask = ae->t_addr;
 		if (tl->t->tok == '/') {
-			VSB_printf(tl->sb, "/mask only allowed once.\n");
+			VSB_cat(tl->sb, "/mask only allowed once.\n");
 			vcc_ErrWhere(tl, tl->t);
 			return;
 		}
diff --git a/lib/libvcc/vcc_action.c b/lib/libvcc/vcc_action.c
index c243dbcb7..a8f3ca20a 100644
--- a/lib/libvcc/vcc_action.c
+++ b/lib/libvcc/vcc_action.c
@@ -273,11 +273,11 @@ vcc_act_return_vcl(struct vcc *tl)
 	ERRCHK(tl);
 	AN(sym);
 	if (sym->eval_priv == NULL) {
-		VSB_printf(tl->symtab, ",\n    {\n");
-		VSB_printf(tl->symtab, "\t\"dir\": \"import\",\n");
-		VSB_printf(tl->symtab, "\t\"type\": \"$VCL\",\n");
+		VSB_cat(tl->symtab, ",\n    {\n");
+		VSB_cat(tl->symtab, "\t\"dir\": \"import\",\n");
+		VSB_cat(tl->symtab, "\t\"type\": \"$VCL\",\n");
 		VSB_printf(tl->symtab, "\t\"name\": \"%s\"\n", sym->name);
-		VSB_printf(tl->symtab, "   }");
+		VSB_cat(tl->symtab, "   }");
 
 		bprintf(buf, "vgc_vcl_%u", tl->unique++);
 		sym->eval_priv = strdup(buf);
diff --git a/lib/libvcc/vcc_backend.c b/lib/libvcc/vcc_backend.c
index b943cf5fd..c8f5ee8cd 100644
--- a/lib/libvcc/vcc_backend.c
+++ b/lib/libvcc/vcc_backend.c
@@ -41,9 +41,9 @@ vcc_default_probe(struct vcc *tl)
 
 	if (tl->default_probe != NULL)
 		return (tl->default_probe);
-	VSB_printf(tl->sb, "No default probe defined\n");
+	VSB_cat(tl->sb, "No default probe defined\n");
 	vcc_ErrToken(tl, tl->t);
-	VSB_printf(tl->sb, " at\n");
+	VSB_cat(tl->sb, " at\n");
 	vcc_ErrWhere(tl, tl->t);
 	return ("");
 }
@@ -95,7 +95,7 @@ vcc_Redef(struct vcc *tl, const char *redef, struct token **t_did,
 	if (*t_did != NULL) {
 		VSB_printf(tl->sb, "%s redefinition at:\n", redef);
 		vcc_ErrWhere(tl, t_field);
-		VSB_printf(tl->sb, "Previous definition:\n");
+		VSB_cat(tl->sb, "Previous definition:\n");
 		vcc_ErrWhere(tl, *t_did);
 		return;
 	}
@@ -195,7 +195,7 @@ vcc_ParseProbeSpec(struct vcc *tl, const struct symbol *sym, char **name)
 		} else if (vcc_IdIs(t_field, "expected_response")) {
 			status = vcc_UintVal(tl);
 			if (status < 100 || status > 999) {
-				VSB_printf(tl->sb,
+				VSB_cat(tl->sb,
 				    "Must specify .expected_response with "
 				    "exactly three digits "
 				    "(100 <= x <= 999)\n");
@@ -219,13 +219,13 @@ vcc_ParseProbeSpec(struct vcc *tl, const struct symbol *sym, char **name)
 
 	if (t_threshold != NULL || t_window != NULL) {
 		if (t_threshold == NULL && t_window != NULL) {
-			VSB_printf(tl->sb,
+			VSB_cat(tl->sb,
 			    "Must specify .threshold with .window\n");
 			vcc_ErrWhere(tl, t_window);
 			return;
 		} else if (t_threshold != NULL && t_window == NULL) {
 			if (threshold > 64) {
-				VSB_printf(tl->sb,
+				VSB_cat(tl->sb,
 				    "Threshold must be 64 or less.\n");
 				vcc_ErrWhere(tl, t_threshold);
 				return;
@@ -233,12 +233,12 @@ vcc_ParseProbeSpec(struct vcc *tl, const struct symbol *sym, char **name)
 			window = threshold + 1;
 		} else if (window > 64) {
 			AN(t_window);
-			VSB_printf(tl->sb, "Window must be 64 or less.\n");
+			VSB_cat(tl->sb, "Window must be 64 or less.\n");
 			vcc_ErrWhere(tl, t_window);
 			return;
 		}
 		if (threshold > window ) {
-			VSB_printf(tl->sb,
+			VSB_cat(tl->sb,
 			    "Threshold can not be greater than window.\n");
 			AN(t_threshold);
 			vcc_ErrWhere(tl, t_threshold);
@@ -344,12 +344,12 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 
 	/* Check for old syntax */
 	if (tl->t->tok == ID && vcc_IdIs(tl->t, "set")) {
-		VSB_printf(tl->sb,
+		VSB_cat(tl->sb,
 		    "NB: Backend Syntax has changed:\n"
 		    "Remove \"set\" and \"backend\" in front"
 		    " of backend fields.\n" );
 		vcc_ErrToken(tl, tl->t);
-		VSB_printf(tl->sb, " at ");
+		VSB_cat(tl->sb, " at ");
 		vcc_ErrWhere(tl, tl->t);
 		return;
 	}
@@ -374,11 +374,11 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 			SkipToken(tl, ';');
 		} else if (vcc_IdIs(t_field, "path")) {
 			if (tl->syntax < VCL_41) {
-				VSB_printf(tl->sb,
+				VSB_cat(tl->sb,
 				    "Unix socket backends only supported"
 				    " in VCL4.1 and higher.\n");
 				vcc_ErrToken(tl, tl->t);
-				VSB_printf(tl->sb, " at ");
+				VSB_cat(tl->sb, " at ");
 				vcc_ErrWhere(tl, tl->t);
 				return;
 			}
@@ -423,7 +423,7 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 			u = vcc_UintVal(tl);
 			ERRCHK(tl);
 			if (u != 1 && u != 2) {
-				VSB_printf(tl->sb,
+				VSB_cat(tl->sb,
 				    ".proxy_header must be 1 or 2\n");
 				vcc_ErrWhere(tl, t_val);
 				return;
@@ -447,10 +447,9 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 			}
 			SkipToken(tl, ';');
 		} else if (vcc_IdIs(t_field, "probe")) {
-			VSB_printf(tl->sb,
-			    "Expected '{' or name of probe, got ");
+			VSB_cat(tl->sb, "Expected '{' or name of probe, got ");
 			vcc_ErrToken(tl, tl->t);
-			VSB_printf(tl->sb, " at\n");
+			VSB_cat(tl->sb, " at\n");
 			vcc_ErrWhere(tl, tl->t);
 			return;
 		} else {
@@ -464,7 +463,7 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 	ERRCHK(tl);
 
 	if (t_host == NULL && t_path == NULL) {
-		VSB_printf(tl->sb, "Expected .host or .path.\n");
+		VSB_cat(tl->sb, "Expected .host or .path.\n");
 		vcc_ErrWhere(tl, t_be);
 		return;
 	}
@@ -537,7 +536,7 @@ vcc_ParseBackend(struct vcc *tl)
 			tl->default_director = NULL;
 		}
 		if (tl->default_director != NULL) {
-			VSB_printf(tl->sb,
+			VSB_cat(tl->sb,
 			    "Only one default director possible.\n");
 			vcc_ErrWhere(tl, t_first);
 			return;
@@ -573,8 +572,8 @@ vcc_Backend_Init(struct vcc *tl)
 
 	Fh(tl, 0, "\nstatic struct vsmw_cluster *vsc_cluster;\n");
 	ifp = New_IniFin(tl);
-	VSB_printf(ifp->ini, "\tvsc_cluster = VRT_VSM_Cluster_New(ctx,\n"
+	VSB_cat(ifp->ini, "\tvsc_cluster = VRT_VSM_Cluster_New(ctx,\n"
 	    "\t    ndirector * VRT_backend_vsm_need(ctx));\n");
-	VSB_printf(ifp->ini, "\tif (vsc_cluster == 0)\n\t\treturn(1);");
-	VSB_printf(ifp->fin, "\t\tVRT_VSM_Cluster_Destroy(ctx, &vsc_cluster);");
+	VSB_cat(ifp->ini, "\tif (vsc_cluster == 0)\n\t\treturn(1);");
+	VSB_cat(ifp->fin, "\t\tVRT_VSM_Cluster_Destroy(ctx, &vsc_cluster);");
 }
diff --git a/lib/libvcc/vcc_backend_util.c b/lib/libvcc/vcc_backend_util.c
index 757026e89..e7c68edfc 100644
--- a/lib/libvcc/vcc_backend_util.c
+++ b/lib/libvcc/vcc_backend_util.c
@@ -104,17 +104,17 @@ vcc_IsField(struct vcc *tl, struct token **t, struct fld_spec *fs)
 			fs->found = t_field;
 			return;
 		}
-		VSB_printf(tl->sb, "Field ");
+		VSB_cat(tl->sb, "Field ");
 		vcc_ErrToken(tl, t_field);
-		VSB_printf(tl->sb, " redefined at:\n");
+		VSB_cat(tl->sb, " redefined at:\n");
 		vcc_ErrWhere(tl, t_field);
-		VSB_printf(tl->sb, "\nFirst defined at:\n");
+		VSB_cat(tl->sb, "\nFirst defined at:\n");
 		vcc_ErrWhere(tl, fs->found);
 		return;
 	}
-	VSB_printf(tl->sb, "Unknown field: ");
+	VSB_cat(tl->sb, "Unknown field: ");
 	vcc_ErrToken(tl, t_field);
-	VSB_printf(tl->sb, " at\n");
+	VSB_cat(tl->sb, " at\n");
 	vcc_ErrWhere(tl, t_field);
 }
 
diff --git a/lib/libvcc/vcc_compile.c b/lib/libvcc/vcc_compile.c
index cfbc62c2e..4d83b490a 100644
--- a/lib/libvcc/vcc_compile.c
+++ b/lib/libvcc/vcc_compile.c
@@ -243,33 +243,33 @@ EmitCoordinates(const struct vcc *tl, struct vsb *vsb)
 	struct source *sp;
 	const char *p;
 
-	VSB_printf(vsb, "/* ---===### Source Code ###===---*/\n");
+	VSB_cat(vsb, "/* ---===### Source Code ###===---*/\n");
 
 	VSB_printf(vsb, "\n#define VGC_NSRCS %u\n", tl->nsources);
 
-	VSB_printf(vsb, "\nstatic const char *srcname[VGC_NSRCS] = {\n");
+	VSB_cat(vsb, "\nstatic const char *srcname[VGC_NSRCS] = {\n");
 	VTAILQ_FOREACH(sp, &tl->sources, list) {
-		VSB_printf(vsb, "\t");
+		VSB_cat(vsb, "\t");
 		VSB_quote(vsb, sp->name, -1, VSB_QUOTE_CSTR);
-		VSB_printf(vsb, ",\n");
+		VSB_cat(vsb, ",\n");
 	}
-	VSB_printf(vsb, "};\n");
+	VSB_cat(vsb, "};\n");
 
 	VSB_printf(vsb, "\nstatic const char *srcbody[%u] = {\n", tl->nsources);
 	VTAILQ_FOREACH(sp, &tl->sources, list) {
-		VSB_printf(vsb, "    /* ");
+		VSB_cat(vsb, "    /* ");
 		VSB_quote(vsb, sp->name, -1, VSB_QUOTE_CSTR);
-		VSB_printf(vsb, " */\n");
+		VSB_cat(vsb, " */\n");
 		VSB_quote_pfx(vsb, "\t", sp->b, sp->e - sp->b, VSB_QUOTE_CSTR);
-		VSB_printf(vsb, ",\n");
+		VSB_cat(vsb, ",\n");
 	}
-	VSB_printf(vsb, "};\n\n");
+	VSB_cat(vsb, "};\n\n");
 
-	VSB_printf(vsb, "/* ---===### Location Counters ###===---*/\n");
+	VSB_cat(vsb, "/* ---===### Location Counters ###===---*/\n");
 
 	VSB_printf(vsb, "\n#define VGC_NREFS %u\n\n", tl->cnt + 1);
 
-	VSB_printf(vsb, "static const struct vpi_ref VGC_ref[VGC_NREFS] = {\n");
+	VSB_cat(vsb, "static const struct vpi_ref VGC_ref[VGC_NREFS] = {\n");
 	lin = 1;
 	pos = 0;
 	sp = 0;
@@ -300,11 +300,11 @@ EmitCoordinates(const struct vcc *tl, struct vsb *vsb)
 		VSB_printf(vsb, "  [%3u] = { %u, %8tu, %4u, %3u, ",
 		    t->cnt, sp->idx, t->b - sp->b, lin, pos + 1);
 		if (t->tok == CSRC)
-			VSB_printf(vsb, " \"C{\"},\n");
+			VSB_cat(vsb, " \"C{\"},\n");
 		else
 			VSB_printf(vsb, " \"%.*s\" },\n", PF(t));
 	}
-	VSB_printf(vsb, "};\n\n");
+	VSB_cat(vsb, "};\n\n");
 }
 
 /*--------------------------------------------------------------------
@@ -551,7 +551,7 @@ vcc_resolve_includes(struct vcc *tl)
 		t1 = VTAILQ_NEXT(t, list);
 		AN(t1);			/* There's always an EOI */
 		if (t1->tok != CSTR) {
-			VSB_printf(tl->sb,
+			VSB_cat(tl->sb,
 			    "include not followed by string constant.\n");
 			vcc_ErrWhere(tl, t1);
 			return;
@@ -560,7 +560,7 @@ vcc_resolve_includes(struct vcc *tl)
 		AN(t2);			/* There's always an EOI */
 
 		if (t2->tok != ';') {
-			VSB_printf(tl->sb,
+			VSB_cat(tl->sb,
 			    "include <string> not followed by semicolon.\n");
 			vcc_ErrWhere(tl, t1);
 			return;
@@ -573,7 +573,7 @@ vcc_resolve_includes(struct vcc *tl)
 			 * the include directive.
 			 */
 			if (t1->src->name[0] != '/') {
-				VSB_printf(tl->sb,
+				VSB_cat(tl->sb,
 				    "include \"./xxxxx\"; needs absolute "
 				    "filename of including file.\n");
 				vcc_ErrWhere(tl, t1);
@@ -671,7 +671,7 @@ vcc_CompileSource(struct vcc *tl, struct source *sp, const char *jfile)
 
 	/* Check if we have any backends at all */
 	if (tl->default_director == NULL) {
-		VSB_printf(tl->sb,
+		VSB_cat(tl->sb,
 		    "No backends or directors found in VCL program, "
 		    "at least one is necessary.\n");
 		tl->err = 1;
@@ -692,13 +692,13 @@ vcc_CompileSource(struct vcc *tl, struct source *sp, const char *jfile)
 
 	/* Tie vcl_init/fini in */
 	ifp = New_IniFin(tl);
-	VSB_printf(ifp->ini, "\tVGC_function_vcl_init(ctx);\n");
+	VSB_cat(ifp->ini, "\tVGC_function_vcl_init(ctx);\n");
 	/*
 	 * Because the failure could be half way into vcl_init{} so vcl_fini{}
 	 * must always be called, also on failure.
 	 */
 	ifp->ignore_errors = 1;
-	VSB_printf(ifp->fin, "\t\tVGC_function_vcl_fini(ctx);");
+	VSB_cat(ifp->fin, "\t\tVGC_function_vcl_fini(ctx);");
 
 	/* Emit method functions */
 	Fh(tl, 1, "\n");
@@ -715,7 +715,7 @@ vcc_CompileSource(struct vcc *tl, struct source *sp, const char *jfile)
 
 	VCC_XrefTable(tl);
 
-	VSB_printf(tl->symtab, "\n]\n");
+	VSB_cat(tl->symtab, "\n]\n");
 	AZ(VSB_finish(tl->symtab));
 	if (TLWriteVSB(tl, jfile, tl->symtab, "Symbol table"))
 		return (NULL);
@@ -815,7 +815,7 @@ VCC_New(void)
 
 	tl->symtab = VSB_new_auto();
 	assert(tl->symtab != NULL);
-	VSB_printf(tl->symtab, "[\n    {\"version\": 0}");
+	VSB_cat(tl->symtab, "[\n    {\"version\": 0}");
 
 	tl->fc = VSB_new_auto();
 	assert(tl->fc != NULL);
diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c
index 29d82a23d..d7c32a8dc 100644
--- a/lib/libvcc/vcc_expr.c
+++ b/lib/libvcc/vcc_expr.c
@@ -290,7 +290,7 @@ vcc_expr_tostring(struct vcc *tl, struct expr **e, vcc_type_t fmt)
 		(*e)->nstr = 1;
 	} else {
 		if ((*e)->fmt == BLOB)
-			VSB_printf(tl->sb,
+			VSB_cat(tl->sb,
 			    "Wrong use of BLOB value.\n"
 			    "BLOBs can only be used as arguments to VMOD"
 			    " functions.\n");
@@ -436,8 +436,8 @@ vcc_do_arg(struct vcc *tl, struct func_arg *fa)
 			if (vcc_IdIs(tl->t, vv->value))
 				break;
 		if (vv == NULL) {
-			VSB_printf(tl->sb, "Wrong enum value.");
-			VSB_printf(tl->sb, "  Expected one of:\n");
+			VSB_cat(tl->sb, "Wrong enum value.");
+			VSB_cat(tl->sb, "  Expected one of:\n");
 			VTAILQ_FOREACH(vv, &fa->enums->children, list)
 				VSB_printf(tl->sb, "\t%s\n", vv->value);
 			vcc_ErrWhere(tl, tl->t);
@@ -626,7 +626,7 @@ vcc_Eval_Func(struct vcc *tl, const struct vjsn_val *spec,
 
 	vcc_func(tl, &e, spec, extra, sym);
 	if (tl->err)
-		VSB_printf(tl->sb, "While compiling function call:\n");
+		VSB_cat(tl->sb, "While compiling function call:\n");
 	ERRCHK(tl);
 	vcc_expr_fmt(tl->fb, tl->indent, e);
 	VSB_cat(tl->fb, ";\n");
@@ -693,7 +693,7 @@ vcc_expr5(struct vcc *tl, struct expr **e, vcc_type_t fmt)
 		ERRCHK(tl);
 		AN(sym);
 		if (sym->kind == SYM_FUNC && sym->type == VOID) {
-			VSB_printf(tl->sb, "Function returns VOID:\n");
+			VSB_cat(tl->sb, "Function returns VOID:\n");
 			vcc_ErrWhere(tl, tl->t);
 			return;
 		}
@@ -702,7 +702,7 @@ vcc_expr5(struct vcc *tl, struct expr **e, vcc_type_t fmt)
 			AZ(*e);
 			sym->eval(tl, e, t, sym, fmt);
 			if (tl->err) {
-				VSB_printf(tl->sb,
+				VSB_cat(tl->sb,
 				    "While compiling function call:\n\n");
 				vcc_ErrWhere2(tl, t, tl->t);
 			}
@@ -719,7 +719,7 @@ vcc_expr5(struct vcc *tl, struct expr **e, vcc_type_t fmt)
 		    PF(t), sym->kind->name);
 		vcc_ErrWhere(tl, t);
 		if (sym->def_b != NULL) {
-			VSB_printf(tl->sb, "That symbol was defined here:\n");
+			VSB_cat(tl->sb, "That symbol was defined here:\n");
 			vcc_ErrWhere(tl, sym->def_b);
 		}
 		return;
@@ -733,8 +733,8 @@ vcc_expr5(struct vcc *tl, struct expr **e, vcc_type_t fmt)
 				 * sockaddr_un if it happens to exist and
 				 * is a socket. So don't let that happen.
 				 */
-				VSB_printf(tl->sb,
-					"Cannot convert to an IP address: ");
+				VSB_cat(tl->sb,
+				    "Cannot convert to an IP address: ");
 				vcc_ErrToken(tl, tl->t);
 				vcc_ErrWhere(tl, tl->t);
 				return;
@@ -794,7 +794,7 @@ vcc_expr5(struct vcc *tl, struct expr **e, vcc_type_t fmt)
 	default:
 		break;
 	}
-	VSB_printf(tl->sb, "Unknown token ");
+	VSB_cat(tl->sb, "Unknown token ");
 	vcc_ErrToken(tl, tl->t);
 	VSB_printf(tl->sb, " when looking for %s\n\n", vcc_utype(fmt));
 	vcc_ErrWhere(tl, tl->t);
@@ -852,7 +852,7 @@ vcc_expr4(struct vcc *tl, struct expr **e, vcc_type_t fmt)
 		}
 
 		if (vm->type_from == NULL) {
-			VSB_printf(tl->sb, "Unknown property ");
+			VSB_cat(tl->sb, "Unknown property ");
 			vcc_ErrToken(tl, tl->t);
 			VSB_printf(tl->sb,
 			 " for type %s\n", (*e)->fmt->name);
@@ -1223,7 +1223,7 @@ vcc_expr_not(struct vcc *tl, struct expr **e, vcc_type_t fmt)
 	vcc_expr_tobool(tl, e);
 	ERRCHK(tl);
 	if ((*e)->fmt != BOOL) {
-		VSB_printf(tl->sb, "'!' must be followed by BOOL, found ");
+		VSB_cat(tl->sb, "'!' must be followed by BOOL, found ");
 		VSB_printf(tl->sb, "%s.\n", vcc_utype((*e)->fmt));
 		vcc_ErrWhere2(tl, tk, tl->t);
 	} else {
@@ -1391,7 +1391,7 @@ vcc_Expr(struct vcc *tl, vcc_type_t fmt)
 	assert(e->fmt == fmt);
 
 	vcc_expr_fmt(tl->fb, tl->indent, e);
-	VSB_printf(tl->fb, "\n");
+	VSB_cat(tl->fb, "\n");
 	vcc_delete_expr(e);
 }
 
diff --git a/lib/libvcc/vcc_parse.c b/lib/libvcc/vcc_parse.c
index bb695ba92..d501ba8e0 100644
--- a/lib/libvcc/vcc_parse.c
+++ b/lib/libvcc/vcc_parse.c
@@ -294,7 +294,7 @@ vcc_ParseVcl(struct vcc *tl)
 
 	Expect(tl, FNUM);
 	if (tl->t->e - tl->t->b != 3 || tl->t->b[1] != '.') {
-		VSB_printf(tl->sb,
+		VSB_cat(tl->sb,
 		    "Don't play silly buggers with VCL version numbers\n");
 		vcc_ErrWhere(tl, tl->t);
 		ERRCHK(tl);
@@ -314,8 +314,7 @@ vcc_ParseVcl(struct vcc *tl)
 		 * in the built-in VCL, and that would give a very
 		 * confusing error message
 		 */
-		VSB_printf(tl->sb,
-		    "Expected 'vcl N.N;' found no semi-colon\n");
+		VSB_cat(tl->sb, "Expected 'vcl N.N;' found no semi-colon\n");
 		vcc_ErrWhere2(tl, tok0, tl->t);
 		ERRCHK(tl);
 	}
@@ -367,7 +366,7 @@ vcc_Parse(struct vcc *tl)
 
 	AZ(tl->indent);
 	if (tl->t->tok != ID || !vcc_IdIs(tl->t, "vcl")) {
-		VSB_printf(tl->sb,
+		VSB_cat(tl->sb,
 		    "VCL version declaration missing\n"
 		    "Update your VCL to Version 4 syntax, and add\n"
 		    "\tvcl 4.1;\n"
@@ -389,8 +388,7 @@ vcc_Parse(struct vcc *tl)
 				    tl->t->b + 2);
 				vcc_NextToken(tl);
 			} else {
-				VSB_printf(tl->sb,
-				    "Inline-C not allowed\n");
+				VSB_cat(tl->sb, "Inline-C not allowed\n");
 				vcc_ErrWhere(tl, tl->t);
 			}
 			break;
@@ -410,17 +408,17 @@ vcc_Parse(struct vcc *tl)
 			/* FALLTHROUGH */
 		default:
 			/* We deliberately do not mention inline-C */
-			VSB_printf(tl->sb, "Expected one of\n\t");
+			VSB_cat(tl->sb, "Expected one of\n\t");
 			for (tp = toplev; tp->name != NULL; tp++) {
 				if (tp[1].name == NULL)
-					VSB_printf(tl->sb, " or ");
+					VSB_cat(tl->sb, " or ");
 				VSB_printf(tl->sb, "'%s'", tp->name);
 				if (tp[1].name != NULL)
-					VSB_printf(tl->sb, ", ");
+					VSB_cat(tl->sb, ", ");
 			}
-			VSB_printf(tl->sb, "\nFound: ");
+			VSB_cat(tl->sb, "\nFound: ");
 			vcc_ErrToken(tl, tl->t);
-			VSB_printf(tl->sb, " at\n");
+			VSB_cat(tl->sb, " at\n");
 			vcc_ErrWhere(tl, tl->t);
 			return;
 		}
diff --git a/lib/libvcc/vcc_symb.c b/lib/libvcc/vcc_symb.c
index 6da0505e8..4d1a4ee72 100644
--- a/lib/libvcc/vcc_symb.c
+++ b/lib/libvcc/vcc_symb.c
@@ -251,8 +251,7 @@ VCC_SymbolGet(struct vcc *tl, vcc_kind_t kind,
 	    (tl->t->b[1] == 'c'|| tl->t->b[1] == 'C') &&
 	    (tl->t->b[2] == 'l'|| tl->t->b[2] == 'L') &&
 	    (tl->t->b[3] == '_')) {
-		VSB_printf(tl->sb,
-		    "Symbols named 'vcl_*' are reserved.\nAt:");
+		VSB_cat(tl->sb, "Symbols named 'vcl_*' are reserved.\nAt:");
 		vcc_ErrWhere(tl, tl->t);
 		return (NULL);
 	}
@@ -293,36 +292,36 @@ VCC_SymbolGet(struct vcc *tl, vcc_kind_t kind,
 		VSB_printf(tl->sb, "%s: '", e->name);
 		for (tn1 = t0; tn1 != tl->t; tn1 = VTAILQ_NEXT(tn1, list))
 			VSB_printf(tl->sb, "%.*s", PF(tn1));
-		VSB_printf(tl->sb, "'");
+		VSB_cat(tl->sb, "'");
 		sym = vcc_sym_in_tab(tl, st, kind, VCL_LOW, VCL_HIGH);
 		if (sym != NULL) {
-			VSB_printf(tl->sb, " (Only available when");
+			VSB_cat(tl->sb, " (Only available when");
 			if (sym->lorev >= VCL_LOW)
 				VSB_printf(tl->sb, " %.1f <=", .1 * sym->lorev);
-			VSB_printf(tl->sb, " VCL syntax");
+			VSB_cat(tl->sb, " VCL syntax");
 			if (sym->hirev <= VCL_HIGH)
 				VSB_printf(tl->sb, " <= %.1f", .1 * sym->hirev);
-			VSB_printf(tl->sb, ")");
+			VSB_cat(tl->sb, ")");
 		}
 		VSB_cat(tl->sb, "\nAt: ");
 		vcc_ErrWhere2(tl, t0, tl->t);
 		return (NULL);
 	}
 	if (kind != SYM_NONE && kind != sym->kind) {
-		VSB_printf(tl->sb, "Symbol '");
+		VSB_cat(tl->sb, "Symbol '");
 		for (tn1 = t0; tn1 != tl->t; tn1 = VTAILQ_NEXT(tn1, list))
 			VSB_printf(tl->sb, "%.*s", PF(tn1));
 		VSB_printf(tl->sb, "' has wrong type (%s): ", sym->kind->name);
 		VSB_cat(tl->sb, "\nAt: ");
 		vcc_ErrWhere2(tl, t0, tl->t);
 		if (sym->def_b != NULL) {
-			VSB_printf(tl->sb, "Symbol was defined here: ");
+			VSB_cat(tl->sb, "Symbol was defined here: ");
 			vcc_ErrWhere(tl, sym->def_b);
 		} else if (sym->ref_b != NULL) {
-			VSB_printf(tl->sb, "Symbol was declared here: ");
+			VSB_cat(tl->sb, "Symbol was declared here: ");
 			vcc_ErrWhere(tl, sym->ref_b);
 		} else {
-			VSB_printf(tl->sb, "Symbol was builtin\n");
+			VSB_cat(tl->sb, "Symbol was builtin\n");
 		}
 		return (NULL);
 	}
@@ -437,14 +436,14 @@ VCC_HandleSymbol(struct vcc *tl, vcc_type_t fmt, const char *pfx)
 		VSB_printf(tl->sb, "%c%s '%.*s' redefined.\n",
 		    toupper(*p), p + 1, PF(t));
 		vcc_ErrWhere(tl, t);
-		VSB_printf(tl->sb, "First definition:\n");
+		VSB_cat(tl->sb, "First definition:\n");
 		AN(sym->def_b);
 		vcc_ErrWhere(tl, sym->def_b);
 		return (sym);
 	} else if (sym != NULL && sym->def_b != NULL) {
 		VSB_printf(tl->sb, "Name '%.*s' already defined.\n", PF(t));
 		vcc_ErrWhere(tl, t);
-		VSB_printf(tl->sb, "First definition:\n");
+		VSB_cat(tl->sb, "First definition:\n");
 		AN(sym->def_b);
 		vcc_ErrWhere(tl, sym->def_b);
 		return (sym);
diff --git a/lib/libvcc/vcc_token.c b/lib/libvcc/vcc_token.c
index 090c84439..04e10f96b 100644
--- a/lib/libvcc/vcc_token.c
+++ b/lib/libvcc/vcc_token.c
@@ -43,9 +43,9 @@ vcc_ErrToken(const struct vcc *tl, const struct token *t)
 {
 
 	if (t->tok == EOI)
-		VSB_printf(tl->sb, "end of input");
+		VSB_cat(tl->sb, "end of input");
 	else if (t->tok == CSRC)
-		VSB_printf(tl->sb, "C{ ... }C");
+		VSB_cat(tl->sb, "C{ ... }C");
 	else
 		VSB_printf(tl->sb, "'%.*s'", PF(t));
 }
@@ -104,7 +104,7 @@ vcc_icoord(struct vsb *vsb, const struct token *t, int tail)
 		} else
 			pos++;
 	}
-	VSB_printf(vsb, "(");
+	VSB_cat(vsb, "(");
 	if (tail < 2)
 		VSB_printf(vsb, "'%s' Line %u ", t->src->name, lin);
 	VSB_printf(vsb, "Pos %u)", pos + 1);
@@ -255,7 +255,7 @@ vcc_NextToken(struct vcc *tl)
 
 	tl->t = VTAILQ_NEXT(tl->t, list);
 	if (tl->t == NULL) {
-		VSB_printf(tl->sb,
+		VSB_cat(tl->sb,
 		    "Ran out of input, something is missing or"
 		    " maybe unbalanced (...) or {...}\n");
 		tl->err = 1;
@@ -398,7 +398,7 @@ vcc_Lexer(struct vcc *tl, struct source *sp)
 		if (*p == '/' && p[1] == '*') {
 			for (q = p + 2; q < sp->e; q++) {
 				if (*q == '/' && q[1] == '*') {
-					VSB_printf(tl->sb,
+					VSB_cat(tl->sb,
 					    "/* ... */ comment contains /*\n");
 					vcc_AddToken(tl, EOI, p, p + 2);
 					vcc_ErrWhere(tl, tl->t);
@@ -414,7 +414,7 @@ vcc_Lexer(struct vcc *tl, struct source *sp)
 			if (q < sp->e)
 				continue;
 			vcc_AddToken(tl, EOI, p, p + 2);
-			VSB_printf(tl->sb,
+			VSB_cat(tl->sb,
 			    "Unterminated /* ... */ comment, starting at\n");
 			vcc_ErrWhere(tl, tl->t);
 			return;
@@ -440,7 +440,7 @@ vcc_Lexer(struct vcc *tl, struct source *sp)
 				continue;
 			}
 			vcc_AddToken(tl, EOI, p, p + 2);
-			VSB_printf(tl->sb,
+			VSB_cat(tl->sb,
 			    "Unterminated inline C source, starting at\n");
 			vcc_ErrWhere(tl, tl->t);
 			return;
@@ -465,7 +465,7 @@ vcc_Lexer(struct vcc *tl, struct source *sp)
 				continue;
 			}
 			vcc_AddToken(tl, EOI, p, p + 2);
-			VSB_printf(tl->sb,
+			VSB_cat(tl->sb,
 			    "Unterminated long-string, starting at\n");
 			vcc_ErrWhere(tl, tl->t);
 			return;
@@ -488,7 +488,7 @@ vcc_Lexer(struct vcc *tl, struct source *sp)
 				}
 				if (*q == '\r' || *q == '\n') {
 					vcc_AddToken(tl, EOI, p, q);
-					VSB_printf(tl->sb,
+					VSB_cat(tl->sb,
 					    "Unterminated string at\n");
 					vcc_ErrWhere(tl, tl->t);
 					return;
@@ -529,7 +529,7 @@ vcc_Lexer(struct vcc *tl, struct source *sp)
 			continue;
 		}
 		vcc_AddToken(tl, EOI, p, p + 1);
-		VSB_printf(tl->sb, "Syntax error at\n");
+		VSB_cat(tl->sb, "Syntax error at\n");
 		vcc_ErrWhere(tl, tl->t);
 		return;
 	}
diff --git a/lib/libvcc/vcc_utils.c b/lib/libvcc/vcc_utils.c
index 8a08e9af9..c0e598532 100644
--- a/lib/libvcc/vcc_utils.c
+++ b/lib/libvcc/vcc_utils.c
@@ -75,7 +75,7 @@ vcc_regexp(struct vcc *tl, struct vsb *vgc_name)
 	ifp = New_IniFin(tl);
 	VSB_printf(ifp->ini, "\tVRT_re_init(&%s, ",buf);
 	EncToken(ifp->ini, tl->t);
-	VSB_printf(ifp->ini, ");");
+	VSB_cat(ifp->ini, ");");
 	VSB_printf(ifp->fin, "\t\tVRT_re_fini(%s);", buf);
 	vcc_NextToken(tl);
 }
@@ -282,7 +282,7 @@ Emit_UDS_Path(struct vcc *tl, const struct token *t_path, const char *errid)
 			   strerror(errno));
 		vcc_ErrWhere(tl, t_path);
 		if (err == ENOENT || err == EACCES) {
-			VSB_printf(tl->sb, "(That was just a warning)\n");
+			VSB_cat(tl->sb, "(That was just a warning)\n");
 			tl->err = 0;
 		} else
 			return;
@@ -311,7 +311,7 @@ vcc_DurationUnit(struct vcc *tl)
 		vcc_NextToken(tl);
 		return (sc);
 	}
-	VSB_printf(tl->sb, "Unknown duration unit ");
+	VSB_cat(tl->sb, "Unknown duration unit ");
 	vcc_ErrToken(tl, tl->t);
 	VSB_printf(tl->sb, "\n%s\n", VNUM_LEGAL_DURATION);
 	vcc_ErrWhere(tl, tl->t);
@@ -382,15 +382,15 @@ vcc_ByteVal(struct vcc *tl, double *d)
 	v = vcc_DoubleVal(tl);
 	ERRCHK(tl);
 	if (tl->t->tok != ID) {
-		VSB_printf(tl->sb, "Expected BYTES unit (B, KB, MB...) got ");
+		VSB_cat(tl->sb, "Expected BYTES unit (B, KB, MB...) got ");
 		vcc_ErrToken(tl, tl->t);
-		VSB_printf(tl->sb, "\n");
+		VSB_cat(tl->sb, "\n");
 		vcc_ErrWhere(tl, tl->t);
 		return;
 	}
 	sc = VNUM_bytes_unit(1.0, tl->t->b, tl->t->e, 0);
 	if (isnan(sc)) {
-		VSB_printf(tl->sb, "Unknown BYTES unit ");
+		VSB_cat(tl->sb, "Unknown BYTES unit ");
 		vcc_ErrToken(tl, tl->t);
 		VSB_printf(tl->sb, "\n%s\n", VNUM_LEGAL_BYTES);
 		vcc_ErrWhere(tl, tl->t);
diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c
index 1a5c13453..846f6d502 100644
--- a/lib/libvcc/vcc_vmod.c
+++ b/lib/libvcc/vcc_vmod.c
@@ -169,7 +169,7 @@ vcc_VmodSanity(struct vcc *tl, void *hdl, struct token *mod, char *fnp)
 	if (vmd == NULL) {
 		VSB_printf(tl->sb, "Malformed VMOD %.*s\n", PF(mod));
 		VSB_printf(tl->sb, "\tFile name: %s\n", fnp);
-		VSB_printf(tl->sb, "\t(no Vmod_Data symbol)\n");
+		VSB_cat(tl->sb, "\t(no Vmod_Data symbol)\n");
 		vcc_ErrWhere(tl, mod);
 		return (NULL);
 	}
@@ -200,7 +200,7 @@ vcc_VmodSanity(struct vcc *tl, void *hdl, struct token *mod, char *fnp)
 	    vmd->abi == NULL) {
 		VSB_printf(tl->sb, "Mangled VMOD %.*s\n", PF(mod));
 		VSB_printf(tl->sb, "\tFile name: %s\n", fnp);
-		VSB_printf(tl->sb, "\tInconsistent metadata\n");
+		VSB_cat(tl->sb, "\tInconsistent metadata\n");
 		vcc_ErrWhere(tl, mod);
 		return (NULL);
 	}
@@ -246,13 +246,13 @@ vcc_ParseImport(struct vcc *tl)
 
 	if (tl->t->tok == ID) {
 		if (!vcc_IdIs(tl->t, "from")) {
-			VSB_printf(tl->sb, "Expected 'from path ...'\n");
+			VSB_cat(tl->sb, "Expected 'from path ...'\n");
 			vcc_ErrWhere(tl, tl->t);
 			return;
 		}
 		vcc_NextToken(tl);
 		if (!tl->unsafe_path && strchr(tl->t->dec, '/')) {
-			VSB_printf(tl->sb,
+			VSB_cat(tl->sb,
 			    "'import ... from path ...' is unsafe.\nAt:");
 			vcc_ErrToken(tl, tl->t);
 			vcc_ErrWhere(tl, tl->t);
@@ -328,32 +328,32 @@ vcc_ParseImport(struct vcc *tl)
 
 	ifp = New_IniFin(tl);
 
-	VSB_printf(ifp->ini, "\tif (VPI_Vmod_Init(ctx,\n");
+	VSB_cat(ifp->ini, "\tif (VPI_Vmod_Init(ctx,\n");
 	VSB_printf(ifp->ini, "\t    &VGC_vmod_%.*s,\n", PF(mod));
 	VSB_printf(ifp->ini, "\t    %u,\n", tl->vmod_count++);
 	VSB_printf(ifp->ini, "\t    &%s,\n", vmd->func_name);
 	VSB_printf(ifp->ini, "\t    sizeof(%s),\n", vmd->func_name);
 	VSB_printf(ifp->ini, "\t    \"%.*s\",\n", PF(mod));
-	VSB_printf(ifp->ini, "\t    ");
+	VSB_cat(ifp->ini, "\t    ");
 	VSB_quote(ifp->ini, fnpx, -1, VSB_QUOTE_CSTR);
-	VSB_printf(ifp->ini, ",\n");
+	VSB_cat(ifp->ini, ",\n");
 	AN(vmd);
 	AN(vmd->file_id);
 	VSB_printf(ifp->ini, "\t    \"%s\",\n", vmd->file_id);
 	VSB_printf(ifp->ini, "\t    \"./vmod_cache/_vmod_%.*s.%s\"\n",
 	    PF(mod), vmd->file_id);
-	VSB_printf(ifp->ini, "\t    ))\n");
-	VSB_printf(ifp->ini, "\t\treturn(1);");
+	VSB_cat(ifp->ini, "\t    ))\n");
+	VSB_cat(ifp->ini, "\t\treturn(1);");
 
-	VSB_printf(tl->symtab, ",\n    {\n");
-	VSB_printf(tl->symtab, "\t\"dir\": \"import\",\n");
-	VSB_printf(tl->symtab, "\t\"type\": \"$VMOD\",\n");
+	VSB_cat(tl->symtab, ",\n    {\n");
+	VSB_cat(tl->symtab, "\t\"dir\": \"import\",\n");
+	VSB_cat(tl->symtab, "\t\"type\": \"$VMOD\",\n");
 	VSB_printf(tl->symtab, "\t\"name\": \"%.*s\",\n", PF(mod));
 	VSB_printf(tl->symtab, "\t\"file\": \"%s\",\n", fnpx);
 	VSB_printf(tl->symtab,
 	    "\t\"dst\": \"./vmod_cache/_vmod_%.*s.%s\"\n",
 	    PF(mod), vmd->file_id);
-	VSB_printf(tl->symtab, "    }");
+	VSB_cat(tl->symtab, "    }");
 
 	/* XXX: zero the function pointer structure ?*/
 	VSB_printf(ifp->fin, "\t\tVRT_priv_fini(&vmod_priv_%.*s);", PF(mod));
diff --git a/lib/libvcc/vcc_xref.c b/lib/libvcc/vcc_xref.c
index dd0893c23..e5f5f2858 100644
--- a/lib/libvcc/vcc_xref.c
+++ b/lib/libvcc/vcc_xref.c
@@ -84,7 +84,7 @@ vcc_checkref(struct vcc *tl, const struct symbol *sym)
 		    sym->kind->name, PF(sym->def_b));
 		vcc_ErrWhere(tl, sym->def_b);
 		if (!tl->err_unref) {
-			VSB_printf(tl->sb, "(That was just a warning)\n");
+			VSB_cat(tl->sb, "(That was just a warning)\n");
 			tl->err = 0;
 		}
 	}
@@ -155,7 +155,7 @@ vcc_CheckActionRecurse(struct vcc *tl, struct proc *p, unsigned bitmap)
 
 	AN(p);
 	if (p->active) {
-		VSB_printf(tl->sb, "Function recurses on\n");
+		VSB_cat(tl->sb, "Function recurses on\n");
 		vcc_ErrWhere(tl, p->name);
 		return (1);
 	}
@@ -209,13 +209,13 @@ vcc_checkaction1(struct vcc *tl, const struct symbol *sym)
 	if (vcc_CheckActionRecurse(tl, p, p->method->ret_bitmap)) {
 		VSB_printf(tl->sb,
 		    "\n...which is the \"%s\" method\n", p->method->name);
-		VSB_printf(tl->sb, "Legal returns are:");
+		VSB_cat(tl->sb, "Legal returns are:");
 #define VCL_RET_MAC(l, U, B)						\
 		if (p->method->ret_bitmap & ((1 << VCL_RET_##U)))	\
 			VSB_printf(tl->sb, " \"%s\"", #l);
 
 #include "tbl/vcl_returns.h"
-		VSB_printf(tl->sb, "\n");
+		VSB_cat(tl->sb, "\n");
 		tl->err = 1;
 	}
 
@@ -231,10 +231,10 @@ vcc_checkaction2(struct vcc *tl, const struct symbol *sym)
 
 	if (p->called)
 		return;
-	VSB_printf(tl->sb, "Function unused\n");
+	VSB_cat(tl->sb, "Function unused\n");
 	vcc_ErrWhere(tl, p->name);
 	if (!tl->err_unref) {
-		VSB_printf(tl->sb, "(That was just a warning)\n");
+		VSB_cat(tl->sb, "(That was just a warning)\n");
 		tl->err = 0;
 	}
 }
diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c
index d6171d677..fd408f767 100644
--- a/lib/libvmod_debug/vmod_debug.c
+++ b/lib/libvmod_debug/vmod_debug.c
@@ -357,7 +357,7 @@ event_load(VRT_CTX, struct vmod_priv *priv)
 	loads++;
 
 	if (cache_param->nuke_limit == 42) {
-		VSB_printf(ctx->msg, "nuke_limit is not the answer.");
+		VSB_cat(ctx->msg, "nuke_limit is not the answer.");
 		return (-1);
 	}
 
@@ -414,7 +414,7 @@ event_warm(VRT_CTX, const struct vmod_priv *priv)
 
 	AN(ctx->msg);
 	if (cache_param->max_esi_depth == 42) {
-		VSB_printf(ctx->msg, "max_esi_depth is not the answer.");
+		VSB_cat(ctx->msg, "max_esi_depth is not the answer.");
 		return (-1);
 	}
 
diff --git a/tools/coccinelle/printf_nofmt.cocci b/tools/coccinelle/printf_nofmt.cocci
new file mode 100644
index 000000000..a4c6d559b
--- /dev/null
+++ b/tools/coccinelle/printf_nofmt.cocci
@@ -0,0 +1,10 @@
+/*
+ * This patch removes useless calls to printf-type functions.
+ */
+
+@@
+expression vsb, fmt;
+@@
+
+- VSB_printf(vsb, fmt);
++ VSB_cat(vsb, fmt);

From dridi.boukelmoune at gmail.com  Fri Nov  8 18:36:07 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Fri,  8 Nov 2019 18:36:07 +0000 (UTC)
Subject: [master] 46a6a3af6 Apply bandaid to missing VMOD symbol errors
Message-ID: <20191108183607.8C39964D2A@lists.varnish-cache.org>


commit 46a6a3af681644f43e2e813ee68f8bd53b4cd1bb
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Nov 8 19:10:48 2019 +0100

    Apply bandaid to missing VMOD symbol errors
    
    The VCL compilation error message when a VMOD symbol does not exist
    simply disappeared. When I figured how to bring it back I was lucky
    my test case exhibited another quirk for the following case:
    
        new foo = bar.foo()
    
    Where bar is a successfully imported VMOD, and foo is the missing
    constructor. For some reason instance symbols are created with VCL
    low and high values, so the missing foo constructor ended up being
    confused with the existing foo instance.
    
    The regression I initially hunted down (the lack of error message
    in the first place) was introduced by 340abd0456e4. I suggest we
    don't give VMOD-induced symbols a VCL low/high since by definition
    they are not tied to a VCL version (even though they may do so at
    run time) and use that criteria to filter out the spurious error
    message:
    
    > Symbol not found: 'directors.foo' (Only available when 4.0 <= VCL
    > syntax <= 4.0)
    
    If there is no proper low or high (and 4.0 is legit) then we don't
    print the "Only available when" part? For now I needed the bandaid.
    
    Refs 340abd0456e4d018fd1589f406287057ab6740ff

diff --git a/bin/varnishtest/tests/v00016.vtc b/bin/varnishtest/tests/v00016.vtc
index 56ba92c4f..f1cab3db6 100644
--- a/bin/varnishtest/tests/v00016.vtc
+++ b/bin/varnishtest/tests/v00016.vtc
@@ -117,6 +117,19 @@ varnish v1 -syntax 4.0 -errvcl {Undefined acl foo} {
 	}
 }
 
+# NB: The line break in -errvcl is here on purpose, it prevents
+# a spurious "Only available when" addition to be missed when the
+# foo constructor could be confused with the foo instance name.
+varnish v1 -syntax 4.0 -errvcl {Symbol not found: 'directors.foo'
+At:} {
+	import directors;
+	backend b { .host = "127.0.0.1"; }
+
+	sub vcl_init {
+		new foo = directors.foo();
+	}
+}
+
 # 'foo' overloaded
 varnish v1 -syntax 4.0 -errvcl {Symbol not found: 'foo'} {
 	backend b { .host = "127.0.0.1"; }
diff --git a/lib/libvcc/vcc_symb.c b/lib/libvcc/vcc_symb.c
index 4d1a4ee72..3ef753220 100644
--- a/lib/libvcc/vcc_symb.c
+++ b/lib/libvcc/vcc_symb.c
@@ -275,6 +275,8 @@ VCC_SymbolGet(struct vcc *tl, vcc_kind_t kind,
 			break;
 		tn = tn1;
 	}
+	if (sym != NULL && sym->kind == SYM_VMOD && e == SYMTAB_PARTIAL)
+		e = SYMTAB_EXISTING;
 	if (sym != NULL && e == SYMTAB_PARTIAL) {
 		st = st2;
 		tn = tn2;
@@ -294,7 +296,8 @@ VCC_SymbolGet(struct vcc *tl, vcc_kind_t kind,
 			VSB_printf(tl->sb, "%.*s", PF(tn1));
 		VSB_cat(tl->sb, "'");
 		sym = vcc_sym_in_tab(tl, st, kind, VCL_LOW, VCL_HIGH);
-		if (sym != NULL) {
+		if (sym != NULL && sym->kind != SYM_OBJECT &&
+		    sym->kind != SYM_INSTANCE) { /* XXX: too specific */
 			VSB_cat(tl->sb, " (Only available when");
 			if (sym->lorev >= VCL_LOW)
 				VSB_printf(tl->sb, " %.1f <=", .1 * sym->lorev);

From nils.goroll at uplex.de  Sat Nov  9 14:48:06 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Sat,  9 Nov 2019 14:48:06 +0000 (UTC)
Subject: [master] dfc71c665 some more rfc references
Message-ID: <20191109144806.1F311117FFE@lists.varnish-cache.org>


commit dfc71c66582852f2b53c58da55d935c0abb07f4e
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Sat Nov 9 15:27:40 2019 +0100

    some more rfc references

diff --git a/bin/varnishd/cache/cache_rfc2616.c b/bin/varnishd/cache/cache_rfc2616.c
index e411f6aaf..a870ba05c 100644
--- a/bin/varnishd/cache/cache_rfc2616.c
+++ b/bin/varnishd/cache/cache_rfc2616.c
@@ -242,6 +242,7 @@ RFC2616_Req_Gzip(const struct http *hp)
 
 /*--------------------------------------------------------------------*/
 
+// rfc7232,l,547,548
 static inline int
 rfc2616_strong_compare(const char *p, const char *e)
 {
@@ -251,6 +252,7 @@ rfc2616_strong_compare(const char *p, const char *e)
 	return (strcmp(p, e) == 0);
 }
 
+// rfc7232,l,550,552
 static inline int
 rfc2616_weak_compare(const char *p, const char *e)
 {
@@ -271,8 +273,8 @@ RFC2616_Do_Cond(const struct req *req)
 		return (0);
 
 	/*
-	 * We MUST ignore If-Modified-Since if we have an If-None-Match
-	 * header [RFC7232 3.3 p16].
+	 * rfc7232,l,861,866
+	 * We MUST ignore If-Modified-Since if we have an If-None-Match header
 	 */
 	if (http_GetHdr(req->http, H_If_None_Match, &p)) {
 		if (!http_GetHdr(req->resp, H_ETag, &e))
@@ -285,7 +287,7 @@ RFC2616_Do_Cond(const struct req *req)
 
 	if (http_GetHdr(req->http, H_If_Modified_Since, &p)) {
 		ims = VTIM_parse(p);
-		if (!ims || ims > req->t_req)	/* [RFC7232 3.3 p16] */
+		if (!ims || ims > req->t_req)	// rfc7232,l,868,869
 			return (0);
 		if (http_GetHdr(req->resp, H_Last_Modified, &p)) {
 			lm = VTIM_parse(p);

From nils.goroll at uplex.de  Sun Nov 10 16:08:05 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Sun, 10 Nov 2019 16:08:05 +0000 (UTC)
Subject: [master] 0496921fe document that we also consider an internal
 timestamp for 304s
Message-ID: <20191110160805.D822C114B8D@lists.varnish-cache.org>


commit 0496921fea70bb6470c52082efb520cf9a2cf3d1
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Sun Nov 10 17:05:59 2019 +0100

    document that we also consider an internal timestamp for 304s
    
    closes #3118

diff --git a/doc/sphinx/reference/vcl_var.rst b/doc/sphinx/reference/vcl_var.rst
index 7474e02fc..4736826bd 100644
--- a/doc/sphinx/reference/vcl_var.rst
+++ b/doc/sphinx/reference/vcl_var.rst
@@ -1159,6 +1159,17 @@ resp.status
 	a return(deliver) from vcl_deliver for conditional requests
 	to cached content if validation succeeds.
 
+	For the validation, first ``req.http.If-None-Match`` is
+	compared against ``resp.http.Etag``. If they compare equal
+	according to the rules for weak validation (see RFC7232), a
+	304 is sent.
+
+	Secondly, ``req.http.If-Modified-Since`` is compared against
+	``resp.http.Last-Modified`` or, if it is unset, against the
+	point in time when the object was last modified based on the
+	``Date`` and ``Age`` headers received with the backend
+	response which created the object. If the object has not been
+	modified based on that comparison, a 304 is sent.
 
 resp.reason
 

From phk at FreeBSD.org  Mon Nov 11 08:49:06 2019
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 11 Nov 2019 08:49:06 +0000 (UTC)
Subject: [master] 23de6b115 Monday morning flexeLinting
Message-ID: <20191111084906.BA959104C23@lists.varnish-cache.org>


commit 23de6b115533d205ccb53bb08a7d24e6b044cb77
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Nov 11 08:48:03 2019 +0000

    Monday morning flexeLinting

diff --git a/bin/varnishd/cache/cache_acceptor.c b/bin/varnishd/cache/cache_acceptor.c
index 9e86c9b62..73430d76e 100644
--- a/bin/varnishd/cache/cache_acceptor.c
+++ b/bin/varnishd/cache/cache_acceptor.c
@@ -384,7 +384,7 @@ vca_make_session(struct worker *wrk, void *arg)
 	wa->acceptsock = -1;
 	sp->listen_sock = wa->acceptlsock;
 
-	assert(wa->acceptaddrlen <= vsa_suckaddr_len);
+	assert((size_t)wa->acceptaddrlen <= vsa_suckaddr_len);
 
 	if (wa->acceptlsock->uds)
 		vca_mk_uds(wa, sp, laddr, lport, raddr, rport);
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 8796e2adf..9bffb8855 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -578,7 +578,7 @@ HSH_Lookup(struct req *req, struct objcore **ocp, struct objcore **bocp)
 static void
 hsh_rush1(const struct worker *wrk, struct objhead *oh, struct rush *r, int max)
 {
-	unsigned u;
+	int i;
 	struct req *req;
 
 	if (max == 0)
@@ -592,7 +592,7 @@ hsh_rush1(const struct worker *wrk, struct objhead *oh, struct rush *r, int max)
 	CHECK_OBJ_NOTNULL(r, RUSH_MAGIC);
 	VTAILQ_INIT(&r->reqs);
 	Lck_AssertHeld(&oh->mtx);
-	for (u = 0; u < max; u++) {
+	for (i = 0; i < max; i++) {
 		req = VTAILQ_FIRST(&oh->waitinglist);
 		if (req == NULL)
 			break;
diff --git a/bin/varnishd/cache/cache_session.c b/bin/varnishd/cache/cache_session.c
index 647b43665..c94ad1f11 100644
--- a/bin/varnishd/cache/cache_session.c
+++ b/bin/varnishd/cache/cache_session.c
@@ -151,12 +151,16 @@ SES_Set_String_Attr(struct sess *sp, enum sess_attr a, const char *src)
 	CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
 	AN(src);
 
+	/*lint -save -e568 -e685 */
+
 	switch (a) {
 #define SESS_ATTR(UP, low, typ, len)	case SA_##UP: assert(len < 0); break;
 #include "tbl/sess_attr.h"
 	default:  WRONG("wrong sess_attr");
 	}
 
+	/*lint -restore */
+
 	ses_reserve_attr(sp, a, &q, strlen(src) + 1);
 	strcpy(q, src);
 }
@@ -168,12 +172,16 @@ SES_Get_String_Attr(const struct sess *sp, enum sess_attr a)
 
 	CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
 
+	/*lint -save -e568 -e685 */
+
 	switch (a) {
 #define SESS_ATTR(UP, low, typ, len)	case SA_##UP: assert(len < 0); break;
 #include "tbl/sess_attr.h"
 	default:  WRONG("wrong sess_attr");
 	}
 
+	/*lint -restore */
+
 	if (ses_get_attr(sp, a, &q) < 0)
 		return (NULL);
 	return (q);
diff --git a/bin/varnishd/http1/cache_http1_fsm.c b/bin/varnishd/http1/cache_http1_fsm.c
index fcd914ba4..9f1796092 100644
--- a/bin/varnishd/http1/cache_http1_fsm.c
+++ b/bin/varnishd/http1/cache_http1_fsm.c
@@ -188,9 +188,8 @@ http1_minimal_response(struct req *req, uint16_t status)
 
 	reason = http_Status2Reason(status, NULL);
 
-	l = snprintf(buf, sizeof(buf),
-	    "HTTP/1.1 %03d %s\r\n\r\n", status, reason);
-	assert (l < sizeof(buf));
+	bprintf(buf, "HTTP/1.1 %03d %s\r\n\r\n", status, reason);
+	l = strlen(buf);
 
 	VSLb(req->vsl, SLT_RespProtocol, "HTTP/1.1");
 	VSLb(req->vsl, SLT_RespStatus, "%03d", status);
diff --git a/bin/varnishd/http2/cache_http2_session.c b/bin/varnishd/http2/cache_http2_session.c
index 8eb5c0506..aa91bc15d 100644
--- a/bin/varnishd/http2/cache_http2_session.c
+++ b/bin/varnishd/http2/cache_http2_session.c
@@ -163,7 +163,7 @@ h2_del_sess(struct worker *wrk, struct h2_sess *h2, enum sess_close reason)
 enum htc_status_e v_matchproto_(htc_complete_f)
 H2_prism_complete(struct http_conn *htc)
 {
-	int l;
+	ptrdiff_t l;
 
 	CHECK_OBJ_NOTNULL(htc, HTTP_CONN_MAGIC);
 	l = htc->rxbuf_e - htc->rxbuf_b;
diff --git a/bin/varnishd/mgt/mgt_cli.c b/bin/varnishd/mgt/mgt_cli.c
index 5ab6db68c..4abcfe8a6 100644
--- a/bin/varnishd/mgt/mgt_cli.c
+++ b/bin/varnishd/mgt/mgt_cli.c
@@ -245,16 +245,16 @@ mgt_cli_stop_child(void)
 static void
 mgt_cli_challenge(struct cli *cli)
 {
-	int i;
+	size_t z;
 	uint8_t u;
 
 	AZ(VRND_RandomCrypto(cli->challenge, sizeof cli->challenge - 2));
-	for (i = 0; i < (sizeof cli->challenge) - 2; i++) {
+	for (z = 0; z < (sizeof cli->challenge) - 2; z++) {
 		AZ(VRND_RandomCrypto(&u, sizeof u));
-		cli->challenge[i] = (u % 26) + 'a';
+		cli->challenge[z] = (u % 26) + 'a';
 	}
-	cli->challenge[i++] = '\n';
-	cli->challenge[i] = '\0';
+	cli->challenge[z++] = '\n';
+	cli->challenge[z] = '\0';
 	VCLI_Out(cli, "%s", cli->challenge);
 	VCLI_Out(cli, "\nAuthentication required.\n");
 	VCLI_SetResult(cli, CLIS_AUTH);
@@ -691,11 +691,12 @@ mgt_DumpRstCli(void)
 {
 	const struct cli_cmd_desc *cp;
 	const char *p;
-	int i, j;
+	int z;
+	size_t j;
 
 	qsort(cmds, ncmds, sizeof cmds[0], cli_cmp);
-	for (i = 0; i < ncmds; i++, cp++) {
-		cp = cmds[i];
+	for (z = 0; z < ncmds; z++, cp++) {
+		cp = cmds[z];
 		if (!strncmp(cp->request, "debug.", 6))
 			continue;
 		printf(".. _ref_cli_");
diff --git a/bin/varnishd/mgt/mgt_param.c b/bin/varnishd/mgt/mgt_param.c
index 67bacd946..93afb9d35 100644
--- a/bin/varnishd/mgt/mgt_param.c
+++ b/bin/varnishd/mgt/mgt_param.c
@@ -565,7 +565,7 @@ MCF_AddParams(struct parspec *ps)
 			exit(4);
 		}
 		mcf_addpar(pp);
-		if (strlen(pp->name) + 1 > margin2)
+		if (strlen(pp->name) + 1L > margin2)
 			margin2 = strlen(pp->name) + 1;
 	}
 }
@@ -733,7 +733,7 @@ MCF_DumpRstParam(void)
 	struct plist *pl;
 	const struct parspec *pp;
 	const char *p, *q, *t1, *t2;
-	int j;
+	size_t z;
 
 	printf("\n.. The following is the autogenerated "
 	    "output from varnishd -x parameter\n\n");
@@ -741,7 +741,7 @@ MCF_DumpRstParam(void)
 		pp = pl->spec;
 		printf(".. _ref_param_%s:\n\n", pp->name);
 		printf("%s\n", pp->name);
-		for (j = 0; j < strlen(pp->name); j++)
+		for (z = 0; z < strlen(pp->name); z++)
 			printf("~");
 		printf("\n");
 
diff --git a/bin/varnishd/proxy/cache_proxy_proto.c b/bin/varnishd/proxy/cache_proxy_proto.c
index bb142f568..20a626d40 100644
--- a/bin/varnishd/proxy/cache_proxy_proto.c
+++ b/bin/varnishd/proxy/cache_proxy_proto.c
@@ -472,7 +472,8 @@ vpx_proto2(const struct worker *wrk, struct req *req)
 static enum htc_status_e v_matchproto_(htc_complete_f)
 vpx_complete(struct http_conn *htc)
 {
-	int i, l, j;
+	size_t z, l;
+	unsigned j;
 	char *p, *q;
 
 	CHECK_OBJ_NOTNULL(htc, HTTP_CONN_MAGIC);
@@ -483,26 +484,26 @@ vpx_complete(struct http_conn *htc)
 	l = htc->rxbuf_e - htc->rxbuf_b;
 	p = htc->rxbuf_b;
 	j = 0x3;
-	for (i = 0; i < l; i++) {
-		if (i < sizeof vpx1_sig && p[i] != vpx1_sig[i])
+	for (z = 0; z < l; z++) {
+		if (z < sizeof vpx1_sig && p[z] != vpx1_sig[z])
 			j &= ~1;
-		if (i < sizeof vpx2_sig && p[i] != vpx2_sig[i])
+		if (z < sizeof vpx2_sig && p[z] != vpx2_sig[z])
 			j &= ~2;
 		if (j == 0)
 			return (HTC_S_JUNK);
-		if (j == 1 && i == sizeof vpx1_sig) {
-			q = memchr(p + i, '\n', htc->rxbuf_e - (p + i));
+		if (j == 1 && z == sizeof vpx1_sig) {
+			q = memchr(p + z, '\n', htc->rxbuf_e - (p + z));
 			if (q != NULL && (q - htc->rxbuf_b) > 107)
 				return (HTC_S_OVERFLOW);
 			if (q == NULL)
 				return (HTC_S_MORE);
 			return (HTC_S_COMPLETE);
 		}
-		if (j == 2 && i == sizeof vpx2_sig) {
+		if (j == 2 && z == sizeof vpx2_sig) {
 			if (l < 16)
 				return (HTC_S_MORE);
 			j = vbe16dec(p + 14);
-			if (l < 16 + j)
+			if (l < 16L + j)
 				return (HTC_S_MORE);
 			return (HTC_S_COMPLETE);
 		}
diff --git a/bin/varnishd/waiter/cache_waiter_poll.c b/bin/varnishd/waiter/cache_waiter_poll.c
index 1736ba300..57f7bbcd2 100644
--- a/bin/varnishd/waiter/cache_waiter_poll.c
+++ b/bin/varnishd/waiter/cache_waiter_poll.c
@@ -118,7 +118,6 @@ vwp_del(struct vwp *vwp, int n)
 		vwp->pollfd[n] = vwp->pollfd[vwp->hpoll];
 		vwp->idx[n] = vwp->idx[vwp->hpoll];
 	}
-VSL(SLT_Debug, vwp->pollfd[vwp->hpoll].fd, "DEL");
 	memset(&vwp->pollfd[vwp->hpoll], 0, sizeof(*vwp->pollfd));
 	vwp->pollfd[vwp->hpoll].fd = -1;
 	vwp->idx[vwp->hpoll] = NULL;
@@ -154,12 +153,12 @@ vwp_dopipe(struct vwp *vwp)
 static void *
 vwp_main(void *priv)
 {
-	int v;
+	int t, v;
 	struct vwp *vwp;
 	struct waiter *w;
 	struct waited *wp;
 	double now, then;
-	int i;
+	size_t z;
 
 	THR_SetName("cache-poll");
 	THR_Init();
@@ -169,38 +168,38 @@ vwp_main(void *priv)
 	while (1) {
 		then = Wait_HeapDue(w, &wp);
 		if (wp == NULL)
-			i = -1;
+			t = -1;
 		else
-			i = (int)floor(1e3 * (then - VTIM_real()));
+			t = (int)floor(1e3 * (then - VTIM_real()));
 		assert(vwp->hpoll > 0);
 		AN(vwp->pollfd);
-		v = poll(vwp->pollfd, vwp->hpoll, i);
+		v = poll(vwp->pollfd, vwp->hpoll, t);
 		assert(v >= 0);
 		now = VTIM_real();
 		if (vwp->pollfd[0].revents)
 			v--;
-		for (i = 1; i < vwp->hpoll;) {
-			assert(vwp->pollfd[i].fd != vwp->pipes[0]);
-			wp = vwp->idx[i];
+		for (z = 1; z < vwp->hpoll;) {
+			assert(vwp->pollfd[z].fd != vwp->pipes[0]);
+			wp = vwp->idx[z];
 			CHECK_OBJ_NOTNULL(wp, WAITED_MAGIC);
 
 			if (v == 0 && Wait_HeapDue(w, NULL) > now)
 				break;
-			if (vwp->pollfd[i].revents)
+			if (vwp->pollfd[z].revents)
 				v--;
 			then = Wait_When(wp);
 			if (then <= now) {
 				AN(Wait_HeapDelete(w, wp));
 				Wait_Call(w, wp, WAITER_TIMEOUT, now);
-				vwp_del(vwp, i);
-			} else if (vwp->pollfd[i].revents & POLLIN) {
+				vwp_del(vwp, z);
+			} else if (vwp->pollfd[z].revents & POLLIN) {
 				assert(wp->fd > 0);
-				assert(wp->fd == vwp->pollfd[i].fd);
+				assert(wp->fd == vwp->pollfd[z].fd);
 				AN(Wait_HeapDelete(w, wp));
 				Wait_Call(w, wp, WAITER_ACTION, now);
-				vwp_del(vwp, i);
+				vwp_del(vwp, z);
 			} else {
-				i++;
+				z++;
 			}
 		}
 		if (vwp->pollfd[0].revents)
diff --git a/include/vrt.h b/include/vrt.h
index c78f2c402..ad1d78d51 100644
--- a/include/vrt.h
+++ b/include/vrt.h
@@ -52,6 +52,7 @@
  * binary/load-time compatible, increment MAJOR version
  *
  * unreleased (planned for 2020-03-15)
+ *	Changed type of vsa_suckaddr_len from int to size_t
  *	New prefix_{ptr|len} fields in vrt_backend
  *	VRT_HashStrands32() added
  * 10.0 (2019-09-15)
diff --git a/include/vsa.h b/include/vsa.h
index 67c3e6266..2e4f0da91 100644
--- a/include/vsa.h
+++ b/include/vsa.h
@@ -31,7 +31,7 @@
 #define VSA_H_INCLUDED
 
 struct suckaddr;
-extern const int vsa_suckaddr_len;
+extern const size_t vsa_suckaddr_len;
 extern const struct suckaddr *bogo_ip;
 
 void VSA_Init(void);
diff --git a/lib/libvarnish/vsa.c b/lib/libvarnish/vsa.c
index 836747eac..438126ccc 100644
--- a/lib/libvarnish/vsa.c
+++ b/lib/libvarnish/vsa.c
@@ -173,7 +173,7 @@ struct suckaddr {
 	};
 };
 
-const int vsa_suckaddr_len = sizeof(struct suckaddr);
+const size_t vsa_suckaddr_len = sizeof(struct suckaddr);
 
 /*
  * Bogus IPv4 address 0.0.0.0:0 to be used for VCL *.ip variables when the

From phk at FreeBSD.org  Mon Nov 11 09:13:05 2019
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 11 Nov 2019 09:13:05 +0000 (UTC)
Subject: [master] 940ba5763 Flip sense of test,
 hoping compilers will then not be upset about asserts not happening.
Message-ID: <20191111091305.7C4FD1054FB@lists.varnish-cache.org>


commit 940ba5763037c0020d1bb6bb442b800fcc8e7332
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Nov 11 09:12:16 2019 +0000

    Flip sense of test, hoping compilers will then not be upset about asserts not happening.

diff --git a/bin/varnishd/cache/cache_session.c b/bin/varnishd/cache/cache_session.c
index c94ad1f11..3548d6c57 100644
--- a/bin/varnishd/cache/cache_session.c
+++ b/bin/varnishd/cache/cache_session.c
@@ -151,14 +151,14 @@ SES_Set_String_Attr(struct sess *sp, enum sess_attr a, const char *src)
 	CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
 	AN(src);
 
-	/*lint -save -e568 -e685 */
-
+	/*lint -save -e506 -e774 */
 	switch (a) {
-#define SESS_ATTR(UP, low, typ, len)	case SA_##UP: assert(len < 0); break;
+#define SESS_ATTR(UP, low, typ, len) \
+		case SA_##UP: if (len > 0) WRONG("wrong: " #UP); break;
 #include "tbl/sess_attr.h"
-	default:  WRONG("wrong sess_attr");
+		default:
+			WRONG("wrong sess_attr");
 	}
-
 	/*lint -restore */
 
 	ses_reserve_attr(sp, a, &q, strlen(src) + 1);
@@ -172,14 +172,14 @@ SES_Get_String_Attr(const struct sess *sp, enum sess_attr a)
 
 	CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
 
-	/*lint -save -e568 -e685 */
-
+	/*lint -save -e506 -e774 */
 	switch (a) {
-#define SESS_ATTR(UP, low, typ, len)	case SA_##UP: assert(len < 0); break;
+#define SESS_ATTR(UP, low, typ, len) \
+		case SA_##UP: if (len > 0) WRONG("wrong: " #UP); break;
 #include "tbl/sess_attr.h"
-	default:  WRONG("wrong sess_attr");
+		default:
+			WRONG("wrong sess_attr");
 	}
-
 	/*lint -restore */
 
 	if (ses_get_attr(sp, a, &q) < 0)

From martin at varnish-software.com  Mon Nov 11 10:01:06 2019
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon, 11 Nov 2019 10:01:06 +0000 (UTC)
Subject: [master] f402e5ff3 Correct the probe heap comparison function
Message-ID: <20191111100106.F26F71065C2@lists.varnish-cache.org>


commit f402e5ff3063026f6ac779295b318da491a934a9
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Tue Nov 5 16:25:28 2019 +0100

    Correct the probe heap comparison function
    
    Fix the probe scheduler heap comparison function to be consistent with
    regard to different running state of the two arguments. With this fix,
    probes that are not running will always bubble to the top before those
    that are already running.

diff --git a/bin/varnishd/cache/cache_backend_probe.c b/bin/varnishd/cache/cache_backend_probe.c
index 97538cade..3339b722c 100644
--- a/bin/varnishd/cache/cache_backend_probe.c
+++ b/bin/varnishd/cache/cache_backend_probe.c
@@ -721,8 +721,8 @@ vbp_cmp(void *priv, const void *a, const void *b)
 	CAST_OBJ_NOTNULL(aa, a, VBP_TARGET_MAGIC);
 	CAST_OBJ_NOTNULL(bb, b, VBP_TARGET_MAGIC);
 
-	if (aa->running && !bb->running)
-		return (0);
+	if ((aa->running == 0) != (bb->running == 0))
+		return (aa->running == 0);
 
 	return (aa->due < bb->due);
 }

From phk at FreeBSD.org  Mon Nov 11 10:17:06 2019
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 11 Nov 2019 10:17:06 +0000 (UTC)
Subject: [master] 8756d0dcf Try to work around compilers which insist asserts
 must hold risk.
Message-ID: <20191111101706.B197F106CAF@lists.varnish-cache.org>


commit 8756d0dcfd3ec65992b5bfc412599fb23c869b61
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Nov 11 10:15:47 2019 +0000

    Try to work around compilers which insist asserts must hold risk.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 3c3f1ba94..a8dcc0395 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -546,7 +546,6 @@ struct req {
 
 enum sess_attr {
 #define SESS_ATTR(UP, low, typ, len)	SA_##UP,
-	SA_TRANSPORT,
 #include "tbl/sess_attr.h"
 	SA_LAST
 };
diff --git a/bin/varnishd/cache/cache_session.c b/bin/varnishd/cache/cache_session.c
index 3548d6c57..548674bec 100644
--- a/bin/varnishd/cache/cache_session.c
+++ b/bin/varnishd/cache/cache_session.c
@@ -49,6 +49,13 @@
 #include "vtim.h"
 #include "waiter/waiter.h"
 
+static const struct {
+	const char		*type;
+} sess_attr[SA_LAST] = {
+#define SESS_ATTR(UC, lc, typ, len) [SA_##UC] = { #typ },
+#include "tbl/sess_attr.h"
+};
+
 /*--------------------------------------------------------------------*/
 
 void
@@ -151,15 +158,9 @@ SES_Set_String_Attr(struct sess *sp, enum sess_attr a, const char *src)
 	CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
 	AN(src);
 
-	/*lint -save -e506 -e774 */
-	switch (a) {
-#define SESS_ATTR(UP, low, typ, len) \
-		case SA_##UP: if (len > 0) WRONG("wrong: " #UP); break;
-#include "tbl/sess_attr.h"
-		default:
-			WRONG("wrong sess_attr");
-	}
-	/*lint -restore */
+	assert (a >= SA_TRANSPORT && a <  SA_LAST);
+	if (strcmp(sess_attr[a].type, "char"))
+		WRONG("wrong sess_attr: not char");
 
 	ses_reserve_attr(sp, a, &q, strlen(src) + 1);
 	strcpy(q, src);
@@ -172,15 +173,9 @@ SES_Get_String_Attr(const struct sess *sp, enum sess_attr a)
 
 	CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
 
-	/*lint -save -e506 -e774 */
-	switch (a) {
-#define SESS_ATTR(UP, low, typ, len) \
-		case SA_##UP: if (len > 0) WRONG("wrong: " #UP); break;
-#include "tbl/sess_attr.h"
-		default:
-			WRONG("wrong sess_attr");
-	}
-	/*lint -restore */
+	assert (a >= SA_TRANSPORT && a <  SA_LAST);
+	if (strcmp(sess_attr[a].type, "char"))
+		WRONG("wrong sess_attr: not char");
 
 	if (ses_get_attr(sp, a, &q) < 0)
 		return (NULL);
diff --git a/include/tbl/sess_attr.h b/include/tbl/sess_attr.h
index 5276a1861..f4f4de0b0 100644
--- a/include/tbl/sess_attr.h
+++ b/include/tbl/sess_attr.h
@@ -33,6 +33,7 @@
 /*lint -save -e525 -e539 */
 
 //        upper           lower         type		    len
+SESS_ATTR(TRANSPORT,	  transport,	void,    	    0)
 SESS_ATTR(REMOTE_ADDR,	  remote_addr,	struct suckaddr,    vsa_suckaddr_len)
 SESS_ATTR(LOCAL_ADDR,	  local_addr,	struct suckaddr,    vsa_suckaddr_len)
 SESS_ATTR(CLIENT_ADDR,	  client_addr,	struct suckaddr,    vsa_suckaddr_len)

From phk at FreeBSD.org  Mon Nov 11 10:34:06 2019
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 11 Nov 2019 10:34:06 +0000 (UTC)
Subject: [master] 36477c6f6 Do not allow reservation of len <= 0 sess_attrs
Message-ID: <20191111103406.782CE107393@lists.varnish-cache.org>


commit 36477c6f6f2e68f1a76938cd03229e514ef01cb7
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Nov 11 10:32:58 2019 +0000

    Do not allow reservation of len <= 0 sess_attrs

diff --git a/bin/varnishd/cache/cache_session.c b/bin/varnishd/cache/cache_session.c
index 548674bec..afda89314 100644
--- a/bin/varnishd/cache/cache_session.c
+++ b/bin/varnishd/cache/cache_session.c
@@ -132,19 +132,21 @@ ses_reserve_attr(struct sess *sp, enum sess_attr a, void **dst, int sz)
 	int								\
 	SES_Set_##low(const struct sess *sp, const typ *src)		\
 	{								\
+		assert(len > 0);					\
 		return (ses_set_attr(sp, SA_##UP, src, len));		\
 	}								\
 									\
 	int								\
 	SES_Get_##low(const struct sess *sp, typ **dst)			\
 	{								\
+		assert(len > 0);					\
 		return (ses_get_attr(sp, SA_##UP, (void**)dst));	\
 	}								\
 									\
 	void								\
 	SES_Reserve_##low(struct sess *sp, typ **dst)			\
 	{								\
-		assert(len >= 0);					\
+		assert(len > 0);					\
 		ses_reserve_attr(sp, SA_##UP, (void**)dst, len);	\
 	}
 
diff --git a/include/tbl/sess_attr.h b/include/tbl/sess_attr.h
index f4f4de0b0..e61365c4d 100644
--- a/include/tbl/sess_attr.h
+++ b/include/tbl/sess_attr.h
@@ -38,8 +38,8 @@ SESS_ATTR(REMOTE_ADDR,	  remote_addr,	struct suckaddr,    vsa_suckaddr_len)
 SESS_ATTR(LOCAL_ADDR,	  local_addr,	struct suckaddr,    vsa_suckaddr_len)
 SESS_ATTR(CLIENT_ADDR,	  client_addr,	struct suckaddr,    vsa_suckaddr_len)
 SESS_ATTR(SERVER_ADDR,	  server_addr,	struct suckaddr,    vsa_suckaddr_len)
-SESS_ATTR(CLIENT_IP,	  client_ip,	char,		    -1)
-SESS_ATTR(CLIENT_PORT,	  client_port,	char,		    -1)
+SESS_ATTR(CLIENT_IP,	  client_ip,	char,		    0)
+SESS_ATTR(CLIENT_PORT,	  client_port,	char,		    0)
 SESS_ATTR(PROXY_TLV,	  proxy_tlv,	uintptr_t,	    sizeof(uintptr_t))
 SESS_ATTR(PROTO_PRIV,	  proto_priv,	uintptr_t,	    sizeof(uintptr_t))
 #undef SESS_ATTR

From phk at FreeBSD.org  Mon Nov 11 10:49:06 2019
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 11 Nov 2019 10:49:06 +0000 (UTC)
Subject: [master] 3b4fef08f Sigh, are enums signed or unsigned ?  Who knows...
Message-ID: <20191111104906.45320107982@lists.varnish-cache.org>


commit 3b4fef08f86a2bad0d8fe9e5dc027cd7680b5dd7
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Nov 11 10:48:00 2019 +0000

    Sigh, are enums signed or unsigned ?  Who knows...

diff --git a/bin/varnishd/cache/cache_session.c b/bin/varnishd/cache/cache_session.c
index afda89314..d536962bf 100644
--- a/bin/varnishd/cache/cache_session.c
+++ b/bin/varnishd/cache/cache_session.c
@@ -160,7 +160,7 @@ SES_Set_String_Attr(struct sess *sp, enum sess_attr a, const char *src)
 	CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
 	AN(src);
 
-	assert (a >= SA_TRANSPORT && a <  SA_LAST);
+	assert(a <  SA_LAST);
 	if (strcmp(sess_attr[a].type, "char"))
 		WRONG("wrong sess_attr: not char");
 
@@ -175,7 +175,7 @@ SES_Get_String_Attr(const struct sess *sp, enum sess_attr a)
 
 	CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
 
-	assert (a >= SA_TRANSPORT && a <  SA_LAST);
+	assert(a <  SA_LAST);
 	if (strcmp(sess_attr[a].type, "char"))
 		WRONG("wrong sess_attr: not char");
 

From gquintard at users.noreply.github.com  Mon Nov 11 17:22:06 2019
From: gquintard at users.noreply.github.com (guillaume quintard)
Date: Mon, 11 Nov 2019 17:22:06 +0000 (UTC)
Subject: [master] 649ad2ba7 add if-range logic
Message-ID: <20191111172206.3EA30110678@lists.varnish-cache.org>


commit 649ad2ba79f44fb0213331e8692361d1f6aa0daa
Author: Guillaume Quintard <guillaume at varnish-software.com>
Date:   Sat Nov 2 18:06:16 2019 -0700

    add if-range logic

diff --git a/bin/varnishd/cache/cache_range.c b/bin/varnishd/cache/cache_range.c
index ab96954fd..dcbfeef49 100644
--- a/bin/varnishd/cache/cache_range.c
+++ b/bin/varnishd/cache/cache_range.c
@@ -33,6 +33,7 @@
 #include "cache_filter.h"
 
 #include "vct.h"
+#include <vtim.h>
 
 /*--------------------------------------------------------------------*/
 
@@ -180,6 +181,58 @@ vrg_dorange(struct req *req, const char *r, void **priv)
 	return (NULL);
 }
 
+/*
+ * return 1 if range should be observed, based on if-range value
+ * if-range can either be a date or an ETag [RFC7233 3.2 p8]
+ */
+static int
+vrg_ifrange(struct req *req)
+{
+	const char *p, *e;
+	vtim_real ims, lm, d;
+
+	if (!http_GetHdr(req->http, H_If_Range, &p))	// rfc7233,l,455,456
+		return (1);
+
+	/* strong validation needed */
+	if (p[0] == 'W' && p[1] == '/')			// rfc7233,l,500,501
+		return (0);
+
+	/* ETag */
+	if (p[0] == '"') {				// rfc7233,l,512,514
+		if (!http_GetHdr(req->resp, H_ETag, &e))
+			return (0);
+		if ((e[0] == 'W' && e[1] == '/'))	// rfc7232,l,547,548
+			return (0);
+		return (strcmp(p, e) == 0);		// rfc7232,l,548,548
+	}
+
+	/* assume date, strong check [RFC7232 2.2.2 p7] */
+	if (!(ims = VTIM_parse(p)))			// rfc7233,l,502,512
+		return (0);
+
+	/* the response needs a Date */
+	// rfc7232 fc7232,l,439,440
+	if (!http_GetHdr(req->resp, H_Date, &p) || !(d = VTIM_parse(p)))
+		return (0);
+
+	/* grab the Last Modified value */
+	if (!http_GetHdr(req->resp, H_Last_Modified, &p))
+		return (0);
+
+	lm = VTIM_parse(p);
+	if (!lm)
+		return (0);
+	
+	/* Last Modified must be 60 seconds older than Date */
+	if (lm > d + 60)				// rfc7232,l,442,443
+		return (0);
+
+	if (lm != ims)					// rfc7233,l,455,456
+		return (0);
+	return (1);
+}
+
 static int v_matchproto_(vdp_init_f)
 vrg_range_init(struct req *req, void **priv)
 {
@@ -187,6 +240,8 @@ vrg_range_init(struct req *req, void **priv)
 	const char *err;
 
 	assert(http_GetHdr(req->http, H_Range, &r));
+	if (!vrg_ifrange(req))	// rfc7233,l,455,456
+		return (1);
 	err = vrg_dorange(req, r, priv);
 	if (err == NULL)
 		return (*priv == NULL ? 1 : 0);
diff --git a/bin/varnishtest/tests/c00100.vtc b/bin/varnishtest/tests/c00100.vtc
new file mode 100644
index 000000000..ebf690430
--- /dev/null
+++ b/bin/varnishtest/tests/c00100.vtc
@@ -0,0 +1,88 @@
+varnishtest "if-range header"
+
+server s1 {
+	rxreq
+	txresp -hdr {etag: "foo"} -hdr "last-modified: Wed, 21 Oct 2015 07:28:00 GMT" -bodylen 16
+
+	rxreq
+	txresp -bodylen 16
+} -start
+
+varnish v1 -vcl+backend {} -start
+
+client c1 {
+	txreq
+	rxresp
+	expect resp.status == 200
+	expect resp.bodylen == 16
+
+	# if-range, but no range
+	txreq -hdr {if-range: "foo"}
+	rxresp
+	expect resp.status == 200
+	expect resp.bodylen == 16
+
+	# non-matching etag if-range
+	txreq -hdr {if-range: "fooled"} -hdr "range: bytes=5-9"
+	rxresp
+	expect resp.status == 200
+	expect resp.bodylen == 16
+
+	# matching etag if-range
+	txreq -hdr {if-range: "foo"} -hdr "range: bytes=5-9"
+	rxresp
+	expect resp.status == 206
+	expect resp.bodylen == 5
+
+	# non-matching date if-range (past)
+	txreq -hdr "if-range: Wed, 21 Oct 2015 07:18:00 GMT" -hdr "range: bytes=5-9"
+	rxresp
+	expect resp.status == 200
+	expect resp.bodylen == 16
+
+	# non-matching date if-range (future)
+	txreq -hdr "if-range: Wed, 21 Oct 2015 07:38:00 GMT" -hdr "range: bytes=5-9"
+	rxresp
+	expect resp.status == 200
+	expect resp.bodylen == 16
+
+	# matching etag if-range
+	txreq -hdr "if-range: Wed, 21 Oct 2015 07:28:00 GMT" -hdr "range: bytes=5-9"
+	rxresp
+	expect resp.status == 206
+	expect resp.bodylen == 5
+}-run
+
+varnish v1 -cliok "ban obj.status != x"
+
+# no etag/LM header
+client c1 {
+	txreq
+	rxresp
+	expect resp.status == 200
+	expect resp.bodylen == 16
+
+	# non-matching etag if-range
+	txreq -hdr {if-range: "fooled"} -hdr "range: bytes=5-9"
+	rxresp
+	expect resp.status == 200
+	expect resp.bodylen == 16
+
+	# matching etag if-range
+	txreq -hdr {if-range: "foo"} -hdr "range: bytes=5-9"
+	rxresp
+	expect resp.status == 200
+	expect resp.bodylen == 16
+
+	# non-matching date if-range (past)
+	txreq -hdr "if-range: Wed, 21 Oct 2015 07:18:00 GMT" -hdr "range: bytes=5-9"
+	rxresp
+	expect resp.status == 200
+	expect resp.bodylen == 16
+
+	# non-matching date if-range (future)
+	txreq -hdr "if-range: Wed, 21 Oct 2015 07:38:00 GMT" -hdr "range: bytes=5-9"
+	rxresp
+	expect resp.status == 200
+	expect resp.bodylen == 16
+} -run

From guillaume at varnish-software.com  Mon Nov 11 17:44:06 2019
From: guillaume at varnish-software.com (Guillaume Quintard)
Date: Mon, 11 Nov 2019 17:44:06 +0000 (UTC)
Subject: [master] da6b9216d mention if-range in changelog
Message-ID: <20191111174406.40DFB110E79@lists.varnish-cache.org>


commit da6b9216dfdae0a2578169f8708a15eaedf1cb0d
Author: Guillaume Quintard <guillaume at varnish-software.com>
Date:   Mon Nov 11 09:43:20 2019 -0800

    mention if-range in changelog

diff --git a/doc/changes.rst b/doc/changes.rst
index 37c638da8..e0c5acadd 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -49,6 +49,9 @@ NEXT (2020-03-15)
   before the VCL callback, now it gets emitted after VCL returns for
   consistency with ``vcl_deliver {}``
 
+* The ``if-range`` header is now handled, allowing clients to conditionally
+  request a range based on a date or an ETag
+
 ================================
 Varnish Cache 6.3.0 (2019-09-15)
 ================================

From phk at FreeBSD.org  Tue Nov 12 09:11:07 2019
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Tue, 12 Nov 2019 09:11:07 +0000 (UTC)
Subject: [master] c638b74b9 Tuesday morning FlexeLinting
Message-ID: <20191112091107.7176264B14@lists.varnish-cache.org>


commit c638b74b9ee96d8a7205bd276cd3285c3a4fb107
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Tue Nov 12 09:09:51 2019 +0000

    Tuesday morning FlexeLinting

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index a8dcc0395..75bbbe4ad 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -388,7 +388,7 @@ struct busyobj {
 	 * All fields from retries and down are zeroed when the busyobj
 	 * is recycled.
 	 */
-	int			retries;
+	unsigned		retries;
 	struct req		*req;
 	struct sess		*sp;
 	struct worker		*wrk;
@@ -449,7 +449,7 @@ struct req {
 	enum req_step		req_step;
 	volatile enum req_body_state_e	req_body_status;
 	enum sess_close		doclose;
-	int			restarts;
+	unsigned		restarts;
 	unsigned		esi_level;
 	struct req		*topreq;	/* esi_level == 0 request */
 	struct vcl		*vcl0;
diff --git a/bin/varnishd/cache/cache_esi_parse.c b/bin/varnishd/cache/cache_esi_parse.c
index 1c1f13a04..ed67f0e5f 100644
--- a/bin/varnishd/cache/cache_esi_parse.c
+++ b/bin/varnishd/cache/cache_esi_parse.c
@@ -938,7 +938,7 @@ VEP_Parse(struct vep_state *vep, const char *p, size_t l)
 				vep->match = NULL;
 				vep->tag_i = 0;
 			} else {
-				assert(e - p <= sizeof(vep->tag));
+				assert(p + sizeof(vep->tag) >= e);
 				memcpy(vep->tag, p, e - p);
 				vep->tag_i = e - p;
 				vep->state = VEP_MATCHBUF;
diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c
index 9bffb8855..b98d3e535 100644
--- a/bin/varnishd/cache/cache_hash.c
+++ b/bin/varnishd/cache/cache_hash.c
@@ -248,8 +248,8 @@ static struct hsh_magiclist {
 static void
 hsh_testmagic(void *result)
 {
-	int i, j;
-	static int nused = 0;
+	size_t i, j;
+	static size_t nused = 0;
 
 	for (i = 0; i < nused; i++)
 		if (!memcmp(hsh_magiclist[i].was, result, VSHA256_LEN))
diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index c5ad6eb8e..343fc47dc 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -430,7 +430,7 @@ pan_busyobj(struct vsb *vsb, const struct busyobj *bo)
 	VSB_indent(vsb, 2);
 	PAN_CheckMagic(vsb, bo, BUSYOBJ_MAGIC);
 	VSB_printf(vsb, "end = %p,\n", bo->end);
-	VSB_printf(vsb, "retries = %d,\n", bo->retries);
+	VSB_printf(vsb, "retries = %u,\n", bo->retries);
 
 	if (bo->req != NULL)
 		pan_req(vsb, bo->req);
@@ -526,7 +526,7 @@ pan_req(struct vsb *vsb, const struct req *req)
 		    "err_code = %d, err_reason = %s,\n", req->err_code,
 		    req->err_reason ? req->err_reason : "(null)");
 
-	VSB_printf(vsb, "restarts = %d, esi_level = %u,\n",
+	VSB_printf(vsb, "restarts = %u, esi_level = %u,\n",
 	    req->restarts, req->esi_level);
 
 	if (req->sp != NULL)
diff --git a/bin/varnishd/http1/cache_http1_line.c b/bin/varnishd/http1/cache_http1_line.c
index ae924d66e..0fff1dd39 100644
--- a/bin/varnishd/http1/cache_http1_line.c
+++ b/bin/varnishd/http1/cache_http1_line.c
@@ -143,7 +143,7 @@ V1L_Close(struct worker *wrk, uint64_t *cnt)
 }
 
 static void
-v1l_prune(struct v1l *v1l, ssize_t bytes)
+v1l_prune(struct v1l *v1l, size_t bytes)
 {
 	ssize_t used = 0;
 	ssize_t j, used_here;
diff --git a/bin/varnishd/http2/cache_http2.h b/bin/varnishd/http2/cache_http2.h
index 430ab060c..4c263ce04 100644
--- a/bin/varnishd/http2/cache_http2.h
+++ b/bin/varnishd/http2/cache_http2.h
@@ -149,7 +149,7 @@ struct h2_sess {
 
 	struct sess			*sess;
 	int				refcnt;
-	int				open_streams;
+	unsigned			open_streams;
 	uint32_t			highest_stream;
 	int				bogosity;
 	int				do_sweep;
diff --git a/bin/varnishd/http2/cache_http2_proto.c b/bin/varnishd/http2/cache_http2_proto.c
index 0f2a21230..3803cbc81 100644
--- a/bin/varnishd/http2/cache_http2_proto.c
+++ b/bin/varnishd/http2/cache_http2_proto.c
@@ -906,19 +906,13 @@ static enum htc_status_e v_matchproto_(htc_complete_f)
 h2_frame_complete(struct http_conn *htc)
 {
 	struct h2_sess *h2;
-	int l;
-	unsigned u;
 
 	CHECK_OBJ_NOTNULL(htc, HTTP_CONN_MAGIC);
 	CAST_OBJ_NOTNULL(h2, htc->priv, H2_SESS_MAGIC);
-	l = htc->rxbuf_e - htc->rxbuf_b;
-	if (l < 9)
+	if (htc->rxbuf_b + 9 > htc->rxbuf_e ||
+	    htc->rxbuf_b + 9 + (vbe32dec(htc->rxbuf_b) >> 8) > htc->rxbuf_e)
 		return (HTC_S_MORE);
-	u = vbe32dec(htc->rxbuf_b) >> 8;
-	if (l >= u + 9)
-		return (HTC_S_COMPLETE);
-
-	return (HTC_S_MORE);
+	return (HTC_S_COMPLETE);
 }
 
 /**********************************************************************/

From nils.goroll at uplex.de  Wed Nov 13 07:08:09 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 13 Nov 2019 07:08:09 +0000 (UTC)
Subject: [master] 391b4551e h2: do not send zero data frames
Message-ID: <20191113070809.5D6FC11929D@lists.varnish-cache.org>


commit 391b4551e6849159947db37fd581c3d78c9137b5
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 13 07:57:47 2019 +0100

    h2: do not send zero data frames
    
    I noticed that the stream 0 header accounting changes with len == 0 VDP
    bytes, which seems useless.

diff --git a/bin/varnishd/http2/cache_http2_deliver.c b/bin/varnishd/http2/cache_http2_deliver.c
index f5faedce8..529042e33 100644
--- a/bin/varnishd/http2/cache_http2_deliver.c
+++ b/bin/varnishd/http2/cache_http2_deliver.c
@@ -109,6 +109,8 @@ h2_bytes(struct req *req, enum vdp_action act, void **priv,
 
 	if ((r2->h2sess->error || r2->error))
 		return (-1);
+	if (len == 0)
+		return (0);
 	H2_Send_Get(req->wrk, r2->h2sess, r2);
 	H2_Send(req->wrk, r2, H2_F_DATA, H2FF_NONE, len, ptr,
 	    &req->acct.resp_bodybytes);

From nils.goroll at uplex.de  Wed Nov 13 07:08:09 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 13 Nov 2019 07:08:09 +0000 (UTC)
Subject: [master] fe687fbe9 ws ocd
Message-ID: <20191113070809.798E21192A0@lists.varnish-cache.org>


commit fe687fbe9b9514e7c3a7d0d332404e8538d5a804
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 13 08:04:53 2019 +0100

    ws ocd

diff --git a/bin/varnishd/cache/cache_range.c b/bin/varnishd/cache/cache_range.c
index dcbfeef49..6370c0951 100644
--- a/bin/varnishd/cache/cache_range.c
+++ b/bin/varnishd/cache/cache_range.c
@@ -223,7 +223,7 @@ vrg_ifrange(struct req *req)
 	lm = VTIM_parse(p);
 	if (!lm)
 		return (0);
-	
+
 	/* Last Modified must be 60 seconds older than Date */
 	if (lm > d + 60)				// rfc7232,l,442,443
 		return (0);

From nils.goroll at uplex.de  Wed Nov 13 08:48:06 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 13 Nov 2019 08:48:06 +0000 (UTC)
Subject: [master] 6dfcc79d7 Reinstantiate "actually,
 the scope must not change either"
Message-ID: <20191113084806.5E68411B403@lists.varnish-cache.org>


commit 6dfcc79d708d9013a07aab8f135003323b607bca
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue May 28 13:09:22 2019 +0200

    Reinstantiate "actually, the scope must not change either"
    
    This reverts commit d6d3416090b8b7db62d2cd0958e756e23ea92352 which
    reverted 86af5ce095a6e39e313cf7c6c23a49edba4aace6.
    
    The regression documented in #3003 will be solved differently in
    follow-up commits.
    
    r03003.vtc added in d6d3416090b8b7db62d2cd0958e756e23ea92352 is kept

diff --git a/bin/varnishd/cache/cache_vrt_priv.c b/bin/varnishd/cache/cache_vrt_priv.c
index d8694ccdd..33bfc4926 100644
--- a/bin/varnishd/cache/cache_vrt_priv.c
+++ b/bin/varnishd/cache/cache_vrt_priv.c
@@ -42,7 +42,6 @@ struct vrt_priv {
 #define VRT_PRIV_MAGIC			0x24157a52
 	VRBT_ENTRY(vrt_priv)		entry;
 	struct vmod_priv		priv[1];
-	uintptr_t			id;
 	uintptr_t			vmod_id;
 };
 
@@ -95,9 +94,9 @@ VRTPRIV_init(struct vrt_privs *privs)
 static inline int
 vrt_priv_dyncmp(const struct vrt_priv *vp1, const struct vrt_priv *vp2)
 {
-	if (vp1->vmod_id < vp2->vmod_id || vp1->id < vp2->id)
+	if (vp1->vmod_id < vp2->vmod_id)
 		return (-1);
-	if (vp1->vmod_id > vp2->vmod_id || vp1->id > vp2->id)
+	if (vp1->vmod_id > vp2->vmod_id)
 		return (1);
 	return (0);
 }
@@ -105,22 +104,16 @@ vrt_priv_dyncmp(const struct vrt_priv *vp1, const struct vrt_priv *vp2)
 VRBT_GENERATE_STATIC(vrt_privs, vrt_priv, entry, vrt_priv_dyncmp)
 
 static struct vmod_priv *
-vrt_priv_dynamic(struct ws *ws, struct vrt_privs *privs, uintptr_t id,
-    uintptr_t vmod_id)
+vrt_priv_dynamic(struct ws *ws, struct vrt_privs *privs, uintptr_t vmod_id)
 {
 	struct vrt_priv *vp;
-	const struct vrt_priv needle = {
-		.id = id,
-		.vmod_id = vmod_id,
-	};
+	const struct vrt_priv needle = {.vmod_id = vmod_id};
 
-	AN(id);
 	AN(vmod_id);
 
 	vp = VRBT_FIND(vrt_privs, privs, &needle);
 	if (vp) {
 		CHECK_OBJ(vp, VRT_PRIV_MAGIC);
-		assert(vp->id == id);
 		assert(vp->vmod_id == vmod_id);
 		return (vp->priv);
 	}
@@ -129,7 +122,6 @@ vrt_priv_dynamic(struct ws *ws, struct vrt_privs *privs, uintptr_t id,
 	if (vp == NULL)
 		return (NULL);
 	INIT_OBJ(vp, VRT_PRIV_MAGIC);
-	vp->id = id;
 	vp->vmod_id = vmod_id;
 	VRBT_INSERT(vrt_privs, privs, vp);
 	return (vp->priv);
@@ -148,7 +140,6 @@ VRT_priv_task(VRT_CTX, const void *vmod_id)
 		return (vrt_priv_dynamic(
 		    ctx->ws,
 		    ctx->req->privs,
-		    (uintptr_t)ctx->req,
 		    (uintptr_t)vmod_id
 		));
 	}
@@ -157,7 +148,6 @@ VRT_priv_task(VRT_CTX, const void *vmod_id)
 		return (vrt_priv_dynamic(
 		    ctx->ws,
 		    ctx->bo->privs,
-		    (uintptr_t)ctx->bo,
 		    (uintptr_t)vmod_id
 		));
 	}
@@ -166,7 +156,6 @@ VRT_priv_task(VRT_CTX, const void *vmod_id)
 	return (vrt_priv_dynamic(
 	    ctx->ws,
 	    cli_task_privs,
-	    (uintptr_t)cli_task_privs,
 	    (uintptr_t)vmod_id
 	));
 }
@@ -178,7 +167,6 @@ VRT_priv_top(VRT_CTX, const void *vmod_id)
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	if (ctx->req == NULL) {
-		/* XXX: should we VRT_fail here instead? */
 		WRONG("PRIV_TOP is only accessible in client VCL context");
 		NEEDLESS(return (NULL));
 	}
@@ -187,7 +175,6 @@ VRT_priv_top(VRT_CTX, const void *vmod_id)
 	return (vrt_priv_dynamic(
 	    req->ws,
 	    req->privs,
-	    (uintptr_t)&req->topreq,
 	    (uintptr_t)vmod_id
 	));
 }

From nils.goroll at uplex.de  Wed Nov 13 08:48:06 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 13 Nov 2019 08:48:06 +0000 (UTC)
Subject: [master] a8e34499f Revert "Now that we have decided bump VRT_MAJOR,
 drop the reqtop struct"
Message-ID: <20191113084806.8DC5811B407@lists.varnish-cache.org>


commit a8e34499f295f20ca28c6c1b108c71e107c854cb
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue May 28 13:14:55 2019 +0200

    Revert "Now that we have decided bump VRT_MAJOR, drop the reqtop struct"
    
    This reverts commit d67993451bda7eeb7c0ea3c5528a8353f283a190.
    
    We will use struct reqtop for vcl0 and PRIV_TOP in follow-up commits.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 75bbbe4ad..689fe90df 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -441,6 +441,11 @@ struct busyobj {
 
 /*--------------------------------------------------------------------*/
 
+struct reqtop {
+	unsigned		magic;
+#define REQTOP_MAGIC		0x57fbda52
+	struct req		*topreq;
+};
 
 struct req {
 	unsigned		magic;
@@ -451,7 +456,7 @@ struct req {
 	enum sess_close		doclose;
 	unsigned		restarts;
 	unsigned		esi_level;
-	struct req		*topreq;	/* esi_level == 0 request */
+	struct reqtop		*top;	/* esi_level == 0 request */
 	struct vcl		*vcl0;
 
 #define REQ_FLAG(l, r, w, d) unsigned	l:1;
@@ -532,7 +537,7 @@ struct req {
 	struct vcf		*vcf;
 };
 
-#define IS_TOPREQ(req) ((req)->topreq == (req))
+#define IS_TOPREQ(req) ((req)->top == NULL || (req)->top->topreq == (req))
 
 /*--------------------------------------------------------------------
  * Struct sess is a high memory-load structure because sessions typically
diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c
index f9fe82241..51413eaf4 100644
--- a/bin/varnishd/cache/cache_esi_deliver.c
+++ b/bin/varnishd/cache/cache_esi_deliver.c
@@ -113,7 +113,7 @@ ved_include(struct req *preq, const char *src, const char *host,
 	enum req_fsm_nxt s;
 
 	CHECK_OBJ_NOTNULL(preq, REQ_MAGIC);
-	CHECK_OBJ_NOTNULL(preq->topreq, REQ_MAGIC);
+	CHECK_OBJ_NOTNULL(preq->top, REQTOP_MAGIC);
 	sp = preq->sp;
 	CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
 	CHECK_OBJ_NOTNULL(ecx, ECX_MAGIC);
@@ -139,7 +139,7 @@ ved_include(struct req *preq, const char *src, const char *host,
 
 	req->esi_level = preq->esi_level + 1;
 
-	req->topreq = preq->topreq;
+	req->top = preq->top;
 
 	HTTP_Setup(req->http, req->ws, req->vsl, SLT_ReqMethod);
 	HTTP_Dup(req->http, preq->http0);
@@ -171,8 +171,8 @@ ved_include(struct req *preq, const char *src, const char *host,
 	req->req_body_status = REQ_BODY_NONE;
 
 	AZ(req->vcl);
-	if (req->topreq->vcl0)
-		req->vcl = req->topreq->vcl0;
+	if (req->vcl0)
+		req->vcl = req->vcl0;
 	else
 		req->vcl = preq->vcl;
 	VCL_Ref(req->vcl);
@@ -264,6 +264,16 @@ ved_vdp_esi_init(struct req *req, void **priv)
 	*priv = ecx;
 	RFC2616_Weaken_Etag(req->resp);
 
+	if (IS_TOPREQ(req)) {
+		Req_MakeTop(req);
+		if (req->top == NULL) {
+			VSLb(req->vsl, SLT_Error,
+			    "(top)request workspace overflow");
+			Req_Fail(req, SC_OVERLOAD);
+			return (-1);
+		}
+	}
+
 	req->res_mode |= RES_ESI;
 	if (req->resp_len != 0)
 		req->resp_len = -1;
diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index 343fc47dc..6d7391f44 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -485,6 +485,20 @@ pan_busyobj(struct vsb *vsb, const struct busyobj *bo)
 
 /*--------------------------------------------------------------------*/
 
+static void
+pan_top(struct vsb *vsb, const struct reqtop *top)
+{
+	VSB_printf(vsb, "top = %p {\n", top);
+	if (PAN_already(vsb, top))
+		return;
+	VSB_indent(vsb, 2);
+	pan_req(vsb, top->topreq);
+	VSB_indent(vsb, -2);
+	VSB_cat(vsb, "},\n");
+}
+
+/*--------------------------------------------------------------------*/
+
 static void
 pan_req(struct vsb *vsb, const struct req *req)
 {
@@ -561,11 +575,8 @@ pan_req(struct vsb *vsb, const struct req *req)
 
 	pan_privs(vsb, req->privs);
 
-	VSB_cat(vsb, "topreq = {\n");
-	VSB_indent(vsb, 2);
-	pan_req(vsb, req->topreq);
-	VSB_indent(vsb, -2);
-	VSB_cat(vsb, "},\n");
+	if (req->top != NULL)
+		pan_top(vsb, req->top);
 
 	VSB_indent(vsb, -2);
 	VSB_cat(vsb, "},\n");
diff --git a/bin/varnishd/cache/cache_req.c b/bin/varnishd/cache/cache_req.c
index 767903682..7812172de 100644
--- a/bin/varnishd/cache/cache_req.c
+++ b/bin/varnishd/cache/cache_req.c
@@ -63,7 +63,6 @@ Req_AcctLogCharge(struct VSC_main_wrk *ds, struct req *req)
 		    (uintmax_t)(a->resp_hdrbytes + a->resp_bodybytes));
 	}
 
-	AN(req->topreq);
 	if (IS_TOPREQ(req)) {
 #define ACCT(foo) ds->s_##foo += a->foo;
 #include "tbl/acct_fields_req.h"
@@ -146,8 +145,6 @@ Req_New(const struct worker *wrk, struct sess *sp)
 	req->t_prev = NAN;
 	req->t_req = NAN;
 
-	req->topreq = req;
-
 	return (req);
 }
 
@@ -173,7 +170,7 @@ Req_Release(struct req *req)
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 	MPL_AssertSane(req);
 	VSL_Flush(req->vsl, 0);
-	req->topreq = NULL;
+	req->top = NULL;
 	MPL_Free(pp->mpl_req, req);
 }
 
@@ -192,6 +189,7 @@ Req_Rollback(struct req *req)
 	if (WS_Overflowed(req->ws))
 		req->wrk->stats->ws_client_overflow++;
 	WS_Reset(req->ws, req->ws_req);
+	req->top = NULL;
 }
 
 /*----------------------------------------------------------------------
@@ -205,7 +203,6 @@ Req_Cleanup(struct sess *sp, struct worker *wrk, struct req *req)
 	CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
-	CHECK_OBJ_NOTNULL(req->topreq, REQ_MAGIC);
 	assert(sp == req->sp);
 	AZ(req->vcl0);
 
@@ -238,6 +235,7 @@ Req_Cleanup(struct sess *sp, struct worker *wrk, struct req *req)
 	req->hash_ignore_busy = 0;
 	req->esi_level = 0;
 	req->is_hit = 0;
+	req->top = 0;
 
 	if (WS_Overflowed(req->ws))
 		wrk->stats->ws_client_overflow++;
@@ -256,3 +254,20 @@ Req_Fail(struct req *req, enum sess_close reason)
 	AN(req->transport->req_fail);
 	req->transport->req_fail(req, reason);
 }
+
+/*----------------------------------------------------------------------
+ */
+
+void
+Req_MakeTop(struct req *req)
+{
+
+	CHECK_OBJ_ORNULL(req->top, REQTOP_MAGIC);
+	if (req->top != NULL)
+		return;
+	req->top = WS_Alloc(req->ws, sizeof *req->top);
+	if (req->top != NULL) {
+		INIT_OBJ(req->top, REQTOP_MAGIC);
+		req->top->topreq = req;
+	}
+}
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index 4fb23dd92..fe53f756e 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -1041,7 +1041,6 @@ CNT_Embark(struct worker *wrk, struct req *req)
 
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
-	CHECK_OBJ_NOTNULL(req->topreq, REQ_MAGIC);
 
 	/* wrk can have changed for restarts */
 	req->vfc->wrk = req->wrk = wrk;
diff --git a/bin/varnishd/cache/cache_varnishd.h b/bin/varnishd/cache/cache_varnishd.h
index b8ccb4704..b455600ca 100644
--- a/bin/varnishd/cache/cache_varnishd.h
+++ b/bin/varnishd/cache/cache_varnishd.h
@@ -351,6 +351,7 @@ void Req_Rollback(struct req *req);
 void Req_Cleanup(struct sess *sp, struct worker *wrk, struct req *req);
 void Req_Fail(struct req *req, enum sess_close reason);
 void Req_AcctLogCharge(struct VSC_main_wrk *, struct req *);
+void Req_MakeTop(struct req *req);
 
 /* cache_req_body.c */
 int VRB_Ignore(struct req *);
diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index f9072c068..51cc21430 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -97,13 +97,15 @@ VCL_Req2Ctx(struct vrt_ctx *ctx, struct req *req)
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
-	CHECK_OBJ_ORNULL(req->topreq, REQ_MAGIC);
-	CHECK_OBJ_NOTNULL(req->wrk, WORKER_MAGIC);
 
 	ctx->vcl = req->vcl;
 	ctx->vsl = req->vsl;
 	ctx->http_req = req->http;
-	ctx->http_req_top = req->topreq->http;
+	CHECK_OBJ_ORNULL(req->top, REQTOP_MAGIC);
+	if (req->top != NULL)
+		ctx->http_req_top = req->top->topreq->http;
+	else
+		ctx->http_req_top = req->http;
 	ctx->http_resp = req->resp;
 	ctx->req = req;
 	ctx->sp = req->sp;
diff --git a/bin/varnishd/cache/cache_vrt_priv.c b/bin/varnishd/cache/cache_vrt_priv.c
index 33bfc4926..d8b2988d0 100644
--- a/bin/varnishd/cache/cache_vrt_priv.c
+++ b/bin/varnishd/cache/cache_vrt_priv.c
@@ -171,7 +171,12 @@ VRT_priv_top(VRT_CTX, const void *vmod_id)
 		NEEDLESS(return (NULL));
 	}
 	CHECK_OBJ_NOTNULL(ctx->req, REQ_MAGIC);
-	req = ctx->req->topreq;
+	req = ctx->req;
+	if (req->top != NULL) {
+		CHECK_OBJ_NOTNULL(req->top, REQTOP_MAGIC);
+		CHECK_OBJ_NOTNULL(req->top->topreq, REQ_MAGIC);
+		req = req->top->topreq;
+	}
 	return (vrt_priv_dynamic(
 	    req->ws,
 	    req->privs,
diff --git a/bin/varnishd/cache/cache_vrt_vcl.c b/bin/varnishd/cache/cache_vrt_vcl.c
index 2617db9e1..453499b99 100644
--- a/bin/varnishd/cache/cache_vrt_vcl.c
+++ b/bin/varnishd/cache/cache_vrt_vcl.c
@@ -428,6 +428,7 @@ vcl_call_method(struct worker *wrk, struct req *req, struct busyobj *bo,
 		CHECK_OBJ(req, REQ_MAGIC);
 		CHECK_OBJ_NOTNULL(req->sp, SESS_MAGIC);
 		CHECK_OBJ_NOTNULL(req->vcl, VCL_MAGIC);
+		CHECK_OBJ_ORNULL(req->top, REQTOP_MAGIC);
 		VCL_Req2Ctx(&ctx, req);
 	}
 	if (bo != NULL) {

From nils.goroll at uplex.de  Wed Nov 13 08:48:06 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 13 Nov 2019 08:48:06 +0000 (UTC)
Subject: [master] 244705bd6 give all requests a struct reqtop
Message-ID: <20191113084806.AE71911B40A@lists.varnish-cache.org>


commit 244705bd619c914526663d6943d573e0a835d724
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue Nov 5 17:44:21 2019 +0100

    give all requests a struct reqtop
    
    Dynamically creating it through Req_MakeTop() would further complicate
    rollbacks.
    
    The memory overhead is basically identical to embedding struct reqtop
    into struct req, except that, for ESI, we have the (struct req).top
    point to the top request's struct reqtop.
    
    With this commit, tests/r02849.vtc and tests/r03003.vtc are failing as
    excpected. While this may impose issues with git bisect, I still think
    that this extra commit helps clarity.

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 689fe90df..898b6b83e 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -537,7 +537,7 @@ struct req {
 	struct vcf		*vcf;
 };
 
-#define IS_TOPREQ(req) ((req)->top == NULL || (req)->top->topreq == (req))
+#define IS_TOPREQ(req) ((req)->top->topreq == (req))
 
 /*--------------------------------------------------------------------
  * Struct sess is a high memory-load structure because sessions typically
diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c
index 51413eaf4..cb7a6e1c8 100644
--- a/bin/varnishd/cache/cache_esi_deliver.c
+++ b/bin/varnishd/cache/cache_esi_deliver.c
@@ -139,6 +139,7 @@ ved_include(struct req *preq, const char *src, const char *host,
 
 	req->esi_level = preq->esi_level + 1;
 
+	memset(req->top, 0, sizeof *req->top);
 	req->top = preq->top;
 
 	HTTP_Setup(req->http, req->ws, req->vsl, SLT_ReqMethod);
@@ -264,16 +265,6 @@ ved_vdp_esi_init(struct req *req, void **priv)
 	*priv = ecx;
 	RFC2616_Weaken_Etag(req->resp);
 
-	if (IS_TOPREQ(req)) {
-		Req_MakeTop(req);
-		if (req->top == NULL) {
-			VSLb(req->vsl, SLT_Error,
-			    "(top)request workspace overflow");
-			Req_Fail(req, SC_OVERLOAD);
-			return (-1);
-		}
-	}
-
 	req->res_mode |= RES_ESI;
 	if (req->resp_len != 0)
 		req->resp_len = -1;
diff --git a/bin/varnishd/cache/cache_req.c b/bin/varnishd/cache/cache_req.c
index 7812172de..10b986472 100644
--- a/bin/varnishd/cache/cache_req.c
+++ b/bin/varnishd/cache/cache_req.c
@@ -137,6 +137,11 @@ Req_New(const struct worker *wrk, struct sess *sp)
 	INIT_OBJ(req->htc, HTTP_CONN_MAGIC);
 	p = (void*)PRNDUP(p + sizeof(*req->htc));
 
+	req->top = (void*)p;
+	INIT_OBJ(req->top, REQTOP_MAGIC);
+	req->top->topreq = req;
+	p = (void*)PRNDUP(p + sizeof(*req->top));
+
 	assert(p < e);
 
 	WS_Init(req->ws, "req", p, e - p);
@@ -170,7 +175,6 @@ Req_Release(struct req *req)
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 	MPL_AssertSane(req);
 	VSL_Flush(req->vsl, 0);
-	req->top = NULL;
 	MPL_Free(pp->mpl_req, req);
 }
 
@@ -189,7 +193,6 @@ Req_Rollback(struct req *req)
 	if (WS_Overflowed(req->ws))
 		req->wrk->stats->ws_client_overflow++;
 	WS_Reset(req->ws, req->ws_req);
-	req->top = NULL;
 }
 
 /*----------------------------------------------------------------------
@@ -235,7 +238,6 @@ Req_Cleanup(struct sess *sp, struct worker *wrk, struct req *req)
 	req->hash_ignore_busy = 0;
 	req->esi_level = 0;
 	req->is_hit = 0;
-	req->top = 0;
 
 	if (WS_Overflowed(req->ws))
 		wrk->stats->ws_client_overflow++;
@@ -254,20 +256,3 @@ Req_Fail(struct req *req, enum sess_close reason)
 	AN(req->transport->req_fail);
 	req->transport->req_fail(req, reason);
 }
-
-/*----------------------------------------------------------------------
- */
-
-void
-Req_MakeTop(struct req *req)
-{
-
-	CHECK_OBJ_ORNULL(req->top, REQTOP_MAGIC);
-	if (req->top != NULL)
-		return;
-	req->top = WS_Alloc(req->ws, sizeof *req->top);
-	if (req->top != NULL) {
-		INIT_OBJ(req->top, REQTOP_MAGIC);
-		req->top->topreq = req;
-	}
-}
diff --git a/bin/varnishd/cache/cache_varnishd.h b/bin/varnishd/cache/cache_varnishd.h
index b455600ca..b8ccb4704 100644
--- a/bin/varnishd/cache/cache_varnishd.h
+++ b/bin/varnishd/cache/cache_varnishd.h
@@ -351,7 +351,6 @@ void Req_Rollback(struct req *req);
 void Req_Cleanup(struct sess *sp, struct worker *wrk, struct req *req);
 void Req_Fail(struct req *req, enum sess_close reason);
 void Req_AcctLogCharge(struct VSC_main_wrk *, struct req *);
-void Req_MakeTop(struct req *req);
 
 /* cache_req_body.c */
 int VRB_Ignore(struct req *);
diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 51cc21430..fe905fea8 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -101,11 +101,8 @@ VCL_Req2Ctx(struct vrt_ctx *ctx, struct req *req)
 	ctx->vcl = req->vcl;
 	ctx->vsl = req->vsl;
 	ctx->http_req = req->http;
-	CHECK_OBJ_ORNULL(req->top, REQTOP_MAGIC);
-	if (req->top != NULL)
-		ctx->http_req_top = req->top->topreq->http;
-	else
-		ctx->http_req_top = req->http;
+	CHECK_OBJ_NOTNULL(req->top, REQTOP_MAGIC);
+	ctx->http_req_top = req->top->topreq->http;
 	ctx->http_resp = req->resp;
 	ctx->req = req;
 	ctx->sp = req->sp;
diff --git a/bin/varnishd/cache/cache_vrt_vcl.c b/bin/varnishd/cache/cache_vrt_vcl.c
index 453499b99..66f0ce9e5 100644
--- a/bin/varnishd/cache/cache_vrt_vcl.c
+++ b/bin/varnishd/cache/cache_vrt_vcl.c
@@ -428,7 +428,7 @@ vcl_call_method(struct worker *wrk, struct req *req, struct busyobj *bo,
 		CHECK_OBJ(req, REQ_MAGIC);
 		CHECK_OBJ_NOTNULL(req->sp, SESS_MAGIC);
 		CHECK_OBJ_NOTNULL(req->vcl, VCL_MAGIC);
-		CHECK_OBJ_ORNULL(req->top, REQTOP_MAGIC);
+		CHECK_OBJ_NOTNULL(req->top, REQTOP_MAGIC);
 		VCL_Req2Ctx(&ctx, req);
 	}
 	if (bo != NULL) {

From nils.goroll at uplex.de  Wed Nov 13 08:48:06 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 13 Nov 2019 08:48:06 +0000 (UTC)
Subject: [master] fc04ea211 move vcl0 to req->top
Message-ID: <20191113084806.D041411B40E@lists.varnish-cache.org>


commit fc04ea211f2769cdeb540115ed84ee8908a48cb8
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed May 29 11:18:01 2019 +0200

    move vcl0 to req->top
    
    Also fix some errors in vcl0 handling:
    
    - Only the top request may release vcl0 because it owns it
    - because we can re-embark for ESI, we can not assert that vcl0 is NULL
      in CNT_Embark()
    
    passes tests/r02849.vtc again.
    
    still fails r03003.vtc, which will get fixed in a follow up commit
    
    Fixes #3019 with test case by @Dridi

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index 898b6b83e..c9933dad1 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -445,6 +445,7 @@ struct reqtop {
 	unsigned		magic;
 #define REQTOP_MAGIC		0x57fbda52
 	struct req		*topreq;
+	struct vcl		*vcl0;
 };
 
 struct req {
@@ -457,7 +458,6 @@ struct req {
 	unsigned		restarts;
 	unsigned		esi_level;
 	struct reqtop		*top;	/* esi_level == 0 request */
-	struct vcl		*vcl0;
 
 #define REQ_FLAG(l, r, w, d) unsigned	l:1;
 #include "tbl/req_flags.h"
diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c
index cb7a6e1c8..99547f73c 100644
--- a/bin/varnishd/cache/cache_esi_deliver.c
+++ b/bin/varnishd/cache/cache_esi_deliver.c
@@ -172,8 +172,9 @@ ved_include(struct req *preq, const char *src, const char *host,
 	req->req_body_status = REQ_BODY_NONE;
 
 	AZ(req->vcl);
-	if (req->vcl0)
-		req->vcl = req->vcl0;
+	AN(req->top);
+	if (req->top->vcl0)
+		req->vcl = req->top->vcl0;
 	else
 		req->vcl = preq->vcl;
 	VCL_Ref(req->vcl);
diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index 6d7391f44..d48b396ee 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -493,6 +493,7 @@ pan_top(struct vsb *vsb, const struct reqtop *top)
 		return;
 	VSB_indent(vsb, 2);
 	pan_req(vsb, top->topreq);
+	VCL_Panic(vsb, "vcl0", top->vcl0);
 	VSB_indent(vsb, -2);
 	VSB_cat(vsb, "},\n");
 }
@@ -559,7 +560,6 @@ pan_req(struct vsb *vsb, const struct req *req)
 		pan_vdp(vsb, req->vdc);
 
 	VCL_Panic(vsb, "vcl", req->vcl);
-	VCL_Panic(vsb, "vcl0", req->vcl0);
 
 	if (req->body_oc != NULL)
 		pan_objcore(vsb, "BODY", req->body_oc);
diff --git a/bin/varnishd/cache/cache_req.c b/bin/varnishd/cache/cache_req.c
index 10b986472..addfda991 100644
--- a/bin/varnishd/cache/cache_req.c
+++ b/bin/varnishd/cache/cache_req.c
@@ -207,7 +207,8 @@ Req_Cleanup(struct sess *sp, struct worker *wrk, struct req *req)
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
 	assert(sp == req->sp);
-	AZ(req->vcl0);
+	if (IS_TOPREQ(req))
+		AZ(req->top->vcl0);
 
 	req->director_hint = NULL;
 	req->restarts = 0;
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index fe53f756e..cc64b31b7 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -1052,7 +1052,6 @@ CNT_Embark(struct worker *wrk, struct req *req)
 		VSLb(req->vsl, SLT_VCL_use, "%s", VCL_Name(req->vcl));
 	}
 
-	AZ(req->vcl0);
 	AN(req->vcl);
 }
 
@@ -1063,7 +1062,6 @@ CNT_Request(struct req *req)
 	enum req_fsm_nxt nxt;
 
 	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
-	assert(IS_TOPREQ(req) || req->vcl0 == NULL);
 
 	wrk = req->wrk;
 	CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC);
@@ -1106,8 +1104,8 @@ CNT_Request(struct req *req)
 	}
 	wrk->vsl = NULL;
 	if (nxt == REQ_FSM_DONE) {
-		if (IS_TOPREQ(req) && req->vcl0 != NULL)
-			VCL_Rel(&req->vcl0);
+		if (IS_TOPREQ(req) && req->top->vcl0 != NULL)
+			VCL_Rel(&req->top->vcl0);
 		VCL_TaskLeave(req->vcl, req->privs);
 		AN(req->vsl->wid);
 		VRB_Free(req);
diff --git a/bin/varnishd/cache/cache_vpi.c b/bin/varnishd/cache/cache_vpi.c
index 17ac4d1dc..b9740b267 100644
--- a/bin/varnishd/cache/cache_vpi.c
+++ b/bin/varnishd/cache/cache_vpi.c
@@ -91,12 +91,14 @@ VPI_vcl_select(VRT_CTX, VCL_VCL vcl)
 
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
 
-	if (IS_TOPREQ(req) && req->vcl0 != NULL)
+	if (IS_TOPREQ(req) && req->top->vcl0 != NULL)
 		return;		// Illegal, req-FSM will fail this later.
 
 	VCL_TaskLeave(req->vcl, req->privs);
 	if (IS_TOPREQ(req)) {
-		req->vcl0 = req->vcl;
+		AN(req->top);
+		AZ(req->top->vcl0);
+		req->top->vcl0 = req->vcl;
 		req->vcl = NULL;
 	} else {
 		VCL_Rel(&req->vcl);
diff --git a/bin/varnishd/http1/cache_http1_fsm.c b/bin/varnishd/http1/cache_http1_fsm.c
index 9f1796092..9418ce8c2 100644
--- a/bin/varnishd/http1/cache_http1_fsm.c
+++ b/bin/varnishd/http1/cache_http1_fsm.c
@@ -414,7 +414,7 @@ HTTP1_Session(struct worker *wrk, struct req *req)
 				VCL_TaskEnter(req->vcl, req->privs);
 			if (CNT_Request(req) == REQ_FSM_DISEMBARK)
 				return;
-			AZ(req->vcl0);
+			AZ(req->top->vcl0);
 			req->task.func = NULL;
 			req->task.priv = NULL;
 			AZ(req->ws->r);
diff --git a/bin/varnishd/http2/cache_http2_proto.c b/bin/varnishd/http2/cache_http2_proto.c
index 3803cbc81..b9017e76f 100644
--- a/bin/varnishd/http2/cache_http2_proto.c
+++ b/bin/varnishd/http2/cache_http2_proto.c
@@ -529,7 +529,7 @@ h2_do_req(struct worker *wrk, void *priv)
 	wrk->stats->client_req++;
 	if (CNT_Request(req) != REQ_FSM_DISEMBARK) {
 		AZ(req->ws->r);
-		AZ(req->vcl0);
+		AZ(req->top->vcl0);
 		h2 = r2->h2sess;
 		CHECK_OBJ_NOTNULL(h2, H2_SESS_MAGIC);
 		Lck_Lock(&h2->sess->mtx);
diff --git a/bin/varnishtest/tests/r03019.vtc b/bin/varnishtest/tests/r03019.vtc
new file mode 100644
index 000000000..f3eebf026
--- /dev/null
+++ b/bin/varnishtest/tests/r03019.vtc
@@ -0,0 +1,35 @@
+varnishtest "return(vcl) then reembark"
+
+barrier b1 cond 2
+
+server s1 {
+	rxreq
+	barrier b1 sync
+	txresp
+} -start
+
+varnish v1 -vcl+backend ""
+varnish v1 -cliok "vcl.label lbl vcl1"
+varnish v1 -vcl {
+	backend be { .host = "${bad_backend}"; }
+
+	sub vcl_recv {
+		return (vcl(lbl));
+	}
+} -start
+
+client c1 {
+	txreq
+	rxresp
+	expect resp.status == 200
+} -start
+
+client c2 {
+	barrier b1 sync
+	txreq
+	rxresp
+	expect resp.status == 200
+} -start
+
+client c1 -wait
+client c2 -wait

From nils.goroll at uplex.de  Wed Nov 13 08:48:06 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 13 Nov 2019 08:48:06 +0000 (UTC)
Subject: [master] 542bf9b82 give PRIV_TOP a head separate from (req)PRIV_TASK
 in struct reqtop
Message-ID: <20191113084807.0276111B413@lists.varnish-cache.org>


commit 542bf9b820bbc40c2c95b5fa0282621b997ac594
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Jun 5 18:32:50 2019 +0200

    give PRIV_TOP a head separate from (req)PRIV_TASK in struct reqtop
    
    This fixes #3003 properly
    
    restore tests/r02219.vtc to the same headroom as before
    
    we need additional workspace for the priv_top which now always gets
    initialized (32 bytes on my machine)

diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h
index c9933dad1..f45e9f1d2 100644
--- a/bin/varnishd/cache/cache.h
+++ b/bin/varnishd/cache/cache.h
@@ -446,6 +446,7 @@ struct reqtop {
 #define REQTOP_MAGIC		0x57fbda52
 	struct req		*topreq;
 	struct vcl		*vcl0;
+	struct vrt_privs	privs[1];
 };
 
 struct req {
diff --git a/bin/varnishd/cache/cache_panic.c b/bin/varnishd/cache/cache_panic.c
index d48b396ee..5f29632e3 100644
--- a/bin/varnishd/cache/cache_panic.c
+++ b/bin/varnishd/cache/cache_panic.c
@@ -493,6 +493,7 @@ pan_top(struct vsb *vsb, const struct reqtop *top)
 		return;
 	VSB_indent(vsb, 2);
 	pan_req(vsb, top->topreq);
+	pan_privs(vsb, top->privs);
 	VCL_Panic(vsb, "vcl0", top->vcl0);
 	VSB_indent(vsb, -2);
 	VSB_cat(vsb, "},\n");
diff --git a/bin/varnishd/cache/cache_req.c b/bin/varnishd/cache/cache_req.c
index addfda991..3516d2a10 100644
--- a/bin/varnishd/cache/cache_req.c
+++ b/bin/varnishd/cache/cache_req.c
@@ -182,13 +182,19 @@ Req_Release(struct req *req)
  * TODO:
  * - check for code duplication with cnt_recv_prep
  * - re-check if complete
+ * - XXX should PRIV_TOP use vcl0?
+ * - XXX PRIV_TOP does not get rolled back, should it for !IS_TOPREQ ?
  */
 
 void
 Req_Rollback(struct req *req)
 {
+	if (IS_TOPREQ(req))
+		VCL_TaskLeave(req->vcl, req->top->privs);
 	VCL_TaskLeave(req->vcl, req->privs);
 	VCL_TaskEnter(req->vcl, req->privs);
+	if (IS_TOPREQ(req))
+		VCL_TaskEnter(req->vcl, req->top->privs);
 	HTTP_Clone(req->http, req->http0);
 	if (WS_Overflowed(req->ws))
 		req->wrk->stats->ws_client_overflow++;
diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c
index cc64b31b7..60090c8c0 100644
--- a/bin/varnishd/cache/cache_req_fsm.c
+++ b/bin/varnishd/cache/cache_req_fsm.c
@@ -1104,8 +1104,11 @@ CNT_Request(struct req *req)
 	}
 	wrk->vsl = NULL;
 	if (nxt == REQ_FSM_DONE) {
-		if (IS_TOPREQ(req) && req->top->vcl0 != NULL)
-			VCL_Rel(&req->top->vcl0);
+		if (IS_TOPREQ(req)) {
+			VCL_TaskLeave(req->vcl, req->top->privs);
+			if (req->top->vcl0 != NULL)
+				VCL_Rel(&req->top->vcl0);
+		}
 		VCL_TaskLeave(req->vcl, req->privs);
 		AN(req->vsl->wid);
 		VRB_Free(req);
diff --git a/bin/varnishd/cache/cache_vpi.c b/bin/varnishd/cache/cache_vpi.c
index b9740b267..2a0095f79 100644
--- a/bin/varnishd/cache/cache_vpi.c
+++ b/bin/varnishd/cache/cache_vpi.c
@@ -94,6 +94,11 @@ VPI_vcl_select(VRT_CTX, VCL_VCL vcl)
 	if (IS_TOPREQ(req) && req->top->vcl0 != NULL)
 		return;		// Illegal, req-FSM will fail this later.
 
+	/* XXX VCL_Task* are somewhat duplicated to those in Req_Rollback called
+	 * from FSM for VCL_RET_VCL. Keeping them here to ensure there are no
+	 * tasks during calls to VCL_Rel / vcl_get
+	 */
+	VCL_TaskLeave(req->vcl, req->top->privs);
 	VCL_TaskLeave(req->vcl, req->privs);
 	if (IS_TOPREQ(req)) {
 		AN(req->top);
@@ -107,4 +112,5 @@ VPI_vcl_select(VRT_CTX, VCL_VCL vcl)
 	VSLb(ctx->req->vsl, SLT_VCL_use, "%s via %s",
 	    req->vcl->loaded_name, vcl->loaded_name);
 	VCL_TaskEnter(req->vcl, req->privs);
+	VCL_TaskEnter(req->vcl, req->top->privs);
 }
diff --git a/bin/varnishd/cache/cache_vrt_priv.c b/bin/varnishd/cache/cache_vrt_priv.c
index d8b2988d0..bcfd7d8d4 100644
--- a/bin/varnishd/cache/cache_vrt_priv.c
+++ b/bin/varnishd/cache/cache_vrt_priv.c
@@ -170,16 +170,12 @@ VRT_priv_top(VRT_CTX, const void *vmod_id)
 		WRONG("PRIV_TOP is only accessible in client VCL context");
 		NEEDLESS(return (NULL));
 	}
-	CHECK_OBJ_NOTNULL(ctx->req, REQ_MAGIC);
 	req = ctx->req;
-	if (req->top != NULL) {
-		CHECK_OBJ_NOTNULL(req->top, REQTOP_MAGIC);
-		CHECK_OBJ_NOTNULL(req->top->topreq, REQ_MAGIC);
-		req = req->top->topreq;
-	}
+	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
+	CHECK_OBJ_NOTNULL(req->top, REQTOP_MAGIC);
 	return (vrt_priv_dynamic(
 	    req->ws,
-	    req->privs,
+	    req->top->privs,
 	    (uintptr_t)vmod_id
 	));
 }
diff --git a/bin/varnishd/http1/cache_http1_fsm.c b/bin/varnishd/http1/cache_http1_fsm.c
index 9418ce8c2..9c0998550 100644
--- a/bin/varnishd/http1/cache_http1_fsm.c
+++ b/bin/varnishd/http1/cache_http1_fsm.c
@@ -410,8 +410,10 @@ HTTP1_Session(struct worker *wrk, struct req *req)
 			req->task.priv = req;
 			wrk->stats->client_req++;
 			CNT_Embark(wrk, req);
-			if (req->req_step == R_STP_TRANSPORT)
+			if (req->req_step == R_STP_TRANSPORT) {
 				VCL_TaskEnter(req->vcl, req->privs);
+				VCL_TaskEnter(req->vcl, req->top->privs);
+			}
 			if (CNT_Request(req) == REQ_FSM_DISEMBARK)
 				return;
 			AZ(req->top->vcl0);
diff --git a/bin/varnishd/http2/cache_http2_proto.c b/bin/varnishd/http2/cache_http2_proto.c
index b9017e76f..4d33525c0 100644
--- a/bin/varnishd/http2/cache_http2_proto.c
+++ b/bin/varnishd/http2/cache_http2_proto.c
@@ -523,8 +523,10 @@ h2_do_req(struct worker *wrk, void *priv)
 	CAST_OBJ_NOTNULL(r2, req->transport_priv, H2_REQ_MAGIC);
 	THR_SetRequest(req);
 	CNT_Embark(wrk, req);
-	if (req->req_step == R_STP_TRANSPORT)
+	if (req->req_step == R_STP_TRANSPORT) {
 		VCL_TaskEnter(req->vcl, req->privs);
+		VCL_TaskEnter(req->vcl, req->top->privs);
+	}
 
 	wrk->stats->client_req++;
 	if (CNT_Request(req) != REQ_FSM_DISEMBARK) {
diff --git a/bin/varnishtest/tests/r02219.vtc b/bin/varnishtest/tests/r02219.vtc
index 8b4d1c188..0315a2b4a 100644
--- a/bin/varnishtest/tests/r02219.vtc
+++ b/bin/varnishtest/tests/r02219.vtc
@@ -19,7 +19,7 @@ varnish v1 -arg "-p workspace_client=9k" -proto PROXY -vcl+backend {
 } -start
 
 client c1 {
-	send "PROXY TCP4 127.0.0.1 127.0.0.1 1111 2222\r\nGET /AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA HTTP/1.1\r\n\r\n"
+	send "PROXY TCP4 127.0.0.1 127.0.0.1 1111 2222\r\nGET /AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA HTTP/1.1\r\n\r\n"
 	rxresp
 } -run
 
@@ -58,14 +58,13 @@ client c2 {
 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42
 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42
 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42
-42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42 42
-42 42 42 42 42 42 42 42 42 42 42 42 42 42
+42 42 42 42 42 42 42
 20 48 54 54 50 2f 31 2e 31 0d 0a 0d 0a
 }
 	rxresp
 } -run
 
 client c3 {
-	send "PROXY TCP4 127.0.0.1 127.0.0.1 1111 2222\r\nGET /CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC HTTP/1.1\r\n\r\n"
+	send "PROXY TCP4 127.0.0.1 127.0.0.1 1111 2222\r\nGET /CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC HTTP/1.1\r\n\r\n"
 	rxresp
 } -run

From nils.goroll at uplex.de  Wed Nov 13 08:48:07 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 13 Nov 2019 08:48:07 +0000 (UTC)
Subject: [master] d1cea9bb4 add assertions requested by @Dridi
Message-ID: <20191113084807.1CFD111B416@lists.varnish-cache.org>


commit d1cea9bb4b3429b6953ee2d68721c8bb33b8f4d5
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 13 09:33:03 2019 +0100

    add assertions requested by @Dridi

diff --git a/bin/varnishd/cache/cache_vpi.c b/bin/varnishd/cache/cache_vpi.c
index 2a0095f79..1ffc62a3e 100644
--- a/bin/varnishd/cache/cache_vpi.c
+++ b/bin/varnishd/cache/cache_vpi.c
@@ -90,6 +90,8 @@ VPI_vcl_select(VRT_CTX, VCL_VCL vcl)
 	struct req *req = ctx->req;
 
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
+	CHECK_OBJ_NOTNULL(req, REQ_MAGIC);
+	CHECK_OBJ_NOTNULL(req->top, REQTOP_MAGIC);
 
 	if (IS_TOPREQ(req) && req->top->vcl0 != NULL)
 		return;		// Illegal, req-FSM will fail this later.

From dridi.boukelmoune at gmail.com  Wed Nov 13 09:08:06 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Wed, 13 Nov 2019 09:08:06 +0000 (UTC)
Subject: [master] df243af68 Allow non-literal negative VCL expressions
Message-ID: <20191113090806.4B60524D8@lists.varnish-cache.org>


commit df243af68312df8869cd00d2826817a4deeffebc
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Tue Nov 12 19:13:50 2019 +0100

    Allow non-literal negative VCL expressions

diff --git a/bin/varnishtest/tests/v00019.vtc b/bin/varnishtest/tests/v00019.vtc
index d81455a15..e37751869 100644
--- a/bin/varnishtest/tests/v00019.vtc
+++ b/bin/varnishtest/tests/v00019.vtc
@@ -70,3 +70,10 @@ varnish v1 -errvcl {Symbol not found: 'req.http.req.http.foo'} {
 		set req.http.req.http.foo = "bar";
 	}
 }
+
+varnish v1 -errvcl {Unknown token '--' when looking for INT} {
+	backend be none;
+	sub vcl_synth {
+		set resp.status = --200;
+	}
+}
diff --git a/bin/varnishtest/tests/v00020.vtc b/bin/varnishtest/tests/v00020.vtc
index 919624a41..238927c34 100644
--- a/bin/varnishtest/tests/v00020.vtc
+++ b/bin/varnishtest/tests/v00020.vtc
@@ -327,6 +327,7 @@ varnish v1 -vcl+backend {
 }
 
 varnish v1 -vcl+backend {
+	import std;
 	import debug;
 	sub vcl_deliver {
 		// Ticket 2745
@@ -339,6 +340,9 @@ varnish v1 -vcl+backend {
 	sub vcl_deliver {
 		set resp.http.p = (0 + 9223372036854775807);
 		set resp.http.n = (0 - 9223372036854775807);
+		if (resp.status == -(-200)) {
+			set resp.http.o = -std.integer("-200");
+		}
 	}
 } -start
 
@@ -350,4 +354,32 @@ client c1 {
 	expect resp.http.bar == "false"
 	expect resp.http.p == 9223372036854775807
 	expect resp.http.n == -9223372036854775807
+	expect resp.http.o == 200
+} -run
+
+varnish v1 -vcl {
+	import std;
+	backend be none;
+	sub vcl_recv {
+		return (synth(200));
+	}
+	sub vcl_synth {
+		if (req.url ~ "double-minus") {
+			set resp.status = -(-204);
+		}
+		if (req.url ~ "minus-std") {
+			set resp.status = -std.integer("-204");
+		}
+		return (deliver);
+	}
+}
+
+client c1 {
+	txreq -url "/double-minus"
+	rxresp
+	expect resp.status == 204
+
+	txreq -url "/minus-std"
+	rxresp
+	expect resp.status == 204
 } -run
diff --git a/lib/libvcc/vcc_expr.c b/lib/libvcc/vcc_expr.c
index d7c32a8dc..bab9251b5 100644
--- a/lib/libvcc/vcc_expr.c
+++ b/lib/libvcc/vcc_expr.c
@@ -764,8 +764,12 @@ vcc_expr5(struct vcc *tl, struct expr **e, vcc_type_t fmt)
 		    fmt != STRINGS)
 			break;
 		vcc_NextToken(tl);
-		if (tl->t->tok != FNUM && tl->t->tok != CNUM)
-			break;
+		if (tl->t->tok != FNUM && tl->t->tok != CNUM) {
+			vcc_expr_cor(tl, &e1, fmt);
+			ERRCHK(tl);
+			*e = vcc_expr_edit(tl, e1->fmt, "-(\v1)", e1, NULL);
+			return;
+		}
 		sign = "-";
 		/* FALLTHROUGH */
 	case FNUM:

From nils.goroll at uplex.de  Wed Nov 13 11:49:06 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 13 Nov 2019 11:49:06 +0000 (UTC)
Subject: [master] 4a147fa7c Add Ctrl-f and Ctrl-b for pagedown/up in
 varnishstat.
Message-ID: <20191113114906.3F9696E72@lists.varnish-cache.org>


commit 4a147fa7c91e0d3c526a6f15c92eb0c8b26a06e6
Author: Lasse Karstensen <lasse.karstensen at gmail.com>
Date:   Tue Nov 12 09:41:42 2019 +0100

    Add Ctrl-f and Ctrl-b for pagedown/up in varnishstat.
    
    This adds another vim-style key binding to varnishstat.
    
    Couldn't find any good way to extend u00008.vtc to test it. The
    cursor is always at column 22 and page-downing to the bottom
    will break the test case when we add another counter.

diff --git a/bin/varnishstat/varnishstat_curses.c b/bin/varnishstat/varnishstat_curses.c
index a97fe0c62..8d3d1c5ef 100644
--- a/bin/varnishstat/varnishstat_curses.c
+++ b/bin/varnishstat/varnishstat_curses.c
@@ -62,6 +62,8 @@
 
 #define VALUE_MAX		999999999999
 
+#define CTRL(c) ((c) & 037)
+
 struct ma {
 	unsigned n, nmax;
 	double acc;
@@ -881,11 +883,13 @@ handle_keypress(int ch)
 		current++;
 		break;
 	case KEY_PPAGE:
+	case CTRL('b'):
 	case 'b':
 		current -= l_points;
 		page_start -= l_points;
 		break;
 	case KEY_NPAGE:
+	case CTRL('f'):
 	case ' ':
 		current += l_points;
 		if (page_start + l_points < n_ptarray - 1)
diff --git a/doc/sphinx/reference/varnishstat.rst b/doc/sphinx/reference/varnishstat.rst
index a6f81c8b8..38684472a 100644
--- a/doc/sphinx/reference/varnishstat.rst
+++ b/doc/sphinx/reference/varnishstat.rst
@@ -79,10 +79,10 @@ The following keys control the interactive display:
 <DOWN>
 	Navigate the counter list one line down.
 
-<PAGEUP> or <b>
+<PAGEUP> or <b> or <CTRL-b>
 	Navigate the counter list one page up.
 
-<PAGEDOWN> or <SPACE>
+<PAGEDOWN> or <SPACE> or <CTRL-f>
 	Navigate the counter list one page down.
 
 <HOME>

From nils.goroll at uplex.de  Wed Nov 13 11:53:06 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 13 Nov 2019 11:53:06 +0000 (UTC)
Subject: [master] d25230ac1 use the CTRL() macro
Message-ID: <20191113115306.150687204@lists.varnish-cache.org>


commit d25230ac1d746d5ec3f9e58f48c6c54d09abf005
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 13 12:51:28 2019 +0100

    use the CTRL() macro
    
    added by @lkarsten as suggested by @Driri in #3122

diff --git a/bin/varnishstat/varnishstat_curses.c b/bin/varnishstat/varnishstat_curses.c
index 8d3d1c5ef..0b5338225 100644
--- a/bin/varnishstat/varnishstat_curses.c
+++ b/bin/varnishstat/varnishstat_curses.c
@@ -944,13 +944,13 @@ handle_keypress(int ch)
 	case 'q':
 		keep_running = 0;
 		return;
-	case '\003':		/* Ctrl-C */
+	case CTRL('c'):
 		AZ(raise(SIGINT));
 		return;
-	case '\024':		/* Ctrl-T */
+	case CTRL('t'):
 		sample = 1;
 		return;
-	case '\032':		/* Ctrl-Z */
+	case CTRL('z'):
 		AZ(raise(SIGTSTP));
 		return;
 	default:

From nils.goroll at uplex.de  Wed Nov 13 15:53:05 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Wed, 13 Nov 2019 15:53:05 +0000 (UTC)
Subject: [master] 0d6361433 make more obvious what happens during director
 destruction
Message-ID: <20191113155305.B5C2263ED3@lists.varnish-cache.org>


commit 0d636143386645a34547f6faa52bb6177d66d4b1
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 13 16:29:38 2019 +0100

    make more obvious what happens during director destruction

diff --git a/bin/varnishd/cache/cache_vrt_vcl.c b/bin/varnishd/cache/cache_vrt_vcl.c
index 66f0ce9e5..a660c23e2 100644
--- a/bin/varnishd/cache/cache_vrt_vcl.c
+++ b/bin/varnishd/cache/cache_vrt_vcl.c
@@ -225,6 +225,7 @@ VRT_DelDirector(VCL_BACKEND *bp)
 	AZ(errno=pthread_rwlock_unlock(&vcl->temp_rwl));
 	if(vdir->methods->destroy != NULL)
 		vdir->methods->destroy(d);
+	assert (d == vdir->dir);
 	free(vdir->cli_name);
 	FREE_OBJ(vdir->dir);
 	FREE_OBJ(vdir);

From dridi.boukelmoune at gmail.com  Wed Nov 13 16:16:06 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Wed, 13 Nov 2019 16:16:06 +0000 (UTC)
Subject: [master] da8ea9164 Teach a rudimentary alignment rule to
 vtc.typesize()
Message-ID: <20191113161606.DD5FD64866@lists.varnish-cache.org>


commit da8ea91648d9a3ef65379a49ad9c8d449c8c52d8
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Wed Nov 13 16:56:06 2019 +0100

    Teach a rudimentary alignment rule to vtc.typesize()

diff --git a/lib/libvmod_vtc/vmod_vtc.c b/lib/libvmod_vtc/vmod_vtc.c
index d349e565f..9b382a6c0 100644
--- a/lib/libvmod_vtc/vmod_vtc.c
+++ b/lib/libvmod_vtc/vmod_vtc.c
@@ -267,14 +267,14 @@ vmod_workspace_dump(VRT_CTX, VCL_ENUM which, VCL_ENUM where,
 VCL_INT v_matchproto_(td_vtc_typesize)
 vmod_typesize(VRT_CTX, VCL_STRING s)
 {
-	size_t i = 0;
+	size_t i = 0, l, a;
 	const char *p;
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 
 	for (p = s; *p; p++) {
 		switch (*p) {
-#define VTC_TYPESIZE(c, t) case c: i += sizeof(t); break;
+#define VTC_TYPESIZE(c, t) case c: l = sizeof(t); break;
 		VTC_TYPESIZE('d', double)
 		VTC_TYPESIZE('f', float)
 		VTC_TYPESIZE('i', int)
@@ -283,10 +283,15 @@ vmod_typesize(VRT_CTX, VCL_STRING s)
 		VTC_TYPESIZE('o', off_t)
 		VTC_TYPESIZE('p', void *)
 		VTC_TYPESIZE('s', short)
+		VTC_TYPESIZE('u', unsigned)
 		VTC_TYPESIZE('z', size_t)
 #undef VTC_TYPESIZE
 		default:	return (-1);
 		}
+		a = i % l;
+		if (a != 0)
+			i += (l - a);
+		i += l;
 	}
 	return ((VCL_INT)i);
 }

From dridi.boukelmoune at gmail.com  Wed Nov 13 21:14:09 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Wed, 13 Nov 2019 21:14:09 +0000 (UTC)
Subject: [master] a3df17bda Whitespace OCD
Message-ID: <20191113211409.32979948FC@lists.varnish-cache.org>


commit a3df17bda463f412243af317efb84213da542fd0
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Wed Nov 13 19:04:05 2019 +0100

    Whitespace OCD

diff --git a/bin/varnishtest/vtc_client.c b/bin/varnishtest/vtc_client.c
index 9777da7ba..2722488a1 100644
--- a/bin/varnishtest/vtc_client.c
+++ b/bin/varnishtest/vtc_client.c
@@ -67,8 +67,7 @@ struct client {
 	pthread_t		tp;
 };
 
-static VTAILQ_HEAD(, client)	clients =
-    VTAILQ_HEAD_INITIALIZER(clients);
+static VTAILQ_HEAD(, client)	clients = VTAILQ_HEAD_INITIALIZER(clients);
 
 /**********************************************************************
  * Send the proxy header
diff --git a/bin/varnishtest/vtc_http.h b/bin/varnishtest/vtc_http.h
index 0091eaa1a..0e6337839 100644
--- a/bin/varnishtest/vtc_http.h
+++ b/bin/varnishtest/vtc_http.h
@@ -74,5 +74,3 @@ struct http {
 	uint64_t		iws;
 	int64_t			ws;
 };
-
-
diff --git a/bin/varnishtest/vtc_logexp.c b/bin/varnishtest/vtc_logexp.c
index 7369a8642..ea14e3e68 100644
--- a/bin/varnishtest/vtc_logexp.c
+++ b/bin/varnishtest/vtc_logexp.c
@@ -68,7 +68,7 @@
  *         Filter records using a query expression, see ``man vsl-query`` for
  *         more information.
  * \-m
- *	   Also emit log records for misses (only for debugging)
+ *         Also emit log records for misses (only for debugging)
  *
  * \-start
  *         Start the logexpect thread in the background.
diff --git a/bin/varnishtest/vtc_server.c b/bin/varnishtest/vtc_server.c
index 3ff49705e..9a1a0d68e 100644
--- a/bin/varnishtest/vtc_server.c
+++ b/bin/varnishtest/vtc_server.c
@@ -67,8 +67,7 @@ struct server {
 
 static pthread_mutex_t		server_mtx;
 
-static VTAILQ_HEAD(, server)	servers =
-    VTAILQ_HEAD_INITIALIZER(servers);
+static VTAILQ_HEAD(, server)	servers = VTAILQ_HEAD_INITIALIZER(servers);
 
 /**********************************************************************
  * Allocate and initialize a server

From dridi.boukelmoune at gmail.com  Wed Nov 13 21:14:09 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Wed, 13 Nov 2019 21:14:09 +0000 (UTC)
Subject: [master] 7a28f076e Move the VTC client connection to its own function
Message-ID: <20191113211409.1BC9C948F9@lists.varnish-cache.org>


commit 7a28f076ebb6845459a5ef108eb4547d0618b1a0
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Wed Nov 13 19:03:36 2019 +0100

    Move the VTC client connection to its own function

diff --git a/bin/varnishtest/vtc_client.c b/bin/varnishtest/vtc_client.c
index 29915a0b3..9777da7ba 100644
--- a/bin/varnishtest/vtc_client.c
+++ b/bin/varnishtest/vtc_client.c
@@ -54,6 +54,7 @@ struct client {
 	char			*spec;
 
 	char			connect[256];
+	const char		*addr;
 	int			rcvbuf;
 
 	char			*proxy_spec;
@@ -180,6 +181,27 @@ client_uds_connect(struct vtclog *vl, const char *path, double tmo,
 	return (fd);
 }
 
+static int
+client_connect(struct vtclog *vl, struct client *c)
+{
+	const char *err;
+	int fd;
+
+	vtc_log(vl, 3, "Connect to %s", c->addr);
+	if (*c->addr == '/')
+		fd = client_uds_connect(vl, c->addr, 10., &err);
+	else
+		fd = client_tcp_connect(vl, c->addr, 10., &err);
+	if (fd < 0)
+		vtc_fatal(c->vl, "Failed to open %s: %s",
+		    c->addr, err);
+	/* VTCP_blocking does its own checks, trust it */
+	VTCP_blocking(fd);
+	if (c->proxy_spec != NULL)
+		client_proxy(vl, fd, c->proxy_version, c->proxy_spec);
+	return (fd);
+}
+
 /**********************************************************************
  * Client thread
  */
@@ -192,7 +214,6 @@ client_thread(void *priv)
 	int fd;
 	int i;
 	struct vsb *vsb;
-	const char *err;
 
 	CAST_OBJ_NOTNULL(c, priv, CLIENT_MAGIC);
 	AN(*c->connect);
@@ -202,6 +223,7 @@ client_thread(void *priv)
 
 	vsb = macro_expand(vl, c->connect);
 	AN(vsb);
+	c->addr = VSB_data(vsb);
 
 	if (c->repeat == 0)
 		c->repeat = 1;
@@ -209,27 +231,15 @@ client_thread(void *priv)
 		vtc_log(vl, 2, "Started (%u iterations%s)", c->repeat,
 			c->keepalive ? " using keepalive" : "");
 	for (i = 0; i < c->repeat; i++) {
-		char *addr = VSB_data(vsb);
+		fd = client_connect(vl, c);
 
-		vtc_log(vl, 3, "Connect to %s", addr);
-		if (*addr == '/')
-			fd = client_uds_connect(vl, addr, 10., &err);
-		else
-			fd = client_tcp_connect(vl, VSB_data(vsb), 10., &err);
-		if (fd < 0)
-			vtc_fatal(c->vl, "Failed to open %s: %s",
-			    VSB_data(vsb), err);
-		/* VTCP_blocking does its own checks, trust it */
-		VTCP_blocking(fd);
-		if (c->proxy_spec != NULL)
-			client_proxy(vl, fd, c->proxy_version, c->proxy_spec);
 		if (! c->keepalive)
-			fd = http_process(vl, c->spec, fd, NULL, addr,
+			fd = http_process(vl, c->spec, fd, NULL, c->addr,
 			    c->rcvbuf);
 		else
 			while (fd >= 0 && i++ < c->repeat)
-				fd = http_process(vl, c->spec, fd, NULL, addr,
-				    c->rcvbuf);
+				fd = http_process(vl, c->spec, fd, NULL,
+				    c->addr, c->rcvbuf);
 		vtc_log(vl, 3, "closing fd %d", fd);
 		VTCP_close(&fd);
 	}

From dridi.boukelmoune at gmail.com  Wed Nov 13 21:14:09 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Wed, 13 Nov 2019 21:14:09 +0000 (UTC)
Subject: [master] 4495a0e0b Sync documentation with reality
Message-ID: <20191113211409.519D594900@lists.varnish-cache.org>


commit 4495a0e0b78aca83f93acc4552abb922e01c2f9a
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Wed Nov 13 20:15:43 2019 +0100

    Sync documentation with reality

diff --git a/bin/varnishtest/vtc_logexp.c b/bin/varnishtest/vtc_logexp.c
index ea14e3e68..31a9cb450 100644
--- a/bin/varnishtest/vtc_logexp.c
+++ b/bin/varnishtest/vtc_logexp.c
@@ -66,7 +66,8 @@
  *
  * \-q query
  *         Filter records using a query expression, see ``man vsl-query`` for
- *         more information.
+ *         more information. Multiple -q options are not supported.
+ *
  * \-m
  *         Also emit log records for misses (only for debugging)
  *
@@ -78,9 +79,6 @@
  *
  * VSL arguments (similar to the varnishlog options):
  *
- * \-b|-c
- *         Process only backend/client records.
- *
  * \-C
  *         Use caseless regex
  *

From dridi.boukelmoune at gmail.com  Wed Nov 13 21:14:09 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Wed, 13 Nov 2019 21:14:09 +0000 (UTC)
Subject: [master] 27bed0edc Teach struct padding to vtc.typesize()
Message-ID: <20191113211409.6BBFC94906@lists.varnish-cache.org>


commit 27bed0edcb190aca32ea50b47b2c08305c1b282c
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Wed Nov 13 21:58:04 2019 +0100

    Teach struct padding to vtc.typesize()
    
    And start adding some coverage, certainly not exhaustive.

diff --git a/bin/varnishtest/tests/m00000.vtc b/bin/varnishtest/tests/m00000.vtc
index a62ff527b..a60147dde 100644
--- a/bin/varnishtest/tests/m00000.vtc
+++ b/bin/varnishtest/tests/m00000.vtc
@@ -143,3 +143,124 @@ varnish v1 -errvcl {Failed initialization} {
 		new shp = directors.shard_param();
 	}
 }
+
+varnish v1 -cliok "param.set vcc_allow_inline_c on"
+varnish v1 -vcl {
+	import std;
+	import vtc;
+
+	backend be none;
+
+	C{
+		struct empty { };
+		struct zc {
+			size_t	z;
+			char	c;
+		};
+		struct ic {
+			int	i;
+			char	c;
+		};
+		struct sc {
+			short	s;
+			char	c;
+		};
+		struct cc {
+			char	c1;
+			char	c2;
+		};
+		struct czc {
+			char	c1;
+			size_t	z;
+			char	c2;
+		};
+		struct cic {
+			char	c1;
+			int	i;
+			char	c2;
+		};
+		struct csc {
+			char	c1;
+			short	s;
+			char	c2;
+		};
+		struct uzp { /* same as vrt_blob */
+			unsigned	u;
+			size_t		z;
+			void		*p;
+		};
+
+		#define SETHDR(type)					\
+			VRT_SetHdr(ctx,					\
+			    &VGC_HDR_RESP_ ## type ## _2d_sizeof,	\
+			    VRT_INT_string(ctx, sizeof(struct type)),	\
+			    vrt_magic_string_end)
+	}C
+
+	sub vcl_recv {
+		return (synth(200));
+	}
+
+	sub vcl_synth {
+		C{ SETHDR(empty); }C
+		set resp.http.empty-typesize = vtc.typesize("");
+		set resp.http.empty-match =
+		    (resp.http.empty-sizeof == resp.http.empty-typesize);
+
+		C{ SETHDR(zc); }C
+		set resp.http.zc-typesize = vtc.typesize("zc");
+		set resp.http.zc-match =
+		    (resp.http.zc-sizeof == resp.http.zc-typesize);
+
+		C{ SETHDR(ic); }C
+		set resp.http.ic-typesize = vtc.typesize("ic");
+		set resp.http.ic-match =
+		    (resp.http.ic-sizeof == resp.http.ic-typesize);
+
+		C{ SETHDR(sc); }C
+		set resp.http.sc-typesize = vtc.typesize("sc");
+		set resp.http.sc-match =
+		    (resp.http.sc-sizeof == resp.http.sc-typesize);
+
+		C{ SETHDR(cc); }C
+		set resp.http.cc-typesize = vtc.typesize("cc");
+		set resp.http.cc-match =
+		    (resp.http.cc-sizeof == resp.http.cc-typesize);
+
+		C{ SETHDR(czc); }C
+		set resp.http.czc-typesize = vtc.typesize("czc");
+		set resp.http.czc-match =
+		    (resp.http.czc-sizeof == resp.http.czc-typesize);
+
+		C{ SETHDR(cic); }C
+		set resp.http.cic-typesize = vtc.typesize("cic");
+		set resp.http.cic-match =
+		    (resp.http.cic-sizeof == resp.http.cic-typesize);
+
+		C{ SETHDR(csc); }C
+		set resp.http.csc-typesize = vtc.typesize("csc");
+		set resp.http.csc-match =
+		    (resp.http.csc-sizeof == resp.http.csc-typesize);
+
+		C{ SETHDR(uzp); }C
+		set resp.http.uzp-typesize = vtc.typesize("uzp");
+		set resp.http.uzp-match =
+		    (resp.http.uzp-sizeof == resp.http.uzp-typesize);
+	}
+}
+
+client c1 {
+	txreq
+	rxresp
+	expect resp.http.empty-match == true
+
+	expect resp.http.zc-match == true
+	expect resp.http.ic-match == true
+	expect resp.http.sc-match == true
+	expect resp.http.cc-match == true
+
+	expect resp.http.czc-match == true
+	expect resp.http.cic-match == true
+	expect resp.http.csc-match == true
+	expect resp.http.uzp-match == true
+} -run
diff --git a/lib/libvmod_vtc/vmod_vtc.c b/lib/libvmod_vtc/vmod_vtc.c
index 9b382a6c0..51a279d27 100644
--- a/lib/libvmod_vtc/vmod_vtc.c
+++ b/lib/libvmod_vtc/vmod_vtc.c
@@ -267,14 +267,15 @@ vmod_workspace_dump(VRT_CTX, VCL_ENUM which, VCL_ENUM where,
 VCL_INT v_matchproto_(td_vtc_typesize)
 vmod_typesize(VRT_CTX, VCL_STRING s)
 {
-	size_t i = 0, l, a;
-	const char *p;
+	size_t i = 0, l, a, p = 0;
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+	AN(s);
 
-	for (p = s; *p; p++) {
-		switch (*p) {
+	for (; *s; s++) {
+		switch (*s) {
 #define VTC_TYPESIZE(c, t) case c: l = sizeof(t); break;
+		VTC_TYPESIZE('c', char)
 		VTC_TYPESIZE('d', double)
 		VTC_TYPESIZE('f', float)
 		VTC_TYPESIZE('i', int)
@@ -288,10 +289,17 @@ vmod_typesize(VRT_CTX, VCL_STRING s)
 #undef VTC_TYPESIZE
 		default:	return (-1);
 		}
+		if (l > p)
+			p = l;
 		a = i % l;
 		if (a != 0)
-			i += (l - a);
+			i += (l - a); /* align */
 		i += l;
 	}
+	if (i > 0) {
+		a = i % p;
+		if (a != 0)
+			i += (p - a); /* pad */
+	}
 	return ((VCL_INT)i);
 }

From dridi.boukelmoune at gmail.com  Thu Nov 14 07:15:06 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Thu, 14 Nov 2019 07:15:06 +0000 (UTC)
Subject: [master] 175af0b91 Big bang rewrite of m00049.vtc
Message-ID: <20191114071506.AABDAA5B43@lists.varnish-cache.org>


commit 175af0b91474c539d78ae49e5496a3c8936f1fb0
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Thu Nov 14 08:08:01 2019 +0100

    Big bang rewrite of m00049.vtc
    
    Now that vtc.typesize() is a bit more reliable, it's easier to deal with
    architecture differences for the sizeof struct vrt_blob. For a much more
    reliable failure check, it now looks at logs.
    
    There may also have been a time when depending on the error we trigger
    we could fail with a 500 or a 503, this is no longer the case.
    
    For the nitty-gritty of the new test case, there are some comments to
    hopefully help (at least my future self) decipher this beast.
    
    This new test case remains stable even with the changes from #3123.

diff --git a/bin/varnishtest/tests/m00049.vtc b/bin/varnishtest/tests/m00049.vtc
index b92968dea..01ad05b66 100644
--- a/bin/varnishtest/tests/m00049.vtc
+++ b/bin/varnishtest/tests/m00049.vtc
@@ -1,186 +1,329 @@
 varnishtest "VMOD blob workspace overflow conditions"
 
-varnish v1 -vcl {
+# This test case is tricky to get right and is somewhat repetitive. The VCL
+# below provides off-the-shelves subs to abstract away some of the complexity.
+# Since not all of them may be called by all VCLs we also need to ensure this
+# will not result in a compilation failure.
+
+varnish v1 -cliok "param.set vcc_err_unref off"
+
+shell {
+	cat >vrt_blob.vcl <<-EOF
 	import blob;
+	import std;
 	import vtc;
-	backend b None;
+
+	backend be none;
 
 	sub vcl_recv {
-		if (req.url == "/1") {
-			# Too small for the vmod_priv object.
-			vtc.workspace_alloc(client, -8);
-		}
-		elsif (req.url == "/2") {
-			# Likely large enough for the vmod_priv object,
-			# but not enough left over for the decode.
-			vtc.workspace_alloc(client, -33);
-		}
-		elsif (req.url == "/3") {
-			# Enough for the decode, but not enough left
-			# over for the encode.
-			vtc.workspace_alloc(client, -50);
-		}
-		set req.http.Decode
-			= blob.encode(blob=blob.decode(encoded="1234567890"));
+		set req.http.vrt_blob = vtc.typesize("uzp");
 	}
 
-	sub vcl_miss {
-		if (req.url == "/4") {
-			# Enough for the req.hash BLOB (vmod_priv + 32
-			# bytes), but not enough left over for the
-			# encoding.
-			vtc.workspace_alloc(client, -65);
-			set req.http.Encode = blob.encode(blob=req.hash);
-		}
-		return( synth(200) );
+	sub shrink {
+		std.log("shrink");
+		vtc.workspace_alloc(client, -std.integer(req.http.leave));
 	}
-} -start
 
-client c1 {
-	txreq -url "/1"
+	sub leave_struct {
+		set req.http.leave = std.integer(req.http.vrt_blob);
+		call shrink;
+	}
+
+	sub leave_half_struct {
+		set req.http.leave = std.integer(req.http.vrt_blob) / 2;
+		call shrink;
+	}
+
+	sub leave_blob {
+		set req.http.leave = std.integer(req.http.vrt_blob) +
+		    std.integer(req.http.blob);
+		call shrink;
+	}
+	EOF
+}
+
+# For each type of BLOB codec we may use some of the prepared clients. It is
+# important not to run them concurently to ensure we can predictably peek at
+# logs.
+
+client c-struct {
+	txreq -url "/struct"
 	rxresp
-	expect resp.status >= 500
-	expect resp.status <= 503
-	expect resp.reason ~ {(?i)^VCL Failed$|^Internal Server Error$}
-} -run
+	expect resp.status == 503
+	expect resp.reason == "VCL failed"
+	expect_close
+}
 
-client c2 {
-	txreq -url "/2"
+client c-encode {
+	txreq -url "/encode"
 	rxresp
-	expect resp.status >= 500
-	expect resp.status <= 503
-	expect resp.reason ~ {(?i)^VCL Failed$|^Internal Server Error$}
-} -run
+	expect resp.status == 503
+	expect resp.reason == "VCL failed"
+	expect_close
+}
 
-client c3 {
-	txreq -url "/3"
+client c-decode {
+	txreq -url "/decode"
 	rxresp
-	expect resp.status >= 500
-	expect resp.status <= 503
-	expect resp.reason ~ {(?i)^VCL Failed$|^Internal Server Error$}
-} -run
+	expect resp.status == 503
+	expect resp.reason == "VCL failed"
+	expect_close
+}
 
-client c4 {
-	txreq -url "/4"
+client c-req-hash {
+	txreq -url "/req.hash"
 	rxresp
-	expect resp.status >= 500
-	expect resp.status <= 503
-	expect resp.reason ~ {(?i)^VCL Failed$|^Internal Server Error$}
-} -run
+	expect resp.status == 503
+	expect resp.reason == "VCL failed"
+	expect_close
+}
+
+client c-sub {
+	txreq -url "/sub"
+	rxresp
+	expect resp.status == 503
+	expect resp.reason == "VCL failed"
+	expect_close
+}
+
+# For each type of BLOB codec we will check failure conditions from the logs
+# so we provision a log reader that will only deal with meaningful log records
+# for this job. This means that all logexpect expect commands must use 0 as
+# their first argument. Meaning we make a comprehensive check of all the log
+# records that aren't filtered out.
+
+logexpect l1 -v v1 -i ReqURL,VCL_Error,VCL_Log,VCL_use
+
+# IDENTITY codec
 
-# Similar tests with BASE64 encoding
 varnish v1 -vcl {
-	import blob;
-	import vtc;
-	backend b None;
+	include "${tmpdir}/vrt_blob.vcl";
 
 	sub vcl_recv {
-		if (req.url == "/1") {
-			vtc.workspace_alloc(client, -33);
+		if (req.url ~ "decode") {
+			# Not enough space to collect the string.
+			set req.http.leave = 5;
+			call shrink;
+		}
+		if (req.url ~ "struct") {
+			# Enough space to collect the decoded string.
+			# Not enough space to allocate a blob (aligned).
+			set req.http.leave = 16;
+			call shrink;
 		}
-		elsif (req.url == "/2") {
-			vtc.workspace_alloc(client, -50);
+		if (req.url ~ "encode") {
+			# Enough space to decode the string.
+			# Not enough space to encode the blob.
+			set req.http.blob = 16;
+			call leave_blob;
 		}
-		set req.http.Decode
-			= blob.encode(blob=blob.decode(BASE64,
-						encoded="MTIzNDU2Nzg5MA=="));
+		blob.encode(blob=blob.decode(encoded="1234567890"));
+		return (synth(200));
 	}
+} -start
 
-	sub vcl_miss {
-		if (req.url == "/3") {
-			vtc.workspace_alloc(client, -65);
-			set req.http.Encode
-				= blob.encode(BASE64, blob=req.hash);
+logexpect l1 {
+	expect 0 * VCL_use	vcl1
+	expect 0 = ReqURL	decode
+	expect 0 = VCL_Log	shrink
+	expect 0 = VCL_Error	"cannot decode, out of space"
+	expect 0 * VCL_use	vcl1
+	expect 0 = ReqURL	struct
+	expect 0 = VCL_Log	shrink
+	expect 0 = VCL_Error	"Workspace overflow .blob.decode."
+	expect 0 * VCL_use	vcl1
+	expect 0 = ReqURL	encode
+	expect 0 = VCL_Log	shrink
+	expect 0 = VCL_Error	"cannot encode, out of space"
+} -start
+
+client c-decode -run
+client c-struct -run
+client c-encode -run
+
+logexpect l1 -wait
+
+# BASE64 codec
+
+varnish v1 -vcl {
+	include "${tmpdir}/vrt_blob.vcl";
+
+	sub vcl_recv {
+		if (req.url ~ "decode") {
+			# Not enough space to collect the string.
+			set req.http.leave = 5;
+			call shrink;
+		}
+		if (req.url ~ "struct") {
+			# Enough space to collect the decoded string.
+			# Not enough space to allocate a blob.
+			set req.http.leave = 16;
+			call shrink;
 		}
-		return( synth(200) );
+		if (req.url ~ "encode") {
+			# Enough space to decode the string.
+			# Not enough space to encode the blob.
+			set req.http.blob = 16;
+			call leave_blob;
+		}
+		blob.encode(
+		    blob=blob.decode(BASE64, encoded="MTIzNDU2Nzg5MA=="));
+		return (synth(200));
 	}
 }
 
-client c1 -run
-client c2 -run
-client c3 -run
+logexpect l1 {
+	expect 0 * VCL_use	vcl2
+	expect 0 = ReqURL	decode
+	expect 0 = VCL_Log	shrink
+	expect 0 = VCL_Error	"cannot decode, out of space"
+	expect 0 * VCL_use	vcl2
+	expect 0 = ReqURL	struct
+	expect 0 = VCL_Log	shrink
+	expect 0 = VCL_Error	"Workspace overflow .blob.decode."
+	expect 0 * VCL_use	vcl2
+	expect 0 = ReqURL	encode
+	expect 0 = VCL_Log	shrink
+	expect 0 = VCL_Error	"cannot encode, out of space"
+} -start
+
+client c-decode -run
+client c-struct -run
+client c-encode -run
+
+logexpect l1 -wait
+
+# URL codec
 
-# URL encoding
 varnish v1 -vcl {
-	import blob;
-	import vtc;
-	backend b None;
+	include "${tmpdir}/vrt_blob.vcl";
 
 	sub vcl_recv {
-		if (req.url == "/1") {
-			vtc.workspace_alloc(client, -33);
+		if (req.url ~ "decode") {
+			# Not enough space to collect the string.
+			set req.http.leave = 5;
+			call shrink;
 		}
-		elsif (req.url == "/2") {
-			vtc.workspace_alloc(client, -50);
+		if (req.url ~ "struct") {
+			# Enough space to collect the decoded string.
+			# Not enough space to allocate a blob (aligned).
+			set req.http.leave = 16;
+			call shrink;
 		}
-		set req.http.Decode
-			= blob.encode(blob=blob.decode(URL,
-						encoded="1234567890"));
-	}
-
-	sub vcl_miss {
-		if (req.url == "/3") {
-			vtc.workspace_alloc(client, -65);
-			set req.http.Encode = blob.encode(URL, blob=req.hash);
+		if (req.url ~ "encode") {
+			# Enough space to decode the string.
+			# Not enough space to encode the blob.
+			set req.http.blob = 16;
+			call leave_blob;
 		}
-		return( synth(200) );
+		blob.encode(blob=blob.decode(URL, encoded="1234567890"));
+		return (synth(200));
 	}
 }
 
-client c1 -run
-client c2 -run
-client c3 -run
+logexpect l1 {
+	expect 0 * VCL_use	vcl3
+	expect 0 = ReqURL	decode
+	expect 0 = VCL_Log	shrink
+	expect 0 = VCL_Error	"cannot decode, out of space"
+	expect 0 * VCL_use	vcl3
+	expect 0 = ReqURL	struct
+	expect 0 = VCL_Log	shrink
+	expect 0 = VCL_Error	"Workspace overflow .blob.decode."
+	expect 0 * VCL_use	vcl3
+	expect 0 = ReqURL	encode
+	expect 0 = VCL_Log	shrink
+	expect 0 = VCL_Error	"cannot encode, out of space"
+} -start
+
+client c-decode -run
+client c-struct -run
+client c-encode -run
+
+logexpect l1 -wait
+
+# HEX codec
 
-# HEX encoding
 varnish v1 -vcl {
-	import blob;
-	import vtc;
-	backend b None;
+	include "${tmpdir}/vrt_blob.vcl";
 
 	sub vcl_recv {
-		if (req.url == "/1") {
-			vtc.workspace_alloc(client, -33);
+		if (req.url ~ "decode") {
+			# Not enough space to collect the string.
+			set req.http.leave = 5;
+			call shrink;
 		}
-		elsif (req.url == "/2") {
-			vtc.workspace_alloc(client, -50);
+		if (req.url ~ "struct") {
+			# Enough space to collect the decoded string.
+			# Not enough space to allocate a blob (aligned).
+			set req.http.leave = 20;
+			call shrink;
 		}
-		set req.http.Decode
-			= blob.encode(blob=blob.decode(HEX,
-					encoded="31323334353637383930"));
-	}
-
-	sub vcl_miss {
-		if (req.url == "/3") {
-			vtc.workspace_alloc(client, -65);
-			set req.http.Encode = blob.encode(HEX, blob=req.hash);
+		if (req.url ~ "encode") {
+			# Enough space to decode the string.
+			# Not enough space to encode the blob.
+			set req.http.blob = 20;
+			call leave_blob;
 		}
-		return( synth(200) );
+		blob.encode(
+		    blob=blob.decode(HEX, encoded="31323334353637383930"));
+		return (synth(200));
 	}
 }
 
-client c1 -run
-client c2 -run
-client c3 -run
+logexpect l1 {
+	expect 0 * VCL_use	vcl4
+	expect 0 = ReqURL	decode
+	expect 0 = VCL_Log	shrink
+	expect 0 = VCL_Error	"cannot decode, out of space"
+	expect 0 * VCL_use	vcl4
+	expect 0 = ReqURL	struct
+	expect 0 = VCL_Log	shrink
+	expect 0 = VCL_Error	"Workspace overflow .blob.decode."
+	expect 0 * VCL_use	vcl4
+	expect 0 = ReqURL	encode
+	expect 0 = VCL_Log	shrink
+	expect 0 = VCL_Error	"cannot encode, out of space"
+} -start
+
+client c-decode -run
+client c-struct -run
+client c-encode -run
+
+logexpect l1 -wait
+
+# blob.sub() function
 
-# sub() function
 varnish v1 -vcl {
-	import blob;
-	import vtc;
-	backend b None;
+	include "${tmpdir}/vrt_blob.vcl";
 
 	sub vcl_miss {
-		if (req.url == "/1") {
-			# Not enough for req.hash + vmod_priv
-			vtc.workspace_alloc(client, -65);
+		if (req.url ~ "req.hash") {
+			# Not enough to create the req.hash blob.
+			call leave_half_struct;
 		}
-		elsif (req.url == "/2") {
-			# Not enough for req.hash + vmod_priv + 30 bytes
-			vtc.workspace_alloc(client, -90);
+		if (req.url ~ "struct") {
+			# Enough for the req.hash blob.
+			# Not enough for the sub-blob.
+			call leave_struct;
 		}
-		set req.http.Encode = blob.encode(blob=blob.sub(req.hash, 30B));
-		return( synth(200) );
+		blob.encode(blob=blob.sub(req.hash, 30B));
+		return (synth(200));
 	}
 }
 
-client c1 -run
-client c2 -run
+logexpect l1 {
+	expect 0 * VCL_use	vcl5
+	expect 0 = ReqURL	req.hash
+	expect 0 = VCL_Log	shrink
+	expect 0 = VCL_Error	"Workspace overflow .req.hash."
+	expect 0 * VCL_use	vcl5
+	expect 0 = ReqURL	struct
+	expect 0 = VCL_Log	shrink
+	expect 0 = VCL_Error	"Workspace overflow .blob.sub."
+} -start
+
+client c-req-hash -run
+client c-struct -run
+
+logexpect l1 -wait
diff --git a/lib/libvmod_blob/vmod_blob.c b/lib/libvmod_blob/vmod_blob.c
index 2a14e8b20..60c90d9ff 100644
--- a/lib/libvmod_blob/vmod_blob.c
+++ b/lib/libvmod_blob/vmod_blob.c
@@ -534,6 +534,6 @@ vmod_sub(VRT_CTX, VCL_BLOB b, VCL_BYTES n, VCL_BYTES off)
 		return null_blob;
 
 
-	return (VRT_blob(ctx, "blob.sub()",
+	return (VRT_blob(ctx, "blob.sub",
 	    (const char *)b->blob + off, n, b->type));
 }

From dridi.boukelmoune at gmail.com  Thu Nov 14 07:15:06 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Thu, 14 Nov 2019 07:15:06 +0000 (UTC)
Subject: [master] 2215057dd We can't have fun outside the ISO C99 standard
Message-ID: <20191114071506.C2AD1A5B46@lists.varnish-cache.org>


commit 2215057dd50959f6697448770e2f77dfd3931b40
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Thu Nov 14 08:10:31 2019 +0100

    We can't have fun outside the ISO C99 standard
    
    Because someone brought a suncc to a vtest party.

diff --git a/bin/varnishtest/tests/m00000.vtc b/bin/varnishtest/tests/m00000.vtc
index a60147dde..206919fa2 100644
--- a/bin/varnishtest/tests/m00000.vtc
+++ b/bin/varnishtest/tests/m00000.vtc
@@ -152,7 +152,6 @@ varnish v1 -vcl {
 	backend be none;
 
 	C{
-		struct empty { };
 		struct zc {
 			size_t	z;
 			char	c;
@@ -202,11 +201,6 @@ varnish v1 -vcl {
 	}
 
 	sub vcl_synth {
-		C{ SETHDR(empty); }C
-		set resp.http.empty-typesize = vtc.typesize("");
-		set resp.http.empty-match =
-		    (resp.http.empty-sizeof == resp.http.empty-typesize);
-
 		C{ SETHDR(zc); }C
 		set resp.http.zc-typesize = vtc.typesize("zc");
 		set resp.http.zc-match =
diff --git a/lib/libvmod_vtc/vmod_vtc.c b/lib/libvmod_vtc/vmod_vtc.c
index 51a279d27..9370a403e 100644
--- a/lib/libvmod_vtc/vmod_vtc.c
+++ b/lib/libvmod_vtc/vmod_vtc.c
@@ -271,6 +271,7 @@ vmod_typesize(VRT_CTX, VCL_STRING s)
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	AN(s);
+	AN(*s);
 
 	for (; *s; s++) {
 		switch (*s) {
@@ -296,10 +297,8 @@ vmod_typesize(VRT_CTX, VCL_STRING s)
 			i += (l - a); /* align */
 		i += l;
 	}
-	if (i > 0) {
-		a = i % p;
-		if (a != 0)
-			i += (p - a); /* pad */
-	}
+	a = i % p;
+	if (a != 0)
+		i += (p - a); /* pad */
 	return ((VCL_INT)i);
 }

From dridi.boukelmoune at gmail.com  Thu Nov 14 07:34:06 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Thu, 14 Nov 2019 07:34:06 +0000 (UTC)
Subject: [master] 85fca27b3 GC stale check
Message-ID: <20191114073406.49DF6A64AF@lists.varnish-cache.org>


commit 85fca27b320e0c28caa08f24ffc6b852d2c73843
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Thu Nov 14 08:32:39 2019 +0100

    GC stale check

diff --git a/bin/varnishtest/tests/m00000.vtc b/bin/varnishtest/tests/m00000.vtc
index 206919fa2..e75bcd6de 100644
--- a/bin/varnishtest/tests/m00000.vtc
+++ b/bin/varnishtest/tests/m00000.vtc
@@ -246,7 +246,6 @@ varnish v1 -vcl {
 client c1 {
 	txreq
 	rxresp
-	expect resp.http.empty-match == true
 
 	expect resp.http.zc-match == true
 	expect resp.http.ic-match == true

From dridi.boukelmoune at gmail.com  Thu Nov 14 10:49:06 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Thu, 14 Nov 2019 10:49:06 +0000 (UTC)
Subject: [master] 1d23a7335 Correct WS allocation in tlv_string
Message-ID: <20191114104906.608B6ADC27@lists.varnish-cache.org>


commit 1d23a7335eda7741574e9b680401f5acc2a4d67d
Author: Emmanuel Hocdet <manu at gandi.net>
Date:   Thu Nov 14 11:14:07 2019 +0100

    Correct WS allocation in tlv_string
    
    Fixes #3131

diff --git a/lib/libvmod_proxy/vmod_proxy.c b/lib/libvmod_proxy/vmod_proxy.c
index 58f2db295..222de9c53 100644
--- a/lib/libvmod_proxy/vmod_proxy.c
+++ b/lib/libvmod_proxy/vmod_proxy.c
@@ -105,12 +105,13 @@ tlv_string(VRT_CTX, int tlv)
 
 	if (VPX_tlv(ctx->req, tlv, (void **)&dst, &len))
 		return (NULL);
-	if (!WS_ReserveSize(ctx->ws, len+1))
+	d = WS_Alloc(ctx->ws, len+1);
+	if (d == NULL) {
+		VRT_fail(ctx, "proxy.TLV: out of workspace");
 		return (NULL);
-	d = ctx->ws->f;
+	}
 	memcpy(d, dst, len);
 	d[len] = '\0';
-	WS_Release(ctx->ws, len+1);
 	return (d);
 }
 

From phk at phk.freebsd.dk  Thu Nov 14 23:07:39 2019
From: phk at phk.freebsd.dk (Poul-Henning Kamp)
Date: Thu, 14 Nov 2019 23:07:39 +0000
Subject: [master] 1d23a7335 Correct WS allocation in tlv_string
In-Reply-To: <20191114104906.608B6ADC27@lists.varnish-cache.org>
References: <20191114104906.608B6ADC27@lists.varnish-cache.org>
Message-ID: <30168.1573772859@critter.freebsd.dk>

--------

I thought the whole point was to *not* cause VRT_fail in *that* function ?

In message <20191114104906.608B6ADC27 at lists.varnish-cache.org>, Dridi Boukelmoune writes:
>
>commit 1d23a7335eda7741574e9b680401f5acc2a4d67d
>Author: Emmanuel Hocdet <manu at gandi.net>
>Date:   Thu Nov 14 11:14:07 2019 +0100
>
>    Correct WS allocation in tlv_string
>    
>    Fixes #3131
>
>diff --git a/lib/libvmod_proxy/vmod_proxy.c b/lib/libvmod_proxy/vmod_proxy.c
>index 58f2db295..222de9c53 100644
>--- a/lib/libvmod_proxy/vmod_proxy.c
>+++ b/lib/libvmod_proxy/vmod_proxy.c
>@@ -105,12 +105,13 @@ tlv_string(VRT_CTX, int tlv)
> 
> 	if (VPX_tlv(ctx->req, tlv, (void **)&dst, &len))
> 		return (NULL);
>-	if (!WS_ReserveSize(ctx->ws, len+1))
>+	d = WS_Alloc(ctx->ws, len+1);
>+	if (d == NULL) {
>+		VRT_fail(ctx, "proxy.TLV: out of workspace");
> 		return (NULL);
>-	d = ctx->ws->f;
>+	}
> 	memcpy(d, dst, len);
> 	d[len] = '\0';
>-	WS_Release(ctx->ws, len+1);
> 	return (d);
> }
> 
>_______________________________________________
>varnish-commit mailing list
>varnish-commit at varnish-cache.org
>https://www.varnish-cache.org/lists/mailman/listinfo/varnish-commit
>

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

From dridi.boukelmoune at gmail.com  Fri Nov 15 07:01:07 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Fri, 15 Nov 2019 07:01:07 +0000 (UTC)
Subject: [master] b9eaa9436 Revert "Correct WS allocation in tlv_string"
Message-ID: <20191115070107.BC163A3C9E@lists.varnish-cache.org>


commit b9eaa94361aa1da0634cf2d140b2366948660209
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Nov 15 08:00:01 2019 +0100

    Revert "Correct WS allocation in tlv_string"
    
    This reverts commit 1d23a7335eda7741574e9b680401f5acc2a4d67d.

diff --git a/lib/libvmod_proxy/vmod_proxy.c b/lib/libvmod_proxy/vmod_proxy.c
index 222de9c53..58f2db295 100644
--- a/lib/libvmod_proxy/vmod_proxy.c
+++ b/lib/libvmod_proxy/vmod_proxy.c
@@ -105,13 +105,12 @@ tlv_string(VRT_CTX, int tlv)
 
 	if (VPX_tlv(ctx->req, tlv, (void **)&dst, &len))
 		return (NULL);
-	d = WS_Alloc(ctx->ws, len+1);
-	if (d == NULL) {
-		VRT_fail(ctx, "proxy.TLV: out of workspace");
+	if (!WS_ReserveSize(ctx->ws, len+1))
 		return (NULL);
-	}
+	d = ctx->ws->f;
 	memcpy(d, dst, len);
 	d[len] = '\0';
+	WS_Release(ctx->ws, len+1);
 	return (d);
 }
 

From dridi at varni.sh  Fri Nov 15 07:01:17 2019
From: dridi at varni.sh (Dridi Boukelmoune)
Date: Fri, 15 Nov 2019 07:01:17 +0000
Subject: [master] 1d23a7335 Correct WS allocation in tlv_string
In-Reply-To: <30168.1573772859@critter.freebsd.dk>
References: <20191114104906.608B6ADC27@lists.varnish-cache.org>
 <30168.1573772859@critter.freebsd.dk>
Message-ID: <CABoVN9C84yBRWtd8-jbkwU0aUAjRfBgjJhw55gYRmePDhW66jA@mail.gmail.com>

On Thu, Nov 14, 2019 at 11:07 PM Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
>
> --------
>
> I thought the whole point was to *not* cause VRT_fail in *that* function ?

Yes, Nils caught this first and continued the discussion on IRC and github.

I will push a revert of this patch.

Dridi

From dridi.boukelmoune at gmail.com  Fri Nov 15 16:18:05 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Fri, 15 Nov 2019 16:18:05 +0000 (UTC)
Subject: [master] 3ff762862 Make sure to use none backends in generated C code
Message-ID: <20191115161805.EFFD5B1751@lists.varnish-cache.org>


commit 3ff7628628b0c1fb846fd86576bd1c9048517251
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Nov 15 17:04:40 2019 +0100

    Make sure to use none backends in generated C code
    
    Otherwise you might run into this:
    
        Message from VCC-compiler:
        Unused backend nil, defined:
        ('<vcl.inline>' Line 4 Pos 17)
                backend nil none;
        ----------------###------
    
        (That was just a warning)
        Message from C-compiler:
        vgc.c:1476:20: error: unused variable 'vgc_backend_nil' [-Werror,-Wunused-variable]
        static VCL_BACKEND vgc_backend_nil;
                           ^
        1 error generated.
        Running C-compiler failed, exited with 1
        VCL compilation failed
    
    This is done in both init and discard code to maintain the balance.

diff --git a/bin/varnishtest/tests/v00060.vtc b/bin/varnishtest/tests/v00060.vtc
index b1540dd5a..28bbcb89f 100644
--- a/bin/varnishtest/tests/v00060.vtc
+++ b/bin/varnishtest/tests/v00060.vtc
@@ -43,3 +43,9 @@ client c1 {
 	rxresp
 	expect resp.status == 503
 } -run
+
+varnish v1 -cliok "param.set vcc_err_unref off"
+varnish v1 -vcl {
+	backend bad { .host = "${bad_backend}"; }
+	backend nil none;
+}
diff --git a/lib/libvcc/vcc_backend.c b/lib/libvcc/vcc_backend.c
index c8f5ee8cd..374eeec5c 100644
--- a/lib/libvcc/vcc_backend.c
+++ b/lib/libvcc/vcc_backend.c
@@ -330,6 +330,9 @@ vcc_ParseHostDef(struct vcc *tl, const struct token *t_be, const char *vgcname)
 		Fb(tl, 0, "\n\t%s = (NULL);\n", vgcname);
 		vcc_NextToken(tl);
 		SkipToken(tl, ';');
+		ifp = New_IniFin(tl);
+		VSB_printf(ifp->ini, "\t(void)%s;\n", vgcname);
+		VSB_printf(ifp->fin, "\t\t(void)%s;\n", vgcname);
 		return;
 	}
 

From nils.goroll at uplex.de  Sat Nov 16 11:11:05 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Sat, 16 Nov 2019 11:11:05 +0000 (UTC)
Subject: [master] cd92f289a Set SA_CLIENT_{IP,PORT} only once
Message-ID: <20191116111105.80373A1995@lists.varnish-cache.org>


commit cd92f289a2388a0b18e6c7412e6e6def7c49b4a8
Author: Emmanuel Hocdet <manu at gandi.net>
Date:   Fri Nov 15 12:48:22 2019 +0100

    Set SA_CLIENT_{IP,PORT} only once

diff --git a/bin/varnishd/proxy/cache_proxy_proto.c b/bin/varnishd/proxy/cache_proxy_proto.c
index 20a626d40..78b120c2a 100644
--- a/bin/varnishd/proxy/cache_proxy_proto.c
+++ b/bin/varnishd/proxy/cache_proxy_proto.c
@@ -126,8 +126,6 @@ vpx_proto1(const struct worker *wrk, const struct req *req)
 		    "PROXY1: Cannot resolve destination address");
 		return (-1);
 	}
-	SES_Set_String_Attr(req->sp, SA_CLIENT_IP, fld[1]);
-	SES_Set_String_Attr(req->sp, SA_CLIENT_PORT, fld[3]);
 
 	VSL(SLT_Proxy, req->sp->vxid, "1 %s %s %s %s",
 	    fld[1], fld[3], fld[2], fld[4]);

From nils.goroll at uplex.de  Sat Nov 16 12:05:06 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Sat, 16 Nov 2019 12:05:06 +0000 (UTC)
Subject: [master] 99a3ee6a0 sensible lower bound for vcl_cooldown parameter
Message-ID: <20191116120506.24EE2A2EB1@lists.varnish-cache.org>


commit 99a3ee6a0dbeb99c4325d2d2a1cbe697782d1454
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Sat Nov 16 13:04:04 2019 +0100

    sensible lower bound for vcl_cooldown parameter
    
    Fixes #3135

diff --git a/include/tbl/params.h b/include/tbl/params.h
index f8c9e6de9..272829a10 100644
--- a/include/tbl/params.h
+++ b/include/tbl/params.h
@@ -1522,7 +1522,7 @@ PARAM(
 PARAM(
 	/* name */	vcl_cooldown,
 	/* typ */	timeout,
-	/* min */	"0.000",
+	/* min */	"1.000",
 	/* max */	NULL,
 	/* default */	"600.000",
 	/* units */	"seconds",

From nils.goroll at uplex.de  Sat Nov 16 14:25:06 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Sat, 16 Nov 2019 14:25:06 +0000 (UTC)
Subject: [master] e79105520 fix oops from previous commit
Message-ID: <20191116142506.C7693A609F@lists.varnish-cache.org>


commit e791055204dc53b833d8735dd868a96e88912324
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Sat Nov 16 15:24:13 2019 +0100

    fix oops from previous commit

diff --git a/bin/varnishtest/tests/r02433.vtc b/bin/varnishtest/tests/r02433.vtc
index 26cceeb08..9f93df073 100644
--- a/bin/varnishtest/tests/r02433.vtc
+++ b/bin/varnishtest/tests/r02433.vtc
@@ -2,7 +2,7 @@ varnishtest "label a cold vcl"
 
 server s1 { } -start
 
-varnish v1 -cliok "param.set vcl_cooldown 0.1"
+varnish v1 -cliok "param.set vcl_cooldown 1"
 varnish v1 -vcl+backend {
 	import debug; # can fail a VCL warmup
 } -start

From dridi.boukelmoune at gmail.com  Sun Nov 17 13:00:06 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Sun, 17 Nov 2019 13:00:06 +0000 (UTC)
Subject: [master] dbe38ae7f Fixes #3136: Typos in user guide.
Message-ID: <20191117130006.9A38397B57@lists.varnish-cache.org>


commit dbe38ae7faca29d51d88d627315c29504860b8eb
Author: Deepjyoti Mondal <deepjyoti.m at media.net>
Date:   Sun Nov 17 17:56:30 2019 +0530

    Fixes #3136: Typos in user guide.
    
    Fixed the typos in the files mentioned in the corresponding issue.

diff --git a/doc/sphinx/tutorial/peculiarities.rst b/doc/sphinx/tutorial/peculiarities.rst
index ca95e6356..60709b0be 100644
--- a/doc/sphinx/tutorial/peculiarities.rst
+++ b/doc/sphinx/tutorial/peculiarities.rst
@@ -22,7 +22,7 @@ handled.
 varnishadm
 ~~~~~~~~~~
 
-Varnish Cache has an admin console. You can connect it it through the
+Varnish Cache has an admin console. You can connect it through the
 :ref:`varnishadm(1)` command. In order to connect the user needs to be
 able to read `/etc/varnish/secret` in order to authenticate.
 
diff --git a/doc/sphinx/users-guide/run_security.rst b/doc/sphinx/users-guide/run_security.rst
index 5159ef3db..9d847df6b 100644
--- a/doc/sphinx/users-guide/run_security.rst
+++ b/doc/sphinx/users-guide/run_security.rst
@@ -150,7 +150,7 @@ HTTP service, but a few can do more damage than others:
 	Execute arbitrary programs
 
 :ref:`ref_param_vcc_allow_inline_c`
-        Allow inline C in VCL, which would any C code from VCL to be executed by Varnish.
+        Allow inline C in VCL, which would allow any C code from VCL to be executed by Varnish.
 
 Furthermore you may want to look at and lock down:
 

From martin at varnish-software.com  Mon Nov 18 14:33:07 2019
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon, 18 Nov 2019 14:33:07 +0000 (UTC)
Subject: [master] bdcdaf936 Govern thread creation by queue length instead of
 dry signals
Message-ID: <20191118143307.8A6D69234A@lists.varnish-cache.org>


commit bdcdaf936ca13e07b8a71584e93669a0d0fcaf41
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Wed Mar 13 14:14:44 2019 +0100

    Govern thread creation by queue length instead of dry signals
    
    When getidleworker signals the pool herder to spawn a thread, it increases
    the dry counter, and the herder resets dry again when spawning a single
    thread. This will in many cases only create a single thread even though
    the herder was signaled dry multiple times, and may cause a situation
    where the cache acceptor is queued and no new thread is created. Together
    with long lasting tasks (ie endless pipelines), and all other tasks having
    higher priority, this will prevent the cache acceptor from being
    rescheduled.
    
    c00096.vtc demonstrates how this can lock up.
    
    To fix this, spawn threads if we have queued tasks and we are below the
    thread maximum level.

diff --git a/bin/varnishd/cache/cache_pool.h b/bin/varnishd/cache/cache_pool.h
index 9902e32ad..4e668fa0b 100644
--- a/bin/varnishd/cache/cache_pool.h
+++ b/bin/varnishd/cache/cache_pool.h
@@ -48,7 +48,6 @@ struct pool {
 	struct taskhead			idle_queue;
 	struct taskhead			queues[TASK_QUEUE_END];
 	unsigned			nthr;
-	unsigned			dry;
 	unsigned			lqueue;
 	uintmax_t			sdropped;
 	uintmax_t			rdropped;
diff --git a/bin/varnishd/cache/cache_wrk.c b/bin/varnishd/cache/cache_wrk.c
index 9f1386c5f..eeabe4061 100644
--- a/bin/varnishd/cache/cache_wrk.c
+++ b/bin/varnishd/cache/cache_wrk.c
@@ -194,13 +194,8 @@ pool_getidleworker(struct pool *pp, enum task_prio prio)
 			AZ(pp->nidle);
 	}
 
-	if (pt == NULL) {
-		if (pp->nthr < cache_param->wthread_max) {
-			pp->dry++;
-			AZ(pthread_cond_signal(&pp->herder_cond));
-		}
+	if (pt == NULL)
 		return (NULL);
-	}
 
 	AZ(pt->func);
 	CAST_OBJ_NOTNULL(wrk, pt->priv, WORKER_MAGIC);
@@ -303,6 +298,7 @@ Pool_Task(struct pool *pp, struct pool_task *task, enum task_prio prio)
 		pp->nqueued++;
 		pp->lqueue++;
 		VTAILQ_INSERT_TAIL(&pp->queues[prio], task, list);
+		AZ(pthread_cond_signal(&pp->herder_cond));
 	} else {
 		if (prio == TASK_QUEUE_REQ)
 			pp->sdropped++;
@@ -473,7 +469,6 @@ pool_breed(struct pool *qp)
 		Lck_Unlock(&pool_mtx);
 		VTIM_sleep(cache_param->wthread_fail_delay);
 	} else {
-		qp->dry = 0;
 		qp->nthr++;
 		Lck_Lock(&pool_mtx);
 		VSC_C_main->threads++;
@@ -552,7 +547,7 @@ pool_herder(void *priv)
 
 		/* Make more threads if needed and allowed */
 		if (pp->nthr < wthread_min ||
-		    (pp->dry && pp->nthr < cache_param->wthread_max)) {
+		    (pp->lqueue > 0 && pp->nthr < cache_param->wthread_max)) {
 			pool_breed(pp);
 			continue;
 		}
@@ -613,16 +608,14 @@ pool_herder(void *priv)
 			continue;
 		}
 		Lck_Lock(&pp->mtx);
-		if (!pp->dry) {
+		if (pp->lqueue == 0) {
 			if (DO_DEBUG(DBG_VTC_MODE))
 				delay = 0.5;
 			(void)Lck_CondWait(&pp->herder_cond, &pp->mtx,
 				VTIM_real() + delay);
-		} else {
+		} else
 			/* XXX: unsafe counters */
 			VSC_C_main->threads_limited++;
-			pp->dry = 0;
-		}
 		Lck_Unlock(&pp->mtx);
 	}
 	return (NULL);
diff --git a/bin/varnishtest/tests/c00096.vtc b/bin/varnishtest/tests/c00096.vtc
new file mode 100644
index 000000000..cf80a0f53
--- /dev/null
+++ b/bin/varnishtest/tests/c00096.vtc
@@ -0,0 +1,106 @@
+varnishtest "Test thread creation on acceptor thread queuing"
+
+# This tests that we are able to spawn new threads in the event that the
+# cache acceptor has been queued. It does this by starting 6 long lasting
+# fetches, which will consume 12 threads. That exceeds the initial
+# allotment of 10 threads, giving some probability that the acceptor
+# thread is queued. Then a single quick fetch is done, which should be
+# served since we are well below the maximum number of threads allowed.
+
+# Barrier b1 blocks the slow servers from finishing until the quick fetch
+# is done.
+barrier b1 cond 7
+
+# Barrier b2 blocks the start of the quick fetch until all slow fetches
+# are known to hold captive two threads each.
+barrier b2 cond 7
+
+server s0 {
+	rxreq
+	txresp -nolen -hdr "Content-Length: 10" -hdr "Connection: close"
+	send "123"
+	barrier b1 sync
+	send "4567890"
+	expect_close
+} -dispatch
+
+server stest {
+	rxreq
+	txresp -body "All good"
+} -start
+
+varnish v1 -arg "-p debug=+syncvsl -p debug=+flush_head"
+varnish v1 -arg "-p thread_pools=1 -p thread_pool_min=10"
+varnish v1 -vcl+backend {
+	sub vcl_backend_fetch {
+		if (bereq.url == "/test") {
+			set bereq.backend = stest;
+		} else {
+			set bereq.backend = s0;
+		}
+	}
+} -start
+
+# NB: we might go above 10 threads when early tasks are submitted to
+# the pool since at least one idle thread must be kept in the pool
+# reserve.
+varnish v1 -expect MAIN.threads >= 10
+
+client c1 {
+	txreq -url /1
+	rxresphdrs
+	barrier b2 sync
+	rxrespbody
+} -start
+
+client c2 {
+	txreq -url /2
+	rxresphdrs
+	barrier b2 sync
+	rxrespbody
+} -start
+
+client c3 {
+	txreq -url /3
+	rxresphdrs
+	barrier b2 sync
+	rxrespbody
+} -start
+
+client c4 {
+	txreq -url /4
+	rxresphdrs
+	barrier b2 sync
+	rxrespbody
+} -start
+
+client c5 {
+	txreq -url /5
+	rxresphdrs
+	barrier b2 sync
+	rxrespbody
+} -start
+
+client c6 {
+	txreq -url /6
+	rxresphdrs
+	barrier b2 sync
+	rxrespbody
+} -start
+
+client ctest {
+	barrier b2 sync
+	txreq -url "/test"
+	rxresp
+	expect resp.status == 200
+	expect resp.body == "All good"
+} -run
+
+barrier b1 sync
+
+client c1 -wait
+client c2 -wait
+client c3 -wait
+client c4 -wait
+client c5 -wait
+client c6 -wait

From martin at varnish-software.com  Mon Nov 18 14:33:07 2019
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon, 18 Nov 2019 14:33:07 +0000 (UTC)
Subject: [master] dbd95ce2e Fix a small memory leak
Message-ID: <20191118143307.A22469234F@lists.varnish-cache.org>


commit dbd95ce2efa334c682b47c3cd47adfe735484f09
Author: Martin Blix Grydeland <martin at varnish-software.com>
Date:   Wed Mar 13 14:43:01 2019 +0100

    Fix a small memory leak
    
    Fix a small memory leak when failing to spawn a new thread.

diff --git a/bin/varnishd/cache/cache_wrk.c b/bin/varnishd/cache/cache_wrk.c
index eeabe4061..dd032850f 100644
--- a/bin/varnishd/cache/cache_wrk.c
+++ b/bin/varnishd/cache/cache_wrk.c
@@ -462,6 +462,7 @@ pool_breed(struct pool *qp)
 	pi->qp = qp;
 
 	if (pthread_create(&tp, &tp_attr, pool_thread, pi)) {
+		FREE_OBJ(pi);
 		VSL(SLT_Debug, 0, "Create worker thread failed %d %s",
 		    errno, vstrerror(errno));
 		Lck_Lock(&pool_mtx);

From martin at varnish-software.com  Mon Nov 18 14:33:07 2019
From: martin at varnish-software.com (Martin Blix Grydeland)
Date: Mon, 18 Nov 2019 14:33:07 +0000 (UTC)
Subject: [master] 36606226f SQUASHME: c96 stabilization
Message-ID: <20191118143307.BAB9592352@lists.varnish-cache.org>


commit 36606226f29f589a6104b3d6514693306953b0e2
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Sun Oct 27 15:42:06 2019 +0100

    SQUASHME: c96 stabilization

diff --git a/bin/varnishtest/tests/c00096.vtc b/bin/varnishtest/tests/c00096.vtc
index cf80a0f53..6c1ca372a 100644
--- a/bin/varnishtest/tests/c00096.vtc
+++ b/bin/varnishtest/tests/c00096.vtc
@@ -31,6 +31,7 @@ server stest {
 
 varnish v1 -arg "-p debug=+syncvsl -p debug=+flush_head"
 varnish v1 -arg "-p thread_pools=1 -p thread_pool_min=10"
+varnish v1 -arg "-p thread_pool_add_delay=0.01"
 varnish v1 -vcl+backend {
 	sub vcl_backend_fetch {
 		if (bereq.url == "/test") {
@@ -41,10 +42,7 @@ varnish v1 -vcl+backend {
 	}
 } -start
 
-# NB: we might go above 10 threads when early tasks are submitted to
-# the pool since at least one idle thread must be kept in the pool
-# reserve.
-varnish v1 -expect MAIN.threads >= 10
+varnish v1 -expect MAIN.threads == 10
 
 client c1 {
 	txreq -url /1

From dridi.boukelmoune at gmail.com  Mon Nov 18 14:38:06 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Mon, 18 Nov 2019 14:38:06 +0000 (UTC)
Subject: [master] bec707045 Make a note of the under-breeding issue
Message-ID: <20191118143806.B5B7492AF9@lists.varnish-cache.org>


commit bec7070455c87b2173cc94b320bb3a7a1ee3fb60
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Mon Nov 18 15:36:55 2019 +0100

    Make a note of the under-breeding issue

diff --git a/bin/varnishtest/tests/c00096.vtc b/bin/varnishtest/tests/c00096.vtc
index 6c1ca372a..31443ac1e 100644
--- a/bin/varnishtest/tests/c00096.vtc
+++ b/bin/varnishtest/tests/c00096.vtc
@@ -42,6 +42,10 @@ varnish v1 -vcl+backend {
 	}
 } -start
 
+# XXX: there is a bug in the pool_herder logic that could
+# lead to over-breeding of threads resulting in more than
+# 10 threads. This is currently mitigated by increasing the
+# thread_pool_add_delay parameter.
 varnish v1 -expect MAIN.threads == 10
 
 client c1 {

From nils.goroll at uplex.de  Mon Nov 18 14:48:07 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 18 Nov 2019 14:48:07 +0000 (UTC)
Subject: [master] 90952d584 I hope this won't start a commit-war with @Dridi :)
Message-ID: <20191118144807.964C893065@lists.varnish-cache.org>


commit 90952d5846360d3f598e998786d858a7f7ac3dd5
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon Nov 18 15:46:26 2019 +0100

    I hope this won't start a commit-war with @Dridi :)

diff --git a/bin/varnishtest/tests/c00096.vtc b/bin/varnishtest/tests/c00096.vtc
index 31443ac1e..23f6a4c87 100644
--- a/bin/varnishtest/tests/c00096.vtc
+++ b/bin/varnishtest/tests/c00096.vtc
@@ -42,10 +42,11 @@ varnish v1 -vcl+backend {
 	}
 } -start
 
-# XXX: there is a bug in the pool_herder logic that could
-# lead to over-breeding of threads resulting in more than
-# 10 threads. This is currently mitigated by increasing the
+# XXX: to minimize the amount of synchronization in the pool_herder
+# logic, it may over-breed threads, resulting in more than 10
+# threads. This is currently mitigated by increasing the
 # thread_pool_add_delay parameter.
+
 varnish v1 -expect MAIN.threads == 10
 
 client c1 {

From nils.goroll at uplex.de  Mon Nov 18 15:05:07 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 18 Nov 2019 15:05:07 +0000 (UTC)
Subject: [master] 465f2f8c3 Ensure all director_list operations happen under
 the vcl_mtx
Message-ID: <20191118150507.A8409938E0@lists.varnish-cache.org>


commit 465f2f8c364cb7e5c6ae93bf46f2ba0f7c757d92
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu Nov 7 09:14:44 2019 +0100

    Ensure all director_list operations happen under the vcl_mtx
    
    Ref #3094

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index fe905fea8..721622738 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -278,6 +278,7 @@ vcl_iterdir(struct cli *cli, const char *pat, const struct vcl *vcl,
 	int i, found = 0;
 	struct vcldir *vdir;
 
+	Lck_AssertHeld(&vcl_mtx);
 	VTAILQ_FOREACH(vdir, &vcl->director_list, list) {
 		if (fnmatch(pat, vdir->cli_name, 0))
 			continue;
@@ -343,8 +344,10 @@ vcl_BackendEvent(const struct vcl *vcl, enum vcl_event_e e)
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
 	AZ(vcl->busy);
 
+	Lck_Lock(&vcl_mtx);
 	VTAILQ_FOREACH(vdir, &vcl->director_list, list)
 		VDI_Event(vdir->dir, e);
+	Lck_Unlock(&vcl_mtx);
 }
 
 static void
@@ -355,6 +358,7 @@ vcl_KillBackends(struct vcl *vcl)
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
 	AZ(vcl->busy);
 	assert(VTAILQ_EMPTY(&vcl->ref_list));
+	Lck_Lock(&vcl_mtx);
 	while (1) {
 		vdir = VTAILQ_FIRST(&vcl->director_list);
 		if (vdir == NULL)
@@ -366,6 +370,7 @@ vcl_KillBackends(struct vcl *vcl)
 		FREE_OBJ(vdir->dir);
 		FREE_OBJ(vdir);
 	}
+	Lck_Unlock(&vcl_mtx);
 }
 
 /*--------------------------------------------------------------------*/

From nils.goroll at uplex.de  Mon Nov 18 15:05:07 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 18 Nov 2019 15:05:07 +0000 (UTC)
Subject: [master] 025f4dac9 change vcl temperature macros from vcl to
 temperature argument
Message-ID: <20191118150507.F2A4E938E4@lists.varnish-cache.org>


commit 025f4dac952afd40f7d22ab2de8bc4d953c95970
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 13 16:18:48 2019 +0100

    change vcl temperature macros from vcl to temperature argument
    
    we are going to need to check a temperature on its own

diff --git a/bin/varnishd/cache/cache_backend.c b/bin/varnishd/cache/cache_backend.c
index 328e9078b..402f9f6f4 100644
--- a/bin/varnishd/cache/cache_backend.c
+++ b/bin/varnishd/cache/cache_backend.c
@@ -604,7 +604,7 @@ VRT_new_backend_clustered(VRT_CTX, struct vsmw_cluster *vc,
 
 	if (be->director != NULL) {
 		/* for cold VCL, update initial director state */
-		if (be->probe != NULL && ! VCL_WARM(vcl))
+		if (be->probe != NULL && ! VCL_WARM(vcl->temp))
 			VBP_Update_Backend(be->probe);
 
 		Lck_Lock(&backends_mtx);
diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 721622738..bdff0e489 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -265,7 +265,7 @@ vcl_get(struct vcl **vcc, struct vcl *vcl)
 	(*vcc)->busy++;
 	Lck_Unlock(&vcl_mtx);
 	AZ(errno=pthread_rwlock_rdlock(&(*vcc)->temp_rwl));
-	assert(VCL_WARM(*vcc));
+	assert(VCL_WARM((*vcc)->temp));
 	AZ(errno=pthread_rwlock_unlock(&(*vcc)->temp_rwl));
 }
 
@@ -503,7 +503,7 @@ vcl_set_state(VRT_CTX, const char *state)
 	case '0':
 		if (vcl->temp == VCL_TEMP_COLD)
 			break;
-		if (vcl->busy == 0 && VCL_WARM(vcl)) {
+		if (vcl->busy == 0 && VCL_WARM(vcl->temp)) {
 			vcl->temp = VTAILQ_EMPTY(&vcl->ref_list) ?
 			    VCL_TEMP_COLD : VCL_TEMP_COOLING;
 			AZ(vcl_send_event(ctx, VCL_EVENT_COLD));
diff --git a/bin/varnishd/cache/cache_vcl.h b/bin/varnishd/cache/cache_vcl.h
index ac01706e2..cfe489956 100644
--- a/bin/varnishd/cache/cache_vcl.h
+++ b/bin/varnishd/cache/cache_vcl.h
@@ -80,7 +80,5 @@ extern const char * const VCL_TEMP_COOLING;
  * NB: The COOLING temperature is neither COLD nor WARM.
  * And LABEL is not a temperature, it's a different kind of VCL.
  */
-#define VCL_WARM(v) ((v)->temp == VCL_TEMP_WARM || (v)->temp == VCL_TEMP_BUSY)
-#define VCL_COLD(v) ((v)->temp == VCL_TEMP_INIT || (v)->temp == VCL_TEMP_COLD)
-
-
+#define VCL_WARM(t) ((t) == VCL_TEMP_WARM || (t) == VCL_TEMP_BUSY)
+#define VCL_COLD(t) ((t) == VCL_TEMP_INIT || (t) == VCL_TEMP_COLD)
diff --git a/bin/varnishd/cache/cache_vrt_vcl.c b/bin/varnishd/cache/cache_vrt_vcl.c
index a660c23e2..9356bceec 100644
--- a/bin/varnishd/cache/cache_vrt_vcl.c
+++ b/bin/varnishd/cache/cache_vrt_vcl.c
@@ -112,7 +112,7 @@ VCL_Ref(struct vcl *vcl)
 
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
 	AZ(errno=pthread_rwlock_rdlock(&vcl->temp_rwl));
-	assert(!VCL_COLD(vcl));
+	assert(!VCL_COLD(vcl->temp));
 	AZ(errno=pthread_rwlock_unlock(&vcl->temp_rwl));
 	Lck_Lock(&vcl_mtx);
 	assert(vcl->busy > 0);
@@ -193,7 +193,7 @@ VRT_AddDirector(VRT_CTX, const struct vdi_methods *m, void *priv,
 	VTAILQ_INSERT_TAIL(&vcl->director_list, vdir, list);
 	Lck_Unlock(&vcl_mtx);
 
-	if (VCL_WARM(vcl))
+	if (VCL_WARM(vcl->temp))
 		/* Only when adding backend to already warm VCL */
 		VDI_Event(d, VCL_EVENT_WARM);
 	else if (vcl->temp != VCL_TEMP_INIT)
@@ -220,7 +220,7 @@ VRT_DelDirector(VCL_BACKEND *bp)
 	Lck_Unlock(&vcl_mtx);
 
 	AZ(errno=pthread_rwlock_rdlock(&vcl->temp_rwl));
-	if (VCL_WARM(vcl))
+	if (VCL_WARM(vcl->temp))
 		VDI_Event(d, VCL_EVENT_COLD);
 	AZ(errno=pthread_rwlock_unlock(&vcl->temp_rwl));
 	if(vdir->methods->destroy != NULL)
@@ -363,7 +363,7 @@ VRT_VCL_Prevent_Discard(VRT_CTX, const char *desc)
 
 	vcl = ctx->vcl;
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
-	assert(VCL_WARM(vcl));
+	assert(VCL_WARM(vcl->temp));
 
 	ALLOC_OBJ(ref, VCLREF_MAGIC);
 	AN(ref);

From nils.goroll at uplex.de  Mon Nov 18 15:05:08 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 18 Nov 2019 15:05:08 +0000 (UTC)
Subject: [master] c671bb669 split out director deletion core
Message-ID: <20191118150508.1D613938E7@lists.varnish-cache.org>


commit c671bb6691f27172ae9103226cb09a99c9812483
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 13 16:56:33 2019 +0100

    split out director deletion core

diff --git a/bin/varnishd/cache/cache_vrt_vcl.c b/bin/varnishd/cache/cache_vrt_vcl.c
index 9356bceec..c1ee2c057 100644
--- a/bin/varnishd/cache/cache_vrt_vcl.c
+++ b/bin/varnishd/cache/cache_vrt_vcl.c
@@ -41,6 +41,8 @@
 #include "cache_director.h"
 #include "cache_vcl.h"
 
+static void deldirector(struct vcldir *);
+
 /*--------------------------------------------------------------------*/
 
 const char *
@@ -203,6 +205,16 @@ VRT_AddDirector(VRT_CTX, const struct vdi_methods *m, void *priv,
 	return (d);
 }
 
+static void
+deldirector(struct vcldir *vdir)
+{
+	if(vdir->methods->destroy != NULL)
+		vdir->methods->destroy(vdir->dir);
+	free(vdir->cli_name);
+	FREE_OBJ(vdir->dir);
+	FREE_OBJ(vdir);
+}
+
 void
 VRT_DelDirector(VCL_BACKEND *bp)
 {
@@ -223,12 +235,8 @@ VRT_DelDirector(VCL_BACKEND *bp)
 	if (VCL_WARM(vcl->temp))
 		VDI_Event(d, VCL_EVENT_COLD);
 	AZ(errno=pthread_rwlock_unlock(&vcl->temp_rwl));
-	if(vdir->methods->destroy != NULL)
-		vdir->methods->destroy(d);
 	assert (d == vdir->dir);
-	free(vdir->cli_name);
-	FREE_OBJ(vdir->dir);
-	FREE_OBJ(vdir);
+	deldirector(vdir);
 }
 
 void

From nils.goroll at uplex.de  Mon Nov 18 15:05:08 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 18 Nov 2019 15:05:08 +0000 (UTC)
Subject: [master] 615298547 remove the temperature lock which serves no purpose
Message-ID: <20191118150508.4F5EF938F4@lists.varnish-cache.org>


commit 615298547adfb06aa785b42552062bc391f56ecd
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Wed Nov 13 17:30:35 2019 +0100

    remove the temperature lock which serves no purpose
    
    we ensure that only backends on the director_list receive events.
    Because events happen in the cli thread, we do not need another level of
    serialization.
    
    There is no explicit vtc, the test case has been added to vmod_debug and
    runs with every invocation of that vmod.
    
    Fixes #3094

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index bdff0e489..c320d6273 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -264,9 +264,7 @@ vcl_get(struct vcl **vcc, struct vcl *vcl)
 	AZ((*vcc)->discard);
 	(*vcc)->busy++;
 	Lck_Unlock(&vcl_mtx);
-	AZ(errno=pthread_rwlock_rdlock(&(*vcc)->temp_rwl));
 	assert(VCL_WARM((*vcc)->temp));
-	AZ(errno=pthread_rwlock_unlock(&(*vcc)->temp_rwl));
 }
 
 /*--------------------------------------------------------------------*/
@@ -415,7 +413,6 @@ VCL_Open(const char *fn, struct vsb *msg)
 	}
 	ALLOC_OBJ(vcl, VCL_MAGIC);
 	AN(vcl);
-	AZ(errno=pthread_rwlock_init(&vcl->temp_rwl, NULL));
 	vcl->dlh = dlh;
 	vcl->conf = cnf;
 	return (vcl);
@@ -432,7 +429,6 @@ VCL_Close(struct vcl **vclp)
 	assert(VTAILQ_EMPTY(&vcl->vfps));
 	assert(VTAILQ_EMPTY(&vcl->vdps));
 	AZ(dlclose(vcl->dlh));
-	AZ(errno=pthread_rwlock_destroy(&vcl->temp_rwl));
 	FREE_OBJ(vcl);
 }
 
@@ -496,7 +492,6 @@ vcl_set_state(VRT_CTX, const char *state)
 	assert(ctx->msg != NULL || *state == '0');
 
 	vcl = ctx->vcl;
-	AZ(errno=pthread_rwlock_wrlock(&vcl->temp_rwl));
 	AN(vcl->temp);
 
 	switch (state[0]) {
@@ -543,7 +538,6 @@ vcl_set_state(VRT_CTX, const char *state)
 	}
 	if (i == 0 && state[1])
 		bprintf(vcl->state, "%s", state + 1);
-	AZ(errno=pthread_rwlock_unlock(&vcl->temp_rwl));
 
 	return (i);
 }
@@ -811,7 +805,6 @@ vcl_cli_discard(struct cli *cli, const char * const *av, void *priv)
 	if (!strcmp(vcl->state, VCL_TEMP_LABEL)) {
 		VTAILQ_REMOVE(&vcl_head, vcl, list);
 		free(vcl->loaded_name);
-		AZ(errno=pthread_rwlock_destroy(&vcl->temp_rwl));
 		FREE_OBJ(vcl);
 	} else if (vcl->temp == VCL_TEMP_COLD) {
 		VCL_Poll();
@@ -836,7 +829,6 @@ vcl_cli_label(struct cli *cli, const char * const *av, void *priv)
 		bprintf(lbl->state, "%s", VCL_TEMP_LABEL);
 		lbl->temp = VCL_TEMP_WARM;
 		REPLACE(lbl->loaded_name, av[2]);
-		AZ(errno=pthread_rwlock_init(&lbl->temp_rwl, NULL));
 		VTAILQ_INSERT_TAIL(&vcl_head, lbl, list);
 	}
 	if (lbl->label != NULL)
diff --git a/bin/varnishd/cache/cache_vcl.h b/bin/varnishd/cache/cache_vcl.h
index cfe489956..df45e9bf3 100644
--- a/bin/varnishd/cache/cache_vcl.h
+++ b/bin/varnishd/cache/cache_vcl.h
@@ -47,7 +47,6 @@ struct vcl {
 	unsigned		busy;
 	unsigned		discard;
 	const char		*temp;
-	pthread_rwlock_t	temp_rwl;
 	VTAILQ_HEAD(,vcldir)	director_list;
 	VTAILQ_HEAD(,vclref)	ref_list;
 	int			nrefs;
diff --git a/bin/varnishd/cache/cache_vrt_vcl.c b/bin/varnishd/cache/cache_vrt_vcl.c
index c1ee2c057..a8ddee7ac 100644
--- a/bin/varnishd/cache/cache_vrt_vcl.c
+++ b/bin/varnishd/cache/cache_vrt_vcl.c
@@ -113,9 +113,7 @@ VCL_Ref(struct vcl *vcl)
 {
 
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
-	AZ(errno=pthread_rwlock_rdlock(&vcl->temp_rwl));
 	assert(!VCL_COLD(vcl->temp));
-	AZ(errno=pthread_rwlock_unlock(&vcl->temp_rwl));
 	Lck_Lock(&vcl_mtx);
 	assert(vcl->busy > 0);
 	vcl->busy++;
@@ -152,6 +150,7 @@ VRT_AddDirector(VRT_CTX, const struct vdi_methods *m, void *priv,
 	struct vcl *vcl;
 	struct vcldir *vdir;
 	struct director *d;
+	const char *temp;
 	va_list ap;
 	int i;
 
@@ -160,11 +159,10 @@ VRT_AddDirector(VRT_CTX, const struct vdi_methods *m, void *priv,
 	AN(fmt);
 	vcl = ctx->vcl;
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
-	AZ(errno=pthread_rwlock_rdlock(&vcl->temp_rwl));
-	if (vcl->temp == VCL_TEMP_COOLING) {
-		AZ(errno=pthread_rwlock_unlock(&vcl->temp_rwl));
+
+	// opportunistic, re-checked again under lock
+	if (vcl->temp == VCL_TEMP_COOLING)
 		return (NULL);
-	}
 
 	ALLOC_OBJ(d, DIRECTOR_MAGIC);
 	AN(d);
@@ -192,15 +190,19 @@ VRT_AddDirector(VRT_CTX, const struct vdi_methods *m, void *priv,
 	vdir->health_changed = VTIM_real();
 
 	Lck_Lock(&vcl_mtx);
-	VTAILQ_INSERT_TAIL(&vcl->director_list, vdir, list);
+	temp = vcl->temp;
+	if (temp != VCL_TEMP_COOLING)
+		VTAILQ_INSERT_TAIL(&vcl->director_list, vdir, list);
+	if (VCL_WARM(temp))
+		VDI_Event(d, VCL_EVENT_WARM);
 	Lck_Unlock(&vcl_mtx);
 
-	if (VCL_WARM(vcl->temp))
-		/* Only when adding backend to already warm VCL */
-		VDI_Event(d, VCL_EVENT_WARM);
-	else if (vcl->temp != VCL_TEMP_INIT)
+	if (temp == VCL_TEMP_COOLING) {
+		deldirector(vdir);
+		return (NULL);
+	}
+	if (!VCL_WARM(temp) && temp != VCL_TEMP_INIT)
 		WRONG("Dynamic Backends can only be added to warm VCLs");
-	AZ(errno=pthread_rwlock_unlock(&vcl->temp_rwl));
 
 	return (d);
 }
@@ -220,6 +222,7 @@ VRT_DelDirector(VCL_BACKEND *bp)
 {
 	struct vcl *vcl;
 	struct vcldir *vdir;
+	const char *temp;
 	VCL_BACKEND d;
 
 	TAKE_OBJ_NOTNULL(d, bp, DIRECTOR_MAGIC);
@@ -227,14 +230,14 @@ VRT_DelDirector(VCL_BACKEND *bp)
 	CHECK_OBJ_NOTNULL(vdir, VCLDIR_MAGIC);
 	vcl = vdir->vcl;
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
+
 	Lck_Lock(&vcl_mtx);
+	temp = vcl->temp;
 	VTAILQ_REMOVE(&vcl->director_list, vdir, list);
 	Lck_Unlock(&vcl_mtx);
 
-	AZ(errno=pthread_rwlock_rdlock(&vcl->temp_rwl));
-	if (VCL_WARM(vcl->temp))
+	if (VCL_WARM(temp))
 		VDI_Event(d, VCL_EVENT_COLD);
-	AZ(errno=pthread_rwlock_unlock(&vcl->temp_rwl));
 	assert (d == vdir->dir);
 	deldirector(vdir);
 }
diff --git a/lib/libvmod_debug/vmod_debug.c b/lib/libvmod_debug/vmod_debug.c
index fd408f767..1a547e172 100644
--- a/lib/libvmod_debug/vmod_debug.c
+++ b/lib/libvmod_debug/vmod_debug.c
@@ -50,6 +50,7 @@ struct priv_vcl {
 	struct vclref		*vclref_discard;
 	struct vclref		*vclref_cold;
 	VCL_DURATION		vcl_discard_delay;
+	VCL_BACKEND		be;
 };
 
 
@@ -403,6 +404,10 @@ xyzzy_vcl_allow_cold(VRT_CTX, struct vmod_priv *priv)
 	VRT_VCL_Allow_Cold(&priv_vcl->vclref_cold);
 }
 
+static const struct vdi_methods empty_methods[1] = {{
+	.magic =	VDI_METHODS_MAGIC,
+	.type =	"debug.dummy"
+}};
 
 static int
 event_warm(VRT_CTX, const struct vmod_priv *priv)
@@ -423,6 +428,11 @@ event_warm(VRT_CTX, const struct vmod_priv *priv)
 
 	bprintf(buf, "vmod-debug ref on %s", VCL_Name(ctx->vcl));
 	priv_vcl->vclref_discard = VRT_VCL_Prevent_Discard(ctx, buf);
+
+	AZ(priv_vcl->be);
+	priv_vcl->be = VRT_AddDirector(ctx, empty_methods,
+	    NULL, "%s", "dir_warmcold");
+
 	return (0);
 }
 
@@ -450,6 +460,8 @@ event_cold(VRT_CTX, const struct vmod_priv *priv)
 
 	VSL(SLT_Debug, 0, "%s: VCL_EVENT_COLD", VCL_Name(ctx->vcl));
 
+	VRT_DelDirector(&priv_vcl->be);
+
 	if (priv_vcl->vcl_discard_delay == 0.0) {
 		VRT_VCL_Allow_Discard(&priv_vcl->vclref_discard);
 		return (0);

From nils.goroll at uplex.de  Mon Nov 18 15:32:06 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Mon, 18 Nov 2019 15:32:06 +0000 (UTC)
Subject: [master] b0c0eaeaa refactor the vcl temperature into a struct
Message-ID: <20191118153206.1AAC59491B@lists.varnish-cache.org>


commit b0c0eaeaaa1aa156feeaba2e46d339b4212a5221
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Mon Nov 18 16:30:57 2019 +0100

    refactor the vcl temperature into a struct
    
    as suggested by @Dridi with reference to @bsdphk in
    https://github.com/varnishcache/varnish-cache/pull/3130#pullrequestreview-316916213

diff --git a/bin/varnishd/cache/cache_backend.c b/bin/varnishd/cache/cache_backend.c
index 402f9f6f4..a2c437416 100644
--- a/bin/varnishd/cache/cache_backend.c
+++ b/bin/varnishd/cache/cache_backend.c
@@ -604,7 +604,7 @@ VRT_new_backend_clustered(VRT_CTX, struct vsmw_cluster *vc,
 
 	if (be->director != NULL) {
 		/* for cold VCL, update initial director state */
-		if (be->probe != NULL && ! VCL_WARM(vcl->temp))
+		if (be->probe != NULL && ! vcl->temp->is_warm)
 			VBP_Update_Backend(be->probe);
 
 		Lck_Lock(&backends_mtx);
diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index c320d6273..6b365cc56 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -45,14 +45,14 @@
 #include "vcli_serve.h"
 #include "vtim.h"
 
-const char * const VCL_TEMP_INIT = "init";
-const char * const VCL_TEMP_COLD = "cold";
-const char * const VCL_TEMP_WARM = "warm";
-const char * const VCL_TEMP_BUSY = "busy";
-const char * const VCL_TEMP_COOLING = "cooling";
+const struct vcltemp VCL_TEMP_INIT[1] = {{ .name = "init", .is_cold = 1 }};
+const struct vcltemp VCL_TEMP_COLD[1] = {{ .name = "cold", .is_cold = 1 }};
+const struct vcltemp VCL_TEMP_WARM[1] = {{ .name = "warm", .is_warm = 1 }};
+const struct vcltemp VCL_TEMP_BUSY[1] = {{ .name = "busy", .is_warm = 1 }};
+const struct vcltemp VCL_TEMP_COOLING[1] = {{ .name = "cooling" }};
 
 // not really a state
-static const char * const VCL_TEMP_LABEL = "label";
+const struct vcltemp VCL_TEMP_LABEL[1] = {{ .name = "label" }};
 
 /*
  * XXX: Presently all modifications to this list happen from the
@@ -254,17 +254,17 @@ vcl_get(struct vcl **vcc, struct vcl *vcl)
 				   * race */
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
 	if (vcl->label == NULL) {
-		AN(strcmp(vcl->state, VCL_TEMP_LABEL));
+		AN(strcmp(vcl->state, VCL_TEMP_LABEL->name));
 		*vcc = vcl;
 	} else {
-		AZ(strcmp(vcl->state, VCL_TEMP_LABEL));
+		AZ(strcmp(vcl->state, VCL_TEMP_LABEL->name));
 		*vcc = vcl->label;
 	}
 	CHECK_OBJ_NOTNULL(*vcc, VCL_MAGIC);
 	AZ((*vcc)->discard);
 	(*vcc)->busy++;
 	Lck_Unlock(&vcl_mtx);
-	assert(VCL_WARM((*vcc)->temp));
+	assert((*vcc)->temp->is_warm);
 }
 
 /*--------------------------------------------------------------------*/
@@ -498,7 +498,7 @@ vcl_set_state(VRT_CTX, const char *state)
 	case '0':
 		if (vcl->temp == VCL_TEMP_COLD)
 			break;
-		if (vcl->busy == 0 && VCL_WARM(vcl->temp)) {
+		if (vcl->busy == 0 && vcl->temp->is_warm) {
 			vcl->temp = VTAILQ_EMPTY(&vcl->ref_list) ?
 			    VCL_TEMP_COLD : VCL_TEMP_COOLING;
 			AZ(vcl_send_event(ctx, VCL_EVENT_COLD));
@@ -682,8 +682,8 @@ vcl_cli_list(struct cli *cli, const char * const *av, void *priv)
 			flg = "discarded";
 		} else
 			flg = "available";
-		VSB_printf(vsb, "%s\t%s\t%s\t%6u\t%s",
-		    flg, vcl->state, vcl->temp, vcl->busy, vcl->loaded_name);
+		VSB_printf(vsb, "%s\t%s\t%s\t%6u\t%s", flg, vcl->state,
+		    vcl->temp->name, vcl->busy, vcl->loaded_name);
 		if (vcl->label != NULL) {
 			VSB_printf(vsb, "\t->\t%s", vcl->label->loaded_name);
 			if (vcl->nrefs)
@@ -718,7 +718,7 @@ vcl_cli_list_json(struct cli *cli, const char * const *av, void *priv)
 		} else
 			VCLI_Out(cli, "\"available\",\n");
 		VCLI_Out(cli, "\"state\": \"%s\",\n", vcl->state);
-		VCLI_Out(cli, "\"temperature\": \"%s\",\n", vcl->temp);
+		VCLI_Out(cli, "\"temperature\": \"%s\",\n", vcl->temp->name);
 		VCLI_Out(cli, "\"busy\": %u,\n", vcl->busy);
 		VCLI_Out(cli, "\"name\": \"%s\"", vcl->loaded_name);
 		if (vcl->label != NULL) {
@@ -796,13 +796,13 @@ vcl_cli_discard(struct cli *cli, const char * const *av, void *priv)
 	VSC_C_main->n_vcl_avail--;
 	vcl->discard = 1;
 	if (vcl->label != NULL) {
-		AZ(strcmp(vcl->state, VCL_TEMP_LABEL));
+		AZ(strcmp(vcl->state, VCL_TEMP_LABEL->name));
 		vcl->label->nlabels--;
 		vcl->label= NULL;
 	}
 	Lck_Unlock(&vcl_mtx);
 
-	if (!strcmp(vcl->state, VCL_TEMP_LABEL)) {
+	if (!strcmp(vcl->state, VCL_TEMP_LABEL->name)) {
 		VTAILQ_REMOVE(&vcl_head, vcl, list);
 		free(vcl->loaded_name);
 		FREE_OBJ(vcl);
@@ -826,7 +826,7 @@ vcl_cli_label(struct cli *cli, const char * const *av, void *priv)
 	if (lbl == NULL) {
 		ALLOC_OBJ(lbl, VCL_MAGIC);
 		AN(lbl);
-		bprintf(lbl->state, "%s", VCL_TEMP_LABEL);
+		bprintf(lbl->state, "%s", VCL_TEMP_LABEL->name);
 		lbl->temp = VCL_TEMP_WARM;
 		REPLACE(lbl->loaded_name, av[2]);
 		VTAILQ_INSERT_TAIL(&vcl_head, lbl, list);
diff --git a/bin/varnishd/cache/cache_vcl.h b/bin/varnishd/cache/cache_vcl.h
index df45e9bf3..e6ac097c1 100644
--- a/bin/varnishd/cache/cache_vcl.h
+++ b/bin/varnishd/cache/cache_vcl.h
@@ -32,6 +32,7 @@
  */
 
 struct vfilter;
+struct vcltemp;;
 
 VTAILQ_HEAD(vfilter_head, vfilter);
 
@@ -46,7 +47,7 @@ struct vcl {
 	char			*loaded_name;
 	unsigned		busy;
 	unsigned		discard;
-	const char		*temp;
+	const struct vcltemp	*temp;
 	VTAILQ_HEAD(,vcldir)	director_list;
 	VTAILQ_HEAD(,vclref)	ref_list;
 	int			nrefs;
@@ -69,15 +70,18 @@ extern struct vcl		*vcl_active; /* protected by vcl_mtx */
 struct vcl *vcl_find(const char *);
 void vcl_get(struct vcl **, struct vcl *);
 
-extern const char * const VCL_TEMP_INIT;
-extern const char * const VCL_TEMP_COLD;
-extern const char * const VCL_TEMP_WARM;
-extern const char * const VCL_TEMP_BUSY;
-extern const char * const VCL_TEMP_COOLING;
+struct vcltemp {
+	const char * const	name;
+	unsigned		is_warm;
+	unsigned		is_cold;
+};
 
 /*
  * NB: The COOLING temperature is neither COLD nor WARM.
  * And LABEL is not a temperature, it's a different kind of VCL.
  */
-#define VCL_WARM(t) ((t) == VCL_TEMP_WARM || (t) == VCL_TEMP_BUSY)
-#define VCL_COLD(t) ((t) == VCL_TEMP_INIT || (t) == VCL_TEMP_COLD)
+extern const struct vcltemp VCL_TEMP_INIT[1];
+extern const struct vcltemp VCL_TEMP_COLD[1];
+extern const struct vcltemp VCL_TEMP_WARM[1];
+extern const struct vcltemp VCL_TEMP_BUSY[1];
+extern const struct vcltemp VCL_TEMP_COOLING[1];
diff --git a/bin/varnishd/cache/cache_vrt_vcl.c b/bin/varnishd/cache/cache_vrt_vcl.c
index a8ddee7ac..42bbf3b9b 100644
--- a/bin/varnishd/cache/cache_vrt_vcl.c
+++ b/bin/varnishd/cache/cache_vrt_vcl.c
@@ -113,7 +113,7 @@ VCL_Ref(struct vcl *vcl)
 {
 
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
-	assert(!VCL_COLD(vcl->temp));
+	assert(!vcl->temp->is_cold);
 	Lck_Lock(&vcl_mtx);
 	assert(vcl->busy > 0);
 	vcl->busy++;
@@ -150,7 +150,7 @@ VRT_AddDirector(VRT_CTX, const struct vdi_methods *m, void *priv,
 	struct vcl *vcl;
 	struct vcldir *vdir;
 	struct director *d;
-	const char *temp;
+	const struct vcltemp *temp;
 	va_list ap;
 	int i;
 
@@ -193,7 +193,7 @@ VRT_AddDirector(VRT_CTX, const struct vdi_methods *m, void *priv,
 	temp = vcl->temp;
 	if (temp != VCL_TEMP_COOLING)
 		VTAILQ_INSERT_TAIL(&vcl->director_list, vdir, list);
-	if (VCL_WARM(temp))
+	if (temp->is_warm)
 		VDI_Event(d, VCL_EVENT_WARM);
 	Lck_Unlock(&vcl_mtx);
 
@@ -201,7 +201,7 @@ VRT_AddDirector(VRT_CTX, const struct vdi_methods *m, void *priv,
 		deldirector(vdir);
 		return (NULL);
 	}
-	if (!VCL_WARM(temp) && temp != VCL_TEMP_INIT)
+	if (!temp->is_warm && temp != VCL_TEMP_INIT)
 		WRONG("Dynamic Backends can only be added to warm VCLs");
 
 	return (d);
@@ -222,7 +222,7 @@ VRT_DelDirector(VCL_BACKEND *bp)
 {
 	struct vcl *vcl;
 	struct vcldir *vdir;
-	const char *temp;
+	const struct vcltemp *temp;
 	VCL_BACKEND d;
 
 	TAKE_OBJ_NOTNULL(d, bp, DIRECTOR_MAGIC);
@@ -236,7 +236,7 @@ VRT_DelDirector(VCL_BACKEND *bp)
 	VTAILQ_REMOVE(&vcl->director_list, vdir, list);
 	Lck_Unlock(&vcl_mtx);
 
-	if (VCL_WARM(temp))
+	if (temp->is_warm)
 		VDI_Event(d, VCL_EVENT_COLD);
 	assert (d == vdir->dir);
 	deldirector(vdir);
@@ -374,7 +374,7 @@ VRT_VCL_Prevent_Discard(VRT_CTX, const char *desc)
 
 	vcl = ctx->vcl;
 	CHECK_OBJ_NOTNULL(vcl, VCL_MAGIC);
-	assert(VCL_WARM(vcl->temp));
+	assert(vcl->temp->is_warm);
 
 	ALLOC_OBJ(ref, VCLREF_MAGIC);
 	AN(ref);

From dridi.boukelmoune at gmail.com  Tue Nov 19 17:27:06 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Tue, 19 Nov 2019 17:27:06 +0000 (UTC)
Subject: [master] 34261afd8 Why-does-slink-hate-capitalization OCD
Message-ID: <20191119172706.C2408644B9@lists.varnish-cache.org>


commit 34261afd8466e99a898eb661d1f7190b95bae1f0
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Tue Nov 19 18:07:03 2019 +0100

    Why-does-slink-hate-capitalization OCD
    
    Is it because German capitalizes so many words that his shift keys have
    become hit or miss?

diff --git a/include/vrt.h b/include/vrt.h
index ad1d78d51..70a153bf3 100644
--- a/include/vrt.h
+++ b/include/vrt.h
@@ -188,9 +188,9 @@ struct strands {
 /*
  * VCL_BLOB:
  *
- * opaque, immutable data (pointer + length), minimum lifetime is the VCL task.
+ * Opaque, immutable data (pointer + length), minimum lifetime is the VCL task.
  *
- * type (optional) is owned by the creator of the blob. blob consumers may use
+ * Type (optional) is owned by the creator of the blob. blob consumers may use
  * it for checks, but should not assert on it.
  *
  * The data behind the blob pointer is assumed to be immutable for the blob's

From dridi.boukelmoune at gmail.com  Tue Nov 19 17:27:06 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Tue, 19 Nov 2019 17:27:06 +0000 (UTC)
Subject: [master] 11d55148a Put some serious red tape on VCL_STRANDS
Message-ID: <20191119172706.D6BD4644BE@lists.varnish-cache.org>


commit 11d55148abcaa64e843df0dceca6c7371c0a018a
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Tue Nov 19 18:09:27 2019 +0100

    Put some serious red tape on VCL_STRANDS
    
    I started suspecting that we need this clarification during the review
    of #3123 [1] and was able to verify it with a simple test case. First I
    needed a function I put in vmod_debug:
    
        $Function STRANDS hoard_strands(PRIV_TASK, STRANDS s)
    
        Return the first value of s for the rest of the task.
    
    The implementation is very straightforward:
    
        struct hoarder {
               VCL_STRANDS     s;
        };
    
        VCL_STRANDS
        xyzzy_hoard_strands(VRT_CTX, struct vmod_priv *priv, VCL_STRANDS s)
        {
               struct hoarder *h;
    
               CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
               AN(priv);
    
               if (priv->priv == NULL) {
                       h = malloc(sizeof *h);
                       AN(h);
                       h->s = s;
                       priv->priv = h;
                       priv->free = free;
               }
    
               return (priv->priv);
        }
    
    And then the following test case results in a panic on my system, but I
    suspect this is generally undefined behavior and other nasty results may
    occur under different circumstances:
    
        varnishtest "Beware of STRANDS"
    
        varnish v1 -vcl {
                import debug;
                backend be none;
                sub vcl_recv {
                        debug.hoard_strands("recv: " + req.url);
                }
                sub vcl_miss {
                        debug.hoard_strands("miss: " + req.url);
                        return (synth(200));
                }
                sub vcl_synth {
                        set resp.body = debug.hoard_strands("synth: " + req.url);
                        return (deliver);
                }
        } -start
    
        client c1 {
                txreq
                rxresp
                expect resp.body ~ recv
        } -run
    
    This also begs the following question: can it ever be safe to let a VMOD
    function return a STRANDS? Maybe it should be banned from return types.
    
    [1] https://github.com/varnishcache/varnish-cache/pull/3123#discussion_r345617108

diff --git a/include/vrt.h b/include/vrt.h
index 70a153bf3..863e60f65 100644
--- a/include/vrt.h
+++ b/include/vrt.h
@@ -180,6 +180,22 @@ struct vsl_log;
 struct ws;
 struct VSC_main;
 
+/*
+ * VCL_STRANDS:
+ *
+ * An argc+argv type of data structure where n indicates the number of strings
+ * in the p array. Individual components of a strands may be null.
+ *
+ * A STRANDS allows you to work on a strings concatenation with the option to
+ * collect it into a single STRING, or if possible work directly on individual
+ * parts.
+ *
+ * The memory management is very strict: a VMOD function receiving a STRANDS
+ * argument should keep no reference after the function returns. Retention of
+ * a STRANDS further in the ongoing task is undefined behavior and may result
+ * in a panic or data corruption.
+ */
+
 struct strands {
 	int		n;
 	const char	**p;

From dridi at varni.sh  Tue Nov 19 18:07:39 2019
From: dridi at varni.sh (Dridi Boukelmoune)
Date: Tue, 19 Nov 2019 18:07:39 +0000
Subject: [master] 11d55148a Put some serious red tape on VCL_STRANDS
In-Reply-To: <20191119172706.D6BD4644BE@lists.varnish-cache.org>
References: <20191119172706.D6BD4644BE@lists.varnish-cache.org>
Message-ID: <CABoVN9CKMdQ_CL+4O1JmaV1LGDWeNLV5d+pHR3tDX1U0bWhJ7g@mail.gmail.com>

>     This also begs the following question: can it ever be safe to let a VMOD
>     function return a STRANDS? Maybe it should be banned from return types.

I think the rules I laid out makes this fine in the scope of a single
VCL statement.

Dridi

From dridi.boukelmoune at gmail.com  Wed Nov 20 18:18:06 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Wed, 20 Nov 2019 18:18:06 +0000 (UTC)
Subject: [master] 7becfb8db Refactor the vcl state into a struct
Message-ID: <20191120181806.74AB09F5C@lists.varnish-cache.org>


commit 7becfb8dbe0679fb6bed0bced0019f489a410aa5
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Wed Nov 20 19:12:57 2019 +0100

    Refactor the vcl state into a struct
    
    Refs b0c0eaeaaa1aa156feeaba2e46d339b4212a5221

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 6b365cc56..3e3b98cac 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -51,7 +51,7 @@ const struct vcltemp VCL_TEMP_WARM[1] = {{ .name = "warm", .is_warm = 1 }};
 const struct vcltemp VCL_TEMP_BUSY[1] = {{ .name = "busy", .is_warm = 1 }};
 const struct vcltemp VCL_TEMP_COOLING[1] = {{ .name = "cooling" }};
 
-// not really a state
+// not really a temperature
 const struct vcltemp VCL_TEMP_LABEL[1] = {{ .name = "label" }};
 
 /*
diff --git a/bin/varnishd/mgt/mgt_vcl.c b/bin/varnishd/mgt/mgt_vcl.c
index 80404fee6..8478887ae 100644
--- a/bin/varnishd/mgt/mgt_vcl.c
+++ b/bin/varnishd/mgt/mgt_vcl.c
@@ -45,11 +45,15 @@
 #include "vev.h"
 #include "vtim.h"
 
-#define VCL_STATE(sym, str)					\
-	static const char * const VCL_STATE_ ## sym = str;
+struct vclstate {
+	const char		*name;
+};
+
+#define VCL_STATE(sym, str)						\
+	static const struct vclstate VCL_STATE_ ## sym[1] = {{ str }};
 #include "tbl/vcl_states.h"
 
-static const char * const VCL_STATE_LABEL = "label";
+static const struct vclstate VCL_STATE_LABEL[1] = {{ "label" }};
 
 static int vcl_count;
 
@@ -58,19 +62,20 @@ struct vmodfilehead vmodhead = VTAILQ_HEAD_INITIALIZER(vmodhead);
 static struct vclprog		*active_vcl;
 static struct vev *e_poker;
 
-static int mgt_vcl_setstate(struct cli *, struct vclprog *, const char *);
+static int mgt_vcl_setstate(struct cli *, struct vclprog *,
+    const struct vclstate *);
 static int mgt_vcl_settemp(struct cli *, struct vclprog *, unsigned);
 static int mgt_vcl_askchild(struct cli *, struct vclprog *, unsigned);
 static void mgt_vcl_set_cooldown(struct vclprog *, vtim_mono);
 
 /*--------------------------------------------------------------------*/
 
-static const char *
+static const struct vclstate *
 mcf_vcl_parse_state(struct cli *cli, const char *s)
 {
 	if (s != NULL) {
-#define VCL_STATE(sym, str)						\
-		if (!strcmp(s, VCL_STATE_ ## sym))			\
+#define VCL_STATE(sym, str)				\
+		if (!strcmp(s, str))			\
 			return (VCL_STATE_ ## sym);
 #include "tbl/vcl_states.h"
 	}
@@ -188,7 +193,7 @@ mgt_vcl_dep_del(struct vcldep *vd)
 /*--------------------------------------------------------------------*/
 
 static struct vclprog *
-mgt_vcl_add(const char *name, const char *state)
+mgt_vcl_add(const char *name, const struct vclstate *state)
 {
 	struct vclprog *vp;
 
@@ -347,7 +352,7 @@ mgt_vcl_askchild(struct cli *cli, struct vclprog *vp, unsigned warm)
 	}
 
 	i = mgt_cli_askchild(&status, &p, "vcl.state %s %d%s\n",
-	    vp->name, warm, vp->state);
+	    vp->name, warm, vp->state->name);
 	if (i && cli != NULL) {
 		VCLI_SetResult(cli, status);
 		VCLI_Out(cli, "%s", p);
@@ -355,7 +360,7 @@ mgt_vcl_askchild(struct cli *cli, struct vclprog *vp, unsigned warm)
 		MGT_Complain(C_ERR,
 		    "Please file ticket: VCL poker problem: "
 		    "'vcl.state %s %d%s' -> %03d '%s'",
-		    vp->name, warm, vp->state, i, p);
+		    vp->name, warm, vp->state->name, i, p);
 	} else {
 		/* Success, update mgt's VCL state to reflect child's
 		   state */
@@ -367,11 +372,11 @@ mgt_vcl_askchild(struct cli *cli, struct vclprog *vp, unsigned warm)
 }
 
 static int
-mgt_vcl_setstate(struct cli *cli, struct vclprog *vp, const char *vs)
+mgt_vcl_setstate(struct cli *cli, struct vclprog *vp, const struct vclstate *vs)
 {
 	unsigned warm;
 	int i;
-	const char *os;
+	const struct vclstate *os;
 
 	CHECK_OBJ_NOTNULL(vp, VCLPROG_MAGIC);
 
@@ -416,6 +421,7 @@ mgt_new_vcl(struct cli *cli, const char *vclname, const char *vclsrc,
 	unsigned status;
 	char *lib, *p;
 	struct vclprog *vp;
+	const struct vclstate *vs;
 	char buf[32];
 
 	AN(cli);
@@ -432,14 +438,14 @@ mgt_new_vcl(struct cli *cli, const char *vclname, const char *vclsrc,
 	}
 
 	if (state == NULL)
-		state = VCL_STATE_AUTO;
+		vs = VCL_STATE_AUTO;
 	else
-		state = mcf_vcl_parse_state(cli, state);
+		vs = mcf_vcl_parse_state(cli, state);
 
-	if (state == NULL)
+	if (vs == NULL)
 		return;
 
-	vp = mgt_vcl_add(vclname, state);
+	vp = mgt_vcl_add(vclname, vs);
 	lib = mgt_VccCompile(cli, vp, vclname, vclsrc, vclsrcfile, C_flag);
 	if (lib == NULL) {
 		mgt_vcl_del(vp);
@@ -464,7 +470,7 @@ mgt_new_vcl(struct cli *cli, const char *vclname, const char *vclsrc,
 		return;
 
 	if (mgt_cli_askchild(&status, &p, "vcl.load %s %s %d%s\n",
-	    vp->name, vp->fname, vp->warm, vp->state)) {
+	    vp->name, vp->fname, vp->warm, vp->state->name)) {
 		mgt_vcl_del(vp);
 		VCLI_Out(cli, "%s", p);
 		VCLI_SetResult(cli, status);
@@ -534,7 +540,8 @@ mgt_push_vcls(struct cli *cli, unsigned *status, char **p)
 			} else {
 				if (mgt_cli_askchild(status, p,
 				    "vcl.load \"%s\" %s %d%s\n",
-				    vp->name, vp->fname, vp->warm, vp->state))
+				    vp->name, vp->fname, vp->warm,
+				    vp->state->name))
 					return (1);
 			}
 			vp->loaded = 1;
@@ -579,7 +586,7 @@ mcf_vcl_load(struct cli *cli, const char * const *av, void *priv)
 static void v_matchproto_(cli_func_t)
 mcf_vcl_state(struct cli *cli, const char * const *av, void *priv)
 {
-	const char *state;
+	const struct vclstate *state;
 	struct vclprog *vp;
 
 	(void)priv;
@@ -707,6 +714,7 @@ mcf_vcl_list(struct cli *cli, const char * const *av, void *priv)
 	char *p;
 	struct vclprog *vp;
 	struct vcldep *vd;
+	const struct vclstate *vs;
 	struct vsb *vsb;
 
 	/* NB: Shall generate same output as vcl_cli_list() */
@@ -727,8 +735,8 @@ mcf_vcl_list(struct cli *cli, const char * const *av, void *priv)
 		VTAILQ_FOREACH(vp, &vclhead, list) {
 			VSB_printf(vsb, "%s",
 			    vp == active_vcl ? "active" : "available");
-			VSB_printf(vsb, "\t%s\t%s", vp->state, vp->warm ?
-			    VCL_STATE_WARM : VCL_STATE_COLD);
+			vs = vp->warm ?  VCL_STATE_WARM : VCL_STATE_COLD;
+			VSB_printf(vsb, "\t%s\t%s", vp->state->name, vs->name);
 			VSB_printf(vsb, "\t%6s\t%s", "-", vp->name);
 			if (mcf_is_label(vp)) {
 				vd = VTAILQ_FIRST(&vp->dfrom);
@@ -754,6 +762,7 @@ mcf_vcl_list_json(struct cli *cli, const char * const *av, void *priv)
 	char *p;
 	struct vclprog *vp;
 	struct vcldep *vd;
+	const struct vclstate *vs;
 
 	/* NB: Shall generate same output as vcl_cli_list() */
 
@@ -772,9 +781,9 @@ mcf_vcl_list_json(struct cli *cli, const char * const *av, void *priv)
 			VSB_indent(cli->sb, 2);
 			VCLI_Out(cli, "\"status\": \"%s\",\n",
 			    vp == active_vcl ? "active" : "available");
-			VCLI_Out(cli, "\"state\": \"%s\",\n", vp->state);
-			VCLI_Out(cli, "\"temperature\": \"%s\",\n",
-			    vp->warm ? VCL_STATE_WARM : VCL_STATE_COLD);
+			VCLI_Out(cli, "\"state\": \"%s\",\n", vp->state->name);
+			vs = vp->warm ?  VCL_STATE_WARM : VCL_STATE_COLD;
+			VCLI_Out(cli, "\"temperature\": \"%s\",\n", vs->name);
 			VCLI_Out(cli, "\"name\": \"%s\"", vp->name);
 			if (mcf_is_label(vp)) {
 				vd = VTAILQ_FIRST(&vp->dfrom);
diff --git a/bin/varnishd/mgt/mgt_vcl.h b/bin/varnishd/mgt/mgt_vcl.h
index e6a32fc8d..7f55c5a45 100644
--- a/bin/varnishd/mgt/mgt_vcl.h
+++ b/bin/varnishd/mgt/mgt_vcl.h
@@ -31,6 +31,7 @@
 struct vclprog;
 struct vmodfile;
 struct vjsn_val;
+struct vclstate;
 
 struct vmoddep {
 	unsigned		magic;
@@ -57,7 +58,7 @@ struct vclprog {
 	char			*name;
 	char			*fname;
 	unsigned		warm;
-	const char *		state;
+	const struct vclstate	*state;
 	double			go_cold;
 	struct vjsn		*symtab;
 	VTAILQ_HEAD(, vcldep)	dfrom;

From nils.goroll at uplex.de  Thu Nov 21 11:05:19 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu, 21 Nov 2019 12:05:19 +0100
Subject: [master] 11d55148a Put some serious red tape on VCL_STRANDS
In-Reply-To: <CABoVN9CKMdQ_CL+4O1JmaV1LGDWeNLV5d+pHR3tDX1U0bWhJ7g@mail.gmail.com>
References: <20191119172706.D6BD4644BE@lists.varnish-cache.org>
 <CABoVN9CKMdQ_CL+4O1JmaV1LGDWeNLV5d+pHR3tDX1U0bWhJ7g@mail.gmail.com>
Message-ID: <5384e539-228d-3b15-6085-6e091471d302@uplex.de>

On 19/11/2019 19:07, Dridi Boukelmoune wrote:
>>     This also begs the following question: can it ever be safe to let a VMOD
>>     function return a STRANDS? Maybe it should be banned from return types.
> 
> I think the rules I laid out makes this fine in the scope of a single
> VCL statement.

My understanding is that we should apply the same rule as for any other data
types in VCL: Ensure that they have at least TASK lifetime.

Then, the rules for keeping references to STRANDS become the same as for BLOBs:
If you want to keep a reference for longer than a TASK, you need to make a copy.

For that copy, I would think that the STRANDS should be collapsed and we might
want to add some convenience utilities.


-- 

** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstra?e 32
22301 Hamburg

tel +49 40 28805731
mob +49 170 2723133
fax +49 40 42949753

xmpp://slink at jabber.int.uplex.de/

http://uplex.de/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-commit/attachments/20191121/76362ad4/attachment.bin>

From dridi at varni.sh  Thu Nov 21 11:37:36 2019
From: dridi at varni.sh (Dridi Boukelmoune)
Date: Thu, 21 Nov 2019 11:37:36 +0000
Subject: [master] 11d55148a Put some serious red tape on VCL_STRANDS
In-Reply-To: <5384e539-228d-3b15-6085-6e091471d302@uplex.de>
References: <20191119172706.D6BD4644BE@lists.varnish-cache.org>
 <CABoVN9CKMdQ_CL+4O1JmaV1LGDWeNLV5d+pHR3tDX1U0bWhJ7g@mail.gmail.com>
 <5384e539-228d-3b15-6085-6e091471d302@uplex.de>
Message-ID: <CABoVN9AEdG1gtm=w1Gm2x91X6=AGDaePLRuBiJ45kVWp9zeD9g@mail.gmail.com>

On Thu, Nov 21, 2019 at 11:05 AM Nils Goroll <nils.goroll at uplex.de> wrote:
>
> On 19/11/2019 19:07, Dridi Boukelmoune wrote:
> >>     This also begs the following question: can it ever be safe to let a VMOD
> >>     function return a STRANDS? Maybe it should be banned from return types.
> >
> > I think the rules I laid out makes this fine in the scope of a single
> > VCL statement.
>
> My understanding is that we should apply the same rule as for any other data
> types in VCL: Ensure that they have at least TASK lifetime.
>
> Then, the rules for keeping references to STRANDS become the same as for BLOBs:
> If you want to keep a reference for longer than a TASK, you need to make a copy.
>
> For that copy, I would think that the STRANDS should be collapsed and we might
> want to add some convenience utilities.

I have no strong opinion on this, I'll happily amend the comment if
the expectations change but I believe the red tape is currently in
sync with reality. However, I'm a bit wary of making this kind of
implicit memory retention possible for strings, unlike blobs that will
require active vmod involvement.


Dridi

From dridi.boukelmoune at gmail.com  Fri Nov 22 14:17:06 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Fri, 22 Nov 2019 14:17:06 +0000 (UTC)
Subject: [master] 92ad90fe9 Simplify the vmod_blob codec interface
Message-ID: <20191122141706.29EA6A118B@lists.varnish-cache.org>


commit 92ad90fe989669764af74a976a122cf568eeac15
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Nov 22 14:45:24 2019 +0100

    Simplify the vmod_blob codec interface
    
    This is a mechanical change that hides the const and restrict details of
    the codec contract in vmod_blob and makes the function signatures more
    manageable.

diff --git a/lib/libvmod_blob/base64.c b/lib/libvmod_blob/base64.c
index f900cbb76..7162b2da5 100644
--- a/lib/libvmod_blob/base64.c
+++ b/lib/libvmod_blob/base64.c
@@ -55,8 +55,8 @@ base64_decode_l(size_t l)
 }
 
 static inline int
-decode(char *restrict *restrict dest, const char *restrict const buf,
-    const size_t buflen, unsigned u, const int n)
+decode(char *restrict *restrict dest, blob_src_t buf,
+    blob_len_t buflen, unsigned u, const int n)
 {
 	char *d;
 
@@ -79,8 +79,8 @@ decode(char *restrict *restrict dest, const char *restrict const buf,
 
 ssize_t
 base64_encode(const enum encoding enc, const enum case_e kase,
-    char *restrict const buf, const size_t buflen,
-    const char *restrict const inbuf, const size_t inlength)
+    blob_dest_t buf, blob_len_t buflen,
+    blob_src_t inbuf, blob_len_t inlength)
 {
 	const struct b64_alphabet *alpha = &b64_alphabet[enc];
 	char *p = buf;
@@ -130,8 +130,8 @@ base64_encode(const enum encoding enc, const enum case_e kase,
 }
 
 ssize_t
-base64_decode(const enum encoding dec, char *restrict const buf,
-    const size_t buflen, ssize_t inlen, VCL_STRANDS strings)
+base64_decode(const enum encoding dec, blob_dest_t buf,
+    blob_len_t buflen, ssize_t inlen, VCL_STRANDS strings)
 {
 	const struct b64_alphabet *alpha = &b64_alphabet[dec];
 	const char *s;
diff --git a/lib/libvmod_blob/hex.c b/lib/libvmod_blob/hex.c
index 6476308ae..91c6124b6 100644
--- a/lib/libvmod_blob/hex.c
+++ b/lib/libvmod_blob/hex.c
@@ -77,8 +77,8 @@ hex2byte(const unsigned char hi, const unsigned char lo)
 
 ssize_t
 hex_encode(const enum encoding enc, const enum case_e kase,
-    char *restrict const buf, const size_t buflen,
-    const char *restrict const in, const size_t inlen)
+    blob_dest_t buf, blob_len_t buflen,
+    blob_src_t in, blob_len_t inlen)
 {
 	char *p = buf;
 	const char *alphabet = hex_alphabet[0];
@@ -103,8 +103,8 @@ hex_encode(const enum encoding enc, const enum case_e kase,
 }
 
 ssize_t
-hex_decode(const enum encoding dec, char *restrict const buf,
-    const size_t buflen, ssize_t n, VCL_STRANDS strings)
+hex_decode(const enum encoding dec, blob_dest_t buf,
+    blob_len_t buflen, ssize_t n, VCL_STRANDS strings)
 {
 	char *dest = buf;
 	const char *b, *s;
diff --git a/lib/libvmod_blob/id.c b/lib/libvmod_blob/id.c
index cc5efab4b..3bca1b9ab 100644
--- a/lib/libvmod_blob/id.c
+++ b/lib/libvmod_blob/id.c
@@ -50,8 +50,8 @@ id_decode_l(size_t l)
 
 ssize_t
 id_encode(const enum encoding enc, const enum case_e kase,
-    char *restrict const buf, const size_t buflen,
-    const char *restrict const in, const size_t inlen)
+    blob_dest_t buf, blob_len_t buflen,
+    blob_src_t in, blob_len_t inlen)
 {
 	(void) enc;
 	(void) kase;
@@ -67,8 +67,8 @@ id_encode(const enum encoding enc, const enum case_e kase,
 }
 
 ssize_t
-id_decode(const enum encoding enc, char *restrict const buf,
-    const size_t buflen, ssize_t n, VCL_STRANDS strings)
+id_decode(const enum encoding enc, blob_dest_t buf,
+    blob_len_t buflen, ssize_t n, VCL_STRANDS strings)
 {
 	const char *s;
 	char *dest = buf;
diff --git a/lib/libvmod_blob/url.c b/lib/libvmod_blob/url.c
index 0134d3859..65d678879 100644
--- a/lib/libvmod_blob/url.c
+++ b/lib/libvmod_blob/url.c
@@ -80,8 +80,8 @@ isoutofrange(const uint8_t c)
 
 ssize_t
 url_encode(const enum encoding enc, const enum case_e kase,
-    char *restrict const buf, const size_t buflen,
-    const char *restrict const in, const size_t inlen)
+    blob_dest_t buf, blob_len_t buflen,
+    blob_src_t in, blob_len_t inlen)
 {
 	char *p = buf;
 	const char * const end = buf + buflen;
@@ -115,8 +115,8 @@ url_encode(const enum encoding enc, const enum case_e kase,
 }
 
 ssize_t
-url_decode(const enum encoding dec, char *restrict const buf,
-    const size_t buflen, ssize_t n, VCL_STRANDS strings)
+url_decode(const enum encoding dec, blob_dest_t buf,
+    blob_len_t buflen, ssize_t n, VCL_STRANDS strings)
 {
 	char *dest = buf;
 	const char * const end = buf + buflen;
diff --git a/lib/libvmod_blob/vmod_blob.h b/lib/libvmod_blob/vmod_blob.h
index ad9f0c1f7..77ae232dc 100644
--- a/lib/libvmod_blob/vmod_blob.h
+++ b/lib/libvmod_blob/vmod_blob.h
@@ -49,6 +49,11 @@ enum case_e {
 #include "tbl_case.h"
 };
 
+typedef const size_t			blob_len_t;
+typedef const ssize_t			blob_slen_t;
+typedef char *restrict const		blob_dest_t;
+typedef const char *restrict const	blob_src_t;
+
 /*
  * Length estimate interface
  */
@@ -82,8 +87,8 @@ size_t  len_f(size_t);
  */
 typedef
 ssize_t encode_f(const enum encoding enc, const enum case_e kase,
-		 char *restrict const buf, const size_t buflen,
-		 const char *restrict const in, const size_t inlen);
+		 blob_dest_t buf, blob_len_t buflen,
+		 blob_src_t in, blob_len_t inlen);
 
 /*
  * General interface for a decoder: decode the concatenation of strings
@@ -107,8 +112,8 @@ ssize_t encode_f(const enum encoding enc, const enum case_e kase,
  *    a static constant empty BLOB
  * otherwise, the number of bytes written
  */
-typedef ssize_t decode_f(const enum encoding dec, char *restrict const buf,
-    const size_t buflen, const ssize_t inlen, VCL_STRANDS strings);
+typedef ssize_t decode_f(const enum encoding dec, blob_dest_t buf,
+    blob_len_t buflen, blob_slen_t inlen, VCL_STRANDS strings);
 
 /* id.c */
 len_f	 id_encode_l;

From dridi.boukelmoune at gmail.com  Fri Nov 22 14:17:06 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Fri, 22 Nov 2019 14:17:06 +0000 (UTC)
Subject: [master] 1e1bef8fe Code style improvements in vmod_blob
Message-ID: <20191122141706.3F8C7A118E@lists.varnish-cache.org>


commit 1e1bef8fe025adc3e430f1d4777773fe1e83f671
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Nov 22 15:14:49 2019 +0100

    Code style improvements in vmod_blob
    
    Better diff with the --ignore-all-space option.

diff --git a/lib/libvmod_blob/base64.c b/lib/libvmod_blob/base64.c
index 7162b2da5..3c6eb852c 100644
--- a/lib/libvmod_blob/base64.c
+++ b/lib/libvmod_blob/base64.c
@@ -59,13 +59,14 @@ decode(char *restrict *restrict dest, blob_src_t buf,
     blob_len_t buflen, unsigned u, const int n)
 {
 	char *d;
+	int i;
 
 	if (n <= 1) {
 		errno = EINVAL;
 		return (-1);
 	}
 	d = *dest;
-	for (int i = 0; i < n - 1; i++) {
+	for (i = 0; i < n - 1; i++) {
 		if (d == buf + buflen) {
 			errno = ENOMEM;
 			return (-1);
@@ -87,16 +88,15 @@ base64_encode(const enum encoding enc, const enum case_e kase,
 	const uint8_t *in = (const uint8_t *)inbuf;
 	const uint8_t * const end = in + inlength;
 
-	(void) kase;
+	(void)kase;
 	AN(buf);
 	AN(alpha);
 	if (in == NULL || inlength == 0)
 		return (0);
 
 	if ((enc == BASE64URLNOPAD &&
-	     buflen < base64nopad_encode_l(inlength)) ||
-	    (enc != BASE64URLNOPAD &&
-	     buflen < base64_encode_l(inlength))) {
+	    buflen < base64nopad_encode_l(inlength)) ||
+	    (enc != BASE64URLNOPAD && buflen < base64_encode_l(inlength))) {
 		errno = ENOMEM;
 		return (-1);
 	}
diff --git a/lib/libvmod_blob/id.c b/lib/libvmod_blob/id.c
index 3bca1b9ab..3eb5ee28d 100644
--- a/lib/libvmod_blob/id.c
+++ b/lib/libvmod_blob/id.c
@@ -73,15 +73,16 @@ id_decode(const enum encoding enc, blob_dest_t buf,
 	const char *s;
 	char *dest = buf;
 	size_t len, outlen = 0, c = SIZE_MAX;
+	int i;
 
-	(void) enc;
+	(void)enc;
 	AN(buf);
 	AN(strings);
 
 	if (n >= 0)
 		c = n;
 
-	for (int i = 0; c > 0 && i < strings->n; i++) {
+	for (i = 0; c > 0 && i < strings->n; i++) {
 		s = strings->p[i];
 		if (s == NULL || *s == '\0')
 			continue;
diff --git a/lib/libvmod_blob/vmod_blob.c b/lib/libvmod_blob/vmod_blob.c
index 60c90d9ff..a68bf68bf 100644
--- a/lib/libvmod_blob/vmod_blob.c
+++ b/lib/libvmod_blob/vmod_blob.c
@@ -118,7 +118,7 @@ static const struct vrt_blob null_blob[1] = {{
 static enum encoding
 parse_encoding(VCL_ENUM e)
 {
-#define VMODENUM(n) if (e == VENUM(n)) return(n);
+#define VMODENUM(n) if (e == VENUM(n)) return (n);
 #include "tbl_encodings.h"
 	WRONG("illegal encoding enum");
 }
@@ -126,7 +126,7 @@ parse_encoding(VCL_ENUM e)
 static enum case_e
 parse_case(VCL_ENUM e)
 {
-#define VMODENUM(n) if (e == VENUM(n)) return(n);
+#define VMODENUM(n) if (e == VENUM(n)) return (n);
 #include "tbl_case.h"
 	WRONG("illegal case enum");
 }
@@ -171,7 +171,7 @@ encodes_hex(enum encoding enc)
 
 static inline int
 check_enc_case(VRT_CTX, VCL_ENUM encs, VCL_ENUM case_s, enum encoding enc,
-	       enum case_e kase)
+    enum case_e kase)
 {
 	if (!encodes_hex(enc) && kase != DEFAULT) {
 		VERR(ctx, "case %s is illegal with encoding %s", case_s, encs);
@@ -184,7 +184,7 @@ check_enc_case(VRT_CTX, VCL_ENUM encs, VCL_ENUM case_s, enum encoding enc,
 
 VCL_VOID v_matchproto_(td_blob_blob__init)
 vmod_blob__init(VRT_CTX, struct vmod_blob_blob **blobp, const char *vcl_name,
-		VCL_ENUM decs, VCL_STRANDS strings)
+    VCL_ENUM decs, VCL_STRANDS strings)
 {
 	struct vmod_blob_blob *b;
 	enum encoding dec = parse_encoding(decs);
@@ -246,7 +246,7 @@ vmod_blob_get(VRT_CTX, struct vmod_blob_blob *b)
 
 VCL_STRING v_matchproto_(td_blob_blob_encode)
 vmod_blob_encode(VRT_CTX, struct vmod_blob_blob *b, VCL_ENUM encs,
-		 VCL_ENUM case_s)
+    VCL_ENUM case_s)
 {
 	enum encoding enc = parse_encoding(encs);
 	AENC(enc);
@@ -256,9 +256,9 @@ vmod_blob_encode(VRT_CTX, struct vmod_blob_blob *b, VCL_ENUM encs,
 	CHECK_OBJ_NOTNULL(b, VMOD_BLOB_MAGIC);
 
 	if (!check_enc_case(ctx, encs, case_s, enc, kase))
-		return NULL;
+		return (NULL);
 	if (b->blob.len == 0)
-		return "";
+		return ("");
 	if (kase == DEFAULT)
 		kase = LOWER;
 
@@ -293,13 +293,15 @@ vmod_blob_encode(VRT_CTX, struct vmod_blob_blob *b, VCL_ENUM encs,
 		}
 		AZ(pthread_mutex_unlock(&b->lock));
 	}
-	return b->encoding[enc][kase];
+	return (b->encoding[enc][kase]);
 }
 
 VCL_VOID v_matchproto_(td_blob_blob__fini)
 vmod_blob__fini(struct vmod_blob_blob **blobp)
 {
 	struct vmod_blob_blob *b;
+	blob_dest_t s;
+	int i, j;
 
 	TAKE_OBJ_NOTNULL(b, blobp, VMOD_BLOB_MAGIC);
 
@@ -308,9 +310,9 @@ vmod_blob__fini(struct vmod_blob_blob **blobp)
 		b->blob.blob = NULL;
 	}
 
-	for (int i = 0; i < __MAX_ENCODING; i++)
-		for (int j = 0; j < 2; j++) {
-			char *s = b->encoding[i][j];
+	for (i = 0; i < __MAX_ENCODING; i++)
+		for (j = 0; j < 2; j++) {
+			s = b->encoding[i][j];
 			if (s != NULL && s != empty) {
 				free(s);
 				b->encoding[i][j] = NULL;
@@ -347,11 +349,11 @@ vmod_decode(VRT_CTX, VCL_ENUM decs, VCL_INT length, VCL_STRANDS strings)
 	if (len == -1) {
 		err_decode(ctx, strings->p[0]);
 		WS_Release(ctx->ws, 0);
-		return NULL;
+		return (NULL);
 	}
 	if (len == 0) {
 		WS_Release(ctx->ws, 0);
-		return null_blob;
+		return (null_blob);
 	}
 	WS_Release(ctx->ws, len);
 
@@ -404,8 +406,8 @@ vmod_encode(VRT_CTX, VCL_ENUM encs, VCL_ENUM case_s, VCL_BLOB b)
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	if (!check_enc_case(ctx, encs, case_s, enc, kase))
-		return NULL;
-	return encode(ctx, enc, kase, b);
+		return (NULL);
+	return (encode(ctx, enc, kase, b));
 }
 
 VCL_STRING v_matchproto_(td_blob_transcode)
@@ -417,6 +419,7 @@ vmod_transcode(VRT_CTX, VCL_ENUM decs, VCL_ENUM encs, VCL_ENUM case_s,
 	enum case_e kase = parse_case(case_s);
 	struct vrt_blob b;
 	VCL_STRING r;
+	size_t l;
 
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	CHECK_OBJ_NOTNULL(ctx->ws, WS_MAGIC);
@@ -426,15 +429,15 @@ vmod_transcode(VRT_CTX, VCL_ENUM decs, VCL_ENUM encs, VCL_ENUM case_s,
 	AENC(enc);
 
 	if (!check_enc_case(ctx, encs, case_s, enc, kase))
-		return NULL;
+		return (NULL);
 
 	/*
 	 * Allocate space for the decoded blob on the stack
 	 * ignoring the limitation imposed by n
 	 */
-	size_t l = decode_l(dec, strings);
+	l = decode_l(dec, strings);
 	if (l == 0)
-		return "";
+		return ("");
 
 	/* XXX: handle stack overflow? */
 	char buf[l];
@@ -447,7 +450,7 @@ vmod_transcode(VRT_CTX, VCL_ENUM decs, VCL_ENUM encs, VCL_ENUM case_s,
 
 	if (b.len == -1) {
 		err_decode(ctx, strings->p[0]);
-		return NULL;
+		return (NULL);
 	}
 
 	/*
@@ -472,66 +475,67 @@ vmod_transcode(VRT_CTX, VCL_ENUM decs, VCL_ENUM encs, VCL_ENUM case_s,
 VCL_BOOL v_matchproto_(td_blob_same)
 vmod_same(VRT_CTX, VCL_BLOB b1, VCL_BLOB b2)
 {
-	(void) ctx;
 
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	if (b1 == b2)
-		return 1;
+		return (1);
 	if (b1 == NULL || b2 == NULL)
-		return 0;
+		return (0);
 	return (b1->len == b2->len && b1->blob == b2->blob);
 }
 
 VCL_BOOL v_matchproto_(td_blob_equal)
 vmod_equal(VRT_CTX, VCL_BLOB b1, VCL_BLOB b2)
 {
-	(void) ctx;
 
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	if (b1 == b2)
-		return 1;
+		return (1);
 	if (b1 == NULL || b2 == NULL)
-		return 0;
+		return (0);
 	if (b1->len != b2->len)
-		return 0;
+		return (0);
 	if (b1->blob == b2->blob)
-		return 1;
+		return (1);
 	if (b1->blob == NULL || b2->blob == NULL)
-		return 0;
+		return (0);
 	return (memcmp(b1->blob, b2->blob, b1->len) == 0);
 }
 
 VCL_INT v_matchproto_(td_blob_length)
 vmod_length(VRT_CTX, VCL_BLOB b)
 {
-	(void) ctx;
 
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	if (b == NULL)
-		return 0;
-	return b->len;
+		return (0);
+	return (b->len);
 }
 
 VCL_BLOB v_matchproto_(td_blob_sub)
 vmod_sub(VRT_CTX, VCL_BLOB b, VCL_BYTES n, VCL_BYTES off)
 {
+
 	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
 	assert(n >= 0);
 	assert(off >= 0);
 
 	if (b == NULL || b->len == 0 || b->blob == NULL) {
 		ERR(ctx, "blob is empty in blob.sub()");
-		return NULL;
+		return (NULL);
 	}
 
 	assert(b->len > 0);
 
 	if (off + n > b->len) {
 		VERR(ctx, "size %jd from offset %jd requires more bytes than "
-		     "blob length %zd in blob.sub()",
-		     (intmax_t)n, (intmax_t)off, b->len);
-		return NULL;
+		    "blob length %zd in blob.sub()",
+		    (intmax_t)n, (intmax_t)off, b->len);
+		return (NULL);
 	}
 
 	if (n == 0)
-		return null_blob;
+		return (null_blob);
 
 
 	return (VRT_blob(ctx, "blob.sub",
diff --git a/lib/libvmod_blob/vmod_blob.h b/lib/libvmod_blob/vmod_blob.h
index 77ae232dc..5fd560816 100644
--- a/lib/libvmod_blob/vmod_blob.h
+++ b/lib/libvmod_blob/vmod_blob.h
@@ -57,8 +57,7 @@ typedef const char *restrict const	blob_src_t;
 /*
  * Length estimate interface
  */
-typedef
-size_t  len_f(size_t);
+typedef size_t len_f(size_t);
 
 /*
  * General interface for an encoder: encode the data at in of length inlen
@@ -116,30 +115,26 @@ typedef ssize_t decode_f(const enum encoding dec, blob_dest_t buf,
     blob_len_t buflen, blob_slen_t inlen, VCL_STRANDS strings);
 
 /* id.c */
-len_f	 id_encode_l;
-len_f	 id_decode_l;
-
-encode_f id_encode;
-decode_f id_decode;
+len_f		id_encode_l;
+len_f		id_decode_l;
+encode_f	id_encode;
+decode_f	id_decode;
 
 /* base64.c */
-len_f	 base64_decode_l;
-len_f	 base64nopad_encode_l;
-len_f	 base64_encode_l;
-
-encode_f base64_encode;
-decode_f base64_decode;
+len_f		base64_decode_l;
+len_f		base64nopad_encode_l;
+len_f		base64_encode_l;
+encode_f	base64_encode;
+decode_f	base64_decode;
 
  /* hex.c */
-len_f	 hex_encode_l;
-len_f	 hex_decode_l;
-
-encode_f hex_encode;
-decode_f hex_decode;
+len_f		hex_encode_l;
+len_f		hex_decode_l;
+encode_f	hex_encode;
+decode_f	hex_decode;
 
 /* url.c */
-len_f	 url_encode_l;
-len_f	 url_decode_l;
-
-encode_f url_encode;
-decode_f url_decode;
+len_f		url_encode_l;
+len_f		url_decode_l;
+encode_f	url_encode;
+decode_f	url_decode;

From dridi.boukelmoune at gmail.com  Fri Nov 22 14:19:05 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Fri, 22 Nov 2019 14:19:05 +0000 (UTC)
Subject: [master] 8caa56754 Fix mishap from the previous commit
Message-ID: <20191122141905.E5883A157C@lists.varnish-cache.org>


commit 8caa567546d0f5799f735a5629cc9bd95edb0bcc
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Nov 22 15:17:41 2019 +0100

    Fix mishap from the previous commit

diff --git a/lib/libvmod_blob/vmod_blob.c b/lib/libvmod_blob/vmod_blob.c
index a68bf68bf..0092686e9 100644
--- a/lib/libvmod_blob/vmod_blob.c
+++ b/lib/libvmod_blob/vmod_blob.c
@@ -300,7 +300,7 @@ VCL_VOID v_matchproto_(td_blob_blob__fini)
 vmod_blob__fini(struct vmod_blob_blob **blobp)
 {
 	struct vmod_blob_blob *b;
-	blob_dest_t s;
+	char *s;
 	int i, j;
 
 	TAKE_OBJ_NOTNULL(b, blobp, VMOD_BLOB_MAGIC);

From dridi.boukelmoune at gmail.com  Fri Nov 22 14:28:05 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Fri, 22 Nov 2019 14:28:05 +0000 (UTC)
Subject: [master] 1b1e7892e Kill base64.h from vmod_blob
Message-ID: <20191122142805.C2AB7A1A24@lists.varnish-cache.org>


commit 1b1e7892ecb00ddf35b305c439de8f3a5aeb109c
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Nov 22 15:24:23 2019 +0100

    Kill base64.h from vmod_blob
    
    It's only ever used by base64.c, so we might as well put it there.

diff --git a/lib/libvmod_blob/Makefile.am b/lib/libvmod_blob/Makefile.am
index fafce9acd..040b1aceb 100644
--- a/lib/libvmod_blob/Makefile.am
+++ b/lib/libvmod_blob/Makefile.am
@@ -4,7 +4,6 @@ libvmod_blob_la_SOURCES = \
 	vmod_blob.c \
 	vmod_blob.h \
 	id.c \
-	base64.h \
 	base64.c \
 	hex.h \
 	hex.c \
@@ -12,7 +11,5 @@ libvmod_blob_la_SOURCES = \
 	tbl_encodings.h \
 	tbl_case.h
 
-base64.o: base64.c base64.h
-
 # Use vmodtool.py generated automake boilerplate
 include $(srcdir)/automake_boilerplate.am
diff --git a/lib/libvmod_blob/base64.c b/lib/libvmod_blob/base64.c
index 3c6eb852c..63f45a8c8 100644
--- a/lib/libvmod_blob/base64.c
+++ b/lib/libvmod_blob/base64.c
@@ -32,8 +32,134 @@
 #include "vrt.h"
 #include "vas.h"
 
-#include "base64.h"
+#include "vmod_blob.h"
 
+#define ILL ((int8_t) 127)
+#define PAD ((int8_t) 126)
+
+static const struct b64_alphabet {
+	const char b64[64];
+	const int8_t i64[256];
+	const int padding;
+} b64_alphabet[] = {
+	[BASE64] = {
+		"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdef"
+		"ghijklmnopqrstuvwxyz0123456789+/",
+		{
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL,  62, ILL, ILL, ILL,  63, /* +, -    */
+			 52,  53,  54,  55,  56,  57,  58,  59, /* 0 - 7   */
+			 60,  61, ILL, ILL, ILL, PAD, ILL, ILL, /* 8, 9, = */
+			ILL,   0,   1,   2,   3,   4,   5,   6, /* A - G   */
+			  7,   8,   9,  10,  11,  12,  13,  14, /* H - O   */
+			 15,  16,  17,  18,  19,  20,  21,  22, /* P - W   */
+			 23,  24,  25, ILL, ILL, ILL, ILL, ILL, /* X, Y, Z */
+			ILL,  26,  27,  28,  29,  30,  31,  32, /* a - g   */
+			 33,  34,  35,  36,  37,  38,  39,  40, /* h - o   */
+			 41,  42,  43,  44,  45,  46,  47,  48, /* p - w   */
+			 49,  50,  51, ILL, ILL, ILL, ILL, ILL, /* x, y, z */
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+		},
+		'='
+	},
+	[BASE64URL] = {
+		"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdef"
+		"ghijklmnopqrstuvwxyz0123456789-_",
+		{
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL,  62, ILL, ILL, /* -       */
+			 52,  53,  54,  55,  56,  57,  58,  59, /* 0 - 7   */
+			 60,  61, ILL, ILL, ILL, PAD, ILL, ILL, /* 8, 9, = */
+			ILL,   0,   1,   2,   3,   4,   5,   6, /* A - G   */
+			  7,   8,   9,  10,  11,  12,  13,  14, /* H - O   */
+			 15,  16,  17,  18,  19,  20,  21,  22, /* P - W   */
+			 23,  24,  25, ILL, ILL, ILL, ILL,  63, /* X-Z, _  */
+			ILL,  26,  27,  28,  29,  30,  31,  32, /* a - g   */
+			 33,  34,  35,  36,  37,  38,  39,  40, /* h - o   */
+			 41,  42,  43,  44,  45,  46,  47,  48, /* p - w   */
+			 49,  50,  51, ILL, ILL, ILL, ILL, ILL, /* x, y, z */
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+		},
+		'='
+	},
+	[BASE64URLNOPAD] = {
+		"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdef"
+		"ghijklmnopqrstuvwxyz0123456789-_",
+		{
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL,  62, ILL, ILL, /* -       */
+			 52,  53,  54,  55,  56,  57,  58,  59, /* 0 - 7   */
+			 60,  61, ILL, ILL, ILL, ILL, ILL, ILL, /* 8, 9    */
+			ILL,   0,   1,   2,   3,   4,   5,   6, /* A - G   */
+			  7,   8,   9,  10,  11,  12,  13,  14, /* H - O   */
+			 15,  16,  17,  18,  19,  20,  21,  22, /* P - W   */
+			 23,  24,  25, ILL, ILL, ILL, ILL,  63, /* X-Z, _  */
+			ILL,  26,  27,  28,  29,  30,  31,  32, /* a - g   */
+			 33,  34,  35,  36,  37,  38,  39,  40, /* h - o   */
+			 41,  42,  43,  44,  45,  46,  47,  48, /* p - w   */
+			 49,  50,  51, ILL, ILL, ILL, ILL, ILL, /* x, y, z */
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
+		},
+		0
+	},
+};
 #define base64_l(l)		(((l) << 2) / 3)
 
 size_t
diff --git a/lib/libvmod_blob/base64.h b/lib/libvmod_blob/base64.h
deleted file mode 100644
index 962ae22cd..000000000
--- a/lib/libvmod_blob/base64.h
+++ /dev/null
@@ -1,156 +0,0 @@
-/*-
- * Copyright 2015-2016 UPLEX - Nils Goroll Systemoptimierung
- * All rights reserved.
- *
- * Authors: Nils Goroll <nils.goroll at uplex.de>
- *          Geoffrey Simmons <geoffrey.simmons at uplex.de>
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- * 1. Redistributions of source code must retain the above copyright notice,
- *    this list of conditions and the following disclaimer.
- * 2. Redistributions in binary form must reproduce the above copyright notice,
- *    this list of conditions and the following disclaimer in the documentation
- *    and/or other materials provided with the distribution.
- *
- * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND ANY
- * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
- * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
- * DISCLAIMED.  IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE FOR ANY
- * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
- * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
- * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
- * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
- * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- *
- */
-
-#include "vmod_blob.h"
-
-#define ILL ((int8_t) 127)
-#define PAD ((int8_t) 126)
-
-static const struct b64_alphabet {
-	const char b64[64];
-	const int8_t i64[256];
-	const int padding;
-} b64_alphabet[] = {
-	[BASE64] = {
-		"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdef"
-		"ghijklmnopqrstuvwxyz0123456789+/",
-		{
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL,  62, ILL, ILL, ILL,  63, /* +, -    */
-			 52,  53,  54,  55,  56,  57,  58,  59, /* 0 - 7   */
-			 60,  61, ILL, ILL, ILL, PAD, ILL, ILL, /* 8, 9, = */
-			ILL,   0,   1,   2,   3,   4,   5,   6, /* A - G   */
-			  7,   8,   9,  10,  11,  12,  13,  14, /* H - O   */
-			 15,  16,  17,  18,  19,  20,  21,  22, /* P - W   */
-			 23,  24,  25, ILL, ILL, ILL, ILL, ILL, /* X, Y, Z */
-			ILL,  26,  27,  28,  29,  30,  31,  32, /* a - g   */
-			 33,  34,  35,  36,  37,  38,  39,  40, /* h - o   */
-			 41,  42,  43,  44,  45,  46,  47,  48, /* p - w   */
-			 49,  50,  51, ILL, ILL, ILL, ILL, ILL, /* x, y, z */
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-		},
-		'='
-	},
-	[BASE64URL] = {
-		"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdef"
-		"ghijklmnopqrstuvwxyz0123456789-_",
-		{
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL,  62, ILL, ILL, /* -       */
-			 52,  53,  54,  55,  56,  57,  58,  59, /* 0 - 7   */
-			 60,  61, ILL, ILL, ILL, PAD, ILL, ILL, /* 8, 9, = */
-			ILL,   0,   1,   2,   3,   4,   5,   6, /* A - G   */
-			  7,   8,   9,  10,  11,  12,  13,  14, /* H - O   */
-			 15,  16,  17,  18,  19,  20,  21,  22, /* P - W   */
-			 23,  24,  25, ILL, ILL, ILL, ILL,  63, /* X-Z, _  */
-			ILL,  26,  27,  28,  29,  30,  31,  32, /* a - g   */
-			 33,  34,  35,  36,  37,  38,  39,  40, /* h - o   */
-			 41,  42,  43,  44,  45,  46,  47,  48, /* p - w   */
-			 49,  50,  51, ILL, ILL, ILL, ILL, ILL, /* x, y, z */
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-		},
-		'='
-	},
-	[BASE64URLNOPAD] = {
-		"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdef"
-		"ghijklmnopqrstuvwxyz0123456789-_",
-		{
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL,  62, ILL, ILL, /* -       */
-			 52,  53,  54,  55,  56,  57,  58,  59, /* 0 - 7   */
-			 60,  61, ILL, ILL, ILL, ILL, ILL, ILL, /* 8, 9    */
-			ILL,   0,   1,   2,   3,   4,   5,   6, /* A - G   */
-			  7,   8,   9,  10,  11,  12,  13,  14, /* H - O   */
-			 15,  16,  17,  18,  19,  20,  21,  22, /* P - W   */
-			 23,  24,  25, ILL, ILL, ILL, ILL,  63, /* X-Z, _  */
-			ILL,  26,  27,  28,  29,  30,  31,  32, /* a - g   */
-			 33,  34,  35,  36,  37,  38,  39,  40, /* h - o   */
-			 41,  42,  43,  44,  45,  46,  47,  48, /* p - w   */
-			 49,  50,  51, ILL, ILL, ILL, ILL, ILL, /* x, y, z */
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-			ILL, ILL, ILL, ILL, ILL, ILL, ILL, ILL,
-		},
-		0
-	},
-};

From dridi.boukelmoune at gmail.com  Fri Nov 22 14:34:06 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Fri, 22 Nov 2019 14:34:06 +0000 (UTC)
Subject: [master] 4e568f8fc Polish
Message-ID: <20191122143406.6CD92A1DE6@lists.varnish-cache.org>


commit 4e568f8fc49cb80797d98a8f0baf097a27b5c1f6
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Nov 22 15:33:37 2019 +0100

    Polish

diff --git a/lib/libvmod_blob/base64.c b/lib/libvmod_blob/base64.c
index 63f45a8c8..45987c2ae 100644
--- a/lib/libvmod_blob/base64.c
+++ b/lib/libvmod_blob/base64.c
@@ -177,7 +177,7 @@ base64_encode_l(size_t l)
 size_t
 base64_decode_l(size_t l)
 {
-	return (((l) * 3) >> 2);
+	return ((l * 3) >> 2);
 }
 
 static inline int
diff --git a/lib/libvmod_blob/hex.c b/lib/libvmod_blob/hex.c
index 91c6124b6..2fe05a422 100644
--- a/lib/libvmod_blob/hex.c
+++ b/lib/libvmod_blob/hex.c
@@ -60,13 +60,13 @@ const uint8_t nibble[] = {
 size_t
 hex_encode_l(size_t l)
 {
-	return (((l) << 1) + 1);
+	return ((l << 1) + 1);
 }
 
 size_t
 hex_decode_l(size_t l)
 {
-	return (((l) + 1) >> 1);
+	return ((l + 1) >> 1);
 }
 
 static inline char

From phk at FreeBSD.org  Mon Nov 25 09:18:08 2019
From: phk at FreeBSD.org (Poul-Henning Kamp)
Date: Mon, 25 Nov 2019 09:18:08 +0000 (UTC)
Subject: [master] 7c89c9688 Monday morning flexelinting & whitespace OCD
Message-ID: <20191125091808.A7138112305@lists.varnish-cache.org>


commit 7c89c96880ec445e673a3b31a69d4414ebf8432e
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Nov 25 09:17:11 2019 +0000

    Monday morning flexelinting & whitespace OCD

diff --git a/bin/varnishd/cache/cache_vcl.c b/bin/varnishd/cache/cache_vcl.c
index 3e3b98cac..4b8a0bb0d 100644
--- a/bin/varnishd/cache/cache_vcl.c
+++ b/bin/varnishd/cache/cache_vcl.c
@@ -52,7 +52,7 @@ const struct vcltemp VCL_TEMP_BUSY[1] = {{ .name = "busy", .is_warm = 1 }};
 const struct vcltemp VCL_TEMP_COOLING[1] = {{ .name = "cooling" }};
 
 // not really a temperature
-const struct vcltemp VCL_TEMP_LABEL[1] = {{ .name = "label" }};
+static const struct vcltemp VCL_TEMP_LABEL[1] = {{ .name = "label" }};
 
 /*
  * XXX: Presently all modifications to this list happen from the
diff --git a/bin/varnishd/cache/cache_vcl.h b/bin/varnishd/cache/cache_vcl.h
index e6ac097c1..2c4c18016 100644
--- a/bin/varnishd/cache/cache_vcl.h
+++ b/bin/varnishd/cache/cache_vcl.h
@@ -32,7 +32,7 @@
  */
 
 struct vfilter;
-struct vcltemp;;
+struct vcltemp;
 
 VTAILQ_HEAD(vfilter_head, vfilter);
 
diff --git a/bin/varnishd/flint.lnt b/bin/varnishd/flint.lnt
index 74fb46180..3d682d52f 100644
--- a/bin/varnishd/flint.lnt
+++ b/bin/varnishd/flint.lnt
@@ -146,6 +146,8 @@
 
 // cache_vcl.c
 -esym(528, vcl_call_method)	// Not referenced
+-esym(765, VCL_TEMP_*)
+-esym(759, VCL_TEMP_*)
 
 -e441	//  for clause irregularity: loop variable '___' not found in 2nd for expression
 
diff --git a/bin/varnishtest/tests/r02872.vtc b/bin/varnishtest/tests/r02872.vtc
index e7d1174fe..ac483a906 100644
--- a/bin/varnishtest/tests/r02872.vtc
+++ b/bin/varnishtest/tests/r02872.vtc
@@ -1,23 +1,22 @@
 varnishtest "VSL quoted fields"
 
 server s1 {
-        rxreq
-        txresp
+	rxreq
+	txresp
 } -start
 
 varnish v1 -vcl {
 	import std;
-        backend be {
-                .host = "${s1_sock}";
-                .probe = { .interval = 1m; }
-        }
+	backend be {
+		.host = "${s1_sock}";
+		.probe = { .interval = 1m; }
+	}
 	sub vcl_recv {
 		# a series of 3-fields log records
 		std.log({"  custom   log     "ok" "});
 		std.log({" "valid"  "fields"  ok "});
 		std.log({" "missing""blank"   ko "});
-		std.log({"  missing  dquote  "ko "});
-		# "
+		std.log({"  missing  dquote  "ko "});	# "
 		return (synth(200));
 	}
 } -start
@@ -30,17 +29,17 @@ client c1 {
 
 # records with malformed fields don't show up
 shell -expect "2" {
-        varnishlog -d -n ${v1_name} -g raw -q 'VCL_Log[3]' | wc -l
+	 varnishlog -d -n ${v1_name} -g raw -q 'VCL_Log[3]' | wc -l
 }
 
 server s1 -wait
 
 shell -expect "Went healthy" {
-        varnishlog -d -n ${v1_name} -g raw \
-		-q 'Backend_health[10] eq "HTTP/1.1 200 OK"'
+	varnishlog -d -n ${v1_name} -g raw \
+	    -q 'Backend_health[10] eq "HTTP/1.1 200 OK"'
 }
 
 # s1 starts sick before the first probe request is made
 shell -expect "Went sick" {
-        varnishlog -d -n ${v1_name} -g raw -q 'Backend_health[10] eq ""'
+	varnishlog -d -n ${v1_name} -g raw -q 'Backend_health[10] eq ""'
 }
diff --git a/include/tbl/sess_attr.h b/include/tbl/sess_attr.h
index e61365c4d..7702e463a 100644
--- a/include/tbl/sess_attr.h
+++ b/include/tbl/sess_attr.h
@@ -32,8 +32,8 @@
 
 /*lint -save -e525 -e539 */
 
-//        upper           lower         type		    len
-SESS_ATTR(TRANSPORT,	  transport,	void,    	    0)
+//	  upper		  lower		type		    len
+SESS_ATTR(TRANSPORT,	  transport,	void,		    0)
 SESS_ATTR(REMOTE_ADDR,	  remote_addr,	struct suckaddr,    vsa_suckaddr_len)
 SESS_ATTR(LOCAL_ADDR,	  local_addr,	struct suckaddr,    vsa_suckaddr_len)
 SESS_ATTR(CLIENT_ADDR,	  client_addr,	struct suckaddr,    vsa_suckaddr_len)
diff --git a/lib/libvmod_vtc/vmod_vtc.c b/lib/libvmod_vtc/vmod_vtc.c
index 9370a403e..f7e58db5c 100644
--- a/lib/libvmod_vtc/vmod_vtc.c
+++ b/lib/libvmod_vtc/vmod_vtc.c
@@ -297,6 +297,7 @@ vmod_typesize(VRT_CTX, VCL_STRING s)
 			i += (l - a); /* align */
 		i += l;
 	}
+	AN(p);
 	a = i % p;
 	if (a != 0)
 		i += (p - a); /* pad */

From nils.goroll at uplex.de  Tue Nov 26 12:44:06 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 26 Nov 2019 12:44:06 +0000 (UTC)
Subject: [master] ed3b095c0 add a facility to test WS_ReserveSize()
Message-ID: <20191126124406.A5B3210F125@lists.varnish-cache.org>


commit ed3b095c0414af453493580a654f63a02d22e0e8
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue Nov 26 13:21:37 2019 +0100

    add a facility to test WS_ReserveSize()

diff --git a/lib/libvmod_vtc/vmod.vcc b/lib/libvmod_vtc/vmod.vcc
index 51154f366..4f15d0d07 100644
--- a/lib/libvmod_vtc/vmod.vcc
+++ b/lib/libvmod_vtc/vmod.vcc
@@ -96,6 +96,14 @@ as much as needed to leave that many bytes free. The actual allocation size
 may be higher to comply with memory alignment requirements of the CPU
 architecture. A failed allocation fails the transaction.
 
+$Function BYTES workspace_reserve(ENUM { client, backend, session, thread },
+	INT size)
+
+Attempt to reserve *size* bytes and release the reservation right
+away. Return the size of the reservation.
+
+See `vtc.workspace_alloc`_ for semantics of the *size* argument.
+
 $Function INT workspace_free(ENUM { client, backend, session, thread })
 
 Find how much unallocated space there is left in a workspace.
diff --git a/lib/libvmod_vtc/vmod_vtc.c b/lib/libvmod_vtc/vmod_vtc.c
index f7e58db5c..e52c40d5a 100644
--- a/lib/libvmod_vtc/vmod_vtc.c
+++ b/lib/libvmod_vtc/vmod_vtc.c
@@ -167,6 +167,34 @@ vmod_workspace_alloc(VRT_CTX, VCL_ENUM which, VCL_INT size)
 		memset(p, '\0', size);
 }
 
+VCL_BYTES v_matchproto_(td_vtc_workspace_reserve)
+vmod_workspace_reserve(VRT_CTX, VCL_ENUM which, VCL_INT size)
+{
+	struct ws *ws;
+	unsigned r;
+
+	CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
+
+	ws = vtc_ws_find(ctx, which);
+	if (ws == NULL)
+		return (0);
+	WS_Assert(ws);
+
+	if (size < 0) {
+		size += WS_ReserveAll(ws);
+		WS_Release(ws, 0);
+	}
+	if (size <= 0) {
+		VRT_fail(ctx, "Attempted negative WS reservation");
+		return (0);
+	}
+	r = WS_ReserveSize(ws, size);
+	if (r == 0)
+		return (0);
+	WS_Release(ws, 0);
+	return (1);
+}
+
 VCL_INT v_matchproto_(td_vtc_workspace_free)
 vmod_workspace_free(VRT_CTX, VCL_ENUM which)
 {

From nils.goroll at uplex.de  Tue Nov 26 12:44:06 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 26 Nov 2019 12:44:06 +0000 (UTC)
Subject: [master] 505b7bd96 WS_ReserveSize() must not hold a reservation for
 zero return value
Message-ID: <20191126124406.BC9AA10F128@lists.varnish-cache.org>


commit 505b7bd9643006fa8e3977f920564ce12a2b24a2
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue Nov 26 13:27:09 2019 +0100

    WS_ReserveSize() must not hold a reservation for zero return value
    
    This originates from a3d47c258fb7938f67a053f6d041257edb69afe9, but
    was overlooked in 4e33359772a3d751b2ef4b5c4b40259f1bcd6903:
    
    When there is insufficient space to fulfil the reservation request, we
    must not leave the workspace reserved.
    
    Fixes #3131

diff --git a/bin/varnishd/cache/cache_ws.c b/bin/varnishd/cache/cache_ws.c
index 7b1b6d8f8..a40bdbc1b 100644
--- a/bin/varnishd/cache/cache_ws.c
+++ b/bin/varnishd/cache/cache_ws.c
@@ -261,7 +261,7 @@ WS_ReserveSize(struct ws *ws, unsigned bytes)
 	if (bytes < b2)
 		b2 = PRNDUP(bytes);
 
-	if (ws->f + b2 > ws->e) {
+	if (ws->f + b2 >= ws->e) {
 		WS_MarkOverflow(ws);
 		return (0);
 	}
diff --git a/bin/varnishtest/tests/r03131.vtc b/bin/varnishtest/tests/r03131.vtc
new file mode 100644
index 000000000..8834e50f7
--- /dev/null
+++ b/bin/varnishtest/tests/r03131.vtc
@@ -0,0 +1,24 @@
+varnishtest "Test workspace functions in vmod_vtc"
+
+varnish v1 -vcl {
+	import vtc;
+	import std;
+
+	backend dummy None;
+
+	sub vcl_recv {
+		return (synth(200));
+	}
+
+	sub vcl_synth {
+		set resp.http.res1 = vtc.workspace_reserve(client, 1024 * 1024);
+		vtc.workspace_alloc(client, -1);
+		set resp.http.res2 = vtc.workspace_reserve(client, 8);
+		set resp.http.res3 = vtc.workspace_reserve(client, 8);
+	}
+} -start
+
+client c1 {
+	txreq
+	rxresp
+} -run
diff --git a/lib/libvmod_vtc/vmod.vcc b/lib/libvmod_vtc/vmod.vcc
index 4f15d0d07..d111f9eb6 100644
--- a/lib/libvmod_vtc/vmod.vcc
+++ b/lib/libvmod_vtc/vmod.vcc
@@ -102,7 +102,7 @@ $Function BYTES workspace_reserve(ENUM { client, backend, session, thread },
 Attempt to reserve *size* bytes and release the reservation right
 away. Return the size of the reservation.
 
-See `vtc.workspace_alloc`_ for semantics of the *size* argument.
+See `vtc.workspace_alloc()`_ for semantics of the *size* argument.
 
 $Function INT workspace_free(ENUM { client, backend, session, thread })
 

From dridi at varni.sh  Tue Nov 26 13:10:21 2019
From: dridi at varni.sh (Dridi Boukelmoune)
Date: Tue, 26 Nov 2019 13:10:21 +0000
Subject: [master] ed3b095c0 add a facility to test WS_ReserveSize()
In-Reply-To: <20191126124406.A5B3210F125@lists.varnish-cache.org>
References: <20191126124406.A5B3210F125@lists.varnish-cache.org>
Message-ID: <CABoVN9Cro4KuBnROfo_BvjgO4N9i1nKfS19GsoSn=Y-c0jHHqg@mail.gmail.com>

On Tue, Nov 26, 2019 at 12:44 PM Nils Goroll <nils.goroll at uplex.de> wrote:
>
>
> commit ed3b095c0414af453493580a654f63a02d22e0e8
> Author: Nils Goroll <nils.goroll at uplex.de>
> Date:   Tue Nov 26 13:21:37 2019 +0100
>
>     add a facility to test WS_ReserveSize()
>
> diff --git a/lib/libvmod_vtc/vmod.vcc b/lib/libvmod_vtc/vmod.vcc
> index 51154f366..4f15d0d07 100644
> --- a/lib/libvmod_vtc/vmod.vcc
> +++ b/lib/libvmod_vtc/vmod.vcc
> @@ -96,6 +96,14 @@ as much as needed to leave that many bytes free. The actual allocation size
>  may be higher to comply with memory alignment requirements of the CPU
>  architecture. A failed allocation fails the transaction.
>
> +$Function BYTES workspace_reserve(ENUM { client, backend, session, thread },
> +       INT size)
> +
> +Attempt to reserve *size* bytes and release the reservation right
> +away. Return the size of the reservation.
> +
> +See `vtc.workspace_alloc`_ for semantics of the *size* argument.
> +
>  $Function INT workspace_free(ENUM { client, backend, session, thread })
>
>  Find how much unallocated space there is left in a workspace.
> diff --git a/lib/libvmod_vtc/vmod_vtc.c b/lib/libvmod_vtc/vmod_vtc.c
> index f7e58db5c..e52c40d5a 100644
> --- a/lib/libvmod_vtc/vmod_vtc.c
> +++ b/lib/libvmod_vtc/vmod_vtc.c
> @@ -167,6 +167,34 @@ vmod_workspace_alloc(VRT_CTX, VCL_ENUM which, VCL_INT size)
>                 memset(p, '\0', size);
>  }
>
> +VCL_BYTES v_matchproto_(td_vtc_workspace_reserve)
> +vmod_workspace_reserve(VRT_CTX, VCL_ENUM which, VCL_INT size)
> +{
> +       struct ws *ws;
> +       unsigned r;
> +
> +       CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC);
> +
> +       ws = vtc_ws_find(ctx, which);
> +       if (ws == NULL)
> +               return (0);
> +       WS_Assert(ws);
> +
> +       if (size < 0) {
> +               size += WS_ReserveAll(ws);
> +               WS_Release(ws, 0);
> +       }
> +       if (size <= 0) {
> +               VRT_fail(ctx, "Attempted negative WS reservation");
> +               return (0);
> +       }

I'm not sure to follow the logic above. If we want to allow a negative
size it should be handled separately from the "normal" case.

> +       r = WS_ReserveSize(ws, size);
> +       if (r == 0)
> +               return (0);
> +       WS_Release(ws, 0);
> +       return (1);
> +}
> +
>  VCL_INT v_matchproto_(td_vtc_workspace_free)
>  vmod_workspace_free(VRT_CTX, VCL_ENUM which)
>  {
> _______________________________________________
> varnish-commit mailing list
> varnish-commit at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-commit

From nils.goroll at uplex.de  Tue Nov 26 13:11:05 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 26 Nov 2019 13:11:05 +0000 (UTC)
Subject: [master] 815331b3a fix copy-pasta vtc description
Message-ID: <20191126131105.DF6FA10FC2C@lists.varnish-cache.org>


commit 815331b3aecca0eb7e4e29f6dd4d778124c2ac9d
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue Nov 26 14:09:32 2019 +0100

    fix copy-pasta vtc description

diff --git a/bin/varnishtest/tests/r03131.vtc b/bin/varnishtest/tests/r03131.vtc
index 8834e50f7..5de9b7bfe 100644
--- a/bin/varnishtest/tests/r03131.vtc
+++ b/bin/varnishtest/tests/r03131.vtc
@@ -1,4 +1,4 @@
-varnishtest "Test workspace functions in vmod_vtc"
+varnishtest "Test WS_ReserveSize() overflow behavior"
 
 varnish v1 -vcl {
 	import vtc;

From nils.goroll at uplex.de  Tue Nov 26 13:12:16 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 26 Nov 2019 14:12:16 +0100
Subject: [master] ed3b095c0 add a facility to test WS_ReserveSize()
In-Reply-To: <CABoVN9Cro4KuBnROfo_BvjgO4N9i1nKfS19GsoSn=Y-c0jHHqg@mail.gmail.com>
References: <20191126124406.A5B3210F125@lists.varnish-cache.org>
 <CABoVN9Cro4KuBnROfo_BvjgO4N9i1nKfS19GsoSn=Y-c0jHHqg@mail.gmail.com>
Message-ID: <a5eeb5f2-2bc6-4d2d-d501-a244c35ff447@uplex.de>

On 26/11/2019 14:10, Dridi Boukelmoune wrote:
> I'm not sure to follow the logic above. If we want to allow a negative
> size it should be handled separately from the "normal" case.

this is your code, Dridi :)

I like how you handle the "leave x bytes free" in .alloc, so I stuck to it

-- 

** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstra?e 32
22301 Hamburg

tel +49 40 28805731
mob +49 170 2723133
fax +49 40 42949753

xmpp://slink at jabber.int.uplex.de/

http://uplex.de/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-commit/attachments/20191126/1ed3a962/attachment.bin>

From dridi at varni.sh  Tue Nov 26 13:28:57 2019
From: dridi at varni.sh (Dridi Boukelmoune)
Date: Tue, 26 Nov 2019 13:28:57 +0000
Subject: [master] ed3b095c0 add a facility to test WS_ReserveSize()
In-Reply-To: <a5eeb5f2-2bc6-4d2d-d501-a244c35ff447@uplex.de>
References: <20191126124406.A5B3210F125@lists.varnish-cache.org>
 <CABoVN9Cro4KuBnROfo_BvjgO4N9i1nKfS19GsoSn=Y-c0jHHqg@mail.gmail.com>
 <a5eeb5f2-2bc6-4d2d-d501-a244c35ff447@uplex.de>
Message-ID: <CABoVN9DZuo9gnv_B9=kkW5O=DrM7Zm7VOBOxm7t3Xa37hN5+_Q@mail.gmail.com>

On Tue, Nov 26, 2019 at 1:12 PM Nils Goroll <nils.goroll at uplex.de> wrote:
>
> On 26/11/2019 14:10, Dridi Boukelmoune wrote:
> > I'm not sure to follow the logic above. If we want to allow a negative
> > size it should be handled separately from the "normal" case.
>
> this is your code, Dridi :)
>
> I like how you handle the "leave x bytes free" in .alloc, so I stuck to it

Please note that while git will point to me, the only thing I did was
to move the code from vmod_debug where it originated. I may have
polished a few things here and there but to be honest, I didn't
remember.

Thanks for reminding me, and forget my remarks :)

Dridi

From nils.goroll at uplex.de  Tue Nov 26 16:09:06 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 26 Nov 2019 16:09:06 +0000 (UTC)
Subject: [master] 287dc4a67 Add Session Attribute workspace overflow handling
Message-ID: <20191126160906.79CD0115BE4@lists.varnish-cache.org>


commit 287dc4a6745c374e0b229bfa861d664989a3a9e8
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Tue Nov 26 14:56:24 2019 +0100

    Add Session Attribute workspace overflow handling
    
    Notes:
    
    * for the acceptor, I think it makes sense to keep AN assertion (pun!)
      because varnish is not viable if the session workspace is too small
      to even hold the attributes initialized in the acceptor.
    
      If this was an issue, we should rather revisit the minimum values for
      the session workspace
    
    * for h1 and h2 session setup, I have used XXXAN() because I am not sure
      how we should best handle allocation failures.
    
    * The relevant bit, for now, is the proxy code which may allocate
      arbitrarily long TLV attributes, so this is the code for which we now
      actually handle errors and test that we do
    
    On the vtc: I added the test to o00005.vtc because there existed a
    previous overflow test from 267504b8143bdb8ef96240455a3aa788f96b579b,
    but that only tested for the one case of a WS overflow which was already
    handled.
    
    Fixes #3145

diff --git a/bin/varnishd/cache/cache_acceptor.c b/bin/varnishd/cache/cache_acceptor.c
index 73430d76e..0e7ee9877 100644
--- a/bin/varnishd/cache/cache_acceptor.c
+++ b/bin/varnishd/cache/cache_acceptor.c
@@ -317,17 +317,17 @@ vca_mk_tcp(const struct wrk_accept *wa,
 	struct sockaddr_storage ss;
 	socklen_t sl;
 
-	SES_Reserve_remote_addr(sp, &sa);
+	AN(SES_Reserve_remote_addr(sp, &sa));
 	AN(VSA_Build(sa, &wa->acceptaddr, wa->acceptaddrlen));
 	sp->sattr[SA_CLIENT_ADDR] = sp->sattr[SA_REMOTE_ADDR];
 
 	VTCP_name(sa, raddr, VTCP_ADDRBUFSIZE, rport, VTCP_PORTBUFSIZE);
-	SES_Set_String_Attr(sp, SA_CLIENT_IP, raddr);
-	SES_Set_String_Attr(sp, SA_CLIENT_PORT, rport);
+	AN(SES_Set_String_Attr(sp, SA_CLIENT_IP, raddr));
+	AN(SES_Set_String_Attr(sp, SA_CLIENT_PORT, rport));
 
 	sl = sizeof ss;
 	AZ(getsockname(sp->fd, (void*)&ss, &sl));
-	SES_Reserve_local_addr(sp, &sa);
+	AN(SES_Reserve_local_addr(sp, &sa));
 	AN(VSA_Build(sa, &ss, sl));
 	sp->sattr[SA_SERVER_ADDR] = sp->sattr[SA_LOCAL_ADDR];
 	VTCP_name(sa, laddr, VTCP_ADDRBUFSIZE, lport, VTCP_PORTBUFSIZE);
@@ -340,13 +340,13 @@ vca_mk_uds(struct wrk_accept *wa, struct sess *sp, char *laddr, char *lport,
 	struct suckaddr *sa;
 
 	(void) wa;
-	SES_Reserve_remote_addr(sp, &sa);
+	AN(SES_Reserve_remote_addr(sp, &sa));
 	AZ(SES_Set_remote_addr(sp, bogo_ip));
 	sp->sattr[SA_CLIENT_ADDR] = sp->sattr[SA_REMOTE_ADDR];
 	sp->sattr[SA_LOCAL_ADDR] = sp->sattr[SA_REMOTE_ADDR];
 	sp->sattr[SA_SERVER_ADDR] = sp->sattr[SA_REMOTE_ADDR];
-	SES_Set_String_Attr(sp, SA_CLIENT_IP, "0.0.0.0");
-	SES_Set_String_Attr(sp, SA_CLIENT_PORT, "0");
+	AN(SES_Set_String_Attr(sp, SA_CLIENT_IP, "0.0.0.0"));
+	AN(SES_Set_String_Attr(sp, SA_CLIENT_PORT, "0"));
 
 	strcpy(laddr, "0.0.0.0");
 	strcpy(raddr, "0.0.0.0");
diff --git a/bin/varnishd/cache/cache_session.c b/bin/varnishd/cache/cache_session.c
index d536962bf..3e131943b 100644
--- a/bin/varnishd/cache/cache_session.c
+++ b/bin/varnishd/cache/cache_session.c
@@ -110,8 +110,8 @@ ses_set_attr(const struct sess *sp, enum sess_attr a, const void *src, int sz)
 	return (0);
 }
 
-static void
-ses_reserve_attr(struct sess *sp, enum sess_attr a, void **dst, int sz)
+static int
+ses_res_attr(struct sess *sp, enum sess_attr a, void **dst, int sz)
 {
 	ssize_t o;
 
@@ -120,12 +120,14 @@ ses_reserve_attr(struct sess *sp, enum sess_attr a, void **dst, int sz)
 	assert(sz >= 0);
 	AN(dst);
 	o = WS_ReserveSize(sp->ws, sz);
-	assert(o >= sz);
+	if (o < sz)
+		return (0);
 	*dst = sp->ws->f;
 	o = sp->ws->f - sp->ws->s;
 	WS_Release(sp->ws, sz);
 	assert(o >= 0 && o <= 0xffff);
 	sp->sattr[a] = (uint16_t)o;
+	return (1);
 }
 
 #define SESS_ATTR(UP, low, typ, len)					\
@@ -143,16 +145,16 @@ ses_reserve_attr(struct sess *sp, enum sess_attr a, void **dst, int sz)
 		return (ses_get_attr(sp, SA_##UP, (void**)dst));	\
 	}								\
 									\
-	void								\
+	int								\
 	SES_Reserve_##low(struct sess *sp, typ **dst)			\
 	{								\
 		assert(len > 0);					\
-		ses_reserve_attr(sp, SA_##UP, (void**)dst, len);	\
+		return (ses_res_attr(sp, SA_##UP, (void**)dst, len));	\
 	}
 
 #include "tbl/sess_attr.h"
 
-void
+int
 SES_Set_String_Attr(struct sess *sp, enum sess_attr a, const char *src)
 {
 	void *q;
@@ -164,8 +166,10 @@ SES_Set_String_Attr(struct sess *sp, enum sess_attr a, const char *src)
 	if (strcmp(sess_attr[a].type, "char"))
 		WRONG("wrong sess_attr: not char");
 
-	ses_reserve_attr(sp, a, &q, strlen(src) + 1);
+	if (! ses_res_attr(sp, a, &q, strlen(src) + 1))
+		return (0);
 	strcpy(q, src);
+	return (1);
 }
 
 const char *
diff --git a/bin/varnishd/cache/cache_varnishd.h b/bin/varnishd/cache/cache_varnishd.h
index b8ccb4704..b9e7e32ca 100644
--- a/bin/varnishd/cache/cache_varnishd.h
+++ b/bin/varnishd/cache/cache_varnishd.h
@@ -389,9 +389,9 @@ enum htc_status_e HTC_RxStuff(struct http_conn *, htc_complete_f *,
 
 #define SESS_ATTR(UP, low, typ, len)					\
 	int SES_Set_##low(const struct sess *sp, const typ *src);	\
-	void SES_Reserve_##low(struct sess *sp, typ **dst);
+	int SES_Reserve_##low(struct sess *sp, typ **dst);
 #include "tbl/sess_attr.h"
-void SES_Set_String_Attr(struct sess *sp, enum sess_attr a, const char *src);
+int SES_Set_String_Attr(struct sess *sp, enum sess_attr a, const char *src);
 
 /* cache_shmlog.c */
 extern struct VSC_main *VSC_C_main;
diff --git a/bin/varnishd/http1/cache_http1_fsm.c b/bin/varnishd/http1/cache_http1_fsm.c
index 9c0998550..b5536d545 100644
--- a/bin/varnishd/http1/cache_http1_fsm.c
+++ b/bin/varnishd/http1/cache_http1_fsm.c
@@ -112,7 +112,7 @@ http1_new_session(struct worker *wrk, void *arg)
 	CHECK_OBJ_NOTNULL(sp, SESS_MAGIC);
 
 	HTC_RxInit(req->htc, req->ws);
-	SES_Reserve_proto_priv(sp, &u);
+	XXXAN(SES_Reserve_proto_priv(sp, &u));
 	http1_setstate(sp, H1NEWREQ);
 	wrk->task.func = http1_req;
 	wrk->task.priv = req;
diff --git a/bin/varnishd/http2/cache_http2_session.c b/bin/varnishd/http2/cache_http2_session.c
index aa91bc15d..0fce32bd8 100644
--- a/bin/varnishd/http2/cache_http2_session.c
+++ b/bin/varnishd/http2/cache_http2_session.c
@@ -101,7 +101,7 @@ h2_init_sess(const struct worker *wrk, struct sess *sp,
 
 	if (SES_Get_proto_priv(sp, &up)) {
 		/* Already reserved if we came via H1 */
-		SES_Reserve_proto_priv(sp, &up);
+		XXXAN(SES_Reserve_proto_priv(sp, &up));
 		*up = 0;
 	}
 	if (*up == 0) {
@@ -130,7 +130,7 @@ h2_init_sess(const struct worker *wrk, struct sess *sp,
 		AZ(VHT_Init(h2->dectbl,
 			h2->local_settings.header_table_size));
 
-		SES_Reserve_proto_priv(sp, &up);
+		XXXAN(SES_Reserve_proto_priv(sp, &up));
 		*up = (uintptr_t)h2;
 	}
 	AN(up);
diff --git a/bin/varnishd/proxy/cache_proxy_proto.c b/bin/varnishd/proxy/cache_proxy_proto.c
index 78b120c2a..80d93349e 100644
--- a/bin/varnishd/proxy/cache_proxy_proto.c
+++ b/bin/varnishd/proxy/cache_proxy_proto.c
@@ -50,6 +50,13 @@ struct vpx_tlv {
 	char			tlv[1];
 };
 
+static inline int
+vpx_ws_err(const struct req *req)
+{
+	VSL(SLT_Error, req->sp->vxid, "insufficient workspace");
+	return (-1);
+}
+
 /**********************************************************************
  * PROXY 1 protocol
  */
@@ -109,17 +116,23 @@ vpx_proto1(const struct worker *wrk, const struct req *req)
 		return (-1);
 	}
 
-	SES_Reserve_client_addr(req->sp, &sa);
+	if (! SES_Reserve_client_addr(req->sp, &sa))
+		return (vpx_ws_err(req));
+
 	if (VSS_ResolveOne(sa, fld[1], fld[3],
 	    pfam, SOCK_STREAM, AI_NUMERICHOST | AI_NUMERICSERV) == NULL) {
 		VSL(SLT_ProxyGarbage, req->sp->vxid,
 		    "PROXY1: Cannot resolve source address");
 		return (-1);
 	}
-	SES_Set_String_Attr(req->sp, SA_CLIENT_IP, fld[1]);
-	SES_Set_String_Attr(req->sp, SA_CLIENT_PORT, fld[3]);
+	if (! SES_Set_String_Attr(req->sp, SA_CLIENT_IP, fld[1]))
+		return (vpx_ws_err(req));
+	if (! SES_Set_String_Attr(req->sp, SA_CLIENT_PORT, fld[3]))
+		return (vpx_ws_err(req));
+
+	if (! SES_Reserve_server_addr(req->sp, &sa))
+		return (vpx_ws_err(req));
 
-	SES_Reserve_server_addr(req->sp, &sa);
 	if (VSS_ResolveOne(sa, fld[2], fld[4],
 	    pfam, SOCK_STREAM, AI_NUMERICHOST | AI_NUMERICSERV) == NULL) {
 		VSL(SLT_ProxyGarbage, req->sp->vxid,
@@ -377,14 +390,16 @@ vpx_proto2(const struct worker *wrk, struct req *req)
 		/* dst/server */
 		memcpy(&sin4.sin_addr, p + 20, 4);
 		memcpy(&sin4.sin_port, p + 26, 2);
-		SES_Reserve_server_addr(req->sp, &sa);
+		if (! SES_Reserve_server_addr(req->sp, &sa))
+			return (vpx_ws_err(req));
 		AN(VSA_Build(sa, &sin4, sizeof sin4));
 		VTCP_name(sa, ha, sizeof ha, pa, sizeof pa);
 
 		/* src/client */
 		memcpy(&sin4.sin_addr, p + 16, 4);
 		memcpy(&sin4.sin_port, p + 24, 2);
-		SES_Reserve_client_addr(req->sp, &sa);
+		if (! SES_Reserve_client_addr(req->sp, &sa))
+			return (vpx_ws_err(req));
 		AN(VSA_Build(sa, &sin4, sizeof sin4));
 		break;
 	case 0x21:
@@ -403,14 +418,16 @@ vpx_proto2(const struct worker *wrk, struct req *req)
 		/* dst/server */
 		memcpy(&sin6.sin6_addr, p + 32, 16);
 		memcpy(&sin6.sin6_port, p + 50, 2);
-		SES_Reserve_server_addr(req->sp, &sa);
+		if (! SES_Reserve_server_addr(req->sp, &sa))
+			return (vpx_ws_err(req));
 		AN(VSA_Build(sa, &sin6, sizeof sin6));
 		VTCP_name(sa, ha, sizeof ha, pa, sizeof pa);
 
 		/* src/client */
 		memcpy(&sin6.sin6_addr, p + 16, 16);
 		memcpy(&sin6.sin6_port, p + 48, 2);
-		SES_Reserve_client_addr(req->sp, &sa);
+		if (! SES_Reserve_client_addr(req->sp, &sa))
+			return (vpx_ws_err(req));
 		AN(VSA_Build(sa, &sin6, sizeof sin6));
 		break;
 	default:
@@ -422,8 +439,10 @@ vpx_proto2(const struct worker *wrk, struct req *req)
 
 	AN(sa);
 	VTCP_name(sa, hb, sizeof hb, pb, sizeof pb);
-	SES_Set_String_Attr(req->sp, SA_CLIENT_IP, hb);
-	SES_Set_String_Attr(req->sp, SA_CLIENT_PORT, pb);
+	if (! SES_Set_String_Attr(req->sp, SA_CLIENT_IP, hb))
+		return (vpx_ws_err(req));
+	if (! SES_Set_String_Attr(req->sp, SA_CLIENT_PORT, pb))
+		return (vpx_ws_err(req));
 
 	VSL(SLT_Proxy, req->sp->vxid, "2 %s %s %s %s", hb, pb, ha, pa);
 
@@ -450,15 +469,13 @@ vpx_proto2(const struct worker *wrk, struct req *req)
 		return (-1);
 	}
 	tlv = WS_Alloc(req->sp->ws, sizeof *tlv + tlv_len);
-	if (tlv == NULL) {
-		VSL(SLT_ProxyGarbage, req->sp->vxid,
-		    "PROXY2: TLV overflows WS");
-		return (-1);
-	}
+	if (tlv == NULL)
+		return (vpx_ws_err(req));
 	INIT_OBJ(tlv, VPX_TLV_MAGIC);
 	tlv->len = tlv_len;
 	memcpy(tlv->tlv, tlv_start, tlv_len);
-	SES_Reserve_proxy_tlv(req->sp, &up);
+	if (! SES_Reserve_proxy_tlv(req->sp, &up))
+		return (vpx_ws_err(req));
 	*up = (uintptr_t)tlv;
 	return (0);
 }
diff --git a/bin/varnishtest/tests/o00005.vtc b/bin/varnishtest/tests/o00005.vtc
index 3ae1aacfe..d84b92d4d 100644
--- a/bin/varnishtest/tests/o00005.vtc
+++ b/bin/varnishtest/tests/o00005.vtc
@@ -5,7 +5,8 @@ server s1 {
 	txresp
 } -start
 
-varnish v1 -proto "PROXY" -vcl+backend {
+varnish v1 -arg "-p pool_sess=0,0,0" -proto "PROXY" -vcl+backend {
+	import vtc;
 	import proxy;
 
 	sub vcl_deliver {
@@ -20,6 +21,7 @@ varnish v1 -proto "PROXY" -vcl+backend {
 		set resp.http.key = proxy.cert_key();
 		set resp.http.sign = proxy.cert_sign();
 		set resp.http.cn = proxy.client_cert_cn();
+		set resp.http.ws_free = vtc.workspace_free(session);
 	}
 } -start
 
@@ -243,4 +245,68 @@ client c1 {
 	expect_close
 } -run
 
+delay 1
+
 varnish v1 -expect ws_session_overflow == 1
+
+# error handling elsewhere in the proxy code
+# request is the same as initial
+
+varnish v1 -cliok "param.set workspace_session 450"
+varnish v1 -cliok "param.set pool_sess 10,100,1"
+
+delay 1
+
+# get rid of the surplus session mpl
+client c10 -proxy1 "1.2.3.4:1111 5.6.7.8:5678" {
+	txreq
+	rxresp
+} -start
+client c11 -proxy1 "1.2.3.4:1111 5.6.7.8:5678" {
+	txreq
+	rxresp
+} -start
+client c12 -proxy1 "1.2.3.4:1111 5.6.7.8:5678" {
+	txreq
+	rxresp
+} -start
+client c13 -proxy1 "1.2.3.4:1111 5.6.7.8:5678" {
+	txreq
+	rxresp
+} -start
+client c14 -proxy1 "1.2.3.4:1111 5.6.7.8:5678" {
+	txreq
+	rxresp
+} -start
+
+client c10 -wait
+client c11 -wait
+client c12 -wait
+client c13 -wait
+client c14 -wait
+
+client c2 {
+	# PROXY2 with CRC32C TLV
+	sendhex {
+		0d 0a 0d 0a 00 0d 0a 51 55 49 54 0a
+		21 11 00 65
+		d9 46 b5 21
+		5f 8e a8 22
+		ed 96
+		01 bb
+		03 00 04  95 03 ee 75
+		01 00 02  68 32
+		02 00 0a  68 6f 63 64 65 74 2e 6e 65 74
+		20 00 3d
+		01 00 00 00 00
+		21 00 07  54 4c 53 76 31 2e 33
+		25 00 05  45 43 32 35 36
+		24 00 0a  52 53 41 2d 53 48 41 32 35 36
+		23 00 16  41 45 41 44 2d 41 45 53 31 32 38
+			  2d 47 43 4d 2d 53 48 41 32 35 36
+	}
+	txreq
+	expect_close
+} -run
+
+varnish v1 -expect ws_session_overflow == 2

From nils.goroll at uplex.de  Tue Nov 26 16:21:06 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Tue, 26 Nov 2019 16:21:06 +0000 (UTC)
Subject: [master] e74f9e871 Simplify WS allocation in tlv_string
Message-ID: <20191126162106.4E729116180@lists.varnish-cache.org>


commit e74f9e871bc0eab014b7dd359d6ae83153c3ee37
Author: Emmanuel Hocdet <manu at gandi.net>
Date:   Thu Nov 14 11:14:07 2019 +0100

    Simplify WS allocation in tlv_string
    
    Patch by @ehocdet, commit message edited by @nigoroll:
    
    The root cause of #3131 was misdiagnosed to the extent that, while this
    change had prevented it, the root cause was a bug in WS_ReserveSize()
    fixed in 505b7bd9643006fa8e3977f920564ce12a2b24a2
    
    The previous tlv_string() code was correct except for the
    fact that error handling should have checked for WS_ReserveSize(ctx->ws,
    len+1) <= len (also spotted by @ehocdet).
    
    Someone had mentioned at some point that we would not want to VRT_fail(),
    but I think this must have been related to the proxy transport code, not
    the proxy vmod.
    
    Ref #3131

diff --git a/lib/libvmod_proxy/vmod_proxy.c b/lib/libvmod_proxy/vmod_proxy.c
index 58f2db295..222de9c53 100644
--- a/lib/libvmod_proxy/vmod_proxy.c
+++ b/lib/libvmod_proxy/vmod_proxy.c
@@ -105,12 +105,13 @@ tlv_string(VRT_CTX, int tlv)
 
 	if (VPX_tlv(ctx->req, tlv, (void **)&dst, &len))
 		return (NULL);
-	if (!WS_ReserveSize(ctx->ws, len+1))
+	d = WS_Alloc(ctx->ws, len+1);
+	if (d == NULL) {
+		VRT_fail(ctx, "proxy.TLV: out of workspace");
 		return (NULL);
-	d = ctx->ws->f;
+	}
 	memcpy(d, dst, len);
 	d[len] = '\0';
-	WS_Release(ctx->ws, len+1);
 	return (d);
 }
 

From dridi.boukelmoune at gmail.com  Thu Nov 28 06:20:07 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Thu, 28 Nov 2019 06:20:07 +0000 (UTC)
Subject: [master] 79373b9b5 Kill dead vmod.constructor descriptor check
Message-ID: <20191128062007.758F71039FB@lists.varnish-cache.org>


commit 79373b9b59752663e4004b399abf7a809c1b5c1b
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Thu Nov 28 07:14:10 2019 +0100

    Kill dead vmod.constructor descriptor check
    
    There's only one place where a symbol may be assigned the SYM_OBJECT
    kind and it is ensured that the JSON descriptor is found and added to
    the symbol.

diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c
index 846f6d502..2ff90ad31 100644
--- a/lib/libvcc/vcc_vmod.c
+++ b/lib/libvcc/vcc_vmod.c
@@ -405,14 +405,6 @@ vcc_Act_New(struct vcc *tl, struct token *t, struct symbol *sym)
 	sy2 = VCC_SymbolGet(tl, SYM_OBJECT, SYMTAB_EXISTING, XREF_NONE);
 	ERRCHK(tl);
 	AN(sy2);
-	if (sy2->eval_priv == NULL) {
-		VSB_printf(tl->sb, "Constructor not found: ");
-		vcc_ErrToken(tl, t);
-		VSB_printf(tl->sb, " at ");
-		vcc_ErrWhere(tl, t);
-		return;
-	}
-
 	CAST_OBJ_NOTNULL(vv, sy2->eval_priv, VJSN_VAL_MAGIC);
 	// vv = object name
 

From dridi.boukelmoune at gmail.com  Thu Nov 28 07:22:05 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Thu, 28 Nov 2019 07:22:05 +0000 (UTC)
Subject: [master] 23e7ff8ab Ignore unused argument
Message-ID: <20191128072205.CB9F9106561@lists.varnish-cache.org>


commit 23e7ff8ab216793397a9cd3b3360119fa486d176
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Thu Nov 28 08:21:30 2019 +0100

    Ignore unused argument

diff --git a/lib/libvcc/vcc_vmod.c b/lib/libvcc/vcc_vmod.c
index 2ff90ad31..dca8eabd8 100644
--- a/lib/libvcc/vcc_vmod.c
+++ b/lib/libvcc/vcc_vmod.c
@@ -389,6 +389,8 @@ vcc_Act_New(struct vcc *tl, struct token *t, struct symbol *sym)
 	int null_ok = 0;
 
 	(void)sym;
+	(void)t;
+
 	ExpectErr(tl, ID);
 	vcc_ExpectVid(tl, "VCL object");
 	ERRCHK(tl);
@@ -401,7 +403,6 @@ vcc_Act_New(struct vcc *tl, struct token *t, struct symbol *sym)
 	vcc_NextToken(tl);
 
 	ExpectErr(tl, ID);
-	t = tl->t;
 	sy2 = VCC_SymbolGet(tl, SYM_OBJECT, SYMTAB_EXISTING, XREF_NONE);
 	ERRCHK(tl);
 	AN(sy2);

From dridi.boukelmoune at gmail.com  Thu Nov 28 07:36:07 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Thu, 28 Nov 2019 07:36:07 +0000 (UTC)
Subject: [master] f8141cc42 Typo in user guide.
Message-ID: <20191128073607.E5BEB106B56@lists.varnish-cache.org>


commit f8141cc42ca5c2f1e3e2fb32c03a6a14b67fd9c4
Author: Deepjyoti Mondal <deepjyoti.m at media.net>
Date:   Thu Nov 28 12:53:48 2019 +0530

    Typo in user guide.
    
    Fixes typo in storage backends guide.

diff --git a/doc/sphinx/users-guide/storage-backends.rst b/doc/sphinx/users-guide/storage-backends.rst
index 9ba6076a4..b99972120 100644
--- a/doc/sphinx/users-guide/storage-backends.rst
+++ b/doc/sphinx/users-guide/storage-backends.rst
@@ -142,7 +142,7 @@ the storage file using `dd(1)` will reduce fragmentation to a minimum.
 .. XXX:1? benc
 
 The 'granularity' parameter specifies the granularity of
-allocation. All allocations are rounded up to this size. The granularity is
+allocation. All allocations are rounded up to this size. The granularity
 is assumed to be expressed in bytes, unless followed by one of the
 suffixes described for size.
 

From nils.goroll at uplex.de  Thu Nov 28 10:24:06 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu, 28 Nov 2019 10:24:06 +0000 (UTC)
Subject: [master] e1a57eb7a Try to make the proxy code session workspace
 overflow test on 32bit
Message-ID: <20191128102406.52FD9109E9E@lists.varnish-cache.org>


commit e1a57eb7a7350ac4c8a0235dc991bef400b118c1
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu Nov 28 11:19:08 2019 +0100

    Try to make the proxy code session workspace overflow test on 32bit
    
    Ref #3145 / 287dc4a6745c374e0b229bfa861d664989a3a9e8

diff --git a/bin/varnishtest/tests/o00005.vtc b/bin/varnishtest/tests/o00005.vtc
index d84b92d4d..2ac11d1a8 100644
--- a/bin/varnishtest/tests/o00005.vtc
+++ b/bin/varnishtest/tests/o00005.vtc
@@ -249,10 +249,16 @@ delay 1
 
 varnish v1 -expect ws_session_overflow == 1
 
-# error handling elsewhere in the proxy code
-# request is the same as initial
-
-varnish v1 -cliok "param.set workspace_session 450"
+# workspace overflow handling elsewhere in the proxy code
+#
+# the workspace_session size is chosen to fail as closely as possible
+# below the minimum required value for the vtc to work (= the
+# workspace to overflow) both on 64 and 32 bit.
+#
+# This value is fragile, ideally we would want something like
+# vtc.alloc(-x), yet there is no vcl code being run before we parse
+# proxy headers
+varnish v1 -cliok "param.set workspace_session 410"
 varnish v1 -cliok "param.set pool_sess 10,100,1"
 
 delay 1

From nils.goroll at uplex.de  Thu Nov 28 11:22:06 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Thu, 28 Nov 2019 11:22:06 +0000 (UTC)
Subject: [master] d6dec0310 more tweaking for 32bit vtest machines
Message-ID: <20191128112206.0FF0810C3DD@lists.varnish-cache.org>


commit d6dec03101a67c38addf549435ae51df38f1d7db
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Thu Nov 28 12:21:17 2019 +0100

    more tweaking for 32bit vtest machines
    
    Ref #3145 / 287dc4a6745c374e0b229bfa861d664989a3a9e8

diff --git a/bin/varnishtest/tests/o00005.vtc b/bin/varnishtest/tests/o00005.vtc
index 2ac11d1a8..b09a8fcaf 100644
--- a/bin/varnishtest/tests/o00005.vtc
+++ b/bin/varnishtest/tests/o00005.vtc
@@ -258,7 +258,7 @@ varnish v1 -expect ws_session_overflow == 1
 # This value is fragile, ideally we would want something like
 # vtc.alloc(-x), yet there is no vcl code being run before we parse
 # proxy headers
-varnish v1 -cliok "param.set workspace_session 410"
+varnish v1 -cliok "param.set workspace_session 402"
 varnish v1 -cliok "param.set pool_sess 10,100,1"
 
 delay 1

From dridi.boukelmoune at gmail.com  Fri Nov 29 07:07:08 2019
From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune)
Date: Fri, 29 Nov 2019 07:07:08 +0000 (UTC)
Subject: [master] 935b6bad4 Polish
Message-ID: <20191129070708.9A155100536@lists.varnish-cache.org>


commit 935b6bad4b7b58a254e4b3dcbd4f9ac7409abd67
Author: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
Date:   Fri Nov 29 07:44:27 2019 +0100

    Polish

diff --git a/lib/libvcc/vcc_symb.c b/lib/libvcc/vcc_symb.c
index 3ef753220..4bb0a72a7 100644
--- a/lib/libvcc/vcc_symb.c
+++ b/lib/libvcc/vcc_symb.c
@@ -113,10 +113,8 @@ vcc_dup_be(const char *b, const char *e)
 	AN(e);
 	assert(e >= b);
 
-	p = malloc((e - b) + 1);
+	p = strndup(b, e - b);
 	AN(p);
-	memcpy(p, b, e - b);
-	p[e - b] = '\0';
 	return (p);
 }
 

From nils.goroll at uplex.de  Fri Nov 29 16:34:05 2019
From: nils.goroll at uplex.de (Nils Goroll)
Date: Fri, 29 Nov 2019 16:34:05 +0000 (UTC)
Subject: [master] c8585914a guard against accidental regressions regarding obj
Message-ID: <20191129163405.ECA8810C76C@lists.varnish-cache.org>


commit c8585914a9abab4d8bad4548821d4e8abce81640
Author: Nils Goroll <nils.goroll at uplex.de>
Date:   Fri Nov 29 17:27:52 2019 +0100

    guard against accidental regressions regarding obj
    
    enum gethdr_e and VCL_HTTP have a 1:1 relation except for HDR_OBJ which
    does not have a corresponding VCL_HTTP pointer.
    
    As I am about to use a "reverse VRT_selecthttp()" to get from VCL_HTTP
    to enum gethdr_e, I would want to guard against accidental introduction
    of a regression.

diff --git a/bin/varnishtest/tests/m00000.vtc b/bin/varnishtest/tests/m00000.vtc
index e75bcd6de..342a861e3 100644
--- a/bin/varnishtest/tests/m00000.vtc
+++ b/bin/varnishtest/tests/m00000.vtc
@@ -127,6 +127,20 @@ varnish v1 -errvcl {Expression has type STRING, expected REAL} {
 	}
 }
 
+varnish v1 -errvcl {Symbol not found: 'obj'} {
+	import debug;
+	sub vcl_hit {
+		debug.rot52(obj);
+	}
+}
+
+varnish v1 -errvcl {Symbol not found: 'obj'} {
+	import debug;
+	sub vcl_deliver {
+		debug.rot52(obj);
+	}
+}
+
 varnish v1 -errvcl {Failed initialization} {
 	import debug;
 	import directors;