From nils.goroll at uplex.de Thu Sep 1 11:18:08 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 1 Sep 2022 11:18:08 +0000 (UTC) Subject: [master] ae69c63be Polish WS_Panic pointer printing Message-ID: <20220901111808.3DF8E120699@lists.varnish-cache.org> commit ae69c63bee7caf4e4122133e38a93352a3d9aeed Author: Nils Goroll Date: Thu Sep 1 13:14:00 2022 +0200 Polish WS_Panic pointer printing diff --git a/bin/varnishd/cache/cache_ws.c b/bin/varnishd/cache/cache_ws.c index 45295fcb3..d327ad757 100644 --- a/bin/varnishd/cache/cache_ws.c +++ b/bin/varnishd/cache/cache_ws.c @@ -358,6 +358,15 @@ WS_Dump(const struct ws *ws, char where, size_t off, void *buf, size_t len) /*--------------------------------------------------------------------*/ +static inline void +ws_printptr(struct vsb *vsb, const char *s, const char *p) +{ + if (p >= s) + VSB_printf(vsb, ", +%ld", (long) (p - s)); + else + VSB_printf(vsb, ", %p", p); +} + void WS_Panic(struct vsb *vsb, const struct ws *ws) { @@ -368,18 +377,9 @@ WS_Panic(struct vsb *vsb, const struct ws *ws) VSB_cat(vsb, "OVERFLOWED "); VSB_printf(vsb, "id = \"%s\",\n", ws->id); VSB_printf(vsb, "{s, f, r, e} = {%p", ws->s); - if (ws->f >= ws->s) - VSB_printf(vsb, ", +%ld", (long) (ws->f - ws->s)); - else - VSB_printf(vsb, ", %p", ws->f); - if (ws->r >= ws->s) - VSB_printf(vsb, ", +%ld", (long) (ws->r - ws->s)); - else - VSB_printf(vsb, ", %p", ws->r); - if (ws->e >= ws->s) - VSB_printf(vsb, ", +%ld", (long) (ws->e - ws->s)); - else - VSB_printf(vsb, ", %p", ws->e); + ws_printptr(vsb, ws->s, ws->f); + ws_printptr(vsb, ws->s, ws->r); + ws_printptr(vsb, ws->s, ws->e); VSB_cat(vsb, "},\n"); VSB_indent(vsb, -2); VSB_cat(vsb, "},\n"); From dridi.boukelmoune at gmail.com Mon Sep 5 08:46:09 2022 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Mon, 5 Sep 2022 08:46:09 +0000 (UTC) Subject: [master] 719994efe Start skeleton release notes for the next version. Message-ID: <20220905084609.0E345105560@lists.varnish-cache.org> commit 719994efe6c56fbc35d12a8d797929ac5944bd7e Author: Geoff Simmons Date: Tue Sep 25 16:31:17 2018 +0200 Start skeleton release notes for the next version. Restructured so that: * 'Upgrading' is limited to work that has to be done to upgrade from a current deployment to the new version. * 'Changes' is a comprehensive, user-level description of changes and new features. diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-trunk.rst new file mode 100644 index 000000000..fbda9d6b9 --- /dev/null +++ b/doc/sphinx/whats-new/changes-trunk.rst @@ -0,0 +1,73 @@ +**Note: This is a working document for a future release, with running +updates for changes in the development branch. For changes in the +released versions of Varnish, see:** :ref:`whats-new-index` + +.. _whatsnew_changes_CURRENT: + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +Changes in Varnish **$NEXT_RELEASE** +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +For information about updating your current Varnish deployment to the +new version, see :ref:`whatsnew_upgrading_CURRENT`. + +A more detailed and technical account of changes in Varnish, with +links to issues that have been fixed and pull requests that have been +merged, may be found in the `change log`_. + +.. _change log: https://github.com/varnishcache/varnish-cache/blob/master/doc/changes.rst + +varnishd +======== + +Parameters +~~~~~~~~~~ + +**XXX changes in -p parameters** + +Other changes in varnishd +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Changes to VCL +============== + +VCL variables +~~~~~~~~~~~~~ + +**XXX new, deprecated or removed variables, or changed semantics** + +Other changes to VCL +~~~~~~~~~~~~~~~~~~~~ + +VMODs +===== + +**XXX changes in the bundled VMODs** + +varnishlog +========== + +**XXX changes concerning varnishlog(1) and/or vsl(7)** + +varnishadm +========== + +**XXX changes concerning varnishadm(1) and/or varnish-cli(7)** + +varnishstat +=========== + +**XXX changes concerning varnishstat(1) and/or varnish-counters(7)** + +varnishtest +=========== + +**XXX changes concerning varnishtest(1) and/or vtc(7)** + +Changes for developers and VMOD authors +======================================= + +**XXX changes concerning VRT, the public APIs, source code organization, +builds etc.** + +*eof* diff --git a/doc/sphinx/whats-new/index.rst b/doc/sphinx/whats-new/index.rst index 0b441cb8c..a8a7f5b64 100644 --- a/doc/sphinx/whats-new/index.rst +++ b/doc/sphinx/whats-new/index.rst @@ -13,6 +13,19 @@ This section describes the changes and improvements between different versions of Varnish, and what upgrading between the different versions entail. +Varnish **$NEXT_RELEASE** +------------------------- + +**Note: These are working documents for a future release, with running +updates for changes in the development branch. For changes in the +released versions of Varnish, see the chapters listed below.** + +.. toctree:: + :maxdepth: 2 + + changes-trunk + upgrading-trunk + Varnish **7.1** --------------- diff --git a/doc/sphinx/whats-new/upgrading-trunk.rst b/doc/sphinx/whats-new/upgrading-trunk.rst new file mode 100644 index 000000000..6143fde99 --- /dev/null +++ b/doc/sphinx/whats-new/upgrading-trunk.rst @@ -0,0 +1,33 @@ +**Note: This is a working document for a future release, with running +updates for changes in the development branch. For changes in the +released versions of Varnish, see:** :ref:`whats-new-index` + +.. _whatsnew_upgrading_CURRENT: + +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +Upgrading to Varnish **$NEXT_RELEASE** +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% + +**XXX: how to upgrade from previous deployments to this +version. Limited to work that has to be done for an upgrade, new +features are listed in "Changes". Explicitly mention what does *not* +have to be changed, especially in VCL. May include, but is not limited +to:** + +* Elements of VCL that have been removed or are deprecated, or whose + semantics have changed. + +* -p parameters that have been removed or are deprecated, or whose + semantics have changed. + +* Changes in the CLI. + +* Changes in the output or interpretation of stats or the log, including + changes affecting varnishncsa/-hist/-top. + +* Changes that may be necessary in VTCs or in the use of varnishtest. + +* Changes in public APIs that may require changes in VMODs or VAPI/VUT + clients. + +*eof* From dridi.boukelmoune at gmail.com Mon Sep 5 09:51:06 2022 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Mon, 5 Sep 2022 09:51:06 +0000 (UTC) Subject: [master] 06518feb8 changes: First sweep since 7.1.0 Message-ID: <20220905095106.223621073E2@lists.varnish-cache.org> commit 06518feb872462eb6e05e5e2126460445e33cc1d Author: Dridi Boukelmoune Date: Mon Sep 5 11:50:16 2022 +0200 changes: First sweep since 7.1.0 diff --git a/doc/changes.rst b/doc/changes.rst index 654d06150..200c3cfd2 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -35,10 +35,88 @@ release process. Varnish Cache NEXT (2022-09-15) =============================== +* Functions ``VRT_AddVDP()``, ``VRT_AddVFP()``, ``VRT_RemoveVDP()`` and + ``VRT_RemoveVFP()`` are deprecated. + * Cookie headers generated by vmod_cookie no longer have a spurious trailing semi-colon (``';'``) at the end of the string. This could break VCL relying on the previous incorrect behavior. +* Prototypical Varnish Extensions (VEXT). Similar to VMODs, a VEXT is loaded + by the cache process. Unlike VMODs that have the combined lifetime of all + the VCLs that reference them, a VEXT has the lifetime of the cache process + itself. There are no built-in extensions so far. + +* The VCC (compilation) process no longer loads VMODs with ``dlopen(3)`` to + collect their metadata. + +* The parameter ``tcp_keepalive_time`` is supported on MacOS. + +* Duration parameters can optionally take a unit, with the same syntax as + duration units in VCL. Example: ``param.set default_grace 1h``. + +* New ``"B"`` string for the package branch in ``VCS_String()``. For the 7.2.0 + version, it would yield the 7.2 branch. + +* The Varnish version and branch are available in ``varnishtest`` through the + ``${pkg_version}`` and ``${pkg_branch}`` macros. + +* New ``${topsrc}`` macro in ``varnishtest -i`` mode. + +* New ``process pNAME -match-text`` command in ``varnishtest`` to expect + text matching a regular expression on screen. + +* New ``filewrite [-a]`` command in ``varnishtest`` to put or append a string + into a file. + +* The new ``vcc_feature`` bits parameter replaces previous ``vcc_*`` boolean + parameters. The latter still exist as deprecated aliases. + +* The ``-k`` option from ``varnishlog`` is now supported by ``varnishncsa``. + +* New functions ``std.now()`` and ``std.timed_call()`` in vmod_std. + +* New ``MAIN.shm_bytes`` counter. + +* A ``req.http.via`` header is set before entering ``vcl_recv``. Via headers + are generated using the ``server.identity`` value. It defaults to the host + name and can be turned into a pseudonym with the ``varnishd -i`` option. + Via headers are appended in both directions, to work with other hops that + may advertise themselves. + +* The ``server.identity`` syntax is now limited to a "token" as defined in + the HTTP grammar to be suitable for Via headers. + +* In ``varnishtest`` a Varnish instance will use its VTC instance name as its + instance name (``varnishd -i``) by default for predictable Via headers in + test cases. + +* VMOD and VEXT authors can use functions from ``vnum.h``. + +* The termination rules for ``WRK_BgThread()`` were relaxed to allow VMODs to + use it. + +* The ``Begin`` log record has a 4th field for subtasks like ESI sub-requests. + +* The ``-E`` option for log utilities now works as documented, with any type + of sub-task based on the ``Begin[4]`` field. This covers ESI like before, + and sub-tasks spawned by VMODs (provided that they log the new field). + +* No more ``req.http.transfer-encoding`` for ESI sub-requests. + +* New ``tools/coccinelle/vcocci.sh`` refactoring script for internal use. + +* The thread pool reserve is now limited to tasks that can be queued. A + backend background fetch is no longer eligible for queueing. It would + otherwise slow a grace hit down significantly when thread pools are + saturated. + +* The unused ``fetch_no_thread`` counter was renamed to ``bgfetch_no_thread`` + because regular backend fetch tasks are always scheduled. + +* There is a new ``contrib/`` directory in the Varnish source tree. The first + contribution is a ``varnishstatdiff`` script. + ================================ Varnish Cache 7.1.0 (2022-03-15) ================================ From phk at FreeBSD.org Mon Sep 5 11:28:06 2022 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Mon, 5 Sep 2022 11:28:06 +0000 (UTC) Subject: [master] 4354c58cc Check if bo->htc is there, like in the rest of the function Message-ID: <20220905112806.20F3110A25D@lists.varnish-cache.org> commit 4354c58cc7905eab616bc97444a4abd656371250 Author: Poul-Henning Kamp Date: Mon Sep 5 11:26:45 2022 +0000 Check if bo->htc is there, like in the rest of the function diff --git a/bin/varnishd/cache/cache_fetch.c b/bin/varnishd/cache/cache_fetch.c index b9be6f4a1..ffbc893fc 100644 --- a/bin/varnishd/cache/cache_fetch.c +++ b/bin/varnishd/cache/cache_fetch.c @@ -488,7 +488,7 @@ vbf_stp_startfetch(struct worker *wrk, struct busyobj *bo) if (http_IsStatus(bo->beresp, 304) && vbf_304_logic(bo) < 0) return (F_STP_ERROR); - if (bo->htc->doclose == SC_NULL && + if (bo->htc != NULL && bo->htc->doclose == SC_NULL && http_GetHdrField(bo->bereq, H_Connection, "close", NULL)) bo->htc->doclose = SC_REQ_CLOSE; From dridi.boukelmoune at gmail.com Mon Sep 5 14:19:06 2022 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Mon, 5 Sep 2022 14:19:06 +0000 (UTC) Subject: [master] 5e3f752d3 changes: First pass for 7.2.0 Message-ID: <20220905141906.40F5C1113A6@lists.varnish-cache.org> commit 5e3f752d3fe223324a317d1c912d388ef4eb73a9 Author: Dridi Boukelmoune Date: Mon Sep 5 16:16:54 2022 +0200 changes: First pass for 7.2.0 diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-trunk.rst index fbda9d6b9..cc4b2ea61 100644 --- a/doc/sphinx/whats-new/changes-trunk.rst +++ b/doc/sphinx/whats-new/changes-trunk.rst @@ -20,54 +20,137 @@ merged, may be found in the `change log`_. varnishd ======== +Extensions +~~~~~~~~~~ + +TODO: VEXT + Parameters ~~~~~~~~~~ -**XXX changes in -p parameters** +Duration values (with a unit in seconds) can optionally take a duration +unit with the same syntax as VCL. For example, the default values of +``default_ttl``, ``default_grace`` and ``default_keep`` were changed +respectively from ``120.000``, ``10.000`` and ``0.000`` to ``2m``, ``10s`` +and ``0s``. + +The platform-dependent ``tcp_keepalive_time`` parameter is supported on +MacOS. + +The new ``vcc_feature`` bits parameter replaces previous ``vcc_*`` boolean +parameters. The latter still exist as deprecated aliases. Other changes in varnishd ~~~~~~~~~~~~~~~~~~~~~~~~~ +The VCC (compilation) process no longer loads VMODs using ``dlopen(3)`` to +collect their metadata. + +Background fetch tasks are no longer queued as this could result in slow +grace hits subject to indefinite delays when thread pools are saturated. + Changes to VCL ============== VCL variables ~~~~~~~~~~~~~ -**XXX new, deprecated or removed variables, or changed semantics** +ESI sub-requests can no longer inherit a ``req.http.transfer-encoding`` +header since the request body is strictly handled by the top request. -Other changes to VCL -~~~~~~~~~~~~~~~~~~~~ +The ``resp.http.via`` header generated by Varnish uses ``server.identity`` +which defaults to the host name. A ``req.http.via`` header is generated +also before entering ``vcl_recv``. If a client request or backend response +already had a Via header, it is now appended to instead of overwritten. + +The ``server.identity`` variable is guaranteed to be a single token as +defined in the HTTP grammar, to safely be used as either a host name or +pseudonym in Via headers. + +The ``now`` variable remains constant in a VCL subroutine. This was already +the case, but is now (pun intended) formally defined behavior. It keeps the +same value even if the execution blocks for a significant time, for example +while calling a VMOD function. VMODs ===== -**XXX changes in the bundled VMODs** +For a real time timestamp, the function ``std.now()`` can be used instead. +There is also a new ``std.timed_call()`` to measure the execution time of a +subroutine. + +Cookie headers generated by vmod_cookie no longer have a spurious trailing +semi-colon (``';'``) at the end of the string. varnishlog ========== -**XXX changes concerning varnishlog(1) and/or vsl(7)** +The ``Begin`` log records may contain a 4th field with the sub-level of +sub-tasks. The ``Begin[4]`` field is used by the ``-E`` option (or lack +thereof) in log utilities to include sub-tasks or not. Internally, only ESI +tasks are subject to this filtering, but it can apply to tasks spawned by +VMODs too. -varnishadm -========== +.. XXX: any reason against ``varnish{hist,top} -k``? -**XXX changes concerning varnishadm(1) and/or varnish-cli(7)** +The ``-k`` option from ``varnishlog`` is now available in ``varnishncsa``. varnishstat =========== -**XXX changes concerning varnishstat(1) and/or varnish-counters(7)** +The unused counter ``MAIN.fetch_no_thread`` was repurposed and renamed to +``MAIN.bgfetch_no_thread`` to signal when background fetch tasks fail to +be scheduled because thread pools are saturated. + +To help estimate the rate of ``vsl_space`` consumption, the new counter +``MAIN.shm_bytes`` was added. It offers a finer-grained metric than the +existing ``MAIN.shm_cycles`` that depends on the ``vsl_space`` setting. + +A new contribution script called ``varnishstatdiff`` can be used to compare +the output of two ``varnishstat -1`` executions with a friendly diff format +for ``varnishstat``'s specific output. varnishtest =========== -**XXX changes concerning varnishtest(1) and/or vtc(7)** +New macros ``${pkg_version}`` and ``${pkg_branch}`` expanding respectively +to ``7.2.0`` and ``7.2`` for the current release. + +It is possible to match the text on screen against a regular expression +with the new ``process -match`` command. + +The new ``filewrite [-a]`` command can put or append text into a file. + +A Varnish instance name in a VTC is used by default as the server identity +for predictable Via headers. + +For example:: + + varnish v1 -vcl+backend { ... } + +The expected Via header is:: + + Via: 1.1 v1 (Varnish/7.2) + +The instance name can still be set to a different value using the ``-arg`` +command to change the ``varnishd -i`` option. Changes for developers and VMOD authors ======================================= -**XXX changes concerning VRT, the public APIs, source code organization, -builds etc.** +The ``varnishtest -i`` option only works from a Varnish source tree, in +which case the new macro ``${topsrc}`` is available in addition to the +old ``${topbuild}`` macro. + +The functions ``VRT_AddVDP()``, ``VRT_AddVFP()``, ``VRT_RemoveVDP()`` and +``VRT_RemoveVFP()`` are deprecated. + +The ``VCS_String()`` function can take the string ``"B"`` for the package +branch. + +The ``vnum.h`` functions are exposed to VMOD and VEXT authors. + +The termination rules for ``WRK_BgThread()`` were relaxed to allow VMODs to +use it. *eof* From dridi.boukelmoune at gmail.com Mon Sep 5 15:31:05 2022 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Mon, 5 Sep 2022 15:31:05 +0000 (UTC) Subject: [master] 6c7cf83a7 circleci: Ignore varnishstatdiff on Alpine Linux Message-ID: <20220905153105.C15991136FF@lists.varnish-cache.org> commit 6c7cf83a79ab3570b522c650af9dfb725156482b Author: Dridi Boukelmoune Date: Mon Sep 5 17:29:16 2022 +0200 circleci: Ignore varnishstatdiff on Alpine Linux There seems to be no output right now, and the test case is failing. diff --git a/.circleci/config.yml b/.circleci/config.yml index 7dc228ff0..accac691a 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -438,7 +438,7 @@ workflows: name: build_alpine dist: alpine release: "latest" - #extra_conf: --without-jemalloc + extra_conf: --without-contrib - build: name: build_archlinux dist: archlinux From dridi.boukelmoune at gmail.com Mon Sep 5 16:23:07 2022 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Mon, 5 Sep 2022 16:23:07 +0000 (UTC) Subject: [master] 8aee5e264 build: Always include the contrib/ subdirectory Message-ID: <20220905162307.5F348114F6F@lists.varnish-cache.org> commit 8aee5e264d9e5707c140c3dbab7e68915de210c9 Author: Dridi Boukelmoune Date: Mon Sep 5 18:14:17 2022 +0200 build: Always include the contrib/ subdirectory Otherwise a source tree bootstrapped --without-config would not distribute the contrib script and test, that are needed for the distcheck target and general redistribution. diff --git a/Makefile.am b/Makefile.am index d33f79a70..c0480e143 100644 --- a/Makefile.am +++ b/Makefile.am @@ -1,10 +1,6 @@ ACLOCAL_AMFLAGS = -I m4 -I . -SUBDIRS = include lib bin vmod etc doc man - -if WITH_CONTRIB -SUBDIRS += contrib -endif +SUBDIRS = include lib bin vmod etc doc man contrib TESTS = tools/magic_check.sh diff --git a/configure.ac b/configure.ac index a76725e30..e8aa9c757 100644 --- a/configure.ac +++ b/configure.ac @@ -892,6 +892,20 @@ VMOD_TESTS="$(cd $srcdir/vmod && echo tests/*.vtc)" AC_SUBST(VMOD_TESTS) AM_SUBST_NOTMAKE(VMOD_TESTS) +AC_ARG_WITH([contrib], + [AS_HELP_STRING([--with-contrib], + [Build Varnish with external contributions.])]) + +AM_CONDITIONAL([WITH_CONTRIB], [test "$with_contrib" = yes]) +CONTRIB_TESTS="$(cd $srcdir/contrib && echo tests/*.vtc)" +AC_SUBST(CONTRIB_TESTS) +AM_SUBST_NOTMAKE(CONTRIB_TESTS) + +AM_COND_IF([WITH_CONTRIB], [ + AC_DEFINE([WITH_CONTRIB], [1], + [Define to 1 when Varnish is built with contributions.]) +]) + # Make sure this include dir exists AC_CONFIG_COMMANDS([mkdir], [$MKDIR_P doc/sphinx/include]) @@ -907,6 +921,7 @@ AC_CONFIG_FILES([ bin/varnishhist/Makefile bin/varnishtest/Makefile bin/varnishncsa/Makefile + contrib/Makefile doc/Makefile doc/graphviz/Makefile doc/sphinx/Makefile @@ -925,21 +940,4 @@ AC_CONFIG_FILES([ vmod/Makefile ]) -AC_ARG_WITH([contrib], - [AS_HELP_STRING([--with-contrib], - [Build Varnish with external contributions.])]) - -AM_CONDITIONAL([WITH_CONTRIB], [test "$with_contrib" = yes]) - -AM_COND_IF([WITH_CONTRIB], [ - CONTRIB_TESTS="$(cd $srcdir/contrib && echo tests/*.vtc)" - AC_SUBST(CONTRIB_TESTS) - AM_SUBST_NOTMAKE(CONTRIB_TESTS) - - AC_DEFINE([WITH_CONTRIB], [1], - [Define to 1 when Varnish is built with contributions.]) - - AC_CONFIG_FILES([contrib/Makefile]) -]) - AC_OUTPUT diff --git a/contrib/Makefile.am b/contrib/Makefile.am index 54482b52b..59607f5bf 100644 --- a/contrib/Makefile.am +++ b/contrib/Makefile.am @@ -1,10 +1,15 @@ # +if !WITH_CONTRIB +dist_noinst_SCRIPTS = \ + varnishstatdiff +else dist_bin_SCRIPTS = \ varnishstatdiff TESTS = @CONTRIB_TESTS@ include $(top_srcdir)/vtc.am +endif -EXTRA_DIST = $(TESTS) +EXTRA_DIST = @CONTRIB_TESTS@ From phk at FreeBSD.org Tue Sep 6 08:24:05 2022 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Tue, 6 Sep 2022 08:24:05 +0000 (UTC) Subject: [master] 4711ac210 Sync libvgz with upstream zlib. Message-ID: <20220906082405.2D9BB107C3C@lists.varnish-cache.org> commit 4711ac210b48aea57bf3e5635f720b4dc3977a8f Author: Poul-Henning Kamp Date: Tue Sep 6 08:22:50 2022 +0000 Sync libvgz with upstream zlib. Varnish is not subject to CVE 2022 37434, we never use "extra" data. diff --git a/lib/libvgz/crc32.c b/lib/libvgz/crc32.c index 2c92d0acb..3ce5afd96 100644 --- a/lib/libvgz/crc32.c +++ b/lib/libvgz/crc32.c @@ -635,7 +635,7 @@ unsigned long ZEXPORT crc32_z(crc, buf, len) #endif /* DYNAMIC_CRC_TABLE */ /* Pre-condition the CRC */ - crc ^= 0xffffffff; + crc = (~crc) & 0xffffffff; /* Compute the CRC up to a word boundary. */ while (len && ((z_size_t)buf & 7) != 0) { @@ -754,7 +754,7 @@ unsigned long ZEXPORT crc32_z(crc, buf, len) #endif /* DYNAMIC_CRC_TABLE */ /* Pre-condition the CRC */ - crc ^= 0xffffffff; + crc = (~crc) & 0xffffffff; #ifdef W @@ -1082,7 +1082,7 @@ uLong ZEXPORT crc32_combine64(crc1, crc2, len2) #ifdef DYNAMIC_CRC_TABLE once(&made, make_crc_table); #endif /* DYNAMIC_CRC_TABLE */ - return multmodp(x2nmodp(len2, 3), crc1) ^ crc2; + return multmodp(x2nmodp(len2, 3), crc1) ^ (crc2 & 0xffffffff); } /* ========================================================================= */ @@ -1117,5 +1117,5 @@ uLong crc32_combine_op(crc1, crc2, op) uLong crc2; uLong op; { - return multmodp(op, crc1) ^ crc2; + return multmodp(op, crc1) ^ (crc2 & 0xffffffff); } diff --git a/lib/libvgz/inflate.c b/lib/libvgz/inflate.c index bb6db9ce9..9577e8312 100644 --- a/lib/libvgz/inflate.c +++ b/lib/libvgz/inflate.c @@ -771,8 +771,9 @@ int flush; if (copy > have) copy = have; if (copy) { if (state->head != Z_NULL && - state->head->extra != Z_NULL) { - len = state->head->extra_len - state->length; + state->head->extra != Z_NULL && + (len = state->head->extra_len - state->length) < + state->head->extra_max) { zmemcpy(state->head->extra + len, next, len + copy > state->head->extra_max ? state->head->extra_max - len : copy); diff --git a/lib/libvgz/zconf.h b/lib/libvgz/zconf.h index 5e1d68a00..5113d217e 100644 --- a/lib/libvgz/zconf.h +++ b/lib/libvgz/zconf.h @@ -8,6 +8,12 @@ #ifndef ZCONF_H #define ZCONF_H +#if defined(Z_SOLO) +#include +#define Z_U8 __uint64_t +#define Z_U4 __uint32_t +#endif + /* * If you *really* need a unique prefix for all types and library functions, * compile with -DZ_PREFIX. The "standard" zlib should be compiled without it. From dridi.boukelmoune at gmail.com Wed Sep 7 05:46:06 2022 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Wed, 7 Sep 2022 05:46:06 +0000 (UTC) Subject: [6.0] 388047efa varnishncsa: log request time in milliseconds Message-ID: <20220907054606.9E2B0106CB9@lists.varnish-cache.org> commit 388047efadcf93d3a61239a274c8d0581a8f8f76 Author: Asad Sajjad Ahmed Date: Thu Jan 14 11:01:23 2021 +0100 varnishncsa: log request time in milliseconds Allow the user to log the request time in milliseconds through the new format specifier: %{ms}T . Signed-off-by: Asad Sajjad Ahmed diff --git a/bin/varnishncsa/varnishncsa.c b/bin/varnishncsa/varnishncsa.c index c92b0a317..145199d10 100644 --- a/bin/varnishncsa/varnishncsa.c +++ b/bin/varnishncsa/varnishncsa.c @@ -348,10 +348,6 @@ format_time(const struct format *format) t_end = t_start; switch (format->time_type) { - case 'D': - AZ(VSB_printf(CTX.vsb, "%d", - (int)((t_end - t_start) * 1e6))); - break; case 't': AN(format->time_fmt); t = t_start; @@ -360,7 +356,16 @@ format_time(const struct format *format) AZ(VSB_cat(CTX.vsb, buf)); break; case 'T': - AZ(VSB_printf(CTX.vsb, "%d", (int)(t_end - t_start))); + AN(format->time_fmt); + if (!strcmp(format->time_fmt, "s")) /* same as %T */ + t = t_end - t_start; + else if (!strcmp(format->time_fmt, "ms")) + t = (t_end - t_start) * 1e3; + else if (!strcmp(format->time_fmt, "us")) /* same as %D */ + t = (t_end - t_start) * 1e6; + else + WRONG("Unreachable branch"); + AZ(VSB_printf(CTX.vsb, "%d", (int)t)); break; default: WRONG("Time format specifier"); @@ -498,12 +503,16 @@ addf_time(char type, const char *fmt) ALLOC_OBJ(f, FORMAT_MAGIC); AN(f); + AN(fmt); f->func = format_time; f->time_type = type; - if (fmt != NULL) { - f->time_fmt = strdup(fmt); - AN(f->time_fmt); - } + f->time_fmt = strdup(fmt); + AN(f->time_fmt); + + if (f->time_type == 'T' && strcmp(f->time_fmt, "s") && + strcmp(f->time_fmt, "ms") && strcmp(f->time_fmt, "us")) + VUT_Error(vut, 1, "Unknown specifier: %%{%s}T", f->time_fmt); + VTAILQ_INSERT_TAIL(&CTX.format, f, list); } @@ -719,7 +728,7 @@ parse_format(const char *format) addf_fragment(&CTX.frag[F_b], "-"); break; case 'D': /* Float request time */ - addf_time(*p, NULL); + addf_time('T', "us"); break; case 'h': /* Client host name / IP Address */ addf_fragment(&CTX.frag[F_h], "-"); @@ -752,7 +761,7 @@ parse_format(const char *format) addf_time(*p, TIME_FMT); break; case 'T': /* Int request time */ - addf_time(*p, NULL); + addf_time(*p, "s"); break; case 'u': /* Remote user from auth */ addf_auth(); @@ -782,6 +791,9 @@ parse_format(const char *format) case 't': addf_time(*q, buf); break; + case 'T': + addf_time(*q, buf); + break; case 'x': parse_x_format(buf); break; diff --git a/doc/sphinx/reference/varnishncsa.rst b/doc/sphinx/reference/varnishncsa.rst index 9c959d3ae..ee5e17d65 100644 --- a/doc/sphinx/reference/varnishncsa.rst +++ b/doc/sphinx/reference/varnishncsa.rst @@ -84,7 +84,7 @@ Supported formatters are: %D In client mode, time taken to serve the request, in microseconds. In backend mode, time from the request was sent to the entire body - had been received. + had been received. This is equivalent to %{us}T. %H The request protocol. Defaults to HTTP/1.0 if not known. @@ -138,7 +138,13 @@ Supported formatters are: %T In client mode, time taken to serve the request, in seconds. In backend mode, time from the request was sent to the entire body had - been received. + been received. This is equivalent to %{s}T. + +%{X}T + In client mode, time taken to serve the request, in the format + specified by X. In backend mode, time from the request was sent to + the entire body had been received. The time specification format can + be one of the following: s (same as %T), ms or us (same as %D). %U The request URL without the query string. Defaults to '-' if not From phk at FreeBSD.org Wed Sep 7 09:03:06 2022 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Wed, 7 Sep 2022 09:03:06 +0000 (UTC) Subject: [master] 14b62912a Document VEXT Message-ID: <20220907090306.48F3210F8F7@lists.varnish-cache.org> commit 14b62912a8d5ffe489005282144323df5165c076 Author: Poul-Henning Kamp Date: Wed Sep 7 09:02:44 2022 +0000 Document VEXT diff --git a/doc/sphinx/reference/index.rst b/doc/sphinx/reference/index.rst index 09ae6342d..cb50ea6ff 100644 --- a/doc/sphinx/reference/index.rst +++ b/doc/sphinx/reference/index.rst @@ -97,6 +97,7 @@ For Developers & DevOps Shell tricks VMODS - Extensions to VCL + VEXT - Varnish Extensions VSM - Shared memory use VDIR - Backends & Directors VCLI - CLI protocol API diff --git a/doc/sphinx/reference/vext.rst b/doc/sphinx/reference/vext.rst new file mode 100644 index 000000000..6570947d0 --- /dev/null +++ b/doc/sphinx/reference/vext.rst @@ -0,0 +1,29 @@ +.. + Copyright (c) 2010-2021 Varnish Software AS + SPDX-License-Identifier: BSD-2-Clause + See LICENSE file for full text of license + +.. _ref-vext: + +%%%%%%%%%%%%%%%%%%%%%%%%% +VEXT - Varnish Extensions +%%%%%%%%%%%%%%%%%%%%%%%%% + +A Varnish Extension is a shared library, loaded into the worker +process during startup, before privilges are dropped for good. This +allows a VEXT to do pretty much anything it wants to do in the +worker process. + +A VEXT can (also) contain a VMOD, and it will work just like any +other VMOD, which also means that VMODs can be loaded as VEXTs. + +The VEXTs are loaded in the child process, in the order they are +specified on the commandline, after the ``heritage`` has been +established, but before stevedores are initialized and jail +privileges are dropped. + +There is currently no ``init`` entrypoint defined, but a +VEXT can use a static initializer to get activated on loading. + +If those static initializers want to bail out, ``stderr`` and +``exit(3)`` can be used to convey diagnostics. diff --git a/doc/sphinx/reference/vtla.rst b/doc/sphinx/reference/vtla.rst index 0a6b04543..ab017c523 100644 --- a/doc/sphinx/reference/vtla.rst +++ b/doc/sphinx/reference/vtla.rst @@ -59,6 +59,9 @@ VEND VEV Varnish EVent -- library functions to implement a simple event-dispatcher. +VEXT + Varnish Extension -- Shared library loaded into the child process. + VGB Varnish Governing Board -- May or may not exist. If you need to ask, you are not on it. diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-trunk.rst index cc4b2ea61..f6745bb4e 100644 --- a/doc/sphinx/whats-new/changes-trunk.rst +++ b/doc/sphinx/whats-new/changes-trunk.rst @@ -23,7 +23,16 @@ varnishd Extensions ~~~~~~~~~~ -TODO: VEXT +From the very first days of Varnish, we have been talking about having +an extension points for "more advanced stuff" and we did, by and large, +keep a place ready for it in the overall architecture. + +Now a credible use-case finally appeared, and we have implemented +"Varnish Extensions" (VTLA: "VEXT"), which can both be used to load +ambient VMODs and to implement entirely new functionaly, for instance +stevedores. + +See :ref:`ref-vext` in the reference manual for more information. Parameters ~~~~~~~~~~ @@ -43,8 +52,11 @@ parameters. The latter still exist as deprecated aliases. Other changes in varnishd ~~~~~~~~~~~~~~~~~~~~~~~~~ -The VCC (compilation) process no longer loads VMODs using ``dlopen(3)`` to -collect their metadata. +The metadata VMODs exposes to Varnishd has changed to a non-binary +format, and it is incompatible with all previous releases. +That makes it possible for the VCC (compilation) process to avoid +opening the VMODs with ``dlopen(3)``, which is both faster and +safer. Background fetch tasks are no longer queued as this could result in slow grace hits subject to indefinite delays when thread pools are saturated. @@ -72,8 +84,8 @@ the case, but is now (pun intended) formally defined behavior. It keeps the same value even if the execution blocks for a significant time, for example while calling a VMOD function. -VMODs -===== +Bundled VMODs +============= For a real time timestamp, the function ``std.now()`` can be used instead. There is also a new ``std.timed_call()`` to measure the execution time of a From dridi.boukelmoune at gmail.com Wed Sep 7 15:21:07 2022 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Wed, 7 Sep 2022 15:21:07 +0000 (UTC) Subject: [master] 7941e75e5 changes: Mention SLT_Link[4] Message-ID: <20220907152107.103B511A401@lists.varnish-cache.org> commit 7941e75e59bf3d09c7dd6037815b94035d35cfdb Author: Dridi Boukelmoune Date: Wed Sep 7 17:19:26 2022 +0200 changes: Mention SLT_Link[4] diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-trunk.rst index f6745bb4e..9772d1e34 100644 --- a/doc/sphinx/whats-new/changes-trunk.rst +++ b/doc/sphinx/whats-new/changes-trunk.rst @@ -103,6 +103,8 @@ thereof) in log utilities to include sub-tasks or not. Internally, only ESI tasks are subject to this filtering, but it can apply to tasks spawned by VMODs too. +Similarly, the ``Link`` record has the same optional 4th field. + .. XXX: any reason against ``varnish{hist,top} -k``? The ``-k`` option from ``varnishlog`` is now available in ``varnishncsa``. From dridi.boukelmoune at gmail.com Wed Sep 7 15:21:07 2022 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Wed, 7 Sep 2022 15:21:07 +0000 (UTC) Subject: [master] e45e98d06 upgrading: First pass for the 7.2 release Message-ID: <20220907152107.2F03111A404@lists.varnish-cache.org> commit e45e98d06922aaa2fe29ab7f9357312eb6e57ebb Author: Dridi Boukelmoune Date: Wed Sep 7 17:20:05 2022 +0200 upgrading: First pass for the 7.2 release diff --git a/doc/sphinx/whats-new/upgrading-trunk.rst b/doc/sphinx/whats-new/upgrading-trunk.rst index 6143fde99..1f46dd179 100644 --- a/doc/sphinx/whats-new/upgrading-trunk.rst +++ b/doc/sphinx/whats-new/upgrading-trunk.rst @@ -8,26 +8,95 @@ released versions of Varnish, see:** :ref:`whats-new-index` Upgrading to Varnish **$NEXT_RELEASE** %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -**XXX: how to upgrade from previous deployments to this -version. Limited to work that has to be done for an upgrade, new -features are listed in "Changes". Explicitly mention what does *not* -have to be changed, especially in VCL. May include, but is not limited -to:** +varnishd +======== -* Elements of VCL that have been removed or are deprecated, or whose - semantics have changed. +Parameters +~~~~~~~~~~ -* -p parameters that have been removed or are deprecated, or whose - semantics have changed. +The following parameters are deprecated: -* Changes in the CLI. +- ``vcc_allow_inline_c`` +- ``vcc_err_unref`` +- ``vcc_unsafe_path`` -* Changes in the output or interpretation of stats or the log, including - changes affecting varnishncsa/-hist/-top. +They can still be set as individual boolean parameters. The deprecated +aliases will be removed in a future release. -* Changes that may be necessary in VTCs or in the use of varnishtest. +They are replaced by the ``vcc_feature`` bits parameter: -* Changes in public APIs that may require changes in VMODs or VAPI/VUT - clients. +- ``allow_inline_c`` +- ``err_unref`` (enabled by default) +- ``unsafe_path`` (enabled by default) + +The following commands are equivalent:: + + param.set vcc_err_unref off + param.set vcc_feature -err_unref + +Identity +~~~~~~~~ + +The server identity must be a valid HTTP token, which may pose a problem +to existing setups. For example ``varnishd -i "edge server 1"`` is no +longer accepted. You can use something like ``varnishd -i "edge-server-1"`` +instead. + +VCL +=== + +Varnish generates a Via header and forwards it to the backend by default. +This can be prevented for example in ``vcl_recv`` or ``vcl_backend_fetch``. + +:: + + sub vcl_recv { + unset req.http.via; + } + + sub vcl_backend_fetch { + unset bereq.http.via; + } + +The Via header is generated with the ``server.identity`` variable for +the ``received-by`` field. See `rfc9110_` for a description of the Via +header. + +.. _rfc9110: https://www.rfc-editor.org/rfc/rfc9110#name-via + +VMODs +===== + +Cookies generated by vmod_cookie used to have a trailing semi-colon that +goes against the recommandations from `rfc6265`_. This should not pose a +problem, unless a piece of VCL code or a backend have come to rely on this +incorrect behavior. + +.. _rfc6265: https://www.rfc-editor.org/rfc/rfc6265 + +varnishlog +========== + +The ``Begin`` and ``Link`` log records have an optional 4th field for the +sub-request level. This may break log processors, log queries or NCSA +formats that expect those records to have exactly 3 fields. + +varnishstat +=========== + +The ``MAIN.fetch_no_thread`` counter is gone, it never worked. Track the +``MAIN.bgfetch_no_thread`` counter instead. + +Changes for developers and VMOD authors +======================================= + +The functions ``VRT_AddVDP()``, ``VRT_AddVFP()``, ``VRT_RemoveVDP()`` and +``VRT_RemoveVFP()`` are deprecated. Use ``VRT_AddFilter()`` to add a pair +(VFP and VDP) and ``VRT_RemoveFilter()`` to remove it. A filter pair needs +at least one member of the pair. + +The ``varnishtest -i`` option no longer works outside of a Varnish source +tree. There shouldn't be a reason to use ``-i`` outside of the Varnish +test suite. *eof* From dridi.boukelmoune at gmail.com Mon Sep 12 14:03:11 2022 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Mon, 12 Sep 2022 14:03:11 +0000 (UTC) Subject: [master] 20d356b3d tranport: Increment MAIN.client_req exactly once Message-ID: <20220912140311.421AB62D62@lists.varnish-cache.org> commit 20d356b3d08a458c6f56117a65f9041b5f6fd405 Author: Lachlan Abbott Date: Wed Sep 7 16:43:34 2022 +1000 tranport: Increment MAIN.client_req exactly once When a request comes back from the waiting list, it is accounted for in the client_req counter, despite not being a new request. To avoid the spurious increments, a request is counted only after leaving the state machine for good. This has no effect on when the client_req increment becomes visible since it is part of the "wrk" group in the first place. In 2e66e0b93ba6e50fa5ef499cd6a4860d0fa56134 the client_req increment was moved from the request state machine (R_STP_TRANSPORT) to individual transport. This patch can't be reverted because shortly aftwerwards the ESI transport changed in 2c59afe22ba108c7f2ab5ff75b2e30baf4ef9ec7 and now runs the R_STP_TRANSPORT step and the client_req counter does not count sub-requests. Refs 2e66e0b93ba6e50fa5ef499cd6a4860d0fa56134 Refs 2c59afe22ba108c7f2ab5ff75b2e30baf4ef9ec7 diff --git a/bin/varnishd/http1/cache_http1_fsm.c b/bin/varnishd/http1/cache_http1_fsm.c index cc61649fd..ab92dcf38 100644 --- a/bin/varnishd/http1/cache_http1_fsm.c +++ b/bin/varnishd/http1/cache_http1_fsm.c @@ -386,10 +386,10 @@ HTTP1_Session(struct worker *wrk, struct req *req) } else if (st == H1PROC) { req->task->func = http1_req; req->task->priv = req; - wrk->stats->client_req++; CNT_Embark(wrk, req); if (CNT_Request(req) == REQ_FSM_DISEMBARK) return; + wrk->stats->client_req++; AZ(req->top->vcl0); req->task->func = NULL; req->task->priv = NULL; diff --git a/bin/varnishd/http2/cache_http2_proto.c b/bin/varnishd/http2/cache_http2_proto.c index c408acd65..63c491fbd 100644 --- a/bin/varnishd/http2/cache_http2_proto.c +++ b/bin/varnishd/http2/cache_http2_proto.c @@ -540,8 +540,8 @@ h2_do_req(struct worker *wrk, void *priv) THR_SetRequest(req); CNT_Embark(wrk, req); - wrk->stats->client_req++; if (CNT_Request(req) != REQ_FSM_DISEMBARK) { + wrk->stats->client_req++; assert(!WS_IsReserved(req->ws)); AZ(req->top->vcl0); h2 = r2->h2sess; diff --git a/bin/varnishtest/tests/c00013.vtc b/bin/varnishtest/tests/c00013.vtc index ef4cc76f8..be2a734c9 100644 --- a/bin/varnishtest/tests/c00013.vtc +++ b/bin/varnishtest/tests/c00013.vtc @@ -47,6 +47,7 @@ client c2 { client c1 -wait varnish v1 -vsl_catchup +varnish v1 -expect client_req == 2 varnish v1 -expect busy_sleep >= 1 varnish v1 -expect busy_wakeup >= 1 varnish v1 -stop diff --git a/bin/varnishtest/tests/t02009.vtc b/bin/varnishtest/tests/t02009.vtc index 511f54f67..223149531 100644 --- a/bin/varnishtest/tests/t02009.vtc +++ b/bin/varnishtest/tests/t02009.vtc @@ -29,3 +29,5 @@ client c1 { } -run stream 1 -wait } -run + +varnish v1 -expect client_req == 2 From dridi.boukelmoune at gmail.com Mon Sep 12 14:22:05 2022 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Mon, 12 Sep 2022 14:22:05 +0000 (UTC) Subject: [master] 6b5ad52f6 changes: Mention #3841 Message-ID: <20220912142205.B56F963979@lists.varnish-cache.org> commit 6b5ad52f64177aba07e680d3961afdd075e86b22 Author: Dridi Boukelmoune Date: Mon Sep 12 16:21:04 2022 +0200 changes: Mention #3841 diff --git a/doc/changes.rst b/doc/changes.rst index 200c3cfd2..fc340426a 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -117,6 +117,11 @@ Varnish Cache NEXT (2022-09-15) * There is a new ``contrib/`` directory in the Varnish source tree. The first contribution is a ``varnishstatdiff`` script. +* A regression in the transport code led MAIN.client_req to be incremented + for requests coming back from the waiting list, it was fixed. (3841_) + +.. _3841: https://github.com/varnishcache/varnish-cache/pull/3841 + ================================ Varnish Cache 7.1.0 (2022-03-15) ================================ From nils.goroll at uplex.de Mon Sep 12 17:44:06 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 12 Sep 2022 17:44:06 +0000 (UTC) Subject: [master] eb7004aa9 Fodder for #3844 discussion Message-ID: <20220912174406.4F851104912@lists.varnish-cache.org> commit eb7004aa90c6a000dfe08ea15d8a6f28bb263b63 Author: Nils Goroll Date: Mon Sep 12 19:40:20 2022 +0200 Fodder for #3844 discussion diff --git a/bin/varnishtest/tests/v00020.vtc b/bin/varnishtest/tests/v00020.vtc index 96c318c57..848c7252e 100644 --- a/bin/varnishtest/tests/v00020.vtc +++ b/bin/varnishtest/tests/v00020.vtc @@ -480,6 +480,13 @@ varnish v1 -vcl+backend { (resp.http.foo + resp.http.bar) == ("X" + resp.http.foo); // Ticket 2809 set resp.http.bar = true == false; + + set resp.http.a = req.http.inexistent; + // correct? #3844 + set resp.http.a-eq-inexistent = resp.http.a == req.http.inexistent; + // discussion in #3844 - and should !!hdr work? + set resp.http.is-a = !(!resp.http.a); + set resp.http.is-inexistent = !(!req.http.inexistent); } sub vcl_deliver { set resp.http.p = (0 + 999999999999999); @@ -499,6 +506,13 @@ client c1 { expect resp.http.p == 999999999999999 expect resp.http.n == -999999999999999 expect resp.http.o == 200 + + expect resp.http.a == "" + # correct? #3844 + expect resp.http.a-eq-inexistent == "false" + # discussion in #3844 + expect resp.http.is-a == "true" + expect resp.http.is-inexistent == "false" } -run varnish v1 -vcl { From martin at varnish-software.com Thu Sep 15 08:35:06 2022 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Thu, 15 Sep 2022 08:35:06 +0000 (UTC) Subject: [master] 085c21d3f Updating devicedetect.vcl to upstream Message-ID: <20220915083506.5313B100C9F@lists.varnish-cache.org> commit 085c21d3fe9cf6ddc8ae24f47359f934ce775c27 Author: Martin Blix Grydeland Date: Thu Sep 15 10:33:39 2022 +0200 Updating devicedetect.vcl to upstream diff --git a/etc/devicedetect.vcl b/etc/devicedetect.vcl index 006cc6843..88cfe6721 100644 --- a/etc/devicedetect.vcl +++ b/etc/devicedetect.vcl @@ -34,7 +34,7 @@ sub devicedetect { unset req.http.X-UA-Device; set req.http.X-UA-Device = "pc"; - # Handle that a cookie may override the detection alltogether. + # Handle that a cookie may override the detection altogether. if (req.http.Cookie ~ "(?i)X-UA-Device-force") { /* ;?? means zero or one ;, non-greedy to match the first. */ set req.http.X-UA-Device = regsub(req.http.Cookie, "(?i).*X-UA-Device-force=([^;]+);??.*", "\1"); From nils.goroll at uplex.de Thu Sep 15 09:32:05 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 15 Sep 2022 09:32:05 +0000 (UTC) Subject: [master] df683e46f Add std.cache_req_body() coverage Message-ID: <20220915093205.63BCD102888@lists.varnish-cache.org> commit df683e46f38f61dc8807120569e7b6ea96e89f4f Author: Nils Goroll Date: Thu Sep 15 11:29:22 2022 +0200 Add std.cache_req_body() coverage Ref c23ef4c3d874871061792add005fdfda9912db43 870f16cebdf2aac9e9a5e0c8d50c4e632528ae54 Closes #3846 diff --git a/bin/varnishtest/tests/c00055.vtc b/bin/varnishtest/tests/c00055.vtc index 10c87f292..6e97e1d40 100644 --- a/bin/varnishtest/tests/c00055.vtc +++ b/bin/varnishtest/tests/c00055.vtc @@ -14,6 +14,9 @@ varnish v1 -vcl+backend { import std; sub vcl_recv { + if (req.url == "/wrong-sub") { + return (pass); + } if (std.cache_req_body(1KB)) { set req.http.stored = true; } else { @@ -27,6 +30,15 @@ varnish v1 -vcl+backend { } set resp.http.stored = req.http.stored; } + sub vcl_backend_fetch { + if (bereq.url == "/wrong-sub") { + if (std.cache_req_body(1KB)) { + return (error(200)); + } else { + return (error(503)); + } + } + } } -start # check log for the aborted POST @@ -60,6 +72,12 @@ server s1 { txresp } -start +client c1 { + txreq -url "/wrong-sub" + rxresp + expect resp.status == 503 +} -run + client c1 { txreq -url "/is_varnish_still_running" rxresp @@ -72,5 +90,16 @@ client c2 { expect_close expect resp.http.stored == } -run + +varnish v1 -errvcl {req.body can only be cached in vcl_recv} { + import std; + backend none none; + sub vcl_init { + if (! std.cache_req_body(1KB)) { + return (fail); + } + } +} + varnish v1 -stop logexpect l1 -wait From nils.goroll at uplex.de Thu Sep 15 09:38:05 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 15 Sep 2022 09:38:05 +0000 (UTC) Subject: [master] d72caa00c Bump vrt major version Message-ID: <20220915093805.E172D102C9E@lists.varnish-cache.org> commit d72caa00c2834474194538d126d0fb5c62b8be47 Author: Nils Goroll Date: Thu Sep 15 11:35:14 2022 +0200 Bump vrt major version due to vrt_ctx change in a8483cf5c864a87f59b2cc26315f37e9edda2d97 diff --git a/include/vrt.h b/include/vrt.h index e5e5647bf..032012a1f 100644 --- a/include/vrt.h +++ b/include/vrt.h @@ -46,7 +46,7 @@ # error "include vdef.h before vrt.h" #endif -#define VRT_MAJOR_VERSION 15U +#define VRT_MAJOR_VERSION 16U #define VRT_MINOR_VERSION 0U From nils.goroll at uplex.de Thu Sep 15 10:09:05 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 15 Sep 2022 10:09:05 +0000 (UTC) Subject: [master] df2049d79 Last-minute changes.rst polishing Message-ID: <20220915100905.35E47103C04@lists.varnish-cache.org> commit df2049d79d862004b14ce2370c897d904315fe62 Author: Nils Goroll Date: Thu Sep 15 11:53:24 2022 +0200 Last-minute changes.rst polishing The #3830 info is taken from 137d9814d7b214971c3ddcb04a6d68ca479f0a10 diff --git a/doc/changes.rst b/doc/changes.rst index fc340426a..49c883356 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -42,6 +42,10 @@ Varnish Cache NEXT (2022-09-15) semi-colon (``';'``) at the end of the string. This could break VCL relying on the previous incorrect behavior. +* The ``SessClose`` and ``BackendClose`` reason ``rx_body``, which + previously output ``Failure receiving req.body``, has been rewritten + to ``Failure receiving body``. + * Prototypical Varnish Extensions (VEXT). Similar to VMODs, a VEXT is loaded by the cache process. Unlike VMODs that have the combined lifetime of all the VCLs that reference them, a VEXT has the lifetime of the cache process @@ -50,11 +54,18 @@ Varnish Cache NEXT (2022-09-15) * The VCC (compilation) process no longer loads VMODs with ``dlopen(3)`` to collect their metadata. +* Stevedore initialization via the ``.init()`` callback has been moved + to the worker process. + * The parameter ``tcp_keepalive_time`` is supported on MacOS. * Duration parameters can optionally take a unit, with the same syntax as duration units in VCL. Example: ``param.set default_grace 1h``. +* Calls to ``VRT_CacheReqBody()`` and ``std.cache_req_body`` from outside + client vcl subs now fail properly instead of triggering an + assertion failure (3846_). + * New ``"B"`` string for the package branch in ``VCS_String()``. For the 7.2.0 version, it would yield the 7.2 branch. @@ -84,6 +95,9 @@ Varnish Cache NEXT (2022-09-15) Via headers are appended in both directions, to work with other hops that may advertise themselves. +* A ``resp.http.via`` header is no longer overwritten by varnish, but + rather appended to. + * The ``server.identity`` syntax is now limited to a "token" as defined in the HTTP grammar to be suitable for Via headers. @@ -93,9 +107,21 @@ Varnish Cache NEXT (2022-09-15) * VMOD and VEXT authors can use functions from ``vnum.h``. +* Do not filter pseudo-headers as regular headers (VSV00009_ / 3830_). + * The termination rules for ``WRK_BgThread()`` were relaxed to allow VMODs to use it. +* ``(struct worker).handling`` has been moved to the newly introduced + ``struct wrk_vpi`` and replaced by a pointer to it, as well as + ``(struct vrt_ctx).handling`` has been replaced by that pointer. + + ``struct wrk_vpi`` is for state at the interface between VRT and VGC + and, in particular, is not const as ``struct vrt_ctx`` aka + ``VRT_CTX``. + +* Panics now contain information about VCL source files and lines. + * The ``Begin`` log record has a 4th field for subtasks like ESI sub-requests. * The ``-E`` option for log utilities now works as documented, with any type @@ -114,13 +140,21 @@ Varnish Cache NEXT (2022-09-15) * The unused ``fetch_no_thread`` counter was renamed to ``bgfetch_no_thread`` because regular backend fetch tasks are always scheduled. +* The macros ``FEATURE()``, ``EXPERIMENT()``, ``DO_DEBUG()``, + ``MGT_FEATURE()``, ``MGT_EXPERIMENT()``, ``MGT_DO_DEBUG()`` and + ``MGT_VCC_FEATURE()`` now return a boolean value (``0`` or ``1``) + instead of the (private) flag value. + * There is a new ``contrib/`` directory in the Varnish source tree. The first contribution is a ``varnishstatdiff`` script. * A regression in the transport code led MAIN.client_req to be incremented for requests coming back from the waiting list, it was fixed. (3841_) +.. _3830: https://github.com/varnishcache/varnish-cache/issues/3830 .. _3841: https://github.com/varnishcache/varnish-cache/pull/3841 +.. _3846: https://github.com/varnishcache/varnish-cache/issues/3846 +.. _VSV00009: https://varnish-cache.org/security/VSV00009.html ================================ Varnish Cache 7.1.0 (2022-03-15) From nils.goroll at uplex.de Thu Sep 15 10:32:04 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 15 Sep 2022 10:32:04 +0000 (UTC) Subject: [master] 619d92a71 Last-minute release docs polishing Message-ID: <20220915103204.EB8AA1048D7@lists.varnish-cache.org> commit 619d92a71797c1a30545126bd40fe88783dedd50 Author: Nils Goroll Date: Thu Sep 15 12:13:33 2022 +0200 Last-minute release docs polishing I think that the other late additions to changes.rst do not qualify for the release docs. Thank you to Dridi and phk for preparing most of the docs and thank you to Martin for the opportunity for some last minute additions. diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-trunk.rst index 9772d1e34..dd3113f43 100644 --- a/doc/sphinx/whats-new/changes-trunk.rst +++ b/doc/sphinx/whats-new/changes-trunk.rst @@ -75,6 +75,9 @@ which defaults to the host name. A ``req.http.via`` header is generated also before entering ``vcl_recv``. If a client request or backend response already had a Via header, it is now appended to instead of overwritten. +A ``resp.http.via`` header is no longer overwritten by varnish, but +rather appended to. + The ``server.identity`` variable is guaranteed to be a single token as defined in the HTTP grammar, to safely be used as either a host name or pseudonym in Via headers. diff --git a/doc/sphinx/whats-new/upgrading-trunk.rst b/doc/sphinx/whats-new/upgrading-trunk.rst index 1f46dd179..d5f8566c2 100644 --- a/doc/sphinx/whats-new/upgrading-trunk.rst +++ b/doc/sphinx/whats-new/upgrading-trunk.rst @@ -62,6 +62,9 @@ The Via header is generated with the ``server.identity`` variable for the ``received-by`` field. See `rfc9110_` for a description of the Via header. +A ``resp.http.via`` header is no longer overwritten by varnish, but +rather appended to. + .. _rfc9110: https://www.rfc-editor.org/rfc/rfc9110#name-via VMODs From nils.goroll at uplex.de Thu Sep 15 10:32:05 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 15 Sep 2022 10:32:05 +0000 (UTC) Subject: [master] cc1c4fd76 Track VRT major version change in test cases Message-ID: <20220915103205.0AF541048DA@lists.varnish-cache.org> commit cc1c4fd76819dcb6608ba2ab3f3bb294e17aab1a Author: Nils Goroll Date: Thu Sep 15 12:27:34 2022 +0200 Track VRT major version change in test cases Ref d72caa00c2834474194538d126d0fb5c62b8be47 diff --git a/bin/varnishtest/tests/m00003.vtc b/bin/varnishtest/tests/m00003.vtc index 1660b43b3..70dd4ec9c 100644 --- a/bin/varnishtest/tests/m00003.vtc +++ b/bin/varnishtest/tests/m00003.vtc @@ -96,7 +96,7 @@ filewrite -a ${tmpdir}/libvmod_wrong.so "\x03" varnish v1 -errvcl {VMOD wants ABI version 1.0} { import wrong; } ############################################################# -# NB: in the tests below "15" should track VRT_MAJOR_VERSION +# NB: in the tests below "16" should track VRT_MAJOR_VERSION filewrite ${tmpdir}/libvmod_wrong.so "VMOD_JSON_SPEC\x02" filewrite -a ${tmpdir}/libvmod_wrong.so { @@ -108,7 +108,7 @@ filewrite -a ${tmpdir}/libvmod_wrong.so { "Vmod_vmod_wrong_Func", "0000000000000000000000000000000000000000000000000000000000000000", "0000000000000000000000000000000000000000000000000000000000000000", - "15", + "16", "0" ], [ "$FOOBAR" @@ -128,7 +128,7 @@ filewrite -a ${tmpdir}/libvmod_wrong.so { "Vmod_vmod_wrong_Func", "0000000000000000000000000000000000000000000000000000000000000000", "0000000000000000000000000000000000000000000000000000000000000000", - "15", + "16", "0" ] ] @@ -146,7 +146,7 @@ filewrite -a ${tmpdir}/libvmod_wrong.so { "Vmod_vmod_wrong_Func", "0000000000000000000000000000000000000000000000000000000000000000", "0000000000000000000000000000000000000000000000000000000000000000", - "15", + "16", "0" ], [ "$CPROTO" @@ -168,7 +168,7 @@ filewrite -a ${tmpdir}/libvmod_wrong.so { "Vmod_vmod_std_Func", "0000000000000000000000000000000000000000000000000000000000000000", "0000000000000000000000000000000000000000000000000000000000000000", - "15", + "16", "0" ], [ "$CPROTO", "/* blabla */" From nils.goroll at uplex.de Thu Sep 15 10:51:03 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 15 Sep 2022 10:51:03 +0000 (UTC) Subject: [7.2] df683e46f Add std.cache_req_body() coverage Message-ID: <20220915105103.43E7510542F@lists.varnish-cache.org> commit df683e46f38f61dc8807120569e7b6ea96e89f4f Author: Nils Goroll Date: Thu Sep 15 11:29:22 2022 +0200 Add std.cache_req_body() coverage Ref c23ef4c3d874871061792add005fdfda9912db43 870f16cebdf2aac9e9a5e0c8d50c4e632528ae54 Closes #3846 diff --git a/bin/varnishtest/tests/c00055.vtc b/bin/varnishtest/tests/c00055.vtc index 10c87f292..6e97e1d40 100644 --- a/bin/varnishtest/tests/c00055.vtc +++ b/bin/varnishtest/tests/c00055.vtc @@ -14,6 +14,9 @@ varnish v1 -vcl+backend { import std; sub vcl_recv { + if (req.url == "/wrong-sub") { + return (pass); + } if (std.cache_req_body(1KB)) { set req.http.stored = true; } else { @@ -27,6 +30,15 @@ varnish v1 -vcl+backend { } set resp.http.stored = req.http.stored; } + sub vcl_backend_fetch { + if (bereq.url == "/wrong-sub") { + if (std.cache_req_body(1KB)) { + return (error(200)); + } else { + return (error(503)); + } + } + } } -start # check log for the aborted POST @@ -60,6 +72,12 @@ server s1 { txresp } -start +client c1 { + txreq -url "/wrong-sub" + rxresp + expect resp.status == 503 +} -run + client c1 { txreq -url "/is_varnish_still_running" rxresp @@ -72,5 +90,16 @@ client c2 { expect_close expect resp.http.stored == } -run + +varnish v1 -errvcl {req.body can only be cached in vcl_recv} { + import std; + backend none none; + sub vcl_init { + if (! std.cache_req_body(1KB)) { + return (fail); + } + } +} + varnish v1 -stop logexpect l1 -wait From nils.goroll at uplex.de Thu Sep 15 10:51:03 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 15 Sep 2022 10:51:03 +0000 (UTC) Subject: [7.2] d72caa00c Bump vrt major version Message-ID: <20220915105103.5C32D105433@lists.varnish-cache.org> commit d72caa00c2834474194538d126d0fb5c62b8be47 Author: Nils Goroll Date: Thu Sep 15 11:35:14 2022 +0200 Bump vrt major version due to vrt_ctx change in a8483cf5c864a87f59b2cc26315f37e9edda2d97 diff --git a/include/vrt.h b/include/vrt.h index e5e5647bf..032012a1f 100644 --- a/include/vrt.h +++ b/include/vrt.h @@ -46,7 +46,7 @@ # error "include vdef.h before vrt.h" #endif -#define VRT_MAJOR_VERSION 15U +#define VRT_MAJOR_VERSION 16U #define VRT_MINOR_VERSION 0U From nils.goroll at uplex.de Thu Sep 15 10:51:03 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 15 Sep 2022 10:51:03 +0000 (UTC) Subject: [7.2] df2049d79 Last-minute changes.rst polishing Message-ID: <20220915105103.7C08E105437@lists.varnish-cache.org> commit df2049d79d862004b14ce2370c897d904315fe62 Author: Nils Goroll Date: Thu Sep 15 11:53:24 2022 +0200 Last-minute changes.rst polishing The #3830 info is taken from 137d9814d7b214971c3ddcb04a6d68ca479f0a10 diff --git a/doc/changes.rst b/doc/changes.rst index fc340426a..49c883356 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -42,6 +42,10 @@ Varnish Cache NEXT (2022-09-15) semi-colon (``';'``) at the end of the string. This could break VCL relying on the previous incorrect behavior. +* The ``SessClose`` and ``BackendClose`` reason ``rx_body``, which + previously output ``Failure receiving req.body``, has been rewritten + to ``Failure receiving body``. + * Prototypical Varnish Extensions (VEXT). Similar to VMODs, a VEXT is loaded by the cache process. Unlike VMODs that have the combined lifetime of all the VCLs that reference them, a VEXT has the lifetime of the cache process @@ -50,11 +54,18 @@ Varnish Cache NEXT (2022-09-15) * The VCC (compilation) process no longer loads VMODs with ``dlopen(3)`` to collect their metadata. +* Stevedore initialization via the ``.init()`` callback has been moved + to the worker process. + * The parameter ``tcp_keepalive_time`` is supported on MacOS. * Duration parameters can optionally take a unit, with the same syntax as duration units in VCL. Example: ``param.set default_grace 1h``. +* Calls to ``VRT_CacheReqBody()`` and ``std.cache_req_body`` from outside + client vcl subs now fail properly instead of triggering an + assertion failure (3846_). + * New ``"B"`` string for the package branch in ``VCS_String()``. For the 7.2.0 version, it would yield the 7.2 branch. @@ -84,6 +95,9 @@ Varnish Cache NEXT (2022-09-15) Via headers are appended in both directions, to work with other hops that may advertise themselves. +* A ``resp.http.via`` header is no longer overwritten by varnish, but + rather appended to. + * The ``server.identity`` syntax is now limited to a "token" as defined in the HTTP grammar to be suitable for Via headers. @@ -93,9 +107,21 @@ Varnish Cache NEXT (2022-09-15) * VMOD and VEXT authors can use functions from ``vnum.h``. +* Do not filter pseudo-headers as regular headers (VSV00009_ / 3830_). + * The termination rules for ``WRK_BgThread()`` were relaxed to allow VMODs to use it. +* ``(struct worker).handling`` has been moved to the newly introduced + ``struct wrk_vpi`` and replaced by a pointer to it, as well as + ``(struct vrt_ctx).handling`` has been replaced by that pointer. + + ``struct wrk_vpi`` is for state at the interface between VRT and VGC + and, in particular, is not const as ``struct vrt_ctx`` aka + ``VRT_CTX``. + +* Panics now contain information about VCL source files and lines. + * The ``Begin`` log record has a 4th field for subtasks like ESI sub-requests. * The ``-E`` option for log utilities now works as documented, with any type @@ -114,13 +140,21 @@ Varnish Cache NEXT (2022-09-15) * The unused ``fetch_no_thread`` counter was renamed to ``bgfetch_no_thread`` because regular backend fetch tasks are always scheduled. +* The macros ``FEATURE()``, ``EXPERIMENT()``, ``DO_DEBUG()``, + ``MGT_FEATURE()``, ``MGT_EXPERIMENT()``, ``MGT_DO_DEBUG()`` and + ``MGT_VCC_FEATURE()`` now return a boolean value (``0`` or ``1``) + instead of the (private) flag value. + * There is a new ``contrib/`` directory in the Varnish source tree. The first contribution is a ``varnishstatdiff`` script. * A regression in the transport code led MAIN.client_req to be incremented for requests coming back from the waiting list, it was fixed. (3841_) +.. _3830: https://github.com/varnishcache/varnish-cache/issues/3830 .. _3841: https://github.com/varnishcache/varnish-cache/pull/3841 +.. _3846: https://github.com/varnishcache/varnish-cache/issues/3846 +.. _VSV00009: https://varnish-cache.org/security/VSV00009.html ================================ Varnish Cache 7.1.0 (2022-03-15) From nils.goroll at uplex.de Thu Sep 15 10:51:03 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 15 Sep 2022 10:51:03 +0000 (UTC) Subject: [7.2] 619d92a71 Last-minute release docs polishing Message-ID: <20220915105103.9798910543B@lists.varnish-cache.org> commit 619d92a71797c1a30545126bd40fe88783dedd50 Author: Nils Goroll Date: Thu Sep 15 12:13:33 2022 +0200 Last-minute release docs polishing I think that the other late additions to changes.rst do not qualify for the release docs. Thank you to Dridi and phk for preparing most of the docs and thank you to Martin for the opportunity for some last minute additions. diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-trunk.rst index 9772d1e34..dd3113f43 100644 --- a/doc/sphinx/whats-new/changes-trunk.rst +++ b/doc/sphinx/whats-new/changes-trunk.rst @@ -75,6 +75,9 @@ which defaults to the host name. A ``req.http.via`` header is generated also before entering ``vcl_recv``. If a client request or backend response already had a Via header, it is now appended to instead of overwritten. +A ``resp.http.via`` header is no longer overwritten by varnish, but +rather appended to. + The ``server.identity`` variable is guaranteed to be a single token as defined in the HTTP grammar, to safely be used as either a host name or pseudonym in Via headers. diff --git a/doc/sphinx/whats-new/upgrading-trunk.rst b/doc/sphinx/whats-new/upgrading-trunk.rst index 1f46dd179..d5f8566c2 100644 --- a/doc/sphinx/whats-new/upgrading-trunk.rst +++ b/doc/sphinx/whats-new/upgrading-trunk.rst @@ -62,6 +62,9 @@ The Via header is generated with the ``server.identity`` variable for the ``received-by`` field. See `rfc9110_` for a description of the Via header. +A ``resp.http.via`` header is no longer overwritten by varnish, but +rather appended to. + .. _rfc9110: https://www.rfc-editor.org/rfc/rfc9110#name-via VMODs From nils.goroll at uplex.de Thu Sep 15 10:51:03 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 15 Sep 2022 10:51:03 +0000 (UTC) Subject: [7.2] cc1c4fd76 Track VRT major version change in test cases Message-ID: <20220915105103.B8F7810543F@lists.varnish-cache.org> commit cc1c4fd76819dcb6608ba2ab3f3bb294e17aab1a Author: Nils Goroll Date: Thu Sep 15 12:27:34 2022 +0200 Track VRT major version change in test cases Ref d72caa00c2834474194538d126d0fb5c62b8be47 diff --git a/bin/varnishtest/tests/m00003.vtc b/bin/varnishtest/tests/m00003.vtc index 1660b43b3..70dd4ec9c 100644 --- a/bin/varnishtest/tests/m00003.vtc +++ b/bin/varnishtest/tests/m00003.vtc @@ -96,7 +96,7 @@ filewrite -a ${tmpdir}/libvmod_wrong.so "\x03" varnish v1 -errvcl {VMOD wants ABI version 1.0} { import wrong; } ############################################################# -# NB: in the tests below "15" should track VRT_MAJOR_VERSION +# NB: in the tests below "16" should track VRT_MAJOR_VERSION filewrite ${tmpdir}/libvmod_wrong.so "VMOD_JSON_SPEC\x02" filewrite -a ${tmpdir}/libvmod_wrong.so { @@ -108,7 +108,7 @@ filewrite -a ${tmpdir}/libvmod_wrong.so { "Vmod_vmod_wrong_Func", "0000000000000000000000000000000000000000000000000000000000000000", "0000000000000000000000000000000000000000000000000000000000000000", - "15", + "16", "0" ], [ "$FOOBAR" @@ -128,7 +128,7 @@ filewrite -a ${tmpdir}/libvmod_wrong.so { "Vmod_vmod_wrong_Func", "0000000000000000000000000000000000000000000000000000000000000000", "0000000000000000000000000000000000000000000000000000000000000000", - "15", + "16", "0" ] ] @@ -146,7 +146,7 @@ filewrite -a ${tmpdir}/libvmod_wrong.so { "Vmod_vmod_wrong_Func", "0000000000000000000000000000000000000000000000000000000000000000", "0000000000000000000000000000000000000000000000000000000000000000", - "15", + "16", "0" ], [ "$CPROTO" @@ -168,7 +168,7 @@ filewrite -a ${tmpdir}/libvmod_wrong.so { "Vmod_vmod_std_Func", "0000000000000000000000000000000000000000000000000000000000000000", "0000000000000000000000000000000000000000000000000000000000000000", - "15", + "16", "0" ], [ "$CPROTO", "/* blabla */" From martin at varnish-software.com Thu Sep 15 11:12:04 2022 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Thu, 15 Sep 2022 11:12:04 +0000 (UTC) Subject: [7.2] 75d4c1de9 Prepare for 7.2.0 Message-ID: <20220915111204.F27141064B5@lists.varnish-cache.org> commit 75d4c1de9673da2ae3df3904fae960d8ae534a00 Author: Martin Blix Grydeland Date: Thu Sep 15 10:38:30 2022 +0200 Prepare for 7.2.0 diff --git a/configure.ac b/configure.ac index e8aa9c757..0eb3af56f 100644 --- a/configure.ac +++ b/configure.ac @@ -2,7 +2,7 @@ AC_PREREQ(2.69) AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS Copyright (c) 2006-2022 Varnish Software]) AC_REVISION([$Id$]) -AC_INIT([Varnish], [trunk], [varnish-dev at varnish-cache.org]) +AC_INIT([Varnish], [7.2.0], [varnish-dev at varnish-cache.org]) AC_CONFIG_SRCDIR(include/miniobj.h) AC_CONFIG_HEADERS([config.h]) AC_CONFIG_MACRO_DIR([m4]) diff --git a/doc/changes.rst b/doc/changes.rst index 49c883356..51175b85a 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -31,9 +31,9 @@ http://varnish-cache.org/docs/trunk/whats-new/index.html and via individual releases. These documents are updated as part of the release process. -=============================== -Varnish Cache NEXT (2022-09-15) -=============================== +================================ +Varnish Cache 7.2.0 (2022-09-15) +================================ * Functions ``VRT_AddVDP()``, ``VRT_AddVFP()``, ``VRT_RemoveVDP()`` and ``VRT_RemoveVFP()`` are deprecated. diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-7.2.rst similarity index 93% rename from doc/sphinx/whats-new/changes-trunk.rst rename to doc/sphinx/whats-new/changes-7.2.rst index dd3113f43..dee76fd9b 100644 --- a/doc/sphinx/whats-new/changes-trunk.rst +++ b/doc/sphinx/whats-new/changes-7.2.rst @@ -1,15 +1,11 @@ -**Note: This is a working document for a future release, with running -updates for changes in the development branch. For changes in the -released versions of Varnish, see:** :ref:`whats-new-index` +.. _whatsnew_changes_7.2: -.. _whatsnew_changes_CURRENT: - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -Changes in Varnish **$NEXT_RELEASE** -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%%%%%% +Changes in Varnish **7.2** +%%%%%%%%%%%%%%%%%%%%%%%%%% For information about updating your current Varnish deployment to the -new version, see :ref:`whatsnew_upgrading_CURRENT`. +new version, see :ref:`whatsnew_upgrading_7.2`. A more detailed and technical account of changes in Varnish, with links to issues that have been fixed and pull requests that have been diff --git a/doc/sphinx/whats-new/index.rst b/doc/sphinx/whats-new/index.rst index a8a7f5b64..9d2561e96 100644 --- a/doc/sphinx/whats-new/index.rst +++ b/doc/sphinx/whats-new/index.rst @@ -13,18 +13,14 @@ This section describes the changes and improvements between different versions of Varnish, and what upgrading between the different versions entail. -Varnish **$NEXT_RELEASE** -------------------------- - -**Note: These are working documents for a future release, with running -updates for changes in the development branch. For changes in the -released versions of Varnish, see the chapters listed below.** +Varnish **7.2** +--------------- .. toctree:: :maxdepth: 2 - changes-trunk - upgrading-trunk + changes-7.2 + upgrading-7.2 Varnish **7.1** --------------- diff --git a/doc/sphinx/whats-new/upgrading-trunk.rst b/doc/sphinx/whats-new/upgrading-7.2.rst similarity index 88% rename from doc/sphinx/whats-new/upgrading-trunk.rst rename to doc/sphinx/whats-new/upgrading-7.2.rst index d5f8566c2..a1ae38ea3 100644 --- a/doc/sphinx/whats-new/upgrading-trunk.rst +++ b/doc/sphinx/whats-new/upgrading-7.2.rst @@ -1,12 +1,8 @@ -**Note: This is a working document for a future release, with running -updates for changes in the development branch. For changes in the -released versions of Varnish, see:** :ref:`whats-new-index` +.. _whatsnew_upgrading_7.2: -.. _whatsnew_upgrading_CURRENT: - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -Upgrading to Varnish **$NEXT_RELEASE** -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%%%%%%%% +Upgrading to Varnish **7.2** +%%%%%%%%%%%%%%%%%%%%%%%%%%%% varnishd ======== diff --git a/include/vrt.h b/include/vrt.h index 032012a1f..fca3c0295 100644 --- a/include/vrt.h +++ b/include/vrt.h @@ -57,7 +57,7 @@ * Whenever something is deleted or changed in a way which is not * binary/load-time compatible, increment MAJOR version * - * NEXT (2022-09-15) + * 16.0 (2022-09-15) * VMOD C-prototypes moved into JSON * VRT_AddVDP() deprecated * VRT_AddVFP() deprecated From martin at varnish-software.com Thu Sep 15 11:38:04 2022 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Thu, 15 Sep 2022 11:38:04 +0000 (UTC) Subject: [7.2] b3ff8decc Switch to the 7.2 branch of pkg-varnish-cache Message-ID: <20220915113804.D3E691071E9@lists.varnish-cache.org> commit b3ff8decc0033de10fd953113e6daebe1671ef1d Author: Martin Blix Grydeland Date: Thu Sep 15 13:36:55 2022 +0200 Switch to the 7.2 branch of pkg-varnish-cache diff --git a/.circleci/config.yml b/.circleci/config.yml index accac691a..e75fab54a 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -6,7 +6,7 @@ parameters: default: "HEAD" pkg-commit: type: string - default: "master" + default: "7.2" dist-url: type: string default: "" From martin at varnish-software.com Thu Sep 15 11:48:05 2022 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Thu, 15 Sep 2022 11:48:05 +0000 (UTC) Subject: [master] 75d4c1de9 Prepare for 7.2.0 Message-ID: <20220915114805.66A631078B3@lists.varnish-cache.org> commit 75d4c1de9673da2ae3df3904fae960d8ae534a00 Author: Martin Blix Grydeland Date: Thu Sep 15 10:38:30 2022 +0200 Prepare for 7.2.0 diff --git a/configure.ac b/configure.ac index e8aa9c757..0eb3af56f 100644 --- a/configure.ac +++ b/configure.ac @@ -2,7 +2,7 @@ AC_PREREQ(2.69) AC_COPYRIGHT([Copyright (c) 2006 Verdens Gang AS Copyright (c) 2006-2022 Varnish Software]) AC_REVISION([$Id$]) -AC_INIT([Varnish], [trunk], [varnish-dev at varnish-cache.org]) +AC_INIT([Varnish], [7.2.0], [varnish-dev at varnish-cache.org]) AC_CONFIG_SRCDIR(include/miniobj.h) AC_CONFIG_HEADERS([config.h]) AC_CONFIG_MACRO_DIR([m4]) diff --git a/doc/changes.rst b/doc/changes.rst index 49c883356..51175b85a 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -31,9 +31,9 @@ http://varnish-cache.org/docs/trunk/whats-new/index.html and via individual releases. These documents are updated as part of the release process. -=============================== -Varnish Cache NEXT (2022-09-15) -=============================== +================================ +Varnish Cache 7.2.0 (2022-09-15) +================================ * Functions ``VRT_AddVDP()``, ``VRT_AddVFP()``, ``VRT_RemoveVDP()`` and ``VRT_RemoveVFP()`` are deprecated. diff --git a/doc/sphinx/whats-new/changes-trunk.rst b/doc/sphinx/whats-new/changes-7.2.rst similarity index 93% rename from doc/sphinx/whats-new/changes-trunk.rst rename to doc/sphinx/whats-new/changes-7.2.rst index dd3113f43..dee76fd9b 100644 --- a/doc/sphinx/whats-new/changes-trunk.rst +++ b/doc/sphinx/whats-new/changes-7.2.rst @@ -1,15 +1,11 @@ -**Note: This is a working document for a future release, with running -updates for changes in the development branch. For changes in the -released versions of Varnish, see:** :ref:`whats-new-index` +.. _whatsnew_changes_7.2: -.. _whatsnew_changes_CURRENT: - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -Changes in Varnish **$NEXT_RELEASE** -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%%%%%% +Changes in Varnish **7.2** +%%%%%%%%%%%%%%%%%%%%%%%%%% For information about updating your current Varnish deployment to the -new version, see :ref:`whatsnew_upgrading_CURRENT`. +new version, see :ref:`whatsnew_upgrading_7.2`. A more detailed and technical account of changes in Varnish, with links to issues that have been fixed and pull requests that have been diff --git a/doc/sphinx/whats-new/index.rst b/doc/sphinx/whats-new/index.rst index a8a7f5b64..9d2561e96 100644 --- a/doc/sphinx/whats-new/index.rst +++ b/doc/sphinx/whats-new/index.rst @@ -13,18 +13,14 @@ This section describes the changes and improvements between different versions of Varnish, and what upgrading between the different versions entail. -Varnish **$NEXT_RELEASE** -------------------------- - -**Note: These are working documents for a future release, with running -updates for changes in the development branch. For changes in the -released versions of Varnish, see the chapters listed below.** +Varnish **7.2** +--------------- .. toctree:: :maxdepth: 2 - changes-trunk - upgrading-trunk + changes-7.2 + upgrading-7.2 Varnish **7.1** --------------- diff --git a/doc/sphinx/whats-new/upgrading-trunk.rst b/doc/sphinx/whats-new/upgrading-7.2.rst similarity index 88% rename from doc/sphinx/whats-new/upgrading-trunk.rst rename to doc/sphinx/whats-new/upgrading-7.2.rst index d5f8566c2..a1ae38ea3 100644 --- a/doc/sphinx/whats-new/upgrading-trunk.rst +++ b/doc/sphinx/whats-new/upgrading-7.2.rst @@ -1,12 +1,8 @@ -**Note: This is a working document for a future release, with running -updates for changes in the development branch. For changes in the -released versions of Varnish, see:** :ref:`whats-new-index` +.. _whatsnew_upgrading_7.2: -.. _whatsnew_upgrading_CURRENT: - -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% -Upgrading to Varnish **$NEXT_RELEASE** -%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% +%%%%%%%%%%%%%%%%%%%%%%%%%%%% +Upgrading to Varnish **7.2** +%%%%%%%%%%%%%%%%%%%%%%%%%%%% varnishd ======== diff --git a/include/vrt.h b/include/vrt.h index 032012a1f..fca3c0295 100644 --- a/include/vrt.h +++ b/include/vrt.h @@ -57,7 +57,7 @@ * Whenever something is deleted or changed in a way which is not * binary/load-time compatible, increment MAJOR version * - * NEXT (2022-09-15) + * 16.0 (2022-09-15) * VMOD C-prototypes moved into JSON * VRT_AddVDP() deprecated * VRT_AddVFP() deprecated From martin at varnish-software.com Thu Sep 15 11:48:05 2022 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Thu, 15 Sep 2022 11:48:05 +0000 (UTC) Subject: [master] 6c1007ad3 Merge tag 'varnish-7.2.0' Message-ID: <20220915114805.B666B1078B7@lists.varnish-cache.org> commit 6c1007ad3043beb31f921224e887b7aecefe7bfe Merge: cc1c4fd76 75d4c1de9 Author: Martin Blix Grydeland Date: Thu Sep 15 13:46:17 2022 +0200 Merge tag 'varnish-7.2.0' Releasing 7.2.0 From nils.goroll at uplex.de Sat Sep 17 11:52:05 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Sat, 17 Sep 2022 11:52:05 +0000 (UTC) Subject: [master] 489280859 Start a NEXT section in changes.rst as an anchor for PR diffs Message-ID: <20220917115205.7F042116EFD@lists.varnish-cache.org> commit 489280859dca563d5deb80301e3129cbb8ad7ffd Author: Nils Goroll Date: Sat Sep 17 13:50:55 2022 +0200 Start a NEXT section in changes.rst as an anchor for PR diffs diff --git a/doc/changes.rst b/doc/changes.rst index 51175b85a..043ad7f5f 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -31,6 +31,10 @@ http://varnish-cache.org/docs/trunk/whats-new/index.html and via individual releases. These documents are updated as part of the release process. +=============================== +Varnish Cache NEXT (2023-03-15) +=============================== + ================================ Varnish Cache 7.2.0 (2022-09-15) ================================ From phk at FreeBSD.org Mon Sep 19 12:19:05 2022 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Mon, 19 Sep 2022 12:19:05 +0000 (UTC) Subject: [master] 0748f3f0d Prefer the "screen-dump" spelling in docs. Message-ID: <20220919121905.9A97311F467@lists.varnish-cache.org> commit 0748f3f0d96e5275f922a9af1bb04cc536f14f67 Author: Poul-Henning Kamp Date: Mon Sep 19 11:13:33 2022 +0000 Prefer the "screen-dump" spelling in docs. diff --git a/bin/varnishtest/vtc_process.c b/bin/varnishtest/vtc_process.c index 30871a0c0..f5ca77bd7 100644 --- a/bin/varnishtest/vtc_process.c +++ b/bin/varnishtest/vtc_process.c @@ -911,7 +911,7 @@ process_close(struct process *p) * process pNAME SPEC [-log] [-dump] [-hexdump] [-expect-exit N] * [-start] [-run] * [-write STRING] [-writeln STRING] - * [-kill STRING] [-stop] [-wait] [-close] + * [-kill STRING] [-stop] [-wait] [-close] [-screen-dump] * * pNAME * Name of the process. It must start with 'p'. @@ -1001,7 +1001,7 @@ process_close(struct process *p) * \-close * Alias for "-kill HUP" * - * \-screen_dump + * \-screen-dump * Dump the virtual screen into vtc_log * */ From phk at FreeBSD.org Mon Sep 19 12:19:05 2022 From: phk at FreeBSD.org (Poul-Henning Kamp) Date: Mon, 19 Sep 2022 12:19:05 +0000 (UTC) Subject: [master] 32af98d0b Update 'process' docs Message-ID: <20220919121905.AED5911F46A@lists.varnish-cache.org> commit 32af98d0ba935edb254117c48046f953a8c3a890 Author: Poul-Henning Kamp Date: Mon Sep 19 11:39:04 2022 +0000 Update 'process' docs diff --git a/bin/varnishtest/vtc_process.c b/bin/varnishtest/vtc_process.c index f5ca77bd7..b24404c89 100644 --- a/bin/varnishtest/vtc_process.c +++ b/bin/varnishtest/vtc_process.c @@ -908,10 +908,14 @@ process_close(struct process *p) * Output from the stderr-pipe is copied verbatim to ${pNAME_err}, and * is always included in the vtc_log. * - * process pNAME SPEC [-log] [-dump] [-hexdump] [-expect-exit N] - * [-start] [-run] - * [-write STRING] [-writeln STRING] - * [-kill STRING] [-stop] [-wait] [-close] [-screen-dump] + * process pNAME SPEC [-allow-core] [-expect-exit N] [-expect-signal N] + * [-dump] [-hexdump] [-log] + * [-run] [-close] [-kill SIGNAL] [-start] [-stop] [-wait] + * [-write STRING] [-writeln STRING] [-writehex HEXSTRING] + * [-need-bytes [+]NUMBER] + * [-screen-dump] [-winsz LINES COLUMNSS] [-ansi-response] + * [-expect-cursor LINE COLUMN] [-expect-text LINE COLUMN TEXT] + * [-match-text LINE COLUMN REGEXP] * * pNAME * Name of the process. It must start with 'p'. @@ -934,6 +938,12 @@ process_close(struct process *p) * \-expect-exit N * Expect exit status N * + * \-expect-signal N + * Expect signal in exit status N + * + * \-allow-core + * Core dump in exit status is OK + * * \-wait * Wait for the process to finish. * @@ -954,7 +964,7 @@ process_close(struct process *p) * expression from either output, consider using it if you only need * to match one. * - * \-kill STRING + * \-kill SIGNAL * Send a signal to the process. The argument can be either * the string "TERM", "INT", or "KILL" for SIGTERM, SIGINT or SIGKILL * signals, respectively, or a hyphen (-) followed by the signal @@ -970,7 +980,10 @@ process_close(struct process *p) * \-stop * Shorthand for -kill TERM. * - * \-winsz LIN COL + * \-close + * Alias for "-kill HUP" + * + * \-winsz LINES COLUMNS * Change the terminal window size to LIN lines and COL columns. * * \-write STRING @@ -986,20 +999,24 @@ process_close(struct process *p) * Wait until at least NUMBER bytes have been received in total. * If '+' is prefixed, NUMBER new bytes must be received. * - * \-expect-text LIN COL PAT - * Wait for PAT to appear at LIN,COL on the virtual screen. + * \-ansi-response + * Respond to terminal respond-back sequences + * + * \-expect-cursor LINE COLUMN + * Expect cursors location + * + * \-expect-text LINE COLUMNS TEXT + * Wait for TEXT to appear at LIN,COL on the virtual screen. * Lines and columns are numbered 1...N * LIN==0 means "on any line" * COL==0 means "anywhere on the line" * - * \-match-text LIN COL PAT + * \-match-text LINE COLUMN REGEXP * Wait for the PAT regular expression to match the text at LIN,COL on the virtual screen. * Lines and columns are numbered 1...N * LIN==0 means "on any line" * COL==0 means "anywhere on the line" * - * \-close - * Alias for "-kill HUP" * * \-screen-dump * Dump the virtual screen into vtc_log From nils.goroll at uplex.de Mon Sep 19 13:26:05 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 19 Sep 2022 13:26:05 +0000 (UTC) Subject: [master] 03a36ac7e Polish contrib Makefile Message-ID: <20220919132605.564E0121445@lists.varnish-cache.org> commit 03a36ac7eaa73e14a4de33d5cbe688456e14525f Author: Nils Goroll Date: Sat Sep 17 13:43:04 2022 +0200 Polish contrib Makefile Avoid this warning: autoreconf2.69: running: automake --add-missing --copy --no-force vtc.am:12: warning: 'TEST_EXTENSIONS' cannot have conditional contents contrib/Makefile.am:12: 'vtc.am' included from here Always defining the tests even if contrib is not built should not cause any harm. diff --git a/contrib/Makefile.am b/contrib/Makefile.am index 59607f5bf..889d3f5ca 100644 --- a/contrib/Makefile.am +++ b/contrib/Makefile.am @@ -1,5 +1,7 @@ # +include $(top_srcdir)/vtc.am + if !WITH_CONTRIB dist_noinst_SCRIPTS = \ varnishstatdiff @@ -9,7 +11,6 @@ dist_bin_SCRIPTS = \ TESTS = @CONTRIB_TESTS@ -include $(top_srcdir)/vtc.am endif EXTRA_DIST = @CONTRIB_TESTS@ From nils.goroll at uplex.de Mon Sep 19 13:44:05 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 19 Sep 2022 13:44:05 +0000 (UTC) Subject: [master] e51197f1d Output the name of duplicate filters Message-ID: <20220919134405.E37A6121E1D@lists.varnish-cache.org> commit e51197f1df1b0c6bf4c4ab01bf647c75d94b08ab Author: Nils Goroll Date: Tue Sep 13 18:53:30 2022 +0200 Output the name of duplicate filters The error message for duplicate filters did not mention the filter name and was thus not particularly helpful. It does now, for example: rot13: VFP already registered (per-vcl) diff --git a/bin/varnishd/cache/cache_vrt_filter.c b/bin/varnishd/cache/cache_vrt_filter.c index 1cfedc449..bf45372d1 100644 --- a/bin/varnishd/cache/cache_vrt_filter.c +++ b/bin/varnishd/cache/cache_vrt_filter.c @@ -103,7 +103,7 @@ vrt_addfilter(VRT_CTX, const struct vfp *vfp, const struct vdp *vdp) err = is_dup_filter(hd, vfp, vdp, name); if (err != NULL) { if (ctx != NULL) - VRT_fail(ctx, "%s (global)", err); + VRT_fail(ctx, "%s: %s (global)", name, err); return (err); } if (ctx != NULL) { @@ -112,7 +112,7 @@ vrt_addfilter(VRT_CTX, const struct vfp *vfp, const struct vdp *vdp) hd = &ctx->vcl->filters; err = is_dup_filter(hd, vfp, vdp, name); if (err != NULL) { - VRT_fail(ctx, "%s (per-vcl)", err); + VRT_fail(ctx, "%s: %s (per-vcl)", name, err); return (err); } } From nils.goroll at uplex.de Mon Sep 19 13:44:06 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 19 Sep 2022 13:44:06 +0000 (UTC) Subject: [master] e675d1c6b Make the duplicate filter colverage test explicit Message-ID: <20220919134406.13AB9121E20@lists.varnish-cache.org> commit e675d1c6b3e73713d5e2c1f6062805bc092bb4c0 Author: Nils Goroll Date: Tue Sep 13 19:08:41 2022 +0200 Make the duplicate filter colverage test explicit Implicitly testing from all loads of vmod_debug caused confusing error messages and prevented explicitly testing for the error message. diff --git a/bin/varnishtest/tests/m00000.vtc b/bin/varnishtest/tests/m00000.vtc index dc674c3ae..73cda64f1 100644 --- a/bin/varnishtest/tests/m00000.vtc +++ b/bin/varnishtest/tests/m00000.vtc @@ -140,6 +140,14 @@ varnish v1 -errvcl {Symbol not found: 'obj'} { } } +varnish v1 -errvcl {rot13: VFP already registered (per-vcl)} { + import debug; + backend none none; + sub vcl_init { + debug.rot104(); + } +} + varnish v1 -errvcl {Failed initialization} { import debug; import directors; @@ -281,7 +289,7 @@ varnish v1 -cliok "vcl.list" varnish v1 -expect vmods == 3 -varnish v1 -cliok "vcl.discard vcl[1-8]" +varnish v1 -cliok "vcl.discard vcl[1-9]" varnish v1 -cliok "vcl.list" varnish v1 -cliok "debug.vmod" diff --git a/vmod/vmod_debug.c b/vmod/vmod_debug.c index f8c603204..50a2b8eeb 100644 --- a/vmod/vmod_debug.c +++ b/vmod/vmod_debug.c @@ -334,6 +334,15 @@ xyzzy_test_priv_vcl(VRT_CTX, struct vmod_priv *priv) assert(!strcmp(priv_vcl->foo, "FOO")); } +VCL_VOID v_matchproto_(td_debug_rot104) +xyzzy_rot104(VRT_CTX) +{ + + CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC); + // This should fail + AN(VRT_AddFilter(ctx, &xyzzy_vfp_rot13, &xyzzy_vdp_rot13)); +} + VCL_VOID v_matchproto_(td_debug_rot52) xyzzy_rot52(VRT_CTX, VCL_HTTP hp) { @@ -456,15 +465,6 @@ event_load(VRT_CTX, struct vmod_priv *priv) priv->methods = priv_vcl_methods; AZ(VRT_AddFilter(ctx, &xyzzy_vfp_rot13, &xyzzy_vdp_rot13)); - - // This should fail - AN(VRT_AddFilter(ctx, &xyzzy_vfp_rot13, &xyzzy_vdp_rot13)); - // Reset the error, we know what we're doing. - ctx->vpi->handling = 0; - - VRT_RemoveFilter(ctx, &xyzzy_vfp_rot13, &xyzzy_vdp_rot13); - AZ(VRT_AddFilter(ctx, &xyzzy_vfp_rot13, &xyzzy_vdp_rot13)); - AZ(VRT_AddFilter(ctx, NULL, &xyzzy_vdp_pedantic)); return (0); } diff --git a/vmod/vmod_debug.vcc b/vmod/vmod_debug.vcc index df8ba6247..6f9d8d71d 100644 --- a/vmod/vmod_debug.vcc +++ b/vmod/vmod_debug.vcc @@ -121,6 +121,11 @@ $Method STRING .test_priv_top(STRING s="") Test per-object priv_top via VRT_priv_top() +$Function VOID rot104() + +Try to register the rot52 filter again. This should always fail +the vcl. + $Function VOID rot52(HTTP hdr) Encrypt the HTTP header with quad-ROT13 encryption, From nils.goroll at uplex.de Mon Sep 19 13:44:06 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 19 Sep 2022 13:44:06 +0000 (UTC) Subject: [master] b30af4dd3 Ensure the test fails for the right reason Message-ID: <20220919134406.2F540121E24@lists.varnish-cache.org> commit b30af4dd37776c891f3970a0a03ab89c5f2cfa7b Author: Nils Goroll Date: Tue Sep 13 19:09:41 2022 +0200 Ensure the test fails for the right reason diff --git a/bin/varnishtest/tests/m00000.vtc b/bin/varnishtest/tests/m00000.vtc index 73cda64f1..3aed21140 100644 --- a/bin/varnishtest/tests/m00000.vtc +++ b/bin/varnishtest/tests/m00000.vtc @@ -148,7 +148,7 @@ varnish v1 -errvcl {rot13: VFP already registered (per-vcl)} { } } -varnish v1 -errvcl {Failed initialization} { +varnish v1 -errvcl {Failed from VCL} { import debug; import directors; backend default { .host = "${localhost}"; } From nils.goroll at uplex.de Mon Sep 19 13:44:06 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 19 Sep 2022 13:44:06 +0000 (UTC) Subject: [master] 5e72440e0 Test that VRT_fail from an object constructor works as expected Message-ID: <20220919134406.5741E121E29@lists.varnish-cache.org> commit 5e72440e0f0dbfa285971c036afa3a1cbb514fd9 Author: Nils Goroll Date: Tue Sep 13 19:15:58 2022 +0200 Test that VRT_fail from an object constructor works as expected diff --git a/bin/varnishtest/tests/m00000.vtc b/bin/varnishtest/tests/m00000.vtc index 3aed21140..cc78c3c1f 100644 --- a/bin/varnishtest/tests/m00000.vtc +++ b/bin/varnishtest/tests/m00000.vtc @@ -165,6 +165,14 @@ varnish v1 -errvcl {Failed from VCL} { } } +varnish v1 -errvcl {Failed initialization} { + import debug; + backend none none; + sub vcl_init { + new fails = debug.obj("fail"); + } +} + varnish v1 -cliok "param.set vcc_feature +allow_inline_c" varnish v1 -vcl { import vtc; @@ -289,7 +297,7 @@ varnish v1 -cliok "vcl.list" varnish v1 -expect vmods == 3 -varnish v1 -cliok "vcl.discard vcl[1-9]" +varnish v1 -cliok "vcl.discard vcl[1-9] vcl10" varnish v1 -cliok "vcl.list" varnish v1 -cliok "debug.vmod" diff --git a/vmod/vmod_debug.vcc b/vmod/vmod_debug.vcc index 6f9d8d71d..90be677d6 100644 --- a/vmod/vmod_debug.vcc +++ b/vmod/vmod_debug.vcc @@ -73,6 +73,8 @@ $Object obj(STRING string="default", ENUM { one, two, three } number=one) Test object +Fail for string="fail". + .. NOTE: .enum before .foo as part of test r01332.vtc $Method VOID .enum(ENUM { phk, des, kristian, mithrandir, martin }) diff --git a/vmod/vmod_debug_obj.c b/vmod/vmod_debug_obj.c index 94331f047..28a6bfa00 100644 --- a/vmod/vmod_debug_obj.c +++ b/vmod/vmod_debug_obj.c @@ -60,6 +60,10 @@ xyzzy_obj__init(VRT_CTX, struct xyzzy_debug_obj **op, CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC); AN(op); AZ(*op); + + if (! strcmp(s, "fail")) { + VRT_fail(ctx, "failing as requested"); + } ALLOC_OBJ(o, VMOD_DEBUG_OBJ_MAGIC); AN(o); *op = o; From nils.goroll at uplex.de Mon Sep 19 13:44:06 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 19 Sep 2022 13:44:06 +0000 (UTC) Subject: [master] 64ccd4464 Test vcl.load failure via vtc cli and varnishadm Message-ID: <20220919134406.77A80121E30@lists.varnish-cache.org> commit 64ccd4464fe7cf04052ee0a288bdbfdb38de2151 Author: Nils Goroll Date: Thu Sep 15 10:32:22 2022 +0200 Test vcl.load failure via vtc cli and varnishadm diff --git a/bin/varnishtest/tests/m00000.vtc b/bin/varnishtest/tests/m00000.vtc index cc78c3c1f..a5d94aca4 100644 --- a/bin/varnishtest/tests/m00000.vtc +++ b/bin/varnishtest/tests/m00000.vtc @@ -173,6 +173,23 @@ varnish v1 -errvcl {Failed initialization} { } } +shell { + cat >${tmpdir}/f1 <<-EOF + vcl 4.1; + import debug; + backend none none; + sub vcl_init { + new fails = debug.obj("fail"); + } + EOF +} + +varnish v1 -clierr 300 "vcl.load f1 ${tmpdir}/f1" + +shell -exit 1 -expect {failing as requested} { + varnishadm -n ${tmpdir}/v1 vcl.load f1 ${tmpdir}/f1 +} + varnish v1 -cliok "param.set vcc_feature +allow_inline_c" varnish v1 -vcl { import vtc; From nils.goroll at uplex.de Mon Sep 19 13:44:06 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 19 Sep 2022 13:44:06 +0000 (UTC) Subject: [master] 7791c04df varnishadm does not fail if an eror is received after the varnishadm -t timeout Message-ID: <20220919134406.8F43D121E3F@lists.varnish-cache.org> commit 7791c04df7ae422272d92111b7e35da751ee77c5 Author: Nils Goroll Date: Thu Sep 15 13:02:44 2022 +0200 varnishadm does not fail if an eror is received after the varnishadm -t timeout diff --git a/bin/varnishtest/tests/m00000.vtc b/bin/varnishtest/tests/m00000.vtc index a5d94aca4..4fbe4e86f 100644 --- a/bin/varnishtest/tests/m00000.vtc +++ b/bin/varnishtest/tests/m00000.vtc @@ -177,8 +177,10 @@ shell { cat >${tmpdir}/f1 <<-EOF vcl 4.1; import debug; + import vtc; backend none none; sub vcl_init { + vtc.sleep(10s); new fails = debug.obj("fail"); } EOF From nils.goroll at uplex.de Mon Sep 19 13:53:05 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 19 Sep 2022 13:53:05 +0000 (UTC) Subject: [master] 3d210ea74 Revert "varnishadm does not fail if an eror is received after the varnishadm -t timeout" Message-ID: <20220919135306.02BB812298A@lists.varnish-cache.org> commit 3d210ea74a4f87949d15e7e62c2818bfa3098296 Author: Nils Goroll Date: Mon Sep 19 15:52:19 2022 +0200 Revert "varnishadm does not fail if an eror is received after the varnishadm -t timeout" This reverts commit 7791c04df7ae422272d92111b7e35da751ee77c5. Premature, can only go in once #3848 is done diff --git a/bin/varnishtest/tests/m00000.vtc b/bin/varnishtest/tests/m00000.vtc index 4fbe4e86f..a5d94aca4 100644 --- a/bin/varnishtest/tests/m00000.vtc +++ b/bin/varnishtest/tests/m00000.vtc @@ -177,10 +177,8 @@ shell { cat >${tmpdir}/f1 <<-EOF vcl 4.1; import debug; - import vtc; backend none none; sub vcl_init { - vtc.sleep(10s); new fails = debug.obj("fail"); } EOF From nils.goroll at uplex.de Sat Sep 24 15:29:05 2022 From: nils.goroll at uplex.de (Nils Goroll) Date: Sat, 24 Sep 2022 15:29:05 +0000 (UTC) Subject: [master] 08430eb16 Add a trivial clarifying assertion Message-ID: <20220924152905.E8B26117FE1@lists.varnish-cache.org> commit 08430eb1666640580ac17c03764809c4bd320b1b Author: Nils Goroll Date: Sat Sep 24 17:26:29 2022 +0200 Add a trivial clarifying assertion diff --git a/bin/varnishd/cache/cache_hash.c b/bin/varnishd/cache/cache_hash.c index a8bd61f5b..337727c99 100644 --- a/bin/varnishd/cache/cache_hash.c +++ b/bin/varnishd/cache/cache_hash.c @@ -861,6 +861,8 @@ HSH_Unbusy(struct worker *wrk, struct objcore *oc) CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC); CHECK_OBJ_NOTNULL(oc, OBJCORE_MAGIC); + CHECK_OBJ_NOTNULL(oc->boc, BOC_MAGIC); + oh = oc->objhead; CHECK_OBJ(oh, OBJHEAD_MAGIC); INIT_OBJ(&rush, RUSH_MAGIC); From hermunn at varnish-software.com Tue Sep 27 12:03:10 2022 From: hermunn at varnish-software.com (Pål Hermunn Johansen) Date: Tue, 27 Sep 2022 12:03:10 +0000 (UTC) Subject: [master] 8801f3a47 Typo - the IRC channel is our channel Message-ID: <20220927120310.4F15111DF5A@lists.varnish-cache.org> commit 8801f3a47d9cd358141c23e9be6327bf1c7aa0fc Author: P?l Hermunn Johansen Date: Tue Sep 27 14:01:56 2022 +0200 Typo - the IRC channel is our channel diff --git a/CONTRIBUTING b/CONTRIBUTING index 804b83385..ed97b395f 100644 --- a/CONTRIBUTING +++ b/CONTRIBUTING @@ -9,7 +9,7 @@ These days we prefer patches as pull requests directly to that tree. Bugreports go there too. -Our main project communication is through or developer IRC channel:: +Our main project communication is through our developer IRC channel:: #varnish-hacking on server irc.linpro.no From martin at varnish-software.com Fri Sep 30 15:14:08 2022 From: martin at varnish-software.com (Martin Blix Grydeland) Date: Fri, 30 Sep 2022 15:14:08 +0000 (UTC) Subject: [master] c61f79948 Limit cifuzz.yml to the `varnishcache` github org Message-ID: <20220930151408.9F06F10F666@lists.varnish-cache.org> commit c61f799485d9da14128603019478a65d0976dd18 Author: Martin Blix Grydeland Date: Fri Sep 30 17:10:54 2022 +0200 Limit cifuzz.yml to the `varnishcache` github org Limit the fuzzer runs to PRs on `varnishcache`'s repositories. The build scripts won't work correctly from anywhere else anyways. diff --git a/.github/workflows/cifuzz.yml b/.github/workflows/cifuzz.yml index a18f1333c..411fbe2ef 100644 --- a/.github/workflows/cifuzz.yml +++ b/.github/workflows/cifuzz.yml @@ -2,6 +2,7 @@ name: CIFuzz on: [pull_request] jobs: Fuzzing: + if: github.repository_owner == 'varnishcache' runs-on: ubuntu-latest steps: - name: Build Fuzzers