From nils.goroll at uplex.de Fri Nov 1 07:35:12 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Fri, 1 Nov 2024 07:35:12 +0000 (UTC) Subject: [master] d24cc294e SunCC: Don't complain about integer promotions in vcl generated C (vgc) Message-ID: <20241101073512.73E504861@lists.varnish-cache.org> commit d24cc294eb75b769def195575c574f4b44e872e5 Author: Nils Goroll Date: Fri Nov 1 08:33:31 2024 +0100 SunCC: Don't complain about integer promotions in vcl generated C (vgc) seen with vmod/tests/directors_c00004.vtc: **** v1 CLI RX|"vgc.c", line 4473: constant promoted according to the 1999 ISO C standard (E_C99_INTEGER_PROMOTION) diff --git a/configure.ac b/configure.ac index b36c599f3..44e1ed879 100644 --- a/configure.ac +++ b/configure.ac @@ -860,7 +860,7 @@ else VCC_CC="exec $VCC_CC %w $PTHREAD_CFLAGS -fpic -shared -o %o %s" ;; *cc) - VCC_CC="exec $VCC_CC %w -errwarn=%%all,no%%E_STATEMENT_NOT_REACHED $PTHREAD_CFLAGS -Kpic -G -o %o %s" + VCC_CC="exec $VCC_CC %w -errwarn=%%all,no%%E_STATEMENT_NOT_REACHED,no%E_C99_INTEGER_PROMOTION $PTHREAD_CFLAGS -Kpic -G -o %o %s" ;; esac ;; From nils.goroll at uplex.de Sun Nov 3 11:03:09 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Sun, 3 Nov 2024 11:03:09 +0000 (UTC) Subject: [master] 121e57c10 acceptor: add private pointers Message-ID: <20241103110309.E9234116CCE@lists.varnish-cache.org> commit 121e57c100d20774c1ac2984372bdea35646c339 Author: Nils Goroll Date: Mon Sep 16 08:59:57 2024 +0200 acceptor: add private pointers Because of the small number of acceptor related objects (acceptor, listen sock, poolsock), this is very low overall overhead and helps avoid complications in acceptor implementations. Complements #3976 diff --git a/bin/varnishd/acceptor/acceptor_priv.h b/bin/varnishd/acceptor/acceptor_priv.h index 4f18d23c5..d24ce5d5b 100644 --- a/bin/varnishd/acceptor/acceptor_priv.h +++ b/bin/varnishd/acceptor/acceptor_priv.h @@ -51,6 +51,7 @@ struct poolsock { struct listen_sock *lsock; struct pool_task task[1]; struct pool *pool; + void *vca_priv; }; struct conn_heritage { diff --git a/bin/varnishd/acceptor/cache_acceptor.h b/bin/varnishd/acceptor/cache_acceptor.h index 6dcc2e2f7..79c7a0dec 100644 --- a/bin/varnishd/acceptor/cache_acceptor.h +++ b/bin/varnishd/acceptor/cache_acceptor.h @@ -61,6 +61,7 @@ struct acceptor { VTAILQ_ENTRY(acceptor) list; VTAILQ_HEAD(,listen_sock) socks; const char *name; + void *vca_priv; acceptor_config_f *config; acceptor_init_f *init; diff --git a/bin/varnishd/common/heritage.h b/bin/varnishd/common/heritage.h index bf207b17b..043343bbc 100644 --- a/bin/varnishd/common/heritage.h +++ b/bin/varnishd/common/heritage.h @@ -55,6 +55,7 @@ struct listen_sock { unsigned test_heritage; struct conn_heritage *conn_heritage; struct acceptor *vca; + void *vca_priv; }; VTAILQ_HEAD(listen_sock_head, listen_sock); From nils.goroll at uplex.de Sun Nov 3 11:03:10 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Sun, 3 Nov 2024 11:03:10 +0000 (UTC) Subject: [master] 2a6f3f43e Flexelint: Change include order Message-ID: <20241103110310.14C0F116CD1@lists.varnish-cache.org> commit 2a6f3f43e4f8466ced6eeaaf8025b7dcabea6a23 Author: Nils Goroll Date: Sun Nov 3 11:56:22 2024 +0100 Flexelint: Change include order to avoid an "already included" complaint on linux: -- Module: ../../lib/libvarnish/vsha256.c (C) File ../../lib/libvarnish/vsha256.c, Line 37 _ # include Warning 537: Repeated include file '/usr/include/endian.h' diff --git a/lib/libvarnish/vsha256.c b/lib/libvarnish/vsha256.c index 8d92abc8c..6b6f756cb 100644 --- a/lib/libvarnish/vsha256.c +++ b/lib/libvarnish/vsha256.c @@ -31,8 +31,6 @@ #include "config.h" -#include - #ifndef __DARWIN_BYTE_ORDER # include # define VBYTE_ORDER __BYTE_ORDER @@ -42,6 +40,7 @@ # define VBIG_ENDIAN __DARWIN_BIG_ENDIAN #endif +#include #include #include From nils.goroll at uplex.de Sun Nov 3 11:03:10 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Sun, 3 Nov 2024 11:03:10 +0000 (UTC) Subject: [master] 58e2e8a2e Flexelint: include cleanup Message-ID: <20241103110310.2E41D116CD4@lists.varnish-cache.org> commit 58e2e8a2e60d029aff901f2a99a47c8f7770cd07 Author: Nils Goroll Date: Sun Nov 3 11:57:36 2024 +0100 Flexelint: include cleanup Info 766: Header file './common/heritage.h' not used in module 'mgt/mgt_jail_linux.c' diff --git a/bin/varnishd/mgt/mgt_jail_linux.c b/bin/varnishd/mgt/mgt_jail_linux.c index d344d7c76..19889c2d0 100644 --- a/bin/varnishd/mgt/mgt_jail_linux.c +++ b/bin/varnishd/mgt/mgt_jail_linux.c @@ -45,7 +45,6 @@ #include #include "mgt/mgt.h" -#include "common/heritage.h" static int vjl_init(char **args) From nils.goroll at uplex.de Sun Nov 3 13:59:06 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Sun, 3 Nov 2024 13:59:06 +0000 (UTC) Subject: [master] 5f7bd4f60 doc: add missing semicolon Message-ID: <20241103135906.4529911CC27@lists.varnish-cache.org> commit 5f7bd4f60e664342eed506658ee81da3e2309ed9 Author: Nils Goroll Date: Sun Nov 3 13:12:18 2024 +0100 doc: add missing semicolon diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst index 959901352..cb8eeb4ef 100644 --- a/doc/sphinx/reference/vcl.rst +++ b/doc/sphinx/reference/vcl.rst @@ -326,7 +326,7 @@ VCL objects A VCL object can be instantiated with the ``new`` keyword:: sub vcl_init { - new b = directors.round_robin() + new b = directors.round_robin(); b.add_backend(node1); } From nils.goroll at uplex.de Sun Nov 3 13:59:06 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Sun, 3 Nov 2024 13:59:06 +0000 (UTC) Subject: [master] 9e96c4c15 Update validhdr() / validate_headers related rfc reference Message-ID: <20241103135906.6F39011CC2A@lists.varnish-cache.org> commit 9e96c4c15ea012ca31c4791153ab92e7ae6caf43 Author: Nils Goroll Date: Sun Nov 3 14:35:32 2024 +0100 Update validhdr() / validate_headers related rfc reference See #4221 for additional finding diff --git a/bin/varnishd/cache/cache_vrt.c b/bin/varnishd/cache/cache_vrt.c index 7f4378fb9..39c8ef53c 100644 --- a/bin/varnishd/cache/cache_vrt.c +++ b/bin/varnishd/cache/cache_vrt.c @@ -539,8 +539,14 @@ VRT_UpperLowerStrands(VRT_CTX, VCL_STRANDS s, int up) return (r); } -// rfc7230,l,1243,1244 -// ASCII VCHAR + TAB + obs-text (0x80-ff) + +// rfc9110,l,1585,1589 +// field-content = field-vchar +// [ 1*( SP / HTAB / field-vchar ) field-vchar ] +// field-vchar = VCHAR / obs-text +// obs-text = %x80-FF +// +// This implementation is less strict, see #4221 static inline VCL_BOOL validhdr(const char *p) { diff --git a/bin/varnishtest/tests/b00049.vtc b/bin/varnishtest/tests/b00049.vtc index 4ad6d4988..b21d01b64 100644 --- a/bin/varnishtest/tests/b00049.vtc +++ b/bin/varnishtest/tests/b00049.vtc @@ -1,4 +1,4 @@ -varnishtest "RFC 7230 compliance" +varnishtest "RFC 9110 compliance" server s1 { rxreq diff --git a/include/tbl/feature_bits.h b/include/tbl/feature_bits.h index 808efe917..c7412ea2d 100644 --- a/include/tbl/feature_bits.h +++ b/include/tbl/feature_bits.h @@ -79,7 +79,7 @@ FEATURE_BIT(WAIT_SILO, wait_silo, ) FEATURE_BIT(VALIDATE_HEADERS, validate_headers, - "Validate all header set operations to conform to RFC7230." + "Validate all header set operations to conform to RFC9110 section 5.5." ) FEATURE_BIT(BUSY_STATS_RATE, busy_stats_rate, diff --git a/vmod/vmod_debug.vcc b/vmod/vmod_debug.vcc index e3f80b0cc..8e9a25c3e 100644 --- a/vmod/vmod_debug.vcc +++ b/vmod/vmod_debug.vcc @@ -374,7 +374,7 @@ A function mixing a named PRIV_TASK with optional parameters. $Function BOOL validhdr(STRANDS) -Test if the argument is a valid header according to RFC7230 section 3.2. +Test if the argument is a valid header according to RFC9110 section 5.5. $Function REGEX just_return_regex(REGEX) From nils.goroll at uplex.de Sun Nov 3 13:59:06 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Sun, 3 Nov 2024 13:59:06 +0000 (UTC) Subject: [master] 11f8d3159 doc: Talk about valid identifiers, strings and headers Message-ID: <20241103135906.91A7211CC2E@lists.varnish-cache.org> commit 11f8d31592e8d3e52a92ab8ca6cd3d54d596c6ed Author: Nils Goroll Date: Sun Nov 3 13:59:55 2024 +0100 doc: Talk about valid identifiers, strings and headers diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst index cb8eeb4ef..90215b2af 100644 --- a/doc/sphinx/reference/vcl.rst +++ b/doc/sphinx/reference/vcl.rst @@ -36,6 +36,39 @@ Starting with Varnish 4.0, each VCL file must start by declaring its version with ``vcl`` *.*\ ``;`` marker at the top of the file. See more about this under Versioning below. +.. _Identifiers: + +Identifiers +----------- + +VCL language elements are generally written in ASCII. + +Identifiers start with an ASCII alphabetic character and continue with any +combination of alphabetic or numerical character, ``-`` or ``_``. + +To illustrate, ``e1xam_p-le`` is a valid identifier, ``1try`` and +``a%remainder`` are not. + +Character Sets +-------------- + +.. _VMODs: https://varnish-cache.org/docs/trunk/reference/vmod.html + +While identifiers are written in ASCII, strings can contain any character set +as long as the *NUL* (zero, 0) byte is reserved. The Varnish Configuration +Language itself is not concerned with the character encoding of strings, VCL +code handling strings in different character sets needs to track encodings +itself. `VMODs`_ exist to help with such tasks. + +To illustrate, ``"??"`` is a valid UTF-8 *string* in UTF-8, but ``??`` is not a +valid *identifier*. + +Character Sets in HTTP +---------------------- + +Not all valid strings in VCL are valid in HTTP elements other than the body +(e.g. URLs and headers). For example, HTTP header field values can not contain +control characters, see also the ``validate_headers`` :ref:`ref_param_feature`. Operators --------- @@ -330,6 +363,9 @@ A VCL object can be instantiated with the ``new`` keyword:: b.add_backend(node1); } +The object name (``b`` in the example) needs to be a valid :ref:`Identifier +`. + This is only available in ``vcl_init``. Subroutines From nils.goroll at uplex.de Sun Nov 3 15:55:06 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Sun, 3 Nov 2024 15:55:06 +0000 (UTC) Subject: [master] 3d0adb402 doc: Improve user guide Message-ID: <20241103155507.0D7071208A7@lists.varnish-cache.org> commit 3d0adb40240145ef2420fcb651f14dd5eac452ca Author: Nils Goroll Date: Sun Nov 3 16:54:14 2024 +0100 doc: Improve user guide WOW, was this outdated diff --git a/doc/sphinx/users-guide/command-line.rst b/doc/sphinx/users-guide/command-line.rst index 3cecbae61..7519d347e 100644 --- a/doc/sphinx/users-guide/command-line.rst +++ b/doc/sphinx/users-guide/command-line.rst @@ -8,10 +8,10 @@ Required command line arguments ------------------------------- -There a two command line arguments you have to set when starting Varnish, these are: +There only one command line argument you have to provide when starting Varnish, +which is '-b' for where the backend server can be contacted. -* what TCP port to serve HTTP from, and -* where the backend server can be contacted. +'-a' is another argument which is likely to require adjustment. If you have installed Varnish through using a provided operating system bound package, you will find the startup options here: @@ -21,31 +21,31 @@ you will find the startup options here: * FreeBSD: `/etc/rc.conf` (See also: /usr/local/etc/rc.d/varnishd) -'-a' *listen_address* -^^^^^^^^^^^^^^^^^^^^^ +'-a' *<[name=][listen_address[,PROTO|,option=value,...]]>* +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -The '-a' argument defines what address Varnish should listen to, and service HTTP requests from. +Each '-a' argument defines one endpoint which Varnish should service HTTP +requests on. -You will most likely want to set this to ":80" which is the Well -Known Port for HTTP. +The default is ``:80,http`` to listen on the Well Known Port for HTTP. If your +webserver runs on the same machine, you will likely have to move it to another +port number or bind it to a loopback address first. -You can specify multiple addresses separated by a comma, and you -can use numeric or host/service names if you like, Varnish will try -to open and service as many of them as possible, but if none of them -can be opened, `varnishd` will not start. +Multiple '-a' arguments can be provided to service multiple endpoints. *name* is +the ``local.socket`` name for VCL. *listen_address* can be an IPv4 or IPv6 +address with a port, a unix domain socket path or an abstract socket. See +:ref:`varnishd(1)` for more details. Here are some examples:: - -a :80 - -a localhost:80 + -a http=:80 + -a localhost:80,HTTP -a 192.168.1.100:8080 -a '[fe80::1]:80' -a '0.0.0.0:8080,[::]:8081' + -a uds=/my/path,PROXY,mode=666 + -a @abstract_socket -.. XXX:brief explanation of some of the more complex examples perhaps? benc - -If your webserver runs on the same machine, you will have to move -it to another port number first. '-f' *VCL-file* or '-b' *backend* ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ From nils.goroll at uplex.de Mon Nov 4 09:19:08 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 4 Nov 2024 09:19:08 +0000 (UTC) Subject: [master] 3dfabb8b2 doc: Point out that the working directory (-n argument) is per instance Message-ID: <20241104091908.3FE4A11D7C3@lists.varnish-cache.org> commit 3dfabb8b28d840bff01e945868469892f21d3b0c Author: Nils Goroll Date: Mon Nov 4 10:17:48 2024 +0100 doc: Point out that the working directory (-n argument) is per instance Motivated by https://github.com/varnishcache/pkg-varnish-cache/issues/178#issuecomment-2453792934 diff --git a/bin/varnishd/mgt/mgt_main.c b/bin/varnishd/mgt/mgt_main.c index 8a9143285..a31ef0506 100644 --- a/bin/varnishd/mgt/mgt_main.c +++ b/bin/varnishd/mgt/mgt_main.c @@ -92,6 +92,8 @@ mgt_DumpOptions(void) "compiled VCLs etc.\n\n"); printf(" In performance critical applications, this directory " "should be on a RAM backed filesystem.\n\n"); + printf(" When running multiple varnishd instances, separate " + "directories need to be used.\n\n"); VIN_DumpDefaults(); } From nils.goroll at uplex.de Mon Nov 4 13:13:04 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 4 Nov 2024 13:13:04 +0000 (UTC) Subject: [master] 778cda340 sync vrt.h with #4222 Message-ID: <20241104131304.96576651C8@lists.varnish-cache.org> commit 778cda3405c679a572ac74357b16fffbed67145c Author: Nils Goroll Date: Mon Nov 4 14:12:15 2024 +0100 sync vrt.h with #4222 diff --git a/include/vrt.h b/include/vrt.h index 9d7d90c14..e400e6a31 100644 --- a/include/vrt.h +++ b/include/vrt.h @@ -58,7 +58,7 @@ * binary/load-time compatible, increment MAJOR version * * NEXT (2024-03-15) - * 20.1 + * 20.1 (2024-11-08) * VDI_EVENT_SICK added to enum vcl_event_e * 20.0 (2024-09-13) * struct vrt_backend.backend_wait_timeout added From nils.goroll at uplex.de Mon Nov 4 13:13:04 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 4 Nov 2024 13:13:04 +0000 (UTC) Subject: [master] dbea8997b sync vrt.h with #4222 Message-ID: <20241104131304.AADD6651CB@lists.varnish-cache.org> commit dbea8997bf02d58550fd667d0719edfbeb8949f9 Author: Nils Goroll Date: Mon Nov 4 14:12:47 2024 +0100 sync vrt.h with #4222 diff --git a/include/vrt.h b/include/vrt.h index e400e6a31..1309604b9 100644 --- a/include/vrt.h +++ b/include/vrt.h @@ -58,7 +58,7 @@ * binary/load-time compatible, increment MAJOR version * * NEXT (2024-03-15) - * 20.1 (2024-11-08) + * 20.1 (2024-11-08 7.6.1) * VDI_EVENT_SICK added to enum vcl_event_e * 20.0 (2024-09-13) * struct vrt_backend.backend_wait_timeout added From nils.goroll at uplex.de Mon Nov 4 15:07:05 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 4 Nov 2024 15:07:05 +0000 (UTC) Subject: [master] 21b155cae Generic Copyright Message-ID: <20241104150705.D176710328B@lists.varnish-cache.org> commit 21b155caef3637dfd7b0c09e54398a8d45a602ff Author: Nils Goroll Date: Fri Oct 11 14:47:58 2024 +0200 Generic Copyright This commit mirrors https://github.com/varnishcache/homepage/commit/0f25620719c3568e166b9beff5c2c176236bf29a as of today, each page redendered from the sphinx source contains the line ? Copyright 2010-2014, Varnish Software AS. While this is true for some of the content, it is not for all of it and because we, several people from UPLEX, intend to contribute significant amounts of documentation in the near future, we would like to change this into something more generic, without the need to state specific copyright and authorship on each page. diff --git a/doc/sphinx/conf.py.in b/doc/sphinx/conf.py.in index 4984b660b..86e277de4 100644 --- a/doc/sphinx/conf.py.in +++ b/doc/sphinx/conf.py.in @@ -38,7 +38,7 @@ master_doc = 'index' # General information about the project. project = u'Varnish Cache' -copyright = u'2010-2014, Varnish Software AS' +copyright = u'2010-2024, The Varnish Cache Contributors' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the From dridi.boukelmoune at gmail.com Mon Nov 4 16:49:05 2024 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Mon, 4 Nov 2024 16:49:05 +0000 (UTC) Subject: [master] 61f386d68 circleci: Fix group install on Fedora 41 with dnf5 Message-ID: <20241104164905.2FFAB1071F4@lists.varnish-cache.org> commit 61f386d68b1a6dc38e65ab8117b80d3033a68e8e Author: Dridi Boukelmoune Date: Mon Nov 4 17:33:41 2024 +0100 circleci: Fix group install on Fedora 41 with dnf5 diff --git a/.circleci/config.yml b/.circleci/config.yml index 185e76fe2..8c930ec8a 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -228,19 +228,23 @@ jobs: command: | case "<< parameters.dist >>" in almalinux|fedora) - dnf -y groupinstall "Development Tools" case "<< parameters.dist >>:<< parameters.release >>" in almalinux:9) dnf -y install "dnf-command(config-manager)" dnf config-manager --set-enabled crb dnf -y install diffutils dnf -y install epel-release + dnf -y groupinstall "Development Tools" ;; almalinux:8) dnf -y install "dnf-command(config-manager)" dnf config-manager --set-enabled powertools dnf -y install diffutils dnf -y install epel-release + dnf -y groupinstall "Development Tools" + ;; + fedora:*) + dnf -y group install development-tools ;; esac dnf -y install \ From nils.goroll at uplex.de Tue Nov 5 18:41:10 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Tue, 5 Nov 2024 18:41:10 +0000 (UTC) Subject: [master] c75b8f55f Export XPORT_Register() Message-ID: <20241105184110.32995119668@lists.varnish-cache.org> commit c75b8f55f6cb87ec66829d92fc45ed33b5deb048 Author: Nils Goroll Date: Tue Nov 5 19:33:58 2024 +0100 Export XPORT_Register() Trivial prerequisite for transports from extensions. diff --git a/bin/varnishd/acceptor/cache_acceptor.c b/bin/varnishd/acceptor/cache_acceptor.c index ca08665c9..2d6e6350b 100644 --- a/bin/varnishd/acceptor/cache_acceptor.c +++ b/bin/varnishd/acceptor/cache_acceptor.c @@ -272,7 +272,7 @@ static VTAILQ_HEAD(,transport) transports = static uint16_t next_xport; -static void +void XPORT_Register(struct transport *xp) { diff --git a/bin/varnishd/cache/cache_transport.h b/bin/varnishd/cache/cache_transport.h index 79f0a7b2d..701f0dbfd 100644 --- a/bin/varnishd/cache/cache_transport.h +++ b/bin/varnishd/cache/cache_transport.h @@ -85,6 +85,7 @@ void H2_PU_Sess(struct worker *, struct sess *, struct req *); void H2_OU_Sess(struct worker *, struct sess *, struct req *); const struct transport *XPORT_ByNumber(uint16_t no); +void XPORT_Register(struct transport *xp); int VPX_Send_Proxy(int fd, int version, const struct sess *); /* cache_session.c */ From nils.goroll at uplex.de Tue Nov 5 20:03:05 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Tue, 5 Nov 2024 20:03:05 +0000 (UTC) Subject: [master] febf5a0c7 Fix expected year of NEXT release Message-ID: <20241105200305.3E55B11C2E1@lists.varnish-cache.org> commit febf5a0c73a970dcdbc6db2b5c0a849d1e148df4 Author: Nils Goroll Date: Tue Nov 5 21:01:51 2024 +0100 Fix expected year of NEXT release diff --git a/include/vrt.h b/include/vrt.h index 1309604b9..1b4be285e 100644 --- a/include/vrt.h +++ b/include/vrt.h @@ -57,7 +57,7 @@ * Whenever something is deleted or changed in a way which is not * binary/load-time compatible, increment MAJOR version * - * NEXT (2024-03-15) + * NEXT (2025-03-15) * 20.1 (2024-11-08 7.6.1) * VDI_EVENT_SICK added to enum vcl_event_e * 20.0 (2024-09-13) From nils.goroll at uplex.de Thu Nov 7 18:07:06 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 7 Nov 2024 18:07:06 +0000 (UTC) Subject: [master] 61f2841df vcl: add unset req.grace and unset req.ttl Message-ID: <20241107180706.83B001067BB@lists.varnish-cache.org> commit 61f2841df6b8ff6767615cf08565f71dced09ae1 Author: Nils Goroll Date: Thu Nov 7 18:38:32 2024 +0100 vcl: add unset req.grace and unset req.ttl There was no way to reset these from VCL to the "unset" value of -1. diff --git a/bin/varnishd/cache/cache_vrt_var.c b/bin/varnishd/cache/cache_vrt_var.c index 60409beca..f0726cc81 100644 --- a/bin/varnishd/cache/cache_vrt_var.c +++ b/bin/varnishd/cache/cache_vrt_var.c @@ -576,12 +576,24 @@ VRT_r_req_##nm(VRT_CTX) \ return (ctx->req->elem); \ } +#define REQ_VAR_U(nm, elem, val) \ + \ +VCL_VOID \ +VRT_u_req_##nm(VRT_CTX) \ +{ \ + CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC); \ + CHECK_OBJ_NOTNULL(ctx->req, REQ_MAGIC); \ + ctx->req->elem = val; \ +} + REQ_VAR_R(backend_hint, director_hint, VCL_BACKEND) REQ_VAR_L(ttl, d_ttl, VCL_DURATION, if (!(arg>0.0)) arg = 0;) REQ_VAR_R(ttl, d_ttl, VCL_DURATION) +REQ_VAR_U(ttl, d_ttl, -1) REQ_VAR_L(grace, d_grace, VCL_DURATION, if (!(arg>0.0)) arg = 0;) REQ_VAR_R(grace, d_grace, VCL_DURATION) +REQ_VAR_U(grace, d_grace, -1) VCL_VOID VRT_l_req_backend_hint(VRT_CTX, VCL_BACKEND be) diff --git a/bin/varnishtest/tests/b00064.vtc b/bin/varnishtest/tests/b00064.vtc index 47271831b..73c7d2dff 100644 --- a/bin/varnishtest/tests/b00064.vtc +++ b/bin/varnishtest/tests/b00064.vtc @@ -33,6 +33,10 @@ varnish v1 -vcl+backend { if (req.http.X-no-grace) { set req.grace = 0s; } + if (req.http.X-but-grace) { + unset req.grace; + } + set req.http.req-grace = req.grace; } sub vcl_hit { set req.http.X-grace = obj.grace; @@ -106,12 +110,11 @@ client c3 { delay .1 -# c4 does not disable grace, and should get the grace object even -# though c3 is waiting on the background thread to deliver a new -# version. +# c4 unsets req.grace after sessing it and should get the grace object even +# though c3 is waiting on the background thread to deliver a new version. client c4 { - txreq + txreq -hdr "X-no-grace: true" -hdr "X-but-grace: true" rxresp barrier b1 sync expect resp.status == 200 diff --git a/doc/sphinx/reference/vcl_var.rst b/doc/sphinx/reference/vcl_var.rst index aff4dd323..450c64fc7 100644 --- a/doc/sphinx/reference/vcl_var.rst +++ b/doc/sphinx/reference/vcl_var.rst @@ -272,12 +272,15 @@ req.grace Writable from: client + Unsettable from: client Upper limit on the object grace. During lookup the minimum of req.grace and the object's stored grace value will be used as the object's grace. + When reading the unset value, it is returned as -1. + .. _req.hash: @@ -536,9 +539,12 @@ req.ttl Writable from: client + Unsettable from: client + Upper limit on the object age for cache lookups to return hit. + When reading the unset value, it is returned as -1. .. _req.url: From nils.goroll at uplex.de Thu Nov 7 18:08:05 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Thu, 7 Nov 2024 18:08:05 +0000 (UTC) Subject: [master] 5ecf47d4b Fix whitespace Message-ID: <20241107180805.82183106991@lists.varnish-cache.org> commit 5ecf47d4ba4825431577a9b41da2d1ac07b21154 Author: Nils Goroll Date: Thu Nov 7 19:07:33 2024 +0100 Fix whitespace it looked right in git diff, but was not diff --git a/bin/varnishd/cache/cache_vrt_var.c b/bin/varnishd/cache/cache_vrt_var.c index f0726cc81..5d5be780d 100644 --- a/bin/varnishd/cache/cache_vrt_var.c +++ b/bin/varnishd/cache/cache_vrt_var.c @@ -579,7 +579,7 @@ VRT_r_req_##nm(VRT_CTX) \ #define REQ_VAR_U(nm, elem, val) \ \ VCL_VOID \ -VRT_u_req_##nm(VRT_CTX) \ +VRT_u_req_##nm(VRT_CTX) \ { \ CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC); \ CHECK_OBJ_NOTNULL(ctx->req, REQ_MAGIC); \ From dridi at varni.sh Thu Nov 7 18:33:00 2024 From: dridi at varni.sh (Dridi Boukelmoune) Date: Thu, 7 Nov 2024 18:33:00 +0000 Subject: [master] 61f2841df vcl: add unset req.grace and unset req.ttl In-Reply-To: <20241107180706.83B001067BB@lists.varnish-cache.org> References: <20241107180706.83B001067BB@lists.varnish-cache.org> Message-ID: On Thu, Nov 7, 2024 at 6:07?PM Nils Goroll wrote: > > > commit 61f2841df6b8ff6767615cf08565f71dced09ae1 > Author: Nils Goroll > Date: Thu Nov 7 18:38:32 2024 +0100 > > vcl: add unset req.grace and unset req.ttl > > There was no way to reset these from VCL to the "unset" value of -1. There was a way awaiting your review: https://github.com/varnishcache/varnish-cache/pull/4041 > diff --git a/bin/varnishd/cache/cache_vrt_var.c b/bin/varnishd/cache/cache_vrt_var.c > index 60409beca..f0726cc81 100644 > --- a/bin/varnishd/cache/cache_vrt_var.c > +++ b/bin/varnishd/cache/cache_vrt_var.c > @@ -576,12 +576,24 @@ VRT_r_req_##nm(VRT_CTX) \ > return (ctx->req->elem); \ > } > > +#define REQ_VAR_U(nm, elem, val) \ > + \ > +VCL_VOID \ > +VRT_u_req_##nm(VRT_CTX) \ > +{ \ > + CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC); \ > + CHECK_OBJ_NOTNULL(ctx->req, REQ_MAGIC); \ > + ctx->req->elem = val; \ > +} > + > REQ_VAR_R(backend_hint, director_hint, VCL_BACKEND) > > REQ_VAR_L(ttl, d_ttl, VCL_DURATION, if (!(arg>0.0)) arg = 0;) > REQ_VAR_R(ttl, d_ttl, VCL_DURATION) > +REQ_VAR_U(ttl, d_ttl, -1) > REQ_VAR_L(grace, d_grace, VCL_DURATION, if (!(arg>0.0)) arg = 0;) > REQ_VAR_R(grace, d_grace, VCL_DURATION) > +REQ_VAR_U(grace, d_grace, -1) > > VCL_VOID > VRT_l_req_backend_hint(VRT_CTX, VCL_BACKEND be) > diff --git a/bin/varnishtest/tests/b00064.vtc b/bin/varnishtest/tests/b00064.vtc > index 47271831b..73c7d2dff 100644 > --- a/bin/varnishtest/tests/b00064.vtc > +++ b/bin/varnishtest/tests/b00064.vtc > @@ -33,6 +33,10 @@ varnish v1 -vcl+backend { > if (req.http.X-no-grace) { > set req.grace = 0s; > } > + if (req.http.X-but-grace) { > + unset req.grace; > + } > + set req.http.req-grace = req.grace; > } > sub vcl_hit { > set req.http.X-grace = obj.grace; > @@ -106,12 +110,11 @@ client c3 { > > delay .1 > > -# c4 does not disable grace, and should get the grace object even > -# though c3 is waiting on the background thread to deliver a new > -# version. > +# c4 unsets req.grace after sessing it and should get the grace object even > +# though c3 is waiting on the background thread to deliver a new version. > > client c4 { > - txreq > + txreq -hdr "X-no-grace: true" -hdr "X-but-grace: true" > rxresp > barrier b1 sync > expect resp.status == 200 > diff --git a/doc/sphinx/reference/vcl_var.rst b/doc/sphinx/reference/vcl_var.rst > index aff4dd323..450c64fc7 100644 > --- a/doc/sphinx/reference/vcl_var.rst > +++ b/doc/sphinx/reference/vcl_var.rst > @@ -272,12 +272,15 @@ req.grace > > Writable from: client > > + Unsettable from: client > > Upper limit on the object grace. > > During lookup the minimum of req.grace and the object's stored > grace value will be used as the object's grace. > > + When reading the unset value, it is returned as -1. > + > > .. _req.hash: > > @@ -536,9 +539,12 @@ req.ttl > > Writable from: client > > + Unsettable from: client > + > > Upper limit on the object age for cache lookups to return hit. > > + When reading the unset value, it is returned as -1. > > .. _req.url: > > _______________________________________________ > varnish-commit mailing list > varnish-commit at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-commit From nils.goroll at uplex.de Fri Nov 8 08:06:13 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Fri, 8 Nov 2024 08:06:13 +0000 (UTC) Subject: [master] 550f76ddb doc: Polish identifiers / strings section Message-ID: <20241108080613.2C226907A@lists.varnish-cache.org> commit 550f76ddbd2c95bc5e795e329993df8f8ee36e0d Author: Nils Goroll Date: Fri Nov 8 09:05:08 2024 +0100 doc: Polish identifiers / strings section Follow-up 11f8d31592e8d3e52a92ab8ca6cd3d54d596c6ed diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst index 90215b2af..295a55e79 100644 --- a/doc/sphinx/reference/vcl.rst +++ b/doc/sphinx/reference/vcl.rst @@ -54,13 +54,15 @@ Character Sets .. _VMODs: https://varnish-cache.org/docs/trunk/reference/vmod.html -While identifiers are written in ASCII, strings can contain any character set -as long as the *NUL* (zero, 0) byte is reserved. The Varnish Configuration -Language itself is not concerned with the character encoding of strings, VCL -code handling strings in different character sets needs to track encodings -itself. `VMODs`_ exist to help with such tasks. - -To illustrate, ``"??"`` is a valid UTF-8 *string* in UTF-8, but ``??`` is not a +While identifiers can only consist of this subset of ASCII, **strings** can +contain any bytes except *NUL* (zero, 0), which marks the end of the string. The +Varnish Configuration Language itself is not concerned with the character +encoding of strings. VCL code handling strings in different character sets needs +to track encodings itself. `VMODs`_ exist to help with such tasks (e.g. +``iconv``). + +To illustrate, ``"??"`` is a valid **string** which, when interpreted as UTF-8, +happens to represent code point ``U+1F96C`` (Leafy Green) but ``??`` is not a valid *identifier*. Character Sets in HTTP From nils.goroll at uplex.de Sat Nov 9 19:14:08 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Sat, 9 Nov 2024 19:14:08 +0000 (UTC) Subject: [master] 41059cf68 changes: Populate changelog for 7.6.1 Message-ID: <20241109191409.061EC643CF@lists.varnish-cache.org> commit 41059cf6815d0f366a5145fdb69d310e37e6d380 Author: Walid Boudebouda Date: Thu Nov 7 16:24:37 2024 +0100 changes: Populate changelog for 7.6.1 Minor formatting edits by committer Conflicts: doc/changes.rst diff --git a/doc/changes.rst b/doc/changes.rst index 5f920f8e6..e465cd0ed 100644 --- a/doc/changes.rst +++ b/doc/changes.rst @@ -41,9 +41,33 @@ Varnish Cache NEXT (2025-03-15) .. PLEASE keep this roughly in commit order as shown by git-log / tig (new to old) -* The scope of VCL variables `req.is_hitmiss` and `req.is_hitpass` is now restricted - to `vcl_miss, vcl_deliver, vcl_pass, vcl_synth` and `vcl_pass, vcl_deliver, vcl_synth` - respectively. +* The scope of VCL variables ``req.is_hitmiss`` and ``req.is_hitpass`` is now + restricted to ``vcl_miss, vcl_deliver, vcl_pass, vcl_synth`` and ``vcl_pass, + vcl_deliver, vcl_synth`` respectively. + +================================ +Varnish Cache 7.6.1 (2024-11-08) +================================ + +* Fixed a bug introduced in 7.6.0 that could trigger a panic when using dynamic + backends (4183_). + +* Resolved a race condition that caused Varnish to exceed the connection limit + set to a backend with the ``.max_connections`` attribute (4154_). + +* Fixed an assertion that was added in 7.6.0 and that could lead to a panic in + the waiter code under certain conditions (4204_). + +* Removed an assertion on the pid value of varnishd that could trigger a panic in + container environments. + +* Added attempt to raise RLIMIT_MEMLOCK to infinity on startup and improved logging + for VSM mlock() errors. (4193_) + +.. _4183: https://github.com/varnishcache/varnish-cache/issues/4183 +.. _4154: https://github.com/varnishcache/varnish-cache/pull/4154 +.. _4204: https://github.com/varnishcache/varnish-cache/issues/4204 +.. _4193: https://github.com/varnishcache/varnish-cache/issues/4193 ================================ Varnish Cache 7.6.0 (2024-09-13) From nils.goroll at uplex.de Mon Nov 18 13:45:07 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 18 Nov 2024 13:45:07 +0000 (UTC) Subject: [master] 117161c85 acceptor_uds: Rename for consistency Message-ID: <20241118134507.DABC6A2430@lists.varnish-cache.org> commit 117161c851df7661b18b262ef502dbabe7065115 Author: Nils Goroll Date: Sat Nov 16 14:53:40 2024 +0100 acceptor_uds: Rename for consistency the make_session function of uds still had a generic name. diff --git a/bin/varnishd/acceptor/cache_acceptor_uds.c b/bin/varnishd/acceptor/cache_acceptor_uds.c index 513d5161e..b71cd1cc4 100644 --- a/bin/varnishd/acceptor/cache_acceptor_uds.c +++ b/bin/varnishd/acceptor/cache_acceptor_uds.c @@ -311,7 +311,7 @@ vca_mk_uds(struct wrk_accept *wa, struct sess *sp) } static void v_matchproto_(task_func_t) -vca_make_session(struct worker *wrk, void *arg) +vca_uds_make_session(struct worker *wrk, void *arg) { struct wrk_accept *wa; struct sess *sp; @@ -452,7 +452,7 @@ vca_uds_accept_task(struct worker *wrk, void *arg) wa.acceptsock = i; if (!Pool_Task_Arg(wrk, TASK_QUEUE_REQ, - vca_make_session, &wa, sizeof wa)) { + vca_uds_make_session, &wa, sizeof wa)) { /* * We couldn't get another thread, so we will handle * the request in this worker thread, but first we From nils.goroll at uplex.de Mon Nov 18 14:08:03 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 18 Nov 2024 14:08:03 +0000 (UTC) Subject: [master] a865812c0 vsha256: Insist that our byte order macros are defined Message-ID: <20241118140803.B1E75A3380@lists.varnish-cache.org> commit a865812c070bce1c4e6f5946862c994e5c57b9e9 Author: Nils Goroll Date: Mon Nov 18 14:42:36 2024 +0100 vsha256: Insist that our byte order macros are defined to avoid errors and confusion as documented in #4226 diff --git a/lib/libvarnish/vsha256.c b/lib/libvarnish/vsha256.c index 6b6f756cb..7e55ec3ad 100644 --- a/lib/libvarnish/vsha256.c +++ b/lib/libvarnish/vsha256.c @@ -40,6 +40,14 @@ # define VBIG_ENDIAN __DARWIN_BIG_ENDIAN #endif +#ifndef VBYTE_ORDER +# error VBYTE_ORDER not defined +#endif +#ifndef VBIG_ENDIAN +# error VBIG_ENDIAN not defined +#endif + + #include #include #include @@ -50,7 +58,7 @@ #include "vend.h" #include "vsha256.h" -#if defined(VBYTE_ORDER) && VBYTE_ORDER == VBIG_ENDIAN +#if VBYTE_ORDER == VBIG_ENDIAN /* Copy a vector of big-endian uint32_t into a vector of bytes */ #define be32enc_vect(dst, src, len) \ From nils.goroll at uplex.de Mon Nov 18 14:32:05 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 18 Nov 2024 14:32:05 +0000 (UTC) Subject: [master] d05d20e08 Tolerate when endian.h is symlink to sys/endian.h Message-ID: <20241118143205.BC9A3A43BD@lists.varnish-cache.org> commit d05d20e08b3c6435afbecee329d9171105f25652 Author: Kirill A. Korinsky Date: Thu Nov 7 17:58:08 2024 +0100 Tolerate when endian.h is symlink to sys/endian.h Since 5a60b36fe9b45b68244596d93db2b922d53e7b5c varnish drop support of sys/endian.h and they had a bit different API one or two leading `_`. Some system may have symlink from endian.h to sys/endian.h, for example OpenBSD. On such system build fails due to wrong SHA256 hash. diff --git a/lib/libvarnish/vsha256.c b/lib/libvarnish/vsha256.c index 7e55ec3ad..29f97fe40 100644 --- a/lib/libvarnish/vsha256.c +++ b/lib/libvarnish/vsha256.c @@ -33,8 +33,16 @@ #ifndef __DARWIN_BYTE_ORDER # include -# define VBYTE_ORDER __BYTE_ORDER -# define VBIG_ENDIAN __BIG_ENDIAN +# ifdef _BYTE_ORDER +# define VBYTE_ORDER _BYTE_ORDER +# else +# define VBYTE_ORDER __BYTE_ORDER +# endif +# ifdef _BIG_ENDIAN +# define VBIG_ENDIAN _BIG_ENDIAN +# else +# define VBIG_ENDIAN __BIG_ENDIAN +# endif #else # define VBYTE_ORDER __DARWIN_BYTE_ORDER # define VBIG_ENDIAN __DARWIN_BIG_ENDIAN From nils.goroll at uplex.de Mon Nov 18 14:41:03 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 18 Nov 2024 14:41:03 +0000 (UTC) Subject: [master] c3b64c8a6 http1: Reserve session workspace for the protocol private pointer only once Message-ID: <20241118144103.144ABA4AED@lists.varnish-cache.org> commit c3b64c8a6f35182133416a0971ef2955e862b887 Author: Nils Goroll Date: Tue Nov 5 20:29:02 2024 +0100 http1: Reserve session workspace for the protocol private pointer only once This allows to re-use http1_new_session from custom implementations, in that it can serve as an entry point into the state machine also for existing sessions (without requiting 8 bytes of session workspace for each request served). diff --git a/bin/varnishd/http1/cache_http1_fsm.c b/bin/varnishd/http1/cache_http1_fsm.c index cc3420362..ed60f5d5b 100644 --- a/bin/varnishd/http1/cache_http1_fsm.c +++ b/bin/varnishd/http1/cache_http1_fsm.c @@ -114,7 +114,10 @@ http1_new_session(struct worker *wrk, void *arg) CHECK_OBJ_NOTNULL(sp, SESS_MAGIC); HTC_RxInit(req->htc, req->ws); - if (!SES_Reserve_proto_priv(sp, &u, &sz)) { + + sz = sizeof u; + if (SES_Get_proto_priv(sp, &u) && + !SES_Reserve_proto_priv(sp, &u, &sz)) { /* Out of session workspace. Free the req, close the sess, * and do not set a new task func, which will exit the * worker thread. */ From nils.goroll at uplex.de Mon Nov 18 14:41:03 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 18 Nov 2024 14:41:03 +0000 (UTC) Subject: [master] f279c0c96 http1: Do not reset the transport pointer Message-ID: <20241118144103.28586A4AF0@lists.varnish-cache.org> commit f279c0c96c1d39e21ac0dd181c71a1b76faaf1d3 Author: Nils Goroll Date: Tue Nov 5 19:14:26 2024 +0100 http1: Do not reset the transport pointer This allows code-reuse from custom implementations where only the deliver deliver callback differs. diff --git a/bin/varnishd/http1/cache_http1_fsm.c b/bin/varnishd/http1/cache_http1_fsm.c index ed60f5d5b..1d9efb1aa 100644 --- a/bin/varnishd/http1/cache_http1_fsm.c +++ b/bin/varnishd/http1/cache_http1_fsm.c @@ -82,7 +82,6 @@ http1_req(struct worker *wrk, void *arg) CAST_OBJ_NOTNULL(req, arg, REQ_MAGIC); THR_SetRequest(req); - req->transport = &HTTP1_transport; assert(!WS_IsReserved(wrk->aws)); HTTP1_Session(wrk, req); AZ(wrk->v1l); @@ -309,8 +308,7 @@ HTTP1_Session(struct worker *wrk, struct req *req) */ if (http1_getstate(sp) == H1NEWREQ) VTCP_blocking(sp->fd); - - req->transport = &HTTP1_transport; + req->transport = XPORT_ByNumber(sp->sattr[SA_TRANSPORT]); while (1) { st = http1_getstate(sp); From nils.goroll at uplex.de Mon Nov 25 14:43:03 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 25 Nov 2024 14:43:03 +0000 (UTC) Subject: [master] 8c872002c cache_http1_line: Refactor V1L_Open() to return struct v1l * Message-ID: <20241125144303.361D611296E@lists.varnish-cache.org> commit 8c872002c75c9dd38c5f114720dd5f893449a03c Author: Nils Goroll Date: Thu Oct 31 10:00:03 2024 +0100 cache_http1_line: Refactor V1L_Open() to return struct v1l * diff --git a/bin/varnishd/http1/cache_http1.h b/bin/varnishd/http1/cache_http1.h index f758a74c9..8c161d5f5 100644 --- a/bin/varnishd/http1/cache_http1.h +++ b/bin/varnishd/http1/cache_http1.h @@ -61,7 +61,7 @@ void V1P_Charge(struct req *, const struct v1p_acct *, struct VSC_vbe *); /* cache_http1_line.c */ void V1L_Chunked(const struct worker *w); void V1L_EndChunk(const struct worker *w); -void V1L_Open(struct worker *, struct ws *, int *fd, struct vsl_log *, +struct v1l * V1L_Open(struct ws *, int *fd, struct vsl_log *, vtim_real deadline, unsigned niov); stream_close_t V1L_Flush(const struct worker *w); stream_close_t V1L_Close(struct worker *w, uint64_t *cnt); diff --git a/bin/varnishd/http1/cache_http1_deliver.c b/bin/varnishd/http1/cache_http1_deliver.c index 15bc706b9..6bce2558c 100644 --- a/bin/varnishd/http1/cache_http1_deliver.c +++ b/bin/varnishd/http1/cache_http1_deliver.c @@ -46,8 +46,10 @@ v1d_error(struct req *req, struct boc *boc, const char *msg) "HTTP/1.1 500 Internal Server Error\r\n" "Server: Varnish\r\n" "Connection: close\r\n\r\n"; + uint64_t bytes; - AZ(req->wrk->v1l); + if (req->wrk->v1l != NULL) + (void) V1L_Close(req->wrk, &bytes); VSLbs(req->vsl, SLT_Error, TOSTRAND(msg)); VSLb(req->vsl, SLT_RespProtocol, "HTTP/1.1"); @@ -71,6 +73,7 @@ V1D_Deliver(struct req *req, struct boc *boc, int sendbody) int err = 0, chunked = 0; stream_close_t sc; uint64_t hdrbytes, bytes; + struct v1l *v1l; CHECK_OBJ_NOTNULL(req, REQ_MAGIC); CHECK_OBJ_ORNULL(boc, BOC_MAGIC); @@ -87,6 +90,20 @@ V1D_Deliver(struct req *req, struct boc *boc, int sendbody) } else if (!http_GetHdr(req->resp, H_Connection, NULL)) http_SetHeader(req->resp, "Connection: keep-alive"); + CHECK_OBJ_NOTNULL(req->wrk, WORKER_MAGIC); + + v1l = V1L_Open(req->wrk->aws, &req->sp->fd, req->vsl, + req->t_prev + SESS_TMO(req->sp, send_timeout), + cache_param->http1_iovs); + + if (v1l == NULL) { + v1d_error(req, boc, "Failure to init v1d (workspace_thread overflow)"); + return; + } + + AZ(req->wrk->v1l); + req->wrk->v1l = v1l; + if (sendbody) { if (!http_GetHdr(req->resp, H_Content_Length, NULL)) { if (req->http->protover == 11) { @@ -115,10 +132,6 @@ V1D_Deliver(struct req *req, struct boc *boc, int sendbody) return; } - V1L_Open(req->wrk, req->wrk->aws, &req->sp->fd, req->vsl, - req->t_prev + SESS_TMO(req->sp, send_timeout), - cache_param->http1_iovs); - if (WS_Overflowed(req->wrk->aws)) { v1d_error(req, boc, "workspace_thread overflow"); return; diff --git a/bin/varnishd/http1/cache_http1_fetch.c b/bin/varnishd/http1/cache_http1_fetch.c index 474175ce0..817f7a969 100644 --- a/bin/varnishd/http1/cache_http1_fetch.c +++ b/bin/varnishd/http1/cache_http1_fetch.c @@ -72,6 +72,7 @@ V1F_SendReq(struct worker *wrk, struct busyobj *bo, uint64_t *ctr_hdrbytes, struct vdp_ctx vdc[1] = {{ 0 }}; intmax_t cl; const char *err = NULL; + struct v1l *v1l = NULL; CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC); CHECK_OBJ_NOTNULL(bo, BUSYOBJ_MAGIC); @@ -100,9 +101,8 @@ V1F_SendReq(struct worker *wrk, struct busyobj *bo, uint64_t *ctr_hdrbytes, if (bo->vdp_filter_list != NULL && VCL_StackVDP(vdc, bo->vcl, bo->vdp_filter_list, NULL, bo)) err = "Failure to push processors"; - else if (V1L_Open(wrk, wrk->aws, htc->rfd, bo->vsl, nan(""), 0), - wrk->v1l == NULL) { - /* ^^^^^^ + else if ((v1l = V1L_Open(wrk->aws, htc->rfd, bo->vsl, nan(""), 0)) == NULL) { + /* ^^^^^^^^ * XXX: need a send_timeout for the backend side * XXX: use cache_param->http1_iovs ? */ @@ -111,8 +111,11 @@ V1F_SendReq(struct worker *wrk, struct busyobj *bo, uint64_t *ctr_hdrbytes, else if (v1f_stackv1l(vdc, bo)) err = "Failure to push V1L"; + AZ(wrk->v1l); + wrk->v1l = v1l; + if (err != NULL) { - if (wrk->v1l != NULL) + if (v1l != NULL) (void) V1L_Close(wrk, &bytes); if (VALID_OBJ(vdc, VDP_CTX_MAGIC)) (void) VDP_Close(vdc, NULL, NULL); @@ -122,6 +125,7 @@ V1F_SendReq(struct worker *wrk, struct busyobj *bo, uint64_t *ctr_hdrbytes, return (-1); } + assert(cl >= -1); if (cl < 0) http_PrintfHeader(hp, "Transfer-Encoding: chunked"); diff --git a/bin/varnishd/http1/cache_http1_line.c b/bin/varnishd/http1/cache_http1_line.c index b1087d1e6..d18bd6d61 100644 --- a/bin/varnishd/http1/cache_http1_line.c +++ b/bin/varnishd/http1/cache_http1_line.c @@ -75,19 +75,16 @@ struct v1l { * otherwise, up to niov */ -void -V1L_Open(struct worker *wrk, struct ws *ws, int *fd, struct vsl_log *vsl, +struct v1l * +V1L_Open(struct ws *ws, int *fd, struct vsl_log *vsl, vtim_real deadline, unsigned niov) { struct v1l *v1l; unsigned u; uintptr_t ws_snap; - CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC); - AZ(wrk->v1l); - if (WS_Overflowed(ws)) - return; + return (NULL); if (niov != 0) assert(niov >= 3); @@ -96,7 +93,7 @@ V1L_Open(struct worker *wrk, struct ws *ws, int *fd, struct vsl_log *vsl, v1l = WS_Alloc(ws, sizeof *v1l); if (v1l == NULL) - return; + return (NULL); INIT_OBJ(v1l, V1L_MAGIC); v1l->ws = ws; @@ -107,7 +104,7 @@ V1L_Open(struct worker *wrk, struct ws *ws, int *fd, struct vsl_log *vsl, /* Must have at least 3 in case of chunked encoding */ WS_Release(ws, 0); WS_MarkOverflow(ws); - return; + return (NULL); } if (u > IOV_MAX) u = IOV_MAX; @@ -121,10 +118,8 @@ V1L_Open(struct worker *wrk, struct ws *ws, int *fd, struct vsl_log *vsl, v1l->vsl = vsl; v1l->werr = SC_NULL; - AZ(wrk->v1l); - wrk->v1l = v1l; - WS_Release(ws, u * sizeof(struct iovec)); + return (v1l); } stream_close_t From nils.goroll at uplex.de Mon Nov 25 14:43:03 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 25 Nov 2024 14:43:03 +0000 (UTC) Subject: [master] 006cb7aff cache_http1_line: Make the v1l VDP a bit less special Message-ID: <20241125144303.6BFC4112972@lists.varnish-cache.org> commit 006cb7aff21ef51cbfaf00eb116cedd30265a699 Author: Nils Goroll Date: Thu Oct 31 10:08:12 2024 +0100 cache_http1_line: Make the v1l VDP a bit less special V1L is used to send HTTP headers and the body, so it is used directly from delivery/fetch first and then as a VDP. From the times before VDPs, the V1L VDP still had its private pointer in struct wrk. This commit is to move the private pointer to the VDP entry. The caller remains responsible for calling V1L_Close() to keep V1L independent of VDP. diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h index b52cbab01..d9cd0094f 100644 --- a/bin/varnishd/cache/cache.h +++ b/bin/varnishd/cache/cache.h @@ -252,8 +252,6 @@ struct worker { vtim_real lastused; - struct v1l *v1l; - pthread_cond_t cond; struct ws aws[1]; diff --git a/bin/varnishd/cache/cache_varnishd.h b/bin/varnishd/cache/cache_varnishd.h index 31fd57eb6..a0e4987fa 100644 --- a/bin/varnishd/cache/cache_varnishd.h +++ b/bin/varnishd/cache/cache_varnishd.h @@ -300,7 +300,8 @@ htc_complete_f HTTP1_Complete; uint16_t HTTP1_DissectRequest(struct http_conn *, struct http *); uint16_t HTTP1_DissectResponse(struct http_conn *, struct http *resp, const struct http *req); -unsigned HTTP1_Write(const struct worker *w, const struct http *hp, const int*); +struct v1l; +unsigned HTTP1_Write(struct v1l *v1l, const struct http *hp, const int*); /* cache_main.c */ vxid_t VXID_Get(const struct worker *, uint64_t marker); diff --git a/bin/varnishd/http1/cache_http1.h b/bin/varnishd/http1/cache_http1.h index 8c161d5f5..98ed1e1df 100644 --- a/bin/varnishd/http1/cache_http1.h +++ b/bin/varnishd/http1/cache_http1.h @@ -59,11 +59,11 @@ stream_close_t V1P_Process(const struct req *, int fd, struct v1p_acct *, void V1P_Charge(struct req *, const struct v1p_acct *, struct VSC_vbe *); /* cache_http1_line.c */ -void V1L_Chunked(const struct worker *w); -void V1L_EndChunk(const struct worker *w); +void V1L_Chunked(struct v1l *v1l); +void V1L_EndChunk(struct v1l *v1l); struct v1l * V1L_Open(struct ws *, int *fd, struct vsl_log *, vtim_real deadline, unsigned niov); -stream_close_t V1L_Flush(const struct worker *w); -stream_close_t V1L_Close(struct worker *w, uint64_t *cnt); -size_t V1L_Write(const struct worker *w, const void *ptr, ssize_t len); +stream_close_t V1L_Flush(struct v1l *v1l); +stream_close_t V1L_Close(struct v1l **v1lp, uint64_t *cnt); +size_t V1L_Write(struct v1l *v1l, const void *ptr, ssize_t len); extern const struct vdp * const VDP_v1l; diff --git a/bin/varnishd/http1/cache_http1_deliver.c b/bin/varnishd/http1/cache_http1_deliver.c index 6bce2558c..462a9357f 100644 --- a/bin/varnishd/http1/cache_http1_deliver.c +++ b/bin/varnishd/http1/cache_http1_deliver.c @@ -40,7 +40,7 @@ /*--------------------------------------------------------------------*/ static void -v1d_error(struct req *req, struct boc *boc, const char *msg) +v1d_error(struct req *req, struct boc *boc, struct v1l **v1lp, const char *msg) { static const char r_500[] = "HTTP/1.1 500 Internal Server Error\r\n" @@ -48,8 +48,9 @@ v1d_error(struct req *req, struct boc *boc, const char *msg) "Connection: close\r\n\r\n"; uint64_t bytes; - if (req->wrk->v1l != NULL) - (void) V1L_Close(req->wrk, &bytes); + AN(v1lp); + if (*v1lp != NULL) + (void) V1L_Close(v1lp, &bytes); VSLbs(req->vsl, SLT_Error, TOSTRAND(msg)); VSLb(req->vsl, SLT_RespProtocol, "HTTP/1.1"); @@ -97,13 +98,10 @@ V1D_Deliver(struct req *req, struct boc *boc, int sendbody) cache_param->http1_iovs); if (v1l == NULL) { - v1d_error(req, boc, "Failure to init v1d (workspace_thread overflow)"); + v1d_error(req, boc, &v1l, "Failure to init v1d (workspace_thread overflow)"); return; } - AZ(req->wrk->v1l); - req->wrk->v1l = v1l; - if (sendbody) { if (!http_GetHdr(req->resp, H_Content_Length, NULL)) { if (req->http->protover == 11) { @@ -116,41 +114,40 @@ V1D_Deliver(struct req *req, struct boc *boc, int sendbody) } INIT_OBJ(ctx, VRT_CTX_MAGIC); VCL_Req2Ctx(ctx, req); - if (VDP_Push(ctx, req->vdc, req->ws, VDP_v1l, NULL)) { - v1d_error(req, boc, "Failure to push v1d processor"); + if (VDP_Push(ctx, req->vdc, req->ws, VDP_v1l, v1l)) { + v1d_error(req, boc, &v1l, "Failure to push v1d processor"); return; } } if (WS_Overflowed(req->ws)) { - v1d_error(req, boc, "workspace_client overflow"); + v1d_error(req, boc, &v1l, "workspace_client overflow"); return; } if (WS_Overflowed(req->sp->ws)) { - v1d_error(req, boc, "workspace_session overflow"); + v1d_error(req, boc, &v1l, "workspace_session overflow"); return; } if (WS_Overflowed(req->wrk->aws)) { - v1d_error(req, boc, "workspace_thread overflow"); + v1d_error(req, boc, &v1l, "workspace_thread overflow"); return; } - hdrbytes = HTTP1_Write(req->wrk, req->resp, HTTP1_Resp); + hdrbytes = HTTP1_Write(v1l, req->resp, HTTP1_Resp); if (sendbody) { if (DO_DEBUG(DBG_FLUSH_HEAD)) - (void)V1L_Flush(req->wrk); + (void)V1L_Flush(v1l); if (chunked) - V1L_Chunked(req->wrk); + V1L_Chunked(v1l); err = VDP_DeliverObj(req->vdc, req->objcore); if (!err && chunked) - V1L_EndChunk(req->wrk); + V1L_EndChunk(v1l); } - sc = V1L_Close(req->wrk, &bytes); - AZ(req->wrk->v1l); + sc = V1L_Close(&v1l, &bytes); req->acct.resp_hdrbytes += hdrbytes; req->acct.resp_bodybytes += VDP_Close(req->vdc, req->objcore, boc); diff --git a/bin/varnishd/http1/cache_http1_fetch.c b/bin/varnishd/http1/cache_http1_fetch.c index 817f7a969..e0c53b96b 100644 --- a/bin/varnishd/http1/cache_http1_fetch.c +++ b/bin/varnishd/http1/cache_http1_fetch.c @@ -43,13 +43,13 @@ #include "cache_http1.h" static int -v1f_stackv1l(struct vdp_ctx *vdc, struct busyobj *bo) +v1f_stackv1l(struct vdp_ctx *vdc, struct busyobj *bo, struct v1l *v1l) { struct vrt_ctx ctx[1]; INIT_OBJ(ctx, VRT_CTX_MAGIC); VCL_Bo2Ctx(ctx, bo); - return (VDP_Push(ctx, vdc, ctx->ws, VDP_v1l, NULL)); + return (VDP_Push(ctx, vdc, ctx->ws, VDP_v1l, v1l)); } /*-------------------------------------------------------------------- @@ -108,15 +108,12 @@ V1F_SendReq(struct worker *wrk, struct busyobj *bo, uint64_t *ctr_hdrbytes, */ err = "Failure to open V1L (workspace_thread overflow)"; } - else if (v1f_stackv1l(vdc, bo)) + else if (v1f_stackv1l(vdc, bo, v1l)) err = "Failure to push V1L"; - AZ(wrk->v1l); - wrk->v1l = v1l; - if (err != NULL) { if (v1l != NULL) - (void) V1L_Close(wrk, &bytes); + (void) V1L_Close(&v1l, &bytes); if (VALID_OBJ(vdc, VDP_CTX_MAGIC)) (void) VDP_Close(vdc, NULL, NULL); VSLb(bo->vsl, SLT_FetchError, "%s", err); @@ -131,7 +128,7 @@ V1F_SendReq(struct worker *wrk, struct busyobj *bo, uint64_t *ctr_hdrbytes, http_PrintfHeader(hp, "Transfer-Encoding: chunked"); VTCP_blocking(*htc->rfd); /* XXX: we should timeout instead */ - hdrbytes = HTTP1_Write(wrk, hp, HTTP1_Req); + hdrbytes = HTTP1_Write(v1l, hp, HTTP1_Req); /* Deal with any message-body the request might (still) have */ i = 0; @@ -144,7 +141,7 @@ V1F_SendReq(struct worker *wrk, struct busyobj *bo, uint64_t *ctr_hdrbytes, } else if (bo->req != NULL && bo->req->req_body_status != BS_NONE) { if (cl < 0) - V1L_Chunked(wrk); + V1L_Chunked(v1l); i = VRB_Iterate(wrk, bo->vsl, bo->req, VDP_ObjIterate, vdc); if (bo->req->req_body_status != BS_CACHED) @@ -167,10 +164,10 @@ V1F_SendReq(struct worker *wrk, struct busyobj *bo, uint64_t *ctr_hdrbytes, bo->req->doclose = SC_RX_BODY; } if (cl < 0) - V1L_EndChunk(wrk); + V1L_EndChunk(v1l); } - sc = V1L_Close(wrk, &bytes); + sc = V1L_Close(&v1l, &bytes); CHECK_OBJ_NOTNULL(sc, STREAM_CLOSE_MAGIC); /* Bytes accounting */ diff --git a/bin/varnishd/http1/cache_http1_fsm.c b/bin/varnishd/http1/cache_http1_fsm.c index 1d9efb1aa..e4fd595f4 100644 --- a/bin/varnishd/http1/cache_http1_fsm.c +++ b/bin/varnishd/http1/cache_http1_fsm.c @@ -84,7 +84,6 @@ http1_req(struct worker *wrk, void *arg) THR_SetRequest(req); assert(!WS_IsReserved(wrk->aws)); HTTP1_Session(wrk, req); - AZ(wrk->v1l); WS_Assert(wrk->aws); THR_SetRequest(NULL); } diff --git a/bin/varnishd/http1/cache_http1_line.c b/bin/varnishd/http1/cache_http1_line.c index d18bd6d61..771490986 100644 --- a/bin/varnishd/http1/cache_http1_line.c +++ b/bin/varnishd/http1/cache_http1_line.c @@ -68,6 +68,7 @@ struct v1l { ssize_t cnt; /* Flushed byte count */ struct ws *ws; uintptr_t ws_snap; + void **vdp_priv; }; /*-------------------------------------------------------------------- @@ -123,17 +124,20 @@ V1L_Open(struct ws *ws, int *fd, struct vsl_log *vsl, } stream_close_t -V1L_Close(struct worker *wrk, uint64_t *cnt) +V1L_Close(struct v1l **v1lp, uint64_t *cnt) { struct v1l *v1l; struct ws *ws; uintptr_t ws_snap; stream_close_t sc; - CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC); AN(cnt); - sc = V1L_Flush(wrk); - TAKE_OBJ_NOTNULL(v1l, &wrk->v1l, V1L_MAGIC); + TAKE_OBJ_NOTNULL(v1l, v1lp, V1L_MAGIC); + if (v1l->vdp_priv != NULL) { + assert(*v1l->vdp_priv == v1l); + *v1l->vdp_priv = NULL; + } + sc = V1L_Flush(v1l); *cnt = v1l->cnt; ws = v1l->ws; ws_snap = v1l->ws_snap; @@ -167,15 +171,12 @@ v1l_prune(struct v1l *v1l, size_t bytes) } stream_close_t -V1L_Flush(const struct worker *wrk) +V1L_Flush(struct v1l *v1l) { ssize_t i; int err; - struct v1l *v1l; char cbuf[32]; - CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC); - v1l = wrk->v1l; CHECK_OBJ_NOTNULL(v1l, V1L_MAGIC); CHECK_OBJ_NOTNULL(v1l->werr, STREAM_CLOSE_MAGIC); AN(v1l->wfd); @@ -261,12 +262,9 @@ V1L_Flush(const struct worker *wrk) } size_t -V1L_Write(const struct worker *wrk, const void *ptr, ssize_t len) +V1L_Write(struct v1l *v1l, const void *ptr, ssize_t len) { - struct v1l *v1l; - CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC); - v1l = wrk->v1l; CHECK_OBJ_NOTNULL(v1l, V1L_MAGIC); AN(v1l->wfd); if (len == 0 || *v1l->wfd < 0) @@ -280,19 +278,16 @@ V1L_Write(const struct worker *wrk, const void *ptr, ssize_t len) v1l->niov++; v1l->cliov += len; if (v1l->niov >= v1l->siov) { - (void)V1L_Flush(wrk); + (void)V1L_Flush(v1l); VSC_C_main->http1_iovs_flush++; } return (len); } void -V1L_Chunked(const struct worker *wrk) +V1L_Chunked(struct v1l *v1l) { - struct v1l *v1l; - CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC); - v1l = wrk->v1l; CHECK_OBJ_NOTNULL(v1l, V1L_MAGIC); assert(v1l->ciov == v1l->siov); @@ -302,7 +297,7 @@ V1L_Chunked(const struct worker *wrk) * a chunk tail, we might as well flush right away. */ if (v1l->niov + 3 >= v1l->siov) { - (void)V1L_Flush(wrk); + (void)V1L_Flush(v1l); VSC_C_main->http1_iovs_flush++; } v1l->siov--; @@ -320,27 +315,39 @@ V1L_Chunked(const struct worker *wrk) */ void -V1L_EndChunk(const struct worker *wrk) +V1L_EndChunk(struct v1l *v1l) { - struct v1l *v1l; - CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC); - v1l = wrk->v1l; CHECK_OBJ_NOTNULL(v1l, V1L_MAGIC); assert(v1l->ciov < v1l->siov); - (void)V1L_Flush(wrk); + (void)V1L_Flush(v1l); v1l->siov++; v1l->ciov = v1l->siov; v1l->niov = 0; v1l->cliov = 0; - (void)V1L_Write(wrk, "0\r\n\r\n", -1); + (void)V1L_Write(v1l, "0\r\n\r\n", -1); } /*-------------------------------------------------------------------- * VDP using V1L */ +/* remember priv pointer for V1L_Close() to clear */ +static int v_matchproto_(vdp_init_f) +v1l_init(VRT_CTX, struct vdp_ctx *vdc, void **priv) +{ + struct v1l *v1l; + + (void) ctx; + (void) vdc; + AN(priv); + CAST_OBJ_NOTNULL(v1l, *priv, V1L_MAGIC); + + v1l->vdp_priv = priv; + return (0); +} + static int v_matchproto_(vdp_bytes_f) v1l_bytes(struct vdp_ctx *vdc, enum vdp_action act, void **priv, const void *ptr, ssize_t len) @@ -348,13 +355,13 @@ v1l_bytes(struct vdp_ctx *vdc, enum vdp_action act, void **priv, ssize_t wl = 0; CHECK_OBJ_NOTNULL(vdc, VDP_CTX_MAGIC); - (void)priv; + AN(priv); AZ(vdc->nxt); /* always at the bottom of the pile */ if (len > 0) - wl = V1L_Write(vdc->wrk, ptr, len); - if (act > VDP_NULL && V1L_Flush(vdc->wrk) != SC_NULL) + wl = V1L_Write(*priv, ptr, len); + if (act > VDP_NULL && V1L_Flush(*priv) != SC_NULL) return (-1); if (len != wl) return (-1); @@ -363,5 +370,6 @@ v1l_bytes(struct vdp_ctx *vdc, enum vdp_action act, void **priv, const struct vdp * const VDP_v1l = &(struct vdp){ .name = "V1B", + .init = v1l_init, .bytes = v1l_bytes, }; diff --git a/bin/varnishd/http1/cache_http1_proto.c b/bin/varnishd/http1/cache_http1_proto.c index f56a29b52..acb8e11dd 100644 --- a/bin/varnishd/http1/cache_http1_proto.c +++ b/bin/varnishd/http1/cache_http1_proto.c @@ -465,23 +465,21 @@ HTTP1_DissectResponse(struct http_conn *htc, struct http *hp, /*--------------------------------------------------------------------*/ static unsigned -http1_WrTxt(const struct worker *wrk, const txt *hh, const char *suf) +http1_WrTxt(struct v1l *v1l, const txt *hh, const char *suf) { unsigned u; - CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC); - AN(wrk); AN(hh); AN(hh->b); AN(hh->e); - u = V1L_Write(wrk, hh->b, hh->e - hh->b); + u = V1L_Write(v1l, hh->b, hh->e - hh->b); if (suf != NULL) - u += V1L_Write(wrk, suf, -1); + u += V1L_Write(v1l, suf, -1); return (u); } unsigned -HTTP1_Write(const struct worker *w, const struct http *hp, const int *hf) +HTTP1_Write(struct v1l *v1l, const struct http *hp, const int *hf) { unsigned u, l; @@ -489,12 +487,12 @@ HTTP1_Write(const struct worker *w, const struct http *hp, const int *hf) AN(hp->hd[hf[0]].b); AN(hp->hd[hf[1]].b); AN(hp->hd[hf[2]].b); - l = http1_WrTxt(w, &hp->hd[hf[0]], " "); - l += http1_WrTxt(w, &hp->hd[hf[1]], " "); - l += http1_WrTxt(w, &hp->hd[hf[2]], "\r\n"); + l = http1_WrTxt(v1l, &hp->hd[hf[0]], " "); + l += http1_WrTxt(v1l, &hp->hd[hf[1]], " "); + l += http1_WrTxt(v1l, &hp->hd[hf[2]], "\r\n"); for (u = HTTP_HDR_FIRST; u < hp->nhd; u++) - l += http1_WrTxt(w, &hp->hd[u], "\r\n"); - l += V1L_Write(w, "\r\n", -1); + l += http1_WrTxt(v1l, &hp->hd[u], "\r\n"); + l += V1L_Write(v1l, "\r\n", -1); return (l); } diff --git a/bin/varnishd/http2/cache_http2_deliver.c b/bin/varnishd/http2/cache_http2_deliver.c index 5de2a7fc4..4c2f80480 100644 --- a/bin/varnishd/http2/cache_http2_deliver.c +++ b/bin/varnishd/http2/cache_http2_deliver.c @@ -331,8 +331,6 @@ h2_deliver(struct req *req, struct boc *boc, int sendbody) sendbody = 0; } - AZ(req->wrk->v1l); - r2->t_send = req->t_prev; H2_Send_Get(req->wrk, r2->h2sess, r2); @@ -351,6 +349,5 @@ h2_deliver(struct req *req, struct boc *boc, int sendbody) (void)VDP_DeliverObj(req->vdc, req->objcore); } - AZ(req->wrk->v1l); req->acct.resp_bodybytes += VDP_Close(req->vdc, req->objcore, boc); } From nils.goroll at uplex.de Mon Nov 25 14:43:03 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 25 Nov 2024 14:43:03 +0000 (UTC) Subject: [master] a6bfa312e cache_http1_line: Flexelint: Avoid unchecked downcast Message-ID: <20241125144303.88E25112979@lists.varnish-cache.org> commit a6bfa312e288a9764a020e67b8c54d50037eb2d7 Author: Nils Goroll Date: Fri Nov 1 09:04:57 2024 +0100 cache_http1_line: Flexelint: Avoid unchecked downcast Found with less Flexelint silencing diff --git a/bin/varnishd/http1/cache_http1_line.c b/bin/varnishd/http1/cache_http1_line.c index 771490986..39ba2356f 100644 --- a/bin/varnishd/http1/cache_http1_line.c +++ b/bin/varnishd/http1/cache_http1_line.c @@ -83,6 +83,7 @@ V1L_Open(struct ws *ws, int *fd, struct vsl_log *vsl, struct v1l *v1l; unsigned u; uintptr_t ws_snap; + size_t sz; if (WS_Overflowed(ws)) return (NULL); @@ -119,7 +120,9 @@ V1L_Open(struct ws *ws, int *fd, struct vsl_log *vsl, v1l->vsl = vsl; v1l->werr = SC_NULL; - WS_Release(ws, u * sizeof(struct iovec)); + sz = u * sizeof(struct iovec); + assert(sz < UINT_MAX); + WS_Release(ws, (unsigned)sz); return (v1l); } From nils.goroll at uplex.de Mon Nov 25 14:43:03 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 25 Nov 2024 14:43:03 +0000 (UTC) Subject: [master] 5bc20e62f cache_http1_line: Flexelint: signedness polish and other minor improvements Message-ID: <20241125144303.A422F11297E@lists.varnish-cache.org> commit 5bc20e62f9dad5bf46b36eac83fbbad2243c396c Author: Nils Goroll Date: Fri Nov 1 09:06:15 2024 +0100 cache_http1_line: Flexelint: signedness polish and other minor improvements Found with less Flexelint silencing Change of data type of struct v1l members was guided by their main use: {s,n,c}iov are all related to the iovcnt argument of writev(), which is of type int. {l,cl}iov are lengths (to be) written, which can not be negative. cnt is ultimately going to be returned as a uint64_t, so it makes sense to have that in the first place. The other changes are consequences of these, fixing all sign and type related Flexelint complaints. diff --git a/bin/varnishd/http1/cache_http1_line.c b/bin/varnishd/http1/cache_http1_line.c index 39ba2356f..d19fb4f08 100644 --- a/bin/varnishd/http1/cache_http1_line.c +++ b/bin/varnishd/http1/cache_http1_line.c @@ -58,14 +58,14 @@ struct v1l { int *wfd; stream_close_t werr; /* valid after V1L_Flush() */ struct iovec *iov; - unsigned siov; - unsigned niov; - ssize_t liov; - ssize_t cliov; - unsigned ciov; /* Chunked header marker */ + int siov; + int niov; + size_t liov; + size_t cliov; + int ciov; /* Chunked header marker */ vtim_real deadline; struct vsl_log *vsl; - ssize_t cnt; /* Flushed byte count */ + uint64_t cnt; /* Flushed byte count */ struct ws *ws; uintptr_t ws_snap; void **vdp_priv; @@ -113,8 +113,8 @@ V1L_Open(struct ws *ws, int *fd, struct vsl_log *vsl, if (niov != 0 && u > niov) u = niov; v1l->iov = WS_Reservation(ws); - v1l->siov = u; - v1l->ciov = u; + v1l->siov = (int)u; + v1l->ciov = (int)u; v1l->wfd = fd; v1l->deadline = deadline; v1l->vsl = vsl; @@ -150,10 +150,14 @@ V1L_Close(struct v1l **v1lp, uint64_t *cnt) } static void -v1l_prune(struct v1l *v1l, size_t bytes) +v1l_prune(struct v1l *v1l, ssize_t abytes) { - ssize_t used = 0; - ssize_t j, used_here; + size_t used = 0; + size_t sz, bytes, used_here; + int j; + + assert(abytes > 0); + bytes = (size_t)abytes; for (j = 0; j < v1l->niov; j++) { if (used + v1l->iov[j].iov_len > bytes) { @@ -162,9 +166,11 @@ v1l_prune(struct v1l *v1l, size_t bytes) v1l->iov[j].iov_len -= used_here; v1l->iov[j].iov_base = (char*)v1l->iov[j].iov_base + used_here; - memmove(v1l->iov, &v1l->iov[j], - (v1l->niov - j) * sizeof(struct iovec)); + sz = (unsigned)v1l->niov - (unsigned)j; + sz *= sizeof(struct iovec); + memmove(v1l->iov, &v1l->iov[j], sz); v1l->niov -= j; + assert(v1l->liov >= bytes); v1l->liov -= bytes; return; } @@ -177,6 +183,7 @@ stream_close_t V1L_Flush(struct v1l *v1l) { ssize_t i; + size_t sz; int err; char cbuf[32]; @@ -190,13 +197,13 @@ V1L_Flush(struct v1l *v1l) if (v1l->ciov < v1l->siov && v1l->cliov > 0) { /* Add chunk head & tail */ bprintf(cbuf, "00%zx\r\n", v1l->cliov); - i = strlen(cbuf); + sz = strlen(cbuf); v1l->iov[v1l->ciov].iov_base = cbuf; - v1l->iov[v1l->ciov].iov_len = i; - v1l->liov += i; + v1l->iov[v1l->ciov].iov_len = sz; + v1l->liov += sz; /* This is OK, because siov was --'ed */ - v1l->iov[v1l->niov].iov_base = cbuf + i - 2; + v1l->iov[v1l->niov].iov_base = cbuf + sz - 2; v1l->iov[v1l->niov++].iov_len = 2; v1l->liov += 2; } else if (v1l->ciov < v1l->siov) { @@ -217,11 +224,11 @@ V1L_Flush(struct v1l *v1l) } i = writev(*v1l->wfd, v1l->iov, v1l->niov); - if (i > 0) - v1l->cnt += i; - - if (i == v1l->liov) - break; + if (i > 0) { + v1l->cnt += (size_t)i; + if ((size_t)i == v1l->liov) + break; + } /* we hit a timeout, and some data may have been sent: * Remove sent data from start of I/O vector, then retry @@ -265,15 +272,21 @@ V1L_Flush(struct v1l *v1l) } size_t -V1L_Write(struct v1l *v1l, const void *ptr, ssize_t len) +V1L_Write(struct v1l *v1l, const void *ptr, ssize_t alen) { + size_t len = 0; CHECK_OBJ_NOTNULL(v1l, V1L_MAGIC); AN(v1l->wfd); - if (len == 0 || *v1l->wfd < 0) + if (alen == 0 || *v1l->wfd < 0) return (0); - if (len == -1) + if (alen > 0) + len = (size_t)alen; + else if (alen == -1) len = strlen(ptr); + else + WRONG("alen"); + assert(v1l->niov < v1l->siov); v1l->iov[v1l->niov].iov_base = TRUST_ME(ptr); v1l->iov[v1l->niov].iov_len = len; @@ -355,7 +368,7 @@ static int v_matchproto_(vdp_bytes_f) v1l_bytes(struct vdp_ctx *vdc, enum vdp_action act, void **priv, const void *ptr, ssize_t len) { - ssize_t wl = 0; + size_t wl = 0; CHECK_OBJ_NOTNULL(vdc, VDP_CTX_MAGIC); AN(priv); @@ -366,7 +379,7 @@ v1l_bytes(struct vdp_ctx *vdc, enum vdp_action act, void **priv, wl = V1L_Write(*priv, ptr, len); if (act > VDP_NULL && V1L_Flush(*priv) != SC_NULL) return (-1); - if (len != wl) + if ((size_t)len != wl) return (-1); return (0); } From nils.goroll at uplex.de Mon Nov 25 15:01:05 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 25 Nov 2024 15:01:05 +0000 (UTC) Subject: [master] 84813b20d Do not call VSHA256Update() for zero length input Message-ID: <20241125150105.38927113898@lists.varnish-cache.org> commit 84813b20db2b83aad2c89567eb30460ec620b496 Author: Nils Goroll Date: Mon Nov 25 15:59:35 2024 +0100 Do not call VSHA256Update() for zero length input Should address an ASAN complaint via CCI: vsha256.c:298:3: runtime error: null pointer passed as argument 2, which is declared to never be null https://app.circleci.com/pipelines/github/varnishcache/varnish-cache/6494/workflows/6c831825-db4d-4dbb-a071-efdee38996ff/jobs/83887?invite=true#step-102-217155_51 diff --git a/vmod/vmod_debug_filters.c b/vmod/vmod_debug_filters.c index 68fbb5566..53777996d 100644 --- a/vmod/vmod_debug_filters.c +++ b/vmod/vmod_debug_filters.c @@ -463,7 +463,8 @@ xyzzy_chksha256_bytes(struct vdp_ctx *vdc, enum vdp_action act, void **priv, struct vdp_chksha256_s *vdps; CAST_OBJ_NOTNULL(vdps, *priv, VDP_CHKSHA256_MAGIC); - VSHA256_Update(vdps->cx, ptr, len); + if (len != 0) + VSHA256_Update(vdps->cx, ptr, len); vdps->called++; vdps->bytes += len; return (VDP_bytes(vdc, act, ptr, len)); From nils.goroll at uplex.de Mon Nov 25 17:30:04 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 25 Nov 2024 17:30:04 +0000 (UTC) Subject: [master] 25e7b9a17 cache_req_fsm: Refactor: move busy objcore reference to struct req Message-ID: <20241125173004.5178D11891D@lists.varnish-cache.org> commit 25e7b9a170160603dac61c95f5ab4969e47d51c5 Author: Nils Goroll Date: Fri Nov 1 15:46:32 2024 +0100 cache_req_fsm: Refactor: move busy objcore reference to struct req This is in preparation of simplifications and to be able disembark during delivery. diff --git a/bin/varnishd/cache/cache.h b/bin/varnishd/cache/cache.h index d9cd0094f..9346f0980 100644 --- a/bin/varnishd/cache/cache.h +++ b/bin/varnishd/cache/cache.h @@ -537,6 +537,7 @@ struct req { struct ws ws[1]; struct objcore *objcore; struct objcore *stale_oc; + struct boc *boc; /* valid during cnt_transmit */ /* resp.body */ struct vdp_ctx *vdc; diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c index 9dfebfd99..56b15fcd4 100644 --- a/bin/varnishd/cache/cache_req_fsm.c +++ b/bin/varnishd/cache/cache_req_fsm.c @@ -429,7 +429,6 @@ cnt_synth(struct worker *wrk, struct req *req) static enum req_fsm_nxt v_matchproto_(req_state_f) cnt_transmit(struct worker *wrk, struct req *req) { - struct boc *boc; uint16_t status; int sendbody, head; intmax_t clval; @@ -440,17 +439,18 @@ cnt_transmit(struct worker *wrk, struct req *req) CHECK_OBJ_NOTNULL(req->objcore, OBJCORE_MAGIC); AZ(req->stale_oc); AZ(req->res_mode); + AZ(req->boc); /* Grab a ref to the bo if there is one (=streaming) */ - boc = HSH_RefBoc(req->objcore); - if (boc && boc->state < BOS_STREAM) + req->boc = HSH_RefBoc(req->objcore); + if (req->boc && req->boc->state < BOS_STREAM) ObjWaitState(req->objcore, BOS_STREAM); clval = http_GetContentLength(req->resp); /* RFC 7230, 3.3.3 */ status = http_GetStatus(req->resp); head = http_method_eq(req->http0->hd[HTTP_HDR_METHOD].b, HEAD); - if (boc != NULL || (req->objcore->flags & (OC_F_FAILED))) + if (req->boc != NULL || (req->objcore->flags & (OC_F_FAILED))) req->resp_len = clval; else req->resp_len = ObjGetLen(req->wrk, req->objcore); @@ -470,7 +470,7 @@ cnt_transmit(struct worker *wrk, struct req *req) VSLb(req->vsl, SLT_Error, "Failure to push processors"); req->doclose = SC_OVERLOAD; req->acct.resp_bodybytes += - VDP_Close(req->vdc, req->objcore, boc); + VDP_Close(req->vdc, req->objcore, req->boc); } else { if (status < 200 || status == 204) { // rfc7230,l,1691,1695 @@ -497,7 +497,7 @@ cnt_transmit(struct worker *wrk, struct req *req) } if (req->resp_len == 0) sendbody = 0; - req->transport->deliver(req, boc, sendbody); + req->transport->deliver(req, req->boc, sendbody); } VSLb_ts_req(req, "Resp", W_TIM_real(wrk)); @@ -508,8 +508,10 @@ cnt_transmit(struct worker *wrk, struct req *req) req->doclose = SC_TX_ERROR; } - if (boc != NULL) + if (req->boc != NULL) { HSH_DerefBoc(wrk, req->objcore); + req->boc = NULL; + } (void)HSH_DerefObjCore(wrk, &req->objcore, HSH_RUSH_POLICY); http_Teardown(req->resp); From nils.goroll at uplex.de Mon Nov 25 17:30:04 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 25 Nov 2024 17:30:04 +0000 (UTC) Subject: [master] 402a969d4 cache_transport: Remove boc argument from vtr_deliver_f Message-ID: <20241125173004.6DBCC118920@lists.varnish-cache.org> commit 402a969d45773c1fff3e4705e362686b6830a82d Author: Nils Goroll Date: Fri Nov 1 15:55:55 2024 +0100 cache_transport: Remove boc argument from vtr_deliver_f now that boc lives in struct req, it is obsolete also change some subordinate helper functions diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c index c4063f355..158251337 100644 --- a/bin/varnishd/cache/cache_esi_deliver.c +++ b/bin/varnishd/cache/cache_esi_deliver.c @@ -855,9 +855,9 @@ static const struct vdp ved_ved = { }; static void -ved_close(struct req *req, struct boc *boc, int error) +ved_close(struct req *req, int error) { - req->acct.resp_bodybytes += VDP_Close(req->vdc, req->objcore, boc); + req->acct.resp_bodybytes += VDP_Close(req->vdc, req->objcore, req->boc); if (! error) return; @@ -868,7 +868,7 @@ ved_close(struct req *req, struct boc *boc, int error) /*--------------------------------------------------------------------*/ static void v_matchproto_(vtr_deliver_f) -ved_deliver(struct req *req, struct boc *boc, int wantbody) +ved_deliver(struct req *req, int wantbody) { int i = 0; const char *p; @@ -878,7 +878,7 @@ ved_deliver(struct req *req, struct boc *boc, int wantbody) struct vrt_ctx ctx[1]; CHECK_OBJ_NOTNULL(req, REQ_MAGIC); - CHECK_OBJ_ORNULL(boc, BOC_MAGIC); + CHECK_OBJ_ORNULL(req->boc, BOC_MAGIC); CHECK_OBJ_NOTNULL(req->objcore, OBJCORE_MAGIC); CAST_OBJ_NOTNULL(ecx, req->transport_priv, ECX_MAGIC); @@ -887,17 +887,17 @@ ved_deliver(struct req *req, struct boc *boc, int wantbody) if (FEATURE(FEATURE_ESI_INCLUDE_ONERROR) && status != 200 && status != 204) { - ved_close(req, boc, ecx->abrt); + ved_close(req, ecx->abrt); return; } if (wantbody == 0) { - ved_close(req, boc, 0); + ved_close(req, 0); return; } - if (boc == NULL && ObjGetLen(req->wrk, req->objcore) == 0) { - ved_close(req, boc, 0); + if (req->boc == NULL && ObjGetLen(req->wrk, req->objcore) == 0) { + ved_close(req, 0); return; } @@ -913,7 +913,7 @@ ved_deliver(struct req *req, struct boc *boc, int wantbody) /* A gzipped include which is not ESI processed */ /* OA_GZIPBITS are not valid until BOS_FINISHED */ - if (boc != NULL) + if (req->boc != NULL) ObjWaitState(req->objcore, BOS_FINISHED); if (req->objcore->flags & OC_F_FAILED) { @@ -921,7 +921,7 @@ ved_deliver(struct req *req, struct boc *boc, int wantbody) * the ESI body. Omit this ESI fragment. * XXX change error argument to 1 */ - ved_close(req, boc, 0); + ved_close(req, 0); return; } @@ -947,5 +947,5 @@ ved_deliver(struct req *req, struct boc *boc, int wantbody) if (i && req->doclose == SC_NULL) req->doclose = SC_REM_CLOSE; - ved_close(req, boc, i && ecx->abrt ? 1 : 0); + ved_close(req, i && ecx->abrt ? 1 : 0); } diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c index 56b15fcd4..a167b2839 100644 --- a/bin/varnishd/cache/cache_req_fsm.c +++ b/bin/varnishd/cache/cache_req_fsm.c @@ -497,7 +497,7 @@ cnt_transmit(struct worker *wrk, struct req *req) } if (req->resp_len == 0) sendbody = 0; - req->transport->deliver(req, req->boc, sendbody); + req->transport->deliver(req, sendbody); } VSLb_ts_req(req, "Resp", W_TIM_real(wrk)); diff --git a/bin/varnishd/cache/cache_transport.h b/bin/varnishd/cache/cache_transport.h index 701f0dbfd..d5cd3d428 100644 --- a/bin/varnishd/cache/cache_transport.h +++ b/bin/varnishd/cache/cache_transport.h @@ -38,7 +38,7 @@ struct req; struct boc; -typedef void vtr_deliver_f (struct req *, struct boc *, int sendbody); +typedef void vtr_deliver_f (struct req *, int sendbody); typedef void vtr_req_body_f (struct req *); typedef void vtr_sess_panic_f (struct vsb *, const struct sess *); typedef void vtr_req_panic_f (struct vsb *, const struct req *); diff --git a/bin/varnishd/http1/cache_http1.h b/bin/varnishd/http1/cache_http1.h index 98ed1e1df..7aadb5cfa 100644 --- a/bin/varnishd/http1/cache_http1.h +++ b/bin/varnishd/http1/cache_http1.h @@ -42,7 +42,7 @@ extern const int HTTP1_Req[3]; extern const int HTTP1_Resp[3]; /* cache_http1_deliver.c */ -void V1D_Deliver(struct req *, struct boc *, int sendbody); +void V1D_Deliver(struct req *, int sendbody); /* cache_http1_pipe.c */ struct v1p_acct { diff --git a/bin/varnishd/http1/cache_http1_deliver.c b/bin/varnishd/http1/cache_http1_deliver.c index 462a9357f..ab3826a92 100644 --- a/bin/varnishd/http1/cache_http1_deliver.c +++ b/bin/varnishd/http1/cache_http1_deliver.c @@ -40,7 +40,7 @@ /*--------------------------------------------------------------------*/ static void -v1d_error(struct req *req, struct boc *boc, struct v1l **v1lp, const char *msg) +v1d_error(struct req *req, struct v1l **v1lp, const char *msg) { static const char r_500[] = "HTTP/1.1 500 Internal Server Error\r\n" @@ -61,14 +61,14 @@ v1d_error(struct req *req, struct boc *boc, struct v1l **v1lp, const char *msg) VTCP_Assert(write(req->sp->fd, r_500, sizeof r_500 - 1)); req->doclose = SC_TX_EOF; - req->acct.resp_bodybytes += VDP_Close(req->vdc, req->objcore, boc); + req->acct.resp_bodybytes += VDP_Close(req->vdc, req->objcore, req->boc); } /*-------------------------------------------------------------------- */ void v_matchproto_(vtr_deliver_f) -V1D_Deliver(struct req *req, struct boc *boc, int sendbody) +V1D_Deliver(struct req *req, int sendbody) { struct vrt_ctx ctx[1]; int err = 0, chunked = 0; @@ -77,7 +77,7 @@ V1D_Deliver(struct req *req, struct boc *boc, int sendbody) struct v1l *v1l; CHECK_OBJ_NOTNULL(req, REQ_MAGIC); - CHECK_OBJ_ORNULL(boc, BOC_MAGIC); + CHECK_OBJ_ORNULL(req->boc, BOC_MAGIC); CHECK_OBJ_NOTNULL(req->objcore, OBJCORE_MAGIC); if (req->doclose == SC_NULL && @@ -98,7 +98,7 @@ V1D_Deliver(struct req *req, struct boc *boc, int sendbody) cache_param->http1_iovs); if (v1l == NULL) { - v1d_error(req, boc, &v1l, "Failure to init v1d (workspace_thread overflow)"); + v1d_error(req, &v1l, "Failure to init v1d (workspace_thread overflow)"); return; } @@ -115,23 +115,23 @@ V1D_Deliver(struct req *req, struct boc *boc, int sendbody) INIT_OBJ(ctx, VRT_CTX_MAGIC); VCL_Req2Ctx(ctx, req); if (VDP_Push(ctx, req->vdc, req->ws, VDP_v1l, v1l)) { - v1d_error(req, boc, &v1l, "Failure to push v1d processor"); + v1d_error(req, &v1l, "Failure to push v1d processor"); return; } } if (WS_Overflowed(req->ws)) { - v1d_error(req, boc, &v1l, "workspace_client overflow"); + v1d_error(req, &v1l, "workspace_client overflow"); return; } if (WS_Overflowed(req->sp->ws)) { - v1d_error(req, boc, &v1l, "workspace_session overflow"); + v1d_error(req, &v1l, "workspace_session overflow"); return; } if (WS_Overflowed(req->wrk->aws)) { - v1d_error(req, boc, &v1l, "workspace_thread overflow"); + v1d_error(req, &v1l, "workspace_thread overflow"); return; } @@ -150,7 +150,7 @@ V1D_Deliver(struct req *req, struct boc *boc, int sendbody) sc = V1L_Close(&v1l, &bytes); req->acct.resp_hdrbytes += hdrbytes; - req->acct.resp_bodybytes += VDP_Close(req->vdc, req->objcore, boc); + req->acct.resp_bodybytes += VDP_Close(req->vdc, req->objcore, req->boc); if (sc == SC_NULL && err && req->sp->fd >= 0) sc = SC_REM_CLOSE; diff --git a/bin/varnishd/http2/cache_http2_deliver.c b/bin/varnishd/http2/cache_http2_deliver.c index 4c2f80480..b80b91924 100644 --- a/bin/varnishd/http2/cache_http2_deliver.c +++ b/bin/varnishd/http2/cache_http2_deliver.c @@ -293,7 +293,7 @@ h2_build_headers(struct vsb *resp, struct req *req) } void v_matchproto_(vtr_deliver_f) -h2_deliver(struct req *req, struct boc *boc, int sendbody) +h2_deliver(struct req *req, int sendbody) { size_t sz; const char *r; @@ -304,7 +304,6 @@ h2_deliver(struct req *req, struct boc *boc, int sendbody) uintptr_t ss; CHECK_OBJ_NOTNULL(req, REQ_MAGIC); - CHECK_OBJ_ORNULL(boc, BOC_MAGIC); CHECK_OBJ_NOTNULL(req->objcore, OBJCORE_MAGIC); CAST_OBJ_NOTNULL(r2, req->transport_priv, H2_REQ_MAGIC); sp = req->sp; @@ -349,5 +348,5 @@ h2_deliver(struct req *req, struct boc *boc, int sendbody) (void)VDP_DeliverObj(req->vdc, req->objcore); } - req->acct.resp_bodybytes += VDP_Close(req->vdc, req->objcore, boc); + req->acct.resp_bodybytes += VDP_Close(req->vdc, req->objcore, req->boc); } From nils.goroll at uplex.de Mon Nov 25 17:30:04 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 25 Nov 2024 17:30:04 +0000 (UTC) Subject: [master] 27203a6d1 cache_req_fsm: Split the tail of cnt_transmit() into a new finish step Message-ID: <20241125173004.9337A118924@lists.varnish-cache.org> commit 27203a6d1d1b75c3fb93cbeb0dee7db473541312 Author: Nils Goroll Date: Fri Nov 1 15:59:45 2024 +0100 cache_req_fsm: Split the tail of cnt_transmit() into a new finish step To prepare for the next commit, we introduce a new finish step to the request FSM, which is just the former tail of cnt_transmit(). diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c index a167b2839..bc4568911 100644 --- a/bin/varnishd/cache/cache_req_fsm.c +++ b/bin/varnishd/cache/cache_req_fsm.c @@ -66,7 +66,8 @@ REQ_STEP(deliver, DELIVER, static) \ REQ_STEP(vclfail, VCLFAIL, static) \ REQ_STEP(synth, SYNTH, static) \ - REQ_STEP(transmit, TRANSMIT, static) + REQ_STEP(transmit, TRANSMIT, static) \ + REQ_STEP(finish, FINISH, static) #define REQ_STEP(l, U, priv) \ static req_state_f cnt_##l; \ @@ -499,6 +500,16 @@ cnt_transmit(struct worker *wrk, struct req *req) sendbody = 0; req->transport->deliver(req, sendbody); } + req->req_step = R_STP_FINISH; + return (REQ_FSM_MORE); +} + +static enum req_fsm_nxt v_matchproto_(req_state_f) +cnt_finish(struct worker *wrk, struct req *req) +{ + + CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC); + CHECK_OBJ_NOTNULL(req, REQ_MAGIC); VSLb_ts_req(req, "Resp", W_TIM_real(wrk)); From nils.goroll at uplex.de Mon Nov 25 17:30:04 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 25 Nov 2024 17:30:04 +0000 (UTC) Subject: [master] 2bd280a34 cache_transport: change vtr_deliver_f to return vtr_deliver_e Message-ID: <20241125173004.BF1CA11892E@lists.varnish-cache.org> commit 2bd280a341213b7a8bf66c5a1c0ec52d9abf90d5 Author: Nils Goroll Date: Fri Nov 1 16:13:55 2024 +0100 cache_transport: change vtr_deliver_f to return vtr_deliver_e This allows the transport's delivery function to disembark the thread by returning VTR_D_DISEMBARK. In this case, CNT_Request() needs to be called at a later time to continue. It is the vtr_deliver_f's responsibility to clear req->wrk because only it can ensure proper synchronization with another thread which might already be running. diff --git a/bin/varnishd/cache/cache_esi_deliver.c b/bin/varnishd/cache/cache_esi_deliver.c index 158251337..6a336b549 100644 --- a/bin/varnishd/cache/cache_esi_deliver.c +++ b/bin/varnishd/cache/cache_esi_deliver.c @@ -867,7 +867,7 @@ ved_close(struct req *req, int error) /*--------------------------------------------------------------------*/ -static void v_matchproto_(vtr_deliver_f) +static enum vtr_deliver_e v_matchproto_(vtr_deliver_f) ved_deliver(struct req *req, int wantbody) { int i = 0; @@ -888,17 +888,17 @@ ved_deliver(struct req *req, int wantbody) if (FEATURE(FEATURE_ESI_INCLUDE_ONERROR) && status != 200 && status != 204) { ved_close(req, ecx->abrt); - return; + return (VTR_D_DONE); } if (wantbody == 0) { ved_close(req, 0); - return; + return (VTR_D_DONE); } if (req->boc == NULL && ObjGetLen(req->wrk, req->objcore) == 0) { ved_close(req, 0); - return; + return (VTR_D_DONE); } if (http_GetHdr(req->resp, H_Content_Encoding, &p)) @@ -922,7 +922,7 @@ ved_deliver(struct req *req, int wantbody) * XXX change error argument to 1 */ ved_close(req, 0); - return; + return (VTR_D_DONE); } INIT_OBJ(foo, VED_FOO_MAGIC); @@ -948,4 +948,5 @@ ved_deliver(struct req *req, int wantbody) req->doclose = SC_REM_CLOSE; ved_close(req, i && ecx->abrt ? 1 : 0); + return (VTR_D_DONE); } diff --git a/bin/varnishd/cache/cache_req_fsm.c b/bin/varnishd/cache/cache_req_fsm.c index bc4568911..bbcb3824f 100644 --- a/bin/varnishd/cache/cache_req_fsm.c +++ b/bin/varnishd/cache/cache_req_fsm.c @@ -430,6 +430,8 @@ cnt_synth(struct worker *wrk, struct req *req) static enum req_fsm_nxt v_matchproto_(req_state_f) cnt_transmit(struct worker *wrk, struct req *req) { + enum req_fsm_nxt nxt = REQ_FSM_MORE; + enum vtr_deliver_e dnxt; uint16_t status; int sendbody, head; intmax_t clval; @@ -441,6 +443,7 @@ cnt_transmit(struct worker *wrk, struct req *req) AZ(req->stale_oc); AZ(req->res_mode); AZ(req->boc); + req->req_step = R_STP_FINISH; /* Grab a ref to the bo if there is one (=streaming) */ req->boc = HSH_RefBoc(req->objcore); @@ -498,10 +501,13 @@ cnt_transmit(struct worker *wrk, struct req *req) } if (req->resp_len == 0) sendbody = 0; - req->transport->deliver(req, sendbody); + dnxt = req->transport->deliver(req, sendbody); + if (dnxt == VTR_D_DISEMBARK) + nxt = REQ_FSM_DISEMBARK; + else + assert(dnxt == VTR_D_DONE); } - req->req_step = R_STP_FINISH; - return (REQ_FSM_MORE); + return (nxt); } static enum req_fsm_nxt v_matchproto_(req_state_f) @@ -1180,6 +1186,7 @@ CNT_Request(struct req *req) */ assert( req->req_step == R_STP_LOOKUP || + req->req_step == R_STP_FINISH || req->req_step == R_STP_TRANSPORT); AN(VXID_TAG(req->vsl->wid) & VSL_CLIENTMARKER); diff --git a/bin/varnishd/cache/cache_transport.h b/bin/varnishd/cache/cache_transport.h index d5cd3d428..6fd297739 100644 --- a/bin/varnishd/cache/cache_transport.h +++ b/bin/varnishd/cache/cache_transport.h @@ -38,7 +38,13 @@ struct req; struct boc; -typedef void vtr_deliver_f (struct req *, int sendbody); +enum vtr_deliver_e { + VTR_D_INVAL = 0, + VTR_D_DONE = 1, + VTR_D_DISEMBARK +}; + +typedef enum vtr_deliver_e vtr_deliver_f (struct req *, int sendbody); typedef void vtr_req_body_f (struct req *); typedef void vtr_sess_panic_f (struct vsb *, const struct sess *); typedef void vtr_req_panic_f (struct vsb *, const struct req *); diff --git a/bin/varnishd/http1/cache_http1.h b/bin/varnishd/http1/cache_http1.h index 7aadb5cfa..85203295f 100644 --- a/bin/varnishd/http1/cache_http1.h +++ b/bin/varnishd/http1/cache_http1.h @@ -42,7 +42,7 @@ extern const int HTTP1_Req[3]; extern const int HTTP1_Resp[3]; /* cache_http1_deliver.c */ -void V1D_Deliver(struct req *, int sendbody); +enum vtr_deliver_e V1D_Deliver(struct req *, int sendbody); /* cache_http1_pipe.c */ struct v1p_acct { diff --git a/bin/varnishd/http1/cache_http1_deliver.c b/bin/varnishd/http1/cache_http1_deliver.c index ab3826a92..9015f3cc0 100644 --- a/bin/varnishd/http1/cache_http1_deliver.c +++ b/bin/varnishd/http1/cache_http1_deliver.c @@ -34,6 +34,7 @@ #include "cache/cache_varnishd.h" #include "cache/cache_filter.h" #include "cache_http1.h" +#include "cache/cache_transport.h" #include "vtcp.h" @@ -67,7 +68,7 @@ v1d_error(struct req *req, struct v1l **v1lp, const char *msg) /*-------------------------------------------------------------------- */ -void v_matchproto_(vtr_deliver_f) +enum vtr_deliver_e v_matchproto_(vtr_deliver_f) V1D_Deliver(struct req *req, int sendbody) { struct vrt_ctx ctx[1]; @@ -99,7 +100,7 @@ V1D_Deliver(struct req *req, int sendbody) if (v1l == NULL) { v1d_error(req, &v1l, "Failure to init v1d (workspace_thread overflow)"); - return; + return (VTR_D_DONE); } if (sendbody) { @@ -116,23 +117,23 @@ V1D_Deliver(struct req *req, int sendbody) VCL_Req2Ctx(ctx, req); if (VDP_Push(ctx, req->vdc, req->ws, VDP_v1l, v1l)) { v1d_error(req, &v1l, "Failure to push v1d processor"); - return; + return (VTR_D_DONE); } } if (WS_Overflowed(req->ws)) { v1d_error(req, &v1l, "workspace_client overflow"); - return; + return (VTR_D_DONE); } if (WS_Overflowed(req->sp->ws)) { v1d_error(req, &v1l, "workspace_session overflow"); - return; + return (VTR_D_DONE); } if (WS_Overflowed(req->wrk->aws)) { v1d_error(req, &v1l, "workspace_thread overflow"); - return; + return (VTR_D_DONE); } hdrbytes = HTTP1_Write(v1l, req->resp, HTTP1_Resp); @@ -156,4 +157,5 @@ V1D_Deliver(struct req *req, int sendbody) sc = SC_REM_CLOSE; if (sc != SC_NULL) Req_Fail(req, sc); + return (VTR_D_DONE); } diff --git a/bin/varnishd/http2/cache_http2_deliver.c b/bin/varnishd/http2/cache_http2_deliver.c index b80b91924..401301243 100644 --- a/bin/varnishd/http2/cache_http2_deliver.c +++ b/bin/varnishd/http2/cache_http2_deliver.c @@ -292,7 +292,7 @@ h2_build_headers(struct vsb *resp, struct req *req) } } -void v_matchproto_(vtr_deliver_f) +enum vtr_deliver_e v_matchproto_(vtr_deliver_f) h2_deliver(struct req *req, int sendbody) { size_t sz; @@ -349,4 +349,5 @@ h2_deliver(struct req *req, int sendbody) } req->acct.resp_bodybytes += VDP_Close(req->vdc, req->objcore, req->boc); + return (VTR_D_DONE); } From nils.goroll at uplex.de Mon Nov 25 17:30:04 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 25 Nov 2024 17:30:04 +0000 (UTC) Subject: [master] 74292b35f http1_deliver polish: save read/write once local variable Message-ID: <20241125173004.E0570118937@lists.varnish-cache.org> commit 74292b35f31bb3bf8636d2378304ef8611a28d38 Author: Nils Goroll Date: Mon Nov 25 18:04:22 2024 +0100 http1_deliver polish: save read/write once local variable diff --git a/bin/varnishd/http1/cache_http1_deliver.c b/bin/varnishd/http1/cache_http1_deliver.c index 9015f3cc0..edb32573e 100644 --- a/bin/varnishd/http1/cache_http1_deliver.c +++ b/bin/varnishd/http1/cache_http1_deliver.c @@ -74,7 +74,7 @@ V1D_Deliver(struct req *req, int sendbody) struct vrt_ctx ctx[1]; int err = 0, chunked = 0; stream_close_t sc; - uint64_t hdrbytes, bytes; + uint64_t bytes; struct v1l *v1l; CHECK_OBJ_NOTNULL(req, REQ_MAGIC); @@ -136,7 +136,7 @@ V1D_Deliver(struct req *req, int sendbody) return (VTR_D_DONE); } - hdrbytes = HTTP1_Write(v1l, req->resp, HTTP1_Resp); + req->acct.resp_hdrbytes += HTTP1_Write(v1l, req->resp, HTTP1_Resp); if (sendbody) { if (DO_DEBUG(DBG_FLUSH_HEAD)) @@ -150,7 +150,6 @@ V1D_Deliver(struct req *req, int sendbody) sc = V1L_Close(&v1l, &bytes); - req->acct.resp_hdrbytes += hdrbytes; req->acct.resp_bodybytes += VDP_Close(req->vdc, req->objcore, req->boc); if (sc == SC_NULL && err && req->sp->fd >= 0) From nils.goroll at uplex.de Mon Nov 25 17:30:05 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 25 Nov 2024 17:30:05 +0000 (UTC) Subject: [master] 0cfd6cd06 vmod_debug: demo a reembarking transport Message-ID: <20241125173005.0B067118942@lists.varnish-cache.org> commit 0cfd6cd06e186e0a36094a57294124f2f1cf2cfa Author: Nils Goroll Date: Fri Nov 1 18:00:28 2024 +0100 vmod_debug: demo a reembarking transport vmod_debug_transports.c demonstrates the basic mechanics of disembarking a request from the vrt_deliver_f, delivering a body from another worker and finishing the request. diff --git a/bin/varnishtest/tests/m00060.vtc b/bin/varnishtest/tests/m00060.vtc new file mode 100644 index 000000000..072fadfbe --- /dev/null +++ b/bin/varnishtest/tests/m00060.vtc @@ -0,0 +1,39 @@ +varnishtest "VMOD debug reembarking transport" + +server s1 { + rxreq + txresp -bodylen 131072 +} -start + +varnish v1 \ + -vcl+backend { + import debug; + + sub vcl_hash { + hash_data(""); + return (lookup); + } + + sub vcl_deliver { + if (req.url == "/chunked") { + set resp.filters = "debug.chunked"; + } + debug.use_reembarking_http1(); + } +} -start + +varnish v1 -cliok "param.set debug +syncvsl" +varnish v1 -cliok "param.set debug +req_state" + +client c1 -repeat 16 -keepalive { + txreq + rxresp +} -start + +client c2 -repeat 16 -keepalive { + txreq -url "/chunked" + rxresp +} -start + +client c1 -wait +client c2 -wait diff --git a/vmod/automake_boilerplate_debug.am b/vmod/automake_boilerplate_debug.am index 51e632f66..2a70c5e8f 100644 --- a/vmod/automake_boilerplate_debug.am +++ b/vmod/automake_boilerplate_debug.am @@ -12,7 +12,8 @@ libvmod_debug_la_SOURCES = \ vmod_debug_acl.c \ vmod_debug_dyn.c \ vmod_debug_filters.c \ - vmod_debug_obj.c + vmod_debug_obj.c \ + vmod_debug_transports.c libvmod_debug_la_CFLAGS = diff --git a/vmod/vmod_debug.c b/vmod/vmod_debug.c index 7e399f7b7..a5a097163 100644 --- a/vmod/vmod_debug.c +++ b/vmod/vmod_debug.c @@ -332,6 +332,7 @@ event_load(VRT_CTX, struct vmod_priv *priv) priv->methods = priv_vcl_methods; debug_add_filters(ctx); + debug_transport_init(); return (0); } @@ -1281,3 +1282,9 @@ xyzzy_resolve_range(VRT_CTX, struct VARGS(resolve_range) *args) *(p.errp)); return (WS_VSB_finish(p.vsb, ctx->ws, NULL)); } + +VCL_VOID +xyzzy_use_reembarking_http1(VRT_CTX) +{ + debug_transport_use_reembarking_http1(ctx); +} diff --git a/vmod/vmod_debug.h b/vmod/vmod_debug.h index 05093be07..ddff80dc8 100644 --- a/vmod/vmod_debug.h +++ b/vmod/vmod_debug.h @@ -33,3 +33,9 @@ void debug_add_filters(VRT_CTX); void debug_remove_filters(VRT_CTX); + +/* vmod_debug_transports.c */ +void +debug_transport_use_reembarking_http1(VRT_CTX); +void +debug_transport_init(void); diff --git a/vmod/vmod_debug.vcc b/vmod/vmod_debug.vcc index 8e9a25c3e..3d791a6c8 100644 --- a/vmod/vmod_debug.vcc +++ b/vmod/vmod_debug.vcc @@ -440,6 +440,13 @@ be hanged to zero. Any larger value will be taken modulo UINT32_MAX. The *mode* argument behaves as for `debug.chksha256()`_. +$Function VOID use_reembarking_http1() + +$Restrict vcl_deliver + +Switch to the reembarking http1 debug transport. Calling it from any other +transport than http1 results in VCL failure. + DEPRECATED ========== diff --git a/vmod/vmod_debug_transports.c b/vmod/vmod_debug_transports.c new file mode 100644 index 000000000..9b9ec1c7a --- /dev/null +++ b/vmod/vmod_debug_transports.c @@ -0,0 +1,224 @@ +/*- + * Copyright (c) 2006 Verdens Gang AS + * Copyright (c) 2006-2015 Varnish Software AS + * Copyright 2024 UPLEX - Nils Goroll Systemoptimierung + * All rights reserved. + * + * Authors: Poul-Henning Kamp + * Nils Goroll + * + * SPDX-License-Identifier: BSD-2-Clause + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include "config.h" + +#include "cache/cache_varnishd.h" + +#include "cache/cache_filter.h" +#include "cache/cache_transport.h" +#include "http1/cache_http1.h" + +#include "vmod_debug.h" + +static void +dbg_error(struct req *req, struct v1l **v1lp, const char *msg) +{ + + (void)req; + (void)v1lp; + (void)msg; + INCOMPL(); +} + +static void dbg_deliver_finish(struct req *req, struct v1l **v1lp, int err); +static void dbg_sendbody(struct worker *wrk, void *arg); + +static task_func_t *hack_http1_req = NULL; + +// copied from cache_http_deliver.c, then split & modified +static enum vtr_deliver_e v_matchproto_(vtr_deliver_f) +dbg_deliver(struct req *req, int sendbody) +{ + struct vrt_ctx ctx[1]; + struct v1l *v1l; + + CHECK_OBJ_NOTNULL(req, REQ_MAGIC); + CHECK_OBJ_ORNULL(req->boc, BOC_MAGIC); + CHECK_OBJ_NOTNULL(req->objcore, OBJCORE_MAGIC); + + if (req->doclose == SC_NULL && + http_HdrIs(req->resp, H_Connection, "close")) { + req->doclose = SC_RESP_CLOSE; + } else if (req->doclose != SC_NULL) { + if (!http_HdrIs(req->resp, H_Connection, "close")) { + http_Unset(req->resp, H_Connection); + http_SetHeader(req->resp, "Connection: close"); + } + } else if (!http_GetHdr(req->resp, H_Connection, NULL)) + http_SetHeader(req->resp, "Connection: keep-alive"); + + CHECK_OBJ_NOTNULL(req->wrk, WORKER_MAGIC); + + v1l = V1L_Open(req->wrk->aws, &req->sp->fd, req->vsl, + req->t_prev + SESS_TMO(req->sp, send_timeout), + cache_param->http1_iovs); + + if (v1l == NULL) { + dbg_error(req, &v1l, "Failure to init v1d (workspace_thread overflow)"); + return (VTR_D_DONE); + } + + if (sendbody) { + if (!http_GetHdr(req->resp, H_Content_Length, NULL)) { + if (req->http->protover == 11) { + http_SetHeader(req->resp, + "Transfer-Encoding: chunked"); + } else { + req->doclose = SC_TX_EOF; + } + } + INIT_OBJ(ctx, VRT_CTX_MAGIC); + VCL_Req2Ctx(ctx, req); + if (VDP_Push(ctx, req->vdc, req->ws, VDP_v1l, v1l)) { + dbg_error(req, &v1l, "Failure to push v1d processor"); + return (VTR_D_DONE); + } + } + + if (WS_Overflowed(req->ws)) { + dbg_error(req, &v1l, "workspace_client overflow"); + return (VTR_D_DONE); + } + + if (WS_Overflowed(req->sp->ws)) { + dbg_error(req, &v1l, "workspace_session overflow"); + return (VTR_D_DONE); + } + + if (WS_Overflowed(req->wrk->aws)) { + dbg_error(req, &v1l, "workspace_thread overflow"); + return (VTR_D_DONE); + } + + req->acct.resp_hdrbytes += HTTP1_Write(v1l, req->resp, HTTP1_Resp); + + if (! sendbody) { + dbg_deliver_finish(req, &v1l, 0); + return (VTR_D_DONE); + } + + (void)V1L_Flush(v1l); + + if (hack_http1_req == NULL) + hack_http1_req = req->task->func; + AN(hack_http1_req); + + VSLb(req->vsl, SLT_Debug, "w=%p scheduling dbg_sendbody", req->wrk); + + req->task->func = dbg_sendbody; + req->task->priv = req; + + req->wrk = NULL; + req->vdc->wrk = NULL; + req->transport_priv = v1l; + + AZ(Pool_Task(req->sp->pool, req->task, TASK_QUEUE_RUSH)); + return (VTR_D_DISEMBARK); +} + +static void v_matchproto_(task_func_t) +dbg_sendbody(struct worker *wrk, void *arg) +{ + struct req *req; + struct v1l *v1l; + const char *p; + int err, chunked; + + CHECK_OBJ_NOTNULL(wrk, WORKER_MAGIC); + CAST_OBJ_NOTNULL(req, arg, REQ_MAGIC); + v1l = req->transport_priv; + req->transport_priv = NULL; + AN(v1l); + + THR_SetRequest(req); + VSLb(req->vsl, SLT_Debug, "w=%p enter dbg_sendbody", wrk); + AZ(req->wrk); + CNT_Embark(wrk, req); + req->vdc->wrk = wrk; // move to CNT_Embark? + + chunked = http_GetHdr(req->resp, H_Transfer_Encoding, &p) && strcmp(p, "chunked") == 0; + if (chunked) + V1L_Chunked(v1l); + err = VDP_DeliverObj(req->vdc, req->objcore); + if (!err && chunked) + V1L_EndChunk(v1l); + dbg_deliver_finish(req, &v1l, err); + + VSLb(req->vsl, SLT_Debug, "w=%p resuming http1_req", wrk); + wrk->task->func = hack_http1_req; + wrk->task->priv = req; +} + +static void +dbg_deliver_finish(struct req *req, struct v1l **v1lp, int err) +{ + stream_close_t sc; + uint64_t bytes; + + sc = V1L_Close(v1lp, &bytes); + + req->acct.resp_bodybytes += VDP_Close(req->vdc, req->objcore, req->boc); + + if (sc == SC_NULL && err && req->sp->fd >= 0) + sc = SC_REM_CLOSE; + if (sc != SC_NULL) + Req_Fail(req, sc); +} + +struct transport DBG_transport; + +void +debug_transport_init(void) +{ + DBG_transport = HTTP1_transport; + DBG_transport.name = "DBG"; + DBG_transport.deliver = dbg_deliver; +} + +void +debug_transport_use_reembarking_http1(VRT_CTX) +{ + struct req *req; + + CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC); + req = ctx->req; + CHECK_OBJ_NOTNULL(req, REQ_MAGIC); + + if (req->transport != &HTTP1_transport) { + VRT_fail(ctx, "Only works on built-in http1 transport"); + return; + } + AZ(req->transport_priv); + req->transport = &DBG_transport; +} From nils.goroll at uplex.de Mon Nov 25 17:51:05 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Mon, 25 Nov 2024 17:51:05 +0000 (UTC) Subject: [master] f7b6e247e Add missing static declaration Message-ID: <20241125175105.66281119DA5@lists.varnish-cache.org> commit f7b6e247ef43de3a9c15aa6ae0c129a7e4b673fe Author: Nils Goroll Date: Mon Nov 25 18:50:29 2024 +0100 Add missing static declaration reported by Flexelint diff --git a/vmod/vmod_debug_transports.c b/vmod/vmod_debug_transports.c index 9b9ec1c7a..90bd0a48b 100644 --- a/vmod/vmod_debug_transports.c +++ b/vmod/vmod_debug_transports.c @@ -196,7 +196,7 @@ dbg_deliver_finish(struct req *req, struct v1l **v1lp, int err) Req_Fail(req, sc); } -struct transport DBG_transport; +static struct transport DBG_transport; void debug_transport_init(void) From dridi.boukelmoune at gmail.com Fri Nov 29 07:28:11 2024 From: dridi.boukelmoune at gmail.com (Dridi Boukelmoune) Date: Fri, 29 Nov 2024 07:28:11 +0000 (UTC) Subject: [master] 027c91f21 circleci: Remove EOL'd debian:bullseye Message-ID: <20241129072811.A6E0B10973A@lists.varnish-cache.org> commit 027c91f2166e260d2922e933e27c01cff81a5228 Author: Dridi Boukelmoune Date: Fri Nov 29 08:22:36 2024 +0100 circleci: Remove EOL'd debian:bullseye And drop 32bit coverage that recently broke altogether and a little out of the blue. diff --git a/.circleci/config.yml b/.circleci/config.yml index 8c930ec8a..b1491127f 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -203,7 +203,7 @@ jobs: description: the Linux distribution (debian|ubuntu) type: string release: - description: the release name (bullseye|bookworm|focal|jammy|noble) + description: the release name (bookworm|focal|jammy|noble) type: string make_target: description: the make target to execute during the build @@ -385,12 +385,6 @@ workflows: dist: fedora release: latest make_target: witness.dot - # oldest debian goes 32bit - - build: - name: build_debian_bullseye - dist: debian - release: bullseye - prefix: i386/ - build: name: build_debian_bookworm dist: debian @@ -437,7 +431,6 @@ workflows: - ubuntu:focal - ubuntu:jammy - ubuntu:noble - - debian:bullseye - debian:bookworm - almalinux:8 - almalinux:9 From nils.goroll at uplex.de Fri Nov 29 08:34:43 2024 From: nils.goroll at uplex.de (Nils Goroll) Date: Fri, 29 Nov 2024 09:34:43 +0100 Subject: [master] 027c91f21 circleci: Remove EOL'd debian:bullseye In-Reply-To: <20241129072811.A6E0B10973A@lists.varnish-cache.org> References: <20241129072811.A6E0B10973A@lists.varnish-cache.org> Message-ID: On 29.11.24 08:28, Dridi Boukelmoune wrote: > And drop 32bit coverage that recently broke altogether and a little out > of the blue. Where did that happen? My 32bit vtesters are still happy. -- Nils Goroll (he/him) ** * * UPLEX - Nils Goroll Systemoptimierung Scheffelstra?e 32 22301 Hamburg tel +49 40 28805731 mob +49 170 2723133 fax +49 40 42949753 xmpp://slink at jabber.int.uplex.de/ http://uplex.de/ -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_0x1DCD8F57A3868BD7.asc Type: application/pgp-keys Size: 3943 bytes Desc: OpenPGP public key URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature.asc Type: application/pgp-signature Size: 488 bytes Desc: OpenPGP digital signature URL: From dridi at varni.sh Fri Nov 29 09:02:42 2024 From: dridi at varni.sh (Dridi Boukelmoune) Date: Fri, 29 Nov 2024 09:02:42 +0000 Subject: [master] 027c91f21 circleci: Remove EOL'd debian:bullseye In-Reply-To: References: <20241129072811.A6E0B10973A@lists.varnish-cache.org> Message-ID: On Fri, Nov 29, 2024 at 8:35?AM Nils Goroll wrote: > > On 29.11.24 08:28, Dridi Boukelmoune wrote: > > And drop 32bit coverage that recently broke altogether and a little out > > of the blue. > > Where did that happen? My 32bit vtesters are still happy. It's the docker logic selecting the i386 image for bullseye in our cci configuration that broke. Never write commit messages in a hurry...