From jb-varnish at wisemo.com Wed May 10 01:10:08 2023 From: jb-varnish at wisemo.com (Jakob Bohm) Date: Wed, 10 May 2023 03:10:08 +0200 Subject: Mysterious no content result, from an URL with pass action Message-ID: <9d1f9ed6-1b52-88ba-9cdf-b1d613fe4069@wisemo.com> Dear Varnish mailing list, When testing varnish as a reverse proxy for multiple services including a local JetBrains TeamCity instance, requests to that teamcity server get corrupted into "204 No Content" replies. Once again, Varnish fails to say why it is refusing to do its job. Any sane program should explicitly and loudly report any fatal error that stops it working.? Loudly means the sysadmin or other user invoking the program receives the exaxt error message by default instead of something highly indirect, hidden behind a debug option or otherwisse highly non-obvious. Here's a relevant clip from the VCL: # Various top comments vcl 4.1; import std; import proxy; # Backend sending requests to the teamcity main server backend teamcity { ??? .host = "2a01:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx"; ??? .port = "8111"; } # IP ranges allowed to access the build server and staging server acl buildtrust { ??? "127.0.0.0"/8; ??? "::"/128; ??? "various others"/??; } # IP ranges allowed to attempt login to things that use our common login #??? database acl logintrust { ??? "various others"/??; } sub vcl_recv { ??? # Happens before we check if we have this in cache already. ??? # ??? # Typically you clean up the request here, removing cookies you don't need, ??? # rewriting the request, etc. ??? if (proxy.is_ssl()) { ??????? set req.http.Scheme = "https"; ??????? set req.http.ssl-version = proxy.ssl_version(); ??????? set req.http.X-Forwarded-Proto = "https"; ??????? set req.http.X-SSL-cipher = proxy.ssl_cipher(); ??????? std.log("TLS-SSL-VERSION: " + proxy.ssl_version()); ??? } else { ??????? set req.http.X-Forwarded-Proto = req.http.Scheme; ??????? unset req.http.ssl-version; ??????? unset req.http.X-SSL-cipher; ??????? std.log("TLS-SSL-VERSION: none"); ??? } ??? unset req.http.X-SSL-Subject; ??? unset req.http.X-SSL-Issuer; ??? unset req.http.X-SSL-notBefore; ??? unset req.http.X-SSL-notAfter; ??? unset req.http.X-SSL-serial; ??? unset req.http.X-SSL-certificate; ??? set req.http.X-Forwarded-For = client.ip; ??? call vcl_req_host; ??? if (req.url ~ "^/something") { ??????? set req.backend_hint = be1; ??? } else if (req.url !~ "^/somethingelse" && ?????????????? !(client.ip ~ logintrust) && ?????????????? !(client.ip ~ buildtrust)) { ??????? # Treat as unknown by redirecting to public website ??????? if ((req.url ~ "^/yeatanother") || ??????????? (req.url ~ "^/yetsomeother")) { ??????????? return (synth(752)); ??????? } else if (req.url ~ "^/yetsomethird") { ??????????? return (synth(753)); ??????? } ??????? return (synth(751)); ??? } else if (req.http.Scheme && req.http.Scheme != "https") { ??????? # See example at https://www.varnish-software.com/developers/tutorials/redirect/ ??????? return (synth(750)); ??? } else if (req.url ~ "^/somethingelse") { ??????? set req.backend_hint = be1; ??? } else if (req.url ~ "^/somethingfourth") { ??????? set req.backend_hint = be2; ??? } else if (req.url ~ "^/somethingfifth") { ??????? set req.backend_hint = be2; ??? } else if (!(client.ip ~ buildtrust)) { ??????? # Treat as unknown by redirecting to public website ??????? if ((req.url ~ "^/yeatanother") || ??????????? (req.url ~ "^/yetsomeother")) { ??????????? return (synth(752)); ??????? } else if (req.url ~ "^/yetsomethird") { ??????????? return (synth(753)); ??????? } ??????? return (synth(751)); ??? } else if (req.url ~ "^/teamcity") { ??????? set req.backend_hint= teamcity; ??????? return (pass); #??? } else if (req.http.host ~ "^somethingsixths") { #?????? set req.backend_hint= be4; ??? } else { ??????? set req.backend_hint = be5; ??? } ??? call vcl_req_method; ??? call vcl_req_authorization; ??? call vcl_req_cookie; ??? return (hash); } sub vcl_backend_response { ??? # Happens after we have read the response headers from the backend. ??? # ??? # Here you clean the response headers, removing silly Set-Cookie headers ??? # and other mistakes your backend does. ??? # The Java webserver in teamcity is incompatible with varnish connection ??? #??? pooling ??? if (beresp.backend == teamcity) { ??????? if (beresp.http.Connection && ??????????? beresp.http.Connection !~ "keep-alive") { ??????????? set beresp.http.Connection += ", close"; ??????? } else { ??????????? set beresp.http.Connection = "close"; ??????? } ??? } } +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ First 43 lines of varnishlog -v 2>&1 *????????????? << BeReq??? >>?? 9 -??????????? 9 Begin????????? b bereq 8 pass -??????????? 9 VCL_use??????? b boot -??????????? 9 Timestamp????? b Start: 1681484803.177212 0.000000 0.000000 -??????????? 9 BereqMethod??? b GET -??????????? 9 BereqURL?????? b /teamcity/app/agents/v1/commands/next -??????????? 9 BereqProtocol? b HTTP/1.1 -??????????? 9 BereqHeader??? b TeamCity-AgentSessionId: L6juFAAt1awJDt6UKToPIxQq7wpBF89C -??????????? 9 BereqHeader??? b User-Agent: TeamCity Agent 2021.2.3 -??????????? 9 BereqHeader??? b Host: vmachine.example.com -??????????? 9 BereqHeader??? b Via: 1.1 vmachine (Varnish/7.2) -??????????? 9 BereqHeader??? b Scheme: https -??????????? 9 BereqHeader??? b ssl-version: TLSv1.3 -??????????? 9 BereqHeader??? b X-Forwarded-Proto: https -??????????? 9 BereqHeader??? b X-SSL-cipher: TLS_AES_256_GCM_SHA384 -??????????? 9 BereqHeader??? b X-Forwarded-For: 192.168.2.112 -??????????? 9 BereqHeader??? b X-Varnish: 9 -??????????? 9 VCL_call?????? b BACKEND_FETCH -??????????? 9 VCL_return???? b fetch -??????????? 9 Timestamp????? b Fetch: 1681484803.177227 0.000014 0.000014 -??????????? 9 Timestamp????? b Connected: 1681484803.177603 0.000390 0.000375 -??????????? 9 BackendOpen??? b 24 teamcity 2a01:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx 8111 2a01:yyyy:yyyy:yyyy::yyyy 59548 connect -??????????? 9 Timestamp????? b Bereq: 1681484803.177645 0.000432 0.000042 -??????????? 9 BerespReason?? b No Content -??????????? 9 Timestamp????? b Beresp: 1681484830.672487 27.495274 27.494842 -??????????? 9 BerespProtocol b HTTP/1.1 -??????????? 9 BerespStatus?? b 204 -??????????? 9 BerespReason?? b No Content -??????????? 9 BerespHeader?? b TeamCity-Node-Id: MAIN_SERVER -??????????? 9 BerespHeader?? b Date: Fri, 14 Apr 2023 15:07:10 GMT -??????????? 9 VCL_call?????? b BACKEND_RESPONSE -??????????? 9 BerespHeader?? b Connection: close -??????????? 9 VCL_return???? b deliver -??????????? 9 Timestamp????? b Process: 1681484830.672563 27.495350 0.000075 -??????????? 9 Filters??????? b -??????????? 9 Storage??????? b malloc Transient -??????????? 9 Fetch_Body???? b 0 none - -??????????? 9 BackendClose?? b 24 teamcity close Backend/VCL requested close -??????????? 9 Timestamp????? b BerespBody: 1681484830.672926 27.495713 0.000362 -??????????? 9 Length???????? b 0 -??????????? 9 BereqAcct????? b 345 0 345 85 0 85 -??????????? 9 End??????????? b Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 S?borg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded From guillaume.quintard at gmail.com Wed May 10 04:49:06 2023 From: guillaume.quintard at gmail.com (Guillaume Quintard) Date: Tue, 9 May 2023 18:49:06 -1000 Subject: Mysterious no content result, from an URL with pass action In-Reply-To: <9d1f9ed6-1b52-88ba-9cdf-b1d613fe4069@wisemo.com> References: <9d1f9ed6-1b52-88ba-9cdf-b1d613fe4069@wisemo.com> Message-ID: Hi Jakob, (Sorry i didn't see that email sooner, it was in my spam folder) Looking at the log, I'm not sure what varnish should be loud about :-) 204 is a success code, and more importantly it's generated by the backend, so varnish is happily passing it along. At the http level, everything looks about right, but I can guess from your apparent irritation that something wrong one level up, let's try to debug that. What kind of response are you expecting, if not a 204? And maybe, what is that endpoint supposed to do? Given that the method was GET, and that there's no body, my only guess is that there's something happening with the TeamCity-AgentSessionId header, maybe? Is the 27 seconds processing time expected? Cheers, On Tue, May 9, 2023, 15:12 Jakob Bohm wrote: > Dear Varnish mailing list, > > When testing varnish as a reverse proxy for multiple services > including a local JetBrains TeamCity instance, requests to that > teamcity server get corrupted into "204 No Content" replies. > > Once again, Varnish fails to say why it is refusing to do its job. > Any sane program should explicitly and loudly report any fatal error > that stops it working. Loudly means the sysadmin or other user > invoking the program receives the exaxt error message by default > instead of something highly indirect, hidden behind a debug option > or otherwisse highly non-obvious. > > Here's a relevant clip from the VCL: > > # Various top comments > vcl 4.1; > > import std; > import proxy; > > # Backend sending requests to the teamcity main server > backend teamcity { > .host = "2a01:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx"; > .port = "8111"; > } > > # IP ranges allowed to access the build server and staging server > acl buildtrust { > "127.0.0.0"/8; > "::"/128; > "various others"/??; > } > > # IP ranges allowed to attempt login to things that use our common login > # database > acl logintrust { > "various others"/??; > } > > sub vcl_recv { > # Happens before we check if we have this in cache already. > # > # Typically you clean up the request here, removing cookies you > don't need, > # rewriting the request, etc. > if (proxy.is_ssl()) { > set req.http.Scheme = "https"; > set req.http.ssl-version = proxy.ssl_version(); > set req.http.X-Forwarded-Proto = "https"; > set req.http.X-SSL-cipher = proxy.ssl_cipher(); > std.log("TLS-SSL-VERSION: " + proxy.ssl_version()); > } else { > set req.http.X-Forwarded-Proto = req.http.Scheme; > unset req.http.ssl-version; > unset req.http.X-SSL-cipher; > std.log("TLS-SSL-VERSION: none"); > } > unset req.http.X-SSL-Subject; > unset req.http.X-SSL-Issuer; > unset req.http.X-SSL-notBefore; > unset req.http.X-SSL-notAfter; > unset req.http.X-SSL-serial; > unset req.http.X-SSL-certificate; > > set req.http.X-Forwarded-For = client.ip; > > call vcl_req_host; > > if (req.url ~ "^/something") { > set req.backend_hint = be1; > } else if (req.url !~ "^/somethingelse" && > !(client.ip ~ logintrust) && > !(client.ip ~ buildtrust)) { > # Treat as unknown by redirecting to public website > if ((req.url ~ "^/yeatanother") || > (req.url ~ "^/yetsomeother")) { > return (synth(752)); > } else if (req.url ~ "^/yetsomethird") { > return (synth(753)); > } > return (synth(751)); > } else if (req.http.Scheme && req.http.Scheme != "https") { > # See example at > https://www.varnish-software.com/developers/tutorials/redirect/ > return (synth(750)); > } else if (req.url ~ "^/somethingelse") { > set req.backend_hint = be1; > } else if (req.url ~ "^/somethingfourth") { > set req.backend_hint = be2; > } else if (req.url ~ "^/somethingfifth") { > set req.backend_hint = be2; > } else if (!(client.ip ~ buildtrust)) { > # Treat as unknown by redirecting to public website > if ((req.url ~ "^/yeatanother") || > (req.url ~ "^/yetsomeother")) { > return (synth(752)); > } else if (req.url ~ "^/yetsomethird") { > return (synth(753)); > } > return (synth(751)); > } else if (req.url ~ "^/teamcity") { > set req.backend_hint= teamcity; > return (pass); > # } else if (req.http.host ~ "^somethingsixths") { > # set req.backend_hint= be4; > } else { > set req.backend_hint = be5; > } > call vcl_req_method; > call vcl_req_authorization; > call vcl_req_cookie; > return (hash); > } > > sub vcl_backend_response { > # Happens after we have read the response headers from the backend. > # > # Here you clean the response headers, removing silly Set-Cookie > headers > # and other mistakes your backend does. > > # The Java webserver in teamcity is incompatible with varnish > connection > # pooling > if (beresp.backend == teamcity) { > if (beresp.http.Connection && > beresp.http.Connection !~ "keep-alive") { > set beresp.http.Connection += ", close"; > } else { > set beresp.http.Connection = "close"; > } > } > } > > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > First 43 lines of varnishlog -v 2>&1 > > * << BeReq >> 9 > - 9 Begin b bereq 8 pass > - 9 VCL_use b boot > - 9 Timestamp b Start: 1681484803.177212 0.000000 0.000000 > - 9 BereqMethod b GET > - 9 BereqURL b /teamcity/app/agents/v1/commands/next > - 9 BereqProtocol b HTTP/1.1 > - 9 BereqHeader b TeamCity-AgentSessionId: > L6juFAAt1awJDt6UKToPIxQq7wpBF89C > - 9 BereqHeader b User-Agent: TeamCity Agent 2021.2.3 > - 9 BereqHeader b Host: vmachine.example.com > - 9 BereqHeader b Via: 1.1 vmachine (Varnish/7.2) > - 9 BereqHeader b Scheme: https > - 9 BereqHeader b ssl-version: TLSv1.3 > - 9 BereqHeader b X-Forwarded-Proto: https > - 9 BereqHeader b X-SSL-cipher: TLS_AES_256_GCM_SHA384 > - 9 BereqHeader b X-Forwarded-For: 192.168.2.112 > - 9 BereqHeader b X-Varnish: 9 > - 9 VCL_call b BACKEND_FETCH > - 9 VCL_return b fetch > - 9 Timestamp b Fetch: 1681484803.177227 0.000014 0.000014 > - 9 Timestamp b Connected: 1681484803.177603 0.000390 > 0.000375 > - 9 BackendOpen b 24 teamcity > 2a01:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx 8111 2a01:yyyy:yyyy:yyyy::yyyy > 59548 connect > - 9 Timestamp b Bereq: 1681484803.177645 0.000432 0.000042 > - 9 BerespReason b No Content > - 9 Timestamp b Beresp: 1681484830.672487 27.495274 > 27.494842 > - 9 BerespProtocol b HTTP/1.1 > - 9 BerespStatus b 204 > - 9 BerespReason b No Content > - 9 BerespHeader b TeamCity-Node-Id: MAIN_SERVER > - 9 BerespHeader b Date: Fri, 14 Apr 2023 15:07:10 GMT > - 9 VCL_call b BACKEND_RESPONSE > - 9 BerespHeader b Connection: close > - 9 VCL_return b deliver > - 9 Timestamp b Process: 1681484830.672563 27.495350 > 0.000075 > - 9 Filters b > - 9 Storage b malloc Transient > - 9 Fetch_Body b 0 none - > - 9 BackendClose b 24 teamcity close Backend/VCL requested > close > - 9 Timestamp b BerespBody: 1681484830.672926 27.495713 > 0.000362 > - 9 Length b 0 > - 9 BereqAcct b 345 0 345 85 0 85 > - 9 End b > > > > > Enjoy > > Jakob > -- > Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com > Transformervej 29, 2860 S?borg, Denmark. Direct +45 31 13 16 10 > This public discussion message is non-binding and may contain errors. > WiseMo - Remote Service Management for PCs, Phones and Embedded > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jb-varnish at wisemo.com Wed May 10 08:43:24 2023 From: jb-varnish at wisemo.com (Jakob Bohm) Date: Wed, 10 May 2023 10:43:24 +0200 Subject: Mysterious no content result, from an URL with pass action In-Reply-To: References: <9d1f9ed6-1b52-88ba-9cdf-b1d613fe4069@wisemo.com> Message-ID: On 2023-05-10 06:49, Guillaume Quintard wrote: > Hi Jakob, > > (Sorry i didn't see that email sooner, it was in my spam folder) > > Looking at the log, I'm not sure what varnish should be loud about :-) > 204 is a success code, and more importantly it's generated by the > backend, so varnish is happily passing it along. > > At the http level, everything looks about right, but I can guess from > your apparent irritation that something wrong one level up, let's try to > debug that. > > What kind of response are you expecting, if not a 204? And maybe, what > is that endpoint supposed to do? Given that the method was GET, and that > there's no body, my only guess is that there's something happening with > the?TeamCity-AgentSessionId header, maybe? > Is the 27 seconds processing time expected? > Expecting uncachable results that vary with time and are only sometimes 204, and the response time is also somewhat unexpected, but is not clearly logged (only a Varnish expert like you can decrypt that it is 27 seconds). It is also unclear if Varnish is always receiving those responses from the backend. I also expected some other URLs in the log, but don't see them. Maybe I should find another day to run the experiments again. > Cheers, > > On Tue, May 9, 2023, 15:12 Jakob Bohm > wrote: > > Dear Varnish mailing list, > > When testing varnish as a reverse proxy for multiple services > including a local JetBrains TeamCity instance, requests to that > teamcity server get corrupted into "204 No Content" replies. > > Once again, Varnish fails to say why it is refusing to do its job. > Any sane program should explicitly and loudly report any fatal error > that stops it working.? Loudly means the sysadmin or other user > invoking the program receives the exaxt error message by default > instead of something highly indirect, hidden behind a debug option > or otherwisse highly non-obvious. > > Here's a relevant clip from the VCL: > > # Various top comments > vcl 4.1; > > import std; > import proxy; > > # Backend sending requests to the teamcity main server > backend teamcity { > ???? .host = "2a01:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx"; > ???? .port = "8111"; > } > > # IP ranges allowed to access the build server and staging server > acl buildtrust { > ???? "127.0.0.0"/8; > ???? "::"/128; > ???? "various others"/??; > } > > # IP ranges allowed to attempt login to things that use our common login > #??? database > acl logintrust { > ???? "various others"/??; > } > > sub vcl_recv { > ???? # Happens before we check if we have this in cache already. > ???? # > ???? # Typically you clean up the request here, removing cookies you > don't need, > ???? # rewriting the request, etc. > ???? if (proxy.is_ssl()) { > ???????? set req.http.Scheme = "https"; > ???????? set req.http.ssl-version = proxy.ssl_version(); > ???????? set req.http.X-Forwarded-Proto = "https"; > ???????? set req.http.X-SSL-cipher = proxy.ssl_cipher(); > ???????? std.log("TLS-SSL-VERSION: " + proxy.ssl_version()); > ???? } else { > ???????? set req.http.X-Forwarded-Proto = req.http.Scheme; > ???????? unset req.http.ssl-version; > ???????? unset req.http.X-SSL-cipher; > ???????? std.log("TLS-SSL-VERSION: none"); > ???? } > ???? unset req.http.X-SSL-Subject; > ???? unset req.http.X-SSL-Issuer; > ???? unset req.http.X-SSL-notBefore; > ???? unset req.http.X-SSL-notAfter; > ???? unset req.http.X-SSL-serial; > ???? unset req.http.X-SSL-certificate; > > ???? set req.http.X-Forwarded-For = client.ip; > > ???? call vcl_req_host; > > ???? if (req.url ~ "^/something") { > ???????? set req.backend_hint = be1; > ???? } else if (req.url !~ "^/somethingelse" && > ??????????????? !(client.ip ~ logintrust) && > ??????????????? !(client.ip ~ buildtrust)) { > ???????? # Treat as unknown by redirecting to public website > ???????? if ((req.url ~ "^/yeatanother") || > ???????????? (req.url ~ "^/yetsomeother")) { > ???????????? return (synth(752)); > ???????? } else if (req.url ~ "^/yetsomethird") { > ???????????? return (synth(753)); > ???????? } > ???????? return (synth(751)); > ???? } else if (req.http.Scheme && req.http.Scheme != "https") { > ???????? # See example at > https://www.varnish-software.com/developers/tutorials/redirect/ > ???????? return (synth(750)); > ???? } else if (req.url ~ "^/somethingelse") { > ???????? set req.backend_hint = be1; > ???? } else if (req.url ~ "^/somethingfourth") { > ???????? set req.backend_hint = be2; > ???? } else if (req.url ~ "^/somethingfifth") { > ???????? set req.backend_hint = be2; > ???? } else if (!(client.ip ~ buildtrust)) { > ???????? # Treat as unknown by redirecting to public website > ???????? if ((req.url ~ "^/yeatanother") || > ???????????? (req.url ~ "^/yetsomeother")) { > ???????????? return (synth(752)); > ???????? } else if (req.url ~ "^/yetsomethird") { > ???????????? return (synth(753)); > ???????? } > ???????? return (synth(751)); > ???? } else if (req.url ~ "^/teamcity") { > ???????? set req.backend_hint= teamcity; > ???????? return (pass); > #??? } else if (req.http.host ~ "^somethingsixths") { > #?????? set req.backend_hint= be4; > ???? } else { > ???????? set req.backend_hint = be5; > ???? } > ???? call vcl_req_method; > ???? call vcl_req_authorization; > ???? call vcl_req_cookie; > ???? return (hash); > } > > sub vcl_backend_response { > ???? # Happens after we have read the response headers from the > backend. > ???? # > ???? # Here you clean the response headers, removing silly Set-Cookie > headers > ???? # and other mistakes your backend does. > > ???? # The Java webserver in teamcity is incompatible with varnish > connection > ???? #??? pooling > ???? if (beresp.backend == teamcity) { > ???????? if (beresp.http.Connection && > ???????????? beresp.http.Connection !~ "keep-alive") { > ???????????? set beresp.http.Connection += ", close"; > ???????? } else { > ???????????? set beresp.http.Connection = "close"; > ???????? } > ???? } > } > > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > First 43 lines of varnishlog -v 2>&1 > > *????????????? << BeReq??? >>?? 9 > -??????????? 9 Begin????????? b bereq 8 pass > -??????????? 9 VCL_use??????? b boot > -??????????? 9 Timestamp????? b Start: 1681484803.177212 0.000000 > 0.000000 > -??????????? 9 BereqMethod??? b GET > -??????????? 9 BereqURL?????? b /teamcity/app/agents/v1/commands/next > -??????????? 9 BereqProtocol? b HTTP/1.1 > -??????????? 9 BereqHeader??? b TeamCity-AgentSessionId: > L6juFAAt1awJDt6UKToPIxQq7wpBF89C > -??????????? 9 BereqHeader??? b User-Agent: TeamCity Agent 2021.2.3 > -??????????? 9 BereqHeader??? b Host: vmachine.example.com > > -??????????? 9 BereqHeader??? b Via: 1.1 vmachine (Varnish/7.2) > -??????????? 9 BereqHeader??? b Scheme: https > -??????????? 9 BereqHeader??? b ssl-version: TLSv1.3 > -??????????? 9 BereqHeader??? b X-Forwarded-Proto: https > -??????????? 9 BereqHeader??? b X-SSL-cipher: TLS_AES_256_GCM_SHA384 > -??????????? 9 BereqHeader??? b X-Forwarded-For: 192.168.2.112 > -??????????? 9 BereqHeader??? b X-Varnish: 9 > -??????????? 9 VCL_call?????? b BACKEND_FETCH > -??????????? 9 VCL_return???? b fetch > -??????????? 9 Timestamp????? b Fetch: 1681484803.177227 0.000014 > 0.000014 > -??????????? 9 Timestamp????? b Connected: 1681484803.177603 0.000390 > 0.000375 > -??????????? 9 BackendOpen??? b 24 teamcity > 2a01:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx 8111 2a01:yyyy:yyyy:yyyy::yyyy > 59548 connect > -??????????? 9 Timestamp????? b Bereq: 1681484803.177645 0.000432 > 0.000042 > -??????????? 9 BerespReason?? b No Content > -??????????? 9 Timestamp????? b Beresp: 1681484830.672487 27.495274 > 27.494842 > -??????????? 9 BerespProtocol b HTTP/1.1 > -??????????? 9 BerespStatus?? b 204 > -??????????? 9 BerespReason?? b No Content > -??????????? 9 BerespHeader?? b TeamCity-Node-Id: MAIN_SERVER > -??????????? 9 BerespHeader?? b Date: Fri, 14 Apr 2023 15:07:10 GMT > -??????????? 9 VCL_call?????? b BACKEND_RESPONSE > -??????????? 9 BerespHeader?? b Connection: close > -??????????? 9 VCL_return???? b deliver > -??????????? 9 Timestamp????? b Process: 1681484830.672563 27.495350 > 0.000075 > -??????????? 9 Filters??????? b > -??????????? 9 Storage??????? b malloc Transient > -??????????? 9 Fetch_Body???? b 0 none - > -??????????? 9 BackendClose?? b 24 teamcity close Backend/VCL requested > close > -??????????? 9 Timestamp????? b BerespBody: 1681484830.672926 27.495713 > 0.000362 > -??????????? 9 Length???????? b 0 > -??????????? 9 BereqAcct????? b 345 0 345 85 0 85 > -??????????? 9 End??????????? b > Enjoy Jakob -- Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com Transformervej 29, 2860 S?borg, Denmark. Direct +45 31 13 16 10 This public discussion message is non-binding and may contain errors. WiseMo - Remote Service Management for PCs, Phones and Embedded From guillaume.quintard at gmail.com Thu May 11 19:26:59 2023 From: guillaume.quintard at gmail.com (Guillaume Quintard) Date: Thu, 11 May 2023 09:26:59 -1000 Subject: Mysterious no content result, from an URL with pass action In-Reply-To: References: <9d1f9ed6-1b52-88ba-9cdf-b1d613fe4069@wisemo.com> Message-ID: On Tue, May 9, 2023, 22:45 Jakob Bohm wrote: > Expecting uncachable results that vary with time and are only sometimes > 204, Understood, but that one looks like a backend issue. Also, just to be clear, the response is uncatchable because the back looked at the url and deemed the request wouldn't lead to cacheable content, so we knew the response would be uncatchable before even contacting the backend and the response time is also somewhat unexpected, but is not > clearly logged (only a Varnish expert like you can decrypt that it is 27 > seconds). To be fair, varnishlog's goal is just to provide all the info it can, in an unopinionated matter. The fact that the response took a long time may or may not be normal, so there's no reason for varnishlog to fret about it, and it doesn't necessarily know either what duration you are interested in, so it gives them all) For anyone reading along and trying to make sense of the Timestamp lines: https://varnish-cache.org/docs/6.0/reference/vsl.html#timestamps Note that varnisncsa would have probably been more concise and maybe useful to check the timing. It is also unclear if Varnish is always receiving those > responses from the backend. > As a rule of thumb, by default, varnish only generates 503s in case of an error (your VCL can also generate other errors, but then you are expected to know about that). A quick way to tag the transport-level error that varnish will generate on the backend side it to have this in your VCL: ``` vcl sub vcl_backend_error { set beresp.http.is-a-varnish-error = "true"; } > > I also expected some other URLs in the log, but don't see them. > You could maybe log more, on disk, and filter for the urls you care about? If that's not what you are already doing? This page might help: https://docs.varnish-software.com/tutorials/vsl-query/ Hope that helps! > -------------- next part -------------- An HTML attachment was scrubbed... URL: From uday.polu at indiamart.com Tue May 30 10:43:33 2023 From: uday.polu at indiamart.com (Uday Kumar) Date: Tue, 30 May 2023 16:13:33 +0530 Subject: Need Assistance in Configuring Varnish to Retain and Ignore Unique Parameter in Request URL while caching Message-ID: Hello everyone, In our system, we're currently using Varnish Cache in front of our Tomcat Server for caching content. As part of our new requirement, we've started passing a unique parameter with every URL. The addition of this unique parameter in each request is causing a cache miss, as Varnish treats each request as distinct due to the difference in the parameter. Our intent is to have Varnish ignore this specific parameter for caching purposes, so that it can treat similar requests with different unique parameters as identical for caching purposes. Expected Functionality of Varnish: 1. We need Varnish to ignore the unique parameter when determining if a request is in the cache or while caching a request. 2. We also need Varnish to retain this unique parameter in the request URL when it's passed along to the Tomcat Server. We're looking for a way to modify our Varnish configuration to address the above issues, your assistance would be greatly appreciated. Thanks & Regards Uday Kumar -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume.quintard at gmail.com Tue May 30 14:49:13 2023 From: guillaume.quintard at gmail.com (Guillaume Quintard) Date: Tue, 30 May 2023 07:49:13 -0700 Subject: Need Assistance in Configuring Varnish to Retain and Ignore Unique Parameter in Request URL while caching In-Reply-To: References: Message-ID: Hi Uday, Ultimately, you'll probably want to learn and use this vmod: https://github.com/Dridi/libvmod-querystring , but in the meantime, we can use a quick hack. Essentially, we don't need to modify the URL, but we can just alter the cache key computation. By default, the key logic looks like this: sub vcl_hash { hash_data(req.url); if (req.http.host) { hash_data(req.http.host); } else { hash_data(server.ip); } return (lookup); } (It's done by default if you don't have a vcl_hash section in your VCL) We can tweak it slightly so that we ignore the whole querystring: sub vcl_hash { hash_data(regsub(req.url, "\?.*","")); if (req.http.host) { hash_data(req.http.host); } else { hash_data(server.ip); } return (lookup); } It's crude, but should do the job. To use it, just copy the code above in your VCL, for example just after the vcl_recv definition (not inside it though). Of course, if you already have a vlc_hash definition in your code, you'll need to modify that one, instead of adding a new one. Relevant documentation: - https://varnish-cache.org/docs/trunk/users-guide/vcl-hashing.html - https://www.varnish-software.com/developers/tutorials/varnish-builtin-vcl/ Last note: it would probably be better if the tomcat server didn't need that unique parameter, or at the very least, if Varnish could just add it itself rather than relying on client information as you're caching something public using something that was user-specific, so there's potential for snafus here. Hope that helps, On Tue, May 30, 2023, 03:45 Uday Kumar wrote: > Hello everyone, > > In our system, we're currently using Varnish Cache in front of our Tomcat > Server for caching content. > > As part of our new requirement, we've started passing a unique parameter > with every URL. The addition of this unique parameter in each request is > causing a cache miss, as Varnish treats each request as distinct due to the > difference in the parameter. Our intent is to have Varnish ignore this > specific parameter for caching purposes, so that it can treat similar > requests with different unique parameters as identical for caching purposes. > > Expected Functionality of Varnish: > > 1. We need Varnish to ignore the unique parameter when determining if a > request is in the cache or while caching a request. > > 2. We also need Varnish to retain this unique parameter in the request > URL when it's passed along to the Tomcat Server. > > We're looking for a way to modify our Varnish configuration to address the > above issues, your assistance would be greatly appreciated. > > > Thanks & Regards > Uday Kumar > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From uday.polu at indiamart.com Wed May 31 06:17:59 2023 From: uday.polu at indiamart.com (Uday Kumar) Date: Wed, 31 May 2023 11:47:59 +0530 Subject: Need Assistance in Configuring Varnish to Retain and Ignore Unique Parameter in Request URL while caching In-Reply-To: References: Message-ID: Hello Guillaume, Thank you so much for your help, will try modifying vcl_hash as suggested! > Last note: it would probably be better if the tomcat server didn't need > that unique parameter, or at the very least, if Varnish could just add it > itself rather than relying on client information as you're caching > something public using something that was user-specific, so there's > potential for snafus here. > Could you please also suggest how to configure Varnish so that Varnish can add Unique Parameter by itself?? -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume.quintard at gmail.com Wed May 31 08:04:19 2023 From: guillaume.quintard at gmail.com (Guillaume Quintard) Date: Wed, 31 May 2023 01:04:19 -0700 Subject: Need Assistance in Configuring Varnish to Retain and Ignore Unique Parameter in Request URL while caching In-Reply-To: References: Message-ID: > Could you please also suggest how to configure Varnish so that Varnish can add Unique Parameter by itself?? We'd need more context, is there any kind of check that tomcat does on this parameter, does it need to have a specific length, or match a regex? If we know that, we can have Varnish check the user request to make sure it's valid, and potentially generate its own parameter. But it all depends on what Tomcat expects from that parameter. -- Guillaume Quintard On Tue, May 30, 2023 at 11:18?PM Uday Kumar wrote: > Hello Guillaume, > > Thank you so much for your help, will try modifying vcl_hash as suggested! > > >> Last note: it would probably be better if the tomcat server didn't need >> that unique parameter, or at the very least, if Varnish could just add >> it itself rather than relying on client information as you're caching >> something public using something that was user-specific, so there's >> potential for snafus here. >> > > Could you please also suggest how to configure Varnish so that Varnish > can add Unique Parameter by itself?? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From uday.polu at indiamart.com Wed May 31 18:06:58 2023 From: uday.polu at indiamart.com (Uday Kumar) Date: Wed, 31 May 2023 23:36:58 +0530 Subject: Need Assistance in Configuring Varnish to Retain and Ignore Unique Parameter in Request URL while caching In-Reply-To: References: Message-ID: Hello, We would like to configure varnish to create unique parameter such that its value should be of 20 characters (alphanumeric characters that are URL safe). On Wed, May 31, 2023, 13:34 Guillaume Quintard wrote: > > Could you please also suggest how to configure Varnish so that Varnish > can add Unique Parameter by itself?? > > We'd need more context, is there any kind of check that tomcat does on > this parameter, does it need to have a specific length, or match a regex? > If we know that, we can have Varnish check the user request to make sure > it's valid, and potentially generate its own parameter. > > But it all depends on what Tomcat expects from that parameter. > > -- > Guillaume Quintard > > > On Tue, May 30, 2023 at 11:18?PM Uday Kumar > wrote: > >> Hello Guillaume, >> >> Thank you so much for your help, will try modifying vcl_hash as suggested! >> >> >>> Last note: it would probably be better if the tomcat server didn't need >>> that unique parameter, or at the very least, if Varnish could just add >>> it itself rather than relying on client information as you're caching >>> something public using something that was user-specific, so there's >>> potential for snafus here. >>> >> >> Could you please also suggest how to configure Varnish so that Varnish >> can add Unique Parameter by itself?? >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume.quintard at gmail.com Wed May 31 18:38:57 2023 From: guillaume.quintard at gmail.com (Guillaume Quintard) Date: Wed, 31 May 2023 11:38:57 -0700 Subject: Need Assistance in Configuring Varnish to Retain and Ignore Unique Parameter in Request URL while caching In-Reply-To: References: Message-ID: Does it need to be unique? can't we just get away with "aaaaaaaaaaaaaaaaaaaa"? the crude VCL code would look like: set req.url = regsub(req.url, "\?.*","?yourparam=aaaaaaaaaaaaaaaaaaaa"); i.e. getting rid of the whole query string and just putting yours in place. -- Guillaume Quintard On Wed, May 31, 2023 at 11:07?AM Uday Kumar wrote: > Hello, > > We would like to configure varnish to create unique parameter such that > its value should be of 20 characters (alphanumeric characters that are URL > safe). > > On Wed, May 31, 2023, 13:34 Guillaume Quintard < > guillaume.quintard at gmail.com> wrote: > >> > Could you please also suggest how to configure Varnish so that Varnish >> can add Unique Parameter by itself?? >> >> We'd need more context, is there any kind of check that tomcat does on >> this parameter, does it need to have a specific length, or match a regex? >> If we know that, we can have Varnish check the user request to make sure >> it's valid, and potentially generate its own parameter. >> >> But it all depends on what Tomcat expects from that parameter. >> >> -- >> Guillaume Quintard >> >> >> On Tue, May 30, 2023 at 11:18?PM Uday Kumar >> wrote: >> >>> Hello Guillaume, >>> >>> Thank you so much for your help, will try modifying vcl_hash as >>> suggested! >>> >>> >>>> Last note: it would probably be better if the tomcat server didn't need >>>> that unique parameter, or at the very least, if Varnish could just add >>>> it itself rather than relying on client information as you're caching >>>> something public using something that was user-specific, so there's >>>> potential for snafus here. >>>> >>> >>> Could you please also suggest how to configure Varnish so that Varnish >>> can add Unique Parameter by itself?? >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From geoff at uplex.de Wed May 31 19:15:58 2023 From: geoff at uplex.de (Geoff Simmons) Date: Wed, 31 May 2023 21:15:58 +0200 Subject: Need Assistance in Configuring Varnish to Retain and Ignore Unique Parameter in Request URL while caching In-Reply-To: References: Message-ID: On 5/31/23 20:06, Uday Kumar wrote: > > We would like to configure varnish to create unique parameter such that > its value should be of 20 characters (alphanumeric characters that are > URL safe). I have used VMOD uuid to set unique values in each request: https://github.com/otto-de/libvmod-uuid The standard UUID string is 36 characters long. If you really have to have exactly 20 characters, you could take the last 3 blocks of hex characters, removing the hyphens. It would take a bit of regex slinging. For example from: 6ba7b810-9dad-11d1-80b4-00c04fd430c8 extract: 11d180b400c04fd430c8 Then you'd have 80 bits represented in 20 hex characters, instead of the 128 bits represented by the standard string. The 80 bits are all obtained from the random number generator (the urandom device on Linux), so I suppose that would be enough to ensure uniqueness. Best, Geoff -- ** * * UPLEX - Nils Goroll Systemoptimierung Scheffelstra?e 32 22301 Hamburg Tel +49 40 2880 5731 Mob +49 176 636 90917 Fax +49 40 42949753 http://uplex.de -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature Type: application/pgp-signature Size: 840 bytes Desc: OpenPGP digital signature URL: