From joh.hendriks at gmail.com Tue Sep 27 13:07:41 2022 From: joh.hendriks at gmail.com (Johan Hendriks) Date: Tue, 27 Sep 2022 15:07:41 +0200 Subject: TTL <=0 ? Message-ID: <79f3c738-775f-2bb8-c2b1-0a71b0260539@gmail.com> Hello all, varnish tells me that the TTL is smaller or equal to 0, but looking at the response headers that is not the case as the BerespHeader has? Expires: Wed, 27 Sep 2023 12:23:11 GMT which is in 2023! This is the part of the log. -?? Begin????????? bereq 8203147 pass -?? VCL_use??????? boot -?? Timestamp????? Start: 1664281391.802802 0.000000 0.000000 -?? BereqMethod??? GET -?? BereqURL /wp-content/plugins/canvas/assets/fonts/canvas-icons.woff -?? BereqProtocol? HTTP/1.1 -?? BereqHeader??? Host: www.site.nl -?? BereqHeader??? X-Forwarded-Proto: https -?? BereqHeader??? Origin: https://www.site.nl -?? BereqHeader??? Accept: */* -?? BereqHeader??? User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 15_6_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/19G82 [FBAN/FBIOS;FBDV/iPhone12,1;FBMD/iPhone;FBSN/iOS;FBSV/15.6.1;FBSS/2;FBID/phone;FBLC/nl_NL;FBOP/5] -?? BereqHeader??? Accept-Language: nl-NL,nl;q=0.9 -?? BereqHeader??? Referer: https://www.site.nl/url -?? BereqHeader??? Accept-Encoding: gzip, deflate, br -?? BereqHeader??? X-forwarded-for: 123.123.123.1 -?? BereqHeader??? Cookie: _sharedid=redacted; cto_bundle=redacted -?? BereqHeader??? X-Varnish-Nocache-Recv: Got request cookie (_sharedid=redacted; cto_bundle=redacted -?? BereqHeader??? X-Varnish: 8203148 -?? VCL_call?????? BACKEND_FETCH -?? VCL_return???? fetch -?? BackendOpen??? 47 default 127.0.0.1 8080 127.0.0.1 48076 -?? BackendStart?? 127.0.0.1 8080 -?? Timestamp????? Bereq: 1664281391.802852 0.000050 0.000050 -?? Timestamp????? Beresp: 1664281391.802989 0.000186 0.000137 -?? BerespProtocol HTTP/1.1 -?? BerespStatus?? 200 -?? BerespReason?? OK -?? BerespHeader?? Server: nginx -?? BerespHeader?? Date: Tue, 27 Sep 2022 12:23:11 GMT -?? BerespHeader?? Content-Type: application/font-woff -?? BerespHeader?? Content-Length: 4316 -?? BerespHeader?? Last-Modified: Tue, 06 Sep 2022 23:45:48 GMT -?? BerespHeader?? Connection: keep-alive -?? BerespHeader?? ETag: "6317dbac-10dc" -?? BerespHeader?? Expires: Wed, 27 Sep 2023 12:23:11 GMT -?? BerespHeader?? Cache-Control: max-age=31536000 -?? BerespHeader?? Accept-Ranges: bytes -?? VCL_call?????? BACKEND_RESPONSE -?? BerespHeader?? X-Req-Host: www.site.nl -?? BerespHeader?? X-Req-URL: /wp-content/plugins/canvas/assets/fonts/canvas-icons.woff -?? BerespHeader?? X-Req-URL-Base: /wp-content/plugins/canvas/assets/fonts/canvas-icons.woff -?? BerespHeader?? X-Varnish-Nocache-Beresp: TTL <= 0s -?? TTL??????????? VCL 120 0 0 1664281392 uncacheable -?? TTL??????????? VCL 120 0 0 1664281392 uncacheable -?? BerespHeader?? X-Varnish-TTL: 120.000 -?? BerespHeader?? X-Varnish-Backend: default -?? VCL_return???? deliver The config from the vcl backend_responce is as follows, this sets the X-Varnish-Nocache-Beresp header. sub vcl_backend_response { ??? if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Surrogate-control ~ "no-store" || ????? (!beresp.http.Surrogate-Control && beresp.http.Cache-Control ~ "no-cache|no-store|private") || ????? beresp.http.Vary == "*") { ??????? # DEBUG INFORMATION - DOES NOT INFLUENCE CACHING # ??????? if (beresp.ttl <= 0s) { ??????????? if (beresp.http.X-Varnish-Nocache-Beresp) { ??????????????? set beresp.http.X-Varnish-Nocache-Beresp = beresp.http.X-Varnish-Nocache-Beresp + ", TTL <= 0s"; ??????????? } ??????????? else { ??????????????? set beresp.http.X-Varnish-Nocache-Beresp = "TTL <= 0s"; ??????????? } ??????? } ??????? if(beresp.http.Set-Cookie) { ??????????? if (beresp.http.X-Varnish-Nocache-Beresp) { ??????????????? set beresp.http.X-Varnish-Nocache-Beresp = beresp.http.X-Varnish-Nocache-Beresp + ", got response cookie (" + beresp.http.Set-Cookie + ")" ; ??????????? } ??????????? else { ??????????????? set beresp.http.X-Varnish-Nocache-Beresp = "Response cookie (" + beresp.http.Set-Cookie + ")"; ??????????? } ??????? } ??????? if (beresp.http.Surrogate-control ~ "no-store" || (!beresp.http.Surrogate-Control && beresp.http.Cache-Control ~ "no-cache|no-store|private")) { ??????????? if (beresp.http.X-Varnish-Nocache-Beresp) { ??????????????? set beresp.http.X-Varnish-Nocache-Beresp = beresp.http.X-Varnish-Nocache-Beresp + ", got no-cache|no-store|private"; ??????????? } ??????????? else { ??????????????? set beresp.http.X-Varnish-Nocache-Beresp = "Got no-cache|no-store|private"; ??????????? } ??????? } ??????? # DEBUG INFORMATION - DOES NOT INFLUENCE CACHING # ??????? /* ??????? * Mark as "Hit-For-Pass" for the next 2 minutes ??????? */ ??????? set beresp.ttl = 120s; ??????? set beresp.uncacheable = true; ??? } Am i right that the TTL is <=0 because it sends a cookie to the backend? Thank you for your time! From geoff at uplex.de Tue Sep 27 13:31:52 2022 From: geoff at uplex.de (Geoff Simmons) Date: Tue, 27 Sep 2022 15:31:52 +0200 Subject: TTL <=0 ? In-Reply-To: <79f3c738-775f-2bb8-c2b1-0a71b0260539@gmail.com> References: <79f3c738-775f-2bb8-c2b1-0a71b0260539@gmail.com> Message-ID: <7cb97d21-9e92-a677-a161-fe67f2861996@uplex.de> On 9/27/22 15:07, Johan Hendriks wrote: > Hello all, varnish tells me that the TTL is smaller or equal to 0, but > looking at the response headers that is not the case as the BerespHeader > has? Expires: Wed, 27 Sep 2023 12:23:11 GMT which is in 2023! > > -?? Begin????????? bereq 8203147 pass The request was set to pass on the client side; that sets bereq.uncacheable=true, which is passed along to the backend side as beresp.uncacheable=true. The Expires response header (and also Cache-Control in your example) might at least tell browser caches that they can cache the response. But Varnish won't cache it. [...] > -?? BereqHeader??? Cookie: _sharedid=redacted; cto_bundle=redacted [...] > Am i right that the TTL is <=0 because it sends a cookie to the backend? If you haven't changed this part of builtin.vcl, then yes: sub vcl_req_cookie { if (req.http.Cookie) { # Risky to cache by default. return (pass); } } If a request/response has a property such as a Cookie header, and a number of other things that suggest that the response may be personalized, then it can't take the chance of caching it by default. That can be one of the worst mistakes you can make with a caching proxy. So if you need to be able to cache despite the presence of cookies, as do many sites these days, you need to write rules for that in VCL. Default VCL has to make the safest choice. Best, Geoff -- ** * * UPLEX - Nils Goroll Systemoptimierung Scheffelstra?e 32 22301 Hamburg Tel +49 40 2880 5731 Mob +49 176 636 90917 Fax +49 40 42949753 http://uplex.de -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature Type: application/pgp-signature Size: 840 bytes Desc: OpenPGP digital signature URL: From guillaume.quintard at gmail.com Tue Sep 27 16:00:51 2022 From: guillaume.quintard at gmail.com (Guillaume Quintard) Date: Tue, 27 Sep 2022 09:00:51 -0700 Subject: TTL <=0 ? In-Reply-To: <7cb97d21-9e92-a677-a161-fe67f2861996@uplex.de> References: <79f3c738-775f-2bb8-c2b1-0a71b0260539@gmail.com> <7cb97d21-9e92-a677-a161-fe67f2861996@uplex.de> Message-ID: One minor addition to Geoff's excellent answer: you may want to try this VCL: https://docs.varnish-software.com/tutorials/hit-miss-logging/ Specially at the beginning it helps knowing what happened to the request. -- Guillaume Quintard On Tue, Sep 27, 2022 at 6:33 AM Geoff Simmons wrote: > On 9/27/22 15:07, Johan Hendriks wrote: > > Hello all, varnish tells me that the TTL is smaller or equal to 0, but > > looking at the response headers that is not the case as the BerespHeader > > has Expires: Wed, 27 Sep 2023 12:23:11 GMT which is in 2023! > > > > - Begin bereq 8203147 pass > > The request was set to pass on the client side; that sets > bereq.uncacheable=true, which is passed along to the backend side as > beresp.uncacheable=true. > > The Expires response header (and also Cache-Control in your example) > might at least tell browser caches that they can cache the response. But > Varnish won't cache it. > > [...] > > - BereqHeader Cookie: _sharedid=redacted; cto_bundle=redacted > [...] > > > Am i right that the TTL is <=0 because it sends a cookie to the backend? > > If you haven't changed this part of builtin.vcl, then yes: > > sub vcl_req_cookie { > if (req.http.Cookie) { > # Risky to cache by default. > return (pass); > } > } > > If a request/response has a property such as a Cookie header, and a > number of other things that suggest that the response may be > personalized, then it can't take the chance of caching it by default. > That can be one of the worst mistakes you can make with a caching proxy. > > So if you need to be able to cache despite the presence of cookies, as > do many sites these days, you need to write rules for that in VCL. > Default VCL has to make the safest choice. > > > Best, > Geoff > -- > ** * * UPLEX - Nils Goroll Systemoptimierung > > Scheffelstra?e 32 > 22301 Hamburg > > Tel +49 40 2880 5731 > Mob +49 176 636 90917 > Fax +49 40 42949753 > > http://uplex.de > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From joh.hendriks at gmail.com Fri Sep 30 12:45:36 2022 From: joh.hendriks at gmail.com (Johan Hendriks) Date: Fri, 30 Sep 2022 14:45:36 +0200 Subject: TTL <=0 ? In-Reply-To: References: <79f3c738-775f-2bb8-c2b1-0a71b0260539@gmail.com> <7cb97d21-9e92-a677-a161-fe67f2861996@uplex.de> Message-ID: <10c334ca-31ac-6670-71b2-558cfcdb3402@gmail.com> On 27/09/2022 18:00, Guillaume Quintard wrote: > One minor addition to Geoff's excellent answer: you may want to try > this VCL: https://docs.varnish-software.com/tutorials/hit-miss-logging/ > Specially at the beginning it helps knowing what happened to the request. > > -- > Guillaume Quintard > > > On Tue, Sep 27, 2022 at 6:33 AM Geoff Simmons wrote: > > On 9/27/22 15:07, Johan Hendriks wrote: > > Hello all, varnish tells me that the TTL is smaller or equal to > 0, but > > looking at the response headers that is not the case as the > BerespHeader > > has? Expires: Wed, 27 Sep 2023 12:23:11 GMT which is in 2023! > > > > -?? Begin????????? bereq 8203147 pass > > The request was set to pass on the client side; that sets > bereq.uncacheable=true, which is passed along to the backend side as > beresp.uncacheable=true. > > The Expires response header (and also Cache-Control in your example) > might at least tell browser caches that they can cache the > response. But > Varnish won't cache it. > > [...] > > -?? BereqHeader??? Cookie: _sharedid=redacted; cto_bundle=redacted > [...] > > > Am i right that the TTL is <=0 because it sends a cookie to the > backend? > > If you haven't changed this part of builtin.vcl, then yes: > > sub vcl_req_cookie { > ? ? ? ? if (req.http.Cookie) { > ? ? ? ? ? ? ? ? # Risky to cache by default. > ? ? ? ? ? ? ? ? return (pass); > ? ? ? ? } > } > > If a request/response has a property such as a Cookie header, and a > number of other things that suggest that the response may be > personalized, then it can't take the chance of caching it by default. > That can be one of the worst mistakes you can make with a caching > proxy. > > So if you need to be able to cache despite the presence of > cookies, as > do many sites these days, you need to write rules for that in VCL. > Default VCL has to make the safest choice. > > > Best, > Geoff > -- > ** * * UPLEX - Nils Goroll Systemoptimierung > > Scheffelstra?e 32 > 22301 Hamburg > > Tel +49 40 2880 5731 > Mob +49 176 636 90917 > Fax +49 40 42949753 > > http://uplex.de > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc Thank you all, that explains a lot. regards Johan -------------- next part -------------- An HTML attachment was scrubbed... URL: