From dridi at varni.sh Tue Apr 3 08:38:57 2018 From: dridi at varni.sh (Dridi Boukelmoune) Date: Tue, 3 Apr 2018 10:38:57 +0200 Subject: Docs : Only handle actual PURGE HTTP methods, everything else is discarded In-Reply-To: <001a01d3c839$64cc5480$2e64fd80$@solutions> References: <001a01d3c839$64cc5480$2e64fd80$@solutions> Message-ID: On Fri, Mar 30, 2018 at 5:11 PM, AWA SOLUTIONS wrote: > > Hi there, > > While reading the docs, I found this in https://www.varnish-software.com/wiki/content/tutorials/varnish/sample_vclTemplate.html : > > HANDLING HTTP PURGE > > sub vcl_purge { > # Only handle actual PURGE HTTP methods, everything else is discarded > if (req.method != "PURGE") { > # restart request > set req.http.X-Purge = "Yes"; > return(restart); > } > } > > But in mattiasgeniar/varnish-4.0-configuration-templates https://github.com/mattiasgeniar/varnish-4.0-configuration-templates/blob/master/default.vcl > > This is just the opposite logic : > > sub vcl_purge { > # Only handle actual PURGE HTTP methods, everything else is discarded > if (req.method == "PURGE") { > # restart request > set req.http.X-Purge = "Yes"; > return (restart); > } > } > > So I assume the latter, wiki page, has a typo ?! I think the wiki is meant to feed from the community and give a single location to find all sorts of tips. If Mattias is not credited we have a problem, but I'm personally not into community tutorials as I way too often (as in almost always) see irrelevant or harmful examples or habits from the Varnish 2 or 3 days that have become irrelevant today but refuse to die (yes, I'm looking at you, deflate=). Doing a git blame on the if statement shows that it was a typo in Mattias' repo probably inherited as-is in the wiki, changed 3 weeks ago: https://github.com/mattiasgeniar/varnish-4.0-configuration-templates/commit/18efd041a9f5c27b31cd89396ef6ea649acc0b70 > Last, in https://book.varnish-software.com/4.0/chapters/Cache_Invalidation.html , after similar vcl_recv conditions than above to return purge, vcl_purge subroutine sets req.method instead of req.http.X-Purge : > > sub vcl_purge { > set req.method = "GET"; > return (restart); > } > > Sounds logical to change req.method to avoid infinite loops, so why isn?t it the same in above examples ? Full disclosure as I reveal my trainer hat, yes that's the way to go, assuming you only cache responses to GET requests. My question is rather "why isn't it documented upstream?" https://varnish-cache.org/docs/trunk/users-guide/purging.html We know our docs lack in certain areas. They aren't bad, and they actually cover a lot, but they don't offer the best experience (and we plan to at least try to improve that). It's only my guess but I'm sure we'd all be happy to review and vet pull requests around documentation. At least I'd be happy to review "now that learned this the hard way I wish it was documented here" patches. Cheers From t.winkelmann at ffh.de Wed Apr 4 14:29:21 2018 From: t.winkelmann at ffh.de (Winkelmann, Thomas (RADIO TELE FFH - Online)) Date: Wed, 4 Apr 2018 16:29:21 +0200 Subject: How to install Varnish 6.0 on Ubuntu 17.10 or 18.04 Message-ID: <8A76B2DB46634944BEEBB218DAE22BC2654D51EC18@exchange-srv2.office.radioteleffh.de> Hello, I want to test http/2 of Varnish 6.0 on Ubuntu. So far I managed to install the varnish packages from package cloud as described here: https://packagecloud.io/varnishcache/varnish60/install#manual-deb Also varnish is starting but it is crashing as soon as I'm using my existing config. It will include the vsthrottle VMOD which seems not to be installed on the system. How can I get the Vmods for Ubuntu? I tried to download it from here https://github.com/varnish/varnish-modules but the compilation failed with thousands of errors. Is there any documentation? Thanks, Thomas ________________________________ RADIO / TELE FFH GmbH & Co. Betriebs-KG FFH-Platz 1, 61111 Bad Vilbel HRA - Nr. 26092 Frankfurt/Main USt.IdNr. DE 112152620 Gesch?ftsf?hrer / Programmdirektor: Hans-Dieter Hillmoth -------------- next part -------------- An HTML attachment was scrubbed... URL: From dridi at varni.sh Wed Apr 4 15:21:07 2018 From: dridi at varni.sh (Dridi Boukelmoune) Date: Wed, 4 Apr 2018 17:21:07 +0200 Subject: How to install Varnish 6.0 on Ubuntu 17.10 or 18.04 In-Reply-To: <8A76B2DB46634944BEEBB218DAE22BC2654D51EC18@exchange-srv2.office.radioteleffh.de> References: <8A76B2DB46634944BEEBB218DAE22BC2654D51EC18@exchange-srv2.office.radioteleffh.de> Message-ID: On Wed, Apr 4, 2018 at 4:29 PM, Winkelmann, Thomas (RADIO TELE FFH - Online) wrote: > Hello, > > I want to test http/2 of Varnish 6.0 on Ubuntu. So far I managed to install > the varnish packages from package cloud as described here: > > https://packagecloud.io/varnishcache/varnish60/install#manual-deb > > Also varnish is starting but it is crashing as soon as I?m using my existing > config. It will include the vsthrottle VMOD which seems not to be installed > on the system. Careful with words like "crashing", you'll skyrocket our blood pressure :) > How can I get the Vmods for Ubuntu? I tried to download it from here > https://github.com/varnish/varnish-modules but the compilation failed with > thousands of errors. > > Is there any documentation? We have yet to produce a release compatible with Varnish 6... Do you see these errors when building from the master branch? If so, and if you have a github account, please open an issue. If you don't have a github account please share compilation errors from the master branch. Cheers From t.winkelmann at ffh.de Thu Apr 5 06:20:28 2018 From: t.winkelmann at ffh.de (Winkelmann, Thomas (RADIO TELE FFH - Online)) Date: Thu, 5 Apr 2018 08:20:28 +0200 Subject: AW: How to install Varnish 6.0 on Ubuntu 17.10 or 18.04 In-Reply-To: References: <8A76B2DB46634944BEEBB218DAE22BC2654D51EC18@exchange-srv2.office.radioteleffh.de> Message-ID: <8A76B2DB46634944BEEBB218DAE22BC2654D51EC1F@exchange-srv2.office.radioteleffh.de> Thank you very much for the response... I'm replacing "crashing" with "not starting" :) This is no surprise as it can not load the Vmod... What are the dependencies for successfully building the vmods on Ubuntu 17.10? Is it only the "varnish-dev" package out of the packagecloud repository? Or do I need anything else? Sorry for the questions, we are more webdevelopers than linux developers. Are there any plans to provide a varnish-modules package for 6.0? -----Urspr?ngliche Nachricht----- Von: Dridi Boukelmoune [mailto:dridi at varni.sh] Gesendet: Mittwoch, 4. April 2018 17:21 An: Winkelmann, Thomas (RADIO TELE FFH - Online) Cc: varnish-misc at varnish-cache.org Betreff: Re: How to install Varnish 6.0 on Ubuntu 17.10 or 18.04 On Wed, Apr 4, 2018 at 4:29 PM, Winkelmann, Thomas (RADIO TELE FFH - Online) wrote: > Hello, > > I want to test http/2 of Varnish 6.0 on Ubuntu. So far I managed to > install the varnish packages from package cloud as described here: > > https://packagecloud.io/varnishcache/varnish60/install#manual-deb > > Also varnish is starting but it is crashing as soon as I?m using my > existing config. It will include the vsthrottle VMOD which seems not > to be installed on the system. Careful with words like "crashing", you'll skyrocket our blood pressure :) > How can I get the Vmods for Ubuntu? I tried to download it from here > https://github.com/varnish/varnish-modules but the compilation failed > with thousands of errors. > > Is there any documentation? We have yet to produce a release compatible with Varnish 6... Do you see these errors when building from the master branch? If so, and if you have a github account, please open an issue. If you don't have a github account please share compilation errors from the master branch. Cheers RADIO / TELE FFH GmbH & Co. Betriebs-KG FFH-Platz 1, 61111 Bad Vilbel HRA - Nr. 26092 Frankfurt/Main USt.IdNr. DE 112152620 Gesch?ftsf?hrer / Programmdirektor: Hans-Dieter Hillmoth From dridi at varni.sh Thu Apr 5 14:32:26 2018 From: dridi at varni.sh (Dridi Boukelmoune) Date: Thu, 5 Apr 2018 16:32:26 +0200 Subject: How to install Varnish 6.0 on Ubuntu 17.10 or 18.04 In-Reply-To: <8A76B2DB46634944BEEBB218DAE22BC2654D51EC1F@exchange-srv2.office.radioteleffh.de> References: <8A76B2DB46634944BEEBB218DAE22BC2654D51EC18@exchange-srv2.office.radioteleffh.de> <8A76B2DB46634944BEEBB218DAE22BC2654D51EC1F@exchange-srv2.office.radioteleffh.de> Message-ID: > What are the dependencies for successfully building the vmods on Ubuntu 17.10? Is it only the "varnish-dev" package out of the packagecloud repository? Or do I need anything else? I'm more a Fedora person myself so I may get the package names wrong, but you need pkg-config, python >= 2.7 and python-docutils. When building from git you also need the autotools (autoconf automake libtool) that may be part of build-essential. > Sorry for the questions, we are more webdevelopers than linux developers. Are there any plans to provide a varnish-modules package for 6.0? We provide "dist" releases (autotools-independent source archives) and we should get one soon. Regarding packages, we rely on downstream distributors for this one. Dridi From ciapnz at gmail.com Sat Apr 7 10:10:00 2018 From: ciapnz at gmail.com (Danila Vershinin) Date: Sat, 7 Apr 2018 13:10:00 +0300 Subject: Softpurge and nearly immediate refresh? Message-ID: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com> Hi, What I work with: * Grace mode configured to be 60 seconds when backend is healthy * Using softpurge module to adjust TTL to 0 upon PURGE. The whole idea is increasing chances that visitors will get cached page after cache was PURGEd for a page. Standard piece: sub vcl_hit { if (obj.ttl >= 0s) { # normal hit return (deliver); } if (std.healthy(req.backend_hint)) { # Backend is healthy. Limit age to 60s. if (obj.ttl + 60s > 0s) { set req.http.grace = "normal(limited)"; return (deliver); } else { return(fetch); } } else { # ... } } And use of softpurge: sub vcl_miss { if (req.method == "PURGE") { softpurge.softpurge(); return (synth(200, "Successful softpurge")); } } sub vcl_hit { if (req.method == "PURGE") { softpurge.softpurge(); return (synth(200, "Successful softpurge")); } } Current behaviour: * send PURGE for cached page * Visitor goes to the page within 60 seconds and sees a stale cached page (triggering background refresh) * Further visits to the page will show refreshed page What I?m looking for: Trigger the background refresh right after PURGE while still leveraging grace mode :) That is, serve stale cache for only as long as it takes to actually generate the new page, and not wait for 60 seconds: * upon PURGE: set TTL to 0 (softpurge) + trigger background page request (possible?) * serve stale cache only while the page is generated I could have adjusted the ?healthy backend grace period? to lower than 60s, but I?m basically checking to see if it?s possible to refresh ?nearly? immediately in this kind of setup. Hope I made any sense :) Best Regards, Danila -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From mghhimu at gmail.com Mon Apr 9 09:12:07 2018 From: mghhimu at gmail.com (Md. Golam Hossain) Date: Mon, 09 Apr 2018 09:12:07 +0000 Subject: Setting a variable/header in vcl_recv and checking it in other sub routines Message-ID: If I want to make some decision by checking some header(eg: req.http.test) in vcl_backend_response being set in vcl_recv but not sent as http header while fetching from backend or while sending response to client, how should I do this? Let's say, I want to store all the objects that is set to 'true' in req.http.test header in vcl_recv and check it in vcl_backend_beresponse to store that object in "test.storage". But, as I am unsetting the header(bereq.http.test) in vcl_backend_fetch, it can't be used in vcl_backend_response while setting storage hint. Any suggestion? Thanks in advance. On Mon, Apr 9, 2018, 11:00 Md. Golam Hossain wrote: > If I want to make some decision by checking some header(eg: > req.http.test) in vcl_backend_response being set in vcl_recv but not > sent as http header while fetching from backend or while sending > response to client, how should I do this? > > Let's say, I want to store all the objects that is set to 'true' in > req.http.test header in vcl_recv and check it in vcl_backend_beresponse > to store that object in "test.storage". But, as I am unsetting the > header(bereq.http.test) in vcl_backend_fetch, it can't be used in > vcl_backend_response while setting storage hint. > > Any suggestion? > Thanks in advance. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Mon Apr 9 09:16:49 2018 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 9 Apr 2018 11:16:49 +0200 Subject: Setting a variable/header in vcl_recv and checking it in other sub routines In-Reply-To: References: Message-ID: Hi, Use vmod_var, from varnish-modules. You'll store variables without setting headers. -- Guillaume Quintard On Mon, Apr 9, 2018 at 11:12 AM, Md. Golam Hossain wrote: > If I want to make some decision by checking some header(eg: > req.http.test) in vcl_backend_response being set in vcl_recv but not > sent as http header while fetching from backend or while sending > response to client, how should I do this? > > Let's say, I want to store all the objects that is set to 'true' in > req.http.test header in vcl_recv and check it in vcl_backend_beresponse > to store that object in "test.storage". But, as I am unsetting the > header(bereq.http.test) in vcl_backend_fetch, it can't be used in > vcl_backend_response while setting storage hint. > > Any suggestion? > Thanks in advance. > > > On Mon, Apr 9, 2018, 11:00 Md. Golam Hossain wrote: > >> If I want to make some decision by checking some header(eg: >> req.http.test) in vcl_backend_response being set in vcl_recv but not >> sent as http header while fetching from backend or while sending >> response to client, how should I do this? >> >> Let's say, I want to store all the objects that is set to 'true' in >> req.http.test header in vcl_recv and check it in vcl_backend_beresponse >> to store that object in "test.storage". But, as I am unsetting the >> header(bereq.http.test) in vcl_backend_fetch, it can't be used in >> vcl_backend_response while setting storage hint. >> >> Any suggestion? >> Thanks in advance. >> >> > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Mon Apr 9 09:18:45 2018 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 9 Apr 2018 11:18:45 +0200 Subject: Softpurge and nearly immediate refresh? In-Reply-To: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com> References: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com> Message-ID: Hi, You can purge then set the method to GET then restart. Would that work for you? Other way is to use req.hash_always_miss that will only revalidate if we are able to fetch a new object. -- Guillaume Quintard On Sat, Apr 7, 2018 at 12:10 PM, Danila Vershinin wrote: > Hi, > > What I work with: > > * Grace mode configured to be 60 seconds when backend is healthy > * Using softpurge module to adjust TTL to 0 upon PURGE. > > The whole idea is increasing chances that visitors will get cached page > after cache was PURGEd for a page. > > Standard piece: > > sub vcl_hit { > if (obj.ttl >= 0s) { > # normal hit > return (deliver); > } > > if (std.healthy(req.backend_hint)) { > # Backend is healthy. Limit age to 60s. > if (obj.ttl + 60s > 0s) { > set req.http.grace = "normal(limited)"; > return (deliver); > } else { > return(fetch); > } > } else { > # ... > } > } > > And use of softpurge: > > sub vcl_miss { > if (req.method == "PURGE") { > softpurge.softpurge(); > return (synth(200, "Successful softpurge")); > } > } > > sub vcl_hit { > if (req.method == "PURGE") { > softpurge.softpurge(); > return (synth(200, "Successful softpurge")); > } > } > > > > Current behaviour: > > * send PURGE for cached page > * Visitor goes to the page within 60 seconds and sees a stale cached page > (triggering background refresh) > * Further visits to the page will show refreshed page > > What I?m looking for: > > Trigger the background refresh right after PURGE while still leveraging > grace mode :) That is, serve stale cache for only as long as it takes to > actually generate the new page, and not wait for 60 seconds: > > * upon PURGE: set TTL to 0 (softpurge) + trigger background page request > (possible?) > * serve stale cache only while the page is generated > > I could have adjusted the ?healthy backend grace period? to lower than > 60s, but I?m basically checking to see if it?s possible to refresh ?nearly? > immediately in this kind of setup. > > Hope I made any sense :) > > Best Regards, > Danila > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ciapnz at gmail.com Mon Apr 9 14:47:26 2018 From: ciapnz at gmail.com (Danila Vershinin) Date: Mon, 9 Apr 2018 17:47:26 +0300 Subject: Softpurge and nearly immediate refresh? In-Reply-To: References: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com> Message-ID: Hi Guillaume, This should absolutely work. Thank you :) Best Regards, Danila > On 9 Apr 2018, at 12:18, Guillaume Quintard wrote: > > You can purge then set the method to GET then restart. Would that work for you? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From ciapnz at gmail.com Wed Apr 11 09:26:30 2018 From: ciapnz at gmail.com (Danila Vershinin) Date: Wed, 11 Apr 2018 12:26:30 +0300 Subject: Softpurge and nearly immediate refresh? In-Reply-To: References: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com> Message-ID: <15B57C74-5AD8-4A83-AE4E-5431F5FE127C@gmail.com> Hi Guillaume, A bit puzzled on something. If we use Vary: by some header.. am I correct that we need multiple restarts to refresh each object variation? Since the background fetch would only refresh the variation that matched initial purge request. Sent from my iPhone > On 9 Apr 2018, at 12:18, Guillaume Quintard wrote: > > Hi, > > You can purge then set the method to GET then restart. Would that work for you? > > Other way is to use req.hash_always_miss that will only revalidate if we are able to fetch a new object. > > -- > Guillaume Quintard > >> On Sat, Apr 7, 2018 at 12:10 PM, Danila Vershinin wrote: >> Hi, >> >> What I work with: >> >> * Grace mode configured to be 60 seconds when backend is healthy >> * Using softpurge module to adjust TTL to 0 upon PURGE. >> >> The whole idea is increasing chances that visitors will get cached page after cache was PURGEd for a page. >> >> Standard piece: >> sub vcl_hit { >> if (obj.ttl >= 0s) { >> # normal hit >> return (deliver); >> } >> >> if (std.healthy(req.backend_hint)) { >> # Backend is healthy. Limit age to 60s. >> if (obj.ttl + 60s > 0s) { >> set req.http.grace = "normal(limited)"; >> return (deliver); >> } else { >> return(fetch); >> } >> } else { >> # ... >> } >> } >> And use of softpurge: >> >> sub vcl_miss { >> if (req.method == "PURGE") { >> softpurge.softpurge(); >> return (synth(200, "Successful softpurge")); >> } >> } >> >> sub vcl_hit { >> if (req.method == "PURGE") { >> softpurge.softpurge(); >> return (synth(200, "Successful softpurge")); >> } >> } >> >> >> Current behaviour: >> >> * send PURGE for cached page >> * Visitor goes to the page within 60 seconds and sees a stale cached page (triggering background refresh) >> * Further visits to the page will show refreshed page >> >> What I?m looking for: >> >> Trigger the background refresh right after PURGE while still leveraging grace mode :) That is, serve stale cache for only as long as it takes to actually generate the new page, and not wait for 60 seconds: >> >> * upon PURGE: set TTL to 0 (softpurge) + trigger background page request (possible?) >> * serve stale cache only while the page is generated >> >> I could have adjusted the ?healthy backend grace period? to lower than 60s, but I?m basically checking to see if it?s possible to refresh ?nearly? immediately in this kind of setup. >> >> Hope I made any sense :) >> >> Best Regards, >> Danila >> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Wed Apr 11 09:37:23 2018 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 11 Apr 2018 11:37:23 +0200 Subject: Softpurge and nearly immediate refresh? In-Reply-To: <15B57C74-5AD8-4A83-AE4E-5431F5FE127C@gmail.com> References: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com> <15B57C74-5AD8-4A83-AE4E-5431F5FE127C@gmail.com> Message-ID: Hi, That's indeed correct, a purge will kill all variations, and the restart only fetches one. The req.hash_always_miss trick however only kills/revalidate one variation. At this moment, we have no way to purge/revalidate all the object under one hash key. -- Guillaume Quintard On Wed, Apr 11, 2018 at 11:26 AM, Danila Vershinin wrote: > Hi Guillaume, > > A bit puzzled on something. If we use Vary: by some header.. am I correct > that we need multiple restarts to refresh each object variation? > > Since the background fetch would only refresh the variation that matched > initial purge request. > > Sent from my iPhone > > On 9 Apr 2018, at 12:18, Guillaume Quintard com> wrote: > > Hi, > > You can purge then set the method to GET then restart. Would that work for > you? > > Other way is to use req.hash_always_miss that will only revalidate if we > are able to fetch a new object. > > -- > Guillaume Quintard > > On Sat, Apr 7, 2018 at 12:10 PM, Danila Vershinin > wrote: > >> Hi, >> >> What I work with: >> >> * Grace mode configured to be 60 seconds when backend is healthy >> * Using softpurge module to adjust TTL to 0 upon PURGE. >> >> The whole idea is increasing chances that visitors will get cached page >> after cache was PURGEd for a page. >> >> Standard piece: >> >> sub vcl_hit { >> if (obj.ttl >= 0s) { >> # normal hit >> return (deliver); >> } >> >> if (std.healthy(req.backend_hint)) { >> # Backend is healthy. Limit age to 60s. >> if (obj.ttl + 60s > 0s) { >> set req.http.grace = "normal(limited)"; >> return (deliver); >> } else { >> return(fetch); >> } >> } else { >> # ... >> } >> } >> >> And use of softpurge: >> >> sub vcl_miss { >> if (req.method == "PURGE") { >> softpurge.softpurge(); >> return (synth(200, "Successful softpurge")); >> } >> } >> >> sub vcl_hit { >> if (req.method == "PURGE") { >> softpurge.softpurge(); >> return (synth(200, "Successful softpurge")); >> } >> } >> >> >> >> Current behaviour: >> >> * send PURGE for cached page >> * Visitor goes to the page within 60 seconds and sees a stale cached page >> (triggering background refresh) >> * Further visits to the page will show refreshed page >> >> What I?m looking for: >> >> Trigger the background refresh right after PURGE while still leveraging >> grace mode :) That is, serve stale cache for only as long as it takes to >> actually generate the new page, and not wait for 60 seconds: >> >> * upon PURGE: set TTL to 0 (softpurge) + trigger background page request >> (possible?) >> * serve stale cache only while the page is generated >> >> I could have adjusted the ?healthy backend grace period? to lower than >> 60s, but I?m basically checking to see if it?s possible to refresh ?nearly? >> immediately in this kind of setup. >> >> Hope I made any sense :) >> >> Best Regards, >> Danila >> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cosimo at streppone.it Wed Apr 11 10:47:48 2018 From: cosimo at streppone.it (Cosimo Streppone) Date: Wed, 11 Apr 2018 12:47:48 +0200 Subject: vmod_vsthrottle memory usage Message-ID: <1523443668.3231337.1334128512.596EC649@webmail.messagingengine.com> Hi, I'm looking at the vsthrottle vmod. Have a proof of concept VCL up and running, and it seems to work very nicely (with varnish 4.1.9). Thanks for this work! I'm wondering what vsthrottle's behaviour is when there's no memory available, specifically when tb_alloc() fails. >From what I can understand from the code[1]: b = tb_alloc(digest, limit, period, now); AZ(VRB_INSERT(tbtree, &v->buckets, b)); and tb_alloc() also checks for a non-zero malloc result (AN()). I understand we're talking about a remote possibility, and I see vsthrottle reclaims unused memory. However, when no memory is available for the malloc, will varnish leave a trace in syslog, exit and restart by way of the asserts? -- Cosimo [1] https://github.com/varnish/varnish-modules/blob/master/src/vmod_vsthrottle.c#L126-L127 From guillaume at varnish-software.com Wed Apr 11 12:02:13 2018 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 11 Apr 2018 14:02:13 +0200 Subject: vmod_vsthrottle memory usage In-Reply-To: <1523443668.3231337.1334128512.596EC649@webmail.messagingengine.com> References: <1523443668.3231337.1334128512.596EC649@webmail.messagingengine.com> Message-ID: Hi cosimo, Very simple, we crash and bring the whole thing down: https://github.com/varnish/varnish-modules/blob/master/src/vmod_vsthrottle.c#L99 IIRC, on Linux, malloc always returns with an allocation, or OOM-kills you -- Guillaume Quintard On Wed, Apr 11, 2018 at 12:47 PM, Cosimo Streppone wrote: > Hi, > > I'm looking at the vsthrottle vmod. Have a proof of concept VCL up and > running, > and it seems to work very nicely (with varnish 4.1.9). Thanks for this > work! > > I'm wondering what vsthrottle's behaviour is when there's no memory > available, > specifically when tb_alloc() fails. > > From what I can understand from the code[1]: > > b = tb_alloc(digest, limit, period, now); > AZ(VRB_INSERT(tbtree, &v->buckets, b)); > > and tb_alloc() also checks for a non-zero malloc result (AN()). > > I understand we're talking about a remote possibility, and I see vsthrottle > reclaims unused memory. However, when no memory is available for the > malloc, > will varnish leave a trace in syslog, exit and restart by way of the > asserts? > > -- > Cosimo > > [1] https://github.com/varnish/varnish-modules/blob/master/ > src/vmod_vsthrottle.c#L126-L127 > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dridi at varni.sh Wed Apr 11 12:07:46 2018 From: dridi at varni.sh (Dridi Boukelmoune) Date: Wed, 11 Apr 2018 14:07:46 +0200 Subject: vmod_vsthrottle memory usage In-Reply-To: References: <1523443668.3231337.1334128512.596EC649@webmail.messagingengine.com> Message-ID: > IIRC, on Linux, malloc always returns with an allocation, or OOM-kills you That's the default behavior but I believe you can also change that. From cosimo at streppone.it Wed Apr 11 13:59:22 2018 From: cosimo at streppone.it (Cosimo Streppone) Date: Wed, 11 Apr 2018 15:59:22 +0200 Subject: VCL: returning early from a custom function Message-ID: <1523455162.3350556.1334316448.10C69177@webmail.messagingengine.com> Hi again, two messages in a day after a few years :-) I have code similar to the following (also here[1]): sub vcl_recv { ... call rate_limit; ... } # Throttling based on request inspection sub rate_limit { if (req.url ~ "pattern1") { std.log("pattern1 requests must never be throttled"); return; # <---- Need to return early here, but can't do it } if (req.url ~ "pattern2") { if (vsthrottle.is_denied("pattern2" + client.identity, 100, 10s)) { std.log("pattern2 throttling for ip " + client.identity); return(synth(429, "ETOOMANYREQUESTS")); } } if (vsthrottle.is_denied("ip:" + client.identity, 500, 10s)) { std.log("global throttling for ip " + client.identity); return(synth(429, "ETOOMANYREQUESTS")); } } Ideally, I'd like to return early from rate_limit() when I know that I don't need to enforce any rate limiting for some types of requests, but I understand that's not implemented. I found some alternatives, though generally they feel uglier. Any ideas? -- Cosimo [1] https://gist.github.com/cosimo/6d3318bf173357dfb2652b7b2e81e1e0#file-gistfile1-txt-L18 From dridi at varni.sh Wed Apr 11 14:30:11 2018 From: dridi at varni.sh (Dridi Boukelmoune) Date: Wed, 11 Apr 2018 16:30:11 +0200 Subject: VCL: returning early from a custom function In-Reply-To: <1523455162.3350556.1334316448.10C69177@webmail.messagingengine.com> References: <1523455162.3350556.1334316448.10C69177@webmail.messagingengine.com> Message-ID: > I found some alternatives, though generally they feel uglier. > Any ideas? This has been discussed in the past, including the possibility to return a value, but as of today you can't break out of a subroutine. https://github.com/varnishcache/varnish-cache/wiki/VIP2%3A-VCL-typed-functions With this feature you could break out of a subroutine by turning it into a function that returns VOID. We are not there yet. Dridi From Tom.Anheyer at berlinonline.de Wed Apr 11 14:43:11 2018 From: Tom.Anheyer at berlinonline.de (Anheyer, Tom) Date: Wed, 11 Apr 2018 14:43:11 +0000 Subject: VCL: returning early from a custom function References: <1523455162.3350556.1334316448.10C69177@webmail.messagingengine.com> Message-ID: <8AE07C2AEF8F2D4CA8887FDC94F779B012220B46@klnsmxmbxp005.DMSYS.de> What about: sub rate_limit { if (req.url ~ "pattern1") { std.log("pattern1 requests must never be throttled"); } else { if (req.url ~ "pattern2") { if (vsthrottle.is_denied("pattern2" + client.identity, 100, 10s)) { std.log("pattern2 throttling for ip " + client.identity); return(synth(429, "ETOOMANYREQUESTS")); } } if (vsthrottle.is_denied("ip:" + client.identity, 500, 10s)) { std.log("global throttling for ip " + client.identity); return(synth(429, "ETOOMANYREQUESTS")); } } } tom Am 11.04.2018 um 15:59 schrieb Cosimo Streppone: > Hi again, > > two messages in a day after a few years :-) > > I have code similar to the following (also here[1]): > > sub vcl_recv { > ... > call rate_limit; > ... > } > > # Throttling based on request inspection > sub rate_limit { > > if (req.url ~ "pattern1") { > std.log("pattern1 requests must never be throttled"); > return; # <---- Need to return early here, but can't do it > } > > if (req.url ~ "pattern2") { > if (vsthrottle.is_denied("pattern2" + client.identity, 100, 10s)) { > std.log("pattern2 throttling for ip " + client.identity); > return(synth(429, "ETOOMANYREQUESTS")); > } > } > > if (vsthrottle.is_denied("ip:" + client.identity, 500, 10s)) { > std.log("global throttling for ip " + client.identity); > return(synth(429, "ETOOMANYREQUESTS")); > } > > } > > Ideally, I'd like to return early from rate_limit() when I know that I don't need > to enforce any rate limiting for some types of requests, but I understand > that's not implemented. > > I found some alternatives, though generally they feel uglier. > Any ideas? > From ciapnz at gmail.com Wed Apr 11 20:39:36 2018 From: ciapnz at gmail.com (Danila Vershinin) Date: Wed, 11 Apr 2018 23:39:36 +0300 Subject: Softpurge and nearly immediate refresh? In-Reply-To: References: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com> <15B57C74-5AD8-4A83-AE4E-5431F5FE127C@gmail.com> Message-ID: <6D649EE6-CB1B-43CE-BC27-507FB1E8CCE2@gmail.com> So I guess I?m looking into having one background fetch trigger another background fetch (to sequentially refresh different object variants). In this fashion: 1. Client PURGE request will do softpurge.softpurge(); return(restart); with GET method, etc. which will all lead to "return deliver()" in limited grace logic to fire background fetch . 2. Then in background fetch: ? in its vcl_deliver() - the current object variation has already entered cache, so setting varied header to value2, removing grace limited flag and calling restart(). This way it should continue revalidation for the other object variant ? we land inside limited grace logic again (as it?s a different object variant) and return deliver() again thus firing off second background fetch (which will refresh second object variant). So the standard grace logic + something like this: sub vcl_deliver { if (req.http.grace == "normal(limited)" && req.http.Varied-Header = 'value1') { set req.http.Varied-Header = 'value2?; unset req.http.grace; return (restart); } } However, it won?t work at least because req.http.grace flag will be set for both the background fetch and the request that kicked it off. (it will be there in vcl_deliver of both). Question is how can we tell if we are inside background fetch? Best Regards, Danila > On 11 Apr 2018, at 12:37, Guillaume Quintard wrote: > > Hi, > > That's indeed correct, a purge will kill all variations, and the restart only fetches one. > > The req.hash_always_miss trick however only kills/revalidate one variation. > > At this moment, we have no way to purge/revalidate all the object under one hash key. > > -- > Guillaume Quintard > > On Wed, Apr 11, 2018 at 11:26 AM, Danila Vershinin > wrote: > Hi Guillaume, > > A bit puzzled on something. If we use Vary: by some header.. am I correct that we need multiple restarts to refresh each object variation? > > Since the background fetch would only refresh the variation that matched initial purge request. > > Sent from my iPhone > > On 9 Apr 2018, at 12:18, Guillaume Quintard > wrote: > >> Hi, >> >> You can purge then set the method to GET then restart. Would that work for you? >> >> Other way is to use req.hash_always_miss that will only revalidate if we are able to fetch a new object. >> >> -- >> Guillaume Quintard >> >> On Sat, Apr 7, 2018 at 12:10 PM, Danila Vershinin > wrote: >> Hi, >> >> What I work with: >> >> * Grace mode configured to be 60 seconds when backend is healthy >> * Using softpurge module to adjust TTL to 0 upon PURGE. >> >> The whole idea is increasing chances that visitors will get cached page after cache was PURGEd for a page. >> >> Standard piece: >> sub vcl_hit { >> if (obj.ttl >= 0s) { >> # normal hit >> return (deliver); >> } >> >> if (std.healthy(req.backend_hint)) { >> # Backend is healthy. Limit age to 60s. >> if (obj.ttl + 60s > 0s) { >> set req.http.grace = "normal(limited)"; >> return (deliver); >> } else { >> return(fetch); >> } >> } else { >> # ... >> } >> } >> And use of softpurge: >> >> sub vcl_miss { >> if (req.method == "PURGE") { >> softpurge.softpurge(); >> return (synth(200, "Successful softpurge")); >> } >> } >> >> sub vcl_hit { >> if (req.method == "PURGE") { >> softpurge.softpurge(); >> return (synth(200, "Successful softpurge")); >> } >> } >> >> >> Current behaviour: >> >> * send PURGE for cached page >> * Visitor goes to the page within 60 seconds and sees a stale cached page (triggering background refresh) >> * Further visits to the page will show refreshed page >> >> What I?m looking for: >> >> Trigger the background refresh right after PURGE while still leveraging grace mode :) That is, serve stale cache for only as long as it takes to actually generate the new page, and not wait for 60 seconds: >> >> * upon PURGE: set TTL to 0 (softpurge) + trigger background page request (possible?) >> * serve stale cache only while the page is generated >> >> I could have adjusted the ?healthy backend grace period? to lower than 60s, but I?m basically checking to see if it?s possible to refresh ?nearly? immediately in this kind of setup. >> >> Hope I made any sense :) >> >> Best Regards, >> Danila >> >> >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From ciapnz at gmail.com Thu Apr 12 07:24:42 2018 From: ciapnz at gmail.com (Danila Vershinin) Date: Thu, 12 Apr 2018 10:24:42 +0300 Subject: Softpurge and nearly immediate refresh? In-Reply-To: References: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com> <15B57C74-5AD8-4A83-AE4E-5431F5FE127C@gmail.com> <6D649EE6-CB1B-43CE-BC27-507FB1E8CCE2@gmail.com> Message-ID: Couldn?t really find anything useful besides obj.hits which indirectly tells us it?s a background fetch. So: sub vcl_deliver { if (req.http.grace == "normal(limited)" && obj.hits == 0 && req.http.Varied-Header = 'value1') { set req.http.Varied-Header = 'value2?; unset req.http.grace; return (restart); } } ? :) Best Regards, Danila > On 12 Apr 2018, at 00:24, Guillaume Quintard wrote: > > There's a beresp attribute for that last one (I'm on mobile, so I'm going to point you to man vcl :-)) > > -- > Guillaume Quintard > > On Wed, Apr 11, 2018, 22:39 Danila Vershinin > wrote: > So I guess I?m looking into having one background fetch trigger another background fetch (to sequentially refresh different object variants). In this fashion: > > 1. Client PURGE request will do softpurge.softpurge(); return(restart); with GET method, etc. which will all lead to "return deliver()" in limited grace logic to fire background fetch . > > 2. Then in background fetch: > ? in its vcl_deliver() - the current object variation has already entered cache, so setting varied header to value2, removing grace limited flag and calling restart(). This way it should continue revalidation for the other object variant > ? we land inside limited grace logic again (as it?s a different object variant) and return deliver() again thus firing off second background fetch (which will refresh second object variant). > > So the standard grace logic + something like this: > > sub vcl_deliver { > if (req.http.grace == "normal(limited)" && req.http.Varied-Header = 'value1') { > set req.http.Varied-Header = 'value2?; > unset req.http.grace; > return (restart); > } > } > > However, it won?t work at least because req.http.grace flag will be set for both the background fetch and the request that kicked it off. (it will be there in vcl_deliver of both). > Question is how can we tell if we are inside background fetch? > > Best Regards, > Danila > >> On 11 Apr 2018, at 12:37, Guillaume Quintard > wrote: >> >> Hi, >> >> That's indeed correct, a purge will kill all variations, and the restart only fetches one. >> >> The req.hash_always_miss trick however only kills/revalidate one variation. >> >> At this moment, we have no way to purge/revalidate all the object under one hash key. >> >> -- >> Guillaume Quintard >> >> On Wed, Apr 11, 2018 at 11:26 AM, Danila Vershinin > wrote: >> Hi Guillaume, >> >> A bit puzzled on something. If we use Vary: by some header.. am I correct that we need multiple restarts to refresh each object variation? >> >> Since the background fetch would only refresh the variation that matched initial purge request. >> >> Sent from my iPhone >> >> On 9 Apr 2018, at 12:18, Guillaume Quintard > wrote: >> >>> Hi, >>> >>> You can purge then set the method to GET then restart. Would that work for you? >>> >>> Other way is to use req.hash_always_miss that will only revalidate if we are able to fetch a new object. >>> >>> -- >>> Guillaume Quintard >>> >>> On Sat, Apr 7, 2018 at 12:10 PM, Danila Vershinin > wrote: >>> Hi, >>> >>> What I work with: >>> >>> * Grace mode configured to be 60 seconds when backend is healthy >>> * Using softpurge module to adjust TTL to 0 upon PURGE. >>> >>> The whole idea is increasing chances that visitors will get cached page after cache was PURGEd for a page. >>> >>> Standard piece: >>> sub vcl_hit { >>> if (obj.ttl >= 0s) { >>> # normal hit >>> return (deliver); >>> } >>> >>> if (std.healthy(req.backend_hint)) { >>> # Backend is healthy. Limit age to 60s. >>> if (obj.ttl + 60s > 0s) { >>> set req.http.grace = "normal(limited)"; >>> return (deliver); >>> } else { >>> return(fetch); >>> } >>> } else { >>> # ... >>> } >>> } >>> And use of softpurge: >>> >>> sub vcl_miss { >>> if (req.method == "PURGE") { >>> softpurge.softpurge(); >>> return (synth(200, "Successful softpurge")); >>> } >>> } >>> >>> sub vcl_hit { >>> if (req.method == "PURGE") { >>> softpurge.softpurge(); >>> return (synth(200, "Successful softpurge")); >>> } >>> } >>> >>> >>> Current behaviour: >>> >>> * send PURGE for cached page >>> * Visitor goes to the page within 60 seconds and sees a stale cached page (triggering background refresh) >>> * Further visits to the page will show refreshed page >>> >>> What I?m looking for: >>> >>> Trigger the background refresh right after PURGE while still leveraging grace mode :) That is, serve stale cache for only as long as it takes to actually generate the new page, and not wait for 60 seconds: >>> >>> * upon PURGE: set TTL to 0 (softpurge) + trigger background page request (possible?) >>> * serve stale cache only while the page is generated >>> >>> I could have adjusted the ?healthy backend grace period? to lower than 60s, but I?m basically checking to see if it?s possible to refresh ?nearly? immediately in this kind of setup. >>> >>> Hope I made any sense :) >>> >>> Best Regards, >>> Danila >>> >>> >>> _______________________________________________ >>> varnish-misc mailing list >>> varnish-misc at varnish-cache.org >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From cosimo at streppone.it Thu Apr 12 08:27:34 2018 From: cosimo at streppone.it (Cosimo Streppone) Date: Thu, 12 Apr 2018 10:27:34 +0200 Subject: VCL: returning early from a custom function In-Reply-To: <8AE07C2AEF8F2D4CA8887FDC94F779B012220B46@klnsmxmbxp005.DMSYS.de> References: <1523455162.3350556.1334316448.10C69177@webmail.messagingengine.com> <8AE07C2AEF8F2D4CA8887FDC94F779B012220B46@klnsmxmbxp005.DMSYS.de> Message-ID: <1523521654.1523074.1335350472.039AD34C@webmail.messagingengine.com> On Wed, Apr 11, 2018, at 16:43, Anheyer, Tom wrote: > What about: > > sub rate_limit { > > if (req.url ~ "pattern1") { > std.log("pattern1 requests must never be throttled"); > } > else { > if (req.url ~ "pattern2") { > if (vsthrottle.is_denied("pattern2" + client.identity, > 100, 10s)) { > std.log("pattern2 throttling for ip " + > client.identity); > return(synth(429, "ETOOMANYREQUESTS")); > } > } > > if (vsthrottle.is_denied("ip:" + client.identity, 500, 10s)) { > std.log("global throttling for ip " + client.identity); > return(synth(429, "ETOOMANYREQUESTS")); > } > } > > } Yes, that's one of the alternatives, though the "whitelisted" patterns are more than just one. I ended up doing something like: set req.http.Rate-Limit = "1"; if (whitelisted-pattern1) { set req.http.Rate-Limit = "0"; } if (whitelisted-pattern2) { set req.http.Rate-Limit = "0"; } ... if (req.http.Rate-Limit == "0") { # no rate limiting } else { # yes, do rate limit } I intend to look at vmod_var to see if I can make this prettier. -- Cosimo From guillaume at varnish-software.com Thu Apr 12 09:25:47 2018 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Thu, 12 Apr 2018 09:25:47 +0000 Subject: Softpurge and nearly immediate refresh? In-Reply-To: References: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com> <15B57C74-5AD8-4A83-AE4E-5431F5FE127C@gmail.com> <6D649EE6-CB1B-43CE-BC27-507FB1E8CCE2@gmail.com> Message-ID: bereq.is_bgfetch? -- Guillaume Quintard On Thu, Apr 12, 2018, 09:24 Danila Vershinin wrote: > Couldn?t really find anything useful besides obj.hits which indirectly > tells us it?s a background fetch. So: > > sub vcl_deliver { > if (req.http.grace == "normal(limited)" && *obj.hits == 0* && > req.http.Varied-Header = 'value1') { > set req.http.Varied-Header = 'value2?; > unset req.http.grace; > return (restart); > } > } > > ? :) > > Best Regards, > Danila > > On 12 Apr 2018, at 00:24, Guillaume Quintard < > guillaume at varnish-software.com> wrote: > > There's a beresp attribute for that last one (I'm on mobile, so I'm going > to point you to man vcl :-)) > > -- > Guillaume Quintard > > On Wed, Apr 11, 2018, 22:39 Danila Vershinin wrote: > >> So I guess I?m looking into having one background fetch trigger another >> background fetch (to sequentially refresh different object variants). In >> this fashion: >> >> 1. Client PURGE request will do softpurge.softpurge(); return(restart); >> with GET method, etc. which will all lead to "return deliver()" in limited >> grace logic to fire background fetch . >> >> 2. Then in background fetch: >> ? in its vcl_deliver() - the current object variation has already >> entered cache, so setting varied header to value2, removing grace limited >> flag and calling restart(). This way it should continue revalidation for >> the other object variant >> ? we land inside limited grace logic again (as it?s a different object >> variant) and return deliver() again thus firing off second background fetch >> (which will refresh second object variant). >> >> So the standard grace logic + something like this: >> >> sub vcl_deliver { >> if (req.http.grace == "normal(limited)" && req.http.Varied-Header = >> 'value1') { >> set req.http.Varied-Header = 'value2?; >> unset req.http.grace; >> return (restart); >> } >> } >> >> However, it won?t work at least because req.http.grace flag will be set >> for both the background fetch and the request that kicked it off. (it will >> be there in vcl_deliver of both). >> Question is how can we tell if we are inside background fetch? >> >> Best Regards, >> Danila >> >> On 11 Apr 2018, at 12:37, Guillaume Quintard < >> guillaume at varnish-software.com> wrote: >> >> Hi, >> >> That's indeed correct, a purge will kill all variations, and the restart >> only fetches one. >> >> The req.hash_always_miss trick however only kills/revalidate one >> variation. >> >> At this moment, we have no way to purge/revalidate all the object under >> one hash key. >> >> -- >> Guillaume Quintard >> >> On Wed, Apr 11, 2018 at 11:26 AM, Danila Vershinin >> wrote: >> >>> Hi Guillaume, >>> >>> A bit puzzled on something. If we use Vary: by some header.. am I >>> correct that we need multiple restarts to refresh each object variation? >>> >>> Since the background fetch would only refresh the variation that matched >>> initial purge request. >>> >>> Sent from my iPhone >>> >>> On 9 Apr 2018, at 12:18, Guillaume Quintard < >>> guillaume at varnish-software.com> wrote: >>> >>> Hi, >>> >>> You can purge then set the method to GET then restart. Would that work >>> for you? >>> >>> Other way is to use req.hash_always_miss that will only revalidate if we >>> are able to fetch a new object. >>> >>> -- >>> Guillaume Quintard >>> >>> On Sat, Apr 7, 2018 at 12:10 PM, Danila Vershinin >>> wrote: >>> >>>> Hi, >>>> >>>> What I work with: >>>> >>>> * Grace mode configured to be 60 seconds when backend is healthy >>>> * Using softpurge module to adjust TTL to 0 upon PURGE. >>>> >>>> The whole idea is increasing chances that visitors will get cached page >>>> after cache was PURGEd for a page. >>>> >>>> Standard piece: >>>> >>>> sub vcl_hit { >>>> if (obj.ttl >= 0s) { >>>> # normal hit >>>> return (deliver); >>>> } >>>> >>>> if (std.healthy(req.backend_hint)) { >>>> # Backend is healthy. Limit age to 60s. >>>> if (obj.ttl + 60s > 0s) { >>>> set req.http.grace = "normal(limited)"; >>>> return (deliver); >>>> } else { >>>> return(fetch); >>>> } >>>> } else { >>>> # ... >>>> } >>>> } >>>> >>>> And use of softpurge: >>>> >>>> sub vcl_miss { >>>> if (req.method == "PURGE") { >>>> softpurge.softpurge(); >>>> return (synth(200, "Successful softpurge")); >>>> } >>>> } >>>> >>>> sub vcl_hit { >>>> if (req.method == "PURGE") { >>>> softpurge.softpurge(); >>>> return (synth(200, "Successful softpurge")); >>>> } >>>> } >>>> >>>> >>>> >>>> Current behaviour: >>>> >>>> * send PURGE for cached page >>>> * Visitor goes to the page within 60 seconds and sees a stale cached >>>> page (triggering background refresh) >>>> * Further visits to the page will show refreshed page >>>> >>>> What I?m looking for: >>>> >>>> Trigger the background refresh right after PURGE while still leveraging >>>> grace mode :) That is, serve stale cache for only as long as it takes to >>>> actually generate the new page, and not wait for 60 seconds: >>>> >>>> * upon PURGE: set TTL to 0 (softpurge) + trigger background page >>>> request (possible?) >>>> * serve stale cache only while the page is generated >>>> >>>> I could have adjusted the ?healthy backend grace period? to lower than >>>> 60s, but I?m basically checking to see if it?s possible to refresh ?nearly? >>>> immediately in this kind of setup. >>>> >>>> Hope I made any sense :) >>>> >>>> Best Regards, >>>> Danila >>>> >>>> >>>> _______________________________________________ >>>> varnish-misc mailing list >>>> varnish-misc at varnish-cache.org >>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>>> >>> >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ciapnz at gmail.com Thu Apr 12 09:28:57 2018 From: ciapnz at gmail.com (Danila Vershinin) Date: Thu, 12 Apr 2018 12:28:57 +0300 Subject: Softpurge and nearly immediate refresh? In-Reply-To: References: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com> <15B57C74-5AD8-4A83-AE4E-5431F5FE127C@gmail.com> <6D649EE6-CB1B-43CE-BC27-507FB1E8CCE2@gmail.com> Message-ID: Ah, I?m using Varnish 4.1, guess might have to look into >= 5.2. Thanks! :) Best Regards, Danila > On 12 Apr 2018, at 12:25, Guillaume Quintard wrote: > > bereq.is_bgfetch? > > -- > Guillaume Quintard > > On Thu, Apr 12, 2018, 09:24 Danila Vershinin > wrote: > Couldn?t really find anything useful besides obj.hits which indirectly tells us it?s a background fetch. So: > > sub vcl_deliver { > if (req.http.grace == "normal(limited)" && obj.hits == 0 && req.http.Varied-Header = 'value1') { > set req.http.Varied-Header = 'value2?; > unset req.http.grace; > return (restart); > } > } > > ? :) > > Best Regards, > Danila > >> On 12 Apr 2018, at 00:24, Guillaume Quintard > wrote: >> >> There's a beresp attribute for that last one (I'm on mobile, so I'm going to point you to man vcl :-)) >> >> -- >> Guillaume Quintard >> >> On Wed, Apr 11, 2018, 22:39 Danila Vershinin > wrote: >> So I guess I?m looking into having one background fetch trigger another background fetch (to sequentially refresh different object variants). In this fashion: >> >> 1. Client PURGE request will do softpurge.softpurge(); return(restart); with GET method, etc. which will all lead to "return deliver()" in limited grace logic to fire background fetch . >> >> 2. Then in background fetch: >> ? in its vcl_deliver() - the current object variation has already entered cache, so setting varied header to value2, removing grace limited flag and calling restart(). This way it should continue revalidation for the other object variant >> ? we land inside limited grace logic again (as it?s a different object variant) and return deliver() again thus firing off second background fetch (which will refresh second object variant). >> >> So the standard grace logic + something like this: >> >> sub vcl_deliver { >> if (req.http.grace == "normal(limited)" && req.http.Varied-Header = 'value1') { >> set req.http.Varied-Header = 'value2?; >> unset req.http.grace; >> return (restart); >> } >> } >> >> However, it won?t work at least because req.http.grace flag will be set for both the background fetch and the request that kicked it off. (it will be there in vcl_deliver of both). >> Question is how can we tell if we are inside background fetch? >> >> Best Regards, >> Danila >> >>> On 11 Apr 2018, at 12:37, Guillaume Quintard > wrote: >>> >>> Hi, >>> >>> That's indeed correct, a purge will kill all variations, and the restart only fetches one. >>> >>> The req.hash_always_miss trick however only kills/revalidate one variation. >>> >>> At this moment, we have no way to purge/revalidate all the object under one hash key. >>> >>> -- >>> Guillaume Quintard >>> >>> On Wed, Apr 11, 2018 at 11:26 AM, Danila Vershinin > wrote: >>> Hi Guillaume, >>> >>> A bit puzzled on something. If we use Vary: by some header.. am I correct that we need multiple restarts to refresh each object variation? >>> >>> Since the background fetch would only refresh the variation that matched initial purge request. >>> >>> Sent from my iPhone >>> >>> On 9 Apr 2018, at 12:18, Guillaume Quintard > wrote: >>> >>>> Hi, >>>> >>>> You can purge then set the method to GET then restart. Would that work for you? >>>> >>>> Other way is to use req.hash_always_miss that will only revalidate if we are able to fetch a new object. >>>> >>>> -- >>>> Guillaume Quintard >>>> >>>> On Sat, Apr 7, 2018 at 12:10 PM, Danila Vershinin > wrote: >>>> Hi, >>>> >>>> What I work with: >>>> >>>> * Grace mode configured to be 60 seconds when backend is healthy >>>> * Using softpurge module to adjust TTL to 0 upon PURGE. >>>> >>>> The whole idea is increasing chances that visitors will get cached page after cache was PURGEd for a page. >>>> >>>> Standard piece: >>>> sub vcl_hit { >>>> if (obj.ttl >= 0s) { >>>> # normal hit >>>> return (deliver); >>>> } >>>> >>>> if (std.healthy(req.backend_hint)) { >>>> # Backend is healthy. Limit age to 60s. >>>> if (obj.ttl + 60s > 0s) { >>>> set req.http.grace = "normal(limited)"; >>>> return (deliver); >>>> } else { >>>> return(fetch); >>>> } >>>> } else { >>>> # ... >>>> } >>>> } >>>> And use of softpurge: >>>> >>>> sub vcl_miss { >>>> if (req.method == "PURGE") { >>>> softpurge.softpurge(); >>>> return (synth(200, "Successful softpurge")); >>>> } >>>> } >>>> >>>> sub vcl_hit { >>>> if (req.method == "PURGE") { >>>> softpurge.softpurge(); >>>> return (synth(200, "Successful softpurge")); >>>> } >>>> } >>>> >>>> >>>> Current behaviour: >>>> >>>> * send PURGE for cached page >>>> * Visitor goes to the page within 60 seconds and sees a stale cached page (triggering background refresh) >>>> * Further visits to the page will show refreshed page >>>> >>>> What I?m looking for: >>>> >>>> Trigger the background refresh right after PURGE while still leveraging grace mode :) That is, serve stale cache for only as long as it takes to actually generate the new page, and not wait for 60 seconds: >>>> >>>> * upon PURGE: set TTL to 0 (softpurge) + trigger background page request (possible?) >>>> * serve stale cache only while the page is generated >>>> >>>> I could have adjusted the ?healthy backend grace period? to lower than 60s, but I?m basically checking to see if it?s possible to refresh ?nearly? immediately in this kind of setup. >>>> >>>> Hope I made any sense :) >>>> >>>> Best Regards, >>>> Danila >>>> >>>> >>>> _______________________________________________ >>>> varnish-misc mailing list >>>> varnish-misc at varnish-cache.org >>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From hermunn at varnish-software.com Thu Apr 12 13:13:27 2018 From: hermunn at varnish-software.com (=?UTF-8?Q?P=C3=A5l_Hermunn_Johansen?=) Date: Thu, 12 Apr 2018 15:13:27 +0200 Subject: Softpurge and nearly immediate refresh? In-Reply-To: References: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com> <15B57C74-5AD8-4A83-AE4E-5431F5FE127C@gmail.com> <6D649EE6-CB1B-43CE-BC27-507FB1E8CCE2@gmail.com> Message-ID: > On 12 Apr 2018, at 12:25, Guillaume Quintard > wrote: > > bereq.is_bgfetch? 2018-04-12 11:28 GMT+02:00 Danila Vershinin : > Ah, I?m using Varnish 4.1, guess might have to look into >= 5.2. Not necessary - is_bgfetch was back ported to 4.1, and is in the latest release 4.1.9 (but not in 4.1.8). P?l From michael at kennedy.ie Fri Apr 13 15:42:29 2018 From: michael at kennedy.ie (Michael Kennedy) Date: Fri, 13 Apr 2018 15:42:29 +0000 Subject: Compare Hex to Decimal Message-ID: Hi, I have a URL that contains a Unix hex timestamp (something like 5ace8c5d). The timestamp should be within X minutes of now, and is used as part of an access control mechanism. I want to do something like this: if (std.time(req.http.x-hextime, now) < now - 1w) { ... } Is there an efficient way in VCL to either convert Hex to Dec so that this comparison can be done, or some other solution that does not involve changing the data in the URL. Thanks, Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From dannysantos1985 at gmail.com Fri Apr 13 21:52:41 2018 From: dannysantos1985 at gmail.com (Daniel Santos) Date: Fri, 13 Apr 2018 22:52:41 +0100 Subject: Fwd: Dont understand how to implement this FAQ topic In-Reply-To: References: Message-ID: I don't understand the this FAQ topic . I want to implement it. In my /etc/varnish/default.vcl I have this: backend apache { .host = "127.0.0.1"; .port = "5555"; } backend node_hello { .host = "127.0.0.1"; .port = "5556"; } import std; sub vcl_recv { # Happens before we check if we have this in cache already. # # Typically you clean up the request here, removing cookies you don't need, # rewriting the request, etc. if ((std.port(local.ip) == 6086) && (std.port(server.ip) == 443)) { set req.http.X-Forwarded-Proto = "https"; } if(req.http.host ~ "domain1.com") { set req.backend_hint = apache; } elsif(req.http.host == "domain2.com") { set req.backend_hint = node_hello; } } How should my implementation become? -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Fri Apr 13 21:57:14 2018 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Fri, 13 Apr 2018 21:57:14 +0000 Subject: Dont understand how to implement this FAQ topic In-Reply-To: References: Message-ID: What is your goal, what don't you understand? -- Guillaume Quintard On Fri, Apr 13, 2018, 23:52 Daniel Santos wrote: > I don't understand the this FAQ topic > > . > > I want to implement it. In my /etc/varnish/default.vcl I have this: > > backend apache { > .host = "127.0.0.1"; > .port = "5555"; > } > > backend node_hello { > .host = "127.0.0.1"; > .port = "5556"; > } > > import std; > > sub vcl_recv { > # Happens before we check if we have this in cache already. > # > # Typically you clean up the request here, removing cookies you don't need, > # rewriting the request, etc. > > if ((std.port(local.ip) == 6086) && (std.port(server.ip) == 443)) { > set req.http.X-Forwarded-Proto = "https"; > } > > if(req.http.host ~ "domain1.com") { > set req.backend_hint = apache; > } > elsif(req.http.host == "domain2.com") { > set req.backend_hint = node_hello; > } > } > > How should my implementation become? > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Sat Apr 14 06:17:12 2018 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Sat, 14 Apr 2018 06:17:12 +0000 Subject: Dont understand how to implement this FAQ topic In-Reply-To: References: Message-ID: Please keep the mail-in in the loop so others can benefit from this discussion. Have a look at this: https://info.varnish-software.com/blog/rewriting-urls-with-varnish-redirection It covers exactly your use-case, I think. -- Guillaume Quintard On Sat, Apr 14, 2018, 00:01 Daniel Santos wrote: > I don't understand where I will put the set req.backend_hint on > https://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-cliehttps://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-clients-to-httpsnts-to-https > > . > And if in order to have http redirect to https will I have to put: > > if (std.port(local.ip) == 80 && req.http.host ~ "^(?i)example.com$") { > set req.http.x-redir = "https://" + req.http.host + req.url; > return(synth(301)); > } > > > for every domain or is there a way to make a force http to https rule for > all the domains? > > 2018-04-13 22:57 GMT+01:00 Guillaume Quintard < > guillaume at varnish-software.com>: > >> What is your goal, what don't you understand? >> >> -- >> Guillaume Quintard >> >> On Fri, Apr 13, 2018, 23:52 Daniel Santos >> wrote: >> >>> I don't understand the this FAQ topic >>> >>> . >>> >>> I want to implement it. In my /etc/varnish/default.vcl I have this: >>> >>> backend apache { >>> .host = "127.0.0.1"; >>> .port = "5555"; >>> } >>> >>> backend node_hello { >>> .host = "127.0.0.1"; >>> .port = "5556"; >>> } >>> >>> import std; >>> >>> sub vcl_recv { >>> # Happens before we check if we have this in cache already. >>> # >>> # Typically you clean up the request here, removing cookies you don't need, >>> # rewriting the request, etc. >>> >>> if ((std.port(local.ip) == 6086) && (std.port(server.ip) == 443)) { >>> set req.http.X-Forwarded-Proto = "https"; >>> } >>> >>> if(req.http.host ~ "domain1.com") { >>> set req.backend_hint = apache; >>> } >>> elsif(req.http.host == "domain2.com") { >>> set req.backend_hint = node_hello; >>> } >>> } >>> >>> How should my implementation become? >>> >>> >>> _______________________________________________ >>> varnish-misc mailing list >>> varnish-misc at varnish-cache.org >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jpotter-varnish at codepuppy.com Sat Apr 14 14:06:26 2018 From: jpotter-varnish at codepuppy.com (jpotter-varnish at codepuppy.com) Date: Sat, 14 Apr 2018 10:06:26 -0400 Subject: Compare Hex to Decimal In-Reply-To: References: Message-ID: <56EFAA5A-CE63-44C4-92D8-DB50DF362866@codepuppy.com> Hi Michael, I did something similar to this several years back using inline C: https://varnish-cache.org/docs/5.1/users-guide/vcl-inline-c.html I?m super-rusty on my C, so ask better minds, but I?d start by doing the if check in C and leaving the rest of the logic in VCL. I.e. strtol() with a base of 16 (see http://pubs.opengroup.org/onlinepubs/7908799/xsh/strtol.html ) and then a direct check of that against gmtime() - delta (hopefully your unix hex timestamps in UTC?). Double-check the input to strtol is safe ? this approach has untrusted inputs getting run through into C. -Jeff > On Apr 13, 2018, at 11:42 AM, Michael Kennedy wrote: > > Hi, > > I have a URL that contains a Unix hex timestamp (something like 5ace8c5d). The timestamp should be within X minutes of now, and is used as part of an access control mechanism. I want to do something like this: > > if (std.time(req.http.x-hextime, now) < now - 1w) { > ... > } > > Is there an efficient way in VCL to either convert Hex to Dec so that this comparison can be done, or some other solution that does not involve changing the data in the URL. > > Thanks, > > Michael > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: From dannysantos1985 at gmail.com Sat Apr 14 17:53:52 2018 From: dannysantos1985 at gmail.com (Daniel Santos) Date: Sat, 14 Apr 2018 17:53:52 +0000 Subject: Dont understand how to implement this FAQ topic In-Reply-To: References: Message-ID: Thanks a lot for sharing. A s?bado, 14/04/2018, 07:17, Guillaume Quintard < guillaume at varnish-software.com> escreveu: > Please keep the mail-in in the loop so others can benefit from this > discussion. > > Have a look at this: > https://info.varnish-software.com/blog/rewriting-urls-with-varnish-redirection > > It covers exactly your use-case, I think. > > -- > Guillaume Quintard > > > On Sat, Apr 14, 2018, 00:01 Daniel Santos > wrote: > >> I don't understand where I will put the set req.backend_hint on >> https://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-cliehttps://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-clients-to-httpsnts-to-https >> >> . >> And if in order to have http redirect to https will I have to put: >> >> if (std.port(local.ip) == 80 && req.http.host ~ "^(?i)example.com$") { >> set req.http.x-redir = "https://" + req.http.host + req.url; >> return(synth(301)); >> } >> >> >> for every domain or is there a way to make a force http to https rule for >> all the domains? >> >> 2018-04-13 22:57 GMT+01:00 Guillaume Quintard < >> guillaume at varnish-software.com>: >> >>> What is your goal, what don't you understand? >>> >>> -- >>> Guillaume Quintard >>> >>> On Fri, Apr 13, 2018, 23:52 Daniel Santos >>> wrote: >>> >>>> I don't understand the this FAQ topic >>>> >>>> . >>>> >>>> I want to implement it. In my /etc/varnish/default.vcl I have this: >>>> >>>> backend apache { >>>> .host = "127.0.0.1"; >>>> .port = "5555"; >>>> } >>>> >>>> backend node_hello { >>>> .host = "127.0.0.1"; >>>> .port = "5556"; >>>> } >>>> >>>> import std; >>>> >>>> sub vcl_recv { >>>> # Happens before we check if we have this in cache already. >>>> # >>>> # Typically you clean up the request here, removing cookies you don't need, >>>> # rewriting the request, etc. >>>> >>>> if ((std.port(local.ip) == 6086) && (std.port(server.ip) == 443)) { >>>> set req.http.X-Forwarded-Proto = "https"; >>>> } >>>> >>>> if(req.http.host ~ "domain1.com") { >>>> set req.backend_hint = apache; >>>> } >>>> elsif(req.http.host == "domain2.com") { >>>> set req.backend_hint = node_hello; >>>> } >>>> } >>>> >>>> How should my implementation become? >>>> >>>> >>>> _______________________________________________ >>>> varnish-misc mailing list >>>> varnish-misc at varnish-cache.org >>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Sat Apr 14 17:55:05 2018 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Sat, 14 Apr 2018 17:55:05 +0000 Subject: Dont understand how to implement this FAQ topic In-Reply-To: References: Message-ID: No problem, did you manage to solve your issue? -- Guillaume Quintard On Sat, Apr 14, 2018, 19:54 Daniel Santos wrote: > Thanks a lot for sharing. > > A s?bado, 14/04/2018, 07:17, Guillaume Quintard < > guillaume at varnish-software.com> escreveu: > >> Please keep the mail-in in the loop so others can benefit from this >> discussion. >> >> Have a look at this: >> https://info.varnish-software.com/blog/rewriting-urls-with-varnish-redirection >> >> It covers exactly your use-case, I think. >> >> -- >> Guillaume Quintard >> >> >> On Sat, Apr 14, 2018, 00:01 Daniel Santos >> wrote: >> >>> I don't understand where I will put the set req.backend_hint on >>> https://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-cliehttps://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-clients-to-httpsnts-to-https >>> >>> . >>> And if in order to have http redirect to https will I have to put: >>> >>> if (std.port(local.ip) == 80 && req.http.host ~ "^(?i)example.com$") { >>> set req.http.x-redir = "https://" + req.http.host + req.url; >>> return(synth(301)); >>> } >>> >>> >>> for every domain or is there a way to make a force http to https rule >>> for all the domains? >>> >>> 2018-04-13 22:57 GMT+01:00 Guillaume Quintard < >>> guillaume at varnish-software.com>: >>> >>>> What is your goal, what don't you understand? >>>> >>>> -- >>>> Guillaume Quintard >>>> >>>> On Fri, Apr 13, 2018, 23:52 Daniel Santos >>>> wrote: >>>> >>>>> I don't understand the this FAQ topic >>>>> >>>>> . >>>>> >>>>> I want to implement it. In my /etc/varnish/default.vcl I have this: >>>>> >>>>> backend apache { >>>>> .host = "127.0.0.1"; >>>>> .port = "5555"; >>>>> } >>>>> >>>>> backend node_hello { >>>>> .host = "127.0.0.1"; >>>>> .port = "5556"; >>>>> } >>>>> >>>>> import std; >>>>> >>>>> sub vcl_recv { >>>>> # Happens before we check if we have this in cache already. >>>>> # >>>>> # Typically you clean up the request here, removing cookies you don't need, >>>>> # rewriting the request, etc. >>>>> >>>>> if ((std.port(local.ip) == 6086) && (std.port(server.ip) == 443)) { >>>>> set req.http.X-Forwarded-Proto = "https"; >>>>> } >>>>> >>>>> if(req.http.host ~ "domain1.com") { >>>>> set req.backend_hint = apache; >>>>> } >>>>> elsif(req.http.host == "domain2.com") { >>>>> set req.backend_hint = node_hello; >>>>> } >>>>> } >>>>> >>>>> How should my implementation become? >>>>> >>>>> >>>>> _______________________________________________ >>>>> varnish-misc mailing list >>>>> varnish-misc at varnish-cache.org >>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>>> >>>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From dannysantos1985 at gmail.com Sat Apr 14 18:03:58 2018 From: dannysantos1985 at gmail.com (Daniel Santos) Date: Sat, 14 Apr 2018 18:03:58 +0000 Subject: Dont understand how to implement this FAQ topic In-Reply-To: References: Message-ID: Yes, I did. A s?bado, 14/04/2018, 18:55, Guillaume Quintard < guillaume at varnish-software.com> escreveu: > No problem, did you manage to solve your issue? > > -- > Guillaume Quintard > > On Sat, Apr 14, 2018, 19:54 Daniel Santos > wrote: > >> Thanks a lot for sharing. >> >> A s?bado, 14/04/2018, 07:17, Guillaume Quintard < >> guillaume at varnish-software.com> escreveu: >> >>> Please keep the mail-in in the loop so others can benefit from this >>> discussion. >>> >>> Have a look at this: >>> https://info.varnish-software.com/blog/rewriting-urls-with-varnish-redirection >>> >>> It covers exactly your use-case, I think. >>> >>> -- >>> Guillaume Quintard >>> >>> >>> On Sat, Apr 14, 2018, 00:01 Daniel Santos >>> wrote: >>> >>>> I don't understand where I will put the set req.backend_hint on >>>> https://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-cliehttps://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-clients-to-httpsnts-to-https >>>> >>>> . >>>> And if in order to have http redirect to https will I have to put: >>>> >>>> if (std.port(local.ip) == 80 && req.http.host ~ "^(?i)example.com$") { >>>> set req.http.x-redir = "https://" + req.http.host + req.url; >>>> return(synth(301)); >>>> } >>>> >>>> >>>> for every domain or is there a way to make a force http to https rule >>>> for all the domains? >>>> >>>> 2018-04-13 22:57 GMT+01:00 Guillaume Quintard < >>>> guillaume at varnish-software.com>: >>>> >>>>> What is your goal, what don't you understand? >>>>> >>>>> -- >>>>> Guillaume Quintard >>>>> >>>>> On Fri, Apr 13, 2018, 23:52 Daniel Santos >>>>> wrote: >>>>> >>>>>> I don't understand the this FAQ topic >>>>>> >>>>>> . >>>>>> >>>>>> I want to implement it. In my /etc/varnish/default.vcl I have this: >>>>>> >>>>>> backend apache { >>>>>> .host = "127.0.0.1"; >>>>>> .port = "5555"; >>>>>> } >>>>>> >>>>>> backend node_hello { >>>>>> .host = "127.0.0.1"; >>>>>> .port = "5556"; >>>>>> } >>>>>> >>>>>> import std; >>>>>> >>>>>> sub vcl_recv { >>>>>> # Happens before we check if we have this in cache already. >>>>>> # >>>>>> # Typically you clean up the request here, removing cookies you don't need, >>>>>> # rewriting the request, etc. >>>>>> >>>>>> if ((std.port(local.ip) == 6086) && (std.port(server.ip) == 443)) { >>>>>> set req.http.X-Forwarded-Proto = "https"; >>>>>> } >>>>>> >>>>>> if(req.http.host ~ "domain1.com") { >>>>>> set req.backend_hint = apache; >>>>>> } >>>>>> elsif(req.http.host == "domain2.com") { >>>>>> set req.backend_hint = node_hello; >>>>>> } >>>>>> } >>>>>> >>>>>> How should my implementation become? >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> varnish-misc mailing list >>>>>> varnish-misc at varnish-cache.org >>>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>>>> >>>>> >>>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From pinakee at waltzz.com Fri Apr 27 14:57:01 2018 From: pinakee at waltzz.com (Pinakee BIswas) Date: Fri, 27 Apr 2018 20:27:01 +0530 Subject: Set cookie creating issues Message-ID: <3b74b839-8ff5-f5ea-23dc-41d6fd5113bc@waltzz.com> Hi, We have been using Varnish for caching our web pages. We have an ecommerce site. Things have been working fine till today but suddenly things have started breaking down and I am not sure why. Following is the issue: We use session cookie to store user sessions. The session cookie is getting changed as Cached responses from varnish is having set-cookie header which is messing up the session cookie. We are using varnish 4.8. Following is a snippet of the VCL: sub vcl_recv { # Happens before we check if we have this in cache already. # # Typically you clean up the request here, removing cookies you don't need, # rewriting the request, etc. set req.backend_hint = uwsgi; #if (req.http.cookie ~ "jivaana_country=") { # Set the country header # set req.http.X-CLIENT-COUNTRY = regsub(req.http.cookie, ".*jivaana_country=([^;]+);.*", "\1"); # } std.log("ga:" + ga.extract(req.url, mode = keep)); set req.url = ga.apply(req.url); # remove Google Analytics parameters if (req.method == "GET") { if ((req.url !~ "^/accounts/userheader") && (req.url !~ "^/accounts/new-userheader") && (req.url !~ "^/product/recently-viewed") && (req.url !~ "^/product/recommended-products") && (req.url !~ "^/product/addtobasket")) { unset req.http.cookie; # strip the cookies - we don't need them } } call devicedetect; } sub vcl_backend_response { # Happens after we have read the response headers from the backend. # # Here you clean the response headers, removing silly Set-Cookie headers # and other mistakes your backend does. if (bereq.method == "GET") { set beresp.do_esi = true; if ((bereq.url !~ "^/accounts/userheader") && (bereq.url !~ "^/accounts/new-userheader") && (bereq.url !~ "^/product/recently-viewed") && (bereq.url !~ "^/product/recommended-products") && (bereq.url !~ "^/product/addtobasket")) { #unset beresp.http.Set-Cookie; set beresp.uncacheable = false; #std.log("Caching the url : **********************" + bereq.url); } } sub vcl_deliver { # Happens when we have all the pieces we need, and are about to send the # response to the client. # # You can do accounting or modifying the final object here. } sub vcl_hash { } Would really appreciate any support as this is messing up our user sessions. Thanks, Pinakee From Deepak.Verma at comtechtel.com Mon Apr 30 20:38:43 2018 From: Deepak.Verma at comtechtel.com (Deepak Verma) Date: Mon, 30 Apr 2018 20:38:43 +0000 Subject: Query regarding Varnish modules for parsing XML body Message-ID: Hi, I am trying to see if Varnish as a reverse proxy cache is suitable for my use case. Are there any Varnish modules available which can help me parse XML data present in POST response body. Regards, Deepak NOTICE TO RECIPIENT: This email, including attachments, may contain information which is confidential, proprietary, attorney-client privileged and / or controlled under U.S. export laws and regulations and may be restricted from disclosure by applicable State and Federal law. Nothing in this email shall create any legal binding agreement between the parties unless expressly stated herein and provided by an authorized representative of Comtech Telecommunications Corp. or its subsidiaries. If you are not the intended recipient of this message, be advised that any dissemination, distribution, or use of the contents of this message is strictly prohibited. If you received this message in error, please notify us immediately by return email and permanently delete all copies of the original email and any attached documentation from any computer or other media. -------------- next part -------------- An HTML attachment was scrubbed... URL: