From guillaume at varnish-software.com Wed Jan 2 11:33:48 2019 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 2 Jan 2019 12:33:48 +0100 Subject: htaccess redirects not working after a while In-Reply-To: <1986281426.5271873.1544692478381@mail.yahoo.com> References: <330025186.3775662.1544542010660.ref@mail.yahoo.com> <330025186.3775662.1544542010660@mail.yahoo.com> <1986281426.5271873.1544692478381@mail.yahoo.com> Message-ID: Hi, You can. Alternatively, if the SSL terminator is sending traffic to varnish via another port, you can unconditionally hash the listening port: hash_data(std.port(server.ip)); (don't forget to import std first) -- Guillaume Quintard On Thu, Dec 13, 2018 at 10:16 AM Miguel Gonzalez wrote: > I use Varnish 4.1, can I just add in the vcl_hash this? > > # Cache the HTTP vs HTTPs separately > if (req.http.X-Forwarded-Proto) { > hash_data(req.http.X-Forwarded-Proto); > } > > > > > En jueves, 13 de diciembre de 2018 10:05:20 CET, Mattias Geniar < > mattias at nucleus.be> escribi?: > > > > Here you have it: > > You are not caching the protocol (http or https), which will lead to > different caching results depending on the backend response. > > As an example, have a look here: https://github.com/mattiasgeniar/varnish-6.0-configuration-templates/blob/master/default.vcl#L236-L238 > > > In your Apache proxy config you can set additional headers to mark they've > been passed through it (in this example, X-Forwarded-Proto) so Varnish > knows to treat them differently. > > > Mattias > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Wed Jan 2 12:01:37 2019 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Wed, 2 Jan 2019 13:01:37 +0100 Subject: banning content from a multiuser web server In-Reply-To: <070a399e-2c3e-e396-0533-c654855ff99a@yahoo.es> References: <070a399e-2c3e-e396-0533-c654855ff99a@yahoo.es> Message-ID: (this was in my spam folder) Hi, I think I would create a script that would link an SSH/user account to a domain, and then do "varnishadm ban req.http.host == $DOMAIN" (no ban lurker-friendly, but you get the idea) -- Guillaume Quintard On Thu, Nov 29, 2018 at 8:20 PM Miguel Gonz?lez wrote: > Hi, > > I do have a Cpanel server with Wordpress sites. Varnish plugin is > working fine but some users do have non-Wordpress websites and sometimes > they want to purge the whole website and currently I have to do it as root. > > How can I ban a domain within the SSH user account and make sure it > only purges the domains that are hosted in his account? > > Regards, > > Miguel > > > --- > This email has been checked for viruses by AVG. > https://www.avg.com > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Tue Jan 15 18:23:01 2019 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Tue, 15 Jan 2019 10:23:01 -0800 Subject: Search engine query cache In-Reply-To: References: Message-ID: Redirecting to varnish-misc On Tue, Jan 15, 2019, 09:51 chandan khatri Hi All, > > We are trying to use Varnish as API accelerator in order to speed up the > search engine query response time. > > After the required setup was done, we could see it working and caching the > API calls to search engine but when we performed stress testing we ran into > connection time out exceptions. > > We tried changing max connection and timeout configuration but it didn't > work; moreover, we aren't able to see any error logs in varnish logs the > exception comes in the client application logs only. > > We are stuck and unable to progress any further. Any help would be > appreciated. > > Thanks > Chandan > _______________________________________________ > varnish-dev mailing list > varnish-dev at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mandys at gmail.com Sun Jan 20 12:12:48 2019 From: mandys at gmail.com (Maninder Singh) Date: Sun, 20 Jan 2019 17:42:48 +0530 Subject: Same URL: Don't cache with store response header, but cache when it's not there Message-ID: Hi Everyone, I am unable to get caching to work in a particular scenario. I need to cache the url eg: /app/profile/GDPR1?hostingUrl=http%3A%2F% 2Fitvidffya.blogspot.com%2F2019%2F01%2Fvar-nish-issue-fix.html However, this is embedded in a 3rd party site (www.anotherdomain.com calls mydomain.com in iframe ) so on Apple Safari we can't drop cookies ( as 3rd party cookies are blocked by default ). As a result, for Apple Safari, when this url is hit, our backend returns top.location.href='someurlonoursite_to_set_cookie' which is fired in context of 3rd party site since they embed using our javascript. This way a cookie is dropped on our domain and user gets back to the url inside the iframe. Now, when I hit this url in chrome, it always picks up top.location.href='....' as it got cached by safari. But, chrome was supposed to get the actual page content since it's not blocking 3rd party cookies. So, I went ahead and added a custom header, "store" for cases of apple safari in our backend. I skip unsetting cookie (hence varnish cache) for this url in this case. But, it still doesn't cache on chrome in subsequent hits. Always goes to backend, never goes to the cache. Is it because I have one url - - that can return 2 responses based on browser - I told it to not cache first time when store header was there, but when store header is not there i ask it to cache it. Still doesn't work. Config sub vcl_recv { std.log("vtglog: in vcl_recv " + req.url); # Only cache GET or HEAD requests. This makes sure the POST requests are always passed. if (req.method != "GET" && req.method != "HEAD") { return (pass); } if(req.http.host == "sqa03.mydomain.com" && req.url ~ "/app/profile"){ std.log("vtglog: unsetting cookies"); unset req.http.cookie; } else{ return(pass); } if (req.http.Authorization) { # Not cacheable by default return (pass); } } sub vcl_backend_response { std.log("vtglog: vcl_backend_response" + bereq.url) ; # Happens after we have read the response headers from the backend. # # Here you clean the response headers, removing silly Set-Cookie headers # and other mistakes your backend does. if (bereq.http.host == "sqa03.mydomain.com" && bereq.url ~ "/app/profile") { std.log("vtglog: inside condition in backend response"); std.log("vtglog: store header value is " + beresp.http.store); if ( beresp.http.Set-Cookie ) { if ( !beresp.http.store ) { std.log("vtglog: since no store headers, cache it by unsetting cookies"); unset beresp.http.Set-Cookie; } else { std.log("vtglog: store header found, dont cache"); } } } if (beresp.status == 301 || beresp.status == 302) { set beresp.http.Location = regsub(beresp.http.Location, ":[0-9]+", ""); } # Don't cache 50x responses if (beresp.status == 500 || beresp.status == 502 || beresp.status == 503 || beresp.status == 504) { return (abandon); } } -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Mon Jan 21 03:39:14 2019 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Sun, 20 Jan 2019 19:39:14 -0800 Subject: Same URL: Don't cache with store response header, but cache when it's not there In-Reply-To: References: Message-ID: Hi, > Is it because I have one url - > - that can return 2 responses based on browser That's the one, you need to tell varnish to care about the browser. You have two ways to do that: either put that in in the hash key directly, or put it in in a header, and tell that to varnish via the vary header. First option looks like this: sub vcl_recv { if (SOME_TEST_TO_CHECK_ITS_SAFARI) { set req.http.browser = "safari"; } else { set req.http.browser = "somethingelse"; } } sub vcl_hash { hash_data(req.http.browser); # do NOT return, let the built-in vcl run } The pro is that you don't need to change the backend, the con is that if you purge, you have to purge twice because they are two different objects. The second option is to tell varnish that the browser is important, but just a variant of the same object: sub vcl_recv { if (SOME_TEST_TO_CHECK_ITS_SAFARI) { set req.http.browser = "safari"; } else { set req.http.browser = "somethingelse"; } } sub vcl_backend_response { set beresp.http.vary = beresp.http.vary + ",browser"; } Note that you can also have the backend return the vary header for you directly There are a couple of cleaner versions, but they are a bit more involved, so let's start with that. Side note: I understand where you are coming from with the std.log debugging, but that info is actually super redundant with what's in the log. -- Guillaume Quintard On Sun, Jan 20, 2019 at 4:14 AM Maninder Singh wrote: > Hi Everyone, > > I am unable to get caching to work in a particular scenario. > > I need to cache the url eg: /app/profile/GDPR1?hostingUrl=http%3A%2F% > 2Fitvidffya.blogspot.com%2F2019%2F01%2Fvar-nish-issue-fix.html > > However, this is embedded in a 3rd party site (www.anotherdomain.com > calls mydomain.com in iframe ) so on Apple Safari we can't drop cookies ( > as 3rd party cookies are blocked by default ). > > As a result, for Apple Safari, when this url is hit, our backend returns > top.location.href='someurlonoursite_to_set_cookie' which is fired in > context of 3rd party site since they embed using our javascript. > > This way a cookie is dropped on our domain and user gets back to the url > inside the iframe. > > Now, when I hit this url in chrome, it always picks up > top.location.href='....' as it got cached by safari. > > But, chrome was supposed to get the actual page content since it's not > blocking 3rd party cookies. > > So, I went ahead and added a custom header, "store" for cases of apple > safari in our backend. > I skip unsetting cookie (hence varnish cache) for this url in this case. > > But, it still doesn't cache on chrome in subsequent hits. > > Always goes to backend, never goes to the cache. > > Is it because I have one url - > - that can return 2 responses based on browser > - I told it to not cache first time when store header was there, but when > store header is not there i ask it to cache it. > > Still doesn't work. > > Config > > sub vcl_recv { > std.log("vtglog: in vcl_recv " + req.url); > # Only cache GET or HEAD requests. This makes sure the POST requests > are always passed. > if (req.method != "GET" && req.method != "HEAD") { > return (pass); > } > > if(req.http.host == "sqa03.mydomain.com" && req.url ~ "/app/profile"){ > std.log("vtglog: unsetting cookies"); > unset req.http.cookie; > } else{ > return(pass); > } > > if (req.http.Authorization) { > # Not cacheable by default > return (pass); > } > } > > sub vcl_backend_response { > std.log("vtglog: vcl_backend_response" + bereq.url) ; > # Happens after we have read the response headers from the backend. > # > # Here you clean the response headers, removing silly Set-Cookie > headers > # and other mistakes your backend does. > if (bereq.http.host == "sqa03.mydomain.com" && bereq.url ~ > "/app/profile") { > std.log("vtglog: inside condition in backend response"); > std.log("vtglog: store header value is " + beresp.http.store); > if ( beresp.http.Set-Cookie ) { > if ( !beresp.http.store ) { > std.log("vtglog: since no store headers, cache it by > unsetting cookies"); > unset beresp.http.Set-Cookie; > } else { > std.log("vtglog: store header found, dont cache"); > } > } > } > > if (beresp.status == 301 || beresp.status == 302) { > set beresp.http.Location = regsub(beresp.http.Location, ":[0-9]+", > ""); > } > > # Don't cache 50x responses > if (beresp.status == 500 || beresp.status == 502 || beresp.status == > 503 || beresp.status == 504) { > return (abandon); > } > } > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mandys at gmail.com Wed Jan 23 04:53:53 2019 From: mandys at gmail.com (Maninder Singh) Date: Wed, 23 Jan 2019 10:23:53 +0530 Subject: Same URL: Don't cache with store response header, but cache when it's not there In-Reply-To: References: Message-ID: Thank you Guillaume for your suggestions. This seemed to have worked. I am now using the following vcl config. >> I understand where you are coming from with the std.log debugging, but that info is actually super redundant with what's in the log. I know. But, varnishlog has lot of info and this was done so that I could just do varnishlog | grep "vtglog" to see the flow as per my liking :-) This has now been removed. This should now keep 2 copies of the same url - one for Safari and one for the rest - right ? Question: What's a good way to check how many cache copies are present for a url ? Once again thank you for your help! - Mandy. vcl 4.0; # Default backend definition. Set this to point to your content server. backend default { .host = "127.0.0.1"; .port = "8080"; } sub vcl_recv { if (req.http.User-Agent ~ "Safari") { if (req.http.User-Agent ~ "Chrome") { set req.http.browser = "other"; } else { set req.http.browser = "safari"; } } else { set req.http.browser = "other"; } # Only cache GET or HEAD requests. This makes sure the POST requests are always passed. if (req.method != "GET" && req.method != "HEAD") { return (pass); } if(req.http.host == "sqa03.mydomain.com" && req.url ~ "/app/profile"){ unset req.http.cookie; } else{ return(pass); } if (req.http.Authorization) { # Not cacheable by default return (pass); } } sub vcl_backend_response { # Happens after we have read the response headers from the backend. # # Here you clean the response headers, removing silly Set-Cookie headers # and other mistakes your backend does. if (bereq.http.host == "sqa03.mydomain.com" && bereq.url ~ "/app/profile") { set beresp.ttl = 180 s; if ( beresp.http.Set-Cookie ) { unset beresp.http.Set-Cookie; } } if (beresp.status == 301 || beresp.status == 302) { set beresp.http.Location = regsub(beresp.http.Location, ":[0-9]+", ""); } # Don't cache 50x responses if (beresp.status == 500 || beresp.status == 502 || beresp.status == 503 || beresp.status == 504) { return (abandon); } } sub vcl_hash { hash_data(req.http.browser); } sub vcl_deliver { # Happens when we have all the pieces we need, and are about to send the # response to the client. # # You can do accounting or modifying the final object here. if (obj.hits > 0) { # Add debug header to see if it's a HIT/MISS and the number of hits, disable when not needed set resp.http.X-Cache = "HIT"; } else { set resp.http.X-Cache = "MISS"; } set resp.http.X-Cache-Hits = obj.hits; } On Mon, 21 Jan 2019 at 09:09, Guillaume Quintard < guillaume at varnish-software.com> wrote: > Hi, > > > Is it because I have one url - > > - that can return 2 responses based on browser > > That's the one, you need to tell varnish to care about the browser. You > have two ways to do that: either put that in in the hash key directly, or > put it in in a header, and tell that to varnish via the vary header. > > First option looks like this: > > sub vcl_recv { > if (SOME_TEST_TO_CHECK_ITS_SAFARI) { > set req.http.browser = "safari"; > } else { > set req.http.browser = "somethingelse"; > } > } > > sub vcl_hash { > hash_data(req.http.browser); > # do NOT return, let the built-in vcl run > } > > The pro is that you don't need to change the backend, the con is that if > you purge, you have to purge twice because they are two different objects. > > The second option is to tell varnish that the browser is important, but > just a variant of the same object: > > sub vcl_recv { > if (SOME_TEST_TO_CHECK_ITS_SAFARI) { > set req.http.browser = "safari"; > } else { > set req.http.browser = "somethingelse"; > } > } > > sub vcl_backend_response { > set beresp.http.vary = beresp.http.vary + ",browser"; > } > > Note that you can also have the backend return the vary header for you > directly > > There are a couple of cleaner versions, but they are a bit more involved, > so let's start with that. > > Side note: I understand where you are coming from with the std.log > debugging, but that info is actually super redundant with what's in the log. > > -- > Guillaume Quintard > > > On Sun, Jan 20, 2019 at 4:14 AM Maninder Singh wrote: > >> Hi Everyone, >> >> I am unable to get caching to work in a particular scenario. >> >> I need to cache the url eg: /app/profile/GDPR1?hostingUrl=http%3A%2F% >> 2Fitvidffya.blogspot.com%2F2019%2F01%2Fvar-nish-issue-fix.html >> >> However, this is embedded in a 3rd party site (www.anotherdomain.com >> calls mydomain.com in iframe ) so on Apple Safari we can't drop cookies >> ( as 3rd party cookies are blocked by default ). >> >> As a result, for Apple Safari, when this url is hit, our backend returns >> top.location.href='someurlonoursite_to_set_cookie' which is fired in >> context of 3rd party site since they embed using our javascript. >> >> This way a cookie is dropped on our domain and user gets back to the url >> inside the iframe. >> >> Now, when I hit this url in chrome, it always picks up >> top.location.href='....' as it got cached by safari. >> >> But, chrome was supposed to get the actual page content since it's not >> blocking 3rd party cookies. >> >> So, I went ahead and added a custom header, "store" for cases of apple >> safari in our backend. >> I skip unsetting cookie (hence varnish cache) for this url in this case. >> >> But, it still doesn't cache on chrome in subsequent hits. >> >> Always goes to backend, never goes to the cache. >> >> Is it because I have one url - >> - that can return 2 responses based on browser >> - I told it to not cache first time when store header was there, but when >> store header is not there i ask it to cache it. >> >> Still doesn't work. >> >> Config >> >> sub vcl_recv { >> std.log("vtglog: in vcl_recv " + req.url); >> # Only cache GET or HEAD requests. This makes sure the POST requests >> are always passed. >> if (req.method != "GET" && req.method != "HEAD") { >> return (pass); >> } >> >> if(req.http.host == "sqa03.mydomain.com" && req.url ~ >> "/app/profile"){ >> std.log("vtglog: unsetting cookies"); >> unset req.http.cookie; >> } else{ >> return(pass); >> } >> >> if (req.http.Authorization) { >> # Not cacheable by default >> return (pass); >> } >> } >> >> sub vcl_backend_response { >> std.log("vtglog: vcl_backend_response" + bereq.url) ; >> # Happens after we have read the response headers from the backend. >> # >> # Here you clean the response headers, removing silly Set-Cookie >> headers >> # and other mistakes your backend does. >> if (bereq.http.host == "sqa03.mydomain.com" && bereq.url ~ >> "/app/profile") { >> std.log("vtglog: inside condition in backend response"); >> std.log("vtglog: store header value is " + beresp.http.store); >> if ( beresp.http.Set-Cookie ) { >> if ( !beresp.http.store ) { >> std.log("vtglog: since no store headers, cache it by >> unsetting cookies"); >> unset beresp.http.Set-Cookie; >> } else { >> std.log("vtglog: store header found, dont cache"); >> } >> } >> } >> >> if (beresp.status == 301 || beresp.status == 302) { >> set beresp.http.Location = regsub(beresp.http.Location, >> ":[0-9]+", ""); >> } >> >> # Don't cache 50x responses >> if (beresp.status == 500 || beresp.status == 502 || beresp.status == >> 503 || beresp.status == 504) { >> return (abandon); >> } >> } >> _______________________________________________ >> varnish-misc mailing list >> varnish-misc at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Fri Jan 25 21:24:10 2019 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Fri, 25 Jan 2019 13:24:10 -0800 Subject: Same URL: Don't cache with store response header, but cache when it's not there In-Reply-To: References: Message-ID: > I know. But, varnishlog has lot of info and this was done so that I could just do varnishlog | grep "vtglog" to see the flow as per my liking :-) > This has now been removed. varnishlog -i NameOfTheTag is your friend > This should now keep 2 copies of the same url - one for Safari and one for the rest - right ? Yessir > What's a good way to check how many cache copies are present for a url ? that's not something you can do at the moment -- Guillaume Quintard On Tue, Jan 22, 2019 at 8:54 PM Maninder Singh wrote: > Thank you Guillaume for your suggestions. > > This seemed to have worked. > > I am now using the following vcl config. > > >> I understand where you are coming from with the std.log debugging, but > that info is actually super redundant with what's in the log. > I know. But, varnishlog has lot of info and this was done so that I could > just do varnishlog | grep "vtglog" to see the flow as per my liking :-) > This has now been removed. > > This should now keep 2 copies of the same url - one for Safari and one for > the rest - right ? > > Question: > What's a good way to check how many cache copies are present for a url ? > > Once again thank you for your help! > > - Mandy. > > vcl 4.0; > > # Default backend definition. Set this to point to your content server. > backend default { > .host = "127.0.0.1"; > .port = "8080"; > } > > sub vcl_recv { > if (req.http.User-Agent ~ "Safari") { > if (req.http.User-Agent ~ "Chrome") { > set req.http.browser = "other"; > } else { > set req.http.browser = "safari"; > } > } else { > set req.http.browser = "other"; > } > > # Only cache GET or HEAD requests. This makes sure the POST requests > are always passed. > if (req.method != "GET" && req.method != "HEAD") { > return (pass); > } > > if(req.http.host == "sqa03.mydomain.com" && req.url ~ "/app/profile"){ > unset req.http.cookie; > } else{ > return(pass); > } > > if (req.http.Authorization) { > # Not cacheable by default > return (pass); > } > } > > sub vcl_backend_response { > # Happens after we have read the response headers from the backend. > # > # Here you clean the response headers, removing silly Set-Cookie > headers > # and other mistakes your backend does. > if (bereq.http.host == "sqa03.mydomain.com" && bereq.url ~ > "/app/profile") { > set beresp.ttl = 180 s; > if ( beresp.http.Set-Cookie ) { > unset beresp.http.Set-Cookie; > } > } > > if (beresp.status == 301 || beresp.status == 302) { > set beresp.http.Location = regsub(beresp.http.Location, ":[0-9]+", > ""); > } > > # Don't cache 50x responses > if (beresp.status == 500 || beresp.status == 502 || beresp.status == > 503 || beresp.status == 504) { > return (abandon); > } > } > > sub vcl_hash { > hash_data(req.http.browser); > } > > sub vcl_deliver { > # Happens when we have all the pieces we need, and are about to send > the > # response to the client. > # > # You can do accounting or modifying the final object here. > if (obj.hits > 0) { > # Add debug header to see if it's a HIT/MISS and the number of hits, > disable when not needed > set resp.http.X-Cache = "HIT"; > } else { > set resp.http.X-Cache = "MISS"; > } > > set resp.http.X-Cache-Hits = obj.hits; > } > > On Mon, 21 Jan 2019 at 09:09, Guillaume Quintard < > guillaume at varnish-software.com> wrote: > >> Hi, >> >> > Is it because I have one url - >> > - that can return 2 responses based on browser >> >> That's the one, you need to tell varnish to care about the browser. You >> have two ways to do that: either put that in in the hash key directly, or >> put it in in a header, and tell that to varnish via the vary header. >> >> First option looks like this: >> >> sub vcl_recv { >> if (SOME_TEST_TO_CHECK_ITS_SAFARI) { >> set req.http.browser = "safari"; >> } else { >> set req.http.browser = "somethingelse"; >> } >> } >> >> sub vcl_hash { >> hash_data(req.http.browser); >> # do NOT return, let the built-in vcl run >> } >> >> The pro is that you don't need to change the backend, the con is that if >> you purge, you have to purge twice because they are two different objects. >> >> The second option is to tell varnish that the browser is important, but >> just a variant of the same object: >> >> sub vcl_recv { >> if (SOME_TEST_TO_CHECK_ITS_SAFARI) { >> set req.http.browser = "safari"; >> } else { >> set req.http.browser = "somethingelse"; >> } >> } >> >> sub vcl_backend_response { >> set beresp.http.vary = beresp.http.vary + ",browser"; >> } >> >> Note that you can also have the backend return the vary header for you >> directly >> >> There are a couple of cleaner versions, but they are a bit more involved, >> so let's start with that. >> >> Side note: I understand where you are coming from with the std.log >> debugging, but that info is actually super redundant with what's in the log. >> >> -- >> Guillaume Quintard >> >> >> On Sun, Jan 20, 2019 at 4:14 AM Maninder Singh wrote: >> >>> Hi Everyone, >>> >>> I am unable to get caching to work in a particular scenario. >>> >>> I need to cache the url eg: /app/profile/GDPR1?hostingUrl=http%3A%2F% >>> 2Fitvidffya.blogspot.com%2F2019%2F01%2Fvar-nish-issue-fix.html >>> >>> However, this is embedded in a 3rd party site (www.anotherdomain.com >>> calls mydomain.com in iframe ) so on Apple Safari we can't drop cookies >>> ( as 3rd party cookies are blocked by default ). >>> >>> As a result, for Apple Safari, when this url is hit, our backend returns >>> top.location.href='someurlonoursite_to_set_cookie' which is fired in >>> context of 3rd party site since they embed using our javascript. >>> >>> This way a cookie is dropped on our domain and user gets back to the url >>> inside the iframe. >>> >>> Now, when I hit this url in chrome, it always picks up >>> top.location.href='....' as it got cached by safari. >>> >>> But, chrome was supposed to get the actual page content since it's not >>> blocking 3rd party cookies. >>> >>> So, I went ahead and added a custom header, "store" for cases of apple >>> safari in our backend. >>> I skip unsetting cookie (hence varnish cache) for this url in this case. >>> >>> But, it still doesn't cache on chrome in subsequent hits. >>> >>> Always goes to backend, never goes to the cache. >>> >>> Is it because I have one url - >>> - that can return 2 responses based on browser >>> - I told it to not cache first time when store header was there, but >>> when store header is not there i ask it to cache it. >>> >>> Still doesn't work. >>> >>> Config >>> >>> sub vcl_recv { >>> std.log("vtglog: in vcl_recv " + req.url); >>> # Only cache GET or HEAD requests. This makes sure the POST requests >>> are always passed. >>> if (req.method != "GET" && req.method != "HEAD") { >>> return (pass); >>> } >>> >>> if(req.http.host == "sqa03.mydomain.com" && req.url ~ >>> "/app/profile"){ >>> std.log("vtglog: unsetting cookies"); >>> unset req.http.cookie; >>> } else{ >>> return(pass); >>> } >>> >>> if (req.http.Authorization) { >>> # Not cacheable by default >>> return (pass); >>> } >>> } >>> >>> sub vcl_backend_response { >>> std.log("vtglog: vcl_backend_response" + bereq.url) ; >>> # Happens after we have read the response headers from the backend. >>> # >>> # Here you clean the response headers, removing silly Set-Cookie >>> headers >>> # and other mistakes your backend does. >>> if (bereq.http.host == "sqa03.mydomain.com" && bereq.url ~ >>> "/app/profile") { >>> std.log("vtglog: inside condition in backend response"); >>> std.log("vtglog: store header value is " + beresp.http.store); >>> if ( beresp.http.Set-Cookie ) { >>> if ( !beresp.http.store ) { >>> std.log("vtglog: since no store headers, cache it by >>> unsetting cookies"); >>> unset beresp.http.Set-Cookie; >>> } else { >>> std.log("vtglog: store header found, dont cache"); >>> } >>> } >>> } >>> >>> if (beresp.status == 301 || beresp.status == 302) { >>> set beresp.http.Location = regsub(beresp.http.Location, >>> ":[0-9]+", ""); >>> } >>> >>> # Don't cache 50x responses >>> if (beresp.status == 500 || beresp.status == 502 || beresp.status == >>> 503 || beresp.status == 504) { >>> return (abandon); >>> } >>> } >>> _______________________________________________ >>> varnish-misc mailing list >>> varnish-misc at varnish-cache.org >>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From info+varnish at shee.org Thu Jan 31 14:21:00 2019 From: info+varnish at shee.org (info+varnish at shee.org) Date: Thu, 31 Jan 2019 15:21:00 +0100 Subject: Locating connection reset issue / h2 vs http/1.1 Message-ID: Hi all, I have following stack: hitch-1.5 - varnish-5.2.0 - httpd-2.2/2.4 On a high traffic node I am observing a lot of "Socket error: Connection reset by peer" log entries coming from hitch. I am trying to locate the cause of the issue (hitch or varnish site). So far I can say; that disabling h2 on hitch the "Connection resets" doesn't appear anymore. Does this have to do with varnish-5.2.'s h2 implementation? Thanks Leon Jan 30 19:02:37 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :0 10:11 proxy connect Jan 30 19:02:37 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :0 10:11 ssl handshake start Jan 30 19:02:37 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :0 10:11 ssl client handshake revents=1 Jan 30 19:02:37 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :0 10:11 ssl client handshake err=SSL_ERROR_WANT_READ Jan 30 19:02:37 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :0 10:11 ssl client handshake revents=1 Jan 30 19:02:37 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :0 10:11 NPN/ALPN protocol: h2 Jan 30 19:02:37 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :0 10:11 ssl end handshake Jan 30 19:02:37 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :42884 10:11 backend connected Jan 30 19:02:39 srv-s01 hitch[4006]: {backend} Socket error: Connection reset by peer Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :42884 10:11 proxy shutdown req=SHUTDOWN_CLEAR Jan 30 19:02:39 srv-s01 hitch[4006]: {backend} Socket error: Broken pipe Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :42884 10:11 proxy shutdown req=SHUTDOWN_CLEAR Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :0 10:11 proxy connect Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :0 10:11 ssl handshake start Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :0 10:11 ssl client handshake revents=1 Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :0 10:11 ssl client handshake err=SSL_ERROR_WANT_READ Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :0 10:11 ssl client handshake revents=1 Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :0 10:11 NPN/ALPN protocol: h2 Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :0 10:11 ssl end handshake Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :42922 10:11 backend connected Jan 30 19:02:40 srv-s01 hitch[4006]: {backend} Socket error: Connection reset by peer Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :42922 10:11 proxy shutdown req=SHUTDOWN_CLEAR Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :42922 10:11 proxy shutdown req=SHUTDOWN_HARD Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :0 10:11 proxy connect Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :0 10:11 ssl handshake start Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :0 10:11 ssl client handshake revents=1 Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :0 10:11 ssl client handshake err=SSL_ERROR_WANT_READ Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :0 10:11 ssl client handshake revents=1 Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :0 10:11 NPN/ALPN protocol: h2 Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :0 10:11 ssl end handshake Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :42924 10:11 backend connected Jan 30 19:03:41 srv-s01 hitch[4006]: ww.xx.yy.zz:59418 :0 12:13 proxy connect Jan 30 19:03:41 srv-s01 hitch[4006]: ww.xx.yy.zz:59418 :0 12:13 ssl handshake start Jan 30 19:03:41 srv-s01 hitch[4006]: ww.xx.yy.zz:59418 :0 12:13 ssl client handshake revents=1 Jan 30 19:03:41 srv-s01 hitch[4006]: ww.xx.yy.zz:59418 :0 12:13 ssl client handshake err=SSL_ERROR_WANT_READ Jan 30 19:03:41 srv-s01 hitch[4006]: ww.xx.yy.zz:59418 :0 12:13 ssl client handshake revents=1 Jan 30 19:03:41 srv-s01 hitch[4006]: ww.xx.yy.zz:59418 :0 12:13 NPN/ALPN protocol: h2 Jan 30 19:03:41 srv-s01 hitch[4006]: ww.xx.yy.zz:59418 :0 12:13 ssl end handshake Jan 30 19:03:41 srv-s01 hitch[4006]: ww.xx.yy.zz:59418 :42926 12:13 backend connected Jan 30 19:03:54 srv-s01 hitch[4006]: {backend} Socket error: Connection reset by peer Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :42924 10:11 proxy shutdown req=SHUTDOWN_CLEAR Jan 30 19:03:54 srv-s01 hitch[4006]: {backend} Socket error: Broken pipe Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :42924 10:11 proxy shutdown req=SHUTDOWN_CLEAR Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59478 :0 10:11 proxy connect Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59478 :0 10:11 ssl handshake start Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59478 :0 10:11 ssl client handshake revents=1 Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59478 :0 10:11 ssl client handshake err=SSL_ERROR_WANT_READ Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59478 :0 10:11 ssl client handshake revents=1 Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59478 :0 10:11 NPN/ALPN protocol: h2 Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59478 :0 10:11 ssl end handshake Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59478 :42970 10:11 backend connected From guillaume at varnish-software.com Thu Jan 31 16:38:55 2019 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Thu, 31 Jan 2019 08:38:55 -0800 Subject: Locating connection reset issue / h2 vs http/1.1 In-Reply-To: References: Message-ID: Hi, Have you activated h2 support in Varnish? (it's not on by default) Cheers, -- Guillaume Quintard On Thu, Jan 31, 2019 at 6:22 AM wrote: > Hi all, > > I have following stack: hitch-1.5 - varnish-5.2.0 - httpd-2.2/2.4 > > On a high traffic node I am observing a lot of "Socket error: Connection > reset by peer" log entries coming from hitch. > > I am trying to locate the cause of the issue (hitch or varnish site). > > So far I can say; that disabling h2 on hitch the "Connection resets" > doesn't appear anymore. > > Does this have to do with varnish-5.2.'s h2 implementation? > > Thanks > Leon > > > Jan 30 19:02:37 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :0 10:11 proxy > connect > Jan 30 19:02:37 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :0 10:11 ssl > handshake start > Jan 30 19:02:37 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :0 10:11 ssl client > handshake revents=1 > Jan 30 19:02:37 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :0 10:11 ssl client > handshake err=SSL_ERROR_WANT_READ > Jan 30 19:02:37 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :0 10:11 ssl client > handshake revents=1 > Jan 30 19:02:37 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :0 10:11 NPN/ALPN > protocol: h2 > Jan 30 19:02:37 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :0 10:11 ssl end > handshake > Jan 30 19:02:37 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :42884 10:11 > backend connected > Jan 30 19:02:39 srv-s01 hitch[4006]: {backend} Socket error: Connection > reset by peer > Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :42884 10:11 proxy > shutdown req=SHUTDOWN_CLEAR > Jan 30 19:02:39 srv-s01 hitch[4006]: {backend} Socket error: Broken pipe > Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :42884 10:11 proxy > shutdown req=SHUTDOWN_CLEAR > Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :0 10:11 proxy > connect > Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :0 10:11 ssl > handshake start > Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :0 10:11 ssl client > handshake revents=1 > Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :0 10:11 ssl client > handshake err=SSL_ERROR_WANT_READ > Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :0 10:11 ssl client > handshake revents=1 > Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :0 10:11 NPN/ALPN > protocol: h2 > Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :0 10:11 ssl end > handshake > Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :42922 10:11 > backend connected > Jan 30 19:02:40 srv-s01 hitch[4006]: {backend} Socket error: Connection > reset by peer > Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :42922 10:11 proxy > shutdown req=SHUTDOWN_CLEAR > Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :42922 10:11 proxy > shutdown req=SHUTDOWN_HARD > Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :0 10:11 proxy > connect > Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :0 10:11 ssl > handshake start > Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :0 10:11 ssl client > handshake revents=1 > Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :0 10:11 ssl client > handshake err=SSL_ERROR_WANT_READ > Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :0 10:11 ssl client > handshake revents=1 > Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :0 10:11 NPN/ALPN > protocol: h2 > Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :0 10:11 ssl end > handshake > Jan 30 19:02:40 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :42924 10:11 > backend connected > Jan 30 19:03:41 srv-s01 hitch[4006]: ww.xx.yy.zz:59418 :0 12:13 proxy > connect > Jan 30 19:03:41 srv-s01 hitch[4006]: ww.xx.yy.zz:59418 :0 12:13 ssl > handshake start > Jan 30 19:03:41 srv-s01 hitch[4006]: ww.xx.yy.zz:59418 :0 12:13 ssl client > handshake revents=1 > Jan 30 19:03:41 srv-s01 hitch[4006]: ww.xx.yy.zz:59418 :0 12:13 ssl client > handshake err=SSL_ERROR_WANT_READ > Jan 30 19:03:41 srv-s01 hitch[4006]: ww.xx.yy.zz:59418 :0 12:13 ssl client > handshake revents=1 > Jan 30 19:03:41 srv-s01 hitch[4006]: ww.xx.yy.zz:59418 :0 12:13 NPN/ALPN > protocol: h2 > Jan 30 19:03:41 srv-s01 hitch[4006]: ww.xx.yy.zz:59418 :0 12:13 ssl end > handshake > Jan 30 19:03:41 srv-s01 hitch[4006]: ww.xx.yy.zz:59418 :42926 12:13 > backend connected > Jan 30 19:03:54 srv-s01 hitch[4006]: {backend} Socket error: Connection > reset by peer > Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :42924 10:11 proxy > shutdown req=SHUTDOWN_CLEAR > Jan 30 19:03:54 srv-s01 hitch[4006]: {backend} Socket error: Broken pipe > Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59413 :42924 10:11 proxy > shutdown req=SHUTDOWN_CLEAR > Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59478 :0 10:11 proxy > connect > Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59478 :0 10:11 ssl > handshake start > Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59478 :0 10:11 ssl client > handshake revents=1 > Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59478 :0 10:11 ssl client > handshake err=SSL_ERROR_WANT_READ > Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59478 :0 10:11 ssl client > handshake revents=1 > Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59478 :0 10:11 NPN/ALPN > protocol: h2 > Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59478 :0 10:11 ssl end > handshake > Jan 30 19:03:54 srv-s01 hitch[4006]: ww.xx.yy.zz:59478 :42970 10:11 > backend connected > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From info+varnish at shee.org Thu Jan 31 17:45:17 2019 From: info+varnish at shee.org (info+varnish at shee.org) Date: Thu, 31 Jan 2019 18:45:17 +0100 Subject: Locating connection reset issue / h2 vs http/1.1 In-Reply-To: References: Message-ID: <678C443B-1889-40EA-9835-9A2C7EA3091A@shee.org> Am 31.01.2019 um 17:38 schrieb Guillaume Quintard : > > On Thu, Jan 31, 2019 at 6:22 AM wrote: >> >> I have following stack: hitch-1.5 - varnish-5.2.0 - httpd-2.2/2.4 >> >> On a high traffic node I am observing a lot of "Socket error: Connection reset by peer" log entries coming from hitch. >> >> I am trying to locate the cause of the issue (hitch or varnish site). >> >> So far I can say; that disabling h2 on hitch the "Connection resets" doesn't appear anymore. >> >> Does this have to do with varnish-5.2.'s h2 implementation? >> >> Jan 30 19:02:37 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :0 10:11 NPN/ALPN protocol: h2 >> Jan 30 19:02:37 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :0 10:11 ssl end handshake >> Jan 30 19:02:37 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :42884 10:11 backend connected >> Jan 30 19:02:39 srv-s01 hitch[4006]: {backend} Socket error: Connection reset by peer >> Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :42884 10:11 proxy shutdown req=SHUTDOWN_CLEAR >> Jan 30 19:02:39 srv-s01 hitch[4006]: {backend} Socket error: Broken pipe >> Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59395 :42884 10:11 proxy shutdown req=SHUTDOWN_CLEAR >> Jan 30 19:02:39 srv-s01 hitch[4006]: ww.xx.yy.zz:59399 :0 10:11 proxy connect > > Have you activated h2 support in Varnish? (it's not on by default) > Sure, DAEMON_OPTS has -p feature=+http2 passed. The content is delivered via h2 (verified in browsers) but sometimes lot of assets (client view) produce ERR_CONNECTION_CLOSED errors in the browser and on server site the mentioned "connection reset by peer" log entries appears ... -- Leon