From eric.m.vandenhout at kpn.com Tue Oct 1 11:20:15 2019 From: eric.m.vandenhout at kpn.com (eric.m.vandenhout at kpn.com) Date: Tue, 1 Oct 2019 11:20:15 +0000 Subject: data throughput Message-ID: Hi Does anyone know if there is a neat way to see the varnish data throughput per second? We are moving from free varnish to a paid plan and I would like to see if the currently chosen plan will be sufficient for our needs. Since this is based on Gb/s I want to know how much data is passing by on the varnish server without taking in account data from monitoring and management of the machine itselves. Hence the wish to find some metric inside Varnish, but could not find it yet? Met vriendelijke groet/Kind Regards Eric van den Hout Devops Engineer / Technical administrator Cluster Klant & Content / AAB [cid:image001.png at 01D5785A.F58D3700] Stationsstraat 115 3811 MH Amersfoort Mofo: +31 6 83569993 Email: aab at kpn.com eric.m.vandenhout at kpn.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 10891 bytes Desc: image001.png URL: From slink at schokola.de Tue Oct 1 11:37:21 2019 From: slink at schokola.de (Nils Goroll) Date: Tue, 1 Oct 2019 13:37:21 +0200 Subject: data throughput In-Reply-To: References: Message-ID: <5930deef-c35e-a831-9795-d74904e814a1@schokola.de> On 01/10/2019 13:20, eric.m.vandenhout at kpn.com wrote: > varnish data throughput per second? You might want to look at |s_req_hdrbytes + | ||s_req_bodybytes + || |||s_resp_hdrbytes| + || |||s_resp_bodybytes| || varnishstat in curses (interactive) mode will also show a current "change" value, if your update rate is 1s(detailt), that will be per second. For longer intervals, I suggest you just snapshot the being/end value and calculate the average. Ref: https://varnish-cache.org/docs/trunk/reference/varnish-counters.html#varnish-counters-7 -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.m.vandenhout at kpn.com Tue Oct 1 11:55:55 2019 From: eric.m.vandenhout at kpn.com (eric.m.vandenhout at kpn.com) Date: Tue, 1 Oct 2019 11:55:55 +0000 Subject: data throughput In-Reply-To: <5930deef-c35e-a831-9795-d74904e814a1@schokola.de> References: <5930deef-c35e-a831-9795-d74904e814a1@schokola.de> Message-ID: Thnx. Will have a look at it. Van: Nils Goroll Verzonden: Tuesday, 1 October 2019 13:37 Aan: Hout, Eric van den ; varnish-misc at varnish-cache.org Onderwerp: Re: data throughput On 01/10/2019 13:20, eric.m.vandenhout at kpn.com wrote: varnish data throughput per second? You might want to look at s_req_hdrbytes + s_req_bodybytes + s_resp_hdrbytes + s_resp_bodybytes varnishstat in curses (interactive) mode will also show a current "change" value, if your update rate is 1s(detailt), that will be per second. For longer intervals, I suggest you just snapshot the being/end value and calculate the average. Ref: https://varnish-cache.org/docs/trunk/reference/varnish-counters.html#varnish-counters-7 -------------- next part -------------- An HTML attachment was scrubbed... URL: From kernelfreak at gmail.com Tue Oct 8 13:17:37 2019 From: kernelfreak at gmail.com (Akshay Sulakhe) Date: Tue, 8 Oct 2019 18:47:37 +0530 Subject: Varnish : Varnish not loading CSS and JS. Message-ID: <994f65ae-cb31-3f98-130b-25b7aade1bf1@gmail.com> Hello friends, I am running magento 2.2.6 on a nginx server. I have Varnish installed on another dedicated server. When I use it via proxy-pass and contact varnish server first, I am able to load the magento website, but CSS and JS are not loading due to strict mime-type issue. Error log : |pub/static/version123456/frontend/company/default/de_DE/css/styles-1.min.css was blocked due to MIME type ("text/html")mismatch (X-Content-Type-Options:nosniff) | We don't have no-sniff added in our Nginx configuration. Varnish config : default.vcl |backend server1c {.host ="IP1";.port ="80";.probe ={.url ="/balancer.html";.timeout =500s;.interval =5s;.window =5;.threshold =3;}}backend server1b {.host ="IP2";.port ="80";.probe ={.url ="/balancer.html";.timeout =500s;.interval =5s;.window =5;.threshold =3;}}backend server1a {.host ="IP3";.port ="80";.probe ={.url ="/pub/balancer.html";.timeout =500s;.interval =5s;.window =5;.threshold =3;}}subvcl_recv {# Happens before we check if we have this in cache already.## Typically you clean up the request here, removing cookies you don't need,# rewriting the request, etc.if(req.url =="/lazyload/prices/load/"){return(pass);}if(req.url =="/roobeo_config/store/changemode/mode/0"){return(pass);}if(req.url =="/roobeo_config/store/changemode/mode/1"){return(pass);}if(req.url =="/rest/default/V1/carts/mine/estimate-shipping-methods"){return(pass);}if(req.url =="/lazywishlist/"){return(pass);}if(req.url =="/index.php/rest/V1/"){return(pass);}if(req.url =="/customer/account/createpassword/"){return(pass);}if(req.url =="/customer/account/resetpasswordpost/"){return(pass);}if(req.url =="/roobeo_shipping/zipcode/validate/"){return(pass);}if(req.url =="/roobeo_shipping/zipcode/save/"){return(pass);}if(req.url =="/roobeo_shipping"){return(pass);}}subvcl_init {newbar =directors.round_robin();bar.add_backend(server1a);bar.add_backend(server1b);bar.add_backend(server1c);}| Nginx config : |location /{proxy_pass http://VARNISH_IP:80;proxy_set_header Host$http_host;proxy_set_header X-Forwarded-Host$http_host;proxy_set_header X-Real-IP $remote_addr;proxy_set_header X-Forwarded-For$proxy_add_x_forwarded_for;proxy_set_header Ssl-Offloaded"1";proxy_set_header X-Forwarded-Protohttps;proxy_set_header X-Forwarded-Port443;#proxy_hide_header X-Varnish;#proxy_hide_header Via;proxy_set_header X-Forwarded-Proto$scheme;try_files $uri $uri//index.php$is_args$args;} Any ideas. Thank you. :-) | || -------------- next part -------------- An HTML attachment was scrubbed... URL: From ciapnz at gmail.com Tue Oct 8 22:36:26 2019 From: ciapnz at gmail.com (Danila Vershinin) Date: Wed, 09 Oct 2019 01:36:26 +0300 Subject: Varnish : Varnish not loading CSS and JS. In-Reply-To: <994f65ae-cb31-3f98-130b-25b7aade1bf1@gmail.com> References: <994f65ae-cb31-3f98-130b-25b7aade1bf1@gmail.com> Message-ID: <1570574186.14515.0@gmail.com> Hi, I don't think it's related to Varnish. 1. You should not be using pub/ in your URLs. The secure Magento 2 setup assumes that pub/ directory is your web root, so you should end up with URLs without it. 2. Your CSS/JS is not loaded because they are returning 404s (thus text/html MIME type, and browser refuses to load them because they are not scripts/styles, they are HTML error message) On Tue, Oct 8, 2019 at 4:17 PM, Akshay Sulakhe wrote: > Hello friends, > > I am running magento 2.2.6 on a nginx server. I have Varnish > installed on another dedicated server. When I use it via proxy-pass > and contact varnish server first, I am able to load the magento > website, but CSS and JS are not loading due to strict mime-type issue. > > Error log : > > pub/static/version123456/frontend/company/default/de_DE/css/styles-1.min.css > was blocked due to MIME type ("text/html") mismatch > (X-Content-Type-Options: nosniff) > > We don't have no-sniff added in our Nginx configuration. > > Varnish config : > > default.vcl > > backend server1c { > .host = "IP1"; > .port = "80"; > .probe = { > .url = "/balancer.html"; > .timeout = 500s; > .interval = 5s; > .window = 5; > .threshold = 3; > } > } > > backend server1b { > .host = "IP2"; > .port = "80"; > .probe = { > .url = "/balancer.html"; > .timeout = 500s; > .interval = 5s; > .window = 5; > .threshold = 3; > } > } > > > backend server1a { > .host = "IP3"; > .port = "80"; > .probe = { > .url = "/pub/balancer.html"; > .timeout = 500s; > .interval = 5s; > .window = 5; > .threshold = 3; > } > } > > sub vcl_recv { > # Happens before we check if we have this in cache already. > # > # Typically you clean up the request here, removing cookies you > don't need, > # rewriting the request, etc. > if (req.url == "/lazyload/prices/load/") { > return(pass); > } > if (req.url == "/roobeo_config/store/changemode/mode/0") { > return(pass); > } > if (req.url == "/roobeo_config/store/changemode/mode/1") { > return(pass); > } > if (req.url == > "/rest/default/V1/carts/mine/estimate-shipping-methods") { > return(pass); > } > if (req.url == "/lazywishlist/") { > return(pass); > } > if (req.url == "/index.php/rest/V1/") { > return(pass); > } > if (req.url == "/customer/account/createpassword/") { > return(pass); > } > if (req.url == "/customer/account/resetpasswordpost/") { > return(pass); > } > if (req.url == "/roobeo_shipping/zipcode/validate/") { > return(pass); > } > if (req.url == "/roobeo_shipping/zipcode/save/") { > return(pass); > } > if (req.url == "/roobeo_shipping") { > return(pass); > } > } > > sub vcl_init { > new bar = directors.round_robin(); > bar.add_backend(server1a); > bar.add_backend(server1b); > bar.add_backend(server1c); > } > > Nginx config : > > location / { > proxy_pass http://VARNISH_IP:80; > proxy_set_header Host $http_host; > proxy_set_header X-Forwarded-Host $http_host; > proxy_set_header X-Real-IP $remote_addr; > proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; > proxy_set_header Ssl-Offloaded "1"; > proxy_set_header X-Forwarded-Proto https; > proxy_set_header X-Forwarded-Port 443; > #proxy_hide_header X-Varnish; > #proxy_hide_header Via; > proxy_set_header X-Forwarded-Proto $scheme; > try_files $uri $uri/ /index.php$is_args$args; > } > > Any ideas. Thank you. :-) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kernelfreak at gmail.com Wed Oct 9 06:48:48 2019 From: kernelfreak at gmail.com (Akshay Sulakhe) Date: Wed, 9 Oct 2019 12:18:48 +0530 Subject: Varnish : Varnish not loading CSS and JS. In-Reply-To: <1570574186.14515.0@gmail.com> References: <994f65ae-cb31-3f98-130b-25b7aade1bf1@gmail.com> <1570574186.14515.0@gmail.com> Message-ID: Hi, So the only problem you are describing is related to pub/, correct? But when I don't use varnish, then these files are loading properly... they are found and loaded. The pub/static and the path is the one used by magento, not by Varnish. Without varnish also the path remains same, and I get the CSS and JS. What changes would you recommend? Thank you. :-) On 09/10/19 4:06 AM, Danila Vershinin wrote: > Hi, > > I don't think it's related to Varnish. > > 1. You should not be using pub/ in your URLs. The secure Magento 2 > setup assumes that pub/ directory is your web root, so you should end > up with URLs without it. > 2. Your CSS/JS is not loaded because they are returning 404s (thus > text/html MIME type, and browser refuses to load them because they are > not scripts/styles, they are HTML error message) > > On Tue, Oct 8, 2019 at 4:17 PM, Akshay Sulakhe > wrote: >> >> Hello friends, >> >> I am running magento 2.2.6 on a nginx server. I have Varnish >> installed on another dedicated server. When I use it via proxy-pass >> and contact varnish server first, I am able to load the magento >> website, but CSS and JS are not loading due to strict mime-type issue. >> >> Error log : >> >> |pub/static/version123456/frontend/company/default/de_DE/css/styles-1.min.css >> was blocked due to MIME type ("text/html")mismatch >> (X-Content-Type-Options:nosniff) | >> >> We don't have no-sniff added in our Nginx configuration. >> >> Varnish config : >> >> default.vcl >> >> |backend server1c {.host ="IP1";.port ="80";.probe ={.url >> ="/balancer.html";.timeout =500s;.interval =5s;.window =5;.threshold >> =3;}}backend server1b {.host ="IP2";.port ="80";.probe ={.url >> ="/balancer.html";.timeout =500s;.interval =5s;.window =5;.threshold >> =3;}}backend server1a {.host ="IP3";.port ="80";.probe ={.url >> ="/pub/balancer.html";.timeout =500s;.interval =5s;.window >> =5;.threshold =3;}}subvcl_recv {# Happens before we check if we have >> this in cache already.## Typically you clean up the request here, >> removing cookies you don't need,# rewriting the request, >> etc.if(req.url =="/lazyload/prices/load/"){return(pass);}if(req.url >> =="/roobeo_config/store/changemode/mode/0"){return(pass);}if(req.url >> =="/roobeo_config/store/changemode/mode/1"){return(pass);}if(req.url >> =="/rest/default/V1/carts/mine/estimate-shipping-methods"){return(pass);}if(req.url >> =="/lazywishlist/"){return(pass);}if(req.url >> =="/index.php/rest/V1/"){return(pass);}if(req.url >> =="/customer/account/createpassword/"){return(pass);}if(req.url >> =="/customer/account/resetpasswordpost/"){return(pass);}if(req.url >> =="/roobeo_shipping/zipcode/validate/"){return(pass);}if(req.url >> =="/roobeo_shipping/zipcode/save/"){return(pass);}if(req.url >> =="/roobeo_shipping"){return(pass);}}subvcl_init {newbar >> =directors.round_robin();bar.add_backend(server1a);bar.add_backend(server1b);bar.add_backend(server1c);}| >> >> >> Nginx config : >> >> |location /{proxy_pass http://VARNISH_IP:80;proxy_set_header >> Host$http_host;proxy_set_header >> X-Forwarded-Host$http_host;proxy_set_header X-Real-IP >> $remote_addr;proxy_set_header >> X-Forwarded-For$proxy_add_x_forwarded_for;proxy_set_header >> Ssl-Offloaded"1";proxy_set_header >> X-Forwarded-Protohttps;proxy_set_header >> X-Forwarded-Port443;#proxy_hide_header X-Varnish;#proxy_hide_header >> Via;proxy_set_header X-Forwarded-Proto$scheme;try_files $uri >> $uri//index.php$is_args$args;} Any ideas. Thank you. :-) | >> >> >> || -------------- next part -------------- An HTML attachment was scrubbed... URL: From ciapnz at gmail.com Wed Oct 9 07:50:52 2019 From: ciapnz at gmail.com (Danila Vershinin) Date: Wed, 9 Oct 2019 10:50:52 +0300 Subject: Varnish : Varnish not loading CSS and JS. In-Reply-To: References: <994f65ae-cb31-3f98-130b-25b7aade1bf1@gmail.com> <1570574186.14515.0@gmail.com> Message-ID: I'm only saying you have way far from the correct configuration, and anything may happen :) Like try_files $uri $uri/ /index.php$is_args$args; alongside proxy_pass. I cannot see how this can work together, or the purpose of it. Best regards, Danila On Wed, Oct 9, 2019 at 9:48 AM Akshay Sulakhe wrote: > Hi, > > So the only problem you are describing is related to pub/, correct? But > when I don't use varnish, then these files are loading properly... they are > found and loaded. The pub/static and the path is the one used by magento, > not by Varnish. Without varnish also the path remains same, and I get the > CSS and JS. What changes would you recommend? Thank you. :-) > On 09/10/19 4:06 AM, Danila Vershinin wrote: > > Hi, > > I don't think it's related to Varnish. > > 1. You should not be using pub/ in your URLs. The secure Magento 2 setup > assumes that pub/ directory is your web root, so you should end up with > URLs without it. > 2. Your CSS/JS is not loaded because they are returning 404s (thus > text/html MIME type, and browser refuses to load them because they are not > scripts/styles, they are HTML error message) > > On Tue, Oct 8, 2019 at 4:17 PM, Akshay Sulakhe > wrote: > > Hello friends, > > I am running magento 2.2.6 on a nginx server. I have Varnish installed on > another dedicated server. When I use it via proxy-pass and contact varnish > server first, I am able to load the magento website, but CSS and JS are not > loading due to strict mime-type issue. > > Error log : > > pub/static/version123456/frontend/company/default/de_DE/css/styles-1.min.css was blocked due to MIME type ("text/html") mismatch (X-Content-Type-Options: nosniff) > > > We don't have no-sniff added in our Nginx configuration. > > Varnish config : > > default.vcl > > backend server1c { > .host = "IP1"; > .port = "80"; > .probe = { > .url = "/balancer.html"; > .timeout = 500s; > .interval = 5s; > .window = 5; > .threshold = 3; > }} > > backend server1b { > .host = "IP2"; > .port = "80"; > .probe = { > .url = "/balancer.html"; > .timeout = 500s; > .interval = 5s; > .window = 5; > .threshold = 3; > }} > > > backend server1a { > .host = "IP3"; > .port = "80"; > .probe = { > .url = "/pub/balancer.html"; > .timeout = 500s; > .interval = 5s; > .window = 5; > .threshold = 3; > }} > sub vcl_recv { > # Happens before we check if we have this in cache already. > # > # Typically you clean up the request here, removing cookies you don't need, > # rewriting the request, etc. > if (req.url == "/lazyload/prices/load/") { > return(pass); > } > if (req.url == "/roobeo_config/store/changemode/mode/0") { > return(pass); > } > if (req.url == "/roobeo_config/store/changemode/mode/1") { > return(pass); > } > if (req.url == "/rest/default/V1/carts/mine/estimate-shipping-methods") { > return(pass); > } > if (req.url == "/lazywishlist/") { > return(pass); > } > if (req.url == "/index.php/rest/V1/") { > return(pass); > } > if (req.url == "/customer/account/createpassword/") { > return(pass); > } > if (req.url == "/customer/account/resetpasswordpost/") { > return(pass); > } > if (req.url == "/roobeo_shipping/zipcode/validate/") { > return(pass); > } > if (req.url == "/roobeo_shipping/zipcode/save/") { > return(pass); > } > if (req.url == "/roobeo_shipping") { > return(pass); > }} > sub vcl_init { > new bar = directors.round_robin(); > bar.add_backend(server1a); > bar.add_backend(server1b); > bar.add_backend(server1c);} > > > Nginx config : > > location / { > proxy_pass http://VARNISH_IP:80; > proxy_set_header Host $http_host; > proxy_set_header X-Forwarded-Host $http_host; > proxy_set_header X-Real-IP $remote_addr; > proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; > proxy_set_header Ssl-Offloaded "1"; > proxy_set_header X-Forwarded-Proto https; > proxy_set_header X-Forwarded-Port 443; > #proxy_hide_header X-Varnish; > #proxy_hide_header Via; > proxy_set_header X-Forwarded-Proto $scheme; > try_files $uri $uri/ /index.php$is_args$args; > } > > Any ideas. Thank you. :-) > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From shafeequeaslam at gmail.com Wed Oct 9 08:28:39 2019 From: shafeequeaslam at gmail.com (shafeeque aslam) Date: Wed, 9 Oct 2019 13:58:39 +0530 Subject: Varnish : Varnish not loading CSS and JS. In-Reply-To: References: <994f65ae-cb31-3f98-130b-25b7aade1bf1@gmail.com> <1570574186.14515.0@gmail.com> Message-ID: Hi Akshay, Please find the CSS link and try to load the CSS link in a browser window and check the varnish and nginx logs. It seems like CSS links throwing 404. On Wed, Oct 9, 2019 at 12:19 PM Akshay Sulakhe wrote: > Hi, > > So the only problem you are describing is related to pub/, correct? But > when I don't use varnish, then these files are loading properly... they are > found and loaded. The pub/static and the path is the one used by magento, > not by Varnish. Without varnish also the path remains same, and I get the > CSS and JS. What changes would you recommend? Thank you. :-) > On 09/10/19 4:06 AM, Danila Vershinin wrote: > > Hi, > > I don't think it's related to Varnish. > > 1. You should not be using pub/ in your URLs. The secure Magento 2 setup > assumes that pub/ directory is your web root, so you should end up with > URLs without it. > 2. Your CSS/JS is not loaded because they are returning 404s (thus > text/html MIME type, and browser refuses to load them because they are not > scripts/styles, they are HTML error message) > > On Tue, Oct 8, 2019 at 4:17 PM, Akshay Sulakhe > wrote: > > Hello friends, > > I am running magento 2.2.6 on a nginx server. I have Varnish installed on > another dedicated server. When I use it via proxy-pass and contact varnish > server first, I am able to load the magento website, but CSS and JS are not > loading due to strict mime-type issue. > > Error log : > > pub/static/version123456/frontend/company/default/de_DE/css/styles-1.min.css was blocked due to MIME type ("text/html") mismatch (X-Content-Type-Options: nosniff) > > > We don't have no-sniff added in our Nginx configuration. > > Varnish config : > > default.vcl > > backend server1c { > .host = "IP1"; > .port = "80"; > .probe = { > .url = "/balancer.html"; > .timeout = 500s; > .interval = 5s; > .window = 5; > .threshold = 3; > }} > > backend server1b { > .host = "IP2"; > .port = "80"; > .probe = { > .url = "/balancer.html"; > .timeout = 500s; > .interval = 5s; > .window = 5; > .threshold = 3; > }} > > > backend server1a { > .host = "IP3"; > .port = "80"; > .probe = { > .url = "/pub/balancer.html"; > .timeout = 500s; > .interval = 5s; > .window = 5; > .threshold = 3; > }} > sub vcl_recv { > # Happens before we check if we have this in cache already. > # > # Typically you clean up the request here, removing cookies you don't need, > # rewriting the request, etc. > if (req.url == "/lazyload/prices/load/") { > return(pass); > } > if (req.url == "/roobeo_config/store/changemode/mode/0") { > return(pass); > } > if (req.url == "/roobeo_config/store/changemode/mode/1") { > return(pass); > } > if (req.url == "/rest/default/V1/carts/mine/estimate-shipping-methods") { > return(pass); > } > if (req.url == "/lazywishlist/") { > return(pass); > } > if (req.url == "/index.php/rest/V1/") { > return(pass); > } > if (req.url == "/customer/account/createpassword/") { > return(pass); > } > if (req.url == "/customer/account/resetpasswordpost/") { > return(pass); > } > if (req.url == "/roobeo_shipping/zipcode/validate/") { > return(pass); > } > if (req.url == "/roobeo_shipping/zipcode/save/") { > return(pass); > } > if (req.url == "/roobeo_shipping") { > return(pass); > }} > sub vcl_init { > new bar = directors.round_robin(); > bar.add_backend(server1a); > bar.add_backend(server1b); > bar.add_backend(server1c);} > > > Nginx config : > > location / { > proxy_pass http://VARNISH_IP:80; > proxy_set_header Host $http_host; > proxy_set_header X-Forwarded-Host $http_host; > proxy_set_header X-Real-IP $remote_addr; > proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; > proxy_set_header Ssl-Offloaded "1"; > proxy_set_header X-Forwarded-Proto https; > proxy_set_header X-Forwarded-Port 443; > #proxy_hide_header X-Varnish; > #proxy_hide_header Via; > proxy_set_header X-Forwarded-Proto $scheme; > try_files $uri $uri/ /index.php$is_args$args; > } > > Any ideas. Thank you. :-) > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -- Thanks & Regards * Shafeeque Aslam A.S* * +91-9633-506-503 **https://teq.kefh.in * https://www.linkedin.com/in/shafeequeaslam -------------- next part -------------- An HTML attachment was scrubbed... URL: From miguel_3_gonzalez at yahoo.es Wed Oct 9 09:25:35 2019 From: miguel_3_gonzalez at yahoo.es (Miguel Gonzalez) Date: Wed, 09 Oct 2019 11:25:35 +0200 Subject: Migration from 4.1 vcl Message-ID: Hi, I am migrating from 4.1 and I am scared that with latest version varnish would break. What should I take into account when migrating from 4.1? Regards, Miguel -------------- next part -------------- An HTML attachment was scrubbed... URL: From dridi at varni.sh Wed Oct 9 09:39:08 2019 From: dridi at varni.sh (Dridi Boukelmoune) Date: Wed, 9 Oct 2019 09:39:08 +0000 Subject: Migration from 4.1 vcl In-Reply-To: References: Message-ID: On Wed, Oct 9, 2019 at 9:27 AM Miguel Gonzalez wrote: > > Hi, > > I am migrating from 4.1 and I am scared that with latest version varnish would break. > > What should I take into account when migrating from 4.1? Probably mostly third-party VMODs, since Varnish 6.x still supports the VCL 4.0 syntax. There may be other non-VCL breaking changes and I recommend you go through the release notes from 5.0 to 6.x, whichever you choose (latest is 6.3 and LTS is 6.0). Dridi From cosimo at streppone.it Wed Oct 9 09:56:44 2019 From: cosimo at streppone.it (Cosimo Streppone) Date: Wed, 09 Oct 2019 11:56:44 +0200 Subject: Migration from 4.1 vcl In-Reply-To: References: Message-ID: <0e8932d8-d134-4333-a3a1-531416fc5889@www.fastmail.com> On Wed, Oct 9, 2019, at 11:39, Dridi Boukelmoune wrote: > On Wed, Oct 9, 2019 at 9:27 AM Miguel Gonzalez wrote: > > > > I am migrating from 4.1 and I am scared that with latest version varnish would break. > > What should I take into account when migrating from 4.1? > > Probably mostly third-party VMODs, since Varnish 6.x still supports > the VCL 4.0 syntax. Any specific info on migration of vmods? I mostly use the "bundled" vmods[1], but occasionally others like libvmod-sqlite3[2]. If I wanted to convert or upgrade them, do you have any hints or info on what is likely to break and why? Can these vmods still be built in the same way with 6.0? Thanks, -- Cosimo [1] https://github.com/varnish/varnish-modules [2] https://github.com/fgsch/libvmod-sqlite3 From dridi at varni.sh Wed Oct 9 15:53:00 2019 From: dridi at varni.sh (Dridi Boukelmoune) Date: Wed, 9 Oct 2019 15:53:00 +0000 Subject: Migration from 4.1 vcl In-Reply-To: <0e8932d8-d134-4333-a3a1-531416fc5889@www.fastmail.com> References: <0e8932d8-d134-4333-a3a1-531416fc5889@www.fastmail.com> Message-ID: > > Probably mostly third-party VMODs, since Varnish 6.x still supports > > the VCL 4.0 syntax. > > Any specific info on migration of vmods? > I mostly use the "bundled" vmods[1], but occasionally others like libvmod-sqlite3[2]. > > If I wanted to convert or upgrade them, do you have any hints or info on what is likely to break and why? > Can these vmods still be built in the same way with 6.0? The problem with third-party VMODs is that we don't control them, so if some idiot [1] decides to change a VMOD's API then VCL making use of it will need some amount of rewriting when upgrading. You shouldn't have a problem with bundled VMODs, otherwise we made a mistake. When it comes to third-party VMODS an upgrade from Varnish 4.1 to 6.x needs to be studied on a case-by-case basis. Worst case scenario the module is not even available for the Varnish version you with to upgrade to. Dridi [1] https://github.com/dridi/libvmod-querystring From cosimo at streppone.it Wed Oct 9 16:03:04 2019 From: cosimo at streppone.it (Cosimo Streppone) Date: Wed, 09 Oct 2019 18:03:04 +0200 Subject: Migration from 4.1 vcl In-Reply-To: References: <0e8932d8-d134-4333-a3a1-531416fc5889@www.fastmail.com> Message-ID: On Wed, Oct 9, 2019, at 17:53, Dridi Boukelmoune wrote: > > > Probably mostly third-party VMODs, since Varnish 6.x still supports > > > the VCL 4.0 syntax. > > > > Any specific info on migration of vmods? > > I mostly use the "bundled" vmods[1], but occasionally others like libvmod-sqlite3[2]. > > When it comes to third-party VMODS an upgrade from Varnish 4.1 to 6.x > needs to be studied on a case-by-case basis. Worst case scenario the > module is not even available for the Varnish version you with to > upgrade to. What I meant was if there's any vmod interface/api change between 4.1 and 6.0 that could break third party vmod code, if that makes any sense. In that case, if a vmod I'd like to use on 6.x doesn't exist or hasn't been upgraded yet, I could perhaps try and do that myself. Maybe too much of a generic question :-) -- Cosimo From geoff at uplex.de Wed Oct 9 16:12:33 2019 From: geoff at uplex.de (Geoff Simmons) Date: Wed, 9 Oct 2019 18:12:33 +0200 Subject: Migration from 4.1 vcl In-Reply-To: References: <0e8932d8-d134-4333-a3a1-531416fc5889@www.fastmail.com> Message-ID: On 10/9/19 18:03, Cosimo Streppone wrote: > > What I meant was if there's any vmod interface/api change between 4.1 and 6.0 > that could break third party vmod code, if that makes any sense. Yes. With just about every new version of Varnish, there's always something. It may be not be much for each version (for Varnish 6.3.0, for example, you need to replace WS_Reserve() with WS_ReserveSize(), or else you get deprecation warnings), but enough so that some kind of fix is often necessary. If you're jumping from 4.1 to 6.0, chances are that quite a few of things accumulated over time. The What's New/Upgrading docs for the various versions often have a section about changes for developers of VMODs and other third-party software, so you might want to look through those. The log in changes.rst in the git repo also goes into such changes, in more technical detail. Good luck, Geoff -- ** * * UPLEX - Nils Goroll Systemoptimierung Scheffelstra?e 32 22301 Hamburg Tel +49 40 2880 5731 Mob +49 176 636 90917 Fax +49 40 42949753 http://uplex.de -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From jpotter-varnish at codepuppy.com Tue Oct 15 14:21:03 2019 From: jpotter-varnish at codepuppy.com (Jeff Potter) Date: Tue, 15 Oct 2019 10:21:03 -0400 Subject: How to send only whitelisted http headers to backend? Message-ID: Hi All, This seems like an easy task, but I haven?t been able to figure out how to do it or find any posts online. Is there a way to only send certain headers to a backend? I.e. in our application, we know we only need X-Forwarded-For and Cookie headers. I know I can unset other known headers (User-Agent, etc) ? but how can I unset *all* other headers? (We?re on VCL format 4.0.) Thanks! -Jeff From dridi at varni.sh Tue Oct 15 21:46:37 2019 From: dridi at varni.sh (Dridi Boukelmoune) Date: Tue, 15 Oct 2019 21:46:37 +0000 Subject: How to send only whitelisted http headers to backend? In-Reply-To: References: Message-ID: On Tue, Oct 15, 2019 at 2:22 PM Jeff Potter wrote: > > > Hi All, > > This seems like an easy task, but I haven?t been able to figure out how to do it or find any posts online. Is there a way to only send certain headers to a backend? > > I.e. in our application, we know we only need X-Forwarded-For and Cookie headers. I know I can unset other known headers (User-Agent, etc) ? but how can I unset *all* other headers? > > (We?re on VCL format 4.0.) Hi Jeff, This is not doable in VCL, this kind of header whitelisting could be implemented with a VMOD but I'm not aware of any one doing that. Dridi From geoff at uplex.de Wed Oct 16 16:06:38 2019 From: geoff at uplex.de (Geoff Simmons) Date: Wed, 16 Oct 2019 18:06:38 +0200 Subject: How to send only whitelisted http headers to backend? In-Reply-To: References: Message-ID: <2b7dcc68-b9f3-e2a5-80af-f3fe7e066c9a@uplex.de> On 10/15/19 16:21, Jeff Potter wrote: > > This seems like an easy task, but I haven?t been able to figure out > how to do it or find any posts online. Is there a way to only send > certain headers to a backend? > > I.e. in our application, we know we only need X-Forwarded-For and > Cookie headers. I know I can unset other known headers (User-Agent, etc) > ? but how can I unset *all* other headers? VMOD re2 has the .hdr_filter() method for the set object: https://code.uplex.de/uplex-varnish/libvmod-re2 https://code.uplex.de/uplex-varnish/libvmod-re2/blob/master/README.rst#L1775 VOID myset.hdr_filter(HTTP, BOOL whitelist) The HTTP parameter can be one of req, resp, bereq or beresp. If the whitelist parameter is true (default true), then only matching headers are retained. Otherwise it's a blacklist -- matching headers are removed. So for your use case: sub vcl_init { new whitelist = re2.set(anchor=start, case_sensitive=false); whitelist.add("X-Forwarded-For:"); whitelist.add("Cookie:"); whitelist.add("Host:"); whitelist.compile(); } sub vcl_backend_fetch { whitelist.hdr_filter(bereq); } I took the liberty of adding the Host header to your whitelist, since it's required since HTTP/1.1. Even if your backends "happen" to work without it, I wouldn't leave it out, since it's not well-formed HTTP otherwise (might stop working, for example, if the backend apps are upgraded). HTH, Geoff -- ** * * UPLEX - Nils Goroll Systemoptimierung Scheffelstra?e 32 22301 Hamburg Tel +49 40 2880 5731 Mob +49 176 636 90917 Fax +49 40 42949753 http://uplex.de -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From dridi at varni.sh Wed Oct 16 17:36:56 2019 From: dridi at varni.sh (Dridi Boukelmoune) Date: Wed, 16 Oct 2019 17:36:56 +0000 Subject: How to send only whitelisted http headers to backend? In-Reply-To: <2b7dcc68-b9f3-e2a5-80af-f3fe7e066c9a@uplex.de> References: <2b7dcc68-b9f3-e2a5-80af-f3fe7e066c9a@uplex.de> Message-ID: On Wed, Oct 16, 2019 at 4:08 PM Geoff Simmons wrote: > > On 10/15/19 16:21, Jeff Potter wrote: > > > > This seems like an easy task, but I haven?t been able to figure out > > how to do it or find any posts online. Is there a way to only send > > certain headers to a backend? > > > > I.e. in our application, we know we only need X-Forwarded-For and > > Cookie headers. I know I can unset other known headers (User-Agent, etc) > > ? but how can I unset *all* other headers? > > VMOD re2 has the .hdr_filter() method for the set object: > > https://code.uplex.de/uplex-varnish/libvmod-re2 > > https://code.uplex.de/uplex-varnish/libvmod-re2/blob/master/README.rst#L1775 > > VOID myset.hdr_filter(HTTP, BOOL whitelist) > > The HTTP parameter can be one of req, resp, bereq or beresp. If the > whitelist parameter is true (default true), then only matching headers > are retained. Otherwise it's a blacklist -- matching headers are removed. > > So for your use case: > > sub vcl_init { > new whitelist = re2.set(anchor=start, case_sensitive=false); > whitelist.add("X-Forwarded-For:"); > whitelist.add("Cookie:"); > whitelist.add("Host:"); > whitelist.compile(); > } > > sub vcl_backend_fetch { > whitelist.hdr_filter(bereq); > } TIL, thanks! > I took the liberty of adding the Host header to your whitelist, since > it's required since HTTP/1.1. Even if your backends "happen" to work > without it, I wouldn't leave it out, since it's not well-formed HTTP > otherwise (might stop working, for example, if the backend apps are > upgraded). Agreed, there are other control headers that one may want to keep in the whitelist, otherwise you may break conditional or partial requests, and everything else I don't remember off the top of my head. Dridi From jpotter-varnish at codepuppy.com Thu Oct 17 12:49:02 2019 From: jpotter-varnish at codepuppy.com (Jeff Potter) Date: Thu, 17 Oct 2019 08:49:02 -0400 Subject: How to send only whitelisted http headers to backend? In-Reply-To: References: <2b7dcc68-b9f3-e2a5-80af-f3fe7e066c9a@uplex.de> Message-ID: <4AAFC050-B560-4EDE-AFC5-79DA6FFE4B30@codepuppy.com> Thanks, Geoff and Dridi! We?ll give this a try. And Dridi, thanks also for maintaining varnish and this list ? ?long time lurker; very rare poster? ? since I have the microphone, just wanted to send a short note of appreciation. -Jeff > On Oct 16, 2019, at 1:36 PM, Dridi Boukelmoune wrote: > > On Wed, Oct 16, 2019 at 4:08 PM Geoff Simmons wrote: >> >> On 10/15/19 16:21, Jeff Potter wrote: >>> >>> This seems like an easy task, but I haven?t been able to figure out >>> how to do it or find any posts online. Is there a way to only send >>> certain headers to a backend? >>> >>> I.e. in our application, we know we only need X-Forwarded-For and >>> Cookie headers. I know I can unset other known headers (User-Agent, etc) >>> ? but how can I unset *all* other headers? >> >> VMOD re2 has the .hdr_filter() method for the set object: >> >> https://code.uplex.de/uplex-varnish/libvmod-re2 >> >> https://code.uplex.de/uplex-varnish/libvmod-re2/blob/master/README.rst#L1775 >> >> VOID myset.hdr_filter(HTTP, BOOL whitelist) >> >> The HTTP parameter can be one of req, resp, bereq or beresp. If the >> whitelist parameter is true (default true), then only matching headers >> are retained. Otherwise it's a blacklist -- matching headers are removed. >> >> So for your use case: >> >> sub vcl_init { >> new whitelist = re2.set(anchor=start, case_sensitive=false); >> whitelist.add("X-Forwarded-For:"); >> whitelist.add("Cookie:"); >> whitelist.add("Host:"); >> whitelist.compile(); >> } >> >> sub vcl_backend_fetch { >> whitelist.hdr_filter(bereq); >> } > > TIL, thanks! > >> I took the liberty of adding the Host header to your whitelist, since >> it's required since HTTP/1.1. Even if your backends "happen" to work >> without it, I wouldn't leave it out, since it's not well-formed HTTP >> otherwise (might stop working, for example, if the backend apps are >> upgraded). > > Agreed, there are other control headers that one may want to keep in > the whitelist, otherwise you may break conditional or partial requests, > and everything else I don't remember off the top of my head. From dridi at varni.sh Fri Oct 18 09:39:36 2019 From: dridi at varni.sh (Dridi Boukelmoune) Date: Fri, 18 Oct 2019 09:39:36 +0000 Subject: How to send only whitelisted http headers to backend? In-Reply-To: <4AAFC050-B560-4EDE-AFC5-79DA6FFE4B30@codepuppy.com> References: <2b7dcc68-b9f3-e2a5-80af-f3fe7e066c9a@uplex.de> <4AAFC050-B560-4EDE-AFC5-79DA6FFE4B30@codepuppy.com> Message-ID: On Thu, Oct 17, 2019 at 12:50 PM Jeff Potter wrote: > > > Thanks, Geoff and Dridi! We?ll give this a try. > > And Dridi, thanks also for maintaining varnish and this list ? ?long time lurker; very rare poster? ? since I have the microphone, just wanted to send a short note of appreciation. Very appreciated too, but you are crediting me much more than I deserve ;-) PHK, Martin and Nils are the current maintainers and someone from Uplex is maintaining this list. Dridi From yassine.aouadi90 at gmail.com Fri Oct 18 15:31:54 2019 From: yassine.aouadi90 at gmail.com (Yassine Aouadi) Date: Fri, 18 Oct 2019 17:31:54 +0200 Subject: Varnish version 6.3 vcl test fails with custom http header starting with a number Message-ID: Message from VCC-compiler: Symbol 'req.http' has wrong type (none): At: ('/etc/varnish/default.vcl' Line 182 Pos 9) -- (Pos 16) set req.http.7hello = "world"; --------########------------------- Symbol was builtin Running VCC-compiler failed, exited with 2 VCL compilation failed same code works when using a letter before the number : set req.http.x7hello = "world"; before submitting a bug a report in github wanted to check is someone is familiar with this issue ? Thanks, --yaouadi -------------- next part -------------- An HTML attachment was scrubbed... URL: From dridi at varni.sh Mon Oct 21 09:14:14 2019 From: dridi at varni.sh (Dridi Boukelmoune) Date: Mon, 21 Oct 2019 09:14:14 +0000 Subject: Varnish version 6.3 vcl test fails with custom http header starting with a number In-Reply-To: References: Message-ID: > before submitting a bug a report in github wanted to check is someone is familiar with this issue ? Can you tell us from which version you are upgrading to 6.3? I know that at some point header names parsing changed so that they'd have to be proper VCL symbols, leaving no room for otherwise valid HTTP headers that don't happen to match VCL expectations. We have no alternate syntax for "exotic" header names, I suggested req.http[7hello], someone else suggested req.http."7hello" but nothing happened in this direction. So your only solution today is to use vmod_header. https://github.com/varnishcache/varnish-cache/issues/2573 Dridi From slink at schokola.de Mon Oct 21 10:27:02 2019 From: slink at schokola.de (Nils Goroll) Date: Mon, 21 Oct 2019 12:27:02 +0200 Subject: BAN variant of varnish cache In-Reply-To: References: Message-ID: <96011a90-ec2a-6663-7c21-58e8cf3e3585@schokola.de> (redirecting the response to varnish-misc, where the question would have belonged) On 14/10/2019 16:13, Santosh Abraham wrote: > Using *BAN*, is it possible to invalidate a particular variant of a varnish cache? What you are trying to do should just work, and I cannot spot an error in your VCL with regards to the ban handling. But your use of the Vary header looks wrong, It is not a request header but rather a response header (which you need to set in vcl_backend_response) and also the value if the Vary header is the name of a request header. So in your case it should be something like: sub vcl_backend_response { if (beresp.http.Vary) { set beresp.http.Vary = beresp.http.Vary + ", X-BOLT-SITE-LOCALE"; } else { set beresp.http.Vary = "X-BOLT-SITE-LOCALE"; } } Nils -------------- next part -------------- An HTML attachment was scrubbed... URL: From yassine.aouadi90 at gmail.com Mon Oct 21 13:21:15 2019 From: yassine.aouadi90 at gmail.com (Yassine Aouadi) Date: Mon, 21 Oct 2019 15:21:15 +0200 Subject: Varnish version 6.3 vcl test fails with custom http header starting with a number In-Reply-To: References: Message-ID: >Can you tell us from which version you are upgrading to 6.3? I am upgrading from 5.2.1 but same code tested and worked with varnish 6.0.5 lts and 6.2.1 (all downloaded from package cloud) on ubuntu bionic. Testing 6 series version to choose to which one I should upgrade . Sadly ,Some backends only accept requests with an "exotic" header name . Should I expect this restriction included in a new 6.0 .* lts package as well ? If I have no choice and have to import the header vmod would you advise how can i set brand new key-value http header in vcl using this vmod? I only saw operations on existing headers in doc : https://github.com/varnish/varnish-modules/blob/master/docs/vmod_header.rst Thanks, --Yassine On Mon, Oct 21, 2019 at 11:14 AM Dridi Boukelmoune wrote: > > before submitting a bug a report in github wanted to check is someone is > familiar with this issue ? > > Can you tell us from which version you are upgrading to 6.3? > > I know that at some point header names parsing changed so that they'd > have to be proper VCL symbols, leaving no room for otherwise valid > HTTP headers that don't happen to match VCL expectations. We have no > alternate syntax for "exotic" header names, I suggested req.http[7hello], > someone else suggested req.http."7hello" but nothing happened in this > direction. So your only solution today is to use vmod_header. > > https://github.com/varnishcache/varnish-cache/issues/2573 > > Dridi > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Tue Oct 22 05:30:42 2019 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Mon, 21 Oct 2019 22:30:42 -0700 Subject: PSA: Buster support for weeklies Message-ID: Hello everyone, TL;DR: there are buster weeklies, please test so we can soo have official support for regular releases. Get them here: https://packagecloud.io/varnishcache/varnish-weekly This has been a frequently asked question since the Debian Buster release,so I wanted to report progress on the "where are the buster packages front?". The weeklies are historically produced by Varnish Software's Jenkins systems, but these last few days we have been working on migrating these jobs to CircleCI, meaning that the CI pipeline is now open to everyone: https://circleci.com/gh/varnishcache/workflows/varnish-cache/tree/master Note that we get now packages for every commit, but we also distcheck on a few distributions, if you have sensible additions to these, well, the .circle/config.yml file lives in the varnish-cache repo, so, PRs are welcome. Regarding packages, the goal is to push them as weeklies for a few...weeks, to gather feedback before gaining official support. So you are definitely encouraged to test these out: https://packagecloud.io/varnishcache/varnish-weekly We are notably very interested in feedback regarding the memory usage. In Buster, our allocator, JEMalloc is now at version 5, so a lot has changed, and some configuration may need tweaking. Once we are confident that both our dependencies and the packaging ( https://github.com/varnishcache/pkg-varnish-cache) is solid, we'll package regular releases for buster. Go and try them out! -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From ewasser at maxcluster.de Wed Oct 23 09:37:18 2019 From: ewasser at maxcluster.de (Erik Wasser) Date: Wed, 23 Oct 2019 11:37:18 +0200 Subject: Affected 5.x versions of VSV00004 Workspace information leak Message-ID: <957fdd2b-bfe4-803a-6867-f668ef745d55@maxcluster.de> Hello list, sometimes I'm confused about the supported versions of Varnish. This resulted in the post "LTS time frame for Varnish 6.0.X?" on https://varnish-cache.org/lists/pipermail/varnish-dist/2019-September/000173.html. But now I'm confused about the "VSV00004 Workspace information leak" (https://varnish-cache.org/security/VSV00004.html) and the affected versions. "VSV00004 Workspace information leak" writes: > Versions affected > > 5.0 and forward So the version 5.0, 5.1 and 5.2 are affected by VSV00004, is that correct? The page http://varnish-cache.org/releases/index.html states that only versions 6.X are supported. So all varnish 5.X should update to 6.X?! Is that conclusion correct? -- Greetings Erik Wasser From dridi at varni.sh Wed Oct 23 10:41:22 2019 From: dridi at varni.sh (Dridi Boukelmoune) Date: Wed, 23 Oct 2019 10:41:22 +0000 Subject: Affected 5.x versions of VSV00004 Workspace information leak In-Reply-To: <957fdd2b-bfe4-803a-6867-f668ef745d55@maxcluster.de> References: <957fdd2b-bfe4-803a-6867-f668ef745d55@maxcluster.de> Message-ID: On Wed, Oct 23, 2019 at 9:38 AM Erik Wasser wrote: > > Hello list, > > sometimes I'm confused about the supported versions of Varnish. This > resulted in the post "LTS time frame for Varnish 6.0.X?" on > https://varnish-cache.org/lists/pipermail/varnish-dist/2019-September/000173.html. > > But now I'm confused about the "VSV00004 Workspace information leak" > (https://varnish-cache.org/security/VSV00004.html) and the affected > versions. > > "VSV00004 Workspace information leak" writes: > > > Versions affected > > > > 5.0 and forward > > So the version 5.0, 5.1 and 5.2 are affected by VSV00004, is that > correct? The page http://varnish-cache.org/releases/index.html states > that only versions 6.X are supported. So all varnish 5.X should update > to 6.X?! Is that conclusion correct? Correct, and if you want some stability I recommend the 6.0 LTS branch that will be maintained for a while, like the previous 4.1 LTS branch that reached EOL in March 2019. Dridi From slink at schokola.de Tue Oct 29 08:19:25 2019 From: slink at schokola.de (Nils Goroll) Date: Tue, 29 Oct 2019 09:19:25 +0100 Subject: varnish don't set malloc size,varnish cache data only 200M size In-Reply-To: References: Message-ID: <13967bb6-22dd-f25c-e2ef-16a246a91fcb@schokola.de> You are right, the documentation is wrong. The default is 100m I opened https://github.com/varnishcache/varnish-cache/issues/3108 - thank you for reporting. The only way to get a bigger cache is to configure the maximum size. On 29/10/2019 04:43, li yiyi wrote: > hi, i use varnish6.2.0 to cache image, my machine memory is 16g, varnish > malloc do not set size and use the default? > i see the documentation from?varnish-cache.org, the default size is unlimited. > my test images size about 6g,when i send http request to > varnish,only a little images can cache from varnish, about 200m size. most of > them is miss. > my some varnish settings as follows: > /usr/local/varnish/sbin/varnishd -a 0.0.0.0:80,HTTP -f > /usr/local/varnish/etc/default.vcl > when i set malloc size(8g),varnish is working ok, malloc must be seted specify > size?how i can do if use?unlimited ?? > > thank you?regards > -------------------------------------------------------------------------------- > > > _______________________________________________ > varnish-commit mailing list > varnish-commit at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-commit -------------- next part -------------- An HTML attachment was scrubbed... URL: From slink at schokola.de Tue Oct 29 08:22:04 2019 From: slink at schokola.de (Nils Goroll) Date: Tue, 29 Oct 2019 09:22:04 +0100 Subject: varnish don't set malloc size,varnish cache data only 200M size In-Reply-To: <13967bb6-22dd-f25c-e2ef-16a246a91fcb@schokola.de> References: <13967bb6-22dd-f25c-e2ef-16a246a91fcb@schokola.de> Message-ID: Sorry, my response was wrong: If no -s argument is given, the default is - 100m malloc for cache objects - unbound transient storage If -smalloc is given, then the size is actually unbound. Nils On 29/10/2019 09:19, Nils Goroll wrote: > You are right, the documentation is wrong. The default is 100m > > I opened https://github.com/varnishcache/varnish-cache/issues/3108 - thank you > for reporting. > > The only way to get a bigger cache is to configure the maximum size. > > On 29/10/2019 04:43, li yiyi wrote: >> hi, i use varnish6.2.0 to cache image, my machine memory is 16g, varnish >> malloc do not set size and use the default? >> i see the documentation from?varnish-cache.org, the default size is >> unlimited. my test images size about 6g,when i send http request to >> varnish,only a little images can cache from varnish, about 200m size. most of >> them is miss. >> my some varnish settings as follows: >> /usr/local/varnish/sbin/varnishd -a 0.0.0.0:80,HTTP -f >> /usr/local/varnish/etc/default.vcl >> when i set malloc size(8g),varnish is working ok, malloc must be seted >> specify size?how i can do if use?unlimited ?? >> >> thank you?regards >> -------------------------------------------------------------------------------- >> >> >> _______________________________________________ >> varnish-commit mailing list >> varnish-commit at varnish-cache.org >> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-commit > > > > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc -------------- next part -------------- An HTML attachment was scrubbed... URL: From rainer.canavan+varnish at sevenval.com Thu Oct 31 16:16:22 2019 From: rainer.canavan+varnish at sevenval.com (Rainer Canavan) Date: Thu, 31 Oct 2019 17:16:22 +0100 Subject: varnishncsa %s in backend mode: 304 logged as 200 Message-ID: Hi, we want to get a better understanding how well our varnish setup is performing. We've got two varnishncsa running, one with '-b' and one without, and each with their own logformat configuration. Each of those configurations contains "status":%s. Varnish version is varnish-6.2.1. If varnish successfully revalidates a stored response, i would expect the backend log to show a status of 304, and the client log a status of 200, but in reality, the vanishncsa running with "-b" also reports a status of 200. Just to make sure, I've watched with tcpdump what's going on, and varnish does really receive a 304 from the backend. The client sends GET /test/5db010a2-07a5-4119-b90b-ed7328c42231 HTTP/1.1 Host: cachetest.backend [...] and receives from varnish: HTTP/1.1 200 OK Cache-Control: max-age=3600 Date: Thu, 31 Oct 2019 14:43:19 GMT Etag: "ASKCUMEWOGYQIAS" [...] The same request between varnish and the backend: GET /test/5db010a2-07a5-4119-b90b-ed7328c42231 HTTP/1.1 Host: cachetest.backend If-None-Match: "ASKCUMEWOGYQIAS" [...] HTTP/1.1 304 Not Modified Cache-Control: max-age=3600 Date: Thu, 31 Oct 2019 14:43:19 GMT Etag: "ASKCUMEWOGYQIAS" Connection: close Is there another statement for the varnishncsa log format that shows the "real" backend HTTP status? Does this work as intended, or is this a bug? Rainer From guillaume at varnish-software.com Thu Oct 31 16:48:26 2019 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Thu, 31 Oct 2019 09:48:26 -0700 Subject: varnishncsa %s in backend mode: 304 logged as 200 In-Reply-To: References: Message-ID: Hi, Varnish automatically transforms the 304 response into the object it represents, this is why you are seeing the 200 (but it could be pretty much any status that you are revalidating). If you want to check for 304s, you can do something like this: sub vcl_backend_response { if (beresp.was_304) { std.log("true_status: 304"); } else { std.log("true_status: " + beresp.status); } } and then use %{VSL:VCL_Log:true_status}x in the varnishncsa format line to retrieve it -- Guillaume Quintard On Thu, Oct 31, 2019 at 9:17 AM Rainer Canavan < rainer.canavan+varnish at sevenval.com> wrote: > Hi, > > we want to get a better understanding how well our varnish setup is > performing. We've got two varnishncsa running, one with '-b' and one > without, and each with their own logformat configuration. Each of > those configurations contains "status":%s. Varnish version is > varnish-6.2.1. > > If varnish successfully revalidates a stored response, i would expect > the backend log to show a status of 304, and the client log a status > of 200, but in reality, the vanishncsa running with "-b" also reports > a status of 200. > > Just to make sure, I've watched with tcpdump what's going on, and > varnish does really receive a 304 from the backend. The client sends > > GET /test/5db010a2-07a5-4119-b90b-ed7328c42231 HTTP/1.1 > Host: cachetest.backend > [...] > > and receives from varnish: > > HTTP/1.1 200 OK > Cache-Control: max-age=3600 > Date: Thu, 31 Oct 2019 14:43:19 GMT > Etag: "ASKCUMEWOGYQIAS" > [...] > > The same request between varnish and the backend: > > GET /test/5db010a2-07a5-4119-b90b-ed7328c42231 HTTP/1.1 > Host: cachetest.backend > If-None-Match: "ASKCUMEWOGYQIAS" > [...] > > HTTP/1.1 304 Not Modified > Cache-Control: max-age=3600 > Date: Thu, 31 Oct 2019 14:43:19 GMT > Etag: "ASKCUMEWOGYQIAS" > Connection: close > > Is there another statement for the varnishncsa log format that shows > the "real" backend HTTP status? Does this work as intended, or is this > a bug? > > Rainer > _______________________________________________ > varnish-misc mailing list > varnish-misc at varnish-cache.org > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rainer.canavan+varnish at sevenval.com Thu Oct 31 17:14:24 2019 From: rainer.canavan+varnish at sevenval.com (Rainer Canavan) Date: Thu, 31 Oct 2019 18:14:24 +0100 Subject: varnishncsa %s in backend mode: 304 logged as 200 In-Reply-To: References: Message-ID: On Thu, Oct 31, 2019 at 5:48 PM Guillaume Quintard < guillaume at varnish-software.com> wrote: > > Hi, > > Varnish automatically transforms the 304 response into the object it represents, this is why you are seeing the 200 (but it could be pretty much any status that you are revalidating). > If you want to check for 304s, you can do something like this: > > sub vcl_backend_response { > if (beresp.was_304) { > std.log("true_status: 304"); > } else { > std.log("true_status: " + beresp.status); > } > } > > and then use %{VSL:VCL_Log:true_status}x in the varnishncsa format line to retrieve it Thanks, that construct logs exactly what I wanted. Rainer -------------- next part -------------- An HTML attachment was scrubbed... URL: