From hrhosseini at hotmail.com Mon Aug 3 11:26:37 2020 From: hrhosseini at hotmail.com (hamidreza hosseini) Date: Mon, 3 Aug 2020 11:26:37 +0000 Subject: Error in varnish child Message-ID: Hi I have this error on my varnish instance , is this important or i can ignore it?: varnish-03 varnishd/varnish[43288]: Child (110695) Last panic at: Mon, 06 Jul 2020 10:20:46 GMT "Assert error in child_sigsegv_handler(), mgt/mgt_child.c line 282: Condition(Segmentation fault by instruction at 0x7f1481c75e70) not true. errno = 11 (Resource temporarily unavailable) thread = (cache-worker) version = varnish-4.1.1 revision 66bb824 ident = Linux,4.4.0-174-generic,x86_64,-junix,-smalloc,-smalloc,-hcritbit,epoll Backtrace: 0x433295: varnishd() [0x433295] 0x452543: varnishd() [0x452543] 0x7fe2ebc0e390: libpthread.so.0(+0x11390) [0x7fe2ebc0e390] 0x42e82d: varnishd(Lck__Lock+0xd) [0x42e82d] 0x44cde9: varnishd() [0x44cde9] 0x429122: varnishd(HSH_Lookup+0x512) [0x429122] 0x43743a: varnishd(CNT_Request+0x17ba) [0x43743a] 0x44e49e: varnishd(HTTP1_Session+0xbe) [0x44e49e] 0x43952d: varnishd(SES_Proto_Req+0x5d) [0x43952d] 0x448e4a: varnishd() [0x448e4a] req = 0x7f3ffcc89020 { vxid = 839124106, step = R_STP_LOOKUP, req_body = R_BODY_NONE, restarts = 0, esi_level = 0, sp = 0x7f10b2f9fa20 { fd = 37, vxid = 839123508, client = 172.20.30.235 37944, step = S_STP_H1PROC, }, worker = 0x7f0683bf2c80 { stack = {0x7f0683bf3000 -> 0x7f0683be7000}, ws = 0x7f0683bf2e78 { id = \"wrk\", {s,f,r,e} = {0x7f0683bf2420,0x7f0683bf2420,(nil),+2040}, }, VCL::method = HASH, VCL::return = lookup, VCL::methods = {RECV, PASS, HASH, MISS, HIT, DELIVER}, }, ws = 0x7f3ffcc89200 { id = \"req\", {s,f,r,e} = {0x7f3ffcc8b000,+440,+57336,+57336}, }, http_conn = 0x7f3ffcc89128 { fd = 37, doclose = NULL, ws = 0x7f3ffcc89200, {rxbuf_b, rxbuf_e} = {0x7f3ffcc8b000, 0x7f3ffcc8b185}, {pipeline_b, pipeline_e} = {(nil), (nil)}, content_length = -1, body_status = none, first_byte_timeout = 0.000000, first_byte_timeout = 0.000000, between_bytes_timeout = 0.000000, }, http[req] = 0x7f3ffcc89298 { ws[req] = 0x7f3ffcc89200, hdrs { \"GET\", \"/v1/AUTH_731f10a999d448c386af549af42849e7/471296768/228582\", \"HTTP/1.1\", \"Host: 172.20.30.235:6081\", \"Accept: */*\", \"X-Auth-Token:gAAAAABdXPtucjrhDJFszhLvKGI32-udc8jHUN5MjdX0GYd-4FysjFwgs4LM26dAybpl \"Accept-Encoding: gzip\", \"X-Varnish: 513085692\", \"X-Forwarded-For: 172.20.30.170, 172.20.30.235\", }, }, vcl = { temp = warm srcname = { \"/etc/varnish/default.vcl\", \"Builtin\", }, }, flags = { }, }, " Best Regards -------------- next part -------------- An HTML attachment was scrubbed... URL: From dridi at varni.sh Mon Aug 3 15:47:15 2020 From: dridi at varni.sh (Dridi Boukelmoune) Date: Mon, 3 Aug 2020 15:47:15 +0000 Subject: Error in varnish child In-Reply-To: References: Message-ID: On Mon, Aug 3, 2020 at 11:28 AM hamidreza hosseini wrote: > > Hi > I have this error on my varnish instance , is this important or i can ignore it?: No, you should promptly do something about it. First off, Varnish 4.1 is no longer supported. You should be able to update to 4.1.10 without a configuration change to benefit from years of bug fixes, then consider upgrading to 6.0 sooner than later. Dridi From hrhosseini at hotmail.com Tue Aug 4 06:17:20 2020 From: hrhosseini at hotmail.com (hamidreza hosseini) Date: Tue, 4 Aug 2020 06:17:20 +0000 Subject: Error in varnish child In-Reply-To: References: , Message-ID: Hi 1.First of all, I send my private information , please remove the last email that i send to varnish-misc 2.My varnish's version is 6.0 and the VCL version is 4.1 I think its the last version that are supported now 3.About the problem that i send the log, can you say something about it? Best Regards, ________________________________ From: Dridi Boukelmoune Sent: Monday, August 3, 2020 8:47 AM To: hamidreza hosseini Cc: varnish-misc at varnish-cache.org Subject: Re: Error in varnish child On Mon, Aug 3, 2020 at 11:28 AM hamidreza hosseini wrote: > > Hi > I have this error on my varnish instance , is this important or i can ignore it?: No, you should promptly do something about it. First off, Varnish 4.1 is no longer supported. You should be able to update to 4.1.10 without a configuration change to benefit from years of bug fixes, then consider upgrading to 6.0 sooner than later. Dridi -------------- next part -------------- An HTML attachment was scrubbed... URL: From guillaume at varnish-software.com Fri Aug 7 00:21:43 2020 From: guillaume at varnish-software.com (Guillaume Quintard) Date: Thu, 6 Aug 2020 17:21:43 -0700 Subject: Some information Varnish packages and CI Message-ID: Hi everyone, This is going to be a long email, and I apologize in advance for that. We get a lot of questions about packages and hopefully, this can act as an FAQ that newcomers can refer to. # Sources, packages, and other packages ## Source tarballs First in the pipeline, they are produced by the Varnish Cache project and available here: https://varnish-cache.org/releases/index.html We get a new release every 6 months (15/03 and 15/09 usually), whatever's in the trunk goes in, with a code freeze period before the release, and a code thaw one after it. The decision to make the release a major or minor occurs a few weeks before the actual release date. Patch releases can happen for bug and security fixes, but only for the supported releases, meaning: - the two latest releases - the Long Term Support release, which is currently 6.0 ## Distribution packages Those are your regular distribution (debian, freeBSD, openSolaris, etc.) packages and are handled by said distribution. Each maintainer generally grabs the source tarball, applies their patches and packages Varnish to their liking. These packages aren't handled at all by the Varnish Cache project and are each distribution's responsibility. We do however try compatible with all interested upstreams so that patches aren't necessary on their side. ## Packagecloud packages The Varnish Cache project produces its own packages for a limited number of linux platforms (see below) and host them here: https://packagecloud.io/varnishcache . The packages are generally released within a week after the source tarball. Each version is isolated in a separate repository to avoid unplanned major upgrades. # Distribution support ## Source Varnish aims to play nice with the maximum number of POSIX systems: BSDs, Solaris, OSX, etc. You can see this by checking: - the vtest page that references tests run by individual contributors: https://varnish-cache.org/vtest/ (let us know if you want to add your target(s)!) - circleci that builds and checks both the source (every commit) and packages (nightly) for a selection of linux platforms: https://app.circleci.com/pipelines/github/varnishcache - travis: that also builds for a couple of major linuxes, and for OSX ## Packagecloud packages At the time of writing, we produce packages for both arm64 and amd64 architectures on: - alpine:3 - centos:7 - centos:8 - debian:bullseye - debian:buster - debian:stretch - ubuntu:bionic - ubuntu:focal - ubuntu:xenial I'd like to thank Martin Grigorov (martin-g on github) for the awesome work he did on expanding the distribution pool, and on supporting arm64, great job! Important note: once a new distribution is added, it only gets packages when the next Varnish release happens. In other words, we don't build old packages for new platforms. All the packaging scripts are located in https://github.com/varnishcache/pkg-varnish-cache and may or may not be used by distributions downstream. We welcome new platforms too, so don't hesitate to open a PR if you want to include your OS (openSUSE maybe?). And we're happy to help if some stuff needs explaining. # Nightlies and Weeklies Every night, Circle CI produces nightly packages (click on the "collect_packages" job, then "artifacts" ). They reflect whatever is in the master branch when the jobs run and allow you to live on the bleeding edge with the freshest of the freshest packages. Every Monday at 15:00 CEST, on the #varnish-hacking IRC channel, in what we call the bugwash, Varnish contributors decide whether or not to push the latest nightly packages into packagecloud, turning them into "weeklies" ( https://packagecloud.io/varnishcache/varnish-weekly). This means that there may be weeks without weeklies, simply because the decision to push was veto'd (weeklies are otherwise automatically accepted). # Docker images In addition, we also care for the official Varnish Docker images. You can check https://hub.docker.com/_/varnish for the hub page, and https://github.com/varnish/docker-varnish for the source. The images track the latest release and the LTS one, they are based on the latest debian release at the time of build and of course use the packagecloud packages. By the way, we'd love to get some feedback on those, so don't hesitate to open issues on actionable items, or to open PRs here: https://github.com/varnish/docker-varnish I have possibly forgotten to include the one bit ot info that you were after, if that's the case, please ask and I can add it! Cheers, -- Guillaume Quintard -------------- next part -------------- An HTML attachment was scrubbed... URL: From batanun at hotmail.com Mon Aug 24 17:42:05 2020 From: batanun at hotmail.com (Batanun B) Date: Mon, 24 Aug 2020 17:42:05 +0000 Subject: Using user defined variable in backend definition? Message-ID: Hi, I'm experimenting with user defined variables, and when using them in regular string concatenation, and in synch output, it works fine. But I would like to use them in the backend definitions, and I simply can't get it to work. backend myBackend { .host = var.get("myBackendHost"); } But that fails with a VCC-compiler error on that line, saying "Expected CSTR got 'var.get'". I also tried with the "variable" vmod, but it resulted in the same type of error. Is there any way to get this to work? I would really like to have all environment specific configuration (including backend hostnames) in an environment specific vcl file, and then include it in the main vcl where the backend definitions should be (and use the variables). And I would like to achive this using just VCL (and vmods), so no custom script that does search and replace or anything like that. So, I would like something like this to work: default.vcl: ... import var; include "environment.vcl"; backend myBackend { .host = var.get("myBackendHost"); } ... environment.vcl: sub vcl_init { var.set("myBackendHost", "myHostName"); } -------------- next part -------------- An HTML attachment was scrubbed... URL: From dridi at varni.sh Mon Aug 24 21:07:21 2020 From: dridi at varni.sh (Dridi Boukelmoune) Date: Mon, 24 Aug 2020 21:07:21 +0000 Subject: Using user defined variable in backend definition? In-Reply-To: References: Message-ID: On Mon, Aug 24, 2020 at 5:43 PM Batanun B wrote: > > Hi, > > I'm experimenting with user defined variables, and when using them in regular string concatenation, and in synch output, it works fine. But I would like to use them in the backend definitions, and I simply can't get it to work. > > backend myBackend { > .host = var.get("myBackendHost"); > } > > But that fails with a VCC-compiler error on that line, saying "Expected CSTR got 'var.get'". > I also tried with the "variable" vmod, but it resulted in the same type of error. > > Is there any way to get this to work? I would really like to have all environment specific configuration (including backend hostnames) in an environment specific vcl file, and then include it in the main vcl where the backend definitions should be (and use the variables). And I would like to achive this using just VCL (and vmods), so no custom script that does search and replace or anything like that. > > So, I would like something like this to work: > > default.vcl: > ... > import var; > > include "environment.vcl"; > > backend myBackend { > .host = var.get("myBackendHost"); > } > ... > > environment.vcl: > sub vcl_init { > var.set("myBackendHost", "myHostName"); > } Hi, You can't do that, but you can move the backend definition inside environment.vcl instead to keep your default.vcl the same across all environments. Dridi From batanun at hotmail.com Tue Aug 25 09:18:22 2020 From: batanun at hotmail.com (Batanun B) Date: Tue, 25 Aug 2020 09:18:22 +0000 Subject: Using user defined variable in backend definition? In-Reply-To: References: , Message-ID: On Mon, Aug 24, 2020 at 11:07 PM Dridi Boukelmoune wrote: > Hi, > > You can't do that, but you can move the backend definition inside > environment.vcl instead to keep your default.vcl the same across all > environments. Hi, Too bad... Strange thing to require hard coded strings. Is there a technical reason for that? Yeah, I actually ended up doing just that, moving the entire backend definition to the separate file. But I would have preferred having the base structure in the main vcl file, and only the actual host names (and other environment specific configuration) in the separate vcl file. Also, if they were variables, I would be able to use them elsewhere in the vcl (like in synth output, which was my main goal originally). If I want to do that now, I would have to define the same host name multiple times. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dridi at varni.sh Tue Aug 25 12:59:44 2020 From: dridi at varni.sh (Dridi Boukelmoune) Date: Tue, 25 Aug 2020 12:59:44 +0000 Subject: Using user defined variable in backend definition? In-Reply-To: References: Message-ID: On Tue, Aug 25, 2020 at 9:20 AM Batanun B wrote: > > On Mon, Aug 24, 2020 at 11:07 PM Dridi Boukelmoune wrote: > > > Hi, > > > > You can't do that, but you can move the backend definition inside > > environment.vcl instead to keep your default.vcl the same across all > > environments. > > Hi, > > Too bad... Strange thing to require hard coded strings. Is there a technical reason for that? Yes, definitions need to be constant! > Yeah, I actually ended up doing just that, moving the entire backend definition to the separate file. But I would have preferred having the base structure in the main vcl file, and only the actual host names (and other environment specific configuration) in the separate vcl file. Also, if they were variables, I would be able to use them elsewhere in the vcl (like in synth output, which was my main goal originally). If I want to do that now, I would have to define the same host name multiple times. I guess, what you want is constants: https://github.com/varnishcache/varnish-cache/pull/3134 Specifically, from this example, but redacted to remove test-case syntax: https://github.com/varnishcache/varnish-cache/pull/3134/files#diff-996416d1d725c18f8a7bf688cc2f4a52 environment.vcl: const string be_host = "example.com"; const duration be_tmo = 3s; main vcl: vcl 4.1; include "environment.vcl"; backend be { .host = be_host; .connect_timeout = be_tmo; } Right now you have the option of either moving the whole backend definition inside environment.vcl or treat your VCL as a template and populate the environment-specific parts at some stage of your deployment pipeline. Cheers, Dridi From slink at schokola.de Tue Aug 25 15:22:45 2020 From: slink at schokola.de (Nils Goroll) Date: Tue, 25 Aug 2020 17:22:45 +0200 Subject: Using user defined variable in backend definition? In-Reply-To: References: Message-ID: <82550e9f-e45b-7744-f793-95879ed8b29d@schokola.de> On 24/08/2020 19:42, Batanun B wrote: > > So, I would like something like this to work: With the "constant" vmod from https://code.uplex.de/uplex-varnish/varnish-objvar you can do something like this /* backends_X.inc.vcl */ backend foo { ??? .host = "127.0.0.1"; } sub vcl_init { ??? new envbackend = constant.backend(foo); } and then in your main vcl include "environment.vcl", which points to some variant of the above and do sub vcl_backend_fetch { ??? set bereq.backend = envbackend.get(); } See https://code.uplex.de/uplex-varnish/varnish-objvar/blob/master/src/vmod_constant.rst for more details on the constant vmod You could do the same thing with a string and the host header or even use a string and https://github.com/nigoroll/libvmod-dynamic to turn that into a backend at run time. hf, Nils -------------- next part -------------- An HTML attachment was scrubbed... URL: From slink at schokola.de Tue Aug 25 15:28:06 2020 From: slink at schokola.de (Nils Goroll) Date: Tue, 25 Aug 2020 17:28:06 +0200 Subject: Using user defined variable in backend definition? In-Reply-To: <82550e9f-e45b-7744-f793-95879ed8b29d@schokola.de> References: <82550e9f-e45b-7744-f793-95879ed8b29d@schokola.de> Message-ID: <3e0c6d2a-e3ef-5827-e2cd-c89506abbf14@schokola.de> On 25/08/2020 17:22, Nils Goroll wrote: > You could do the same thing with a string and the host header or even use a > string and https://github.com/nigoroll/libvmod-dynamic to turn that into a > backend at run time. I noticed that this is probably what you want. So you might want to consider /* env.inc.vcl */ sub vcl_init { ??? new be_name = constant.string("foo.bar.com"); } /* main.vcl */ vcl 4.1; import constant;??? // I forgot this in my previous email include "env.inc.vcl"; sub vcl_init { ??? new dyn = dynamic.director(); } sub vcl_backend_fetch { ??? set bereq.backend = dyn.backend(be_name.get()); } From slink at schokola.de Tue Aug 25 15:29:32 2020 From: slink at schokola.de (Nils Goroll) Date: Tue, 25 Aug 2020 17:29:32 +0200 Subject: Using user defined variable in backend definition? In-Reply-To: <3e0c6d2a-e3ef-5827-e2cd-c89506abbf14@schokola.de> References: <82550e9f-e45b-7744-f793-95879ed8b29d@schokola.de> <3e0c6d2a-e3ef-5827-e2cd-c89506abbf14@schokola.de> Message-ID: <2eab1546-a12a-0efe-6f10-98a23d6eb1c2@schokola.de> On 25/08/2020 17:28, Nils Goroll wrote: > import constant;??? // I forgot this in my previous email and in this one I forgot ??? import dynamic; Sorry for posting in a rush From batanun at hotmail.com Wed Aug 26 18:12:36 2020 From: batanun at hotmail.com (Batanun B) Date: Wed, 26 Aug 2020 18:12:36 +0000 Subject: Using user defined variable in backend definition? In-Reply-To: <2eab1546-a12a-0efe-6f10-98a23d6eb1c2@schokola.de> References: <82550e9f-e45b-7744-f793-95879ed8b29d@schokola.de> <3e0c6d2a-e3ef-5827-e2cd-c89506abbf14@schokola.de>, <2eab1546-a12a-0efe-6f10-98a23d6eb1c2@schokola.de> Message-ID: Thanks for the suggestions! I'm not sure if we are in the right moment of this project to introduce more competitivity in the form of switching to directors, and a dynamic on top of that. At least not just for this simple thing. But we might need/want to use directors later on, for the dynamic address resolution. And then I might revisit this thread, and look at your example. Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: