Vcl.list and the maximum number of loaded VCL

[Dridi Boukelmoune]

Hello,

On Tue, Dec 25, 2018 at 6:23 PM <alexis at madrzejewski.com> wrote:
>
> Hello,
>
> I use Varnish 6.1 and I'm using the vcl.load technique to load custom
> VCL based on the domain name.
> Basically, I follow what is explained on this article :
> https://info.varnish-software.com/blog/one-vcl-per-domain
>
> I have a huge number of domains and each domain have a specific and
> different configuration. Like a vhost.

Sounds about right.

> I used to have multiple VCL file and include them all but it wasn't
> effective. Each time I changed the configuration of one domain, I had to
> reload varnish and all the VCL were recompiled which took a huge amount
> of time (5+ minutes).
> So I switched to the "load.vcl" technique to just recompile the VCL I
> needed and it's much more effective.
>
> But I have an issue related to the huge number of domains and loaded VCL
> I manage.
> This problem was first reported here :
> https://github.com/varnishcache/varnish-cache/issues/2713
>
> I understood that it's fixed and that I needed to add the "max_vcl" to
> define the maximum number of loaded VCL.
> I did that, the max_vcl is on 2000 on my case but the vcl.list is not
> showing the loaded VCL.
> The vcl are loaded, my configuration works but I can't list the loaded
> VCL and it's a problem for me because I need discard old config.

If I understand correctly, most of the 2000 VCLs are active VCL
labels. If that's the case, you will likely suffer from some
significant overhead when it comes VCL operations. In particular, the
addition or removal of VCLs will update the VCL and backend counters
and create numerous files that need to be walked in the working
directory.

> [root at cache-01 ~]# varnishadm
> 200
> -----------------------------
> Varnish Cache CLI 1.0
> -----------------------------
> Linux,3.10.0-514.el7.x86_64,x86_64,-junix,-smalloc,-sfile,-sdefault,-hcritbit
> varnish-6.1.1 revision efc2f6c1536cf2272e471f5cff5f145239b19460
>
> Type 'help' for command list.
> Type 'quit' to close CLI session.
>
> param.show max_vcl
> 200
> max_vcl
>          Value is: 2000
>          Default is: 100
>          Minimum is: 0
>
>          Threshold of loaded VCL programs.  (VCL labels are not
>          counted.)  Parameter max_vcl_handling determines behaviour.
>
>
> vcl.list
> 200
>
> quit
> 500
> Closing CLI connection
>
> Right now, I have ~960 active vcl :
>
> [root at cache-01 ~]# ls /etc/varnish/vhosts/ | wc -l
> 957
>
> On another somewhat related note, is it a good idea to have that much
> loaded VCL ?
> Can it become a problem in the long term ?

See above, there currently is a bottleneck that shows up when you have
many loaded VCLs.

> I ask because I noticed, two time, that my instance of varnish is a 100%
> CPU utilization after 2 weeks.
> I dynamically add, remove, update vhosts but I can't clean old loaded
> VCL because of this vcl.list problem.
> After two weeks, varnish seems to use all the CPU and I suspected that
> it's related. If a restart varnish, then it's OK for another two weeks.
> I suspect that all the old loaded VCL, not discarded, clogged up my
> instance of varnish. Is that plausible, possible ?

Very plausible.

> I'm not familiar with varnish internals so it's a guess.

It's fine. An alternative could be to run multiple Varnish instance to
spread the VCLs. You can use the -n option to varnishd and any other
varnish* program to target a given instance.

Dridi