Varnishncsa Random Log Sampling ?

Dridi Boukelmoune dridi at
Mon Mar 2 11:43:25 UTC 2020

On Mon, Mar 2, 2020 at 11:15 AM Yassine Aouadi
<yassine.aouadi90 at> wrote:
> Hello,
> I  am sending my Varnish log to  to remote SAAS solution  I and want to improve  logs costs by  implementing a server side sampling solution .
> First I splitted Varnishnncsa  into two service one for error logs and the other for acces logs  :
> CGroup: /system.slice/varnishncsa-error.service
>            └─18458 /usr/bin/varnishncsa -c -b -a -w /var/log/varnish/varnishncsa-error.log -D -P /run/varnishncsa/ -f /etc/varnish/varnishncsa_logmatic.format -q *Status > 399
>   CGroup: /system.slice/varnishncsa.service
>            └─18347 /usr/bin/varnishncsa -c -b -a -w /var/log/varnish/varnishncsa-access.log -D -P /run/varnishncsa/ -f /etc/varnish/varnishncsa_logmatic.format -q *Status < 400
> Is there Any way to go further with  varnishncsa  and perform and random sampling of my access logs ? for example write only 10 % of access logs
> If it's not possible with varnishncsa any   Suggestion ? I tried rsyslog random sampling but  I am facing memory leaks while stress testing server with high load


I think the closest to what you want is rate limiting, see the
documentation for the varnishstat -R option. Otherwise you can always
do the sampling one step downstream and instead of sending
varnishncsa-<something>.log whenever logrotate triggers a rotation you
run script that sends 1 line every 10 lines. But I think rate limiting
with -R is simpler and instead of a percentage that depends highly on
your traffic you can actually get a limit according to a budget since
you wish to reduce costs.


More information about the varnish-misc mailing list