Varnish returning synthetic 500 error even though it has stale content it should serve. But only seems to happen during/after a burst of traffic

[Batanun B]

> I recommend permanent logging, if you want to be able to debug older incidents.
> We do it like this:
> 
> ```
> /usr/bin/varnishlog -w /var/log/varnish/varnish-500.log -a -D \
>         -P /var/run/varnishlog-500.pid \
>         -q 'RespStatus >= 500 or BerespStatus >=500'
> ```
> 
> I attach our systemd unit file, for you may be interested.

Thanks. I have thought about that too. But I think we might want to include non-error transactions as well. I mean, with the problems this post is about we want to see when the cached version of the start page was generated and when it was last served from cache successfully. But maybe we could have a permanent logging just for the start page, regardless of http status. That should hopefully reduce the logging intensity enough so that logging to disk isn't effecting the Varnish performance.

One thing though... If you log all "status: 500+" transactions to disk, isn't there a risk that your logging might exacerbate a situation where your site is overwhelmed with traffic? Where a large load causes your backends to start failing, and that triggers intense logging of those erroneous transactions which might reduce the performance of Varnish, causing more timeouts etc which cause more logging and so on...