VSL Timeout and Varnishkafka

[Luca Toscano]

2016-06-22 13:33 GMT+02:00 Geoff Simmons <geoff at uplex.de>:

> On 06/22/2016 01:04 PM, Luca Toscano wrote:
> >
> > Nice test that I found in the Varnish codebase (meant to trigger a
> > VSL timeout):
> >
> >
> https://github.com/varnishcache/varnish-cache/blob/4.1/bin/varnishtest/tests/r01762.vtc#L29
>
> That's
> >
> probably your best bet. Otherwise, triggering VSL timeouts can
> be tricky, because there are a lot of things going on that are hard to
> control.
>
> You said that your PHP script sleeps before emitting "HTML" -- if
> that's what you really meant, then your response probably sent
> headers, and then went to sleep before sending the body. In that case,
> there won't be a first_byte_timeout -- the timeout is until any part
> of the response arrives, including headers.
>

Sorry, what I meant to say was the whole request (headers included).


>
> You could try the same trick but with a large between_bytes_timeout.
> Or find a way to send nothing at all, not even headers, until the
> first_byte_timeout elapses.
>
> I have in fact seen VSL timeouts in production as a result of
> long-running responses, but if you want to reliably reproduce it that
> way, try making the time differences very large -- say timeouts=150s,
> and have your backend delay the response for five minutes.
>
>
Thanks to the Varnish devs on IRC I found a way to repro and also an
explanation. Varnish workers write log tags to a buffer, that gets flushed
to shmlog when it is full. A busy server might flush the buffer often, and
it can happen that a Begin tag gets flushed to shmlog without the rest
(including the End tag, for example, due to long backend processing). The
VSL API will check in the Dispatch function that a Begin tag is not delayed
more than what specified by the -T parameter (or by the default 120
seconds). This behavior is of course a bit difficult to reproduce, but the
-p debug=+syncvsl helps forcing Varnish to make VSL synchronous.

Thanks a lot for all the help!

Luca
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20160623/911a90a8/attachment-0001.html>