<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif; font-size: 14.6667px; background-color: rgb(255, 255, 255); display: inline !important">> It's not obvious
why a response</span><span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif; font-size: 14.6667px; background-color: rgb(255, 255, 255); display: inline !important"> would
be delayed, but it could </span></div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif; font-size: 14.6667px; background-color: rgb(255, 255, 255); display: inline !important">> happen to be
related to the </span><span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif; font-size: 14.6667px; background-color: rgb(255, 255, 255); display: inline !important">fetch_chunksize
parameter.</span></div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hmm... <span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif; font-size: 14.6667px; background-color: rgb(255, 255, 255); display: inline !important">fetch_chunksize<span> is <span style="color: rgb(34, 34, 34); font-family: sans-serif; font-size: 16px; text-align: justify; background-color: rgb(255, 255, 255); display: inline !important">16k
by default, and 4k minmum. Does this mean that even if streaming is working fine, it will still buffer at least 4kb of data before sending that to the client? That would be way to much for this use case. The output is basically plain text, and it is flushed
after a few lines of text has been written. We are talking maybe 50-100 bytes per flush.</span></span></span><br style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif; font-size: 14.6667px; background-color: rgb(255, 255, 255)">
<br>
<span style="font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif; font-size: 14.6667px; background-color: rgb(255, 255, 255); display: inline !important">> However I have
never come across a setup where we needed to</span><br>
</div>
<div id="appendonsend"></div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Well, I actually bit the bullet and decided to refactor this particular admin page that I needed this for, so that the job is done in the background and the current status can be fetched whenever one feels like. That way the response is always quick. Hopefully
we won't need this for too many of these old legacy admin pages.</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Dridi Boukelmoune <dridi@varni.sh><br>
<b>Sent:</b> Monday, September 14, 2020 8:35 AM<br>
<b>To:</b> Batanun B <batanun@hotmail.com><br>
<b>Cc:</b> varnish-misc@varnish-cache.org <varnish-misc@varnish-cache.org><br>
<b>Subject:</b> Re: Can't get "streaming" or pipe to work, Varnish still waits for the full response</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt">
<div class="PlainText">On Sat, Sep 12, 2020 at 10:08 PM Batanun B <batanun@hotmail.com> wrote:<br>
><br>
> Hi,<br>
><br>
> We have some old (legacy) internal admin pages that do some classic old school processing while the page is loading, and outputting the current status as it is working. When requesting these pages directly (through Tomcat), I can see the results in the browser
at the same time as the results are written on the other end. But when I go through Varnish, no matter what I try, I only see a blank page that is loading/waiting, and then when the backend is done writing, then I get the entire result in one go.<br>
><br>
> How can I configure Varnish to bring any content to the client the moment it gets it from the backend, and not wait until the entire response is done?<br>
><br>
> In vcl_backend_response I do this:<br>
> set beresp.do_stream = true;<br>
> set beresp.uncacheable = true;<br>
> return (deliver);<br>
<br>
Streaming is on by default, you don't need to do anything.<br>
<br>
> I have also tried returning (pipe) in vcl_recv (with and without do_stream and uncacheable). And gzip is turn off. But nothing helps. What can I do more? And how can I debug this? Varnishlog shows nothing that is telling me that it is buffering, or waiting
for the response, or anything like that.<br>
<br>
It is indeed hard to get that information just from Varnish, you could<br>
try to capture TCP packets to check how long it takes for backend<br>
traffic to be forwarded to clients. It's not obvious why a response<br>
would be delayed, but it could happen to be related to the<br>
fetch_chunksize parameter.<br>
<br>
However I have never come across a setup where we needed to<br>
tune that knob...<br>
<br>
Dridi<br>
</div>
</span></font></div>
</body>
</html>