pipe mode and sendfile?

[Martin Blix Grydeland]

On Tue, 29 Mar 2016 at 10:51 Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:

> So vcl_sess{} would allow you to do things with the 4 ip numbers
> and the protocol type (H1/H2/whatever) but you do not have a request.
>
> So IIUC this is really an implementation of what is known as a dumb proxy?
No attempt is made at looking at the contents of the stream at all, and
everything is piped through to the other end including the original request
even if it happened to be HTTP-alike (which it wouldn't have to be).

While a dumb proxy certainly has its uses, it is quite far from what
Varnish is, and there are other tools that do the job well. Complementing
Varnish with this functionality could be useful, but I guess this would
count for <1% of the pipe use-cases.

For H2 vcl_sess{} probably has access to the HEADERS, and if the
> PROXY offers it, also ALPN and client cert info.  This is where you
> decide to pipe the entire connection.
>

Here I don't really follow you. The HEADERS won't come until the first H2
stream is opened and the HEADERS frame read, and we are not talking about
piping an entire connection any more. Wouldn't we then need to obviously
start with a fresh connection, create a first stream using exactly the same
HEADERS (if not the real client and the server would become desynched in
the header compression for subsequent streams) and the start piping. While
maybe this would work, it sounds dirty. And the same hack would be
available to H1 (without the need to deny any header changes).

I believe it's important to this discussion to remember that by my account
>90% of all pipe uses in Varnish is to deal with very large content. Both
from the client (large post) and from a server (.iso download), without
using excessive amounts of memory / stevedore space and LRU-purging cached
content in the process. We have the strangely named
stevedore-bypass-for-pass changes that should take away most of the need
for pipe as we know it. So getting that feature in becomes important, and
would allow this discussion to take more innovative paths.

Martin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20160329/20c09320/attachment.html>