Websocket's better support

[Dridi Boukelmoune]

On Thu, Apr 20, 2017 at 9:57 AM,  <jonathan.huot at thomsonreuters.com> wrote:
> Hi Varnish dev & users,
>
> Websocket is (still) not dying, and for us, it seems we have to handle more and more this kind of traffic.
> It's why I would like to open a discussion and see how we can enhance the websocket support in Varnish.
>
> Currently, the implementation is done thru pipe like this :
>
> cli (Upgrade) -> recv -> pipe <--> backend
>
> So basically, we're putting the ball into backend's hands and nothing else.
> It has several disadvantages; one of them is that we cannot interact with the handshake's response which is still in HTTP/1.1.
> E.g. adding a set-cookie for stickiness is not possible.
> E.g. testing if status code is 101 is not possible
>
> So, first problem, first question: do you think it is possible to allow opening the pipe after vcl_backend_response ?
> The flow will be:
>
> cli (Upgrade) -> recv -> pass -> b_fetch -> b_response -> pipe <---> backend

It's not, but it's not the first time we discussed it:

https://github.com/varnishcache/varnish-cache/wiki/VIP8%3A-No-pipe-in-builtin.vcl-in-V5#discussion

See around 13:41.

You can emulate an "Expect: 100-continue" dance with the backend to at
least make sure an upgrade would be allowed before actually piping:

    varnishtest "websocket example"

    # Varnish won't let you use 100-continue so we need an ad-hoc solution
    # to ask the backend whether websocket is acceptable beforehand.

    server s1 {
        rxreq
        expect req.http.Connection == upgrade
        expect req.http.Upgrade == websocket
        expect req.http.Expect == 200-ok
        txresp

        rxreq
        expect req.http.Connection == upgrade
        expect req.http.Upgrade == websocket
        expect req.http.Expect == <undef>
        txresp -status 101 \
            -hdr "Connection: upgrade" \
            -hdr "Upgrade: websocket"

        send "pretend we do websocket here"
    } -start

    varnish v1 -vcl+backend {
        sub vcl_recv {
            if (req.restarts == 0) {
                unset req.http.Pipe;
                unset req.http.X-Upgrade;
            }
            if (req.http.Pipe) {
                set req.http.Upgrade = req.http.X-Upgrade;
                return (pipe);
            }
            elsif (req.http.Connection == "upgrade") {
                set req.http.X-Upgrade = req.http.Upgrade;
                return (pass);
            }
        }

        sub vcl_backend_fetch {
            if (bereq.http.X-Upgrade) {
                set bereq.http.Connection = "upgrade";
                set bereq.http.Upgrade = bereq.http.X-Upgrade;
                set bereq.http.Expect = "200-ok";
                unset bereq.http.X-Upgrade;
            }
        }

        sub vcl_backend_response {
            if (bereq.http.Upgrade && beresp.status == 200) {
                set beresp.http.Pipe = bereq.http.Upgrade;
                return (deliver);
            }
        }

        sub vcl_deliver {
            if (resp.http.Pipe) {
                set req.http.Pipe = resp.http.Pipe;
                return (restart);
            }
        }
    } -start

    client c1 {
        txreq -hdr "Connection: upgrade" -hdr "Upgrade: websocket"
        rxresp
        expect resp.status == 101

        # receive the fake websocket traffic
        recv 28

        # this will fail because there isn't anything left to read
        # recv 1
    } -run

This example is over-simplified, it only shows that you can act upon
websocket requests. That doesn't solve the fact that you don't have
access to the beresp once you return pipe.

> Only this extra step would be a huge improvement of what we can do on websockets connections and will be very beneficial.
>
>
> Then, a bonus question, cuz it requires probably much more time:
> I was wondering if later, websocket protocol can be integrated to the core (e.g. similarly to HTTP2?), to have benefits of metrics, params and logs.
> Because, No, websocket messages are not just "bytes thru a pipe" :-)

Well, websocket is not HTTP, unlilke... HTTP/2? I don't think it would
fit nicely in VCL, applications do what they want on top of the
session. Unlike HTTP that specifies what happens on the session, we
couldn't make more assumption than bytes passing through.

Dridi