Varnish 3.1 becoming 4.0 instead (?)

[Poul-Henning Kamp]

I've been mildly disciplined for not keeping the -dev mailing list
in the loop, and that's perfectly true: I have not.

I have no other excuse than I felt like I was talking into an empty
room, but I am assured now that the room is no longer empty so I
try to make sure that IRC doesn't become CABAL and use -dev more.

So a bit of catch-up is probably in order:

The big ticket item right now is streaming.  Martin is working
furiously on that, and I'm trying to slow him down enough to make
sure that streaming becomes an integral part and not some bolt-on
feature.

After streaming backend-If-Modified-Since (Geoffs patches) are next
in the line.

The thinking until now was that we would try to release Varnish 3.1
with both of these as default-off features, and make them default-on
feautures in 4.0

However, as we work the code a couple of details and one 500 pound
gorilla has materialized, and I am now leaning towards skipping the
3.1 step and going directly for 4.0.

The gorilla is vcl_error{} which I am increasingly convinced was a
bad idea to begin with.

The only reason why we added vcl_error{} was to avoid every vcl_fetch{}
having to have to start with:

	if (beresp.status == 503) {
		/* do something */
	}

But with streaming, you can only detect failures to fetch the
headers, once you have started to receive the body, you have committed
to a reply to the client, and it cannot be uncommitted.

That makes vcl_error{} basically pointless with streaming on.

The way I think it will look in 4.0 is that failures to fetch
the reponse-headers from the backend, will be reported as
a 503 reponse in vcl_fetch{}, from where a restart will
be possible.

If you are streaming, and the body fails, you won't hear about it
at VCL level, there will be nothing we can do about it.

If you are not streaming, and the body fetch fails, you will
see a 503 in vcl_deliver{}.

That leaves the other part of what we use vcl_error for today:
synthetic responses.

I thought I had a good idea for "synthetic responses from
everywhere" last year, but it transpired to be a tar-pit
of special-casing, so I abandonned that idea.

My thinking now is that functionality can and should move to
vcl_deliver{} where it rightfully belongs, with all other responses
to the client, and doing it (only) there enables us to provide a
much more sane and usable access to the response body.

Some sort of VCL syntax for "go to vcl_reponse and deliver
response=X" will be needed, and I think it will simply be

	return (response(812));

which leaves the responsibility for synthesizing headers
and body entirely to vcl_deliver{}.

So if we do streaming in 3.1 as planned, I'll have to find some way
to hack up vcl_error{} to still work as we are used to, and I'm
increasingly finding that more trouble than it is worth and leaning
more and more towards the 4.0 option where we can do away with
vcl_error{} and avoid all the complications.

This email represents your chance to come up with really good
arguments for why we should not simply skip 3.1

It is also your chance to remind us what other VCL syntax/semantics
affecting changes should do/have promised for 4.0.

In either case, we are probably looking at a alpha-release with
streaming in jan/early-feb, but you will be able to play with
-trunk before that.

At a more detailed level, Martin and I are trying to pave the road
to streaming by shuffling things around, and the cheat-sheat on
that looks like this:

        Everything related to backend fetch goes into busyobj.
        including htc & (new) workspace for bereq/beresp

	The new workspace means that we will have three workspaces
	in total:
		sess->ws  	holds req.*
		ws->ws	  	holds resp.*
		busyobj->ws	holds bereq.* and beresp.*

        Eventually sess->ws should only exist when sessions do not
        wait, and then worker->ws can then be eliminated and sess->ws
	contain both req.* and resp.*  (Hopefully in 4.0)

        Client->body is be dealt with (FetchHdr() before fetcher-thread
        is spawned.

        Busyobj must be refcounted, since client->deliver may be much
        slower than worker->fetch.

        Busyobj will be malloced, and two strategies for caching free
        ones will be $param selectable:

	Either we cache a single free busyobj (in a static variable,
	under lock), which will work well for high hit-rates.

	Or we cache one per worker thread in wrk->nbusyobj, without
	locking, which will be good for low hit-rates.

	Stats counters will allow us to monitor the situation and
	improve this policy.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.