WS/http_conn (Lasse & Martins patches)

[Poul-Henning Kamp]

Lasse and Martin have sent in a group of patches which hit WS and
http_conn from either side and I have tried to see if something
sensible could be done before 4.1 with this area.

I've cherry-picked some bits from the patches and rearranged some
deck-chairs for better Feng-Shui, but what really needs to happen
is not something I want to attempt right before 4.1-R.

Here are some gory entrails for those interested:

We originally had two operations on a workspace:  Allocate and Reserve.

The original idea was that one could still do allocations while part of
the workspace was reserved, but that soon fell by the road-side.

The main user of Reserve was the http_conn receive buffer - we didn't
want to Alloc a buffer and drag the unused part around, so http_conn
would reserve and only hang onto what was actually needed.

The secondary user of Reserve is when we don't know how much space
we will need, but these not really Reservations as much as time-extended
Allocs.

Later we got the Snapshot/Reset trick as a means for reusing WS,
but that is really just another shape of Reserve once you look hard
enough at it.

Next we defined the backend-side timeouts (first/between bytes) in a
way that makes it very hard to reuse the client side "receive a
HTTP header" code for the backend.

The http_conn ended up as a sort of H1-proto-state struct which keeps
track of the existence of a body to be fetched etc.

And then we added PROXY, which uses the http_conn, and has a rather
incestuous relationship with the HTTP1 code all in all.

And all along we have worked to make out-of-WS a handled error, rather
than a panic.

Now we're facing H2, which as entirely different wire/mem properties
than H1, which pretty much violates the assumptions under http_conn.

Some major work will happen here once 4.1 is cut, but to be honest,
after mucking with it for a week I'm still not entirely sure how it
should look in the long run...

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.