Request for code review of streaming patches and merging into master
Martin Blix Grydeland
martin at varnish-software.com
Sun Jan 22 18:52:23 CET 2012
The streaming implementation has reached a state now where I believe it
should be considered for merging into master. I will post all of the
patches to varnish-dev.
There are some outstanding points left, outlined below. And also there will
be some cache_center.c restructuring coming up, but it should be possible
to do that without needing major rewriting of the streaming stuff, and
having the code in master when starting on that should be an advantage.
Any comments and inputs much appreciated.
Outstanding streaming work items:
* Streaming and client 304
If we can deliver a 304, we should for the pass-case just drop the
backend connection (alternately do the bg fetch and throw away the
result?). For the non-pass-case we should perform the bg fetch while
still delivering a 304 to the client. The decision making will have
to be influenced by wether we manage to grab a thread or not, and
what to do when we don't have a 2nd thread has to be decided.
This is best implemented after the vcl changes and cache_center.c
changes coming from that.
* Streaming and range
Currently ranged delivery isn't performed when doing streaming, full
object is always delivered when do_stream is true. This should be
fixed. Some patches from the 3.0 streaming by Thomas Souvignet has
been mailed to varnish-dev. Should be possible to use this as input
when adding it to the new streaming implementation.
* Break backend connection when streaming, pass-mode and client goes away
If we loose the client while streaming in pass-mode we should
probably just drop the backend connection as well.
* Adaptive streaming
I don't think it makes sense to do streaming for very small
objects. For things less than a kilobyte, by the time we have the
2nd thread ready the full object is probably already on the wire and
it would be faster to just read this in full before delivering the
This is easy if there is a content-length header, we could just test
this against a configurable minimum size. This decision policy
probably makes the much sense in VCL?
For chunked/eof transfer from backend, we can't know the size at
decision making time. We could perhaps implement that streaming is
first started after having received a minimum number of bytes?
Martin Blix Grydeland
Varnish Software AS
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the varnish-dev