Future plans (was: setting the port number)

[Poul-Henning Kamp]

In message <ujry7v4fkox.fsf at cat.linpro.no>, Dag-Erling =?iso-8859-1?Q?Sm=F8rgra
v?= writes:

>The attached VCL file trigges the following assertion:

Yes, VCL is only just able to handle the default config, pretty much
all other variables than those you see in the default code are not 
implemented yet.


But I would like us to spend at least a few emails on project
management at this point.

I am approximately 60% through the 400.000 NOK, so that leaves 
something like 300 hours on my part.

Knowing the (100-X)/X rule for whatever X is applicable, that
means that we will come up short somewhere on some subset of
our ambitions.

I often use a little excercise to prioritize my work that goes like
"If you had to deliver one hour from now, what would you do ?, what
if you had two hours ?  What if you had 1 day ?" etc.

The conclusion I reached on the way back home was quite different
from the one I reached on the way up to Oslo.

On the way to Oslo, I had expected performance to be better than
squid by maybe a factor of two, and I therefore saw VCL as a major
product selling point relative to Squid, and I expected us to put
a fair bit of effort into VCL before the release date.

However, considering that we totally smoked Squid performance wise
in the tests, and did so with a stability which if not exactly
rivals Squid, then at least seems to have more potential I think
we should maybe tone VCL ambitions down a bit, and instead aim
at being vastly more reliable and a lot faster than Squid.

Despite the results of the livetest, I am painfully aware that there
is quite a bit of work yet in the HTTP processing department, and
some work in the rest of the "basic mechanics", optimizing the
storage allocation, making the worker thread pool dynamic etc.

We have the VCL compiler, we have the compiled code for the
default config running and working.  The major actions (pipe,
pass, deliver, fetch, lookup etc) needs to be made to work
correctly and reliably because they influence the architecture
of the cache process.

But other than that I am inclined to leave VCL alone for now.
Incrementally adding stuff to VCL later on is very easy, because
changes can be trivially tested with telnet as client.

In comparison changing the worker thread pool, buffer manangement
or HTTP header fritzing code requires testing in a live environment
which is slightly more involved.

So my proposed plan from now on would be:

	Analyse logfiles from live test

	Review and clean up changes made during live test.

	Implement all VCL actions {pipe, pass, fetch ...}
	   and get cache process structure finally nailed down.

	Make worker thread pool dynamic.

	(start running live in VG.no ?)

	(Invite alpha testers ?)

	Implement management process more functions (start/stop etc).

	Improve HTTP header handling & compliance.

	Improve storage_file allocation policy.

	(Invite beta testers ?)

And then for the remaining time make it buffet work, picking whatever
we desire from:

	Adding more statistics

	Logfile handling

	Add VCL features

Until we run out of time & money and

	(Release Varnish 1.0)


In addition to all of this, there is the Linux portability, documentation
and test-harness tasks, but I sort of hope they will be mostly in
Dag-Erlings area ?

Does this make sense ?

Poul-Henning


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.