Varnish version 2 and beyond

[Poul-Henning Kamp]

Varnish version 1 has been out for some time now, and it is time
to talk about varnish version 2 and the future in general.

I told Anders that I wanted a bit of vacation before I even
contemplated the future.  That is why I have been relatively silent
and inactive since the release.

But I have also talked to a number of early Varnish users and given
a couple of talks about Varnish in various settings and the reception
have been very warm and postive.

And then everybody asks me "What happens next ?  What about version 2 ?"

So lets me open the discussion about version 2 with a loooong email:


What ?
======

The features I have heard people ask for so far are:

    Things we already know how to do:
	Bugfixes
	Docs
	More VCL
	Simple Server side includes.
	Vary: header processing (requirement for compression etc.)
	Compression.
	tar storage method for static/emergency contents
	The cluster controller

    Tasks which can be delegated:
	Tcpdump-like specification to varnishlog(1) etc.
	RRD statistics
	GUI / Dashboard

    Things we need to think more about how to do:
	Prefetching
	backend loadbalancing

    Things we need to think a lot about how to do:
	Cookie/Auth handling
	URL-scrubber functionality (exploits etc)

As a programmer I want to add, high on the list:

	Regression test driver
	Regression test cases


How ?
=====

The plan Anders talked about for version 2, was to go through a
process setting up a user-group with a governing board, writing
specification documents for Varnish version 2, produce time estimates,
write a budget and pass the hat around and see what happened.

Even though I am not very interested in the administrative side of
the project, I would strongly advice setting up some kind of
"core-team" or governing board, well ahead of the time where it
becomes necessary to have one.


Who ?
=====

Some of the tasks above are probably best left in my hands while
others would be better off delegated.  As good first approximation
is that if it is in the Varnishd binary it is probably "my turf"
if it is outside it can delegated.

If you guys get a better offer from somebody else or decide you
can't afford my services, I won't disappear from Varnish in toto.

Chances are I might still land a contract or two with people who
need help with Varnish, but my involvement would be a sparetime
activity and obviously a lot less intense than otherwise.


How Much ?
==========

As you know VG.no sponsored version 1.0 and I don't know if the
amount they spent is really secret or not.  I trust Anders will
disclose it if he feels this is necessary for the discussion.

(Currently linpro.no runs the project servers, I don't know under
what arrangement that happens.  If they cannot find enough Varnish
work in Norway, we may have to pay them to do so, or migrate it to
sourceforge or whatever.)

For my own part, I feed my kids by working as an independent
contractor and as such I have to send invoices to somebody every
so often.  I have plenty of uncommitted hours in 2007 still, and
if people want to pay me to continue on Varnish, I'm game.  (I
do have a little bit of available time left in 2006 also).

For reasons of personal development I prefer to not get nailed down
in just one area of computing.  I also have a number of faithful
customers which keep coming back with interesting stuff for me to
work on.

So I try to only put half of my time up for any one customer when
it comes to long term contracts.  I work better that way too, it
gives me more time to think about the issues in the background.

For the magnitude of work we are talking about here, my listprice
is DKR610/h (USD105/h) which after my 10% discount on BSD licensed
Open Source, becomes DKR550/h (USD94/h) (For danish customers sales
tax would be on top of that).

If you guys prefer to go through the formal excercise and define
tasks, milestones etc. etc, then it's fine for me.  The only two
conditions I set is that you guys set up a governing board so I
have a competent and authoritative management to work with on all
the paper and that you tell me where to send my invoices.  This
last bit could easily be the most tricky.

But Seeing the domainnames we get email from, I get the impression
that finding funding for future Varnish development may not be as
troublesome as we anticipated when we began, so a much simpler
alternative is also an option:

If you guys are willing to pay me DKR50K/month (USD8600/month) and
agree to a 3 month notice of termination, I will commit to using
half of my time on Varnish in whatever direction we together decide
to take it.  In that scenario I'm willing to send invoices directly
to whoever the donor of the month is.  That way we don't have to
wait for a formal user organization to get the financial side
hashed out.

The main downside to this way of doing things is that it does
exactly take away alot of the incentive to get a solid user
organization set up.

My advice is that you first decide the organizational aspect(s)
before you start the technical bits and prioritzation of the wishlist.

I think that was what I had to say right now and I think you guys
should talk/email/whisper in smokefilled rooms/fight it out with
EMACS macros or however else you want to settle between yourselves
how you want to progress from here.


Poul-Henning

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.