The stevedore API

[Poul-Henning Kamp]

I've been mucking about with the stevedore api for some days in order
to resolve a number of silly issues, including the vast overallocation
for tiny objects when streaming.

There are suprisingly many dead ends in this area.

I think I have finally managed to come up with something that is usable,
both from stevedore and varnishd side.

The crucial insight is that we do not need to store the OA first
since we now have accessor functions.  Instead we can collect the
OAs in the busyobj until we know their final size, and only then
allocate space for them.

But we still need to alert persistent stevedores before we make the
first allocation for a new object, and we also (which is new!) want
to give stevedores a chance to relocate segments once their size is
known for sure.   In that case it's the stevedores responsibility
to keep track of streamers of the interrim segment before freeing it.

Persistent stevedores also must be told what "magic key" it must
present to resurrect an object.

And finally we make it the cache_obj.c codes responsibility to free
all allocated segments individually, so that the stevedore does not
need to keep track of that. 

Seen from the stevedore:

    New objects are created with stv->newobj().

	Arguments:
		busyobj
		total size estimate
	Returns:
		yes/no

	busyobj/objcore has private field(s) for the stevedores
	private use from here on.

	The stevedore can fail this if the object is undesirable,
	and another stevedore will be attempted, if nothing else, Transient.

    Storage segments are allocated with stv->alloc()
	Arguments:
		busyobj
		desired number of bytes
	Returns:
		interrim seg_id (uintptr_t)
		data pointer
		length

    Once the segment has been written to:  stv->commit()
	Arguments:
		busyobj
		interrim seg_id
		length used
	Returns:
		final seg_id, may be different from interim seg_id

    Other threads may reference segments with stv->ref()
	Arguments:
		busyobj (possibly NULL)
		seg_id (interrim or final)
	Returns:
		data pointer
		length

    References are released with stv->rel()
	Arguments:
		busyobj (possibly NULL)
		seg_id (interrim or final)
	Returns:
		none

    Object is finalized with stv->final()
	Arguments:
		busyobj (not NULL)
		seg_id (key to resurrect persistent object)
		off_t  (subkey to resurrect persistent object)
	Returns:
		none

    Objects are killed with stv->deleteobj()
	Arguments:
		objcore

    All allocated segments are individually freed with stv->free()
	Arguments:
		seg_id

Seen from varnishd:

	stv->newobj()
	collect OA's in busyobj and access from there
	stv->alloc/commit until body is received
	all OA's defined.
	byte-stream encode OAs into last storage segment
	(including OA listing storage segment id's)
	(if space allows) else into separate segment
	stv->final(coords for OA bytestream)
	OA's now pulled out of stored object

Comments most welcome...

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.