on the vsl format - VIP23 (VSL refactoring) design sketch

[Nils Goroll]

Hi,

I spend some time pondering the VSL format, here's what I got at the moment.

First of all, a recap of the existing format. In short:

* a record can be an end- or wrapmarker. These are not relevant in the current
  context

* batch records are containers for individual records as produced by the VSLb*
  functions on a vsl buffer

* records are tag, length, vxid and payload

endmarker:

| 31 ... 24 | 23 ... 0 |
| 0xfe      | 0x454545 |

wrapmarker:

| 31 ... 24 | 23 ... 0 |
| 0xfe      | 0x575757 |

batch:

| 31 ... 24 | 23 ... 0 |
| 0xff      | zero     |

| 31 ... 0 |
| len      |

<record><record>...

record:

| 31 ... 24 | 23 ... 2 | 1 .. 0 |
| tag       | len >> 2 | zero   |

| 31 | 30 | 29... 0 |
|  B |  C | vxid    |

| (len bytes) |
| log         |

B = BACKENDMARKER
C = CLIENTMARKER
actually those are contained in the vxid passed to VSL


There is a number of VSL Tags for which this format can remain unchanged,
because they either contain exactly one string or are relevant only for
debugging/tracing purposes.

PROPOSAL:

For the binary VSL records we already agreed that fixed length fields always
come first. Because we have the length and know the size of the fixed part, we
can could put in another STRING or header as follows:

fixed:

| 31 ... 24 | 23 ... 2 | 1 .. 0 |
| tag       | len >> 2 | 1    0 |

| 31 | 30 | 29... 0 |
|  B |  C | vxid    |

<DBL><INT>...[<STRING|HDR>]

HDR: <UINT8><STRING><STRING>


This would accommodate exactly one additional STRING or, with one extra byte to
indicate the header name length, one additional header.

I would also propose to indicate this format by setting one of the lower bits of
the length.

This format would accommodate more VSL Tags where exactly one string/header is
logged.

The question then is if, for additional variable data, we should actually embed
it in one VSL record. We could also use additional VSL records, under the
condition that such "continuation records" are written within the same Batch:

variable:

like record, but
- field instead of tag
- p[0] & 3 == 3
- no xvid, data starts at p[1]

| 31 ... 24 | 23 ... 2 | 1 .. 0 |
| field     | len >> 2 | 1    1 |

| (len bytes) |
| log         |

The advantage would be that we get the full 24 bits length per string.

But my main motivation was actually pondering about a different function
interface, for which this format would be advantageous because the caller would
write directly to VSL. Here's a taste of this suggestion, but this is not cooked
yet. Feedback welcome anyway:

struct SLTS_XXX t[1];

VSLS_Open(t, vsl, SLT_XXX, vxid);
t->fixed->dbl = 0.5;
t->fixed->int = 42;
VSLS_printf(t, t->var->brz, "%s_%s", bar, baz);
VSLS_printf(t, t->var->foo, "%s/%s", bar, baz);
VSLS_Close(t);

Nils


-- 

** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

tel +49 40 28805731
mob +49 170 2723133
fax +49 40 42949753

xmpp://slink@jabber.int.uplex.de/

http://uplex.de/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20190412/ce81eb5a/attachment.bin>