varnishlog-json: inclusion in core

[Guillaume Quintard]

Hi everybody,

After a few months of simmering, I think it's time to formally propose
varnishlog-json (https://github.com/varnish/varnishlog-json) for inclusion
in the main repository.

A tiny bit of context first, feel free to skip if you've already heard me
ramble about this.
Currently we have two main loggers: varnishncsa and varnishlog, both with
their caveats. The former requires the user to know exactly what they are
looking for so they can define a format line, on top of using ncsa which is
a terrible format. One can emulate JSON (completely random example) but the
format line quickly becomes insane and unreadable. And I'll skip over the
issues of which record value is picked (first one? last one? it depends!).
The latter presents less a synthetic report of a transaction and more a
sequence of events which is absolutely brilliant for debugging, but isn't
super useful outside of that.

So after years of ranting, I finally built a tool that would provide a
compact report of a transaction, opinionated and hopefully both immediately
useful but whose output can be used by other tools (so they don't have to
link against libvarnish).

I think the README of the project is fairly clear about the important
point, but to save you a click, here are the takeaways:
- uses VUT
- tries to present requests/responses as they are when entering/leaving
varnish
- JSON output using libcjson (https://github.com/tspspi/libcjson) we link
against it, but we could embed it as when given how small it is
- supports all the bells and whistles you expect from a libVUT tool,
including filtering, grouping and daemonizing
- reports VCL_Log records so users can inject extra information for future
processing

I've been using it for a while in a bunch of projects in go and JS, saving
me from writing bindings, and I think we can start thinking about making it
a prime citizen given the gap it fills. I know there are V/CBOR works in
the pipeline, which could possibly change the internal implementation
details, but I reckon it doesn't prevent us from including the current code
until we have better internals to hook into.
Internally, I'm going to start pushing to make this part of Enterprise, but
I'd like to try and upstream it first to get some broader feedback from the
community.

I'll be at the bugwash today so we can hopefully discuss it.

Cheers!

-- 
Guillaume Quintard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20240722/4b8e739e/attachment.html>