VSC self-documentation

[Reza Naghibi]

Looks pretty decent. Definitely makes sense to have some kind of JSON
support in Varnish, especially on the parsing and reading end. I too had to
implement my own JSON lib just because the options out there just dont hit
all requirements, libraries generally optimize for 1 thing and somehow do
everything else poorly, like navigation vs performance or strictness vs
performance. What I got is ~1600 LOC, but it has very strict JSON grammer
parsing. Very fast on both ends too, parsing is done in 1 pass (streamed)
and the result is a nice search index which acts very close to a hashtable
when searching. Infact, the search index get stored into cache (see below).

I could definitely see open sourcing this in the form of a VMOD. Yes, I
said VMOD. Check out the following VCL. Sorry for the hijack, and yes, im
digging up object support in VCL again :)

---
import request;
import objcore;
import json;
import types;

sub vcl_recv {
  new json_req = request.new();

  if (req.url == "/") {
    json_req.copy_headers();
    json_req.set_host(local.ip);
    json_req.set_port(std.port(local.ip));
    json_req.set_url("/some/data.json");

    # Tell the VMOD if this request goes back into Varnish
    # safely grab the objcore, if possible
    # (there is a more elegant way to do this
    #  but I rather get the point across)
    json_req.want_objcore();

    json_req.send();
  }
}

sub vcl_deliver {
  new oc = objcore.new();
  new json_doc = json.new();
  new json_field = types.string();

  if (json_req.sent()) {
    json_req.wait();

    # Load the objcore from the json request
    oc.get_reference(json_req.get_objcore());

    if (!oc.is_valid()) {
      return(synth(401));
    }

    # Similar to ESI data
    json_doc.load_index(oc.get_attribute("json"));

    if (!json_doc.is_valid()) {
      # Parse the response text into JSON
      json_doc.load_text(json_req.get_body());
      # Write back the index into cache
      oc.set_attribute("json", json_doc.get_index());
    }

    json_field.set(json_doc.get("some_json_field"));

    # We now read a JSON field from the body of an object in cache
    # Future lookups will hit the JSON search index directly
  }
}
---

Basically, make a request back to Varnish, grab the objcore, and then
load/store/cache JSON into it. That JSON could have been a cache miss, it
gets cached, it could have been a 404, this all plays really nice with how
Varnish works. A client can even request that JSON back out (minus the
search index, obviously). Dont get all up in arms, this all works, this is
pretty much how Edgestash works over at Varnish Software :) Also, remember,
the point of VCL is to allow people to write what they want, so if someone
wants to write the above VMODs/VCL, good for them.

So ya, if I can write a JSON vmod object, then that would be a good case to
release JSON code. The flipside is that with objects, JSON parsing is just
the tip of the iceburg in terms of providing a nice standard library with
all the things dev expect, plus interop with Varnish internals, like
objcore above.

--
Reza Naghibi
Varnish Software

On Fri, May 19, 2017 at 10:48 AM, Poul-Henning Kamp <phk at phk.freebsd.dk>
wrote:

> --------
> In message <CADe=ujYbWs09K_UgsB+A6+u97OnTf3xR9HYk1pwKoNNr99EBjQ@
> mail.gmail.com>
> , "Devon H. O'Dell" writes:
>
> >> [2] I my check for control-chars in strings I forgot that char is
> signed.
> >
> >Except when it isn't! Whether "plain" char is signed or unsigned is
> >implementation-defined
>
> Yeah, compliments to the ISO-C people for that bit of insanity...
>
> --
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
>
> _______________________________________________
> varnish-dev mailing list
> varnish-dev at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20170519/cb9918b4/attachment-0001.html>