Expanding VCL object support

[Reza Naghibi]

Maybe something like "new (top)blah = ..." and "blah.foo()" would allow us
to target different scopes but keep the variable names and VCL syntax more
clean. Just thinking out loud.

--
Reza Naghibi
Varnish Software

On Thu, Apr 7, 2016 at 10:04 AM, Reza Naghibi <reza at varnish-software.com>
wrote:

> I was thinking about this a bit more last night. I agree that
> "req.var.blah.foo()" is pretty verbose. We could drop it all and just do
> "new blah = ..." with "blah.foo()" and let vcc and the user reason about
> the scope. Maybe a shorthand prefix is best. One issue is if you want to
> target the top request lifetime (ESI), then without a prefix, there is no
> way to do that. There is no way to tell is "new blah = ..." should be
> request or top request scope. With a prefix, vcc would know which scope you
> want it to be. Im thinking having top request scopes would be useful, but
> if not, then we could favor dropping the prefix altogether.
>
> --
> Reza Naghibi
> Varnish Software
>
> On Thu, Apr 7, 2016 at 7:45 AM, Federico Schwindt <fgsch at lodoss.net>
> wrote:
>
>> I definitely like this and I can see a lot of places where it will become
>> handy.
>>
>> What I'm not particularly keen is the xxx.var namespace.  While I
>> understand the reasoning behind and I'd like to see some kind of variable
>> support in Varnish I'm not sure reusing the *req/*resp space is the way
>> forward.
>> If you really want to tie the variables to a particular namespace you
>> could name them as `req_var_xxx`, `bereq_var_xxx`, etc. so this feels a bit
>> superfluous.
>>
>> Speaking of variable support, if we were to have these new scopes isn't
>> this exactly giving us that, perhaps with some extra cost?
>>
>> On Wed, Apr 6, 2016 at 10:09 PM, Reza Naghibi <reza at varnish-software.com>
>> wrote:
>>
>>> Below are some thoughts and a prototype patch for expanding object
>>> support. Today, objects are limited to global objects which have a lifetime
>>> of the entire VCL. I feel its useful to have objects which can be created
>>> during the request and their scope is limited to that request. When the
>>> request is done, the objects are finalized.
>>>
>>> The driver for this is creating a new curl/http vmod (I have several
>>> other vmods in mind which would benefit from this). When making curl
>>> requests from VCL, you may want to have multiple outstanding (async)
>>> requests, so we need to have the ability to encapsulate these requests in
>>> isolated, independent, and request scoped objects. Another use case is
>>> creating simple type objects, like VCL_STRING, VCL_INT, and VCL_BLOB. We
>>> can wrap these types into an object and we now have VCL variables which
>>> don't require conversion into headers and we can define them on the fly per
>>> request. This will likely open the door for some very interesting VCL
>>> functionality :)
>>>
>>> Here is a VCL snippet which compiles and works with the patch and uses
>>> my new libvmod_types [0].
>>>
>>> ---
>>> import types;
>>>
>>> sub vcl_init
>>> {
>>>   //global objects, these are unchanged from 4.0
>>>   new s = types.string("Hello!");
>>>   new reqs = types.integer(0);
>>> }
>>>
>>> sub vcl_recv
>>> {
>>>   //new req scoped objects
>>>   new req.var.slocal = types.string("Request scoped string");
>>>   new req.var.s2 = types.string("request string two");
>>>   new req.var.count = types.integer(1);
>>> }
>>>
>>> sub vcl_backend_fetch
>>> {
>>>   //new bereq scoped objects
>>>   new bereq.var.sbe = types.string("berequest string v1");
>>>   set bereq.http.sbe = bereq.var.sbe.value();
>>>
>>>   bereq.var.sbe.set("berequest string v2");
>>>   set bereq.http.sbe2 = bereq.var.sbe.value();
>>> }
>>>
>>> sub vcl_deliver
>>> {
>>>   //referencing a mix of global and req scoped objects
>>>   set resp.http.X-s = s.value();
>>>   set resp.http.X-s-length = s.length();
>>>
>>>   set resp.http.X-slocal = req.var.slocal.value();
>>>   set resp.http.X-slocal-length = req.var.slocal.length();
>>>
>>>   req.var.count.increment(10);
>>>   set resp.http.count = req.var.count.value();
>>>   set resp.http.reqs = reqs.increment_get(1);
>>> }
>>> ---
>>>
>>> The theoretical curl/http example:
>>>
>>> ---
>>> import http;
>>>
>>> sub vcl_recv
>>> {
>>>   //http request #1
>>>   new req.var.h1 = http.request();
>>>   req.var.h1.set_header("foo", "bar");
>>>   req.var.h1.set_url("POST", "http://host1/blah?ok=true");
>>>   req.var.h1.send();
>>>
>>>   //http request #2 (we dont read it so its async)
>>>   new req.var.h2 = http.request();
>>>   req.var.h2.set_url("GET", "http://host2/ping");
>>>   req.var.h2.send();
>>> }
>>>
>>> sub vcl_deliver
>>> {
>>>   //reference and read http request #1 and block for result
>>>   set resp.http.X-test-response-code = req.var.h1.get_response_code();
>>> }
>>> ---
>>>
>>> I left the legacy global objects alone in code and syntax. I introduced
>>> 2 new variable name scopes: req.var.* and bereq.var.*. This is completely
>>> cosmetic as these variables can still be request scoped without the
>>> (be)req.var prefix. However, the reason for adding it is to give the user
>>> some kind of indication that their variable is tied to a frontend, backend,
>>> or global scope. Otherwise I have the feeling having a bunch of un-prefixed
>>> variables throwing vcc scope errors when used incorrectly will be confusing.
>>>
>>> Also, the implementation is fairly simple because I piggybacked on the
>>> vmod/vrt priv_task implementation. Request scoped objects are basically
>>> given a shimmed struct vmod_priv. I had to jump thru a few small hoops in
>>> vcc code to get the priv->priv to cast into an actual struct that the VMOD
>>> expects. This may or may not be related to VIP#1, but it would be cleaner
>>> to move objects to something more priv like than trying to pass in an
>>> explicit struct. However, for the patch, I kept the object interface the
>>> same and made use of the previously mentioned vcc/vrt shims.
>>>
>>> The patch is enough to have the examples work and give you guys an idea
>>> of how it would work. I wanted to get some feedback before spending more
>>> time on this. Its based off of this commit [1], so feel free to comment on
>>> github if you want.
>>>
>>>
>>> [0] https://github.com/rezan/libvmod-types
>>> [1]
>>> https://github.com/rezan/varnish-cache/commit/b547bd9ad2fca9db1ef17ee73b8e9b7df9950c34
>>>
>>> Thanks!
>>>
>>> --
>>> Reza Naghibi
>>> Varnish Software
>>>
>>> _______________________________________________
>>> varnish-dev mailing list
>>> varnish-dev at varnish-cache.org
>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20160407/31a8caab/attachment.html>