Expanding VCL object support

[Reza Naghibi]

I was thinking about this a bit more last night. I agree that
"req.var.blah.foo()" is pretty verbose. We could drop it all and just do
"new blah = ..." with "blah.foo()" and let vcc and the user reason about
the scope. Maybe a shorthand prefix is best. One issue is if you want to
target the top request lifetime (ESI), then without a prefix, there is no
way to do that. There is no way to tell is "new blah = ..." should be
request or top request scope. With a prefix, vcc would know which scope you
want it to be. Im thinking having top request scopes would be useful, but
if not, then we could favor dropping the prefix altogether.

--
Reza Naghibi
Varnish Software

On Thu, Apr 7, 2016 at 7:45 AM, Federico Schwindt <fgsch at lodoss.net> wrote:

> I definitely like this and I can see a lot of places where it will become
> handy.
>
> What I'm not particularly keen is the xxx.var namespace.  While I
> understand the reasoning behind and I'd like to see some kind of variable
> support in Varnish I'm not sure reusing the *req/*resp space is the way
> forward.
> If you really want to tie the variables to a particular namespace you
> could name them as `req_var_xxx`, `bereq_var_xxx`, etc. so this feels a bit
> superfluous.
>
> Speaking of variable support, if we were to have these new scopes isn't
> this exactly giving us that, perhaps with some extra cost?
>
> On Wed, Apr 6, 2016 at 10:09 PM, Reza Naghibi <reza at varnish-software.com>
> wrote:
>
>> Below are some thoughts and a prototype patch for expanding object
>> support. Today, objects are limited to global objects which have a lifetime
>> of the entire VCL. I feel its useful to have objects which can be created
>> during the request and their scope is limited to that request. When the
>> request is done, the objects are finalized.
>>
>> The driver for this is creating a new curl/http vmod (I have several
>> other vmods in mind which would benefit from this). When making curl
>> requests from VCL, you may want to have multiple outstanding (async)
>> requests, so we need to have the ability to encapsulate these requests in
>> isolated, independent, and request scoped objects. Another use case is
>> creating simple type objects, like VCL_STRING, VCL_INT, and VCL_BLOB. We
>> can wrap these types into an object and we now have VCL variables which
>> don't require conversion into headers and we can define them on the fly per
>> request. This will likely open the door for some very interesting VCL
>> functionality :)
>>
>> Here is a VCL snippet which compiles and works with the patch and uses my
>> new libvmod_types [0].
>>
>> ---
>> import types;
>>
>> sub vcl_init
>> {
>>   //global objects, these are unchanged from 4.0
>>   new s = types.string("Hello!");
>>   new reqs = types.integer(0);
>> }
>>
>> sub vcl_recv
>> {
>>   //new req scoped objects
>>   new req.var.slocal = types.string("Request scoped string");
>>   new req.var.s2 = types.string("request string two");
>>   new req.var.count = types.integer(1);
>> }
>>
>> sub vcl_backend_fetch
>> {
>>   //new bereq scoped objects
>>   new bereq.var.sbe = types.string("berequest string v1");
>>   set bereq.http.sbe = bereq.var.sbe.value();
>>
>>   bereq.var.sbe.set("berequest string v2");
>>   set bereq.http.sbe2 = bereq.var.sbe.value();
>> }
>>
>> sub vcl_deliver
>> {
>>   //referencing a mix of global and req scoped objects
>>   set resp.http.X-s = s.value();
>>   set resp.http.X-s-length = s.length();
>>
>>   set resp.http.X-slocal = req.var.slocal.value();
>>   set resp.http.X-slocal-length = req.var.slocal.length();
>>
>>   req.var.count.increment(10);
>>   set resp.http.count = req.var.count.value();
>>   set resp.http.reqs = reqs.increment_get(1);
>> }
>> ---
>>
>> The theoretical curl/http example:
>>
>> ---
>> import http;
>>
>> sub vcl_recv
>> {
>>   //http request #1
>>   new req.var.h1 = http.request();
>>   req.var.h1.set_header("foo", "bar");
>>   req.var.h1.set_url("POST", "http://host1/blah?ok=true");
>>   req.var.h1.send();
>>
>>   //http request #2 (we dont read it so its async)
>>   new req.var.h2 = http.request();
>>   req.var.h2.set_url("GET", "http://host2/ping");
>>   req.var.h2.send();
>> }
>>
>> sub vcl_deliver
>> {
>>   //reference and read http request #1 and block for result
>>   set resp.http.X-test-response-code = req.var.h1.get_response_code();
>> }
>> ---
>>
>> I left the legacy global objects alone in code and syntax. I introduced 2
>> new variable name scopes: req.var.* and bereq.var.*. This is completely
>> cosmetic as these variables can still be request scoped without the
>> (be)req.var prefix. However, the reason for adding it is to give the user
>> some kind of indication that their variable is tied to a frontend, backend,
>> or global scope. Otherwise I have the feeling having a bunch of un-prefixed
>> variables throwing vcc scope errors when used incorrectly will be confusing.
>>
>> Also, the implementation is fairly simple because I piggybacked on the
>> vmod/vrt priv_task implementation. Request scoped objects are basically
>> given a shimmed struct vmod_priv. I had to jump thru a few small hoops in
>> vcc code to get the priv->priv to cast into an actual struct that the VMOD
>> expects. This may or may not be related to VIP#1, but it would be cleaner
>> to move objects to something more priv like than trying to pass in an
>> explicit struct. However, for the patch, I kept the object interface the
>> same and made use of the previously mentioned vcc/vrt shims.
>>
>> The patch is enough to have the examples work and give you guys an idea
>> of how it would work. I wanted to get some feedback before spending more
>> time on this. Its based off of this commit [1], so feel free to comment on
>> github if you want.
>>
>>
>> [0] https://github.com/rezan/libvmod-types
>> [1]
>> https://github.com/rezan/varnish-cache/commit/b547bd9ad2fca9db1ef17ee73b8e9b7df9950c34
>>
>> Thanks!
>>
>> --
>> Reza Naghibi
>> Varnish Software
>>
>> _______________________________________________
>> varnish-dev mailing list
>> varnish-dev at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20160407/57668221/attachment-0001.html>