Need Assistance in Configuring Varnish to Retain and Ignore Unique Parameter in Request URL while caching

[Guillaume Quintard]

Hi all,

Turns out install-vmod works, just needed to grab the right tarballs and
have the right dependencies installed. Here's the Dockerfile I used:
FROM varnish:7.3

USER root
RUN set -e; \
    EXTRA_DEPS="autoconf-archive libossp-uuid-dev"; \
    apt-get update; \
    apt-get -y install $VMOD_DEPS $EXTRA_DEPS libossp-uuid16 libuuid1
/pkgs/*.deb; \
    # vmod_querystring
    install-vmod
https://github.com/Dridi/libvmod-querystring/releases/download/v2.0.3/vmod-querystring-2.0.3.tar.gz;
\
    # vmod_uuid
    install-vmod
https://github.com/otto-de/libvmod-uuid/archive/refs/heads/master.tar.gz; \
    apt-get -y purge --auto-remove $VMOD_DEPS $EXTRA_DEPS varnish-dev; \
    rm -rf /var/lib/apt/lists/*
USER varnish

and here's the VCL:

vcl 4.1;

import querystring;
import uuid;

backend default {
    .host = "localhost";
    .port = "4444";
}

sub vcl_init {
    new qf = querystring.filter(sort = true);
    qf.add_string("myparam");
}

# clear the url from param as it goes in
sub vcl_recv {
    # clear myparam from the incoming url
    set req.url = qf.apply(mode = drop);
}

# add the querystring parameter back if we go to the backend
sub vcl_backend_fetch {
    # create the unique string
    set bereq.http.mynewparam = regsub(uuid.uuid_v4(), "^(.{20}).*", "\1");

    # add our own myparam
    if (bereq.url ~ "\?") {
        set bereq.url = bereq.url + "&myparam=" + bereq.http.mynewparam;
    } else {
        set bereq.url = bereq.url + "?myparam=" + bereq.http.mynewparam;
    }
}

It's a bit crude, but it fulfills your requirements. Make sure you test it
though.

-- 
Guillaume Quintard


On Thu, Jun 1, 2023 at 6:10 AM Uday Kumar <uday.polu at indiamart.com> wrote:

> Thanks for the prompt response!
>
> Thanks & Regards
> Uday Kumar
>
>
> On Thu, Jun 1, 2023 at 11:12 AM Guillaume Quintard <
> guillaume.quintard at gmail.com> wrote:
>
>> Thanks, so, to make things clean you are going to need to use a couple of
>> vmods, which means being able to compile them first:
>> - https://github.com/otto-de/libvmod-uuid as Geoff offered
>> - https://github.com/Dridi/libvmod-querystring that will allow easy
>> manipulation of the querystring
>>
>> unfortunately, the install-vmod tool that is bundled into the Varnish
>> docker image isn't able to cleanly compile/install them. I'll have a look
>> this week-end if I can, or at least I'll open a ticket on
>> https://github.com/varnish/docker-varnish
>>
>> But, if you are able to install those two, then your life is easy:
>> - once you receive a request, you can start by creating a unique ID,
>> which'll be the the vcl equivalent of `uuidgen | sed -E
>> 's/(\w+)-(\w+)-(\w+)-(\w+).*/\1\2\3\4/'` (without having testing it,
>> probably `regsub(uuid.uuid_v4(), "s/(\w+)-(\w+)-(\w+)-(\w+).*",
>> "\1\2\3\4/"`)
>> - then just add/replace the parameter in the querystring with
>> vmod_querystring
>>
>> and...that's about it?
>>
>> Problem is getting the vmods to compile/install which I can help with
>> this week-end. There's black magic that you can do using regex to
>> manipulate querystring, but it's a terrible idea.
>>
>> --
>> Guillaume Quintard
>>
>>
>> On Wed, May 31, 2023 at 6:48 PM Uday Kumar <uday.polu at indiamart.com>
>> wrote:
>>
>>>
>>> Does it need to be unique? can't we just get away with
>>>> "aaaaaaaaaaaaaaaaaaaa"?
>>>>
>>>
>>> Our Requirements:
>>> 1. New Parameter should be *appended *to already existing parameters in
>>> Query String. (should not replace entire query string)
>>> 2. Parameter Value *Must be Unique for each request* (ideally unique
>>> randomness is preferred)
>>> 3. Allowed Characters are Alphanumeric which are *URL safe* [can be
>>> lowercase, uppercase in case of alphabets]
>>> 4. Characters can be repeated in parameter value EX: Gn4lT*Y*
>>> gBgpPaRi6hw6*Y*S (here, Y is repeated) But as mentioned above the value
>>> must be unique as a whole.
>>>
>>> Ex: Parameter value for 1st request can be "Gn4lT*Y*gBgpPaRi6hw6*Y*S",
>>>                                         2nd request can be
>>> "G34lTYgBgpPaRi6hyaaF" and so on
>>>
>>>
>>> Thanks & Regards
>>> Uday Kumar
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20230604/bafb30d3/attachment.html>