Caching Modified URLs by Varnish instead of the original requested URL

[Guillaume Quintard]

Hi Uday,

I'm not exactly sure how to read those diagrams, so I apologize if I'm
missing the mark or if I'm too broad here.

There are a few points I'd like to attract your attention to. The first one
is that varnish doesn't cache the request or the URL. The cache is
essentially a big hashmap/dictionary/database, in which you store the
response. The request/url is the key for it, so you need to have it in its
"final" form before you do anything.

>From what I read, you are not against it, and you just want to sanitize the
URL in vcl_recv, but you don't like the idea of making the main file too
unwieldy. If I got that right, then I have a nice answer for you: use
includes and function calls.

As an example:

# cat /etc/varnish/url.vcl
sub sanitize_url {
  # do whatever modifications you need here
}

# cat /etc/varnish/default.vcl
include "./url.vcl";

sub vcl_recvl {
  call sanitize_url;
}


That should get you going.

Hopefully I didn't miss the mark too much here, let me know if I did.

-- 
Guillaume Quintard


On Tue, Aug 22, 2023 at 3:45 AM Uday Kumar <uday.polu at indiamart.com> wrote:

> Hello All,
>
>
> For our spring boot application, we are using Varnish Caching in a
> production environment.
>
>
>
>
> Requirement: [To utilize cache effectively]
>
> Modify the URL (Removal of unnecessary parameters) while caching the user
> request, so that the modified URL can be cached by varnish which helps
> improve cache HITS for similar URLs.
>
>
> For Example:
>
> Let's consider the below Request URL
>
> Url at time t, 1. samplehost.com/search/ims?q=bags&source=android
> &options.start=0
>
>
> Our Requirement:
>
> To make varnish consider URLs with options.start=0 and without
> options.start parameter as EQUIVALENT, such that a single cached
> response(Single Key) can be utilized in both cases.
>
>
> *1st URL after modification:*
>
> samplehost.com/search/ims?q=bags&source=android
>
>
> *Cached URL at Varnish:*
>
> samplehost.com/search/ims?q=bags&source=android
>
>
>
> Now, Url at time t+1, 2. samplehost.com/search/ims?q=bags&source=android
>
>
> At present, varnish considers the above URL as different from 1st URL and
> uses a different key while caching the 2nd URL[So, it will be a miss]
>
>
> *So, URL after Modification:*
>
> samplehost.com/search/ims?q=bags&source=android
>
>
> Now, 2nd URL will be a HIT at varnish, effectively utilizing the cache.
>
>
>
> NOTE:
>
> We aim to execute this URL Modification without implementing the logic directly
> within the default.VCL file. Our intention is to maintain a clean and
> manageable codebase in the VCL.
>
>
>
> To address this requirement effectively, we have explored two potential
> Approaches:
>
>
> Approach-1:
>
>
>
> Approach-2:
>
>
>
>
> 1. Please go through the approaches mentioned above and let me know the
> effective solution.
>
> 2. Regarding Approach-2
>
> At Step 2:
>
> May I know if there is any way to access and execute a custom subroutine
> from another VCL, for modifying the Request URL? if yes, pls help with
> details.
>
> At Step 3:
>
> Tomcat Backend should receive the Original Request URL instead of the
> Modified URL.
>
> 3. Please let us know if there is any better approach that can be
> implemented.
>
>
>
> Thanks & Regards
> Uday Kumar
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20230822/af026cec/attachment-0001.html>