Block Unauthorized Requests at Varnish [Code Optimization]
Uday Kumar
uday.polu at indiamart.com
Mon Oct 16 06:43:39 UTC 2023
Hello Guillaume,
Thank you so much!
I'll check it out!
Thanks & Regards
Uday Kumar
On Sun, Oct 15, 2023 at 7:32 AM Guillaume Quintard <
guillaume.quintard at gmail.com> wrote:
> Hello Uday,
>
> Quick follow-up as I realize that templating can be a bit scary when
> confronted for the first time, and you are far from the first one to be
> curious about, so I've committed this:
> https://github.com/varnish/toolbox/tree/master/gotemplate-example
> It probably won't get you very far, but it should at least get you
> started, and help understand how templating can make things a tiny be
> simpler but splitting data from business logic, for example to add more
> IPs/ACLs or source without edit the VCL manually.
>
> Hope that helps.
>
> --
> Guillaume Quintard
>
>
> On Thu, Oct 12, 2023 at 12:36 PM Uday Kumar <uday.polu at indiamart.com>
> wrote:
>
>> > That's mainly how computers work, processing will be linear. You
>> *could* create a vmod that packs ACLs into a hashmap to simplify the
>> apparent logic, but you will pay that price developing the vmod, and for a
>> very modest performance gain. If you have less than 50 sources, or even
>> less than 100, I don't think it's worth agonizing over that kind of
>> optimization (unless you've actually measured and you did see a
>> performance drop).
>>
>> Okay, Thanks for your suggestion!
>>
>> > I assume that the VCL is currently committed in a repo somewhere and
>> gets edited every time you need to add a new IP or source. If so, it's not
>> great because editing such repetitive code is error-prone, and therefore
>> you should use templating to create the VCL from a simpler, more
>> maintainable source.
>>
>> Sure, will definitely explore!
>>
>> Thanks & Regards
>> Uday Kumar
>>
>>
>> On Fri, Oct 13, 2023 at 12:35 AM Guillaume Quintard <
>> guillaume.quintard at gmail.com> wrote:
>>
>>> > In the above example, if the request URL is source=tablet [for which
>>> condition is present at the end], still I have to check all the above
>>> conditions.
>>>
>>> That's mainly how computers work, processing will be linear. You *could*
>>> create a vmod that packs ACLs into a hashmap to simplify the apparent
>>> logic, but you will pay that price developing the vmod, and for a very
>>> modest performance gain. If you have less than 50 sources, or even less
>>> than a 100, I don't think it's worth agonizing over that kind of
>>> optimization (unless you've actually measured and you did see a
>>> performance drop).
>>>
>>> > One thing I would do though is to generate the VCL from a source file,
>>> like a YAML one:
>>>
>>> All I'm saying is that you should focus on increasing the
>>> maintainability of the project before worrying about performance. I assume
>>> that the VCL is currently committed in a repo somewhere and gets edited
>>> every time you need to add a new IP or source. If so, it's not great
>>> because editing such repetitive code is error-prone, and therefore you
>>> should use templating to create the VCL from a simpler, more maintainable
>>> source.
>>>
>>> Tools like go templates or jinja can provide that feature and save you
>>> from repeating yourself when writing configuration.
>>>
>>> --
>>> Guillaume Quintard
>>>
>>>
>>> On Thu, Oct 12, 2023 at 11:46 AM Uday Kumar <uday.polu at indiamart.com>
>>> wrote:
>>>
>>>> Hi Guillaume,
>>>>
>>>> I don't think those are redundant checks, from what you are showing,
>>>> they are all justified. Sure, there may be a bunch of them, but you have to
>>>> go through to them.
>>>>
>>>> By redundant I meant, I have to write multiple checks for each source
>>>> and list of IPs associated with it. [which would be *worse *if the
>>>> number of sources are huge]
>>>>
>>>> *Example:*
>>>>
>>>> If(
>>>>
>>>> (req.url ~ "source=mobile" && client.ip != mobile_source) ||
>>>>
>>>> (req.url ~ "source=desktop" && client.ip != desktop_source) ||
>>>>
>>>> (req.url ~ "source=laptop" && client.ip != laptop_source) ||
>>>>
>>>> (req.url ~ "source=tablet" && client.ip != tablet_source)
>>>>
>>>> ){
>>>>
>>>> return(Synth(403, "access denied!"))
>>>>
>>>> }
>>>>
>>>>
>>>> In the above example, if the request URL is source=tablet *[for which
>>>> condition is present at the end]*, still I have to check all the above
>>>> conditions.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> One thing I would do though is to generate the VCL from a source file,
>>>> like a YAML one:
>>>>
>>>> Didn't understand, can you please elaborate?
>>>>
>>>> Thanks & Regards
>>>> Uday Kumar
>>>>
>>>>
>>>> On Thu, Oct 12, 2023 at 11:11 PM Guillaume Quintard <
>>>> guillaume.quintard at gmail.com> wrote:
>>>>
>>>>> Hi Uday,
>>>>>
>>>>> I don't think those are redundant checks, from what you are showing,
>>>>> they are all justified. Sure, there may be a bunch of them, but you have to
>>>>> go through to them.
>>>>>
>>>>> One thing I would do though is to generate the VCL from a source file,
>>>>> like a YAML one:
>>>>>
>>>>> mobile:
>>>>> - IP1
>>>>> - IP2
>>>>> - IP3
>>>>> desktop:
>>>>> - IP4
>>>>> - IP5
>>>>> - IP6
>>>>>
>>>>>
>>>>> From that, you can build the VCL without having to manually write
>>>>> "client.ip" or "(req.url ~ "source=" every time.
>>>>>
>>>>> --
>>>>> Guillaume Quintard
>>>>>
>>>>>
>>>>> On Thu, Oct 12, 2023 at 10:17 AM Uday Kumar <uday.polu at indiamart.com>
>>>>> wrote:
>>>>>
>>>>>> Hello everyone,
>>>>>>
>>>>>> We use varnish in our production environment for caching content.
>>>>>>
>>>>>> Our Requirement:
>>>>>>
>>>>>> We are trying to block unauthorized requests at varnish based on the
>>>>>> source parameter in the URL and the client IP in the request header.
>>>>>>
>>>>>> For example:
>>>>>>
>>>>>> Sample URL:
>>>>>>
>>>>>> www.hostname:port/path?source=mobile&keyword= bags
>>>>>>
>>>>>> Let's assume there are 3 IPs [which are allowed to access varnish]
>>>>>> associated with the above request of mobile source.
>>>>>>
>>>>>> i.e *IP1, IP2, IP3*
>>>>>>
>>>>>> So if any request comes with the source as *mobile *and client-ip as
>>>>>> *IP4*, it's treated as an unauthorized request and should be blocked
>>>>>> at varnish.
>>>>>>
>>>>>>
>>>>>> What we have done for blocking?
>>>>>>
>>>>>> *Sample URL:*
>>>>>> www.hostname:port/path?source=mobile&keyword= bags
>>>>>>
>>>>>> Created a map using ACL as below:
>>>>>>
>>>>>> acl mobile_source{
>>>>>>
>>>>>> "IP1";
>>>>>>
>>>>>> "IP2";
>>>>>>
>>>>>> "IP3";
>>>>>>
>>>>>> }
>>>>>>
>>>>>> If(req.url ~ "source=mobile" && client.ip !~ mobile_source) {
>>>>>>
>>>>>> return(Synth(403, "varnish access denied!"))
>>>>>>
>>>>>> }
>>>>>>
>>>>>>
>>>>>> The problem we are facing:
>>>>>>
>>>>>> The source parameter can have different values like mobile, desktop,
>>>>>> laptop, tablet, etc. and each value can have different IPs associated with
>>>>>> it.
>>>>>>
>>>>>> ACL Rules will be as below:
>>>>>>
>>>>>> acl mobile_source{
>>>>>>
>>>>>> "IP1";
>>>>>>
>>>>>> "IP2";
>>>>>>
>>>>>> "IP3";
>>>>>>
>>>>>> }
>>>>>>
>>>>>> acl desktop_source{
>>>>>>
>>>>>> "IP4";
>>>>>>
>>>>>> "IP5";
>>>>>>
>>>>>> "IP6";
>>>>>>
>>>>>> }
>>>>>>
>>>>>> and so on,
>>>>>>
>>>>>>
>>>>>> If we wanted to block unauthorized access from different source vs IP
>>>>>> combinations, we would have to add that many conditions as below.
>>>>>>
>>>>>> If(
>>>>>>
>>>>>> (req.url ~ "source=mobile" && client.ip != mobile_source) ||
>>>>>>
>>>>>> (req.url ~ "source=desktop" && client.ip != desktop_source) ||
>>>>>>
>>>>>> (req.url ~ "source=laptop" && client.ip != laptop_source) ||
>>>>>>
>>>>>> (req.url ~ "source=tablet" && client.ip != tablet_source)
>>>>>>
>>>>>> ){
>>>>>>
>>>>>> return(Synth(403, "access denied!"))
>>>>>>
>>>>>> }
>>>>>>
>>>>>> This becomes worse, if we have 10's or 20's of source values.
>>>>>>
>>>>>> Our question:
>>>>>>
>>>>>> We would like to know if there is any way to optimize the code by
>>>>>> removing redundant checks so that we can scale it even if we have many
>>>>>> sources vs IP combinations.
>>>>>>
>>>>>>
>>>>>> Thanks & Regards
>>>>>> Uday Kumar
>>>>>> _______________________________________________
>>>>>> varnish-misc mailing list
>>>>>> varnish-misc at varnish-cache.org
>>>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>>>>
>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20231016/aa2902e6/attachment-0001.html>
More information about the varnish-misc
mailing list