varnish with apache mod_auth

[Andrei]

whoops, I totally meant to reply to the load balancing thread :|

On Sat, Mar 18, 2017 at 1:11 AM, Andrei <lagged at gmail.com> wrote:

> Out of curiosity, has anyone done a CDN of Varnish servers? I have 4
> Varnish servers in different datacenters around the world, and use anycast
> IPs to direct traffic based on the region. I managed to do cache
> replication using a "fanout" method for new cache hits to be replicated
> through an intermediary server to the related group of Varnish servers, but
> was wondering if anyone had a better method.
>
> On Fri, Mar 17, 2017 at 2:48 PM, Guillaume Quintard <
> guillaume at varnish-software.com> wrote:
>
>> Actually, Varnish should set the XFF header even before you enter
>> vcl_recv.
>>
>> --
>> Guillaume Quintard
>>
>> On Mar 17, 2017 19:23, "Hernán Marsili" <hernan at cmsmedios.com> wrote:
>>
>>> Ok, so I finally make it work with the suggested rule.
>>>
>>> On the vcl_recv I have:
>>>
>>> if (req.http.x-forwarded-for) {
>>>
>>>         set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", "
>>> + client.ip;
>>>
>>> set req.http.x-cdn-ip = regsub(req.http.X-Forwarded-For, "([^,]+), *([^
>>> ,]+)[ ,]?.*", "\2");
>>>
>>>     } else {
>>>
>>>         set req.http.X-Forwarded-For = client.ip;
>>>
>>> set req.http.x-cdn-ip = regsub(req.http.X-Forwarded-For, "([^,]+), *([^
>>> ,]+)[ ,]?.*", "\2");
>>>
>>>     }
>>>
>>> I then use Apache remote_ip to listen to x-cd-ip with this:
>>>
>>>  RemoteIPHeader x-cdn-ip
>>>
>>>  RemoteIPTrustedProxy 127.0.0.1 172.31.29.204
>>>
>>> I don't probable need the IF but since this was in place for some
>>> reason, I just leave it.
>>>
>>> It seems to be working just fine. What do you think?
>>>
>>> On Fri, Mar 17, 2017 at 10:32 AM Andrei <lagged at gmail.com> wrote:
>>>
>>>> Does the CDN not provide the IP you want in a separate header?
>>>> Typically CDN's have custom headers for just that which you can use as well
>>>>
>>>> On Fri, Mar 17, 2017 at 3:31 PM, Guillaume Quintard <
>>>> guillaume at varnish-software.com> wrote:
>>>>
>>>> If you have the ability to compile a vmod, you can use split() from
>>>> vmod-str (disclaimer: I wrote that) https://github.com/gquin
>>>> tard/libvmod-str/blob/master/src/vmod_str.vcc
>>>>
>>>> otherwise, to get the second ip, something like :
>>>>
>>>> regsub(req.http.xff, "([^,]+), *([^ ,]+)[ ,]?.*", "\2")
>>>>
>>>> should work. Fell free to test, using regex101.com for example. or
>>>> better, a Varnish Test case Case: https://gist.github.com/
>>>> gquintard/ee47432bb8b5c97b615d973b57b6338e
>>>> test it using: varnishtest foo.vtc
>>>>
>>>> --
>>>> Guillaume Quintard
>>>>
>>>> On Fri, Mar 17, 2017 at 1:33 PM, Hernán Marsili <hernan at cmsmedios.com>
>>>> wrote:
>>>>
>>>> Thank you! so, I figure I can parse the x-forwarded-for in which I have
>>>> 3 ips. The first one is the customer, the second one is the one 1 need (the
>>>> CDN) and the third I think is the load balancer.
>>>>
>>>> I can assign it to a new header x-cdn-ip and use apache_remoteip to use
>>>> that ip as the connecting ip.
>>>>
>>>> What do you think?
>>>>
>>>> Only problem here is to parse the second iP. I have something like this:
>>>>
>>>> set req.http.x-cdn-ip = regsub(req.http.X-Forwarded-For,
>>>> "^([^,]+),?.*$", "\1");
>>>>
>>>> I was able to get the first IP but not the second only which is the one
>>>> I need. Any one can point me in the right direction with the regsub?
>>>>
>>>> Thank you!
>>>>
>>>> On Fri, Mar 17, 2017 at 4:43 AM Andrei <lagged at gmail.com> wrote:
>>>>
>>>> Authenticated requests should typically bypass cache, unless you want
>>>> to hash the related session id(s), however that can get "interesting". I
>>>> suggest using an Apache module such as rpaf or remoteip in order for Apache
>>>> to set the client IP from the X-Forwarded-For header set by Varnish. This
>>>> way, you will not need to worry about whitelisting localhost, or other
>>>> cucumbersome iptables rules, and your IP restrictions will work as intended.
>>>>
>>>> On Fri, Mar 17, 2017 at 1:32 AM, Jason Price <japrice at gmail.com> wrote:
>>>>
>>>> I don't believe there's a trivial way to do this.
>>>>
>>>> Varnish will return the cached response to any IP address that comes
>>>> calling.  Even if the first request comes from a valid IP, which gets
>>>> passed through via X-Forward or similar, and mod_auth is tweaked to respond
>>>> to that, any subsequent request will not be seen by either apache or
>>>> mod_auth at all.
>>>>
>>>> You have a few options:
>>>> 1) IP Whitelists are a rather poor means of authentication.  Moving to
>>>> something else might be prudent.  But that's not easy.
>>>> 2) There are probably VMODs that do something similar.  If not and if
>>>> the list of IPs isn't too long, you could limit the IPs in VCL rather than
>>>> mod_auth.
>>>> 3) Push the list of IP addresses that can connect to the external port
>>>> down to IPTables or similar.
>>>> 4) Push the list of IP addresses to external Firewall, or Security
>>>> Group or whatever.
>>>>
>>>>
>>>>
>>>> On Thu, Mar 16, 2017 at 5:46 PM, Hernán Marsili <hernan at cmsmedios.com>
>>>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> We are having an issue with VARNISH and apache mod_auth. Varnish is on
>>>> port 80 serving users and Apache is the backend.
>>>>
>>>> We have servers restricting access only to authenticated users or
>>>> certain IP addresses. Since we installed Varnish the issue is that we need
>>>> to enable 127.0.0.1 as a permitted IP (required ip rule) so the Varnish can
>>>> fetch content. The problem, is that the real IP is not used and all the
>>>> other rules does not apply.
>>>>
>>>> Bottom line, how can we still control who is requesting using MOD_AUTH
>>>> and having Varnish?
>>>>
>>>> Regards
>>>> Hernán.
>>>>
>>>> _______________________________________________
>>>> varnish-misc mailing list
>>>> varnish-misc at varnish-cache.org
>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> varnish-misc mailing list
>>>> varnish-misc at varnish-cache.org
>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> varnish-misc mailing list
>>>> varnish-misc at varnish-cache.org
>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>>
>>>>
>>>>
>>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20170318/7ed6f388/attachment-0001.html>