varnish with apache mod_auth

[Andrei]

Out of curiosity, has anyone done a CDN of Varnish servers? I have 4
Varnish servers in different datacenters around the world, and use anycast
IPs to direct traffic based on the region. I managed to do cache
replication using a "fanout" method for new cache hits to be replicated
through an intermediary server to the related group of Varnish servers, but
was wondering if anyone had a better method.

On Fri, Mar 17, 2017 at 2:48 PM, Guillaume Quintard <
guillaume at varnish-software.com> wrote:

> Actually, Varnish should set the XFF header even before you enter
> vcl_recv.
>
> --
> Guillaume Quintard
>
> On Mar 17, 2017 19:23, "Hernán Marsili" <hernan at cmsmedios.com> wrote:
>
>> Ok, so I finally make it work with the suggested rule.
>>
>> On the vcl_recv I have:
>>
>> if (req.http.x-forwarded-for) {
>>
>>         set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " +
>> client.ip;
>>
>> set req.http.x-cdn-ip = regsub(req.http.X-Forwarded-For, "([^,]+), *([^
>> ,]+)[ ,]?.*", "\2");
>>
>>     } else {
>>
>>         set req.http.X-Forwarded-For = client.ip;
>>
>> set req.http.x-cdn-ip = regsub(req.http.X-Forwarded-For, "([^,]+), *([^
>> ,]+)[ ,]?.*", "\2");
>>
>>     }
>>
>> I then use Apache remote_ip to listen to x-cd-ip with this:
>>
>>  RemoteIPHeader x-cdn-ip
>>
>>  RemoteIPTrustedProxy 127.0.0.1 172.31.29.204
>>
>> I don't probable need the IF but since this was in place for some reason,
>> I just leave it.
>>
>> It seems to be working just fine. What do you think?
>>
>> On Fri, Mar 17, 2017 at 10:32 AM Andrei <lagged at gmail.com> wrote:
>>
>>> Does the CDN not provide the IP you want in a separate header? Typically
>>> CDN's have custom headers for just that which you can use as well
>>>
>>> On Fri, Mar 17, 2017 at 3:31 PM, Guillaume Quintard <
>>> guillaume at varnish-software.com> wrote:
>>>
>>> If you have the ability to compile a vmod, you can use split() from
>>> vmod-str (disclaimer: I wrote that) https://github.com/gquin
>>> tard/libvmod-str/blob/master/src/vmod_str.vcc
>>>
>>> otherwise, to get the second ip, something like :
>>>
>>> regsub(req.http.xff, "([^,]+), *([^ ,]+)[ ,]?.*", "\2")
>>>
>>> should work. Fell free to test, using regex101.com for example. or
>>> better, a Varnish Test case Case: https://gist.github.com/
>>> gquintard/ee47432bb8b5c97b615d973b57b6338e
>>> test it using: varnishtest foo.vtc
>>>
>>> --
>>> Guillaume Quintard
>>>
>>> On Fri, Mar 17, 2017 at 1:33 PM, Hernán Marsili <hernan at cmsmedios.com>
>>> wrote:
>>>
>>> Thank you! so, I figure I can parse the x-forwarded-for in which I have
>>> 3 ips. The first one is the customer, the second one is the one 1 need (the
>>> CDN) and the third I think is the load balancer.
>>>
>>> I can assign it to a new header x-cdn-ip and use apache_remoteip to use
>>> that ip as the connecting ip.
>>>
>>> What do you think?
>>>
>>> Only problem here is to parse the second iP. I have something like this:
>>>
>>> set req.http.x-cdn-ip = regsub(req.http.X-Forwarded-For,
>>> "^([^,]+),?.*$", "\1");
>>>
>>> I was able to get the first IP but not the second only which is the one
>>> I need. Any one can point me in the right direction with the regsub?
>>>
>>> Thank you!
>>>
>>> On Fri, Mar 17, 2017 at 4:43 AM Andrei <lagged at gmail.com> wrote:
>>>
>>> Authenticated requests should typically bypass cache, unless you want to
>>> hash the related session id(s), however that can get "interesting". I
>>> suggest using an Apache module such as rpaf or remoteip in order for Apache
>>> to set the client IP from the X-Forwarded-For header set by Varnish. This
>>> way, you will not need to worry about whitelisting localhost, or other
>>> cucumbersome iptables rules, and your IP restrictions will work as intended.
>>>
>>> On Fri, Mar 17, 2017 at 1:32 AM, Jason Price <japrice at gmail.com> wrote:
>>>
>>> I don't believe there's a trivial way to do this.
>>>
>>> Varnish will return the cached response to any IP address that comes
>>> calling.  Even if the first request comes from a valid IP, which gets
>>> passed through via X-Forward or similar, and mod_auth is tweaked to respond
>>> to that, any subsequent request will not be seen by either apache or
>>> mod_auth at all.
>>>
>>> You have a few options:
>>> 1) IP Whitelists are a rather poor means of authentication.  Moving to
>>> something else might be prudent.  But that's not easy.
>>> 2) There are probably VMODs that do something similar.  If not and if
>>> the list of IPs isn't too long, you could limit the IPs in VCL rather than
>>> mod_auth.
>>> 3) Push the list of IP addresses that can connect to the external port
>>> down to IPTables or similar.
>>> 4) Push the list of IP addresses to external Firewall, or Security Group
>>> or whatever.
>>>
>>>
>>>
>>> On Thu, Mar 16, 2017 at 5:46 PM, Hernán Marsili <hernan at cmsmedios.com>
>>> wrote:
>>>
>>> Hi,
>>>
>>> We are having an issue with VARNISH and apache mod_auth. Varnish is on
>>> port 80 serving users and Apache is the backend.
>>>
>>> We have servers restricting access only to authenticated users or
>>> certain IP addresses. Since we installed Varnish the issue is that we need
>>> to enable 127.0.0.1 as a permitted IP (required ip rule) so the Varnish can
>>> fetch content. The problem, is that the real IP is not used and all the
>>> other rules does not apply.
>>>
>>> Bottom line, how can we still control who is requesting using MOD_AUTH
>>> and having Varnish?
>>>
>>> Regards
>>> Hernán.
>>>
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc at varnish-cache.org
>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>
>>>
>>>
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc at varnish-cache.org
>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>
>>>
>>>
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc at varnish-cache.org
>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>
>>>
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20170318/da1fb36d/attachment-0001.html>