From dridi at varni.sh  Tue Apr  3 08:38:57 2018
From: dridi at varni.sh (Dridi Boukelmoune)
Date: Tue, 3 Apr 2018 10:38:57 +0200
Subject: Docs : Only handle actual PURGE HTTP methods,
 everything else is discarded
In-Reply-To: <001a01d3c839$64cc5480$2e64fd80$@solutions>
References: <001a01d3c839$64cc5480$2e64fd80$@solutions>
Message-ID: <CABoVN9ChEwRzKq7-zEN+Z0amrkqkEFihiCBi8LPCuQO+iUTv4A@mail.gmail.com>

On Fri, Mar 30, 2018 at 5:11 PM, AWA SOLUTIONS <contact at awa.solutions> wrote:
>
> Hi there,
>
> While reading the docs, I found this in https://www.varnish-software.com/wiki/content/tutorials/varnish/sample_vclTemplate.html :
>
> HANDLING HTTP PURGE
>
>   sub vcl_purge {
>     # Only handle actual PURGE HTTP methods, everything else is discarded
>     if (req.method != "PURGE") {
>       # restart request
>       set req.http.X-Purge = "Yes";
>       return(restart);
>     }
>   }
>
> But in mattiasgeniar/varnish-4.0-configuration-templates https://github.com/mattiasgeniar/varnish-4.0-configuration-templates/blob/master/default.vcl
>
> This is just the opposite logic :
>
> sub vcl_purge {
>   # Only handle actual PURGE HTTP methods, everything else is discarded
>   if (req.method == "PURGE") {
>     # restart request
>     set req.http.X-Purge = "Yes";
>     return (restart);
>   }
> }
>
> So I assume the latter, wiki page, has a typo ?!

I think the wiki is meant to feed from the community and give a single
location to find all sorts of tips. If Mattias is not credited we have
a problem, but I'm personally not into community tutorials as I way
too often (as in almost always) see irrelevant or harmful examples or
habits from the Varnish 2 or 3 days that have become irrelevant today
but refuse to die (yes, I'm looking at you, deflate=).

Doing a git blame on the if statement shows that it was a typo in
Mattias' repo probably inherited as-is in the wiki, changed 3 weeks
ago:

https://github.com/mattiasgeniar/varnish-4.0-configuration-templates/commit/18efd041a9f5c27b31cd89396ef6ea649acc0b70

> Last, in https://book.varnish-software.com/4.0/chapters/Cache_Invalidation.html , after similar vcl_recv conditions than above to return purge, vcl_purge subroutine sets req.method instead of req.http.X-Purge :
>
> sub vcl_purge {
>     set req.method = "GET";
>     return (restart);
> }
>
> Sounds logical to change req.method to avoid infinite loops, so why isn?t it the same in above examples ?

Full disclosure as I reveal my trainer hat, yes that's the way to go,
assuming you only cache responses to GET requests.

My question is rather "why isn't it documented upstream?"

https://varnish-cache.org/docs/trunk/users-guide/purging.html

We know our docs lack in certain areas. They aren't bad, and they
actually cover a lot, but they don't offer the best experience (and we
plan to at least try to improve that). It's only my guess but I'm sure
we'd all be happy to review and vet pull requests around
documentation.

At least I'd be happy to review "now that learned this the hard way I
wish it was documented here" patches.

Cheers

From t.winkelmann at ffh.de  Wed Apr  4 14:29:21 2018
From: t.winkelmann at ffh.de (Winkelmann, Thomas (RADIO TELE FFH - Online))
Date: Wed, 4 Apr 2018 16:29:21 +0200
Subject: How to install Varnish 6.0 on Ubuntu 17.10 or 18.04
Message-ID: <8A76B2DB46634944BEEBB218DAE22BC2654D51EC18@exchange-srv2.office.radioteleffh.de>

Hello,

I want to test http/2 of Varnish 6.0 on Ubuntu. So far I managed to install the varnish packages from package cloud as described here:

https://packagecloud.io/varnishcache/varnish60/install#manual-deb<https://packagecloud.io/varnishcache/varnish60/install%23manual-deb>

Also varnish is starting but it is crashing as soon as I'm using my existing config. It will include the vsthrottle VMOD which seems not to be installed on the system.

How can I get the Vmods for Ubuntu? I tried to download it from here https://github.com/varnish/varnish-modules but the compilation failed with thousands of errors.

Is there any documentation?

Thanks,
Thomas

________________________________
RADIO / TELE FFH GmbH & Co. Betriebs-KG
FFH-Platz 1, 61111 Bad Vilbel
HRA - Nr. 26092 Frankfurt/Main
USt.IdNr. DE 112152620
Gesch?ftsf?hrer / Programmdirektor: Hans-Dieter Hillmoth
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180404/670f0eec/attachment.html>

From dridi at varni.sh  Wed Apr  4 15:21:07 2018
From: dridi at varni.sh (Dridi Boukelmoune)
Date: Wed, 4 Apr 2018 17:21:07 +0200
Subject: How to install Varnish 6.0 on Ubuntu 17.10 or 18.04
In-Reply-To: <8A76B2DB46634944BEEBB218DAE22BC2654D51EC18@exchange-srv2.office.radioteleffh.de>
References: <8A76B2DB46634944BEEBB218DAE22BC2654D51EC18@exchange-srv2.office.radioteleffh.de>
Message-ID: <CABoVN9DMdSnmxv4kMYmdjD1qF_VErztS=B7SzL4X83D1U+WatQ@mail.gmail.com>

On Wed, Apr 4, 2018 at 4:29 PM, Winkelmann, Thomas (RADIO TELE FFH -
Online) <t.winkelmann at ffh.de> wrote:
> Hello,
>
> I want to test http/2 of Varnish 6.0 on Ubuntu. So far I managed to install
> the varnish packages from package cloud as described here:
>
> https://packagecloud.io/varnishcache/varnish60/install#manual-deb
>
> Also varnish is starting but it is crashing as soon as I?m using my existing
> config. It will include the vsthrottle VMOD which seems not to be installed
> on the system.

Careful with words like "crashing", you'll skyrocket our blood pressure :)

> How can I get the Vmods for Ubuntu? I tried to download it from here
> https://github.com/varnish/varnish-modules but the compilation failed with
> thousands of errors.
>
> Is there any documentation?

We have yet to produce a release compatible with Varnish 6...

Do you see these errors when building from the master branch? If so,
and if you have a github account, please open an issue. If you don't
have a github account please share compilation errors from the master
branch.

Cheers

From t.winkelmann at ffh.de  Thu Apr  5 06:20:28 2018
From: t.winkelmann at ffh.de (Winkelmann, Thomas (RADIO TELE FFH - Online))
Date: Thu, 5 Apr 2018 08:20:28 +0200
Subject: AW: How to install Varnish 6.0 on Ubuntu 17.10 or 18.04
In-Reply-To: <CABoVN9DMdSnmxv4kMYmdjD1qF_VErztS=B7SzL4X83D1U+WatQ@mail.gmail.com>
References: <8A76B2DB46634944BEEBB218DAE22BC2654D51EC18@exchange-srv2.office.radioteleffh.de>
 <CABoVN9DMdSnmxv4kMYmdjD1qF_VErztS=B7SzL4X83D1U+WatQ@mail.gmail.com>
Message-ID: <8A76B2DB46634944BEEBB218DAE22BC2654D51EC1F@exchange-srv2.office.radioteleffh.de>

Thank you very much for the response... I'm replacing "crashing" with "not starting" :) This is no surprise as it can not load the Vmod...

What are the dependencies for successfully building the vmods on Ubuntu 17.10? Is it only the "varnish-dev" package out of the packagecloud repository? Or do I need anything else?

Sorry for the questions, we are more webdevelopers than linux developers. Are there any plans to provide a varnish-modules package for 6.0?

-----Urspr?ngliche Nachricht-----
Von: Dridi Boukelmoune [mailto:dridi at varni.sh]
Gesendet: Mittwoch, 4. April 2018 17:21
An: Winkelmann, Thomas (RADIO TELE FFH - Online)
Cc: varnish-misc at varnish-cache.org
Betreff: Re: How to install Varnish 6.0 on Ubuntu 17.10 or 18.04

On Wed, Apr 4, 2018 at 4:29 PM, Winkelmann, Thomas (RADIO TELE FFH -
Online) <t.winkelmann at ffh.de> wrote:
> Hello,
>
> I want to test http/2 of Varnish 6.0 on Ubuntu. So far I managed to
> install the varnish packages from package cloud as described here:
>
> https://packagecloud.io/varnishcache/varnish60/install#manual-deb
>
> Also varnish is starting but it is crashing as soon as I?m using my
> existing config. It will include the vsthrottle VMOD which seems not
> to be installed on the system.

Careful with words like "crashing", you'll skyrocket our blood pressure :)

> How can I get the Vmods for Ubuntu? I tried to download it from here
> https://github.com/varnish/varnish-modules but the compilation failed
> with thousands of errors.
>
> Is there any documentation?

We have yet to produce a release compatible with Varnish 6...

Do you see these errors when building from the master branch? If so, and if you have a github account, please open an issue. If you don't have a github account please share compilation errors from the master branch.

Cheers

RADIO / TELE FFH GmbH & Co. Betriebs-KG
FFH-Platz 1, 61111 Bad Vilbel
HRA - Nr. 26092 Frankfurt/Main
USt.IdNr. DE 112152620
Gesch?ftsf?hrer / Programmdirektor: Hans-Dieter Hillmoth

From dridi at varni.sh  Thu Apr  5 14:32:26 2018
From: dridi at varni.sh (Dridi Boukelmoune)
Date: Thu, 5 Apr 2018 16:32:26 +0200
Subject: How to install Varnish 6.0 on Ubuntu 17.10 or 18.04
In-Reply-To: <8A76B2DB46634944BEEBB218DAE22BC2654D51EC1F@exchange-srv2.office.radioteleffh.de>
References: <8A76B2DB46634944BEEBB218DAE22BC2654D51EC18@exchange-srv2.office.radioteleffh.de>
 <CABoVN9DMdSnmxv4kMYmdjD1qF_VErztS=B7SzL4X83D1U+WatQ@mail.gmail.com>
 <8A76B2DB46634944BEEBB218DAE22BC2654D51EC1F@exchange-srv2.office.radioteleffh.de>
Message-ID: <CABoVN9DtVEOMCrfn_5za6XNN82dCoRbh8CeOiP_=f-+3=EkTWQ@mail.gmail.com>

> What are the dependencies for successfully building the vmods on Ubuntu 17.10? Is it only the "varnish-dev" package out of the packagecloud repository? Or do I need anything else?

I'm more a Fedora person myself so I may get the package names wrong,
but you need pkg-config, python >= 2.7 and python-docutils. When
building from git you also need the autotools (autoconf automake
libtool) that may be part of build-essential.

> Sorry for the questions, we are more webdevelopers than linux developers. Are there any plans to provide a varnish-modules package for 6.0?

We provide "dist" releases (autotools-independent source archives) and
we should get one soon. Regarding packages, we rely on downstream
distributors for this one.

Dridi

From ciapnz at gmail.com  Sat Apr  7 10:10:00 2018
From: ciapnz at gmail.com (Danila Vershinin)
Date: Sat, 7 Apr 2018 13:10:00 +0300
Subject: Softpurge and nearly immediate refresh?
Message-ID: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com>

Hi,

What I work with:

* Grace mode configured to be 60 seconds when backend is healthy
* Using softpurge module to adjust TTL to 0 upon PURGE.

The whole idea is increasing chances that visitors will get cached page after cache was PURGEd for a page.

Standard piece:
sub vcl_hit {
    if (obj.ttl >= 0s) {
        # normal hit
        return (deliver);
    }

    if (std.healthy(req.backend_hint)) {
        # Backend is healthy. Limit age to 60s.
        if (obj.ttl + 60s > 0s) {
            set req.http.grace = "normal(limited)";
            return (deliver);
        } else {
            return(fetch);
        }
    } else {
        # ...
    }
}
And use of softpurge:

sub vcl_miss {
    if (req.method == "PURGE") {
        softpurge.softpurge();
        return (synth(200, "Successful softpurge"));
    }
}

sub vcl_hit {
    if (req.method == "PURGE") {
        softpurge.softpurge();
        return (synth(200, "Successful softpurge"));
    }
}


Current behaviour:

* send PURGE for cached page
* Visitor goes to the page within 60 seconds and sees a stale cached page (triggering background refresh)
* Further visits to the page will show refreshed page

What I?m looking for:

Trigger the background refresh right after PURGE while still leveraging grace mode :) That is, serve stale cache for only as long as it takes to actually generate the new page, and not wait for 60 seconds:

* upon PURGE: set TTL to 0 (softpurge) + trigger background page request (possible?)
* serve stale cache only while the page is generated

I could have adjusted the ?healthy backend grace period? to lower than 60s, but I?m basically checking to see if it?s possible to refresh ?nearly? immediately in this kind of setup.

Hope I made any sense :)

Best Regards,
Danila

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180407/b0003f31/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180407/b0003f31/attachment-0001.bin>

From mghhimu at gmail.com  Mon Apr  9 09:12:07 2018
From: mghhimu at gmail.com (Md. Golam Hossain)
Date: Mon, 09 Apr 2018 09:12:07 +0000
Subject: Setting a variable/header in vcl_recv and checking it in other sub
 routines
Message-ID: <CAO+qN0SKNK1sJcVJ7zNQXfNrLgXB-yMLV3PpbLWU-UPCBLCp4A@mail.gmail.com>

If I want to make some decision by checking some header(eg:
req.http.test) in vcl_backend_response being set in vcl_recv but not
sent as http header while fetching from backend or while sending
response to client, how should I do this?

Let's say, I want to store all the objects that is set to 'true' in
req.http.test header in vcl_recv and check it in vcl_backend_beresponse
to store that object in "test.storage". But, as I am unsetting the
header(bereq.http.test) in vcl_backend_fetch, it can't be used in
vcl_backend_response while setting storage hint.

Any suggestion?
Thanks in advance.


On Mon, Apr 9, 2018, 11:00 Md. Golam Hossain <mghhimu at gmail.com> wrote:

> If I want to make some decision by checking some header(eg:
> req.http.test) in vcl_backend_response being set in vcl_recv but not
> sent as http header while fetching from backend or while sending
> response to client, how should I do this?
>
> Let's say, I want to store all the objects that is set to 'true' in
> req.http.test header in vcl_recv and check it in vcl_backend_beresponse
> to store that object in "test.storage". But, as I am unsetting the
> header(bereq.http.test) in vcl_backend_fetch, it can't be used in
> vcl_backend_response while setting storage hint.
>
> Any suggestion?
> Thanks in advance.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180409/413e0dcb/attachment.html>

From guillaume at varnish-software.com  Mon Apr  9 09:16:49 2018
From: guillaume at varnish-software.com (Guillaume Quintard)
Date: Mon, 9 Apr 2018 11:16:49 +0200
Subject: Setting a variable/header in vcl_recv and checking it in other
 sub routines
In-Reply-To: <CAO+qN0SKNK1sJcVJ7zNQXfNrLgXB-yMLV3PpbLWU-UPCBLCp4A@mail.gmail.com>
References: <CAO+qN0SKNK1sJcVJ7zNQXfNrLgXB-yMLV3PpbLWU-UPCBLCp4A@mail.gmail.com>
Message-ID: <CAJ6ZYQyG6Czh6n8A_QiQw5wMvLYOasSAFN=z-dZDr+HHCPewKw@mail.gmail.com>

Hi,

Use vmod_var, from varnish-modules. You'll store variables without setting
headers.

-- 
Guillaume Quintard

On Mon, Apr 9, 2018 at 11:12 AM, Md. Golam Hossain <mghhimu at gmail.com>
wrote:

> If I want to make some decision by checking some header(eg:
> req.http.test) in vcl_backend_response being set in vcl_recv but not
> sent as http header while fetching from backend or while sending
> response to client, how should I do this?
>
> Let's say, I want to store all the objects that is set to 'true' in
> req.http.test header in vcl_recv and check it in vcl_backend_beresponse
> to store that object in "test.storage". But, as I am unsetting the
> header(bereq.http.test) in vcl_backend_fetch, it can't be used in
> vcl_backend_response while setting storage hint.
>
> Any suggestion?
> Thanks in advance.
>
>
> On Mon, Apr 9, 2018, 11:00 Md. Golam Hossain <mghhimu at gmail.com> wrote:
>
>> If I want to make some decision by checking some header(eg:
>> req.http.test) in vcl_backend_response being set in vcl_recv but not
>> sent as http header while fetching from backend or while sending
>> response to client, how should I do this?
>>
>> Let's say, I want to store all the objects that is set to 'true' in
>> req.http.test header in vcl_recv and check it in vcl_backend_beresponse
>> to store that object in "test.storage". But, as I am unsetting the
>> header(bereq.http.test) in vcl_backend_fetch, it can't be used in
>> vcl_backend_response while setting storage hint.
>>
>> Any suggestion?
>> Thanks in advance.
>>
>>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180409/5ba5540d/attachment.html>

From guillaume at varnish-software.com  Mon Apr  9 09:18:45 2018
From: guillaume at varnish-software.com (Guillaume Quintard)
Date: Mon, 9 Apr 2018 11:18:45 +0200
Subject: Softpurge and nearly immediate refresh?
In-Reply-To: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com>
References: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com>
Message-ID: <CAJ6ZYQzJ6a6PPrVOeKZSCCsZMnhuoh2rMp3dmT2xQTjs=JLKcg@mail.gmail.com>

Hi,

You can purge then set the method to GET then restart. Would that work for
you?

Other way is to use req.hash_always_miss that will only revalidate if we
are able to fetch a new object.

-- 
Guillaume Quintard

On Sat, Apr 7, 2018 at 12:10 PM, Danila Vershinin <ciapnz at gmail.com> wrote:

> Hi,
>
> What I work with:
>
> * Grace mode configured to be 60 seconds when backend is healthy
> * Using softpurge module to adjust TTL to 0 upon PURGE.
>
> The whole idea is increasing chances that visitors will get cached page
> after cache was PURGEd for a page.
>
> Standard piece:
>
> sub vcl_hit {
>     if (obj.ttl >= 0s) {
>         # normal hit
>         return (deliver);
>     }
>
>     if (std.healthy(req.backend_hint)) {
>         # Backend is healthy. Limit age to 60s.
>         if (obj.ttl + 60s > 0s) {
>             set req.http.grace = "normal(limited)";
>             return (deliver);
>         } else {
>             return(fetch);
>         }
>     } else {
>         # ...
>     }
> }
>
> And use of softpurge:
>
> sub vcl_miss {
>     if (req.method == "PURGE") {
>         softpurge.softpurge();
>         return (synth(200, "Successful softpurge"));
>     }
> }
>
> sub vcl_hit {
>     if (req.method == "PURGE") {
>         softpurge.softpurge();
>         return (synth(200, "Successful softpurge"));
>     }
> }
>
>
>
> Current behaviour:
>
> * send PURGE for cached page
> * Visitor goes to the page within 60 seconds and sees a stale cached page
> (triggering background refresh)
> * Further visits to the page will show refreshed page
>
> What I?m looking for:
>
> Trigger the background refresh right after PURGE while still leveraging
> grace mode :) That is, serve stale cache for only as long as it takes to
> actually generate the new page, and not wait for 60 seconds:
>
> * upon PURGE: set TTL to 0 (softpurge) + trigger background page request
> (possible?)
> * serve stale cache only while the page is generated
>
> I could have adjusted the ?healthy backend grace period? to lower than
> 60s, but I?m basically checking to see if it?s possible to refresh ?nearly?
> immediately in this kind of setup.
>
> Hope I made any sense :)
>
> Best Regards,
> Danila
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180409/bbedcd7c/attachment.html>

From ciapnz at gmail.com  Mon Apr  9 14:47:26 2018
From: ciapnz at gmail.com (Danila Vershinin)
Date: Mon, 9 Apr 2018 17:47:26 +0300
Subject: Softpurge and nearly immediate refresh?
In-Reply-To: <CAJ6ZYQzJ6a6PPrVOeKZSCCsZMnhuoh2rMp3dmT2xQTjs=JLKcg@mail.gmail.com>
References: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com>
 <CAJ6ZYQzJ6a6PPrVOeKZSCCsZMnhuoh2rMp3dmT2xQTjs=JLKcg@mail.gmail.com>
Message-ID: <DD4F6695-CAC4-430B-81A5-820A48999801@gmail.com>

Hi Guillaume,

This should absolutely work. Thank you :)

Best Regards,
Danila

> On 9 Apr 2018, at 12:18, Guillaume Quintard <guillaume at varnish-software.com> wrote:
> 
> You can purge then set the method to GET then restart. Would that work for you?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180409/6658704e/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180409/6658704e/attachment-0001.bin>

From ciapnz at gmail.com  Wed Apr 11 09:26:30 2018
From: ciapnz at gmail.com (Danila Vershinin)
Date: Wed, 11 Apr 2018 12:26:30 +0300
Subject: Softpurge and nearly immediate refresh?
In-Reply-To: <CAJ6ZYQzJ6a6PPrVOeKZSCCsZMnhuoh2rMp3dmT2xQTjs=JLKcg@mail.gmail.com>
References: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com>
 <CAJ6ZYQzJ6a6PPrVOeKZSCCsZMnhuoh2rMp3dmT2xQTjs=JLKcg@mail.gmail.com>
Message-ID: <15B57C74-5AD8-4A83-AE4E-5431F5FE127C@gmail.com>

Hi Guillaume,

A bit puzzled on something. If we use Vary: by some header.. am I correct that we need multiple restarts to refresh each object variation?

Since the background fetch would only refresh the variation that matched initial purge request. 

Sent from my iPhone

> On 9 Apr 2018, at 12:18, Guillaume Quintard <guillaume at varnish-software.com> wrote:
> 
> Hi,
> 
> You can purge then set the method to GET then restart. Would that work for you?
> 
> Other way is to use req.hash_always_miss that will only revalidate if we are able to fetch a new object.
> 
> -- 
> Guillaume Quintard
> 
>> On Sat, Apr 7, 2018 at 12:10 PM, Danila Vershinin <ciapnz at gmail.com> wrote:
>> Hi,
>> 
>> What I work with:
>> 
>> * Grace mode configured to be 60 seconds when backend is healthy
>> * Using softpurge module to adjust TTL to 0 upon PURGE.
>> 
>> The whole idea is increasing chances that visitors will get cached page after cache was PURGEd for a page.
>> 
>> Standard piece:
>> sub vcl_hit {
>>     if (obj.ttl >= 0s) {
>>         # normal hit
>>         return (deliver);
>>     }
>> 
>>     if (std.healthy(req.backend_hint)) {
>>         # Backend is healthy. Limit age to 60s.
>>         if (obj.ttl + 60s > 0s) {
>>             set req.http.grace = "normal(limited)";
>>             return (deliver);
>>         } else {
>>             return(fetch);
>>         }
>>     } else {
>>         # ...
>>     }
>> }
>> And use of softpurge:
>> 
>> sub vcl_miss {
>>     if (req.method == "PURGE") {
>>         softpurge.softpurge();
>>         return (synth(200, "Successful softpurge"));
>>     }
>> }
>> 
>> sub vcl_hit {
>>     if (req.method == "PURGE") {
>>         softpurge.softpurge();
>>         return (synth(200, "Successful softpurge"));
>>     }
>> }
>> 
>> 
>> Current behaviour:
>> 
>> * send PURGE for cached page
>> * Visitor goes to the page within 60 seconds and sees a stale cached page (triggering background refresh)
>> * Further visits to the page will show refreshed page
>> 
>> What I?m looking for:
>> 
>> Trigger the background refresh right after PURGE while still leveraging grace mode :) That is, serve stale cache for only as long as it takes to actually generate the new page, and not wait for 60 seconds:
>> 
>> * upon PURGE: set TTL to 0 (softpurge) + trigger background page request (possible?)
>> * serve stale cache only while the page is generated
>> 
>> I could have adjusted the ?healthy backend grace period? to lower than 60s, but I?m basically checking to see if it?s possible to refresh ?nearly? immediately in this kind of setup.
>> 
>> Hope I made any sense :) 
>> 
>> Best Regards,
>> Danila
>> 
>> 
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180411/d3e57eb2/attachment.html>

From guillaume at varnish-software.com  Wed Apr 11 09:37:23 2018
From: guillaume at varnish-software.com (Guillaume Quintard)
Date: Wed, 11 Apr 2018 11:37:23 +0200
Subject: Softpurge and nearly immediate refresh?
In-Reply-To: <15B57C74-5AD8-4A83-AE4E-5431F5FE127C@gmail.com>
References: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com>
 <CAJ6ZYQzJ6a6PPrVOeKZSCCsZMnhuoh2rMp3dmT2xQTjs=JLKcg@mail.gmail.com>
 <15B57C74-5AD8-4A83-AE4E-5431F5FE127C@gmail.com>
Message-ID: <CAJ6ZYQztsih2M7mVPbfbye714XE=DGaO4qDKD1Cxh1+xjKjyhA@mail.gmail.com>

Hi,

That's indeed correct, a purge will kill all variations, and the restart
only fetches one.

The req.hash_always_miss trick however only kills/revalidate one variation.

At this moment, we have no way to purge/revalidate all the object under one
hash key.

-- 
Guillaume Quintard

On Wed, Apr 11, 2018 at 11:26 AM, Danila Vershinin <ciapnz at gmail.com> wrote:

> Hi Guillaume,
>
> A bit puzzled on something. If we use Vary: by some header.. am I correct
> that we need multiple restarts to refresh each object variation?
>
> Since the background fetch would only refresh the variation that matched
> initial purge request.
>
> Sent from my iPhone
>
> On 9 Apr 2018, at 12:18, Guillaume Quintard <guillaume at varnish-software.
> com> wrote:
>
> Hi,
>
> You can purge then set the method to GET then restart. Would that work for
> you?
>
> Other way is to use req.hash_always_miss that will only revalidate if we
> are able to fetch a new object.
>
> --
> Guillaume Quintard
>
> On Sat, Apr 7, 2018 at 12:10 PM, Danila Vershinin <ciapnz at gmail.com>
> wrote:
>
>> Hi,
>>
>> What I work with:
>>
>> * Grace mode configured to be 60 seconds when backend is healthy
>> * Using softpurge module to adjust TTL to 0 upon PURGE.
>>
>> The whole idea is increasing chances that visitors will get cached page
>> after cache was PURGEd for a page.
>>
>> Standard piece:
>>
>> sub vcl_hit {
>>     if (obj.ttl >= 0s) {
>>         # normal hit
>>         return (deliver);
>>     }
>>
>>     if (std.healthy(req.backend_hint)) {
>>         # Backend is healthy. Limit age to 60s.
>>         if (obj.ttl + 60s > 0s) {
>>             set req.http.grace = "normal(limited)";
>>             return (deliver);
>>         } else {
>>             return(fetch);
>>         }
>>     } else {
>>         # ...
>>     }
>> }
>>
>> And use of softpurge:
>>
>> sub vcl_miss {
>>     if (req.method == "PURGE") {
>>         softpurge.softpurge();
>>         return (synth(200, "Successful softpurge"));
>>     }
>> }
>>
>> sub vcl_hit {
>>     if (req.method == "PURGE") {
>>         softpurge.softpurge();
>>         return (synth(200, "Successful softpurge"));
>>     }
>> }
>>
>>
>>
>> Current behaviour:
>>
>> * send PURGE for cached page
>> * Visitor goes to the page within 60 seconds and sees a stale cached page
>> (triggering background refresh)
>> * Further visits to the page will show refreshed page
>>
>> What I?m looking for:
>>
>> Trigger the background refresh right after PURGE while still leveraging
>> grace mode :) That is, serve stale cache for only as long as it takes to
>> actually generate the new page, and not wait for 60 seconds:
>>
>> * upon PURGE: set TTL to 0 (softpurge) + trigger background page request
>> (possible?)
>> * serve stale cache only while the page is generated
>>
>> I could have adjusted the ?healthy backend grace period? to lower than
>> 60s, but I?m basically checking to see if it?s possible to refresh ?nearly?
>> immediately in this kind of setup.
>>
>> Hope I made any sense :)
>>
>> Best Regards,
>> Danila
>>
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180411/e377e2e2/attachment.html>

From cosimo at streppone.it  Wed Apr 11 10:47:48 2018
From: cosimo at streppone.it (Cosimo Streppone)
Date: Wed, 11 Apr 2018 12:47:48 +0200
Subject: vmod_vsthrottle memory usage
Message-ID: <1523443668.3231337.1334128512.596EC649@webmail.messagingengine.com>

Hi,

I'm looking at the vsthrottle vmod. Have a proof of concept VCL up and running,
and it seems to work very nicely (with varnish 4.1.9). Thanks for this work!

I'm wondering what vsthrottle's behaviour is when there's no memory available,
specifically when tb_alloc() fails.

>From what I can understand from the code[1]:

    b = tb_alloc(digest, limit, period, now);
    AZ(VRB_INSERT(tbtree, &v->buckets, b));

and tb_alloc() also checks for a non-zero malloc result (AN()).

I understand we're talking about a remote possibility, and I see vsthrottle
reclaims unused memory. However, when no memory is available for the malloc,
will varnish leave a trace in syslog, exit and restart by way of the asserts?

-- 
Cosimo

[1] https://github.com/varnish/varnish-modules/blob/master/src/vmod_vsthrottle.c#L126-L127

From guillaume at varnish-software.com  Wed Apr 11 12:02:13 2018
From: guillaume at varnish-software.com (Guillaume Quintard)
Date: Wed, 11 Apr 2018 14:02:13 +0200
Subject: vmod_vsthrottle memory usage
In-Reply-To: <1523443668.3231337.1334128512.596EC649@webmail.messagingengine.com>
References: <1523443668.3231337.1334128512.596EC649@webmail.messagingengine.com>
Message-ID: <CAJ6ZYQxEpk6Zb9_NBExvDy=6gM+JQa1Nx-y5HJf-QXda0=1sxg@mail.gmail.com>

Hi cosimo,

Very simple, we crash and bring the whole thing down:
https://github.com/varnish/varnish-modules/blob/master/src/vmod_vsthrottle.c#L99

IIRC, on Linux, malloc always returns with an allocation, or OOM-kills you

-- 
Guillaume Quintard

On Wed, Apr 11, 2018 at 12:47 PM, Cosimo Streppone <cosimo at streppone.it>
wrote:

> Hi,
>
> I'm looking at the vsthrottle vmod. Have a proof of concept VCL up and
> running,
> and it seems to work very nicely (with varnish 4.1.9). Thanks for this
> work!
>
> I'm wondering what vsthrottle's behaviour is when there's no memory
> available,
> specifically when tb_alloc() fails.
>
> From what I can understand from the code[1]:
>
>     b = tb_alloc(digest, limit, period, now);
>     AZ(VRB_INSERT(tbtree, &v->buckets, b));
>
> and tb_alloc() also checks for a non-zero malloc result (AN()).
>
> I understand we're talking about a remote possibility, and I see vsthrottle
> reclaims unused memory. However, when no memory is available for the
> malloc,
> will varnish leave a trace in syslog, exit and restart by way of the
> asserts?
>
> --
> Cosimo
>
> [1] https://github.com/varnish/varnish-modules/blob/master/
> src/vmod_vsthrottle.c#L126-L127
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180411/a10ac132/attachment-0001.html>

From dridi at varni.sh  Wed Apr 11 12:07:46 2018
From: dridi at varni.sh (Dridi Boukelmoune)
Date: Wed, 11 Apr 2018 14:07:46 +0200
Subject: vmod_vsthrottle memory usage
In-Reply-To: <CAJ6ZYQxEpk6Zb9_NBExvDy=6gM+JQa1Nx-y5HJf-QXda0=1sxg@mail.gmail.com>
References: <1523443668.3231337.1334128512.596EC649@webmail.messagingengine.com>
 <CAJ6ZYQxEpk6Zb9_NBExvDy=6gM+JQa1Nx-y5HJf-QXda0=1sxg@mail.gmail.com>
Message-ID: <CABoVN9DqdEtkktroKJd=KYjK2WPaRyif8Eb-BR6QZ63cwrK__w@mail.gmail.com>

> IIRC, on Linux, malloc always returns with an allocation, or OOM-kills you

That's the default behavior but I believe you can also change that.

From cosimo at streppone.it  Wed Apr 11 13:59:22 2018
From: cosimo at streppone.it (Cosimo Streppone)
Date: Wed, 11 Apr 2018 15:59:22 +0200
Subject: VCL: returning early from a custom function
Message-ID: <1523455162.3350556.1334316448.10C69177@webmail.messagingengine.com>

Hi again,

two messages in a day after a few years :-)

I have code similar to the following (also here[1]):

    sub vcl_recv {
        ...
        call rate_limit;
        ...
    }

    # Throttling based on request inspection
    sub rate_limit {

        if (req.url ~ "pattern1") {
           std.log("pattern1 requests must never be throttled");
           return; # <---- Need to return early here, but can't do it
        }

        if (req.url ~ "pattern2") {
           if (vsthrottle.is_denied("pattern2" + client.identity, 100, 10s)) {
               std.log("pattern2 throttling for ip " + client.identity);
               return(synth(429, "ETOOMANYREQUESTS"));
           }
        }

        if (vsthrottle.is_denied("ip:" + client.identity, 500, 10s)) {
            std.log("global throttling for ip " + client.identity);
            return(synth(429, "ETOOMANYREQUESTS"));
        }

    }

Ideally, I'd like to return early from rate_limit() when I know that I don't need
to enforce any rate limiting for some types of requests, but I understand
that's not implemented.

I found some alternatives, though generally they feel uglier.
Any ideas?

-- 
Cosimo

[1] https://gist.github.com/cosimo/6d3318bf173357dfb2652b7b2e81e1e0#file-gistfile1-txt-L18

From dridi at varni.sh  Wed Apr 11 14:30:11 2018
From: dridi at varni.sh (Dridi Boukelmoune)
Date: Wed, 11 Apr 2018 16:30:11 +0200
Subject: VCL: returning early from a custom function
In-Reply-To: <1523455162.3350556.1334316448.10C69177@webmail.messagingengine.com>
References: <1523455162.3350556.1334316448.10C69177@webmail.messagingengine.com>
Message-ID: <CABoVN9C9Sm-BujfiGQkXv8Mjep8-qS=ck7ej7cgPHaQ58ZxV5w@mail.gmail.com>

> I found some alternatives, though generally they feel uglier.
> Any ideas?

This has been discussed in the past, including the possibility to
return a value, but as of today you can't break out of a subroutine.

https://github.com/varnishcache/varnish-cache/wiki/VIP2%3A-VCL-typed-functions

With this feature you could break out of a subroutine by turning it
into a function that returns VOID. We are not there yet.

Dridi

From Tom.Anheyer at berlinonline.de  Wed Apr 11 14:43:11 2018
From: Tom.Anheyer at berlinonline.de (Anheyer, Tom)
Date: Wed, 11 Apr 2018 14:43:11 +0000
Subject: VCL: returning early from a custom function
References: <1523455162.3350556.1334316448.10C69177@webmail.messagingengine.com>
Message-ID: <8AE07C2AEF8F2D4CA8887FDC94F779B012220B46@klnsmxmbxp005.DMSYS.de>

What about:

sub rate_limit {

        if (req.url ~ "pattern1") {
           std.log("pattern1 requests must never be throttled");
        }
        else {
            if (req.url ~ "pattern2") {
                if (vsthrottle.is_denied("pattern2" + client.identity,
100, 10s)) {
                    std.log("pattern2 throttling for ip " +
client.identity);
                    return(synth(429, "ETOOMANYREQUESTS"));
                }
            }

            if (vsthrottle.is_denied("ip:" + client.identity, 500, 10s)) {
                std.log("global throttling for ip " + client.identity);
                return(synth(429, "ETOOMANYREQUESTS"));
            }
        }

    }

tom


Am 11.04.2018 um 15:59 schrieb Cosimo Streppone:
> Hi again,
>
> two messages in a day after a few years :-)
>
> I have code similar to the following (also here[1]):
>
>     sub vcl_recv {
>         ...
>         call rate_limit;
>         ...
>     }
>
>     # Throttling based on request inspection
>     sub rate_limit {
>
>         if (req.url ~ "pattern1") {
>            std.log("pattern1 requests must never be throttled");
>            return; # <---- Need to return early here, but can't do it
>         }
>
>         if (req.url ~ "pattern2") {
>            if (vsthrottle.is_denied("pattern2" + client.identity, 100, 10s)) {
>                std.log("pattern2 throttling for ip " + client.identity);
>                return(synth(429, "ETOOMANYREQUESTS"));
>            }
>         }
>
>         if (vsthrottle.is_denied("ip:" + client.identity, 500, 10s)) {
>             std.log("global throttling for ip " + client.identity);
>             return(synth(429, "ETOOMANYREQUESTS"));
>         }
>
>     }
>
> Ideally, I'd like to return early from rate_limit() when I know that I don't need
> to enforce any rate limiting for some types of requests, but I understand
> that's not implemented.
>
> I found some alternatives, though generally they feel uglier.
> Any ideas?
>


From ciapnz at gmail.com  Wed Apr 11 20:39:36 2018
From: ciapnz at gmail.com (Danila Vershinin)
Date: Wed, 11 Apr 2018 23:39:36 +0300
Subject: Softpurge and nearly immediate refresh?
In-Reply-To: <CAJ6ZYQztsih2M7mVPbfbye714XE=DGaO4qDKD1Cxh1+xjKjyhA@mail.gmail.com>
References: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com>
 <CAJ6ZYQzJ6a6PPrVOeKZSCCsZMnhuoh2rMp3dmT2xQTjs=JLKcg@mail.gmail.com>
 <15B57C74-5AD8-4A83-AE4E-5431F5FE127C@gmail.com>
 <CAJ6ZYQztsih2M7mVPbfbye714XE=DGaO4qDKD1Cxh1+xjKjyhA@mail.gmail.com>
Message-ID: <6D649EE6-CB1B-43CE-BC27-507FB1E8CCE2@gmail.com>

So I guess I?m looking into having one background fetch trigger another background fetch (to sequentially refresh different object variants). In this fashion:

1. Client PURGE request will do softpurge.softpurge(); return(restart); with GET method, etc. which will all lead to "return deliver()" in limited grace logic to fire background fetch .

2. Then in background fetch:
   ? in its vcl_deliver() - the current object variation has already entered cache, so setting varied header to value2, removing grace limited flag and calling restart(). This way it should continue revalidation for the other object variant
   ? we land inside limited grace logic again (as it?s a different object variant) and return deliver() again thus firing off second background fetch (which will refresh second object variant).

So the standard grace logic + something like this:

sub vcl_deliver {
   if (req.http.grace == "normal(limited)" && req.http.Varied-Header = 'value1') {
     set req.http.Varied-Header = 'value2?;
     unset req.http.grace;
     return (restart);
   }
}

However, it won?t work at least because req.http.grace flag will be set for both the background fetch and the request that kicked it off. (it will be there in vcl_deliver of both).
Question is how can we tell if we are inside background fetch?

Best Regards,
Danila

> On 11 Apr 2018, at 12:37, Guillaume Quintard <guillaume at varnish-software.com> wrote:
> 
> Hi,
> 
> That's indeed correct, a purge will kill all variations, and the restart only fetches one.
> 
> The req.hash_always_miss trick however only kills/revalidate one variation.
> 
> At this moment, we have no way to purge/revalidate all the object under one hash key.
> 
> --
> Guillaume Quintard
> 
> On Wed, Apr 11, 2018 at 11:26 AM, Danila Vershinin <ciapnz at gmail.com <mailto:ciapnz at gmail.com>> wrote:
> Hi Guillaume,
> 
> A bit puzzled on something. If we use Vary: by some header.. am I correct that we need multiple restarts to refresh each object variation?
> 
> Since the background fetch would only refresh the variation that matched initial purge request.
> 
> Sent from my iPhone
> 
> On 9 Apr 2018, at 12:18, Guillaume Quintard <guillaume at varnish-software.com <mailto:guillaume at varnish-software.com>> wrote:
> 
>> Hi,
>> 
>> You can purge then set the method to GET then restart. Would that work for you?
>> 
>> Other way is to use req.hash_always_miss that will only revalidate if we are able to fetch a new object.
>> 
>> --
>> Guillaume Quintard
>> 
>> On Sat, Apr 7, 2018 at 12:10 PM, Danila Vershinin <ciapnz at gmail.com <mailto:ciapnz at gmail.com>> wrote:
>> Hi,
>> 
>> What I work with:
>> 
>> * Grace mode configured to be 60 seconds when backend is healthy
>> * Using softpurge module to adjust TTL to 0 upon PURGE.
>> 
>> The whole idea is increasing chances that visitors will get cached page after cache was PURGEd for a page.
>> 
>> Standard piece:
>> sub vcl_hit {
>>     if (obj.ttl >= 0s) {
>>         # normal hit
>>         return (deliver);
>>     }
>> 
>>     if (std.healthy(req.backend_hint)) {
>>         # Backend is healthy. Limit age to 60s.
>>         if (obj.ttl + 60s > 0s) {
>>             set req.http.grace = "normal(limited)";
>>             return (deliver);
>>         } else {
>>             return(fetch);
>>         }
>>     } else {
>>         # ...
>>     }
>> }
>> And use of softpurge:
>> 
>> sub vcl_miss {
>>     if (req.method == "PURGE") {
>>         softpurge.softpurge();
>>         return (synth(200, "Successful softpurge"));
>>     }
>> }
>> 
>> sub vcl_hit {
>>     if (req.method == "PURGE") {
>>         softpurge.softpurge();
>>         return (synth(200, "Successful softpurge"));
>>     }
>> }
>> 
>> 
>> Current behaviour:
>> 
>> * send PURGE for cached page
>> * Visitor goes to the page within 60 seconds and sees a stale cached page (triggering background refresh)
>> * Further visits to the page will show refreshed page
>> 
>> What I?m looking for:
>> 
>> Trigger the background refresh right after PURGE while still leveraging grace mode :) That is, serve stale cache for only as long as it takes to actually generate the new page, and not wait for 60 seconds:
>> 
>> * upon PURGE: set TTL to 0 (softpurge) + trigger background page request (possible?)
>> * serve stale cache only while the page is generated
>> 
>> I could have adjusted the ?healthy backend grace period? to lower than 60s, but I?m basically checking to see if it?s possible to refresh ?nearly? immediately in this kind of setup.
>> 
>> Hope I made any sense :)
>> 
>> Best Regards,
>> Danila
>> 
>> 
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org <mailto:varnish-misc at varnish-cache.org>
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc <https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc>
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180411/4252e936/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180411/4252e936/attachment-0001.bin>

From ciapnz at gmail.com  Thu Apr 12 07:24:42 2018
From: ciapnz at gmail.com (Danila Vershinin)
Date: Thu, 12 Apr 2018 10:24:42 +0300
Subject: Softpurge and nearly immediate refresh?
In-Reply-To: <CAJ6ZYQz8YJACmAbSYYz6JLJUX=P3T5ZX6C6azWRWLFhgaUbLUw@mail.gmail.com>
References: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com>
 <CAJ6ZYQzJ6a6PPrVOeKZSCCsZMnhuoh2rMp3dmT2xQTjs=JLKcg@mail.gmail.com>
 <15B57C74-5AD8-4A83-AE4E-5431F5FE127C@gmail.com>
 <CAJ6ZYQztsih2M7mVPbfbye714XE=DGaO4qDKD1Cxh1+xjKjyhA@mail.gmail.com>
 <6D649EE6-CB1B-43CE-BC27-507FB1E8CCE2@gmail.com>
 <CAJ6ZYQz8YJACmAbSYYz6JLJUX=P3T5ZX6C6azWRWLFhgaUbLUw@mail.gmail.com>
Message-ID: <D2642FE3-4339-42C9-83A4-ED734C66086D@gmail.com>

Couldn?t really find anything useful besides obj.hits which indirectly tells us it?s a background fetch. So:

sub vcl_deliver {
   if (req.http.grace == "normal(limited)" && obj.hits == 0 && req.http.Varied-Header = 'value1') {
     set req.http.Varied-Header = 'value2?;
     unset req.http.grace;
     return (restart);
   }
}

? :)

Best Regards,
Danila

> On 12 Apr 2018, at 00:24, Guillaume Quintard <guillaume at varnish-software.com> wrote:
> 
> There's a beresp attribute for that last one (I'm on mobile, so I'm going to point you to man vcl :-))
> 
> --
> Guillaume Quintard
> 
> On Wed, Apr 11, 2018, 22:39 Danila Vershinin <ciapnz at gmail.com <mailto:ciapnz at gmail.com>> wrote:
> So I guess I?m looking into having one background fetch trigger another background fetch (to sequentially refresh different object variants). In this fashion:
> 
> 1. Client PURGE request will do softpurge.softpurge(); return(restart); with GET method, etc. which will all lead to "return deliver()" in limited grace logic to fire background fetch .
> 
> 2. Then in background fetch:
>    ? in its vcl_deliver() - the current object variation has already entered cache, so setting varied header to value2, removing grace limited flag and calling restart(). This way it should continue revalidation for the other object variant
>    ? we land inside limited grace logic again (as it?s a different object variant) and return deliver() again thus firing off second background fetch (which will refresh second object variant).
> 
> So the standard grace logic + something like this:
> 
> sub vcl_deliver {
>    if (req.http.grace == "normal(limited)" && req.http.Varied-Header = 'value1') {
>      set req.http.Varied-Header = 'value2?;
>      unset req.http.grace;
>      return (restart);
>    }
> }
> 
> However, it won?t work at least because req.http.grace flag will be set for both the background fetch and the request that kicked it off. (it will be there in vcl_deliver of both).
> Question is how can we tell if we are inside background fetch?
> 
> Best Regards,
> Danila
> 
>> On 11 Apr 2018, at 12:37, Guillaume Quintard <guillaume at varnish-software.com <mailto:guillaume at varnish-software.com>> wrote:
>> 
>> Hi,
>> 
>> That's indeed correct, a purge will kill all variations, and the restart only fetches one.
>> 
>> The req.hash_always_miss trick however only kills/revalidate one variation.
>> 
>> At this moment, we have no way to purge/revalidate all the object under one hash key.
>> 
>> --
>> Guillaume Quintard
>> 
>> On Wed, Apr 11, 2018 at 11:26 AM, Danila Vershinin <ciapnz at gmail.com <mailto:ciapnz at gmail.com>> wrote:
>> Hi Guillaume,
>> 
>> A bit puzzled on something. If we use Vary: by some header.. am I correct that we need multiple restarts to refresh each object variation?
>> 
>> Since the background fetch would only refresh the variation that matched initial purge request.
>> 
>> Sent from my iPhone
>> 
>> On 9 Apr 2018, at 12:18, Guillaume Quintard <guillaume at varnish-software.com <mailto:guillaume at varnish-software.com>> wrote:
>> 
>>> Hi,
>>> 
>>> You can purge then set the method to GET then restart. Would that work for you?
>>> 
>>> Other way is to use req.hash_always_miss that will only revalidate if we are able to fetch a new object.
>>> 
>>> --
>>> Guillaume Quintard
>>> 
>>> On Sat, Apr 7, 2018 at 12:10 PM, Danila Vershinin <ciapnz at gmail.com <mailto:ciapnz at gmail.com>> wrote:
>>> Hi,
>>> 
>>> What I work with:
>>> 
>>> * Grace mode configured to be 60 seconds when backend is healthy
>>> * Using softpurge module to adjust TTL to 0 upon PURGE.
>>> 
>>> The whole idea is increasing chances that visitors will get cached page after cache was PURGEd for a page.
>>> 
>>> Standard piece:
>>> sub vcl_hit {
>>>     if (obj.ttl >= 0s) {
>>>         # normal hit
>>>         return (deliver);
>>>     }
>>> 
>>>     if (std.healthy(req.backend_hint)) {
>>>         # Backend is healthy. Limit age to 60s.
>>>         if (obj.ttl + 60s > 0s) {
>>>             set req.http.grace = "normal(limited)";
>>>             return (deliver);
>>>         } else {
>>>             return(fetch);
>>>         }
>>>     } else {
>>>         # ...
>>>     }
>>> }
>>> And use of softpurge:
>>> 
>>> sub vcl_miss {
>>>     if (req.method == "PURGE") {
>>>         softpurge.softpurge();
>>>         return (synth(200, "Successful softpurge"));
>>>     }
>>> }
>>> 
>>> sub vcl_hit {
>>>     if (req.method == "PURGE") {
>>>         softpurge.softpurge();
>>>         return (synth(200, "Successful softpurge"));
>>>     }
>>> }
>>> 
>>> 
>>> Current behaviour:
>>> 
>>> * send PURGE for cached page
>>> * Visitor goes to the page within 60 seconds and sees a stale cached page (triggering background refresh)
>>> * Further visits to the page will show refreshed page
>>> 
>>> What I?m looking for:
>>> 
>>> Trigger the background refresh right after PURGE while still leveraging grace mode :) That is, serve stale cache for only as long as it takes to actually generate the new page, and not wait for 60 seconds:
>>> 
>>> * upon PURGE: set TTL to 0 (softpurge) + trigger background page request (possible?)
>>> * serve stale cache only while the page is generated
>>> 
>>> I could have adjusted the ?healthy backend grace period? to lower than 60s, but I?m basically checking to see if it?s possible to refresh ?nearly? immediately in this kind of setup.
>>> 
>>> Hope I made any sense :)
>>> 
>>> Best Regards,
>>> Danila
>>> 
>>> 
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc at varnish-cache.org <mailto:varnish-misc at varnish-cache.org>
>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc <https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc>
>>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180412/57c8cfaf/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180412/57c8cfaf/attachment.bin>

From cosimo at streppone.it  Thu Apr 12 08:27:34 2018
From: cosimo at streppone.it (Cosimo Streppone)
Date: Thu, 12 Apr 2018 10:27:34 +0200
Subject: VCL: returning early from a custom function
In-Reply-To: <8AE07C2AEF8F2D4CA8887FDC94F779B012220B46@klnsmxmbxp005.DMSYS.de>
References: <1523455162.3350556.1334316448.10C69177@webmail.messagingengine.com>
 <8AE07C2AEF8F2D4CA8887FDC94F779B012220B46@klnsmxmbxp005.DMSYS.de>
Message-ID: <1523521654.1523074.1335350472.039AD34C@webmail.messagingengine.com>

On Wed, Apr 11, 2018, at 16:43, Anheyer, Tom wrote:
> What about:
> 
> sub rate_limit {
> 
>         if (req.url ~ "pattern1") {
>            std.log("pattern1 requests must never be throttled");
>         }
>         else {
>             if (req.url ~ "pattern2") {
>                 if (vsthrottle.is_denied("pattern2" + client.identity,
> 100, 10s)) {
>                     std.log("pattern2 throttling for ip " +
> client.identity);
>                     return(synth(429, "ETOOMANYREQUESTS"));
>                 }
>             }
> 
>             if (vsthrottle.is_denied("ip:" + client.identity, 500, 10s)) {
>                 std.log("global throttling for ip " + client.identity);
>                 return(synth(429, "ETOOMANYREQUESTS"));
>             }
>         }
> 
>     }

Yes, that's one of the alternatives, though the "whitelisted" patterns
are more than just one.

I ended up doing something like:

   set req.http.Rate-Limit = "1";
   
   if (whitelisted-pattern1) { set req.http.Rate-Limit = "0"; }
   if (whitelisted-pattern2) { set req.http.Rate-Limit = "0"; }
   ...

   if (req.http.Rate-Limit == "0") {
       # no rate limiting
   } else {
       # yes, do rate limit
   }

I intend to look at vmod_var to see if I can make this prettier.

-- 
Cosimo

From guillaume at varnish-software.com  Thu Apr 12 09:25:47 2018
From: guillaume at varnish-software.com (Guillaume Quintard)
Date: Thu, 12 Apr 2018 09:25:47 +0000
Subject: Softpurge and nearly immediate refresh?
In-Reply-To: <D2642FE3-4339-42C9-83A4-ED734C66086D@gmail.com>
References: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com>
 <CAJ6ZYQzJ6a6PPrVOeKZSCCsZMnhuoh2rMp3dmT2xQTjs=JLKcg@mail.gmail.com>
 <15B57C74-5AD8-4A83-AE4E-5431F5FE127C@gmail.com>
 <CAJ6ZYQztsih2M7mVPbfbye714XE=DGaO4qDKD1Cxh1+xjKjyhA@mail.gmail.com>
 <6D649EE6-CB1B-43CE-BC27-507FB1E8CCE2@gmail.com>
 <CAJ6ZYQz8YJACmAbSYYz6JLJUX=P3T5ZX6C6azWRWLFhgaUbLUw@mail.gmail.com>
 <D2642FE3-4339-42C9-83A4-ED734C66086D@gmail.com>
Message-ID: <CAJ6ZYQzX0ZYFONv2S1ORi=8HnA54LmT5Ese3bN_RUnEVpD4oig@mail.gmail.com>

bereq.is_bgfetch?

-- 
Guillaume Quintard

On Thu, Apr 12, 2018, 09:24 Danila Vershinin <ciapnz at gmail.com> wrote:

> Couldn?t really find anything useful besides obj.hits which indirectly
> tells us it?s a background fetch. So:
>
> sub vcl_deliver {
>    if (req.http.grace == "normal(limited)" && *obj.hits == 0* &&
> req.http.Varied-Header = 'value1') {
>      set req.http.Varied-Header = 'value2?;
>      unset req.http.grace;
>      return (restart);
>    }
> }
>
> ? :)
>
> Best Regards,
> Danila
>
> On 12 Apr 2018, at 00:24, Guillaume Quintard <
> guillaume at varnish-software.com> wrote:
>
> There's a beresp attribute for that last one (I'm on mobile, so I'm going
> to point you to man vcl :-))
>
> --
> Guillaume Quintard
>
> On Wed, Apr 11, 2018, 22:39 Danila Vershinin <ciapnz at gmail.com> wrote:
>
>> So I guess I?m looking into having one background fetch trigger another
>> background fetch (to sequentially refresh different object variants). In
>> this fashion:
>>
>> 1. Client PURGE request will do softpurge.softpurge(); return(restart);
>> with GET method, etc. which will all lead to "return deliver()" in limited
>> grace logic to fire background fetch .
>>
>> 2. Then in background fetch:
>>    ? in its vcl_deliver() - the current object variation has already
>> entered cache, so setting varied header to value2, removing grace limited
>> flag and calling restart(). This way it should continue revalidation for
>> the other object variant
>>    ? we land inside limited grace logic again (as it?s a different object
>> variant) and return deliver() again thus firing off second background fetch
>> (which will refresh second object variant).
>>
>> So the standard grace logic + something like this:
>>
>> sub vcl_deliver {
>>    if (req.http.grace == "normal(limited)" && req.http.Varied-Header =
>> 'value1') {
>>      set req.http.Varied-Header = 'value2?;
>>      unset req.http.grace;
>>      return (restart);
>>    }
>> }
>>
>> However, it won?t work at least because req.http.grace flag will be set
>> for both the background fetch and the request that kicked it off. (it will
>> be there in vcl_deliver of both).
>> Question is how can we tell if we are inside background fetch?
>>
>> Best Regards,
>> Danila
>>
>> On 11 Apr 2018, at 12:37, Guillaume Quintard <
>> guillaume at varnish-software.com> wrote:
>>
>> Hi,
>>
>> That's indeed correct, a purge will kill all variations, and the restart
>> only fetches one.
>>
>> The req.hash_always_miss trick however only kills/revalidate one
>> variation.
>>
>> At this moment, we have no way to purge/revalidate all the object under
>> one hash key.
>>
>> --
>> Guillaume Quintard
>>
>> On Wed, Apr 11, 2018 at 11:26 AM, Danila Vershinin <ciapnz at gmail.com>
>> wrote:
>>
>>> Hi Guillaume,
>>>
>>> A bit puzzled on something. If we use Vary: by some header.. am I
>>> correct that we need multiple restarts to refresh each object variation?
>>>
>>> Since the background fetch would only refresh the variation that matched
>>> initial purge request.
>>>
>>> Sent from my iPhone
>>>
>>> On 9 Apr 2018, at 12:18, Guillaume Quintard <
>>> guillaume at varnish-software.com> wrote:
>>>
>>> Hi,
>>>
>>> You can purge then set the method to GET then restart. Would that work
>>> for you?
>>>
>>> Other way is to use req.hash_always_miss that will only revalidate if we
>>> are able to fetch a new object.
>>>
>>> --
>>> Guillaume Quintard
>>>
>>> On Sat, Apr 7, 2018 at 12:10 PM, Danila Vershinin <ciapnz at gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> What I work with:
>>>>
>>>> * Grace mode configured to be 60 seconds when backend is healthy
>>>> * Using softpurge module to adjust TTL to 0 upon PURGE.
>>>>
>>>> The whole idea is increasing chances that visitors will get cached page
>>>> after cache was PURGEd for a page.
>>>>
>>>> Standard piece:
>>>>
>>>> sub vcl_hit {
>>>>     if (obj.ttl >= 0s) {
>>>>         # normal hit
>>>>         return (deliver);
>>>>     }
>>>>
>>>>     if (std.healthy(req.backend_hint)) {
>>>>         # Backend is healthy. Limit age to 60s.
>>>>         if (obj.ttl + 60s > 0s) {
>>>>             set req.http.grace = "normal(limited)";
>>>>             return (deliver);
>>>>         } else {
>>>>             return(fetch);
>>>>         }
>>>>     } else {
>>>>         # ...
>>>>     }
>>>> }
>>>>
>>>> And use of softpurge:
>>>>
>>>> sub vcl_miss {
>>>>     if (req.method == "PURGE") {
>>>>         softpurge.softpurge();
>>>>         return (synth(200, "Successful softpurge"));
>>>>     }
>>>> }
>>>>
>>>> sub vcl_hit {
>>>>     if (req.method == "PURGE") {
>>>>         softpurge.softpurge();
>>>>         return (synth(200, "Successful softpurge"));
>>>>     }
>>>> }
>>>>
>>>>
>>>>
>>>> Current behaviour:
>>>>
>>>> * send PURGE for cached page
>>>> * Visitor goes to the page within 60 seconds and sees a stale cached
>>>> page (triggering background refresh)
>>>> * Further visits to the page will show refreshed page
>>>>
>>>> What I?m looking for:
>>>>
>>>> Trigger the background refresh right after PURGE while still leveraging
>>>> grace mode :) That is, serve stale cache for only as long as it takes to
>>>> actually generate the new page, and not wait for 60 seconds:
>>>>
>>>> * upon PURGE: set TTL to 0 (softpurge) + trigger background page
>>>> request (possible?)
>>>> * serve stale cache only while the page is generated
>>>>
>>>> I could have adjusted the ?healthy backend grace period? to lower than
>>>> 60s, but I?m basically checking to see if it?s possible to refresh ?nearly?
>>>> immediately in this kind of setup.
>>>>
>>>> Hope I made any sense :)
>>>>
>>>> Best Regards,
>>>> Danila
>>>>
>>>>
>>>> _______________________________________________
>>>> varnish-misc mailing list
>>>> varnish-misc at varnish-cache.org
>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>>
>>>
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180412/71912da7/attachment-0001.html>

From ciapnz at gmail.com  Thu Apr 12 09:28:57 2018
From: ciapnz at gmail.com (Danila Vershinin)
Date: Thu, 12 Apr 2018 12:28:57 +0300
Subject: Softpurge and nearly immediate refresh?
In-Reply-To: <CAJ6ZYQzX0ZYFONv2S1ORi=8HnA54LmT5Ese3bN_RUnEVpD4oig@mail.gmail.com>
References: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com>
 <CAJ6ZYQzJ6a6PPrVOeKZSCCsZMnhuoh2rMp3dmT2xQTjs=JLKcg@mail.gmail.com>
 <15B57C74-5AD8-4A83-AE4E-5431F5FE127C@gmail.com>
 <CAJ6ZYQztsih2M7mVPbfbye714XE=DGaO4qDKD1Cxh1+xjKjyhA@mail.gmail.com>
 <6D649EE6-CB1B-43CE-BC27-507FB1E8CCE2@gmail.com>
 <CAJ6ZYQz8YJACmAbSYYz6JLJUX=P3T5ZX6C6azWRWLFhgaUbLUw@mail.gmail.com>
 <D2642FE3-4339-42C9-83A4-ED734C66086D@gmail.com>
 <CAJ6ZYQzX0ZYFONv2S1ORi=8HnA54LmT5Ese3bN_RUnEVpD4oig@mail.gmail.com>
Message-ID: <A46F56F5-69F5-4FA9-BC2B-8EB1D8CBCDD3@gmail.com>

Ah, I?m using Varnish 4.1, guess might have to look into >= 5.2.

Thanks! :)

Best Regards,
Danila

> On 12 Apr 2018, at 12:25, Guillaume Quintard <guillaume at varnish-software.com> wrote:
> 
> bereq.is_bgfetch?
> 
> --
> Guillaume Quintard
> 
> On Thu, Apr 12, 2018, 09:24 Danila Vershinin <ciapnz at gmail.com <mailto:ciapnz at gmail.com>> wrote:
> Couldn?t really find anything useful besides obj.hits which indirectly tells us it?s a background fetch. So:
> 
> sub vcl_deliver {
>    if (req.http.grace == "normal(limited)" && obj.hits == 0 && req.http.Varied-Header = 'value1') {
>      set req.http.Varied-Header = 'value2?;
>      unset req.http.grace;
>      return (restart);
>    }
> }
> 
> ? :)
> 
> Best Regards,
> Danila
> 
>> On 12 Apr 2018, at 00:24, Guillaume Quintard <guillaume at varnish-software.com <mailto:guillaume at varnish-software.com>> wrote:
>> 
>> There's a beresp attribute for that last one (I'm on mobile, so I'm going to point you to man vcl :-))
>> 
>> --
>> Guillaume Quintard
>> 
>> On Wed, Apr 11, 2018, 22:39 Danila Vershinin <ciapnz at gmail.com <mailto:ciapnz at gmail.com>> wrote:
>> So I guess I?m looking into having one background fetch trigger another background fetch (to sequentially refresh different object variants). In this fashion:
>> 
>> 1. Client PURGE request will do softpurge.softpurge(); return(restart); with GET method, etc. which will all lead to "return deliver()" in limited grace logic to fire background fetch .
>> 
>> 2. Then in background fetch:
>>    ? in its vcl_deliver() - the current object variation has already entered cache, so setting varied header to value2, removing grace limited flag and calling restart(). This way it should continue revalidation for the other object variant
>>    ? we land inside limited grace logic again (as it?s a different object variant) and return deliver() again thus firing off second background fetch (which will refresh second object variant).
>> 
>> So the standard grace logic + something like this:
>> 
>> sub vcl_deliver {
>>    if (req.http.grace == "normal(limited)" && req.http.Varied-Header = 'value1') {
>>      set req.http.Varied-Header = 'value2?;
>>      unset req.http.grace;
>>      return (restart);
>>    }
>> }
>> 
>> However, it won?t work at least because req.http.grace flag will be set for both the background fetch and the request that kicked it off. (it will be there in vcl_deliver of both).
>> Question is how can we tell if we are inside background fetch?
>> 
>> Best Regards,
>> Danila
>> 
>>> On 11 Apr 2018, at 12:37, Guillaume Quintard <guillaume at varnish-software.com <mailto:guillaume at varnish-software.com>> wrote:
>>> 
>>> Hi,
>>> 
>>> That's indeed correct, a purge will kill all variations, and the restart only fetches one.
>>> 
>>> The req.hash_always_miss trick however only kills/revalidate one variation.
>>> 
>>> At this moment, we have no way to purge/revalidate all the object under one hash key.
>>> 
>>> --
>>> Guillaume Quintard
>>> 
>>> On Wed, Apr 11, 2018 at 11:26 AM, Danila Vershinin <ciapnz at gmail.com <mailto:ciapnz at gmail.com>> wrote:
>>> Hi Guillaume,
>>> 
>>> A bit puzzled on something. If we use Vary: by some header.. am I correct that we need multiple restarts to refresh each object variation?
>>> 
>>> Since the background fetch would only refresh the variation that matched initial purge request.
>>> 
>>> Sent from my iPhone
>>> 
>>> On 9 Apr 2018, at 12:18, Guillaume Quintard <guillaume at varnish-software.com <mailto:guillaume at varnish-software.com>> wrote:
>>> 
>>>> Hi,
>>>> 
>>>> You can purge then set the method to GET then restart. Would that work for you?
>>>> 
>>>> Other way is to use req.hash_always_miss that will only revalidate if we are able to fetch a new object.
>>>> 
>>>> --
>>>> Guillaume Quintard
>>>> 
>>>> On Sat, Apr 7, 2018 at 12:10 PM, Danila Vershinin <ciapnz at gmail.com <mailto:ciapnz at gmail.com>> wrote:
>>>> Hi,
>>>> 
>>>> What I work with:
>>>> 
>>>> * Grace mode configured to be 60 seconds when backend is healthy
>>>> * Using softpurge module to adjust TTL to 0 upon PURGE.
>>>> 
>>>> The whole idea is increasing chances that visitors will get cached page after cache was PURGEd for a page.
>>>> 
>>>> Standard piece:
>>>> sub vcl_hit {
>>>>     if (obj.ttl >= 0s) {
>>>>         # normal hit
>>>>         return (deliver);
>>>>     }
>>>> 
>>>>     if (std.healthy(req.backend_hint)) {
>>>>         # Backend is healthy. Limit age to 60s.
>>>>         if (obj.ttl + 60s > 0s) {
>>>>             set req.http.grace = "normal(limited)";
>>>>             return (deliver);
>>>>         } else {
>>>>             return(fetch);
>>>>         }
>>>>     } else {
>>>>         # ...
>>>>     }
>>>> }
>>>> And use of softpurge:
>>>> 
>>>> sub vcl_miss {
>>>>     if (req.method == "PURGE") {
>>>>         softpurge.softpurge();
>>>>         return (synth(200, "Successful softpurge"));
>>>>     }
>>>> }
>>>> 
>>>> sub vcl_hit {
>>>>     if (req.method == "PURGE") {
>>>>         softpurge.softpurge();
>>>>         return (synth(200, "Successful softpurge"));
>>>>     }
>>>> }
>>>> 
>>>> 
>>>> Current behaviour:
>>>> 
>>>> * send PURGE for cached page
>>>> * Visitor goes to the page within 60 seconds and sees a stale cached page (triggering background refresh)
>>>> * Further visits to the page will show refreshed page
>>>> 
>>>> What I?m looking for:
>>>> 
>>>> Trigger the background refresh right after PURGE while still leveraging grace mode :) That is, serve stale cache for only as long as it takes to actually generate the new page, and not wait for 60 seconds:
>>>> 
>>>> * upon PURGE: set TTL to 0 (softpurge) + trigger background page request (possible?)
>>>> * serve stale cache only while the page is generated
>>>> 
>>>> I could have adjusted the ?healthy backend grace period? to lower than 60s, but I?m basically checking to see if it?s possible to refresh ?nearly? immediately in this kind of setup.
>>>> 
>>>> Hope I made any sense :)
>>>> 
>>>> Best Regards,
>>>> Danila
>>>> 
>>>> 
>>>> _______________________________________________
>>>> varnish-misc mailing list
>>>> varnish-misc at varnish-cache.org <mailto:varnish-misc at varnish-cache.org>
>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc <https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc>
>>>> 
>>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180412/a673f5e4/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180412/a673f5e4/attachment.bin>

From hermunn at varnish-software.com  Thu Apr 12 13:13:27 2018
From: hermunn at varnish-software.com (=?UTF-8?Q?P=C3=A5l_Hermunn_Johansen?=)
Date: Thu, 12 Apr 2018 15:13:27 +0200
Subject: Softpurge and nearly immediate refresh?
In-Reply-To: <A46F56F5-69F5-4FA9-BC2B-8EB1D8CBCDD3@gmail.com>
References: <4B189FC7-F241-4D79-9580-1E3EC60E294E@gmail.com>
 <CAJ6ZYQzJ6a6PPrVOeKZSCCsZMnhuoh2rMp3dmT2xQTjs=JLKcg@mail.gmail.com>
 <15B57C74-5AD8-4A83-AE4E-5431F5FE127C@gmail.com>
 <CAJ6ZYQztsih2M7mVPbfbye714XE=DGaO4qDKD1Cxh1+xjKjyhA@mail.gmail.com>
 <6D649EE6-CB1B-43CE-BC27-507FB1E8CCE2@gmail.com>
 <CAJ6ZYQz8YJACmAbSYYz6JLJUX=P3T5ZX6C6azWRWLFhgaUbLUw@mail.gmail.com>
 <D2642FE3-4339-42C9-83A4-ED734C66086D@gmail.com>
 <CAJ6ZYQzX0ZYFONv2S1ORi=8HnA54LmT5Ese3bN_RUnEVpD4oig@mail.gmail.com>
 <A46F56F5-69F5-4FA9-BC2B-8EB1D8CBCDD3@gmail.com>
Message-ID: <CAMNFQgNPEo2d0owv-h85kJG8jgJLx8gMQtdbNoBPG11+MqzdAQ@mail.gmail.com>

> On 12 Apr 2018, at 12:25, Guillaume Quintard
> <guillaume at varnish-software.com> wrote:
>
> bereq.is_bgfetch?

2018-04-12 11:28 GMT+02:00 Danila Vershinin <ciapnz at gmail.com>:
> Ah, I?m using Varnish 4.1, guess might have to look into >= 5.2.

Not necessary - is_bgfetch was back ported to 4.1, and is in the
latest release 4.1.9 (but not in 4.1.8).

P?l

From michael at kennedy.ie  Fri Apr 13 15:42:29 2018
From: michael at kennedy.ie (Michael Kennedy)
Date: Fri, 13 Apr 2018 15:42:29 +0000
Subject: Compare Hex to Decimal
Message-ID: <DB5PR08MB0648AABB77F848DFE6D8E138C9BD0@DB5PR08MB0648.eurprd08.prod.outlook.com>

Hi,


I have a URL that contains a Unix hex timestamp (something like 5ace8c5d). The timestamp should be within X minutes of now, and is used as part of an access control mechanism. I want to do something like this:


if (std.time(req.http.x-hextime, now) < now - 1w) {
...
}

Is there an efficient way in VCL to either convert Hex to Dec so that this comparison can be done, or some other solution that does not involve changing the data in the URL.

Thanks,

Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180413/2553b313/attachment.html>

From dannysantos1985 at gmail.com  Fri Apr 13 21:52:41 2018
From: dannysantos1985 at gmail.com (Daniel Santos)
Date: Fri, 13 Apr 2018 22:52:41 +0100
Subject: Fwd: Dont understand how to implement this FAQ topic
In-Reply-To: <CAL6VdkQ81HqiMD3RY15oBDm0HFbbZbykoxw9bv6VM1OG7Z1ngQ@mail.gmail.com>
References: <CAL6VdkQ81HqiMD3RY15oBDm0HFbbZbykoxw9bv6VM1OG7Z1ngQ@mail.gmail.com>
Message-ID: <CAL6VdkS1yZzrhTEowW0v4MPbWfUQtKFdJjQ1yOMSJSXCe-jrRw@mail.gmail.com>

I don't understand the this FAQ topic
<https://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-clients-to-https>
.

I want to implement it. In my /etc/varnish/default.vcl I have this:

backend apache {
    .host = "127.0.0.1";
    .port = "5555";
}

backend node_hello {
    .host = "127.0.0.1";
    .port = "5556";
}

import std;

sub vcl_recv {
    # Happens before we check if we have this in cache already.
    #
    # Typically you clean up the request here, removing cookies you don't need,
    # rewriting the request, etc.

    if ((std.port(local.ip) == 6086) && (std.port(server.ip) == 443)) {
        set req.http.X-Forwarded-Proto = "https";
    }

    if(req.http.host ~ "domain1.com") {
        set req.backend_hint = apache;
    }
    elsif(req.http.host == "domain2.com") {
        set req.backend_hint = node_hello;
    }
}

How should my implementation become?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180413/5dcd910f/attachment.html>

From guillaume at varnish-software.com  Fri Apr 13 21:57:14 2018
From: guillaume at varnish-software.com (Guillaume Quintard)
Date: Fri, 13 Apr 2018 21:57:14 +0000
Subject: Dont understand how to implement this FAQ topic
In-Reply-To: <CAL6VdkS1yZzrhTEowW0v4MPbWfUQtKFdJjQ1yOMSJSXCe-jrRw@mail.gmail.com>
References: <CAL6VdkQ81HqiMD3RY15oBDm0HFbbZbykoxw9bv6VM1OG7Z1ngQ@mail.gmail.com>
 <CAL6VdkS1yZzrhTEowW0v4MPbWfUQtKFdJjQ1yOMSJSXCe-jrRw@mail.gmail.com>
Message-ID: <CAJ6ZYQydwcNR9nmcKWgifXtJgVcGrdt3sgeKRHbXRJdBWfp=bg@mail.gmail.com>

What is your goal, what don't you understand?

-- 
Guillaume Quintard

On Fri, Apr 13, 2018, 23:52 Daniel Santos <dannysantos1985 at gmail.com> wrote:

> I don't understand the this FAQ topic
> <https://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-clients-to-https>
> .
>
> I want to implement it. In my /etc/varnish/default.vcl I have this:
>
> backend apache {
>     .host = "127.0.0.1";
>     .port = "5555";
> }
>
> backend node_hello {
>     .host = "127.0.0.1";
>     .port = "5556";
> }
>
> import std;
>
> sub vcl_recv {
>     # Happens before we check if we have this in cache already.
>     #
>     # Typically you clean up the request here, removing cookies you don't need,
>     # rewriting the request, etc.
>
>     if ((std.port(local.ip) == 6086) && (std.port(server.ip) == 443)) {
>         set req.http.X-Forwarded-Proto = "https";
>     }
>
>     if(req.http.host ~ "domain1.com") {
>         set req.backend_hint = apache;
>     }
>     elsif(req.http.host == "domain2.com") {
>         set req.backend_hint = node_hello;
>     }
> }
>
> How should my implementation become?
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180413/c4b05cbe/attachment.html>

From guillaume at varnish-software.com  Sat Apr 14 06:17:12 2018
From: guillaume at varnish-software.com (Guillaume Quintard)
Date: Sat, 14 Apr 2018 06:17:12 +0000
Subject: Dont understand how to implement this FAQ topic
In-Reply-To: <CAL6VdkTizckBkVk+3hTPTgo1HtTJUWZvQh25hLVUY8LTOzENrw@mail.gmail.com>
References: <CAL6VdkQ81HqiMD3RY15oBDm0HFbbZbykoxw9bv6VM1OG7Z1ngQ@mail.gmail.com>
 <CAL6VdkS1yZzrhTEowW0v4MPbWfUQtKFdJjQ1yOMSJSXCe-jrRw@mail.gmail.com>
 <CAJ6ZYQydwcNR9nmcKWgifXtJgVcGrdt3sgeKRHbXRJdBWfp=bg@mail.gmail.com>
 <CAL6VdkTizckBkVk+3hTPTgo1HtTJUWZvQh25hLVUY8LTOzENrw@mail.gmail.com>
Message-ID: <CAJ6ZYQycaJGDBBfv73LNaxJB5mGUEZyfWhTW-aoKLM5N8nfguQ@mail.gmail.com>

Please keep the mail-in in the loop so others can benefit from this
discussion.

Have a look at this:
https://info.varnish-software.com/blog/rewriting-urls-with-varnish-redirection

It covers exactly your use-case, I think.

-- 
Guillaume Quintard


On Sat, Apr 14, 2018, 00:01 Daniel Santos <dannysantos1985 at gmail.com> wrote:

> I don't understand where I will put the set req.backend_hint on
> https://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-cliehttps://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-clients-to-httpsnts-to-https
> <https://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-clients-to-https>
> .
> And if in order to have http redirect to https will I have to put:
>
> if (std.port(local.ip) == 80 && req.http.host ~ "^(?i)example.com$") {
>         set req.http.x-redir = "https://" + req.http.host + req.url;
>         return(synth(301));
>     }
>
>
> for every domain or is there a way to make a force http to https rule for
> all the domains?
>
> 2018-04-13 22:57 GMT+01:00 Guillaume Quintard <
> guillaume at varnish-software.com>:
>
>> What is your goal, what don't you understand?
>>
>> --
>> Guillaume Quintard
>>
>> On Fri, Apr 13, 2018, 23:52 Daniel Santos <dannysantos1985 at gmail.com>
>> wrote:
>>
>>> I don't understand the this FAQ topic
>>> <https://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-clients-to-https>
>>> .
>>>
>>> I want to implement it. In my /etc/varnish/default.vcl I have this:
>>>
>>> backend apache {
>>>     .host = "127.0.0.1";
>>>     .port = "5555";
>>> }
>>>
>>> backend node_hello {
>>>     .host = "127.0.0.1";
>>>     .port = "5556";
>>> }
>>>
>>> import std;
>>>
>>> sub vcl_recv {
>>>     # Happens before we check if we have this in cache already.
>>>     #
>>>     # Typically you clean up the request here, removing cookies you don't need,
>>>     # rewriting the request, etc.
>>>
>>>     if ((std.port(local.ip) == 6086) && (std.port(server.ip) == 443)) {
>>>         set req.http.X-Forwarded-Proto = "https";
>>>     }
>>>
>>>     if(req.http.host ~ "domain1.com") {
>>>         set req.backend_hint = apache;
>>>     }
>>>     elsif(req.http.host == "domain2.com") {
>>>         set req.backend_hint = node_hello;
>>>     }
>>> }
>>>
>>> How should my implementation become?
>>>
>>>
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc at varnish-cache.org
>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180414/7068b5e6/attachment-0001.html>

From jpotter-varnish at codepuppy.com  Sat Apr 14 14:06:26 2018
From: jpotter-varnish at codepuppy.com (jpotter-varnish at codepuppy.com)
Date: Sat, 14 Apr 2018 10:06:26 -0400
Subject: Compare Hex to Decimal
In-Reply-To: <DB5PR08MB0648AABB77F848DFE6D8E138C9BD0@DB5PR08MB0648.eurprd08.prod.outlook.com>
References: <DB5PR08MB0648AABB77F848DFE6D8E138C9BD0@DB5PR08MB0648.eurprd08.prod.outlook.com>
Message-ID: <56EFAA5A-CE63-44C4-92D8-DB50DF362866@codepuppy.com>


Hi Michael,

I did something similar to this several years back using inline C:
	https://varnish-cache.org/docs/5.1/users-guide/vcl-inline-c.html <https://varnish-cache.org/docs/5.1/users-guide/vcl-inline-c.html>

I?m super-rusty on my C, so ask better minds, but I?d start by doing the if check in C and leaving the rest of the logic in VCL. I.e.
strtol() with a base of 16 (see http://pubs.opengroup.org/onlinepubs/7908799/xsh/strtol.html <http://pubs.opengroup.org/onlinepubs/7908799/xsh/strtol.html>) and then a direct check of that against gmtime() - delta (hopefully your unix hex timestamps in UTC?). Double-check the input to strtol is safe ? this approach has untrusted inputs getting run through into C.

-Jeff

> On Apr 13, 2018, at 11:42 AM, Michael Kennedy <michael at kennedy.ie> wrote:
> 
> Hi,
> 
> I have a URL that contains a Unix hex timestamp (something like 5ace8c5d). The timestamp should be within X minutes of now, and is used as part of an access control mechanism. I want to do something like this:
> 
> if (std.time(req.http.x-hextime, now) < now - 1w) {
> ...
> }
> 
> Is there an efficient way in VCL to either convert Hex to Dec so that this comparison can be done, or some other solution that does not involve changing the data in the URL.
> 
> Thanks,
> 
> Michael
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org <mailto:varnish-misc at varnish-cache.org>
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc <https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180414/4c1d75b3/attachment.html>

From dannysantos1985 at gmail.com  Sat Apr 14 17:53:52 2018
From: dannysantos1985 at gmail.com (Daniel Santos)
Date: Sat, 14 Apr 2018 17:53:52 +0000
Subject: Dont understand how to implement this FAQ topic
In-Reply-To: <CAJ6ZYQycaJGDBBfv73LNaxJB5mGUEZyfWhTW-aoKLM5N8nfguQ@mail.gmail.com>
References: <CAL6VdkQ81HqiMD3RY15oBDm0HFbbZbykoxw9bv6VM1OG7Z1ngQ@mail.gmail.com>
 <CAL6VdkS1yZzrhTEowW0v4MPbWfUQtKFdJjQ1yOMSJSXCe-jrRw@mail.gmail.com>
 <CAJ6ZYQydwcNR9nmcKWgifXtJgVcGrdt3sgeKRHbXRJdBWfp=bg@mail.gmail.com>
 <CAL6VdkTizckBkVk+3hTPTgo1HtTJUWZvQh25hLVUY8LTOzENrw@mail.gmail.com>
 <CAJ6ZYQycaJGDBBfv73LNaxJB5mGUEZyfWhTW-aoKLM5N8nfguQ@mail.gmail.com>
Message-ID: <CAL6VdkR_VQMHgUa1mf9WMRYyhMwzpck0bhzwHWMws4Pv0zgUfA@mail.gmail.com>

Thanks a lot for sharing.

A s?bado, 14/04/2018, 07:17, Guillaume Quintard <
guillaume at varnish-software.com> escreveu:

> Please keep the mail-in in the loop so others can benefit from this
> discussion.
>
> Have a look at this:
> https://info.varnish-software.com/blog/rewriting-urls-with-varnish-redirection
>
> It covers exactly your use-case, I think.
>
> --
> Guillaume Quintard
>
>
> On Sat, Apr 14, 2018, 00:01 Daniel Santos <dannysantos1985 at gmail.com>
> wrote:
>
>> I don't understand where I will put the set req.backend_hint on
>> https://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-cliehttps://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-clients-to-httpsnts-to-https
>> <https://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-clients-to-https>
>> .
>> And if in order to have http redirect to https will I have to put:
>>
>> if (std.port(local.ip) == 80 && req.http.host ~ "^(?i)example.com$") {
>>         set req.http.x-redir = "https://" + req.http.host + req.url;
>>         return(synth(301));
>>     }
>>
>>
>> for every domain or is there a way to make a force http to https rule for
>> all the domains?
>>
>> 2018-04-13 22:57 GMT+01:00 Guillaume Quintard <
>> guillaume at varnish-software.com>:
>>
>>> What is your goal, what don't you understand?
>>>
>>> --
>>> Guillaume Quintard
>>>
>>> On Fri, Apr 13, 2018, 23:52 Daniel Santos <dannysantos1985 at gmail.com>
>>> wrote:
>>>
>>>> I don't understand the this FAQ topic
>>>> <https://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-clients-to-https>
>>>> .
>>>>
>>>> I want to implement it. In my /etc/varnish/default.vcl I have this:
>>>>
>>>> backend apache {
>>>>     .host = "127.0.0.1";
>>>>     .port = "5555";
>>>> }
>>>>
>>>> backend node_hello {
>>>>     .host = "127.0.0.1";
>>>>     .port = "5556";
>>>> }
>>>>
>>>> import std;
>>>>
>>>> sub vcl_recv {
>>>>     # Happens before we check if we have this in cache already.
>>>>     #
>>>>     # Typically you clean up the request here, removing cookies you don't need,
>>>>     # rewriting the request, etc.
>>>>
>>>>     if ((std.port(local.ip) == 6086) && (std.port(server.ip) == 443)) {
>>>>         set req.http.X-Forwarded-Proto = "https";
>>>>     }
>>>>
>>>>     if(req.http.host ~ "domain1.com") {
>>>>         set req.backend_hint = apache;
>>>>     }
>>>>     elsif(req.http.host == "domain2.com") {
>>>>         set req.backend_hint = node_hello;
>>>>     }
>>>> }
>>>>
>>>> How should my implementation become?
>>>>
>>>>
>>>> _______________________________________________
>>>> varnish-misc mailing list
>>>> varnish-misc at varnish-cache.org
>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180414/a08a03bb/attachment.html>

From guillaume at varnish-software.com  Sat Apr 14 17:55:05 2018
From: guillaume at varnish-software.com (Guillaume Quintard)
Date: Sat, 14 Apr 2018 17:55:05 +0000
Subject: Dont understand how to implement this FAQ topic
In-Reply-To: <CAL6VdkR_VQMHgUa1mf9WMRYyhMwzpck0bhzwHWMws4Pv0zgUfA@mail.gmail.com>
References: <CAL6VdkQ81HqiMD3RY15oBDm0HFbbZbykoxw9bv6VM1OG7Z1ngQ@mail.gmail.com>
 <CAL6VdkS1yZzrhTEowW0v4MPbWfUQtKFdJjQ1yOMSJSXCe-jrRw@mail.gmail.com>
 <CAJ6ZYQydwcNR9nmcKWgifXtJgVcGrdt3sgeKRHbXRJdBWfp=bg@mail.gmail.com>
 <CAL6VdkTizckBkVk+3hTPTgo1HtTJUWZvQh25hLVUY8LTOzENrw@mail.gmail.com>
 <CAJ6ZYQycaJGDBBfv73LNaxJB5mGUEZyfWhTW-aoKLM5N8nfguQ@mail.gmail.com>
 <CAL6VdkR_VQMHgUa1mf9WMRYyhMwzpck0bhzwHWMws4Pv0zgUfA@mail.gmail.com>
Message-ID: <CAJ6ZYQy-RyS8m=5a0C-RYcYE4k63RaBtH_pX3xA2k7K51PrDag@mail.gmail.com>

No problem, did you manage to solve your issue?

-- 
Guillaume Quintard

On Sat, Apr 14, 2018, 19:54 Daniel Santos <dannysantos1985 at gmail.com> wrote:

> Thanks a lot for sharing.
>
> A s?bado, 14/04/2018, 07:17, Guillaume Quintard <
> guillaume at varnish-software.com> escreveu:
>
>> Please keep the mail-in in the loop so others can benefit from this
>> discussion.
>>
>> Have a look at this:
>> https://info.varnish-software.com/blog/rewriting-urls-with-varnish-redirection
>>
>> It covers exactly your use-case, I think.
>>
>> --
>> Guillaume Quintard
>>
>>
>> On Sat, Apr 14, 2018, 00:01 Daniel Santos <dannysantos1985 at gmail.com>
>> wrote:
>>
>>> I don't understand where I will put the set req.backend_hint on
>>> https://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-cliehttps://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-clients-to-httpsnts-to-https
>>> <https://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-clients-to-https>
>>> .
>>> And if in order to have http redirect to https will I have to put:
>>>
>>> if (std.port(local.ip) == 80 && req.http.host ~ "^(?i)example.com$") {
>>>         set req.http.x-redir = "https://" + req.http.host + req.url;
>>>         return(synth(301));
>>>     }
>>>
>>>
>>> for every domain or is there a way to make a force http to https rule
>>> for all the domains?
>>>
>>> 2018-04-13 22:57 GMT+01:00 Guillaume Quintard <
>>> guillaume at varnish-software.com>:
>>>
>>>> What is your goal, what don't you understand?
>>>>
>>>> --
>>>> Guillaume Quintard
>>>>
>>>> On Fri, Apr 13, 2018, 23:52 Daniel Santos <dannysantos1985 at gmail.com>
>>>> wrote:
>>>>
>>>>> I don't understand the this FAQ topic
>>>>> <https://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-clients-to-https>
>>>>> .
>>>>>
>>>>> I want to implement it. In my /etc/varnish/default.vcl I have this:
>>>>>
>>>>> backend apache {
>>>>>     .host = "127.0.0.1";
>>>>>     .port = "5555";
>>>>> }
>>>>>
>>>>> backend node_hello {
>>>>>     .host = "127.0.0.1";
>>>>>     .port = "5556";
>>>>> }
>>>>>
>>>>> import std;
>>>>>
>>>>> sub vcl_recv {
>>>>>     # Happens before we check if we have this in cache already.
>>>>>     #
>>>>>     # Typically you clean up the request here, removing cookies you don't need,
>>>>>     # rewriting the request, etc.
>>>>>
>>>>>     if ((std.port(local.ip) == 6086) && (std.port(server.ip) == 443)) {
>>>>>         set req.http.X-Forwarded-Proto = "https";
>>>>>     }
>>>>>
>>>>>     if(req.http.host ~ "domain1.com") {
>>>>>         set req.backend_hint = apache;
>>>>>     }
>>>>>     elsif(req.http.host == "domain2.com") {
>>>>>         set req.backend_hint = node_hello;
>>>>>     }
>>>>> }
>>>>>
>>>>> How should my implementation become?
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> varnish-misc mailing list
>>>>> varnish-misc at varnish-cache.org
>>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>>
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180414/7aef46c4/attachment-0001.html>

From dannysantos1985 at gmail.com  Sat Apr 14 18:03:58 2018
From: dannysantos1985 at gmail.com (Daniel Santos)
Date: Sat, 14 Apr 2018 18:03:58 +0000
Subject: Dont understand how to implement this FAQ topic
In-Reply-To: <CAJ6ZYQy-RyS8m=5a0C-RYcYE4k63RaBtH_pX3xA2k7K51PrDag@mail.gmail.com>
References: <CAL6VdkQ81HqiMD3RY15oBDm0HFbbZbykoxw9bv6VM1OG7Z1ngQ@mail.gmail.com>
 <CAL6VdkS1yZzrhTEowW0v4MPbWfUQtKFdJjQ1yOMSJSXCe-jrRw@mail.gmail.com>
 <CAJ6ZYQydwcNR9nmcKWgifXtJgVcGrdt3sgeKRHbXRJdBWfp=bg@mail.gmail.com>
 <CAL6VdkTizckBkVk+3hTPTgo1HtTJUWZvQh25hLVUY8LTOzENrw@mail.gmail.com>
 <CAJ6ZYQycaJGDBBfv73LNaxJB5mGUEZyfWhTW-aoKLM5N8nfguQ@mail.gmail.com>
 <CAL6VdkR_VQMHgUa1mf9WMRYyhMwzpck0bhzwHWMws4Pv0zgUfA@mail.gmail.com>
 <CAJ6ZYQy-RyS8m=5a0C-RYcYE4k63RaBtH_pX3xA2k7K51PrDag@mail.gmail.com>
Message-ID: <CAL6VdkQDFumgHH-n=e7GzjCYtH5CgON5sVF43e0BsWOydkn_+g@mail.gmail.com>

Yes, I did.

A s?bado, 14/04/2018, 18:55, Guillaume Quintard <
guillaume at varnish-software.com> escreveu:

> No problem, did you manage to solve your issue?
>
> --
> Guillaume Quintard
>
> On Sat, Apr 14, 2018, 19:54 Daniel Santos <dannysantos1985 at gmail.com>
> wrote:
>
>> Thanks a lot for sharing.
>>
>> A s?bado, 14/04/2018, 07:17, Guillaume Quintard <
>> guillaume at varnish-software.com> escreveu:
>>
>>> Please keep the mail-in in the loop so others can benefit from this
>>> discussion.
>>>
>>> Have a look at this:
>>> https://info.varnish-software.com/blog/rewriting-urls-with-varnish-redirection
>>>
>>> It covers exactly your use-case, I think.
>>>
>>> --
>>> Guillaume Quintard
>>>
>>>
>>> On Sat, Apr 14, 2018, 00:01 Daniel Santos <dannysantos1985 at gmail.com>
>>> wrote:
>>>
>>>> I don't understand where I will put the set req.backend_hint on
>>>> https://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-cliehttps://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-clients-to-httpsnts-to-https
>>>> <https://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-clients-to-https>
>>>> .
>>>> And if in order to have http redirect to https will I have to put:
>>>>
>>>> if (std.port(local.ip) == 80 && req.http.host ~ "^(?i)example.com$") {
>>>>         set req.http.x-redir = "https://" + req.http.host + req.url;
>>>>         return(synth(301));
>>>>     }
>>>>
>>>>
>>>> for every domain or is there a way to make a force http to https rule
>>>> for all the domains?
>>>>
>>>> 2018-04-13 22:57 GMT+01:00 Guillaume Quintard <
>>>> guillaume at varnish-software.com>:
>>>>
>>>>> What is your goal, what don't you understand?
>>>>>
>>>>> --
>>>>> Guillaume Quintard
>>>>>
>>>>> On Fri, Apr 13, 2018, 23:52 Daniel Santos <dannysantos1985 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I don't understand the this FAQ topic
>>>>>> <https://github.com/varnish/hitch/wiki/FAQ#redirect-varnish-http-clients-to-https>
>>>>>> .
>>>>>>
>>>>>> I want to implement it. In my /etc/varnish/default.vcl I have this:
>>>>>>
>>>>>> backend apache {
>>>>>>     .host = "127.0.0.1";
>>>>>>     .port = "5555";
>>>>>> }
>>>>>>
>>>>>> backend node_hello {
>>>>>>     .host = "127.0.0.1";
>>>>>>     .port = "5556";
>>>>>> }
>>>>>>
>>>>>> import std;
>>>>>>
>>>>>> sub vcl_recv {
>>>>>>     # Happens before we check if we have this in cache already.
>>>>>>     #
>>>>>>     # Typically you clean up the request here, removing cookies you don't need,
>>>>>>     # rewriting the request, etc.
>>>>>>
>>>>>>     if ((std.port(local.ip) == 6086) && (std.port(server.ip) == 443)) {
>>>>>>         set req.http.X-Forwarded-Proto = "https";
>>>>>>     }
>>>>>>
>>>>>>     if(req.http.host ~ "domain1.com") {
>>>>>>         set req.backend_hint = apache;
>>>>>>     }
>>>>>>     elsif(req.http.host == "domain2.com") {
>>>>>>         set req.backend_hint = node_hello;
>>>>>>     }
>>>>>> }
>>>>>>
>>>>>> How should my implementation become?
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> varnish-misc mailing list
>>>>>> varnish-misc at varnish-cache.org
>>>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>>>
>>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180414/1241ff5e/attachment.html>

From pinakee at waltzz.com  Fri Apr 27 14:57:01 2018
From: pinakee at waltzz.com (Pinakee BIswas)
Date: Fri, 27 Apr 2018 20:27:01 +0530
Subject: Set cookie creating issues
Message-ID: <3b74b839-8ff5-f5ea-23dc-41d6fd5113bc@waltzz.com>

Hi,

We have been using Varnish for caching our web pages. We have an 
ecommerce site. Things have been working fine till today but suddenly 
things have started breaking down and I am not sure why. Following is 
the issue:

We use session cookie to store user sessions. The session cookie is 
getting changed as Cached responses from varnish is having set-cookie 
header which is messing up the session cookie. We are using varnish 4.8. 
Following is a snippet of the VCL:


sub vcl_recv {
     # Happens before we check if we have this in cache already.
     #
     # Typically you clean up the request here, removing cookies you 
don't need,
     # rewriting the request, etc.
     set req.backend_hint = uwsgi;

     #if (req.http.cookie ~ "jivaana_country=") {
         # Set the country header
#       set req.http.X-CLIENT-COUNTRY = regsub(req.http.cookie, 
".*jivaana_country=([^;]+);.*", "\1");
#    }

    std.log("ga:" + ga.extract(req.url, mode = keep));
    set req.url = ga.apply(req.url); #  remove Google Analytics parameters

    if (req.method == "GET") {
         if ((req.url !~ "^/accounts/userheader") &&
             (req.url !~ "^/accounts/new-userheader") &&
             (req.url !~ "^/product/recently-viewed") &&
             (req.url !~ "^/product/recommended-products") &&
             (req.url !~ "^/product/addtobasket")) {
                 unset req.http.cookie; # strip the cookies - we don't 
need them
         }
     }

     call devicedetect;
}

sub vcl_backend_response {
     # Happens after we have read the response headers from the backend.
     #
     # Here you clean the response headers, removing silly Set-Cookie 
headers
     # and other mistakes your backend does.
     if (bereq.method == "GET") {
         set beresp.do_esi = true;
         if ((bereq.url !~ "^/accounts/userheader") &&
             (bereq.url !~ "^/accounts/new-userheader") &&
             (bereq.url !~ "^/product/recently-viewed") &&
             (bereq.url !~ "^/product/recommended-products") &&
             (bereq.url !~ "^/product/addtobasket")) {
                 #unset beresp.http.Set-Cookie;
                 set beresp.uncacheable = false;
                 #std.log("Caching the url : **********************" + 
bereq.url);
    }
}

sub vcl_deliver {
     # Happens when we have all the pieces we need, and are about to 
send the
     # response to the client.
     #
     # You can do accounting or modifying the final object here.
}

sub vcl_hash {
}

Would really appreciate any support as this is messing up our user sessions.

Thanks,

Pinakee


From Deepak.Verma at comtechtel.com  Mon Apr 30 20:38:43 2018
From: Deepak.Verma at comtechtel.com (Deepak Verma)
Date: Mon, 30 Apr 2018 20:38:43 +0000
Subject: Query regarding Varnish modules for parsing XML body
Message-ID: <BY1PR09MB0822CF692490ED173F8E7967EE820@BY1PR09MB0822.namprd09.prod.outlook.com>

Hi,

I am trying to see if Varnish as a reverse proxy cache is suitable for my use case. Are there any Varnish modules available which can help me parse XML data present in POST response body.

Regards,
Deepak
NOTICE TO RECIPIENT: This email, including attachments, may contain information which is confidential, proprietary, attorney-client privileged and / or controlled under U.S. export laws and regulations and may be restricted from disclosure by applicable State and Federal law. Nothing in this email shall create any legal binding agreement between the parties unless expressly stated herein and provided by an authorized representative of Comtech Telecommunications Corp. or its subsidiaries. If you are not the intended recipient of this message, be advised that any dissemination, distribution, or use of the contents of this message is strictly prohibited. If you received this message in error, please notify us immediately by return email and permanently delete all copies of the original email and any attached documentation from any computer or other media.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180430/245bdfb8/attachment.html>