[Varnish 4] Respecting client's Cache-Control: max-age= as TTL

[Guillaume Quintard]

Ah, right, I totally forgot about that, sorry.

Sooooooo, there's no real clean way to do it (that I can see, smarter
people than me may have a solution), but here's what I can offer.

sub vcl_recv {
    if (req.restarts == 0) {
        unset req.http.force_miss;
    } else if (req.http.force_miss) {
        set req.hash_alway_miss
    }
}

sub vcl_hit {
    if (CONDITION TO BYPASS CACHE) {
        set req.http.force_miss = "yes";
        return(restart);
    }
}



-- 
Guillaume Quintard

On Tue, Aug 1, 2017 at 12:10 AM, Martynas Jusevičius <martynas at atomgraph.com
> wrote:

> Sorry, sent too soon. Here it goes:
>
> Thanks Guillaume.
>
> First I tried return(fetch) as you suggested
>
> sub vcl_hit {
>     if (req.http.Cache-Control ~ "max-age=[0-9]*") {
>         set req.http.Max-Age = regsub(req.http.Cache-Control,
> "max-age=([0-9]*)", "\1");
>         if (obj.age > std.duration(req.http.Max-Age + "s", 1000000s)) {
>             std.log("obj.age: " + obj.age + " req.http.Max-Age: " +
> req.http.Max-Age);
>             return(fetch);
>         }
>     }
>     ...
>
> but I got an error:
>
> -   VCL_call       HIT
> -   ReqHeader      Max-Age: 69
> -   VCL_Log        obj.age: 102.306 req.http.Max-Age: 69
> -   VCL_return     fetch
> -   VCL_Error      change return(fetch) to return(miss) in vcl_hit{}
> -   VCL_Error      vcl_hit{} returns miss without busy object.  Doing pass.
> -   VCL_call       PASS
> -   VCL_return     fetch
>
> I did as told and I tried return(miss)
>
> sub vcl_hit {
>     if (req.http.Cache-Control ~ "max-age=[0-9]*") {
>         set req.http.Max-Age = regsub(req.http.Cache-Control,
> "max-age=([0-9]*)", "\1");
>         if (obj.age > std.duration(req.http.Max-Age + "s", 1000000s)) {
>             std.log("obj.age: " + obj.age + " req.http.Max-Age: " +
> req.http.Max-Age);
>             return(miss);
>         }
>     }
>     ...
>
> but then I got another error:
>
> -   VCL_call       HIT
> -   ReqHeader      Max-Age: 69
> -   VCL_Log        obj.age: 195.391 req.http.Max-Age: 69
> -   VCL_return     miss
> -   VCL_Error      vcl_hit{} returns miss without busy object.  Doing pass.
> -   VCL_call       PASS
> -   VCL_return     fetch
>
> So it looks like the max-age logic is triggered correctly, but what is
> wrong with the return values?
>
> On Tue, Aug 1, 2017 at 12:01 AM, Martynas Jusevičius <
> martynas at atomgraph.com> wrote:
>
>> Thanks Guillaume.
>>
>> First I tried
>>
>> sub vcl_hit {
>>     if (req.http.Cache-Control ~ "max-age=[0-9]*") {
>>         set req.http.Max-Age = regsub(req.http.Cache-Control,
>> "max-age=([0-9]*)", "\1");
>>         if (obj.age > std.duration(req.http.Max-Age + "s", 1000000s)) {
>>             std.log("obj.age: " + obj.age + " req.http.Max-Age: " +
>> req.http.Max-Age);
>>             return(fetch);
>>         }
>>     }
>>     ...
>>
>> but I got an error:
>>
>> -   VCL_call       HIT
>> -   ReqHeader      Max-Age: 69
>> -   VCL_Log        obj.age: 102.306 req.http.Max-Age: 69
>> -   VCL_return     fetch
>> -   VCL_Error      change return(fetch) to return(miss) in vcl_hit{}
>> -   VCL_Error      vcl_hit{} returns miss without busy object.  Doing
>> pass.
>> -   VCL_call       PASS
>> -   VCL_return     fetch
>>
>> I did as told and I tried
>>
>>
>> On Mon, Jul 31, 2017 at 9:11 PM, Guillaume Quintard <
>> guillaume at varnish-software.com> wrote:
>>
>>> man vcl
>>>
>>> bereq is filtered to avoid side effects of the client forcing the ttl to
>>> the backed.
>>>
>>> Anyway, by the time you have access to bereq, it's too late for you
>>> since the decision to go to the backend has already been been made.
>>>
>>> --
>>> Guillaume Quintard
>>>
>>>
>>> On Jul 31, 2017 19:56, "Martynas Jusevičius" <martynas at atomgraph.com>
>>> wrote:
>>>
>>> Thanks. What was mostly unclear to me is passing the req header value
>>> all the way to where it's used to set TTL.
>>>
>>> Why doesn't bereq contain the req headers? At least Cache-Control is
>>> gone.
>>>
>>> But I guess that can be done using obj.ttl, which I didn't know about.
>>> Any documentation on that?
>>>
>>> On Mon, 31 Jul 2017 at 18.38, Guillaume Quintard <
>>> guillaume at varnish-software.com> wrote:
>>>
>>>> On github I pointed to the doc explaining how you can return(fetch) to
>>>> ignore a cached object, possibly based on ttl, so you already have half the
>>>> answer.
>>>>
>>>> The other part of the equation is just converting
>>>> req.http.cache-control to a duration and comparing that to obj.ttl. It will
>>>> be similar to what you have done on v3.
>>>>
>>>> --
>>>> Guillaume Quintard
>>>>
>>>> On Jul 31, 2017 18:25, "Martynas Jusevičius" <martynas at atomgraph.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I have been reading quite a bit about Varnish and VCL but found almost
>>>>> no examples with Cache-Control coming from the client request [1].
>>>>>
>>>>> What I want to achieve: if the client sends Cache-Control: max-age=60,
>>>>> TTL becomes 60 s. If the cache hit is fresher than 60 s, deliver it,
>>>>> otherwise fetch a new response from backend (I hope I'm not misusing the
>>>>> VCL terms here) *and* cache it.
>>>>>
>>>>> I had hacked this together in the vcl_fetch section in Varnish 3.x by
>>>>> setting the req.http.Cache-Control max-age value as beresp.ttl, but
>>>>> vcl_fetch is gone in Varnish 4.x.
>>>>>
>>>>> I have received a suggestion to use vcl_hit and/or grace [2], but
>>>>> again -- no examples...
>>>>>
>>>>> Could anyone provide some VCL pseudo-code that
>>>>> uses req.http.Cache-Control value to override TTL? max-age number parsing
>>>>> not necessary, I have figure that out.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Martynas
>>>>>
>>>>> [1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Header
>>>>> s/Cache-Control#Cache_request_directives
>>>>> [2] https://github.com/varnishcache/varnish-cache/issues/201
>>>>> 4#issuecomment-319096566
>>>>>
>>>>> _______________________________________________
>>>>> varnish-misc mailing list
>>>>> varnish-misc at varnish-cache.org
>>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20170801/1bf5b939/attachment.html>