[Varnish 4] Respecting client's Cache-Control: max-age= as TTL
Martynas Jusevičius
martynas at atomgraph.com
Tue Aug 1 19:17:15 CEST 2017
Guillaume,
after I fixed a couple of typos, this *seems* to work now:
sub vcl_recv {
...
if (req.restarts == 0) {
unset req.http.force_miss;
if (req.http.x-forwarded-for) {
set req.http.X-Forwarded-For =
req.http.X-Forwarded-For + ", " + client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
}
else if (req.http.force_miss) {
set req.hash_always_miss = true;
}
...
sub vcl_hit {
if (req.http.Cache-Control ~ "max-age=[0-9]*") {
set req.http.Max-Age = regsub(req.http.Cache-Control,
"max-age=([0-9]*)", "\1");
if (obj.age > std.duration(req.http.Max-Age + "s", 1000000s)) {
set req.http.force_miss = "yes";
return(restart);
}
}
...
After return(restart) in vcl_hit I can see a new BeReq which calls
BACKEND_FETCH, new BeResp which calls BACKEND_RESPONSE, and then restarted
Request which triggers hash_always_miss and then calls RECV, HASH, MISS,
and finally DELIVER.
If that looks right, it would be nice to have it documented somewhere for
future reference.
Thanks for your help.
On Tue, Aug 1, 2017 at 9:34 AM, Guillaume Quintard <
guillaume at varnish-software.com> wrote:
> Ah, right, I totally forgot about that, sorry.
>
> Sooooooo, there's no real clean way to do it (that I can see, smarter
> people than me may have a solution), but here's what I can offer.
>
> sub vcl_recv {
> if (req.restarts == 0) {
> unset req.http.force_miss;
> } else if (req.http.force_miss) {
> set req.hash_alway_miss
> }
> }
>
> sub vcl_hit {
> if (CONDITION TO BYPASS CACHE) {
> set req.http.force_miss = "yes";
> return(restart);
> }
> }
>
>
>
> --
> Guillaume Quintard
>
> On Tue, Aug 1, 2017 at 12:10 AM, Martynas Jusevičius <
> martynas at atomgraph.com> wrote:
>
>> Sorry, sent too soon. Here it goes:
>>
>> Thanks Guillaume.
>>
>> First I tried return(fetch) as you suggested
>>
>> sub vcl_hit {
>> if (req.http.Cache-Control ~ "max-age=[0-9]*") {
>> set req.http.Max-Age = regsub(req.http.Cache-Control,
>> "max-age=([0-9]*)", "\1");
>> if (obj.age > std.duration(req.http.Max-Age + "s", 1000000s)) {
>> std.log("obj.age: " + obj.age + " req.http.Max-Age: " +
>> req.http.Max-Age);
>> return(fetch);
>> }
>> }
>> ...
>>
>> but I got an error:
>>
>> - VCL_call HIT
>> - ReqHeader Max-Age: 69
>> - VCL_Log obj.age: 102.306 req.http.Max-Age: 69
>> - VCL_return fetch
>> - VCL_Error change return(fetch) to return(miss) in vcl_hit{}
>> - VCL_Error vcl_hit{} returns miss without busy object. Doing
>> pass.
>> - VCL_call PASS
>> - VCL_return fetch
>>
>> I did as told and I tried return(miss)
>>
>> sub vcl_hit {
>> if (req.http.Cache-Control ~ "max-age=[0-9]*") {
>> set req.http.Max-Age = regsub(req.http.Cache-Control,
>> "max-age=([0-9]*)", "\1");
>> if (obj.age > std.duration(req.http.Max-Age + "s", 1000000s)) {
>> std.log("obj.age: " + obj.age + " req.http.Max-Age: " +
>> req.http.Max-Age);
>> return(miss);
>> }
>> }
>> ...
>>
>> but then I got another error:
>>
>> - VCL_call HIT
>> - ReqHeader Max-Age: 69
>> - VCL_Log obj.age: 195.391 req.http.Max-Age: 69
>> - VCL_return miss
>> - VCL_Error vcl_hit{} returns miss without busy object. Doing
>> pass.
>> - VCL_call PASS
>> - VCL_return fetch
>>
>> So it looks like the max-age logic is triggered correctly, but what is
>> wrong with the return values?
>>
>> On Tue, Aug 1, 2017 at 12:01 AM, Martynas Jusevičius <
>> martynas at atomgraph.com> wrote:
>>
>>> Thanks Guillaume.
>>>
>>> First I tried
>>>
>>> sub vcl_hit {
>>> if (req.http.Cache-Control ~ "max-age=[0-9]*") {
>>> set req.http.Max-Age = regsub(req.http.Cache-Control,
>>> "max-age=([0-9]*)", "\1");
>>> if (obj.age > std.duration(req.http.Max-Age + "s", 1000000s)) {
>>> std.log("obj.age: " + obj.age + " req.http.Max-Age: " +
>>> req.http.Max-Age);
>>> return(fetch);
>>> }
>>> }
>>> ...
>>>
>>> but I got an error:
>>>
>>> - VCL_call HIT
>>> - ReqHeader Max-Age: 69
>>> - VCL_Log obj.age: 102.306 req.http.Max-Age: 69
>>> - VCL_return fetch
>>> - VCL_Error change return(fetch) to return(miss) in vcl_hit{}
>>> - VCL_Error vcl_hit{} returns miss without busy object. Doing
>>> pass.
>>> - VCL_call PASS
>>> - VCL_return fetch
>>>
>>> I did as told and I tried
>>>
>>>
>>> On Mon, Jul 31, 2017 at 9:11 PM, Guillaume Quintard <
>>> guillaume at varnish-software.com> wrote:
>>>
>>>> man vcl
>>>>
>>>> bereq is filtered to avoid side effects of the client forcing the ttl
>>>> to the backed.
>>>>
>>>> Anyway, by the time you have access to bereq, it's too late for you
>>>> since the decision to go to the backend has already been been made.
>>>>
>>>> --
>>>> Guillaume Quintard
>>>>
>>>>
>>>> On Jul 31, 2017 19:56, "Martynas Jusevičius" <martynas at atomgraph.com>
>>>> wrote:
>>>>
>>>> Thanks. What was mostly unclear to me is passing the req header value
>>>> all the way to where it's used to set TTL.
>>>>
>>>> Why doesn't bereq contain the req headers? At least Cache-Control is
>>>> gone.
>>>>
>>>> But I guess that can be done using obj.ttl, which I didn't know about.
>>>> Any documentation on that?
>>>>
>>>> On Mon, 31 Jul 2017 at 18.38, Guillaume Quintard <
>>>> guillaume at varnish-software.com> wrote:
>>>>
>>>>> On github I pointed to the doc explaining how you can return(fetch) to
>>>>> ignore a cached object, possibly based on ttl, so you already have half the
>>>>> answer.
>>>>>
>>>>> The other part of the equation is just converting
>>>>> req.http.cache-control to a duration and comparing that to obj.ttl. It will
>>>>> be similar to what you have done on v3.
>>>>>
>>>>> --
>>>>> Guillaume Quintard
>>>>>
>>>>> On Jul 31, 2017 18:25, "Martynas Jusevičius" <martynas at atomgraph.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I have been reading quite a bit about Varnish and VCL but found
>>>>>> almost no examples with Cache-Control coming from the client request [1].
>>>>>>
>>>>>> What I want to achieve: if the client sends Cache-Control:
>>>>>> max-age=60, TTL becomes 60 s. If the cache hit is fresher than 60 s,
>>>>>> deliver it, otherwise fetch a new response from backend (I hope I'm not
>>>>>> misusing the VCL terms here) *and* cache it.
>>>>>>
>>>>>> I had hacked this together in the vcl_fetch section in Varnish 3.x by
>>>>>> setting the req.http.Cache-Control max-age value as beresp.ttl, but
>>>>>> vcl_fetch is gone in Varnish 4.x.
>>>>>>
>>>>>> I have received a suggestion to use vcl_hit and/or grace [2], but
>>>>>> again -- no examples...
>>>>>>
>>>>>> Could anyone provide some VCL pseudo-code that
>>>>>> uses req.http.Cache-Control value to override TTL? max-age number parsing
>>>>>> not necessary, I have figure that out.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Martynas
>>>>>>
>>>>>> [1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Header
>>>>>> s/Cache-Control#Cache_request_directives
>>>>>> [2] https://github.com/varnishcache/varnish-cache/issues/201
>>>>>> 4#issuecomment-319096566
>>>>>>
>>>>>> _______________________________________________
>>>>>> varnish-misc mailing list
>>>>>> varnish-misc at varnish-cache.org
>>>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20170801/bda565b2/attachment.html>
More information about the varnish-misc
mailing list