Conditional requests for cached 404 responses

[Mark Slater]

Hi,

I'm running Varnish in front of a back end that has to do some work to
determine whether a request should receive a 404 response.  However, it can
cheaply determine whether a previous 404 is still valid.

I see Varnish issuing conditional requests for cached 200 responses, but I
haven't managed to achieve the same for cached 404 responses.

Here's my sample VCL:


vcl 4.1;

backend default {
    .host = "localhost";
    .port = "8081";
}

sub vcl_backend_response {
     set beresp.keep = 5m;
}


I'm testing with canned responses on port 8081  For the working 200 case, I
return:


HTTP/1.1 200 OK
cache-control: max-age=5
etag: "foo"
content-length: 13
connection: close

Hello, World!


When I make requests to Varnish, I see, as expected, a first request to the
back end, followed by five seconds of nothing to the back end, because
Varnish is responding with its cached copy, followed by a conditional
request to the back end:


GET / HTTP/1.1
Host: localhost:8080
User-Agent: curl/7.68.0
Accept: */*
X-Forwarded-For: 127.0.0.1
Accept-Encoding: gzip
X-Varnish: 3

GET / HTTP/1.1
Host: localhost:8080
User-Agent: curl/7.68.0
Accept: */*
X-Forwarded-For: 127.0.0.1
Accept-Encoding: gzip
If-None-Match: "foo"
X-Varnish: 32773


For the failing 404 case, my canned back end responds:


HTTP/1.1 404 Not Found
cache-control: max-age=5
etag: "foo"
content-length: 13
connection: close

Hello, World!


Now when I make requests to Varnish, I get a cached response for five
seconds as before, but when the response goes stale, rather than issuing a
conditional request to revalidate it, Varnish is issuing unconditional
requests:


GET / HTTP/1.1
Host: localhost:8080
User-Agent: curl/7.68.0
Accept: */*
X-Forwarded-For: 127.0.0.1
Accept-Encoding: gzip
X-Varnish: 3

GET / HTTP/1.1
Host: localhost:8080
User-Agent: curl/7.68.0
Accept: */*
X-Forwarded-For: 127.0.0.1
Accept-Encoding: gzip
X-Varnish: 32771


Is that something I can adjust with configuration?  If it's relevant, I'm
running:

Debug: Version: varnish-6.2.1 revision
9f8588e4ab785244e06c3446fe09bf9db5dd8753
Debug: Platform:
Linux,5.4.0-153-generic,x86_64,-jnone,-sdefault,-sdefault,-hcritbit

Incidentally, 200 responses with content-length 0 also seem to exhibit this
behaviour.

Thanks in advance,

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20230714/fbfd74d7/attachment.html>