varnish fetching, stroring and serving cached 404 from backend server
mloftis at wgops.com
Wed Aug 18 19:49:50 CEST 2010
--On Wednesday, August 18, 2010 12:50 AM -0700 David Birdsong
<david.birdsong at gmail.com> wrote:
> On Tue, Aug 17, 2010 at 7:59 PM, Michael Loftis <mloftis at wgops.com> wrote:
>> Varnish caching, or not caching, is completely controlled by the VCL.
>> Not having yours noone can help you. However, the *default* VCL
>> makes anything with a Cookie or HTTP Authentication header from the
>> client uncacheable.
> Fair enough, I should have known vcl would be helpful, *but I didn't
> think that statement was entirely true.
> "completely controlled by the VCL"...I thought there were boundaries
> that varnish operated in and no amount of vcl magic would override
> it...such as beresp.cacheable, if that boolean evaluated to false, I
> thought that varnish will never cache. Is that incorrect?
<http://www.gossamer-threads.com/lists/varnish/misc/14286> i think that
thread should answer your ?'s on beresp.cacheable, in short, it is not
authoritative. Further more 404's are perfectly cacheable.
You're caching all requests for a big list of extensions on line 91...
I'm not sure about case sensitivity in the req.http.cookie statement but
since you're passing everything to lookup anyway in the end you *should* be
caching 404s. I'm actually reading this and trying to figure out why a 404
*wouldn't* cache. In fact the only places I don't see you caching are if
it has an auth header, a cookie AND that cookie looks like one of your auth
cookies. That's the only reason I could see your code NOT caching.
Expect, Auth headers, and a cookie that looks like your auth cookie.
So I'm guessing that once in a while a miss is resulting in a 404 which
then (appropriately) gets cached. If you don't want 404's getting cached
set their TTL to 0 in vcl_fetch. Just remember that'll leave your backends
open to that sort of mischief.
It looks like tour vcl_fetch does do a fall through to the default
vcl_fetch, so unless there are overriding expires or cache-control headers
coming back on 404's, they'll be cached. If you don't want them cached you
need to pass on them in vcl_fetch.
> Anyway, here's the vcl:
More information about the varnish-misc