Value of builtin.vcl - vcl_recv on modern websites

[Dridi Boukelmoune]

> ==> Maybe modern web site SHOULD NOT use cookies on every request!
> Because of the way cookies interfere with downstream caching.

Does it really matter? Cookies or not, they should rather *always*
include at least a Cache-Control header in responses.

> HTTP was conceived from the beginning to enable caching as a means of
> solving performance problems in slow networks. A well-configured

I must politely disagree here. HTTP/0.9 had no such thing, 1.0
introduced limited caching support and 1.1 got its act together but
ended up with broken semantics. Spoiler alert, covering this in the
part 2 draft.

>> A classic example of this is someone adding javascript via Google
>> Tag Manager which then sets a cookie.
>
> One might have hoped that the Googlers, of all people, would have more
> awareness of the trouble that they could cause by doing that.

I don't see this as a bad thing from a technical point of view. If you
need to carry state in requests, you need cookies.

> In a project where I am able to work with the app devs, I have had
> good experience with working out a policy with them: if you MUST have
> cookies in every request (although I WISH YOU WOULD RECONSIDER THAT),
> then the caching proxy cannot make caching decisions on your behalf.
> Only you can know if your response is cacheable, despite the presence
> of cookie foo or bar, but is not cacheable if the cookie is baz.

Here I disagree...

> So if you want your response to be cached, you MUST say so in a
> Cache-Control header. The proxy will not cache any other responses.

...but here I agree. If you need cookies, use cookies, but if you
serve responses in the first place make sure to let downstreams know
what to do with said response. For example it could be cacheable by
the client but not by proxies in-between.

> Then we write VCL to bypass builtin's vcl_recv, and start Varnish with
> - -t 0 (default TTL is 0s). Responses are then cached only if they
> announce that they are cacheable.

+1

Ideally Varnish shouldn't make decisions in the lack of information.
So default TTL and grace periods should ideally be zero and instead
rely on Cache-Control entries (max-age, s-maxage,
stale-while-revalidate...)

> That boils down to answering your question by saying no, you can't use
> builtin vcl_recv in a situation like that. When the cookies, like the
> Evil, are always and everywhere (to paraphrase a saying in Germany),
> and some cookies lead to cacheable responses while others don't, then
> there's no other option for a caching proxy.

Well, the proxy could always believe that the web application did its
homework. That's what nginx does (or so I'm lead to believe) but
experience shows that homework is often skipped for non-business
topics like HTTP that are handed over to the underlying framework or
CMS. I wish more webdevs would realize that HTTP is an application
protocol, not transport (for 5 minutes when HTTP APIs became a thing I
thought it was going to happen).

Dridi