varnish-misc Digest, Vol 136, Issue 12

[Matthew Johnson]

> Date: Wed, 12 Jul 2017 11:20:52 +0200
> From: Geoff Simmons <geoff at uplex.de>
> To: varnish-misc at varnish-cache.org
> Subject: Re: Value of builtin.vcl - vcl_recv on modern websites
> Message-ID: <bd1c8f9d-270f-90f0-afd5-226ffde3eeeb at uplex.de>
> Content-Type: text/plain; charset=windows-1252
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On 07/12/2017 09:14 AM, Matthew Johnson wrote:
>>
>> Since Varnish 1.0 the builtin (or default) vcl_recv has had this
>> statement: if (req.http.Authorization || req.http.Cookie) {
>>
>> /* Not cacheable by default */
>>
>> return (pass);
>>
>> }
>>
>> My issue with this is the req.http.Cookie check, In any modern
>> website cookies are always present.
>
> While I'm sympathetic to all the things you said and don't mean to
> disregard it but cutting things short, the answer here is simple:
> there is no other choice for the default configuration of a caching
> proxy, because those two headers mean that the response may be
> personalized.

Agreed. I think the area im looking to explore is how you move
forwards from the default configuration as there are a few paths that
can be taken.

>
> Many uses of cookies don't have that effect, of course, but Varnish
> has no way of knowing that. As bad as the effect of the default config
> may seem on sites that use cookies on every request -- Varnish doesn't
> cache anything -- it would be much worse if someone sets up Varnish,
> doesn't think of the consequences of not changing the default
> configuration, and you end up seeing someone else's personal data in
> cached responses.
>
> This problem is not specific to Varnish, but to any server that tries
> to do what Varnish tries to do. I know from experience that it's
> generally futile to say so, but this situation really ought to lead to
> some widespread re-thinking throughout the industry. Forgive me for
> shouting, soapbox-style, but this gives me an opportunity to sound off
> on a pet peeve:
>
> ==> Maybe modern web site SHOULD NOT use cookies on every request!
> Because of the way cookies interfere with downstream caching.
>
> I have come to the conviction that many uses of cookies are a result
> of lazy thinking in app development. Many PHP devs, for example, are
> in the habit of saying session_start(); at the beginning of every
> script, without thinking twice about whether they really need it. I
> have seen uses of cookies where "just toss that thing into a cookie"
> was evidently the easy decision to make. I have seen cookies with
> values that are 3KB long.

Most .Net sites are the same, Almost any application I come across
takes a "session first" approach.


>
> (Sometime over beer I'll tell you about that little database that
> someone wanted to transport over a cookie, a base64-encoded CSV file
> whose data was *also* base64-encoded, leading to a doubly
> base64-encoded cookie value, in every request.)
Beer sounds good. There are definitely many war stories on my side of
madness in the use of cookies. Database in cookie in a novel approach!

>
> This is an instance of an issue that you encounter a lot with the use
> of Varnish in practice: app development that doesn't think outside of
> its own box in terms of functionality and performance. Rather than
> thinking about the benefits of handing off some of your work, by
> letting someone else serve your cached responses for you.
>
> HTTP was conceived from the beginning to enable caching as a means of
> solving performance problems in slow networks. A well-configured
> deployment of Varnish shows how beneficial that can be. But the
> universal and unreflected use of cookies is one of the forces
> presently at work that actively undermine that part of the equation.
>
>> A classic example of this is someone adding javascript via Google
>> Tag Manager which then sets a cookie.
>
> One might have hoped that the Googlers, of all people, would have more
> awareness of the trouble that they could cause by doing that.
>
Whilst Google do contribute to the issue with their own scripts,
Google Tag Manager allows non technical people to deploy additional
3rd party javascript on a website. In the web performance space this
always leads to slow loading websites in the browser but the cookie
problem then plays into caching rules aswell.

>> Do you still recommend configurations fall through to the
>> underlying vcl_recv logic?
>>
>> Options i can think of:
>
> In a project where I am able to work with the app devs, I have had
> good experience with working out a policy with them: if you MUST have
> cookies in every request (although I WISH YOU WOULD RECONSIDER THAT),
> then the caching proxy cannot make caching decisions on your behalf.
> Only you can know if your response is cacheable, despite the presence
> of cookie foo or bar, but is not cacheable if the cookie is baz.
>
> So if you want your response to be cached, you MUST say so in a
> Cache-Control header. The proxy will not cache any other responses.

I consider this ideal if it's possible to take that approach. I am
often caught with customers using off the shelf platforms and less
than ideal control over their application.

>
> Then we write VCL to bypass builtin's vcl_recv, and start Varnish with
> - -t 0 (default TTL is 0s). Responses are then cached only if they
> announce that they are cacheable.
>
> Of course, this has the effect that you're lamenting -- Varnish
> doesn't cache anything by default -- but in my experience, the result
> is that devs have become very good at thinking about the cacheability
> of their responses.
>
> That boils down to answering your question by saying no, you can't use
> builtin vcl_recv in a situation like that. When the cookies, like the
> Evil, are always and everywhere (to paraphrase a saying in Germany),
> and some cookies lead to cacheable responses while others don't, then
> there's no other option for a caching proxy.

A great summary, I think that summarises Dridi views earlier around
the way HTTP has been implemented in applications. No perfect wins!

>
>
> Best,
> Geoff
> - --
> ** * * UPLEX - Nils Goroll Systemoptimierung
>
> Scheffelstra?e 32
> 22301 Hamburg
>
> Tel +49 40 2880 5731
> Mob +49 176 636 90917
> Fax +49 40 42949753
>
> http://uplex.de
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
>
> iQIcBAEBCAAGBQJZZen0AAoJEOUwvh9pJNURnWEQAK9ucYSXEcrEwbmOrroBWoGK
> iR9a8OFhst1rVKFQ2vNTUpw+OYM8vf8SJYToyq2VxG5/f/uGsT6nSVRWGKThgeV1
> geyyUQfbbDte1at4aFy6HX6LeCt62Si0L9KUZMZMkI5C6m6FKgA/5HecUchhuXdP
> Li17DXKvzrQyTBJpvbk2vBZGPVlErnVAUz75IeJMrD6t/WGO0PsZvkC/l8LZhqcD
> 2S2R9SHYMIyBrWSZm+YsI1DxMwvH6Gt84NRPpKcHQQ7TKEfvtOq0NwoqcNOB26EL
> KIfOuVJdbiMvD5D+BZud/7a7UzSpJz5klLMdTcJMN60MrHJjGcok/5KiG7TNNowj
> hMy5YUYpuIybsWzcvB5Ie/nteb0WyXt5+LYkxjP9dbN7AN3k+aU1PboSOyqYXO3u
> KK1al00LMKHfzMHs1vF3QHRt2Q1Udud6dCdHuw6TyJ7eWCc9YGgU8NyboMLkXBhO
> fBVNUQQjfNjaDWhKFvUIMEsGZhgvzzuMvjlZNhkc/lcDLmU8wVXyiMFSoBcuR1sX
> 7EM1wBUKcKix0wE4QPl9608ql/5LF3Ms+wqDpmS0ECgFIf1yKMWFZt9iHhMUbch2
> 7fhr77vVjVD1K6nKHqDGOuLp4Cq+lfBJkd7PX2huQUV/hc00C8+NEieD77wuAwk7
> OPiGNt5YqmDNjtZmFUVH
> =BlAn
> -----END PGP SIGNATURE-----
>
>
>
> *********************************************