Reintroduce a req.uncacheable flag

[Dridi Boukelmoune]

Greetings,

I would like to reopen a discussion started by Guillaume via a pull
request:

https://github.com/varnishcache/varnish-cache/pull/3457

For reasons other than the ones he exposed I have reached the
conclusion that such a flag would be both relevant and useful. Not
req.hash_always_pass as originally suggested but req.uncacheable
(and to Guillaume's credit he did make the suggestion first in the pull
request description).

I didn't take part in the pull request discussion and don't remember
if I took a position but chances are I would have opposed a flag
called hash_always_pass. On the other hand, here are the reasons why I
think req.uncacheable would make sense.

First, we already have a relationship between bereq.uncacheable and
bereq.uncacheable: the former implies the latter. I believe the same
relationship could exist between req and bereq.

Second, since the VCL built-in split it makes even more sense to get
the ability to mark a request as uncacheable. If you wish to keep your
code organized as per the split, but don't want an early return to
prevent a later step to be taken, then a flag instead of a
return(pass) lets the flow of execution continue and guarantees that a
lookup attempt would result in a pass transition.

As we are heading towards a dot-zero release, we could even break the
built-in VCL to replace pass transitions with req flagging. That would
be a breaking change in the sense that we would no longer break the
flow of execution, but in practice that does not change the outcome of
existing VCL not taking advantage of the split (for example upgrading
from 6.0 LTS) and for VCL authors who already adopted the split, I
assume they'd be savvy enough to check their code.

Third, one more reason to bring it back as req.uncacheable instead of
hash_always_pass is consistency across the board.

I'm not diving into implementation details on purpose, my goal with
this thread is to discuss the possibility to reintroduce a new
iteration on #3457 and shine a new light on it from the perspective
of the recent split and the additional opportunity it grants us.

Cheers,
Dridi