backend-304 magic header merge in backend_response

[Martin Gaitzsch]

Good morning!

Yesterday I spent quite a while to understand the following behavior of
varnish (simplified):

first request
-------------
hash, backend response 200, no cache-control, cacheable, ttl 2m, 6h
grace and set "Cache-Control: no-cache" to make the client return to varnish

<...2m ttl passes...>

another request
---------------
triggers bg_fetch because of grace
backend response 304, again no cache-control from backend
BUT: the "OBJ".http.Cache-Control is merged to the response! So in
backend_response beresp.http.Cache-Control is "no-cache" - which is not
intuitive and also triggers code like this in backend_response (also see
builitin.vcl):

	if (!beresp.http.Surrogate-Control && beresp.http.Cache-Control ~
"(?i:no-cache|no-store|private)") {
		# Mark as "Hit-For-Miss" for the next 2 minutes
		set beresp.ttl = 120s;
		set beresp.uncacheable = true;
		return (deliver);
	}

...and that's not what I intended because my very useful grace-object
gets lost! (which we need during deployment 'downtimes')

Putting (! beresp.was_304) around the if-statement is not a solution
because this will fail in other situations. Valid workarounds are:

1. deleting Cache-Control in backend_response and only set it in deliver
downsides:
  - split "object preparation code" to backend_response and deliver,
which make vcl more complex than needed
  - needs to be done on every single response again instead of once in
backend_response

or
2. saving the original Cache-Control to an additional header and do
additional evaluations based on this header
downsides:
  - need to delete this header again in deliver on every response
  - real backend response headers of the current response still not
available for decisions; the merged headers are not sufficient to
conclude to the real backend response headers


Both options are not very appealing to me. How about giving access to
obj.* in backend_response and putting the real backend response headers
to beresp.* like the name indicates? This would be a clean transparent
and more intuitive solution which does not need header merge-magic.

Best
Martin