(be)(req|resp).body

[Nils Goroll]

I would like to wrap up the discussion so far and start with a review and
outlook on body access. I will prepare a separate email regarding the
abandon/backend_error changes proposed.

To avoid working around the missing features in core at the moment, I would like
to go ahead with the actual implementation ASAP. So I'm asking phk for a review.

>From a VCL autor's perspective, (be)?(req|resp).body could be first class
citizens in VCL and/or we could have better vmod support for working with them.
The basic issue is that bodies are not strings, but stevedore objects made up of
several (discontinuous) allocations. So, for example, for a regular expression
match, we'd need to use multi segment matching, which has specific issues (see
pcrepartial(3)).

I see this boiling down to three steps

#1 VCL support for set/unset (be)?(req|resp).body

#2 Better vmod support for bodies
   - iterate over bodies?
   - push fetch processors?

#3 more complete support for working with bodies
   - string operators
   - reading (cast to STRING_LIST ?) bodies

How exactly #2 and #3 are going to look is unclear to me still, but I guess the
major change would be that some of the cases would require vcl_recv /
vcl_backend_fetch to pull the body into cache early.

Nevertheless, I can't see a case where implementing #1 would prevent us from
DTRT regarding #2 and #3.


To implement #1 we have/need:

- HAVE

  vcl_backend_fetch {
	unset bereq.body;		# in buitlin.vcl
  }

  vcl_backend_error {
	unset beresp.body;		# will add a VTC
	set   beresp.body = "...";	# in buitlin.vcl
  }

  vcl_synth {
	unset resp.body;		# will add  a VTC
	set   resp.body = "...";	# in buitlin.vcl
  }

- NEED

  # synthetic backend request body
  vcl_backend_fetch {
	set   bereq.body = "...";
  }

  # synthetic cache object without vcl_backend_error de-tour
  # will drain-read the backend response
  vcl_backend_response {
	unset beresp.body;
	set   beresp.body = "...";
  }

  # synthetic client response without the vcl_synth de-tour
  vcl_deliver {
	unset resp.body;
	set   resp.body = "...";
  }


Nils