Problem with error and (large) POST requests

[Olivier G]

I've done additional tests, and I'm probably misunderstanding some stuff… :/

First, it seems to me that Varnish do consume the entire payload *before* sending anything to the backend (naively, I was thinking that varnish would start the backend request "sooner").


So, I set-up ngynx as a "dumb" backend to absorbe the post payloads.
Additionally, I told ngynx to have a very small POST size limit, and send 413.

If curl-ing ngynx with a (too) big payload, ngynx immediately validates the content-length and sends the 413 answer. The upload stops gracefully without much content being sent, and curl happily reads the 413.

Interestingly, if I use that setup as a backend to Varnish to workaround my problem (as I described in the previous mail), Varnish ends-up again with 
    5 FetchError   c backend write error: 32 (Broken pipe)
and does 503.
Meaning to me that it couldn't handle ngynx answer… while curl can…

</confused>

- Olivier


Le 15 sept. 2011 à 04:13, Olivier G a écrit :

> Hi there.
> 
> I have a problem with calling "error" and POST requests.
> 
> Reproducing is rather simple - use a default varnish install, then just do:
> 
> sub vcl_recv{
> 	error 413 "Whatever";
> }
> 
> Now, curl your server by posting a (small) payload.
> -> (might) work, and the client (might) get a 413 answer
> 
> Now, curl your server by posting a (somewhat huge ~ 1MB) payload.
> -> curl will end-up with a "broken pipe" without getting any answer.
> 
> Browsers (at least Firefox and Chrome) doing XHR in the same setup will end-up in the same state (readyState 4, status 0) (without being able to read the answer).
> 
> On the other hand, Varnishlog demonstrates that the answer clearly is sent, and everything is normal from Varnish POV.
> 
> 
> Interestingly, even Varnish as a client (if pointing a backend to the Varnish sending the error in recv), exhibits:
> 5 FetchError   c backend write error: 32 (Broken pipe) 
> and returns a 503 no matter the actual code sent by the "backend".
> … which makes me think that varnish (server) behavior in that case is wrong.
> 
> 
> Also:
> If you do the "error" in "vcl_pass", you get the same problem.
> If you "error" in "vcl_fetch", though… things work as expected.
> 
> 
> 
> So, it looks to me like:
> 1- varnish doesn't read the request body before it actually queries the backend (in which case it passes it), or before it gets out of hit (in which case the body is dropped - though, if the request has a body, it's quite likely we will call "pass"...)
> 2- calling "error" means Varnish closes the tcp connection immediately
> 3- a client that finds itself in a position where the connection is closed *before it has time to write the entire request* just stops here and fail
> 4- calling "error" means Varnish doesn't wait for the entire request body to be available/consumed (if it hasn't been already)
> 
> 
> Assuming I'm correct in these assumptions, then it means that "error" when handling requests with a body is NOT usable at all before having queried a backend.
> 
> 
> I'm not sure this behavior is intended - if it is, then maybe the documentation should be updated? (I couldn't find a reference to that problem in here: https://www.varnish-cache.org/docs/3.0/reference/vcl.html), and error is clearly mentioned as being usable in recv, etc.
> If it's not, then possibly the solution would be that the error routine actually makes sure that the entire request body has been sent.
> 
> 
> 
> Right now, I'm workarounding by calling "error" inside vcl_fetch after having queried a dumb backend which only purpose is to force varnish to consume the body - though this is quite convoluted and suboptimal.
> 
> 
> What do you guys think?
> 
> 
> Thanks a lot!
> 
> - Olivier

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110915/e9fd4f2e/attachment-0003.html>