Dealing with "http format error"

[Jay Luker]

After continuing to debug my problem I've had to reassess what the
actual issue is. Again, the request URL I'm trying to proxy is this:

http://nedwww.ipac.caltech.edu/cgi-bin/nph-objsearch?z_unit=z&z_value1=&z_value2=&radius=9.99996&zv_breaker=30000.0&hconst=73&omegam=0.27&out_equinox=J2000.0&list_limit=5&in_equinox=J2000.0&lon=05+23+34.6&in_csys=Equatorial&omegav=0.73&obj_sort=Distance+to+search+center&corr_z=1&out_csys=Equatorial&img_stamp=NO&lat=-69+45+22&ot_include=ANY&of=ascii_bar&nmp_op=ANY&z_constraint=Unconstrained&search_type=Near+Position+Search

It appears that because this backend does not provide any http headers
in the response that varnish is interpreting part of the response
itself as the protocol, status code, etc. The log output below
indicates that the RxProtocol, RxStatus, RxResponse and RxHeader
values actually contain strings from what should be the body of the
response.

"""
   12 TxRequest    b GET
   12 TxURL        b
http://nedwww.ipac.caltech.edu/cgi-bin/nph-objsearch?z_unit=z&z_value1=&z_value2=&radius=9.99996
&zv_breaker=30000.0&hconst=73&omegam=0.27&out_equinox=J2000.0&list_limit=5&in_equinox=J2000.0&lon=05+23+34.6&in_csys=
Equatorial&omegav=0.73&obj_sort=Distance+t
   12 TxProtocol   b HTTP/1.1
   12 TxHeader     b Host: nedwww.ipac.caltech.edu
   12 TxHeader     b User-agent: Python-urllib/1.16
   12 TxHeader     b X-Varnish: 1646987917
   12 TxHeader     b X-Forwarded-For: 131.142.185.41
    0 CLI          - Rd ping
    0 CLI          - Wr 0 200 PONG 1285100629 1.0
   12 RxProtocol   b Results
   12 RxStatus     b from
   12 RxResponse   b query to  NASA/IPAC Extragalactic Database (NED),
   12 RxHeader     b which is operated by the Jet Propulsion
Laboratory, California Institute of
   12 RxHeader     b Technology, under contract with the National
Aeronautics and Space Administration.
   12 RxHeader     b This work was (partially) supported by the US
National Virtual Observatory
   12 RxHeader     b development project, which is funded by the
National Science Foundation
   12 RxHeader     b under cooperative agreement AST0122449 with The
Johns Hopkins University.
   12 HttpGarbage  b

Results
   11 FetchError   c http format error
   12 BackendClose b ned
   11 VCL_call     c error
   11 VCL_return   c deliver
   11 Length       c 488
   11 VCL_call     c deliver
   11 VCL_return   c deliver
   11 TxProtocol   c HTTP/1.1
   11 TxStatus     c 503
"""

Can someone tell me if it's possible to correct this response prior to
the point varnish tries to interpret it? vcl_fetch appears to be too
late in the process.

Thanks,
--jay

On Wed, Sep 15, 2010 at 2:14 PM, Jay Luker <lbjay at reallywow.com> wrote:
> I have a backend that I do not control that is causing an error in
> varnish due (I think) to a lack of http headers in the response. The
> URL in question is http://bit.ly/c8AXGI (shortened). If you look
> you'll see that the http response consists of "HTTP/1.0 200 OK" and
> that's all. No content-type, content-length, etc. This results in the
> following varnishlog output:
>
> Results
>   11 FetchError   c http format error
>   12 BackendClose b ned
>   11 VCL_call     c error
>   ...
>
> My question is, since I don't have any control of this backend, is
> there some VCL magic I can use to fudge whatever missing headers are
> causing varnish to spit the bit on the response? I have already
> unsuccessfully tried the only obvious thing I could think of, which
> was to add to vcl_fetch:
>
>    set obj.http.Content-Type = "text/plain; charset=ascii";
>    set obj.http.Content-Length = "100000";
>    set obj.http.Content-Encoding = "none";
>
> I'm using v2.06.
>
> Thanks,
> --jay
>