400 Bad Request and whitespace in headers

[Guillaume Quintard]

Hi Justin,

The URL you won't get, since it comes after the bad method, so varnish
bails out before that. The IP you have in the session information. Update
your command to

    varnishlog -q 'RespReason ~ "Bad Request"'

However, the reported IP will most likely be the ALB's, since it's the one
sending you the request.

-- 
Guillaume Quintard


On Tue, Jul 16, 2024 at 10:01 AM Justin Lloyd <justinl at arena.net> wrote:

> I got a response but it hasn't been helpful so far, basically just saying
> "look at ALB an WAF logs". The problem is that the varnishlog output
> doesn't give me enough to use to filter the WAF logs (ALB logging is
> currently not enabled), and even during 1 minute the number of requests is
> too much to filter through via the AWS console (and I don't yet have a good
> system for parsing the actual logs via Splunk or some other tool). Is there
> any way, for example, to get Varnish to display the IP and/or URL requested
> in the type of bad request output I showed in my original message? The
> command I used for that output was just
>
> varnishlog -q 'RespReason ~ "Bad Request"'
>
> Thanks,
> Justin
>
> -----Original Message-----
> From: varnish-misc <varnish-misc-bounces+justinl=
> arena.net at varnish-cache.org> On Behalf Of Justin Lloyd
> Sent: Tuesday, July 16, 2024 6:28 AM
> To: geoff at uplex.de; Guillaume Quintard <guillaume.quintard at gmail.com>
> Cc: varnish-misc at varnish-cache.org
> Subject: RE: 400 Bad Request and whitespace in headers
>
> > That should be intelligible to WAF support, without any reference to
> Varnish at all.
>
> I would hope so, I just wanted to cover all the bases.
>
> I do have a number of managed WAF rulesets in use, but maybe there's one
> that would cover this particular issue. I would think this should be
> checked for in all cases, but perhaps not. Hopefully Support can help
> explain why these would be getting through.
>
> As for the additional work, there is so much garbage that still gets
> through, despite my best efforts with the WAF configuration so far, the
> more obvious stuff I can reasonably block, the better.
>
> I appreciate the feedback! I'll respond once I get more info from Support,
> especially regarding the nul byte issue.
>
> Justin
>
> -----Original Message-----
> From: Geoff Simmons <geoff at uplex.de>
> Sent: Tuesday, July 16, 2024 1:19 AM
> To: Justin Lloyd <justinl at arena.net>; Guillaume Quintard <
> guillaume.quintard at gmail.com>
> Cc: varnish-misc at varnish-cache.org
> Subject: Re: 400 Bad Request and whitespace in headers
>
> On 7/16/24 03:15, Justin Lloyd wrote:
> >
> > I meant blocking them at the AWS WAF, before they even get to any of
> > the web servers, i.e. less work for Varnish. I'd need to get the raw
> > headers and I wasn't having luck with that so far in the WAF
> > CloudTrail logs, so I've opened up a support case about it, but I was
> > hoping to possibly get some insight here, as well, since I don't know
> > whether the WAF support specialists will know much about using Varnish.
>
>  From what you've described, there were evidently requests with whitespace
> in header field names, a violation of HTTP syntax. That should be
> intelligible to WAF support, without any reference to Varnish at all.
>
> Why isn't a WAF rejecting requests like that by default?
>
> The invalid header names, and also your previous Varnish log excerpt
> showing "GET" followed by a nul byte, have the whiff of someone attempting
> a request smuggling attack. But it could be just a de-synchronized HTTP
> client. Either way, I would have expected a WAF to filter such requests,
> without having to ask support.
>
> And to agree with what Guillaume said, Varnish is not getting much
> additional work when it rejects those requests. The one in your previous
> example was probably taken care of in single-digit microseconds. It is true
> that the client connection would be spared if the request hadn't been
> forwarded at all. And it helps to use connections efficiently at a heavily
> loaded site.
>
>
> Best,
> Geoff
> --
> ** * * UPLEX - Nils Goroll Systemoptimierung
>
> Scheffelstraße 32
> 22301 Hamburg
>
> Tel +49 40 2880 5731
> Mob +49 176 636 90917
> Fax +49 40 42949753
>
> http://uplex.de/
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20240716/1912851b/attachment.html>