Varnish 503 on POST requests once in a while

[Maninder Singh]

Hi,

I am facing a strange problem.

I am running varnish 4.1.11 on port 80 which is behind a LB.

Varnish then connects to apache 2.4 ( on port 8080 ) and also have php-fpm
running.

Now, everything works fine but once in a while I am seeing 503 response
from varnish for a POST request. During this time there is no log in apache
also ( neither access log nor error log ).

Where does this request vanish ?

How do I debug this issue ?

Some more details:

* I have varnishncsa turned on with following flags.

ExecStart=/usr/bin/varnishncsa -q 'not (ReqURL ~
"\.(js|css|png|jpeg|jpg|ico|gif|swf|woff)" or ReqHeader:User-Agent ~
"HTTP-Monitor" or ReqHeader:User-Agent ~ "Monitoring/1.1")' -a -w
/var/log/varnish/varnishncsa.log -D -f /etc/sysconfig/varnishncsa

Here is the entry in varnish log during that time.

X.X.X.X, LB.IP.0.0 - - [25/Dec/2019:00:26:40 -0600] 2629363 "POST
http://mydomain.com/LetsCelebrate HTTP/1.1" 503 1288 "
https://referringdomain.com/LetsCelebrate?hostingUrl=https%3A%2F%2Fxy.abc.com"
"Mozilla/5.0 (Linux; Android 9; SM-A920F) AppleWebKit/537.36 (KHTML, like
Gecko) Chrome/79.0.3945.93 Mobile Safari/537.36" "mydomain.com" miss

In .vcl file I have following set.
 .first_byte_timeout = 120s;

Now, I am not logging varnish to apache requests ( which backend fetch ).
As you can see above I am only logging client requests.

Is their a way to log client requests like above but then also log backend
fetch failures to another log file ?
[ Running 2 instances of varnishncsa ? ]

Any pointers ?

Please let me know.

Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20191226/b78f272b/attachment-0001.html>