Hi there!<br><br>I have the exact same problem, and it comes and goes as it pleases..<br>I'm testing varnish with different backends (different customers sites in the same instance) and every once in a while it locks up at the same place (the X-Forwarded-For header on varnishlog) for any site. Not the others though, as they are accessible through varnish just fine. And after a while of working the cache the site that locks up comes back to life.<br>
This is what varnishlog shows just before the 503 error:<br>Request:<br><font size="1"><span style="font-family: courier new,monospace;"> 9 SessionOpen c MY_IP_ADDRESS 56186 VARNISH_SERVER_IP:80</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 ReqStart c </span></font><font size="1"><span style="font-family: courier new,monospace;">MY_IP_ADDRESS</span></font><font size="1"><span style="font-family: courier new,monospace;"> 56186 444163595</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 RxRequest c GET</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 RxURL c /</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 RxProtocol c HTTP/1.1</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 RxHeader c Host: </span></font><font size="1"><span style="font-family: courier new,monospace;">BACKEND_HOSTNAME</span></font><font size="1"><span style="font-family: courier new,monospace;"></span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 RxHeader c User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en; rv:<a href="http://1.9.0.3">1.9.0.3</a>) Gecko/2008101315 Ubuntu/8.10 (intrepid) Firefox/3.0.3</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 RxHeader c Accept-Language: q=0.8,en-us;q=0.5,en;q=0.3</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 RxHeader c Accept-Encoding: gzip,deflate</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 RxHeader c Keep-Alive: 300</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 RxHeader c Connection: keep-alive</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"></span><span style="font-family: courier new,monospace;"> 9 VCL_call c recv</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 VCL_return c lookup</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 VCL_call c hash</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 VCL_return c hash</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 VCL_call c miss</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 VCL_return c fetch</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 Backend c 17 </span></font><font size="1"><span style="font-family: courier new,monospace;">BACKEND_NAME</span></font><font size="1"><span style="font-family: courier new,monospace;"> </span></font><font size="1"><span style="font-family: courier new,monospace;">BACKEND_NAME</span></font><font size="1"><span style="font-family: courier new,monospace;"></span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 17 TxRequest - GET</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 17 TxURL - /</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 17 TxProtocol - HTTP/1.1</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 17 TxHeader - Host: </span></font><font size="1"><span style="font-family: courier new,monospace;">BACKEND_HOSTNAME</span></font><font size="1"><span style="font-family: courier new,monospace;"></span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 17 TxHeader - User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en; rv:<a href="http://1.9.0.3">1.9.0.3</a>) Gecko/2008101315 Ubuntu/8.10 (intrepid) Firefox/3.0.3</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 17 TxHeader - Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 17 TxHeader - Accept-Language: q=0.8,en-us;q=0.5,en;q=0.3</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 17 TxHeader - Accept-Encoding: gzip,deflate</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 17 TxHeader - Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"></span><span style="font-family: courier new,monospace;"> 17 TxHeader - X-Varnish: 444163595</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 17 TxHeader - X-Forwarded-For: MY_IP_ADDRESS</span><br style="font-family: courier new,monospace;">
</font><br>===> Long waiting time (sorry, didn't really time it, but it's like 5 minutes)<br>Response:<br><font size="1"><span style="font-family: courier new,monospace;"> 17 BackendClose - BACKEND_NAME</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 VCL_call c error</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 VCL_return c deliver</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 Length c 452</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 VCL_call c deliver</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 VCL_return c deliver</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 TxProtocol c HTTP/1.1</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 TxStatus c 503</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 TxResponse c Service Unavailable</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 TxHeader c Server: Varnish</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 TxHeader c Retry-After: 0</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 TxHeader c Content-Type: text/html; charset=utf-8</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 TxHeader c Content-Length: 452</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 TxHeader c Date: Tue, 11 Nov 2008 11:49:00 GMT</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 TxHeader c X-Varnish: 444163595</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 TxHeader c Age: 945</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 TxHeader c Via: 1.1 varnish</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 TxHeader c Connection: close</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 ReqEnd c 444163595 1226403195.496297359 1226404140.571825743 0.000217438 945.075489759 0.000038624</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 SessionClose c error</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 StatSess c MY_IP_ADDRESS 56186 945 1 1 0 0 0 236 452</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 0 StatAddr - </span></font><font size="1"><span style="font-family: courier new,monospace;">MY_IP_ADDRESS</span></font><font size="1"><span style="font-family: courier new,monospace;"> 0 427594 323 1326 0 0 548 435744 9005255</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 SessionOpen c </span></font><font size="1"><span style="font-family: courier new,monospace;">MY_IP_ADDRESS</span></font><font size="1"><span style="font-family: courier new,monospace;"> 44754 </span></font><font size="1"><span style="font-family: courier new,monospace;">VARNISH_SERVER_IP</span></font><font size="1"><span style="font-family: courier new,monospace;">:80</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 ReqStart c </span></font><font size="1"><span style="font-family: courier new,monospace;">MY_IP_ADDRESS</span></font><font size="1"><span style="font-family: courier new,monospace;"> 44754 444163596</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 RxRequest c GET</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 RxURL c /favicon.ico</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 RxProtocol c HTTP/1.1</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 RxHeader c Host: BACKEND_HOSTNAME</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 RxHeader c User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en; rv:<a href="http://1.9.0.3">1.9.0.3</a>) Gecko/2008101315 Ubuntu/8.10 (intrepid) Firefox/3.0.3</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 RxHeader c Accept: image/png,image/*;q=0.8,*/*;q=0.5</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 RxHeader c Accept-Language: q=0.8,en-us;q=0.5,en;q=0.3</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 RxHeader c Accept-Encoding: gzip,deflate</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 RxHeader c Keep-Alive: 300</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 RxHeader c Connection: keep-alive</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"></span><span style="font-family: courier new,monospace;"> 9 VCL_call c recv</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 VCL_return c lookup</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 VCL_call c hash</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 VCL_return c hash</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 VCL_call c miss</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 9 VCL_return c fetch</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 9 Backend c 14 </span></font><font size="1"><span style="font-family: courier new,monospace;">BACKEND_NAME</span></font><font size="1"><span style="font-family: courier new,monospace;"> </span></font><font size="1"><span style="font-family: courier new,monospace;">BACKEND_NAME</span></font><font size="1"><span style="font-family: courier new,monospace;"></span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 14 TxRequest - GET</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 14 TxURL - /favicon.ico</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 14 TxProtocol - HTTP/1.1</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 14 TxHeader - Host: </span></font><font size="1"><span style="font-family: courier new,monospace;">BACKEND_HOSTNAME</span></font><font size="1"><span style="font-family: courier new,monospace;"></span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 14 TxHeader - User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en; rv:<a href="http://1.9.0.3">1.9.0.3</a>) Gecko/2008101315 Ubuntu/8.10 (intrepid) Firefox/3.0.3</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 14 TxHeader - Accept: image/png,image/*;q=0.8,*/*;q=0.5</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 14 TxHeader - Accept-Language: q=0.8,en-us;q=0.5,en;q=0.3</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"> 14 TxHeader - Accept-Encoding: gzip,deflate</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 14 TxHeader - Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7</span><br style="font-family: courier new,monospace;">
<span style="font-family: courier new,monospace;"></span><span style="font-family: courier new,monospace;"> 14 TxHeader - X-Varnish: 444163596</span><br style="font-family: courier new,monospace;"><span style="font-family: courier new,monospace;"> 14 TxHeader - X-Forwarded-For: MY_IP_ADDRESS</span><br style="font-family: courier new,monospace;">
</font><br>Any one else seeing this? Any one have any idea why this happens?<br><br>Cheers<br><br>Alecs<br><br><div class="gmail_quote">On Tue, Nov 11, 2008 at 5:42 AM, Brano <span dir="ltr"><<a href="mailto:admin@opensubtitles.org">admin@opensubtitles.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Hi all,<br>
<br>
recently we installed Varnish on our server. Everything works fine,<br>
but on download we get this error:<br>
<br>
<a href="http://www.opensubtitles.org/en/download/file/1951965961" target="_blank">http://www.opensubtitles.org/en/download/file/1951965961</a><br>
<br>
Varnish Error 503 Service Unavailable<br>
<br>
Service Unavailable<br>
Guru Meditation:<br>
<br>
XID: 2138997704<br>
<br>
My backend works ok:<br>
<a href="http://web1.opensubtitles.org/en/download/file/1951965961" target="_blank">http://web1.opensubtitles.org/en/download/file/1951965961</a><br>
<a href="http://web2.opensubtitles.org/en/download/file/1951965961" target="_blank">http://web2.opensubtitles.org/en/download/file/1951965961</a><br>
<br>
Server info:<br>
varnishd (varnish-2.0.1)<br>
OS: FreeBSD 7.1-PRERELEASE<br>
PHP Version 5.2.6<br>
httpd: Lighttpd 1.4.20<br>
<br>
Here is log:<br>
276 ReqStart c <a href="http://124.157.249.3" target="_blank">124.157.249.3</a> 50073 2138997704<br>
276 RxRequest c GET<br>
276 RxURL c /en/download/file/1951965961<br>
276 RxProtocol c HTTP/1.1<br>
276 RxHeader c Host: <a href="http://www.opensubtitles.org" target="_blank">www.opensubtitles.org</a><br>
276 RxHeader c User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:<a href="http://1.9.0.3" target="_blank">1.9.0.3</a>) Gecko/2008092417 Firefox/3.0.3 (.NET CLR 3.5.30729)<br>
276 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8<br>
276 RxHeader c Accept-Language: en<br>
276 RxHeader c Accept-Encoding: gzip,deflate<br>
276 RxHeader c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7<br>
276 RxHeader c Keep-Alive: 300<br>
276 RxHeader c Connection: keep-alive<br>
276 RxHeader c Referer: <a href="http://www.opensubtitles.org/en/subtitles/3363915/prison-break-en" target="_blank">http://www.opensubtitles.org/en/subtitles/3363915/prison-break-en</a><br>
276 RxHeader c Cookie: __utma=188827125.1293269160.1192786702.1226384704.1226387542.495; __utmz=188827125.1226166768.475.44.utmccn=(referral)|utmcsr=<a href="http://forum.opensubtitles.org" target="_blank">forum.opensubtitles.org</a>|utmcct=/index.php|utmcmd=refe<br>
276 RxHeader c Cache-Control: max-age=0<br>
276 VCL_call c recv<br>
276 VCL_return c lookup<br>
276 VCL_call c hash<br>
276 VCL_return c hash<br>
276 VCL_call c miss<br>
276 VCL_return c fetch<br>
139 BackendOpen b web2 <a href="http://92.240.234.126" target="_blank">92.240.234.126</a> 61634 <a href="http://92.240.234.119" target="_blank">92.240.234.119</a> 80<br>
276 Backend c 139 www_director web2<br>
139 TxRequest b GET<br>
139 TxURL b /en/download/file/1951965961<br>
139 TxProtocol b HTTP/1.1<br>
139 TxHeader b Host: <a href="http://www.opensubtitles.org" target="_blank">www.opensubtitles.org</a><br>
139 TxHeader b User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:<a href="http://1.9.0.3" target="_blank">1.9.0.3</a>) Gecko/2008092417 Firefox/3.0.3 (.NET CLR 3.5.30729)<br>
139 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8<br>
139 TxHeader b Accept-Language: en<br>
139 TxHeader b Accept-Encoding: gzip,deflate<br>
139 TxHeader b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7<br>
139 TxHeader b Referer: <a href="http://www.opensubtitles.org/en/subtitles/3363915/prison-break-en" target="_blank">http://www.opensubtitles.org/en/subtitles/3363915/prison-break-en</a><br>
139 TxHeader b Cookie: __utma=188827125.1293269160.1192786702.1226384704.1226387542.495; __utmz=188827125.1226166768.475.44.utmccn=(referral)|utmcsr=<a href="http://forum.opensubtitles.org" target="_blank">forum.opensubtitles.org</a>|utmcct=/index.php|utmcmd=refe<br>
139 TxHeader b X-Varnish: 2138997704<br>
139 TxHeader b X-Forwarded-For: <a href="http://124.157.249.3" target="_blank">124.157.249.3</a><br>
<br>
If you need more info, please let me know.<br>
<br>
Any help appreciated.<br>
<br>
Thank you.<br>
<font color="#888888"><br>
/Brano<br>
_______________________________________________<br>
varnish-misc mailing list<br>
<a href="mailto:varnish-misc@projects.linpro.no">varnish-misc@projects.linpro.no</a><br>
<a href="http://projects.linpro.no/mailman/listinfo/varnish-misc" target="_blank">http://projects.linpro.no/mailman/listinfo/varnish-misc</a><br>
</font></blockquote></div><br>