"straight insufficient bytes" 503 error
Adam Arrowood
adam.arrowood at oit.gatech.edu
Thu Oct 17 16:10:46 CEST 2013
Hi,
I am having a problem with a varnish cache installation that I cannot find the answer to... I'm getting 503 (Service Unavailable) errors when trying to view an image served from a drupal 7 backend.
Here are details (ips, names are changed for privacy):
varnish cache -> foo.gatech.edu (192.168.0.1), RHEL6, varnish-3.0.4-4.el6.art.x86_64 from the atomicorp repo
drupal server -> bar.gatech.edu (192.168.0.2), RHEL6, apache 2.2, drupal 7, defined in vcl as backend 'bar', has 'foo.gatech.edu' as a ServerAlias
web client (Chrome) -> 10.10.10.1
Users can access the site via http://foo.gatech.edu/ , which goes through the varnish cache, and works well, generally. The admins of the site can access the site via http://bar.gatech.edu/ , which does not go through the cache.
When I use my web browser to try an fetch a JPEG image from http://foo.gatech.edu/home/images/50384 , I get a 503 error from varnish.
This is odd for several reasons:
- I can see a successful (HTTP 200 result) request for the image both in apache logs of bar.gatech.edu, and in the back-end (b) entries from the varnishlog on foo.gatech.edu
- It happens for any default-sized being served from the drupal application on bar.gatech.edu, so it's not specific to one image. Images in the /home/images/ directory are served via the "Adaptive Image Styles" varnish module
- I can always successfully fetch the image via http://bar.gatech.edu/home/images/50384 , bypassing the cache
- It does *not* happen for smaller versions of the same image. So, http://foo.gatech.edu/home/images/50384/thumbnail_scaled returns a smaller jpeg version of the image, through the cache, successfully every time. Using HTTPFox (similar to HTTP Live Headers) on Firefox shows nothing significantly different between fetching the original image and the thumbnail version of the image from bar.gatec.edu directly
Things I've tried:
- restarting varnish, rebooting the box, etc.
- using both file and malloc storage options in /etc/sysconfig/varnish
- upping the backend-timeouts to very large values
- eliminate any vhosts/hostname issues by changing my /etc/hosts so that foo.gatech.edu resolves to 198.168.0.2, and then verifying http://foo.gatech.edu/home/images/50384 returns a good image to my browser
Here is the varnishlog output from a failed request... The key line is "11 FetchError c straight insufficient bytes" :
-----------------------------------
13 BackendOpen b bar 192.168.0.1 44950 192.168.0.2 80
13 TxRequest b GET
13 TxURL b /home/images/50384
13 TxProtocol b HTTP/1.1
13 TxHeader b Host: foo.gatech.edu
13 TxHeader b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
13 TxHeader b Pragma: no-cache
13 TxHeader b User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36
13 TxHeader b DNT: 1
13 TxHeader b Accept-Language: en-US,en;q=0.8
13 TxHeader b X-Forwarded-For: 10.10.10.1
13 TxHeader b X-Varnish: 1892649961
13 TxHeader b Accept-Encoding: gzip
13 RxProtocol b HTTP/1.1
13 RxStatus b 200
13 RxResponse b OK
13 RxHeader b Date: Wed, 16 Oct 2013 20:17:23 GMT
13 RxHeader b Server: Apache/2.2.3 (Red Hat)
13 RxHeader b X-Powered-By: PHP/5.3.14 ZendServer/5.0
13 RxHeader b X-Drupal-Cache: MISS
13 RxHeader b Expires: Sun, 19 Nov 1978 05:00:00 GMT
13 RxHeader b Last-Modified: Wed, 16 Oct 2013 20:17:23 +0000
13 RxHeader b Cache-Control: public, max-age=21600
13 RxHeader b ETag: "1381954643-1"
13 RxHeader b Content-Length: 55293
13 RxHeader b Content-Language: en
13 RxHeader b Vary: Accept-Encoding
13 RxHeader b Content-Encoding: gzip
13 RxHeader b Connection: close
13 RxHeader b Content-Type: image/jpeg
13 Fetch_Body b 4(length) cls -1 mklen 1
13 BackendClose b bar
11 SessionOpen c 10.10.10.1 64106 :80
11 ReqStart c 10.10.10.1 64106 1892649961
11 RxRequest c GET
11 RxURL c /home/images/50384
11 RxProtocol c HTTP/1.1
11 RxHeader c Host: foo.gatech.edu
11 RxHeader c Connection: keep-alive
11 RxHeader c Cache-Control: no-cache
11 RxHeader c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
11 RxHeader c Pragma: no-cache
11 RxHeader c User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.101 Safari/537.36
11 RxHeader c DNT: 1
11 RxHeader c Accept-Encoding: gzip,deflate,sdch
11 RxHeader c Accept-Language: en-US,en;q=0.8
11 VCL_call c recv
11 VCL_return c lookup
11 VCL_call c hash
11 Hash c /home/images/50384
11 Hash c foo.gatech.edu
11 VCL_return c hash
11 VCL_call c miss fetch
11 Backend c 13 bar bar
11 TTL c 1892649961 RFC 21600 -1 -1 1381954643 0 1381954643 280299600 21600
11 VCL_call c fetch deliver
11 ObjProtocol c HTTP/1.1
11 ObjResponse c OK
11 ObjHeader c Date: Wed, 16 Oct 2013 20:17:23 GMT
11 ObjHeader c Server: Apache/2.2.3 (Red Hat)
11 ObjHeader c X-Powered-By: PHP/5.3.14 ZendServer/5.0
11 ObjHeader c X-Drupal-Cache: MISS
11 ObjHeader c Expires: Sun, 19 Nov 1978 05:00:00 GMT
11 ObjHeader c Last-Modified: Wed, 16 Oct 2013 20:17:23 +0000
11 ObjHeader c Cache-Control: public, max-age=21600
11 ObjHeader c ETag: "1381954643-1"
11 ObjHeader c Content-Language: en
11 ObjHeader c Vary: Accept-Encoding
11 ObjHeader c Content-Encoding: gzip
11 ObjHeader c Content-Type: image/jpeg
11 ObjHeader c X-Varnish-IP: 192.168.0.1
11 FetchError c straight insufficient bytes
11 Gzip c u F - 29537 55293 80 80 236227
11 VCL_call c error deliver
11 VCL_call c deliver deliver
11 TxProtocol c HTTP/1.1
11 TxStatus c 503
11 TxResponse c Service Unavailable
11 TxHeader c Server: Varnish
11 TxHeader c Content-Type: text/html; charset=utf-8
11 TxHeader c Content-Length: 829
11 TxHeader c Accept-Ranges: bytes
11 TxHeader c Date: Wed, 16 Oct 2013 20:17:23 GMT
11 TxHeader c X-Varnish: 1892649961
11 TxHeader c Age: 0
11 TxHeader c Via: 1.1 varnish
11 TxHeader c Connection: close
11 TxHeader c X-Cache: MISS
11 Length c 829
11 ReqEnd c 1892649961 1381954643.203719378 1381954643.275374174 0.000027418 0.071626902 0.000027895
11 SessionClose c error
11 StatSess c 10.10.10.1 64106 0 1 1 0 0 0 256 829
-----------------------------------
As I mentioned, I've Goolged around for this error "straight insufficient bytes" and didn't find any solution. I can find the line that returns that error in the varnish cache source code, but without knowning the internals of the src, it's Greek to me.
Anyone got any guesses/suggestions? Any help will be greatly appreciated.
Thanks,
Adam A
--
:: Adam Arrowood :: adam.arrowood at oit.gatech.edu
:: Office of Information Technology/A&I
:: Georgia Institute of Technology, Atlanta, GA USA
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20131017/51151e06/attachment.html>
More information about the varnish-misc
mailing list