Vary Header Strange Behavior

[Jeff Anderson]

The latest version of Varnish seems to behave unexpectedly with regard  
to the Vary on Accept-Encoding headers.

Our production servers report back different headers than our  
development and test varnish servers even though they are identical  
VCL. In production it serves up uncompressed versions no matter what  
version is requested.  The vary header doesn't even show up like it  
does on the development severs.


$ curl http://prodweb9 --head  --compressed

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
X-Runtime: 0.58210
ETag: "359535e914cddcc7ce0c879f1b3d4668"
Cache-Control: no-cache, public, max-age=300
Server: LiteSpeed
Content-Length: 60711
Date: Mon, 01 Dec 2008 23:10:47 GMT
X-Varnish: 1013927736 1013926899
Age: 59
Via: 1.1 varnish
Served-by: prodweb9/prodapp7
Connection: close


$ curl http://prodweb9 --head

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
X-Runtime: 0.58210
ETag: "359535e914cddcc7ce0c879f1b3d4668"
Cache-Control: no-cache, public, max-age=300
Server: LiteSpeed
Content-Length: 60711
Date: Mon, 01 Dec 2008 23:10:56 GMT
X-Varnish: 1013927873 1013926899
Age: 69
Via: 1.1 varnish
Served-by: prodweb9/prodapp7
Connection: close



However on our development servers:

$ curl http://devweb1 --head

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
ETag: "4ebc8c72bde3a68dc965f52d392ac630"
X-Runtime: 0.12171
Cache-Control: no-cache, public, max-age=300
Vary: Accept-Encoding
X-Varnish: 191681248
Age: 0
Via: 1.1 varnish
Served-by: devweb1/devapp1
Date: Mon, 01 Dec 2008 23:04:15 GMT
Server: LiteSpeed
Connection: Keep-Alive
Keep-Alive: timeout=5, max=100
WWW-Authenticate: Basic realm="Protected Area"

$ curl http://devweb1 --head  --compressed

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
ETag: "4ebc8c72bde3a68dc965f52d392ac630"
X-Runtime: 0.12171
Cache-Control: no-cache, public, max-age=300
Content-Encoding: gzip
Vary: Accept-Encoding
X-Varnish: 191681255 191681248
Age: 30
Via: 1.1 varnish
Served-by: devweb1/devapp1
Date: Mon, 01 Dec 2008 23:04:45 GMT
Server: LiteSpeed
Connection: Keep-Alive
Keep-Alive: timeout=5, max=100
WWW-Authenticate: Basic realm="Protected Area"


The lines below were added to the VCL to resolve the problem.  And  
while they did resolve the issue it seems to have made varnish double  
the look up of incoming requests according to the varnishncsa log.

sub vcl_hash {
   if (req.http.Accept-Encoding ~ "gzip" || req.http.Accept-Encoding ~  
"deflate") {
     set req.hash += req.http.Accept-Encoding;
   }


Is this normal or is something overriding the proper functioning of  
varnish?

Thanks,
--Jeff