Compressed and uncompressed cached object handling
Daniel Rodriguez
coolbomb at gmail.com
Tue Nov 17 11:31:52 CET 2009
Hi guys,
I'm having a problem with a varnish implementation that we are testing
to replace an ugly appliance. We were almost ready to place our server
in a more real environment (some of our production sites), but I
found out that there is something not working properly with the
compression handling in my varnishd (varnish-2.0.5 - Debian)
Varnish its returning the first object cached no matter if i ask for a
clear object (no Accept-encoding specified) or a gzip/deflate object.
If the object cached is a gzip object it will return that no matter if
I later ask for a clear one later.
According to what I have seen in the documentation varnish should keep
both object versions (compressed and no-compressed) in the cache and
deliver the one that its asked by the client.
Step 1
I ask for a non-compressed object (no Accept-encoding specified). This
works "great"
GET -H "TE:" -sed "http://foo.bar/test/prueba.php"
200 OK
Cache-Control: max-age=20
Connection: close
Date: Mon, 16 Nov 2009 16:56:06 GMT
Via: 1.1 varnish
Age: 0
Server: Apache
Content-Length: 11013
Content-Type: text/html; charset=iso-8859-15
Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT
Client-Date: Mon, 16 Nov 2009 16:56:06 GMT
Client-Peer: 10.10.10.10:80
Client-Response-Num: 1
X-Varnish: 1655545411
The request goes like this in the log:
12 SessionOpen c 10.20.20.20 57909 :80
12 ReqStart c 10.20.20.20 57909 1655545411
12 RxRequest c GET
12 RxURL c /test/prueba.php
12 RxProtocol c HTTP/1.1
12 RxHeader c Connection: TE, close
12 RxHeader c Host: foo.bar
12 RxHeader c TE:
12 RxHeader c User-Agent: lwp-request/5.827 libwww-perl/5.831
12 VCL_call c recv
12 VCL_return c lookup
12 VCL_call c hash
12 VCL_return c hash
12 VCL_call c miss
12 VCL_return c fetch
14 BackendOpen b default 10.10.10.10 33484 10.30.30.30 80
12 Backend c 14 default default
14 TxRequest b GET
14 TxURL b /test/prueba.php
14 TxProtocol b HTTP/1.1
14 TxHeader b Host: foo.bar
14 TxHeader b User-Agent: lwp-request/5.827 libwww-perl/5.831
14 TxHeader b X-Varnish: 1655545411
14 TxHeader b X-Forwarded-For: 10.20.20.20
0 CLI - Rd ping
0 CLI - Wr 0 200 PONG 1258390564 1.0
14 RxProtocol b HTTP/1.1
14 RxStatus b 200
14 RxResponse b OK
14 RxHeader b Date: Mon, 16 Nov 2009 16:56:01 GMT
14 RxHeader b Server: Apache
14 RxHeader b Cache-control: max-age=20
14 RxHeader b Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT
14 RxHeader b Connection: close
14 RxHeader b Transfer-Encoding: chunked
14 RxHeader b Content-Type: text/html; charset=iso-8859-15
12 ObjProtocol c HTTP/1.1
12 ObjStatus c 200
12 ObjResponse c OK
12 ObjHeader c Date: Mon, 16 Nov 2009 16:56:01 GMT
12 ObjHeader c Server: Apache
12 ObjHeader c Cache-control: max-age=20
12 ObjHeader c Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT
12 ObjHeader c Content-Type: text/html; charset=iso-8859-15
14 BackendClose b default
12 TTL c 1655545411 RFC 20 1258390566 0 0 20 0
12 VCL_call c fetch
12 VCL_return c deliver
12 Length c 11013
12 VCL_call c deliver
12 VCL_return c deliver
12 TxProtocol c HTTP/1.1
12 TxStatus c 200
12 TxResponse c OK
12 TxHeader c Server: Apache
12 TxHeader c Cache-control: max-age=20
12 TxHeader c Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT
12 TxHeader c Content-Type: text/html; charset=iso-8859-15
12 TxHeader c Content-Length: 11013
12 TxHeader c Date: Mon, 16 Nov 2009 16:56:06 GMT
12 TxHeader c X-Varnish: 1655545411
12 TxHeader c Age: 0
12 TxHeader c Via: 1.1 varnish
12 TxHeader c Connection: close
12 ReqEnd c 1655545411 1258390561.316438675
1258390566.327898026 0.000134945 5.010995150 0.000464201
12 SessionClose c Connection: close
12 StatSess c 10.20.20.20 57909 5 1 1 0 0 1 282 11013
Step 2
Then the next request will go with a (Accept-encoding: gzip), and
returns me a clear object :(
GET -H "Accept-encoding: gzip" -H "TE:" -sed "http://foo.bar/test/prueba.php"
200 OK
Cache-Control: max-age=20
Connection: close
Date: Mon, 16 Nov 2009 16:56:09 GMT
Via: 1.1 varnish
Age: 3
Server: Apache
Content-Length: 11013
Content-Type: text/html; charset=iso-8859-15
Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT
Client-Date: Mon, 16 Nov 2009 16:56:08 GMT
Client-Peer: 10.10.10.10:80
Client-Response-Num: 1
X-Varnish: 1655545412 1655545411
12 SessionOpen c 10.20.20.20 57910 :80
12 ReqStart c 10.20.20.20 57910 1655545412
12 RxRequest c GET
12 RxURL c /test/prueba.php
12 RxProtocol c HTTP/1.1
12 RxHeader c Connection: TE, close
12 RxHeader c Accept-Encoding: gzip
12 RxHeader c Host: foo.bar
12 RxHeader c TE:
12 RxHeader c User-Agent: lwp-request/5.827 libwww-perl/5.831
12 VCL_call c recv
12 VCL_return c lookup
12 VCL_call c hash
12 VCL_return c hash
12 Hit c 1655545411
12 VCL_call c hit
12 VCL_return c deliver
12 Length c 11013
12 VCL_call c deliver
12 VCL_return c deliver
12 TxProtocol c HTTP/1.1
12 TxStatus c 200
12 TxResponse c OK
12 TxHeader c Server: Apache
12 TxHeader c Cache-control: max-age=20
12 TxHeader c Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT
12 TxHeader c Content-Type: text/html; charset=iso-8859-15
12 TxHeader c Content-Length: 11013
12 TxHeader c Date: Mon, 16 Nov 2009 16:56:09 GMT
12 TxHeader c X-Varnish: 1655545412 1655545411
12 TxHeader c Age: 3
12 TxHeader c Via: 1.1 varnish
12 TxHeader c Connection: close
12 ReqEnd c 1655545412 1258390569.036545277
1258390569.036923647 0.000121355 0.000098705 0.000279665
12 SessionClose c Connection: close
12 StatSess c 10.20.20.20 57910 0 1 1 0 0 0 293 11013
My config:
backend default {
.host = "10.30.30.30";
.port = "80";
}
acl purge {
"localhost";
}
sub vcl_recv {
set req.grace = 2m;
if (req.http.Accept-Encoding) {
if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
# No se comprimen estos, no tiene logica
remove req.http.Accept-Encoding;
} elsif (req.http.Accept-Encoding ~ "gzip") {
set req.http.Accept-Encoding = "gzip";
} elsif (req.http.Accept-Encoding ~ "deflate") {
set req.http.Accept-Encoding = "deflate";
} else {
# No se conoce el algoritmo
remove req.http.Accept-Encoding;
}
}
if (req.request == "PURGE") {
if (!client.ip ~ purge) {
error 405 "Not allowed.";
}
purge("req.url == " req.url);
}
if (req.http.Authorization) {
return (pass);
}
return(lookup);
}
sub vcl_fetch {
set obj.grace = 2m;
if(obj.http.Pragma ~ "no-cache" || obj.http.Cache-Control ~
"no-cache" || obj.http.Cache-Control ~ "private" ||
obj.http.Cache-Control ~ "max-age=0" || obj.http.Cache-Control ~
"must-revalidate" || obj.http.Cache-Control ~ "private" ) {
pass;
}
if (!obj.cacheable) {
return (pass);
}
if (obj.http.Set-Cookie) {
return (pass);
}
set obj.prefetch = -3s;
if (req.http.Authorization && !obj.http.Cache-Control ~ "public") {
pass;
}
return (deliver);
}
I'm doing something wrong?...
Best Regards,
More information about the varnish-misc
mailing list