Compressed and uncompressed cached object handling
Michael S. Fischer
michael at dynamine.net
Tue Nov 17 11:42:36 CET 2009
Are you returning a "Vary: Accept-Encoding" in your origin server's
response headers?
--Michael
On Nov 17, 2009, at 4:01 PM, Daniel Rodriguez wrote:
> Hi guys,
>
> I'm having a problem with a varnish implementation that we are testing
> to replace an ugly appliance. We were almost ready to place our server
> in a more real environment (some of our production sites), but I
> found out that there is something not working properly with the
> compression handling in my varnishd (varnish-2.0.5 - Debian)
>
> Varnish its returning the first object cached no matter if i ask for a
> clear object (no Accept-encoding specified) or a gzip/deflate object.
> If the object cached is a gzip object it will return that no matter if
> I later ask for a clear one later.
>
> According to what I have seen in the documentation varnish should keep
> both object versions (compressed and no-compressed) in the cache and
> deliver the one that its asked by the client.
>
> Step 1
>
> I ask for a non-compressed object (no Accept-encoding specified). This
> works "great"
>
> GET -H "TE:" -sed "http://foo.bar/test/prueba.php"
> 200 OK
> Cache-Control: max-age=20
> Connection: close
> Date: Mon, 16 Nov 2009 16:56:06 GMT
> Via: 1.1 varnish
> Age: 0
> Server: Apache
> Content-Length: 11013
> Content-Type: text/html; charset=iso-8859-15
> Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT
> Client-Date: Mon, 16 Nov 2009 16:56:06 GMT
> Client-Peer: 10.10.10.10:80
> Client-Response-Num: 1
> X-Varnish: 1655545411
>
> The request goes like this in the log:
>
> 12 SessionOpen c 10.20.20.20 57909 :80
> 12 ReqStart c 10.20.20.20 57909 1655545411
> 12 RxRequest c GET
> 12 RxURL c /test/prueba.php
> 12 RxProtocol c HTTP/1.1
> 12 RxHeader c Connection: TE, close
> 12 RxHeader c Host: foo.bar
> 12 RxHeader c TE:
> 12 RxHeader c User-Agent: lwp-request/5.827 libwww-perl/5.831
> 12 VCL_call c recv
> 12 VCL_return c lookup
> 12 VCL_call c hash
> 12 VCL_return c hash
> 12 VCL_call c miss
> 12 VCL_return c fetch
> 14 BackendOpen b default 10.10.10.10 33484 10.30.30.30 80
> 12 Backend c 14 default default
> 14 TxRequest b GET
> 14 TxURL b /test/prueba.php
> 14 TxProtocol b HTTP/1.1
> 14 TxHeader b Host: foo.bar
> 14 TxHeader b User-Agent: lwp-request/5.827 libwww-perl/5.831
> 14 TxHeader b X-Varnish: 1655545411
> 14 TxHeader b X-Forwarded-For: 10.20.20.20
> 0 CLI - Rd ping
> 0 CLI - Wr 0 200 PONG 1258390564 1.0
> 14 RxProtocol b HTTP/1.1
> 14 RxStatus b 200
> 14 RxResponse b OK
> 14 RxHeader b Date: Mon, 16 Nov 2009 16:56:01 GMT
> 14 RxHeader b Server: Apache
> 14 RxHeader b Cache-control: max-age=20
> 14 RxHeader b Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT
> 14 RxHeader b Connection: close
> 14 RxHeader b Transfer-Encoding: chunked
> 14 RxHeader b Content-Type: text/html; charset=iso-8859-15
> 12 ObjProtocol c HTTP/1.1
> 12 ObjStatus c 200
> 12 ObjResponse c OK
> 12 ObjHeader c Date: Mon, 16 Nov 2009 16:56:01 GMT
> 12 ObjHeader c Server: Apache
> 12 ObjHeader c Cache-control: max-age=20
> 12 ObjHeader c Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT
> 12 ObjHeader c Content-Type: text/html; charset=iso-8859-15
> 14 BackendClose b default
> 12 TTL c 1655545411 RFC 20 1258390566 0 0 20 0
> 12 VCL_call c fetch
> 12 VCL_return c deliver
> 12 Length c 11013
> 12 VCL_call c deliver
> 12 VCL_return c deliver
> 12 TxProtocol c HTTP/1.1
> 12 TxStatus c 200
> 12 TxResponse c OK
> 12 TxHeader c Server: Apache
> 12 TxHeader c Cache-control: max-age=20
> 12 TxHeader c Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT
> 12 TxHeader c Content-Type: text/html; charset=iso-8859-15
> 12 TxHeader c Content-Length: 11013
> 12 TxHeader c Date: Mon, 16 Nov 2009 16:56:06 GMT
> 12 TxHeader c X-Varnish: 1655545411
> 12 TxHeader c Age: 0
> 12 TxHeader c Via: 1.1 varnish
> 12 TxHeader c Connection: close
> 12 ReqEnd c 1655545411 1258390561.316438675
> 1258390566.327898026 0.000134945 5.010995150 0.000464201
> 12 SessionClose c Connection: close
> 12 StatSess c 10.20.20.20 57909 5 1 1 0 0 1 282 11013
>
> Step 2
>
> Then the next request will go with a (Accept-encoding: gzip), and
> returns me a clear object :(
>
> GET -H "Accept-encoding: gzip" -H "TE:" -sed "http://foo.bar/test/prueba.php
> "
> 200 OK
> Cache-Control: max-age=20
> Connection: close
> Date: Mon, 16 Nov 2009 16:56:09 GMT
> Via: 1.1 varnish
> Age: 3
> Server: Apache
> Content-Length: 11013
> Content-Type: text/html; charset=iso-8859-15
> Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT
> Client-Date: Mon, 16 Nov 2009 16:56:08 GMT
> Client-Peer: 10.10.10.10:80
> Client-Response-Num: 1
> X-Varnish: 1655545412 1655545411
>
> 12 SessionOpen c 10.20.20.20 57910 :80
> 12 ReqStart c 10.20.20.20 57910 1655545412
> 12 RxRequest c GET
> 12 RxURL c /test/prueba.php
> 12 RxProtocol c HTTP/1.1
> 12 RxHeader c Connection: TE, close
> 12 RxHeader c Accept-Encoding: gzip
> 12 RxHeader c Host: foo.bar
> 12 RxHeader c TE:
> 12 RxHeader c User-Agent: lwp-request/5.827 libwww-perl/5.831
> 12 VCL_call c recv
> 12 VCL_return c lookup
> 12 VCL_call c hash
> 12 VCL_return c hash
> 12 Hit c 1655545411
> 12 VCL_call c hit
> 12 VCL_return c deliver
> 12 Length c 11013
> 12 VCL_call c deliver
> 12 VCL_return c deliver
> 12 TxProtocol c HTTP/1.1
> 12 TxStatus c 200
> 12 TxResponse c OK
> 12 TxHeader c Server: Apache
> 12 TxHeader c Cache-control: max-age=20
> 12 TxHeader c Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT
> 12 TxHeader c Content-Type: text/html; charset=iso-8859-15
> 12 TxHeader c Content-Length: 11013
> 12 TxHeader c Date: Mon, 16 Nov 2009 16:56:09 GMT
> 12 TxHeader c X-Varnish: 1655545412 1655545411
> 12 TxHeader c Age: 3
> 12 TxHeader c Via: 1.1 varnish
> 12 TxHeader c Connection: close
> 12 ReqEnd c 1655545412 1258390569.036545277
> 1258390569.036923647 0.000121355 0.000098705 0.000279665
> 12 SessionClose c Connection: close
> 12 StatSess c 10.20.20.20 57910 0 1 1 0 0 0 293 11013
>
> My config:
>
> backend default {
> .host = "10.30.30.30";
> .port = "80";
> }
>
> acl purge {
> "localhost";
> }
>
> sub vcl_recv {
> set req.grace = 2m;
> if (req.http.Accept-Encoding) {
> if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
> # No se comprimen estos, no tiene logica
> remove req.http.Accept-Encoding;
> } elsif (req.http.Accept-Encoding ~ "gzip") {
> set req.http.Accept-Encoding = "gzip";
> } elsif (req.http.Accept-Encoding ~ "deflate") {
> set req.http.Accept-Encoding = "deflate";
> } else {
> # No se conoce el algoritmo
> remove req.http.Accept-Encoding;
> }
> }
> if (req.request == "PURGE") {
> if (!client.ip ~ purge) {
> error 405 "Not allowed.";
> }
> purge("req.url == " req.url);
> }
> if (req.http.Authorization) {
> return (pass);
> }
> return(lookup);
> }
>
> sub vcl_fetch {
> set obj.grace = 2m;
> if(obj.http.Pragma ~ "no-cache" || obj.http.Cache-Control ~
> "no-cache" || obj.http.Cache-Control ~ "private" ||
> obj.http.Cache-Control ~ "max-age=0" || obj.http.Cache-Control ~
> "must-revalidate" || obj.http.Cache-Control ~ "private" ) {
> pass;
> }
> if (!obj.cacheable) {
> return (pass);
> }
> if (obj.http.Set-Cookie) {
> return (pass);
> }
> set obj.prefetch = -3s;
> if (req.http.Authorization && !obj.http.Cache-Control ~ "public") {
> pass;
> }
> return (deliver);
> }
>
> I'm doing something wrong?...
>
> Best Regards,
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc
More information about the varnish-misc
mailing list