Compressed and uncompressed cached object handling

Daniel Rodriguez coolbomb at gmail.com
Tue Nov 17 12:24:03 CET 2009


Hi Michael,

That was the problem, the server was not returning the "Vary:
Accept-Encoding", I didn't notice that detail in the headers sent by
the server. Its working perfect right now.

Thank you very much.
Regards,

On Tue, Nov 17, 2009 at 11:42 AM, Michael S. Fischer
<michael at dynamine.net> wrote:
> Are you returning a "Vary: Accept-Encoding" in your origin server's response
> headers?
>
> --Michael
>
> On Nov 17, 2009, at 4:01 PM, Daniel Rodriguez wrote:
>
>> Hi guys,
>>
>> I'm having a problem with a varnish implementation that we are testing
>> to replace an ugly appliance. We were almost ready to place our server
>> in a more real  environment (some of our production sites), but I
>> found out that there is something not working properly with the
>> compression handling in my varnishd (varnish-2.0.5 - Debian)
>>
>> Varnish its returning the first object cached no matter if i ask for a
>> clear object (no Accept-encoding specified) or a gzip/deflate object.
>> If the object cached is a gzip object it will return that no matter if
>> I later ask for a clear one later.
>>
>> According to what I have seen in the documentation varnish should keep
>> both object versions (compressed and no-compressed) in the cache and
>> deliver the one that its asked by the client.
>>
>> Step 1
>>
>> I ask for a non-compressed object (no Accept-encoding specified). This
>> works "great"
>>
>> GET -H "TE:" -sed "http://foo.bar/test/prueba.php"
>> 200 OK
>> Cache-Control: max-age=20
>> Connection: close
>> Date: Mon, 16 Nov 2009 16:56:06 GMT
>> Via: 1.1 varnish
>> Age: 0
>> Server: Apache
>> Content-Length: 11013
>> Content-Type: text/html; charset=iso-8859-15
>> Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT
>> Client-Date: Mon, 16 Nov 2009 16:56:06 GMT
>> Client-Peer: 10.10.10.10:80
>> Client-Response-Num: 1
>> X-Varnish: 1655545411
>>
>> The request goes like this in the log:
>>
>>  12 SessionOpen  c 10.20.20.20 57909 :80
>>    12 ReqStart     c 10.20.20.20 57909 1655545411
>>    12 RxRequest    c GET
>>    12 RxURL        c /test/prueba.php
>>    12 RxProtocol   c HTTP/1.1
>>    12 RxHeader     c Connection: TE, close
>>    12 RxHeader     c Host: foo.bar
>>    12 RxHeader     c TE:
>>    12 RxHeader     c User-Agent: lwp-request/5.827 libwww-perl/5.831
>>    12 VCL_call     c recv
>>    12 VCL_return   c lookup
>>    12 VCL_call     c hash
>>    12 VCL_return   c hash
>>    12 VCL_call     c miss
>>    12 VCL_return   c fetch
>>    14 BackendOpen  b default 10.10.10.10 33484 10.30.30.30 80
>>    12 Backend      c 14 default default
>>    14 TxRequest    b GET
>>    14 TxURL        b /test/prueba.php
>>    14 TxProtocol   b HTTP/1.1
>>    14 TxHeader     b Host: foo.bar
>>    14 TxHeader     b User-Agent: lwp-request/5.827 libwww-perl/5.831
>>    14 TxHeader     b X-Varnish: 1655545411
>>    14 TxHeader     b X-Forwarded-For: 10.20.20.20
>>     0 CLI          - Rd ping
>>     0 CLI          - Wr 0 200 PONG 1258390564 1.0
>>    14 RxProtocol   b HTTP/1.1
>>    14 RxStatus     b 200
>>    14 RxResponse   b OK
>>    14 RxHeader     b Date: Mon, 16 Nov 2009 16:56:01 GMT
>>    14 RxHeader     b Server: Apache
>>    14 RxHeader     b Cache-control: max-age=20
>>    14 RxHeader     b Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT
>>    14 RxHeader     b Connection: close
>>    14 RxHeader     b Transfer-Encoding: chunked
>>    14 RxHeader     b Content-Type: text/html; charset=iso-8859-15
>>    12 ObjProtocol  c HTTP/1.1
>>    12 ObjStatus    c 200
>>    12 ObjResponse  c OK
>>    12 ObjHeader    c Date: Mon, 16 Nov 2009 16:56:01 GMT
>>    12 ObjHeader    c Server: Apache
>>    12 ObjHeader    c Cache-control: max-age=20
>>    12 ObjHeader    c Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT
>>    12 ObjHeader    c Content-Type: text/html; charset=iso-8859-15
>>    14 BackendClose b default
>>    12 TTL          c 1655545411 RFC 20 1258390566 0 0 20 0
>>    12 VCL_call     c fetch
>>    12 VCL_return   c deliver
>>    12 Length       c 11013
>>    12 VCL_call     c deliver
>>    12 VCL_return   c deliver
>>    12 TxProtocol   c HTTP/1.1
>>    12 TxStatus     c 200
>>    12 TxResponse   c OK
>>    12 TxHeader     c Server: Apache
>>    12 TxHeader     c Cache-control: max-age=20
>>    12 TxHeader     c Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT
>>    12 TxHeader     c Content-Type: text/html; charset=iso-8859-15
>>    12 TxHeader     c Content-Length: 11013
>>    12 TxHeader     c Date: Mon, 16 Nov 2009 16:56:06 GMT
>>    12 TxHeader     c X-Varnish: 1655545411
>>    12 TxHeader     c Age: 0
>>    12 TxHeader     c Via: 1.1 varnish
>>    12 TxHeader     c Connection: close
>>    12 ReqEnd       c 1655545411 1258390561.316438675
>> 1258390566.327898026 0.000134945 5.010995150 0.000464201
>>    12 SessionClose c Connection: close
>>    12 StatSess     c 10.20.20.20 57909 5 1 1 0 0 1 282 11013
>>
>> Step 2
>>
>> Then the next request will go with a (Accept-encoding: gzip), and
>> returns me a clear object :(
>>
>> GET -H "Accept-encoding: gzip" -H "TE:" -sed
>> "http://foo.bar/test/prueba.php"
>> 200 OK
>> Cache-Control: max-age=20
>> Connection: close
>> Date: Mon, 16 Nov 2009 16:56:09 GMT
>> Via: 1.1 varnish
>> Age: 3
>> Server: Apache
>> Content-Length: 11013
>> Content-Type: text/html; charset=iso-8859-15
>> Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT
>> Client-Date: Mon, 16 Nov 2009 16:56:08 GMT
>> Client-Peer: 10.10.10.10:80
>> Client-Response-Num: 1
>> X-Varnish: 1655545412 1655545411
>>
>>   12 SessionOpen  c 10.20.20.20 57910 :80
>>   12 ReqStart     c 10.20.20.20 57910 1655545412
>>   12 RxRequest    c GET
>>   12 RxURL        c /test/prueba.php
>>   12 RxProtocol   c HTTP/1.1
>>   12 RxHeader     c Connection: TE, close
>>   12 RxHeader     c Accept-Encoding: gzip
>>   12 RxHeader     c Host: foo.bar
>>   12 RxHeader     c TE:
>>   12 RxHeader     c User-Agent: lwp-request/5.827 libwww-perl/5.831
>>   12 VCL_call     c recv
>>   12 VCL_return   c lookup
>>   12 VCL_call     c hash
>>   12 VCL_return   c hash
>>   12 Hit          c 1655545411
>>   12 VCL_call     c hit
>>   12 VCL_return   c deliver
>>   12 Length       c 11013
>>   12 VCL_call     c deliver
>>   12 VCL_return   c deliver
>>   12 TxProtocol   c HTTP/1.1
>>   12 TxStatus     c 200
>>   12 TxResponse   c OK
>>   12 TxHeader     c Server: Apache
>>   12 TxHeader     c Cache-control: max-age=20
>>   12 TxHeader     c Last-Modified: Mon, 16 Nov 2009 16:56:06 GMT
>>   12 TxHeader     c Content-Type: text/html; charset=iso-8859-15
>>   12 TxHeader     c Content-Length: 11013
>>   12 TxHeader     c Date: Mon, 16 Nov 2009 16:56:09 GMT
>>   12 TxHeader     c X-Varnish: 1655545412 1655545411
>>   12 TxHeader     c Age: 3
>>   12 TxHeader     c Via: 1.1 varnish
>>   12 TxHeader     c Connection: close
>>   12 ReqEnd       c 1655545412 1258390569.036545277
>> 1258390569.036923647 0.000121355 0.000098705 0.000279665
>>   12 SessionClose c Connection: close
>>   12 StatSess     c 10.20.20.20 57910 0 1 1 0 0 0 293 11013
>>
>> My config:
>>
>> backend default {
>>       .host = "10.30.30.30";
>>        .port = "80";
>> }
>>
>> acl purge {
>>       "localhost";
>> }
>>
>> sub vcl_recv {
>>  set req.grace = 2m;
>>  if (req.http.Accept-Encoding) {
>>     if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
>>        # No se comprimen estos, no tiene logica
>>           remove req.http.Accept-Encoding;
>>     } elsif (req.http.Accept-Encoding ~ "gzip") {
>>        set req.http.Accept-Encoding = "gzip";
>>     } elsif (req.http.Accept-Encoding ~ "deflate") {
>>        set req.http.Accept-Encoding = "deflate";
>>     } else {
>>        # No se conoce el algoritmo
>>       remove req.http.Accept-Encoding;
>>     }
>>  }
>>  if (req.request == "PURGE") {
>>     if (!client.ip ~ purge) {
>>        error 405 "Not allowed.";
>>     }
>>     purge("req.url == " req.url);
>>  }
>>  if (req.http.Authorization) {
>>      return (pass);
>>  }
>>  return(lookup);
>> }
>>
>> sub vcl_fetch {
>>   set obj.grace = 2m;
>>   if(obj.http.Pragma ~ "no-cache" || obj.http.Cache-Control ~
>> "no-cache" || obj.http.Cache-Control ~ "private" ||
>> obj.http.Cache-Control ~ "max-age=0" || obj.http.Cache-Control ~
>> "must-revalidate" || obj.http.Cache-Control ~ "private" ) {
>>      pass;
>>   }
>>   if (!obj.cacheable) {
>>       return (pass);
>>   }
>>   if (obj.http.Set-Cookie) {
>>       return (pass);
>>   }
>>   set obj.prefetch =  -3s;
>>   if (req.http.Authorization && !obj.http.Cache-Control ~ "public") {
>>      pass;
>>   }
>>   return (deliver);
>> }
>>
>> I'm doing something wrong?...
>>
>> Best Regards,
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at projects.linpro.no
>> http://projects.linpro.no/mailman/listinfo/varnish-misc
>
>



More information about the varnish-misc mailing list