Cannot figure out why, sometimes, I am getting a miss
MAGNIEN, Thierry
thierry.magnien at sfr.com
Fri Sep 4 16:21:28 CEST 2015
Hi,
Your Apache sends a "Vary: Accept-Encoding,User-Agent" header, which means you cache a different object for each User-Agent string.
The two requests having a different user agent, you get a miss on the second one even if the first is in cache :
User-Agent: Hippique/2.9.2 (iPad; iOS 8.2; Scale/2.00)
and
User-Agent: Hippique/2.9.2 (iPad; iOS 7.0.6; Scale/1.00)
Best regards,
Thierry
-----------------------------
De : varnish-misc-bounces+thierry.magnien=sfr.com at varnish-cache.org [mailto:varnish-misc-bounces+thierry.magnien=sfr.com at varnish-cache.org] De la part de Parmentier Benjamin
Envoyé : vendredi 4 septembre 2015 15:51
À : varnish-misc at varnish-cache.org
Objet : Cannot figure out why, sometimes, I am getting a miss
Hi,
I have a varnish 3.0.6 in-between a F5 Big-IP and an apache backend :
F5 Big-IP -> varnish -> apache
I have a request that I set a TTL for 24h but I sometimes get miss on this URL even if the TTL is not reached and I don't know what is causing this.
Extract from varnishncsa :
78.195.19.79 - - [04/Sep/2015:14:58:20 +0200] "GET http://www.mydomain.tld/services/info/client/16/sdm/config? HTTP/1.1" 200 38 "-" "Info/2.9.2 (iPad; iOS 8.2; Scale/2.00)" hit 134
77.152.91.156 - - [04/Sep/2015:14:59:22 +0200] "GET http://www.mydomain.tld/services/info/client/16/sdm/config? HTTP/1.1" 200 38 "-" "Info/2.9.2 (iPad; iOS 7.0.6; Scale/1.00)" miss 2685
The first request hit the cache :
124 ReqStart c 10.98.40.11 61080 349021016
124 RxRequest c GET
124 RxURL c /services/info/client/16/sdm/config?
124 RxProtocol c HTTP/1.1
124 RxHeader c Host: www.mydomain.tld
124 RxHeader c Accept: */*
124 RxHeader c Connection: keep-alive
124 RxHeader c Cookie: UID=69740d-185.43.182.11-1426874236; UIDR=1426874236; s1=550C600D31A40171; s1=550C60007096022A; ai_session=92dd6dfb13354c93ada5cf9abb619e32|2015-07-02T11:35:24.6019731+00:00|
2015-07-02T11:35:24.9926130+00:00; ai_user=e271bdbc381d4212a07bc0b02e0e32
124 RxHeader c User-Agent: Hippique/2.9.2 (iPad; iOS 8.2; Scale/2.00)
124 RxHeader c Accept-Language: fr;q=1
124 RxHeader c Info-Session-Id: 72440dc7e08b268927e6cbe9f3842b70
124 RxHeader c Accept-Encoding: gzip, deflate
124 RxHeader c X-Forwarded-For: 78.195.19.79
124 VCL_call c recv lookup
124 VCL_call c hash
124 Hash c /services/info/client/16/sdm/config
124 Hash c www.mydomain.tld
124 VCL_return c hash
124 Hit c 346131254
124 VCL_call c hit deliver
124 VCL_call c deliver deliver
124 TxProtocol c HTTP/1.1
124 TxStatus c 200
124 TxResponse c OK
124 TxHeader c Server: Apache
124 TxHeader c Vary: Accept-Encoding,User-Agent
124 TxHeader c Content-Encoding: gzip
124 TxHeader c Content-Type: application/json;charset=UTF-8
124 TxHeader c Content-Length: 38
124 TxHeader c Accept-Ranges: bytes
124 TxHeader c Date: Fri, 04 Sep 2015 12:59:21 GMT
124 TxHeader c X-Varnish: 349021016 346131254
124 TxHeader c Age: 82715
124 TxHeader c Via: 1.1 varnish
124 TxHeader c Connection: keep-alive
124 TxHeader c Var-Cache: HIT
124 Length c 38
124 ReqEnd c 349021016 1441371561.449951649 1441371561.450107813 0.000079632 0.000083208 0.000072956
One minute later the second request miss the cache and fetch the backend :
53 SessionOpen c 10.98.40.12 24242 0.0.0.0:84
53 ReqStart c 10.98.40.12 24242 349021058
53 RxRequest c GET
53 RxURL c /services/info/client/16/sdm/config?
53 RxProtocol c HTTP/1.1
53 RxHeader c Host: www.mydomain.tld
53 RxHeader c Accept: */*
53 RxHeader c Accept-Encoding: gzip, deflate
53 RxHeader c Accept-Language: fr;q=1, en;q=0.9, de;q=0.8, ja;q=0.7, nl;q=0.6, it;q=0.5
53 RxHeader c Connection: keep-alive
53 RxHeader c Cookie: matchfbx=1; wfivefivec=scx3J3CkSI312SXL3Fo06SRLnunHpCoS; ptc=%7b%22UserSyncItems%22%3a%5b%7b%22PartnerId%22%3a%2229%22%2c%22PartnerUserId%22%3anull%2c%22TimeStamp%22%3a%2201%2f21%2f2015+12%3a49%3a20%22%7d%5d%7d; tidal_ttid=1a27cbd8-4429-44f2-9f31-
53 RxHeader c User-Agent: Hippique/2.9.2 (iPad; iOS 7.0.6; Scale/1.00)
53 RxHeader c X-Forwarded-For: 77.152.91.156
53 VCL_call c recv lookup
53 VCL_call c hash
53 Hash c /services/info/client/16/sdm/config
53 Hash c www.mydomain.tld
53 VCL_return c hash
53 VCL_call c miss fetch
53 Backend c 45 info_varnish APACHE_BACKEND
53 TTL c 349021058 RFC 120 -1 -1 1441371562 0 1441371562 0 0
53 VCL_call c fetch
53 TTL c 349021058 VCL 86400 -1 -1 1441371562 -0
53 VCL_return c deliver
53 ObjProtocol c HTTP/1.1
53 ObjResponse c OK
53 ObjHeader c Date: Fri, 04 Sep 2015 12:59:22 GMT
53 ObjHeader c Server: Apache
53 ObjHeader c Vary: Accept-Encoding,User-Agent
53 ObjHeader c Content-Encoding: gzip
53 ObjHeader c Content-Type: application/json;charset=UTF-8
53 Gzip c u F - 38 18 80 80 234
53 VCL_call c deliver deliver
53 TxProtocol c HTTP/1.1
53 TxStatus c 200
53 TxResponse c OK
53 TxHeader c Server: Apache
53 TxHeader c Vary: Accept-Encoding,User-Agent
53 TxHeader c Content-Encoding: gzip
53 TxHeader c Content-Type: application/json;charset=UTF-8
53 TxHeader c Content-Length: 38
53 TxHeader c Accept-Ranges: bytes
53 TxHeader c Date: Fri, 04 Sep 2015 12:59:22 GMT
53 TxHeader c X-Varnish: 349021058
53 TxHeader c Age: 0
53 TxHeader c Via: 1.1 varnish
53 TxHeader c Connection: keep-alive
53 TxHeader c Var-Cache: MISS
53 Length c 38
53 ReqEnd c 349021058 1441371562.030749321 1441371562.033434391 0.000071526 0.002643108 0.000041962
Here my vcl config :
backend APACHE_BACKEND {
.host = "127.0.0.1";
.port = "80";
.probe = {
.url = "/surveillance/.surveillance.txt";
.interval = 10s;
.timeout = 1s;
.window = 5;
.threshold = 3;
}
}
director info_varnish random {
{ .backend = APACHE_BACKEND ;
.weight = 1 ;
}
}
sub normalize_req_url {
# strip out "_" querystring values so varnish does not vary cache.
set req.url = regsuball(req.url, "([\?|&])_=[^&\s]*&?", "\1");
# get rid of trailing & or ?
set req.url = regsuball(req.url, "[\?|&]+$", "");
}
sub vcl_recv {
set req.backend = info_varnish;
if ( req.url ~ "^.*/info/client/\d+/programme[s]?/.*/part.*$" ||
req.url ~ "^.*/info/client/\d+/programme[s]?/\d{8,8}/R\d{1,2}/C\d{1,2}.*$" ||
req.url ~ "^.*/info/client/\d+/multimedia/videos/streaming.*$" ||
req.url ~ "^.*/info/client/\d+/sdm/config.*$" ) {
call normalize_req_url;
return(lookup);
}
return(pass);
}
sub vcl_fetch {
if ( req.url ~ "^.*/info/client/\d+/programme[s]?/.*/participants.*$" ) {
set beresp.ttl = 5s;
} else if ( req.url ~ "^.*/info/client/\d+/programme[s]?/\d{8,8}/R\d{1,2}/C\d{1,2}.*$" ) {
set beresp.ttl = 900s;
} else if ( req.url ~ "^.*/info/client/\d+/multimedia/videos/streaming.*$" ) {
set beresp.ttl = 60s;
} else if ( req.url ~ "^.*/info/client/\d+/sdm/config.*$" ) {
set beresp.ttl = 86400s;
}
return(deliver);
}
sub vcl_deliver {
# Set a header to track a cache HIT/MISS.
if (obj.hits > 0) {
set resp.http.Var-Cache = "HIT";
} else {
set resp.http.Var-Cache = "MISS";
}
}
Any ideas ?
Regards,
Benjamin
________________________________________
Ce message et les pièces jointes sont confidentiels et réservés à l'usage exclusif de ses destinataires. Il peut également être protégé par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir immédiatement l'expéditeur et de le détruire. L'intégrité du message ne pouvant être assurée sur Internet, la responsabilité de Worldline ne pourra être recherchée quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'expéditeur ne donne aucune garantie à cet égard et sa responsabilité ne saurait être recherchée pour tout dommage résultant d'un virus transmis.
This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Worldline liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted.
More information about the varnish-misc
mailing list