Cannot figure out why I am getting a miss on a static URL

Fri Jan 16 16:52:37 CET 2015

Hi,

Please try something like this :

varnishlog –c –m VCL_call:miss

This will show you requests leading to a miss. If this is always on the same URL, please try :

varnishlog –c –m RxURL:your_url

This will show what TTL is applied to your object.

Regards,
Thierry
---------------

De : varnish-misc-bounces+thierry.magnien=sfr.com at varnish-cache.org [mailto:varnish-misc-bounces+thierry.magnien=sfr.com at varnish-cache.org] De la part de Jennings III, Raymond
Envoyé : vendredi 16 janvier 2015 16:34
À : Viktor Gunnarson
Cc : varnish-misc
Objet : Re: Cannot figure out why I am getting a miss on a static URL

As far as I know there is no "-q" option on varnishlog for Varnish 3.0.6

The VCL is too complicated to post but I have added a ton of syslog conditional calls to show that the right lines and blocks in the VCL are being run.

but here is some:

sub vcl_hash {
 hash_data(req.url);
 return(hash);
}

sub vcl_fetch {
...

   if ( req.url ~ "^/my/url/.*" ) {
       set beresp.ttl = 24h;
C{
     syslog(LOG_ERR, "TTL was set to 24h");
}C
   }

   return(deliver);
}

sub vcl_deliver {

 return(deliver);
}

In my varnishncsa I print "hit" and "miss" with the option:   %{Varnish:handling}x

That is where I am seeing the miss requests.

tail -f varnishncsa.access_log | grep "\/my\/url" | grep "miss"

Sometimes I get a miss within 2 minutes, other times it can go 20 minutes.

I also have a staging system with the exact same VCL and it never happens.  So it does not appear to be something forcing a cache clearing (There re no PURGE or BAN requests) and I am currently ignoring all cache related headers. and force a return(deliver) so that the default varnish handler for vcl_fetch does not run.

Again, I see no lru_objects, I do see some lru_moved objects and I see quite a few lock operations but I do not know what they mean.  So my guess is that this problem has to do with some type of contention whether it be threads or locks.

Raymond Jennings III
nytimes.com
Office: 212.556.7786
iPhone: 914.330.5074
E-mail: Raymond.Jennings at nytimes.com
FaceTime: Raymond.Jennings at nytimes.com

On Fri, Jan 16, 2015 at 10:21 AM, Viktor Gunnarson <viktor.gunnarson at ericsson.com> wrote:
Have you tried logging the requests that get misses? Perhaps that could give some additional info.

varnishlog -q "VCL_call ~ 'MISS'"

Also you could provide the vcl because without that it’s really hard to know what happens...

Best regards,
Viktor

From: varnish-misc-bounces+viktor.gunnarson=ericsson.com at varnish-cache.org [mailto:varnish-misc-bounces+viktor.gunnarson=ericsson.com at varnish-cache.org] On Behalf Of Jennings III, Raymond
Sent: den 16 januari 2015 16:07
To: Dridi Boukelmoune
Cc: varnish-misc
Subject: Re: Cannot figure out why I am getting a miss on a static URL

varnishncsa clearly shows hits and misses within a 24 hour period despite clearly setting the TTL to 24 hours.  Varnishlog clearly shows the TTL being set to 24 hours.  I have NO lru_nuked objects as clearly shown by the varnish stats.

I'm not sure what is vague.  I am setting an object for a TTL of 24 hours yet during that 24 hour period it is still getting miss requests.  The URLs in question have no GET parameters.  I am hashing only on the URL and not the hostname.

So what does this tell me?  It tells me that my cache size is big enough.

My guess is there is some type of contention from locking of some type given other requests coming through.  There are quite a few PUT and POST requests that pass through this Varnish server.

Ummmm, I hardly think that "anyone can't help me."  I think most definitely "someone" that has some insight about the internals could shed some light on this problem.

Raymond Jennings III
nytimes.com
Office: 212.556.7786
iPhone: 914.330.5074
E-mail: Raymond.Jennings at nytimes.com
FaceTime: Raymond.Jennings at nytimes.com

On Fri, Jan 16, 2015 at 10:01 AM, Dridi Boukelmoune <dridi.boukelmoune at zenika.com> wrote:
Hi Raymond,

Have you looked at the logs?

Your description of the issue is too vague, I don't think anyone can
help you with just that.

Cheers,
Dridi

On Thu, Jan 15, 2015 at 3:36 PM, Jennings III, Raymond
<raymond.jennings at nytimes.com> wrote:
> I have a static URL that I set a TTL for 24h but yet every so many minutes I
> get a "miss" (as shown in my varnishncsa.)
>
> I purge this URL every 24 hours.
>
> 1 - The URL never has any parameters.
> 2 - I do not hash over the hostname only the URI
> 3 - Nobody is issuing any PURGE or BAN requests
> 4 - n_lru_nuked is always 0
> 5 - There are no 500 or 400 responses
>
> What is causing this URL to get a miss when everything I see looks good?  If
> the cache size has been exceeded I would have expected to see the nuked
> value be something other than 0.
>
> vcl_fetch is basically just:
>
> set beresp.ttl = 24h;
> return(deliver);
>
> Any reason why Varnish seems to be taking it upon itself to cause this miss?
> In 24 hours I get about 12000 requests and out of those 120 misses and I
> should see 2 miss requests at most.
>
> The backends are healthy.
>
> This is varnish-3.0.6-1.el6.x86_64
>
> Thanks.
>
>
>
> Raymond Jennings III
> nytimes.com
> Office: 212.556.7786
> iPhone: 914.330.5074
> E-mail: Raymond.Jennings at nytimes.com
> FaceTime: Raymond.Jennings at nytimes.com
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc