Cannot figure out why I am getting a miss on a static URL

[Jennings III, Raymond]

As far as I know there is no "-q" option on varnishlog for Varnish 3.0.6

The VCL is too complicated to post but I have added a ton of syslog
conditional calls to show that the right lines and blocks in the VCL are
being run.

but here is some:

sub vcl_hash {
 hash_data(req.url);
 return(hash);
}


sub vcl_fetch {
...

   if ( req.url ~ "^/my/url/.*" ) {
       set beresp.ttl = 24h;
C{
     syslog(LOG_ERR, "TTL was set to 24h");
}C
   }

   return(deliver);
}


sub vcl_deliver {

 return(deliver);
}


In my varnishncsa I print "hit" and "miss" with the option:
  %{Varnish:handling}x

That is where I am seeing the miss requests.


tail -f varnishncsa.access_log | grep "\/my\/url" | grep "miss"


Sometimes I get a miss within 2 minutes, other times it can go 20 minutes.


I also have a staging system with the exact same VCL and it never happens.
So it does not appear to be something forcing a cache clearing (There re no
PURGE or BAN requests) and I am currently ignoring all cache related
headers. and force a return(deliver) so that the default varnish handler
for vcl_fetch does not run.


Again, I see no lru_objects, I do see some lru_moved objects and I see
quite a few lock operations but I do not know what they mean.  So my guess
is that this problem has to do with some type of contention whether it be
threads or locks.



Raymond Jennings III
*nytimes.com <http://nytimes.com/>*
*Office: 212.556.7786 <212-556-7786>*

*iPhone: 914.330.5074 <914-330-5074>E-mail: Raymond.Jennings at nytimes.com
<Raymond.Jennings at nytimes.com>FaceTime: Raymond.Jennings at nytimes.com
<Raymond.Jennings at nytimes.com>*

On Fri, Jan 16, 2015 at 10:21 AM, Viktor Gunnarson <
viktor.gunnarson at ericsson.com> wrote:

>  Have you tried logging the requests that get misses? Perhaps that could
> give some additional info.
>
>
>
> varnishlog -q "VCL_call ~ 'MISS'"
>
>
>
> Also you could provide the vcl because without that it’s really hard to
> know what happens...
>
>
>
> Best regards,
>
> Viktor
>
>
>
>
>
> *From:* varnish-misc-bounces+viktor.gunnarson=
> ericsson.com at varnish-cache.org [mailto:
> varnish-misc-bounces+viktor.gunnarson=ericsson.com at varnish-cache.org] *
> On Behalf Of *Jennings III, Raymond
> *Sent:* den 16 januari 2015 16:07
> *To:* Dridi Boukelmoune
> *Cc:* varnish-misc
> *Subject:* Re: Cannot figure out why I am getting a miss on a static URL
>
>
>
> varnishncsa clearly shows hits and misses within a 24 hour period despite
> clearly setting the TTL to 24 hours.  Varnishlog clearly shows the TTL
> being set to 24 hours.  I have NO lru_nuked objects as clearly shown by the
> varnish stats.
>
>
>
> I'm not sure what is vague.  I am setting an object for a TTL of 24 hours
> yet during that 24 hour period it is still getting miss requests.  The URLs
> in question have no GET parameters.  I am hashing only on the URL and not
> the hostname.
>
>
>
> So what does this tell me?  It tells me that my cache size is big enough.
>
>
>
> My guess is there is some type of contention from locking of some type
> given other requests coming through.  There are quite a few PUT and POST
> requests that pass through this Varnish server.
>
>
>
> Ummmm, I hardly think that "anyone can't help me."  I think most
> definitely "someone" that has some insight about the internals could shed
> some light on this problem.
>
>
>
>
>
>
>       Raymond Jennings III
>
> *nytimes.com <http://nytimes.com/>*
>
> *Office:* 212.556.7786 <212-556-7786>
>
> *iPhone:* 914.330.5074 <914-330-5074>
>
> *E-mail: **Raymond.Jennings at nytimes.com <Raymond.Jennings at nytimes.com>*
>
> *FaceTime:* Raymond.Jennings at nytimes.com
>
>
>
> On Fri, Jan 16, 2015 at 10:01 AM, Dridi Boukelmoune <
> dridi.boukelmoune at zenika.com> wrote:
>
> Hi Raymond,
>
> Have you looked at the logs?
>
> Your description of the issue is too vague, I don't think anyone can
> help you with just that.
>
> Cheers,
> Dridi
>
>
> On Thu, Jan 15, 2015 at 3:36 PM, Jennings III, Raymond
> <raymond.jennings at nytimes.com> wrote:
> > I have a static URL that I set a TTL for 24h but yet every so many
> minutes I
> > get a "miss" (as shown in my varnishncsa.)
> >
> > I purge this URL every 24 hours.
> >
> > 1 - The URL never has any parameters.
> > 2 - I do not hash over the hostname only the URI
> > 3 - Nobody is issuing any PURGE or BAN requests
> > 4 - n_lru_nuked is always 0
> > 5 - There are no 500 or 400 responses
> >
> > What is causing this URL to get a miss when everything I see looks
> good?  If
> > the cache size has been exceeded I would have expected to see the nuked
> > value be something other than 0.
> >
> > vcl_fetch is basically just:
> >
> > set beresp.ttl = 24h;
> > return(deliver);
> >
> > Any reason why Varnish seems to be taking it upon itself to cause this
> miss?
> > In 24 hours I get about 12000 requests and out of those 120 misses and I
> > should see 2 miss requests at most.
> >
> > The backends are healthy.
> >
> > This is varnish-3.0.6-1.el6.x86_64
> >
> > Thanks.
> >
> >
> >
> > Raymond Jennings III
> > nytimes.com
> > Office: 212.556.7786
> > iPhone: 914.330.5074
> > E-mail: Raymond.Jennings at nytimes.com
> > FaceTime: Raymond.Jennings at nytimes.com
> >
>
> > _______________________________________________
> > varnish-misc mailing list
> > varnish-misc at varnish-cache.org
> > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20150116/9fd45490/attachment-0001.html>