Logging expired objects/URLs

[Arthur Lutz]

Hi list,

I'm trying to log the URLs that expire in my varnish cache (version 2.1).

My first approximation was to use the following command :

  varnishlog -u 2>&1 | grep -v RFC --line-buffered | grep -E 
"(ExpKill|TxURL|TTL)" --line-buffered > /tmp/logvarnish

(I couldn't find a successful combination of -i and -I to make the same log)

My idea was that I could then correlate between TxURL & the object_id
found in TTL and ExpKill, but under heavy load it's not that simple,
there is not simple way of getting the objectid from previous lines.

I then started a python script to obtain URLs correlated to have
expired, but results are not great. I even have an option to do a
urlopen and check the "Age" in the headers (can share if needed).

Any tips ?

Arthur

-- 
Arthur Lutz - LOGILAB, Paris (France).
Formations - http://www.logilab.fr/formations
Développements - http://www.logilab.fr/services
Gestion de connaissances - http://www.cubicweb.org/
Conférence Web Sémantique (mai 2012) - http://www.semweb.pro