[Varnish] #1659: ban lurker doesn't purge all effected objects

[Varnish]

#1659: ban lurker doesn't purge all effected objects
-----------------------+----------------------
 Reporter:  varnish@…  |       Type:  defect
   Status:  new        |   Priority:  normal
Milestone:             |  Component:  varnishd
  Version:  3.0.6      |   Severity:  normal
 Keywords:             |
-----------------------+----------------------
 Hi *,

 couple months ago I had a purging problem, where it seems that not all
 objects where purged by the ban lurker thread. Today this issue happens
 again and I had the chance to capture some logs and requests.

 Story:
 Our customer purges varnish every night at 1pm CET, which he does with a
 PURGE-Request which is handled by the following vcl:
 {{{
 if (req.url ~ "\*") {
             ban("obj.http.X-Purge-URL ~ " + req.url + " && obj.http.X
 -Purge-Host ~ " + req.http.host);
             error 200 "Purge added";
 }
 }}}

 This request results in the following ban entry:
 {{{
 1421192472.105130     0G        obj.http.X-Purge-URL ~ /* && obj.http.X
 -Purge-Host ~ domain.tld
 }}}

 I know, the regex "/*" is some kind of weird, but it shoul match every URL
 since the URL starts with a "/".

 But now we got the problem, that not all URLs are purged, following a
 varnishlog trace for an object which had have purged last night:
 {{{
   420 SessionOpen  c X.X.X.X 38584 X.X.X.X:80
   420 ReqStart     c X.X.X.X 38584 2713433276
   420 RxRequest    c HEAD
   420 RxURL        c /en-gb/printed-denim-skirt-with-leather-trim.html
   420 RxProtocol   c HTTP/1.1
   420 RxHeader     c User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu)
 libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
   420 RxHeader     c Host: www.domain.tld
   420 RxHeader     c Accept: */*
   420 RxHeader     c X-DEBUG:XXXXX
   420 VCL_call     c recv lookup
   420 VCL_call     c hash
   420 Hash         c /en-gb/printed-denim-skirt-with-leather-trim.html
   420 Hash         c www.domain.tld
   420 VCL_return   c hash
   420 Hit          c 2700988225
   420 VCL_call     c hit deliver
   420 VCL_call     c deliver deliver
   420 TxProtocol   c HTTP/1.1
   420 TxStatus     c 200
   420 TxResponse   c OK
   420 TxHeader     c Server: Apache
   420 TxHeader     c P3p: CP="CAO PSA OUR"
   420 TxHeader     c X-Cache-Debug: 1
   420 TxHeader     c X-Cache-Tags: catalog_product_327804
   420 TxHeader     c Vary: Accept-Encoding
   420 TxHeader     c Content-Type: text/html; charset=UTF-8
   420 TxHeader     c X-Purge-URL: /en-gb/printed-denim-skirt-with-leather-
 trim.html
   420 TxHeader     c X-Purge-Host: www.domain.tld
   420 TxHeader     c X-Orig-Query-String:
   420 TxHeader     c X-Striped-Query-String:
   420 TxHeader     c Cache-Control: no-store, no-cache, must-revalidate,
 post-check=0, pre-check=0
   420 TxHeader     c Pragma: no-cache
   420 TxHeader     c Expires: Mon, 31 Mar 2008 10:00:00 GMT
   420 TxHeader     c Date: Wed, 14 Jan 2015 12:42:02 GMT
   420 TxHeader     c X-Varnish: 2713433276 2700988225
   420 TxHeader     c Age: 74851
   420 TxHeader     c Via: 1.1 varnish
   420 TxHeader     c Connection: keep-alive
   420 TxHeader     c X-Cache: HIT
   420 TxHeader     c X-Cache-Hits: 12
   420 TxHeader     c X-BE: dir_domain_tld_random
   420 TxHeader     c X-BE-Name:
   420 TxHeader     c X-Lifetime: 11548.540
   420 Length       c 0
   420 ReqEnd       c 2713433276 1421239322.510934830 1421239322.511025429
 0.000041962 0.000051498 0.000039101
 }}}

 The indicator for myself is the Age header of the object.

 Some background infos, normally the varnish cache holds around 300.000
 objects when the nightly purging starts. It takes about 30 minutes to
 purge all objects from the cache.

 Give me a hint if you need some more information.

 Best regards,
 Martin

-- 
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1659>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator