mass purge causes high load?

[Sascha Ottolski]

Am Sonntag 13 April 2008 12:08:55 schrieben Sie:
> Sascha Ottolski <ottolski at web.de> writes:
> > I just needed to get rid of about 27,000 stale URLs, that were
> > cached as 404 or 302 due to a configuration error on the backends.
>
> Why couldn't you just wait for them to expire?

If I only knew when this would happen...I set the default_ttl to 360 
days...

>
> > So I did a url.purge in a loop, sleeping 0.1 seconds after each
> > URL:
> >
> > for i in `cat notfound.txt.sorted` ; do varnishadm -T:81 url.purge
> > $i; sleep 0.1; done
>
> That's a very, very bad idea.  Varnish must now check every object in
> cache against 27,000 regular expressions.
>
> If you know the exact URL to purge, use an HTTP PURGE (see VCL code
> examples in the vcl man page)

I'm aware of this, but had expected that the semantic of both method 
would be identical. Especially as I did not pass regular expresssions, 
but the complete URLs. Well, at least that's what I thought :-)

After all, good news is, that the proxies slowly came down to normal 
operation (took 3 or 4 hours), without crashing. However, since then I 
have the feeling that the load is a bit higher than it used to be 
before, now it seems to stay at around 1 even at low traffic periods. 
Anyway, response time is excellent.


Thanks a lot, 

Sascha

>
> DES