High CPU usage when using ban with HTTP PURGE

[Connor Walls]

Hi Lasse, thanks for the prompt reply. I've done a little bit more investigation on this issue.

1. I tried using smart bans but I saw no change in performance. I tried change the ban from a regex to a straight match on the URL (i.e. from ban("obj.http.x-url ~ " + [header containing regex]) to ban("obj.http.x-url == " + [header containing url]) ) and I still observed the same performance issues. I also tried changing from ban() to ban_url() - no performance improvement. Removing that line from the VCL brings the CPU right back down, so I've concluded the problem is adding the ban to the ban list. Is this an inherently CPU intensive operation, is there going to be no way around this?

2. I've also gone down the route of trying purges because they are significantly faster and look like they can handle the load okay, but my only issue is matching variations of a page. My vcl_hash function looks like this:

sub vcl_hash {
   hash_data(req.http.Accept);
   hash_data(req.http.X-Forwarded-Proto);
}

So the problem I'm having is purging all variations of the file based on "Accept" and "X-Forwarded-Proto" headers, as well as the host. Is this something that is even possible? I personally wouldn't have thought so but after seeing in the documentation (https://www.varnish-cache.org/docs/trunk/tutorial/purging.html):  "The purge in vcl_miss is necessary to purge all variants in the cases where you hit an object, but miss a particular variant." Seems to imply that this functionality is present but I can't find any more documentation on how it works. Do I need to remove the hash_data() calls in vcl_hash and instead include "Accept" and "X-Forwarded-Proto" in the "Vary" header coming from the backend server?

Thanks,
Connor Walls

-----Original Message-----
From: varnish-misc-bounces at varnish-cache.org [mailto:varnish-misc-bounces at varnish-cache.org] On Behalf Of Lasse Karstensen
Sent: 26 July 2012 10:36
To: varnish-misc at varnish-cache.org
Subject: Re: High CPU usage when using ban with HTTP PURGE

Connor Walls:
> So, I've been having an issue with CPU usage recently. We run a 
> cluster of
> 3 varnish boxes, running 3.0.2, serving content on a particular port, 
> and accepting HTTP PURGE request on another (:82). This is used so 
> whenever a user on the site changes something on their profile page, 
> one of the backend servers will send a HTTP PURGE request to each box 
> in the cluster, containing a regex in the header, which is then used 
> to ban based on a particular URL - the relevant VCL is as follows:
[..]
> Now what we've been seeing is very high CPU usage with this 
> functionality, which has been slowly growing over time with increased 
> traffic on the website, and is now averaging above 70%, peaking at 
> 100% at times and causing some user requests to fail. The CPU usage 
> correlates very closely with timeout exceptions we get when making the 
> HTTP PURGE requests. This became an issue yesterday to the point where 
> we disabled the HTTP PURGE - CPU usage has now fallen from averaging > 
> 70% to averaging around 8%, so clearly the purges are the issue. What 
> I'm wondering is whether or not banning like this is going to be 
> inherently heavy and I need to find a way to throttle the purge 
> requests, or if I could rework the VCL so it will be less CPU heavy to 
> process these requests? Will the CPU usage depend on what the actual regex is?

There are two methods for cache invalidation in Varnish 3.0: Purges and bans.
Bans are flexible and cool, and purges are less flexible but very fast. Use purges if you can.

If you are using a high amount of bans, you should use what Kristian calls "Smart bans" in the Varnish Book: 

    https://www.varnish-software.com/static/book/Cache_invalidation.html#smart-bans


--
Lasse Karstensen
Varnish Software AS
http://www.varnish-software.com/

_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc