High CPU usage when using ban with HTTP PURGE

[Connor Walls]

Hi all,

So, I've been having an issue with CPU usage recently. We run a cluster of 3 varnish boxes, running 3.0.2, serving content on a particular port, and accepting HTTP PURGE request on another (:82). This is used so whenever a user on the site changes something on their profile page, one of the backend servers will send a HTTP PURGE request to each box in the cluster, containing a regex in the header, which is then used to ban based on a particular URL - the relevant VCL is as follows:

//to allow purging
    if (req.request == "PURGE") {
        if (!server.port == 82) {
            error 405 "Not Allowed.";
            return (lookup);
        } else {
            if(req.http.Purge-Regex)
            {
                ban("req.url ~ " + req.http.Purge-Regex);
                error 200 "Purged";
            }
            else
            {
                error 200 "Not Purged";
            }
        }
    }

Now what we've been seeing is very high CPU usage with this functionality, which has been slowly growing over time with increased traffic on the website, and is now averaging above 70%, peaking at 100% at times and causing some user requests to fail. The CPU usage correlates very closely with timeout exceptions we get when making the HTTP PURGE requests. This became an issue yesterday to the point where we disabled the HTTP PURGE - CPU usage has now fallen from averaging > 70% to averaging around 8%, so clearly the purges are the issue. What I'm wondering is whether or not banning like this is going to be inherently heavy and I need to find a way to throttle the purge requests, or if I could rework the VCL so it will be less CPU heavy to process these requests? Will the CPU usage depend on what the actual regex is?

Thanks,
Connor Walls
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20120726/27f6e69a/attachment.html>